Best Practice when deploying a single mdb into a cluster

Question

At a high level, we are converting all of our components to Weblogic processes that use Stateless Session Beans and Message Driven Beans. All of the weblogic processes will be clustered, and all of the topic queues will be distributed (Uniform Distributed Topics / Queues).
          We have one component that is a single MDB reading from a single queue on 1 machine. It is a requirement that the JMS messages on that queue be processed in order, and the processing of messages frequently requires that the same row in the DB be updated. Does anyone have any thoughts on the best design for that in our clustered environment?
          One possible solution we have come up with (not working):
          Possible Solution 1: Use a distributed topic and enforce a single client via client-id on the connection factory, causing a single consumer.
          1.Deploy a uniform-distributed Topic to the cluster.
          2.Create a connection factory with a client-id.
          3.Deploy a single FooMDB to the cluster.
          Problem with Solution 1: WL allows multiple consumers on Topic with same client-id
          1.Start (2) servers in cluster
          2.FooMDB running on Server_A connects to Topic
          3.FooMDB running on Server_B fails with unique id exception (yeah).
          4.Send messages - Messages are processed only once by FooMDB on Server_A (yeah).
          5.Stop Server_A.
          6.FooMDB running on Server_B connects automatically to Topic.
          7.Send messages - Messages are processed by FooMDB on Server_B (yeah).
          8.Start Server_A
          9.FooMDB successfully connects to Topic, even though FooMDB on Server_B is already connected (bad). Is this a WL bug or our config bug??
          10.Send messages - Messages are processed by both FooMDB on Server_A and Server_B (bad). Is this a WL bug or our config bug??
          Conclusion: Does anyone have any thoughts on the best design for that in our clustered environment? and if the above solution is doable, what mistake might we have made?
          Thank you in advance for your help!
          kb

Answer 1

Thanks for the helpful info Tom.
          Kevin - It seems that for both the MDB, and the JMS provider, there are (manual or scripted) actions to be taken during any failure event + failure probes possibly required to launch these actions...?
          In the case of the JMS provider, the JMS destination needs to be migrated in the event of managed-server or host failure; if this host is the one that also runs the Admin server then the Admin server also needs to be restarted on a new host too, in order that it can become available to receive the migration instructions and thus update the config of the managed server which is to be newly targetted to serve the JMS destination.
          In the case of the MDB, a deployment action of some sort would need to take place on another managed-server, in the event of a failure of the managed server or the host, where the original MDB had been initally deployed.
          The JMS Destination migration actions can be totally avoided by the use of another JMS implementation which has a design philosophy of "failover" built into it (for example, Tibco EMS has totally automatic JMS failover features) and could be accessed gracefully by using Weblogic foreign JMS. The sinlge MDB deployed on one of the Weblogic managed servers in the cluster would still need some kind of (possibly scripted) redeployment action, and on top of this, there would need to be some kind of health check process to establish if this re-deployment action was actually required to be launched. It is possible that the logic and actions required just to establish the true functional health of this MDB could themsevles be as difficult as the original design requirement :-)
          All of this suggests that for the given requirement; the BEA environment is not well suited; and if no other environment or JMS provider is available at your site, then a manipulation of process itself may be required to enable it to be handled in a highly-available way which can be gracefully administered in a Weblogic cluster.
          We have not discussed the message payload design and the reasons that message order must be respected - by changing the message payload design and possibly adding additional data, this requirement "can", "in certain circumstances", be avoided.
          If you can't do that, I suggest you buy a 2 node Sun Cluster with shared HA storage and use this to monitor a simple JMS client java program that periodically checks for items on the Queue. The Tibco EMS servers could also be configured on this platform and give totally automatic failover protection for both process and host failure scenarios. With the spare money we can go to the pub.
          P.S. I don't work for Tibco or Sun and am a BIG Weblogic fan :-)