JMS Server on a cluster stores all msgs posted to Distributed Queue on 1 server failure

Question

          Hi
          I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running its own
          JMS server.
          Product version in use is Weblogic SP2.
          Configured a Distributed Queue and deployed it on both the JMS servers.
          Set the load-balancing to round-Robin and Server-Affinity to False on the connection
          factory.
          The producer is an external client which posts 100 messages to the Distributed
          queue.
          Consumers are MDBs pinned to the respective physical queues on the respective
          JMS servers.
          Each server has its own store.
          Once 100 messages are posted to the Distributed Queue. The messages are distributed
          to
          the 2 Physical Destinations equally i.e 50-50 each.
          While the MDBs are processing the msgs, server srvr1 is brought down at a point
          when it has consumed only 10 of its 50 msgs.
          When the srvr1 is brought up again, i am finding that the store for that JMS server
          contains the 40 unprocessed msgs in its physical queue + the msgs on the other
          queue that were unprocessed at the time this server crashed.
          When srvr1 went down, srvr2 was running and it went on to process all its 50 msgs
          completly. Why would those processed msgs show up again when srvr1 is brought
          up ?
          Please provide some input !!
          Ravi.
          

Answer 1

          The log message is from the distributed destination
          "forwarders". In distributed queues, these forwarders
          automatically move messages from queues without
          consumers to queues that have them (queue
          forwarders are disabled by default). In distributed topics,
          the forwarders serve
          to replicate a published message from the local
          topic member to each of the remote topic members.
          "Raviprasad Athivilli" <[email protected]> wrote:
          >
          >Tom
          >
          >I think you are right. When the server1 is brought up again, its picking
          >up all
          >the messages
          >that were written to the store.
          >
          >But as it starts processing them, i get this exception on the server2
          >that has
          >been running from the beginning.
          >
          ><Oct 10, 2003 6:20:19 PM EDT> <Error> <JMS> <040366> <JMS Distributed
          >Destination
          >member "MyQueue@JMSServer-02" in "MyQueue"
          >unable to accept connection from remote member "MyQueue@JMSServer-01"
          >due to exception
          >weblogic.jms.common.JMSException: can't find
          >queue target jndi table for MyQueue@JMSServer-01 != MyQueue
          >weblogic.jms.common.JMSException: can't find queue target jndi table
          >for MyQueue@JMSServer-01
          >!= MyQueue
          > at weblogic.jms.backend.BEDestination.setupSystemSubscription(BEDestination.java:3466)
          > at weblogic.jms.backend.BEManager.sessionAndTopicSubscriberCreate(BEManager.java:234)
          > at weblogic.jms.backend.BEManager.invoke(BEManager.java:384)
          > at weblogic.jms.dispatcher.Request.wrappedFiniteStateMachine(Request.java:602)
          > at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsync(DispatcherImpl.java:152)
          > at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsyncFuture(DispatcherImpl.java:396)
          > at weblogic.jms.dispatcher.DispatcherImpl_WLSkel.invoke(Unknown
          >Source)
          > at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:362)
          > at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:313)
          > at weblogic.security.service.SecurityServiceManager.runAs(SecurityServiceManager.java:821)
          > at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:308)
          > at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
          > at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:213)
          > at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:189)
          >
          >What does this mean ?
          >
          >Ravi.
          >Tom Barnes <[email protected]> wrote:
          >>Seems more than odd. Are you sure that q1 got unprocessed
          >>messages from q2? How did you confirm?
          >>
          >>Perhaps q1 got messages its MDB already processed back again -
          >>an indication that the MDB is throwing Runtime exceptions
          >>or Errors, and the container is rolling back transactions.
          >>
          >>Raviprasad Athivilli wrote:
          >>> Hi
          >>>
          >>> I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running
          >>its own
          >>> JMS server.
          >>> Product version in use is Weblogic SP2.
          >>> Configured a Distributed Queue and deployed it on both the JMS servers.
          >>> Set the load-balancing to round-Robin and Server-Affinity to False
          >>on the connection
          >>> factory.
          >>>
          >>> The producer is an external client which posts 100 messages to the
          >>Distributed
          >>> queue.
          >>> Consumers are MDBs pinned to the respective physical queues on the
          >>respective
          >>> JMS servers.
          >>> Each server has its own store.
          >>>
          >>> Once 100 messages are posted to the Distributed Queue. The messages
          >>are distributed
          >>> to
          >>> the 2 Physical Destinations equally i.e 50-50 each.
          >>>
          >>> While the MDBs are processing the msgs, server srvr1 is brought down
          >>at a point
          >>> when it has consumed only 10 of its 50 msgs.
          >>>
          >>> When the srvr1 is brought up again, i am finding that the store for
          >>that JMS server
          >>> contains the 40 unprocessed msgs in its physical queue + the msgs
          >on
          >>the other
          >>> queue that were unprocessed at the time this server crashed.
          >>>
          >>> When srvr1 went down, srvr2 was running and it went on to process
          >all
          >>its 50 msgs
          >>> completly. Why would those processed msgs show up again when srvr1
          >>is brought
          >>> up ?
          >>>
          >>> Please provide some input !!
          >>>
          >>> Ravi.
          >>
          >