JMS Issues!

Question

h1. {color:#000000}Hi Guys, {color}
We are having issues when using multiple JMS queues and Message DrivenBeans. Its Glassfish application server, version 2. The way it currently works is we create some objects add add them to the first JMS queue. The queue destination is a message driven bean when receives the message acknoladges it and then creates several new objects and adds them to another queue when is received by another message driven bean. This means that objects recevied on the first quest generally create 10 - 20 new objects (the first object is normally a URI to a web resource of some kind). The receiver then will create 10 -20 new objects that are downloaded from a server. Each of these objects are then added to a separate queue.
All the functionality of sending messages is all contained in the code below:
private QueueConnectionFactory pconnectionFactory = null;
private Queue pqueue = null;
private QueueConnection queueConnection = null;
private QueueSession queueSession = null;
private QueueSender queueSender = null;
private Queue replyTo = null;
private boolean isQueueOpen = false;
    *  Sends a JMS object message, if exception occurs, a new connection is started.
     public void sendJMSMessage(Serializable object)     {
          log.trace("JMSProvider:Sending JMS object message");
          try
               trySendJMSMessage(object);
          } catch(JMSException jmsException) {
               log.trace("JMSException caught when sending message... attempting to reopen connection");
               log.trace(jmsException);
               closeJms();
               openOnce();
               sendJMSMessage(object);
     private void trySendJMSMessage(Serializable object) throws JMSException {
          ObjectMessage objMsg = queueSession.createObjectMessage(object);
          objMsg.setJMSExpiration(0);
         if (replyTo != null)
              objMsg.setJMSReplyTo(replyTo);     
         queueSender.send(objMsg);
     private void openJms() throws JMSException {
          isQueueOpen = true;
          log.trace("JMSProvider: Opening JMS connection");
         queueConnection = pconnectionFactory.createQueueConnection();
        queueSession = queueConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
        queueSender = queueSession.createSender(pqueue);
        replyTo = null;
     private void closeJms() {
         isQueueOpen = false;
          log.trace("JMSProvider: Closing JMS connection");
        if (queueConnection != null) {
            try {
                queueConnection.close();
            } catch (JMSException e) {
                 log.trace("JmsProvider: JMSExcetion caught when closeing JMS Connection");
            queueConnection = null;
     }Iv not used jms before. The first problem that I notice is if I try to send an object message and then imediatly close the connection, when the server is handling large loads of data. I receive messages in the log stating that the connection pool on the server has reached its limit. The JMS connections are P2P and not topic based. There is a single producer and a single consumer, however its on a cluseterd server, meaning multiple threads. Knowning this, we trying not to close queue connections when sending messages and try and use the same queue to send all message to the desired destination using one queue. The problem is that a no point to we know howmany objects we are going to be processing so we do not know at what point to close the connections.
As we cannot close the connections, it is left up to the garbage collected to destroy the connections, this also produces warnings in the log file stating that a connection has been destoyed on an active connection.
The last problem that I we have is that occassionaly I receive an XAException. The exception is listed below in the next post:
[#|2009-12-23T14:39:53.762+0100|SEVERE|sun-appserver2.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=58;_ThreadName=p: thread-pool-1; w: 50;_RequestID=d953eff9-afe6-4f65-bf08-7ae8328c421a;|endTransaction (XA) on JMSService:jmsdirect failed for connectionId:1621055879205632000 and flags=67108864 due to unkown JMSService server error.|#]
[#|2009-12-23T14:39:53.762+0100|WARNING|sun-appserver2.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=58;_ThreadName=p: thread-pool-1; w: 50;_RequestID=d953eff9-afe6-4f65-bf08-7ae8328c421a;|MQJMSRA_MC2001: destroy:Previously destroyed-mcId=221|#]
[#|2009-12-23T14:39:53.762+0100|SEVERE|sun-appserver2.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=58;_ThreadName=p: thread-pool-1; w: 50;_RequestID=d953eff9-afe6-4f65-bf08-7ae8328c421a;|rollbackTransaction (XA) on JMSService:jmsdirect failed for connectionId:1621055879205632000:transactionId=1621055879249863681 due to unkown JMSService server error.|#]
[#|2009-12-23T14:39:53.762+0100|SEVERE|sun-appserver2.1|javax.enterprise.system.core.transaction|_ThreadID=58;_ThreadName=p: thread-pool-1; w: 50;org.omg.CORBA.INTERNAL:   vmcid: 0x0  minor code: 0 completed: Maybe;rollback;_RequestID=d953eff9-afe6-4f65-bf08-7ae8328c421a;|JTS5031: Exception [org.omg.CORBA.INTERNAL:   vmcid: 0x0  minor code: 0 completed: Maybe] on Resource [rollback] operation.|#]
[#|2009-12-23T14:39:53.762+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=66;_ThreadName=p: thread-pool-1; w: 82;|- getQueuedUpdateById() called, 1 updates in queue
[#|2009-12-23T14:39:53.762+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=66;_ThreadName=p: thread-pool-1; w: 82;|- All resources handled of Update 1261575585540 - 5 sources
|#]
[#|2009-12-23T14:39:53.762+0100|SEVERE|sun-appserver2.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=83;_ThreadName=p: thread-pool-1; w: 65;_RequestID=43da22b0-7829-4dff-9483-95d536d7f38a;|startTransaction (XA) on JMSService:jmsdirect failed for connectionId:1621055879205636098 due to unkown JMSService server error.|#]
[#|2009-12-23T14:39:53.762+0100|WARNING|sun-appserver2.1|javax.enterprise.system.core.transaction|_ThreadID=83;_ThreadName=p: thread-pool-1; w: 65;_RequestID=43da22b0-7829-4dff-9483-95d536d7f38a;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException
     at com.sun.messaging.jms.ra.DirectXAResource.start(DirectXAResource.java:680)
     at com.sun.jts.jta.TransactionState.startAssociation(TransactionState.java:305)
     at com.sun.jts.jta.TransactionImpl.enlistResource(TransactionImpl.java:205)
     at com.sun.enterprise.distributedtx.J2EETransaction.enlistRes
.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:245)
     at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:76)
     at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:555)
Caused by: com.sun.messaging.jmq.jmsservice.JMSServiceException: startTransaction: connection ID not found: 1621055879205636098
     at com.sun.messaging.jmq.jmsserver.service.imq.IMQDirectService.checkConnectionId(IMQDirectService.java:2464)
     at com.sun.messaging.jmq.jmsserver.service.imq.IMQDirectService.startTransaction(IMQDirectService.java:1514)
     at com.sun.messaging.jms.ra.DirectXAResource.start(DirectXAResource.java:646)
I would be eternally greatfull for any ideas?
Thanks Dan

Answer 1

This isn't much information to go on. Some thoughts:
Is your consumer app throwing exceptions or otherwise somehow failing to acknowledge/commit its received messages?
Are there any messages in the server logs?
Have you tried looking at your thread dumps?
I recommend starting your investigation by examining the message count statistics for each destination and each subscriber.
Also, in case you haven't done so already, I highly recommend configuring a quota on each JMS server.
Tom