Losing messages from queue

Question

(OT: i can't figure out these newsgroups too well. i tried using the public newsgroups from my isp usenet server, but they don't seem to get posted here...)
          we have a mdb which acts like a pass through to a stateless session bean. (command pattern?). anyway. the mdb requires a transaction, as does all the slsb's involved. the slsb basically posts orders to a database. during load testing, the whole thing goes flaky. a client application submits 1000 orders to be processed (simulates 10 users, 2 orders per minute per user, 100 orders total per user.). the thing is, 1000 orders never make it to the database, but most times the jms queue has processed all it's messages (no messges pending). we're using guaranteed delivery on the client side, and looking at the wls console can see that there were 1000 messages in the queue.
          the mdb's have a pool of 20 beans, and there is 60 max db connections. we know the db connections are having issues, timing out all over the place and such. i would hope that our app can recover from that ok. when there's any error in processing, we set rollback on the context.
          we're noticing that messages seem to get committed from the jms queue and not committed to the database. maybe the db is getting rolled back and the db isn't. maybe the db never has a chance to commit the item.
          i can see that messages are getting re-process ok. so i do know that rollbacks are happening on the db side of things. it's just a little challenging to trace all 1000 messages through the system to find what happened.
          does anyone have some ideas where we can start to troubleshoot these "missing" messages/transactions?
          we're using oracle's thin xa driver with wls 8.1sp2.
          if anyone could perhaps shed a bit of light on this (troubleshooting tips), it would be most certainly appreciated!!
          our next step we're going to take is to enable DebugJMSXA and DebugMessagePath logging from the server, and also logging the message id's and message retry attributes from the applicatin.

Answer 1

(OT: i can't figure out these newsgroups too well. i          > tried using the public newsgroups from my isp usenet
          > server, but they don't seem to get posted here...)
          OT: I'm told that that the transfer between usenet and forums is not immediate - it may take up to 15 minutes. I will forward your comment.
          >
          > we have a mdb which acts like a pass through to a
          > stateless session bean. (command pattern?). anyway.
          > the mdb requires a transaction,
          First thing to check: Is the MDB descriptor marked BOTH with "Container" and "Required"? (Actually, my bet is that your OK here.)
          > as does all the
          > slsb's involved. the slsb basically posts orders to a
          > database. during load testing, the whole thing goes
          > flaky. a client application submits 1000 orders to be
          > processed (simulates 10 users, 2 orders per minute
          > per user, 100 orders total per user.). the thing is,
          > 1000 orders never make it to the database,
          All 1000?
          > but most
          > times the jms queue has processed all it's messages
          > (no messges pending). we're using guaranteed delivery
          > on the client side, and looking at the wls console
          > can see that there were 1000 messages in the queue.
          >
          > the mdb's have a pool of 20 beans, and there is 60
          > 0 max db connections. we know the db connections are
          > having issues, timing out all over the place and
          > such.
          What operation in particular is timing out? And can you post your config.xml for the connection pool and data source?
          > i would hope that our app can recover from
          > that ok. when there's any error in processing, we set
          > rollback on the context.
          >
          > we're noticing that messages seem to get committed
          > from the jms queue and not committed to the database.
          > maybe the db is getting rolled back and the db
          > b isn't. maybe the db never has a chance to commit
          > the item.
          BTW: The Oracle 10g thin driver handles XA much faster than their earlier drivers. I think that BEA consequently now generally recommends using this driver now.
          >
          > i can see that messages are getting re-process ok.
          > so i do know that rollbacks are happening on the db
          > b side of things. it's just a little challenging to
          > trace all 1000 messages through the system to find
          > what happened.
          >
          > does anyone have some ideas where we can start to
          > troubleshoot these "missing" messages/transactions?
          What do you mean by "re-proces ok". Does this mean that once all reprocessing completes, all messages have been processed correctly?
          >
          > we're using oracle's thin xa driver with wls 8.1sp2.
          >
          > if anyone could perhaps shed a bit of light on this
          > (troubleshooting tips), it would be most certainly
          > appreciated!!
          >
          > our next step we're going to take is to enable
          > DebugJMSXA and DebugMessagePath logging from the
          > server, and also logging the message id's and message
          > retry attributes from the applicatin.
          One thing to do is sanity check the entry points in each of your beans (MDB and SSB) to ensure that there is a transaction on the thread. The JMS FAQ contains info on this: TxHelper.getTransaction()...
          Another is to investigate whether transactions are timing out. (Increase the transaction timeout and see if either (A) rollbacks stop occuring or (B) if rollbacks take longer to occur.)