JMS message persistance

Hi
I have an interface receiving SMS from a SMSC. This message is dumped into a JMS queue where a MDB receives and processes this.
This MDB relies on a DB connection writing a lot of information away and execute some PL/SQL.
Question is: When the DB is not available, ie the JDBC connection fails, how do I "re-send" this or save it to process it later when the DB is up again.
Is there any method available that will "auto re-submit" the JMS message if I don't acknowledge it?
The currentfunctionlaity works well except for this problem.
Tks
Andre

Isaac -- If you need high reliability or transactional behaviors you really need to be using Oracle JMS. In general we recommend using Oracle JMS for now becasue OC4J JMS is still not JMS 1.0.2 compliant. We are working on that for a future release.
Thanks -- Jeff

Similar Messages

What is the recommended way for persisting JMS messages?

What is the recommended way for persisting JMS messages?. As per the IMQ admin documentation , using the default built-in persistence type which is through unix flat files is much efficient and faster, compared to the database persistence .
Tried setting up the jdbc stuff for database persistence on iAS 6.5 . I am getting the following
error .
[24/Apr/2002:16:09:20 PDT] [B1060]: Loading persistent data...
[24/Apr/2002:16:09:21 PDT] Using plugged in persistent store: database connection
url=jdbc:oracle:thin:@dbatool.mygazoo.com:1521:qa1 brokerid=ias01
[24/Apr/2002:16:09:23 PDT] [B1039]: Broker "jmqbroker" ready.
[24/Apr/2002:16:11:56 PDT] ERROR [B4012]: Failed to persist interest
SystemManager%3ASystemManagerEngine%2BiMQ+Destination%0AgetName%28%29%3A%09%09SM_Response%0AClass%3A%09%09%09com.sun.messaging.Topic%0AgetVERSION%28%29%3A%09%092.0%0AisReadonly%28%29%3A%09%09false%0AgetProperties%28%29%3A%09%7BJMQDestinationName%3DSM_Response%2C+JMQDestinationDescription%3DA+Description+for+the+Destination+Object%7D:
java.sql.SQLException: ORA-01401: inserted value too large for column
[24/Apr/2002:16:11:56 PDT] WARNING [B2009]: Creation of consumer SM_Response to destination 1
failed:com.sun.messaging.jmq.jmsserver.util.BrokerException: Failed to persist interest
SystemManager%3ASystemManagerEngine%2BiMQ+Destination%0AgetName%28%29%3A%09%09SM_Response%0AClass%3A%09%09%09com.sun.messaging.Topic%0AgetVERSION%28%29%3A%09%092.0%0AisReadonly%28%29%3A%09%09false%0AgetProperties%28%29%3A%09%7BJMQDestinationName%3DSM_Response%2C+JMQDestinationDescription%3DA+Description+for+the+Destination+Object%7D:
java.sql.SQLException: ORA-01401: inserted value too large for column
Any thoughts?

From the output, you are using imq 2.0. In that release
the key used to persist a durable subscriber in the database
table has a limit of 100 characters. The output shows that
your value is:
SystemManager%3ASystemManagerEngine%2BiMQ+Destination%0AgetName%28%29%3A%09%09SM_Res
ponse%0AClass%3A%09%09%09com.sun.messaging.Topic%0AgetVERSION%28%29%3A%09%092.0%0Ais
Readonly%28%29%3A%09%09false%0AgetProperties%28%29%3A%09%7BJMQDestinationName%3DSM_R
esponse%2C+JMQDestinationDescription%3DA+Description+for+the+Destination+Object%7D:
which is much longer than 100 characters.
You might want to shorten the string you use for the
durable name.
And yes, the default file-based persistence store is
more efficient when compared to the plugged-in persistence
through a database.

JMS - remove persistent messages from memory

Hi,
I have a problem with RAM and JMS. I have to store many JMS-messages (overall size
of more than 2GB). By default they are stored in RAM and in the database.
How can I configure WL not to store all messages in RAM.

Do you mean Time to Live Override value that we set in Overrides tab for the jms queue.Yes. Time to Live override can be used for that purpose.
Do all messages get stored in persistent (file)store while a jms server is up or only the messages with certain status like "Pending" get stored in jms.If yes,do the >pending messages get automatically deleted once the messages are read by the client application.All persistent messages gets stored in the jms store which includes messages in Messages Current ( status - visible ) state.
Some of the non persistent messages which gets paged out to free physical memory also end up in the store.
Messages get deleted from the store, but file stores doesn't relieve the disk space to OS after deleting ie file stores never shrink in size automatically. You can even end up with a huge file store even though no messages are pending in any of the queues under it.

Saving a JMS Message in a Persistent Store

Hello Friends,
I am going to store a JMS message in a persistent store like a database and then recreate it when needed. What I will be doing is storing the JMS headers individually. Then storing all the properties and then the message body. I just wanted to confirm that with this data I would be able to recreate the entire message.

Not sure of the answer to this (I suspect re-creating the message with exactly the same values as it started with will be a bit more tricky than you would imaging). Why don't you try creating a blank message and setting every single header / property? You'll probably get some exceptions / values being over written when you send.
The real question is are you 100% sure you need to do this? The whole idea smells fishy. Why do you need the JMS Headers? Things would be far easier if the body of the message contained everything you needed. Why do you have to store the message in a database rather than leave it on a persistent queue?

EntityManager.persist and JMS messaging issues?

Hi everyone:
I am seeing a problem that I could not figure out a solution. Can anyone shed me some lights on this?
I have a EJB method which basically does 2 things: 1). persists a record into a database table using EntityManager 2). sends out a message to a JMS topic. So it is th like:
myMethod() {
entityManager.persist(myObj);
notifier.notify(...);
The client will then listen to the JMS topic and read the new record from database. Usually this works pretty well, but sometimes, the client cannot find the new record from database at all and a few seconds later it can. It is as if there is a delay between the JMS messaging and database persistence - somehow the client receives the message first and at this time the record is not really saved into database yet!
Can anyone explain to me what is going on here? What kind of solutions might help to resolve this?
BTW, I am using the Hibernate EntityManager. My app server is Glassfish.
Thanks a lot for your help
jmao

You need to use flush method to persist the value before you calling notifier.notify(...);
below code should works
myMethod() {
entityManager.persist(myObj);
*entityManager.flush();</<br />
notifier.notify(...);
}

Weird things when persisting the jms messages of topic in the wls9?

I am testing the store of the jms messages in wls9.
          But when i sent a topic message to the topic, it seems that the message cannot be stored to the persistent stores.
          The things were that:
          when i send a message of queue and refreshed the page of console of wls9.I saw the number changed in :"Home > Summary of Persistent Stores > exampleJDBCStore "-Monitoring-Statistics,the "Create Count" had increased.
          But when i use the topic message, the "Create Count" never changed.
          Anybody can tell me why?
          Is the topic message cannot be stored in persistent stores, or others ?
          thx and rgds.

Topic messages are not persisted unless both the publisher qos is persistent and there are existing durable subscribers. Otherwise, there is no need to persist them. Do you have any durable subscribers?
          Tom

Need suggestion on implementing JMS message error recovery

Hi,
Our application has a JMS topic where we publish application events. Now, there can be scenarios where the consumers cannot process the message due to some infrastructure issues and would error out. We need a way so that those messages can be reprocessed again later. we are thinking of the following design for JMS message error recovery
1. Use a persistent TOPIC (this would ensure guaranteed delivery)
2. Configure a error destination on JMS topic e.g a jms queue
3. Have an error handling MDB listening to the Error destination. An error handling MDB would dequeue the errored messages from error destination and persist it to a Data base "error" table..
4. Provide a mechanism to republish those messages to topic (e.g a scheduler or admin ui or a command line utility) .. The messages would deleted from database "error" table and published to topic again....
A. Are there any issues with the above design which we need to handle?
B. Are there any additional steps required in a Cluster environment with a distributed topic and distribute error destination? (our error mdb will have one-copy-per-application setting)
B. From a performance angle, Is it OK to use persistent TOPIC ? Or will it better to persist the message to the db table and then publish it as a non persistent message ... ? (But i guess the performance should be more or less the same in both of these approaches)
C. Are there any other recommended design patterns for error recovery of JMS messages
Please advise.
Regards,
Arif

Thanks Tom !
We may not be able to go with the approach of delaying/pausing redelivery of the messsage because
1. Pausing entire MDB approach: Our MDB application consumes messages generated by different producers and our MDB needs to continue processing the messages even if messages corresponding to one producer is erroring out
2. Redelivery delay : This would only delay the retry of an errored message. But there would still be a problem if the message fails during all retries (i.e redelivery limit count). We don't want to lose this message. In our case, It is possible that a particular message cannot be processed due to unavailability of a third party system for hours or may be a day.
Basically, i am looking on approaches for a robust and performant error recovery/retry framework for our application (refer details in my first post on this thread) while fully making use of all features provided by middleware (WLS). Please advise.
Regards,
Arif

Lost (and found) JMS - messages

          Using WLS 6.1 and have experienced that JMS messages has not been delivered to
          its consumer correctly. We are interested in any suggestions to what has happend,
          what can be done to avoid the problem in the future, or perhaps how we can find
          out more about the problem.
          We have a message driven bean A which sends messages to the queue Q, of which
          the message driven bean B is the only consumer.
          What we recently have experienced is that some of the messages delivered by A
          to Q never are delivered to B. But the messages seems to be persisted correctly,
          because after a restart they reappear!
          Following is a more detailed description of the scenario:
          1. A message is sendt with DeliveryMode=PERSISTENT and TimeToLive=0. QueueSender.send
          is invoked and returns normally.
          2. The consumers never gets the message (B.onMessage i never invoked with this
          message).
          3. Point 1-2 is repeated for 13 different messages over a period when ~100 similar
          messages are delivered normally.
          4. The server is then restarted.
          5. In the starup log it is indicated that 44 records are found in the JMSStore.
          6. When the server restarts, the "lost" messages gets delivered to B.
          After the server restart we have not experienced the problem again.
          Logs does not show anything unusual about the previous restart.
          Other details:
          There is always exatly 1 instance of A running, and 5-20 instances of B. A implements
          and uses a pool of javax.jms.XAQueueSession instances, which shares a single instance
          of javax.jms.XAQueueConnection. The aknowledge-mode for B is "Auto-aknowledge".
          We're using Weblogic Server 6.1.3.0 / Red Hat Linux 7.3.
          Any help on this problem is appreciated!


          I believe we've seen just the behaviour you describe on 6.1 SP2, but only on a
          multi-cpu box (Solaris), and only under load. We can't reproduce the problem reliably,
          and get no error messages/warnings anywhere, so very little to send to customer
          support; by the time we notice this, the problem's long passed, so a thread dump's
          of no use.
          We're going to try and see how SP5 (which has loads of JMS fixes) pans out.
          But if you find anything out, could you post here?
          simon.
          "Harald Stendal" <[email protected]> wrote:
          >
          >Using WLS 6.1 and have experienced that JMS messages has not been delivered
          >to
          >its consumer correctly. We are interested in any suggestions to what
          >has happend,
          >what can be done to avoid the problem in the future, or perhaps how we
          >can find
          >out more about the problem.
          >
          >We have a message driven bean A which sends messages to the queue Q,
          >of which
          >the message driven bean B is the only consumer.
          >What we recently have experienced is that some of the messages delivered
          >by A
          >to Q never are delivered to B. But the messages seems to be persisted
          >correctly,
          >because after a restart they reappear!
          >
          >Following is a more detailed description of the scenario:
          >
          >1. A message is sendt with DeliveryMode=PERSISTENT and TimeToLive=0.
          >QueueSender.send
          >is invoked and returns normally.
          >2. The consumers never gets the message (B.onMessage i never invoked
          >with this
          >message).
          >3. Point 1-2 is repeated for 13 different messages over a period when
          >~100 similar
          >messages are delivered normally.
          >4. The server is then restarted.
          >5. In the starup log it is indicated that 44 records are found in the
          >JMSStore.
          >6. When the server restarts, the "lost" messages gets delivered to B.
          >
          >After the server restart we have not experienced the problem again.
          >
          >Logs does not show anything unusual about the previous restart.
          >
          >Other details:
          >There is always exatly 1 instance of A running, and 5-20 instances of
          >B. A implements
          >and uses a pool of javax.jms.XAQueueSession instances, which shares a
          >single instance
          >of javax.jms.XAQueueConnection. The aknowledge-mode for B is "Auto-aknowledge".
          >
          >We're using Weblogic Server 6.1.3.0 / Red Hat Linux 7.3.
          >
          >Any help on this problem is appreciated!
          >

How to stop and start MDBs to listen for JMS messages

Hello! This might be more of an architecture question rather than a technical questing, but I post it here, in the lack of a better place...
For several years I have been using Webster and Webster MQ to send messages back and forth to clients. The architecture for the middleware platform has been designed around the functionality of ListenerPorts. Listenerports is a Webster functionality that let you "stop" and start to listen to JMS messages for a given MDB either by admin console, or by JMX.
The arhitecture we used was something like this :
1.Client sends a persistent message to a queue. (order)
2. An MDB pics up the order message, calls a SessionBean that "fronts" the IIOP back end system.
This is quite "out of the box" architecture, but we used the Listenerports to solve a stability problem for our back end system.
The backEnd system was quite unstable, so the EJB threw an exception (typically because of timeout or connection refused) back to the MDB.
The Listenerport was "defined" to handle 3 errors before stopping, so the MDB resent the message twice before the listenerport automatically stopped. The nice thing about this feature was that the client could continue to send orders, and was not bothered when the back end system was down.
When the backEnd system was up and running, we could start the listenerport and it was all back to normal state. We even had a start script that checked the state of the listenerport every 30 minute, and started it automatically if it was stopped, making the admin task of starting the MDB unnecessary..
Now.. Why am I telling this story..
I would like to know how this could be done using "clean" J2EE technologies..That is..Not using any technology that is not portable..
I`m playing with glassfish at home, and it struck me that there is nothing in the J2EE spec that defines the functionality described above ? Am i missing something ?
I Do not want to stop My application because the back end system is down? That is the only way (i know of) that i can "stop" to listen for messages..
I can put all the messages on an error queue when the backbend system is down, but that would lead to more code just to handle the error messages. Maybe the easiest way to solve this is to "move" (programmatically) all messages from the error queue over to the "standard" queue every XX minute, but somehow that sounds "wrong"
Can anyone give me some advice to how this problem situation should be solved ? I`m interested in the solution both from an architectural perspective and from a more technical perspective..
Best regards
Hans-Jacob Melby

Breakpoints, whether normal or conditional, are just meant for debugging of your code. I had the sense from your question that the pausing you want to do is a part of normal operation of your code. I would NOT recommend using a breakpoint for that situation. It would bring up the block diagram showing the breakpoint when it occurs. A user besides the programmer would not know what to do in that case.
Yes, both the inner and outer loops would have shift registers.
Putting a case structure with a small while loop inside the "Pausing Case" is doable. It just depends on what you are doing or waiting for while the program operation is "paused".

JMS messages stuck in queue in weblogic v923 cluster

Hi all,
Enviornment details:
In our pre-production weblogic v923 cluster enviornment with Oracle 10g database, there are 10 server boxes with 2 managed nodes running on each server box in linux enviornment.
There are 3 JMS queues and 1 error queue per managed node.
All the queues are durable and the persitence store is a database table. There is a single database table per single managed node.
High loads of JMS messages from client applications coming on each of the 3 JMS queue are processed by the cluster on daily basis.
The JMS messages that failed to be processed the very first time due to the application logic exceptions are put on error queue since the retry is set to zero on the non-error queue.
The application exceptions are logged to a database table for futher reference.
The falied messages from the error queue are read by spring based MDP and resent to the application for reprocessing.
If no application exceptions, the message will be persisted in a relational database; in the case of failures, MDP code puts the failed message back on to the error queue.
The auto_ack is true on the JMS message.
It is observed about less than 1% of the messages fail the original processing and are in error queue on daily basis.
99.9% of the time, the reprocessing of the messages from the error queue is a success.
Currently due to some obvious reasons, the system administrator stops a managed node when the disk is full and start a different managed node on the same server box. This approach will be stopped in the very near future so that all nodes on all servers are running at any point of time by making few enviornment changes.
Problem:
1a) Occasionally, it is observed that 1 - 4 JMS messages are stuck in the error queue. There are no signs of reprocessing as there are no application exceptions or the corresponing insert into the database. The JMS persistence store table still has the JMS message.
1b) It is observed that over time, the same JMS messages that were originally stuck on server 1's managed node A are no longer there but are found in a different node (say node B) on a different server box.
As far as I understand the JMS, a message could be in a Pending state if MDP (in my case) does not automatically send ACKNOWLEGMENT back due to an issue in the Spring Defaultlisternercontainer before calling MDP's onMessage(). I am not sure as what happens to that status over time).
It is also observed that the stuck messages content is not much different than the other messages that are processed fine.
Any pointers/input is highly appreciated.
Thanks in advance for your valuable input and time.

Off the top of my head, I can only think of two reasons why a message might move from node to node:
(A) An application is dequeueing them and enquening them elsewhere.
(B) The application is using distributed queues, and the distributed queue has been configured to enable "queue forwarding". The forwarding feature automatically moves messages from distributed queue members that have no consumers to members that do have consumers. Queue forwarding is sometimes enabled as a work-around to better enable MDP support of WL distributed destinations, unlike WebLogic MDBs, MDPs have no container logic to ensure that all members of a distributed destination are serviced by a consumer under all circumstances...
Regards,
Tom

Weblogic Message Bridge Producing Duplicate JMS Messages

We have a message bridge that we use to integrate with TIBCO. The message bridge is deployed on a WL cluster. We see duplicate JMS messages produced by the bridge when it is deployed on the cluster. We don't see this behavior when only one ManagedServer in the cluster is active,.
TIBCO version: 4.4.3 V5
WLS: 10.3.3 with patch for bug BUG8732539 (BQC6)
java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Oracle JRockit(R) (build R28.0.0-679-130297-1.6.0_17-20100312-2121-linux-x86_64, compiled mode)
We are also seeing a lot of exceptions related to
<May 27, 2011 4:11:57 PM PDT> <Warning> <Connector> <BEA-190032> << eis/jms/WLSConnectionFactoryJNDIXA > ResourceAllocationException thrown by resource adapter on call to ManagedConnectionFactory.createManagedConnection(): "javax.resource.ResourceException: Failed to start the connection ">
However the bridge is active. We are monitoring the message-out count on TIBCO and we observed that we see duplicates whenever there a message is re-delivered. We opened a case with Oracle support and they suggested patch 10258751 which didn't help. The next suggestion was a patch for "Bug 11852771 - [WLS10.3.2]MISSING AND DUPLICATE IN JMS QUEUE MESSAGE MONITORING" but I don't think it will help since our JMS consumers are getting duplicate messages and this is not just a UI issue.
Here is the message behavior when we send a single message:
Timelines
11:34:17 --> Message sent to TIBCO EMS queue
11:34:17 --> WL MS3 throws an javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMFAIL : Resource manager is unavailable exception
11:34:17 --> WL MS1 says it got the message and committed the transaction
11:34:17 --> Message is visible in the JMS queue in the admin console
11:34:17 --> TIBCO EMS has still not received an ACK from WL for the message
11:34:31 --> TIBCO EMS resends the same JMS message to WL
11:34:31 --> WL MS2 says it got the message and committed the transaction
11:34:31 --> WL MS1 throws an javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMFAIL : Resource manager is unavailable exception
11:34:31 --> Message is visible in the JMS queue in the admin console (At this point there are two messages in the JMS queue)
11:34:31 --> TIBCO EMS has still not received an ACK from WL for the message
11:34:54 --> TIBCO EMS resends the same JMS message to WL
11:34:31 --> WL MS4 says it got the message and committed the transaction
11:34:31 --> WL MS2 throws an javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMFAIL : Resource manager is unavailable exception
11:34:31 --> Message is visible in the JMS queue in the admin console (At this point there are three messages in the JMS queue)
11:34:31 --> TIBCO EMS receives an ACK from WL for the message
MS04
####<May 27, 2011 11:34:54 AM PDT> <Debug> <MessagingBridgeRuntime> <> <MS04> <[ACTIVE] ExecuteThread: '109' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-6A59E8FE76C4FC4CA149> <> <1306521294611> <BEA-000000> <Bridge: MessageBridge1 (processMessages()) received message:
JMS Message Class: TextMessage
JMSMessageID: ID:MS01.4B044DDFEE1F1:25354AD
JMSCorrelationID: null
JMSDeliveryMode: PERSISTENT
JMSDestination: Queue[SourceQueue1]
JMSExpiration: 0
JMSPriority: 4
JMSRedelivered: true
JMSReplyTo: null
JMSTimestamp: 1306521256905 (Fri May 27 11:34:16 PDT 2011)
JMSType: null
Transaction Id: BEA1-6A59E8FE76C4FC4CA149
<?xml version="1.0" encoding="UTF-8"?>
<commands ...
>
####<May 27, 2011 11:34:54 AM PDT> <Debug> <MessagingBridgeRuntime> <MS04> <MS04> <[ACTIVE] ExecuteThread: '109' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-6A59E8FE76C4FC4CA149> <> <1306521294611> <BEA-000000> <Bridge: MessageBridge1 (processMessages()) successfully sent message:
JMS Message Class: TextMessage
Old JMS MessageID: ID:MS01.4B044DDFEE1F1:25354AD
New JMS MessageID: ID:<193654.1306521294611.0>
<?xml version="1.0" encoding="UTF-8"?>
<commands ...
>
MS03
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntime> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521257342> <BEA-000000> <Bridge MessageBridge1 Successfully got connection to the target destination>
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntime> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521257343> <BEA-000000> <Bridge MessageBridge1: both source and target adapters support XA = true>
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntime> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521257343> <BEA-000000> <Bridge MessageBridge1 supportsMDBTX = false supportsXA = true async = true>
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntime> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521257343> <BEA-000000> <Bridge MessageBridge1 WorkMode = Exactly-once>
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntimeVerbose> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521257343> <BEA-000000> <Bridge MessageBridge1 Entering processMessages() ------ >
####<May 27, 2011 11:34:17 AM PDT> <Error> <MessagingBridge> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-3AA29116A4CA622834EA> <> <1306521257343> <BEA-200015> <An error occurred in bridge "MessageBridge1" during the transfer of messages (java.lang.Exception: javax.resource.ResourceException: Failed to setup the Resource Adapter Connection for enlistment in the transaction, Pool = 'eis/jms/WLSConnectionFactoryJNDIXA', javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMFAIL : Resource manager is unavailable
javax.transaction.xa.XAException: Internal error: XAResource 'eis/jms/WLSConnectionFactoryJNDIXA' is unavailable
at weblogic.transaction.internal.XAResourceDescriptor.checkResource(XAResourceDescriptor.java:948)
at weblogic.transaction.internal.XAResourceDescriptor.startResourceUse(XAResourceDescriptor.java:634)
at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1227)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1161)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.ServerTransactionImpl.enlistResource(ServerTransactionImpl.java:507)
at weblogic.transaction.internal.ServerTransactionImpl.enlistResource(ServerTransactionImpl.java:434)
at weblogic.connector.transaction.outbound.XATxConnectionHandler.enListResource(XATxConnectionHandler.java:117)
at weblogic.connector.outbound.ConnectionWrapper.invoke(ConnectionWrapper.java:218)
at $Proxy62.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1431)
at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1006)
at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1083)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
at weblogic.connector.transaction.outbound.XATxConnectionHandler.enListResource(XATxConnectionHandler.java:200)
at weblogic.connector.outbound.ConnectionWrapper.invoke(ConnectionWrapper.java:218)
at $Proxy62.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1431)
at $Proxy62.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1431)
at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1006)
at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1083)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMFAIL : Resource manager is unavailable
javax.transaction.xa.XAException: Internal error: XAResource 'eis/jms/WLSConnectionFactoryJNDIXA' is unavailable
at weblogic.transaction.internal.XAResourceDescriptor.checkResource(XAResourceDescriptor.java:948)
at weblogic.transaction.internal.XAResourceDescriptor.startResourceUse(XAResourceDescriptor.java:634)
at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1227)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1161)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.ServerTransactionImpl.enlistResource(ServerTransactionImpl.java:507)
at weblogic.transaction.internal.ServerTransactionImpl.enlistResource(ServerTransactionImpl.java:434)
at weblogic.connector.transaction.outbound.XATxConnectionHandler.enListResource(XATxConnectionHandler.java:117)
at weblogic.connector.outbound.ConnectionWrapper.invoke(ConnectionWrapper.java:218)
at $Proxy62.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1431)
at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1006)
at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1083)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1185)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.ServerTransactionImpl.enlistResource(ServerTransactionImpl.java:507)
at weblogic.transaction.internal.ServerTransactionImpl.enlistResource(ServerTransactionImpl.java:434)
at weblogic.connector.transaction.outbound.XATxConnectionHandler.enListResource(XATxConnectionHandler.java:117)
... 8 more
).>
####<May 27, 2011 11:34:17 AM PDT> <Warning> <MessagingBridge> <MS03> <MS03> <[ACTIVE] ExecuteThread: '30' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521257344> <BEA-200026> <Bridge "MessageBridge1" encountered some problems in one of its adapters or underlying systems. It stopped transferring messages and will try to reconnect to the adapters shortly. (The exception caught was java.lang.Exception: MessagingBridgeException: Messaging bridge operation failed
.)>
MS02
####<May 27, 2011 11:34:31 AM PDT> <Debug> <MessagingBridgeRuntime> <MS02> <MS02> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-679AF074F381152F047C> <> <1306521271386> <BEA-000000> <Bridge: MessageBridge1 (processMessages()) received message:
JMS Message Class: TextMessage
JMSMessageID: ID:MS01.4B044DDFEE1F1:25354AD
JMSCorrelationID: null
JMSDeliveryMode: PERSISTENT
JMSDestination: Queue[SourceQueue1]
JMSExpiration: 0
JMSPriority: 4
JMSRedelivered: true
JMSReplyTo: null
JMSTimestamp: 1306521256905 (Fri May 27 11:34:16 PDT 2011)
JMSType: null
Transaction Id: BEA1-679AF074F381152F047C
<?xml version="1.0" encoding="UTF-8"?>
<commands ...
>
####<May 27, 2011 11:34:31 AM PDT> <Debug> <MessagingBridgeRuntime> <MS02> <MS02> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-679AF074F381152F047C> <> <1306521271392> <BEA-000000> <Bridge: MessageBridge1 (processMessages()) successfully sent message:
JMS Message Class: TextMessage
Old JMS MessageID: ID:MS01.4B044DDFEE1F1:25354AD
New JMS MessageID: ID:<196075.1306521271386.0>
<?xml version="1.0" encoding="UTF-8"?>
<commands ...
>
####<May 27, 2011 11:34:54 AM PDT> <Error> <MessagingBridge> <MS02> <MS02> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521294610> <BEA-200015> <An error occurred in bridge "MessageBridge1" during the transfer of messages (java.lang.Exception: weblogic.transaction.RollbackException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMERR : A resource manager error has occured in the transaction branch
javax.transaction.xa.XAException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:495)
at weblogic.connector.transaction.outbound.XAWrapper.setTransactionTimeout(XAWrapper.java:457)
at weblogic.connector.transaction.outbound.RecoveryOnlyXAWrapper.setTransactionTimeout(RecoveryOnlyXAWrapper.java:34)
at weblogic.transaction.internal.XAResourceDescriptor.setXAResourceTransactionTimeoutIfAppropriate(XAResourceDescriptor.java:1422)
at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1223)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1161)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:181)
at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1565)
at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:551)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:132)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.lang.NullPointerException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:485)
... 14 more
at weblogic.transaction.internal.TransactionImpl.throwRollbackException(TransactionImpl.java:1871)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:330)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:225)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:293)
at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1534)
at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1006)
at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1083)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMERR : A resource manager error has occured in the transaction branch
javax.transaction.xa.XAException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:495)
at weblogic.connector.transaction.outbound.XAWrapper.setTransactionTimeout(XAWrapper.java:457)
at weblogic.connector.transaction.outbound.RecoveryOnlyXAWrapper.setTransactionTimeout(RecoveryOnlyXAWrapper.java:34)
at weblogic.transaction.internal.XAResourceDescriptor.setXAResourceTransactionTimeoutIfAppropriate(XAResourceDescriptor.java:1422)
at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1223)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1161)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:181)
at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1565)
at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:551)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:132)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.lang.NullPointerException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:485)
... 14 more
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1185)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:181)
at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1565)
at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:551)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:132)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
... 2 more
).>
####<May 27, 2011 11:34:54 AM PDT> <Warning> <MessagingBridge> <MS02> <MS02> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521294610> <BEA-200026> <Bridge "MessageBridge1" encountered some problems in one of its adapters or underlying systems. It stopped transferring messages and will try to reconnect to the adapters shortly. (The exception caught was java.lang.Exception: MessagingBridgeException: Messaging bridge operation failed
MS01
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntime> <MS01> <MS01> <[ACTIVE] ExecuteThread: '36' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-72BC818FE2BB8C2655C6> <> <1306521257536> <BEA-000000> <Bridge: MessageBridge1 (processMessages()) received message:
JMS Message Class: TextMessage
JMSMessageID: ID:MS01.4B044DDFEE1F1:25354AD
JMSCorrelationID: null
JMSDeliveryMode: PERSISTENT
JMSDestination: Queue[SourceQueue1]
JMSExpiration: 0
JMSPriority: 4
JMSRedelivered: false
JMSReplyTo: null
JMSTimestamp: 1306521256905 (Fri May 27 11:34:16 PDT 2011)
JMSType: null
Transaction Id: BEA1-72BC818FE2BB8C2655C6
<?xml version="1.0" encoding="UTF-8"?>
<commands ...
>
####<May 27, 2011 11:34:17 AM PDT> <Debug> <MessagingBridgeRuntime> <MS01> <MS01> <[ACTIVE] ExecuteThread: '36' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-72BC818FE2BB8C2655C6> <> <1306521257542> <BEA-000000> <Bridge: MessageBridge1 (processMessages()) successfully sent message:
JMS Message Class: TextMessage
Old JMS MessageID: ID:MS01.4B044DDFEE1F1:25354AD
New JMS MessageID: ID:<195581.1306521257536.0>
<?xml version="1.0" encoding="UTF-8"?>
<commands ...
>
####<May 27, 2011 11:34:31 AM PDT> <Error> <MessagingBridge> <MS01> <MS01> <[ACTIVE] ExecuteThread: '36' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521271384> <BEA-200015> <An error occurred in bridge "MessageBridge1" during the transfer of messages (java.lang.Exception: weblogic.transaction.RollbackException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMERR : A resource manager error has occured in the transaction branch
javax.transaction.xa.XAException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:495)
at weblogic.connector.transaction.outbound.XAWrapper.setTransactionTimeout(XAWrapper.java:457)
at weblogic.connector.transaction.outbound.RecoveryOnlyXAWrapper.setTransactionTimeout(RecoveryOnlyXAWrapper.java:34)
at weblogic.transaction.internal.XAResourceDescriptor.setXAResourceTransactionTimeoutIfAppropriate(XAResourceDescriptor.java:1422)
at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1223)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1161)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:181)
at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1565)
at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:551)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:132)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.lang.NullPointerException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:485)
... 14 more
at weblogic.transaction.internal.TransactionImpl.throwRollbackException(TransactionImpl.java:1871)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:330)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:224)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:293)
at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1534)
at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1006)
at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1083)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: javax.transaction.SystemException: start() failed on resource 'eis/jms/WLSConnectionFactoryJNDIXA': XAER_RMERR : A resource manager error has occured in the transaction branch
javax.transaction.xa.XAException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:495)
at weblogic.connector.transaction.outbound.XAWrapper.setTransactionTimeout(XAWrapper.java:457)
at weblogic.connector.transaction.outbound.RecoveryOnlyXAWrapper.setTransactionTimeout(RecoveryOnlyXAWrapper.java:34)
at weblogic.transaction.internal.XAResourceDescriptor.setXAResourceTransactionTimeoutIfAppropriate(XAResourceDescriptor.java:1422)
at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1223)
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1161)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:181)
at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1565)
at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:551)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:132)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.lang.NullPointerException
at weblogic.connector.security.layer.AdapterLayer.setTransactionTimeout(AdapterLayer.java:485)
... 14 more
at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1185)
at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:282)
at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:181)
at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1565)
at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:551)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:132)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
... 1 more
).>
####<May 27, 2011 11:34:31 AM PDT> <Warning> <MessagingBridge> <MS01> <MS01> <[ACTIVE] ExecuteThread: '36' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1306521271384> <BEA-200026> <Bridge "MessageBridge1" encountered some problems in one of its adapters or underlying systems. It stopped transferring messages and will try to reconnect to the adapters shortly. (The exception caught was java.lang.Exception: MessagingBridgeException: Messaging bridge operation failed
.)>

@Tom, thanks for your hints and sorry for my delayed response. Here are my answers to your queries…
-- Does messaging work, without any exceptions, when only one bridge is active?
It works fine when only one node of the cluster is active. -- I'm not familiar with Tibco monitoring, but I suspect that the "message-out count on TIBCO" stat doesn't always imply that there's a duplicate message. It could mean that an initial message delivery attempt failed, it's transaction rolled back, and the message was redelivered. Redelivery after a failure is normal - the original delivered message, if one actually made it onto a WebLogic queue, should be discarded by WebLogic when its failed transaction eventually rolls back (before its ever handed to a consumer application).
You are absolutely right. The mismatch in count was due to redelivery attempts. We could see that as soon as we turned on trace in TIBCO-- The posted stack traces look like they may be JVM stdout/stderr scrapes, which are often automatically truncated, for example, see the "... 8 more" listed at the bottom of one of the stack traces below. There might be more information in the full stack trace for the "Resource manager is unavailable" exception. Full traces are logged in each WebLogic Server's log files.
I tried redirecting the output and even set "stdout Stack Trace Depth:" to -1 but I am not albe to see the complete trace.-- The setTransactionTimeout NullPointerException error message looks like it is collateral damage from the original reported "Resource manager is unavailable" exception that's thrown earlier during the enlistResource() step. If you are truly getting duplicates, then I think this is a clue that the bridge is somehow (stupidly) trying to proceed with forwarding the message despite the fact that there's no current transaction on the thread... 
I have a case open with Oracle support and we have extensive debugging turned on. Here is what the engineer came back with. There is an issue with XA transactions in JCA adapter when it talks to third party providers like TIBCO. Since the resource names are not unique across the cluster, a XA transaction is started on multiple nodes in the cluster for the same message. All transactions abort but the message is persisted and TIBCO redelivers the message since the connection is severed when the transaction aborts. We were able to reproduce this scenario at will with our setup and Oracle was able to reproduce the issue too.  -- Can you check the state of the message on the WebLogic Q? The fact that a message is shown on the WL console doesn't necessarily mean that the state of the message is visible (it could be waiting for the transaction to commit - something that apparently never happens). On the console: Services -> Messaging -> JMS Modules -> [module name] -> [destination name]. 
Our consumer was getting duplicate messages. This confirms that the console was not giving us incorrect information. The console showed that 2/3/4 messages were in "current" state when we expected only one message. -- Make sure that the bridge's Tibco destination is a Tibco queue not a Tibco topic (it's happened). 
Yeah we made sure it was a Queue. -- Can you post your bridge and destination stanzas from your config.xml?  
Will try to do this shortly but have to strip out lot of information. -- I vaguely recall that there was a problem with untyped connection factories in Tibco at one point (not sure). You might want to try changing the Tibco configuration to spin-up "XA Queue Connection Factory" instead of just an "XA Connection Factory" (plus modify the bridge configuration accordingly). 
We tried this but it didn't eliminate this issue. One thing we did notice however was that the number of rollbacks were greatly reduced. 
Here is the workaround that Oracle is suggesting: Create pinned destinations and assign it to a distributed queue. This will keep the transactions local and hence eliminate the duplicate transactions and rollback. We tried it and it eliminated the dups issue. However I discovered another bug during my testing. Our cluster messaging mode is "Unicast" and message forwarding from zero consumer queues to queues with consumers fails. We have four/five nodes in the cluster and two consumer threads. I observed that the zero consumer queues always forward to one queue in the cluster that has consumers. The forwarding stalls when I bring down the node that has consumers. Again, we were able to reproduce this issue at will and Oracle has acknowledged that it is a bug. They've reported that Unicast fails without any warning whereas Multicast fails with a warning in the log. We have observed this message forwarding stalling even with UDQs.
I have another case open for an issue where our message bridges that transfer messages from TIBCO to WLS stalls after sometime. The status is shown as active but it is not doing anything. The server instance shows stuck threads trying to roll back a message. TIBCO logs indicate that it is receiving a request to rollback a transaction but it is not finding the transaction to rollback. The only way to recover from this condition is to restart the cluster. I am fairly confident that he root case for this issue is again the XA transaction issue. I changed our UDQ to pinned destination in one environment and we are no longer seeing dups or bridge stalling there. Hopefully Oracle can resolve the message forwarding issue.Thanks again….

JMS messages sometimes not received by Flex client

We have a problem that Flex client is sometimes not able to
receive JMS messages from a JBoss JMS Queue. Most of the time it
works, but if users logout/login using the same account frequently
or you restart JBoss 4.0.4 during their session then it might
happen. Then it doesn't work at all even if you wait. Client is
able to send messages when this problem occurs, which can then be
intercepted by other clients but cannot receive any messages from
himself or from the others. JBoss 4.0.4 is running in cluster mode.
Could it be that there is an old frozen session which is receiving
the messages meant for the new client?(consumer selector is unique
to each user, but if a user logs in twice, the same selector is
used) If I run 2 instances of my JMS test consumer using the same
selector then the consumer which registered later receives the
messages and if it quits, then the older consumer starts receving
them. This doesn't occur in Flex.
Flex is running on Tomcat 5.5.17.
Our messaging-config.xml is
<?xml version="1.0" encoding="UTF-8"?>
<service id="message-service"
class="flex.messaging.services.MessageService"
messageTypes="flex.messaging.messages.AsyncMessage">
<adapters>
<adapter-definition id="actionscript"
class="flex.messaging.services.messaging.adapters.ActionScriptAdapter"
default="true" />
<adapter-definition id="jms"
class="flex.messaging.services.messaging.adapters.JMSAdapter"/>
</adapters>
<destination id="JMSQueueB">
<properties>
<network>
<session-timeout>10</session-timeout>
</network>
<jms>
<initial-context-environment>
<property>
<name>java.naming.factory.url.pkgs</name>
<value>org.jboss.naming:org.jnp.interfaces</value>
</property>
<property>
<name>Context.PROVIDER_URL</name>
<value>jnp://x.x.x.x:1100,jnp://x.x.x.y:1100</value>
</property>
<property>
<name>Context.INITIAL_CONTEXT_FACTORY</name>
<value>org.jnp.interfaces.NamingContextFactory</value>
</property>
</initial-context-environment>
<destination-type>Queue</destination-type>
<message-type>javax.jms.TextMessage</message-type>
<connection-factory>ConnectionFactory</connection-factory>
<destination-jndi-name>queue/B</destination-jndi-name>
<destination-name>B</destination-name>
<delivery-mode>PERSISTENT</delivery-mode>
<message-priority>DEFAULT_PRIORITY</message-priority>
<acknowledge-mode>AUTO_ACKNOWLEDGE</acknowledge-mode>
<transacted-sessions>false</transacted-sessions>
</jms>
</properties>
<channels>
<channel ref="my-rtmp"/>
<channel ref="my-secure-amf"/>
</channels>
<adapter ref="jms"/>
</destination>
</service>
Part of our actionscript code:
pmConsumer = new Consumer();
pmConsumer.destination = "JMSQueueB";
pmConsumer.selector = "some jms selector ....";
pmConsumer.addEventListener(MessageEvent.MESSAGE,
presenceMessageHandler);
pmConsumer.addEventListener(MessageFaultEvent.FAULT,
faultHandler);
pmConsumer.addEventListener(ChannelFaultEvent.FAULT,
channelFaultHandler);
pmConsumer.subscribe();
Also Flex seems to be handing channel switching incorrectly,
if rtmp port is firewalled, it won't use amf properly even though
it was given 2 channels in configuration file and JMS messaging
doesn't work. We have to alter the channel set during runtime.
Flex also doesn't automatically switch to other JMS server if
one goes down. Is this a feature or a bug? We have to reset
producer and consumer in order for this to work.

Hi,
It was in earlier version of the OS that you could turn Bonjour On and Off.
If this can be done in Leopard and Snow Leopard I have not found it yet. (But I don't look that often and nor for very long)
This Forum may help with that
I would also try a PRAM reset. (on Both computers)
Shut down the computer.
Restart it holding down APPLE(or ⌘)ALT+PR Keys until you have heard three Start Up Bongs.
7:18 PM Tuesday; December 8, 2009
Please, if posting Logs, do not post any Log info after the line "Binary Images for iChat"

JMS - Message Loss

Hi All,
I am trying to read message from SUN - JMS Queue to XI
After reading the message XI is trying to persist the message to Database and here I am getting a SQL Exception That The JMS Message ID is of 71 characters
where as the host field in XI DataBase is of size 63 only
This message is not reaching Integration Engine ( I dont see the msg in SXMB_MONI)
Please suggest me some fix.
Thanks
Kavitha

Getting this error from server log :
com.sap.sql.log.OpenSQLException: Cannot assign a java.lang.String object of length 71 to host variable 1 which has JDBC type VARCHAR(63).
Other details in the server log
Category: /System/Database/sql/types
Location: com.sap.sql.types.VarcharResultColumn
Application: sap.com/com.sap.xi.mdt
Regards
Kavitha

Foreign JMS Server Persistance\Retry in case Remote JMS Provider goes down

Hi All
I would need more clarity on this. How does the Foreign JMS Server behave in case the remote JMS provider goes down. Does it take the message from the Publishing Proxy Service and retries\persists the JMS Message till the remote provide is up?
Please let us know. This would effect our design as retrial is very important here. Thanks

If you want to provide guaranteed delivery with automatic retries when using remote JMS implementations, there are multiple ways to implement it:
1. While reading from a foreign JMS destination
Create a Foreign JMS server using transactional drivers (use XA in case of JMS). The messages remain on remote server JMS queue and OSB proxy polls the message directly on remote server. So if Remote Server is down OSB will not be able to read the message. Once remote server is up OSB will pick up the message (as long as foreign JMS server provides persistence and maintains the messages in the queue during a server restart). In case of error in OSB foreign queue can do retry if it provides a retry mechanism.
2. While writing the message to a foreign JMS destination
a. Use a foreign JMS server using transactional connection. Put the retry mechanism in OSB Business service which writes to the foreign JMS/MQ queue. In case of failure Business Service can retry based on configuration. If you want to provide guaranteed delivery then create a local JMS queue to store the messages. So when remote destination is down for a long time undelivered messages will be rolled back to the local queue. You can put retry mechanism on the local queue.
b. Use a messaging bridge which will write the messages to the remote destination. You can configure retry delivery rules in messaging bridge so once busienss service writes the message to the bridge, a SAF agent will ensure the delivery to remote destination whenever it is up.

Sending a JMS message inside a PostPersist entity listener method

Hi to all,
I have an entity called "Batch" that I want to persist. After the persist, I want to send a JMS message to notify batch insertion to the other clients.
So I created the "Batch" entity as following :
@Entity
@EntityListeners({BatchServiceBean.class})
@Table(name = "batch")The "BatchServiceBean" is a session bean which persists the batch and sends the JMS notification message.
Inside this class we have the following method :
@PostPersist
    private void sendInsertMessage(Batch insertedBatch) throws JMSException {
        AddBatchMessage message = new AddBatchMessage();
        message.setBatch(insertedBatch);
        try {
            messageSenderServiceBean.sendRefreshMessage(message);
        } catch (Exception oups) {
            oups.printStackTrace();
    }In the previous code, I recuperate the inserted batch and the "messageSenderServiceBean" facade to send a refresh message.
My problem is that I can get the inserted batch, but the object "messageSenderServiceBean" is null knowing that I have the following annotation to get the "messageSenderServiceBean" facade.
@EJB
    private MessageSenderServiceLocal                   messageSenderServiceBean;Whyhe object "messageSenderServiceBean" is null ??
Can any one help me ??
Thanks in advance.

Hello,
Injection only occurs on server managed objects, not JPA managed entities or other java objects such as the listeners registered with JPA. So your Listener class will need to look up the server objects or other resources directly from the context rather than rely on them being injected. I don't have any good examples handy, but similar to how you would look up an ejb/datasource etc before injection.
Best Regards,
Chris

JMS message persistance

Similar Messages

Maybe you are looking for