Weblogic.tran saction.RollbackException: SubCoordinator ......

Question

Settings:
When we try to call a remote Ejb API the below error is thrown.
xxx.xxx.xx.xx : represents ip for the localhost.
yyy.yyy.yy.yy represents ip for the remote Ejb call
<Apr 24, 2009 3:08:22 PM EDT> <Warning> <JTA> <BEA-110486> <Transaction BEA1-015A5D3211663154D65A cannot complete commit processing beca
use resource [weblogic.jdbc.wrapper.JTSXAResourceImpl] is unavailable. The transaction will be abandoned after 82,022 seconds unless all
resources acknowledge the commit decision.>
<Apr 24, 2009 3:08:50 PM EDT> <Error> <EJB> <BEA-010026> <Exception occurred during commit of transaction Xid=BEA1-00FFFAB936AC3154D65A(
27212457),Status=Rolling Back. [Reason=javax.transaction.SystemException: SubCoordinator 'AdminServer+yyy.yyy.yy.yyt:9002+emall
+admin+admin+9002+' not available],numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds since begin=306,seconds left=4,XAServerResourceInfo
[WLStore_JMSIntegration_cgJMSStore]=(ServerResourceInfo[WLStore_JMSIntegration_cgJMSStore]=(state=rolledback,assigned=AdminServer),xar=W
LStore_JMSIntegration_cgJMSStore3903528,re-Registered = false),XAServerResourceInfo[RayEmall_XA]=(ServerResourceInfo[RayEmall_XA]=(state
=new,assigned=AdminServer),xar=null,re-Registered = false),SCInfo[emall+AdminServer]=(state=rolling-back),SCInfo[JMSIntegration+AdminSer
ver]=(state=rolledback),properties=({}),local properties=({class com.bea.wli.bpm.runtime.JpdContainer$TxnListenerxx.xx.xx.xx-69a754d.1
20d9486666.-7fc9=com.bea.wli.bpm.runtime.JpdContainer$TxnListener@13b5abd, class com.bea.wli.bpm.runtime.JpdContainer$TxnListener147.120
.51.42-69a754d.120d9486666.-7fcb=com.bea.wli.bpm.runtime.JpdContainer$TxnListener@1a930f4}),OwnerTransactionManager=ServerTM[ServerCoord
inatorDescriptor=(CoordinatorURL=AdminServer+xxx.xxx.xx.xx:8001+JMSIntegration+t3+, XAResources={WLStore_JMSIntegration_cgJMSStore, webl
ogic.jdbc.wrapper.JTSXAResourceImpl, WLStore_JMSIntegration_WseeFileStore, WLStore_JMSIntegration_gex-edaFileStore, WLStore_JMSIntegrati
on__WLS_AdminServer, LTHDatasource},NonXAResources={})],CoordinatorURL=AdminServer+xxx.xxx.xx.xx:8001+JMSIntegration+t3+): weblogic.tran
saction.RollbackException: SubCoordinator 'AdminServer+yyy.yyy.yy.yy:9002+emall+admin+admin+9002+' not available
at weblogic.transaction.internal.TransactionImpl.throwRollbackException(TransactionImpl.java:1809)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:331)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:227)
at weblogic.ejb.container.internal.MDListener.execute(MDListener.java:463)
at weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:335)
at weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:291)
at weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4072)
at weblogic.jms.client.JMSSession.execute(JMSSession.java:3962)
at weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:4490)
at weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
.>
<Apr 24, 2009 3:09:22 PM EDT> <Warning> <JTA> <BEA-110486> <Transaction BEA1-006AA78459E63154D65A cannot complete commit processing beca
use resource [weblogic.jdbc.wrapper.JTSXAResourceImpl] is unavailable. The transaction will be abandoned after 77,408 seconds unless all
resources acknowledge the commit decision.>
None of the transactions are recorded on the WLICONSOLE when the Remote EJB call is envoked. When i comment out the code for the Remote API Call there is no error being thrown, and the WLICONSOLE records the transactions as completed. I tried to debug the error, but the process seems to complete the execution, with the result as expected, but the error is still present despite of the process complete execution, with no other errors.
One more thing: Despite of the message "AdminServer+yyy.yyy.yy.yyt:9002+emall+admin+admin+9002+' not available" if i copy the IPaddress on the browser the server is present and its the Admin server console of the cluster.
Edmond.

Answer 1

We have 2 domains that need to commnicate. Web domain has the webapp deployed and invokes the Integration app in the integration domain.
Web Domain DS:
URL : jdbc:oracle:thin:@xxx.xxx.xx.xx:1521:dbname
Driver Class Name: oracle.jdbc.xa.client.OracleXADataSource
DS name : lthDatasource
Integration Domain DS:
URL : jdbc:oracle:thin:@xxx.xxx.xx.xx:1521:dbname
Driver Class Name: oracle.jdbc.xa.client.OracleXADataSource
DS Name: LTHDatasource
The reason we need the DS from both domains is because we right data to the DB from both apps that are deployed in both domains.
Below is the error that continues to be thrown despite of the Server gets restarted or not, and terminating all of the Transactions from the WLI Console. A bit confusing, but due to the fact that is throwing the below error, it seems as it is an Local issue, and i don't know where the problem is.
<Apr 27, 2009 8:32:21 AM EDT> <Warning> <JTA> <BEA-110486> <Transaction BEA1-006AA78459E63154D65A cannot complete commit processing bec
ause resource [weblogic.jdbc.wrapper.JTSXAResourceImpl] is unavailable. The transaction will be abandoned after 54 seconds unless all r
esources acknowledge the commit decision.>
<Apr 27, 2009 8:33:21 AM EDT> <Error> <JTA> <BEA-110423> <Abandoning transaction after 244,431 seconds: Xid=BEA1-006AA78459E63154D65A(4
297209),Status=Committing,numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds since begin=244431,seconds left=0,XAServerResourceInfo[WLSt
ore_JMSIntegration_cgJMSStore]=(ServerResourceInfo[WLStore_JMSIntegration_cgJMSStore]=(state=committed,assigned=AdminServer),xar=WLStor
e_JMSIntegration_cgJMSStore6065567,re-Registered = true),XAServerResourceInfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(ServerResourceI
nfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(state=new,assigned=none),xar=weblogic.jdbc.wrapper.JTSXAResourceImpl@21a87e5,re-Registere
d = true),SCInfo[JMSIntegration+AdminServer]=(state=committed),properties=({ISOLATION LEVEL=2, weblogic.jdbc=t3://XXX.xxx.xx.xx:8001}),
local properties=({weblogic.transaction.recoveredTransaction=true}),OwnerTransactionManager=ServerTM[ServerCoordinatorDescriptor=(Coord
inatorURL=AdminServer+XXX.xxx.xx.xx:8001+JMSIntegration+t3+, XAResources={WLStore_JMSIntegration_cgJMSStore, weblogic.jdbc.wrapper.JTSX
AResourceImpl, WLStore_JMSIntegration_WseeFileStore, WLStore_JMSIntegration_gex-edaFileStore, WLStore_JMSIntegration__WLS_AdminServer,
LTHDatasource},NonXAResources={})],CoordinatorURL=AdminServer+XXX.xxx.xx.xx:8001+JMSIntegration+t3+)>
<Apr 27, 2009 8:33:21 AM EDT> <Error> <JTA> <BEA-110423> <Abandoning transaction after 239,877 seconds: Xid=BEA1-015A5D3211663154D65A(4
267614),Status=Committing,numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds since begin=239876,seconds left=0,XAServerResourceInfo[WLSt
ore_JMSIntegration_cgJMSStore]=(ServerResourceInfo[WLStore_JMSIntegration_cgJMSStore]=(state=committed,assigned=AdminServer),xar=WLStor
e_JMSIntegration_cgJMSStore6065567,re-Registered = true),XAServerResourceInfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(ServerResourceI
nfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(state=new,assigned=none),xar=weblogic.jdbc.wrapper.JTSXAResourceImpl@21a87e5,re-Registere
d = true),SCInfo[JMSIntegration+AdminServer]=(state=committed),properties=({ISOLATION LEVEL=2, weblogic.jdbc=t3://XXX.xxx.xx.xx:8001}),
local properties=({weblogic.transaction.recoveredTransaction=true}),OwnerTransactionManager=ServerTM[ServerCoordinatorDescriptor=(Coord
inatorURL=AdminServer+XXX.xxx.xx.xx:8001+JMSIntegration+t3+, XAResources={WLStore_JMSIntegration_cgJMSStore, weblogic.jdbc.wrapper.JTSX
AResourceImpl, WLStore_JMSIntegration_WseeFileStore, WLStore_JMSIntegration_gex-edaFileStore, WLStore_JMSIntegration__WLS_AdminServer,
LTHDatasource},NonXAResources={})],CoordinatorURL=AdminServer+XXX.xxx.xx.xx:8001+JMSIntegration+t3+)>