WLS 9.2: Interpreting the error - This RJVM has already been shutdown

Question

Hi,
I have encountered the error message (stacktrace below) in one of the managed servers of my Weblogic 9.2 cluster (containing 4 managed servers). There are ALSB proxies consuming and producing to JMS distributed queues on the cluster.
The server reporting this error (server01_01) is complaining about the Remote JVM of the other servers (server01_02, 02_01, 02_02) being shutdown, but what I have observed is that server01_01 was killed by the Nodemanager(may be due to failure or being unavailable).
I would like to know:
1) When does this error occur usually?
2) How is it different from "weblogic.jms.common.LostServerException: [JMSClientExceptions:055169]Network connection to server was lost."?
3) Why did I get an error complaining about the other healthy servers when actually the problem is with the server reporting the error?
Any hints are appreciated.
+####<Mar 26, 2009 11:44:08 AM EST> <Warning> <RMI> <ifappd111d-v04t> <+server01_01+> <[ACTIVE] ExecuteThread: '37' for queue:+
+'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1238028248453> <BEA-080006> <Failed to associate the transaction context with the response while marshalling an exception to the client:+
java.rmi.ConnectException: This RJVM has already been shutdown -3786093667493708317S:ifappd112d-v04t:[8003,8003,-1,-1,-1,-1,-1]:ifappd111d-v04t:8003,ifappd112d-v04t:8003,ifappd111d-v04t:8005,ifappd112d-v04t:8005:domServiceBus:server02_01.
java.rmi.ConnectException: This RJVM has already been shutdown -3786093667493708317S:ifappd112d-v04t:[8003,8003,-1,-1,-1,-1,-1]:ifappd111d-v04t:8003,ifappd112d-v04t:8003,ifappd111d-v04t:8005,ifappd112d-v04t:8005:domServiceBus:*server02_01*+
     at weblogic.rjvm.RJVMImpl.getOutputStream(RJVMImpl.java:336)
     at weblogic.rjvm.RJVMImpl.getResponseStream(RJVMImpl.java:679)
     at weblogic.rjvm.MsgAbbrevInputStream.getMsgAbbrevOutputStream(MsgAbbrevInputStream.java:275)
     at weblogic.rjvm.MsgAbbrevInputStream.getOutboundResponse(MsgAbbrevInputStream.java:534)
     at weblogic.rmi.internal.BasicServerRef.postInvoke(BasicServerRef.java:599)
     at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:453)
     at weblogic.rmi.internal.BasicServerRef.access$300(BasicServerRef.java:57)
     at weblogic.rmi.internal.BasicServerRef$BasicExecuteRequest.run(BasicServerRef.java:965)
     at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
     at weblogic.work.ExecuteThread.run(ExecuteThread.java:181)

Answer 1

EJB keep traks of Bea CLUSTER using JVMID as an identifier of each manged server's JVM. When this JVMID is no longer valid, you can see "JVM is already been shutdowm" as client exception. This issue happen when, on client side, EJB can't recognize server side JVM shutdown. In WLS 8.1 there was a PATCH to solve this problem. Ask to Bea support if you need a patch also for WLS 9.2.
Regards