JTS Hangs

After a while, and sometimes randomly, our system hangs while getting a
connection. We religiously close connections, so I don't think that is the
problem. When I do a thread dump, here is the result. Can someone help?
WLS 4.5.1 sp8
JDBC2.0 JTS driver
HPUX 11
Full thread dump:
"ListenThread" prio=9 tid=0x4221a0 nid=0x2a runnable
[0x74e63000..0x74e634e0]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:406)
at java.net.ServerSocket.implAccept(ServerSocket.java:240)
at java.net.ServerSocket.accept(ServerSocket.java:224)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:280)
"GET_ZB_ECOMM_QUEUE-0" prio=8 tid=0x454eb0 nid=0x29 waiting on monitor
[0x74ddf000..0x74ddf4e0]
at java.lang.Thread.sleep(Native Method)
at com.onrp.ecomm.engine.BatchListener.run(BatchListener.java:73)
"aq_inventory-0" prio=9 tid=0x453278 nid=0x28 waiting on monitor
[0x74e00000..0x74e004e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:34)
at weblogic.rjvm.ResponseImpl.getThrowable(ResponseImpl.java:41)
at weblogic.common.T3Client.sendRecv(T3Client.java:755)
at
weblogic.jdbcbase.t3.Connection.executeRequest(Connection.java:367)
at
weblogic.jdbcbase.t3.Connection.executeRequest(Connection.java:360)
at weblogic.jdbcbase.t3.Driver.connect(Driver.java:243)
at java.sql.DriverManager.getConnection(DriverManager.java:457)
at java.sql.DriverManager.getConnection(DriverManager.java:106)
at weblogic.jdbcbase.jts.Driver.connect(Driver.java:271)
at
com.zanybrainy.ecomm.inventory.InventoryHandler.dbdb(InventoryHandler.java:1
58)
at
com.zanybrainy.ecomm.inventory.InventoryHandler.service(InventoryHandler.jav
a:123)
at
com.zanybrainy.ecomm.inventory.InventoryHandlerEOImpl.service(InventoryHandl
erEOImpl.java:48)
at
com.zanybrainy.ecomm.inventory.InventoryHandlerEOImpl_ServiceStub.service(In
ventoryHandlerEOImpl_ServiceStub.java:130)
at
com.onrp.ecomm.engine.BusinessHandlersDispatcher.service(BusinessHandlersDis
patcher.java:117)
at com.onrp.ecomm.engine.QueueBean.dequeue(QueueBean.java:122)
at
com.onrp.ecomm.engine.QueueBeanEOImpl.dequeue(QueueBeanEOImpl.java:48)
at
com.onrp.ecomm.engine.QueueBeanEOImpl_ServiceStub.dequeue(QueueBeanEOImpl_Se
rviceStub.java:317)
at com.onrp.ecomm.engine.QueueListener.run(QueueListener.java:101)
"Thread-1" prio=8 tid=0x3bfe30 nid=0x27 waiting on monitor
[0x74e21000..0x74e214e0]
at java.lang.Thread.sleep(Native Method)
at
com.zanybrainy.ecomm.submit.SubmitListener.run(SubmitListener.java:280)
at java.lang.Thread.run(Thread.java:479)
"Thread-0" daemon prio=8 tid=0x3b1ee0 nid=0x26 waiting on monitor
[0x74e42000..0x74e424e0]
at java.lang.Thread.sleep(Native Method)
at
com.onrp.tools.error.ErrorHandler$EmailThread.run(ErrorHandler.java:379)
at java.lang.Thread.run(Thread.java:479)
"ExecuteThread-9" daemon prio=9 tid=0x2b7a90 nid=0x13 waiting on monitor
[0x75094000..0x750944e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-8" daemon prio=9 tid=0x2b51f0 nid=0x12 waiting on monitor
[0x750b5000..0x750b54e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-7" daemon prio=9 tid=0x2b4598 nid=0x11 waiting on monitor
[0x750d6000..0x750d64e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-6" daemon prio=9 tid=0x2b39b0 nid=0x10 waiting on monitor
[0x750f7000..0x750f74e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-5" daemon prio=9 tid=0x2b1600 nid=0xf waiting on monitor
[0x75118000..0x751184e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-4" daemon prio=9 tid=0x2b0ad0 nid=0xe waiting on monitor
[0x75139000..0x751394e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-3" daemon prio=9 tid=0x27f768 nid=0xd runnable
[0x7515a000..0x7515a4e0]
at java.sql.DriverManager.println(DriverManager.java:364)
at
weblogic.jdbc.common.internal.DriverProxy.execute(DriverProxy.java:140)
at weblogic.t3.srvr.ClientRequest.execute(ClientContext.java:1034)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:106)
"ExecuteThread-2" daemon prio=9 tid=0x27d428 nid=0xc waiting on monitor
[0x7517b000..0x7517b4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-1" daemon prio=9 tid=0x277d70 nid=0xb waiting on monitor
[0x7519c000..0x7519c4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-0" daemon prio=9 tid=0x277a98 nid=0xa waiting on monitor
[0x751bd000..0x751bd4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"TimeEventGenerator" daemon prio=9 tid=0x26c6f0 nid=0x9 waiting on monitor
[0x751de000..0x751de4e0]
at java.lang.Object.wait(Native Method)
at
weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:252)
at
weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java
:146)
at java.lang.Thread.run(Thread.java:479)
"SpinnerRandomSource" daemon prio=9 tid=0x26bc38 nid=0x8 waiting on monitor
[0x7f41d000..0x7f41d4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.security.SpinnerThread.stopSpinning(SpinnerRandomBitsSource.java:10
2)
at
weblogic.security.SpinnerThread.run(SpinnerRandomBitsSource.java:119)
"Signal Dispatcher" daemon prio=9 tid=0xf9bb8 nid=0x7 runnable
[0..0xffffffff]
"Finalizer" daemon prio=9 tid=0xef168 nid=0x5 waiting on monitor
[0x7f4d0000..0x7f4d04e0]
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:112)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:174)
"Reference Handler" daemon prio=9 tid=0xee528 nid=0x4 waiting on monitor
[0x7f4f1000..0x7f4f14e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:114)
"main" prio=9 tid=0x133f0 nid=0x1 waiting on monitor
[0x7f7f1000..0x7f7f1480]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:1701)
at java.lang.reflect.Method.invoke(Native Method)
at weblogic.Server.startServerDynamically(Server.java:95)
at weblogic.Server.main(Server.java:61)
"VM Thread" prio=3 tid=0x1e7e0 nid=0x3 runnable
"VM Periodic Task Thread" prio=8 tid=0xf9918 nid=0x6 waiting on monitor

"Andrew R. Gregory" wrote:
>
This new JTS driver works much better. I am not able to confuse it into
using T3. I was not using clusters in my server.cool, glad to hear it.
Joe
>
"Joseph Weinstein" <[email protected]> wrote in message
news:[email protected]...
Hi Andrew.
Is this in a cluster? I see the problem. It appears the jts driver
is sometimes mistakenly using the type-3 driver, thinking there's
a cluster involved and that it must connect to another server instance,
but the instance is itself! This is causing a deadlock where the
out-calling thread has the DriverManager class lock, and the incoming-
call-handling thread needs it. I'm attaching a patch jar for you to try.
Please put this at the head of your weblogic class path, and restart,
and let me know?
thanks,
Joe
"Andrew R. Gregory" wrote:
After a while, and sometimes randomly, our system hangs while getting a
connection. We religiously close connections, so I don't think that is
the
problem. When I do a thread dump, here is the result. Can someonehelp?
WLS 4.5.1 sp8
JDBC2.0 JTS driver
HPUX 11
Full thread dump:
"ListenThread" prio=9 tid=0x4221a0 nid=0x2a runnable
[0x74e63000..0x74e634e0]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:406)
at java.net.ServerSocket.implAccept(ServerSocket.java:240)
at java.net.ServerSocket.accept(ServerSocket.java:224)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:280)
"GET_ZB_ECOMM_QUEUE-0" prio=8 tid=0x454eb0 nid=0x29 waiting on monitor
[0x74ddf000..0x74ddf4e0]
at java.lang.Thread.sleep(Native Method)
atcom.onrp.ecomm.engine.BatchListener.run(BatchListener.java:73)
"aq_inventory-0" prio=9 tid=0x453278 nid=0x28 waiting on monitor
[0x74e00000..0x74e004e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:34)
at weblogic.rjvm.ResponseImpl.getThrowable(ResponseImpl.java:41)
at weblogic.common.T3Client.sendRecv(T3Client.java:755)
at
weblogic.jdbcbase.t3.Connection.executeRequest(Connection.java:367)
at
weblogic.jdbcbase.t3.Connection.executeRequest(Connection.java:360)
at weblogic.jdbcbase.t3.Driver.connect(Driver.java:243)
at java.sql.DriverManager.getConnection(DriverManager.java:457)
at java.sql.DriverManager.getConnection(DriverManager.java:106)
at weblogic.jdbcbase.jts.Driver.connect(Driver.java:271)
at
com.zanybrainy.ecomm.inventory.InventoryHandler.dbdb(InventoryHandler.java:1
58)
at
com.zanybrainy.ecomm.inventory.InventoryHandler.service(InventoryHandler.jav
a:123)
at
com.zanybrainy.ecomm.inventory.InventoryHandlerEOImpl.service(InventoryHandl
erEOImpl.java:48)
at
com.zanybrainy.ecomm.inventory.InventoryHandlerEOImpl_ServiceStub.service(In
ventoryHandlerEOImpl_ServiceStub.java:130)
at
com.onrp.ecomm.engine.BusinessHandlersDispatcher.service(BusinessHandlersDis
patcher.java:117)
at com.onrp.ecomm.engine.QueueBean.dequeue(QueueBean.java:122)
at
com.onrp.ecomm.engine.QueueBeanEOImpl.dequeue(QueueBeanEOImpl.java:48)
at
com.onrp.ecomm.engine.QueueBeanEOImpl_ServiceStub.dequeue(QueueBeanEOImpl_Se
rviceStub.java:317)
atcom.onrp.ecomm.engine.QueueListener.run(QueueListener.java:101)
"Thread-1" prio=8 tid=0x3bfe30 nid=0x27 waiting on monitor
[0x74e21000..0x74e214e0]
at java.lang.Thread.sleep(Native Method)
at
com.zanybrainy.ecomm.submit.SubmitListener.run(SubmitListener.java:280)
at java.lang.Thread.run(Thread.java:479)
"Thread-0" daemon prio=8 tid=0x3b1ee0 nid=0x26 waiting on monitor
[0x74e42000..0x74e424e0]
at java.lang.Thread.sleep(Native Method)
at
com.onrp.tools.error.ErrorHandler$EmailThread.run(ErrorHandler.java:379)
at java.lang.Thread.run(Thread.java:479)
"ExecuteThread-9" daemon prio=9 tid=0x2b7a90 nid=0x13 waiting on monitor
[0x75094000..0x750944e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-8" daemon prio=9 tid=0x2b51f0 nid=0x12 waiting on monitor
[0x750b5000..0x750b54e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-7" daemon prio=9 tid=0x2b4598 nid=0x11 waiting on monitor
[0x750d6000..0x750d64e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-6" daemon prio=9 tid=0x2b39b0 nid=0x10 waiting on monitor
[0x750f7000..0x750f74e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-5" daemon prio=9 tid=0x2b1600 nid=0xf waiting on monitor
[0x75118000..0x751184e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-4" daemon prio=9 tid=0x2b0ad0 nid=0xe waiting on monitor
[0x75139000..0x751394e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-3" daemon prio=9 tid=0x27f768 nid=0xd runnable
[0x7515a000..0x7515a4e0]
at java.sql.DriverManager.println(DriverManager.java:364)
at
weblogic.jdbc.common.internal.DriverProxy.execute(DriverProxy.java:140)
atweblogic.t3.srvr.ClientRequest.execute(ClientContext.java:1034)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:106)
"ExecuteThread-2" daemon prio=9 tid=0x27d428 nid=0xc waiting on monitor
[0x7517b000..0x7517b4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-1" daemon prio=9 tid=0x277d70 nid=0xb waiting on monitor
[0x7519c000..0x7519c4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"ExecuteThread-0" daemon prio=9 tid=0x277a98 nid=0xa waiting on monitor
[0x751bd000..0x751bd4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.t3.srvr.ExecuteThread.waitForRequest(ExecuteThread.java:88)
at weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:103)
"TimeEventGenerator" daemon prio=9 tid=0x26c6f0 nid=0x9 waiting onmonitor
[0x751de000..0x751de4e0]
at java.lang.Object.wait(Native Method)
at
weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:252)
at
weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java
:146)
at java.lang.Thread.run(Thread.java:479)
"SpinnerRandomSource" daemon prio=9 tid=0x26bc38 nid=0x8 waiting onmonitor
[0x7f41d000..0x7f41d4e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at
weblogic.security.SpinnerThread.stopSpinning(SpinnerRandomBitsSource.java:10
2)
at
weblogic.security.SpinnerThread.run(SpinnerRandomBitsSource.java:119)
"Signal Dispatcher" daemon prio=9 tid=0xf9bb8 nid=0x7 runnable
[0..0xffffffff]
"Finalizer" daemon prio=9 tid=0xef168 nid=0x5 waiting on monitor
[0x7f4d0000..0x7f4d04e0]
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:112)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
atjava.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:174)
"Reference Handler" daemon prio=9 tid=0xee528 nid=0x4 waiting on monitor
[0x7f4f1000..0x7f4f14e0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
atjava.lang.ref.Reference$ReferenceHandler.run(Reference.java:114)
"main" prio=9 tid=0x133f0 nid=0x1 waiting on monitor
[0x7f7f1000..0x7f7f1480]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:424)
at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:1701)
at java.lang.reflect.Method.invoke(Native Method)
at weblogic.Server.startServerDynamically(Server.java:95)
at weblogic.Server.main(Server.java:61)
"VM Thread" prio=3 tid=0x1e7e0 nid=0x3 runnable
"VM Periodic Task Thread" prio=8 tid=0xf9918 nid=0x6 waiting on monitor--
PS: Folks: BEA WebLogic is in S.F., and now has some entry-level positionsfor
people who want to work with Java and E-Commerce infrastructure products.Send
resumes to [email protected]
The Weblogic Application Server from BEA
JavaWorld Editor's Choice Award: Best Web Application Server
Java Developer's Journal Editor's Choice Award: Best Web ApplicationServer
Crossroads A-List Award: Rapid Application Development Tools for Java
Intelligent Enterprise RealWare: Best Application Using a ComponentArchitecture
http://weblogic.beasys.com/press/awards/index.htm
PS: Folks: BEA WebLogic is in S.F., and now has some entry-level positions for
people who want to work with Java and E-Commerce infrastructure products. Send
resumes to [email protected]
The Weblogic Application Server from BEA
JavaWorld Editor's Choice Award: Best Web Application Server
Java Developer's Journal Editor's Choice Award: Best Web Application Server
Crossroads A-List Award: Rapid Application Development Tools for Java
Intelligent Enterprise RealWare: Best Application Using a Component Architecture
http://weblogic.beasys.com/press/awards/index.htm

Similar Messages

  • Getconnection hangs ( WLS 6.1 SP4,  UDB 7.2 Fixpak7, Type 2 Driver)

    Hello,
    We sometimes experience application hang when a user logon due to all threads waiting to get connection from UDB server. Any help is appreciated.
    Environment
    App Server: WLS 6.1 SP4
    Database: DB2 UDB 7.1
    JDBC Adapter: Type 2 IBM Driver (COM.ibm.db2.jdbc.app.DB2Driver)
    We have setup connection pool and a TX data source.
    Connection pool related config.xml
    <JDBCConnectionPool CapacityIncrement="1"
    DriverName="COM.ibm.db2.jdbc.app.DB2Driver" InitialCapacity="40"
    LoginDelaySeconds="1" MaxCapacity="50" Name="LPDSS_DB2_POOL1"
    PreparedStatementCacheSize="100"
    Properties="user=xxx;password=xxx;language=C"
    RefreshMinutes="999999" ShrinkingEnabled="false"
    Targets="ms1" TestConnectionsOnRelease="false"
    TestConnectionsOnReserve="true" TestTableName="SYSIBM.SYSDUMMY1" URL="jdbc:db2:xxxx"/>
    The thread dump analysis shows that getconnection() call waiting for a lock (waiting for monitor entry) and java.weblogic.Admin RESET_POOL doesn't work either. The call never returns.
    "ExecuteThread: '14' for queue: 'default'" daemon prio=5 tid=0x494e78 nid=0x1b waiting for monitor entry [0xaf580000..0xaf5819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    "ExecuteThread: '15' for queue: 'default'" daemon prio=5 tid=0x495970 nid=0x1c waiting for monitor entry [0xaf481000..0xaf4819d8]
    at weblogic.common.internal.ResourceAllocator.reset(ResourceAllocator.java:848)
    at weblogic.common.internal.ResourceAllocator_WLSkel.invoke(Unknown Source)
    at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:305)
    at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:274)
    at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:139)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '16' for queue: 'default'" daemon prio=5 tid=0x496468 nid=0x1d waiting for monitor entry [0xaf37f000..0xaf3819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.jts.Connection.getOrCreateConnection(Connection.java:569)
    at weblogic.jdbc.jts.Connection.prepareStatement(Connection.java:117)
    at weblogic.jdbc.rmi.internal.ConnectionImpl.prepareStatement(ConnectionImpl.java:135)
    at weblogic.jdbc.rmi.SerialConnection.prepareStatement(SerialConnection.java:78)
    "ExecuteThread: '17' for queue: 'default'" daemon prio=5 tid=0x497568 nid=0x1e waiting for monitor entry [0xaf280000..0xaf2819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    "ExecuteThread: '13' for queue: 'default'" daemon prio=5 tid=0x494380 nid=0x1a waiting for monitor entry [0xaf680000..0xaf6819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    "ExecuteThread: '12' for queue: 'default'" daemon prio=5 tid=0x493888 nid=0x19 waiting for monitor entry [0xaf780000..0xaf7819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    Exception in JDBC log shows there is a communication problem.
    SQLException: SQLState(40003) vendor code(-30081)
    COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver][DB2/6000] SQL30081N A communication error has been detected. Communication protocol being
    used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "xx.xx.xx.xx". Communication function dete
    cting the error: "recv". Protocol specific error code(s): "145", "*", "*". SQLSTATE=08001
    ---------------------------------------------------

    ganesh narayanan wrote:
    Hello,
    We sometimes experience application hang when a user logon due to all threads waiting to get connection from UDB server. Any help is appreciated.
    Environment
    App Server: WLS 6.1 SP4
    Database: DB2 UDB 7.1
    JDBC Adapter: Type 2 IBM Driver (COM.ibm.db2.jdbc.app.DB2Driver)
    We have setup connection pool and a TX data source.
    Connection pool related config.xml
    <JDBCConnectionPool CapacityIncrement="1"
    DriverName="COM.ibm.db2.jdbc.app.DB2Driver" InitialCapacity="40"
    LoginDelaySeconds="1" MaxCapacity="50" Name="LPDSS_DB2_POOL1"
    PreparedStatementCacheSize="100"
    Properties="user=xxx;password=xxx;language=C"
    RefreshMinutes="999999" ShrinkingEnabled="false"
    Targets="ms1" TestConnectionsOnRelease="false"
    TestConnectionsOnReserve="true" TestTableName="SYSIBM.SYSDUMMY1" URL="jdbc:db2:xxxx"/>
    The thread dump analysis shows that getconnection() call waiting for a lock (waiting for monitor entry) and java.weblogic.Admin RESET_POOL doesn't work either.
    The call never returns.
    >
    We'd need to see all the threads, preferably before you try (repeatedly it seems) to
    reset the pool. There may well be problems with the IBM driver and DBMS...
    Joe
    "ExecuteThread: '14' for queue: 'default'" daemon prio=5 tid=0x494e78 nid=0x1b waiting for monitor entry [0xaf580000..0xaf5819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    "ExecuteThread: '15' for queue: 'default'" daemon prio=5 tid=0x495970 nid=0x1c waiting for monitor entry [0xaf481000..0xaf4819d8]
    at weblogic.common.internal.ResourceAllocator.reset(ResourceAllocator.java:848)
    at weblogic.common.internal.ResourceAllocator_WLSkel.invoke(Unknown Source)
    at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:305)
    at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:274)
    at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:139)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '16' for queue: 'default'" daemon prio=5 tid=0x496468 nid=0x1d waiting for monitor entry [0xaf37f000..0xaf3819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.jts.Connection.getOrCreateConnection(Connection.java:569)
    at weblogic.jdbc.jts.Connection.prepareStatement(Connection.java:117)
    at weblogic.jdbc.rmi.internal.ConnectionImpl.prepareStatement(ConnectionImpl.java:135)
    at weblogic.jdbc.rmi.SerialConnection.prepareStatement(SerialConnection.java:78)
    "ExecuteThread: '17' for queue: 'default'" daemon prio=5 tid=0x497568 nid=0x1e waiting for monitor entry [0xaf280000..0xaf2819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    "ExecuteThread: '13' for queue: 'default'" daemon prio=5 tid=0x494380 nid=0x1a waiting for monitor entry [0xaf680000..0xaf6819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    "ExecuteThread: '12' for queue: 'default'" daemon prio=5 tid=0x493888 nid=0x19 waiting for monitor entry [0xaf780000..0xaf7819d8]
    at weblogic.common.internal.ResourceAllocator.resetThisOne(ResourceAllocator.java:877)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:486)
    at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:405)
    at weblogic.common.internal.ResourceAllocator.reserveWaitSecs(ResourceAllocator.java:395)
    at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:163)
    at weblogic.jdbc.common.internal.ConnectionPool.reserveWaitSecs(ConnectionPool.java:117)
    at weblogic.jdbc.pool.Driver.connect(Driver.java:152)
    at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:319)
    at weblogic.jdbc.jts.Driver.connect(Driver.java:120)
    at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:214)
    Exception in JDBC log shows there is a communication problem.
    SQLException: SQLState(40003) vendor code(-30081)
    COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver][DB2/6000] SQL30081N A communication error has been detected. Communication protocol being
    used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "xx.xx.xx.xx". Communication function dete
    cting the error: "recv". Protocol specific error code(s): "145", "*", "*". SQLSTATE=08001

  • Hanging problem with Oracle JDBC thin driver

    Hi. We have an application running on top of WLS 6.1 accessing Oracle 8I with oracle's
    JDBC thin driver. Under stress testing, it appears to hang sometimes. Thread dump
    shows that many threads seem to be stuck at the same line of code in JDBC driver
    (at oracle.sql.NUMBER._isPositive(NUMBER.java:2882)). Has anyone run into similar
    issue? Any help is much appreciated.
    "ExecuteThread: '1' for queue: 'default'" daemon prio=5 tid=0xe18960 nid=0xf wai
    ting on monitor [0xb3680000..0xb36819d8]
    at oracle.sql.NUMBER._isPositive(NUMBER.java:2882)
    at oracle.sql.NUMBER._fromLnxFmt(NUMBER.java:2953)
    at oracle.sql.NUMBER.toBigDecimal(NUMBER.java:557)
    at oracle.sql.NUMBER.bigDecimalValue(NUMBER.java:1872)
    at oracle.jdbc.dbaccess.DBConversion.NumberBytesToBigDecimal(DBConversion.ja
    va:1771)
    at oracle.jdbc.driver.OracleStatement.getBigDecimalValue(OracleStatement.jav
    a:3357)
    at oracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    66)
    at oracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    23)
    at oracle.jdbc.driver.OracleResultSetImpl.getObject(OracleResultSetImpl.java
    :401)
    at weblogic.jdbc.jts.ResultSet.getObject(ResultSet.java:268)
    at weblogic.jdbc.rmi.internal.ResultSetImpl.getObject(ResultSetImpl.java:592
    at weblogic.jdbc.rmi.internal.ResultSetStraightReader.getObject(ResultSetStr
    aightReader.java:198)
    at weblogic.jdbc.rmi.SerialResultSet.getObject(SerialResultSet.java:682)
    "ExecuteThread: '6' for queue: 'default'" daemon prio=5 tid=0x6d93a8 nid=0x14
    wa
    iting on monitor [0xb3180000..0xb31819d8]
    at oracle.sql.NUMBER._isPositive(NUMBER.java:2882)
    at oracle.sql.NUMBER._fromLnxFmt(NUMBER.java:2953)
    at oracle.sql.NUMBER.toBigDecimal(NUMBER.java:557)
    at oracle.sql.NUMBER.bigDecimalValue(NUMBER.java:1872)
    at oracle.jdbc.dbaccess.DBConversion.NumberBytesToBigDecimal(DBConversion.ja
    va:1771)
    at oracle.jdbc.driver.OracleStatement.getBigDecimalValue(OracleStatement.jav
    a:3357)
    at oracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    66)
    at oracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    23)
    at oracle.jdbc.driver.OracleResultSetImpl.getObject(OracleResultSetImpl.java
    :401)
    at weblogic.jdbc.jts.ResultSet.getObject(ResultSet.java:268)
    at weblogic.jdbc.rmi.internal.ResultSetImpl.getObject(ResultSetImpl.java:592
    at weblogic.jdbc.rmi.internal.ResultSetStraightReader.getObject(ResultSetStr
    aightReader.java:198)
    at weblogic.jdbc.rmi.SerialResultSet.getObject(SerialResultSet.java:682)

    Try get the latest thin driver from oracle and place it in the beginning of
    the classpath in the server startup script to use it. This seems like a
    oracle thin driver bug, the newer driver might help.
    sree
    "Andrew Dong" <[email protected]> wrote in message
    news:[email protected]...
    >
    Hi. We have an application running on top of WLS 6.1 accessing Oracle 8Iwith oracle's
    JDBC thin driver. Under stress testing, it appears to hang sometimes.Thread dump
    shows that many threads seem to be stuck at the same line of code in JDBCdriver
    (at oracle.sql.NUMBER._isPositive(NUMBER.java:2882)). Has anyone run intosimilar
    issue? Any help is much appreciated.
    "ExecuteThread: '1' for queue: 'default'" daemon prio=5 tid=0xe18960nid=0xf wai
    ting on monitor [0xb3680000..0xb36819d8]
    at oracle.sql.NUMBER._isPositive(NUMBER.java:2882)
    at oracle.sql.NUMBER._fromLnxFmt(NUMBER.java:2953)
    at oracle.sql.NUMBER.toBigDecimal(NUMBER.java:557)
    at oracle.sql.NUMBER.bigDecimalValue(NUMBER.java:1872)
    atoracle.jdbc.dbaccess.DBConversion.NumberBytesToBigDecimal(DBConversion.ja
    va:1771)
    atoracle.jdbc.driver.OracleStatement.getBigDecimalValue(OracleStatement.jav
    a:3357)
    atoracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    66)
    atoracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    23)
    atoracle.jdbc.driver.OracleResultSetImpl.getObject(OracleResultSetImpl.java
    :401)
    at weblogic.jdbc.jts.ResultSet.getObject(ResultSet.java:268)
    atweblogic.jdbc.rmi.internal.ResultSetImpl.getObject(ResultSetImpl.java:592
    atweblogic.jdbc.rmi.internal.ResultSetStraightReader.getObject(ResultSetStr
    aightReader.java:198)
    atweblogic.jdbc.rmi.SerialResultSet.getObject(SerialResultSet.java:682)
    >
    "ExecuteThread: '6' for queue: 'default'" daemon prio=5 tid=0x6d93a8nid=0x14
    wa
    iting on monitor [0xb3180000..0xb31819d8]
    at oracle.sql.NUMBER._isPositive(NUMBER.java:2882)
    at oracle.sql.NUMBER._fromLnxFmt(NUMBER.java:2953)
    at oracle.sql.NUMBER.toBigDecimal(NUMBER.java:557)
    at oracle.sql.NUMBER.bigDecimalValue(NUMBER.java:1872)
    atoracle.jdbc.dbaccess.DBConversion.NumberBytesToBigDecimal(DBConversion.ja
    va:1771)
    atoracle.jdbc.driver.OracleStatement.getBigDecimalValue(OracleStatement.jav
    a:3357)
    atoracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    66)
    atoracle.jdbc.driver.OracleStatement.getObjectValue(OracleStatement.java:41
    23)
    atoracle.jdbc.driver.OracleResultSetImpl.getObject(OracleResultSetImpl.java
    :401)
    at weblogic.jdbc.jts.ResultSet.getObject(ResultSet.java:268)
    atweblogic.jdbc.rmi.internal.ResultSetImpl.getObject(ResultSetImpl.java:592
    atweblogic.jdbc.rmi.internal.ResultSetStraightReader.getObject(ResultSetStr
    aightReader.java:198)
    atweblogic.jdbc.rmi.SerialResultSet.getObject(SerialResultSet.java:682)
    >
    >

  • Weblogic server hangs during stress-testing -- possibly EJB deadlocks

    Hi
    While running stress tests on our application, our WLS 4.5.1 hangs even at a
    mere load of 20 users. We are running on a Solaris machine. Database: Oracle
    8.1.5, running on the same machine. Using oracle.jdbc.driver.OracleDriver as
    the JDBC driver, and using JTS as the transaction manager.
    When we did a thread dump (complete lis at the end of this post) and
    basically we are seeing a bunch of our entity beans in a "locked" state. A
    common pattern we see in the thread dumps is that
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5] com.nps.ers.ejb.organization.<Our Entiy Bean>EOImpl.<a get
    method>(ERSSystemConfigurationBeanEOImpl.java:1539)
    [6] com.nps.ers.ejb.awardnomination.<Our entity bean>.<some method>
    After more thorough studying of EJB transactions, we changed the isolation
    level of all our beans from TRANSACTION_SERIALIZED to
    TRANSACTION_READ_COMMITTED and made sure all our beans say TX_REQUIRED for
    their transactionAttribue. Still, this problem persists.
    The basic scenario we are load testing is that of multiple users logging
    into the system and trying to access a particular user's record, and right
    now, this is causing deadlocks at 20 users.
    What we fail to understand is that --
    (a) even if a method is attempting a read-only access to the data, why would
    it try to obtain a lock? or why would it matter that the data is locked?
    Some of the locks we are seeing are actually on a bean that is used for
    read-only purposes.
    (b) if 2 or more processes are trying to access the same record, based on
    whichever process got their first, shouldn't one succeed, and the other go
    into a 'waiting' state?
    We have also opened this issue with weblogic support, and the response so
    far has been that there "may" be a bug in weblogic 4.5.1 itself that's
    causing it, and they need to look into it further. Other similar posts in
    the newsgroup haven't been resolved either -- hence this general plea for
    help. Please advise if you've any suggestions. Thanks
    Sandeep
    ---------------------------Complete Thread Dump Listing -------------------
    "HighPriority TimeEventGenerator" (TID:0x1488278, sys_thread_t:0x14881b8,
    state:CW, thread_t: t@46, threadID:0xd9271dc8, stack_bottom:0xd9272000,
    stack_size:0x20000) prio=9
    [1] weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:249)
    [2]
    weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java
    :143)
    [3] java.lang.Thread.run(Thread.java:479)
    "SSLListenThread" (TID:0x1588220, sys_thread_t:0x1588160, state:R, thread_t:
    t@45, threadID:0xd9321dc8, stack_bottom:0xd9322000, stack_size:0x20000)
    prio=5
    [1] java.net.PlainSocketImpl.socketAccept(Native Method)
    [2] java.net.PlainSocketImpl.accept(PlainSocketImpl.java:406)
    [3] java.net.ServerSocket.implAccept(ServerSocket.java:240)
    [4] java.net.ServerSocket.accept(ServerSocket.java:224)
    [5]
    weblogic.security.SSL.SSLServerSocket.acceptNoHandshake(SSLServerSocket.java
    :121)
    [6] weblogic.security.SSL.SSLServerSocket.accept(SSLServerSocket.java:112)
    [7] weblogic.t3.srvr.ListenThread.run(ListenThread.java:230)
    "ListenThread" (TID:0x15f7d48, sys_thread_t:0x15f7c88, state:R, thread_t:
    t@44, threadID:0xd92a1dc8, stack_bottom:0xd92a2000, stack_size:0x20000)
    prio=5
    [1] java.net.PlainSocketImpl.socketAccept(Native Method)
    [2] java.net.PlainSocketImpl.accept(PlainSocketImpl.java:406)
    [3] java.net.ServerSocket.implAccept(ServerSocket.java:238)
    [4] java.net.ServerSocket.accept(ServerSocket.java:223)
    [5] weblogic.t3.srvr.ListenThread.run(ListenThread.java:230)
    "ExecuteThread-29" (TID:0xad1c90, sys_thread_t:0xad1bd0, state:CW, thread_t:
    t@40, threadID:0xd94e1dc8, stack_bottom:0xd94e2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5] com.nps.ers.ejb.user.UserBeanEOImpl.display(UserBeanEOImpl.java:1384)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.updatePointsUsed(AwardNo
    minationBean.java:1530)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.nominate(AwardNomination
    Bean.java:1279)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.nominate(AwardNomi
    nationBeanEOImpl.java:282)
    [9]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.nomina
    te(AwardNominationBeanEOImpl_ServiceStub.java:811)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [13] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [14] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [16] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [17]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [19]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [20]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [21] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [22] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-28" (TID:0xad1a78, sys_thread_t:0xad19b8, state:CW, thread_t:
    t@39, threadID:0xd9511dc8, stack_bottom:0xd9512000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-27" (TID:0xad1678, sys_thread_t:0xad15b8, state:CW, thread_t:
    t@38, threadID:0xd9541dc8, stack_bottom:0xd9542000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-26" (TID:0xac84e8, sys_thread_t:0xac8428, state:CW, thread_t:
    t@37, threadID:0xd9571dc8, stack_bottom:0xd9572000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2] weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3] weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-25" (TID:0xab7d50, sys_thread_t:0xab7c90, state:CW, thread_t:
    t@36, threadID:0xd95a1dc8, stack_bottom:0xd95a2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-24" (TID:0xab9b60, sys_thread_t:0xab9aa0, state:CW, thread_t:
    t@35, threadID:0xd95d1dc8, stack_bottom:0xd95d2000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2] weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3] weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-23" (TID:0xab9948, sys_thread_t:0xab9888, state:CW, thread_t:
    t@34, threadID:0xd9601dc8, stack_bottom:0xd9602000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5] com.nps.ers.ejb.user.UserBeanEOImpl.getUserId(UserBeanEOImpl.java:182)
    [6]
    com.nps.ers.ejb.login.LoginSessionBean.getUserStatus(LoginSessionBean.java:1
    61)
    [7]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl.getUserStatus(LoginSessionBeanE
    OImpl.java:212)
    [8]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl_ServiceStub.getUserStatus(Login
    SessionBeanEOImpl_ServiceStub.java:554)
    [9]
    com.nps.ers.servlet.LoginRequest.processLoginRequest(LoginRequest.java:248)
    [10] com.nps.ers.servlet.LoginRequest.getNextPage(LoginRequest.java:90)
    [11] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [12] com.nps.common.servlet.BaseServlet.doGet(BaseServlet.java:124)
    [13] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [14] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [15]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [16]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [17]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [18]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [19] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [20] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-22" (TID:0xabcc20, sys_thread_t:0xabcb60, state:CW, thread_t:
    t@33, threadID:0xd9631dc8, stack_bottom:0xd9632000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-21" (TID:0xa9ccf0, sys_thread_t:0xa9cc30, state:CW, thread_t:
    t@32, threadID:0xd9661dc8, stack_bottom:0xd9662000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2] weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3] weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-20" (TID:0xab2550, sys_thread_t:0xab2490, state:R, thread_t:
    t@31, threadID:0xd9691dc8, stack_bottom:0xd9692000, stack_size:0x20000)
    prio=5
    [1] java.net.SocketInputStream.socketRead(Native Method)
    [2] java.net.SocketInputStream.read(SocketInputStream.java:85)
    [3] weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [4] weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [5] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-19" (TID:0xa640c0, sys_thread_t:0xa64000, state:CW, thread_t:
    t@30, threadID:0xd96c1dc8, stack_bottom:0xd96c2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5] com.nps.ers.ejb.user.UserBeanEOImpl.display(UserBeanEOImpl.java:1384)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.updatePointsUsed(AwardNo
    minationBean.java:1530)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.nominate(AwardNomination
    Bean.java:1279)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.nominate(AwardNomi
    nationBeanEOImpl.java:282)
    [9]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.nomina
    te(AwardNominationBeanEOImpl_ServiceStub.java:811)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [13] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [14] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [16] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [17]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [19]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [20]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [21] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [22] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-18" (TID:0xa502d8, sys_thread_t:0xa50218, state:CW, thread_t:
    t@29, threadID:0xd96f1dc8, stack_bottom:0xd96f2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getCashWithinD
    ays(ERSSystemConfigurationBeanEOImpl.java:1231)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.getAwardNominationNotifi
    cationStates(AwardNominationBean.java:772)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.getAwardNomination
    NotificationStates(AwardNominationBeanEOImpl.java:247)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.getAwa
    rdNominationNotificationStates(AwardNominationBeanEOImpl_ServiceStub.java:41
    5)
    [9]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [12] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [13] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [14] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [16]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [17]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [19]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [20] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [21] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-17" (TID:0xa242d8, sys_thread_t:0xa24218, state:CW, thread_t:
    t@28, threadID:0xd9721dc8, stack_bottom:0xd9722000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.user.UserBeanEOImpl.getUserStatusCode(UserBeanEOImpl.java:22
    77)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.getApprovers(ApproverS
    electionBean.java:352)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.getApprovers(App
    roverSelectionBeanEOImpl.java:140)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.getA
    pprovers(ApproverSelectionBeanEOImpl_ServiceStub.java:554)
    [9]
    com.nps.ers.servlet.ApproverSelection.getApproverList(ApproverSelection.java
    :81)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-16" (TID:0xaa0ed8, sys_thread_t:0xaa0e18, state:CW, thread_t:
    t@27, threadID:0xd9751dc8, stack_bottom:0xd9752000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2] weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3] weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-15" (TID:0xad0220, sys_thread_t:0xad0160, state:CW, thread_t:
    t@26, threadID:0xd9781dc8, stack_bottom:0xd9782000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5] com.nps.ers.ejb.user.UserBeanEOImpl.getUserId(UserBeanEOImpl.java:182)
    [6]
    com.nps.ers.ejb.login.LoginSessionBean.getUserStatus(LoginSessionBean.java:1
    61)
    [7]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl.getUserStatus(LoginSessionBeanE
    OImpl.java:212)
    [8]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl_ServiceStub.getUserStatus(Login
    SessionBeanEOImpl_ServiceStub.java:554)
    [9]
    com.nps.ers.servlet.LoginRequest.processLoginRequest(LoginRequest.java:248)
    [10] com.nps.ers.servlet.LoginRequest.getNextPage(LoginRequest.java:90)
    [11] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [12] com.nps.common.servlet.BaseServlet.doGet(BaseServlet.java:124)
    [13] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [14] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [15]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [16]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [17]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [18]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [19] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [20] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-14" (TID:0xa8f4c0, sys_thread_t:0xa8f400, state:CW, thread_t:
    t@25, threadID:0xd97b1dc8, stack_bottom:0xd97b2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2] weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4] weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getCashWithinD
    ays(ERSSystemConfigurationBeanEOImpl.java:1231)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.getAwardNominationNotifi
    cationStates(AwardNominationBean.java:772)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.getAwardNomination
    NotificationStates(AwardNominationBeanEOImpl.java:247)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.getAwa
    rdNominationNotificationStates(AwardNominationBeanEOImpl_ServiceStub.java:41
    5)
    [9]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [12] com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [13] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [14] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [16]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [17]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [19]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [20] weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [21] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-13" (TID:0xa194f0, sys_thread_t:0xa19430, state:CW, thread_t:
    t@24, threadID:0xd97e1dc8, stack_bottom:0xd97e2000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2] weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3] weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.E

    Can you post details on this instrumentation?
    I know in the past when we ran into this type of problem, it took us a while
    to think through and identify the problems.
    Thanks,
    Gary
    Rob Woollen <[email protected]> wrote in message
    news:[email protected]...
    Generally this indicates a deadlock in the order that you access your
    entity beans. Please follow-up with [email protected]. There is
    some instrumentation that you can turn on to help spot why this is
    occurring.
    -- Rob
    Sandeep Giri wrote:
    Hi
    While running stress tests on our application, our WLS 4.5.1 hangs even
    at a
    mere load of 20 users. We are running on a Solaris machine. Database:Oracle
    8.1.5, running on the same machine. Usingoracle.jdbc.driver.OracleDriver as
    the JDBC driver, and using JTS as the transaction manager.
    When we did a thread dump (complete lis at the end of this post) and
    basically we are seeing a bunch of our entity beans in a "locked" state.A
    common pattern we see in the thread dumps is that
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5] com.nps.ers.ejb.organization.<Our Entiy Bean>EOImpl.<a get
    method>(ERSSystemConfigurationBeanEOImpl.java:1539)
    [6] com.nps.ers.ejb.awardnomination.<Our entity bean>.<some method>
    After more thorough studying of EJB transactions, we changed theisolation
    level of all our beans from TRANSACTION_SERIALIZED to
    TRANSACTION_READ_COMMITTED and made sure all our beans say TX_REQUIREDfor
    their transactionAttribue. Still, this problem persists.
    The basic scenario we are load testing is that of multiple users logging
    into the system and trying to access a particular user's record, andright
    now, this is causing deadlocks at 20 users.
    What we fail to understand is that --
    (a) even if a method is attempting a read-only access to the data, whywould
    it try to obtain a lock? or why would it matter that the data is locked?
    Some of the locks we are seeing are actually on a bean that is used for
    read-only purposes.
    (b) if 2 or more processes are trying to access the same record, basedon
    whichever process got their first, shouldn't one succeed, and the othergo
    into a 'waiting' state?
    We have also opened this issue with weblogic support, and the responseso
    far has been that there "may" be a bug in weblogic 4.5.1 itself that's
    causing it, and they need to look into it further. Other similar postsin
    the newsgroup haven't been resolved either -- hence this general pleafor
    help. Please advise if you've any suggestions. Thanks
    Sandeep
    ---------------------------Complete Thread DumpListing -------------------
    >>
    "HighPriority TimeEventGenerator" (TID:0x1488278,sys_thread_t:0x14881b8,
    state:CW, thread_t: t@46, threadID:0xd9271dc8, stack_bottom:0xd9272000,
    stack_size:0x20000) prio=9
    [1] weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:249)
    [2]
    weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java
    :143)
    [3] java.lang.Thread.run(Thread.java:479)
    "SSLListenThread" (TID:0x1588220, sys_thread_t:0x1588160, state:R,thread_t:
    t@45, threadID:0xd9321dc8, stack_bottom:0xd9322000, stack_size:0x20000)
    prio=5
    [1] java.net.PlainSocketImpl.socketAccept(Native Method)
    [2] java.net.PlainSocketImpl.accept(PlainSocketImpl.java:406)
    [3] java.net.ServerSocket.implAccept(ServerSocket.java:240)
    [4] java.net.ServerSocket.accept(ServerSocket.java:224)
    [5]
    weblogic.security.SSL.SSLServerSocket.acceptNoHandshake(SSLServerSocket.java
    :121)
    [6]weblogic.security.SSL.SSLServerSocket.accept(SSLServerSocket.java:112)
    [7] weblogic.t3.srvr.ListenThread.run(ListenThread.java:230)
    "ListenThread" (TID:0x15f7d48, sys_thread_t:0x15f7c88, state:R,thread_t:
    t@44, threadID:0xd92a1dc8, stack_bottom:0xd92a2000, stack_size:0x20000)
    prio=5
    [1] java.net.PlainSocketImpl.socketAccept(Native Method)
    [2] java.net.PlainSocketImpl.accept(PlainSocketImpl.java:406)
    [3] java.net.ServerSocket.implAccept(ServerSocket.java:238)
    [4] java.net.ServerSocket.accept(ServerSocket.java:223)
    [5] weblogic.t3.srvr.ListenThread.run(ListenThread.java:230)
    "ExecuteThread-29" (TID:0xad1c90, sys_thread_t:0xad1bd0, state:CW,thread_t:
    t@40, threadID:0xd94e1dc8, stack_bottom:0xd94e2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]com.nps.ers.ejb.user.UserBeanEOImpl.display(UserBeanEOImpl.java:1384)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.updatePointsUsed(AwardNo
    minationBean.java:1530)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.nominate(AwardNomination
    Bean.java:1279)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.nominate(AwardNomi
    nationBeanEOImpl.java:282)
    [9]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.nomina
    te(AwardNominationBeanEOImpl_ServiceStub.java:811)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [13]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [14] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [16] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [17]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [19]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [20]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [21]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [22] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-28" (TID:0xad1a78, sys_thread_t:0xad19b8, state:CW,thread_t:
    t@39, threadID:0xd9511dc8, stack_bottom:0xd9512000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-27" (TID:0xad1678, sys_thread_t:0xad15b8, state:CW,thread_t:
    t@38, threadID:0xd9541dc8, stack_bottom:0xd9542000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-26" (TID:0xac84e8, sys_thread_t:0xac8428, state:CW,thread_t:
    t@37, threadID:0xd9571dc8, stack_bottom:0xd9572000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2]weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3]weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-25" (TID:0xab7d50, sys_thread_t:0xab7c90, state:CW,thread_t:
    t@36, threadID:0xd95a1dc8, stack_bottom:0xd95a2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-24" (TID:0xab9b60, sys_thread_t:0xab9aa0, state:CW,thread_t:
    t@35, threadID:0xd95d1dc8, stack_bottom:0xd95d2000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2]weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3]weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-23" (TID:0xab9948, sys_thread_t:0xab9888, state:CW,thread_t:
    t@34, threadID:0xd9601dc8, stack_bottom:0xd9602000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]com.nps.ers.ejb.user.UserBeanEOImpl.getUserId(UserBeanEOImpl.java:182)
    [6]
    com.nps.ers.ejb.login.LoginSessionBean.getUserStatus(LoginSessionBean.java:1
    61)
    [7]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl.getUserStatus(LoginSessionBeanE
    OImpl.java:212)
    [8]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl_ServiceStub.getUserStatus(Login
    SessionBeanEOImpl_ServiceStub.java:554)
    [9]
    com.nps.ers.servlet.LoginRequest.processLoginRequest(LoginRequest.java:248)
    [10] com.nps.ers.servlet.LoginRequest.getNextPage(LoginRequest.java:90)
    [11]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [12] com.nps.common.servlet.BaseServlet.doGet(BaseServlet.java:124)
    [13] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [14] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [15]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [16]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [17]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [18]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [19]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [20] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-22" (TID:0xabcc20, sys_thread_t:0xabcb60, state:CW,thread_t:
    t@33, threadID:0xd9631dc8, stack_bottom:0xd9632000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getEMTRoleId(E
    RSSystemConfigurationBeanEOImpl.java:1539)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.checkEMT(ApproverSelec
    tionBean.java:473)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.checkEMT(Approve
    rSelectionBeanEOImpl.java:71)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.chec
    kEMT(ApproverSelectionBeanEOImpl_ServiceStub.java:501)
    [9]
    com.nps.ers.servlet.NortelApproverSelection.updateApproverList(NortelApprove
    rSelection.java:87)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-21" (TID:0xa9ccf0, sys_thread_t:0xa9cc30, state:CW,thread_t:
    t@32, threadID:0xd9661dc8, stack_bottom:0xd9662000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2]weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3]weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-20" (TID:0xab2550, sys_thread_t:0xab2490, state:R,thread_t:
    t@31, threadID:0xd9691dc8, stack_bottom:0xd9692000, stack_size:0x20000)
    prio=5
    [1] java.net.SocketInputStream.socketRead(Native Method)
    [2] java.net.SocketInputStream.read(SocketInputStream.java:85)
    [3]weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [4]weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [5] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-19" (TID:0xa640c0, sys_thread_t:0xa64000, state:CW,thread_t:
    t@30, threadID:0xd96c1dc8, stack_bottom:0xd96c2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]com.nps.ers.ejb.user.UserBeanEOImpl.display(UserBeanEOImpl.java:1384)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.updatePointsUsed(AwardNo
    minationBean.java:1530)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.nominate(AwardNomination
    Bean.java:1279)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.nominate(AwardNomi
    nationBeanEOImpl.java:282)
    [9]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.nomina
    te(AwardNominationBeanEOImpl_ServiceStub.java:811)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [13]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [14] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [16] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [17]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [19]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [20]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [21]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [22] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-18" (TID:0xa502d8, sys_thread_t:0xa50218, state:CW,thread_t:
    t@29, threadID:0xd96f1dc8, stack_bottom:0xd96f2000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.organization.ERSSystemConfigurationBeanEOImpl.getCashWithinD
    ays(ERSSystemConfigurationBeanEOImpl.java:1231)
    [6]
    com.nps.ers.ejb.awardnomination.AwardNominationBean.getAwardNominationNotifi
    cationStates(AwardNominationBean.java:772)
    [7]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl.getAwardNomination
    NotificationStates(AwardNominationBeanEOImpl.java:247)
    [8]
    com.nps.ers.ejb.awardnomination.AwardNominationBeanEOImpl_ServiceStub.getAwa
    >>
    rdNominationNotificationStates(AwardNominationBeanEOImpl_ServiceStub.java:41
    5)
    [9]
    com.nps.ers.servlet.AwardNominationRequest.nominate(AwardNominationRequest.j
    ava:1339)
    [10]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [12]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [13] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [14] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [15] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [16]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [17]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [18]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [19]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [20]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [21] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-17" (TID:0xa242d8, sys_thread_t:0xa24218, state:CW,thread_t:
    t@28, threadID:0xd9721dc8, stack_bottom:0xd9722000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]
    com.nps.ers.ejb.user.UserBeanEOImpl.getUserStatusCode(UserBeanEOImpl.java:22
    77)
    [6]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBean.getApprovers(ApproverS
    electionBean.java:352)
    [7]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl.getApprovers(App
    roverSelectionBeanEOImpl.java:140)
    [8]
    com.nps.ers.ejb.awardnomination.ApproverSelectionBeanEOImpl_ServiceStub.getA
    pprovers(ApproverSelectionBeanEOImpl_ServiceStub.java:554)
    [9]
    com.nps.ers.servlet.ApproverSelection.getApproverList(ApproverSelection.java
    :81)
    [10]
    com.nps.ers.servlet.NortelApproverSelection.getApproverList(NortelApproverSe
    lection.java:62)
    [11]
    com.nps.ers.servlet.AwardNominationRequest.getApprovers(AwardNominationReque
    st.java:1412)
    [12]
    com.nps.ers.servlet.AwardNominationRequest.previewNominate(AwardNominationRe
    quest.java:1096)
    [13]
    com.nps.ers.servlet.AwardNominationRequest.processNextPage(AwardNominationRe
    quest.java:276)
    [14]
    com.nps.ers.servlet.AwardNominationRequest.getNextPage(AwardNominationReques
    t.java:109)
    [15]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [16] com.nps.common.servlet.BaseServlet.doPost(BaseServlet.java:184)
    [17] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [18] javax.servlet.http.HttpServlet.service(HttpServlet.java:835)
    [19]
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :75)
    [20]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:259)
    [21]
    weblogic.servlet.internal.ServletContextImpl.invokeServlet(ServletContextImp
    l.java:244)
    [22]
    weblogic.socket.MuxableSocketHTTP.invokeServlet(MuxableSocketHTTP.java:353)
    [23]weblogic.socket.MuxableSocketHTTP.execute(MuxableSocketHTTP.java:247)
    [24] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-16" (TID:0xaa0ed8, sys_thread_t:0xaa0e18, state:CW,thread_t:
    t@27, threadID:0xd9751dc8, stack_bottom:0xd9752000, stack_size:0x20000)
    prio=5
    [1] weblogic.common.internal.DynaQueue.getW(DynaQueue.java:219)
    [2]weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:212)
    [3]weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:19)
    [4] weblogic.t3.srvr.ExecuteThread.run(ExecuteThread.java:94)
    "ExecuteThread-15" (TID:0xad0220, sys_thread_t:0xad0160, state:CW,thread_t:
    t@26, threadID:0xd9781dc8, stack_bottom:0xd9782000, stack_size:0x20000)
    prio=5
    [1] weblogic.ejb.internal.BaseEJBContext.lock(BaseEJBContext.java:180)
    [2]weblogic.ejb.internal.StatefulEJBCache.bind(StatefulEJBCache.java:250)
    [3]
    weblogic.ejb.internal.StatefulEJBObject.getContextForInvoke(StatefulEJBObjec
    t.java:88)
    [4]weblogic.ejb.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:164)
    [5]com.nps.ers.ejb.user.UserBeanEOImpl.getUserId(UserBeanEOImpl.java:182)
    [6]
    com.nps.ers.ejb.login.LoginSessionBean.getUserStatus(LoginSessionBean.java:1
    61)
    [7]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl.getUserStatus(LoginSessionBeanE
    OImpl.java:212)
    [8]
    com.nps.ers.ejb.login.LoginSessionBeanEOImpl_ServiceStub.getUserStatus(Login
    SessionBeanEOImpl_ServiceStub.java:554)
    [9]
    com.nps.ers.servlet.LoginRequest.processLoginRequest(LoginRequest.java:248)
    [10] com.nps.ers.servlet.LoginRequest.getNextPage(LoginRequest.java:90)
    [11]com.nps.common.servlet.BaseServlet.handleRequest(BaseServlet.java:245)
    [12] com.nps.common.servlet.BaseServlet.doGet(BaseServlet.java:124)
    [13] javax.servlet.http.HttpServlet.service(HttpServlet.java:708)
    [14

  • Weblogic server 5.1 goes to hang within an hour

    I have the problem with weblogic server 5.1 that it goes to hang within around
    an hour. There is no obvious error or exceptions shown in the weblogic log.
    But at the time the weblogic server is hanging, I just observed from the weblogic
    console GUI that once a user logon to the web application (the web server is iPlanet
    6.0), the number of bean in cache of the corresponding stateful session bean increases
    one but does not drop. Every single logon makes an increment 1 to the # of the
    bean in cache of the corresponding session bean but no dropping.
    What the session bean does is actually obtain a connection from the weblogic server
    and get data from database. I went down to the database (Sybase ASE 11.9) at
    the hanging time but did not see any long-running process nor table blocking that
    may hinder the database response time. I do not understand what the session bean
    was waiting for.
    The only hints that I have is the below error message which was found in the iPlanet
    web server log. Anyone please help to tell me what the problem can be?
    Error found in iPlanet web server
    ============================
    [03/Jun/2003:02:51:26] failure ( 381): Error receiving connection (No process
    is on the other end of the pipe.)

    Thanks, Rob.
    I will try and see if it helps. Thanks again.
    Rob Woollen <[email protected]> wrote:
    Thanks for the thread dump.
    Yes, DriverManager is the problem. Unfortunately this class is
    well-known for causing deadlocks.
    You really do not ever want to use DriverManager.getConnection from
    code that runs in the server. Instead, you should define a TxDataSource
    for your connection pool and use DataSource.getConnection.
    I would recommend you lookup the DataSource in your stateless session
    bean's ejbCreate method and store the DataSource in a member variable.
    Your methods can call DataSource.getConnection when they need a JDBC
    connection.
    -- Rob
    Carol wrote:
    Rob,
    I got the same symptom with weblogic server 6.1 today and I got thethread dump
    for it this time. I read through the log and I wonder if the methodcall from
    "java.sql.DriverManager.println" led to the hanging problem.
    I attach the thread dump for your reference.
    Do you have any advice? Thanks in advance.
    "Carol" <[email protected]> wrote:
    Rob,
    Thanks for your advice firstly.
    I am not able to reproduce the problem at this moment so I do not have
    the thread
    dump for review. But I did take a look at the Queue Length in the
    WL
    console
    GUI at the time hanging and I found it kept to be 0 or 1. Also, the
    heap size
    was just about 60%.
    But I have missed out one thing in my previous post. Besides the stateful
    session
    bean, another stateless session bean was used at the time user logon.
    And the
    stateless session bean just behaved the same as what the stateful one
    did at the
    time the WL was hanging, i.e. its "bean in cache" increased one when
    a user tried
    to logon.
    What else shall I look into in order to find out the root cause?
    Rob Woollen <[email protected]> wrote:
    Take a thread dump (or several) when the server is hung. If you can't
    figure out the problem from that, post the thread dump here, and we'll
    have a look.
    -- Rob
    Carol wrote:
    I have the problem with weblogic server 5.1 that it goes to hang
    within
    around
    an hour. There is no obvious error or exceptions shown in the weblogiclog.
    But at the time the weblogic server is hanging, I just observed fromthe weblogic
    console GUI that once a user logon to the web application (the webserver is iPlanet
    6.0), the number of bean in cache of the corresponding stateful sessionbean increases
    one but does not drop. Every single logon makes an increment 1 tothe # of the
    bean in cache of the corresponding session bean but no dropping.
    What the session bean does is actually obtain a connection from theweblogic server
    and get data from database. I went down to the database (Sybase
    ASE
    11.9) at
    the hanging time but did not see any long-running process nor tableblocking that
    may hinder the database response time. I do not understand what
    the
    session bean
    was waiting for.
    The only hints that I have is the below error message which was foundin the iPlanet
    web server log. Anyone please help to tell me what the problem canbe?
    Error found in iPlanet web server
    ============================
    [03/Jun/2003:02:51:26] failure ( 381): Error receiving connection(No process
    is on the other end of the pipe.)
    Full thread dump:
    "HighPriority TimeEventGenerator" daemon prio=9 tid=0x6ef920 nid=0x77waiting on monitor [0xaf681000..0xaf6819d8]
         at java.lang.Object.wait(Native Method)
         at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
         at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:138)
         at java.lang.Thread.run(Thread.java:484)
    "SSLListenThread" prio=5 tid=0x88e860 nid=0x76 runnable [0xafd81000..0xafd819d8]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:468)
         at java.net.ServerSocket.implAccept(ServerSocket.java:243)
         at java.net.ServerSocket.accept(ServerSocket.java:222)
         at weblogic.security.SSL.SSLServerSocket.acceptNoHandshake(SSLServerSocket.java:126)
         at weblogic.security.SSL.SSLServerSocket.accept(SSLServerSocket.java:117)
         at weblogic.t3.srvr.ListenThread.run(ListenThread.java:255)
    "ListenThread" prio=5 tid=0x88e110 nid=0x75 runnable [0xaf781000..0xaf7819d8]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:468)
         at java.net.ServerSocket.implAccept(ServerSocket.java:243)
         at java.net.ServerSocket.accept(ServerSocket.java:222)
         at weblogic.t3.srvr.ListenThread.run(ListenThread.java:255)
    "GC Daemon" daemon prio=2 tid=0x724590 nid=0x72 waiting on monitor[0xaff81000..0xaff819d8]
         at java.lang.Object.wait(Native Method)
         at sun.misc.GC$Daemon.run(GC.java:103)
    "RMI RenewClean-[127.0.0.1:55045]" daemon prio=5 tid=0x724460 nid=0x71waiting on monitor [0xb0581000..0xb05819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:108)
         at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:495)
         at java.lang.Thread.run(Thread.java:484)
    "ExecuteThread: '14' for queue: 'JmsDispatcher'" daemon prio=5 tid=0xe9160nid=0x6e waiting on monitor [0xb0781000..0xb07819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '13' for queue: 'JmsDispatcher'" daemon prio=5 tid=0xe8678nid=0x6d waiting on monitor [0xb0881000..0xb08819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '12' for queue: 'JmsDispatcher'" daemon prio=5 tid=0xe7ca8nid=0x6c waiting on monitor [0xb0981000..0xb09819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '11' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b6498nid=0x6b waiting on monitor [0xb0a81000..0xb0a819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '10' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b59b0nid=0x6a waiting on monitor [0xb0b81000..0xb0b819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '9' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b4ec8nid=0x69 waiting on monitor [0xb0c81000..0xb0c819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '8' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b43e0nid=0x68 waiting on monitor [0xb0d81000..0xb0d819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '7' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b38f8nid=0x67 waiting on monitor [0xb0e81000..0xb0e819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '6' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b2e10nid=0x66 waiting on monitor [0xb0f81000..0xb0f819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '5' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b10d8nid=0x65 waiting on monitor [0xb1081000..0xb10819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '4' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b05f0nid=0x64 waiting on monitor [0xb1181000..0xb11819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '3' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b1c50nid=0x63 waiting on monitor [0xb1281000..0xb12819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b1aa8nid=0x62 waiting on monitor [0xb1381000..0xb13819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b1900nid=0x61 waiting on monitor [0xb1481000..0xb14819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: 'JmsDispatcher'" daemon prio=5 tid=0x3b13d0nid=0x60 waiting on monitor [0xb1581000..0xb15819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: 'weblogic.transaction.AsyncQueue'" daemonprio=5 tid=0x2e9e18 nid=0x5f waiting on monitor [0xb1681000..0xb16819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: 'weblogic.transaction.AsyncQueue'" daemonprio=5 tid=0x2e8160 nid=0x5e waiting on monitor [0xb1781000..0xb17819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: 'weblogic.transaction.AsyncQueue'" daemonprio=5 tid=0x2e98e8 nid=0x5d waiting on monitor [0xb1881000..0xb18819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '9' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x553848 nid=0x5c waiting on monitor [0xb1981000..0xb19819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '8' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x552d60 nid=0x5b waiting on monitor [0xb1a81000..0xb1a819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '7' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x5522f0 nid=0x5a waiting on monitor [0xb1b81000..0xb1b819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '6' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x4f0150 nid=0x59 waiting on monitor [0xb1c81000..0xb1c819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '5' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x4f0f18 nid=0x58 waiting on monitor [0xb1d81000..0xb1d819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '4' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x4f0d48 nid=0x57 waiting on monitor [0xb1e81000..0xb1e819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '3' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x541870 nid=0x56 waiting on monitor [0xb1f81000..0xb1f819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x548ac8 nid=0x55 waiting on monitor [0xb2081000..0xb20819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x547390 nid=0x54 waiting on monitor [0xb2181000..0xb21819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: '__weblogic_admin_rmi_queue'" daemonprio=5 tid=0x5414c8 nid=0x53 waiting on monitor [0xb2281000..0xb22819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: '__weblogic_admin_html_queue'" daemonprio=5 tid=0x4f3d60 nid=0x52 waiting on monitor [0xb2381000..0xb23819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: '__weblogic_admin_html_queue'" daemonprio=5 tid=0x5469c0 nid=0x51 waiting on monitor [0xb2481000..0xb24819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "TimeEventGenerator" daemon prio=5 tid=0x4f0378 nid=0x50 waiting onmonitor [0xb2581000..0xb25819d8]
         at java.lang.Object.wait(Native Method)
         at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
         at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:138)
         at java.lang.Thread.run(Thread.java:484)
    "TimeEventGenerator" daemon prio=5 tid=0x542270 nid=0x4f waiting onmonitor [0xb2681000..0xb26819d8]
         at java.lang.Object.wait(Native Method)
         at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
         at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:138)
         at java.lang.Thread.run(Thread.java:484)
    "SpinnerRandomSource" daemon prio=5 tid=0x4f2ba0 nid=0x4e waiting onmonitor [0xb2781000..0xb27819d8]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:57)
         at java.lang.Thread.run(Thread.java:484)
    "ExecuteThread: '64' for queue: 'default'" daemon prio=5 tid=0x4ee2c8nid=0x4d waiting for monitor entry [0xb2881000..0xb28819d8]
         at weblogic.jdbc.jts.Connection.internalRollback(Connection.java:673)
         at weblogic.jdbc.jts.Connection.rollback(Connection.java:431)
         at weblogic.transaction.internal.ServerResourceInfo.rollback(ServerResourceInfo.java:1226)
         at weblogic.transaction.internal.ServerResourceInfo.rollback(ServerResourceInfo.java:545)
         at weblogic.transaction.internal.ServerSCInfo.startRollback(ServerSCInfo.java:372)
         at weblogic.transaction.internal.ServerTransactionImpl.localRollback(ServerTransactionImpl.java:1419)
         at weblogic.transaction.internal.ServerTransactionImpl.globalRollback(ServerTransactionImpl.java:2012)
         at weblogic.transaction.internal.TransactionImpl$1.execute(TransactionImpl.java:1559)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '63' for queue: 'default'" daemon prio=5 tid=0x4ed7e0nid=0x4c waiting for monitor entry [0xb2981000..0xb29819d8]
         at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:371)
         at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:24)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '62' for queue: 'default'" daemon prio=5 tid=0x4eccf8nid=0x4b waiting for monitor entry [0xb2a80000..0xb2a819d8]
         at java.sql.DriverManager.getConnection(DriverManager.java:144)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:79)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:96)
         at com.xxxxx.infra.ffff.agent.ppppoo.ppppooBean.init(ppppooBean.java:141)
         at com.xxxxx.infra.ffff.agent.ppppoo.ppppooBean.ejbCreate(ppppooBean.java:121)
         at com.xxxxx.infra.ffff.agent.ppppoo.ppppooBean_9cncmc_Impl.ejbCreate(ppppooBean_9cncmc_Impl.java:112)
         at java.lang.reflect.Method.invoke(Native Method)
         at weblogic.ejb20.pool.StatelessSessionPool.createBean(StatelessSessionPool.java:148)
         at weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:112)
         at weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:160)
         at weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:117)
         at weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java:63)
         at com.xxxxx.infra.ffff.agent.ppppoo.ppppooBean_9cncmc_EOImpl.startRequest(ppppooBean_9cncmc_EOImpl.java:26)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean.executeRequest(ffffShellRequestBean.java:326)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean.startRequest(ffffShellRequestBean.java:104)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean_7gya1w_EOImpl.startRequest(ffffShellRequestBean_7gya1w_EOImpl.java:37)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean_7gya1w_EOImpl_WLSkel.invoke(UnknownSource)
         at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
         at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:93)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
         at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '61' for queue: 'default'" daemon prio=5 tid=0x4eb3f8nid=0x4a runnable [0xb2b81000..0xb2b819d8]
         at weblogic.socket.PosixSocketMuxer.poll(Native Method)
         at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:371)
         at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:24)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '60' for queue: 'default'" daemon prio=5 tid=0x4ea910nid=0x49 waiting for monitor entry [0xb2c81000..0xb2c819d8]
         at weblogic.jdbc.jts.Connection.internalRollback(Connection.java:673)
         at weblogic.jdbc.jts.Connection.rollback(Connection.java:431)
         at weblogic.transaction.internal.ServerResourceInfo.rollback(ServerResourceInfo.java:1226)
         at weblogic.transaction.internal.ServerResourceInfo.rollback(ServerResourceInfo.java:545)
         at weblogic.transaction.internal.ServerSCInfo.startRollback(ServerSCInfo.java:372)
         at weblogic.transaction.internal.ServerTransactionImpl.localRollback(ServerTransactionImpl.java:1419)
         at weblogic.transaction.internal.ServerTransactionImpl.globalRetryRollback(ServerTransactionImpl.java:2152)
         at weblogic.transaction.internal.ServerTransactionImpl.access$3(ServerTransactionImpl.java:2123)
         at weblogic.transaction.internal.ServerTransactionImpl$1.execute(ServerTransactionImpl.java:2291)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '59' for queue: 'default'" daemon prio=5 tid=0x4e9e28nid=0x48 waiting for monitor entry [0xb2d81000..0xb2d819d8]
         at java.sql.DriverManager.getConnection(DriverManager.java:144)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:79)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:96)
         at com.xxxxx.apps.aaaa.ejb.uuuii.loaduuuii(uuuii.java:73)
         at com.xxxxx.apps.aaaa.ejb.uuuii.<init>(uuuii.java:63)
         at com.xxxxx.apps.aaaa.ejb.eeeeessBean.enInit(eeeeessBean.java:30)
         at com.xxxxx.apps.aaaa.ejb.eeeeessBean_2i6_EOImpl.enInit(eeeeessBean_2i6_EOImpl.java:221)
         at com.xxxxx.apps.aaaa.ejb.eeeeessBean_2i6_EOImpl_WLSkel.invoke(UnknownSource)
         at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
         at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '58' for queue: 'default'" daemon prio=5 tid=0x4e9340nid=0x47 waiting for monitor entry [0xb2e81000..0xb2e819d8]
         at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:371)
         at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:24)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '57' for queue: 'default'" daemon prio=5 tid=0x4e8858nid=0x46 waiting for monitor entry [0xb2f81000..0xb2f819d8]
         at java.sql.DriverManager.getConnection(DriverManager.java:144)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:79)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:96)
         at com.xxxxx.apps.aaaa.ejb.uuuii.loaduuuii(uuuii.java:73)
         at com.xxxxx.apps.aaaa.ejb.uuuii.<init>(uuuii.java:63)
         at com.xxxxx.apps.aaaa.ejb.eeeeessBean.enInit(eeeeessBean.java:30)
         at com.xxxxx.apps.aaaa.ejb.eeeeessBean_2i6_EOImpl.enInit(eeeeessBean_2i6_EOImpl.java:221)
         at com.xxxxx.apps.aaaa.ejb.eeeeessBean_2i6_EOImpl_WLSkel.invoke(UnknownSource)
         at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
         at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '56' for queue: 'default'" daemon prio=5 tid=0x4e7d70nid=0x45 waiting for monitor entry [0xb3080000..0xb30819d8]
         at java.sql.DriverManager.getConnection(DriverManager.java:144)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:79)
         at com.xxxxx.infra.db.ejb.weblogic.DBAccess.<init>(DBAccess.java:96)
         at com.xxxxx.infra.ffff.agent.ppppooII.ppppooBean.init(ppppooBean.java:164)
         at com.xxxxx.infra.ffff.agent.ppppooII.ppppooBean.ejbCreate(ppppooBean.java:142)
         at com.xxxxx.infra.ffff.agent.ppppooII.ppppooBean_sl938_Impl.ejbCreate(ppppooBean_sl938_Impl.java:112)
         at java.lang.reflect.Method.invoke(Native Method)
         at weblogic.ejb20.pool.StatelessSessionPool.createBean(StatelessSessionPool.java:148)
         at weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:112)
         at weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:160)
         at weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:117)
         at weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java:63)
         at com.xxxxx.infra.ffff.agent.ppppooII.ppppooBean_sl938_EOImpl.startRequest(ppppooBean_sl938_EOImpl.java:26)
         at com.xxxxx.infra.ffff.agent.ppppooII.ppppooBean_sl938_EOImpl_WLSkel.invoke(UnknownSource)
         at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
         at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:93)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
         at weblogic.rmi.internal.BasicServerRef.dispatch(BasicServerRef.java:158)
         at weblogic.rmi.internal.ServerRequest.sendOneWayRaw(ServerRequest.java:92)
         at weblogic.rmi.internal.ServerRequest.sendReceive(ServerRequest.java:112)
         at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:253)
         at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:220)
         at weblogic.rmi.internal.ProxyStub.invoke(ProxyStub.java:35)
         at $Proxy222.startRequest(Unknown Source)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean.executeRequest(ffffShellRequestBean.java:326)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean.startRequest(ffffShellRequestBean.java:104)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean_7gya1w_EOImpl.startRequest(ffffShellRequestBean_7gya1w_EOImpl.java:37)
         at com.xxxxx.infra.ffff.agent.ffffShellRequest.ffffShellRequestBean_7gya1w_EOImpl_WLSkel.invoke(UnknownSource)
         at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
         at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:93)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
         at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '55' for queue: 'default'" daemon prio=5 tid=0x4e7280nid=0x44 waiting for monitor entry [0xb3180000..0xb31819d8]
         at weblogic.common.internal.ResourceAllocator.markBorrowed(ResourceAllocator.java:332)
         at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:446)
         at weblogic.common.internal.ResourceAllocator.reserve(ResourceAllocator.java:400)
         at weblogic.common.internal.ResourceAllocator.reserveNoWait(ResourceAllocator.java:368)
         at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:166)
         at weblogic.jdbc.common.internal.ConnectionPool.reserveNoWait(ConnectionPool.java:127)
         at weblogic.jdbc.common.internal.RmiDataSource.getPoolConnection(RmiDataSource.java:194)
         at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:219)
         at weblogic.jdbc.common.internal.RmiDataSource_WLSkel.invoke(UnknownSource)
         at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
         at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:93)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
         at weblogic.rmi.internal.BasicE

  • JMS server hangs!?

    Hi all:
    It seems the JMS server hangs in WLS 6.1 after one Tx timeout due to
    some reason. The following is the tread dump. Can anyone tell me why? or
    any hints?
    The WLS server still functions, web server and app server both are
    responding to request. Only the JMS server hangs.
    The exception is also attached as follows.
    Thanks.
    wang minjiang
    ------------------------------------------------exception
    ERROR | 2001-08-30 16:27:09.136 |
    com.kbf.ejb.exchangerate.ExchangeRateBean_nnqi6b_Impl | |
    java.sql.SQLException: The transaction is no longer active (status =
    Marked rollback.
    [Reason=weblogic.transaction.internal.TimedOutException: Transaction
    timed out after 31 seconds
    Name=[EJB
    com.kbf.ejb.exchangerate.ExchangeRateBean.getLastUpdatedRates()],Xid=18:1c1de34f6566c5b7(6971341),Status=Active,numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds
    since begin=31,seconds left=30,activeThread=Thread[ExecuteThread: '13'
    for queue: 'default',5,Thread Group for Queue:
    'default'],ServerResourceInfo[weblogic.jdbc.jts.Connection]=(state=started,assigned=none),SCInfo[kbf+kbfServer]=(state=active),properties=({ISOLATION
    LEVEL=2, weblogic.transaction.name=[EJB
    com.kbf.ejb.exchangerate.ExchangeRateBean.getLastUpdatedRates()],
    weblogic.jdbc=t3://192.168.100.89:80}),OwnerTransactionManager=ServerTM[ServerCoordinatorDescriptor=(CoordinatorURL=kbfServer+192.168.100.89:80+kbf+,
    Resources={})],CoordinatorURL=kbfServer+192.168.100.89:80+kbf+)]). No
    further JDBC access is allowed within this transaction.
    at weblogic.jdbc.jts.Connection.checkIfRolledBack(Connection.java:501)
    at weblogic.jdbc.jts.ResultSet.next(ResultSet.java:279)
    at
    weblogic.jdbc.rmi.internal.ResultSetImpl.next(ResultSetImpl.java:132)
    at
    weblogic.jdbc.rmi.internal.ResultSetStraightReader.next(ResultSetStraightReader.java:27)
    at weblogic.jdbc.rmi.SerialResultSet.next(SerialResultSet.java:71)
    at
    com.fairex.base.coarsegrain.CoarseGrainBean.selectData(CoarseGrainBean.java:427)
    at
    com.kbf.ejb.exchangerate.ExchangeRateBean.getLastUpdatedRates(ExchangeRateBean.java:51)
    at
    com.kbf.ejb.exchangerate.ExchangeRateBean_nnqi6b_EOImpl.getLastUpdatedRates(ExchangeRateBean_nnqi6b_EOImpl.java:207)
    at
    com.kbf.ejb.remotemgr.RemoteMgrBean.getOldCurrPairRates(RemoteMgrBean.java:1176)
    at
    com.kbf.ejb.remotemgr.RemoteMgrBean_l6efiq_EOImpl.getOldCurrPairRates(RemoteMgrBean_l6efiq_EOImpl.java:125)
    at
    com.kbf.ejb.remotemgr.RemoteMgrBean_l6efiq_EOImpl_WLSkel.invoke(Unknown
    Source)
    at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:288)
    at
    weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:257)
    at
    weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:22)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    ------------------------------------------------thread dump
    Full thread dump:
    "HighPriority TimeEventGenerator" daemon prio=9 tid=0x989888 nid=0x8dc
    waiting o
    n monitor [0x1229f000..0x1229fdc4]
    at java.lang.Object.wait(Native Method)
    at
    weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
    at
    weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerat
    or.java:138)
    at java.lang.Thread.run(Thread.java:484)
    "SSLListenThread" prio=5 tid=0xfbdf940 nid=0x460 runnable
    [0x1224f000..0x1224fdc
    4]
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:421)
    at java.net.ServerSocket.implAccept(ServerSocket.java:243)
    at java.net.ServerSocket.accept(ServerSocket.java:222)
    at
    weblogic.security.SSL.SSLServerSocket.acceptNoHandshake(SSLServerSock
    et.java:126)
    at
    weblogic.security.SSL.SSLServerSocket.accept(SSLServerSocket.java:117
    at weblogic.t3.srvr.ListenThread.run(ListenThread.java:255)
    "ListenThread" prio=5 tid=0xfbdf818 nid=0x348 runnable
    [0x1220f000..0x1220fdc4]
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:421)
    at java.net.ServerSocket.implAccept(ServerSocket.java:243)
    at java.net.ServerSocket.accept(ServerSocket.java:222)
    at weblogic.t3.srvr.ListenThread.run(ListenThread.java:255)
    "Application Manager Thread" prio=5 tid=0xf7194e8 nid=0x6fc waiting on
    monitor [
    0x121cf000..0x121cfdc4]
    at java.lang.Thread.sleep(Native Method)
    at
    weblogic.management.mbeans.custom.ApplicationManager$ApplicationPolle
    r.run(ApplicationManager.java:1016)
    "MatchMaker" prio=5 tid=0xf64d410 nid=0x894 waiting on monitor
    [0x1218f000..0x12
    18fdc4]
    at java.lang.Object.wait(Native Method)
    at com.kbf.ejb.ordermgr.MatchMaker.run(MatchMaker.java:86)
    at java.lang.Thread.run(Thread.java:484)
    "OrderExpiryCheck" prio=5 tid=0xf11a038 nid=0x80c waiting on monitor
    [0x1214f000
    ..0x1214fdc4]
    at java.lang.Object.wait(Native Method)
    at
    com.kbf.startup.OrderExpiryCheck.run(OrderExpiryCheck.java:102)
    at java.lang.Thread.run(Thread.java:484)
    "ExcelRateServer" prio=5 tid=0xf989d48 nid=0x770 runnable
    [0x1210f000..0x1210fdc
    4]
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:421)
    at java.net.ServerSocket.implAccept(ServerSocket.java:243)
    at java.net.ServerSocket.accept(ServerSocket.java:222)
    at
    com.kbf.rates.ExcelRateServer.monitorRates(ExcelRateServer.java:118)
    at com.kbf.rates.ARateServer.run(ARateServer.java:129)
    at java.lang.Thread.run(Thread.java:484)
    "RateBuffer" prio=5 tid=0xf989c20 nid=0x8d4 waiting on monitor
    [0x120cf000..0x12
    0cfdc4]
    at java.lang.Object.wait(Native Method)
    at com.kbf.rates.RateBuffer.run(RateBuffer.java:67)
    at java.lang.Thread.run(Thread.java:484)
    "Thread-3" prio=5 tid=0xfba9538 nid=0x580 waiting on monitor
    [0x1208f000..0x1208
    fdc4]
    at java.lang.Thread.sleep(Native Method)
    at
    com.fairex.ordermatch.intf.liquidity.LiquidityImpl.run(LiquidityImpl.
    java:532)
    at java.lang.Thread.run(Thread.java:484)
    "T754" prio=5 tid=0xfba9410 nid=0x5a8 waiting on monitor
    [0x1204f000..0x1204fdc4
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at com.fairex.resource.LinearQueue.dequeue(LinearQueue.java:82)
    at
    com.fairex.resource.ThreadPool$PooledThread.run(ThreadPool.java:199)
    "T754" prio=5 tid=0xfaed538 nid=0x78c waiting on monitor
    [0x1200f000..0x1200fdc4
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at com.fairex.resource.LinearQueue.dequeue(LinearQueue.java:82)
    at
    com.fairex.resource.ThreadPool$PooledThread.run(ThreadPool.java:199)
    "T754" prio=5 tid=0xfaed410 nid=0x5c0 waiting on monitor
    [0x11fcf000..0x11fcfdc4
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at com.fairex.resource.LinearQueue.dequeue(LinearQueue.java:82)
    at
    com.fairex.resource.ThreadPool$PooledThread.run(ThreadPool.java:199)
    "T754" prio=5 tid=0xf21b540 nid=0x248 waiting on monitor
    [0x11f8f000..0x11f8fdc4
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at com.fairex.resource.LinearQueue.dequeue(LinearQueue.java:82)
    at
    com.fairex.resource.ThreadPool$PooledThread.run(ThreadPool.java:199)
    "T754" prio=5 tid=0xf21b010 nid=0x43c waiting on monitor
    [0x11f4f000..0x11f4fdc4
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at com.fairex.resource.LinearQueue.dequeue(LinearQueue.java:82)
    at
    com.fairex.resource.ThreadPool$PooledThread.run(ThreadPool.java:199)
    "RateDB" prio=5 tid=0xfa80720 nid=0x780 waiting on monitor
    [0x11f0f000..0x11f0fd
    c4]
    at java.lang.Object.wait(Native Method)
    at com.kbf.rates.RateDB.run(RateDB.java:62)
    at java.lang.Thread.run(Thread.java:484)
    "T43" prio=5 tid=0xeef5ff0 nid=0x918 waiting on monitor
    [0x11ecf000..0x11ecfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at com.fairex.resource.LinearQueue.dequeue(LinearQueue.java:82)
    at
    com.fairex.resource.ThreadPool$PooledThread.run(ThreadPool.java:199)
    "Thread-2" prio=5 tid=0xefa5d38 nid=0x930 waiting on monitor
    [0x11e8f000..0x11e8
    fdc4]
    at java.lang.Thread.sleep(Native Method)
    at
    com.fairex.dispatch.WebSessionManager.run(WebSessionManager.java:413)
    at java.lang.Thread.run(Thread.java:484)
    "ExecuteThread: '0' for queue: 'JMS.TimerTreePool'" daemon prio=5
    tid=0xf0594b8
    nid=0x724 waiting on monitor [0x10e4f000..0x10e4fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '3' for queue: 'JMS.TimerClientPool'" daemon prio=5
    tid=0xf2d4e4
    0 nid=0x6bc waiting on monitor [0x10e0f000..0x10e0fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: 'JMS.TimerClientPool'" daemon prio=5
    tid=0xf2d4c9
    0 nid=0x6b8 waiting on monitor [0x10dcf000..0x10dcfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: 'JMS.TimerClientPool'" daemon prio=5
    tid=0xf06fb4
    8 nid=0x6a4 waiting on monitor [0x10d8f000..0x10d8fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: 'JMS.TimerClientPool'" daemon prio=5
    tid=0xef0c9a
    8 nid=0x8f4 waiting on monitor [0x10d4f000..0x10d4fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '14' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf1e9e50 nid
    =0x5a4 waiting on monitor [0x10d0f000..0x10d0fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '13' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf160e80 nid
    =0x704 waiting on monitor [0x10ccf000..0x10ccfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '12' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf226e98 nid
    =0x72c waiting on monitor [0x10c8f000..0x10c8fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '11' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf226c20 nid
    =0x6e0 waiting on monitor [0x10c4f000..0x10c4fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '10' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf0b9c70 nid
    =0x850 waiting on monitor [0x10c0f000..0x10c0fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '9' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xef2c558 nid=
    0x2bc waiting on monitor [0x10bcf000..0x10bcfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '8' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf3e4cc8 nid=
    0x8f8 waiting on monitor [0x10b8f000..0x10b8fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '7' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf1e0c20 nid=
    0x608 waiting on monitor [0x10b4f000..0x10b4fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '6' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf650528 nid=
    0x880 waiting on monitor [0x10b0f000..0x10b0fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '5' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf127558 nid=
    0x5fc waiting on monitor [0x10acf000..0x10acfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '4' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf178ea8 nid=
    0x790 waiting on monitor [0x10a8f000..0x10a8fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '3' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf127430 nid=
    0x810 waiting on monitor [0x10a4f000..0x10a4fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf11bd08 nid=
    0x604 waiting on monitor [0x10a0f000..0x10a0fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf11bb58 nid=
    0x35c waiting on monitor [0x109cf000..0x109cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: 'JmsDispatcher'" daemon prio=5
    tid=0xf1ae170 nid=
    0x538 waiting on monitor [0x1098f000..0x1098fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: 'weblogic.transaction.AsyncQueue'" daemon
    prio=5
    tid=0xeee1650 nid=0x3c8 waiting on monitor [0x1094f000..0x1094fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: 'weblogic.transaction.AsyncQueue'" daemon
    prio=5
    tid=0xf0205a8 nid=0x378 waiting on monitor [0x1090f000..0x1090fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: 'weblogic.transaction.AsyncQueue'" daemon
    prio=5
    tid=0xefb0450 nid=0x7b4 waiting on monitor [0x108cf000..0x108cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '9' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf365b38 nid=0x890 waiting on monitor [0x1088f000..0x1088fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '8' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf3658c0 nid=0x220 waiting on monitor [0x1084f000..0x1084fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '7' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf2d5d18 nid=0x240 waiting on monitor [0x1080f000..0x1080fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '6' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf4b5e90 nid=0x4b4 waiting on monitor [0x107cf000..0x107cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '5' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf4b5c18 nid=0x7f8 waiting on monitor [0x1078f000..0x1078fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '4' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf739a38 nid=0x7e4 waiting on monitor [0x1074f000..0x1074fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '3' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf098d88 nid=0x59c waiting on monitor [0x1070f000..0x1070fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf394bb0 nid=0x65c waiting on monitor [0x106cf000..0x106cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    x9ad970 nid=0x4f0 waiting on monitor [0x1068f000..0x1068fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: '__weblogic_admin_rmi_queue'" daemon
    prio=5 tid=0
    xf1c0618 nid=0x754 waiting on monitor [0x1064f000..0x1064fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: '__weblogic_admin_html_queue'" daemon
    prio=5 tid=
    0xf090210 nid=0x82c waiting on monitor [0x1060f000..0x1060fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: '__weblogic_admin_html_queue'" daemon
    prio=5 tid=
    0xf0a8a28 nid=0x620 waiting on monitor [0x105cf000..0x105cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "TimeEventGenerator" daemon prio=5 tid=0xf01ee28 nid=0x630 waiting on
    monitor [0
    x1058f000..0x1058fdc4]
    at java.lang.Object.wait(Native Method)
    at
    weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
    at
    weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerat
    or.java:138)
    at java.lang.Thread.run(Thread.java:484)
    "TimeEventGenerator" daemon prio=5 tid=0x9adc50 nid=0x75c waiting on
    monitor [0x
    1054f000..0x1054fdc4]
    at java.lang.Object.wait(Native Method)
    at
    weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
    at
    weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerat
    or.java:138)
    at java.lang.Thread.run(Thread.java:484)
    "SpinnerRandomSource" daemon prio=5 tid=0xf11fdb8 nid=0x658 waiting on
    monitor [
    0x1050f000..0x1050fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource
    .java:57)
    at java.lang.Thread.run(Thread.java:484)
    "ExecuteThread: '14' for queue: 'default'" daemon prio=5 tid=0xf3b4df0
    nid=0x758
    runnable [0x104cf000..0x104cfdc4]
    at weblogic.socket.NTSocketMuxer.getNextSocket(Native Method)
    at
    weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:501)
    at
    weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:
    24)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '13' for queue: 'default'" daemon prio=5 tid=0xf14fe08
    nid=0x1bc
    waiting on monitor [0x1048f000..0x1048fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '12' for queue: 'default'" daemon prio=5 tid=0xf426818
    nid=0x4d0
    runnable [0x1044f000..0x1044fdc4]
    at weblogic.socket.NTSocketMuxer.getNextSocket(Native Method)
    at
    weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:501)
    at
    weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:
    24)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '11' for queue: 'default'" daemon prio=5 tid=0xef9cd40
    nid=0x8e0
    waiting on monitor [0x1040f000..0x1040fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '10' for queue: 'default'" daemon prio=5 tid=0xefa78c0
    nid=0x514
    waiting on monitor [0x103cf000..0x103cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '9' for queue: 'default'" daemon prio=5 tid=0xf2d3008
    nid=0x5d0
    waiting on monitor [0x1038f000..0x1038fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '8' for queue: 'default'" daemon prio=5 tid=0xef51818
    nid=0x6f0
    waiting on monitor [0x1034f000..0x1034fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '7' for queue: 'default'" daemon prio=5 tid=0x93dda8
    nid=0x7e8 w
    aiting on monitor [0x1030f000..0x1030fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '6' for queue: 'default'" daemon prio=5 tid=0xf2e7db8
    nid=0x71c
    waiting on monitor [0x102cf000..0x102cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '5' for queue: 'default'" daemon prio=5 tid=0xee98e78
    nid=0x79c
    waiting on monitor [0x1028f000..0x1028fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '4' for queue: 'default'" daemon prio=5 tid=0xf40ee70
    nid=0x5b0
    waiting on monitor [0x1024f000..0x1024fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '3' for queue: 'default'" daemon prio=5 tid=0xf42ee90
    nid=0x650
    waiting on monitor [0x1020f000..0x1020fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '2' for queue: 'default'" daemon prio=5 tid=0xf317e90
    nid=0x8f0
    waiting on monitor [0x101cf000..0x101cfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '1' for queue: 'default'" daemon prio=5 tid=0xef5f8c8
    nid=0x6d4
    waiting on monitor [0x1018f000..0x1018fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '0' for queue: 'default'" daemon prio=5 tid=0xefd2ec0
    nid=0x8b4
    waiting on monitor [0x1014f000..0x1014fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:420)
    at
    weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "Thread-0" daemon prio=10 tid=0xf212698 nid=0x900 waiting on monitor
    [0x1010f000
    ..0x1010fdc4]
    at java.lang.Thread.sleep(Native Method)
    at
    weblogic.transaction.internal.TransactionManagerImpl$1.run(Transactio
    nManagerImpl.java:1543)
    at java.lang.Thread.run(Thread.java:484)
    "Signal Dispatcher" daemon prio=10 tid=0x9a1d30 nid=0x77c waiting on
    monitor [0.
    .0]
    "Finalizer" daemon prio=9 tid=0x952630 nid=0x76c waiting on monitor
    [0xebcf000..
    0xebcfdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:108)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:123)
    at
    java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:162)
    "Reference Handler" daemon prio=10 tid=0x9519a0 nid=0x63c waiting on
    monitor [0x
    eb8f000..0xeb8fdc4]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java

    Wow Zach, comprehnsive response. I'm gonna take not of that response and
    use it on my team!
    Minjiang, the stack trace reports a transaction timeout. This can happen
    druing a long running bean process , and will be further aggrevated if you
    are running in debug mode. Try increasing the timeout in your deployment
    descriptor, or, optimize your code ;)
    "Zach" <[email protected]> wrote in message
    news:3b8e5ac8$[email protected]..
    JMS does not appear on any of the stacks in the thread dump.
    Please describe the actual symptoms you are seeing (as oppposed
    to the moer general statement that the JMS server hangs).
    Can you get an application to attach to the server (create
    an initial context and do some JNDI lookups). Can that
    application then create a JMS connection, a JMS session,
    and so forth. At what point does it hang. How do you
    know it is hung? what is not happening? what do you
    expect to happen? Do you have listeners that stop
    responding? Do you see messages in the destinations
    from the console? What JVM is this?
    Nteo that the exception/error you posted is for an invocation of
    com.kbf.ejb.remotemgr.RemoteMgrBean.getOldCurrPairRates()
    which has exactly nothing to do with JMS. It is not being invoked
    by JMS and does not appear to be invoking JMS.
    "minjiang" <[email protected]> wrote in message
    news:[email protected]..
    Hi all:
    It seems the JMS server hangs in WLS 6.1 after one Tx timeout due to
    some reason. The following is the tread dump. Can anyone tell me why? or
    any hints?
    The WLS server still functions, web server and app server both are
    responding to request. Only the JMS server hangs.
    The exception is also attached as follows.
    Thanks.
    wang minjiang

  • CC 2014 hangs all the time

    It's a disaster!
    (all new CC2014 apps installed and updated) premiere pro CC2014 ; trying to export media to media enconder CC2014 which hangs all the time. Working on the latest mbp i7.
    When one timeline is rendering in media encoder nothing can be added to it and premiere pro 2014 is significantly behaving like a mpb2,1 generation laptop.
    nothing of this was happening with my brand new laptop on CC
    Not ready with this release yet?
    A far from smooth update which I'am not used from U at adobe!
    [Moving to Premiere Pro... JTS]

    Mine works ok until I render out a video. It brought my i7 system to its knees. Rendering out an hour long H264 video  (in ppro, not media encoder) once the render was at 100% it just sat there for about 10 minutes finalizing or something? Anyways the whole system became almost unusable and every click or mouse move takes forever, like trying to eat peanut butter with a straw.
    My system is an i7 4770k with 16gb of ram and a GeForce GTX760
    windows 8.1

  • Weblogic 6.1 sp2 hangs after sometimes.

    Our server hangs after 1 hour or tow for no apperent reason. It seem like the port get chocked and there in no way I can ping it. The JVM is running and I do not see any core dump. Neiter I see any other error but the server just stops listening on its port.

    I cannot ping the server, I cannot get the core dump there is nothing I can to
    to the server. Bellow is the thread dump but it is not much help either.
    "ExecuteThread: '19' for queue: 'default'" daemon prio=5 tid=0x5d3778 nid=0x21
    waiting on monitor [0xb2a81000..0xb2a819d8]
         at java.lang.Object.wait(Native Method)
         - waiting on <b8c3b430> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         - locked <b8c3b430> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '18' for queue: 'default'" daemon prio=5 tid=0x5d2da8 nid=0x20
    waiting on monitor [0xb2b81000..0xb2b819d8]
         at java.lang.Object.wait(Native Method)
         - waiting on <b8c3b3c0> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         - locked <b8c3b3c0> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '17' for queue: 'default'" daemon prio=5 tid=0x597358 nid=0x1f
    runnable [0xb2c7f000..0xb2c819d8]
         at oracle.jdbc.oci8.OCIDBAccess.do_fetch(Native Method)
         at oracle.jdbc.oci8.OCIDBAccess.fetch(OCIDBAccess.java:1424)
         - locked <c53c7e90> (a oracle.jdbc.oci8.OCIDBAccess)
         at oracle.jdbc.driver.OracleStatement.doExecuteQuery(OracleStatement.java:1918)
         at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:2107)
         at oracle.jdbc.driver.OracleStatement.doScrollStmtExecuteQuery(OracleStatement.java:5597)
         at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:590)
         - locked <b85d6058> (a oracle.jdbc.driver.OracleStatement)
         - locked <c53c8fc0> (a oracle.jdbc.driver.OracleConnection)
         at weblogic.jdbc.jts.Statement.executeQuery(Statement.java:157)
         at weblogic.jdbc.rmi.internal.StatementImpl.executeQuery(StatementImpl.java:79)
         at weblogic.jdbc.rmi.SerialStatement.executeQuery(SerialStatement.java:80)
         at com.silknet.ebiz.silkdataobject.jdbc2.SilkJDBC2Executor.executeQuery(SilkJDBC2Executor.java:71)
         at com.silknet.ebiz.silkdataobject.jdbc2.SilkJDBC2Executor.executeQuery(SilkJDBC2Executor.java:36)
         at com.silknet.ebiz.silkdataobject.SilkDatabase.executeQuery(SilkDatabase.java:175)
         at com.silknet.ebiz.silkdataobject.SilkDatabase.executeQuery(SilkDatabase.java:85)
         at com.silknet.ebiz.silkdataobject.PlanExecuter.sqlToXML(PlanExecuter.java:1408)
         at com.silknet.ebiz.silkdataobject.PlanExecuter.executePlan(PlanExecuter.java:159)
         at com.silknet.ebiz.silkdataobject.SDOViewPlan.execute(SDOViewPlan.java:213)
         at com.silknet.ebiz.silkdataobject.SDOViewPlan.execute(SDOViewPlan.java:191)
         at com.silknet.ebiz.silkdataobject.Query.listInstances(Query.java:229)
         at com.silknet.ebiz.silkdataobject.ejb.QueryBean.listInstances(QueryBean.java:74)
         at com.silknet.ebiz.silkdataobject.ejb.QueryBean_rlzhdu_EOImpl.listInstances(QueryBean_rlzhdu_EOImpl.java:37)
         at java.lang.reflect.Method.invoke(Native Method)
         at com.silknet.ebiz.silkrequest.server.adapter.AdapterDirector.invokeEJB(AdapterDirector.java:451)
         at com.silknet.ebiz.silkrequest.server.adapter.AdapterDirector.invokeAdapter(AdapterDirector.java:153)
         at com.silknet.ebiz.silkrequest.server.bs.ServiceDirector.invokeService(ServiceDirector.java:100)
         at com.silknet.ebiz.silkrequest.server.bs.ServiceDirector.invokeService(ServiceDirector.java:159)
         at com.silknet.ebiz.silkrequest.server.bs.ServiceDirector.invokeService(ServiceDirector.java:228)
         at com.silknet.ebiz.silkrequest.client.LocalClient.invokeService(LocalClient.java:49)
         at com.silknet.ebiz.client.BSManager.sendRequests(BSManager.java:191)
         at com.silknet.ebiz.client.BSManager.sendRequests(BSManager.java:147)
         at com.silknet.ebiz.silkmodels.SilkModel.doQuery(SilkModel.java:174)
         at com.silknet.ebiz.silkmodels.BusinessObjectView.getData(BusinessObjectView.java:633)
         at com.bcbs.ebiz.uhv.RequestPopulationNew.populateFromSCR(Unknown Source)
         at jsp_servlet._uhv.__Request._jspService(__Request.java:231)
         at weblogic.servlet.jsp.JspBase.service(JspBase.java:27)
         at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:265)
         at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:200)
         at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:2495)
         at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2204)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:139)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    "ExecuteThread: '16' for queue: 'default'" daemon prio=5 tid=0x596870 nid=0x1e
    waiting on monitor [0xb2d81000..0xb2d819d8]
         at java.lang.Object.wait(Native Method)
         - waiting on <b8c3b2e0> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         - locked <b8c3b2e0> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)
    "ExecuteThread: '15' for queue: 'default'" daemon prio=5 tid=0x595d88 nid=0x1d
    waiting on monitor [0xb2e81000..0xb2e819d8]
         at java.lang.Object.wait(Native Method)
         - waiting on <b8c3b270> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:420)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:94)
         - locked <b8c3b270> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:118)

  • My Excel with PowerPivots acts weird, it hangs when i try to change font color

    Hi 
    I am facing a weird issue using PowerPivot in my workbook, there is a worksheet named Charts in my workbook and all the charts i am using resides in that worksheet. Whenever i try to change the font color in that sheet, my excel's color palette goes blur
    and then everything just freezes and turns black. This problem is happening only with Charts worksheet, when i try to change the color in other sheets, it works just, the problem is only with Charts sheet. File
    size is 16 MB, there are just 10 pivot tables with some calculations and some calculated fields. There are no add-ins other than PowerPivot. 
    Thanks,
    Shanker

    Hi,
    In regarding of the issue, please provide us more information to assist you better.
    Do you receive any error message?
    Which Excel version are you using?  Excel 2013 or other?
    Which workbook format are you using? XLS or XLSX?
    Are you performing any specific task?
    According to your description, this issue seems only occur with the special worksheet "Charts".
    Firstly, I recommend we copy all of the content to a new/blank Excel file to test.
    If it works fine, this issue might be caused by the "Charts" workbook itself. We'd better check the "Charts" workbook. Or re-build the file with a new workbook.
    If it still makes Excel hang with a new file, this issue might due to the content. Please check the content first.
    Secondly, we could follow below KB to troubleshoot this issue:
    https://support2.microsoft.com/kb/2758592/en-us?wa=wsignin1.0
    Thirdly, if the issue still exists, we may try to collect the Event log and App crash dump file to do advanced troubleshooting.
    Event log:
    http://windows.microsoft.com/en-US/windows7/Open-Event-Viewer
    App crash dump file:
    First enable app crash dump collection by copying following words into notepad, saving it as dump.reg and importing
    it:
    Windows Registry Editor Version 5.00
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\localdumps\EXCEL.EXE]
    "DumpFolder"=hex(2):63,00,3a,00,5c,00,63,00,72,00,61,00,73,00,68,00,64,00,75,\
      00,6d,00,70,00,73,00,00,00
    "DumpCount"=dword:00000010
    "DumpType"=dword:00000001
    "CustomDumpFlags"=dword:00000000
    Then, open Excel
    to repro the issue. If crash issue appeared, please find the crashdump file under c:\.
    To further help you, please upload this file into Skydrive and shared the link here.
    Also, you can try to analyze dump by yourself if you would like to:
    How to analyze app crash dump file:
    http://blogs.technet.com/b/askperf/archive/2007/05/29/basic-debugging-of-an-application-crash.aspx
    Hope it's helpful.
    Regards,
    George Zhao
    TechNet Community Support
    It's recommended to download and install
    Configuration Analyzer Tool (OffCAT), which is developed by Microsoft Support teams. Once the tool is installed, you can run it at any time to scan for hundreds of known issues in Office
    programs.

  • Just installed Firefox 4. It hangs when I open any website and type in, for example, an ID, or even just click on something, for example, "Compose", in my webmail site.

    Here's the detail of what happens.
    1. Turn on computer and then click on Firefox icon on desktop.
    2. Firefox opens. Everything looks OK. Mr Bunsen's graphic is working fine.
    3. Click on any webpage link. For example, a newspaper or an online webmail page. All looks OK until
    4. Click on anything or type anything, such as in ID in a box, and bingo, the top line of Firefox goes from dark blue to very pale blue and the computer is hung. You cannot get out of Firefox or get Firefox to work.
    5. Switch off computer.
    Is this perhaps due to a clash of Add-ons. I run Avast anti-virus. McAfee came with Firefox 4 and I did not reject it. Is it conflicting wth something in my system and causing it to hang?

    You can modify the pref <b>keyword.URL</b> on the <b>about:config</b> page to use Google's "I'm Feeling Lucky" or Google's "Browse By Name".
    * Google "I'm Feeling Lucky": http://www.google.com/search?btnI=I%27m+Feeling+Lucky&ie=UTF-8&oe=UTF-8&q=
    * Google "Browse by Name": http://www.google.com/search?ie=UTF-8&sourceid=navclient&gfns=1&q=
    * http://kb.mozillazine.org/keyword.URL
    * http://kb.mozillazine.org/Location_Bar_search

  • I upgraded to the latest version of Firefox but it hangs when I use a search site such as Google or Bing. How can I fix this?

    putting a search option in the search box to the right of the address bar opens up the full search window but then Firefox hangs.

    Your UserAgent string in Firefox is messed up by another program that you installed, and those websites don't know you are running Firefox 3.6.3 (which is what you do have installed).
    [http://en.wikipedia.org/wiki/User_Agent]
    type '''about:config''' in the URL bar and hit Enter
    ''If you see the warning, you can confirm that you want to access that page.''
    Filter = '''general.useragent.'''
    Right-click the preferences that are '''bold''', one line at a time, and select '''''Reset''''',
    Then restart Firefox

  • It hangs when playing games

    My system is Windows 7 - 64 bit operating system
    I've used Firefox for many many years and always praised it compared to EI. I am afraid I wont be able to praise it much more. It stinks. It hangs mostly when I am playing games or looking at pictures or videos. Its worse when on Facebook but also happens on other web site. I did all the troubleshooting, updates, remove old versions and re install, disable add ons all and all Its driving me insane. Can you solve the problem with a new versionof Firefox or do we have to flush the darn browser for good.
    Thank you

    anyone has a solution until a new version comes out?

  • When I boot up it hangs on the login screen for awhile, why?

    I have a mid 2011 Macbook Air 13".  It came with Lion but I recently updated to Mountain Lion.  When I wake the laptop from sleep everything is fine.  If it's been awhile since it slept (a couple of hours), the login screen hangs.  What happens is that it boots like normal and looks like everything is correct, but it won't let me type in my password.  After 10 seconds or so I can see a cursor come up but it won't let me type (but anything I do type at this point will later fill in the field after another 2-3 seconds or so).  One other thing that's weird is that when the screen first comes up it shows the battery percentage from when I put it back on the charger, then after everything catches up, it changes to show 100%.
    I hope I explained the problem well enough.  Basicly a "wake from sleep" is fine if it hasn't been long since the laptop slept, but if it's been awhile it takes about 15-20 seconds to boot up.
    Ideas?

    Here are some suggestions based on my recent issue with unable to get Mavericks up after upgrade - The Apple geniuses just simply couldnt figure this one out...despite it being in their shop for 3 days (They probably did not look at it...and asked if they can just wipe out everything and install fresh...i had said NO):
    Restart computer and hold Cmd+V when you hear chime. This will start the computer in Verbose mode.
    See the messages and see where the system boot fails. In my case, it was some backtrace IOStorageFamily issue with com.paragon-software.
    Google may be able to solve your Apple issues - just type the error message in and see if you get any useful suggestions. in most cases - this would be some faulty 3rd party drivers.....
    In my case, it was something stupid i had probably installed....I had to boot to safe mode (Hold Shift at boot chime) - open Terminal, delete the folder com.paragon-software.kext from System/Library/Extensions....that fixed the problem. Mavericks is working great now....
    Let me know if your debugging thing works...

  • Firefox hangs on opening requiring force quit I have snow leopard mac It hangs when I try safe mode. I need to know how to remove all traces of firefox on my mac so I can reinstall. I have tried terminal as advised on firefox webpage. Please advise m

    Firefox hangs on opening requiring force quit I have snow leopard Mac It even hangs when I try safe mode. I need to know how to remove all traces of firefox on my mac so I can reinstall. I have tried terminal as advised on firefox webpage. Please advise me
    == This happened ==
    Every time Firefox opened
    == I tried to update firefox add-ons ==
    == User Agent ==
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7

    Try a new profile. See [[Recovering important data from an old profile]]

  • I am having a problem opening my browser, the broswer is hanging up after slowly opening

    Hello, I have been having trouble with my firefox browser for quite some time now, it just stopped working on me about 5 or 6 months ago. I have switched over to Internet Explorer which has still been working fine. I like using Firefox much more than Explorer these days, but I am unfortunately not good much with fixing computer problems and have pretty much extinguished all my available options for repairing it myself.
    The problem with Firefox is now when I click on the shortcut link on my desktop, it takes a good 5 to 10 minutes to open up the browser now. Once it does finally open, you cannot type or click on anything it is hung up and will not allow you to go to another site(the home page will still load, with current news and other information)
    I cannot remember anything that would have caused it to crash or get hung up like this, but like I said I am not a computer expert by any means. I do know that I am ready to get Mozilla back to running smoothly as before, I had just gotten really used to it(I had been using internet explorer).
    I also uninstalled Firefox completely and reinstalled it, I figured if nothing else I tried worked that usually is the best option...it still did/does the same thing, hangs up everytime you try to run/open it.
    I checked your online solution for what to do on a hanging first page, I saw something about pentium 4's running Windows Vista or XP...my computer runs Windows 7 (was actually purchased running the terribly bug ridden and error-prone Vista, and lucky I was able to obtain a free upgrade to 7). So I was thinking maybe that had something to do with it, I had some problem in the past, with what I cannot remember, but it traced back to something from when Vista was on my computer(even though it was no longer my operating system I'll be darn if it wasn't still causing me headaches! How fitting.)
    My computer in question is a Dell Studio XPS 1640 which I bought new almost two years ago exactly)

    Did you check what I would consider the first place to look:
    * http://kb.mozillazine.org/Problematic_extensions
    If the site is still down (at least 12 hours) when you read this try [http://webcache.googleusercontent.com/search?q=cache:http%3A//kb.mozillazine.org/Problematic_extensions Google's cache]
    Link or url would make it easier to see which page you tried rather than "your online solution for ...".
    * http://kb.mozillazine.org/Firefox_hangs ([http://webcache.googleusercontent.com/search?q=cache:http%3A//kb.mozillazine.org/Firefox_hangs cached version])
    * http://kb.mozillazine.org/Firefox_crashes ([http://webcache.googleusercontent.com/search?q=cache:http%3A//kb.mozillazine.org/Firefox_crashes cached version])
    * [https://support.mozilla.com/kb/Firefox%20hangs#os=mac&amp;browser=fx4 Firefox hangs | Troubleshooting | Firefox Help]

Maybe you are looking for

  • Can't drag files... need help, please!

    Hoping someone can help me. I came the other day to find that none of the files on our iMac G5 could be dragged. Stuff on the desktop couldn't be moved, re-arranged, etc. Even in iTunes, files couldn't be dragged to new playlists. They'll still open

  • Hmm...HELP!!!

    Hello everyone, Hey! Right now I am looking to purchase my first ever Mac and am trying to seek some advice from you guys. I am inbetween a new 12" powerbook or a refurbished 15" powerbook (on sale section of site for $1449). The refurb has a combo d

  • Exchange rate updation in VA02

    Dear Experts We have sales order which is having around 5 line item and in that 1st line item alone has been invoiced in January and later items still not invoiced In the month of February, the invoice for 1st line item has been cancelled , so to avo

  • Missing link to Adapter Engine from XI 3.0 web start page.

    All, After deploying sda's for adapter framework core, framework and XI tools the Link to the Adapter Engine from the XI start page is now missing. Any ideas?? thanks, Linwood

  • KM Document iView and User Permissions

    Hi all, I have created a KM Document iView to display a document (stored within KM) to my users. However, it is happening something I believe is not correct. When I set only READ permission for this folder, the KM Document iView is not able to displa