Error Thread Dump

Hello, we have WebLogic Server 6.1 whit JDK 1.3.1_04 over Solaris 5.8 .
When we tried to take a stack dump (kill -3), caused the server to crash:
"TimeEventGenerator" daemon prio=5 tid=0xba9b98 nid=0x2f waiting on monitor [0x2d07f000..0x2d07fc68]
at java.lang.Object.wait(Native Method)
at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:279)
at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:138)
at java.lang.Thread.run(Thread.java:479)
"SpinnerRandomSource" daemon prio=5 tid=0x903138 nid=0x2e waiting on monitor [0x2d17f000..0x2d17fc68]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:415)
at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:57)
at java.lang.Thread.run(Thread.java:479)
"ExecuteThread: '4' for queue: 'SourcingConsole'" daemon prio=5 tid=0x9c7600 nid=0x2d waiting on monitor [0x2d27f000..0x2d27fc68]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:415)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:95)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
"ExecuteThread: '3' for queue: 'SourcingConsole'" daemon prio=5 tid=0x9c6b08 nid=0x2c waiting on monitor [0x2d37f000..0x2d37fc68]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:415)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:95)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
"ExecuteThread: '2' for queue: 'SourcingConsole'" daemon prio=5 tid=0x9c5f60 nid=0x2b waiting on monitor [0x2d47f000..0x2d47fc68]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:415)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:95)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
"ExecuteThread: '1' for queue: 'SourcingConsole'" daemon prio=5 tid=0xb07068 nid=0x2a waiting on monitor [0x2d57f000..0x2d57fc68]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:415)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:95)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
"ExecuteThread: '0' for queue: 'SourcingConsole'" daemon prio=5 tid=0x90a100 nid=0x29 waiting on monitor [0x2d67f000..0x2d67fc68]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:415)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:95)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
"ExecuteThread: '29' for queue: 'default'" daemon prio=5 tid=0x909608 nid=0x28 waiting for monitor entry [0x2d77f000..0x2d77fc68]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:584)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:24)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:140)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:121)
"ExecuteThread: '28' for queue: 'default'" daemon prio=5 tid=0x907ae8 nid=0x27 waiting for monitor entry [0x2d87f000..0x2d87fc68]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:584)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:24)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:140)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:121)
"ExecuteThread: '27' for queue: 'default'" daemon prio=5 tid=0x906ff0 nid=0x26 runnable [0x2d97f000..0x2d97fc68]
# HotSpot Virtual Machine Error, Internal Error
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
# Error happened during: print threads
# Error ID: 4E4D4554484F440E435050015E 01
# Problematic Thread: prio=5 tid=0x1faea8 nid=0x2 runnable
The starts options are -server -XX:SurvivorRatio=2 -XX:MaxNewSize=512m -XX:NewSize=512m -XX:PermSize=128m -XX:MaxPermSize=128m -XX:+UseLWPSynchronization -XX:+DisableExplicitGC-verbose:gc -Xnoclassgc -Xms3072m -Xmx3072m -DMAX_MEMORY=3072m
What do you reckon caused this?
Thanks

Hello,
Try dropping your -server option (or run in client mode -client)
This looks like your bug:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4339224
On another note your GC settings look a bit odd, are you sure they are optimal for your application?
cheers
Hussein Badakhchani
www.orbism.com

Similar Messages

  • A StuckThreadMaxTime error; who can help me to analyse the thread dumps

    The StuckThreadMaxTime error freeze my system, no one can logged in.I have double checked the thread dumps, but get no help.
    The following is the thread dumps:
    <2004-12-7 ÉÏÎç05ʱ56·Ö47Ãë GMT+08:00> <Alert> <Log Management> <BEA-170017> <The log file ./myserver/myserver.log will be rotated. Reopen the log file if tailing has stopped. This can happen on some platforms like Windows.>
    <2004-12-7 ÉÏÎç05ʱ56·Ö47Ãë GMT+08:00> <Alert> <Log Management> <BEA-170018> <The log file has been rotated to myserver.log00035. Log messages will continue to be logged in ./myserver/myserver.log.>
    <2004-12-7 ÉÏÎç10ʱ09·Ö32Ãë GMT+08:00> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default' has been busy for "646" seconds working on the request "com.huawei.oss.rms.resourceaccess.ResourceMgmtSession_1x9h7n_EOImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
    <2004-12-7 ÉÏÎç10ʱ09·Ö32Ãë GMT+08:00> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '13' for queue: 'weblogic.kernel.Default' has been busy for "651" seconds working on the request "com.huawei.oss.rms.resourceaccess.ResourceMgmtSession_1x9h7n_EOImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
    <2004-12-7 ÉÏÎç10ʱ09·Ö32Ãë GMT+08:00> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '14' for queue: 'weblogic.kernel.Default' has been busy for "650" seconds working on the request "com.huawei.oss.rms.resourceaccess.ResourceMgmtSession_1x9h7n_EOImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
    Full thread dump Java HotSpot(TM) VM (1.4.2 1.4.2.03-040401-18:59-PA_RISC2.0 PA2.0 (aCC_AP) mixed mode):
    "ExecuteThread: '14' for queue: 'JmsDispatcher'" daemon prio=5 tid=01c3f778 nid=71 lwp_id=39624 in Object.wait() [0x61abf000..0x61abf4f0]
         at java.lang.Object.wait(Native Method)
         - waiting on <6d1d24f8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <6d1d24f8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '13' for queue: 'JmsDispatcher'" daemon prio=5 tid=01c3f610 nid=70 lwp_id=39623 in Object.wait() [0x61b40000..0x61b404f0]
         at java.lang.Object.wait(Native Method)
         - waiting on <6d1d2578> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <6d1d2578> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '12' for queue: 'JmsDispatcher'" daemon prio=5 tid=01c3f4a8 nid=69 lwp_id=39622 in Object.wait() [0x61bc1000..0x61bc14f0]
         at java.lang.Object.wait(Native Method)
         - waiting on <6d1d25f8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <6d1d25f8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '11' for queue: 'JmsDispatcher'" daemon prio=5 tid=01c3f340 nid=68 lwp_id=39621 in Object.wait() [0x61c42000..0x61c424f0]
         at java.lang.Object.wait(Native Method)
         - waiting on <6d1d2678> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.Eÿ

    "ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=000883f8 nid=27 lwp_id=39205 runnable [0x630ec000..0x630eb4f0]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at oracle.net.ns.Packet.receive(Unknown Source)
         at oracle.net.ns.DataPacket.receive(Unknown Source)
         at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
         at oracle.net.ns.NetInputStream.read(Unknown Source)
         at oracle.net.ns.NetInputStream.read(Unknown Source)
         at oracle.net.ns.NetInputStream.read(Unknown Source)
         at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
         at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
         at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
         at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181)
         at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543)
         at oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:913)
         at oracle.jdbc.driver.T4CPreparedStatement.execute_maybe_describe(T4CPreparedStatement.java:452)
         at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:984)
         at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:2885)
         at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:2926)
         - locked <762d0bd8> (a oracle.jdbc.driver.T4CPreparedStatement)
         - locked <6c837e90> (a oracle.jdbc.driver.T4CConnection)
         at weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:92)
         at com.huawei.oss.rms.persistence.ReadOperationPersistenceImpl.getAttributeValue(ReadOperationPersistenceImpl.java:2054)
         at com.huawei.oss.rms.persistence.PersistenceOperationImpl.getAttributeValue(PersistenceOperationImpl.java:516)
         at com.huawei.oss.services.common.Utility.getNameByID(Utility.java:78)
         at com.huawei.oss.services.port.common.RMSPortUtility.setExtraToVO(RMSPortUtility.java:45)
         at com.huawei.oss.services.port.server.PhyPortConnHelper.setExtraToVO(PhyPortConnHelper.java:2046)
         at com.huawei.oss.services.port.server.PhyPortConnHelper.afterGet(PhyPortConnHelper.java:719)
         at com.huawei.oss.services.common.AbstractResourceHelper.getResource(AbstractResourceHelper.java:293)
         at com.huawei.oss.services.port.server.PhyPortConnHelper.rmsGetByPort(PhyPortConnHelper.java:809)
         at com.huawei.oss.services.port.server.PhyPortConnHelper.rmsGetAllPhyConnOfBoard(PhyPortConnHelper.java:1144)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:324)
         at com.huawei.oss.services.common.AbstractResourceHelper.directMethod(AbstractResourceHelper.java:489)
         at com.huawei.oss.services.common.AbstractResourceHelper.invokeMethod(AbstractResourceHelper.java:404)
         at com.huawei.oss.rms.resourceaccess.ResourceMgmtSessionEJB.invokeMethod(ResourceMgmtSessionEJB.java:263)
         at com.huawei.oss.rms.resourceaccess.ResourceMgmtSession_1x9h7n_EOImpl.invokeMethod(ResourceMgmtSession_1x9h7n_EOImpl.java:424)
         at com.huawei.oss.rms.resourceaccess.ResourceMgmtSession_1x9h7n_EOImpl_WLSkel.invoke(Unknown Source)
         at weblogic.rmi.internal.activation.ActivatableServerRef.invoke(ActivatableServerRef.java:90)
         at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
         at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
         at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:144)
         at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
         at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
    "ExecuteThread: '13' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=00088290 nid=26 lwp_id=39204 runnable [0x6316d000..0x6316c4f0]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at oracle.net.ns.Packet.receive(Unknown Source)
         at oracle.net.ns.DataPacket.receive(Unknown Source)
         at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
         at oracle.net.ns.NetInputStream.read(Unknown Source)
         at oracle.net.ns.NetInputStream.read(Unknown Source)
         at oracle.net.ns.NetInputStream.read(Unknown Source)
         at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
         at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
         at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
         at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:112)
         at oracle.jdbc.driver.T4CStatement.execute_for_rows(T4CStatement.java:474)
         at oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:913)
         at oracle.jdbc.driver.T4CStatement.execute_maybe_describe(T4CStatement.java:383)
         at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:984)
         at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1124)
         - locked <76c0d1f8> (a oracle.jdbc.driver.T4CStatement)
         - locked <6f1ab788> (a oracle.jdbc.driver.T4CConnection)
         at weblogic.jdbc.wrapper.Statement.executeQuery(Statement.java:345)
         at com.huawei.oss.rms.persistence.PersistenceOperationImpl.executeSQL(PersistenceOperationImpl.java:2884)
         at com.huawei.oss.rms.persistence.PersistenceOperationImpl.setVersionFlagNResourceName(PersistenceOperationImpl.java:3137)
         at com.huawei.oss.rms.persistence.PersistenceOperationImpl.makeAssociationValue(PersistenceOperationImpl.java:2600)
         at com.huawei.oss.rms.persistence.ReadOperationPersistenceImpl.getResourceAssociationSet(ReadOperationPersistenceImpl.java:2575)
         at com.huawei.oss.rms.persistence.PersistenceOperationImpl.getResourceAssociationSet(PersistenceOperationImpl.java:603)
         at com.huawei.oss.services.port.common.RMSPortUtility.getSlotByEquip(RMSPortUtility.jÿ

  • How to get thread dump in case of out of memory error.

    Hi Team,
    In case of out of memory, heap dump can be generated using option '- XX:+HeapDumpOnOutOfMemoryError'. Is is possible to generate thread dump too using any option? Can it be done using option -XX:OnOutOfMemoryError="<cmd args>?
    Thanks in advance.
    Regards,
    Atul.

    Jay SenSharma wrote:
    Hi Mukul,
    Complete Steps can be Devided into 3-parts
    A). Configuring a Mail-Session
    B). Configuring WLDF Module
    C). Configuring Watch And Notification
    For detailed <A class=bodylinkwhite href="http://www.software-to-convert.com/divx-avi-conversion-software/divx-avi-to-h264-software.html"><FONT face=tahoma,verdana,sans-serif color=#000 size=1>steps</FONT></A> please refer to: <B><A class=jive-link-external href="http://jaysensharma.wordpress.com/2010/01/07/e-mail-notification-using-wldf/">http://jaysensharma.wordpress.com/2010/01/07/e-mail-notification-using-wldf/</A></B>
    <P>
    Thanks
    Jay SenSharma
    <B><A class=jive-link-external href="http://jaysensharma.wordpress.com">http://jaysensharma.wordpress.com</A> (WebLogic Wonders Are here)</B></P>
    It is just the solution for my problem, It's very useful.

  • What's a good way to do a thread dump into a separate file

    What is a good way to do a thread dump automatically into a separate file.
    Example. I run a script to do the thread dump, but unfortunetly, it goes into my stdout log file with the rest of my weblogic errors.
    Any ideas? I want it in a separate file when I run my script?

    Do a Google search on "Drobo S" "benchmark."  I don't have a Drobo S, only the regular Drobo.  But here's a guy who tested one on Windows:
    http://mansurovs.com/drobo-s-review-usb-3-0-2nd-generation
    This one has it a bit faster:
    http://the-gadgeteer.com/2011/12/31/drobo-s-storage-array-review/
    Do read up on a few reviews of it, and be absolutely clear that interface speed (i.e. eSATA versus Firewire versus Thunderbolt) is NOT the same as the performance of the system.  The Drobo cannot keep up with any interfaces... at least the Drobo and the Drobo S cannot.
    I am not using the FS model which is a NAS.  I am using the plain old "Drobo" which is slower than the Drobo S, but that's not to say that the Drobo S is fast, because it is not.
    The Drobo in theory is really attractive: Dead simple to manage, can mix and match drive sizes, offers you some data protection, etc.  However do note that protected storage is not, in and of itself, a backup.  You need other backups besides just the data on the Drobo.  And, because it's so slow, it's really not a great fit for photo storage.  See this review from a guy who used to think the Drobo was great for that and then appended his review:
    http://www.stuckincustoms.com/drobo-review/
    To be as clear as possible, IMO the BEST backup strategy with something like Aperture (so long as your managed Aperture library is of a manageable size, like < 800 GB), is to get a few small portable Firewire 800 drives and keep vaults on each one.  They are great because they are easy to use, to have with you, are bus powered, and you WILL offsite them.

  • Weblogic sp4 on linux out of memory expections - Thread dump

    Any body has idea why out of memory expections?
    Here is the thread dump.
    Full thread dump Java HotSpot(TM) Server VM (1.4.2_05-b04 mixed mode):
    "Thread-33" daemon prio=1 tid=0x086175f0 nid=0x4f8e runnable [c8d57000..c8d5723c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0xd48f7820> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.apache.axis.monitor.SOAPMonitorService$ServerSocketThread.run(SOAPMonitorService.java:189)
         at java.lang.Thread.run(Thread.java:534)
    "CustomerClientDispatcherThread" daemon prio=1 tid=0x08877100 nid=0x4f8e in Object.wait() [c77ff000..c77ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd45b7330> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
         at java.lang.Object.wait(Object.java:429)
         at com.amgen.ea.orx.queue.impl.PersistenceQueue.peek(PersistenceQueue.java:58)
         - locked <0xd45b7330> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
         at com.amgen.ea.orx.custclient.impl.CustomerClientDispatcher.run(CustomerClientDispatcher.java:50)
         at java.lang.Thread.run(Thread.java:534)
    "CustomerClientDispatcherThread" daemon prio=1 tid=0xcb625c28 nid=0x4f8e in Object.wait() [c7b8c000..c7b8c23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3ce7958> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
         at java.lang.Object.wait(Object.java:429)
         at com.amgen.ea.orx.queue.impl.PersistenceQueue.peek(PersistenceQueue.java:58)
         - locked <0xd3ce7958> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
         at com.amgen.ea.orx.custclient.impl.CustomerClientDispatcher.run(CustomerClientDispatcher.java:52)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-8" daemon prio=1 tid=0x08710e80 nid=0x4f8e in Object.wait() [c7cc4000..c7cc423c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd38b44c8> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
         at java.lang.Object.wait(Object.java:429)
         at com.amgen.ea.orx.queue.impl.PersistenceQueue.peek(PersistenceQueue.java:58)
         - locked <0xd38b44c8> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
         at com.amgen.ea.orx.custclient.impl.CustomerClientDispatcher.run(CustomerClientDispatcher.java:52)
         at java.lang.Thread.run(Thread.java:534)
    "ListenThread.Default" prio=1 tid=0xcc21b658 nid=0x4f8e runnable [c7c0c000..c7c0c23c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0xd3811cf8> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:26)
         at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:735)
         at weblogic.t3.srvr.ListenThread.run(ListenThread.java:301)
    "Thread-6" prio=1 tid=0xcb3ef610 nid=0x4f8e in Object.wait() [c92a0000..c92a023c]
         at java.lang.Object.wait(Native Method)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0xd348db48> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "Thread-5" prio=1 tid=0x08c46fd8 nid=0x4f8e in Object.wait() [c9500000..c950023c]
         at java.lang.Object.wait(Native Method)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0xd34901c8> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "LDAPConnThread-0 ldap://0.0.0.0:7013" daemon prio=1 tid=0xcb67b150 nid=0x4f8e runnable [c9580000..c958023c]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
         - locked <0xd3490a98> (a java.io.BufferedInputStream)
         at netscape.ldap.ber.stream.BERElement.getElement(BERElement.java:101)
         at netscape.ldap.LDAPConnThread.run(LDAPConnThread.java:538)
         at java.lang.Thread.run(Thread.java:534)
    "VDE Transaction Processor Thread" prio=1 tid=0xcb2fe110 nid=0x4f8e in Object.wait() [c9600000..c960023c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3490b30> (a com.octetstring.vde.backend.standard.TransactionProcessor)
         at java.lang.Object.wait(Object.java:429)
         at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:365)
         - locked <0xd3490b30> (a com.octetstring.vde.backend.standard.TransactionProcessor)
         at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212)
    "ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" daemon prio=1 tid=0x08665ba8 nid=0x4f8e in Object.wait() [c9680000..c968023c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3490ba8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" daemon prio=1 tid=0x0830a7e0 nid=0x4f8e in Object.wait() [c9700000..c970023c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3490c28> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" daemon prio=1 tid=0x080a0c00 nid=0x4f8e in Object.wait() [c9780000..c978023c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3490ca8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon prio=1 tid=0xcafb3be8 nid=0x4f8e runnable [c98ff000..c98ff23c]
         at weblogic.socket.PosixSocketMuxer.poll(Native Method)
         at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:100)
         - locked <0xd31100e0> (a java.lang.String)
         at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
    "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon prio=1 tid=0xcaf65228 nid=0x4f8e waiting for monitor entry [c997f000..c997f23c]
         at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)
         - waiting to lock <0xd31100e0> (a java.lang.String)
         at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
    "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon prio=1 tid=0xcafb0c20 nid=0x4f8e waiting for monitor entry [c99ff000..c99ff23c]
         at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)
         - waiting to lock <0xd31100e0> (a java.lang.String)
         at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
    "weblogic.security.SpinnerRandomSource" daemon prio=1 tid=0xcaf64f98 nid=0x4f8e in Object.wait() [c9a7f000..c9a7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31182d8> (a java.lang.Object)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)
         - locked <0xd31182d8> (a java.lang.Object)
         at java.lang.Thread.run(Thread.java:534)
    "weblogic.time.TimeEventGenerator" daemon prio=1 tid=0xcb3e1400 nid=0x4f8e in Object.wait() [c9aff000..c9aff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118348> (a weblogic.time.common.internal.TimeTable)
         at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:272)
         - locked <0xd3118348> (a weblogic.time.common.internal.TimeTable)
         at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:118)
         at java.lang.Thread.run(Thread.java:534)
    "ExecuteThread: '4' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb3df3a8 nid=0x4f8e in Object.wait() [c9b7f000..c9b7f23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31183b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '3' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb3de7a0 nid=0x4f8e in Object.wait() [c9bff000..c9bff23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118438> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '2' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb3ddcc0 nid=0x4f8e in Object.wait() [c9c7f000..c9c7f23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31184b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '1' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb1c66f0 nid=0x4f8e in Object.wait() [c9cff000..c9cff23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118538> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '0' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb1c5b58 nid=0x4f8e in Object.wait() [c9d7f000..c9d7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31185b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31185b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '39' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c4f50 nid=0x4f8e in Object.wait() [c9dff000..c9dff23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118638> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '38' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c4348 nid=0x4f8e in Object.wait() [c9e7f000..c9e7f23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31186b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '37' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c3740 nid=0x4f8e in Object.wait() [c9eff000..c9eff23c]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118738> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '36' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c2ba8 nid=0x4f8e in Object.wait() [c9f7f000..c9f7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31187b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31187b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '35' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb674078 nid=0x4f8e in Object.wait() [c9fff000..c9fff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118838> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118838> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '34' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb673470 nid=0x4f8e in Object.wait() [ca07f000..ca07f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31188b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31188b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '33' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb6728d8 nid=0x4f8e in Object.wait() [ca0ff000..ca0ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118938> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118938> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '32' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb671cd0 nid=0x4f8e in Object.wait() [ca17f000..ca17f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31189b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31189b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '31' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb6710e0 nid=0x4f8e in Object.wait() [ca1ff000..ca1ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118a38> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118a38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '30' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfdb548 nid=0x4f8e in Object.wait() [ca27f000..ca27f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118ab8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118ab8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '29' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfda940 nid=0x4f8e in Object.wait() [ca2ff000..ca2ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118b38> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118b38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '28' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfd9d58 nid=0x4f8e in Object.wait() [ca37f000..ca37f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118bb8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118bb8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '27' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfd91c0 nid=0x4f8e in Object.wait() [ca3ff000..ca3ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118c38> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118c38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '26' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb119a40 nid=0x4f8e in Object.wait() [ca47f000..ca47f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118cb8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118cb8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '25' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb118e38 nid=0x4f8e in Object.wait() [ca4ff000..ca4ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118d38> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118d38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '24' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb118230 nid=0x4f8e in Object.wait() [ca57f000..ca57f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118db8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118db8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '23' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcc207638 nid=0x4f8e in Object.wait() [ca5ff000..ca5ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118e38> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118e38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '22' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcc206a30 nid=0x4f8e in Object.wait() [ca67f000..ca67f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118eb8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118eb8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcc205e28 nid=0x4f8e in Object.wait() [ca6ff000..ca6ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118f38> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118f38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '20' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfbf1e0 nid=0x4f8e in Object.wait() [ca77f000..ca77f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3118fb8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3118fb8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '19' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfbe648 nid=0x4f8e in Object.wait() [ca7ff000..ca7ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119038> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119038> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '18' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfbda40 nid=0x4f8e in Object.wait() [ca87f000..ca87f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31190b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31190b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '17' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb62c500 nid=0x4f8e in Object.wait() [ca8ff000..ca8ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119138> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119138> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '16' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb62b8f8 nid=0x4f8e in Object.wait() [ca97f000..ca97f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31191b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31191b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '15' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb62acf0 nid=0x4f8e in Object.wait() [ca9ff000..ca9ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119238> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119238> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb61f5a8 nid=0x4f8e in Object.wait() [caa7f000..caa7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31192b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31192b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '13' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb61e9e0 nid=0x4f8e in Object.wait() [caaff000..caaff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119338> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119338> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '12' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf13af0 nid=0x4f8e in Object.wait() [cab7f000..cab7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31193b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31193b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '11' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf12ee8 nid=0x4f8e in Object.wait() [cabff000..cabff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119438> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119438> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '10' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf12350 nid=0x4f8e in Object.wait() [cac7f000..cac7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31194b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31194b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '9' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb309598 nid=0x4f8e in Object.wait() [cacff000..cacff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119538> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119538> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '8' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb3089c8 nid=0x4f8e in Object.wait() [cad7f000..cad7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31195b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31195b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '7' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf56250 nid=0x4f8e in Object.wait() [cadff000..cadff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119638> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119638> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '6' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55cb0 nid=0x4f8e in Object.wait() [cae7f000..cae7f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31196b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31196b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '5' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55918 nid=0x4f8e in Object.wait() [caeff000..caeff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119738> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119738> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '4' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55510 nid=0x4f8e in Object.wait() [cb47f000..cb47f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31197b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31197b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55150 nid=0x4f8e in Object.wait() [cb4ff000..cb4ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119838> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119838> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '2' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfc7bd8 nid=0x4f8e in Object.wait() [cb57f000..cb57f23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31198b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31198b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '1' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb3cd250 nid=0x4f8e in Object.wait() [cb5ff000..cb5ff23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119938> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd3119938> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '0' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb3cd0b8 nid=0x4f8e in Object.wait() [cc1fe000..cc1fe23c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd31199b8> (a weblogic.kernel.ExecuteThread)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xd31199b8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "Thread-1" daemon prio=1 tid=0x086d6478 nid=0x4f8e in Object.wait() [cc3e1000..cc3e123c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xd3119a88> (a java.util.TaskQueue)
         at java.lang.Object.wait(Object.java:429)
         at java.util.TimerThread.mainLoop(Timer.java:403)
         - locked <0xd3119a88> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "Signal Dispatcher" daemon prio=1 tid=0x080c4170 nid=0x4f8e waiting on condition [0..0]
    "Finalizer" daemon prio=1 tid=0x080bfac0 nid=0x4f8e in Object.wait() [cf509000..cf50923c]
         at java.lang.Object.wait(Native Method)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
         - locked <0xd3129c78> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
         at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
    "Refe

    <p>Can you post the stack trace. I assume these thread dumps were taken after the error occured. Can you take a few dumps just before you run out of memory.</p>
    <p>
    Hussein Badakhchani</br>
    </p>

  • Full thread dump Java HotSpot(TM) Server VM (1.4.2_05-b04 mixed mode):

    Hi All,
    I am getting below error message when i am starting JBoss
    Anyone has any idea, why it is happening?
    Full thread dump Java HotSpot(TM) Server VM (1.4.2_05-b04 mixed mode):
    "RMI ConnectionExpiration-[127.0.0.1:33030]" daemon prio=1 tid=0x2a375e90 nid=0xcf9 waiting on condition [2820d000..2820d86c]
         at java.lang.Thread.sleep(Native Method)
         at sun.rmi.transport.tcp.TCPChannel$Reaper.run(TCPChannel.java:447)
         at java.lang.Thread.run(Thread.java:534)
    "RMI TCP Connection(1681)-127.0.0.1" daemon prio=1 tid=0x2cc3e008 nid=0xcf9 runnable [27b7b000..27b7b86c]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
         - locked <0x2ed63640> (a java.io.BufferedInputStream)
         at java.io.FilterInputStream.read(FilterInputStream.java:66)
         at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:442)
         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:701)
         at java.lang.Thread.run(Thread.java:534)
    "RMI RenewClean-[127.0.0.1:33030]" daemon prio=1 tid=0x2ca76ad0 nid=0xcf9 in Object.wait() [29dcd000..29dcd86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x2ed61c50> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
         - locked <0x2ed61c50> (a java.lang.ref.ReferenceQueue$Lock)
         at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:500)
         at java.lang.Thread.run(Thread.java:534)
    "JNP Server" prio=1 tid=0x082dd9c0 nid=0xcf9 runnable [27afa000..27afa86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d145770> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jnp.server.Main.run(Main.java:257)
         at java.lang.Thread.run(Thread.java:534)
    "RMI TCP Connection(1680)-127.0.0.1" daemon prio=1 tid=0x2ca55d80 nid=0xcf9 runnable [27f88000..27f8886c]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
         - locked <0x2ecd1e40> (a java.io.BufferedInputStream)
         at java.io.FilterInputStream.read(FilterInputStream.java:66)
         at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:442)
         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:701)
         at java.lang.Thread.run(Thread.java:534)
    "RMI ConnectionExpiration-[127.0.0.1:33030]" daemon prio=1 tid=0x2df71458 nid=0xcf9 waiting on condition [29e4e000..29e4e86c]
         at java.lang.Thread.sleep(Native Method)
         at sun.rmi.transport.tcp.TCPChannel$Reaper.run(TCPChannel.java:447)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-36" daemon prio=1 tid=0x08541f10 nid=0xcf9 runnable [279f8000..279f886c]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
         at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:277)
         - locked <0x3d8dbc38> (a java.io.BufferedInputStream)
         at com.sun.jndi.ldap.Connection.run(Connection.java:780)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-35" daemon prio=1 tid=0x0811fd00 nid=0xcf9 runnable [27a79000..27a7986c]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
         at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:277)
         - locked <0x3d8d7d50> (a java.io.BufferedInputStream)
         at com.sun.jndi.ldap.Connection.run(Connection.java:780)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-34" daemon prio=1 tid=0x08bc3588 nid=0xcf9 in Object.wait() [27e86000..27e8686c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d8d6050> (a org.jboss.mx.util.ThreadPool$Worker)
         at java.lang.Object.wait(Object.java:429)
         at org.jboss.mx.util.ThreadPool$Worker.run(ThreadPool.java:243)
         - locked <0x3d8d6050> (a org.jboss.mx.util.ThreadPool$Worker)
    "Thread-33" daemon prio=1 tid=0x08bc47b8 nid=0xcf9 in Object.wait() [27f07000..27f0786c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d8d4780> (a org.jboss.mx.util.ThreadPool$Worker)
         at java.lang.Object.wait(Object.java:429)
         at org.jboss.mx.util.ThreadPool$Worker.run(ThreadPool.java:243)
         - locked <0x3d8d4780> (a org.jboss.mx.util.ThreadPool$Worker)
    "Thread-32" daemon prio=1 tid=0x08407010 nid=0xcf9 in Object.wait() [2d2af000..2d2af86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d895fa8> (a org.jboss.mx.util.ThreadPool$Worker)
         at java.lang.Object.wait(Object.java:429)
         at org.jboss.mx.util.ThreadPool$Worker.run(ThreadPool.java:243)
         - locked <0x3d895fa8> (a org.jboss.mx.util.ThreadPool$Worker)
    "Thread-31" prio=1 tid=0x084065d0 nid=0xcf9 in Object.wait() [27bfc000..27bfc86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d895bd8> (a java.lang.Object)
         at java.lang.Object.wait(Object.java:429)
         at org.jboss.system.server.ServerImpl$LifeThread.run(ServerImpl.java:731)
         - locked <0x3d895bd8> (a java.lang.Object)
    "Thread-30" prio=1 tid=0x082ef440 nid=0xcf9 in Object.wait() [27c7d000..27c7d86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d852290> (a org.jboss.mx.util.RunnableScheduler)
         at org.jboss.mx.util.RunnableScheduler.waitOutstanding(RunnableScheduler.java:188)
         - locked <0x3d852290> (a org.jboss.mx.util.RunnableScheduler)
         at org.jboss.mx.util.RunnableScheduler.run(RunnableScheduler.java:93)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-29" prio=1 tid=0x085357c0 nid=0xcf9 in Object.wait() [27cfe000..27cfe86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d8502e8> (a org.jboss.mx.util.RunnableScheduler)
         at org.jboss.mx.util.RunnableScheduler.waitOutstanding(RunnableScheduler.java:188)
         - locked <0x3d8502e8> (a org.jboss.mx.util.RunnableScheduler)
         at org.jboss.mx.util.RunnableScheduler.run(RunnableScheduler.java:93)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-28" prio=1 tid=0x08fc8d58 nid=0xcf9 in Object.wait() [27d7f000..27d7f86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d84d6b0> (a org.jboss.mx.util.RunnableScheduler)
         at org.jboss.mx.util.RunnableScheduler.waitOutstanding(RunnableScheduler.java:188)
         - locked <0x3d84d6b0> (a org.jboss.mx.util.RunnableScheduler)
         at org.jboss.mx.util.RunnableScheduler.run(RunnableScheduler.java:93)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-27" daemon prio=1 tid=0x08a3c5f0 nid=0xcf9 runnable [27e05000..27e0586c]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
         at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:277)
         - locked <0x3d8494b8> (a java.io.BufferedInputStream)
         at com.sun.jndi.ldap.Connection.run(Connection.java:780)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-26" daemon prio=1 tid=0x08829d80 nid=0xcf9 in Object.wait() [2bf3a000..2bf3a86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d83f190> (a java.util.TaskQueue)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0x3d83f190> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "RMI LeaseChecker" daemon prio=1 tid=0x08d475b0 nid=0xcf9 waiting on condition [29bff000..29bff86c]
         at java.lang.Thread.sleep(Native Method)
         at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:307)
         at java.lang.Thread.run(Thread.java:534)
    "SessionScavenger" daemon prio=1 tid=0x08b8c420 nid=0xcf9 waiting on condition [2a1ff000..2a1ff86c]
         at java.lang.Thread.sleep(Native Method)
         at org.mortbay.jetty.servlet.AbstractSessionManager$SessionScavenger.run(AbstractSessionManager.java:428)
    "SessionScavenger" daemon prio=1 tid=0x082f2760 nid=0xcf9 waiting on condition [2ab72000..2ab7286c]
         at java.lang.Thread.sleep(Native Method)
         at org.mortbay.jetty.servlet.AbstractSessionManager$SessionScavenger.run(AbstractSessionManager.java:428)
    "UILServerILService Accept Thread" prio=1 tid=0x2cc3e9c0 nid=0xcf9 runnable [2abfb000..2abfb86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d63d830> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jboss.mq.il.uil2.UILServerILService.run(UILServerILService.java:118)
         at java.lang.Thread.run(Thread.java:534)
    "OIL2 Worker Server" prio=1 tid=0x2dfe0df0 nid=0xcf9 runnable [2ac7c000..2ac7c86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d63d958> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jboss.mq.il.oil2.OIL2ServerILService.run(OIL2ServerILService.java:366)
         at java.lang.Thread.run(Thread.java:534)
    "UIL Worker" prio=1 tid=0x2ccfca90 nid=0xcf9 runnable [2acfd000..2acfd86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d63da18> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jboss.mq.il.uil.UILServerILService.run(UILServerILService.java:184)
         at java.lang.Thread.run(Thread.java:534)
    "OIL Worker Server" prio=1 tid=0x2bab0a70 nid=0xcf9 runnable [2ad7e000..2ad7e86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d63dad8> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jboss.mq.il.oil.OILServerILService.run(OILServerILService.java:471)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-25" prio=1 tid=0x2cc9ef88 nid=0xcf9 in Object.wait() [2adff000..2adff86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d63db88> (a java.util.LinkedList)
         at java.lang.Object.wait(Object.java:429)
         at org.jboss.resource.connectionmanager.PoolFiller$1.run(PoolFiller.java:75)
         - locked <0x3d63db88> (a java.util.LinkedList)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-24" prio=1 tid=0x2dfe0268 nid=0xcf9 in Object.wait() [2afc4000..2afc486c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d52f320> (a java.util.ArrayList)
         at org.jboss.resource.connectionmanager.IdleRemover$1.run(IdleRemover.java:66)
         - locked <0x3d52f320> (a java.util.ArrayList)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-23" daemon prio=1 tid=0x2dfce680 nid=0xcf9 in Object.wait() [2b045000..2b04586c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d52f390> (a org.jboss.util.timeout.TimeoutFactory)
         at org.jboss.util.timeout.TimeoutFactory.doWork(TimeoutFactory.java:503)
         - locked <0x3d52f390> (a org.jboss.util.timeout.TimeoutFactory)
         at org.jboss.util.timeout.TimeoutFactory.access$300(TimeoutFactory.java:27)
         at org.jboss.util.timeout.TimeoutFactory$1.run(TimeoutFactory.java:542)
    "Thread-22" prio=1 tid=0x2ccef150 nid=0xcf9 runnable [2b0c6000..2b0c686c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d52f400> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jboss.invocation.pooled.server.PooledInvoker.run(PooledInvoker.java:216)
         at java.lang.Thread.run(Thread.java:534)
    "JBossMQ Cache Reference Softner" daemon prio=1 tid=0x2dfe1a28 nid=0xcf9 in Object.wait() [2b147000..2b14786c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d411a00> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
         - locked <0x3d411a00> (a java.lang.ref.ReferenceQueue$Lock)
         at org.jboss.mq.server.MessageCache.run(MessageCache.java:233)
         at java.lang.Thread.run(Thread.java:534)
    "RMI TCP Accept-0" daemon prio=1 tid=0x2c388ff0 nid=0xcf9 runnable [2b1ef000..2b1ef86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d2c0f90> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
         at java.lang.Thread.run(Thread.java:534)
    "SessionScavenger" daemon prio=1 tid=0x2c3864e0 nid=0xcf9 waiting on condition [2b270000..2b27086c]
         at java.lang.Thread.sleep(Native Method)
         at org.mortbay.jetty.servlet.AbstractSessionManager$SessionScavenger.run(AbstractSessionManager.java:428)
    "PoolThread-4" prio=1 tid=0x2c384e10 nid=0xcf9 in Object.wait() [2b2f1000..2b2f186c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c10c8> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c10c8> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-3" prio=1 tid=0x2ba1d4b0 nid=0xcf9 in Object.wait() [2b372000..2b37286c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1120> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1120> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-2" prio=1 tid=0x2ba1c960 nid=0xcf9 in Object.wait() [2b3f3000..2b3f386c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1178> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1178> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-1" prio=1 tid=0x2ba1be10 nid=0xcf9 in Object.wait() [2b474000..2b47486c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c11d0> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c11d0> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-0" prio=1 tid=0x2ba1b2c0 nid=0xcf9 in Object.wait() [2b4f5000..2b4f586c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1228> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1228> (a org.mortbay.util.ThreadPool$PoolThread)
    "Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=8009]" prio=1 tid=0x2ba1af70 nid=0xcf9 runnable [2b576000..2b57686c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d2c12d8> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.mortbay.util.ThreadedServer.acceptSocket(ThreadedServer.java:346)
         at org.mortbay.util.ThreadedServer$Acceptor.run(ThreadedServer.java:507)
    "PoolThread-9" prio=1 tid=0x2ba1a420 nid=0xcf9 in Object.wait() [2b5f7000..2b5f786c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c13a8> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c13a8> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-8" prio=1 tid=0x2ba198d0 nid=0xcf9 in Object.wait() [2b678000..2b67886c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1400> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1400> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-7" prio=1 tid=0x2c3a4ad8 nid=0xcf9 in Object.wait() [2b6f9000..2b6f986c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1458> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1458> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-6" prio=1 tid=0x2c3a4788 nid=0xcf9 in Object.wait() [2b77a000..2b77a86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c14b0> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c14b0> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-5" prio=1 tid=0x2c3a3c58 nid=0xcf9 in Object.wait() [2b7fb000..2b7fb86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1508> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1508> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-4" prio=1 tid=0x2ba8bfe8 nid=0xcf9 in Object.wait() [2b87c000..2b87c86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1560> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1560> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-3" prio=1 tid=0x2ba78040 nid=0xcf9 in Object.wait() [2b8fd000..2b8fd86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c15b8> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c15b8> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-2" prio=1 tid=0x2ba69c18 nid=0xcf9 in Object.wait() [2b97e000..2b97e86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1610> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1610> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-1" prio=1 tid=0x2ba69a08 nid=0xcf9 in Object.wait() [2b9ff000..2b9ff86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c1668> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c1668> (a org.mortbay.util.ThreadPool$PoolThread)
    "PoolThread-0" prio=1 tid=0x2ba7ebc8 nid=0xcf9 in Object.wait() [2bb8e000..2bb8e86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d2c16c0> (a org.mortbay.util.ThreadPool$PoolThread)
         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:450)
         - locked <0x3d2c16c0> (a org.mortbay.util.ThreadPool$PoolThread)
    "Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=8020]" prio=1 tid=0x2ba7ea18 nid=0xcf9 runnable [2bc0f000..2bc0f86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d2c1770> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.mortbay.util.ThreadedServer.acceptSocket(ThreadedServer.java:346)
         at org.mortbay.util.ThreadedServer$Acceptor.run(ThreadedServer.java:507)
    "SessionScavenger" daemon prio=1 tid=0x2ba8bb18 nid=0xcf9 waiting on condition [2bd90000..2bd9086c]
         at java.lang.Thread.sleep(Native Method)
         at org.mortbay.jetty.servlet.AbstractSessionManager$SessionScavenger.run(AbstractSessionManager.java:428)
    "Thread-4" daemon prio=1 tid=0x2ba8c6f0 nid=0xcf9 in Object.wait() [2beb9000..2beb986c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d23d438> (a java.util.TaskQueue)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0x3d23d438> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "GC Daemon" daemon prio=1 tid=0x2c32cd20 nid=0xcf9 in Object.wait() [2bfbb000..2bfbb86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d0b4b28> (a sun.misc.GC$LatencyLock)
         at sun.misc.GC$Daemon.run(GC.java:100)
         - locked <0x3d0b4b28> (a sun.misc.GC$LatencyLock)
    "RMI Reaper" prio=1 tid=0x2c325c88 nid=0xcf9 in Object.wait() [2c03c000..2c03c86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d0b4078> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
         - locked <0x3d0b4078> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
         at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:330)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-3" daemon prio=1 tid=0x2c33be48 nid=0xcf9 in Object.wait() [2c0bd000..2c0bd86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d0b4378> (a java.util.TaskQueue)
         at java.lang.Object.wait(Object.java:429)
         at java.util.TimerThread.mainLoop(Timer.java:403)
         - locked <0x3d0b4378> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "RMI TCP Accept-0" daemon prio=1 tid=0x2c33b8e0 nid=0xcf9 runnable [2c13e000..2c13e86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d0b40d8> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-2" daemon prio=1 tid=0x2c340060 nid=0xcf9 runnable [2c1bf000..2c1bf86c]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0x3d0b1730> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at org.jboss.web.WebServer.run(WebServer.java:258)
         at org.jboss.web.ThreadPool$Worker.run(ThreadPool.java:163)
    "ScannerThread" daemon prio=1 tid=0x2c3403d8 nid=0xcf9 waiting on condition [2cdc7000..2cdc786c]
         at java.lang.Thread.sleep(Native Method)
         at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:217)
         at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:190)
    "Thread-0" daemon prio=1 tid=0x2c327b00 nid=0xcf9 in Object.wait() [2c240000..2c24086c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d05d2d0> (a java.util.TaskQueue)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0x3d05d2d0> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "DestroyJavaVM" prio=1 tid=0x2df28218 nid=0xcf9 waiting on condition [0..bfffccc4]
    "Signal Dispatcher" daemon prio=1 tid=0x080c33f8 nid=0xcf9 waiting on condition [0..0]
    "Finalizer" daemon prio=1 tid=0x080bed60 nid=0xcf9 in Object.wait() [2e3fc000..2e3fc86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d0b5930> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
         - locked <0x3d0b5930> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
         at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
    "Reference Handler" daemon prio=1 tid=0x080be9d0 nid=0xcf9 in Object.wait() [2e47d000..2e47d86c]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x3d0b52f8> (a java.lang.ref.Reference$Lock)
         at java.lang.Object.wait(Object.java:429)
         at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:115)
         - locked <0x3d0b52f8> (a java.lang.ref.Reference$Lock)

    have you checked bugs.sun.com to see if this has been reported?
    And if not, file one and attach your dump for them to look into it.

  • SIGQUIT ignored / no thread dump in 1.5/5.0 ?

    Trying out 1.5 on a dual-xeon, Linux 2.4.27 machine. - 'java -version' returns:
    java version "1.5.0"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0-b64)
    Java HotSpot(TM) Client VM (build 1.5.0-b64, mixed mode, sharing)
    Java process seems to ignore 'kill -SIGQUIT' -- nothing to stderr/stdout, nothing to working directory. A thread dump would be really handy. No special options were used to launch JVM.
    Any ideas?

    I'm having a problem with the same symptom: SIGQUIT is ignored. So I ran jstack and got this:
    Error occurred during stack walking:
    sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:134)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:437)
    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:48)
    at sun.jvm.hotspot.runtime.linux_x86.LinuxX86JavaThreadPDAccess.getCurrentFrameGuess(LinuxX86JavaThreadPDAccess.java:75)
    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:252)
    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:211)
    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:42)
    at sun.jvm.hotspot.tools.JStack.run(JStack.java:41)
    at sun.jvm.hotspot.tools.Tool.start(Tool.java:204)
    at sun.jvm.hotspot.tools.JStack.main(JStack.java:58)
    Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native Method)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:34)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:431)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:109)
    Any ideas/suggestions? Anyone seen this before?

  • 100% CPU load but no clue in thread dump, EP. Win a bottle of champagne...

    Hi,
    We are suddenly facing 100% CPU load in our EP cluster (5 x 8-way Xeon multiprocessor machines). We have a serious performance problem that is burning a lot of our time and causes a lot of stress for over 3 weeks now.
    We have taken tens of thread dumps from Application Nodes.
    - In none of the thread dumps we see the Finalizer thread
    running.
    - From the garbage collector log we see that full garbage
    collection runs occur very rarily (once an hour).
    - We use the compacting garbage collector.
    - In a lot of thread dumps we do not even see our own portal code in the stack traces of the threads!
    - In Windows Task Manager we see the jlaunch.exe processes
    consume all available CPU time.
    - Portal users see a blank portal page when the CPU load hits 100%. When the load goes down again, things return to normal.
    - There are no errors logged in any of the log files of the portal. We checked all of them. We expected a log full of errors somewhere but nothing even remotely interesting was found. Windows Event Viewer shows nothing either.
    - The amount of sockets in CLOSE_WAIT status is < 10 on every machine in the cluster.
    This forces me to conclude that something in the jlaunch.exe executable consumes the CPU time. This raises the following 3 questions:
    - What does the mystifying jlaunch.exe do besides executing java.exe ?
    - Why is the Java virtual machine launched by a custom executable like jlaunch (What is it that cannot be program med in Java) ? Can it be GZip compression ?
    - If the problem is not caused by jlaunch.exe, then it must be caused by the JVM. What activity, invisible in thread dumps, is performed by the JVM that can cause the high CPU load ?
    Our development- and support teams are desperate. All suggestions are welcome. The person that comes up with
    the solution to our problem gets a nice bottle of champagne.
    Regards,
    Chris Twigt

    Hi,
    - What does the mystifying jlaunch.exe do besides executing java.exe ?
    This is so that the startupframework can connect more easily and take control of the JVM in some situations
    - Why is the Java virtual machine launched by a custom executable like jlaunch (What is it that cannot be program med in Java) ?
    I assume when SAP release their JVM (in the next major release), that will be called directly.
    - If the problem is not caused by jlaunch.exe, then it must be caused by the JVM. What activity, invisible in thread dumps, is performed by the JVM that can cause the high CPU load ?
    Loads of thing, but there should be clues in the thread dumps.  (the reason why your code is not in those thread dumps is that your code is only active during the processing of a particular request, unless you have a service, afterwards there are no trace of it as the thread which does the processing goes back to sleep)
    I've experienced a similar situation with a an 6.40 portal, and it was then caused by the following:
    1. User A comes logs in and sees that some cache timeout has occured , therefore it issues a SQL query which does a full table scan on a table of approx. 1 GB (on of the UME tables)
    2. User B comes in just afterwards and also sees that the cache timeout has occured, and issues the same SQL query as User A
    3. User C .... and so on untill the query from User A eventually finishes
    So
    1. Check database. Any big queries running ?
    2. Check file activity, are there a lot of writing ?
    3. Check network activity, especially to the state controller which is the weakest link
    4. What about portal logs ? Any activity during the hang ?
    Also, please provide one of the thread dumps for further analysis..
    cheers
    Dagfinn

  • WebLogic 8.1 SP4 on Linux - thread dump does not work

    I have setup WebLogic 8.1 SP4 on RedHat Linux AS 3.3 with Sun JVM (142_05). When I try to perform a thread dump using weblogic.Admin, I receive the following error in the stderr file. I have also tried the kill -3 but still no thread dump. Is there some trick to this ?
    cannot load libary 'stackdump': java.lang.UnsatisfiedLinkError: no stackdump in java.library.path

    I am getting the same error as well
    cannot load libary 'stackdump': java.lang.UnsatisfiedLinkError: no stackdump in java.library.path
    I am creating a simple domain using the domain config wizard and I hope all the PATH and LD_LIBRARY_PATH are set by the default scripts. But I am still getting this error.
    But Kill -3 <PID> works though!
    Is is a know problem OR am I missing something?!
    Bala

  • Thread dump data received from VMWare

    When Lots of trigger threads are waiting on monitor entry for the release of lock on SQLServer connection. I've pasted the snippet of the culprit
    thread:
    The thread dump is below:
    Lots of trigger threads are waiting on monitor entry for the release of
    lock on SQLServer connection. I've pasted the snippet of the culprit
    thread:
    "Thread-382" daemon prio=5 tid=0x00343e58 nid=0x1b2 runnable
    [2477f000..2477fc30]
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at
    com.wm.dd.jdbc.sqlserver.SQLServerByteOrderedDataReader.readPacketIntoPr
    imaryBuffer(Unknown Source)
    at
    com.wm.dd.jdbc.sqlserver.SQLServerByteOrderedDataReader.receive(Unknown
    Source)
    at com.wm.dd.jdbc.sqlserver.tds.TDSRequest.receive(Unknown
    Source)
    at
    com.wm.dd.jdbc.sqlserver.tds.TDSExecuteRequest.submitRequest(Unknown
    Source)
    at
    com.wm.dd.jdbc.sqlserver.SQLServerImplConnection.rollbackTransaction(Unk
    nown Source)
    at com.wm.dd.jdbc.base.BaseConnection.rollback(Unknown Source)
    - locked <0x56a3ac88> (a
    com.wm.dd.jdbc.sqlserver.SQLServerConnection)
    at com.wm.dd.jdbc.base.BaseConnection.commonCloseReset(Unknown
    Source)
    at com.wm.dd.jdbc.base.BaseConnection.reset(Unknown Source)
    - locked <0x56a3ac88> (a
    com.wm.dd.jdbc.sqlserver.SQLServerConnection)
    at com.wm.dd.jdbcx.base.BasePooledConnection.reset(Unknown
    Source)
    - locked <0x56a3ac88> (a
    com.wm.dd.jdbc.sqlserver.SQLServerConnection)
    - locked <0x56a47e10> (a
    com.wm.dd.jdbcx.base.BasePooledConnection)
    at com.wm.dd.jdbcx.base.BaseConnectionWrapper.close(Unknown
    Source)
    - locked <0x56a47e78> (a
    com.wm.dd.jdbcx.base.BaseConnectionWrapper)
    at
    com.wm.adapter.wmjdbc.connection.JDBCConnection.destroyConnection(JDBCCo
    nnection.java:277)
    at
    com.wm.adk.connection.WmManagedConnection.destroy(WmManagedConnection.ja
    va:243)
    - locked <0x56a3aae0> (a
    com.wm.adapter.wmjdbc.connection.JDBCConnection)
    at
    com.wm.app.b2b.server.jca.WmConnectionPool$ConnectionTimeoutMgr.run(WmCo
    nnectionPool.java:1048)
    - locked <0x52ff1a68> (a java.util.Vector)
    This thread locked the resource and not releasing it. It's waiting on
    the connection from sql server. So, kindly check for any connectivity
    problems between the IS and sql server and rectify the problem.
    Let me know about the nohup_p2.out. Is this a clustered srever? Here
    also I can observe that there is a problem with database connectivity

    We are get this error while useing webmethod.

  • Full thread dump/segmentation violation encountered in NAS4.0

    hi,
    the application is running on NAS4.0 and BillerXpert4. while accessing the database via a billerXpert api, we encounter a segmentation violation ==> full thread dump. this happens sporadically. below is an extract from the NAS log showing the error:-
    SIGSEGV 11* segmentation violation
    si_signo [11]: SIGSEGV 11* segmentation violation
    si_errno [0]:
    si_code [1]: SEGV_MAPERR [addr: 0x20302000]
    stackbase=DC841D64, stackpointer=DC83F42C
    Full thread dump:
    "Thread-3357" (TID:0xe17215c8, sys_thread_t:0x12040a0, state:R, thread_t: t@
    3767, sp:0xdc580db8 threadID:0xdc581dd0, stack_base:0xdc581d64, stack_size:0x200
    00) prio=5
    java.net.SocketInputStream.read(Compiled Code)
    java.io.BufferedInputStream.fill(Compiled Code)
    java.io.BufferedInputStream.read(Compiled Code)
    netscape.ldap.ber.stream.BERElement.getElement(Compiled Code)
    netscape.ldap.LDAPConnThread.run(Compiled Code)
    any pointers/advice as to what cld have caused this to happen?
    many thanks in advance!!

    Ah well if you're back on 1.2 then that's about as much error information as you will get.
    JDK 1.2 is way past end-of-life. There is nothing you can do here to fix this problem.
    Note: just because your application code was written for 1.2 doesn't mean it won't run on a later version of the VM. Try 1.3.1 if you don't want to jump up to 1.5 or 1.6 (but 1.3.1 will soon hit end-of-life too).
    Sorry.

  • Full Thread Dump...Deadlock occured in the code

    The following code generates error saying the following description:
    ERROR Start:
    SIGBUS occured.
    Full thread dump:
    "main" (TID:0x66c9360, sys_thread_t:0x7ad4f10, taskId:0x7ae447c state:runnable name:tJmain, stack_base:0x7ae447c, stack_high:0x1080 stack_size:19696) prio=5
    "Finalizer thread" (TID:0x66c9390, sys_thread_t:0x7ad0538, taskId:0x7ad4590 state:condition variable pend name:Finalizer threa, stack_base:0x7ad4588, stack_high:0x144 stack_size:15624) prio=1
    "Thread-0" (TID:0x66dd778, sys_thread_t:0x6560460, taskId:0x65644b8 state:condition variable pend name:Thread-0, stack_base:0x65644b0, stack_high:0xb48 stack_size:15628) prio=5
    Page Fault
    Program Counter: 0x0013c6a3
    Status Register: 0x00010202
    Error Code: 0x00130000
    Task: 0x6551d20 "Thread-3"
    ERROR End
    This is the code..........
    public class abc{
         private boolean ready;
         public synchronized void toggle1() {
              ready = false;                     // de-block the other thread
              try {
              notify();                     // inform the other thread
              }catch(IllegalMonitorStateException e) {
              e.printStackTrace();
              while (!ready) {               // block until the other
                   try {                    // thread sets "ready" to
                        wait();               // "true".
                   } catch (InterruptedException e) {
                   System.out.println("Error from toggle1:InterruptedException");
                   e.printStackTrace();
                   catch(Throwable e)
                   System.out.println("Error from toggle1:Throwable");
                   e.printStackTrace();
         public synchronized void toggle0() {
              ready = true;                    // de-block the other thread
              notify();                    // inform the other thread
              while (ready) {                    // block until the other
                   try {                    // thread sets "ready" to
                        wait();               // "false".
                   } catch (InterruptedException e) {}
    Would you please put look into it and suggest a solution
    Thanks
    Message was edited by:
    Indranil.Sarkar

    Ah well if you're back on 1.2 then that's about as much error information as you will get.
    JDK 1.2 is way past end-of-life. There is nothing you can do here to fix this problem.
    Note: just because your application code was written for 1.2 doesn't mean it won't run on a later version of the VM. Try 1.3.1 if you don't want to jump up to 1.5 or 1.6 (but 1.3.1 will soon hit end-of-life too).
    Sorry.

  • Can't Redirect -classic thread dump/stack trace

    Hi,
    But how can you redirect the output so you can view it easily, instead of the info just scrolling by into oblivion? I'm using Win98 and JDK1.3.1_02, and w/ the default hotspot jvm, from the console, I could redirect the stack trace/thread dump from standard out (w/c it went into) into a file.
    But w/ the -classic option (from the SDK java version, not the JRE), the stack trace/thread dump does NOT seem to sent to standard out, because I can redirect standard out, but the trace/dump is STILL going to the screen. I have redirected standard err (from w/in my java program) as well, but still no dice!
    Help! TIA,
    Reggie

    Thanks for your reply guys! I also tried redirecting std err from inside my java app. My System.err outputs WERE redirected, but NOT the thread dump - that STILL went to the screen.
    Then I tried the "java 2>error.txt myclass" but win98 thought 2 was the name of my class and threw a NoClassDefFoundError ;-).
    Any more ideas? regards,
    Reggie

  • Full thread dump Classic VM

    When I'm trying to connect with a few clients to the IAS Server the Apache-server crashes. In the log file following error is shown:
    full thread dump classic VM (JDK-1.2.2_005, native threads)
    "SocketTimeout" at java.lang.thread.sleep at HTTP Client.SocketTimeOut.run
    Does anyone know how I can fix this?
    Nathalie

    Did you ever figure this out?

  • Help needed with diagnosis of WebLogic server thread dump

     

    Sorry, my mistake - I was trying to search for "weblogic" before the dump file
    loaded fully.... I am not very familiar with VisiBroker and may be totally
    wrong, but my guess is it creates a thread for each connection/client. I would
    look for VisiBroker properties or settings that could limit the runaway creation
    of threads....
    Arunabh
    "Dimitri I. Rakitine" wrote:
    It is, preceded by lots and lots of CORBA (visibroker) threads. Is it
    possible that application code
    creates new ORBs more often than it should ?
    "Arunabh Hazarika" <[email protected]> wrote in message
    news:[email protected]...
    The attached file does not look like a WebLogic Server thread dump.....
    Arunabh
    Kapil Singh Pawar wrote:
    I am running a web application on WebLogic server 6.0 sp2 + rp1 (HP-UX).
    After
    I start loading the web application with more users I constantly get thejava.lang.OutOfMemoryError.
    I tried increasing the max_thread_proc kernel parameter to 2048 but thatdoes
    not seem to help much. Here is the error that I get on the WebLogicServer console.
    <Oct 7, 2002 5:00:27 PM GMT+02:00> <Error> <HTTP><[WebAppServletContext(1828590,i2)]
    Servlet failed with Exception
    java.lang.OutOfMemoryError: max threads limit being reached in VM
    at java.lang.Thread.start(Native Method)
    at netscape.ldap.LDAPConnThread.<init>(LDAPConnThread.java:112)
    atnetscape.ldap.LDAPConnection.getNewThread(LDAPConnection.java:1117)
    atnetscape.ldap.LDAPConnection.connect(LDAPConnection.java:1034)
    at netscape.ldap.LDAPConnection.connect(LDAPConnection.java:915)
    at netscape.ldap.LDAPConnection.connect(LDAPConnection.java:759)
    atcom.i2.scm.LDAP.ScmLDAPDefaultAuthenticate.authenticate(ScmLDAPDefaultAuthen
    ticate.java:129)
    at scmldap.service(scmldap.java:452)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
    atweblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :213)
    atweblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:1302)
    atweblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :1676)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
    >
    I am posting the thread dump obtained at the OutOfMemoryError stage. Anyhelp
    in diagnosing the thread dump and possible fix for this error is highlyappreciated.
    For information sake, while the web application encountersOutOfMemoryError the
    example web apps and WebLogic console application work fine. So it looksto be
    a case of memory outage only for my custom application.
    Thanks
    Kapil Singh Pawar
    Name: thread_dump.txt
    thread_dump.txt Type: Plain Text (text/plain)
    Encoding: base64--
    Dimitri

Maybe you are looking for