Daemons in java
Hello,
Is there any way to have a Java program detach itself from the console and run in the background as a daemon?
TheORlyFactor wrote:
Hello,
Is there any way to have a Java program detach itself from the console and run in the background as a daemon?Same way you would any other program. The fact that you wrote it in Java is irrelevant, as it's not the program that does it, but the execution environment.
Similar Messages
-
I am making a server in Java, and when I run it, it seems to hog all of the CPU time. Is there any way to optimize code so that it uses less resources?
I know one way to create daemons is to use "daemon threads." However, daemon threads die when all normal threads die (such as the main program thread). However, I want one thread to operate until explicitly closed.
The main reason I did not post this under the "Concurrency" section is because it is more about optimization in general. For example, will empty for (;;) loops tax the system more since the code will have to be re-evaluated? Is there any other way to block a thread so it doesn't die?
Help would be appreciated. A resource on Java optimization would be greatly appreciated.http://java.sun.com/docs/books/tutorial/essential/concurrency/simple.html
Actually, this is very much a concurrency question.
A for(;;) loop will tax 100% of the CPU, so you need to yield control when you have nothing to do. The page above has a nice "while thread.isAlive()" loop to give you an example.
Here is the beginning of the Concurrency tutorial. Wouldn't hurt for you to go through it, I'm sure.
http://java.sun.com/docs/books/tutorial/essential/concurrency/index.html -
Hello,
I'm working on a large scale GUI and running in to OOMs. Looking at the threads, there are hundreds of them sitting around named Java FX task daemon thread.
Daemon Thread [Java FX task daemon thread] (Running)
Daemon Thread [Java FX task daemon thread] (Suspended)
Unsafe.park(boolean, long) line: not available [native method]
LockSupport.park(Object) line: 158
AbstractQueuedSynchronizer$ConditionObject.await() line: 1987
DelayQueue<E>.take() line: 160
ScheduledThreadPoolExecutor$DelayedWorkQueue.take() line: 609
ScheduledThreadPoolExecutor$DelayedWorkQueue.take() line: 602
ScheduledThreadPoolExecutor(ThreadPoolExecutor).getTask() line: 947
ThreadPoolExecutor$Worker.run() line: 907
Thread.run() line: 662 Can anyone shed some light on what is starting these threads?I had similar issues with a server running on XP, where it kept dying randomly with no problems.
I tracked down what was happening by getting the exit value of the java process - which may give you information regarding what happened to the process.
For example, if CTRL+C was pressed you get a value of 130 returned in the Linux environment. Using this error information I found out why my server kept dying (which was the XP telnet window was sending dodgy key presses to the window the server was running in).
The exit code of a process is available after it exits via $status in Unix or %errorlevel% in a PC .bat file. -
All u java gurus -please help me with my DAEMONS
I had already posted one query regarding the same but due to some unsatisfactory answers I am forced to ask the same question.
JUST TELL ME HOW TO WRITE DAEMONS IN JAVA- REST I WILL MANAGE
heres my problem:
I have to make a java program to backup and restore encrypted files over a network.
The daemon should :
1 . The daemon has to be able to send messages to all other comps in the network that a user on some machine wants to backup or restore a file.
2. It also needs to be able to find all the files backed up of others as well as my own files on my PC.
3. I will maintain files placed on each PC using hash tables for efficient searching.When this is done the daemon will figure out from a heap of free disk space how many comps are to be involved.I then use a TCPserver or TCPclient class to open connections to them and send or receive encrypted files.
4. Daemon will send messages across the network if any system change(disk space +/-) occurs.Can anyone give me some documentation and links on the same?
Yeah,one more thing how will that daemon calculate free disk space on my computer -any methods would be appeciated -This willl determine what computers to select for back up.
IF all the above doesnt make any sense just tell me how to write a daemon-I need the code for this.
Also how will ensure that my daemon will run at the same time a machine boots up.
help for any part will be appreciatedCheck out the apache commons daemon library and associated stub program. http://jakarta.apache.org/commons/daemon/
AFAIK There's no quick way in the Java libraries of measuring free disk space. Best bet is to exec the appropriate command line utility and read the results programatically. -
Can Java be used to write service/daemon at solaris?
I new to solaris server side developement, thinking not go back to c/c++ on solaris, to write the system service or daemon. Can I use java to achieve this instead of c/c++ ? Or where I can find the samples / resources about this?
Thanks,
BenThere is no reason you cannot write a daemon in Java. The cloudscape database daemon that was shipped with the original Forte for Java is written in Java. You need to decide if Java can give the the performance you require for your service/daemon, however.
You can find some examples and an API for writing Java Daemons at:
http://www.jcp.org/en/jsr/detail?id=96
http://jakarta.apache.org/commons/sandbox/daemon/
http://sourceforge.net/projects/telnetd/ -
Hello everyone,
I'm searching for a design pattern / framework to manage exception handling. I'm currently working on a distributed document management system for PC / AS/400, which consists of Commandline clients, a Socket Server and a windows NT daemon in java, which accepts network requests from the Socket Server. Communication is done via serialized Objects.
My Problem is, that exceptions can be thrown either on the server or on the client side and have to be transferred to the user. Error Messages should be read from the database. Exception handling should prefferably take place in a central piece of code, such as the two endpoints of network connections, the SocketServer as service under win32 and another SocketServer as Application on another box, currently with UNIX OS.
thanks in advance for any answers
regardsYou should look into Bridge [GOF:151] and Memento [GOF:273].
Bridge allows you to decouple your mechanism from the implementation memento on how to propergate the decoupled exception information. -
Hello, first of all I'm new to Java. We've a service written with PHP which sends a lot of emails to our users. Right now all new email messages added to MongoDB and by cron script(PHP) we send them out. I'm trying to implement a Mail Queue Daemon with Java, so it could process them asynchronous and without cron script. (We need to remove delays with cron, and if we will call php script faster, we will stuck in a concurrency with fetching, sending, removing emails from Mongo)
The question is does Java community have any kind of class I can use or should I write it from scratch? I would appreciate any advices.
Thanks.Well, sorry for not replying sooner. I wrote a little script, it works as I wanted to and everything good, except the size. I put it on github please take a look maybe i missed something or it can be tune. I tested it with hundreds of emails to be sent, and postfix log showed that for each email my daemon creates new connection. I didn't find how to use persistent connection with JavaMail, but i actually haven't searched it yet &) But what is more important is ps -aux on our freebsd:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
68094 root 18 44 0 *1208M* 32388K ucond 0 0:01 0.00% java
1G for simple script? this is when it just waiting and not sending mail. How it can be optimized?
2DrClap I prefer to name it daemon because it runs and lives permanent and not using cron. I dunno about threads, but i believe if we will send ~10k and more emails per hour, would be better to use threads, I don't want situation when client registering on site and waiting for five mins when activation email arrives. And as I mention above and as you can see in script we use Mongo, its fast. Really fast. I generate 100k emails for test in ~1 second. So Mongo isn't bottleneck and will never be. -
Blocked trying to get lock: java.lang.Class@[fat lock]
Joe,
Can you please help me understand what is going on with this thread dump? There is no dead lock; but I have stuck thread. Is it waiting an Oracle prepared statement?
===== FULL THREAD DUMP ===============
Wed Sep 4 16:12:27 2013
Oracle JRockit(R) R28.0.1-21-133393-1.6.0_20-20100512-2126-linux-x86_64
"Main Thread" id=1 idx=0x4 tid=26484 prio=5 alive, waiting, native_blocked
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x1398a8228[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:902)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x1398a8228[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:465)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(OC Main Thread)" id=2 idx=0x8 tid=26487 prio=5 alive, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0xc tid=26488 prio=5 alive, daemon
"(GC Worker Thread 2)" id=? idx=0x10 tid=26489 prio=5 alive, daemon
"(GC Worker Thread 3)" id=? idx=0x14 tid=26490 prio=5 alive, daemon
"(GC Worker Thread 4)" id=? idx=0x18 tid=26491 prio=5 alive, daemon
"(Code Generation Thread 1)" id=3 idx=0x1c tid=26494 prio=5 alive, native_waiting, daemon
"(Code Optimization Thread 1)" id=4 idx=0x20 tid=26495 prio=5 alive, native_waiting, daemon
"(VM Periodic Task)" id=5 idx=0x24 tid=26496 prio=10 alive, native_blocked, daemon
"(Attach Listener)" id=6 idx=0x28 tid=26497 prio=5 alive, native_blocked, daemon
"Finalizer" id=7 idx=0x2c tid=26498 prio=8 alive, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:189)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x30 tid=26499 prio=10 alive, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x34 tid=26500 prio=5 alive, native_blocked, daemon
"VM JFR Buffer Thread" id=10 idx=0x38 tid=26501 prio=5 alive, in native, daemon
"Timer-0" id=13 idx=0x3c tid=26502 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x1399c29d0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0x1399c29d0[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Timer-1" id=14 idx=0x40 tid=26504 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x1399c2a58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x1399c2a58[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x44 tid=26505 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x1399c3848[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1964)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)[optimized]
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x1399c3848[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JFR request timer" id=16 idx=0x48 tid=26506 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x1399c2ae0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0x1399c2ae0[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.time.TimeEventGenerator" id=18 idx=0x4c tid=26507 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/time/common/internal/TimeTable@0x1399c5070[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0x1399c5070[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JMAPI event thread" id=19 idx=0x50 tid=26508 prio=5 alive, in native, daemon
"weblogic.timers.TimerThread" id=20 idx=0x54 tid=26509 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/timers/internal/TimerThread@0x1399c2c08[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0x1399c2c08[fat lock]
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" id=21 idx=0x58 tid=26510 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x1399c3930[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1964)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)[optimized]
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x1399c3930[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"NM Reaper" id=24 idx=0x5c tid=26512 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x1399c5938[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x1399c5938[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=25 idx=0x60 tid=26514 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x13898d7f0[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1402)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1503)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1045)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:996)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2170)[optimized]
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=26 idx=0x64 tid=26515 prio=5 alive, in native, daemon
at jrockit/ext/epoll/EPoll.epollWait0(ILjava/nio/ByteBuffer;II)I(Native Method)
at jrockit/ext/epoll/EPoll.epollWait(EPoll.java:115)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:156)
^-- Holding lock: java/lang/String@0x13898d7f0[fat lock]
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=27 idx=0x68 tid=26516 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x13898d7f0[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1402)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1503)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1045)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:996)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2170)[optimized]
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" id=28 idx=0x6c tid=26517 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x13898d7f0[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1402)[inlined]
at jrockit/vm/Locks.fatLockReacquire(Locks.java:1640)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1508)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1045)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:996)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2170)[optimized]
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"VDE Transaction Processor Thread" id=31 idx=0x70 tid=26559 prio=2 alive, waiting, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0x1399c3ab8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:367)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0x1399c3ab8[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"UCP-worker-thread-16" id=485 idx=0x74 tid=1649 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x106f68908
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)[inlined]
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)[optimized]
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)[optimized]
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)[optimized]
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"DoSManager" id=34 idx=0x78 tid=26561 prio=6 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at com/octetstring/vde/DoSManager.run(DoSManager.java:433)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Timer-2" id=35 idx=0x7c tid=26562 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x1399c5df8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x1399c5df8[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"OracleTimeoutPollingThread" id=36 idx=0x80 tid=26585 prio=10 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/jdbc/driver/OracleTimeoutPollingThread.run(OracleTimeoutPollingThread.java:155)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Thread-14" id=37 idx=0x84 tid=26586 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x1399c6018
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)[inlined]
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)[optimized]
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)[optimized]
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JPS FileChangeNotifier<clean-only>" id=38 idx=0x88 tid=26587 prio=5 alive, parked, native_blocked, daemon
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:198)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)[optimized]
at java/util/concurrent/DelayQueue.take(DelayQueue.java:164)[optimized]
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)[inlined]
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)[optimized]
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)[optimized]
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"LogFlusher" id=39 idx=0x8c tid=26588 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"LogFlusher" id=40 idx=0x90 tid=26589 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"LogFlusher" id=41 idx=0x94 tid=26590 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" id=265 idx=0x98 tid=28296 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x106f29288[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1964)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)[optimized]
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x106f29288[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Thread-71" id=266 idx=0x9c tid=28297 prio=10 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/wsm/policyaccess/impl/cache/PolicyCacheImpl$BackTracingSynchronizationThread.run(PolicyCacheImpl.java:759)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Thread-70" id=264 idx=0xa0 tid=28295 prio=10 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x106f3df50
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at oracle/mds/internal/notifications/EventDispatcherThread.run(EventDispatcherThread.java:54)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"oracle.dfw.impl.incident - ADR Executor (created: Fri Aug 30 01:12:40 GMT 2013)" id=45 idx=0xa4 tid=26601 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x1399c6698
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)[inlined]
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)[optimized]
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)[optimized]
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)[optimized]
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@37a58fc8" id=394 idx=0xa8 tid=28767 prio=10 alive, in native, daemon
at jrockit/net/SocketNativeIO.readBytesPinned(Ljava/io/FileDescriptor;[BIII)I(Native Method)
at jrockit/net/SocketNativeIO.socketRead(SocketNativeIO.java:32)[inlined]
at java/net/SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I(SocketInputStream.java)[inlined]
at java/net/SocketInputStream.read(SocketInputStream.java:129)[optimized]
at oracle/net/nt/MetricsEnabledInputStream.read(TcpNTAdapter.java:564)[optimized]
at oracle/net/ns/Packet.receive(Packet.java:295)[inlined]
at oracle/net/ns/DataPacket.receive(DataPacket.java:94)[optimized]
at oracle/net/ns/NetInputStream.getNextPacket(NetInputStream.java:176)[optimized]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:121)[inlined]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:96)[inlined]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:81)[optimized]
at oracle/jdbc/driver/T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)[optimized]
at oracle/jdbc/driver/T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)[inlined]
at oracle/jdbc/driver/T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1087)[optimized]
at oracle/jdbc/driver/T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)[inlined]
at oracle/jdbc/driver/T4C8Oall.receive(T4C8Oall.java:625)[optimized]
at oracle/jdbc/driver/T4CCallableStatement.doOall8(T4CCallableStatement.java:201)[inlined]
at oracle/jdbc/driver/T4CCallableStatement.executeForRows(T4CCallableStatement.java:1036)[optimized]
at oracle/jdbc/driver/OracleStatement.doExecuteWithTimeout(OracleStatement.java:1379)[optimized]
at oracle/jdbc/driver/OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3568)[optimized]
at oracle/jdbc/driver/OraclePreparedStatement.execute(OraclePreparedStatement.java:3739)[optimized]
^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x125a827f8[recursive]
at oracle/jdbc/driver/OracleCallableStatement.execute(OracleCallableStatement.java:8150)[optimized]
^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x125a827f8[thin lock]
at oracle/jdbc/driver/OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1535)[optimized]
at weblogic/jdbc/wrapper/PreparedStatement.execute(PreparedStatement.java:99)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.readEvent(SAQBusinessEventBus.java:1232)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.handleSingleQueueEvent(SAQBusinessEventBus.java:687)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.handleQueueEvents(SAQBusinessEventBus.java:660)
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.access$000(SAQBusinessEventBus.java:71)
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus$1.run(SAQBusinessEventBus.java:319)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:105)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:183)
at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"oracle.dfw.incident.IncidentCreatorThread" id=47 idx=0xac tid=26603 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/ArrayList@0x1399c6780[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at oracle/dfw/impl/incident/DiagnosticsDataExtractorImpl$IncidentCreatorThread.run(DiagnosticsDataExtractorImpl.java:1597)
^-- Lock released while waiting: java/util/ArrayList@0x1399c6780[fat lock]
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"oracle.dfw.incident.FloodController" id=49 idx=0xb4 tid=26606 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/lang/Object@0x1399c6868[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at oracle/dfw/impl/incident/FloodController.run(FloodController.java:136)
^-- Lock released while waiting: java/lang/Object@0x1399c6868[fat lock]
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"UCP-worker-thread-16" id=487 idx=0xb8 tid=1681 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x106f89660
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)[inlined]
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)[optimized]
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)[optimized]
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)[optimized]
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Thread-72" id=267 idx=0xbc tid=28298 prio=10 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/wsm/policyaccess/impl/cache/PolicyCacheImpl$PolicyCacheRefreshThread.run(PolicyCacheImpl.java:915)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@37a58d77" id=393 idx=0xc0 tid=28766 prio=10 alive, in native, daemon
at jrockit/net/SocketNativeIO.readBytesPinned(Ljava/io/FileDescriptor;[BIII)I(Native Method)
at jrockit/net/SocketNativeIO.socketRead(SocketNativeIO.java:32)[inlined]
at java/net/SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I(SocketInputStream.java)[inlined]
at java/net/SocketInputStream.read(SocketInputStream.java:129)[optimized]
at oracle/net/nt/MetricsEnabledInputStream.read(TcpNTAdapter.java:564)[optimized]
at oracle/net/ns/Packet.receive(Packet.java:295)[inlined]
at oracle/net/ns/DataPacket.receive(DataPacket.java:94)[optimized]
at oracle/net/ns/NetInputStream.getNextPacket(NetInputStream.java:176)[optimized]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:121)[inlined]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:96)[inlined]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:81)[optimized]
at oracle/jdbc/driver/T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)[optimized]
at oracle/jdbc/driver/T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)[inlined]
at oracle/jdbc/driver/T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1087)[optimized]
at oracle/jdbc/driver/T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)[inlined]
at oracle/jdbc/driver/T4C8Oall.receive(T4C8Oall.java:625)[optimized]
at oracle/jdbc/driver/T4CCallableStatement.doOall8(T4CCallableStatement.java:201)[inlined]
at oracle/jdbc/driver/T4CCallableStatement.executeForRows(T4CCallableStatement.java:1036)[optimized]
at oracle/jdbc/driver/OracleStatement.doExecuteWithTimeout(OracleStatement.java:1379)[optimized]
at oracle/jdbc/driver/OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3568)[optimized]
at oracle/jdbc/driver/OraclePreparedStatement.execute(OraclePreparedStatement.java:3739)[optimized]
^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x12e33e6b8[recursive]
at oracle/jdbc/driver/OracleCallableStatement.execute(OracleCallableStatement.java:8150)[optimized]
^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x12e33e6b8[thin lock]
at oracle/jdbc/driver/OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1535)[optimized]
at weblogic/jdbc/wrapper/PreparedStatement.execute(PreparedStatement.java:99)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.readOAOODelivery(SAQBusinessEventBus.java:1302)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.handleSingleOAOODelivery(SAQBusinessEventBus.java:750)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.handleQueueEvents(SAQBusinessEventBus.java:669)
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.access$000(SAQBusinessEventBus.java:71)
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus$1.run(SAQBusinessEventBus.java:319)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:105)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:183)
at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@37a4ecb8" id=392 idx=0xc4 tid=28765 prio=10 alive, in native, daemon
at jrockit/net/SocketNativeIO.readBytesPinned(Ljava/io/FileDescriptor;[BIII)I(Native Method)
at jrockit/net/SocketNativeIO.socketRead(SocketNativeIO.java:32)[inlined]
at java/net/SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I(SocketInputStream.java)[inlined]
at java/net/SocketInputStream.read(SocketInputStream.java:129)[optimized]
at oracle/net/nt/MetricsEnabledInputStream.read(TcpNTAdapter.java:564)[optimized]
at oracle/net/ns/Packet.receive(Packet.java:295)[inlined]
at oracle/net/ns/DataPacket.receive(DataPacket.java:94)[optimized]
at oracle/net/ns/NetInputStream.getNextPacket(NetInputStream.java:176)[optimized]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:121)[inlined]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:96)[inlined]
at oracle/net/ns/NetInputStream.read(NetInputStream.java:81)[optimized]
at oracle/jdbc/driver/T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)[optimized]
at oracle/jdbc/driver/T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)[inlined]
at oracle/jdbc/driver/T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1087)[optimized]
at oracle/jdbc/driver/T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)[inlined]
at oracle/jdbc/driver/T4C8Oall.receive(T4C8Oall.java:625)[optimized]
at oracle/jdbc/driver/T4CCallableStatement.doOall8(T4CCallableStatement.java:201)[inlined]
at oracle/jdbc/driver/T4CCallableStatement.executeForRows(T4CCallableStatement.java:1036)[optimized]
at oracle/jdbc/driver/OracleStatement.doExecuteWithTimeout(OracleStatement.java:1379)[optimized]
at oracle/jdbc/driver/OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3568)[optimized]
at oracle/jdbc/driver/OraclePreparedStatement.execute(OraclePreparedStatement.java:3739)[optimized]
^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x125a80ae8[recursive]
at oracle/jdbc/driver/OracleCallableStatement.execute(OracleCallableStatement.java:8150)[optimized]
^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x125a80ae8[thin lock]
at oracle/jdbc/driver/OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1535)[optimized]
at weblogic/jdbc/wrapper/PreparedStatement.execute(PreparedStatement.java:99)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.readEvent(SAQBusinessEventBus.java:1232)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.handleSingleQueueEvent(SAQBusinessEventBus.java:687)[optimized]
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.handleQueueEvents(SAQBusinessEventBus.java:660)
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus.access$000(SAQBusinessEventBus.java:71)
at oracle/integration/platform/blocks/event/saq/SAQBusinessEventBus$1.run(SAQBusinessEventBus.java:319)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:105)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:183)
at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"UCP-worker-thread-16" id=488 idx=0xc8 tid=1682 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x102a76870
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)[inlined]
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)[optimized]
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)[optimized]
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)[optimized]
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@37afc2f4" id=396 idx=0xcc tid=28769 prio=10 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/tip/mediator/common/listener/DBLocker.enqueueLockedMessages(DBLocker.java:213)[optimized]
at oracle/tip/mediator/common/listener/DBLocker.run(DBLocker.java:84)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:105)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:183)
at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"LogFlusher" id=56 idx=0xd0 tid=26614 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@37afb4b1" id=395 idx=0xd4 tid=28768 prio=10 alive, parked, native_blocked, daemon
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)
at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:592)[inlined]
at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:198)[inlined]
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)[optimized]
at java/util/concurrent/LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)[optimized]
at oracle/tip/mediator/common/listener/AbstractWorker.run(AbstractWorker.java:64)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:105)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:183)
You need to open a support case, and it's for the application, not WebLogic. You have threads deadlocked
in application code.
Circular (deadlocked) lock chains
=================================
Chain 5:
"[STUCK] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'" id=577 idx=0x560 tid=5192 waiting for java/util/concurrent/locks/ReentrantLock$NonfairSync@0x3895ee610 held by:
"[STUCK] ExecuteThread: '9' for queue: 'weblogic.kernel.Default (self-tuning)'" id=433 idx=0x278 tid=30215 waiting for java/util/concurrent/locks/ReentrantLock$NonfairSync@0x38a904880 held by:
"[STUCK] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'" id=577 idx=0x560 tid=5192 -
Ghostship: runs the windows proxy application "Ultrasurf" as a daemon
I like to run "Ultrasurf" with wine on Archlinux, because it is a free and for that pretty fast nice proxy application which tunnels everything through SSL. While doing so, I've started writing scripts that automate things for me. Lately I decided to advance these scripts even further to a fully functional daemon I call ghostship. So here's a quick overview what that daemon does:
* Create its own user and group to limit its rights
* Set up a wineprefix
* Install/update Ultrasurf
* Install the needed dlls by downloading and installing the m$ visual c++ runtime
* Runs Ultrasurf hidden in a vnc server, thus no GUI will be annoying you
* Sends data through Ultrasurf from time to time, to make sure that it keeps connected
* If Ultrasurf is not connected for whatever reason, it restarts the whole vnc server
Feedback is welcome
Homepage, AURThis is partly what I needed:
* To change this template, choose Tools | Templates
* and open the template in the editor.
package daemon;
import java.util.Timer;
import java.util.TimerTask;
import java.util.logging.Level;
import java.util.logging.Logger;
//import javax.swing.Timer;
* @author arthur
public class Daemon
public static void main(String[] args)
int delay = 5000; // delay for 5 sec. before start
int interval = 1000; // iterate every sec.
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask()
public void run()
// Task here ...
log(Level.INFO, "Hello World");
}, delay, interval);
private static void log(Level level, String message)
Logger.getLogger(Daemon.class.getName()).log(level, message);
} -
High CPU usage with Coherence 3.5.0 cluster node
We're noticing sustained high CPU usage on one of our Coherence nodes - this happens after running a job that pretty intensively interacts with the cache. The jobs complete, however we still see the Coherence process using approximately the equivalent of one core's worth of CPU. I took a couple of thread dumps about fifteen minutes apart, and the only noticeable differences are within the PacketListener1 and PacketListenerN threads - they're holding locks on different DatagramPacket instances, which suggests to be that this may be the piece of code that is looping.
There are two other cache nodes in this particular cluster - both of them on a different machine - and neither of them are exhibiting the same CPU utilization.
Environment:
Coherence 3.5.0
Java - BEA JRockit(R) (build R27.6.3-40_o-112056-1.6.0_11-20090318-2103-linux-x86_64, compiled mode)
Has anyone encountered this scenario before?
Here's a copy of the two thread dumps:
Thread Dump 1 -
===== FULL THREAD DUMP ===============
Thu Feb 24 21:45:00 2011
BEA JRockit(R) R27.6.3-40_o-112056-1.6.0_11-20090318-2103-linux-x86_64
"Main Thread" id=1 idx=0x4 tid=18143 prio=5 alive, in native, waiting
-- Waiting for notification on: java/lang/Class@0x43587b58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/net/DefaultCacheServer.main(DefaultCacheServer.java:80)
^-- Lock released while waiting: java/lang/Class@0x43587b58[fat lock]
at com/zzzz/carbon/cacheserver/ZzzzCoherenceServerStartup.doWork(ZzzzCoherenceServerStartup.java:29)
at com/zzzz/util/runner/ZzzzRunnerBase.run(ZzzzRunnerBase.java:23)
at com/zzzz/carbon/cacheserver/ZzzzCoherenceServerStartup.main(ZzzzCoherenceServerStartup.java:16)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=18144 prio=5 alive, in native, daemon
"(GC Main Thread)" id=3 idx=0xc tid=18145 prio=5 alive, in native, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=18146 prio=5 alive, in native, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=18147 prio=5 alive, in native, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=18148 prio=5 alive, in native, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=18149 prio=5 alive, in native, daemon
"(Code Generation Thread 1)" id=4 idx=0x20 tid=18150 prio=5 alive, in native, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x24 tid=18151 prio=5 alive, in native, native_waiting, daemon
"(VM Periodic Task)" id=6 idx=0x28 tid=18152 prio=10 alive, in native, daemon
"Finalizer" id=7 idx=0x2c tid=18153 prio=8 alive, in native, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees([Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$500(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:159)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x30 tid=18154 prio=10 alive, in native, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue()Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:79)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x34 tid=18155 prio=5 alive, in native, daemon
"Logger@9217551 3.5/459" id=12 idx=0x38 tid=18158 prio=3 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/application/console/Coherence$Logger$Queue@0x4e1faef8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/application/console/Coherence$Logger$Queue@0x4e1faef8[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketListener1" id=14 idx=0x3c tid=18159 prio=8 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb21360[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)[inlined]
at java/net/DatagramSocket.receive(DatagramSocket.java:712)[optimized]
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb21360[thin lock]
^-- Holding lock: java/net/DatagramPacket@0x73aa4e80[thin lock]
^-- Holding lock: java/net/DatagramSocket@0x4cb21338[thin lock]
at com/tangosol/coherence/component/net/socket/UdpSocket.receive(UdpSocket.CDB:20)[optimized]
at com/tangosol/coherence/component/net/UdpPacket.receive(UdpPacket.CDB:4)[optimized]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketListener.onNotify(PacketListener.CDB:19)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketReceiver" id=15 idx=0x40 tid=18160 prio=7 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/net/Cluster$PacketReceiver$InQueue@0x4cf60d58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketReceiver.onWait(PacketReceiver.CDB:2)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketReceiver$InQueue@0x4cf60d58[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketPublisher" id=16 idx=0x44 tid=18161 prio=6 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/net/Cluster$PacketPublisher$InQueue@0x4ca053a8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketPublisher.onWait(PacketPublisher.CDB:2)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketPublisher$InQueue@0x4ca053a8[fat lock]
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketPublisher$InQueue@0x4ca053a8[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketSpeaker" id=17 idx=0x48 tid=18162 prio=8 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/net/Cluster$PacketSpeaker$BundlingQueue@0x4cabe9d8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/queue/ConcurrentQueue.waitForEntry(ConcurrentQueue.CDB:16)
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketSpeaker$BundlingQueue@0x4cabe9d8[fat lock]
at com/tangosol/coherence/component/util/queue/ConcurrentQueue.remove(ConcurrentQueue.CDB:7)
at com/tangosol/coherence/component/util/Queue.remove(Queue.CDB:1)
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketSpeaker.onNotify(PacketSpeaker.CDB:62)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketListenerN" id=18 idx=0x4c tid=18163 prio=8 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb221b0[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)[inlined]
at java/net/DatagramSocket.receive(DatagramSocket.java:712)[optimized]
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb221b0[thin lock]
^-- Holding lock: java/net/DatagramPacket@0x73e59890[thin lock]
^-- Holding lock: java/net/MulticastSocket@0x4cb22178[thin lock]
at com/tangosol/coherence/component/net/socket/UdpSocket.receive(UdpSocket.CDB:20)[optimized]
at com/tangosol/coherence/component/net/UdpPacket.receive(UdpPacket.CDB:4)[optimized]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketListener.onNotify(PacketListener.CDB:19)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Cluster|Member(Id=2, Timestamp=2011-02-23 16:25:36.488, Address=10.10.100.77:8095, MachineId=25677, Location=site:emtzzzz.com,machine:njdev04,process:18142,member:cldev201~carboncachelauncher~181)" id=19 idx=0x50 tid=18164 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4cac64d8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/service/Grid.onWait(Grid.CDB:9)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4cac64d8[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"TcpRingListener" id=20 idx=0x54 tid=18165 prio=6 alive, in native, daemon
at java/net/PlainSocketImpl.socketAccept(Ljava/net/SocketImpl;)V(Native Method)
at java/net/PlainSocketImpl.accept(PlainSocketImpl.java:384)
^-- Holding lock: java/net/SocksSocketImpl@0x4cb227d8[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:453)
at java/net/ServerSocket.accept(ServerSocket.java:421)
at com/tangosol/coherence/component/net/socket/TcpSocketAccepter.accept(TcpSocketAccepter.CDB:18)
at com/tangosol/coherence/component/util/daemon/TcpRingListener.acceptConnection(TcpRingListener.CDB:10)
at com/tangosol/coherence/component/util/daemon/TcpRingListener.onNotify(TcpRingListener.CDB:9)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Invocation:Management:EventDispatcher" id=22 idx=0x58 tid=18166 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/daemon/queueProcessor/Service$EventDispatcher$Queue@0x4ab76140[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/daemon/queueProcessor/Service$EventDispatcher$Queue@0x4ab76140[fat lock]
at com/tangosol/coherence/component/util/daemon/queueProcessor/Service$EventDispatcher.onWait(Service.CDB:7)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Invocation:Management" id=23 idx=0x5c tid=18167 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4ab75508[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/service/Grid.onWait(Grid.CDB:9)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4ab75508[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"DistributedCache" id=25 idx=0x60 tid=18168 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4a3869b0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/service/Grid.onWait(Grid.CDB:9)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4a3869b0[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
===== END OF THREAD DUMP ===============
Thread Dump 2 -
===== FULL THREAD DUMP ===============
Thu Feb 24 21:27:53 2011
BEA JRockit(R) R27.6.3-40_o-112056-1.6.0_11-20090318-2103-linux-x86_64
"Main Thread" id=1 idx=0x4 tid=18143 prio=5 alive, in native, waiting
-- Waiting for notification on: java/lang/Class@0x43587b58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/net/DefaultCacheServer.main(DefaultCacheServer.java:80)
^-- Lock released while waiting: java/lang/Class@0x43587b58[fat lock]
at com/zzzz/carbon/cacheserver/ZzzzCoherenceServerStartup.doWork(ZzzzCoherenceServerStartup.java:29)
at com/zzzz/util/runner/ZzzzRunnerBase.run(ZzzzRunnerBase.java:23)
at com/zzzz/carbon/cacheserver/ZzzzCoherenceServerStartup.main(ZzzzCoherenceServerStartup.java:16)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=18144 prio=5 alive, in native, daemon
"(GC Main Thread)" id=3 idx=0xc tid=18145 prio=5 alive, in native, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=18146 prio=5 alive, in native, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=18147 prio=5 alive, in native, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=18148 prio=5 alive, in native, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=18149 prio=5 alive, in native, daemon
"(Code Generation Thread 1)" id=4 idx=0x20 tid=18150 prio=5 alive, in native, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x24 tid=18151 prio=5 alive, in native, native_waiting, daemon
"(VM Periodic Task)" id=6 idx=0x28 tid=18152 prio=10 alive, in native, daemon
"Finalizer" id=7 idx=0x2c tid=18153 prio=8 alive, in native, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees([Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$500(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:159)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x30 tid=18154 prio=10 alive, in native, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue()Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:79)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x34 tid=18155 prio=5 alive, in native, daemon
"Logger@9217551 3.5/459" id=12 idx=0x38 tid=18158 prio=3 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/application/console/Coherence$Logger$Queue@0x4e1faef8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/application/console/Coherence$Logger$Queue@0x4e1faef8[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketListener1" id=14 idx=0x3c tid=18159 prio=8 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb21360[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)[inlined]
at java/net/DatagramSocket.receive(DatagramSocket.java:712)[optimized]
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb21360[thin lock]
^-- Holding lock: java/net/DatagramPacket@0x73a80930[thin lock]
^-- Holding lock: java/net/DatagramSocket@0x4cb21338[thin lock]
at com/tangosol/coherence/component/net/socket/UdpSocket.receive(UdpSocket.CDB:20)[optimized]
at com/tangosol/coherence/component/net/UdpPacket.receive(UdpPacket.CDB:4)[optimized]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketListener.onNotify(PacketListener.CDB:19)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketReceiver" id=15 idx=0x40 tid=18160 prio=7 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/net/Cluster$PacketReceiver$InQueue@0x4cf60d58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketReceiver.onWait(PacketReceiver.CDB:2)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketReceiver$InQueue@0x4cf60d58[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketPublisher" id=16 idx=0x44 tid=18161 prio=6 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/net/Cluster$PacketPublisher$InQueue@0x4ca053a8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketPublisher.onWait(PacketPublisher.CDB:2)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketPublisher$InQueue@0x4ca053a8[fat lock]
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketPublisher$InQueue@0x4ca053a8[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketSpeaker" id=17 idx=0x48 tid=18162 prio=8 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/net/Cluster$PacketSpeaker$BundlingQueue@0x4cabe9d8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/queue/ConcurrentQueue.waitForEntry(ConcurrentQueue.CDB:16)
^-- Lock released while waiting: com/tangosol/coherence/component/net/Cluster$PacketSpeaker$BundlingQueue@0x4cabe9d8[fat lock]
at com/tangosol/coherence/component/util/queue/ConcurrentQueue.remove(ConcurrentQueue.CDB:7)
at com/tangosol/coherence/component/util/Queue.remove(Queue.CDB:1)
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketSpeaker.onNotify(PacketSpeaker.CDB:62)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"PacketListenerN" id=18 idx=0x4c tid=18163 prio=8 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb221b0[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)[inlined]
at java/net/DatagramSocket.receive(DatagramSocket.java:712)[optimized]
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x4cb221b0[thin lock]
^-- Holding lock: java/net/DatagramPacket@0x5e55a240[thin lock]
^-- Holding lock: java/net/MulticastSocket@0x4cb22178[thin lock]
at com/tangosol/coherence/component/net/socket/UdpSocket.receive(UdpSocket.CDB:20)[optimized]
at com/tangosol/coherence/component/net/UdpPacket.receive(UdpPacket.CDB:4)[optimized]
at com/tangosol/coherence/component/util/daemon/queueProcessor/packetProcessor/PacketListener.onNotify(PacketListener.CDB:19)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Cluster|Member(Id=2, Timestamp=2011-02-23 16:25:36.488, Address=10.10.100.77:8095, MachineId=25677, Location=site:emtzzzz.com,machine:njdev04,process:18142,member:cldev201~carboncachelauncher~181)" id=19 idx=0x50 tid=18164 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4cac64d8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/service/Grid.onWait(Grid.CDB:9)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4cac64d8[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"TcpRingListener" id=20 idx=0x54 tid=18165 prio=6 alive, in native, daemon
at java/net/PlainSocketImpl.socketAccept(Ljava/net/SocketImpl;)V(Native Method)
at java/net/PlainSocketImpl.accept(PlainSocketImpl.java:384)
^-- Holding lock: java/net/SocksSocketImpl@0x4cb227d8[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:453)
at java/net/ServerSocket.accept(ServerSocket.java:421)
at com/tangosol/coherence/component/net/socket/TcpSocketAccepter.accept(TcpSocketAccepter.CDB:18)
at com/tangosol/coherence/component/util/daemon/TcpRingListener.acceptConnection(TcpRingListener.CDB:10)
at com/tangosol/coherence/component/util/daemon/TcpRingListener.onNotify(TcpRingListener.CDB:9)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:37)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Invocation:Management:EventDispatcher" id=22 idx=0x58 tid=18166 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/daemon/queueProcessor/Service$EventDispatcher$Queue@0x4ab76140[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/daemon/queueProcessor/Service$EventDispatcher$Queue@0x4ab76140[fat lock]
at com/tangosol/coherence/component/util/daemon/queueProcessor/Service$EventDispatcher.onWait(Service.CDB:7)
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Invocation:Management" id=23 idx=0x5c tid=18167 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4ab75508[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/service/Grid.onWait(Grid.CDB:9)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4ab75508[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"DistributedCache" id=25 idx=0x60 tid=18168 prio=5 alive, in native, waiting, daemon
-- Waiting for notification on: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4a3869b0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at com/tangosol/coherence/component/util/Daemon.onWait(Daemon.CDB:18)[inlined]
at com/tangosol/coherence/component/util/daemon/queueProcessor/service/Grid.onWait(Grid.CDB:9)[optimized]
^-- Lock released while waiting: com/tangosol/coherence/component/util/queue/concurrentQueue/DualQueue@0x4a3869b0[fat lock]
at com/tangosol/coherence/component/util/Daemon.run(Daemon.CDB:34)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
===== END OF THREAD DUMP ===============Charlie, Cameron,
Thanks for following up. To answer Cameron's question - yes, running top shows that it is this process using a significant amount of the CPU. I captured a JRA this morning setting nativesamples=true.
I see the following when I look at the Hot Methods - let me know if there is something else that would shed some more light on this issue.
Method Percent(%) #Samples Optimized Native Method Id.
libjvm.so#mmDetGcFollowReferencesWorkers 28.575 2,062 0 1 0x2AE6B82EC2C1
libpthread.so.0#__recvfrom_nocancel 26.012 1,877 0 1 0x3CC1A0D689
libpthread.so.0#__accept_nocancel 26.012 1,877 0 1 0x3CC1A0D4A9
libjvm.so#mmBalanceGetReference 4.975 359 0 1 0x2AE6B817A65B
libjvm.so#objIterStepFieldFromBitmaskWord 4.753 343 0 1 0x2AE6B82A6B34
libjvm.so#objIterStep 3.88 280 0 1 0x2AE6B82A6F8D
libjvm.so#objIterInitInner 2.744 198 0 1 0x2AE6B82A71B7
libjvm.so#mmBalanceStoreReference 1.414 102 0 1 0x2AE6B817A781
libjvm.so#objIterSetupFields 0.637 46 0 1 0x2AE6B82A6B08
libjvm.so#nativeInnerLockNotLinked 0.236 17 0 1 0x2AE6B829DE76
libjvm.so#mmWorkPacketPoolGet 0.125 9 0 1 0x2AE6B817A4F5
libjvm.so#qBitSetIsBitSetInWord 0.069 5 0 1 0x2AE6B82C8AA6
libjvm.so#nativeInnerUnlockNotLinked 0.069 5 0 1 0x2AE6B829E028
libjvm.so#mmWorkPacketPoolAdd 0.042 3 0 1 0x2AE6B817A48E
libjvm.so#objPoolMarkAllHard 0.042 3 0 1 0x2AE6B82A4923
libjvm.so#objIterSetupArray 0.028 2 0 1 0x2AE6B82A6BDB
libjvm.so#markAcquired 0.028 2 0 1 0x2AE6B829DBD3
libjvm.so#mmPointerMatrixTryInsertAtIndex 0.028 2 0 1 0x2AE6B82BEFEB
libjvm.so#mmPointerMatrixClear 0.028 2 0 1 0x2AE6B82BF61C
libjvm.so#objPoolMarkWeakConcurrently 0.028 2 0 1 0x2AE6B82A5358
libjvm.so#qBitSetFindFirstNonClearedWord 0.014 1 0 1 0x2AE6B82C888E
libjvm.so#utilCounterToNanos 0.014 1 0 1 0x2AE6B8313B53
libjvm.so#waitForLockIfNeededAndMarkAcquired 0.014 1 0 1 0x2AE6B829DC30
libjvm.so#vmtLinkData 0.014 1 0 1 0x2AE6B8325B2E
libjvm.so#mmBitsIsObjectMarkedGrey 0.014 1 0 1 0x2AE6B82ED7E9
libjvm.so#qBitSetFindLastSetBitInWord 0.014 1 0 1 0x2AE6B82C8854
libjvm.so#mark_writebarriers 0.014 1 0 1 0x2AE6B8250EEA
libjvm.so#signalNextInLockQueueIfNeeded 0.014 1 0 1 0x2AE6B829DF52
libjvm.so#mmGetUsingMatrixes 0.014 1 0 1 0x2AE6B819F33C
libjvm.so#ptGetThreadId 0.014 1 0 1 0x2AE6B82BD9ED
libc.so.6#memset 0.014 1 0 1 0x3CC0A7A000
libjvm.so#setupNodeForSelf 0.014 1 0 1 0x2AE6B829DD87
libc.so.6#_int_free 0.014 1 0 1 0x3CC0A714E0
libjvm.so#mmAddChunkToList 0.014 1 0 1 0x2AE6B830977C
libjvm.so#vmtiUnlinkData 0.014 1 0 1 0x2AE6B8325B4A
libjvm.so#nativeLockInSuspendCritical 0.014 1 0 1 0x2AE6B829E0E3
libjvm.so#mmSweepHeapPart 0.014 1 0 1 0x2AE6B830989A
libjvm.so#mmBalanceWorkSetSwapPackets 0.014 1 0 1 0x2AE6B817A647
libc.so.6#_int_malloc 0.014 1 0 1 0x3CC0A71E80
libjvm.so#charToJlcType 0.014 1 0 1 0x2AE6B8253633
unknown#unknown functions 0 0 0 1 0x2AE6B8597090
Running with nativemethods=false, I see the following under hot methods
Method Percent(%) #Samples Optimized Native Method Id.
jrockit.vm.Locks.monitorExitSecondStage(Object) 50 1 1 0 0x161D30D0
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.ClusterService.onNotify() 50 1 1 0 0x169D2C70
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketReceiver.onNotify() 0 0 1 0 0x1551C050
com.tangosol.coherence.component.util.queue.ConcurrentQueue.checkFlush(int) 0 0 1 0 0x151C2180
java.lang.Thread.run() 0 0 0 0 0x14E98EB0
jrockit.vm.RNI.c2java(long, long, long, long, long) 0 0 0 0 0x14C07830
unknown#unknown functions 0 0 0 1 0x2AE6B8597090
com.tangosol.coherence.component.util.Daemon.run() 0 0 0 0 0x15CFEBC0
com.tangosol.coherence.component.util.queue.ConcurrentQueue.add(Object) 0 0 1 0 0x15ED6B00
com.tangosol.coherence.component.util.queue.ConcurrentQueue.onAddElement() 0 0 1 0 0x151C9C20
com.tangosol.coherence.component.util.queue.ConcurrentQueue.flush(boolean) 0 0 1 0 0x1618CAD0
Thanks,
Jason -
Issue in Deploying application in Weblogic 10 3 3 + Oracle JRockit(R) R28.1
Hi
I have Weblogic 10 3 3 with Oracle JRockit(R) R28.1.0-123-138454-1.6.0_20-20101014-1350-linux-ia32 as my JVM on my Linux box.
When I deploy my application and starts it, app goes to New state and my Managed servers (on which I deploy my app) are forcefully shutdown.
I am building my application using Oracle JRockit(R) (build R28.1.0-123-138454-1.6.0_20-20101014-1351-windows-ia32
Below is the complete dump I get once my servers are forcefully shutdown
It will great if any one could throw light on what is happening
===== FULL THREAD DUMP ===============
Fri Dec 31 06:45:50 2010
Oracle JRockit(R) R28.1.0-123-138454-1.6.0_20-20101014-1350-linux-ia32
"Main Thread" id=1 idx=0x4 tid=23019 prio=5 alive, waiting, native_blocked
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x78cbd5d0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:902)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x78cbd5d0[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:465)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=23021 prio=5 alive, native_blocked, daemon
"(OC Main Thread)" id=3 idx=0xc tid=23022 prio=5 alive, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=23023 prio=5 alive, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=23024 prio=5 alive, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=23025 prio=5 alive, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=23026 prio=5 alive, daemon
"(Code Generation Thread 1)" id=4 idx=0x20 tid=23033 prio=5 alive, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x24 tid=23034 prio=5 alive, native_waiting, daemon"(VM Periodic Task)" id=6 idx=0x28 tid=23035 prio=10 alive, native_blocked, daemon
"Finalizer" id=7 idx=0x2c tid=23036 prio=8 alive, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:189)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x30 tid=23037 prio=10 alive, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x34 tid=23038 prio=5 alive, native_blocked, daemon
"Timer-0" id=13 idx=0x3c tid=23047 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x7923ea80[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0x7923ea80[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Timer-1" id=14 idx=0x40 tid=23049 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x769fe1b8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x769fe1b8[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x44 tid=23050 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x76aee438[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x76aee438[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"weblogic.time.TimeEventGenerator" id=18 idx=0x4c tid=23052 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/time/common/internal/TimeTable@0x76b7a7c8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0x76b7a7c8[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"JMAPI event thread" id=19 idx=0x50 tid=23053 prio=5 alive, in native, daemon
"weblogic.timers.TimerThread" id=20 idx=0x54 tid=23054 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/timers/internal/TimerThread@0x76bc13b0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0x76bc13b0[fat lock]
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" id=21 idx=0x58 tid=23055 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x772f4e30[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x772f4e30[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"NM Reaper" id=24 idx=0x5c tid=23056 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x77134a50[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x77134a50[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=25 idx=0x60 tid=23057 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x771354a8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)
at jrockit/vm/Locks.lockFat(Locks.java:1512)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=26 idx=0x64 tid=23058 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x771354a8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)
at jrockit/vm/Locks.lockFat(Locks.java:1512)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=27 idx=0x68 tid=23059 prio=5 alive, in native, daemon
at jrockit/ext/epoll/EPoll.epollWait0(ILjava/nio/ByteBuffer;II)I(Native Method)
at jrockit/ext/epoll/EPoll.epollWait(EPoll.java:115)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:156)
^-- Holding lock: java/lang/String@0x771354a8[fat lock]
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" id=28 idx=0x6c tid=23060 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x771354a8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)
at jrockit/vm/Locks.lockFat(Locks.java:1512)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"VDE Transaction Processor Thread" id=31 idx=0x70 tid=23064 prio=2 alive, waiting, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0x77137470[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:367)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0x77137470[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"DoSManager" id=34 idx=0x78 tid=23066 prio=6 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at com/octetstring/vde/DoSManager.run(DoSManager.java:433)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-13" id=35 idx=0x7c tid=23079 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x7a663848
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-14" id=36 idx=0x80 tid=23081 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x78b0d530
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-15" id=37 idx=0x84 tid=23083 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x7a2ab0b0
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"weblogic.cluster.MessageReceiver" id=38 idx=0x88 tid=23084 prio=5 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x778d0da0[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x778d0da0[biased lock]
at java/net/DatagramSocket.receive(DatagramSocket.java:725)
^-- Holding lock: java/net/DatagramPacket@0x778d09d8[biased lock]
^-- Holding lock: java/net/MulticastSocket@0x778d0d40[biased lock]
at weblogic/cluster/MulticastFragmentSocket.receive(MulticastFragmentSocket.java:239)
at weblogic/cluster/FragmentSocketWrapper.receive(FragmentSocketWrapper.java:98)
at weblogic/cluster/MulticastManager.run(MulticastManager.java:460)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" id=44 idx=0xa0 tid=23092 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x7a2bde50[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x7a2bde50[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" id=45 idx=0xa4 tid=23093 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x7a5941e0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x7a5941e0[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" id=46 idx=0xa8 tid=23103 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr$2@0x78a33620[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Thread.join(Thread.java:1143)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr$2@0x78a33620[fat lock]
at java/lang/Thread.join(Thread.java:1196)
at java/lang/ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
at java/lang/ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
at java/lang/Shutdown.runHooks(Shutdown.java:79)
at java/lang/Shutdown.sequence(Shutdown.java:123)
at java/lang/Shutdown.exit(Shutdown.java:168)
^-- Holding lock: java/lang/Class@0x78ee8648[biased lock]
at java/lang/Runtime.exit(Runtime.java:90)
at java/lang/System.exit(System.java:904)
at antlr/CharScanner.panic(CharScanner.java:274)
at antlr/CharScanner.setTokenObjectClass(CharScanner.java:340)
at org/hibernate/hql/ast/HqlLexer.setTokenObjectClass(HqlLexer.java:25)
at antlr/CharScanner.<init>(CharScanner.java:51)
at antlr/CharScanner.<init>(CharScanner.java:60)
at org/hibernate/hql/antlr/HqlBaseLexer.<init>(HqlBaseLexer.java:55)
at org/hibernate/hql/antlr/HqlBaseLexer.<init>(HqlBaseLexer.java:52)
at org/hibernate/hql/antlr/HqlBaseLexer.<init>(HqlBaseLexer.java:46)
at org/hibernate/hql/ast/HqlLexer.<init>(HqlLexer.java:20)
at org/hibernate/hql/ast/HqlParser.getInstance(HqlParser.java:41)
at org/hibernate/hql/ast/QueryTranslatorImpl.parse(QueryTranslatorImpl.java:203)
at org/hibernate/hql/ast/QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:127)
^-- Holding lock: org/hibernate/hql/ast/QueryTranslatorImpl@0x7946d160[biased lock]
at org/hibernate/hql/ast/QueryTranslatorImpl.compile(QueryTranslatorImpl.java:83)
at org/hibernate/impl/SessionFactoryImpl.getQuery(SessionFactoryImpl.java:427)
at org/hibernate/impl/SessionFactoryImpl.checkNamedQueries(SessionFactoryImpl.java:388)
at org/hibernate/impl/SessionFactoryImpl.<init>(SessionFactoryImpl.java:291)
at org/hibernate/cfg/Configuration.buildSessionFactory(Configuration.java:1005)
at org/springframework/orm/hibernate3/LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:825)
at org/springframework/orm/hibernate3/LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:751)
at org/springframework/beans/factory/support/AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1091)
at org/springframework/beans/factory/support/AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:396)
at org/springframework/beans/factory/support/AbstractBeanFactory.getBean(AbstractBeanFactory.java:233)
^-- Holding lock: java/util/HashMap@0x7a0b1010[thin lock]
at org/springframework/beans/factory/support/AbstractBeanFactory.getBean(AbstractBeanFactory.java:145)
at org/springframework/beans/factory/support/DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:277)
at org/springframework/context/support/AbstractApplicationContext.refresh(AbstractApplicationContext.java:313)
at org/springframework/web/context/support/AbstractRefreshableWebApplicationContext.refresh(AbstractRefreshableWebApplicationContext.java:139)
at org/springframework/web/context/ContextLoader.createWebApplicationContext(ContextLoader.java:252)
at org/springframework/web/context/ContextLoader.initWebApplicationContext(ContextLoader.java:190)
at org/springframework/web/context/ContextLoaderListener.contextInitialized(ContextLoaderListener.java:49)
at weblogic/servlet/internal/EventsManager$FireContextListenerAction.run(EventsManager.java:481)
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)
at weblogic/servlet/internal/EventsManager.notifyContextCreatedEvent(EventsManager.java:181)
at weblogic/servlet/internal/WebAppServletContext.preloadResources(WebAppServletContext.java:1863)
^-- Holding lock: weblogic/servlet/internal/WebAppServletContext@0x7a2e8670[thin lock]
at weblogic/servlet/internal/WebAppServletContext.start(WebAppServletContext.java:3126)
at weblogic/servlet/internal/WebAppModule.startContexts(WebAppModule.java:1512)
at weblogic/servlet/internal/WebAppModule.start(WebAppModule.java:486)
at weblogic/application/internal/flow/ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic/application/utils/StateMachineDriver.nextState(StateMachineDriver.java:41)
at weblogic/application/internal/flow/ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic/application/internal/flow/ScopedModuleDriver.start(ScopedModuleDriver.java:200)
at weblogic/application/internal/flow/ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
at weblogic/application/internal/flow/ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic/application/utils/StateMachineDriver.nextState(StateMachineDriver.java:41)
at weblogic/application/internal/flow/ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic/application/internal/flow/StartModulesFlow.activate(StartModulesFlow.java:27)
at weblogic/application/internal/BaseDeployment$2.next(BaseDeployment.java:1267)
at weblogic/application/utils/StateMachineDriver.nextState(StateMachineDriver.java:41)
at weblogic/application/internal/BaseDeployment.activate(BaseDeployment.java:409)
at weblogic/application/internal/SingleModuleDeployment.activate(SingleModuleDeployment.java:43)
at weblogic/application/internal/DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic/deploy/internal/targetserver/AppContainerInvoker.activate(AppContainerInvoker.java:79)
at weblogic/deploy/internal/targetserver/operations/AbstractOperation.activate(AbstractOperation.java:569)
at weblogic/deploy/internal/targetserver/operations/ActivateOperation.activateDeployment(ActivateOperation.java:150)
at weblogic/deploy/internal/targetserver/operations/ActivateOperation.doCommit(ActivateOperation.java:116)
at weblogic/deploy/internal/targetserver/operations/StartOperation.doCommit(StartOperation.java:143)
at weblogic/deploy/internal/targetserver/operations/AbstractOperation.commit(AbstractOperation.java:323)
at weblogic/deploy/internal/targetserver/DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844)
at weblogic/deploy/internal/targetserver/DeploymentManager.activateDeploymentList(DeploymentManager.java:1253)
^-- Holding lock: java/util/ArrayList@0x7a032928[biased lock]
at weblogic/deploy/internal/targetserver/DeploymentManager.handleCommit(DeploymentManager.java:440)
at weblogic/deploy/internal/targetserver/DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163)
at weblogic/deploy/service/internal/targetserver/DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
at weblogic/deploy/service/internal/targetserver/DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
at weblogic/deploy/service/internal/targetserver/DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
at weblogic/work/SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:201)
at weblogic/work/ExecuteThread.run(ExecuteThread.java:173)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of traceHi
I have Weblogic 10 3 3 with Oracle JRockit(R) R28.1.0-123-138454-1.6.0_20-20101014-1350-linux-ia32 as my JVM on my Linux box.
When I deploy my application and starts it, app goes to New state and my Managed servers (on which I deploy my app) are forcefully shutdown.
I am building my application using Oracle JRockit(R) (build R28.1.0-123-138454-1.6.0_20-20101014-1351-windows-ia32
Below is the complete dump I get once my servers are forcefully shutdown
It will great if any one could throw light on what is happening
===== FULL THREAD DUMP ===============
Fri Dec 31 06:45:50 2010
Oracle JRockit(R) R28.1.0-123-138454-1.6.0_20-20101014-1350-linux-ia32
"Main Thread" id=1 idx=0x4 tid=23019 prio=5 alive, waiting, native_blocked
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x78cbd5d0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:902)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x78cbd5d0[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:465)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=23021 prio=5 alive, native_blocked, daemon
"(OC Main Thread)" id=3 idx=0xc tid=23022 prio=5 alive, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=23023 prio=5 alive, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=23024 prio=5 alive, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=23025 prio=5 alive, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=23026 prio=5 alive, daemon
"(Code Generation Thread 1)" id=4 idx=0x20 tid=23033 prio=5 alive, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x24 tid=23034 prio=5 alive, native_waiting, daemon"(VM Periodic Task)" id=6 idx=0x28 tid=23035 prio=10 alive, native_blocked, daemon
"Finalizer" id=7 idx=0x2c tid=23036 prio=8 alive, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:189)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x30 tid=23037 prio=10 alive, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x34 tid=23038 prio=5 alive, native_blocked, daemon
"Timer-0" id=13 idx=0x3c tid=23047 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x7923ea80[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0x7923ea80[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Timer-1" id=14 idx=0x40 tid=23049 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x769fe1b8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x769fe1b8[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x44 tid=23050 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x76aee438[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x76aee438[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"weblogic.time.TimeEventGenerator" id=18 idx=0x4c tid=23052 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/time/common/internal/TimeTable@0x76b7a7c8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0x76b7a7c8[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"JMAPI event thread" id=19 idx=0x50 tid=23053 prio=5 alive, in native, daemon
"weblogic.timers.TimerThread" id=20 idx=0x54 tid=23054 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/timers/internal/TimerThread@0x76bc13b0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0x76bc13b0[fat lock]
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" id=21 idx=0x58 tid=23055 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x772f4e30[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x772f4e30[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"NM Reaper" id=24 idx=0x5c tid=23056 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x77134a50[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x77134a50[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=25 idx=0x60 tid=23057 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x771354a8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)
at jrockit/vm/Locks.lockFat(Locks.java:1512)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=26 idx=0x64 tid=23058 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x771354a8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)
at jrockit/vm/Locks.lockFat(Locks.java:1512)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=27 idx=0x68 tid=23059 prio=5 alive, in native, daemon
at jrockit/ext/epoll/EPoll.epollWait0(ILjava/nio/ByteBuffer;II)I(Native Method)
at jrockit/ext/epoll/EPoll.epollWait(EPoll.java:115)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:156)
^-- Holding lock: java/lang/String@0x771354a8[fat lock]
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" id=28 idx=0x6c tid=23060 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0x771354a8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)
at jrockit/vm/Locks.lockFat(Locks.java:1512)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"VDE Transaction Processor Thread" id=31 idx=0x70 tid=23064 prio=2 alive, waiting, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0x77137470[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:367)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0x77137470[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"DoSManager" id=34 idx=0x78 tid=23066 prio=6 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at com/octetstring/vde/DoSManager.run(DoSManager.java:433)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-13" id=35 idx=0x7c tid=23079 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x7a663848
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-14" id=36 idx=0x80 tid=23081 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x78b0d530
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-15" id=37 idx=0x84 tid=23083 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x7a2ab0b0
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"weblogic.cluster.MessageReceiver" id=38 idx=0x88 tid=23084 prio=5 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x778d0da0[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
^-- Holding lock: java/net/PlainDatagramSocketImpl@0x778d0da0[biased lock]
at java/net/DatagramSocket.receive(DatagramSocket.java:725)
^-- Holding lock: java/net/DatagramPacket@0x778d09d8[biased lock]
^-- Holding lock: java/net/MulticastSocket@0x778d0d40[biased lock]
at weblogic/cluster/MulticastFragmentSocket.receive(MulticastFragmentSocket.java:239)
at weblogic/cluster/FragmentSocketWrapper.receive(FragmentSocketWrapper.java:98)
at weblogic/cluster/MulticastManager.run(MulticastManager.java:460)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" id=44 idx=0xa0 tid=23092 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x7a2bde50[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x7a2bde50[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" id=45 idx=0xa4 tid=23093 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x7a5941e0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x7a5941e0[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" id=46 idx=0xa8 tid=23103 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr$2@0x78a33620[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Thread.join(Thread.java:1143)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr$2@0x78a33620[fat lock]
at java/lang/Thread.join(Thread.java:1196)
at java/lang/ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
at java/lang/ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
at java/lang/Shutdown.runHooks(Shutdown.java:79)
at java/lang/Shutdown.sequence(Shutdown.java:123)
at java/lang/Shutdown.exit(Shutdown.java:168)
^-- Holding lock: java/lang/Class@0x78ee8648[biased lock]
at java/lang/Runtime.exit(Runtime.java:90)
at java/lang/System.exit(System.java:904)
at antlr/CharScanner.panic(CharScanner.java:274)
at antlr/CharScanner.setTokenObjectClass(CharScanner.java:340)
at org/hibernate/hql/ast/HqlLexer.setTokenObjectClass(HqlLexer.java:25)
at antlr/CharScanner.<init>(CharScanner.java:51)
at antlr/CharScanner.<init>(CharScanner.java:60)
at org/hibernate/hql/antlr/HqlBaseLexer.<init>(HqlBaseLexer.java:55)
at org/hibernate/hql/antlr/HqlBaseLexer.<init>(HqlBaseLexer.java:52)
at org/hibernate/hql/antlr/HqlBaseLexer.<init>(HqlBaseLexer.java:46)
at org/hibernate/hql/ast/HqlLexer.<init>(HqlLexer.java:20)
at org/hibernate/hql/ast/HqlParser.getInstance(HqlParser.java:41)
at org/hibernate/hql/ast/QueryTranslatorImpl.parse(QueryTranslatorImpl.java:203)
at org/hibernate/hql/ast/QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:127)
^-- Holding lock: org/hibernate/hql/ast/QueryTranslatorImpl@0x7946d160[biased lock]
at org/hibernate/hql/ast/QueryTranslatorImpl.compile(QueryTranslatorImpl.java:83)
at org/hibernate/impl/SessionFactoryImpl.getQuery(SessionFactoryImpl.java:427)
at org/hibernate/impl/SessionFactoryImpl.checkNamedQueries(SessionFactoryImpl.java:388)
at org/hibernate/impl/SessionFactoryImpl.<init>(SessionFactoryImpl.java:291)
at org/hibernate/cfg/Configuration.buildSessionFactory(Configuration.java:1005)
at org/springframework/orm/hibernate3/LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:825)
at org/springframework/orm/hibernate3/LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:751)
at org/springframework/beans/factory/support/AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1091)
at org/springframework/beans/factory/support/AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:396)
at org/springframework/beans/factory/support/AbstractBeanFactory.getBean(AbstractBeanFactory.java:233)
^-- Holding lock: java/util/HashMap@0x7a0b1010[thin lock]
at org/springframework/beans/factory/support/AbstractBeanFactory.getBean(AbstractBeanFactory.java:145)
at org/springframework/beans/factory/support/DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:277)
at org/springframework/context/support/AbstractApplicationContext.refresh(AbstractApplicationContext.java:313)
at org/springframework/web/context/support/AbstractRefreshableWebApplicationContext.refresh(AbstractRefreshableWebApplicationContext.java:139)
at org/springframework/web/context/ContextLoader.createWebApplicationContext(ContextLoader.java:252)
at org/springframework/web/context/ContextLoader.initWebApplicationContext(ContextLoader.java:190)
at org/springframework/web/context/ContextLoaderListener.contextInitialized(ContextLoaderListener.java:49)
at weblogic/servlet/internal/EventsManager$FireContextListenerAction.run(EventsManager.java:481)
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)
at weblogic/servlet/internal/EventsManager.notifyContextCreatedEvent(EventsManager.java:181)
at weblogic/servlet/internal/WebAppServletContext.preloadResources(WebAppServletContext.java:1863)
^-- Holding lock: weblogic/servlet/internal/WebAppServletContext@0x7a2e8670[thin lock]
at weblogic/servlet/internal/WebAppServletContext.start(WebAppServletContext.java:3126)
at weblogic/servlet/internal/WebAppModule.startContexts(WebAppModule.java:1512)
at weblogic/servlet/internal/WebAppModule.start(WebAppModule.java:486)
at weblogic/application/internal/flow/ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic/application/utils/StateMachineDriver.nextState(StateMachineDriver.java:41)
at weblogic/application/internal/flow/ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic/application/internal/flow/ScopedModuleDriver.start(ScopedModuleDriver.java:200)
at weblogic/application/internal/flow/ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
at weblogic/application/internal/flow/ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic/application/utils/StateMachineDriver.nextState(StateMachineDriver.java:41)
at weblogic/application/internal/flow/ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic/application/internal/flow/StartModulesFlow.activate(StartModulesFlow.java:27)
at weblogic/application/internal/BaseDeployment$2.next(BaseDeployment.java:1267)
at weblogic/application/utils/StateMachineDriver.nextState(StateMachineDriver.java:41)
at weblogic/application/internal/BaseDeployment.activate(BaseDeployment.java:409)
at weblogic/application/internal/SingleModuleDeployment.activate(SingleModuleDeployment.java:43)
at weblogic/application/internal/DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic/deploy/internal/targetserver/AppContainerInvoker.activate(AppContainerInvoker.java:79)
at weblogic/deploy/internal/targetserver/operations/AbstractOperation.activate(AbstractOperation.java:569)
at weblogic/deploy/internal/targetserver/operations/ActivateOperation.activateDeployment(ActivateOperation.java:150)
at weblogic/deploy/internal/targetserver/operations/ActivateOperation.doCommit(ActivateOperation.java:116)
at weblogic/deploy/internal/targetserver/operations/StartOperation.doCommit(StartOperation.java:143)
at weblogic/deploy/internal/targetserver/operations/AbstractOperation.commit(AbstractOperation.java:323)
at weblogic/deploy/internal/targetserver/DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844)
at weblogic/deploy/internal/targetserver/DeploymentManager.activateDeploymentList(DeploymentManager.java:1253)
^-- Holding lock: java/util/ArrayList@0x7a032928[biased lock]
at weblogic/deploy/internal/targetserver/DeploymentManager.handleCommit(DeploymentManager.java:440)
at weblogic/deploy/internal/targetserver/DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163)
at weblogic/deploy/service/internal/targetserver/DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
at weblogic/deploy/service/internal/targetserver/DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
at weblogic/deploy/service/internal/targetserver/DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
at weblogic/work/SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:201)
at weblogic/work/ExecuteThread.run(ExecuteThread.java:173)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace -
Error while starting Managed Server - Oracle_SOA1 and WLS_Spaces
Hi,
I have installed Weblogic 10.3.3, Oracle Soa Suite 11.1.1.3 and Oracle Webcenter 11.1.1.3 32 bit on a RHEL5.5 64bit. The JRE used for all is Oracle JRockit(R) R28.0.0-679-130297-1.6.0_17-20100312-2128-linux-ia32.
The Weblogic Server starts up fine. But when starting the Managed Servers, I get the following errors in both
================================================================================
####<Feb 1, 2012 7:02:42 PM IST> <Error> <> <wlsstaging> <WLS_Spaces> <Thread-29> <<WLS Kernel>> <> <> <1328103162978> <BEA-000000> <
===== FULL THREAD DUMP ===============
Wed Feb 1 19:02:42 2012
Oracle JRockit(R) R28.0.0-679-130297-1.6.0_17-20100312-2128-linux-ia32
"Main Thread" id=1 idx=0x4 tid=1330 prio=5 alive, waiting, native_blocked
-- Waiting for notification on: java/lang/Object@0x9e3ace70[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/rjvm/ConnectionManager.bootstrap(ConnectionManager.java:320)
^-- Lock released while waiting: java/lang/Object@0x9e3ace70[fat lock]
at weblogic/rjvm/RJVMManager.findOrCreateRemoteInternal(RJVMManager.java:254)
^-- Holding lock: java/lang/Object@0xb09589c8[thin lock]
at weblogic/rjvm/RJVMManager.findOrCreate(RJVMManager.java:197)
at weblogic/rjvm/RJVMFinder.findOrCreateRemoteServer(RJVMFinder.java:238)
at weblogic/rjvm/RJVMFinder.findOrCreateInternal(RJVMFinder.java:200)
at weblogic/rjvm/RJVMFinder.findOrCreate(RJVMFinder.java:170)
^-- Holding lock: weblogic/rjvm/RJVMFinder@0x9e3ac210[biased lock]
at weblogic/rjvm/ServerURL.findOrCreateRJVM(ServerURL.java:153)
at weblogic/jndi/WLInitialContextFactoryDelegate.getInitialReference(WLInitialContextFactoryDelegate.java:401)
at weblogic/jndi/Environment.getInitialReference(Environment.java:245)
at weblogic/server/ServerLifeCycleRuntime.getLifeCycleOperationsRemote(ServerLifeCycleRuntime.java:1083)
at weblogic/t3/srvr/ServerRuntime.sendStateToAdminServer(ServerRuntime.java:431)
at weblogic/t3/srvr/ServerRuntime.updateRunState(ServerRuntime.java:417)
at weblogic/t3/srvr/T3Srvr.setState(T3Srvr.java:187)
^-- Holding lock: weblogic/t3/srvr/T3Srvr@0xb1184e38[recursive]
at weblogic/t3/srvr/T3Srvr.shutdown(T3Srvr.java:1038)
at weblogic/t3/srvr/T3Srvr.setFailedState(T3Srvr.java:321)
^-- Holding lock: weblogic/t3/srvr/T3Srvr@0xb1184e38[biased lock]
at weblogic/t3/srvr/T3Srvr.initializeAdmin(T3Srvr.java:833)
at weblogic/t3/srvr/T3Srvr.startup(T3Srvr.java:506)
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:446)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=1331 prio=5 alive, native_blocked, daemon
"(OC Main Thread)" id=3 idx=0xc tid=1332 prio=5 alive, native_waiting, daemon
"(Code Generation Thread 1)" id=4 idx=0x10 tid=1333 prio=5 alive, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x14 tid=1334 prio=5 alive, native_waiting, daemon
"(VM Periodic Task)" id=6 idx=0x18 tid=1335 prio=10 alive, native_blocked, daemon
"Finalizer" id=7 idx=0x1c tid=1336 prio=8 alive, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:189)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x20 tid=1337 prio=10 alive, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x24 tid=1338 prio=5 alive, native_blocked, daemon
"VM JFR Buffer Thread" id=10 idx=0x28 tid=1339 prio=5 alive, in native, daemon
"Timer-0" id=13 idx=0x2c tid=1340 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0xb10fee88[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0xb10fee88[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Timer-1" id=14 idx=0x30 tid=1408 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0xb10fef08[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0xb10fef08[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x34 tid=1409 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0xb0eac5e0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0xb0eac5e0[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"JFR request timer" id=16 idx=0x38 tid=1410 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0xb10fef88[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0xb10fef88[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"weblogic.time.TimeEventGenerator" id=18 idx=0x3c tid=1411 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/time/common/internal/TimeTable@0xb10fe310[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0xb10fe310[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"JMAPI event thread" id=19 idx=0x40 tid=1412 prio=5 alive, in native, daemon
"weblogic.timers.TimerThread" id=20 idx=0x44 tid=1413 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/timers/internal/TimerThread@0xb10ff030[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0xb10ff030[fat lock]
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" id=21 idx=0x48 tid=1414 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0xb0eac6b8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0xb0eac6b8[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" id=24 idx=0x4c tid=1415 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0xb0eac790[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0xb0eac790[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=25 idx=0x50 tid=1416 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0xb11009d0[thin lock]
at jrockit/vm/Threads.sleep(I)V(Native Method)
at jrockit/vm/Locks.waitForThinRelease(Locks.java:946)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1074)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:996)
at jrockit/vm/Locks.monitorEnter(Locks.java:2170)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=26 idx=0x54 tid=1417 prio=5 alive, in native, daemon
at jrockit/ext/epoll/EPoll.epollWait0(ILjava/nio/ByteBuffer;II)I(Native Method)
at jrockit/ext/epoll/EPoll.epollWait(EPoll.java:115)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:156)
^-- Holding lock: java/lang/String@0xb11009d0[thin lock]
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"VDE Transaction Processor Thread" id=29 idx=0x58 tid=1424 prio=2 alive, waiting, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0xb102c588[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:367)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0xb102c588[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"DoSManager" id=32 idx=0x60 tid=1426 prio=6 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at com/octetstring/vde/DoSManager.run(DoSManager.java:433)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" id=33 idx=0x64 tid=1429 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0xb0eac868[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:157)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0xb0eac868[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:178)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-12" id=34 idx=0x68 tid=1435 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0xacc072a8
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:650)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:707)
at weblogic/store/internal/PersistentStoreImpl$2.run(PersistentStoreImpl.java:464)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"JPS FileChangeNotifier<clean-only>" id=35 idx=0x6c tid=1436 prio=5 alive, parked, native_blocked, daemon
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:198)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
at java/util/concurrent/DelayQueue.take(DelayQueue.java:164)
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"LogFlusher" id=36 idx=0x70 tid=1437 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"LogFlusher" id=37 idx=0x74 tid=1438 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"LogFlusher" id=38 idx=0x78 tid=1439 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"oracle.dfw.impl.incident - ADR Executor (created: Wed Feb 01 18:54:55 IST 2012)" id=42 idx=0x88 tid=1446 prio=5 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0x982f9e48
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2221)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:158)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"oracle.dfw.incident.IncidentCreatorThread" id=45 idx=0x90 tid=1449 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/ArrayList@0x982d0530[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at oracle/dfw/impl/incident/DiagnosticsDataExtractorImpl$IncidentCreatorThread.run(DiagnosticsDataExtractorImpl.java:1597)
^-- Lock released while waiting: java/util/ArrayList@0x982d0530[fat lock]
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"oracle.dfw.incident.FloodController" id=46 idx=0x94 tid=1450 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/lang/Object@0x97b491a0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at oracle/dfw/impl/incident/FloodController.run(FloodController.java:136)
^-- Lock released while waiting: java/lang/Object@0x97b491a0[fat lock]
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"LogFlusher" id=48 idx=0xa0 tid=1452 prio=5 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/core/ojdl/BufferedLogWriter$Flusher.run(BufferedLogWriter.java:409)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"JMX Framework document pooling thread" id=52 idx=0xb0 tid=1457 prio=4 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at oracle/as/config/notification/filesystem/WatchingDocumentChangeNotifier.run(WatchingDocumentChangeNotifier.java:200)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"Thread-29" id=56 idx=0xc0 tid=1538 prio=5 alive, daemon
at bea/jmapi/DiagnosticCommandImpl.execute(Ljava/lang/String;Ljava/io/FileDescriptor;)V(Native Method)
at bea/jmapi/DiagnosticCommandImpl.execute(DiagnosticCommandImpl.java:53)
at com/bea/jvm/DiagnosticCommand.execute(DiagnosticCommand.java:245)
at com/bea/jvm/DiagnosticCommand$Command.execute(DiagnosticCommand.java:404)
at bea/jmapi/ThreadSystemImpl.getThreadStackDump(ThreadSystemImpl.java:102)
at weblogic/platform/JRockitVM.threadDump(JRockitVM.java:61)
at weblogic/t3/srvr/T3Srvr.logThreadDump(T3Srvr.java:280)
at weblogic/t3/srvr/ServerLifeCycleTimerThread.run(ServerLifeCycleTimerThread.java:77)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
Open lock chains
================
Chain 1:
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=25 idx=0x50 tid=1416 waiting for java/lang/String@0xb11009d0 held by:
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=26 idx=0x54 tid=1417 (active)
===== END OF THREAD DUMP ===============
Please help.
Thank you.
Regards.
Debesh.The default open file limit in Linux is 1024 while WebCenter installer requires the open file limit to be 4096 or more.
To change the open file limits, login as root and edit the /etc/security/limits.conf file. Look for the following two lines:
* soft nofile 2048
* hard nofile 2048
If these lines do not exist, add them.
Change the values from 2048 to 4096 or higher, then reboot the machine. -
Managed Server stuck in starting
Hi every, thank you for reading.
We have a Oracle SOA domain running on WLS 10.3.6. The OS is windows server 2003.
Adminserver and soa_server1 are on vmmachine1.
proxy server and soa_server2 are on vmmachine2.
soa_server1 and soa_server2 are in a cluster.
This domain has been working normally for serveral months. Managed server is controlled by nodemanager.
Yesterday, we increased the memory of vmmachine1 and vmmachine2 from 8g to 16g.
After that, soa_server2 cannot be started while other managed server of the domain starts normally.
I looked into the soa_server2.log and found that the log stucks at "<Notice> <Log Management> <BEA-170019> <The server log file C:\Oracle\Middleware\user_projects\domains\ctst_soa_domain\servers\soa_server2\logs\soa_server2.log is opened. All server side log events will be written to this file.> " forever.
The task manager shows that the memory of java.exe process of the stucked managed server is about 350m and at an increase rate of about 4k.
The cpu consumption of the process is 0 percent.
Below is part of JRockit Mission Control fly record event of the stucked JVM:
1,359,447,962,480,625,883 4,453,053,500 weblogic.timers.TimerThread Java Wait
1,359,447,962,480,688,068 4,453,037,334 [ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)' Java Wait
1,359,447,962,558,579,199 109,474,384 VDE Transaction Processor Thread Java Wait
1,359,447,962,564,083,662 869,606,634 RMI TCP Connection(26)-192.9.200.148 Socket Read
1,359,447,962,668,068,530 109,075,316 VDE Transaction Processor Thread Java Wait
1,359,447,962,777,159,184 109,367,902 VDE Transaction Processor Thread Java Wait
1,359,447,962,886,542,218 109,638,015 VDE Transaction Processor Thread Java Wait
1,359,447,962,996,257,842 109,285,831 VDE Transaction Processor Thread Java Wait
1,359,447,963,105,558,278 109,143,508 VDE Transaction Processor Thread Java Wait
1,359,447,963,214,709,536 109,312,043 VDE Transaction Processor Thread Java Wait
1,359,447,963,324,052,457 109,455,297 VDE Transaction Processor Thread Java Wait
1,359,447,963,433,516,028 109,259,502 VDE Transaction Processor Thread Java Wait
1,359,447,963,433,924,695 93,414,304 RMI TCP Connection(26)-192.9.200.148 Socket Read
1,359,447,963,527,635,047 2,515,422,874 RMI TCP Connection(26)-192.9.200.148 Socket Read
1,359,447,963,542,789,729 152,068,282 VDE Transaction Processor Thread Java Wait
1,359,447,963,694,879,147 113,581,233 VDE Transaction Processor Thread Java Wait
1,359,447,963,697,060,581 1,002,528,122 RMI TCP Connection(idle) Socket Read
1,359,447,963,808,488,263 109,555,192 VDE Transaction Processor Thread Java Wait
1,359,447,963,918,076,105 109,076,613 VDE Transaction Processor Thread Java Wait
1,359,447,964,027,167,650 109,358,282 VDE Transaction Processor Thread Java Wait
1,359,447,964,136,541,541 109,442,748 VDE Transaction Processor Thread Java Wait
1,359,447,964,245,999,192 109,830,184 VDE Transaction Processor Thread Java Wait
1,359,447,964,324,342,462 2,015,390,696 Timer-1 Java Wait
1,359,447,964,355,844,714 108,827,865 VDE Transaction Processor Thread Java Wait
1,359,447,964,464,687,365 109,297,368 VDE Transaction Processor Thread Java Wait
1,359,447,964,574,003,355 109,396,656 VDE Transaction Processor Thread Java Wait
1,359,447,964,683,418,068 109,366,571 VDE Transaction Processor Thread Java Wait
1,359,447,964,705,600,116 743,862,091 RMI TCP Connection(idle) Socket Read
1,359,447,964,792,799,741 109,347,456 VDE Transaction Processor Thread Java Wait
1,359,447,964,902,156,236 109,395,625 VDE Transaction Processor Thread Java Wait
1,359,447,965,011,567,244 109,331,778 VDE Transaction Processor Thread Java Wait
1,359,447,965,120,914,342 109,367,906 VDE Transaction Processor Thread Java Wait
1,359,447,965,230,297,996 109,602,297 VDE Transaction Processor Thread Java Wait
1,359,447,965,339,915,098 109,369,203 VDE Transaction Processor Thread Java Wait
1,359,447,965,449,297,323 109,126,121 VDE Transaction Processor Thread Java Wait
1,359,447,965,449,946,395 265,099,402 RMI TCP Connection(idle) Socket Read
1,359,447,965,558,468,725 109,441,305 VDE Transaction Processor Thread Java Wait
1,359,447,965,667,925,444 109,229,624 VDE Transaction Processor Thread Java Wait
1,359,447,965,720,533,365 322,363,229 RMI TCP Connection(idle) Socket Read
1,359,447,965,777,169,428 109,580,914 VDE Transaction Processor Thread Java Wait
1,359,447,965,886,766,019 109,140,868 VDE Transaction Processor Thread Java Wait
Any help is welcome.Thread Dump:
"Main Thread" id=1 idx=0x4 tid=1728 prio=5 alive, waiting, native_blocked
-- Waiting for notification on: weblogic/t3/srvr/SubsystemRequest@0x00000001
82879510[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/t3/srvr/SubsystemRequest.rendezvouz(SubsystemRequest.java:100)
^-- Lock released while waiting: weblogic/t3/srvr/SubsystemRequest@0x0000000
182879510[fat lock]
at weblogic/t3/srvr/SubsystemRequest.action(SubsystemRequest.java:81)
at weblogic/t3/srvr/SubsystemRequest.start(SubsystemRequest.java:51)
at weblogic/t3/srvr/ServerServicesManager.startService(ServerServicesManager
.java:459)
at weblogic/t3/srvr/ServerServicesManager.startInStandbyState(ServerServices
Manager.java:166)
^-- Holding lock: java/lang/Class@0x0000000182852658[biased lock]
at weblogic/t3/srvr/T3Srvr.initializeStandby(T3Srvr.java:881)
at weblogic/t3/srvr/T3Srvr.startup(T3Srvr.java:568)
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:469)
at weblogic/Server.main(Server.java:71)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=4040 prio=5 alive, daemon
"(OC Main Thread)" id=3 idx=0xc tid=2300 prio=5 alive, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=220 prio=5 alive, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=2056 prio=5 alive, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=1096 prio=5 alive, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=2580 prio=5 alive, daemon
"(Code Generation Thread 1)" id=4 idx=0x20 tid=3404 prio=5 alive, native_waiting
, daemon
"(Code Optimization Thread 1)" id=5 idx=0x24 tid=3228 prio=5 alive, native_waiti
ng, daemon
"(VM Periodic Task)" id=6 idx=0x28 tid=1564 prio=10 alive, native_blocked, daemo
n
"(Attach Listener)" id=7 idx=0x2c tid=640 prio=5 alive, native_blocked, daemon
"Finalizer" id=8 idx=0x30 tid=248 prio=8 alive, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native
Method)
at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:183)
at java/lang/Thread.run(Thread.java:662)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Reference Handler" id=9 idx=0x34 tid=2092 prio=10 alive, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;
(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=10 idx=0x38 tid=3548 prio=5 alive, native_blocked, da
emon
"VM JFR Buffer Thread" id=11 idx=0x3c tid=3768 prio=5 alive, in native, daemon
"Timer-0" id=14 idx=0x40 tid=3800 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x000000018238DF88[fat l
ock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0x000000018238DF88[fat
lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Timer-1" id=15 idx=0x44 tid=1572 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0x0000000185AC9490[fat l
ock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at java/util/TimerThread.mainLoop(Timer.java:509)
^-- Lock released while waiting: java/util/TaskQueue@0x0000000185AC9490[fat
lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'"
id=16 idx=0x48 tid=3360 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/Transac
tionProcessor@0x000000018A83F800[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitTransaction
sCompleted(TransactionProcessor.java:353)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/Transa
ctionProcessor@0x000000018A83F800[fat lock]
at com/octetstring/vde/backend/standard/BackendStandard.<init>(BackendStanda
rd.java:275)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
at jrockit/vm/Reflect.invokeMethod(Ljava/lang/Object;Ljava/lang/Object;[Ljav
a/lang/Object;)Ljava/lang/Object;(Native Method)
at sun/reflect/NativeConstructorAccessorImpl.newInstance0(Ljava/lang/reflect
/Constructor;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
at sun/reflect/NativeConstructorAccessorImpl.newInstance(NativeConstructorAc
cessorImpl.java:39)
at sun/reflect/DelegatingConstructorAccessorImpl.newInstance(DelegatingConst
ructorAccessorImpl.java:27)
at java/lang/reflect/Constructor.newInstance(Constructor.java:513)
at com/octetstring/vde/backend/BackendHandler.<init>(BackendHandler.java:248
at com/octetstring/vde/backend/BackendHandler.getInstance(BackendHandler.jav
a:344)
at weblogic/ldap/EmbeddedLDAP.start(EmbeddedLDAP.java:299)
at weblogic/t3/srvr/SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:256)
at weblogic/work/ExecuteThread.run(ExecuteThread.java:221)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JFR request timer" id=17 idx=0x4c tid=2260 prio=5 alive, waiting, native_blocke
d, daemon
-- Waiting for notification on: java/util/TaskQueue@0x0000000185B49530[fat l
ock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0x0000000185B49530[fat
lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.time.TimeEventGenerator" id=19 idx=0x50 tid=600 prio=9 alive, waiting,
native_blocked, daemon
-- Waiting for notification on: weblogic/time/common/internal/TimeTable@0x00
0000018C653048[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0x0
00000018C653048[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.j
ava:117)
at java/lang/Thread.run(Thread.java:662)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JMAPI event thread" id=20 idx=0x54 tid=2916 prio=5 alive, in native, daemon
"weblogic.timers.TimerThread" id=21 idx=0x58 tid=3288 prio=9 alive, waiting, nat
ive_blocked, daemon
-- Waiting for notification on: weblogic/timers/internal/TimerThread@0x00000
0018C6ABCA0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0x0000
00018C6ABCA0[fat lock]
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'
" id=22 idx=0x5c tid=1368 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/work/ExecuteThread@0x000000018C2A1C
00[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:205)
^-- Lock released while waiting: weblogic/work/ExecuteThread@0x000000018C2A1
C00[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:226)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=25 idx=0x60 tid=1484
prio=5 alive, in native, daemon
at weblogic/socket/NTSocketMuxer.getIoCompletionResult(Lweblogic/socket/NTSo
cketMuxer$IoCompletionData;)Z(Native Method)
at weblogic/socket/NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=26 idx=0x64 tid=2540
prio=5 alive, in native, daemon
at weblogic/socket/NTSocketMuxer.getIoCompletionResult(Lweblogic/socket/NTSo
cketMuxer$IoCompletionData;)Z(Native Method)
at weblogic/socket/NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=27 idx=0x68 tid=3640
prio=5 alive, in native, daemon
at weblogic/socket/NTSocketMuxer.getIoCompletionResult(Lweblogic/socket/NTSo
cketMuxer$IoCompletionData;)Z(Native Method)
at weblogic/socket/NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" id=28 idx=0x6c tid=1904
prio=5 alive, in native, daemon
at weblogic/socket/NTSocketMuxer.getIoCompletionResult(Lweblogic/socket/NTSo
cketMuxer$IoCompletionData;)Z(Native Method)
at weblogic/socket/NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"VDE Transaction Processor Thread" id=31 idx=0x70 tid=1520 prio=2 alive, waiting
, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/Transac
tionProcessor@0x000000018A83F800[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Metho
d)
at java/lang/Object.wait(J)V(Native Method)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitwrite(Trans
actionProcessor.java:378)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/Transa
ctionProcessor@0x000000018A83F800[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(Transaction
Processor.java:228)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
===== END OF THREAD DUMP =============== -
JRockit 1.4.2_04....is this a locking situation ?
We migrated recently to JRockit VM with Weblogic 7 on multi-cpu mahcine running Linux. After multiple simultaneous calls to a bean, we see that the managed server stops responding ( no OutOfMemory errors - just no response for any http calls ). I was wondering if the following thread dump shows any locking issue ( although there doesnt seem to be any deadlocks )...
===== FULL THREAD DUMP ===============
Thu Mar 17 09:01:00 2005
"Main Thread" prio=5 id=0x80 pid=22480 waiting
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x405b25d8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:1083)@0x2420c414
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x405b25d8[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:318)@0x21b4551d
at weblogic/Server.main(Server.java:32)@0x21b44e52
--- End of stack trace
"(Signal Handler)" prio=5 id=0x100 pid=22487 active, daemon
"(Code Generation Thread 1)" prio=5 id=0x180 pid=22488 active, native_waiting, daemon
"(Code Optimization Thread 1)" prio=5 id=0x200 pid=22489 active, native_waiting, daemon
"(GC Main Thread)" prio=5 id=0x280 pid=22490 active, daemon
"(GC Worker Thread 1)" prio=5 id=0x300 pid=22491 active, native_waiting, daemon
"(GC Worker Thread 2)" prio=5 id=0x380 pid=22492 active, native_waiting, daemon
"Finalizer" prio=8 id=0x400 pid=22493 waiting, daemon
-- Waiting for notification on: java/lang/ref/ReferenceQueue$Lock@0x404cfed8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at jrockit/vm/Locks.wait(Unknown Source)@0x21af6b85
at java/lang/ref/ReferenceQueue.remove(ReferenceQueue.java:111)@0x21af9511
^-- Lock released while waiting: java/lang/ref/ReferenceQueue$Lock@0x404cfed8[fat lock]
at java/lang/ref/ReferenceQueue.remove(ReferenceQueue.java:127)@0x21af93ec
at java/lang/ref/Finalizer$FinalizerThread.run(Unknown Source)@0x21af939e
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"Reference Handler" prio=10 id=0x480 pid=22494 waiting for pending references, daemon
at java/lang/ref/Reference.getPending(Native Method)@0x21b146f0
at java/lang/ref/Reference.access$000(Unknown Source)@0x21b14765
at java/lang/ref/Reference$ReferenceHandler.run(Unknown Source)@0x21b14670
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"[JRockit] Connection Listener" prio=5 id=0x500 pid=22496 active, daemon
at jrockit/net/SocketNativeIO.accept(Native Method)@0x22415850
at jrockit/net/SocketNativeIO.accept(Unknown Source)@0x2241590b
at java/net/AbstractSocketImpl.accept(Unknown Source)@0x22415755
^-- Holding lock: java/net/PlainSocketImpl@0x404ed1f0[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:448)@0x22415665
at java/net/ServerSocket.accept(ServerSocket.java:419)@0x2241555c
at com/jrockit/management/rmp/RmpSocketListener.run(RmpSocketListener.java:170)@0x21b44aea
at java/lang/Thread.run(Unknown Source)@0x2241550f
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"(Hotspot Detector)" prio=10 id=0x580 pid=22497 active, daemon
"Thread-1" prio=10 id=0x600 pid=22498 waiting, daemon
-- Waiting for notification on: java/lang/Object@0x40dd8030[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0x21af6a30
at jrockit/vm/Locks.wait(Unknown Source)@0x21af6c19
at java/lang/Thread.sleep(Unknown Source)@0x21be0615
^-- Lock released while waiting: java/lang/Object@0x40dd8030[fat lock]
at weblogic/transaction/internal/TransactionManagerImpl$1.run(TransactionManagerImpl.java:1660)@0x2267a1ac
at java/lang/Thread.run(Unknown Source)@0x2241550f
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '0' for queue: 'default'" prio=5 id=0x680 pid=22499 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40dddca0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40dddca0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '1' for queue: 'default'" prio=5 id=0x700 pid=22500 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40dde368[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40dde368[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '2' for queue: 'default'" prio=5 id=0x780 pid=22501 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40dde848[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40dde848[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '3' for queue: 'default'" prio=5 id=0x800 pid=22502 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40dded28[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40dded28[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '4' for queue: 'default'" prio=5 id=0x880 pid=22503 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40ddf208[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40ddf208[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '5' for queue: 'default'" prio=5 id=0x900 pid=22504 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40ddf6e8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40ddf6e8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '6' for queue: 'default'" prio=5 id=0x980 pid=22505 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40ddfbc8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40ddfbc8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '7' for queue: 'default'" prio=5 id=0xa00 pid=22506 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40de00a8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40de00a8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '8' for queue: 'default'" prio=5 id=0xa80 pid=22507 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40de0588[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40de0588[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '9' for queue: 'default'" prio=5 id=0xb00 pid=22508 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40de0a68[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40de0a68[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '10' for queue: 'default'" prio=5 id=0xb80 pid=22509 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40de0f48[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40de0f48[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '11' for queue: 'default'" prio=5 id=0xc00 pid=22510 waiting, daemon
-- Waiting for notification on: weblogic/kernel/ExecuteThread@0x40de1428[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.wait(Unknown Source)@0x21af96d5
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:146)@0x2267c946
^-- Lock released while waiting: weblogic/kernel/ExecuteThread@0x40de1428[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:172)@0x2267c669
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '12' for queue: 'default'" prio=5 id=0xc80 pid=22511 active, daemon
at weblogic/socket/PosixSocketMuxer.poll(Native Method)@0x2288c800
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:621)@0x2288c534
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:23)@0x2288c483
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:213)@0x2288c451
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:189)@0x2267c697
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '13' for queue: 'default'" prio=5 id=0xd00 pid=22512 blocked, daemon
-- Blocked trying to get lock: java/lang/Object@0x41140860[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.raceForTheFatLock(Unknown Source)@0x21af50f8
at jrockit/vm/Locks.lockFat(Optimized Method)@0x24afd67d
at jrockit/vm/Locks.monitorEnterSlowCase(Optimized Method)@0x24affd5b
at jrockit/vm/Locks.monitorEnter(Native Method)@0x21af3f24
at jrockit/vm/Locks.monitorEnterForced(Unknown Source)@0x21bd7fd5
at jrockit/vm/Locks.monitorEnterUnmatched(Native Method)@0x21af3f30
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:617)@0x2288c51c
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:23)@0x2288c483
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:213)@0x2288c451
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:189)@0x2267c697
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
"ExecuteThread: '14' for queue: 'default'" prio=5 id=0xd80 pid=22513 blocked, daemon
-- Blocked trying to get lock: java/lang/Object@0x41140860[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0x21af5020
at jrockit/vm/Locks.lockFat(Optimized Method)@0x24afd66e
at jrockit/vm/Locks.monitorEnterSlowCase(Optimized Method)@0x24affd5b
at jrockit/vm/Locks.monitorEnter(Native Method)@0x21af3f24
at jrockit/vm/Locks.monitorEnterForced(Unknown Source)@0x21bd7fd5
at jrockit/vm/Locks.monitorEnterUnmatched(Native Method)@0x21af3f30
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:617)@0x2288c51c
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:23)@0x2288c483
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:213)@0x2288c451
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:189)@0x2267c697
at java/lang/Thread.startThreadFromVM(Unknown Source)@0x21af92f3
--- End of stack trace
Blocked lock chains
===================
Chain 2:
"ExecuteThread: '14' for queue: 'default'" (id: 0x00000d80) waiting for java/lang/Object@0x41140860 held by:
"ExecuteThread: '12' for queue: 'default'" (id: 0x00000c80) in chain 1
Open lock chains
================
Chain 1:
"ExecuteThread: '13' for queue: 'default'" (id: 0x00000d00) waiting for java/lang/Object@0x41140860 held by:
"ExecuteThread: '12' for queue: 'default'" (id: 0x00000c80) (active)
===== END OF THREAD DUMP ===============JRockit usually detects and prints out deadlocks at the end of the thread stack dump. Since you most likely have a support contract, I suggest you open a support case so we can take a look at this.
Thanks,
Henrik, JRockit team -
Weblogic reaches 100% CPU utilisation
I am using WebLogic Server Version 10.3.1.0. I got an application deployed in this server, which is developed with java1.6, ejb, hibernate. I start the server with 8GB of RAM assigned to it and after a day or two the weblogic server causes the use of the CPU to reach to 100% and as a result the server sort of doesn't response anymore. I will be creating 30-40 thread per managed server and one server has two managed server. The volume will be around 30k per day. I dont know which is causing the 100% cpu utilisation and here below are the thread dumps which have taken at the time of high cpu in a server. Please let me know where is the exact issue and where is the stuck thread (if any)? In which part the issue is occuring?
10774:
===== FULL THREAD DUMP ===============
Thu Jun 19 16:57:39 2014
Oracle JRockit(R) R28.2.8-10-156881-1.6.0_51-20130611-1146-linux-x86_64
"Main Thread" id=1 idx=0x4 tid=10776 prio=5 alive, waiting, native_blocked
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0xa4db4c58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:981)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0xa4db4c58[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:490)
at weblogic/Server.main(Server.java:71)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=10777 prio=5 alive, native_blocked, daemon
"(OC Main Thread)" id=3 idx=0xc tid=10778 prio=5 alive, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=10779 prio=5 alive, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=10780 prio=5 alive, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=10781 prio=5 alive, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=10782 prio=5 alive, daemon
"(Code Generation Thread 1)" id=4 idx=0x20 tid=10783 prio=5 alive, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x24 tid=10784 prio=5 alive, native_waiting, daemon
"(VM Periodic Task)" id=6 idx=0x28 tid=10785 prio=10 alive, native_waiting, daemon
"Finalizer" id=7 idx=0x2c tid=10786 prio=8 alive, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:201)
at java/lang/Thread.run(Thread.java:662)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Reference Handler" id=8 idx=0x30 tid=10787 prio=10 alive, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"(Sensor Event Thread)" id=9 idx=0x34 tid=10788 prio=5 alive, native_blocked, daemon
"VM JFR Buffer Thread" id=10 idx=0x38 tid=10789 prio=5 alive, in native, daemon
"Timer-0" id=13 idx=0x3c tid=10793 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0xa4d5d260[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[optimized]
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0xa4d5d260[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'default'" id=14 idx=0x40 tid=10839 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'default'" id=15 idx=0x44 tid=10840 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/LinkedList@0xa8cbed08[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/ejb/container/internal/MDListener.getListener(MDListener.java:240)[optimized]
^-- Lock released while waiting: java/util/LinkedList@0xa8cbed08[fat lock]
at weblogic/ejb/container/internal/MDListener.transactionalOnMessage(MDListener.java:480)[inlined]
at weblogic/ejb/container/internal/MDListener.onMessage(MDListener.java:379)[optimized]
at weblogic/jms/client/JMSSession.onMessage(JMSSession.java:4659)[optimized]
at weblogic/jms/client/JMSSession.execute(JMSSession.java:4345)[optimized]
at weblogic/jms/client/JMSSession.executeMessage(JMSSession.java:3821)[optimized]
at weblogic/jms/client/JMSSession.access$000(JMSSession.java:115)
at weblogic/jms/client/JMSSession$UseForRunnable.run(JMSSession.java:5170)
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'default'" id=16 idx=0x48 tid=10841 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d5f0d0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d5f0d0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'default'" id=17 idx=0x4c tid=10842 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'default'" id=18 idx=0x50 tid=10843 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d5fe50[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d5fe50[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '5' for queue: 'default'" id=19 idx=0x54 tid=10844 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '6' for queue: 'default'" id=20 idx=0x58 tid=10845 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '7' for queue: 'default'" id=21 idx=0x5c tid=10846 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '8' for queue: 'default'" id=22 idx=0x60 tid=10847 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '9' for queue: 'default'" id=23 idx=0x64 tid=10848 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d62010[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d62010[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '10' for queue: 'default'" id=24 idx=0x68 tid=10849 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d626d0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d626d0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '11' for queue: 'default'" id=25 idx=0x6c tid=10850 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '12' for queue: 'default'" id=26 idx=0x70 tid=10851 prio=5 alive, native_blocked, daemon
at java/util/HashMap.get(HashMap.java:385)[optimized]
at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '13' for queue: 'default'" id=27 idx=0x74 tid=10852 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d648e8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d648e8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '14' for queue: 'default'" id=28 idx=0x78 tid=10853 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d65568[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d65568[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.kernel.Non-Blocking'" id=29 idx=0x7c tid=10854 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d65cc0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d65cc0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.kernel.Non-Blocking'" id=30 idx=0x80 tid=10855 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d663c0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d663c0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.kernel.Non-Blocking'" id=31 idx=0x84 tid=10856 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d66ac0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d66ac0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.kernel.System'" id=32 idx=0x88 tid=10857 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d671c0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d671c0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.kernel.System'" id=33 idx=0x8c tid=10858 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d67f48[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d67f48[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.kernel.System'" id=34 idx=0x90 tid=10859 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d68cc0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d68cc0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.kernel.System'" id=35 idx=0x94 tid=10860 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d69a38[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d69a38[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'weblogic.kernel.System'" id=36 idx=0x98 tid=10861 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d977b8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d977b8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.Rejector'" id=37 idx=0x9c tid=10862 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d99570[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d99570[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.Rejector'" id=38 idx=0xa0 tid=10863 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf2e38[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf2e38[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"weblogic.time.TimeEventGenerator" id=39 idx=0xa4 tid=10864 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/time/common/internal/TimeTable@0xa4dbed18[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0xa4dbed18[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
at java/lang/Thread.run(Thread.java:662)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JMAPI event thread" id=40 idx=0xa8 tid=10865 prio=5 alive, in native, daemon
"weblogic.timers.TimerThread" id=41 idx=0xac tid=10866 prio=9 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/timers/internal/TimerThread@0xa4dc3f70[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0xa4dc3f70[fat lock]
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" id=42 idx=0xb0 tid=10867 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf3528[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf3528[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" id=43 idx=0xb4 tid=10868 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf35f0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf35f0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" id=44 idx=0xb8 tid=10869 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf36b8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf36b8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.admin.RMI'" id=45 idx=0xbc tid=10870 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf3780[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf3780[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'weblogic.admin.RMI'" id=46 idx=0xc0 tid=10871 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf3848[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf3848[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'ImageWorkManager'" id=47 idx=0xc4 tid=10896 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf57e0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf57e0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'ImageWorkManager'" id=48 idx=0xc8 tid=10899 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf58a8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf58a8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"JFR request timer" id=49 idx=0xcc tid=10900 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/util/TaskQueue@0xa4dc69d8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at java/util/TimerThread.mainLoop(Timer.java:483)
^-- Lock released while waiting: java/util/TaskQueue@0xa4dc69d8[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.logging.DomainLogBroadcasterClient'" id=53 idx=0xd0 tid=10902 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf6550[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf6550[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.logging.LogBroadcaster'" id=54 idx=0xd4 tid=10903 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf6d50[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf6d50[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=55 idx=0xd8 tid=10904 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0xa4cf78f8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1512)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)[optimized]
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=56 idx=0xdc tid=10905 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0xa4cf78f8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1512)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)[optimized]
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=57 idx=0xe0 tid=10906 prio=5 alive, blocked, native_blocked, daemon
-- Blocked trying to get lock: java/lang/String@0xa4cf78f8[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[inlined]
at jrockit/vm/Locks.fatLockReacquire(Locks.java:1649)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1517)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2179)[optimized]
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" id=58 idx=0xe4 tid=10907 prio=5 alive, in native, daemon
at jrockit/ext/epoll/EPoll.epollWait0(ILjava/nio/ByteBuffer;II)I(Native Method)
at jrockit/ext/epoll/EPoll.epollWait(EPoll.java:123)
at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:156)
^-- Holding lock: java/lang/String@0xa4cf78f8[fat lock]
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"VDE Transaction Processor Thread" id=61 idx=0xe8 tid=10909 prio=2 alive, waiting, native_blocked, daemon
-- Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0xa46c3738[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:367)
^-- Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0xa46c3738[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"DoSManager" id=64 idx=0xf0 tid=10911 prio=6 alive, sleeping, native_waiting, daemon
at java/lang/Thread.sleep(J)V(Native Method)
at com/octetstring/vde/DoSManager.run(DoSManager.java:433)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'UserLockout'" id=65 idx=0xf4 tid=10920 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf93c8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf93c8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'UserLockout'" id=66 idx=0xf8 tid=10921 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf9490[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf9490[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'UserLockout'" id=67 idx=0xfc tid=10922 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf9558[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf9558[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'UserLockout'" id=68 idx=0x100 tid=10923 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf9620[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf9620[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'UserLockout'" id=69 idx=0x104 tid=10924 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf96e8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf96e8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.unicast.DispatchWorkManager'" id=70 idx=0x108 tid=10958 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbbb0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbbb0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.unicast.DispatchWorkManager'" id=71 idx=0x10c tid=10959 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbc78[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbc78[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.unicast.DispatchWorkManager'" id=72 idx=0x110 tid=10960 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbd40[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbd40[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.unicast.DispatchWorkManager'" id=73 idx=0x114 tid=10961 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbe08[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbe08[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'weblogic.unicast.DispatchWorkManager'" id=74 idx=0x118 tid=10962 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbed0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbed0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=75 idx=0x11c tid=10963 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c66f40[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c66f40[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=76 idx=0x120 tid=10964 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c67008[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c67008[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=77 idx=0x124 tid=10965 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c670d0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c670d0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=78 idx=0x128 tid=10966 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c67198[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c67198[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=79 idx=0x12c tid=10967 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c67260[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c67260[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'ClusterMessaging'" id=81 idx=0x134 tid=10969 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c692e0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c692e0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'wl_oldBootStrap'" id=82 idx=0x138 tid=10972 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69a90[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69a90[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'wl_oldBootStrap'" id=83 idx=0x13c tid=10973 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69b58[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69b58[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'wl_oldBootStrap'" id=84 idx=0x140 tid=10974 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69c20[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69c20[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'wl_oldBootStrap'" id=85 idx=0x144 tid=10975 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69ce8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69ce8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'wl_oldBootStrap'" id=86 idx=0x148 tid=10980 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69db0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69db0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '5' for queue: 'wl_oldBootStrap'" id=87 idx=0x14c tid=10984 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69e78[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69e78[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '6' for queue: 'wl_oldBootStrap'" id=88 idx=0x150 tid=10987 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69f40[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69f40[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '7' for queue: 'wl_oldBootStrap'" id=89 idx=0x154 tid=10988 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6a008[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6a008[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"Thread-12" id=90 idx=0x158 tid=10989 prio=5 alive, parked, native_blocked
-- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0xa4c66b48
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:156)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:672)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:721)
at java/lang/Thread.run(Thread.java:662)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'JTACoordinatorWM'" id=91 idx=0x15c tid=10990 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6d128[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6d128[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'JTACoordinatorWM'" id=92 idx=0x160 tid=10991 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6d1f0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6d1f0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'JTACoordinatorWM'" id=93 idx=0x164 tid=10992 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6d2b8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:485)[inlined]
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6d2b8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '0' for queue: 'JmsDispatcher'" id=94 idx=0x168 tid=10995 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e688[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e688[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '1' for queue: 'JmsDispatcher'" id=95 idx=0x16c tid=10996 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e750[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e750[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '2' for queue: 'JmsDispatcher'" id=96 idx=0x170 tid=10997 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e818[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e818[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '3' for queue: 'JmsDispatcher'" id=97 idx=0x174 tid=10998 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e8e0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e8e0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"ExecuteThread: '4' for queue: 'JmsDispatcher'" id=98 idx=0x178 tid=10999 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e9a8[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
Hi,
Based on the below doc , you can narrow down the actual Thread that is causing this high CPU.
Unexpected High CPU Usage with WebLogic Server (WLS) Support Pattern (Doc ID 779349.1)
Thanks,
Sharmela
Maybe you are looking for
-
Problem with multiple same emails - thinking of clean reinstall
Hi. My OS is windows vista home edition and I am using Thunderbird version 31.0. My email provider is MSN. When thunderbird checks mail, Instead of receiving one copy of an email, I am sometimes getting two and even three copies of the same email. Th
-
why is there no search option on the app store? i just don't always know the category for the app type i'm looking for. this is nuts to have no search. or maybe i'm nuts not to find it? even that would be okay as long as it's there!
-
can i connect the play station 3 on Apple cinema display 27" (year 2011)? Please help me :-(
-
What is the Forms 10.1.2.3 download site?
Hi all: I want to install forms10g in Vista, and then oracle.com said forms version must is 10.1.2.3. But in the Software Downloads , Oracle Developer Suite 10g last version is 10.1.2.0.2. Can anyone tell me what is the Forms 10.1.2.3 download site?
-
PLM table for Internal Order budget, Actual Cost and Actual Hrs
Dear all In cProjects version 4.0, we have integration with existing ECC 6.0. We create Internal Order manually and assign it to cProjects. One cProject having one Internal Order. Resource Planning part is managed in cProjects while Actual Time keep