Managed server start very slowly

Question

Hi All,
We are currently facing issues in starting a managed weblogic server.
We have 2 machines, say MAC1 and MAC2.
MAC1 contains the admin server and 5 managed servers.
MAC2 contains only 5 managed servers with the same name as that of the 5 managed servers which are on MAC1.
We faced some issues starting the managed servers on MAC2.
So we decided to use pack and unpack.
We were able to start the managed servers on MAC2 however each server tooks about 25 minutes to start.
This is some what unexpected behaviour.
I had a look at the logs of one of the managed server.
It appears that its waits most of its time on :
<Log Management> <BEA-170019> <The server log file xyz.log is opened. All server side log events will be written to this file.>
The next log after this appears after about 20 minutes.
So i feel that its stuck reading or writing the logs.
Any pointers!!
Regards,
Stacey.

Answer 1

Hi All,
After taking the thread dumps , I am suspecting the following two threads which might be causing the issue :
"Thread-3" daemon prio=10 tid=0x73939800 nid=0x6784 waiting on condition [0x72530000]
java.lang.Thread.State: WAITING (parking)
     at sun.misc.Unsafe.park(Native Method)
     - parking to wait for <0x76610450> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118)
     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1735)
     at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:452)
     at weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:19)
     at weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:567)
     at weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:175)
     at weblogic.store.internal.PersistentStoreImpl$2.run(PersistentStoreImpl.java233)
"ExecuteThread: '7' for queue: 'weblogic.socket.Muxer'" daemon prio=10 tid=0x7391ec00 nid=0x6789 waiting for monitor entry [0x73344000]
java.lang.Thread.State: BLOCKED (on object monitor)
     at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:95)
     - waiting to lock <0x765902f0> (a weblogic.socket.PosixSocketMuxer$1)
     at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
Any pointers !!
Regards,
Stacey.