Stuck replication thread

We are using WLS 920 in production on RHEL 4. Getting the following stuck threads when a managed server is brought down. Any ideas?
          Thread-57 "[STUCK] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, waiting, priority=1, DAEMON> {
          jrockit.vm.Locks.convertThinLockedToFatLocked(Native Method)
          jrockit.vm.Locks.createMonitorAndConvert(Unknown Source)
          java.lang.Object.wait(Native Method)
          weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:73)
          ^-- Holding lock: weblogic.rjvm.ResponseImpl@38b34d4[thin lock]
          weblogic.rjvm.ResponseImpl.getTxContext(ResponseImpl.java:100)
          weblogic.rjvm.BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:109)
          weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
          weblogic.cluster.replication.ReplicationManager_920_WLStub.create(Unknown Source)
          weblogic.cluster.replication.ReplicationManager.trySecondary(ReplicationManager.java:658)
          weblogic.cluster.replication.ReplicationManager.createSecondary(ReplicationManager.java:630)
          weblogic.cluster.replication.ReplicationManager.registerLocally(ReplicationManager.java:412)
          weblogic.servlet.internal.session.ReplicatedSessionContext.lookupRO(ReplicatedSessionContext.java:193)
          weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:340)
          weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getValidSession(ServletRequestImpl.java:2521)
          weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getSessionInternal(ServletRequestImpl.java:2090)
          weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getSession(ServletRequestImpl.java:2057)
          weblogic.servlet.internal.ServletRequestImpl.getSession(ServletRequestImpl.java:1189)
          weblogic.servlet.security.internal.SecurityModule$SessionRetrievalAction.run(SecurityModule.java:535)
          weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
          weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
          weblogic.servlet.security.internal.SecurityModule.getUserSession(SecurityModule.java:426)
          weblogic.servlet.security.internal.ServletSecurityManager.checkAccess(ServletSecurityManager.java:81)
          weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:1920)
          weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1890)
          weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1344)
          weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
          weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
          jrockit.vm.RNI.c2java(Native Method)
          We implemented MP1 this weekend and did not see this issue with MP1. However, new issues surfaced that caused use to revert back to 920.

FYI, this was also cross-posted to the Berkeley DB XML forum and
responded to there.
Replication client gets stuck reading pages during queries

Similar Messages

Clearing a Stuck Execute Thread

HI.
If you get a stuck execute thread, is there anyway of clearing it or rolling it back so the action (a POST in this case) can be repeated again?
STUCK] ExecuteThread: '16' for queue: 'weblogic.kernel.Default (self-tuning)'
684959
weblogic.servlet.internal.ServletRequestImpl@da2109[ POST /ols/uds/request HTTP/1.0 Accept: image/gif, image/x-xbitmap,
image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint,
application/msword, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap,
application/x-ms-application, */* Referer: https://www.workcoverqld.com.au/ols/uds/file Accept-Language: en-au
Content-Type: application/x-www-form-urlencoded UA-CPU: x86 User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT
5.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729) Content-Length: 260 Pragma:
no-cache Cookie: JSESSIONID=Tsh3LLqRZrn0pYf4hdyrBbtLTxWf2sVg2Q8LBLgYpvrTThy9LYlJ!1280830650 X-Forwarded-For:
202.158.241.132 Connection: Keep-Alive Proxy-Client-IP: 202.158.241.132 X-WebLogic-KeepAliveSecs: 25
X-WebLogic-Request-ClusterInfo: true x-weblogic-cluster-hash: eu0SzdBVUqV5a6fkg3X2d1IDltA ]

Hi,
We can get more info about execute threads by getting thread dumps. Below are few ways to get
1. From weblogic admin console. Servers --> Monitoring --> Performance --> Dump Thread.
2. On the unix box enter the command "kill -3 <pid>".
3. Using WLST
java weblogic.WLST
>connect('<adminUsername>','<adminPassword>','<t3://adminServerUrl:port>')
>threadDump('true','VASLog.txt','<ServerName>')
>exit()
4. Using welogic.Admin command tool
java weblogic.Admin -adminurl t3://<adminServer>:<port> -username <adminUsername> -password <adminPassword> THREAD_DUMP
Except for point 3, all the output is written to "<ManagedServerLog>.out" file.
Thanks.
Vijay Bheemineni.

Reclaiming stuck execute threads

Does anyone out there know of a setting in WLS 7 where you can tell it to reclaim
an execute thread that has been stuck for a while? There are settings to detect
stuck threads, but I would like it to reclaim (kill stuck one and create a new
one and add back to execute queue) that stuck thread after a set amount of time.
Will it already do this on its own; am I just not waiting long enough?

Hi Chris,
"Chris" <[email protected]> wrote in message
news:3eb143e6$[email protected]..
Does anyone out there know of a setting in WLS 7 where you can tell it toreclaim
an execute thread that has been stuck for a while? There are settings todetect
stuck threads, but I would like it to reclaim (kill stuck one and create anew
one and add back to execute queue) that stuck thread after a set amount oftime.
In Java there is no way to "kill" a stuck thread, and there is a good
reason for this.
The feature you are taking about is for diadnistic purposes
and should be used as such.
Will it already do this on its own; am I just not waiting long enough?If it's really stuck, you have to get a thread dump and find out
why thread stuck. You can get a thread dump by pressing
<Ctrl+Break> in Widows shell or sending kill -3 to the server
process under Unix.
Regards,
Slava Imeshev

Need help with STUCK Threads. What's causing them?

In our production environment, we have a soa suite cluster with 3 nodes. All 3 of the servers keep turning up stuck and hogging threads, and eventually they all grind to a halt where I can't even view composite instances through em. There is a bpel process that is using a dbadapter to poll for records every 5 seconds and then performs some service call outs and other dbadapters. This process is generating thousands of instances a day and I think it is the culprit. If I view the logs, I see one of the dbadapters that it calls out to fails occasionally with a PK violation. I looked at the developer's code, and they have no error handling in the process at all. In the logs, there will the a hit for the PK violation, then about 10 minutes later the a STUCK thread gets logged(the configured timeout is 600 seconds, or 10 minutes). Would not handling an error in a BPEL process hog/STUCK a thread? I need to be able to show what code is causing the issue so I can have a resolution path.
I also see a note in the trace about something holding a lock:
Holding lock: oracle.jdbc.driver.T4CConnection@1fbc3486[thin lock]....would that be an un-rolled back transaction?
Here is the dump of the STUCK thread. Any help is greatly appreciated. Thanks!
<Aug 5, 2011 10:30:02 AM EDT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "634" seconds working on the request "oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-517 "[STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
java.net.SocketInputStream.read(SocketInputStream.java:107)
oracle.net.nt.MetricsEnabledInputStream.read(TcpNTAdapter.java:553)
oracle.net.ns.Packet.receive(Packet.java:249)
oracle.net.ns.DataPacket.receive(DataPacket.java:94)
oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:168)
oracle.net.ns.NetInputStream.read(NetInputStream.java:113)
oracle.net.ns.NetInputStream.read(NetInputStream.java:96)
oracle.net.ns.NetInputStream.read(NetInputStream.java:81)
oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:120)
oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:75)
oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1081)
oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)
oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:588)
oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:70)
oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:854)
oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:918)
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1252)
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3541)
oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3605)
^-- Holding lock: oracle.jdbc.driver.T4CConnection@1fbc3486[thin lock]
oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1667)
weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:123)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:887)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:541)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:530)
org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:521)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:202)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:192)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:254)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelect(DatasourceCallQueryMechanism.java:245)
org.eclipse.persistence.queries.DataReadQuery.executeNonCursor(DataReadQuery.java:188)
org.eclipse.persistence.queries.DataReadQuery.executeDatabaseQuery(DataReadQuery.java:141)
org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:603)
org.eclipse.persistence.queries.DataReadQuery.execute(DataReadQuery.java:115)
org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:589)
org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2898)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1218)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1190)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1176)
org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:378)
org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:701)
weblogic.deployment.TransactionalEntityManagerProxyImpl$QueryImpl.getSingleResult(TransactionalEntityManagerProxyImpl.java:232)
com.collaxa.cube.persistence.dao.impl.jpa.JpaCubeInstanceDaoImpl.getNumberOfInstances(JpaCubeInstanceDaoImpl.java:439)
oracle.fabric.CubeServiceEngine.getNumberOfComponentInstances(CubeServiceEngine.java:1962)
oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getNumberOfComponentInstances(FacadeFinderBeanImpl.java:1618)
oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBeanImpl.java:1653)
sun.reflect.GeneratedMethodAccessor5852.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:575)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.jee.intercept.MethodInvocationInvocationContext.proceed(MethodInvocationInvocationContext.java:104)
oracle.security.jps.ee.ejb.JpsAbsInterceptor$1.run(JpsAbsInterceptor.java:88)
oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:299)
oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:406)
oracle.security.jps.wls.JpsWeblogicEjbInterceptor.runJaasMode(JpsWeblogicEjbInterceptor.java:61)
oracle.security.jps.ee.ejb.JpsAbsInterceptor.intercept(JpsAbsInterceptor.java:81)
oracle.security.jps.ee.ejb.JpsInterceptor.intercept(JpsInterceptor.java:101)
sun.reflect.GeneratedMethodAccessor863.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:575)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
com.bea.core.repackaged.springframework.jee.intercept.JeeInterceptorInterceptor.invoke(JeeInterceptorInterceptor.java:66)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)
weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:22)
com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:49)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:86)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:151)
$Proxy309.getComponentSnapShot(Unknown Source)
oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.java:1604)
oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl_WLSkel.invoke(Unknown Source)
weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:567)
weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:143)
weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:425)
weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:117)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
>
<Aug 5, 2011 10:30:02 AM EDT> <Notice> <Diagnostics> <BEA-320068> <Watch 'StuckThread' with severity 'Notice' on server 'soa_server1' has triggered at Aug 5, 2011 10:30:02 AM EDT. Notification details:
WatchRuleType: Log
WatchRule: (SEVERITY = 'Error') AND (MSGID = 'BEA-000337')
WatchData: DATE = Aug 5, 2011 10:30:02 AM EDT SERVER = soa_server1 MESSAGE = [STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "634" seconds working on the request "oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-517 "[STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
java.net.SocketInputStream.read(SocketInputStream.java:107)
oracle.net.nt.MetricsEnabledInputStream.read(TcpNTAdapter.java:553)
oracle.net.ns.Packet.receive(Packet.java:249)
oracle.net.ns.DataPacket.receive(DataPacket.java:94)
oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:168)
oracle.net.ns.NetInputStream.read(NetInputStream.java:113)
oracle.net.ns.NetInputStream.read(NetInputStream.java:96)
oracle.net.ns.NetInputStream.read(NetInputStream.java:81)
oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:120)
oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:75)
oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1081)
oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)
oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:588)
oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:70)
oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:854)
oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:918)
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1252)
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3541)
oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3605)
^-- Holding lock: oracle.jdbc.driver.T4CConnection@1fbc3486[thin lock]
oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1667)
weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:123)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:887)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:541)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:530)
org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:521)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:202)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:192)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:254)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelect(DatasourceCallQueryMechanism.java:245)
org.eclipse.persistence.queries.DataReadQuery.executeNonCursor(DataReadQuery.java:188)
org.eclipse.persistence.queries.DataReadQuery.executeDatabaseQuery(DataReadQuery.java:141)
org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:603)
org.eclipse.persistence.queries.DataReadQuery.execute(DataReadQuery.java:115)
org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:589)
org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2898)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1218)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1190)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1176)
org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:378)
org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:701)
weblogic.deployment.TransactionalEntityManagerProxyImpl$QueryImpl.getSingleResult(TransactionalEntityManagerProxyImpl.java:232)
com.collaxa.cube.persistence.dao.impl.jpa.JpaCubeInstanceDaoImpl.getNumberOfInstances(JpaCubeInstanceDaoImpl.java:439)
oracle.fabric.CubeServiceEngine.getNumberOfComponentInstances(CubeServiceEngine.java:1962)
oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getNumberOfComponentInstances(FacadeFinderBeanImpl.java:1618)
oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBeanImpl.java:1653)
sun.reflect.GeneratedMethodAccessor5852.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:575)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.jee.intercept.MethodInvocationInvocationContext.proceed(MethodInvocationInvocationContext.java:104)
oracle.security.jps.ee.ejb.JpsAbsInterceptor$1.run(JpsAbsInterceptor.java:88)
oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:299)
oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:406)
oracle.security.jps.wls.JpsWeblogicEjbInterceptor.runJaasMode(JpsWeblogicEjbInterceptor.java:61)
oracle.security.jps.ee.ejb.JpsAbsInterceptor.intercept(JpsAbsInterceptor.java:81)
oracle.security.jps.ee.ejb.JpsInterceptor.intercept(JpsInterceptor.java:101)
sun.reflect.GeneratedMethodAccessor863.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:575)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
com.bea.core.repackaged.springframework.jee.intercept.JeeInterceptorInterceptor.invoke(JeeInterceptorInterceptor.java:66)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)
weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:22)
com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:49)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:86)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:151)
$Proxy309.getComponentSnapShot(Unknown Source)
oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.java:1604)
oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl_WLSkel.invoke(Unknown Source)
weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:567)
weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:143)
weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:425)
weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:117)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
SUBSYSTEM = WebLogicServer USERID = <WLS Kernel> SEVERITY = Error THREAD = [ACTIVE] ExecuteThread: '96' for queue: 'weblogic.kernel.Default (self-tuning)' MSGID = BEA-000337 MACHINE = soaprod TXID = CONTEXTID = f5d0b85a929630a2:6eada0f7:1304e0a2bdc:-7ffb-00000000017e1605 TIMESTAMP = 1312554602698
WatchAlarmType: AutomaticReset
WatchAlarmResetPeriod: 600000
>
Edited by: Brutus35 on Aug 5, 2011 8:04 AM

my impression is that you have trouble on the DB,
monitor the sessions to see if there is some table locked, and increase the number of processes on the DB
you can use SQLDeveloper for that
anyway your TX should rollback after some time, check that you have a timeout in place, it's better to rollback a TX than having the entire server crash

Socket stucking thread and program

Hi!
I have an applet that updates itself through a sockets thread .
The thread objects is being called in the init() method ,
for some reason in the constructor of the thread the program doesnt pass the line :
in=new ObjectInputStream(new BufferedInputStream(s.getInputStream()));
s is the socket .
The most wierd thing is , that this line stucks the thread , that stucks the applet , which is a wierd concept-its a thread !!!
What can it be ?

Let me guess. You have this:
in=new ObjectInputStream(new BufferedInputStream(s.getInputStream()));
out=new ObjectOutputStream(new BufferedOutputStream(s.getOutputStream()));in both your client and your server. Try it this way:
out=new ObjectOutputStream(new BufferedOutputStream(s.getOutputStream()));
in=new ObjectInputStream(new BufferedInputStream(s.getInputStream()));EJP

How to increase the Weblogic Session Threads

Hi Everyone,
We have been using replicated-if-clustered property of session-descriptor element of weblogic.xml and things were smooth and fine until our application didnt grow. Now there is a heavy traffic on the application and we want to increase the session replication thread count to more then 2 servers. Does anyone have any idea how to go about it? I dont see a way of defining how many servers the session should be replicated. Right now it does on the 2 servers which forms the cluster. Is there really any difference between replicated and replicated-if-clustered property?
Thanks a Lot in Advance for answering my queries.
Yogendra N Joshi.

user13017505 wrote:
Hi Everyone,
We have been using replicated-if-clustered property of session-descriptor element of weblogic.xml and things were smooth and fine until our application didnt grow. Now there is a heavy traffic on the application and we want to increase the session replication thread count to more then 2 servers. Does anyone have any idea how to go about it? I dont see a way of defining how many servers the session should be replicated. Right now it does on the 2 servers which forms the cluster. Is there really any difference between replicated and replicated-if-clustered property?
Thanks a Lot in Advance for answering my queries.
Yogendra N Joshi.To start with persistent store/session replication options:
•memory—Disables persistent session storage.
•replicated—Same as memory, but session data is replicated across the clustered servers.(No Persistent store)
•replicated_if_clustered—If the Web application is deployed on a clustered server, the in-effect persistent-store-type will be replicated. Otherwise, memory is the default.
http://download.oracle.com/docs/cd/E12840_01/wls/docs103/webapp/weblogic_xml.html
Regarding increasing traffic on the servers:
If there is heavy traffic on the servers and both servers in a cluster are utilized fully, you can add more managed servers to the cluster and load balance the requests amongst them equally.
The sessions will be replicated across all the servers in a cluster as you have session replication enabled already.
But for increasing load/traffic we will have to analyze the capacity of the present setup and then add more servers to the cluster as required.
Regards,
Swapna

Replication port used in TimesTen 5.1.35

Hi,
Can you please tell me what is the default port number used by TimesTen replication in version 5.1.35 ?
Regards
Pratheej

If you're using AUTO port assignment for replication (ie not specifying a PORT in your Create Replication SQL), then use ttRepAdmin -showstatus to see what the port is that's been assigned by the OS. This is also specified in the timestend.log when the replication threads are starting up. There isn't a default port used otherwise. The agent will just use whatever it's assigned by the OS on each startup.

Doubt in GG replication

Hi All,
posting this thread in this section since i dont see much activity under replication thread
We have a Table level GG replication setup between source and target.
My question is, say suppose i create a DELETE Trigger on a table (which is under replication) in target, will it cause any issue in replication?
Thanks,
John

Sorry returning late to the thread.
In the source database the trigger will be active and function as normal. Golden Gate will capture the SQL the causes the trigger to fire in the source database and will replicate that SQL to the destination database. Golden Gate will also capture any changes made by the trigger in source database replicate those changes to the destination database.
If you do not suppress the trigger in the destination database then the following will take place. The SQL that fired the trigger will be executed in the destination database, thus re-firing the trigger destination. Any changes made by the trigger will happen twice in the destination database. Once by the SQL the other by the replication of any changes made by the trigger in the source database.
You are not going to loose any changes made because the SQL the fired the trigger and the changes made by the trigger are shipped to the destination.

Weblogic em and console crashes in Forms 11gR2 domain

Hi,
I am working towards an APEX and/or Forms11gR2 environment from Forms and Reports 6i.
Coming from a client-server background,with the concept of application servers etc totally unfamiliar territory.
and a near-zero knowledge of Java its been a tough few weeks,but I have managed to install a test 11gR2 Windows 32 Oracle Database win 7,
Weblogic 10_3_5 Admin server and a managed server on Windows Server 2008 32 bit 4GB memory,install and deploy
APEX and the APEX Listener and also install Forms and Reports 11gR2 using a separate domain as required by the installer.
The Weblogic Admin server and its managed server are stable and i can access and administer them no problem from a browser
with em and console.
My problem is with the domain created by Forms and Reports for deployment. I can login to the console and em of
that domain but whenever I click on a link inside, the domain crashes with a "fatal error detected by the Java Runtime environment".
The Domain name is FormsDomain.
As I need to start the WLS_FORM and WLS_REPORT in the Forms Domain through the console, I am stuck.
Can anyone help,I have pasted the output form the created log file:
# A fatal error has been detected by the Java Runtime Environment:
# java.lang.OutOfMemoryError: requested 131080 bytes for Chunk::new. Out of swap space?
# Internal Error (allocation.cpp:272), pid=4116, tid=6032
# Error: Chunk::new
# JRE version: 6.0_24-b50
# Java VM: Java HotSpot(TM) Server VM (19.1-b02 mixed mode windows-x86 )
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
--------------- T H R E A D ---------------
Current thread (0x6591b000): JavaThread "CompilerThread1" daemon [_thread_in_native, id=6032, stack(0x65b80000,0x65bd0000)]
Stack: [0x65b80000,0x65bd0000], sp=0x65bce62c, free space=313k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x3379f7]
V [jvm.dll+0x10548c]
V [jvm.dll+0x3ee6c]
V [jvm.dll+0x3f103]
V [jvm.dll+0x2c30ae]
V [jvm.dll+0x172ae8]
V [jvm.dll+0x6fdc7]
V [jvm.dll+0x13f819]
V [jvm.dll+0x141cb9]
V [jvm.dll+0x14383c]
V [jvm.dll+0x143db6]
V [jvm.dll+0xe36ea]
V [jvm.dll+0xe409a]
V [jvm.dll+0xa679a]
V [jvm.dll+0xe5ffc]
V [jvm.dll+0xe6baf]
V [jvm.dll+0x317ed4]
V [jvm.dll+0x28ce0c]
C [msvcr71.dll+0x9565]
C [kernel32.dll+0x44b29]
C [ntdll.dll+0x3e1c6]
C [ntdll.dll+0x3e199]
Current CompileTask:
C2:3075 java.beans.Introspector.getTargetMethodInfo()[Ljava/beans/MethodDescriptor; (255 bytes)
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x6f4adc00 JavaThread "DmsThread-7" daemon [_thread_blocked, id=4388, stack(0x73090000,0x730e0000)]
0x6f4ad400 JavaThread "DmsThread-6" daemon [_thread_blocked, id=5720, stack(0x72f70000,0x72fc0000)]
0x6f4ad000 JavaThread "DmsThread-5" daemon [_thread_blocked, id=5900, stack(0x72f20000,0x72f70000)]
0x6f4ac800 JavaThread "DmsThread-4" daemon [_thread_blocked, id=5904, stack(0x72ed0000,0x72f20000)]
0x6f4ac400 JavaThread "DmsThread-3" daemon [_thread_blocked, id=6080, stack(0x6db20000,0x6db70000)]
0x6f4abc00 JavaThread "DmsThread-2" daemon [_thread_blocked, id=6116, stack(0x6dad0000,0x6db20000)]
0x6f4ab000 JavaThread "ClientNotifForwarder-1" daemon [_thread_blocked, id=5072, stack(0x6da80000,0x6dad0000)]
0x6f4aac00 JavaThread "CacheManager" daemon [_thread_blocked, id=4528, stack(0x6bc90000,0x6bce0000)]
0x6f4aa400 JavaThread "DynamicListenThread[Default[4]]" daemon [_thread_in_native, id=5608, stack(0x6da30000,0x6da80000)]
0x69063c00 JavaThread "DynamicListenThread[Default[2]]" daemon [_thread_in_native, id=4204, stack(0x6d9e0000,0x6da30000)]
0x69063400 JavaThread "DynamicListenThread[Default[1]]" daemon [_thread_in_native, id=3324, stack(0x6d990000,0x6d9e0000)]
0x69063000 JavaThread "DynamicListenThread[Default]" daemon [_thread_in_native, id=3560, stack(0x6d940000,0x6d990000)]
0x69062800 JavaThread "DynamicListenThread[Default[3]]" daemon [_thread_in_native, id=5712, stack(0x6d8f0000,0x6d940000)]
0x69062400 JavaThread "DynamicListenThread[Default[5]]" daemon [_thread_in_native, id=4088, stack(0x6d8a0000,0x6d8f0000)]
0x69061c00 JavaThread "weblogic.GCMonitor" daemon [_thread_blocked, id=4508, stack(0x6d660000,0x6d6b0000)]
0x69061800 JavaThread "Thread-41" [_thread_blocked, id=1876, stack(0x6d610000,0x6d660000)]
0x69060400 JavaThread "CacheCleaner" daemon [_thread_blocked, id=5624, stack(0x6d580000,0x6d5d0000)]
0x671d8000 JavaThread "Worker-2" daemon [_thread_blocked, id=3380, stack(0x6d530000,0x6d580000)]
0x6a8ce400 JavaThread "Worker-1" daemon [_thread_blocked, id=5328, stack(0x6d4e0000,0x6d530000)]
0x6a8cd800 JavaThread "Worker-0" daemon [_thread_blocked, id=4988, stack(0x6d490000,0x6d4e0000)]
0x6a8ce800 JavaThread "SocketTimeout" daemon [_thread_blocked, id=4480, stack(0x6d440000,0x6d490000)]
0x6a8cdc00 JavaThread "RepolessTargetManager" daemon [_thread_blocked, id=1132, stack(0x6d3f0000,0x6d440000)]
0x6a8cd000 JavaThread "HealthMonitor" daemon [_thread_blocked, id=3192, stack(0x6d340000,0x6d390000)]
0x6a8cc400 JavaThread "Timer-5" daemon [_thread_blocked, id=5716, stack(0x6d070000,0x6d0c0000)]
0x6a8cb800 JavaThread "EmChartCacheWorker" daemon [_thread_blocked, id=4780, stack(0x6d020000,0x6d070000)]
0x6a8cb400 JavaThread "Timer-4" daemon [_thread_blocked, id=4604, stack(0x6cf80000,0x6cfd0000)]
0x69269800 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=2096, stack(0x6cee0000,0x6cf30000)]
0x6a8ccc00 JavaThread "JarIndex Timer" daemon [_thread_blocked, id=5772, stack(0x6cda0000,0x6cdf0000)]
0x671d7c00 JavaThread "DmsThread-1" daemon [_thread_blocked, id=4200, stack(0x6b8c0000,0x6b910000)]
0x671d5c00 JavaThread "[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=3672, stack(0x6d160000,0x6d1b0000)]
0x671d5800 JavaThread "[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=5080, stack(0x6d110000,0x6d160000)]
0x671d5000 JavaThread "[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=1900, stack(0x6d0c0000,0x6d110000)]
0x6926a000 JavaThread "LogFlusher" daemon [_thread_blocked, id=3716, stack(0x6cfd0000,0x6d020000)]
0x69269400 JavaThread "oracle.dfw.incident.FloodController" daemon [_thread_blocked, id=4612, stack(0x6cf30000,0x6cf80000)]
0x69267c00 JavaThread "oracle.dfw.incident.IncidentCreatorThread" daemon [_thread_blocked, id=4092, stack(0x6cdf0000,0x6ce40000)]
0x69268000 JavaThread "oracle.dfw.impl.incident - ADR Executor (created: Thu Feb 21 13:50:40 GST 2013)" daemon [_thread_blocked, id=5528, stack(0x6ce40000,0x6ce90000)]
0x69267000 JavaThread "LogFlusher" daemon [_thread_blocked, id=5480, stack(0x6bd80000,0x6bdd0000)]
0x69266800 JavaThread "LogFlusher" daemon [_thread_blocked, id=4720, stack(0x6bd30000,0x6bd80000)]
0x69266400 JavaThread "LogFlusher" daemon [_thread_blocked, id=3756, stack(0x6bce0000,0x6bd30000)]
0x69265800 JavaThread "Thread-14" [_thread_blocked, id=6044, stack(0x6ba00000,0x6ba50000)]
0x69265000 JavaThread "Timer-2" daemon [_thread_blocked, id=5640, stack(0x6b9b0000,0x6ba00000)]
0x69264c00 JavaThread "Runtime Authorization Event Notifier" daemon [_thread_blocked, id=4808, stack(0x6b960000,0x6b9b0000)]
0x69264400 JavaThread "JMX Framework document pooling thread" daemon [_thread_blocked, id=4412, stack(0x6b910000,0x6b960000)]
0x69263400 JavaThread "JPS FileChangeNotifier<clean-only>" daemon [_thread_blocked, id=4400, stack(0x6b820000,0x6b870000)]
0x69263800 JavaThread "DoSManager" daemon [_thread_blocked, id=4160, stack(0x6b870000,0x6b8c0000)]
0x69262c00 JavaThread "VDE Replication Thread" daemon [_thread_blocked, id=5376, stack(0x6b7d0000,0x6b820000)]
0x69262800 JavaThread "VDE Transaction Processor Thread" daemon [_thread_blocked, id=4536, stack(0x6b780000,0x6b7d0000)]
0x67156400 JavaThread "ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" daemon [_thread_in_native, id=5064, stack(0x6b730000,0x6b780000)]
0x675bd800 JavaThread "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon [_thread_in_native, id=3316, stack(0x67d50000,0x67da0000)]
0x675bd000 JavaThread "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon [_thread_in_native, id=3772, stack(0x67d00000,0x67d50000)]
0x68467800 JavaThread "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon [_thread_in_native, id=1308, stack(0x67cb0000,0x67d00000)]
0x672ec400 JavaThread "[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=4264, stack(0x67c60000,0x67cb0000)]
0x69c4d800 JavaThread "weblogic.timers.TimerThread" daemon [_thread_blocked, id=6096, stack(0x67a60000,0x67ab0000)]
0x67798800 JavaThread "weblogic.time.TimeEventGenerator" daemon [_thread_blocked, id=6112, stack(0x67a10000,0x67a60000)]
0x69563400 JavaThread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_in_Java, id=3968, stack(0x67940000,0x67990000)]
0x68529800 JavaThread "Timer-1" daemon [_thread_blocked, id=4852, stack(0x67890000,0x678e0000)]
0x68a4a400 JavaThread "Timer-0" daemon [_thread_blocked, id=5136, stack(0x67840000,0x67890000)]
0x6591bc00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=4252, stack(0x65bd0000,0x65c20000)]
=>0x6591b000 JavaThread "CompilerThread1" daemon [_thread_in_native, id=6032, stack(0x65b80000,0x65bd0000)]
0x65916c00 JavaThread "CompilerThread0" daemon [_thread_in_native, id=3320, stack(0x65b30000,0x65b80000)]
0x65914000 JavaThread "Attach Listener" daemon [_thread_blocked, id=4048, stack(0x65ae0000,0x65b30000)]
0x6590b400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=4860, stack(0x022e0000,0x02330000)]
0x008a9400 JavaThread "Finalizer" daemon [_thread_blocked, id=4448, stack(0x02290000,0x022e0000)]
0x008a8400 JavaThread "Reference Handler" daemon [_thread_blocked, id=5292, stack(0x00c00000,0x00c50000)]
0x007d1000 JavaThread "main" [_thread_blocked, id=5704, stack(0x002e0000,0x00330000)]
Other Threads:
0x008a2800 VMThread [stack: 0x009b0000,0x00a00000] [id=4084]
0x6592cc00 WatcherThread [stack: 0x65c20000,0x65c70000] [id=5976]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
Heap
PSYoungGen total 291968K, used 153215K [0x4fe20000, 0x64cb0000, 0x65370000)
eden space 241984K, 54% used [0x4fe20000,0x57ff3458,0x5ea70000)
from space 49984K, 40% used [0x61be0000,0x62faca78,0x64cb0000)
to space 50304K, 0% used [0x5ea70000,0x5ea70000,0x61b90000)
PSOldGen total 349568K, used 184605K [0x25370000, 0x3a8d0000, 0x4fe20000)
object space 349568K, 52% used [0x25370000,0x307b77a8,0x3a8d0000)
PSPermGen total 262144K, used 166001K [0x05370000, 0x15370000, 0x25370000)
object space 262144K, 63% used [0x05370000,0x0f58c610,0x15370000)
Dynamic libraries:
0x00400000 - 0x00424000      D:\Oracle\MIDDLE~1\JDK160~1\bin\java.exe
0x76e70000 - 0x76f98000      C:\Windows\system32\ntdll.dll
0x768e0000 - 0x769bc000      C:\Windows\system32\kernel32.dll
0x756d0000 - 0x75796000      C:\Windows\system32\ADVAPI32.dll
0x77000000 - 0x770c2000      C:\Windows\system32\RPCRT4.dll
0x7c340000 - 0x7c396000      D:\Oracle\MIDDLE~1\JDK160~1\jre\bin\msvcr71.dll
0x6db70000 - 0x6dfb9000      D:\Oracle\MIDDLE~1\JDK160~1\jre\bin\server\jvm.dll
0x76840000 - 0x768dd000      C:\Windows\system32\USER32.dll
0x76af0000 - 0x76b3b000      C:\Windows\system32\GDI32.dll
0x73bf0000 - 0x73c22000      C:\Windows\system32\WINMM.dll
0x76ba0000 - 0x76c4a000      C:\Windows\system32\msvcrt.dll
0x76c50000 - 0x76d94000      C:\Windows\system32\ole32.dll
0x75900000 - 0x7598d000      C:\Windows\system32\OLEAUT32.dll
0x73ae0000 - 0x73b19000      C:\Windows\system32\OLEACC.dll
0x76fd0000 - 0x76fee000      C:\Windows\system32\IMM32.DLL
0x76da0000 - 0x76e68000      C:\Windows\system32\MSCTF.dll
0x75510000 - 0x7553c000      C:\Windows\system32\apphelp.dll
0x76fb0000 - 0x76fb9000      C:\Windows\system32\LPK.DLL
0x76730000 - 0x767ad000      C:\Windows\system32\USP10.dll
0x6d850000 - 0x6d85c000      D:\Oracle\MIDDLE~1\JDK160~1\jre\bin\verify.dll
0x6d3d0000 - 0x6d3ef000      D:\Oracle\MIDDLE~1\JDK160~1\jre\bin\java.dll
0x6d330000 - 0x6d338000      D:\Oracle\MIDDLE~1\JDK160~1\jre\bin\hpi.dll
0x76fa0000 - 0x76fa7000      C:\Windows\system32\PSAPI.DLL
0x6d890000 - 0x6d89f000      D:\Oracle\MIDDLE~1\JDK160~1\jre\bin\zip.dll
0x6d6b0000 - 0x6d6c3000      D:\Oracle\Middleware\jdk160_24\jre\bin\net.dll
0x75990000 - 0x759bd000      C:\Windows\system32\WS2_32.dll
0x76fc0000 - 0x76fc6000      C:\Windows\system32\NSI.dll
0x74cf0000 - 0x74d2b000      C:\Windows\system32\mswsock.dll
0x74df0000 - 0x74df5000      C:\Windows\System32\wship6.dll
0x748f0000 - 0x748ff000      C:\Windows\system32\NLAapi.dll
0x74ec0000 - 0x74ed9000      C:\Windows\system32\IPHLPAPI.DLL
0x74f20000 - 0x74f55000      C:\Windows\system32\dhcpcsvc.DLL
0x75180000 - 0x751ac000      C:\Windows\system32\DNSAPI.dll
0x75570000 - 0x75584000      C:\Windows\system32\Secur32.dll
0x74f10000 - 0x74f17000      C:\Windows\system32\WINNSI.DLL
0x74ee0000 - 0x74f01000      C:\Windows\system32\dhcpcsvc6.DLL
0x73230000 - 0x73238000      C:\Windows\System32\winrnr.dll
0x758b0000 - 0x758fa000      C:\Windows\system32\WLDAP32.dll
0x73240000 - 0x7324f000      C:\Windows\system32\napinsp.dll
0x749b0000 - 0x749b5000      C:\Windows\System32\wshtcpip.dll
0x731b0000 - 0x731b6000      C:\Windows\system32\rasadhlp.dll
0x6d6d0000 - 0x6d6d9000      D:\Oracle\Middleware\jdk160_24\jre\bin\nio.dll
0x003a0000 - 0x003a5000      D:\Oracle\Middleware\wlserver_10.3\server\native\win\32\terminalio.dll
0x6d600000 - 0x6d609000      D:\Oracle\Middleware\jdk160_24\jre\bin\management.dll
0x003c0000 - 0x003c5000      D:\Oracle\Middleware\wlserver_10.3\server\native\win\32\stackdump.dll
0x74c20000 - 0x74c5b000      C:\Windows\system32\rsaenh.dll
0x75590000 - 0x755ae000      C:\Windows\system32\USERENV.dll
0x003d0000 - 0x003d5000      D:\Oracle\Middleware\wlserver_10.3\server\native\win\32\wlntio.dll
0x6d830000 - 0x6d838000      D:\Oracle\Middleware\jdk160_24\jre\bin\sunmscapi.dll
0x74fe0000 - 0x750d1000      C:\Windows\system32\CRYPT32.dll
0x75120000 - 0x75132000      C:\Windows\system32\MSASN1.dll
0x003e0000 - 0x003f3000      D:\Oracle\Middleware\wlserver_10.3\server\native\win\32\wlfileio3.dll
0x6d1b0000 - 0x6d2fa000      D:\Oracle\Middleware\jdk160_24\jre\bin\awt.dll
0x6ce90000 - 0x6ced2000      C:\Windows\system32\WINSPOOL.DRV
0x74630000 - 0x747ce000      C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.6001.18523_none_5cdd65e20837faf2\COMCTL32.dll
0x76b40000 - 0x76b98000      C:\Windows\system32\SHLWAPI.dll
0x02350000 - 0x02356000      D:\Oracle\Middleware\jdk160_24\jre\bin\rmi.dll
VM Arguments:
jvm_args: -Xms256m -Xmx512m -XX:MaxPermSize=512m -Dweblogic.Name=AdminServer -Djava.security.policy=D:\Oracle\MIDDLE~1\WLSERV~1.3\server\lib\weblogic.policy -Dweblogic.ProductionModeEnabled=true -da -Dplatform.home=D:\Oracle\MIDDLE~1\WLSERV~1.3 -Dwls.home=D:\Oracle\MIDDLE~1\WLSERV~1.3\server -Dweblogic.home=D:\Oracle\MIDDLE~1\WLSERV~1.3\server -Xms512m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m -Doracle.home=D:\Oracle\Middleware\FRHome1 -Ddomain.home=D:\Weblogic_Domains\domains\FormsDomain -Dcommon.components.home=D:\Oracle\MIDDLE~1\ORACLE~1 -Djrf.version=11.1.1 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Ddomain.home=D:\WEBLOG~1\domains\FORMSD~1 -Djrockit.optfile=D:\Oracle\MIDDLE~1\ORACLE~1\modules\oracle.jrf_11.1.1\jrocket_optfile.txt -Doracle.server.config.dir=D:\WEBLOG~1\domains\FORMSD~1\config\FMWCON~1\servers\AdminServer -Doracle.domain.config.dir=D:\WEBLOG~1\domains\FORMSD~1\config\FMWCON~1 -Digf.arisidbeans.carmlloc=D:\WEBLOG~1\domains\FORMSD~1\config\FMWCON~1\carml -Digf.arisidstack.home=D:\WEBLOG~1\domains\FORMSD~1\config\FMWCON~1\arisidprovider -Doracle.security.jps.config=D:\WEBLOG~1\domains\FORMSD~1\config\fmwconfig\jps-config.xml -Doracle.deployed.app.dir=D:\WEBLOG~1\domains\FORMSD~1\servers\AdminServer\tmp\_WL_user -Doracle.deployed.app.ext=\- -Dweblogic.alternateTypesDirectory=D:\Oracle\MIDDLE~1\ORACLE~1\modules\oracle.ossoiap_11.1.1,D:\Oracle\MIDDLE~1\ORACLE~1\modules\oracle.oamprovider_11.1.1 -Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dweblogic.jdbc.remoteEnabled=false -Dwc.oracle.home=D:\Oracle\Middleware\FRHome1 -Dclassic.oracle.home=D:\Oracle\Middleware\FRHome1 -Dem.oracle.home=D:\Oracle\Middleware\oracle_common -Djava.awt.headless=true -Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext.dirs=D:\Oracle\MIDDLE~1\patch_wls1035\profiles\default\sysext_manifest_classpath;D:\Oracle\MIDDLE~1\patch_jdev1111\profiles\default\sysext_manifest_classpath
java_command: weblogic.Server
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=D:\Oracle\MIDDLE~1\JDK160~1
PATH=D:\Oracle\MIDDLE~1\patch_wls1035\profiles\default\native;D:\Oracle\MIDDLE~1\patch_jdev1111\profiles\default\native;D:\Oracle\MIDDLE~1\WLSERV~1.3\server\native\win\32;D:\Oracle\MIDDLE~1\WLSERV~1.3\server\bin;D:\Oracle\MIDDLE~1\modules\ORGAPA~1.1\bin;D:\Oracle\MIDDLE~1\JDK160~1\jre\bin;D:\Oracle\MIDDLE~1\JDK160~1\bin;D:\Oracle\MIDDLE~1\FRHome1\BIN;C:\Windows\System32;C:\Windows;C:\Windows\System32\wbem;C:\Windows\System32\WINDOW~1\v1.0\;d:\Oracle\MIDDLE~1\JDK160~1\bin;D:\Oracle\MIDDLE~1\FRHome1\opmn\bin;D:\Oracle\MIDDLE~1\FRHome1\opmn\lib;D:\Oracle\MIDDLE~1\FRHome1\perl\bin;D:\Oracle\MIDDLE~1\WLSERV~1.3\server\native\win\32\oci920_8
USERNAME=Administrator
LD_LIBRARY_PATH=D:\Oracle\Middleware\FRHome1\lib
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 10, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows Server 2008 Build 6001 Service Pack 1
CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 23 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1
Memory: 4k page, physical 4192516k(1701856k free), swap 13055652k(10385168k free)
vm_info: Java HotSpot(TM) Server VM (19.1-b02) for windows-x86 JRE (1.6.0_24-b50), built on Feb 10 2011 01:48:01 by "java_re" with MS VC++ 7.1 (VS2003)
time: Thu Feb 21 13:53:53 2013
elapsed time: 215 seconds

Hi,
a hint anyone?
The domains I created with JDeveloper work without any problems and are totally stable,
while the one created through Oracle Forms just crashes all the time when accessed through the console
or em.
Is there any difference between these domains?
They all use jdk160_24 and the Forms Domain crashes even when running alone, so it can't be lack of
physical memory, I would think..
I would have created a domain through JDeveloper and extend it, but extending a domain is only possible if
the domain is created with Forms, so I am well and truly stuck here.
Regards
Leckraj

Weblogic 10.3.0 managed server + Spring Framework slow startup

Hello
Iam running weblogic 10.3.0 plus an application using spring framework on this:
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_20-b02
java.specification.name = Java Platform API Specification
java.specification.vendor = Sun Microsystems Inc.
java.specification.version = 1.6
java.vendor = Oracle Corporation
java.version = 1.6.0_20
java.vm.info = compiled mode
java.vm.name = Oracle JRockit(R)
os.arch = amd64
os.name = Windows 2003
os.version = 5.2
sun.arch.data.model = 64
sun.io.unicode.encoding = UnicodeLittle
sun.jnu.encoding = Cp1252
sun.management.compiler = Oracle JRockit(R) Optimizing Compiler
sun.os.patch.level = Service Pack 2
Problem Description: A weblogic managed server is taking more than an hour to start, the delay is due to the deployments during startup, passing from STATE_PREPARED to STATE_ADMIN and to the initialization of the Spring root WebApplicationContex, which takes an avg of 7 mins per module deployed.
I have a development server with the same application which starts in a few minutes, the only difference between the environments are tha the development server runs in 32bits.
Any help will be appreciated
-luis
Server log file:
####<Feb 17, 2011 5:09:06 PM VET> <Info> <ServletContext-/Micropago/ServiciosOperacionTransferencia> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1297978746371> <BEA-000000> <Initializing Spring root WebApplicationContext>
####<Feb 17, 2011 5:19:16 PM VET> <Info> <Deployer> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1297979356364> <BEA-149059> <Module operaciondispatcher-ws-1.0-SNAPSHOT.war of application operaciondispatcher-ws-1 is transitioning from STATE_PREPARED to STATE_ADMIN on server micropago_server.>
####<Feb 17, 2011 5:19:16 PM VET> <Info> <Deployer> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1297979356411> <BEA-149060> <Module operaciondispatcher-ws-1.0-SNAPSHOT.war of application operaciondispatcher-ws-1 successfully transitioned from STATE_PREPARED to STATE_ADMIN on server micropago_server.>
####<Feb 17, 2011 5:19:24 PM VET> <Info> <ServletContext-/Micropago/ServiciosOperacionD> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1297979364083> <BEA-000000> <Initializing Spring root WebApplicationContext>
####<Feb 17, 2011 5:23:35 PM VET> <Info> <Deployer> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1297979615333> <BEA-149059> <Module operacionutil-ws-1.0-SNAPSHOT.war of application operacionutil-ws-1 is transitioning from STATE_PREPARED to STATE_ADMIN on server micropago_server.>
####<Feb 17, 2011 5:23:35 PM VET> <Info> <Deployer> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1297979615333> <BEA-149060> <Module operacionutil-ws-1.0-SNAPSHOT.war of application operacionutil-ws-1 successfully transitioned from STATE_PREPARED to STATE_ADMIN on server micropago_server.>
####<Feb 17, 2011 5:23:40 PM VET> <Info> <ServletContext-/Micropago/ServiciosUtilidad> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1297979620318> <BEA-000000> <Initializing Spring root WebApplicationContext>
####<Feb 17, 2011 5:30:10 PM VET> <Info> <Deployer> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1297980010468> <BEA-149059> <Module securitymanager-ws-1.0-SNAPSHOT.war of application securitymanager-ws-1 is transitioning from STATE_PREPARED to STATE_ADMIN on server micropago_server.>
####<Feb 17, 2011 5:30:10 PM VET> <Info> <Deployer> <SAPL734> <micropago_server> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1297980010515> <BEA-149060> <Module securitymanager-ws-1.0-SNAPSHOT.war of application securitymanager-ws-1 successfully transitioned from STATE_PREPARED to STATE_ADMIN on server micropago_server.>

Hi, thanks for your answer.
How can I do a Thread Dump snapshot in a windows server starting as a service (no console)?
Admin_server Thread Stack Trace
Dump 1/3
"Main Thread" waiting for lock weblogic.t3.srvr.T3Srvr@f14693 WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:811)
     weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:459)
     weblogic.Server.main(Server.java:67)
"(Signal Handler)" RUNNABLE
null
"(OC Main Thread)" RUNNABLE
null
"(Code Generation Thread 1)" RUNNABLE
null
"(Code Optimization Thread 1)" RUNNABLE
null
"(VM Periodic Task)" RUNNABLE
null
"(Attach Listener)" RUNNABLE
null
"Finalizer" RUNNABLE
     jrockit.memory.Finalizer.waitForFinalizees(Native Method)
     jrockit.memory.Finalizer.access$700(Finalizer.java:12)
     jrockit.memory.Finalizer$4.run(Finalizer.java:189)
     java.lang.Thread.run(Thread.java:619)
"Reference Handler" RUNNABLE
     java.lang.ref.Reference.waitForActivatedQueue(Native Method)
     java.lang.ref.Reference.access$100(Reference.java:11)
     java.lang.ref.Reference$ReferenceHandler.run(Reference.java:82)
"(Sensor Event Thread)" RUNNABLE
null
"VM JFR Buffer Thread" RUNNABLE native
null
"Timer-0" waiting for lock java.util.TaskQueue@e259a7 WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     java.util.TimerThread.mainLoop(Timer.java:483)
     java.util.TimerThread.run(Timer.java:462)
"Timer-1" waiting for lock java.util.TaskQueue@10ad080 TIMED_WAITING
     java.lang.Object.wait(Native Method)
     java.util.TimerThread.mainLoop(Timer.java:509)
     java.util.TimerThread.run(Timer.java:462)
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" RUNNABLE
DUMP 2/3
"Main Thread" waiting for lock weblogic.t3.srvr.T3Srvr@f14693 WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:811)
     weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:459)
     weblogic.Server.main(Server.java:67)
"(Signal Handler)" RUNNABLE
null
"(OC Main Thread)" RUNNABLE
null
"(Code Generation Thread 1)" RUNNABLE
null
"(Code Optimization Thread 1)" RUNNABLE
null
"(VM Periodic Task)" RUNNABLE
null
"(Attach Listener)" RUNNABLE
null
"Finalizer" RUNNABLE
     jrockit.memory.Finalizer.waitForFinalizees(Native Method)
     jrockit.memory.Finalizer.access$700(Finalizer.java:12)
     jrockit.memory.Finalizer$4.run(Finalizer.java:189)
     java.lang.Thread.run(Thread.java:619)
"Reference Handler" RUNNABLE
     java.lang.ref.Reference.waitForActivatedQueue(Native Method)
     java.lang.ref.Reference.access$100(Reference.java:11)
     java.lang.ref.Reference$ReferenceHandler.run(Reference.java:82)
"(Sensor Event Thread)" RUNNABLE
null
"VM JFR Buffer Thread" RUNNABLE native
null
"Timer-0" waiting for lock java.util.TaskQueue@e259a7 WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     java.util.TimerThread.mainLoop(Timer.java:483)
     java.util.TimerThread.run(Timer.java:462)
"Timer-1" waiting for lock java.util.TaskQueue@10ad080 TIMED_WAITING
     java.lang.Object.wait(Native Method)
     java.util.TimerThread.mainLoop(Timer.java:509)
     java.util.TimerThread.run(Timer.java:462)
"[STANDBY] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.work.ExecuteThread@10ad1bd WAITING
     java.lang.Object.wait(Object.java:485)
     weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:157)
     weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
"weblogic.time.TimeEventGenerator" waiting for lock weblogic.time.common.internal.TimeTable@10aff60 TIMED_WAITING
     weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:286)
     weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:117)
     java.lang.Thread.run(Thread.java:619)
"weblogic.timers.TimerThread" waiting for lock weblogic.timers.internal.TimerThread@11216cb TIMED_WAITING
     java.lang.Object.wait(Native Method)
     weblogic.timers.internal.TimerThread$Thread.run(TimerThread.java:267)
"[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.work.ExecuteThread@10ad3e0 WAITING
     java.lang.Object.wait(Object.java:485)
     weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:157)
     weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
"Thread-7" waiting for lock java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@10ad595 WAITING
     sun.misc.Unsafe.park(Native Method)
     java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
     java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
     java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
     weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
     weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:567)
     weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:615)
     weblogic.store.internal.PersistentStoreImpl$2.run(PersistentStoreImpl.java:383)
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" RUNNABLE native
     weblogic.socket.NTSocketMuxer.getIoCompletionResult(Native Method)
     weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
     weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" RUNNABLE native
     weblogic.socket.NTSocketMuxer.getIoCompletionResult(Native Method)
     weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
     weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" RUNNABLE native
     weblogic.socket.NTSocketMuxer.getIoCompletionResult(Native Method)
     weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
     weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
"VDE Transaction Processor Thread" waiting for lock [email protected] WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:367)
     com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212)
"VDE Replication Thread" waiting for lock com.octetstring.vde.replication.Replication@10ad426 TIMED_WAITING
     com.octetstring.vde.replication.Replication.wait30sec(Replication.java:384)
     com.octetstring.vde.replication.Replication.run(Replication.java:350)
"DoSManager" TIMED_WAITING
     java.lang.Thread.sleep(Native Method)
     com.octetstring.vde.DoSManager.run(DoSManager.java:433)
"Thread-11" waiting for lock java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4f576f WAITING
     sun.misc.Unsafe.park(Native Method)
DUMP 3/3
"Main Thread" waiting for lock weblogic.t3.srvr.T3Srvr@f14693 WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:811)
     weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:459)
     weblogic.Server.main(Server.java:67)
"(Signal Handler)" RUNNABLE
null
"(OC Main Thread)" RUNNABLE
null
"(Code Generation Thread 1)" RUNNABLE
null
"(Code Optimization Thread 1)" RUNNABLE
null
"(VM Periodic Task)" RUNNABLE
null
"(Attach Listener)" RUNNABLE
null
"Finalizer" RUNNABLE
     jrockit.memory.Finalizer.waitForFinalizees(Native Method)
     jrockit.memory.Finalizer.access$700(Finalizer.java:12)
     jrockit.memory.Finalizer$4.run(Finalizer.java:189)
     java.lang.Thread.run(Thread.java:619)
"Reference Handler" RUNNABLE
     java.lang.ref.Reference.waitForActivatedQueue(Native Method)
     java.lang.ref.Reference.access$100(Reference.java:11)
     java.lang.ref.Reference$ReferenceHandler.run(Reference.java:82)
"(Sensor Event Thread)" RUNNABLE
null
"VM JFR Buffer Thread" RUNNABLE native
null
"Timer-0" waiting for lock java.util.TaskQueue@e259a7 WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     java.util.TimerThread.mainLoop(Timer.java:483)
     java.util.TimerThread.run(Timer.java:462)
"Timer-1" waiting for lock java.util.TaskQueue@10ad080 TIMED_WAITING
     java.lang.Object.wait(Native Method)
     java.util.TimerThread.mainLoop(Timer.java:509)
     java.util.TimerThread.run(Timer.java:462)
"[STANDBY] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.work.ExecuteThread@10ad1bd WAITING
     java.lang.Object.wait(Object.java:485)
     weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:157)
     weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
"weblogic.time.TimeEventGenerator" waiting for lock weblogic.time.common.internal.TimeTable@10aff60 TIMED_WAITING
     weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:286)
     weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:117)
     java.lang.Thread.run(Thread.java:619)
"weblogic.timers.TimerThread" waiting for lock weblogic.timers.internal.TimerThread@11216cb TIMED_WAITING
     java.lang.Object.wait(Native Method)
     weblogic.timers.internal.TimerThread$Thread.run(TimerThread.java:267)
"[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.work.ExecuteThread@10ad3e0 WAITING
     java.lang.Object.wait(Object.java:485)
     weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:157)
     weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
"Thread-7" waiting for lock java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@10ad595 WAITING
     sun.misc.Unsafe.park(Native Method)
     java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
     java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
     java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
     weblogic.utils.concurrent.JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
     weblogic.store.internal.PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:567)
     weblogic.store.internal.PersistentStoreImpl.run(PersistentStoreImpl.java:615)
     weblogic.store.internal.PersistentStoreImpl$2.run(PersistentStoreImpl.java:383)
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" RUNNABLE native
     weblogic.socket.NTSocketMuxer.getIoCompletionResult(Native Method)
     weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
     weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" RUNNABLE native
     weblogic.socket.NTSocketMuxer.getIoCompletionResult(Native Method)
     weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
     weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" RUNNABLE native
     weblogic.socket.NTSocketMuxer.getIoCompletionResult(Native Method)
     weblogic.socket.NTSocketMuxer.processSockets(NTSocketMuxer.java:81)
     weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
     weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
     weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
     weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
"VDE Transaction Processor Thread" waiting for lock [email protected] WAITING
     java.lang.Object.wait(Native Method)
     java.lang.Object.wait(Object.java:485)
     com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:367)
     com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212)
"VDE Replication Thread" waiting for lock com.octetstring.vde.replication.Replication@10ad426 TIMED_WAITING
     com.octetstring.vde.replication.Replication.wait30sec(Replication.java:384)
     com.octetstring.vde.replication.Replication.run(Replication.java:350)
"DoSManager" TIMED_WAITING
     java.lang.Thread.sleep(Native Method)
     com.octetstring.vde.DoSManager.run(DoSManager.java:433)
"Thread-11" waiting for lock java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4f576f WAITING
     sun.misc.Unsafe.park(Native Method)

Upgrading to PT 8.49.14 - PermGen problem with weblogic console

All Peoplesoft domain Comes you fine in our env setup (Admin, PIA1, and PIA2).
When we try to bring up the console, we're able to get the initial signon screen.
We enter the account and password, hit the signon button, and it never comes back.
The log ends up showing a PermGen error:
hrdmo@peo1asdv-z1:/psa1asdv/apps/hrdmo/hr890/webserv/hrdmo/servers/WebLo
gicAdmin/logs> more WebLogicAdmin_stderr.log
Exception in thread "VDE Replication Thread" java.lang.OutOfMemoryError:
PermGen space
Exception in thread "[STANDBY] ExecuteThread: '6' for queue:
'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError:
PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '0' for queue:
'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError:
PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '7' for queue:
'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError:
PermGen space
Changed Java heap from Solution:
from:
JAVA_OPTIONS_SOLARIS="-server -Xms512m -Xmx512m -XX:MaxPermSize=256m
-Dtoplink.xml.platform=oracle.toplink.platform.xml.jaxp.JAXPPlatform"
To:
JAVA_OPTIONS_SOLARIS="-server -Xms1024m -Xmx1024m -XX:MaxPermSize=256m
-Dtoplink.xml.platform=oracle.toplink.platform.xml.jaxp.JAXPPlatform"
Changed the adminserver:
from:
JAVA_OPTIONS_ADMINSERVER="-Xms256m -Xmx1024m"
to:
JAVA_OPTIONS_ADMINSERVER="-Xms1024m -Xmx1024m"
still getting PermGen space error, any thoughts?
Env setup: Solaris 10, WLS 92 MP3 patch1, PTools 849.14
Thanks

Increasing PermGen size doesn't solve problem at all. Source of the problem is related with class loader. After you web app is undeployed class loader can not unload all classes from PermGen memory. This leads to extra memory consumption after every undeploy, deploy cycle. Make sure every class can be released and undeployed after undeploy.
Regards
Remis B
Remigijus Bauzys
Java EE freelancer
IT Consultant.

Is this a bug in __env_close_pp?

I'm using Berkeley DB 4.6.21.
in __env_close_pp, this function calls __repmgr_close, then __env_close, and
__env_close calls __rep_env_close.
__repmgr_close will join any replication threads(selector, messengers, and elect_thread).
But __rep_env_close will send any outstanding bulk buffers when close, my question is that the relevant threads are over already, all the connection fd were closed, the sending function may return error.
I do an experiment to verify this, and enable DB_REP_CONF_BULK in replication config, when the master node exits,
the program will complain occasionally :socket writing failure: Bad file descriptor.
Though the error seems was not fatal and trivial.
It seems that the error message was thrown from __repmgr_send_internal, and it was called from __repmgr_send (via __repmgr_send_broadcast and __repmgr_send_one), and __repmgr_send was the sending handler of replication handler.

Thanks for your reply.
Please forgive me,I am pretty busy in these days.
My OS is
6.2-RELEASE FreeBSD 6.2-RELEASE #0: Fri Jan 12 10:40:27 UTC 2007.
I modified the official examples in db-4.6.21/examples_c/ex_rep/mgr as following:
1. enable bulk in replication:
dbenv->rep_set_config(dbenv,DB_REP_CONF_BULK,1);
2. enable verbose in replication
add -v option to command line;
3. catch SIGINT
when SIGINT happens, "goto err" to close databases(break the "for(;;)" loop in rep_common.c), return and close the env.
Running the replication in 2 nodes , after election finishes, input something, the client will sync with the master, then Ctrl+C to stop master process:
QUOTESERVER> c 4
ex_rep: bulk_msg: Copying LSN [1][3051] of 73 bytes to 0x28f4ff0c
ex_rep: bulk_msg: Copying LSN [1][3136] of 32 bytes to 0x28f4ff61
ex_rep: bulk_msg: Send buffer after copy due to PERM
ex_rep: send_bulk: Send 165 (0xa5) bulk buffer bytes
ex_rep: env2 rep_send_message: msgv = 4 logv 13 gen = 7 eid -1, type bulk_log, LSN [1][3136] perm
ex_rep: will await acknowledgement: need 0
ex_rep: got ack [1][3136](7) from site localhost:8080
ex_rep: bulk_msg: Copying LSN [1][3180] of 53 bytes to 0x28f4fee8
ex_rep: bulk_msg: Copying LSN [1][3245] of 53 bytes to 0x28f4ff29
ex_rep: bulk_msg: Copying LSN [1][3310] of 68 bytes to 0x28f4ff6a
ex_rep: bulk_msg: Send buffer after copy due to PERM
ex_rep: send_bulk: Send 210 (0xd2) bulk buffer bytes
ex_rep: env2 rep_send_message: msgv = 4 logv 13 gen = 7 eid -1, type bulk_log, LSN [1][3310] perm
ex_rep: will await acknowledgement: need 0
ex_rep: got ack [1][3310](7) from site localhost:8080
QUOTESERVER> ^Cex_rep: bulk_msg: Copying LSN [1][3390] of 73 bytes to 0x28f4fee8
ex_rep: bulk_msg: Copying LSN [1][3475] of 44 bytes to 0x28f4ff3d
ex_rep: bulk_msg: Send buffer after copy due to PERM
ex_rep: send_bulk: Send 141 (0x8d) bulk buffer bytes
ex_rep: env2 rep_send_message: msgv = 4 logv 13 gen = 7 eid -1, type bulk_log, LSN [1][3475] flush perm
ex_rep: will await acknowledgement: need 0
ex_rep: got ack [1][3475](7) from site localhost:8080
Symbol Price
====== =====
a 2
b 3
c 4
ex_rep: bulk_msg: Copying LSN [1][3531] of 73 bytes to 0x28f4fee8
ex_rep: Stopping repmgr threads
ex_rep: Repmgr threads are finished
ex_rep: send_bulk: Send 85 (0x55) bulk buffer bytes
ex_rep: env2 rep_send_message: msgv = 4 logv 13 gen = 7 eid -1, type bulk_log, LSN [0][0] nobuf
ex_rep: socket writing failure: Bad file descriptor
ex_rep: rep_send_function returned: 9

ISE 1.1.3.124 secondary node not reachable after registration

G'day All,
I'm constantly seeing that the sync and replication status for my secondary admin/monitor node in the primary node as node not reachable. The secondary still thinks it is in standalone mode. When I run the ISE diag tool connectivity tests I am able successfully ping the devices from each other using both hostname and ip and the nslookup also works fine between both nodes. Ping and nslookups also work from different networks within the environment. The two nodes are in the same vlan on a 6500 vss pair but on different switches of the pair. I'm new to ISE so any help is greatly appreciated.
Thanks All.
JS
Sent from Cisco Technical Support iPhone App

Hi Saurav,
Thanks for your prompt repsonse...
I have worked through that section of the document. The registration completes successfully, I've got NTP sync on both nodes and the system time on both nodes is identical.
I am only using the self signed certificates, but following the user guide instructions I have imported the secondary's cert into the primary node.
Just as of about 30 minutes ago, I saw an alarm on the Secondary ISE node stating that a Slow or Stuck Replication has been detected...
As I said in the original post, I can ping the fqdn's from each other so it appears that the DNS requirements have been satisfied.
I've changed the admin account password, I am certain that the ISE DB passwords are correct and the same on both nodes and the timezones for both nodes is the same also....
It looks to me that registration is fine, but the first full replication isn't completing successfully
Thanks,
JS

Help with StuckThread in java.lang.String.equals(String.java:619)

I periodically get a stuck Execute thread on my Weblogic 8.15 server
during a call to a Stateless Session Bean. When I do thread dumps, the
thread is always stuck in the same place (in
java.lang.String.equals(String.java:619) or in the call immediately above it
java.util.LinkedList.indexOf(LinkedList.java:397)). Even though the thread
dump indicates that the thread is in a runnable state, if I do multiple
thread dumps over a period of time, the stack trace always indicates that
the thread is in the same place. The thread remains stuck until Weblogic is
restarted. Other client applictions can make session bean calls, but each
stuck thread seems to still take up lots of CPU time. I have let the stuck
threads run overnight, and the stack trace from the thread dump always shows
them executing the same String/LinkedList code. In each case, our code is
trying to iterate over a collection
Does anybody know what could cause this problem, and how to fix it? I get
StuckThreadMaxTime errors in the log:
####<Dec 4, 2005 10:47:25 AM EST> <Error> <WebLogicServer> <nybill>
<myserver> <weblogic.health.CoreHealthMonitor> <<WLS Kernel>> <>
<BEA-000337> <ExecuteThread: '4' for queue: 'weblogic.kernel.Default' has
been busy for "1,263" seconds working on the request
"ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl", which
is more than the configured time (StuckThreadMaxTime) of "1,200" seconds.>
Here are some stack traces from different thread dumps (I have the full
thread dumps if necessary):
"ExecuteThread: '10' for queue: 'weblogic.kernel.Default'" daemon prio=5
tid=0x7720eb98 nid=0xd68 runnable [571f000..571fdb0]
at java.lang.String.equals(String.java:619)
at java.util.LinkedList.indexOf(LinkedList.java:398)
at java.util.LinkedList.contains(LinkedList.java:176)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getDailyCallSummary(XMLBillCr
eation.java:1992)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getGraphs(XMLBillCreation.jav
a:1931)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getLocations(XMLBillCreation.
java:2618)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.createXMLBill(XMLBillCreation
.java:236)
at
ncss.billing.ejb.billAdmin.session.BillAdminSessionEJB.getBillAsXml(BillAdmi
nSessionEJB.java:341)
at
ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl.getBillAsX
ml(BillAdminSession_uli3xb_EOImpl.java:100)
at
ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl_WLSkel.inv
oke(Unknown Source)
at
weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:477)
at
weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java
:108)
at
weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
at
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
t.java:363)
at
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at
weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
at
weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:3
0)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
"ExecuteThread: '22' for queue: 'weblogic.kernel.Default'" daemon prio=5
tid=0x772538d0 nid=0xe24 runnable [497f000..4fdb0]
at java.lang.String.equals(String.java:619)
at java.util.LinkedList.indexOf(LinkedList.java:398)
at java.util.LinkedList.contains(LinkedList.java:176)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getMostExpensiveOrLongestCall
s(XMLBillCreation.java:1892)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getTopTenReport(XMLBillCreati
on.java:1798)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getTopTenReports(XMLBillCreat
ion.java:1751)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getLocations(XMLBillCreation.
java:2612)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.createXMLBill(XMLBillCreation
.java:236)
at
ncss.billing.ejb.billAdmin.session.BillAdminSessionEJB.getBillAsXml(BillAdmi
nSessionEJB.java:341)
at
ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl.getBillAsX
ml(BillAdminSession_uli3xb_EOIm.java:100)
at
ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl_WLSkel.inv
oke(Unknown Source)
at
weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:477)
at
weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java
:108)
at
weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
at
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
t.java:363)
at
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at
weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
at
weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:3
0)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
"ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon prio=5
tid=0x76afb060 nid=0x498 runnable [48af000..48fdb0]
at java.lang.String.equals(String.java:619)
at java.util.LinkedList.indexOf(LinkedList.java:398)
at java.util.LinkedList.contains(LinkedList.java:176)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getMostFrequentlyCalledNumber
sOrCitiesReport(XMLBillCreation.java:1839)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getTopTenReport(XMLBillCreati
on.java:1772)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getTopTenReports(XMLBillCreat
ion.java:1743)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.getLocations(XMLBillCreation.
java:2612)
at
ncss.billing.broadviewBill.xml.XMLBillCreation.createXMLBill(XMLBillCreation
.java:236)
at
ncss.billing.ejb.billAdmin.session.BillAdminSessionEJB.getBillAsXml(BillAdmi
nSessionEJB.java:341)
at
ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl.getBillAsX
ml(BillAdminSession_uli3xb_EOImp.java:100)
at
ncss.billing.ejb.billAdmin.session.BillAdminSession_uli3xb_EOImpl_WLSkel.inv
oke(Unknown Source)
at
weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:477)
at
weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java
:108)
at
weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
at
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
t.java:363)
at
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at
weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
at
weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:3
0)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
The code in LinkedList that it seems to be executing is the following:
Line#
397 for (Entry e = header.next; e != header; e = e.next) {
398 if (o.equals(e.element))
399 return index;
400 index++;
401 }
I am running Weblogic 8.15 on Windows 2000.
Thanks for any help,
- Don

njb7ty wrote:
I suggest dropping that example program and concentrating on reading a book on Java such as 'Head First in Java'. Otherwise, you will spend a lot of time trying to get something to work and gain little value from it.Likewise... Jumping into reflections before you can [read a stack-trace|http://www.0xcafefeed.com/2004/06/of-thread-dumps-and-stack-traces/] is like signing up a toddler for the New York Marathon... it's probably simply beyond your skill level... so step back... go read a book, do some tutorials, get your head around just the process of the designing, writing, compiling, running, and debugging java programs... and what the different diagnostics mean... Then, equipped with your nose-clip and your trusty stone ;-) you contemplate leaping into the deep end ;-)
Cheers. Keith.

Please help me in analysing JVM crash

Guys,
Can some one help me in analyzing this hotspot err.log :
# An unexpected error has been detected by Java Runtime Environment:
# SIGSEGV (0xb) at pc=0xfec2af88, pid=3772, tid=110
# Java VM: Java HotSpot(TM) Server VM (11.0-b16 mixed mode solaris-sparc)
# Problematic frame:
# V [libjvm.so+0x42af88]
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
--------------- T H R E A D ---------------
Current thread (0x017de400): JavaThread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_in_vm, id=110, stack(0xcc480000,0xcc500000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000010
Registers:
O0=0xcc4f6b08 O1=0xcc4f6b20 O2=0xcc4f6b20 O3=0x00000000
O4=0xfc000224 O5=0xcc0f8158 O6=0xcc4f6aa8 O7=0xfef73384
G1=0xfc00021c G2=0x00000006 G3=0xfc400000 G4=0x00000000
G5=0xfc000000 G6=0x00000000 G7=0xfe066200 Y=0x00000000
PC=0xfec2af88 nPC=0xfec2af8c
Top of Stack: (sp=0xcc4f6aa8)
0xcc4f6aa8: fc00022c ff139a80 0003ba80 0003b800
0xcc4f6ab8: fc005eb8 fffffffc ff13c524 027141c8
0xcc4f6ac8: 027141a8 ff139a80 cc4f6b24 0003ba80
0xcc4f6ad8: 027141ac ff0fe000 cc4f6d68 fef733c0
0xcc4f6ae8: 00000000 ff1491f4 00000000 0000000e
0xcc4f6af8: 00000000 00000001 00000000 c58bdd66
0xcc4f6b08: cc0f8158 fc000224 fc000108 00000000
0xcc4f6b18: cc0f80d0 00000000 ff13c524 cc4febd0
Instructions: (pc=0xfec2af88)
0xfec2af78: 81 c7 e0 08 81 e8 20 00 da 02 20 00 d6 03 60 20
0xfec2af88: d4 02 e0 10 92 20 00 0a 8b 32 70 3f 81 c3 e0 08
Stack: [0xcc480000,0xcc500000], sp=0xcc4f6aa8, free space=474k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x42af88]
V [libjvm.so+0x7733c8]
V [libjvm.so+0x2d60a8]
V [libjvm.so+0x2d6410]
V [libjvm.so+0x2d72f8]
V [libjvm.so+0x70aa6c]
V [libjvm.so+0x11a308]
j java.util.Collections$SynchronizedMap.get(Ljava/lang/Object;)Ljava/lang/Object;+7
j weblogic.management.jmx.ObjectNameManagerBase.lookupRegisteredObjectName(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.lookupOrRegister(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.mapToJMX(Ljava/lang/Class;Ljava/lang/Object;Ljavax/management/openmbean/OpenType;)Ljava/lang/Object;+290
j weblogic.management.jmx.modelmbean.WLSModelMBean.getAttribute(Ljava/lang/String;)Ljava/lang/Object;+182
j com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+107
j com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+10
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
v ~StubRoutines::call_stub
V [libjvm.so+0x134704]
V [libjvm.so+0x192b74]
C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
v ~StubRoutines::call_stub
V [libjvm.so+0x134704]
V [libjvm.so+0x192b74]
C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
v ~StubRoutines::call_stub
V [libjvm.so+0x134704]
V [libjvm.so+0x192b74]
C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
v ~StubRoutines::call_stub
V [libjvm.so+0x134704]
V [libjvm.so+0x192b74]
C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
j weblogic.management.mbeanservers.internal.SecurityInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+9
j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5$1.run()Ljava/lang/Object;+18
v ~StubRoutines::call_stub
V [libjvm.so+0x134704]
V [libjvm.so+0x192b74]
C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5.run()Ljava/lang/Object;+20
j weblogic.security.acl.internal.AuthenticatedSubject.doAs(Lweblogic/security/subject/AbstractSubject;Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+34
j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+22
j weblogic.management.jmx.mbeanserver.WLSMBeanServer.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+10
j javax.management.remote.rmi.RMIConnectionImpl.doOperation(I[Ljava/lang/Object;)Ljava/lang/Object;+268
j javax.management.remote.rmi.RMIConnectionImpl.access$200(Ljavax/management/remote/rmi/RMIConnectionImpl;I[Ljava/lang/Object;)Ljava/lang/Object;+3
j javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run()Ljava/lang/Object;+12
v ~StubRoutines::call_stub
V [libjvm.so+0x134704]
V [libjvm.so+0x192b74]
C [libjava.so+0xf5d8] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2Ljava_security_AccessControlContext_2+0x1c
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+32
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0
j javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(I[Ljava/lang/Object;Ljavax/security/auth/Subject;)Ljava/lang/Object;+121
j javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+77
j javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(I[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+411
j weblogic.rmi.internal.ServerRequest.sendReceive()Lweblogic/rmi/spi/InboundResponse;+216
j weblogic.rmi.internal.BasicRemoteRef.invoke(Ljava/rmi/Remote;Lweblogic/rmi/extensions/server/RuntimeMethodDescriptor;[Ljava/lang/Object;Ljava/lang/reflect/Method;)Ljava/lang/Object;+129
j javax.management.remote.rmi.RMIConnectionImpl_1030_WLStub.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+50
j javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+66
j javax.management.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+162
j weblogic.management.jmx.MBeanServerInvocationHandler.doInvoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+219
j weblogic.management.jmx.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+495
j $Proxy60.getConfigurationManager()Lweblogic/management/mbeanservers/edit/ConfigurationManagerMBean;+9
j com.bea.console.utils.MBeanChangeManager.getCurrentLockOwner()Ljava/lang/String;+7
j com.bea.console.utils.MBeanChangeManager.isCurrentLockOwner(Ljavax/servlet/http/HttpServletRequest;)Z+0
j jsp_servlet._jsp._contentheader.__contentmenu._jspService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+443
j weblogic.servlet.jsp.JspBase.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+9
j weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run()Ljava/lang/Object;+43
j weblogic.servlet.internal.StubSecurityHelper.invokeServlet(Ljavax/servlet/ServletRequest;Ljavax/servlet/http/HttpServletRequest;Lweblogic/servlet/internal/ServletRequestImpl;Ljavax/servlet/ServletResponse;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/Servlet;)Ljava/lang/Throwable;+110
j weblogic.servlet.internal.ServletStubImpl.execute(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/FilterChainImpl;)V+356
j weblogic.servlet.internal.ServletStubImpl.onAddToMapException(Ljava/lang/Throwable;Lweblogic/servlet/internal/ServletRequestImpl;Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/FilterChainImpl;)V+261
j weblogic.servlet.internal.ServletStubImpl.execute(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/FilterChainImpl;)V+382
j weblogic.servlet.internal.ServletStubImpl.execute(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+4
j weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(ZLjavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/ServletResponseImpl;I)V+105
j weblogic.servlet.internal.RequestDispatcherImpl.include(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+495
j com.bea.netuix.servlets.controls.content.JspContent.beginRender(Ljava/io/Writer;)Z+854
j com.bea.netuix.nf.ControlLifecycle$7.visit(Lcom/bea/netuix/nf/UIControl;Ljava/lang/Object;)Z+94
j com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Ljava/lang/Object;)V+568
j com.bea.netuix.nf.ControlTreeWalker.walk(Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;)V+535
j com.bea.netuix.nf.Lifecycle.processLifecycles(II[Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/UIContext;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;Z)I+147
j com.bea.netuix.nf.Lifecycle.processLifecycles(I[Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/UIContext;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;)I+11
j com.bea.netuix.nf.Lifecycle.processLifecycles(I[Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;)I+12
...<more frames>...
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j java.util.Collections$SynchronizedMap.get(Ljava/lang/Object;)Ljava/lang/Object;+7
j weblogic.management.jmx.ObjectNameManagerBase.lookupRegisteredObjectName(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.lookupOrRegister(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.mapToJMX(Ljava/lang/Class;Ljava/lang/Object;Ljavax/management/openmbean/OpenType;)Ljava/lang/Object;+290
j weblogic.management.jmx.modelmbean.WLSModelMBean.getAttribute(Ljava/lang/String;)Ljava/lang/Object;+182
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
v ~StubRoutines::call_stub
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
j weblogic.management.mbeanservers.internal.SecurityInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+9
j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5$1.run()Ljava/lang/Object;+18
v ~StubRoutines::call_stub
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5.run()Ljava/lang/Object;+20
j weblogic.security.acl.internal.AuthenticatedSubject.doAs(Lweblogic/security/subject/AbstractSubject;Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+34
j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+22
j weblogic.management.jmx.mbeanserver.WLSMBeanServer.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+10
j javax.management.remote.rmi.RMIConnectionImpl.doOperation(I[Ljava/lang/Object;)Ljava/lang/Object;+268
j javax.management.remote.rmi.RMIConnectionImpl.access$200(Ljavax/management/remote/rmi/RMIConnectionImpl;I[Ljava/lang/Object;)Ljava/lang/Object;+3
j javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run()Ljava/lang/Object;+12
v ~StubRoutines::call_stub
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0
j javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(I[Ljava/lang/Object;Ljavax/security/auth/Subject;)Ljava/lang/Object;+121
j javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+77
j javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(I[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+411
j weblogic.rmi.internal.ServerRequest.sendReceive()Lweblogic/rmi/spi/InboundResponse;+216
j weblogic.rmi.internal.BasicRemoteRef.invoke(Ljava/rmi/Remote;Lweblogic/rmi/extensions/server/RuntimeMethodDescriptor;[Ljava/lang/Object;Ljava/lang/reflect/Method;)Ljava/lang/Object;+129
j javax.management.remote.rmi.RMIConnectionImpl_1030_WLStub.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+50
j javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+66
j javax.management.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+162
j weblogic.management.jmx.MBeanServerInvocationHandler.doInvoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+219
j weblogic.management.jmx.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+495
j $Proxy60.getConfigurationManager()Lweblogic/management/mbeanservers/edit/ConfigurationManagerMBean;+9
j com.bea.console.utils.MBeanChangeManager.getCurrentLockOwner()Ljava/lang/String;+7
j com.bea.console.utils.MBeanChangeManager.isCurrentLockOwner(Ljavax/servlet/http/HttpServletRequest;)Z+0
j jsp_servlet._jsp._contentheader.__contentmenu._jspService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+443
j weblogic.servlet.jsp.JspBase.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+9
j weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run()Ljava/lang/Object;+43
j weblogic.servlet.internal.StubSecurityHelper.invokeServlet(Ljavax/servlet/ServletRequest;Ljavax/servlet/http/HttpServletRequest;Lweblogic/servlet/internal/ServletRequestImpl;Ljavax/servlet/ServletResponse;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/Servlet;)Ljava/lang/Throwable;+110
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x021e1000 JavaThread "Thread-12" daemon [_thread_in_native, id=223, stack(0xc5280000,0xc5300000)]
0x031f4800 JavaThread "[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=222, stack(0xc5380000,0xc5400000)]
0x0323c400 JavaThread "[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=221, stack(0xc5480000,0xc5500000)]
0x0323d000 JavaThread "[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=220, stack(0xc5580000,0xc5600000)]
0x01e39000 JavaThread "[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=219, stack(0xc5680000,0xc5700000)]
0x01e39c00 JavaThread "DynamicListenThread[Default]" daemon [_thread_in_native, id=218, stack(0xc5780000,0xc5800000)]
0x01b7a400 JavaThread "weblogic.GCMonitor" daemon [_thread_blocked, id=217, stack(0xc5880000,0xc5900000)]
0x031ff000 JavaThread "Thread-11" daemon [_thread_blocked, id=216, stack(0xc5c80000,0xc5d00000)]
0x02996400 JavaThread "DoSManager" daemon [_thread_blocked, id=215, stack(0xc5b80000,0xc5c00000)]
0x02997000 JavaThread "VDE Replication Thread" daemon [_thread_blocked, id=213, stack(0xc5d80000,0xc5e00000)]
0x02997400 JavaThread "VDE Transaction Processor Thread" daemon [_thread_blocked, id=212, stack(0xc5e80000,0xc5f00000)]
0x02997c00 JavaThread "ExecuteThread: '96' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=211, stack(0xc5f80000,0xc6000000)]
0x0298a400 JavaThread "ExecuteThread: '95' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=210, stack(0xc6080000,0xc6100000)]
0x0298a800 JavaThread "ExecuteThread: '94' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=209, stack(0xc6180000,0xc6200000)]
0x0298b000 JavaThread "ExecuteThread: '93' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=208, stack(0xc6280000,0xc6300000)]
0x0298b400 JavaThread "ExecuteThread: '92' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=207, stack(0xc6380000,0xc6400000)]
0x0298bc00 JavaThread "ExecuteThread: '91' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=206, stack(0xc6480000,0xc6500000)]
0x02980400 JavaThread "ExecuteThread: '90' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=205, stack(0xc6580000,0xc6600000)]
0x02980800 JavaThread "ExecuteThread: '89' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=204, stack(0xc6680000,0xc6700000)]
0x02981000 JavaThread "ExecuteThread: '88' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=203, stack(0xc6780000,0xc6800000)]
0x02981400 JavaThread "ExecuteThread: '87' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=202, stack(0xc6880000,0xc6900000)]
0x02981c00 JavaThread "ExecuteThread: '86' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=201, stack(0xc6980000,0xc6a00000)]
0x02974400 JavaThread "ExecuteThread: '85' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=200, stack(0xc6a80000,0xc6b00000)]
0x02974800 JavaThread "ExecuteThread: '84' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=199, stack(0xc6b80000,0xc6c00000)]
0x02975000 JavaThread "ExecuteThread: '83' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=198, stack(0xc6c80000,0xc6d00000)]
0x02975400 JavaThread "ExecuteThread: '82' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=197, stack(0xc6d80000,0xc6e00000)]
0x02975c00 JavaThread "ExecuteThread: '81' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=196, stack(0xc6e80000,0xc6f00000)]
0x02956400 JavaThread "ExecuteThread: '80' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=195, stack(0xc6f80000,0xc7000000)]

Cross posted multiple times.
[http://forums.sun.com/thread.jspa?threadID=5369763&messageID=10622671#10622671]

Stuck replication thread

Similar Messages

Maybe you are looking for