PAPI client calls - stuck threads with albpm57 engine

PAPI client running on weblogic portal 10.3, connecting to ALBPM 5.7 engine (on weblogic 8.1 cluster).
threads get stuck in WLP servers (where PAPI client runs), and the call is for getActiveInstances from default view. We have around 15000 active instances, and the papi cache size is configured for 25000.
"[STUCK] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" id=17 idx=0x7c tid=6615 prio=1 alive, in native, waiting, daemon
-- Waiting for notification on: fuego/papi/impl/lock/SimpleLock@0x1f443568[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at java/lang/Object.wait(Object.java:485)
at fuego/papi/impl/lock/SimpleLock.readLock(SimpleLock.java:50)
^-- Lock released while waiting: fuego/papi/impl/lock/SimpleLock@0x1f443568[fat lock]
at fuego/papi/impl/InstanceCache$Entry.readLock(InstanceCache.java:708)
at fuego/papi/impl/InstanceCache.getInstancesByFilter(InstanceCache.java:505)
at fuego/papi/impl/InstanceCache.getInstancesByFilter(InstanceCache.java:231)
at fuego/papi/impl/ProcessServiceImpl.getInstancesByFilter(ProcessServiceImpl.java:1084)
at fuego/papi/impl/ProcessServiceNESessionImpl.getInstancesByFilter(ProcessServiceNESessionImpl.java:754)
Please help how to resolve this.
thanks in advance.
Edited by: 811778 on May 17, 2013 5:05 PM

support team suggested to use,
-Dfuego.papi.lockManagerClass=fuego.papi.impl.lock.ParanoidLockManager for each java process, acting as PAPI client. This class available in fuegopapi-client.jar
additional comments from support team,
Implementing this property does not generate side-effects. This property detects and kills/recovers deadlocks in the PAPI instance cache

Similar Messages

  • Connect Java PAPI client to 5.7 Studio embedded engine?

    I am coding and unit testing some integration of java PAPI client code against a process inALBPM 5.7. But, it is very cumbersome to have to do a fully deploy to a bpm server to do this unit testing.
    Is it possible to configure the properties used to create the ProcessService object to connect to the engine that is launched from within the Studio so that I can test locally on my PC? What would the directory.properties have to look like to do so?
    Todd

    Hi Adam,
    Could you please explain more about this?
    I want to use PAPI against my local BPM studio. I have the project at "C:\OracleBPMWorkspace\TestProcess". I tried to point the directory to the project, as you suggested. Am i doing it correctly? I get the below exception. Could somebody help me?
    Thanks
    raj
    Configuration:
    Properties configuration = new Properties();
    configuration.setProperty(ProcessService.DIRECTORY_ID, "Default");
    configuration.setProperty(ProcessService.DIRECTORY_PROPERTIES_FILE, "C:\\OracleBPMWorkspace\\TestProcess");
    configuration.setProperty(ProcessService.WORKING_FOLDER, "/tmp");
    Exception:
    java.lang.IllegalStateException: directory configuration runtime fails to initilize with resource: C:\OracleBPMWorkspace\TestProcess
         at fuego.directory.DirectoryConfigurationManager.getDirectoryConfiguration(DirectoryConfigurationManager.java:335)
         at fuego.directory.DirectoryConfigurationManager.getDirectoryConfigType(DirectoryConfigurationManager.java:313)
         at fuego.directory.provider.Factory.getDirectoryFactory(Factory.java:216)
         at fuego.directory.Directory.startAnonymousSession(Directory.java:211)
         at fuego.papi.impl.ProcessServiceFactoryImpl.obtainSchemaId(ProcessServiceFactoryImpl.java:193)
         at fuego.papi.impl.ProcessServiceFactoryImpl.create(ProcessServiceFactoryImpl.java:75)
         at fuego.papi.impl.ProcessServiceFactoryImpl.create(ProcessServiceFactoryImpl.java:63)
         at fuego.papi.ProcessService.create(ProcessService.java:335)
         at com.thomson.tssq.j2012.bpms.PAPI_Test.main(PAPI_Test.java:35)

  • Problems calling VIs remotely with RT-engine

    I have problems when I call VIs remotely and I create executables to run with the Run Time Engine 7.0.
    My program consists of a server that show some data (to the user) calling a VI that store such data. Another independent part of the program call such VI (which store the data) by remotely calling it with the VI Server. If I run the 2 programs under Labview 7.0 (directly installed) all goes well, but when I create the 2 executables and I run them, the data appeare only in the server. In the client I don't see any data even if I have no error detected.
    In the past I used the same methodology with Labview 5.1 in some application and I never had such problem.
    Could someone help me ?
    Thanks in advance.
    Linus

    I send you (as attachment) a llb with 3 VIs (Labview 7.0) : a server VI that call directly the message VI, and a client VI that call the message VI indirectly.
    I have compiled the server VI into an exe program (using the option that create the data.llb file).
    After installing that program (in the directory C:\programmi\Test\ ) and running it (with the Run Time Engine), if now I run the client VI (in the same machine) the value of the message displayed by the panel is not the same of the server.
    If I also compile the client VI into an exe program and then I run it (with the Run Time Engine), I obtain the same result (different values displayed).
    Only if I run the server VI and the client VI directly under Labview (in the same machine) the two VIs
    display the same message.
    If I also run the compiled exe version of the server VI and the client VI in two different machines I obtain the same message displayed.
    Best regads,
    Linus
    Attachments:
    Test.llb ‏51 KB

  • Need help with STUCK Threads.  What's causing them?

    In our production environment, we have a soa suite cluster with 3 nodes. All 3 of the servers keep turning up stuck and hogging threads, and eventually they all grind to a halt where I can't even view composite instances through em. There is a bpel process that is using a dbadapter to poll for records every 5 seconds and then performs some service call outs and other dbadapters. This process is generating thousands of instances a day and I think it is the culprit. If I view the logs, I see one of the dbadapters that it calls out to fails occasionally with a PK violation. I looked at the developer's code, and they have no error handling in the process at all. In the logs, there will the a hit for the PK violation, then about 10 minutes later the a STUCK thread gets logged(the configured timeout is 600 seconds, or 10 minutes). Would not handling an error in a BPEL process hog/STUCK a thread? I need to be able to show what code is causing the issue so I can have a resolution path.
    I also see a note in the trace about something holding a lock:
    Holding lock: oracle.jdbc.driver.T4CConnection@1fbc3486[thin lock]....would that be an un-rolled back transaction?
    Here is the dump of the STUCK thread. Any help is greatly appreciated. Thanks!
    <Aug 5, 2011 10:30:02 AM EDT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "634" seconds working on the request "oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-517 "[STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
    jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
    jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
    java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
    java.net.SocketInputStream.read(SocketInputStream.java:107)
    oracle.net.nt.MetricsEnabledInputStream.read(TcpNTAdapter.java:553)
    oracle.net.ns.Packet.receive(Packet.java:249)
    oracle.net.ns.DataPacket.receive(DataPacket.java:94)
    oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:168)
    oracle.net.ns.NetInputStream.read(NetInputStream.java:113)
    oracle.net.ns.NetInputStream.read(NetInputStream.java:96)
    oracle.net.ns.NetInputStream.read(NetInputStream.java:81)
    oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:120)
    oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:75)
    oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1081)
    oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)
    oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:588)
    oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:70)
    oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:854)
    oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:918)
    oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1252)
    oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3541)
    oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3605)
    ^-- Holding lock: oracle.jdbc.driver.T4CConnection@1fbc3486[thin lock]
    oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1667)
    weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:123)
    org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:887)
    org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:541)
    org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:530)
    org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:521)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:202)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:192)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:254)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelect(DatasourceCallQueryMechanism.java:245)
    org.eclipse.persistence.queries.DataReadQuery.executeNonCursor(DataReadQuery.java:188)
    org.eclipse.persistence.queries.DataReadQuery.executeDatabaseQuery(DataReadQuery.java:141)
    org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:603)
    org.eclipse.persistence.queries.DataReadQuery.execute(DataReadQuery.java:115)
    org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:589)
    org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2898)
    org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1218)
    org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1190)
    org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1176)
    org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:378)
    org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:701)
    weblogic.deployment.TransactionalEntityManagerProxyImpl$QueryImpl.getSingleResult(TransactionalEntityManagerProxyImpl.java:232)
    com.collaxa.cube.persistence.dao.impl.jpa.JpaCubeInstanceDaoImpl.getNumberOfInstances(JpaCubeInstanceDaoImpl.java:439)
    oracle.fabric.CubeServiceEngine.getNumberOfComponentInstances(CubeServiceEngine.java:1962)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getNumberOfComponentInstances(FacadeFinderBeanImpl.java:1618)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBeanImpl.java:1653)
    sun.reflect.GeneratedMethodAccessor5852.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.jee.intercept.MethodInvocationInvocationContext.proceed(MethodInvocationInvocationContext.java:104)
    oracle.security.jps.ee.ejb.JpsAbsInterceptor$1.run(JpsAbsInterceptor.java:88)
    oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:299)
    oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:406)
    oracle.security.jps.wls.JpsWeblogicEjbInterceptor.runJaasMode(JpsWeblogicEjbInterceptor.java:61)
    oracle.security.jps.ee.ejb.JpsAbsInterceptor.intercept(JpsAbsInterceptor.java:81)
    oracle.security.jps.ee.ejb.JpsInterceptor.intercept(JpsInterceptor.java:101)
    sun.reflect.GeneratedMethodAccessor863.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    com.bea.core.repackaged.springframework.jee.intercept.JeeInterceptorInterceptor.invoke(JeeInterceptorInterceptor.java:66)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)
    weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:22)
    com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:49)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:86)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:151)
    $Proxy309.getComponentSnapShot(Unknown Source)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.java:1604)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl_WLSkel.invoke(Unknown Source)
    weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:567)
    weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:143)
    weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:425)
    weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:117)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
    >
    <Aug 5, 2011 10:30:02 AM EDT> <Notice> <Diagnostics> <BEA-320068> <Watch 'StuckThread' with severity 'Notice' on server 'soa_server1' has triggered at Aug 5, 2011 10:30:02 AM EDT. Notification details:
    WatchRuleType: Log
    WatchRule: (SEVERITY = 'Error') AND (MSGID = 'BEA-000337')
    WatchData: DATE = Aug 5, 2011 10:30:02 AM EDT SERVER = soa_server1 MESSAGE = [STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "634" seconds working on the request "oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-517 "[STUCK] ExecuteThread: '64' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
    jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
    jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
    java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
    java.net.SocketInputStream.read(SocketInputStream.java:107)
    oracle.net.nt.MetricsEnabledInputStream.read(TcpNTAdapter.java:553)
    oracle.net.ns.Packet.receive(Packet.java:249)
    oracle.net.ns.DataPacket.receive(DataPacket.java:94)
    oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:168)
    oracle.net.ns.NetInputStream.read(NetInputStream.java:113)
    oracle.net.ns.NetInputStream.read(NetInputStream.java:96)
    oracle.net.ns.NetInputStream.read(NetInputStream.java:81)
    oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:120)
    oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:75)
    oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1081)
    oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1062)
    oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:588)
    oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:70)
    oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:854)
    oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:918)
    oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1252)
    oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3541)
    oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3605)
    ^-- Holding lock: oracle.jdbc.driver.T4CConnection@1fbc3486[thin lock]
    oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1667)
    weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:123)
    org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:887)
    org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:541)
    org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:530)
    org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:521)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:202)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:192)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:254)
    org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelect(DatasourceCallQueryMechanism.java:245)
    org.eclipse.persistence.queries.DataReadQuery.executeNonCursor(DataReadQuery.java:188)
    org.eclipse.persistence.queries.DataReadQuery.executeDatabaseQuery(DataReadQuery.java:141)
    org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:603)
    org.eclipse.persistence.queries.DataReadQuery.execute(DataReadQuery.java:115)
    org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:589)
    org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2898)
    org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1218)
    org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1190)
    org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1176)
    org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:378)
    org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:701)
    weblogic.deployment.TransactionalEntityManagerProxyImpl$QueryImpl.getSingleResult(TransactionalEntityManagerProxyImpl.java:232)
    com.collaxa.cube.persistence.dao.impl.jpa.JpaCubeInstanceDaoImpl.getNumberOfInstances(JpaCubeInstanceDaoImpl.java:439)
    oracle.fabric.CubeServiceEngine.getNumberOfComponentInstances(CubeServiceEngine.java:1962)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getNumberOfComponentInstances(FacadeFinderBeanImpl.java:1618)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBeanImpl.java:1653)
    sun.reflect.GeneratedMethodAccessor5852.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.jee.intercept.MethodInvocationInvocationContext.proceed(MethodInvocationInvocationContext.java:104)
    oracle.security.jps.ee.ejb.JpsAbsInterceptor$1.run(JpsAbsInterceptor.java:88)
    oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:299)
    oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:406)
    oracle.security.jps.wls.JpsWeblogicEjbInterceptor.runJaasMode(JpsWeblogicEjbInterceptor.java:61)
    oracle.security.jps.ee.ejb.JpsAbsInterceptor.intercept(JpsAbsInterceptor.java:81)
    oracle.security.jps.ee.ejb.JpsInterceptor.intercept(JpsInterceptor.java:101)
    sun.reflect.GeneratedMethodAccessor863.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    com.bea.core.repackaged.springframework.jee.intercept.JeeInterceptorInterceptor.invoke(JeeInterceptorInterceptor.java:66)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)
    weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:22)
    com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:49)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:86)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:151)
    $Proxy309.getComponentSnapShot(Unknown Source)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.getComponentSnapShot(FacadeFinderBean_4vacyo_FacadeFinderBeanImpl.java:1604)
    oracle.soa.management.internal.ejb.impl.FacadeFinderBean_4vacyo_FacadeFinderBeanImpl_WLSkel.invoke(Unknown Source)
    weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:567)
    weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:143)
    weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:425)
    weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:117)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
    SUBSYSTEM = WebLogicServer USERID = <WLS Kernel> SEVERITY = Error THREAD = [ACTIVE] ExecuteThread: '96' for queue: 'weblogic.kernel.Default (self-tuning)' MSGID = BEA-000337 MACHINE = soaprod TXID = CONTEXTID = f5d0b85a929630a2:6eada0f7:1304e0a2bdc:-7ffb-00000000017e1605 TIMESTAMP = 1312554602698
    WatchAlarmType: AutomaticReset
    WatchAlarmResetPeriod: 600000
    >
    Edited by: Brutus35 on Aug 5, 2011 8:04 AM

    my impression is that you have trouble on the DB,
    monitor the sessions to see if there is some table locked, and increase the number of processes on the DB
    you can use SQLDeveloper for that
    anyway your TX should rollback after some time, check that you have a timeout in place, it's better to rollback a TX than having the entire server crash

  • Regarding Engine Run state using PAPI Client

    Is there anyway to find if ALBPM Execution engine is running using PAPI Client?
    Thank you.
    Regards,
    Ravi.

    Hi,
    IMHO the best way to find if the engine is up or down is by connecting through JMX,
    Hope this helps,
    Ariel

  • Stuck thread in adf application with weblogic

    hi every body
    I am working in JDev 11.1.1.4 and weblogic 10.3.4
    when I deploy my Application in my production weblogic
    after some time (it does not constant it is variable) I got warnning on the server
    when I check the server I got some stuck thread
    any idea or notes how can I know what is the reason of these stuck threads?
    is there any objects have thread unsafe issue, resource contention or race condition in ADF?
    BR,
    Alaa

    actully I do that guys
    but what I get does not have any related to my application
    even those http request does not request the same page
    this is the stuck thread when I dump
    "[STUCK] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" id=61 idx=0xe8 tid=13369 prio=1 alive, waiting, native_blocked, daemon
    -- Waiting for notification on: oracle/adfinternal/controller/util/RequestLock@0x88e06b58[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
    at java/lang/Object.wait(Object.java:485)[inlined]
    at oracle/adfinternal/controller/util/RequestLock.lock(RequestLock.java:42)[inlined]
    at oracle/adfinternal/controller/state/RootViewPortContextImpl.lockViewPortRequestLock(RootViewPortContextImpl.java:604)[optimized]
    ^-- Lock released while waiting: oracle/adfinternal/controller/util/RequestLock@0x88e06b58[fat lock]
    at oracle/adfinternal/controller/state/ControllerState.initializeRequest(ControllerState.java:833)[inlined]
    at oracle/adfinternal/controller/state/ControllerState.initializeRequest(ControllerState.java:754)[inlined]
    at oracle/adfinternal/controller/application/AdfcConfigurator.beginRequest(AdfcConfigurator.java:50)[optimized]
    at org/apache/myfaces/trinidadinternal/config/GlobalConfiguratorImpl._startConfiguratorServiceRequest(GlobalConfiguratorImpl.java:562)[inlined]
    at org/apache/myfaces/trinidadinternal/config/GlobalConfiguratorImpl.beginRequest(GlobalConfiguratorImpl.java:212)[optimized]
    at org/apache/myfaces/trinidadinternal/webapp/TrinidadFilterImpl.doFilter(TrinidadFilterImpl.java:155)[optimized]
    at org/apache/myfaces/trinidad/webapp/TrinidadFilter.doFilter(TrinidadFilter.java:92)[optimized]
    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
    at oracle/adf/library/webapp/LibraryFilter.doFilter(LibraryFilter.java:175)[optimized]
    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
    at oracle/security/jps/ee/http/JpsAbsFilter$1.run(JpsAbsFilter.java:111)[optimized]
    at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)[inlined]
    at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:313)[inlined]
    at oracle/security/jps/ee/util/JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:413)[inlined]
    at oracle/security/jps/ee/http/JpsAbsFilter.runJaasMode(JpsAbsFilter.java:94)[inlined]
    at oracle/security/jps/ee/http/JpsAbsFilter.doFilter(JpsAbsFilter.java:161)[optimized]
    at oracle/security/jps/ee/http/JpsFilter.doFilter(JpsFilter.java:71)[optimized]
    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
    at oracle/dms/servlet/DMSServletFilter.doFilter(DMSServletFilter.java:136)[optimized]
    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
    at weblogic/servlet/internal/RequestEventsFilter.doFilter(RequestEventsFilter.java:27)[optimized]
    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[inlined]
    at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3715)[inlined]
    at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)[optimized]
    at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized]
    at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:120)[inlined]
    at weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2277)[inlined]
    at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:2183)[optimized]
    at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1454)[optimized]
    at weblogic/work/ExecuteThread.execute(ExecuteThread.java:207)[optimized]
    at weblogic/work/ExecuteThread.run(ExecuteThread.java:176)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    any another ideas?
    BR,
    Alaa

  • Need some help with stucked threads

    Hi, I am running an ADF application on Weblogic 10.3.4. I get a lot of stucked threads and can't figure out the reason.
    Here is my stack dump:
    <31.mar.2011 kl 08.36 CEST> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "625" seconds working on the request "weblogic.servlet.internal.ServletRequestImpl@237aaa4c[
    POST /faces/hovedside?_adf.ctrl-state=13zq3miluz_4 HTTP/1.1
    Connection: keep-alive
    Referer: https://please.help.me/?_afrLoop=8410499084123&_afrWindowMode=0&_afrWindowId=null
    Content-Length: 356
    Adf-Ads-Page-Id: 3
    Origin: https://please.help.me
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16
    Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    Adf-Rich-Message: true
    Accept: */*
    Accept-Encoding: gzip,deflate,sdch
    Accept-Language: nb-NO,nb;q=0.8,no;q=0.6,nn;q=0.4,en-US;q=0.2,en;q=0.2
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
    Cookie: JSESSIONID=1dQ7NJpXZhYGM11Sf2ppDz4LZwQM5FpL6yxDrj0cSlMTsTdWvPPN!-222523808
    ]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-133 "[STUCK] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, suspended, priority=1, DAEMON> {
    java.util.ArrayList.clone(ArrayList.java:250)
    oracle.jbo.uicli.binding.JUCtrlHierNodeBinding.getChildren(JUCtrlHierNodeBinding.java:719)
    oracle.jbo.uicli.binding.JUCtrlHierNodeBinding.findChildNode(JUCtrlHierNodeBinding.java:844)
    oracle.jbo.uicli.binding.JUCtrlHierBinding.bringNodeToRangeKeyPath(JUCtrlHierBinding.java:772)
    oracle.adfinternal.view.faces.model.binding.FacesCtrlHierBinding.bringNodeToRangeKeyPath(FacesCtrlHierBinding.java:101)
    oracle.adfinternal.view.faces.model.binding.RowDataManager.setRowKey(RowDataManager.java:119)
    oracle.adfinternal.view.faces.model.binding.FacesCtrlHierBinding$FacesModel.setRowKey(FacesCtrlHierBinding.java:633)
    org.apache.myfaces.trinidad.model.RowKeySetTreeImpl$PathIterator.nextModelKey(RowKeySetTreeImpl.java:745)
    org.apache.myfaces.trinidad.model.RowKeySetTreeImpl$PathIterator.nextItem(RowKeySetTreeImpl.java:740)
    org.apache.myfaces.trinidad.model.RowKeySetTreeImpl$PathIterator.next(RowKeySetTreeImpl.java:721)
    oracle.adfinternal.view.faces.renderkit.rich.TreeRendererUtils.initModel(TreeRendererUtils.java:288)
    oracle.adfinternal.view.faces.renderkit.rich.TreeRendererUtils.wrapModel(TreeRendererUtils.java:527)
    oracle.adfinternal.view.faces.renderkit.rich.TreeRendererUtils.getCollectionObject(TreeRendererUtils.java:340)
    oracle.adfinternal.view.faces.renderkit.rich.TreeTableRenderer.getCollectionObject(TreeTableRenderer.java:762)
    oracle.adfinternal.view.faces.renderkit.rich.TableRenderer.storeHeightAutosized(TableRenderer.java:2365)
    oracle.adfinternal.view.faces.renderkit.rich.TableRenderer.encodeAll(TableRenderer.java:482)
    oracle.adfinternal.view.faces.renderkit.rich.TreeTableRenderer.encodeAll(TreeTableRenderer.java:585)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    org.apache.myfaces.trinidad.component.UIXCollection.encodeEnd(UIXCollection.java:527)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer._encodeChild(PanelGroupLayoutRenderer.java:398)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer.access$300(PanelGroupLayoutRenderer.java:30)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer$EncoderCallback.processComponent(PanelGroupLayoutRenderer.java:582)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer$EncoderCallback.processComponent(PanelGroupLayoutRenderer.java:573)
    org.apache.myfaces.trinidad.component.UIXComponent.processFlattenedChildren(UIXComponent.java:154)
    org.apache.myfaces.trinidad.component.UIXComponent.processFlattenedChildren(UIXComponent.java:285)
    org.apache.myfaces.trinidad.component.UIXComponent.encodeFlattenedChildren(UIXComponent.java:252)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer.encodeAll(PanelGroupLayoutRenderer.java:236)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeAllChildren(CoreRenderer.java:407)
    oracle.adfinternal.view.faces.renderkit.rich.PanelHeaderRenderer.renderChildrenAfterHelpAndInfo(PanelHeaderRenderer.java:471)
    oracle.adfinternal.view.faces.renderkit.rich.PanelHeaderRenderer._renderContentCell(PanelHeaderRenderer.java:939)
    oracle.adfinternal.view.faces.renderkit.rich.PanelHeaderRenderer.renderContentRow(PanelHeaderRenderer.java:425)
    oracle.adfinternal.view.faces.renderkit.rich.PanelHeaderRenderer.encodeAll(PanelHeaderRenderer.java:160)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer._encodeChild(PanelGroupLayoutRenderer.java:398)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer.access$300(PanelGroupLayoutRenderer.java:30)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer$EncoderCallback.processComponent(PanelGroupLayoutRenderer.java:582)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer$EncoderCallback.processComponent(PanelGroupLayoutRenderer.java:573)
    org.apache.myfaces.trinidad.component.UIXComponent.processFlattenedChildren(UIXComponent.java:154)
    org.apache.myfaces.trinidad.component.UIXComponent.processFlattenedChildren(UIXComponent.java:285)
    org.apache.myfaces.trinidad.component.UIXComponent.encodeFlattenedChildren(UIXComponent.java:252)
    oracle.adfinternal.view.faces.renderkit.rich.PanelGroupLayoutRenderer.encodeAll(PanelGroupLayoutRenderer.java:236)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adfinternal.view.faces.renderkit.rich.PanelStretchLayoutRenderer.encodeCenterFacet(PanelStretchLayoutRenderer.java:769)
    oracle.adfinternal.view.faces.renderkit.rich.PanelStretchLayoutRenderer._encodeCenterPane(PanelStretchLayoutRenderer.java:1055)
    oracle.adfinternal.view.faces.renderkit.rich.PanelStretchLayoutRenderer._encodeMiddlePanes(PanelStretchLayoutRenderer.java:345)
    oracle.adfinternal.view.faces.renderkit.rich.PanelStretchLayoutRenderer.encodeAll(PanelStretchLayoutRenderer.java:122)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adf.view.rich.render.RichRenderer.encodeStretchedChild(RichRenderer.java:1998)
    oracle.adfinternal.view.faces.renderkit.rich.RegionRenderer.access$400(RegionRenderer.java:46)
    oracle.adfinternal.view.faces.renderkit.rich.RegionRenderer$ChildEncoderCallback.processComponent(RegionRenderer.java:562)
    oracle.adfinternal.view.faces.renderkit.rich.RegionRenderer$ChildEncoderCallback.processComponent(RegionRenderer.java:553)
    org.apache.myfaces.trinidad.component.UIXComponent.processFlattenedChildren(UIXComponent.java:154)
    org.apache.myfaces.trinidad.component.UIXComponent.processFlattenedChildren(UIXComponent.java:285)
    org.apache.myfaces.trinidad.component.UIXComponent.encodeFlattenedChildren(UIXComponent.java:252)
    oracle.adfinternal.view.faces.renderkit.rich.RegionRenderer._encodeChildren(RegionRenderer.java:229)
    oracle.adfinternal.view.faces.renderkit.rich.RegionRenderer.encodeAll(RegionRenderer.java:144)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    oracle.adf.view.rich.component.fragment.UIXRegion.encodeEnd(UIXRegion.java:292)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    oracle.adfinternal.view.faces.taglib.region.IncludeTag$FacetWrapper.encodeAll(IncludeTag.java:511)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adfinternal.view.faces.renderkit.rich.DecorativeBoxRenderer.encodeFacet(DecorativeBoxRenderer.java:361)
    oracle.adfinternal.view.faces.renderkit.rich.DecorativeBoxRenderer._encodeCenterPane(DecorativeBoxRenderer.java:529)
    oracle.adfinternal.view.faces.renderkit.rich.DecorativeBoxRenderer.encodeAll(DecorativeBoxRenderer.java:167)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adf.view.rich.render.RichRenderer.encodeStretchedChild(RichRenderer.java:1998)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer._renderPane(PanelSplitterRenderer.java:1267)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer.encodeAll(PanelSplitterRenderer.java:88)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adf.view.rich.render.RichRenderer.encodeStretchedChild(RichRenderer.java:1998)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer._renderPane(PanelSplitterRenderer.java:1267)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer.encodeAll(PanelSplitterRenderer.java:88)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer.encodeAll(PanelSplitterRenderer.java:88)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adfinternal.view.faces.renderkit.rich.DecorativeBoxRenderer.encodeFacet(DecorativeBoxRenderer.java:361)
    oracle.adfinternal.view.faces.renderkit.rich.DecorativeBoxRenderer._encodeCenterPane(DecorativeBoxRenderer.java:529)
    oracle.adfinternal.view.faces.renderkit.rich.DecorativeBoxRenderer.encodeAll(DecorativeBoxRenderer.java:167)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adf.view.rich.render.RichRenderer.encodeStretchedChild(RichRenderer.java:1998)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer._renderPane(PanelSplitterRenderer.java:1267)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer.encodeAll(PanelSplitterRenderer.java:88)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    oracle.adf.view.rich.render.RichRenderer.encodeStretchedChild(RichRenderer.java:1998)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer._renderPane(PanelSplitterRenderer.java:1267)
    oracle.adfinternal.view.faces.renderkit.rich.PanelSplitterRenderer.encodeAll(PanelSplitterRenderer.java:88)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeAllChildren(CoreRenderer.java:407)
    oracle.adfinternal.view.faces.renderkit.rich.PageTemplateRenderer.encodeAll(PageTemplateRenderer.java:69)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    oracle.adf.view.rich.component.fragment.ContextSwitchingComponent.encodeEnd(ContextSwitchingComponent.java:155)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeAllChildren(CoreRenderer.java:407)
    oracle.adfinternal.view.faces.renderkit.rich.FormRenderer.encodeAll(FormRenderer.java:158)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeChild(CoreRenderer.java:398)
    oracle.adf.view.rich.render.RichRenderer.encodeChild(RichRenderer.java:2608)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeAllChildren(CoreRenderer.java:407)
    oracle.adfinternal.view.faces.renderkit.rich.DocumentRenderer.encodeAll(DocumentRenderer.java:725)
    oracle.adf.view.rich.render.RichRenderer.encodeAll(RichRenderer.java:1252)
    org.apache.myfaces.trinidad.render.CoreRenderer.encodeEnd(CoreRenderer.java:326)
    org.apache.myfaces.trinidad.component.UIXComponentBase.encodeEnd(UIXComponentBase.java:758)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    javax.faces.component.UIComponent.encodeAll(UIComponent.java:920)
    com.sun.faces.application.ViewHandlerImpl.doRenderView(ViewHandlerImpl.java:256)
    com.sun.faces.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:131)
    javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:189)
    org.apache.myfaces.trinidadinternal.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:155)
    oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._renderResponse(LifecycleImpl.java:773)
    oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._executePhase(LifecycleImpl.java:252)
    oracle.adfinternal.view.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:207)
    javax.faces.webapp.FacesServlet.service(FacesServlet.java:246)
    oracle.adfinternal.view.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:207)
    javax.faces.webapp.FacesServlet.service(FacesServlet.java:246)
    weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:224)
    weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:108)
    weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:206)
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    oracle.adf.model.servlet.ADFBindingFilter.doFilter(ADFBindingFilter.java:147)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    oracle.adfinternal.view.faces.webapp.rich.RegistrationFilter.doFilter(RegistrationFilter.java:69)
    org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:437)
    oracle.adfinternal.view.faces.activedata.AdsFilter.doFilter(AdsFilter.java:44)
    org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:437)
    org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl._doFilterImpl(TrinidadFilterImpl.java:231)
    org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl.doFilter(TrinidadFilterImpl.java:133)
    org.apache.myfaces.trinidad.webapp.TrinidadFilter.doFilter(TrinidadFilter.java:91)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    oracle.adf.library.webapp.LibraryFilter.doFilter(LibraryFilter.java:163)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:111)
    oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:299)
    oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:405)
    oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:94)
    oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:98)
    oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:70)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:86)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:25)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:55)
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3687)
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:308)
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:116)
    weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2213)
    weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2135)
    weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1420)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:168)
    Please help.
    Thomas

    Hi Thomas,
    Your Thread is showing a processor computing operation ArrayList.clone() triggered from the Oracle JBO framework / API.
    Few questions for you:
    -     Are you using Solaris OS? If yes can you please capture the PRSTAT at the same time as your JVM Thread Dump using command:
    prstat -L 0 1 | grep <java pid>;kill -3 <java pid>
    -     How many threads are showing this condition?
    -     Was CPU utilization of the affected Java process very high on the server at that time?
    -     What is the utilization of your Java Heap at the time of the Thread Dump capture?
    This Thread could be stuck in an infinite loop since it is stuck for at least 600 seconds (10 minutes). It would probably help to generate a few more snapshots earlier and understand if Thread is hanging in the same code portion (possibility Oracle JBO) for every snapshot or not.
    This could also be data related (too many data rows) returned and causing high processing time for the clone operation from the Myfaces / JSF table that you are trying to render.
    Regards,
    P-H
    http://javaeesupportpatterns.blogspot.com/

  • PAPI client in J2EE (Oracle app server, OC4J)

    Hello,
    We have a problem with using RMI-based PAPI in our J2EE application. Our client code runs fine as a standalone program, but doesn't work when it's used as a part of
    a J2EE application deployed to OC4J. There seems to some incompatibility between the XML parser implementation that Oracle supplies (xmlparserv2.jar) and Oracle's
    BPM Papi client.
    Here are the errors that we're getting:
    [Error] :3:69: <Line 3, Column 69>: XML-20129: (Error) Namespace prefix 'xsi' us
    ed but not declared.
    [Error] :8:33: <Line 8, Column 33>: XML-20129: (Error) Namespace prefix 'xsi' us
    ed but not declared.
    [Error] :10:34: <Line 10, Column 34>: XML-20129: (Error) Namespace prefix 'xsi'
    used but not declared.
    [Error] :12:40: <Line 12, Column 40>: XML-20129: (Error) Namespace prefix 'xsi'
    used but not declared.
    [Error] :42:43: <Line 42, Column 43>: XML-20129: (Error) Namespace prefix 'xsi'
    used but not declared.
    [Error] :44:33: <Line 44, Column 33>: XML-20129: (Error) Namespace prefix 'xsi'
    used but not declared.
    [Error] :46:38: <Line 46, Column 38>: XML-20129: (Error) Namespace prefix 'xsi'
    used but not declared.
    [Error] :48:35: <Line 48, Column 35>: XML-20129: (Error) Namespace prefix 'xsi'
    used but not declared.
    Exception in thread "main" java.lang.IllegalArgumentException
    at oracle.xml.jaxp.JXTransformer.setOutputProperty(JXTransformer.java:762)
    at fuego.prefs.engine.storage.EnginePreferencesNode.export(EnginePreferencesNode.java:179)
    at fuego.prefs.engine.EnginePreferences.doExport(EnginePreferences.java:143)
    at fuego.prefs.engine.EnginePreferences.doExport(EnginePreferences.java:134)
    at fuego.directory.project.engine.ProjectEngineAccessor.getEnginePropertiesString(ProjectEngineAccessor.java:137)
    at fuego.directory.project.engine.ProjectEngineAccessor.fetchEngineConfiguration(ProjectEngineAccessor.java:83)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at fuego.directory.provider.DirectorySessionImpl$AccessorProxy.invoke(DirectorySessionImpl.java:756)
    at $Proxy5.fetchEngineConfiguration(Unknown Source)
    at fuego.directory.DirEngineConfiguration.fetch(DirEngineConfiguration.java:102)
    at fuego.prefs.engine.storage.EnginePreferencesNode.fetchEngineConfiguration(EnginePreferencesNode.java:112)
    at fuego.prefs.engine.EnginePreferences.initialize(EnginePreferences.java:438)
    at fuego.prefs.engine.EnginePreferences.loadBasePreferences(EnginePreferences.java:220)
    at fuego.papi.impl.ProcessServiceImpl.initializePollingHandledType(ProcessServiceImpl.java:1937)
    at fuego.papi.impl.ProcessServiceImpl.initializeInstanceCache(ProcessServiceImpl.java:1924)
    at fuego.papi.impl.ProcessServiceImpl.<init>(ProcessServiceImpl.java:178)
    at fuego.papi.impl.ProcessServiceFactoryImpl.getOrCreateProcessService(ProcessServiceFactoryImpl.java:127)
    at fuego.papi.impl.ProcessServiceFactoryImpl.create(ProcessServiceFactoryImpl.java:79)
    at fuego.papi.impl.ProcessServiceFactoryImpl.create(ProcessServiceFactoryImpl.java:63)
    at fuego.papi.ProcessService.create(ProcessService.java:335)
    at oasas.wpr.demo.bpm.PapiTest.createProcess(PapiTest.java:36)
    at oasas.wpr.demo.bpm.PapiTest.<init>(PapiTest.java:15)
    at oasas.wpr.demo.bpm.PapiTest.main(PapiTest.java:104)
    The errors above are generated by our standalone client when it runs with xmlparserv2.jar in the classpath. Exactly the same errors are generated by the J2EE app, but we can't remove xmlparserv.jar from OC4J's classpath because it seems to be required by the container itself.
    Please advise if there are any configuration changes that are necessary to have a PAPI client running in Oracle's J2EE environment.
    Thank you in advance,
    Igor

    Hi,
    I only was successful with running the PAPI WS client in OC4J environment, as described here: Re: Error in JAX-WS 2.0 & JDeveloper 10.1.3
    In my case, I copied all jar files from latest Sun's Metro distro and that fixed the problem!
    I also tested the RMI-based PAPI call, but that one still does not work.
    -Igor

  • Stuck Threads on weblogic 10.3.2 Holding lock HttpURLConnection

    We are having issue with STUCK THREADS on weblogic 10.3.2. Basically my application makes a lot of web service calls over HTTPS and once in a while I see few STUCK THREADS and most of the time they clear but sometimes I have to bounce my server to clean.
    I have a thread dump, according to this its holding lock at weblogic.net.http.HttpURLConnection.getInputStream
    Any help to resolve this problem is highly appreciated.
    ThreadDump:
    <May 24, 2011 9:59:08 AM EDT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "604" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@b607d75", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-75 "[STUCK] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
    jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
    jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:31)
    java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
    java.net.SocketInputStream.read(SocketInputStream.java:107)
    weblogic.utils.io.ChunkedInputStream.read(ChunkedInputStream.java:149)
    java.io.InputStream.read(InputStream.java:85)
    com.certicom.tls.record.ReadHandler.readFragment(Unknown Source)
    com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
    com.certicom.tls.record.ReadHandler.read(Unknown Source)
    ^-- Holding lock: com.certicom.tls.record.ReadHandler@bec563c[thin lock]
    com.certicom.io.InputSSLIOStreamWrapper.read(Unknown Source)
    java.io.BufferedInputStream.fill(BufferedInputStream.java:189)
    java.io.BufferedInputStream.read(BufferedInputStream.java:236)
    ^-- Holding lock: java.io.BufferedInputStream@bec5d9f[thin lock]
    weblogic.net.http.MessageHeader.isHTTP(MessageHeader.java:214)
    weblogic.net.http.MessageHeader.parseHeader(MessageHeader.java:141)
    weblogic.net.http.HttpClient.parseHTTP(HttpClient.java:452)
    ^-- Holding lock: weblogic.net.http.HttpsClient@bec5202[thin lock]
    weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:328)
    weblogic.net.http.SOAPHttpsURLConnection.getInputStream(SOAPHttpsURLConnection.java:37)
    ^-- Holding lock: weblogic.net.http.SOAPHttpsURLConnection@bd3b209[thin lock]
    weblogic.net.http.HttpURLConnection.getResponseCode(HttpURLConnection.java:939)
    com.sun.xml.ws.transport.http.client.HttpClientTransport.readResponseCodeAndMessage(HttpClientTransport.java:213)
    com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:116)
    com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:101)
    com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:580)
    com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:553)
    com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:535)
    com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:433)
    ^-- Holding lock: com.sun.xml.ws.api.pipe.Fiber@bd3afa8[thin lock]
    com.sun.xml.ws.client.Stub.process(Stub.java:238)
    com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:135)
    com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:102)
    com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
    com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:116)
    $Proxy107.jobSubmit(Unknown Source)
    com.somejob.invokeService(JobOperation.java:53)
    com.somejobws.operations.ticket.execute(ticket.java:30)
    com.somejobutil.Somehelper.process(Somehelper.java:174)
    com.somejobutil.Somehelper.processJob(Somehelper.java:147)
    com.somejobutil.Somehelper.processMDB(Somehelper.java:91)
    com.somejobejb.ProcessMDB.onMessage(ProcessMDB.java:28)
    sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:86)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:102)
    com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
    com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:151)
    $Proxy67.onMessage(Unknown Source)
    weblogic.ejb.container.internal.MDListener.execute(MDListener.java:403)
    weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:352)
    weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:264)
    weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4564)
    weblogic.jms.client.JMSSession.execute(JMSSession.java:4060)
    weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3746)
    weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
    weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5096)
    weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
    }

    Yes, we have the same problem, stuck thread on weblogic.net.http.HttpsURLConnection.getResponseCode() (actually implemented by weblogic.net.http.SOAPHttpsURLConnection which also appears to be deprecated ???)
    When we use the class javax.net.ssl.HttpsURLConnection the getResponseCode() method works perfectly well, so it looks like the weblogic implementation is to blame.
    This is becoming rather urgent now because we're looking to migrate from OC4J to Weblogic and this issue has therefore become a blocker.

  • PAPI client is running outside a j2ee container

    Hi,
    I install BPM J2EE 10.3.1 for linux and weblogic 10.3 for linux on Red Hat Linux Enterprise 5. The database' version is 10.2.0.1 for windows and the database is install in window server 2003 sp1 on another machine.
    I search this forum and copy and change the code. But I still get error "PAPI client is running outside a j2ee container"
    The java code runs in the 3rd machine.
    -------------------------java code-------------------------------------------------
    Properties configuration = new Properties();
    configuration.setProperty(ProcessService.DIRECTORY_ID, "Default");
    configuration.setProperty(ProcessService.DIRECTORY_PROPERTIES_FILE, "F:\\share\\192.168.1.103\\directory.xml");
    Properties systemProperties = System.getProperties();
    systemProperties.setProperty("java.naming.factory.initial", "weblogic.jndi.WLInitialContextFactory");
    systemProperties.setProperty("java.naming.provider.url", "t3://192.168.1.103:7001");
    ProcessService processService = ProcessService.create(configuration);
    ConnectionPassport Conpassport = processService.createPassportWithPreset("container-auth");
    Conpassport.setParticipant("jcooper");
    Conpassport.setPassword("welcome1");
    System.out.println("34");
    Conpassport.fillPassport();
    System.out.println("36");
    ProcessServiceSession session = processService.createSession(Conpassport, null);
    System.out.println("38");
    String processName = "/processA";
    Arguments arguments = Arguments.create();
    System.out.println("41");
    session.processCreateInstance(processName, "BeginIn", arguments);//
    System.out.println("43");
    -----------------------error message----------------------------------------------------
    D:\JDeveloper11g_20090318\jdk160_05\bin\javaw.exe -client -classpath E:\Jdeveloper10131_20090318\jdev\mywork\GDHZDJDCIntegrationTest\HZGTprocess\classes;D:\BPM_HOME_standalone\client\papi\lib\b1oracle.jar;D:\BPM_HOME_standalone\client\papi\lib\b1util.jar;D:\BPM_HOME_standalone\client\papi\lib\b1base.jar;D:\BPM_HOME_standalone\client\papi\lib\fuegopapi-client.jar hzgtprocess.VM_192_168_1_103_linux
    Creating connector [fuego:SQL]
    Creating ProcessService with id 'oracle/2009-09-14 13:23:12+08:00'.
    ********* PAPI (Version 10.3.1.0.0 Mantainance Pack 1 Build # 98982 ) is starting *********
    Local folder /tmp\system\Schema3387192-437105321\catalogs found.
    Loading catalogs from local folder: /tmp\system\Schema3387192-437105321\catalogs
    0 jars found locally.
    [CatalogMgrCache] =======================
    Registering CatalogMgr [oracle/2009-09-14 13:23:12+08:00] ...CatalogManagerCache 20195943:
    Managers:
    Counters:
    [CatalogMgrCache] =======================
    CatalogMgr [oracle/2009-09-14 13:23:12+08:00] REGISTERED!CatalogManagerCache 20195943:
    Managers:
    {oracle/2009-09-14 13:23:12+08:00=fuego.util.LocalCatalogManager@1cbda0a}
    Counters:
    ProcessService 'oracle/2009-09-14 13:23:12+08:00' created successfully.
    34
    36
    38
    41
    Unreachable Engine Tolerance (seconds):
    by default: 0
    to be used: 0
    This papi client will not cache exceptions which imply that an engine could not be reached.
    A user transaction provider is not available for reading Papi News received through the JMS Topic (probable reason: PAPI client is running outside a j2ee container). Papi News will be processed anyway. Details: javax.naming.NoInitialContextException: Cannot instantiate class: weblogic.jndi.WLInitialContextFactory [Root exception is java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory]
    The connection to the JMS Topic engine was completed successfully.
    Exception in thread "main" fuego.papi.ProcessNotAvailableException: Process '/processA#Default-1.0' not available.
         at fuego.papi.impl.AbstractProcessLoader.createProcess(AbstractProcessLoader.java:96)
         at fuego.papi.impl.AbstractProcessLoader.createProcess(AbstractProcessLoader.java:36)
         at fuego.papi.impl.SessionProcessLoader.load(SessionProcessLoader.java:254)
         at fuego.papi.impl.ProcessManager.get(ProcessManager.java:894)
         at fuego.papi.impl.ProcessServiceImpl.getProcess(ProcessServiceImpl.java:1380)
         at fuego.papi.impl.SessionProcessManager.getProcess(SessionProcessManager.java:143)
         at fuego.papi.impl.ProcessServiceSessionImpl.processGet(ProcessServiceSessionImpl.java:2665)
         at fuego.papi.impl.ProcessServiceSessionImpl.checkBeginPermissions(ProcessServiceSessionImpl.java:4286)
         at fuego.papi.impl.ProcessServiceSessionImpl.processCreateInstance(ProcessServiceSessionImpl.java:3177)
         at hzgtprocess.VM_192_168_1_103_linux.main(VM_192_168_1_103_linux.java:42)
    Caused by: fuego.papi.impl.EngineNotAvailableException: Cannot reach engine 'bpmengine' at URL: ''.
         at fuego.papi.impl.EngineNotAvailableException.wrapConnectionException(EngineNotAvailableException.java:80)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.createSecureEngine(J2EEEngineAccessImpl.java:181)
         at fuego.papi.impl.ProcessServiceImpl.createSecureEngine(ProcessServiceImpl.java:1111)
         at fuego.papi.impl.ProcessServiceSessionImpl$2.run(ProcessServiceSessionImpl.java:3987)
         at fuego.papi.impl.ProcessServiceImpl.executeEngineOp(ProcessServiceImpl.java:1693)
         at fuego.papi.impl.ProcessServiceSessionImpl.getSecureEngine(ProcessServiceSessionImpl.java:3995)
         at fuego.papi.impl.AbstractProcessLoader.createProcess(AbstractProcessLoader.java:48)
         ... 9 more
    Caused by: javax.naming.NoInitialContextException: Cannot instantiate class: weblogic.jndi.WLInitialContextFactory [Root exception is java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory]
         at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:657)
         at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)
         at javax.naming.InitialContext.init(InitialContext.java:223)
         at javax.naming.InitialContext.<init>(InitialContext.java:197)
         at fuego.util.InitialContextProvider.create(InitialContextProvider.java:44)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.createInitialContext(J2EEEngineAccessImpl.java:492)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.getInitialContext(J2EEEngineAccessImpl.java:293)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.getEngineHome(J2EEEngineAccessImpl.java:277)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.getSecureEngineInterface(J2EEEngineAccessImpl.java:333)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.createSecureEngine(J2EEEngineAccessImpl.java:178)
         ... 14 more
    Caused by: java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory
         at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
         at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
         at java.lang.Class.forName0(Native Method)
         at java.lang.Class.forName(Class.java:247)
         at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:46)
         at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:654)
         ... 23 more
    The connectivity to the JMS Topic for the server synchronization information has been temporarily interrupted.
    Details:
    The connectivity to the JMS Topic has been broken due to a JNDI Context initialization problem.
    Details:
    javax.naming.NoInitialContextException: Cannot instantiate class: weblogic.jndi.WLInitialContextFactory [Root exception is java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory]
    Caused by: Cannot instantiate class: weblogic.jndi.WLInitialContextFactory
    Caused by: weblogic.jndi.WLInitialContextFactory
    fuego.papi.impl.j2ee.JMSTopicLookUpException: The connectivity to the JMS Topic has been broken due to a JNDI Context initialization problem.
    Details:
    javax.naming.NoInitialContextException: Cannot instantiate class: weblogic.jndi.WLInitialContextFactory [Root exception is java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory]
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl$TopicSubscriberImpl.lookupTopic(J2EEEngineAccessImpl.java:665)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl$TopicSubscriberImpl.run(J2EEEngineAccessImpl.java:574)
    Caused by: javax.naming.NoInitialContextException: Cannot instantiate class: weblogic.jndi.WLInitialContextFactory [Root exception is java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory]
         at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:657)
         at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)
         at javax.naming.InitialContext.init(InitialContext.java:223)
         at javax.naming.InitialContext.<init>(InitialContext.java:197)
         at fuego.util.InitialContextProvider.create(InitialContextProvider.java:44)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.createInitialContext(J2EEEngineAccessImpl.java:492)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl.access$1000(J2EEEngineAccessImpl.java:68)
         at fuego.papi.impl.j2ee.J2EEEngineAccessImpl$TopicSubscriberImpl.lookupTopic(J2EEEngineAccessImpl.java:660)
         ... 1 more
    Caused by: java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory
         at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
         at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
         at java.lang.Class.forName0(Native Method)
         at java.lang.Class.forName(Class.java:247)
         at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:46)
         at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:654)
         ... 8 more
    Process exited.

    Could this be a CLASSPATH issue?
    Caused by: java.lang.ClassNotFoundException: weblogic.jndi.WLInitialContextFactory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)

  • Stuck threads reading blob column from db table

    WLS 10.3.5, JDK 1.6u29, Oracle 11g RAC, ojdbc6 latest driver
    We're having problems with stuck threads trying to read a blob column from a DB table. The query to extract the blob is a simple select, without any locking such as "for update" clauses or whatever. The blob's size is <= 100k.
    The thread dump shows the following stack trace:
    +"[STUCK] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" RUNNABLE native+
    +     java.net.SocketInputStream.socketRead0(Native Method)+
    +     java.net.SocketInputStream.read(SocketInputStream.java:129)+
    +     oracle.net.ns.Packet.receive(Packet.java:300)+
    +     oracle.net.ns.DataPacket.receive(DataPacket.java:106)+
    +     oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)+
    +     oracle.net.ns.NetInputStream.read(NetInputStream.java:260)+
    +     oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:105)+
    +     oracle.jdbc.driver.T4CMAREngine.getNBytes(T4CMAREngine.java:1517)+
    +     oracle.jdbc.driver.T4C8TTILobd.unmarshalLobData(T4C8TTILobd.java:476)+
    +     oracle.jdbc.driver.T4C8TTILob.readLOBD(T4C8TTILob.java:770)+
    +     oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:361)+
    +     oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)+
    +     oracle.jdbc.driver.T4C8TTILob.read(T4C8TTILob.java:146)+
    +     oracle.jdbc.driver.T4CConnection.getBytes(T4CConnection.java:2392)+
    +     oracle.sql.BLOB.getBytes(BLOB.java:348)+
    +     oracle.sql.BLOB.getBytes(BLOB.java:222)+
    +     weblogic.jdbc.wrapper.Blob_oracle_sql_BLOB.getBytes(Unknown Source)+
    +     com.ibatis.sqlmap.engine.type.BlobTypeHandlerCallback.getResult(BlobTypeHandlerCallback.java:33)+
    +     com.ibatis.sqlmap.engine.type.CustomTypeHandler.getResult(CustomTypeHandler.java:52)+
    +     com.ibatis.sqlmap.engine.mapping.result.ResultMap.getPrimitiveResultMappingValue(ResultMap.java:619)+
    +     com.ibatis.sqlmap.engine.mapping.result.ResultMap.getResults(ResultMap.java:345)+
    +     com.ibatis.sqlmap.engine.execution.SqlExecutor.handleResults(SqlExecutor.java:384)+
    +     com.ibatis.sqlmap.engine.execution.SqlExecutor.handleMultipleResults(SqlExecutor.java:300)+
    +     com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQuery(SqlExecutor.java:189)+
    +     com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.sqlExecuteQuery(MappedStatement.java:221)+
    +     com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:189)+
    +     com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryForList(MappedStatement.java:139)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:567)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:541)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:118)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryForList(SqlMapClientImpl.java:94)+
    +     com.ibatis.dao.client.template.SqlMapDaoTemplate.queryForList(SqlMapDaoTemplate.java:282)"[STUCK] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" RUNNABLE native+
    +     java.net.SocketInputStream.socketRead0(Native Method)+
    +     java.net.SocketInputStream.read(SocketInputStream.java:129)+
    +     oracle.net.ns.Packet.receive(Packet.java:300)+
    +     oracle.net.ns.DataPacket.receive(DataPacket.java:106)+
    +     oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)+
    +     oracle.net.ns.NetInputStream.read(NetInputStream.java:260)+
    +     oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:105)+
    +     oracle.jdbc.driver.T4CMAREngine.getNBytes(T4CMAREngine.java:1517)+
    +     oracle.jdbc.driver.T4C8TTILobd.unmarshalLobData(T4C8TTILobd.java:476)+
    +     oracle.jdbc.driver.T4C8TTILob.readLOBD(T4C8TTILob.java:770)+
    +     oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:361)+
    +     oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)+
    +     oracle.jdbc.driver.T4C8TTILob.read(T4C8TTILob.java:146)+
    +     oracle.jdbc.driver.T4CConnection.getBytes(T4CConnection.java:2392)+
    +     oracle.sql.BLOB.getBytes(BLOB.java:348)+
    +     oracle.sql.BLOB.getBytes(BLOB.java:222)+
    +     weblogic.jdbc.wrapper.Blob_oracle_sql_BLOB.getBytes(Unknown Source)+
    +     com.ibatis.sqlmap.engine.type.BlobTypeHandlerCallback.getResult(BlobTypeHandlerCallback.java:33)+
    +     com.ibatis.sqlmap.engine.type.CustomTypeHandler.getResult(CustomTypeHandler.java:52)+
    +     com.ibatis.sqlmap.engine.mapping.result.ResultMap.getPrimitiveResultMappingValue(ResultMap.java:619)+
    +     com.ibatis.sqlmap.engine.mapping.result.ResultMap.getResults(ResultMap.java:345)+
    +     com.ibatis.sqlmap.engine.execution.SqlExecutor.handleResults(SqlExecutor.java:384)+
    +     com.ibatis.sqlmap.engine.execution.SqlExecutor.handleMultipleResults(SqlExecutor.java:300)+
    +     com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQuery(SqlExecutor.java:189)+
    +     com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.sqlExecuteQuery(MappedStatement.java:221)+
         com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:189)
    +     com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryForList(MappedStatement.java:139)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:567)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:541)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:118)+
    +     com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryForList(SqlMapClientImpl.java:94)+
    +     com.ibatis.dao.client.template.SqlMapDaoTemplate.queryForList(SqlMapDaoTemplate.java:282)+
    Some threads eventually end (after 1-2 hours), most of them remain there for days.
    Any hint would be quite useful, thanks.

    Threads are executing the actual allocated request from the Weblogic Kernel. Most of the problems happen when the Thread execution is reaching the application or business layer.
    At this point your application Java code module is sending or receiving data from external sources such as a an Oracle database for example. Any problem with such external system will cause the Thread to hang and wait for data to come back.
    Other situations can occur such as internal deadlock, infinite looping, heavy IO contention on your server etc.
    Doesn't loo like a driver issue.
    http://docs.oracle.com/cd/E21764_01/doc.1111/e14770/weblogic_server_issues.htm#autoId2
    Check at the Database end
    Cheers ...

  • Stuck thread in weblogic

    Hello All,
    We are getting stuck thread in weblogic under high load.
    following is the thread dump from weblogic 10
    "[STANDBY] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'" id=83 idx=0x134 tid=5376 prio=5 alive, in native, waiting, daemon
    -- Waiting for notification on: weblogic/work/ExecuteThread@0x06DEC880[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
    ^-- Lock released while waiting: weblogic/work/ExecuteThread@0x06DEC880[fat lock]
    at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "[STANDBY] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'" id=84 idx=0x138 tid=1688 prio=5 alive, in native, waiting, daemon
    -- Waiting for notification on: weblogic/work/ExecuteThread@0x06E50070[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
    ^-- Lock released while waiting: weblogic/work/ExecuteThread@0x06E50070[fat lock]
    at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "[STUCK] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'" id=85 idx=0x13c tid=3984 prio=1 alive, daemon
    at jrockit/vm/Allocator.allocArray(Allocator.java:392)
    at java/lang/AbstractStringBuilder.<init>(AbstractStringBuilder.java:44)
    at java/lang/StringBuffer.<init>(StringBuffer.java:80)
    at com/vignette/exolab/castor/jdo/oql/ParseTreeWalker.addWhereClause(ParseTreeWalker.java:994)
    at com/vignette/exolab/castor/jdo/oql/ParseTreeWalker.createQueryExpression(ParseTreeWalker.java:816)
    at com/vignette/exolab/castor/jdo/oql/ParseTreeWalker.<init>(ParseTreeWalker.java:139)
    at com/vignette/exolab/castor/jdo/engine/OQLQueryImpl.create(OQLQueryImpl.java:274)
    at com/vignette/exolab/castor/jdo/engine/DatabaseImpl.getOQLQuery(DatabaseImpl.java:502)
    at com/vignette/as/server/persist/dao/JDODAOPersistor.findByQuery(JDODAOPersistor.java:395)
    at com/vignette/as/server/persist/dao/JDODAOPersistor.findByIds(JDODAOPersistor.java:360)
    at com/vignette/as/server/persist/ExtensibleApplicationObjectPersistor.findByIdsHelper(ExtensibleApplicationObjectPersistor.java:159)
    at com/vignette/as/server/persist/ExtensibleApplicationObjectPersistor.findByIds(ExtensibleApplicationObjectPersistor.java:90)
    at com/vignette/as/server/logic/channel/ChannelOpsLogic.findById(ChannelOpsLogic.java:194)
    at com/vignette/as/server/logic/channel/ChannelOpsLogicGen.findById(ChannelOpsLogicGen.java:1553)
    at com/vignette/as/client/javabean/Channel.findById(Channel.java:247)
    at com/vignette/as/server/logic/channel/ChannelOpsLogic.getSubchannels(ChannelOpsLogic.java:563)
    at com/vignette/as/server/logic/channel/ChannelOpsLogicGen.getSubchannels(ChannelOpsLogicGen.java:3237)
    at com/vignette/as/client/javabean/ChannelBase.getSubchannels(ChannelBase.java:480)
    at com/vignette/as/client/javabean/ChannelBase.getAllSubchannels(ChannelBase.java:375)
    at com/astrazeneca/portal/dpm/ChannelWrapper.getSubchannels(ChannelWrapper.java:203)
    at jsp_servlet/_jsp/_grid/__xmlgeneration.subChannelRec(__xmlgeneration.java:97)
    at jsp_servlet/_jsp/_grid/__xmlgeneration.subChannelRec(__xmlgeneration.java:146)
    at jsp_servlet/_jsp/_grid/__xmlgeneration._jspService(__xmlgeneration.java:430)
    at weblogic/servlet/jsp/JspBase.service(JspBase.java:34)
    at weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:226)
    at weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:124)
    at weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)
    at weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:175)
    at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3395)
    at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic/security/service/SecurityManager.runAs(Lweblogic/security/acl/internal/AuthenticatedSubject;Lweblogic/security/acl/internal/AuthenticatedSubject;Ljava/security/PrivilegedAction;)Ljava/lang/Object;(Unknown Source)
    at weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2140)
    at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:2046)
    at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1366)
    at weblogic/work/ExecuteThread.execute(ExecuteThread.java:200)
    at weblogic/work/ExecuteThread.run(ExecuteThread.java:172)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "[STANDBY] ExecuteThread: '16' for queue: 'weblogic.kernel.Default (self-tuning)'" id=86 idx=0x140 tid=1192 prio=5 alive, in native, waiting, daemon
    -- Waiting for notification on: weblogic/work/ExecuteThread@0x06E50AF0[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
    ^-- Lock released while waiting: weblogic/work/ExecuteThread@0x06E50AF0[fat lock]
    at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "VAP Topic Consumer Thread - 0" id=87 idx=0x144 tid=6040 prio=5 alive, in native, parked, daemon
    at jrockit/vm/Locks.park0(J)V(Native Method)
    at jrockit/vm/Locks.park(Locks.java:2416)
    at sun/misc/Unsafe.park(ZJ)V(Native Method)
    at java/util/concurrent/locks/LockSupport.park(LockSupport.java:118)
    at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1767)
    at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:359)
    at com/epicentric/topic/internal/MessageQueue.remove(MessageQueue.java:67)
    at com/epicentric/topic/internal/Consumer.run(Consumer.java:72)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "VAP Topic Consumer Thread - 1" id=88 idx=0x148 tid=4752 prio=5 alive, in native, parked, daemon
    at jrockit/vm/Locks.park0(J)V(Native Method)
    at jrockit/vm/Locks.park(Locks.java:2416)
    at sun/misc/Unsafe.park(ZJ)V(Native Method)
    at java/util/concurrent/locks/LockSupport.park(LockSupport.java:118)
    at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1767)
    at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:359)
    at com/epicentric/topic/internal/MessageQueue.remove(MessageQueue.java:67)
    at com/epicentric/topic/internal/Consumer.run(Consumer.java:72)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "VAP Topic Consumer Thread - 2" id=89 idx=0x14c tid=4504 prio=5 alive, in native, parked, daemon
    at jrockit/vm/Locks.park0(J)V(Native Method)
    at jrockit/vm/Locks.park(Locks.java:2416)
    at sun/misc/Unsafe.park(ZJ)V(Native Method)
    at java/util/concurrent/locks/LockSupport.park(LockSupport.java:118)
    at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1767)
    at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:359)
    at com/epicentric/topic/internal/MessageQueue.remove(MessageQueue.java:67)
    at com/epicentric/topic/internal/Consumer.run(Consumer.java:72)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "VAP Topic Consumer Thread - 3" id=90 idx=0x150 tid=5572 prio=5 alive, in native, parked, daemon
    at jrockit/vm/Locks.park0(J)V(Native Method)
    at jrockit/vm/Locks.park(Locks.java:2416)
    at sun/misc/Unsafe.park(ZJ)V(Native Method)
    at java/util/concurrent/locks/LockSupport.park(LockSupport.java:118)
    at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1767)
    at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:359)
    at com/epicentric/topic/internal/MessageQueue.remove(MessageQueue.java:67)
    at com/epicentric/topic/internal/Consumer.run(Consumer.java:72)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "VAP Topic Consumer Thread - 4" id=91 idx=0x154 tid=4340 prio=5 alive, in native, parked, daemon
    at jrockit/vm/Locks.park0(J)V(Native Method)
    at jrockit/vm/Locks.park(Locks.java:2416)
    at sun/misc/Unsafe.park(ZJ)V(Native Method)
    at java/util/concurrent/locks/LockSupport.park(LockSupport.java:118)
    at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1767)
    at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:359)
    at com/epicentric/topic/internal/MessageQueue.remove(MessageQueue.java:67)
    at com/epicentric/topic/internal/Consumer.run(Consumer.java:72)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "CS_Local_Monitor_0" id=92 idx=0x158 tid=3864 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/vignette/config/client/cachefile/ConfigSpaceLocal$Monitor.run(ConfigSpaceLocal.java:1364)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-41" id=93 idx=0x15c tid=4884 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x0A899918[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x0A899918[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-42" id=94 idx=0x160 tid=4968 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x06BE2238[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x06BE2238[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "TaskScheduler thread" id=98 idx=0x164 tid=1580 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/epicentric/task/TaskScheduler.sleepUntilRuntime(TaskScheduler.java:226)
    at com/epicentric/task/TaskScheduler.run(TaskScheduler.java:83)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "TaskQueue Lease Manager" id=100 idx=0x168 tid=3908 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/epicentric/task/internal/LeaseManager.waitForNextExecutionTime(LeaseManager.java:69)
    at com/epicentric/task/internal/LeaseManager.run(LeaseManager.java:52)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread System Expiration and Ostracizing Thread" id=101 idx=0x16c tid=4200 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/epicentric/threadpool/standard/StandardThreadPool$ExpirationThread.run(StandardThreadPool.java:238)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Task Executor" id=102 idx=0x170 tid=3592 prio=1 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/epicentric/task/internal/TaskExecutor.waitForNextExecutionTime(TaskExecutor.java:205)
    at com/epicentric/task/internal/TaskExecutor.run(TaskExecutor.java:186)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Task Metric Gatherer" id=103 idx=0x174 tid=5988 prio=1 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/epicentric/task/internal/TaskExecutor$1.run(TaskExecutor.java:142)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "CS_Local_Monitor_0" id=142 idx=0x178 tid=5196 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/vignette/config/client/cachefile/ConfigSpaceLocal$Monitor.run(ConfigSpaceLocal.java:1364)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-54" id=130 idx=0x17c tid=648 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x0AD3C798[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x0AD3C798[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Timer-10" id=113 idx=0x180 tid=1092 prio=5 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/TaskQueue@0x00B22F40[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/util/TimerThread.mainLoop(Timer.java:509)
    ^-- Lock released while waiting: java/util/TaskQueue@0x00B22F40[fat lock]
    at java/util/TimerThread.run(Timer.java:462)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "JNDI Monitor" id=116 idx=0x184 tid=1856 prio=5 alive, in native, waiting, daemon
    -- Waiting for notification on: com/epicentric/jndi/Monitor@0x00B12D60[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at com/epicentric/jndi/Monitor.run(Monitor.java:100)
    ^-- Lock released while waiting: com/epicentric/jndi/Monitor@0x00B12D60[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-47" id=117 idx=0x188 tid=816 prio=5 alive, in native, daemon
    at jrockit/net/SocketNativeIO.readBytesPinned(Ljava/io/FileDescriptor;[BIII)I(Native Method)
                    at jrockit/net/SocketNativeIO.socketRead(SocketNativeIO.java:31)
                    at java/net/SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I(SocketInputStream.java)
                    at java/net/SocketInputStream.read(SocketInputStream.java:129)
                    at java/io/BufferedInputStream.fill(BufferedInputStream.java:218)
                    at java/io/BufferedInputStream.read1(BufferedInputStream.java:256)
                    at java/io/BufferedInputStream.read(BufferedInputStream.java:313)
                    ^-- Holding lock: java/io/BufferedInputStream@0x00B3EA48[thin lock]
    at com/sun/jndi/ldap/Connection.run(Connection.java:784)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "TaskScheduler thread" id=119 idx=0x18c tid=4520 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/epicentric/task/TaskScheduler.run(TaskScheduler.java:94)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "OracleTimeoutPollingThread" id=124 idx=0x190 tid=2628 prio=10 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at oracle/jdbc/driver/OracleTimeoutPollingThread.run(OracleTimeoutPollingThread.java:158)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-55" id=131 idx=0x194 tid=5700 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x0973F378[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x0973F378[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-56" id=132 idx=0x198 tid=5596 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at org/apache/commons/pool/impl/GenericObjectPool$Evictor.run(GenericObjectPool.java:1122)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "CacheSyncThread" id=133 idx=0x19c tid=708 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/vignette/as/server/caching/CacheSynchronization.run(CacheSynchronization.java:135)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-58" id=135 idx=0x1a0 tid=5652 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at org/apache/commons/pool/impl/GenericObjectPool$Evictor.run(GenericObjectPool.java:1122)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-61" id=143 idx=0x1a4 tid=4196 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x0BEF1160[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x0BEF1160[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-62" id=144 idx=0x1a8 tid=5456 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x0D650770[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x0D650770[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-63" id=145 idx=0x1ac tid=1056 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at org/apache/commons/pool/impl/GenericObjectPool$Evictor.run(GenericObjectPool.java:1122)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "CacheSyncThread" id=146 idx=0x1b0 tid=3496 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/vignette/as/server/caching/CacheSynchronization.run(CacheSynchronization.java:135)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-64" id=147 idx=0x1b4 tid=4132 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at org/apache/commons/pool/impl/GenericObjectPool$Evictor.run(GenericObjectPool.java:1122)
    at java/lang/Thread.run(Thread.java:595)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "CS_Local_Monitor_0" id=221 idx=0x1b8 tid=6104 prio=5 alive, in native, sleeping, native_waiting, daemon
    at java/lang/Thread.sleep(J)V(Native Method)
    at com/vignette/config/client/cachefile/ConfigSpaceLocal$Monitor.run(ConfigSpaceLocal.java:1364)
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-84" id=222 idx=0x1bc tid=3500 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x1673EAD8[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x1673EAD8[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "Thread-85" id=223 idx=0x1c0 tid=5492 prio=1 alive, in native, waiting, daemon
    -- Waiting for notification on: java/util/ArrayList@0x1932F128[fat lock]
    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
    at java/lang/Object.wait(J)V(Native Method)
    at java/lang/Object.wait(Object.java:474)
    at com/vignette/logging/LocaleAppender$Dispatcher.run(LocaleAppender.java:393)
    ^-- Lock released while waiting: java/util/ArrayList@0x1932F128[fat lock]
    at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
    -- end of trace
    "[STANDBY] ExecuteThread: '17' for queue

    Hi,
    We are facing a similar stuck thread problem with the following stack trace...
    Any help on this is much appreciated.
    Thanks,
    Sen
    Thread-86 "[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, waiting, priority=1, DAEMON> {
    -- Waiting for notification on: netscape.ldap.LDAPSearchListener@2accce6[fat lock]
    java.lang.Object.wait(Object.java:???)
    java.lang.Object.wait(Object.java:485)
    netscape.ldap.LDAPMessageQueue.waitForMessage(LDAPMessageQueue.java:198)
    netscape.ldap.LDAPMessageQueue.waitFirstMessage(LDAPMessageQueue.java:100)
    netscape.ldap.LDAPConnection.sendRequest(LDAPConnection.java:1782)
    ^-- Holding lock: netscape.ldap.LDAPConnection@138555c[thin lock]
    netscape.ldap.LDAPConnection.search(LDAPConnection.java:2500)
    weblogic.security.providers.authentication.LDAPAtnDelegate.getDNForUser(LDAPAtnDelegate.java:3317)
    weblogic.security.providers.authentication.LDAPAtnDelegate.listMemberGroups(LDAPAtnDelegate.java:2091)
    weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.addAllGroups(LDAPAtnLoginModuleImpl.java:419)
    weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.addGroups(LDAPAtnLoginModuleImpl.java:411)
    weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:136)
    com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
    com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:101)
    sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:???)
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:27)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    javax.security.auth.login.LoginContext.invoke(LoginContext.java:720)
    javax.security.auth.login.LoginContext.access$000(LoginContext.java:186)
    javax.security.auth.login.LoginContext$4.run(LoginContext.java:683)
    javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    javax.security.auth.login.LoginContext.login(LoginContext.java:566)
    com.bea.common.security.internal.service.JAASLoginServiceImpl.login(JAASLoginServiceImpl.java:77)
    sun.reflect.GeneratedMethodAccessor2726.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
    $Proxy17.login(Unknown Source)
    weblogic.security.service.internal.WLSJAASLoginServiceImpl$ServiceImpl.login(Unknown Source)
    com.bea.common.security.internal.service.JAASAuthenticationServiceImpl.authenticate(JAASAuthenticationServiceImpl.java:74)
    sun.reflect.GeneratedMethodAccessor2727.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
    $Proxy37.authenticate(Unknown Source)
    weblogic.security.service.WLSJAASAuthenticationServiceWrapper.authenticate(Unknown Source)
    weblogic.security.service.PrincipalAuthenticator.authenticate(Unknown Source)
    weblogic.servlet.security.internal.SecurityModule.checkAuthenticate(SecurityModule.java:213)
    weblogic.servlet.security.ServletAuthentication.login(ServletAuthentication.java:391)
    weblogic.servlet.security.ServletAuthentication.weak(ServletAuthentication.java:371)
    com.bea.p13n.security.Authentication.login(Authentication.java:118)
    com.ppluk.moneyout.portal.auth.LoginServlet.doGet(LoginServlet.java:40)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:700)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:815)
    weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:224)
    weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:108)
    weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:198)
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
    com.bea.portal.tools.servlet.http.HttpContextFilter.doFilter(HttpContextFilter.java:50)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
    com.bea.p13n.servlets.PortalServletFilter.doFilter(PortalServletFilter.java:241)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
    com.bea.jsptools.servlet.PagedResultServiceFilter.doFilter(PagedResultServiceFilter.java:61)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
    weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:25)
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3468)
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:308)
    weblogic.security.service.SecurityManager.runAs(Unknown Source)
    weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2116)
    weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2038)
    weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1372)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
    }

  • WL Server Stuck Thread on Network Read

    I am experiencing an issue with the Weblogic deployment at my organisation.
    Firstly an introduction; My Application works by receiving a Web Service (SOAP) Request from a client, the Web Service request is then processed and a internal message packet is generated and sent to a C++ Application ( which is listening on a particular port). The C++ application then processes the information, and returns the response to the web service Layer, which then process the Web Service request back to the calling client.
    Currently we are having a situation where some packets are sent to the C++ application as expected. But then we are having situations where some packets NEVER reach the C++ side and the calling client time outs the entire request ( configured timeout is set for 180 seconds) and on the WebLogic Layer we have noticed a significant amount of Stuck ( and Hogging ) Threads. It should be noted that even after a stuck thread is recorded some requests are still being processed normally.
    A deeper investigation into the Weblogic Thread dump shows the Weblogic thread is stuck at:
    _at sun/nio/ch/IOUtil.read(Ljava/io/FileDescriptor;Ljava/nio/ByteBuffer;JLsun/nio/ch/NativeDispatcher;Ljava/lang/Object;)I(IOUtil.java)[inlined]_+
    _at sun/nio/ch/SocketChannelImpl.read(SocketChannelImpl.java:236)[optimized]_+
    +_^-- Holding lock: java/lang/Object@0x1a27db60[thin lock]_+
    _at com/reso/number/service/comm/SoaSocket.receive(SoaSocket.java:128)[optimized]_+
    _at com/reso/number/service/event/OTEvent.submitEvent(TEvent.java:219)[optimized]_+
    _at com/reso/number/service/event/SIMQueryEvent.process(QQEvent.java:368)[optimized]_+
    It seems that a write is sent to the C++ Application and the thread is 'WAITING' for the response.
    Now, the problem is that due to the Infrastructure in place and architectural setup ( We are using Solaris Zone for Node deployment and we have the WebLogic AS on the same node/zone as the C++ application) we are unable to do a network trace to determine whether WebLogic actually sends the request.
    Now my questions: i) Are there any known ways to trace the networks packets sent by Java ( maybe via the JVM) ii) Could Garbage collection cause the symptoms being observed? iii)Any recommended ways to resolve this kind of problem?
    Thanks for all the help.

    Increasing the Stuck Thread Max Time will not help. " Stuck Thread Max Time" tells us, after how much amount of time the thread should mark as stuck. Because by default it is 600 sec, any thread which takes more than 600sec for execution will mark as stuck. If you mention 700 sec, after 700sec the thread is marked as stuck. Even though the thread is marked stuck it still process.
    Here we need to find why the thread is stuck at
    ]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-15 "[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
    sun.nio.ch.SocketDispatcher.read1(SocketDispatcher.java:???)
    sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:46)
    sun.nio.ch.IOUtil.read(IOUtil.java:228)
    sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:159)
    ^-- Holding lock: java.lang.Object@16a47a8b[thin lock]
    weblogic.socket.NIOSocketMuxer$NIOInputStream.read(NIOSocketMuxer.java:505)
    weblogic.servlet.internal.PostInputStream.read(PostInputStream.java:156)
    weblogic.servlet.internal.ServletInputStreamImpl.read(ServletInputStreamImpl.java:228)
    weblogic.servlet.internal.ServletRequestImpl$RequestParameters.mergePostParams(ServletRequestImpl.java:2062)
    You may have to increase the buffer size at sun.nio.

  • Stuck Threads Leading to server hang

    Hi ,
    We have been facing a problem with Stuck Threads in our production environment. We are currently running version 8.1. Though the stuck Threads do point to JDBC issues, we suspect that these problems may be related to some Weblogic/OS configuration issue as well. These probelms eventually lead to a server hang.
    The general pattern of the problem is that we see messages like :
    Dec 24, 2006 8:56:09 AM EST> <Debug> <Common> <BEA-000610> < RP(rtqdbConnectionPool):timeoutInactiveRes (20)>
    ####<Dec 25, 2006 8:54:51 AM EST> <Error> <WebLogicServer> <hygeia> <hygeia-rtq-vm7> <weblogic.health.CoreHealthMonitor> <<WLS Kernel>> <> <BEA-000337> <UNKNOWN_SESSION has been busy for "896" seconds working on the request "Http Request: /quote.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.> ==================== This email/fax message is for the sole use of the intended
    Sometimes a deadlock Exception would be thrown from the Database,
    nested exception is java.sql.SQLException: ORA-00060: deadlock detected while waiting for resource
    But this is not as often as we get the Stuck Thread messages.
    This is a typical Thread dump from one of our servers when it complained of a stuck thread.
    ///*************Thread Dump Start**********************///
    Full thread dump Java HotSpot(TM) Server VM (1.4.2_08-b03 mixed mode):
    "idle thread" daemon prio=5 tid=0x00d2c150 nid=0x7f in Object.wait() [94e7f000..94e7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb2010000> (a org.xbill.Task.WorkerThread)
         at org.xbill.Task.WorkerThread.run(WorkerThread.java:130)
         - locked <0xb2010000> (a org.xbill.Task.WorkerThread)
    "Thread-28" daemon prio=5 tid=0x00c6a7c0 nid=0x7d in Object.wait() [92c7f000..92c7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xbce667f8> (a com.geico.util.LockManager)
         at com.geico.util.LockManager.waitForWork(LockManager.java:85)
         - locked <0xbce667f8> (a com.geico.util.LockManager)
         at com.geico.mail.Mail.run(Mail.java:259)
         at java.lang.Thread.run(Thread.java:534)
    "CacheManager-idle" daemon prio=5 tid=0x00c7d950 nid=0x7c in Object.wait() [92d7f000..92d7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xbce66878> (a com.geico.util.LockManager)
         at com.geico.util.LockManager.waitForWork(LockManager.java:85)
         - locked <0xbce66878> (a com.geico.util.LockManager)
         at com.geico.cache.CacheManager.run(CacheManager.java:353)
         at java.lang.Thread.run(Thread.java:534)
    "ExecuteThread: '2' for queue: 'weblogic.kernel.Non-Blocking'" daemon prio=5 tid=0x006a6c90 nid=0x7a in Object.wait() [92e7f000..92e7fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xbc2a5db8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '1' for queue: 'weblogic.kernel.Non-Blocking'" daemon prio=5 tid=0x0120c568 nid=0x79 in Object.wait() [92f7f000..92f7fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xbc2a5e38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '0' for queue: 'weblogic.kernel.Non-Blocking'" daemon prio=5 tid=0x006a6af0 nid=0x78 in Object.wait() [9307f000..9307fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xbc2a5eb8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '0' for queue: 'JMSStore<AptSoftFileStore-morpheus-rtq-vm7>.ioThreadPool'" daemon prio=5 tid=0x0052a198 nid=0x77 in Object.wait() [9317f000..9317fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xbc2a5f38> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "Thread-26" daemon prio=5 tid=0x00635d58 nid=0x75 in Object.wait() [9327f000..9327fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xbb97ad20> (a com.geico.util.LockManager)
         at java.lang.Object.wait(Object.java:429)
         at com.geico.util.LockManager.waitForWork(LockManager.java:63)
         - locked <0xbb97ad20> (a com.geico.util.LockManager)
         at com.geico.util.providers.StorageProvider.run(StorageProvider.java:158)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-25" daemon prio=5 tid=0x00fac430 nid=0x72 waiting on condition [9337f000..9337fc28]
         at java.lang.Thread.sleep(Native Method)
         at org.apache.commons.pool.impl.GenericObjectPool$Evictor.run(GenericObjectPool.java:1080)
         at java.lang.Thread.run(Thread.java:534)
    "ListenThread.Default" prio=5 tid=0x014ec6e8 nid=0x71 runnable [9347f000..9347fc28]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0xbb97ae60> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:26)
         at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:735)
         at weblogic.t3.srvr.ListenThread.run(ListenThread.java:301)
    "Thread-24" daemon prio=5 tid=0x011e9668 nid=0x70 waiting on condition [9357f000..9357fc28]
         at java.lang.Thread.sleep(Native Method)
         at com.geico.log.HeartbeatLogger$1.run(HeartbeatLogger.java:133)
    "Thread-22" daemon prio=5 tid=0x011e8610 nid=0x6f waiting on condition [9367f000..9367fc28]
         at java.lang.Thread.sleep(Native Method)
         at com.geico.log.StreamLogger.pause(StreamLogger.java:263)
         at com.geico.log.StreamLogger.access$300(StreamLogger.java:15)
         at com.geico.log.StreamLogger$WorkerThread.run(StreamLogger.java:217)
         at java.lang.Thread.run(Thread.java:534)
    "filewatcher" daemon prio=5 tid=0x0111ca50 nid=0x6e waiting on condition [9377f000..9377fc28]
         at java.lang.Thread.sleep(Native Method)
         at com.geico.watcher.FileWatcher.sleep(FileWatcher.java:135)
         at com.geico.watcher.FileWatcher.run(FileWatcher.java:118)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-18" prio=5 tid=0x00de2388 nid=0x6d in Object.wait() [9437e000..9437fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xa20f1a60> (a weblogic.jms.backend.BEConsumerReceiveRequest)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.jms.dispatcher.Request.sleepTillNotified(Request.java:240)
         at weblogic.jms.dispatcher.Request.wrappedFiniteStateMachine(Request.java:685)
         - locked <0xa20f1a60> (a weblogic.jms.backend.BEConsumerReceiveRequest)
         at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsyncInternal(DispatcherImpl.java:132)
         at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsync(DispatcherImpl.java:123)
         at weblogic.jms.dispatcher.Request.dispatchAsync(Request.java:878)
         at weblogic.jms.frontend.FEConsumer.receive(FEConsumer.java:452)
         at weblogic.jms.frontend.FEConsumer.invoke(FEConsumer.java:643)
         at weblogic.jms.dispatcher.Request.wrappedFiniteStateMachine(Request.java:643)
         at weblogic.jms.dispatcher.DispatcherImpl.dispatchSyncTran(DispatcherImpl.java:192)
         at weblogic.jms.client.JMSSession.receiveMessage(JMSSession.java:456)
         at weblogic.jms.client.JMSConsumer.receive(JMSConsumer.java:421)
         at org.springframework.jms.core.JmsTemplate.doReceive(JmsTemplate.java:800)
         at org.springframework.jms.core.JmsTemplate.doReceive(JmsTemplate.java:783)
         at org.springframework.jms.core.JmsTemplate$8.doInJms(JmsTemplate.java:741)
         at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:582)
         at org.springframework.jms.core.JmsTemplate.receive(JmsTemplate.java:739)
         at org.springframework.jms.core.JmsTemplate.receive(JmsTemplate.java:731)
         at com.geico.uierrorlog.UiErrorUtil.getMessage(UiErrorUtil.java:89)
         at com.geico.uierrorlog.UiErrorDaemon.getMessage(UiErrorDaemon.java:108)
         at com.geico.uierrorlog.UiErrorDaemon.run(UiErrorDaemon.java:166)
         at java.lang.Thread.run(Thread.java:534)
    "StatisticsThread" daemon prio=5 tid=0x00d0c798 nid=0x6c in Object.wait() [9447f000..9447fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee7b18> (a com.geico.tangosol.stats.StatsDaemon)
         at com.geico.tangosol.stats.StatsDaemon.run(StatsDaemon.java:144)
         - locked <0xb8ee7b18> (a com.geico.tangosol.stats.StatsDaemon)
         at com.tangosol.util.Daemon$1.run(Daemon.java:63)
    "StatisticsThread" daemon prio=5 tid=0x01202c18 nid=0x6b in Object.wait() [9457f000..9457fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee7bd8> (a com.geico.tangosol.stats.StatsDaemon)
         at com.geico.tangosol.stats.StatsDaemon.run(StatsDaemon.java:144)
         - locked <0xb8ee7bd8> (a com.geico.tangosol.stats.StatsDaemon)
         at com.tangosol.util.Daemon$1.run(Daemon.java:63)
    "StatisticsThread" daemon prio=5 tid=0x00b33f18 nid=0x6a in Object.wait() [9467f000..9467fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee7c90> (a com.geico.tangosol.stats.StatsDaemon)
         at com.geico.tangosol.stats.StatsDaemon.run(StatsDaemon.java:144)
         - locked <0xb8ee7c90> (a com.geico.tangosol.stats.StatsDaemon)
         at com.tangosol.util.Daemon$1.run(Daemon.java:63)
    "DistributedCache:DistributedRTQOverflow" daemon prio=5 tid=0x009895e8 nid=0x69 in Object.wait() [9477f000..9477fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee7d40> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee7d40> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "DistributedCache:PoolStatsDistributedCache:EventDispatcher" daemon prio=5 tid=0x00dff060 nid=0x68 in Object.wait() [9487f000..9487fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee7f18> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee7f18> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "DistributedCache:PoolStatsDistributedCache" daemon prio=5 tid=0x006f3e98 nid=0x67 in Object.wait() [9497f000..9497fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee7fe8> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "StatisticsThread" daemon prio=5 tid=0x0090d8d0 nid=0x66 in Object.wait() [94a7f000..94a7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee81c8> (a com.geico.tangosol.stats.StatsDaemon)
         at com.geico.tangosol.stats.StatsDaemon.run(StatsDaemon.java:144)
         - locked <0xb8ee81c8> (a com.geico.tangosol.stats.StatsDaemon)
         at com.tangosol.util.Daemon$1.run(Daemon.java:63)
    "DistributedCache:dist-RTQSession:EventDispatcher" daemon prio=5 tid=0x00a05110 nid=0x65 in Object.wait() [94b7f000..94b7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee8278> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8278> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "DistributedCache:dist-RTQSession" daemon prio=5 tid=0x0036e600 nid=0x64 in Object.wait() [94c7f000..94c7fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8348> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "ReplicatedCache" daemon prio=5 tid=0x00e9f9f8 nid=0x63 in Object.wait() [94d7f000..94d7fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8520> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "Scheduler_Worker-9" prio=5 tid=0x00339820 nid=0x61 in Object.wait() [94f7f000..94f7fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-8" prio=5 tid=0x0033af58 nid=0x60 in Object.wait() [9507f000..9507fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-7" prio=5 tid=0x0033a878 nid=0x5f in Object.wait() [9517f000..9517fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-6" prio=5 tid=0x00935488 nid=0x5e in Object.wait() [9527f000..9527fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-5" prio=5 tid=0x00bb2190 nid=0x5d in Object.wait() [9537f000..9537fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-4" prio=5 tid=0x010b9038 nid=0x5c in Object.wait() [9547f000..9547fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-3" prio=5 tid=0x00a34678 nid=0x5b in Object.wait() [9557f000..9557fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-2" prio=5 tid=0x014f9860 nid=0x5a in Object.wait() [9567f000..9567fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-1" prio=5 tid=0x0044e530 nid=0x59 in Object.wait() [9577f000..9577fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "Scheduler_Worker-0" prio=5 tid=0x0110b8e8 nid=0x58 in Object.wait() [9587f000..9587fc28]
         at java.lang.Object.wait(Native Method)
         at org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:423)
         - locked <0xb8ee8788> (a java.lang.Object)
         at org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:53)
         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:514)
    "StatisticsThread" daemon prio=5 tid=0x011352b8 nid=0x57 in Object.wait() [9597f000..9597fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee8b20> (a com.geico.tangosol.stats.StatsDaemon)
         at com.geico.tangosol.stats.StatsDaemon.run(StatsDaemon.java:144)
         - locked <0xb8ee8b20> (a com.geico.tangosol.stats.StatsDaemon)
         at com.tangosol.util.Daemon$1.run(Daemon.java:63)
    "DistributedCache:dist-rtq-sessionheader:EventDispatcher" daemon prio=5 tid=0x004222a0 nid=0x56 in Object.wait() [95a7f000..95a7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee8bd8> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8bd8> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "DistributedCache:dist-rtq-sessionheader" daemon prio=5 tid=0x01278fa0 nid=0x55 in Object.wait() [95b7f000..95b7fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8ca8> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "DistributedCache:dist-hibernate-default:EventDispatcher" daemon prio=5 tid=0x015273f0 nid=0x54 in Object.wait() [95c7f000..95c7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8ee8e80> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8e80> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "DistributedCache:dist-hibernate-default" daemon prio=5 tid=0x00ea1328 nid=0x53 in Object.wait() [9607f000..9607fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8ee8f50> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "Invocation:Management" daemon prio=5 tid=0x003d8968 nid=0x52 in Object.wait() [95d7f000..95d7fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8448698> (a com.tangosol.coherence.component.util.daemon.QueueProcessor$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "Invocation:Management:EventDispatcher" daemon prio=5 tid=0x007e6040 nid=0x51 in Object.wait() [95e7f000..95e7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb84487f0> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb84487f0> (a com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "TcpRingListener" daemon prio=6 tid=0x007e6f40 nid=0x50 runnable [95f7f000..95f7fc28]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0xb84488d0> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at java.net.ServerSocket.accept(ServerSocket.java:419)
         at com.tangosol.coherence.component.net.socket.TcpSocketAccepter.accept(TcpSocketAccepter.CDB:17)
         at com.tangosol.coherence.component.util.daemon.TcpRingListener.acceptConnection(TcpRingListener.CDB:9)
         at com.tangosol.coherence.component.util.daemon.TcpRingListener.onNotify(TcpRingListener.CDB:1)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:34)
         at java.lang.Thread.run(Thread.java:534)
    "Cluster" daemon prio=5 tid=0x00c94738 nid=0x4e in Object.wait() [9617f000..9617fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8448ac0> (a com.tangosol.coherence.component.net.Cluster$ClusterService$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "PacketPublisher" daemon prio=6 tid=0x00d83418 nid=0x4d runnable [9627f000..9627fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8448c78> (a com.tangosol.coherence.component.net.Cluster$PacketPublisher$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "PacketReceiver" daemon prio=7 tid=0x00b3cd70 nid=0x4c in Object.wait() [9637f000..9637fc28]
         at java.lang.Object.wait(Native Method)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8448dc0> (a com.tangosol.coherence.component.net.Cluster$PacketReceiver$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "PacketListenerN" daemon prio=8 tid=0x003d7cd0 nid=0x4b runnable [9647f000..9647fc28]
         at java.net.PlainDatagramSocketImpl.receive(Native Method)
         at java.net.DatagramSocket.receive(DatagramSocket.java:711)
         - locked <0xb84494c8> (a java.net.DatagramPacket)
         - locked <0xb84494e8> (a java.net.MulticastSocket)
         at com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:18)
         at com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:17)
         at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onWait(PacketListener.CDB:5)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "PacketListener1" daemon prio=8 tid=0x00df5538 nid=0x4a runnable [9657f000..9657fc28]
         at java.net.PlainDatagramSocketImpl.receive(Native Method)
         at java.net.DatagramSocket.receive(DatagramSocket.java:711)
         - locked <0xb8449cb8> (a java.net.DatagramPacket)
         - locked <0xb8449cd8> (a java.net.DatagramSocket)
         at com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:18)
         at com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:17)
         at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onWait(PacketListener.CDB:5)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "Logger@13785534 3.1.2/343" daemon prio=3 tid=0x006b7358 nid=0x49 in Object.wait() [9667f000..9667fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8449e90> (a com.tangosol.coherence.component.application.console.Coherence$Logger$Queue)
         at com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:9)
         - locked <0xb8449e90> (a com.tangosol.coherence.component.application.console.Coherence$Logger$Queue)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:31)
         at java.lang.Thread.run(Thread.java:534)
    "ExecuteThread: '5' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x011384b0 nid=0x48 in Object.wait() [9677f000..9677fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xb82c91f8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '4' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x01137bc8 nid=0x47 in Object.wait() [9687f000..9687fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xb82c9278> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '3' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x0111d4e0 nid=0x46 in Object.wait() [9697f000..9697fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xb82c92f8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '2' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x010dd218 nid=0x45 in Object.wait() [96a7f000..96a7fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xb82c9378> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '1' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00bb1c98 nid=0x44 in Object.wait() [96b7f000..96b7fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xb82c93f8> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "ExecuteThread: '0' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00979ac8 nid=0x43 in Object.wait() [96c7f000..96c7fc28]
         at java.lang.Object.wait(Native Method)
         at java.lang.Object.wait(Object.java:429)
         at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
         - locked <0xb82c9478> (a weblogic.kernel.ExecuteThread)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
    "Thread-12" daemon prio=5 tid=0x00200610 nid=0x42 waiting on condition [96d7f000..96d7fc28]
         at java.lang.Thread.sleep(Native Method)
         at com.geico.log.HeartbeatLogger$1.run(HeartbeatLogger.java:133)
    "Thread-10" daemon prio=5 tid=0x00200470 nid=0x41 waiting on condition [96e7f000..96e7fc28]
         at java.lang.Thread.sleep(Native Method)
         at com.geico.log.StreamLogger.pause(StreamLogger.java:267)
         at com.geico.log.StreamLogger.access$300(StreamLogger.java:19)
         at com.geico.log.StreamLogger$WorkerThread.run(StreamLogger.java:221)
         at java.lang.Thread.run(Thread.java:534)
    "ClockDaemonThread-0" daemon prio=5 tid=0x00ab6a48 nid=0x40 in Object.wait() [96f7f000..96f7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8051b20> (a com.geico.util.concurrent.ClockDaemon)
         at com.geico.util.concurrent.ClockDaemon.nextTask(ClockDaemon.java:328)
         - locked <0xb8051b20> (a com.geico.util.concurrent.ClockDaemon)
         at com.geico.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:362)
         at java.lang.Thread.run(Thread.java:534)
    "Thread-9" prio=5 tid=0x008912e0 nid=0x3f in Object.wait() [9ad7f000..9ad7fc28]
         at java.lang.Object.wait(Native Method)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0xb8051bc0> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "ExecuteThread: '0' for queue: 'JMS.TimerTreePool'" daemon prio=5 tid=0x001d7fa8 nid=0x3e in Object.wait() [9ae7f000..9ae7fc28]
         at java.lang.Object.wait(Native Method)
         at weblogic.jms.backend.BETimerTree.execute(BETimerTree.java:146)
         - locked <0xb8051c90> (a weblogic.jms.backend.BETimerTree)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
    "Thread-8" prio=5 tid=0x00247418 nid=0x3d in Object.wait() [9af7f000..9af7fc28]
         at java.lang.Object.wait(Native Method)
         - waiting on <0xb8051d20> (a java.util.TaskQueue)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0xb8051d20> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "Thread-7" prio=5 tid=0x001d8a28 nid=0x3c in Object.wait() [9b07f000..9b07fc28]
         at java.lang.Object.wait(Native Method)
         at java.util.TimerThread.mainLoop(Timer.java:429)
         - locked <0xb8051dc0> (a java.util.TaskQueue)
         at java.util.TimerThread.run(Timer.java:382)
    "ExecuteThread: '0' for queue: 'weblogic.cluster.MulticastManager'" daemon prio=5 tid=0x00da0a60 nid=0x3b runnable [9b17f000..9b17fc28]
         at java.net.PlainDatagramSocketImpl.receive(Native Method)
         at java.net.DatagramSocket.receive(DatagramSocket.java:711)
         - locked <0xa22dc3e8> (a java.net.DatagramPacket)
         - locked <0xb7558130> (a java.net.MulticastSocket)
         at weblogic.cluster.FragmentSocket.receive(FragmentSocket.java:169)
         at weblogic.cluster.MulticastManager.execute(MulticastManager.java:395)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
    "SSLListenThread.Administrator" prio=5 tid=0x00c7db08 nid=0x3a runnable [9b27f000..9b27fc28]
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
         - locked <0xb75582a0> (a java.net.PlainSocketImpl)
         at java.net.ServerSocket.implAccept(ServerSocket.java:448)
         at javax.net.ssl.impl.SSLServerSocketImp

    Hi,
    I guess this issue is related to your connectivity and hyperion services. Can you take a restart and try to execute the reports. Also try to check all your Hyperion services are up and running before executing the FR Reports. You can also check the Reporting services by login to Weblogic server (if you have applied it in your environment). In weblogic it should be in Running mode and not in Admin mode. If it is not in Running mode then try to check the Data Source setting from Connection Pool in Weblogic Server.
    Soumya

  • Java client for OSB proxy with JMS Transport

    Hi,
    I am trying to call OSB proxy with JMS Transport. I am generating the client through ant task clientgen and following this article
    http://www.oracle.com/technetwork/articles/murphy-soa-jms-092653.html
    The osb proxy is req-response and is simply routing to BS which return a string value.
    When I run my client, it get stuck and does not return at all. Has any one trying java client in such scenario?
    What I may be missing?
    Below is snipped of client code:
    String url = "http://localhost:7021/sbresource?PROXY/MySample/MyJMSProxyService";
    CreditLoanApprovalServiceSoapBindingQSService service = new CreditLoanApprovalServiceSoapBindingQSService_Impl(url);
    MyPortType port = service.getCreditLoanApprovalServiceSoapBindingQSPort();
    LoanStruct in = new LoanStruct(); //populated the data structure
    String loanResult = port.processLoanApp(in); // Stuck here without any error
    System.out.println("LoanResult--> " + loanResult);
    Thx
    /Ashwani

    http://localhost:7021/sbresource?PROXY/MySample/MyJMSProxyService is the WSDL URL of the proxy.
    Transport is is picked by the client from wsdl.
    As far as the documentation of client generation is there, there is no change.
    But meanwhile I have started working on sending the message directly to queue. JMSProxy is getting called. May be I will first run the proxy this way and then try troubleshooting the java client.
    Regards
    Ashwani

Maybe you are looking for

  • Apple's ink preference option, randomly appears 2008 Mac Pro???

    Ink or inkwell. Is Apple's own handwriting recognition program It is included in leopard and is only available if your mac detects the presence of a wacom tablet. I don't have a wacom tablet, nor have I ever used one. Yet the ink icon has just recent

  • Itunes wont sync pictures from my phone to laptop

    I've read that some people are having this problem also but I have yet to find a solution so I'm begging desperately for help! When I connect my iPhone 5 to my laptop, it will sync everything but the photos and videos. I get error 54. Please help. I

  • Using Firefox 7 loads Facebook very very slowly whereas Firefox 6.0.2 has no problems nor does IE

    When I or my family attempt to load Facebook and the apps it either takes forever or not at all to load. Other websites are slow, but not as bad. This happens only when we upgrade to Firefox 7, but when using Firefox 6.0.2 we do not have any problems

  • SP02 showing update was terminated and not showing Spool no and other data

    Hi Friends, Can anyone guide me in the below problem After going to change mode of transaction when i go to message and i have done repeat output and further data and communication methods. After that i have saved document. After that when i enter in

  • Error message using formcalc

    I created my form, used the formcalc, previewed it, and got this message: Script failed (language is formcalc; context is xfa[0].form[0].#subform[0].tuitioncost[0]) script=NumericField8 Error: accessor 'NumericField8' is unknown However, when i "ok"e