Hung CFDocument threads

Hi! I'm using ColdFusion 7 and I've been experiencing intermittent hung threads related to CFDocument. I finally got around to configuring SeeFusion to give me thread dumps of running threads, and I caught one in action today. The thread had been "running" for a couple of hours, even though the requestTimeout on the page was set to 500 seconds.
Here's the top of the thread dump:
Full thread dump Java HotSpot(TM) Server VM (1.4.2_13):
"jrpp-2833" runnable
        - locked <257> (a java.lang.Object)
        at ice.pilots.html4.CSSMatcher.OEAB(OEAB)
        at ice.pilots.html4.CSSMatcher.I(OEAB)
        at ice.pilots.html4.CSSMatcher.build(OEAB)
        at ice.pilots.html4.CSSLayout._maxCharacters(OEAB)
        at ice.pilots.html4.CSSLayout.S(OEAB)
        at ice.pilots.html4.CSSLayout.layout(OEAB)
        at ice.pilots.html4.ThePrinter.getPageCount(Unknown source)
        at ice.storm.print.AbstractStormPrinter.setStormPageFormat(OEAB)
        at ice.pilots.html4.ThePrinter.setStormPageFormat(Unknown source)
        at com.icesoft.util.PageDecorator.HeaderFooterDecoration.paint(HeaderFooterDecoration.java:2 03)
        at ice.pilots.html4.ThePrinter.printPage(Unknown source)
        at coldfusion.document.DocumentSection.process(DocumentSection.java:225)
Since I can't see the internals of these classes, I can't really troubleshoot any further. Can an Adobe person with access to the source code help to shed some light on what might be going on here?
I will continue to try to reproduce this problem and attempt to boil it down to a simple-ish test case. I just don't know how successful I'll be since it seems to be an intermittent problem.
Oh, and obviously I know that CF 7 is getting pretty long in the tooth, but we can't afford to pay thousands to upgrade right now and then spend hours and hours setting it up, testing, and fixing the inevetible upgrade issues.
Thanks for reading! Hope someone can help...
CF Version:                  7,0,2,142559  
Update Level: cfusion.ear/cfusion.war/WEB-INF/cfusion/lib/updates/hf702-70523.jar 

As far as I can tell, running the exact same request (that hung before) works for me, and quickly (<10 sec). As I suspected, this problem is intermittent and non-deterministic, so I don't think I'm going to be able to come up with a simplified test case.

Similar Messages

  • Sql loader problem - thread appears to be hung

    Sorry to ask such a strang question but I am not sure which direction to go with this one. Hoping someone might have an idea.
    I am running oracle 10g on windows. I have 3 servers all patched at the same levels. All 3 servers run a purchased utility called dbutility which copies data from a dms database to an oracle database using oracle's sql*loader. On one server the dbutility hangs when the dbutility program tries to start sql*loader. I must close the dos window to end the job - ctl c does not work.
    If I let the job sit long enought, it eventually gets the message:"BULK LOADER thread appears to be hung: main thread waiting".
    I called the company which supports the dbutility program and they think it is a windows or oracle problem.
    I can get the dbutility to work if I start the dbutility in verbose (trace) mode. For some reason slowing the dbutility down before starting the sql*load program fixes the problem.
    Has anyone ever had issues starting sql*loader?
    Does anyone have any ideas what might be causing this problem?
    Do you think reinstalling the sql*loader might help?
    Thank you!
    Confused:)

    Try to built a test case on your own.
    Make a little test creating an sql*loader ctl and try to load the file on a test table.
    If that work you should ask againg with your tool support.
    load data
    infile 'YOUR_FILE_HERE'
    badfile 'BAD_FILE'
    append
    into table test_ldr
    fields terminated by "," optionally enclosed by '"'          
    (caracter1,caracter2,caracter3,caracter4,caracter5)
    caraterN = your table column name.
    sqlldr.exe userid=username/password@db rows=1000 control=conrtol.ctl log=log.log errors=200 direct=y

  • How to properly implement custom ServiceFailurePolicy

    I'd like to use Coherence service guardian to interrupt & cancel hung cacheloader threads, which are going into infinite loop due to database connection became unresponsive.
    For that purpose I've tried first to create working testcase in order to learn how service guardian works.
    I've created a basic distributed cache with fake cache loader to imitate a delay in response, then an infinite loop, and custom test service guardian policy.
    I do see cache loader being interrupted by a call to onGuardableRecovery, but then strange thing happens: despite cacheloader thread being successfully released, service guardian still thinks that the service is stuck and tries several times to recover by calling onGuardableRecovery and finally to terminate service with the call to onGuardableTerminate.
    What I'm missing here or doing wrong ?
    Below are testcase classes & cache config:
    package test.guardian;
    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.cache.CacheStore;
    import com.tangosol.util.Base;
    import java.util.Collection;
    import java.util.Map;
    public class testCacheStore extends Base implements CacheStore {
    private String datasource_name = null;
    public static volatile boolean loop_thread = true;
    public testCacheStore(String p_datasource_name) {
    CacheFactory.log("inside cacheloader constructor, p_datasource_name="+p_datasource_name,CacheFactory.LOG_DEBUG);
    this.datasource_name = p_datasource_name;
    public Object load(Object oKey) {
    CacheFactory.log("inside cacheloader",CacheFactory.LOG_DEBUG);
    try {
    Thread.sleep(120*1000);
    catch (InterruptedException e) {
    CacheFactory.log("loader thread was interrupted"+"\n"+CacheFactory.getStackTrace(e),CacheFactory.LOG_DEBUG);
    while (testCacheStore.loop_thread) {String n = "1"+"2";}
    CacheFactory.log("loader stop_thread flag was set to stop thread",CacheFactory.LOG_DEBUG);
    CacheFactory.log("return value from the store",CacheFactory.LOG_DEBUG);
    return ((String) oKey)+"-value";
    public Map loadAll(Collection colKeys) {
    throw new UnsupportedOperationException();
    public void store(Object oKey, Object oValue) {
    public void storeAll(Map mapEntries) {
    throw new UnsupportedOperationException();
    public void erase(Object oKey) {
    public void eraseAll(Collection colKeys) {
    CacheFactory.log("inside eraseAll method", CacheFactory.LOG_DEBUG);
    throw new UnsupportedOperationException();
    package test.guardian;
    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.Guardable;
    import com.tangosol.net.ServiceFailurePolicy;
    import com.tangosol.net.Service;
    import com.tangosol.net.Cluster;
    public class mycustomGPolicy implements ServiceFailurePolicy {   
    public static mycustomGPolicy getInstance() {
    return new mycustomGPolicy();
    public mycustomGPolicy(String p_info) {
    CacheFactory.log("parametrized policy class instance created",CacheFactory.LOG_DEBUG);
    CacheFactory.log("p_info="+p_info,CacheFactory.LOG_DEBUG);
    public mycustomGPolicy() {
    CacheFactory.log("policy class instance created",CacheFactory.LOG_DEBUG);
    //CacheFactory.log("p_info="+p_info,CacheFactory.LOG_DEBUG);
    public void onGuardableRecovery(Guardable p_guardable, Service p_service) {
    CacheFactory.log("inside onGuardableRecovery",CacheFactory.LOG_DEBUG);
    CacheFactory.log("p_guardable="+p_guardable.toString(),CacheFactory.LOG_DEBUG);
    CacheFactory.log("p_guardable.getClass().getCanonicalName()="+p_guardable.getClass().getCanonicalName(),CacheFactory.LOG_DEBUG);
    testCacheStore.loop_thread = false;
    p_guardable.recover();
    p_guardable.getContext().heartbeat();
    public void onGuardableTerminate(Guardable p_guardable, Service p_service) {
    CacheFactory.log("inside onGuardableTerminate",CacheFactory.LOG_DEBUG);
    CacheFactory.log("p_guardable="+p_guardable.toString(),CacheFactory.LOG_DEBUG);
    CacheFactory.log("p_guardable.getClass().getCanonicalName()="+p_guardable.getClass().getCanonicalName(),CacheFactory.LOG_DEBUG);
    testCacheStore.loop_thread = false;
    p_guardable.terminate();
    public void onServiceFailed(Cluster p_cluster) {
    CacheFactory.log("inside onServiceFailed",CacheFactory.LOG_DEBUG);
    package test.guardian;
    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.NamedCache;
    public class testGuardianCache {
    public static void main(String[] args) {
    try {
    log("building cache instance");
    NamedCache cache = CacheFactory.getCache("test-guardable");
    log("requesting cache data");
    String resp4 = (String)cache.get("test1223g");
    log("response:");
    System.out.println(resp4+"\n");
    } catch (com.tangosol.io.pof.PortableException e) {
    Throwable e_src = e;
    while (e_src.getCause() != null ) {
    e_src = e_src.getCause();
    e_src.printStackTrace();
    public static void log(String info) {
    System.out.println(info);
    <?xml version="1.0" encoding="UTF-8"?>
    <cache-config>
         <caching-scheme-mapping>
              <cache-mapping>
                   <cache-name>test-guardable</cache-name>
                   <scheme-name>test-guardable-schema</scheme-name>
              </cache-mapping>
         </caching-scheme-mapping>
         <caching-schemes>
              <distributed-scheme>
                   <scheme-name>test-guardable-schema</scheme-name>
                   <service-name>test-guardable-service</service-name>
                   <backing-map-scheme>
                        <read-write-backing-map-scheme>
                             <cachestore-timeout>0</cachestore-timeout>
                             <internal-cache-scheme>
                                  <external-scheme>
                                  <scheme-name>LocalNioLtd</scheme-name>
                                  <high-units>10000</high-units>
                                  <unit-calculator>FIXED</unit-calculator>
                                  <nio-memory-manager>
                                  <initial-size>10M</initial-size>
                                  <maximum-size>50M</maximum-size>
                                  </nio-memory-manager>
                                  </external-scheme>
                             </internal-cache-scheme>
                             <cachestore-scheme>
                                  <class-scheme>
                                       <class-name>test.guardian.testCacheStore</class-name>
                                       <init-params>
                                       <init-param>
                                       <param-type>java.lang.String</param-type>
                                       <param-value>jdbc-coherence/MY-CACHE-DS</param-value>
                                       </init-param>
                                       </init-params>
                                  </class-scheme>
                             </cachestore-scheme>
                             <rollback-cachestore-failures>true</rollback-cachestore-failures>
                             <read-only>true</read-only>
                             <write-delay-seconds>0</write-delay-seconds>
                        </read-write-backing-map-scheme>
                   </backing-map-scheme>
                   <listener/>
                   <autostart>true</autostart>
                   <thread-count>10</thread-count>
                   <guardian-timeout>10000</guardian-timeout>
                   <service-failure-policy>
              <class-name>test.guardian.mycustomGPolicy</class-name>
                                       <init-params>
                                       <init-param>
                                       <param-name>datasource</param-name>
                                       <param-type>java.lang.String</param-type>
                                       <param-value>jdbc-coherence/MY-CACHE-DS</param-value>
                                       </init-param>
                                       </init-params>
              </service-failure-policy>
              </distributed-scheme>
         </caching-schemes>
    </cache-config>
    <?xml version="1.0"?>
    <coherence>
         <services>
              <service>
                   <service-type>DistributedCache</service-type>
                   <service-component>DistributedCache</service-component>
                   <init-params>
                        <init-param id="4">
                             <param-name>local-storage</param-name>
                             <param-value system-property="tangosol.coherence.distributed.localstorage">false</param-value>
                        </init-param>
                   </init-params>
              </service>
         </services>
         <cluster-config>
    <member-identity>
    <cluster-name>WORKSTATION</cluster-name>
    </member-identity>
              <unicast-listener>
                   <address system-property="tangosol.coherence.localhost">localhost</address>
                   <port system-property="tangosol.coherence.localport">7070</port>
                   <port-auto-adjust system-property="tangosol.coherence.localport.adjust">true</port-auto-adjust>
                   <well-known-addresses>
    <socket-address id="1"><address>localhost</address><port>7070</port></socket-address>
    </well-known-addresses>
              </unicast-listener>
              <authorized-hosts/>
              <logging-config>
                   <severity-level system-property="tangosol.coherence.log.level">5</severity-level>
                   <character-limit system-property="tangosol.coherence.log.limit">0</character-limit>
              </logging-config>
         </cluster-config>
         <logging-config>
              <severity-level>5</severity-level>
              <message-format>{date} &lt;{level}&gt; (thread={thread}, member={member}): {text}</message-format>
              <character-limit>8192</character-limit>
         </logging-config>
    </coherence>
    setlocal
    set COHERENCE_HOME=c:\coherence36
    set PROG_HOME=C:\coherence36\deploy
    set COH_OPTS=-cp %COHERENCE_HOME%\lib\coherence.jar;%PROG_HOME%\testguardable.jar
    set JMX_OPTS=-Dcom.sun.management.jmxremote -Dtangosol.coherence.management=all -Dtangosol.coherence.management.remote=true -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.ssl=false
    set OVERRIDE_OPTS=-Dtangosol.coherence.override=%PROG_HOME%\tangosol-coherence-override.xml
    rem
    rem
    start "1testguardable" /MAX C:\jdev10g\jdk\jre\bin\java.exe %COH_OPTS% %OVERRIDE_OPTS% -Xms256m -Xmx256m -Xloggc: -Dtangosol.coherence.distributed.localstorage=true -Dtangosol.coherence.cacheconfig=%PROG_HOME%\testguardable-cache-config.xml com.tangosol.net.DefaultCacheServer

    INFO - 2010-07-23 23:43:07.906 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/C:/coherence36/lib/coherence.jar!/tangosol-coherence.xml"
    INFO - 2010-07-23 23:43:07.921 <Info> (thread=main, member=n/a): Loaded operational overrides from "file:/C:/-PRG/coherence_ws/Guardian/deploy/tangosol-coherence-override.xml"
    DEBUG - 2010-07-23 23:43:07.937 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
    DEBUG -
    Oracle Coherence Version 3.6.0.0 Build 17229
    Grid Edition: Development mode
    Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
    INFO - 2010-07-23 23:43:08.531 <Info> (thread=main, member=n/a): Loaded cache configuration from "file:/C:/-PRG/coherence_ws/Guardian/deploy/testguardable-cache-config.xml"
    DEBUG - 2010-07-23 23:43:09.515 <D4> (thread=main, member=n/a): TCMP bound to /192.168.1.10:7070 using SystemSocketProvider
    INFO - 2010-07-23 23:43:41.000 <Info> (thread=Cluster, member=n/a): Created a new cluster "WORKSTATION" with Member(Id=1, Timestamp=2010-07-23 23:43:09.531, Address=192.168.1.10:7070, MachineId=26890, Location=site:local,machine:WS,process:5880, Role=CoherenceServer, Edition=Grid Edition, Mode=Development, CpuCount=2, SocketCount=1) UID=0xC0A8010A0000012A01427FDB690A1B9E
    INFO - 2010-07-23 23:43:41.015 <Info> (thread=main, member=n/a): Started cluster Name=WORKSTATION
    WellKnownAddressList(Size=1,
    WKA{Address=192.168.1.10, Port=7070}
    MasterMemberSet
    ThisMember=Member(Id=1, Timestamp=2010-07-23 23:43:09.531, Address=192.168.1.10:7070, MachineId=26890, Location=site:local,machine:WS,process:5880, Role=CoherenceServer)
    OldestMember=Member(Id=1, Timestamp=2010-07-23 23:43:09.531, Address=192.168.1.10:7070, MachineId=26890, Location=site:local,machine:WS,process:5880, Role=CoherenceServer)
    ActualMemberSet=MemberSet(Size=1, BitSetCount=2
    Member(Id=1, Timestamp=2010-07-23 23:43:09.531, Address=192.168.1.10:7070, MachineId=26890, Location=site:local,machine:WS,process:5880, Role=CoherenceServer)
    RecycleMillis=1200000
    RecycleSet=MemberSet(Size=0, BitSetCount=0
    TcpRing{Connections=[]}
    IpMonitor{AddressListSize=0}
    DEBUG - 2010-07-23 23:43:41.093 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
    DEBUG - 2010-07-23 23:43:41.718 <D5> (thread=main, member=1): parametrized policy class instance created
    DEBUG - 2010-07-23 23:43:41.718 <D5> (thread=main, member=1): p_info=jdbc-coherence/MY-CACHE-DS
    DEBUG - 2010-07-23 23:43:41.750 <D5> (thread=DistributedCache:test-guardable-service, member=1): Service test-guardable-service joined the cluster with senior service member 1
    INFO - 2010-07-23 23:43:41.890 <Info> (thread=main, member=1):
    Services
    ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.6, OldestMemberId=1}
    InvocationService{Name=Management, State=(SERVICE_STARTED), Id=1, Version=3.1, OldestMemberId=1}
    PartitionedCache{Name=test-guardable-service, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}
    Started DefaultCacheServer...
    DEBUG - 2010-07-23 23:44:50.250 <D5> (thread=Cluster, member=1): Member(Id=2, Timestamp=2010-07-23 23:44:50.258, Address=192.168.1.10:7072, MachineId=26890, Location=site:local,machine:WS,process:3376, Role=TestGuardianTestGuardianCache) joined Cluster with senior member 1
    DEBUG - 2010-07-23 23:44:50.390 <D5> (thread=Cluster, member=1): Member 2 joined Service Management with senior member 1
    DEBUG - 2010-07-23 23:44:51.093 <D5> (thread=Cluster, member=1): Member 2 joined Service test-guardable-service with senior member 1
    DEBUG - 2010-07-23 23:44:51.218 <D5> (thread=DistributedCache:test-guardable-service, member=1): inside cacheloader constructor, p_datasource_name=jdbc-coherence/MY-CACHE-DS
    DEBUG - 2010-07-23 23:44:51.359 <D5> (thread=test-guardable-serviceWorker:4, member=1): inside cacheloader
    DEBUG - 2010-07-23 23:45:00.375 <D5> (thread=Recovery Thread, member=1): inside onGuardableRecovery
    DEBUG - 2010-07-23 23:45:00.375 <D5> (thread=Recovery Thread, member=1): p_guardable=Guard{Daemon=test-guardable-serviceWorker:4}
    DEBUG - 2010-07-23 23:45:00.375 <D5> (thread=Recovery Thread, member=1): p_guardable.getClass().getCanonicalName()=com.tangosol.coherence.component.util.daemon.queueProcessor.Service$DaemonPool$Daemon$Guard
    ERROR - 2010-07-23 23:45:00.375 <Error> (thread=DistributedCache:test-guardable-service, member=1): Attempting recovery (due to soft timeout) of {WrapperGuardable Guard{Daemon=test-guardable-serviceWorker:4} Service=PartitionedCache{Name=test-guardable-service, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}}
    WARN - 2010-07-23 23:45:00.375 <Warning> (thread=Recovery Thread, member=1): A worker thread has been executing task: Message "GetRequest"
    FromMember=Member(Id=2, Timestamp=2010-07-23 23:44:50.258, Address=192.168.1.10:7072, MachineId=26890, Location=site:local,machine:WS,process:3376, Role=TestGuardianTestGuardianCache)
    FromMessageId=29
    Internal=false
    MessagePartCount=1
    PendingCount=0
    MessageType=59
    ToPollId=0
    Poll=null
    Packets
    [000]=Directed{PacketType=0x0DDF00D5, ToId=1, FromId=2, Direction=Incoming, ReceivedMillis=23:44:51.343, ToMemberSet=null, ServiceId=2, MessageType=59, FromMessageId=29, ToMessageId=27, MessagePartCount=1, MessagePartIndex=0, NackInProgress=false, ResendScheduled=none, Timeout=none, PendingResendSkips=0, DeliveryState=unsent, Body=0}
    Service=PartitionedCache{Name=test-guardable-service, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}
    ToMemberSet=MemberSet(Size=1, BitSetCount=2
    Member(Id=1, Timestamp=2010-07-23 23:43:09.531, Address=192.168.1.10:7070, MachineId=26890, Location=site:local,machine:WS,process:5880, Role=CoherenceServer)
    NotifySent=false
    } for 9016ms and appears to be stuck; attempting to interrupt: test-guardable-serviceWorker:4
    DEBUG - 2010-07-23 23:45:00.375 <D5> (thread=test-guardable-serviceWorker:4, member=1): loader thread was interrupted
         at java.lang.Thread.sleep(Native Method)
         at test.guardian.testCacheStore.load(testCacheStore.java:26)
         at com.tangosol.net.cache.ReadWriteBackingMap$CacheStoreWrapper.load(ReadWriteBackingMap.java:4759)
         at com.tangosol.net.cache.ReadWriteBackingMap$CacheStoreWrapper.loadInternal(ReadWriteBackingMap.java:4344)
         at com.tangosol.net.cache.ReadWriteBackingMap.get(ReadWriteBackingMap.java:807)
         at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onGetRequest(PartitionedCache.CDB:22)
         at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$GetRequest.run(PartitionedCache.CDB:1)
         at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:1)
         at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:32)
         at com.tangosol.coherence.component.util.DaemonPool$Daemon.onNotify(DaemonPool.CDB:63)
         at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
         at java.lang.Thread.run(Thread.java:595)
    DEBUG - 2010-07-23 23:45:00.375 <D5> (thread=test-guardable-serviceWorker:4, member=1): loader stop_thread flag was set to stop thread
    DEBUG - 2010-07-23 23:45:00.375 <D5> (thread=test-guardable-serviceWorker:4, member=1): return value from the store
    DEBUG - 2010-07-23 23:45:00.390 <D5> (thread=Cluster, member=1): TcpRing disconnected from Member(Id=2, Timestamp=2010-07-23 23:44:50.258, Address=192.168.1.10:7072, MachineId=26890, Location=site:local,machine:WS,process:3376, Role=TestGuardianTestGuardianCache) due to a peer departure; removing the member.
    DEBUG - 2010-07-23 23:45:00.390 <D5> (thread=Cluster, member=1): Member 2 left service Management with senior member 1
    DEBUG - 2010-07-23 23:45:00.390 <D5> (thread=Cluster, member=1): Member 2 left service test-guardable-service with senior member 1
    DEBUG - 2010-07-23 23:45:00.390 <D5> (thread=Cluster, member=1): Member(Id=2, Timestamp=2010-07-23 23:45:00.39, Address=192.168.1.10:7072, MachineId=26890, Location=site:local,machine:WS,process:3376, Role=TestGuardianTestGuardianCache) left Cluster with senior member 1
    ERROR - 2010-07-23 23:48:16.250 <Error> (thread=Cluster, member=1): Attempting recovery (due to soft timeout) of {WrapperGuardable Guard{Daemon=DistributedCache:test-guardable-service} Service=PartitionedCache{Name=test-guardable-service, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}}
    DEBUG - 2010-07-23 23:48:16.250 <D5> (thread=Recovery Thread, member=1): inside onGuardableRecovery
    DEBUG - 2010-07-23 23:48:16.250 <D5> (thread=Recovery Thread, member=1): p_guardable=Guard{Daemon=DistributedCache:test-guardable-service}
    DEBUG - 2010-07-23 23:48:16.250 <D5> (thread=Recovery Thread, member=1): p_guardable.getClass().getCanonicalName()=com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$Guard
    DEBUG - Interrupted PartitionedCache, Thread[DistributedCache:test-guardable-service,5,Cluster]
    ERROR - 2010-07-23 23:48:46.765 <Error> (thread=Cluster, member=1): Terminating guarded execution (due to hard timeout) of {WrapperGuardable Guard{Daemon=DistributedCache:test-guardable-service} Service=PartitionedCache{Name=test-guardable-service, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}}
    DEBUG - 2010-07-23 23:48:46.765 <D5> (thread=Termination Thread, member=1): inside onGuardableTerminate
    DEBUG - 2010-07-23 23:48:46.765 <D5> (thread=Termination Thread, member=1): p_guardable=Guard{Daemon=DistributedCache:test-guardable-service}
    DEBUG - 2010-07-23 23:48:46.765 <D5> (thread=Termination Thread, member=1): p_guardable.getClass().getCanonicalName()=com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$Guard
    DEBUG - 2010-07-23 23:48:46.765 <D5> (thread=DistributedCache:test-guardable-service, member=1): Service test-guardable-service left the cluster
    INFO - 2010-07-23 23:48:46.890 <Info> (thread=main, member=1): Restarting Service: test-guardable-service
    DEBUG - 2010-07-23 23:48:46.906 <D5> (thread=main, member=1): parametrized policy class instance created
    DEBUG - 2010-07-23 23:48:46.906 <D5> (thread=main, member=1): p_info=jdbc-coherence/MY-CACHE-DS
    DEBUG - 2010-07-23 23:48:46.906 <D5> (thread=DistributedCache:test-guardable-service, member=1): Service test-guardable-service joined the cluster with senior service member 1
    ERROR - 2010-07-23 23:48:55.265 <Error> (thread=Cluster, member=1): Attempting recovery (due to soft timeout) of {WrapperGuardable Guard{Daemon=DistributedCache:test-guardable-service} Service=PartitionedCache{Name=test-guardable-service, State=(SERVICE_STOPPED), Not initialized}}
    DEBUG - 2010-07-23 23:48:55.265 <D5> (thread=Recovery Thread, member=1): inside onGuardableRecovery
    DEBUG - 2010-07-23 23:48:55.265 <D5> (thread=Recovery Thread, member=1): p_guardable=Guard{Daemon=DistributedCache:test-guardable-service}
    DEBUG - 2010-07-23 23:48:55.265 <D5> (thread=Recovery Thread, member=1): p_guardable.getClass().getCanonicalName()=com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$Guard
    ERROR - 2010-07-23 23:48:56.281 <Error> (thread=Cluster, member=1): Terminating guarded execution (due to hard timeout) of {WrapperGuardable Guard{Daemon=DistributedCache:test-guardable-service} Service=PartitionedCache{Name=test-guardable-service, State=(SERVICE_STOPPED), Not initialized}}
    DEBUG - 2010-07-23 23:48:56.281 <D5> (thread=Termination Thread, member=1): inside onGuardableTerminate
    DEBUG - 2010-07-23 23:48:56.281 <D5> (thread=Termination Thread, member=1): p_guardable=Guard{Daemon=DistributedCache:test-guardable-service}
    DEBUG - 2010-07-23 23:48:56.281 <D5> (thread=Termination Thread, member=1): p_guardable.getClass().getCanonicalName()=com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$Guard

  • MedRec example under stress

    Hi,
    I tested MedRec example of WLS 8.1 cluster. The configuration is
    Tiger 4 box (IA 64, 4* 1.4 GHZ CPU, 8GB RAM):
    OS - RedHat AS 2.1 (kernel: 2.4.18-e.12smp)
    3 WLS instance - one Admin, 2 managed server in one cluster.
    JVM options of managed servers are -Xms1024m -Xmx 1024m -Xgc:parellel.
    Each managed server has 35 default threads.
    HP box (IA 64, 2* CPU) runs Oracle 9.2.0.2 as persistent store. 35
    connections from each managed server.
    OS - RedHat AS 2.1 (kernel: 2.4.18-e.12smp)
    There are 2 kind of virtual users: patient - edit profile, view records;
    physician - search patient, view patient's records, prescribe.
    When there are 35 patients and 35 physicians request any one of the
    managed server, it makes that server hung.
    Thread dump of JRockit is follows.
    Is it a kind of Out-of-Thread deadlock?
    Thanks,
    Kevin Huang
    ===== FULL THREAD DUMP ===============
    Sun Sep 14 17:38:46 2003
    "Noname" id: 0x0x00010000 prio: 5 WAITING, DAEMON, GCABLE
    thread: 0x0x2000000000ae54d8 lastj: 0x0x2000000041bfeea0
    at COM.jrockit.vm.MemSystem.mmParallelEP(Native
    Method)@0x2000000041a69270
    --- End of stack trace
    "Noname" id: 0x0x00020000 prio: 5 WAITING, DAEMON, GCABLE
    thread: 0x0x2000000000ae5660 lastj: 0x0x2000000041c8eea0
    at COM.jrockit.vm.MemSystem.mmParallelEP(Native
    Method)@0x2000000041a69270
    --- End of stack trace
    "Noname" id: 0x0x00030000 prio: 5 WAITING, DAEMON, GCABLE
    thread: 0x0x2000000000ae57e8 lastj: 0x0x2000000041d1eea0
    at COM.jrockit.vm.MemSystem.mmParallelEP(Native
    Method)@0x2000000041a69270
    --- End of stack trace
    "Noname" id: 0x0x00040000 prio: 5 WAITING, DAEMON, GCABLE
    thread: 0x0x2000000000ae5970 lastj: 0x0x2000000041dbeea0
    at COM.jrockit.vm.MemSystem.mmParallelEP(Native
    Method)@0x2000000041a69270
    --- End of stack trace
    "Finalizer" id: 0x0x00070000 prio: 8 WAITING, DAEMON, GCABLE
    thread: 0x0x2000000000961218 lastj: 0x0x200000004240eea0
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.wait0(Unknown Source)@0x2000000041a699c0
    at java.lang.Object.wait0(Unknown Source)@0x2000000041a763c0
    at
    java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)@0x2000000041a82
    990
    at
    java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)@0x2000000041a82
    7b0
    at
    java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)@0x2000000041
    a7ea10
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "SignalHandler" id: 0x0x00080000 prio: 5 ACTIVE, DAEMON, GCABLE
    thread: 0x0x20000000009009d8 lastj: 0x0x2000000042c1eea0
    at COM.jrockit.vm.SignalHandler.waitForSignals(Native
    Method)@0x2000000041ae3070
    at COM.jrockit.vm.SignalHandler$1.run(Unknown Source)@0x2000000041ae3320
    at java.lang.Thread.run(Unknown Source)@0x2000000041a70320
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "Hotspot Detector" id: 0x0x00090000 prio: 5 ACTIVE, DAEMON, GCABLE
    thread: 0x0x2000000000900ce0 lastj: 0x0x2000000042d7eea0
    at COM.jrockit.vm.HotspotDetector.findHotSpotCandidate(Native
    Method)@0x2000000041ae3440
    at COM.jrockit.vm.HotspotDetector.run(Unknown Source)@0x2000000041ae35b0
    at java.lang.Thread.run(Unknown Source)@0x2000000041a70320
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "Reference Handler" id: 0x0x000a0000 prio: 10 WAITING, DAEMON, GCABLE
    thread: 0x0x2000000000901090 lastj: 0x0x2000000042eceea0
    at java.lang.ref.Reference.getPending(Native Method)@0x2000000041ae44f0
    at java.lang.ref.Reference.access$000(Unknown Source)@0x2000000041ae4690
    at java.lang.ref.Reference$ReferenceHandler.run(Unknown
    Source)@0x2000000041ae42a0
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "Thread-1" id: 0x0x000b0000 prio: 5 WAITING, NONDAEMON, GCABLE
    thread: 0x0x200000000085fff8 lastj: 0x0x60000fffffff8dd0
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.wait0(Unknown Source)@0x2000000041a699c0
    at
    weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:1170)@0x200000005751a1f0
    at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:384)@0x2000000043142890
    at weblogic.Server.main(Server.java:32)@0x20000000431404d0
    --- End of stack trace
    "ExecuteThread: '0' for queue: 'weblogic.kernel.Default'" id: 0x0x000c0000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318bc50 lastj: 0x0x200000004681ed60
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.releaseBean(StatelessSessionPool.ja
    va:209)@0x20000000588b9df0
    at
    weblogic.ejb20.manager.StatelessManager.releaseBean(StatelessManager.java:38
    6)@0x20000000588b9d90
    at
    weblogic.ejb20.internal.StatelessEJBObject.postInvoke(StatelessEJBObject.jav
    a:147)@0x20000000588ab050
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.addRecord(Physic
    ianSessionEJB_n7enxc_EOImpl.java:316)@0x200000005
    2eea780
    at
    com.bea.medrec.actions.CreateVisitAction.save(CreateVisitAction.java:289)@0x
    2000000056b195c0
    at COM.jrockit.reflect.NativeMethodInvoker.invoke0(Native
    Method)@0x2000000041b54440
    at COM.jrockit.reflect.NativeMethodInvoker.invoke(Unknown
    Source)@0x2000000041b54a60
    at COM.jrockit.reflect.VirtualNativeMethodInvoker.invoke(Unknown
    Source)@0x2000000043ba6070
    at java.lang.reflect.Method.invoke(Unknown Source)@0x2000000041b52c10
    at
    org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:
    280)@0x2000000052e9fb10
    at
    org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.
    java:252)@0x2000000052e9ed00
    at
    com.bea.medrec.actions.BaseLookupDispatchAction.execute(BaseLookupDispatchAc
    tion.java:59)@0x2000000052e9dbf0
    at
    com.bea.medrec.actions.PhysBaseLookupDispatchAction.execute(PhysBaseLookupDi
    spatchAction.java:47)@0x2000000056b12de0
    at
    com.bea.medrec.actions.CreateVisitAction.execute(CreateVisitAction.java:61)@
    0x2000000056b16d90
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:520)@0x2000
    000052e91f80
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:760)@0x2000000058841
    780
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '1' for queue: 'weblogic.kernel.Default'" id: 0x0x000d0000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318c700 lastj: 0x0x200000004692eea0
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:8
    8)@0x200000005887df50
    at
    weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:140)
    @0x200000005887d6c0
    at
    weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:130)@0x20
    0000005887bea0
    at
    weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java
    :70)@0x2000000058874880
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.getRecordsSummar
    y(PhysicianSessionEJB_n7enxc_EOImpl.java:81)@0x20
    00000052ecb1c0
    at
    com.bea.medrec.actions.PhysViewRecordsSummaryAction.getMedicalRecords(PhysVi
    ewRecordsSummaryAction.java:103)@0x2000000056b153
    c0
    at
    com.bea.medrec.actions.PhysViewRecordsSummaryAction.executeAction(PhysViewRe
    cordsSummaryAction.java:89)@0x2000000056b14860
    at
    com.bea.medrec.actions.BaseAction.execute(BaseAction.java:70)@0x2000000052e7
    6220
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:502)@0x20000
    00052e651e0
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:740)@0x2000000058841
    b50
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '2' for queue: 'weblogic.kernel.Default'" id: 0x0x000e0000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318c8d8 lastj: 0x0x2000000046a3eea0
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:8
    8)@0x200000005887df50
    at
    weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:140)
    @0x200000005887d6c0
    at
    weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:130)@0x20
    0000005887bea0
    at
    weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java
    :70)@0x2000000058874880
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.getRecord(Physic
    ianSessionEJB_n7enxc_EOImpl.java:133)@0x200000005
    2ed8a90
    at
    com.bea.medrec.actions.PhysViewRecordAction.executeAction(PhysViewRecordActi
    on.java:60)@0x2000000056b160d0
    at
    com.bea.medrec.actions.BaseAction.execute(BaseAction.java:70)@0x2000000052e7
    6220
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:502)@0x20000
    00052e651e0
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:740)@0x2000000058841
    b50
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" id: 0x0x000f0000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318cab0 lastj: 0x0x2000000046b4eea0
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.releaseBean(StatelessSessionPool.ja
    va:209)@0x20000000588b9df0
    at
    weblogic.ejb20.manager.StatelessManager.releaseBean(StatelessManager.java:38
    6)@0x20000000588b9d90
    at
    weblogic.ejb20.internal.StatelessEJBObject.postInvoke(StatelessEJBObject.jav
    a:147)@0x20000000588ab050
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.getRecordsSummar
    y(PhysicianSessionEJB_n7enxc_EOImpl.java:108)@0x2
    000000052ecb510
    at
    com.bea.medrec.actions.PhysViewRecordsSummaryAction.getMedicalRecords(PhysVi
    ewRecordsSummaryAction.java:103)@0x2000000056b153
    c0
    at
    com.bea.medrec.actions.PhysViewRecordsSummaryAction.executeAction(PhysViewRe
    cordsSummaryAction.java:89)@0x2000000056b14860
    at
    com.bea.medrec.actions.BaseAction.execute(BaseAction.java:70)@0x2000000052e7
    6220
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:502)@0x20000
    00052e651e0
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:740)@0x2000000058841
    b50
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '4' for queue: 'weblogic.kernel.Default'" id: 0x0x00100000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318cc88 lastj: 0x0x2000000046c5ed60
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:8
    8)@0x200000005887df50
    at
    weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:140)
    @0x200000005887d6c0
    at
    weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:130)@0x20
    0000005887bea0
    at
    weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java
    :70)@0x2000000058874880
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.searchPatients(P
    hysicianSessionEJB_n7enxc_EOImpl.java:29)@0x20000
    00052ea40a0
    at
    com.bea.medrec.actions.SearchResultsAction.processSearch(SearchResultsAction
    .java:111)@0x2000000056b13950
    at
    com.bea.medrec.actions.SearchResultsAction.search(SearchResultsAction.java:9
    2)@0x2000000056b135d0
    at COM.jrockit.reflect.NativeMethodInvoker.invoke0(Native
    Method)@0x2000000041b54440
    at COM.jrockit.reflect.NativeMethodInvoker.invoke(Unknown
    Source)@0x2000000041b54a60
    at COM.jrockit.reflect.VirtualNativeMethodInvoker.invoke(Unknown
    Source)@0x2000000043ba6070
    at java.lang.reflect.Method.invoke(Unknown Source)@0x2000000041b52c10
    at
    org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:
    280)@0x2000000052e9fb10
    at
    org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.
    java:252)@0x2000000052e9ed00
    at
    com.bea.medrec.actions.BaseLookupDispatchAction.execute(BaseLookupDispatchAc
    tion.java:59)@0x2000000052e9dbf0
    at
    com.bea.medrec.actions.PhysBaseLookupDispatchAction.execute(PhysBaseLookupDi
    spatchAction.java:47)@0x2000000056b12de0
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:520)@0x2000
    000052e91f80
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:760)@0x2000000058841
    780
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '5' for queue: 'weblogic.kernel.Default'" id: 0x0x00110000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318ce60 lastj: 0x0x2000000046d6eea0
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:8
    8)@0x200000005887df50
    at
    weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:140)
    @0x200000005887d6c0
    at
    weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:130)@0x20
    0000005887bea0
    at
    weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java
    :70)@0x2000000058874880
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.getRecord(Physic
    ianSessionEJB_n7enxc_EOImpl.java:133)@0x200000005
    2ed8a90
    at
    com.bea.medrec.actions.PhysViewRecordAction.executeAction(PhysViewRecordActi
    on.java:60)@0x2000000056b160d0
    at
    com.bea.medrec.actions.BaseAction.execute(BaseAction.java:70)@0x2000000052e7
    6220
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:502)@0x20000
    00052e651e0
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:740)@0x2000000058841
    b50
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '6' for queue: 'weblogic.kernel.Default'" id: 0x0x00120000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318d038 lastj: 0x0x2000000046e7ed60
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x2000000041a62d30
    at java.lang.Object.monitorEnter(Native Method)@0x2000000041a632a0
    at
    weblogic.ejb20.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:8
    8)@0x200000005887df50
    at
    weblogic.ejb20.manager.StatelessManager.preInvoke(StatelessManager.java:140)
    @0x200000005887d6c0
    at
    weblogic.ejb20.internal.BaseEJBObject.preInvoke(BaseEJBObject.java:130)@0x20
    0000005887bea0
    at
    weblogic.ejb20.internal.StatelessEJBObject.preInvoke(StatelessEJBObject.java
    :70)@0x2000000058874880
    at
    com.bea.medrec.controller.PhysicianSessionEJB_n7enxc_EOImpl.searchPatients(P
    hysicianSessionEJB_n7enxc_EOImpl.java:29)@0x20000
    00052ea40a0
    at
    com.bea.medrec.actions.SearchResultsAction.processSearch(SearchResultsAction
    .java:111)@0x2000000056b13950
    at
    com.bea.medrec.actions.SearchResultsAction.search(SearchResultsAction.java:9
    2)@0x2000000056b135d0
    at COM.jrockit.reflect.NativeMethodInvoker.invoke0(Native
    Method)@0x2000000041b54440
    at COM.jrockit.reflect.NativeMethodInvoker.invoke(Unknown
    Source)@0x2000000041b54a60
    at COM.jrockit.reflect.VirtualNativeMethodInvoker.invoke(Unknown
    Source)@0x2000000043ba6070
    at java.lang.reflect.Method.invoke(Unknown Source)@0x2000000041b52c10
    at
    org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:
    280)@0x2000000052e9fb10
    at
    org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.
    java:252)@0x2000000052e9ed00
    at
    com.bea.medrec.actions.BaseLookupDispatchAction.execute(BaseLookupDispatchAc
    tion.java:59)@0x2000000052e9dbf0
    at
    com.bea.medrec.actions.PhysBaseLookupDispatchAction.execute(PhysBaseLookupDi
    spatchAction.java:47)@0x2000000056b12de0
    at
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProces
    sor.java:480)@0x2000000052e75fb0
    at
    org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
    @0x2000000052e69550
    at
    org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)@0x20
    00000052e655c0
    at
    org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:520)@0x2000
    000052e91f80
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:760)@0x2000000058841
    780
    at
    javax.servlet.http.HttpServlet.service(HttpServlet.java:853)@0x200000005756a
    500
    at
    weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(Servle
    tStubImpl.java:1053)@0x200000005756a080
    at
    weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
    :387)@0x2000000057561d20
    at
    weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)@0x20000000
    57560a60
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    com.bea.medrec.filters.RequestEncodingFilter.doFilter(RequestEncodingFilter.
    java:44)@0x2000000052e64f80
    at
    weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27)@
    0x2000000057560830
    at
    weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(W
    ebAppServletContext.java:6316)@0x200000005755ed10
    at
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubjec
    t.java:317)@0x200000004b0edad0
    at
    weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)@0x
    200000004b0ed670
    at
    weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
    ntext.java:3622)@0x200000005754ae20
    at
    weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
    :2569)@0x2000000057548c30
    at
    weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)@0x2000000045c4
    30e0
    at
    weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)@0x2000000045bd6310
    at java.lang.Thread.startThreadFromVM(Unknown Source)@0x2000000041a6fbc0
    --- End of stack trace
    "ExecuteThread: '7' for queue: 'weblogic.kernel.Default'" id: 0x0x00130000
    prio: 5 LOCKED, DAEMON, GCABLE
    thread: 0x0x200000000318d210 lastj: 0x0x2000000046f8ed60
    at java.lang.Thread.doYield(Native Method)@0x2000000041a624d0
    at java.lang.Object.lockFat0(Unknown Source)@0x20000

    I'm sure there are details on this in the documentation, but, in short:
              -- If you want to support persistent messages (which I'm sure MedRec does), then your JMS server has to be configured with a store.
              -- You can use either a file store or a JDBC store.
              -- If you use a JDBC store, it uses the JDBC connection pool you provide and the username/password you provide there to create two tables in which it stores persistent messages.
              -- The database username you use for this has to have permission to create a table.

  • Java.util.Collections$SynchronizedSet locked???

    Hi,
    I met a problem which is hard to diagnose:
    platform: HPUX 11i
    oracle client: Oracle 9.2.0.3
    weblogic: weblogic7.0 sp1
    pool type: oci
    after running for a while, the whole system hung, and thread dump is like the
    following:
    the locked thread:
    "ExecuteThread: '9' for queue: 'default'" daemon prio=10 tid=0x00318858 nid=21
    lwp_id=77267 waiting for monitor entry [0x45e81000..0x45e80500]
         at oracle.jdbc.driver.OraclePreparedStatement.close(OraclePreparedStatement.java:394)
         - waiting to lock <0x505f6fd8> (a oracle.jdbc.driver.OracleConnection)
         at weblogic.jdbc.common.internal.ConnectionEnv.dropStatement(ConnectionEnv.java:836)
         at weblogic.jdbc.jts.Statement.close(Statement.java:237)
         at weblogic.jdbc.rmi.internal.StatementImpl.close(StatementImpl.java:112)
         at weblogic.jdbc.rmi.SerialStatement.close(SerialStatement.java:127)
         at weblogic.jdbc.rmi.SerialConnection.closeAndClearAllStatements(SerialConnection.java:485)
         - locked <0x774f04b8> (a java.util.Collections$SynchronizedSet)
         at weblogic.jdbc.rmi.SerialConnection.$BIP$close(SerialConnection.java:210)
         at weblogic.jdbc.rmi.SerialConnection.close(SerialConnection.java)
         at com.ebao.life.bean.pub.common.DBean.close(DBean.java:222)
    please note that 0x774f04b8was locked by this thread.
    other thread are all waiting for the lock:
    "ExecuteThread: '10' for queue: 'default'" daemon prio=10 tid=0x00319188 nid=22
    lwp_id=77268 waiting for monitor entry [0x45dc1000..0x45dc0500]
         at weblogic.jdbc.rmi.SerialConnection.closeAndClearAllStatements(SerialConnection.java:480)
         - waiting to lock <0x774f04b8> (a java.util.Collections$SynchronizedSet)
         at weblogic.jdbc.rmi.SerialConnection.$BIP$close(SerialConnection.java:210)
         at weblogic.jdbc.rmi.SerialConnection.close(SerialConnection.java)
         at com.ebao.life.bean.pub.common.DBean.close(DBean.java:222)
         at com.ebao.pub.util.DBean.close(DBean.java:455)
         at com.ebao.general.bs.service.autounderwriting.DbFlag.closeCon(DbFlag.java:111)
    What happened and what may be the cause?
    Hope your answer.
    Thanks&Rgds
    Lumin

    lumin wrote:
    Hi,
    I met a problem which is hard to diagnose:
    platform: HPUX 11i
    oracle client: Oracle 9.2.0.3
    weblogic: weblogic7.0 sp1
    pool type: oci
    after running for a while, the whole system hung, and thread dump is like the
    following:
    the locked thread:
    "ExecuteThread: '9' for queue: 'default'" daemon prio=10 tid=0x00318858 nid=21
    lwp_id=77267 waiting for monitor entry [0x45e81000..0x45e80500]
         at oracle.jdbc.driver.OraclePreparedStatement.close(OraclePreparedStatement.java:394)
         - waiting to lock <0x505f6fd8> (a oracle.jdbc.driver.OracleConnection)
         at weblogic.jdbc.common.internal.ConnectionEnv.dropStatement(ConnectionEnv.java:836)
         at weblogic.jdbc.jts.Statement.close(Statement.java:237)
         at weblogic.jdbc.rmi.internal.StatementImpl.close(StatementImpl.java:112)
         at weblogic.jdbc.rmi.SerialStatement.close(SerialStatement.java:127)
         at weblogic.jdbc.rmi.SerialConnection.closeAndClearAllStatements(SerialConnection.java:485)
         - locked <0x774f04b8> (a java.util.Collections$SynchronizedSet)
         at weblogic.jdbc.rmi.SerialConnection.$BIP$close(SerialConnection.java:210)
         at weblogic.jdbc.rmi.SerialConnection.close(SerialConnection.java)
         at com.ebao.life.bean.pub.common.DBean.close(DBean.java:222)
    please note that 0x774f04b8was locked by this thread.
    other thread are all waiting for the lock:
    "ExecuteThread: '10' for queue: 'default'" daemon prio=10 tid=0x00319188 nid=22
    lwp_id=77268 waiting for monitor entry [0x45dc1000..0x45dc0500]
         at weblogic.jdbc.rmi.SerialConnection.closeAndClearAllStatements(SerialConnection.java:480)
         - waiting to lock <0x774f04b8> (a java.util.Collections$SynchronizedSet)
         at weblogic.jdbc.rmi.SerialConnection.$BIP$close(SerialConnection.java:210)
         at weblogic.jdbc.rmi.SerialConnection.close(SerialConnection.java)
         at com.ebao.life.bean.pub.common.DBean.close(DBean.java:222)
         at com.ebao.pub.util.DBean.close(DBean.java:455)
         at com.ebao.general.bs.service.autounderwriting.DbFlag.closeCon(DbFlag.java:111)
    What happened and what may be the cause?
    Hope your answer.
    Thanks&Rgds
    LuminHi. What is the chance you have more than one thread closing the same connection?
    Joe

  • Re: [SunONE-JATO] Frame Loading

    Divakar,
    A very challenging scenario for all app servers, especially J2EE web
    containers, is concurrent access to session. The HttpSession API does
    not provide an interface for either the container vendor or the
    developer to manage concurrency issues or events discretely; therefore,
    there is little JATO or any framework can do to help you with
    concurrency issues AND not add container behavior. The container
    vendors have to rely upon implementation details to manage these complex
    scenarios. Be that as it may (which is mostly my opinion anyways), the
    iAS engineers have worked long and hard the last year from iAS 6.0 sp2
    thru 6.5 to deal with issues around session concurrency; the discussion
    of which can be fairly verbose. What is important and in some ways
    unfortunate, is that the handling on these concurrency scenarios is
    different between releases. I believe that there is no blocking/locking
    in sp2 and earlier (resembling NetDynamics 3.x behavior). In sp3 (your
    case) concurrent access was serialized (resembling in some ways
    NetDynamics 4.x and 5.x) In sp4 it was changed again and finally in 6.5
    it was improved to ensure data coherency and stability with locking
    while eliminating locks within the same JVM. If you would like details
    email me offline at [matthew.stevens@s...] and I will see what I can do.
    No matter what version of iAS you are using, please ensure that your
    application is deployed with Sticky load-balancing such that request
    process is handled by the same JVM per session. In this way, you avoid
    the 'distributed' computing problem of locking across processes and
    machines. In other words, you have far less chance of hung request
    threads if you use sticky load-balancing.
    You need to establish some debug techniques in your development
    environment for J2EE to assess hangs or latencies (there are plenty of
    synchronous APIs in any J2EE container so this is not just an iAS thing
    or Session concurrency thing). If you are on Solaris you may issue a
    kill -3 PID
    of the JVM running the web container. Sending this SIGQUIT signal to
    the JVM will trap the JVM and result in a full thread dump to the
    STDERR. For instance, for iAS you would find the JVM (.kjs) spawned by
    the kjs script and send it a SIGQUIT. You should very easily see in the
    stack traces of the threads, your own thread which is above the dispatch
    to your module servlet [ie. you will see the
    <ApplicationServletBase.processRequest()] stack frame in the stack trace.
    If you are on Win32 you can run the iAS engines in the foreground by
    invoking iAS from the command line [kas -debug] in which case the KJS
    will appear in its own console window which you can CTRLBREAK to see the
    stack trace.
    In this way, you can see what your web container is doing while it is
    latent or hung.
    In your case, you will probably see your request thread blocked in calls
    to PlatformNASSession or PlatformLITESession. These two classes are the
    HttpSession implementations for iAS. The 'NAS' session uses the HA
    Dsync sub-system for replicated attribute backing store and the 'LITE'
    session uses an regular HashTable for the backing store.
    Again, concurrency issues are FAR MORE complex with the distributed
    implementation. Therefore, as an isolation technique, you may want to
    confirm that you are using session-impl=lite in your iAS deployment
    descriptor so as to avoid the Dsync implementation. Using Session LITE
    should eliminate the hangs during concurrent frame loads.
    Is it possible for you to upgrade to iAS 6.5? What about iAS 6.0 sp4?
    Please remember to leverage the JATO front controller events, even if it
    is for debug. You have the onBeforeRequest event where you could
    intercept the front controller before it has bothered with the
    HttpSession APIs. For isolation purposes, you could enter a critical
    section from your derived onBeforeRequest event to see whether
    concurrency is an issue in your case.
    I hope this "frames" the problem space for you 8-) Please post any
    questions or results from your testing and we can try to help further.
    The best information we can use to help you is the stack trace for the
    hang condition.
    thanks,
    matt
    padiyar wrote:
    I am facing problems with FrameSet loading.
    I have a frameset having two frames and src defined
    for both. Sometimes one of the frames doesnt get loaded.
    I am using iAS 6.0 with SP3 and JATO 1.2
    <FRAMESET COLS="200,*" BORDER=0 FRAMEBORDER=no>
    <FRAME NAME="Frame1" SRC="../AppName/pgMenu" MARGINHEIGHT=0
    MARGINWIDTH=10 SCROLLING=auto NORESIZE >
    <FRAME NAME="Frame2" SRC="../AppName/pgDetail" MARGINHEIGHT=6
    MARGINWIDTH=12 SCROLLING=auto RESIZE >
    </FRAMESET>
    From the kjs logs i noticed that onAfterRequest is not getting called
    while first frame's viewbean is being executed and kjs just hangs.
    If I set the SRC for one of the frames as viewbean and the other one
    a blank.html and trigger the loading of second frame using javascript
    on load of the first frame it works fine. But i dont
    want to go for this approach.
    Did anybody faced this problem and know how to fix this without using
    javascripts?
    Thanks,
    Divakar
    To download the latest version of JATO, please visit:
    http://www.sun.com/software/download/developer/5102.html
    For more information about JATO, please visit:
    http://developer.iplanet.com/tech/appserver/framework/index.jsp

    test

  • OIM 9.1.0.1870.1 with Oracle database 11.1.0.7 - Hung threads

    Anyone seen issues like hung threads using OIM9.1.0 with Oracle Database 11.1.0.7?

    Thanks guys for the quick responses.
    But I have tried both the options for putting the jar in Third Party and also by uploading using the OOTB utility UploadJar.sh.
    But it is giving the same error.
    I have tired rebouncing the server and also Purge cached, but no success.
    Just to mention again, I have tried with all the last 3 postgres JDBC4 driver available on the site (u mentioned).
    So, any other clue?
    Thanks in advance.

  • Hung Threads with multi thread processing

    We use TopLink v10.1.3.5
    One of our applications uses multi-core processing and when multiple threads try to access lazily loaded 1-m relationships at the same time, the threads just hang forever. There are no errors but the thread dump says it’s waiting on a condition.
    We do not use cache Synchronization.
    The 1-m relationship is privately owned, uses Batch reading and Indirection.
    The problem occurs intermittently and we cannot reproduce it at will.
    Found this discussion, but since we are on 10.3.5, I think we should already have the patch?
    Hung Threads (Toplink 10.1.3)
    Can anyone please provide any help on this.
    Thanks for the help.
    Thread dump -
    at java/lang/Object.wait(Native Method)
    at java/lang/Object.wait(Object.java:167(Compiled Code))
    at oracle/toplink/internal/helper/ConcurrencyManager.acquire(ConcurrencyManager.java:76(Compiled Code))
    at oracle/toplink/internal/identitymaps/CacheKey.acquire(CacheKey.java:85(Compiled Code))
    at oracle/toplink/internal/identitymaps/IdentityMap.acquireLock(IdentityMap.java:85(Compiled Code))
    at oracle/toplink/internal/identitymaps/IdentityMapManager.acquireLock(IdentityMapManager.java:101(Compiled Code))
    at oracle/toplink/internal/sessions/IdentityMapAccessor.acquireLock(IdentityMapAccessor.java:68(Compiled Code))
    at oracle/toplink/internal/sessions/IdentityMapAccessor.acquireLock(IdentityMapAccessor.java:58(Compiled Code))
    at oracle/toplink/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:502(Compiled Code))
    at oracle/toplink/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:382(Compiled Code))
    at oracle/toplink/mappings/OneToOneMapping.valueFromRow(OneToOneMapping.java:1020(Compiled Code))
    at oracle/toplink/mappings/DatabaseMapping.readFromRowIntoObject(DatabaseMapping.java:1045(Compiled Code))
    at oracle/toplink/internal/descriptors/ObjectBuilder.buildAttributesIntoObject(ObjectBuilder.java:245(Compiled Code))
    at oracle/toplink/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:564(Compiled Code))
    at oracle/toplink/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:382(Compiled Code))
    at oracle/toplink/internal/descriptors/ObjectBuilder.buildObjectsInto(ObjectBuilder.java:678(Compiled Code))
    at oracle/toplink/internal/queryframework/DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:142(Compiled Code))
    at oracle/toplink/queryframework/ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:483(Compiled Code))
    at oracle/toplink/queryframework/ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:813(Compiled Code))
    at oracle/toplink/queryframework/DatabaseQuery.execute(DatabaseQuery.java:620(Compiled Code))
    at oracle/toplink/queryframework/ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:781(Compiled Code))
    at oracle/toplink/queryframework/ReadAllQuery.execute(ReadAllQuery.java:451(Compiled Code))
    at oracle/toplink/publicinterface/Session.internalExecuteQuery(Session.java:2089(Compiled Code))
    at oracle/toplink/publicinterface/Session.executeQuery(Session.java:993(Compiled Code))
    at oracle/toplink/internal/indirection/QueryBasedValueHolder.instantiate(QueryBasedValueHolder.java:62(Compiled Code))
    at oracle/toplink/internal/indirection/QueryBasedValueHolder.instantiate(QueryBasedValueHolder.java:55(Compiled Code))
    at oracle/toplink/internal/indirection/DatabaseValueHolder.getValue(DatabaseValueHolder.java:61(Compiled Code))
    at oracle/toplink/indirection/IndirectList.buildDelegate(IndirectList.java:202(Compiled Code))
    at oracle/toplink/indirection/IndirectList.getDelegate(IndirectList.java:359(Compiled Code))
    at oracle/toplink/indirection/IndirectList.size(IndirectList.java:703(Compiled Code))

    All I can tell from the stack shown is that you are triggering indirection on a collection to an object that has an eagerly fetched 1:1 mapping to an object that is locked in the cache.  The thread is waiting on the lock owner to finish building the object before it can be returned.
    To tell what is going wrong, you will need to look at the complete thread dump to see what other threads might be building the object with its cache key locked.  For instance, a long running query might make it appear the system is hung when it is just a bottleneck in the application.  If you can, you should also try to call session.getIdentityMapAccessor().printIdentityMapLocks(); at the time of the problem as it will print off information on the locks such as the object involved and which threads hold them.  As for patches; 10.1.3 is an older stream but I am unaware of fixes that are not included in 10.1.3.5 that might resolve this - you should go through support to be sure.
    Best Regards,
    Chris

  • Hung Threads (Toplink 10.1.3)

    We are experiencing hung threads (waiting in TopLink code) in our app server, each one consuming a db connection so we eventually run out of db connections and die.
    We've got 9 threads waiting over 10 minutes on ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    And 1 thread waiting over 10 minutes on WriteLockManager.acquireLocksForClone(WriteLockManager.java:66)
    Thread dump Java HotSpot(TM) Client VM (1.4.2_05-b04 mixed mode):
    "Servlet.Engine.Transports : 12" daemon prio=5 tid=0x02976988 nid=0x5a waiting on condition [5e57c000..5e57fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 11" daemon prio=5 tid=0x028181a0 nid=0x59 in Object.wait() [5e67c000..5e67fc30]
    at java.lang.Object.wait(Native Method)
    - waiting on <0x9e5bc300> (a oracle.toplink.internal.helper.ConcurrencyManager)
    at java.lang.Object.wait(Object.java:429)
    at oracle.toplink.internal.helper.WriteLockManager.acquireLocksForClone(WriteLockManager.java:66)
    - locked <0x9e5bc300> (a oracle.toplink.internal.helper.ConcurrencyManager)
    at oracle.toplink.publicinterface.UnitOfWork.cloneAndRegisterObject(UnitOfWork.java:715)
    at oracle.toplink.internal.sessions.UnitOfWorkIdentityMapAccessor.getAndCloneCacheKeyFromParent(UnitOfWorkIdentityMapAccessor.java:154)
    at oracle.toplink.internal.sessions.UnitOfWorkIdentityMapAccessor.getFromIdentityMap(UnitOfWorkIdentityMapAccessor.java:99)
    at oracle.toplink.internal.sessions.IdentityMapAccessor.getFromIdentityMap(IdentityMapAccessor.java:257)
    at oracle.toplink.publicinterface.UnitOfWork.registerExistingObject(UnitOfWork.java:3465)
    - locked <0x9e97ac68> (a oracle.toplink.internal.sessions.UnitOfWorkImpl)
    at oracle.toplink.publicinterface.UnitOfWork.registerExistingObject(UnitOfWork.java:3427)
    - locked <0x9e97ac68> (a oracle.toplink.internal.sessions.UnitOfWorkImpl)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.registerIndividualResult(ObjectLevelReadQuery.java:1717)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:449)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:413)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:376)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:451)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.registerIndividualResult(ObjectLevelReadQuery.java:1701)
    at oracle.toplink.queryframework.ReadAllQuery.registerResultInUnitOfWork(ReadAllQuery.java:686)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:466)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:825)
    at oracle.toplink.publicinterface.UnitOfWork.internalExecuteQuery(UnitOfWork.java:2532)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingUowOld(BaseDAO.java:256)
    at com.saic.ct.sys.persistence.common.BaseDAO.findVOsForUpdate(BaseDAO.java:485)
    at com.saic.ct.sys.persistence.common.BaseDAO.findVOByPK(BaseDAO.java:291)
    at com.saic.ct.sys.persistence.schedule.ScheduleDAO.findWorkGroupByPK(ScheduleDAO.java:1248)
    "Servlet.Engine.Transports : 10" daemon prio=5 tid=0x029b0168 nid=0x58 waiting on condition [5e77c000..5e77fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 9" daemon prio=5 tid=0x02c880c8 nid=0x57 waiting on condition [5e87c000..5e87fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 8" daemon prio=5 tid=0x0257e900 nid=0x56 waiting on condition [5ec7c000..5ec7fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 5" daemon prio=5 tid=0x02d99b40 nid=0x53 waiting on condition [5e97c000..5e97fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 3" daemon prio=5 tid=0x02407580 nid=0x4f waiting on condition [5f17c000..5f17fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 2" daemon prio=5 tid=0x027d1bf0 nid=0x49 waiting on condition [5ef7c000..5ef7fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:443)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:413)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:376)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:451)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.registerIndividualResult(ObjectLevelReadQuery.java:1701)
    at oracle.toplink.queryframework.ReadAllQuery.registerResultInUnitOfWork(ReadAllQuery.java:686)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:466)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:825)
    at oracle.toplink.publicinterface.UnitOfWork.internalExecuteQuery(UnitOfWork.java:2532)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingUowOld(BaseDAO.java:256)
    at com.saic.ct.sys.persistence.common.BaseDAO.findVOsForUpdate(BaseDAO.java:485)
    at com.saic.ct.sys.persistence.common.BaseDAO.findVOByPK(BaseDAO.java:291)
    at com.saic.ct.sys.persistence.schedule.ScheduleDAO.findWorkGroupByPK(ScheduleDAO.java:1248)
    "Servlet.Engine.Transports : 1" daemon prio=5 tid=0x029773e0 nid=0x46 waiting on condition [664fc000..664ffc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    "Servlet.Engine.Transports : 0" daemon prio=5 tid=0x0241ce38 nid=0x44 waiting on condition [5fe7c000..5fe7fc30]
    at java.lang.Thread.sleep(Native Method)
    at oracle.toplink.internal.helper.ConcurrencyManager.releaseDeferredLock(ConcurrencyManager.java:389)
    at oracle.toplink.internal.identitymaps.CacheKey.releaseDeferredLock(CacheKey.java:311)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:619)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:378)
    at oracle.toplink.internal.descriptors.ObjectBuilder.buildObjectsInto(ObjectBuilder.java:667)
    at oracle.toplink.internal.queryframework.DatabaseQueryMechanism.buildObjectsFromRows(DatabaseQueryMechanism.java:141)
    at oracle.toplink.queryframework.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:468)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:800)
    at oracle.toplink.queryframework.DatabaseQuery.execute(DatabaseQuery.java:603)
    at oracle.toplink.queryframework.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:768)
    at oracle.toplink.queryframework.ReadAllQuery.execute(ReadAllQuery.java:436)
    at oracle.toplink.publicinterface.Session.internalExecuteQuery(Session.java:2062)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:981)
    at oracle.toplink.publicinterface.Session.executeQuery(Session.java:938)
    at com.saic.ct.sys.persistence.common.BaseDAO.executeQueryUsingSession(BaseDAO.java:1249)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1432)
    at com.saic.ct.sys.persistence.common.BaseDAO.findMany(BaseDAO.java:1407)
    at com.saic.ct.sys.persistence.common.CommonDAO.findClassifiers(CommonDAO.java:1609)
    ...

    Zev,
    The patch I applied was distributed in the file p5206620_101300_GENERIC.zip and contains the ObjectBuilder and Version classes. I'm pretty sure it is appplied correctly, I will verify again.
    This are the Toplink startup logs:
    [TopLink Info]: 2006.05.18 02:29:32.737--TopLink, version: Oracle TopLink - 10g Release 3 (10.1.3.0.0) (Build Patch for Bugs 5145690 and 5156075)
    [TopLink Info]: 2006.05.18 02:29:32.775--Server: WebLogic Server 8.1 SP4 Mon Nov 29 16:21:29 PST 2004 471647
    [TopLink Config]: 2006.05.18 02:29:32.807--connecting(DatabaseLogin(
         platform=>Oracle9Platform
         user name=> "epharm_owner"
         connector=>JNDIConnector datasource name=>com/pfizer/pgrd/epharm/ds1
    [TopLink Config]: 2006.05.18 02:29:35.390--Connected: jdbc:oracle:thin:@nlsunk770b:1521:clnval3
         User: EPHARM_OWNER
         Database: Oracle Version: Oracle9i Enterprise Edition Release 9.2.0.5.0 - Production
    With the Partitioning, OLAP and Oracle Data Mining options
    JServer Release 9.2.0.5.0 - Production
         Driver: Oracle JDBC driver Version: 9.2.0.4.0
    [TopLink Config]: 2006.05.18 02:29:35.392--connecting(DatabaseLogin(
         platform=>Oracle9Platform
         user name=> "epharm_owner"
         connector=>JNDIConnector datasource name=>com/pfizer/pgrd/epharm/ds1
    [TopLink Config]: 2006.05.18 02:29:35.405--Connected: jdbc:oracle:thin:@nlsunk770b:1521:clnval3
         User: EPHARM_OWNER
         Database: Oracle Version: Oracle9i Enterprise Edition Release 9.2.0.5.0 - Production
    With the Partitioning, OLAP and Oracle Data Mining options
    JServer Release 9.2.0.5.0 - Production
         Driver: Oracle JDBC driver Version: 9.2.0.4.0
    Thomas Buchan

  • Weblogic Hung threads

    What exactly in the WL log files indicate a hung thread? I'm looking to write a script to monitor WL logs and notify by email when a hung thread situation happens.
    TIA
    Vinayan Menon

    Stuck thread. Weblogic logs stuck thread in log file and you can specify stuck thread interval in admin console.
    http://e-docs.bea.com/wls/docs92/ConsoleHelp/taskhelp/tuning/TuningExecuteThreads.html
    jayesh@yagnasys

  • Kill/stop hung thread to generate an exception

    In production certain requests cause the thread executing them to hang. We are unable to reproduce that problem outside of production. One thought I had was to call stop on the thread executing that request which should cause a ThreadDeath which should be logged by our application so I could hopefully find the source of the problem. I made a way to call stop on a Thread and that seems to cause the Thread to die on my machine but not production. Any suggestions on that tactic or on a better tactic to use?

    See http://www.experts-exchange.com/Programming/Programming_Languages/Java/Q_20599112.html for the resolution.

  • Subscriber getting a hung on a ZfS Java thread

    In order to resolve high utilization on NW 6.5 sp7 with ZfS 7 sp1 subscriber I want to update the JVM.NLM with Sun's latest update for Java 1.4.2_13. In the previous post I've found the following version resolved the problem:
    JVM.NLM
    Loaded from [SYS:\JAVA\BIN\CLIENT\] on May 18, 2008 7:05:08 am
    (Address Space = OS)
    Java Hotspot 1.4.2_13 Interpreter
    Version 1.43 May 12, 2008
    (C) Copyright 2003-2006 Novell, Inc. All Rights Reserved
    However, I cannot find it anywhere on the Novell site so I need help.
    Sinisa

    shabulin,
    It appears that in the past few days you have not received a response to your
    posting. That concerns us, and has triggered this automated reply.
    Has your problem been resolved? If not, you might try one of the following options:
    - Visit http://support.novell.com and search the knowledgebase and/or check all
    the other self support options and support programs available.
    - You could also try posting your message again. Make sure it is posted in the
    correct newsgroup. (http://forums.novell.com)
    Be sure to read the forum FAQ about what to expect in the way of responses:
    http://forums.novell.com/faq.php
    If this is a reply to a duplicate posting, please ignore and accept our apologies
    and rest assured we will issue a stern reprimand to our posting bot.
    Good luck!
    Your Novell Product Support Forums Team
    http://support.novell.com/forums/

  • Problem with threads hanging

    We have a problem where our application stops responding after a few days of usage. Things will for fine for a day or two, and then pretty quickly threads will start getting hung up, usually in places where they are allocating memory
    We are running WebLogic 8.1 SP2 on Sun JDK 1.4.2_04 on Solaris 8 using the alternate threading model and the -server hotspot vm. We are running pretty much the same code that we had no problems with under WebLogic 6.1 SP4 and Sun JDK 1.3.1.
    A thread dump usually shows that some or all of our execute threads are in the state "waiting for monitor entry" even though they are not currently waiting on any java locks. Here is a sample thread from the thread dump (we have ~120 threads so I don't want to post the full dump).
    =============================================================================================
    "ExecuteThread: '8' for queue: 'itgCrmWarExecutionQueue'" daemon prio=5 tid=0x005941d0 nid=0x2c waiting for monitor entry [c807f000..c807fc28]
    at java.lang.String.substring(String.java:1446)
    at java.lang.String.substring(String.java:1411)
    at weblogic.servlet.internal.ServletRequestImpl.getRelativeUri(ServletRequestImpl.java:1872)
    at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3492)
    at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2585)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
    =============================================================================================
    String.java line 1446 for this jdk allocates a new String object, and all the other threads in this state also are creating new objects or arrays, etc.
    We've done a pstack on this process when it's in this state, and the threads that are in the "waiting for monitor entry" that look like they're allocating memory are all waiting on the same lwp_mutex_lock with some allocation method that's calling into the native TwoGenerationCollectorPolicy.mem_allocate_work (see pstack output below for the same thread as in the thread dump above)
    =============================================================================================
    ----------------- lwp# 44 / thread# 44 --------------------
    ff31f364 lwp_mutex_lock (e3d70)
    fee92384 __1cNObjectMonitorGenter26MpnGThread__v_ (5000, 525c, 5000, 50dc, 4800, 4af0) + 2d8
    fee324d4 __1cSObjectSynchronizerKfast_enter6FnGHandle_pnJBasicLock_pnGThread__v_ (c807f65c, c807f7d4, 5941d0, 0, 35d654, fee328ec) + 68
    fee32954 __1cQinstanceRefKlassZacquire_pending_list_lock6FpnJBasicLock__v_ (c807f7d4, ff170000, d4680000, 4491d4, fee1bc2c,
    0) + 78
    fee3167c __1cPVM_GC_OperationNdoit_prologue6M_i_ (c807f7bc, 4400, ff170000, 2d2b8, 4a6268, c807fa18) + 38
    fee2e0b0 __1cIVMThreadHexecute6FpnMVM_Operation__v_ (c807f7bc, 963a8, 0, 0, 1, 0) + 90
    fed2c2a4 __1cbCTwoGenerationCollectorPolicyRmem_allocate_work6MIii_pnIHeapWord__ (962c0, ff1c29ec, ff1c297c, ff131a26, 4800, 4998) + 160
    fed22940 __1cNinstanceKlassRallocate_instance6MpnGThread__pnPinstanceOopDesc__ (ee009020, 5941d0, 15ca581, 3647f0, 4a6268, c807f8c8) + 180
    fed34928 __1cLOptoRuntimeFnew_C6FpnMklassOopDesc_pnKJavaThread__v_ (ee009018, 5941d0, 0, 0, 0, 0) + 28
    fa435a58 ???????? (ee009018, e86de, 15ca4de, 50dc, 5941d0, c807f9c8)
    fb36f9a4 ???????? (0, d412ccd8, ee046c28, ff170000, 0, 0)
    fad8b278 ???????? (ee046c28, d6000c90, ee046530, 8, db8e8450, c807f9e8)
    fad62abc ???????? (d412ccd8, ee046530, d6000c90, ee3bfa38, 8, c807fa18)
    fa4b3c38 ???????? (c807fb9c, 0, f2134700, fa415e50, 8, c807faa8)
    fa40010c ???????? (c807fc28, c807fe90, a, ee9e1e20, 4, c807fb40)
    fed5d48c __1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_ (c807fe88, c807fcf0, c807fda8, 5941d0, 5941d0, c807fd00) + 27c
    fee4b784 __1cJJavaCallsMcall_virtual6FpnJJavaValue_nLKlassHandle_nMsymbolHandle_4pnRJavaCallArguments_pnGThread__v_ (ff170000, 594778, c807fd9c, c807fd98, c807fda8, 5941d0) + 164
    fee5e8dc __1cJJavaCallsMcall_virtual6FpnJJavaValue_nGHandle_nLKlassHandle_nMsymbolHandle_5pnGThread__v_ (c807fe88, c807fe84, c807fe7c, c807fe74, c807fe6c, 5941d0) + 6c
    fee6fc74 __1cMthread_entry6FpnKJavaThread_pnGThread__v_ (5941d0, 5941d0, 838588, 594778, 306d10, fee69254) + 128
    fee6927c __1cKJavaThreadDrun6M_v_ (5941d0, 2c, 40, 0, 40, 0) + 284
    fee6575c _start   (5941d0, fa1a1600, 0, 0, 0, 0) + 134
    ff3758c0 lwpstart (0, 0, 0, 0, 0, 0)
    =============================================================================================
    Also when it's having this problem, the "VM Thread" is always using a lot of processor time. We did a couple of pstacks today while it was having this problem, and this thread was stuck in the ONMethodSweeper.sweep for over 15 minutes when we finally killed the server.
    From the thread dump:
    "VM Thread" prio=5 tid=0x000e2d20 nid=0x2 runnable
    From the first pstack:
    =============================================================================================
    ----------------- lwp# 2 / thread# 2 --------------------
    fed40c04 __1cXvirtual_call_RelocationIparse_ic6FrpnICodeBlob_rpC5rppnHoopDesc_pi_nNRelocIterator__ (42a2f4, fa5fa46d, ffffffff, fc4ffcb8, 42a2f4, 42a324) + 124
    fed46318 __1cKCompiledIC2t5B6MpnKRelocation__v_ (42a2f0, fc4ffd24, fc4ffd4c, e802, 0, 6) + 38
    fed90c38 __1cHnmethodVcleanup_inline_caches6M_v_ (fa5f7f88, fa608940, 1, 0, fa400000, 6) + 1ac
    fede18b4 __1cONMethodSweeperFsweep6F_v_ (2cf38, 0, ffffffff, ff1cf1fc, ff1c66e8, fede1d44) + 1b0
    fede1e6c __1cUSafepointSynchronizeFbegin6F_v_ (2cf38, ff1ba138, 5000, 50dc, 5000, 525c) + 248
    feef1fd4 __1cIVMThreadEloop6M_v_ (4400, 4000, 4324, 4000, 42b0, 3800) + 3d4
    feef1ae4 __1cIVMThreadDrun6M_v_ (e2d20, 2, 40, 0, 40, 0) + 8c
    fee6575c _start   (e2d20, ff270200, 0, 0, 0, 0) + 134
    ff3758c0 lwpstart (0, 0, 0, 0, 0, 0)
    =============================================================================================
    Second pstack
    =============================================================================================
    ----------------- lwp# 2 / thread# 2 --------------------
    fed41180 __1cXvirtual_call_RelocationIparse_ic6FrpnICodeBlob_rpC5rppnHoopDesc_pi_nNRelocIterator__ (0, ff1b9664, ffffffff, fc4ffcb8, a6f2cc, fc4ffbd0) + 6a0
    fed46318 __1cKCompiledIC2t5B6MpnKRelocation__v_ (a6f2c8, fc4ffd24, fc4ffd4c, e802, 0, 6) + 38
    fed90c38 __1cHnmethodVcleanup_inline_caches6M_v_ (faded4c8, fadf2c80, 1, 0, fa400000, 6) + 1ac
    fede18b4 __1cONMethodSweeperFsweep6F_v_ (2cf38, 0, ffffffff, ff1cf1fc, ff1c66e8, fede1d44) + 1b0
    fede1e6c __1cUSafepointSynchronizeFbegin6F_v_ (2cf38, ff1ba138, 5000, 50dc, 5000, 525c) + 248
    feef1fd4 __1cIVMThreadEloop6M_v_ (4400, 4000, 4324, 4000, 42b0, 3800) + 3d4
    feef1ae4 __1cIVMThreadDrun6M_v_ (e2d20, 2, 40, 0, 40, 0) + 8c
    fee6575c _start   (e2d20, ff270200, 0, 0, 0, 0) + 134
    ff3758c0 lwpstart (0, 0, 0, 0, 0, 0)
    =============================================================================================
    Has anyone ever seen anything like this? I'm trying to figure out if this is caused by something we're doing, or something relating to our environment and jvm options. Any ideas?

    Thanks for the reply - I'm testing our app with the +UseConcMarkSweepGC now in our test environment to make sure it doesn't cause any problems there.  Unfortunately the only place we've had this problem is on the production server, so it's extra difficult debugging this. 
    We're using the following memory options:
    -ms512m -mx512m -XX:NewSize=128m -XX:PermSize=192m -XX:MaxNewSize=128m -XX:MaxPermSize=192m -XX:SurvivorRatio=8and the following debugging options, as we've also been seeing OutOfMemoryErrors ( see http://forum.java.sun.com/thread.jsp?forum=37&thread=522354&tstart=45&trange=15 )
    -verbosegc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGCBTW, which c++filt version and options are you using? Our Solaris boxes only seem to have the GNU version installed. I was trying to run that on some of the other stack traces and wasn't getting anywhere, and didn't know if because it was GNU version wouldn't work on something compiled with the Sun compiler.
    Thanks!
    --Andy                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

  • Image repaint preformance and threading

    Folks,
    I'm trying to make this sucker run faster.
    My question is, can anyone please guide me, especially with regards synchronising the Threads more efficiently... I'm thinking of using join and i]notify to make the "navigator" threads yield to the swing threads, to give it a chance to repaint before continuing... does this sound sane to you?
    Currently, without the thread.sleep it paints get the first "burst", and then nothing until the alrorithm has completed... exactly not what I wanted, because the whole point of this GUI is to watch the algorithm at work... sort of a "visual debugger"... I find that watching an algorithm play out helps me to "imagineer" ways of improving it... and in this case improvement means optimisation... it's all about getting from A-J faster than anyone else on the planet, especially those smarty-wishbone-legs C# programmers ;-)
    The code is too big to post (darn that 7500 char limit!) so I'll split it over several posts here, and I've also posted it as a single download to [MazeOfBoltonGUI2.java|http://groups.google.com/group/comp_lang_java_exchange/web/MazeOfBoltonGUI2.java] on my google group (comp lang java exchange).
    Cheers all. Keith.
    package forums.maze;
    import java.util.List;
    import java.util.ArrayList;
    import java.util.Map;
    import java.util.HashMap;
    import java.util.SortedMap;
    import java.util.TreeMap;
    import java.util.Stack;
    import java.util.Queue;
    import java.util.PriorityQueue;
    import java.util.Iterator;
    import java.util.Set;
    import java.util.concurrent.Callable;
    import java.util.concurrent.ExecutorService;
    import java.util.concurrent.Executors;
    import java.util.concurrent.Future;
    import java.util.concurrent.ExecutionException;
    import java.awt.Dimension;
    import java.awt.Color;
    import java.awt.Graphics;
    import java.awt.Graphics2D;
    import java.awt.Font;
    import java.awt.image.BufferedImage;
    import java.awt.event.ActionListener;
    import java.awt.event.ActionEvent;
    import javax.swing.JPanel;
    import javax.swing.JButton;
    import javax.swing.JFrame;
    import javax.swing.SwingUtilities;
    import javax.swing.SwingWorker;
    import java.io.PrintWriter;
    import java.io.BufferedWriter;
    import java.io.FileWriter;
    import java.io.BufferedReader;
    import java.io.FileReader;
    import java.io.IOException;
    * A Visual debugger,
    * for the [A* Alogorithm|http://en.wikipedia.org/wiki/A*_search_algorithm] navigator
    * of the [Maze Of Bolton|http://cplus.about.com/od/programmingchallenges/a/challenge12.htm]
    * as implemented by [Prometheuz|http://forums.sun.com/profile.jspa?userID=550123]
    * with GUI by [Kajbj|http://forums.sun.com/profile.jspa?userID=91610]
    * hacked together by [Keith Corlett|http://forums.sun.com/profile.jspa?userID=640846]
    * and posted on [Sun's Java Forum|http://forums.sun.com/thread.jspa?threadID=5319334]
    * and posted on [Google news group|http://groups.google.com.au/group/comp_lang_java_exchange/]
    public class MazeOfBoltonGUI2
      static final char[][] matrix = readMatrix("map.txt");
      public static void main(String[] args) {
        SwingUtilities.invokeLater(
          new Runnable() {
            public void run() {
              try {
                MazeNavigator navigator = new MazeNavigator(matrix);
                JFrame frame = new JFrame("MazeOfBoltonGUI2");
                frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                frame.setContentPane(new MainPanel(navigator));
                frame.pack();
                frame.setVisible(true);
              } catch (Exception e) {
                e.printStackTrace();
       * Reads the file into a char matrix[rows,cols] ie: an array of char arrays.
       * @param String filename - the name of the file to read
       * @return a fixed length array of strings containing file contents.
      private static char[][] readMatrix(String filename) {
        try {
          BufferedReader input = null;
          try {
            input = new BufferedReader(new FileReader(filename));
            char[][] matrix = null;
            List<String> lines = new ArrayList<String>();
            String line = null;
            while ( (line = input.readLine()) != null ) {
              lines.add(line);
            int rows = lines.size();
            matrix = new char[rows][];
            for (int i=0; i<rows; i++) {
              matrix[i] = lines.get(i).toCharArray();
            System.err.println("DEBUG: rows="+rows+", cols="+matrix[0].length);
            return matrix;
          } finally {
            if(input!=null)input.close();
        } catch (IOException e) {
          e.printStackTrace();
          throw new IllegalStateException("Failed to readMatrix!", e);
    class MainPanel extends JPanel
      private static final long serialVersionUID = 1L;
      // button panel
      private final JButton goButton;
      // maze panel
      private final MazeNavigator navigator;
      private final Monitor<Path> monitor;
      private BufferedImage background;
      private BufferedImage image;
      private List<Path>currentPaths;
      public MainPanel(MazeNavigator navigator) {
        this.navigator = navigator;
        this.monitor = new SwingMonitor();
        this.goButton = new JButton("Go");
        goButton.addActionListener(
          new ActionListener() {
            public void actionPerformed(ActionEvent event) {
              final String caption = goButton.getText();
              goButton.setVisible(false);
              monitor.execute();
        add(goButton);
        setPreferredSize(new Dimension(navigator.maze.cols*3, navigator.maze.rows*3)); //w,h
      public void paintComponent(Graphics g) {
        super.paintComponent(g);
        if (image==null) {
          image = (BufferedImage)createImage(navigator.maze.cols, navigator.maze.rows);
          mazeColors = createMazeColors(navigator.maze);
        this.draw(image.createGraphics());
        ((Graphics2D)g).drawImage(image, 0, 0, super.getWidth(), super.getHeight(), null);
      private static Color[][] mazeColors;
      private static Color[][] createMazeColors(Maze maze) {
        Color[][] colors = new Color[maze.rows][maze.cols];
        for (int r=0; r<maze.rows; r++) {
          for (int c=0; c<maze.cols; c++) {
            colors[r][c] = getColor(maze.matrix[r][c].ch);
        return colors;
      }*... PTO ...*

    I'm persuaded that the main issue (no intermediate results drawn) is the improper use of SwingWorker.
    When you've got over it, you may want to consider other smaller-effect optimizations:
    Reconsider usage of an offscreen image*
    public void paintComponent(Graphics g) {
        super.paintComponent(g);
        if (image==null) {
          image = (BufferedImage)createImage(navigator.maze.cols, navigator.maze.rows);
          mazeColors = createMazeColors(navigator.maze);
        this.draw(image.createGraphics());
        ((Graphics2D)g).drawImage(image, 0, 0, super.getWidth(), super.getHeight(), null);
      }At first I didn't get why you wanted to draw an offscreen image, then paint it to the screen, all that in the EDT.
    After reading the draw() method more closely, I guess you want to ease the coding of the scaling: you draw an image where one cell = one pixel, then paint the image, scaled to the panel's display size.
    In terms of performance, I don't know how it stands:
    On one hand, the image creation if lighter (1 pixel per cell). And you have a point that the built-in scaling offered by Graphics2D.drawImage(image, size) may be efficient. I can't comment on that, I hope the granphics and hardware acceleration folks will pop in the thread.
    On the other hand, if the built-in scaling had poor performance, it may be good to try what "manual" scaling would bring you. That means, in a simplified version, skip the offscreen image creation, and draw directly on the paintComponent()'s Graphics2D argument. the drawing of a cell at coordinates c,r, for example, would look like:
    g.fillRect(c*CELL_WIDTH, r*CELL_HEIGHT, WIDTH, HEIGHT);Performance apart, the scaling as you do it currently has functional drawbacks, if you want pathes as 1-pixel width lines over large cells:
    - if the maze is smaller (in rows, columns) than the panel displaying it (in pixels), the cells will be scaled but the pathes too: so your 1-pixel lines appear as large as the cells. May or may not be a desired effect.
    - if the maze is larger than the display panel, the cells are shrinked, fine, but the so are the path lines, to a point where they may be invisible (probably depending on color blending, I'm a n00b at graphics operations).
    But maybe I misunderstood the need, and maybe the intended drawing of a path is actually the drawing of the rectangles of all its traversed cells, in special path colors?
    Reconsider intermediate allocations*
    Each paintComponent() call results in the allocation of a 2D-array of Color objects (method createMazeColors(Maze)).
    I don't see what the mazeColors array brings you. I assume you wanted to decouple the determination of colors (depending on cell state) and the rendering of the colors: what does it bring: no performance advantage (on the contrary, it adds a 2D array allocation, and 2xN^2 2D-array access), and does not improve the code readability either (subjective rant, sorry).
    Why don't you pass the Maze as an argument to the draw() method, and call the getColor(cell.ch) from there?
    Side note, maybe a bit subjective: performance apart, the design of the usage of this mazeColor array is a no-go!
    An instance method alters a static variable reference,which is used subsequently in another instance method, with no synchronization. The current code does that in a single thread (+paintxxx()+ is only called in the EDT), which keeps that safe (I'd dare to say: by luck), but considerations exposed below may have you refactor the design to introduce other threads, and may exhibit the thread-unsafety of this design.
    Consider drawing the image in a background thread.*
    Indeed the technique of drawing to an offscreen image is quite common, but it is often done to improve responsiveness (not raw performance) of Swing applications. Here is a resource about this (what the author calls the passive approach), although it doesn't use a background thread.
    The idea is that if a paintCompobnent() methods involves lots of computation (arithmetics of traversing a 2D model, scaling things, etc.), this takes CPU times in the EDT, and all subsequent events (such as, a MouseEvent, but also other painting events) keep pending on the event queue, which is consumed by the single event-dispatch thread. The result is that the UI appear unresponsive, and painting of other areas may seem hung.
    The idea is to move the computation to a background thread, which posts rendering to the EDT when the Image is ready to be displayed.
    Of course this doesn't gain any CPU time. This only ensures the EDT uses a minimal part of this CPU (only render and image and process events), instead of performing the whole computation.
    In your case you already have a background thread, and indeed an appropriate choice, a SwingWorker. The application of this technique would consist in calling the draw() method in the worker thread (in the update(Path) method), and invoke super.publish() only after the image has been updated. Note that the process(List<Path>) could then ignore its argument (you may reconsider the choice of type parameter of the worker), and simply get the latest version of the image attribute).
    Of course in this technique, the offscreen image filling is called synchronously from the Navigator, so this halts the algorithm part itself, for the duration of the image generation. You may refine the technique by spawning a dedicated thread for the image generation - with subtle guard code to handle occasions when the algorithm goes faster than the image generation, and posts a new update(Path) while the image generation for the previous path has not completed yet...
    Recuce the number of things to redraw*
    Two parts:
    first, depending on the number of cells and pathes, there may be (yet another) optimization, to not redraw the whole offscreen image, but only the cells/path that have changed in the last update(). In particular, if a path is not a line but a list of cells, then it's quite easy, reusing the current offscreen image, to only fillRect(...) the appropriate cells.
    Second, if a path is not rendered as a thin line over larger cells, but as cells themselves rendered in special path colors, you may paint cells and path in one go: instead of drawing, first the cells, then the path, draw only the cells, electing the color using a decision method such as:
    private Color getColor(Cell) {
        if (cell.getPathState()!=NOT_IN_ANY_PATH) {
            return getColor(cell.getPathState());
        else {
            return getColor(cell.ch);
    }Of course this forces you to modify your data model, and update the new pathState as part of the algorithm (or better isolated, in the update(Path) method, before invoking the drawing machinery). Maybe that was the intention of the mazeColors array?
    I haven't studied your other posts on the logic of the MazeOfBolton algorithm, so I don't know if it's acceptable for you that a cell appear to have only one path state, as opposed to one for each of the pathes that traverse it. This last trick may then seem incorrect, but please consider it as only a graphical information, and indeed your current image drawing draws only ONE path for a given cell (the last path in currentPaths that traverses this cell ).

  • Thread are gettin stuck causing the WebLogic server to hang

    We are developing an application for an US client which inclueds the following technology.
    - WebLogic 9.2 JMS (Foreign Server, Queue, Connection Factor, persistent Store)
    - Oracle 10g & Oracle 9i
    - J2EE (MDB)
    Different application uses different databases.
    After the server run for a few day it is getting a stuck thread and reported the same as a Hogging Thread in server -> monitoring -> thread. At this point the server hangs and stopped working with the status in 'warning' state (in the same page itself).
    The problem remains until the managed server is restarted.
    We have taken thread dump of different times and one of the error that we have got is attached below:
    "[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.rjvm.ResponseImpl@1279315 WAITING
         java.lang.Object.wait(Native Method)
         weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:82)
         weblogic.rjvm.ResponseImpl.getTxContext(ResponseImpl.java:109)
         weblogic.messaging.dispatcher.DispatcherProxy.unmarshalResponse(DispatcherProxy.java:238)
         weblogic.messaging.dispatcher.DispatcherProxy.dispatchSyncTranFuture(DispatcherProxy.java:131)
         weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchSyncTran(DispatcherWrapperState.java:241)
         weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchSyncNoTran(DispatcherWrapperState.java:266)
         weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncNoTran(DispatcherAdapter.java:59)
         weblogic.jms.client.JMSProducer.toFEProducer(JMSProducer.java:1061)
         weblogic.jms.client.JMSProducer.deliveryInternal(JMSProducer.java:670)
         weblogic.jms.client.JMSProducer.sendInternal(JMSProducer.java:455)
         weblogic.jms.client.JMSProducer.send(JMSProducer.java:337)
         weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:966)
         com.ctb.dex.messagehost.InputMessageListener.onMessage(Unknown Source)
         weblogic.ejb.container.internal.MDListener.execute(MDListener.java:429)
         weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:335)
         weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:291)
         weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4072)
         weblogic.jms.client.JMSSession.execute(JMSSession.java:3962)
         weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:4490)
         weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)
         weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
         weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
    Any body has any idea why this is happening? Please let me know in case you need to know more about the problem. Mail me at [email protected].

    Tom, we are experiencing the similar stuck thread max time out errors on our weblogic server(weblogic 8.1 sp5). Below is the thread dump:
    Blocked lock chains
    ===================
    Chain 4:
    "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=101 idx=0xca tid=19298 waiting for java/lang/String@0x9c1fb10 held by:
    "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=102 idx=0xcc tid=19299 in chain 3
    Open lock chains
    ================
    Chain 1:
    "ExecuteThread: '33' for queue: 'weblogic.kernel.Default'" id=62 idx=0x80 tid=19223 waiting for java/lang/String@0xbd00b10 held by:
    "ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" id=32 idx=0x44 tid=19192 (waiting on notification)
    Chain 2:
    "ExecuteThread: '45' for queue: 'weblogic.kernel.Default'" id=74 idx=0x98 tid=19235 waiting for java/lang/String@0x1d1083a8 held by:
    "ExecuteThread: '11' for queue: 'weblogic.kernel.Default'" id=40 idx=0x54 tid=19200 (waiting on notification)
    Chain 3:
    "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=100 idx=0xc8 tid=19297 waiting for java/lang/String@0x9c1fb10 held by:
    "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=102 idx=0xcc tid=19299 (active)
    Can you help us in resolving the issue? This causes the weblogic server to hung and eventually non responsive.
    Below are our java version details:
    java version "1.4.2_13"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_13-b06)
    Java HotSpot(TM) Client VM (build 1.4.2_13-b06, mixed mode)
    Can you let me know how to proceed and what can be done to avoid this issue?
    Thanks a bunch Tom.
    Regards
    RV

Maybe you are looking for

  • Job Cancelled in SM37

    Hi all, Job is cancelled in sm37 daily from 7th december onwards. We have 4 process chains in our project and all the chains are ok when i am checking in RSPC,and the data is being loaded into the respective cubes and ods. The error message is like J

  • Can't login to my yahoo account

    When trying to get mail from 1 yahoo account I get a login failed message. After several retries, a error message pops up: Server [email protected] has disconnected. The server may have gone down or there may be a network problem. Reentering the pass

  • ShowDocument url permission denied in 1.6.0_24

    Have a JWS application, which was bringing up the default browser. Sometime the url was http & others was file ( sort of a print preview function). Starting in 1.6.0_24, I am getting a SecurityException "ShowDocument url permission denied" for both.

  • Unable to create Business logic Callable Object

    Hi All, We are working on CE 7.1 SDN trial version. while creating Business Logic Callble object, it is giving an error: Error while loading configuration dialog: Failed to create delegate for component com.sap.caf.eu.gp.ui.co.CExpConfig. (Hint: Is t

  • Can animated graphics be used?

    I love iWeb, but I don't know if "animated graphics" can be used.  If this is possible, please provide information. RJohnson