Jrun crash

hi all
i manage a server with cf8.
my customer decided to move server to a cloud.
from that moment, i lost my sleep...
one or twice a day jRun gets the error
"JRun closed connection".
this is a little report of situation... any help is very appreciated... :-)
== SERVER SPECS ==
virtual machine on wmware cloud
with Linux CentOS, 32 bit
4 core CPU
3 GB Ram
1,5 GB Ram dedicated for JRun
JVM use 20% of allocated memory.
80.000 pages a day
== JRUN ==
Maximum number of simultaneous Template requests: 25
Maximum JVM Heap Size (MB): 1524
i activate jRun log. this is a optimal situation:
THREADS: Listen=22 Idle=60 Running=1 Delayed/Waiting=0
  Time Delayed=0 Dropped=0 Completed=62 Time To Complete=11253 Total=83
THREADS: Listen=23 Idle=86 Running=1 Delayed/Waiting=0
  Time Delayed=0 Dropped=0 Completed=134 Time To Complete=30495 Total=110
THREADS: Listen=22 Idle=77 Running=1 Delayed/Waiting=0
  Time Delayed=0 Dropped=0 Completed=106 Time To Complete=19309 Total=100
before crash, the situation is similar... :-(
== DB ==
db (postgres 9.1) is in local lan on other virtual server,
no particulary data to provide.
== DURING THE CRASH... ==
this is the situation just before crash cf crash (from fusionreactor)
http://i.imgur.com/VwXts.png
http://i.imgur.com/ezzuV.png
there are no reasons for this situation.
no overload on cpu, no overload on ram.
in screenshot, over the first 3/4 page in list, other pages
are normally generated in 1/4 of second.
from this reports a see a strage "time question"...
== "TIME QUESTION"... ==
on this server happen at least two strage events...
- 1 -
in fusionreactor report, such as in seefusion report, such as in cf log,
i see a very strange long long long request.
this request taken 4,398 secs, 1h and 22m to complete. (ALWAYS 1h 22m!!!).
this happens randomly, in normal situation this page are generated
in few millisecons. moreover, the end-time of this tasks is IN THE FUTURE!
time of machine is updated regularly via ntp.
can this happen because of the virtual machine?
12/19 12:20:55 Warning [jrpp-871] - Thread: jrpp-871, template:
/jackets/audio.cfm, completed in 4,398 secs
12/19 12:23:00 Warning [jrpp-1068] - Thread: jrpp-1068, template:
/articoli/index.cfm, completed in 4,398 secs
12/19 13:00:05 Warning [jrpp-1126] - Thread: jrpp-1126, template:
/articoli/index.cfm, completed in 4,398 secs
12/19 13:23:10 Warning [jrpp-1292] - Thread: jrpp-1292, template:
/home/index.cfm, completed in 4,398 secs
- 2 -
second strange event, is the frequently timeout of cfloop, cfoutout,
cffile tags... but is not "real" timeout.
when i meet up with the error, there is no waiting time that should
generate the timeout error.
evidently cf with this error, want to say me such other,
but i don't know what... :-/
you have some idea?
i add "var" scope to cfc in my code...
== APACHE PROCESS ==
i activeted a simple log for apache connections.
sometimes, just before the crash the apache load
is bit overload..
H: apache process - C: connections on 80 port.
-- optimal situation --
H:79 C:195
H:108 C:255
H:85 C:264
H:81 C:163
H:77 C:198
H:88 C:402
H:82 C:243
H:82 C:196
-- just before crash --
H:68 C:63
H:81 C:81
H:143 C:130
H:156 C:152
H:163 C:169
H:163 C:163
-- just before crash --
H:178 C:186
H:178 C:170
H:178 C:187
H:178 C:176
H:178 C:188
H:178 C:215
== CFSERVER.LOG ==
from cfserver.log i get
12/18 18:48:36 error Error in getRealPathFromConn
12/18 21:51:00 error Error in getRealPathFromConn
12/18 21:51:00 error Error in getRealPathFromConn
12/18 21:51:03 error Error in getRealPathFromConn
12/18 21:51:03 error Error in getRealPathFromConn
but i do not think this is an important matter...
===
have you got some ideas for this strage case?
links, documents... any idea is welcome!
many thanks
Roberto

for future...
the "strange" timeouts are caused by a linux kernel bug...
http://archives.postgresql.org/pgsql-general/2011-07/msg00300.php

Similar Messages

  • How To Fix Jrun Crashes in ColdFusion 8

    I have ColdFusion 8 running on Windows Server 2008 with IIS 7.  Lately JRun has been crashing every few minutes.  I don't know what the cause could be.  Can anyone give advice on how to fix it?
    The jron error I am getting in the server log mentions MSVCR100.dll and states:

    From "OUT log" details sounds like there is a problem with the CF webserver connector.
    What do you do to overcome the outage eg restart server, restart CF or restart IIS?
    It would be nice to narrow down problem a bit more. Maybe there will be some interesting detail in ColdFusion8\runtime\lib\wsconfig\1\LogFiles\ jrun_iis6_wildcardYYMMDD.log file?
    I want to suggest delete the old connector and rebuild it using WSCONFIG.EXE tool from ColdFusion8\runtime\bin however I expect when you unintalled CF8 you likely removed the IIS connector information then the subsequent install added the connector setup information again. Maybe you can comment on how you attached CF8 to IIS?
    Sorry more questions than answers, Carl.

  • JRun Crashing alot

    are there any other ways to get Jrun from crashing on a
    frequent basis. I have increased the number of concurrent users
    sightly already, and it still seems to crash.

    If JRun is crashing a lot then it is probably a memory leak
    in your code. So you can load test your application and use a
    memory monitoring tool like JConsole and try and find where your
    leak is coming from and solve it. You can also tune your JVM
    settings in the jvm.config which could help you.

  • CF8 (64bit) on Win 2008 Ent - JRun crashes (503 err) when files copied

    We have recently upgraded some of these servers to Win 2008 with CF8 64 bit.  On these machines (especially when pushing CFCs over), the machine shows 503 errors until JRUN service is restarted.
    We are using a custom script that uses RoboCopy to push files to a number of servers in a web farm.  The script pushes code to both CF7 (Windows 2003) and CF8 (Windows 2003 and Windows 2008) servers.  There are no issues on any of the Windows 2003 servers.  After most deployments, the Windows 2008 servers start throwing 503 errors...
    There are no errors listed in the CF or JRUN logs -- or in the Windows Application, System, or Security logs.  The CF logs just 'stop'...
    The server is responsive from the console.  Jrun is not using an inordinate amount of memory (~300mb) and the processors are fine (~2%).
    The JVM being used is Sun (64bit) 1.6.0_13 (we also have one on 1.6.0_17 as a test); they all present the same issue. We are running CF 8.0.1 (no hotfixes installed).
    Has anyone had a similar problem?
    Any ideas would be appreciated...
    Thanks,
    Blake

    Some clarification...
    We have tried a number of things - but it looks like this happens most consistently when pushing updates of CFC files that are in heavy use.  A short bit after the updated file is copied to the server - connections start to fail and 503 errors are returned by the CF server.
    CF ADMIN still runs fine.
    I think there has to be something locking these files - something specific to Windows 2008; because we use the same deployment scripts on CF8 servers running Windows 2003 and there are no issues...
    Any ideas?
    Blake

  • Sleeping 'watcher' servlet crashes JRUN

    I have set up a servlet that runs a query once every 2 hours. I have made a while(true) loop that loops forever then executes the query then tells my thread that i created to sleep for 7200000 milliseconds. I tried using
    Thread.currentThread.sleep(7200000) ;
    and then also tried creating a new thread
    Thread myThread = new Thread();
    myThread.sleep(7200000);
    (BTW: This is all in a method i call from the init method)
    Either way if I invoke the servlet from a jsp it works fine but the jsp is always stuck working on getting a response. If I go into JRUN and set up the servlet to be preloaded when JRUN is started, it executes once then JRUN locks up. And when I try to view the site I see the "Could not connect to JRun Server" error. From this point on I have to complete hose the entire jrun instance and create a new jrun server instance. The JRUN instance cant be restarted, even rebooting doesnt work.
    Is the servlet sleeping the jrun init thread itself? Im sorry Im new to threads.
    Thanks for any help anyone can give me!

    Is the servlet sleeping the jrun init thread itself?
    Im sorry Im new to threads.Yes. This is exactly what is happening.
    You need to spawn another thread, that does nothing but runs through your while ( true ) ... loop, sleeping for the required two hours every now and then.
    Something like this:
    class MySleepingThread implements Runnable
       public void run()
          while ( true )
              doTheStuffINeedToDoEveryTwoHours();
              try
                 Thread.sleep( 1000 * 60 * 60 * 2 }
              catch ( InterruptedException e )
                 break;
    And in the servlet class you normally had this, do this:
    Thread t = new Thread( new MySleepingThread() );
    // This starts the thread and eventually calls the run() method, starting your loop on a separate thread, so this one will continue.
    t.start();.P.

  • Help me stabilize this jRun configuration (CF9/Win2k3/IIS6)

    Hi everyone,
    We've been struggling to keep our multi-server configuration stable for quite some time now.  At the end of last month we were running under CF 7.0.2 on a two servers setup (one instance each).  At that point we managed to get our uptime to around 1 week per instance before they would restart by themselves.  Since the beginning of the month we upgraded to CF 9 and we're back to square one with multi-restart a day.
    Our current configuration is 2 Win2k3 servers, running a cluster of 4 instances, 2 instances per server. At this point we are pretty certain this is due to improper JVM settings.
    We've been toying with them and while some are more stable than others we never quite got it right.
    From the default:
    java.args=-server -Xmx512m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/
    To currently:
    java.args=-server -Xmx896m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=512m -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/ -verbose:gc -Xloggc:c:/Jrun4/logs/gc/gcInstance1b.log
    We have determined that we do need more than the default 512MB simply by monitoring with FusionReactor, on average our amount of memory consumed is hovering in the mid 300MB and can go up to low 700MB under heavy load.
    Most of the crash will be logged in jrun4/bin/hs_err_pid*.log always an "Out of swap space"
    The latest crash from yesterday generated the following log:
    From it, I gather that its the PSPermGen that is full (most logs will show the same before a crash), which is why we increased MaxPermSize but the total still show as 107520K!??! I've attached the GC log after the HS_ERR.  No one here is a jRun expert, just developpers tasked with fixing this as best as we possibly can so any help would be greatly appreciated!!
    # A fatal error has been detected by the Java Runtime Environment:
    # java.lang.OutOfMemoryError: requested 2264248 bytes for Chunk::new. Out of swap space?
    #  Internal Error (allocation.cpp:215), pid=15224, tid=4960
    #  Error: Chunk::new
    # JRE version: 6.0_14-b08
    # Java VM: Java HotSpot(TM) Server VM (14.0-b16 mixed mode windows-x86 )
    # If you would like to submit a bug report, please visit:
    #   http://java.sun.com/webapps/bugreport/crash.jsp
    ---------------  T H R E A D  ---------------
    Current thread (0x5c69c800):  JavaThread "CompilerThread1" daemon [_thread_in_native, id=4960, stack(0x5ce70000,0x5cf70000)]
    Stack: [0x5ce70000,0x5cf70000],  sp=0x5cf6e90c,  free space=1018k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    V  [jvm.dll+0x306157]
    V  [jvm.dll+0xf7d2c]
    V  [jvm.dll+0x3915c]
    V  [jvm.dll+0x393f3]
    V  [jvm.dll+0x16a470]
    V  [jvm.dll+0xa9552]
    V  [jvm.dll+0xd7843]
    V  [jvm.dll+0xd7f6c]
    V  [jvm.dll+0x9e082]
    V  [jvm.dll+0xda0e0]
    V  [jvm.dll+0xdac6f]
    V  [jvm.dll+0x2e63f4]
    V  [jvm.dll+0x267dac]
    C  [MSVCR71.dll+0x9565]
    C  [kernel32.dll+0x2482f]
    Current CompileTask:
    C2:3749  !   cfdspInit2ecfm988339296.runPage()Ljava/lang/Object; (4792 bytes)
    ---------------  P R O C E S S  ---------------
    Java Threads: ( => current thread )
      0x6f812800 JavaThread "task" daemon [_thread_blocked, id=12580, stack(0x6acb0000,0x6adb0000)]
      0x79981800 JavaThread "task" daemon [_thread_blocked, id=14328, stack(0x6a940000,0x6aa40000)]
      0x7882a800 JavaThread "task" daemon [_thread_blocked, id=6132, stack(0x6a840000,0x6a940000)]
      0x6eb97800 JavaThread "task" daemon [_thread_blocked, id=1756, stack(0x6a740000,0x6a840000)]
      0x6f221800 JavaThread "task" daemon [_thread_blocked, id=14024, stack(0x6a640000,0x6a740000)]
      0x6f668800 JavaThread "task" daemon [_thread_blocked, id=15956, stack(0x69740000,0x69840000)]
      0x62887400 JavaThread "RMI TCP Connection(10140)-192.168.40.35" daemon [_thread_in_native, id=11024, stack(0x628d0000,0x629d0000)]
      0x61ec3800 JavaThread "RMI TCP Connection(idle)" daemon [_thread_blocked, id=5848, stack(0x60b80000,0x60c80000)]
      0x788d2c00 JavaThread "RMI TCP Connection(idle)" daemon [_thread_blocked, id=13348, stack(0x60460000,0x60560000)]
      0x7955e400 JavaThread "RMI TCP Connection(10136)-192.168.40.35" daemon [_thread_in_native, id=10912, stack(0x639d0000,0x63ad0000)]
      0x621c5400 JavaThread "RMI TCP Connection(idle)" daemon [_thread_blocked, id=3308, stack(0x64ed0000,0x64fd0000)]
      0x619fb400 JavaThread "RMI TCP Connection(10132)-192.168.40.34" daemon [_thread_in_native, id=7784, stack(0x642d0000,0x643d0000)]
      0x62719c00 JavaThread "jrpp-821" [_thread_in_native, id=5124, stack(0x60360000,0x60460000)]
      0x78751400 JavaThread "jrpp-820" [_thread_in_native, id=16240, stack(0x60180000,0x60280000)]
      0x7fa71400 JavaThread "jrpp-819" [_thread_in_native, id=15228, stack(0x5fb80000,0x5fc80000)]
      0x78ccb400 JavaThread "RMI TCP Connection(idle)" daemon [_thread_blocked, id=6224, stack(0x64ad0000,0x64bd0000)]
      0x6edc8c00 JavaThread "RMI TCP Connection(10123)-192.168.40.34" daemon [_thread_in_native, id=4592, stack(0x632d0000,0x633d0000)]
      0x6e37a400 JavaThread "jrpp-818" [_thread_in_native, id=2712, stack(0x617d0000,0x618d0000)]
      0x62816c00 JavaThread "jrpp-817" [_thread_in_native, id=11972, stack(0x60980000,0x60a80000)]
      0x6fab2400 JavaThread "RMI TCP Connection(10074)-192.168.40.34" daemon [_thread_in_native, id=8124, stack(0x64dd0000,0x64ed0000)]
      0x78769400 JavaThread "RMI TCP Connection(10050)-192.168.40.34" daemon [_thread_in_native, id=8720, stack(0x60a80000,0x60b80000)]
      0x5c82ac00 JavaThread "RMI TCP Connection(10030)-192.168.40.34" daemon [_thread_in_native, id=12620, stack(0x62cd0000,0x62dd0000)]
      0x61ee6800 JavaThread "task" daemon [_thread_in_native, id=5936, stack(0x60d80000,0x60e80000)]
      0x6e239c00 JavaThread "scheduler-28" [_thread_blocked, id=13084, stack(0x69a40000,0x69b40000)]
      0x61cc2400 JavaThread "RMI TCP Connection(10001)-192.168.40.34" daemon [_thread_in_native, id=3300, stack(0x5fe80000,0x5ff80000)]
      0x5da0a400 JavaThread "scheduler-27" [_thread_blocked, id=5764, stack(0x62fd0000,0x630d0000)]
      0x5c82a400 JavaThread "scheduler-26" [_thread_blocked, id=15592, stack(0x61290000,0x61390000)]
      0x7963bc00 JavaThread "scheduler-25" [_thread_blocked, id=1564, stack(0x60880000,0x60980000)]
      0x61f3a800 JavaThread "RMI RenewClean-[192.168.40.34:4835]" daemon [_thread_in_native, id=12824, stack(0x62bd0000,0x62cd0000)]
      0x6df88800 JavaThread "web-3" [_thread_in_native, id=10008, stack(0x667d0000,0x668d0000)]
      0x6e27dc00 JavaThread "scheduler-3" [_thread_blocked, id=7512, stack(0x63ad0000,0x63bd0000)]
      0x5dd3b400 JavaThread "scheduler-24" [_thread_blocked, id=7552, stack(0x629d0000,0x62ad0000)]
      0x6ebcd800 JavaThread "AWT-Windows" daemon [_thread_in_native, id=8736, stack(0x60080000,0x60180000)]
      0x6f595400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-8617)" daemon [_thread_blocked, id=7408, stack(0x5fd80000,0x5fe80000)]
      0x5d9d3800 JavaThread "scheduler-2" [_thread_blocked, id=1884, stack(0x60e80000,0x60f80000)]
      0x619d4400 JavaThread "scheduler-23" [_thread_blocked, id=7984, stack(0x616d0000,0x617d0000)]
      0x62748800 JavaThread "RMI RenewClean-[192.168.40.34:4547]" daemon [_thread_blocked, id=12652, stack(0x60c80000,0x60d80000)]
      0x61a28400 JavaThread "MultiThreadedHttpConnectionManager cleanup" daemon [_thread_blocked, id=7212, stack(0x5fc80000,0x5fd80000)]
      0x00717400 JavaThread "DestroyJavaVM" [_thread_blocked, id=12764, stack(0x00030000,0x00130000)]
      0x6261dc00 JavaThread "scheduler-1" [_thread_blocked, id=9928, stack(0x64800000,0x64900000)]
      0x5d508400 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=11168, stack(0x65fd0000,0x660d0000)]
      0x62186800 JavaThread "worker #4" daemon [_thread_blocked, id=15864, stack(0x66cd0000,0x66dd0000)]
      0x6e827c00 JavaThread "worker #3" daemon [_thread_blocked, id=15700, stack(0x663d0000,0x664d0000)]
      0x625a4c00 JavaThread "worker #2" daemon [_thread_blocked, id=2940, stack(0x662d0000,0x663d0000)]
      0x622d5000 JavaThread "worker #1" daemon [_thread_blocked, id=12548, stack(0x661d0000,0x662d0000)]
      0x5d7ffc00 JavaThread "worker #0" daemon [_thread_blocked, id=9384, stack(0x660d0000,0x661d0000)]
      0x5d406800 JavaThread "Thread-49" [_thread_blocked, id=9248, stack(0x65c20000,0x65d20000)]
      0x622a8000 JavaThread "obj-skimmer" daemon [_thread_blocked, id=3680, stack(0x640d0000,0x641d0000)]
      0x620d7800 JavaThread "obj-skimmer" daemon [_thread_blocked, id=12312, stack(0x63fd0000,0x640d0000)]
      0x62351800 JavaThread "obj-skimmer" daemon [_thread_blocked, id=11708, stack(0x63dd0000,0x63ed0000)]
      0x5dedd800 JavaThread "obj-skimmer" daemon [_thread_blocked, id=8800, stack(0x5f9d0000,0x5fad0000)]
      0x5dec6400 JavaThread "obj-skimmer" daemon [_thread_blocked, id=14688, stack(0x5f8d0000,0x5f9d0000)]
      0x5d519c00 JavaThread "obj-skimmer" daemon [_thread_blocked, id=4868, stack(0x5f7d0000,0x5f8d0000)]
      0x5d7cec00 JavaThread "obj-skimmer" daemon [_thread_blocked, id=12844, stack(0x5f6d0000,0x5f7d0000)]
      0x62515800 JavaThread "obj-skimmer" daemon [_thread_blocked, id=7396, stack(0x5f5d0000,0x5f6d0000)]
      0x6283e400 JavaThread "obj-skimmer" daemon [_thread_blocked, id=6164, stack(0x5f4d0000,0x5f5d0000)]
      0x5c731c00 JavaThread "scheduler-0" [_thread_blocked, id=11724, stack(0x5f3d0000,0x5f4d0000)]
      0x621e9000 JavaThread "Thread-41" daemon [_thread_blocked, id=3196, stack(0x5ef90000,0x5f090000)]
      0x5d32f800 JavaThread "jndi-5" [_thread_blocked, id=2808, stack(0x63ed0000,0x63fd0000)]
      0x5da96c00 JavaThread "Timer-8" daemon [_thread_blocked, id=9864, stack(0x733a0000,0x734a0000)]
      0x61f0e000 JavaThread "Timer-7" [_thread_blocked, id=3116, stack(0x732a0000,0x733a0000)]
      0x623f2c00 JavaThread "Timer-6" [_thread_blocked, id=3528, stack(0x731a0000,0x732a0000)]
      0x5d327c00 JavaThread "scheduler-22" [_thread_blocked, id=3612, stack(0x730a0000,0x731a0000)]
      0x621bc400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-36)" daemon [_thread_blocked, id=13668, stack(0x72ea0000,0x72fa0000)]
      0x61981400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-35)" daemon [_thread_blocked, id=15540, stack(0x72da0000,0x72ea0000)]
      0x5d9dcc00 JavaThread "FusionReactor Request Monitor" [_thread_blocked, id=3464, stack(0x72ca0000,0x72da0000)]
      0x5ddeb400 JavaThread "FusionReactor FD/R Remote Data Collector for {CF_Instance2a.jrun4.lcbwebott02}" [_thread_blocked, id=9760, stack(0x72ba0000,0x72ca0000)]
      0x5dcf9400 JavaThread "FusionReactor Web Server" [_thread_in_native, id=12396, stack(0x72aa0000,0x72ba0000)]
      0x5df7a400 JavaThread "FusionReactor Enterprise Service - Local Metrics Cache" daemon [_thread_blocked, id=9780, stack(0x729a0000,0x72aa0000)]
      0x5def8400 JavaThread "Timer-5" daemon [_thread_blocked, id=1640, stack(0x728a0000,0x729a0000)]
      0x621a4c00 JavaThread "Timer-4" daemon [_thread_blocked, id=13496, stack(0x727a0000,0x728a0000)]
      0x5da2a400 JavaThread "Timer-3" daemon [_thread_blocked, id=5568, stack(0x726a0000,0x727a0000)]
      0x61b60c00 JavaThread "Timer-2" daemon [_thread_blocked, id=10724, stack(0x725a0000,0x726a0000)]
      0x622bbc00 JavaThread "Timer-1" daemon [_thread_blocked, id=8252, stack(0x724a0000,0x725a0000)]
      0x5d840c00 JavaThread "Thread-25" daemon [_thread_blocked, id=13204, stack(0x71fa0000,0x720a0000)]
      0x6e037c00 JavaThread "Timer-0" daemon [_thread_blocked, id=10944, stack(0x70670000,0x70770000)]
      0x5d815400 JavaThread "task" daemon [_thread_blocked, id=8680, stack(0x71da0000,0x71ea0000)]
      0x5dea9800 JavaThread "task" daemon [_thread_blocked, id=8052, stack(0x71570000,0x71670000)]
      0x6228c000 JavaThread "Thread-23" daemon [_thread_blocked, id=11888, stack(0x71170000,0x71270000)]
      0x6228c800 JavaThread "Thread-22" daemon [_thread_blocked, id=15232, stack(0x70f70000,0x71070000)]
      0x62289000 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=8972, stack(0x70e70000,0x70f70000)]
      0x5da2ac00 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=11732, stack(0x70d70000,0x70e70000)]
      0x6e181c00 JavaThread "task" daemon [_thread_blocked, id=15432, stack(0x70870000,0x70970000)]
      0x5db7e400 JavaThread "task" daemon [_thread_blocked, id=15828, stack(0x70770000,0x70870000)]
      0x61c3dc00 JavaThread "Thread-21" daemon [_thread_blocked, id=12384, stack(0x6d9b0000,0x6dab0000)]
      0x6e052800 JavaThread "Thread-20" daemon [_thread_blocked, id=1760, stack(0x6d500000,0x6d600000)]
      0x61af4400 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=8832, stack(0x6d400000,0x6d500000)]
      0x627dd400 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=5612, stack(0x6d1b0000,0x6d2b0000)]
      0x5da64000 JavaThread "Thread-19" [_thread_blocked, id=15120, stack(0x6c6b0000,0x6c7b0000)]
      0x6229d800 JavaThread "task" daemon [_thread_blocked, id=6736, stack(0x6ccb0000,0x6cdb0000)]
      0x5de23400 JavaThread "task" daemon [_thread_blocked, id=15424, stack(0x6cbb0000,0x6ccb0000)]
      0x621f6c00 JavaThread "task" daemon [_thread_blocked, id=6324, stack(0x6b0b0000,0x6b1b0000)]
      0x5d3b5400 JavaThread "scheduler-21" [_thread_blocked, id=4284, stack(0x6aeb0000,0x6afb0000)]
      0x624c2800 JavaThread "Thread-17" daemon [_thread_blocked, id=6100, stack(0x6bdb0000,0x6beb0000)]
      0x624e6800 JavaThread "Thread-16" daemon [_thread_blocked, id=9080, stack(0x6bcb0000,0x6bdb0000)]
      0x621e5400 JavaThread "task" daemon [_thread_blocked, id=14136, stack(0x6bbb0000,0x6bcb0000)]
      0x6215e800 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=9112, stack(0x6b9b0000,0x6bab0000)]
      0x62522000 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=7220, stack(0x6b8b0000,0x6b9b0000)]
      0x622a8800 JavaThread "Thread-15" daemon [_thread_blocked, id=15848, stack(0x6afb0000,0x6b0b0000)]
      0x622b1400 JavaThread "task" daemon [_thread_blocked, id=4512, stack(0x6adb0000,0x6aeb0000)]
      0x62163400 JavaThread "Thread-14" daemon [_thread_blocked, id=3944, stack(0x6abb0000,0x6acb0000)]
      0x622b2c00 JavaThread "task" daemon [_thread_blocked, id=12724, stack(0x6aab0000,0x6abb0000)]
      0x623aa800 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=7736, stack(0x6a540000,0x6a640000)]
      0x623a9400 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=13556, stack(0x6a440000,0x6a540000)]
      0x62010000 JavaThread "scheduler-20" [_thread_blocked, id=13884, stack(0x69d40000,0x69e40000)]
      0x5d4e0c00 JavaThread "Thread-13" daemon [_thread_blocked, id=14880, stack(0x69640000,0x69740000)]
      0x621a0000 JavaThread "Thread-12" daemon [_thread_blocked, id=10732, stack(0x69440000,0x69540000)]
      0x5d41e800 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=8964, stack(0x69340000,0x69440000)]
      0x62216c00 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=6644, stack(0x69240000,0x69340000)]
      0x61b5b800 JavaThread "scheduler-19" [_thread_blocked, id=10856, stack(0x68f40000,0x69040000)]
      0x62178400 JavaThread "scheduler-18" [_thread_blocked, id=10184, stack(0x68e40000,0x68f40000)]
      0x62177000 JavaThread "scheduler-17" [_thread_blocked, id=6816, stack(0x68d40000,0x68e40000)]
      0x62176000 JavaThread "scheduler-16" [_thread_blocked, id=7700, stack(0x68c40000,0x68d40000)]
      0x62174000 JavaThread "scheduler-15" [_thread_blocked, id=8296, stack(0x68b40000,0x68c40000)]
      0x61b60000 JavaThread "scheduler-14" [_thread_blocked, id=6212, stack(0x68a40000,0x68b40000)]
      0x61b5e800 JavaThread "scheduler-13" [_thread_blocked, id=6868, stack(0x68940000,0x68a40000)]
      0x61b5d800 JavaThread "scheduler-12" [_thread_blocked, id=12472, stack(0x68840000,0x68940000)]
      0x62168400 JavaThread "scheduler-11" [_thread_blocked, id=9468, stack(0x68740000,0x68840000)]
      0x62167000 JavaThread "scheduler-10" [_thread_blocked, id=7224, stack(0x68640000,0x68740000)]
      0x62165800 JavaThread "scheduler-9" [_thread_blocked, id=3520, stack(0x68540000,0x68640000)]
      0x5d41b400 JavaThread "scheduler-8" [_thread_blocked, id=11824, stack(0x68440000,0x68540000)]
      0x5d419c00 JavaThread "scheduler-7" [_thread_blocked, id=4956, stack(0x68340000,0x68440000)]
      0x620dc400 JavaThread "scheduler-6" [_thread_blocked, id=14988, stack(0x68240000,0x68340000)]
      0x620db400 JavaThread "scheduler-5" [_thread_blocked, id=9580, stack(0x68140000,0x68240000)]
      0x5d3cf400 JavaThread "scheduler-4" [_thread_blocked, id=10220, stack(0x68040000,0x68140000)]
      0x5dc40c00 JavaThread "scheduler-3" [_thread_blocked, id=6780, stack(0x67ed0000,0x67fd0000)]
      0x621a4400 JavaThread "scheduler-2" [_thread_blocked, id=9216, stack(0x67dd0000,0x67ed0000)]
      0x619d5400 JavaThread "scheduler-1" [_thread_blocked, id=7032, stack(0x67ad0000,0x67bd0000)]
      0x5dd32400 JavaThread "scheduler-0" [_thread_blocked, id=15420, stack(0x665d0000,0x666d0000)]
      0x5df2bc00 JavaThread "Transaction Domain PrivateJmsDomain" [_thread_blocked, id=15384, stack(0x67cd0000,0x67dd0000)]
      0x5d2f7800 JavaThread "clock" daemon [_thread_blocked, id=7236, stack(0x67bd0000,0x67cd0000)]
      0x5d41f400 JavaThread "Transaction Domain DefaultDomain" [_thread_blocked, id=3908, stack(0x679d0000,0x67ad0000)]
      0x6223bc00 JavaThread "tyrex.util.daemonMaster" daemon [_thread_blocked, id=15468, stack(0x677d0000,0x678d0000)]
      0x621bcc00 JavaThread "task" daemon [_thread_blocked, id=7364, stack(0x676d0000,0x677d0000)]
      0x5df2d400 JavaThread "task" daemon [_thread_blocked, id=6052, stack(0x670d0000,0x671d0000)]
      0x5d927800 JavaThread "Thread-10" daemon [_thread_blocked, id=8776, stack(0x66bd0000,0x66cd0000)]
      0x61944c00 JavaThread "Thread-9" daemon [_thread_blocked, id=12100, stack(0x66ad0000,0x66bd0000)]
      0x620be400 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=13248, stack(0x669d0000,0x66ad0000)]
      0x5d52f400 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=15988, stack(0x668d0000,0x669d0000)]
      0x5d8dd400 JavaThread "task" daemon [_thread_blocked, id=12096, stack(0x664d0000,0x665d0000)]
      0x5d4cd400 JavaThread "task" daemon [_thread_blocked, id=12332, stack(0x65ed0000,0x65fd0000)]
      0x5dc4e400 JavaThread "Thread-8" daemon [_thread_blocked, id=11804, stack(0x659d0000,0x65ad0000)]
      0x5dce2400 JavaThread "Thread-7" daemon [_thread_blocked, id=13224, stack(0x658d0000,0x659d0000)]
      0x61ca1c00 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=992, stack(0x657d0000,0x658d0000)]
      0x61abb400 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=8984, stack(0x656d0000,0x657d0000)]
      0x5d87ac00 JavaThread "jndi-3" [_thread_blocked, id=10016, stack(0x654d0000,0x655d0000)]
      0x5da66800 JavaThread "jndi-2" [_thread_in_native, id=15032, stack(0x653d0000,0x654d0000)]
      0x5de6bc00 JavaThread "jndi-1" [_thread_blocked, id=10964, stack(0x652d0000,0x653d0000)]
      0x61ca2400 JavaThread "jndi-0" [_thread_blocked, id=13736, stack(0x651d0000,0x652d0000)]
      0x61918000 JavaThread "task" daemon [_thread_blocked, id=1892, stack(0x64cd0000,0x64dd0000)]
      0x61917c00 JavaThread "task" daemon [_thread_blocked, id=2692, stack(0x64bd0000,0x64cd0000)]
      0x61a64400 JavaThread "Thread-6" daemon [_thread_blocked, id=3580, stack(0x644d0000,0x645d0000)]
      0x5db8d400 JavaThread "Thread-5" daemon [_thread_blocked, id=15852, stack(0x645d0000,0x646d0000)]
      0x5df10800 JavaThread "Thread-4" daemon [_thread_blocked, id=12984, stack(0x643d0000,0x644d0000)]
      0x5da1c000 JavaThread "RMI RenewClean-[192.168.40.35:1941]" daemon [_thread_blocked, id=9096, stack(0x641d0000,0x642d0000)]
      0x5df7b800 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=6672, stack(0x63cd0000,0x63dd0000)]
      0x5dbef800 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=13076, stack(0x63bd0000,0x63cd0000)]
      0x5d335c00 JavaThread "jms-fifo-5" [_thread_blocked, id=5108, stack(0x638d0000,0x639d0000)]
      0x5dc50400 JavaThread "jms-fifo-4" [_thread_blocked, id=588, stack(0x637d0000,0x638d0000)]
      0x5daaa000 JavaThread "jms-fifo-3" [_thread_blocked, id=9968, stack(0x636d0000,0x637d0000)]
      0x5d4eb800 JavaThread "jms-fifo-2" [_thread_blocked, id=5636, stack(0x635d0000,0x636d0000)]
      0x61a40400 JavaThread "jms-fifo-1" [_thread_blocked, id=15884, stack(0x634d0000,0x635d0000)]
      0x5dbf6800 JavaThread "RMI RenewClean-[192.168.40.35:2178]" daemon [_thread_blocked, id=8716, stack(0x631d0000,0x632d0000)]
      0x5d455c00 JavaThread "RMI RenewClean-[192.168.40.34:2233]" daemon [_thread_blocked, id=14028, stack(0x614d0000,0x615d0000)]
      0x5dc71400 JavaThread "RMI RenewClean-[192.168.40.34:3534]" daemon [_thread_blocked, id=7068, stack(0x613d0000,0x614d0000)]
      0x5d509000 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=10984, stack(0x5ed90000,0x5ee90000)]
      0x5d819c00 JavaThread "snapshot thread" daemon [_thread_blocked, id=10840, stack(0x5eb90000,0x5ec90000)]
      0x5d7b3400 JavaThread "discovery announcement" daemon [_thread_blocked, id=6580, stack(0x5ea90000,0x5eb90000)]
      0x5c810800 JavaThread "multicast request" daemon [_thread_in_native, id=11208, stack(0x5e990000,0x5ea90000)]
      0x5c74a400 JavaThread "unicast request" daemon [_thread_in_native, id=1328, stack(0x5e890000,0x5e990000)]
      0x5d303800 JavaThread "event expire" daemon [_thread_blocked, id=7796, stack(0x5e790000,0x5e890000)]
      0x5c7b8800 JavaThread "service expire" daemon [_thread_blocked, id=15348, stack(0x5e690000,0x5e790000)]
      0x5c7b8400 JavaThread "Thread-3" daemon [_thread_blocked, id=13080, stack(0x5e590000,0x5e690000)]
      0x5d3b6800 JavaThread "Thread-2" daemon [_thread_blocked, id=6772, stack(0x5e490000,0x5e590000)]
      0x5d3b6400 JavaThread "multicast announcement timer" daemon [_thread_blocked, id=13304, stack(0x5e390000,0x5e490000)]
      0x5d4e1c00 JavaThread "multicast discovery announcement listener" daemon [_thread_in_native, id=10632, stack(0x5e290000,0x5e390000)]
      0x5d3d5400 JavaThread "GC Daemon" daemon [_thread_blocked, id=13068, stack(0x5e190000,0x5e290000)]
      0x5d492800 JavaThread "RMI Reaper" [_thread_blocked, id=6540, stack(0x5e090000,0x5e190000)]
      0x5c6cc000 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=10780, stack(0x5df90000,0x5e090000)]
      0x5c69fc00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=8196, stack(0x5cf70000,0x5d070000)]
    =>0x5c69c800 JavaThread "CompilerThread1" daemon [_thread_in_native, id=4960, stack(0x5ce70000,0x5cf70000)]
      0x5c697c00 JavaThread "CompilerThread0" daemon [_thread_blocked, id=5152, stack(0x5cd70000,0x5ce70000)]
      0x5c696400 JavaThread "Attach Listener" daemon [_thread_blocked, id=14576, stack(0x5cc70000,0x5cd70000)]
      0x5c695400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=16064, stack(0x5cb70000,0x5cc70000)]
      0x5bf5f800 JavaThread "Finalizer" daemon [_thread_blocked, id=16192, stack(0x5ca70000,0x5cb70000)]
      0x5bf5e400 JavaThread "Reference Handler" daemon [_thread_blocked, id=7228, stack(0x5c970000,0x5ca70000)]
    Other Threads:
      0x5c691000 VMThread [stack: 0x5c870000,0x5c970000] [id=8468]
      0x5c6a1c00 WatcherThread [stack: 0x5d070000,0x5d170000] [id=10620]
    VM state:not at safepoint (normal execution)
    VM Mutex/Monitor currently owned by a thread: None
    Heap
    PSYoungGen      total 89856K, used 19025K [0x55490000, 0x5b6f0000, 0x5b810000)
      eden space 79232K, 16% used [0x55490000,0x561a64c0,0x5a1f0000)
      from space 10624K, 52% used [0x5ac90000,0x5b20e2f8,0x5b6f0000)
      to   space 10752K, 0% used [0x5a1f0000,0x5a1f0000,0x5ac70000)
    PSOldGen        total 460416K, used 308422K [0x23810000, 0x3f9b0000, 0x55490000)
      object space 460416K, 66% used [0x23810000,0x36541bb8,0x3f9b0000)
    PSPermGen       total 107520K, used 106079K [0x03810000, 0x0a110000, 0x23810000)
      object space 107520K, 98% used [0x03810000,0x09fa7e40,0x0a110000)
    Dynamic libraries:
    0x00400000 - 0x00411000     C:\JRun4\bin\jrun.exe
    0x7c800000 - 0x7c8c2000     C:\WINDOWS\system32\ntdll.dll
    0x77e40000 - 0x77f42000     C:\WINDOWS\system32\kernel32.dll
    0x7d1e0000 - 0x7d27c000     C:\WINDOWS\system32\ADVAPI32.dll
    0x77c50000 - 0x77cef000     C:\WINDOWS\system32\RPCRT4.dll
    0x76f50000 - 0x76f63000     C:\WINDOWS\system32\Secur32.dll
    0x78520000 - 0x785c3000     C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_D08D0375\MSVCR 90.dll
    0x6db60000 - 0x6df6a000     C:\JRun4\jre\bin\server\jvm.dll
    0x77380000 - 0x77411000     C:\WINDOWS\system32\USER32.dll
    0x77c00000 - 0x77c49000     C:\WINDOWS\system32\GDI32.dll
    0x76aa0000 - 0x76acd000     C:\WINDOWS\system32\WINMM.dll
    0x7c340000 - 0x7c396000     C:\JRun4\bin\MSVCR71.dll
    0x76290000 - 0x762ad000     C:\WINDOWS\system32\IMM32.DLL
    0x7f000000 - 0x7f009000     C:\WINDOWS\system32\LPK.DLL
    0x75490000 - 0x754f5000     C:\WINDOWS\system32\USP10.dll
    0x6d860000 - 0x6d86c000     C:\JRun4\jre\bin\verify.dll
    0x6d3e0000 - 0x6d3ff000     C:\JRun4\jre\bin\java.dll
    0x6d340000 - 0x6d348000     C:\JRun4\jre\bin\hpi.dll
    0x76b70000 - 0x76b7b000     C:\WINDOWS\system32\PSAPI.DLL
    0x6d8a0000 - 0x6d8af000     C:\JRun4\jre\bin\zip.dll
    0x6d6c0000 - 0x6d6d3000     C:\JRun4\jre\bin\net.dll
    0x71c00000 - 0x71c17000     C:\WINDOWS\system32\WS2_32.dll
    0x77ba0000 - 0x77bfa000     C:\WINDOWS\system32\msvcrt.dll
    0x71bf0000 - 0x71bf8000     C:\WINDOWS\system32\WS2HELP.dll
    0x71b20000 - 0x71b61000     C:\WINDOWS\System32\mswsock.dll
    0x76ed0000 - 0x76efa000     C:\WINDOWS\system32\DNSAPI.dll
    0x76f70000 - 0x76f77000     C:\WINDOWS\System32\winrnr.dll
    0x76f10000 - 0x76f3e000     C:\WINDOWS\system32\WLDAP32.dll
    0x76f80000 - 0x76f85000     C:\WINDOWS\system32\rasadhlp.dll
    0x5d630000 - 0x5d63c000     C:\JRun4\bin\portscan.dll
    0x5f270000 - 0x5f2ca000     C:\WINDOWS\system32\hnetcfg.dll
    0x71ae0000 - 0x71ae8000     C:\WINDOWS\System32\wshtcpip.dll
    0x68000000 - 0x68035000     C:\WINDOWS\system32\rsaenh.dll
    0x76920000 - 0x769e2000     C:\WINDOWS\system32\USERENV.dll
    0x71c40000 - 0x71c97000     C:\WINDOWS\system32\netapi32.dll
    0x6d7b0000 - 0x6d7b6000     C:\JRun4\jre\bin\rmi.dll
    0x6d6e0000 - 0x6d6e9000     C:\JRun4\jre\bin\nio.dll
    0x5f190000 - 0x5f197000     C:\JRun4\bin\FusionReactor.dll
    0x77d00000 - 0x77d8b000     C:\WINDOWS\system32\OLEAUT32.dll
    0x77670000 - 0x777a9000     C:\WINDOWS\system32\ole32.dll
    0x5f1a0000 - 0x5f1aa000     C:\WINDOWS\system32\perfos.dll
    0x6d840000 - 0x6d848000     C:\JRun4\jre\bin\sunmscapi.dll
    0x761b0000 - 0x76243000     C:\WINDOWS\system32\CRYPT32.dll
    0x76190000 - 0x761a2000     C:\WINDOWS\system32\MSASN1.dll
    0x5f090000 - 0x5f110000     C:\JRun4\servers\CF_Instance2a\cfusion.ear\cfusion.war\WEB-INF\cfusion\lib\izmjniado.dll
    0x76cf0000 - 0x76d0a000     C:\WINDOWS\system32\iphlpapi.dll
    0x76cd0000 - 0x76ce9000     C:\WINDOWS\system32\MPRAPI.dll
    0x76df0000 - 0x76e24000     C:\WINDOWS\system32\ACTIVEDS.dll
    0x76dc0000 - 0x76de8000     C:\WINDOWS\system32\adsldpc.dll
    0x76b80000 - 0x76bae000     C:\WINDOWS\system32\credui.dll
    0x7c8d0000 - 0x7d0cf000     C:\WINDOWS\system32\SHELL32.dll
    0x7d180000 - 0x7d1d2000     C:\WINDOWS\system32\SHLWAPI.dll
    0x76a80000 - 0x76a98000     C:\WINDOWS\system32\ATL.DLL
    0x76e30000 - 0x76e3c000     C:\WINDOWS\system32\rtutils.dll
    0x7e020000 - 0x7e02f000     C:\WINDOWS\system32\SAMLIB.dll
    0x770e0000 - 0x771e8000     C:\WINDOWS\system32\SETUPAPI.dll
    0x77420000 - 0x77523000     C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.3790.3959_x- ww_D8713E55\comctl32.dll
    0x5f200000 - 0x5f223000     C:\JRun4\servers\CF_Instance2a\cfusion.ear\cfusion.war\WEB-INF\cfusion\lib\CFXNeo.dll
    0x5f230000 - 0x5f243000     C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\lib\cfregistry.dll
    0x647d0000 - 0x647e3000     C:\JRun4\servers\CF_Instance2a\cfusion.ear\cfusion.war\WEB-INF\cfusion\lib\cfregistry.dll
    0x65ad0000 - 0x65c1a000     C:\JRun4\jre\bin\awt.dll
    0x64a00000 - 0x64a27000     C:\WINDOWS\system32\WINSPOOL.DRV
    0x77530000 - 0x775c7000     C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_5.82.3790.3959_x -ww_78FCF8D0\comctl32.dll
    0x6d2e0000 - 0x6d334000     C:\JRun4\jre\bin\fontmanager.dll
    0x64900000 - 0x6490e000     C:\JRun4\servers\CF_Instance2a\cfusion.ear\cfusion.war\WEB-INF\cfusion\lib\NeoUUID.dll
    0x6d610000 - 0x6d619000     C:\JRun4\jre\bin\management.dll
    0x5f2d0000 - 0x5f380000     K:\Instance\Core\WEB02_2a_cfx_image_mx.dll
    0x73eb0000 - 0x73fd1000     C:\WINDOWS\system32\MFC42.DLL
    0x60280000 - 0x60351000     C:\WINDOWS\system32\WININET.dll
    0x5f380000 - 0x5f389000     C:\WINDOWS\system32\Normaliz.dll
    0x5fad0000 - 0x5fb15000     C:\WINDOWS\system32\iertutil.dll
    0x71bb0000 - 0x71bb9000     C:\WINDOWS\system32\WSOCK32.dll
    0x5f390000 - 0x5f3cd000     C:\WINDOWS\system32\ODBC32.dll
    0x762b0000 - 0x762f9000     C:\WINDOWS\system32\comdlg32.dll
    0x5fb20000 - 0x5fb30000     C:\WINDOWS\system32\MSVCIRT.dll
    0x607e0000 - 0x607f7000     C:\WINDOWS\system32\odbcint.dll
    0x5ec90000 - 0x5ecb4000     C:\JRun4\jre\bin\jpeg.dll
    0x5ecd0000 - 0x5ecdb000     C:\JRun4\servers\CF_Instance2a\cfusion.ear\cfusion.war\WEB-INF\cfusion\lib\clib_jiio_util .dll
    0x61180000 - 0x6128c000     C:\JRun4\servers\CF_Instance2a\cfusion.ear\cfusion.war\WEB-INF\cfusion\lib\clib_jiio_sse2 .dll
    0x5ed00000 - 0x5ed23000     C:\JRun4\jre\bin\dcpr.dll
    VM Arguments:
    jvm_args: -Xmx896m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=512m -XX:+UseParallelGC -Dcoldfusion.rootDir=C:\JRun4/ -verbose:gc -Xloggc:c:/Jrun4/logs/gc/gcInstance2a.log -Djmx.invoke.getters=true
    java_command: <unknown>
    Launcher Type: generic
    Environment Variables:
    PATH=C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\lib;C:\JRun4\servers \cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\jintegra\bin;C:\JRun4\servers\cfusion\cfu sion-ear\cfusion-war\WEB-INF\cfusion\jintegra\bin\international;C:\JRun4\servers\cfusion\c fusion-ear\cfusion-war\WEB-INF\cfusion\lib\oosdk\classes\win;C:\Program Files\Windows Resource Kits\Tools\;C:\Program Files\Support Tools\;C:\Perl\bin\;C:\JRun4\verity\k2\_nti40\bin;C:\Program Files\EMC\PowerPath\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Dell\SysMgt\RAC4;C:\Program Files\Dell\SysMgt\oma\bin;C:\Program Files\Microsoft SQL Server\80\Tools\Binn\;C:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\Microsoft SQL Server\90\DTS\Binn\;C:\Program Files\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7\IDE\;C:\PROGRA~1\SAAZOD\zScripts\
    USERNAME=lcbadmin
    OS=Windows_NT
    PROCESSOR_IDENTIFIER=x86 Family 15 Model 4 Stepping 8, GenuineIntel
    ---------------  S Y S T E M  ---------------
    OS: Windows Server 2003 family Build 3790 Service Pack 2
    CPU:total 8 (2 cores per cpu, 2 threads per core) family 15 model 4 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht
    Memory: 4k page, physical 2097151k(2097151k free), swap 4194303k(4194303k free)
    vm_info: Java HotSpot(TM) Server VM (14.0-b16) for windows-x86 JRE (1.6.0_14-b08), built on May 21 2009 07:57:57 by "java_re" with MS VC++ 7.1
    time: Tue May 18 21:48:13 2010
    elapsed time: 154379 seconds
    0.381: [GC 3072K->717K(7680K), 0.0028050 secs]
    0.578: [GC 3789K->923K(7680K), 0.0026605 secs]
    2.581: [GC 3995K->1086K(7680K), 0.0028255 secs]
    2.860: [GC 4158K->1283K(7680K), 0.0042886 secs]
    3.056: [GC 4355K->1467K(7680K), 0.0028442 secs]
    3.231: [GC 4537K->1648K(11008K), 0.0025800 secs]
    3.713: [GC 7792K->2190K(11008K), 0.0040999 secs]
    3.985: [GC 4856K->2263K(17408K), 0.0025673 secs]
    3.988: [Full GC 2263K->2146K(17408K), 0.0487271 secs]
    4.696: [GC 14434K->3018K(16832K), 0.0051747 secs]
    5.486: [GC 14858K->3805K(16192K), 0.0045260 secs]
    6.045: [GC 15197K->4709K(16000K), 0.0074686 secs]
    6.053: [Full GC 4709K->4451K(22464K), 0.1103924 secs]
    6.787: [GC 15039K->5937K(22528K), 0.0117950 secs]
    7.218: [GC 16625K->7804K(22912K), 0.0099954 secs]
    7.573: [GC 18492K->10050K(24704K), 0.0144209 secs]
    8.256: [GC 21826K->10582K(25024K), 0.0079624 secs]
    8.639: [GC 22358K->10701K(27776K), 0.0130334 secs]
    8.652: [Full GC 10701K->7499K(36096K), 0.1193615 secs]
    10.349: [GC 21067K->8260K(36160K), 0.0036341 secs]
    14.083: [GC 21828K->10306K(37184K), 0.0113857 secs]
    15.701: [GC 25090K->9822K(37696K), 0.0059270 secs]
    16.614: [GC 24606K->11610K(38272K), 0.0164009 secs]
    16.824: [GC 26842K->11008K(38976K), 0.0064352 secs]
    17.759: [GC 26240K->11017K(40128K), 0.0056167 secs]
    17.967: [GC 27657K->11008K(40384K), 0.0056861 secs]
    18.194: [GC 27648K->11008K(42240K), 0.0062386 secs]
    18.530: [GC 29504K->11009K(42176K), 0.0052893 secs]
    19.610: [GC 29505K->11657K(43200K), 0.0098857 secs]
    19.912: [GC 31561K->11633K(43712K), 0.0067032 secs]
    20.364: [GC 31537K->12066K(45184K), 0.0081479 secs]
    20.805: [GC 32470K->12014K(45440K), 0.0109562 secs]
    22.058: [GC 33262K->17043K(46848K), 0.0195642 secs]
    23.211: [GC 39571K->19376K(47424K), 0.0259335 secs]
    24.402: [GC 41904K->20994K(50112K), 0.0253877 secs]
    25.890: [GC 44930K->21307K(50624K), 0.0184082 secs]
    26.371: [GC 45243K->22727K(52160K), 0.0178302 secs]
    26.884: [GC 47687K->23246K(53120K), 0.0208570 secs]
    27.995: [GC 48206K->25664K(54592K), 0.0241083 secs]
    29.207: [GC 51298K->31755K(57792K), 0.0290945 secs]
    29.236: [Full GC 31755K->27853K(74304K), 0.3884927 secs]
    29.924: [GC 46184K->38399K(76288K), 0.0460258 secs]
    30.401: [GC 62847K->55516K(80192K), 0.0404456 secs]
    30.441: [Full GC 55516K->51591K(107648K), 0.3037522 secs]
    30.873: [GC 62219K->59963K(115968K), 0.0441025 secs]
    31.388: [GC 84732K->72340K(118976K), 0.0392704 secs]
    31.756: [GC 97172K->90818K(126720K), 0.0627947 secs]
    31.819: [Full GC 90818K->89854K(162432K), 0.3339941 secs]
    32.169: [GC 92791K->90051K(162432K), 0.0469329 secs]
    32.477: [GC 117269K->98790K(164992K), 0.0352808 secs]
    33.032: [GC 128657K->112415K(164992K), 0.0569482 secs]
    33.625: [GC 142303K->129501K(167872K), 0.0829067 secs]
    33.708: [Full GC 129501K->128716K(214720K), 0.3245804 secs]
    34.410: [GC 133817K->130014K(214720K), 0.0563804 secs]
    47.633: [GC 162782K->137909K(216000K), 0.0515256 secs]
    49.988: [GC 171957K->141420K(216000K), 0.0516951 secs]
    51.694: [GC 175468K->145781K(216000K), 0.1004621 secs]
    51.795: [Full GC 145781K->134519K(260224K), 0.5717566 secs]
    53.350: [GC 168567K->136461K(260224K), 0.0109643 secs]
    54.436: [GC 170509K->141065K(260224K), 0.0167210 secs]
    55.290: [GC 175113K->145039K(260224K), 0.0231981 secs]
    56.200: [GC 179087K->151640K(260224K), 0.0254553 secs]
    57.069: [GC 185688K->159187K(260224K), 0.0374343 secs]
    58.092: [GC 193235K->165658K(260224K), 0.0415577 secs]
    58.735: [GC 199706K->171695K(260224K), 0.0381981 secs]
    59.441: [GC 205743K->177923K(260224K), 0.0466761 secs]
    60.138: [GC 211971K->184639K(260224K), 0.0398748 secs]
    60.178: [Full GC 184639K->184545K(308672K), 0.6346645 secs]
    61.473: [GC 218593K->191727K(308672K), 0.0228897 secs]
    62.103: [GC 225775K->198230K(308672K), 0.0452323 secs]
    62.593: [GC 232278K->204937K(308672K), 0.0441287 secs]
    63.152: [GC 238985K->210234K(308672K), 0.0466643 secs]
    63.631: [GC 244282K->214332K(308672K), 0.0308136 secs]
    64.069: [GC 248380K->220520K(308672K), 0.0310264 secs]
    64.490: [GC 254568K->224630K(307648K), 0.0338149 secs]
    64.887: [GC 258678K->227375K(281664K), 0.0177588 secs]
    65.336: [GC 261423K->232312K(307456K), 0.0275194 secs]
    66.622: [GC 268984K->233248K(307584K), 0.0294454 secs]
    68.689: [GC 269920K->233475K(307200K), 0.0303114 secs]
    70.063: [GC 272323K->233989K(308224K), 0.0319816 secs]
    70.948: [GC 272827K->236267K(309504K), 0.0381517 secs]
    72.151: [GC 276971K->243396K(309760K), 0.0479739 secs]
    72.810: [GC 284100K->256970K(305792K), 0.0670952 secs]
    73.202: [GC 291018K->241617K(308672K), 0.0469614 secs]
    73.250: [Full GC 241617K->241374K(365888K), 1.0516054 secs]
    75.419: [GC 274751K->244364K(365888K), 0.0663203 secs]
    76.501: [GC 278366K->244598K(365888K), 0.0116506 secs]
    77.218: [GC 278646K->249688K(365376K), 0.0161159 secs]
    77.761: [GC 283736K->249237K(340544K), 0.0173527 secs]
    78.002: [GC 283285K->249652K(366336K), 0.1664675 secs]
    78.802: [GC 286066K->250024K(364736K), 0.0352561 secs]
    86.046: [GC 286428K->246641K(365952K), 0.0169778 secs]
    89.807: [GC 286513K->247751K(366080K), 0.0128220 secs]
    104.515: [GC 287623K->247447K(365824K), 0.0143550 secs]
    105.456: [GC 290071K->247027K(367168K), 0.0130806 secs]
    107.171: [GC 289651K->251837K(364800K), 0.0211884 secs]
    107.490: [GC 296701K->253512K(366848K), 0.0189826 secs]
    107.981: [GC 298376K->251486K(365760K), 0.0193742 secs]
    109.163: [GC 298971K->250285K(367168K), 0.0158604 secs]
    115.811: [GC 297773K->250474K(365760K), 0.0139898 secs]
    118.378: [GC 299946K->254646K(367168K), 0.0217229 secs]
    119.027: [GC 304118K->259176K(367296K), 0.0324158 secs]
    119.615: [GC 308840K->262151K(368896K), 0.0358532 secs]
    120.623: [GC 311815K->270706K(366784K), 0.0824720 secs]
    121.470: [GC 316146K->279753K(371584K), 0.0766884 secs]
    122.268: [GC 325193K->288492K(360192K), 0.1021346 secs]
    122.929: [GC 322540K->294521K(365888K), 0.0907757 secs]
    123.531: [GC 328569K->300541K(365888K), 0.0978875 secs]
    124.140: [GC 334589K->305907K(365888K), 0.0988647 secs]
    124.703: [GC 339955K->311069K(365888K), 0.0951130 secs]
    125.315: [GC 345117K->312745K(365888K), 0.0801517 secs]
    125.533: [GC 346793K->313321K(365888K), 0.0943218 secs]
    125.627: [Full GC 313321K->279735K(412288K), 1.3003860 secs]
    126.992: [GC 313783K->280075K(412288K), 0.0125182 secs]
    127.076: [GC 314123K->280267K(412288K), 0.0065298 secs]
    128.996: [GC 314315K->283281K(412288K), 0.0107040 secs]
    129.398: [GC 317329K->283830K(412288K), 0.0095192 secs]
    129.695: [GC 317878K->283843K(412288K), 0.0104606 secs]
    134.828: [GC 317891K->284435K(412288K), 0.0091083 secs]
    135.574: [GC 318483K->287393K(412288K), 0.0108201 secs]
    135.749: [GC 321441K->286125K(412288K), 0.0087844 secs]
    135.795: [GC 320173K->286269K(412288K), 0.0064130 secs]
    135.841: [GC 320317K->286401K(412288K), 0.0059679 secs]
    136.118: [GC 320449K->286916K(412288K), 0.0069953 secs]
    136.761: [GC 320964K->288092K(412288K), 0.0079408 secs]
    138.931: [GC 322140K->288414K(412288K), 0.0071464 secs]
    155.109: [GC 322462K->290915K(412288K), 0.0073039 secs]
    162.080: [GC 324963K->290026K(378752K), 0.0085490 secs]
    166.335: [GC 324074K->292428K(409984K), 0.0075349 secs]
    166.668: [GC 327820K->293701K(383808K), 0.0110931 secs]
    167.470: [GC 329093K->295465K(406464K), 0.0431141 secs]
    167.814: [GC 332777K->297419K(408256K), 0.0173732 secs]
    168.042: [GC 334731K->295698K(406976K), 0.0249668 secs]
    168.530: [GC 336018K->294099K(408128K), 0.0117701 secs]
    186.485: [GC 334406K->294290K(405568K), 0.0099247 secs]
    189.467: [GC 337042K->293803K(407552K), 0.0092321 secs]
    192.025: [GC 336555K->294114K(405184K), 0.0144861 secs]
    192.537: [GC 338658K->293848K(406848K), 0.0096140 secs]
    198.694: [GC 338392K->294634K(404800K), 0.0275907 secs]
    200.414: [GC 340714K->295225K(406208K), 0.0243144 secs]
    201.828: [GC 341305K->294499K(404800K), 0.0191665 secs]
    202.519: [GC 342051K->294865K(405824K), 0.0105044 secs]
    207.017: [GC 342417K->295663K(404992K), 0.0190171 secs]
    218.114: [GC 344495K->297980K(405760K), 0.0192997 secs]
    218.433: [GC 346812K->296451K(407424K), 0.0127834 secs]
    219.292: [GC 346819K->296114K(407168K), 0.0103118 secs]
    227.933: [GC 346482K->298582K(407936K), 0.0205685 secs]
    228.202: [GC 350358K->299398K(408192K), 0.0224917 secs]
    228.984: [GC 351174K->297324K(410048K), 0.0120500 secs]
    231.777: [GC 351722K->297050K(410816K), 0.0146803 secs]
    236.016: [GC 351431K->297635K(411136K), 0.0126529 secs]
    239.092: [GC 353955K->298634K(411840K), 0.0155092 secs]
    239.552: [GC 354954K->299904K(413120K), 0.0215851 secs]
    252.632: [GC 357566K->298439K(413440K), 0.0137175 secs]
    269.175: [GC 356103K->298922K(414016K), 0.0170458 secs]
    273.686: [GC 358166K->299811K(414592K), 0.0213339 secs]
    275.229: [GC 359066K->301064K(415616K), 0.0209921 secs]
    288.095: [GC 361412K->304481K(416000K), 0.0232993 secs]
    288.392: [GC 364833K->303245K(418624K), 0.0226790 secs]
    301.675: [GC 364685K->302009K(418816K), 0.0362261 secs]
    307.093: [GC 363449K->300955K(419392K), 0.0126385 secs]
    309.805: [GC 364430K->301618K(420160K), 0.0126741 secs]
    325.062: [GC 365106K->302651K(420352K), 0.0151472 secs]
    339.201: [GC 367545K->301935K(420928K), 0.0181489 secs]
    346.999: [GC 366831K->301900K(420864K), 0.0125003 secs]
    348.433: [GC 367756K->303969K(421312K), 0.0198808 secs]
    357.384: [GC 369825K->302469K(421888K), 0.0198447 secs]
    358.633: [GC 369349K->303806K(422272K), 0.3034111 secs]
    359.588: [GC 370686K->306106K(427136K), 0.3337606 secs]
    360.523: [GC 378170K->306180K(427392K), 0.0179965 secs]
    361.119: [GC 378244K->304632K(434688K), 0.0256676 secs]
    364.956: [GC 384248K->303336K(434880K), 0.0169394 secs]
    365.935: [GC 382952K->305060K(434624K), 0.0194296 secs]
    368.833: [GC 385060K->305320K(434944K), 0.0149410 secs]
    370.448: [GC 385320K->304360K(435200K), 0.0216550 secs]
    377.487: [GC 385185K->306224K(435456K), 0.0202275 secs]
    385.574: [GC 387056K->305525K(435392K), 0.0180600 secs]
    388.927: [GC 386805K->306564K(435648K), 0.0175783 secs]
    391.674: [GC 387844K->309037K(434304K), 0.0254813 secs]
    407.514: [GC 388909K->308487K(433600K), 0.0175757 secs]
    409.194: [GC 388359K->311467K(432512K), 0.0209529 secs]
    414.063: [GC 388395K->314784K(433728K), 0.0241777 secs]
    415.431: [GC 391712K->313315K(430912K), 0.0223951 secs]
    430.083: [GC 387427K->314564K(430592K), 0.0232336 secs]
    439.450: [GC 388676K->315193K(430400K), 0.0247885 secs]
    441.171: [GC 386745K->312782K(431040K), 0.0169260 secs]
    452.250: [GC 384334K->313844K(430720K), 0.0202268 secs]
    460.522: [GC 385844K->319503K(430976K), 0.2599954 secs]
    468.974: [GC 391503K->322586K(424064K), 0.0422984 secs]
    469.474: [GC 387095K->323360K(424512K), 0.0407064 secs]
    477.136: [GC 387872K->322728K(425728K), 0.0311824 secs]
    478.559: [GC 384552K->323311K(421760K), 0.0272584 secs]
    484.216: [GC 385128K->322961K(425280K), 0.0298063 secs]
    495.042: [GC 383377K->323348K(425472K), 0.0294568 secs]
    507.955: [GC 383750K->323970K(424128K), 0.0292337 secs]
    513.449: [GC 384514K->323429K(424832K), 0.0228552 secs]
    522.316: [GC 383973K->324246K(424256K), 0.0451143 secs]
    529.871: [GC 385935K->326326K(424896K), 0.0519706 secs]
    530.225: [GC 388022K->324837K(426240K), 0.0314384 secs]
    534.490: [GC 388133K->324714K(425792K), 0.0241508 secs]
    563.372: [GC 388010K->324722K(427456K), 0.0246593 secs]
    572.215: [GC 390130K->324496K(427200K), 0.0251314 secs]
    573.388: [GC 389904K->324967K(427712K), 0.0333442 secs]
    585.876: [GC 391143K->325337K(416128K), 0.0146273 secs]
    588.092: [GC 391513K->326307K(427840K), 0.0150402 secs]
    590.242: [GC 391587K->327561K(417344K), 0.0243195 secs]
    597.280: [GC 392841K->326350K(427904K), 0.0166214 secs]
    605.425: [GC 391822K->325727K(427776K), 0.0130987 secs]
    621.244: [GC 391199K->326293K(427840K), 0.0137431 secs]
    631.514: [GC 392213K->325980K(427968K), 0.0116427 secs]
    635.911: [GC 391900K->326451K(427840K), 0.0144855 secs]
    642.943: [GC 393075K->327191K(428096K), 0.0141549 secs]
    650.865: [GC 393815K->328001K(427136K), 0.0262237 secs]
    654.381: [GC 395201K->327424K(427712K), 0.0161351 secs]
    654.834: [GC 394624K->327019K(428288K), 0.0150662 secs]
    665.058: [GC 395243K->327139K(427904K), 0.0115758 secs]
    673.535: [GC 395352K->326908K(427776K), 0.0198125 secs]
    695.278: [GC 396284K->327746K(428352K), 0.0181528 secs]
    704.497: [GC 397122K->328052K(427456K), 0.0190775 secs]
    711.387: [GC 398196K->329218K(428096K), 0.0499620 secs]
    719.837: [GC 399362K->328605K(427200K), 0.0170412 secs]
    735.922: [GC 399517K->327069K(427776K), 0.0145175 secs]
    736.804: [GC 397981K->327242K(428416K), 0.0110488 secs]
    747.130: [GC 398986K->328367K(428224K), 0.0190119 secs]
    749.678: [GC 400104K->327812K(428224K), 0.0143051 secs]
    767.357: [GC 400452K->328673K(428416K), 0.0338326 secs]
    772.008: [GC 401313K->330038K(428352K), 0.0205973 secs]
    773.157: [GC 403446K->328909K(428544K), 0.0178709 secs]
    773.757: [GC 402310K->328315K(429696K), 0.0140103 secs]
    779.192: [GC 403187K->329304K(429440K), 0.0122486 secs]
    784.114: [GC 404184K->329444K(429376K), 0.0136336 secs]
    800.024: [GC 405604K->330673K(430016K), 0.0161595 secs]
    810.128: [GC 406833K->329507K(430208K), 0.0155031 secs]
    822.611: [GC 406518K->328618K(430656K), 0.0159114 secs]
    830.465: [GC 405674K->329259K(430720K), 0.0134547 secs]
    832.320: [GC 406955K->331849K(431040K), 0.0192622 secs]
    839.589: [GC 409543K->330317K(431488K), 0.0152272 secs]
    846.029: [GC 408841K->330677K(431936K), 0.0146644 secs]
    855.691: [GC 409205K->330917K(432000K), 0.0147851 secs]
    857.089: [GC 410085K->329282K(432256K), 0.0113335 secs]
    862.172: [GC 408450K->330507K(433280K), 0.0744561 secs]
    864.102: [GC 410827K->329939K(433152K), 0.0602524 secs]
    871.764: [GC 410259K->337849K(432896K), 0.0464115 secs]
    873.725: [GC 418041K->340453K(435264K), 0.0492662 secs]
    875.230: [GC 420645K->343085K(429760K), 0.0488546 secs]
    876.560: [GC 417581K->348660K(432512K), 0.0499122 secs]
    881.246: [GC 423156K->354299K(420736K), 0.0444023 secs]
    881.291: [Full GC 354299K->184625K(324608K), 1.7573846 secs]
    885.122: [GC 247345K->189442K(330496K), 0.0344186 secs]
    890.421: [GC 252162K->189400K(329792K), 0.0117897 secs]
    892.905: [GC 252312K->191219K(330240K), 0.0194622 secs]
    894.713: [GC 254126K->190575K(329984K), 0.0155164 secs]
    896.839: [GC 254816K->190146K(330368K), 0.0139860 secs]
    901.632: [GC 254400K->189745K(330304K), 0.0130588 secs]
    909.589: [GC 255600K->191848K(331008K), 0.0234403 secs]
    916.343: [GC 257704K->191739K(329920K), 0.0422734 secs]
    921.147: [GC 258939K->193093K(330752K), 0.0297285 secs]
    925.261: [GC 260293K->192604K(330368K), 0.0226062 secs]
    929.613: [GC 261084K->191169K(331008K), 0.0156673 secs]
    944.701: [GC 259649K->192220K(330240K), 0.0410416 secs]
    949.753: [GC 261788K->192155K(331008K), 0.0406027 secs]
    953.023: [GC 261717K->192548K(331008K), 0.0233991 secs]
    956.654: [GC 263396K->192558K(331328K), 0.0204636 secs]
    958.453: [GC 263401K->191823K(332352K), 0.0191734 secs]
    962.069: [GC 264143K->192447K(332288K), 0.0143854 secs]
    966.391: [GC 264767K->191895K(332928K), 0.0149928 secs]
    986.458: [GC 265488K->194857K(333184K), 0.0228897 secs]
    998.709: [GC 268457K->192952K(333568K), 0.0181598 secs]
    1004.050: [GC 267448K->192375K(333952K), 0.0159302 secs]
    1010.544: [GC 266871K->192397K(334144K), 0.0150398 secs]
    1013.933: [GC 267661K->194230K(334400K), 0.0254282 secs]
    1018.907: [GC 269494K->194579K(334784K), 0.0176419 secs]
    1034.138: [GC 270803K->193420K(334976K), 0.0201663 secs]
    1068.738: [GC 269638K->192210K(335360K), 0.0204423 secs]
    1071.498: [GC 269128K->193375K(335488K), 0.0151018 secs]
    1074.150: [GC 270303K->195766K(335936K), 0.0306906 secs]
    1078.711: [GC 273334K->194600K(336128K), 0.0151313 secs]
    1091.405: [GC 272168K->193745K(336192K), 0.0133192 secs]
    1117.409: [GC 272273K->193573K(336576K), 0.0178818 secs]
    1131.229: [GC 272101K->193468K(336960K), 0.0162684 secs]
    1135.116: [GC 272700K->195603K(337152K), 0.0260614 secs]
    1150.247: [GC 274835K->193585K(337536K), 0.0128450 secs]
    1179.512: [GC 273585K->194432K(337792K), 0.0153779 secs]
    1186.196: [GC 274432K->194550K(337728K), 0.0154796 secs]
    1195.271: [GC 275190K->196552K(337984K), 0.0223260 secs]
    1203.048: [GC 277192K->196154K(338816K), 0.0183032 secs]
    1233.017: [GC 277370K->193641K(338880K), 0.0131341 secs]
    1244.191: [GC 274857K->194742K(339008K), 0.0112069 secs]
    1250.311: [GC 276399K->193065K(339200K), 0.0254185 secs]
    1253.819: [GC 274729K->194369K(339840K), 0.0129195 secs]
    1256.108: [GC 276737K->195947K(339904K), 0.0238248 secs]
    1267.525: [GC 278315K->195033K(340352K), 0.0569615 secs]
    1271.692: [GC 278425K->196492K(340480K), 0.0190379 secs]
    1277.693: [GC 279852K->197949K(340480K), 0.0191917 secs]
    1280.238: [GC 281405K->196790K(340864K), 0.0157921 secs]
    1285.199: [GC 280246K->197847K(340608K), 0.0181497 secs]
    1286.007: [GC 281175K->196440K(337472K), 0.0111939 secs]
    1304.891: [GC 279768K->197845K(340608K), 0.0182715 secs]
    1315.601: [GC 281237K->197856K(340736K), 0.0167214 secs]
    1316.421: [GC 281248K->198430K(340608K), 0.0246224 secs]
    1328.348: [GC 281758K->198595K(339136K), 0.0165731 secs]
    1349.267: [GC 281915K->197864K(340928K), 0.0170268 secs]
    1368.354: [GC 281192K->199896K(340288K), 0.0262476 secs]
    1376.770: [GC 283224K->200903K(339136K), 0.0309814 secs]
    1381.040: [GC 282183K->200807K(338496K), 0.0238972 secs]
    1392.801: [GC 282087K->200812K(338880K), 0.0230000 secs]
    1402.978: [GC 280812K->199419K(335424K), 0.0188024 secs]
    1423.983: [GC 279419K->199784K(339264K), 0.0155648 secs]
    1427.064: [GC 280360K->200673K(339072K), 0.0203093 secs]
    1434.942: [GC 281249K->200341K(339584K), 0.0223844 secs]
    1438.108: [GC 281621K->200922K(339584K), 0.0322821 secs]
    1443.355: [GC 282202K->200559K(339840K), 0.0218897 secs]
    1456.235: [GC 282351K->201313K(339712K), 0.0227011 secs]
    1470.330: [GC 283105K->202311K(339840K), 0.0259252 secs]
    1485.071: [GC 284423K->201711K(340032K), 0.0171362 secs]
    1494.809: [GC 283823K->201909K(340160K), 0.0157363 secs]
    1497.843: [GC 284469K->203516K(340224K), 0.0455427 secs]
    1514.800: [GC 286076K->201843K(340224K), 0.0183698 secs]
    1532.011: [GC 284019K->200888K(335296K), 0.0148204 secs]
    1542.673: [GC 283064K->202547K(340032K), 0.0133714 secs]
    1546.169: [GC 284275K->201936K(339968K), 0.0131052 secs]
    1558.229: [GC 283664K->203817K(339840K), 0.0270803 secs]
    1563.074: [GC 285609K->204088K(339904K), 0.0235084 secs]
    1565.882: [GC 285880K->203251K(339968K), 1.4040290 secs]
    1568.277: [GC 285427K->206133K(339904K), 1.4095963 secs]
    1572.106: [GC 288309K->218429K(329664K), 1.3890185 secs]
    1574.922: [GC 289719K->228796K(334784K), 1.5292558 secs]
    1577.851: [GC 300091K->234048K(321472K), 0.0349367 secs]
    1578.641: [GC 292032K->235261K(322496K), 1.4253011 secs]
    1580.562: [GC 293231K->236958K(325248K), 0.0272196 secs]
    1581.905: [GC 291230K->242541K(325376K), 1.4795405 secs]
    1584.465: [GC 296813K->242311K(318848K), 0.0404727 secs]
    1586.335: [GC 288263K->244486K(322112K), 0.0371221 secs]
    1587.556: [GC 290426K->247801K(321856K), 0.0331232 secs]
    1587.795: [GC 294007K->243424K(322112K), 0.0236656 secs]
    1588.929: [GC 289632K->247212K(321472K), 0.0233302 secs]
    1590.843: [GC 294188K->251257K(321792K), 1.5163833 secs]
    1594.496: [GC 298233K->251601K(323072K), 0.0251024 secs]
    1594.521: [Full GC 251601K->229607K(373056K), 1.2681137 secs]
    1597.865: [GC 278951K->240314K(372800K), 0.0389260 secs]
    1599.989: [GC 289658K->242870K(373952K), 0.0228907 secs]
    1604.417: [GC 294768K->242335K(373760K), 0.0189292 secs]
    1606.399: [GC 294239K->238333K(375168K), 0.0103706 secs]
    1609.289: [GC 292093K->237476K(352576K), 0.0084749 secs]
    1614.643: [GC 291229K->238324K(375552K), 0.0109861 secs]
    1618.865: [GC 291316K->242479K(356800K), 0.0357355 secs]
    1619.215: [GC 295471K->243649K(375872K), 0.0218387 secs]
    1621.325: [GC 297793K->241168K(374784K), 0.0128466 secs]
    1627.656: [GC 295312K->240383K(376512K), 0.0130406 secs]
    1633.452: [GC 296831K->241343K(375936K), 0.1381361 secs]
    1639.291: [GC 297731K->240512K(377664K), 0.0167985 secs]
    1641.070: [GC 299836K->241890K(377408K), 0.0126484 secs]
    1643.135: [GC 301218K->242024K(379200K), 0.0830174 secs]
    1657.391: [GC 304808K->242766K(379008K), 0.0230685 secs]
    1660.963: [GC 305550K->241718K(379200K), 0.0121118 secs]
    1672.385: [GC 307247K->242373K(380032K), 0.0148897 secs]
    1679.189: [GC 307909K->242663K(379456K), 0.0336012 secs]
    1679.806: [GC 310311K->242331K(380544K), 0.0137211 secs]
    1698.018: [GC 309972K->242272K(380288K), 0.0129662 secs]
    1707.166: [GC 312209K->241933K(381504K), 0.0170909 secs]
    1710.486: [GC 311885K->242797K(381568K), 0.0167249 secs]
    1713.338: [GC 314555K->243384K(382272K), 0.0192025 secs]
    1717.070: [GC 315192K->242810K(382144K), 0.0123115 secs]
    1724.488: [GC 316282K->244688K(382912K), 0.0156554 secs]
    1731.274: [GC 318160K->241808K(383744K), 0.0144968 secs]
    1737.801: [GC 316560K->243104K(384064K), 0.0121330 secs]
    1739.864: [GC 317856K->245407K(384896K), 0.0341079 secs]
    1743.852: [GC 321311K->244312K(384896K), 0.0141990 secs]
    1746.971: [GC 320216K->244438K(385344K), 0.0183935 secs]
    1766.204: [GC 322134K->243594K(385792K), 0.0132516 secs]
    1770.080: [GC 321290K->243829K(386240K), 0.0130751 secs]
    1784.406: [GC 322869K->244602K(386624K), 0.0156597 secs]
    1796.876: [GC 323597K->243561K(386816K), 0.0148723 secs]
    1798.332: [GC 323561K->245704K(387136K), 0.0154095 secs]
    1800.109: [GC 325704K->247336K(388032K), 0.0409818 secs]
    1804.904: [GC 328168K->245655K(388736K), 0.0166685 secs]
    1806.600: [GC 326487K->245580K(389760K), 0.0219245 secs]
    1818.889: [GC 328204K->244470K(390144K), 0.0157939 secs]
    1822.358: [GC 327094K->245905K(390336K), 0.0175462 secs]
    1827.555: [GC 329224K->246088K(390592K), 0.0208042 secs]
    1837.514: [GC 329416K->245605K(390336K), 0.0137855 secs]
    1847.001: [GC 329445K->246131K(390720K), 0.0146275 secs]
    1848.864: [GC 329971K->246494K(391296K), 0.0175766 secs]
    1859.203: [GC 331036K->246937K(391360K), 0.0146327 secs]
    1860.966: [GC 331481K->248414K(390208K), 0.0216726 secs]
    1864.134: [GC 331742K->247451K(389056K), 0.0162971 secs]
    1873.550: [GC 330779K->248640K(389760K), 0.0218835 secs]
    1876.250: [GC 330803K->248184K(388224K), 0.0208859 secs]
    1880.577: [GC 330360K->247325K(390336K), 0.0147365 secs]
    1892.113: [GC 329565K->247859K(387456K), 0.0144426 secs]
    1900.820: [GC 330099K->248412K(389952K), 0.0195921 secs]
    1903.069: [GC 330716K->247796K(389888K), 0.0155545 secs]
    1910.944: [GC 330100K->248981K(390336K), 0.0165833 secs]
    1912.105: [GC 331910K->253094K(390400K), 0.0212416 secs]
    1912.393: [GC 336038K->252108K(388096K), 0.0182545 secs]
    1917.882: [GC 332364K->252928K(387456K), 0.0226416 secs]
    1920.975: [GC 333184K->255435K(387264K), 0.0831686 secs]
    1921.451: [GC 333515K->253724K(388160K), 0.0265402 secs]
    1930.344: [GC 331804K->253979K(387904K), 0.0208346 secs]
    1934.661: [GC 332059K->253520K(388032K), 0.0179034 secs]
    1941.664: [GC 331600K->254297K(388096K), 0.0206944 secs]
    1947.160: [GC 332761K->253651K(388096K), 0.0194694 secs]
    1948.391: [GC 332115K->254824K(388288K), 0.0213165 secs]
    1955.883: [GC 333798K->252957K(388288K), 0.0172918 secs]
    1962.014: [GC 331933K->254893K(388864K), 0.0256110 secs]
    1981.902: [GC 334509K->256567K(388352K), 0.0445018 secs]
    1992.276: [GC 336183K->254832K(388736K), 0.0185104 secs]
    1993.072: [GC 333808K->253054K(388480K), 0.0171761 secs]
    2006.441: [GC 332030K->254222K(388160K), 0.0331208 secs]
    2012.620: [GC 332686K->254008K(381568K), 0.0121851 secs]
    2029.610: [GC 332472K->254865K(388224K), 0.0141391 secs]
    2033.518: [GC 333009K->254448K(388160K), 0.0171172 secs]
    2037.370: [GC 332592K->254761K(388032K), 0.0137820 secs]
    2042.150: [GC 332962K->257542K(388096K), 0.0195266 secs]
    2048.972: [GC 335750K->256642K(387520K), 0.0175746 secs]
    2054.523: [GC 335234K->255301K(387904K), 0.0131941 secs]
    2057.202: [GC 333884K->255117K(388800K), 0.0174713 secs]
    2059.506: [GC 334861K->255720K(388864K), 0.0172076 secs]
    2067.436: [GC 335464K->256201K(388800K), 0.0154266 secs]
    2075.384: [GC 336457K->255145K(388864K), 0.0134541 secs]
    2077.148: [GC 335401K->255820K(389312K), 0.0158148 secs]
    2084.353: [GC 336972K->255983K(389440K), 0.0134647 secs]
    2093.116: [GC 337135K->256751K(389440K), 0.0155239 secs]
    2102.787: [GC 338658K->258431K(389632K), 0.0215284 secs]
    2104.462: [GC 340351K->256945K(390144K), 0.0153015 secs]
    2109.728: [GC 339505K->257115K(390208K), 0.0159563 secs]
    2122.379: [GC 339675K->256748K(390400K), 0.0161834 secs]
    2133.977: [GC 339948K->257442K(390400K), 0.0158151 secs]
    2139.718: [GC 340642K->257842K(390912K), 0.0182630 secs]
    2143.532: [GC 341810K->255648K(390784K), 0.0156949 secs]
    2156.980: [GC 339610K->256127K(391296K), 0.0232994 secs]
    2162.709: [GC 340479K->259011K(389952K), 0.0216918 secs]
    2165.392: [GC 343363K->257891K(391296K), 0.0156548 secs]
    2172.867: [GC 341987K->256830K(387328K), 0.1029279 secs]
    2175.829: [GC 340926K->258830K(390848K), 0.0175429 secs]
    2193.642: [GC 342860K->257384K(390912K), 0.0158681 secs]
    2198.592: [GC 341416K->257722K(391040K), 0.0145218 secs]
    2216.548: [GC 342330K->258612K(391168K), 0.0147463 secs]
    2218.664: [GC 343218K->258786K(391360K), 0.0167079 secs]
    2223.931: [GC 343970K->261044K(391552K), 0.0325592 secs]
    2229.163: [GC 346228K->260589K(390656K), 0.0185048 secs]
    2233.913: [GC 344557K->259905K(391104K), 0.0178199 secs]
    2237.120: [GC 343873K->259165K(390720K), 0.0131946 secs]
    2241.975: [GC 343261K->259505K(390976K), 0.0162827 secs]
    2250.736: [GC 343601K->259841K(390912K), 0.0159241 secs]
    2259.976: [GC 344513K->260319K(391168K), 0.0205832 secs]
    2265.025: [GC 344956K->259597K(391488K), 0.0152182 secs]
    2266.097: [GC 344973K->259896K(391680K), 0.0151583 secs]
    2283.470: [GC 345272K->261445K(391808K), 0.0237256 secs]
    2284.093: [GC 347077K->260412K(391936K), 0.0237551 secs]
    2301.941: [GC 346044K->261961K(391616K), 0.0202204 secs]
    2305.248: [GC 347337K->261329K(391808K), 0.0165993 secs]
    2331.961: [GC 346705K->262034K(391488K), 0.0184896 secs]
    2340.389: [GC 347154K->261811K(391680K), 0.0146796 secs]
    2345.010: [GC 346931K->261477K(391360K), 0.0261079 secs]
    2362.483: [GC 346725K->261484K(391552K), 0.0150812 secs]
    2366.469: [GC 346732K->261782K(391616K), 0.0176710 secs]
    2381.854: [GC 347478K->262227K(391808K), 0.0178963 secs]
    2405.250: [GC 347923K->263473K(391552K), 0.0290353 secs]
    2407.033: [GC 348977K->262593K(391872K), 0.0134900 secs]
    2410.744: [GC 348097K->262613K(391680K), 0.0152754 secs]
    2430.766: [GC 348181K->262504K(391808K), 0.0168945 secs]
    2445.377: [GC 348072K->262613K(391872K), 0.0150553 secs]
    2457.907: [GC 348373K->261561K(391936K), 0.0132542 secs]
    2464.252: [GC 347321K->262665K(391936K), 0.0156359 secs]
    2466.159: [GC 348425K->263117K(391936K), 0.0366962 secs]
    2501.045: [GC 348877K->262979K(391936K), 0.0179926 secs]
    2515.249: [GC 348995K->262886K(391936K), 0.0150738 secs]
    2526.311: [GC 348902K->264968K(391488K), 0.0330411 secs]
    2529.752: [GC 350582K->263343K(391936K), 0.0154606 secs]
    2539.265: [GC 348975K->263855K(391552K), 0.0177899 secs]
    2543.927: [GC 349614K->264173K(391808K), 0.0240386 secs]
    2549.568: [GC 349933K->263423K(392000K), 0.0161444 secs]
    2556.347: [GC 349628K->264100K(392128K), 0.0160603 secs]
    2561.802: [GC 350308K->263122K(392256K), 0.0182325 secs]
    2566.114: [GC 349586K->263404K(392320K), 0.0137574 secs]
    2586.702: [GC 349853K->266723K(390848K), 0.0274725 secs]
    2589.854: [GC 351715K->266060K(391616K), 0.0168321 secs]
    2591.739: [GC 351052K->265623K(391232K), 0.0195907 secs]
    2607.384: [GC 350743K->265415K(391488K), 0.0174390 secs]
    2618.555: [GC 350535K->265563K(391616K), 0.0167459 secs]
    2637.109: [GC 351195K->266551K(391808K), 0.0148909 secs]
    2642.146: [GC 352183K->266672K(392000K), 0.0180908 secs]
    2643.168: [GC 352624K->266330K(392064K), 0.0147989 secs]
    2647.188: [GC 352282K->267880K(391616K), 0.0173880 secs]
    2655.985: [GC 353512K->267766K(391296K), 0.0149468 secs]
    2656.891: [GC 353398K->266275K(391744K), 0.0144024 secs]
    2664.408: [GC 351444K->266017K(388928K), 0.0209675 secs]
    2668.686: [GC 351201K->266578K(391680K), 0.0153836 secs]
    2686.271: [GC 351826K->266792K(391616K), 0.0148674 secs]
    2687.701: [GC 352040K->266975K(391744K), 0.0195823 secs]
    2689.757: [GC 352543K->268009K(391680K), 0.0159841 secs]
    2703.374: [GC 353573K->267804K(392000K), 0.0164064 secs]
    2707.609: [GC 353867K->267629K(391936K), 0.0251624 secs]
    2712.220: [GC 353709K->265927K(391936K), 0.0125782 secs]
    2737.914: [GC 352071K->268752K(391168K), 0.0185614 secs]
    2750.502: [GC 354890K->266908K(392064K), 0.0142372 secs]
    2760.888: [GC 352796K->268633K(392064K), 0.0171518 secs]
    2766.927: [GC 354521K->268366K(391808K), 0.0181752 secs]
    2768.030: [GC 354254K->269350K(391936K), 0.0206453 secs]
    2771.794: [GC 355238K->268665K(392064K), 0.0168314 secs]
    2809.208: [GC 354553K->270422K(391872K), 0.0188068 secs]
    2809.559: [GC 356294K->268022K(391616K), 0.0139145 secs]
    2819.650: [GC 353270K->268398K(391744K), 0.0131524 secs]
    2824.293: [GC 353646K->268658K(391424K), 0.0141748 secs]
    2826.554: [GC 353962K->268920K(391616K), 0.0155995 secs]
    2828.896: [GC 354232K->270468K(391488K), 0.0282409 secs]
    2843.234: [GC 355844K->270455K(390848K), 0.0208270 secs]
    2861.999: [GC 355831K->269224K(391744K), 0.1919214 secs]
    2870.048: [GC 354344K->270798K(390720K), 0.0189311 secs]
    2882.122: [GC 355918K->269501K(391680K), 0.0173705 secs]
    2889.172: [GC 354493K->271061K(390656K), 0.0332939 secs]
    2894.716: [GC 356053K->270154K(391680K), 0.0169549 secs]
    2896.430: [GC 355082K->271813K(391104K), 0.0194223 secs]
    2896.842: [GC 356741K->272049K(390656K), 0.0313532 secs]
    2901.564: [GC 355761K->272403K(390272K), 0.0189123 secs]
    2908.595: [GC 356115K->272029K(390336K), 0.0179135 secs]
    2914.717: [GC 354717K->271988K(390464K), 0.0146900 secs]
    2923.602: [GC 354672K->271800K(390208K), 0.0173560 secs]
    2939.749: [GC 3

    Hi! Thanks for the reply,
    After much googling we've finally found a stable configuration.
    java.args=-server -Xmx768m -Xss64k -Dsun.io.useCanonCaches=false -XX:MaxPermSize=512m -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/ -verbose:gc -Xloggc:c:/Jrun4/logs/gc/gcInstance1b.log
    I tried playing with the survivor ratio but came back to the default when I didn't notice much improvement.  In the end what did it (pretty sure) is -Xss to reduce the requests size living on the stack.  Our large initial heap, perm + overhead combined with the default request size was causing the JVM to go over and blow up I think.
    We've been stable for over a month now, the next step for us is to reduce the MaxPermSize a little at a time to find our optimal range.

  • Server crashes running CFMX7 and IIS

    The sequence of events is that the server runs fine....for a
    short while, then either CF services or all the Web services become
    unavailalbe. Sometimes the whole machine goes down, and must be
    rebooted. Sometimes if the machine does not go down, restarting CF
    services or the Web services IIS solves the issue and the site
    becomes available againl.
    There are no server logs pointing to any hardware or Windows
    operating system system problems. Running Windows 2003 Server and
    IIS 6 with CFMX7.
    Sometimes when the server goes down, I get a Jrun connection
    error. But I really think that the server is just getting
    overloaded with these erroneous 404 errors that are showing up
    every couple of minutes/seconds in the application log.
    These are 404 errors for pages that have been off the Website
    for years, in some cases. It feels like a Web crawler is going
    through the site looking for old erroneous pages and not finding
    them.
    I have built a completely new server, and now have twice the
    hard-drive space and twice the RAM, but it is beginning to crash
    just like the previous server. And the application log is filling
    up, just like the prevous server.
    We have re-assigned IP addresses, so this new server now has
    the same IP address that the old one had, and even though it has a
    new name, all references to the old name are being forwarded to the
    new server....So our problem has followed us to the new machine.
    Iiiiiiiieeeeeee. I am in the twlight zone, and can't get out.
    Does anyone have any idea how I can tell where these 404 errors
    might be coming from? How do I get a referrer on a log file error?
    Or any other ideas at all. I am at my wits end, and if I
    don't solve this problem am in serious trouble here. WCW

    Willow315 wrote:
    > The sequence of events is that the server runs
    fine....for a short while, then
    > either CF services or all the Web services become
    unavailalbe. Sometimes the
    > whole machine goes down, and must be rebooted.
    Goes down? Do you mean 100% CPU and unresponsive or do you
    mean it
    completely locks up and doesn't even answer pings anymore?
    > Sometimes when the server goes down, I get a Jrun
    connection error. But I
    > really think that the server is just getting overloaded
    with these erroneous
    > 404 errors that are showing up every couple of
    minutes/seconds in the
    > application log.
    What is the first error you see in your *-out.log? Is JRun
    maximizing
    the allocated RAM when it goes down?
    > Iiiiiiiieeeeeee. I am in the twlight zone, and can't get
    out. Does anyone have
    > any idea how I can tell where these 404 errors might be
    coming from? How do I
    > get a referrer on a log file error?
    In your IIS logging settings, enable referrer logging. And
    you might
    check if there is a "Check that file exists" checkbox in IIS
    where you
    mapped the .cfm extension to CF.
    Jochem
    Jochem van Dieten
    Adobe Community Expert for ColdFusion

  • JRun Thread Pool Issue

    I'm running CF 9.0.1 on Ubuntu on an "Medium" Amazon EC2 instance. CF has been crashing intermittently (several times per day). At such times, running top gets me this (or something similar):
    PID
    USER
    PR
    NI
    VIRT
    RES
    SHR
    S
    %CPU
    %MEM
    TIME+COMMAND                                                                                                   
    15855
    wwwrun
    20
    0
    1762m
    730m
    20m
    S
    99.3
    19.4
    13:22.96 coldfusion9
    So, it's obviously consuming most of the server resources. The following error has been showing up in my cfserver.log in the leadup to each crash:
    java.lang.RuntimeException: Request timed out waiting for an available thread to run. You may want to consider increasing the number of active threads in the thread pool.
    If I run /opt/coldfusion9/bin/coldfusion status, I get:
    Pg/Sec  DB/Sec  CP/Sec  Reqs  Reqs  Reqs  AvgQ   AvgReq AvgDB  Bytes  Bytes
    Now Hi  Now Hi  Now Hi  Q'ed  Run'g TO'ed Time   Time   Time   In/Sec Out/Sec
    0   0   0   0   -1  -1  150   25    0     0      -1352560      0      0
    In the administrator, under Server Settings > Request Tuning, the setting for Maximum number of simultaneous Template requests is 25. So this makes sense so far. I could just increase the thread pool to cover these sort of load spikes. I could make it 200. (Which I did just now as a test.)
    However, there's also this file /opt/coldfusion9/runtime/servers/coldfusion/SERVER-INF/jrun.xml. And some of the settings in there appear to conflict. For example, it reads:
    <service class="jrunx.scheduler.SchedulerService" name="SchedulerService">
      <attribute name="bindToJNDI">true</attribute>
      <attribute name="activeHandlerThreads">25</attribute>
      <attribute name="maxHandlerThreads">1000</attribute>
      <attribute name="minHandlerThreads">20</attribute>
      <attribute name="threadWaitTimeout">180</attribute>
      <attribute name="timeout">600</attribute>
    </service>
    Which a) has fewer active threads (what does this mean?), and b) has a max threads that exceed the simultaneous request limit set in the admin. So, I'm not sure. Are these independent configs that need to be made to match manually? Or is the jrun.xml file supposed to be written by the CF Admin when changes are made there? Hmm. But maybe this is different because presumably the CF Scheduler should only use a subset of all available threads, right...so we'd always have some threads for real live users. We also have this in there:
    <service class="jrun.servlet.http.WebService" name="WebService">
      <attribute name="port">8500</attribute>
      <attribute name="interface">*</attribute>
      <attribute name="deactivated">true</attribute>
      <attribute name="activeHandlerThreads">200</attribute>
      <attribute name="minHandlerThreads">1</attribute>
      <attribute name="maxHandlerThreads">1000</attribute>
      <attribute name="mapCheck">0</attribute>
      <attribute name="threadWaitTimeout">300</attribute>
      <attribute name="backlog">500</attribute>
      <attribute name="timeout">300</attribute>
    </service>
    This appears to have changed when I changed the CF Admin setting...maybe...but it's the activeHandlerThreads that matches my new maximum simulataneous requests setting...rather than the maxHandlerThreads, which again exceeds it. Finally, we have this:
    <service class="jrun.servlet.jrpp.JRunProxyService" name="ProxyService">
      <attribute name="activeHandlerThreads">200</attribute>
      <attribute name="minHandlerThreads">1</attribute>
      <attribute name="maxHandlerThreads">1000</attribute>
      <attribute name="mapCheck">0</attribute>
      <attribute name="threadWaitTimeout">300</attribute>
      <attribute name="backlog">500</attribute>
      <attribute name="deactivated">false</attribute>
      <attribute name="interface">*</attribute>
      <attribute name="port">51800</attribute>
      <attribute name="timeout">300</attribute>
      <attribute name="cacheRealPath">true</attribute>
    </service>
    So, I'm not certain which (if any) of these I should change and what exactly the relationship is between maximum requests and maximum threads. Also, since several of these list the maxHandlerThreads as 1000, I'm wondering if I should just set the maximum simultaneous requests to 1000. There must be some upper limit that depends on available server resources...but I'm not sure what it is and I don't really want to play around with it since it's a production environment.
    I'm not sure if it pertains to this issue at all, but when I run a ps aux | grep coldfusion I get the following:
    wwwrun   15853  0.0  0.0   8704   760 pts/1
    S
    20:22   0:00 /opt/coldfusion9/runtime/bin/coldfusion9 -jar jrun.jar -autorestart -start coldfusion
    wwwrun   15855  5.4 18.2 1678552 701932 pts/1  
    Sl
    20:22   1:38 /opt/coldfusion9/runtime/bin/coldfusion9 -jar jrun.jar -start coldfusion
    There are always these two and never more than these two processes. So there does not appear to be a one-to-one relationship between processes and threads. I recall from an MX 6.1 install I maintained for many years that additional CF processes were visible in the process list. It seemed to me at the time like I had a process for each thread...so either I was wrong or something is quite different in version 9 since it's reporting 25 running requests and only showing these two processes. If a single process can have multiple threads in the background, then I'm given to wonder why I have two processes instead of one...just curious.
    So, anyway, I've been experimenting while composing this post. As noted above I adjusted the maximum simulataneous requests up to 200. I was hoping this would solve my problem, but CF just crashed again (rather it slogged down and requests started timing out...so effectively "crashed"). This time, top looked similar (still consuming more than 99% of the CPU), but CF status looked different:
    Pg/Sec  DB/Sec  CP/Sec  Reqs  Reqs  Reqs  AvgQ   AvgReq AvgDB  Bytes  Bytes
    Now Hi  Now Hi  Now Hi  Q'ed  Run'g TO'ed Time   Time   Time   In/Sec Out/Sec
    0   0   0   0   -1  -1  0     150   0     0      0      0      0      0
    Obviously, since I'd increased the maximum simultaneous requests, it was allowing more requests to run simultaneously...but it was still maxing out the server resources.
    Further experiments (after restarting CF) showed me that the server became unusably slogged after about 30-35 "Reqs Run'g", with all additional requests headed for an inevitible timeout:
    Pg/Sec  DB/Sec  CP/Sec  Reqs  Reqs  Reqs  AvgQ   AvgReq AvgDB  Bytes  Bytes
    Now Hi  Now Hi  Now Hi  Q'ed  Run'g TO'ed Time   Time   Time   In/Sec Out/Sec
    0   0   0   0   -1  -1  0     33    0     0      -492   0      0      0
    So, it's clear that increasing the maximum simultaneous requests has not helped. I guess what it comes down to is this: What is it having such a hard time with? Where are these spikes coming from? Bursts of traffic? On what pages? What requests are running at any given time? I guess I simply need more information to continue troubleshooting. If there are long-running requests, or other issues, I'm not seeing it in the logs (although I do have that option checked in the admin). I need to know which requests exactly are those responsible for these spikes. Any help would be much appreciated. Thanks.
    ~Day

    I really appreciate your help. However, I haven't been able to find the JRun Thread settings you describe above.
    Under Request Tuning, I see:
    Server Settings > Request Tuning
    Request Limits
    Maximum number of simultaneous Template requests
      Restricts the number of simultaneously processed requests. Use this setting to increase overall system performance for heavy load applications. Requests beyond the specified limit are queued. On Standard Edition, you must restart ColdFusion to enable this setting. 
    Maximum number of simultaneous Flash Remoting requests
      The number of Flash Remoting requests that can be processed concurrently.
    Maximum number of simultaneous Web Service requests
      The number of Web Service requests that can be processed concurrently.
    Maximum number of simultaneous CFC function requests
      The number of ColdFusion Component methods that can be processed concurrently via HTTP. This does not affect invocation of CFC methods from within CFML, only methods requested via an HTTP request.
    Tag Limit Settings
    Maximum number of simultaneous Report threads
      The maximum number of ColdFusion reports that can be processed concurrently.
    Maximum number of threads available for CFTHREAD
      The maximum number of threads created by CFTHREAD that will be run concurrently. Threads created by CFTHREAD in excess of this are queued.  On Standard Edition, the maximum limit is 10. 
    And under Java and JVM, I see:
    Server Settings > Java and JVM
        Java and JVM settings control the way ColdFusion starts the Java Virtual Machine when it starts.  You can control settings like what classpaths are used and how memory is allocated as well as add custom command line arguments.  Changing these settings requires restarting ColdFusion.  If you enter an incorrect setting, ColdFusion may not restart properly. 
       Backups of the jvm.config file are created when you hit the submit button. You can use this backup to restore from a critical change. 
       Java Virtual Machine Path
      Specifies the location of the Java Virtual Machine.
       Minimum JVM Heap Size (MB)         Maximum JVM Heap Size  (MB)       
       The Memory Size settings determine the amount of memory that the JVM can use for programs and data. 
       ColdFusion Class Path
      Specifies any additional class paths for the JVM, with multiple directories separated by  commas.
       JVM Arguments
      -server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib
      Specifies any specific JVM initialization options, separated by spaces.
    I did go take a look at FusionReactor and found it's not free (which would be fine, of course, if it would actually help). It looks like there's a fully functional demo, which is cool...but I've haven't been able to get it to install yet, so we'll see.
    Thanks again!
    ~Day
    (By the way, I've cross-posted this inquiry on StackOverflow. So if you're able to help me arrive at a solution you might want to answer there as well.)

  • Recurring JVM Crash related to jai.RasterAccessor?

    We are having an intermittent problem with our JVM crashing and CF restarting on an almost daily basis. We are running CF                                      8,0,1,195765 in a Windows environment with JDK 1.6.0_04. The dump is below. Every time that this occurs, the "current compiletask" is the same (as listed below). Our JVM settings are also as follows with 1GB of Heap Space allocated:
    -server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:PermSize=128m -XX:NewSize=256m -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:SurvivorRatio=4 -XX:+UseParNewGC -XX:+HeapDumpOnOutOfMemoryError -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib
    I'm not sure if upgrading our JDK will help solve this problem but I'm thinking that there is some code somewhere that is calling the javax.media.jai.RasterAccessor java method and that is the root of this problem. I do not know what code might be calling this Java method but I was thinking that it might be related to CFDocument or CFImage. However, our use of these tags is pretty minimal.
    Here is the dump:
    # An unexpected error has been detected by Java Runtime Environment:
    # java.lang.OutOfMemoryError: requested 67108872 bytes for Chunk::new. Out of swap space?
    #  Internal Error (allocation.cpp:218), pid=6044, tid=4680
    #  Error: Chunk::new
    # Java VM: Java HotSpot(TM) Server VM (10.0-b19 mixed mode windows-x86)
    # If you would like to submit a bug report, please visit:
    #   http://java.sun.com/webapps/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    ---------------  T H R E A D  ---------------
    Current thread (0x52624000):  JavaThread "CompilerThread0" daemon [_thread_in_native, id=4680, stack(0x52d40000,0x52e40000)]
    Stack: [0x52d40000,0x52e40000]
    [error occurred during error reporting (printing stack bounds), id 0x80000001]
    Current CompileTask:
    C2:1459      javax.media.jai.RasterAccessor.<init>(Ljava/awt/image/Raster;Ljava/awt/Rectangle;Ljavax/m edia/jai/RasterFormatTag;Ljava/awt/image/ColorModel;)V (3287 bytes)
    ---------------  P R O C E S S  ---------------
    Java Threads: ( => current thread )
      0x533fe800 JavaThread "jrpp-199" [_thread_in_native, id=3200, stack(0x5cec0000,0x5cfc0000)]
      0x53270400 JavaThread "jrpp-198" [_thread_in_native, id=4460, stack(0x5a6a0000,0x5a7a0000)]
      0x53270800 JavaThread "jrpp-197" [_thread_in_native, id=6048, stack(0x5f160000,0x5f260000)]
      0x533cb800 JavaThread "jrpp-191" [_thread_in_native, id=1136, stack(0x5ada0000,0x5aea0000)]
      0x593cbc00 JavaThread "jrpp-189" [_thread_in_native, id=3960, stack(0x5fee0000,0x5ffe0000)]
      0x569f4400 JavaThread "jrpp-185" [_thread_in_native, id=4180, stack(0x67960000,0x67a60000)]
      0x56f52800 JavaThread "FusionReactor FD/R Remote Data Collector for {coldfusion.cfmx8.web1}" [_thread_blocked, id=4824, stack(0x5aba0000,0x5aca0000)]
      0x56f52400 JavaThread "FusionReactor Enterprise Service - Local Metrics Cache" [_thread_blocked, id=6076, stack(0x53650000,0x53750000)]
      0x59815800 JavaThread "Timer-16" [_thread_blocked, id=4684, stack(0x67160000,0x67260000)]
      0x53e75800 JavaThread "Timer-15" [_thread_blocked, id=2416, stack(0x66e60000,0x66f60000)]
      0x602dec00 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-98)" [_thread_in_native, id=1644, stack(0x67060000,0x67160000)]
      0x602de800 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-97)" [_thread_blocked, id=4172, stack(0x66f60000,0x67060000)]
      0x610e8400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-96)" [_thread_blocked, id=3148, stack(0x66d60000,0x66e60000)]
      0x5a364400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-95)" [_thread_in_native, id=2252, stack(0x66c60000,0x66d60000)]
      0x61676400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-94)" [_thread_blocked, id=5792, stack(0x66b60000,0x66c60000)]
      0x611e0400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-93)" [_thread_in_native, id=3492, stack(0x66a60000,0x66b60000)]
      0x56d12800 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-92)" [_thread_in_native, id=5244, stack(0x66960000,0x66a60000)]
      0x5875d400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-91)" [_thread_blocked, id=4364, stack(0x66860000,0x66960000)]
      0x60a0a400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-90)" [_thread_blocked, id=976, stack(0x5fae0000,0x5fbe0000)]
      0x595b8400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-89)" [_thread_blocked, id=4944, stack(0x5d5c0000,0x5d6c0000)]
      0x60775800 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-88)" [_thread_in_native, id=5832, stack(0x5fde0000,0x5fee0000)]
      0x6077d400 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-87)" [_thread_blocked, id=3384, stack(0x5fce0000,0x5fde0000)]
      0x53193800 JavaThread "FusionReactor Web Server (Server Thread Pool Member Thread-86)" [_thread_blocked, id=2440, stack(0x5fbe0000,0x5fce0000)]
      0x531cc800 JavaThread "Timer-10" [_thread_blocked, id=5060, stack(0x5d4c0000,0x5d5c0000)]
      0x58e8f000 JavaThread "AWT-Windows" daemon [_thread_in_native, id=4864, stack(0x5e0d0000,0x5e1d0000)]
      0x584c8c00 JavaThread "scheduler-3" [_thread_blocked, id=2092, stack(0x5d3c0000,0x5d4c0000)]
      0x595f5400 JavaThread "scheduler-21" [_thread_blocked, id=3396, stack(0x57cb0000,0x57db0000)]
      0x53c3c800 JavaThread "scheduler-2" [_thread_blocked, id=744, stack(0x5c2c0000,0x5c3c0000)]
      0x56838800 JavaThread "SocketTimeout" daemon [_thread_blocked, id=3072, stack(0x5cfc0000,0x5d0c0000)]
      0x5921fc00 JavaThread "scheduler-1" [_thread_blocked, id=3476, stack(0x5cdc0000,0x5cec0000)]
      0x569b8800 JavaThread "MulticastListener" [_thread_in_native, id=5332, stack(0x5ccc0000,0x5cdc0000)]
      0x56b81800 JavaThread "MulticastListener" [_thread_in_native, id=5272, stack(0x5cbc0000,0x5ccc0000)]
      0x5273d800 JavaThread "MulticastListener" [_thread_in_native, id=2376, stack(0x5cac0000,0x5cbc0000)]
      0x58a63c00 JavaThread "MulticastListener" [_thread_in_native, id=1128, stack(0x5c9c0000,0x5cac0000)]
      0x56a81800 JavaThread "MulticastListener" [_thread_in_native, id=924, stack(0x5c8c0000,0x5c9c0000)]
      0x53212400 JavaThread "MulticastListener" [_thread_in_native, id=5140, stack(0x5c7c0000,0x5c8c0000)]
      0x5681bc00 JavaThread "MulticastListener" [_thread_in_native, id=4380, stack(0x5c6c0000,0x5c7c0000)]
      0x591ffc00 JavaThread "MulticastListener" [_thread_in_native, id=5420, stack(0x5c5c0000,0x5c6c0000)]
      0x59227c00 JavaThread "MulticastListener" [_thread_in_native, id=5616, stack(0x5c4c0000,0x5c5c0000)]
      0x53a18400 JavaThread "MulticastListener" [_thread_in_native, id=988, stack(0x5c3c0000,0x5c4c0000)]
      0x003f6400 JavaThread "DestroyJavaVM" [_thread_blocked, id=620, stack(0x00030000,0x00130000)]
      0x5702ec00 JavaThread "Timer-9" [_thread_blocked, id=5580, stack(0x5b8a0000,0x5b9a0000)]
      0x585db800 JavaThread "Timer-8" [_thread_blocked, id=2556, stack(0x5b7a0000,0x5b8a0000)]
      0x53241c00 JavaThread "Timer-7" daemon [_thread_blocked, id=3184, stack(0x5b6a0000,0x5b7a0000)]
      0x567eb400 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=460, stack(0x5c1a0000,0x5c2a0000)]
      0x53980400 JavaThread "worker #4" daemon [_thread_blocked, id=5496, stack(0x5c0a0000,0x5c1a0000)]
      0x56e48c00 JavaThread "worker #3" daemon [_thread_blocked, id=2304, stack(0x5bfa0000,0x5c0a0000)]
      0x53a04c00 JavaThread "worker #2" daemon [_thread_blocked, id=4248, stack(0x5bea0000,0x5bfa0000)]
      0x53c63800 JavaThread "worker #1" daemon [_thread_blocked, id=3488, stack(0x5bda0000,0x5bea0000)]
      0x58a64400 JavaThread "worker #0" daemon [_thread_blocked, id=192, stack(0x5bca0000,0x5bda0000)]
      0x56a1d400 JavaThread "ServerThread" daemon [_thread_in_native, id=1920, stack(0x5bba0000,0x5bca0000)]
      0x588b9400 JavaThread "Thread-26" [_thread_blocked, id=4456, stack(0x5baa0000,0x5bba0000)]
      0x56a72400 JavaThread "Thread-22" [_thread_in_native, id=3872, stack(0x5b5a0000,0x5b6a0000)]
      0x53aaf800 JavaThread "obj-skimmer" daemon [_thread_blocked, id=3060, stack(0x5b4a0000,0x5b5a0000)]
      0x534d9400 JavaThread "obj-skimmer" daemon [_thread_blocked, id=3964, stack(0x5b3a0000,0x5b4a0000)]
      0x53a99400 JavaThread "obj-skimmer" daemon [_thread_blocked, id=300, stack(0x5b2a0000,0x5b3a0000)]
      0x53492400 JavaThread "obj-skimmer" daemon [_thread_blocked, id=2312, stack(0x5b1a0000,0x5b2a0000)]
      0x538fb800 JavaThread "obj-skimmer" daemon [_thread_blocked, id=3400, stack(0x5b0a0000,0x5b1a0000)]
      0x5267b400 JavaThread "obj-skimmer" daemon [_thread_blocked, id=5944, stack(0x5afa0000,0x5b0a0000)]
      0x56a72c00 JavaThread "scheduler-0" [_thread_blocked, id=2380, stack(0x5aea0000,0x5afa0000)]
      0x53285c00 JavaThread "FusionReactor Web Server" [_thread_in_native, id=3332, stack(0x5aca0000,0x5ada0000)]
      0x567b5800 JavaThread "Timer-6" daemon [_thread_blocked, id=1876, stack(0x5aaa0000,0x5aba0000)]
      0x56991000 JavaThread "Timer-5" [_thread_blocked, id=5424, stack(0x5a9a0000,0x5aaa0000)]
      0x56993c00 JavaThread "Timer-4" [_thread_blocked, id=332, stack(0x5a8a0000,0x5a9a0000)]
      0x53f2a400 JavaThread "Thread-13" [_thread_blocked, id=2008, stack(0x5a4a0000,0x5a5a0000)]
      0x56958800 JavaThread "Timer-3" daemon [_thread_blocked, id=5648, stack(0x58320000,0x58420000)]
      0x53da6000 JavaThread "Timer-2" daemon [_thread_blocked, id=4076, stack(0x57db0000,0x57eb0000)]
      0x5765d800 JavaThread "Timer-1" [_thread_blocked, id=4140, stack(0x57850000,0x57950000)]
      0x53b10000 JavaThread "Timer-0" [_thread_blocked, id=3748, stack(0x57750000,0x57850000)]
      0x5378ac00 JavaThread "SeeFusion HTTP Listener" daemon [_thread_in_native, id=2120, stack(0x58220000,0x58320000)]
      0x537d7400 JavaThread "SeeFusion Request Watcher" [_thread_blocked, id=3692, stack(0x58120000,0x58220000)]
      0x57542400 JavaThread "SeeFusion History Sleepythread" daemon [_thread_blocked, id=5416, stack(0x58020000,0x58120000)]
      0x5755a800 JavaThread "SeeFusion File Watcher - seefusion.properties" daemon [_thread_blocked, id=4404, stack(0x57f20000,0x58020000)]
      0x53b11000 JavaThread "scheduler-20" [_thread_blocked, id=4196, stack(0x57bb0000,0x57cb0000)]
      0x539c8400 JavaThread "mipc-1" [_thread_in_native, id=5096, stack(0x57ab0000,0x57bb0000)]
      0x53c07400 JavaThread "Thread-4" [_thread_blocked, id=4932, stack(0x579b0000,0x57ab0000)]
      0x53537400 JavaThread "scheduler-19" [_thread_blocked, id=5828, stack(0x56650000,0x56750000)]
      0x53536400 JavaThread "scheduler-18" [_thread_blocked, id=796, stack(0x56550000,0x56650000)]
      0x53535000 JavaThread "scheduler-17" [_thread_blocked, id=2452, stack(0x56450000,0x56550000)]
      0x53534000 JavaThread "scheduler-16" [_thread_blocked, id=5872, stack(0x56350000,0x56450000)]
      0x53533400 JavaThread "scheduler-15" [_thread_blocked, id=3800, stack(0x56250000,0x56350000)]
      0x53532000 JavaThread "scheduler-14" [_thread_blocked, id=5772, stack(0x56150000,0x56250000)]
      0x53531000 JavaThread "scheduler-13" [_thread_blocked, id=3828, stack(0x56050000,0x56150000)]
      0x53530000 JavaThread "scheduler-12" [_thread_blocked, id=5336, stack(0x55f50000,0x56050000)]
      0x5352f000 JavaThread "scheduler-11" [_thread_blocked, id=5112, stack(0x55e50000,0x55f50000)]
      0x5352e000 JavaThread "scheduler-10" [_thread_blocked, id=4988, stack(0x55d50000,0x55e50000)]
      0x5352d400 JavaThread "scheduler-9" [_thread_blocked, id=2472, stack(0x55c50000,0x55d50000)]
      0x534dfc00 JavaThread "scheduler-8" [_thread_blocked, id=2340, stack(0x55b50000,0x55c50000)]
      0x534dec00 JavaThread "scheduler-7" [_thread_blocked, id=2672, stack(0x55a50000,0x55b50000)]
      0x53519c00 JavaThread "scheduler-6" [_thread_blocked, id=4056, stack(0x55950000,0x55a50000)]
      0x53518c00 JavaThread "scheduler-5" [_thread_blocked, id=2620, stack(0x55850000,0x55950000)]
      0x53517c00 JavaThread "scheduler-4" [_thread_blocked, id=4236, stack(0x55750000,0x55850000)]
      0x53516c00 JavaThread "scheduler-3" [_thread_blocked, id=3936, stack(0x55650000,0x55750000)]
      0x53515c00 JavaThread "scheduler-2" [_thread_blocked, id=5868, stack(0x55550000,0x55650000)]
      0x5352c400 JavaThread "scheduler-1" [_thread_blocked, id=4964, stack(0x55450000,0x55550000)]
      0x534da400 JavaThread "scheduler-0" [_thread_blocked, id=2512, stack(0x55350000,0x55450000)]
      0x5351ac00 JavaThread "Transaction Domain PrivateJmsDomain" [_thread_blocked, id=2392, stack(0x55250000,0x55350000)]
      0x53505400 JavaThread "clock" daemon [_thread_blocked, id=2096, stack(0x55150000,0x55250000)]
      0x534e1400 JavaThread "Transaction Domain DefaultDomain" [_thread_blocked, id=5864, stack(0x55050000,0x55150000)]
      0x53504400 JavaThread "tyrex.util.daemonMaster" daemon [_thread_blocked, id=6108, stack(0x54f50000,0x55050000)]
      0x534a2800 JavaThread "jndi-4" [_thread_blocked, id=6092, stack(0x54e50000,0x54f50000)]
      0x534f9000 JavaThread "jndi-3" [_thread_blocked, id=4720, stack(0x54d50000,0x54e50000)]
      0x534f8400 JavaThread "jndi-2" [_thread_in_native, id=704, stack(0x54c50000,0x54d50000)]
      0x534f7800 JavaThread "jndi-1" [_thread_blocked, id=4564, stack(0x54b50000,0x54c50000)]
      0x534ac800 JavaThread "jndi-0" [_thread_blocked, id=5016, stack(0x54a50000,0x54b50000)]
      0x534f5400 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=3420, stack(0x54850000,0x54950000)]
      0x534c0c00 JavaThread "jms-fifo-5" [_thread_blocked, id=2420, stack(0x54650000,0x54750000)]
      0x53479800 JavaThread "jms-fifo-4" [_thread_blocked, id=3188, stack(0x54550000,0x54650000)]
      0x53479400 JavaThread "jms-fifo-3" [_thread_blocked, id=5652, stack(0x54450000,0x54550000)]
      0x53478c00 JavaThread "jms-fifo-2" [_thread_blocked, id=4064, stack(0x54350000,0x54450000)]
      0x53286800 JavaThread "jms-fifo-1" [_thread_blocked, id=3940, stack(0x54250000,0x54350000)]
      0x534fd800 JavaThread "GC Daemon" daemon [_thread_blocked, id=2660, stack(0x54150000,0x54250000)]
      0x53461800 JavaThread "RMI Reaper" [_thread_blocked, id=3456, stack(0x54050000,0x54150000)]
      0x533dcc00 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=1340, stack(0x53f50000,0x54050000)]
      0x5262b000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=4880, stack(0x52f40000,0x53040000)]
      0x52625400 JavaThread "CompilerThread1" daemon [_thread_blocked, id=4912, stack(0x52e40000,0x52f40000)]
    =>0x52624000 JavaThread "CompilerThread0" daemon [_thread_in_native, id=4680, stack(0x52d40000,0x52e40000)]
      0x52622c00 JavaThread "Attach Listener" daemon [_thread_blocked, id=4928, stack(0x52c40000,0x52d40000)]
      0x52621c00 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=544, stack(0x52b40000,0x52c40000)]
      0x52621000 JavaThread "Surrogate Locker Thread (CMS)" daemon [_thread_blocked, id=2700, stack(0x52a40000,0x52b40000)]
      0x52606400 JavaThread "Finalizer" daemon [_thread_blocked, id=5912, stack(0x52940000,0x52a40000)]
      0x52605400 JavaThread "Reference Handler" daemon [_thread_blocked, id=4664, stack(0x52840000,0x52940000)]
    Other Threads:
      0x52602400 VMThread [stack: 0x52740000,0x52840000] [id=2656]
      0x5262c400 WatcherThread [stack: 0x53040000,0x53140000] [id=4884]
    VM state:not at safepoint (normal execution)
    VM Mutex/Monitor currently owned by a thread: None
    Heap
    par new generation   total 218496K, used 75145K [0x03c60000, 0x13c60000, 0x13c60000)
      eden space 174848K,  35% used [0x03c60000, 0x0786c1c0, 0x0e720000)
      from space 43648K,  31% used [0x0e720000, 0x0f476510, 0x111c0000)
      to   space 43648K,   0% used [0x111c0000, 0x111c0000, 0x13c60000)
    concurrent mark-sweep generation total 786432K, used 85404K [0x13c60000, 0x43c60000, 0x43c60000)
    concurrent-mark-sweep perm gen total 131072K, used 69573K [0x43c60000, 0x4bc60000, 0x4fc60000)
    Dynamic libraries:
    0x00400000 - 0x00410000     C:\ColdFusion8\runtime\bin\jrun.exe
    0x7c800000 - 0x7c8c2000     C:\WINDOWS\system32\ntdll.dll
    0x77e40000 - 0x77f42000     C:\WINDOWS\system32\kernel32.dll
    0x7d1e0000 - 0x7d27c000     C:\WINDOWS\system32\ADVAPI32.dll
    0x77c50000 - 0x77cef000     C:\WINDOWS\system32\RPCRT4.dll
    0x76f50000 - 0x76f63000     C:\WINDOWS\system32\Secur32.dll
    0x77ba0000 - 0x77bfa000     C:\WINDOWS\system32\MSVCRT.dll
    0x6dac0000 - 0x6de7a000     C:\ColdFusion8\runtime\jre\bin\server\jvm.dll
    0x77380000 - 0x77411000     C:\WINDOWS\system32\USER32.dll
    0x77c00000 - 0x77c49000     C:\WINDOWS\system32\GDI32.dll
    0x76aa0000 - 0x76acd000     C:\WINDOWS\system32\WINMM.dll
    0x7c340000 - 0x7c396000     C:\WINDOWS\system32\MSVCR71.dll
    0x76290000 - 0x762ad000     C:\WINDOWS\system32\IMM32.DLL
    0x71bc0000 - 0x71bc8000     C:\WINDOWS\system32\rdpsnd.dll
    0x771f0000 - 0x77201000     C:\WINDOWS\system32\WINSTA.dll
    0x71c40000 - 0x71c97000     C:\WINDOWS\system32\NETAPI32.dll
    0x76b70000 - 0x76b7b000     C:\WINDOWS\system32\PSAPI.DLL
    0x6d320000 - 0x6d328000     C:\ColdFusion8\runtime\jre\bin\hpi.dll
    0x6d820000 - 0x6d82c000     C:\ColdFusion8\runtime\jre\bin\verify.dll
    0x6d3c0000 - 0x6d3df000     C:\ColdFusion8\runtime\jre\bin\java.dll
    0x6d860000 - 0x6d86f000     C:\ColdFusion8\runtime\jre\bin\zip.dll
    0x6d620000 - 0x6d633000     C:\ColdFusion8\runtime\jre\bin\net.dll
    0x71c00000 - 0x71c17000     C:\WINDOWS\system32\WS2_32.dll
    0x71bf0000 - 0x71bf8000     C:\WINDOWS\system32\WS2HELP.dll
    0x71b20000 - 0x71b61000     C:\WINDOWS\System32\mswsock.dll
    0x76ed0000 - 0x76efa000     C:\WINDOWS\system32\DNSAPI.dll
    0x76f70000 - 0x76f77000     C:\WINDOWS\System32\winrnr.dll
    0x76f10000 - 0x76f3e000     C:\WINDOWS\system32\WLDAP32.dll
    0x76f80000 - 0x76f85000     C:\WINDOWS\system32\rasadhlp.dll
    0x53600000 - 0x5360c000     C:\ColdFusion8\runtime\bin\portscan.dll
    0x5f270000 - 0x5f2ca000     C:\WINDOWS\system32\hnetcfg.dll
    0x71ae0000 - 0x71ae8000     C:\WINDOWS\System32\wshtcpip.dll
    0x68000000 - 0x68035000     C:\WINDOWS\system32\rsaenh.dll
    0x76920000 - 0x769e2000     C:\WINDOWS\system32\USERENV.dll
    0x6d7c0000 - 0x6d7c6000     C:\ColdFusion8\runtime\jre\bin\rmi.dll
    0x6d640000 - 0x6d649000     C:\ColdFusion8\runtime\jre\bin\nio.dll
    0x57950000 - 0x57957000     C:\ColdFusion8\lib\FusionReactor.dll
    0x77d00000 - 0x77d8b000     C:\WINDOWS\system32\OLEAUT32.dll
    0x77670000 - 0x777a9000     C:\WINDOWS\system32\ole32.dll
    0x5e2f0000 - 0x5e2fa000     C:\WINDOWS\system32\perfos.dll
    0x6d800000 - 0x6d808000     C:\ColdFusion8\runtime\jre\bin\sunmscapi.dll
    0x761b0000 - 0x76243000     C:\WINDOWS\system32\CRYPT32.dll
    0x76190000 - 0x761a2000     C:\WINDOWS\system32\MSASN1.dll
    0x54950000 - 0x549d0000     C:\ColdFusion8\lib\izmjniado.dll
    0x76cf0000 - 0x76d0a000     C:\WINDOWS\system32\iphlpapi.dll
    0x76cd0000 - 0x76ce9000     C:\WINDOWS\system32\MPRAPI.dll
    0x76df0000 - 0x76e24000     C:\WINDOWS\system32\ACTIVEDS.dll
    0x76dc0000 - 0x76de8000     C:\WINDOWS\system32\adsldpc.dll
    0x76b80000 - 0x76bae000     C:\WINDOWS\system32\credui.dll
    0x7c8d0000 - 0x7d0cf000     C:\WINDOWS\system32\SHELL32.dll
    0x77da0000 - 0x77df2000     C:\WINDOWS\system32\SHLWAPI.dll
    0x76a80000 - 0x76a92000     C:\WINDOWS\system32\ATL.DLL
    0x76e30000 - 0x76e3c000     C:\WINDOWS\system32\rtutils.dll
    0x7e020000 - 0x7e02f000     C:\WINDOWS\system32\SAMLIB.dll
    0x770e0000 - 0x771e8000     C:\WINDOWS\system32\SETUPAPI.dll
    0x77420000 - 0x77523000     C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.3790.3959_x- ww_D8713E55\comctl32.dll
    0x54a00000 - 0x54a08000     C:\ColdFusion8\lib\CFXNeo.dll
    0x60020000 - 0x60030000     C:\WINDOWS\system32\MSVCIRT.dll
    0x57eb0000 - 0x57f15000     C:\WINDOWS\system32\MSVCP60.dll
    0x54a30000 - 0x54a3e000     C:\ColdFusion8\lib\cfregistry.dll
    0x57970000 - 0x57980000     C:\ColdFusion8\lib\PerfmonClient.dll
    0x6d0b0000 - 0x6d1de000     C:\ColdFusion8\runtime\jre\bin\awt.dll
    0x73070000 - 0x73097000     C:\WINDOWS\system32\WINSPOOL.DRV
    0x73860000 - 0x738ab000     C:\WINDOWS\system32\ddraw.dll
    0x73b30000 - 0x73b36000     C:\WINDOWS\system32\DCIMAN32.dll
    0x6d2c0000 - 0x6d313000     C:\ColdFusion8\runtime\jre\bin\fontmanager.dll
    0x54750000 - 0x54757000     C:\ColdFusion8\lib\NeoUUID.dll
    0x6d200000 - 0x6d22f000     C:\ColdFusion8\runtime\jre\bin\cmm.dll
    0x6d460000 - 0x6d484000     C:\ColdFusion8\runtime\jre\bin\jpeg.dll
    0x54770000 - 0x5477b000     C:\ColdFusion8\lib\clib_jiio_util.dll
    0x5d6c0000 - 0x5d7cc000     C:\ColdFusion8\lib\clib_jiio_sse2.dll
    0x547b0000 - 0x547bb000     C:\ColdFusion8\lib\mlib_jai_util.dll
    0x5d8d0000 - 0x5dd1e000     C:\ColdFusion8\lib\mlib_jai_mmx.dll
    VM Arguments:
    jvm_args: -Xmx1024m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:PermSize=128m -XX:NewSize=256m -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:SurvivorRatio=4 -XX:+UseParNewGC -XX:+HeapDumpOnOutOfMemoryError -Dcoldfusion.rootDir=C:\ColdFusion8\runtime/../ -Dcoldfusion.libPath=C:\ColdFusion8\runtime/../lib -Dcoldfusion.classPath=C:\ColdFusion8\runtime/../lib/updates,C:\ColdFusion8\runtime/../li b,C:\ColdFusion8\runtime/../gateway/lib/,C:\ColdFusion8\runtime/../wwwroot/WEB-INF/flex/ja rs,C:\ColdFusion8\runtime/../wwwroot/WEB-INF/cfform/jars -Djmx.invoke.getters=true
    java_command: <unknown>
    Launcher Type: generic
    Environment Variables:
    PATH=C:\ColdFusion8\runtime\..\lib;C:\ColdFusion8\runtime\..\jintegra\bin;C:\ColdFusion8\r untime\..\jintegra\bin\international;C:\ColdFusion8\verity\k2\_nti40\bin;C:\WINDOWS\system 32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
    USERNAME=xxxxx
    OS=Windows_NT
    PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 8, GenuineIntel
    ---------------  S Y S T E M  ---------------
    OS: Windows Server 2003 family Build 3790 Service Pack 2
    CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 7 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3
    Memory: 4k page, physical 2097151k(1871136k free), swap 4194303k(3129772k free)
    vm_info: Java HotSpot(TM) Server VM (10.0-b19) for windows-x86 JRE (1.6.0_04-b12), built on Dec 14 2007 00:46:56 by "java_re" with MS VC++ 7.1
    time: Wed Nov 11 23:00:05 2009
    elapsed time: 71488 seconds

    Yes, that site is down. That is why I mentioned using google cache. Just google that url, and use the "Cached" option. 
    There is a lot of information in the link. So I would definitely review it to see if it matches your situation. But here is one snippet:
    "While these errors are found among various Sun bug reports, the only easy solution is an indirect one, to disable the HotSpot compiler entirely with the -Xint optional argument which forces it to only run in interpreted mode. The errors might be caused when HotSpot begins to optimized certain frequently run methods or tight blocks of code. When this happens the compiler will convert the interpreted bytecode to native code and tuck it away in a little piece of memory. The compiler probably needs a little chunk of contiguous memory to store the optimized native code and the OS was unable to provide it. Notice the errors usually indicate that attempts to allocate 1-2MB of memory failed, even though its often noted that neither the JVM heap nor the OS memory usage has maxed out at the time of crash.
    A more careful approach to resolve the crashes would be..."
    -Leigh

  • Server Crash OutOfMemory

    This night our coldfusion server crashed. It gave a jrun
    error
    . Any help on this error would be greatly appreciated
    Many thanks in advance,
    Nebu
    ------------ERROR--------------------
    unable to create new native thread The specific sequence of
    files included or processed is: D:\Inetpub\NieuwBouw\index.cfm,
    line: 35
    1555
    "Error","jrpp-2057","11/17/06","02:08:32","QasaOnWeb","unable to
    create new native thread The specific sequence of files included or
    processed is: D:\Inetpub\NieuwBouw\projecten\index.cfm, line: 35 "
    1556 java.lang.OutOfMemoryError: unable to create new native
    thread
    1557 at java.lang.Thread.start(Native Method)
    1558 at
    coldfusion.server.j2ee.sql.pool.ConnectionRunner.fetchConnection(ConnectionRunner.java:42 )
    1559 at
    coldfusion.server.j2ee.sql.pool.JDBCPool.create(JDBCPool.java:519)
    1560 at
    coldfusion.server.j2ee.sql.pool.JDBCPool._checkOut(JDBCPool.java:444)
    1561 at
    coldfusion.server.j2ee.sql.pool.JDBCPool.checkOut(JDBCPool.java:350)
    1562 at
    coldfusion.server.j2ee.sql.pool.JDBCPool.requestConnection(JDBCPool.java:720)
    1563 at
    coldfusion.server.j2ee.sql.pool.JDBCManager.requestConnection(JDBCManager.java:123)
    1564 at
    coldfusion.server.j2ee.sql.JRunDataSource.getConnection(JRunDataSource.java:138)
    1565 at
    coldfusion.server.j2ee.sql.JRunDataSource.getConnection(JRunDataSource.java:125)
    1566 at
    coldfusion.sql.CFDataSource.getConnection(CFDataSource.java:37)
    1567 at
    coldfusion.sql.DataSrcImpl.getCachedConnection(DataSrcImpl.java:149)
    1568 at
    coldfusion.sql.DataSrcImpl.getConnection(DataSrcImpl.java:103)
    1569 at coldfusion.sql.SqlImpl.execute(SqlImpl.java:214)
    1570 at
    coldfusion.tagext.sql.QueryTag.doEndTag(QueryTag.java:499)
    1571 at
    cffooter2ecfm1264563072.runPage(D:\Inetpub\NieuwBouw\footer.cfm:35)
    1572 at
    coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:152)
    1573 at
    coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:349)
    1574 at
    coldfusion.runtime.CfJspPage._emptyTag(CfJspPage.java:1915)
    1575 at
    cfbottom2ecfm236715733.runPage(D:\Inetpub\NieuwBouw\projecten\bottom.cfm:4)
    1576 at
    coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:152)
    1577 at
    coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:349)
    1578 at
    coldfusion.runtime.CfJspPage._emptyTag(CfJspPage.java:1915)
    1579 at
    cfsub_samengevoegd2ecfm992510435.runPage(D:\Inetpub\NieuwBouw\projecten\sub_samengevoegd. cfm:477)
    1580 at
    coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:152)
    1581 at
    coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:349)
    1582 at
    coldfusion.runtime.CfJspPage._emptyTag(CfJspPage.java:1915)
    1583 at
    cfindex2ecfm1730678187.runPage(D:\Inetpub\NieuwBouw\projecten\index.cfm:19)
    1584 at
    coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:152)
    1585 at
    coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:349)
    1586 at
    coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
    1587 at
    coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:209)
    1588 at
    coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:51)
    1589 at
    coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
    1590 at
    coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:69)
    1591 at
    coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
    1592 at
    coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
    1593 at
    coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
    1594 at
    coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
    1595 at
    coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:115)
    1596 at coldfusion.CfmServlet.service(CfmServlet.java:107)
    1597 at
    coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:78)
    1598 at
    jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)
    1599 at
    jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
    1600 at
    jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)
    1601 at
    jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)
    1602 at
    jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:204)
    1603 at
    jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)
    1604 at
    jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)
    1605 at
    jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)
    1606 at
    jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

    Answered by Adobe technical support
    Since the error does mention "out of memory" in the context
    of not being able to create a new thread, it could be worth trying
    to decrease the default stack size per thread using a JVM switch,
    -Xss. So you could edit (after backing up)
    {JRun4_Home}/bin/jvm.config to add the -Xss argument on the
    java.args line just after the -Xmx (max heap) setting like this:
    java.args=-server -Xmx512m -Xss256k .....
    DO NOT USE THE CF ADMIN TO MAKE THIS CHANGE
    This example would tell the jvm to use a thread stack size of
    256k instead of the default stack (assumed to be 1024k). A CFMX
    server restart is required. You might want to first try 256k, but
    if any other problem turns up you could try increasing it to 512k.

  • More jrun using up memory problems

    There seem to be a lot of problems like this but everyone's situation seems to be a little different.  It makes it hard to find solutions.  Here is what we are dealing with.
    We are taking an XML file containing product data and importing it into a SQL 2000 database.  The xml file is about 6mb and it has product data for about 4700 products.  When we wrote the import we were running coldfusion 7 standard updated to 7.02.  We run the import every night and it seemed to run fine.  It took about 45 minutes to run but the server didn't crash.  We just recently upgraded to cf 9 standard.  After the upgrade when the import ran we were seeing lots of out of memory errors in the log.  The second night it ran coldfusion had to be restarted.
    We setup a testing server and ran the import there and watched the memory usage of jrun.  As the import ran memory usage kept climbing until it reached the max heap size.  After this the import got very slow and eventually coldfusion became completely unresponsive.
    We changed the import so that it would do the products in smaller batches as separate requests.  This seemed to be the answer.  Between the requests the memory usage would drop down some and the whole thing was able to run without using up all the available memory.  The import also takes a lot less time now that the memory usage isn't so high.  We thought we had it fixed so we started running it on the production server.
    After a couple days we noticed that our memory usage was very high on the production server.  We went back to the testing server and ran the import a few times.  After this we could see that each time it was run the memory usage would increase.  After about 4 times the memory reached the max heap size and coldfusion became unresponsive.  The memory usage never goes back down.

    Some things to check / consider.
    1. What are your JVM settings?  What is your Maximum JVM Heap Size?  If you are using the default value, 512 MB, consider a larger value.  On a 32 bit server the upper limit is 1.8 GB.  A 64 bit server can allocate more memory to the JVM, but I haven't seen any documentation that specifies the upper limits on 64 bit.  (If anyone can point me to some info on this I'd be grateful.)
    2. Consider moving the XML to SQL import process off of ColdFusion to another tool.
    Some options:
    a. A SQL Server DTS package
    b. Use the bulk insert feature of SQL Server
    c. Use a third party tool such as Altova Mapforce
    Java and JVM page in CF administration
    http://help.adobe.com/en_US/ColdFusion/9.0/Admin/WSc3ff6d0ea77859461172e0811cbf3638e6-7ffc .html#WSc3ff6d0ea77859461172e0811cbf3638e6-7feb
    "Maximum JVM heap size greater than 1.8GB will prevent ColdFusion MX from starting"
    http://go.adobe.com/kb/ts_tn_19359_en-us
    "Examples of Bulk Importing and Exporting XML Documents" (SQL Server 2000)
    http://msdn.microsoft.com/en-us/library/ms191184.aspx
    Altova Mapforce
    http://www.altova.com/mapforce.html
    "Troubleshooting a Leaky Heap in Your JVM" (blog entry)
    http://www.coldfusionmuse.com/index.cfm/2008/2/12/leaky.heap.jvm

  • JDBC OCI on Linux crashes, pls help

    Hi!
    I'm trying to use jdk 1.2.2, and jdbc2 of 8.1.6(classes12_01.zip), on linux Oracle8i 8.1.5, and at some moment the app crashes with a SIGSEGV, after making several sqlqueries:
    SIGSEGV 11* segmentation violation
    si_signo [11]: SIGSEGV 11* segmentation violation
    si_errno [0]: Success
    si_code [0]: SI_USER [pid: 0, uid: 0]
    stackpointer=0x44bbcff4
    Full thread dump Classic VM (1.2.2-L, green threads):
    "CaducatorThread" (TID:0x40eb5ab8, sys_thread_t:0x89af9e8, state:R) prio=1
    at oracle.jdbc.oci8.OCIDBAccess.open_statement(Native Method)
    at oracle.jdbc.oci8.OCIDBAccess.open(OCIDBAccess.java:458)
    at oracle.jdbc.driver.OracleStatement.<init>(OracleStatement.java:288)
    at oracle.jdbc.driver.OracleStatement.<init>(OracleStatement.java:307)
    at oracle.jdbc.driver.OraclePreparedStatement.<init>(OraclePreparedStatement.java:194)
    at oracle.jdbc.driver.OraclePreparedStatement.<init>(OraclePreparedStatement.java:177)
    at oracle.jdbc.driver.OracleConnection.prepareStatement(OracleConnection.java:339)
    at server.persistence.jdbc.JdbcProcessInstancePersMgr.getInstancesToDispose(JdbcProcessInstancePersMgr.java:1111)
    I tried with the examples on jdbc/OCI and they worked, but my app, that is multithreaded, crashes.
    Pls I don't know where to look anymore, I'm desperate.
    Million Thanks in advance.
    Mariano

    Try placing the jar file in the
    {cfmx-root}/runtime/servers/lib (create if not already there)
    directory if you have the server install or the
    {jrun-root}/servers/lib directory if you have the multiserver or
    j2ee install on JRun. Restart the Coldfusion applicatioin server.
    Ted Zimmerman

  • CF crashes 1 to 2 times a weeks

    hey there guys, it does not fail! CF keeps crashing on us 1 to 2 times a week sometimes even 3! i have applied all the latest patches to the system and CF but it just keep getting worse and worse! I'm not a CF guy and im tired of google'n stuff. please let me know what kind of info you guys would need to help me out with these problems! i have researched and read eveything on this site. thanks in advanced.
    exception.log
    "    "Error","jrpp-799","09/06/12","03:17:45","XXX","Connection Failure: Status code unavailable The specific sequence of files included or processed is: C:\wwwroot\scripts\processEmail.cfm, line: 145 "
    coldfusion.tagext.net.HttpTag$HttpConnectionFailureException: Connection Failure: Status code unavailable
    "    "Error","jrpp-925","09/07/12","00:32:55","XXX","Java heap space The specific sequence of files included or processed is: C:\wwwroot\XXX-micro\index.cfm, line: 9 "
    java.lang.OutOfMemoryError: Java heap space
    "    "Error","jrpp-938","09/07/12","00:32:55","XXX","The request has exceeded the allowable time limit Tag: cfoutput The specific sequence of files included or processed is: C:\wwwroot\locator\search.cfm, line: 338 "
    coldfusion.runtime.RequestTimedOutException: The request has exceeded the allowable time limit Tag: cfoutput
    "    "Error","jrpp-929","09/07/12","00:34:16","XXX","The request has exceeded the allowable time limit Tag: CFLOOP The specific sequence of files included or processed is: C:\wwwroot\scripts\processEmail.cfm, line: 107 "
    coldfusion.runtime.RequestTimedOutException: The request has exceeded the allowable time limit Tag: CFLOOP
    server.log
    "    "Warning","jrpp-935","09/07/12","00:26:25",,"Thread: jrpp-935, processing template: C:\wwwroot\XXX-micro\index.cfm, completed in 55 seconds, exceeding the 45 second warning limit"
    "    "Warning","jrpp-913","09/07/12","00:26:27",,"Thread: jrpp-913, processing template: C:\wwwroot\XXX-micro\candidate.cfm, completed in 47 seconds, exceeding the 45 second warning limit"
    "    "Warning","jrpp-938","09/07/12","00:32:12",,"Thread: jrpp-938, processing template: C:\wwwroot\locator\search.cfm, completed in 60 seconds, exceeding the 45 second warning limit"
    "    "Warning","jrpp-925","09/07/12","00:32:53",,"Thread: jrpp-925, processing template: C:\wwwroot\XXX-micro\index.cfm, completed in 150 seconds, exceeding the 45 second warning limit"
    "    "Warning","jrpp-928","09/07/12","00:32:58",,"Thread: jrpp-928, processing template: C:\wwwroot\locator\search.cfm, completed in 163 seconds, exceeding the 45 second warning limit"
    "    "Warning","jrpp-932","09/07/12","00:33:01",,"Thread: jrpp-932, processing template: C:\wwwroot\locator\search.cfm, completed in 168 seconds, exceeding the 45 second warning
    application.log
    "    "Error","jrpp-925","09/07/12","00:32:55","XXX","Java heap space The specific sequence of files included or processed is: C:\wwwroot\XXX-micro\index.cfm, line: 9 "
    "    "Error","jrpp-938","09/07/12","00:32:55","XXX","The request has exceeded the allowable time limit Tag: cfoutput The specific sequence of files included or processed is: C:\wwwroot\locator\search.cfm, line: 338 "
    "    "Error","jrpp-929","09/07/12","00:34:16","XXX","The request has exceeded the allowable time limit Tag: CFLOOP The specific sequence of files included or processed is: C:\wwwroot\scripts\processEmail.cfm, line: 107 "

    this is what i got so far.
    09/07 00:32:58 error ROOT CAUSE:
    java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2786)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
        at java.io.DataOutputStream.write(DataOutputStream.java:90)
        at java.io.DataOutputStream.writeUTF(DataOutputStream.java:384)
        at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
        at org.apache.bcel.classfile.ConstantUtf8.dump(ConstantUtf8.java:125)
        at org.apache.bcel.classfile.ConstantPool.dump(ConstantPool.java:230)
        at org.apache.bcel.classfile.JavaClass.dump(JavaClass.java:322)
        at org.apache.bcel.classfile.JavaClass.getBytes(JavaClass.java:290)
        at coldfusion.bytecode.JavaAssembler.getBytes(JavaAssembler.java:428)
        at coldfusion.compiler.TemplateAssembler.assemble(TemplateAssembler.java:111)
        at coldfusion.compiler.NeoTranslator.translateJava(NeoTranslator.java:377)
        at coldfusion.compiler.NeoTranslator.translateJava(NeoTranslator.java:147)
        at coldfusion.runtime.TemplateClassLoader$TemplateCache$1.fetch(TemplateClassLoader.java:442 )
        at coldfusion.util.LruCache.get(LruCache.java:180)
        at coldfusion.runtime.TemplateClassLoader$TemplateCache.fetchSerial(TemplateClassLoader.java :362)
        at coldfusion.util.AbstractCache.fetch(AbstractCache.java:58)
        at coldfusion.util.SoftCache.get_statsOff(SoftCache.java:133)
        at coldfusion.util.SoftCache.get(SoftCache.java:81)
        at coldfusion.runtime.TemplateClassLoader.findClass(TemplateClassLoader.java:609)
        at coldfusion.runtime.RuntimeServiceImpl.getFile(RuntimeServiceImpl.java:785)
        at coldfusion.runtime.RuntimeServiceImpl.resolveTemplatePath(RuntimeServiceImpl.java:766)
        at coldfusion.tagext.lang.IncludeTag.setTemplate(IncludeTag.java:337)
        at cfindex2ecfm1647231854.runPage(C:\wwwroot\XXX-micro\index.cfm:9)
        at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231)
        at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:416)
        at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
        at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:381)
        at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
        at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
        at coldfusion.filter.PathFilter.invoke(PathFilter.java:94)
        at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
    javax.servlet.ServletException: ROOT CAUSE:
    java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2786)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
        at java.io.DataOutputStream.write(DataOutputStream.java:90)
        at java.io.DataOutputStream.writeUTF(DataOutputStream.java:384)
        at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
        at org.apache.bcel.classfile.ConstantUtf8.dump(ConstantUtf8.java:125)
        at org.apache.bcel.classfile.ConstantPool.dump(ConstantPool.java:230)
        at org.apache.bcel.classfile.JavaClass.dump(JavaClass.java:322)
        at org.apache.bcel.classfile.JavaClass.getBytes(JavaClass.java:290)
        at coldfusion.bytecode.JavaAssembler.getBytes(JavaAssembler.java:428)
        at coldfusion.compiler.TemplateAssembler.assemble(TemplateAssembler.java:111)
        at coldfusion.compiler.NeoTranslator.translateJava(NeoTranslator.java:377)
        at coldfusion.compiler.NeoTranslator.translateJava(NeoTranslator.java:147)
        at coldfusion.runtime.TemplateClassLoader$TemplateCache$1.fetch(TemplateClassLoader.java:442 )
        at coldfusion.util.LruCache.get(LruCache.java:180)
        at coldfusion.runtime.TemplateClassLoader$TemplateCache.fetchSerial(TemplateClassLoader.java :362)
        at coldfusion.util.AbstractCache.fetch(AbstractCache.java:58)
        at coldfusion.util.SoftCache.get_statsOff(SoftCache.java:133)
        at coldfusion.util.SoftCache.get(SoftCache.java:81)
        at coldfusion.runtime.TemplateClassLoader.findClass(TemplateClassLoader.java:609)
        at coldfusion.runtime.RuntimeServiceImpl.getFile(RuntimeServiceImpl.java:785)
        at coldfusion.runtime.RuntimeServiceImpl.resolveTemplatePath(RuntimeServiceImpl.java:766)
        at coldfusion.tagext.lang.IncludeTag.setTemplate(IncludeTag.java:337)
        at cfindex2ecfm1647231854.runPage(C:\wwwroot\XXX-micro\index.cfm:9)
        at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231)
        at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:416)
        at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
        at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:381)
        at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
        at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
        at coldfusion.filter.PathFilter.invoke(PathFilter.java:94)
        at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
        at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:70 )
        at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
        at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
        at jrun.servlet.FilterChain.service(FilterChain.java:101)
        at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
        at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
        at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
        at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
        at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
        at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
        at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
        at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
        at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
    09/07 00:33:03 error (JRun Service: ProxyService [jrun.servlet.jrpp.JRunProxyService@51fffa02]) JRunPRoxyServer.invokeRunnable:
    java.lang.IllegalStateException
        at jrun.servlet.JRunResponse.getWriter(JRunResponse.java:205)
        at jrun.servlet.JRunResponse.sendError(JRunResponse.java:597)
        at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:328)
        at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
        at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
        at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
        at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
        at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
        at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
    java.lang.IllegalStateException
        at jrun.servlet.JRunResponse.getWriter(JRunResponse.java:205)
        at jrun.servlet.JRunResponse.sendError(JRunResponse.java:597)
        at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:328)
        at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
        at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
        at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
        at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
        at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
        at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

  • Problems with Windows 2003, JRun and Java 1.4.2_07

    Hi!
    I have installed JRun 4 with J2SE 1.4.2_07 on a Windows 2003 Server and all goes well until someone install a system patch, Office. After that JRun doesn�t star. The logs says:
    Starting Macromedia JRun 4 (Build 84683), admin server
    An unexpected exception has been detected in native code outside the VM.
    Unexpected Signal : EXCEPTION_ACCESS_VIOLATION (0xc0000005) occurred at PC=0x77F46A07
    Function=RtlFreeHeap+0x322
    Library=C:\WINDOWS\system32\ntdll.dll
    Current Java thread:
         at java.util.zip.ZipFile.getEntry(Native Method)
         at java.util.zip.ZipFile.getEntry(ZipFile.java:146)
         - locked <0x10a83218> (a java.util.jar.JarFile)
         at java.util.jar.JarFile.getEntry(JarFile.java:194)
         at java.util.jar.JarFile.getJarEntry(JarFile.java:181)
         at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:671)
         at sun.misc.URLClassPath.getResource(URLClassPath.java:160)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:191)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:187)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:289)
         - locked <0x10a63608> (a sun.misc.Launcher$AppClassLoader)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:274)
         - locked <0x10a63608> (a sun.misc.Launcher$AppClassLoader)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
         at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:302)
         - locked <0x10a63608> (a sun.misc.Launcher$AppClassLoader)
         at java.lang.Class.getDeclaredConstructors0(Native Method)
         at java.lang.Class.privateGetDeclaredConstructors(Class.java:1618)
         at java.lang.Class.getConstructor0(Class.java:1930)
         at java.lang.Class.getConstructor(Class.java:1027)
         at com.sun.management.jmx.MetaData.findConstructor(MetaData.java:256)
         at com.sun.management.jmx.MBeanServerImpl.internal_instantiate(MBeanServerImpl.java:2111)
         at com.sun.management.jmx.MBeanServerImpl.instantiate(MBeanServerImpl.java:152)
         at jrunx.kernel.ConfigurableServicePartition.loadAndInit(ConfigurableServicePartition.java:127)
         at jrunx.kernel.ConfigurableServicePartition.loadChildren(ConfigurableServicePartition.java:89)
         at jrunx.kernel.ConfigurableServicePartition.setChildElements(ConfigurableServicePartition.java:60)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:324)
         at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1628)
         at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1523)
         at jrunx.kernel.ServiceAdapter.invokeMethod(ServiceAdapter.java:705)
         at jrunx.kernel.JRunServiceDeployer.loadMBeans(JRunServiceDeployer.java:179)
         at jrunx.kernel.JRunServiceDeployer.deployServices(JRunServiceDeployer.java:85)
         at jrunx.kernel.DeploymentService.loadServices(DeploymentService.java:46)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:324)
         at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1628)
         at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1523)
         at jrunx.kernel.JRun.startServer(JRun.java:575)
         at jrunx.kernel.JRun.<init>(JRun.java:493)
         at jrunx.kernel.JRun$1.run(JRun.java:346)
         at java.security.AccessController.doPrivileged(Native Method)
         at jrunx.kernel.JRun.start(JRun.java:343)
         at jrunx.kernel.JRun.startByNTService(JRun.java:427)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:324)
         at jrunx.kernel.JRun.invoke(JRun.java:180)
         at jrunx.kernel.JRun.main(JRun.java:168)
    Dynamic libraries:
    0x00400000 - 0x00410000      C:\JRun4\bin\jrun.exe
    0x77F40000 - 0x77FFA000      C:\WINDOWS\system32\ntdll.dll
    0x77E40000 - 0x77F34000      C:\WINDOWS\system32\kernel32.dll
    0x77DA0000 - 0x77E30000      C:\WINDOWS\system32\ADVAPI32.dll
    0x77C50000 - 0x77CF5000      C:\WINDOWS\system32\RPCRT4.dll
    0x77BA0000 - 0x77BF4000      C:\WINDOWS\system32\MSVCRT.dll
    0x08000000 - 0x08138000      C:\Java\j2sdk1.4.2_07\jre\bin\client\jvm.dll
    0x77D00000 - 0x77D8F000      C:\WINDOWS\system32\USER32.dll
    0x77C00000 - 0x77C44000      C:\WINDOWS\system32\GDI32.dll
    0x76AA0000 - 0x76ACC000      C:\WINDOWS\system32\WINMM.dll
    0x10000000 - 0x10007000      C:\Java\j2sdk1.4.2_07\jre\bin\hpi.dll
    0x76F50000 - 0x76F63000      C:\WINDOWS\system32\Secur32.dll
    0x003F0000 - 0x003FE000      C:\Java\j2sdk1.4.2_07\jre\bin\verify.dll
    0x005F0000 - 0x00609000      C:\Java\j2sdk1.4.2_07\jre\bin\java.dll
    0x00610000 - 0x0061D000      C:\Java\j2sdk1.4.2_07\jre\bin\zip.dll
    0x006E0000 - 0x006EF000      C:\Java\j2sdk1.4.2_07\jre\bin\net.dll
    0x71C00000 - 0x71C18000      C:\WINDOWS\system32\WS2_32.dll
    0x71BF0000 - 0x71BF8000      C:\WINDOWS\system32\WS2HELP.dll
    0x71B20000 - 0x71B63000      C:\WINDOWS\System32\mswsock.dll
    0x76ED0000 - 0x76EF7000      C:\WINDOWS\system32\DNSAPI.dll
    0x76F70000 - 0x76F77000      C:\WINDOWS\System32\winrnr.dll
    0x76F10000 - 0x76F3F000      C:\WINDOWS\system32\WLDAP32.dll
    0x76F80000 - 0x76F85000      C:\WINDOWS\system32\rasadhlp.dll
    0x03230000 - 0x0323C000      C:\JRun4\bin\portscan.dll
    0x71AE0000 - 0x71AE8000      C:\WINDOWS\System32\wshtcpip.dll
    0x76C10000 - 0x76C38000      C:\WINDOWS\system32\imagehlp.dll
    0x6D580000 - 0x6D621000      C:\WINDOWS\system32\dbghelp.dll
    0x77B90000 - 0x77B98000      C:\WINDOWS\system32\VERSION.dll
    0x76B70000 - 0x76B7B000      C:\WINDOWS\system32\PSAPI.DLL
    Heap at VM Abort:
    Heap
    def new generation total 2304K, used 1312K [0x10010000, 0x10280000, 0x109e0000)
    eden space 2112K, 56% used [0x10010000, 0x1013a8d0, 0x10220000)
    from space 192K, 61% used [0x10250000, 0x1026d958, 0x10280000)
    to space 192K, 0% used [0x10220000, 0x10220000, 0x10250000)
    tenured generation total 30272K, used 1139K [0x109e0000, 0x12770000, 0x18010000)
    the space 30272K, 3% used [0x109e0000, 0x10afce80, 0x10afd000, 0x12770000)
    compacting perm gen total 4864K, used 4680K [0x18010000, 0x184d0000, 0x1c010000)
    the space 4864K, 96% used [0x18010000, 0x184a2348, 0x184a2400, 0x184d0000)
    Local Time = Wed Feb 09 10:43:31 2005
    Elapsed Time = 3
    # The exception above was detected in native code outside the VM
    # Java VM: Java HotSpot(TM) Client VM (1.4.2_07-b05 mixed mode)
    # An error report file has been saved as hs_err_pid1712.log.
    # Please refer to the file for further information.
    Can someone help me. Thanx and sorry about my english.
    Manu

    It crashed 3 seconds after it is running. Should be reading some jar file.
    Could be a something corrupted a zip file or something. Can you still do a java -version? You may want to try reinstalling 1.4.2_07

  • HELP Planning Crashes

    We are running a Planning server and consistently find it crashes on large forms. Sometimes these forms work perfectly well and sometimes they crash. It stops the web forms from displaying and the only fix we have found is to restart the JRun services. Does anyone know how to permanently fix this?We have a WIndows 2003 server without service packs and JRun version 4 with Planning version 3.5

    How many rows are you trying to display in your planning web form?Probably your form has too many rows and many members having dynamic calc.Split the form accordingly. Create a form with less number of rows and scale it up.Other parameters include..memory of your server..Increase your retrieval buffer on essbase and other cache settings...etc

Maybe you are looking for

  • I live in Sweden and Portugal, can I have two apple accounts one for each country?

    Hi, I need your help, at the present moment I devide my time between two countries Sweden and Portugal, my Apple acoount is based in sweden, my question is can I open an account in Portugal and keep the Swedish account or i am only allowd an account?

  • How can I set up a default (pre-recorded) voicemail greeting on iphone 4s?

    How can I set up a default (pre-recorded) voicemail greeting on iphone 4s?i dont want a custom one with my voice i want the pre-recorded where a lady says my number . how do i get that?

  • Updating Thumbnails... each time I open iPhoto

    Hello- Every time I open iPhoto, it updates the thumbnails. I have tried the following to no avail: 1) Remove the iPhoto preferences from home/library/preferences, 2) Remove the iPhoto cache from home/library/Caches/com.apple.iPhoto 3) Rebuild the Th

  • Problem retrieving large amount of data!

    Hi, I'm currently working with a database application accessing very big amount of data. One query could result in 500 000+ hits! I would like to present this data in a JTable. When the query is executed, I create a two-dimensional array and store th

  • HORRIBLE QUALITY

    I am having some problems with iMovie and I am wondering if you could give me some advice. I am trying to screen-cast some Google Earth shots and then add microphone audio and music. Ideally, I would like to be able to submit the video to online vide