Permanent Generation GC and concurrent mode failures

Hello
I have experienced the following symptoms when a concurrent mode failure occurs (1.4.2_19):
Perm Gen 100% full but ample space in old generation.
First, I thought that even though there was ample space in OG, probably there was no ample "contiguous" space due to fragmentation. However, the GC logs always revealed a coincidence between "perm gen full", concurrent mode failure and "ample space in old gen" such that I think that the CMF errors in this case do not occur due to fragmentation.
So, my question is: If there is ample contiguous space in old generation for a full promotion guarantee, can concurrent mode failures still occur because perm gen is 100%? (assuming not using CMS perm gen sweeping options). My understanding is that the answer is "yes" because by default, perm gen is collected by Full GC which interferes with CMS and causes a concurrent mode failure. Can somebody please confirm? Plan to increase perm gen size, monitor perm gen growth and the enable perm gen sweeping.
Thank you
Gavin

cybergavin wrote:
My perm gen was not exactly 100% full.
The perm gen size was set to 128 MBSo
-XX:MaxPermSize=128m -XX:PermSize=128m
and every time it reaches around 127 MB, a concurrent mode failure occurs even though there is ample free space in the old generation and the parallel scavenge in the young generation is successful. And I did not get OOM exceptions.
So, I wanted to confirm whether the error "concurrent mode failure" can be caused by a Full GC caused by a full perm gen as opposed to its typical causes like full promotion failures and old gen fragmentation.Maybe you need to enable [-XX:+CMSIncrementalMode|http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html#icms.available_options] and maybe [-XX:+CMSIncrementalPacing|http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html#icms.available_options] and use [-XX:CMSIncrementalSafetyFactor=|http://blogs.sun.com/jonthecollector/entry/what_the_heck_s_a] and/or [-XX:CMSIncrementalDutyCycle=|http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html#icms.available_options] / [-XX:CMSIncrementalDutyCycleMin=|http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html#icms.available_options].
Or you could just try with a larger -XX:MaxPermSize=256m

Similar Messages

Concurrent mode failure triggered by a full permanent generation?

Hello
I've read various forum posts and blogs on concurrent mode failures and I'm aware why such failures occur (small old gen, promotion guarantee failure, fragmentation) in the old gen and some options (increase heap size/old gen, use CMSInitiatingFraction, explicit System.gc() at quiet hours for compaction, etc.) that may be used to alleviate the problem. In fact, thanks to these forums, I used some of these options to effectively deal with concurrent mode failures on a SPARC system (Solaris 9). However, with my current JVM tuning exercise, the following GC log snippet from a JVM 1.4.2_10 running on HP-UX 11.23 (ia64) leaves me in doubt.
346179.949: [GC 346179.950: [ParNew: 419022K->27014K(491520K), 0.1540543 secs] 976980K->585383K(1474560K), 0.1542645 secs]
346444.644: [GC 346444.645: [ParNew: 420230K->420230K(491520K), 0.0000416 secs]346444.645: [CMS (concurrent mode failure)[Unloading class sun.reflect.GeneratedMethodAccessor1107]
[Unloading class sun.reflect.GeneratedMethodAccessor1087]
[Unloading class org.springframework.ejb.support.AbstractJmsMessageDrivenBean]
[Unloading class sun.reflect.GeneratedConstructorAccessor552]
...#lots of classes unloaded here#
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor155]
: 558369K->317371K(983040K), 9.5144462 secs] 978599K->317371K(1474560K), 9.5147394 secs]
346677.529: [GC 346677.530: [ParNew: 393216K->13960K(491520K), 0.2163942 secs] 710587K->331331K(1474560K), 0.2166204 secs]
JVM 1.4.2_10 Flags: -Xms1536m -Xmx1536m -Xmn576m -XX:PermSize=512m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:MaxTenuringThreshold=31 -XX:TargetSurvivorRatio=80 -XX:SurvivorRatio=4 -XX:+TraceClassLoading -XX:+TraceClassUnloading -XX:+DisableExplicitGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xverbosegc:file=.....
Question: When using CMS, perm gen collection is turned off by default and a serial Full GC collects the perm gen. Could this have led to the concurrent mode failure seen above causing classes to be unloaded? i.e. when CMS collector needs to clean up perm gen via a serial STW GC, it generates a concurrent mode failure?
I doubt the concurrent mode failure could be caused by a full promotion guarantee failure or fragmentation given the available free heap and the fact that this was the first GC in the old generation since server startup.
Note: We do have a perm gen memory leak caused by spring/CGLIB. While we are fixing the class loading issue, we increased MaxPermSize from 256m to 512m to buy some time. With MaxPermSize=512m and -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled, we had very frequent CMS GCs (every 5 seconds) and I read that this happens because the CMSInitiatingthreshold is the same for both old gen and perm gen. So, we removed -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled and the CMS GCs are no longer frequent.
Thanks
Edited by: cybergavin on Dec 28, 2009 12:09 AM

Hmm..I think I was a bit hasty there...
[ParNew: 420230K->420230K(491520K), 0.0000416 secs]346444.645: [CMS (concurrent mode failure)[Unloading class........ indicates a parallel scavenge could not be performed most probably due to a "young generation guarantee" failure. Also, HPjmeter revealed no issues with perm gen growth. So, I have decreased the size of the young generation and triggered CMS earlier (changes in bold below):
*JVM 1.4.2_10 Flags:* -Xms1536m -Xmx1536m *-Xmn512m* -XX:PermSize=512m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:MaxTenuringThreshold=31 -XX:TargetSurvivorRatio=80 -XX:SurvivorRatio=4 -XX:+TraceClassLoading -XX:+TraceClassUnloading -XX:+DisableExplicitGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps *-XX:CMSInitiatingOccupancyFraction=50 -XX:+UseCMSInitiatingOccupancyOnly* -Xverbosegc:file=.....
Will post back on results of the above....

Concurrent mode failure- low memory -- OutOfMemory Exception

WebLogic Web-tier instance frequently causing low memory. Where GC logs shows
Total time for which application threads were stopped: 0.7241688 seconds
46903.931: [GC [1 CMS-initial-mark: 375909K(524288K)] 441597K(983040K), 0.1330196 secs]
Total time for which application threads were stopped: 0.2807213 seconds
46904.064: [CMS-concurrent-mark-start]
46907.915: [CMS-concurrent-mark: 3.850/3.850 secs]
46907.915: [CMS-concurrent-preclean-start]
46907.952: [CMS-concurrent-preclean: 0.035/0.037 secs]
46907.952: [CMS-concurrent-abortable-preclean-start]
46938.527: [GC {Heap before gc invocations=218:
par new generation total 458752K, used 458752K [0xaf800000, 0xcf800000, 0xcf800000)
eden space 393216K, 100% used [0xaf800000, 0xc7800000, 0xc7800000)
from space 65536K, 100% used [0xc7800000, 0xcb800000, 0xcb800000)
to space 65536K, 0% used [0xcb800000, 0xcb800000, 0xcf800000)
concurrent mark-sweep generation total 524288K, used 375909K [0xcf800000, 0xef800000, 0xef800000)
concurrent-mark-sweep perm gen total 131072K, used 97681K [0xef800000, 0xf7800000, 0xf7800000)
46938.528: [ParNew (promotion failed): 458752K->458752K(458752K), 2.2293867 secs]46940.757: [CMS46941.097: [CMS-concurrent-abortable-preclean: 8.221/33.145 secs]
(concurrent mode failure): 500649K->524287K(524288K), 9.2616939 secs] 834661K->599632K(983040K)Heap after gc invocations=219:
par new generation total 458752K, used 75345K [0xaf800000, 0xcf800000, 0xcf800000)
eden space 393216K, 19% used [0xaf800000, 0xb4194420, 0xc7800000)
from space 65536K, 0% used [0xc7800000, 0xc7800000, 0xcb800000)
to space 65536K, 0% used [0xcb800000, 0xcb800000, 0xcf800000)
concurrent mark-sweep generation total 524288K, used 524287K [0xcf800000, 0xef800000, 0xef800000)
concurrent-mark-sweep perm gen total 131072K, used 96428K [0xef800000, 0xf7800000, 0xf7800000)
, 11.4919889 secs]
For your reference JVM arguments for these instances are as follows:
-Xms1024m -Xmx1024m
-XX:NewSize=512m
-XX:MaxNewSize=512m
-XX:SurvivorRatio=6
-Xnoclassgc -XX:PermSize=128m
-XX:MaxPermSize=128m
-XX:+HeapDumpOnOutOfMemoryError
-XX:+DisableExplicitGC
-XX:+UseConcMarkSweepGC
-verbose:gc -XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintHeapAtGC
-Xloggc:gc.log
-XX:MaxPermSize=128m
JDK: Sun JDK 1.5
Operating Environment: Solaris 5.10
WebLogic version 9.2 MP3
Thread dump shows:
"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x009679c0 nid=0x26 waiting for monitor entry [0xa3e7e000..0xa3e7f970]
at weblogic.utils.io.Chunk.<init>(Chunk.java:293)
at weblogic.utils.io.Chunk.getChunk(Chunk.java:141)
at weblogic.utils.io.ChunkedOutputStream.advance(ChunkedOutputStream.java:52)
at weblogic.utils.io.ChunkedOutputStream.write(ChunkedOutputStream.java:65)
at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1685)
We had applied patches as suggested by Oracle WebLogic. But, still issue persists :(
Any help on this issue appriciated.

1) Yes, a System.gc() will be reported by cms as a concurrent mode failure.
2) See if the release you're using has the flag -XX:+ExplicitGCInvokesConcurrent.
This will tell cms to start a concurrent collection when a System.gc() is involved.
It is in jdk6. It may still not do what you want if the the concurrent collections
are happening too frequently.
RMI schedules the full GC's because GC's are local to a JVM and
the collector on one JVM (call it JVM A) does not know about state of the heap
on another JVM (call it JVM B).
3) I would have expected setting the gcInterval to
a larger number would have had more of an affect. What was the value?
Post a fragment of a gc log (-XX:+PrintGCDetails) that shows the concurrent
mode failure. Maybe there's something else going on.

Concurrent Mode Failure

With -XX:+UseConcMarkSweepGC turned on, I am getting a "Concurrent Mode Failure" when a minor collection that leads to a full GC (due to a full old generation) occurs. The VM runs normally although the full GC was very long.
Does anyone know what this means? Surprisingly little documentation on this error message. A suggestion was to enable class unloading with CMS, which I did, but the error still comes up.
Any ideas?

First of all, thanks for your replies, they have been
most helpful in our attempt to tune GC activities for
our uncommon application.We are happy that the forums have been a useful device for
providing direct help to customers such as yourself.
During the bursty allocation,
(1) most objects are promoted very quickly into
tenured, replacing old objects in the tenured
(2) a lot of temporary objects that only live for the
duration of the burst activity are created and
promoted into tenured
Obviously the solution to the second is to tune and
resize the young generation but we have reached a
limit where the pause times for minor collections are
acceptable.I am assuming from that that you have tried to tune
the sizes of Eden, the survivor spaces and tenuring
thresholds to what you believe are optimal settings
that reduce premature promotion while still staying
under your threshold of acceptable pause times.
Fair enough.
>
My point is that the size of old generation
after bursty allocation should be roughly the
same as before bursty allocation, given that
there is little floating garbage after the burst.
Our problem is that we rely on CMS to happen after
the burst (when CPU load is low and significantly
fewer objects are created), and this works using
vanilla CMS, but not with iCMS. Thus, my question
about whether iCMS is idle if no minor collections
occur.Yes, you are right. If the "steady state" iCMS duty cycle
is lower than 100%, then if allocation rates in Eden
fall substantially, then iCMS will not necessarily start
after the scavenge that promoted these objects into
the tenured generation. On the other hand, vanilla
CMS would (both those statements modulo the
understanding that the tenured generation occupancy
exceeds the appropriate triggering threshold).
However, it seems to me that it should not matter for
you that the tenured generation is not immediately
cleaned up following said promotion activity, because
presumably there is no pressure from further promotion?
Ah, OK, I think I understand your problem. You get these bursts
of activity, and the computed steady state duty cycle is too
low to be able to effectively deal with such bursty behaviour?
Are there verbose GC logs (form iCMS certainly, and as well, if possible,
from CMS) that you might be able to share with us at
hotspotgc dash feedback at sun dot com? It could be that we may
be able to tune our adaptive pacing filters to deal effectively
with such burstiness without your having to pay the price of
a 100% durty cycle (aka vanilla CMS), to deal with these
bursty periods while little collection activity is needed during
the not-so-bursty peiods. (It could well be that we can't, because
of the time-scale of said burstiness, but it's worth a try.)

Concurrent mode failure - CMS & RMI

Hi,
I've been struggling with this concurrent mode failures for quite some time and I would be very nice if somebody could answer some more or less basic questions:
1.) If I scedule a full gc when using CMS do I get an concurrent mode failure? In my log I get one like every 60 seconds even as more than 50% of the heap is free, so I guess it's due to RMI?!
2.) If 1.) is true, what can I do? I can disable explicit gcs, but will there remain uncollected garbage or will there be any other disadvantage? I mean SUN won't do this ugly FullGC-scheduling-feature for fun, do they?
3.) If I need explicit gcs (or if I at least suppose it's better to schedule it manually in the night), what can I do in that case? As far as I understood the sun.rmi.dgc.server.gcInterval just set's a maximum interval! Actually I put it to a very high value, but I get the FullGC with a concurrent mode failure every 63 seconds, at memory usage where the CMS should not even kick-in.
Thanks a lot for any help,
Robert.

1) Yes, a System.gc() will be reported by cms as a concurrent mode failure.
2) See if the release you're using has the flag -XX:+ExplicitGCInvokesConcurrent.
This will tell cms to start a concurrent collection when a System.gc() is involved.
It is in jdk6. It may still not do what you want if the the concurrent collections
are happening too frequently.
RMI schedules the full GC's because GC's are local to a JVM and
the collector on one JVM (call it JVM A) does not know about state of the heap
on another JVM (call it JVM B).
3) I would have expected setting the gcInterval to
a larger number would have had more of an affect. What was the value?
Post a fragment of a gc log (-XX:+PrintGCDetails) that shows the concurrent
mode failure. Maybe there's something else going on.

Any help understanding this (concurrent mode failure)?

Hi
Using JDK 1.4.2 on red hat enterprise, haven't yet seen this problem on HP-UX but we are running a different set of settings there.
-verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:+PrintVMOptions -Xms2g -Xmx2g -Xss256k -XX:PermSize=192m -XX:MaxPermSize=192m -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=1 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -XX:+CMSPermGenSweepingEnabled -XX:+UseTLAB -XX:CMSMarkStackSize=8M -XX:+DisableExplicitGC
[GC VM option '+PrintVMOptions'
VM option 'PermSize=192m'
VM option 'MaxPermSize=192m'
VM option '+UseConcMarkSweepGC'
VM option '+UseCMSCompactAtFullCollection'
VM option 'CMSFullGCsBeforeCompaction=1'
VM option '+CMSClassUnloadingEnabled'
VM option '+CMSParallelRemarkEnabled'
VM option '+CMSPermGenSweepingEnabled'
VM option '+UseTLAB'
VM option 'CMSMarkStackSize=8M'
VM option '+DisableExplicitGC'
Here's the snippet from the log file showing the steady growth before finally this particular instance is restarted.
This happens maybe once per day on 1 or 2 of the 30 odd processes, the rest seem to hover just fine around the 1 gig odd mark and lower.
I've read Jon's blogs, they've helped a lot, could still use a hand making more sense of this and since we're in production we can't fiddle too much.
This as simple as saying "you're allocating way too much every so often, find the guilty code"?
Thanks in advance
43608.056: [GC 43608.056: [ParNew: 53120K->0K(53184K), 0.0206690 secs] 692909K->643384K(2097088K), 0.0208900 secs]
Total time for which application threads were stopped: 0.0221600 seconds
Application time: 5.2950500 seconds
43613.373: [GC 43613.374: [ParNew: 52564K->0K(53184K), 0.0190870 secs] 695949K->646749K(2097088K), 0.0193270 secs]
Total time for which application threads were stopped: 0.0206910 seconds
Application time: 10.7180170 seconds
43624.112: [GC 43624.113: [ParNew: 52836K->0K(53184K), 0.0267140 secs] 699586K->653006K(2097088K), 0.0270510 secs]
Total time for which application threads were stopped: 0.0284920 seconds
Application time: 11.8978000 seconds
43636.039: [GC 43636.039: [ParNew: 53120K->0K(53184K), 0.0355370 secs] 706126K->657203K(2097088K), 0.0358680 secs]
Total time for which application threads were stopped: 0.0377480 seconds
Application time: 6.4194150 seconds
43642.496: [GC 43642.496: [ParNew: 52790K->0K(53184K), 0.0249040 secs] 709993K->662587K(2097088K), 0.0251460 secs]
Total time for which application threads were stopped: 0.0265180 seconds
Application time: 2.2362170 seconds
43644.759: [GC 43644.759: [ParNew: 53120K->0K(53184K), 0.0174680 secs] 715707K->665330K(2097088K), 0.0176920 secs]
Total time for which application threads were stopped: 0.0189960 seconds
Application time: 5.5910270 seconds
43650.369: [GC 43650.369: [ParNew: 52989K->0K(53184K), 0.0205030 secs] 718320K->669851K(2097088K), 0.0207310 secs]
Total time for which application threads were stopped: 0.0219250 seconds
Application time: 1.6515340 seconds
43652.043: [GC 43652.043: [ParNew: 53117K->0K(53184K), 0.0160820 secs] 722969K->672528K(2097088K), 0.0163420 secs]
Total time for which application threads were stopped: 0.0176470 seconds
Application time: 8.4822620 seconds
43660.543: [GC 43660.543: [ParNew: 53120K->0K(53184K), 0.0326680 secs] 725648K->680627K(2097088K), 0.0328980 secs]
Total time for which application threads were stopped: 0.0341570 seconds
Application time: 11.0827410 seconds
43671.660: [GC 43671.660: [ParNew: 52560K->0K(53184K), 0.0247530 secs] 733188K->685702K(2097088K), 0.0249980 secs]
Total time for which application threads were stopped: 0.0264230 seconds
Application time: 4.6662200 seconds
43676.353: [GC 43676.353: [ParNew: 52910K->0K(53184K), 0.0206240 secs] 738613K->689324K(2097088K), 0.0208810 secs]
Total time for which application threads were stopped: 0.0223920 seconds
Application time: 6.6525520 seconds
43683.027: [GC 43683.027: [ParNew: 52649K->0K(53184K), 0.0195830 secs] 741973K->692332K(2097088K), 0.0198070 secs]
Total time for which application threads were stopped: 0.0209330 seconds
Application time: 1.5601640 seconds
43684.609: [GC 43684.609: [ParNew: 52508K->0K(53184K), 0.0181740 secs] 744840K->695786K(2097088K), 0.0184020 secs]
Total time for which application threads were stopped: 0.0196160 seconds
Application time: 4.1460570 seconds
43688.774: [GC 43688.775: [ParNew: 53120K->0K(53184K), 0.0203780 secs] 748906K->699231K(2097088K), 0.0206190 secs]
Total time for which application threads were stopped: 0.0218880 seconds
Application time: 5.8668450 seconds
43694.663: [GC 43694.663: [ParNew: 53112K->0K(53184K), 0.0213090 secs] 752344K->703173K(2097088K), 0.0215650 secs]
Total time for which application threads were stopped: 0.0227280 seconds
Application time: 2.7179640 seconds
43697.405: [GC 43697.405: [ParNew: 53120K->0K(53184K), 0.0637350 secs] 756293K->705966K(2097088K), 0.0641660 secs]
Total time for which application threads were stopped: 0.0662140 seconds
Application time: 15.9736600 seconds
43713.444: [GC 43713.444: [ParNew: 53120K->0K(53184K), 0.0280160 secs] 759086K->711062K(2097088K), 0.0282570 secs]
Total time for which application threads were stopped: 0.0296280 seconds
Application time: 3.8590560 seconds
43717.333: [GC 43717.334: [ParNew: 52784K->0K(53184K), 0.0289100 secs] 763846K->715157K(2097088K), 0.0292600 secs]
Total time for which application threads were stopped: 0.0309510 seconds
Application time: 11.0081190 seconds
43728.377: [GC 43728.377: [ParNew: 53058K->0K(53184K), 0.0396730 secs] 768216K->719969K(2097088K), 0.0399280 secs]
Total time for which application threads were stopped: 0.0461840 seconds
Application time: 1.9920560 seconds
43730.420: [GC 43730.420: [ParNew: 53120K->0K(53184K), 0.0195590 secs] 773089K->722236K(2097088K), 0.0198520 secs]
Total time for which application threads were stopped: 0.0305260 seconds
Application time: 9.5348620 seconds
43739.984: [GC 43739.985: [ParNew: 53120K->0K(53184K), 0.0210960 secs] 775356K->725317K(2097088K), 0.0213940 secs]
Total time for which application threads were stopped: 0.0311520 seconds
Application time: 7.3824160 seconds
43747.408: [GC 43747.408: [ParNew: 53120K->0K(53184K), 0.0274210 secs] 778437K->729550K(2097088K), 0.0277470 secs]
Total time for which application threads were stopped: 0.0468940 seconds
Application time: 8.7856540 seconds
43756.227: [GC 43756.227: [ParNew: 53120K->0K(53184K), 0.0252000 secs] 782670K->734045K(2097088K), 0.0254550 secs]
Total time for which application threads were stopped: 0.0315440 seconds
Application time: 5.8074140 seconds
43762.070: [GC 43762.071: [ParNew: 52718K->0K(53184K), 0.0268620 secs] 786764K->740127K(2097088K), 0.0271090 secs]
Total time for which application threads were stopped: 0.0373050 seconds
Application time: 4.4915470 seconds
43766.598: [GC 43766.598: [ParNew: 53117K->0K(53184K), 0.0207740 secs] 793245K->743248K(2097088K), 0.0210680 secs]
Total time for which application threads were stopped: 0.0300810 seconds
Application time: 4.8663360 seconds
43771.504: [GC 43771.504: [ParNew: 53120K->0K(53184K), 0.0201270 secs] 796368K->745712K(2097088K), 0.0204750 secs]
Total time for which application threads were stopped: 0.0388450 seconds
Application time: 8.6802310 seconds
43780.212: [GC 43780.212: [ParNew: 53013K->0K(53184K), 0.0219580 secs] 798725K->749746K(2097088K), 0.0222020 secs]
Total time for which application threads were stopped: 0.0290250 seconds
Application time: 2.1775650 seconds
43782.422: [GC 43782.422: [ParNew: 53120K->0K(53184K), 0.0190600 secs] 802866K->752281K(2097088K), 0.0194110 secs]
Total time for which application threads were stopped: 0.0298550 seconds
Application time: 7.6248630 seconds
43790.075: [GC 43790.075: [ParNew: 52950K->0K(53184K), 0.0212610 secs] 805232K->756051K(2097088K), 0.0214950 secs]
Total time for which application threads were stopped: 0.0298600 seconds
Application time: 12.4794810 seconds
43802.596: [GC 43802.596: [ParNew: 52484K->0K(53184K), 0.0249110 secs] 808535K->761359K(2097088K), 0.0252320 secs]
Total time for which application threads were stopped: 0.0449290 seconds
Application time: 11.7985830 seconds
43814.427: [GC 43814.428: [ParNew: 53120K->0K(53184K), 0.0235200 secs] 814479K->765573K(2097088K), 0.0238420 secs]
Total time for which application threads were stopped: 0.0316530 seconds
Application time: 8.7485040 seconds
43823.211: [GC 43823.212: [ParNew: 52661K->0K(53184K), 0.0291330 secs] 818235K->771406K(2097088K), 0.0294070 secs]
Total time for which application threads were stopped: 0.0419130 seconds
Application time: 6.9341000 seconds
43830.186: [GC 43830.186: [ParNew: 53115K->0K(53184K), 0.0226130 secs] 824522K->776196K(2097088K), 0.0229300 secs]
Total time for which application threads were stopped: 0.0325970 seconds
Application time: 5.7942000 seconds
43836.020: [GC 43836.021: [ParNew: 52733K->0K(53184K), 0.0198250 secs] 828929K->780013K(2097088K), 0.0200940 secs]
Total time for which application threads were stopped: 0.0376430 seconds
Application time: 10.6327130 seconds
43846.675: [GC 43846.675: [ParNew: 52737K->0K(53184K), 0.0246010 secs] 832751K->785407K(2097088K), 0.0248280 secs]
Total time for which application threads were stopped: 0.0261080 seconds
Application time: 14.3177570 seconds
43861.019: [GC 43861.019: [ParNew: 53120K->0K(53184K), 0.0218920 secs] 838527K->789600K(2097088K), 0.0221180 secs]
Total time for which application threads were stopped: 0.0233230 seconds
Application time: 5.3604960 seconds
43866.412: [GC 43866.412: [ParNew: 52743K->0K(53184K), 0.0244120 secs] 842344K->793748K(2097088K), 0.0247340 secs]
Total time for which application threads were stopped: 0.0354410 seconds
Application time: 11.9592830 seconds
43878.415: [GC 43878.416: [ParNew: 52790K->0K(53184K), 0.0184740 secs] 846538K->796512K(2097088K), 0.0189340 secs]
Total time for which application threads were stopped: 0.0383250 seconds
07/09/12 10:33:05 RE-ATTACHING: Row = [enatis.data.per.PerViewRowImpl] Entity = [enatis.data.per.PerEoImpl] State = [CLEAN] with Key = [oracle.jbo.Key[4291000018PZ ]]
Application time: 17.7943560 seconds
43896.238: [GC 43896.238: [ParNew: 52696K->0K(53184K), 0.0260340 secs] 849208K->802267K(2097088K), 0.0263540 secs]
Total time for which application threads were stopped: 0.0348860 seconds
Application time: 8.3590650 seconds
43904.640: [GC 43904.640: [ParNew: 53120K->0K(53184K), 0.0182380 secs] 855387K->805434K(2097088K), 0.0185050 secs]
Total time for which application threads were stopped: 0.0355610 seconds
Application time: 6.7479770 seconds
43911.413: [GC 43911.413: [ParNew: 52766K->0K(53184K), 0.0500520 secs] 858200K->817084K(2097088K), 0.0503110 secs]
Total time for which application threads were stopped: 0.0567100 seconds
Application time: 6.7541480 seconds
43918.228: [GC 43918.228: [ParNew: 53120K->0K(53184K), 0.1331180 secs] 870204K->851295K(2097088K), 0.1333800 secs]
Total time for which application threads were stopped: 0.1432520 seconds
Application time: 3.8274410 seconds
43922.197: [GC 43922.197: [ParNew: 53095K->0K(53184K), 0.1036830 secs] 904391K->875877K(2097088K), 0.1039570 secs]
Total time for which application threads were stopped: 0.1126310 seconds
Application time: 4.9308070 seconds
43927.249: [GC 43927.249: [ParNew: 53120K->0K(53184K), 0.1200000 secs] 928997K->900366K(2097088K), 0.1202910 secs]
Total time for which application threads were stopped: 0.1375030 seconds
Application time: 1.9715840 seconds
43929.348: [GC 43929.349: [ParNew: 53120K->0K(53184K), 0.0779690 secs] 953486K->912753K(2097088K), 0.0783190 secs]
Total time for which application threads were stopped: 0.0854850 seconds
Application time: 4.8621560 seconds
43934.302: [GC 43934.302: [ParNew: 53120K->0K(53184K), 0.1753970 secs] 965873K->938610K(2097088K), 0.1757580 secs]
Total time for which application threads were stopped: 0.1889440 seconds
Application time: 3.1126770 seconds
43937.599: [GC 43937.599: [ParNew: 53120K->0K(53184K), 0.1094920 secs] 991730K->955346K(2097088K), 0.1097650 secs]
Total time for which application threads were stopped: 0.1182550 seconds
Application time: 3.8963880 seconds
43941.623: [GC 43941.623: [ParNew: 53120K->0K(53184K), 0.1480600 secs] 1008466K->976954K(2097088K), 0.1483400 secs]
Total time for which application threads were stopped: 0.1653780 seconds
Application time: 7.1542900 seconds
43948.926: [GC 43948.927: [ParNew: 53120K->0K(53184K), 0.2435230 secs] 1030074K->1014822K(2097088K), 0.2437620 secs]
Total time for which application threads were stopped: 0.2450340 seconds
Application time: 6.8893410 seconds
43956.061: [GC 43956.061: [ParNew: 53120K->0K(53184K), 0.3473630 secs] 1067942K->1050315K(2097088K), 0.3476000 secs]
Total time for which application threads were stopped: 0.3488460 seconds
Application time: 0.0000900 seconds
43956.410: [GC [1 CMS-initial-mark: 1050315K(2043904K)] 1050411K(2097088K), 0.0104150 secs]
Total time for which application threads were stopped: 0.0109390 seconds
43956.420: [CMS-concurrent-mark-start]
43959.018: [CMS-concurrent-mark: 2.597/2.598 secs]
43959.018: [CMS-concurrent-preclean-start]
43959.018: [CMS-concurrent-preclean: 0.000/0.000 secs]
Application time: 6.2326630 seconds
43962.654: [GC43962.654: [Rescan (parallel) , 0.2701930 secs]43962.924: [weak refs processing, 0.5280520 secs]43963.453: [class unloading[Unloading class sun.reflect.GeneratedMethodAccessor1120]
[Unloading class sun.reflect.GeneratedMethodAccessor1129]
[Unloading class sun.reflect.GeneratedMethodAccessor1128]
[Unloading class sun.reflect.GeneratedMethodAccessor1143]
[Unloading class sun.reflect.GeneratedMethodAccessor1137]
[Unloading class sun.reflect.GeneratedMethodAccessor1124]
[Unloading class sun.reflect.GeneratedMethodAccessor1125]
[Unloading class sun.reflect.GeneratedMethodAccessor1138]
[Unloading class sun.reflect.GeneratedMethodAccessor1152]
[Unloading class sun.reflect.GeneratedMethodAccessor1155]
[Unloading class sun.reflect.GeneratedMethodAccessor1136]
[Unloading class sun.reflect.GeneratedConstructorAccessor1022]
[Unloading class sun.reflect.GeneratedMethodAccessor1118]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor163]
[Unloading class sun.reflect.GeneratedMethodAccessor1126]
[Unloading class sun.reflect.GeneratedMethodAccessor1117]
[Unloading class sun.reflect.GeneratedMethodAccessor1116]
[Unloading class sun.reflect.GeneratedMethodAccessor1135]
[Unloading class sun.reflect.GeneratedMethodAccessor1119]
[Unloading class sun.reflect.GeneratedMethodAccessor1127]
, 0.0499860 secs]43963.503: [scrub symbol & string tables, 0.0332330 secs] [1 CMS-remark: 1050315K(2043904K)] 1096705K(2097088K), 0.9034220 secs]
Total time for which application threads were stopped: 0.9046590 seconds
43963.558: [CMS-concurrent-sweep-start]
Application time: 0.4523320 seconds
43964.016: [GC 43964.016: [ParNew: 53120K->0K(53184K), 0.2078860 secs] 955304K->935315K(2097088K), 0.2081220 secs]
Total time for which application threads were stopped: 0.2142380 seconds
43966.450: [CMS-concurrent-sweep: 2.678/2.893 secs]
43966.451: [CMS-concurrent-reset-start]
43966.494: [CMS-concurrent-reset: 0.043/0.043 secs]
Application time: 4.2227670 seconds
43968.457: [GC 43968.457: [ParNew: 53120K->0K(53184K), 0.1068680 secs] 436338K->405964K(2097088K), 0.1071390 secs]
Total time for which application threads were stopped: 0.1171590 seconds
Application time: 6.3307910 seconds
43974.905: [GC 43974.905: [ParNew: 53120K->0K(53184K), 0.1405810 secs] 459084K->438194K(2097088K), 0.1408370 secs]
Total time for which application threads were stopped: 0.1511130 seconds
Application time: 2.1166730 seconds
43977.180: [GC 43977.180: [ParNew: 53087K->0K(53184K), 0.0598370 secs] 491282K->450305K(2097088K), 0.0601160 secs]
Total time for which application threads were stopped: 0.0775190 seconds
Application time: 4.7535500 seconds
43982.000: [GC 43982.001: [ParNew: 53120K->0K(53184K), 0.1475350 secs] 503425K->478341K(2097088K), 0.1477800 secs]
Total time for which application threads were stopped: 0.1543110 seconds
Application time: 5.6461180 seconds
43987.804: [GC 43987.805: [ParNew: 53120K->0K(53184K), 0.1502640 secs] 531461K->510120K(2097088K), 0.1505320 secs]
Total time for which application threads were stopped: 0.1605520 seconds
Application time: 4.6071550 seconds
43992.571: [GC 43992.571: [ParNew: 53120K->0K(53184K), 0.1269550 secs] 563240K->536014K(2097088K), 0.1272820 secs]
Total time for which application threads were stopped: 0.1360560 seconds
Application time: 0.9160970 seconds
43993.633: [GC 43993.634: [ParNew: 52889K->0K(53184K), 0.0339760 secs] 588904K->543199K(2097088K), 0.0343070 secs]
Total time for which application threads were stopped: 0.0532110 seconds
Application time: 3.7609360 seconds
43997.435: [GC 43997.436: [ParNew: 53120K->0K(53184K), 0.0976550 secs] 596319K->564648K(2097088K), 0.0979260 secs]
Total time for which application threads were stopped: 0.1045230 seconds
Application time: 3.6492120 seconds
44001.194: [GC 44001.194: [ParNew: 53120K->0K(53184K), 0.0945310 secs] 617768K->583964K(2097088K), 0.0947840 secs]
Total time for which application threads were stopped: 0.1057890 seconds
Application time: 3.4590660 seconds
44004.757: [GC 44004.757: [ParNew: 52427K->0K(53184K), 0.1735420 secs] 636392K->606952K(2097088K), 0.1739010 secs]
Total time for which application threads were stopped: 0.1833310 seconds
Application time: 3.9927710 seconds
44008.941: [GC 44008.942: [ParNew: 53101K->0K(53184K), 0.0994210 secs] 660053K->631526K(2097088K), 0.0997020 secs]
Total time for which application threads were stopped: 0.1171670 seconds
Application time: 3.7763590 seconds
44012.819: [GC 44012.819: [ParNew: 53064K->0K(53184K), 0.1047140 secs] 684591K->653119K(2097088K), 0.1049430 secs]
Total time for which application threads were stopped: 0.1061730 seconds
Application time: 6.3306170 seconds
44019.256: [GC 44019.256: [ParNew: 53120K->0K(53184K), 0.1091900 secs] 706239K->677429K(2097088K), 0.1094120 secs]
Total time for which application threads were stopped: 0.1107130 seconds
Application time: 3.3295540 seconds
44022.696: [GC 44022.696: [ParNew: 53120K->0K(53184K), 0.0845340 secs] 730549K->692228K(2097088K), 0.0847660 secs]
Total time for which application threads were stopped: 0.0859710 seconds
Application time: 5.1413240 seconds
44027.924: [GC 44027.924: [ParNew: 53120K->0K(53184K), 0.1623040 secs] 745348K->720629K(2097088K), 0.1625430 secs]
Total time for which application threads were stopped: 0.1638290 seconds
Application time: 4.4772970 seconds
44032.565: [GC 44032.565: [ParNew: 53114K->0K(53184K), 0.1628460 secs] 773743K->744931K(2097088K), 0.1631360 secs]
Total time for which application threads were stopped: 0.1646070 seconds
Application time: 3.1443530 seconds
44035.874: [GC 44035.874: [ParNew: 53120K->0K(53184K), 0.1092600 secs] 798051K->761281K(2097088K), 0.1095100 secs]
Total time for which application threads were stopped: 0.1108940 seconds
Application time: 3.6947170 seconds
44039.680: [GC 44039.680: [ParNew: 53120K->0K(53184K), 0.1451310 secs] 814401K->780990K(2097088K), 0.1453770 secs]
Total time for which application threads were stopped: 0.1468070 seconds
Application time: 1.5045480 seconds
44041.331: [GC 44041.331: [ParNew: 52921K->0K(53184K), 0.0628230 secs] 833912K->789345K(2097088K), 0.0630510 secs]
Total time for which application threads were stopped: 0.0642340 seconds
Application time: 3.5619510 seconds
44044.957: [GC 44044.957: [ParNew: 53120K->0K(53184K), 0.1462690 secs] 842465K->810010K(2097088K), 0.1465080 secs]
Total time for which application threads were stopped: 0.1477730 seconds
Application time: 2.6063530 seconds
44047.712: [GC 44047.712: [ParNew: 52725K->0K(53184K), 0.1095910 secs] 862735K->828029K(2097088K), 0.1098200 secs]
Total time for which application threads were stopped: 0.1110810 seconds
Application time: 2.8685230 seconds
44050.691: [GC 44050.691: [ParNew: 53120K->0K(53184K), 0.1112890 secs] 881149K->844230K(2097088K), 0.1115370 secs]
Total time for which application threads were stopped: 0.1128490 seconds
Application time: 2.6718830 seconds
44053.476: [GC 44053.476: [ParNew: 52804K->0K(53184K), 0.1049940 secs] 897034K->859437K(2097088K), 0.1052220 secs]
Total time for which application threads were stopped: 0.1065130 seconds
Application time: 2.0643630 seconds
44055.647: [GC 44055.647: [ParNew: 53003K->0K(53184K), 0.0905500 secs] 912440K->872838K(2097088K), 0.0908130 secs]
Total time for which application threads were stopped: 0.0921630 seconds
Application time: 3.8447810 seconds
44059.584: [GC 44059.584: [ParNew: 53120K->0K(53184K), 0.2533330 secs] 925958K->901400K(2097088K), 0.2535730 secs]
Total time for which application threads were stopped: 0.2547760 seconds
Application time: 2.8353870 seconds
44062.675: [GC 44062.675: [ParNew: 52520K->0K(53184K), 0.1265640 secs] 953920K->916558K(2097088K), 0.1268670 secs]
Total time for which application threads were stopped: 0.1283310 seconds
Application time: 2.3321870 seconds
44065.135: [GC 44065.135: [ParNew: 53120K->0K(53184K), 0.1436290 secs] 969678K->930508K(2097088K), 0.1438740 secs]
Total time for which application threads were stopped: 0.1452900 seconds
Application time: 1.1871350 seconds
44066.473: [GC 44066.473: [ParNew: 53118K->0K(53184K), 0.1240660 secs] 983627K->942342K(2097088K), 0.1243940 secs]
Total time for which application threads were stopped: 0.1308180 seconds
Application time: 4.1949150 seconds
44070.802: [GC 44070.803: [ParNew: 52504K->0K(53184K), 0.2460170 secs] 994847K->966218K(2097088K), 0.2462940 secs]
Total time for which application threads were stopped: 0.2565070 seconds
Application time: 2.3949770 seconds
44073.452: [GC 44073.453: [ParNew: 53120K->0K(53184K), 0.1351640 secs] 1019338K->980150K(2097088K), 0.1354470 secs]
Total time for which application threads were stopped: 0.1439290 seconds
Application time: 4.3352340 seconds
44077.940: [GC 44077.941: [ParNew: 53101K->0K(53184K), 0.2399110 secs] 1033251K->1003714K(2097088K), 0.2402050 secs]
Total time for which application threads were stopped: 0.2574500 seconds
Application time: 3.8057930 seconds
44081.994: [GC 44081.994: [ParNew: 53120K->0K(53184K), 0.2154480 secs] 1056834K->1022831K(2097088K), 0.2157980 secs]
Total time for which application threads were stopped: 0.2234510 seconds
Application time: 0.0000730 seconds
44082.224: [GC [1 CMS-initial-mark: 1022831K(2043904K)] 1022927K(2097088K), 0.0130050 secs]
Total time for which application threads were stopped: 0.0266360 seconds
44082.237: [CMS-concurrent-mark-start]
Application time: 5.4437320 seconds
44087.689: [GC 44087.689: [ParNew: 53120K->0K(53184K), 0.3132650 secs] 1075951K->1048686K(2097088K), 0.3135120 secs]
Total time for which application threads were stopped: 0.3218810 seconds
44090.635: [CMS-concurrent-mark: 8.076/8.398 secs]
44090.635: [CMS-concurrent-preclean-start]
44090.635: [CMS-concurrent-preclean: 0.000/0.000 secs]
Application time: 4.1030320 seconds
44092.146: [GC 44092.146: [ParNew: 53120K->0K(53184K), 0.2526600 secs] 1101806K->1068987K(2097088K), 0.2529880 secs]
Total time for which application threads were stopped: 0.2930800 seconds
Application time: 2.8570730 seconds
44095.286: [GC 44095.287: [ParNew: 53120K->0K(53184K), 0.1991800 secs] 1122107K->1085754K(2097088K), 0.1994280 secs]
Total time for which application threads were stopped: 0.2299700 seconds
Application time: 1.7219600 seconds
44097.219: [GC 44097.219: [ParNew: 53113K->0K(53184K), 0.1431840 secs] 1138867K->1098491K(2097088K), 0.1434430 secs]
Total time for which application threads were stopped: 0.1541960 seconds
Application time: 0.5872220 seconds
44097.997: [GC 44097.997: [ParNew: 52671K->0K(53184K), 0.1976300 secs] 1151163K->1109541K(2097088K), 0.1978960 secs]
Total time for which application threads were stopped: 0.2452470 seconds
Application time: 2.8295910 seconds
44101.047: [GC 44101.047: [ParNew: 53120K->0K(53184K), 0.1880870 secs] 1162661K->1125669K(2097088K), 0.1883390 secs]
Total time for which application threads were stopped: 0.2109970 seconds
Application time: 4.4293580 seconds
44105.729: [GC 44105.730: [ParNew: 53120K->0K(53184K), 0.2848100 secs] 1178789K->1148409K(2097088K), 0.2850410 secs]
Total time for which application threads were stopped: 0.3494680 seconds
Application time: 4.5104910 seconds
44110.528: [GC 44110.528: [ParNew: 53120K->0K(53184K), 0.2996120 secs] 1201529K->1171112K(2097088K), 0.2999760 secs]
Total time for which application threads were stopped: 0.3026420 seconds
Application time: 3.7846210 seconds
44114.625: [GC 44114.625: [ParNew: 53120K->0K(53184K), 0.2557830 secs] 1224232K->1191281K(2097088K), 0.2560210 secs]
Total time for which application threads were stopped: 0.2687430 seconds
Application time: 2.6413800 seconds
44117.528: [GC 44117.528: [ParNew: 53120K->0K(53184K), 0.1739240 secs] 1244401K->1205940K(2097088K), 0.1741620 secs]
Total time for which application threads were stopped: 0.1793890 seconds
Application time: 4.7872250 seconds
44122.492: [GC 44122.492: [ParNew: 53120K->0K(53184K), 0.3045890 secs] 1259060K->1231422K(2097088K), 0.3048290 secs]
Total time for which application threads were stopped: 0.3072850 seconds
Application time: 6.2000570 seconds
44129.021: [GC 44129.021: [ParNew: 53120K->0K(53184K), 0.3628320 secs] 1284542K->1261727K(2097088K), 0.3630650 secs]
Total time for which application threads were stopped: 0.3874490 seconds
Application time: 4.9197640 seconds
44134.341: [GC 44134.341: [ParNew: 53120K->0K(53184K), 0.3119360 secs] 1314847K->1287281K(2097088K), 0.3121950 secs]
Total time for which application threads were stopped: 0.3488480 seconds
Application time: 4.4958250 seconds
44139.202: [GC 44139.202: [ParNew: 53120K->0K(53184K), 0.2921870 secs] 1340401K->1313317K(2097088K), 0.2924300 secs]
Total time for which application threads were stopped: 0.3458420 seconds
Application time: 2.9204910 seconds
44142.417: [GC 44142.417: [ParNew: 53120K->0K(53184K), 0.1904860 secs] 1366437K->1329808K(2097088K), 0.1907150 secs]
Total time for which application threads were stopped: 0.1919930 seconds
Application time: 3.9970250 seconds
44146.636: [GC 44146.637: [ParNew: 53019K->0K(53184K), 0.2588980 secs] 1382828K->1351164K(2097088K), 0.2591250 secs]
Total time for which application threads were stopped: 0.2910660 seconds
Application time: 1.3237030 seconds
44148.227: [GC 44148.227: [ParNew: 53120K->0K(53184K), 0.0949690 secs] 1404284K->1358754K(2097088K), 0.0952310 secs]
Total time for which application threads were stopped: 0.1024530 seconds
Application time: 5.8729190 seconds
44154.233: [GC 44154.233: [ParNew: 53120K->0K(53184K), 0.3753130 secs] 1411874K->1389102K(2097088K), 0.3755370 secs]
Total time for which application threads were stopped: 0.4137460 seconds
Application time: 2.8788090 seconds
44157.526: [GC 44157.526: [ParNew: 53120K->0K(53184K), 0.1987470 secs] 1442222K->1406367K(2097088K), 0.1990490 secs]
Total time for which application threads were stopped: 0.2371510 seconds
Application time: 4.9930280 seconds
44162.732: [GC 44162.732: [ParNew: 53120K->0K(53184K), 0.3251770 secs] 1459487K->1431734K(2097088K), 0.3254560 secs]
Total time for which application threads were stopped: 0.3392560 seconds
Application time: 5.3928510 seconds
44168.505: [GC 44168.506: [ParNew: 53120K->0K(53184K), 0.3478970 secs] 1484854K->1461670K(2097088K), 0.3481250 secs]
Total time for which application threads were stopped: 0.4034160 seconds
Application time: 4.3012280 seconds
44173.178: [GC 44173.178: [ParNew: 53120K->0K(53184K), 0.2655450 secs] 1514790K->1483729K(2097088K), 0.2657630 secs]
Total time for which application threads were stopped: 0.2891130 seconds
Application time: 5.2461070 seconds
44178.692: [GC 44178.692: [ParNew: 53120K->0K(53184K), 0.3227640 secs] 1536849K->1509975K(2097088K), 0.3230000 secs]
Total time for which application threads were stopped: 0.3243440 seconds
Application time: 3.9882730 seconds
44183.019: [GC 44183.019: [ParNew: 53120K->0K(53184K), 0.2809640 secs] 1563095K->1533122K(2097088K), 0.2812210 secs]
Total time for which application threads were stopped: 0.2974160 seconds
Application time: 4.9134590 seconds
44188.246: [GC 44188.247: [ParNew: 53120K->0K(53184K), 0.3009460 secs] 1586242K->1558260K(2097088K), 0.3012500 secs]
Total time for which application threads were stopped: 0.3337030 seconds
Application time: 0.0064750 seconds
44188.555: [GC44188.556: [Rescan (parallel) , 0.2483640 secs]44188.804: [weak refs processing, 0.7967400 secs]44189.601: [class unloading[Unloading class sun.reflect.GeneratedMethodAccessor1184]
[Unloading class sun.reflect.GeneratedMethodAccessor1183]
[Unloading class sun.reflect.GeneratedMethodAccessor1179]
[Unloading class sun.reflect.GeneratedMethodAccessor1185]
[Unloading class sun.reflect.GeneratedMethodAccessor1182]
[Unloading class sun.reflect.GeneratedMethodAccessor1181]
[Unloading class sun.reflect.GeneratedMethodAccessor1180]
, 0.0502070 secs]44189.651: [scrub symbol & string tables, 0.0314740 secs] [1 CMS-remark: 1558260K(2043904K)] 1561812K(2097088K), 1.1403420 secs]
Total time for which application threads were stopped: 1.1420510 seconds
44189.698: [CMS-concurrent-sweep-start]
Application time: 2.0153710 seconds
44191.713: [GC 44191.714: [ParNew: 53015K->0K(53184K), 0.4850080 secs] 1474478K->1433341K(2097088K), 0.4853390 secs]
Total time for which application threads were stopped: 0.4872470 seconds
44192.717: [CMS-concurrent-sweep: 2.530/3.018 secs]
44192.717: [CMS-concurrent-reset-start]
44192.755: [CMS-concurrent-reset: 0.038/0.038 secs]
Application time: 2.5580970 seconds
44194.758: [GC [1 CMS-initial-mark: 1401675K(2043904K)] 1453513K(2097088K), 0.0884220 secs]
Total time for which application threads were stopped: 0.0896100 seconds
44194.847: [CMS-concurrent-mark-start]
Application time: 0.1899980 seconds
44195.038: [GC 44195.038: [ParNew: 53120K->0K(53184K

Refer to http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4758307.
The symptom is similar and you can see various useful/successful workarounds.

I have an EX495 WHS and I get an Error Code. 1 aqua and 4 red : Recovery Mode Failure.

Hi,
I have an MediaSmart Server EX495 WHS and I get an Error Code. 1 aqua and 4 red : Recovery Mode Failure. I tried Server recovery to no avail. This all started when I lost connection to the Server via Control Panel - it stopped connecting to the network totally.
Can anyone help?
Thanks in advance, John

I had the same problem and am now in a worse situation. Saw your posting and hoped someone would have advised what to do. When I re-boot my EX495, the Health indicator light comes up red. Pressing the status/recovery button causes the Health light to flash 1 Blue, 4 Red: Recovery Mode Failure. User manual says contact HP. HP says I'm out of warranty. Does anyone out there have a suggestion what I can do/try? Is it worth paying HP for tech support on the issue?

Permanent Generation and GC

Our server application runs fine using ConcMarkSweep GC (or so we thought) on Java 1.5 with
subsecond pauses.
However, we see the permanent generation growing slowly per request, e.g. by 2k.
The unfortunate effect is that when the permanent generation is full, a Full GC cycle takes place
and the 15 seconds spent on that are unacceptable to our client. Ouch.
Q1: Is there a way to avoid the Full GC on the permanent GC? Why isn't it included in the CMS GC
cycles?
From the little information I can find on the permanent generation, it contains
- class data
- interned Strings
Q2: What could be the cause of the substantial growth in the permanent generation.
(I checked the XML parsing, and the invocations of String.intern are not sufficient to explain the
situation). We do a lot of different things with third party jars, XML, XLST, jdom, JDBC (Sybase jconnect), some object (de)serialization and zipping, SSL, Tomcat.. No RMI/EJB.
Any wll-known offenders?
Q3: Are there any tools other than jmap -heap to monitor the object in the permanent generation
specifically? Ideally a full profiling JMVTI app or tracing the calls to the String.intern.
Q4: Is there anything else I'm missing? Have there been changes in this obscure corner of the JVM since 1.4.2?

Probably nothing new for you, up until Q3...
The unfortunate effect is that when the permanent
generation is full, a Full GC cycle takes place
and the 15 seconds spent on that are unacceptable to
our client. Ouch.Have you looked at this tuning doc?
http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html
Q1: Is there a way to avoid the Full GC on the
permanent GC? Why isn't it included in the CMS GC
cycles?For more information on heap sizing and garbage collection issues,
check the white papers here:
http://developers.sun.com/techtopics/mobility/midp/articles/garbagecollection2/
The benchmark application they describe is rather specialized, but the
paper contains some very useful information about heap layout and
garbage collection in general.
From the little information I can find on the
permanent generation, it contains
- class data
- interned Strings
Q2: What could be the cause of the substantial growth
in the permanent generation.
(I checked the XML parsing, and the invocations of
String.intern are not sufficient to explain the
situation). We do a lot of different things with
third party jars, XML, XLST, jdom, JDBC (Sybase
jconnect), some object (de)serialization and zipping,
SSL, Tomcat.. No RMI/EJB.
Any wll-known offenders?See Q3. Hopefully one of the queries listed in the README will help track down the anomaly.
Q3: Are there any tools other than jmap -heap to
monitor the object in the permanent generation
specifically? Ideally a full profiling JMVTI app or
tracing the calls to the String.intern.Have you looked at HAT (Heap Analysis Tool)?
https://hat.dev.java.net
https://hat.dev.java.net/doc/README.html
Hope this helps.
"Troubleshooting Guide for J2SE 5.0",
http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf

Difference between maximum concurrency and batch mode in JDBC

Hi All,
Could som one tell me the difference between maximum concurrency and batch mode in receiver JDBC adapter?
Regards,
XIer

Hi,
The Max Concurrancy and Batch mode are used for performance tunning for JDBC adapter.
Max Concurrancy is possible upto 5 only. so even if you will be taking up 10 it will by-default take 5.
The number of records being processed depend on the size of each record. for e.g Per minit XI Is Processing 50 Records,It will Take lot of time to process all Records.
So if you set Max concurrancy as 5 then at time 5 messages will be processed simultaneously.
In Batch mode it will, collect the multiple SQL statements that will be created for records to be processed and processed at a time. It will improve performance considerably.
But Batch Mode is possible with only INSERT, UPDATE, DELETE etc.
The Stored procedures can be executed with max concurrency
Thus it completely depends that on the no of records to be processed at a time and volume of Mesasges.
Thanks
Swarup

HT201210 i have a 4th generation 8gb and its stuck in recovery mode. I have tried everything i even got a new computer. When i try to recover it it has to do an update and when its done and error pops up. What do i do i seriously need help

i have a 4th generation 8gb and its stuck in recovery mode. I have tried everything i even got a new computer. When i try to recover it it has to do an update and when its done and error pops up. What do i do i seriously need help

You said "and error pops up. " Just what does the error message say?

CSM concurrent bridge and router mode

Hi,
Is it possible on the CSM to use bridge and router mode at the same time ? Or is it only router mode or only bridge mode ?
E.g. in the example below, when using HTTPS entering the vlan 3 , it will be bridged to vlan 3....But when using HTTP entering vlan 3...it will be routed to vlan 4... Will that work ?
Thanks
vlan 3 client
ip address 3.3.3.1 255.255.255.0
vlan 3 server
ip address 3.3.3.1 255.255.255.0
vlan 4 server
ip address 4.4.4.1 255.255.255.0
vserver HTTPS
vlan 3
virtual 3.3.3.10 tcp https
serverfarm HTTPS
serverfarm HTTPS
no nat server
no nat client
real 3.3.3.11
inservice
real 3.3.3.12
inservice
vserver HTTP
vlan 3
virtual 3.3.3.11 tcp http
serverfarm HTTP
serverfarm HTTP
nat server
no nat client
real 4.4.4.10
inservice
real 4.4.4.11
inservice

HI Michel,
first of all you can run bridged and routed mode at the same time but you can not define the same vlan as client and server. If you would change the above config from vlan 3 server to vlan 30 server and place the reals in vlan 30 it will work. A proper layer 2 configuration is for sure the prerequisit.
Kind regards,
Joerg

How to diagnose increase in permanent generation?

Hello,
My team is observing the following problem which is important to us and that we
did not yet manage to identify:
We observe from the gc traces (see extracts below) that the permanent memory
size keeps increasing slowly (to the point were an java.lang.OutOfMemoryError is
thrown) but the -verbose:class traces do not show anymore class loading after
the system "warmup" phase. This was observed using Sun's JDK version 1.4.1.01.04 on
HPUX, with the following JVM options java -verbose:gc
-Xverbosegc:file=/tmp/gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
-XX:+PrintHeapAtGC -Xms256M -Xmx256M -XX:NewSize=16M
-XX:MaxNewSize=16M -XX:SurvivorRatio=2 -XX:+UseConcMarkSweepGC
I would need help to answer the following questions:
1- what other actions than class loading could possibly explain an increase of
used space in the permanent generation? The GC FAQ says "The permanent
generation is used to hold reflective of the VM itself such as class objects and
method objects". Could use of JNI also affect this space?
2- how can I configure or instrument the JVM 1.4.1 to track cause of increase of
used space in the permanent generation? The program we are testing uses 500 MB
RAM and we often lack physical space on the server to make use of standard
profiler tools such as JProbe or -hprof standard JVM option: HPUX JVM complains
about insufficient MAXDSIZE kernel parameters that is already set to 512 MB, and
plenty swap space is available)
3- In which version are the "-XX:+TraceClassloading and
-XX:+TraceClassUnloading" options supported? I read from
http://java.sun.com/docs/hotspot/gc1.4.2/faq.html that the options
"-XX:+TraceClassloading and -XX:+TraceClassUnloading" should trace class
loading, however they don't seem available under JDK 1.4.1 on Windows, and they
are not either documented in the toolsdocs for JDK 1.4.2
4- What does -verbose:class option really shows? The toolsdoc says
"-verbose:class Display information about each class loaded.". Would this also
display classes dynamically generated such as dynamic proxies (or classes
loading from custom classloaders)? I suspect this only displays class loaded
from the system classloader...
Many thanks for your help,
Regards,
Guillaume Berche.
19735.5: [DefNew: 10511K->2379K(12288K), 0.1241661 secs] 160415K->152624K(258048K) Heap after GC invocations=11590:
Heap
def new generation total 12288K, used 2379K [77c00000, 78c00000, 78c00000)
eden space 8192K, 0% used [77c00000, 77c00000, 78400000)
from space 4096K, 58% used [78400000, 78652e48, 78800000)
to space 4096K, 0% used [78800000, 78800000, 78c00000)
concurrent mark-sweep generation total 245760K, used 150245K [68c00000, 77c00000, 77c00000)
CompactibleFreeListSpace space 245760K, 61% used [68c00000, 77c00000)
concurrent-mark-sweep perm gen total 43772K, used 33105K [64c00000, 676bf000, 68c00000)
CompactibleFreeListSpace space 43772K, 75% used [64c00000, 676bf000)
} , 0.1262338 secs]
19737: [GC {Heap before GC invocations=11590:
Heap
def new generation   total 12288K, used 10571K [77c00000, 78c00000, 78c00000)
eden space   8192K, 100% used [77c00000, 78400000, 78400000)
from space   4096K, 58% used [78400000, 78652e48, 78800000)
to   space   4096K,   0% used [78800000, 78800000, 78c00000)
concurrent mark-sweep generation total 245760K, used 150245K [68c00000, 77c00000, 77c00000)
CompactibleFreeListSpace space 245760K, 61% used [68c00000, 77c00000)
concurrent-mark-sweep perm gen total 43772K, used 33106K [64c00000, 676bf000, 68c00000)
CompactibleFreeListSpace space 43772K, 75% used [64c00000, 676bf000)
} , 0.1262338 secs]

I did check the JDK bug database, and you are right, String.intern() does indeed seem to increase size of permanent generation (argnn!)
Have you experienced cases in which "internalised" String are prevented by the JVM 1.4.1 from being GCed?
I saw an old bug on JDK 1.1 similar to that:
http://developer.java.sun.com/developer/bugParade/bugs/4497186.html
Thanks for your help,
Guillaume.

The difference between VTP server and transparent mode on Catalyst Switch.

Hello
I have a question about the difference between VTP server mode and VTP transparent mode on general catalyst switch.
Basically VTP server mode can create and modify VLAN configuration but actually there is not any VLAN configuration through running-config, is it true? When I checked it on Cat3550, certainly there is not VLAN configuration on VTP server mode. But VTP transparent can create VLAN and configuration but does not synchronize with other switch VLAN status. I appreciate any related information and reason of the VTP server mode specification, thank you very much.
[VTP Transparent mode]
3550#sh vtp status
VTP Version : 2
Configuration Revision : 0
Maximum VLANs supported locally : 1005
Number of existing VLANs : 27
VTP Operating Mode : Transparent
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
*omit
3550#
3550#sh run
Building configuration...
*omit
vlan 99
name TEST-VLAN
[VTP Server mode]
3550#sh vtp status
VTP Version : 2
Configuration Revision : 0
Maximum VLANs supported locally : 1005
Number of existing VLANs : 27
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
*omit
3550#
3550#sh run
Building configuration...
*no VLAN like above configuration on VTP transparent mode.
Best Regards,
Masanobu Hiyoshi

Hi mhiyoshi,
3550#sh vtp status
VTP Version : 2
Configuration Revision : 0
Maximum VLANs supported locally : 1005
Number of existing VLANs : 27
VTP Operating Mode : Transparent
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
*omit
3550#
3550#sh run
Building configuration...
*omit
vlan 99
name TEST-VLAN
The above out put indicates that Vlan is created and then mode changed to transparent. i.e why revision no is 0.
3550#sh vtp status
VTP Version : 2
Configuration Revision : 0
Maximum VLANs supported locally : 1005
Number of existing VLANs : 27
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
*omit
3550#
3550#sh run
Building configuration...
*no VLAN like above configuration on VTP transparent mode.
This indicates that vlan never created in server mode nor learnt from another switch as revision no is 0

How to put ipod 3rd generation on the disk mode manually?

how to put ipod 3rd generation on the disk mode manually?

I am having this problem as well. I've tried dozens of times at this point. I can produce the apple icon but when I release and press play+select it just reverts back to the red x. The red x is the whole reason I'm trying to get this into disk mode. I need to reset it but I can't until it is in disk mode. I've tried the flat surface, one finger etc etc but disk mode still eludes me. Any other thoughts?

Second generation ipod and now when I turn it stays on the screen where it says the date and time and try to log in to use it and I can not to do.

second generation ipod and now when I turn it stays on the screen where it says the date and time and try to log in to use it and I can not to do.

Try:
- Reset the iOS device. Nothing will be lost
Reset iOS device: Hold down the On/Off button and the Home button at the same time for at
least ten seconds, until the Apple logo appears.
- Restore from backup. See:
iOS: How to back up
. Place the iOS device in Recovery Mode if necessary to allow the restore.
- Restore to factory settings/new iOS device.
If still problem, make an appointment at the Genius Bar of an Apple store since it appears you have a hardware problem.
Apple Retail Store - Genius Bar

Permanent Generation GC and concurrent mode failures

Similar Messages

Maybe you are looking for