Full Garbage Collection Problem

Hi All,
We are working on NetWeaver Application Server JAVA 7.0
I am getting an error message in of the the EWA reports for JAVA system. The red alert says as below:
The maximum ratio of full garbage collections to total garbage collections in the reported interval was higher than 90%.
In order to solve the above problem, I increased Heap Memory for all JAVA Server nodes to 3072 (earlier it was 2048 for all the server nodes). However, still I am getting same error in EWA report.
Can any one help me in further analysing and solving the above problem?
Your help is appreciated.

Here are links to some of the tools. I have worked with [IBM GC for IBM JVM|http://www.ibm.com/developerworks/java/library/j-ibmtools2/index.html]. You may have to try others that can read Sun JVM's GC log.
http://www.tagtraum.com/gcviewer.html
http://www.yourkit.com/overview/index.jsp
https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPJMETER
http://java.sun.com/performance/jvmstat/visualgc.html

Similar Messages

Can I force full garbage collection?

Hi, my program is memory bound , as users load more files more memory is required , if the user decides to load a very large number of files they will eventually run out of memory. So I am trying to detect when there is less 15% of heap memory left, then force garbage collection and if it cant free up more than 15% of the heap I will stop the user from loading any more files. But the problem is though I call System.gc() to try and force a full garbage collect it rarely
retrieve enough memory to get below the 15% limit. But using the Yourkit Profiler I can select the Force GarbageCollection option and this always manages to free up more memory to get the figure under the 15% limit. in support of this I found that sometimes my program stop me loading more files when there is still quite a bit available.
So my questions are.
1. I know System.gc() is only hint to garbage collect, but the docs imply it only replys after the garbage collection (if any) has been done, is this right or do I have to wait.
2. Is there any way to Force complete Garbage Collectionas profiler appears to do.
3. is there a VM option I could set instead to force the JVM to completely garbage collect at say 83% so that if I then polled that 85% of heap was being used I would know that it really was, and I wouldnt need to bother trying to garbage collect further. (Im using Suns 1.6. JVM on Windows and Linux, and Apples 1.5 or 1.6 JVM on Macs)
public static void checkMemoryWhilstLoadingFiles() throws LowMemoryException
        MemoryUsage mu = ManagementFactory.getMemoryMXBean().getHeapMemoryUsage();
        //No max defined future proofing
        if(mu.getMax()==-1)
            return;
        if (mu.getUsed() > (mu.getMax() * 0.85f))
            MainWindow.logger.warning("Memory low:" + mu);
            System.gc();
            MainWindow.logger.warning("Memory low gc1:" + ManagementFactory.getMemoryMXBean().getHeapMemoryUsage());
            System.gc();
            MainWindow.logger.warning("Memory low gc2:" + ManagementFactory.getMemoryMXBean().getHeapMemoryUsage());
            System.gc();
            MainWindow.logger.warning("Memory low gc3:" + ManagementFactory.getMemoryMXBean().getHeapMemoryUsage());
            mu = ManagementFactory.getMemoryMXBean().getHeapMemoryUsage();
            if (mu.getUsed() > (mu.getMax() * 0.85f))
                MainWindow.logger.severe("Memory too low:" + mu);
                throw new LowMemoryException("Running out of memory:"+mu.getUsed());
            else
                 MainWindow.logger.warning("Memory usage reduced to:" + mu);
    }thanks for any help Paul
Edited by: paultaylor on 27-Jun-2008 11:10

On all of the current Sun HotSpot JVM's, calling System.gc() will cause a full compacting collection. Unless you have -XX:+DisableExplicitGC on your command line, in which case the call is a noop. Or if you are running the mostly-concurrent collector (-XX:+UseConcMarkSweepGC) and have the -XX:+ExplicitGCInvokesConcurrent flag on your command line, in which case calling System.gc() will start a concurrent collection (and the calling thread will block until the cycle is finished).
But calling System.gc() isn't enough to recover all the space that might be recovered. For example, System.gc() will identify objects that are unreferenced but need to have their finalize() methods called before their space becomes available again. So one call to System.gc() won't recover their space. Those finalize() methods need some cycles to run in, so back-to-back (or back-to-back-to-back :-) calls to System.gc() won't help. If you use a lot of finalize() methods, you should leave a lot of time for the finalize() methods to run between the calls to System.gc(). (Better would be to convert your code to use WeakReferences and run your own reference processing queues, and then you could tell when you were done processing references. But that's real work.) Some people try calling System.runFinalization() and wait for that to return, but that has at least two failure modes (details left to the reader).
In addition, there are details like: if there is still 15% of the heap free, then we won't aggressively clear SoftReferences when you call System.gc(). We might if you waited until the heap was full and we collected it on our own, since we know how much free space there will be after a collection at the point where we are choosing which SoftReferences to clear, and use that to decide how aggressively to clear SoftReferences.
There is no method to force the collector to do a compacting collection at, say 85% full. There is an option to have the mostly-concurrent collector start a collection cycle that way. But there's no way to find out if a collection cycle is running.
You are skating on the edge of the qualities of service offered by the different collectors in the various JVM's available. That weakens your ability to "write once, run anywhere".

Full garbage collection issue, not releasing/flagging memory

I have the following problem running on a multi-cpu windows server with Java 1.4.2_05 using WebLogic 8.1:
During a lifecyle of the web application (under load, but not to heavy) memory usage seems ok and garbage collection is called regularly. Suddenly, the used heap starts to rize very fast and after a while, even a full garbage collection cylce, does not release any memory anymore.
I am sure that, from our coding, we release memory ok, and normally we should only use about 5 to 10 mb for each user max (with0 normal defnew garbage collections).
I tried changing the garbage collection parameters, but this does not solve the problem. Best scenario was with the concurrent collector and I got this output at +/- the end:
[GC 100202K->93511K(115628K), 0.0091472 secs]
[GC 148480K->139612K(163808K), 0.0225914 secs]
[Full GC[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor289]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor290]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor273]
153750K->133006K(164064K), 1.2434402 secs]
[GC 148939K->137948K(203264K), 0.0223085 secs]
[GC 188789K->177116K(203264K), 0.0180729 secs]
[Full GC[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor312]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor322]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor309]
189788K->170264K(203264K), 1.1851945 secs]
[Full GC 203228K->203227K(203264K), 1.2876122 secs]
[Full GC 203263K->203233K(203264K), 1.3354548 secs]
[Full GC 203263K->203258K(203264K), 1.2873518 secs]
<Jan 17, 2007 9:40:40 AM EST> <Error> <HTTP> <BEA-101017> <[ServletContext(id=33114655,name=console,context-path=/console)] Root cause of ServletException.
java.lang.OutOfMemoryError
>
[Full GC 203263K->203233K(203264K), 1.2814516 secs]
[Full GC 203233K->203231K(203264K), 1.6029044 secs]
[Full GC 203263K->203242K(203264K), 1.3081352 secs]
<Jan 17, 2007 9:41:51 AM EST> <Emergency> <WebLogicServer> <BEA-000210> <The WebLogic Server is no longer listening for connections.>
[Full GC 203263K->203247K(203264K), 1.3161194 secs]
[Full GC 203263K->203249K(203264K), 1.2954988 secs]
[Full GC 203263K->203247K(203264K), 1.6423404 secs]
<Jan 17, 2007 9:41:57 AM EST> <Alert> <WebLogicServer> <BEA-000218> <Server shutdown has been requested by <WLS Kernel>>
[Full GC 203263K->203250K(203264K), 1.3161025 secs]
Another strange item is: I maximized the amount of memory it uses to 512m with the Xmx parameter, I am almost sure that that one is used, but it never gets higher than 203M? Does anyone know why this is?
Another strange item: the monitoring in the weblogic code indicates 32MB of usage (relative memory usage seems to be ok, but the quanity indication is just plain wrong) with 15 threads running.
This problem does not exist when using JBoss 4.0.2 or 4.0.3 (standard j2ee settings).
If anyone has an idea or can help me, I would appreciate it very very much. :)

Hi ,
Is this issue resolved ?
we are facing same problem.
1. We have checked the CPU and memory utilization everything is normal
2. GC logs showing FULL GC calls continuously
3. After restart the resin server system is working normally.
Environment detail
Resin ./resin-pro-3.0.18 on suse Linux
Java JDK1.4.2_08
Please suggest

How to specify when Full Garbage Collections occur in the Old Generation

Hi. We seem to be having a problem with a number of JVMs (1.5.0_17-b04) that run a component of a Document Management application. This component stores a large amount of information in caches which reside in the Old Generation. Although these cache sizes can be somewhat controlled by the application, they are currently taking about 85% of the Old Generation space. Fortunately, very few objects get tenured into the Old Generation - they all are cleaned up in the New Generation space.
The problem we are seeing is that with the Old Generation at 85% full, there are constant full GC's occurring. Since the caches cannot be removed, the system frantically tries to remove objects that can't be removed.
We have three solutions in mind. The first is to increase the memory allocation to the Old Generation so that the caches take a smaller percentage of the available memory allocation. The second would be to decrease the size of the caches; but this is set more by the number of documents in the application and cannot be made much smaller.
The third solution is to configure the JVM so that Garbage Collections in the Old Generation do not occur until the memory is more than a specific percentage of memory in the Old Generation. We would then set this percentage to be higher than the amount of memory being used by the caches.
So, is it possible to tell the JVM to only run a Full GC when the memory in the Old Generation is greater than a specific value (say 85% full)?
Thanks for your help.
Andre Fischer.

afischer wrote:
The third solution is to configure the JVM so that Garbage Collections in the Old Generation do not occur until the memory is more than a specific percentage of memory in the Old Generation. We would then set this percentage to be higher than the amount of memory being used by the caches.
So, is it possible to tell the JVM to only run a Full GC when the memory in the Old Generation is greater than a specific value (say 85% full)?Switch to the CMS collector.
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=86

Full Garbage Collection

Hi Friends,
I'm using weblogic workflow for my project. Last night i got one error, once i gone throw the bea ...i come to know that the error will comes because of "once the application calls webservice and the webservice intern calls the EJB stateless or stateful will fail". But my application is not using any sort of webservices. So i try to find the problem finally i found tht the problem is because of Garbage collection of Heap Size. It is taking 3.8508577 sec's. I feel in this time the JVM thread gets heighest priority and it is killing the application thread which is to be executed as usal.
Can u guide me how to catch this exception so tht my application won;t get affected. The actual error says like this which is related to Garbage Collection...[Full GC 313152K -> 105060K (1004928K), 3.8508577 secs]. I'm using JDK 1.4.
Thaks & Regards
[email protected]

Replies in this thread.

Help needed!! Novice with Garbage Collection problems.

Hi Guys,
Really hoping somebody can help me here. I am a relative novice when it comes to all things Java but i am slowly trying to learn. I have come across an issue which i have identified but i am just not sure what to do about it.
Ok, in a nut shell the issue seems to be revolving around the frequency of garbage collection. From the default-err.log file i am seeing (on average) an Allocation Failure occur every 2 secs. Here is a sample from the log with verbose:gc active:
<AF[4986]: Allocation Failure. need 208480 bytes, 78 ms since last AF>
<AF[4986]: managing allocation failure, action=2 (559165976/1342176248)>
<GC: Mon Oct 11 11:51:12 2004
<GC(4986): freed 4101528 bytes in 1559 ms, 41% free (563267504/1342176248)>
<GC(4986): mark: 1301 ms, sweep: 258 ms, compact: 0 ms>
<GC(4986): refs: soft 0 (age >= 32), weak 0, final 0, phantom 0>
<AF[4986]: completed in 1563 ms>
<AF[4987]: Allocation Failure. need 208536 bytes, 78 ms since last AF>
<AF[4987]: managing allocation failure, action=2 (559138336/1342176248)>
<GC: Mon Oct 11 11:51:14 2004
<GC(4987): freed 4105128 bytes in 1563 ms, 41% free (563243464/1342176248)>
<GC(4987): mark: 1293 ms, sweep: 270 ms, compact: 0 ms>
<GC(4987): refs: soft 0 (age >= 32), weak 0, final 0, phantom 0>
<AF[4987]: completed in 1563 ms>
As you can see, allocation failures are occuring all the time, and with 2secs between events, and each GC taking around 1.5secs, i am having massive problems with the response of the server. It seems that the javaw.exe process is just pegged at 100% CPU the whole time and then it will eventually grind to a halt, and the users will get terrible response times.
OK, the questions are:
- Even to me (a novice) the above extract from the log doesnt look good. Am i right?
- what would be causing this? (i know - how long is a peice of string but i am hoping somebody can point me in the right direction so i can look some more)
- what can i do about it? Is there any parameters i can put into the java args to help me out?
Currently i am running -Xms of 128meg and an -Xmx of 1024m with no other settings. There are at the moment about 200users logged onto this server concurrently, and it seems to die a couple of hours into them all being logged on. I then have to kick everybody out and reboot to get it in a working state again.
I am in some serious need of help from some gurus!! any help would be invaluable, thanks heaps guys.
Tim

Hi Again guys, thanks for all your replies.
I have been working my butt of on this issue and i just cant seem to get anywhere... probably due to my complete lack of knowledge on this whole GC thing! :)
One thing that i have noticed that seems to be very consistant is the fact that as soon as i get an "action=2" in my default-err.log from an allocation failure, thats when the system goes nuts and tends to not recover.
At all other times it is an action=1, and the system seems to be running OK, but as soon as i get action=2, the time between GC events drop from seconds to miliseconds, and the bytes required just skyrockets. It keeps on this upward spiral till i just have to reboot the box.
From all my reading, i have found that an action=2 means "2 - The Garbage Collector has tried to allocate out of the wilderness, and failed."
This is the only item in the logs that i see is directly related to the server performance.
Can anybody please explain to me (in laymans terms) what the action=2 means, what may cause it and what i should be looking at changing to fix it?
The extract from the log files (in my first post in this thread) are still valid, as are the min/max memory settings.
Any help at all would be invaluable.
Thanks very much.
Tim

DataGrid Garbage Collection problem

Hi,
I'm quite confused about the DataGrid component's garbage collection. I've made a simple test code which adds a datagrid to the display list. If I remove the component from its parent without clicking any item, the element is collected correctly. However, if I click an item, the datagrid is no longer removed. Otherwise, if I use the keyborad to navigate through the items, the component is removed.
The profiler doesn't display any inbound reference to the datagrid. So, what's the problem? Is Flex retaining any reference when I use the mouse?
Thanks in advance.

Hi,
here it goes:
the main application:
<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">
     <mx:Script>
          <![CDATA[
               import com.adur.nodes.Child;
               private var _child:Child;
               private function onLinkClickHandler(event:MouseEvent) : void {
                    _linkButton.enabled      = false;
                    _unlinkButton.enabled      = true;
                    _child = new Child();
                    addChild(_child);
               private function onUnlinkClickHandler(event:MouseEvent) : void {
                    _linkButton.enabled      = true;
                    _unlinkButton.enabled      = false;
                    removeChild(_child);
                    _child = null;
                    System.gc();
          ]]>
     </mx:Script>
     <mx:VBox width="100%" height="100%">
          <mx:HBox width="100%" height="30">
               <mx:Button id="_linkButton" width="100" label="Link" click="onLinkClickHandler(event)" />
               <mx:Button id="_unlinkButton" width="100" label="Unlink" enabled="false" click="onUnlinkClickHandler(event)" />
          </mx:HBox>
          <mx:Box     id="_content" width="100%" height="100%" />
     </mx:VBox>
</mx:Application>
the Child class:
<?xml version="1.0" encoding="utf-8"?>
<mx:Canvas xmlns:mx="http://www.adobe.com/2006/mxml" width="100%" height="100%" >
     <mx:ArrayCollection id="collection">
          <mx:source>
          <mx:Array>
               <mx:Object label="Option1" score="85" />
          </mx:Array>
          </mx:source>
     </mx:ArrayCollection>
     <mx:List dataProvider="{collection}" />
</mx:Canvas>
thanks harUI.

Garbage Collection Problem

Hi All,
I am facing some problem with frequent Garbage Collection application running in the background. Issue in production system. Frequently GC is happening, which is affecting the performance. Is there any work around. Issue is affecting the employees and data is inconsistent.
Please help on this asap.
Regards,
Soorya

Hi,
>>>I am facing some problem with frequent Garbage Collection application running in the background.
usually issues with GC are not that simple
also you need to check what kind of GC is happening first then you can start checking why
but my guess would be there is something wrong with the flow and this is causing the GC to work in a wrong way
(if it was working before)
when I did GC checks for my clients it was at least one day of work sometimes to find out what was wrong + repairs later on
Regards,
Michal Krawczyk

Reducing full Garbage Collection frequency.

I've been trying to improve the performance while inserting a large number of records to an embedded H2 database. Monitoring memory useage suggests that it's being used rather innefficiently. A lot of the objects created by H2 seem to find their way into "tenured" space before being freed. Full mark-and-sweap garbage collections are occuring every couple of seconds, despite the fact that only about 10% of the available heap is occupied.
Any advise on tuning the garbage collector to improve throughput in this case?

I think if surviving objects max out the to-space (part of the young generation) the remaining objects are copied into the tenured generation.
You can get more info via:
-XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGC -Xloggc:<filename>

Garbage Collection problem with BI 4.0

Hi experts,
I've installed BI 4.0, the operating system is SUSE Linux. In the logging directory are several *._gc.log-Files:
E.g. crproc_bobSIAN.CrystalReportsProcessingServer_child_gc.log:
34.609: [Full GC [PSYoungGen: 8352K->2707K(86976K)] [ParOldGen: 33449K->33470K(49472K)] 41801K->36177K(136448K) [PSPermGen: 35
487K->35460K(56640K)], 0.2510370 secs] [Times: user=0.25 sys=0.00, real=0.24 secs]
39.664: [GC [PSYoungGen: 81283K->4033K(100544K)] 114753K->40232K(150016K), 0.0387570 secs] [Times: user=0.03 sys=0.02, real=0.
04 secs]
41.271: [GC [PSYoungGen: 91137K->14332K(101440K)] 127336K->51242K(150912K), 0.0316510 secs] [Times: user=0.02 sys=0.01, real=0
.03 secs]
47.628: [GC [PSYoungGen: 101436K->15431K(112320K)] 138346K->66686K(163712K), 0.0623110 secs] [Times: user=0.06 sys=0.01, real=
0.06 secs]
47.690: [Full GC [PSYoungGen: 15431K->12343K(112320K)] [ParOldGen: 51255K->51388K(73088K)] 66686K->63731K(185408K) [PSPermGen:
63948K->63918K(96064K)], 0.5285130 secs] [Times: user=0.49 sys=0.04, real=0.53 secs]
659.998: [GC [PSYoungGen: 107383K->14966K(114752K)] 158771K->66354K(187840K), 0.1000770 secs]
or bobSIAN_gc.log:
3542.210: [GC 3542.233: [DefNew: 20756K->925K(22208K), 0.0036560 secs] 54323K->34493K(71208K), 0.0037360 secs] [Times: user=0.00 sys=0.00, real=0.03 secs]
3781.915: [GC 3781.942: [DefNew: 20701K->962K(22208K), 0.0031300 secs] 54269K->34530K(71208K), 0.0031840 secs] [Times: user=0.00 sys=0.00, real=0.03 secs]
4081.483: [GC 4081.499: [DefNew: 20738K->981K(22208K), 0.0030550 secs] 54306K->34549K(71208K), 0.0031110 secs]
It's also interesting, that I'm missing free space in the installation directory of Business Objects. There are no reports scheduled, but the directory still grows. Maybe it depends on problems with GC.
Any idea? Is there a best practice, how to configure the jvm?
Thanks and best regards,
Max

Hello Deepu,
The current plan for BI OnDemand is to support BOE 4.0 in the 2012 timeframe. We have not yet worked out a specific date for the release, though. The specific upgrade plan and long term feature roadmap is still under discussion for BI OnDemand.
Regards,
Terry Penner
BI OnDemand Product Owner

Simulate full garbage collection pauses

Hi,
I'm doing some performance tuning for our Java systems, and one thing I want to test is how the applications behave when "stop-the-world" Full GC pauses of various different durations occur in our JVMs. We have seen some problems in our systems where longer GC pauses have negative effects. However, it is difficult to recreate these conditions using the applications themselves. Does anyone know if it is possible to force a JVM to do a "stop-the-world" pause of a specified duration, that would simulate a GC?
Notes -
1) I'm not talking about forcing a GC - I know how to do this but this give no control on the duration of the GC. Appreciate might be able to work a solution by writing a program to use up X amount of memory followed by a forced GC, but hoping for something simpler!
2) We have already exhaustively tuned our JVMs to reduces frequency and duration of Full GC as much as possible but they do still need to occur occassionally. Have engaged with forums alredy on how best to do this, so no need for more info on this!
3) We are using multiple JVMs which communicate with each other (RMI, etc.), and a large part of what I'm testing is the effect on JVM A of a long GC pause in JVM B while A is communicating with B
4) JVM version is 1.5.0_u18, normally running on Solaris 9/10
Thanks in advance for any insights!
Regards,
Adrian

Hi,
The docs here:
[http://docs.sun.com/app/docs/doc/806-1367/6jalj6mv1?a=view|http://docs.sun.com/app/docs/doc/806-1367/6jalj6mv1?a=view]
seem to suggest something like what I want to do could have been achieved in Java 1.2, by sending SIGQUIT to the process. However, in Java 5, SIGQUIT just creates a thread dump. Anyone know if its possible to achive the behaviour from that link in Java 5 by any other means / signal?
Regards,
Adrian
Edited by: AdrianFitz on 05-Mar-2010 13:40

EWA Alert A- high ratio of full garbage

Hi All,
I have been receiving "RED" EWA report for pure JAVA stack from past 3 weeks. Earlier also I had got it and then the heap memory size was 1024. I had increased to 3GB and then I started getting a "YELLO" EWA report alert. However, from past 3 weeks I have been getting a red EWA report for full Garbage Collection problem.
I had gone throug some of the threads here and found some usefule info like, Paul mentioned in below link:
[EWA Alert A- high ratio of garbage collection|EarlyWatch Alert A - high ratio of full garbage;
How have gone to th dev_server0 file and saw some of the garbage collections there. I am quite unable to get the details mentioned there like:
1.
Mon Dec 12 07:43:12 2011
796020.432: [GC 796020.432: [ParNew: 319604K->28169K(320128K),
0.0588451 secs] 2671315K->2382219K(3116672K), 0.0589239 secs]
2.
Thu Dec 08 10:40:24 2011
461123.817: [Full GC 461123.817: [Tenured
Thu Dec 08 10:40:25 2011
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor150]
[Unloading class sun.reflect.GeneratedMethodAccessor543]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor220]......
I think the first one is the small garbage collections and the latter one is the "Full GC" as the name itself says.---Correct me if I am wrong.
Can any one help me in understanding the above details?
What should I do to remove this from my EWA alert?
Regards,
Faisal

Hi,
I have gone through the server nodes , std_server<n>.out (I have 3 server nodes) and found that on today's date, there are Full GC written in the logs multiple times. But can any one help me in diagnosing this and reaching out to the cause of this problem?
Regards
Faisal

Problem with video garbage collection

I have problem with a cummulative memory issue. I have developed an air application that reproduces video when it is needed. Although my air process (myAir.exe) uses the same ammount of memory for days (stable, illustrating lack of development memory leak problems) the total system memory increases every time a video is being reproduced. Every time the video is stopped the memory used by this video is released but not completely resulting to a continuously increasing system memory. Is this a graphics card problem, an OS graphics garbage collection problem or an air development problem! Has anyone ever had a similar problem? Every response would be helpful.
Thank you in advance.

Just a question ..??.. as a result of this dbconn getting freed before method ends , are you getting resultsetclosed error ?..

Hotspot core dumping during JVM garbage collection ?

We have an application which calls a 3rd party supplied server API which has recently been upgraded to use Java 1.5
We are getting the following error reported by our client application. The application is also now running Java 1.5 but references many classes in jar files which would have quite old code in.
The supplier of the API has stated that the problem requires us to recompile all our jar files using v 1.5 ( including things like jconnect and jms ?!?!? ). This sounds like a bit of a cop-out to me, not to mention being impossible since we don't have the source for things like jconnect.
I suspect that there is a garbage collection problem at the bottom of all this, but I'm not sure how I can "prove" this, nor do I currently have any real clue as to how to fix any GC problem that may exist.
The application is supposed to wait for a message on a MQSeries queue and then transforms it via Xalan XSLT and sends it to a Server application. I've tried playing around with heap sizes etc but that just seems to make it worse. If I leave it at the settings that the previous version used then the client at least manages to process a couple of messages before core dumping. There doesn't seem to be a consistent trigger event to cause the core dump ( it's not a message arriving on a queue for example ) but it does seem to be fairly consistent timewise, i.e. after
Any ideas gratefully accepted.
Here's a logfile excerpt from my applications showing the Hotspot error message :
=====================================================================================
08-Jul-2008 10:01:05 Waiting for messages from COLT.BBFS
08-Jul-2008 10:02:05 Waiting for messages from COLT.BBFS
08-Jul-2008 10:03:05 Waiting for messages from COLT.BBFS
405.815: [GC [PSYoungGen: 17331K->9244K(37632K)] 111702K->103615K(192128K), 0.1615910 secs]
405.977: [Full GC#
# An unexpected error has been detected by HotSpot Virtual Machine:
# SIGBUS (0xa) at pc=0xfe141348, pid=2600, tid=8
# Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
# Problematic frame:
# V [libjvm.so+0x141348]
# An error report file with more information is saved as hs_err_pid2600.log
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
=====================================================================================
The logfile referred to in the error message contains the following.
=====================================================================================
# An unexpected error has been detected by HotSpot Virtual Machine:
# SIGBUS (0xa) at pc=0xfe141348, pid=2600, tid=8
# Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
# Problematic frame:
# V [libjvm.so+0x141348]
--------------- T H R E A D ---------------
Current thread (0x001484d8): VMThread [id=8]
siginfo:si_signo=10, si_errno=0, si_code=1, si_addr=0x0000080f
Registers:
O0=0x00487588 O1=0xfe7d6454 O2=0x000079b0 O3=0x00007800
O4=0x00008868 O5=0x00147d48 O6=0xf8781460 O7=0xfe0f7938
G1=0xe52aaae8 G2=0x00000003 G3=0x00000003 G4=0x001484d8
G5=0xf8781d98 G6=0x00000002 G7=0xf8781d98 Y=0x805683e2
PC=0xfe141348 nPC=0xfe14134c
Top of Stack: (sp=0xf8781460)
0xf8781460: fe786000 00c6ba20 fe10013c e54bab68
0xf8781470: 0000080f e54bab6c 0000080f 00000004
0xf8781480: 00487588 00000134 e54bab6c 00000004
0xf8781490: 00000001 00000000 f87814c0 fe17aef4
0xf87814a0: fe6485b4 fe7d899c 001484d8 0011da40
0xf87814b0: 00148988 00148c10 00148d7c f8781880
0xf87814c0: 007c3389 007c3c0e 00000f87 00008868
0xf87814d0: 00008800 00487588 fe141310 fe7d6454
Instructions: (pc=0xfe141348)
0xfe141338: ec 06 c0 1a 80 a5 a0 00 22 40 00 0a ba 07 60 01
0xfe141348: f2 05 a0 00 ae 0e 60 03 80 a5 e0 03 22 40 00 05
Stack: [0xf8702000,0xf8781d98), sp=0xf8781460, free space=509k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x141348]
V [libjvm.so+0x17aefc]
V [libjvm.so+0x2d557c]
V [libjvm.so+0x300ef8]
V [libjvm.so+0x301e84]
V [libjvm.so+0x2ff950]
V [libjvm.so+0x29df30]
V [libjvm.so+0x362b44]
V [libjvm.so+0x6436f0]
VM_Operation (0xe03012b0): parallel gc system gc, mode: safepoint, requested by thread 0x0031bca0
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00b2c028 JavaThread "Thread-4" [_thread_in_native, id=85]
0x007f5048 JavaThread "Thread-0" [_thread_blocked, id=84]
0x00c27cf0 JavaThread "Notification Delivery" [_thread_blocked, id=81]
0x0026fa08 JavaThread "RMI LeaseChecker" daemon [_thread_blocked, id=73]
0x00821048 JavaThread "RMI RenewClean-[162.11.2.32:44425]" daemon [_thread_blocked, id=70]
0x0031bca0 JavaThread "GC Daemon" daemon [_thread_blocked, id=67]
0x00cd5d28 JavaThread "RMI Reaper" [_thread_blocked, id=66]
0x003c9300 JavaThread "Timer-0" daemon [_thread_blocked, id=65]
0x00929fe0 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=64]
0x0089bf18 JavaThread "SeedGenerator Thread" daemon [_thread_blocked, id=42]
0x00c47248 JavaThread "Pool thread #7" daemon [_thread_blocked, id=38]
0x00c466a0 JavaThread "Pool thread #6" daemon [_thread_blocked, id=37]
0x00311850 JavaThread "Pool thread #5" daemon [_thread_blocked, id=36]
0x00287a40 JavaThread "Pool thread #4" daemon [_thread_blocked, id=35]
0x00286e98 JavaThread "Pool thread #3" daemon [_thread_blocked, id=34]
0x00c134b0 JavaThread "Pool thread #2" daemon [_thread_blocked, id=33]
0x00ad09e0 JavaThread "Pool thread #1" daemon [_thread_blocked, id=32]
0x00286cd8 JavaThread "PoolThreadManager" daemon [_thread_blocked, id=31]
0x00c129e0 JavaThread "Channel Reaper" daemon [_thread_blocked, id=30]
0x00c669e8 JavaThread "ORB Daemon Thread" daemon [_thread_blocked, id=29]
0x00b10170 JavaThread "Worker for ServerProtocol: (iiop) /0.0.0.0:20168" daemon [_thread_blocked, id=22]
0x008a17e0 JavaThread "Syn~ Client" daemon [_thread_blocked, id=21]
0x003dc378 JavaThread "PoolScavenger0" daemon [_thread_blocked, id=20]
0x0015a928 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=15]
0x00159880 JavaThread "CompilerThread1" daemon [_thread_blocked, id=14]
0x00158a18 JavaThread "CompilerThread0" daemon [_thread_blocked, id=13]
0x00157b98 JavaThread "AdapterThread" daemon [_thread_blocked, id=12]
0x00156dc8 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=11]
0x0014ccd8 JavaThread "Finalizer" daemon [_thread_blocked, id=10]
0x0014ad90 JavaThread "Reference Handler" daemon [_thread_blocked, id=9]
0x00038238 JavaThread "main" [_thread_in_native, id=1]
Other Threads:
=>0x001484d8 VMThread [id=8]
0x0015c3b0 WatcherThread [id=16]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x00037728/0x00037758] Threads_lock - owner thread: 0x001484d8
[0x00033650/0x00037ba8] Heap_lock - owner thread: 0x0031bca0
Heap
PSYoungGen total 37632K, used 9244K [0xf2eb0000, 0xf7ba0000, 0xf8400000)
eden space 28352K, 0% used [0xf2eb0000,0xf2eb0000,0xf4a60000)
from space 9280K, 99% used [0xf4a60000,0xf5367080,0xf5370000)
to space 25216K, 0% used [0xf6300000,0xf6300000,0xf7ba0000)
PSOldGen total 154496K, used 94371K [0xe8400000, 0xf1ae0000, 0xf2eb0000)
object space 154496K, 61% used [0xe8400000,0xee028e78,0xf1ae0000)
PSPermGen total 35584K, used 18260K [0xe4400000, 0xe66c0000, 0xe8400000)
object space 35584K, 51% used [0xe4400000,0xe55d5158,0xe66c0000)
Dynamic libraries:
0x00010000      /dsdvlp/java/jvm/jdk1.5.0_03/bin/java
0xff350000      /usr/lib/libthread.so.1
0xff340000      /usr/lib/libdl.so.1
0xff200000      /usr/lib/libc.so.1
0xff390000      /usr/platform/SUNW,Sun-Fire-880/lib/libc_psr.so.1
0xfe000000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/server/libjvm.so
0xff1e0000      /usr/lib/libsocket.so.1
0xff2d0000      /usr/lib/libsched.so.1
0xff1b0000      /usr/lib/libCrun.so.1
0xff160000      /usr/lib/libm.so.1
0xff080000      /usr/lib/libnsl.so.1
0xff060000      /usr/lib/libmp.so.2
0xff030000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/native_threads/libhpi.so
0xfdfc0000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libverify.so
0xfdf80000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libjava.so
0xfdf50000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libzip.so
0xfb7e0000      /usr/lib/locale/en_GB.ISO8859-1/en_GB.ISO8859-1.so.2
0xe4190000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libnet.so
0xe3bd0000      /dsdvlp/lib/5/libSolarisNatives.so
0xe3e90000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/librmi.so
VM Arguments:
jvm_args: -Djava.ext.dirs=/dsdvlp/java/tmijar/firs7/lib/cli:/dsdvlp/java/tmijar/firs7/lib/cli/ext:/dsdvlp/java/tmijar/firs7/lib/cmn/OpenORB:/dsdvlp/java/tmijar/firs7/lib/cmn/OpenORB/ext:/dsdvlp/java/tmijar/firs7/lib/cmn:/dsdvlp/java/tmijar/firs7/lib/cmn/ext:/dsdvlp/java/tmijar/firs7/daemonlib -Duser.dir=/dsdvlp/java/tmijar/firs7 -Dopenorb.config=file:/dsdvlp/java/tmijar/firs7/configs/OpenORB/config/SynOpenORB.xml -Dopenorb.home=file:/dsdvlp/java/tmijar/firs7/configs/OpenORB -Dcom.coexis.syn.general.orbbinding=com.coexis.syn.general.orbbinding.openorb.OpenORBBinding_1_4 -Dsun.rmi.dgc.client.gcInterval=360000 -Dsun.rmi.dgc.server.gcInterval=360000000 -Xms32m -Xmx256m -Dcom.coexis.syn.clientcommandsconfiglocation=file://localhost//dsdvlp/java/tmijar/firs7/configs/clientcommands.xml -Dcom.coexis.syn.clientconfiglocation=file://localhost//dsdvlp/java/tmijar/firs7/configs/fsbbtd_client.xml -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
java_command: com.coexis.syn.mqmessaging.daemon.RunDaemon -p /dsdvlp/bin/5/lndsfsd_fsbbtd.properties start
Environment Variables:
JAVA_HOME=/dsdvlp/java/jvm/jdk150
CLASSPATH=.:/dsdvlp/java/jar/jconnect520.jar:/dsdvlp/java/jar/vbjapp340.jar:/dsdvlp/java/jar/vbjorb340.jar:/dsdvlp/java/jar/javax_jndi120.jar
PATH=/usr/local/etc:/usr/lang:/usr/openwin/bin:/usr/ucb:/bin:/usr/etc:/usr/local/5/bin:/dsdvlp/bin/5:/dsdvlp/bin/4:/home/app/sybase/5/bin:/home/app/sybase/5/localscripts:/home/app/sybase/5/sqr:/home/app/lang:/home/app/lang/SC2.0.1:/usr/ccs/bin:/usr/local/opt/Acrobat3/bin:/dsdvlp/bin:.
LD_LIBRARY_PATH=/dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/server:/dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc:/dsdvlp/java/jvm/jdk1.5.0_03/jre/../lib/sparc:/usr/lib:/usr/openwin/lib:/usr/local/5/lib:/dsdvlp/lib/5:/dstest/lib/5:/home/app/sybase/5/lib:/dstest/cats/sun4/lib:/tmitest/Opus/opus/lib
SHELL=/bin/csh
DISPLAY=CLI00184.mfil.local:1.0
OS=5
--------------- S Y S T E M ---------------
OS: Solaris 8 2/02 s28s_u7wos_08a SPARC
Copyright 2002 Sun Microsystems, Inc. All Rights Reserved.
Assembled 18 December 2001
uname:SunOS 5.8 Generic_117350-20 sun4u (T1 libthread)
rlimit: STACK 8192k, CORE 9216k, NOFILE 4096, AS infinity
load average:2.24 2.67 2.68
CPU:total 4 has_v8, has_v9, has_vis1, has_vis2, is_ultra3
Memory: 8k page, physical 8388608k(166384k free)
vm_info: Java HotSpot(TM) Server VM (1.5.0_03-b07) for solaris-sparc, built on Apr 13 2005 03:31:26 by unknown with unknown Workshop:0x550

The very first suggestion I have is to move your VM to a more recent update of 1.5.0.
It looks like you are crashing with 5.0u3, and I'm pretty sure 5.0u16 is available. You
don't want to waste your time chasing a bug that's already been fixed.

Garbage Collection Mysteries

I have an application that consumes huge amounts of heap at time, on the order of 100 MB. I added some code to call System.gc() when I dispose of the window that is responsible for making such huge allocations. My understanding is that System.gc() is supposed to do a full garbage collection, but when I am watching the heap in the Eclipse profiler it does not seem to reclaim very much. In almost ever case, when I click the garbage collect button in the profiler it seems to do a much better job of collecting a lot.
What is the difference between explicitly invoking the garbage collector from the Eclipse profiler and calling System.gc() in code when the window is disposed?
How can I get my code to automatically make the garbage collector work so well?
Cheers, Eric

jschell wrote:
Might note however that calling gc() is unlikely to do anything to make your application better.Ain't it the truth. In fact it may even make it worse. Sane applications should not need manual GC calls - the runtime does what it is supposed to do itself and if it doesn't you should investigate and fix the problem that is preventing it from doing what it is supposed to do. First making sure that there is in fact an issue of course, perhaps the app simply needs 100mb of heap space at some point in the lifetime of your application. It isn't exactly a HUGE amount of memory, especially in a Java or .NET VM environment.

Full Garbage Collection Problem

Similar Messages

Maybe you are looking for