How to take regular heap dumps using HPROF

Question

Hi Folks,
I am using Oracle App server as my application server. I found that the memory is growing gradualy and gets maxed out with in 1 hour. I am using 1 GB of heap.
I defently feel this is a memory leak issue. Once the Heap usage reaches 100%, I will start getting the FULL GCs and my whole server hangs and nothing will work. Some times even the JVM crashes and restarts again.
I didn't find Out of Memory exception also in any of my logs.
I came to know that we can use Hprof to deal with this.
I use the below as my JVM agrs...
-agentlib:hprof=heap=all,format=b,depth=10,file=$ORACLE_HOME\hprof\Data.hprof
I run my load run for 10 mins, now my heap usage has been grown to some extent.
My Questions:
1. Why there are 2 files generated, one is with the name Data.hprof and another with Data.hprof.tmp. Which is what?
2. How to get the dump at 2 different points. So that I can compare the the 2 dumps and I can say which object is growing more.
I downloaded the HAT and If I use to open this Data.hprof file from HAT, I am getting this error. This error will come if I open the file with out stoping the JVM process.
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at hat.parser.HprofReader.read(HprofReader.java:202)
at hat.parser.Reader.readFile(Reader.java:90)
at hat.Main.main(Main.java:149)
If I stop the JVM process, and then open through HAT I am getting this error,
Started HTTP server on port 7000
Reading from hprofData.hprof...
Dump file created Wed Dec 13 02:35:03 MST 2006
Warning: Weird stack frame line number: -688113664
java.io.IOException: Bad record length of -1551478782 at byte 0x0008ffab of file.
at hat.parser.HprofReader.read(HprofReader.java:193)
at hat.parser.Reader.readFile(Reader.java:90)
at hat.Main.main(Main.java:149)
JVm version I am using is: Sun JVM 1.5.0_06
I am seriously fed up of this memory leak issue... Please help me out folks... I need this as early as possible..
I hope I get early replys...
Thanks in advance...

Answer 1

First, the suggestion of using jmap is an excellent one, you should try it. On large applications, using the hprof agent you have to restart your VM, and hprof can disturb your JVM process, you may not be able to see the problem as quickly. With jmap, you can get a heap snapshot of a running JVM when it is in the state you want to understand more of, and it's really fast compared to using the hprof agent. The hprof dump file you get from jmap will not have the stack traces of where objects were allocated, which was a concern of mine a while back, but all indications are that these stack traces are not critical to finding memory leak problems. The allocation sites can usually be found with a good IDE ot search tool,
like the NetBeans 'Find Usages' feature.
On hprof, there is a temp file created during the heap dump creation, ignore the tmp file.
The HAT utility has been added to JDK6 (as jhat) and many problems have been fixed. But most importantly, this JDK6 jhat can read ANY hprof dump file, from JDK5 or even JDK1.4.2. So even though the JDK6 jhat is using JDK6 itself, the hprof dump file it is given could have come from pretty much anywhere, including jmap. As long as it's a valid hprof binary dump file.
So even if it's just to have jhat handy, you should get JDK6.
Also, the Netbeans profiler (http://www.netbeans.org) might be helpful too. But it will require a restart of the VM.
-kto