Libumem and java native leaks

Question

I'm diagnosing a native memory leak in a java process(not caused by a user JNI library, probably due to not closing some stream tied to native resources). I had used this technique before with some success, but this time I'm running into some problems. Basically what I'm doing is using libumem and mdb to help find the leak. I realize that these tools can give some strange results when looking at the JVM, but previously when I have it used, I can just focus on the leaked buffers with a large count and that pointed me right to the problem(the problem is bad enough that eventually I run out of memory space for the process...so I know the leak is being triggered repeatedly). The first problem I have is with libumem/mdb and the stack printed out through bufctl_audit which only displays symbol addresses for java routines. If it displayed the java symbol names, or if there were a way to make it display them then I wouldn't have much of a problem(though I've found I need to increase the audit size because of the large stack frame for a java process).
So what I've done before is find the C library/routines where the memory is being leaked at, and then use dtrace to print stack traces for calls into that C library. Something like this:
dtrace -n 'pid$target:libzip:ZIP_GetEntry:entry { @s[jstack(60,3000)] = count() }' -p <PID>
This has pointed me to the right place in the past, because the place where the leak was happening was getting called frequently. The problem is with the current leak is this is much too course grained. The C library being called is being called quite often, so I'm getting way too many stacks, and sorting through to find the problematic one is difficult. Now if jstack() would show the java symbol name AND its address I could easily correlate it with what's in the findleaks/bufctl_audit output of libumem/mdb.
Is there something I can do inside mdb to help find the java symbol names, or is there a way to use dtrace to correlate the java symbol name with its address to help me out here? I can do some rather ugly iterative stuff with dtrace where I don't give it a large enough buffer to print the whole stack trace and with small increases probably find the mappings from the java symbols to the address, but I was hoping there was something a little less painful. This of course needs to be diagnosed in a production system...so what I can do is somewhat limited. There is redundancy, so I obviously can for a short period pause execution on one server to grab the findleaks output and things like that. Thanks,
Micah

Answer 1

Since your question is about tracing Java internals, you might do better posting in a Java-related forum. (This forum is for Sun Studio, for developing in C, C++, and Fortran.) Try one of the forums listed under Java here:
[http://forums.sun.com/index.jspa?tab=forums]
and especially here:
[http://forums.sun.com/category.jspa?categoryID=39]