Permanent Generation and GC

Question

Our server application runs fine using ConcMarkSweep GC (or so we thought) on Java 1.5 with
subsecond pauses.
However, we see the permanent generation growing slowly per request, e.g. by 2k.
The unfortunate effect is that when the permanent generation is full, a Full GC cycle takes place
and the 15 seconds spent on that are unacceptable to our client. Ouch.
Q1: Is there a way to avoid the Full GC on the permanent GC? Why isn't it included in the CMS GC
cycles?
From the little information I can find on the permanent generation, it contains
- class data
- interned Strings
Q2: What could be the cause of the substantial growth in the permanent generation.
(I checked the XML parsing, and the invocations of String.intern are not sufficient to explain the
situation). We do a lot of different things with third party jars, XML, XLST, jdom, JDBC (Sybase jconnect), some object (de)serialization and zipping, SSL, Tomcat.. No RMI/EJB.
Any wll-known offenders?
Q3: Are there any tools other than jmap -heap to monitor the object in the permanent generation
specifically? Ideally a full profiling JMVTI app or tracing the calls to the String.intern.
Q4: Is there anything else I'm missing? Have there been changes in this obscure corner of the JVM since 1.4.2?

Answer 1

Probably nothing new for you, up until Q3...
The unfortunate effect is that when the permanent
generation is full, a Full GC cycle takes place
and the 15 seconds spent on that are unacceptable to
our client. Ouch.Have you looked at this tuning doc?
http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html
Q1: Is there a way to avoid the Full GC on the
permanent GC? Why isn't it included in the CMS GC
cycles?For more information on heap sizing and garbage collection issues,
check the white papers here:
http://developers.sun.com/techtopics/mobility/midp/articles/garbagecollection2/
The benchmark application they describe is rather specialized, but the
paper contains some very useful information about heap layout and
garbage collection in general.
From the little information I can find on the
permanent generation, it contains
- class data
- interned Strings
Q2: What could be the cause of the substantial growth
in the permanent generation.
(I checked the XML parsing, and the invocations of
String.intern are not sufficient to explain the
situation). We do a lot of different things with
third party jars, XML, XLST, jdom, JDBC (Sybase
jconnect), some object (de)serialization and zipping,
SSL, Tomcat.. No RMI/EJB.
Any wll-known offenders?See Q3. Hopefully one of the queries listed in the README will help track down the anomaly.
Q3: Are there any tools other than jmap -heap to
monitor the object in the permanent generation
specifically? Ideally a full profiling JMVTI app or
tracing the calls to the String.intern.Have you looked at HAT (Heap Analysis Tool)?
https://hat.dev.java.net
https://hat.dev.java.net/doc/README.html
Hope this helps.
"Troubleshooting Guide for J2SE 5.0",
http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf