Long running GC - High cpu% utilization

Question

I have a tomcat process running on linux which generates a lot of garbage.
Throughout the day the machine will be at around 80% idle, and then suddenly jump to 20% idle cpu. It can stay at 20% idle for 1-4 hours, then goes back to 80% idle where it can stay for hours as well. The graph looks like a step-function.
This does not correlate to the usage graph; the usage graph is smooth, typical of a B->C webapp.
In conjunction with the 20% idle, i also see about 20% spent in "system" cpu, as well as an increase in "user" cpu consumption. When at 80% idle, system% is around 1%-2%.
I log GC stats, and when the cpu is at 80% idle, i see:
75324.001: [GC 75324.001: [ParNew: 65408K->0K(65472K), 0.0102260 secs] 472166K->407009K(1535936K), 0.0104630 secs]
75327.998: [GC 75327.998: [ParNew: 65408K->0K(65472K), 0.0118190 secs] 472417K->407212K(1535936K), 0.0120560 secs]
75333.342: [GC 75333.343: [ParNew: 65408K->0K(65472K), 0.0105900 secs] 472620K->407480K(1535936K), 0.0108260 secs]
About 10 msecs spent in GC every 4 secs, but when it gets to 20% idle i see:
30336.581: [GC 30336.581: [ParNew: 65408K->0K(65472K), 0.5277730 secs] 244204K->203369K(1535936K), 0.5280800 secs]
30338.403: [GC 30338.403: [ParNew: 65408K->0K(65472K), 0.4632960 secs] 268777K->224890K(1535936K), 0.4635790 secs]
30340.256: [GC 30340.256: [ParNew: 65408K->0K(65472K), 0.5050930 secs] 290298K->248727K(1535936K), 0.5053670 secs]
So now it's taking 500 msecs every 2 secs or so.
I'm using 1.5 with the following options:
-Xms1500m -Xmx1500m -XX:+UseConcMarkSweepGC -XX:NewSize=64m
So a 1.5G heap, with a 64M young generation. My inclination is to bump up the size of the young generation, but when i see these long 500 msecs young gen collection times, i get worried.
Can anybody explain the strange step-function behavior of the process over time that would be strictly related to GC?
Note that i see exactly the same pattern using the jrockit jvm (although the absolute numbers are different). I see the same step-function graph for cpu utilization vs a smooth graph for usage.
Thanks.

Answer 1

The fact that you see the same behavior with two completely different VM's makes me think it's not the VM's. -XX:+UseConcMarkSweepGC will cause slightly ParNew collections to take slightly longer, when it is running, but not anything like 50 times slower.
Trying using jconsole to see what going on during the idle periods and the busy periods. That is, get thread stack traces. If you are running JDK-1.6.0, you can just start up jconsole and attach to the running VM. If you are running JDK-1.5.0, I think you need to start the VM with an additional option to allow jconsole to attach to it.