GC accounts for 80% of processing time

Question

Hi,
i'm having a rather serious problem with the garbage collector on one of my Java projects. Basically, what i'm doing is just read an XML file using StAX, and then process its content -- nothing scary here. However, I found the process was being really slow, so I profiled it using the YourKit Java Profiler within Eclipse, and it told me that for a short 12 second run, the GC was responsible for 10 seconds!
Without being an expert, I've done other small projects in Java, and this is the first time I see this. As it is also the first time I'm using StAX, I can't but wonder whether there might be a connection?
I've noticed exactly the same symptoms on several architectures (Mac/WinXP/Linux SuSE, Eclipse/CLI -- I did profiling only on the Mac). Calling java with the -Xmx1024m switch does improve things, as the process then needs only 10.3 instead of 12.5 seconds. However this is still unacceptable, as the programm is eventually meant to be running for several hours straight -- working, not collecting garbage.
I've created a GC log: I don't really know what to make of it, but it basically looks like this:
0.146: [GC 512K->194K(1984K), 0.0024579 secs]
0.180: [GC 704K->268K(1984K), 0.0012318 secs]
0.204: [GC 776K->255K(1984K), 0.0006406 secs]
0.228: [GC 767K->257K(1984K), 0.0003966 secs]
0.234: [GC 355K->270K(1984K), 0.0004048 secs]
0.235: [Full GC 270K->270K(1984K), 0.0063139 secs]
0.281: [GC 2773K->2671K(4036K), 0.0012717 secs]
0.282: [Full GC 2671K->2671K(4036K), 0.0070599 secs]
0.293: [GC 4719K->4719K(7084K), 0.0001165 secs]
These are the first 8 lines. The next 1800+ lines look pretty much like these last 4 ones, with 2 messages being shot every .1 second or so. Apparently the numbers keep rising, and the last messages are
10.197: [Full GC 5262K->3214K(7084K), 0.0103843 secs]
10.209: [GC 5267K->5262K(7084K), 0.0001103 secs]
10.210: [Full GC 5262K->3214K(7084K), 0.0109799 secs]
10.253: [GC 5767K->5302K(7084K), 0.0010396 secs]
So can anyone tell me what's happening here? Do I have a memory leak or something?? Is it possible to find out what the GC is cleaning every .1 second?
I'm really at a loss here, so thx a lot for your help! :-)
cheers
David

Answer 1

Actually, you could try following:
-     Open WF Monitor, connect to DB via sqlplus or other tool to prepare to run process.
-     Start mapping execution, verify your session is waiting “queue message”
-     Wait process to complete, verify it by WF Monitor
-     Check wait state of session – if it still in “queue message”, it waits completion report from runtime platform.
Our current configuration is: OWB 9.2.0.4 + DB 9.2.0.7.
We have faced similar issues with OWF (don’t know your environment, but in ours OWF work under heavy load with concurrent executions of different types of processes) – session could wait indefinitely long for process to “complete” while process had finished long time ago.
Unfortunately, this is not the only implication of the issue. Once you could notice that your processes simply hang in active state. Activities within process are notified, but not started.
We took two steps independently and now our situation with OWB/OWF is much better:
- First. In the runtime schema of OWB there are a lot of foreign keys without indexes on “child” side – we created indexes, rebuild them, “alter move” all tables etc. This came from the recent investigation in this forum about performance of OWB while purging the logs. I suppose the issues that raised in that discussion have their implications on environment with concurrent process executions also.
- Second. I applied 9.2.0.7 Patch Set (plus one one-off-patch) on DB – this make OWF behavior to improve dramatically. No “notified-but-not-started” activities, or “indefinite-waiting-for-process-completion” sessions noted since New Year. Previously we have to fight these issues every day and even worse.
Serhit.