OutOfMemoryError in 1.4.1, not in 1.3.1

Question

I'm having trouble diagnosing an OutOfMemoryError in my application and I would like to double-check my assumptions here, if I may. I apologize in advance for the length of this post... I figured the more information the better.
The situation is this:
When running the application, it will eventually generate an OutOfMemoryError.
The OutOfMemoryError always happens during garbage collection and there is plenty of collectable garbage. What I mean to suggest is that I did not actually run out of memory.
If I specify the minimum heap size to be larger than default (equal to the max heap size at the extreme), I effectively increase the space allowed for allocation before a garbage collection which makes for a longer garbage collection which causes the error to occur much more quickly - usually within the first 3 garbage collections.
I am currently testing with the Windows JDK1.4.1 on Windows 2000/SP3.
My expectation (I have not verified this yet, perhaps someone here can confirm this) is that the JVM is supposed to guarantee that any time an allocation is attempted that, if there isn't enough memory available on the heap, the GC will run and attempt to give me my memory before it throws an error. I do not believe that this is what is happening.
The application uses several threads and my belief is that this must be some sort of race condition, incorrect or incomplete synchronization. When the OutOfMemoryError occurs during the collection, I will get anywhere from 1 to 11 (so far) threads reporting the error.
The reason I really don't think I'm running out of memory is based on observing the output of a profiler (OptimizeIt). I start with "-Xms192m -Xmx192m". Heap usage starts out at a nominal value of 15m. Application runs under load, eventually usage gets to within 1K of the limit and a "Full GC" begins. The first couple of times, memory usage drops back to the starting value of 15m. Then it fails. Either I "sprung" a really big leak or there is some other problem.
I have observed that, when running under 1.3.1, the GC runs while there is a LOT more room before reaching the heap limit. For example, in this case it will run when heap usage reaches around 180m. I have also noticed that, when running "-verbose:gc -XX:+PrintGCDetails", it appears that there is never a non-Full GC.
If anyone has any suggestions on where to look for this problem, I'm all ears. I've used a profiler to determine where most of the allocations are occuring, but that just doesn't help. I'm not really running out of memory, this could be a single byte allocation for all I know that's happening when it shouldn't. I can't believe that it's a VM bug or I would have heard more about it.

Answer 1

Well, as I mentioned, I am starting at only 15m of heap usage. I then climb to 192m and drop back down to 15m after the collection. Even under load, my baseline memory usage is only about 32m. There is a lot of temporary creation (I've done a lot to reduce it, which has helped, but still quite a bit).
The duration of the GC appears to be part of the equation. If the GC lasts a long time, it is more likely to happen.
Maybe I should also point out this: If, rather than specify "-Xms192m -Xmx192m", I use "-Xms128m -Xmx192m", I may get an OutOfMemoryError before heap size even grows to 192m.