Out Of Memory with tons of free mem

Question

I need help diagnosing a problem I'm having with a server application.
The problem:
I'm getting general flakeyness and sometimes "OutOfMemory: cannot create new native thread" after the application has been running several hours (with 150 or so logins and 300 or so signs rendered).
The environment:
We run Tomcat 3.2.4, Xalan2, JAXP1.1, mysql w/ mm.sql JDBC driver v1.2.0.13, JDK 1.3.1_03
on a SunFire v880 w/ 4 processors, 8Gig RAM running Solaris 8. JVM args set to -server -Xms3g -Xmx3g -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:SurvivorRatio=16 -XX:+DisableExplicitGC
What I've tried:
I've run under OptimizeIt and found only small amounts of mem being consumed ongoing. Something the GC ought to be able to handle. I've played with the GC parameters in an effort to get this problem under control. I monitor the freemem and can see it go up and down through out the day. The lowest freemem has come down is to 900Meg, then bounces back up to 2.7Gig or so. I suspected fragmentation, but from what I read, hotspot shouldn't allow that to happen.
That exception is thrown from native code in "jvm.cpp" in the JVM_StartThread() method. Comments say it could happen when memory is low (duh).
Our App:
We render in store signs for a major US department store. Our app provides simplified sign composition. We render the signs for printing (generating postscript for the printers). Sometimes we deal with buffers as large as 112Meg. We use XML for sign templates and jobs. We spit out XML from servlets, using XSLT to transform to HTML for the browser. Our sign templates use some XSLT to add common things into the templates as they are loaded. We support image upload to a repository that the sign creators can use in the signs. So, we deal with lots if image data and XML throughout the app. We hit the database throughout the workflow.
HELP!

Answer 1

Hi,
I had the same problem under Linux, but found a solution:
There are 2 different factors for this problem.
There is the "Xss" option of Java and the "stack size"
as reported/set by ulimit:
ulimit -acore file size (blocks) 0
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 16896
virtual memory (kbytes) unlimited
When using java 1.3.1, the "Xss" option is ignored, so
only the "stack size" of ulimit counts. Setting the
stack size using ulimit is done using the -s option:
ulimit -s 8129This sets the stack size per Java thread to 8MB.
The default stack size in Redhat Enterprise Server 3.1
is 10MB. When a Java thread is started, this 10MB is not taken
immediately, but the Linux kernel does guarantee this thread 10MB
if it needs it. Suppose you hava 2GB memory in your machine, then
after starting 200 Java threads, the light goes out even though
"top" reports 1.5GB of free memory. The memory is free, but the
kernel has already promised all of it to the 200 executing Java
threads.
Under java 1.4.2, the Xss option is not ignored. It has the same
effect as the "stack size" set by ulimit; specifying a large amount
of memory for Xss will limit the amount of threads you can start.
Using java 1.4.2 it seems these settings both have to be tuned.
Greetz,
Chris Twigt