Jvm startup fails with error when using large -Xmx value

Question

I'm running JDK 1.6.0_02-b05 on RHEL5 server. I'm getting error when starting the JVM with large -Xmx value. The host has ample memory to succeed yet it fails. I see this error when I'm starting tomcat with a bunch of options but found that it can be easily reproduced by starting the JVM with -Xmx2048M and -version. So it's this boiled down test case that I've been examining more closely.
host% free -mt
total used free shared buffers cached
Mem: 6084 3084 3000 0 184 1531
-/+ buffers/cache: 1368 4716
Swap: 6143 0 6143
Total: 12228 3084 9144
Free reveals the host has 6 GB of RAM, approximately half is available. Swap is totally free meaning I should have access to about 9 GB of memory at this point.
host% java -version
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)
java -version succeeds
host% java -Xmx2048M -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
java -Xmx2048M -version fails. Trace of this reveals mmap call fails.
mmap2(NULL, 2214592512, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot allocate memory)
Any ideas?

Answer 1

These are the relevant java options we are using:
-server -XX:-OmitStackTraceInFastThrow -XX:+PrintClassHistogram -XX:+UseLargePages -Xms6g -Xmx6g -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=128m -XX:MaxPermSize=192m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:+ExplicitGCInvokesConcurrent -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.awt.headless=true
This is a web application that is very dynamic and uses lots of database calls to build pages. We use a large clustered cache to reduce trips to the database. So being able to acces lots of memory is important to our application.
I'll explain some of the more uncommon options:
We use the Concurrent Garbage collector to reduce stop the world GC's. Here are the CMS options:
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled An explicit coded GC invokes the Concurrent GC instead of the stop the world GC.
-XX:+ExplicitGCInvokesConcurrentThe default PermSizes where not large enough for our application. So we increased them.
-XX:PermSize=128m
-XX:MaxPermSize=192mWe had some exceptions that were omitting their stack traces. This options fixes that problem:
-XX:-OmitStackTraceInFastThrowWe approximate between 10% to 20% performance improvement with Large Page support. This is an advance feature.
-XX:+UseLargePagesUseLargePages requires OS level configuration as well. In SUSE10 we configured the OS's hugepages by executing
echo "vm.nr_hugepages = 3172" >> /etc/sysctl.confand then rebooting. kernel.shmmax may also need to be modified. If you use Large Page be sure to google for complete instructions.
When we transitioned to 64bit we transitioned from much slower systems having 4GB of ram to much faster machines with 8GB of ram, so I can't answer the question of degraded performance, however with our application, the bigger our cache the better our performance, so if 64bit is slower we more than make up for it being able to access more memory. I bet the performance difference depends on the applications. You should do your own profiling.
You can run both the 32bit version and the 64bit version on most 64bit OSes. So if there is a significant difference run the version you need for the application. For example if you need the memory use the 64bit version if you don't then use the 32bit version.