Experiencing a 32-bit Java heap barrier on a 64-bit JVM

Question

We are attempting to run a Java application on a Sun Fire V440 with Solaris 9 and 16GB RAM. We are using the 5.0 JRE.
We are starting the application with the -d64 and -server options, and 6g of heap space with the max/min settings the same.
We have queried the Runtime object to confirm that the heap is what we expect it to be, and have plotted our heap usage.
Our full data matrix requests about 3.4GB of memory at one point in the code. We are unable to analyse the full matrix and encounter an OutOfMemoryError on array allocation. What we notice is that when we scale down the matrix so that heap usage is just under the 32 bit barrier (4.3GB) it runs, but as soon as we cross it we error.
We have checked that we are using the 64-bit JVM via the sun.arch.data.model system property and the java.version property. The Runtime object also clearly shows that the free memory is available.
What we see is that before allocation we are using 1,3GB of heap.
Adding about 3.4GB, on array allocation, to this puts us over the 32-bit threshold and we error. Any input would be greatly appreciated.

Answer 1

Thank you for the replies. It has helped significantly! As was suggested, the problem seems to be a young and old generation heap problem. Unfortunately, it's too late to change the subject heading. The barrier was just an artifact of a number of coincidences.
We are closer to solving the issue, but we're not there yet. We tried an experiment with 8GB of heap split 50/50 for young and old using:
java -d64 -server -Xms8000m -Xmx8000m -Xmn4000m
resulting in settings of:
-XX:MaxHeapSize=8388608000
-XX:MaxNewSize=4194304000
-XX:NewSize=4194304000
-XX:+UseParallelGC
Then we manually GC'd before the array allocation just to make sure the young generation heap was cleared out. Prior to this most of the young generation cycling is due to a BufferedReader readline() command.
After System.gc(), we read in an matrix of 900K x 300 double values = 2.1GB of data. We still run out of heap space. Our manual GC is invocation #3 and clears out the young heap, and even promotes an insignificant amount of data to the old heap, but leaves both generations largely empty:
{Heap before gc invocations=1:
PSYoungGen      total 3584000K, used 3072000K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 100% used [0xfffffffe3d400000,0xfffffffef8c00000,0xfffffffef8c00000)
  from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
  to   space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen        total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a718,0xfffffffd3fc00000)
0.000: [GCAdaptiveSizePolicy::compute_survivor_space_size_and_thresh:  survived: 1032296  promoted: 0  overflow: falseAdaptiveSizeStart: 0.016 collection: 1
AdaptiveSizePolicy::compute_generation_free_space:  minor_time: 0.000157 major_cost: 0.000000 mutator_cost: 0.999843 throughput_goal: 0.990000 live_space: 277243776 free_space: 7340032000 old_promo_size: 4194304000 old_eden_size: 3145728000 desired_promo_size: 4194304000 desired_eden_size: 3145728000
AdaptiveSizeStop: collection: 1
[PSYoungGen: 3072000K->1008K(3584000K)] 3072000K->1008K(7680000K), 0.0162566 secs]
Heap after gc invocations=1:
PSYoungGen total 3584000K, used 1008K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8cfc068,0xffffffff18000000)
to space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
PSOldGen total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a718,0xfffffffd3fc00000)
{Heap before gc invocations=2:
PSYoungGen      total 3584000K, used 2952191K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 96% used [0xfffffffe3d400000,0xfffffffef1603f40,0xfffffffef8c00000)
  from space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8cfc068,0xffffffff18000000)
  to   space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
PSOldGen        total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7596K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6b3d8,0xfffffffd3fc00000)
88.203: [GCAdaptiveSizePolicy::compute_survivor_space_size_and_thresh:  survived: 999520  promoted: 0  overflow: falseAdaptiveSizeStart: 88.212 collection: 2
AdaptiveSizeStop: collection: 2
[PSYoungGen: 2952191K->976K(3584000K)] 2952191K->976K(7680000K), 0.0096021 secs]
Heap after gc invocations=2:
PSYoungGen total 3584000K, used 976K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xffffffff18000000,0xffffffff180f4060,0xffffffff37400000)
to space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen total 24576K, used 7596K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6b3d8,0xfffffffd3fc00000)
{Heap before gc invocations=3:
PSYoungGen      total 3584000K, used 976K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
  from space 512000K, 0% used [0xffffffff18000000,0xffffffff180f4060,0xffffffff37400000)
  to   space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen        total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7596K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6b3d8,0xfffffffd3fc00000)
88.213: [Full GCAdaptiveSizeStart: 88.392 collection: 3
AdaptiveSizeStop: collection: 3
[PSYoungGen: 976K->0K(3584000K)] [PSOldGen: 0K->874K(4096000K)] 976K->874K(7680000K) [PSPermGen: 7596K->7596K(24576K)], 0.1788788 secs]
Heap after gc invocations=3:
PSYoungGen total 3584000K, used 0K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
to space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen total 4096000K, used 874K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd434da8c0,0xfffffffe3d400000)
PSPermGen total 24576K, used 7596K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6b3d8,0xfffffffd3fc00000)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
So next we reduced our data to 890K x 300 double values = 2.14GB of data, not a lot less, and it allocates fine. In fact in GC #4 it has even promoted the signal matrix to the old generation:
{Heap before gc invocations=1:
PSYoungGen      total 3584000K, used 3072000K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 100% used [0xfffffffe3d400000,0xfffffffef8c00000,0xfffffffef8c00000)
  from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
  to   space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen        total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a768,0xfffffffd3fc00000)
0.000: [GCAdaptiveSizePolicy::compute_survivor_space_size_and_thresh:  survived: 1048656  promoted: 0  overflow: falseAdaptiveSizeStart: 0.017 collection: 1
AdaptiveSizePolicy::compute_generation_free_space:  minor_time: 0.000281 major_cost: 0.000000 mutator_cost: 0.999719 throughput_goal: 0.990000 live_space: 277260224 free_space: 7340032000 old_promo_size: 4194304000 old_eden_size: 3145728000 desired_promo_size: 4194304000 desired_eden_size: 3145728000
AdaptiveSizeStop: collection: 1
[PSYoungGen: 3072000K->1024K(3584000K)] 3072000K->1024K(7680000K), 0.0170110 secs]
Heap after gc invocations=1:
PSYoungGen total 3584000K, used 1024K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8d00050,0xffffffff18000000)
to space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
PSOldGen total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a768,0xfffffffd3fc00000)
{Heap before gc invocations=2:
PSYoungGen      total 3584000K, used 2891377K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 94% used [0xfffffffe3d400000,0xfffffffeeda9c6d8,0xfffffffef8c00000)
  from space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8d00050,0xffffffff18000000)
  to   space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
PSOldGen        total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a768,0xfffffffd3fc00000)
37.161: [GCAdaptiveSizePolicy::compute_survivor_space_size_and_thresh:  survived: 1015880  promoted: 0  overflow: falseAdaptiveSizeStart: 37.171 collection: 2
AdaptiveSizeStop: collection: 2
[PSYoungGen: 2891377K->992K(3584000K)] 2891377K->992K(7680000K), 0.0098960 secs]
Heap after gc invocations=2:
PSYoungGen total 3584000K, used 992K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xffffffff18000000,0xffffffff180f8048,0xffffffff37400000)
to space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a768,0xfffffffd3fc00000)
{Heap before gc invocations=3:
PSYoungGen      total 3584000K, used 992K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
  from space 512000K, 0% used [0xffffffff18000000,0xffffffff180f8048,0xffffffff37400000)
  to   space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen        total 4096000K, used 0K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd43400000,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a768,0xfffffffd3fc00000)
37.172: [Full GCAdaptiveSizeStart: 37.352 collection: 3
AdaptiveSizeStop: collection: 3
[PSYoungGen: 992K->0K(3584000K)] [PSOldGen: 0K->874K(4096000K)] 992K->874K(7680000K) [PSPermGen: 7593K->7593K(24576K)], 0.1802284 secs]
Heap after gc invocations=3:
PSYoungGen total 3584000K, used 0K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
to space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen total 4096000K, used 874K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd434da8c0,0xfffffffe3d400000)
PSPermGen total 24576K, used 7593K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6a768,0xfffffffd3fc00000)
{Heap before gc invocations=4:
PSYoungGen      total 3584000K, used 3072000K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 100% used [0xfffffffe3d400000,0xfffffffef8c00000,0xfffffffef8c00000)
  from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
  to   space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen        total 4096000K, used 874K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 0% used [0xfffffffd43400000,0xfffffffd434da8c0,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7611K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6eeb0,0xfffffffd3fc00000)
54.054: [GC-- [PSYoungGen: 3072000K->3072000K(3584000K)] 3072874K->5158809K(7680000K), 23.1725940 secs]
Heap after gc invocations=4:
PSYoungGen total 3584000K, used 3072000K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 100% used [0xfffffffe3d400000,0xfffffffef8c00000,0xfffffffef8c00000)
from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
to space 512000K, 1% used [0xfffffffef8c00000,0xfffffffef947a490,0xffffffff18000000)
PSOldGen total 4096000K, used 2086809K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 50% used [0xfffffffd43400000,0xfffffffdc29e6578,0xfffffffe3d400000)
PSPermGen total 24576K, used 7611K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6eeb0,0xfffffffd3fc00000)
{Heap before gc invocations=5:
PSYoungGen      total 3584000K, used 3072000K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
  eden space 3072000K, 100% used [0xfffffffe3d400000,0xfffffffef8c00000,0xfffffffef8c00000)
  from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
  to   space 512000K, 1% used [0xfffffffef8c00000,0xfffffffef947a490,0xffffffff18000000)
PSOldGen        total 4096000K, used 2086809K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
  object space 4096000K, 50% used [0xfffffffd43400000,0xfffffffdc29e6578,0xfffffffe3d400000)
PSPermGen       total 24576K, used 7611K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
  object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6eeb0,0xfffffffd3fc00000)
77.227: [Full GCAdaptiveSizeStart: 81.903 collection: 5
AdaptiveSizePolicy::compute_generation_free_space limits: desired_promo_size: 5310531271 promo_limit: 4194304000 free_in_old_gen: 2048607488 max_old_gen_size: 4194304000 avg_old_live: 4746786511456829440
AdaptiveSizePolicy::compute_generation_free_space limits: desired_eden_size: 5454285546 old_eden_size: 3145728000 eden_limit: 3145728000 cur_eden: 3145728000 max_eden_size: 3145728000 avg_young_live: 524328
AdaptiveSizePolicy::compute_generation_free_space:  minor_time: 0.289393 major_cost: 0.104945 mutator_cost: 0.605662 throughput_goal: 0.990000 live_space: 2422441472 free_space: 7340032000 old_promo_size: 4194304000 old_eden_size: 3145728000 desired_promo_size: 4194304000 desired_eden_size: 3145728000
AdaptiveSizeStop: collection: 5
[PSYoungGen: 3072000K->0K(3584000K)] [PSOldGen: 2086809K->2095406K(4096000K)] 5158809K->2095406K(7680000K) [PSPermGen: 7611K->7611K(24576K)], 4.6758162 secs]
Heap after gc invocations=5:
PSYoungGen total 3584000K, used 0K [0xfffffffe3d400000, 0xffffffff37400000, 0xffffffff37400000)
eden space 3072000K, 0% used [0xfffffffe3d400000,0xfffffffe3d400000,0xfffffffef8c00000)
from space 512000K, 0% used [0xffffffff18000000,0xffffffff18000000,0xffffffff37400000)
to space 512000K, 0% used [0xfffffffef8c00000,0xfffffffef8c00000,0xffffffff18000000)
PSOldGen total 4096000K, used 2095406K [0xfffffffd43400000, 0xfffffffe3d400000, 0xfffffffe3d400000)
object space 4096000K, 51% used [0xfffffffd43400000,0xfffffffdc324bad0,0xfffffffe3d400000)
PSPermGen total 24576K, used 7611K [0xfffffffd3e400000, 0xfffffffd3fc00000, 0xfffffffd43400000)
object space 24576K, 30% used [0xfffffffd3e400000,0xfffffffd3eb6eeb0,0xfffffffd3fc00000)
Apologies for the long trace information, but it's not clear yet what is going on. We even tried to add:
-XX:PretenureSizeThreshold=1000m
to try to force the signal matrix into old generation first upon allocation. This is where it should be, and would save the heap from having to promote it later. So question is: Is there something we're missing w.r.t. eden/from/to space ? Clearly, there is!
Thanks again!