JNI_CreateJavaVM returns JNI_ENOMEM (-4) from different versions of the jre

Question

I have created a JNI dll, invoked from a VBA Excel add-in to execute Java code. This works fine until I introduce the -Xmx parameter as a vm option.
Before creating the VM, I use a VirtualAlloc/Free loop to make sure the -Xmx parameter is not too large, decrementing it by 2MB each time until an allocable amount is found (thanks to [jurberg's post|http://forums.sun.com/thread.jspa?forumID=37&threadID=5220601]; my version, slightly modified, is posted below). I then pass that value to the VM via the -Xmx option.
This works great when I am using a JRE 1.6 installed in the "C:\Program Files\Java\*jre1.6.0_xx"* directory. When the same* JRE version is installed in the "C:\Program Files\Java\*jre6*" directory, JNI_CreateJavaVM fails with JNI_ENOMEM. Calling GetLastError returns 0 ("operation completed successfully"). Using a *1.5 JRE*, this also fails, returning JNI_ENOMEM, but GetLastError returns error code 6 ("the handle is invalid").
A little about my platform:
     Windows XP Pro
     Building JNI dll using Microsoft Visual C++ 2008 and 1.5 JDK
I have multiple JREs and JDKs installed on my system as this is a dev machine, but I have the same problem on a non-dev machine running XP Home.
Here is a snippet of my code used to create the vm:
     // create the JNI structures
     const int numOptions = args.size();
     JavaVMInitArgs vm_args;
     JavaVMOption* options = new JavaVMOption[numOptions];
        log("Creating JVM with parameters:");
        int i = 0;
        char * nextArg;
        for (itr=args.begin(); itr != args.end(); itr++) {
            nextArg = new char[(*itr).length() + 1];
            strcpy(nextArg, (*itr).c_str());
            options.extraInfo = NULL;
options[i++].optionString = nextArg;
log("\t" + string(nextArg));
     vm_args.version = CRUSH_JNI_VERSION;
     vm_args.options = options;
     vm_args.nOptions = numOptions;
     vm_args.ignoreUnrecognized = JNI_FALSE;
// load and initialize the Java VM, and return a JNI interface pointer
JNIEnv* env = NULL;
     err = (*createVM)(&jvm, (void**)&env, &vm_args);
     // err is -4 (JNI_ENOMEM) in the cases described above
Does anyone have any suggestions on what is going on here and how I might make this code stable for all 1.5 and 1.6 JREs, regardless of where they are installed?
Thanks in advance,
Sarah
Code to determine max -Xmx value:static const DWORD NUM_BYTES_PER_MB = 1024 * 1024;
bool canAllocate(DWORD bytes)
LPVOID lpvBase;
lpvBase = VirtualAlloc(NULL, bytes, MEM_RESERVE, PAGE_READWRITE);
if (lpvBase == NULL) return false;
VirtualFree(lpvBase, 0, MEM_RELEASE);
return true;
int getMaxHeapAvailable(int permGenMB, int maxHeapMB)
DWORD          originalMaxHeapBytes = 0;
DWORD          maxHeapBytes = 0;
int               numMemChunks = 0;
SYSTEM_INFO          sSysInfo;
DWORD          maxPermBytes = permGenMB * NUM_BYTES_PER_MB;     // Perm space is in addition to the heap size
DWORD          numBytesNeeded = 0;
GetSystemInfo(&sSysInfo);
// jvm aligns as follows:
// quoted from size_t GenCollectorPolicy::compute_max_alignment() of jdk 7 hotspot code:
// The card marking array and the offset arrays for old generations are
// committed in os pages as well. Make sure they are entirely full (to
// avoid partial page problems), e.g. if 512 bytes heap corresponds to 1
// byte entry and the os page size is 4096, the maximum heap size should
// be 512*4096 = 2MB aligned.
// card_size computation from CardTableModRefBS::SomePublicConstants of jdk 7 hotspot code
int card_shift = 9;
int card_size = 1 << card_shift;
DWORD alignmentBytes = sSysInfo.dwPageSize * card_size;
maxHeapBytes = maxHeapMB * NUM_BYTES_PER_MB;
// make it fit in the alignment structure
maxHeapBytes = maxHeapBytes + (maxHeapBytes % alignmentBytes);
numMemChunks = maxHeapBytes / alignmentBytes;
originalMaxHeapBytes = maxHeapBytes;
// loop and decrement requested amount by one chunk
// until the available amount is found
numBytesNeeded = maxHeapBytes + maxPermBytes;
while (!canAllocate(numBytesNeeded) && numMemChunks > 0)
numMemChunks --;
maxHeapBytes = numMemChunks * alignmentBytes;
numBytesNeeded = maxHeapBytes + maxPermBytes;
if (numMemChunks == 0) return 0;
// we can allocate the requested size, return it now
if (maxHeapBytes == originalMaxHeapBytes) return maxHeapMB;
// calculate the new MaxHeapSize in megabytes
return maxHeapBytes / NUM_BYTES_PER_MB;

Answer 1

I have a similar, but I think much simpler problem. Namely, I get ENOMEM's when as far as I can tell there's plenty of memory available. It seems to have something to do with how Windows is configured, although I've never been able to determine what it could be.FWIW, in my case, I found that if I loaded my JNI dll into a console process, the max heap requested was always allocated. But when loading into Excel, the same amount would be too much. This was partly due to the fact that Excel has its own memory management, limiting the amount of memory workbooks can use. Also, it could be due to the vm not being able to reserve a contiguous chunk of memory for the max heap space.
Why (and how) separate the permanent generation space from the rest of the max heap? It seems you'll fail if you can't get that much space (which is the why) but how did you determine what it is?The VM uses the perm gen space plus the requested max heap space when attempting the VirtualAlloc call to verify that it can allocate the specified amount. The default perm gen is 64MB, but that can be changed via the -XX:MaxPermSize vm parameter, so I allow for any requested value.
What's CRUSH_JNI_VERSION? It's not in any .h file I have.That's just my own constant defined to be either JNI_VERSION_1_4 or JNI_VERSION_1_6.
Why are you messing with the bootclasspath? (I suspect you're adding something to it. Generally the VM can find it's own damn classpath).Yep, I'm adding the 2.1 JAXB jar to the bootclasspath because earlier 1.6 distributions included JAXB 2.0 and I needed 2.1.
-sarah