Jdeveloper 11g: Could not reserve enough space for object heap

Today morning, all of sudden my application running on jdeveloper 11.1.1.5.3 on a windows 7 64 bit machine stopped running. To be more precise I was hitting the following exception during the application deployment:

GC overhead exceeded

So, the natural answer was to look for permgen space and related memory jvm parameters for the possible issues. On looking into setDomain.env at jdeveloper//DefaultDomain/bin I found the following:


MEM_MAX_PERM_SIZE_64BIT="-XX:MaxPermSize=256M"
export MEM_MAX_PERM_SIZE_64BIT


So, I bumped it to 1024M.  I started my server again and it did not help. So, next thing to look was to check the jvm parameters at wls startup in jdev console and I found as follows:

C:\Oracle\Middleware11153\jdk160_24\bin\java -client   -Xms32M -Xmx512M -XX:CompileThreshold=8000 -XX:PermSize=64M  -XX:MaxPermSize=1024M -D

Looking into it string gave me another hint on parameter -Xmx512M which is also not quite enough and could be the cause of issue. So, I decided to bump it to 1024M. To change this parameter I had options to update it in my jdev launch script which looks like as follows:

myjdev.cmd
============================

set MW_HOME=C:\Oracle\Middleware11153
set JAVA_HOME=%MW_HOME%\jdk1.6.0_34
set PATH =%JAVA_HOME%\bin;%PATH%
set JDEV_USER_HOME=C:\Users\vskumar\AppData\Roaming\JDeveloper
set FADEV_VERBOSE=true
set USER_MEM_ARGS=-Xms256m -Xmx256m -XX:MaxPermSize=1024m -XX:CompileThreshold=8000
%MW_HOME%\jdeveloper\jdeveloper.exe


So, I went ahead and modified the line starting with set USER_MEM_ARGS to look like as follows:
set USER_MEM_ARGS=-Xms256m -Xmx1024m -XX:MaxPermSize=1024m -XX:CompileThreshold=8000

and fired this command. After jdev launch I started the integrated wls and it failed with error as follows:


Error occurred during initialization of VM
Could not reserve enough space for object heap


as I had changed Xmx param as the last thing so kind of guessed it as the issue and reduced it to 768M. And the same problem. So, I reduced it further to 640M and this I did not see the error. Though 640M was not satisfactory but was higher than previous 512M so, I thought its ok to go ahead and hopefully the error will go away as well. And, after couple of minutes later against my expectations it failed again with a new error this time. It was as follows:



**************************************************************************
The WebLogic Server encountered a critical failure
Reason: unable to create new native thread
***************************************************************************
 
Throwing I/O exception: "MapViewOfFile: handle=7212 hiOffset=0 loOffset=0 size=1114112, Not enough storage is available to process this command., errno=8" errno = 8
Throwing I/O exception: "MapViewOfFile: handle=7212 hiOffset=0 loOffset=0 size=1114112, Not enough storage is available to process this command., errno=8" errno = 8
Throwing I/O exception: "MapViewOfFile: handle=7212 hiOffset=0 loOffset=0 size=1048576, Not enough storage is available to process this command., errno=8" errno = 8

This kind of indicated me a physical memory issue with RAM or harddisk. I have 8gb ram on my machine with no app running so was sure that certainly I do not have low memory. I had more than 10 gig free space on my c drive where all this was running. So, what to do next? I reviewed my setDomain.env, ide.conf and jdev.conf files and changed all the places mem related parameters to 1024M but all with no results. I then changed my cmd script Xmx parameter back to 1024M as I thought this error is just an effect of error above it and will go away if I can fix that one. Hence, I was back to error


Error occurred during initialization of VM
Could not reserve enough space for object heap


I decided to google for it and bumped into a thread talking about the possible reason that windows tries to allocate a contiguous memory to run vm and it fails to do so due to fragmentation etc and hence causes it. One of the solution in the same thread suggested to switch to a 64 bit jdk as a possible fix. Having no other options I downloaded jdk6 64bit at location c:\proram files\java\jdk1.6.0_34 and updated my launch script JAVA_HOME line as follows:

set JAVA_HOME = C:\Program Files\Java\jdk1.6.0_34

Hoping all goes well i started jdev and then integrated wls. During startup I observed it is still picking up the old 32 bit version which was strange. So, I started looking again what is the issue for not picking the 64 bit jdk. I looked into jdev.conf and found an entry like:

SetJavaHome C:\Program Files\Java\jdk1.6.0

So, thinking this could be the issue I closed my jdev and commented this line. I kicked my jdev again and on startup as expected it asked me to provide java.exe location. I provided the 64 bit version java.exe location and finally jdev started. However, on starting the integrated WLS I was seeing the same old 32 bit version picked. I thought it might be picking it from setDomain.env and opened it but found its referring JAVA_HOME param rather a hard coded value. I thought to give it a try and added the value manually to it as follows:

set JAVA_HOME = "C:\Program Files\Java\jdk1.6.0_34"

I kicked my domain startup again but was still seeing the old jdk used and was frustrated on whats going on. I had no guesses left here and all of a sudden a friend suggested could it be that it doesn't like spaces? With no other options left we thought to give that too a try and i removed spaces to look it like:

set JAVA_HOME="C:\Program Files\Java\jdk1.6.0_34"

And, that was the culprit here! The spaces which my friend discovered as a possible issue was the least expected thing to be causing this particular issue. So, I then removed this hardcoded value and fixed my script launcher values as well accordingly.

Needless to say, all my issues related memory got resolved and things worked fine after spending almost a day on this issue.

Key Takeaways
===========================
1. Make sure not to have extra spaces in the set parameters in case of using windows.
2. It is strongly recommended to use a 64 bit jdk. I am not sure why oracle ships 32 bit jdk with jdeveloper when this is recommended in dev guide as well.
3. Have a launcherScript to set jvm params on startup rather launching jdev directly from program menu.