Mapping set heap sizes to used memory

Question

Hi all,
I've got a question about the parameters used to control your java process' heap sizes: "-Xms128m -Xmx256m" etc.
Let's say I set my min and max to 2Gb, just for a simplistic example.
If I then look at the linux server my process is running on, I may see a top screen like so:
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
10647 javaprog 20   0 2180m 1.9g  18m S  1.3  3.7   1:57.02 javaWhat I'm trying to understand is what relationship - if any - there is between these arguments and the figures I see within top. One thing in particular that I'm interested in is the fact that I occasionally see a RES (or more commonly a VIRT) size higher than the maximum that I have provided to Java. Naively I would assume that therefore there isn't a relationship between the two... but I wouldn't mind someone clarifiying this for me.
Any resources on the matter would be appreciated, and I apologise if this question is outside the realms of this particular subforum.
Dave.

Answer 1

Peter Lawrey wrote:
user5287726 wrote:
Peter Lawrey wrote:
It will always reserve this much virtual memory, plus. In term of resident memory, even the minimum is not guarenteed to be used. The minimum specifies at what point it will make little effort to recycle memory. i.e. it grows to the minimum size freely, but a "Hello World" program still won't use the minimum size.No, Linux does not reserve virtual memory. Just Google "Linux memory overcommit". Out-of-the-box, every Linux distro I'm aware of will just keep returning virtual memory to processes until things fall apart and the kernel starts killing processes that are using lots of memory - like your database server, web server, or application-critical JVMs. You know - the very processes you built and deployed the machine to run. Just Google "Linux OOM killer".Thats not the behaviour I see. When I start a process which busy waits, but doesn't create any objects, the virtual memory sized used is based on the -mx option, not how much is used. Given virtual memeory is largely free, why would an OS only give virtual memory on an as needs basis.
Busy looping process which does nothing.
In each case the resident size is 16m
option       virtual size
-mx100m      368m = 100m + 268m
-mx250m      517m = 250m + 267m
-mx500m      769m = 500m + 269m
-mx1g        1294m = 1024m + 270m
-mx2g        2321m = 2048m + 273mTo me it appears that the maximum size you ask is immediately added to the virtual memory size, even if its not used (plus an overhead) i.e. the resident size is only 16m.Yes, it's only using 16m. And its virtual size may very well be what you see. But that doesn't mean the OS actually has enough RAM + swap the hold what it tells all running processes they can have.
How much RAM + swap does your machine have? Say it's 4 GB. You can probably run 10 or 20 JVMs simultaneously with the "-mx2g" option. Imagine what happens, though, if they actually try and use that memory - that the OS said they could have, but which doesn't all exist.
What happens?
The OOM killer fires up and starts killing processes. Which ones? Gee, it's a "standard election procedure". Which on a server that's actually doing something tend to be the processes actually doing something, like your DBMS or web server or JVM. Or maybe it's your backups that get whacked because they're "newly started" and got promised access to memory that doesn't exist.
Memory overcommit on a server with availability and reliability requirements more stringent than risible is indefensible.