Understanding Solaris 10 memory use

Question

Hi - does anyone know of a decent, easy to understand guide on understanding the above? I have a T4-1 Solaris 10 DB2 server with "top" installed that shows around 4G of free memory out of 64G real. But vmstat shows around 30G of free memory. The only thing running on the server is a single non-global zone that has memory capped at 32G - so I'm trying to figure out why "top" is showing so much used memory? I use Cacti to graph memory usage and this has crept up over a month or so. Swap used as far as top / Cacti is concerned has remained at 11G pretty much since the server was last rebooted.
vmstat:
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr s4 s5 s6 s7 in sy cs us sy id
0 0 0 86960112 32143992 26 74 0 0 0 0 0 -0 23 13 -0 1448 939 1179 0 0 100
So free is ~30G
top:
Memory: 64G real, 3197M free, 11G swap in use, 55G swap free
Any help would be appreciated, thanks - Julian.

Answer 1

   In general, I tend to be as confused at the array of different values you get from memory statistics as anybody. Clearly, different tools are most likely measuring different things with respect to memory usage and the values aren't necessarily comparable. It is often hard for those of us in the field to determine what these tools are actually measure. However, what you're describing as top's free memory value sounds like the size of the free list. Years ago I remember reading something about this. Let me tray to explain it - hopefully, someone more knowledgeable than I can correct it if I get it wrong.
   To put it simply, empty memory is a waste - you might as well have something loaded in case you need it later. So when you load something into memory, you don't throw it away until you know you need the space for something else. However, you have to have be able to grab memory quickly, so you plan ahead and keep a list of memory pages you can reuse at a moment's notice. The "free memory" size if the size of this list - the memory which is free to be reused at a moment's notice.
   It works like this. Initially, the free list is everything the OS isn't using. As the system runs, the memory get used and the free list gets smaller and smaller. Eventually, it reaches a high water mark which causes the kernel to run an internal task to look for more memory which can be freed if needed. This process works until it adds enough memory to the free list to reach a low water mark. So, at boot time you have a lot of free memory which slowly gets consumed. Once it reaches a certain point, the kernel maintains free memory between the high and low water marks. This sounds like what you describe. The memory use creeps up and then stays high.