Inactive memory. it's not working properly, how do i tweak the system to fix it?

Question

hi.
my system has 24 gigabytes of ram. I use After effects and photoshop and illustrator... all day, every day, going back over 10 years.
when I run a particularly big project in AE, one that uses big fies, or a lot of files, the inactive ram very quickly balloons out of control.
prime example: I am rendering right now, it takes about 12 minutes to render the comp, in 12 minutes the system will go from zero megabytes of inactive ram, to 17 gigabytes, leaving me with no ram for the system, causing in some severe cases: actual hard crashes. I continually have to monitor and run a "purge" command in the terminal in order to cut that out.
the documentation and general discourse I've read agrees that inactive memory is the same thing as free memory, in that it is available for applications with need to use it. But, it contains potentially useful data that was recently used, and the system would rather give you the free stuff before giving you the inactive stuff.
here's the problem:
Aftereffects uses this memory, and then frees it. it's full of gibberish, it gets used quickly, and released quickly, and pooled into inactive memory. At this point it should be availible for other apps to use, and it isn't. it is treated as if it is in use. The system can't use it, aftereffects can't use it, no other app can get this ram allocated to it. Inactive memory, becomes terminal memory, memory that is not available for use by the system.
I know there has to be a system default or a setting in the terminal, I can set to make it trigger happy and automatically free up this useless memory.
what is it?

Answer 1

Inactive memory is the operating system cache.
This cache may contains data which has not been written to disk (called dirty pages).  This is because Mac OS X defers flushing dirty cache data to disk to cut down on contantly starting up idle disks, consuming power, draining laptop batteries, and adding extra wear to solid state disks which have a limited number of writes.
When Mac OS X has lots of inactive memory and the system starts to slow down, this is because now the OS is being forced to write the dirty cached pages to disk before it can give those pages to another applications, or to reuse those pages with the same applciation.
After effects and photoshop and illustrator are very memory intensive apps.
...causing in some severe cases: actual hard crashes.
That is either a hardware problem, or an operating system bug.  You should file a Bug report including your crash dumps to Apple.
BugReporter
<http://bugreporter.apple.com>
Free ADC (Apple Developer Connection) account needed for BugReporter.
Anyone can get a free account at:
<http://developer.apple.com/programs/register/>
Aftereffects uses this memory, and then frees it. it's full of gibberish, it gets used quickly, and released quickly, and pooled into inactive memory. At this point it should be availible for other apps to use, and it isn't. it is treated as if it is in use. The system can't use it, aftereffects can't use it, no other app can get this ram allocated to it. Inactive memory, becomes terminal memory, memory that is not available for use by the system.
The inactive memory may still be part of the After Effects address space, just not in the current process working set.  That is to say, ever so often the operating system will mark pages inactive and see if the process references them again.  If it does, they are put back in the process' working set (quickly because they are still in RAM and it just requies updating some process tables), otherwise they sit in inactive memory.  If they are really part of After Effects address space, then chances are the data in those pages is dirty, and before they can be given to another process the pages have to be written to the /var/vm/swapfile(s).
If After Effects generated this inactive memory by reading files (and only reading), then those pages are clean (not dirty), and can easily be moved to free pages and given to a process as there is no need to write them  back to disk, as the disk has the same values as what is cached.
If After Effects generated this inactive memory by writing data to files, then the pages are dirty and are waiting to be written to disk.  Unless the file has been deleted after the write (such as a scratch file), then the operating system must write the data to disk before it can reuse the inactive pages.  It does not matter if later the file will be deleted, the operating system does not know that, so it cannot give those pages to another process until they have been written to disk.  The except being if and only if, a process is also accessing (reading or writing) that file, then cache starts to pay for itself.
The operating system does not refuse to give inactive memory to other processes, it just has responsibilities to make sure dirty pages are properly moved to stable storage before the pages are reused.  And generally speaking stable storage is slow.  Orders of magnitude slower.
If you want  to control when you the operating system cleans dirty inactive memory, then you may continue to run the 'purge' command.  As long as you are not running it non-stop, and especially if you are using a laptop or solid state disk.
Then again, getting more RAM will allow the operating system to dirty up more RAM before it needs to start flushing to disk.  And replacing your disks with solid state disks (SSDs) will allow the operating system to flush dirty pages much faster when it needs to start cleaning the inactive memory.
Message was edited by: BobHarris