CS5 on mac 10.6.4 slows system to a crawl, render times really long

Question

We recently upgraded our towers to 10.6.4 and all productivity has slowed to a snails pace using After Effects. On 10.5 the speed was almost TOO fast on our renders...it was awesome. Now I'm looking to get that speed back.
I have a few very simple comps. 1920 x 1080, 23.98fps, 5 layers (4 QT renders from final cut XDCAM and Animation codec, 1 adjustment layer with levels and hue/saturation for color correction). Approx 40 seconds long each.
I've followed the Adobe advice of turning the multiprocessing on and using the following settings from here http://forums.adobe.com/thread/543440
leave 4gb for other apps, set minimum 3gb per processor = 33min render
setting it back to what WAS screaming fast on 10.5:
leave 3gb for other apps, set minimum 0.75gb per processor = 19min render
turning multiprocessing off = 5min render
same project on 10.5 system with multiprocessing turned ON with the above mentioned settings = 2min render
what is going on here? is there a compatibility problem with CS5/10.6 and Animation codec files? that seems to be the bulk of the slowdown, but still over twice as slow?
additionally, when AE is rendering the rest of the system becomes unusable...every action, even dragging a finder window around, results in beachballs and a 30 - 40 second lag in response. this happens with or without multiprocessing.
I just want to get back to work...any ideas on how i can configure this beast to get back to my former speeds?
8 core mac Pro
OS 10.6.4
12gb RAM
many thanks in advance. i'm willing to send beer for a winning fix.

Answer 1

Good to know, thanks for the update. 10.6.x does manage memory differently, but our testing for performance has shown a slight improvement in the newer OS. So I am guessing that the OS update is not the issue. I just ran another test on a Mac with your configuration and saw a 4% improvement on 10.6.4 over 10.5.8.
It's uncertain what state the system memory was in when you started your tests. That's why I asked for a restart of the computer. The times reported seemed like there was a possibility that the OS was swapping. XDCAM footage places a high demand on system resources.  But it's hard to say without being there.
What I was seeing with the initial post was that the two extremes of allocation per background cpu were tried, but not the middle ground. To better see this, launch the Activity Monitor utility or look at the last line of the MP preference. With the 4GB reserved and the 3GB/ bkgnd cpu setting you weren't getting any extra processes spawned for multiprocessing. You can see this in the preferences multiprocessing section as "Actual CPUs that will be used = 0". Then, by setting the pref to .75GB/bkgnd the app launched 8 bkgnd processes, but for the task at hand, I am guessing that they were starved for memory and sometimes failed to render. This was faster than the first test, but still slow. By setting the pref to 1.5GB/bkgnd cpu you were now getting 4 new processes spawned for multiprocessing. The background processes had enough memory to succeed so that made it render faster still.
Now, try setting it to 1GB/bkgnd cpu. That will launch 6 processes which should render faster as long as that is enough memory for the task. If  the Memory pref for Reserve RAM for other apps is set back to 3GB, it will allow the spawning of 7 bkgnd processes. If that is a successful balance for this project then it will be the fastest render time. But, our testing shows it will be at the risk of starving the OS for memory, thus the more conservative 4GB recommendation for a 12GB system.
It's a fine line and why we are recommending that one give up a little speed by reserving more memory for other apps. True, there will be less processes spawned for multiprocessing, but there will also be less chance for the OS to start swapping to disk which greatly slows down performance for all tasks. If that compromise is not acceptible, then the alternative is to buy more ram so that all 8 processes can adequately receive enough memory for the job, and yet still maintain adequate reserves for the OS and other apps.