Too much caching in 2008, how can I limit file-cache?

Question

2008 Server with enough RAM, client copies a server stored file to the local drive, the server "eats" up all his ram, and when the server hits the limit it slows down to crawl speed, like he would be swapping out his cache too. Especially annoying if the server runs hyper-v guests, they nearly stop. I saw that effect on a lot of servers, but all my searches failed. 2008-R2 does not show that annoying effect, so I hope again it can be adjusted in 2008.
See this screenshot (Update interval of taskmanager is 5 seconds): http://csb-filderstadt.homepage.t-online.de/Too-much-caching.png
Server 2008 enterprise, 16 GB RAM, 2 Quad-core, uses 650 MB (the hyper-v guests are not running) when without load. One client copies a .vhd file of about 70 GB from the server to himself. The network speed is OK, about 100 MBytes/s, copy speed slows down to about 80 MB/s when the RAM limit is reached.
But how can I prevent that the simple file cache eats up all RAM, and then slows down the machine to crawl, including all hyper-v guest (if they are running). Opening a taskmanager takes about 1 minute when the limit is reached, normally less than a second.
Is there a setting in the registry so I can either limit the RAM used for caching, or a way to tell the filecache to leave more free RAM for the actual work?

Answer 1

Guys,
This is an old post but I have been looking everywhere for a solution. 
Below is the server config:
Windows 2008 R2 Standard 64bit
12 GB Ram
HyperV - 2 Virtual
Virtual 1 - 1 GB - Virtual server only created and powered on not used
Virtual 2 - 2 GB - Virtual server only created and powered on not used
Exchange 2010 on Physical server
less than 10 users on exchange
Current Mail box per user - approx 3 GB
AV - Nod32
BES Express on Physical server
When I reboot the server the Memory usage is around 60% but slowly store.exe starts to eat the memory and finally it reaches 98% and then server hangs. This is so regular that sometimes I have to reboot the server 3 or 4 times a week.
BTW, I tried to limit the memory usage to 3GB via ADSI after googling but store.exe keeps on using and there was no use making this change.
ADSI change: msExchESEParamCacheSizeMax = 393216 
Can someone please please help me.
Regards
J
Your setup is wrong for your usage.
My recommendation would be:
1 Virtual Exchange with 8 GB RAM (or 6GB with dynamic up to 8GB) and four virtual CPU's
1 Virtual Blackberry Server with 2 GB RAM and two virtual CPU's.
Host machine: Only AV and Hyper-V, nothing else, not even connected to an active directory domain if not REALLY needed.
This will solve your memory issues without requiring hacking 'round the cache settings of Exchange.