Disk caching on host or guest?

OK, this is probably a noob question, but if we have 64GB RAM on our HyperV (2008R2) host, and we are running disk intensive software, do we:
a) Allocate the 'minimum' RAM to the guest, and leave the rest for the host to use for disk caching, or
b) Allocate the maximum RAM to the guest (leaving 1GB for the host), and let the guest use it for disk caching?
Allocating half & half would seem to be a waste as they will probably both end up caching the same data (will they?), but it's not clear whether we're best letting the host or the guest do the caching. Or does it actually matter at all?
I've had a good look around and haven't been able to find any relevant recommendations.
More Info - the 'disk intensive' software is mainly a PostgreSQL server. We'll give that about 8GB for its shared buffers, but it seems to be recommended to use OS disk caching beyond that. There is a 1GB BBWC P420i RAID controller so write caching is performed
on that. Currently, our biggest performance bottleneck seems to be due to uncached reads, so we are increasing the host RAM from 16GB to 64GB (and adding an SSD for index storage), but just want to know whether it's best to increase the guest RAM allocation,
or leave it 'spare' on the host.
With Windows Server 2008 R2 / Hyper-V 2.0 you don't have that many options as VHD access is not cached by host. At all... So you'd better allocate move VM memory as I/O would be cached inside a VM. Windows Server 2012 R2 / Hyper-V 3.0 would give you more
caching options that include Read-Only CSV Cache, Flash-based Write-Back Cache coming with Tiering and also SMB access is also extensively cached @ both client and server sides. See:
CSV Cache
http://blogs.msdn.com/b/clustering/archive/2013/07/19/10286676.aspx
Write Back Cache
http://technet.microsoft.com/en-us/library/dn387076.aspx
Hyper-V over SMB
http://technet.microsoft.com/en-us/library/jj134187.aspx
So it could be a good idea to upgrade to Windows Server 2012 R2 now :)
You may deploy third-party software to do a RAM and flash cache but you need to think twice as it could be simply dangerous - no reboot you may lose gigabytes of your transactions...
Hope this helped a bit :)
StarWind VSAN [Virtual SAN] clusters Hyper-V without SAS, Fibre Channel, SMB 3.0 or iSCSI, uses Ethernet to mirror internally mounted SATA disks between hosts.

Similar Messages

Strange performance issue with 3510/3511 SAM-FS disk cache

Hi there!
I'm running a small SAM-QFS environment and have some strange performance issue on the disk storage part, which somebody here might be able to explain.
Configuration: one 3510, dual controller, RAID-5 9+1, one hot spare and one disk not configured for whatever reason. The R5 logical drive hosts a 150GB LUN for SAM-QFS metadata (mm in SAM-FS speak) and a 1TB LUN for data (mr in SAM-FS speak). Further, there are two small LUNs (2GB, 100GB) for some other purpose. Those two LUNs have nearly no I/O. All disks are SUN146G. Host connection is 2GBit, multipathing enabled and working.
Then the disk cache became too small, and the customer added a 3511 expansion unit with SUN300G disks. One logical drive is a RAID-1, 1+1, used for NetBackup catalog. The other is a RAID-5, 8+1, providing two LUNs: 260GB SAM-FS metadata (mm) and 1.999TB SAM-FS data (mr).
For SAM-FS, the LUNs form two file systems: one "residing" in the 3510, the other "residing" in the 3511 expansion. Cabling is according to the manual and checked several times by several independant people. Operating system is Solaris 10, hardware is a V880.
The problem we observe: SAM-FS I/O on LUNs on disks inside the 3510 is fine. With iostat, I see 100MB/s read and 50MB/s write at the same time. On the SAM-FS file system which is running on the two LUNs in the 3511, the limit seems to be at 40MB/s read/write. Both SAM-FS file systems are configured the same in regards of block size.
In case I have activity on both SAM-FS file systems, I see 100MB/s+ on the LUN running inside the controller shelf and another 40MB/s on the disk runnin in the 3511 expansion chassis. So, the controller is easily capable of handling 150MB/s.
Cache settings in the 3510 controller are default I think (wasn't installed by me), batteries are fine.
Is this 40MB/s we experience a limitation by the expansion shelf? Don't think so. Anybody has any ideas on this? What parameters to check or to change? Any hint appreciated. I can also provide further details if needed. Thank you.
wolfgang

SUN300G disks sound like 300GB FC disks.
Depending on how many files are in the SAMFS file system, sharing the mm and mr devices on the same RAID array can be a pretty horrible idea. In my opinion and experience, it's almost always better to NEVER put more than one LUN on a RAID array. Period. Putting more than one LUN on an array results in IO contention on that array. And large, unnaturally configured (9+1? Why?) RAID arrays will have problems from the start.
What are the block sizes used on the RAID arrays? It wouldn't surprise me to see that the RAID array on the expansion tray has a very large block size. Larger block sizes are, in general, not better. Especially for SAMFS metadata - which IIRC is something like 8k or 16k blocks.
I suspect what is happening is most of the metadata updates are going to the mm device on the new array, contending with the IO operations on the file data.
How much space is left on each mm device? What does "iostat -sndxz 2" show when you're having the IO problems?

Install error on CC and ERROR code 50. Also AE would crash due to full disk cache and close unable to select from menu to change.

I have recently installed and used Creative Cloud and also install several applications. The apps and cc ran fine but Adobe AE would crash upon startup of the app in which is prompted a message stating ...the drive which it is installed on does not have enough memory or disk cache is full. Please change in preferences. I attempt to do so but AE shuts down after I select ok. I installed the cleaner tool and uninstalled all Adobe products and attempted to reinstall creative cloud and now I get a "...We've encountered the following issues: Creative Cloud desktop failed to install. (Error code:50)
I have been trying to resolve this issue for three weeks now which I did finally get it to work by opening the AE by selecting a AE file. I was able to use it 3 times for the amount charged for the monthly fee and I also downloaded several of the other apps but was
unable to use them due to this issue then uninstalled with Adobe cleaner tool!
PLEASE HELP!
Message was edited by: Paul Whitehurst

Jeff,
I have tried to install on another drive after downloading Creative Cloud clean-up and uninstalled all adobe creative cloud. So if I erase my temp folders it might work?
Please help I have basically paid for my 2nd month and only used a few times.
Thanks,
Paul

Error Message: Disk Cache folder doesn't exist

I know this has already been asked, but I am aware that I HAVE deleted the disk cache folder by accident, and it is no longer in the recycle bin. Is there anywhere i can go to restore the Disk Cache folder, or redownload it? How does it affect me to work without it? Thanks

Just create a new folder and point to it in the preferences. It's as easy as that.
You could also delete your preferences, re-launch, and everything should be restored.

Cannot ping for either from host to guest nor from guest to host

Hi I have installed virtual box 4.1.20 r80170
my host os is centos 6.3 64 bit
my guest os is oracle linux 5.8 64 bit
I've using bridge adapter for the network settings, but I'm neither able to ping
from host to guest
from guest to host
may I know what's wrong?
how should I trouble shoot this?
my host os network configurations is as follow:
[oracle@localhost ~]$ /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr E8:E0:B7:D2:F9:54
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:20 Memory:c4800000-c4820000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:3488 errors:0 dropped:0 overruns:0 frame:0
TX packets:3488 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:320590 (313.0 KiB) TX bytes:320590 (313.0 KiB)
wlan0 Link encap:Ethernet HWaddr 9C:B7:0D:96:F4:DF
inet addr:192.168.0.199 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::9eb7:dff:fe96:f4df/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:22932 errors:0 dropped:0 overruns:0 frame:0
TX packets:22068 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15525818 (14.8 MiB) TX bytes:5127595 (4.8 MiB)
I'm using wlan0 on the host os
my eth0 configurations on the guest os is as follow:
#Intel Corporation 82540EM Gigabit Ethernet Controller
DEVICE=etho0
BOOTPROTO=static
HWADDR:08:00:27:08:19:50
ONBOOT:yes
DHCPHOSTNAME=source.localdomain
IPADDR=192.168.0.11
NETMASK-192.168.0.1
TYPE=Ethernet
USERCTRL=no
IPV6INIT=no
PEERDNS=yes
I've a router to connect to the Internet. It is dlink DIR-615
I've done a dmesg while pinging from guest to host here's the partial output:
spurious NAK ON isa0060/serio0
some program might be trying to acces hardware directly
e100:eth0 NIC link is up 1000 Mbp
Full Duplex, Flow Control: RX
ADDRCONF (NETDEV_UP) eth0: link is not ready
ADDRCONF (NETDEV_CHANGE): eth0 becomes ready
eth0: no IPv6 routes present
Any assistance is deeply appreciated!
thanks a lot!
Edited by: oraclewannabe2 on Aug 25, 2012 11:28 AM
Edited by: oraclewannabe2 on Aug 25, 2012 12:10 PM

Hi ,
This is OL 6.
Now,after doing guestadditions this is working fine altough I am not sure what exactly guest addition is.
Additionally,I installed firefox in same VM but when I click on firefox then nothing comes up.
Best regards,
Vishal

After Effects on mac error message, Disk Cache, can't do anything (CS6)

Hello,
My name is Lukas.
Everytime when I open my After Effects, the render window opens up and an error about the Disk - Cache appears.
I own the version 11.0.4.2 for CS6.
I cleared the DC already, I do have enough available Disk space, but I cant do anything.
All I can do is to open up the preferences Window and change Settings.
I can't do nothing in the rest of the menu bar (for example open up a new composition).
(I'm sorry for my english, my motherlanguage is german)
I would be very happy if you could help me with my problem!
Thanks in advance!

Na dann erzähl uns doch nochmal alles richtig. Aus dem Gestammel wird keiner schalu, zumal du uns keinerlei Informationen gegeben hast, wo denn der Cache liegt, wie deine Festplatten organisiert sind und welches System du verwendest...
Mylenium

Is After Effects CC disk cache taking up double the space on my RAID 0 SSD? Should I move it?

In AE CC the available space on my RAID 0 (2 x 120GB SSD) drive drops from 100GB to around 40GB as soon as I do even a small RAM preview and even though I have set the disk cache in the preferences to 30GB. It also causes AE to prompt that I may not have enough space on the drive for the amount of cache I have set in preferences even though there appears to be 40GB left on it.
I really value working with long fast RAM previews but other than that I'm not sure if the performance is jeopardised at all here. At one point I was enjoying 60GB of disk cache.
RAID 0 is striped so it shouldn't take up double the space, am I right?
I am about to install a RAID 0 HDD (2 x 3TB) for source files as I work with video a lot and have a 4K RAW project on the go so no time like the present. So should I move the disk cache from the ridiculously fast RAID 0 SSD to the new RAID HDD drive (as suggested in Harm's guide)?

Thank you Mylenium. All this time I have been absolutely convinced that you don't need to defragment SSDs. Is this a complete misunderstanding? Even if you do though, I have Windows 8 (not 8.1) which is doing that once a week automatically (apparently even to SSDs).
Are you saying that the 30GB disk cache set up works like a contiguous file?
i7 3930k (12 virtual cores) on P9X79 Deluxe
2 x 120GB Force GT Raid 0 SSD on SATA 6gb/s is my boot drive as well as where I keep programs and the disk cache. I do a disk clean up to remove temp files a couple of times a week but that's all.
1 x 3TB Barracuda on on SATA 6gb/s for back ups and exports
3TB Lacie on USB 3 for source video and project files
2TB Lacie on USB 3 for archive projects
660ti 2GB
I'll try a manual defrag of my RAID 0 SSDs now.

Forcing the disk cache to be written to disk

Hi all. We are looking for a way to insure the content of the icommon in the ufs on disk as we need to read it. However, calling sync is async and does not seem to provide what we're looking for. When a file is updated, created all the information is not immediately written to disk. It is kept in the unified memory and later flushed to the disk cache which then writes it down on the media. However, since we're reading the hot FS we need to force ALL file metadata AND data to be written to the media.
Seems like when I run the command 'ff', probably as a side-effect, this is happening. ff takes too long and is at the FS level. Is it a side-effect, or does 'ff' calls a specific func (truss did not reveal anything useful).
Thanks all and best regards.

I. What is really happening under the hood?
1.sync(2) passes execution into kernel mode
2.kernel function syssync() is called then
3.then vfs_sync(0)
4.eventually ufs_sync(vfsp,�) is called with NULL as a first argument
in case of NULL(as a 1st arg) ufs_sync() just schedules but does not necessarily completes the writing of ufs metadata before returning.
So, this behavior completely matches with one described in man section for sync(2).
II. directio does not solve your problem because it affects only the way the file data( not metadata) goes to disk. ( read directio(3C) and mount_ufs(1M) carefully )
III. What you really need
May be it seems to be a heavy weapon for you but one of the possibilities is:
To write a loadable system call (loadable kernel module) that will
invoke ufs_sync() with proper arguments (non NULL vfsp) for mounted file
system of interest.

Media/Disk cache, etc. setup for a desktop and laptop dual configuration

Hi,
(New User)
I have a desktop at home, and a laptop. I split my time about 50/50 between the two. I use it for AE, PP, PS mainly
Desktop hard drive setup:
1 SSD (Currently, Windows 8 and Adobe CC installed)
2 - 1TB (7200rpm) drives (1 drive is where I put my project files (gets sync'd with google drive) and the other drive isn't being used at the moment)
Laptop:
1TB Hard drvie (Of course, Win 8 and Adobe CC installed) (And, the folder where I do my projects gets sync'd with google drive).
As you can see, because I split my time between a laptop and desktop, I need to be working with the most current files (which is why I sync them through google drive).
My question is (please remember I am new), where should I set up disk cache, media cache, and anything else so that it is effecient and more importantly so I can use either my laptop or desktop to work on the same files?
Again, I barely know what I am talking about here. I really don't understand what media cache, disk cache, etc. is. I am trying to learn but I want to get this setup first so I can start with the tutorials on lynda.com.
Thanks,
Chris

First you do not need disk cache for all projects. For some it may even get in the way. Second, put your disk cache wherever you would like. If you have a second drive that is fast, use that. If you have room on your SSD drive, use that.
If you get into trouble with a comp and previews are fouled up or things just don't look right, empty your disk cache. It's not a necessity for working with AE, it's just a convenience for some projects that may help you work a little faster.

Disk Cache memory problem

Hello,
I am using 5.5 AF (Mac) my Disk cache memory was full. Whenever i start application I got this massage & its effects on speed. Can some one guide me how do I clear it?
Thanks.

The same answer that I gave you here:
http://www.videocopilot.net/forum/viewtopic.php?f=4&t=116607

Best place to install After Effects, and best place to have AE Disk Cache

Hey, I have a new Samsung 840 EVO 250gb which I know I am already gonna install my OS on it, but not sure what to do with After Effects. I have heard it is good to have your Disk Cache on an SSD for better write performance, but should I have After Effects also installed on the SSD, or have AE installed on the Seagate Mechanical 1TB hard drive that I have now?
Thanks,
Jacob

I'd put the OS & application software on the hard drive. The reason: once launched, there aren't nearly as many read-write instances as there would be for an AE cache. You launch the OS... what, once a day? How about applications.... maybe 3-4 times a day?
On the other hand, when the AE cache is on the SSD, it it reads & writes to it all the time. That's where you want the real speed.
It might be nice to launch in the blink of an eye, but you're taking up valuable -- and pricey -- SSD real estate with OS & applications, things that aren't used all that often in comparison to the AE cache.

Best Drive For "Disk Cache"

Hello Everyone.
By default, the After Effects Disk Cache is located somewhere in the "C Drive", Temp Folder.
Since all the applications are Installed In the "C Drive", Therefore, Is It better to use another Hard Drive for the temp folder or just leave It alone ?
Both my "C Drive", and my Other drive each have a 400 GB of Empty Space.
My Drives are not SSD Drive, just a regular Hard Drives.
Thanks.

Cache should be on a the fastest drive on your system that has a fair amount of free space. Simple as that. If your fastest drive is the same drive that contains the OS and there is a bunch of free space then that may be the best option. If you have other drives on your system that are as fast, then pick one of them. It's as simple as that. Cache needs room and speed before it needs to be on a separate drive.

Setting Disks/Caches/Vault for multiple projects - Best Practices

Please confirm a couple assumptions for me:
1. Because Scratch Disk, Cache and Autosave preferences are all contained in System Settings, I cannot choose different settings for different projects at the same time (i.e. I have to change the settings upon launch of a new project, if I desire a change).
2. It is good practice to set the Video/Render Disks to an external drive, and keep the Cache and Autosave Vault set to the primary drive (e.g. user:Documents:FCP Documents). It is also best practice to save the Project File to your primary drive.
And a question: I see that the Autosave Vault distinguishes between projects, and the Waveform Cache Files distinguishes between clips. But what happens in the Thumbnail Cache Files folder when you have more than one project targeting that folder? Does it lump it into the same file? Overwrite it? Is that something about which I should be concerned?
Thanks!

maxwell wrote:
Please confirm a couple assumptions for me:
1. Because Scratch Disk, Cache and Autosave preferences are all contained in System Settings, I cannot choose different settings for different projects at the same time (i.e. I have to change the settings upon launch of a new project, if I desire a change).
Yes
2. It is good practice to set the Video/Render Disks to an external drive, and keep the Cache and Autosave Vault set to the primary drive (e.g. user:Documents:FCP Documents).
Yes
It is also best practice to save the Project File to your primary drive.
I don't. And I don't think it matters. But you should back that file up to some other drive (like Time Machine).
And a question: I see that the Autosave Vault distinguishes between projects, and the Waveform Cache Files distinguishes between clips. But what happens in the Thumbnail Cache Files folder when you have more than one project targeting that folder? Does it lump it into the same file? Overwrite it? Is that something about which I should be concerned?
I wouldn't worry about it.
o| TOnyTOny |o

Disk cache javascript functions and images

Hi,
How can I force client browsers of my web application to cache the images and other javascript entities like css, functions, js files, etc.
By default it is not caching, though it should have. I have not written any code saying not to cache. Any help.
Environment : Tomcat server - java beans - jsp [struts framework]
In Netscape 7 if I say 'page view' it shows images and media files of my applications are not cached. Thats why application is really slow. If I compare this to some other web application which is fast, I see that media files, js files and images gets 'DISK CACHED'.]
Can anyone explain me what is disk cache and how to get it disk cached?
Thanks,
sunil

Thanks. I now understand whats the concept of disk cache.
But after lot of analysis also my problem still remains the same. Just that there are some patterns available.
I have some images in my own application. I also have some images which reference another site [application]. Interestingly all images which reside in my appication are not getting cached. All image which I refer into other site are getting cached. Can anyone makeout anything from this?
I looked into META tag generated for resulting htmls. But they dint have anything which says browser not to cache.
Can anyone list any java/jsp/html instructions which can tell browser not to cache stuff.
Or more importantly is there any setting or instruction or command which I can write in my java/jsp/html which can override any such setting done.
Thanks,
Sunil

Adobe After Effects Disk Cache folder is massive!!!

Hi. My mac computer is starting to get out of free space so I use omnidisk to do some cleaning on my computer and I notice that there a folder in user/library/preferences/adobe/after effect/11.0/Adobe After Effects Disk Cache - user's MacBook Pro.noindex/.... that has lots of different folders, named with numbers, that has hundreds if not thousands of files that goes from 100kb up to 9mb! The total space taken by these files are 49gb!!!
I did some fast research in here: http://helpx.adobe.com/after-effects/using/memory-storage.html#caches_ram_cache_disk_cache _and_media_cache and it says that after effects store some render data in the computer (i think )... but I'm really not going to open any of the old after effects files so my question is... can i delete that folder??? or is it vital for the program?
By the way my computer has 750gb and 16gb of ram sooo yeah any help will be appreciated...

Welcome to disk cache. You can set the disk cache size in the preferences. On my laptop I keep it at 6GB. If you are using it you should be emptying it on a regular basis.

Disk caching on host or guest?

Similar Messages

Maybe you are looking for