Server Backup fails on HyperV Host - VSS Operation Failed

Question

Hi, I have a Server 2008 Enterprise R2 Host, which holds four virtual machines. I have setup to backup nightly to an external USB drive. Backups had been going fine for months until recently.
I now get this within 5 minutes of the backup starting, and the job fails:
The backup operation that started at '‎2012‎-‎10‎-‎16T00:18:12.638000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review
the event details for a solution, and then rerun the backup operation once the issue is resolved.
When I do a vssadmin list writers, all are Stable and No Error except these two:
I registered the HyperV VSS writers several months ago before I even started backing up.
The VHDMP Log shows these errors on two VM VHD files:
Failed to surface VHD \Device\HarddiskVolumeShadowCopy1284\VirtualHardDrives\VirtualMachine.vhd. Error status 0xC0000061.
I installed all latest updates to both the VM's and the Host machine and rebooted, but this made no difference.
What should I be looking for to resolve these issues?
Thanks

Answer 1

Hi,
MS KB 2013544 records this issue, you receive following event logged in the VHDMP log: 
Event ID 3: Failed to surface VHD <Drive Letter\Path\virtual hard drives\VMNAME_########-####-####-####-############.vhd. Error Status: 0xC0000061
There is a timing issue with FSDepends.sys and with VHDMP.sys. This timing issue occurs when certain backup programs are installed on computers that are running Windows Server 2008. By default, the FSDepends.sys start value in the registry is set to Manual.
When any third-party backup software loads its tape device driver, the software can sometimes result in FSDepends.sys and VHDMP.sys not initializing correctly.
To work around these problems, FSDepends.sys should be set to a start value of Boot (0x0).
Make sure that you back up the registry before you change it.
Make sure that you know how to restore the registry if a problem occurs.
For more information about how to back up, restore, and change the registry, click the following article number to view the article in the Microsoft Knowledge Base:322756How
to back up and restore the registry in Windows
To work around these problems, follow these steps:
1) Start Registry Editor.
2) Visit the following registry key: “HKLM\SYSTEM\CurrentControlSet\Services\FsDepends”
3) Under the “FsDepends” key, change REG_DWORD value “Start” from “3” to “0”
4) Restart the computer
More Information:
As soon as the computer is restarted, FSDepends.sys starts immediately and the timing issue no longer occurs.
Microsoft is aware of the issue and is currently investigating a resolution.
Or, you can try one of the following two workarounds. Neither of these methods is recommended. However, they work to correct the problems if editing the registry is not possible or desired.
- Turn off any attached tape device and restart the server. As soon as the server is started, turn on the tape device.
- Disable the device driver named TPFilter.sys
For more information please refer to following MS articles:
Error message when you manage a VHD file in Windows Server 2008: "A Virtual Disk Provider for the specified file was not found”
http://support.microsoft.com/kb/2013544
Lawrence
TechNet Community Support