Hyper-v Live Migration not completing when using VM with large RAM

Question

hi,
i have a two node server 2012 R2 cluster hyper-v which uses 100GB CSV, and 128GB RAM across 2 physical CPU's (approx 7.1GB used when the VM is not booted), and 1 VM running windows 7 which has 64GB RAM assigned, the VHD size is around 21GB and the BIN file
is 64GB (by the way do we have to have that, can we get rid of the BIN file?). 
NUMA is enabled on both servers, when I attempt to live migrate i get event 1155 in the cluster events, the LM starts and gets into 60 something % but then fails. the event details are "The pending move for the role 'New Virtual Machine' did not complete."
however, when i lower the amount of RAM assigned to the VM to around 56GB (56+7 = 63GB) the LM seems to work, any amount of RAM below this allows LM to succeed, but it seems if the total used RAM from the physical server (including that used for the
VMs) is 64GB or above, the LM fails.... coincidence since the server has 64GB per CPU.....
why would this be?
many thanks
Steve

Answer 1

Hi,
I turned NUMA spanning off on both servers in the cluster - I assigned 62 GB, 64GB and 88GB and each time the VM started up no problems. with 62GB LM completed, but I cant get LM to complete with 64GB+.
my server is a HP DL380 G8, it has the latest BIOS (I just updated it today as it was a couple of months behind), i cant see any settings in BIOS relating to NUMA so i'm guessing it is enabled and cant be changed.
if i run the cmdlt as admin I get ProcessorsAvailability : {0, 0, 0, 0...}
if i run it as standard user i get ProcessorsAvailability
my memory and CPU config are as follows, hyper-threading is enabled for the CPU but I dont
think that would make a difference?
Processor 1 1 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 1 4 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 1 9 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 1 12 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 2 1 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 2 4 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 2 9 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor 2 12 Good, In Use Yes 713756-081 DIMM DDR3 16384 MB 1600 MHz 1.35 V 2 Synchronous
Processor Name
Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz
Processor Status
OK
Processor Speed
2400 MHz
Execution Technology
12/12 cores; 24 threads
Memory Technology
64-bit Capable
Internal L1 cache
384 KB
Internal L2 cache
3072 KB
Internal L3 cache
30720 KB
Processor 2
Processor Name
Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz
Processor Status
OK
Processor Speed
2400 MHz
Execution Technology
12/12 cores; 24 threads
Memory Technology
64-bit Capable
Internal L1 cache
384 KB
Internal L2 cache
3072 KB
Internal L3 cache
30720 KB
thanks
Steve