Dynamic Memory on Linux VM
Hello!
Hyper-V 3.0 is great! After it will be released, I think it will become the most popular hypervisor. But it remains a major drawback.
Nowhere announced support for dynamic memory for Linux VM on Hyper-V.
Planned at least in some perspective to implement this functionality?
Now we have to use two different hypervisors, as Hyper-V does not meet all the requirements of our customers.
Mark Tepterev
Oversun
~
~
Moved P.P.P.S.
Q from Brian Wong:
----- Original Message -----
From: "Brian Wong"
To: <[email protected]>
Sent: Thursday, March 06, 2014 9:24 AM
Subject: Re: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled
On 3/6/2014 1:20 AM, Brian Wong wrote:
> The kernel is built with the full set of Hyper-V drivers, including the
> key "Microsoft Hyper-V Balloon Driver" as well as memory hot-add and
> hot-remove functionality. This is happening with both the Gentoo-patched
> 3.10.32 kernel and the vanilla 3.12.5 kernel. The host machine has a
> total of 24 GB of memory.
>
> For now, I am working around the issue by starting the VM with the
> startup memory set to the maximum and letting Hyper-V take the usused
> memory back when it is not in use. The VM will then get the extra memory
> when it needs it.
>
> Have I encountered a bug in the Hyper-V balloon driver?
>
Just a correction: the vanilla kernel version is 3.13.5, not 3.12.5.
Sorry for any confusion.
Brian Wong
http://www.fierydragonlord.com
----- Original Message -----
From: "Brian Wong"
To: <[email protected]>
Sent: Thursday, March 06, 2014 9:20 AM
Subject: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled
I'm new to LKML, so please don't be too hard on me :)
I'm running Gentoo Linux under Microsoft Client Hyper-V on Windows 8.1
Pro, and I've noticed some odd behavior with respect to dynamic memory
(aka memory ballooning). The system will never use more than the startup
memory defined in the virtual machine's settings.
( VVM: typewriting error viRtual fixed by me, for best search in future )
For example, if I set the startup memory to 512 MB, and enable dynamic
memory with a minimum of 512 MB and a maximum of 8192 MB, the system
will never allocate than 512 MB of physical memory, despite Hyper-V
assigning more memory to the VM and the added memory being visible in
the output of "free" and "htop". Attempting to use more memory causes
the system to start paging to swap, rather than actually allocating the
memory above the startup memory assigned to the VM.
The kernel is built with the full set of Hyper-V drivers, including the
key "Microsoft Hyper-V Balloon Driver" as well as memory hot-add and
hot-remove functionality. This is happening with both the Gentoo-patched
3.10.32 kernel and the vanilla 3.12.5 kernel. The host machine has a
total of 24 GB of memory.
Brian Wong wrote On 3/6/2014 1:20 AM:
Just a correction: the vanilla kernel version is 3.13.5, not 3.12.5. )
For now, I am working around the issue by starting the VM with the
startup memory set to the maximum and letting Hyper-V take the usused
memory back when it is not in use. The VM will then get the extra memory
when it needs it.
Have I encountered a bug in the Hyper-V balloon driver?
Brian Wong
http://www.fierydragonlord.com
----- Original Message -----
From: "Victor Miasnikov"
To: [email protected]; "Brian Wong"
Cc: "Abhishek Gupta (LIS)" ( zzzzzzzzzzzzzzz (at) microsoft.com>; "KY Srinivasan" zzzzzzzzzzz (at) microsoft.com
Sent: Thursday, March 06, 2014 1:07 PM
Subject: Re: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled RE: [PATCH 2/2]
Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
Hi!
Short:
Question to Linux kernel team:
may be patch
>>> [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
can solve problems with dynamic memory hot add in Hyper-V VMs with Linux OS ?
Full:
BW>> .., if I set the startup memory to 512 MB, and enable dynamic
BW>> memory with a minimum of 512 MB and a maximum of 8192 MB,
BW>> the system will never allocate than 512 MB of physical memory
BW>>
BW>> Have I encountered a bug in the Hyper-V balloon driver?
BW>>
Unfortunately, It's long story . . . :-(
a)
I already ( on January 09, 2014 2:18 PM ) write about problems with "Online the hot-added memory" in "user space" see
P.P.S.
b)
See
Bug 979257 -[Hyper-V][RHEL6.5][RFE]in-kernel online support for memory hot-add
https://bugzilla.redhat.com/show_bug.cgi?id=979257
( Info from this topic may be interessant not only for RedHat users )
b2)
Detail about pathes related problem "Online the hot-added memory" in "user space" :
>>> [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
>>>
>>>
>>> === 0001-Drivers-base-memory-Export-functionality-for-in-kern.patch
>>> . . .
>>> +/*
>>> + * Given the start pfn of a memory block; bring the memory
>>> + * block online. This API would be useful for drivers that may
>>> + * want to bring "online" the memory that has been hot-added.
>>> + */
>>> +
>>> +int online_memory_block(unsigned long start_pfn) { struct mem_section
>>> +*cur_section; struct memory_block *cur_memory_block;
>>>
>>> . . .
>>> ===
>>>
>>>
>>> ==
>>> . . .
>>> == 0002-Drivers-hv-balloon-Online-the-hot-added-memory-in-co.patch
>>> . . .
>>> /*
>>> - * Wait for the memory block to be onlined.
>>> - * Since the hot add has succeeded, it is ok to
>>> - * proceed even if the pages in the hot added region
>>> - * have not been "onlined" within the allowed time.
>>> + * Before proceeding to hot add the next segment,
>>> + * online the segment that has been hot added.
>>> */
>>> - wait_for_completion_timeout(&dm_device.ol_waitevent, 5*HZ);
>>> + online_memory_block(start_pfn);
>>>
>>> }
c)
Before apply patches ( see in P.S. about native udev-script) we are need use one of this methods:
[ VVM: URL of this topic skipped ]
c1)
"/bin/cp method" by Nikolay Pushkarev :
Following udev rule works slightly faster for me (assuming that memory0 bank always in online state):
SUBSYSTEM=="memory", ACTION=="add", DEVPATH=="/devices/system/memory/memory[1-9]*",
RUN+="/bin/cp /sys$devpath/../memory0/state /sys$devpath/state"}}
( VVM : of course all need be place in one line, 2 line in this msg. -- only for good visual formating reasons )
c2)
udev rule using "putarg" by Nikolay Pushkarev :
Even "/bin/cp method" udev rule work time to time not as need :-(
As a result, Nikolay Pushkarev write a program putarg.c, that is even faster than "/bin/cp method" :
==
#include <stdio.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <string.h>
int main(int argc, char** argv) {
int i, fd;
if (argc < 2) return 0;
if ((fd = open(argv[1], O_RDWR)) < 0) return 1;
for (i = 2; i < argc; i++) {
if (write(fd, argv[i], strlen(argv[i])) < 0) {
close(fd);
return i;
close(fd);
return 0;
==
The first argument - the name of the output file ,
and argument number 2 ( and all subsequent ( if exist ) ) - are text that are wiil be written in output file.
Compile source code to executable file by run command:
gcc -o putarg -s putarg.c
The resulting binary need be placed an accessible location , such as /usr/bin, or wherever you want.
Now udev rule using "putarg" can be written as :
SUBSYSTEM=="memory", ACTION=="add", RUN+="/usr/bin/putarg /sys$devpath/state online"
This complex solutions ( compiled file and udev-rule ) works exceptionally fast.
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
P.S.
Nikolay Pushkarev about standart udev-script :
Strange, that the native udev-script
SUBSYSTEM=="memory", ACTION=="add", ATTR{state}="online"
triggered somehow through time ( VVM: very often not work as need )
P.P.S.
----- Original Message -----
From: "Victor Miasnikov"
To: "Dan Carpenter"; "K. Y. Srinivasan" ; <[email protected]>
Cc: "Greg KH" ; <[email protected]>; <olaf (at) aepfle.de>; ""Andy Whitcroft"" <zzzzzzzzzzzz (at)
canonical.com>;
<jasowang (at) redhat.com>
Sent: Thursday, January 09, 2014 2:18 PM
Subject: RE: [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
Implement the file copy service
Hi!
> Is there no way we could implement file copying in user space?
For "file copy service" "user space" may be pretty good
But I ( and other Hyper-V sysadmin) see non-Ok ( in "political correct" terminalogy) results with "hv: balloon: Online
the hot-added memory" in "user space"
==
[PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
==
What news? Roadmap?
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
~
~
~
~
~
~
~
~
~
----- Original Message -----
From: "KY Srinivasan"
To: "Victor Miasnikov"; [email protected]; "Brian Wong"
Cc: "Abhishek Gupta (LIS)"
Sent: Thursday, March 06, 2014 1:23 PM
Subject: RE: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled RE: [PATCH 2/2]
Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
> -----Original Message-----
> From: Victor Miasnikov
> Sent: Thursday, March 6, 2014 3:38 PM
> To: [email protected]; Brian Wong
> Cc: Abhishek Gupta (LIS); KY Srinivasan
> Subject: Re: Linux does not use more than the startup RAM under Hyper-V
> with dynamic memory enabled RE: [PATCH 2/2] Drivers: hv: balloon: Online
> the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
>
Victor,
I will try to get my in-context onlining patches accepted upstream.
K. Y
Similar Messages
-
Error message: ORA-27125: unable to create shared memory segment Linux-x86_
Hi,
I am doing an installtion of SAP Netweaver 2004s SR3 on SusE Linux 11/Oracle 10.2
But i am facing the follow issue in Create Database phase of SAPInst.
An error occurred while processing service SAP NetWeaver 7.0 Support Release 3 > SAP Systems > Oracle > Central System > Central System( Last error reported by the step :Caught ESAPinstException in Modulecall: ORA-27125: unable to create shared memory segment Linux-x86_64 Error: 1: Operation not permitted Disconnected
Please help me to resolve the issue.
Thanks,
NishithaHi Ratnajit,
I am too facing the same error but my ORACLE is not starting,
Here are my results of following command:
cat /etc/sysctl.conf
# created by /sapmnt/pss-linux/scripts/sysctl.pl on Wed Oct 23 22:55:01 CEST 2013
fs.inotify.max_user_watches = 65536
kernel.randomize_va_space = 0
##kernel.sem = 1250 256000 100 8192
kernel.sysrq = 1
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.neigh.default.gc_thresh1 = 256
net.ipv4.neigh.default.gc_thresh2 = 1024
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv6.neigh.default.gc_thresh1 = 256
net.ipv6.neigh.default.gc_thresh2 = 1024
net.ipv6.neigh.default.gc_thresh3 = 4096
vm.max_map_count = 2000000
# Modified for SAP on 2013-10-24 07:14:17 UTC
#kernel.shmall = 2097152
kernel.shmall = 16515072
# Modified for SAP on 2013-10-24 07:14:17 UTC
#kernel.shmmax = 2147483648
kernel.shmmax = 67645734912
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
And here is mine Limit.conf File
cat /etc/security/limits.conf
#<domain> <type> <item> <value>
#* soft core 0
#* hard rss 10000
#@student hard nproc 20
#@faculty soft nproc 20
#@faculty hard nproc 50
#ftp hard nproc 0
#@student - maxlogins 4
# Added for SAP on 2012-03-14 10:38:15 UTC
#@sapsys soft nofile 32800
#@sapsys hard nofile 32800
#@sdba soft nofile 32800
#@sdba hard nofile 32800
#@dba soft nofile 32800
#@dba hard nofile 32800
# End of file
# Added for SAP on 2013-10-24
# soft nproc 2047
# hard nproc 16384
# soft nofile 1024
# hard nofile 65536
@sapsys soft nofile 131072
@sapsys hard nofile 131072
@sdba soft nproc 131072
@sdba hard nproc 131072
@dba soft core unlimited
@dba hard core unlimited
soft memlock 50000000
hard memlock 50000000
Here is mine cat /proc/meminfo
MemTotal: 33015980 kB
MemFree: 29890028 kB
Buffers: 82588 kB
Cached: 1451480 kB
SwapCached: 0 kB
Active: 1920304 kB
Inactive: 749188 kB
Active(anon): 1136212 kB
Inactive(anon): 39128 kB
Active(file): 784092 kB
Inactive(file): 710060 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 33553404 kB
SwapFree: 33553404 kB
Dirty: 1888 kB
Writeback: 0 kB
AnonPages: 1135436 kB
Mapped: 161144 kB
Shmem: 39928 kB
Slab: 84096 kB
SReclaimable: 44400 kB
SUnreclaim: 39696 kB
KernelStack: 2840 kB
PageTables: 10544 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 50061392 kB
Committed_AS: 1364300 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 342156 kB
VmallocChunk: 34359386308 kB
HardwareCorrupted: 0 kB
AnonHugePages: 622592 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 67584 kB
DirectMap2M: 33486848 kB
Please let me know where i am going wrong.
Wat thing basically u check on /proc/meminfo command
Regards,
Dipak -
Oracle taking too much memory in Linux
Hi.. I am using Oralce on linux 7.5
No any active session is running on server for oracle ..
But oracle taking too much memory in linux when I check with TOP command ..
16:08:29 up 19 days, 20:14, 2 users, load average: 5.32, 4.63, 3.78
227 processes: 226 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 2.5% 0.0% 2.9% 0.4% 0.0% 91.8% 2.2%
cpu00 3.8% 0.0% 3.8% 0.0% 0.0% 92.2% 0.0%
cpu01 0.4% 0.0% 0.0% 0.0% 0.0% 99.5% 0.0%
cpu02 0.8% 0.0% 4.7% 1.7% 0.0% 85.7% 6.8%
cpu03 5.1% 0.0% 3.0% 0.0% 0.0% 89.6% 2.1%
Mem: 8203404k av, 8186192k used, 17212k free, 0k shrd, 7292k buff
6266516k actv, 1275016k in_d, 136704k in_c
Swap: 6289384k av, 10528k used, 6278856k free 7602904k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
6224 oracle 15 0 1007M 1.0G 998M D 0.0 12.5 30:45 1 oracle
10628 oracle 15 0 1000M 998M 998M S 0.0 12.4 0:16 0 oracle
6230 oracle 15 0 945M 945M 943M S 0.0 11.7 82:33 1 oracle
10188 oracle 15 0 944M 942M 931M S 0.0 11.7 3:02 1 oracle
19128 oracle 15 0 909M 901M 899M S 0.0 11.2 2:49 2 oracle
6864 oracle 15 0 886M 886M 877M S 0.0 11.0 6:12 3 oracle
18586 oracle 22 0 883M 882M 880M D 1.9 11.0 1:15 0 oracle
6870 oracle 15 0 433M 432M 431M S 0.0 5.4 0:21 2 oracle
6421 oracle 15 0 337M 336M 335M S 0.0 4.2 1:07 2 oracle
8478 oracle 15 0 272M 271M 269M S 0.0 3.3 0:12 1 oracle
18250 oracle 15 0 256M 255M 252M S 0.0 3.1 0:38 0 oracle
6876 oracle 15 0 251M 250M 249M S 0.0 3.1 1:03 0 oracle
17926 oracle 15 0 225M 224M 223M S 0.0 2.8 0:01 1 oracle
19320 oracle 15 0 197M 196M 195M S 0.0 2.4 0:23 1 oracle
18116 oracle 15 0 168M 167M 165M S 0.0 2.0 0:04 2 oracle
19596 oracle 15 0 159M 158M 155M D 2.2 1.9 0:17 0 oracle
6228 oracle 15 0 156M 155M 142M S 0.0 1.9 12:05 1 oracle
18902 oracle 15 0 81612 71M 48364 S 0.0 0.8 3:07 3 oracle
19286 oracle 15 0 48420 46M 46412 S 0.0 0.5 0:03 0 oracle
6222 oracle 15 0 41496 40M 39684 S 0.0 0.4 9:38 3 oracle
19602 oracle 15 0 40324 38M 38164 S 0.0 0.4 0:00 2 oracle
5962 mysql 25 0 39168 38M 88 S 0.0 0.4 0:00 1 mysqld
5963 mysql 15 0 39168 38M 88 S 0.0 0.4 0:08 0 mysqld
5964 mysql 20 0 39168 38M 88 S 0.0 0.4 0:00 0 mysqld
5965 mysql 25 0 39168 38M 88 S 0.0 0.4 0:00 0 mysqld
5966 mysql 25 0 39168 38M 88 S 0.0 0.4 0:00 0 mysqld
5967 mysql 20 0 39168 38M 88 S 0.0 0.4 0:00 0 mysqld
5969 mysql 15 0 39168 38M 88 S 0.0 0.4 2:16 2 mysqld
5970 mysql 15 0 39168 38M 88 S 0.0 0.4 2:31 1 mysqld
5971 mysql 24 0 39168 38M 88 S 0.0 0.4 0:00 0 mysqld
5972 mysql 20 0 39168 38M 88 S 0.0 0.4 0:00 0 mysqld
6868 oracle 15 0 27312 26M 15204 S 0.0 0.3 8:21 3 oracle
19554 oracle 15 0 20920 19M 18192 S 0.0 0.2 0:00 3 oracle
6874 oracle 15 0 16904 16M 15400 S 0.0 0.2 1:15 1 oracle
19444 oracle 15 0 13636 12M 12004 S 0.0 0.1 0:00 1 oracle
19442 oracle 15 0 12964 12M 11324 S 0.0 0.1 0:00 1 oracle
19474 oracle 15 0 12708 12M 11076 S 0.0 0.1 0:00 2 oracle
Why oracle taking too much memory?
Any solution for this bec of that my updation takes too much time .
Edited by: harshalpatil on Oct 13, 2008 4:11 PMHarsh,
1) What version of Oracle you running on 7.5 release of linux?
2)How did you come to the conclusion that because of this, your update is slow?
Cheers
Aman.... -
Dynamic Memory and Data Deduplication
Hi,
I'm currently testing W2K12 Deduplication feature on some servers, and I was wondering if I could enable "Dynamic Memory" on the VM in Hyper-V? Because the job only uses percentage of memory for the dedup job, I guess it's not possible.
Thanks in advance!I know it old thread, but I ran into this and could not find anything on it but here, so I am adding my experience.
I can concur with 3magroup...
In my environment Windows Server 2012 with Dynamic Memory and Deduplication enabled, the HyperV does not release the driver locked ram for the dedup optimization. What resulted was an insane amount of paging and a painfully slow VM. Increasing the memory
buffer for the dynamic memory did not cause the HyperV to release the memory for the dedup. Instead, it still "driver locked" nearly all of the ram, which did not make sense to me based on my understanding of the Memory Buffer setting.
I can verify the VM poor performance and dedup using the paging file with the Resource Monitor, disk section, and the Driver locked RAM with RamMap. -
Hi guys,
we are using three node cluster with Windows Server 2012R2. Our virtual machines are virtual
desktops with Windows 8.1 guest OS, 4 vCPU and 8 GB of dynamic memory.
Sometimes dynamic memory stops working, guest OS is not able to allocate more RAM and starts to swap. This swapping slows performance of all our virtual machines, since
all virtual machines are on same cluster shared volume. This happens at random RAM amount allocated, for example at 2.6 GB,
which is way below our 8 GB limit.
Live migration helps to solve this issue for migrated VM, after live migration the guest OS is able to allocate
more ram and starts to work properly, swapping disappears. However after some time, problem appears on other VM.
Our physical machines have 2x 10core Xeon and 394 GB of RAM. Only about 30% of RAM is used, so there is free space for all virtual machines. But we need to solve this issue so dynamic RAM starts to work properly.
Do you have similar experience?Hi manasj,
Please check event log "Microsoft-Windows-Hyper-V Worker/Admin " to see if there is any clue .
Also please refer to the following potetial cause :
" In a Hyper-V Failover Cluster, a virtual machine can have its configuration information stored on cluster shared storage (a Physical disk resource or a Cluster Shared Volume (CSV). If the physical disk resource or the Cluster Shared Volume (CSV) goes Offline or Fails,
the VM placed in a critical state. Once the storage is re-connected, the VM should no longer be in a critical state. However, virtual machine worker process (vmwp.exe) does not refresh all of its file handles."
For details please refer to following link :
http://support.microsoft.com/kb/2504962/en-us
Best Regards
Elton Ji
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place. -
Dynamic Memory in the production environment
Hi,
We configure all production VMs with static memory on Hyper-v 2012
we need to make sure there is no negative impact on VM performance if we use Dynamic memory , as we read for 2 years
not recommend to use dynamic memory in the production.
I preferred the answer with Trusted URL
RamyHiya,
The best answer you can get is, that it is dependent on the application running on the server O/S.
The reason that dynamic memory is not recommended for production environments, is that many applications are not supporting it. That is usually seen on memory intensive applications(SQL is an example) or simply because of caching types of functions. (SharePoint
is an example).
The major concern here is when VM's are decreasing the memory for a VM, the application does not understand this. Usually it is not a problem when increasing the memory. - Again it will depend on the application.
Think of it as hot swap memory and how you used to use that in the physical machine days.
Also when there is not adequate memory, operating system will use paging. Paging uses disks and disks has a lower access time than memory.
In general dynamic memory is easier to control than dynamically expanding disks, as you can set the buffer size of the memory, which is still not available for the dynamic disks.
Besides the above, the following links states:
"Workloads that are not NUMA-aware will not take advantage of virtual NUMA. However, the guest operating system may perform some NUMA optimization. Enabling Dynamic Memory (therefore presenting only a single virtual NUMA) should not cause performance
degradation"
http://technet.microsoft.com/en-us/library/dn282282.aspx -
Dynamic memory allocation failure
Dear reader,
We sometimes have a problem where our windows 2012 r2 RDS virtual servers, that reside on windows 2012r2 hyper-v hosts, loose their dynamic memory and only have their startup memory left to work with. Users start complaining that things are very slow etc.
If I check several screens (RDS Broker load balancing, hyper-v manager, cluster manager and the vm's task manager) it's clear that the vm only has its startup memory allocated. I'm not sure if this happens instantly or immidiatly after the nightly reboot.
To resolve the problem we have to call all users on the vm where it happens and ask them to logoff (if they are even able to), and then we reboot the machine.
I have checked the logs from the machine where the VM resides on and the logs from the vm itself. But I cannot find anything. We also have alot of windows 2008r2 vm's with dynamivc memory, but none of those have ever had this problem.
Searched the internet, but so far it seems we are only.
Can anyone give me a lead to troubleshoot this?
Best regards,
Ruud Boersma
MCITP Enterprise administratorHi all,
I'm going to be "one of those people" who revives dead posts for something that is relevant but obviously not fixed... sorry in advance!
We have the exact same situation, a bunch of RDSH guests with Dynamic memory turned on (60+ of them). every day between 1-8 of them will fail to allocate Dynamic memory and will be stuck on their startup RAM amount. This really hurts our users at peak
times.
I have engaged our TAM and have raised a case with PSS, Usual story "your the only one with this problem". Which obviously isn't true.
So, we have tons of free RAM on the hosts, 600GB+ on most of them, the issue affects RDSH hosts at random, across multiple hosts and clusters.
The screen shots attached are of one of our hosts from this morning. it has 8GB startup, 8GB minimum, 32GB Maximum RAM configured, with a 23% buffer. the host has 752GB RAM FREE. Notice how the perf counter "Hyper-V Dynamic Memory Integration Service"
is reporting "0", it should be reporting "32768". also under task manager on the VM we are missing "Maximum memory" which should be just below "Hardware reserved" in the bottom right hand corner.
Looks like the balloon driver is being delayed at boot time, we are going to XPerf all the servers in the hope that we can catch the critter. It's an unusual problem.
We only have Acrobat PDF viewer, word viewer, excel viewer and two custom dot.NET applications installed on the guests. Some of the servers are also just dumb RDSH hosts, with not connection broker configured, using an F5 loadbalancer for load distribution
and session management. All guests are 2012R2 patched up to March 2015, integration Services are installed and up to date (its an enlightened OS remember). -
Dynamic Memory is not working all the time
We are in the process off moving our 2008R2 VM's from the 2008R2 HyperV servers to new Server 2012R2 Hosts.
We shut down the VM's copy the files and VHD's to the new CSV's en import the VM in the Hyperv Manager. Then we make them high available in the Failover Cluster Manager (Configure role - Virtual machine). We mount the integration tools and update the
VM to version 6.3.9600.16384
For a specific type of VM (mostly RDS Host servers) we always had Dynamic Memory configured (when they were hosted on de 2008R2 platform), so we are using the same settings on the 2012r2 platform. The memory settings were;
Startup memory: 1024 MB
Minimum memory: 1024 MB
Maximum memory: 12288 MB
These VM's reboot every morning, this is done for specific reasons. But now once in a while (once per week/2 weeks) we notice that the VM's are not using more memory then 1024 MB while the demand is much higher. Rebooting the server helps most of the times,
live migrating to another host also helps. In the VM we see that memory usage in the taskmanager is 99-100%, and after the move it immediately starts using more than the minimum configured amount.
Until the failover the memory usage was 1024 MB and it did not get any higher.
This happened several times. Last week we changed the Memory configuration to:
Startup memory : 2048 MB
Minimum memory: 2048 MB
Maximum memory: 12288 MB
But this morning we had a call about the performance of one of the VM's, We saw that it was only using 2 GB memory while the demand was much higher. After live migrating it to another host it started using more memory immediately.
The 2012R2 hosts are not overcommited, there is a lot of memory still available for the VM's. Those VM's never had this problem on the 2008R2 Hyperv platform.
Any idea why this happens?
Peter CampsPeter,
I think this is a bug of some sort. I say that because the components that make up dynamic memory are as follows.
Memory Balancer(Host service, coordinates how memory changes are made.) This is also what shows the memory demand counter i believe.
Dynamic Memory Virtualization Service Provider (this is included your VMWP.exe proccess, one per VM. Essentially how it runs on the host. He listens to the Service Client for metrics)
Dynamic Memory Virtualization Service Client (this is inside the VM and reports to the Dynamic Memory Virtualizaton Service Provider.)
Since you live migrated the machine it made dynamic memory work on the other host. This means the Service Client is running in the client and shouldn't be an issue. The Memory Balancer is the server and shouldn't be the issue, so this means the "Dynamic
Memory Virtualization Service Provider" is in question. When you live migrate the machine its going to create a new VMWP.exe process on the clustered server. So now the question is it the host that couldn't listen to the service or the worker process
skipped a beat and has a bug.
Out of curiosity does it happen to both hosts? Also have you profiled the servers to see how much memory they really require on start-up? When you reboot the RDS servers, how many VM's do you reboot and is it a staggered process? -
Dynamic memory not released to host
Dear Techies,
Ours is a small Hyper V virtual server infrastructure with three DELL power-edge physical hosts(Windows server 2012 Datacenter) and around 15 virtual machines running on top of it. The hosts are added to fail-over cluster. Each host has 95 GB RAM. All the
VMs are running Windows server 2012 standard edition.
We have installed terminal services(TS licensing, TS connection broker, TS session host) in four VMs with the following dynamic memory settings:
Start-up RAM : 2048 MB
Minimum RAM : 2048 MB
Maximum RAM : 8192 MB
Below mentioned applications are configured in the server:
Nova Application Installed
SQL Developer Tool is Configured (ODBC Connection established for Database communication)
FTPs communication allowed for File Transfer
McAfee Agent is configured (Virus Scanner)
Nimsoft Robot Agent Configured – Monitoring
Terminal Service
Enabled Multiple terminal sessions based on customer requirement
BGinfo tool configured through group policy for customized desktop background
The average memory utilization in the terminal servers are 3.6 GB. As per dynamic allocation the maximum RAM requirement/allocation till date is 4GB. As seen in Hyper V console, the current RAM demand is 2300 MB and assigned memory is 2800 MB.
However, the earlier assigned RAM in the server is ballooned/faked to the VM as driver locked memory. This is by design. Despite the memory being released back to the host, the server still shows up the 4Gb which makes the memory utilization report from
monitoring tools look 80% (3.2 GB out of 4 GB).
As a result, the memory utilization report is always based on the current dynamically allocated RAM and not calculated based on the maximum assigned RAM(8GB in this case). To make it clear: If the
currently assigned RAM is 4Gb and utilization is 3.2 GB the utilization % is reported as 80%. However, if calculated in accordance with maximum RAM capacity of the server it would be 40% ((3.2/8)*100).
Is there any way to release the driver locked memory from the VM.?
Regards,
Auditya NI am not really clear on the point of your question.
Allocated RAM is what is currently in use / allocated to a VM out of the physical RAM pool. It is Demand + Buffer. The demand is based on the applications in the VM and what they think they need and how efficiently they return unused memory
to the machine. This has nothing to do with in-application paging (which happens a lot with Terminal Servers).
So yes, the memory utilization is accurate in relation to physical RAM utilization.
Dynamic Memory is about efficiency, not about over-allocation. Hyper-V can never give VMs more RAM than is in the physical RAM pool. The VMs can be configured to have more RAM than is in the physical RAM pool - but the VMs will hit the top of
the pool and not be allowed to have any more. There is no ballooning or paging to disk.
So, you maximum allocated could go beyond what is possible. But that would mean that your utilization would be artificially low if this was used in the calculation.
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat. -
I'm confused about dynamic memory features. I have N73me with 2gb sd card. Does it add it's dynamic memory or only more sotage space for files (mp3, jpg, applications etc) ?
N97 seems to be loaded with lots of memory. Comparing it's memory features to what N73 has, what's new/different ?Symbian/S60 phones do not use storage memory as an extension to available physical DRAM (run-time memory).
See also this thread:
/discussions/board/message?board.id=smartphones&message.id=14962 -
Guest keeps on consuming all the dynamic memory
Software:
Host: Windows Server 2012 R2 x64 fully patched
Guest: Windows 8 Update 1 x64 (Fully Patched).
I've been experience this problem for a few months and really don't know where to go from here. After starting the specific guest that is having the issue it will happen anywhere between 3-7 days. The guest will consume almost all the memory. The only way
i can log into the guest is with hyper-v manager. Remote Desktop will hang. Then when i log into the guest i can't bring up task manager or many things. Every time i have to kill the vm and restart it. Last time i thought to keep process monitor up and
running so i could inspect things. Well it happened today again. According the hyper-v manager the guest was consuming almost 24GB of memory:
here is the memory demand:
What's wierd is when i logged into the vm it was showing it only had 4GB of memory with 3.1GB used. There was hardly anything running other than a few services and they weren't using much memory. Quickly process explorer locked up on me. After logging in
for say 5-10 minutes this always happens too. After trying to diagnose the for a few minutes it will appear i have rebooted the system because i will get the following:
but i definitly did not reboot. If i come back in 24 hours it will still be at the same spot. Again i have to kill the vm but the same vicious cycle will ensue. I'm at a loss how to diagnose this. I've tried to scour the event viewer but it hasn't helped.
Sure i could just turn dynamic memory off or lower the upper threshold of the dynamic memory but i shouldn't have to do that. Thats really just masking the problem. The only default value of dynamic memory i changed was the "Startup Memory". I really
don't want to rebuild the vm but what other options do i have here?Well i updated one of my main programs on that machine (the on that i would think used "Most" of the memory) and i "thought" it went away but nope i was wrong. It came back. Luckily i left rammap & process explorer running. I could
it when it was consuming almost 6GB rather than the 20+ GB like before. Here is a screenshot of rammap:
As you can see "Paged Pool" memory is huge. I closed almost every app running on the system hoping that would help but it really didn't. The only thing that is running at this point outside of the normal windows processes is EMET 5 & Nortan
Antivirus. I tried to stop halt "Auto Protect" & the firewall but that didn't help any.
unfortunately, though i did catch it early, most programs will fail to load. Even file explorer fails to load. As usual after interacting with process explorer for awhile i get the white screen of death out of it. Hmmmm figure out why so much pool memory
is being consumed. -
Windows Server 2012 HyperV Cluster - Dynamic Memory Config
Hello There,
I am about to configure two node HyperV Cluster on Windows 2012 R2.
There is 192 GB RAM on each node but my VMs requirement is around 216 GB but some VMs doesn't require the full memory capacity all the time so in this scenario what is the best method to use dynamic memory utilization.
RAM
48 GB
48 GB
16 GB
48 GB
48 GB
8 GB
I need to provision 6 VMs with the above memory config but want to utilize it effectively through dynamic memory, please suggest.
I don't need the full RAM capacity all the time as in the table.
Thanks,
maqsood
Maqsood Mohammed Senior Systems Engineer MCITP-Enterprise Admin & ITILv3 Foundation CertifiedOf course you can, this is the aim of the dynamic memory feature.
In the VM's memory configuration, you will find three parameters:
Start up RAM : This is the memory allocated to the VM when it's started
Maximum RAM : This is the maximum amount of memory that a VM can request from the Hypervisor. If there is available memory the host will grant it more memory, if not, the VM will not receive memory
Minimum RAM : Because the memory configuration is dynamic, when a VM will not use memory, it will give back its memory resources to the host, but it will never go down its Minimum RAM value (The total minimum RAM of all VMs can never exceed the available
memory, otherwise the VMs will not even start)
And there is the Priority option that tell the hypervisor which VM to satisfy first.
Look here for more explanation
Regards, Samir Farhat Infrastructure and Virtualization Consultant || Virtualization, Cloud, Azure ? Follow and Ask here https://buildwindows.wordpress.com -
Queue of arrays without dynamic memory allocation
Hey folks,
I'm working on optimizing a timing critical VI. This VI is the
producer in a producer consumer architecture. I'm trying to populate
a queue from file in a manner that is as efficient as possible. My
current plan of attack is:
- read block of data from file and populate array (pre-allocated).
- add array (always of the same size) to Queue with a max size defined
(e.g. 50 elements)
- This is in a while loop as is the standard producer consumer model.
To improve the performance I would like to ensure that there is no
dynamic memory allocation on the Queue's behalf. This is easily done,
from what I understand, if the data type in the queue is of the same
type (e.g. double, int). However, since the size of an array can vary
does this mean that any queue of arrays will always dynamically
allocate memory? Is there a way to ensure that the queue will always
use the same memory as in a circular queue?
Thanks,
SteveDuplicate.
Try to take over the world! -
Request additional allocation of dynamic memory
Hyper-V 2012 host
Server 2012 R2 guest
The guest is configured for startup and minimum memory of 2 GB with a maximum memory of 64 GB. Once per day a PowerShell script runs which uses 7-Zip.exe to compress some large files. The 7-Zip command fails if there is less than 10 GB of memory
immediately available. I do not want to allocate that much minimum memory to the guest, as it doesn't need it for anything but this one script.
How can I programmatically request or instruct the host to allocate 10 GB to the guest before I run the 7zip command?
Currently I am launching a PowerShell job running Sysinternals TestLimit64.exe and running it until I see the allocated memory go up on the host. But this workaround is inelegant, inexact, imprecise, and slow.
Thank you.
Tim Curwick
MadWithPowerShell.comHi Tim,
Please refer to the following link which are related to scripting dynamic memory:
Scripting dynamic memory, part 1: reading the configuration<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/07/scripting-dynamic-memory-part-1-reading-the-configuration.aspx<o:p></o:p>
Scripting dynamic memory, part 2: displaying current usage information<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/08/scripting-dynamic-memory-part-2-displaying-current-usage-information.aspx<o:p></o:p>
Scripting dynamic memory, part 3: looking at performance counters<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/13/scripting-dynamic-memory-part-3-looking-at-performance-counters.aspx<o:p></o:p>
Scripting dynamic memory, part 4: configuring memory<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/14/scripting-dynamic-memory-part-4-configuring-memory.aspx<o:p></o:p>
Scripting dynamic memory, part 5: changing minimum memory
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/15/scripting-dynamic-memory-part-5-changing-minimum-memory.aspx
Best Regards,
Vincent Wu
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. -
Exporting snapshotted VMs and dynamic memory
We have a production VM which uses dynamic memory. Unfortunately, someone decided to snapshot this machine and now we cannot export the VM using the hyper-v "Export" facility.
Export gives a warning message indicating that VMs and snapshots cannot be imported into Server 2008R2 and that dynamic memory must be turned off prior to exporting. Here's the message:
I can turn off dynamic memory in the main VM but each snapshot has its own setting for dynamic memory. I can see the setting, dynamic memory is on but it is grayed out.
As a backup, I want to export the VM prior to merging the snapshots. How can I do this?
Thanks,
Alan
[email protected]
Alan MacKenzie [email protected]Yes, you cannot import to Server 2008 R2, you can only import to Server 2008 R2 SP1 or newer.
You cannot go backward in the version of the hypervisor, that is what it is warning about. It is not a block, it is just information. You can still Export and Import (if necessary) to a 2008 R2 SP1 or 2012 machine.
In regards to the snapshot. That is a different issue. Simply delete the saved state portion (the running memory part) of the snapshot.
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat.
Disclaimer: Attempting change is of your own free will.
Maybe you are looking for
-
Sqlpackage.exe Azure export error started few days ago
I was using this for a couple of months and suddenly since few days, I can't do it anymore sqlpackage.exe /a:Export /ssn:xxxxxxx.database.windows.net /sdn:"xxx" /su:"xxx" /sp:"xxxxxxxxxxxxxx" /tf:"xxxxxx.bacpac" Connecting to database 'xxxxx' on serv
-
10.2.0.3 Version question
After updating a 10.2.0.1 client installation to 10.2.0.3 I noticed that the version number format had changed. Before the update the versions were in the form 10.2.0.1. Afterwards the newest version is in the form 1.102.3.0. This looks like a mistak
-
Senior LabVIEW Controls Engineer--GOLDEN, CO
JOB DESCRIPTION Vehicle Projects Inc is seeking a full-time, senior controls engineer for the development and demonstration of high-power hydrogen-fuelcell rail vehicles. For background on our rail-vehicle program, please google the keywords: (“fuel
-
Multi-Language supported by CUPC & presence
Hi, Devices:- Presence Server 7.0 CUPC 7.0 Most of the user are using English as language in CUPC, few user want to use Japanese language for CUPC. Is it possible to use different languages for different user in above mentioned setup. Regards, Abhas
-
Yosemite made my Macbook air 2014 extremely slow!! Help?
After I updated to Yosemite, everything has been extremely slow and lagging!! I am not good with computers and I don't have a back up drive or stuff like that .. Is there any way to fix this??