Request additional allocation of dynamic memory

Hyper-V 2012 host
Server 2012 R2 guest
The guest is configured for startup and minimum memory of 2 GB with a maximum memory of 64 GB. Once per day a PowerShell script runs which uses 7-Zip.exe to compress some large files. The 7-Zip command fails if there is less than 10 GB of memory
immediately available. I do not want to allocate that much minimum memory to the guest, as it doesn't need it for anything but this one script.
How can I programmatically request or instruct the host to allocate 10 GB to the guest before I run the 7zip command?
Currently I am launching a PowerShell job running Sysinternals TestLimit64.exe and running it until I see the allocated memory go up on the host. But this workaround is inelegant, inexact, imprecise, and slow.
Thank you.
Tim Curwick
MadWithPowerShell.com

Hi Tim,
Please refer to the following link which are related to scripting dynamic memory:
Scripting dynamic memory, part 1: reading the configuration<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/07/scripting-dynamic-memory-part-1-reading-the-configuration.aspx<o:p></o:p>
Scripting dynamic memory, part 2: displaying current usage information<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/08/scripting-dynamic-memory-part-2-displaying-current-usage-information.aspx<o:p></o:p>
Scripting dynamic memory, part 3: looking at performance counters<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/13/scripting-dynamic-memory-part-3-looking-at-performance-counters.aspx<o:p></o:p>
Scripting dynamic memory, part 4: configuring memory<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/14/scripting-dynamic-memory-part-4-configuring-memory.aspx<o:p></o:p>
Scripting dynamic memory, part 5: changing minimum memory
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/15/scripting-dynamic-memory-part-5-changing-minimum-memory.aspx
Best Regards,
Vincent Wu
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

Similar Messages

Templates and Dynamic Memory Allocation Templates

Hi , I was reading a detailed article about templates and I came across the following paragraph
template<class T, size_t N>
class Stack
T data[N]; // Fixed capacity is N
size_t count;
public:
void push(const T& t);
};"You must provide a compile-time constant value for the parameter N when you request an instance of this template, such as *Stack<int, 100> myFixedStack;*
Because the value of N is known at compile time, the underlying array (data) can be placed on the run time stack instead of on the free store.
This can improve runtime performance by avoiding the overhead associated with dynamic memory allocation.
Now in the above paragraph what does
"This can improve runtime performance by avoiding the overhead associated with dynamic memory allocation." mean ?? What does template over head mean ??
I am a bit puzzled and i would really appreciate it if some one could explain to me what this sentence means thanks...

The run-time memory model of a C or C++ program consists of statically allocated data, automatically allocated data, and dynamically allocated data.
Data objects (e.g. variables) declared at namespace scope (which includes global scope) are statically allocated. Data objects local to a function that are declared static are also statically allocated. Static allocation means the storage for the data is available when the program is loaded, even before it begins to run. The data remains allocated until after the program exits.
Data objects local to a function that are not declared static are automatically allocated when the function starts to run. Example:
int foo() { int i; ... } Variable i does not exist until function foo begins to run, at which time space for it appears automatically. Each new invocation of foo gets its own location for i independent of other invocations of foo. Automatic allocation is usually referred to as stack allocation, since that is the usual implementation method: an area of storage that works like a stack, referenced by a dedicated machine register. Allocating the automatic data consists of adding (or subtracting) a value to the stack register. Popping the stack involves only subtracting (or adding) a value to the stack register. When the function exits, the stack is popped, releasing storage for all its automatic data.
Dynamically allocated storage is acquired by an explicit use of a new-expression, or a call to an allocation function like malloc(). Example:
int* ip = new int[100]; // allocate space for 100 integers
double* id = (double*)malloc(100*sizeof(double)); // allocate space for 100 doublesDynamic storage is not released until you release it explicitly via a delete-expression or a call to free(). Managing the "heap", the area from where dynamic storage is acquired, and to which it is released, can be quite time-consuming.
Your example of a Stack class (not to be confused with the program stack that is part of the C or C++ implementation) uses a fixed-size (that is, fixed at the point of template instance creation) automatically-allocated array to act as a stack data type. It has the advantage of taking zero time to allocate and release the space for the array. It has the disadvantages of any fixed-size array: it can waste space, or result in a program failure when you try to put N+1 objects into it, and it cannot be re-sized once created.

Dynamic memory allocation failure

Dear reader,
We sometimes have a problem where our windows 2012 r2 RDS virtual servers, that reside on windows 2012r2 hyper-v hosts, loose their dynamic memory and only have their startup memory left to work with. Users start complaining that things are very slow etc.
If I check several screens (RDS Broker load balancing, hyper-v manager, cluster manager and the vm's task manager) it's clear that the vm only has its startup memory allocated. I'm not sure if this happens instantly or immidiatly after the nightly reboot.
To resolve the problem we have to call all users on the vm where it happens and ask them to logoff (if they are even able to), and then we reboot the machine.
I have checked the logs from the machine where the VM resides on and the logs from the vm itself. But I cannot find anything. We also have alot of windows 2008r2 vm's with dynamivc memory, but none of those have ever had this problem.
Searched the internet, but so far it seems we are only.
Can anyone give me a lead to troubleshoot this?
Best regards,
Ruud Boersma
MCITP Enterprise administrator

Hi all,
I'm going to be "one of those people" who revives dead posts for something that is relevant but obviously not fixed... sorry in advance!
We have the exact same situation, a bunch of RDSH guests with Dynamic memory turned on (60+ of them). every day between 1-8 of them will fail to allocate Dynamic memory and will be stuck on their startup RAM amount. This really hurts our users at peak
times.
I have engaged our TAM and have raised a case with PSS, Usual story "your the only one with this problem". Which obviously isn't true.
So, we have tons of free RAM on the hosts, 600GB+ on most of them, the issue affects RDSH hosts at random, across multiple hosts and clusters.
The screen shots attached are of one of our hosts from this morning. it has 8GB startup, 8GB minimum, 32GB Maximum RAM configured, with a 23% buffer. the host has 752GB RAM FREE. Notice how the perf counter "Hyper-V Dynamic Memory Integration Service"
is reporting "0", it should be reporting "32768". also under task manager on the VM we are missing "Maximum memory" which should be just below "Hardware reserved" in the bottom right hand corner.
Looks like the balloon driver is being delayed at boot time, we are going to XPerf all the servers in the hope that we can catch the critter. It's an unusual problem.
We only have Acrobat PDF viewer, word viewer, excel viewer and two custom dot.NET applications installed on the guests. Some of the servers are also just dumb RDSH hosts, with not connection broker configured, using an F5 loadbalancer for load distribution
and session management. All guests are 2012R2 patched up to March 2015, integration Services are installed and up to date (its an enlightened OS remember).

Queue of arrays without dynamic memory allocation

Hey folks,
I'm working on optimizing a timing critical VI. This VI is the
producer in a producer consumer architecture. I'm trying to populate
a queue from file in a manner that is as efficient as possible. My
current plan of attack is:
- read block of data from file and populate array (pre-allocated).
- add array (always of the same size) to Queue with a max size defined
(e.g. 50 elements)
- This is in a while loop as is the standard producer consumer model.
To improve the performance I would like to ensure that there is no
dynamic memory allocation on the Queue's behalf. This is easily done,
from what I understand, if the data type in the queue is of the same
type (e.g. double, int). However, since the size of an array can vary
does this mean that any queue of arrays will always dynamically
allocate memory? Is there a way to ensure that the queue will always
use the same memory as in a circular queue?
Thanks,
Steve

Duplicate.
Try to take over the world!

Crash the dynamic memory allocation

hi
i am new to java, my prof said to crash the java prog. write simple prog. which allocate dynamic memory in C++ and Java. push the limits to see how much allocation cause the programs to crash.
how do i implement this in java.. can any one help out here.
thanks in advances.

Write a program that allocates larger and larger objects until it crashes. Make sure it prints the sizes of the objects as it runs.

Degree in Dynamic memory allocation need help!

I'm a student in the University of Bucharest in Computer-Science in the senior year. I'm looking for some specs for my degree in Dynamic memory allocation. In particular I was looking for specs about how the JVM heap and garbage collector work. Can you please direct me to someone how can help me find the necessary specs?
Thank you.

[http://java.sun.com/javase/technologies/hotspot/]
~

Problem in dynamic memory allocation

Hi,
My name is Ravi Kumar. I'm working on a project to improve organizational performance which include visual studio for simulation. I'm using dynamic memory allocation to allocate space for the array that are used in the program. Now I have run-time error
which I can't understand where it is going wrong. Can someone please help me regrading this issue.
If anyone interested in helping please leave a comment with your email id so that I will share the whole project folder.
Thanks,
Ravi

Hi Ravi,
Don is right that this is the forum to discuss questions and feedback for Microsoft Office client.
Please post in MSDN forum of Visual Studio, where you can get more experienced responses:
https://social.msdn.microsoft.com/Forums/en-US/home?forum=visualstudiogeneral
The reason why we recommend posting appropriately is you will get the most qualified pool of respondents, and other partners who read the forums regularly can either share their knowledge or learn from your interaction with us. Thank you for your understanding.
Regards,
Ethan Hua
TechNet Community Support
It's recommended to download and install
Configuration Analyzer Tool (OffCAT), which is developed by Microsoft Support teams. Once the tool is installed, you can run it at any time to scan for hundreds of known issues in Office
programs.

Allocating dynamic memory

Hi,
my second question for today...;-)
Should I always get dynamic memory via Photoshop API or can I use new[] as well? I understand that photoshop uses own memory management, but I'm talking about relatively small amounts of memory, say some kB.
If there a rule?
Greetings Thomas

For small allocations use new/delete DO NOT use PIUNew to override. This will just give you headaches. For large allocations, image buffers, use the buffer suite. <[email protected]> wrote in message news:[email protected].. > Hi, > > my second question for today...;-) > > Should I always get dynamic memory via Photoshop API or can I use new[] as > well? I understand that photoshop uses own memory management, but I'm > talking about relatively small amounts of memory, say some kB. > If there a rule? > > Greetings Thomas

Windows Server 2012 HyperV Cluster - Dynamic Memory Config

Hello There,
I am about to configure two node HyperV Cluster on Windows 2012 R2.
There is 192 GB RAM on each node but my VMs requirement is around 216 GB but some VMs doesn't require the full memory capacity all the time so in this scenario what is the best method to use dynamic memory utilization.
RAM
48 GB
48 GB
16 GB
48 GB
48 GB
8 GB
I need to provision 6 VMs with the above memory config but want to utilize it effectively through dynamic memory, please suggest.
I don't need the full RAM capacity all the time as in the table.
Thanks,
maqsood
Maqsood Mohammed Senior Systems Engineer MCITP-Enterprise Admin & ITILv3 Foundation Certified

Of course you can, this is the aim of the dynamic memory feature.
In the VM's memory configuration, you will find three parameters:
Start up RAM : This is the memory allocated to the VM when it's started
Maximum RAM : This is the maximum amount of memory that a VM can request from the Hypervisor. If there is available memory the host will grant it more memory, if not, the VM will not receive memory
Minimum RAM : Because the memory configuration is dynamic, when a VM will not use memory, it will give back its memory resources to the host, but it will never go down its Minimum RAM value (The total minimum RAM of all VMs can never exceed the available
memory, otherwise the VMs will not even start)
And there is the Priority option that tell the hypervisor which VM to satisfy first.
Look here for more explanation
Regards, Samir Farhat Infrastructure and Virtualization Consultant || Virtualization, Cloud, Azure ? Follow and Ask here https://buildwindows.wordpress.com

Relationship between Dynamic Memory Heap and Heap Data Structure

This question is not strictly related to Java, but rather to programming in general, and I tend to get better answers from this community than any where else.
Somehow, my school and industry experience have somehow not given me the opportunity to explore and understand heaps (the data structure), so I'm investigating them now, and in particular, I've been looking at applications. I know they can be used for priority queues, heap sorts, and shortest path searches. However, I would have thought that, obviously, there must be some sort of relationship between the heap data structure, and the dynamic memory heap. Otherwise, I can think of no good reason why the dynamic memory heap would be named "heap". Surprisingly, after searching the web for 90 minutes or so, I've seen vague references, but nothing conclusive (trouble seems to be that it's hard to get Google to understand that I'm using the word "heap" in two different contexts, and similarly, it would not likely understand that web authors would use the word in two different contexts).
The Java Virtual Machine Spec is silent on the subject, as "The Java virtual machine assumes no particular type of automatic storage management system, and the storage management technique may be chosen according to the implementor's system requirements."
I've seen things like:
[of dynamic memory] "All the blocks of a particular size are kept in a sorted linked list or tree (I extrapolate that sorted tree could imply heap)"
[of dynamic memory] "The free and reserved areas of memory are maintained in a data structure similar to binary trees called a heap"
[of dynamic memory] "This is not related to the heap data structure"
[of dynamic memory] "Not to be confused with the data structure known as a "heap"
[of data structure] "Not to be confused with the dynamic memory pool, often known as TheHeap"
At this point, I've come to surmise that some (but not all) memory management algorithms use heaps to track which (pages? blocks? bytes?) of memory are used, and which are not. However, the point of a heap is to store data so that the max (or min) key is at the root of the heap. But we might want to allocate memory of different sizes at different times, so it wouldn't make sense to key on the amount of available memory in a particular region of the free store.
I must assume then that there would be a different heap maintained for each size of memory block that can be allocated, and the key must have something to do with the attractiveness of the particular memory block in the heap (perhaps the lowest address, resulting, hopefully, in growing the free store space less often, leaving more space for the stack to grow, or perhaps keyed based on the fragmentation, to hopefully result in less fragmentation, and therefore more efficient use of the memory space, or perhaps based on page boundaries, keeping as much data in the same page as possible, etc).
So at this point, I have a few questions I've been unable to resolve completely:
1. Am I correct that the heap was so named because (perhaps at one point in time), a heap is/was commonly used to track the available memory in the free store?
2. If so, would it be correct that there would be a heap per standard block size?
3. Also, at what level of granularity would a heap typically be used (memory page, memory blocks, individual words (4-bytes))?
4. What would be the most likely property one would use as a key. That is, what makes the root item on the heap ideal?
5. Would a industrial strength system like the jvm use a (perhaps modified or tuned) heap for this sort of task, or would this typically be too naive for an real world solution today?
Any insight would be awesome!
Thanks,
A.

jschell wrote:
I think you are not only mixing terms but domains.
For starters the OS allocs memory. Applications, regardless of language, request memory from the OS and use it in various ways.
There are many variations of the term "heap" like the following.
[http://en.wikipedia.org/wiki/Heap_(data_structure)]
[http://en.wikipedia.org/wiki/Dynamic_memory_allocation]
A java VM will request memory from the OS (from a 'heap') and use it in its application 'heap' (C/C++) and then create the Java 'heap'. There can be variations of that along the way that can and likely will include variations of how each heap is used, potentially code that creates its own heap, and potentially other allocators which use something which is not a heap.This last part, I find a bit confusing. By "use something which is not a heap", do you mean the heap data structure, or the dynamic memory pool meaning of heap? If the former, then you would be implying that it would be common for a heap data structure to be used to manage the heap dynamic memory pool. If the latter, what would this "something which is not a heap" be? The best definition of "heap" I've found simply states that it is a pool of memory that can be dynamically allocated. If there is some other way of allocating dynamic memory, then it would suggest that the previous definition of "heap" is incomplete.
>
So to terms.
1. Am I correct that the heap was so named because (perhaps at one point in time), a heap is/was commonly used to track the available memory in the free store?Which 'heap'? The VM one? It is probably named that because the implementors of the Sun VM were familar with how C++ and Smalltalk allocated memory.Okay, but that begs the question, was the heap in C++ and/or Smalltalk so named for the above queried reason?
>
2. If so, would it be correct that there would be a heap per standard block size?Not sure what you are referring to but probably a detail of the implementation. And since there are different levels the question doesn't mean much.
However OS allocations are always by block if that helps. After that it requires making the question much, much more specific.
3. Also, at what level of granularity would a heap typically be used (memory page, memory blocks, individual words (4-bytes))?Again not specific enough. A typical standard implementation of heap could not be at the word level. And it is unlikely, but not impossible, that variations would support word size allocations.
The VM heap might use word boundaries (but not size), where the application heap certainly does (word boundary.)My understanding of it is that the application would request blocks from the OS, and then something like malloc would manage the memory within the allocated blocks. malloc (or whatever equivalent Java uses) would have to keep track of the memory it has allocated somehow, and I would think it would have to do this at the word level, since it's most commonly going to allocate memory at the word level to be references to other objects, etc.
So I guess my question here would really be, if the dynamic memory heap is so named because there has been a memory management strategy that relied upon a heap data structure (which I've found no proof, but have found some suggestive literature), then would that probably have applied at the OS Page Fault level, tracking allocated blocks, or would that have applied at the malloc level, allocating individual words as necessary?
>
4. What would be the most likely property one would use as a key. That is, what makes the root item on the heap ideal?"Key" is not a term that will apply in this discussion.
You appear to be referring to strategies for effective allocation of memory such as allocations from different regions by size comparison.
It is possible that all levels might use such an allocator. General purpose applications do not sort allocations though (as per your one reference that mentions 'key'.) Sorry, I got the term "key" from an article I read regarding heaps, that indicates that a "key" is used to sort the elements, which I guess would be a more generalized way to make a heap than assuming a natural ordering on the elements in the heap. I'm not sure if the terminology is standard.
>
5. Would a industrial strength system like the jvm use a (perhaps modified or tuned) heap for this sort of task, or would this typically be too naive for an real world solution today?Again too indefinite. The Sun VM uses a rather complicated allocator, the model for which originated after years of proceeding research certainly in Smalltalk and in Lisp as well, both commercially and academically.
I am sure the default is rules driven either explicitly or implicitly as well. So it is self tuning.
There are command line options that allow you to change how it works as well.I guess perhaps I could attempt to clarify my initial question a bit.
There is a 1:1 correspondence between the runtime stack, and a stack data structure. That is, when you call a function, it pushes a stack frame onto the runtime stack. When you return from a function, it pops a stack frame from the runtime stack. This is almost certainly the reasons the runtime stack is named as it is.
The question is, is there or has there ever been a 1:1 correspondence between some aspect of the dynamic memory heap or how it is managed, and a heap data structure? If so, it would explain the name, but I'm a bit puzzled as to how a heap data structure would be of assistance in creating or managing the dynamic memory heap. If not, on the other hand, then does anybody know where the name "heap" came from, as it applies to the dynamic memory pool?
A.

Hyper-V, Dynamic Memory, Memory Demand more than Maximum memory

Hi,
I have found that for some VMs Demand Memory more that Maximum Memory set.
Where can I found any documentation about this?

A VM can 'demand' more memory than is possible'
But it will never actually be given more then the maximum.
In the VM the application is asking for more RAM from the virtual memory manager of the OS in the VM (not to be confused with any memory management that the hypervisor is doing).
When the VM hits its Maximum setting it is no different than a physical machine hitting its physical RAM cap - the OS begins paging to give the application the additional RAM it is asking for.
It simply means that you have a greedy application. SQL and the Exchange mailbox role have a long history of being very greedy applications (SQL made many changes to work in concert with dynamic memory) - taking all of the RAM they could get.
In your case, you have a startup of 32GB a minimum of 24GB and a Maximum of 36GB. And the application in the VM would 'like' to have 54GB. If you look in your VM OS you will probably find that it is paging to disk.
What is your allocation setting? The default of 20% Higher?
Personally - I see little value in using dynamic memory with such a small window of difference between the startup, minimum, and maximum. It is as if you are trying to micro manage the feature, instead of allowing the feature to work as designed.
Dynamic memory is designed to balance RAM between all of the VMs on a particular hypervisor at the same time. And to allow use of all the RAM the machine has, if there is a workload that can use it. If the VM wants 64GB, why not give it that.
Dynamic memory will not starve other VMs to give this VM more RAM (unless you give this VM a higher priority). But it will take RAM away from a VM (if absolutely necessary) to allow another VM to meet its requirements.
Any VM is always guaranteed its startup, at boot time. And dynamic memory will not starve it below it minimum - and if the VM is not using RAM it will take RAM away down to the minimum. And a VM will never be given more than the maximum, even
if the VM wants more.
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat.
Disclaimer: Attempting change is of your own free will.

Query is allocating too large memory

I’m building an Analysis in OBIEE against an ASO cube and am seeing the following error:
Query is allocating too large memory ( > 4GB) and cannot be executed. Query allocation exceeds allocation limits
The report we’re trying to build is intended to show information from eight dimensions. However, when I try to add just a few of the dimensions we get the “Query is allocating too large memory” error. Even if I filter down the information so that I only have 1 or 2 rows in the Analysis I get the error. It seems like there is something wrong that is causing our queries to become so bloated. We're using OBIEE 11.1.1.6.0.
Any help would be appreciated.

950121 wrote:
I’m building an Analysis in OBIEE against an ASO cube and am seeing the following error:
Query is allocating too large memory ( > 4GB) and cannot be executed. Query allocation exceeds allocation limits
The report we’re trying to build is intended to show information from eight dimensions. However, when I try to add just a few of the dimensions we get the “Query is allocating too large memory” error. Even if I filter down the information so that I only have 1 or 2 rows in the Analysis I get the error. It seems like there is something wrong that is causing our queries to become so bloated. We're using OBIEE 11.1.1.6.0.
Any help would be appreciated.Hi,
This sounds like a known Bug 13331507 : RFA - DEBUGGING 'QUERY IS ALLOCATING TOO LARGE MEMORY ( > 4GB)' FROM ESSBASE.
Cause:
A filter has been added in several lines in the 'Data Filters' Tab of the 'Users Permissions' Screen in the Administration Tool (click on Manage and then Identity menu items). This caused the MDX Filter statement to be added several times to the MDX issues to the underlying Database, which in turn caused too much memory to be used in processing the request.
Refer to Doc ID: 1389873.1 for more information on My Oracle Support.

Query is allocating too large memory Error ( 4GB) in Essbase 11.1.2

Hi All,
Currently we are preparing dashboards in OBIEE from the Hyperion Essbase ASO (11.1.2) Cubes.When are trying to retrieve data with more attributes we are facing the below error
"Odbc driver returned an error (SQLExecDirectW).
Error Details
Error Codes: OPR4ONWY:U9IM8TAC:OI2DL65P
State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred. [nQSError: 43113] Message returned from OBIS. [nQSError: 96002] Essbase Error: Internal error: Query is allocating too large memory ( > 4GB) and cannot be executed. Query allocation exceeds allocation limits. (HY000)"
Currently we have data file size less than 2GB so we are using "Pending Cache Size=64MB".
Please let me know which memory I have to increase to resolve this issue
Thanks,
SatyaB

Hi,
Do you have any dynamic hierarchies? What is the size of the data set?
Thanks,
Nathan

Dynamic Memory on Linux VM

Hello!
Hyper-V 3.0 is great! After it will be released, I think it will become the most popular hypervisor. But it remains a major drawback.
Nowhere announced support for dynamic memory for Linux VM on Hyper-V.
Planned at least in some perspective to implement this functionality?
Now we have to use two different hypervisors, as Hyper-V does not meet all the requirements of our customers.
Mark Tepterev
Oversun

~
~
Moved P.P.P.S.
Q from Brian Wong:
----- Original Message -----
From: "Brian Wong"
To: <[email protected]>
Sent: Thursday, March 06, 2014 9:24 AM
Subject: Re: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled
On 3/6/2014 1:20 AM, Brian Wong wrote:
> The kernel is built with the full set of Hyper-V drivers, including the
> key "Microsoft Hyper-V Balloon Driver" as well as memory hot-add and
> hot-remove functionality. This is happening with both the Gentoo-patched
> 3.10.32 kernel and the vanilla 3.12.5 kernel. The host machine has a
> total of 24 GB of memory.
>
> For now, I am working around the issue by starting the VM with the
> startup memory set to the maximum and letting Hyper-V take the usused
> memory back when it is not in use. The VM will then get the extra memory
> when it needs it.
>
> Have I encountered a bug in the Hyper-V balloon driver?
>
Just a correction: the vanilla kernel version is 3.13.5, not 3.12.5.
Sorry for any confusion.
Brian Wong
http://www.fierydragonlord.com
----- Original Message -----
From: "Brian Wong"
To: <[email protected]>
Sent: Thursday, March 06, 2014 9:20 AM
Subject: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled
I'm new to LKML, so please don't be too hard on me :)
I'm running Gentoo Linux under Microsoft Client Hyper-V on Windows 8.1
Pro, and I've noticed some odd behavior with respect to dynamic memory
(aka memory ballooning). The system will never use more than the startup
memory defined in the virtual machine's settings.
( VVM: typewriting error viRtual fixed by me, for best search in future )
For example, if I set the startup memory to 512 MB, and enable dynamic
memory with a minimum of 512 MB and a maximum of 8192 MB, the system
will never allocate than 512 MB of physical memory, despite Hyper-V
assigning more memory to the VM and the added memory being visible in
the output of "free" and "htop". Attempting to use more memory causes
the system to start paging to swap, rather than actually allocating the
memory above the startup memory assigned to the VM.
The kernel is built with the full set of Hyper-V drivers, including the
key "Microsoft Hyper-V Balloon Driver" as well as memory hot-add and
hot-remove functionality. This is happening with both the Gentoo-patched
3.10.32 kernel and the vanilla 3.12.5 kernel. The host machine has a
total of 24 GB of memory.
Brian Wong wrote On 3/6/2014 1:20 AM:
Just a correction: the vanilla kernel version is 3.13.5, not 3.12.5. )
For now, I am working around the issue by starting the VM with the
startup memory set to the maximum and letting Hyper-V take the usused
memory back when it is not in use. The VM will then get the extra memory
when it needs it.
Have I encountered a bug in the Hyper-V balloon driver?
Brian Wong
http://www.fierydragonlord.com
----- Original Message -----
From: "Victor Miasnikov"
To: [email protected]; "Brian Wong"
Cc: "Abhishek Gupta (LIS)" ( zzzzzzzzzzzzzzz (at) microsoft.com>; "KY Srinivasan" zzzzzzzzzzz (at) microsoft.com
Sent: Thursday, March 06, 2014 1:07 PM
Subject: Re: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled RE: [PATCH 2/2]
Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
Hi!
Short:
Question to Linux kernel team:
may be patch
>>> [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
can solve problems with dynamic memory hot add in Hyper-V VMs with Linux OS ?
Full:
BW>> .., if I set the startup memory to 512 MB, and enable dynamic
BW>> memory with a minimum of 512 MB and a maximum of 8192 MB,
BW>> the system will never allocate than 512 MB of physical memory
BW>>
BW>> Have I encountered a bug in the Hyper-V balloon driver?
BW>>
Unfortunately, It's long story . . . :-(
a)
I already ( on January 09, 2014 2:18 PM ) write about problems with "Online the hot-added memory" in "user space" see
P.P.S.
b)
See
Bug 979257 -[Hyper-V][RHEL6.5][RFE]in-kernel online support for memory hot-add
https://bugzilla.redhat.com/show_bug.cgi?id=979257
( Info from this topic may be interessant not only for RedHat users )
b2)
Detail about pathes related problem "Online the hot-added memory" in "user space" :
>>> [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
>>>
>>>
>>> === 0001-Drivers-base-memory-Export-functionality-for-in-kern.patch
>>> . . .
>>> +/*
>>> + * Given the start pfn of a memory block; bring the memory
>>> + * block online. This API would be useful for drivers that may
>>> + * want to bring "online" the memory that has been hot-added.
>>> + */
>>> +
>>> +int online_memory_block(unsigned long start_pfn) { struct mem_section
>>> +*cur_section; struct memory_block *cur_memory_block;
>>>
>>> . . .
>>> ===
>>>
>>>
>>> ==
>>> . . .
>>> == 0002-Drivers-hv-balloon-Online-the-hot-added-memory-in-co.patch
>>> . . .
>>> /*
>>> - * Wait for the memory block to be onlined.
>>> - * Since the hot add has succeeded, it is ok to
>>> - * proceed even if the pages in the hot added region
>>> - * have not been "onlined" within the allowed time.
>>> + * Before proceeding to hot add the next segment,
>>> + * online the segment that has been hot added.
>>> */
>>> - wait_for_completion_timeout(&dm_device.ol_waitevent, 5*HZ);
>>> + online_memory_block(start_pfn);
>>>
>>> }
c)
Before apply patches ( see in P.S. about native udev-script) we are need use one of this methods:
[ VVM: URL of this topic skipped ]
c1)
"/bin/cp method" by Nikolay Pushkarev :
Following udev rule works slightly faster for me (assuming that memory0 bank always in online state):
SUBSYSTEM=="memory", ACTION=="add", DEVPATH=="/devices/system/memory/memory[1-9]*",
RUN+="/bin/cp /sys$devpath/../memory0/state /sys$devpath/state"}}
( VVM : of course all need be place in one line, 2 line in this msg. -- only for good visual formating reasons )
c2)
udev rule using "putarg" by Nikolay Pushkarev :
Even "/bin/cp method" udev rule work time to time not as need :-(
As a result, Nikolay Pushkarev write a program putarg.c, that is even faster than "/bin/cp method" :
==
#include <stdio.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <string.h>
int main(int argc, char** argv) {
int i, fd;
if (argc < 2) return 0;
if ((fd = open(argv[1], O_RDWR)) < 0) return 1;
for (i = 2; i < argc; i++) {
 if (write(fd, argv[i], strlen(argv[i])) < 0) {
 close(fd);
 return i;
close(fd);
return 0;
==
The first argument - the name of the output file ,
and argument number 2 ( and all subsequent ( if exist ) ) - are text that are wiil be written in output file.
Compile source code to executable file by run command:
gcc -o putarg -s putarg.c
The resulting binary need be placed an accessible location , such as /usr/bin, or wherever you want.
Now udev rule using "putarg" can be written as :
SUBSYSTEM=="memory", ACTION=="add", RUN+="/usr/bin/putarg /sys$devpath/state online"
This complex solutions ( compiled file and udev-rule ) works exceptionally fast.
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
P.S.
Nikolay Pushkarev about standart udev-script :
Strange, that the native udev-script
SUBSYSTEM=="memory", ACTION=="add", ATTR{state}="online"
triggered somehow through time ( VVM: very often not work as need )
P.P.S.
----- Original Message -----
From: "Victor Miasnikov"
To: "Dan Carpenter"; "K. Y. Srinivasan" ; <[email protected]>
Cc: "Greg KH" ; <[email protected]>; <olaf (at) aepfle.de>; ""Andy Whitcroft"" <zzzzzzzzzzzz (at)
canonical.com>;
<jasowang (at) redhat.com>
Sent: Thursday, January 09, 2014 2:18 PM
Subject: RE: [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
Implement the file copy service
Hi!
> Is there no way we could implement file copying in user space?
For "file copy service" "user space" may be pretty good
But I ( and other Hyper-V sysadmin) see non-Ok ( in "political correct" terminalogy) results with "hv: balloon: Online
the hot-added memory" in "user space"
==
[PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
==
What news? Roadmap?
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
~
~
~
~
~
~
~
~
~
----- Original Message -----
From: "KY Srinivasan"
To: "Victor Miasnikov"; [email protected]; "Brian Wong"
Cc: "Abhishek Gupta (LIS)"
Sent: Thursday, March 06, 2014 1:23 PM
Subject: RE: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled RE: [PATCH 2/2]
Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
> -----Original Message-----
> From: Victor Miasnikov
> Sent: Thursday, March 6, 2014 3:38 PM
> To: [email protected]; Brian Wong
> Cc: Abhishek Gupta (LIS); KY Srinivasan
> Subject: Re: Linux does not use more than the startup RAM under Hyper-V
> with dynamic memory enabled RE: [PATCH 2/2] Drivers: hv: balloon: Online
> the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
>
Victor,
I will try to get my in-context onlining patches accepted upstream.
K. Y

Dynamic memory freezes

Hi guys,
we are using three node cluster with Windows Server 2012R2. Our virtual machines are virtual
desktops with Windows 8.1 guest OS, 4 vCPU and 8 GB of dynamic memory.
Sometimes dynamic memory stops working, guest OS is not able to allocate more RAM and starts to swap. This swapping slows performance of all our virtual machines, since
all virtual machines are on same cluster shared volume. This happens at random RAM amount allocated, for example at 2.6 GB,
which is way below our 8 GB limit.
Live migration helps to solve this issue for migrated VM, after live migration the guest OS is able to allocate
more ram and starts to work properly, swapping disappears. However after some time, problem appears on other VM.
Our physical machines have 2x 10core Xeon and 394 GB of RAM. Only about 30% of RAM is used, so there is free space for all virtual machines. But we need to solve this issue so dynamic RAM starts to work properly.
Do you have similar experience?

Hi manasj,
Please check event log "Microsoft-Windows-Hyper-V Worker/Admin " to see if there is any clue .
Also please refer to the following potetial cause :
" In a Hyper-V Failover Cluster, a virtual machine can have its configuration information stored on cluster shared storage (a Physical disk resource or a Cluster Shared Volume (CSV). If the physical disk resource or the Cluster Shared Volume (CSV) goes Offline or Fails,
the VM placed in a critical state. Once the storage is re-connected, the VM should no longer be in a critical state. However, virtual machine worker process (vmwp.exe) does not refresh all of its file handles."
For details please refer to following link :
http://support.microsoft.com/kb/2504962/en-us
Best Regards
Elton Ji
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

Request additional allocation of dynamic memory

Similar Messages

Maybe you are looking for