Dynamic memory and GC.

Hi,
I have bad memory problems and I would like to
know what is the proper way to handle the following
scenario (The object that will be created will be called
Obj1and each time before using this object I need to
call his constructor ):
1.a. There is a global declaration in main class
Obj1 ObjectItem;
b. When I want to use ObjectItem I do:
ObjectItem = new Obj1();
c. When I finished working with ObjectItem object:
ObjectItem = null;
My question is:
1.1 Is the GC will perform the cleanup of ObjectItem
object ?
1.2 What will happened if before the GC activation
There will be another use in this object and we
call the constructor again ? will the old object
memory will be free ?
1.3 What will happened if I will not write null in
object ? Is every constructor call will increase
the memory usage even though we have only
one object?
1.4 Will there be any problems if the object is
declared in local procedure.
Thanks a lot,
Lior Cohen.

Hi,
I have bad memory problems and I would like to
know what is the proper way to handle the following
g
scenario (The object that will be created will be
e called
Obj1and each time before using this object I need to
o
call his constructor ):
1.a. There is a global declaration in main class
Obj1 ObjectItem;
b. When I want to use ObjectItem I do:
ObjectItem = new Obj1();
c. When I finished working with ObjectItem object:
ObjectItem = null;
My question is:
1.1 Is the GC will perform the cleanup of
up of ObjectItem
object ?ObjectItem is a variable (really a field) and not an object. However, any object that is no longer reachable will be available for garbage collection. So, after you set ObjectItem to null, that object is available for GC.
1.2 What will happened if before the GC
he GC activation
There will be another use in this object
this object and we
call the constructor again ? will the old
will the old object
memory will be free ?You are confusing variables and objects. variables are slots into which references to objects can be placed. Variables only hold the reference, not the storage associated with the object and its fields. If you create a new object it uses some bit of the heap that is distinct from any other live object. Exactly when a garbage collector will identify an object as unreachable and therefore collectable and when the storage will be releases is not controllable or knowable by you. It will occur before out of memory errors will be thrown so don't worry with it.
1.3 What will happened if I will not write null
null in
object ? Is every constructor call will
r call will increase
the memory usage even though we have only
e have only
one object?Again you are confusing different concepts. Ys, every time you consruct an object it will consume a new bit of the heap. Reuse of heap storage only occurs after garbage collection. But don't worry about that. The JVM only needs space for all live, reachable objects. Other will be GC'ed before the JVM refuses to honor your request to construct an object.
If you do not set the variable to null, the the object will remain referenced until the variable goes out of scope or the object that owns it (assuming it is a field) is no longer reachable. You need not set variable to null except for in unusual cases where large nets of objects would remain referenced and thus consume heap space for much longer than desired and that cause heap size problems. Most of the time scope is sufficiently small to handle it for you.
1.4 Will there be any problems if the object is
ect is
declared in local procedure.
Objects are not declared, they are constructed. Variables (fields, arguments, locals) are declared, but no constructed. You can construct an object anywhere you'd like to and associate it with what ever variable of any scope you can reach at the time of assignment.
Thanks a lot,
Lior Cohen.

Similar Messages

Dynamic Memory and Data Deduplication

Hi,
I'm currently testing W2K12 Deduplication feature on some servers, and I was wondering if I could enable "Dynamic Memory" on the VM in Hyper-V? Because the job only uses percentage of memory for the dedup job, I guess it's not possible.
Thanks in advance!

I know it old thread, but I ran into this and could not find anything on it but here, so I am adding my experience.
I can concur with 3magroup...
In my environment Windows Server 2012 with Dynamic Memory and Deduplication enabled, the HyperV does not release the driver locked ram for the dedup optimization. What resulted was an insane amount of paging and a painfully slow VM. Increasing the memory
buffer for the dynamic memory did not cause the HyperV to release the memory for the dedup. Instead, it still "driver locked" nearly all of the ram, which did not make sense to me based on my understanding of the Memory Buffer setting.
I can verify the VM poor performance and dedup using the paging file with the Resource Monitor, disk section, and the Driver locked RAM with RamMap.

Dynamic memory allocation failure

Dear reader,
We sometimes have a problem where our windows 2012 r2 RDS virtual servers, that reside on windows 2012r2 hyper-v hosts, loose their dynamic memory and only have their startup memory left to work with. Users start complaining that things are very slow etc.
If I check several screens (RDS Broker load balancing, hyper-v manager, cluster manager and the vm's task manager) it's clear that the vm only has its startup memory allocated. I'm not sure if this happens instantly or immidiatly after the nightly reboot.
To resolve the problem we have to call all users on the vm where it happens and ask them to logoff (if they are even able to), and then we reboot the machine.
I have checked the logs from the machine where the VM resides on and the logs from the vm itself. But I cannot find anything. We also have alot of windows 2008r2 vm's with dynamivc memory, but none of those have ever had this problem.
Searched the internet, but so far it seems we are only.
Can anyone give me a lead to troubleshoot this?
Best regards,
Ruud Boersma
MCITP Enterprise administrator

Hi all,
I'm going to be "one of those people" who revives dead posts for something that is relevant but obviously not fixed... sorry in advance!
We have the exact same situation, a bunch of RDSH guests with Dynamic memory turned on (60+ of them). every day between 1-8 of them will fail to allocate Dynamic memory and will be stuck on their startup RAM amount. This really hurts our users at peak
times.
I have engaged our TAM and have raised a case with PSS, Usual story "your the only one with this problem". Which obviously isn't true.
So, we have tons of free RAM on the hosts, 600GB+ on most of them, the issue affects RDSH hosts at random, across multiple hosts and clusters.
The screen shots attached are of one of our hosts from this morning. it has 8GB startup, 8GB minimum, 32GB Maximum RAM configured, with a 23% buffer. the host has 752GB RAM FREE. Notice how the perf counter "Hyper-V Dynamic Memory Integration Service"
is reporting "0", it should be reporting "32768". also under task manager on the VM we are missing "Maximum memory" which should be just below "Hardware reserved" in the bottom right hand corner.
Looks like the balloon driver is being delayed at boot time, we are going to XPerf all the servers in the hope that we can catch the critter. It's an unusual problem.
We only have Acrobat PDF viewer, word viewer, excel viewer and two custom dot.NET applications installed on the guests. Some of the servers are also just dumb RDSH hosts, with not connection broker configured, using an F5 loadbalancer for load distribution
and session management. All guests are 2012R2 patched up to March 2015, integration Services are installed and up to date (its an enlightened OS remember).

Exporting snapshotted VMs and dynamic memory

We have a production VM which uses dynamic memory. Unfortunately, someone decided to snapshot this machine and now we cannot export the VM using the hyper-v "Export" facility.
Export gives a warning message indicating that VMs and snapshots cannot be imported into Server 2008R2 and that dynamic memory must be turned off prior to exporting. Here's the message:
I can turn off dynamic memory in the main VM but each snapshot has its own setting for dynamic memory. I can see the setting, dynamic memory is on but it is grayed out.
As a backup, I want to export the VM prior to merging the snapshots. How can I do this?
Thanks,
Alan
[email protected]
Alan MacKenzie [email protected]

Yes, you cannot import to Server 2008 R2, you can only import to Server 2008 R2 SP1 or newer.
You cannot go backward in the version of the hypervisor, that is what it is warning about. It is not a block, it is just information. You can still Export and Import (if necessary) to a 2008 R2 SP1 or 2012 machine.
In regards to the snapshot. That is a different issue. Simply delete the saved state portion (the running memory part) of the snapshot.
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat.
Disclaimer: Attempting change is of your own free will.

Relationship between Dynamic Memory Heap and Heap Data Structure

This question is not strictly related to Java, but rather to programming in general, and I tend to get better answers from this community than any where else.
Somehow, my school and industry experience have somehow not given me the opportunity to explore and understand heaps (the data structure), so I'm investigating them now, and in particular, I've been looking at applications. I know they can be used for priority queues, heap sorts, and shortest path searches. However, I would have thought that, obviously, there must be some sort of relationship between the heap data structure, and the dynamic memory heap. Otherwise, I can think of no good reason why the dynamic memory heap would be named "heap". Surprisingly, after searching the web for 90 minutes or so, I've seen vague references, but nothing conclusive (trouble seems to be that it's hard to get Google to understand that I'm using the word "heap" in two different contexts, and similarly, it would not likely understand that web authors would use the word in two different contexts).
The Java Virtual Machine Spec is silent on the subject, as "The Java virtual machine assumes no particular type of automatic storage management system, and the storage management technique may be chosen according to the implementor's system requirements."
I've seen things like:
[of dynamic memory] "All the blocks of a particular size are kept in a sorted linked list or tree (I extrapolate that sorted tree could imply heap)"
[of dynamic memory] "The free and reserved areas of memory are maintained in a data structure similar to binary trees called a heap"
[of dynamic memory] "This is not related to the heap data structure"
[of dynamic memory] "Not to be confused with the data structure known as a "heap"
[of data structure] "Not to be confused with the dynamic memory pool, often known as TheHeap"
At this point, I've come to surmise that some (but not all) memory management algorithms use heaps to track which (pages? blocks? bytes?) of memory are used, and which are not. However, the point of a heap is to store data so that the max (or min) key is at the root of the heap. But we might want to allocate memory of different sizes at different times, so it wouldn't make sense to key on the amount of available memory in a particular region of the free store.
I must assume then that there would be a different heap maintained for each size of memory block that can be allocated, and the key must have something to do with the attractiveness of the particular memory block in the heap (perhaps the lowest address, resulting, hopefully, in growing the free store space less often, leaving more space for the stack to grow, or perhaps keyed based on the fragmentation, to hopefully result in less fragmentation, and therefore more efficient use of the memory space, or perhaps based on page boundaries, keeping as much data in the same page as possible, etc).
So at this point, I have a few questions I've been unable to resolve completely:
1. Am I correct that the heap was so named because (perhaps at one point in time), a heap is/was commonly used to track the available memory in the free store?
2. If so, would it be correct that there would be a heap per standard block size?
3. Also, at what level of granularity would a heap typically be used (memory page, memory blocks, individual words (4-bytes))?
4. What would be the most likely property one would use as a key. That is, what makes the root item on the heap ideal?
5. Would a industrial strength system like the jvm use a (perhaps modified or tuned) heap for this sort of task, or would this typically be too naive for an real world solution today?
Any insight would be awesome!
Thanks,
A.

jschell wrote:
I think you are not only mixing terms but domains.
For starters the OS allocs memory. Applications, regardless of language, request memory from the OS and use it in various ways.
There are many variations of the term "heap" like the following.
[http://en.wikipedia.org/wiki/Heap_(data_structure)]
[http://en.wikipedia.org/wiki/Dynamic_memory_allocation]
A java VM will request memory from the OS (from a 'heap') and use it in its application 'heap' (C/C++) and then create the Java 'heap'. There can be variations of that along the way that can and likely will include variations of how each heap is used, potentially code that creates its own heap, and potentially other allocators which use something which is not a heap.This last part, I find a bit confusing. By "use something which is not a heap", do you mean the heap data structure, or the dynamic memory pool meaning of heap? If the former, then you would be implying that it would be common for a heap data structure to be used to manage the heap dynamic memory pool. If the latter, what would this "something which is not a heap" be? The best definition of "heap" I've found simply states that it is a pool of memory that can be dynamically allocated. If there is some other way of allocating dynamic memory, then it would suggest that the previous definition of "heap" is incomplete.
>
So to terms.
1. Am I correct that the heap was so named because (perhaps at one point in time), a heap is/was commonly used to track the available memory in the free store?Which 'heap'? The VM one? It is probably named that because the implementors of the Sun VM were familar with how C++ and Smalltalk allocated memory.Okay, but that begs the question, was the heap in C++ and/or Smalltalk so named for the above queried reason?
>
2. If so, would it be correct that there would be a heap per standard block size?Not sure what you are referring to but probably a detail of the implementation. And since there are different levels the question doesn't mean much.
However OS allocations are always by block if that helps. After that it requires making the question much, much more specific.
3. Also, at what level of granularity would a heap typically be used (memory page, memory blocks, individual words (4-bytes))?Again not specific enough. A typical standard implementation of heap could not be at the word level. And it is unlikely, but not impossible, that variations would support word size allocations.
The VM heap might use word boundaries (but not size), where the application heap certainly does (word boundary.)My understanding of it is that the application would request blocks from the OS, and then something like malloc would manage the memory within the allocated blocks. malloc (or whatever equivalent Java uses) would have to keep track of the memory it has allocated somehow, and I would think it would have to do this at the word level, since it's most commonly going to allocate memory at the word level to be references to other objects, etc.
So I guess my question here would really be, if the dynamic memory heap is so named because there has been a memory management strategy that relied upon a heap data structure (which I've found no proof, but have found some suggestive literature), then would that probably have applied at the OS Page Fault level, tracking allocated blocks, or would that have applied at the malloc level, allocating individual words as necessary?
>
4. What would be the most likely property one would use as a key. That is, what makes the root item on the heap ideal?"Key" is not a term that will apply in this discussion.
You appear to be referring to strategies for effective allocation of memory such as allocations from different regions by size comparison.
It is possible that all levels might use such an allocator. General purpose applications do not sort allocations though (as per your one reference that mentions 'key'.) Sorry, I got the term "key" from an article I read regarding heaps, that indicates that a "key" is used to sort the elements, which I guess would be a more generalized way to make a heap than assuming a natural ordering on the elements in the heap. I'm not sure if the terminology is standard.
>
5. Would a industrial strength system like the jvm use a (perhaps modified or tuned) heap for this sort of task, or would this typically be too naive for an real world solution today?Again too indefinite. The Sun VM uses a rather complicated allocator, the model for which originated after years of proceeding research certainly in Smalltalk and in Lisp as well, both commercially and academically.
I am sure the default is rules driven either explicitly or implicitly as well. So it is self tuning.
There are command line options that allow you to change how it works as well.I guess perhaps I could attempt to clarify my initial question a bit.
There is a 1:1 correspondence between the runtime stack, and a stack data structure. That is, when you call a function, it pushes a stack frame onto the runtime stack. When you return from a function, it pops a stack frame from the runtime stack. This is almost certainly the reasons the runtime stack is named as it is.
The question is, is there or has there ever been a 1:1 correspondence between some aspect of the dynamic memory heap or how it is managed, and a heap data structure? If so, it would explain the name, but I'm a bit puzzled as to how a heap data structure would be of assistance in creating or managing the dynamic memory heap. If not, on the other hand, then does anybody know where the name "heap" came from, as it applies to the dynamic memory pool?
A.

Templates and Dynamic Memory Allocation Templates

Hi , I was reading a detailed article about templates and I came across the following paragraph
template<class T, size_t N>
class Stack
T data[N]; // Fixed capacity is N
size_t count;
public:
void push(const T& t);
};"You must provide a compile-time constant value for the parameter N when you request an instance of this template, such as *Stack<int, 100> myFixedStack;*
Because the value of N is known at compile time, the underlying array (data) can be placed on the run time stack instead of on the free store.
This can improve runtime performance by avoiding the overhead associated with dynamic memory allocation.
Now in the above paragraph what does
"This can improve runtime performance by avoiding the overhead associated with dynamic memory allocation." mean ?? What does template over head mean ??
I am a bit puzzled and i would really appreciate it if some one could explain to me what this sentence means thanks...

The run-time memory model of a C or C++ program consists of statically allocated data, automatically allocated data, and dynamically allocated data.
Data objects (e.g. variables) declared at namespace scope (which includes global scope) are statically allocated. Data objects local to a function that are declared static are also statically allocated. Static allocation means the storage for the data is available when the program is loaded, even before it begins to run. The data remains allocated until after the program exits.
Data objects local to a function that are not declared static are automatically allocated when the function starts to run. Example:
int foo() { int i; ... } Variable i does not exist until function foo begins to run, at which time space for it appears automatically. Each new invocation of foo gets its own location for i independent of other invocations of foo. Automatic allocation is usually referred to as stack allocation, since that is the usual implementation method: an area of storage that works like a stack, referenced by a dedicated machine register. Allocating the automatic data consists of adding (or subtracting) a value to the stack register. Popping the stack involves only subtracting (or adding) a value to the stack register. When the function exits, the stack is popped, releasing storage for all its automatic data.
Dynamically allocated storage is acquired by an explicit use of a new-expression, or a call to an allocation function like malloc(). Example:
int* ip = new int[100]; // allocate space for 100 integers
double* id = (double*)malloc(100*sizeof(double)); // allocate space for 100 doublesDynamic storage is not released until you release it explicitly via a delete-expression or a call to free(). Managing the "heap", the area from where dynamic storage is acquired, and to which it is released, can be quite time-consuming.
Your example of a Stack class (not to be confused with the program stack that is part of the C or C++ implementation) uses a fixed-size (that is, fixed at the point of template instance creation) automatically-allocated array to act as a stack data type. It has the advantage of taking zero time to allocate and release the space for the array. It has the disadvantages of any fixed-size array: it can waste space, or result in a program failure when you try to put N+1 objects into it, and it cannot be re-sized once created.

Dynamic Memory on Linux VM

Hello!
Hyper-V 3.0 is great! After it will be released, I think it will become the most popular hypervisor. But it remains a major drawback.
Nowhere announced support for dynamic memory for Linux VM on Hyper-V.
Planned at least in some perspective to implement this functionality?
Now we have to use two different hypervisors, as Hyper-V does not meet all the requirements of our customers.
Mark Tepterev
Oversun

~
~
Moved P.P.P.S.
Q from Brian Wong:
----- Original Message -----
From: "Brian Wong"
To: <[email protected]>
Sent: Thursday, March 06, 2014 9:24 AM
Subject: Re: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled
On 3/6/2014 1:20 AM, Brian Wong wrote:
> The kernel is built with the full set of Hyper-V drivers, including the
> key "Microsoft Hyper-V Balloon Driver" as well as memory hot-add and
> hot-remove functionality. This is happening with both the Gentoo-patched
> 3.10.32 kernel and the vanilla 3.12.5 kernel. The host machine has a
> total of 24 GB of memory.
>
> For now, I am working around the issue by starting the VM with the
> startup memory set to the maximum and letting Hyper-V take the usused
> memory back when it is not in use. The VM will then get the extra memory
> when it needs it.
>
> Have I encountered a bug in the Hyper-V balloon driver?
>
Just a correction: the vanilla kernel version is 3.13.5, not 3.12.5.
Sorry for any confusion.
Brian Wong
http://www.fierydragonlord.com
----- Original Message -----
From: "Brian Wong"
To: <[email protected]>
Sent: Thursday, March 06, 2014 9:20 AM
Subject: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled
I'm new to LKML, so please don't be too hard on me :)
I'm running Gentoo Linux under Microsoft Client Hyper-V on Windows 8.1
Pro, and I've noticed some odd behavior with respect to dynamic memory
(aka memory ballooning). The system will never use more than the startup
memory defined in the virtual machine's settings.
( VVM: typewriting error viRtual fixed by me, for best search in future )
For example, if I set the startup memory to 512 MB, and enable dynamic
memory with a minimum of 512 MB and a maximum of 8192 MB, the system
will never allocate than 512 MB of physical memory, despite Hyper-V
assigning more memory to the VM and the added memory being visible in
the output of "free" and "htop". Attempting to use more memory causes
the system to start paging to swap, rather than actually allocating the
memory above the startup memory assigned to the VM.
The kernel is built with the full set of Hyper-V drivers, including the
key "Microsoft Hyper-V Balloon Driver" as well as memory hot-add and
hot-remove functionality. This is happening with both the Gentoo-patched
3.10.32 kernel and the vanilla 3.12.5 kernel. The host machine has a
total of 24 GB of memory.
Brian Wong wrote On 3/6/2014 1:20 AM:
Just a correction: the vanilla kernel version is 3.13.5, not 3.12.5. )
For now, I am working around the issue by starting the VM with the
startup memory set to the maximum and letting Hyper-V take the usused
memory back when it is not in use. The VM will then get the extra memory
when it needs it.
Have I encountered a bug in the Hyper-V balloon driver?
Brian Wong
http://www.fierydragonlord.com
----- Original Message -----
From: "Victor Miasnikov"
To: [email protected]; "Brian Wong"
Cc: "Abhishek Gupta (LIS)" ( zzzzzzzzzzzzzzz (at) microsoft.com>; "KY Srinivasan" zzzzzzzzzzz (at) microsoft.com
Sent: Thursday, March 06, 2014 1:07 PM
Subject: Re: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled RE: [PATCH 2/2]
Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
Hi!
Short:
Question to Linux kernel team:
may be patch
>>> [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
can solve problems with dynamic memory hot add in Hyper-V VMs with Linux OS ?
Full:
BW>> .., if I set the startup memory to 512 MB, and enable dynamic
BW>> memory with a minimum of 512 MB and a maximum of 8192 MB,
BW>> the system will never allocate than 512 MB of physical memory
BW>>
BW>> Have I encountered a bug in the Hyper-V balloon driver?
BW>>
Unfortunately, It's long story . . . :-(
a)
I already ( on January 09, 2014 2:18 PM ) write about problems with "Online the hot-added memory" in "user space" see
P.P.S.
b)
See
Bug 979257 -[Hyper-V][RHEL6.5][RFE]in-kernel online support for memory hot-add
https://bugzilla.redhat.com/show_bug.cgi?id=979257
( Info from this topic may be interessant not only for RedHat users )
b2)
Detail about pathes related problem "Online the hot-added memory" in "user space" :
>>> [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
>>>
>>>
>>> === 0001-Drivers-base-memory-Export-functionality-for-in-kern.patch
>>> . . .
>>> +/*
>>> + * Given the start pfn of a memory block; bring the memory
>>> + * block online. This API would be useful for drivers that may
>>> + * want to bring "online" the memory that has been hot-added.
>>> + */
>>> +
>>> +int online_memory_block(unsigned long start_pfn) { struct mem_section
>>> +*cur_section; struct memory_block *cur_memory_block;
>>>
>>> . . .
>>> ===
>>>
>>>
>>> ==
>>> . . .
>>> == 0002-Drivers-hv-balloon-Online-the-hot-added-memory-in-co.patch
>>>   . . .
>>>    /*
>>> -   * Wait for the memory block to be onlined.
>>> -   * Since the hot add has succeeded, it is ok to
>>> -   * proceed even if the pages in the hot added region
>>> -   * have not been "onlined" within the allowed time.
>>> +   * Before proceeding to hot add the next segment,
>>> +   * online the segment that has been hot added.
>>>     */
>>> - wait_for_completion_timeout(&dm_device.ol_waitevent, 5*HZ);
>>> + online_memory_block(start_pfn);
>>>
>>>   }
c)
Before apply patches ( see in P.S. about native udev-script) we are need use one of this methods:
[ VVM:   URL of this topic skipped ]
c1)
"/bin/cp method" by Nikolay Pushkarev :
Following udev rule works slightly faster for me (assuming that memory0 bank always in online state):
SUBSYSTEM=="memory", ACTION=="add", DEVPATH=="/devices/system/memory/memory[1-9]*",
RUN+="/bin/cp /sys$devpath/../memory0/state /sys$devpath/state"}}
( VVM : of course all need be place in one line, 2 line in this msg. -- only for good visual formating reasons )
c2)
udev rule using "putarg" by Nikolay Pushkarev :
Even "/bin/cp method" udev rule work time to time not as need :-(
As a result, Nikolay Pushkarev write a program putarg.c, that is even faster than "/bin/cp method" :
==
#include <stdio.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <string.h>
int main(int argc, char** argv) {
int i, fd;
if (argc < 2) return 0;
if ((fd = open(argv[1], O_RDWR)) < 0) return 1;
for (i = 2; i < argc; i++) {
    if (write(fd, argv[i], strlen(argv[i])) < 0) {
      close(fd);
      return i;
close(fd);
return 0;
==
The first argument - the name of the output file ,
and argument number 2 ( and all subsequent ( if exist ) ) - are text that are wiil be written in output file.
Compile source code to executable file by run command:
gcc -o putarg -s putarg.c
The resulting binary need be placed an accessible location , such as /usr/bin, or wherever you want.
Now udev rule using "putarg" can be written as :
SUBSYSTEM=="memory", ACTION=="add", RUN+="/usr/bin/putarg /sys$devpath/state online"
This complex solutions ( compiled file and udev-rule ) works exceptionally fast.
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
P.S.
Nikolay Pushkarev about standart udev-script :
Strange, that the native udev-script
SUBSYSTEM=="memory", ACTION=="add", ATTR{state}="online"
triggered somehow through time ( VVM: very often not work as need )
P.P.S.
----- Original Message -----
From: "Victor Miasnikov"
To: "Dan Carpenter"; "K. Y. Srinivasan" ; <[email protected]>
Cc: "Greg KH" ; <[email protected]>; <olaf (at) aepfle.de>; ""Andy Whitcroft"" <zzzzzzzzzzzz (at)
canonical.com>;
<jasowang (at) redhat.com>
Sent: Thursday, January 09, 2014 2:18 PM
Subject: RE: [PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
Implement the file copy service
Hi!
> Is there no way we could implement file copying in user space?
For "file copy service" "user space" may be pretty good
But I ( and other Hyper-V sysadmin) see non-Ok ( in "political correct" terminalogy) results with "hv: balloon: Online
the hot-added memory" in "user space"
==
[PATCH 2/2] Drivers: hv: balloon: Online the hot-added memory "in context"
==
What news? Roadmap?
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
~
~
~
~
~
~
~
~
~
----- Original Message -----
From: "KY Srinivasan"
To: "Victor Miasnikov"; [email protected]; "Brian Wong"
Cc: "Abhishek Gupta (LIS)"
Sent: Thursday, March 06, 2014 1:23 PM
Subject: RE: Linux does not use more than the startup RAM under Hyper-V with dynamic memory enabled RE: [PATCH 2/2]
Drivers: hv: balloon: Online the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
> -----Original Message-----
> From: Victor Miasnikov
> Sent: Thursday, March 6, 2014 3:38 PM
> To: [email protected]; Brian Wong
> Cc: Abhishek Gupta (LIS); KY Srinivasan
> Subject: Re: Linux does not use more than the startup RAM under Hyper-V
> with dynamic memory enabled RE: [PATCH 2/2] Drivers: hv: balloon: Online
> the hot-added memory "in context" Re: [PATCH 1/1] Drivers: hv:
>
Victor,
I will try to get my in-context onlining patches accepted upstream.
K. Y

Dynamic memory freezes

Hi guys,
we are using three node cluster with Windows Server 2012R2. Our virtual machines are virtual
desktops with Windows 8.1 guest OS, 4 vCPU and 8 GB of dynamic memory.
Sometimes dynamic memory stops working, guest OS is not able to allocate more RAM and starts to swap. This swapping slows performance of all our virtual machines, since
all virtual machines are on same cluster shared volume. This happens at random RAM amount allocated, for example at 2.6 GB,
which is way below our 8 GB limit.
Live migration helps to solve this issue for migrated VM, after live migration the guest OS is able to allocate
more ram and starts to work properly, swapping disappears. However after some time, problem appears on other VM.
Our physical machines have 2x 10core Xeon and 394 GB of RAM. Only about 30% of RAM is used, so there is free space for all virtual machines. But we need to solve this issue so dynamic RAM starts to work properly.
Do you have similar experience?

Hi manasj,
Please check event log "Microsoft-Windows-Hyper-V Worker/Admin " to see if there is any clue .
Also please refer to the following potetial cause :
" In a Hyper-V Failover Cluster, a virtual machine can have its configuration information stored on cluster shared storage (a Physical disk resource or a Cluster Shared Volume (CSV). If the physical disk resource or the Cluster Shared Volume (CSV) goes Offline or Fails,
the VM placed in a critical state. Once the storage is re-connected, the VM should no longer be in a critical state. However, virtual machine worker process (vmwp.exe) does not refresh all of its file handles."
For details please refer to following link :
http://support.microsoft.com/kb/2504962/en-us
Best Regards
Elton Ji
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

Dynamic Memory in the production environment

Hi,
We configure all production VMs with static memory on Hyper-v 2012
we need to make sure there is no negative impact on VM performance if we use Dynamic memory , as we read for 2 years
not recommend to use dynamic memory in the production.
I preferred the answer with Trusted URL
Ramy

Hiya,
The best answer you can get is, that it is dependent on the application running on the server O/S.
The reason that dynamic memory is not recommended for production environments, is that many applications are not supporting it. That is usually seen on memory intensive applications(SQL is an example) or simply because of caching types of functions. (SharePoint
is an example).
The major concern here is when VM's are decreasing the memory for a VM, the application does not understand this. Usually it is not a problem when increasing the memory. - Again it will depend on the application.
Think of it as hot swap memory and how you used to use that in the physical machine days.
Also when there is not adequate memory, operating system will use paging. Paging uses disks and disks has a lower access time than memory.
In general dynamic memory is easier to control than dynamically expanding disks, as you can set the buffer size of the memory, which is still not available for the dynamic disks.
Besides the above, the following links states:
"Workloads that are not NUMA-aware will not take advantage of virtual NUMA. However, the guest operating system may perform some NUMA optimization. Enabling Dynamic Memory (therefore presenting only a single virtual NUMA) should not cause performance
degradation"
http://technet.microsoft.com/en-us/library/dn282282.aspx

Dynamic Memory is not working all the time

We are in the process off moving our 2008R2 VM's from the 2008R2 HyperV servers to new Server 2012R2 Hosts.
We shut down the VM's copy the files and VHD's to the new CSV's en import the VM in the Hyperv Manager. Then we make them high available in the Failover Cluster Manager (Configure role - Virtual machine). We mount the integration tools and update the
VM to version 6.3.9600.16384
For a specific type of VM (mostly RDS Host servers) we always had Dynamic Memory configured (when they were hosted on de 2008R2 platform), so we are using the same settings on the 2012r2 platform. The memory settings were;
Startup memory: 1024 MB
Minimum memory: 1024 MB
Maximum memory: 12288 MB
These VM's reboot every morning, this is done for specific reasons. But now once in a while (once per week/2 weeks) we notice that the VM's are not using more memory then 1024 MB while the demand is much higher. Rebooting the server helps most of the times,
live migrating to another host also helps. In the VM we see that memory usage in the taskmanager is 99-100%, and after the move it immediately starts using more than the minimum configured amount.
Until the failover the memory usage was 1024 MB and it did not get any higher.
This happened several times. Last week we changed the Memory configuration to:
Startup memory : 2048 MB
Minimum memory: 2048 MB
Maximum memory: 12288 MB
But this morning we had a call about the performance of one of the VM's, We saw that it was only using 2 GB memory while the demand was much higher. After live migrating it to another host it started using more memory immediately.
The 2012R2 hosts are not overcommited, there is a lot of memory still available for the VM's. Those VM's never had this problem on the 2008R2 Hyperv platform.
Any idea why this happens?
Peter Camps

Peter,
I think this is a bug of some sort. I say that because the components that make up dynamic memory are as follows.
Memory Balancer(Host service, coordinates how memory changes are made.) This is also what shows the memory demand counter i believe.
Dynamic Memory Virtualization Service Provider (this is included your VMWP.exe proccess, one per VM. Essentially how it runs on the host. He listens to the Service Client for metrics)
Dynamic Memory Virtualization Service Client (this is inside the VM and reports to the Dynamic Memory Virtualizaton Service Provider.)
Since you live migrated the machine it made dynamic memory work on the other host. This means the Service Client is running in the client and shouldn't be an issue. The Memory Balancer is the server and shouldn't be the issue, so this means the "Dynamic
Memory Virtualization Service Provider" is in question. When you live migrate the machine its going to create a new VMWP.exe process on the clustered server. So now the question is it the host that couldn't listen to the service or the worker process
skipped a beat and has a bug.
Out of curiosity does it happen to both hosts? Also have you profiled the servers to see how much memory they really require on start-up? When you reboot the RDS servers, how many VM's do you reboot and is it a staggered process?

Dynamic memory not released to host

Dear Techies,
Ours is a small Hyper V virtual server infrastructure with three DELL power-edge physical hosts(Windows server 2012 Datacenter) and around 15 virtual machines running on top of it. The hosts are added to fail-over cluster. Each host has 95 GB RAM. All the
VMs are running Windows server 2012 standard edition.
We have installed terminal services(TS licensing, TS connection broker, TS session host) in four VMs with the following dynamic memory settings:
Start-up RAM : 2048 MB
Minimum RAM : 2048 MB
Maximum RAM : 8192 MB
Below mentioned applications are configured in the server:
Nova Application Installed
SQL Developer Tool is Configured (ODBC Connection established for Database communication)
FTPs communication allowed for File Transfer
McAfee Agent is configured (Virus Scanner)
Nimsoft Robot Agent Configured – Monitoring
Terminal Service
Enabled Multiple terminal sessions based on customer requirement
BGinfo tool configured through group policy for customized desktop background
The average memory utilization in the terminal servers are 3.6 GB. As per dynamic allocation the maximum RAM requirement/allocation till date is 4GB. As seen in Hyper V console, the current RAM demand is 2300 MB and assigned memory is 2800 MB.
However, the earlier assigned RAM in the server is ballooned/faked to the VM as driver locked memory. This is by design. Despite the memory being released back to the host, the server still shows up the 4Gb which makes the memory utilization report from
monitoring tools look 80% (3.2 GB out of 4 GB).
As a result, the memory utilization report is always based on the current dynamically allocated RAM and not calculated based on the maximum assigned RAM(8GB in this case). To make it clear: If the
currently assigned RAM is 4Gb and utilization is 3.2 GB the utilization % is reported as 80%. However, if calculated in accordance with maximum RAM capacity of the server it would be 40% ((3.2/8)*100).
Is there any way to release the driver locked memory from the VM.?
Regards,
Auditya N

I am not really clear on the point of your question.
Allocated RAM is what is currently in use / allocated to a VM out of the physical RAM pool. It is Demand + Buffer. The demand is based on the applications in the VM and what they think they need and how efficiently they return unused memory
to the machine. This has nothing to do with in-application paging (which happens a lot with Terminal Servers).
So yes, the memory utilization is accurate in relation to physical RAM utilization.
Dynamic Memory is about efficiency, not about over-allocation. Hyper-V can never give VMs more RAM than is in the physical RAM pool. The VMs can be configured to have more RAM than is in the physical RAM pool - but the VMs will hit the top of
the pool and not be allowed to have any more. There is no ballooning or paging to disk.
So, you maximum allocated could go beyond what is possible. But that would mean that your utilization would be artificially low if this was used in the calculation.
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat.

Guest keeps on consuming all the dynamic memory

Software:
Host: Windows Server 2012 R2 x64 fully patched
Guest: Windows 8 Update 1 x64 (Fully Patched).
I've been experience this problem for a few months and really don't know where to go from here. After starting the specific guest that is having the issue it will happen anywhere between 3-7 days. The guest will consume almost all the memory. The only way
i can log into the guest is with hyper-v manager. Remote Desktop will hang. Then when i log into the guest i can't bring up task manager or many things. Every time i have to kill the vm and restart it. Last time i thought to keep process monitor up and
running so i could inspect things. Well it happened today again. According the hyper-v manager the guest was consuming almost 24GB of memory:
here is the memory demand:
What's wierd is when i logged into the vm it was showing it only had 4GB of memory with 3.1GB used. There was hardly anything running other than a few services and they weren't using much memory. Quickly process explorer locked up on me. After logging in
for say 5-10 minutes this always happens too. After trying to diagnose the for a few minutes it will appear i have rebooted the system because i will get the following:
but i definitly did not reboot. If i come back in 24 hours it will still be at the same spot. Again i have to kill the vm but the same vicious cycle will ensue. I'm at a loss how to diagnose this. I've tried to scour the event viewer but it hasn't helped.
Sure i could just turn dynamic memory off or lower the upper threshold of the dynamic memory but i shouldn't have to do that. Thats really just masking the problem. The only default value of dynamic memory i changed was the "Startup Memory". I really
don't want to rebuild the vm but what other options do i have here?

Well i updated one of my main programs on that machine (the on that i would think used "Most" of the memory) and i "thought" it went away but nope i was wrong. It came back. Luckily i left rammap & process explorer running. I could
it when it was consuming almost 6GB rather than the 20+ GB like before. Here is a screenshot of rammap:
As you can see "Paged Pool" memory is huge. I closed almost every app running on the system hoping that would help but it really didn't. The only thing that is running at this point outside of the normal windows processes is EMET 5 & Nortan
Antivirus. I tried to stop halt "Auto Protect" & the firewall but that didn't help any.
unfortunately, though i did catch it early, most programs will fail to load. Even file explorer fails to load. As usual after interacting with process explorer for awhile i get the white screen of death out of it. Hmmmm figure out why so much pool memory
is being consumed.

Windows Server 2012 HyperV Cluster - Dynamic Memory Config

Hello There,
I am about to configure two node HyperV Cluster on Windows 2012 R2.
There is 192 GB RAM on each node but my VMs requirement is around 216 GB but some VMs doesn't require the full memory capacity all the time so in this scenario what is the best method to use dynamic memory utilization.
RAM
48 GB
48 GB
16 GB
48 GB
48 GB
8 GB
I need to provision 6 VMs with the above memory config but want to utilize it effectively through dynamic memory, please suggest.
I don't need the full RAM capacity all the time as in the table.
Thanks,
maqsood
Maqsood Mohammed Senior Systems Engineer MCITP-Enterprise Admin & ITILv3 Foundation Certified

Of course you can, this is the aim of the dynamic memory feature.
In the VM's memory configuration, you will find three parameters:
Start up RAM : This is the memory allocated to the VM when it's started
Maximum RAM : This is the maximum amount of memory that a VM can request from the Hypervisor. If there is available memory the host will grant it more memory, if not, the VM will not receive memory
Minimum RAM : Because the memory configuration is dynamic, when a VM will not use memory, it will give back its memory resources to the host, but it will never go down its Minimum RAM value (The total minimum RAM of all VMs can never exceed the available
memory, otherwise the VMs will not even start)
And there is the Priority option that tell the hypervisor which VM to satisfy first.
Look here for more explanation
Regards, Samir Farhat Infrastructure and Virtualization Consultant || Virtualization, Cloud, Azure ? Follow and Ask here https://buildwindows.wordpress.com

Queue of arrays without dynamic memory allocation

Hey folks,
I'm working on optimizing a timing critical VI. This VI is the
producer in a producer consumer architecture. I'm trying to populate
a queue from file in a manner that is as efficient as possible. My
current plan of attack is:
- read block of data from file and populate array (pre-allocated).
- add array (always of the same size) to Queue with a max size defined
(e.g. 50 elements)
- This is in a while loop as is the standard producer consumer model.
To improve the performance I would like to ensure that there is no
dynamic memory allocation on the Queue's behalf. This is easily done,
from what I understand, if the data type in the queue is of the same
type (e.g. double, int). However, since the size of an array can vary
does this mean that any queue of arrays will always dynamically
allocate memory? Is there a way to ensure that the queue will always
use the same memory as in a circular queue?
Thanks,
Steve

Duplicate.
Try to take over the world!

Request additional allocation of dynamic memory

Hyper-V 2012 host
Server 2012 R2 guest
The guest is configured for startup and minimum memory of 2 GB with a maximum memory of 64 GB. Once per day a PowerShell script runs which uses 7-Zip.exe to compress some large files. The 7-Zip command fails if there is less than 10 GB of memory
immediately available. I do not want to allocate that much minimum memory to the guest, as it doesn't need it for anything but this one script.
How can I programmatically request or instruct the host to allocate 10 GB to the guest before I run the 7zip command?
Currently I am launching a PowerShell job running Sysinternals TestLimit64.exe and running it until I see the allocated memory go up on the host. But this workaround is inelegant, inexact, imprecise, and slow.
Thank you.
Tim Curwick
MadWithPowerShell.com

Hi Tim,
Please refer to the following link which are related to scripting dynamic memory:
Scripting dynamic memory, part 1: reading the configuration<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/07/scripting-dynamic-memory-part-1-reading-the-configuration.aspx<o:p></o:p>
Scripting dynamic memory, part 2: displaying current usage information<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/08/scripting-dynamic-memory-part-2-displaying-current-usage-information.aspx<o:p></o:p>
Scripting dynamic memory, part 3: looking at performance counters<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/13/scripting-dynamic-memory-part-3-looking-at-performance-counters.aspx<o:p></o:p>
Scripting dynamic memory, part 4: configuring memory<o:p></o:p>
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/14/scripting-dynamic-memory-part-4-configuring-memory.aspx<o:p></o:p>
Scripting dynamic memory, part 5: changing minimum memory
http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/09/15/scripting-dynamic-memory-part-5-changing-minimum-memory.aspx
Best Regards,
Vincent Wu
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

Dynamic memory and GC.

Similar Messages

Maybe you are looking for