HT3258 dynamic memory manager

Does Apple fix the problems with the dynamic memory manager in Mountain Lion?

After more tests I can say that this memory consumption
disappear with a release version of the swf executed directly by
the browser.

Similar Messages

  • Using Dynamic Memory Management

    Hi All,
    http://technet.microsoft.com/en-us/library/ms178145(v=sql.90).aspx
    I have red the above article regarding Dynamic Memory Management and now I just want to check whether this option is enabled in my environment or not.
    Is this by default enabled in the server or else Please suggest on how to achieve this
    Regards,
    Kalyan
    Grateful to your time and support. Regards, Shiva

    Is this by default enabled in the server or else Please suggest on how to achieve this
    Regards,
    Kalyan
    Grateful to your time and support. Regards, Shiva
    If you read the first line of the link you posted its clearly written SQL Server memory management is dynamic. You cannot change it . You can just restrict buffer pool usage by puttling limit on MAX Server memory. In that case as well SQL Server might cache
    all Buffer pool memory.
    Event though you cap max server memory still direct memory allocations for large pages will be done directly by windows API and that will also come under SQL Server memory utilization
    In short SQL server memory allocation is always dynamic you cannot change it
    Please mark this reply as answer if it solved your issue or vote as helpful if it helped so that other forum members can benefit from it.
    My TechNet Wiki Articles

  • OSX inactive memory management problems

    On my 2010 Macbook Air 11" running OSX 10.8 Mountain Lion, I have run into some major issues with memory management. This problem persists across both my Mac machines and multiple generations of OSX. I have a 2007 2.2 Ghz Core 2 Duo Macbook Pro Running Lion (upgraded from Tiger) with 3 gb 667Mhz DDR2 RAM and my 2010 Air 1.4Ghz Core 2 Duo (Upgraded from Snow Leopard to Mountain Lion) with 2gb 1067mhz DDR3 RAM. On both machines, for some time now, during normal usage; especially web browsing using Chrome and Safari (respectively); the inactive RAM on both machines will grow to consume around 30% of all RAM and force time consuming page outs to the mass storage drives on both machines. The Pro has a slower hard drive and the problem is the worst here, the air has a much faster SSD. I have found myself constantly having a window of Activity Monitor up on at least one of my desktops watching my RAM usage, using terminal to purge ram upwards of 10 times an hour to prevent costly page outs, especially on the Pro. I know Apple claims that inactive RAM is essentially free RAM that is temporarily storing recently used information for ease of access later and that it's supposed to be released as free memory when needed, but this obviously is not happening. Right now on my air my swap file is over 650Mb and I've seen it top 2Gb before. The air is exponentially better than the Pro due to the faster SSD, but I do notice substantial UI lag and a massive drop in fluidity as soon as my meager 2Gb is full and I start paging. The Pro is another story entirely, the entire system will essentially become unusable, having to wait several seconds for mouse clicks to even register. That's why I upgraded the stock 2Gb of RAM it comes with to 3Gb hoping that a 150% increase in RAM would help, but it just prolonged the inevitable. I still end up paging out just as bad across both systems if un checked. Even when I keep a close eye on memory usage and purge often, I still end up paging out because I'm not vigilant enough.
    I have to limit my browsing to less than 5 tabs and keep my number of open programs less than 2 on both machines. My active and wired memory rarely seem to top 70%, meaning the rest gets taken by inactive, which isn't functioning as Apple claims. Even if my conclusions aobut what is happening under the hood are incorrect, something is going terribly wrong. I can't upgrade the RAM on my Air at all, and the RAM on my Pro is capped at 4Gb. I'm holding on upgrading because I don't think the excess hardware will solve this software memory problem. Whatever is happening is causing a serious drop in performance for me (yes I do know I have underpowered machines), but there has to be something I can do to speed performance. I've read about disabling the dynamic page file entirely, which just seems to crash the system when free memory is gone, and I've read about programs that claim to free memory. Those programs seem to work by taking a high priority in the process heigharchy of the OS and then proceeding to eat up large portions of RAM and releasing them as needed in an attempt to replicate the true intentions of inactive RAM, but I've heard of problems with this method as well. Does anyone have a viable solution? Monitoring my RAM usage myself and ensuring I don't end up paging out is costly, time consuming, annoying, and inefficient since I fail to catch the problem before I page if I get particularly busy. There is no other OS I've ever been acquainted with that has this problem, not any flavor of Linux, not even the dreaded windows. I seriously hope Apple can do something to manage this runaway memory problem. I'd like to be able to open more than 3 windows in Safari. I've had to purge 3 seperate times while writing this on my Air, and I now have 678 inactive memory, 741 inactive, 582 wired, and less than 14Mb free out of 2Gb with a growing swap at 680Mb. Each purge becomes less and less effective and the last one I did freed up only about 100Mb and it got eaten up again by inactive in less than 10 seconds. On my Air, the memory hog is Safari right now at 700Mb between the web content and flash player with only Facebook, youtube, and this Apple Supprt tab open. I have NO other applications running in the fore or background other than Activity Monitor and Terminal. On my Pro the memory hog is always kernel task, I use Chrome and Safari both. While the memory used by the browser does not usually take up the most substantial portion of the total used RAM out of any process, the more tabs I open, the more RAM I use. The browser is usually the second heaviest RAM hog to Kernel Task. So it seems that across the two machines there are two lsightly different manifestations of the same problem with the same results: massive performance drops and extremely annoying and costly page outs no matter the reason. I just want this problem to go away. I've used underpowered windows laptops that can open a dozen tabs in a heavier browser like IE or Firefox while using other programs like Word or Excel and more with no memory lag issues. There's no way in **** I could manage to open that many pages in a browser while using Pages and/or Numbers on either of my machines and expect reliable (swap free performance). This is just kind of sad in my opinion. Does anyone have a way to get my OSX machine running smooth so that I can remove the one thing that windows and Linux fan boys get the right to laugh at my Macs for?

    Hi Zephryl,
    I was actually able to get an initial response from Sun on this a few months ago. However, the Sun Swing team has not followed up on a resolution for this pervasive problem, even though they noticed the same problem when running a test applet I had created for them. Apparently, I.E. is not releasing memory from the heap.
    Below is a quote from a Sun rep. on this in an e-mail sent to me on Dec 4, 2002:
    "I suspected the leak is in the native code because the # of handles and GDI objects keep increasing but no obvious Java objects are left behind in the Java heap during page switch."
    So, until Sun and/or Microsoft work out a solution to this, anyone who uses I.E. 6 and applets for their UI seems to be in a lot of trouble.
    As a note, trying to invoke the Garbage Collector does not do anything, but generally a very small amount of memory will be released (like maybe 5-10% of the memory allocated for the applet).
    Cheers!
    Avi Gray
    Global Computer Enterprises

  • Oracle 9i Automatic PGA Memory Management

    Hello,
    my team and me, we are facing difficulties to change the size of the PGA used by our server processes for HASH JOIN, SORT... operators,
    here you can see the results of "select * from v$pgastat":
    [pgastat dynamic view results|http://pastebin.com/m210314dc]
    We have been increasing consecutively our pga_aggregate_target parameter from 1.7 Gb initially to 4Gb then at the end 6Gb, the value of "Global memory bound" and " aggregate pga auto target" on the link above are still equal to 0.
    I have been reading threads on the forum and documentation see below, I understand how the global memory manager (CKPT) computest the sql memory target and then the global memory bound, as far as I understand I can only "play" on the pga_aggregate_target value in order to increase the size of our PGAs (I exclude to play with hidden parameters).
    - Joze Senegacnik: Advanced Management of working areas in Oracle 9i/10g : http://tonguc.yilmaz.googlepages.com/JozeSenegacnik-PGAMemoryManagementvO.zip
    - Dageville Benoit and Zait Mohamed: SQL memory management in oracle 9i
    Here different information that could be usefull:
    OS: solaris 10 (db running in a non global zone)
    Arch: 64-bit sparcv9 kernel modules
    Physical memory: 32 Gb (being shared between all non global zones)
    Oracle version: 9.2.0.5 32bits
    Values of init parameters and hidden parameters that could be relevant:
    [init parameters|http://pastebin.com/m40340cf4]
    [hidden parameters|http://pastebin.com/m50d74c53]
    Maybe useful queries:
    over work areas views, I use the following script:
    [wa_analysis.sql|http://pastebin.com/d606ebd9b]
    and the result of it:
    [result of script wa_analysis.sql|http://pastebin.com/m5f49a2e5]

    Joze Senegacnik wrote:
    - either your sessions are using a lot of memory for storing variables like pl/sql arrays which is subtracted from automatic management: PGA_AGGREGATE_TARGET - (aggregated persistent area + a part of the run time area of all server processes)
    - you are hitting a bug
    - or maybe something elseI am really happy you come to this conclusion too, they are the same we made with my team and we have submitting to Oracle support via metalink SR 3-1216060641, we were asking if we hit the following bug (in note 1) or we leak about pl/sql or java... or else indeed,
    note 1: PGA_AGGREGATE_TARGET Assigned Memory Is Left Unconsumed When Set High [ID 844542.1]
    Joze Senegacnik wrote:
    I would like to know:
    1.) what were the values for global memory bound and autotarget immediately (or in short time) after the database restart or when you have increased them Just after the restart of the database and just after the change of P_A_T, we query v$pgastat immediately after and the value of global memory bound and auto target were equal to 0 byte,
    2.) If you are able to change value of PGA_AGGREGATE_TARGET (P_A_T) to 10GB what happens with global memory bound and auto traget. They should be positive at least for a short time. As this is a dynamic parameter you can change it for a short time, run queries and set it back.We plan to do this tonight, we have an "heavy" ITIL change management procedures that allow us to make changes approved by change manager and only during night maintenance window on production system, I come back to you tomorrow. But we have been increasing from 1,7Gb to 4Gb to 6Gb, each time I have been querying v$sgastat in the next 2 mins and global memory bound and auto target were equal to 0 byte.
    3.) Have you checked on the OS level how much memory are using server processes - do these numbers come along with what Oracle says. Not during problematic activities, meaning active work areas performing HASH-JOIN, SORT... operators,
    unfortunately it is a production system, even if he performs poorly, we are not allowed to try or retry the poor queries, but if it comes again I'll do it,
    during low activities, here the results paste with the scripts I used:
    [pga processes info in oracle|http://pastebin.com/f2e540062]
    I spooled the result rows of this previous script in /var/tmp/pga_processes.log then I loop over all processes pid and display pmap output anon info like this:
    h5. cat /var/tmp/pga_processes.log | awk -F' ' '{print $5}' | xargs -n 1 -i pmap -x {}| grep -v 'Addres' |egrep 'Kb' 2>&1 > /var/tmp/pga_processes_os.log
    then I merge line by line the two files with unix paste command, here the results:
    [os and oracle pga informations|http://pastebin.com/f4135c8a6]
    4.) How many server processes are running on you system in average/max and are you using just dedicated processes or also shared?in average 250, we are only using dedicated processes,
    5.) At time of low activity is the global memory bound still 0 or becomes > 0. I have been querying every 15 min during more than 24 hours low activities, it still stay to 0,
    5.) Are you experiencing paging/swapping on OS level?No, here orca figures for details:
    [free memory|http://img509.imageshack.us/img509/5897/ohuron1asd2gauge1024xfr.png]
    swap
    [pagein pageout|http://img121.imageshack.us/img121/6946/ohuron1asd2gaugepginper.png]
    [memory usage|http://img19.imageshack.us/img19/2213/ohuron1asd2gaugeppkerne.png]
    6.) Please post the result of: select * from X$QESMMSGA ;during low activities, [results X$QESMMSGA|http://pastebin.com/f61df7093]
    While you will be answering to my questions I'll try to figure out what we can do to properly diagnose the problem. As you are on 9i it is a little bit harder.I am really kind of your help, as we say in my country, "if you need tow arms one day to carry something, call me."
    --Jeremy Baumont                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           

  • Dynamic memory allocation failure

    Dear reader,
    We sometimes have a problem where our windows 2012 r2 RDS virtual servers, that reside on windows 2012r2 hyper-v hosts, loose their dynamic memory and only have their startup memory left to work with. Users start complaining that things are very slow etc.
    If I check several screens (RDS Broker load balancing, hyper-v manager, cluster manager and the vm's task manager) it's clear that the vm only has its startup memory allocated. I'm not sure if this happens instantly or immidiatly after the nightly reboot.
    To resolve the problem we have to call all users on the vm where it happens and ask them to logoff (if they are even able to), and then we reboot the machine.
    I have checked the logs from the machine where the VM resides on and the logs from the vm itself. But I cannot find anything. We also have alot of windows 2008r2 vm's with dynamivc memory, but none of those have ever had this problem.
    Searched the internet, but so far it seems we are only.
    Can anyone give me a lead to troubleshoot this?
    Best regards,
    Ruud Boersma
    MCITP Enterprise administrator

    Hi all,
    I'm going to be "one of those people" who revives dead posts for something that is relevant but obviously not fixed... sorry in advance!
    We have the exact same situation, a bunch of RDSH guests with Dynamic memory turned on (60+ of them). every day between 1-8 of them will fail to allocate Dynamic memory and will be stuck on their startup RAM amount. This really hurts our users at peak
    times.
    I have engaged our TAM and have raised a case with PSS, Usual story "your the only one with this problem". Which obviously isn't true.
    So, we have tons of free RAM on the hosts, 600GB+ on most of them, the issue affects RDSH hosts at random, across multiple hosts and clusters.
    The screen shots attached are of one of our hosts from this morning. it has 8GB startup, 8GB minimum, 32GB Maximum RAM configured, with a 23% buffer. the host has 752GB RAM FREE. Notice how the perf counter "Hyper-V Dynamic Memory Integration Service"
    is reporting "0", it should be reporting "32768". also under task manager on the VM we are missing "Maximum memory" which should be just below "Hardware reserved" in the bottom right hand corner.
    Looks like the balloon driver is being delayed at boot time, we are going to XPerf all the servers in the hope that we can catch the critter. It's an unusual problem.
    We only have Acrobat PDF viewer, word viewer, excel viewer and two custom dot.NET applications installed on the guests. Some of the servers are also just dumb RDSH hosts, with not connection broker configured, using an F5 loadbalancer for load distribution
    and session management. All guests are 2012R2 patched up to March 2015, integration Services are installed and up to date (its an enlightened OS remember).

  • Dynamic Memory is not working all the time

    We are in the process off moving our 2008R2 VM's from the 2008R2 HyperV servers to new Server 2012R2 Hosts.
    We shut down the VM's copy the files and VHD's to the new CSV's en import the VM in the Hyperv Manager. Then we make them high available in the Failover Cluster Manager (Configure role - Virtual machine). We mount the integration tools and update the
    VM to version 6.3.9600.16384
    For a specific type of VM (mostly RDS Host servers) we always had Dynamic Memory configured (when they were hosted on de 2008R2 platform), so we are using the same settings on the 2012r2 platform. The memory settings were;
    Startup memory: 1024 MB
    Minimum memory: 1024 MB
    Maximum memory: 12288 MB
    These VM's reboot every morning, this is done for specific reasons. But now once in a while (once per week/2 weeks) we notice that the VM's are not using more memory then 1024 MB while the demand is much higher. Rebooting the server helps most of the times,
    live migrating to another host also helps. In the VM we see that memory usage in the taskmanager is 99-100%, and after the move it immediately starts using more than the minimum configured amount.
    Until the failover the memory usage was 1024 MB and it did not get any higher.
    This happened several times. Last week we changed the Memory configuration to:
    Startup memory : 2048 MB
    Minimum memory: 2048 MB
    Maximum memory: 12288 MB
    But this morning we had a call about the performance of one of the VM's, We saw that it was only using 2 GB memory while the demand was much higher. After live migrating it to another host it started using more memory immediately.
    The 2012R2 hosts are not overcommited, there is a lot of memory still available for the VM's. Those VM's never had this problem on the 2008R2 Hyperv platform.
    Any idea why this happens?
    Peter Camps

    Peter,
    I think this is a bug of some sort. I say that because the components that make up dynamic memory are as follows.
    Memory Balancer(Host service, coordinates how memory changes are made.) This is also what shows the memory demand counter i believe.
    Dynamic Memory Virtualization Service Provider (this is included your VMWP.exe proccess, one per VM. Essentially how it runs on the host. He listens to the Service Client for metrics)
    Dynamic Memory Virtualization Service Client (this is inside the VM and reports to the Dynamic Memory Virtualizaton Service Provider.)
    Since you live migrated the machine it made dynamic memory work on the other host. This means the Service Client is running in the client and shouldn't be an issue. The Memory Balancer is the server and shouldn't be the issue, so this means the "Dynamic
    Memory Virtualization Service Provider" is in question. When you live migrate the machine its going to create a new VMWP.exe process on the clustered server. So now the question is it the host that couldn't listen to the service or the worker process
    skipped a beat and has a bug.
    Out of curiosity does it happen to both hosts? Also have you profiled the servers to see how much memory they really require on start-up? When you reboot the RDS servers, how many VM's do you reboot and is it a staggered process?

  • Guest keeps on consuming all the dynamic memory

    Software:
    Host: Windows Server 2012 R2 x64 fully patched
    Guest: Windows 8 Update 1 x64 (Fully Patched). 
    I've been experience this problem for a few months and really don't know where to go from here. After starting the specific guest that is having the issue it will happen anywhere between 3-7 days. The guest will consume almost all the memory. The only way
    i can log into the guest is with hyper-v manager. Remote Desktop will hang. Then when i log into the guest i can't bring up task manager or many things.  Every time i have to kill the vm and restart it. Last time i thought to keep process monitor up and
    running so i could inspect things. Well it happened today again. According the hyper-v manager the guest was consuming almost 24GB of memory:
    here is the memory demand:
    What's wierd is when i logged into the vm it was showing it only had 4GB of memory with 3.1GB used. There was hardly anything running other than a few services and they weren't using much memory. Quickly process explorer locked up on me. After logging in
    for say 5-10 minutes this always happens too. After trying to diagnose the for a few minutes it will appear i have rebooted the system because i will get the following:
    but i definitly did not reboot. If i come back in 24 hours it will still be at the same spot. Again i have to kill the vm but the same vicious cycle will ensue. I'm at a loss how to diagnose this. I've tried to scour the event viewer but it hasn't helped.
    Sure i could just turn dynamic memory off or lower the upper threshold of the dynamic memory but i shouldn't have to do that. Thats really just masking the problem. The only default value of dynamic memory i changed was the "Startup Memory". I really
    don't want to rebuild the vm but what other options do i have here?

    Well i updated one of my main programs on that machine (the on that i would think used "Most" of the memory) and i "thought" it went away but nope i was wrong. It came back. Luckily i left rammap & process explorer running. I could
    it when it was consuming almost 6GB rather than the 20+ GB like before. Here is a screenshot of rammap:
    As you can see "Paged Pool" memory is huge. I closed almost every app running on the system hoping that would help but it really didn't. The only thing that is running at this point outside of the normal windows processes is EMET 5 & Nortan
    Antivirus. I tried to stop halt "Auto Protect" & the firewall but that didn't help any.
    unfortunately, though i did catch it early, most programs will fail to load. Even file explorer fails to load. As usual after interacting with process explorer for awhile i get the white screen of death out of it. Hmmmm figure out why so much pool memory
    is being consumed.

  • [2008 R2] Hyper-V Dynamic memory warning with half of vHost memory free

    Our virtual server host system has about 19GB of Memory free (out of 32GB total), yet a virtual guest using Dynamic Memory was only being assigned 8GB and the demand was around 13GB, therefore generating a Warning state on the Memory Status. Logging onto
    the guest machine showed about 8GB of memory being used as well. The end-users were receiving memory errors in their applications. Any idea why the guest system was in this warning state?

    There is a perception in the OS.  And different numbers come from different places.
    In a nutshell, the RDP Server has a memory leak if you constantly disconnect and reconnect - and it ends up chewing up memory, but when available memory is tested this memory consumption is missed. If you logout of the RDP session instead of disconnecting
    then the memory is given back and can actually be used.
    It is a strange interaction with RDP that has been there since the original release.  But it is specific to using RDP to connect to the Hyper-V Server for VM Management and disconnecting without ever logging out.
    There was also a google process that many folks reported a long while back that caused memory consumption that prevented VMs from starting as well.
    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.

  • Relationship between Dynamic Memory Heap and Heap Data Structure

    This question is not strictly related to Java, but rather to programming in general, and I tend to get better answers from this community than any where else.
    Somehow, my school and industry experience have somehow not given me the opportunity to explore and understand heaps (the data structure), so I'm investigating them now, and in particular, I've been looking at applications. I know they can be used for priority queues, heap sorts, and shortest path searches. However, I would have thought that, obviously, there must be some sort of relationship between the heap data structure, and the dynamic memory heap. Otherwise, I can think of no good reason why the dynamic memory heap would be named "heap". Surprisingly, after searching the web for 90 minutes or so, I've seen vague references, but nothing conclusive (trouble seems to be that it's hard to get Google to understand that I'm using the word "heap" in two different contexts, and similarly, it would not likely understand that web authors would use the word in two different contexts).
    The Java Virtual Machine Spec is silent on the subject, as "The Java virtual machine assumes no particular type of automatic storage management system, and the storage management technique may be chosen according to the implementor's system requirements."
    I've seen things like:
    [of dynamic memory] "All the blocks of a particular size are kept in a sorted linked list or tree (I extrapolate that sorted tree could imply heap)"
    [of dynamic memory] "The free and reserved areas of memory are maintained in a data structure similar to binary trees called a heap"
    [of dynamic memory] "This is not related to the heap data structure"
    [of dynamic memory] "Not to be confused with the data structure known as a "heap"
    [of data structure] "Not to be confused with the dynamic memory pool, often known as TheHeap"
    At this point, I've come to surmise that some (but not all) memory management algorithms use heaps to track which (pages? blocks? bytes?) of memory are used, and which are not. However, the point of a heap is to store data so that the max (or min) key is at the root of the heap. But we might want to allocate memory of different sizes at different times, so it wouldn't make sense to key on the amount of available memory in a particular region of the free store.
    I must assume then that there would be a different heap maintained for each size of memory block that can be allocated, and the key must have something to do with the attractiveness of the particular memory block in the heap (perhaps the lowest address, resulting, hopefully, in growing the free store space less often, leaving more space for the stack to grow, or perhaps keyed based on the fragmentation, to hopefully result in less fragmentation, and therefore more efficient use of the memory space, or perhaps based on page boundaries, keeping as much data in the same page as possible, etc).
    So at this point, I have a few questions I've been unable to resolve completely:
    1. Am I correct that the heap was so named because (perhaps at one point in time), a heap is/was commonly used to track the available memory in the free store?
    2. If so, would it be correct that there would be a heap per standard block size?
    3. Also, at what level of granularity would a heap typically be used (memory page, memory blocks, individual words (4-bytes))?
    4. What would be the most likely property one would use as a key. That is, what makes the root item on the heap ideal?
    5. Would a industrial strength system like the jvm use a (perhaps modified or tuned) heap for this sort of task, or would this typically be too naive for an real world solution today?
    Any insight would be awesome!
    Thanks,
    A.

    jschell wrote:
    I think you are not only mixing terms but domains.
    For starters the OS allocs memory. Applications, regardless of language, request memory from the OS and use it in various ways.
    There are many variations of the term "heap" like the following.
    [http://en.wikipedia.org/wiki/Heap_(data_structure)]
    [http://en.wikipedia.org/wiki/Dynamic_memory_allocation]
    A java VM will request memory from the OS (from a 'heap') and use it in its application 'heap' (C/C++) and then create the Java 'heap'. There can be variations of that along the way that can and likely will include variations of how each heap is used, potentially code that creates its own heap, and potentially other allocators which use something which is not a heap.This last part, I find a bit confusing. By "use something which is not a heap", do you mean the heap data structure, or the dynamic memory pool meaning of heap? If the former, then you would be implying that it would be common for a heap data structure to be used to manage the heap dynamic memory pool. If the latter, what would this "something which is not a heap" be? The best definition of "heap" I've found simply states that it is a pool of memory that can be dynamically allocated. If there is some other way of allocating dynamic memory, then it would suggest that the previous definition of "heap" is incomplete.
    >
    So to terms.
    1. Am I correct that the heap was so named because (perhaps at one point in time), a heap is/was commonly used to track the available memory in the free store?Which 'heap'? The VM one? It is probably named that because the implementors of the Sun VM were familar with how C++ and Smalltalk allocated memory.Okay, but that begs the question, was the heap in C++ and/or Smalltalk so named for the above queried reason?
    >
    2. If so, would it be correct that there would be a heap per standard block size?Not sure what you are referring to but probably a detail of the implementation. And since there are different levels the question doesn't mean much.
    However OS allocations are always by block if that helps. After that it requires making the question much, much more specific.
    3. Also, at what level of granularity would a heap typically be used (memory page, memory blocks, individual words (4-bytes))?Again not specific enough. A typical standard implementation of heap could not be at the word level. And it is unlikely, but not impossible, that variations would support word size allocations.
    The VM heap might use word boundaries (but not size), where the application heap certainly does (word boundary.)My understanding of it is that the application would request blocks from the OS, and then something like malloc would manage the memory within the allocated blocks. malloc (or whatever equivalent Java uses) would have to keep track of the memory it has allocated somehow, and I would think it would have to do this at the word level, since it's most commonly going to allocate memory at the word level to be references to other objects, etc.
    So I guess my question here would really be, if the dynamic memory heap is so named because there has been a memory management strategy that relied upon a heap data structure (which I've found no proof, but have found some suggestive literature), then would that probably have applied at the OS Page Fault level, tracking allocated blocks, or would that have applied at the malloc level, allocating individual words as necessary?
    >
    4. What would be the most likely property one would use as a key. That is, what makes the root item on the heap ideal?"Key" is not a term that will apply in this discussion.
    You appear to be referring to strategies for effective allocation of memory such as allocations from different regions by size comparison.
    It is possible that all levels might use such an allocator. General purpose applications do not sort allocations though (as per your one reference that mentions 'key'.) Sorry, I got the term "key" from an article I read regarding heaps, that indicates that a "key" is used to sort the elements, which I guess would be a more generalized way to make a heap than assuming a natural ordering on the elements in the heap. I'm not sure if the terminology is standard.
    >
    5. Would a industrial strength system like the jvm use a (perhaps modified or tuned) heap for this sort of task, or would this typically be too naive for an real world solution today?Again too indefinite. The Sun VM uses a rather complicated allocator, the model for which originated after years of proceeding research certainly in Smalltalk and in Lisp as well, both commercially and academically.
    I am sure the default is rules driven either explicitly or implicitly as well. So it is self tuning.
    There are command line options that allow you to change how it works as well.I guess perhaps I could attempt to clarify my initial question a bit.
    There is a 1:1 correspondence between the runtime stack, and a stack data structure. That is, when you call a function, it pushes a stack frame onto the runtime stack. When you return from a function, it pops a stack frame from the runtime stack. This is almost certainly the reasons the runtime stack is named as it is.
    The question is, is there or has there ever been a 1:1 correspondence between some aspect of the dynamic memory heap or how it is managed, and a heap data structure? If so, it would explain the name, but I'm a bit puzzled as to how a heap data structure would be of assistance in creating or managing the dynamic memory heap. If not, on the other hand, then does anybody know where the name "heap" came from, as it applies to the dynamic memory pool?
    A.

  • Templates and Dynamic Memory Allocation Templates

    Hi , I was reading a detailed article about templates and I came across the following paragraph
    template<class T, size_t N>
    class Stack
    T data[N]; // Fixed capacity is N
    size_t count;
    public:
    void push(const T& t);
    };"You must provide a compile-time constant value for the parameter N when you request an instance of this template, such as *Stack<int, 100> myFixedStack;*
    Because the value of N is known at compile time, the underlying array (data) can be placed on the run time stack instead of on the free store.
    This can improve runtime performance by avoiding the overhead associated with dynamic memory allocation.
    Now in the above paragraph what does
    "This can improve runtime performance by avoiding the overhead associated with dynamic memory allocation." mean ?? What does template over head mean ??
    I am a bit puzzled and i would really appreciate it if some one could explain to me what this sentence means thanks...

    The run-time memory model of a C or C++ program consists of statically allocated data, automatically allocated data, and dynamically allocated data.
    Data objects (e.g. variables) declared at namespace scope (which includes global scope) are statically allocated. Data objects local to a function that are declared static are also statically allocated. Static allocation means the storage for the data is available when the program is loaded, even before it begins to run. The data remains allocated until after the program exits.
    Data objects local to a function that are not declared static are automatically allocated when the function starts to run. Example:
    int foo() { int i; ... } Variable i does not exist until function foo begins to run, at which time space for it appears automatically. Each new invocation of foo gets its own location for i independent of other invocations of foo. Automatic allocation is usually referred to as stack allocation, since that is the usual implementation method: an area of storage that works like a stack, referenced by a dedicated machine register. Allocating the automatic data consists of adding (or subtracting) a value to the stack register. Popping the stack involves only subtracting (or adding) a value to the stack register. When the function exits, the stack is popped, releasing storage for all its automatic data.
    Dynamically allocated storage is acquired by an explicit use of a new-expression, or a call to an allocation function like malloc(). Example:
    int* ip = new int[100]; // allocate space for 100 integers
    double* id = (double*)malloc(100*sizeof(double)); // allocate space for 100 doublesDynamic storage is not released until you release it explicitly via a delete-expression or a call to free(). Managing the "heap", the area from where dynamic storage is acquired, and to which it is released, can be quite time-consuming.
    Your example of a Stack class (not to be confused with the program stack that is part of the C or C++ implementation) uses a fixed-size (that is, fixed at the point of template instance creation) automatically-allocated array to act as a stack data type. It has the advantage of taking zero time to allocate and release the space for the array. It has the disadvantages of any fixed-size array: it can waste space, or result in a program failure when you try to put N+1 objects into it, and it cannot be re-sized once created.

  • Hyper-V, Dynamic Memory, Memory Demand more than Maximum memory

    Hi,
    I have found that for some VMs Demand Memory more that Maximum Memory set.
    Where can I found any documentation about this?

    A VM can 'demand' more memory than is possible'
    But it will never actually be given more then the maximum.
    In the VM the application is asking for more RAM from the virtual memory manager of the OS in the VM (not to be confused with any memory management that the hypervisor is doing).
    When the VM hits its Maximum setting it is no different than a physical machine hitting its physical RAM cap - the OS begins paging to give the application the additional RAM it is asking for.
    It simply means that you have a greedy application.  SQL and the Exchange mailbox role have a long history of being very greedy applications (SQL made many changes to work in concert with dynamic memory) - taking all of the RAM they could get.
    In your case, you have a startup of 32GB a minimum of 24GB and a Maximum of 36GB.  And the application in the VM would 'like' to have 54GB.  If you look in your VM OS you will probably find that it is paging to disk.
    What is your allocation setting?  The default of 20%  Higher?
    Personally - I see little value in using dynamic memory with such a small window of difference between the startup, minimum, and maximum.  It is as if you are trying to micro manage the feature, instead of allowing the feature to work as designed.
    Dynamic memory is designed to balance RAM between all of the VMs on a particular hypervisor at the same time.  And to allow use of all the RAM the machine has, if there is a workload that can use it.  If the VM wants 64GB, why not give it that. 
    Dynamic memory will not starve other VMs to give this VM more RAM (unless you give this VM a higher priority).  But it will take RAM away from a VM (if absolutely necessary) to allow another VM to meet its requirements.
    Any VM is always guaranteed its startup, at boot time.  And dynamic memory will not starve it below it minimum - and if the VM is not using RAM it will take RAM away down to the minimum.  And a VM will never be given more than the maximum, even
    if the VM wants more.
    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.

  • Allocating dynamic memory

    Hi,
    my second question for today...;-)
    Should I always get dynamic memory via Photoshop API or can I use new[] as well? I understand that photoshop uses own memory management, but I'm talking about relatively small amounts of memory, say some kB.
    If there a rule?
    Greetings Thomas

    For small allocations use new/delete DO NOT use PIUNew to override. This <br />will just give you headaches.<br /><br />For large allocations, image buffers, use the buffer suite.<br /><br /><[email protected]> wrote in message <br />news:[email protected]..<br />> Hi,<br />><br />> my second question for today...;-)<br />><br />> Should I always get dynamic memory via Photoshop API or can I use new[] as <br />> well? I understand that photoshop uses own memory management, but I'm <br />> talking about relatively small amounts of memory, say some kB.<br />> If there a rule?<br />><br />> Greetings Thomas

  • MAC OS X Lion performance problem - broken memory management

    Starting with OS X 10.5 there are evident memory management problems in MAC OS X. The web was already then cluttered with complaints about system slowing down dramatically after some time. Back then i had slower machine, Mac Mini with 1GB RAM, so i (wrongly) concluded that it was due to inferior hardware.
    Now i have 2010 MBP, core i7, 8 GB RAM, dual GPU.
    Mac os X Snow Leopard was pain, but after migrating to OS X Lion, working some serious stuff on MAC started to be a nightmare.
    I finally managed to reproduce the problematic scenario, so i run the test and recorded the screen, into video.
    http://www.youtube.com/watch?v=u5wZwZh61_4
    I run the tar+bzip command, which is basic unix stuff, on the large amount of picture files, in my Pictures/ folder. Just before start, i run the "purge" command, to delete inactive/cached program data.
    You can see on the video that free memory starts to drop very fast, and inactive is constantly rising. If you take a look at "bsdtar" command, it takes only a fragment of RAM, so the problem is not in this process. You cannot say that it is a program memory leak, because then the problem would not be in inactive ram, rather in active/wired.
    When the free memory dropped below 100mb, i started some apps, like Safari, iPhoto and MS Word, and you can see in the video, that it takes even minutes to start an app, when normally (when there is free RAM), it would take some 3-5 secs to load.
    I run the same scenario and the same commands on my Linux Centos 6 box, no problem there ! Memory usage is some 10-20mb, no problems with cache/buffer.
    The memory management must be very broken in Mac OS X !

    Broken?  That's a bit harsh.
    Immature?  That's perhaps a better explanation.
    This paper describes Priority Paging as implemented in Solaris 2.7 back in 1998, and that's essentially what Mac OS X is in need of today:
    The problem is that when pages are needed, no differentiation is made between system file cache pages and application pages, and worse, the file cache can actually steal pages needed by applications.
    Finally when Dynamic Pager starts up and needs to start swapping things out, it's fairly heavy weight in operation, and causes the UI not responding cursor (aka the spinning beach ball) to appear.

  • Use JNI application has memory manager, feasible?

    Hi,
    This is an attempt to circumvent javas poor memory management capabilities. Since java is unable to allocate memory dynamically (*) I'm thinking of writing a native library capable of allocating memory and then allowing java applications to stow away data in this area. The application would be able to access the memory directly through some java.nio.ByteArray-type implementation.
    The java application would be responsible for allocating and deallocating, reading and writing data.
    Is this feasible? Has anyone done this already? I guess a JNI is solution would be pretty straightforward?
    (...this will be my first experience with JNI)
    (*) Xmx option can be used assuming you know beforehand the memory requirements.

    Hi,
    This is feasible since JNative already does this.
    See Pointer and MemoryBlockFactory classes to do this.
    --Marc (http://jnative.sf.net)                                                                                                                                                                                                                                                                                           

  • Dynamic Resource Management

    Hi, all.
    I've implemented a internal service cloud portal for my customer.
    Oracle VM 2.2.2 is the virtualization solution for it.
    The question is implementation of DRS(Dynamic Resource Management). This example is like following:
    1. vMemory balooning
    2. vCPU hotplug
    3. vNetwork limit
    I know that xen has already supported these features and OVM use xen as its hypervisor.
    I'd like to know whether I can implement above thigs via OVM manager API or not?
    Best regards,
    Yeonki

    906943 wrote:
    I know that xen has already supported these features and OVM use xen as its hypervisor.
    I'd like to know whether I can implement above thigs via OVM manager API or not?Yes, Oracle VM supports all three of those things. You can balloon memory, hotplug CPU, disks and NICs. Not sure if the network limiting is live or not though.
    Also, this is not technically DRS: DRS usually refers to live migrating guests between physical machines to achieve balanced load across all nodes. Oracle VM 3.0 supports this style of DRS automatically as a server pool policy.

Maybe you are looking for