Low Latency Garbage Collection

Hi,
What is the best garbage collector configuration for a low latency system?
Multi-threaded server process running on a multi-core box (say 8 cores). Reading and writing messages to the network, no disk. Want all messages to be processed in as timely a manner as possible wrt their individual latencies. Want to avoid some messages being delayed by a garbage collection pause. What will give me the 'fairest' latency time amongst the messages?
I should add, Java SE 1.6 recent version.
Thanks in advance for you suggestions.
Rupert Smith
Edited by: rupertlssmith on Dec 2, 2009 10:17 AM

The simplest approach is don't create any objects in your critical path. (Or keep them to a minimum) Javolution and Joda-time have a libraries which helps.
You can try the concurrent mark sweep or G1 collectors to see if they help.
You can get the GC low enough that you will see other delays such as the SMC or network jitter or memory bank effects instead.
i.e. If your server is simple enough, you can see more jitter from the hardware/OS than Java itself.
Edited by: Peter__Lawrey on 02-Dec-2009 20:45

Similar Messages

  • Improving Latency - Garbage Collection

    Hi there,
    Are there any suggestions as to what you can do to reduce latency in real time applications. Any suggestions as to improving performance times or garbage collection?
    Thanks,

    Hi there,
    Are there any suggestions as to what you can do to
    reduce latency in real time applications. High octane Java!
    I would have suggested you search for a specific forum on this site that features such topics but alas, google is not for you, so....

  • Will system.gc() force garbage collection?

    what impact will the method system.gc() have if i call it in one of the user methods.

    If you use Sun's HotSpot 1.5.0 JVM (or later) with the -XX:+UseConcurrentMarkSweepGC (low-latency) garbage collector, you can use the additional flag -XX:+ExplicitGCInvokesConcurrent to have calls to System.gc() initiate a concurrent collection, rather than cause a stop-the-world compacting collection, which calling System.gc() usually does. Most people who run the low-latency collector are trying to avoid the pauses caused by compacting collections.
    If you have some annoying third-party library that calls System.gc() and that upsets your application's performance, you can use the additional flag -XX:+DisableExplicitGC to make calls to System.gc() into no-ops. That works with whatever collector you are using.

  • Skills/Knowledge needed for low latency/high throughput development

    I have a meeting/technical interview next week for a department at work (foreign exchange). It is going to be a technical interview. I know from the team that they use java heavily, using jvm optimisation, modifications to garbage collection, and have servers placed as close to the exchanges as possible to minimise latency. They said they need to work in "micro seconds" opposed to milliseconds and this means being as efficient as possible.
    I love java development, but am relatively inexperienced but i really want this job. What would you suggest needs to be researched for a role like this in order to stand the best chance of getting it. I dont think knowing about inheritance, auto boxing/unboxing is going to help much in this?
    I am thinking potentially looking at new releases to the java platform such as closures to demonstrate i am keeping up to date with current trends but as for the rest of it i am not really sure.
    I would really appreciate some pointers around considerations for low latency / high volume / highly concurrent development in java if possible.
    Just for a little more detail, the backend uses KDB database with a java fx front end
    Thanks

    ea33334c-b8a8-437b-9807-a170194a1950 wrote:
    it is part of my graduate placement. i have to do a rotation to a new team. i hope you were only so blunt because i fogot to mention this ?
    How is any of what you just said relevant? I was 'so blunt' because you seem to be setting yourself (and your potential new teammates) up for failure. Based on what you posted you are nowhere near qualified for the task you described.
    Further there is absolutely NOTHING in what you posted that talks about any skills that you DO have. You didn't mention A SINGLE THING about your skillset or how you might add value to that team or project.
    Your educational experience should provide some guidelines for how you advance your skills in ANY subject. Each college course has prerequisites and for good reason. Taking a course when you don't have the proper foundation and background is the surest way to fail. Colleges don't let you do it. You have likely been in classes where some of your classmates were clearly in over their head. For those people that course will be nothing but headache and heartache and their chances of success are minimal.
    It is the same with most endeavors including the one you mention in your thread. Naturally you want to challenge yourself when you join a new project or team but you have to be able to hold your own and contribute. Taking on a task or project when you don't have the necessary experience will not only subject you to unnecessary problems but you can easily have a large negative impact on the entire team and project.
    I suggest you try to find a different project where whatever (still unknown to us) skills you have can be used to contribute to the team effort. No one expects new team members to know everything or as much as more experienced developers but they also don't want an 'anchor' that is going to drag them down.

  • Hotspot core dumping during JVM garbage collection ?

    We have an application which calls a 3rd party supplied server API which has recently been upgraded to use Java 1.5
    We are getting the following error reported by our client application. The application is also now running Java 1.5 but references many classes in jar files which would have quite old code in.
    The supplier of the API has stated that the problem requires us to recompile all our jar files using v 1.5 ( including things like jconnect and jms ?!?!? ). This sounds like a bit of a cop-out to me, not to mention being impossible since we don't have the source for things like jconnect.
    I suspect that there is a garbage collection problem at the bottom of all this, but I'm not sure how I can "prove" this, nor do I currently have any real clue as to how to fix any GC problem that may exist.
    The application is supposed to wait for a message on a MQSeries queue and then transforms it via Xalan XSLT and sends it to a Server application. I've tried playing around with heap sizes etc but that just seems to make it worse. If I leave it at the settings that the previous version used then the client at least manages to process a couple of messages before core dumping. There doesn't seem to be a consistent trigger event to cause the core dump ( it's not a message arriving on a queue for example ) but it does seem to be fairly consistent timewise, i.e. after
    Any ideas gratefully accepted.
    Here's a logfile excerpt from my applications showing the Hotspot error message :
    =====================================================================================
    08-Jul-2008 10:01:05 Waiting for messages from COLT.BBFS
    08-Jul-2008 10:02:05 Waiting for messages from COLT.BBFS
    08-Jul-2008 10:03:05 Waiting for messages from COLT.BBFS
    405.815: [GC [PSYoungGen: 17331K->9244K(37632K)] 111702K->103615K(192128K), 0.1615910 secs]
    405.977: [Full GC#
    # An unexpected error has been detected by HotSpot Virtual Machine:
    #  SIGBUS (0xa) at pc=0xfe141348, pid=2600, tid=8
    # Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
    # Problematic frame:
    # V  [libjvm.so+0x141348]
    # An error report file with more information is saved as hs_err_pid2600.log
    # If you would like to submit a bug report, please visit:
    # http://java.sun.com/webapps/bugreport/crash.jsp
    =====================================================================================
    The logfile referred to in the error message contains the following.
    =====================================================================================
    # An unexpected error has been detected by HotSpot Virtual Machine:
    # SIGBUS (0xa) at pc=0xfe141348, pid=2600, tid=8
    # Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
    # Problematic frame:
    # V [libjvm.so+0x141348]
    --------------- T H R E A D ---------------
    Current thread (0x001484d8): VMThread [id=8]
    siginfo:si_signo=10, si_errno=0, si_code=1, si_addr=0x0000080f
    Registers:
    O0=0x00487588 O1=0xfe7d6454 O2=0x000079b0 O3=0x00007800
    O4=0x00008868 O5=0x00147d48 O6=0xf8781460 O7=0xfe0f7938
    G1=0xe52aaae8 G2=0x00000003 G3=0x00000003 G4=0x001484d8
    G5=0xf8781d98 G6=0x00000002 G7=0xf8781d98 Y=0x805683e2
    PC=0xfe141348 nPC=0xfe14134c
    Top of Stack: (sp=0xf8781460)
    0xf8781460: fe786000 00c6ba20 fe10013c e54bab68
    0xf8781470: 0000080f e54bab6c 0000080f 00000004
    0xf8781480: 00487588 00000134 e54bab6c 00000004
    0xf8781490: 00000001 00000000 f87814c0 fe17aef4
    0xf87814a0: fe6485b4 fe7d899c 001484d8 0011da40
    0xf87814b0: 00148988 00148c10 00148d7c f8781880
    0xf87814c0: 007c3389 007c3c0e 00000f87 00008868
    0xf87814d0: 00008800 00487588 fe141310 fe7d6454
    Instructions: (pc=0xfe141348)
    0xfe141338: ec 06 c0 1a 80 a5 a0 00 22 40 00 0a ba 07 60 01
    0xfe141348: f2 05 a0 00 ae 0e 60 03 80 a5 e0 03 22 40 00 05
    Stack: [0xf8702000,0xf8781d98), sp=0xf8781460, free space=509k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    V [libjvm.so+0x141348]
    V [libjvm.so+0x17aefc]
    V [libjvm.so+0x2d557c]
    V [libjvm.so+0x300ef8]
    V [libjvm.so+0x301e84]
    V [libjvm.so+0x2ff950]
    V [libjvm.so+0x29df30]
    V [libjvm.so+0x362b44]
    V [libjvm.so+0x6436f0]
    VM_Operation (0xe03012b0): parallel gc system gc, mode: safepoint, requested by thread 0x0031bca0
    --------------- P R O C E S S ---------------
    Java Threads: ( => current thread )
    0x00b2c028 JavaThread "Thread-4" [_thread_in_native, id=85]
    0x007f5048 JavaThread "Thread-0" [_thread_blocked, id=84]
    0x00c27cf0 JavaThread "Notification Delivery" [_thread_blocked, id=81]
    0x0026fa08 JavaThread "RMI LeaseChecker" daemon [_thread_blocked, id=73]
    0x00821048 JavaThread "RMI RenewClean-[162.11.2.32:44425]" daemon [_thread_blocked, id=70]
    0x0031bca0 JavaThread "GC Daemon" daemon [_thread_blocked, id=67]
    0x00cd5d28 JavaThread "RMI Reaper" [_thread_blocked, id=66]
    0x003c9300 JavaThread "Timer-0" daemon [_thread_blocked, id=65]
    0x00929fe0 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=64]
    0x0089bf18 JavaThread "SeedGenerator Thread" daemon [_thread_blocked, id=42]
    0x00c47248 JavaThread "Pool thread #7" daemon [_thread_blocked, id=38]
    0x00c466a0 JavaThread "Pool thread #6" daemon [_thread_blocked, id=37]
    0x00311850 JavaThread "Pool thread #5" daemon [_thread_blocked, id=36]
    0x00287a40 JavaThread "Pool thread #4" daemon [_thread_blocked, id=35]
    0x00286e98 JavaThread "Pool thread #3" daemon [_thread_blocked, id=34]
    0x00c134b0 JavaThread "Pool thread #2" daemon [_thread_blocked, id=33]
    0x00ad09e0 JavaThread "Pool thread #1" daemon [_thread_blocked, id=32]
    0x00286cd8 JavaThread "PoolThreadManager" daemon [_thread_blocked, id=31]
    0x00c129e0 JavaThread "Channel Reaper" daemon [_thread_blocked, id=30]
    0x00c669e8 JavaThread "ORB Daemon Thread" daemon [_thread_blocked, id=29]
    0x00b10170 JavaThread "Worker for ServerProtocol: (iiop) /0.0.0.0:20168" daemon [_thread_blocked, id=22]
    0x008a17e0 JavaThread "Syn~ Client" daemon [_thread_blocked, id=21]
    0x003dc378 JavaThread "PoolScavenger0" daemon [_thread_blocked, id=20]
    0x0015a928 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=15]
    0x00159880 JavaThread "CompilerThread1" daemon [_thread_blocked, id=14]
    0x00158a18 JavaThread "CompilerThread0" daemon [_thread_blocked, id=13]
    0x00157b98 JavaThread "AdapterThread" daemon [_thread_blocked, id=12]
    0x00156dc8 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=11]
    0x0014ccd8 JavaThread "Finalizer" daemon [_thread_blocked, id=10]
    0x0014ad90 JavaThread "Reference Handler" daemon [_thread_blocked, id=9]
    0x00038238 JavaThread "main" [_thread_in_native, id=1]
    Other Threads:
    =>0x001484d8 VMThread [id=8]
    0x0015c3b0 WatcherThread [id=16]
    VM state:at safepoint (normal execution)
    VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
    [0x00037728/0x00037758] Threads_lock - owner thread: 0x001484d8
    [0x00033650/0x00037ba8] Heap_lock - owner thread: 0x0031bca0
    Heap
    PSYoungGen total 37632K, used 9244K [0xf2eb0000, 0xf7ba0000, 0xf8400000)
    eden space 28352K, 0% used [0xf2eb0000,0xf2eb0000,0xf4a60000)
    from space 9280K, 99% used [0xf4a60000,0xf5367080,0xf5370000)
    to space 25216K, 0% used [0xf6300000,0xf6300000,0xf7ba0000)
    PSOldGen total 154496K, used 94371K [0xe8400000, 0xf1ae0000, 0xf2eb0000)
    object space 154496K, 61% used [0xe8400000,0xee028e78,0xf1ae0000)
    PSPermGen total 35584K, used 18260K [0xe4400000, 0xe66c0000, 0xe8400000)
    object space 35584K, 51% used [0xe4400000,0xe55d5158,0xe66c0000)
    Dynamic libraries:
    0x00010000      /dsdvlp/java/jvm/jdk1.5.0_03/bin/java
    0xff350000      /usr/lib/libthread.so.1
    0xff340000      /usr/lib/libdl.so.1
    0xff200000      /usr/lib/libc.so.1
    0xff390000      /usr/platform/SUNW,Sun-Fire-880/lib/libc_psr.so.1
    0xfe000000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/server/libjvm.so
    0xff1e0000      /usr/lib/libsocket.so.1
    0xff2d0000      /usr/lib/libsched.so.1
    0xff1b0000      /usr/lib/libCrun.so.1
    0xff160000      /usr/lib/libm.so.1
    0xff080000      /usr/lib/libnsl.so.1
    0xff060000      /usr/lib/libmp.so.2
    0xff030000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/native_threads/libhpi.so
    0xfdfc0000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libverify.so
    0xfdf80000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libjava.so
    0xfdf50000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libzip.so
    0xfb7e0000      /usr/lib/locale/en_GB.ISO8859-1/en_GB.ISO8859-1.so.2
    0xe4190000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libnet.so
    0xe3bd0000      /dsdvlp/lib/5/libSolarisNatives.so
    0xe3e90000      /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/librmi.so
    VM Arguments:
    jvm_args: -Djava.ext.dirs=/dsdvlp/java/tmijar/firs7/lib/cli:/dsdvlp/java/tmijar/firs7/lib/cli/ext:/dsdvlp/java/tmijar/firs7/lib/cmn/OpenORB:/dsdvlp/java/tmijar/firs7/lib/cmn/OpenORB/ext:/dsdvlp/java/tmijar/firs7/lib/cmn:/dsdvlp/java/tmijar/firs7/lib/cmn/ext:/dsdvlp/java/tmijar/firs7/daemonlib -Duser.dir=/dsdvlp/java/tmijar/firs7 -Dopenorb.config=file:/dsdvlp/java/tmijar/firs7/configs/OpenORB/config/SynOpenORB.xml -Dopenorb.home=file:/dsdvlp/java/tmijar/firs7/configs/OpenORB -Dcom.coexis.syn.general.orbbinding=com.coexis.syn.general.orbbinding.openorb.OpenORBBinding_1_4 -Dsun.rmi.dgc.client.gcInterval=360000 -Dsun.rmi.dgc.server.gcInterval=360000000 -Xms32m -Xmx256m -Dcom.coexis.syn.clientcommandsconfiglocation=file://localhost//dsdvlp/java/tmijar/firs7/configs/clientcommands.xml -Dcom.coexis.syn.clientconfiglocation=file://localhost//dsdvlp/java/tmijar/firs7/configs/fsbbtd_client.xml -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
    java_command: com.coexis.syn.mqmessaging.daemon.RunDaemon -p /dsdvlp/bin/5/lndsfsd_fsbbtd.properties start
    Environment Variables:
    JAVA_HOME=/dsdvlp/java/jvm/jdk150
    CLASSPATH=.:/dsdvlp/java/jar/jconnect520.jar:/dsdvlp/java/jar/vbjapp340.jar:/dsdvlp/java/jar/vbjorb340.jar:/dsdvlp/java/jar/javax_jndi120.jar
    PATH=/usr/local/etc:/usr/lang:/usr/openwin/bin:/usr/ucb:/bin:/usr/etc:/usr/local/5/bin:/dsdvlp/bin/5:/dsdvlp/bin/4:/home/app/sybase/5/bin:/home/app/sybase/5/localscripts:/home/app/sybase/5/sqr:/home/app/lang:/home/app/lang/SC2.0.1:/usr/ccs/bin:/usr/local/opt/Acrobat3/bin:/dsdvlp/bin:.
    LD_LIBRARY_PATH=/dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/server:/dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc:/dsdvlp/java/jvm/jdk1.5.0_03/jre/../lib/sparc:/usr/lib:/usr/openwin/lib:/usr/local/5/lib:/dsdvlp/lib/5:/dstest/lib/5:/home/app/sybase/5/lib:/dstest/cats/sun4/lib:/tmitest/Opus/opus/lib
    SHELL=/bin/csh
    DISPLAY=CLI00184.mfil.local:1.0
    OS=5
    --------------- S Y S T E M ---------------
    OS: Solaris 8 2/02 s28s_u7wos_08a SPARC
    Copyright 2002 Sun Microsystems, Inc. All Rights Reserved.
    Assembled 18 December 2001
    uname:SunOS 5.8 Generic_117350-20 sun4u (T1 libthread)
    rlimit: STACK 8192k, CORE 9216k, NOFILE 4096, AS infinity
    load average:2.24 2.67 2.68
    CPU:total 4 has_v8, has_v9, has_vis1, has_vis2, is_ultra3
    Memory: 8k page, physical 8388608k(166384k free)
    vm_info: Java HotSpot(TM) Server VM (1.5.0_03-b07) for solaris-sparc, built on Apr 13 2005 03:31:26 by unknown with unknown Workshop:0x550

    The very first suggestion I have is to move your VM to a more recent update of 1.5.0.
    It looks like you are crashing with 5.0u3, and I'm pretty sure 5.0u16 is available. You
    don't want to waste your time chasing a bug that's already been fixed.

  • When will wls try to garbage collect ?

    The docs for the option "Low memory GC threshold" say that this is the "Threshold
    level at which WLS will try to garbage collect once the granularity report has
    been met".
    My question is: When is this condition met ? If at least once there has been a
    granularity report in the log ? Or if the server has been set to Warning state
    The problem Im trying to address is that my "Low memory GC threshold" is set to
    20% and my server has reached only 1% free memory and garbage collection did not
    run, I had to force it via console.
    Thanks in advance, Giselle

    I am wonder when the WLS activate or passivate session and how can I          control it?
              > From documents, sessionWillPassivate in HttpSessionActivationListener will
              be
              > executed when a session is about to passivate on one server (ie. WLS
              instance)
              > and the sessionDidActivate method when the same session has been activated
              on
              > a second server.
              >
              > But I have monitor the behavior for a long time by setting
              PersistentStoreType
              > to file, memory, replicated. sessionDidActivate and sessionWillPassivate
              will
              > never executed. I expected it will passivate and activate when session is
              replicated
              > between servers or persist in files. Any ideas?
              What version of WL? the interface you describe was new in Servlet 2.3, so it
              may not be implemented until 6.1 or even 7.x.
              Peace,
              Cameron Purdy
              Tangosol, Inc.
              http://www.tangosol.com/coherence.jsp
              Tangosol Coherence: Clustered Replicated Cache for Weblogic
              "Karen Law" <[email protected]> wrote in message
              news:3ec86188$[email protected]..
              >
              

  • Manual garbage collection

    Im running a huge combinatorial generation algorithm and i'm starting to run into memory allocation errors on larger problem instances. I am using the -Xmx256m run time option on the JVM to give it more memory but im generating huge masses off stuff here.
    The question I have is, if I know an object isnt going to be used again is it worth destroying it in memory yourself; i assume if you have some undesired object you can do something like this;
    myObject = null;
    and then call the garbage collector to free up the memory..firstly will this work, secondly what is there performance tradeoff for calling the garbage collector more often and lastly is the garbage collector allready efficient enough that doing things like this is probably not worth it.
    Regards,
    Dave

    Setting references to null MAY help get immediate relief, but its not guarunteed to help. Same with the System.gc() call.
    Doing both those thing will not enable the system to operate with less memory than before, as variables that are out of scope will be discovered during garbage collection, and garbage collection WILL run when the heap is fully utilized.
    Doing the above may make the GC pause lower at the expense of worse performance.
    Instead of doing the above, I'd recommand using the Incremental GC by specifying -Xincgc on the command line. This will run GC more often in the background, leading to shorter GC pauses, but about 10% performance hit in general.
    If OTOH you are running into OutOfMemoryErrors and don't think you should be, you probably have a memory leak. Most likely you are allocating objects and storing them somewhere then forgetting about them. For example, if you stick an object in a HashMap, and then forget about it, that object will not be GC'd, unless the HashMap can be GC'd.

  • Reconciling garbage collection, heap overview, and object stats

    First, both the JRockit RuntimeAnalyzer and Console are great tools. We use
    them extensively so thank you.
    I'm trying to reconcile the numbers in these three tabs.
    1. How do I reconcile the Runtime Analyzer and Console output?
    - The Heap overview tab in our application profile shows Heap Overview as
    83% free.
    - However, the Garbage Collection tab of the profile and Console shows the
    application as oscillating between 50 Meg and 200 Meg of heap used. That's
    25% (50Meg/200Meg) to 100%(200Meg/200Meg) used. How do I interpret the 83%
    vs. the 100%?
    I don't believe the 83% free, but I'm skeptical that we consume 150Meg of
    memory in 50 seconds.
    2. How do I reconcile the Object stats with the Garbage collection?
    - Take the top heap user at end of recording, character buffer. It's 22.8%
    of heap using 6,328 KB. If the heap is actually only 34Meg ( 17% of 200Meg.
    I get the 17% from the 83% free), then 22.8% makes sense.
    - So what's in the 200Meg of heap?
    I sent this recording to the JRA team if you want to look at it.
    Thanks
    Jeff

    I've never heard it put that way. Very interesting.
    "Johan Walles" <johan@spamalamadingdong> wrote in message
    news:41bf0be3@mail...
    Note that the time it takes for the garbage collector to do its thing
    grows with the amount of live data.
    What the garbage collector really does is more like retaining the live
    data than disposing of the garbage.
    Regards //Johan
    Jeff wrote:
    Staffan,
    Thanks - you clearly answered my questions. Now the follow-on questions:
    1. Is there a way to get insight into what the 'dead' data is composed
    of?
    2. Is this pattern of consuming 3x the live data in about a minute
    'typical' or a disaster in the making?
    I'm trying to get a sense of what a reasonable target of 'dead' data is.
    The system processes about a meg of data per second, including database
    writes.
    Thanks
    Jeff
    "sla" <[email protected]> wrote in message
    news:33533893.1102952170368.JavaMail.root@jserv5...
    Hi Jeff,
    I'll try to answer the questions.
    1) The Heap overview in the application profile is a snapshot of the heap
    at the end of a garbage collection. At this time only live data is still
    on the heap. So it looks like you have 17% of the heap filled with live
    data (and some overhead as seen in the Heap overview).
    In the garbage collection tab you can see the heap usage oscillating
    between 35-40MB and 200MB. The lower value is right after a garbage
    collection and the higher value is right before a garbage collection. The
    garbage collector clears out about 160MB of "dead" data from the heap.
    This is the amount of temporary objects that you created during the
    garbage collection cycles.
    2) The Object statistics are also taken right after a garbage collection.
    At this time there is 34M of live data on the heap and of these about 22%
    is taken up by character arrays (not unusual).
    At this time the rest of the 200MB heap is empty. It's been cleared of
    all temporary objects and is ready for allocation of new objects.
    I hope this answered your questions.
    Regards,
    /Staffan

  • WLS not garbage collecting enough?

    WLS 5.1 SP5
    SunOS 5.6 (2.6)
    JDK 1.2.2_005a (with the JVMARGS directive set as per BEA's docs. The
    JVMARGS directive solved the SIGBUS 10 for us when requesting the
    AdminMain servlet)
    We tried some stress testing of our web app by running about 100 to 200
    virtual clients requesting the same URL (servlet that forwards to a
    JSP). These virtual clients are actually Java threads running
    simultaneously.
    After each test, we notice that, via WL console, the heap usage of WL
    increased. However, after leaving WL server by itself (no HTTP requests
    coming to it) for a long time (say 30 minutes to 1 hour), we noticed
    that the heap usage has not gone down, except for about 1%. As soon as
    we explicitly tell WLS to garbage collect via the WL console, then the
    heap usage goes down ... sometimes even as low as just 10% when we tell
    it to gc again just after a gc a few seconds earlier.
    Is this proper behaviour by WLS? ... or does this have something to do
    with the JVMARGS directive? Why doesn't it gc even after a long time?
    Thanks in advance,
    John Salvo
    Homepage: http://homepages.tig.com.au/~jmsalvo/

    There are a number of problems with memory usage and garbage collection. Going to WLS 5.1 will fix
    some of them. You should also consider having your application call System.runFinalization(); and
    System.gc() from time to time.
    Mike
    "Jim Zhou" <[email protected]> wrote:
    I see the same behavior in my stress test/pre-production run. I think it's
    normal behavior. The GC will kick in when the heap size gets 100%, you might
    see a pause in your WLS because all threads will stop for the GC ( you
    should see CPU really busy when GC is in ). JDK 1.2 should have better GC
    algorithm than JDK 1.1. Usually for people still using JDK1.1.x, tuning the
    heap size for tolerable GC pause is a pain., the tuning guide want you to
    run multiple WLS instances in a box to stagger the effects of GC pause. If
    you use JDK 1.2, you probably set your heap size comparable to your box's
    memory.
    In one of my stress test, I set heap size to 256m, I see on my console the
    heap gets full every 30 seconds, and it take 1 or 2 seconds to GC. But if
    your heap is 2G, it might take longer.
    WLS 4.51 SP11
    Solaris 2.6
    JDK 1.2.2_06.
    Regards,
    Jim Zhou.
    Jesus M. Salvo Jr. <[email protected]> wrote in message
    news:[email protected]...
    WLS 5.1 SP5
    SunOS 5.6 (2.6)
    JDK 1.2.2_005a (with the JVMARGS directive set as per BEA's docs. The
    JVMARGS directive solved the SIGBUS 10 for us when requesting the
    AdminMain servlet)
    We tried some stress testing of our web app by running about 100 to 200
    virtual clients requesting the same URL (servlet that forwards to a
    JSP). These virtual clients are actually Java threads running
    simultaneously.
    After each test, we notice that, via WL console, the heap usage of WL
    increased. However, after leaving WL server by itself (no HTTP requests
    coming to it) for a long time (say 30 minutes to 1 hour), we noticed
    that the heap usage has not gone down, except for about 1%. As soon as
    we explicitly tell WLS to garbage collect via the WL console, then the
    heap usage goes down ... sometimes even as low as just 10% when we tell
    it to gc again just after a gc a few seconds earlier.
    Is this proper behaviour by WLS? ... or does this have something to do
    with the JVMARGS directive? Why doesn't it gc even after a long time?
    Thanks in advance,
    John Salvo
    Homepage: http://homepages.tig.com.au/~jmsalvo/

  • Garbage Collection Pauses in a client application: Trashing.

    Hi,
    we have put in production a financial trading application written in Java. We are experimenting a strange behaviour of the garbage collector. We are experimenting GC pauses every 4-5 hours which raise CPU time to 100% for 30 seconds. These pauses are not-deterministic. Memory consumption of the entire application is very low (about 30MB).
    This is a strange beheaviour we have detected in the log file of GC:
    5020.027: [GC 27847K->14659K(63936K), 0.0086360 secs]
    5020.036: [Inc GC 14659K->24546K(63936K), 0.0149416 secs]
    5020.107: [GC 27842K->14658K(63936K), 0.0086947 secs]
    5020.116: [Inc GC 14658K->24546K(63936K), 0.0094716 secs]
    5020.181: [GC 27842K->14658K(63936K), 0.0086846 secs]
    5020.190: [Inc GC 14658K->24546K(63936K), 0.0095778 secs]
    5020.255: [GC 27842K->14658K(63936K), 0.0102155 secs]
    5020.266: [Inc GC 14658K->24546K(63936K), 0.0084659 secs]
    5020.335: [GC 27842K->14658K(63936K), 0.0088606 secs]
    5020.344: [Inc GC 14658K->24554K(63936K), 0.0084514 secs]
    as you can see in one second the GC and the IncGC are called many times. WHY?
    We have read articles and tried all various known settings. But we didn't find any solution and at the moment we don't know how to move.
    Thank you for your hint
    Piero

    Which settings have you tried? By the looks of things you've not set any of your JVM memory variables.
    Try bumping up your maximum heap space using -Xmx.
    If you are running on a nice server try using -Xmx256m -Xmx256m -Xmn64m, these should help for a start.
    Also why are you using the "incremental garbage collector" when you have such a tiny application as you can see a full garbage collection is only taking 0.01 seconds.
    Also try reading this http://java.sun.com/docs/hotspot/gc/

  • Revisited: static synchronized vs garbage collected

    A forum thread (http://forum.java.sun.com/thread.jsp?forum=4&thread=466049&start=0&range=15&hilite=false&q=) from last November addressed issues involving the relative merits of using static methods vs. the garbage collection issues that might attend conventional object instantiation. The discussion extended into use of synchronized code and threading, but from it, I've failed to gather an understanding of a few subtleties.
    I understand the dangers of using class-level variables within servlets, however, I'm still puzzled as to the thread-safety of 1) static class-level variables in servlets, and 2) using static methods residing in "utility" type classes from within servlets. I also realize these matters are not confined to servlets, but extend to any multi-threaded environment. I'm particularly unclear as to when, or if, the methods of the static class-level variable/objects need to be synchronized; with the same question applied to the static methods of the "utility" classes.
    From the forum thread mentioned above, it would seem that synchronization is NOT needed since parameters and method-level variables are specific to the executing threads. I believe this essentially means that static methods should not rely on the state of their containing class (under most applications). In a situation where the method's code is not trivial or "lightweight" (e.g. database access), would not the overhead associated with the swapping in and out of thread-specific variables lead to performance issues?
    My particular situation involves an ecommerce site running websphere. We are experiencing sporadic episodes where the VM starts running low on memory, will occasionally recover but will also occasionally crash. In pursuing this wild goose, my first suspicion was a thread deadlocking problem since it's not reliably reproducible. Most of the log entries point to a static method in one of these "utility" classes, yet I can find no non-static class level variables either in the utility or the servlet. However, none of these static methods are synchronized.
    Sorry for the ramble, and thanks for any help.
    Regards,
    Joe

    Because static variables, by their very nature, are only instantiated once, you MUST syncronise them to be safe in a multi threaded environment, otherwise you will get unexpected results.
    As you rightly mention, servlets also optimise the instantiation of class level variables and effectively make them static as well, so they are shared amongst all instaniations of a servlet.

  • All Tweens randomly halting (Not Garbage Collection)

    I am developing a game with some AS3 tweens. I have encountered the problem of garbage collection affecting tweens before so I have all my tweens referenced at the class level now to ensure that isn't a problem. However, randomly all the current tweens in my game will stop. Other, non-tweened, frame by frame animations continue. It's just like the garbage collection problem but the tweens shouldn't be being collected...
    The code I am using for tweens:
    Var declaration at the top of the class:
    private var gateTween:Tween;
    This code in a function:
    gateTween = new Tween(currentGate, "x", None.easeOut, 800, 380, 1.5, true);
    gateTween.addEventListener(TweenEvent.MOTION_FINISH, horseJump);
    Any ideas would be appriciated.

    I have Swing based java project. Im setting all the
    references to null as and when i dont need them, and
    then im calling System.gc() also.
    However it seems the memory is not being released,
    when i check in the task manager, memory usage does
    not reduce. It seems garbage collection is not
    happening.Actually I have a small class - where in I try to do this:-
    public class TestDebug extends Object{
    /** Creates a new instance of TestDebug */
    public TestDebug() {
    * @param args the command line arguments
    public static void main(String[] args) {
    System.out.println("Trying to debug this");
    TestDebug testDebug = new TestDebug();
    testDebug.finalize();
    System.out.println("Lasting print");
    public void finalize(){
    System.out.println("Finalize called");
    System.gc();
    System.out.println("Finalize finished");
    But it is also true that you should never try to force it - It has its own way to do it.
    For Example try to execute a big code(so that you can find JVM doing the GC) as "java -verbose:gc class-name"
    You will find at some points during the execution process - that the JVM actually runs the GC on its own- but when not that is not defined.It is a low priority thread running under JVM.

  • Can I force full garbage collection?

    Hi, my program is memory bound , as users load more files more memory is required , if the user decides to load a very large number of files they will eventually run out of memory. So I am trying to detect when there is less 15% of heap memory left, then force garbage collection and if it cant free up more than 15% of the heap I will stop the user from loading any more files. But the problem is though I call System.gc() to try and force a full garbage collect it rarely
    retrieve enough memory to get below the 15% limit. But using the Yourkit Profiler I can select the Force GarbageCollection option and this always manages to free up more memory to get the figure under the 15% limit. in support of this I found that sometimes my program stop me loading more files when there is still quite a bit available.
    So my questions are.
    1. I know System.gc() is only hint to garbage collect, but the docs imply it only replys after the garbage collection (if any) has been done, is this right or do I have to wait.
    2. Is there any way to Force complete Garbage Collectionas profiler appears to do.
    3. is there a VM option I could set instead to force the JVM to completely garbage collect at say 83% so that if I then polled that 85% of heap was being used I would know that it really was, and I wouldnt need to bother trying to garbage collect further. (Im using Suns 1.6. JVM on Windows and Linux, and Apples 1.5 or 1.6 JVM on Macs)
    public static void checkMemoryWhilstLoadingFiles() throws LowMemoryException
            MemoryUsage mu = ManagementFactory.getMemoryMXBean().getHeapMemoryUsage();
            //No max defined future proofing
            if(mu.getMax()==-1)
                return;
            if (mu.getUsed() > (mu.getMax()  *  0.85f))
                MainWindow.logger.warning("Memory low:" + mu);
                System.gc();
                MainWindow.logger.warning("Memory low gc1:" + ManagementFactory.getMemoryMXBean().getHeapMemoryUsage());
                System.gc();
                MainWindow.logger.warning("Memory low gc2:" + ManagementFactory.getMemoryMXBean().getHeapMemoryUsage());
                System.gc();
                MainWindow.logger.warning("Memory low gc3:" + ManagementFactory.getMemoryMXBean().getHeapMemoryUsage());
                mu = ManagementFactory.getMemoryMXBean().getHeapMemoryUsage();
                if (mu.getUsed()  > (mu.getMax()  *  0.85f))
                    MainWindow.logger.severe("Memory too low:" + mu);
                    throw new LowMemoryException("Running out of memory:"+mu.getUsed());
                else
                     MainWindow.logger.warning("Memory usage reduced to:" + mu);   
        }thanks for any help Paul
    Edited by: paultaylor on 27-Jun-2008 11:10

    On all of the current Sun HotSpot JVM's, calling System.gc() will cause a full compacting collection. Unless you have -XX:+DisableExplicitGC on your command line, in which case the call is a noop. Or if you are running the mostly-concurrent collector (-XX:+UseConcMarkSweepGC) and have the -XX:+ExplicitGCInvokesConcurrent flag on your command line, in which case calling System.gc() will start a concurrent collection (and the calling thread will block until the cycle is finished).
    But calling System.gc() isn't enough to recover all the space that might be recovered. For example, System.gc() will identify objects that are unreferenced but need to have their finalize() methods called before their space becomes available again. So one call to System.gc() won't recover their space. Those finalize() methods need some cycles to run in, so back-to-back (or back-to-back-to-back :-) calls to System.gc() won't help. If you use a lot of finalize() methods, you should leave a lot of time for the finalize() methods to run between the calls to System.gc(). (Better would be to convert your code to use WeakReferences and run your own reference processing queues, and then you could tell when you were done processing references. But that's real work.) Some people try calling System.runFinalization() and wait for that to return, but that has at least two failure modes (details left to the reader).
    In addition, there are details like: if there is still 15% of the heap free, then we won't aggressively clear SoftReferences when you call System.gc(). We might if you waited until the heap was full and we collected it on our own, since we know how much free space there will be after a collection at the point where we are choosing which SoftReferences to clear, and use that to decide how aggressively to clear SoftReferences.
    There is no method to force the collector to do a compacting collection at, say 85% full. There is an option to have the mostly-concurrent collector start a collection cycle that way. But there's no way to find out if a collection cycle is running.
    You are skating on the edge of the qualities of service offered by the different collectors in the various JVM's available. That weakens your ability to "write once, run anywhere".

  • How to encourage jvm to garbage collect?

    Hello,
    I am working with an application that would benefit from more frequent garbage collection. It is running on a hefty machine, with multiple processors and more than 4 gigs of memory available for the JVM. Right now, we are setting the max heap size to 3 gigs.
    The problem is that objects are accumulating, but are not being collected. During a particular operation that I am profiling, over 1 gig of objects are created, but are never collected. If I do a manual garbage collect (using jprofiler), they are all collected. There is no good way to load these objects in another way, or to create fewer objects.
    I have spent a few days playing with -XX:MaxNewSize, -XX:NewSize, -XX:MaxHeapFreeRatio, -XX:TargetSurvivorRatio, and even -XX:+UseParNewGC. Unfortunately, I am unable to encourage the JVM to collect these objects automatically.
    Are there any tricks that I am missing? Does anyone know any good way to encourage the JVM to collect garbage after a certain threshhold in the new generation?
    Thanks,
    --Jeff                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

    You can try this!
    How do I set the JVM's heap size?
    On machines with limited memory (less than 384MB), it is recommended that you set the initial heap size lower than the default. Open the configuration file etc/netbeans.confin a text editor and modify the options in netbeans_default_optionssetting. Decrease the option -J-Xmx128mto -J-Xmx96m and the option -J-XX:MaxPermSize=96mto -J-XX:MaxPermSize=64m. Save the file and restart the IDE.
    Bear in mind that UI responsiveness may be affected when the heap utilization gets close to its limit. Should you encounter an OutOfMemoryError, you need to increase Xmx or XX:MaxPermSize back to the default, or even higher.
    Similarly when running on a machine with more memory it might be useful to increase the maximum size of the heap, especially when working with larger projects. Use the page linked below to get more details on this topic.
    Applies to: NetBeans 4.x, 5.0
    Platforms: All

  • High Eden Java Memory Usage/Garbage Collection

    Hi,
    I am trying to make sure that my Coldfusion Server is optomised to the max and to find out what is normal limits.
    Basically it looks like at times my servers can run slow but it is possible that this is caused by a very old bloated code base.
    Jrun can sometimes have very high CPU usage so I purchased Fusion Reactor to see what is going on under the hood.
    Here are my current Java settings (running v6u24):
    java.args=-server -Xmx4096m -Xms4096m -XX:MaxPermSize=256m -XX:PermSize=256m -Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000 -Dsun.io.useCanonCaches=false -XX:+UseParallelGC -Xbatch ........
    With regards Memory, the only memory that seems to be running a lot of Garbage Collection is the Eden Memory Space. It climbs to nearly 1.2GB in total just under every minute at which time it looks like GC kicks in and the usage drops to about 100MB.
    Survivor memory grows to about 80-100MB over the space of 10 minutes but drops to 0 after the scheduled full GC runs. Old Gen memory fluctuates between 225MB and 350MB with small steps (~50MB) up or down when full GC runs every 10 minutes.
    I had the heap set to 2GB initally in total giving about 600MB to the Eden Space. When I looked at the graphs from Fusion Reactor I could see that there was (minor) Garbage Collection about 2-3 times a minute when the memory usage maxed out the entire 600MB which seemed a high frequency to my untrained eye. I then upped the memory to 4GB in total (~1.2GB auto given to Eden space) to see the difference and saw that GC happened 1-2 times per minute.
    Is it normal in Coldfusion that the Eden memory would grow so quickly and have garbage collection run so often? i.e do these graphs look normal?
    Also should I somehow redistribute the memory available to give the Eden memory more since it seems to be where all the action is?
    Any other advice for performance improvements would be much appreciated.
    Note: These graphs are not from a period where jrun had high CPU.
    Here are the graphs:
    PS Eden Space Graph
    PS Survivor Space Graph
    PS Old Gen Graph
    PS Perm Gen Graph
    Heap Memory Graph
    Heap/Non Heap Memory Graph
    CPU Graph
    Request Average Execution Time Graph
    Request Activity Graph
    Code Cache Graph

    Hi,
    >Is it normal in Coldfusion that the Eden memory would grow so quickly and have garbage collection run so often?
    Yes normal to garbage collect Eden often. That is a minor garbage collection.
    >Also should I somehow redistribute the memory available to give the Eden memory more since it seems to be where all the action is?
    Sometimes it is good to set Eden (Eden and its two Survivor Spaces combined make up New or Young Generation part of JVM heap) to a smaller size. I know your thinking - what make it less, but I want to make it bigger. Give less a try (sometimes less = more, bigger not = better) and monitor the situation. I like to use -Xmn switch, some sources say to use other method/s. Perhaps you could try java.args=-server -Xmx4096m -Xms4096m -Xmn172m etc. I better mention make a backup copy of jvm.config before applying changes. Having said that now you know how you can set the size to bigger if you want.
    I think the JVM is perhaps making some poor decisions with sizing the heap. With Eden growing to 1Gb then being evacuated not many objects are surviving and therefore not being promoted to Old Generation. This ultimately means the object will need to be loaded again latter to Eden rather than being referenced in the Old generation part of the heap. Adds up to poor performance.
    >Any other advice for performance improvements would be much appreciated.
    You are using Parallel garbage collector. Perhaps you could enable that to run multi-threaded reducing the time duration of the garbage collections, jvm args ...-XX:+UseParallelGC -XX:ParallelGCThreads=N etc where N = CPU cores (eg quad core = 4).
    HTH, Carl.

Maybe you are looking for