NativeGetNewTLA OOM on Weblogic 10.3.2

All
Need help in fixing nativeGetNewTLA OOM error on weblogic 10.3.2. This can be reproduced by querying data from Database(Oracle 11).
Below is the stacktrace
Any help is highly appreciated
Thanks
####<Jul 15, 2010 4:11:15 PM EDT> <Error> <Cluster> <test.com> <Server01> <weblogic.cluster.MessageReceiver> <<WLS Kernel>> <> <> <1279224675699> <BEA-000110> <Multicast socket receive error: java.lang.OutOfMemoryError: nativeGetNewTLA
java.lang.OutOfMemoryError: nativeGetNewTLA
        at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45)
        at java.lang.StringBuilder.<init>(StringBuilder.java:68)
        at java.lang.Class.argumentTypesToString(Class.java:2768)
        at java.lang.Class.getDeclaredMethod(Class.java:1937)
        at java.io.ObjectStreamClass.getInheritableMethod(ObjectStreamClass.java:1349)
        at java.io.ObjectStreamClass.access$2200(ObjectStreamClass.java:52)
        at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:448)
        at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
        at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:547)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
        at java.util.ArrayList.readObject(ArrayList.java:593)
        at sun.reflect.GeneratedMethodAccessor2143.invoke(Unknown Source)Edited by: user4029460 on Jul 27, 2010 8:06 AM

Please try to tune the following parameters:
-XXlargeObjectLimit=
-XXtlasize=

Similar Messages

  • OOM on weblogic 10.3.2 JVM@check_alloc

    All
    On Linux 64 bit machine and weblogic 10.3.2 we are getting OOM very frequently.
    I have used MissionControl to see if there is any JavaHeap issue but I could not find any. I notice there is a constant increase in physical memory usage where as the Java Heap usage was increasing and decreasing at regular intervals(GC).
    Thanks in advance for your help.
    Below are the details for hardware and appserver startup params
    uname:
    Linux xxxxxx 2.6.18-164.6.1.el5 #1 SMP Tue Oct 27 11:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
    weblogicStartup:
    /apps/jrockit6-64/bin/java -Xms2048m -Xmx2048m -XXsetGC:genparcon -Xns:1536m -da -Dplatform.home=/apps/bea10.3.2/wlserver_10.3 -Dwls.home=/apps/bea10.3.2/wlserver_10.3/server -Dwli.home=/apps/bea10.3.2/wlserver_10.3/integration -Dweblogic.management.discover=false -Dweblogic.management.server=http://server-prod1:9400 -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/apps/bea10.3.2/patch_wls1032/profiles/default/sysext_manifest_classpath -Dweblogic.Name=Server01 -Djava.security.policy=/apps/bea10.3.2/wlserver_10.3/server/lib/weblogic.policy weblogic.Server
    Exception from log file:
    java.lang.OutOfMemoryError: class allocation, 11055935600 loaded, 4140625920 footprint JVM@check_alloc (src/jvm/model/classload/classalloc.c:118). 2760 byt
    at sun.misc.Unsafe.defineClass(Native Method)
    at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:45)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:381)
    at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:377)
    at sun.reflect.MethodAccessorGenerator.generateMethod(MethodAccessorGenerator.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:28)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at weblogic.spring.monitoring.utils.Delegator.invokeGetBoolean(Delegator.java:21)
    at weblogic.spring.monitoring.utils.AbstractBeanDefinitionDelegator.isPrototype(AbstractBeanDefinitionDelegator.java:59)
    at weblogic.spring.monitoring.actions.CreateBeanElapsedTimeActionState.setAbstractBeanDefinition(CreateBeanElapsedTimeActionState.java:19)
    at weblogic.spring.monitoring.actions.AbstractBeanFactoryCreateBeanAction.setArguments(AbstractBeanFactoryCreateBeanAction.java:36)
    at weblogic.spring.monitoring.actions.BaseElapsedTimeAction.preProcess(BaseElapsedTimeAction.java:58)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:379)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:269)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1010)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:880)
    at com.app.distribution.util.ApplicationContextLoader.getBean(ApplicationContextLoader.java:23)
    at com.app.distribution.ejb.BatchDistributionMDB.onMessage(BatchDistributionMDB.java:49)
    at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
    at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
    at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
    at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
    at com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
    at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
    at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
    at com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
    at $Proxy90.onMessage(Unknown Source)
    at weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:327)
    at weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4585)
    at weblogic.jms.client.JMSSession.execute(JMSSession.java:4271)
    at weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3748)
    at weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
    at weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5096)
    at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

    Henrik
    Thanks very much for your response.
    I tried adding -XVerbose:class,gc and noticed that there are many classes created as follows.
    *[INFO ][class  ] created: #23722 com/beans/WSGResponse$JobInfo$JaxbAccessorF_subjobInfo*
    These classes are used to marshal and unmarshal xml payload from webservice request/response.
    There is a subJobInfo inner class in WSGResponse$JobInfo, but it looks like JAXB is creating subjobInfo(many similar classes are created) class on the fly. If this is a concern is there any way we can add few parameters to garbage collect these created classes.
    Below is the code snippet of the WSGResponse Class. I have removed getters and setters.
    package com.beans;
    import javax.xml.bind.annotation.XmlAccessType;
    import javax.xml.bind.annotation.XmlAccessorType;
    import javax.xml.bind.annotation.XmlElement;
    import javax.xml.bind.annotation.XmlRootElement;
    import javax.xml.bind.annotation.XmlType;
    import com.beans.WSGRequest.OperationAttributes;
    @XmlAccessorType(XmlAccessType.FIELD)
    @XmlType(name = "wsg-response", factoryClass = com.beans.ObjectFactory.class, factoryMethod = "createWSGResponse", propOrder = {
    "statusCode", "statusMessage", "operationAttributes", "jobInfo" })
    @XmlRootElement(name = "wsg-response")
    public class WSGResponse {
    @XmlElement(name = "status-code", required = false)
    protected String statusCode;
    @XmlElement(name = "status-message", required = false)
    protected String statusMessage;
    @XmlElement(name = "operation-attributes", required = false)
    protected OperationAttributes operationAttributes;
    @XmlElement(name = "job-info", required = false)
    protected WSGResponse.JobInfo jobInfo;
    @XmlAccessorType(XmlAccessType.FIELD)
    @XmlType(name = "job-info", factoryClass = com.beans.ObjectFactory.class, factoryMethod = "createJobInfo", propOrder = {
    "jobClientId", "statusCode", "statusMessage", "subjobInfo" })
    public static class JobInfo {
    @XmlElement(name = "job-client-id", required = false)
    protected String jobClientId;
    @XmlElement(name = "status-code", required = false)
    protected String statusCode;
    @XmlElement(name = "status-message", required = false)
    protected String statusMessage;
    @XmlElement(name = "subjob-info", required = false)
    protected SubjobInfo subjobInfo;
    @XmlAccessorType(XmlAccessType.FIELD)
    @XmlType(name = "subjob-info", factoryClass = com.beans.ObjectFactory.class, factoryMethod = "createSubjobInfo", propOrder = {
    "statusCode", "statusMessage", "jobIdentifier", "currentJobState", "destination" })
    public static class SubjobInfo {
    @XmlElement(name = "status-code", required = false)
    protected String statusCode;
    @XmlElement(name = "status-message", required = false)
    protected String statusMessage;
    @XmlElement(name = "job-identifier", required = false)
    protected String jobIdentifier;
    @XmlElement(name = "current-job-state", required = false)
    protected String currentJobState;
    @XmlElement(name = "destination", required = false)
    protected String destination;
    }

  • OOM happens inside Weblogic 10.3.6 when application uploads large files

    Oracle Fusion BI Apps application is uploading large files (100+ MB) onto Oracle Cloud Storage. This application works properly when ran outside weblogic server. When deployed on Fusion Middleware Weblogic 10.3.6, during upload of large files we get this OOM error
    java.lang.OutOfMemoryError: allocLargeObjectOrArray: [B, size 268435472
        at jrockit/vm/Allocator.allocLargeObjectOrArray(JIZ)Ljava/lang/Object;(Native Method)
        at jrockit/vm/Allocator.allocObjectOrArray(Allocator.java:349)[optimized]
        at weblogic/utils/io/UnsyncByteArrayOutputStream.resizeBuffer(UnsyncByteArrayOutputStream.java:59)[inlined]
        at weblogic/utils/io/UnsyncByteArrayOutputStream.write(UnsyncByteArrayOutputStream.java:89)[optimized]
        at com/sun/jersey/api/client/CommittingOutputStream.write(CommittingOutputStream.java:90)
        at com/sun/jersey/core/util/ReaderWriter.writeTo(ReaderWriter.java:115)
        at com/sun/jersey/core/provider/AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:76)
        at com/sun/jersey/core/impl/provider/entity/InputStreamProvider.writeTo(InputStreamProvider.java:98)
        at com/sun/jersey/core/impl/provider/entity/InputStreamProvider.writeTo(InputStreamProvider.java:59)
        at com/sun/jersey/api/client/RequestWriter.writeRequestEntity(RequestWriter.java:300)
        at com/sun/jersey/client/urlconnection/URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:213)
        at com/sun/jersey/client/urlconnection/URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
    Looks like weblogic is using its default Weblogic HTTP handler, switching to Sun HTTP handler via start up JVM/Java Option "-DUseSunHttpHandler=true" solves the OOM issue.
    Seems instead of streaming the file content with some fixed size byte array its being on the whole file into memory during upload.
    Is it possible to solve this OOM by changing any setting of Weblogic HTPP handler without switching to Sun HTTP handler as there are many other application deployed on this weblogic instance?
    We are concerned whether there will be any impact on performance or any other issue.
    Please advice, highly appreciate your response.
    Thanks!

    Hi,
    If you have a back then restore the below file back and then try to start weblogic:
    \Oracle\Middleware\user_projects\domains\<domain_name>\config\config.lok
    Thanks,
    Sharmela

  • Re:configure weblogic so that i get mail whenever i get oom error

    Hi i need to configure the weblogic so that whenever i get the OOM error in weblogic Server i get a mail in my outlook
    can anyone tellme how to do it

    Hi,
    You can use WLDF with E-Mail Notification Feature of WebLogic...Please refer to the below Link:
    http://jaysensharma.wordpress.com/2010/01/07/e-mail-notification-using-wldf/ for step by step procedure.
    Thanks
    Jay Sensharma

  • Weblogic 10.3.5 performance problem

    Hi.
    I'm using a weblogic 10.3.5 as a test server (We deploy there the applications before going into production). The problem is that after a certain amount of deploys, the server becomes slower and slower until its mandatory to reestart because it crashes.
    The error trace is this:
    weblogic.application.ModuleException:
         at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1510)
         at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:482)
         at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
         at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
         at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
         at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200)
         at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
         at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
         at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
         at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
         at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
         at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:636)
         at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
         at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:205)
         at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:58)
         at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
         at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
         at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:569)
         at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150)
         at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116)
         at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:323)
         at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844)
         at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1253)
         at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:440)
         at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163)
         at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
         at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
         at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
         at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
         at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
         at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
    Caused By: java.lang.OutOfMemoryError: GC overhead limit exceeded
         at java.util.Arrays.copyOfRange(Arrays.java:3209)
         at java.lang.String.<init>(String.java:215)
         at weblogic.utils.StringUtils$StringMaker.getString(StringUtils.java:605)
         at weblogic.utils.StringUtils$ReflectedStringMaker.getString(StringUtils.java:615)
         at weblogic.utils.StringUtils.getString(StringUtils.java:600)
         at weblogic.utils.classloaders.AbstractClassFinder.getClassSource(AbstractClassFinder.java:31)
         at weblogic.utils.classloaders.MultiClassFinder.getClassSource(MultiClassFinder.java:58)
         at weblogic.utils.classloaders.MultiClassFinder.getClassSource(MultiClassFinder.java:58)
         at weblogic.application.utils.CompositeWebAppFinder.getClassSource(CompositeWebAppFinder.java:88)
         at weblogic.utils.classloaders.DelegateFinder.getClassSource(DelegateFinder.java:30)
         at weblogic.utils.classloaders.MultiClassFinder.getClassSource(MultiClassFinder.java:58)
         at weblogic.application.utils.CompositeWebAppFinder.getClassSource(CompositeWebAppFinder.java:90)
         at weblogic.utils.classloaders.MultiClassFinder.getClassSource(MultiClassFinder.java:58)
         at weblogic.utils.classloaders.MultiClassFinder.getClassSource(MultiClassFinder.java:58)
         at weblogic.utils.classloaders.CodeGenClassFinder.getClassSource(CodeGenClassFinder.java:25)
         at weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:291)
         at weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:270)
         at weblogic.utils.classloaders.ChangeAwareClassLoader.findClass(ChangeAwareClassLoader.java:64)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
         at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:179)
         at weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:43)
         at com.sun.faces.application.ApplicationFactoryImpl.getApplication(ApplicationFactoryImpl.java:107)
         at com.sun.faces.config.processor.AbstractConfigProcessor.getApplication(AbstractConfigProcessor.java:130)
         at com.sun.faces.config.processor.ApplicationConfigProcessor.process(ApplicationConfigProcessor.java:252)
         at com.sun.faces.config.processor.AbstractConfigProcessor.invokeNext(AbstractConfigProcessor.java:114)
         at com.sun.faces.config.processor.LifecycleConfigProcessor.process(LifecycleConfigProcessor.java:116)
         at com.sun.faces.config.processor.AbstractConfigProcessor.invokeNext(AbstractConfigProcessor.java:114)
         at com.sun.faces.config.processor.FactoryConfigProcessor.process(FactoryConfigProcessor.java:216)
         at com.sun.faces.config.ConfigManager.initialize(ConfigManager.java:338)
         at com.sun.faces.config.ConfigureListener.contextInitialized(ConfigureListener.java:226)
         at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:481)
    We tried with more memory and the same problem occurs. What kind of configuration/settings should we use/change in the server to avoid this problem?
    Thanks.

    "Caused By: java.lang.OutOfMemoryError: GC overhead limit exceeded"
    The reason for this error is explained here: http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#par_gc.oom
    and here: http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#cms.oom
    You could try using -XX:-UseGCOverheadLimit, but that does not make the bad performance go away.
    How many applications are you deploying?
    Some tuning HotSpot tuning examples are presented here: http://middlewaremagic.com/weblogic/?p=7340

  • OOM - Java Heap Space on BI

    Hello All,
    I 'am using BI publisher trial version to enable PDF printing in APEX.
    The BI engine hungs and and runs 100% on CPU and in the logs i do see a OOM - Java Heap Space
    ####<Jan 3, 2014 2:13:46 PM UTC> <Error> <HTTP Session> <ac101.us.oracle.com> <bipserver> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1388758426864> <BEA-100060> <An unexpected error occurred while retrieving the session for Web application: ServletContext@1927403765[app:xmlpserver module:xmlpserver path:/xmlpserver spec-version:2.5].
    java.lang.OutOfMemoryError: Java heap space
    In which file should i change the Xms / Xmx parameters ?
    TIA,
    John

    Hi John,
    Look at this threads
    https://forum.java.sun.com/thread/2566826
    java.lang.OutOfMemoryError: Java heap space -previewing 145MB file in Word
    http://www.technicalconfessions.com/posts.php?post_id=174&title=Starting%20up%20OIM%20managed%20server:%20java.lang.OutOfMemoryError:%20PermGen%20space
    Thanks

  • How to read the memory/memdbg log with regards to an OOM?

    We're trying to migrate from Sun JVM 1.5.0_14 to JRockit R27.5.0 running WebLogic 9.2.2 on Sun Solaris 10 on T2000 boxes (8 cores and 32G of memory). When specifying an pausetime target of 200ms (-server -Xms1g -Xmx1g -XgcPrio:pausetime -Xpausetarget=200ms -XXgcThreads:4 -XXoptThreads:4 -XXnoSystemGC -XXallocPrefetch) we're running in OOMs like
    <17.03.2008 18.17 Uhr CET> <Notice> <Log Management> <BEA-170027> <The server initialized the domain log broadcaster successfully. Log messages will now be broadcasted to the domain log.>
    Exception in thread "Dispatcher-Thread-19" java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object size: 7057136, Num elements: 3528560
         at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:99)
         at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:393)
         at java.lang.StringBuilder.append(StringBuilder.java:120)
         at java.lang.Thread.run(Thread.java:595)
    <17.03.2008 18.17 Uhr CET> <Notice> <Cluster> <BEA-000138> <Listening for announcements from cluster PT_VK_Cluster on 239.255.255.22:7001.> From the corresponding memory/memdbg log:
    [memory ][Mon Mar 17 17:45:30 2008][15174] Running with 32 bit heap and compressed references.
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Memory layout after heap allocation:
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] ' ' - free, '-' - OS reserved range, 'r' - reserved,  'x' - committed,
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] '+' - committed/read, 'e' - committed/executable
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 'J' - java heap, 'j' - java heap data structures
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 00 0.00Gb -jjjjjjjj+jJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 10 0.25Gb JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 20 0.50Gb JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 30 0.75Gb JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 40 1.00Gb JJJJJJJJJJJ                                                    
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 50 1.25Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 60 1.50Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 70 1.75Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 80 2.00Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] 90 2.25Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] a0 2.50Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] b0 2.75Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] c0 3.00Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] d0 3.25Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] e0 3.50Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] f0 3.75Gb                                                                
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Minimum TLA size is 2048 bytes
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Preferred TLA size is 40960 bytes
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Large object limit is 2048 bytes
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Minimal blocksize on the freelist is 2048 bytes
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Initial and maximum number of gc threads: 4, of which 4 parallel threads, 4 concurrent threads, and 4 yc threads.
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Maximum nursery percentage of free heap is: 95.
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Prefetch distance in balanced tree: 4
    [compact][Mon Mar 17 17:45:30 2008][15174] Compactset limit: 299900, Using matrixes: 1, Static: 0
    [memory ][Mon Mar 17 17:45:30 2008][15174] GC mode: Garbage collection optimized for short pausetimes, initial strategy: Concurrent Mark & Sweep
    [memory ][Mon Mar 17 17:45:30 2008][15174] heap size: 1048576K, maximal heap size: 1048576K
    [memory ][Mon Mar 17 17:45:30 2008][15174] < s>-<end>: GC <before>K-><after>K (<heap>K), <pause> ms
    [memory ][Mon Mar 17 17:45:30 2008][15174] <s/start> - start time of collection (seconds since jvm start)
    [memory ][Mon Mar 17 17:45:30 2008][15174] <end>     - end time of collection (seconds since jvm start)
    [memory ][Mon Mar 17 17:45:30 2008][15174] <before>  - memory used by objects before collection (KB)
    [memory ][Mon Mar 17 17:45:30 2008][15174] <after>   - memory used by objects after collection (KB)
    [memory ][Mon Mar 17 17:45:30 2008][15174] <heap>    - size of heap after collection (KB)
    [memory ][Mon Mar 17 17:45:30 2008][15174] <pause>   - total sum of pauses during collection (milliseconds)
    [memory ][Mon Mar 17 17:45:30 2008][15174]             run with -Xverbose:gcpause to see individual pauses
    [memdbg ][Mon Mar 17 17:45:30 2008][15174] Using prefetch linesize: 16 bytes  chunks: 512 bytes pf_dist: 128 bytes
    [gcpause][Mon Mar 17 18:17:38 2008][15174] Thread "Dispatcher-Thread-19" id=79 idx=0x138 tid=78 was in object alloc 3524.102 ms from 1924.890 s
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] GC reason: Large object allocation failed (7057136 bytes), cause: Alloc Queue
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Stopping of javathreads took 19.615 ms
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] old collection 181 started
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Alloc Queue size before GC: 7057136, tlas: 0, oldest: 0
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Compacting 8 heap parts at index 24 (type external) (exceptional false)
    [compact][Mon Mar 17 18:17:38 2008][15174] OC 181: 8 parts (max 512), index 24. Type external, (exceptional false)
    [compact][Mon Mar 17 18:17:38 2008][15174] Area start: 0x5b00000, end: 0x6b00000
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Starting initial marking phase (OC1).
    [nursery][Mon Mar 17 18:17:38 2008][15174] KeepAreaStart: 0x150a5378 KeepAreaEnd: 0x1aeb8918
    [nursery][Mon Mar 17 18:17:38 2008][15174] Young collection started. This YC is a part of OC#181 initial marking.
    [nursery][Mon Mar 17 18:17:38 2008][15174] Setting mmNurseryMarker[0] to 0xf803880
    [nursery][Mon Mar 17 18:17:38 2008][15174] Setting mmNurseryMarker[1] to 0x150a5378
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Initial marking phase promoted 2 objects (0K)
    [gcpause][Mon Mar 17 18:17:38 2008][15174] old collection phase 1 pause time: 124.798376 ms, (start time: 1928.495 s)
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Restarting of javathreads took 0.643 ms
    [memdbg ][Mon Mar 17 18:17:38 2008][15174] Starting concurrent marking phase (OC2).
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Starting precleaning phase (OC3).
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Stopping of javathreads took 19.442 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Starting final marking phase (OC4).
    [nursery][Mon Mar 17 18:17:41 2008][15174] KeepAreaStart: 0x0 KeepAreaEnd: 0x0
    [nursery][Mon Mar 17 18:17:41 2008][15174] Young collection started. This YC is a part of OC#181 final marking.
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Final marking phase promoted 75597 objects (2829K)
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Removing 20 permanent work packets from pool, now 376 packets
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] total concurrent mark time: 3261.573 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] ending marking phase
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] current error: -236.220931
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Requesting to run parallel sweep since Alloc Queue is non-empty.
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Will use parallel sweep instead of concurrent sweep due to request.
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] starting parallel sweeping phase
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1473472 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 318008 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 60896 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 129984 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1230896 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1881216 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 262360 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1293112 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1031088 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 3392344 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 520800 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 94160 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 391624 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 2152 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 13144 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1560688 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 3968 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 280952 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 232952 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 838880 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 142616 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 18656 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 103312 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 35496 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 374760 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 4216 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation added 1081256 bytes to the freelist
    [compact][Mon Mar 17 18:17:41 2008][15174] Evacuation found 9621 objects and moved 9587 objects
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] total sweep time: 57.433 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] ending sweeping phase
    [nursery][Mon Mar 17 18:17:41 2008][15174] Setting mmNurseryMarker[0] to 0xf77a8c8
    [nursery][Mon Mar 17 18:17:41 2008][15174] Setting mmNurseryMarker[1] to 0x1501a790
    [nursery][Mon Mar 17 18:17:41 2008][15174] Nursery size increased from 0kb to 327680kb. Parts: 1177
    [gcpause][Mon Mar 17 18:17:41 2008][15174] Threads waited for memory 3398.495 ms starting at 1928.426 s
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Alloc Queue size after GC: 7057136, tlas: 0, oldest: 1
    [compact][Mon Mar 17 18:17:41 2008][15174] Average compact time ratio: 0.953058
    [compact][Mon Mar 17 18:17:41 2008][15174] Compaction pause: 36.552 (target 127.074), update ref pause: 2.080 (target 6.259)
    [compact][Mon Mar 17 18:17:41 2008][15174] Updated 16423 refs (limit: 54054).
    [compact][Mon Mar 17 18:17:41 2008][15174] Compaction ended at index 31, object end address 0x69f8058.
    [compact][Mon Mar 17 18:17:41 2008][15174] Summary: 181;24;31;8;2;0;36.552;127.074;2.080;6.259;16423;54054
    [compact][Mon Mar 17 18:17:41 2008][15174] External default parts to compact set to 16
    [compact][Mon Mar 17 18:17:41 2008][15174] Compaction too short, increasing compact ratio.
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] gc-trigger is 12.339 %
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Nursery size after OC: 335544320
    [gcpause][Mon Mar 17 18:17:41 2008][15174] old collection phase 4-0 pause time: 250.697868 ms, (start time: 1931.576 s)
    [gcpause][Mon Mar 17 18:17:41 2008][15174] (pause includes compaction: 36.552 ms (external), update ref: 2.080 ms)
    [memory ][Mon Mar 17 18:17:41 2008][15174] 1928.495-1931.826: GC 169061K->164649K (1048576K), sum of pauses 375.496 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Page faults before GC: 4527, page faults after GC: 4527, pages in heap: 131072
    [finaliz][Mon Mar 17 18:17:41 2008][15174] (OC) Pending finalizers 0->2
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Restarting of javathreads took 0.726 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] GC reason: Large object allocation failed (7057136 bytes), cause: Alloc Queue
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Stopping of javathreads took 19.403 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] old collection 182 started
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Alloc Queue size before GC: 7057136, tlas: 0, oldest: 1
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Exceptional compaction at end of heap ordered. Requesting to run parallel sweep.
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Compacting 16 heap parts at index 496 (type external) (exceptional true)
    [compact][Mon Mar 17 18:17:41 2008][15174] OC 182: 16 parts (max 512), index 496. Type external, (exceptional true)
    [compact][Mon Mar 17 18:17:41 2008][15174] Area start: 0x40b00000, end: 0x42b00000
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Starting initial marking phase (OC1).
    [nursery][Mon Mar 17 18:17:41 2008][15174] KeepAreaStart: 0x1501a790 KeepAreaEnd: 0x1aabc2d0
    [nursery][Mon Mar 17 18:17:41 2008][15174] Young collection started. This YC is a part of OC#182 initial marking.
    [nursery][Mon Mar 17 18:17:41 2008][15174] Setting mmNurseryMarker[0] to 0xf77a8c8
    [nursery][Mon Mar 17 18:17:41 2008][15174] Setting mmNurseryMarker[1] to 0x1501a790
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Initial marking phase promoted 2351 objects (94K)
    [gcpause][Mon Mar 17 18:17:41 2008][15174] old collection phase 1 pause time: 126.584439 ms, (start time: 1931.874 s)
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Restarting of javathreads took 0.547 ms
    [memdbg ][Mon Mar 17 18:17:41 2008][15174] Starting concurrent marking phase (OC2).
    [compact][Mon Mar 17 18:17:42 2008][15174] Pointermatrix for thread '(GC Main Thread)' failed to extend beyond 43378 elements.
    [memdbg ][Mon Mar 17 18:17:42 2008][15174] Too many pointers, compaction aborted
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Starting precleaning phase (OC3).
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Stopping of javathreads took 19.425 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Starting final marking phase (OC4).
    [nursery][Mon Mar 17 18:17:44 2008][15174] KeepAreaStart: 0x0 KeepAreaEnd: 0x0
    [nursery][Mon Mar 17 18:17:44 2008][15174] Young collection started. This YC is a part of OC#182 final marking.
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Final marking phase promoted 72379 objects (2561K)
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Removing 19 permanent work packets from pool, now 357 packets
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] total concurrent mark time: 3161.521 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] ending marking phase
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] current error: -175.496244
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Requesting to run parallel sweep since Alloc Queue is non-empty.
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Will use parallel sweep instead of concurrent sweep due to request.
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] starting parallel sweeping phase
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] total sweep time: 18.785 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] ending sweeping phase
    [nursery][Mon Mar 17 18:17:44 2008][15174] Setting mmNurseryMarker[0] to 0xf77aa98
    [nursery][Mon Mar 17 18:17:44 2008][15174] Setting mmNurseryMarker[1] to 0x14e0c8f0
    [nursery][Mon Mar 17 18:17:44 2008][15174] Nursery size increased from 0kb to 327680kb. Parts: 1263
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Alloc Queue size after GC: 7057136, tlas: 0, oldest: 2
    [compact][Mon Mar 17 18:17:44 2008][15174] Compaction pause: 0.000 (target 127.074), update ref pause: 0.000 (target 6.259)
    [compact][Mon Mar 17 18:17:44 2008][15174] Updated 43378 refs (limit: 54054).
    [compact][Mon Mar 17 18:17:44 2008][15174] Compaction ended at index 31, object end address 0x69f8058.
    [compact][Mon Mar 17 18:17:44 2008][15174] Summary: 182;24;31;16;0;1;0.000;127.074;0.000;6.259;43378;54054
    [compact][Mon Mar 17 18:17:44 2008][15174] Exceptional compaction size requested by allocator, not adjusting compact ratio.
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] gc-trigger is 12.339 %
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Nursery size after OC: 335544320
    [gcpause][Mon Mar 17 18:17:44 2008][15174] old collection phase 4-0 pause time: 235.501725 ms, (start time: 1934.832 s)
    [gcpause][Mon Mar 17 18:17:44 2008][15174] (pause includes compaction: 0.000 ms (no compaction), update ref: 0.000 ms)
    [memory ][Mon Mar 17 18:17:44 2008][15174] 1931.874-1935.068: GC 168649K->163056K (1048576K), sum of pauses 362.086 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Page faults before GC: 4527, page faults after GC: 4527, pages in heap: 131072
    [finaliz][Mon Mar 17 18:17:44 2008][15174] (OC) Pending finalizers 0->2
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Restarting of javathreads took 0.667 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] GC reason: Large object allocation failed (7057136 bytes), cause: Alloc Queue
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Stopping of javathreads took 20.308 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] old collection 183 started
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Alloc Queue size before GC: 7057136, tlas: 0, oldest: 2
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Compacting 512 heap parts at index 0 (type internal) (exceptional true)
    [compact][Mon Mar 17 18:17:44 2008][15174] OC 183: 512 parts (max 512), index 0. Type internal, (exceptional true)
    [compact][Mon Mar 17 18:17:44 2008][15174] Area start: 0x2b00000, end: 0x42b00000
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Starting initial marking phase (OC1).
    [nursery][Mon Mar 17 18:17:44 2008][15174] KeepAreaStart: 0x14e0c8f0 KeepAreaEnd: 0x1a85bc48
    [nursery][Mon Mar 17 18:17:44 2008][15174] Young collection started. This YC is a part of OC#183 initial marking.
    [nursery][Mon Mar 17 18:17:44 2008][15174] Setting mmNurseryMarker[0] to 0xf77aa98
    [nursery][Mon Mar 17 18:17:44 2008][15174] Setting mmNurseryMarker[1] to 0x14e0c8f0
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Initial marking phase promoted 632 objects (18K)
    [gcpause][Mon Mar 17 18:17:44 2008][15174] old collection phase 1 pause time: 133.589550 ms, (start time: 1935.151 s)
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Restarting of javathreads took 0.465 ms
    [memdbg ][Mon Mar 17 18:17:44 2008][15174] Starting concurrent marking phase (OC2).
    [memdbg ][Mon Mar 17 18:17:48 2008][15174] Starting precleaning phase (OC3).
    [memdbg ][Mon Mar 17 18:17:48 2008][15174] Stopping of javathreads took 19.590 ms
    [memdbg ][Mon Mar 17 18:17:48 2008][15174] Starting final marking phase (OC4).
    [nursery][Mon Mar 17 18:17:48 2008][15174] KeepAreaStart: 0x0 KeepAreaEnd: 0x0
    [nursery][Mon Mar 17 18:17:48 2008][15174] Young collection started. This YC is a part of OC#183 final marking.
    [compact][Mon Mar 17 18:17:48 2008][15174] Pointermatrix for thread '(GC Worker Thread 3)' failed to extend beyond 25391 elements.
    [memdbg ][Mon Mar 17 18:17:48 2008][15174] Too many pointers, compaction aborted
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Final marking phase promoted 126119 objects (4396K)
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Removing 18 permanent work packets from pool, now 339 packets
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] total concurrent mark time: 4378.189 ms
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] ending marking phase
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] current error: -162.086164
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Requesting to run parallel sweep since Alloc Queue is non-empty.
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Will use parallel sweep instead of concurrent sweep due to request.
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] starting parallel sweeping phase
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] total sweep time: 18.297 ms
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] ending sweeping phase
    [nursery][Mon Mar 17 18:17:49 2008][15174] Setting mmNurseryMarker[0] to 0xf75c4e8
    [nursery][Mon Mar 17 18:17:49 2008][15174] Setting mmNurseryMarker[1] to 0x14dd6788
    [nursery][Mon Mar 17 18:17:49 2008][15174] Nursery size increased from 0kb to 327680kb. Parts: 1307
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Alloc Queue size after GC: 0, tlas: 0, oldest: 0
    [compact][Mon Mar 17 18:17:49 2008][15174] Compaction pause: 0.000 (target 127.074), update ref pause: 0.000 (target 6.259)
    [compact][Mon Mar 17 18:17:49 2008][15174] Updated 55716 refs: 0 inside compaction area, and 55716 outside (limit: 54054).
    [compact][Mon Mar 17 18:17:49 2008][15174] Compaction ended at index 31, object end address 0x69f8058.
    [compact][Mon Mar 17 18:17:49 2008][15174] Summary: 183;24;31;512;0;1;0.000;127.074;0.000;6.259;55716;54054
    [compact][Mon Mar 17 18:17:49 2008][15174] Exceptional compaction size requested by allocator, not adjusting compact ratio.
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] gc-trigger is 12.339 %
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Nursery size after OC: 335544320
    [gcpause][Mon Mar 17 18:17:49 2008][15174] old collection phase 4-0 pause time: 301.755262 ms, (start time: 1939.257 s)
    [gcpause][Mon Mar 17 18:17:49 2008][15174] (pause includes compaction: 0.000 ms (no compaction), update ref: 0.000 ms)
    [memory ][Mon Mar 17 18:17:49 2008][15174] 1935.151-1939.559: GC 226936K->166214K (1048576K), sum of pauses 435.345 ms
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Page faults before GC: 4527, page faults after GC: 4529, pages in heap: 131072
    [finaliz][Mon Mar 17 18:17:49 2008][15174] (OC) Pending finalizers 0->24
    [memdbg ][Mon Mar 17 18:17:49 2008][15174] Restarting of javathreads took 0.928 ms
    [gcpause][Mon Mar 17 18:17:49 2008][15174] Thread "Dispatcher-Thread-19" id=79 idx=0x138 tid=78 was in object alloc 11134.471 ms from 1928.426 s
    [memory ][Mon Mar 17 18:17:49 2008][15174] Throwing OutOfMemory: allocLargeObjectOrArray - Object size: 7057136, Num elements: 3528560Why was the thread "Dispatcher-Thread-19" not able to allocate 7057136 bytes within 11134.471 ms?
    Cheers,
    Torsten

    Is it expected by your application to create StringBuilders with more than 3 million characters in them?Yes. Our application (which runs happily for years now on Sun's JVM) is producing and consuming large data streams (mostly XML). So a StringBuilder producing a string of 3 million characters ist very common. Some XML data streams consumed from database clobs have up to 10 million characters. As I said, with Sun's JVM we have no problems with fragmentation or aborted compactions with a similar sized heap (-server -Xms1g -Xmx1g -Xmn256m -XX:MaxPermSize=256m -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=95 -XX:MaxTenuringThreshold=31 -XX:+UseParNewGC -XX:ParallelGCThreads=4 -XX:-BindGCTaskThreadsToCPUs -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+DisableExplicitGC).
    Cheers,
    Torsten

  • OOME when doing RMI with wlclient.jar

    Hi,
    I'm calling EJB-components that are hosted on a WLS 8.1 server.
    The client uses wlclient.jar from WLS 9 distribution (WebLogic Server 9.2 Fri Jun 23 20:47:26 EDT 2006 783464) and jre 1.5.0_11.
    I get OOME at the client after doing several hundred EJB calls.
    I have checked that the client code does not leak ejb-references, initial contexts etc. I have also tried to call ejb remove and close the initialcontext and start anew but it does not help, the "com.sun.corba.se.impl.orbutil.CacheTable" still becomes the top heap consumer.
    The outline of how the program proceeds is:
    1. create initial context
    2. lookup ejb home
    3. call the create-method of a stateless session bean
    4. call some ejb business methods several hundred times
    -- here is the OOME, most often not during the ejb call, but when memory is examined the RMI-implementation objects are filling the heap
    5. call ejb remove
    6. close initial context
    So that's the problem. How could I get around this?
    I included some HPROF statistics below. HPROF made the stats immidiately after the OOME . All the top rank rows are related to RMI implementation classes, not application classes.
    Regards,
    Kari
    SITES BEGIN (ordered by live bytes) Tue Oct 06 12:02:10 2009
    percent live alloc'ed stack class
    rank self accum bytes objs bytes objs trace name
    1 10.29% 10.29% 3758560 3614 39151840 37646 310238 byte[]
    2 9.31% 19.60% 3400848 45698 3400848 45698 312167 char[]
    3 5.46% 25.06% 1994432 62326 3528256 110258 312197 com.sun.corba.se.impl.orbutil.CacheTable$Entry
    4 3.21% 28.27% 1172952 8747 1172952 8747 312066 char[]
    5 3.00% 31.27% 1096752 45698 1096752 45698 312166 java.lang.String
    6 2.90% 34.17% 1059688 6869 1089912 7103 310593 byte[]
    7 2.30% 36.47% 838592 26206 2827456 88358 312171 com.sun.corba.se.impl.orbutil.CacheTable$Entry
    8 2.19% 38.65% 798200 25112 798200 25112 306612 char[]
    9 2.02% 40.67% 738552 30773 769344 32056 310605 java.util.HashMap$Entry
    10 1.99% 42.66% 727064 6991 3915184 37646 310235 com.sun.corba.se.impl.encoding.CDROutputStream_1_2
    11 1.84% 44.50% 671712 1294 1196448 3818 312195 com.sun.corba.se.impl.orbutil.CacheTable$Entry[]
    12 1.84% 46.34% 671712 1294 1196448 3818 312196 com.sun.corba.se.impl.orbutil.CacheTable$Entry[]
    13 1.65% 48.00% 604472 6869 625064 7103 310578 com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl
    14 1.65% 49.65% 603792 25158 628680 26195 300952 java.util.HashMap$Entry
    15 1.65% 51.30% 602688 25112 602688 25112 306611 java.lang.String
    16 1.50% 52.80% 549520 6869 568240 7103 310599 com.sun.corba.se.impl.protocol.giopmsgheaders.RequestMessage_1_2
    17 1.50% 54.31% 549520 6869 568240 7103 310587 java.util.HashMap$Entry[]
    18 1.44% 55.74% 524352 4 524352 4 302751 java.util.HashMap$Entry[]
    19 1.26% 57.00% 460632 3387 475592 3497 310457 com.sun.corba.se.impl.interceptors.ClientRequestInfoImpl
    20 1.21% 58.21% 441072 9189 2515152 52399 304085 java.nio.HeapByteBuffer
    21 1.05% 59.27% 385000 6875 397768 7103 310405 com.sun.corba.se.impl.legacy.connection.SocketFactoryContactInfoImpl
    22 1.03% 60.29% 375120 15630 375120 15630 313808 java.lang.StackTraceElement
    23 1.00% 61.29% 364728 3507 1939496 18649 309771 com.sun.corba.se.impl.encoding.CDRInputStream_1_2
    24 0.96% 62.25% 349696 3877 5040200 58488 310969 char[]
    25 0.90% 63.15% 329712 6869 340944 7103 310231 com.sun.corba.se.impl.encoding.CDROutputObject
    26 0.77% 63.92% 279904 8747 279904 8747 312070 com.sun.corba.se.impl.orbutil.CacheTable$Entry
    27 0.75% 64.67% 275040 6876 365360 9134 304217 char[]
    28 0.75% 65.42% 274760 6869 284120 7103 310586 java.util.HashMap
    29 0.74% 66.16% 271200 3390 289920 3624 310960 java.util.HashMap$Entry[]
    30 0.74% 66.91% 271200 3390 289920 3624 310941 com.sun.corba.se.impl.protocol.giopmsgheaders.ReplyMessage_1_2
    31 0.74% 67.65% 271120 3389 287360 3592 311344 java.util.HashMap$Entry[]
    32 0.67% 68.32% 243984 10166 257856 10744 310884 java.util.HashMap$Entry
    33 0.67% 68.99% 243984 10166 257856 10744 310879 com.sun.corba.se.spi.servicecontext.UnknownServiceContext
    34 0.65% 69.64% 239272 2583 908696 10308 300090 char[]
    35 0.60% 70.24% 219808 6869 227296 7103 310799 com.sun.corba.se.impl.encoding.BufferManagerWriteStream
    36 0.60% 70.84% 219808 6869 227296 7103 310584 com.sun.corba.se.spi.servicecontext.ServiceContexts
    37 0.57% 71.42% 209928 8747 209928 8747 312065 java.lang.String
    38 0.56% 71.98% 205776 8574 205776 8574 313784 java.lang.StackTraceElement
    39 0.54% 72.52% 196800 5427 427112 11183 310920 byte[]
    40 0.54% 73.06% 196352 1700 444904 4043 310971 char[]
    41 0.47% 73.53% 172056 7169 172056 7169 312484 java.lang.StackTraceElement
    42 0.47% 73.99% 170000 2125 178800 2235 311358 java.util.HashMap$Entry[]

    Andy Piper wrote:
    Configuration:
    =============
    * Windows 2000 Prof SP3
    * Sun JDK 1.4.1_01 (1.4.2)Incidentally you should be using 1.4.1_03 (or later) in the client if
    you are using wlclient.jar.I tried some combinations of WL and client JDKs,
    even that:
    * WL started under Sun JDK 1.4.2
    * client started under Sun JDK 1.4.2
    * all classes (EJBs, value objects, etc)
    compiled under Sun JDK 1.4.2
    >
    >
    FAIL because of java.rmi.MarshalException
    weblogic.iiop.IIOPInputStream.checkChunk(IIOPInputStream.java:449)
    at
    weblogic.iiop.IIOPInputStream.read_longlong(IIOPInputStream.java:1002)There is known bug with reading longs inside a chunked value. This is
    fixed in SP2 and the relevant CR is CR124377 - you can get a patch for
    SP1 from support.It's realy very nice news for us.
    Thank you very match for help and quick response.
    Best regards,
    Eugene Voytitsky

  • How to handle OOME in OSB/ALSB

    In WebLogic the advice on OutOfMemoryError is to restart the server via the panic action in overload configuration.
    However, in OSB an OOME is always caught by an OSB errorhandler, so I think that leaves you with the following options
    1. in all your OSB errorhandlers call System.exit(), not nice since errorhandlers cannot be reused
    2. configure JRockit to exit on OOME
    3. monitor WebLogic/OSB log for OOME and take some action on that
    What do you think?

    1. in all your OSB errorhandlers call System.exit(), not nice since errorhandlers cannot be reused
    Not Recommended, OOM are runtime errors and you cannot predict the when/where they can happen. Why would you want to put system.exit in every error handler. this would be over-kill. Its like putting try-catch block for every piece of java code that is written.
    2. configure JRockit to exit on OOME
    May not be good idea either. Only some sub-system might be effected by OOM, other systems might not be...
    3. monitor WebLogic/OSB log for OOME and take some action on that
    This would be the best way if I have to choose from your given 3 options
    You should not how ever forget that with step 3, we should also spend some (painful!) time to identify the root-cause of the OOM and eliminate either by changing program/logic or upgrading the hardware...That is the only long-term solution.
    Thanks
    Manoj

  • Deployed application is not picked up during weblogic startup

    I'm having this strange problem, my application is deployed with java.Deployer
    tool, works fine, then when ever i reboot the weblogic server, the application
    is not accessible untill i explicitly redeploy it again.
    Please help!

    http://e-docs.bea.com/wls/certifications/certs_810/overview.html
    Looks 141_03. But you have to understand what "certification" means.
    Typically within minor revs of the VM 141_03 and 142 you are ok and there
    should be no problems.
    As for the proving it to you... There were lots of changes and improvemetns
    to the 8.1 code line for deployment. As for knowing what your testing for,
    OOM is a general error with no clear indication of where the problem lies,
    its not clear if it is WLS version or the VM itself. I am sure you know the
    hard answer there, it takes digging in with a profiler to find the culprit.
    You have two options:
    1) Drill down with a profiler in your env and figure out exactly WHAT is the
    cause of the leak in the current environment
    2) switch between different app verndors/versions and VMs.
    Either way you have alot of varialbes to track, take care of and compare.
    There is no silver bullet as always.
    Cheers
    mbg
    "Stephen" <[email protected]> wrote in message
    news:[email protected]...
    >
    BTW, is 8.1 certified with 1.4.2 on Solaris/Sparc yet? Otherwise it is anon-starter
    >
    "Stephen" <[email protected]> wrote:
    SOS, we hear this one all the time, the new and improved version fixes
    the problem.
    Is this a 1.4.2 fix that helps 8.1 or something in the code that is
    different?
    be specific, I have this problem as well and am not willing to give it
    another
    try unless I know what I am testing for.
    "Mark Griffith" <[email protected]> wrote:
    My suggestion would be to try 8.1, I developed many sample applications
    over
    the course of the release and never had to restart the server and could
    continually redeploy. The worst it got was having to completely
    undeploy
    and redeploy the application. (undeploy + redeploy is different that
    just
    redeploy).
    Cheers
    mbg
    "Tom Wilson" <[email protected]> wrote in message
    news:[email protected]...
    This probably doesn't help but....we were doing our initial
    development
    using
    Weblogic 7.0 using 1.3.1_08 but finally we got tired of dealing withthe
    out
    of memory issues resulting from continuously deploying new versionsof JSP
    pages on our system. We found that running into the "Out of Memory"exceptions
    combined with restarting and redeploying the app wasted too much time- an
    average of 7 minutes about 7-8 times a day resulting in about 1 hourworth
    of downtime for each developer using Weblogic. We finally trieddeveloping
    our system with Tomcat and OpenEJB and now we never get "Out of
    Memory"
    exceptions requiring us to restart Tomcat - provided we are onlyworking
    on JSP's.
    Also, we used JDK 1.4.2 with Tomcat which helps get around the memoryleak
    issue
    that results from compiling JSP's using JDK 1.3.1.
    My suggestion...try something else
    "Stephen Westbom" <[email protected]> wrote:
    I have had the same problem with Weblogic 7.0 sp4 using 1.3.1_08.
    I
    found that
    turning off deadspot helps (otherwise known as hotspot).
    It makes it unusable after a while in a production environment.
    Our
    firm is considering
    switching to JBoss because it is simpler and more reliable, despiteno
    good user
    friendly deployment tools.
    Weblogic concentrates too much on bells and whistles that sound good
    rather than
    reliability and usability. I guess existing customers are not asimportant
    as
    new business.
    "Dmitry Amelchenko" <[email protected]> wrote:
    Also, I don't know if that helps, but in general we've been
    experiencing
    some
    other problems with the web logic deployment. For instance, aftera
    number
    of
    redeployments, the application throwth OutOfMemoryException (about
    10
    deployments
    MAX).
    Internally we've been discussing what's the best way of deployingthe
    ear application
    into the weblogic -- using java deploy tool, or using hot deployby
    just
    dropping
    the app in a particular bea folder.
    Is there a prefered/recommended way of deployment?
    If not, what people have found to be the most stable way of
    deploying?
    >>>>>>
    Please help!
    P.S. I'm little dissappointed, isn't bea an industry leader? Isn'tit
    supposed
    to take all these little issues away from the developpers so theycan
    concentrate
    on solving the business problems? It has not been the case for usso
    far. BTW
    -- JBOSS just works.
    "Dmitry" <[email protected]> wrote:
    I'm having this strange problem, my application is deployed with
    java.Deployer
    tool, works fine, then when ever i reboot the weblogic server,
    the
    application
    is not accessible untill i explicitly redeploy it again.
    Please help!

  • Unable to start weblogic with JRockit

    i am using a 32 bit machine to start weblogic server on JRockit JVM. Doing this in order to have already installed Oracle SOA suite and when I start weblogic using JRockit it is Running out of memory. Tried the solutions given in the forum but not able to resolve this issue.
    Also when using jdk getting permgen error. Tried with resetting the maxPermSize value etc in the startweblogic server but not able to do so.
    For JRockit OOM has any one faced a similar issue?
    Can any one help me with same ?

    Thanks Carlos. The value was getting overridden in startweblogic.cmd for Sun jdk but was not sufficient for my problem as I was using Jrockit and with JRockit there is no Permsize issue inturn there was OOM.
    I followed these steps:
    1. changed maxpermsize in startweblogic.
    2. Tried increasing system swap space.
    3. Used a profiler which controls JVM running, tried reduce memory leaks, but none worked.
    I switched to JRockit for weblogic
    1. tried increasing memory etc. Finally I switched to 64 bit machine as suggested in one of the forums for JRockit oom.
    Now both jdk and JRockit weblogic with all SOA components working successfully.
    Thanks,
    Dipti Karnataki

  • WebLogic startup --- java.lang.OutOfMemoryError

    Hello,
    after many Error's i get now this trace.
    Until now, i fixed the MBean error, and the Permgen error. [Link | https://kr.forums.oracle.com/forums/thread.jspa?threadID=2306365 ]
    But now i get this ... and google dont help me with the printed errors.
    Does anyone of you got a hint for me?
    greets Jens
    * To start WebLogic Server, use a username and *
    * password assigned to an admin-level user. For *
    * server administration, use the WebLogic Server *
    * console at http://hostname:port/console *
    starting weblogic with Java version:
    java version "1.7.0_19"
    OpenJDK Runtime Environment (rhel-2.3.9.1.el6_4-x86_64)
    OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
    Starting WLS with line:
    /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/bin/java -Xms512m -Xmx512m -Dweblogic.Name=AdminServer -Djava.security.policy=/opt/oracle/wlserver_12.1/server/lib/weblogic.policy -Djava.endorsed.dirs=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/jre/lib/endorsed:/opt/oracle/wlserver_12.1/endorsed -da -Dplatform.home=/opt/oracle/wlserver_12.1 -Dwls.home=/opt/oracle/wlserver_12.1/server -Dweblogic.home=/opt/oracle/wlserver_12.1/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/opt/oracle/patch_wls1211/profiles/default/sysext_manifest_classpath:/opt/oracle/patch_ocp371/profiles/default/sysext_manifest_classpath weblogic.Server
    <May 27, 2013 2:56:51 PM CEST> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
    <May 27, 2013 2:56:51 PM CEST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
    <May 27, 2013 2:56:51 PM CEST> <Notice> <WebLogicServer> <BEA-000395> <The following extensions directory contents added to the end of the classpath:
    /opt/oracle/user_projects/domains/weblogic/lib/hsql.jar:/opt/oracle/user_projects/domains/weblogic/lib/portal-service.jar:/opt/oracle/user_projects/domains/weblogic/lib/portlet.jar.>
    <May 27, 2013 2:56:52 PM CEST> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with OpenJDK 64-Bit Server VM Version 23.7-b01 from Oracle Corporation.>
    <May 27, 2013 2:56:53 PM CEST> <Info> <Management> <BEA-141107> <Version: WebLogic Server Temporary Patch for 13340309 Thu Feb 16 18:30:21 IST 2012
    WebLogic Server Temporary Patch for 13019800 Mon Jan 16 16:53:54 IST 2012
    WebLogic Server Temporary Patch for BUG13391585 Thu Feb 02 10:18:36 IST 2012
    WebLogic Server Temporary Patch for 13516712 Mon Jan 30 15:09:33 IST 2012
    WebLogic Server Temporary Patch for BUG13641115 Tue Jan 31 11:19:13 IST 2012
    WebLogic Server Temporary Patch for BUG13603813 Wed Feb 15 19:34:13 IST 2012
    WebLogic Server Temporary Patch for 13424251 Mon Jan 30 14:32:34 IST 2012
    WebLogic Server Temporary Patch for 13361720 Mon Jan 30 15:24:05 IST 2012
    WebLogic Server Temporary Patch for BUG13421471 Wed Feb 01 11:24:18 IST 2012
    WebLogic Server Temporary Patch for BUG13657792 Thu Feb 23 12:57:33 IST 2012
    WebLogic Server 12.1.1.0 Wed Dec 7 08:40:57 PST 2011 1445491 >
    <May 27, 2013 2:56:56 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
    <May 27, 2013 2:56:56 PM CEST> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool.>
    <May 27, 2013 2:56:56 PM CEST> <Notice> <LoggingService> <BEA-320400> <The log file /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/logs/AdminServer.log will be rotated. Reopen the log file if tailing has stopped. This can happen on some platforms, such as Windows.>
    <May 27, 2013 2:56:56 PM CEST> <Notice> <LoggingService> <BEA-320401> <The log file has been rotated to /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/logs/AdminServer.log00012. Log messages will continue to be logged in /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/logs/AdminServer.log.>
    <May 27, 2013 2:56:56 PM CEST> <Notice> <Log Management> <BEA-170019> <The server log file /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/logs/AdminServer.log is opened. All server side log events will be written to this file.>
    <May 27, 2013 2:57:01 PM CEST> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
    <May 27, 2013 2:57:05 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY.>
    <May 27, 2013 2:57:05 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
    May 27, 2013 2:57:22 PM com.liferay.portal.kernel.log.Jdk14LogImpl info
    INFO: Detected server weblogic
    Loading zip:/opt/oracle/user_projects/domains/weblogic/servers/AdminServer/tmp/_WL_user/liferay_Deployment/wdm7jy/war/WEB-INF/lib/portal-impl.jar!/system.properties
    May 27, 2013 2:57:23 PM com.liferay.portal.kernel.log.Jdk14LogImpl info
    INFO: Global shared lib directory /opt/oracle/modules/
    May 27, 2013 2:57:23 PM com.liferay.portal.kernel.log.Jdk14LogImpl info
    INFO: Global lib directory /opt/oracle/user_projects/domains/weblogic/lib/
    May 27, 2013 2:57:23 PM com.liferay.portal.kernel.log.Jdk14LogImpl info
    INFO: Portal lib directory /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/tmp/_WL_user/liferay_Deployment/wdm7jy/war/WEB-INF/lib/
    May 27, 2013 2:57:23 PM com.liferay.portal.kernel.log.Jdk14LogImpl info
    INFO: Properties for portal loaded from [file:/opt/oracle/user_projects/domains/portal-ext.properties, zip:/opt/oracle/user_projects/domains/weblogic/servers/AdminServer/tmp/_WL_user/liferay_Deployment/wdm7jy/war/WEB-INF/lib/portal-impl.jar!/portal.properties]
    Loading zip:/opt/oracle/user_projects/domains/weblogic/servers/AdminServer/tmp/_WL_user/liferay_Deployment/wdm7jy/war/WEB-INF/lib/portal-impl.jar!/portal.properties
    Loading file:/opt/oracle/user_projects/domains/portal-ext.properties
    <May 27, 2013 2:57:58 PM CEST> <Warning> <HTTP> <BEA-101342> <liferay_Deployment: Error(s) encountered while precompiling JSP jspURI
    configuration.jsp:17:18: Error in "init.jsp" at line 249: The code of method _jspService(HttpServletRequest, HttpServletResponse) is exceeding the 65535 bytes limit
    <%@ include file="/html/portlet/asset_publisher/init.jsp" %>
    ^--------------------------------------^
    >
    <May 27, 2013 2:57:59 PM CEST> <Warning> <J2EE> <BEA-160140> <Unresolved optional package references (in META-INF/MANIFEST.MF): [Extension-Name: javax.crypto, referenced from: /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/tmp/_WL_user/liferay_Deployment/wdm7jy/war/WEB-INF/lib/_wl_cls_gen.jar]. Ensure that the referenced optional package has been deployed as a library.>
    14:58:20,366 INFO [[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'][DialectDetector:71] Determine dialect for Oracle 11
    14:58:20,556 INFO [[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'][DialectDetector:136] Found dialect org.hibernate.dialect.Oracle10gDialect
    May 27, 2013 2:58:33 PM net.sf.ehcache.Cache initialise
    WARNING: Cache: com.liferay.portal.service.impl.PortletLocalServiceImpl has a maxElementsInMemory of 0. In Ehcache 2.0 this has been changed to mean a store with no capacity limit. Set it to 1 if you want no elements cached in memory
    May 27, 2013 2:59:17 PM org.quartz.impl.StdSchedulerFactory instantiate
    INFO: Using default implementation for ThreadExecutor
    ^X ^X^X^X^XException in thread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception in thread "[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception in thread "[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception in thread "[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'"
    >
    <May 27, 2013 2:57:59 PM CEST> <Warning> <J2EE> <BEA-160140> <Unresolved optional package references (in META-INF/MANIFEST.MF): [Extension-Name: javax.crypto, referenced from: /opt/oracle/user_projects/domains/weblogic/servers/AdminServer/tmp/_WL_user/liferay_Deployment/wdm7jy/war/WEB-INF/lib/_wl_cls_gen.jar]. Ensure that the referenced optional package has been deployed as a library.>
    14:58:20,366 INFO [[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'][DialectDetector:71] Determine dialect for Oracle 11
    14:58:20,556 INFO [[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'][DialectDetector:136] Found dialect org.hibernate.dialect.Oracle10gDialect
    May 27, 2013 2:58:33 PM net.sf.ehcache.Cache initialise
    WARNING: Cache: com.liferay.portal.service.impl.PortletLocalServiceImpl has a maxElementsInMemory of 0. In Ehcache 2.0 this has been changed to mean a store with no capacity limit. Set it to 1 if you want no elements cached in memory
    May 27, 2013 2:59:17 PM org.quartz.impl.StdSchedulerFactory instantiate
    INFO: Using default implementation for ThreadExecutor
    ^X ^X^X^X^XException in thread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception in thread "[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'"
    Edited by: 1008097 on 28.05.2013 00:12

    I think you should consider increasing heap size and check whether you still face the OOM. In addition to that you can consider implementing GC logging using: -verbose:gc -XX:PrintGCTimeStamps and other parameters..
    http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html
    You can consider implementing -XX:HeapDumpOnOutOfMemory or capture heap dump using jmap etc and check out the contents in the heap using MAT to understand the possible suspects.
    https://blogs.oracle.com/alanb/entry/heap_dumps_are_back_with
    The above are the basic stuff that needs to be looked into to understand the cause of this issue.
    Cheers!!
    AJ
    Edited by: AJ on May 28, 2013 4:28 PM

  • Exception in thread "DoSManager" java.lang.OutOfMemoryError: nativeGetNewTLA

    Hi,
    We are facing this 'Exception in thread "DoSManager" java.lang.OutOfMemoryError: nativeGetNewTLA' exception in our production logs. Our JVM as part of WebLogic Server is BEA JRockit(R) Version R27.6.5-32_o-121899-1.6.0_14 and linux-x86_64.
    This is how our JVM is configured:
    a. Stack size(-Xss)
    --This is not configured
    b. Heap size
    ---Xms2048m -Xmx2048m
    c. XXtlasize
    ---This is not configured
    d. XXlargeobjectlimit
    ---This is not configured too
    As I understand if we don't configure above parameters then default values will be considered. Our concern is whenever this exception occurs our application crashes obviously because JVM is not able to execute the code because of insufficient memory. Do we need to tune our JVM or do we need to fix issues in our code? My next task is actually to do the Heap Dump analysis on the Production heap dump which would have been taken when there is an 'OutOfMemoryError' outage for better analysis. For heap dump analysis we will be using MAT.  
    Any suggestions regarding this would be welcomed.

    i m getting Exception in thread "main"
    java.lang.OutOfMemoryError,
    please help meHow?
    You are leaking memory or attempting to use to much. There is a bug in your code.

  • Weblogic crashes with  java.lang.OutOfMemoryError: getNewTla.

    Hi,
    We have a clustered Weblogic environment and a custom WebCenter portal application is deployed on it.
    We are using JDev 11.1.1.5.0 and the Weblogic version we are using is 10.3.5.0
    We are frequently getting below error on either cluster. Can anyone help please?
    ExecuteRequest failed
    java.lang.OutOfMemoryError: getNewTla.
    java.lang.OutOfMemoryError: getNewTla
    at java.util.HashMap.newKeyIterator(HashMap.java:1024)
    at java.util.HashMap$KeySet.iterator(HashMap.java:1062)
    at java.util.HashSet.iterator(HashSet.java:153)
    at weblogic.cluster.messaging.internal.GroupImpl.getConfigInformation(GroupImpl.java:271)
    at weblogic.cluster.messaging.internal.GroupManagerImpl.findOrCreateGroupMember(GroupManagerImpl.java:198)
    at weblogic.cluster.messaging.internal.GroupManagerImpl.handleMessage(GroupManagerImpl.java:173)
    at weblogic.cluster.messaging.internal.ConnectionImpl$1.run(ConnectionImpl.java:139)
    at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
    Regards,
    Kanchan

    Hi Kanchan,
    Few things you could do here is,
    1. Check GC logs to check if Garbage collector selected is able to release un-referenced objects. Logs will give you hint about possible memory leak.
    If you have not set debug GC logs then set it using -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGC -Xloggc:/logs/server1_gclogs.txt
    2. Would prefer to take heap dump on OOM and use any tool like jhat or MAT to analyse. It will help in deciding if you require any performance tuning at JVM level. sometime it happens that heap space allocated is tool low causing OOM.
    For taking heap dump either set it as JAVA option - "*-XX:+HeapDumpOnOutOfMemoryError*"
    or
    using Sun tool "jmap"
    $ jmap –heap:format=b <pid>
    Thanks,
    Ranjan

  • Weblogic/jms/backend/BEMessageReference

    Hi ,
    I have a severe OOM problem in production as my servers are crashing very often.. I have 32 GB unix box and I have put -Xmx 3072 -Xms 3072 as MEM_ARGS for Managed1 and Managed3 ... eventhen I am getting OOM ...
    Can anybody suggest me best MEM_ARGS with -Xx:MaxPermsize for Admin ,Managed1 and Managed3 server instances...
    I have put Heap dump options analyzed the heap dump file showing 63% is weblogic/jms/backend/BEMessageReference.. Can anybody tell me whats the rootCause of this .
    using weblogic 8.1 sp6 and jdk 1.4.2_16 version.
    Thanks
    Sanjana

    Hi,
    What's most likely happening is that the application did not configure JMS quotas and/or flow control, plus its inserting messages faster than they can be processed. It may also be the case that JMS paging hasn't been configured. If so, then an examination of JMS statistics on the WebLogic console should help make the root cause obvious, and, since you're on 8.1, I'd recommend reading the old [ WebLogic JMS Performance Guide |  http://ftpna2.bea.com/pub/downloads/WebLogicJMSPerformanceGuide.pdf ] white-paper for ideas about how to address the issue.
    Tom
    PS. In the future, please post to the JMS newsgroup instead of the General newsgroup.

Maybe you are looking for