Hotspot core dumping during JVM garbage collection ?
We have an application which calls a 3rd party supplied server API which has recently been upgraded to use Java 1.5
We are getting the following error reported by our client application. The application is also now running Java 1.5 but references many classes in jar files which would have quite old code in.
The supplier of the API has stated that the problem requires us to recompile all our jar files using v 1.5 ( including things like jconnect and jms ?!?!? ). This sounds like a bit of a cop-out to me, not to mention being impossible since we don't have the source for things like jconnect.
I suspect that there is a garbage collection problem at the bottom of all this, but I'm not sure how I can "prove" this, nor do I currently have any real clue as to how to fix any GC problem that may exist.
The application is supposed to wait for a message on a MQSeries queue and then transforms it via Xalan XSLT and sends it to a Server application. I've tried playing around with heap sizes etc but that just seems to make it worse. If I leave it at the settings that the previous version used then the client at least manages to process a couple of messages before core dumping. There doesn't seem to be a consistent trigger event to cause the core dump ( it's not a message arriving on a queue for example ) but it does seem to be fairly consistent timewise, i.e. after
Any ideas gratefully accepted.
Here's a logfile excerpt from my applications showing the Hotspot error message :
=====================================================================================
08-Jul-2008 10:01:05 Waiting for messages from COLT.BBFS
08-Jul-2008 10:02:05 Waiting for messages from COLT.BBFS
08-Jul-2008 10:03:05 Waiting for messages from COLT.BBFS
405.815: [GC [PSYoungGen: 17331K->9244K(37632K)] 111702K->103615K(192128K), 0.1615910 secs]
405.977: [Full GC#
# An unexpected error has been detected by HotSpot Virtual Machine:
# SIGBUS (0xa) at pc=0xfe141348, pid=2600, tid=8
# Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
# Problematic frame:
# V [libjvm.so+0x141348]
# An error report file with more information is saved as hs_err_pid2600.log
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
=====================================================================================
The logfile referred to in the error message contains the following.
=====================================================================================
# An unexpected error has been detected by HotSpot Virtual Machine:
# SIGBUS (0xa) at pc=0xfe141348, pid=2600, tid=8
# Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
# Problematic frame:
# V [libjvm.so+0x141348]
--------------- T H R E A D ---------------
Current thread (0x001484d8): VMThread [id=8]
siginfo:si_signo=10, si_errno=0, si_code=1, si_addr=0x0000080f
Registers:
O0=0x00487588 O1=0xfe7d6454 O2=0x000079b0 O3=0x00007800
O4=0x00008868 O5=0x00147d48 O6=0xf8781460 O7=0xfe0f7938
G1=0xe52aaae8 G2=0x00000003 G3=0x00000003 G4=0x001484d8
G5=0xf8781d98 G6=0x00000002 G7=0xf8781d98 Y=0x805683e2
PC=0xfe141348 nPC=0xfe14134c
Top of Stack: (sp=0xf8781460)
0xf8781460: fe786000 00c6ba20 fe10013c e54bab68
0xf8781470: 0000080f e54bab6c 0000080f 00000004
0xf8781480: 00487588 00000134 e54bab6c 00000004
0xf8781490: 00000001 00000000 f87814c0 fe17aef4
0xf87814a0: fe6485b4 fe7d899c 001484d8 0011da40
0xf87814b0: 00148988 00148c10 00148d7c f8781880
0xf87814c0: 007c3389 007c3c0e 00000f87 00008868
0xf87814d0: 00008800 00487588 fe141310 fe7d6454
Instructions: (pc=0xfe141348)
0xfe141338: ec 06 c0 1a 80 a5 a0 00 22 40 00 0a ba 07 60 01
0xfe141348: f2 05 a0 00 ae 0e 60 03 80 a5 e0 03 22 40 00 05
Stack: [0xf8702000,0xf8781d98), sp=0xf8781460, free space=509k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x141348]
V [libjvm.so+0x17aefc]
V [libjvm.so+0x2d557c]
V [libjvm.so+0x300ef8]
V [libjvm.so+0x301e84]
V [libjvm.so+0x2ff950]
V [libjvm.so+0x29df30]
V [libjvm.so+0x362b44]
V [libjvm.so+0x6436f0]
VM_Operation (0xe03012b0): parallel gc system gc, mode: safepoint, requested by thread 0x0031bca0
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00b2c028 JavaThread "Thread-4" [_thread_in_native, id=85]
0x007f5048 JavaThread "Thread-0" [_thread_blocked, id=84]
0x00c27cf0 JavaThread "Notification Delivery" [_thread_blocked, id=81]
0x0026fa08 JavaThread "RMI LeaseChecker" daemon [_thread_blocked, id=73]
0x00821048 JavaThread "RMI RenewClean-[162.11.2.32:44425]" daemon [_thread_blocked, id=70]
0x0031bca0 JavaThread "GC Daemon" daemon [_thread_blocked, id=67]
0x00cd5d28 JavaThread "RMI Reaper" [_thread_blocked, id=66]
0x003c9300 JavaThread "Timer-0" daemon [_thread_blocked, id=65]
0x00929fe0 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=64]
0x0089bf18 JavaThread "SeedGenerator Thread" daemon [_thread_blocked, id=42]
0x00c47248 JavaThread "Pool thread #7" daemon [_thread_blocked, id=38]
0x00c466a0 JavaThread "Pool thread #6" daemon [_thread_blocked, id=37]
0x00311850 JavaThread "Pool thread #5" daemon [_thread_blocked, id=36]
0x00287a40 JavaThread "Pool thread #4" daemon [_thread_blocked, id=35]
0x00286e98 JavaThread "Pool thread #3" daemon [_thread_blocked, id=34]
0x00c134b0 JavaThread "Pool thread #2" daemon [_thread_blocked, id=33]
0x00ad09e0 JavaThread "Pool thread #1" daemon [_thread_blocked, id=32]
0x00286cd8 JavaThread "PoolThreadManager" daemon [_thread_blocked, id=31]
0x00c129e0 JavaThread "Channel Reaper" daemon [_thread_blocked, id=30]
0x00c669e8 JavaThread "ORB Daemon Thread" daemon [_thread_blocked, id=29]
0x00b10170 JavaThread "Worker for ServerProtocol: (iiop) /0.0.0.0:20168" daemon [_thread_blocked, id=22]
0x008a17e0 JavaThread "Syn~ Client" daemon [_thread_blocked, id=21]
0x003dc378 JavaThread "PoolScavenger0" daemon [_thread_blocked, id=20]
0x0015a928 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=15]
0x00159880 JavaThread "CompilerThread1" daemon [_thread_blocked, id=14]
0x00158a18 JavaThread "CompilerThread0" daemon [_thread_blocked, id=13]
0x00157b98 JavaThread "AdapterThread" daemon [_thread_blocked, id=12]
0x00156dc8 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=11]
0x0014ccd8 JavaThread "Finalizer" daemon [_thread_blocked, id=10]
0x0014ad90 JavaThread "Reference Handler" daemon [_thread_blocked, id=9]
0x00038238 JavaThread "main" [_thread_in_native, id=1]
Other Threads:
=>0x001484d8 VMThread [id=8]
0x0015c3b0 WatcherThread [id=16]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x00037728/0x00037758] Threads_lock - owner thread: 0x001484d8
[0x00033650/0x00037ba8] Heap_lock - owner thread: 0x0031bca0
Heap
PSYoungGen total 37632K, used 9244K [0xf2eb0000, 0xf7ba0000, 0xf8400000)
eden space 28352K, 0% used [0xf2eb0000,0xf2eb0000,0xf4a60000)
from space 9280K, 99% used [0xf4a60000,0xf5367080,0xf5370000)
to space 25216K, 0% used [0xf6300000,0xf6300000,0xf7ba0000)
PSOldGen total 154496K, used 94371K [0xe8400000, 0xf1ae0000, 0xf2eb0000)
object space 154496K, 61% used [0xe8400000,0xee028e78,0xf1ae0000)
PSPermGen total 35584K, used 18260K [0xe4400000, 0xe66c0000, 0xe8400000)
object space 35584K, 51% used [0xe4400000,0xe55d5158,0xe66c0000)
Dynamic libraries:
0x00010000 /dsdvlp/java/jvm/jdk1.5.0_03/bin/java
0xff350000 /usr/lib/libthread.so.1
0xff340000 /usr/lib/libdl.so.1
0xff200000 /usr/lib/libc.so.1
0xff390000 /usr/platform/SUNW,Sun-Fire-880/lib/libc_psr.so.1
0xfe000000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/server/libjvm.so
0xff1e0000 /usr/lib/libsocket.so.1
0xff2d0000 /usr/lib/libsched.so.1
0xff1b0000 /usr/lib/libCrun.so.1
0xff160000 /usr/lib/libm.so.1
0xff080000 /usr/lib/libnsl.so.1
0xff060000 /usr/lib/libmp.so.2
0xff030000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/native_threads/libhpi.so
0xfdfc0000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libverify.so
0xfdf80000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libjava.so
0xfdf50000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libzip.so
0xfb7e0000 /usr/lib/locale/en_GB.ISO8859-1/en_GB.ISO8859-1.so.2
0xe4190000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/libnet.so
0xe3bd0000 /dsdvlp/lib/5/libSolarisNatives.so
0xe3e90000 /dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/librmi.so
VM Arguments:
jvm_args: -Djava.ext.dirs=/dsdvlp/java/tmijar/firs7/lib/cli:/dsdvlp/java/tmijar/firs7/lib/cli/ext:/dsdvlp/java/tmijar/firs7/lib/cmn/OpenORB:/dsdvlp/java/tmijar/firs7/lib/cmn/OpenORB/ext:/dsdvlp/java/tmijar/firs7/lib/cmn:/dsdvlp/java/tmijar/firs7/lib/cmn/ext:/dsdvlp/java/tmijar/firs7/daemonlib -Duser.dir=/dsdvlp/java/tmijar/firs7 -Dopenorb.config=file:/dsdvlp/java/tmijar/firs7/configs/OpenORB/config/SynOpenORB.xml -Dopenorb.home=file:/dsdvlp/java/tmijar/firs7/configs/OpenORB -Dcom.coexis.syn.general.orbbinding=com.coexis.syn.general.orbbinding.openorb.OpenORBBinding_1_4 -Dsun.rmi.dgc.client.gcInterval=360000 -Dsun.rmi.dgc.server.gcInterval=360000000 -Xms32m -Xmx256m -Dcom.coexis.syn.clientcommandsconfiglocation=file://localhost//dsdvlp/java/tmijar/firs7/configs/clientcommands.xml -Dcom.coexis.syn.clientconfiglocation=file://localhost//dsdvlp/java/tmijar/firs7/configs/fsbbtd_client.xml -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
java_command: com.coexis.syn.mqmessaging.daemon.RunDaemon -p /dsdvlp/bin/5/lndsfsd_fsbbtd.properties start
Environment Variables:
JAVA_HOME=/dsdvlp/java/jvm/jdk150
CLASSPATH=.:/dsdvlp/java/jar/jconnect520.jar:/dsdvlp/java/jar/vbjapp340.jar:/dsdvlp/java/jar/vbjorb340.jar:/dsdvlp/java/jar/javax_jndi120.jar
PATH=/usr/local/etc:/usr/lang:/usr/openwin/bin:/usr/ucb:/bin:/usr/etc:/usr/local/5/bin:/dsdvlp/bin/5:/dsdvlp/bin/4:/home/app/sybase/5/bin:/home/app/sybase/5/localscripts:/home/app/sybase/5/sqr:/home/app/lang:/home/app/lang/SC2.0.1:/usr/ccs/bin:/usr/local/opt/Acrobat3/bin:/dsdvlp/bin:.
LD_LIBRARY_PATH=/dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc/server:/dsdvlp/java/jvm/jdk1.5.0_03/jre/lib/sparc:/dsdvlp/java/jvm/jdk1.5.0_03/jre/../lib/sparc:/usr/lib:/usr/openwin/lib:/usr/local/5/lib:/dsdvlp/lib/5:/dstest/lib/5:/home/app/sybase/5/lib:/dstest/cats/sun4/lib:/tmitest/Opus/opus/lib
SHELL=/bin/csh
DISPLAY=CLI00184.mfil.local:1.0
OS=5
--------------- S Y S T E M ---------------
OS: Solaris 8 2/02 s28s_u7wos_08a SPARC
Copyright 2002 Sun Microsystems, Inc. All Rights Reserved.
Assembled 18 December 2001
uname:SunOS 5.8 Generic_117350-20 sun4u (T1 libthread)
rlimit: STACK 8192k, CORE 9216k, NOFILE 4096, AS infinity
load average:2.24 2.67 2.68
CPU:total 4 has_v8, has_v9, has_vis1, has_vis2, is_ultra3
Memory: 8k page, physical 8388608k(166384k free)
vm_info: Java HotSpot(TM) Server VM (1.5.0_03-b07) for solaris-sparc, built on Apr 13 2005 03:31:26 by unknown with unknown Workshop:0x550
The very first suggestion I have is to move your VM to a more recent update of 1.5.0.
It looks like you are crashing with 5.0u3, and I'm pretty sure 5.0u16 is available. You
don't want to waste your time chasing a bug that's already been fixed.
Similar Messages
-
Long pauses during ParNew garbage collection Please Help !
Hi,
We are running a server application on an large machine (~120 CPU, ~380 GB Memory).
After running 1 or 2 hours we suddenly get exorbitant application pause times during garbage collection and a massive cpu usage from the java vm
We are running on Java 6 (64Bit) with 6GB Heap.
Concurrent garbage collection is turned on using the parameters:
-XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled
-XX:CMSInitiatingOccupancyFraction=80
-XX:+DisableExplicitGC
We turned on verbose garbage collection and are getting the following output:
1. Normal operation:
Application time: 217.4656792 seconds
3180.905: [GC 3180.906: [ParNew
Desired survivor size 20119552 bytes, new threshold 4 (max 4)
- age 1: 2843824 bytes, 2843824 total
- age 2: 2577128 bytes, 5420952 total
- age 3: 5742024 bytes, 11162976 total
- age 4: 625672 bytes, 11788648 total
: 329531K->15764K(353920K), 0.1484379 secs] 2435799K->2122105K(3392144K), 0.1492386 secs]
Total time for which application threads were stopped: 0.1886810 seconds
2. The Problem:
Application time: 2.8858445 seconds
5008.433: [GC 5008.434: [ParNew
Desired survivor size 20119552 bytes, new threshold 2 (max 4)
- age 1: 15837712 bytes, 15837712 total
- age 2: 12284416 bytes, 28122128 total
: 348338K->39296K(353920K), 138.5317715 secs] 2487779K->2192551K(3392144K), 138.5327383 secs]
Total time for which application threads were stopped: 138.5778558 seconds
Application time: 2.9764564 seconds
5149.957: [GC 5149.957: [ParNew
Desired survivor size 20119552 bytes, new threshold 2 (max 4)
- age 1: 9483176 bytes, 9483176 total
- age 2: 14499344 bytes, 23982520 total
: 353920K->39296K(353920K), 231.5110574 secs] 2507175K->2204546K(3392144K), 231.5121011 secs]
Total time for which application threads were stopped: 231.5257754 seconds
Application time: 2.7932907 seconds
5384.277: [GC 5384.278: [ParNew
Desired survivor size 20119552 bytes, new threshold 4 (max 4)
- age 1: 10756376 bytes, 10756376 total
- age 2: 9135888 bytes, 19892264 total
: 353920K->28449K(353920K), 256.2065591 secs] 2519170K->2207651K(3392144K), 256.2076388 secs]
Total time for which application threads were stopped: 256.2221463 seconds
I can't find any significant differences in the log between fast and long running garbage collections.
I urgently need help in solving this problem !
What can I do ?After the exciting reply I did get on my question, we did some more investigations on the problem and it seems that we finally found the solution to our problem.
The number of garbage collection threads used by the virtual machine defaults to the number of cpus of the machine.
This is ok for small machines or machines where the main load is produced by the java application itself.
In our environment the main load is not produced by the java application but oracle database processes.
When java tries to do it's garbage collection using 120 threads (# CPU) on the machine which is already overloaded by non java processes, the thread synchronization seems to produce an exorbitant overhead.
My theory is that spin locking is used on memory access, causing threads to spin while waiting for other blocking threads not getting cpu because of the heavy load on the system.
The solution is now to limit the number of garbage collection threads.
We did that on the first try by setting -XX:ParallelGCThreads=8
For over one day with heavy load no long GC pauses were experienced any more. -
ORA-07445: exception encountered: core dump during database open
Hi,
after running OracleXE for 1 year on my notebook, I am running into a strange problem. During startup of the database ( opening the DB ) I get the following error:
ORA-07445: exception encountered: core dump [ACCESS_VIOLATION] [__VInfreq__kmmlrl+6329] [PC:0x2D7DF01] [ADDR:0x8] [UNABLE_TO_READ] []
I then tried to reinstall the DB but the problem persists. I have no clues what influences this error.
Any ideas welcomeThe machine is a notebook which is remotely administrated. The admin says there has not been a major software update.
The notebook has 2 GB of main memory and 4 GB of free disk space.
The database was up and running since 1 year. The first time I ran into this problem, was when I dropped a user from database. But that for sure was not the reason, as I totally removed oraclexe and reinstalled it from scratch and the problem persists.
Interestingly the installation doesn't fail, but the final startup, so it might be one of the last packages being installed which produces the error.
I'm looking forward to find some more details. -
11gr2 crsd core dump during failover or start attempt on second node
Hi,
I installed 11gr2 with ASM on one node (solaris SPARC). Then I added another node to this cluster (via addNode.sh script).
Than got strange error: If my first node is up, second node is started fine and run well. If I shutdown first node - crsd on second node dump to core and fails to restart. I get the same error if I try to start second node when the first one is down.
In the crsd.log I see the following:
[ clsdmt][2]Listening to (ADDRESS=(PROTOCOL=ipc)(KEY=mskbkp2DBG_CRSD))
2010-03-03 17:31:35.330: [ clsdmt][2]PID for the Process [18669], connkey 1
2010-03-03 17:31:35.331: [ clsdmt][2]Creating PID [18669] file for home /u01/grid/11.2.0 host mskbkp2 bin crs to /u01/grid/11
.2.0/crs/init/
2010-03-03 17:31:35.331: [ clsdmt][2]Writing PID [18669] to the file [u01/grid/11.2.0/crs/init/mskbkp2.pid]
2010-03-03 17:31:35.925: [ default][1] CRS Daemon Starting
2010-03-03 17:31:35.933: [ default][1] ENV Logging level for Module: AGENT 1
2010-03-03 17:31:35.934: [ default][1] ENV Logging level for Module: AGFW 0
2010-03-03 17:31:35.934: [ default][1] ENV Logging level for Module: CLSFRAME 0
2010-03-03 17:31:35.934: [ default][1] ENV Logging level for Module: CLSVER 0
2010-03-03 17:31:35.934: [ default][1] ENV Logging level for Module: CLUCLS 0
2010-03-03 17:31:35.934: [ default][1] ENV Logging level for Module: COMMCRS 0
2010-03-03 17:31:35.934: [ default][1] ENV Logging level for Module: COMMNS 0
2010-03-03 17:31:35.936: [ default][1] ENV Logging level for Module: CRSAPP 0
2010-03-03 17:31:35.936: [ default][1] ENV Logging level for Module: CRSCCL 0
2010-03-03 17:31:35.936: [ default][1] ENV Logging level for Module: CRSCEVT 0
2010-03-03 17:31:35.936: [ default][1] ENV Logging level for Module: CRSCOMM 1
2010-03-03 17:31:35.936: [ CRSD][1] ENV Debug Level(CRSD): 50
2010-03-03 17:31:35.936: [ CRSD][1] ENV Logging level for Module: CRSD 50
2010-03-03 17:31:35.937: [ CRSD][1] ENV Debug Level(CRSEVT): 0
2010-03-03 17:31:35.937: [ CRSD][1] ENV Logging level for Module: CRSEVT 0
2010-03-03 17:31:35.937: [ CRSD][1] ENV Debug Level(CRSMAIN): 1
2010-03-03 17:31:35.937: [ CRSD][1] ENV Logging level for Module: CRSMAIN 1
2010-03-03 17:31:35.937: [ CRSD][1] ENV Debug Level(CRSOCR): 0
2010-03-03 17:31:35.937: [ CRSD][1] ENV Logging level for Module: CRSOCR 0
2010-03-03 17:31:35.939: [ CRSD][1] ENV Debug Level(CRSPE): 0
2010-03-03 17:31:35.939: [ CRSD][1] ENV Logging level for Module: CRSPE 0
2010-03-03 17:31:35.939: [ CRSD][1] ENV Debug Level(CRSPLACE): 0
2010-03-03 17:31:35.939: [ CRSD][1] ENV Logging level for Module: CRSPLACE 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Debug Level(CRSRES): 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Logging level for Module: CRSRES 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Debug Level(CRSRPT): 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Logging level for Module: CRSRPT 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Debug Level(CRSRTI): 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Logging level for Module: CRSRTI 0
2010-03-03 17:31:35.940: [ CRSD][1] ENV Debug Level(CRSSE): 0
2010-03-03 17:31:35.941: [ CRSD][1] ENV Logging level for Module: CRSSE 0
2010-03-03 17:31:35.941: [ CRSD][1] ENV Debug Level(CRSSEC): 0
2010-03-03 17:31:35.941: [ CRSD][1] ENV Logging level for Module: CRSSEC 0
2010-03-03 17:31:35.941: [ CRSD][1] ENV Debug Level(CRSSHARED): 0
2010-03-03 17:31:35.941: [ CRSD][1] ENV Logging level for Module: CRSSHARED 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Debug Level(CRSTIMER): 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Logging level for Module: CRSTIMER 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Debug Level(CRSUI): 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Logging level for Module: CRSUI 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Debug Level(CSSCLNT): 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Logging level for Module: CSSCLNT 0
2010-03-03 17:31:35.942: [ CRSD][1] ENV Debug Level(OCRAPI): 1
2010-03-03 17:31:35.942: [ CRSD][1] ENV Logging level for Module: OCRAPI 1
2010-03-03 17:31:35.942: [ CRSD][1] ENV Debug Level(OCRASM): 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Logging level for Module: OCRASM 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Debug Level(OCRCAC): 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Logging level for Module: OCRCAC 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Debug Level(OCRCLI): 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Logging level for Module: OCRCLI 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Debug Level(OCRMAS): 1
2010-03-03 17:31:35.943: [ CRSD][1] ENV Logging level for Module: OCRMAS 1
2010-03-03 17:31:35.944: [ CRSD][1] ENV Debug Level(OCRMSG): 1
2010-03-03 17:31:35.944: [ CRSD][1] ENV Logging level for Module: OCRMSG 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Debug Level(OCROSD): 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Logging level for Module: OCROSD 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Debug Level(OCRRAW): 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Logging level for Module: OCRRAW 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Debug Level(OCRSRV): 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Logging level for Module: OCRSRV 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Debug Level(OCRUTL): 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Logging level for Module: OCRUTL 1
2010-03-03 17:31:35.945: [ CRSD][1] ENV Debug Level(SuiteTes): 1
2010-03-03 17:31:35.946: [ CRSD][1] ENV Logging level for Module: SuiteTes 1
2010-03-03 17:31:35.946: [ CRSD][1] ENV Debug Level(UiServer): 0
2010-03-03 17:31:35.946: [ CRSD][1] ENV Logging level for Module: UiServer 0
2010-03-03 17:31:35.946: [ CRSMAIN][1] Checking the OCR device
2010-03-03 17:31:35.948: [ CRSMAIN][1] Connecting to the CSS Daemon
2010-03-03 17:31:35.976: [ CRSMAIN][1] Initializing OCR
2010-03-03 17:31:35.981: [ OCRAPI][1]clsu_get_private_ip_addr: Calling clsu_get_private_ip_addresses to get first private ip
2010-03-03 17:31:35.981: [ OCRAPI][1]Check namebufs
2010-03-03 17:31:35.981: [ OCRAPI][1]Finished checking namebufs
2010-03-03 17:31:35.982: [ GIPC][1] gipcCheckInitialization: possible incompatible non-threaded init from [clsinet.c : 3232
], original from [clsss.c : 5026]
2010-03-03 17:31:36.036: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:405] gpnp tracelevel 3, component tracelevel 0
2010-03-03 17:31:36.037: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:535] '/u01/grid/11.2.0' in effect as GPnP home base.
2010-03-03 17:31:36.059: [ GIPC][1] gipcCheckInitialization: possible incompatible non-threaded init from [clsgpnp0.c : 680
], original from [clsss.c : 5026]
2010-03-03 17:31:36.067: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3867] Init gpnp local security key providers (2)
fatal if both fail
2010-03-03 17:31:36.068: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3870] Init gpnp local security key proveders 1 o
f 2: file wallet (LSKP-FSW)
2010-03-03 17:31:36.068: [ GPnP][1]clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:398] Using FS Wallet Location : /u01/grid/11.2.0/
gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:36.069: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3892] Init gpnp local security key provider 1 of
2: file wallet (LSKP-FSW) OK
2010-03-03 17:31:36.069: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3898] Init gpnp local security key proveders 2 o
f 2: OLR wallet (LSKP-CLSW-OLR)
[ CLWAL][1]clsw_Initialize: OLR initlevel [30000]
2010-03-03 17:31:36.080: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3921] Init gpnp local security key provider 2 of
2: OLR wallet (LSKP-CLSW-OLR) OK
2010-03-03 17:31:36.081: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1952] <Get gpnp security keys (wallet) for id:1,typ;7. (2
providers - fatal if all fail)
2010-03-03 17:31:36.081: [ GPnP][1]clsgpnpkwf_getWalletPath: [at clsgpnpkwf.c:501] req_id=1 ck_prov_id=1 wallet path: /u01/
grid/11.2.0/gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:36.152: [ GPnP][1]clsgpnpwu_walletfopen: [at clsgpnpwu.c:496] Opened SSO wallet: '/u01/grid/11.2.0/gpnp/ms
kbkp2/wallets/peer/cwallet.sso'
2010-03-03 17:31:36.152: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1968] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1
of 2 (LSKP-FSW(1))
2010-03-03 17:31:36.152: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2010-03-03 17:31:36.172: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1952] <Get gpnp security keys (wallet) for id:1,typ;4. (2
providers - fatal if all fail)
2010-03-03 17:31:36.172: [ GPnP][1]clsgpnpkwf_getWalletPath: [at clsgpnpkwf.c:501] req_id=1 ck_prov_id=1 wallet path: /u01/
grid/11.2.0/gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:36.238: [ GPnP][1]clsgpnpwu_walletfopen: [at clsgpnpwu.c:496] Opened SSO wallet: '/u01/grid/11.2.0/gpnp/ms
kbkp2/wallets/peer/cwallet.sso'
2010-03-03 17:31:36.239: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1968] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1
of 2 (LSKP-FSW(1))
2010-03-03 17:31:36.239: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2010-03-03 17:31:36.239: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:840] GPnP client pid=18669, tl=3, f=0
2010-03-03 17:31:36.541: [GIPCXCPT][1] gipcShutdownF: skipping shutdown, count 2, from [ clsinet.c : 1735], ret gipcretSuccess
(0)
2010-03-03 17:31:36.552: [GIPCXCPT][1] gipcShutdownF: skipping shutdown, count 1, from [ clsgpnp0.c : 1021], ret gipcretSucces
s (0)
2010-03-03 17:31:36.771: [ OCRRAW][1]proprioo: for disk 0 (+DR2_BIN), id match (1), total id sets, (1) need recover (0), my v
otes (0), total votes (0), commit_lsn (9), lsn (9)
2010-03-03 17:31:36.771: [ OCRRAW][1]proprioo: my id set: (833490748, 1028247821, 0, 0, 0)
2010-03-03 17:31:36.772: [ OCRRAW][1]proprioo: 1st set: (833490748, 1028247821, 0, 0, 0)
2010-03-03 17:31:36.772: [ OCRRAW][1]proprioo: 2nd set: (0, 0, 0, 0, 0)
2010-03-03 17:31:36.830: [ OCRSRV][1]th_init: Successfully retrieved CSS misscount [31].
2010-03-03 17:31:36.830: [ OCRSRV][1]th_init: Successfully query CLSS mode [3].
[ OCRMAS][20]th_calc_av:5': Rturn persisted AV [186646784] [11.2.0.1.0]
2010-03-03 17:31:36.920: [ OCRSRV][20]th_not_master_change: Master change callback not registered
2010-03-03 17:31:36.920: [ OCRMAS][20]th_master:12: I AM THE NEW OCR MASTER at incar 1. Node Number 2
2010-03-03 17:31:37.134: [ OCRASM][20]proprasmo: ASM cache size is [5MB]
2010-03-03 17:31:37.142: [ OCRASM][20]proprasmo: ASM cache [5MB] enabled for disk group [DR2_BIN].
2010-03-03 17:31:37.155: [ OCRRAW][20]proprioo: for disk 0 (+DR2_BIN), id match (1), total id sets, (1) need recover (0), my
votes (0), total votes (0), commit_lsn (9), lsn (9)
2010-03-03 17:31:37.155: [ OCRRAW][20]proprioo: my id set: (833490748, 1028247821, 0, 0, 0)
2010-03-03 17:31:37.155: [ OCRRAW][20]proprioo: 1st set: (833490748, 1028247821, 0, 0, 0)
2010-03-03 17:31:37.155: [ OCRRAW][20]proprioo: 2nd set: (0, 0, 0, 0, 0)
2010-03-03 17:31:37.214: [ OCRMAS][20]proath_master:18: Spawned connection mgr thread
2010-03-03 17:31:37.214: [ OCRMAS][20]proath_master:20: Spawned upgrade thread
2010-03-03 17:31:37.214: [ OCRMAS][20]th_master:19.1: Wake up upgrade thread
2010-03-03 17:31:37.216: [ OCRSRV][1]th_snap_local_spawn: Inside snap local spawn. host is [mskbkp2]
2010-03-03 17:31:37.219: [ CRSMAIN][1] Running as user: root
2010-03-03 17:31:37.219: [ CRSMAIN][1] CRSD running as the Privileged user
2010-03-03 17:31:37.219: [ CLSVER][1] Static Version 11.2.0.1.0
2010-03-03 17:31:37.226: [ OCRMAS][20]th_master:1': Recvd pubdata event from node [2]
2010-03-03 17:31:37.227: [ OCRMAS][20]th_master:2': Recvd pubdata event for self. Do nothing.
2010-03-03 17:31:37.227: [ CLSVER][1] Daemon version: 11.2.0.1.0 Software version: 11.2.0.1.0
2010-03-03 17:31:37.231: [ CLSVER][1] Active Version from OCR:11.2.0.1.0
2010-03-03 17:31:37.232: [ CLSVER][1] Active Version and Software Version are same
2010-03-03 17:31:37.232: [ CLSVER][1] Active Version changed to 11.2.0.1.0
2010-03-03 17:31:37.232: [ OCRSRV][1]th_reg_master_change: Master change callback registered
2010-03-03 17:31:37.232: [ OCRAPI][1]a_reg_master_change: Registered master change callback
2010-03-03 17:31:37.232: [ OCRSRV][1]th_not_master_change: Invoking master change callback. Master [2] Inc [1]
2010-03-03 17:31:37.232: [ OCRAPI][1]a_reg_master_change: Notified master change
2010-03-03 17:31:37.232: [ CRSMAIN][1] CAA Node Group Pri Data size: 128
2010-03-03 17:31:37.233: [ CRSMAIN][1] CAA Node Group Pub Data size: 128
2010-03-03 17:31:37.247: [ CRSMAIN][1] Getting private data of booted nodes
2010-03-03 17:31:37.247: [ CRSMAIN][1] Checking for booted param on nodenum: 2
2010-03-03 17:31:37.306: [ CLSE][1]clse_get_auth_loc: Returning default authloc: /u01/grid/11.2.0/auth/crs/mskbkp2
2010-03-03 17:31:37.306: [ CRSMAIN][1] Using Authorizer location: /u01/grid/11.2.0/auth/crs/mskbkp2
2010-03-03 17:31:37.314: [ OCRSRV][23]th_upgrade: Starting upgrade calculation
2010-03-03 17:31:37.364: [ CLSCLU][1]clsclu_init: rc 0
2010-03-03 17:31:37.381: [ OCRSRV][23]th_upgrade:10.1 AV [186646784]. State [11]. Already upgraded.Updated global data to the
crs version group. Return [0]
2010-03-03 17:31:37.385: [ CRSMAIN][1] Initializing RTI
2010-03-03 17:31:37.433: [ CRSMAIN][1] Initializing ResouceStateListener
2010-03-03 17:31:37.433: [CRSTIMER][37] Timer Thread Starting.
2010-03-03 17:31:37.433: [ CRSMAIN][1] Initializing EVMMgr
2010-03-03 17:31:37.446: [ CRSMAIN][1] Initializing ResourceMap Map
2010-03-03 17:31:37.461: [ CRSMAIN][1] Subscribing to EVM events for apps
2010-03-03 17:31:37.504: [ CRSMAIN][1] CRSD locked during state recovery, please wait.
2010-03-03 17:31:37.516: [ CRSMAIN][1] CRSD recovered, unlocked.
2010-03-03 17:31:37.525: [ default][1]clsu_get_private_ip_addr: Calling clsu_get_private_ip_addresses to get first private ip
2010-03-03 17:31:37.525: [ default][1]Check namebufs
2010-03-03 17:31:37.525: [ default][1]Finished checking namebufs
2010-03-03 17:31:37.526: [ GIPC][1] gipcCheckInitialization: possible incompatible non-threaded init from [clsinet.c : 3232
], original from [clsss.c : 5026]
2010-03-03 17:31:37.569: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:405] gpnp tracelevel 3, component tracelevel 0
2010-03-03 17:31:37.569: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:535] '/u01/grid/11.2.0' in effect as GPnP home base.
2010-03-03 17:31:37.587: [ GIPC][1] gipcCheckInitialization: possible incompatible non-threaded init from [clsgpnp0.c : 680
], original from [clsss.c : 5026]
2010-03-03 17:31:37.595: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3867] Init gpnp local security key providers (2)
fatal if both fail
2010-03-03 17:31:37.595: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3870] Init gpnp local security key proveders 1 o
f 2: file wallet (LSKP-FSW)
2010-03-03 17:31:37.596: [ GPnP][1]clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:398] Using FS Wallet Location : /u01/grid/11.2.0/
gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:37.596: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3892] Init gpnp local security key provider 1 of
2: file wallet (LSKP-FSW) OK
2010-03-03 17:31:37.596: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3898] Init gpnp local security key proveders 2 o
f 2: OLR wallet (LSKP-CLSW-OLR)
[ CLWAL][1]clsw_Initialize: OLR initlevel [30000]
2010-03-03 17:31:37.607: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3921] Init gpnp local security key provider 2 of
2: OLR wallet (LSKP-CLSW-OLR) OK
2010-03-03 17:31:37.607: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1952] <Get gpnp security keys (wallet) for id:1,typ;7. (2
providers - fatal if all fail)
2010-03-03 17:31:37.607: [ GPnP][1]clsgpnpkwf_getWalletPath: [at clsgpnpkwf.c:501] req_id=1 ck_prov_id=1 wallet path: /u01/
grid/11.2.0/gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:37.673: [ GPnP][1]clsgpnpwu_walletfopen: [at clsgpnpwu.c:496] Opened SSO wallet: '/u01/grid/11.2.0/gpnp/ms
kbkp2/wallets/peer/cwallet.sso'
2010-03-03 17:31:37.673: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1968] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1
of 2 (LSKP-FSW(1))
2010-03-03 17:31:37.673: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2010-03-03 17:31:37.690: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1952] <Get gpnp security keys (wallet) for id:1,typ;4. (2
providers - fatal if all fail)
2010-03-03 17:31:37.690: [ GPnP][1]clsgpnpkwf_getWalletPath: [at clsgpnpkwf.c:501] req_id=1 ck_prov_id=1 wallet path: /u01/
grid/11.2.0/gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:37.754: [ GPnP][1]clsgpnpwu_walletfopen: [at clsgpnpwu.c:496] Opened SSO wallet: '/u01/grid/11.2.0/gpnp/ms
kbkp2/wallets/peer/cwallet.sso'
2010-03-03 17:31:37.754: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1968] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1
of 2 (LSKP-FSW(1))
2010-03-03 17:31:37.754: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2010-03-03 17:31:37.755: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:840] GPnP client pid=18669, tl=3, f=0
2010-03-03 17:31:37.806: [GIPCXCPT][1] gipcShutdownF: skipping shutdown, count 2, from [ clsinet.c : 1735], ret gipcretSuccess
(0)
2010-03-03 17:31:37.817: [GIPCXCPT][1] gipcShutdownF: skipping shutdown, count 1, from [ clsgpnp0.c : 1021], ret gipcretSucces
s (0)
2010-03-03 17:31:37.822: [ CRSMAIN][1] CRSD listening on 10 style E2E port (ADDRESS=(PROTOCOL=tcp)(HOST=172.31.25.112)(PORT=38
983))
2010-03-03 17:31:37.835: [ CRSMAIN][1] Starting Threads
2010-03-03 17:31:37.858: [ CLSE][1]clse_get_auth_loc: Returning default authloc: /u01/grid/11.2.0/auth/crs/mskbkp2
2010-03-03 17:31:37.858: [ CRSD][1] AuthLoc /u01/grid/11.2.0/auth/crs/mskbkp2
2010-03-03 17:31:37.859: [ CRSD][1] PE active version: 11.2.0.1.0
2010-03-03 17:31:37.859: [ CRSD][1] PE Engine: NEW
2010-03-03 17:31:37.859: [ CRSD][1] Using OCR batch ops : ENABLED
2010-03-03 17:31:37.860: [ CRSMAIN][1] Initializing Node Down Monitor
2010-03-03 17:31:37.860: [ CRSMAIN][1] CRS Daemon Started.
2010-03-03 17:31:37.860: [ CRSD][1] Connecting to the CSS Daemon
2010-03-03 17:31:37.861: [ CRSD][1] Local CSS Node Number is: 2
2010-03-03 17:31:37.863: [ CRSD][1] Local Css Node Name is: mskbkp2
2010-03-03 17:31:37.863: [ CRSD][1] CRSDPersonality initialized
2010-03-03 17:31:37.864: [ CRSMAIN][1] Process member data: CRSD:mskbkp2
2010-03-03 17:31:37.864: [ CRSD][1][F-ALGO] getIpcPath returning (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_IPC_SOCKET_11))
2010-03-03 17:31:37.865: [CLSFRAME][1] Inited lsf context 102b3f670
2010-03-03 17:31:37.865: [CLSFRAME][1] Initing CLS Framework messaging
2010-03-03 17:31:37.869: [ CRSD][1][F-ALGO] getIpcPath returning (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_IPC_SOCKET_11))
2010-03-03 17:31:37.873: [UiServer][1] UI Comms initalize() 1
2010-03-03 17:31:37.873: [CLSFRAME][1] New Framework state: 2
2010-03-03 17:31:37.873: [CLSFRAME][1] M2M is starting...
2010-03-03 17:31:37.873: [ CRSCCL][1]clsCclInit called by process: 18669
2010-03-03 17:31:37.885: [ CRSCCL][1]USING CLSC ============
2010-03-03 17:31:37.895: [ default][1]clsu_get_private_ip_addr: Calling clsu_get_private_ip_addresses to get first private ip
2010-03-03 17:31:37.895: [ default][1]Check namebufs
2010-03-03 17:31:37.895: [ default][1]Finished checking namebufs
2010-03-03 17:31:37.950: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:405] gpnp tracelevel 3, component tracelevel 0
2010-03-03 17:31:37.951: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:535] '/u01/grid/11.2.0' in effect as GPnP home base.
2010-03-03 17:31:37.970: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3867] Init gpnp local security key providers (2)
fatal if both fail
2010-03-03 17:31:37.970: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3870] Init gpnp local security key proveders 1 o
f 2: file wallet (LSKP-FSW)
2010-03-03 17:31:37.970: [ GPnP][1]clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:398] Using FS Wallet Location : /u01/grid/11.2.0/
gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:37.970: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3892] Init gpnp local security key provider 1 of
2: file wallet (LSKP-FSW) OK
2010-03-03 17:31:37.970: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3898] Init gpnp local security key proveders 2 o
f 2: OLR wallet (LSKP-CLSW-OLR)
[ CLWAL][1]clsw_Initialize: OLR initlevel [70000]
2010-03-03 17:31:37.980: [ GPnP][1]clsgpnp_InitCKProviders: [at clsgpnp0.c:3921] Init gpnp local security key provider 2 of
2: OLR wallet (LSKP-CLSW-OLR) OK
2010-03-03 17:31:37.980: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1952] <Get gpnp security keys (wallet) for id:1,typ;7. (2
providers - fatal if all fail)
2010-03-03 17:31:37.980: [ GPnP][1]clsgpnpkwf_getWalletPath: [at clsgpnpkwf.c:501] req_id=1 ck_prov_id=1 wallet path: /u01/
grid/11.2.0/gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:38.049: [ GPnP][1]clsgpnpwu_walletfopen: [at clsgpnpwu.c:496] Opened SSO wallet: '/u01/grid/11.2.0/gpnp/ms
kbkp2/wallets/peer/cwallet.sso'
2010-03-03 17:31:38.049: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1968] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1
of 2 (LSKP-FSW(1))
2010-03-03 17:31:38.049: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2010-03-03 17:31:38.068: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1952] <Get gpnp security keys (wallet) for id:1,typ;4. (2
providers - fatal if all fail)
2010-03-03 17:31:38.068: [ GPnP][1]clsgpnpkwf_getWalletPath: [at clsgpnpkwf.c:501] req_id=1 ck_prov_id=1 wallet path: /u01/
grid/11.2.0/gpnp/mskbkp2/wallets/peer/
2010-03-03 17:31:38.134: [ GPnP][1]clsgpnpwu_walletfopen: [at clsgpnpwu.c:496] Opened SSO wallet: '/u01/grid/11.2.0/gpnp/ms
kbkp2/wallets/peer/cwallet.sso'
2010-03-03 17:31:38.135: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1968] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1
of 2 (LSKP-FSW(1))
2010-03-03 17:31:38.135: [ GPnP][1]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2010-03-03 17:31:38.135: [ GPnP][1]clsgpnp_Init: [at clsgpnp0.c:840] GPnP client pid=18669, tl=3, f=3
2010-03-03 17:31:38.184: [GIPCXCPT][1] gipcShutdownF: skipping shutdown, count 2, from [ clsinet.c : 1735], ret gipcretSuccess
(0)
2010-03-03 17:31:38.194: [GIPCXCPT][1] gipcShutdownF: skipping shutdown, count 1, from [ clsgpnp0.c : 1021], ret gipcretSucces
s (0)
2010-03-03 17:31:38.200: [ CRSCCL][1]Listening endpoint created sucessfully @ (ADDRESS=(PROTOCOL=tcp)(DEV=54)(HOST=172.31.25.
112)(PORT=38984)).con = 10359a0d0
2010-03-03 17:31:38.209: [ CRSCCL][48]CSS Group Registration complete.
2010-03-03 17:31:38.213: [ CRSCCL][48]cclGetMemberData called
2010-03-03 17:31:38.215: [ CRSCCL][48]Obtained first membership map.
2010-03-03 17:31:38.215: [ CRSCCL][48]Dumping member data ------------------
2010-03-03 17:31:38.215: [ CRSCCL][48]Member (2, 603412550) on node port=.
2010-03-03 17:31:38.216: [ CRSCCL][48]Done ------------------
2010-03-03 17:31:38.216: [ CRSCCL][48]Waiting for reconfigs
2010-03-03 17:31:38.216: [ CRSCCL][49]cclCommunicationHandler started.
2010-03-03 17:31:38.220: [ CRSCOMM][1] Ipc: m_pClscCtx=1020c4850m_pUgblm=1035b2a50
2010-03-03 17:31:38.220: [ CRSCOMM][1] Ipc: Starting send thread
2010-03-03 17:31:38.220: [ CRSCOMM][1] IpcL: Listener instantiated for: (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_IPC_SOCKET_11))
2010-03-03 17:31:38.221: [ CRSCOMM][52] Ipc: sendWork thread started.
2010-03-03 17:31:38.222: [ CRSCOMM][1] IpcL: Listener started listening.
2010-03-03 17:31:38.223: [ CRSCOMM][53] IpcL: thread started listening
2010-03-03 17:31:38.223: [CLSFRAME][1] Starting thread model named: AgfwProxySrvTM
2010-03-03 17:31:38.224: [CLSFRAME][1] Starting thread model named: OcrModuleTM
2010-03-03 17:31:38.225: [CLSFRAME][1] Starting thread model named: PolicyEngineTM
2010-03-03 17:31:38.225: [CLSFRAME][1] Starting thread model named: SharedThreadTM
2010-03-03 17:31:38.225: [CLSFRAME][1] Starting thread model named: UiServerTM
2010-03-03 17:31:38.225: [CLSFRAME][1] New Framework state: 3
2010-03-03 17:31:38.227: [ CRSRPT][62] Enabled
2010-03-03 17:31:38.228: [ CRSPE][61] PE Role|State Update: old role [INVALID] new [INVALID]; old state [Not yet initialized
] new [Enabling: waiting for role]
2010-03-03 17:31:38.229: [ CRSSE][62] Master Change Event; New Master Node ID:2 This Node's ID:2
2010-03-03 17:31:38.230: [ CRSPE][61] PE Role|State Update: old role [INVALID] new [MASTER]; old state [Enabling: waiting fo
r role] new [Configuring]
2010-03-03 17:31:38.230: [ CRSPE][61] PE MASTER NAME: mskbkp2
2010-03-03 17:31:38.230: [ CRSPE][61] Starting to read configuration
2010-03-03 17:31:38.260: [ CRSPE][61] Reading (2) servers
2010-03-03 17:31:38.459: [ CRSPE][61] DM: set global config version to: 150
2010-03-03 17:31:38.459: [ CRSPE][61] DM: set pool freeze timeout to: 60000
2010-03-03 17:31:38.459: [ CRSPE][61] DM: Set event seq number to: 13900000
2010-03-03 17:31:38.459: [ CRSPE][61] DM: Set threshold event seq number to: 13980000
2010-03-03 17:31:38.460: [ CRSPE][61] Sent request to write event sequence number 14000000 to repository
2010-03-03 17:31:38.483: [ CRSPE][61] Wrote new event sequence to repository
2010-03-03 17:31:38.568: [ CRSPE][61] Reading (15) types
2010-03-03 17:31:38.593: [ CRSPE][61] Reading (3) server pools
2010-03-03 17:31:38.624: [ CRSPE][61] Reading (21) resources
2010-03-03 17:31:39.987: [ CRSPE][61] Finished reading configuration. Parsing...
2010-03-03 17:31:39.988: [ CRSPE][61] Parsing resource types...
2010-03-03 17:31:40.030: [ CRSD][61] Initializing the config version for type ora.asm.type to: 1
2010-03-03 17:31:40.035: [ CRSD][61] Initializing the config version for type ora.cluster_resource.type to: 1
2010-03-03 17:31:40.040: [ CRSD][61] Initializing the config version for type ora.cluster_vip.type to: 1
2010-03-03 17:31:40.044: [ CRSD][61] Initializing the config version for type ora.cluster_vip_net1.type to: 1
2010-03-03 17:31:40.048: [ CRSD][61] Dump State Starting ...
2010-03-03 17:31:40.048: [ CRSD][61] State Dump for RTILock
2010-03-03 17:31:40.048: [ CRSD][61] Lock State List is busy, skipping ..
2010-03-03 17:31:40.048: [ CRSD][61] State Dump for Timer
2010-03-03 17:31:40.049: [ CRSD][61] Timer map size=0
2010-03-03 17:31:40.049: [ CRSPE][61] Dumping PE Data Model...:DM has [0 resources][0 types][0 servers][0 spools]
------------- RESOURCES:
------------- TYPES:
------------- SERVERS:
------------- SERVER POOLS:
2010-03-03 17:31:40.049: [ CRSPE][61] Dumping ICE contents...:ICE operation count: 0
2010-03-03 17:31:40.049: [ CRSD][61] Dump State Done.
I guess that there is some thing wrong in configuration, but cannot find out what.
Any help would be appreciated.
ThanksHi,
Please check your disk attributes and permission of OCR/Voting and other ASM devices. The disk attribute should be changed to be shared among all nodes of cluster. It happened with us in 10.2.0.4 where disk was not shared and we were able to start crs from only one node at a time so please check disk attributes. Please see blog keyurmakwanacrs.blogspot.com for AIX which we faced. Not surle whether you've similar problem or not. We had 10.2.0.4 clusterware.
thanks,
Keyur -
Hi,
In one of my C applications, malloc is dumping core even if the pointer was free'd earlier (this is the behavior for both 32-bit and 64-bit).
So I re-built the application with -lmapmalloc, and it worked fine.
Can any one suggest whether it is ok to link the application with libmapmalloc or not.
Thanks.This is almost always cause by overwriting areas that you shouldn't be. Preload watchmalloc (man watchmalloc) and see where it blows up.
-r -
Hi - We are currently using dbxml for many years successfully on CentOS and FreeBSD. Recently,we have been trying to get our dbxml application to run on Joyent's SmartOS platform.
Through the help of Joyent, we are able to build sbxml 2.5.16 binaries on smartos. Build ntes here:
https://github.com/joyent/smartos-live/issues/236
However, when we run the application, the java core dumps - both on jdk 7 and jdk 6. Same code runs find on other platforms. Core dump is included in the above post. Not sure if anyone here might have an idea on what is causing this.Hi Lauren,
Thanks for the response. Here is what I posted about this on the smartos forums. They helped me in getting dbxml to compile on smartos. Including using an xquilla patch:
OK, so the build problem you're having there appears to be an error in the C++ source for xqilla. This diff appears to fix that:
--- pristine/dbxml-2.5.16/xqilla/src/items/DatatypeFactoryTemplate.hpp 2009-01-07 11:46:13.000000000 +0000 +++ dbxml-2.5.16/xqilla/src/items/DatatypeFactoryTemplate.hpp 2013-07-11 08:17:00.638395661 +0000 @@ -79,7 +79,7 @@ AnyAtomicType::Ptr createInstance(const XMLCh* value, const DynamicContext* context) const { - return createInstanceNoCheck(DatatypeFactoryTemplate<TYPE>::getPrimitiveTypeURI(), + return this->createInstanceNoCheck(DatatypeFactoryTemplate<TYPE>::getPrimitiveTypeURI(), DatatypeFactoryTemplate<TYPE>::getPrimitiveTypeName(), value, context); }
Here is the post about the core dump:
Hi - OK, so apparently, the newly compiled code core dumps the JVM every time I try to call the openContainer method.
At first, I thought this was a code issue or a jdk 7 issue. However, I've compiled dbxml on smartos via the instructions in above thread using both jdk 6 and jdk 7 and I get the same core dump on the same call.
Also, tested the code on jdk 7 and jdk 6 on FreeBSD, Linux and Solaris and everything works OK.
I also thought it was a problem with my data, but I tried this create a fresh container using dbxml shell.
Same thing. So it seems to be a problem with the dbxml build specific to smartos.
Anyone have any ideas on where I can start looking for a solution?
$ /opt/local/java/bin/java -d64 -cp ...
srv context: java.io.BufferedInputStream@1f2f0ce
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x000000000028a89d, pid=4151, tid=2
JRE version: 7.0_25-b15
Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode solaris-amd64 compressed oops)
Problematic frame:
C 0x000000000028a89d
Core dump written. Default location: /home/nxd/srv/adm/core or core.4151
An error report file with more information is saved as:
/home/nxd/srv/adm/hs_err_pid4151.log
If you would like to submit a bug report, please visit:
http://bugreport.sun.com/bugreport/crash.jsp
Abort (core dumped)
========CORE DUMP
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x000000000028a89d, pid=4116, tid=2
JRE version: 7.0_25-b15
Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode solaris-amd64 compressed oops)
Problematic frame:
C 0x000000000028a89d
Core dump written. Default location: /home/test/srv/adm/core or core.4116
If you would like to submit a bug report, please visit:
http://bugreport.sun.com/bugreport/crash.jsp
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.
T H R E A D
Current thread (0x000000000041e000): JavaThread "main" [_thread_in_native, id=2, stack(0xfffffd7ffe800000,0xfffffd7ffea00000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x000000000028a89d
Registers:
RAX=0x000000000028a89d, RBX=0x0000000000000001, RCX=0x0000000002b27ea0, RDX=0x474e5543432b2b00
RSP=0xfffffd7ffe9fec88, RBP=0xfffffd7ffe9fee30, RSI=0x0000000000000001, RDI=0x0000000000000001
R8 =0xfffffd7ffe9fecb0, R9 =0xfffffd7fbfeb8e60, R10=0x0000000000000434, R11=0xfffffd7fff202df0
R12=0x0000000002b27ec0, R13=0x0000000000000002, R14=0x0000000000000001, R15=0x0000000002b28630
RIP=0x000000000028a89d, RFLAGS=0x0000000000010286
Top of Stack: (sp=0xfffffd7ffe9fec88)
0xfffffd7ffe9fec88: fffffd7fff2a2e50 fffffd7ffe9fecc0
0xfffffd7ffe9fec98: 00000001ff3fc800 fffffd7ffe9fee50
0xfffffd7ffe9feca8: 0000000002b27ea0 fffffd7ffe9fefa0
0xfffffd7ffe9fecb8: fffffd7fff3b1422 0000000002b27ea0
0xfffffd7ffe9fecc8: 0000000002b9b530 fffffd7fbfeefd90
0xfffffd7ffe9fecd8: fffffd7ff53b7fb3 fffffd7ffe9ff190
0xfffffd7ffe9fece8: fffffd7ffe9ff090 0000000000000000
0xfffffd7ffe9fecf8: 0000000000000001 0000000000000000
0xfffffd7ffe9fed08: ffffff00000000ff 0000000002b8cf00
0xfffffd7ffe9fed18: fffffd7ffe9ff110 0000000002b1ea40
0xfffffd7ffe9fed28: 0000000002582310 0000000000000000
0xfffffd7ffe9fed38: 0000000000000001 6d2e42494c534f5f
0xfffffd7ffe9fed48: 0000000002b1ea40 0000000000000000
0xfffffd7ffe9fed58: 0000000000000000 fffffd7ffe9ff080
0xfffffd7ffe9fed68: fffffd7ffe9ff000 0000000000000000
0xfffffd7ffe9fed78: 0000000000000000 697600656e6f6e2d
0xfffffd7ffe9fed88: 2d65646f6e007765 0000000002b27ec0
0xfffffd7ffe9fed98: 0000000000000002 0000000000000001
0xfffffd7ffe9feda8: 0000000002b28630 fffffd7ffe9ff090
0xfffffd7ffe9fedb8: fffffd7fbfe81442 fffffd7fbfe3be1d
0xfffffd7ffe9fedc8: fffffd7fbfc51f28 000000000028a89d
0xfffffd7ffe9fedd8: fffffd7fbfe81108 fffffd7fbfeb8e60
0xfffffd7ffe9fede8: 0000000000000434 fffffd7ffe9fecb0
0xfffffd7ffe9fedf8: 0000000100000000 fffffd7fbfc9f120
0xfffffd7ffe9fee08: 0000000002b1ea40 0000000002b27ec0
0xfffffd7ffe9fee18: 0000000000000001 fffffd7ffe9fee50
0xfffffd7ffe9fee28: 0000000002b27ea0 fffffd7ffe9fefb0
0xfffffd7ffe9fee38: fffffd7fff2a302f 00000000000006c9
0xfffffd7ffe9fee48: 0000000002b27ea0 fffffd7ffe9fefa0
0xfffffd7ffe9fee58: fffffd7fff3b1422 0000000002b27ea0
0xfffffd7ffe9fee68: 0000000002b9b530 fffffd7fbfeefd90
0xfffffd7ffe9fee78: fffffd7ff53b7fb3 fffffd7ffe9ff190
Instructions: (pc=0x000000000028a89d)
0x000000000028a87d:
[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xb]
Register to memory mapping:
RAX=0x000000000028a89d is an unknown value
RBX=0x0000000000000001 is an unknown value
RCX=0x0000000002b27ea0 is an unknown value
RDX=0x474e5543432b2b00 is an unknown value
RSP=0xfffffd7ffe9fec88 is pointing into the stack for thread: 0x000000000041e000
RBP=0xfffffd7ffe9fee30 is pointing into the stack for thread: 0x000000000041e000
RSI=0x0000000000000001 is an unknown value
RDI=0x0000000000000001 is an unknown value
R8 =0xfffffd7ffe9fecb0 is pointing into the stack for thread: 0x000000000041e000
R9 =0xfffffd7fbfeb8e60: _ZTSN5DbXml23DbXmlDebugHookDecoratorE+0x13040 in /home/test/srv/bdbxml.smartos_jdk7/lib/libdbxml-2.5.so at 0xfffffd7fbfc00000
R10=0x0000000000000434 is an unknown value
R11=0xfffffd7fff202df0: memcpy+0x60 in /lib/amd64/libc.so.1 at 0xfffffd7fff190000
R12=0x0000000002b27ec0 is an unknown value
R13=0x0000000000000002 is an unknown value
R14=0x0000000000000001 is an unknown value
R15=0x0000000002b28630 is an unknown value
Stack: [0xfffffd7ffe800000,0xfffffd7ffea00000], sp=0xfffffd7ffe9fec88, free space=2043k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C 0x000000000028a89d
C [libc.so.1+0x11302f] SUNW_Unwind_RaiseException+0x50
C [libstdc++.so.6.0.13+0x1380db] __cxa_throw+0x9b
C [libdbxml-2.5.so+0x281442] DbXml::SyntaxDatabase::SyntaxDatabase(const DbXml::Syntax(_db_env*, DbXml::Transaction(std::basic_string < char, std::char_traits, std::allocator >, bool, const DbXml::ContainerConfig(bool)&)*)*)+0x33a
C [libdbxml-2.5.so+0x23be1d] DbXml::Container::openIndexDbs(DbXml::Transaction(const DbXml::ContainerConfig&)*)+0x203
C [libdbxml-2.5.so+0x23c5a5] DbXml::Container::openInternal(DbXml::Transaction(const DbXml::ContainerConfig(bool)&)*)+0x649
C [libdbxml-2.5.so+0x23ce52] DbXml::Container::Container(DbXml::Manager(std::basic_string < char, std::char_traits, std::allocator >, DbXml::Transaction(const DbXml::ContainerConfig(bool)&)*)&)+0x212
C [libdbxml-2.5.so+0x26e2f1] DbXml::Manager::ContainerStore::findContainer(void&, std::basic_string < char, std::char_traits, std::allocator >, DbXml::Transaction(const DbXml::ContainerConfig(bool)&)*)+0xb7
C [libdbxml-2.5.so+0x26e4b9] DbXml::Manager::openContainer(std::basic_string < char, std::char_traits, std::allocator >, DbXml::Transaction(const DbXml::ContainerConfig(bool)&)*)+0x11b
C [libdbxml-2.5.so+0x28fa45] DbXml::XmlManager::openContainer(std::basic_string < char, std::char_traits, std::allocator >, const DbXml::XmlContainerConfig&)+0x73
C [libdbxml_java-2.5.so+0x61b68] Java_com_sleepycat_dbxml_dbxml_1javaJNI_XmlManager_1openContainerInternal_1_1SWIG_10+0x106
j com.sleepycat.dbxml.dbxml_javaJNI.XmlManager_openContainerInternal__SWIG_0(JLcom/sleepycat/dbxml/XmlManager;Ljava/lang/String;[ILjava/lang/String;)J+0
j com.sleepycat.dbxml.XmlManager.openContainerInternal(Ljava/lang/String;Lcom/sleepycat/dbxml/XmlContainerConfig;)Lcom/sleepycat/dbxml/XmlContainer;+18
j com.sleepycat.dbxml.XmlManager.openContainer(Ljava/lang/String;Lcom/sleepycat/dbxml/XmlContainerConfig;)Lcom/sleepycat/dbxml/XmlContainer;+3
j com.lightspoke.dbx.dao.DbxContainerManager.()V+411
j com.lightspoke.dbx.service.DbxRMIEngine.()V+45
j com.lightspoke.dbx.service.DbxRMIEngine.main([Ljava/lang/String;)V+29
v ~StubRoutines::call_stub
V [libjvm.so+0x54e5d1] void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x5d1
V [libjvm.so+0x54e81b] void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2b
V [libjvm.so+0x638695] jni_CallStaticVoidMethod+0x5c1
C [libjli.so+0x4d83] JavaMain+0x5e7
C [libc.so.1+0x10cfaa] _thrp_setup+0x8a
C [libc.so.1+0x10d2c0] _lwp_start+0x0
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j com.sleepycat.dbxml.dbxml_javaJNI.XmlManager_openContainerInternal__SWIG_0(JLcom/sleepycat/dbxml/XmlManager;Ljava/lang/String;[ILjava/lang/String;)J+0
j com.sleepycat.dbxml.XmlManager.openContainerInternal(Ljava/lang/String;Lcom/sleepycat/dbxml/XmlContainerConfig;)Lcom/sleepycat/dbxml/XmlContainer;+18
j com.sleepycat.dbxml.XmlManager.openContainer(Ljava/lang/String;Lcom/sleepycat/dbxml/XmlContainerConfig;)Lcom/sleepycat/dbxml/XmlContainer;+3
j com.lightspoke.dbx.dao.DbxContainerManager.()V+411
j com.lightspoke.dbx.service.DbxRMIEngine.()V+45
j com.lightspoke.dbx.service.DbxRMIEngine.main([Ljava/lang/String;)V+29
v ~StubRoutines::call_stub
P R O C E S S
Java Threads: ( => current thread )
0x0000000002410000 JavaThread "Socket Server Thread-2" [_thread_in_native, id=20, stack(0xfffffd7ff4000000,0xfffffd7ff4200000)]
0x0000000002405000 JavaThread "KaRMI Object Queue" daemon [_thread_blocked, id=19, stack(0xfffffd7ff4400000,0xfffffd7ff4600000)]
0x00000000021eb800 JavaThread "Service Thread" daemon [_thread_blocked, id=17, stack(0xfffffd7ff4800000,0xfffffd7ff4a00000)]
0x00000000021e9800 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=16, stack(0xfffffd7ffe6ef000,0xfffffd7ffe7ef000)]
0x00000000021e6800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=15, stack(0xfffffd7ffec0f000,0xfffffd7ffed0f000)]
0x00000000021e4800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=14, stack(0xfffffd7ff4c00000,0xfffffd7ff4e00000)]
0x000000000217b800 JavaThread "Finalizer" daemon [_thread_blocked, id=13, stack(0xfffffd7ff5000000,0xfffffd7ff5200000)]
0x0000000002174800 JavaThread "Reference Handler" daemon [_thread_blocked, id=12, stack(0xfffffd7ffe000000,0xfffffd7ffe200000)]
=>0x000000000041e000 JavaThread "main" [_thread_in_native, id=2, stack(0xfffffd7ffe800000,0xfffffd7ffea00000)]
Other Threads:
0x000000000216c000 VMThread [stack: 0xfffffd7ffe2af000,0xfffffd7ffe3af000] [id=11]
0x0000000002205800 WatcherThread [stack: 0xfffffd7ffdeff000,0xfffffd7ffdfff000] [id=18]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
Heap
PSYoungGen total 19712K, used 16342K [0x00000000eaa00000, 0x00000000ec000000, 0x0000000100000000)
eden space 16896K, 96% used [0x00000000eaa00000,0x00000000eb9f5a00,0x00000000eba80000)
from space 2816K, 0% used [0x00000000ebd40000,0x00000000ebd40000,0x00000000ec000000)
to space 2816K, 0% used [0x00000000eba80000,0x00000000eba80000,0x00000000ebd40000)
ParOldGen total 43008K, used 0K [0x00000000c0000000, 0x00000000c2a00000, 0x00000000eaa00000)
object space 43008K, 0% used [0x00000000c0000000,0x00000000c0000000,0x00000000c2a00000)
PSPermGen total 22528K, used 7661K [0x00000000bae00000, 0x00000000bc400000, 0x00000000c0000000)
object space 22528K, 34% used [0x00000000bae00000,0x00000000bb57b450,0x00000000bc400000)
Card table byte_map: [0xfffffd7ff8c00000,0xfffffd7ff8e2a000] byte_map_base: 0xfffffd7ff8629000
Polling page: 0xfffffd7fff000000
Code Cache [0xfffffd7ff9000000, 0xfffffd7ff9400000, 0xfffffd7ffc000000)
total_blobs=371 nmethods=41 adapters=283 free_code_cache=48554Kb largest_free_block=49715392
Compilation events (10 events):
Event: 0.848 Thread 0x00000000021e9800 nmethod 34 0xfffffd7ff9072a90 code [0xfffffd7ff9072be0, 0xfffffd7ff9072c78]
Event: 0.848 Thread 0x00000000021e9800 35 java.util.ArrayList::get (11 bytes)
Event: 0.848 Thread 0x00000000021e9800 nmethod 35 0xfffffd7ff90788d0 code [0xfffffd7ff9078a20, 0xfffffd7ff9078ad8]
Event: 0.854 Thread 0x00000000021e9800 36 ! sun.misc.URLClassPath$JarLoader::getResource (91 bytes)
Event: 0.894 Thread 0x00000000021e9800 nmethod 36 0xfffffd7ff9080490 code [0xfffffd7ff9080840, 0xfffffd7ff9082128]
Event: 0.907 Thread 0x00000000021e9800 37 java.io.UnixFileSystem::parentOrNull (118 bytes)
Event: 0.917 Thread 0x00000000021e9800 nmethod 37 0xfffffd7ff907ddd0 code [0xfffffd7ff907df40, 0xfffffd7ff907e598]
Event: 0.968 Thread 0x00000000021e6800 nmethod 32 0xfffffd7ff908bf50 code [0xfffffd7ff908c660, 0xfffffd7ff90909c8]
Event: 1.019 Thread 0x00000000021e9800 38 java.util.Arrays::copyOf (19 bytes)
Event: 1.021 Thread 0x00000000021e9800 nmethod 38 0xfffffd7ff908a250 code [0xfffffd7ff908a3a0, 0xfffffd7ff908a578]
GC Heap History (0 events):
No events
Deoptimization events (6 events):
Event: 0.351 Thread 0x000000000041e000 Uncommon trap -34 fr.pc 0xfffffd7ff9062f04
Event: 0.352 Thread 0x000000000041e000 Uncommon trap -34 fr.pc 0xfffffd7ff9062f04
Event: 0.438 Thread 0x000000000041e000 Uncommon trap -83 fr.pc 0xfffffd7ff9067068
Event: 0.638 Thread 0x000000000041e000 Uncommon trap -83 fr.pc 0xfffffd7ff9067908
Event: 0.668 Thread 0x000000000041e000 Uncommon trap -83 fr.pc 0xfffffd7ff9076094
Event: 0.669 Thread 0x000000000041e000 Uncommon trap -12 fr.pc 0xfffffd7ff9065848
Internal exceptions (10 events):
Event: 1.079 Thread 0x000000000041e000 Threw 0x00000000eb94b5b8 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.563 Thread 0x000000000041e000 Threw 0x00000000eb972960 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.564 Thread 0x000000000041e000 Threw 0x00000000eb9756c0 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.564 Thread 0x000000000041e000 Threw 0x00000000eb97fc60 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.565 Thread 0x000000000041e000 Threw 0x00000000eb983ed0 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.565 Thread 0x000000000041e000 Threw 0x00000000eb98d390 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.566 Thread 0x000000000041e000 Threw 0x00000000eb992830 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.566 Thread 0x000000000041e000 Threw 0x00000000eb996070 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.567 Thread 0x000000000041e000 Threw 0x00000000eb9a1d10 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Event: 1.738 Thread 0x000000000041e000 Threw 0x00000000eb9a6840 at /export/HUDSON/workspace/jdk7u25-2-build-solaris-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.c
Events (10 events):
Event: 1.565 loading class 0x0000000002a678c0
Event: 1.565 loading class 0x0000000002a678c0 done
Event: 1.566 loading class 0x0000000002a67960
Event: 1.566 loading class 0x0000000002a67960 done
Event: 1.566 loading class 0x00000000022d9f00
Event: 1.566 loading class 0x00000000022d9f00 done
Event: 1.567 loading class 0x0000000002a67eb0
Event: 1.567 loading class 0x0000000002a67eb0 done
Event: 1.738 loading class 0x00000000022d9d40
Event: 1.738 loading class 0x00000000022d9d40 done
Dynamic libraries:
0x0000000000400000 /opt/local/jdk1.7.0_25/bin/amd64/dbxrmijvm
0xfffffd7ffd85c000 /usr/lib/amd64/libthread.so.1
0xfffffd7ffd700000 /opt/local/jdk1.7.0_25/bin/amd64/../../jre/lib/amd64/jli/libjli.so
0xfffffd7ffe56f000 /usr/lib/amd64/libdl.so.1
0xfffffd7fff190000 /usr/lib/amd64/libc.so.1
0xfffffd7ffc1a0000 /opt/local/jdk1.7.0_25/jre/lib/amd64/server/libjvm.so
0xfffffd7ffea40000 /usr/lib/amd64/libsocket.so.1
0xfffffd7ffd85b000 /usr/lib/amd64/libsched.so.1
0xfffffd7ffc180000 /usr/lib/amd64/libm.so.1
0xfffffd7ffc150000 /usr/lib/amd64/libCrun.so.1
0xfffffd7ffdaef000 /usr/lib/amd64/libdoor.so.1
0xfffffd7ffc110000 /usr/lib/amd64/libdemangle.so.1
0xfffffd7ffeee0000 /usr/lib/amd64/libm.so.2
0xfffffd7ffed10000 /usr/lib/amd64/libnsl.so.1
0xfffffd7ffeab0000 /usr/lib/amd64/libmd.so.1
0xfffffd7ffea90000 /usr/lib/amd64/libmp.so.2
0xfffffd7ffc0f0000 /opt/local/jdk1.7.0_25/jre/lib/amd64/libverify.so
0xfffffd7ffc0a0000 /opt/local/jdk1.7.0_25/jre/lib/amd64/libjava.so
0xfffffd7ffe670000 /usr/lib/amd64/libscf.so.1
0xfffffd7ffeba0000 /usr/lib/amd64/libuutil.so.1
0xfffffd7ffead0000 /usr/lib/amd64/libgen.so.1
0xfffffd7ffedb0000 /usr/lib/amd64/libnvpair.so.1
0xfffffd7ffe650000 /usr/lib/amd64/libsmbios.so.1
0xfffffd7ffc070000 /opt/local/jdk1.7.0_25/jre/lib/amd64/libzip.so
0xfffffd7ffc040000 /opt/local/jdk1.7.0_25/jre/lib/amd64/libnet.so
0xfffffd7ffc020000 /opt/local/jdk1.7.0_25/jre/lib/amd64/libnio.so
0xfffffd7ffeb20000 /usr/lib/amd64/librt.so.1
0xfffffd7ffc000000 /usr/lib/amd64/libsendfile.so.1
0xfffffd7ffdd30000 /home/test/srv/bdbxml.smartos_jdk7/lib/libdb_java-4.8.so
0xfffffd7ffdb60000 /usr/lib/amd64/libresolv.so.2
0xfffffd7ffeacd000 /usr/lib/amd64/libpthread.so.1
0xfffffd7ffe630000 /usr/lib/amd64/libgcc_s.so.1
0xfffffd7ffe220000 /home/test/srv/bdbxml.smartos_jdk7/lib/libdbxml_java-2.5.so
0xfffffd7ffd950000 /home/test/srv/bdbxml/lib/libdb-4.8.so
0xfffffd7fc0400000 /home/test/srv/bdbxml/lib/libxqilla.so.5
0xfffffd7fc0000000 /home/test/srv/bdbxml/lib/libxerces-c-3.0.so
0xfffffd7fbfc00000 /home/test/srv/bdbxml/lib/libdbxml-2.5.so
0xfffffd7ff5280000 /usr/lib/amd64/libstdc++.so.6
VM Arguments:
jvm_args: -Xss2048k -Djava.library.path=/home/test/srv/bdbxml/lib:/home/test/bdb/lib:/usr/lib:/usr/local/lib:/lib:/usr/lib/amd64: -Djava.rmi.server.codebase=/home/test/srv/lib/dbxrmi.jar -Duka.karmi.config=/home/test/srv/adm/.karmi.property -Djava.rmi.server.hostname=test.gaoxiong -Djava.security.policy=/home/test/adm/DbxRMI.policy -Dlog4j.configuration=file:/home/test/adm/log4j.xml
java_command: com.lightspoke.dbx.service.DbxRMIEngine
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=/opt/local/java
PATH=/opt/local/java/bin:/home/test/srv/bdbxml/bin:/opt/local/java/bin:/home/test/srv/bdbxml/bin:/opt/local/java/bin:/home/test/srv/bdbxml/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/usr/sbin:/home/test/srv/bdbxml/bin:
LD_LIBRARY_PATH=/home/test/srv/bdbxml/lib:/home/test/bdb/lib:/usr/lib:/usr/local/lib:/lib:/usr/lib/amd64:
SHELL=/bin/bash
Signal Handlers:
SIGSEGV: [libjvm.so+0x11b37b4], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0x11b37b4], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0x53fc78], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0x53fc78], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGXFSZ: [libjvm.so+0x53fc78], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0x53fc78], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: [libjvm.so+0xff8ea8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGHUP: [libjvm.so+0xff8ea8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: [libjvm.so+0xff8ea8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0xff8ea8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0xffd27c], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0x53fc78], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
S Y S T E M
OS: SmartOS x86_64
Copyright 2010 Sun Microsystems, Inc. All Rights Reserved.
Copyright 2010-2012 Joyent, Inc. All Rights Reserved.
Use is subject to license terms.
See uname -v for assembly date and time.
uname:SunOS 5.11 joyent_20130530T224720Z i86pc
(T2 libthread)
rlimit: STACK 10240k, CORE infinity, NOFILE 65536, AS infinity
load average:0.01 0.00 0.00
CPU:total 8 (4 cores per cpu, 1 threads per core) family 6 model 15 stepping 11, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, tsc
Memory: 4k page, physical 4194304k(4142372k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (23.25-b01) for solaris-amd64 JRE (1.7.0_25-b15), built on Jun 5 2013 21:57:39 by "" with Sun Studio 12u1
time: Tue Aug 6 01:12:44 2013
elapsed time: 2 seconds -
Reconciling garbage collection, heap overview, and object stats
First, both the JRockit RuntimeAnalyzer and Console are great tools. We use
them extensively so thank you.
I'm trying to reconcile the numbers in these three tabs.
1. How do I reconcile the Runtime Analyzer and Console output?
- The Heap overview tab in our application profile shows Heap Overview as
83% free.
- However, the Garbage Collection tab of the profile and Console shows the
application as oscillating between 50 Meg and 200 Meg of heap used. That's
25% (50Meg/200Meg) to 100%(200Meg/200Meg) used. How do I interpret the 83%
vs. the 100%?
I don't believe the 83% free, but I'm skeptical that we consume 150Meg of
memory in 50 seconds.
2. How do I reconcile the Object stats with the Garbage collection?
- Take the top heap user at end of recording, character buffer. It's 22.8%
of heap using 6,328 KB. If the heap is actually only 34Meg ( 17% of 200Meg.
I get the 17% from the 83% free), then 22.8% makes sense.
- So what's in the 200Meg of heap?
I sent this recording to the JRA team if you want to look at it.
Thanks
JeffI've never heard it put that way. Very interesting.
"Johan Walles" <johan@spamalamadingdong> wrote in message
news:41bf0be3@mail...
Note that the time it takes for the garbage collector to do its thing
grows with the amount of live data.
What the garbage collector really does is more like retaining the live
data than disposing of the garbage.
Regards //Johan
Jeff wrote:
Staffan,
Thanks - you clearly answered my questions. Now the follow-on questions:
1. Is there a way to get insight into what the 'dead' data is composed
of?
2. Is this pattern of consuming 3x the live data in about a minute
'typical' or a disaster in the making?
I'm trying to get a sense of what a reasonable target of 'dead' data is.
The system processes about a meg of data per second, including database
writes.
Thanks
Jeff
"sla" <[email protected]> wrote in message
news:33533893.1102952170368.JavaMail.root@jserv5...
Hi Jeff,
I'll try to answer the questions.
1) The Heap overview in the application profile is a snapshot of the heap
at the end of a garbage collection. At this time only live data is still
on the heap. So it looks like you have 17% of the heap filled with live
data (and some overhead as seen in the Heap overview).
In the garbage collection tab you can see the heap usage oscillating
between 35-40MB and 200MB. The lower value is right after a garbage
collection and the higher value is right before a garbage collection. The
garbage collector clears out about 160MB of "dead" data from the heap.
This is the amount of temporary objects that you created during the
garbage collection cycles.
2) The Object statistics are also taken right after a garbage collection.
At this time there is 34M of live data on the heap and of these about 22%
is taken up by character arrays (not unusual).
At this time the rest of the 200MB heap is empty. It's been cleared of
all temporary objects and is ready for allocation of new objects.
I hope this answered your questions.
Regards,
/Staffan -
Netscape Directory Server core dumps
Hi,
I have NDS version 5.1 running on Solaris 10. My software is querying the LDAP at around 10-15 requests/sec. The LDAP memory is runnig at around 50M. (I have around 900 entries in the LDAP tree). At some random instance of time, the LDAP memory shoots up from 50M to 3.2 GB when it goes out of memory and then restarts. My software has a LDAP connection pool to it and one of the threads goes OutOfmemory after the error.
The LDAP does a core dump during the process. From the analysis of the core dump, it seems that it is trying to access some memory which is not part of the LDAP process map.
bash-3.00$ mdb core
mdb: failed to initialize /lib/libc_db.so.1: libthread_db call failed unexpectedly
mdb: warning: debugger will only be able to examine raw LWPs
Loading modules: [ libc.so.1 libuutil.so.1 ld.so.1 ]
::stacklibc_psr.so.1`memcpy+0x234(87c00038, 6b646174, 74776f6a, 8, 6b646174, 0)
libslapd.so`slapi_dup_control+0x94(53db678, 2ce38f8, fc93ff2c, 0, 2ce38f8,2ce38f8)
0x2f3b4(7e55c80, 2f30150, 2ce6bb8, 4c7fde8, 2f30150, ffffffff)
0xfef17d18(2f60164, fc940000, 0, 0, fef17c74, 1)
0xff2c8a20(0, 0, 0, 0, 0, 0)
::statusdebugging core file of ns-slapd (32-bit) from XXXXXX
initial argv:XXXXXXXX
threading model: multi-threaded using native lwps
status: process terminated by SIGSEGV (Segmentation Fault)
>
The failure is at address 0xfee8063c.
0xfee8063c::dislibc_psr.so.1`memcpy+0x20c: 0x2480012
libc_psr.so.1`memcpy+0x210: nop
libc_psr.so.1`memcpy+0x214: ldub [%i1], %o2
libc_psr.so.1`memcpy+0x218: stb %o2, [%i0]
libc_psr.so.1`memcpy+0x21c: add %i1, 1, %i1
libc_psr.so.1`memcpy+0x220: subcc %i3, 1, %i3
libc_psr.so.1`memcpy+0x224: 0x184ffffc
libc_psr.so.1`memcpy+0x228: add %i0, 1, %i0
libc_psr.so.1`memcpy+0x22c: ba +0x60 <libc_psr.so.1`memcpy+0x28c>
libc_psr.so.1`memcpy+0x230: nop
libc_psr.so.1`memcpy+0x234: ld [%i1], %o2
libc_psr.so.1`memcpy+0x238: st %o2, [%i0]
libc_psr.so.1`memcpy+0x23c: add %i1, 4, %i1
libc_psr.so.1`memcpy+0x240: subcc %i3, 4, %i3
// Looking for register i1
::regs%g0 = 0x00000000 %l0 = 0x87c00038
%g1 = 0xff37c840 libsh.so`_shi_sysAllocPool+0xd4 %l1 = 0xff16c2bc
%g2 = 0x00000000 %l2 = 0x07970778
%g3 = 0x0000036c %l3 = 0x05410000
%g4 = 0x00000001 %l4 = 0xff165a34
%g5 = 0x87c00038 %l5 = 0x00000000
%g6 = 0x00000000 %l6 = 0x01000000
%g7 = 0xfd06ca00 %l7 = 0x00000004
%o0 = 0x87c00038 %i0 = 0x87c00038
%o1 = 0x74776f72 %i1 = 0x6b646174
%o2 = 0xefe4617c %i2 = 0x74776f6a
%o3 = 0x00000002 %i3 = 0x00000008
%o4 = 0x00000002 %i4 = 0x6b646174
%o5 = 0x053d9c28 %i5 = 0x00000000
%o6 = 0xfc93fe00 %i6 = 0xfc93fe60
%o7 = 0xff0c0c44 libslapd.so`slapi_ch_malloc+0x64 %i7 = 0xff0e8e78
libslapd.so`slapi_dup_control+0x94
%psr = 0xfe401004 impl=0xf ver=0xe icc=nZvc
ec=0 ef=4096 pil=0 s=0 ps=0 et=0 cwp=0x4
%y = 0x00000002
%pc = 0xfee8063c libc_psr.so.1`memcpy+0x234
%npc = 0xfee80640 libc_psr.so.1`memcpy+0x238
%sp = 0xfc93fe00
0x6b646174::dismdb: failed to read instruction at 6b646174: no mapping for address
>
The register "i1"'s content was memory address 0x6b646174 which is not part of process address space.
Has anyone experienced similar problem before? I was trying to find the same in the forum but without any luck. This problem is happening quiet regularly.Seeems like content of register i1 got missed.. here is the content..
%l2 = 0x07970778
%l3 = 0x05410000
%l4 = 0xff165a34
%l5 = 0x00000000
%l6 = 0x01000000
%l7 = 0x00000004
%i0 = 0x87c00038
%i1 = 0x6b646174
%i2 = 0x74776f6a
%i3 = 0x00000008
%i4 = 0x6b646174
%i5 = 0x00000000
%i6 = 0xfc93fe60 -
Solaris 10, Ruby on Rails, Core Dumps
I am not very savvy with Solaris administration and we are having issues with configuring our system to run Ruby on Rails applications. We opted to use the Coolstack framework for our environment, thankfully without many hiccups. That said, our applications load but core dump during the second page load.
At this point, I'm not sure if it's our recent switch to Coolstack that is causing the problem. We've also worked to get a Mongrel cluster up and running, so that could also be the culprit.
What I'm looking for really is advice on how to pin down what the error is. I've learned about pstack to deal with the core dump file, but I'd have a better time understanding Chinese than that output. I also read about gdb and dbx, neither of which appear to be installed on our system. A strings on the file, another recommendation I found, spits out a bunch of gibberish.
How can I move forward in discovering the problem?I'm having the exact same problem with my Solaris server. I can't seem to get any form of Rails running on the system, and pages generated by Rails crash after a few page loads (not always the same number).
Did you have any luck tracking down the issue? I've asked around in a number of places and come up empty-handed every time. -
Investigation on JVM crash without core dump (by JIT compiler)
Hi, All
I posted "JVM crash without core dump due to CompilerThread1" couple months ago.
http://forum.java.sun.com/thread.jspa?threadID=5253434
I would like show what we found, and ask couple questions
1) The reason of JVM crash is "CompilerThread0/1 wanted to allocate more native memory and eventually exceeded the limit of 32-bit Linux (on Redhat, the limit of VIRT is 3G)
2) After we lower the heap size (to reduce process size) and added process size monitor (track process size every minute) and JIT compilation log (-XX:+PrintCompilation),
We found sometimes JVM process jumped more than 800M when compiling one method. with the following log
Total time for which application threads were stopped: 0.1997400 seconds
5828 xxx.xxx.SomeClass::someMethod (1507 bytes)
5828 COMPILE SKIPPED: out of nodes during split (not retryable)
549002.449: [GC [PSYoungGen: 517897K->28390K(551296K)] 942855K->453348K(1229952K), 0.0726350 secs]
if we saw "COMPILE SKIPPED: out of nodes during split (not retryable)", the process size of JVM (VIRT/RES) always jumped 800M~1000M, and sometimes the memory get reverted in 30mins to couple hours, and sometimes it lasted forever. (so before we have larger footprint, this jump will kill JVM, right now, if the jump lasts forever, any more allocation on top of it also can kill jvm (much rare))
3) This is definitely a bug of JVM, because
it only happened on server mode, not client mode
it is random, that method can be compiled success on other JVM or next restart (we have 20 JVMs)
the method is not that complicate, like 100 lines, (bigger method gets compiled success)
this only happen on PROD environment, we can't reproduce it locally or QA (the method is always compiled success)
It mostly like under some condition (maybe node space is not enough), to compile that method will trigger JVM trying to allocate much more native memory.
4) We are going to disable this method by ".hotspot_compiler" to fix it, right now we are using lower footprint, JVM dies rarely.
+To understand more about this, I have some questions about JIT compiling+
I saw same method are compiled more than once from JIT log, like the method caused our problem, it only happens on 2nd time compilation.
Is it that JIT compiler will recompile the method with deeper optimization level some time, and more optimization it uses, the more memory it requires?
(like gcc has -O2 -O3)
Thanks!
NeoToday, one JVM crashed again, with
{Heap before gc invocations=4188:
PSYoungGen total 540864K, used 487958K [0x8ba50000, 0xb1250000, 0xb1250000)
eden space 467328K, 100% used [0x8ba50000,0xa82b0000,0xa82b0000)
from space 73536K, 28% used [0xa82b0000,0xa96d5850,0xaca80000)
to space 71680K, 0% used [0xacc50000,0xacc50000,0xb1250000)
ParOldGen total 1024000K, used 493477K [0x4d250000, 0x8ba50000, 0x8ba50000)
object space 1024000K, 48% used [0x4d250000,0x6b439610,0x8ba50000)
PSPermGen total 101760K, used 101240K [0x2d250000, 0x335b0000, 0x4d250000)
object space 101760K, 99% used [0x2d250000,0x3352e1f8,0x335b0000)
405290.711: [GC [PSYoungGen: 487958K->17397K(544192K)] 981435K->518248K(1568192K), 0.0769350 secs]
Heap after gc invocations=4188:
PSYoungGen total 544192K, used 17397K [0x8ba50000, 0xb1250000, 0xb1250000)
eden space 472512K, 0% used [0x8ba50000,0x8ba50000,0xa87c0000)
from space 71680K, 24% used [0xacc50000,0xadd4d480,0xb1250000)
to space 70208K, 0% used [0xa87c0000,0xa87c0000,0xacc50000)
ParOldGen total 1024000K, used 500851K [0x4d250000, 0x8ba50000, 0x8ba50000)
object space 1024000K, 48% used [0x4d250000,0x6bb6cf70,0x8ba50000)
PSPermGen total 101760K, used 101240K [0x2d250000, 0x335b0000, 0x4d250000)
object space 101760K, 99% used [0x2d250000,0x3352e1f8,0x335b0000)
Total time for which application threads were stopped: 0.0779620 seconds
Exception in thread "CompilerThread1" java.lang.OutOfMemoryError: requested 4522768 bytes for Chunk::new. Out of swap space? -
Garbage collection Java Virtual Machine : Hewlett-Packard Hotspot release 1.3.1.01
"Hi,
I try and understand the mechanism of garbage collection of the Java Virtual Machine : Hewlett-Packard Hotspot release 1.3.1.01.
There is description of this mechanism in the pdf file : "memory management and garbage collection" available at the paragraph "Java performance tuning tutorial" at the page :
http://h21007.www2.hp.com/dspp/tech/tech_TechDocumentDetailPage_IDX/1,1701,1607,00.html
Regarding my question :
Below is an extract of the log file of garbage collections. This extract has 2 consecutive garbage collections.
(each begins with "<GC:").
<GC: 1 387875.630047 554 1258496 1 161087488 0 161087488 20119552 0 20119552
334758064 238778016 335544320
46294096 46294096 46399488 5.319209 >
<GC: 5 387926.615209 555 1258496 1 161087488 0 161087488 0 0 20119552
240036512 242217264 335544320
46317184 46317184 46399488 5.206192 >
There are 2 "full garbage collections", one of reason "1" and one of reason "5".
For the first one "Old generation After " =238778016
For the second "Old generation After " =238778016
Thus, "Old generation Before garbage collection" of the second is higher than "Old generation After garbage collection". Why?
I expected all objects to be allocated in the "Eden" space. And therefore I did not expect to sI agree but my current Hp support is not very good on JVM issues.
Rob Woollen <[email protected]> wrote:
You'd probably be better off asking this question to HP.
-- Rob
Martial wrote:
The object of this mail is the Hewlett-Packard 1.3.1.01 Hotspot JavaVirtual Machine
release and its garbage collection mechanism.
I am interested in the "-Xverbosegc" option for garbage collectionmonitoring.
I have been through the online document :
http://www.hp.com/products1/unix/java/infolibrary/prog_guide/java1_3/hotspot.html#-Xverbosegc
I would like to find out more about the garbage collection mechanismand need
further information to understand the result of the log file generatedwith the
"-Xverbosegc"
For example here is an extract of a garbage collection log file generatedwith
Hewlett-Packard Hotspot Java Virtual Machine. Release 1.3.1.01.
These are 2 consecutive rows of the files :
<GC: 5 385565.750251 543 48 1 161087488 0 161087488 0 0 20119552 264184480255179792
335544320 46118384 46118384 46137344 5.514721 >
<GC: 1 385876.530728 544 1258496 1 161087488 0 161087488 20119552 020119552 334969696
255530640 335544320 46121664 46106304 46137344 6.768760 >
We have 2 full garbage collections, one of Reason 5 and the next oneof Reason
1.
What happened between these 2 garbage collections as we got : "Oldgeneration
After" of row 2 is higher than "Old generation Before" of row 1? Iexpected Objects
to be initially allocated in eden and so we could not get "old generation2modified
between the end of one garbage collection and before the next one.
Could you please clarify this issue and/or give more information aboutgarbage
collection mechanisms with the Hewlett-Packard Hotspot Java VirtualMachine. Release
1.3.1.01. -
I am using JNI in my Java application , 12 hours after running my test case get a JVM crash -
1) These are the parameters with which i invoke my Java program
java -d64 -Xcheck:jni -Xmx2560M -Xms2560M -Xss256k RunQueries
2) The Heap output at the time of the core shows "from space" as 100% used , does this signify anything?
Heap at VM Abort:
Heap
def new generation total 848128K, used 672342K [0xfffffffe93c00000, 0xfffffffec9150000, 0xfffffffec9150000)
eden space 822464K, 78% used [0xfffffffe93c00000, 0xfffffffebb385b90, 0xfffffffec5f30000)
from space 25664K, 100% used [0xfffffffec7840000, 0xfffffffec9150000, 0xfffffffec9150000)
to space 25664K, 0% used [0xfffffffec5f30000, 0xfffffffec5f30000, 0xfffffffec7840000)
tenured generation total 1747648K, used 1350866K [0xfffffffec9150000, 0xffffffff33c00000, 0xffffffff33c00000)
the space 1747648K, 77% used [0xfffffffec9150000, 0xffffffff1b884830, 0xffffffff1b884a00, 0xffffffff33c00000)
compacting perm gen total 16384K, used 13550K [0xffffffff33c00000, 0xffffffff34c00000, 0xffffffff37c00000)
the space 16384K, 82% used [0xffffffff33c00000, 0xffffffff3493b860, 0xffffffff3493ba00, 0xffffffff34c00000)
Local Time = Mon Feb 12 21:49:40 2007
Elapsed Time = 61687
# The exception above was detected in native code outside the VM
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.4.2_07-b05 mixed mode)
3) A dbx on the Java core shows the location in the JNI code where the core dump occured.
dbx `which java` core
For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 7.3' in your .dbxrc
Reading java
dbx: internal warning: writable memory segment 0x7cb00000[16384] of size 0 in core
core file header read successfully
Reading ld.so.1
Reading libthread.so.1
Reading libdl.so.1
Reading libc.so.1
Reading libc_psr.so.1
Reading libjvm.so
Reading libCrun.so.1
Reading libsocket.so.1
Reading libnsl.so.1
Reading libm.so.1
WARNING!!
A loadobject was found with an unexpected checksum value.
See `help core mismatch' for details, and run `proc -map'
to see what checksum values were expected and found.
dbx: warning: Some symbolic information might be incorrect.
t@1 (l@1) terminated by signal ABRT (Abort)
0xffffffff7eea822c: lwpkill+0x0008: bcc,a,pt %icc,_lwp_kill+0x18 ! 0xffffffff7eea823c
Current function is Java_getLoid (optimized)
7239 e = cod_to_long (*(o_object*) data, &loid_as_long);
(dbx) where
current thread: t@1
[1] lwpkill(0x0, 0x6, 0xffffffffffffffe6, 0x0, 0x0, 0x0), at 0xffffffff7eea822c
[2] raise(0x6, 0x0, 0xffffffff7fffad30, 0x7fbffeff00003ff6, 0x0, 0x2), at 0xffffffff7ee58a8c
[3] abort(0x0, 0xffffffff7fffae10, 0x0, 0xfffffffffffffff8, 0x0, 0xffffffff7fffae39), at 0xffffffff7ee3e3b8
[4] os::abort(0x1, 0xffffffff7e9d295c, 0xffffffff7fffaf10, 0xffffffff7e9d24a9, 0x4b007c, 0xffffffff7eb78878), at 0xffffffff7e90951c
[5] os::handle_unexpected_exception(0x10011e600, 0xa, 0xfffffffe90345704, 0xffffffff7fffbeb0, 0xffffffff7e69c6f8, 0x0), at 0xffffffff7e907e08
[6] JVM_handle_solaris_signal(0xffffffff7fffbeb0, 0xffffffff7e9d443e, 0xffffffff7fffbbd0, 0x1, 0x0, 0x1), at 0xffffffff7e69c800
[7] __sighndlr(0xa, 0xffffffff7fffbeb0, 0xffffffff7fffbbd0, 0xffffffff7e69cb9c, 0x0, 0x0), at 0xffffffff7f2188d8
---- called from signal handler with signal 10 (SIGBUS) ------
=>[8] Java_getLoid(vjEnv = ???, cls = ???, handle = ???, attr = ???) (optimized), at 0xfffffffe90345704 (line ~7239) in "j.c"
I have added checks for this function call to ensure that no invalid pointer is being sent to it which could cause a segmentation violation.
Please could someone give me some pointers on how to solve this problem, the error occurs at this point only 12 hours after running the test case..First step is to add the option -Xcheck:jni to your java command line. It will flag common JNI errors.
If that doesn't identify the problem, then compile your jni code with debugging symbols (-g) and add the following option to the java command line when you run your test: -XX:+ShowMessageBoxOnError. When you hit the problem again, the VM will keep the process alive instead of calling abort(). Then you can use dbx to attach to the live process and should be able to get a better idea of what went wrong. -
Hi all,
We have faced this problem in our production system.
Machine is Solaris 8, sparc, Running Java 1.3.1_08 hotspot version.
I searched around quite a bit but no good solution.
Here is the stack trace that I have just before the JVM crashes.
Also, I am new to gdb. I tried to connect to core dump file through gdb but i
do not see any details. The application is running on japanese platform how can
I read the core file?
Regards
Tapan
~~~~~~~~~~~~~~~~~~~~~~
STACK TRACE -
<2004/04/26 9:44:34:JST> <Info> <T3Services> <Integral5: ReliableTopicConnection.createTopicSession:
INSIDE...>
Tag 'insert' can't insert page '/dealing/Snippet.do?name=pricingDetail.edit'.
Check if it exists.
Null property value for 'allQuotes'
java.lang.IllegalArgumentException: Null property value for 'allQuotes'
at org.apache.commons.beanutils.PropertyUtils.getNestedProperty(PropertyUtils.java:619)
at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:669)
at org.apache.struts.util.RequestUtils.lookup(RequestUtils.java:509)
at org.apache.struts.taglib.bean.DefineTag.doStartTag(DefineTag.java:200)
at directDerivatives._integral._dealing._swap._CC.__PricingDetailEdit._jspService(__PricingDetailEdit.java:325)
at weblogic.servlet.jsp.JspBase.service(JspBase.java:27)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:262)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:198)
at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:530)
at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:350)
at org.apache.struts.tiles.ActionComponentServlet.processForward(ActionComponentServlet.java:262)
at org.apache.struts.tiles.ActionComponentServlet.processActionForward(ActionComponentServlet.java:104)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1529)
at com.integral.jsp.framework.IdcActionServlet.validateRequest(IdcActionServlet.java:331)
at com.integral.jsp.framework.IdcActionServlet.process(IdcActionServlet.java:139)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:487)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:262)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:198)
at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:530)
at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:350)
at weblogic.servlet.jsp.PageContextImpl.include(PageContextImpl.java:123)
at org.apache.struts.taglib.tiles.InsertTag$InsertHandler.doEndTag(InsertTag.java:733)
at org.apache.struts.taglib.tiles.InsertTag.doEndTag(InsertTag.java:368)
at directDerivatives._integral._dealing._workflow._template.__DealingFormLayout._jspService(__DealingFormLayout.java:1912)
at weblogic.servlet.jsp.JspBase.service(JspBase.java:27)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:262)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:198)
at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:284)
at org.apache.struts.tiles.ActionComponentServlet.processForward(ActionComponentServlet.java:264)
at org.apache.struts.tiles.ActionComponentServlet.processActionForward(ActionComponentServlet.java:104)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1529)
at com.integral.jsp.framework.IdcActionServlet.validateRequest(IdcActionServlet.java:331)
at com.integral.jsp.framework.IdcActionServlet.process(IdcActionServlet.java:139)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:487)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:262)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:198)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:2678)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2412)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:140)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:121)<2004/04/26 9:44:34:JST>
<Info> <T3Services> <Integral5: ReliableTopicConnection.createTopicSession: ABOUT
TO ACQUIRE LOCK...>
<2004/04/26 9:44:34:JST> <Info> <T3Services> <Integral5: ReliableTopicConnection.createTopicSession:
ABOUT TO ACQUIRE LOCK...>
<2004/04/26 9:44:34:JST> <Info> <T3Services> <Integral5: ReliableTopicConnection.createTopicSession:
INSIDE...>
<2004/04/26 9:44:34:JST> <Info> <T3Services> <Integral5: ReliableTopicConnection.createTopicSession:
INSIDE...>
An irrecoverable stack overflow has occurred.
Unexpected Signal : 11 occurred at PC=0xfb3d40cc
Function name=getValue (compiled Java code)
Library=(N/A)
Current Java thread:
Dynamic libraries:
0x10000 /opt/weblogic/jdk131/bin/../bin/sparc/native_threads/java
0xff350000 /usr/lib/libthread.so.1
0xff390000 /usr/lib/libdl.so.1
0xff200000 /usr/lib/libc.so.1
0xff330000 /usr/platform/SUNW,Ultra-80/lib/libc_psr.so.1
0xfe400000 /opt/weblogic/jdk131/jre/lib/sparc/client/libjvm.so
0xff2e0000 /usr/lib/libCrun.so.1
0xff1e0000 /usr/lib/libsocket.so.1
0xff100000 /usr/lib/libnsl.so.1
0xff0d0000 /usr/lib/libm.so.1
0xff310000 /usr/lib/libw.so.1
0xff0b0000 /usr/lib/libmp.so.2
0xff060000 /opt/weblogic/jdk131/jre/lib/sparc/native_threads/libhpi.so
0xff030000 /opt/weblogic/jdk131/jre/lib/sparc/libverify.so
0xfe7c0000 /opt/weblogic/jdk131/jre/lib/sparc/libjava.so
0xfe790000 /opt/weblogic/jdk131/jre/lib/sparc/libzip.so
0xfe2d0000 /usr/lib/locale/ja_JP.PCK/ja_JP.PCK.so.2
0xfe2b0000 /usr/lib/locale/ja_JP.PCK/methods_ja_JP.PCK.so.2
0xaf5e0000 /opt/weblogic/jdk131/jre/lib/sparc/libnet.so
0xaf360000 /usr/lib/nss_files.so.1
0xaf340000 /opt/weblogic/wlserver/lib/solaris/libmuxer.so
0xaf320000 /usr/ucblib/libucb.so.1
0xaf230000 /usr/lib/libresolv.so.2
0xaf140000 /usr/lib/libelf.so.1
0xa7080000 /opt/weblogic/wlserver/lib/solaris/oci920_8/libweblogicoci37.so
0xa6400000 /opt/oracle/product/lib32/libclntsh.so.9.0
0xaf010000 /usr/lib/libC.so.5
0xaef60000 /opt/oracle/product/lib32/libwtc9.so
0xaef40000 /usr/lib/libgen.so.1
0xaef20000 /usr/lib/libsched.so.1
0xaee60000 /usr/lib/libaio.so.1
0xaee40000 /usr/lib/librt.so.1
0xad760000 /opt/weblogic/jdk131/jre/lib/sparc/libioser12.so
Local Time = Mon Apr 26 09:44:34 2004
Elapsed Time = 67979
# HotSpot Virtual Machine Error : 11
# Error ID : 4F530E43505002BD 01
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
# Java VM: Java HotSpot(TM) Client VM (1.3.1_08-b03 mixed mode)
<2004/04/26 9:44:34:JST> <Info> <T3Services> <Integral5: ReliableTopicConnection.createTopicSession:
ABOUT TO ACQUIRE LOCK...>
ˆÙíI—¹Before debugging, look at these 20 or so search hits - at least one one of them looks like a possibility.
http://search.java.sun.com/search/java/index.jsp?qp=&nh=10&qt=4F530E43505002CC&col=javabugs&col=javaforums&x=27&y=9
Also - if the problem came on suddenly, what changed....environment, software, power, hardware, etc??? -
Help on JVM Crash with core dump on solaris - 1.5_17
Some times in my load test scenarios on sun os boxes JVM crashing with core dump. Here is some dump from the file
# An unexpected error has been detected by HotSpot Virtual Machine:
# SIGSEGV (0xb) at pc=0xfea07f40, pid=1564, tid=10
# Java VM: Java HotSpot(TM) Server VM (1.5.0_17-b04 mixed mode)
# Problematic frame:
# V [libjvm.so+0x207f40]
--------------- T H R E A D ---------------
Current thread (0x0014e220): JavaThread "CompilerThread1" daemon [_thread_in_native, id=10]
siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000000
Registers:
O0=0x00000010 O1=0x019c1960 O2=0x01e00ec0 O3=0x002bdc48
O4=0x01042c68 O5=0xc467eb4c O6=0xc467e330 O7=0x01042c68
G1=0x01e00ea0 G2=0xff014c94 G3=0x000000e6 G4=0x01c5a4e4
G5=0x01736e20 G6=0x00000000 G7=0xfb9e4200 Y=0x00000000
PC=0xfea07f40 nPC=0xfea07f44
--------------- S Y S T E M ---------------
OS: Solaris 10 5/08 s10s_u5wos_10 SPARC
Copyright 2008 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 24 March 2008
uname:SunOS 5.10 Generic_127127-11 sun4v (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:2.73 2.67 2.21
CPU:total 32 has_v8, has_v9, has_vis1, has_vis2, is_ultra3, is_sun4v, is_niagara1
Memory: 8k page, physical 8257536k(366576k free)
vm_info: Java HotSpot(TM) Server VM (1.5.0_17-b04) for solaris-sparc, built on Nov 10 2008 01:58:40 by unknown with unknown Workshop:0x550
Here is the stack dump of the kill quit thread
----------------- lwp# 10 / thread# 10 --------------------
ff2c5bf0 lwpkill (6, 0, ff2f2e10, ff2a8bd0, ffffffff, 6) + 8
ff2410f8 abort (7400, 1, 7c00, ad314, ff2f12d8, 0) + 110
fee7e58c __1cCosFabort6Fi_v_ (1, 0, ff013084, fefde000, 7d94, 7c00) + 58
fef0de48 __1cHVMErrorOreport_and_die6M_v_ (0, ff03a640, ff033ff4, 1, fee82c88, ff033ff4) + c84
fea74138 JVM_handle_solaris_signal (b, c467e2b0, c467dff8, 8000, ff032fa0, 14e220) + ab4
ff2c4b28 __sighndlr (b, c467e2b0, c467dff8, fea7364c, 0, 1) + c
ff2b9b00 call_user_handler (b, ffbffeff, c, 0, fb9e4200, c467dff8) + 3b8
fea07f40 __1cMPhaseChaitinFSplit6MI_I_ (c467ec2c, 0, 0, 3677ac, 398, c) + 3410
fea13c68 __1cMPhaseChaitinRRegister_Allocate6M_v_ (c467eb4c, e88, dc0, ff0137d8, c467fb14, 48d) + 720
fea17c64 __1cHCompileICode_Gen6M_v_ (c467f218, 9e0c, 9c00, fef56b15, 0, c467ec2c) + 2b0
fea7ff14 __1cHCompile2t5B6MpnFciEnv_pnKC2Compiler_pnIciMethod_ii_v_ (c467f218, 0, 346c8, 0, fef569b8, 0) + c08
fea75fb8 __1cKC2CompilerOcompile_method6MpnFciEnv_pnIciMethod_i_v_ (c467fb14, fef42a90, 1e40f58, 244, 346c8, d1800000) + b0
fea76b68 __1cNCompileBrokerZinvoke_compiler_on_method6FpnLCompileTask__v_ (908928, 14e7fc, 13c900, 14e220, fef57367, c467fb14) + 4cc
feb3357c __1cNCompileBrokerUcompiler_thread_loop6F_v_ (ff0330b8, 13c8a0, 14e220, c5e67700, 14e7f8, 0) + 44c
feadbd20 __1cKJavaThreadDrun6M_v_ (14e220, ff037040, 7820, 0, 7800, 9400) + 2b0
fee7e0a8 __1cG_start6Fpv_0_ (14e220, 61c, fefde000, 0, 5874, 5800) + 208
ff2c49fc lwpstart (0, 0, 0, 0, 0, 0)
Any idea on this dump, helps me a lot.
Thanks.[http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/crashes.html]
-
Weblogic Core Dumps on Solaris with Hotspot Server VM
For Sun Microsystems SPARC with Solaris 2.7 the supported platform is
"SunSoft SDK 1.3.1 JavaTM 2 Runtime Environment, Standard Edition (build
1.3.1-b24) Java HotSpotTM Server VM (build 1.3.1-b24, mixed mode)"
We have set the MaxPermSize as specified below and are still receiving
periodic core dumps in a production environment. The most recent of which
is "Unexpected Signal : 11 occurred at PC=0xee4b5d00 Function
name=JVM_CurrentTimeMillis" a reported bug on Sun's bug parade (Bug Id
#4488864). Are there some other settings that we should try? Is there a
more stable VM we should be using like "SunSoft SDK 1.3.0 JavaTM 2 Runtime
Environment, Standard Edition (build 1.3.0) "?
Help would be appreciated,
Thanks
Problems with JDK 1.3 crashing
If you have problems with OutOfMemory errors and the JVM crashing with JDK
1.3, try setting: -XX:MaxPermSize=128m. There is currently an open bug on
Sun's bug parade that describes this problem. See,
http://developer.java.sun.com/developer/bugParade/bugs/4390238.htmlFYI: Stability seems to have been achieved by setting:
-XX:MaxNewSize=64m
-XX:MaxPermSize=128m
"Paul Hamill" <[email protected]> wrote in message
news:[email protected]..
Unfortunately we tried switching to client and we still periodically getthe
core dumps.
"Dimitri Rakitine" <[email protected]> wrote in message
news:[email protected]..
I'd be interested to know this as well - so far the only stable option
that I know of is to use client JVM. Server Hotspot crashes, sooner
or later. 1.3.1-b24 server crashes as well, but not the client JVM.
Paul Hamill <[email protected]> wrote:
For Sun Microsystems SPARC with Solaris 2.7 the supported platform is
"SunSoft SDK 1.3.1 JavaTM 2 Runtime Environment, Standard Edition
(build
1.3.1-b24) Java HotSpotTM Server VM (build 1.3.1-b24, mixed mode)"
We have set the MaxPermSize as specified below and are still receiving
periodic core dumps in a production environment. The most recent of
which
is "Unexpected Signal : 11 occurred at PC=0xee4b5d00 Function
name=JVM_CurrentTimeMillis" a reported bug on Sun's bug parade (Bug Id
#4488864). Are there some other settings that we should try? Is
there
a
more stable VM we should be using like "SunSoft SDK 1.3.0 JavaTM 2Runtime
Environment, Standard Edition (build 1.3.0) "?
Help would be appreciated,
Thanks
Problems with JDK 1.3 crashing
If you have problems with OutOfMemory errors and the JVM crashing with
JDK
1.3, try setting: -XX:MaxPermSize=128m. There is currently an open bugon
Sun's bug parade that describes this problem. See,
http://developer.java.sun.com/developer/bugParade/bugs/4390238.html
Dimitri
Maybe you are looking for
-
The system is running on Windows 7, after the restore, all bookmarks and all history prior to the restore were missing
-
Material with Different grades
Dear Guru's, Here in my client, for one material they are having 5 different grades based on the quality, Like A,B,C,D & E How we need to maintain them in single mmr, because they are procuring @ different prices based on the quality. My PP guy dont
-
IWork apps cannot be opened in Lion.
Moving to a new MacBook Pro with 10.7.3 and iWork 09 apps "cannot be opened because of a problem".Trying to update iWork and keep getting message: "Alert - A newer version of Keynote is already installed" even after reinstalling iWork from the disk.
-
I have some question related show conn on ace the log like below: ACEAD#1/130# show conn de | be 2633 2633 1 in TCP 330 100.254.130.13:39560 100.254.16.11:389 ESTAB [ idle time : 00:33:21, byte count : 334 ] [ elapsed time: 00:48:35, packet count: 5
-
Finder can't show all the vnc machine connected to my network
i would like to know how to show my windows based machine running vnc server in the "shared" coloumn view in the finder. Finder can find an Ubuntu based machine running a vnc server but not my windows one. how to fix it? thanks too much