Who can give me a suggestion about java crash with Internal Error ID?

Question

I running my server performance test, but after 7 hours, the jvm crashed and give a java internal error ID with "*53484152454432554E54494D450E43505001A8*",
and I can not query any valuable information from Google by this internal Error ID. Who can give me a light?
I have run the performance test on the other machine with the same service logic, the other environment not crash, It's very strange!
h5. some content of The hs_err_xxx.log file:
# An unexpected error has been detected by HotSpot Virtual Machine:
# Internal Error (53484152454432554E54494D450E43505001A8), pid=16908, tid=1060293520
# Java VM: Java HotSpot(TM) Server VM (1.5.0_19-b02 mixed mode)
--------------- T H R E A D ---------------
Current thread (0x425782e8): JavaThread "11d1def534ea1be0:656f6049:124caa54ac8:49c0,EventTypeID[com.telware.protocol.cid.eve
nt.CIDContinueEvent#telware#1.0], #14" [_thread_in_Java, id=17068]
Stack: [0x3f2ac000,0x3f32d000), sp=0x3f32b2a0, free space=508k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x516863]
V [libjvm.so+0x1ca6cc]
V [libjvm.so+0x4a3c5f]
V [libjvm.so+0x43ceb6]
V [libjvm.so+0x43a0c4]
Other Threads:
0x08e0d850 VMThread [id=16914]
0x08e20e20 WatcherThread [id=16923]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
h6. Heap
par new generation total 98304K, used 36214K [0x49360000, 0x51360000, 0x51360000)
eden space 65536K, 29% used [0x49360000, 0x4a667c10, 0x4d360000)
from space 32768K, 51% used [0x4d360000, 0x4e3b5e58, 0x4f360000)
to space 32768K, 0% used [0x4f360000, 0x4f360000, 0x51360000)
concurrent mark-sweep generation total 1507328K, used 974545K [0x51360000, 0xad360000, 0xad360000)
concurrent-mark-sweep perm gen total 40152K, used 24021K [0xad360000, 0xafa96000, 0xb1360000)
h6. VM Arguments:
jvm_args: -Dfile.encoding=utf-8 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationConcurrentTime
-XX:+PrintGCApplicationStoppedTime -Xloggc:/home/sepcs/sep/oper/gc.log -Xms1600m -Xmx1600m -Xmn128m -XX:SurvivorRatio=2 -XX:+
UseFastAccessorMethods -XX:-UseFastEmptyMethods -XX:+UseFastJNIAccessors -XX:ParallelGCThreads=2 -XX:+UseConcMarkSweepGC -XX:
UseParNewGC -XX:CMSFullGCsBeforeCompaction=1 -XX:GCTimeRatio=19 -Xnoclassgc -XX:DisableExplicitGC -XX:CMSInitiatingOccupanc
yFraction=70 -XX:+UseCMSCompactAtFullCollection -XX:+CMSParallelRemarkEnabled -Dsep.home.dir=/home/sepcs/sep
java_command: com.telware.sep.system.SysAdmin -r -gc
Launcher Type: SUN_STANDARD
h6. Environment Variables:
JAVA_HOME=/usr/jdk/jdk1.5.0_19
PATH=/home/sepcs/sep/bin::/usr/jdk/jdk1.5.0_19/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
LD_LIBRARY_PATH=/usr/jdk/jdk1.5.0_19/jre/lib/i386/server:/usr/jdk/jdk1.5.0_19/jre/lib/i386:/usr/jdk/jdk1.5.0_19/jre/../lib/i3
86
SHELL=/bin/csh
DISPLAY=localhost:10.0
HOSTTYPE=i386-linux
OSTYPE=linux
MACHTYPE=i386
--------------- S Y S T E M ---------------
OS:Red Hat Enterprise Linux Server release 5.1 (Tikanga)
uname:Linux 2.6.18-53.el5 #1 SMP Wed Oct 10 16:34:02 EDT 2007 i686
libc:glibc 2.5 NPTL 2.5
rlimit: STACK 10240k, CORE infinity, NPROC 31726, NOFILE 1024, AS infinity
load average:2.05 2.35 2.23
CPU:total 2 (cores per cpu 2, threads per core 1) family 15 model 6 stepping 5, cmov, cx8, fxsr, mmx, sse, sse2, sse3
Memory: 4k page, physical 2009524k(54132k free), swap 1049k(1048k free)
vm_info: Java HotSpot(TM) Server VM (1.5.0_19-b02) for linux-x86, built on May 4 2009 01:43:34 by java_re with gcc 3.2.1-7a
(J2SE release)
time: Sat Nov 7 02:44:28 2009

Answer 1

Thank you very much first.
I have noted that the link [t-739313] but I can not found any valuable info for my exception :(
Yesterday night this crash has been rised again after my server running 1hours, I have analysis the core file, it's stack like this:
{color:#008080}(gdb) bt
#0 0xb7f4e402 in __kernel_vsyscall ()
#1 0x00488c10 in raise () from /lib/libc.so.6
#2 0x0048a521 in abort () from /lib/libc.so.6
#3 0xb7990f75 in os::abort () from /usr/jdk/jdk1.5.0_19/jre/lib/i386/server/libjvm.so
#4 0xb7a6f914 in VMError::report_and_die () from /usr/jdk/jdk1.5.0_19/jre/lib/i386/server/libjvm.so
#5 0xb77236cc in report_fatal () from /usr/jdk/jdk1.5.0_19/jre/lib/i386/server/libjvm.so
#6 0xb79fcc5f in SharedRuntime::continuation_for_implicit_exception ()
from /usr/jdk/jdk1.5.0_19/jre/lib/i386/server/libjvm.so
#7 0xb7995eb6 in JVM_handle_linux_signal () from /usr/jdk/jdk1.5.0_19/jre/lib/i386/server/libjvm.so
#8 0xb79930c4 in signalHandler () from /usr/jdk/jdk1.5.0_19/jre/lib/i386/server/libjvm.so
#9 <signal handler called>
#10 0x0960527c in ?? ()
#11 0x429f2160 in ?? ()
#12 0x429f2158 in ?? ()
#13 0x4cb72e48 in ?? ()
#14 0x4cb72f28 in ?? ()
#15 0x4cb72fe8 in ?? ()
#16 0x00000000 in ?? (){color}
I have print the stack from the core file, I found the crashed thread's state which indicted by hs_err_xxx.log is IN_JAVA and no stack information.
{color:#008080}Thread 12536: (state = IN_JAVA){color}
(no stack info.)
{color:#008080}{color:#000000}And from the gc.log, I found that the crash occured in the CMS phase, the last gc log is :
{color}5814.574: [GC [1 {color:#0000ff}CMS-initial-mark{color}: 829142K(1507328K)] 839697K(1622016K), 0.0281480 secs]
Total time for which application threads were stopped: 0.0284280 seconds
5814.602: [{color:#0000ff}CMS-concurrent-mark-start{color}]
Application time: 1.7093670 seconds
5816.312: [GC 5816.312: [ParNew: 108858K->8866K(114688K), 0.1133760 secs] 938001K->842222K(1622016K), 0.1135920 secs]
Total time for which application threads were stopped: 0.1141250 seconds
Application time: 1.4597160 seconds
5817.886: [GC 5817.886: [ParNew: 107170K->8372K(114688K), 0.0900130 secs] 940526K->844407K(1622016K), 0.0902690 secs]
Total time for which application threads were stopped: 0.0907340 seconds
Application time: 1.4513060 seconds
5819.428: [GC 5819.428: [ParNew: 106676K->8189K(114688K), 0.0871610 secs] 942711K->846695K(1622016K), 0.0873750 secs]
Total time for which application threads were stopped: 0.0878680 seconds
{color:#000000}All above is my found, but I am a newbie in JVM crash, so anybody can help me ?
and I am very interesting in the Internal Error ID ({color}{color}{color:#ff0000}*53484152454432554E54494D450E43505001A8*{color}{color:#008080}{color:#000000}), I think this ID hide a lot of valuable information for SUN JDK developer, who can interpret it? Thanks very much!
{color}{color}