Ttcserver coredumping

Hi,
We are observing that ttcserver process is corebumping.
And we are getting following error in our application logs:
==============================================================
2010-04-17 18:29:53: CARDMANAGER: ERROR: MonitorObj::CheckTTServer Error: ERROR in TTClientConnection.cpp, line 229: TTClientConnection::Connect() -- while connecting to driver
[TimesTen][TimesTen 7.0.5.16.0 ODBC Driver][TimesTen]TT10004: Unable to connect to daemon: Thread create failed -- file "db.c", lineno 9169, procedure "sbDbConnect"
*** ODBC Error/Warning = 08S01, TimesTen Error/Warning = 10004
*** (connect string <TTC_Server=172.16.17.5;TTC_Server_DSN=DG_630051;UID=kodiak;PWD=kodiak;TTC_Timeout=70>)
=============================================================
The remote connections are not blocked for a long time, I mean in between remote connection to the DB was going successful.
Following is the BT of the core file:
=============================================
(gdb) bt
#0 0x000000000042390e in HandoffConnect ()
#1 0x000000000040a081 in main ()
(gdb)
================================================
From the TimesTen logs we can see following error message being printe4d freqently.
====================================================================
12:50:19.22 Err : SRV: 2996: EventID=99| pthread_create() failed with System Error: 11
12:50:19.22 Err : SRV: 2996: EventID=99| pthread_create() failed with stacksize: 2097152
12:50:19.22 Err : SRV: 2996: EventID=99| Failed to create child server thread in main server
12:50:19.22 Info: SRV: 2996: EventID=59| Error creating separate server process. Not enough memory.
=====================================================================
Please help me to know the reason for this.
Regards
Pratheej

What is your OS?
At the time of the problem, how many users are connected to TimesTen? (have a look in the message log around this time (2010-04-17 18:29:53) and see what the value of nUsers is.
Is it possible you are validly out of memory - if you check system memory is there plenty showing as free?
Can you share the 'ulimit -a' and 'ulimit -Ha' outputs for the user the ttcserver daemon runs as?

Similar Messages

  • Coredump when adding new data to a document

    Hi,
    I have managed to get a coredump when adding data to a document,
    initially using the Python API but I can reproduce it with a dbxml script.
    I am using dbxml-2.2.13 on RedHat WS 4.0.
    My original application reads XML data from files, and adds them
    one at a time to a DbXML document using XmlModify.addAppendStep
    and XmlModify.execute. At a particular document (call it "GLU.xml") it
    segfaults during the XmlModify.execute call. It is not malformed data in
    the file, because if I remove some files that are loaded at an earlier stage,
    GLU.xml is loaded quite happily and the segfault happens later. Changing
    my application so that it exits just before reading GLU.xml, and loading GLU.xml's
    data into the container file using the dbxml shell's "append" command produces
    the same segfault. The stacktrace is below. Steps #0 to #7 inclusive are the
    same as the stacktrace I got when using the Python API.
    Can anyone give me any suggestions? I could send the dbxml container file and
    dbxml script to anyone who would be prepared to take a look at this problem.
    Regards,
    Peter.
    #0  ~NsEventGenerator (this=0x9ea32f8) at NsEventGenerator.cpp:110
    110                     _freeList = cur->freeNext;
    (gdb) where
    #0  ~NsEventGenerator (this=0x9ea32f8) at NsEventGenerator.cpp:110
    #1  0x009cacef in DbXml::NsPullToPushConverter8::~NsPullToPushConverter8$delete ()
        at /scratch_bernoulli/pkeller/dbxml-2.2.13/install/include/xercesc/framework/XMLRefInfo.hpp:144
    #2  0x00a5d03c in DbXml::NsDocumentDatabase::updateContentAndIndex (this=0x96b7a60,
        new_document=@0x96e3608, context=@0x96a3fc8, stash=@0x96a4098) at ../scoped_ptr.hpp:44
    #3  0x009a71b1 in DbXml::Container::updateDocument (this=0x96a71d0, txn=0x0, new_document=@0x96e3608,
        context=@0x96a3fc8) at shared_ptr.hpp:72
    #4  0x009b8465 in UpdateDocumentFunctor::method (this=0xb7d3a008, container=@0x96a71d0, txn=0x0, flags=0)
        at TransactedContainer.cpp:167
    #5  0x009b70c5 in DbXml::TransactedContainer::transactedMethod (this=0x96a71d0, txn=0x0, flags=0,
        f=@0xbff66500) at TransactedContainer.cpp:217
    #6  0x009b71e4 in DbXml::TransactedContainer::updateDocument (this=0x96a71d0, txn=0x0,
        document=@0x96e3608, context=@0x96a3fc8) at TransactedContainer.cpp:164
    #7  0x009d7616 in DbXml::Modify::updateDocument (this=0x96c1748, txn=0x0, document=@0xbff665b0,
        context=@0xbff669dc, uc=@0xbff669e4)
        at /scratch_bernoulli/pkeller/dbxml-2.2.13/dbxml/build_unix/../dist/../include/dbxml/XmlDocument.hpp:72
    #8  0x009d9c18 in DbXml::Modify::execute (this=0x96c1748, txn=0x0, toModify=@0x96a7280,
        context=@0xbff669dc, uc=@0xbff669e4) at Modify.cpp:743
    #9  0x009c1c35 in DbXml::XmlModify::execute (this=0xbff666c0, toModify=@0x96a7280, context=@0xbff669dc,
        uc=@0xbff669e4) at XmlModify.cpp:128
    #10 0x08066bda in CommandException::~CommandException ()
    #11 0x0805f64e in CommandException::~CommandException ()
    #12 0x08050c82 in ?? ()
    #13 0x00705de3 in __libc_start_main () from /lib/tls/libc.so.6
    #14 0x0804fccd in ?? ()
    Current language:  auto; currently c++

    Hi George,
    I can get the coredump with the following XML data (cut down from its original
    size of around 900Kb):
    <file name="GLU.xml">
    <_StorageUnit time="Wed Apr  5 11:06:49 2006" release="1.0.212"
    packageName="ccp.ChemComp" root="tempData" originator="CCPN Python XmlIO">
    <parent>
      <key1 tag="molType">protein</key1>
      <key2 tag="ccpCode">GLU</key2>
    </parent>
    <StdChemComp ID="1" code1Letter="E" stdChemCompCode="GLU" molType="protein" ccpCode="GLU" code3Letter="GLU" msdCode="GLU_LFOH" cifCode="GLU" merckCode="12,4477">
      <name>GLUTAMIC ACID</name>
      <commonNames>L-glutamic acid</commonNames>
    </_StorageUnit>
    <!--End of Memops Data-->
    </file>This happens when the data from 106 other files have been inserted beforehand
    (ranging in size from 1Kb to 140Kb). If I manipulate the order so that the above data
    is loaded earlier in the sequence, it inserts fine and I get the coredump when
    loading data from a different file.
    The actual XmlModify calls look something like:
      qry = mgr.prepare("/datapkg/dir[@name='dir1']/dir[@name='dir2']", qc)
      mdfy.addAppendStep(qry, XmlModify.Element, "",
                         '<file name='" + fileName + '">' +
                          data[pos:] + "</file>")
      mdfy.execute(XmlValue(doc), qc, uc)where data[pos:] points to the location in the mmap-ed file containing the
    above data just after the <?xml ...?> header.
    If you want to try to reproduce the crash at your end there are a couple of ways
    we could do it. I have just figured out that this forum software doesn't let me
    upload files or reveal my e-mail address in my profile, but you can contact me with
    username: pkeller; domain name: globalphasing.com and I can send the
    data to you.
    Regards,
    Peter.

  • Cannot figure out how to open and view a coredump

    I've been having frequent issues on not just Linux but also Windows, where the symptoms are always the same: all the applications notify me that they crash, and for Linux, the screen freezes, while on Windows there's a blue screen in addition. Trying to analyze Windows' dump of course proved futile. However, while I did get a core dump on Linux in /var/lib/systemd/coredumps, the file within the xz archive, when opened with a log viewer like KSystemLog, shows timestamps but also binary code in the description field. The `coredumpctl` utility is little help to me too. Not even a Google search on "systemd core dump view" gave any good results, seeming to discard the keyword `view` from the query.
    So my question is: how do I view a core dump or get coredumpctl to analyze it? `coredumpctl gdb` returns nothing

    $ coredumpctl gdb <keyword>
    e.g.
    $ coredumpctl gdb vlc
    will show the latest vlc dump file with gdb.

  • JVM Coredump with libengine.so as the problematic frame

    Hi Experts,
    Our application is running over Tuxedo 9.0, and JVM 1.5.0_07-b03 mixed mode. We are experiencing coredumps frequently in production environment.
    The hs_err file shows the following information:
    # An unexpected error has been detected by HotSpot Virtual Machine:
    # SIGSEGV (0xb) at pc=0x4aa0daa8, pid=21887, tid=45
    # Java VM: Java HotSpot(TM) Server VM (1.5.0_07-b03 mixed mode)
    # Problematic frame:
    # C [libengine.so.71+0x10daa8] gprematch+0x16c
    --------------- T H R E A D ---------------
    Current thread (0x006ce740): JavaThread "[ACTIVE] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_in_native, id=45]
    siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x3d22686b
    Registers:
    O0=0x4997d118 O1=0x4997d5ce O2=0x4a9ebc80 O3=0x02f7a7a0
    O4=0x0017969c O5=0x00000000 O6=0x4997c7f8 O7=0x4aa0d9a0
    G1=0x00efa000 G2=0x02f71dc8 G3=0x4997d07c G4=0x02f71dc8
    G5=0xff018640 G6=0x00000000 G7=0x4bbd5200 Y=0x00000008
    PC=0x4aa0daa8 nPC=0x4aa0daac
    Top of Stack: (sp=0x4997c7f8)
    0x4997c7f8: 00000000 00000000 00000000 02f71dc8
    0x4997c808: 0000478c 00004400 00004bc0 3d226869
    0x4997c818: 02f72838 4997d5ce 00000000 4ab59ebc
    0x4997c828: 3d22686b 02f71dc8 4997d0c8 4acc1714
    0x4997c838: 00000000 7465642f 00000000 752f7574
    0x4997c848: 66382f68 746d6c2f 00000000 02f71dc8
    0x4997c858: 6e742e6a 73700a2a 2a2a2045 4e442044
    0x4997c868: 45425547 20203230 31302d30 342d3031
    Instructions: (pc=0x4aa0daa8)
    0x4aa0da98: f0 27 6b 60 ee 07 bf ec b8 05 e0 02 f8 27 bf ec
    0x4aa0daa8: f2 0d e0 02 80 a6 60 03 12 80 00 09 d0 07 bf fc
    Stack: [0x49880000,0x49980000), sp=0x4997c7f8, free space=1009k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    C [libengine.so.71+0x10daa8] gprematch+0x16c
    C [libwsc.so.71+0x4171c] _tmtrace+0x340
    C [libwsc.so.71+0x108c8] tpalloc+0x144
    C [libtti_tuxedo_wsc.so+0x3330] Java_terrapin_tuxedo_TpType_tpalloc+0xc0
    j terrapin.tuxedo.TpType.tpalloc(Ljava/lang/String;Ljava/lang/String;I)J+1817098320
    j terrapin.tuxedo.TpType.tpalloc(Ljava/lang/String;Ljava/lang/String;I)J+0
    j terrapin.tuxedo.Fml32.<init>()V+13
    v ~C2IAdapter
    VM Arguments:
    jvm_args: -Dinfo=TC5.0.0.37_7003_/totalcarebin1/tcaredomain115 -Xms2048m -Xmx2048m -XX:PermSize=512m -XX:MaxPermSize=512m -verbose:gc -XX:+DisableExplicitGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -Xloggc:/totalcarelog2/gc_20100401060835.log -XX:+UseParNewGC -XX:NewSize=256m -XX:MaxNewSize=256m -XX:+UseCMSCompactAtFullCollection -verbose:jni -Xss1024K -da -Dplatform.home=/totalcarebin1/3p/bea/weblogic91 -Dwls.home=/totalcarebin1/3p/bea/weblogic91/server -Dwli.home=/totalcarebin1/3p/bea/weblogic91/integration -Dweblogic.management.discover=true -Dweblogic.ProductionModeEnabled=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/totalcarebin1/3p/bea/patch_weblogic910/profiles/default/sysext_manifest_classpath -Dcom.sun.xml.namespace.QName.useCompatibleSerialVersionUID=1.0 -Dweblogic.management.username=system -Dweblogic.management.password=systempass -Dweblogic.Name=tcareserver -Djava.security.policy=/totalcarebin1/3p/bea/weblogic91/server/lib/weblogic.policy -Djava.security.auth.login.config==/totalcarebin1/tcaredomain115/serverclasses/propfiles/totalcare_jaas.config
    java_command: weblogic.Server
    Launcher Type: SUN_STANDARD
    Environment Variables:
    JAVA_HOME=/totalcarebin1/3p/jdk1.5.0_07
    PATH=.:/usr/dt/bin:/usr/ccs/bin:/usr/local/bin:/totalcarebin1/3p/jdk1.5.0_07/bin:/bin:/bin:/usr/bin:/usr/sbin:/etc:/usr/local/gnu/bin:/opt/SUNWste/bin:/usr/openwin/bin:/opt/SUNWspro/bin:/totalcarebin1/3p/tuxedo9/bin:/usr/ucb:/usr1/arbor/ccbin:/totalcarebin1/3p/bea/weblogic91/server/bin:/totalcarebin1/3p/jdk1.5.0_07/bin:/totalcarebin1/3p/bea/weblogic91/server/bin:/totalcarebin1/3p/jdk1.5.0_07/jre/bin:/totalcarebin1/3p/jdk1.5.0_07/bin:/u01/app/oracl1/product/10.2/cl_1/bin:/totalcarebin1/3p/perl/bin:/sbin:/usr/sbin:/totalcarebin1/3p/jdk1.5.0_07/bin:/totalcarebin1/tcaredomain115/util/unix:/totalcarebin1/3p/tuxedo9/bin:/usr/sfw/bin:/usr/local/bin:/usr/ccs/bin:/usr/local/sbin:/usr/ccs/bin:/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/ccs/bin:/opt/SUNWexplo/bin:/usr/sfw/bin:/usr/sfw/sbin:/usr/local/sbin:/usr/cluster/bin:/usr/cluster/sbin
    LD_LIBRARY_PATH=/totalcarebin1/3p/jdk1.5.0_07/jre/lib/sparc/server:/totalcarebin1/3p/jdk1.5.0_07/jre/lib/sparc:/totalcarebin1/3p/jdk1.5.0_07/jre/../lib/sparc:/usr/local/lib:/usr/lib:/usr/dt/bin:/totalcarebin1/3p/jdk1.5.0_07/lib:/usr/openwin/lib:/opt/SUNWspro/lib:/usr/atria/lib:/totalcarebin1/3p/tuxedo9/lib:/totalcarebin1/tcaredomain115/serverclasses/3p/terrapin/lib32:/totalcarebin1/3p/bea/weblogic91/server/lib/solaris:/totalcarebin1/3p/bea/weblogic91/server/native/solaris/sparc:/totalcarebin1/3p/bea/weblogic91/server/native/solaris/sparc64
    SHELL=/bin/ksh
    Signal Handlers:
    SIGSEGV: [libjvm.so+0x6f60b8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
    SIGBUS: [libjvm.so+0x6f60b8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
    SIGFPE: [libjvm.so+0x276398], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
    SIGPIPE: [libjvm.so+0x276398], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
    SIGILL: [libjvm.so+0x276398], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
    SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
    SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
    SIGHUP: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
    SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
    SIGQUIT: [libjvm.so+0x66a38c], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
    SIGTERM: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
    --------------- S Y S T E M ---------------
    OS: Solaris 10 6/06 s10s_u2wos_09a SPARC
    Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
    Use is subject to license terms.
    Assembled 09 June 2006
    uname:SunOS 5.10 Generic_137111-03 sun4u (T2 libthread)
    rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
    load average:0.39 0.57 0.44
    CPU:total 2 has_v8, has_v9, has_vis1, has_vis2, is_ultra3
    Memory: 8k page, physical 16777216k(11833704k free)
    vm_info: Java HotSpot(TM) Server VM (1.5.0_07-b03) for solaris-sparc, built on May 3 2006 01:22:35 by unknown with unknown Workshop:0x550
    The PSTACK shows:
    core 'core.1stApril' of 21887:     /totalcarebin1/3p/jdk1.5.0_07/bin/java -Dinfo=TC5.0.0.37_7003_/totalca
    ----------------- lwp# 1 / thread# 1 --------------------
    ff2c6510 ___lwp_cond_wait (3bbf0, 3bbd8, 0, 0, 0, 1) + 8
    feab4f98 __1cNObjectMonitorEwait6MxipnGThread__v_ (136068, 94c8, 8c00, fefc4000, 3a950, 136088) + 59c
    feab61a8 __1cSObjectSynchronizerEwait6FnGHandle_xpnGThread__v_ (ffbfaeb0, 8784, 0, 3a950, 8400, fefc4000) + e8
    feab5e58 JVM_MonitorWait (0, 3b534, 3a950, ff018ffc, 0, 1) + 31c
    f900c280 * java/lang/Object.wait(J)V+99
    f900c224 * java/lang/Object.wait(J)V+0
    f9005764 * java/lang/Object.wait()V+2 (line 474)
    f9005764 * weblogic/t3/srvr/T3Srvr.waitForDeath()V+46 (line 729)
    f9005764 * weblogic/t3/srvr/T3Srvr.run([Ljava/lang/String;)I+120 (line 380)
    f90058b8 * weblogic/Server.main([Ljava/lang/String;)V+43 (line 67)
    f9000218 * StubRoutines (1)
    fe99add4 __1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_ (1, 3a950, ffbfb1b8, ffbfb0e8, 4, ffbfb238) + 5a0
    feadf258 jni_CallStaticVoidMethod (ff018ffc, 3b52c, 3b128, 3a950, 3b120, 3ab20) + 500
    00012224 main     (3bb3c, 348da, ff0108a8, 1d8, fead497c, 1) + 115c
    000110b0 _start   (0, 0, 0, 0, 0, 0) + 108
    I appreciate your comments in advance, please let me know if you require further information.
    Thanks,
    Edited by: user3587921 on Apr 6, 2010 8:01 PM                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

    Hi,
    Unfortunately it appears you are calling Tuxedo from Java using JNI. Although that should work, it isn't something that Oracle supports. My guess is that you are experiencing memory corruption problems due to improper handling of allocated buffers. This is part of the reason we don't support JNI as the issues related to memory management can be very complex. My suggestion is that you go to whomever has provided you the JNI code for support (Comverse?). Alternatively you could look at the various Java integration products Oracle provides for Tuxedo. The Java integration to Tuxedo options are Jolt, WebLogic Tuxedo Connector (WTC - a component of WLS), and the new Tuxedo JCA Adapter. Although if you are using a packaged application like Comverse Total Care, you'll really need to talk with them as you're unlikely to be able to switch to an Oracle supported mechanism without their help or changes on their part.
    Regards,
    Todd Little
    Oracle Tuxedo Chief Architect

  • How to force CoreDump in the Kernel source code

    Hi,
    Anyone know, how to force coredump in the Kernel source code?. And also to collect all information related to CPU Registers, CPU IRAM, Cache Data and complete SDRAM content.

    pull your RAM out....
    Wouldn't recommend doing that really...... And why would you want to cause a kernel panic?
    EDIT: Spelling sucked...
    Last edited by crankyadmin (2009-09-28 22:02:14)

  • Ttcserver restarts when there are more than 5 c/s connections

    Hi,
    I am having some problem with C/S connection.
    When more than 5 c/s connections are connected to datastore, ttcserver restarts.
    No applications are running. Only tried to connection with c/s mode.
    Message logs are like below.
    ## tterrors.log
    2010-04-21 15:52:56.36 Warn: : 4891: TimesTen Daemon Release 11.2.1.4.0.tt1121 started.
    16:11:46.18 Warn: SRV: 4902: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.18 Warn: SRV: 4917: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.18 Warn: SRV: 4925: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.18 Warn: SRV: 4913: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.18 Warn: SRV: 4921: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.18 Warn: SRV: 4909: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.29 Err : SRV: 4926: EventID=99| ttShmCreate: shmget(1560567862, 1048576, flags) failed with system error 17
    ## ttmesg.log
    16:11:45.16 Info: SRV: 4925: EventID=16| Connect succeeded from client: jjwas1 (was1); IP address: xxx.xx.xx.82; Client PID: 4143; DSN: tt; UID: dba
    2010-04-21 16:11:46.18 Info: : 4891: maind 6: socket closed, calling recovery (last cmd was 8)
    16:11:46.18 Warn: SRV: 4902: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    2010-04-21 16:11:46.18 Info: : 4891: Starting daRecovery for 4896
    16:11:46.18 Warn: SRV: 4917: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    2010-04-21 16:11:46.18 Info: : 4891: Finished daRecovery for pid 4896.
    16:11:46.18 Warn: SRV: 4925: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    2010-04-21 16:11:46.18 Info: : 4891: maind 10: socket closed, calling recovery (last cmd was 9)
    16:11:46.18 Warn: SRV: 4913: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    16:11:46.18 Warn: SRV: 4921: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    2010-04-21 16:11:46.18 Info: : 4891: Starting daRecovery for 4896
    16:11:46.18 Warn: SRV: 4909: EventID=99| Main server has unexpectedly exited; child server exiting and client connection terminated
    2010-04-21 16:11:46.18 Info: : 4891: 4896 ------------------: TimesTen Server exited
    2010-04-21 16:11:46.18 Info: : 4891: Spawning '/memdb/TimesTen/tt1121/bin/ttcserver'
    2010-04-21 16:11:46.18 Info: : 4891: Spawned new process, idx 5, handle 0x133e=4926
    2010-04-21 16:11:46.18 Info: : 4891: Finished daRecovery for pid 4896.
    2010-04-21 16:11:46.19 Info: : 4891: child 4896 gone, terminated with signal 11, core dumped
    When i faced it at first, thought shared memory will be a problem. But enough physical memory and enough shmmax(or other kernel).
    Also created other datastore with large size of perm size and worked ok.
    Any idea why this is happening? Seems like log mesg with -verbose mode does not gives any clue to me....:(
    * x86-64bit Solaris10
    * ttversio : TimesTen Release 11.2.1.4.0 (64 bit Solaris/x86)
    Best regards,

    Hi Adam,
    Yes, I opened a SR with this issue.
    Was wondering anyone else had a similar exprience.
    regards,

  • How to use the kernel coredump

    My system generated a kernel coredump. I want to know the reason.

    Hi!
    Sorry, it's not really that easy! The system will coredump if it detects a system problem that would cause data corruption were it to continue, for example an invalid kernel pointer.
    Have a look at "Panic: UNIX Systems Crash Dump Analysis", ISBN 0-13-149386-8, written by SUN, and then post some questions!
    Perhaps simpler would be to ensure your kernel patches are up to date. The chances are someone else has already reported the problem and it has been fixed by a patch.
    Regards,
    Ralph
    SUN DTS

  • Critical CoreDumps in our Application Servers with Coherence

    Dear Support team,
    We are facing few issues with our C++ AppServer binary which are Coherence enabled. We are using the libcoherence.so provided by Oracle. We find the dumps are occuring only when Coherence is enabled and our binary looks good when Coherence is disabled. The below coredump occurred somewhere internally within Coherence, please shed some lights on the below dump and advise.
    ====================
    t@9 (l@9) terminated by signal SEGV (no mapping at the fault address)
    Current function is coherence::lang::Object::_attach
    641 LifeCycle nState(m_atomicLifeCycle.peek());
    (dbx) where
    current thread: t@9
    =>[1] coherence::lang::Object::_attach(this = 0xf71fb328, fEscaped = true), line 641 in "Object.hpp"
    [2] coherence::lang::Object::wait(0x29bfe48, 0xfa63d350, 0xfe275244, 0x1a81, 0x29c27d0, 0xfdf88a80), at 0xfab906d4
    [3] coherence::component::util::Daemon::onWait(0x29bbc30, 0xf71fb510, 0x29bfe48, 0xf71fb510, 0xfdf88a80, 0xf71fb500), at 0xfa757470
    [4] coherence::component::util::Daemon::run(0x29bbc30, 0xfe6ea03c, 0xfe6ea03c, 0xf71fb5a0, 0x1, 0xf71fb5c0), at 0xfa75800c
    [5] coherence::lang::Thread::run(0xf71fbed0, 0x29bbf40, 0xfdf88a80, 0x29c0b20, 0xf71fbf28, 0xfe4eab9c), at 0xfa9d9918
    [6] coh_startPosix(0x17dfd38, 0xf71fc000, 0x0, 0x0, 0xfa629ee8, 0xfe275df0), at 0xfa614f74
    (dbx) dump
    fEscaped = true
    nState = UNION
    this = 0xf71fb328
    cHandle = 4096U
    (dbx)
    ====================

    Can you please provide the following information:
    - Coherence version
    - OS version
    - Compiler version
    I would also suggest that you validate that you are on the minimum supported compiler version paying particular attention the patch level, see http://docs.oracle.com/cd/E24290_01/coh.371/e22839/gs_install.htm#BABDCDFG.
    thanks,
    Mark
    Oracle Coherence

  • Coredump in Proc when connect Oracle

    Hello
    I am programming in pro*C in Solaris with a Oracle 8.1.7 database.
    I have no problem to precompile and compile programs but when i execute then i have a coredump when i execute the exec sql connect instruction.
    I debug the applications with the dbx utility and i get the next:
    signal SEGV (no mapping at the fault address) in kouogini at 0xfed74404
    0xfed74404: kouogini+0x023c: ld [%o0], %o0
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfed74b44: kouoini+0x00a0: call kouogini
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfed39db4: kpuinit0+0x07bc: jmpl %o7, %o7
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfed39538: kpuinit+0x0030: call PROCEDURELINKAGE_TABLE_+0x22d4 [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecb24d0: sqgctx+0x00f8: call PROCEDURELINKAGE_TABLE_+0x69c [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecb23a4: sqgrct+0x003c: call PROCEDURELINKAGE_TABLE_+0x3a8 [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecabdd8: sqlcmex+0x0214: call PROCEDURELINKAGE_TABLE_+0x39c [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecac490: sqlcxt+0x0058: call sqlcmex
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0x000109e0: main+0x0150: call sqlcxt [PLT]
    Please, help me
    thanks
    Sergio saiz.

    The Member Feedback forum is for suggestions and feedback for OTN Developer Services. This forum is not monitored by Oracle support or product teams and so Oracle product and technology related questions will not be answered. We recommend that you post this thread to the Database - General forum. General Database Discussions
    OTN

  • Coredump on using mtmalloc on Solaris 8

    Hi ,
    We are trying to use mtmalloc with our application to compare
    the performance with other allocators.
    The tests go through fine on Solaris 9m, but coredumps immediately
    on Solaris 8 in a free call.
    Can you point out the differences in the mtmalloc between solaris 8 and 9
    that can be helpful to resolve this issue.
    Any pointers would be really helpful.
    regards,
    Johar

    Johar,
    1. Before you start with the tests patch your OS with the latest patch cluster
    http://patchpro.sun.com/
    "Solaris Recommended Patch Cluster"
    2. Are you running a binary compiled on Solaris 9 on Solaris 8?
    3. To level the playground you should use the new 1 to 1 thread model in Solaris 8
    which is not the default thread model (unfortunately).
    4. See thread 16910 in this forum
    http://forum.sun.com/thread.jsp?forum=4&thread=16910
    0. You might want to read this whitepaper too:
    http://wwws.sun.com/software/whitepapers/solaris9/multithread.pdf
    HTH,
    vladimir

  • Coredumpctl coredumps on its own coredump :~-(

    Please excuse this stupid question but "almighty google" has problems to find links to sentences with recurrent structure.
    My systeminfo is:
    Linux GarAus 3.17.4-1-ARCH #1 SMP PREEMPT Fri Nov 21 21:14:42 CET 2014 x86_64 GNU/Linux
    and my systemd folder looks actually like this:
    [134] yousry@GarAus> ls -lt /var/lib/systemd/coredump
    insgesamt 472460
    -rw-r-----+ 1 root root 73211 28. Nov 17:05 core.coredumpctl.1000.41e5176f889441929a1e6f2140a9fb88.6889.1417190748000000.lz4
    -rw-r-----+ 1 root root 73317 28. Nov 17:05 core.coredumpctl.1000.41e5176f889441929a1e6f2140a9fb88.6887.1417190748000000.lz4
    -rw-r-----+ 1 root root 73390 28. Nov 17:05 core.coredumpctl.1000.41e5176f889441929a1e6f2140a9fb88.6885.1417190747000000.lz4
    -rw-r-----+ 1 root root 73323 28. Nov 17:03 core.coredumpctl.1000.41e5176f889441929a1e6f2140a9fb88.6855.1417190618000000.lz4
    if I try to debug the last coredump with
    coredumpctl gdb
    I get an ugly stack smashing:
    yousry@GarAus> coredumpctl gdb
    PID: 7299 (coredumpctl)
    UID: 1000 (yousry)
    GID: 100 (users)
    Signal: 6 (ABRT)
    Timestamp: Fr 2014-11-28 17:22:46 CET (11s ago)
    Command Line: coredumpctl gdb
    Executable: /usr/bin/coredumpctl
    Control Group: /user.slice/user-1000.slice/session-c2.scope
    Unit: session-c2.scope
    Slice: user-1000.slice
    Session: c2
    Owner UID: 1000 (yousry)
    Boot ID: 41e5176f889441929a1e6f2140a9fb88
    Machine ID: fd7c3d7a1dc2498e882c12e8c25400e1
    Hostname: GarAus
    Coredump: /var/lib/systemd/coredump/core.coredumpctl.1000.41e5176f889441929a1e6f2140a9fb88.7299.1417191766000000.lz4
    Message: Process 7299 (coredumpctl) of user 1000 dumped core.
    *** stack smashing detected ***: coredumpctl terminated
    ======= Backtrace: =========
    /usr/lib64/libc.so.6(+0x7361e)[0x7f80655dd61e]
    /usr/lib64/libc.so.6(__fortify_fail+0x37)[0x7f8065662c77]
    /usr/lib64/libc.so.6(__fortify_fail+0x0)[0x7f8065662c40]
    coredumpctl(+0xc340)[0x7f8066670340]
    coredumpctl(+0x52b8)[0x7f80666692b8]
    /usr/lib64/libc.so.6(__libc_start_main+0xf0)[0x7f806558a040]
    coredumpctl(+0x567c)[0x7f806666967c]
    ======= Memory map: ========
    7f8060bfd000-7f8060c13000 r-xp 00000000 08:05 35419847 /usr/lib/libgcc_s.so.1
    7f8060c13000-7f8060e12000 ---p 00016000 08:05 35419847 /usr/lib/libgcc_s.so.1
    7f8060e12000-7f8060e13000 rw-p 00015000 08:05 35419847 /usr/lib/libgcc_s.so.1
    7f8060e13000-7f8060e1e000 r-xp 00000000 08:05 35392811 /usr/lib/libnss_files-2.20.so
    7f8060e1e000-7f806101e000 ---p 0000b000 08:05 35392811 /usr/lib/libnss_files-2.20.so
    7f806101e000-7f806101f000 r--p 0000b000 08:05 35392811 /usr/lib/libnss_files-2.20.so
    7f806101f000-7f8061020000 rw-p 0000c000 08:05 35392811 /usr/lib/libnss_files-2.20.so
    7f8061020000-7f8061660000 r--s 00000000 08:05 46671573 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]cc806c8b.journal
    7f8061660000-7f8061ca0000 r--s 00000000 08:05 46661884 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]762dc2f0.journal
    7f8061ca0000-7f80624a0000 r--s 00000000 08:05 46662534 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/user-120.journal
    7f80624a0000-7f8062ae0000 r--s 00000000 08:05 46671405 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]61688330.journal
    7f8062ae0000-7f8063120000 r--s 00000000 08:05 46671668 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/user-1000.journal
    7f8063120000-7f8063760000 r--s 00000000 08:05 46671571 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]b6018306.journal
    7f8063760000-7f8063da0000 r--s 00000000 08:05 46671629 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]d48a511b.journal
    7f8063da0000-7f80643e0000 r--s 00000000 08:05 46661665 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/system.journal
    7f80643e0000-7f8064a20000 r--s 00000000 08:05 46671685 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]8d6a109b0d7.journal
    7f8064a20000-7f8065060000 r--s 00000000 08:05 46671660 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]e03c2a08.journal
    7f8065060000-7f8065358000 r--p 00000000 08:05 35422454 /usr/lib/locale/locale-archive
    7f8065358000-7f8065369000 r-xp 00000000 08:05 35443880 /usr/lib/libgpg-error.so.0.13.0
    7f8065369000-7f8065568000 ---p 00011000 08:05 35443880 /usr/lib/libgpg-error.so.0.13.0
    7f8065568000-7f8065569000 r--p 00010000 08:05 35443880 /usr/lib/libgpg-error.so.0.13.0
    7f8065569000-7f806556a000 rw-p 00011000 08:05 35443880 /usr/lib/libgpg-error.so.0.13.0
    7f806556a000-7f8065704000 r-xp 00000000 08:05 35392857 /usr/lib/libc-2.20.so
    7f8065704000-7f8065903000 ---p 0019a000 08:05 35392857 /usr/lib/libc-2.20.so
    7f8065903000-7f8065907000 r--p 00199000 08:05 35392857 /usr/lib/libc-2.20.so
    7f8065907000-7f8065909000 rw-p 0019d000 08:05 35392857 /usr/lib/libc-2.20.so
    7f8065909000-7f806590d000 rw-p 00000000 00:00 0
    7f806590d000-7f8065924000 r-xp 00000000 08:05 35392818 /usr/lib/libpthread-2.20.so
    7f8065924000-7f8065b23000 ---p 00017000 08:05 35392818 /usr/lib/libpthread-2.20.so
    7f8065b23000-7f8065b24000 r--p 00016000 08:05 35392818 /usr/lib/libpthread-2.20.so
    7f8065b24000-7f8065b25000 rw-p 00017000 08:05 35392818 /usr/lib/libpthread-2.20.so
    7f8065b25000-7f8065b29000 rw-p 00000000 00:00 0
    7f8065b29000-7f8065b30000 r-xp 00000000 08:05 35392870 /usr/lib/librt-2.20.so
    7f8065b30000-7f8065d2f000 ---p 00007000 08:05 35392870 /usr/lib/librt-2.20.so
    7f8065d2f000-7f8065d30000 r--p 00006000 08:05 35392870 /usr/lib/librt-2.20.so
    7f8065d30000-7f8065d31000 rw-p 00007000 08:05 35392870 /usr/lib/librt-2.20.so
    7f8065d31000-7f8065e08000 r-xp 00000000 08:05 35436468 /usr/lib/libgcrypt.so.20.0.2
    7f8065e08000-7f8066008000 ---p 000d7000 08:05 35436468 /usr/lib/libgcrypt.so.20.0.2
    7f8066008000-7f8066009000 r--p 000d7000 08:05 35436468 /usr/lib/libgcrypt.so.20.0.2
    7f8066009000-7f8066012000 rw-p 000d8000 08:05 35436468 /usr/lib/libgcrypt.so.20.0.2
    7f8066012000-7f806601c000 r-xp 00000000 08:05 35436178 /usr/lib/liblz4.so.1.4.0
    7f806601c000-7f806621b000 ---p 0000a000 08:05 35436178 /usr/lib/liblz4.so.1.4.0
    7f806621b000-7f806621c000 r--p 00009000 08:05 35436178 /usr/lib/liblz4.so.1.4.0
    7f806621c000-7f806621d000 rw-p 0000a000 08:05 35436178 /usr/lib/liblz4.so.1.4.0
    7f806621d000-7f806623f000 r-xp 00000000 08:05 35399308 /usr/lib/liblzma.so.5.0.7
    7f806623f000-7f806643e000 ---p 00022000 08:05 35399308 /usr/lib/liblzma.so.5.0.7
    7f806643e000-7f806643f000 r--p 00021000 08:05 35399308 /usr/lib/liblzma.so.5.0.7
    7f806643f000-7f8066440000 rw-p 00022000 08:05 35399308 /usr/lib/liblzma.so.5.0.7
    7f8066440000-7f8066462000 r-xp 00000000 08:05 35392813 /usr/lib/ld-2.20.so
    7f806664f000-7f8066651000 rw-p 00000000 00:00 0
    7f8066651000-7f8066652000 r--s 00000000 08:05 46671573 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]cc806c8b.journal
    7f8066652000-7f8066653000 r--s 00000000 08:05 46661884 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]762dc2f0.journal
    7f8066653000-7f8066654000 r--s 00000000 08:05 46662534 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/user-120.journal
    7f8066654000-7f8066655000 r--s 00000000 08:05 46671405 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]61688330.journal
    7f8066655000-7f8066656000 r--s 00000000 08:05 46671668 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/user-1000.journal
    7f8066656000-7f8066657000 r--s 00000000 08:05 46671571 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]b6018306.journal
    7f8066657000-7f8066658000 r--s 00000000 08:05 46671629 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]d48a511b.journal
    7f8066658000-7f8066659000 r--s 00000000 08:05 46661665 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/system.journal
    7f8066659000-7f806665a000 r--s 00000000 08:05 46671685 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]8d6a109b0d7.journal
    7f806665a000-7f806665b000 r--s 00000000 08:05 46671660 /var/log/journal/fd7c3d7a1dc2498e882c12e8c25400e1/[email protected]e03c2a08.journal
    7f806665b000-7f8066661000 rw-p 00000000 00:00 0
    7f8066661000-7f8066662000 r--p 00021000 08:05 35392813 /usr/lib/ld-2.20.so
    7f8066662000-7f8066663000 rw-p 00022000 08:05 35392813 /usr/lib/ld-2.20.so
    7f8066663000-7f8066664000 rw-p 00000000 00:00 0
    7f8066664000-7f806667e000 r-xp 00000000 08:05 35443628 /usr/bin/coredumpctl
    7f806667f000-7f8066680000 r--p 0001a000 08:05 35443628 /usr/bin/coredumpctl
    7f8066680000-7f8066681000 rw-p 0001b000 08:05 35443628 /usr/bin/coredumpctl
    7f8066caf000-7f8066cd0000 rw-p 00000000 00:00 0 [heap]
    7fff6be7b000-7fff6be9c000 rw-p 00000000 00:00 0 [stack]
    7fff6bffc000-7fff6bffe000 r--p 00000000 00:00 0 [vvar]
    7fff6bffe000-7fff6c000000 r-xp 00000000 00:00 0 [vdso]
    ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
    zsh: abort (core dumped) coredumpctl gdb
    Of curse the real reason is the crash of another small application. I'm using the default config of coredumpctl:   
    [134] yousry@GarAus> cat /etc/systemd/coredump.conf
    # This file is part of systemd.
    # systemd is free software; you can redistribute it and/or modify it
    # under the terms of the GNU Lesser General Public License as published by
    # the Free Software Foundation; either version 2.1 of the License, or
    # (at your option) any later version.
    # See coredump.conf(5) for details
    [Coredump]
    #Storage=external
    #Compress=yes
    #ProcessSizeMax=2G
    #ExternalSizeMax=2G
    #JournalSizeMax=767M
    #MaxUse=
    #KeepFree=
    Any help would be greatly appreciated!
    TIA

    hmm... I've experienced that as well, and from what I could tell it's related to lz4, specifically downgrading to lz4-123 fixed it for me. See https://bugs.archlinux.org/task/42944
    Edit:
    falconindy wrote:A backtrace would be useful. debug symbols can be grabbed from my repo
    Oh, that's what I've been looking for! Need to write that down. But there doesn't seem to be anything systemd-217-7 there, only up to 217-6?
    Last edited by jjacky (2014-11-28 18:18:40)

  • Coredump in pro*c program when i connect oracle

    Hello
    I am programming in pro*C in Solaris with a Oracle 8.1.7 database.
    I have no problem to precompile and compile programs but when i execute then i have a coredump when i execute the exec sql connect instruction.
    I debug the applications with the dbx utility and i get the next:
    signal SEGV (no mapping at the fault address) in kouogini at 0xfed74404
    0xfed74404: kouogini+0x023c: ld [%o0], %o0
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfed74b44: kouoini+0x00a0: call kouogini
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfed39db4: kpuinit0+0x07bc: jmpl %o7, %o7
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfed39538: kpuinit+0x0030: call PROCEDURELINKAGE_TABLE_+0x22d4 [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecb24d0: sqgctx+0x00f8: call PROCEDURELINKAGE_TABLE_+0x69c [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecb23a4: sqgrct+0x003c: call PROCEDURELINKAGE_TABLE_+0x3a8 [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecabdd8: sqlcmex+0x0214: call PROCEDURELINKAGE_TABLE_+0x39c [PLT]
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0xfecac490: sqlcxt+0x0058: call sqlcmex
    (/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) up
    0x000109e0: main+0x0150: call sqlcxt [PLT]
    Please, help me
    thanks
    Sergio saiz.

    The Member Feedback forum is for suggestions and feedback for OTN Developer Services. This forum is not monitored by Oracle support or product teams and so Oracle product and technology related questions will not be answered. We recommend that you post this thread to the Database - General forum. General Database Discussions
    OTN

  • Weblogic crashes with a coredump on solaris

    Hi All,
    Recently our weblogic server on solaris started to give core dumps.
    It doesnt say why it crashed and the only available information are the
    coredump file and the log file which is created for that crash. One other
    noticable thing was somewhere in the output it said,
    An irrecoverable stack overflow has occurred.
    Unexpected Signal : 11 occurred at PC=0xb77e8
    Function name=(N/A)
    Library=(N/A)
    NOTE: We are unable to locate the function name symbol for the error
    just occurred. Please refer to release documentation for possible
    reason and solutions.
    Current Java thread:
    Dynamic libraries:
    0x10000 /opt/customer/bea/jdk131/jre/bin/sparc/native_threads/java
    0xff350000 /usr/lib/libthread.so.1
    0xff390000 /usr/lib/libdl.so.1
    0xff200000 /usr/lib/libc.so.1
    0xff330000 /usr/platform/SUNW,Ultra-60/lib/libc_psr.so.1
    0xfe000000 /opt/customer/bea/jdk131/jre/lib/sparc/server/libjvm.so
    0xff2e0000 /usr/lib/libCrun.so.1
    0xff1e0000 /usr/lib/libsocket.so.1
    0xff100000 /usr/lib/libnsl.so.1
    0xff0d0000 /usr/lib/libm.so.1
    0xff310000 /usr/lib/libw.so.1
    0xff0b0000 /usr/lib/libmp.so.2
    0xff080000 /opt/customer/bea/jdk131/jre/lib/sparc/native_threads/libhpi.so
    0xff050000 /opt/customer/bea/jdk131/jre/lib/sparc/libverify.so
    0xfdc80000 /opt/customer/bea/jdk131/jre/lib/sparc/libjvm.so
    0xfe7c0000 /opt/customer/bea/jdk131/jre/lib/sparc/libjava.so
    0xff020000 /opt/customer/bea/jdk131/jre/lib/sparc/libzip.so
    0xfe560000 /usr/lib/locale/en_US.ISO8859-15/en_US.ISO8859-15.so.2
    0xfe530000 /opt/customer/bea/jdk131/jre/lib/sparc/libhprof.so
    0xfc920000 /opt/customer/bea/jdk131/jre/lib/sparc/libnet.so
    0xfa6e0000 /usr/lib/nss_files.so.1
    0xfa6c0000 /opt/customer/bea/wlserver6.1/lib/solaris/libmuxer.so
    0xfa6a0000 /usr/ucblib/libucb.so.1
    0xfa460000 /usr/lib/libresolv.so.1
    0xfa420000 /usr/lib/libelf.so.1
    Local Time = Wed May 29 15:21:52 2002
    Elapsed Time = 13736
    # HotSpot Virtual Machine Error : 11
    # Error ID : 4F530E43505002C4 01
    # Please report this error at
    # http://java.sun.com/cgi-bin/bugreport.cgi
    # Java VM: Java HotSpot(TM) Server VM (1.3.1_01 mixed mode)
    # An error report file has been saved as hs_err_pid26165.log.
    # Please refer to the file for further information.
    Abort - core dumped
    LD_LIBRARY_PATH=/opt/customer/bea/wlserver6.1/lib/solaris:/opt/customer/bea/
    wlserver6.1/lib/solaris/oci816_8:/usr/local/lib
    Any ideas? I can send the coredump if required.
    Thank You
    Saman

    So your SP2 machines are more stabel than SP3 and SP4?
    Thanks,
    Chuck
    Kevin Toomey wrote:
    I'm seeing something similiar. I recently upgraded our admin server to SP4 and
    JDK 1.3.1_06. Some of the managed servers in the domain are still at SP2 with
    JDK 1.3.1_02. The admin server is crashing with no dumps and no error messages
    to the log.
    Hella <[email protected]> wrote:
    We have been experiencing numerous crashes (with core dumps) runninig
    6.1sp3 on Solaris 8 64-bit. The server is at the latest recommended
    patch cluster (to include the J2SE patch cluster) We are using J2SE
    1.3.1 and have tried all revisions to include 06. It seems the 04 rev
    is
    the most stable but still we get numerous crashes a day.
    Does anyone know of any problems with the above combinations?
    Thanks greatly,
    Chuck

  • OCIServerAttach() coredump with Oracle10.2 64bit on X64 platform

    hi, i get a problem using OCI to connect to server. On X64 platform, Solaris10 system, and 64bit Oracle10.2 release, when i use OCIServerAttach() to attach a server, it will core dump. However, with the same code and same Oracle version, on a spark platform, it works well.My code is 32bit, so it depens on $ORACLE_HOME/lib32/libclntsh.so.10.1. Under Spark platform it works,but under x64 platform it will coredump. I copied a libclntsh.so.10.1 from a 32bit Oracle 10.2 client instance $ORACLE_HOME/lib on X86 platform to the X64 $ORACLE_HOME/lib32, the code then works. Why?
    My code:
    OCIEnv      *m_pEnvHandle;               
    OCIError      *m_pErrHandle;               
    OCIServer      *m_pSrvHandle;               
    OCISvcCtx      *m_pSvcHandle;               
    const char* i_pszSrv = "iccshis";
    OCIEnvCreate(&m_pEnvHandle, OCI_THREADED|OCI_OBJECT, (dvoid *)0,
                   0, 0, 0, (size_t) 0, (dvoid **)0);
    OCIHandleAlloc((dvoid*)m_pEnvHandle, (dvoid**)&m_pErrHandle,
                   OCI_HTYPE_ERROR, (size_t)0, (dvoid**)0);
    OCIHandleAlloc((dvoid*)m_pEnvHandle, (dvoid**)&m_pSrvHandle,
                   OCI_HTYPE_SERVER, (size_t)0, (dvoid**)0);
    OCIServerAttach(m_pSrvHandle, m_pErrHandle, (text*)i_pszSrv, strlen((char*)i_pszSrv), OCI_DEFAULT);
    The stack back trace when the programme core dumped:
    00000000 ???????? (fb1f3430, 0, 804068c, 822e1d8, b, 8040690)
    fa996a3f nnfgrne (fb1f3430, 8043058, 0, fb1b5e38, 0, 8040e00) + 25b
    faa0b30d nlolgobj (fb1f3430, 8043058, 16, 8040e64, 8040e68) + 669
    fa9951ab nnfun2a (fb1f3430, 8043058, ff, 8043034, 8040ef0, 1000) + 15b
    fa994e9e nnfsn2a (fb1f3430, 8043058, ff, 8043034, 8040ef0, 1000) + 36
    fa983f97 niqname (fb1f3430, 1001, 8043058, 7, c0e887c, 8046290) + 2fb
    fa8cfd38 kwfnran (9721ba8, 7, 804628c, 8046290, c0ee8bc, c0ee8c0) + 168
    fa8a7b33 kwfcinit (c0e475c, 9721ba8, 7, 804628c, 8046290, be896d0) + 1b7
    fa726f04 kpuatch (c0e475c, c0e41d0, 9721ba8, 7, 0, 0) + 7c0
    fa7e6f06 OCIServerAttach (c0e475c, c0e41d0, 9721ba8, 7, 0, 0) + 56
    f7574b07 bool COracleDBI::Open(const char*,const char*,const char*) (be7c998, 9721ba8, 89b6d90, 95f7170) + 12f;
    Can anybody tell me why? thanks.

    I'd hesitate to use SQL*Plus to determine a failure on this single OCI call.
    While I haven't checked lately, the last time I looked, SQL*Plus didn't use OCIServerAttach. Instead, it used UPI to perform most connection and session initialization and then converted its handles for use by OCI calls. Like I said, it may work, but this is a very basic call that should work.
    I know a couple people who worked at Oracle and based on their information, I believe it would be nearly impossible for this blatant of a bug to pass regression. I'd opt for trying one of the OCI demo programs to see if this same issue can be replicated.
    Just my 2 cents :)

  • Synchronous inetd coredumping over multiple zones

    We're experiencing rather strange behaviour on several of our zoned systems (some M5k's, 5220's, T2000's),
    in that some Solaris daemons (mostly the inetd) are repeatedly coredumping, in all zones at the same time.
    Aug 2 13:29:01 so02 genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[18080] core dumped: /var/core/core_zone1_inetd_0_0_1217676540_18080
    Aug 2 13:29:02 so02 genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[18094] core dumped: /var/core/core_zone2_inetd_0_0_1217676541_18094
    Aug 2 13:29:02 so02 genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[18247] core dumped: /var/core/core_zone1_inetd_0_0_1217676542_18247
    Aug 2 13:29:03 so02 genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[18261] core dumped: /var/core/core_zone2_inetd_0_0_1217676543_18261
    Aug 2 13:29:04 so02 genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[18275] core dumped: /var/core/core_zone2_inetd_0_0_1217676543_18275
    Aug 2 13:29:05 so02 genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[18285] core dumped: /var/core/core_zone1_inetd_0_0_1217676544_18285
    These always seem to come in bursts of three crash rounds, then it's running fine for a few days, and suddenly another burst of coredumps - on average twice a week.
    The machines are not visible to the internet, and as it also happens on weekends, so I think I can rule out some malicious user trying to exploit some flaw in inetd causing it to crash.
    Anyone got a hint what might cause this, or how to trace the issue back?

    Smells like a bug in inetd that is being tickled by a port scan. If someone in your organization is doing scanning, it could hit all your zone IPs within a short period of time.
    If you can't find who's responsible via other methods (asking around), I'd set up a zone that isn't doing anything (so it has almost no network traffic). Then I'd run snoop to watch traffic for that hostname just before you might expect it again. Then go back and correlate the traffic after you find inetd down again. By having the zone otherwise idle, the snoop traffic should be light enough that you can run it for hours with little impact.
    Darren

Maybe you are looking for