ECC 6.0 SR3 AIX and ORACLE installation error - work processes dies

Question

Hi ,
My environment is
AIX 6.1
oracle 10.2.0.2
ECC 6.0 SR3
i'm performing the installation on above mentioned system, and got stuck at a Start ABAP instance step.
instance is not coming up and all the process are starting and and then again die one by one.
In trace of all wor processess , i can see the error trace as follows:
PfStatDisconnect: disconnect statistics
M  Entering TH_CALLHOOKS
M  ThCallHooks: call hook >SAP-Trace buffer write< for event BEFORE_DUMP
M  TrThHookFunc: called for WP dump
M  ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP
M  *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c   724]
M  *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c  261]
M  Entering ThSetStatError
M  ThIErrHandle: do not call ThrCoreInfo (no_core_info=0, in_dynp_env=0)
M  ThErrPrivHandle: set th_errno (11)
M  Entering ThReadDetachMode
M  *** ERROR => ThIErrHandle: bad value for th_act_em_hdl (8), detach T16/M0 [thxxhead.c   10973]
M 
M  Modeinfo for User T16/M0
Full trace for work process is
trc file: "dev_w16", trc level: 1, release: "700"
ACTIVE TRACE LEVEL           1
ACTIVE TRACE COMPONENTS      all, MJ
M sysno      00
M sid        DMB
M systemid   324 (IBM RS/6000 with AIX)
M relno      7000
M patchlevel 0
M patchno    144
M intno      20050900
M make:      single threaded, Unicode, 64 bit, optimized
M pid        790666
M
M 
M Tue Jan  6 15:41:58 2009
M  kernel runs with dp version 232000(ext=109000) (@(#) DPLIB-INT-VERSION-232000-UC)
M  length of sys_adm_ext is 576 bytes
M  ***LOG Q01=> ThInit, WPStart (Workproc16 1 790666) [thxxhead.c   1281]
M  ThInit: running on host MBDEVECC
M  calling db_connect ...
C  Got LIBPATH=/usr/sap/DMB/DVEBMGS00/exe:/usr/sap/DMB/DVEBMGS00/exe:/usr/lib:/lib:/usr/sap/DMB/SYS/exe/run:/oracle/client/10x_64/instantclient from environment
C  Oracle Client Version: '10.2.0.2.0'
C  Client NLS settings: AMERICAN_AMERICA.UTF8
C  Logon as OPS$-user to get SAPSR3's password
C  Connecting as /@DMB on connection 0 (nls_hdl 0) ... (dbsl 700 250407)
C  Nls CharacterSet                 NationalCharSet              C      EnvHp      ErrHp ErrHpBatch
C    0 UTF8                                                      1 0x113701b30 0x11371a068 0x113719918
C  Attaching to DB Server DMB (con_hdl=0,svchp=0x11371d268,srvhp=0x11371d4e8)
C  Starting user session (con_hdl=0,svchp=0x11371d268,srvhp=0x11371d4e8,usrhp=0x113709970)
C  Now '/@DMB' is connected (con_hdl 0, nls_hdl 0).
C  Got SAPSR3's password from OPS$-user
C  Disconnecting from connection 0 ...
C  Closing user session (con_hdl=0,svchp=0x11371d268,usrhp=0x113709970)
C  Now I'm disconnected from ORACLE
C  Connecting as SAPSR3/<pwd>@DMB on connection 0 (nls_hdl 0) ... (dbsl 700 250407)
C  Nls CharacterSet                 NationalCharSet              C      EnvHp      ErrHp ErrHpBatch
C    0 UTF8                                                      1 0x113701b30 0x11371a068 0x113719918
C  Starting user session (con_hdl=0,svchp=0x11371d268,srvhp=0x11371d4e8,usrhp=0x113709970)
C  Now 'SAPSR3/<pwd>@DMB' is connected (con_hdl 0, nls_hdl 0).
C  Database NLS settings: AMERICAN_AMERICA.UTF8
C  DB instance DMB is running on MBDEVECC with ORACLE version 10.2.0.2.0 since JAN 06, 2009, 15:41:46
B  Connection 0 opened (DBSL handle 0)
B  Wp  Hdl ConName          ConId     ConState     TX  PRM RCT TIM MAX OPT Date     Time   DBHost         
B  000 000 R/3              000000000 ACTIVE       NO  YES NO  000 255 255 20090106 154158 MBDEVECC       
M  db_connect o.k.
M  ICT: exclude compression: .zip,.cs,.rar,.arj,.z,.gz,.tar,.lzh,.cab,.hqx,.ace,.jar,.ear,.war,.css,.pdf,.js,.gzip,.uue,.bz2,.iso,.sda,.sar,.gif
I 
I Tue Jan  6 15:42:00 2009
I  MtxInit: 16 0 0
M  SHM_PRES_BUF               (addr: 0x7000000057a0000, size: 4400000)
M  SHM_ROLL_AREA          (addr: 0x700000140000000, size: 134217728)
M  SHM_PAGING_AREA          (addr: 0x700000150000000, size: 67108864)
M  SHM_ROLL_ADM               (addr: 0x700000005bd4000, size: 2672386)
M  SHM_PAGING_ADM          (addr: 0x700000160000000, size: 656416)
M  ThCreateNoBuffer          allocated 544152 bytes for 1000 entries at 0x700000170002000
M  ThCreateNoBuffer          index size: 3000 elems
M  ThCreateVBAdm          allocated 11776 bytes (50 server) at 0x700000180000000
X  EmInit: MmSetImplementation( 2 ).
X  MM global diagnostic options set: 0
X  <ES> client 16 initializing ....
X  Using implementation std
X  <ES> Info: use normal pages (no huge table support available)
X  ES initialized.
B  db_con_shm_ini:  WP_ID = 16, WP_CNT = 17, CON_ID = -1
B  dbtbxbuf: Buffer TABL  (addr: 0x7000001b0000100, size: 30000000, end: 0x7000001b1c9c480)
B  dbtbxbuf: Buffer TABLP (addr: 0x700000005e62100, size: 10240000, end: 0x700000006826100)
B  dbexpbuf: Buffer EIBUF (addr: 0x700000006827108, size: 4194304, end: 0x700000006c27108)
B  dbexpbuf: Buffer ESM   (addr: 0x7000001c0000108, size: 4194304, end: 0x7000001c0400108)
B  dbexpbuf: Buffer CUA   (addr: 0x70000017300b108, size: 3072000, end: 0x7000001732f9108)
B  dbexpbuf: Buffer OTR   (addr: 0x7000001d0000108, size: 4194304, end: 0x7000001d0400108)
M  CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.
S 
S Tue Jan  6 15:42:01 2009
S  *** init spool environment
S  initialize debug system
T  Stack direction is downwards.
T  debug control: prepare exclude for printer trace
T  new memory block 0x113a30700
S  spool kernel/ddic check: Ok
S  using table TSP02FX for frontend printing
S  1 spool work process(es) found
S  frontend print via spool service enabled
S  printer list size is 150
S  printer type list size is 50
S  queue size (profile)   = 300
S  hostspool list size = 3000
S  option list size is 30
S      found processing queue enabled
S  found spool memory service RSPO-RCLOCKS at 0x7000001732fb070
S  doing lock recovery
S  setting server cache root
S  found spool memory service RSPO-SERVERCACHE at 0x7000001732fb590
S    using messages for server info
S  size of spec char cache entry: 297032 bytes (timeout 100 sec)
S  size of open spool request entry: 2272 bytes
S  immediate print option for implicitely closed spool requests is disabled
A 
A  -PXA--
A  PXA INITIALIZATION
A  PXA: Locked PXA-Semaphore.
A  System page size: 4kb, total admin_size: 11460kb, dir_size: 11392kb.
A  Attached to PXA (address 0x7000001e0000000, size 300000K)
A  abap/pxa = shared unprotect gen_remote
A  PXA INITIALIZATION FINISHED
A  -PXA--
A 
Signal 6 encountered.
=> 64 bit R/3 Kernel
=> 64 bit AIX Kernel
=> Heap limit      = unlimited
=> Stack limit     = 4294967296
=> Core limit      = 1073741312
=> File size limit = unlimited
=> Heap address  = 0x0x113afcb80
=> Stack address = 0xfffffffffffdd60
=> Stack low     =  0xfffffffffffb470
=> Stack high    =  0xffffffffffff7c0
=> Stack Trace:
     pthread_kill() at 0x9000000005b9450
     praise() at 0x9000000005b8cac
     raise() at 0x90000000002b0f0
     abort() at 0x900000000094528
     ab_init() at 0x1002038d0
     ThrInitRunTime() at 0x1003d1864
     ThStartUp2() at 0x101a9caf0
     ThInit() at 0x1000e9678
     ThStart() at 0x1000fe1b0
     DpMain() at 0x10156c770
     nlsui_main() at 0x101a99998
=> CPU Registers:
     msr = 0xa00000000000d032     iar = 0x09000000005b9450
     ctr = 0xffffffff000000bc      lr = 0xffffffffffffffff
     xer = 0x00000000ffffffff      cr = 0x0000000000585000
     r00 = 0xffffffffffffffff     r01 = 0x0fffffffffffdd60
     r02 = 0xffffffffffffffff     r03 = 0x0000000000000000
     r04 = 0xffffffffffffffff     r05 = 0xffffffffffffffff
     r06 = 0xffffffffffffffff     r07 = 0xffffffffffffffff
     r08 = 0xffffffffffffffff     r09 = 0xffffffffffffffff
     r10 = 0xffffffffffffffff     r11 = 0xffffffffffffffff
     r12 = 0xffffffffffffffff     r13 = 0x0000000112d77bd8
     r14 = 0x0000000110cf5dd8     r15 = 0x0000000110028718
     r16 = 0x0000000110001ee8     r17 = 0x0000000000000000
     r18 = 0x0000000000000000     r19 = 0x0000000110027538
     r20 = 0x0000000110572f50     r21 = 0x0000000110572fb0
     r22 = 0x0000000110572f98     r23 = 0x0000000110572f80
     r24 = 0x0000000110573610     r25 = 0x0000000110000e40
     r26 = 0x00000001105737c0     r27 = 0x0000000101baa950
     r28 = 0x0000000112d6fe50     r29 = 0x0000000000000006
     r30 = 0x0000000000000006     r31 = 0x0000000000000001
M  ***LOG Q0E=> SigIGenAction, signal ( 6) [sigux.c      935]
M  in_ThErrHandle: 1
M  ThSigHandler: signal (step 1, th_errno 11, action 4)
M 
M  Info for wp 16
M 
M    stat = WP_RUN
M    waiting_for = NO_WAITING
M    reqtype = DP_RQ_VB2WP
M    act_reqtype = NO_REQTYPE
M    rq_info = 0
M    tid = 16
M    mode = 0
M    len = -1
M    rq_id = 65535
M    rq_source =
M    last_tid = 0
M    last_mode = 0
M    semaphore = 0
M    act_cs_count = 0
M    csTrack = 0
M    csTrackRwExcl = 0
M    csTrackRwShrd = 0
M    mode_cleaned_counter = 0
M    control_flag = 0
M    int_checked_resource(RFC) = 0
M    ext_checked_resource(RFC) = 0
M    int_checked_resource(HTTP) = 0
M    ext_checked_resource(HTTP) = 0
M    report = >                                        <
M    action = 0
M    tab_name = >                              <
M    attachedVm = no VM
M 
M  Modeinfo for User T16/M0
M 
M    tm state = 2
M    uid = 17
M    term type = 0x2
M    display = 0x0
M    cpic_no = 0
M    cpic_idx = -1
M    usr = >SAPSYS      <
M    terminal = >                    <
M    client = >   <
M    conversation_ID = >        <
M    appc_tm_conv_idx = -1
M    its_plugin = NO
M    allowCreateMode = YES
M    blockSoftCanel = NO
M    session_id = >                                <
M    ext_session_id = >                                <
M    imode = 0
M    mode state = 0x41
M    mode clean_state = 1
M    task_type = 0
M    lastThFc = THFCINVA
M    lastAction = TH_IACT_NO_ACTION
M    th_errno = 11
M    rollout_reason = 0
M    last_rollout_level = 0
M    async_receives = 0
M    cpic_receive = 0
M    em handle = 8
M    roll state = 1
M    abap state = 1
M    em state = 3
M    eg state = 1
M    spa state = 1
M    enq state = 0
M    softcancel = 0
M    cancelInitiator = ?
M    clean_state = DP_DEFAULT_CLEANING
M    next hook = T-1/U-1/M255
M    master hook = T-1/U-1/M255
M    slave hook = T-1/U-1/M255
M    debug_tid = 255
M    debug_mode = 0
M    mode type = 0x1
M    debug = 0
M    msg_count = 0
M    tcode = >                    <
M    last_wp = -1
M    client conversation_ID = >        <
M    server conversation_ID = >        <
M    lock = 0
M    max enq infos = 0
M    act enq infos = 0
M    em_hyper_hdl = 0x700000030532aa8
M    plugin_info = NULL
M    act_plugin_hdl = -1
M    act_plugin_no  = 0
M    max_plugin_no  = 0
M 
M  PfStatDisconnect: disconnect statistics
M  Entering TH_CALLHOOKS
M  ThCallHooks: call hook >SAP-Trace buffer write< for event BEFORE_DUMP
M  TrThHookFunc: called for WP dump
M  ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP
M  *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c   724]
M  *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c  261]
M  Entering ThSetStatError
M  ThIErrHandle: do not call ThrCoreInfo (no_core_info=0, in_dynp_env=0)
M  ThErrPrivHandle: set th_errno (11)
M  Entering ThReadDetachMode
M  *** ERROR => ThIErrHandle: bad value for th_act_em_hdl (8), detach T16/M0 [thxxhead.c   10973]
M 
M  Modeinfo for User T16/M0
M 
M    tm state = 2
M    uid = 17
M    term type = 0x2
M    display = 0x0
M    cpic_no = 0
M    cpic_idx = -1
M    usr = >SAPSYS      <
M    terminal = >                    <
M    client = >   <
M    conversation_ID = >        <
M    appc_tm_conv_idx = -1
M    its_plugin = NO
M    allowCreateMode = YES
M    blockSoftCanel = NO
M    session_id = >                                <
M    ext_session_id = >                                <
M    imode = 0
M    mode state = 0x41
M    mode clean_state = 1
M    task_type = 0
M    lastThFc = THFCINVA
M    lastAction = TH_IACT_NO_ACTION
M    th_errno = 11
M    rollout_reason = 0
M    last_rollout_level = 0
M    async_receives = 0
M    cpic_receive = 0
M    em handle = 8
M    roll state = 1
M    abap state = 1
M    em state = 2
M    eg state = 1
M    spa state = 1
M    enq state = 0
M    softcancel = 0
M    cancelInitiator = ?
M    clean_state = DP_DEFAULT_CLEANING
M    next hook = T-1/U-1/M255
M    master hook = T-1/U-1/M255
M    slave hook = T-1/U-1/M255
M    debug_tid = 255
M    debug_mode = 0
M    mode type = 0x1
M    debug = 0
M    msg_count = 0
M    tcode = >                    <
M    last_wp = -1
M    client conversation_ID = >        <
M    server conversation_ID = >        <
M    lock = 0
M    max enq infos = 0
M    act enq infos = 0
M    em_hyper_hdl = 0x700000030532aa8
M    plugin_info = NULL
M    act_plugin_hdl = -1
M    act_plugin_no  = 0
M    max_plugin_no  = 0
M 
M  ***LOG Q02=> wp_halt, WPStop (Workproc16 790666) [dpuxtool.c   260]
Can some one please help me.
Regards
Saurabh

Answer 1

This looks like some sort of resource issue. Have you looked at this post (similar issue): [JC00 instance could not start on the last phase of NW04 installation|JC00 instance could not start on the last phase of NW04 installation;?
Hope that is helpful.
J. Haynes