Dispatcher stopped and Workprocessors are Ended State

Question

Hi Experts
Suddenly my dispatcher stopped and all work processors are in Ended state in the ECC6.0 with Oracle database in windows 2003.  I have checked the Memory parameters and resetted the values but still getting the same problem here i am pasting the dev_disp and dev_w0 files.  Please help me need full
Regards
Chandra
dev_disp::
trc file: "dev_disp", trc level: 1, release: "700"
sysno      00
sid        CHE
systemid   560 (PC with Windows NT)
relno      7000
patchlevel 0
patchno    175
intno      20050900
make:      multithreaded, Unicode, optimized
pid        1536
Sun Dec 07 13:05:43 2008
kernel runs with dp version 241000(ext=110000) (@(#) DPLIB-INT-VERSION-241000-UC)
length of sys_adm_ext is 576 bytes
SWITCH TRC-HIDE on ***
***LOG Q00=> DpSapEnvInit, DPStart (00 1536) [dpxxdisp.c   1261]
     shared lib "dw_xml.dll" version 175 successfully loaded
     shared lib "dw_xtc.dll" version 175 successfully loaded
     shared lib "dw_stl.dll" version 175 successfully loaded
     shared lib "dw_gui.dll" version 175 successfully loaded
     shared lib "dw_mdm.dll" version 175 successfully loaded
rdisp/softcancel_sequence :  -> 0,5,-1
use internal message server connection to port 3900
Sun Dec 07 13:05:54 2008
WARNING => DpNetCheck: NiHostToAddr(www.doesnotexist0081.qqq.nxst) took 11 seconds
Sun Dec 07 13:05:59 2008
WARNING => DpNetCheck: NiAddrToHost(1.0.0.0) took 5 seconds
***LOG GZZ=> 2 possible network problems detected - check tracefile and adjust the DNS settings [dpxxtool2.c  5418]
MtxInit: 30000 0 0
DpSysAdmExtInit: ABAP is active
DpSysAdmExtInit: VMC (JAVA VM in WP) is not active
DpIPCInit2: start server >I3LSERVSAP1_CHE_00                      <
DpShMCreate: sizeof(wp_adm)          23856     (1704)
DpShMCreate: sizeof(tm_adm)          4415616     (21968)
DpShMCreate: sizeof(wp_ca_adm)          24064     (80)
DpShMCreate: sizeof(appc_ca_adm)     8000     (80)
DpCommTableSize: max/headSize/ftSize/tableSize=500/8/528056/528064
DpShMCreate: sizeof(comm_adm)          528064     (1048)
DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0
DpShMCreate: sizeof(slock_adm)          0     (96)
DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0
DpShMCreate: sizeof(file_adm)          0     (72)
DpShMCreate: sizeof(vmc_adm)          0     (1544)
DpShMCreate: sizeof(wall_adm)          (38456/34360/64/184)
DpShMCreate: sizeof(gw_adm)     48
DpShMCreate: SHM_DP_ADM_KEY          (addr: 06840040, size: 5081056)
DpShMCreate: allocated sys_adm at 06840040
DpShMCreate: allocated wp_adm at 06842118
DpShMCreate: allocated tm_adm_list at 06847E48
DpShMCreate: allocated tm_adm at 06847E78
DpShMCreate: allocated wp_ca_adm at 06C7DEF8
DpShMCreate: allocated appc_ca_adm at 06C83CF8
DpShMCreate: allocated comm_adm at 06C85C38
DpShMCreate: system runs without slock table
DpShMCreate: system runs without file table
DpShMCreate: allocated vmc_adm_list at 06D06AF8
DpShMCreate: allocated gw_adm at 06D06B38
DpShMCreate: system runs without vmc_adm
DpShMCreate: allocated ca_info at 06D06B68
DpShMCreate: allocated wall_adm at 06D06B70
MBUF state OFF
DpCommInitTable: init table for 500 entries
rdisp/queue_size_check_value :  -> off
ThTaskStatus: rdisp/reset_online_during_debug 0
EmInit: MmSetImplementation( 2 ).
MM global diagnostic options set: 0
<ES> client 0 initializing ....
<ES> InitFreeList
<ES> block size is 1024 kByte.
Using implementation view
<EsNT> Using memory model view.
<EsNT> Memory Reset disabled as NT default
<ES> 4075 blocks reserved for free list.
ES initialized.
rdisp/http_min_wait_dia_wp : 1 -> 1
***LOG CPS=> DpLoopInit, ICU ( 3.0 3.0 4.0.1) [dpxxdisp.c   1668]
***LOG Q0K=> DpMsAttach, mscon ( I3LSERVSAP1) [dpxxdisp.c   12364]
DpStartStopMsg: send start message (myname is >I3LSERVSAP1_CHE_00                      <)
DpStartStopMsg: start msg sent
CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.
CCMS: Initalizing shared memory of size 60000000 for monitoring segment.
CCMS: Checking Downtime Configuration of Monitoring Segment.
CCMS: start to initalize 3.X shared alert area (first segment).
DpMsgAdmin: Set release to 7000, patchlevel 0
MBUF state PREPARED
MBUF component UP
DpMBufHwIdSet: set Hardware-ID
***LOG Q1C=> DpMBufHwIdSet [dpxxmbuf.c   1050]
DpMsgAdmin: Set patchno for this platform to 175
Release check o.K.
Sun Dec 07 13:06:39 2008
my types changed after wp death/restart 0xbf --> 0xbe
my types changed after wp death/restart 0xbe --> 0xbc
my types changed after wp death/restart 0xbc --> 0xb8
my types changed after wp death/restart 0xb8 --> 0xb0
my types changed after wp death/restart 0xb0 --> 0xa0
my types changed after wp death/restart 0xa0 --> 0x80
DP_FATAL_ERROR => DpWPCheck: no more work processes
DISPATCHER EMERGENCY SHUTDOWN ***
increase tracelevel of WPs
NiWait: sleep (10000ms) ...
NiISelect: timeout 10000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:49 2008
NiISelect: TIMEOUT occured (10000ms)
dump system status
Workprocess Table (long)               Sun Dec 07 07:36:49 2008
========================
No Ty. Pid      Status  Cause Start Err Sem CPU    Time  Program          Cl  User         Action                    Table
0 DIA     6120 Ended         no      1   0        0                                                                         
1 DIA     2860 Ended         no      1   0        0                                                                         
2 DIA     6020 Ended         no      1   0        0                                                                         
3 DIA     5632 Ended         no      1   0        0                                                                         
4 DIA      588 Ended         no      1   0        0                                                                         
5 DIA     6016 Ended         no      1   0        0                                                                         
6 UPD      484 Ended         no      1   0        0                                                                         
7 ENQ     2268 Ended         no      1   0        0                                                                         
8 BTC      584 Ended         no      1   0        0                                                                         
9 BTC     6064 Ended         no      1   0        0                                                                         
10 BTC     4228 Ended         no      1   0        0                                                                         
11 BTC     5980 Ended         no      1   0        0                                                                         
12 SPO     1984 Ended         no      1   0        0                                                                         
13 UP2     5996 Ended         no      1   0        0                                                                         
Dispatcher Queue Statistics               Sun Dec 07 07:36:49 2008
===========================
--------++++--
+
Typ
now
high
max
writes
reads
--------++++--
+
NOWP
0
2
2000
6
6
--------++++--
+
DIA
5
5
2000
5
0
--------++++--
+
UPD
0
0
2000
0
0
--------++++--
+
ENQ
0
0
2000
0
0
--------++++--
+
BTC
0
0
2000
0
0
--------++++--
+
SPO
0
0
2000
0
0
--------++++--
+
UP2
0
0
2000
0
0
--------++++--
+
max_rq_id          12
wake_evt_udp_now     0
wake events           total     8,  udp     7 ( 87%),  shm     1 ( 12%)
since last update     total     8,  udp     7 ( 87%),  shm     1 ( 12%)
Dump of tm_adm structure:               Sun Dec 07 07:36:49 2008
=========================
Term    uid  man user    term   lastop  mod wp  ta   a/i (modes)
Workprocess Comm. Area Blocks               Sun Dec 07 07:36:49 2008
=============================
Slots: 300, Used: 1, Max: 0
--------++--
+
id
owner
pid
eyecatcher
--------++--
+
0
DISPATCHER
-1
WPCAAD000
NiWait: sleep (5000ms) ...
NiISelect: timeout 5000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:54 2008
NiISelect: TIMEOUT occured (5000ms)
DpHalt: shutdown server >I3LSERVSAP1_CHE_00                      < (normal)
DpJ2eeDisableRestart
DpModState: buffer in state MBUF_PREPARED
NiBufSend starting
NiIWrite: hdl 2 sent data (wrt=110,pac=1,MESG_IO)
MsINiWrite: sent 110 bytes
MsIModState: change state to SHUTDOWN
DpModState: change server state from STARTING to SHUTDOWN
Switch off Shared memory profiling
ShmProtect( 57, 3 )
ShmProtect(SHM_PROFILE, SHM_PROT_RW
ShmProtect( 57, 1 )
ShmProtect(SHM_PROFILE, SHM_PROT_RD
DpWakeUpWps: wake up all wp's
Stop work processes
Stop gateway
killing process (6084) (SOFT_KILL)
Stop icman
killing process (6128) (SOFT_KILL)
Terminate gui connections
wait for end of work processes
wait for end of gateway
[DpProcDied] Process lives  (PID:6084  HANDLE:1484)
waiting for termination of gateway ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:55 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process died  (PID:6084  HANDLE:1484)
wait for end of icman
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:56 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:57 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:58 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:06:59 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:00 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:01 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:02 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:03 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:04 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:05 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process lives  (PID:6128  HANDLE:1488)
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=1513
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Sun Dec 07 13:07:06 2008
NiISelect: TIMEOUT occured (1000ms)
[DpProcDied] Process died  (PID:6128  HANDLE:1488)
DpStartStopMsg: send stop message (myname is >I3LSERVSAP1_CHE_00                      <)
NiIMyHostName: hostname = 'I3LSERVSAP1'
AdGetSelfIdentRecord: >                                                                           <
AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 4 (AD_STARTSTOP), ser 0, ex 0, errno 0
DpConvertRequest: net size = 189 bytes
NiBufSend starting
NiIWrite: hdl 2 sent data (wrt=562,pac=1,MESG_IO)
MsINiWrite: sent 562 bytes
send msg (len 110+452) to name                    -, type 4, key -
DpStartStopMsg: stop msg sent
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER          , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 recv would block (errno=EAGAIN)
NiIRead: read for hdl 2 timed out (0ms)
DpHalt: no more messages from the message server
DpHalt: send keepalive to synchronize with the message server
NiBufSend starting
NiIWrite: hdl 2 sent data (wrt=114,pac=1,MESG_IO)
MsINiWrite: sent 114 bytes
send msg (len 110+4) to name           MSG_SERVER, type 0, key -
MsSndName: MS_NOOP ok
Send 4 bytes to MSG_SERVER
NiIRead: hdl 2 recv would block (errno=EAGAIN)
NiIPeek: peek successful for hdl 2 (r)
NiIRead: hdl 2 received data (rcd=114,pac=1,MESG_IO)
NiBufIIn: NIBUF len=114
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 114 bytes
MSG received, len 110+4, flag 3, from MSG_SERVER          , typ 0, key -
Received 4 bytes from MSG_SERVER                             
Received opcode MS_NOOP from msg_server, reply MSOP_OK
MsOpReceive: ok
MsSendKeepalive : keepalive sent to message server
NiIRead: hdl 2 recv would block (errno=EAGAIN)
Sun Dec 07 13:07:07 2008
NiIPeek: peek for hdl 2 timed out (r; 1000ms)
NiIRead: read for hdl 2 timed out (1000ms)
DpHalt: no more messages from the message server
DpHalt: sync with message server o.k.
detach from message server
***LOG Q0M=> DpMsDetach, ms_detach () [dpxxdisp.c   12710]
NiBufSend starting
NiIWrite: hdl 2 sent data (wrt=110,pac=1,MESG_IO)
MsINiWrite: sent 110 bytes
MsIDetach: send logout to msg_server
MsIDetach: call exit function
DpMsShutdownHook called
NiBufISelUpdate: new MODE -- (r-) for hdl 2 in set0
SiSelNSet: set events of sock 1420 to: ---
NiBufISelRemove: remove hdl 2 from set0
SiSelNRemove: removed sock 1420 (pos=2)
SiSelNRemove: removed sock 1420
NiSelIRemove: removed hdl 2
MBUF state OFF
AdGetSelfIdentRecord: >                                                                           <
AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 40 (AD_MSBUF), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 40 (AD_MSBUF), ser 0, ex 0, errno 0
blks_in_queue/wp_ca_blk_no/wp_max_no = 1/300/14
LOCK WP ca_blk 1
make DISP owner of wp_ca_blk 1
DpRqPutIntoQueue: put request into queue (reqtype 1, prio LOW, rq_id 15)
MBUF component DOWN
NiICloseHandle: shutdown and close hdl 2 / sock 1420
NiBufIClose: clear extension for hdl 2
MsIDetach: detach MS-system
cleanup EM
EsCleanup ....
EmCleanup() -> 0
Es2Cleanup: Cleanup ES2
***LOG Q05=> DpHalt, DPStop ( 1536) [dpxxdisp.c   10908]
Good Bye .....
dev_w0::
trc file: "dev_w0", trc level: 1, release: "700"
ACTIVE TRACE LEVEL           1
ACTIVE TRACE COMPONENTS      all, MJ
B 
B Sun Dec 07 13:05:59 2008
B  create_con (con_name=R/3)
B  Loading DB library 'F:\usr\sap\CHE\DVEBMGS00\exe\dboraslib.dll' ...
B  Library 'F:\usr\sap\CHE\DVEBMGS00\exe\dboraslib.dll' loaded
B  Version of 'F:\usr\sap\CHE\DVEBMGS00\exe\dboraslib.dll' is "700.08", patchlevel (0.167)
B  New connection 0 created
M sysno      00
M sid        CHE
M systemid   560 (PC with Windows NT)
M relno      7000
M patchlevel 0
M patchno    175
M intno      20050900
M make:      multithreaded, Unicode, optimized
M pid        6120
M
M  kernel runs with dp version 241000(ext=110000) (@(#) DPLIB-INT-VERSION-241000-UC)
M  length of sys_adm_ext is 576 bytes
M  ***LOG Q0Q=> tskh_init, WPStart (Workproc 0 6120) [dpxxdisp.c   1323]
I  MtxInit: 30000 0 0
M  DpSysAdmExtCreate: ABAP is active
M  DpSysAdmExtCreate: VMC (JAVA VM in WP) is not active
M  DpShMCreate: sizeof(wp_adm)          23856     (1704)
M  DpShMCreate: sizeof(tm_adm)          4415616     (21968)
M  DpShMCreate: sizeof(wp_ca_adm)          24064     (80)
M  DpShMCreate: sizeof(appc_ca_adm)     8000     (80)
M  DpCommTableSize: max/headSize/ftSize/tableSize=500/8/528056/528064
M  DpShMCreate: sizeof(comm_adm)          528064     (1048)
M  DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0
M  DpShMCreate: sizeof(slock_adm)          0     (96)
M  DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0
M  DpShMCreate: sizeof(file_adm)          0     (72)
M  DpShMCreate: sizeof(vmc_adm)          0     (1544)
M  DpShMCreate: sizeof(wall_adm)          (38456/34360/64/184)
M  DpShMCreate: sizeof(gw_adm)     48
M  DpShMCreate: SHM_DP_ADM_KEY          (addr: 06840040, size: 5081056)
M  DpShMCreate: allocated sys_adm at 06840040
M  DpShMCreate: allocated wp_adm at 06842118
M  DpShMCreate: allocated tm_adm_list at 06847E48
M  DpShMCreate: allocated tm_adm at 06847E78
M  DpShMCreate: allocated wp_ca_adm at 06C7DEF8
M  DpShMCreate: allocated appc_ca_adm at 06C83CF8
M  DpShMCreate: allocated comm_adm at 06C85C38
M  DpShMCreate: system runs without slock table
M  DpShMCreate: system runs without file table
M  DpShMCreate: allocated vmc_adm_list at 06D06AF8
M  DpShMCreate: allocated gw_adm at 06D06B38
M  DpShMCreate: system runs without vmc_adm
M  DpShMCreate: allocated ca_info at 06D06B68
M  DpShMCreate: allocated wall_adm at 06D06B70
M  rdisp/queue_size_check_value :  -> off
M  ThTaskStatus: rdisp/reset_online_during_debug 0
X  EmInit: MmSetImplementation( 2 ).
X  MM global diagnostic options set: 0
X  <ES> client 0 initializing ....
X  Using implementation view
X  <EsNT> Using memory model view.
M  <EsNT> Memory Reset disabled as NT default
X  ES initialized.
M 
M Sun Dec 07 13:06:00 2008
M  ThInit: running on host I3LSERVSAP1
M 
M Sun Dec 07 13:06:01 2008
M  calling db_connect ...
C  Prepending F:\usr\sap\CHE\DVEBMGS00\exe to Path.
C  Oracle Client Version: '10.2.0.1.0'
C  Client NLS settings: AMERICAN_AMERICA.UTF8
C  Logon as OPS$-user to get SAPSR3's password
C  Connecting as /@CHE on connection 0 (nls_hdl 0) ... (dbsl 700 030508)
C  Nls CharacterSet                 NationalCharSet              C      EnvHp      ErrHp ErrHpBatch
C    0 UTF8                                                      1   0672A1A8   0672F744   0672EFCC
C  Attaching to DB Server CHE (con_hdl=0,svchp=0672EF18,srvhp=06740B04)
C  Starting user session (con_hdl=0,svchp=0672EF18,srvhp=06740B04,usrhp=06755878)
C  Now '/@CHE' is connected: con_hdl=0, nls_hdl=0, session_id=85.
C  Got SAPSR3's password from OPS$-user
C  Disconnecting from connection 0 ...
C  Closing user session (con_hdl=0,svchp=0672EF18,usrhp=06755878)
C  Now I'm disconnected from ORACLE
C  Connecting as SAPSR3/<pwd>@CHE on connection 0 (nls_hdl 0) ... (dbsl 700 030508)
C  Nls CharacterSet                 NationalCharSet              C      EnvHp      ErrHp ErrHpBatch
C    0 UTF8                                                      1   0672A1A8   0672F744   0672EFCC
C  Starting user session (con_hdl=0,svchp=0672EF18,srvhp=06740B04,usrhp=06755878)
C  Now 'SAPSR3/<pwd>@CHE' is connected: con_hdl=0, nls_hdl=0, session_id=85.
C  Database NLS settings: AMERICAN_AMERICA.UTF8
C  DB instance CHE is running on I3LSERVSAP1 with ORACLE version 10.2.0.2.0 since DEC 07, 2008, 12:36:31
B  Connection 0 opened (DBSL handle 0)
B  Wp  Hdl ConName          ConId     ConState     TX  PRM RCT TIM MAX OPT Date     Time   DBHost         
B  000 000 R/3              000000000 ACTIVE       NO  YES NO  000 255 255 20081207 130601 I3LSERVSAP1    
C  build_stmt: reallocating stmt buffer: 256 -> 2000 characters
M  db_connect o.k.
M  ICT: exclude compression: .zip,.cs,.rar,.arj,.z,.gz,.tar,.lzh,.cab,.hqx,.ace,.jar,.ear,.war,.css,.pdf,.js,.gzip,.uue,.bz2,.iso,.sda,.sar,.gif
I  MtxInit: 0 0 0
M  SHM_PRES_BUF               (addr: 0B3A0040, size: 4400000)
I  *** ERROR => [MapOsShm] Can't find free space for Shared Memory (Size=720000 KB)
               [shmnt.c      2174]
I  *** ERROR => [CreateOsShm] MapViewOfFile(Key=9,Handle=0x000004DC) failed with Err=0
               [shmnt.c      2174]
I  *** ERROR => ShmCreate: Create (9,737280000,3) failed [shmnt.c      506]
Memory diagnostic                                 *
Systeminformation
Processor-Typ             : Intel Pentium Pro
Processor-Count           : 8
Operating System          : NT 5.2, Build 3790
Service Pack              : Service Pack 2
NT Pagefile Informations
Config. minimum size      : 12570624 K
Config. maximum size      : 12570624 K
Avail.  maximum size      : 12570624 K
Num
Pagefile
Min.Size
Max.Size
Avail.Max
Curr.Size
1
c:\pagefile.sys
4190208 K
4190208 K
4190208 K
4190208 K
2
d:\pagefile.sys
4190208 K
4190208 K
4190208 K
4190208 K
3
f:\pagefile.sys
4190208 K
4190208 K
4190208 K
4190208 K
NT Task Manager Informations
Total Handles             :        0
Total Threads             :        0
Total Processes           :        0
Commit Charge Total       :  6571584 K
Commit Charge Limit       : 15822776 K
Commit Charge Peak        :  7495116 K
Phys.Memory Total         :       54 K
Phys.Memory Available     : 93650276 K
File Cache                : 17399116 K
Kernel Memory Total       : 2084222616 K
Kernel Memory Paged       :        0 K
Kernel Memory Nonpaged    : 93650304 K
Memory usage of current process
Total virt.address space  :  2097024 K
Avail.virt.address space  :   837328 K
Private Pages             :        0 K
Total heap size           :    14532 K
Virtual memory regions    :        0 K
Uncommitted heap memory   :     6440 K
Allocated heap memory     :     6850 K
Moveable heap memory      :        0 K
DDE shared heap memory    :        0 K
Memory usage of all processes
PID
Image
Instance
Work.Set
WS Peak
Priv.Pages
PP Peak
Pg Fault
1940
sapstartsrv.exe
15956 K
15956 K
18656 K
18700 K
4
5716
msg_server.EXE
[MS] CHE_00
8508 K
8508 K
11000 K
11000 K
2
1536
disp+work.EXE
[DP] CHE_00
93256 K
93256 K
69540 K
69540 K
22
5772
igswd.EXE
[**] CHE_00
2796 K
3048 K
1436 K
1724 K
0
1304
igsmux.exe
12012 K
12012 K
13032 K
13032 K
2
5696
igspw.exe
2536 K
2536 K
2336 K
2336 K
0
5688
igspw.exe
2536 K
2536 K
2336 K
2336 K
0
6084
gwrd.EXE
[GW] CHE_00
10164 K
10164 K
9996 K
17924 K
2
6128
icman.EXE
[**] CHE_00
86128 K
86128 K
88096 K
88096 K
21
6120
disp+work.EXE
31808 K
31904 K
82200 K
85356 K
7
2860
disp+work.EXE
[WP] CHE_00
30604 K
30604 K
81656 K
85356 K
7
6020
disp+work.EXE
[WP] CHE_00
16080 K
16080 K
67372 K
67372 K
3
5632
disp+work.EXE
[WP] CHE_00
16104 K
16104 K
67372 K
67372 K
3
588
disp+work.EXE
[WP] CHE_00
28804 K
28804 K
80672 K
85356 K
7
6016
disp+work.EXE
[WP] CHE_00
16084 K
16084 K
67372 K
67372 K
3
484
disp+work.EXE
[WP] CHE_00
16084 K
16084 K
67372 K
67372 K
3
2268
disp+work.EXE
[WP] CHE_00
16080 K
16080 K
67372 K
67372 K
3
584
disp+work.EXE
[WP] CHE_00
19112 K
19112 K
85360 K
85360 K
4
6064
disp+work.EXE
[WP] CHE_00
30972 K
30972 K
81696 K
85356 K
7
4228
disp+work.EXE
[WP] CHE_00
30376 K
30376 K
81488 K
85356 K
7
5980
disp+work.EXE
[WP] CHE_00
16080 K
16080 K
67372 K
67372 K
3
1984
disp+work.EXE
[WP] CHE_00
16080 K
16080 K
67372 K
67372 K
3
5996
disp+work.EXE
[WP] CHE_00
16080 K
16080 K
67372 K
67372 K
3
Sum
534240 K
1248476 K
M  *** ERROR => ThShMCreate: ShmCreate SHM_ROLL_AREA_KEY failed [thxxhead.c   2598]
M  *** ERROR => ThIPCInit: ThShMCreate [thxxhead.c   2095]
M  ***LOG R19=> ThInit, ThIPCInit ( TSKH-IPC-000001) [thxxhead.c   1532]
M  in_ThErrHandle: 1
M  *** ERROR => ThInit: ThIPCInit (step 1, th_errno 17, action 3, level 1) [thxxhead.c   10534]
M 
M  Info for wp 0
M 
M    pid = 6120
M    severity = 0
M    status = 0
M    stat = WP_RUN
M    waiting_for = NO_WAITING
M    reqtype = DP_RQ_DIAWP
M    act_reqtype = NO_REQTYPE
M    rq_info = 0
M    tid = -1
M    mode = 255
M    len = -1
M    rq_id = 65535
M    rq_source =
M    last_tid = 0
M    last_mode = 0
M    semaphore = 0
M    act_cs_count = 0
M    csTrack = 0
M    csTrackRwExcl = 0
M    csTrackRwShrd = 0
M    mode_cleaned_counter = 0
M    control_flag = 0
M    int_checked_resource(RFC) = 0
M    ext_checked_resource(RFC) = 0
M    int_checked_resource(HTTP) = 0
M    ext_checked_resource(HTTP) = 0
M    report = >                                        <
M    action = 0
M    tab_name = >                              <
M    attachedVm = no VM
M 
M  *****************************************************************************
M  *
M  *  LOCATION    SAP-Server I3LSERVSAP1_CHE_00 on host I3LSERVSAP1 (wp 0)
M  *  ERROR       ThInit: ThIPCInit
M  *
M  *  TIME        Sun Dec 07 13:06:01 2008
M  *  RELEASE     700
M  *  COMPONENT   Taskhandler
M  *  VERSION     1
M  *  RC          17
M  *  MODULE      thxxhead.c
M  *  LINE        10754
M  *  COUNTER     1
M  *
M  *****************************************************************************
M 
M  PfStatDisconnect: disconnect statistics
M  Entering TH_CALLHOOKS
M  ThCallHooks: call hook >BtcCallLgCl< for event BEFORE_DUMP
M  ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP
M  *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c   723]
M  *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c  261]
M  Entering ThSetStatError
M  ThIErrHandle: do not call ThrCoreInfo (no_core_info=0, in_dynp_env=0)
M  Entering ThReadDetachMode
M  call ThrShutDown (1)...
M  ***LOG Q02=> wp_halt, WPStop (Workproc 0 6120) [dpnttool.c   333]

Answer 1

Hi Chandra
If you look at the DEV_w0 log you can see the work process are not comming you as they are not getting enough shared memory see this error message
" Can't find free space for Shared Memory (Size=720000 KB) "
Please review the parameter changes made by you in Instance profile before restart of instace. Check the values of these parameters and if you are not sure of the value changed by you  undo the parameter changes made before the instace restart
Restart the instance again after these changes in instance profile. for now you can change the instace profile file from OS but don't forget to import this changed profile when instace is up
if you did not changed any Profile parameters, check the physical memory available. chance could you some hardware memory issue. get your os team to check.
Its a clear case of memory issue ..
Rajeev