SAPOSCOL not running in MS Cluster

Question

Hi, gurus:
We have a problem with SAPOSCOL in a SAP ECC 6.0 system (SAP ECC 6.0 + NetWeaver 7.00 + Oracle 10.2 + Windows Server 2003 R2 Enterprise x64 Edition) running over a MS cluster:
Transactions OS06/ST06 shows no data, and they show an info message wich states: SAPOSCOL not running ? (shared memory not available ). When we checked this issue, we noticed that, in fact, there is no sapcoscol.exe task running in any node.
But when we try to start the service (both using microsoft services console and cmd commands) although we can see the process running in the node which owns all the resources, SAP seems not notice that. The information system shows in ST06>Operating System Collector>Status is the following:
iinterval             0             sec.
Collector Version:
Date/time             05.09.2008 16:55:01
Start of Collector
Status report
Collector Versions                         
  running                                   COLL 20.95     700     - 20.64     NT 07/10/17
  dialog                                   COLL 20.95     700     - 20.65     NT 08/02/06
Shared Memory                              attached
Number of records                         575
Active Flag                              active     (01)
Operating System                         Windows NT     5.2.3790 SP     2 BL-SAP2 4x AMD64 Level 1
Collector PID                              0 (00000000)
Collector                                   not running (process ID not found).
Start time coll.                              Thu Jan 01     01:00:00 1970
Current     Time                              Fri Sep 05     16:55:01 2008
Last write access                         Mon Sep 01     11:28:23 2008
Last Read  Access                         Fri Sep 05     15:54:00 2008
Collection Interval                         10     sec     (next delay).
Collection Interval                         10     sec     (last ).
Status                                   read
Collect     Details                         required
Refresh                                   required
Header Extention Structure                    
Number of x-header          Records          1
Number of Communication     Records          60
Number of free Com.          Records          60
Resulting offset to     1.data rec.               61
Trace level                                   3
Collector in IDLE -     mode ?               NO
  become idle after     300     sec     without     read access.|
  Length of     Idle Interval                    60     sec
  Length of     norm.Interval                    10     sec
But saposcol.exe is running with a certain PID in the same note than SAP and Oracle under user sapservice<sid>
We have tried to run saposcol in several ways (as, I have noted before: from microsoft service console, from cmd line using "net start saposcol", using the saposcol under C:\WINDOWS\SapCluster and the one under
F:\usr\sap\PRD\sys\exe\run, fom the two nodes, accessing the cluster through several IPs...) and tried the commands saposcol -c and saposcol -k but we cannot get the saposcoll run. Moreover, we haven't found any log information. The only log we (and SAP) could find is the one located in C:\WINDOWS\SapCluster\dev_coll.
This log remain frozen at September 1st:
      SAPOSCOL version  COLL 20.95 700 - 20.64 NT 07/10/17, 64 bit, multithreaded, Non-Unicode
      compiled at   Feb  3 2008
      systemid      562 (PC with Windows NT)
      relno         7000
      patch text    COLL 20.95 700 - 20.64 NT 07/10/17
      patchno       146
      intno         20050900
      running on    BL-SAP2 Windows NT 5.2 3790 Service Pack 2 4x AMD64 Level 15 (Mod 65 Step 3)
12:04:16 01.09.2008   LOG: Profile          : no profile used
12:04:16 01.09.2008   LOG: Saposcol Version  : [COLL 20.95 700 - 20.64 NT 07/10/17]
12:04:16 01.09.2008   LOG: Working directory : C:\WINDOWS\SAPCLU~1
12:04:16 01.09.2008   LOG: Allocate Counter Buffer [10000 Bytes]
12:04:16 01.09.2008   LOG: Allocate Instance Buffer [10000 Bytes]
12:04:17 01.09.2008   LOG: Shared Memory Size: 71898.
12:04:17 01.09.2008   LOG: Connected to existing shared memory.
12:04:17 01.09.2008   LOG: MaxRecords = 575 <> RecordCnt + Dta_offset = 614 + 61
12:04:22 01.09.2008 WARNING: WaitFree: could not set new shared memory status after 5 sec
12:04:22 01.09.2008 WARNING: Cannot create Shared Memory
Kernel Info:
Kernel release    700
Compilation        NT 5.2 3790 Service Pack 1 x86 MS VC++ 14.00
Sup.Pkg lvl.       146
ABAP Load       1563
CUA load           30
Mode                opt
Can anyone shed some light on the subject?
Thank you very much and kind regards
Edited by: Jose Enrique Sepulveda on Sep 6, 2008 2:10 AM

Answer 1

Dear bhaskar:
Thanks for your reply. We have considered balancing the system to the other node or reboot the system to free resources, in order to re-create the shared memory, but in the past, the balancing process (move resources from one node to the other) has caused problems. Since this is a critical system, stopping (or balancing) is not an option right now, and updating the kernel requires an ABAP stack reboot plus the kernel change : any changes in system configuration requires a longer approval/planning process than a reboot.
Moreover, the OS collecting system and its display in OS06/ST06 has worked fine until now.
Does anyone knows if a reboot has solved this kind of problem in a similar situation?
Thanks in advance
José Enrique