Ocmstart.sh is not starting in one RAC node
I am having a trouble starting "oracm" on one node. I can see it is working on second node but not on first node. Is there any one haveing the same issue.
Hi Chandra Sorry for the delay, I got the output from cm.out file during restart of ocmstart.sh script
=========================================================
TRACE: HandleJoin(): JOIN from node(1)->(1), tid = ClusterListener:3076 file = nmlisten.c, line = 362 {Thu Apr 10 21:39:10 2003 }
TRACE: HandleStatus(): node(0) UNKNOWN, tid = ClusterListener:3076 file = nmlisten.c, line = 404 {Thu Apr 10 21:39:11 2003 }
TRACE: HandleStatus(): src[0] dest[1] dom[0] seq[33] sync[10], tid = ClusterListener:3076 file = nmlisten.c, line = 415 {Thu Apr 10 21:39:11 2003 }
TRACE: HandleSync(): src[0] dest[1] dom[0] seq[34] sync[10], tid = ClusterListener:3076 file = nmlisten.c, line = 506 {Thu Apr 10 21:39:11 2003 }
TRACE: SendAck(): node(0) domain(0) syncSeqNo(10) type(11), tid = ClusterListener:3076 file = nmmember.c, line = 1913 {Thu Apr 10 21:39:11 2003 }
TRACE: HandleVote(): src[0] dest[1] dom[0] seq[35] sync[10], tid = ClusterListener:3076 file = nmlisten.c, line = 643 {Thu Apr 10 21:39:12 2003 }
TRACE: SendVoteInfo(): node(0) domain(0) syncSeqNo(10), tid = ClusterListener:3076 file = nmmember.c, line = 1727 {Thu Apr 10 21:39:12 2003 }
TRACE: HandleShutdown(): src[0] dest[1] dom[0] seq[0] sync[10] type[4], tid = ClusterListener:3076 file = nmlisten.c, line = 1087 {Thu Apr 10 21:39:13 2003 }
TRACE: IncrementEventValue: *(80f2920) = (1, 1), tid = ClusterListener:3076 file = unixinc.c, line = 253 {Thu Apr 10 21:39:13 2003 }--- End Dump ---
oracm, version[ 9.2.0.2.0.47 ] started {Thu Apr 10 21:40:07 2003 }
KernelModuleName is hangcheck-timer {Thu Apr 10 21:40:07 2003 }
OemNodeConfig(): Network Address of node0: 192.168.0.50 (port 9998)
{Thu Apr 10 21:40:07 2003 }
OemNodeConfig(): Network Address of node1: 192.168.0.51 (port 9998)
{Thu Apr 10 21:40:07 2003 }
WARNING: OemInit2: Opened file(/var/opt/oracle/oradata/orcl/CMQuorumFile 8), tid = main:1024 file = oem.c, line = 491 {Thu Apr 10 21:40:07 2003 }Debug Hang : ClusterListener (PID=2541) Registered withwatchdog daemon. {Thu Apr 10 21:40:09 2003 }
InitializeCM: ModuleName = hangcheck-timer {Thu Apr 10 21:40:09 2003 }
InitializeCM: Kernel module hangcheck-timer is already loaded {Thu Apr 10 21:40:09 2003 }
Debug Hang : CmConnectListener (PID=2542):Registered with watchdog daemon. {Thu Apr 10 21:40:09 2003 }
Debug Hang :StartNMMon (PID=2536) Registered with watchdog daemon. {Thu Apr 10 21:40:09 2003 }
CreateLocalEndpoint(): Network Address: 192.168.0.51
{Thu Apr 10 21:40:09 2003 }
Debug Hang :PollingThread (PID=135159169): Registered with {Thu Apr 10 21:40:09 2003 }
Debug Hang : DiskPingThread (PID=135159169): Registered with {Thu Apr 10 21:40:09 2003 }
Debug Hang :SendingThread (PID=135159169): Registered with {Thu Apr 10 21:40:09 2003 }
--- DUMP GROUP STATE DB ---
--- END OF GROUP STATE DUMP ---
--- Begin Dump ---
TRACE: LogListener: Spawned with tid 0x803., tid = 2051 file = logging.c, line = 116 {Thu Apr 10 21:40:07 2003 }oracm, version[ 9.2.0.2.0.47 ] started {Thu Apr 10 21:40:07 2003 }
TRACE: Can't read registry value for WatchdogTimerMargin, tid = main:1024 file = unixinc.c, line = 1011 {Thu Apr 10 21:40:07 2003 }
TRACE: Can't read registry value for WatchdogSafetyMargin, tid = main:1024 file = unixinc.c, line = 1011 {Thu Apr 10 21:40:07 2003 }KernelModuleName is hangcheck-timer {Thu Apr 10 21:40:07 2003 }
TRACE: Can't read registry value for ClientTimeout, tid = main:1024 file = unixinc.c, line = 1011 {Thu Apr 10 21:40:07 2003 }
TRACE: InitNMInfo: setting clientTimeout to 215s based on MissCount 215 and PollInterval 1000ms, tid = main:1024 file = nmconfig.c, line = 137 {Thu Apr 10 21:40:07 2003 }
TRACE: InitClusterDb(): getservbyname on CMSrvr failed - 22 : assigning 9998, tid = main:1024 file = nmconfig.c, line = 212 {Thu Apr 10 21:40:07 2003 }OemNodeConfig(): Network Address of node0: 192.168.0.50 (port 9998)
{Thu Apr 10 21:40:07 2003 }
OemNodeConfig(): Network Address of node1: 192.168.0.51 (port 9998)
{Thu Apr 10 21:40:07 2003 }
TRACE: OemCreateListenPort: bound at 9998, tid = main:1024 file = oem.c, line = 858 {Thu Apr 10 21:40:07 2003 }
TRACE: InitClusterDb(): found my node info at 1 name rac2, priv rac2priv, port 3623, tid = main:1024 file = nmconfig.c, line = 265 {Thu Apr 10 21:40:07 2003 }
TRACE: InitClusterDb(): Local Node(1) NodeName[rac2], tid = main:1024 file = nmconfig.c, line = 283 {Thu Apr 10 21:40:07 2003 }
TRACE: InitClusterDb(): Cluster(Oracle) with (2) Defined Nodes, tid = main:1024 file = nmconfig.c, line = 286 {Thu Apr 10 21:40:07 2003 }
TRACE: OEMInits(): CM Disk File (/var/opt/oracle/oradata/orcl/CMQuorumFile), tid = main:1024 file = oem.c, line = 243 {Thu Apr 10 21:40:07 2003 }
WARNING: OemInit2: Opened file(/var/opt/oracle/oradata/orcl/CMQuorumFile 8), tid = main:1024 file = oem.c, line = 491 {Thu Apr 10 21:40:07 2003 }
TRACE: ReadOthersDskInfo(): node(0) rcfg(11) wrtcnt(1052) lastcnt(0) alive(1052), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(1) rcfg(10) wrtcnt(2) lastcnt(0) alive(1), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(2) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(3) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(4) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(5) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(6) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(7) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(8) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(9) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(10) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(11) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(12) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(13) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(14) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(15) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(16) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(17) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(18) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(19) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(20) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(21) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(22) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(23) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(24) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(25) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(26) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(27) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(28) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(29) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(30) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(31) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(32) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(33) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(34) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(35) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(36) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(37) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(38) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(39) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(40) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(41) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(42) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(43) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(44) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(45) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(46) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(47) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(48) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(49) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(50) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(51) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(52) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(53) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(54) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(55) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(56) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(57) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(58) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(59) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(60) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(61) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(62) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ReadOthersDskInfo(): node(63) rcfg(0) wrtcnt(0) lastcnt(0) alive(0), tid = main:1024 file = oem.c, line = 1442 {Thu Apr 10 21:40:09 2003 }
TRACE: ClusterListener: Spawned with tid 0xc04., tid = 3076 file = nmlistener.c, line = 53 {Thu Apr 10 21:40:09 2003 }Debug Hang : ClusterListener (PID=2541) Registered withwatchdog daemon. {Thu Apr 10 21:40:09 2003 }
TRACE: ClusterListener (pid=2541, tid=3076): Registered with watchdog daemon., tid = 3076 file = nmlistener.c, line = 75 {Thu Apr 10 21:40:09 2003 }InitializeCM: ModuleName = hangcheck-timer {Thu Apr 10 21:40:09 2003 }
InitializeCM: Kernel module hangcheck-timer is already loaded {Thu Apr 10 21:40:09 2003 }
TRACE: CmConnectListener: Spawned with tid 0x1005., tid = 4101 file = cmclient.c, line = 215 {Thu Apr 10 21:40:09 2003 }Debug Hang : CmConnectListener (PID=2542):Registered with watchdog daemon. {Thu Apr 10 21:40:09 2003 }
TRACE: CmConnectListener (pid=2542, tid=4101): Registered with watchdog daemon., tid = 4101 file = cmclient.c, line = 246 {Thu Apr 10 21:40:09 2003 }Debug Hang :StartNMMon (PID=2536) Registered with watchdog daemon. {Thu Apr 10 21:40:09 2003 }
TRACE: StartNMMon (pid=2536, tid=1024): Registered with watchdog daemon., tid = main:1024 file = cmnodemon.c, line = 251 {Thu Apr 10 21:40:09 2003 }CreateLocalEndpoint(): Network Address: 192.168.0.51
{Thu Apr 10 21:40:09 2003 }
TRACE: StartClusterJoin(): clusterState(0) nodeState(0), tid = main:1024 file = nmmember.c, line = 277 {Thu Apr 10 21:40:09 2003 }
TRACE: PollingThread: Spawned with tid 0x1406., tid = 5126 file = nmmember.c, line = 742 {Thu Apr 10 21:40:09 2003 }Debug Hang :PollingThread (PID=135159169): Registered with {Thu Apr 10 21:40:09 2003 }
TRACE: PollingThread (pid=2543, tid=5126): Registered with watchdog daemon., tid = 5126 file = nmmember.c, line = 760 {Thu Apr 10 21:40:09 2003 }
TRACE: DiskPingThread: Spawned with tid 0x1807., tid = 6151 file = nmmember.c, line = 1050 {Thu Apr 10 21:40:09 2003 }Debug Hang : DiskPingThread (PID=135159169): Registered with {Thu Apr 10 21:40:09 2003 }
TRACE: DiskPingThread (pid=2544, tid=6151): Registered with watchdog daemon., tid = 6151 file = nmmember.c, line = 1074 {Thu Apr 10 21:40:09 2003 }
TRACE: SendingThread: Spawned with tid 0x1c08, 0x0., tid = 7176 file = nmmember.c, line = 511 {Thu Apr 10 21:40:09 2003 }Debug Hang :SendingThread (PID=135159169): Registered with {Thu Apr 10 21:40:09 2003 }
TRACE: SendingThread (pid=2545, tid=7176): Registered with watchdog daemon., tid = 7176 file = nmmember.c, line = 576 {Thu Apr 10 21:40:09 2003 }
TRACE: HandleJoin(): src[1] dest[1] dom[0] seq[1] sync[0], tid = ClusterListener:3076 file = nmlisten.c, line = 346 {Thu Apr 10 21:40:09 2003 }
TRACE: HandleJoin(): JOIN from node(1)->(1), tid = ClusterListener:3076 file = nmlisten.c, line = 362 {Thu Apr 10 21:40:09 2003 }
TRACE: HandleStatus(): node(0) UNKNOWN, tid = ClusterListener:3076 file = nmlisten.c, line = 404 {Thu Apr 10 21:40:10 2003 }
TRACE: HandleStatus(): src[0] dest[1] dom[0] seq[36] sync[11], tid = ClusterListener:3076 file = nmlisten.c, line = 415 {Thu Apr 10 21:40:10 2003 }
TRACE: HandleSync(): src[0] dest[1] dom[0] seq[37] sync[11], tid = ClusterListener:3076 file = nmlisten.c, line = 506 {Thu Apr 10 21:40:10 2003 }
TRACE: SendAck(): node(0) domain(0) syncSeqNo(11) type(11), tid = ClusterListener:3076 file = nmmember.c, line = 1913 {Thu Apr 10 21:40:10 2003 }
TRACE: HandleVote(): src[0] dest[1] dom[0] seq[38] sync[11], tid = ClusterListener:3076 file = nmlisten.c, line = 643 {Thu Apr 10 21:40:11 2003 }
TRACE: SendVoteInfo(): node(0) domain(0) syncSeqNo(11), tid = ClusterListener:3076 file = nmmember.c, line = 1727 {Thu Apr 10 21:40:11 2003 }
TRACE: HandleShutdown(): src[0] dest[1] dom[0] seq[0] sync[11] type[4], tid = ClusterListener:3076 file = nmlisten.c, line = 1087 {Thu Apr 10 21:40:12 2003 }
TRACE: IncrementEventValue: *(80f2920) = (1, 1), tid = ClusterListener:3076 file = unixinc.c, line = 253 {Thu Apr 10 21:40:12 2003 }--- End Dump ---
=========================================================
Similar Messages
-
Dbconsole failed to start on one RAC node
Hi
I have 2 RAC nodes (RHEL 4) and 10.2.0.1. On one dbconsole is running and on other I get the following. Earlier dbconsole
on both the nodes used to run perfectly fine. I will appreacite any suggestions to rectify this problem.
Regards
oracle@rac01<18>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> emctl start dbconsole
TZ set to Canada/Newfoundland
Oracle Enterprise Manager 10g Database Control Release 10.2.0.1.0
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
http://rac01:1158/em/console/aboutApplication
Agent Version : 10.1.0.4.1
OMS Version : Unknown
Protocol Version : 10.1.0.2.0
Agent Home : /u01/app/oracle/product/10.2/db_1/rac01_RACDB1
Agent binaries : /u01/app/oracle/product/10.2/db_1
Agent Process ID : 23329
Parent Process ID : 21132
Agent URL : http://rac01:3938/emd/main
Started at : 2007-07-25 11:37:32
Started by user : oracle
Last Reload : 2007-07-25 11:37:32
Last successful upload : (none)
Last attempted upload : (none)
Total Megabytes of XML files uploaded so far : 0.00
Number of XML files pending upload : 371
Size of XML files pending upload(MB) : 7.66
Available disk space on upload filesystem : 44.78%
Agent is already started. Will restart the agent
Stopping agent ... stopped.
Starting Oracle Enterprise Manager 10g Database Control ............................................................................................. failed.
Logs are generated in directory /u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log
oracle@rac01<19>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log>
ON OTHER NODE:
oracle@rac02<2>:/u01/app/oracle> emctl start dbconsole
TZ set to Canada/Newfoundland
Oracle Enterprise Manager 10g Database Control Release 10.2.0.1.0
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
http://rac01:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 10g Database Control .................................... started.
Logs are generated in directory /u01/app/oracle/product/10.2/db_1/rac02_RACDB2/sysman/log
oracle@rac02<3>:/u01/app/oracle>Thanks for your time and reply .
Well, here is what I got, couldn't make out from here.
Regards
oracle@rac01<19>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> ls -lart
total 13500
drwxr----- 7 oracle dba 4096 Jul 14 10:48 ..
-rw-r----- 1 oracle dba 0 Jul 14 10:48 emdctl.log
drwxrwx--- 2 oracle dba 4096 Jul 14 10:54 nmcRACDB11521
-rw-r----- 1 oracle dba 4655792 Jul 24 23:01 emoms.trc
-rw-r----- 1 oracle dba 4655792 Jul 24 23:01 emoms.log
drwxr----- 3 oracle dba 4096 Jul 25 11:35 .
-rw-r----- 1 oracle dba 4096 Jul 25 12:05 emdb.nohup.lr
-rw-r----- 1 oracle dba 1074 Jul 25 12:05 emagent_perl.trc
-rw-r----- 1 oracle dba 1731 Jul 25 12:06 emagent.log
-rw-r----- 1 oracle dba 1080 Jul 25 12:07 emagentfetchlet.trc
-rw-r----- 1 oracle dba 1080 Jul 25 12:07 emagentfetchlet.log
-rw-r----- 1 oracle dba 81089 Jul 25 13:28 emdctl.trc
-rw-r----- 1 oracle dba 3309143 Jul 25 13:28 emdb.nohup
-rw-r----- 1 oracle dba 1044518 Jul 25 13:28 emagent.trc
oracle@rac01<20>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> cat emagent.log
2007-07-14 10:50:44 Thread-3086936288 Starting Agent 10.1.0.4.1 from /u01/app/oracle/product/10.2/db_1 (00701)
2007-07-14 10:51:16 Thread-3086936288 EMAgent started successfully (00702)
2007-07-14 14:38:21 Thread-3086935744 Starting Agent 10.1.0.4.1 from /u01/app/oracle/product/10.2/db_1 (00701)
2007-07-14 14:39:00 Thread-3086935744 EMAgent started successfully (00702)
2007-07-24 07:05:06 Thread-3086935744 Starting Agent 10.1.0.4.1 from /u01/app/oracle/product/10.2/db_1 (00701)
2007-07-24 07:07:11 Thread-3086935744 target {+ASM1_rac01, osm_instance} is broken: cannot compute dynamic properties in time. (00155)
2007-07-24 07:07:14 Thread-3086935744 EMAgent started successfully (00702)
2007-07-24 12:06:27 Thread-3086935744 EMAgent normal shutdown (00703)
2007-07-24 12:08:26 Thread-3086935744 Starting Agent 10.1.0.4.1 from /u01/app/oracle/product/10.2/db_1 (00701)
2007-07-24 12:08:51 Thread-3086935744 EMAgent started successfully (00702)
2007-07-25 11:35:35 Thread-3086935744 EMAgent normal shutdown (00703)
2007-07-25 11:37:32 Thread-3086935744 Starting Agent 10.1.0.4.1 from /u01/app/oracle/product/10.2/db_1 (00701)
2007-07-25 11:39:29 Thread-3086935744 target {+ASM1_rac01, osm_instance} is broken: cannot compute dynamic properties in time. (00155)
2007-07-25 11:39:30 Thread-3086935744 EMAgent started successfully (00702)
2007-07-25 12:03:36 Thread-3086935744 EMAgent normal shutdown (00703)
2007-07-25 12:05:15 Thread-3086935744 Starting Agent 10.1.0.4.1 from /u01/app/oracle/product/10.2/db_1 (00701)
2007-07-25 12:06:23 Thread-3086935744 target {+ASM1_rac01, osm_instance} is broken: cannot compute dynamic properties in time. (00155)
2007-07-25 12:06:24 Thread-3086935744 EMAgent started successfully (00702)
oracle@rac01<21>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> cat emagentfetchlet.log
2007-07-14 11:01:44,208 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-14 14:40:29,096 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-24 07:10:44,123 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-24 12:12:48,187 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-25 11:41:25,628 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-25 12:07:30,335 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
oracle@rac01<22>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log>
oracle@rac01<22>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> tail -40 emagentfetchlet.trc
2007-07-14 11:01:44,208 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-14 14:40:29,096 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-24 07:10:44,123 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-24 12:12:48,187 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-25 11:41:25,628 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
2007-07-25 12:07:30,335 [main] WARN track.OracleInventory collectInventory.439 - ECM: The inventory location file for the special Windows NT case does not exist or is unreadable.
oracle@rac01<25>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> tail -10 emdctl.trc
2007-07-25 13:01:02 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:04:41 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:07:12 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:10:50 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:14:32 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:18:09 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:20:40 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:24:27 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:28:06 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:31:43 Thread-3086935744 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
oracle@rac01<28>:/u01/app/oracle/product/10.2/db_1/rac01_RACDB1/sysman/log> tail -10 emagent.trc
2007-07-25 13:31:44 Thread-43162528 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:31:44 Thread-43162528 ERROR pingManager: nmepm_pingReposURL: Cannot connect to http://rac01:1158/em/upload/: retStatus=-32
2007-07-25 13:32:14 Thread-74791840 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:32:14 Thread-74791840 ERROR pingManager: nmepm_pingReposURL: Cannot connect to http://rac01:1158/em/upload/: retStatus=-32
2007-07-25 13:32:14 Thread-74791840 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:32:14 Thread-74791840 ERROR pingManager: nmepm_pingReposURL: Cannot connect to http://rac01:1158/em/upload/: retStatus=-32
2007-07-25 13:32:44 Thread-74791840 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:32:44 Thread-74791840 ERROR pingManager: nmepm_pingReposURL: Cannot connect to http://rac01:1158/em/upload/: retStatus=-32
2007-07-25 13:32:44 Thread-74791840 WARN http: snmehl_connect: connect failed to (rac01:1158): Connection refused (error = 111)
2007-07-25 13:32:44 Thread-74791840 ERROR pingManager: nmepm_pingReposURL: Cannot connect to http://rac01:1158/em/upload/: retStatus=-32
Message was edited by:
Singh -
Cssd does not start in non-RAC environment Thus we can not bring up ASM
Non-RAC environment
ASM version = 11.1.0.7
HP-UX Itanium 11.23
After power outage, CSSD does not start on non-RAC environment
Running as root "/sbin/init.d/init.cssd start" does not start cssd
Oracle support tried "$ASM_HOME/bin/localconfig delete" and "$ASM_HOME/bin/localconfig add"
but it did not start CSS
Oracle support tried "$ASM_HOME/bin/localconfig reset $ASM_HOME"
It started the CSSD and the "crsctl check css" came back with CSS is healthy
But around 1 minute later it rebooted the server and when it came up again CSS does not start.
They checked /etc/inittab and it looked fine.
Before the reboot we saw this message in the /var/adm/syslog/OLDsyslog.log:
Cluster Ready Services completed waiting on dependencies
Again it is a NON-RAC environment. We only need CSSD for ASM. We do not have CRS installed on this server.
Our test system has been down for a week and we did not get the resolution from Oracle support yet !
Any pointers are greately appriciated.
Thanks,
DzungHere is the message in $ASM_HOME/log/<hostname>/alert<hostname>.log :
2010-07-16 09:42:02.956
[client(11930)]CRS-1006:The OCR location /db/app/oracle/product/11.1/cdata/localhost/local.ocr is inacce
ssible. Details in /db/app/oracle/product/11.1/log/rmodbd01/client/clscfg10.log.
2010-07-16 09:42:02.971
[client(11930)]CRS-1006:The OCR location /db/app/oracle/product/11.1/cdata/localhost/local.ocr is inacce
ssible. Details in /db/app/oracle/product/11.1/log/rmodbd01/client/clscfg10.log.
2010-07-16 09:42:03.054
[client(11930)]CRS-1013:The OCR at /db/app/oracle/product/11.1/cdata/localhost/local.ocr was successfull
y formatted using version 2. Ignore earlier CRS-1006 messages if any.
2010-07-16 09:42:46.379
[cssd(12297)]CRS-1601:CSSD Reconfiguration complete. Active nodes are rmodbd01 .
Here is the message in $ASM_HOME/log/<hostname>/cssd/cssdOUT.log:
setsid: failed with -1/1
s0clssscGetEnvOracleUser: calling getpwnam_r for user oracle
s0clssscGetEnvOracleUser: info for user oracle complete
07/16/10 09:42:36: CSSD starting
Here is the message in $ASM_HOME/log/<hostname>/cssd/ocssd.log:
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 7, from 6, changes 6
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmSetVersions: properties common to all peers: 1,2,3,4,5,6,7
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmEstablishMasterNode: MASTER for 174732166 is node(0) birth(174732166)
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmChangeMasterNode: requeued 0 RPCs
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 8, from 7, changes 7
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmMasterCMSync: Synchronizing group/lock status
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmMasterSendDBDone: group/lock status synchronization complete
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 9, from 8, changes 8
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 10, from 9, changes 9
[ CSSD]CLSS-3000: reconfiguration successful, incarnation 174732166 with 1 nodes
[ CSSD]CLSS-3001: local node number 0, master node number 0
[ CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssscSAGEInitFenceCompl: Completing kgzf fence initialization
[ CSSD]2010-07-16 09:42:46.394 [12] >TRACE: clssgmUpdateEventValue: Client listener incarn val 174732166, changes 1
[ CSSD]2010-07-16 09:42:46.395 [12] >TRACE: clssgmAllocProc: (60000000003c7120) allocated
[ CSSD]2010-07-16 09:42:46.395 [12] >TRACE: clssgmAllocProc: (60000000003c73a0) allocated
[ CSSD]2010-07-16 09:42:46.396 [14] >TRACE: Connect request from user oracle
[ CSSD]2010-07-16 09:42:46.396 [14] >TRACE: Connect request from user root
[ CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: properties of cmProc 60000000003c7120 - 1,2,3
[ CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: Connect from con(60000000003b2810) proc(60000000003c7120) pid(12350) version 11:1:1:4
[ CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: properties of cmProc 60000000003c73a0 - 1,2,3
[ CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: Connect from con(60000000003b2990) proc(60000000003c73a0) pid(12131) version 11:1:1:4
[ CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmRegisterClient: proc(1/60000000003c7120), client(1/600000000096e7c0)
[ CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (600000000096e7c0)
[ CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmJoinGrock: local grock CSS_INTERNAL_NODE_GROUP new client 600000000096e7c0 with con 60000000003b2b10, requested num 0
[ CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmAddNodeGrpMember: member (60000000009e0030) added
[ CSSD]2010-07-16 09:42:46.403 [12] >TRACE: clssgmGroupState: requested group state of group localhost_NG, member count 0
[ CSSD]2010-07-16 09:42:46.403 [12] >TRACE: clssgmGroupState: requested group state of group localhost_NG, member count 0
[ CSSD]2010-07-16 09:42:46.404 [12] >TRACE: clssgmDeadProc: proc 60000000003c73a0
[ CSSD]2010-07-16 09:42:46.404 [12] >TRACE: clssgmDestroyProc: cleaning up proc(60000000003c73a0) con(60000000003b2990) skgpid ospid 12131 with 0 clients, refcount 0
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmGroupState: requested group state of unknown group MASTER#DISKMON#GROUP
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmGroupState: requested group state of group MASTER#DISKMON#GROUP, member count 0
[ CSSD]2010-07-16 09:42:46.425 [18] >TRACE: KGZF: context successfully initialized, API version 1.4, using pipe default
[ CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssscSAGEInitFenceCompl: kgzf fence initialization successfully completed
[ CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssgmReconfigThread: CSS/GM open for global group registrations
[ CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssgmReconfigThread: completed for reconfig(174732166), with status(1)
[ CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssgmUpdateEventValue: Reconfig Event val 2, changes 2
[ CSSD]2010-07-16 09:42:46.425 [1] >TRACE: clssgmWaitOnEventValue: after Reconfig Event val 2, eval 2 waited 47
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmRegisterClient: proc(1/60000000003c7120), client(2/600000000096e870)
[ CSSD]2010-07-16 09:42:46.425 [1] >TRACE: clssgmUpdateEventValue: Reconfig Event val 0, changes 3
[ CSSD]2010-07-16 09:42:46.425 [1] >TRACE: clssgmStartNMMon: previous reconfig complete, incarnation(174732166)
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 2 (600000000096e870)
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmJoinGrock: global grock MASTER#DISKMON#GROUP#MX new client 600000000096e870 with con 60000000003b2990, requested num -1
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddGrockMember: adding member to grock MASTER#DISKMON#GROUP#MX
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(0) grock (6000000000989e50/MASTER#DISKMON#GROUP#MX)
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmQueueGrockEvent: lockName(MASTER#DISKMON#GROUP#MX) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0)
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmCommonAddMember: global lock grock MASTER#DISKMON#GROUP#MX member(0/Local) node(0) flags 0x2 0x2
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmRegisterClient: proc(1/60000000003c7120), client(3/600000000096e920)
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 3 (600000000096e920)
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmJoinGrock: global grock MASTER#DISKMON#GROUP new client 600000000096e920 with con 60000000003b2bd0, requested num 0
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddGrockMember: adding member to grock MASTER#DISKMON#GROUP
[ CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddMember: new master 0 for group(MASTER#DISKMON#GROUP)
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmAddMember: Adding fencing for member 0, group MASTER#DISKMON#GROUP, death 1, SAGE 0
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmAddMember: member (0/60000000009e0230) added. pbsz(72) prsz(0) flags 0x0 to grock (600000000098a170/MASTER#DISKMON#GROUP)
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmQueueGrockEvent: groupName(MASTER#DISKMON#GROUP) count(1) master(0) event(1), incarn 1, mbrc 1, to member 0, events 0x78, state 0x0
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmCommonAddMember: global group grock MASTER#DISKMON#GROUP member(0/Local) node(0) flags 0x0 0x1e00
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 2 (600000000096e870)
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmExitGrock: client 2 (600000000096e870), grock MASTER#DISKMON#GROUP#MX, member 0
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (60000000009e0130) in global grock MASTER#DISKMON#GROUP#MX
[ CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmRemoveMember: grock MASTER#DISKMON#GROUP#MX, member number 0 (60000000009e0130) node number 0 state 0x14 member refcnt 0 grock type 3
[ CSSD]2010-07-16 09:42:48.405 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:42:48.405 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:42:52.444 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:42:52.444 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:42:56.484 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:42:56.484 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:00.516 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:00.516 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:04.563 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:04.563 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:08.603 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:08.603 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:12.643 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:12.643 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:16.676 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:16.676 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:20.723 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:20.723 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:24.762 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:24.762 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:28.802 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:28.802 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:32.842 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:32.842 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:36.882 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:36.882 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:40.922 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:40.922 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:44.964 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:44.964 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:49.002 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:49.002 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:53.043 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:53.043 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:57.085 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:43:57.085 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:43:58.354 [12] >TRACE: clssgmAllocProc: (60000000003c7ee0) allocated
[ CSSD]2010-07-16 09:43:58.355 [14] >TRACE: Connect request from user oracle
[ CSSD]2010-07-16 09:43:58.356 [12] >TRACE: clssgmClientConnectMsg: properties of cmProc 60000000003c7ee0 - 1,2,3
[ CSSD]2010-07-16 09:43:58.356 [12] >TRACE: clssgmClientConnectMsg: Connect from con(60000000003b2990) proc(60000000003c7ee0) pid(13157) version 11:1:1:4
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmRegisterClient: proc(2/60000000003c7ee0), client(1/600000000096e870)
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (600000000096e870)
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_GROUP new client 600000000096e870 with con 60000000003b2c90, requested num -1
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_GROUP
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddMember: new master 0 for group(CLSSSCHECK_GROUP)
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddMember: Adding fencing for member 0, group CLSSSCHECK_GROUP, death 1, SAGE 0
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddMember: member (0/60000000009e0130) added. pbsz(8) prsz(8) flags 0x0 to grock (6000000000989e50/CLSSSCHECK_GROUP)
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmQueueGrockEvent: groupName(CLSSSCHECK_GROUP) count(1) master(0) event(1), incarn 1, mbrc 1, to member 0, events 0x0, state 0x0
[ CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmCommonAddMember: global group grock CLSSSCHECK_GROUP member(0/Local) node(0) flags 0x0 0x0
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (600000000096e870)
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExitGrock: client 1 (600000000096e870), grock CLSSSCHECK_GROUP, member 0
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (60000000009e0130) in global grock CLSSSCHECK_GROUP
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_GROUP, member number 0 (60000000009e0130) node number 0 state 0x14 member refcnt 0 grock type 2
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmRegisterClient: proc(2/60000000003c7ee0), client(2/600000000096e870)
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 2 (600000000096e870)
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_LOCK new client 600000000096e870 with con 60000000003b2c90, requested num -1
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_LOCK
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(0) grock (6000000000989e50/CLSSSCHECK_LOCK)
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmQueueGrockEvent: lockName(CLSSSCHECK_LOCK) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0)
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmCommonAddMember: global lock grock CLSSSCHECK_LOCK member(0/Local) node(0) flags 0x2 0x2
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 2 (600000000096e870)
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExitGrock: client 2 (600000000096e870), grock CLSSSCHECK_LOCK, member 0
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (60000000009e0130) in global grock CLSSSCHECK_LOCK
[ CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_LOCK, member number 0 (60000000009e0130) node number 0 state 0x14 member refcnt 0 grock type 3
[ CSSD]2010-07-16 09:43:58.362 [12] >TRACE: clssgmDeadProc: proc 60000000003c7ee0
[ CSSD]2010-07-16 09:43:58.362 [12] >TRACE: clssgmDestroyProc: cleaning up proc(60000000003c7ee0) con(60000000003b2990) skgpid ospid 13157 with 0 clients, refcount 0
[ CSSD]2010-07-16 09:44:01.125 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:01.125 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:05.164 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:05.164 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:09.196 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:09.196 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:13.236 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:13.236 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:17.284 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:17.284 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:21.322 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:21.322 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:25.316 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:25.316 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
[ CSSD]2010-07-16 09:44:30.333 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:30.333 [8] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes
[ CSSD]2010-07-16 09:44:34.324 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
[ CSSD]2010-07-16 09:44:34.324 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
Here is the message in $ASM_HOME/log/<hostname>/diskmon/client.log:
[ DISKMON] 07/16/2010 18:33:32.050 dskm_send_command: process 23246 sending command 8 to master diskmon listening on default pipe
[ DISKMON] 07/16/2010 18:33:32.080 dskm_send_command3: skgznp_connect failed with error 56815
[ DISKMON] 07/16/2010 18:33:32.080 dskm_send_command3: error 56815 at location skgznpcon6 - connect() - Connection refused
Here is the message in $ASM_HOME/log/<hostname>/diskmon/diskmonOUT.log:
setsid: failed with -1/1
dskm_getenv_oracle_user: calling getpwnam_r for user oracle
dskm_getenv_oracle_user: info for user oracle complete
07/16/10 18:33:31: Master Diskmon starting
Here is the message in $ASM_HOME/log/<hostname>/diskmon/diskmon.log:
[ DISKMON] 07/16/2010 09:42:35.573 dskm main: starting up
[ DISKMON] 07/16/2010 09:42:35.588 [12350:3] dskm_rac_thrd_main: running
[ DISKMON] 07/16/2010 09:42:35.588 [12350:1] dskm_rac_thrd_creat2: got the post from the css event handling thread
[ DISKMON] 07/16/2010 09:42:35.589 [12350:1] dskm main: startup complete
[ DISKMON] 07/16/2010 09:42:35.589 [12350:1] listening on -> default pipe
[ DISKMON] 07/16/2010 09:42:35.792 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
[ DISKMON] 07/16/2010 09:42:36.385 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
[ DISKMON] 07/16/2010 09:42:36.906 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
[ DISKMON] 07/16/2010 09:42:37.426 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
[ DISKMON] 07/16/2010 09:42:37.945 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
[ DISKMON] 07/16/2010 09:42:38.465 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
[ DISKMON] 07/16/2010 09:42:46.376 [12350:1] dskm_slave_thrd_creat: thread created
[ DISKMON] 07/16/2010 09:42:46.376 [12350:11] dskm_slave_thrd_main1: slave 0 running
[ DISKMON] 07/16/2010 09:42:46.376 [12350:11] dskm_process_msg5: received msg type KGZM_IDENTIFY (0x0001)
[ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_proc_identify8: client kgzf/12297, version 0x01020000, slave 0, reid cid=3e0391f05e06cfafbf7419d7cf085a44,icin=174732166,nmn=0,lnid=174732166,gid=0,gin=0,gmn=0,umemid=0,opid=0,opsn=0,lvl=node
[ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_send_version1:
[ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_send_version4: done
[ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_process_msg7: processed msg 0 type KGZM_IDENTIFY (0x0001), retcode 0
[ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_process_msg5: received msg type KGZM_KGZF_HANDSHAKE (0x0010)
[ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_proc_kgzf_handshake3: client kgzf/12297, kgzf version 0x00010004, slave 0
[ DISKMON] 07/16/2010 09:42:46.401 [12350:3] dskm_clss_ini2: successful clsssinit(), clssvers 2.1
[ DISKMON] 07/16/2010 09:42:46.402 [12350:3] dskm_clss_ini12: node rmodbd01 (0) registered in cluster
[ DISKMON] 07/16/2010 09:42:46.403 [12350:3] dskm_reid_ini12: diskmon reid cid=3e0391f05e06cfafbf7419d7cf085a44,icin=174732166,nmn=0,lnid=174732166,gid=-1,gin=-1,gmn=-1,umemid=-1,opid=12350,opsn=1279291355,lvl=process
[ DISKMON] 07/16/2010 09:42:46.424 [12350:3] dskm_sage_config: CELL storage configuration file cellinit.ora not found
[ DISKMON] 07/16/2010 09:42:46.425 [12350:3] dskm_nfy_kgzf1: notified thread kgzf enabled
[ DISKMON] 07/16/2010 09:42:46.425 [12350:11] dskm_proc_kgzf_handshake5: got the post from the hb thread
[ DISKMON] 07/16/2010 09:42:46.425 [12350:11] dskm_proc_kgzf_handshake9: done, kgzf enabled
[ DISKMON] 07/16/2010 09:42:46.425 [12350:11] dskm_process_msg7: processed msg 0 type KGZM_KGZF_HANDSHAKE (0x0010), retcode 0
[ DISKMON] 07/16/2010 09:42:46.426 [12350:3] dskm_rac_ini22: CELL storage not configured in the cluster; registered in group MASTER#DISKMON#GROUP as memno 0 (GSDGRPSZ 512) -
My apple macbook pro retina 15'' ME664 has problems with sound.Either there will be no sound at all or at startup there il be very scary and broken sound and afterwards still there would be no sound.Secondly , it does not start in one shot.atleast 2 to 3 times pressing power button for longer time starts it up and afterwards still there will be no sound. Please help
This morning i was not able to hear sound on my recently purchased macbookpro retina 15 inch model.My attempts to resolve this issue by going to system preferences and removing the mute box that was ticked as active. i had downloaded much earlier a mac version of logmein app and noticed that in the sound box noted above that there was a bar indicating that the default was now to logmein. i had never used this app. Why it started today, i have no idea. I moved the logmein application to trash and deleted the app , however it did not remove the default option which still blocked the sound.
Going to a macforum gave the tha anser that to remove the logmein app, it first had to be reinstalled a nd them use the uninstaller option ftrom the dropdown menu. for now that has resolved the failure to allow sound oputput. -
How to execute DBMS_JOB at exactly one RAC node
Hello,
after unsuccessfully searching for "RAC" and "DBMS_JOB" I open this thread.
Can you tell me how to dedicate one RAC-node for doing my batch-jobs
which are started by using dbms_job (so there is no tnsnames.ora which is used).
Thanks in advancehi,
Let's say the instances are named:I1, I2, I3, I4
Issue:ALTER SYSTEM SET JOB_QUEUE_PROCESSES=0 SCOPE=BOTH SID='I1';
ALTER SYSTEM SET JOB_QUEUE_PROCESSES=0 SCOPE=BOTH SID='I2';
ALTER SYSTEM SET JOB_QUEUE_PROCESSES=0 SCOPE=BOTH SID='I3';
ALTER SYSTEM SET JOB_QUEUE_PROCESSES=10 SCOPE=BOTH SID='I4';So that only instance I4 will run jobs.
Regards,
Yoann. -
What would happened when one RAC node's public NIC down ?
Dear all,
There's a two-node RAC on my office. As my observation, when one RAC node's public NIC is down, the "crs_stat -t" command wouldn't show any resource is OFFLINE. So at this moment if i use sqlplus to connect to my RAC db, it still will choose node1 or node2 instance randomly right? And if i've been assigned to the node's instance whose public NIC is down, my sqlplus would failed to connect to db?
So, can we say that Oracle RAC can't supply HA function if one node's public NIC is down? Or is there just any other solution to solve this issue? Any suggestion would be appreciated.Public node is down means that only one node would be able to take the load i.e. in your case it will be Node 2. It may happen that CRS is unable to record the status - in such case there may be failed connections to one node.
-
Service ora. database .db not start datbase automaticaly (RAC 11gR2)
Hi all,
after install Grid Infraestructure, RAC database and created database. All services are ok, database is ok. Everything are running fine.
So, when I make a test to reboot my servers, i see that the service ora.<database>.db are OFFLINE and not start automaticaly after reboot.
I´m using 11g R2 on Solaris 10.
Does anybody can help me ?
Thanks
Wander (Brazil)
Edited by: Wander(Brazil) on Oct 15, 2010 6:43 AM
Edited by: Wander(Brazil) on Oct 15, 2010 6:46 AMHi,
You have to follow some order for starting services in RAC database.
Check the below steps can help you:
The ORDER IS AS FOLLOWS
For Starting things:
Step 1 Start Clusterware
Note
When any host in the cluster boots,Clusterware should be started.
The scripts listed below help make this happen:
/etc/inittab
/etc/rc5.d/S96init.crs
/etc/init.d/init.crs
/etc/init.d/init.crsd
/etc/init.d/init.cssd
/etc/init.d/init.evmd
Be aware of this for 2 reasons:
1. It may be unneccesary for you to start Clusterware. It might already be started.
2. The script, /etc/init.d/init.crs, can be used to start Clusterware if you want to use it.
To start Clusterware manually, maybe use this script instead of init.crs:
To start Clusterware manually, maybe use this script instead of init.crs:
#!/bin/sh
# Run as root
# start
ssh dbcl1n1 /home/app/crs/bin/crsctl start crs
or
$ sudo /home/app/crs/bin/crsctl start crs
Step 2: Start nodeapps
After you start Clusterware, you may start nodeapps.
Use this script to start nodeapps:
#! /bin/sh
# start
srvctl start nodeapps -n dbcl1n1
Step 3:Start Database
After you start the nodeapps, you may start the Databases.
Use this script to start the databases:
#! /bin/sh
# ps
ssh dbcl1n1 ps -ef | grep dbw
# status
srvctl status database -d AUCD
# start
srvctl start database -d AUCD
Step 4:Start the application after starting the Database
Best regards,
Rafi.
http://rafioracledba.blogspot.com -
Lync 2010 Cleint will not start on one Windows XP Client
We have on Windows XP (SP3) Client and on this Client lync 2010 with the actaul CU6 will not start.
No entry in the eventlog or any other errors.
Deinstall, restart, install, restart does not help.
Has anyone had the same issue on one Windows Xp client?
regards Holger Technical Specialist UCHi
Can you login with a different account and test it? exit the lync client:
Also go to the %userprofile%\AppData\Local\Microsoft\Communicator folder (this is for windows 7 but you get where i am going) and remove the contents (sip) folder and then try login again.
Hope this helps. Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. -
Scan-vip running only on one RAC node
Hi ,
While setting up RAC11.2 on Centos 5.7 , I was getting this error during the grid installation:
PRCR-1079 : Failed to start resource ora.scan1.vip
CRS-5005: IP Address: 192.168.100.208 is already in use in the network
CRS-2674: Start of 'ora.scan1.vip' on 'falcen6b' failed
CRS-2632: There are no more servers to try to place resource 'ora.scan1.vip' on that would satisfy its placement policy
PRCR-1079 : Failed to start resource ora.scan2.vip
CRS-5005: IP Address: 192.168.100.209 is already in use in the network
CRS-2674: Start of 'ora.scan2.vip' on 'falcen6b' failed
CRS-2632: There are no more servers to try to place resource 'ora.scan2.vip' on that would satisfy its placement policy
PRCR-1079 : Failed to start resource ora.scan3.vip
CRS-5005: IP Address: 192.168.100.210 is already in use in the network
CRS-2674: Start of 'ora.scan3.vip' on 'falcen6b' failed
CRS-2632: There are no more servers to try to place resource 'ora.scan3.vip' on that would satisfy its placement policy
I figured that the scan service is able to run only on one node at a time. When I stopped the service on rac1 and started it on rac2 the service is starting.
But I think for the grid installation the scan service has to simultaneously run on both the nodes.
How do I resolve it?
Any suggestions please.
PS - I am planning to try with the patch 11.0.2.3 but it will be a while till i get access to it.
Till then can someone suggest a workaround?Hi Balazs Papp and onedbguru,
I was able to resolve that error by running the following command on rac2, now that part of the installer passed.
crsctl start res ora.scan1.vip
However the cluster verification utility is failing at the end of installer.
When I executed the below command, this is my output:
[oracle@falcen6a grid]$ ./runcluvfy.sh stage -post crsinst -n falcen6a,falcen6b -verbose
Performing post-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "falcen6a"
Destination Node Reachable?
falcen6a yes
falcen6b yes
Result: Node reachability check passed from node "falcen6a"
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
falcen6b passed
falcen6a passed
Result: User equivalence check passed for user "oracle"
Checking time zone consistency...
Time zone consistency check passed.
Checking Cluster manager integrity...
Checking CSS daemon...
Node Name Status
falcen6b running
falcen6a running
Oracle Cluster Synchronization Services appear to be online.
Cluster manager integrity check passed
UDev attributes check for OCR locations started...
Result: UDev attributes check passed for OCR locations
UDev attributes check for Voting Disk locations started...
Result: UDev attributes check passed for Voting Disk locations
Check default user file creation mask
Node Name Available Required Comment
falcen6b 0022 0022 passed
falcen6a 0022 0022 passed
Result: Default user file creation mask check passed
Checking cluster integrity...
Cluster is divided into 2 partitions
Partition 1 consists of the following members:
Node Name
falcen6b
Partition 2 consists of the following members:
Node Name
falcen6a
Cluster integrity check failed. Cluster is divided into 2 partition(s).
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations
ERROR:
PRVF-4193 : Asm is not running on the following nodes. Proceeding with the remaining nodes.
Checking OCR config file "/etc/oracle/ocr.loc"...
OCR config file "/etc/oracle/ocr.loc" check successful
ERROR:
PRVF-4195 : Disk group for ocr location "+DATA" not available on the following nodes:
Checking size of the OCR location "+DATA" ...
Size check for OCR location "+DATA" successful...
OCR integrity check failed
Checking CRS integrity...
ERROR:
PRVF-5316 : Failed to retrieve version of CRS installed on node "falcen6b"
The Oracle clusterware is healthy on node "falcen6b"
The Oracle clusterware is healthy on node "falcen6a"
CRS integrity check failed
Checking node application existence...
Checking existence of VIP node application
Node Name Required Status Comment
falcen6b yes unknown failed
falcen6a yes unknown failed
Result: Check failed.
Checking existence of ONS node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no online passed
Result: Check ignored.
Checking existence of GSD node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no does not exist ignored
Result: Check ignored.
Checking existence of EONS node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no online passed
Result: Check ignored.
Checking existence of NETWORK node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no online passed
Result: Check ignored.
Checking Single Client Access Name (SCAN)...
SCAN VIP name Node Running? ListenerName Port Running?
falcen6-scan unknown false LISTENER 1521 false
WARNING:
PRVF-5056 : Scan Listener "LISTENER" not running
Checking name resolution setup for "falcen6-scan"...
SCAN Name IP Address Status Comment
falcen6-scan 192.168.100.210 passed
falcen6-scan 192.168.100.208 passed
falcen6-scan 192.168.100.209 passed
Verification of SCAN VIP and Listener setup failed
OCR detected on ASM. Running ACFS Integrity checks...
Starting check to see if ASM is running on all cluster nodes...
PRVF-5137 : Failure while checking ASM status on node "falcen6b"
Starting Disk Groups check to see if at least one Disk Group configured...
Disk Group Check passed. At least one Disk Group configured
Task ACFS Integrity check failed
Checking Oracle Cluster Voting Disk configuration...
Oracle Cluster Voting Disk configuration check passed
Checking to make sure user "oracle" is not in "root" group
Node Name Status Comment
falcen6b does not exist passed
falcen6a does not exist passed
Result: User "oracle" is not part of "root" group. Check passed
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
Check: CTSS Resource running on all nodes
Node Name Status
falcen6b passed
falcen6a passed
Result: CTSS resource check passed
Querying CTSS for time offset on all nodes...
Result: Query of CTSS for time offset passed
Check CTSS state started...
Check: CTSS state
Node Name State
falcen6b Observer
falcen6a Observer
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP Configuration file check passed
Checking daemon liveness...
Check: Liveness for "ntpd"
Node Name Running?
falcen6b yes
falcen6a yes
Result: Liveness check passed for "ntpd"
Checking NTP daemon command line for slewing option "-x"
Check: NTP daemon command line
Node Name Slewing Option Set?
falcen6b yes
falcen6a yes
Result:
NTP daemon slewing option check passed
Checking NTP daemon's boot time configuration, in file "/etc/sysconfig/ntpd", for slewing option "-x"
Check: NTP daemon's boot time configuration
Node Name Slewing Option Set?
falcen6b yes
falcen6a yes
Result:
NTP daemon's boot time configuration check for slewing option passed
NTP common Time Server Check started...
NTP Time Server "133.243.236.19" is common to all nodes on which the NTP daemon is running
NTP Time Server "133.243.236.18" is common to all nodes on which the NTP daemon is running
NTP Time Server "210.173.160.86" is common to all nodes on which the NTP daemon is running
NTP Time Server ".LOCL." is common to all nodes on which the NTP daemon is running
Check of common NTP Time Server passed
Clock time offset check from NTP Time Server started...
Checking on nodes "[falcen6b, falcen6a]"...
Check: Clock time offset from NTP Time Server
Time Server: 133.243.236.19
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 15.332 passed
falcen6a -1.503 passed
Time Server "133.243.236.19" has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Time Server: 133.243.236.18
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 15.115 passed
falcen6a -1.614 passed
Time Server "133.243.236.18" has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Time Server: 210.173.160.86
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 15.219 passed
falcen6a -1.527 passed
Time Server "210.173.160.86" has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Time Server: .LOCL.
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 0.0 passed
falcen6a 0.0 passed
Time Server ".LOCL." has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Clock time offset check passed
Result: Clock synchronization check using Network Time Protocol(NTP) passed
Oracle Cluster Time Synchronization Services check passed
Post-check for cluster services setup was unsuccessful on all the nodes.
[oracle@falcen6a grid]$
Any suggestions? -
Vip not started in newly added node in 10gR2
Hi
I'm facing a issue while adding third node in 10gR2 env.
after pre-requisite check has been success, I run $CRS_HOME/oui/bin/addNode.sh script.
after GUI based installation , I run all the three script , which has been completed successfully,
Then I run vipca as root user to configure vip , but it thrown error ,CRS-1006, CRS 1028, crs-0215, crs-0223
I check status of OCR voting and their permission and status on node 3 , all looks good.
CRS-1006: No more member to consider CRS-0215: could not start resource 'ora.rac3.vip'
CRS-1028: dependency analysis failed because of CRS-0223 resource 'ora.rac1.gsd' has placement error.
CRS-1028: dependency analysis failed because of CRS-0223 resource 'ora.rac2.gsd' has placement error.
CRS-1028: dependency analysis failed because of CRS-0223 resource 'ora.rac1.ons' has placement error.
CRS-1028: dependency analysis failed because of CRS-0223 resource 'ora.rac2.ons' has placement error.
Can u please suggest.....
Regards
rajeev....[oracle@rac1 cluvfy]$ ./runcluvfy.sh stage -post crsinst -n rac1,rac2,rac3 -verbose
Performing post-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "rac1"
Destination Node Reachable?
rac2 yes
rac1 yes
rac3 yes
Result: Node reachability check passed from node "rac1".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
rac2 passed
rac1 passed
rac3 passed
Result: User equivalence check passed for user "oracle".
Checking Cluster manager integrity...
Checking CSS daemon...
Node Name Status
rac2 running
rac1 running
rac3 running
Result: Daemon status check passed for "CSS daemon".
Cluster manager integrity check passed.
Checking cluster integrity...
Node Name
rac1
rac2
rac3
Cluster integrity check passed
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations.
Uniqueness check for OCR device passed.
Checking the version of OCR...
OCR of correct Version "2" exists.
Checking data integrity of OCR...
Data integrity check for OCR passed.
OCR integrity check passed.
Checking CRS integrity...
Checking daemon liveness...
Check: Liveness for "CRS daemon"
Node Name Running
rac2 yes
rac1 yes
rac3 yes
Result: Liveness check passed for "CRS daemon".
Checking daemon liveness...
Check: Liveness for "CSS daemon"
Node Name Running
rac2 yes
rac1 yes
rac3 yes
Result: Liveness check passed for "CSS daemon".
Checking daemon liveness...
Check: Liveness for "EVM daemon"
Node Name Running
rac2 yes
rac1 yes
rac3 yes
Result: Liveness check passed for "EVM daemon".
Liveness of all the daemons
Node Name CRS daemon CSS daemon EVM daemon
rac2 yes yes yes
rac1 yes yes yes
rac3 yes yes yes
Checking CRS health...
Check: Health of CRS
Node Name CRS OK?
rac2 yes
rac1 yes
rac3 yes
Result: CRS health check passed.
CRS integrity check passed.
Checking node application existence...
Checking existence of VIP node application
Node Name Required Status Comment
rac2 yes exists passed
rac1 yes exists passed
rac3 yes exists passed
Result: Check passed.
Checking existence of ONS node application
Node Name Required Status Comment
rac2 no exists passed
rac1 no exists passed
rac3 no exists passed
Result: Check passed.
Checking existence of GSD node application
Node Name Required Status Comment
rac2 no exists passed
rac1 no exists passed
rac3 no exists passed
Result: Check passed.
Post-check for cluster services setup was successful. -
One RAC node is down give the following error when starting the database!
wHEN TRYING TO START THE DATABASE ON RAC ENVIORNMENT
SQL> connect sys as sysdba
Enter password:
Connected to an idle instance.
SQL> startup
ORA-27102: out of memory
HPUX-ia64 Error: 12: Not enough space
SQL> exit
Disconnected
When we r trying to start the database it said
$ bdf
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 2097152 240944 1841896 12% /
/dev/vg00/lvol1 344064 115376 226944 34% /stand
/dev/vg00/lvol8 10485760 9370960 1106232 89% /var
/dev/vg00/lvol7 4866048 2557680 2290400 53% /usr
/dev/vg00/u02 10485760 3502229 6547116 35% /u02
/dev/vg00/u01 10485760 10476596 9164 100% /u01
/dev/vg00/lvol4 2097152 601872 1483944 29% /tmp
/dev/vg00/lvol6 4194304 3231000 955792 77% /opt
/dev/vg00/lvol5 524288 311520 211136 60% /home
WHERE /U01 WAS 100%. Now i emptied the space in /u01 to
$ bdf
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 2097152 240944 1841896 12% /
/dev/vg00/lvol1 344064 115376 226944 34% /stand
/dev/vg00/lvol8 10485760 9370960 1106232 89% /var
/dev/vg00/lvol7 4866048 2557680 2290400 53% /usr
/dev/vg00/u02 10485760 3502229 6547116 35% /u02
/dev/vg00/u01 10485760 9508934 930943 91% /u01
/dev/vg00/lvol4 2097152 601872 1483944 29% /tmp
/dev/vg00/lvol6 4194304 3231000 955792 77% /opt
/dev/vg00/lvol5 524288 311520 211136 60% /home
When trying to start the db again its giving the following error...
SQL> connect sys as sysdba
Enter password:
Connected to an idle instance.
SQL> startup
ORA-27102: out of memory
HPUX-ia64 Error: 12: Not enough space
SQL> exit
Disconnected
here i changed the sga_target and now it says
ORACLE instance started.
Total System Global Area 436207616 bytes
Fixed Size 1297912 bytes
Variable Size 148648456 bytes
Database Buffers 285212672 bytes
Redo Buffers 1048576 bytes
ORA-01105: mount is incompatible with mounts by other instances
ORA-19808: recovery destination parameter mismatch
What could be the issue..
Ur help would be highly appreciated...Hello
SQL> startup
ORA-27102: out of memory
HPUX-ia64 Error: 12: Not enough space
SQL> exiterror is not related to space on your mount point. it is related to memory.
if you are getting this error means chekc at the OS level whether something is consuming more memory due to which it is not allowing oracle to allocate sga.
Check top/sar/glance to see who is consuming more memory
Total System Global Area 436207616 bytes
Fixed Size 1297912 bytes
Variable Size 148648456 bytes
Database Buffers 285212672 bytes
Redo Buffers 1048576 bytes
ORA-01105: mount is incompatible with mounts by other instances
ORA-19808: recovery destination parameter mismatchit is not the best practice to maintain differnet parameters for each instance in RAC env. also check db_recovery_file_dest and db_recovery_file_dest_size is same on all node. it should be same i.e it should be a shared location.
Anil Malkai -
Cluster_resource not starting in oracle RAC 11gR2 after adding
I am using a RAC of 2 nodes (rac1,rac2) on windows 2008 Server R2 with Oracle 11gr2 grid installed.
I am trying to add a cluster resource using
*> crsctl add resource check -type cluster_resource -attr "ACTION_SCRIPT=C:\app\agent.exe"*
It is success and is shown using crs_stat -t
C:\app\11.2.0>crs_stat -t
Name Type Target State Host
check clus...esource ONLINE UNKNOWN rac2
Now I am trying to start this using
*>crsctl start resource check*
Its failing with following error:
CRS-2672: Attempting to start 'check' on 'rac2'
CRS-2674: Start of 'check' on 'rac2' failed
CRS-2679: Attempting to clean 'check' on 'rac2'
CRS-2678: 'check' on 'rac2' has experienced an unrecoverable failure
CRS-0267: Human intervention required to resume its availability.
CRS-5802: Unable to start the agent process
CRS-4000: Command Start failed, or completed with errors.
Please help me in solving this problem.Hi FZheng ,
Thanks for your reply.
agent.exe is a application containing start/stop/clean/check cases.Its a simple application through which we want to check cluster resource addition.
Yes Grid user has privileges to rum agent.exe.
One thing more I want to add,this problem is only coming for ORACLE 11gR2.
All other versions (11gR1,10g) its working fine.
If we check crsd.log in %GRID_HOME%/log/crsd directory ..the logs are as follows:
*[ AGFW][3616] Starting the agent: C:\app\11.2.0\grid\bin\scriptagent.exe with user id: administrator and incarnation:9*
*[ AGFW][3616]Created alert : (:CRSAGF00123:) : Failed to start the agent process: C:\app\11.2.0\grid\bin\scriptagent.exe Category: -2 Operation: OpenService Loc: getusrtok2 OS error: 1060 Other : OpenService Failed*
*[ AGFW][3616]Created alert : (:CRSAGF00126:) : Agent start failed*
I guess the problem is with scriptagent.exe which is used to start 3rd party cluster resources.
Please comment. -
FF will not start for ONE Windows 7 User. All others work fine.
I try to start FF and it hangs. Loads into Project Manager, but no browser window.
So then I need to amend the title to "firefox.exe -P works on all Windows USERS except one.
Can I adjust the profile for User A from a Windows logon for User B?
type into the Run dialog "firefox.exe -P" and the system just stares at me...no UI, no GUI, no dialog, nothing.
and yes, I have uninstalled, uninstalled deleting profiles, started in safe mode, and uninstalled and removed all FF references from Regedit.
I've done everything short of a rollback to ver 2.0 -
Itunes does not start for one user but works for others on the same mac
All of a sudden my itunes does not fire up.
All other users on the same mac it is working fine.
I have deleted and reinstalled itunes.
Any help would be great.
CheersAll of a sudden my itunes does not fire up.
All other users on the same mac it is working fine.
I have deleted and reinstalled itunes.
Any help would be great.
Cheers -
Archive REDO When One RAC Node is Down
I have a question about how redo log get archived when one of the instances in a two node RAC cluster is down (not open, not mounted).
For example, let's assume instance1 was shutdown and only instance 2 is running.
I have 6 redo logs:
<font face="courier">
SQL> SELECT GROUP#, THREAD#, SEQUENCE#, STATUS FROM V$LOG;
GROUP# THREAD# SEQUENCE# STATUS
1 1 3390 INACTIVE
2 1 3389 INACTIVE
3 1 3391 ACTIVE
5 2 3886 INACTIVE
4 2 3887 INACTIVE
6 2 3888 CURRENT
</font>
If I run the following statement what will happen?
<font face="courier">
SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;
</font>
The documentation says +Specify CURRENT to manually archive the current redo log file group of the specified thread, forcing a log switch. If you omit the THREAD parameter, then Oracle Database archives all redo log file groups from all enabled threads, including logs previous to current logs. You can specify CURRENT only when the database is open.+
Would Oracle archive sequence# 3391 from thread 1 even though the instance is not open?When your instance are not working, it means that it doesn't have any CURRENT redo log. So when you issue switching logfile - it launches archiving only current redo log, for working instance only. That 3391 redo log is not current, it has Active status, that means that it could be needed for recovery purpose, but it had to be archived earlier.
Maybe you are looking for
-
Forex Fluctuation entries not to be passed in BRS
Hi I have made payment to Vendor on 12th December $1000 (1$ @ Rs. 45.45) = Rs. 4545 At this time entry passed was: Vendor A/c... Dr Rs. 4545 To Outgoing Bank A/c Rs. 4545 Now on 15th December i am running BRS (FF67). Rate mentioned in OB08 on 15th
-
Installed a few extensions in dreamweaver. As ever it gives a few options to access. Where one is to tick the windows>insert option so that I get extra icons on the screen, I can access the extension. However on these I should be able to go to the in
-
My repaint( ) just re-paints but doesn't clear the back
Hi, i am using repaint( ) with the timer method to randomly draw 100 randomly colored lines.But every time the timer calls actionperformed which contains repaint( ), the application just paints the new 100 lines on top of the old ones, not cleaning t
-
Oracle database product feature matrix
Hi, 1)I just went through the doc http://www.oracle.com/us/products/database/039449.pdf. But it does not mention which feature requires additional license and which one comes by deafault. Can anyone give me link of a doc which mentions whether addit
-
I have a 110 series. One or more of the cartridges are running out. A test print would tell me which one. How do I do a test print?