Oracle clusterware services issue

Dear all,
After i installed oracle clusterware and the installation was successfull. the following case takes place.
1- Both Nodes are running.
2- to test failover, i reboot Node 2.
3- Node 2 doesn't start and stop at "Applying computer setting"
4- to start Node 2, I have to issue "crsctl stop crs" at Node 1.
5- When i become able to log in to Node 2, I log in to the machine and then i issue "crsctl start crs" at Node 1.
the system Architecture is as follows.
1- Windows 2003 SP1.
2- Oracle RAC 10.2.0.4, full installer.
the ocssd.log on Node 1 shows the following,
Oracle Database 10g CRS Release 10.2.0.4.0 Production Copyright 1996, 2008 Oracle. All rights reserved.
[    CSSD]2009-02-07 18:33:08.488 >USER: Copyright 2009, Oracle version 10.2.0.4.0
[    CSSD]2009-02-07 18:33:08.488 >USER: CSS daemon log for node racdbsrv1, number 1, in cluster BOGCluster
[  clsdmt]Listening to (ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61180))
[    CSSD]2009-02-07 18:33:08.488 [2168] >TRACE: clssscmain: local-only set to false
[    CSSD]2009-02-07 18:33:08.504 [2168] >TRACE: clssnmReadNodeInfo: added node 1 (racdbsrv1) to cluster
[    CSSD]2009-02-07 18:33:08.504 [2168] >TRACE: clssnmReadNodeInfo: added node 2 (racdbsrv2) to cluster
[    CSSD]2009-02-07 18:33:08.520 [528] >TRACE: clssnm_skgxninit: Compatible vendor clusterware not in use
[    CSSD]2009-02-07 18:33:08.520 [528] >TRACE: clssnm_skgxnmon: skgxn init failed
[    CSSD]2009-02-07 18:33:08.520 [2168] >TRACE: clssnmNMInitialize: misscount set to (30)
[    CSSD]2009-02-07 18:33:08.520 [2168] >TRACE: clssnmNMInitialize: Network heartbeat thresholds are: impending reconfig 15000 ms, reconfig start (misscount) 30000 ms
[    CSSD]2009-02-07 18:33:08.535 [2168] >TRACE: clssnmDiskStateChange: state from 1 to 2 disk (0/\\.\votedsk1)
[    CSSD]2009-02-07 18:33:08.535 [1580] >TRACE: clssnmvDPT: spawned for disk 0 (\\.\votedsk1)
[    CSSD]2009-02-07 18:33:08.535 [2168] >TRACE: clssnmDiskStateChange: state from 1 to 2 disk (1/\\.\votedsk2)
[    CSSD]2009-02-07 18:33:08.535 [2320] >TRACE: clssnmvDPT: spawned for disk 1 (\\.\votedsk2)
[    CSSD]2009-02-07 18:33:08.535 [2168] >TRACE: clssnmDiskStateChange: state from 1 to 2 disk (2/\\.\votedsk3)
[    CSSD]2009-02-07 18:33:08.535 [188] >TRACE: clssnmvDPT: spawned for disk 2 (\\.\votedsk3)
[    CSSD]2009-02-07 18:33:10.548 [1580] >TRACE: clssnmDiskStateChange: state from 2 to 4 disk (0/\\.\votedsk1)
[    CSSD]2009-02-07 18:33:10.548 [2408] >TRACE: clssnmvKillBlockThread: spawned for disk 0 (\\.\votedsk1) initial sleep interval (1000)ms
[    CSSD]2009-02-07 18:33:10.548 [2320] >TRACE: clssnmDiskStateChange: state from 2 to 4 disk (1/\\.\votedsk2)
[    CSSD]2009-02-07 18:33:10.548 [2520] >TRACE: clssnmvKillBlockThread: spawned for disk 1 (\\.\votedsk2) initial sleep interval (1000)ms
[    CSSD]2009-02-07 18:33:10.564 [188] >TRACE: clssnmDiskStateChange: state from 2 to 4 disk (2/\\.\votedsk3)
[    CSSD]2009-02-07 18:33:10.564 [2168] >TRACE: clssscSclsFatal: read value of disable
[    CSSD]2009-02-07 18:33:10.564 [2168] >TRACE: clssscSclsFatal: read value of disable
[    CSSD]2009-02-07 18:33:10.564 [2280] >TRACE: clssnmvKillBlockThread: spawned for disk 2 (\\.\votedsk3) initial sleep interval (1000)ms
[    CSSD]2009-02-07 18:33:10.564 [2296] >TRACE: clssnmFatalThread: spawned
[    CSSD]2009-02-07 18:33:10.564 [2320] >TRACE: clssnmReadDskHeartbeat: node(2) is down. rcfg(1) wrtcnt(2) LATS(1245125) Disk lastSeqNo(2)
[    CSSD]2009-02-07 18:33:10.564 [1580] >TRACE: clssnmReadDskHeartbeat: node(2) is down. rcfg(1) wrtcnt(1) LATS(1245125) Disk lastSeqNo(1)
[    CSSD]2009-02-07 18:33:10.564 [2248] >TRACE: clssnmClusterListener: Listening on (ADDRESS=(PROTOCOL=tcp)(HOST=racdbsrv1-priv)(PORT=49895))
[    CSSD]2009-02-07 18:33:10.579 [2292] >TRACE: clssgmclientlsnr: listening on (ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61101))
[    CSSD]2009-02-07 18:33:10.579 [2248] >TRACE: clssnmClusterListener: Probing node racdbsrv2 (2), probcon(01619C60)
[    CSSD]2009-02-07 18:33:10.579 [2240] >TRACE: clssgmPeerListener: Listening on (ADDRESS=(PROTOCOL=tcp)(DEV=1188)(HOST=192.168.0.22)(PORT=1125))
[    CSSD]2009-02-07 18:33:10.579 [188] >TRACE: clssnmReadDskHeartbeat: node(2) is down. rcfg(1) wrtcnt(3) LATS(1245140) Disk lastSeqNo(3)
[    CSSD]2009-02-07 18:33:11.172 [2248] >TRACE: clssnmConnComplete: MSGSRC 2, type 5, node 2, flags 0x0001, con 0213A960, probe 01619C60
[    CSSD]2009-02-07 18:33:11.172 [2248] >TRACE: clssnmConnComplete: node 2, racdbsrv2, con(0213A960), probcon(01619C60), ninfcon(00000000), node unique 1234020788, prev unique 0, msg unique 1234020788 node state 0
[    CSSD]2009-02-07 18:33:11.172 [2248] >TRACE: clssnmsendConnAck: node 2, node state 0
[    CSSD]2009-02-07 18:33:11.172 [2248] >TRACE: clssnmConnComplete: connecting to node 2 (con 0213A960), ninfcon (0213A960), state (0)
[    CSSD]2009-02-07 18:33:11.562 [2248] >WARNING: clssnmeventhndlr: Receive failure during connect node 2 (racdbsrv2), con(01619C60), probe(01619C60), rc=10
[    CSSD]2009-02-07 18:33:11.562 [2248] >TRACE: clssnmDiscHelper: racdbsrv2, node(2) connection failed, con (01619C60), probe(01619C60)
[    CSSD]2009-02-07 18:33:11.562 [2248] >TRACE: clssnmDiscHelper: connected to node 2 , ninfcon (0213A960), state (0)
[    CSSD]2009-02-07 18:33:11.562 [1580] >TRACE: clssnmReadDskHeartbeat: node(2) is down. rcfg(1) wrtcnt(3) LATS(1246125) Disk lastSeqNo(3)
[    CSSD]2009-02-07 18:33:11.562 [2320] >TRACE: clssnmReadDskHeartbeat: node(2) is down. rcfg(1) wrtcnt(3) LATS(1246125) Disk lastSeqNo(3)
[    CSSD]2009-02-07 18:33:11.672 [188] >TRACE: clssnmReadDskHeartbeat: node(2) is down. rcfg(1) wrtcnt(4) LATS(1246234) Disk lastSeqNo(4)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmRcfgMgrThread: Local Join
[    CSSD]2009-02-07 18:33:17.570 [2232] >WARNING: clssnmLocalJoinEvent: takeover succ
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmDoSyncUpdate: Initiating sync 1
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmDoSyncUpdate: diskTimeout set to (27000)ms
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: Ack message type (11)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: node(1) is ALIVE
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: node(2) is ALIVE
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSendSync: syncSeqNo(1)
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmHandleSync: diskTimeout set to (27000)ms
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmHandleSync: Acknowledging sync: src[1] srcName[racdbsrv1] seq[1] sync[1]
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitForAcks: Ack message type(11), ackCount(2)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitForAcks: done, msg type(11)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmDoSyncUpdate: node(1) is transitioning from joining state to active state
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmDoSyncUpdate: node(2) is transitioning from joining state to active state
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: Ack message type (13)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: node(2) is ACTIVE
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmSendVoteInfo: node(1) syncSeqNo(1)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitForAcks: Ack message type(13), ackCount(1)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitForAcks: done, msg type(13)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmCheckDskInfo: Checking disk info...
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmCheckDskInfo: diskTimeout set to (200000)ms
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmEvict: Start
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitOnEvictions: Start
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: Ack message type (15)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSetupAckWait: node(2) is ACTIVE
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmSendUpdate: syncSeqNo(1)
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmUpdateNodeState: node 0, state (0/0) unique (0/0) prevConuni(0) birth (0/0) (old/new)
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmUpdateNodeState: node 1, state (2/3) unique (1234020788/1234020788) prevConuni(0) birth (1/1) (old/new)
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmUpdateNodeState: node 2, state (2/3) unique (1234020788/1234020788) prevConuni(0) birth (1/1) (old/new)
[    CSSD]2009-02-07 18:33:17.570 [2248] >USER: clssnmHandleUpdate: SYNC(1) from node(1) completed
[    CSSD]2009-02-07 18:33:17.570 [2248] >USER: clssnmHandleUpdate: NODE 1 (racdbsrv1) IS ACTIVE MEMBER OF CLUSTER
[    CSSD]2009-02-07 18:33:17.570 [2248] >USER: clssnmHandleUpdate: NODE 2 (racdbsrv2) IS ACTIVE MEMBER OF CLUSTER
[    CSSD]2009-02-07 18:33:17.570 [2248] >TRACE: clssnmHandleUpdate: diskTimeout set to (200000)ms
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitForAcks: Ack message type(15), ackCount(1)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmWaitForAcks: done, msg type(15)
[    CSSD]2009-02-07 18:33:17.570 [2232] >TRACE: clssnmDoSyncUpdate: Sync 1 complete!
[    CSSD]2009-02-07 18:33:17.586 [2320] >TRACE: clssnmvFatalCheck: fatal 1, sclsfatal 0
[    CSSD]2009-02-07 18:33:17.586 [1580] >TRACE: clssnmvFatalCheck: fatal 1, sclsfatal 0
[    CSSD]2009-02-07 18:33:17.586 [188] >TRACE: clssnmvFatalCheck: fatal 1, sclsfatal 0
[    CSSD]2009-02-07 18:33:17.648 [2248] >TRACE: clssnmSendFatalOn: req to syncLeader(1)
[    CSSD]2009-02-07 18:33:17.664 [188] >TRACE: clssnmvFatalOnUpdate: handling req from 1 in op 2
[    CSSD]2009-02-07 18:33:17.664 [1580] >TRACE: clssnmvFatalOnUpdate: handling req from 1 in op 2
[    CSSD]2009-02-07 18:33:17.680 [2168] >USER: NMEVENT_SUSPEND [00][00][00][00]
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmReconfigThread: started for reconfig (1)
[    CSSD]2009-02-07 18:33:17.680 [3000] >USER: NMEVENT_RECONFIG [00][00][00][06]
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmEstablishConnections: 2 nodes in cluster incarn 1
[    CSSD]2009-02-07 18:33:17.680 [2240] >TRACE: clssgmInitialRecv: (01619C60) accepted a new connection from node 2 born at 1 active (2, 2), vers (10,3,1,2)
[    CSSD]2009-02-07 18:33:17.680 [2240] >TRACE: clssgmInitialRecv: conns done (2/2)
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmEstablishMasterNode: MASTER for 1 is node(1) birth(1)
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmChangeMasterNode: requeued 0 RPCs
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmMasterCMSync: Synchronizing group/lock status
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmMasterSendDBDone: group/lock status synchronization complete
[    CSSD]CLSS-3000: reconfiguration successful, incarnation 1 with 2 nodes
[    CSSD]CLSS-3001: local node number 1, master node number 1
[    CSSD]2009-02-07 18:33:17.680 [3000] >TRACE: clssgmReconfigThread: completed for reconfig(1), with status(1)
[    CSSD]2009-02-07 18:33:17.680 [188] >TRACE: clssnmvFatalOnUpdate: handling req from 2 in op 2
[    CSSD]2009-02-07 18:33:17.695 [1580] >TRACE: clssnmvFatalOnUpdate: handling req from 2 in op 2
[    CSSD]2009-02-07 18:33:17.695 [2320] >TRACE: clssnmvFatalOnUpdate: handling req from 1 in op 2
[    CSSD]2009-02-07 18:33:17.711 [2320] >TRACE: clssnmvFatalOnUpdate: handling req from 2 in op 2
[    CSSD]2009-02-07 18:33:18.553 [2296] >TRACE: clssnmFatalThread: Fatal mode enabled
[    CSSD]2009-02-07 18:35:59.953 [2248] >WARNING: clssnmeventhndlr: Receive failure with node 2 (racdbsrv2), state 3, con(0213A960), probe(00000000), rc=11
[    CSSD]2009-02-07 18:35:59.953 [2248] >TRACE: clssnmDiscHelper: racdbsrv2, node(2) connection failed, con (0213A960), probe(00000000)
[    CSSD]2009-02-07 18:35:59.953 [2240] >TRACE: clssgmPeerDeactivate: node 2 (racdbsrv2), death 0, state 0x1 connstate 0xf
[    CSSD]2009-02-07 18:36:15.055 [2236] >WARNING: clssnmPollingThread: node racdbsrv2 (2) at 50 1.246708e-307artbeat fatal, eviction in 14.766 seconds
[    CSSD]2009-02-07 18:36:15.055 [2236] >TRACE: clssnmPollingThread: node racdbsrv2 (2) is impending reconfig, flag 1, misstime 15234
[    CSSD]2009-02-07 18:36:15.055 [2236] >TRACE: clssnmPollingThread: diskTimeout set to (27000)ms impending reconfig status(1)
[    CSSD]2009-02-07 18:36:16.054 [2236] >WARNING: clssnmPollingThread: node racdbsrv2 (2) at 50 1.246708e-307artbeat fatal, eviction in 13.766 seconds
[    CSSD]2009-02-07 18:36:23.051 [2236] >WARNING: clssnmPollingThread: node racdbsrv2 (2) at 75 1.246708e-307artbeat fatal, eviction in 6.766 seconds
[    CSSD]2009-02-07 18:36:27.049 [2236] >WARNING: clssnmPollingThread: node racdbsrv2 (2) at 90 1.246708e-307artbeat fatal, eviction in 2.766 seconds
[    CSSD]2009-02-07 18:36:28.049 [2236] >WARNING: clssnmPollingThread: node racdbsrv2 (2) at 90 1.246708e-307artbeat fatal, eviction in 1.766 seconds
[    CSSD]2009-02-07 18:36:29.048 [2236] >WARNING: clssnmPollingThread: node racdbsrv2 (2) at 90 1.246708e-307artbeat fatal, eviction in 0.766 seconds
[    CSSD]2009-02-07 18:36:29.829 [2236] >TRACE: clssnmPollingThread: Eviction started for node racdbsrv2 (2), flags 0x0001, state 3, wt4c 0
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmDoSyncUpdate: Initiating sync 2
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmDoSyncUpdate: diskTimeout set to (27000)ms
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSetupAckWait: Ack message type (11)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSetupAckWait: node(1) is ALIVE
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSendSync: syncSeqNo(2)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitForAcks: Ack message type(11), ackCount(1)
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmHandleSync: diskTimeout set to (27000)ms
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmHandleSync: Acknowledging sync: src[1] srcName[racdbsrv1] seq[6] sync[2]
[    CSSD]2009-02-07 18:36:29.829 [2168] >USER: NMEVENT_SUSPEND [00][00][00][06]
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitForAcks: done, msg type(11)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmDoSyncUpdate: Terminating node 2, racdbsrv2, misstime(30015) state(5)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSetupAckWait: Ack message type (13)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitForAcks: Ack message type(13), ackCount(1)
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmSendVoteInfo: node(1) syncSeqNo(2)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitForAcks: done, msg type(13)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmCheckDskInfo: Checking disk info...
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmEvict: Start
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmEvict: Evicting node 2, racdbsrv2, birth 1, death 2, impendingrcfg 1, stateflags 0x1
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitOnEvictions: Start
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitOnEvictions: node 2, racdbsrv2, undead 0
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSetupAckWait: Ack message type (15)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmSendUpdate: syncSeqNo(2)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitForAcks: Ack message type(15), ackCount(1)
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmUpdateNodeState: node 0, state (0/0) unique (0/0) prevConuni(0) birth (0/0) (old/new)
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmUpdateNodeState: node 1, state (3/3) unique (1234020788/1234020788) prevConuni(0) birth (1/1) (old/new)
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmUpdateNodeState: node 2, state (5/0) unique (1234020788/1234020788) prevConuni(1234020788) birth (1/1) (old/new)
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmDeactivateNode: node 2 (racdbsrv2) left cluster

[    CSSD]2009-02-07 18:36:29.829 [2248] >USER: clssnmHandleUpdate: SYNC(2) from node(1) completed
[    CSSD]2009-02-07 18:36:29.829 [2248] >USER: clssnmHandleUpdate: NODE 1 (racdbsrv1) IS ACTIVE MEMBER OF CLUSTER
[    CSSD]2009-02-07 18:36:29.829 [2248] >TRACE: clssnmHandleUpdate: diskTimeout set to (200000)ms
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmWaitForAcks: done, msg type(15)
[    CSSD]2009-02-07 18:36:29.829 [2232] >TRACE: clssnmDoSyncUpdate: Sync 2 complete!
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmReconfigThread: started for reconfig (2)
[    CSSD]2009-02-07 18:36:29.829 [2832] >USER: NMEVENT_RECONFIG [00][00][00][02]
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupGrocks: cleaning up grock ORACRS_MEMBER_racdbsrv1 type 3
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupGrocks: cleaning up grock ORACRS_MEMBER_racdbsrv2 type 3
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupOrphanMembers: cleaning up remote mbr(0) grock(_ORA_CRS_MEMBER_racdbsrv2) birth(1/0)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupGrocks: cleaning up grock crs_version type 2
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupOrphanMembers: cleaning up remote mbr(0) grock(crs_version) birth(1/0)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupOrphanMembers: cleaning up remote mbr(4) grock(crs_version) birth(1/0)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupGrocks: cleaning up grock ocr_BOGCluster type 2
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupOrphanMembers: cleaning up remote mbr(2) grock(ocr_BOGCluster) birth(1/0)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupGrocks: cleaning up grock CRSDMAIN type 2
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupOrphanMembers: cleaning up remote mbr(2) grock(CRSDMAIN) birth(1/0)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupGrocks: cleaning up grock EVMDMAIN type 2
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmCleanupOrphanMembers: cleaning up remote mbr(2) grock(EVMDMAIN) birth(1/0)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmEstablishConnections: 1 nodes in cluster incarn 2
[    CSSD]2009-02-07 18:36:29.829 [2240] >TRACE: clssgmPeerListener: connects done (1/1)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmEstablishMasterNode: MASTER for 2 is node(1) birth(1)
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmMasterCMSync: Synchronizing group/lock status
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmMasterSendDBDone: group/lock status synchronization complete
[    CSSD]CLSS-3000: reconfiguration successful, incarnation 2 with 1 nodes
[    CSSD]CLSS-3001: local node number 1, master node number 1
[    CSSD]2009-02-07 18:36:29.829 [2832] >TRACE: clssgmReconfigThread: completed for reconfig(2), with status(1)
[    CSSD]2009-02-07 18:39:08.230 [2248] >TRACE: clssnmConnComplete: MSGSRC 2, type 5, node 2, flags 0x0001, con 02148A68, probe 00000000
[    CSSD]2009-02-07 18:39:08.230 [2248] >TRACE: clssnmConnComplete: node 2, racdbsrv2, con(02148A68), probcon(00000000), ninfcon(00000000), node unique 1234021161, prev unique 0, msg unique 1234021161 node state 0
[    CSSD]2009-02-07 18:39:08.230 [2248] >TRACE: clssnmsendConnAck: node 2, node state 0
[    CSSD]2009-02-07 18:39:08.230 [2248] >TRACE: clssnmSendConnAck: connected to node 2 , ninfcon (02148A68), state (0)
[    CSSD]2009-02-07 18:39:08.230 [2248] >TRACE: clssnmConnComplete: connecting to node 2 (con 02148A68), ninfcon (02148A68), state (0)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmDoSyncUpdate: Initiating sync 3
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmDoSyncUpdate: diskTimeout set to (27000)ms
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: Ack message type (11)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: node(1) is ALIVE
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: node(2) is ALIVE
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSendSync: syncSeqNo(3)
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmHandleSync: diskTimeout set to (27000)ms
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmHandleSync: Acknowledging sync: src[1] srcName[racdbsrv1] seq[10] sync[3]
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitForAcks: Ack message type(11), ackCount(2)
[    CSSD]2009-02-07 18:39:09.571 [2168] >USER: NMEVENT_SUSPEND [00][00][00][02]
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitForAcks: done, msg type(11)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmDoSyncUpdate: node(2) is transitioning from joining state to active state
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: Ack message type (13)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: node(2) is ACTIVE
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmSendVoteInfo: node(1) syncSeqNo(3)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitForAcks: Ack message type(13), ackCount(1)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitForAcks: done, msg type(13)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmCheckDskInfo: Checking disk info...
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmEvict: Start
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitOnEvictions: Start
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: Ack message type (15)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSetupAckWait: node(2) is ACTIVE
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmSendUpdate: syncSeqNo(3)
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmUpdateNodeState: node 0, state (0/0) unique (0/0) prevConuni(0) birth (0/0) (old/new)
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmUpdateNodeState: node 1, state (3/3) unique (1234020788/1234020788) prevConuni(0) birth (1/1) (old/new)
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmUpdateNodeState: node 2, state (2/3) unique (1234021161/1234021161) prevConuni(0) birth (3/3) (old/new)
[    CSSD]2009-02-07 18:39:09.571 [2248] >USER: clssnmHandleUpdate: SYNC(3) from node(1) completed
[    CSSD]2009-02-07 18:39:09.571 [2248] >USER: clssnmHandleUpdate: NODE 1 (racdbsrv1) IS ACTIVE MEMBER OF CLUSTER
[    CSSD]2009-02-07 18:39:09.571 [2248] >USER: clssnmHandleUpdate: NODE 2 (racdbsrv2) IS ACTIVE MEMBER OF CLUSTER
[    CSSD]2009-02-07 18:39:09.571 [2248] >TRACE: clssnmHandleUpdate: diskTimeout set to (200000)ms
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitForAcks: Ack message type(15), ackCount(1)
[    CSSD]2009-02-07 18:39:09.571 [3444] >TRACE: clssgmReconfigThread: started for reconfig (3)
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmWaitForAcks: done, msg type(15)
[    CSSD]2009-02-07 18:39:09.571 [3444] >USER: NMEVENT_RECONFIG [00][00][00][06]
[    CSSD]2009-02-07 18:39:09.571 [2232] >TRACE: clssnmDoSyncUpdate: Sync 3 complete!
[    CSSD]2009-02-07 18:39:09.571 [3444] >TRACE: clssgmEstablishConnections: 2 nodes in cluster incarn 3
[    CSSD]2009-02-07 18:39:30.802 [2240] >TRACE: clsc_send_msg: (01FD5E08) NS err (12571, 12560), transport (533, 57, 0)
[    CSSD]2009-02-07 18:39:30.802 [2240] >ERROR: clssgmPeerEventHndlr: send failed (01619C60), rc 3
[    CSSD]2009-02-07 18:39:35.575 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:39:35.575 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:39:37.759 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:39:37.759 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:40:36.038 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:40:36.054 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:40:38.238 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:40:38.238 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:41:36.519 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:41:36.519 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:41:38.705 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:41:38.705 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:42:37.040 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:42:37.040 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:42:39.241 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:42:39.241 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:43:37.560 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:43:37.560 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:43:39.762 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:43:39.778 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:38.097 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:38.112 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:40.298 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:40.298 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:40.532 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:40.548 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:46.762 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:44:46.762 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:45:38.633 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:45:38.633 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:45:40.834 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:45:40.850 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:46:39.155 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:46:39.155 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:46:41.389 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:46:41.389 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:47:39.713 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:47:39.728 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:47:41.931 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:47:41.931 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:48:40.271 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:48:40.271 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:48:42.474 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:48:42.474 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:49:40.814 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:49:40.814 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:49:43.016 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:49:43.016 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:50:41.357 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:50:41.357 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:50:43.559 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:50:43.559 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:51:41.915 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:51:41.915 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:51:44.102 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:51:44.102 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:52:42.459 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:52:42.475 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:52:44.646 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:52:44.646 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:53:43.025 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:53:43.041 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:53:45.182 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:53:45.197 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:40.777 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:40.777 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:43.605 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:43.605 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:45.777 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:45.777 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:47.339 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:54:47.339 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:55:44.184 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:55:44.184 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:55:46.340 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:55:46.340 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:56:44.779 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:56:44.795 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:56:46.888 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:56:46.904 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:57:45.358 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:57:45.374 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:57:47.468 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:57:47.468 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:58:45.937 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:58:45.937 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:58:48.030 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:58:48.030 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:59:46.494 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:59:46.494 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:59:48.588 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 18:59:48.588 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:00:47.052 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:00:47.052 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:00:49.161 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:00:49.161 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:01:47.625 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:01:47.625 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:01:49.718 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:01:49.718 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:02:48.182 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:02:48.182 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:02:50.291 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:02:50.291 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:03:48.739 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:03:48.755 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:03:50.849 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:03:50.864 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:41.093 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:41.093 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:48.092 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:48.108 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:49.311 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:49.311 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:51.420 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:04:51.435 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:05:49.882 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:05:49.882 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:05:51.991 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:05:52.006 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:06:50.437 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:06:50.437 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:06:52.546 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:06:52.562 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:07:50.993 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:07:50.993 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:07:53.102 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:07:53.117 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:08:51.548 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:08:51.548 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:08:53.673 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:08:53.673 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:09:52.103 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:09:52.118 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:09:54.243 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:09:54.243 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:10:52.673 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:10:52.673 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:10:54.798 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:10:54.798 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:11:53.257 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:11:53.257 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:11:55.382 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:11:55.382 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:12:53.826 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:12:53.826 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:12:55.966 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:12:55.966 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:13:54.426 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:13:54.426 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced
[    CSSD]2009-02-07 19:13:56.536 [2292] >WARNING: clssgmBroadcastMap: node 2 quiesced

Similar Messages

  • Ntp Server for Oracle 11gr2 (oracle clusterware) on HP-UX

    we are planning to configure a ntp server on Windows 2003 server for our Oracle 11gr2 RAC (2 nodes) on HP-UX but we have some issues to be clear. Following are the details for our environment.
    RAC : Oracle Clusterware (2 nodes)
    Database : 11gr2
    OS: HP-UX B11
    Hardware: HP Intergrity Blade Servers BL860c i2
    ntp server : HP Proliant
    if our ntp server fails how the both nodes will manage the time?

    if our ntp server fails how the both nodes will manage the time?Use NTP insure that all cluster nodes must be synchronized to the same time ,in 9i , 10g and 11gR2 Oracle Clusterware includes the Cluster Time Synchronization Service (CTSS) . On startup Clusterware checks for a NTP configuration, if found, CTSS goes into Observer mode. This means it will monitor the clock synchronization and report in the Oracle Clusterware alert log if it finds a problem. If it does not find a NTP configuration, CTSS will be active .
    Check
    http://docs.oracle.com/cd/E26996_01/E18548/html/manager_ntpconfig.html
    http://docs.oracle.com/cd/E23824_01/html/821-1454/time-20.html

  • HACMP and Oracle Clusterware

    Hi Guys , I need some technical guidance regarding HACMP and Oracle Clusterware. I am designing an Oracle maximum Availability architecture for a client on 4 Nodes of IBM 570 PSeries servers on Oracle 10G RAC. The configuration includes IBM HACMP and Oracle Clusterware. No I need do know if I can fully rely on Oracle Clusterware as my Clusterware or I can configure both IBM HACMP and Oracle Clusterware for some services. Can these two clusterware coexist ??

    Yes they can co-exist. But my question is why complicate things. You cannot have a RAC cluster without the Oracle Clusterware. Meaning if you install HACMP you will have to install Oracle Clusterware also on top of this. Why complicate the stack... keep it simple.. we have been using Oracle clusteware on AIX without HACMP without any issues so far.
    Also you should check www.oracleracsig.org website for some good leads into implementing RAC on AIX.

  • Oracle 10g to 11g Upgrade - Oracle Clusterware problem

    Oracle10g RAC (2 Nodes) RHEL 4 64bit
    Hi All,
    I have began the procedure for upgrading Oracle Clusterware to 11g. However, during the install OUI was giving me certain error messages like it couldn't transfer the OUI installlogs to node 2. Well, I kept pushing through the install and after I ran the last rootupgrade script on node 2 it gave me the following error:
    Checking the existence of nodeapps on this node
    Exception in thread "main" java.lang.UnsupportedClassVersionError: oracle/ops/opsctl/OPSCTLDriver (Unsupported major.minor version 49.0)
    at java.lang.ClassLoader.defineClass0(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:539)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:251)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:55)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:194)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:187)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:289)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:274)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:302)
    Creating '/crs/home/install/paramfile.crs' with data used for CRS configuration
    Exception in thread "main" java.lang.UnsupportedClassVersionError: oracle/ops/opsctl/OPSCTLDriver (Unsupported major.minor version 49.0)
    at java.lang.ClassLoader.defineClass0(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:539)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:251)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:55)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:194)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:187)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:289)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:274)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:302)
    Failed to retrieve VIP details
    Exception in thread "main" java.lang.UnsupportedClassVersionError: oracle/ops/opsctl/OPSCTLDriver (Unsupported major.minor version 49.0)
    at java.lang.ClassLoader.defineClass0(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:539)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:251)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:55)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:194)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:187)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:289)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:274)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:302)
    Failed to retrieve VIP details
    Setting CRS configuration values in /crs/home/install/paramfile.crs
    So I ignored the error for the time being and after that I checked the status of crs and it gave me the following:
    [oracle@vtl-rac2 crsd]$ crsstatus
    HA Resource Target State
    ora.VMRACDEV.VMRACDEV1.inst ONLINE ONLINE on vtl-rac1
    ora.VMRACDEV.VMRACDEV2.inst ONLINE ONLINE on vtl-rac2
    ora.VMRACDEV.db ONLINE ONLINE on vtl-rac1
    ora.vtl-rac1.ASM1.asm ONLINE ONLINE on vtl-rac1
    ora.vtl-rac1.LISTENER_VTL-RAC1.lsnr ONLINE ONLINE on vtl-rac1
    ora.vtl-rac1.gsd ONLINE ONLINE on vtl-rac1
    ora.vtl-rac1.ons ONLINE OFFLINE
    ora.vtl-rac1.vip ONLINE ONLINE on vtl-rac1
    ora.vtl-rac2.ASM2.asm ONLINE ONLINE on vtl-rac2
    ora.vtl-rac2.LISTENER_VTL-RAC2.lsnr ONLINE ONLINE on vtl-rac2
    ora.vtl-rac2.gsd ONLINE ONLINE on vtl-rac2
    ora.vtl-rac2.ons ONLINE OFFLINE
    ora.vtl-rac2.vip ONLINE ONLINE on vtl-rac2
    So i tried to bring it back up by doing a crs_stop -all and crs_start -all and it gave me the following error:
    vtl-rac1 : CRS-1019: Resource ora.vtl-rac2.ons (application) cannot run on vtl-rac1
    Start of `ora.vtl-rac1.ons` on member `vtl-rac1` failed.
    vtl-rac2 : CRS-1019: Resource ora.vtl-rac1.ons (application) cannot run on vtl-rac2
    CRS-0223: Resource 'ora.VMRACDEV.db' has placement error.
    CRS-0215: Could not start resource 'ora.vtl-rac1.ons'.
    CRS-0215: Could not start resource 'ora.vtl-rac2.ons'.
    I am thinking that all of this is caused by the various issues I had with the install. If I am incorrect, then please let me know. If it is true, I would like to know if there are any 11g docs on how to clean up a failed clusterware upgrade. Any advice would be greatly appreciated on any of the situations I am having.
    Thank you

    Hi Chandra,
    Did CVU report any problems before the upgrade?No, there were no errors reported by CVU before the upgrade.
    I don't there is note out there for cleaning 11g CRS
    install...and I think can very use the 10g CRS note -
    239998.1.Yeah I might have to go that way.
    I have both the 11g CRS install and upgrade from 10g
    to 11CRS at
    http://chandradba.blogspot.com/2007/08/oracle-11g-rac-
    install-on-red-hat-50.html
    and
    http://chandradba.blogspot.com/2008/02/oracle-10g-crs-
    upgrade-to-11g-crs.html
    see if it helps.Yup, your guide is very simple, clear and error proof :) That's how mine when pretty much except right around 75% I started getting these strange errors of files not being able to be transferred to node 2. Anyways, it shouldn't be a problem as none of those errors were configuration related...or else I would have a messed up cluster.
    Well, I actually rebooted both machines and now the whole CRS stack is up!! So I guess I am ok. We'll just have to wait and see.
    Thanks for your help Chandra...I always appreciate it.

  • Oracle Authentication Services for OS11.1.1.3 - Unix Support

    Hello
    I am trying to locate what versions of Oracle Auth services for OS support unix? and versions of unix that oracle supports?
    appreciate any guidance.
    j

    If you contact Oracle Support, they can give you an estimated release date. This will be a 'no sooner than' release date. Releases may always slip if issues have been found that need to be fixed before the patchset is released to the public.

  • Install OEM 12c fails on "start oracle management service". win2008

    Windows Server 2008 R2 Enterprise SP1 x64
    Oracle Database 11g release 11.2.0.3.0
    I install OEM 12c(12.1.0.3) following this manual: Installing Enterprise Manager System
    before start i've installed WebLogic Server 10.3.6 and aplied 2 patches specified in this manual
    installation type is advanced, small, so i have created database from the appropriate template
    i also installed cygwin and started ssh-service
    the installation of OEM process fails on 73% "start oracle management service" with this message:
    INFO: oracle.sysman.top.oms:Starting OMS ...
    INFO: oracle.sysman.top.oms:Executing command: cmd /c set NEED_EXIT_CODE=1&&C:\Oracle\Middleware\oms\bin\emctl.bat start oms
    INFO: oracle.sysman.top.oms:Oracle Enterprise Manager Cloud Control 12c Release 3 
    INFO: oracle.sysman.top.oms:Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
    INFO: oracle.sysman.top.oms:Windows service OracleManagementServer_EMGC_OMS1_1 failed to be started
    INFO: oracle.sysman.top.oms:Oracle Management Server is Down
    INFO: oracle.sysman.top.oms:Please check C:/Oracle/gc_inst/em/EMGC_OMS1\sysman\log\emctl.log for error details
    SEVERE: oracle.sysman.top.oms:Starting of OMS failed.
    in C:/Oracle/gc_inst/em/EMGC_OMS1\sysman\log\emctl.log i got this:
    2013-09-06 12:24:27,012 [main] ERROR commands.BaseCommand logAndPrint.620 - Oracle Management Server Could Not Be Started
    Check EM Server log file for details: C:\Oracle\gc_inst\user_projects\domains\GCDomain\servers\EMGC_OMS1\logs\EMGC_OMS1.out
    2013-09-06 12:24:27,012 [main] INFO  commands.StartCommand startOMS.337 - startOMS finished with result: 1
    2013-09-06 12:24:30,974 [main] INFO  commands.StartCommand startOMS.371 - net start return code is 2
    2013-09-06 12:24:30,974 [main] ERROR commands.BaseCommand logAndPrint.620 - Windows service OracleManagementServer_EMGC_OMS1_1 failed to be started
    2013-09-06 12:24:34,094 [main] INFO  commands.BaseCommand printMessage.404 - statusOMS finished with result: 8
    2013-09-06 12:24:34,094 [main] ERROR commands.BaseCommand logAndPrint.620 - Please check C:/Oracle/gc_inst/em/EMGC_OMS1\sysman\log\emctl.log for error details
    and in C:\Oracle\gc_inst\user_projects\domains\GCDomain\servers\EMGC_OMS1\logs\EMGC_OMS1.out this:
    <06.09.2013 12:24:01 ICT> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) 64-Bit Server VM Version 23.25-b01 from Oracle Corporation>
    <06.09.2013 12:24:01 ICT> <Info> <Management> <BEA-141107> <Version: WebLogic Server Temporary Patch for 13349651 Thu May 24 10:18:28 IST 2012
    WebLogic Server Temporary Patch for BUG14482558 Fri Jan 11 11:20:16 IST 2013
    WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050 >
    <06.09.2013 12:24:03 ICT> <Info> <Security> <BEA-090065> <Getting boot identity from user.>
    Enter username to boot WebLogic server:Error: Failed to get value from Standard Input
    Enter password to boot WebLogic server:
    <06.09.2013 12:24:04 ICT> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:
    There are 1 nested errors:
    weblogic.management.ManagementException: Booting as admin server, but servername, EMGC_OMS1, does not match the admin server name, EMGC_ADMINSERVER
         at weblogic.management.provider.internal.RuntimeAccessService.start(RuntimeAccessService.java:67)
         at weblogic.t3.srvr.ServerServicesManager.startService(ServerServicesManager.java:461)
         at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(ServerServicesManager.java:166)
         at weblogic.t3.srvr.T3Srvr.initializeStandby(T3Srvr.java:881)
         at weblogic.t3.srvr.T3Srvr.startup(T3Srvr.java:568)
         at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:469)
         at weblogic.Server.main(Server.java:71)
    >
    <06.09.2013 12:24:04 ICT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
    wlst:
    wls:/offline>
    what I did wrong? how to start WLS server?

    Hi,
    First of all with EM 12.1.0.x any version you are not required to install wls separately, we have single installer for em and wls. Is there any specific reason you installer wls separately?
    Coming back to your issue: can you check
    %OMS_HOME%\BIN>emctl istart oms completes successfully.
    %OMS_HOME%\BIN>emctl istop oms completes successfully.
    Also open an SR with Oracle support and share the number. We should debug such issue through SR so that we can share logs and come find a root cause.

  • Need link for official oracle clusterware & RAC installaiton on sol10 SPARC

    Hi Forum,
    I have been trying out all types of possible combination of installations for oracle clusterware and RAC on solaris 10 SPARC system. But as every document is not exact in all the steps so i have not succeeded at all.
    It would be nice if anyone can give me the link for ORACLE CLUSTERWARE AND RAC INSTALLATION ON SOLARIS 10 SPARC. It would be nice if the document covers almost all the steps.
    setup : sun fire 445 (2 No)
    RAW disk Setup for storage.
    OS: solaris 10 11/08
    oracle: oracle 10.2.0.1 for DB and clusterware.
    Please help me out. Its urgent.
    Regards
    Prakash

    I have been using the same document b14205. But the steps for the oracle clusterware OUI installations are not at all specified in. in 4 steps it has completed the whole OUI setup. where as i ran into so many issues that i had to look into google to find the soultion... and doc which is more presise with the steps.. well thanks for the other document... its given a new dimention to my efforts
    regards
    prakash

  • Solaris SMF configuration for Oracle CSS service

    Below is the code to create oracle CSS service with solaris SMF, it will create smf service with instance name as "default" where as i need to change it to "css"
    svc:/application/oracle/css:default
    change service instance name to
    svc:/application/oracle/css:css
    i don't know about smf and xml, can somebody help me to change css service name ?
    <?xml version="1.0"?>
    <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
    <!-- $Id: oracle-database-css.xml,v 1.4 2006/03/19 21:19:40 joomul Exp $
      The contents of this file are subject to the terms of the
      Common Development and Distribution License, Version 1.0 only
      (the "License").  You may not use this file except in compliance
      with the License.
      You can obtain a copy of the license at
       http://www.opensource.org/licenses/cddl1.php
      See the License for the specific language governing permissions and
      limitations under the License.
      Copyright 2005, 2006 Joost Mulders.
      This document contains a service definition for the
        Cluster Synchronization Services (CSS) process.
      This process is required for ASM and RAC. It should be started via smf(5)
      only for *non-RAC* systems. On RA clusters, this daemon is started via crs.
    -->
    <service_bundle type='manifest' name='oracle-database-css'>
    <service
      name='application/oracle/css'
      type='service'
      version='1'>
      <create_default_instance enabled='false' />
      <!-- CSS depends on multi-user. That takes care of details like filesystem -->
      <dependency
       name='multi-user'
       grouping='require_all'
       restart_on='error'
       type='service'>
       <service_fmri
        value='svc:/milestone/multi-user:default' />
      </dependency>
      <exec_method
       type='method'
       name='start'
       exec='$ORACLE_HOME/bin/ocssd'
       timeout_seconds='30' >
       <method_context>
        <method_credential
         user='oracle'
         group='dba'
         supp_groups=':default'
         privileges=':default'
         limit_privileges=':default' />
        <method_environment>
         <envvar name='ORACLE_HOME' value='/u01/app/oracle/product/10.2.0.1/asm_1' />
        </method_environment>
       </method_context>
      </exec_method>
      <exec_method
       type='method'
       name='stop'
       exec=':kill'
       timeout_seconds='60' >
       <method_context>
        <method_credential
         user='oracle'
         group='dba'
         supp_groups=':default'
         privileges=':default'
         limit_privileges=':default' />
        <method_environment>
         <envvar name='ORACLE_HOME' value='/u01/app/oracle/product/10.2.0.1/asm_1' />
        </method_environment>
       </method_context>
      </exec_method>
      <!-- ocssd.bin does not detach -->
      <property_group name='startd' type='framework'>
       <propval name='duration' type='astring' value='child' />
       <propval name='modify_authorization' type='astring'
        value='solaris.smf.manage.oracle.database' />
      </property_group>
      <property_group name='general' type='framework'>
       <propval name='modify_authorization' type='astring'
        value='solaris.smf.manage.oracle.database' />
       <propval name='action_authorization' type='astring'
        value='solaris.smf.manage.oracle.database' />
      </property_group>
      <stability value='Unstable' />
      <template>
       <common_name>
        <loctext xml:lang='C'>
         Oracle Cluster Synchronization Services (CSS)
        </loctext>
       </common_name>
       <documentation>
        <doc_link
         name='Intro to Oracle Clusterware and Oracle Real Application Clusters'
         uri='http://download-east.oracle.com/docs'/>
       </documentation>
      </template>
    </service>
    </service_bundle>Edited by: sachinonnet on Jan 12, 2010 2:01 AM

    Hi,
    i got the solution as below
    removed below line from service xml
    <create_default_instance enabled='false' />
    created another xml file for instance xml, where i specified the name of instance.
    <?xml version="1.0"?>
    <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
    <!-- $Id: oracle-database-instance.xml,v 1.12 2006/03/12 19:51:20 joost Exp $
      The contents of this file are subject to the terms of the
      Common Development and Distribution License, Version 1.0 only
      (the "License").  You may not use this file except in compliance
      with the License.
      You can obtain a copy of the license at
       http://www.opensource.org/licenses/cddl1.php
      See the License for the specific language governing permissions and
      limitations under the License.
      Copyright 2005,2006 Joost Mulders.
      This document contains a service definition for an Oracle database instance.
      It contains some instance specific variables like ORACLE_SID and ORACLE_HOME.
      These must be filled in below.
      Other -generic database- service properties are inherited from the parent
      service, 'application/oracle/database'.
      A database instance can be brought under SMF control by:
       * running /lib/svc/method/ora-smf autoimport
    *OR*
       * filling in the blanks (marked with '' or ':default') below
       * svccfg import <file>
    -->
    <service_bundle type='manifest' name='oracle-database-instance'>
    <service
      name='application/oracle/database'
      type='service'
      version='1'>
      <dependency name='oracle-asm' grouping='require_all' restart_on='none' type='service'>
        <service_fmri value='svc:/application/oracle/database:ASM' />
      </dependency>
      <!-- The SMF instance name MUST match the database instance -->
      <instance name='PDB' enabled='false'>
       <method_context
        working_directory='/u01/app/oracle/product/10.2.0.1/db'
        project=':default'
        resource_pool=':default'>
        <!--
          The credentials of the user with which the method is executed.
        -->
        <method_credential
         user='oracle'
         group='dba'
         supp_groups=':default'
         privileges=':default'
         limit_privileges=':default'/>
        <method_environment>
         <envvar name='ORACLE_SID' value='pdb' />
         <envvar name='ORACLE_HOME' value='/u01/app/oracle/product/10.2.0.1/db' />
        </method_environment>
       </method_context>
      <!--
       A generic options property group for this database.
       This property group has currently only one variable: instance_type.
       instance_type describes the type of database instance. It's value can be
       RDBMS or ASM. RDBMS denotes a classic database instance and ASM is a storage
       manager instance.
      -->
      <property_group name='options' type='application'>
       <stability value='External' />
       <propval name='instance_type' type='astring' value='RDBMS' />
       <propval name='modify_authorization' type='astring'
        value='solaris.smf.manage.oracle.database' />
      </property_group>
      </instance>
    </service>
    </service_bundle>

  • Oracle Clusterware Configuration Assistant fails

    I am installing clusterware 11.1.0.6 and the runcluvfy commands all came back as passed, except for the VIP, which the documentation says to ignore. The whole install goes to 100%, then it runs the configuration assistants.
    It falls over on the first of these: Oracle Configuration Assistant. If I retry it, the same addition is placed in the logfile, as below:
    INFO: Starting to execute configuration assistants
    INFO: Command = C:\WINDOWS\system32\cmd /c call C:\oracle\11.1.0\crs/install/crssetup.config.bat
    clscfg: EXISTING configuration version 4 detected.
    clscfg: version 4 is 11 Release 1.
    Successfully accumulated necessary OCR keys.
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 1: vm-rac1 vm-rac1-priv vm-rac1
    node 2: vm-rac2 vm-rac2-priv vm-rac2
    clscfg: Arguments check out successfully.
    NO KEYS WERE WRITTEN. Supply -force parameter to override.
    -force is destructive and will destroy any previous cluster
    configuration.
    Step 1: checking status of CRS cluster
    Step 2: creating directories (C:\oracle\11.1.0\crs)
    Step 3: configuring OCR repository
    Oracle Cluster Registry already initialized
    Step 4: configuring safe mode for CRS stack
    Step 5: Starting up CRS stack on all nodes
    vm-rac1 failed to startup service OracleCSService, err(1068)
    vm-rac2 failed to startup service OracleCSService, err(1068)
    Command = C:\WINDOWS\system32\cmd /c call C:\oracle\11.1.0\crs/install/crssetup.config.bat has failed
    INFO: Configuration assistant "Oracle Clusterware Configuration Assistant" failed
    *** Starting OUICA ***
    Oracle Home set to C:\oracle\11.1.0\crs
    Configuration directory is set to C:\oracle\11.1.0\crs\cfgtoollogs. All xml files under the directory will be processed
    INFO: The "C:\oracle\11.1.0\crs\cfgtoollogs\configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
    INFO:
    The Runconfig command constructed is null
    INFO: Since the option is to overwrite the existing C:\oracle\11.1.0\crs\cfgtoollogs\configToolFailedCommands file, backing it up
    INFO: The backed up file name is C:\oracle\11.1.0\crs\cfgtoollogs\configToolFailedCommands.bak.1
    SEVERE: OUI-25031:Some of the configuration assistants failed/cancelled. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any "Recommended" assistants means your system will not be correctly configured.
    1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures.
    2. Fix the errors causing these failures.
    3. Select the failed assistants and click the 'Retry' button to retry them.
    Please help.

    Sorry:
    alertvm-rac1.log:
    Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
    2009-05-20 15:44:25.531
    [client(3792)]CRS-1006:The OCR location \\.\ocrcfg is inaccessible. Details in C:\oracle\11.1.0\crs\log\vm-rac1\client\ocrconfig_3788.log.
    2009-05-20 15:44:25.609
    [client(3792)]CRS-1006:The OCR location \\.\ocrcfg is inaccessible. Details in C:\oracle\11.1.0\crs\log\vm-rac1\client\ocrconfig_3788.log.
    2009-05-20 15:44:25.671
    [client(3792)]CRS-1006:The OCR location \\.\ocrcfg is inaccessible. Details in C:\oracle\11.1.0\crs\log\vm-rac1\client\ocrconfig_3788.log.
    2009-05-20 15:44:25.687
    [client(3792)]CRS-1006:The OCR location \\.\ocrcfg is inaccessible. Details in C:\oracle\11.1.0\crs\log\vm-rac1\client\ocrconfig_3788.log.
    2009-05-20 15:44:28.921
    [client(3792)]CRS-1013:The OCR at \\.\ocrcfg was successfully formatted using version 2. Ignore earlier CRS-1006 messages if any.
    2009-05-20 15:44:40.875
    [client(3796)]CRS-1801:Cluster vm-rac_cluster configured with nodes vm-rac1 vm-rac2 .
    client\clscfg.log:
    Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
    2009-05-20 16:31:03.796: [  OCROSD][3196]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 16:31:03.812: [  OCROSD][3196]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 16:31:03.875: [  OCROSD][3196]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 16:31:03.875: [  OCROSD][3196]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 16:31:03.921: [  OCROSD][3196]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 16:31:03.937: [  OCROSD][3196]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    client\clsfmt.log:
    Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
    2009-05-20 15:44:40.031: [  OCROSD][3872]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:40.031: [  OCROSD][3872]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:40.187: [  CLSFMT][3872]clsfmt: successfully initialized file \\.\votedsk1
    client\ocrconfig_3788.log:
    Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
    2009-05-20 15:44:25.453: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.484: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.484: [  OCRRAW][3792]propriogid:1: INVALID FORMAT
    2009-05-20 15:44:25.531: [  OCRRAW][3792]ibctx:1:ERROR: INVALID FORMAT
    2009-05-20 15:44:25.562: [  OCRRAW][3792]proprinit:problem reading the bootblock or superbloc 22
    2009-05-20 15:44:25.562: [ default][3792]a_init:7!: Backend init unsuccessful : [22]
    2009-05-20 15:44:25.562: [ OCRCONF][3792]ocrconfig starts...
    2009-05-20 15:44:25.578: [ OCRCONF][3792]Upgrading OCR data
    2009-05-20 15:44:25.609: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.609: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.609: [  OCRRAW][3792]propriogid:1: INVALID FORMAT
    2009-05-20 15:44:25.609: [  OCRRAW][3792]ibctx:1:ERROR: INVALID FORMAT
    2009-05-20 15:44:25.609: [  OCRRAW][3792]proprinit:problem reading the bootblock or superbloc 22
    2009-05-20 15:44:25.609: [ default][3792]a_init:7!: Backend init unsuccessful : [22]
    2009-05-20 15:44:25.609: [ OCRCONF][3792]Exporting OCR data to [OCRUPGRADEFILE]
    2009-05-20 15:44:25.640: [  OCRAPI][3792]a_init:7!: Backend init unsuccessful : [33]
    2009-05-20 15:44:25.640: [ OCRCONF][3792]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]
    2009-05-20 15:44:25.656: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.671: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.671: [  OCRRAW][3792]propriogid:1: INVALID FORMAT
    2009-05-20 15:44:25.671: [  OCRRAW][3792]ibctx:1:ERROR: INVALID FORMAT
    2009-05-20 15:44:25.671: [  OCRRAW][3792]proprinit:problem reading the bootblock or superbloc 22
    2009-05-20 15:44:25.671: [ default][3792]a_init:7!: Backend init unsuccessful : [22]
    2009-05-20 15:44:25.687: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.687: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.687: [  OCRRAW][3792]propriogid:1: INVALID FORMAT
    2009-05-20 15:44:25.687: [  OCRRAW][3792]ibctx:1:ERROR: INVALID FORMAT
    2009-05-20 15:44:25.687: [  OCRRAW][3792]proprinit:problem reading the bootblock or superbloc 22
    2009-05-20 15:44:25.687: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.703: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.703: [  OCRRAW][3792]propriogid:1: INVALID FORMAT
    2009-05-20 15:44:25.875: [  OCROSD][3792]utgdv:11:could not read reg value ocrmirrorconfig_loc os error= The system could not find the environment option that was entered.
    2009-05-20 15:44:25.921: [  OCRRAW][3792]propriowv: Vote information on disk 0 [\\.\ocrcfg] is adjusted from [0/0] to [2/2]
    2009-05-20 15:44:28.921: [  OCRRAW][3792]iniconfig:No 92 configuration
    2009-05-20 15:44:28.921: [  OCRAPI][3792]a_init:6a: Backend init successful
    2009-05-20 15:44:30.593: [ OCRCONF][3792]Initialized DATABASE keys in OCR
    2009-05-20 15:44:31.453: [ OCRCONF][3792]csetskgfrblock0: output from clsmft: [clsfmt: Failure trying to resize OCR file \\.\ocrcfg
    from 10725732864 bytes to 10725736448 bytes
    clsfmt: successfully initialized file \\.\ocrcfg
    g
    2009-05-20 15:44:31.531: [ OCRCONF][3792]Successfully set skgfr block 0
    2009-05-20 15:44:31.531: [ OCRCONF][3792]Exiting [status=success]...
    So, it seems to be all about the OCR not setup correctly, however, all I've done is follow the notes. This was a 'clean' VM install of win2003 Server, so nothing has been accidently tinkered with.

  • Oracle DBconsole service does not start

    Hi,
    Db 10g 2 on Win XP pro. When I want start Oracle DBconsole service , it does not start. I receive a message from windows to see the provider and says that error code is 2. When I look at event logs of windows it is said abnormal exit and event code is 4. Cosequently it is impossible to see DB control in Internet Explorer browser.
    Thanks for your help.

    Thank you Madrid. I resolved the problem. I had this :
    set ORACLE_SID=ORCL2
    C:\Documents and Settings\> emctl stop dbconsole
    OC4J Configuration issue. C:\oracle\product\10.2.0\db_1/oc4j/j2ee/OC4J_DBConsole_LAPTOP-145_ORCL2 not found.
    Actually I had C:\oracle\product\10.2.0\db_1/oc4j/j2ee\LAPTOP-145.b-a-w.com_orcl. Then I should have create OC4J_DBConsole_LAPTOP-145_ORCL2.I created that and no more problem.
    My question is where emctl look for this name OC4J_DBConsole_LAPTOP-145_ORCL2 ? In tnsnames.ora, in sqlnet. ora or some where else ? Thank you.
    can anyone answer please ?
    Message was edited by:
    user522961
    Message was edited by:
    user522961

  • Oracle clusterware upgrade 10.2.0.4

    Will there be any separate patch to upgrade oracle clusterware from 10.2.0.1 to 10.2.0.4....
    why because we installed oracle clusterware 10.2.0.1 and Oracle RAC database 10.2.0.1 and upgraded with patch 10.2.0.4.....which upgraded both clusterware and database to 10.2.0.4
    Now due to some problems clusterware got crashed and we installed new clusterware 10.2.0.1 and need to upgrade clusterware to 10.2.0.4 with out touching the Oracle RAC database which is already upgraded to 10.2.0.4...
    early response is appreciated

    Clusterware sits in its own home ... what specifically is the issue not answered by the docs so we can help with the current issue?
    Installation docs can be found at http://docs.oracle.com and there is additional support at metalink.

  • Feed back needed about the Oracle support service request

    Dear Oracle professional,
    When you have an issues in the Oracle softwares while doing the installations or working funcnalities.
    Is it good to get it touch with Oracle support services to get supports or Self efforts is good.
    Some company's are feeling get supports is not good way of approach from the Oracle supports services.
    Rather they would keep thier business on hold till they get the supports by themself.the daly will causes in to the business losses because of the time.
    And their business production will be less.
    By getting Oracle supports you get the solutions faster this helps to increase the productivities high.
    Some IT Manages will like to play this role always find a solutions by themself and eaten up the time.
    Why this is happending with them?
    Any issues or eagostics with them?
    Feel free to express it to me, i would be happy to hear this please.
    Thanks
    Riyas
    email [email protected]

    Clearly it is a trade of between high productivity vs time & money.
    So now a days companies which i have seen that they will have internal team of SMEs which will try to look into issues and there are certain escalation points from where in you will get some support from Oracle support.
    and again this changes from organization to organization not a thumb rule.
    Regards
    Prashant Pathak

  • Oracle heterogeneous services in 10g r2

    The dba configured oracle heterogeneous services to connect to a SQL Server database.
    The purpose of doing this is to select from a view in SQL Server.
    We are able to select. But as soon as a where clause is introduced, the session just hangs.
    So we thought it might be because we are accessing a view.
    We then created a table in SQL Server and inserted all the data into the table. However, the select with a where clause does not work.
    We then created a table in Oracle and selected from the view in SQL Server to insert data into the oracle table . Once the data is in an Oracle table, we are able to select and use a where clause. We do not want to bring data into Oracle as the data architect is trying to avoid a duplication scenario.
    The DBA is not able to find what is wrong. Can someone please help. Have you seen a similar issue?
    The oracle version is 10.2.0.4 and sql server is 2000.
    Thanks,
    ksd
    Edited by: user8699561 on Mar 16, 2010 8:30 AM

    If the application is using sequences, it cannot require that values are not skipped. Oracle sequences will never be gap-free. And assuming that there are multiple users, the order of transactions and the order of sequence generated values cannot be guaranteed.
    I suppose that if your failover process was manual and scheduled and the downtime window was sufficient (i.e. you're failing over because you're taking one server down for scheduled maintenance, not because there was an unexpected fault), you could set up a script that would shut down the application, iterate over the sequences on the master getting the next value, alter the local sequences (presumably by altering the increment, selecting the next value from the local sequences, and returning the increment to 1), and then allow the application to start on the new server. If your downtime window is sufficient, when the primary has an unscheduled failure, you could potentially do a SELECT MAX(primary key) on each table on the local system (while the application is down), alter the sequences and then allow the application to start running. Neither is a particularly pretty option, and the latter option can be quite expensive if there is a lot of data.
    Justin

  • Problem while starting the oracle clusterware

    Hi,
    In my envirnoment there are two rac nodes. When both systems are shutdown state but when i start the system 1 (node1) the oracle clusterware does not start it give the error while running this command crs_stat -t crs cannot communicate.
    But when I start system 2 (node 2) the oracle clusterware starts.
    Can anyone help me out regarding this issue?
    Thank you for your cooperation.
    Regards,
    Adnan Hamdus Salam.

    Hi,
    No that is not the case. My oracle rac clusterware is perfectly working fine.
    In case 1 when both nodes are shutdown when I start node 2 the oracle clusterware starts and then I start node 1 the whole oracle rac clusterware up successfull.
    in case 2 when both nodes are shutdown when I start node 1 the oracle rac clusterware does not starts and then when I start node 2 the whole oracle rac clusterware up successfully.
    Regards,
    Adnan Hamdus Salam

  • Oracle.bpel.services.datacontrol.data.DataCollection error in BPM work-list

    Hi,
    I am working with Human task form. I Have created one form using create auto-generate option from task options. The form is fetching and displaying data properly in BPM work-list, but when i click on approve/reject button in form, it gives the error oracle.bpel.services.datacontrol.data.DataCollection error . Not sure, from where this error is coming.
    Please reply if anyone is having idea why this error is coming. I am working with Jdev 11.1.1.5 and SOA suite 11g and this project is part of new development. ( no migration issue, i guess)
    thanks,
    rps

    was this issue resolved?
    I am facing the same issue and when I click on Approve/Reject in the form I receive a pop up saying      
    oracle.bpel.services.datacontrol.data.DataCollection cannot be cast to oracle.bpel.services.datacontrol.data.DataObject.

Maybe you are looking for