Failure at final check of Oracle CRS stack. 10 on the first node.

Hi everyone
I trying to install an Oracle RAC 10gr2 on an Oracle Enterprise Linux AS release 4 (October Update 7) , but I'm having this problem
root@fporn01 crs# ./root.sh
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname fporn01 for node 1.
assigning default hostname fporn02 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: fporn01 fporn01-priv fporn01
node 2: fporn02 fporn02-priv fporn02
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
+10+
forget about the node names!!!!
but on the second node everything went fine, so I'm sure this is not a connectivity issue.
the iptables service is stopped and disabled
check the results after running the root.sh script
root@fporn02 ~# /u01/app/crs/root.sh
Checking to see if Oracle CRS stack is already configured
+/etc/oracle does not exist. Creating it now.+
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname fporn01 for node 1.
assigning default hostname fporn02 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: fporn01 fporn01-priv fporn01
node 2: fporn02 fporn02-priv fporn02
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
fporn02
CSS is inactive on these nodes.
fporn01
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
this is the log of crs on the first node
root@fporn01 bin# cat /u01/app/crs/log/fporn01/alertfporn01.log
+2009-06-24 17:27:37.695+
client(9045)CRS-1006:The OCR location /u02/oradata/orcl/OCRFile_mirror is inaccessible. Details in /u01/app/crs/log/fporn01/client/ocrconfig_9045.log.
+2009-06-24 17:27:37.741+
client(9045)CRS-1001:The OCR was formatted using version 2.
+2009-06-24 17:28:24.544+
client(9092)CRS-1801:Cluster pdb-rac configured with nodes fporn01 fporn02 .
this is the log of crs on the second node
root@fporn02 ~# cat /u01/app/crs/log/fporn02/alertfporn02.log
+2009-06-24 18:09:09.307+
cssd(16991)CRS-1605:CSSD voting file is online: /u02/oradata/orcl/CSSFile. Details in /u01/app/crs/log/fporn02/cssd/ocssd.log.
+2009-06-24 18:09:09.307+
cssd(16991)CRS-1605:CSSD voting file is online: /u02/oradata/orcl/CSSFile_mirror1. Details in /u01/app/crs/log/fporn02/cssd/ocssd.log.
+2009-06-24 18:09:09.310+
cssd(16991)CRS-1605:CSSD voting file is online: /u02/oradata/orcl/CSSFile_mirror2. Details in /u01/app/crs/log/fporn02/cssd/ocssd.log.
+2009-06-24 18:09:12.441+
cssd(16991)CRS-1601:CSSD Reconfiguration complete. Active nodes are fporn02 .
I have rechecked the Remote Access / User Equivalence
after run the OCRCHECK command ia have this information
root@fporn01 bin# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 262144
Used space (kbytes) : 312
Available space (kbytes) : 261832
ID : 255880615
Device/File Name : /u02/oradata/orcl/OCRFile
Device/File integrity check succeeded
Device/File Name : /u02/oradata/orcl/OCRFile_mirror
Device/File integrity check succeeded
Cluster registry integrity check succeeded
on the second node i get the same output
root@fporn02 bin# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 262144
Used space (kbytes) : 312
Available space (kbytes) : 261832
ID : 255880615
Device/File Name : /u02/oradata/orcl/OCRFile
Device/File integrity check succeeded
Device/File Name : /u02/oradata/orcl/OCRFile_mirror
Device/File integrity check succeeded
Cluster registry integrity check succeeded
I have reviewed the following metalink notes but none of them seems to solve my problem
*344994.1*
*240001.1*
*725878.1*
*329450.1*
*734221.1*
I have done a research trough many forums, but always the fail is on the second node, but my fail is on the first node.
I hope anyone could help me.
this is the output of cluvfy
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "fporn01"
Destination Node Reachable?
fporn01 yes
fporn02 yes
Result: Node reachability check passed from node "fporn01".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
fporn02 passed
fporn01 passed
Result: User equivalence check passed for user "oracle".
Checking administrative privileges...
Check: Existence of user "oracle"
Node Name User Exists Comment
fporn02 yes passed
fporn01 yes passed
Result: User existence check passed for "oracle".
Check: Existence of group "oinstall"
Node Name Status Group ID
fporn02 exists 501
fporn01 exists 501
Result: Group existence check passed for "oinstall".
Check: Membership of user "oracle" in group "oinstall" as Primary
Node Name User Exists Group Exists User in Group Primary Comment
fporn02 yes yes yes yes passed
fporn01 yes yes yes yes passed
Result: Membership check for user "oracle" in group "oinstall" as Primary passed.
Administrative privileges check passed.
Checking node connectivity...
Interface information for node "fporn02"
Interface Name IP Address Subnet
eth0 10.218.108.245 10.218.108.0
eth1 192.168.1.2 192.168.1.0
Interface information for node "fporn01"
Interface Name IP Address Subnet
eth0 10.218.108.244 10.218.108.0
eth1 192.168.1.1 192.168.1.0
eth2 172.16.9.210 172.16.9.0
Check: Node connectivity of subnet "10.218.108.0"
Source Destination Connected?
fporn02:eth0 fporn01:eth0 yes
Result: Node connectivity check passed for subnet "10.218.108.0" with node(s) fporn02,fporn01.
Check: Node connectivity of subnet "192.168.1.0"
Source Destination Connected?
fporn02:eth1 fporn01:eth1 yes
Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) fporn02,fporn01.
Check: Node connectivity of subnet "172.16.9.0"
Result: Node connectivity check passed for subnet "172.16.9.0" with node(s) fporn01.
Suitable interfaces for the private interconnect on subnet "10.218.108.0":
fporn02 eth0:10.218.108.245
fporn01 eth0:10.218.108.244
Suitable interfaces for the private interconnect on subnet "192.168.1.0":
fporn02 eth1:192.168.1.2
fporn01 eth1:192.168.1.1
ERROR:
Could not find a suitable set of interfaces for VIPs.
Result: Node connectivity check failed.
Checking system requirements for 'crs'...
Check: Total memory
Node Name Available Required Comment
fporn02 7.93GB (8310276KB) 512MB (524288KB) passed
fporn01 7.93GB (8310276KB) 512MB (524288KB) passed
Result: Total memory check passed.
Check: Free disk space in "/tmp" dir
Node Name Available Required Comment
fporn02 9.57GB (10037300KB) 400MB (409600KB) passed
fporn01 9.55GB (10012168KB) 400MB (409600KB) passed
Result: Free disk space check passed.
Check: Swap space
Node Name Available Required Comment
fporn02 8.81GB (9240568KB) 1GB (1048576KB) passed
fporn01 8.81GB (9240568KB) 1GB (1048576KB) passed
Result: Swap space check passed.
Check: System architecture
Node Name Available Required Comment
fporn02 i686 i686 passed
fporn01 i686 i686 passed
Result: System architecture check passed.
Check: Kernel version
Node Name Available Required Comment
fporn02 2.6.9-78.0.0.0.1.ELhugemem 2.4.21-15EL passed
fporn01 2.6.9-78.0.0.0.1.ELhugemem 2.4.21-15EL passed
Result: Kernel version check passed.
Check: Package existence for "make-3.79"
Node Name Status Comment
fporn02 make-3.80-7.EL4 passed
fporn01 make-3.80-7.EL4 passed
Result: Package existence check passed for "make-3.79".
Check: Package existence for "binutils-2.14"
Node Name Status Comment
fporn02 binutils-2.15.92.0.2-25 passed
fporn01 binutils-2.15.92.0.2-25 passed
Result: Package existence check passed for "binutils-2.14".
Check: Package existence for "gcc-3.2"
Node Name Status Comment
fporn02 gcc-3.4.6-10.0.1 passed
fporn01 gcc-3.4.6-10.0.1 passed
Result: Package existence check passed for "gcc-3.2".
Check: Package existence for "glibc-2.3.2-95.27"
Node Name Status Comment
fporn02 glibc-2.3.4-2.41 passed
fporn01 glibc-2.3.4-2.41 passed
Result: Package existence check passed for "glibc-2.3.2-95.27".
Check: Package existence for "compat-db-4.0.14-5"
Node Name Status Comment
fporn02 compat-db-4.1.25-9 passed
fporn01 compat-db-4.1.25-9 passed
Result: Package existence check passed for "compat-db-4.0.14-5".
Check: Package existence for "compat-gcc-7.3-2.96.128"
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
Result: Package existence check failed for "compat-gcc-7.3-2.96.128".
++Check: Package existence for "compat-gcc-c++-7.3-2.96.128"++
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
++Result: Package existence check failed for "compat-gcc-c++-7.3-2.96.128".++
++Check: Package existence for "compat-libstdc++-7.3-2.96.128"++
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
++Result: Package existence check failed for "compat-libstdc++-7.3-2.96.128".++
++Check: Package existence for "compat-libstdc++-devel-7.3-2.96.128"++
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
++Result: Package existence check failed for "compat-libstdc++-devel-7.3-2.96.128".++
Check: Package existence for "openmotif-2.2.3"
Node Name Status Comment
fporn02 openmotif-2.2.3-10.2.el4 passed
fporn01 openmotif-2.2.3-10.2.el4 passed
Result: Package existence check passed for "openmotif-2.2.3".
Check: Package existence for "setarch-1.3-1"
Node Name Status Comment
fporn02 setarch-1.6-1 passed
fporn01 setarch-1.6-1 passed
Result: Package existence check passed for "setarch-1.3-1".
Check: Group existence for "dba"
Node Name Status Comment
fporn02 exists passed
fporn01 exists passed
Result: Group existence check passed for "dba".
Check: Group existence for "oinstall"
Node Name Status Comment
fporn02 exists passed
fporn01 exists passed
Result: Group existence check passed for "oinstall".
Check: User existence for "nobody"
Node Name Status Comment
fporn02 exists passed
fporn01 exists passed
Result: User existence check passed for "nobody".
System requirement failed for 'crs'
Pre-check for cluster services setup was unsuccessful on all the nodes.

forget about my last post, it was my mistake, I rebooted the server and the clustered file system service did not start up at boot time.
sorry
this is what I really got in /var/log/messages
after manually running crs daemons
Jun 26 16:43:07 fporn01 su(pam_unix)[10020]: session opened for user oracle by (uid=0)
Jun 26 16:43:07 fporn01 su(pam_unix)[10020]: session closed for user oracle
Jun 26 16:43:07 fporn01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 26 16:44:07 fporn01 su(pam_unix)[9977]: session opened for user oracle by (uid=0)
Jun 26 16:45:31 fporn01 su(pam_unix)[10293]: session opened for user oracle by (uid=0)
Jun 26 16:45:32 fporn01 su(pam_unix)[10293]: session closed for user oracle
Jun 26 16:45:32 fporn01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 26 16:45:40 fporn01 su(pam_unix)[10351]: session opened for user oracle by (uid=0)
Jun 26 16:45:40 fporn01 su(pam_unix)[10351]: session closed for user oracle
Jun 26 16:45:40 fporn01 su(pam_unix)[10415]: session opened for user oracle by (uid=0)
Jun 26 16:45:40 fporn01 su(pam_unix)[10415]: session closed for user oracle
Jun 26 16:45:40 fporn01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 26 16:46:32 fporn01 su(pam_unix)[10591]: session opened for user oracle by (uid=0)
Jun 26 16:46:40 fporn01 logger: Running CRSD with TZ =
after running ps -ef | grep -E 'init|d.bin|ocls|oprocd|diskmon|evmlogger|PID'
[root@fporn01 ~]# ps -ef | grep -E 'init|d.bin|ocls|oprocd|diskmon|evmlogger|PID'
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 15:33 ? 00:00:00 init [5]
root 9869 7951 0 16:40 pts/1 00:00:00 [init.crsd] <defunct>
oracle 10053 9977 0 16:44 ? 00:00:00 /u01/app/crs/bin/evmd.bin
root 10249 7951 0 16:45 pts/1 00:00:00 /bin/sh /etc/init.d/init.cssd fatal
root 10341 7951 0 16:45 pts/1 00:00:00 /u01/app/crs/bin/crsd.bin reboot
root 10551 10249 0 16:46 pts/1 00:00:00 /bin/sh /etc/init.d/init.cssd daemon
oracle 10618 10592 0 16:46 ? 00:00:00 /u01/app/crs/bin/ocssd.bin
oracle 10926 10053 0 16:46 ? 00:00:00 /u01/app/crs/bin/evmlogger.bin -o /u01/app/crs/evm/log/evmlogger.info -l /u01/app/crs/evm/log/evmlogger.log
root 16658 9461 0 16:50 pts/2 00:00:00 grep -E init|d.bin|ocls|oprocd|diskmon|evmlogger|PID
CRS daemons finally work
*but i get this error when i run [oracle@fporn01 cluvfy]$ ./runcluvfy.sh stage -post crsinst -n fporn01,fporn02 -verbose*
Performing post-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "fporn01"
Destination Node Reachable?
fporn01 yes
fporn02 yes
Result: Node reachability check passed from node "fporn01".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
fporn02 passed
fporn01 passed
Result: User equivalence check passed for user "oracle".
ERROR:
CRS is not installed on any of the nodes.
Verification cannot proceed.
Post-check for cluster services setup was unsuccessful on all the nodes.

Similar Messages

Failure at final check of Oracle CRS stack.10 on the second node

Hi,
I am trying to install Oracle Clusterware 10.2.0.1.0 in VM machines (2 nodes config) in Linux (OEL5) using VMware Server (2.0). Everything went very well one the first node upto running the root.sh. Running root.sh ended with Failure at final check of Oracle CRS stack 10 error.
RAC1 root.sh output
[root@rac1 crs]# ./root.sh
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac1
CSS is inactive on these nodes.
rac2
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
[root@rac1 crs]#
RAC2 root.sh output
[root@rac2 crs]# ./root.sh
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
10
[root@rac2 crs]#
Output of alterrac2.log
[root@rac2 rac2]# more alertrac2.log
2009-08-14 23:02:44.699
[client(5935)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible. Details in /u01/crs/oracle/product/10.2.
0/crs/log/rac2/client/ocrconfig_5935.log.
2009-08-14 23:02:44.704
[client(5935)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible. Details in /u01/crs/oracle/product/10.2.
0/crs/log/rac2/client/ocrconfig_5935.log.
2009-08-14 23:02:44.707
[client(5935)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible. Details in /u01/crs/oracle/product/10.2.
0/crs/log/rac2/client/ocrconfig_5935.log.
2009-08-14 23:02:44.864
[client(5935)]CRS-1001:The OCR was formatted using version 2.
2009-08-14 23:02:50.339
[client(6004)]CRS-1801:Cluster crs configured with nodes rac1 rac2 .
2009-08-14 23:05:07.603
[cssd(6600)]CRS-1605:CSSD voting file is online: /dev/raw/raw2. Details in /u01/crs/oracle/product/10.2.0/crs/l
og/rac2/cssd/ocssd.log.
[root@rac2 rac2]#
Since raw devices are not supported from OEL5, I did do the workaround in *63-oracle-raw.rules file under /etc/udev/rules.d* dir.
ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", KERNEL=="sdc1", RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", KERNEL=="sdd1", RUN+="/bin/raw /dev/raw/raw3 %N"
ACTION=="add", KERNEL=="sde1", RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add", KERNEL=="sdf1", RUN+="/bin/raw /dev/raw/raw5 %N"
KERNEL=="raw[1-2]*", OWNER="root", GROUP="oinstall", MODE="640"
KERNEL=="raw[3-5]*", OWNER="oracle", GROUP="oinstall", MODE="640"
One thing I have noticed after running root.sh on both the nodes is the permissons on raw devices changed from
Before root.sh
[root@rac2 crs]# ls -ls /dev/raw*
0 crw------- 1 root root 162, 0 Aug 14 22:42 /dev/rawctl
/dev/raw:
total 0
0 crw-r----- 1 root oinstall 162, 1 Aug 14 22:42 raw1
0 crw-r----- 1 root oinstall 162, 2 Aug 14 22:42 raw2
0 crw-r----- 1 oracle oinstall 162, 3 Aug 14 22:42 raw3
0 crw-r----- 1 oracle oinstall 162, 4 Aug 14 22:42 raw4
0 crw-r----- 1 oracle oinstall 162, 5 Aug 14 22:42 raw5
to
[root@rac2 crs]# ls -ls /dev/raw*
0 crw------- 1 root root 162, 0 Aug 14 22:31 /dev/rawctl
/dev/raw:
total 0
0 crw-r----- 1 root oinstall 162, 1 Aug 14 22:56 raw1
0 crw-r--r-- 1 oracle oinstall 162, 2 Aug 14 23:01 raw2
0 crw-r----- 1 oracle oinstall 162, 3 Aug 14 22:31 raw3
0 crw-r----- 1 oracle oinstall 162, 4 Aug 14 22:31 raw4
0 crw-r----- 1 oracle oinstall 162, 5 Aug 14 22:31 raw5
[root@rac1 crs]#
My shared disk listing
[root@www shared]# ls -ltr
total 8780
-rw------- 1 root root 640 Aug 14 22:43 votingdisk.vmdk
-rw------- 1 root root 598 Aug 14 22:43 ocr.vmdk
-rw------- 1 root root 604 Aug 14 22:43 asm3.vmdk
-rw------- 1 root root 604 Aug 14 22:43 asm2.vmdk
-rw------- 1 root root 604 Aug 14 22:43 asm1.vmdk
-rw------- 1 root root 65536 Aug 14 22:44 votingdisk-s006.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s005.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s004.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s003.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s002.vmdk
-rw------- 1 root root 393216 Aug 14 22:44 votingdisk-s001.vmdk
-rw------- 1 root root 65536 Aug 14 22:44 ocr-s006.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 ocr-s005.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 ocr-s004.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 ocr-s003.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 ocr-s002.vmdk
-rw------- 1 root root 393216 Aug 14 22:44 ocr-s001.vmdk
-rw------- 1 root root 65536 Aug 14 22:44 asm3-s006.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm3-s005.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm3-s004.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm3-s003.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm3-s002.vmdk
-rw------- 1 root root 393216 Aug 14 22:44 asm3-s001.vmdk
-rw------- 1 root root 65536 Aug 14 22:44 asm2-s006.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm2-s005.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm2-s004.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm2-s003.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm2-s002.vmdk
-rw------- 1 root root 393216 Aug 14 22:44 asm2-s001.vmdk
-rw------- 1 root root 65536 Aug 14 22:44 asm1-s006.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm1-s005.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm1-s004.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm1-s003.vmdk
-rw------- 1 root root 327680 Aug 14 22:44 asm1-s002.vmdk
-rw------- 1 root root 393216 Aug 14 22:44 asm1-s001.vmdk
[root@www shared]#
I don't know how to fix this problem. I did go through many docs and metalink notes.
I am new to RAC world. It took 3 days to come to this stage. Please help me.
Thanks
Leo

Hi Surachart,
Here is my messages output..
*/var/log/messages*
Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for fe80::20c:29ff:fe6b:f9a8 on eth1.
Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for 192.168.1.196 on eth1.
Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for fe80::20c:29ff:fe6b:f99e on eth0.
Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for 192.168.0.196 on eth0.
Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering HINFO record with values 'I686'/'LINUX'.
Aug 20 14:05:02 rac2 avahi-daemon[3627]: Server startup complete. Host name is rac2.local. Local service cookie is 927471131.
Aug 20 14:05:03 rac2 avahi-daemon[3627]: Service "SFTP File Transfer on rac2" (/services/sftp-ssh.service) successfully established.
Aug 20 14:05:08 rac2 smartd[3739]: smartd version 5.38 [i686-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Aug 20 14:05:08 rac2 smartd[3739]: Home page is http://smartmontools.sourceforge.net/
Aug 20 14:05:08 rac2 smartd[3739]: Opened configuration file /etc/smartd.conf
Aug 20 14:05:08 rac2 smartd[3739]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/hdc, opened
Aug 20 14:05:08 rac2 kernel: hdc: drive_cmd: status=0x51 { DriveReady SeekComplete Error }
Aug 20 14:05:08 rac2 kernel: hdc: drive_cmd: error=0x04 { AbortedCommand }
Aug 20 14:05:08 rac2 kernel: ide: failed opcode was: 0xec
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/hdc, not ATA, no IDENTIFY DEVICE Structure
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sda, opened
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sda, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sda' to turn on SMART features
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdb, opened
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdb, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdb' to turn on SMART features
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdc, opened
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdc, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdc' to turn on SMART features
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdd, opened
Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdd, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdd' to turn on SMART features
Aug 20 14:05:09 rac2 smartd[3739]: Device: /dev/sde, opened
Aug 20 14:05:09 rac2 smartd[3739]: Device: /dev/sde, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sde' to turn on SMART features
Aug 20 14:05:10 rac2 smartd[3739]: Device: /dev/sdf, opened
Aug 20 14:05:10 rac2 smartd[3739]: Device: /dev/sdf, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdf' to turn on SMART features
Aug 20 14:05:10 rac2 smartd[3739]: Monitoring 0 ATA and 0 SCSI devices
Aug 20 14:05:10 rac2 smartd[3741]: smartd has fork()ed into background mode. New PID=3741.
Aug 20 14:05:13 rac2 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
Aug 20 14:05:13 rac2 last message repeated 3 times
Aug 20 14:05:27 rac2 gconfd (root-3967): starting (version 2.14.0), pid 3967 user 'root'
Aug 20 14:05:27 rac2 gconfd (root-3967): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
Aug 20 14:05:27 rac2 gconfd (root-3967): Resolved address "xml:readwrite:/root/.gconf" to a writable configuration source at position 1
Aug 20 14:05:27 rac2 gconfd (root-3967): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
Aug 20 14:05:29 rac2 gconfd (root-3967): Resolved address "xml:readwrite:/root/.gconf" to a writable configuration source at position 0
Aug 20 14:05:29 rac2 hald: mounted /dev/hdc on behalf of uid 0
Aug 20 14:05:29 rac2 hcid[3311]: Default passkey agent (:1.8, /org/bluez/applet) registered
Aug 20 14:05:31 rac2 nm-system-settings: Loaded plugin ifcfg-rh: (c) 2007 - 2008 Red Hat, Inc. To report bugs please use the NetworkManager mailing list.
Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-eth1 ...
Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: read connection 'System eth1'
Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-lo ...
Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: error: Ignoring loopback device config.
Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-eth0 ...
Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: read connection 'System eth0'
Aug 20 14:05:31 rac2 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
Aug 20 14:05:32 rac2 last message repeated 4 times
Aug 20 14:12:51 rac2 kernel: FS-Cache: Loaded
Aug 20 14:22:06 rac2 xinetd[3488]: START: shell pid=5193 from=192.168.0.195
Aug 20 14:22:06 rac2 xinetd[3488]: EXIT: shell status=0 pid=5193 duration=0(sec)
Aug 20 14:22:07 rac2 xinetd[3488]: START: shell pid=5217 from=192.168.0.195
Aug 20 14:22:07 rac2 xinetd[3488]: EXIT: shell status=0 pid=5217 duration=0(sec)
Aug 20 14:22:07 rac2 xinetd[3488]: START: shell pid=5241 from=192.168.0.195
Aug 20 14:22:07 rac2 xinetd[3488]: EXIT: shell status=0 pid=5241 duration=0(sec)
Aug 20 14:22:16 rac2 xinetd[3488]: EXIT: shell status=0 pid=6236 duration=0(sec)
Aug 20 14:22:16 rac2 xinetd[3488]: START: shell pid=6265 from=192.168.0.195
Aug 20 14:22:16 rac2 xinetd[3488]: EXIT: shell status=0 pid=6265 duration=0(sec)
Aug 20 14:22:16 rac2 xinetd[3488]: START: shell pid=6291 from=192.168.0.195
Aug 20 14:22:17 rac2 xinetd[3488]: EXIT: shell status=0 pid=6291 duration=1(sec)
Aug 20 14:22:17 rac2 xinetd[3488]: START: shell pid=6317 from=192.168.0.195
Aug 20 14:22:17 rac2 xinetd[3488]: EXIT: shell status=0 pid=6317 duration=0(sec)
[root@rac2 log]#

Failure at final check of oracle crs stack

Hi,
I am installing oracle clusterware 11, when I run on the first node root.sh it's ok, but when I run it on the second node recive this message:
Failure at final check of Oracle CRS stack 10
I have stopped firewall and ssh,scp works fine without password using node.domain and node without domain between the nodes.
please help me!

thanks for the answer.
this is root.sh output of second node
[root@orac-asbe-c cluster]# ./root.sh
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: orac-asbe-d orac-asbe-d-priv orac-asbe-d
node 2: orac-asbe-c orac-asbe-c-priv orac-asbe-c
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/hdd1
Now formatting voting device: /dev/hdd2
Now formatting voting device: /dev/hdd3
Format of 3 voting devices complete.
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
10
the ocssd.log in very long, I take fiew pieces
Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
[ clsdmt]Listening to (ADDRESS=(PROTOCOL=ipc)(KEY=orac-asbe-cDBG_CSSD))
[ CSSD]2011-06-17 11:04:17.918 >USER: Oracle Database 10g CSS Release 11.1.0.6.0 Production Copyright 1996, 2004 Oracle. All rights reserved.
[ CSSD]2011-06-17 11:04:17.918 >USER: CSS daemon log for node orac-asbe-c, number 2, in cluster orac-as_cluster
[ CSSD]2011-06-17 11:04:17.936 [649053344] >TRACE: clssscmain: local-only set to false
[ CSSD]2011-06-17 11:04:17.970 [649053344] >TRACE: clssnmReadNodeInfo: added node 1 (orac-asbe-d) to cluster
[ CSSD]2011-06-17 11:04:17.994 [649053344] >TRACE: clssnmReadNodeInfo: added node 2 (orac-asbe-c) to cluster
[ CSSD]2011-06-17 11:04:17.997 [649053344] >WARNING: clssnmReadWallet: Open Wallet returned 28759
[ CSSD]2011-06-17 11:04:17.997 [649053344] >WARNING: clssnmInitNMInfo: Node not configured for node kill
[ CSSD]2011-06-17 11:04:18.011 [1133824320] >TRACE: clssnm_skgxninit: Compatible vendor clusterware not in use
[ CSSD]2011-06-17 11:04:18.011 [1133824320] >TRACE: clssnm_skgxnmon: skgxn init failed
[ CSSD]2011-06-17 11:04:18.023 [649053344] >TRACE: clssnmNMInitialize: Network heartbeat thresholds are: impending reconfig 15000 ms, reconfig start (misscount) 30000 ms
[ CSSD]2011-06-17 11:04:18.027 [649053344] >TRACE: clssnmNMInitialize: Voting file I/O timeouts are: short 27000 ms, long 200000 ms
[ CSSD]2011-06-17 11:04:18.039 [649053344] >TRACE: clssnmDiskStateChange: state from 1 to 2 disk (0//dev/hdd1)
[ CSSD]2011-06-17 11:04:18.040 [1133824320] >TRACE: clssnmvDPT: spawned for disk 0 (/dev/hdd1)
[ CSSD]2011-06-17 11:04:18.083 [1133824320] >TRACE: clssnmDiskStateChange: state from 2 to 4 disk (0//dev/hdd1)
[ CSSD]2011-06-17 11:04:18.088 [1144314176] >TRACE: clssnmvKillBlockThread: spawned for disk 0 (/dev/hdd1) initial : sleep interval (1000)ms
[ CSSD]2011-06-17 11:04:18.185 [649053344] >TRACE: clssnmDiskStateChange: state from 1 to 2 disk (1//dev/hdd2)
[ CSSD]2011-06-17 11:04:18.189 [1154804032] >TRACE: clssnmvDPT: spawned for disk 1 (/dev/hdd2)
[ CSSD]2011-06-17 11:04:18.208 [1154804032] >TRACE: clssnmDiskStateChange: state from 2 to 4 disk (1//dev/hdd2)
[ CSSD]2011-06-17 11:04:18.212 [1102223680] >TRACE: clssnmvKillBlockThread: spawned for disk 1 (/dev/hdd2) initial sleep interval (1000)ms
[ CSSD]2011-06-17 11:04:18.219 [649053344] >TRACE: clssnmDiskStateChange: state from 1 to 2 disk (2//dev/hdd3)
[ CSSD]2011-06-17 11:04:18.223 [1165293888] >TRACE: clssnmvDPT: spawned for disk 2 (/dev/hdd3)
[ CSSD]2011-06-17 11:04:18.251 [1165293888] >TRACE: clssnmDiskStateChange: state from 2 to 4 disk (2//dev/hdd3)
[ CSSD]2011-06-17 11:04:18.255 [1175783744] >TRACE: clssnmvKillBlockThread: spawned for disk 2 (/dev/hdd3) initial sleep interval (1000)ms
CSSD]2011-06-17 11:04:18.367 [649053344] >TRACE: clssscSclsFatal: read value of disable
[ CSSD]2011-06-17 11:04:18.370 [649053344] >TRACE: clssscSclsFatal: read value of disable
[ CSSD]2011-06-17 11:04:18.373 [1196763456] >TRACE: clssnmFatalThread: spawned
[ CSSD]2011-06-17 11:04:18.375 [1207253312] >TRACE: clssnmClusterListener: Listening on (ADDRESS=(PROTOCOL=tcp)(HOST=orac-asbe-c-priv)(PORT=49895))
[ CSSD]2011-06-17 11:04:18.375 [1207253312] >TRACE: clssnmconnect: connecting to node(1), con(0xc3af000), flags 0x0003
[ CSSD]2011-06-17 11:04:18.381 [1217743168] >TRACE: clssgmDeathChkThread: Spawned
[ CSSD]------- Begin Dump -------
[ CSSD]
[ CSSD]
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >TRACE: clssnmConnComplete: MSGSRC 1, type 6, node 1, flags 0x0003, con 0xc3af000, probe (nil), nodekillsz 0
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >TRACE: clssnmConnComplete: msg src=1 dst=2 seq=0 type=6 birth=203767168 state=3 name=()
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >ERROR: ASSERT clssnm.c 11562
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >ERROR: clssnmConnComplete: OCR id mismatch (1700660325, 1307814030)
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >ERROR: ###################################
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >ERROR: clssscExit: CSSD aborting from thread clssnmClusterListener
[ CSSD]2011-06-17 11:04:18.383 [1207253312] >ERROR: ###################################
[ CSSD]
----- Call Stack Trace -----
[ CSSD]calling call entry argument values in hex
[ CSSD]location type point (? means dubious value)
[ CSSD]-------------------- -------- -------------------- ----------------------------
[ CSSD]sskgds_getexecname: using /proc/self/status and $PATH to get ocssd.bin
[ CSSD]Cannot open ocssd.bin for reading: errno=2
[ CSSD]Cannot open ocssd.bin for reading: errno=2
[ CSSD]Cannot open ocssd.bin for reading: errno=2
[ CSSD]000000000040C2D6 call kgdsdst() 000000000 ? 000000001 ?
[ CSSD] 047F4FB68 ? 047F4E650 ?
[ CSSD] 000000000 ? 000000003 ?
thanks again

Clusterware Install:root.sh- Failure at final check of Oracle CRS stack. 10

Hello All,
Image: !http://systemwars.com/rac/cluster_back.jpg!
I was attempting to perform the steps in:
Link: http://www.oracle-base.com/articles/11g/OracleDB11gR1RACInstallationOnLinuxUsingNFS.php
The only difference is that I decided to use fedora core 12 instead. I did this because I added a second NIC card (USB) and only FC12 would recognize it. I tried to get it to work on Cent 5 but it just wouldn't. The second nic on each machine eth1 are connected via crossover cable, and the interfaces can ping each other just fine, rac1-priv and rac2-priv.
So here is my setup:
# Public
192.168.2.11 rac1.localdomain rac1
192.168.2.12 rac2.localdomain rac2
#Private
192.168.0.11 rac1-priv.localdomain rac1-priv
192.168.0.12 rac2-priv.localdomain rac2-priv
#Virtual
192.168.2.111 rac1-vip.localdomain rac1-vip
192.168.2.112 rac2-vip.localdomain rac2-vip
#NAS
192.168.2.10 mini.localdomain mini
Mini refers to my Mac mini which I decided to use as the 3rd "server" in the group. I was able to mount/read & write to the file systems just fine. As you can see.
[root@rac1 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_rac1-lv_root
8063408 5156268 2497540 68% /
tmpfs 1417456 0 1417456 0% /dev/shm
/dev/sda1 198337 22080 166017 12% /boot
mini:/shared_config 488050688 76719808 411074880 16% /u01/shared_config
mini:/shared_crs 488050688 76719808 411074880 16% /u01/app/crs/product/11.1.0/crs
mini:/shared_home 488050688 76719808 411074880 16% /u01/app/oracle/product/11.1.0/db_1
mini:/shared_data 488050688 76719808 411074880 16% /u01/oradata
[root@rac1 ~]# ssh rac2
Last login: Mon Dec 21 19:33:38 2009 from rac1.localdomain
[root@rac2 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_rac2-lv_root
8063408 4958008 2695800 65% /
tmpfs 1417456 0 1417456 0% /dev/shm
/dev/sda1 198337 22063 166034 12% /boot
mini:/shared_config 488050688 76719808 411074880 16% /u01/shared_config
mini:/shared_crs 488050688 76719808 411074880 16% /u01/app/crs/product/11.1.0/crs
mini:/shared_home 488050688 76719808 411074880 16% /u01/app/oracle/product/11.1.0/db_1
mini:/shared_data 488050688 76719808 411074880 16% /u01/oradata[color]
CLUSTER VERIFY SEEMS OK APART FROM ONE WARNING
WARNING:
Could not find a suitable set of interfaces for VIPs.
which according to this link, "can be safety ignored", although I noticed in the link its an actual ERROR and not a WARNING => http://www.idevelopment.info/data/Oracle/DBA_tips/Oracle10gRAC/CLUSTER_11.shtml . I also noted that it saw the public IPs as the possible priv IPs, which I also thought could safety be ignored.
oracle@rac1 clusterware]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "rac1"
Destination Node Reachable?
rac2 yes
rac1 yes
Result: Node reachability check passed from node "rac1".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
rac2 passed
rac1 passed
Result: User equivalence check passed for user "oracle".
Checking administrative privileges...
Check: Existence of user "oracle"
Node Name User Exists Comment
rac2 yes passed
rac1 yes passed
Result: User existence check passed for "oracle".
Check: Existence of group "oinstall"
Node Name Status Group ID
rac2 exists 501
rac1 exists 501
Result: Group existence check passed for "oinstall".
Check: Membership of user "oracle" in group "oinstall" [as Primary]
Node Name User Exists Group Exists User in Group Primary Comment
rac2 yes yes yes yes passed
rac1 yes yes yes yes passed
Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.
Administrative privileges check passed.
Checking node connectivity...
Interface information for node "rac2"
Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address
eth0 192.168.2.12 192.168.2.0 0.0.0.0 192.168.2.1 00:01:6C:XXXX
eth2 192.168.0.12 192.168.0.0 0.0.0.0 192.168.2.1 00:25:4B:XXXX
Interface information for node "rac1"
Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address
eth0 192.168.2.11 192.168.2.0 0.0.0.0 192.168.2.1 00:01:6CXXXXX
eth1 192.168.0.11 192.168.0.0 0.0.0.0 192.168.2.1 00:25:4B:XXXX
Check: Node connectivity of subnet "192.168.2.0"
Source Destination Connected?
rac2:eth0 rac1:eth0 yes
Result: Node connectivity check passed for subnet "192.168.2.0" with node(s) rac2,rac1.
Check: Node connectivity of subnet "192.168.0.0"
Source Destination Connected?
rac2:eth2 rac1:eth1 yes
Result: Node connectivity check passed for subnet "192.168.0.0" with node(s) rac2,rac1.
Interfaces found on subnet "192.168.2.0" that are likely candidates for a private interconnect:
rac2 eth0:192.168.2.12
rac1 eth0:192.168.2.11
WARNING:
Could not find a suitable set of interfaces for VIPs.
Result: Node connectivity check passed.
Checking system requirements for 'crs'...
Check: Total memory
Node Name Available Required Comment
rac2 2.7GB (2834912KB) 1GB (1048576KB) passed
rac1 2.7GB (2834912KB) 1GB (1048576KB) passed
Result: Total memory check passed.
Check: Free disk space in "/tmp" dir
Node Name Available Required Comment
rac2 4.58GB (4805204KB) 400MB (409600KB) passed
rac1 10.51GB (11015624KB) 400MB (409600KB) passed
Result: Free disk space check passed.
Check: Swap space
Node Name Available Required Comment
rac2 2GB (2097144KB) 1.5GB (1572864KB) passed
rac1 3GB (3145720KB) 1.5GB (1572864KB) passed
Result: Swap space check passed.
Check: System architecture
Node Name Available Required Comment
rac2 i686 i686 passed
rac1 i686 i686 passed
Result: System architecture check passed.
Check: Kernel version
Node Name Available Required Comment
rac2 2.6.31.5-127.fc12.i686.PAE 2.6.9 passed
rac1 2.6.31.5-127.fc12.i686.PAE 2.6.9 passed
Result: Kernel version check passed.
Check: Package existence for "make-3.81"
Node Name Status Comment
rac2 make-3.81-18.fc12.i686 passed
rac1 make-3.81-18.fc12.i686 passed
Result: Package existence check passed for "make-3.81".
Check: Package existence for "binutils-2.17.50.0.6"
Node Name Status Comment
rac2 binutils-2.19.51.0.14-34.fc12.i686 passed
rac1 binutils-2.19.51.0.14-34.fc12.i686 passed
Result: Package existence check passed for "binutils-2.17.50.0.6".
Check: Package existence for "gcc-4.1.1"
Node Name Status Comment
rac2 gcc-4.4.2-7.fc12.i686 passed
rac1 gcc-4.4.2-7.fc12.i686 passed
Result: Package existence check passed for "gcc-4.1.1".
Check: Package existence for "libaio-0.3.106"
Node Name Status Comment
rac2 libaio-0.3.107-9.fc12.i686 passed
rac1 libaio-0.3.107-9.fc12.i686 passed
Result: Package existence check passed for "libaio-0.3.106".
Check: Package existence for "libaio-devel-0.3.106"
Node Name Status Comment
rac2 libaio-devel-0.3.107-9.fc12.i686 passed
rac1 libaio-devel-0.3.107-9.fc12.i686 passed
Result: Package existence check passed for "libaio-devel-0.3.106".
Check: Package existence for "libstdc++-4.1.1"
Node Name Status Comment
rac2 libstdc++-4.4.2-7.fc12.i686 passed
rac1 libstdc++-4.4.2-7.fc12.i686 passed
Result: Package existence check passed for "libstdc++-4.1.1".
Check: Package existence for "elfutils-libelf-devel-0.125"
Node Name Status Comment
rac2 elfutils-libelf-devel-0.143-1.fc12.i686 passed
rac1 elfutils-libelf-devel-0.143-1.fc12.i686 passed
Result: Package existence check passed for "elfutils-libelf-devel-0.125".
Check: Package existence for "sysstat-7.0.0"
Node Name Status Comment
rac2 sysstat-9.0.4-4.fc12.i686 passed
rac1 sysstat-9.0.4-4.fc12.i686 passed
Result: Package existence check passed for "sysstat-7.0.0".
Check: Package existence for "compat-libstdc++-33-3.2.3"
Node Name Status Comment
rac2 compat-libstdc++-33-3.2.3-68.i686 passed
rac1 compat-libstdc++-33-3.2.3-68.i686 passed
Result: Package existence check passed for "compat-libstdc++-33-3.2.3".
Check: Package existence for "libgcc-4.1.1"
Node Name Status Comment
rac2 libgcc-4.4.2-7.fc12.i686 passed
rac1 libgcc-4.4.2-7.fc12.i686 passed
Result: Package existence check passed for "libgcc-4.1.1".
Check: Package existence for "libstdc++-devel-4.1.1"
Node Name Status Comment
rac2 libstdc++-devel-4.4.2-7.fc12.i686 passed
rac1 libstdc++-devel-4.4.2-7.fc12.i686 passed
Result: Package existence check passed for "libstdc++-devel-4.1.1".
Check: Package existence for "unixODBC-2.2.11"
Node Name Status Comment
rac2 unixODBC-2.2.14-6.fc12.i686 passed
rac1 unixODBC-2.2.14-9.fc12.i686 passed
Result: Package existence check passed for "unixODBC-2.2.11".
Check: Package existence for "unixODBC-devel-2.2.11"
Node Name Status Comment
rac2 unixODBC-devel-2.2.14-6.fc12.i686 passed
rac1 unixODBC-devel-2.2.14-9.fc12.i686 passed
Result: Package existence check passed for "unixODBC-devel-2.2.11".
Check: Package existence for "glibc-2.5-12"
Node Name Status Comment
rac2 glibc-2.11-2.i686 passed
rac1 glibc-2.11-2.i686 passed
Result: Package existence check passed for "glibc-2.5-12".
Check: Group existence for "dba"
Node Name Status Comment
rac2 exists passed
rac1 exists passed
Result: Group existence check passed for "dba".
Check: Group existence for "oinstall"
Node Name Status Comment
rac2 exists passed
rac1 exists passed
Result: Group existence check passed for "oinstall".
Check: User existence for "nobody"
Node Name Status Comment
rac2 exists passed
rac1 exists passed
Result: User existence check passed for "nobody".
System requirement passed for 'crs'
Pre-check for cluster services setup was successful. So now here is the actual problem:
After the installation and during the run of the root.sh I get:
Failure at final check of Oracle CRS stack.
10
[root@rac1 crs]# ./root.sh
WARNING: directory '/u01/app/crs/product/11.1.0' is not owned by root
WARNING: directory '/u01/app/crs/product' is not owned by root
WARNING: directory '/u01/app/crs' is not owned by root
WARNING: directory '/u01/app' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
The directory '/u01/app/crs/product/11.1.0' is not owned by root. Changing owner to root
The directory '/u01/app/crs/product' is not owned by root. Changing owner to root
The directory '/u01/app/crs' is not owned by root. Changing owner to root
The directory '/u01/app' is not owned by root. Changing owner to root
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /u01/shared_config/voting_disk
Format of 1 voting devices complete.
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
10According to this link => http://blog.contractoracle.com/2009/01/failure-at-final-check-of-oracle-crs.html
To recover from a status 10, one must check:
check firewall / routing / iptables issues
Now I have turned iptables off completely it doesnt even start up at boot time, so I know it can't be that.
ROUTE
[oracle@rac1 clusterware]$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.0 * 255.255.255.0 U 1 0 0 eth0
192.168.0.0 * 255.255.255.0 U 1 0 0 eth1
default 192.168.2.1 0.0.0.0 UG 0 0 0 eth0
[oracle@rac2 ~]$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.0 * 255.255.255.0 U 1 0 0 eth0
192.168.0.0 * 255.255.255.0 U 1 0 0 eth2
default 192.168.2.1 0.0.0.0 UG 0 0 0 eth0
[oracle@rac1 clusterware]$ traceroute rac2
traceroute to rac2 (192.168.2.12), 30 hops max, 60 byte packets
1 rac2.localdomain (192.168.2.12) 0.424 ms 0.427 ms 0.096 ms
[oracle@rac1 clusterware]$ traceroute rac2-priv
traceroute to rac2-priv (192.168.0.12), 30 hops max, 60 byte packets
1 rac2-priv.localdomain (192.168.0.12) 1.336 ms 1.238 ms 1.188 ms
[oracle@rac1 clusterware]$ traceroute rac2-vip
traceroute to rac2-vip (192.168.2.112), 30 hops max, 60 byte packets
1 rac1.localdomain (192.168.2.11) 2999.599 ms !H 2999.560 ms !H 2999.523 ms !H
[oracle@rac1 bin]$ ./crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.
Both rac1 and rac2 get the same output above with the -vip getting !H => !H, !N, or !P (host, network or protocol unreachable), I am assuming this is normal as CRS install did not complete successfully and the virtual IP is not bound yet.
Im pretty sure I have some kind of networking issue here, but I cant put my finger on it. I have tried absolutely everything that is suggested on the internet that I could find. Even deleting the /tmp/.oracle and /var/tmp/.oracle but nothing works. Ssh keys for root and oracle users exist and Ive connected using every possible combination to avoid that first time ssh prompt so users oracle on each node goes directly into rac1/rac2 rac1-priv/rac2-priv & actual IPs as well. Any ideas?
Edited by: Javier on Dec 30, 2009 12:34 PM
Edited by: Javier on Dec 30, 2009 6:58 PM

Hello
Note 370605.1 (Clusterware Intermittently Hangs And Commands Fail With CRS-184) is telling this.
"This is caused by a cron job that cleans up the /tmp directory which also removes the Oracle socket files in /tmp/.oracle
Do not remove /tmp/.oracle or /var/tmp/.oracle or its files while Oracle Clusterware is up."
Best Regards...

CRS installation: Failure at final check of Oracle CRS stack.10

Hello,
I am trying to install Oracle RAC in 10GR2 to simulate a migration from 1024 to 11GR2. I am using VMWARE with two Linux CentOS 64b 6.2 and shared disks as raw devices. I got "Failure at final check of Oracle CRS stack.10" when running root.sh, on both nodes. The ocrcheck is fine, but I have two different IDs... which is not good and I do not understand why:
- I have shared raw devices
- the devices are the same, I checked this twice
Can anyoane help?
I thank you all in advance.

Hello,
The exact error message is the one in my subject description: Failure at final check of Oracle CRS stack.10
This occurs when runing root.sh after the installation is successfull and there is no way to continue, the assistants are failing, which is logic, as the root.sh attempt to configure and run the CRS agents are not done successfully. The OCR ar ok with the same ID on both nodes, the minor in major values for the raw devices are and the firewall is disabled on both nodes. I checked the rights, they are also ok. The exact and complete return of the root.sh script is the following:
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: cygnus cygnus-priv cygnus
node 2: taurus taurus-priv taurus
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /apps/oracle/oradat/vot10
Now formatting voting device: /apps/oracle/oradat/vot20
Now formatting voting device: /apps/oracle/oradat/vot30
Format of 3 voting devices complete.
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
10
Any help will be really apreciated and I thank you in advancen guys...

Failure at final check of Oracle CRS stack.AIX oracle 10g RAC with GPFS

HI ，I install oracle 10R2 RAC using GPFS，os is AIX 6.1,but when I installed CRS，at executing the second root.sh ,I am in trouble ，the error information as follow :
Failure at final check of Oracle CRS stack.
10
I look up the log file the information :
The OCR location /data_gpfs/CRS/ocr_disk2 is inaccessible. Details in /orac
leapp/product/10.2.0/crs/log/p615a/client/ocrconfig_6881286.log.
the ocrconfig_6881286.log.information:
2014-01-24 01:32:20.361: [ OCRCONF][1]ocrconfig starts...
2014-01-24 01:32:20.389: [ OCRCONF][1]Upgrading OCR data
2014-01-24 01:32:20.391: [ OCROSD][1]utread:3: problem reading buffer 100f21d0 buflen 512 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.391: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.391: [ OCROSD][1]utread:3: problem reading buffer 100f21d0 buflen 512 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.391: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.391: [ OCROSD][1]utread:3: problem reading buffer ffffb1d0 buflen 4096 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.392: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.392: [ OCRRAW][1]propriogid:1: INVALID FORMAT
2014-01-24 01:32:20.392: [ OCROSD][1]utread:3: problem reading buffer ffffb1d0 buflen 4096 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.392: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.392: [ OCRRAW][1]propriogid:1: INVALID FORMAT
2014-01-24 01:32:20.392: [ OCRRAW][1]proprioini: both disks are not OCR formatted
2014-01-24 01:32:20.392: [ OCRRAW][1]proprinit: Could not open raw device
2014-01-24 01:32:20.392: [ default][1]a_init:7!: Backend init unsuccessful : [26]
2014-01-24 01:32:20.393: [ OCRCONF][1]Exporting OCR data to [OCRUPGRADEFILE]
2014-01-24 01:32:20.393: [ OCRAPI][1]a_init:7!: Backend init unsuccessful : [33]
propriogid:1: INVALID FORMAT
2014-01-24 01:32:20.516: [ OCRRAW][1]propriowv: Vote information on disk 0 [/data_gpfs/CRS/ocr_disk1] is adjusted from [0/0] to [1/2]
2014-01-24 01:32:20.527: [ OCRRAW][1]propriowv: Vote information on disk 1 [/data_gpfs/CRS/ocr_disk2] is adjusted from [0/0] to [1/2]
2014-01-24 01:32:20.960: [ OCRRAW][1]propriniconfig:No 92 configuration
2014-01-24 01:32:20.960: [ OCRAPI][1]a_init:6a: Backend init successful
2014-01-24 01:32:21.191: [ OCRCONF][1]Initialized DATABASE keys in OCR
2014-01-24 01:32:21.349: [ OCRCONF][1]Successfully set skgfr block 0
2014-01-24 01:32:21.351: [ OCRCONF][1]Exiting [status=success]...
I dont know what cause this error,i am really trouble,who can help me !!!

HI ，I install oracle 10R2 RAC using GPFS，os is AIX 6.1,but when I installed CRS，at executing the second root.sh ,I am in trouble ，the error information as follow :
Failure at final check of Oracle CRS stack.
10
I look up the log file the information :
The OCR location /data_gpfs/CRS/ocr_disk2 is inaccessible. Details in /orac
leapp/product/10.2.0/crs/log/p615a/client/ocrconfig_6881286.log.
the ocrconfig_6881286.log.information:
2014-01-24 01:32:20.361: [ OCRCONF][1]ocrconfig starts...
2014-01-24 01:32:20.389: [ OCRCONF][1]Upgrading OCR data
2014-01-24 01:32:20.391: [ OCROSD][1]utread:3: problem reading buffer 100f21d0 buflen 512 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.391: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.391: [ OCROSD][1]utread:3: problem reading buffer 100f21d0 buflen 512 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.391: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.391: [ OCROSD][1]utread:3: problem reading buffer ffffb1d0 buflen 4096 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.392: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.392: [ OCRRAW][1]propriogid:1: INVALID FORMAT
2014-01-24 01:32:20.392: [ OCROSD][1]utread:3: problem reading buffer ffffb1d0 buflen 4096 retval 0 phy_offset 102400 retry 0
2014-01-24 01:32:20.392: [ OCROSD][1]utread:4: problem reading the buffer errno 2 errstring No such file or directory
2014-01-24 01:32:20.392: [ OCRRAW][1]propriogid:1: INVALID FORMAT
2014-01-24 01:32:20.392: [ OCRRAW][1]proprioini: both disks are not OCR formatted
2014-01-24 01:32:20.392: [ OCRRAW][1]proprinit: Could not open raw device
2014-01-24 01:32:20.392: [ default][1]a_init:7!: Backend init unsuccessful : [26]
2014-01-24 01:32:20.393: [ OCRCONF][1]Exporting OCR data to [OCRUPGRADEFILE]
2014-01-24 01:32:20.393: [ OCRAPI][1]a_init:7!: Backend init unsuccessful : [33]
propriogid:1: INVALID FORMAT
2014-01-24 01:32:20.516: [ OCRRAW][1]propriowv: Vote information on disk 0 [/data_gpfs/CRS/ocr_disk1] is adjusted from [0/0] to [1/2]
2014-01-24 01:32:20.527: [ OCRRAW][1]propriowv: Vote information on disk 1 [/data_gpfs/CRS/ocr_disk2] is adjusted from [0/0] to [1/2]
2014-01-24 01:32:20.960: [ OCRRAW][1]propriniconfig:No 92 configuration
2014-01-24 01:32:20.960: [ OCRAPI][1]a_init:6a: Backend init successful
2014-01-24 01:32:21.191: [ OCRCONF][1]Initialized DATABASE keys in OCR
2014-01-24 01:32:21.349: [ OCRCONF][1]Successfully set skgfr block 0
2014-01-24 01:32:21.351: [ OCRCONF][1]Exiting [status=success]...
I dont know what cause this error,i am really trouble,who can help me !!!

Failure at Final Check of Oracle CRS Stack. ... 10

Hi,
when i run the root.sh script on second node,i'm facing this issue.i checked with my alert log. " /dev/raw/raw2" is showing as inaccessible. how to resolve this issue.
[client(1449)]CRS-1006:The OCR location /dev/raw/raw2 is inaccessible. Details in /u01/app/oracle/product/10.2.0/crs_home/log/rac5/client/ocrconfig_1449.log.
2012-08-16 12:12:32.628
[client(1449)]CRS-1006:The OCR location /dev/raw/raw2 is inaccessible. Details in /u01/app/oracle/product/10.2.0/crs_home/log/rac5/client/ocrconfig_1449.log.
2012-08-16 12:12:32.637
[client(1449)]CRS-1006:The OCR location /dev/raw/raw2 is inaccessible. Details in /u01/app/oracle/product/10.2.0/crs_home/log/rac5/client/ocrconfig_1449.log.
2012-08-16 12:12:32.742
[client(1449)]CRS-1001:The OCR was formatted using version 2.
2012-08-16 12:12:40.144
[client(1500)]CRS-1801:Cluster crs configured with nodes rac5 rac6 .
2012-08-16 12:15:00.465
[cssd(2185)]CRS-1605:CSSD voting file is online: /dev/raw/raw3. Details in /u01/app/oracle/product/10.2.0/crs_home/log/rac5/cssd/ocssd.log.
2012-08-16 12:15:03.672
[cssd(2185)]CRS-1601:CSSD Reconfiguration complete. Active nodes are rac5

Hi,
after checking (and correcting) the situtation with you raw device permission (probably you should check udev rulels), you can do the following:
as root /u01..../crs_home/crs/install/rootcrsl.pl -deconfig -force
and then run root.sh again.

Failure at final check of Oracle CRS stac

+++crs ID conflicts ocrcheck scsi
++Referred metalink note:344994.1 but it was specific to RAW devices, Need a solution for scsi device.
oracle@vx0302 bin # ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 3306636
Used space (kbytes) : 440
Available space (kbytes) : 3306196
ID : 1425438992 <<different from node1
Device/File Name : /dev/sdb1
Device/File integrity check failed
Device/File not configured
Cluster registry integrity check failed
oracle@vx0301 bin # ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 40269168
Used space (kbytes) : 308
Available space (kbytes) : 40268860
ID : 68510624 << different from node1
Device/File Name : /dev/sdb1
Device/File integrity check succeeded
Device/File not configured
Cluster registry integrity check succeeded
According to the above output, the IDs are same. I followed this http://surachartopun.com/2009/01/failure-at-final-check-of-oracle-crs.html link for the workaround. I have taken the following steps:
1. Firewall is off.
2. I have referred to the metalink, but it provides a solution only for raw devices. I have used the link http://www.oracle.com/technology/pub/articles/hunter-rac11gr2-iscsi.html to configure Openfiler for iscsi devices. The configuration is EXACTLY the same. However, when i ran ls -l /dev/iscsi/*, the output was different on both nodes.
Node1:
[oracle@vx0301 ~]# ls -l /dev/iscsi/*
/dev/iscsi/crs1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdc
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdc1
/dev/iscsi/data1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdb1
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdb1
/dev/iscsi/fra1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdd
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdd1
Node2:
[oracle@vx0302 ~]# ls -l /dev/iscsi/*
/dev/iscsi/crs1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdb
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdb1
/dev/iscsi/data1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdd1
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdd1
/dev/iscsi/fra1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdc
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdc1
Is there a way i can fix this?
Edited by: user10594250 on Jan 22, 2010 4:41 AM

+++crs ID conflicts ocrcheck scsi
++Referred metalink note:344994.1 but it was specific to RAW devices, Need a solution for scsi device.
oracle@vx0302 bin # ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 3306636
Used space (kbytes) : 440
Available space (kbytes) : 3306196
ID : 1425438992 <<different from node1
Device/File Name : /dev/sdb1
Device/File integrity check failed
Device/File not configured
Cluster registry integrity check failed
oracle@vx0301 bin # ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 40269168
Used space (kbytes) : 308
Available space (kbytes) : 40268860
ID : 68510624 << different from node1
Device/File Name : /dev/sdb1
Device/File integrity check succeeded
Device/File not configured
Cluster registry integrity check succeeded
According to the above output, the IDs are same. I followed this http://surachartopun.com/2009/01/failure-at-final-check-of-oracle-crs.html link for the workaround. I have taken the following steps:
1. Firewall is off.
2. I have referred to the metalink, but it provides a solution only for raw devices. I have used the link http://www.oracle.com/technology/pub/articles/hunter-rac11gr2-iscsi.html to configure Openfiler for iscsi devices. The configuration is EXACTLY the same. However, when i ran ls -l /dev/iscsi/*, the output was different on both nodes.
Node1:
[oracle@vx0301 ~]# ls -l /dev/iscsi/*
/dev/iscsi/crs1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdc
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdc1
/dev/iscsi/data1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdb1
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdb1
/dev/iscsi/fra1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdd
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdd1
Node2:
[oracle@vx0302 ~]# ls -l /dev/iscsi/*
/dev/iscsi/crs1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdb
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdb1
/dev/iscsi/data1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdd1
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdd1
/dev/iscsi/fra1:
total 0
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part -> ../../sdc
lrwxrwxrwx 1 root root 9 Nov 3 18:13 part1 -> ../../sdc1
Is there a way i can fix this?
Edited by: user10594250 on Jan 22, 2010 4:41 AM

CRS installation roo.sh failed -- FAILURE AT FINAL CHECK

CRS 10.2.0.1 installation on Solaris 10
root. sh failed, following error appeared.
bash-3.00# sh -x root.sh
+ /opt/oracrs/install/rootinstall
+ /opt/oracrs/install/rootconfig
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: proddb02 proddb02-priv proddb02
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/rdsk/vpath9a
Now formatting voting device: /dev/rdsk/vpath10a
Now formatting voting device: /dev/rdsk/vpath11a
Format of 3 voting devices complete.
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
10
bash-3.00#
following metalink note 240001.1 as well, no success.
any idea where should I look?

You can start looking at the following files to check if there are any errors reported :
a) The OS Messages file ( /var/adm/messages )
b) Any files with names like crsctl* under the /tmp directory
c) The client trace files under $ORA_CRS_HOME/log/<hostname>/client/*
d) The crsd and the cssd logs under $ORA_CRS_HOME/log/<hostname>/[crsd/cssd]/*
Also what does a ps -ef | grep d.bin indicate.
Which daemons get started and which do not.
Vishwa

Oracle CRS stack is already configured while running root.sh

Hi,
I am trying to install cluster ware in for 10g (10.2.0.1)RAC in RHEL5 node 1 and node 2 servers. Already i tried install clusterware but my installation failed in last step. So i uninstalled and tried my installation again from node 1. But when i execute the ./root.sh as root user in node 1 or node 2 i get the below error
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
Oracle CRS stack is already configured and will be running under init(1M)
How to remove or deconfig this CRS and CSS service properly and proceed the installation.????
Regards,
007

cd <GRID_HOME>/crs/install
run as root:
# rootcrs.pl -deconfig -force
then run under oracle user:
<GRID_HOME>/deinstall/deinstall
This should do.
If you still see this warnings, check if you have any oracle-owned files under /etc, for example /etc/oracle dir with local ocr/olr location files.
Consider renaming/removing them and try again.

Timed out waiting for the CRS stack to start on Second node.

Hi,
I am trying to setup 2 node 11gR2 RAC in vmware. I face the Timed out waiting for the CRS stack to start error on second node while running root.sh.I checked the cluster log files located in /u01/app/11.2.0/grid/log/node1/alertnode2.log it shows as mentioned below.But when i logged into ASM instance and checked the diskgroup and it is in mount state on both nodes,So i am really confused why i ended up with error in 2nd node on running root.sh. Can anyone tell me how to correct this error or any other checking needs to be done?
/u01/app/11.2.0/grid/log/node1/alertnode2.log
[ohasd(23261)]CRS-2765:Resource 'ora.crsd' has failed on server 'node2'.
2014-01-21 21:40:10.314
[crsd(25309)]CRS-1006:The OCR location +DATA is inaccessible. Details in /u01/app/11.2.0/grid/log/node2/crsd/crsd.log.
2014-01-21 21:40:11.217
[ohasd(23261)]CRS-2765:Resource 'ora.crsd' has failed on server 'node2'.
2014-01-21 21:40:12.422
[crsd(25335)]CRS-1006:The OCR location +DATA is inaccessible. Details in /u01/app/11.2.0/grid/log/node2/crsd/crsd.log.
2014-01-21 21:40:13.327
[ohasd(23261)]CRS-2765:Resource 'ora.crsd' has failed on server 'node2'.
2014-01-21 21:40:14.514
[crsd(25356)]CRS-1006:The OCR location +DATA is inaccessible. Details in /u01/app/11.2.0/grid/log/node2/crsd/crsd.log.
2014-01-21 21:40:15.439
[ohasd(23261)]CRS-2765:Resource 'ora.crsd' has failed on server 'node2'.
2014-01-21 21:40:15.440
[ohasd(23261)]CRS-2771:Maximum restart attempts reached for resource 'ora.crsd'; will not restart.
root.sh output
[root@node2 ~]# /u01/app/11.2.0/grid/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
   Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2014-01-21 21:37:55: Parsing the host name
2014-01-21 21:37:55: Checking for super user privileges
2014-01-21 21:37:55: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_p
arams
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node 11gdb, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'node2'
CRS-2676: Start of 'ora.mdnsd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'node2'
CRS-2676: Start of 'ora.gipcd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'node2'
CRS-2676: Start of 'ora.gpnpd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node2'
CRS-2676: Start of 'ora.cssdmonitor' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'node2'
CRS-2672: Attempting to start 'ora.diskmon' on 'node2'
CRS-2676: Start of 'ora.diskmon' on 'node2' succeeded
CRS-2676: Start of 'ora.cssd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'node2'
CRS-2676: Start of 'ora.ctssd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'node2'
CRS-2676: Start of 'ora.drivers.acfs' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'node2'
CRS-2676: Start of 'ora.asm' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'node2'
CRS-2676: Start of 'ora.crsd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'node2'
CRS-2676: Start of 'ora.evmd' on 'node2' succeeded
Timed out waiting for the CRS stack to start.
Regards,
007

Hi,
Are you trying to install on Vmware Workstation or Vmware Server?
If you are using vmware server add below line in your .vmx file for both node
scsi1.sharedBus = "VIRTUAL"
First check which scsi serial number are you using as above I have selected scsi1 serial during disk addition,

Oracle XE 10g connect for the first time

I have just installed Oracle10 g express edition on Windows XP service pack 3, when trying to connect for the first time
I get the ORA-12631 message error when trying to connect!!
SQL> connect system
Immettere la password:
ERROR:
ORA-12631: Recupero nome utente non riuscito
Can anyone help me?

sorry here is:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
C:\>set
ALLUSERSPROFILE=C:\Documents and Settings\All Users
ambiente=AC
APPDATA=C:\Documents and Settings\UR00780.HD01\Application Data
CLIENTNAME=Console
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=ACMI1252
ComSpec=C:\WINDOWS\system32\cmd.exe
cstella=VERONA
FP_NO_HOST_CHECK=NO
fserver=ACSWDWV01
HOMEDRIVE=H:
HOMEPATH=\
HOMESHARE=\\acnas200.intranet.unicredit.it\UR00780
KITID=AC
LOGONSERVER=\\USADSW340
LSERVER=ACSWDWV01.hd01.unicreditgroup.eu
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
Path=C:\oraclexe\app\oracle\product\10.2.0\server\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
Files\Microsoft SQL Server\80\Tools\BINN;C:\Program Files\IBM\Personal Communications\;C:\Program Files\IBM\Trace Facil
ity\;C:\Program Files\Symantec\pcAnywhere\;C:\PROGRA~1\IBM\CLIENT~1;C:\PROGRA~1\IBM\CLIENT~1\Shared;C:\Program Files\Int
el\DMIX;c:\Program Files\Microsoft SQL Server\90\Tools\binn\
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
PCOMM_Root=C:\Program Files\IBM\Personal Communications\
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 37 Stepping 2, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=2502
ProgramFiles=C:\Program Files
PROMPT=$P$G
SESSIONNAME=Console
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=C:\DOCUME~1\UR0078~1.HD0\LOCALS~1\Temp
TMP=C:\DOCUME~1\UR0078~1.HD0\LOCALS~1\Temp
USERDNSDOMAIN=HD01.UNICREDITGROUP.EU
USERDOMAIN=HD01
USERNAME=UR00780
USERPROFILE=C:\Documents and Settings\UR00780.HD01
windir=C:\WINDOWS
C:\>

Oracle CRS(10.1) installation fails

Hi,
Following is a detailed synopsis of my unsuccessful attempt at installing Oracle Cluster Ready Services(10.1) on a two node Windows 2000 advanced server system with external SCSI hard disk.
Following is the Configuration of both the Nodes
1.Intel Pentium P4 2.80 Ghz CPU
2.512 MB RAM
3.37 GB HDD
4.Windows 2000 Advanced Server SP4
5.Adaptec SCSI CARD 39160 ULTRA160 SCSI Controller
6.Two -10/100 Mbps Fast Ethernet PCI Adapter (DLINK)
Clusterware
Oracle Cluster Ready Services 10.1.0.2
Configuration for SCSI
1.Model Make :SEAGATE
2.Model Number :ST336607
3.Firmware: 0007
4.Model Name: Cheetah 10K.6
5.Capacity: 36.7GB
6.Speed: 10000 rpm
Following are the pre-installation steps.
1.The system setup involved 2 identical nodes with Windows 2000 Advanced Server SP4 OS.
2.Two static Public IP's for both the nodes
3.Two private IP's which can be pinged. (1st Node Private IP 10.10.10.2 2nd Node Private IP 10.10.10.3)
4.Entry of the private IP's can also be found in the host file.
5.ORA_DBA win group was created on both the nodes.
6.Both the nodes are connected to a external SCSI hard disk
CRS installation Steps
1.On the Cluster Configuration page the Public Node Names and corresponding Private Node Names were entered.
2.On the Network Interface Usage Page one interface was selected as Public (192.168.144.0) and the other interface was selected as Private (10.10.10.0)
3.On Disk formatting Options page the âDo not format any logical drives was selectedâ.
4.Finally the Logical partitions made on the SCSI hard disk were assigned to the OCR and voting disk.
OUI installed the Oracle folder and services on the primary node and on the remote node. But failed to start the services on both the nodes.
Following is snippet of the OUI installing log.
Save Cluster Inventory
Error occured while trying to stop service OracleOUIOraCr10g_homeService on one or more cluster nodes. Please stop service on nodes pc-p41300 using the Service Control Manager.
Invoking OUI on cluster nodes pc-p41300
C:\DOCUME~1\salil\LOCALS~1\Temp\OraInstall2005-01-04_09-19-56PM\oui\bin\setup.exe -jreLoc C:\DOCUME~1\salil\LOCALS~1\Temp\OraInstall2005-01-04_09-19-56PM\jre\1.4.2 -paramFile C:\DOCUME~1\salil\LOCALS~1\Temp\OraInstall2005-01-04_09-19-56PM\oui/clusterparam.ini -silent -ignoreSysPrereqs -attachHome -local -noClusterEnabled ORACLE_HOME=D:\oracle\product\10.1.0\crs ORACLE_HOME_NAME=OraCr10g_home CLUSTER_NODES=PC-P41295,pc-p41300 CRS=true
PRKC-1010 : Failed to stop the service on all nodes
Starting to execute configuration assistants
Launched configuration assistant 'Oracle Cluster Ready Services Configuration Assistant'
Tool type is: Optional.
The command being spawned is: 'D:\oracle\product\10.1.0\crs/install/crssetup.config.bat'
Start output from spawned process:
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: pc-p41300 pc-p41300-node pc-p41300
node 2: pc-p41295 pc-p41295-node pc-p41295
Creating OCR keys for user 'salil', privgrp ''..
Operation successful.
Now formatting voting device: \\.\votedsk
Successful in setting block0 for voting disk.
Format complete.
Step 1: checking status of CRS cluster
Step 2: configuring OCR repository
Step 3: configuring safe mode for CRS stack
Step 4: Starting up CRS stack on all nodes
pc-p41300 service OracleCSService in improper STOPPED state, err(997)
PC-P41295 service OracleCSService in improper PENDING state, err(997)
Following errors were also observed at Operating System Level
Event ID:11
Description: The driver detected a controller error on \Device\SCSI\adpu160m1
Note: adpu160m is the driver for the adaptec controllers
Event ID:9
Description: The device, \Device\SCSI\adpu160m1, did not respond within the time out period.
Since the above method failed following alternate method was tired
The CRS software was installed on just one node. This installation went about without any problems with all the services up and running.
Using the ADDNode utility provided by CRS another node was added. But again when CRSSetup.exe tried to start the services on the remote node following error was encountered.
PC-P41300 service OracleCSService in improper PENDING state, err(997)
I would be grateful if some could help me with this problem.
Thanking you in Anticipation,
Regards,
Salil([email protected])

sa**** wrote:
Hi,
Following is a detailed synopsis of my unsuccessful attempt at installing Oracle Cluster Ready Services(10.1) on a two node Windows 2000 advanced server system with external SCSI hard disk.
Following is the Configuration of both the Nodes
1.Intel Pentium P4 2.80 Ghz CPU
2.512 MB RAM
3.37 GB HDD
4.Windows 2000 Advanced Server SP4
5.Adaptec SCSI CARD 39160 ULTRA160 SCSI Controller
6.Two -10/100 Mbps Fast Ethernet PCI Adapter (DLINK)
Clusterware
Oracle Cluster Ready Services 10.1.0.2
Configuration for SCSI
1.Model Make :SEAGATE
2.Model Number :ST336607
3.Firmware: 0007
4.Model Name: Cheetah 10K.6
5.Capacity: 36.7GB
6.Speed: 10000 rpm
Following are the pre-installation steps.
1.The system setup involved 2 identical nodes with Windows 2000 Advanced Server SP4 OS.
2.Two static Public IP's for both the nodes
3.Two private IP's which can be pinged. (1st Node Private IP 10.10.10.2 2nd Node Private IP 10.10.10.3)
4.Entry of the private IP's can also be found in the host file.
5.ORA_DBA win group was created on both the nodes.
6.Both the nodes are connected to a external SCSI hard disk
CRS installation Steps
1.On the Cluster Configuration page the Public Node Names and corresponding Private Node Names were entered.
2.On the Network Interface Usage Page one interface was selected as Public (192.168.144.0) and the other interface was selected as Private (10.10.10.0)
3.On Disk formatting Options page the âDo not format any logical drives was selectedâ.
4.Finally the Logical partitions made on the SCSI hard disk were assigned to the OCR and voting disk.
OUI installed the Oracle folder and services on the primary node and on the remote node. But failed to start the services on both the nodes.
Following is snippet of the OUI installing log.
<HR>
Save Cluster Inventory
Error occured while trying to stop service OracleOUIOraCr10g_homeService on one or more cluster nodes. Please stop service on nodes pc-p41300 using the Service Control Manager.
Invoking OUI on cluster nodes pc-p41300
C:\DOCUME~1\salil\LOCALS~1\Temp\OraInstall2005-01-04_09-19-56PM\oui\bin\setup.exe -jreLoc C:\DOCUME~1\salil\LOCALS~1\Temp\OraInstall2005-01-04_09-19-56PM\jre\1.4.2 -paramFile C:\DOCUME~1\salil\LOCALS~1\Temp\OraInstall2005-01-04_09-19-56PM\oui/clusterparam.ini -silent -ignoreSysPrereqs -attachHome -local -noClusterEnabled ORACLE_HOME=D:\oracle\product\10.1.0\crs ORACLE_HOME_NAME=OraCr10g_home CLUSTER_NODES=PC-P41295,pc-p41300 CRS=true
PRKC-1010 : Failed to stop the service on all nodes
Starting to execute configuration assistants
Launched configuration assistant 'Oracle Cluster Ready Services Configuration Assistant'
Tool type is: Optional.
The command being spawned is: 'D:\oracle\product\10.1.0\crs/install/crssetup.config.bat'
Start output from spawned process:
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <NODENUMBER>: <NODENAME><PRIVATE name interconnect><HOSTNAME>
node 1: pc-p41300 pc-p41300-node pc-p41300
node 2: pc-p41295 pc-p41295-node pc-p41295
Creating OCR keys for user 'salil', privgrp ''..
Operation successful.
Now formatting voting device: \\.\votedsk
Successful in setting block0 for voting disk.
Format complete.
Step 1: checking status of CRS cluster
Step 2: configuring OCR repository
Step 3: configuring safe mode for CRS stack
Step 4: Starting up CRS stack on all nodes
<HR>
pc-p41300 service OracleCSService in improper STOPPED state, err(997)
PC-P41295 service OracleCSService in improper PENDING state, err(997)
<HR>
Following errors were also observed at Operating System Level
Event ID:11
Description: The driver detected a controller error on \Device\SCSI\adpu160m1
Note: adpu160m is the driver for the adaptec controllers
Event ID:9
Description: The device, \Device\SCSI\adpu160m1, did not respond within the time out period.
Since the above method failed following alternate method was tired
The CRS software was installed on just one node. This installation went about without any problems with all the services up and running.
Using the ADDNode utility provided by CRS another node was added. But again when CRSSetup.exe tried to start the services on the remote node following error was encountered.
PC-P41300 service OracleCSService in improper PENDING state, err(997)
I would be grateful if some could help me with this problem.
Thanking you in Anticipation,
Regards,
Salil([email protected]) 
I do not know what you're talking about, I'm new to this.

CLSRSC-117: Failed to start Oracle Clusterware stack Oracle 12c clusterware

I'm installing Oracle 12c clusterware on 2 nodes.
i installed it on node 1 and after executing the ./root.sh is was executed successfully.
but after trying to execute it on the other node it fails with the next error:
CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'dbrac1'
CRS-2883: Resource 'ora.cssdmonitor' failed during Clusterware stack start.
CRS-4406: Oracle High Availability Services synchronous start failed.
CRS-4000: Command Start failed, or completed with errors.
2015/02/11 01:53:15 CLSRSC-117: Failed to start Oracle Clusterware stack
my ASM disks are fine and i can see them via oracleasm scandisks and oracleasm listdiks.

Sorry it was successful on node 2 dbrac2 (which is my main node and has the oracle 12c sources)
here is the complete logs:
2015-02-11 00:58:11: Checking parameters from paramfile /u01/app/12.1.0/grid_1/crs/install/crsconfig_params to validate installer variables
2015-02-11 00:58:11: Set BC to 1
2015-02-11 00:58:11: ASM_CONFIG is near
2015-02-11 00:58:11: The configuration parameter file /u01/app/12.1.0/grid_1/crs/install/crsconfig_params is valid
2015-02-11 00:58:11: ### Printing the configuration values from files:
2015-02-11 00:58:11:    /u01/app/12.1.0/grid_1/crs/install/crsconfig_params
2015-02-11 00:58:11:    /u01/app/12.1.0/grid_1/crs/install/s_crsconfig_defs
2015-02-11 00:58:11: ASM_AU_SIZE=1
2015-02-11 00:58:11: ASM_CONFIG=near
2015-02-11 00:58:11: ASM_CREDENTIALS=
2015-02-11 00:58:11: ASM_DISCOVERY_STRING=
2015-02-11 00:58:11: ASM_DISKS=ORCL:OC1,ORCL:OC2,ORCL:OC3
2015-02-11 00:58:11: ASM_DISK_GROUP=Votting
2015-02-11 00:58:11: ASM_REDUNDANCY=EXTERNAL
2015-02-11 00:58:11: ASM_SPFILE=
2015-02-11 00:58:11: ASM_UPGRADE=false
2015-02-11 00:58:11: BIG_CLUSTER=true
2015-02-11 00:58:11: CLSCFG_MISSCOUNT=
2015-02-11 00:58:11: CLUSTER_GUID=
2015-02-11 00:58:11: CLUSTER_NAME=dbrac-cluster
2015-02-11 00:58:11: CRFHOME="/u01/app/12.1.0/grid_1"
2015-02-11 00:58:11: CRS_LIMIT_CORE=unlimited
2015-02-11 00:58:11: CRS_LIMIT_MEMLOCK=unlimited
2015-02-11 00:58:11: CRS_LIMIT_OPENFILE=65536
2015-02-11 00:58:11: CRS_LIMIT_STACK=2048
2015-02-11 00:58:11: CRS_LSNR_STACK=10240
2015-02-11 00:58:11: CRS_NODEVIPS='AUTO/255.255.255.0/bond0,AUTO/255.255.255.0/bond0'
2015-02-11 00:58:11: CRS_STORAGE_OPTION=1
2015-02-11 00:58:11: CSS_LEASEDURATION=400
2015-02-11 00:58:11: DIRPREFIX=
2015-02-11 00:58:11: DISABLE_OPROCD=0
2015-02-11 00:58:11: EXTERNAL_ORACLE=/opt/oracle
2015-02-11 00:58:11: EXTERNAL_ORACLE_BIN=/opt/oracle/bin
2015-02-11 00:58:11: GNS_ADDR_LIST=172.20.30.12
2015-02-11 00:58:11: GNS_ALLOW_NET_LIST=
2015-02-11 00:58:11: GNS_CONF=true
2015-02-11 00:58:11: GNS_CREDENTIALS=
2015-02-11 00:58:11: GNS_DENY_ITF_LIST=
2015-02-11 00:58:11: GNS_DENY_NET_LIST=
2015-02-11 00:58:11: GNS_DOMAIN_LIST=gns.kbe.com
2015-02-11 00:58:11: GNS_TYPE=local
2015-02-11 00:58:11: GPNPCONFIGDIR=/u01/app/12.1.0/grid_1
2015-02-11 00:58:11: GPNPGCONFIGDIR=/u01/app/12.1.0/grid_1
2015-02-11 00:58:11: GPNP_PA=
2015-02-11 00:58:11: HOST_NAME_LIST=dbrac2,dbrac1
2015-02-11 00:58:11: HUB_NODE_LIST=dbrac2,dbrac1
2015-02-11 00:58:11: HUB_NODE_VIPS=AUTO,AUTO
2015-02-11 00:58:11: HUB_SIZE=32
2015-02-11 00:58:11: ID=/etc/init.d
2015-02-11 00:58:11: INIT=/sbin/init
2015-02-11 00:58:11: INITCTL=/sbin/initctl
2015-02-11 00:58:11: ISROLLING=true
2015-02-11 00:58:11: IT=/etc/inittab
2015-02-11 00:58:11: JLIBDIR=/u01/app/12.1.0/grid_1/jlib
2015-02-11 00:58:11: JREDIR=/u01/app/12.1.0/grid_1/jdk/jre/
2015-02-11 00:58:11: LANGUAGE_ID=AMERICAN_AMERICA.AL32UTF8
2015-02-11 00:58:11: LISTENER_USERNAME=grid
2015-02-11 00:58:11: MGMT_DB=true
2015-02-11 00:58:11: MSGFILE=/var/adm/messages
2015-02-11 00:58:11: NETWORKS="bond0"/172.20.30.0:public,"bond1"/172.20.190.0:asm,"bond1"/172.20.190.0:cluster_interconnect
2015-02-11 00:58:11: NEW_HOST_NAME_LIST=
2015-02-11 00:58:11: NEW_NODEVIPS='AUTO/255.255.255.0/bond0,AUTO/255.255.255.0/bond0'
2015-02-11 00:58:11: NEW_NODE_NAME_LIST=
2015-02-11 00:58:11: NEW_PRIVATE_NAME_LIST=
2015-02-11 00:58:11: NODELIST=dbrac2,dbrac1
2015-02-11 00:58:11: NODE_NAME_LIST=dbrac2,dbrac1
2015-02-11 00:58:11: OCRCONFIG=/etc/oracle/ocr.loc
2015-02-11 00:58:11: OCRCONFIGDIR=/etc/oracle
2015-02-11 00:58:11: OCRID=
2015-02-11 00:58:11: OCRLOC=ocr.loc
2015-02-11 00:58:11: OCR_LOCATIONS=NO_VAL
2015-02-11 00:58:11: OLASTGASPDIR=/etc/oracle/lastgasp
2015-02-11 00:58:11: OLD_CRS_HOME=
2015-02-11 00:58:11: OLRCONFIG=/etc/oracle/olr.loc
2015-02-11 00:58:11: OLRCONFIGDIR=/etc/oracle
2015-02-11 00:58:11: OLRLOC=olr.loc
2015-02-11 00:58:11: OPROCDCHECKDIR=/etc/oracle/oprocd/check
2015-02-11 00:58:11: OPROCDDIR=/etc/oracle/oprocd
2015-02-11 00:58:11: OPROCDFATALDIR=/etc/oracle/oprocd/fatal
2015-02-11 00:58:11: OPROCDSTOPDIR=/etc/oracle/oprocd/stop
2015-02-11 00:58:11: ORACLE_BASE=/u01/app/grid
2015-02-11 00:58:11: ORACLE_HOME=/u01/app/12.1.0/grid_1
2015-02-11 00:58:11: ORACLE_OWNER=grid
2015-02-11 00:58:11: ORA_ASM_GROUP=asmadmin
2015-02-11 00:58:11: ORA_DBA_GROUP=oinstall
2015-02-11 00:58:11: PRIVATE_NAME_LIST=
2015-02-11 00:58:11: RCALLDIR=/etc/rc.d/rc0.d /etc/rc.d/rc1.d /etc/rc.d/rc2.d /etc/rc.d/rc3.d /etc/rc.d/rc4.d /etc/rc.d/rc5.d /etc/rc.d/rc6.d
2015-02-11 00:58:11: RCKDIR=/etc/rc.d/rc0.d /etc/rc.d/rc1.d /etc/rc.d/rc2.d /etc/rc.d/rc4.d /etc/rc.d/rc6.d
2015-02-11 00:58:11: RCSDIR=/etc/rc.d/rc3.d /etc/rc.d/rc5.d
2015-02-11 00:58:11: RC_KILL=K15
2015-02-11 00:58:11: RC_KILL_OLD=K96
2015-02-11 00:58:11: RC_KILL_OLD2=K19
2015-02-11 00:58:11: RC_START=S96
2015-02-11 00:58:11: REUSEDG=false
2015-02-11 00:58:11: RIM_NODE_LIST=
2015-02-11 00:58:11: SCAN_NAME=kberac-scan.gns.kbe.com
2015-02-11 00:58:11: SCAN_PORT=1521
2015-02-11 00:58:11: SCRBASE=/etc/oracle/scls_scr
2015-02-11 00:58:11: SILENT=false
2015-02-11 00:58:11: SO_EXT=so
2015-02-11 00:58:11: SRVCFGLOC=srvConfig.loc
2015-02-11 00:58:11: SRVCONFIG=/var/opt/oracle/srvConfig.loc
2015-02-11 00:58:11: SRVCONFIGDIR=/var/opt/oracle
2015-02-11 00:58:11: TZ=Asia/Riyadh
2015-02-11 00:58:11: UPSTART_INIT_DIR=/etc/init
2015-02-11 00:58:11: USER_IGNORED_PREREQ=true
2015-02-11 00:58:11: VNDR_CLUSTER=false
2015-02-11 00:58:11: VOTING_DISKS=NO_VAL
2015-02-11 00:58:11: ### Printing other configuration values ###
2015-02-11 00:58:11: CLSCFG_EXTRA_PARMS=
2015-02-11 00:58:11: HAS_GROUP=oinstall
2015-02-11 00:58:11: HAS_USER=root
2015-02-11 00:58:11: HOST=dbrac1
2015-02-11 00:58:11: OLR_DIRECTORY=/u01/app/12.1.0/grid_1/cdata
2015-02-11 00:58:11: OLR_LOCATION=/u01/app/12.1.0/grid_1/cdata/dbrac1.olr
2015-02-11 00:58:11: ORA_CRS_HOME=/u01/app/12.1.0/grid_1
2015-02-11 00:58:11: SIHA=0
2015-02-11 00:58:11: SUCC_REBOOT=0
2015-02-11 00:58:11: SUPERUSER=root
2015-02-11 00:58:11: addfile=/u01/app/12.1.0/grid_1/crs/install/crsconfig_addparams
2015-02-11 00:58:11: crscfg_trace=1
2015-02-11 00:58:11: crscfg_trace_file=/u01/app/12.1.0/grid_1/cfgtoollogs/crsconfig/rootcrs_dbrac1_2015-02-11_00-58-11AM.log
2015-02-11 00:58:11: old_nodevips=
2015-02-11 00:58:11: osdfile=/u01/app/12.1.0/grid_1/crs/install/s_crsconfig_defs
2015-02-11 00:58:11: parameters_valid=1
2015-02-11 00:58:11: paramfile=/u01/app/12.1.0/grid_1/crs/install/crsconfig_params
2015-02-11 00:58:11: platform_family=unix
2015-02-11 00:58:11: srvctl_trc_suff=0
2015-02-11 00:58:11: stackStartLevel=11
2015-02-11 00:58:11: user_is_superuser=1
2015-02-11 00:58:11: ### Printing of configuration values complete ###
2015-02-11 00:58:11: Save the ASM password file location: +Votting/orapwASM
2015-02-11 00:58:11: Performing few checks before running scripts
2015-02-11 00:58:11: Attempt to get current working directory
2015-02-11 00:58:11: Running as user grid: pwd
2015-02-11 00:58:11: s_run_as_user2: Running /bin/su grid -c ' pwd '
2015-02-11 00:58:11: Removing file /tmp/fileCnF377
2015-02-11 00:58:11: Successfully removed file: /tmp/fileCnF377
2015-02-11 00:58:11: /bin/su successfully executed
2015-02-11 00:58:11: Pre-checkes for running scripts passed
2015-02-11 00:58:11: Instantiating scripts in GI home: /u01/app/12.1.0/grid_1
2015-02-11 00:58:11: saving current owner/permisssios of parent dir of /u01/app/12.1.0
2015-02-11 00:58:11: saving parent dir ACLs in /u01/app/12.1.0/grid_1/crs/install/ParentDirPerm_dbrac1.txt
2015-02-11 00:58:11: loading... /u01/app/12.1.0/grid_1/crs/install/ParentDirPerm_dbrac1.txt
2015-02-11 00:58:11: Getting file permissions for /u01/app/12.1.0
2015-02-11 00:58:11: Getting file permissions for /u01/app
2015-02-11 00:58:11: Getting file permissions for /u01
2015-02-11 00:58:11: Executing cmd: /u01/app/12.1.0/grid_1/bin/sqlplus -V
2015-02-11 00:58:11: Command output:
>
> SQL*Plus: Release 12.1.0.1.0 Production
>
>End Command output
2015-02-11 00:58:11: Got CRS release version: 12.1.0.1.0
2015-02-11 00:58:11:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK
2015-02-11 00:58:11: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK
2015-02-11 00:58:11: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK '
2015-02-11 00:58:11: Removing file /tmp/fileBau02q
2015-02-11 00:58:11: Successfully removed file: /tmp/fileBau02q
2015-02-11 00:58:11: /bin/su successfully executed
2015-02-11 00:58:11: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -pname VERSION
2015-02-11 00:58:11: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -pname VERSION '
2015-02-11 00:58:12: Removing file /tmp/fileMP6nyK
2015-02-11 00:58:12: Successfully removed file: /tmp/fileMP6nyK
2015-02-11 00:58:12: /bin/su successfully executed
2015-02-11 00:58:12: The property VERSION for checkpoint:ROOTCRS_STACK is there
2015-02-11 00:58:12: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -listckpt -name ROOTCRS_STACK -pname VERSION
2015-02-11 00:58:12: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -listckpt -name ROOTCRS_STACK -pname VERSION '
2015-02-11 00:58:12: Removing file /tmp/filelarvE4
2015-02-11 00:58:12: Successfully removed file: /tmp/filelarvE4
2015-02-11 00:58:12: /bin/su successfully executed
2015-02-11 00:58:12: Succeeded to get property value:VERSION=12.1.0.1.0
2015-02-11 00:58:12: version1 is 12.1.0.1.0
2015-02-11 00:58:12: version2 is 12.1.0.1.0
2015-02-11 00:58:12: Version match status is 1
2015-02-11 00:58:12: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -status
2015-02-11 00:58:12: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -status '
2015-02-11 00:58:12: Removing file /tmp/file2xL0lp
2015-02-11 00:58:12: Successfully removed file: /tmp/file2xL0lp
2015-02-11 00:58:12: /bin/su successfully executed
2015-02-11 00:58:12: The 'ROOTCRS_STACK' status is FAILED
2015-02-11 00:58:12: USER_IGNORED_PREREQ is set to 1
2015-02-11 00:58:12: Executing cmd: /u01/app/12.1.0/grid_1/bin/clsecho -p has -f clsrsc -m 363
2015-02-11 00:58:12: Command output:
> CLSRSC-363: User ignored prerequisites during installation
>End Command output
2015-02-11 00:58:12: Executing cmd: /u01/app/12.1.0/grid_1/bin/clsecho -p has -f clsrsc -m 363
2015-02-11 00:58:12: Command output:
> CLSRSC-363: User ignored prerequisites during installation
>End Command output
2015-02-11 00:58:12: CLSRSC-363: User ignored prerequisites during installation
2015-02-11 00:58:12: Set BC to 1
2015-02-11 00:58:12: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -status
2015-02-11 00:58:12: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -status '
2015-02-11 00:58:12: Removing file /tmp/fileUO8xNK
2015-02-11 00:58:12: Successfully removed file: /tmp/fileUO8xNK
2015-02-11 00:58:12: /bin/su successfully executed
2015-02-11 00:58:12: The 'ROOTCRS_STACK' status is FAILED
2015-02-11 00:58:12: Saving the configuration parameter file data
2015-02-11 00:58:12: checkpoint status of ROOTCRS_STACK is FAIL
2015-02-11 00:58:12:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_PARAM
2015-02-11 00:58:12: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_PARAM
2015-02-11 00:58:12: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_PARAM '
2015-02-11 00:58:13: Removing file /tmp/file53D9Q6
2015-02-11 00:58:13: Successfully removed file: /tmp/file53D9Q6
2015-02-11 00:58:13: /bin/su successfully executed
2015-02-11 00:58:13:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OSDSETUP
2015-02-11 00:58:13: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OSDSETUP
2015-02-11 00:58:13: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OSDSETUP '
2015-02-11 00:58:13: Removing file /tmp/fileJAZWvt
2015-02-11 00:58:13: Successfully removed file: /tmp/fileJAZWvt
2015-02-11 00:58:13: /bin/su successfully executed
2015-02-11 00:58:13: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OSDSETUP -status
2015-02-11 00:58:13: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OSDSETUP -status '
2015-02-11 00:58:13: Removing file /tmp/filesnoyLQ
2015-02-11 00:58:13: Successfully removed file: /tmp/filesnoyLQ
2015-02-11 00:58:13: /bin/su successfully executed
2015-02-11 00:58:13: The 'ROOTCRS_OSDSETUP' status is SUCCESS
2015-02-11 00:58:13: Oracle CRS home = /u01/app/12.1.0/grid_1
2015-02-11 00:58:13: Host name = dbrac1
2015-02-11 00:58:13: CRS user = grid
2015-02-11 00:58:13: Oracle CRS home = /u01/app/12.1.0/grid_1
2015-02-11 00:58:13: GPnP host = dbrac1
2015-02-11 00:58:13: Oracle GPnP home = /u01/app/12.1.0/grid_1/gpnp
2015-02-11 00:58:13: Oracle GPnP local home = /u01/app/12.1.0/grid_1/gpnp/dbrac1
2015-02-11 00:58:13: GPnP directories verified.
2015-02-11 00:58:13: Checking to see if Oracle CRS stack is already configured
2015-02-11 00:58:13: Oracle CRS stack is already configured and will be running under init (1M)
2015-02-11 00:58:13: ---Checking local gpnp setup...
2015-02-11 00:58:13: chk gpnphome /u01/app/12.1.0/grid_1/gpnp/dbrac1: profile_ok 1 wallet_ok 1 r/o_wallet_ok 1
2015-02-11 00:58:13: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/peer" -wu=peer
2015-02-11 00:58:13: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/peer" -wu=peer
2015-02-11 00:58:13: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/peer" -wu=peer '
2015-02-11 00:58:13: Removing file /tmp/fileC5RBEe
2015-02-11 00:58:13: Successfully removed file: /tmp/fileC5RBEe
2015-02-11 00:58:13: /bin/su successfully executed
2015-02-11 00:58:13: gpnptool: rc=0
2015-02-11 00:58:13: gpnptool output:
Profile signature is valid.
2015-02-11 00:58:13: Profile "/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" signature is VALID for wallet "file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/peer"
2015-02-11 00:58:13: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/prdr" -wu=peer
2015-02-11 00:58:13: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/prdr" -wu=peer
2015-02-11 00:58:13: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/prdr" -wu=peer '
2015-02-11 00:58:14: Removing file /tmp/filebJUCzD
2015-02-11 00:58:14: Successfully removed file: /tmp/filebJUCzD
2015-02-11 00:58:14: /bin/su successfully executed
2015-02-11 00:58:14: gpnptool: rc=0
2015-02-11 00:58:14: gpnptool output:
Profile signature is valid.
2015-02-11 00:58:14: Profile "/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" signature is VALID for wallet "file:/u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/prdr"
2015-02-11 00:58:14: chk gpnphome /u01/app/12.1.0/grid_1/gpnp/dbrac1: OK
2015-02-11 00:58:14: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool getpval -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -o="/tmp/fileW3oHv3" -prf_cn -prf_cid -prf_sq -prf_pa
2015-02-11 00:58:14: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool getpval -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -o="/tmp/fileW3oHv3" -prf_cn -prf_cid -prf_sq -prf_pa
2015-02-11 00:58:14: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool getpval -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -o="/tmp/fileW3oHv3" -prf_cn -prf_cid -prf_sq -prf_pa '
2015-02-11 00:58:14: Removing file /tmp/fileXuqOrt
2015-02-11 00:58:14: Successfully removed file: /tmp/fileXuqOrt
2015-02-11 00:58:14: /bin/su successfully executed
2015-02-11 00:58:14: gpnptool: rc=0
2015-02-11 00:58:14: gpnptool output:
2015-02-11 00:58:14: Removing file /tmp/fileW3oHv3
2015-02-11 00:58:14: Successfully removed file: /tmp/fileW3oHv3
2015-02-11 00:58:14: setting profile permissions
2015-02-11 00:58:14: resfiles are /u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/root/ewallet.p12 /u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/peer/cwallet.sso /u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/prdr/cwallet.sso /u01/app/12.1.0/grid_1/gpnp/dbrac1/wallets/pa/cwallet.sso
2015-02-11 00:58:14: GPnP Wallets ownership/permissions successfully set. [node-local=1]
2015-02-11 00:58:14: ---Checking cluster-wide gpnp setup...
2015-02-11 00:58:14: chk gpnphome /u01/app/12.1.0/grid_1/gpnp: profile_ok 1 wallet_ok 1 r/o_wallet_ok 1
2015-02-11 00:58:14: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer" -wu=peer
2015-02-11 00:58:14: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer" -wu=peer
2015-02-11 00:58:14: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer" -wu=peer '
2015-02-11 00:58:15: Removing file /tmp/filefqLPlU
2015-02-11 00:58:15: Successfully removed file: /tmp/filefqLPlU
2015-02-11 00:58:15: /bin/su successfully executed
2015-02-11 00:58:15: gpnptool: rc=0
2015-02-11 00:58:15: gpnptool output:
Profile signature is valid.
2015-02-11 00:58:15: Profile "/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" signature is VALID for wallet "file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer"
2015-02-11 00:58:15: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/prdr" -wu=peer
2015-02-11 00:58:15: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/prdr" -wu=peer
2015-02-11 00:58:15: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/prdr" -wu=peer '
2015-02-11 00:58:15: Removing file /tmp/fileudaEhm
2015-02-11 00:58:15: Successfully removed file: /tmp/fileudaEhm
2015-02-11 00:58:15: /bin/su successfully executed
2015-02-11 00:58:15: gpnptool: rc=0
2015-02-11 00:58:15: gpnptool output:
Profile signature is valid.
2015-02-11 00:58:15: Profile "/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" signature is VALID for wallet "file:/u01/app/12.1.0/grid_1/gpnp/wallets/prdr"
2015-02-11 00:58:15: chk gpnphome /u01/app/12.1.0/grid_1/gpnp: OK
2015-02-11 00:58:15: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool getpval -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -o="/tmp/fileYSmOeP" -prf_cn -prf_cid -prf_sq -prf_pa
2015-02-11 00:58:15: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool getpval -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -o="/tmp/fileYSmOeP" -prf_cn -prf_cid -prf_sq -prf_pa
2015-02-11 00:58:15: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool getpval -p="/u01/app/12.1.0/grid_1/gpnp/profiles/peer/profile.xml" -o="/tmp/fileYSmOeP" -prf_cn -prf_cid -prf_sq -prf_pa '
2015-02-11 00:58:15: Removing file /tmp/filernP1bi
2015-02-11 00:58:15: Successfully removed file: /tmp/filernP1bi
2015-02-11 00:58:15: /bin/su successfully executed
2015-02-11 00:58:15: gpnptool: rc=0
2015-02-11 00:58:15: gpnptool output:
2015-02-11 00:58:15: Removing file /tmp/fileYSmOeP
2015-02-11 00:58:15: Successfully removed file: /tmp/fileYSmOeP
2015-02-11 00:58:15: setting profile permissions
2015-02-11 00:58:15: resfiles are /u01/app/12.1.0/grid_1/gpnp/wallets/root/ewallet.p12 /u01/app/12.1.0/grid_1/gpnp/wallets/peer/cwallet.sso /u01/app/12.1.0/grid_1/gpnp/wallets/prdr/cwallet.sso /u01/app/12.1.0/grid_1/gpnp/wallets/pa/cwallet.sso
2015-02-11 00:58:15: GPnP Wallets ownership/permissions successfully set. [node-local=0]
2015-02-11 00:58:15: gpnp setup checked: local valid? 1 cluster-wide valid? 1
2015-02-11 00:58:15: gpnptool: run /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer" -wu=peer
2015-02-11 00:58:15: Running as user grid: /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer" -wu=peer
2015-02-11 00:58:15: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/gpnptool verify -p="/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" -w="file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer" -wu=peer '
2015-02-11 00:58:16: Removing file /tmp/fileaK8v7L
2015-02-11 00:58:16: Successfully removed file: /tmp/fileaK8v7L
2015-02-11 00:58:16: /bin/su successfully executed
2015-02-11 00:58:16: gpnptool: rc=0
2015-02-11 00:58:16: gpnptool output:
Profile signature is valid.
2015-02-11 00:58:16: Profile "/u01/app/12.1.0/grid_1/gpnp/dbrac1/profiles/peer/profile.xml" signature is VALID for wallet "file:/u01/app/12.1.0/grid_1/gpnp/wallets/peer"
2015-02-11 00:58:16: Local and Cluster-wide setups signed with same wallet.
gpnp setup: CLUSTERWIDE
2015-02-11 00:58:16: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl query crs releaseversion
2015-02-11 00:58:16: Command output:
> Oracle High Availability Services release version on the local node is [12.1.0.1.0]
>End Command output
2015-02-11 00:58:16: Version String passed is: [Oracle High Availability Services release version on the local node is [12.1.0.1.0]]
2015-02-11 00:58:16: Version Info returned is : [12.1.0.1.0]
2015-02-11 00:58:16: Got CRS release version: 12.1.0.1.0
2015-02-11 00:58:16: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -status
2015-02-11 00:58:16: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_STACK -status '
2015-02-11 00:58:16: Removing file /tmp/fileshWmsh
2015-02-11 00:58:16: Successfully removed file: /tmp/fileshWmsh
2015-02-11 00:58:16: /bin/su successfully executed
2015-02-11 00:58:16: The 'ROOTCRS_STACK' is either in START/FAILED state
2015-02-11 00:58:16:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR
2015-02-11 00:58:16: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR
2015-02-11 00:58:16: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR '
2015-02-11 00:58:16: Removing file /tmp/fileIB67nN
2015-02-11 00:58:16: Successfully removed file: /tmp/fileIB67nN
2015-02-11 00:58:16: /bin/su successfully executed
2015-02-11 00:58:16: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR -status
2015-02-11 00:58:16: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR -status '
2015-02-11 00:58:16: Removing file /tmp/filenbKqUj
2015-02-11 00:58:16: Successfully removed file: /tmp/filenbKqUj
2015-02-11 00:58:16: /bin/su successfully executed
2015-02-11 00:58:16: The 'ROOTCRS_OLR' status is SUCCESS
2015-02-11 00:58:16: 'ROOTCRS_OLR' state is 'SUCCESS'
2015-02-11 00:58:16: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR -status
2015-02-11 00:58:16: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OLR -status '
2015-02-11 00:58:17: Removing file /tmp/filevkRq2Q
2015-02-11 00:58:17: Successfully removed file: /tmp/filevkRq2Q
2015-02-11 00:58:17: /bin/su successfully executed
2015-02-11 00:58:17: The 'ROOTCRS_OLR' status is SUCCESS
2015-02-11 00:58:17: OLR is already initialized
2015-02-11 00:58:17: Executing /u01/app/12.1.0/grid_1/bin/crsctl create scr grid
2015-02-11 00:58:17: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl create scr grid
2015-02-11 00:58:17:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_GPNPSETUP
2015-02-11 00:58:17: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_GPNPSETUP
2015-02-11 00:58:17: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_GPNPSETUP '
2015-02-11 00:58:17: Removing file /tmp/filevuHPep
2015-02-11 00:58:17: Successfully removed file: /tmp/filevuHPep
2015-02-11 00:58:17: /bin/su successfully executed
2015-02-11 00:58:17: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_GPNPSETUP -status
2015-02-11 00:58:17: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_GPNPSETUP -status '
2015-02-11 00:58:17: Removing file /tmp/filegPte2X
2015-02-11 00:58:17: Successfully removed file: /tmp/filegPte2X
2015-02-11 00:58:17: /bin/su successfully executed
2015-02-11 00:58:17: The 'ROOTCRS_GPNPSETUP' status is SUCCESS
2015-02-11 00:58:17: 'ROOTCRS_GPNPSETUP' state is SUCCESS
2015-02-11 00:58:17: Local GPNP is already initialized
2015-02-11 00:58:17: Pin process list is osysmond.bin,ologgerd,ocssd.bin,cssdmonitor,cssdagent,mdb_pmon_-MGMTDB,kswapd0
2015-02-11 00:58:17:   copy "/u01/app/12.1.0/grid_1/crf/admin/crfdbrac1.ora" => "/tmp/DRDpfvHuaC"
2015-02-11 00:58:17: Copied existing orafile
2015-02-11 00:58:17: Creating CHM/OS config file /u01/app/12.1.0/grid_1/crf/admin/crfdbrac1.ora
2015-02-11 00:58:17: Trying to check if new CRS stack is partially up
2015-02-11 00:58:17: check new crs stack
2015-02-11 00:58:17: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl check resource ora.cssd -init
2015-02-11 00:58:17: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.cssd -init
2015-02-11 00:58:18: Checking the status of ora.cssd
2015-02-11 00:58:23: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.cssd -init
2015-02-11 00:58:23: Checking the status of ora.cssd
2015-02-11 00:58:28: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.cssd -init
2015-02-11 00:58:28: Checking the status of ora.cssd
2015-02-11 00:58:33: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl check resource ora.crsd -init
2015-02-11 00:58:33: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.crsd -init
2015-02-11 00:58:33: Checking the status of ora.crsd
2015-02-11 00:58:38: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.crsd -init
2015-02-11 00:58:38: Checking the status of ora.crsd
2015-02-11 00:58:43: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.crsd -init
2015-02-11 00:58:44: Checking the status of ora.crsd
2015-02-11 00:58:49: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl check resource ora.evmd -init
2015-02-11 00:58:49: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.evmd -init
2015-02-11 00:58:49: Checking the status of ora.evmd
2015-02-11 00:58:54: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.evmd -init
2015-02-11 00:58:54: Checking the status of ora.evmd
2015-02-11 00:58:59: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl status resource ora.evmd -init
2015-02-11 00:58:59: Checking the status of ora.evmd
2015-02-11 00:59:04: Oracle Grid Infrastructure is not running from /u01/app/12.1.0/grid_1
2015-02-11 00:59:04: Deleting older CHM/OS repository at
2015-02-11 00:59:04:   copy "/tmp/DRDpfvHuaC" => "/u01/app/12.1.0/grid_1/crf/admin/crfdbrac1.ora"
2015-02-11 00:59:04: Check and delete older IPD/OS installation
2015-02-11 00:59:04: INFO: The OS Tool is not installed at /usr/lib/oracrf.
2015-02-11 00:59:04: Registering ohasd
2015-02-11 00:59:04:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OHASD
2015-02-11 00:59:04: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OHASD
2015-02-11 00:59:04: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OHASD '
2015-02-11 00:59:05: Removing file /tmp/fileF65ZJD
2015-02-11 00:59:05: Successfully removed file: /tmp/fileF65ZJD
2015-02-11 00:59:05: /bin/su successfully executed
2015-02-11 00:59:05: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OHASD -status
2015-02-11 00:59:05: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_OHASD -status '
2015-02-11 00:59:05: Removing file /tmp/filewDoW1j
2015-02-11 00:59:05: Successfully removed file: /tmp/filewDoW1j
2015-02-11 00:59:05: /bin/su successfully executed
2015-02-11 00:59:05: The 'ROOTCRS_OHASD' status is SUCCESS
2015-02-11 00:59:05: 'ROOTCRS_OHASD' state is SUCCESS
2015-02-11 00:59:05: ohasd already registered
2015-02-11 00:59:05: Starting ohasd
2015-02-11 00:59:05: Checking the status of ohasd
2015-02-11 00:59:05: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl check has
2015-02-11 00:59:05: ohasd is already running
2015-02-11 00:59:05: Checking the status of ohasd
2015-02-11 00:59:05: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl check has
2015-02-11 00:59:05: ohasd started successfully
2015-02-11 00:59:05: Creating OHASD resources and dependencies
2015-02-11 00:59:05: Configuring HASD
2015-02-11 00:59:05:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_INITRES
2015-02-11 00:59:05: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_INITRES
2015-02-11 00:59:05: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_INITRES '
2015-02-11 00:59:05: Removing file /tmp/fileral9N1
2015-02-11 00:59:05: Successfully removed file: /tmp/fileral9N1
2015-02-11 00:59:05: /bin/su successfully executed
2015-02-11 00:59:05: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_INITRES -status
2015-02-11 00:59:05: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_INITRES -status '
2015-02-11 00:59:06: Removing file /tmp/fileL9iNaK
2015-02-11 00:59:06: Successfully removed file: /tmp/fileL9iNaK
2015-02-11 00:59:06: /bin/su successfully executed
2015-02-11 00:59:06: The 'ROOTCRS_INITRES' status is SUCCESS
2015-02-11 00:59:06: 'ROOTCRS_INITRES' state is SUCCESS
2015-02-11 00:59:06: OHASD Resources are already configured
2015-02-11 00:59:06: Successfully created OHASD resources for cluster and ASM
2015-02-11 00:59:06:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_AFDINST
2015-02-11 00:59:06: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_AFDINST
2015-02-11 00:59:06: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_AFDINST '
2015-02-11 00:59:06: Removing file /tmp/fileqVR08s
2015-02-11 00:59:06: Successfully removed file: /tmp/fileqVR08s
2015-02-11 00:59:06: /bin/su successfully executed
2015-02-11 00:59:06: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_AFDINST -status
2015-02-11 00:59:06: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_AFDINST -status '
2015-02-11 00:59:06: Removing file /tmp/fileOqJoHc
2015-02-11 00:59:06: Successfully removed file: /tmp/fileOqJoHc
2015-02-11 00:59:06: /bin/su successfully executed
2015-02-11 00:59:06: The 'ROOTCRS_AFDINST' status is SUCCESS
2015-02-11 00:59:06:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_ACFSINST
2015-02-11 00:59:06: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_ACFSINST
2015-02-11 00:59:06: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_ACFSINST '
2015-02-11 00:59:06: Removing file /tmp/file1r4ZQW
2015-02-11 00:59:06: Successfully removed file: /tmp/file1r4ZQW
2015-02-11 00:59:06: /bin/su successfully executed
2015-02-11 00:59:06: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_ACFSINST -status
2015-02-11 00:59:06: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_ACFSINST -status '
2015-02-11 00:59:06: Removing file /tmp/filePXzECH
2015-02-11 00:59:06: Successfully removed file: /tmp/filePXzECH
2015-02-11 00:59:06: /bin/su successfully executed
2015-02-11 00:59:06: The 'ROOTCRS_ACFSINST' status is SUCCESS
2015-02-11 00:59:06: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl stop crs -f
2015-02-11 00:59:09: Command output:
> CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbrac1'
> CRS-2673: Attempting to stop 'ora.evmd' on 'dbrac1'
> CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'dbrac1'
> CRS-2673: Attempting to stop 'ora.mdnsd' on 'dbrac1'
> CRS-2673: Attempting to stop 'ora.gipcd' on 'dbrac1'
> CRS-2673: Attempting to stop 'ora.gpnpd' on 'dbrac1'
> CRS-2677: Stop of 'ora.cssdmonitor' on 'dbrac1' succeeded
> CRS-2677: Stop of 'ora.gipcd' on 'dbrac1' succeeded
> CRS-2677: Stop of 'ora.evmd' on 'dbrac1' succeeded
> CRS-2677: Stop of 'ora.mdnsd' on 'dbrac1' succeeded
> CRS-2677: Stop of 'ora.gpnpd' on 'dbrac1' succeeded
> CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dbrac1' has completed
> CRS-4133: Oracle High Availability Services has been stopped.
>End Command output
2015-02-11 00:59:09: The return value of stop of CRS: 0
2015-02-11 00:59:09: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl check crs
2015-02-11 00:59:09: Command output:
> CRS-4639: Could not contact Oracle High Availability Services
>End Command output
2015-02-11 00:59:09: Oracle CRS stack has been shut down
2015-02-11 00:59:09:      ckpt: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_BOOTCFG
2015-02-11 00:59:09: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_BOOTCFG
2015-02-11 00:59:09: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_BOOTCFG '
2015-02-11 00:59:10: Removing file /tmp/file9QLcAB
2015-02-11 00:59:10: Successfully removed file: /tmp/file9QLcAB
2015-02-11 00:59:10: /bin/su successfully executed
2015-02-11 00:59:10: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_BOOTCFG -status
2015-02-11 00:59:10: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_BOOTCFG -status '
2015-02-11 00:59:10: Removing file /tmp/fileMeQD8v
2015-02-11 00:59:10: Successfully removed file: /tmp/fileMeQD8v
2015-02-11 00:59:10: /bin/su successfully executed
2015-02-11 00:59:10: The 'ROOTCRS_BOOTCFG' status is SUCCESS
2015-02-11 00:59:10: 'ROOTCRS_BOOTCFG' state is SUCCESS
2015-02-11 00:59:10: Node specific initial boot configuration already completed
2015-02-11 00:59:10: Startup level is 11
2015-02-11 00:59:10: Attempt to start the whole CRS stack
2015-02-11 00:59:10: Executing cmd: /u01/app/12.1.0/grid_1/bin/crsctl start crs -wait
2015-02-11 01:09:57: Command output:
> CRS-4123: Starting Oracle High Availability Services-managed resources
> CRS-2672: Attempting to start 'ora.mdnsd' on 'dbrac1'
> CRS-2672: Attempting to start 'ora.evmd' on 'dbrac1'
> CRS-2676: Start of 'ora.mdnsd' on 'dbrac1' succeeded
> CRS-2676: Start of 'ora.evmd' on 'dbrac1' succeeded
> CRS-2672: Attempting to start 'ora.gpnpd' on 'dbrac1'
> CRS-2676: Start of 'ora.gpnpd' on 'dbrac1' succeeded
> CRS-2672: Attempting to start 'ora.gipcd' on 'dbrac1'
> CRS-2676: Start of 'ora.gipcd' on 'dbrac1' succeeded
> CRS-2672: Attempting to start 'ora.cssdmonitor' on 'dbrac1'
> CRS-2676: Start of 'ora.cssdmonitor' on 'dbrac1' succeeded
> CRS-2672: Attempting to start 'ora.cssd' on 'dbrac1'
> CRS-2672: Attempting to start 'ora.diskmon' on 'dbrac1'
> CRS-2676: Start of 'ora.diskmon' on 'dbrac1' succeeded
> CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'dbrac1'
> CRS-2883: Resource 'ora.cssdmonitor' failed during Clusterware stack start.
> CRS-4406: Oracle High Availability Services synchronous start failed.
> CRS-4000: Command Start failed, or completed with errors.
>End Command output
2015-02-11 01:09:57: The return value of blocking start of CRS: 1
2015-02-11 01:09:57: Failed to start the Oracle CRS stack
2015-02-11 01:09:57: Executing cmd: /u01/app/12.1.0/grid_1/bin/clsecho -p has -f clsrsc -m 117
2015-02-11 01:09:57: Command output:
> CLSRSC-117: Failed to start Oracle Clusterware stack
>End Command output
2015-02-11 01:09:57: Executing cmd: /u01/app/12.1.0/grid_1/bin/clsecho -p has -f clsrsc -m 117
2015-02-11 01:09:57: Command output:
> CLSRSC-117: Failed to start Oracle Clusterware stack
>End Command output
2015-02-11 01:09:57: CLSRSC-117: Failed to start Oracle Clusterware stack
2015-02-11 01:09:57: ###### Begin DIE Stack Trace ######
2015-02-11 01:09:57:     Package         File                 Line Calling
2015-02-11 01:09:57:     --------------- -------------------- ---- ----------
2015-02-11 01:09:57: 1: main            rootcrs.pl            211 crsutils::dietrap
2015-02-11 01:09:57: 2: crsinstall      crsinstall.pm         941 main::__ANON__
2015-02-11 01:09:57: 3: crsinstall      crsinstall.pm         857 crsinstall::start_cluster
2015-02-11 01:09:57: 4: crsinstall      crsinstall.pm         281 crsinstall::prepare_to_start_clusterware
2015-02-11 01:09:57: 5: crsinstall      crsinstall.pm         219 crsinstall::CRSInstall
2015-02-11 01:09:57: 6: main            rootcrs.pl            334 crsinstall::new
2015-02-11 01:09:57: ####### End DIE Stack Trace #######
2015-02-11 01:09:57: ROOTCRS_STACK checkpoint has failed
2015-02-11 01:09:57: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_STACK -state FAIL
2015-02-11 01:09:57: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_STACK -state FAIL '
2015-02-11 01:09:57: Removing file /tmp/fileJKiQVi
2015-02-11 01:09:57: Successfully removed file: /tmp/fileJKiQVi
2015-02-11 01:09:57: /bin/su successfully executed
2015-02-11 01:09:57: Succeeded in writing the checkpoint:'ROOTCRS_STACK' with status:FAIL
2015-02-11 01:09:57: Running as user grid: /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_STACK -state FAIL
2015-02-11 01:09:57: s_run_as_user2: Running /bin/su grid -c ' /u01/app/12.1.0/grid_1/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_STACK -state FAIL '
2015-02-11 01:09:58: Removing file /tmp/file3h3DN6
2015-02-11 01:09:58: Successfully removed file: /tmp/file3h3DN6
2015-02-11 01:09:58: /bin/su successfully executed
2015-02-11 01:09:58: Succeeded in writing the checkpoint:'ROOTCRS_STACK' with status:FAIL

Veritas Cluster (SFRAC) and oracle CRS

We have oracle 9i RAC running on Solaris 9 with over veritas cluster.
We want to run 10g RAC on the same same servers which will use the
veritas cluster file system
While trying to install 10g CRS the node selection list doesnt show node
information.Entered information using response file .After the installation
root.sh fails to start CRS.
Is it ok to run 9i and 10g RAC on same servers.

Chandra/Vishwa
We have a libskgxn2.so in /opt/ORCLcluster/lib directory. But the size of the file is
different from /opt/VRTSvcs/rac/lib/libskgxn2_64.so.1 file
Since we already have 9i rac running, I am not sure if copying the file from
/opt/VRTSvcs/rac/lib directory if it will affect 9i rac.
I went ahead with installation of CRS. I am getting following error while running root.sh
Checking to see if Oracle CRS stack is already configured
Checking to see if any 9i GSD is up
/oracle/crshome/AUTEST/bin/lsdb: Failed to initialize Cluster Context
skgxn error number 1311719766
operation skgxnqtsz
location SKGXN not av
errno 0: Error 0
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
clscfg -install -nn nodeA,nodeAnum,nodeB,nodeBnum... -o crshome
-l languageid -c clustername -q votedisk
[-t p1,p2,p3,p4] [-pn privA,privAnum,privB,privBnum...]
[-hn hostA,hostAnum,hostB,hostBnum...]
With CRS and veritas running on the same server , will CRS always communicate
across nodes using VCS since we will be using veritas cluster file system
for database.
Shankar

Failure at final check of Oracle CRS stack. 10 on the first node.

Similar Messages

Maybe you are looking for