Oracle 11gR2 RAC - Recover Crashed node

Question

We have Oracle 11gR2 (11.2.0.3) RAC (2-node) on RHEL5.
One of the node in this cluster has been crashed, and we will have to rebuilt that server from the scratch, we don't have other choice. All our instances are running on node-1.
In this case how do we rebuilt the second node and instances without affecting the database in node-1. Any input would be appreciated.
Edited by: MR139 on May 23, 2012 11:44 AM

Answer 1

Balazs
I followed the doc (http://docs.oracle.com/cd/E11882_01/rac.112/e16794/adddelclusterware.htm#CHDFIAIE) to add node to the cluster. And it was successful. In the last step when i ran root.sh script on the new node I got the problem. Its again creating +ASM1 instance and it saying cannot create a diskgroup.
[root@hublpr2 11.2.0.3]# ./root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /app/grid/product/11.2.0.3
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /app/grid/product/11.2.0.3/crs/install/crsconfig_params
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'hublpr2'
CRS-2676: Start of 'ora.cssdmonitor' on 'hublpr2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'hublpr2'
CRS-2672: Attempting to start 'ora.diskmon' on 'hublpr2'
CRS-2676: Start of 'ora.diskmon' on 'hublpr2' succeeded
CRS-2676: Start of 'ora.cssd' on 'hublpr2' succeeded
Disk Group MDMPVTD01 creation failed with the following message:
ORA-15018: diskgroup cannot be created
ORA-15031: disk specification 'ORCL:MDMPVDT03' matches no disks
ORA-15031: disk specification 'ORCL:MDMPVDT02' matches no disks
ORA-15031: disk specification 'ORCL:MDMPVDT01' matches no disks
Configuration of ASM ... failed
see asmca logs at /app/oracle/cfgtoollogs/asmca for details
Did not succssfully configure and start ASM at /app/grid/product/11.2.0.3/crs/install/crsconfig_lib.pm line 6763.
/app/grid/product/11.2.0.3/perl/bin/perl -I/app/grid/product/11.2.0.3/perl/lib -I/app/grid/product/11.2.0.3/crs/install /app/grid/product/11.2.0.3/crs/install/rootcrs.pl execution failed