Is RAC node configuration when disk array fails on one node .

Question

Hi ,
We recently had all the filesystem of node 1 of RAC cluster , turned into read only mode. Upon further investigation it was revealed that it was due to disks array failure on node 1 . The database instance on node 2 is up and running fine . The OS team are rebuilding the node 1 from scratch and will restore oracle installable from the backup .
My question is once all files are restored :
Do we need to add the node to the RAC configuration ?
Do we need to do relink of oracle binary files ?
Can the node be brought up directly once all the oracle installables are restored properly or will the oRacle team require to perform addition steps to bring the node into RAC configuration .Thanks,
Sachin K

Answer 1

Hi ,
If the restore fails in some way . We will require to first remove and then add the nodes to the node 1 cluster right ? Kindly confirm on the below steps.
In case of such situation below are the steps we plan to follow:
version ; 10.2.0.5
Affected node :prd_node1
Affected instance :PRDB1
Surviving Node :prd_node2
Surviving instance: PRDB2
DB Listener on prd_node1:LISTENER_PRD01
ASM listener on prd_node1:LISTENER_PRDASM01
DB Listener on prd_node2:LISTENER_PRD02
ASM listener on prd_node2:LISTENER_PRDASM02
Login to the surviving node .In our case its prd_node2
Step 1 - Remove ONS information :
Execute as root the following command to find out the remote port number to be used
$cat $CRS_HOME/opmn/conf/ons.config
and remove the information pertaining the node to be deleted using
#$CRS_HOME/bin/racgons remove_config prd_node1:6200
Step 2 - Remove resources :
In this step, the resources that were defined on this node has to be removed. These resources include (a) Database (b) Instance (c) ASM. A list of this can
be acquired by running crs_stat -t command from any node
The srvctl remove listener command used below is only applicable in 10204 and higher releases including 11.1.0.6. The command will report an error if the
clusterware version is less than 10204. If clusterware version is less than 10204, use netca to remove the listener
srvctl remove listener -n prd_node1 -l LISTENER_PRD01
srvctl remove listener -n prd_node1 -l LISTENER_PRDASM01
srvctl remove instance -d PRDB -i PRDB1
srvctl remove asm -n prd_node1 -i +ASM1
Step 3 Execute rootdeletenode.sh :
From the node that you are not deleting execute as root the following command which will help find out the node number of the node that you want to delete
#$CRS_HOME/bin/olsnodes -n
this number can be passed to the rootdeletenode.sh command which is to be executed as root from any node which is going to remain in the cluster.
#$CRS_HOME/install/rootdeletenode.sh prd_node1,1
Step 5 Update the Inventory :
From the node which is going to remain in the cluster run the following command as owner of the CRS_HOME. The argument to be passed to the CLUSTER_NODES is a
comma seperated list of node names of the cluster which are going to remain in the cluster. This step needs to be performed from once per home (Clusterware,
ASM and RDBMS homes).
## Example of running runInstaller to update inventory in Clusterware home
$CRS_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ORA_CRS_HOME "CLUSTER_NODES=prd_node2" CRS=TRUE
## Optionally enclose the host names with {}
## Example of running runInstaller to update inventory in ASM home
$CRS_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ASM_HOME "CLUSTER_NODES=prd_node2"
## Optionally enclose the host names with {}
## Example of running runInstaller to update inventory in RDBMS home
$CRS_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=prd_node2"
## Optionally enclose the host names with {}
We need steps to add the node back into the cluster . Can anyone please help us on this ?
Thanks,
Sachin K