Root.sh on second node fails

I am running Linux 64bit. Installing Oracle 11gR2.
It passed all the per-requisite. Run root.sh on first node. It finished with no errorrs.
On second node I got the following:
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-07-13 12:51:28: Parsing the host name
2010-07-13 12:51:28: Checking for super user privileges
2010-07-13 12:51:28: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node fred0224, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'fred0225'
CRS-2676: Start of 'ora.mdnsd' on 'fred0225' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'fred0225'
CRS-2676: Start of 'ora.gipcd' on 'fred0225' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'fred0225'
CRS-2676: Start of 'ora.gpnpd' on 'fred0225' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fred0225'
CRS-2676: Start of 'ora.cssdmonitor' on 'fred0225' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'fred0225'
CRS-2672: Attempting to start 'ora.diskmon' on 'fred0225'
CRS-2676: Start of 'ora.diskmon' on 'fred0225' succeeded
CRS-2676: Start of 'ora.cssd' on 'fred0225' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'fred0225'
Start action for octssd aborted
CRS-2676: Start of 'ora.ctssd' on 'fred0225' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'fred0225'
CRS-2672: Attempting to start 'ora.asm' on 'fred0225'
CRS-2676: Start of 'ora.drivers.acfs' on 'fred0225' succeeded
CRS-2676: Start of 'ora.asm' on 'fred0225' succeeded
CRS-2664: Resource 'ora.ctssd' is already running on 'fred0225'
CRS-4000: Command Start failed, or completed with errors.
Command return code of 1 (256) from command: /u01/app/11.2.0/grid/bin/crsctl start resource ora.asm -init
Start of resource "ora.asm -init" failed
Failed to start ASM
Failed to start Oracle Clusterware stack
In the ocssd.log I found
[    CSSD][3559689984]clssnmvDHBValidateNCopy: node 1, fred0224, has a disk HB, but no network HB, DHB has rcfg 174483948, wrtcnt, 232, LATS 521702664, lastSeqNo 232, uniqueness 1279039649, timestamp 1279039959/521874274
In oraagent_oracle.log I found
[  clsdmc][1212365120]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GPNPD)) with status 9
2010-07-13 12:54:07.234: [ora.gpnpd][1212365120] [check] Error = error 9 encountered when connecting to GPNPD
2010-07-13 12:54:07.238: [ora.gpnpd][1212365120] [check] Calling PID check for daemon
2010-07-13 12:54:07.238: [ora.gpnpd][1212365120] [check] Trying to check PID = 20584
2010-07-13 12:54:07.432: [ COMMCRS][1285794112]clsc_connect: (0x1304d850) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GPNPD))
[  clsdmc][1222854976]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_MDNSD)) with status 9
2010-07-13 12:54:08.649: [ora.mdnsd][1222854976] [check] Error = error 9 encountered when connecting to MDNSD
2010-07-13 12:54:08.649: [ora.mdnsd][1222854976] [check] Calling PID check for daemon
2010-07-13 12:54:08.649: [ora.mdnsd][1222854976] [check] Trying to check PID = 20571
2010-07-13 12:54:08.841: [ COMMCRS][1201875264]clsc_connect: (0x12f3b1d0) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_MDNSD))
[  clsdmc][1159915840]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GIPCD)) with status 9
2010-07-13 12:54:10.051: [ora.gipcd][1159915840] [check] Error = error 9 encountered when connecting to GIPCD
2010-07-13 12:54:10.051: [ora.gipcd][1159915840] [check] Calling PID check for daemon
2010-07-13 12:54:10.051: [ora.gipcd][1159915840] [check] Trying to check PID = 20566
2010-07-13 12:54:10.242: [ COMMCRS][1254324544]clsc_connect: (0x12f35630) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GIPCD))
In oracssdagent_root.log I found
2010-07-13 12:52:28.698: [ CSSCLNT][1102481728]clssscConnect: gipc request failed with 29 (0x16)
2010-07-13 12:52:28.698: [ CSSCLNT][1102481728]clsssInitNative: connect failed, rc 29
2010-07-13 12:53:55.222: [ CSSCLNT][1102481728]clssnsqlnum: RPC failed rc 3
2010-07-13 12:53:55.222: [ USRTHRD][1102481728] clsnomon_cssini: failed 3 to fetch node number
2010-07-13 12:53:55.222: [ USRTHRD][1102481728] clsnomon_init: css init done, nodenum -1.
2010-07-13 12:53:55.222: [ CSSCLNT][1102481728]clsssRecvMsg: got a disconnect from the server while waiting for message type 43
2010-07-13 12:53:55.222: [ CSSCLNT][1102481728]clsssGetNLSData: Failure receiving a msg, rc 3
If anyone needs more info please let me know.

On all nodes,
1. Modify the /etc/sysconfig/oracleasm with:
ORACLEASM_SCANORDER="dm"
ORACLEASM_SCANEXCLUDE="sd"
2. restart the asmlib by :
# /etc/init.d/oracleasm restart
3. Run root.sh on the 2nd node
hope this helps you

Similar Messages

  • Error while executing root.sh on second node: Failed 3 to bind listening ..

    (root@slpas20)[oracle/crshome/crs11g/css/admin] /oracle/crshome/crs11g/root.sh
    Checking to see if Oracle CRS stack is already configured
    Checking to see if any 9i GSD is up
    Setting the permissions on OCR backup directory
    Setting up Network socket directories
    Oracle Cluster Registry configuration upgraded successfully
    clscfg: EXISTING configuration version 4 detected.
    clscfg: version 4 is 11 Release 1.
    Successfully accumulated necessary OCR keys.
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 0: slpas04 slpas04_priv slpas04
    node 1: slpas20 slpas20_priv slpas20
    clscfg: Arguments check out successfully.
    NO KEYS WERE WRITTEN. Supply -force parameter to override.
    -force is destructive and will destroy any previous cluster
    configuration.
    Oracle Cluster Registry for cluster has already been initialized
    Failed 3 to bind listening endpoint: (ADDRESS=(PROTOCOL=tcp)(HOST=slpas20_priv))
    CRS daemons not set to start.
    (root@slpas20)[oracle/crshome/crs11g/css/admin]

    Hi Anand
    Can you please control the following points please?
    That on node1 in the /etc/hosts file the name of the private interconnect exist. But on node2 or the failing node it's missing the private interconnect in the /etc/hosts file. That should be the reason why the issue occurs.
    Solution of The Problem might be to :
    1. Add the missing hostnames to /etc/hosts
    2. Stop and start CRS stack
    Please confirm that this solves your issue.
    thanks

  • Error after running root.sh on second node

    hi,
    i have installed clusterware on a 2 node system running on RHEL 5
    i followed the prereqs , and solved all the errors i encountered
    after clusterware installation , it asks to run root.sh on all the nodes
    when i ran root.sh on second node
    it gave this error
    Running vipca(silent) for configuring nodeapps
    /home/oracle/crs/oracle/product/10/crs/jdk/jre//bin/java: error while loading
    shared libraries: libpthread.so.0: cannot open shared object file:
    No such file or directory
    so i follwed metalink note 414163.1
    after that i called it a day
    in the morning , wheni started both nodes
    and started vipca on second node
    it gave this error
    PRKH:1010 unable to communicate with crs services
    the i ran ps-ef | grep crs
    root 3201 1 0 15:37 ? 00:00:00 /bin/sh /etc/init.d/init.crsd run
    crsctl check crs gave
    failure 1 contacting css daemon
    cannot communicate with crs
    cannot communicate with evm
    what should i do to start these services?

    crsd and cssd logs were empty and there was no relevant info in crs alert
    i am just reinstalling clusterware now
    one thing i wanted to ask
    why does owner ship of raw files change back to root (after node restart)
    even though i chnged them to oracle

  • Error CLSRSC-507 during the execution of root.sh on second node

    Hi all.
       OS.......: Red-Hat 6.5
       RDBMS: Oracle 12.1.0.2.0
       During the installation of a 2-node RAC in a RHEL 6.5, during the execution of the root.sh script in the second node, i get the following error:
    [root@oraprd02 grid]# ./root.sh
    Performing root user operation.
    The following environment variables are set as:
        ORACLE_OWNER= grid
        ORACLE_HOME=  /u01/app/12.1.0/grid
    Enter the full pathname of the local bin directory: [/usr/local/bin]:
       Copying dbhome to /usr/local/bin ...
       Copying oraenv to /usr/local/bin ...
       Copying coraenv to /usr/local/bin ...
    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params
    2015/05/04 22:47:16 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
    2015/05/04 22:47:59 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
    2015/05/04 22:48:00 CLSRSC-363: User ignored prerequisites during installation
    OLR initialization - successful
    2015/05/04 22:48:46 CLSRSC-507: The root script cannot proceed on this node oraprd02 because either the first-node operations have not completed on node oraprd01 or there was an error in obtaining the status of the first-node operations.
    Died at /u01/app/12.1.0/grid/crs/install/crsutils.pm line 3681.
    The command '/u01/app/12.1.0/grid/perl/bin/perl -I/u01/app/12.1.0/grid/perl/lib -I/u01/app/12.1.0/grid/crs/install /u01/app/12.1.0/grid/crs/install/rootcrs.pl ' execution failed
    The root.sh on the first node completed successfully. I get the succeeded message from the script in the first node.
    Have anyone faced this problem? Any assistance will be most helpfull.
    Thanks in advance.

    crsd and cssd logs were empty and there was no relevant info in crs alert
    i am just reinstalling clusterware now
    one thing i wanted to ask
    why does owner ship of raw files change back to root (after node restart)
    even though i chnged them to oracle

  • Root.sh failed on second node while installing CRS 10g on centos 5.5

    root.sh failed on second node while installing CRS 10g
    Hi all,
    I am able to install Oracle 10g RAC clusterware on first node of the cluster. However, when I run the root.sh script as root
    user on second node of the cluster, it fails with following error message:
    NO KEYS WERE WRITTEN. Supply -force parameter to override.
    -force is destructive and will destroy any previous cluster
    configuration.
    Oracle Cluster Registry for cluster has already been initialized
    Startup will be queued to init within 90 seconds.
    Adding daemons to inittab
    Expecting the CRS daemons to be up within 600 seconds.
    Failure at final check of Oracle CRS stack.
    10
    and run cluvfy stage -post hwos -n all -verbose,it show message:
    ERROR:
    Could not find a suitable set of interfaces for VIPs.
    Result: Node connectivity check failed.
    Checking shared storage accessibility...
    Disk Sharing Nodes (2 in count)
    /dev/sda db2 db1
    and run cluvfy stage -pre crsinst -n all -verbose,it show message:
    ERROR:
    Could not find a suitable set of interfaces for VIPs.
    Result: Node connectivity check failed.
    Checking system requirements for 'crs'...
    No checks registered for this product.
    and run cluvfy stage -post crsinst -n all -verbose,it show message:
    Result: Node reachability check passed from node "DB2".
    Result: User equivalence check passed for user "oracle".
    Node Name CRS daemon CSS daemon EVM daemon
    db2 no no no
    db1 yes yes yes
    Check: Health of CRS
    Node Name CRS OK?
    db1 unknown
    Result: CRS health check failed.
    check crsd.log and show message:
    clsc_connect: (0x143ca610) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_db2_crs))
    clsssInitNative: connect failed, rc 9
    Any help would be greatly appreciated.
    Edited by: 868121 on 2011-6-24 上午12:31

    Hello, it took a little searching, but I found this in a note in the GRID installation guide for Linux/UNIX:
    Public IP addresses and virtual IP addresses must be in the same subnet.
    In your case, you are using two different subnets for the VIPs.

  • 11gR2 RAC install fail when running root.sh script on second node

    I get the errors:
    ORA-15018: diskgroup cannot be created
    ORA-15072: command requires at least 2 regular failure groups, discovered only 0
    ORA-15080: synchronous I/O operation to a disk failed
    [main] [ 2012-04-10 16:44:12.564 EDT ] [UsmcaLogger.logException:175] oracle.sysman.assistants.util.sqlEngine.SQLFatalErrorException: ORA-15018: diskgroup cannot be created
    ORA-15072: command requires at least 2 regular failure groups, discovered only 0
    ORA-15080: synchronous I/O operation to a disk failed
    I have tried the fix solutions from metalink note, but did not fix issue
    11GR2 GRID INFRASTRUCTURE INSTALLATION FAILS WHEN RUNNING ROOT.SH ON NODE 2 OF RAC USING ASMLIB [ID 1059847.1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

    Hi,
    it looks like, that your "shared device" you are using is not really shared.
    The second node does "create an ASM diskgroup" and create OCR and Voting disks. If this indeed would be a shared device, he should have recognized, that your disk is shared.
    So as a result your VMware configuration must be wrong, and the disk you presented as shared disk is not really shared.
    Which VMWare version did you use? It will not work correctly with the workstation or player edition, since shared disks are only really working with the server version.
    If you indeed using the server, could you paste your vm configurations?
    Furthermore I recommend using Virtual Box. There is a nice how-to:
    http://www.oracle-base.com/articles/11g/OracleDB11gR2RACInstallationOnOEL5UsingVirtualBox.php
    Sebastian

  • Root.sh failed at second node OUL 6.3 Oracle GRID 11.2.0.3

    Hi, im installing a two node cluster mounted on Oracle Linux 6.3 with Oracle DB 11.2.0.3, the installation went smooth up until the execution of the root.sh script on the second node.
    THe script return this final lines:
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node nodo1, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    Start of resource "ora.crsd" failed
    CRS-2800: Cannot start resource 'ora.asm' as it is already in the INTERMEDIATE state on server 'nodo2'
    CRS-4000: Command Start failed, or completed with errors.
    Failed to start Oracle Grid Infrastructure stack
    Failed to start Cluster Ready Services at /u01/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 1286.
    /u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed
    In $GRID_HOME/log/node2/alertnode.log It appears to be a Cluster Time Synchronization Service issue, (i didn't synchronyze the nodes..) however the CTSS is running in observer mode, wich i believe it shouldn't affect the installation process. After that i lost it...there's an entry CRS-5018 indicating that an unused HAIP route was removed... and then, out of the blue: CRS-5818:Aborted command 'start' for resource 'ora.asm'. Some clarification will be deeply apreciated.
    Here's the complete log:
    2013-04-01 13:39:35.358
    [client(12163)]CRS-2101:The OLR was formatted using version 3.
    2013-04-01 19:40:19.597
    [ohasd(12338)]CRS-2112:The OLR service started on node nodo2.
    2013-04-01 19:40:19.657
    [ohasd(12338)]CRS-1301:Oracle High Availability Service started on node nodo2.
    [client(12526)]CRS-10001:01-Apr-13 13:41 ACFS-9459: ADVM/ACFS is not supported on this OS version: '2.6.39-400.17.2.el6uek.i686'
    [client(12528)]CRS-10001:01-Apr-13 13:41 ACFS-9201: Not Supported
    [client(12603)]CRS-10001:01-Apr-13 13:41 ACFS-9459: ADVM/ACFS is not supported on this OS version: '2.6.39-400.17.2.el6uek.i686'
    2013-04-01 19:41:17.509
    [ohasd(12338)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
    2013-04-01 19:41:17.618
    [gpnpd(12695)]CRS-2328:GPNPD started on node nodo2.
    2013-04-01 19:41:21.363
    [cssd(12755)]CRS-1713:CSSD daemon is started in exclusive mode
    2013-04-01 19:41:23.194
    [ohasd(12338)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
    2013-04-01 19:41:56.144
    [cssd(12755)]CRS-1707:Lease acquisition for node nodo2 number 2 completed
    2013-04-01 19:41:57.545
    [cssd(12755)]CRS-1605:CSSD voting file is online: /dev/oracleasm/disks/ASM_DISK_1; details in /u01/app/11.2.0/grid/log/nodo2/cssd/ocssd.log.
    [cssd(12755)]CRS-1636:The CSS daemon was started in exclusive mode but found an active CSS daemon on node nodo1 and is terminating; details at (:CSSNM00006:) in /u01/app/11.2.0/grid/log/nodo2/cssd/ocssd.log
    2013-04-01 19:41:58.549
    [ohasd(12338)]CRS-2765:Resource 'ora.cssdmonitor' has failed on server 'nodo2'.
    2013-04-01 19:42:10.025
    [gpnpd(12695)]CRS-2329:GPNPD on node nodo2 shutdown.
    2013-04-01 19:42:11.407
    [mdnsd(12685)]CRS-5602:mDNS service stopping by request.
    2013-04-01 19:42:29.642
    [gpnpd(12947)]CRS-2328:GPNPD started on node nodo2.
    2013-04-01 19:42:33.241
    [cssd(13012)]CRS-1713:CSSD daemon is started in clustered mode
    2013-04-01 19:42:35.104
    [ohasd(12338)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
    2013-04-01 19:42:44.065
    [cssd(13012)]CRS-1707:Lease acquisition for node nodo2 number 2 completed
    2013-04-01 19:42:45.484
    [cssd(13012)]CRS-1605:CSSD voting file is online: /dev/oracleasm/disks/ASM_DISK_1; details in /u01/app/11.2.0/grid/log/nodo2/cssd/ocssd.log.
    2013-04-01 19:42:52.138
    [cssd(13012)]CRS-1601:CSSD Reconfiguration complete. Active nodes are nodo1 nodo2 .
    2013-04-01 19:42:55.081
    [ctssd(13076)]CRS-2403:The Cluster Time Synchronization Service on host nodo2 is in observer mode.
    2013-04-01 19:42:55.581
    [ctssd(13076)]CRS-2401:The Cluster Time Synchronization Service started on host nodo2.
    2013-04-01 19:42:55.581
    [ctssd(13076)]CRS-2407:The new Cluster Time Synchronization Service reference node is host nodo1.
    2013-04-01 19:43:08.875
    [ctssd(13076)]CRS-2412:The Cluster Time Synchronization Service detects that the local time is significantly different from the mean cluster time. Details in /u01/app/11.2.0/grid/log/nodo2/ctssd/octssd.log.
    2013-04-01 19:43:08.876
    [ctssd(13076)]CRS-2409:The clock on host nodo2 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode.
    2013-04-01 19:43:13.565
    [u01/app/11.2.0/grid/bin/orarootagent.bin(13064)]CRS-5018:(:CLSN00037:) Removed unused HAIP route: 169.254.0.0 / 255.255.0.0 / 0.0.0.0 / eth0
    2013-04-01 19:53:09.800
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5818:Aborted command 'start' for resource 'ora.asm'. Details at (:CRSAGF00113:) {0:0:223} in /u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log.
    2013-04-01 19:53:11.827
    [ohasd(12338)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.asm'. Details at (:CRSPE00111:) {0:0:223} in /u01/app/11.2.0/grid/log/nodo2/ohasd/ohasd.log.
    2013-04-01 19:53:12.779
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:53:13.892
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:53:43.877
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:54:13.891
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:54:43.906
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:55:13.914
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:55:43.918
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:56:13.922
    [u01/app/11.2.0/grid/bin/oraagent.bin(12922)]CRS-5019:All OCR locations are on ASM disk groups [DATA], and none of these disk groups are mounted. Details are at "(:CLSN00100:)" in "/u01/app/11.2.0/grid/log/nodo2/agent/ohasd/oraagent_oracle/oraagent_oracle.log".
    2013-04-01 19:56:53.209
    [crsd(13741)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 20:07:01.128
    [crsd(13741)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 20:07:01.278
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 20:07:08.689
    [crsd(15248)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 20:13:10.138
    [ctssd(13076)]CRS-2412:The Cluster Time Synchronization Service detects that the local time is significantly different from the mean cluster time. Details in /u01/app/11.2.0/grid/log/nodo2/ctssd/octssd.log.
    2013-04-01 20:17:13.024
    [crsd(15248)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 20:17:13.171
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 20:17:20.826
    [crsd(16746)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 20:27:25.020
    [crsd(16746)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 20:27:25.176
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 20:27:31.591
    [crsd(18266)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 20:37:35.668
    [crsd(18266)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 20:37:35.808
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 20:37:43.209
    [crsd(19762)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 20:43:11.160
    [ctssd(13076)]CRS-2412:The Cluster Time Synchronization Service detects that the local time is significantly different from the mean cluster time. Details in /u01/app/11.2.0/grid/log/nodo2/ctssd/octssd.log.
    2013-04-01 20:47:47.487
    [crsd(19762)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 20:47:47.637
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 20:47:55.086
    [crsd(21242)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 20:57:59.343
    [crsd(21242)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 20:57:59.492
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 20:58:06.996
    [crsd(22744)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 21:08:11.046
    [crsd(22744)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 21:08:11.192
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 21:08:18.726
    [crsd(24260)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 21:13:12.000
    [ctssd(13076)]CRS-2412:The Cluster Time Synchronization Service detects that the local time is significantly different from the mean cluster time. Details in /u01/app/11.2.0/grid/log/nodo2/ctssd/octssd.log.
    2013-04-01 21:18:22.262
    [crsd(24260)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 21:18:22.411
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 21:18:29.927
    [crsd(25759)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 21:28:34.467
    [crsd(25759)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 21:28:34.616
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 21:28:41.990
    [crsd(27291)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 21:38:45.012
    [crsd(27291)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 21:38:45.160
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 21:38:52.790
    [crsd(28784)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 21:43:12.378
    [ctssd(13076)]CRS-2412:The Cluster Time Synchronization Service detects that the local time is significantly different from the mean cluster time. Details in /u01/app/11.2.0/grid/log/nodo2/ctssd/octssd.log.
    2013-04-01 21:48:56.285
    [crsd(28784)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 21:48:56.435
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 21:49:04.421
    [crsd(30272)]CRS-1012:The OCR service started on node nodo2.
    2013-04-01 21:59:08.183
    [crsd(30272)]CRS-0810:Cluster Ready Service aborted due to failure to communicate with Event Management Service with error [1]. Details at (:CRSD00120:) in /u01/app/11.2.0/grid/log/nodo2/crsd/crsd.log.
    2013-04-01 21:59:08.318
    [ohasd(12338)]CRS-2765:Resource 'ora.crsd' has failed on server 'nodo2'.
    2013-04-01 21:59:15.860
    [crsd(31772)]CRS-1012:The OCR service started on node nodo2.

    Hi santysharma, thanks for the reply, i have two ethernet interfaces: eth0 (public network 192.168.1.0) and eth1 (private network 10.5.3.0), there is no device using that ip range, here's the output of route command:
    (Sorry for the alignment, i tried to tab it but the editor trims it again)
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    default 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
    private * 255.255.255.0 U 0 0 0 eth1
    link-local * 255.255.0.0 U 1002 0 0 eth0
    link-local * 255.255.0.0 U 1003 0 0 eth1
    public * 255.255.255.0 U 0 0 0 eth0
    And the /etc/hosts file
    127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
    10.5.3.1 nodo1.cluster nodo1
    10.5.3.2 nodo2.cluster nodo2
    192.168.1.13 cluster-scan
    192.168.1.14 nodo1-vip
    192.168.1.15 nodo2-vip
    And the ifconfig -a
    eth0 Link encap:Ethernet HWaddr C8:3A:35:D9:C6:2B
    inet addr:192.168.1.12 Bcast:192.168.1.255 Mask:255.255.255.0
    inet6 addr: fe80::ca3a:35ff:fed9:c62b/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:34708 errors:0 dropped:18 overruns:0 frame:0
    TX packets:24693 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:48545969 (46.2 MiB) TX bytes:1994381 (1.9 MiB)
    eth1 Link encap:Ethernet HWaddr 00:0D:87:D0:A3:8E
    inet addr:10.5.3.2 Bcast:10.5.3.255 Mask:255.255.255.0
    inet6 addr: fe80::20d:87ff:fed0:a38e/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:44 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 b) TX bytes:5344 (5.2 KiB)
    Interrupt:23 Base address:0x6000
    lo Link encap:Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:16436 Metric:1
    RX packets:20 errors:0 dropped:0 overruns:0 frame:0
    TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:1320 (1.2 KiB) TX bytes:1320 (1.2 KiB)
    Now that i'm thinking i've read somewhere that ipv6 was no supported...yet there's no relation with the 169.254.x.x ip range.

  • Root.sh fails on second node

    I already posted this issue on database installation forum, and was suggested to post it on this forum.
    Here are the details.
    I am running Linux 64bit on ESx clients. Installing Oracle 11gR2.
    It passed all the per-requisite. Run root.sh on first node. It finished with no errorrs.
    On second node I got the following:
    Running Oracle 11g root.sh script...
    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /u01/app/11.2.0/grid
    Enter the full pathname of the local bin directory: [usr/local/bin]:
    The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-07-13 12:51:28: Parsing the host name
    2010-07-13 12:51:28: Checking for super user privileges
    2010-07-13 12:51:28: User has super user privileges
    Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node fred0224, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    CRS-2672: Attempting to start 'ora.mdnsd' on 'fred0225'
    CRS-2676: Start of 'ora.mdnsd' on 'fred0225' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'fred0225'
    CRS-2676: Start of 'ora.gipcd' on 'fred0225' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'fred0225'
    CRS-2676: Start of 'ora.gpnpd' on 'fred0225' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fred0225'
    CRS-2676: Start of 'ora.cssdmonitor' on 'fred0225' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'fred0225'
    CRS-2672: Attempting to start 'ora.diskmon' on 'fred0225'
    CRS-2676: Start of 'ora.diskmon' on 'fred0225' succeeded
    CRS-2676: Start of 'ora.cssd' on 'fred0225' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'fred0225'
    Start action for octssd aborted
    CRS-2676: Start of 'ora.ctssd' on 'fred0225' succeeded
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'fred0225'
    CRS-2672: Attempting to start 'ora.asm' on 'fred0225'
    CRS-2676: Start of 'ora.drivers.acfs' on 'fred0225' succeeded
    CRS-2676: Start of 'ora.asm' on 'fred0225' succeeded
    CRS-2664: Resource 'ora.ctssd' is already running on 'fred0225'
    CRS-4000: Command Start failed, or completed with errors.
    Command return code of 1 (256) from command: /u01/app/11.2.0/grid/bin/crsctl start resource ora.asm -init
    Start of resource "ora.asm -init" failed
    Failed to start ASM
    Failed to start Oracle Clusterware stack
    In the ocssd.log I found
    [ CSSD][3559689984]clssnmvDHBValidateNCopy: node 1, fred0224, has a disk HB, but no network HB, DHB has rcfg 174483948, wrtcnt, 232, LATS 521702664, lastSeqNo 232, uniqueness 1279039649, timestamp 1279039959/521874274
    In oraagent_oracle.log I found
    [ clsdmc][1212365120]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GPNPD)) with status 9
    2010-07-13 12:54:07.234: [ora.gpnpd][1212365120] [check] Error = error 9 encountered when connecting to GPNPD
    2010-07-13 12:54:07.238: [ora.gpnpd][1212365120] [check] Calling PID check for daemon
    2010-07-13 12:54:07.238: [ora.gpnpd][1212365120] [check] Trying to check PID = 20584
    2010-07-13 12:54:07.432: [ COMMCRS][1285794112]clsc_connect: (0x1304d850) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GPNPD))
    [ clsdmc][1222854976]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_MDNSD)) with status 9
    2010-07-13 12:54:08.649: [ora.mdnsd][1222854976] [check] Error = error 9 encountered when connecting to MDNSD
    2010-07-13 12:54:08.649: [ora.mdnsd][1222854976] [check] Calling PID check for daemon
    2010-07-13 12:54:08.649: [ora.mdnsd][1222854976] [check] Trying to check PID = 20571
    2010-07-13 12:54:08.841: [ COMMCRS][1201875264]clsc_connect: (0x12f3b1d0) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_MDNSD))
    [ clsdmc][1159915840]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GIPCD)) with status 9
    2010-07-13 12:54:10.051: [ora.gipcd][1159915840] [check] Error = error 9 encountered when connecting to GIPCD
    2010-07-13 12:54:10.051: [ora.gipcd][1159915840] [check] Calling PID check for daemon
    2010-07-13 12:54:10.051: [ora.gipcd][1159915840] [check] Trying to check PID = 20566
    2010-07-13 12:54:10.242: [ COMMCRS][1254324544]clsc_connect: (0x12f35630) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=fred0225DBG_GIPCD))
    In oracssdagent_root.log I found
    2010-07-13 12:52:28.698: [ CSSCLNT][1102481728]clssscConnect: gipc request failed with 29 (0x16)
    2010-07-13 12:52:28.698: [ CSSCLNT][1102481728]clsssInitNative: connect failed, rc 29
    2010-07-13 12:53:55.222: [ CSSCLNT][1102481728]clssnsqlnum: RPC failed rc 3
    2010-07-13 12:53:55.222: [ USRTHRD][1102481728] clsnomon_cssini: failed 3 to fetch node number
    2010-07-13 12:53:55.222: [ USRTHRD][1102481728] clsnomon_init: css init done, nodenum -1.
    2010-07-13 12:53:55.222: [ CSSCLNT][1102481728]clsssRecvMsg: got a disconnect from the server while waiting for message type 43
    2010-07-13 12:53:55.222: [ CSSCLNT][1102481728]clsssGetNLSData: Failure receiving a msg, rc 3
    If you need more info, let me know.

    Well, the error clearly indicates that a communication problem exists on the private interconnect.
    Could this be a setting in ESX, which prevents some communication between the clients on the second network card? Any routing table in ESX not configured correctly?
    Sebastian

  • Oracle 11gR2 RAC Root.sh Failed On The Second Node

    Hello,
    When i installing Oracle 11gR2 RAC on AIX 7.1 , root.sh succeeds on first node but fails on the second node:
    I get error "Root.sh Failed On The Second Node With Error ORA-15018 ORA-15031 ORA-15025 ORA-27041 [ID 1459711.1]" within Oracle installation.
    Applies to:
    Oracle Server - 11gR2 RAC
    EMC VNX 500
    IBM AIX on POWER Systems (64-bit)
    in /dev/rhdiskpower0 does not show in kfod output on second node. It is an EMC multipath disk device.
    But the disk can be found with AIX command.
    any help!!
    Thanks

    the soluation that uninstall "EMC solutitons enabler" but in the machine i just find "EMC migration enabler" and conn't remove without remove EMC Powerpath.

  • Ora.asm -init failed on second node root.sh

    Hi All,
    Installing Grid Infrastructure for a 11gr2 Cluster on two nodes Oracle Linux 5 + Vsware vSphere v4, shared disk on same host machine. When run root.sh, first node was success but the second node got following error message (actually the first node was cloned from the seoncd):
    CRS-2672: Attempting to start 'ora.ctssd' on 'wandrac2'
    Start action for octssd aborted
    CRS-2676: Start of 'ora.ctssd' on 'wandrac2' succeeded
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'wandrac2'
    CRS-2672: Attempting to start 'ora.asm' on 'wandrac2'
    CRS-2676: Start of 'ora.drivers.acfs' on 'wandrac2' succeeded
    CRS-2676: Start of 'ora.asm' on 'wandrac2' succeeded
    CRS-2664: Resource 'ora.ctssd' is already running on 'wandrac2'
    CRS-4000: Command Start failed, or completed with errors.
    Command return code of 1 (256) from command: /orapp/racsl/11.2.0/bin/crsctl start resource ora.asm -init
    Start of resource "ora.asm -init" failed
    Failed to start ASM
    Failed to start Oracle Clusterware stack
    Thanks in advance for any information and helps,

    Hi,
    I came across this error and I am about to start a fresh installation of the grid. (ealier one failed because it was unable to read the memory in rac2 )
    Is there anything specific I can change before I start my installation.
    PS - I didnt get what exactly is going on with the hosts file.
    My files are as follows :
    RAC1 - etc/hosts
    [oracle@falcen6a ~]$ cat /etc/hosts
    # Do not remove the following line, or various programs
    # that require network functionality will fail.
    127.0.0.1 localhost.localdomain localhost
    ::1 localhost6.localdomain6 localhost6
    # Public
    192.168.100.218 falcen6a.a.pri falcen6a
    192.168.100.219 falcen6b.a.pri falcen6b
    # Private
    192.168.210.101 falcen6a-priv.a.pri falcen6a-priv
    192.168.210.102 falcen6b-priv.a.pri falcen6b-priv
    # Virtual
    192.168.100.212 falcen6a-vip.a.pri falcen6a-vip
    192.168.100.213 falcen6b-vip.a.pri falcen6b-vip
    # SCAN
    #192.168.100.208 falcen6-scan.a.pri falcen6-scan
    #192.168.100.209 falcen6-scan.a.pri falcen6-scan
    #192.168.100.210 falcen6-scan.a.pri falcen6-scan
    on RAC2
    [oracle@falcen6b ~]$ cat /etc/hosts
    # Do not remove the following line, or various programs
    # that require network functionality will fail.
    127.0.0.1 localhost.localdomain localhost
    ::1 localhost6.localdomain6 localhost6
    #Public
    192.168.100.218 falcen6a.a.pri falcen6a
    192.168.100.219 falcen6b.a.pri falcen6b
    # Private
    192.168.210.101 falcen6a-priv.a.pri falcen6a-priv
    192.168.210.102 falcen6b-priv.a.pri falcen6b-priv
    # Virtual
    192.168.100.212 falcen6a-vip.a.pri falcen6a-vip
    192.168.100.213 falcen6b-vip.a.pri falcen6b-vip
    # SCAN
    #192.168.100.208 falcen6-scan.a.pri falcen6-scan
    #192.168.100.209 falcen6-scan.a.pri falcen6-scan
    #192.168.100.210 falcen6-scan.a.pri falcen6-scan
    Can someone please confirm this??

  • 11gR2 clusterware installation problem on root.sh script on second node

    Hi all,
    I wanna install the *11gR2 RAC* on ORA-Linux 5.5 (x86_64) using VMware server but on the second node i get two "*failed*" at the end of root.sh script.
    After that i try to install DB but ı can see only one node.What is the problem...
    I will send the output, ı need your help.
    Thank you all for helping..
    Hosts file:(we have no ping problem )
    [root@rac2 ~]# cat /etc/hosts
    # Do not remove the following line, or various programs
    # that require network functionality will fail.
    127.0.0.1       localhost.localdomain   localhost
    # Public
    192.168.2.101   rac1.localdomain        rac1
    192.168.2.102   rac2.localdomain        rac2
    # Private
    192.168.0.101   rac1-priv.localdomain   rac1-priv
    192.168.0.102   rac2-priv.localdomain   rac2-priv
    # Virtual
    192.168.2.111   rac1-vip.localdomain    rac1-vip
    192.168.2.112   rac2-vip.localdomain    rac2-vip
    # SCAN
    192.168.2.201   rac-scan.localdomain rac-scan
    [root@rac2 ~]#
    FIRST NODE root.sh script output...
    [root@rac2 ~]# /u01/app/11.2.0/db_1/root.sh
    Running Oracle 11g root.sh script...
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/app/11.2.0/db_1
    Enter the full pathname of the local bin directory: [/usr/local/bin]:
       Copying dbhome to /usr/local/bin ...
       Copying oraenv to /usr/local/bin ...
       Copying coraenv to /usr/local/bin ...
    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-12-06 14:45:06: Parsing the host name
    2010-12-06 14:45:06: Checking for super user privileges
    2010-12-06 14:45:06: User has super user privileges
    Using configuration parameter file: /u01/app/11.2.0/db_1/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
      root wallet
      root wallet cert
      root cert export
      peer wallet
      profile reader wallet
      pa wallet
      peer wallet keys
      pa wallet keys
      peer cert request
      pa cert request
      peer cert
      pa cert
      peer root cert TP
      profile reader root cert TP
      pa root cert TP
      peer pa cert TP
      pa peer cert TP
      profile reader pa cert TP
      profile reader peer cert TP
      peer user cert
      pa user cert
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
    CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
    CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
    CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
    CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
    CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
    ASM created and started successfully.
    DiskGroup DATA created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
    CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
    CRS-4256: Updating the profile
    Successful addition of voting disk 587cc69413ce4fd3bf0c2c2548fb9017.
    Successfully replaced voting disk group with +DATA.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ##  STATE    File Universal Id                File Name Disk group
    1. ONLINE   587cc69413ce4fd3bf0c2c2548fb9017 (/dev/oracleasm/disks/DISK1) [DATA]
    Located 1 voting disk(s).
    CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'
    CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
    CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'
    CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac2'
    CRS-2677: Stop of 'ora.cssdmonitor' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'
    CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'
    CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'
    CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'
    CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
    CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
    CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
    CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
    CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
    CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac2'
    CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
    CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'rac2'
    CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac2'
    CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac2'
    CRS-2676: Start of 'ora.DATA.dg' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.registry.acfs' on 'rac2'
    CRS-2676: Start of 'ora.registry.acfs' on 'rac2' succeeded
    rac2     2010/12/06 14:52:06     /u01/app/11.2.0/db_1/cdata/rac2/backup_20101206_145206.olr
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
    Checking swap space: must be greater than 500 MB.   Actual 6847 MB    Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /u01/app/oraInventory
    'UpdateNodeList' was successful.
    [root@rac2 ~]#
    SECOND NODE root.sh script output
    [root@rac1 db_1]# ./root.sh
    Running Oracle 11g root.sh script...
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/app/11.2.0/db_1
    Enter the full pathname of the local bin directory: [/usr/local/bin]:
       Copying dbhome to /usr/local/bin ...
       Copying oraenv to /usr/local/bin ...
       Copying coraenv to /usr/local/bin ...
    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-12-06 14:54:11: Parsing the host name
    2010-12-06 14:54:11: Checking for super user privileges
    2010-12-06 14:54:11: User has super user privileges
    Using configuration parameter file: /u01/app/11.2.0/db_1/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
    CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
    CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
    CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
    CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
    CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
    ASM created and started successfully.
    DiskGroup DATA created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
    CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
    Successful addition of voting disk 2761ce8d47b44fbabf73462151e3ba1d.
    Successfully replaced voting disk group with +DATA.
    CRS-4266: Voting file(s) successfully replaced
    ##  STATE    File Universal Id                File Name Disk group
    1. ONLINE   2761ce8d47b44fbabf73462151e3ba1d (/dev/oracleasm/disks/DISK1) [DATA]
    Located 1 voting disk(s).
    CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
    CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
    CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
    CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
    CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
    CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
    CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
    CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
    CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
    CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
    CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
    CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
    CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
    CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac1'
    CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
    CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
    CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac1'
    CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1'
    CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded
    PRCR-1079 : *Failed* to start resource ora.scan1.vip
    CRS-5005: IP Address: 192.168.2.201 is already in use in the network
    CRS-2674: Start of 'ora.scan1.vip' on 'rac1' *failed*
    CRS-2632: There are no more servers to try to place resource 'ora.scan1.vip' on that would satisfy its placement policy
    start scan ... *failed*
    Configure Oracle Grid Infrastructure for a Cluster ... *failed*
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
    Checking swap space: must be greater than 500 MB.   Actual 6847 MB    Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /u01/app/oraInventory
    'UpdateNodeList' was successful.
    [root@rac1 db_1]# * "./runcluvfy.sh stage -pre -crsinst -n rac1,rac2 " outputs are same each node....*
    [oracle@rac2 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2
    Performing pre-checks for cluster services setup
    Checking node reachability...
    Node reachability check passed from node "rac2"
    Checking user equivalence...
    User equivalence check passed for user "oracle"
    Checking node connectivity...
    Checking hosts config file...
    Verification of the hosts config file successful
    Node connectivity passed for subnet "192.168.2.0" with node(s) rac2,rac1
    TCP connectivity check passed for subnet "192.168.2.0"
    Node connectivity passed for subnet "192.168.122.0" with node(s) rac2,rac1
    TCP connectivity check failed for subnet "192.168.122.0"
    Node connectivity passed for subnet "192.168.0.0" with node(s) rac2,rac1
    TCP connectivity check passed for subnet "192.168.0.0"
    Interfaces found on subnet "192.168.2.0" that are likely candidates for VIP are:
    rac2 eth0:192.168.2.102 eth0:192.168.2.112 eth0:192.168.2.201
    rac1 eth0:192.168.2.101 eth0:192.168.2.111
    Interfaces found on subnet "192.168.122.0" that are likely candidates for a private interconnect are:
    rac2 virbr0:192.168.122.1
    rac1 virbr0:192.168.122.1
    Interfaces found on subnet "192.168.0.0" that are likely candidates for a private interconnect are:
    rac2 eth1:192.168.0.102
    rac1 eth1:192.168.0.101
    Node connectivity check passed
    Total memory check passed
    Available memory check passed
    Swap space check passed
    Free disk space check passed for "rac2:/tmp"
    Free disk space check passed for "rac1:/tmp"
    User existence check passed for "oracle"
    Group existence check passed for "oinstall"
    Group existence check passed for "dba"
    Membership check for user "oracle" in group "oinstall" [as Primary] passed
    Membership check for user "oracle" in group "dba" passed
    Run level check passed
    Hard limits check passed for "maximum open file descriptors"
    Soft limits check passed for "maximum open file descriptors"
    Hard limits check passed for "maximum user processes"
    Soft limits check passed for "maximum user processes"
    System architecture check passed
    Kernel version check passed
    Kernel parameter check passed for "semmsl"
    Kernel parameter check passed for "semmns"
    Kernel parameter check passed for "semopm"
    Kernel parameter check passed for "semmni"
    Kernel parameter check passed for "shmmax"
    Kernel parameter check passed for "shmmni"
    Kernel parameter check passed for "shmall"
    Kernel parameter check passed for "file-max"
    Kernel parameter check passed for "ip_local_port_range"
    Kernel parameter check passed for "rmem_default"
    Kernel parameter check passed for "rmem_max"
    Kernel parameter check passed for "wmem_default"
    Kernel parameter check passed for "wmem_max"
    Kernel parameter check passed for "aio-max-nr"
    Package existence check passed for "make-3.81"
    Package existence check passed for "binutils-2.17.50.0.6"
    Package existence check passed for "gcc-4.1.2"
    Package existence check passed for "libaio-0.3.106 (i386)"
    Package existence check passed for "libaio-0.3.106 (x86_64)"
    Package existence check passed for "glibc-2.5-24 (i686)"
    Package existence check passed for "glibc-2.5-24 (x86_64)"
    Package existence check passed for "compat-libstdc++-33-3.2.3 (i386)"
    Package existence check passed for "compat-libstdc++-33-3.2.3 (x86_64)"
    Package existence check passed for "elfutils-libelf-0.125 (x86_64)"
    Package existence check passed for "elfutils-libelf-devel-0.125"
    Package existence check passed for "glibc-common-2.5"
    Package existence check passed for "glibc-devel-2.5 (i386)"
    Package existence check passed for "glibc-devel-2.5 (x86_64)"
    Package existence check passed for "glibc-headers-2.5"
    Package existence check passed for "gcc-c++-4.1.2"
    Package existence check passed for "libaio-devel-0.3.106 (i386)"
    Package existence check passed for "libaio-devel-0.3.106 (x86_64)"
    Package existence check passed for "libgcc-4.1.2 (i386)"
    Package existence check passed for "libgcc-4.1.2 (x86_64)"
    Package existence check passed for "libstdc++-4.1.2 (i386)"
    Package existence check passed for "libstdc++-4.1.2 (x86_64)"
    Package existence check passed for "libstdc++-devel-4.1.2 (x86_64)"
    Package existence check passed for "sysstat-7.0.2"
    Package existence check passed for "unixODBC-2.2.11 (i386)"
    Package existence check passed for "unixODBC-2.2.11 (x86_64)"
    Package existence check passed for "unixODBC-devel-2.2.11 (i386)"
    Package existence check passed for "unixODBC-devel-2.2.11 (x86_64)"
    Package existence check passed for "ksh-20060214"
    Check for multiple users with UID value 0 passed
    Current group ID check passed
    Core file name pattern consistency check passed.
    User "oracle" is not part of "root" group. Check passed
    Default user file creation mask check passed
    Starting Clock synchronization checks using Network Time Protocol(NTP)...
    NTP Configuration file check started...
    NTP Configuration file check passed
    Checking daemon liveness...
    Liveness check passed for "ntpd"
    NTP daemon slewing option check passed
    NTP daemon's boot time configuration check for slewing option passed
    NTP common Time Server Check started...
    Check of common NTP Time Server passed
    Clock time offset check from NTP Time Server started...
    Clock time offset check passed
    Clock synchronization check using Network Time Protocol(NTP) passed
    Pre-check for cluster services setup was successful.
    [oracle@rac2 grid]$ I'm confused :)
    Edited by: Eren GULERYUZ on 06.Ara.2010 05:57

    Hi,
    it looks like, that your "shared device" you are using is not really shared.
    The second node does "create an ASM diskgroup" and create OCR and Voting disks. If this indeed would be a shared device, he should have recognized, that your disk is shared.
    So as a result your VMware configuration must be wrong, and the disk you presented as shared disk is not really shared.
    Which VMWare version did you use? It will not work correctly with the workstation or player edition, since shared disks are only really working with the server version.
    If you indeed using the server, could you paste your vm configurations?
    Furthermore I recommend using Virtual Box. There is a nice how-to:
    http://www.oracle-base.com/articles/11g/OracleDB11gR2RACInstallationOnOEL5UsingVirtualBox.php
    Sebastian

  • RAC, ASM failed to start up on second node , ORA-03113: end-of-file on comm

    i'm installing an RAC with 2 nodes on top of ASM
    when creating ASM Diskgroup , it failed and reported error CRS-0215 failed to start asm on node2
    Oracle 10.2.0.1
    linux CentOs 4.x
    u01/app/oracle/product/10.2.0/db_1/bin/dbca  -progress_only   -configureASM -templateName NO_VALUE -gdbName NO -sid NO      -emConf
    iguration NONE    -diskList /dev/raw/raw2,/dev/raw/raw3  -diskGroupName DATA -datafileJarLocation /u01/app/oracle/product/10.2.0/db_
    1/assistants/dbca/templates  -responseFile NO_VALUE  -nodeinfo node1,node2    -obfuscatedPasswords true   -oratabLocation /u01/app/o
    racle/product/10.2.0/db_1/install/oratab   -asmSysPassword 05dbb0be38ecf8cca822cf3cf99e675448  -redundancy EXTERNA
    [oracle@node2 bin]$ ./crs_stat -t -v
    Name           Type           R/RA   F/FT   Target    State     Host       
    ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    node1      
    ora....E1.lsnr application    0/5    0/0    ONLINE    ONLINE    node1      
    ora.node1.gsd  application    0/5    0/0    ONLINE    ONLINE    node1      
    ora.node1.ons  application    0/3    0/0    ONLINE    ONLINE    node1      
    ora.node1.vip  application    0/0    0/0    ONLINE    ONLINE    node1      
    ora....SM2.asm application    0/5    0/0    OFFLINE   OFFLINE              
    ora....E2.lsnr application    0/5    0/0    ONLINE    ONLINE    node2      
    ora.node2.gsd  application    0/5    0/0    ONLINE    ONLINE    node2      
    ora.node2.ons  application    0/3    0/0    ONLINE    ONLINE    node2      
    ora.node2.vip  application    0/0    0/0    ONLINE    ONLINE    node2  
    i checked the status , asm is able to start on both nodes if not at the same time ,
    when trying to start the second node , with srvctl or sqlplus , each give the error 03113
    can anyone suggest me of how to bring up both instances ,
    thanks~
    [oracle@node2 bin]$ srvctl stop asm -n node1
    [oracle@node2 bin]$ srvctl start asm -n node1
    [oracle@node2 bin]$ srvctl start asm -n node2
    PRKS-1009 : Failed to start ASM instance "+ASM2" on node "node2", [PRKS-1009 : Failed to start ASM instance "+ASM2" on node "node2", [node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:SQL*Plus: Release 10.2.0.1.0 - Production on Wed May 27 16:14:50 2009
    node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:Enter user-name: Connected to an idle instance.
    node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:SQL> ORA-03113: end-of-file on communication channel
    node2:ora.node2.ASM2.asm:SQL> Disconnected
    node2:ora.node2.ASM2.asm:
    [code/]
    Edited by: zs_hzh on May 27, 2009 1:25 AM                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

    Is it possible to start ASM on second node with SQL*Plus in NOMOUNT state?

  • ONS failed to start on second node

    Hi,
    I have a problem with ons on 10g rac running on linux 5.3
    on node 1 it is running without problem but on second node i got this error
    2009-04-08 16:30:41.318: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    Number of onsconfiguration retrieved, numcfg = 2
    onscfg[0]
    {node = rac1, port = 6200}
    Adding remote host rac1:6200
    onscfg[1]
    {node = rac2, port = 6
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: 200}
    Adding remote host rac2:6200
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission d
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: enied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server loca
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: l port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    Number of onsconfiguration retrieved, numcfg = 2
    onscfg[0]
    {node = rac1, port = 6200}
    Adding remote host rac1:6200
    o
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: nscfg[1]
    {node = rac2, port = 6200}
    Adding remote host rac2:6200
    onsctl: ons failed to start
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/crs
    2009-04-08 16:30:41.319: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: clsrcexecut: cmd = /u01/app/crs/bin/racgeut -e USRORA_DEBUG=0 540 /u01/app/crs/bin/onsctl start
    2009-04-08 16:30:41.320: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: clsrcexecut: rc = 1, time = 2.580s
    2009-04-08 16:30:42.148: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the
    2009-04-08 16:30:42.150: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    2009-04-08 16:30:42.150: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: RCV: Permission denied
    Communication error with the OPMN server local port.
    Check the OPMN log files
    Number of onsconfiguration retrieved, numcfg = 2
    onscfg[0]
    {node = rac1, port = 6200}
    Adding remote host rac1:6200
    onscfg[1]
    {node = rac2, port = 6
    2009-04-08 16:30:42.150: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: 200}
    Adding remote host rac2:6200
    ons is not running ...
    2009-04-08 16:30:42.151: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/crs
    2009-04-08 16:30:42.151: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: clsrcexecut: cmd = /u01/app/crs/bin/racgeut -e USRORA_DEBUG=0 540 /u01/app/crs/bin/onsctl ping
    2009-04-08 16:30:42.151: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: clsrcexecut: rc = 1, time = 0.840s
    2009-04-08 16:30:42.153: [    RACG][3065611968] [16553][3065611968][ora.rac2.ons]: end for resource = ora.rac2.ons, action = start, status = 1, time = 3.620s
    2009-04-08 16:30:44.376: [    RACG][3066242752] [17061][3066242752][ora.rac2.ons]: onsctl: shutting down ons daemon ...
    Number of onsconfiguration retrieved, numcfg = 2
    onscfg[0]
    {node = rac1, port = 6200}
    Adding remote host rac1:6200
    onscfg[1]
    {node = rac2, port = 6200}
    Adding remote host rac2:6200
    Any idea how to fix this?
    Thanks

    check the output for crs_getperm for the resource from both nodes. If you could, post them here.
    Regards,
    Ganesh

  • Hotfix (4.1.3559) installation on the second portal node fails

    Hi,
    I'm installing FIM hotfix (4.1.3559). It installed without issues on sync server and first node of the portal/service. On second node it is failing. Here is place from the log (most probably failing somewhere here):
    StartServices: Service: Forefront Identity Manager Service
    MSI (s) (70:D4) [16:57:19:635]: Executing op: ActionStart(Name=PatchRemoveFIMPortal,,)
    Action 16:57:19: PatchRemoveFIMPortal.
    MSI (s) (70:D4) [16:57:19:635]: Executing op: CustomActionSchedule(Action=PatchRemoveFIMPortal,ActionType=1025,Source=BinaryData,Target=CAQuietExec,CustomActionData="C:\Program Files\Microsoft Forefront Identity Manager\2010\Portal\Microsoft.IdentityManagement.SolutionPackUtility.exe"
    action=uninstall mode=ServiceAndPortal log=event SHAREPOINTTIMEOUT=180 SolutionPack=MicrosoftIdentityManagement.wsp deleteweb=no continueonerror=no UILevel=5)
    MSI (s) (70:2C) [16:57:19:790]: Invoking remote custom action. DLL: C:\Windows\Installer\MSICD1E.tmp, Entrypoint: CAQuietExec
    CAQuietExec:  Microsoft.IdentityManagement.SolutionPackUtility.exe will deploy and/or retract the FIM solution packs. This operation may take long time in a SharePoint farm environment.
    CAQuietExec:  Executing all administrative timer jobs in preparation for FIM solution pack retraction.
    CAQuietExec:  An exception occurred while deploying/retracting FIM Portal solution packs. Exception : Exception has been thrown by the target of an invocation.
    CAQuietExec:  Error 0xfffffff9: Command line returned an error.
    CAQuietExec:  Error 0xfffffff9: CAQuietExec Failed
    CustomAction PatchRemoveFIMPortal returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
    09/27/2014 16:57:28.791 [7024]: Assembly Install: Failing with hr=80070005 at RemoveDirectoryAndChildren, line 398
    09/27/2014 16:57:28.791 [7024]: Detailed info about C:\Windows\assembly\tmp\S9M6DS80\Microsoft.ResourceManagement.dll
    09/27/2014 16:57:28.791 [7024]:  File attributes: 00000080
    09/27/2014 16:57:28.948 [7024]:  Restart Manager Info: 1 entries
    09/27/2014 16:57:28.948 [7024]:   App[0]: (7024) Windows Installer (msiserver), type = 3
    09/27/2014 16:57:28.948 [7024]:  Security info:
    09/27/2014 16:57:28.948 [7024]:   Owner: S-1-5-18
    09/27/2014 16:57:28.948 [7024]:   Group: S-1-5-18
    09/27/2014 16:57:28.948 [7024]:   DACL information: 4 entries:
    09/27/2014 16:57:28.948 [7024]:   ACE[0]: Type = 0x00, Flags = 010, Mask = 001f01ff, SID = S-1-5-18
    09/27/2014 16:57:28.948 [7024]:   ACE[1]: Type = 0x00, Flags = 010, Mask = 001f01ff, SID = S-1-5-32-544
    09/27/2014 16:57:28.948 [7024]:   ACE[2]: Type = 0x00, Flags = 010, Mask = 001200a9, SID = S-1-5-32-545
    09/27/2014 16:57:28.948 [7024]:   ACE[3]: Type = 0x00, Flags = 010, Mask = 001200a9, SID = S-1-15-2-1
    09/27/2014 16:57:28.963 [7024]: Assembly Install: Failing with hr=80070005 at RemoveDirectoryAndChildren, line 398
    09/27/2014 16:57:28.963 [7024]: Detailed info about C:\Windows\assembly\tmp\WGGUV4IP\Microsoft.IdentityManagement.Logging.dll
    09/27/2014 16:57:28.963 [7024]:  File attributes: 00000080
    09/27/2014 16:57:29.088 [7024]:  Restart Manager Info: 1 entries
    09/27/2014 16:57:29.088 [7024]:   App[0]: (7024) Windows Installer (msiserver), type = 3
    09/27/2014 16:57:29.088 [7024]:  Security info:
    09/27/2014 16:57:29.088 [7024]:   Owner: S-1-5-18
    09/27/2014 16:57:29.088 [7024]:   Group: S-1-5-18
    09/27/2014 16:57:29.088 [7024]:   DACL information: 4 entries:
    09/27/2014 16:57:29.088 [7024]:   ACE[0]: Type = 0x00, Flags = 010, Mask = 001f01ff, SID = S-1-5-18
    09/27/2014 16:57:29.088 [7024]:   ACE[1]: Type = 0x00, Flags = 010, Mask = 001f01ff, SID = S-1-5-32-544
    09/27/2014 16:57:29.104 [7024]:   ACE[2]: Type = 0x00, Flags = 010, Mask = 001200a9, SID = S-1-5-32-545
    09/27/2014 16:57:29.104 [7024]:   ACE[3]: Type = 0x00, Flags = 010, Mask = 001200a9, SID = S-1-15-2-1
    09/27/2014 16:57:29.104 [7024]: Assembly Install: Failing with hr=80070005 at RemoveDirectoryAndChildren, line 398
    09/27/2014 16:57:29.104 [7024]: Detailed info about C:\Windows\assembly\tmp\NJBBJ7LC\Microsoft.IdentityManagement.CredentialManagement.Portal.Gates.dll
    09/27/2014 16:57:29.104 [7024]:  File attributes: 00000080
    09/27/2014 16:57:29.244 [7024]:  Restart Manager Info: 1 entries
    09/27/2014 16:57:29.244 [7024]:   App[0]: (7024) Windows Installer (msiserver), type = 3
    09/27/2014 16:57:29.244 [7024]:  Security info:
    09/27/2014 16:57:29.244 [7024]:   Owner: S-1-5-18
    09/27/2014 16:57:29.244 [7024]:   Group: S-1-5-18
    09/27/2014 16:57:29.244 [7024]:   DACL information: 4 entries:
    09/27/2014 16:57:29.244 [7024]:   ACE[0]: Type = 0x00, Flags = 010, Mask = 001f01ff, SID = S-1-5-18
    09/27/2014 16:57:29.244 [7024]:   ACE[1]: Type = 0x00, Flags = 010, Mask = 001f01ff, SID = S-1-5-32-544
    09/27/2014 16:57:29.244 [7024]:   ACE[2]: Type = 0x00, Flags = 010, Mask = 001200a9, SID = S-1-5-32-545
    09/27/2014 16:57:29.244 [7024]:   ACE[3]: Type = 0x00, Flags = 010, Mask = 001200a9, SID = S-1-15-2-1
    09/27/2014 16:57:29.275 [7024]: Assembly Install: Failing with hr=80070005 at RemoveDirectoryAndChildren, line 398
     Any suggestions/hints? I belive it is something with SharePoint (SP Foundation 2013 is used/ no farms).
    By the way - in the similar setup in test environment it went OK. Right now it is failing on the production
    Borys Majewski, Identity Management Solutions Architect (Blog: IDArchitect.NET)

    Same situation with update 4.1.3599 (newer one)
    Finally issue has been solved - it was some well know SharePoint problem. Clearing some sharepoint cach helped (http://officepoint.blogspot.com/2008/06/update-conflict-has-occurred-and-you.html)
    Borys Majewski, Identity Management Solutions Architect (Blog: IDArchitect.NET)

  • Oracle Grid Infrastructure 11.2.0.2 root.sh hangs in the second node

    Hi,
    I am installing Oracle Grid Infrastructure 11.2.0.2.0 using VM's. Both the virtual machines are running RHEL 5.7 32 bit version.
    Until the point where the installation suggests to run two scripts as root user, everything runs fine.
    root.sh runs successfully on the first node.
    On the second node it hangs at,
    Now product-specific root actions will be performed.
    Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

    user10193468 wrote:
    Hi,
    Following is the last few lines of the $GRID_HOME/cfgtoollogs/crsconfig/rootcrs_<node>.log file
    2012-03-19 17:20:17: OLR successfully created or upgraded
    2012-03-19 17:20:17: /u01/app/11.2.0/grid/bin/clscfg -localadd
    2012-03-19 17:20:17: Executing /u01/app/11.2.0/grid/bin/clscfg -localadd
    2012-03-19 17:20:17: Executing cmd: /u01/app/11.2.0/grid/bin/clscfg -localadd
    I am using ASM for OCR and Voting Disks.
    Private network is up and running.Please let me know what have you set in asm_diskstring parameter?
    I have find following note from metalink which says that you should not set asm_diskstring to '/dev/' or '/dev/*'. This is not supported
    root.sh Hangs or Reboots Node on Non First Node if ASM Discovery String asm_diskstring is Set to /dev/ or /dev/* [ID 1356567.1]

Maybe you are looking for

  • Error while creating Query based taxonomy.

    Hi, I have created an index and trying to build a query based taxonomy though Taxonomy Query Builder in content management. Earlier I have created another index but after that I have deleted it but still while trying to build a query based taxonomy i

  • Is there a way to copy and paste info from an attachment on the ipad into a document or note?

    I sent myself an email with an attachment to my iPad.  I want to copy and paste the info from the attachment so I can have it accessible on the iPad either as a Note or a document.  I've been able to copy the info but the Paste does not do anything. 

  • Does OS Lion have a screenshot feature?

    I'm a new Mac user and I recently bought my new MacBook Pro! . I was wondering if you can take screenshots like you can in windows. (press Print screen in windows to take screenshot). I was wondering if you could use a something like "command" (key).

  • Error  while installing forms 4.5 client w32prd.tmp not found

    can someone tell me about w32prd.tmp file.I copied it to the setup folder from forms 5.0 installtion cd. machine is on windows NT 4.0.

  • Premiere Pro has encountered an error  [URGENT]

    I can´t open my recent project. I receive the following error Message: "Premiere Pro has encountered an error. [..\..\Src\Clip\MasterClip.cpp-2802]" Even if i want to open the auto-saved projects i get this error message. I tried to import the projec