Why root.sh fails in the second node?

Hi
After successful install of oracle11 grid in 2 nodes and running root.sh on
node1, root.sh on node2 fails:
[root@vmorarac2 dev]#
/u01/app/product/11.2.0/oracle/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/product/11.2.0/oracle
Enter the full pathname of the local bin directory:
[/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. 
Overwrite it? (y/n)
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. 
Overwrite it? (y/n)
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. 
Overwrite it? (y/n)
[n]: y
   Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed
by
Database Configuration Assistant when a database is
created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2013-07-17 08:37:10: Parsing the host name
2013-07-17 08:37:10: Checking for super user
privileges
2013-07-17 08:37:10: User has super user privileges
Using configuration parameter file:
/u01/app/product/11.2.0/oracle/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been
started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but
found an active CSS daemon on node vmorarac1, number 1, and is terminating
CRS-2673: Attempting to stop 'ora.cssdmonitor' on
'vmorarac2'
CRS-2677: Stop of 'ora.cssdmonitor' on 'vmorarac2'
succeeded
An active cluster was found during exclusive startup,
restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on
'vmorarac2'
CRS-2676: Start of 'ora.mdnsd' on 'vmorarac2'
succeeded
CRS-2672: Attempting to start 'ora.gipcd' on
'vmorarac2'
CRS-2676: Start of 'ora.gipcd' on 'vmorarac2'
succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on
'vmorarac2'
CRS-2676: Start of 'ora.gpnpd' on 'vmorarac2'
succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on
'vmorarac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'vmorarac2'
succeeded
CRS-2672: Attempting to start 'ora.cssd' on
'vmorarac2'
CRS-2672: Attempting to start 'ora.diskmon' on
'vmorarac2'
CRS-2676: Start of 'ora.diskmon' on 'vmorarac2'
succeeded
CRS-2674: Start of 'ora.cssd' on 'vmorarac2' failed
CRS-2679: Attempting to clean 'ora.cssd' on
'vmorarac2'
CRS-2681: Clean of 'ora.cssd' on 'vmorarac2' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on
'vmorarac2'
CRS-2677: Stop of 'ora.diskmon' on 'vmorarac2'
succeeded
CRS-4000: Command Start failed, or completed with
errors.
CRS-2672: Attempting to start 'ora.cssd' on
'vmorarac2'
CRS-2672: Attempting to start 'ora.diskmon' on
'vmorarac2'
CRS-2674: Start of 'ora.diskmon' on 'vmorarac2' failed
CRS-2679: Attempting to clean 'ora.diskmon' on
'vmorarac2'
CRS-5016: Process
"/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent
"/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed:
details at "(:CLSN00010:)" in
"/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"
CRS-2681: Clean of 'ora.diskmon' on 'vmorarac2'
succeeded
CRS-2674: Start of 'ora.cssd' on 'vmorarac2' failed
CRS-2679: Attempting to clean 'ora.cssd' on
'vmorarac2'
CRS-2681: Clean of 'ora.cssd' on 'vmorarac2' succeeded
CRS-4000: Command Start failed, or completed with
errors.
Command return code of 1 (256) from command:
/u01/app/product/11.2.0/oracle/bin/crsctl start resource ora.ctssd -init -env
USR_ORA_ENV=CTSS_REBOOT=TRUE
Start of resource "ora.ctssd -init -env
USR_ORA_ENV=CTSS_REBOOT=TRUE" failed
Failed to start CTSS
Failed to start Oracle Clusterware stack
[root@vmorarac2 dev]#
[root@vmorarac2 dev]#
try again:
[root@vmorarac2 bin]# ./crsctl start resource ora.ctssd
-init -env USR_ORA_ENV=CTSS_REBOOT=TRUE
CRS-2672: Attempting to start 'ora.cssd' on
'vmorarac2'
CRS-2672: Attempting to start 'ora.diskmon' on
'vmorarac2'
CRS-2674: Start of 'ora.diskmon' on 'vmorarac2' failed
CRS-2679: Attempting to clean 'ora.diskmon' on
'vmorarac2'
CRS-5016: Process
"/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent
"/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed:
details at "(:CLSN00010:)" in
"/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"
CRS-2681: Clean of 'ora.diskmon' on 'vmorarac2'
succeeded
CRS-2674: Start of 'ora.cssd' on 'vmorarac2' failed
CRS-2679: Attempting to clean 'ora.cssd' on
'vmorarac2'
CRS-2681: Clean of 'ora.cssd' on 'vmorarac2' succeeded
CRS-4000: Command Start failed, or completed with
errors.
[root@vmorarac2 bin]# ps -ef | grep u01
root      8913     1  0 08:37 ?        00:00:06
/u01/app/product/11.2.0/oracle/bin/ohasd.bin reboot
oracle   10507     1  0 08:39 ?        00:00:02
/u01/app/product/11.2.0/oracle/bin/oraagent.bin
oracle   10522     1  0 08:39 ?        00:00:00
/u01/app/product/11.2.0/oracle/bin/mdnsd.bin
oracle   10534     1  0 08:39 ?        00:00:00
/u01/app/product/11.2.0/oracle/bin/gipcd.bin
oracle   10548     1  0 08:39 ?        00:00:39
/u01/app/product/11.2.0/oracle/bin/gpnpd.bin
root     11723     1  0 11:00 ?        00:00:03
/u01/app/product/11.2.0/oracle/bin/cssdmonitor
[oracle@vmorarac2 bin]$ ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready
Services
CRS-4530: Communications failure contacting Cluster
Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager

log:
Oracle Database 11g Clusterware Release 11.2.0.1.0 - Production Copyright 1996, 2009 Oracle. All rights reserved.
2013-07-17 08:38:04.485: [    AGFW][3481860944] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/
2013-07-17 08:38:04.485: [   AGENT][3481860944] Agent framework initialized, Process Id = 10319
2013-07-17 08:38:04.487: [ USRTHRD][3481860944] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle
2013-07-17 08:38:04.487: [ USRTHRD][3481860944] Process::convertPidToString pid = 10319
2013-07-17 08:38:04.488: [    AGFW][3481860944] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))
2013-07-17 08:38:04.488: [CLSFRAME][3481860944] Inited lsf context 0x317e9e0
2013-07-17 08:38:04.488: [CLSFRAME][3481860944] Initing CLS Framework messaging
2013-07-17 08:38:04.488: [CLSFRAME][3481860944] New Framework state: 2
2013-07-17 08:38:04.488: [CLSFRAME][3481860944] M2M is starting...
2013-07-17 08:38:04.490: [ CRSCOMM][3481860944] m_pClscCtx=0x31d1bd0m_pUgblm=0x31d5720
2013-07-17 08:38:04.490: [ CRSCOMM][3481860944] Starting send thread
2013-07-17 08:38:04.490: [ CRSCOMM][1119435072] clsIpc: sendWork thread started.
2013-07-17 08:38:04.491: [ CRSCOMM][1129924928] IPC Client thread started listening
2013-07-17 08:38:04.491: [ CRSCOMM][1129924928] init data sent from server
2013-07-17 08:38:04.491: [CLSFRAME][3481860944] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2
2013-07-17 08:38:04.491: [CLSFRAME][3481860944] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2
2013-07-17 08:38:04.492: [CLSFRAME][3481860944] Starting thread model named: MultiThread
2013-07-17 08:38:04.492: [CLSFRAME][3481860944] Starting thread model named: SingleThread
2013-07-17 08:38:04.492: [CLSFRAME][3481860944] Starting thread model named: SingleThreadT
2013-07-17 08:38:04.492: [CLSFRAME][3481860944] New Framework state: 3
2013-07-17 08:38:04.493: [    AGFW][3481860944] Agent Framework started successfully
2013-07-17 08:38:04.493: [    AGFW][1182374208] Agfw engine module has enabled...
2013-07-17 08:38:04.493: [CLSFRAME][1182374208] Module Enabling is complete
2013-07-17 08:38:04.493: [CLSFRAME][1182374208] New Framework state: 6
2013-07-17 08:38:04.493: [    AGFW][1182374208] Agent is started with userid: root , expected user: root
2013-07-17 08:38:04.493: [    AGFW][1182374208] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14
2013-07-17 08:38:04.505: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:358
2013-07-17 08:38:04.506: [    AGFW][1182374208] Added new restype: ora.crs.type
2013-07-17 08:38:04.506: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:358
2013-07-17 08:38:04.506: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:360
2013-07-17 08:38:04.506: [    AGFW][1182374208] Added new restype: ora.ctss.type
2013-07-17 08:38:04.507: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:360
2013-07-17 08:38:04.516: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:362
2013-07-17 08:38:04.516: [    AGFW][1182374208] Added new restype: ora.diskmon.type
2013-07-17 08:38:04.517: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:362
2013-07-17 08:38:04.519: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:364
2013-07-17 08:38:04.520: [    AGFW][1182374208] Added new restype: ora.drivers.acfs.type
2013-07-17 08:38:04.520: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:364
2013-07-17 08:38:04.521: [    AGFW][1182374208] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:366
2013-07-17 08:38:04.521: [    AGFW][1182374208] Added new resource: ora.diskmon 1 1 to the agfw
2013-07-17 08:38:04.522: [    AGFW][1182374208] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:366
2013-07-17 08:38:04.522: [    AGFW][1182374208] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:367
2013-07-17 08:38:04.522: [    AGFW][1182374208] Preparing START command for: ora.diskmon 1 1
2013-07-17 08:38:04.522: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING
2013-07-17 08:38:04.526: [    AGFW][1161394496] Executing command: start for resource: ora.diskmon 1 1
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] clsn_agent::start {
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DaemonAgent{
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] }DaemonAgent
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DiskmonAgent::DiskmonAgent {
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] InitAttrs {
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] __IS_HASD_AGENT=TRUE
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] }InitAttrs
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DiskmonAgent::DiskmonAgent }
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DiskmonAgent::start {
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Arg Value = -d
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Arg Value = -f
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Total Count of Environment Variables = 3
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Adding Environment Variables __IS_HASD_AGENT=
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Adding Environment variable from USR_ORA_ENV ORACLE_USER=oracle
2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon.
2013-07-17 08:38:04.528: [ora.diskmon][1161394496] [start] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle
2013-07-17 08:38:04.528: [ora.diskmon][1161394496] [start] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon
2013-07-17 08:38:04.531: [CRSTIMER][1091324224] Timer Thread Starting.
2013-07-17 08:38:04.533: [ora.diskmon][1161394496] [start] execCmd ret = 0
2013-07-17 08:38:04.533: [ora.diskmon][1161394496] [start] }DaemonAgent::start
2013-07-17 08:38:10.534: [ora.diskmon][1161394496] [start] DiskmonAgent::connect {
2013-07-17 08:38:10.534: [ora.diskmon][1161394496] [start] Process::convertPidToString pid = 10319
2013-07-17 08:38:10.535: [ora.diskmon][1161394496] [start] DiskmonAgent::connect }
2013-07-17 08:38:10.535: [ora.diskmon][1161394496] [start] DiskmonAgent::start }
2013-07-17 08:38:10.535: [ora.diskmon][1161394496] [start] clsn_agent::start }
2013-07-17 08:38:10.535: [    AGFW][1161394496] Command: start for resource: ora.diskmon 1 1 completed with status: SUCCESS
2013-07-17 08:38:10.535: [    AGFW][1182374208] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:367
2013-07-17 08:38:10.537: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:38:10.538: [ora.diskmon][1161394496] [check] DiskmonAgent::check {
2013-07-17 08:38:10.538: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 0
2013-07-17 08:38:10.538: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:38:10.538: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: STARTING to: ONLINE
2013-07-17 08:38:10.538: [    AGFW][1182374208] Started implicit monitor for:ora.diskmon 1 1
2013-07-17 08:38:10.538: [    AGFW][1182374208] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:367
2013-07-17 08:38:30.543: [    AGFW][1182374208] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:38:30.544: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:38:30.544: [ora.diskmon][1161394496] [check] DiskmonAgent::check {
2013-07-17 08:38:30.544: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 0
2013-07-17 08:38:30.545: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:38:50.550: [    AGFW][1182374208] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:38:50.551: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:38:50.551: [ora.diskmon][1161394496] [check] DiskmonAgent::check {
2013-07-17 08:38:50.551: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 0
2013-07-17 08:38:50.551: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:39:02.782: [    AGFW][1182374208] Agent received the message: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:547
2013-07-17 08:39:02.782: [    AGFW][1182374208] Preparing STOP command for: ora.diskmon 1 1
2013-07-17 08:39:02.782: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: ONLINE to: STOPPING
2013-07-17 08:39:02.783: [    AGFW][1161394496] Executing command: stop for resource: ora.diskmon 1 1
2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] clsn_agent::stop {
2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] DiskmonAgent::stop {
2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] DiskmonAgent::stop }
2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] clsn_agent::stop }
2013-07-17 08:39:02.783: [    AGFW][1161394496] Command: stop for resource: ora.diskmon 1 1 completed with status: SUCCESS
2013-07-17 08:39:02.784: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:39:02.784: [ora.diskmon][1161394496] [check] DiskmonAgent::check {
2013-07-17 08:39:02.784: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 2
2013-07-17 08:39:02.784: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: PLANNED_OFFLINE
2013-07-17 08:39:02.784: [    AGFW][1182374208] Agent sending reply for: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:547
2013-07-17 08:39:02.785: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: STOPPING to: PLANNED_OFFLINE
2013-07-17 08:39:02.785: [    AGFW][1182374208] Agent sending last reply for: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:547
2013-07-17 08:39:02.785: [    AGFW][1182374208] Agent has no resources to be monitored.Sending suicide request.
2013-07-17 08:39:02.786: [    AGFW][1182374208] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:72
2013-07-17 08:39:02.789: [    AGFW][1182374208] Agent is commiting suicide.
2013-07-17 08:39:02.790: [    AGFW][1182374208] Agent is exiting with exit code: 1
2013-07-17 08:39:12.176: [    AGFW][1664629584] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/
2013-07-17 08:39:12.176: [   AGENT][1664629584] Agent framework initialized, Process Id = 10581
2013-07-17 08:39:12.178: [ USRTHRD][1664629584] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle
2013-07-17 08:39:12.178: [ USRTHRD][1664629584] Process::convertPidToString pid = 10581
2013-07-17 08:39:12.178: [    AGFW][1664629584] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))
2013-07-17 08:39:12.178: [CLSFRAME][1664629584] Inited lsf context 0xd9309e0
2013-07-17 08:39:12.179: [CLSFRAME][1664629584] Initing CLS Framework messaging
2013-07-17 08:39:12.179: [CLSFRAME][1664629584] New Framework state: 2
2013-07-17 08:39:12.179: [CLSFRAME][1664629584] M2M is starting...
2013-07-17 08:39:12.180: [ CRSCOMM][1664629584] m_pClscCtx=0xd983bd0m_pUgblm=0xd987720
2013-07-17 08:39:12.180: [ CRSCOMM][1664629584] Starting send thread
2013-07-17 08:39:12.181: [ CRSCOMM][1115052352] clsIpc: sendWork thread started.
2013-07-17 08:39:12.181: [ CRSCOMM][1125542208] IPC Client thread started listening
2013-07-17 08:39:12.181: [ CRSCOMM][1125542208] init data sent from server
2013-07-17 08:39:12.181: [CLSFRAME][1664629584] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2
2013-07-17 08:39:12.181: [CLSFRAME][1664629584] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2
2013-07-17 08:39:12.182: [CLSFRAME][1664629584] Starting thread model named: MultiThread
2013-07-17 08:39:12.182: [CLSFRAME][1664629584] Starting thread model named: SingleThread
2013-07-17 08:39:12.182: [CLSFRAME][1664629584] Starting thread model named: SingleThreadT
2013-07-17 08:39:12.182: [CLSFRAME][1664629584] New Framework state: 3
2013-07-17 08:39:12.182: [    AGFW][1664629584] Agent Framework started successfully
2013-07-17 08:39:12.182: [    AGFW][1177991488] Agfw engine module has enabled...
2013-07-17 08:39:12.183: [CLSFRAME][1177991488] Module Enabling is complete
2013-07-17 08:39:12.183: [CLSFRAME][1177991488] New Framework state: 6
2013-07-17 08:39:12.183: [    AGFW][1177991488] Agent is started with userid: root , expected user: root
2013-07-17 08:39:12.183: [    AGFW][1177991488] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14
2013-07-17 08:39:12.192: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:886
2013-07-17 08:39:12.192: [    AGFW][1177991488] Added new restype: ora.crs.type
2013-07-17 08:39:12.192: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:886
2013-07-17 08:39:12.198: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:888
2013-07-17 08:39:12.198: [    AGFW][1177991488] Added new restype: ora.ctss.type
2013-07-17 08:39:12.199: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:888
2013-07-17 08:39:12.204: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:890
2013-07-17 08:39:12.204: [    AGFW][1177991488] Added new restype: ora.diskmon.type
2013-07-17 08:39:12.204: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:890
2013-07-17 08:39:12.209: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:892
2013-07-17 08:39:12.209: [    AGFW][1177991488] Added new restype: ora.drivers.acfs.type
2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:892
2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:894
2013-07-17 08:39:12.210: [    AGFW][1177991488] Added new resource: ora.diskmon 1 1 to the agfw
2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:894
2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:895
2013-07-17 08:39:12.211: [    AGFW][1177991488] Preparing START command for: ora.diskmon 1 1
2013-07-17 08:39:12.211: [    AGFW][1177991488] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING
2013-07-17 08:39:12.216: [    AGFW][1167501632] Executing command: start for resource: ora.diskmon 1 1
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] clsn_agent::start {
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] DaemonAgent{
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] }DaemonAgent
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] DiskmonAgent::DiskmonAgent {
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] InitAttrs {
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] __IS_HASD_AGENT=TRUE
2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] }InitAttrs
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] DiskmonAgent::DiskmonAgent }
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] DiskmonAgent::start {
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Arg Value = -d
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Arg Value = -f
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Total Count of Environment Variables = 3
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Adding Environment Variables __IS_HASD_AGENT=
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Adding Environment variable from USR_ORA_ENV ORACLE_USER=oracle
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon.
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle
2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon
2013-07-17 08:39:12.220: [CRSTIMER][1198971200] Timer Thread Starting.
2013-07-17 08:39:12.220: [ora.diskmon][1167501632] [start] execCmd ret = 0
2013-07-17 08:39:12.220: [ora.diskmon][1167501632] [start] }DaemonAgent::start
2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] DiskmonAgent::connect {
2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] Process::convertPidToString pid = 10581
2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] DiskmonAgent::connect }
2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] DiskmonAgent::start }
2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] clsn_agent::start }
2013-07-17 08:39:18.222: [    AGFW][1167501632] Command: start for resource: ora.diskmon 1 1 completed with status: SUCCESS
2013-07-17 08:39:18.223: [    AGFW][1177991488] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:895
2013-07-17 08:39:18.223: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:39:18.223: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:39:18.224: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:39:18.224: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:39:18.225: [    AGFW][1177991488] ora.diskmon 1 1 state changed from: STARTING to: ONLINE
2013-07-17 08:39:18.225: [    AGFW][1177991488] Started implicit monitor for:ora.diskmon 1 1
2013-07-17 08:39:18.225: [    AGFW][1177991488] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:895
2013-07-17 08:39:38.231: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:39:38.232: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:39:38.232: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:39:38.232: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:39:38.232: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:39:58.237: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:39:58.238: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:39:58.238: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:39:58.238: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:39:58.238: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:40:12.107: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:966
2013-07-17 08:40:18.243: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:40:18.244: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:40:18.244: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:40:18.244: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:40:18.244: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:40:38.250: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:40:38.251: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:40:38.251: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:40:38.251: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:40:38.251: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:40:42.116: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:982
2013-07-17 08:40:58.245: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:40:58.246: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:40:58.247: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:40:58.247: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:40:58.247: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:41:12.125: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:998
2013-07-17 08:41:18.252: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:41:18.252: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:41:18.253: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:41:18.253: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:41:18.253: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:41:38.259: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:41:38.260: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:41:38.260: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:41:38.260: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:41:38.260: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:41:58.255: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:41:58.256: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:41:58.256: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:41:58.256: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:41:58.256: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:42:12.134: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1026
2013-07-17 08:42:18.261: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:42:18.262: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:42:18.262: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:42:18.262: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:42:18.262: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:42:38.268: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:42:38.269: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:42:38.269: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:42:38.269: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:42:38.269: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:42:58.265: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:42:58.266: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:42:58.266: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:42:58.267: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:42:58.267: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:43:12.144: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1054
2013-07-17 08:43:18.272: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:43:18.272: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:43:18.273: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:43:18.273: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:43:18.273: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:43:38.278: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:43:38.278: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:43:38.279: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:43:38.279: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:43:38.279: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:43:42.154: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1070
2013-07-17 08:43:58.284: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:43:58.285: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:43:58.285: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:43:58.285: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:43:58.285: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:44:12.154: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1086
2013-07-17 08:44:18.291: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:44:18.292: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:44:18.292: [ora.diskmon][1167501632] [check] DiskmonAgent::check {
2013-07-17 08:44:18.292: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0
2013-07-17 08:44:18.292: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:44:38.296: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:44:38.297: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:44:38.297: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:44:38.297: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:44:38.297: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:44:42.163: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1102
2013-07-17 08:44:58.302: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:44:58.303: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:44:58.303: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:44:58.304: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:44:58.304: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:45:12.174: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1118
2013-07-17 08:45:18.309: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:45:18.309: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:45:18.310: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:45:18.310: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:45:18.310: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:45:38.315: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:45:38.316: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:45:38.316: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:45:38.316: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:45:38.316: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:45:42.183: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1134
2013-07-17 08:45:58.312: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:45:58.313: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:45:58.313: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:45:58.313: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:45:58.313: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:46:12.192: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1150
2013-07-17 08:46:18.318: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:46:18.319: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:46:18.319: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:46:18.319: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:46:18.319: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:46:38.325: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:46:38.326: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:46:38.326: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:46:38.326: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:46:38.326: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:46:42.203: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1166
2013-07-17 08:46:58.332: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:46:58.333: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:46:58.333: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:46:58.333: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:46:58.333: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:47:12.203: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1182
2013-07-17 08:47:18.338: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:47:18.339: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:47:18.339: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:47:18.339: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:47:18.340: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:47:38.345: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:47:38.345: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:47:38.346: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:47:38.346: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:47:38.346: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:47:42.211: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1198
2013-07-17 08:47:58.351: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:47:58.352: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:47:58.352: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:47:58.352: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:47:58.352: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:48:12.220: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1214
2013-07-17 08:48:18.358: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:48:18.359: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:48:18.359: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:48:18.359: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:48:18.359: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:48:38.365: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:48:38.366: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:48:38.366: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:48:38.366: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:48:38.366: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:48:42.230: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1230
2013-07-17 08:48:58.370: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:48:58.371: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:48:58.371: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:48:58.371: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:48:58.371: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:49:12.238: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1246
2013-07-17 08:49:18.377: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1
2013-07-17 08:49:18.378: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1
2013-07-17 08:49:18.378: [ora.diskmon][1157011776] [check] DiskmonAgent::check {
2013-07-17 08:49:18.379: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0
2013-07-17 08:49:18.379: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE
2013-07-17 08:49:23.262: [    AGFW][1177991488] Agent received the message: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:1310
2013-07-17 0

Similar Messages

  • Oracle 11gR2 RAC Root.sh Failed On The Second Node

    Hello,
    When i installing Oracle 11gR2 RAC on AIX 7.1 , root.sh succeeds on first node but fails on the second node:
    I get error "Root.sh Failed On The Second Node With Error ORA-15018 ORA-15031 ORA-15025 ORA-27041 [ID 1459711.1]" within Oracle installation.
    Applies to:
    Oracle Server - 11gR2 RAC
    EMC VNX 500
    IBM AIX on POWER Systems (64-bit)
    in /dev/rhdiskpower0 does not show in kfod output on second node. It is an EMC multipath disk device.
    But the disk can be found with AIX command.
    any help!!
    Thanks

    the soluation that uninstall "EMC solutitons enabler" but in the machine i just find "EMC migration enabler" and conn't remove without remove EMC Powerpath.

  • Oracle Grid Infrastructure 11.2.0.2 root.sh hangs in the second node

    Hi,
    I am installing Oracle Grid Infrastructure 11.2.0.2.0 using VM's. Both the virtual machines are running RHEL 5.7 32 bit version.
    Until the point where the installation suggests to run two scripts as root user, everything runs fine.
    root.sh runs successfully on the first node.
    On the second node it hangs at,
    Now product-specific root actions will be performed.
    Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

    user10193468 wrote:
    Hi,
    Following is the last few lines of the $GRID_HOME/cfgtoollogs/crsconfig/rootcrs_<node>.log file
    2012-03-19 17:20:17: OLR successfully created or upgraded
    2012-03-19 17:20:17: /u01/app/11.2.0/grid/bin/clscfg -localadd
    2012-03-19 17:20:17: Executing /u01/app/11.2.0/grid/bin/clscfg -localadd
    2012-03-19 17:20:17: Executing cmd: /u01/app/11.2.0/grid/bin/clscfg -localadd
    I am using ASM for OCR and Voting Disks.
    Private network is up and running.Please let me know what have you set in asm_diskstring parameter?
    I have find following note from metalink which says that you should not set asm_diskstring to '/dev/' or '/dev/*'. This is not supported
    root.sh Hangs or Reboots Node on Non First Node if ASM Discovery String asm_diskstring is Set to /dev/ or /dev/* [ID 1356567.1]

  • RAC 11gR2 cluster installation: root.sh failed on the 1st node

    Hi,
    Does anybody know why is possible when I run the root.sh on the 1st node, during the Oracle 11gR2 RAC installation (cluster installation) to get the following error?
    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /oracle/grid
    Enter the full pathname of the local bin directory: [usr/local/bin]:
    The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y
    Copying dbhome to /usr/local/bin ...
    The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y
    Copying oraenv to /usr/local/bin ...
    The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y
    Copying coraenv to /usr/local/bin ...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-06-29 14:17:43: Parsing the host name
    2010-06-29 14:17:43: Checking for super user privileges
    2010-06-29 14:17:43: User has super user privileges
    Using configuration parameter file: /oracle/grid/crs/install/crsconfig_params
    Creating trace directory
    User oracle has the required capabilities to run CSSD in realtime mode
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'system'..
    Operation successful.
    root wallet
    root wallet cert
    root cert export
    peer wallet
    profile reader wallet
    pa wallet
    peer wallet keys
    pa wallet keys
    peer cert request
    pa cert request
    peer cert
    pa cert
    peer root cert TP
    profile reader root cert TP
    pa root cert TP
    peer pa cert TP
    pa peer cert TP
    profile reader pa cert TP
    profile reader peer cert TP
    peer user cert
    pa user cert
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-2672: Attempting to start 'ora.gipcd' on 'trz1test_rac'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.gipcd' on 'trz1test_rac' succeeded
    CRS-2676: Start of 'ora.mdnsd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.gpnpd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'trz1test_rac'
    CRS-2676: Start of 'ora.cssdmonitor' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'trz1test_rac'
    CRS-2672: Attempting to start 'ora.diskmon' on 'trz1test_rac'
    CRS-2676: Start of 'ora.diskmon' on 'trz1test_rac' succeeded
    CRS-2676: Start of 'ora.cssd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.ctssd' on 'trz1test_rac' succeeded
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'system'..
    Operation successful.
    CRS-2672: Attempting to start 'ora.crsd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.crsd' on 'trz1test_rac' succeeded
    Now formatting voting disk: /data_gpfs/oracle/crs/vdsk.
    CRS-4603: Successful addition of voting disk /data_gpfs/oracle/crs/vdsk.
    ## STATE File Universal Id File Name Disk group
    1. ONLINE 653624f2aa1f4f83bf774e8052889a32 (/data_gpfs/oracle/crs/vdsk) []
    Located 1 voting disk(s).
    CRS-2673: Attempting to stop 'ora.crsd' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.crsd' on 'trz1test_rac' succeeded
    CRS-2673: Attempting to stop 'ora.ctssd' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.ctssd' on 'trz1test_rac' succeeded
    CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.cssdmonitor' on 'trz1test_rac' succeeded
    CRS-2673: Attempting to stop 'ora.cssd' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.cssd' on 'trz1test_rac' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.gpnpd' on 'trz1test_rac' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.gipcd' on 'trz1test_rac' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'trz1test_rac'
    CRS-2677: Stop of 'ora.mdnsd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.mdnsd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.mdnsd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.gipcd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.gpnpd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'trz1test_rac'
    CRS-2676: Start of 'ora.cssdmonitor' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'trz1test_rac'
    CRS-2672: Attempting to start 'ora.diskmon' on 'trz1test_rac'
    CRS-2676: Start of 'ora.diskmon' on 'trz1test_rac' succeeded
    CRS-2676: Start of 'ora.cssd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.ctssd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.crsd' on 'trz1test_rac' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'trz1test_rac'
    CRS-2676: Start of 'ora.evmd' on 'trz1test_rac' succeeded
    */oracle/grid/bin/srvctl start nodeapps -n trz1test_rac ... failed*
    Configure Oracle Grid Infrastructure for a Cluster ... failed
    This is because ora.eONS daemon is not starting. There is a Metalink note that we MIGHT start this daemon manually ... but this is not working.
    *./srvctl status nodeapps -n trz1test_rac*
    -n <node_name> option has been deprecated.
    VIP trz1test_rac_vip is enabled
    VIP trz1test_rac_vip is running on node: trz1test_rac
    Network is enabled
    Network is running on node: trz1test_rac
    GSD is disabled
    GSD is not running on node: trz1test_rac
    ONS is enabled
    ONS daemon is running on node: trz1test_rac
    eONS is enabled
    eONS daemon is not running on node: trz1test_rac

    I run my clusterware/DB on AIX 5.3
    When I run runcluvfy.sh here are the things which are not passing:
    Check: Node connectivity of subnet "192.168.1.0"
    Source Destination Connected?
    trz2test_rac:en5 trz2test_rac:en5 yes
    trz2test_rac:en5 trz1test_rac:en5 yes
    trz2test_rac:en5 trz1test_rac:en5 yes
    trz2test_rac:en5 trz1test_rac:en5 yes
    trz2test_rac:en5 trz1test_rac:en5 yes
    trz1test_rac:en5 trz1test_rac:en5 yes
    Result: Node connectivity passed for subnet "192.168.1.0" with node(s) trz2test_rac,trz1test_rac
    Check: TCP connectivity of subnet "192.168.1.0"
    Source Destination Connected?
    trz1test_rac:192.168.1.140 trz2test_rac:192.168.1.142 failed
    trz1test_rac:192.168.1.140 trz2test_rac:192.168.1.142 failed
    Result: TCP connectivity check failed for subnet "192.168.1.0"
    NTP daemon slewing option check failed on some nodes
    PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
    Result: Clock synchronization check using Network Time Protocol(NTP) failed
    NTP mustn't be a problem I guess as the date are identical on the 2 nodes.
    I have no idea how to fix the TCP connectivity issue with the subnet "192.168.1.0". Some posts wrote that could be a firewall issue. Are there any other causes ?
    Thanks to all,
    Paul

  • Grid installation: root.sh failed on the first node on Solaris cluster 4.1

    Hi all,
    I'm trying to install the Grid (11.2.0.3.0) on the 2 node-clusters (OSC 4.1).
    When I run the root.sh on the first node, I got the out put as follow:
    xha239080-root-5.11# root.sh
    Performing root user operation for Oracle 11g
    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /Grid/CRShome
    Enter the full pathname of the local bin directory: [/usr/local/bin]:
    /usr/local/bin is read only. Continue without copy (y/n) or retry (r)? [y]:
    Warning: /usr/local/bin is read only. No files will be copied.
    Creating /var/opt/oracle/oratab file...
    Entries will be added to the /var/opt/oracle/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    Using configuration parameter file: /Grid/CRShome/crs/install/crsconfig_params
    Creating trace directory
    User ignored Prerequisites during installation
    OLR initialization - successful
    root wallet
    root wallet cert
    root cert export
    peer wallet
    profile reader wallet
    pa wallet
    peer wallet keys
    pa wallet keys
    peer cert request
    pa cert request
    peer cert
    pa cert
    peer root cert TP
    profile reader root cert TP
    pa root cert TP
    peer pa cert TP
    pa peer cert TP
    profile reader pa cert TP
    profile reader peer cert TP
    peer user cert
    pa user cert
    Adding Clusterware entries to inittab
    CRS-2672: Attempting to start 'ora.mdnsd' on 'xha239080'
    CRS-2676: Start of 'ora.mdnsd' on 'xha239080' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'xha239080'
    CRS-2676: Start of 'ora.gpnpd' on 'xha239080' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'xha239080'
    CRS-2672: Attempting to start 'ora.gipcd' on 'xha239080'
    CRS-2676: Start of 'ora.cssdmonitor' on 'xha239080' succeeded
    CRS-2676: Start of 'ora.gipcd' on 'xha239080' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'xha239080'
    CRS-2672: Attempting to start 'ora.diskmon' on 'xha239080'
    CRS-2676: Start of 'ora.diskmon' on 'xha239080' succeeded
    CRS-2676: Start of 'ora.cssd' on 'xha239080' succeeded
    ASM created and started successfully.
    Disk Group DATA created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-4256: Updating the profile
    Successful addition of voting disk 9cdb938773bc4f16bf332edac499fd06.
    Successful addition of voting disk 842907db11f74f59bf65247138d6e8f5.
    Successful addition of voting disk 748852d2a5c84f72bfcd50d60f65654d.
    Successfully replaced voting disk group with +DATA.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ## STATE File Universal Id File Name Disk group
    1. ONLINE 9cdb938773bc4f16bf332edac499fd06 (/dev/did/rdsk/d10s6) [DATA]
    2. ONLINE 842907db11f74f59bf65247138d6e8f5 (/dev/did/rdsk/d8s6) [DATA]
    3. ONLINE 748852d2a5c84f72bfcd50d60f65654d (/dev/did/rdsk/d9s6) [DATA]
    Located 3 voting disk(s).
    Start of resource "ora.cssd" failed
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'xha239080'
    CRS-2672: Attempting to start 'ora.gipcd' on 'xha239080'
    CRS-2676: Start of 'ora.cssdmonitor' on 'xha239080' succeeded
    CRS-2676: Start of 'ora.gipcd' on 'xha239080' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'xha239080'
    CRS-2672: Attempting to start 'ora.diskmon' on 'xha239080'
    CRS-2676: Start of 'ora.diskmon' on 'xha239080' succeeded
    CRS-2674: Start of 'ora.cssd' on 'xha239080' failed
    CRS-2679: Attempting to clean 'ora.cssd' on 'xha239080'
    CRS-2681: Clean of 'ora.cssd' on 'xha239080' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'xha239080'
    CRS-2677: Stop of 'ora.gipcd' on 'xha239080' succeeded
    CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'xha239080'
    CRS-2677: Stop of 'ora.cssdmonitor' on 'xha239080' succeeded
    CRS-5804: Communication error with agent process
    CRS-4000: Command Start failed, or completed with errors.
    Failed to start Oracle Grid Infrastructure stack
    Failed to start Cluster Synchorinisation Service in clustered mode at /Grid/CRShome/crs/install/crsconfig_lib.pm line 1211.
    /Grid/CRShome/perl/bin/perl -I/Grid/CRShome/perl/lib -I/Grid/CRShome/crs/install /Grid/CRShome/crs/install/rootcrs.pl execution failed
    xha239080-root-5.11# history
    checking the ocssd.log, I see some thing as follow:
    2013-09-16 18:46:24.238: [    CSSD][1]clssscmain: Starting CSS daemon, version 11.2.0.3.0, in (clustered) mode with uniqueness value 1379371584
    2013-09-16 18:46:24.239: [    CSSD][1]clssscmain: Environment is production
    2013-09-16 18:46:24.239: [    CSSD][1]clssscmain: Core file size limit extended
    2013-09-16 18:46:24.248: [    CSSD][1]clssscmain: GIPCHA down 1
    2013-09-16 18:46:24.249: [    CSSD][1]clssscGetParameterOLR: OLR fetch for parameter logsize (8) failed with rc 21
    2013-09-16 18:46:24.250: [    CSSD][1]clssscExtendLimits: The current soft limit for file descriptors is 65536, hard limit is 65536
    2013-09-16 18:46:24.250: [    CSSD][1]clssscExtendLimits: The current soft limit for locked memory is 4294967293, hard limit is 4294967293
    2013-09-16 18:46:24.250: [    CSSD][1]clssscGetParameterOLR: OLR fetch for parameter priority (15) failed with rc 21
    2013-09-16 18:46:24.250: [    CSSD][1]clssscSetPrivEnv: Setting priority to 4
    2013-09-16 18:46:24.253: [    CSSD][1]clssscSetPrivEnv: unable to set priority to 4
    2013-09-16 18:46:24.253: [    CSSD][1]SLOS: cat=-2, opn=scls_mem_lockdown, dep=11, loc=mlockall
    unable to lock memory
    2013-09-16 18:46:24.253: [    CSSD][1](:CSSSC00011:)clssscExit: A fatal error occurred during initialization
    Do anyone have any idea what going on and how can I fix it ?

    Hi,
    solaris has several issues with DISM, e.g.:
    Solaris 10 and Solaris 11 Shared Memory Locking May Fail (Doc ID 1590151.1)
    Sounds like Solaris Cluster  has a similar bug. A "workaround" is to reboot the (cluster) zone, that "fixes" the mlock error. This bug was introduced with updates in september, atleast to our environment (Solaris 11.1). Prior i did not have the issue and now i have to restart the entire zone, whenever i stop crs.
    With 11.2.0.3 the root.sh script can be rerun without prior cleaning up, so you should be able to continue installation at that point after the reboot. After the root.sh completes some configuration assistants need to be run, to complete the installation. You need to execute this manually as you wipe your oui session
    Kind Regards
    Thomas

  • Root.sh fails inf 11gr2 2 node hp installation

    Hello,
    Env Details:
    This is an 2node RAC 11gR2 installation on HP 11.3.
    Following is the issue:
    The GRid infrastructure installation fails at the point of running root.sh with the following errors on node 1,
    CRS-2677: Stop of 'ora.cssdmonitor' on 'vpar1' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'vpar1'
    CRS-2677: Stop of 'ora.gpnpd' on 'vpar1' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'vpar1'
    CRS-2677: Stop of 'ora.mdnsd' on 'vpar1' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'vpar1'
    CRS-2677: Stop of 'ora.gipcd' on 'vpar1' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    CRS-2672: Attempting to start 'ora.gipcd' on 'vpar1'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'vpar1'
    CRS-2676: Start of 'ora.gipcd' on 'vpar1' succeeded
    CRS-2676: Start of 'ora.mdnsd' on 'vpar1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'vpar1'
    CRS-2676: Start of 'ora.gpnpd' on 'vpar1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'vpar1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'vpar1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'vpar1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'vpar1'
    CRS-2676: Start of 'ora.diskmon' on 'vpar1' succeeded
    CRS-2674: Start of 'ora.cssd' on 'vpar1' failed
    CRS-2679: Attempting to clean 'ora.cssd' on 'vpar1'
    CRS-2678: 'ora.cssd' on 'vpar1' has experienced an unrecoverable failure
    CRS-0267: Human intervention required to resume its availability.
    CRS-2673: Attempting to stop 'ora.diskmon' on 'vpar1'
    CRS-2677: Stop of 'ora.diskmon' on 'vpar1' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'vpar1'
    CRS-2677: Stop of 'ora.gpnpd' on 'vpar1' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'vpar1'
    CRS-2677: Stop of 'ora.mdnsd' on 'vpar1' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'vpar1'
    CRS-2677: Stop of 'ora.gipcd' on 'vpar1' succeeded
    *CRS-4000: Command Start failed, or completed with errors.
    Command return code of 1 (256) from command: /orabinary1/cluster/bin/crsctl start resource ora.ctssd -init
    Start of resource "ora.ctssd -init" failed
    Clusterware exclusive mode start of resource ora.ctssd failed
    CRS-2500: Cannot stop resource 'ora.crsd' as it is not running
    CRS-4000: Command Stop failed, or completed with errors.
    Command return code of 1 (256) from command: /orabinary1/cluster/bin/crsctl stop resource ora.crsd -init
    Stop of resource "ora.crsd -init" failed
    Failed to stop CRSD
    CRS-2500: Cannot stop resource 'ora.asm' as it is not running
    CRS-4000: Command Stop failed, or completed with errors.
    Command return code of 1 (256) from command: /orabinary1/cluster/bin/crsctl stop resource ora.asm -init
    Stop of resource "ora.asm -init" failed
    Failed to stop ASM*
    CRS-2679: Attempting to clean 'ora.cssd' on 'vpar1'
    CRS-2681: Clean of 'ora.cssd' on 'vpar1' succeeded
    Initial cluster configuration failed.
    the ocrcheck command fails with :
    2010-02-25 14:57:08.610: [  OCRASM][1]proprasmo: Failed to open file in dirty mode
    2010-02-25 14:57:08.610: [  OCRASM][1]proprasmo: Error in open/create file in dg [newgrid]
    [  OCRASM][1]SLOS : SLOS: cat=8, opn=kgfolclcpi1, dep=210, loc=kgfokge
    AMDU-00210: No disks found in diskgroup NEWGRID
    AMDU-00210: No disks found in diskgroup NEWGRID
    2010-02-25 14:57:08.630: [  OCRASM][1]proprasmo: kgfoCheckMount returned [7]
    2010-02-25 14:57:08.630: [  OCRASM][1]proprasmo: The ASM instance is down
    the disks are already owned by grid:asmadmin group i have already done chmod 660 for all of them.
    the precheck during the installation does not fail with any particular error and goes ahead to the point of asking to run root.sh
    kindly help.
    regards
    pg

    Unfortunately I can't run any -post checks because the CRS doesn't install correctly. The error occurs during the intial root.sh run on the first node, and doens't allow me to continue.
    cluvfy stage -post crsinst -n sun277z1,sun278z1
    Performing post-checks for cluster services setup
    Checking node reachability...
    Node reachability check passed from node "sun277z1"
    Checking user equivalence...
    User equivalence check passed for user "oracle"
    ERROR:
    CRS is not installed on any of the nodes
    Verification cannot proceed

  • Can't start instance on the second node

    I install RAC on Redhat AS 2.1 with LVM.
    After run "srvctl start db -d RAC" or "srvctl start instance -d RAC -i RAC2",
    the instance on node2 is not running.
    The error message say that the instance is already running even I just start RAC2 only. But the RAC1 runs well.
    I don't know it is the file system or the environment problem.
    Thanks

    I installed Oracle 9ir2 RAC on two RH advance server 2.1. There are some error on creating database, OLAP and ODM.
    Starting db instance on node1 is fine, but the second node is not running. Then I install database on the second node
    With software only(to get sqlplus to work), and start instance by SQL. I found the SGA is not the same.
    To get my SCSI and LVM to work, I recompiler the kernel. Is that possible two different kernel image to make SGA has
    Different size? Will the SGA size be the same on Real Application Cluster? I am sure the sem, shmmax have the same
    Value on both nodes. How can I change the role of node1 to node2 manually in ocm?
    Thanks

  • UCCE calls are failing at the RunExternalScript node

    Dear members,
    It is a UCCE Lab environment with CVP version 9.0
    My issue is that all the calls are failing at the RunExternalScript node.
    In the CVP logs, the below error appears.
    External VXML located at http://192.168.2.215:7000/CVP/en-us/app/Server?_dnis=3000&application=Test&callid=565&_ani=565 resulted in a bad fetch 
    Attached are the ICM script, CVP script, CVP logs and rtr logs.
    Is there any variable that needs to be added in the ICM script?
    Thank you in advance for your help.
    Lara

    Thank you Jameson for replying.
    I was using the variable user.microappp.sys_media_lib instead of user.microapp.app_media_lib.
    But even after I corrected this variable, I'm still facing a bad fetch error.
    70: 192.168.2.215: Mar 09 2015 11:24:49.294 +0200: %CVP_9_0_IVR-3-CALL_ERROR:  CALLGUID=0145638000010000000000421500A8C0 DNIS=012345678918 CVP VXML Server encountered a Bad-Fetch Error - URL: http://192.168.2.215:7000/CVP/en-us/../Server?_dnis=3000&application=Test&callid=565&_ani=565 (Client: 10.1.1.103) [id:3023] 
    71: 192.168.2.215: Mar 09 2015 11:24:49.294 +0200: %CVP_9_0_IVR-3-CALL_ERROR:  RunScript Error from 10.1.1.103 [CVP_BAD_FETCH(45)] CALLGUID: 0145638000010000000000421500A8C0 DNIS=012345678918 {VRUScriptName: 'GS,Server,V' ConfigParam: ''} [id:3023] 
    108: 192.168.2.215: Mar 09 2015 11:24:49.310 +0200: %CVP_9_0_SIP-3-SIP_CALL_ERROR:  CALLGUID = 0145638000010000000000421500A8C0 LEGID = 50331337-C57A11E4-BCB6D284-70B23AF8 - [INBOUND] - DIALOGUE_FAILURE from ICM Router sends 404 rejection to call. errorcode=15 [id:5004] 
    111: 192.168.2.215: Mar 09 2015 11:24:49.310 +0200: %CVP_9_0_SIP-3-SIP_CALL_ERROR:  CALLGUID = 0145638000010000000000421500A8C0 LEGID = 50331337-C57A11E4-BCB6D284-70B23AF8 - [INBOUND] - ABNORMALLY ENDING - SIP code [404], Reason Hdr [SIP;cause=404] Not Found, GW call using SURV TCL flag [false], NON NORMAL flag [true], DNIS [3000], ANI [565] with AGE (msecs) 54334 and Call History : 012345678918|-1; [id:5004] 
    120: 192.168.2.215: Mar 09 2015 11:24:50.152 +0200: %CVP_9_0_SIP-3-SIP_CALL_ERROR:  CALLGUID = 0145638000010000000000421500A8C0 LEGID = 50331337-C57A11E4-BCB6D284-70B23AF8 - [INBOUND]: Refer failed with 503 - Service Unavailable. May be a problem with Routing Configuration or Gateway Dial-Peer. [id:5004] 
    122: 192.168.2.215: Mar 09 2015 11:24:50.152 +0200: %CVP_9_0_SIP-3-SIP_CALL_ERROR:  CALLGUID = 0145638000010000000000421500A8C0 LEGID = 50331337-C57A11E4-BCB6D284-70B23AF8 - [INBOUND] - ABNORMALLY ENDING - SIP code [404], Reason Hdr [SIP;cause=404] Not Found, GW call using SURV TCL flag [false], NON NORMAL flag [true], DNIS [3000], ANI [565] with AGE (msecs) 55176 and Call History : 012345678918|-1; [id:5004] 
    I can play the .wav file from the browser but the calls are still failing at the Run External Script node.
    I restarted the CVP  Call/VXML server but with no luck.
    Your help is highly appreciated.
    Thank you.
    Lara

  • Sap service fails after switching to the second node in HA system.

    Hello all,
    We have installed the HA mscs cluster installation for ECC6.0(my SAP erp 2005) system.The ASCS+SCS isntance  and the DB instance are the failover resources.
    When the ASCS+SCS(sap cluster group) fail over to the next node the service sap_<sid>_00 (ascs) and sap_<sid>_01(scs) fails to start automatically  and we have to manually start the instance from the services console by giving the sapservice<sid> password to start the services.
    when we try to start the service without giving the password then the system gives a message unable to start due to logon failure and after giving the password it shows that the user has been been granted to logon as a service user.
    Pl help resolve this issue.
    thx
    satyajit

    A change to the zpool_import() management of the zpool.cache file, as delivered by Solaris 10 kernel patches 137137-09 (for SPARC) or 137138-09 (for x86), might cause systems that have their shared ZFS (zfs(1M)) storage pools under the control of HAStoragePlus to be simultaneously imported on multiple cluster nodes. Importing a ZFS storage pool on multiple cluster nodes will result in pool corruption, which might cause data integrity issues or cause a cluster node to panic.
    To avoid this problem, install Solaris 10 patch 139579-02 (for SPARC) or 139580-02 (for x86) immediately after you install 137137-09 or 137138-09 but before you reboot the cluster nodes.
    Alternatively, only on the Solaris 10 5/08 OS, remove the affected patch before any ZFS pools are simultaneously imported to multiple cluster nodes. You cannot remove patch 137137-09 or 137138-09 from the Solaris 10 10/08 OS, because these patches are preinstalled on that release.

  • Failure at final check of Oracle CRS stack.10  on the second node

    Hi,
    I am trying to install Oracle Clusterware 10.2.0.1.0 in VM machines (2 nodes config) in Linux (OEL5) using VMware Server (2.0). Everything went very well one the first node upto running the root.sh. Running root.sh ended with Failure at final check of Oracle CRS stack 10 error.
    RAC1 root.sh output
    [root@rac1 crs]# ./root.sh
    WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
    WARNING: directory '/u01/crs/oracle/product' is not owned by root
    WARNING: directory '/u01/crs/oracle' is not owned by root
    WARNING: directory '/u01/crs' is not owned by root
    WARNING: directory '/u01' is not owned by root
    Checking to see if Oracle CRS stack is already configured
    /etc/oracle does not exist. Creating it now.
    Setting the permissions on OCR backup directory
    Setting up NS directories
    Oracle Cluster Registry configuration upgraded successfully
    WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
    WARNING: directory '/u01/crs/oracle/product' is not owned by root
    WARNING: directory '/u01/crs/oracle' is not owned by root
    WARNING: directory '/u01/crs' is not owned by root
    WARNING: directory '/u01' is not owned by root
    assigning default hostname rac1 for node 1.
    assigning default hostname rac2 for node 2.
    Successfully accumulated necessary OCR keys.
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 1: rac1 rac1-priv rac1
    node 2: rac2 rac2-priv rac2
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Now formatting voting device: /dev/raw/raw2
    Format of 1 voting devices complete.
    Startup will be queued to init within 90 seconds.
    Adding daemons to inittab
    Expecting the CRS daemons to be up within 600 seconds.
    CSS is active on these nodes.
    rac1
    CSS is inactive on these nodes.
    rac2
    Local node checking complete.
    Run root.sh on remaining nodes to start CRS daemons.
    [root@rac1 crs]#
    RAC2 root.sh output
    [root@rac2 crs]# ./root.sh
    WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
    WARNING: directory '/u01/crs/oracle/product' is not owned by root
    WARNING: directory '/u01/crs/oracle' is not owned by root
    WARNING: directory '/u01/crs' is not owned by root
    WARNING: directory '/u01' is not owned by root
    Checking to see if Oracle CRS stack is already configured
    /etc/oracle does not exist. Creating it now.
    Setting the permissions on OCR backup directory
    Setting up NS directories
    Oracle Cluster Registry configuration upgraded successfully
    WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
    WARNING: directory '/u01/crs/oracle/product' is not owned by root
    WARNING: directory '/u01/crs/oracle' is not owned by root
    WARNING: directory '/u01/crs' is not owned by root
    WARNING: directory '/u01' is not owned by root
    assigning default hostname rac1 for node 1.
    assigning default hostname rac2 for node 2.
    Successfully accumulated necessary OCR keys.
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 1: rac1 rac1-priv rac1
    node 2: rac2 rac2-priv rac2
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Now formatting voting device: /dev/raw/raw2
    Format of 1 voting devices complete.
    Startup will be queued to init within 90 seconds.
    Adding daemons to inittab
    Expecting the CRS daemons to be up within 600 seconds.
    Failure at final check of Oracle CRS stack.
    10
    [root@rac2 crs]#
    Output of alterrac2.log
    [root@rac2 rac2]# more alertrac2.log
    2009-08-14 23:02:44.699
    [client(5935)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible. Details in /u01/crs/oracle/product/10.2.
    0/crs/log/rac2/client/ocrconfig_5935.log.
    2009-08-14 23:02:44.704
    [client(5935)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible. Details in /u01/crs/oracle/product/10.2.
    0/crs/log/rac2/client/ocrconfig_5935.log.
    2009-08-14 23:02:44.707
    [client(5935)]CRS-1006:The OCR location /dev/raw/raw1 is inaccessible. Details in /u01/crs/oracle/product/10.2.
    0/crs/log/rac2/client/ocrconfig_5935.log.
    2009-08-14 23:02:44.864
    [client(5935)]CRS-1001:The OCR was formatted using version 2.
    2009-08-14 23:02:50.339
    [client(6004)]CRS-1801:Cluster crs configured with nodes rac1 rac2 .
    2009-08-14 23:05:07.603
    [cssd(6600)]CRS-1605:CSSD voting file is online: /dev/raw/raw2. Details in /u01/crs/oracle/product/10.2.0/crs/l
    og/rac2/cssd/ocssd.log.
    [root@rac2 rac2]#
    Since raw devices are not supported from OEL5, I did do the workaround in *63-oracle-raw.rules file under /etc/udev/rules.d* dir.
    ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"
    ACTION=="add", KERNEL=="sdc1", RUN+="/bin/raw /dev/raw/raw2 %N"
    ACTION=="add", KERNEL=="sdd1", RUN+="/bin/raw /dev/raw/raw3 %N"
    ACTION=="add", KERNEL=="sde1", RUN+="/bin/raw /dev/raw/raw4 %N"
    ACTION=="add", KERNEL=="sdf1", RUN+="/bin/raw /dev/raw/raw5 %N"
    KERNEL=="raw[1-2]*", OWNER="root", GROUP="oinstall", MODE="640"
    KERNEL=="raw[3-5]*", OWNER="oracle", GROUP="oinstall", MODE="640"
    One thing I have noticed after running root.sh on both the nodes is the permissons on raw devices changed from
    Before root.sh
    [root@rac2 crs]# ls -ls /dev/raw*
    0 crw------- 1 root root 162, 0 Aug 14 22:42 /dev/rawctl
    /dev/raw:
    total 0
    0 crw-r----- 1 root oinstall 162, 1 Aug 14 22:42 raw1
    0 crw-r----- 1 root oinstall 162, 2 Aug 14 22:42 raw2
    0 crw-r----- 1 oracle oinstall 162, 3 Aug 14 22:42 raw3
    0 crw-r----- 1 oracle oinstall 162, 4 Aug 14 22:42 raw4
    0 crw-r----- 1 oracle oinstall 162, 5 Aug 14 22:42 raw5
    to
    [root@rac2 crs]# ls -ls /dev/raw*
    0 crw------- 1 root root 162, 0 Aug 14 22:31 /dev/rawctl
    /dev/raw:
    total 0
    0 crw-r----- 1 root oinstall 162, 1 Aug 14 22:56 raw1
    0 crw-r--r-- 1 oracle oinstall 162, 2 Aug 14 23:01 raw2
    0 crw-r----- 1 oracle oinstall 162, 3 Aug 14 22:31 raw3
    0 crw-r----- 1 oracle oinstall 162, 4 Aug 14 22:31 raw4
    0 crw-r----- 1 oracle oinstall 162, 5 Aug 14 22:31 raw5
    [root@rac1 crs]#
    My shared disk listing
    [root@www shared]# ls -ltr
    total 8780
    -rw------- 1 root root 640 Aug 14 22:43 votingdisk.vmdk
    -rw------- 1 root root 598 Aug 14 22:43 ocr.vmdk
    -rw------- 1 root root 604 Aug 14 22:43 asm3.vmdk
    -rw------- 1 root root 604 Aug 14 22:43 asm2.vmdk
    -rw------- 1 root root 604 Aug 14 22:43 asm1.vmdk
    -rw------- 1 root root 65536 Aug 14 22:44 votingdisk-s006.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s005.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s004.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s003.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 votingdisk-s002.vmdk
    -rw------- 1 root root 393216 Aug 14 22:44 votingdisk-s001.vmdk
    -rw------- 1 root root 65536 Aug 14 22:44 ocr-s006.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 ocr-s005.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 ocr-s004.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 ocr-s003.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 ocr-s002.vmdk
    -rw------- 1 root root 393216 Aug 14 22:44 ocr-s001.vmdk
    -rw------- 1 root root 65536 Aug 14 22:44 asm3-s006.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm3-s005.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm3-s004.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm3-s003.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm3-s002.vmdk
    -rw------- 1 root root 393216 Aug 14 22:44 asm3-s001.vmdk
    -rw------- 1 root root 65536 Aug 14 22:44 asm2-s006.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm2-s005.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm2-s004.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm2-s003.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm2-s002.vmdk
    -rw------- 1 root root 393216 Aug 14 22:44 asm2-s001.vmdk
    -rw------- 1 root root 65536 Aug 14 22:44 asm1-s006.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm1-s005.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm1-s004.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm1-s003.vmdk
    -rw------- 1 root root 327680 Aug 14 22:44 asm1-s002.vmdk
    -rw------- 1 root root 393216 Aug 14 22:44 asm1-s001.vmdk
    [root@www shared]#
    I don't know how to fix this problem. I did go through many docs and metalink notes.
    I am new to RAC world. It took 3 days to come to this stage. Please help me.
    Thanks
    Leo

    Hi Surachart,
    Here is my messages output..
    */var/log/messages*
    Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for fe80::20c:29ff:fe6b:f9a8 on eth1.
    Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for 192.168.1.196 on eth1.
    Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for fe80::20c:29ff:fe6b:f99e on eth0.
    Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering new address record for 192.168.0.196 on eth0.
    Aug 20 14:05:01 rac2 avahi-daemon[3627]: Registering HINFO record with values 'I686'/'LINUX'.
    Aug 20 14:05:02 rac2 avahi-daemon[3627]: Server startup complete. Host name is rac2.local. Local service cookie is 927471131.
    Aug 20 14:05:03 rac2 avahi-daemon[3627]: Service "SFTP File Transfer on rac2" (/services/sftp-ssh.service) successfully established.
    Aug 20 14:05:08 rac2 smartd[3739]: smartd version 5.38 [i686-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
    Aug 20 14:05:08 rac2 smartd[3739]: Home page is http://smartmontools.sourceforge.net/
    Aug 20 14:05:08 rac2 smartd[3739]: Opened configuration file /etc/smartd.conf
    Aug 20 14:05:08 rac2 smartd[3739]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/hdc, opened
    Aug 20 14:05:08 rac2 kernel: hdc: drive_cmd: status=0x51 { DriveReady SeekComplete Error }
    Aug 20 14:05:08 rac2 kernel: hdc: drive_cmd: error=0x04 { AbortedCommand }
    Aug 20 14:05:08 rac2 kernel: ide: failed opcode was: 0xec
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/hdc, not ATA, no IDENTIFY DEVICE Structure
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sda, opened
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sda, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sda' to turn on SMART features
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdb, opened
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdb, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdb' to turn on SMART features
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdc, opened
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdc, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdc' to turn on SMART features
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdd, opened
    Aug 20 14:05:08 rac2 smartd[3739]: Device: /dev/sdd, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdd' to turn on SMART features
    Aug 20 14:05:09 rac2 smartd[3739]: Device: /dev/sde, opened
    Aug 20 14:05:09 rac2 smartd[3739]: Device: /dev/sde, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sde' to turn on SMART features
    Aug 20 14:05:10 rac2 smartd[3739]: Device: /dev/sdf, opened
    Aug 20 14:05:10 rac2 smartd[3739]: Device: /dev/sdf, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdf' to turn on SMART features
    Aug 20 14:05:10 rac2 smartd[3739]: Monitoring 0 ATA and 0 SCSI devices
    Aug 20 14:05:10 rac2 smartd[3741]: smartd has fork()ed into background mode. New PID=3741.
    Aug 20 14:05:13 rac2 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
    Aug 20 14:05:13 rac2 last message repeated 3 times
    Aug 20 14:05:27 rac2 gconfd (root-3967): starting (version 2.14.0), pid 3967 user 'root'
    Aug 20 14:05:27 rac2 gconfd (root-3967): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
    Aug 20 14:05:27 rac2 gconfd (root-3967): Resolved address "xml:readwrite:/root/.gconf" to a writable configuration source at position 1
    Aug 20 14:05:27 rac2 gconfd (root-3967): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
    Aug 20 14:05:29 rac2 gconfd (root-3967): Resolved address "xml:readwrite:/root/.gconf" to a writable configuration source at position 0
    Aug 20 14:05:29 rac2 hald: mounted /dev/hdc on behalf of uid 0
    Aug 20 14:05:29 rac2 hcid[3311]: Default passkey agent (:1.8, /org/bluez/applet) registered
    Aug 20 14:05:31 rac2 nm-system-settings: Loaded plugin ifcfg-rh: (c) 2007 - 2008 Red Hat, Inc. To report bugs please use the NetworkManager mailing list.
    Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-eth1 ...
    Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: read connection 'System eth1'
    Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-lo ...
    Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: error: Ignoring loopback device config.
    Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-eth0 ...
    Aug 20 14:05:31 rac2 nm-system-settings: ifcfg-rh: read connection 'System eth0'
    Aug 20 14:05:31 rac2 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
    Aug 20 14:05:32 rac2 last message repeated 4 times
    Aug 20 14:12:51 rac2 kernel: FS-Cache: Loaded
    Aug 20 14:22:06 rac2 xinetd[3488]: START: shell pid=5193 from=192.168.0.195
    Aug 20 14:22:06 rac2 xinetd[3488]: EXIT: shell status=0 pid=5193 duration=0(sec)
    Aug 20 14:22:07 rac2 xinetd[3488]: START: shell pid=5217 from=192.168.0.195
    Aug 20 14:22:07 rac2 xinetd[3488]: EXIT: shell status=0 pid=5217 duration=0(sec)
    Aug 20 14:22:07 rac2 xinetd[3488]: START: shell pid=5241 from=192.168.0.195
    Aug 20 14:22:07 rac2 xinetd[3488]: EXIT: shell status=0 pid=5241 duration=0(sec)
    Aug 20 14:22:16 rac2 xinetd[3488]: EXIT: shell status=0 pid=6236 duration=0(sec)
    Aug 20 14:22:16 rac2 xinetd[3488]: START: shell pid=6265 from=192.168.0.195
    Aug 20 14:22:16 rac2 xinetd[3488]: EXIT: shell status=0 pid=6265 duration=0(sec)
    Aug 20 14:22:16 rac2 xinetd[3488]: START: shell pid=6291 from=192.168.0.195
    Aug 20 14:22:17 rac2 xinetd[3488]: EXIT: shell status=0 pid=6291 duration=1(sec)
    Aug 20 14:22:17 rac2 xinetd[3488]: START: shell pid=6317 from=192.168.0.195
    Aug 20 14:22:17 rac2 xinetd[3488]: EXIT: shell status=0 pid=6317 duration=0(sec)
    [root@rac2 log]#

  • Root.sh fails during the last part of cluster installation.

    hi,
    I am trying to install clusterware for Oracle 11g on rhel 5.
    Everything seemed fine untill the end of the clusterware installation...
    Firstly i did not get the option to install orainstRoot.sh script, I mean the "Execute the configuration scripts" has only one script to install that is root.sh
    Well, i ran the script with root user, and i get the below error. At the end i have uploaded the cluster install log aswell
    [root@node1 ~]# sh /u01/app/crs/root.sh
    WARNING: directory '/u01/app' is not owned by root
    WARNING: directory '/u01' is not owned by root
    Checking to see if Oracle CRS stack is already configured
    Setting the permissions on OCR backup directory
    Setting up Network socket directories
    Oracle Cluster Registry configuration upgraded successfully
    The directory '/u01/app' is not owned by root. Changing owner to root
    The directory '/u01' is not owned by root. Changing owner to root
    Successfully accumulated necessary OCR keys.
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 1: node1 node1-priv node1
    node 2: node2 node2-priv node2
    node 3: node3 node3-priv node3
    Creating OCR keys for user 'root', privgrp 'root'..
    OCR create keys failed, OCR error 26.
    Failed to initialize Oracle Cluster Registry for cluster
    0
    The configuration assistant says
    Output generated from configuration assistant "Oracle Notification Server Configuration Assistant":
    Command = /u01/app/crs/install/onsconfig add_config node1:6251 node2:6251 node3:6251
    The ONS configuration failed to create
    Configuration assistant "Oracle Notification Server Configuration Assistant" failed
    The "/u01/app/crs/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any)efore executing the same.-----------------------------------------------------------------------------
    ToolFailedCommands file shows
    # Copyright (c) 1999, 2007, Oracle. All rights reserved.
    /u01/app/crs/install/onsconfig add_config cch_or_db1:*6251* cch_or_db2:*6251* cch_or_db3:*6251*
    /u01/app/crs/bin/oifcfg setif -global eth0/10.2.0.0:public eth1/192.168.2.0:cluster_interconnect
    /u01/app/crs/bin/cluvfy stage -post crsinst -n cch_or_db1,cch_or_db2,cch_or_db3
    I am wondering so as to what exactly is 6251, is this the port number for all the three nodes? Infact port numbers for the three nodes are something else. Should i attach those port numbers to the respective node names and run the failed commands individually??
    Here is the log file for the unsuccessful cluster install, i have uploaded
    http://stashbox.org/313748/installActions2008-12-04_01-05-26PM.log
    Any idea about this??

    Hi
    have your run cluster verify to check if your OS configuration and pre-requisities are done?

  • Why do apps install on the second page?

    I have restored the device and I still, even with plenty of room, have newly downloaded apps being added to my second page??? ***

    Slightly bumping because I believe I understand this strange phenomena...
    It would appear that the iPhone allows you to create folders, but the device still treats these apps even when they are put in folders as if they take a "Space" on the homescreen. That being said if you have more than a few third party applications installed they would " if you were to unfolder all your apps" end up on the second page.
    Please tell me I am wrong and this is a joke...

  • Auto redial fails on the second attempt

    the new feature on the version 7.2.60.103 automatically redial the contact if the first attempt wasn't successful. however the second attempt made automatically will result in the call being dropped.

    Hey Carolyn,
    Thank you for your reply. I tried resetting the SMC but I still have the same issue.
    It doesn't really matter if I put it to sleep from the Apple Menu or if I close the Lid. It always works the first time, but never the second time. What happens is that it stops responding directly after "getting" the sleep signal (either by closing the lid, then the whole screen is black and won't respond, or by choosing sleep from the apple menu, then the screen is not black but nothing responds). In both cases, nothing works, and I have to force the macbook to shut down (by holding the power button).
    Amazing thing, the fan gradually goes to max speed when this issue occurs (always).
    I never installed or used any new applications except one single update which required the system to restart (Safari 4).
    Cheers!
    Wissam

  • Why does FaceTime fail all the time?

    Can't talk to anyone it freezes and then fail b4 the pick up and says its connecting on one side but not the other.

    Unable to make or receive FaceTime calls after April 16, 2014

  • Why do I never hear the second chime when resetting the PRAM?

    I have tried many times to follow the directions in the Help topic "Restting PRAM" but I never "...hear the startup sound for the second time." What might I be doing wrong? Thanks for your input.   

    Cause you don't wait long enough.
    Hold Option-Command-P-R down until the cycle repeats at least 2-3 times: BONG!--dark screen--optical drive recalibrates--dark screen--BONG!------->

Maybe you are looking for