Facing prcr-1079 failed to start resource ora.orcl.db problem
Hi all,
I am about installing an oracle database with asm
I have installed ASM libraries and created asm instance with diskgroup successfully mounted. I started the ASM instance in the oragrid user.
Know I runned the DBCA command to create a database named ORCL from the oracle user,
I followed up the procedure and during creation phase (at 80 %) I got multiple errors,
: prcr-1079 failed to start resource ora.orcl.db
ORA-01031: insufficient privileges
is there anything wrong, or I need to grant system privileges for oracle over directories?
Thanks for help
HI,
Please check following link: PRCR-1079: Failed to start resource ora.test.db, ORA-01031,ORA-2674 | Just Innovation
# usermod -G asmadmin,asmdba,asmoper,dba grid
Now you will notice the Privileges are changed
# id grid
uid=501(grid) gid=1000(oinstall) groups=1000(oinstall),1100(dba),1300(asmadmin),1400(asmdba),1500(asmoper)
Now try to runInstaller again!
Thank you
Similar Messages
-
PRCR-1079 Failed to start resource ora.rac.db - during installation
Hi
After successful installation of Grid Infrasturcture I proceeded with database installation on clusterware and at the stage when the installer was creating clone database I got the following errors ( this was my 2nd attempt and I got the same errors both the time ) :
Errors:
PRCR-1079 : Failed to start resource ora.rac.db
ORA-01092 : ORACLE instance terminated. Disconnection forced
ORA-00704 : bootstrap process failure
ORA-00604 : error occurred at resursiive SQL level 2
ORA-01578 : ORACLE data block corrupted (file # 1, block # 5505)
ORA-01110 : data file 1:'+DATA/rac/datafile/system.256.799676855'
Process ID : 23498
Session ID : 63 Serial number 3
CRS-2674 Start of 'ora.rac.db' on 'rac2' failed
CRS-2632 There are no more servers to try to place resource 'ora.rac.db' on that would satisfy its placement policy
There are no logs on that node (rac2)
I am running Oracle Linux 5.4 64 bit
As mentioned above this was my 2nd attempt afresh and I got the same errors both the times, please let me know what is the problem as the rac2 is replica of rac2 in VMWare.
Thanks for your help
Rgds
THi
I tried again for the 3rd time and go the same error again, this time I rebuilt the node 2 - Can someone ple ase help me with this issue why it keeps failing on node 2 at the same stage for the 2rd time in a row.
Also please help me clone the database manually from node 1 to node 2 so I don't have to try to reinstall it again, there must be ways to do it
Thanks for your help in advance
Rgds
T -
Failed to start resource: Name: ora.racdb.db, node: null, filter: null, ms
Hi DBA's.
Im, running
Finalizing Installation 96% the following Warning:
[Thread-288] [ 2010-01-21 14:28:57.456 ARST ] [CRSNative.internalStartResource:352] Failed to start resource: Name: ora.racdb.db, node: null, filter: null, msg CRS-2674:
Start of 'ora.racdb.db' on 'linux2' failed
CRS-2678: 'ora.racdb.db' on 'linux2' has experienced an unrecoverable failure
CRS-0267: Human intervention required to resume its availability.
CRS-5807: Agent failed to process the message
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
[Thread-288] [ 2010-01-21 14:28:57.457 ARST ] [PostDBCreationStep.executeImpl:828] Exception while Starting with HA Database Resource PRCR-1079 : Failed to start resourc
e ora.racdb.db
CRS-2674: Start of 'ora.racdb.db' on 'linux2' failed
CRS-2678: 'ora.racdb.db' on 'linux2' has experienced an unrecoverable failure
CRS-0267: Human intervention required to resume its availability.
CRS-5807: Agent failed to process the message
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
oracle$ dbcaHi...
Now is Ok.
I did:
srvctl start instance -d racdb -i racdb2
[oracle@linux1 oracle]$ su - grid -c "crsctl status resource -w \"TYPE co 'ora'\" -t"
Password:
NAME TARGET STATE SERVER STATE_DETAILS
Local Resources
ora.CRS.dg
ONLINE ONLINE linux1
ONLINE ONLINE linux2
ora.FRA.dg
ONLINE ONLINE linux1
ONLINE ONLINE linux2
ora.LISTENER.lsnr
ONLINE ONLINE linux1
ONLINE ONLINE linux2
ora.RACDB_DATA.dg
ONLINE ONLINE linux1
ONLINE ONLINE linux2
ora.asm
ONLINE ONLINE linux1 Started
ONLINE ONLINE linux2 Started
ora.eons
ONLINE ONLINE linux1
ONLINE ONLINE linux2
ora.gsd
OFFLINE OFFLINE linux1
OFFLINE OFFLINE linux2
ora.net1.network
ONLINE ONLINE linux1
ONLINE ONLINE linux2
ora.ons
ONLINE ONLINE linux1
ONLINE ONLINE linux2
Cluster Resources
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE linux1
ora.linux1.vip
1 ONLINE ONLINE linux1
ora.linux2.vip
1 ONLINE ONLINE linux2
ora.oc4j
1 OFFLINE OFFLINE
ora.racdb.db
1 ONLINE ONLINE linux1 Open
2 ONLINE ONLINE linux2 Open
ora.scan1.vip
1 ONLINE ONLINE linux1
Thanks. -
Root.sh failed with CRS-2800: Can't start resource 'ora.ctssd' as it is
Cannot start resource 'ora.ctssd' as it is already in the INTERMEDIATE state.
I was trying to install two node RAC on linux REDHAT, I have encountered following errors:
Start of resource "ora.crsd" failed
CRS-2800: Cannot start resource 'ora.ctssd' as it is already in the INTERMEDIATE state on server
CRS-4000: Command Start failed, or completed with errors.
Clusterware exclusive mode start of Clusterware Ready Services failed at /u01/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 6475.
/u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed
How to fix this issue?
runCluvfy ran successfully.
I found out on the second node(oracle02), there is no any daemon running. But on the local installation node(oracle01):
[[email protected] bin]# ./crsctl stat res -t -init
NAME TARGET STATE SERVER STATE_DETAILS
Cluster Resources
ora.asm
1 ONLINE ONLINE oracle01 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE oracle01
ora.crf
1 OFFLINE OFFLINE
ora.crsd
1 ONLINE OFFLINE
ora.cssd
1 ONLINE ONLINE oracle01
ora.cssdmonitor
1 ONLINE ONLINE oracle01
ora.ctssd
1 ONLINE INTERMEDIATE oracle01 Wrong check return.
ora.diskmon
1 ONLINE ONLINE oracle01
ora.drivers.acfs
1 ONLINE ONLINE oracle01
ora.evmd
1 OFFLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE oracle01
ora.gpnpd
1 ONLINE ONLINE oracle01
ora.mdnsd
1 ONLINE ONLINE oracle01
[[email protected] bin]#
How to proceed next step? what to fix in order for my installation to be successful.
Thanks in advance.OK, please post the css last lines in the log.
Also, if you have alert log (for crs, it should be under $GI_HOME/log/<hostname>) post it as well.
It's a bit complicated to trace it here, I'll do my best, but you might want to open an SR on this.
Liron -
Database failed to start up - ORA-03113: end-of-file on communication chann
Hi,
We had power failure, and i had to restore the database from offline backup.
after restore i tried to start the database and i received following errors.
SQL> startup
ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance
ORACLE instance started.
Total System Global Area 663961600 bytes
Fixed Size 2184056 bytes
Variable Size 394267784 bytes
Database Buffers 260046848 bytes
Redo Buffers 7462912 bytes
Database mounted.
ORA-01589: must use RESETLOGS or NORESETLOGS option for database open
SQL>
SQL> alter database open resetlogs;
alter database open resetlogs
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 7179
Session ID: 227 Serial number: 3
SQL>
I checked the alert.log
Tue Jun 19 08:06:33 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options.
Using parameter settings in server-side spfile /oracle/PRD/112_64/dbs/spfilePRD.ora
System parameters with non-default values:
processes = 900
sessions = 1800
event = "10027"
event = "10028"
event = "10142"
event = "10183"
event = "10191"
event = "38068 level 100"
event = "38085"
event = "38087"
event = "44951 level 1024"
shared_pool_size = 352M
filesystemio_options = "setall"
control_files = "/oracle/PRD/sapdata1/cntrl/cntrlPRD.dbf"
control_files = "/oracle/PRD/origlogA/cntrl/cntrlPRD.dbf"
control_files = "/oracle/PRD/saparch/cntrl/cntrlPRD.dbf"
control_file_record_keep_time= 30
db_block_size = 8192
db_cache_size = 248M
compatible = "11.2.0"
log_archive_dest_1 = "LOCATION=/oracle/PRD/saparch/PRDarch"
log_archive_dest_2 = "LOCATION=/oracle/PRD/saparch1/PRDarch"
log_archive_format = "%t_%s_%r.dbf"
db_files = 1024
log_checkpoints_to_alert = TRUE
replication_dependency_tracking= FALSE
undo_tablespace = "PSAPUNDO"
inmemory_undo = FALSE
undo_retention = 16000
recyclebin = "off"
remote_os_authent = TRUE
service_names = "PRD.WORLD"
parallel_max_servers = 80
parallel_execution_message_size= 16384
fixcontrol = "5099019:ON"
fixcontrol = "5705630:ON"
fixcontrol = "6399597:ON"
fixcontrol = "6430500:ON"
fixcontrol = "6440977:ON"
fixcontrol = "6626018:ON"
fixcontrol = "6972291:ON"
fixcontrol = "9495669:ON"
fixcontrol = "9196440:ON"
fixcontrol = "8937971:ON"
audit_file_dest = "/oracle/PRD/saptrace/audit"
hpux_sched_noage = 178
db_name = "PRD"
open_cursors = 900
star_transformation_enabled= "TRUE"
optimizerextended_cursor_sharing_rel= "NONE"
optimizeradaptive_cursor_sharing= FALSE
parallel_threads_per_cpu = 1
optimizer_index_cost_adj = 20
query_rewrite_enabled = "FALSE"
optimpeek_user_binds = FALSE
pga_aggregate_target = 4G
optimizeruse_feedback = FALSE
diagnostic_dest = "/oracle/PRD/saptrace"
max_dump_file_size = "20000"
Deprecated system parameters with specified values:
remote_os_authent
End of deprecated system parameter listing
Tue Jun 19 08:06:34 2012
PMON started with pid=2, OS id=7143
Tue Jun 19 08:06:34 2012
PSP0 started with pid=3, OS id=7145
Tue Jun 19 08:06:35 2012
VKTM started with pid=4, OS id=7147
VKTM running at (100ms) precision
Tue Jun 19 08:06:35 2012
GEN0 started with pid=5, OS id=7149
Tue Jun 19 08:06:35 2012
DIAG started with pid=6, OS id=7151
Tue Jun 19 08:06:35 2012
DBRM started with pid=7, OS id=7153
Tue Jun 19 08:06:35 2012
DIA0 started with pid=8, OS id=7155
Tue Jun 19 08:06:35 2012
MMAN started with pid=9, OS id=7157
Tue Jun 19 08:06:35 2012
DBW0 started with pid=10, OS id=7159
Tue Jun 19 08:06:35 2012
LGWR started with pid=11, OS id=7161
Tue Jun 19 08:06:35 2012
CKPT started with pid=12, OS id=7163
Tue Jun 19 08:06:35 2012
SMON started with pid=13, OS id=7165
Tue Jun 19 08:06:35 2012
RECO started with pid=14, OS id=7167
Tue Jun 19 08:06:35 2012
MMON started with pid=15, OS id=7169
Tue Jun 19 08:06:35 2012
MMNL started with pid=16, OS id=7171
ORACLE_BASE from environment = /oracle
Tue Jun 19 08:06:36 2012
ALTER DATABASE MOUNT
Successful mount of redo thread 1, with mount id 1849974924
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE MOUNT
Tue Jun 19 08:06:40 2012
ALTER DATABASE OPEN
Errors in file /oracle/PRD/saptrace/diag/rdbms/prd/PRD/trace/PRD_ora_7179.trc:
ORA-01589: must use RESETLOGS or NORESETLOGS option for database open
ORA-1589 signalled during: ALTER DATABASE OPEN...
Tue Jun 19 08:08:01 2012
alter database open resetlogs
RESETLOGS after complete recovery through change 23408973429
Exception [type: SIGSEGV, Invalid Permissions for object] [ADDR:0x4] [PC:0x40000000083B8941, $cold_kcrf_val_log_bsz()+929] [flags: 0x0, count: 1]
Errors in file /oracle/PRD/saptrace/diag/rdbms/prd/PRD/trace/PRD_ora_7179.trc (incident=583361):
ORA-07445: exception encountered: core dump [kcrf_val_log_bsz()+929] [SIGSEGV] [ADDR:0x4] [PC:0x40000000083B8941] [Invalid Permissions for object] []
ORA-27041: unable to open file
HPUX-ia64 Error: 13: Permission denied
Additional information: 9
ORA-27041: unable to open file
HPUX-ia64 Error: 13: Permission denied
Additional information: 9
Incident details in: /oracle/PRD/saptrace/diag/rdbms/prd/PRD/incident/incdir_583361/PRD_ora_7179_i583361.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Tue Jun 19 08:08:05 2012
Dumping diagnostic data in directory=[cdmp_20120619080805], requested by (instance=1, osid=7179), summary=[incident=583361].
Tue Jun 19 08:08:06 2012
Sweep [inc][583361]: completed
Sweep [inc2][583361]: completed
I noticed the in spfile that archive log was still being used, but i remember that we disabled the archiving.
oraprd> more spfilePRD.ora
^G^D^D[.M-^D"^K
PRD.__db_cache_size=241591910
PRD.__oracle_base='/oracle'#ORACLE_BASE set from environment
PRD.__shared_pool_size=365743308
*._fix_control='5099019:ON','5705630:ON','6399597:ON','6430500:ON','6440977:ON','6626018:ON','6972291:ON','9495669:ON','9196440:ON','8937971:ON'
*._in_memory_undo=FALSE#RECOMMENDED BY ORACLE/SAP FOR 11.2.0 - SAP note 1431798
*._optim_peek_user_binds=FALSE#RECOMMENDED BY ORACLE/SAP FOR 11.2.0 - SAP note 1431798
*._optimizer_adaptive_cursor_sharing=FALSE#RECOMMENDED BY ORACLE/SAP FOR 11.2.0 - SAP note 1431798
*._optimizer_extended_cursor_sharing_rel='NONE'#RECOMMENDED BY ORACLE/SAP FOR 11.2.0 - SAP note 1431798
*._optimizer_use_feedback=FALSE#RECOMMENDED BY ORACLE/SAP FOR 11.2.0 - SAP note 1431798
*.audit_file_dest='/oracle/PRD/saptrace/audit'
*.compatible='11.2.0'#Parameter setting for SAP on Oracle Release 11.2 (see SAP note 1431798)
*.control_file_record_keep_time=30
*.control_files='/oracle/PRD/sapdata1/cntrl/cntrlPRD.dbf','/oracle/PRD/origlogA/cntrl/cntrlPRD.dbf','/oracle/PRD/saparch/cntrl/cntrlPRD.dbf'
*.db_block_size=8192
*.db_cache_size=257698037
*.db_files=1024
*.db_name='PRD'
*.diagnostic_dest='/oracle/PRD/saptrace'
*.event='10027','10028','10142','10183','10191','38068 level 100','38085','38087','44951 level 1024'#RECOMMENDED BY ORACLE/SAP FOR 11.2.0 - SAP note 1431798
*.filesystemio_options='setall'
*.hpux_sched_noage=178
*.log_archive_dest_1='LOCATION=/oracle/PRD/saparch/PRDarch'
*.log_archive_dest_2='LOCATION=/oracle/PRD/saparch1/PRDarch'
*.log_archive_format='%t_%s_%r.dbf'
*.log_checkpoints_to_alert=TRUE
*.max_dump_file_size='20000'
*.open_cursors=900
*.optimizer_index_cost_adj=20
*.parallel_execution_message_size=16384
*.parallel_max_servers=80
*.parallel_threads_per_cpu=1
*.pga_aggregate_target=4294967296
*.processes=900
*.query_rewrite_enabled='FALSE'
*.recyclebin='off'
*.remote_os_authent=TRUE#Parameter setting for SAP on Oracle Release 11.2 (see SAP note 1431798)
*.replication_dependency_tracking=FALSE
*.service_names='PRD.WORLD'
*.sessions=1800
*.shared_pool_size=365743308
*.star_transformation_enabled='TRUE'
*.undo_retention=16000
*.undo_tablespace='PSAPUNDO'
I reviewed the /oracle/PRD/saptrace/diag/rdbms/prd/PRD/incident/incdir_583361/PRD_ora_7179_i583361.trc, it hugh file with alot of Jargon.
Any idea what could be causing this error. Thanks in advanceALTER DATABASE OPEN RESETLOGS
RESETLOGS after complete recovery through change 23408973429
Exception [type: SIGSEGV, Invalid Permissions for object] [ADDR:0x4] [PC:0x40000000083B8941, $cold_kcrf_val_log_bsz()+929] [flags: 0x0, count: 1]
Errors in file /oracle/PRD/saptrace/diag/rdbms/prd/PRD/trace/PRD_ora_8363.trc (incident=583362):
ORA-07445: exception encountered: core dump [kcrf_val_log_bsz()+929] [SIGSEGV] [ADDR:0x4] [PC:0x40000000083B8941] [Invalid Permissions for object] []
ORA-27041: unable to open file
HPUX-ia64 Error: 13: Permission denied
Additional information: 9
ORA-27041: unable to open file
HPUX-ia64 Error: 13: Permission denied
Additional information: 9
Incident details in: /oracle/PRD/saptrace/diag/rdbms/prd/PRD/incident/incdir_583362/PRD_ora_8363_i583362.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Tue Jun 19 09:16:24 2012
Dumping diagnostic data in directory=[cdmp_20120619091624], requested by (instance=1, osid=8363), summary=[incident=583362].
Tue Jun 19 09:16:26 2012
Sweep [inc][583362]: completed
Sweep [inc2][583362]: completed -
2-node RAC - unable to start ora.orcl.orcl1.inst - Please help.
linux1-> crs_stat -t
Name Type Target State Host
ora....SM1.asm application ONLINE ONLINE linux1
ora....X1.lsnr application ONLINE ONLINE linux1
ora.linux1.gsd application ONLINE ONLINE linux1
ora.linux1.ons application ONLINE ONLINE linux1
ora.linux1.vip application ONLINE ONLINE linux1
ora....SM2.asm application ONLINE ONLINE linux2
ora....X2.lsnr application ONLINE ONLINE linux2
ora.linux2.gsd application ONLINE ONLINE linux2
ora.linux2.ons application ONLINE ONLINE linux2
ora.linux2.vip application ONLINE ONLINE linux2
ora.orcl.db application ONLINE ONLINE linux1
ora....l1.inst application ONLINE OFFLINE
ora....l2.inst application ONLINE ONLINE linux2
ora....test.cs application ONLINE ONLINE linux2
ora....cl1.srv application ONLINE OFFLINE
ora....cl2.srv application ONLINE ONLINE linux2
linux1->
linux1-> crs_start ora.orcl.orcl1.inst
Attempting to start `ora.orcl.orcl1.inst` on member `linux1`
Start of `ora.orcl.orcl1.inst` on member `linux1` failed.
linux2 : CRS-1018: Resource ora.linux1.vip (application) is already running on linux1
CRS-0215: Could not start resource 'ora.orcl.orcl1.inst'.
linux1->crs_start ora.orcl.orcl1.inst
PRKP-1001 : Error starting instance orcl1 on node linux1
CRS-0215: Could not start resource 'ora.orcl.orcl1.inst'.
stopped all the services one by one
crs_stop XXXX
crs_stop XXXX
Again started all the services in sequence order like asm, instance, listener, database. but no use
Even stopped the as root stopped crs itself
#./crsctl stop crs
started back using
#./crsctl start crs
Please help.
Thanks in advance[cssd(5771)]CRS-1605:CSSD voting file is online: /u02/oradata/orcl/VotingFile_Mirror1. Details in /u01/app/oracle/product/crs/log/linux1/cssd/ocssd.log.
2010-02-16 18:55:15.398
[cssd(5772)]CRS-1605:CSSD voting file is online: /u02/oradata/orcl/VotingFile_Mirror2. Details in /u01/app/oracle/product/crs/log/linux1/cssd/ocssd.log.
2010-02-16 18:55:19.311
[cssd(5919)]CRS-1601:CSSD Reconfiguration complete. Active nodes are linux1 linux2 .
2010-02-16 18:55:19.877
[crsd(4275)]CRS-1012:The OCR service started on node linux1.
2010-02-16 18:55:19.957
[evmd(5582)]CRS-1401:EVMD started on node linux1.
2010-02-16 18:55:21.728
[crsd(4275)]CRS-1201:CRSD started on node linux1.
2010-02-16 19:13:03.006
[cssd(5833)]CRS-1603:CSSD on node linux1 shutdown by user.
2010-02-16 19:15:40.379
[cssd(14888)]CRS-1605:CSSD voting file is online: /u02/oradata/orcl/VotingFile. Details in /u01/app/oracle/product/crs/log/linux1/cssd/ocssd.log.
2010-02-16 19:15:40.404
[cssd(14889)]CRS-1605:CSSD voting file is online: /u02/oradata/orcl/VotingFile_Mirror1. Details in /u01/app/oracle/product/crs/log/linux1/cssd/ocssd.log.
2010-02-16 19:15:40.404
[cssd(14890)]CRS-1605:CSSD voting file is online: /u02/oradata/orcl/VotingFile_Mirror2. Details in /u01/app/oracle/product/crs/log/linux1/cssd/ocssd.log.
2010-02-16 19:15:44.567
[cssd(15060)]CRS-1601:CSSD Reconfiguration complete. Active nodes are linux1 linux2 .
2010-02-16 19:15:45.125
[crsd(14066)]CRS-1012:The OCR service started on node linux1.
2010-02-16 19:15:45.196
[evmd(14708)]CRS-1401:EVMD started on node linux1.
2010-02-16 19:15:46.353
[crsd(14066)]CRS-1201:CRSD started on node linux1. -
11gR2 Cluster node does not start, start of ora.gipcd fails
Hi
I have a clusert node which just does not start, and i do know understand what the problem is.
Does anyone have an idea what the problem cound be?
I am using Oracle 11.2.0.1.0 on OEL.
I appreciate any help, thank you.
Regards Bernhard
NAME TARGET STATE SERVER STATE_DETAILS
Cluster Resources
ora.asm
1 ONLINE OFFLINE
ora.crsd
1 ONLINE INTERMEDIATE node001
ora.cssd
1 ONLINE OFFLINE
ora.cssdmonitor
1 ONLINE OFFLINE
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 ONLINE OFFLINE
ora.drivers.acfs
1 ONLINE ONLINE node001
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE OFFLINE
ora.gpnpd
1 ONLINE OFFLINE
ora.mdnsd
1 ONLINE ONLINE node001 alertnode001.log
Oracle Database 11g Clusterware Release 11.2.0.1.0 - Production Copyright 1996, 2009 Oracle. All rights reserved.
2012-06-29 08:43:52.511
[/opt/grid/product/11gR2/grid/bin/cssdmonitor(5144)]CRS-5822:Agent '/opt/grid/product/11gR2/grid/bin/cssdmonitor_root' disconnected from server. Details at (:CRSAGF00117:) in /opt/grid/product/11gR2/grid/log/node001/agent/ohasd/oracssdmonitor_root/oracssdmonitor_root.log.
2012-06-29 08:48:49.713
[ohasd(4346)]CRS-2112:The OLR service started on node node001.
2012-06-29 08:48:50.010
[ohasd(4346)]CRS-8017:location: /etc/oracle/lastgasp has 50 reboot advisory log files, 0 were announced and 0 errors occurred
2012-06-29 08:48:55.134
[ohasd(4346)]CRS-2772:Server 'node001' has been assigned to pool 'Free'.
[client(4634)]CRS-10001:ACFS-9327: Verifying ADVM/ACFS devices.
[client(4652)]CRS-10001:ACFS-9322: done.
2012-06-29 08:50:59.362
[ohasd(4346)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gipcd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.oracssdmonitor_root.log
Oracle Database 11g Clusterware Release 11.2.0.1.0 - Production Copyright 1996, 2009 Oracle. All rights reserved.
2012-06-29 08:48:51.003: [ AGFW][1810312288] Starting the agent: /opt/grid/product/11gR2/grid/log/node001/agent/ohasd/oracssdmonitor_root/
2012-06-29 08:48:51.003: [ AGENT][1810312288] Agent framework initialized, Process Id = 4414
2012-06-29 08:48:51.003: [ USRTHRD][1810312288] to enter agent main
2012-06-29 08:48:54.939: [ USRTHRD][1810312288] clsncssd_main: setting priority to 4
2012-06-29 08:48:54.939: [ USRTHRD][1810312288] *** Agent Framework Started ***
2012-06-29 08:48:54.939: [ USRTHRD][1810312288] clsncssd_vmondisv: Compatible vendor clusterware not in use
2012-06-29 08:48:54.939: [ USRTHRD][1810312288] clsncssd_thrdspawn: spawn OMON succ
2012-06-29 08:48:54.939: [ USRTHRD][1810312288] clsncssd_main: spawn omon succ
2012-06-29 08:48:54.939: [ USRTHRD][1098914112] clsnomon_main: default values for omon, initrate 1000, pollrate 500.
2012-06-29 08:48:54.939: [ USRTHRD][1098914112] clsnomon_main: starting omon
2012-06-29 08:48:54.942: [ CSSCLNT][1098914112]clssscConnect: gipc request failed with 29 (0x16)
2012-06-29 08:48:54.942: [ CSSCLNT][1098914112]clsssInitNative: connect failed, rc 29
2012-06-29 08:48:55.041: [ AGFW][1810312288] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))
2012-06-29 08:48:55.041: [CLSFRAME][1810312288] Inited lsf context 0x1ae1cdd0
2012-06-29 08:48:55.041: [CLSFRAME][1810312288] Initing CLS Framework messaging
2012-06-29 08:48:55.041: [CLSFRAME][1810312288] New Framework state: 2
2012-06-29 08:48:55.041: [CLSFRAME][1810312288] M2M is starting...
2012-06-29 08:48:55.047: [ CRSCOMM][1810312288] m_pClscCtx=0x1ae21d70m_pUgblm=0x1ae754b0
2012-06-29 08:48:55.047: [ CRSCOMM][1810312288] Starting send thread
2012-06-29 08:48:55.052: [ CRSCOMM][1119893824] clsIpc: sendWork thread started.
2012-06-29 08:48:55.057: [ CRSCOMM][1130383680] IPC Client thread started listening
2012-06-29 08:48:55.057: [ CRSCOMM][1130383680] init data sent from server
2012-06-29 08:48:55.057: [CLSFRAME][1810312288] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:node001
2012-06-29 08:48:55.058: [CLSFRAME][1810312288] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:node001
2012-06-29 08:48:55.066: [CLSFRAME][1810312288] Starting thread model named: MultiThread
2012-06-29 08:48:55.073: [CLSFRAME][1810312288] Starting thread model named: SingleThread
2012-06-29 08:48:55.076: [CLSFRAME][1810312288] Starting thread model named: SingleThreadT
2012-06-29 08:48:55.080: [CLSFRAME][1810312288] New Framework state: 3
2012-06-29 08:48:55.080: [ AGFW][1810312288] Agent Framework started successfully
2012-06-29 08:48:55.080: [ AGFW][1182832960] Agfw engine module has enabled...
2012-06-29 08:48:55.080: [CLSFRAME][1182832960] Module Enabling is complete
2012-06-29 08:48:55.080: [CLSFRAME][1182832960] New Framework state: 6
2012-06-29 08:48:55.080: [ AGFW][1182832960] Agent is started with userid: root , expected user: root
2012-06-29 08:48:55.081: [ AGFW][1182832960] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14
2012-06-29 08:48:55.088: [ AGFW][1182832960] Agent received the message: RESTYPE_ADD[ora.cssdmonitor.type] ID 8196:299
2012-06-29 08:48:55.089: [ AGFW][1182832960] Added new restype: ora.cssdmonitor.type
2012-06-29 08:48:55.089: [ AGFW][1182832960] Agent sending last reply for: RESTYPE_ADD[ora.cssdmonitor.type] ID 8196:299
2012-06-29 08:48:55.090: [ AGFW][1182832960] Agent received the message: RESOURCE_ADD[ora.cssdmonitor 1 1] ID 4356:301
2012-06-29 08:48:55.090: [ AGFW][1182832960] Added new resource: ora.cssdmonitor 1 1 to the agfw
2012-06-29 08:48:55.091: [ AGFW][1182832960] Agent sending last reply for: RESOURCE_ADD[ora.cssdmonitor 1 1] ID 4356:301
2012-06-29 08:48:55.091: [ AGFW][1182832960] Agent received the message: RESOURCE_PROBE[ora.cssdmonitor 1 1] ID 4097:302
2012-06-29 08:48:55.091: [ AGFW][1182832960] Preparing CHECK command for: ora.cssdmonitor 1 1
2012-06-29 08:48:55.096: [ AGFW][1172343104] Executing command: check for resource: ora.cssdmonitor 1 1
2012-06-29 08:48:55.096: [ora.cssdmonitor][1172343104] [check] clsncssd_getattr: attr OMON_INITRATE, value 1000
2012-06-29 08:48:55.096: [ora.cssdmonitor][1172343104] [check] clsncssd_getattr: attr OMON_POLLRATE, value 500
2012-06-29 08:48:55.096: [ora.cssdmonitor][1172343104] [check] clsncssd_getattr: attr ORA_OPROCD_MODE, value
2012-06-29 08:48:55.096: [ora.cssdmonitor][1172343104] [check] clsncssd_getattr: attr PROCD_TIMEOUT, value 1000
2012-06-29 08:48:55.096: [ora.cssdmonitor][1172343104] [check] clsncssd_getattr: attr LOGGING_LEVEL, value 1
2012-06-29 08:48:55.096: [ AGFW][1172343104] check for resource: ora.cssdmonitor 1 1 completed with status: OFFLINE
2012-06-29 08:48:55.096: [CRSTIMER][1203812672] Timer Thread Starting.
2012-06-29 08:48:55.096: [ AGFW][1182832960] ora.cssdmonitor 1 1 state changed from: UNKNOWN to: OFFLINE
2012-06-29 08:48:55.096: [ AGFW][1182832960] Agent sending last reply for: RESOURCE_PROBE[ora.cssdmonitor 1 1] ID 4097:302
2012-06-29 08:48:55.096: [ AGFW][1182832960] Agent has no resources to be monitored.Sending suicide request.
2012-06-29 08:48:55.096: [ AGFW][1182832960] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:28
2012-06-29 08:48:55.100: [ AGFW][1182832960] Agent received the message: RESOURCE_DELETE[ora.cssdmonitor 1 1] ID 4358:317
2012-06-29 08:48:55.100: [ AGFW][1182832960] Agent sending last reply for: RESOURCE_DELETE[ora.cssdmonitor 1 1] ID 4358:317
2012-06-29 08:48:55.100: [ AGFW][1182832960] Agent has no resources to be monitored.Sending suicide request.
2012-06-29 08:48:55.100: [ AGFW][1182832960] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:34
2012-06-29 08:48:55.101: [ AGFW][1182832960] ora.cssdmonitor 1 1 marked as deleted.
2012-06-29 08:48:55.101: [ AGFW][1182832960] Deleting the resource: ora.cssdmonitor 1 1
2012-06-29 08:48:55.101: [ AGFW][1182832960] Agent is commiting suicide.
2012-06-29 08:48:55.101: [ USRTHRD][1182832960] clsncssd_exit: CSSD Agent was asked to exit with exit code 1
2012-06-29 08:48:55.101: [ USRTHRD][1182832960] clsncssd_becomeactive: Signaling active state
2012-06-29 08:48:55.101: [ USRTHRD][1182832960] clsncssd_term: still threads up (1)
2012-06-29 08:48:55.101: [ USRTHRD][1182832960] clsncssd_exit: Agent is done.
2012-06-29 08:48:55.101: [ AGFW][1182832960] Agent is exiting with exit code: 1
2012-06-29 08:48:55.947: [ CSSCLNT][1098914112]clssscConnect: gipc request failed with 29 (0x16)
2012-06-29 08:48:55.947: [ CSSCLNT][1098914112]clsssInitNative: connect failed, rc 29
2012-06-29 08:48:56.952: [ CSSCLNT][1098914112]clssscConnect: gipc request failed with 29 (0x16)
2012-06-29 08:48:56.952: [ CSSCLNT][1098914112]clsssInitNative: connect failed, rc 29Edited by: Bernhard W on 03-Jul-2012 02:14I am really getting desperate about this.
The ASM persmissions are on both nodes the same, so this should not be the problem and i can not find any problem on the network.
The log files of the healthy node do not say anything about the node which does not startup now, the last log entry was from 6 months ago, when i shut that node down. There seems to be no communication between the two nodes now.
I tried to manually start the ora.cssd but somehow ora.gipcd just does not start and i can not find the problem.
[oracle@dmrs11srvcla001 ~]$ crsctl stat res -t -init
NAME TARGET STATE SERVER STATE_DETAILS
Cluster Resources
ora.asm
1 ONLINE OFFLINE
ora.crsd
1 ONLINE INTERMEDIATE dmrs11srvcla001
ora.cssd
1 ONLINE OFFLINE
ora.cssdmonitor
1 ONLINE ONLINE dmrs11srvcla001
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.drivers.acfs
1 ONLINE ONLINE dmrs11srvcla001
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE OFFLINE
ora.gpnpd
1 ONLINE OFFLINE
ora.mdnsd
1 ONLINE ONLINE dmrs11srvcla001
[oracle@dmrs11srvcla001 ~]$ crsctl start res ora.cssd -init
CRS-2672: Attempting to start 'ora.gipcd' on 'dmrs11srvcla001'
Start action for daemon aborted
CRS-2674: Start of 'ora.gipcd' on 'dmrs11srvcla001' failed
CRS-2679: Attempting to clean 'ora.gipcd' on 'dmrs11srvcla001'
CRS-2681: Clean of 'ora.gipcd' on 'dmrs11srvcla001' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'dmrs11srvcla001'
Start action for daemon aborted
CRS-2674: Start of 'ora.gpnpd' on 'dmrs11srvcla001' failed
CRS-2679: Attempting to clean 'ora.gpnpd' on 'dmrs11srvcla001'
CRS-2681: Clean of 'ora.gpnpd' on 'dmrs11srvcla001' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'dmrs11srvcla001'
CRS-2672: Attempting to start 'ora.diskmon' on 'dmrs11srvcla001'
CRS-2676: Start of 'ora.diskmon' on 'dmrs11srvcla001' succeededalertnode001.log
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gipcd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-02 17:11:40.789
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gpnpd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-02 17:11:41.654
[ohasd(4351)]CRS-2765:Resource 'ora.mdnsd' has failed on server 'node001'.
2012-07-02 17:11:43.404
[cssd(4997)]CRS-1713:CSSD daemon is started in clustered mode
2012-07-02 17:11:46.634
[cssd(4997)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2012-07-02 17:11:46.745
[ohasd(4351)]CRS-2765:Resource 'ora.cssdmonitor' has failed on server 'node001'.
2012-07-02 17:12:42.384
[/opt/grid/product/11gR2/grid/bin/orarootagent.bin(4488)]CRS-5818:Aborted command 'start for resource: ora.diskmon 1 1' for resource 'ora.diskmon'. Details at (:CRSAGF00113:) in /opt/grid/product/11gR2/grid/log/node001/agent/ohasd/orarootagent_root/orarootagent_root.log.
2012-07-02 17:12:46.390
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.diskmon'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-02 17:14:50.768
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gipcd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-02 17:16:55.819
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gpnpd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-02 17:16:58.354
[cssd(5137)]CRS-1713:CSSD daemon is started in clustered mode
2012-07-02 17:16:58.425
[cssd(5137)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2012-07-02 17:25:23.194
[ohasd(4351)]CRS-2765:Resource 'ora.diskmon' has failed on server 'node001'.
2012-07-02 17:25:23.200
[ohasd(4351)]CRS-2767:Target resource 'ora.diskmon' is offline, will not recover.
2012-07-02 17:26:57.384
[/opt/grid/product/11gR2/grid/bin/cssdagent(5116)]CRS-5818:Aborted command 'start for resource: ora.cssd 1 1' for resource 'ora.cssd'. Details at (:CRSAGF00113:) in /opt/grid/product/11gR2/grid/log/node001/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
2012-07-02 17:27:02.514
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.cssd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-02 17:37:11.971
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.evmd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 02:13:57.821
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 05:14:37.915
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 05:44:44.630
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 06:14:51.303
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 06:44:57.981
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 07:15:04.655
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 07:42:45.015
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gipcd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-03 07:44:50.068
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gpnpd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.
2012-07-03 07:44:52.369
[cssd(10209)]CRS-1713:CSSD daemon is started in clustered mode
2012-07-03 07:44:52.434
[cssd(10209)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2012-07-03 07:45:11.331
[ohasd(4351)]CRS-2765:Resource 'ora.crsd' has failed on server 'node001'.
2012-07-03 07:53:17.222
[ohasd(4351)]CRS-2765:Resource 'ora.diskmon' has failed on server 'node001'.
2012-07-03 07:53:17.222
[ohasd(4351)]CRS-2767:Target resource 'ora.diskmon' is offline, will not recover.
2012-07-03 07:54:51.401
[/opt/grid/product/11gR2/grid/bin/cssdagent(10188)]CRS-5818:Aborted command 'start for resource: ora.cssd 1 1' for resource 'ora.cssd'. Details at (:CRSAGF00113:) in /opt/grid/product/11gR2/grid/log/node001/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
2012-07-03 07:54:56.524
[ohasd(4351)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.cssd'. Details at (:CRSPE00111:) in /opt/grid/product/11gR2/grid/log/node001/ohasd/ohasd.log.ohasd.log
2012-07-03 07:45:11.330: [ AGFW][1347250496] Agfw Proxy Server received the message: RESOURCE_STATUS[Proxy] ID 20481:24919
2012-07-03 07:45:11.330: [ AGFW][1347250496] Received state change for ora.crsd 1 1 [old state = PARTIAL, new state = OFFLINE]
2012-07-03 07:45:11.330: [ AGFW][1347250496] Agfw Proxy Server sending message to PE, Contents = [MIDTo:2|OpID:3|FromA:{Invalid|Node:0|Process:0|Type:0}|ToA:{Invalid|Node:-1|Process:-1|Type:-1}|MIDFrom:0|Type:4|Pri2|Id:23258]
2012-07-03 07:45:11.330: [ AGFW][1347250496] Agfw Proxy Server replying to the message: RESOURCE_STATUS[Proxy] ID 20481:24919
2012-07-03 07:45:11.330: [ CRSPE][1357756736] State change received from node001 for ora.crsd 1 1
2012-07-03 07:45:11.331: [ CRSPE][1357756736] Processing PE command id=139. Description: [Resource State Change (ora.crsd 1 1) : 0x1c284d40]
2012-07-03 07:45:11.331: [ CRSPE][1357756736] RI [ora.crsd 1 1] new external state [OFFLINE] old value: [INTERMEDIATE] on node001 label = []
2012-07-03 07:45:11.331: [ CRSPE][1357756736] Resource Resource Instance ID[ora.crsd 1 1]. Values:
STATE=OFFLINE
TARGET=ONLINE
LAST_SERVER=node001
CURRENT_RCOUNT=1
LAST_RESTART=1341292505
FAILURE_COUNT=0
FAILURE_HISTORY=
STATE_DETAILS=
INCARNATION=29
STATE_CHANGE_VERS=29
LAST_FAULT=1341292504
DEGREE_ID=1
ID=ora.crsd 1 1
CARDINALITY_ID=1
Lock Info:
Write Locks:none
ReadLocks:|STATE INITED||INITIAL CHECK DONE| has failed!
2012-07-03 07:45:11.331: [ CRSPE][1357756736] Processing unplanned state change for [ora.crsd 1 1]
2012-07-03 07:45:11.331: [ CRSPE][1357756736] ora.crsd 1 1: uptime exceeds uptime threshold , resetting restart count
2012-07-03 07:45:11.331: [ CRSPE][1357756736] Local Recovery picked for [ora.crsd 1 1]
2012-07-03 07:45:11.333: [ CRSPE][1357756736] Sending message to agfw: id = 23260
2012-07-03 07:45:11.333: [ AGFW][1347250496] Agfw Proxy Server received the message: RESOURCE_START[ora.crsd 1 1] ID 4098:23260
2012-07-03 07:45:11.333: [ CRSPE][1357756736] CRS-2672: Attempting to start 'ora.crsd' on 'node001'
2012-07-03 07:45:11.333: [ AGFW][1347250496] Agfw Proxy Server forwarding the message: RESOURCE_START[ora.crsd 1 1] ID 4098:23260 to the agent /opt/grid/product/11gR2/grid/bin/orarootagent_root
2012-07-03 07:45:12.349: [ AGFW][1347250496] Received the reply to the message: RESOURCE_START[ora.crsd 1 1] ID 4098:23261 from the agent /opt/grid/product/11gR2/grid/bin/orarootagent_root
2012-07-03 07:45:12.350: [ AGFW][1347250496] Agfw Proxy Server sending the reply to PE for message:RESOURCE_START[ora.crsd 1 1] ID 4098:23260
2012-07-03 07:45:12.350: [ CRSPE][1357756736] Received reply to action [Start] message ID: 23260
2012-07-03 07:45:12.350: [ AGFW][1347250496] Received the reply to the message: RESOURCE_START[ora.crsd 1 1] ID 4098:23261 from the agent /opt/grid/product/11gR2/grid/bin/orarootagent_root
2012-07-03 07:45:12.350: [ AGFW][1347250496] Agfw Proxy Server sending the last reply to PE for message:RESOURCE_START[ora.crsd 1 1] ID 4098:23260
2012-07-03 07:45:12.350: [ CRSPE][1357756736] Received reply to action [Start] message ID: 23260
2012-07-03 07:45:12.351: [ CRSPE][1357756736] RI [ora.crsd 1 1] new external state [INTERMEDIATE] old value: [OFFLINE] on node001 label = []
2012-07-03 07:45:12.351: [ CRSPE][1357756736] CRS-2676: Start of 'ora.crsd' on 'node001' succeeded
2012-07-03 07:45:12.351: [ CRSPE][1357756736] PE Command [ Resource State Change (ora.crsd 1 1) : 0x1c284d40 ] has completed
2012-07-03 07:45:12.351: [ AGFW][1347250496] Agfw Proxy Server received the message: CMD_COMPLETED[Proxy] ID 20482:23269
2012-07-03 07:45:12.351: [ AGFW][1347250496] Agfw Proxy Server replying to the message: CMD_COMPLETED[Proxy] ID 20482:23269
2012-07-03 07:45:12.351: [ AGFW][1347250496] Agfw received reply from PE for resource state change for ora.crsd 1 1
2012-07-03 07:53:17.220: [ AGFW][1347250496] Agfw Proxy Server received the message: RESOURCE_STATUS[Proxy] ID 20481:25353
2012-07-03 07:53:17.221: [ AGFW][1347250496] Received state change for ora.diskmon 1 1 [old state = ONLINE, new state = PLANNED_OFFLINE]
2012-07-03 07:53:17.221: [ AGFW][1347250496] Agfw Proxy Server sending message to PE, Contents = [MIDTo:2|OpID:3|FromA:{Invalid|Node:0|Process:0|Type:0}|ToA:{Invalid|Node:-1|Process:-1|Type:-1}|MIDFrom:0|Type:4|Pri2|Id:23497]
2012-07-03 07:53:17.221: [ AGFW][1347250496] Agfw Proxy Server replying to the message: RESOURCE_STATUS[Proxy] ID 20481:25353
2012-07-03 07:53:17.221: [ CRSPE][1357756736] State change received from node001 for ora.diskmon 1 1
2012-07-03 07:53:17.221: [ CRSPE][1357756736] Processing PE command id=140. Description: [Resource State Change (ora.diskmon 1 1) : 0x2aaab00318a0]
2012-07-03 07:53:17.221: [ CRSPE][1357756736] RI [ora.diskmon 1 1] new external state [OFFLINE] old value: [ONLINE] on node001 label = []
2012-07-03 07:53:17.221: [ CRSPE][1357756736] RI [ora.diskmon 1 1] new target state: [OFFLINE] old value: [ONLINE]
2012-07-03 07:53:17.222: [ CRSOCR][1351452992] Multi Write Batch processing...
2012-07-03 07:53:17.222: [ CRSPE][1357756736] Resource Resource Instance ID[ora.diskmon 1 1]. Values:
STATE=OFFLINE
TARGET=OFFLINE
LAST_SERVER=node001
CURRENT_RCOUNT=0
LAST_RESTART=1341294297
FAILURE_COUNT=0
FAILURE_HISTORY=
STATE_DETAILS=
INCARNATION=2
STATE_CHANGE_VERS=5
LAST_FAULT=1341242723
DEGREE_ID=1
ID=ora.diskmon 1 1
CARDINALITY_ID=1
Lock Info:
Write Locks:START of [ora.cssd 1 1] on [node001] : 0x2aaab000ba10
ReadLocks:|STATE INITED| has failed!
2012-07-03 07:53:17.222: [ CRSPE][1357756736] Processing unplanned state change for [ora.diskmon 1 1]
2012-07-03 07:53:17.222: [ CRSPE][1357756736] Target is not ONLINE, not recovering [ora.diskmon 1 1]
2012-07-03 07:53:17.222: [ CRSPE][1357756736] PE Command [ Resource State Change (ora.diskmon 1 1) : 0x2aaab00318a0 ] has completed
2012-07-03 07:53:17.222: [ AGFW][1347250496] Agfw Proxy Server received the message: CMD_COMPLETED[Proxy] ID 20482:23501
2012-07-03 07:53:17.222: [ AGFW][1347250496] Agfw Proxy Server replying to the message: CMD_COMPLETED[Proxy] ID 20482:23501
2012-07-03 07:53:17.223: [ AGFW][1347250496] Agfw received reply from PE for resource state change for ora.diskmon 1 1
2012-07-03 07:53:17.223: [ CRSOCR][1351452992] Multi Write Batch done.
2012-07-03 07:54:56.523: [ AGFW][1347250496] Received the reply to the message: RESOURCE_START[ora.cssd 1 1] ID 4098:23242 from the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:54:56.523: [ AGFW][1347250496] Agfw Proxy Server sending the reply to PE for message:RESOURCE_START[ora.cssd 1 1] ID 4098:23220
2012-07-03 07:54:56.524: [ CRSPE][1357756736] Received reply to action [Start] message ID: 23220
2012-07-03 07:54:56.524: [ AGFW][1347250496] Received the reply to the message: RESOURCE_START[ora.cssd 1 1] ID 4098:23242 from the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:54:56.524: [ CRSPE][1357756736]Created alert : (:CRSPE00111:) : Start action timed out!
2012-07-03 07:54:56.524: [ CRSPE][1357756736] Start action failed with error code: 3
2012-07-03 07:54:56.524: [ AGFW][1347250496] Agfw Proxy Server sending the last reply to PE for message:RESOURCE_START[ora.cssd 1 1] ID 4098:23220
2012-07-03 07:54:56.524: [ CRSPE][1357756736] Received reply to action [Start] message ID: 23220
2012-07-03 07:54:56.524: [ AGFW][1347250496] Agfw Proxy Server received the message: AGENT_SUICIDE[Proxy] ID 20486:34
2012-07-03 07:54:56.524: [ AGFW][1347250496] Suicide request received from /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:54:56.524: [ AGFW][1347250496] Agfw Proxy Server replying to the message: AGENT_SUICIDE[Proxy] ID 20486:34
2012-07-03 07:54:56.524: [ CRSPE][1357756736] CRS-2674: Start of 'ora.cssd' on 'node001' failed
2012-07-03 07:54:56.525: [UiServer][1361959232] Container [ Name: ORDER
MESSAGE:
TextMessage[CRS-2674: Start of 'ora.cssd' on 'node001' failed]
MSGTYPE:
TextMessage[1]
OBJID:
TextMessage[ora.cssd]
WAIT:
TextMessage[0]
2012-07-03 07:54:56.525: [ CRSPE][1357756736] Sending message to agfw: id = 23555
2012-07-03 07:54:56.525: [ CRSPE][1357756736] CRS-2679: Attempting to clean 'ora.cssd' on 'node001'
2012-07-03 07:54:56.525: [ AGFW][1347250496] Agfw Proxy Server received the message: RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23555
2012-07-03 07:54:56.525: [ AGFW][1347250496] Agent: /opt/grid/product/11gR2/grid/bin/cssdagent with user id root is being stopped. Will be restarted when agent stops completly
2012-07-03 07:54:56.525: [ AGFW][1347250496] Could not forward message [RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23555] to agent. /opt/grid/product/11gR2/grid/bin/cssdagent_root is not running
2012-07-03 07:54:56.525: [ AGFW][1347250496] Can not start Agent: /opt/grid/product/11gR2/grid/bin/cssdagent with user id root is waiting to be restarted.
2012-07-03 07:54:56.525: [UiServer][1361959232] Container [ Name: ORDER
MESSAGE:
TextMessage[CRS-2679: Attempting to clean 'ora.cssd' on 'node001']
MSGTYPE:
TextMessage[3]
OBJID:
TextMessage[ora.cssd]
WAIT:
TextMessage[0]
2012-07-03 07:55:01.627: [ CRSCOMM][1340946752][FFAIL] Couldnt clscreceive message, no message: 11
2012-07-03 07:55:01.627: [ CRSCOMM][1340946752] Client disconnected.
2012-07-03 07:55:01.627: [ CRSCOMM][1340946752][FFAIL] Listener got clsc error 11 for memNum. 14
2012-07-03 07:55:01.628: [ CRSCOMM][1340946752] IPC listener connection to member 14 has been removed
2012-07-03 07:55:01.628: [CLSFRAME][1340946752] Removing IPC Member:{Relative|Node:0|Process:14|Type:3}
2012-07-03 07:55:01.628: [CLSFRAME][1340946752] Disconnected from AGENT process: {Relative|Node:0|Process:14|Type:3}
2012-07-03 07:55:01.628: [ CRSPE][1357756736] Disconnected from server:
2012-07-03 07:55:01.628: [ AGFW][1347250496] Agfw Proxy Server received process disconnected notification, count=1
2012-07-03 07:55:01.628: [ AGFW][1347250496] /opt/grid/product/11gR2/grid/bin/cssdagent_root disconnected.
2012-07-03 07:55:01.628: [ AGFW][1347250496] Restarting the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.628: [ AGFW][1347250496] Agent /opt/grid/product/11gR2/grid/bin/cssdagent_root[10188] stopped!
2012-07-03 07:55:01.628: [ CRSCOMM][1347250496] removeConnection: Member 14 does not exist.
2012-07-03 07:55:01.628: [ AGFW][1347250496] Starting the agent: /opt/grid/product/11gR2/grid/bin/cssdagent with user id: root and incarnation:7
2012-07-03 07:55:01.632: [ AGFW][1347250496] Starting the HB [Interval = 30000, misscount = 10kill allowed=0] for agent: /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.819: [CLSFRAME][1340946752] New IPC Member:{Relative|Node:0|Process:15|Type:3}:AGENT
2012-07-03 07:55:01.819: [CLSFRAME][1340946752] New process connected to us ID:{Relative|Node:0|Process:15|Type:3} Info:AGENT
2012-07-03 07:55:01.837: [ AGFW][1347250496] Agfw Proxy Server received the message: AGENT_HANDSHAKE[Proxy] ID 20484:14
2012-07-03 07:55:01.837: [ AGFW][1347250496] Agent /opt/grid/product/11gR2/grid/bin/cssdagent_root with pid:10530 connected to server.
2012-07-03 07:55:01.837: [ AGFW][1347250496] Agfw Proxy Server sending message: RESTYPE_ADD[ora.cssd.type] ID 8196:23591 to the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.838: [ AGFW][1347250496] Agfw Proxy Server sending message: RESOURCE_ADD[ora.cssd 1 1] ID 4356:23593 to the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.838: [ AGFW][1347250496] Agfw Proxy Server forwarding the message: RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23555 to the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.839: [ AGFW][1347250496] Agfw Proxy Server replying to the message: AGENT_HANDSHAKE[Proxy] ID 20484:14
2012-07-03 07:55:01.851: [ AGFW][1347250496] Received the reply to the message: RESTYPE_ADD[ora.cssd.type] ID 8196:23591 from the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.854: [ AGFW][1347250496] Received the reply to the message: RESOURCE_ADD[ora.cssd 1 1] ID 4356:23593 from the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.861: [ AGFW][1347250496] Received the reply to the message: RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23594 from the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.862: [ AGFW][1347250496] Agfw Proxy Server sending the reply to PE for message:RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23555
2012-07-03 07:55:01.862: [ AGFW][1347250496] Received the reply to the message: RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23594 from the agent /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.862: [ CRSPE][1357756736] Received reply to action [Clean] message ID: 23555
2012-07-03 07:55:01.862: [ AGFW][1347250496] Agfw Proxy Server sending the last reply to PE for message:RESOURCE_CLEAN[ora.cssd 1 1] ID 4100:23555
2012-07-03 07:55:01.863: [ AGFW][1347250496] Agfw Proxy Server received the message: AGENT_SUICIDE[Proxy] ID 20486:34
2012-07-03 07:55:01.863: [ CRSPE][1357756736] Received reply to action [Clean] message ID: 23555
2012-07-03 07:55:01.863: [ AGFW][1347250496] Suicide request received from /opt/grid/product/11gR2/grid/bin/cssdagent_root
2012-07-03 07:55:01.863: [ AGFW][1347250496] Agfw Proxy Server replying to the message: AGENT_SUICIDE[Proxy] ID 20486:34
2012-07-03 07:55:01.863: [ CRSPE][1357756736] CRS-2681: Clean of 'ora.cssd' on 'node001' succeeded
2012-07-03 07:55:01.863: [ CRSPE][1357756736] Sequencer for [ora.cssd 1 1] has completed with error: CRS-0215: Could not start resource 'ora.cssd'.
2012-07-03 07:55:01.863: [UiServer][1361959232] Container [ Name: ORDER
MESSAGE:
TextMessage[CRS-2681: Clean of 'ora.cssd' on 'node001' succeeded]
MSGTYPE:
TextMessage[3]
OBJID:
TextMessage[ora.cssd]
WAIT:
TextMessage[0]
2012-07-03 07:55:01.864: [ CRSPE][1357756736] PE Command [ Start Resource : 0x2aaab0009d00 ] has completed
2012-07-03 07:55:01.864: [ AGFW][1347250496] Agfw Proxy Server received the message: CMD_COMPLETED[Proxy] ID 20482:23606
2012-07-03 07:55:01.864: [ CRSPE][1357756736] UI Command [Start Resource : 0x2aaab0009d00] is replying to sender.
2012-07-03 07:55:01.864: [ AGFW][1347250496] Agfw Proxy Server replying to the message: CMD_COMPLETED[Proxy] ID 20482:23606
2012-07-03 07:55:01.864: [UiServer][1361959232] Container [ Name: UI_DATA
ora.cssd:
TextMessage[215]
2012-07-03 07:55:01.864: [UiServer][1361959232] Done for ctx=0x2aaab004ece0
2012-07-03 07:55:01.869: [UiServer][1364060480] Closed: remote end failed/disc.
2012-07-03 07:55:03.791: [ CRSCOMM][1340946752][FFAIL] Couldnt clscreceive message, no message: 11
2012-07-03 07:55:03.791: [ CRSCOMM][1340946752] Client disconnected.
2012-07-03 07:55:03.791: [ CRSCOMM][1340946752][FFAIL] Listener got clsc error 11 for memNum. 15
2012-07-03 07:55:03.791: [ CRSCOMM][1340946752] IPC listener connection to member 15 has been removed
2012-07-03 07:55:03.791: [CLSFRAME][1340946752] Removing IPC Member:{Relative|Node:0|Process:15|Type:3}
2012-07-03 07:55:03.791: [CLSFRAME][1340946752] Disconnected from AGENT process: {Relative|Node:0|Process:15|Type:3}
2012-07-03 07:55:03.792: [ CRSPE][1357756736] Disconnected from server:
2012-07-03 07:55:03.792: [ AGFW][1347250496] Agfw Proxy Server received process disconnected notification, count=1
2012-07-03 07:55:03.792: [ AGFW][1347250496] /opt/grid/product/11gR2/grid/bin/cssdagent_root disconnected.
2012-07-03 07:55:03.792: [ AGFW][1347250496] Agent /opt/grid/product/11gR2/grid/bin/cssdagent_root[10530] stopped!
2012-07-03 07:55:03.792: [ CRSCOMM][1347250496] removeConnection: Member 15 does not exist. -
11gR2 : Failed to start EVM while running root.sh in Node1
DB verision:11GR2
Platform : Solaris 5.10 (Amd64)
I've reached the last step in the installation in Node1
# ./root.sh
.............all messages say succeeded till here............
CRS-2676: Start of 'ora.crsd' on 'hudson1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'hudson1'
Start action for daemon aborted
CRS-2674: Start of 'ora.evmd' on 'hudson1' failed
CRS-2679: Attempting to clean 'ora.evmd' on 'hudson1'
CRS-2681: Clean of 'ora.evmd' on 'hudson1' succeeded
CRS-4000: Command Start failed, or completed with errors.
Command return code of 1 (256) from command: /u01/grid/bin/crsctl start resource ora.evmd -init
Start of resource "ora.evmd -init" failed
Failed to start EVM
Failed to start Oracle Clusterware stackA search in OTN and MOS returned nothing. Any idea how i could diagnose this issue?Hi,
Check this note on MOS.
*Troubleshooting 11.2 Grid Infastructure Installation Root.sh Issues [ID 1053970.1]*
A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.
Regards,
Levi Pereira -
CRS-0215: Could not start resource
Hello
This is my first time installing clusterware. However, i have not been too successful at it. This is my configuration:
Operating System: RHES 5.3
Oracle Database 11gR1
OpenFiler used to configure shared disks
After several attempts, i was able to run the root.sh script on both nodes.
Output on first node:
[root@vx0301 oracle]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory to 770.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete
[root@vx0301 oracle]# /u01/crs11g/root.sh
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: vx0301 vx0301-priv vx0301
node 2: vx0302 vx0302-priv vx0302
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
vx0301
Cluster Synchronization Services is inactive on these nodes.
vx0302
Local node checking complete. Run root.sh on remaining nodes to start CRS daemons.
Output on second node:
[root@vx0301 oracle]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory to 770.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete
[root@vx0301 oracle]# /u01/crs11g/root.sh
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: vx0301 vx0301-priv vx0301
node 2: vx0302 vx0302-priv vx0302
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
vx0301
vx0302
I also got the errors:
Starting GSD application resource on (2) nodes1:CRS-0215: Could not start resource 'ora.vx0301.gsd'
Starting ONS application resource on (2) nodes1:CRS-0215: Could not start resource 'ora.vx0301.ons'
There were no log files.
When i clicked ok on the execute scripts page, i got the following errors in the configuration assistants page:
Output generated from configuration assistant "Oracle Notification Server Configuration Assistant":
Command = /u01/crs11g/install/onsconfig add_config vx0301:6251 vx0302:6251
The ONS configuration failed to create
Configuration assistant "Oracle Notification Server Configuration Assistant" failed
The "/u01/crs11g/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.-----------------------------------------------------------------------------Output generated from configuration assistant "Oracle Notification Server Configuration Assistant":
Command = /u01/crs11g/install/onsconfig add_config vx0301:6251 vx0302:6251
The ONS configuration failed to create
Configuration assistant "Oracle Notification Server Configuration Assistant" failed
The "/u01/crs11g/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.-----------------------------------------------------------------------------Output generated from configuration assistant "Oracle Notification Server Configuration Assistant" (attempt 2):
Command = /u01/crs11g/install/onsconfig add_config vx0301:6251 vx0302:6251
The ONS configuration failed to create
Configuration assistant "Oracle Notification Server Configuration Assistant" failed
The "/u01/crs11g/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.-----------------------------------------------------------------------------
Contents of the /u01/crs11g/cfgtoollogs/configToolFailedCommands script:
[oracle@vx0301 ~]$ cat /u01/crs11g/cfgtoollogs/configToolFailedCommands
# Copyright (c) 1999, 2007, Oracle. All rights reserved.
/u01/crs11g/install/onsconfig add_config vx0301:6251 vx0302:6251
/u01/crs11g/bin/oifcfg setif -global eth0/172.30.4.0:public eth1/192.168.1.0:cluster_interconnect
/u01/crs11g/bin/cluvfy stage -post crsinst -n vx0301,vx0302
I tried running /u01/crs11g/install/onsconfig add_config vx0301:6251 vx0302:6251 manually from the terminal:
[root@vx0301 oracle]# /u01/crs11g/install/onsconfig add_config vx0301:6251 vx0302:6251
The ONS configuration failed to create
Output of the ons.log file:
[root@vx0301 oracle]# cat /u01/crs11g/log/vx0301/racg/ons.log
Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
2010-01-25 13:59:15.786: [ RACG][3055679168] [10113][3055679168][default]: clsrons: procr_init:PROC-32: Cluster Ready Services on the local node is not running Messaging error [9] status = 32
2010-01-25 13:59:33.359: [ RACG][3055584960] [10229][3055584960][default]: clsrons: procr_init:PROC-32: Cluster Ready Services on the local node is not running Messaging error [9] status = 32
2010-01-25 14:01:00.319: [ RACG][3086849728] [10734][3086849728][default]: clsrons: procr_init:PROC-32: Cluster Ready Services on the local node is not running Messaging error [9] status = 32
2010-01-25 14:02:02.723: [ RACG][3086628544] [11105][3086628544][default]: clsrons: procr_init:PROC-32: Cluster Ready Services on the local node is not running Messaging error [9] status = 32
[root@vx0301 oracle]#
I am absoutely stumped as to what to do next. Any help is greatly appreciated.I tried running ruclufvy.sh stage -post crsinst -n vx0301,vx0302 -verbose. Here is the output:
[oracle@vx0301 clusterware]$ ./runcluvfy.sh stage -post crsinst -n vx0301,vx0302 -verbose
Performing post-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "vx0301"
Destination Node Reachable?
vx0301 yes
vx0302 yes
Result: Node reachability check passed from node "vx0301".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
vx0302 passed
vx0301 passed
Result: User equivalence check passed for user "oracle".
Checking Cluster manager integrity...
Checking CSS daemon...
Node Name Status
vx0302 running
vx0301 running
Result: Daemon status check passed for "CSS daemon".
Cluster manager integrity check passed.
Checking cluster integrity...
Node Name
vx0301
vx0302
Cluster integrity check passed
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations.
Uniqueness check for OCR device passed.
Checking the version of OCR...
OCR of correct Version "2" exists.
Checking data integrity of OCR...
ERROR:
OCR integrity is invalid.
OCR integrity check failed.
Checking CRS integrity...
Checking daemon liveness...
Check: Liveness for "CRS daemon"
Node Name Running
vx0302 yes
vx0301 yes
Result: Liveness check passed for "CRS daemon".
Checking daemon liveness...
Check: Liveness for "CSS daemon"
Node Name Running
vx0302 yes
vx0301 yes
Result: Liveness check passed for "CSS daemon".
Checking daemon liveness...
Check: Liveness for "EVM daemon"
Node Name Running
vx0302 yes
vx0301 yes
Result: Liveness check passed for "EVM daemon".
Liveness of all the daemons
Node Name CRS daemon CSS daemon EVM daemon
vx0302 yes yes yes
vx0301 yes yes yes
Checking CRS health...
Check: Health of CRS
Node Name CRS OK?
vx0302 yes
vx0301 unknown
Result: CRS health check failed.
CRS integrity check failed.
Checking node application existence...
Checking existence of VIP node application
Node Name Required Status Comment
vx0302 yes exists passed
vx0301 yes exists passed
Result: Check passed.
Checking existence of ONS node application
Node Name Required Status Comment
vx0302 no exists passed
vx0301 no exists passed
Result: Check passed.
Checking existence of GSD node application
Node Name Required Status Comment
vx0302 no exists passed
vx0301 no exists passed
Result: Check passed.
Post-check for cluster services setup was unsuccessful on all the nodes.
[oracle@vx0301 clusterware]$
I also tried running ./ocrchek on both nodes. Here is the output:
[oracle@vx0301 bin]$ ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 3306636
Used space (kbytes) : 2056
Available space (kbytes) : 3304580
ID : 1425438992
Device/File Name : /dev/sdb1
Device/File integrity check failed
Device/File not configured
Cluster registry integrity check failed
[oracle@vx0301 bin]$ ssh vx0302
Last login: Mon Jan 25 13:52:03 2010 from vx0301
[oracle@vx0302 ~]$ cd /u01/crs11g/bin/
[oracle@vx0302 bin]$ ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 3306636
Used space (kbytes) : 2056
Available space (kbytes) : 3304580
ID : 1425438992
Device/File Name : /dev/sdb1
Device/File integrity check failed
Device/File not configured
Cluster registry integrity check failed
[oracle@vx0302 bin]$
From the results, the ids are the same, and so is the OCR disk. But the integrity check has failed. How do i interpret this output? -
CRS-0223: Resource 'ora.xt3xx.ons' has placement error.
All,
I recently downgraded my RAC 10.2.0.4 database to 10.2.0.1 . and after that I am not able to start the database using srvctl . however i am able to start the db manually.
this is the error message
srvctl start database -d RDBE
PRKP-1001 : Error starting instance RDBE1 on node xt33db006
CRS-0215: Could not start resource 'ora.RDBE.RDBE1.inst'.
PRKP-1001 : Error starting instance RDBE2 on node xt33db007
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:SQL*Plus: Release 10.2.0.1.0 - Production on Fri May 16 09:32:30 2008
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:Copyright (c) 1982, 2005, Oracle. All rights reserved.
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:Enter user-name: Connected to an idle instance.
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:SQL> ASM instance started
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:Total System Global Area 130023424 bytes
xt33db007:ora.xt33db007.ASM2.asm:Fixed Size 1976920 bytes
xt33db007:ora.xt33db007.ASM2.asm:Variable Size 102880680 bytes
xt33db007:ora.xt33db007.ASM2.asm:ASM Cache 25165824 bytes
xt33db007:ora.xt33db007.ASM2.asm:ASM diskgroups mounted
xt33db007:ora.xt33db007.ASM2.asm:SQL> Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
xt33db007:ora.xt33db007.ASM2.asm:With the Partitioning, Real Application Clusters, OLAP and Data Mining options
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:SQL*Plus: Release 10.2.0.1.0 - Production on Fri May 16 09:32:53 2008
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:Copyright (c) 1982, 2005, Oracle. All rights reserved.
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:Enter user-name:
xt33db007:ora.xt33db007.ASM2.asm:Connected to:
xt33db007:ora.xt33db007.ASM2.asm:Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
xt33db007:ora.xt33db007.ASM2.asm:With the Partitioning, Real Application Clusters, OLAP and Data Mining options
xt33db007:ora.xt33db007.ASM2.asm:
xt33db007:ora.xt33db007.ASM2.asm:SQL> ASM diskgroups dismounted
xt33db007:ora.xt33db007.ASM2.asm:ASM instance shutdown
xt33db007:ora.xt33db007.ASM2.asm:SQL> Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
xt33db007:ora.xt33db007.ASM2.asm:With the Partitioning, Real Application Clusters, OLAP and Data Mining options
xt33db007:ora.xt33db007.ASM2.asm:
CRS-0215: Could not start resource 'ora.RDBE.RDBE2.inst'.
please advise.
thanks
PkWhat is the error message you are seeing in CRS & RACG logs related to the database startup.?
What is the STATE & TARGET fields showing up for the database and instances?
Please paste the srvm log for the srvctl command.
export SRVM_TRACE=TRUE
srvctl start database -d RDBE
exit -
CRS-1019 Resource ora.devdb.devdb1.inst (application) cannot run on rac2
I run a CRS upgrade from 10.2.0.1 to 10.2.0.3. The process runned ok but instance devdb1 on rac1 of the RAC does not start.
crs_stat -t
ora.devdb.db ONLINE ONLINE rac1
ora.devdb.devdb1.inst ONLINE OFFLINE
ora.devdb.devdb2.inst ONLINE ONLINE rac2
I tried the following, without success:
srvctl remove database -d devdb -f
crs_unregister ora.devdb.db
srvctl remove instance -d devdb -i devdb1
srvctl remove instance -d devdb -i devdb2
srvctl add database -d devdb -o $ORACLE_HOME
srvctl add instance -d devdb -i devdb1 -n rac1
srvctl add instance -d devdb -i devdb2 -n rac2
srvctl stop database -d devdb
srvctl start database -d devdb
When I run:
srvctl start instance -d devdb -i devdb1
PRKP-1001: Error starting instance devdb1 on rac1
CRS-0215 Could not start resource 'ora.devdb.devdb1.inst'
When I run:
crs_start ora.devdb.devdb1.inst
Attempting to start 'ora.devdb.devdb1.inst' on member 'rac1'
Start of 'ora.devdb.devdb1.inst' on member 'rac1' faileld.
rac2: CRS-1019 Resource ora.devdb.devdb1.inst (application) cannot run on rac2
CRS-0215 Could not start resource ora.devdb.devdb1.inst
When I run on rac1 as oracle:
sqlplus /nolog
conn / as sysdba
startup
ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file '/01/app/oracle/product/10.2.0/db_1/dbs/initdevdb1.ora'
Does anyone have some tips to help?
Regards
CristianHi Cristian,
Maybe you have already solved your issue.
Have you checked if the necessary pfile (initdevdb1.ora) is in the directory /01/app/oracle/product/10.2.0/db_1/dbs? if it is there then you might need to check the permissions on the path.
I suppose you did the upgrade using a new binary home for your 10.2.0.3 version where the required parameter file(s) is/are still there in $OLD_ORACLE_HOME/dbs.
If so, then copy them from there to the new $ORACLE_HOME/dbs then give it a try.
Hope this helps.
Regards,
Jozsef -
Itunes quite working on home computer. Now trying to re-install. It says serivce Apple Mobil Device failed to start?
Hello home computer problem,
The following article provides steps that can help sort the Apple Mobile Device service.
How to restart the Apple Mobile Device Service (AMDS) on Windows
http://support.apple.com/kb/TS1567
Cheers,
Allen -
Satellite L300-19F doesn't boot up - Windows failed to start
My notebook doesn't start up, "Windows failed to start. To fix the problem: restart with recovery CD".
I may do this, but first I do want to try to save data (and pics) from hard disk.
Is it possible, for example, to go to DOS during start up with recovery CD, and copy data to USB-stick?
By the way, all my data is on E:\ partition. Will this be overwritten when installing from recovery?
Please advise. Regards.> Is it possible, for example, to go to DOS during start up with recovery CD, and copy data to USB-stick?
No, this is not possible!
If you would boot up using recovery disk then the recover procedure would be started and the whole HDD would be formatted!
You can try to boot using an WinPE
http://www.livecd.com/
This will allows you to boot up the unit and to get access to the data on the HDD
You could also built own Windows live CD
Check this:
http://www.makeuseof.com/tag/how-to-create-your-own-custom-windows-live-cd/ -
PRCR-1070 : Failed to check if resource ora.asm is registered
I am facing issues while installing Grid Infrastrcture 11gR2 on a clustertered environment.
This is a 11gR2 installation with ASMFS on RHEL 5.
The binaries were installed on both the servers successfully but when running root.sh on node 1, there were some errors resulting in the failure of cluster installation.
[root@ora11gR2RAC1 grid]# ./root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-08-09 22:56:28: Parsing the host name
2010-08-09 22:56:28: Checking for super user privileges
2010-08-09 22:56:28: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'ora11gr2rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.gipcd' on 'ora11gr2rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.gpnpd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ora11gr2rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.diskmon' on 'ora11gr2rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.ctssd' on 'ora11gr2rac1' succeeded
ASM created and started successfully.
DiskGroup DATA created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.crsd' on 'ora11gr2rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 9994be77f8764f45bfe2d73eccdb6384.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
1. ONLINE 9994be77f8764f45bfe2d73eccdb6384 (ORCL:DISK1) [DATA]
Located 1 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.crsd' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.asm' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.ctssd' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.cssd' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.gipcd' on 'ora11gr2rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'ora11gr2rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.mdnsd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.gipcd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.gpnpd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ora11gr2rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.diskmon' on 'ora11gr2rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.ctssd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.asm' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.crsd' on 'ora11gr2rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'ora11gr2rac1'
CRS-2676: Start of 'ora.evmd' on 'ora11gr2rac1' succeeded
CRS-0184: Cannot communicate with the CRS daemon.
PRCR-1070 : Failed to check if resource ora.asm is registered
Cannot communicate with crsd
add asm ... failed
clsr_start_dg return error at loc: 70 rc=0
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Create failed, or completed with errors.
create diskgroup DATA ... failed
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Add failed, or completed with errors.
Command return code of 1 (256) from command: /u01/app/11.2.0/grid/bin/crsctl add type ora.registry.acfs.type -basetype ora.local_resource.type -file /u01/app/11.2.0/grid/crs/template/registry.acfs.type
add ora.registry.acfs.type ... failed
PRCR-1070 : Failed to check if resource ora.net1.network is registered
Cannot communicate with crsd
add scan=ora11gRAC-scan ... failed
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... failed
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 930 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.crsd.log
2010-08-10 15:20:02.195: [ AGFW][2587470736] Agent /u01/app/11.2.0/grid/bin/oraagent_oracle with pid:14636 connected to server.
2010-08-10 15:20:02.195: [ AGFW][2587470736] Could not find the resource type [ ora.asm.type ]
2010-08-10 15:20:02.195: [ AGFW][2587470736] Could not find the resource type [ ora.database.type ]
2010-08-10 15:20:02.195: [ AGFW][2587470736] Could not find the resource type [ ora.discoveryserver.type ]
2010-08-10 15:20:02.196: [ AGFW][2587470736] Could not find the resource type [ ora.diskgroup.type ]
2010-08-10 15:20:02.196: [ AGFW][2587470736] Agfw Proxy Server sending message: RESTYPE_ADD[ora.eons.type] ID 8196:90 to the agent /u01/app/11.2.0/grid/bin/oraagent_oracle
2010-08-10 15:20:02.197: [ AGFW][2587470736] Could not find the resource type [ ora.evm.type ]
2010-08-10 15:20:02.197: [ AGFW][2587470736] Could not find the resource type [ ora.gipc.type ]
2010-08-10 15:20:02.197: [ AGFW][2587470736] Could not find the resource type [ ora.gpnp.type ]
2010-08-10 15:20:02.198: [ AGFW][2587470736] Agfw Proxy Server sending message: RESTYPE_ADD[ora.gsd.type] ID 8196:92 to the agent /u01/app/11.2.0/grid/bin/oraagent_oracle
2010-08-10 15:20:02.198: [ AGFW][2587470736] Could not find the resource type [ ora.listener.type ]
2010-08-10 15:20:02.198: [ AGFW][2587470736] Could not find the resource type [ ora.mdns.type ]
2010-08-10 15:20:02.198: [ AGFW][2587470736] Agfw Proxy Server sending message: RESTYPE_ADD[ora.ons.type] ID 8196:94 to the agent /u01/app/11.2.0/grid/bin/oraagent_oracle
2010-08-10 15:20:02.199: [ AGFW][2587470736] Could not find the resource type [ ora.scan_listener.type ]
2010-08-10 15:20:02.199: [ AGFW][2587470736] Could not find the resource type [ ora.service.type ]
2010-08-10 15:20:02.199: [ AGFW][2587470736] Could not find the resource type [ ora.supercluster.type ]
2010-08-10 15:20:02.200: [ AGFW][2587470736] Agfw Proxy Server sending message: RESOURCE_ADD[ora.eons ora11gr2rac1 1] ID 4356:96 to the agent /u01/app/11.2.0/grid/bin/oraagent_oracleAny suggestions?Hi user;
I belive its better to close thread here(change status to answered) and move your issue to Grid Computing >>RAC, ASM & Clusterware Installation forum part
You can get more quick response there
Regard
Helios -
Rs-ora:resource group failed to start on chosen node; it may end up failing
I have configured two node failover cluster environment using netra a/d 1000 storage. When I try to deploy oracle server application it throws the following error
rs-ora: resource group failed to start on chosen node; it may end up failing over to other node(s)
I created metaset and gave one raw did disk to that metaset.
I created logical hostname resource, ha-storage plus resource. Later I brought the resource group to online using following command
#clrg online emM rg-ora
Later I created oracle cluster resource using following command.
#clrs create -g rg-ora -t SUNW.oracle_server -p ORACLE_HOME=/global/oracle/product/10.2.0/db_1 -p ORACLE_SID=infra -p Alert_log_file=/global/oracle/product/10.2.0/db_1/admin/infra/bdump/alert_infra.log -p Connect_string=sysdba/dbadmin1@infra -p Resource_dependencies=rs-ora-has rs-ora
node1 - Validation failed. ORACLE_HOME /global/oracle/product/10.2.0/db_1 does not exist
node1 - ALERT_LOG_FILE /global/oracle/product/10.2.0/db_1/admin/infra/bdump/alert_infra.log doesn't exist
node1 - PARAMETER_FILE: /global/oracle/product/10.2.0/db_1/dbs/initinfra.ora nor server PARAMETER_FILE: /global/oracle/product/10.2.0/db_1/dbs/spfileinfra.ora exists
node1 - This resource depends on a HAStoragePlus resouce that is not online on this node. Ignoring validation errors.
rs-ora: resource group failed to start on chosen node; it may end up failing over to other node(s)
The status of oracle resource shows as follows.
Resource Name Node Name State Status Message
rs-ora node1 Start failed Faulted
I used solaris 10 update 6 patch level is Generic_137137-09, Oracle version 10.2.0, Sun clusters 3.2 update1. Following are the vfstab and /var/adm/messages of both nodes.
Node1#grep ora /etc/vfstab
/dev/md/oradg/dsk/d300 /dev/md/oradg/rdsk/d300 /global/oracle ufs 5 no logging
Node2#grep ora /etc/vfstab
/dev/md/oradg/dsk/d300 /dev/md/oradg/rdsk/d300 /global/oracle ufs 5 no logging
Node1#more /var/adm/messages
Oct 17 05:19:17 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hafoip_prenet_start> for resource <ha-
host-1>, resource group <rg-ora>, node <node1>, timeout <300> seconds
Oct 17 05:19:17 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/
lib/rgm/rt/hafoip/hafoip_prenet_start>:tag=<rg-ora.ha-host-1.10>: Calling security_clnt_connect(..., host=<node1>, sec_typ
e {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:17 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hafoip_prenet_start> completed successfully for
resource <ha-host-1>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <300 seconds>
Oct 17 05:19:17 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hastorageplus_prenet_start> for resour
ce <rs-ora-has>, resource group <rg-ora>, node <node1>, timeout <1800> seconds
Oct 17 05:19:17 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/
lib/rgm/rt/hastorageplus/hastorageplus_prenet_start>:tag=<rg-ora.rs-ora-has.10>: Calling security_clnt_connect(..., host=<tes
tlab5>, sec_type {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:18 node1 Cluster.RGM.rgmd: [ID 375444 daemon.notice] 8 fe_rpc_command: cmd_type(enum):<2>:cmd=<null>:tag=<rg-
ora.rs-ora-has.10>: Calling security_clnt_connect(..., host=<node1>, sec_type {0:WEAK, 1:STRONG, 2:DES} =<0>, ...)
Oct 17 05:19:18 node1 Cluster.RGM.rgmd: [ID 316625 daemon.notice] Timeout monitoring on method tag <rg-ora.rs-ora-has.10>
has been suspended.
Oct 17 05:19:20 node1 Cluster.Framework: [ID 801593 daemon.notice] stdout: becoming primary for oradg
Oct 17 05:19:21 node1 Cluster.RGM.rgmd: [ID 375444 daemon.notice] 8 fe_rpc_command: cmd_type(enum):<3>:cmd=<null>:tag=<rg-
ora.rs-ora-has.10>: Calling security_clnt_connect(..., host=<node1>, sec_type {0:WEAK, 1:STRONG, 2:DES} =<0>, ...)
Oct 17 05:19:21 node1 Cluster.RGM.rgmd: [ID 316625 daemon.notice] Timeout monitoring on method tag <rg-ora.rs-ora-has.10>
has been resumed.
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hastorageplus_prenet_start> completed successful
ly for resource <rs-ora-has>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <1800 seconds>
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hafoip_start> for resource <ha-host-1>
, resource group <rg-ora>, node <node1>, timeout <500> seconds
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/
lib/rgm/rt/hafoip/hafoip_start>:tag=<rg-ora.ha-host-1.0>: Calling security_clnt_connect(..., host=<node1>, sec_type {0:WEA
K, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hafoip_start> completed successfully for resourc
e <ha-host-1>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <500 seconds>
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hafoip_monitor_start> for resource <ha
-host-1>, resource group <rg-ora>, node <node1>, timeout <300> seconds
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hastorageplus_start> for resource <rs-
ora-has>, resource group <rg-ora>, node <node1>, timeout <90> seconds
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 510020 daemon.notice] 46 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/
lib/rgm/rt/hafoip/hafoip_monitor_start>:tag=<rg-ora.ha-host-1.7>: Calling security_clnt_connect(..., host=<node1>, sec_typ
e {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/
lib/rgm/rt/hastorageplus/hastorageplus_start>:tag=<rg-ora.rs-ora-has.0>: Calling security_clnt_connect(..., host=<node1>,
sec_type {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hafoip_monitor_start> completed successfully for
resource <ha-host-1>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <300 seconds>
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hastorageplus_start> completed successfully for
resource <rs-ora-has>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <90 seconds>
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hastorageplus_monitor_start> for resou
rce <rs-ora-has>, resource group <rg-ora>, node <node1>, timeout <90> seconds
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/
lib/rgm/rt/hastorageplus/hastorageplus_monitor_start>:tag=<rg-ora.rs-ora-has.7>: Calling security_clnt_connect(..., host=<tes
tlab5>, sec_type {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:25 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hastorageplus_monitor_start> completed successfu
lly for resource <rs-ora-has>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <90 seconds>
Oct 17 05:19:38 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <bin/oracle_server_validate> for resour
ce <rs-ora>, resource group <rg-ora>, node <node1>, timeout <120> secondsOct 17 05:19:38 node1 Cluster.RGM.rgmd: [ID 375444 daemon.notice] 8 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscor/
oracle_server/bin/oracle_server_validate>:tag=<rg-ora.rs-ora.2>: Calling security_clnt_connect(..., host=<node1>, sec_type
{0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:38 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <bin/oracle_server_validate> completed successful
ly for resource <rs-ora>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <120 seconds>
Oct 17 05:19:38 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <bin/oracle_server_init> for resource <
rs-ora>, resource group <rg-ora>, node <node1>, timeout <30> seconds
Oct 17 05:19:38 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscor
/oracle_server/bin/oracle_server_init>:tag=<rg-ora.rs-ora.4>: Calling security_clnt_connect(..., host=<node1>, sec_type {0
:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:38 node1 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <bin/oracle_server_init> completed successfully f
or resource <rs-ora>, resource group <rg-ora>, node <node1>, time used: 0% of timeout <30 seconds>
Oct 17 05:19:38 node1 Cluster.CCR: [ID 973933 daemon.notice] resource rs-ora added.
Oct 17 05:19:39 node1 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <bin/oracle_server_start> for resource
<rs-ora>, resource group <rg-ora>, node <node1>, timeout <600> seconds
Oct 17 05:19:39 node1 Cluster.RGM.rgmd: [ID 751138 daemon.notice] 47 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscor
/oracle_server/bin/oracle_server_start>:tag=<rg-ora.rs-ora.0>: Calling security_clnt_connect(..., host=<node1>, sec_type {
0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 17 05:19:48 node1 SC[SUNWscor.oracle_server.start]:rg-ora:rs-ora: [ID 876834 daemon.error] Could not start server
Oct 17 05:19:48 node1 Cluster.RGM.rgmd: [ID 938318 daemon.error] Method <bin/oracle_server_start> failed on resource <rs-o
ra> in resource group <rg-ora> [exit code <1>, time used: 1% of timeout <600 seconds>]
Node2# more /var/adm/messages
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group rg-ora state on node node2 change to RG_PENDIN
G_OFFLINE
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource rs-ora-has state on node node2 change to R_MON_STOPP
ING
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource ha-host-1 state on node node2 change to R_MON_STOPPI
NG
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hafoip_monitor_stop> for resource <ha-host
-1>, resource group <rg-ora>, node <node2>, timeout <300> seconds
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hastorageplus_monitor_stop> for resource <
rs-ora-has>, resource group <rg-ora>, node <node2>, timeout <90> seconds
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 268902 daemon.notice] 45 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/
rgm/rt/hafoip/hafoip_monitor_stop>:tag=<rg-ora.ha-host-1.8>: Calling security_clnt_connect(..., host=<node2>, sec_type {0:WEAK
, 1:STRONG, 2:DES} =<1>, ...)
Oct 14 20:20:04 node2 Cluster.RGM.rgmd: [ID 510020 daemon.notice] 46 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/
rgm/rt/hastorageplus/hastorageplus_monitor_stop>:tag=<rg-ora.rs-ora-has.8>: Calling security_clnt_connect(..., host=<node2>, s
ec_type {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hastorageplus_monitor_stop> completed successfully f
or resource <rs-ora-has>, resource group <rg-ora>, node <node2>, time used: 0% of timeout <90 seconds>
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource rs-ora-has state on node node2 change to R_ONLINE_UN
MON
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource rs-ora-has state on node node2 change to R_STOPPING
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource rs-ora-has status on node node2 change to R_FM_UNKNO
WN
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource rs-ora-has status msg on node node2 change to <Stopp
ing>
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hastorageplus_stop> for resource <rs-ora-h
as>, resource group <rg-ora>, node <node2>, timeout <1800> seconds
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 510020 daemon.notice] 46 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/
rgm/rt/hastorageplus/hastorageplus_stop>:tag=<rg-ora.rs-ora-has.1>: Calling security_clnt_connect(..., host=<node2>, sec_type
{0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hafoip_monitor_stop> completed successfully for reso
urce <ha-host-1>, resource group <rg-ora>, node <node2>, time used: 0% of timeout <300 seconds>
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource ha-host-1 state on node node2 change to R_ONLINE_UNM
ON
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hastorageplus_stop> completed successfully for resou
rce <rs-ora-has>, resource group <rg-ora>, node <node2>, time used: 0% of timeout <1800 seconds>
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource rs-ora-has state on node node2 change to R_STOPPED
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource ha-host-1 state on node node2 change to R_STOPPING
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 224900 daemon.notice] launching method <hafoip_stop> for resource <ha-host-1>, res
ource group <rg-ora>, node <node2>, timeout <300> seconds
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource ha-host-1 status on node node2 change to R_FM_UNKNOW
N
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource ha-host-1 status msg on node node2 change to <Stoppi
ng>
Oct 14 20:20:05 node2 Cluster.RGM.rgmd: [ID 510020 daemon.notice] 46 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/
rgm/rt/hafoip/hafoip_stop>:tag=<rg-ora.ha-host-1.1>: Calling security_clnt_connect(..., host=<node2>, sec_type {0:WEAK, 1:STRO
NG, 2:DES} =<1>, ...)
Oct 14 20:20:06 node2 ip: [ID 678092 kern.notice] TCP_IOC_ABORT_CONN: local = 192.168.032.244:0, remote = 000.000.000.000:0, s
tart = -2, end = 6
Oct 14 20:20:06 node2 ip: [ID 302654 kern.notice] TCP_IOC_ABORT_CONN: aborted 0 connection
Oct 14 20:20:06 node2 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource ha-host-1 status on node node2 change to R_FM_OFFLIN
E
Oct 14 20:20:06 node2 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource ha-host-1 status msg on node node2 change to <Logica
lHostname offline.>
Oct 14 20:20:06 node2 Cluster.RGM.rgmd: [ID 515159 daemon.notice] method <hafoip_stop> completed successfully for resource <ha
-host-1>, resource group <rg-ora>, node <node2>, time used: 0% of timeout <300 seconds>
Oct 14 20:20:06 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource ha-host-1 state on node node2 change to R_OFFLINE
Oct 14 20:20:06 node2 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource rs-ora-has state on node node2 change to R_POSTNET_S
TOPPING
Maybe you are looking for
-
Error Message when we create a Service Ticket in SAP CRM 2007
Hi All we are having a serious production issue when we create a S.Ticket in SAP CRM 2007 We get the following error message we are not trying to send any information to R/3 We have a planned go live this saturday. So any help would be greatly apprec
-
Can't make it to the login window iMac G5 10.5.8
When I turned my iMac on this morning it went through all of the "normal" steps (gong, grey screen, apple logo, progress wheel, blue screen) and stopped at the blue screen. Thinking it was just a fluke, I shut the machine down and restarted, only to
-
Dears , i want to display the image in sidebar region , i make all steps : 1- add column in the table ( FLAG blob , FILENAME varchar2(4000), MIMTYPE varchar2(4000), FILESIZE varchar2(4000) )2- create new region html type and but in condition PL
-
SET RUNTIMESUBVARS - my favorite new feature in 11.1.2.3
Hi there, I must say it! My favorite new feature in 11.1.2.3: SET RUNTIMESUBVARS see here: http://docs.oracle.com/cd/E40248_01/epm.1112/essbase_tech_ref/frameset.htm?set_runtimesubvars.html No longer the need in a MaxL scipt to create and/or set serv
-
Old Airport card not recognizing Apple Airport Router.
Hello there, I've just been given an old G4 (digital audio) to use for university and after doing some research brought an old Airport card off of ebay. After putting into the relevant slot and plugging it in, it started to recognize all of my neighb