RAC database restarting by itself
All,
I have a 2 node Win 2003 64bit cluster using ASM.
Node 1 fell over yesterday and restarted about 2 minutes later. I know why it fell over (there's another patch to be applied). Is there somewhere within the RAC config to say to RAC to restart the Node 1 instance after a certain time? I can't find anything in the RAC logs (css, evm & crs). Any ideas?
Many thanks
Vic
> Is there somewhere within the RAC config to say to RAC to restart the Node 1 instance after a certain time?
No. It is the purpose of each cluster node to be part of the collective. So there is not twiddling of thumbs on the part of a node outside the cluster. It will attempt to rejoin it now and not later. As it should. (and resistance is futile)
The CRS service is configured to autostart when a node boots. When CRS successfully starts, it will then start all services it has on that node. Node apps, ASM, RAC, etc.
That is the way it works and should work.. not sure what you are expecting by suggesting that it should only join again after a certain time?
Similar Messages
-
Rolling restart of rac database
what happens to the already present connections while doing a rolling restart of rac database?
if it is already established connection to a node and doing something what will happen, will it wait till that session completes what it is doing?>>what happens to the already present connections while doing a rolling restart of rac database?
>> if it is already established connection to a node and doing something what will happen, will it wait till that session completes what it is doing?
As said if TAF is configured only SELECT queries will be failed over to other instance. If it is DDL then it success will depend on the parameter such as IMMEDIATE/TRANSNATIONAL supplied with SHUTDOWN command.
HTH,
Pradeep -
Hi,
Oracle RAC database 10.2.0.3/RedHat4 with 2 nodes.
In the begining we had an error ORA-600[12803] so only sys can connect to database I find the note 1026653.6 this note said that we need to create AUDSES$ sequence but befor that we have to restart the database.
When we stop the datanbase we had another ORA-600 and it's impossible to start it!!
Here is a coppy of our alert file:
Picked latch-free SCN scheme 2
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.3.0.
System parameters with non-default values:
processes = 300
sessions = 335
sga_max_size = 524288000
__shared_pool_size = 310378496
__large_pool_size = 4194304
__java_pool_size = 8388608
__streams_pool_size = 8388608
spfile = +DATA/osista/spfileosista.ora
nls_language = FRENCH
nls_territory = FRANCE
nls_length_semantics = CHAR
sga_target = 524288000
control_files = DATA/osista/controlfile/control01.ctl, DATA/osista/controlfile/control02.ctl
db_block_size = 8192
__db_cache_size = 184549376
compatible = 10.2.0.3.0
log_archive_dest_1 = LOCATION=USE_DB_RECOVERY_FILE_DEST
db_file_multiblock_read_count= 16
cluster_database = TRUE
cluster_database_instances= 2
db_create_file_dest = +DATA
db_recovery_file_dest = +FLASH
db_recovery_file_dest_size= 68543315968
thread = 2
instance_number = 2
undo_management = AUTO
undo_tablespace = UNDOTBS2
undo_retention = 29880
remote_login_passwordfile= EXCLUSIVE
db_domain =
dispatchers = (PROTOCOL=TCP) (SERVICE=OSISTAXDB)
local_listener = (address=(protocol=tcp)(port=1521)(host=132.147.160.243))
remote_listener = LISTENERS_OSISTA
job_queue_processes = 10
background_dump_dest = /oracle/product/admin/OSISTA/bdump
user_dump_dest = /oracle/product/admin/OSISTA/udump
core_dump_dest = /oracle/product/admin/OSISTA/cdump
audit_file_dest = /oracle/product/admin/OSISTA/adump
db_name = OSISTA
open_cursors = 300
pga_aggregate_target = 104857600
aq_tm_processes = 1
Cluster communication is configured to use the following interface(s) for this instance
172.16.0.2
Wed Jun 13 11:04:30 2012
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=8560
DIAG started with pid=3, OS id=8562
PSP0 started with pid=4, OS id=8566
LMON started with pid=5, OS id=8570
LMD0 started with pid=6, OS id=8574
LMS0 started with pid=7, OS id=8576
LMS1 started with pid=8, OS id=8580
MMAN started with pid=9, OS id=8584
DBW0 started with pid=10, OS id=8586
LGWR started with pid=11, OS id=8588
CKPT started with pid=12, OS id=8590
SMON started with pid=13, OS id=8592
RECO started with pid=14, OS id=8594
CJQ0 started with pid=15, OS id=8596
MMON started with pid=16, OS id=8598
Wed Jun 13 11:04:31 2012
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=17, OS id=8600
Wed Jun 13 11:04:31 2012
starting up 1 shared server(s) ...
Wed Jun 13 11:04:31 2012
lmon registered with NM - instance id 2 (internal mem no 1)
Wed Jun 13 11:04:31 2012
Reconfiguration started (old inc 0, new inc 2)
List of nodes:
1
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Wed Jun 13 11:04:31 2012
LMS 0: 0 GCS shadows cancelled, 0 closed
Wed Jun 13 11:04:31 2012
LMS 1: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Wed Jun 13 11:04:31 2012
LMS 0: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:04:31 2012
LMS 1: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:04:31 2012
Submitted all GCS remote-cache requests
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=20, OS id=8877
Wed Jun 13 11:04:43 2012
alter database mount
Wed Jun 13 11:04:43 2012
This instance was first to mount
Wed Jun 13 11:04:43 2012
Starting background process ASMB
ASMB started with pid=25, OS id=10068
Starting background process RBAL
RBAL started with pid=26, OS id=10072
Wed Jun 13 11:04:47 2012
SUCCESS: diskgroup DATA was mounted
Wed Jun 13 11:04:51 2012
Setting recovery target incarnation to 1
Wed Jun 13 11:04:52 2012
Successful mount of redo thread 2, with mount id 3005749259
Wed Jun 13 11:04:52 2012
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: alter database mount
Wed Jun 13 11:05:06 2012
alter database open
Wed Jun 13 11:05:06 2012
This instance was first to open
Wed Jun 13 11:05:06 2012
Beginning crash recovery of 1 threads
parallel recovery started with 2 processes
Wed Jun 13 11:05:07 2012
Started redo scan
Wed Jun 13 11:05:07 2012
Completed redo scan
61 redo blocks read, 4 data blocks need recovery
Wed Jun 13 11:05:07 2012
Started redo application at
Thread 1: logseq 7924, block 3, scn 506098125
Wed Jun 13 11:05:07 2012
Recovery of Online Redo Log: Thread 1 Group 2 Seq 7924 Reading mem 0
Mem# 0: +DATA/osista/onlinelog/group_2.372.742132543
Wed Jun 13 11:05:07 2012
Completed redo application
Wed Jun 13 11:05:07 2012
Completed crash recovery at
Thread 1: logseq 7924, block 64, scn 506118186
4 data blocks read, 4 data blocks written, 61 redo blocks read
Switch log for thread 1 to sequence 7925
Picked broadcast on commit scheme to generate SCNs
db_recovery_file_dest_size of 65368 MB is 0.61% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
SUCCESS: diskgroup FLASH was mounted
SUCCESS: diskgroup FLASH was dismounted
Thread 1 advanced to log sequence 7926
SUCCESS: diskgroup FLASH was mounted
SUCCESS: diskgroup FLASH was dismounted
Thread 1 advanced to log sequence 7927
Wed Jun 13 11:05:11 2012
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=31, OS id=12747
Wed Jun 13 11:05:11 2012
ARC0: Archival started
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=32, OS id=12749
Wed Jun 13 11:05:12 2012
Thread 2 opened at log sequence 7176
Current log# 4 seq# 7176 mem# 0: +DATA/osista/onlinelog/group_4.289.742134597
Wed Jun 13 11:05:12 2012
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
Wed Jun 13 11:05:12 2012
Successful open of redo thread 2
Wed Jun 13 11:05:12 2012
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Jun 13 11:05:12 2012
ARC0: Becoming the heartbeat ARCH
Wed Jun 13 11:05:12 2012
SMON: enabling cache recovery
Wed Jun 13 11:05:15 2012
Successfully onlined Undo Tablespace 20.
Wed Jun 13 11:05:15 2012
SMON: enabling tx recovery
Wed Jun 13 11:05:15 2012
Database Characterset is AL32UTF8
Wed Jun 13 11:05:16 2012
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_9174.trc:
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []
Wed Jun 13 11:05:16 2012
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_9174.trc:
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Instance terminated by USER, pid = 9174
ORA-1092 signalled during: alter database open...
Wed Jun 13 11:06:16 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Interface type 1 eth0 172.16.0.0 configured from OCR for use as a cluster interconnect
Interface type 1 bond0 132.147.160.0 configured from OCR for use as a public interface
Picked latch-free SCN scheme 2
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.3.0.
System parameters with non-default values:
processes = 300
sessions = 335
sga_max_size = 524288000
__shared_pool_size = 314572800
__large_pool_size = 4194304
__java_pool_size = 8388608
__streams_pool_size = 8388608
spfile = +DATA/osista/spfileosista.ora
nls_language = FRENCH
nls_territory = FRANCE
nls_length_semantics = CHAR
sga_target = 524288000
control_files = DATA/osista/controlfile/control01.ctl, DATA/osista/controlfile/control02.ctl
db_block_size = 8192
__db_cache_size = 180355072
compatible = 10.2.0.3.0
log_archive_dest_1 = LOCATION=USE_DB_RECOVERY_FILE_DEST
db_file_multiblock_read_count= 16
cluster_database = TRUE
cluster_database_instances= 2
db_create_file_dest = +DATA
db_recovery_file_dest = +FLASH
db_recovery_file_dest_size= 68543315968
thread = 2
instance_number = 2
undo_management = AUTO
undo_tablespace = UNDOTBS2
undo_retention = 29880
remote_login_passwordfile= EXCLUSIVE
db_domain =
dispatchers = (PROTOCOL=TCP) (SERVICE=OSISTAXDB)
local_listener = (address=(protocol=tcp)(port=1521)(host=132.147.160.243))
remote_listener = LISTENERS_OSISTA
job_queue_processes = 10
background_dump_dest = /oracle/product/admin/OSISTA/bdump
user_dump_dest = /oracle/product/admin/OSISTA/udump
core_dump_dest = /oracle/product/admin/OSISTA/cdump
audit_file_dest = /oracle/product/admin/OSISTA/adump
db_name = OSISTA
open_cursors = 300
pga_aggregate_target = 104857600
aq_tm_processes = 1
Cluster communication is configured to use the following interface(s) for this instance
172.16.0.2
Wed Jun 13 11:06:16 2012
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=18682
DIAG started with pid=3, OS id=18684
PSP0 started with pid=4, OS id=18695
LMON started with pid=5, OS id=18704
LMD0 started with pid=6, OS id=18721
LMS0 started with pid=7, OS id=18735
LMS1 started with pid=8, OS id=18753
MMAN started with pid=9, OS id=18767
DBW0 started with pid=10, OS id=18788
LGWR started with pid=11, OS id=18796
CKPT started with pid=12, OS id=18799
SMON started with pid=13, OS id=18801
RECO started with pid=14, OS id=18803
CJQ0 started with pid=15, OS id=18805
MMON started with pid=16, OS id=18807
Wed Jun 13 11:06:17 2012
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=17, OS id=18809
Wed Jun 13 11:06:17 2012
starting up 1 shared server(s) ...
Wed Jun 13 11:06:17 2012
lmon registered with NM - instance id 2 (internal mem no 1)
Wed Jun 13 11:06:17 2012
Reconfiguration started (old inc 0, new inc 2)
List of nodes:
1
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Wed Jun 13 11:06:18 2012
LMS 0: 0 GCS shadows cancelled, 0 closed
Wed Jun 13 11:06:18 2012
LMS 1: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Wed Jun 13 11:06:18 2012
LMS 0: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:06:18 2012
LMS 1: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:06:18 2012
Submitted all GCS remote-cache requests
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=20, OS id=18816
Wed Jun 13 11:06:18 2012
ALTER DATABASE MOUNT
Wed Jun 13 11:06:18 2012
This instance was first to mount
Wed Jun 13 11:06:18 2012
Reconfiguration started (old inc 2, new inc 4)
List of nodes:
0 1
Wed Jun 13 11:06:18 2012
Starting background process ASMB
Wed Jun 13 11:06:18 2012
Global Resource Directory frozen
Communication channels reestablished
ASMB started with pid=22, OS id=18913
Starting background process RBAL
* domain 0 valid = 0 according to instance 0
Wed Jun 13 11:06:18 2012
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Wed Jun 13 11:06:18 2012
LMS 0: 0 GCS shadows cancelled, 0 closed
Wed Jun 13 11:06:18 2012
LMS 1: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Wed Jun 13 11:06:18 2012
LMS 0: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:06:18 2012
LMS 1: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:06:18 2012
Submitted all GCS remote-cache requests
Fix write in gcs resources
RBAL started with pid=23, OS id=18917
Reconfiguration complete
Wed Jun 13 11:06:22 2012
SUCCESS: diskgroup DATA was mounted
Wed Jun 13 11:06:26 2012
Setting recovery target incarnation to 1
Wed Jun 13 11:06:26 2012
Successful mount of redo thread 2, with mount id 3005703530
Wed Jun 13 11:06:26 2012
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE MOUNT
Wed Jun 13 11:06:27 2012
ALTER DATABASE OPEN
This instance was first to open
Wed Jun 13 11:06:27 2012
Beginning crash recovery of 1 threads
parallel recovery started with 2 processes
Wed Jun 13 11:06:27 2012
Started redo scan
Wed Jun 13 11:06:27 2012
Completed redo scan
61 redo blocks read, 4 data blocks need recovery
Wed Jun 13 11:06:28 2012
Started redo application at
Thread 2: logseq 7176, block 3
Wed Jun 13 11:06:28 2012
Recovery of Online Redo Log: Thread 2 Group 4 Seq 7176 Reading mem 0
Mem# 0: +DATA/osista/onlinelog/group_4.289.742134597
Wed Jun 13 11:06:28 2012
Completed redo application
Wed Jun 13 11:06:28 2012
Completed crash recovery at
Thread 2: logseq 7176, block 64, scn 506138248
4 data blocks read, 4 data blocks written, 61 redo blocks read
Picked broadcast on commit scheme to generate SCNs
Wed Jun 13 11:06:28 2012
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=28, OS id=19692
Wed Jun 13 11:06:28 2012
ARC0: Archival started
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=29, OS id=19695
Wed Jun 13 11:06:28 2012
Thread 2 advanced to log sequence 7177
Thread 2 opened at log sequence 7177
Current log# 3 seq# 7177 mem# 0: +DATA/osista/onlinelog/group_3.291.742134597
Successful open of redo thread 2
Wed Jun 13 11:06:28 2012
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Jun 13 11:06:28 2012
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Wed Jun 13 11:06:28 2012
ARC1: Becoming the heartbeat ARCH
Wed Jun 13 11:06:28 2012
SMON: enabling cache recovery
Wed Jun 13 11:06:28 2012
db_recovery_file_dest_size of 65368 MB is 0.61% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
SUCCESS: diskgroup FLASH was mounted
SUCCESS: diskgroup FLASH was dismounted
Wed Jun 13 11:06:31 2012
Successfully onlined Undo Tablespace 20.
Wed Jun 13 11:06:31 2012
SMON: enabling tx recovery
Wed Jun 13 11:06:31 2012
Database Characterset is AL32UTF8
Wed Jun 13 11:06:31 2012
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_19596.trc:
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []
Wed Jun 13 11:06:32 2012
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_19596.trc:
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Instance terminated by USER, pid = 19596
ORA-1092 signalled during: ALTER DATABASE OPEN...
Wed Jun 13 11:11:35 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Interface type 1 eth0 172.16.0.0 configured from OCR for use as a cluster interconnect
Interface type 1 bond0 132.147.160.0 configured from OCR for use as a public interface
Picked latch-free SCN scheme 2
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.3.0.
System parameters with non-default values:
processes = 300
sessions = 335
sga_max_size = 524288000
__shared_pool_size = 318767104
__large_pool_size = 4194304
__java_pool_size = 8388608
__streams_pool_size = 8388608
spfile = +DATA/osista/spfileosista.ora
nls_language = FRENCH
nls_territory = FRANCE
nls_length_semantics = CHAR
sga_target = 524288000
control_files = DATA/osista/controlfile/control01.ctl, DATA/osista/controlfile/control02.ctl
db_block_size = 8192
__db_cache_size = 176160768
compatible = 10.2.0.3.0
log_archive_dest_1 = LOCATION=USE_DB_RECOVERY_FILE_DEST
db_file_multiblock_read_count= 16
cluster_database = TRUE
cluster_database_instances= 2
db_create_file_dest = +DATA
db_recovery_file_dest = +FLASH
db_recovery_file_dest_size= 68543315968
thread = 2
instance_number = 2
undo_management = AUTO
undo_tablespace = UNDOTBS2
undo_retention = 29880
remote_login_passwordfile= EXCLUSIVE
db_domain =
dispatchers = (PROTOCOL=TCP) (SERVICE=OSISTAXDB)
local_listener = (address=(protocol=tcp)(port=1521)(host=132.147.160.243))
remote_listener = LISTENERS_OSISTA
job_queue_processes = 10
background_dump_dest = /oracle/product/admin/OSISTA/bdump
user_dump_dest = /oracle/product/admin/OSISTA/udump
core_dump_dest = /oracle/product/admin/OSISTA/cdump
audit_file_dest = /oracle/product/admin/OSISTA/adump
db_name = OSISTA
open_cursors = 300
pga_aggregate_target = 104857600
aq_tm_processes = 1
Cluster communication is configured to use the following interface(s) for this instance
172.16.0.2
Wed Jun 13 11:11:35 2012
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=16101
DIAG started with pid=3, OS id=16103
PSP0 started with pid=4, OS id=16105
LMON started with pid=5, OS id=16107
LMD0 started with pid=6, OS id=16110
LMS0 started with pid=7, OS id=16112
LMS1 started with pid=8, OS id=16116
MMAN started with pid=9, OS id=16120
DBW0 started with pid=10, OS id=16132
LGWR started with pid=11, OS id=16148
CKPT started with pid=12, OS id=16169
SMON started with pid=13, OS id=16185
RECO started with pid=14, OS id=16203
CJQ0 started with pid=15, OS id=16219
MMON started with pid=16, OS id=16227
Wed Jun 13 11:11:36 2012
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=17, OS id=16229
Wed Jun 13 11:11:36 2012
starting up 1 shared server(s) ...
Wed Jun 13 11:11:36 2012
lmon registered with NM - instance id 2 (internal mem no 1)
Wed Jun 13 11:11:36 2012
Reconfiguration started (old inc 0, new inc 2)
List of nodes:
1
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Wed Jun 13 11:11:36 2012
LMS 0: 0 GCS shadows cancelled, 0 closed
Wed Jun 13 11:11:36 2012
LMS 1: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Wed Jun 13 11:11:36 2012
LMS 1: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:11:36 2012
LMS 0: 0 GCS shadows traversed, 0 replayed
Wed Jun 13 11:11:36 2012
Submitted all GCS remote-cache requests
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=20, OS id=16235
Wed Jun 13 11:11:37 2012
ALTER DATABASE MOUNT
Wed Jun 13 11:11:37 2012
This instance was first to mount
Wed Jun 13 11:11:37 2012
Starting background process ASMB
ASMB started with pid=22, OS id=16343
Starting background process RBAL
RBAL started with pid=23, OS id=16347
Wed Jun 13 11:11:44 2012
SUCCESS: diskgroup DATA was mounted
Wed Jun 13 11:11:49 2012
Setting recovery target incarnation to 1
Wed Jun 13 11:11:49 2012
Successful mount of redo thread 2, with mount id 3005745065
Wed Jun 13 11:11:49 2012
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE MOUNT
Wed Jun 13 11:22:25 2012
alter database open
This instance was first to open
Wed Jun 13 11:22:26 2012
Beginning crash recovery of 1 threads
parallel recovery started with 2 processes
Wed Jun 13 11:22:26 2012
Started redo scan
Wed Jun 13 11:22:26 2012
Completed redo scan
61 redo blocks read, 4 data blocks need recovery
Wed Jun 13 11:22:26 2012
Started redo application at
Thread 1: logseq 7927, block 3
Wed Jun 13 11:22:26 2012
Recovery of Online Redo Log: Thread 1 Group 1 Seq 7927 Reading mem 0
Mem# 0: +DATA/osista/onlinelog/group_1.283.742132543
Wed Jun 13 11:22:26 2012
Completed redo application
Wed Jun 13 11:22:26 2012
Completed crash recovery at
Thread 1: logseq 7927, block 64, scn 506178382
4 data blocks read, 4 data blocks written, 61 redo blocks read
Switch log for thread 1 to sequence 7928
Picked broadcast on commit scheme to generate SCNs
Wed Jun 13 11:22:27 2012
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=31, OS id=13010
Wed Jun 13 11:22:27 2012
ARC0: Archival started
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=32, OS id=13033
Wed Jun 13 11:22:27 2012
Thread 2 opened at log sequence 7178
Current log# 4 seq# 7178 mem# 0: +DATA/osista/onlinelog/group_4.289.742134597
Successful open of redo thread 2
Wed Jun 13 11:22:27 2012
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Jun 13 11:22:27 2012
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Wed Jun 13 11:22:27 2012
ARC1: Becoming the heartbeat ARCH
Wed Jun 13 11:22:27 2012
SMON: enabling cache recovery
Wed Jun 13 11:22:30 2012
db_recovery_file_dest_size of 65368 MB is 0.61% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
SUCCESS: diskgroup FLASH was mounted
SUCCESS: diskgroup FLASH was dismounted
Wed Jun 13 11:22:31 2012
Successfully onlined Undo Tablespace 20.
Wed Jun 13 11:22:31 2012
SMON: enabling tx recovery
Wed Jun 13 11:22:32 2012
Database Characterset is AL32UTF8
Wed Jun 13 11:22:32 2012
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_11751.trc:
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []
Wed Jun 13 11:22:33 2012
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_11751.trc:
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Instance terminated by USER, pid = 11751
ORA-1092 signalled during: alter database open...
regards,Hi;
Errors in file /oracle/product/admin/OSISTA/udump/osista2_ora_9174.trc:Did you check trc file?
ORA-00600: code d'erreur interne, arguments : [kokiasg1], [], [], [], [], [], [], []You are getting oracle internal error(ORA 600) which mean you could need to work wiht oracle support team. Please see below note, if its not help than i suggest log a sr:
Troubleshoot an ORA-600 or ORA-7445 Error Using the Error Lookup Tool [ID 153788.1]
for your future rac issue please use Forum Home » High Availability » RAC, ASM & Clusterware Installation which is RAC dedicated forum site.
Regard
Helios -
Time to connect to RAC database
Hi,
I have installed 11gr2 RAC database on windows server 2008 on ASM. I can connect to existing prod db withing a second but it is taking almost 20Seconds for me to connect to the RAC database. Is it because of the SCAN IPs?
Can anyone let me know the reason behind it and the resolution on it?
Thanks...
Edited by: user2995637 on Jul 22, 2011 6:15 AM
Edited by: user2995637 on Jul 22, 2011 6:16 AMuser2995637 wrote:
I have installed 11gr2 RAC database on windows server 2008 on ASM. I can connect to existing prod db withing a second but it is taking almost 20Seconds for me to connect to the RAC database. Is it because of the SCAN IPs?No. Should not be the case, normally.
Yes, some TNS connections can be slower than others. There are a number of factors. The initial connection may result in a redirect to another listener on a different IP address (e.g. load balancing). This will be slower than a direct connection (no redirect).
In such a case for example, where a redirect occurs, the client with receive a redirect to a new hostname. It needs to resolve that to an IP. This can be slow due to DNS issues (either with the DNS itself, or with the resolution scope config of the client).
The connection can request a dedicated sever instead of a shared server. A shared server connection is simply a hand-off of that client to an existing dispatcher process. A dedicated server connection requires the Listener to launch a brand new Oracle server process, for that process to attach itself to the SGA, and then resume the connection/conversation with the client. Thus shared server is by its nature a lot faster to establish a connection with than a dedicated server.
So there are a number of moving parts that can be slow. And you will need to isolate these and test each in turn to determine where the performance knock is. -
Upgrade of RAC database in 10g or 11g
Hi,
Can we upgrade RAC database without downtime like we do patch?
Regardsuser602441 wrote:
Hi,
Can we upgrade RAC database without downtime like we do patch?
RegardsHi,
The Oracle Clusterware software always fully supports rolling upgrades, while the ASM software is rolling upgradeable at version 11.1.0.6 and beyond.
Rolling upgrade: we mean upgrading software (Oracle Database, Oracle Clusterware, ASM or the OS itself) while the cluster is operational by shutting down a node, upgrading the software on that node, and then reintegrating it into the cluster, and so forth one node at a time until all the nodes in the cluster are at the new software level.
For the Oracle Database software (RAC), it is possible only for certain single patches that are marked as rolling upgrade compatible. Most Bundle patches and Critical Patch Updates (CPU) are rolling upgradeable. Patchsets and DB version (10g to 11g) changes are not supported in a rolling fashion, one reason that this may be impossible is that across major releases, there may be incompatible versions of the system tablespace, for example. To upgrade these in a rolling fashion one will need to use a logical standby with Oracle Database 10g or 11g.
Regards,
Levi Pereira -
Data pump export full RAC database in window single DB by network_link
Hi Experts,
I have a window 32 bit 10.2 database.
I try to export a full rac database (350G some version with window DB) in window single database by dblink.
exp syntax as
exdpd salemanager/********@sale FULL=y DIRECTORY=dataload NETWORK_LINK=sale.net DUMPFILE=sale20100203.dmp LOGFILE=salelog20100203.log
I created a dblink with fixed instance3. It was working for two day and display message as
ORA-31693: Table data object "SALE_AUDIT"."AU_ITEM_IN" failed to load/unload and is being skipped due to error:
ORA-29913: error in executing ODCIEXTTABLEPOPULATE callout
ORA-01555: snapshot too old: rollback segment number with name "" too small
ORA-02063: preceding line from sale.netL
I stoped export and checked window target alert log.
I saw some message as
kupprdp: master process DM00 started with pid=16, OS id=4444
to execute - SYS.KUPM$MCP.MAIN('SYS_EXPORT_FULL_02', 'SYSTEM', 'KUPC$C_1_20100202235235', 'KUPC$S_1_20100202235235', 0);
Tue Feb 02 23:56:12 2010
The value (30) of MAXTRANS parameter ignored.
kupprdp: master process DM00 started with pid=17, OS id=4024
to execute - SYS.KUPM$MCP.MAIN('SYS_EXPORT_FULL_01', 'SALE', 'KUPC$C_1_20100202235612', 'KUPC$S_1_20100202235612', 0);
kupprdp: worker process DW01 started with worker id=1, pid=18, OS id=2188
to execute - SYS.KUPW$WORKER.MAIN('SYS_EXPORT_FULL_01', 'SALE');
In RAC instance alert.log. I saw message as
SELECT /*+ NO_PARALLEL ("KU$") */ "ID","RAW_DATA","TRANSM_ID","RECEIVED_UTC_DATE ","RECEIVED_FROM","ACTION","ORAUSER",
"ORADATE" FROM RELATIONAL("SALE_AUDIT"."A U_ITEM_IN") "KU$"
How to fixed this error?
add more undotbs space in RAC instance 3 or window database?
Thanbks
Jim
Edited by: user589812 on Feb 4, 2010 10:15 AMI usually increate undo space. Is your undo retention set smaller than the time it takes to run the job? If it is, I would think you would need to do that. If not, then I would think it would be the space. You were in the process of exporting data when the job failed which is what I would have expected. Basically, DataPump want to export each table consistent to itself. Let's say that one of your tables is partitioned and it has a large partition and a smaller partition. DataPump attempts to export the larger partiitons first and it remembers the scn for that partition. When the smaller partitions are exported, it will use the scn to get the data from that partition as it would have looked like if it exported the data when the first partiiton was used. If you don't have partitioned tables, then do you know if some of the tables in the export job (I know it's full so that includes just about all of them) are having data added to them or removed from them? I can't think of anything else that would need undo while exporting data.
Dean -
hi
one of our RAC environment keep restarting.
i've disable the init.cssd, init.crs, init.evmd in the /etc/inittab in order to check the logs.
this is the situation:
crsd.log:
2009-02-04 00:09:00.118: [ COMMCRS][9]clsc_connect: (8000000100318640) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_node1_loud))
2009-02-04 00:09:00.132: [ CSSCLNT][1]clsssInitNative: connect failed, rc 9
2009-02-04 00:09:00.134: [ CRSRTI][1]32CSS is not ready. Received status 3 from CSS. Waiting for good status ..
2009-02-04 00:09:08.016: [ CRSD][1]32Daemon Version: 10.2.0.2.0 Active Version: 10.2.0.2.0
2009-02-04 00:09:08.016: [ CRSD][1]32Active Version and Software Version are same
2009-02-04 00:09:08.017: [ CRSMAIN][1]32Initializing OCR
2009-02-04 00:09:08.037: [ OCRRAW][1]proprioo: for disk 0 (/dev/rdsk/ora_ocr_raw), id match (1), my id set (752560621,1028247821) total id sets (1), 1st set
(752560621,1028247821), 2nd set (0,0) my votes (2), total votes (2)
2009-02-04 00:09:08.140: [ CSSCLNT][24]clssgsGroupJoin: CSS has not reached fatal mode.Registration is not yet safe. Retrying
ocssd.log:
[ CSSD]2009-02-03 21:52:08.651 [9] >USER: clssnmHandleUpdate: NODE 1 (node1l) IS ACTIVE MEMBER OF CLUSTER
[ CSSD]2009-02-03 21:52:08.651 [9] >TRACE: clssnmHandleUpdate: diskTimeout set to (200000)ms
[ CSSD]2009-02-03 21:52:08.651 [16] >TRACE: clssnmWaitForAcks: done, msg type(15)
[ CSSD]2009-02-03 21:52:08.651 [16] >TRACE: clssnmDoSyncUpdate: Sync Complete!
[ CSSD]2009-02-03 21:52:08.722 [1] >USER: NMEVENT_SUSPEND [00][00][00][00]
[ CSSD]2009-02-03 21:52:08.724 [17] >TRACE: clssgmReconfigThread: started for reconfig (1)
[ CSSD]2009-02-03 21:52:08.749 [17] >USER: NMEVENT_RECONFIG [00][00][00][02]
[ CSSD]2009-02-03 21:52:08.749 [17] >TRACE: clssgmEstablishConnections: 1 nodes in cluster incarn 1
[ CSSD]2009-02-03 21:52:08.751 [13] >TRACE: clssgmPeerListener: connects done (1/1)
[ CSSD]2009-02-03 21:52:08.752 [17] >TRACE: clssgmEstablishMasterNode: MASTER for 1 is node(1) birth(1)
[ CSSD]2009-02-03 21:52:08.752 [17] >TRACE: clssgmChangeMasterNode: requeued 0 RPCs
[ CSSD]2009-02-03 21:52:08.752 [17] >TRACE: clssgmMasterCMSync: Synchronizing group/lock status
[ CSSD]2009-02-03 21:52:08.752 [17] >TRACE: clssgmMasterSendDBDone: group/lock status synchronization complete
[ CSSD]CLSS-3000: reconfiguration successful, incarnation 1 with 1 nodes
[ CSSD]CLSS-3001: local node number 1, master node number 1
[ CSSD]2009-02-03 21:52:08.753 [17] >TRACE: clssgmReconfigThread: completed for reconfig(1), with status(1)
[ CSSD]2009-02-03 21:52:08.863 [10] >TRACE: clssgmClientConnectMsg: Connect from con(80000001008fd2a0) proc(8000000100ae26a8) pid() proto(10:2:1:1)
[ CSSD]2009-02-03 21:52:08.864 [10] >TRACE: clssgmClientConnectMsg: Connect from con(8000000100ae0128) proc(8000000100ae2a10) pid() proto(10:2:1:1) from con(8000000100aa32c0) proc(8000000100aa5b90) pid() proto(10:2:1:1)
alertlog:
[cssd(2535)]CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 .
2009-02-03 23:55:20.821
[cssd(2575)]CRS-1605:CSSD voting file is online: /dev/rdsk/ora_voting_raw. Detai ls in /work/crs/product/10.2/crs/log/lourmel/cssd/ocssd.log.
2009-02-03 23:55:28.376
evmd.log:
Oracle Database 10g CRS Release 10.2.0.2.0 Production Copyright 1996, 2004, Oracle. All rights reserved
2009-02-04 00:08:58.331: [ EVMD][1]32EVMD waiting for CSS to be ready err = 3
2009-02-04 00:08:59.939: [ COMMCRS][9]clsc_connect: (800000010007d658) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_node1_loud))
2009-02-04 00:08:59.946: [ CSSCLNT][1]clsssInitNative: connect failed, rc 9
2009-02-04 00:08:59.948: [ EVMD][1]32EVMD waiting for CSS to be ready err = 3
2009-02-04 00:09:07.596: [ CSSCLNT][1]clssgsGroupJoin: CSS has not reached fatal mode.Registration is not yet safe. Retrying
syslog:
Feb 4 00:08:41 lourmel syslog: Oracle Cluster Ready Services starting up automatically.
Feb 4 00:08:45 lourmel sfd[2153]: starting the daemon.
Feb 4 00:08:45 lourmel su: + tty?? root-orac
Feb 4 00:08:45 lourmel krsd[2152]: Delay time is 300 seconds
Feb 4 00:08:43 lourmel syslog: Oracle Cluster Ready Services starting up automatically.
Feb 4 00:08:52 lourmel above message repeats 2 times
Feb 4 00:08:52 lourmel syslog: Cluster Ready Services completed waiting on dependencies.
Feb 4 00:08:53 lourmel syslog: Running CRSD with TZ =
when i checked(befor the restart) the command crs_stat i got the message:
ORA-0184: Cannot communicate wirh CRS
crsctl check crs gives us:
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
as i said befor, the machine always restarting
anyone have an idea?? pleaseDear All,
I recently upgrade the Few RAC setups with Oracle 10g Patchset 3 (10.2.0.4) on Linux Servers
In one of the RAC setup, found servers are rebooting daily. The same setup was working fine and problem started only after applying the Patchset. Checked all the logs and Found nothing relevant.
Then i checked the things which added with this Patchset.
The Most interesting found , Oracle Added a New Daemon- oprocd.
# ps -efl | grep oprocd
4 S root 6440 6063 0 -40 - - 2114 - Mar03 ? 00:00:00 /opt/oracle/product/10.2.0/crs/bin/oprocd.bin run -t 1000 -m 500 -hsi 5:10:50:75:90 -f
These are Interesting Points about above line
1.This Process is running by root user
2. With Highest Priority -40
3. Probing every Seconds (t 1000)
4. waiting CPU response for 500 Milliseconds ( -m 500 means margin time is 500 Milli Seconds)
5. Process status is Fatal (-f)
Now I am concluding these points- This daemon will probe cpu every second and wait for response within 500 Mill seconds. If in the 500 Milli second not getting any response from the cpu, will assume the CPU is hang and try to Reboot the Machine. The OPERATING SYSTEM will not get enough time to write the system logs and server reboots.
So the solution is increase the Margin time for 500 Milli second to 10 seconds.
These are following steps to increase the Margin time.
Please Remember- The Modification process need Downtime and You need to stop cluster service in all member nodes.
1. Stop The CRS Process
#crsctl stop crs
#<CRS_HOME>/bin/oprocd stop
2. Ensure that Clusterware stack is down and not running
#ps -ef |egrep "crsd.bin|ocssd.bin|evmd.bin|oprocd"
This should return no processes.
3. From one node of the cluster, change the value of the "diagwait" parameter to 13 by issuing the command as root:
#crsctl set css diagwait 13 -force
4. Check if diagwait is successfully set.
#crsctl get css diagwait
5. Restart the Oracle Clusterware on all the nodes by executing:
#crsctl start crs
(Note- If facing any problem to restarting the CRS services, ASM and Database, You can reboot the Nodes.The Cluster and Database will come automatically due to init startup scripts.)
6. The oprocd daemon process will show with -m 10000
# ps -efl| grep oprocd
# 4 S root 6440 6063 0 -40 - - 2114 - Feb02 ? 00:00:00 /opt/oracle/product/10.2.0/crs/bin/oprocd.bin run -t 1000 -m 10000 -hsi 5:10:50:75:90 -f
Rollback Procedure-
If You need to unset oprocd value due any reason
#crsctl unset css diagwait
I am confident, The abnormal RAC Node restart problem will solve with this workaround.
Regards,
Sumit
Bangalore,India -
Is X-windows and GUI desktops supported on the ODA "engineered system" running a RAC database? If it is, what is the yum command needed to install the X-windows, Gnome, and KDE package groups?
While I agree with the direction of the suggestions with installing packages for X-windows, we do not have a blanket 'apply any package' recommendation.
In particular we do not support altering the kernel (although we do have exceptions which we review on a case by case basis).
Basically, if the you want to alter functionality that would not impact core functionality you are usually fine.
A good guideling is : The more dependencies that there are between the package / rpm you are considering using the higher the potential impact on functionality - meaning higher chance for problems
Note: We do use VNC including Real and Tiger regularly , but we have no hard recommendation on how you may want to use X-windows. I have never seen a limitation other than comments on bugs
or incompatibility within the X-window product itself with certain kernel levels.
Patching may overwrite some packages that you may install, however, _depending on packages/rpms added_ there is also the possibility that you will break existing functionality to the point
that patching itself will fail ( we have already seen a few cases of this in which case the proper mitigation is to remove / roll-back any alterations to the ODA before patching, and then adding the packages/rpms
back after the patching is completed.
From what you are discussing the impact should be low without conflicts, but please consider the above, and if you have specific packages which you consider potential problems
please create an SR so that we can review packages / rpms on an individual basis.
Once again: the main criteria for not supporting rpms is regarding the kernel itself
Chuck -
Listener target is offline on rac database
hi all,
os-linux
oracle 10.0.2
i am using two node rac (rac1,rac2).
i used rconfig for converting a non rac database to rac database.
Everything is working fine but when i fire command crs_stat
i am getting
NAME=ora.rac1.LISTENER_RCONFIG_RAC1.lsnr
TYPE=application
TARGET=OFFLINE
STATE=ONLINE on nrac1
NAME=ora.rac2.LISTENER_RCONFIG_RAC2.lsnr
TYPE=application
TARGET=OFFLINE
STATE=ONLINE on nrac2
please give me the solution.
Edited by: varun4dba on Nov 15, 2010 12:28 PMWhat is the output of:
crs_stat -v ora.rac1.LISTENER_RCONFIG_RAC1.lsnrTry to stop listeners and and to restart them with crs_start:
crs_start ora.rac1.LISTENER_RCONFIG_RAC1.lsnr -
Restoring Back To 10g After A Failed 11g Upgrade Of a RAC Database
I'm testing this out on a small test RAC database. I successfully upgraded it from 10.2.0.4 to 11.2.0.2 but wanted to test the scenario of having to go back to 10g if the upgrade really hosed up. The first recovery attempt seemed to be successful but after bringing the DB down with srvctl, it failed on the next startup saying it needed to be started in upgrade mode. Something from 11g was still in place or the fact that I was trying to restart a 10g database managed by 11g clusterware was the issue. I tried starting the DB from both 10g and 11g environments and got the same result. Even starting each instance individually got the same result.
In all that I tried, I got the usual incarnation and "until time before reset time" messages. I've been doing this all through RMAN without EM or Grid Ctl. As usual, any docs I found have had just a little information and I have to piece my own instructions together from all of them, not knowing for sure it all steps would apply in my situation.
Can anybody point me to a good doc or other resource that might help me out? Many thanks!Now I have a different issue with apparently the same problem. I successfully did the restore/recover as before but the thing pukes when I open resetlogs at the end. Something, somewhere is still pointing to 11g but I have no idea where or what has changed since the last time I did this. Maybe I've messed things up by doing this multiple times. Here are my RMAN commands:
RMAN> connect target
RMAN> startup force nomount;
RMAN> RESTORE SPFILE TO '+DATA/jimg/spfilejimg.ora' from '/local/oracle/10.2.0/db_1/dbs/c-2526333028-20110915-01';
RMAN> shutdown immediate;
(in a different session, from command line)
% mv /local/oracle/10.2.0/db_1/dbs/initJIMG1.ora.bak /local/oracle/10.2.0/db_1/dbs/initJIMG1.ora
RMAN> startup force nomount pfile='/local/oracle/10.2.0/db_1/dbs/initJIMG1.ora';
RMAN> restore controlfile from '/local/oracle/10.2.0/db_1/dbs/c-2526333028-20110915-01';
RMAN> alter database mount;
RMAN>
run {
restore database;
recover database;
RMAN> alter database open resetlogs;
The errors I get after resetlogs are:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of alter db command at 09/16/2011 09:24:50
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
ORA-03114: not connected to ORACLE
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of alter db command at 09/16/2011 09:24:50
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option
This is being done on a small, expendable database for practice. I think I've learned enough about what NOT to do to keep me from getting in this jam to begin with. Any thoughts? -
Metalink Document IDs for RAC database
Hello,
This is Khalid, Plz any body could give me metalink document id's for RAC database managing and monitoring, backup and recoveries etc.. plz any document id related to RAC databases.....
Thanks & Regards
KhalidClusterware References
Metalink Notes
259301.1 CRS and 10g RAC
This note contains a useful awk script to improve the output of crs_stat -ls
436067.1 Windows CRS_STAT script to display long names correctly
309541.1 How to start/stop the 10g CRS Clusterware
263897.1 How to stop Cluster Ready Services (CRS)
298073.1 How to remove CRS auto start and restart for a RAC instance
295871.1 How to verify if CRS install is valid
316583.1 VIPCA fails complaining that interface is not public
341214.1 How to cleanup after a failed (or successful) Oracle Clusterware installation
280589.1 How to install Oracle 10g CRS on a cluster where one or more nodes are not to be configured with CRS immediately
357808.1 CRS Diagnostics
272331.1 CRS 10g Diagnostic Guide
330358.1 CRS 10g R2 Diagnostic Collection Guide
331168.1 Oracle Clusterware consolidated logging in 10gR2
342590.1 CRS logs not being written
357808.1 Diagnosability for CRS/EVM/RACG
459694.1 Procwatcher: Script to Monitor and Examine Oracle and CRS Processes
289690.1 Data Gathering for Troubleshooting RAC and CRS issues
265769.1 Troubleshooting CRS Reboots
240001.1 Troubleshooting CRS root.sh problems (10g RAC)
239989.1 10g RAC - Stopping Reboot Loops when CRS problems occur
294430.1 CSS Timeout Computation in 10g RAC
284752.1 10gRAC: Steps to Increase CSS Misscount, Reboottime and Disktimeout
462616.1 Reconfiguring the CSS disktimeout of 10gR2 Clusterware for proper LUN failover
293819.1 Placement of voting and OCR disk file in 10g RAC
317628.1 How to replace a corrupt OCR mirror file
452486.1 Moving OCR and Voting Disk to another location
399482.1 How to recreate OCR/Voting disk accidentally deleted
358620.1 How to recreate OCR/Voting disk in 10gR1/R2 RAC
279793.1 How to Restore a Lost Voting Disk in 10g
264847.1 How to Configure Virtual IPs for 10g RAC
283684.1 How to change interconnect/public interface IP subnet in a 10g cluster
276434.1 Modifying the VIP or VIP Hostname of an Oracle 10g Clusterware Node
294336.1 Changing the check interval for the Oracle 10g VIP
219361.1 Troubleshooting Instance Evictions (ORA-29740)
297498.1 Resolving Instance Evictions on Windows platforms
315125.1 What to check if the Cluster Synchronization Services daemon (OCSSD) does not start
270512.1 Adding a node to a 10g RAC Cluster
269320.1 Removing a node from a 10g RAC Cluster
338706.1 Cluster Ready Services (CRS) rolling upgrade
399031.1 Step-by-step installation of Oracle Clusterware one-off and bundle patches for Oracle 10g
401783.1 Changes in Oracle Clusterware after applying 10.2.0.3 Patchset
405820.1 Known Issues After Applying 10.2 CRS bundle patches
316817.1 Cluster Verification Utility (CLUVFY) FAQ
372358.1 Shared disk check with the Cluster Verification Utility
338924.1 CLUVFY Fails with error - could not find a suitable set of interfaces for VIPs
Bugs
5849200 CRS LOGS ARE NOT BEING WRITTEN
5137401 OPROCD LOGFILE IS CLEARED AFTER A REBOOT
Fixed in Oracle 10.2.0.4+ and 11.1.0.6+
source: http://www.juliandyke.com/References/Clusterware.html
regards,
Rajeshkumar Govindarajan.
http://oracleinstance.blogspot.com -
Backup Exec 12.5 and Rac database
We always use rman to backup and restore our rac database without problems but now we would like to try with Backup Exec 12.5. Yesterday we installed the server for backup exec and the oracle agent for linux and by the way we did a full backup of our Rac database. The problems was when we tried to restore it. When we restore a single database it work but when we try with RAC it does not work.
Questions:
Is there a different agent for oracle Rac database or it is the same?
Are there aditional steps to configure the agent for Rac database?
I really appreciate your help.++We are using flash recovery area and all archive logs, flash back logs and backup sets go in there?The system administrator wants to back up the full directory of flash recovery area. Would that be enough?Is a backup of oracle home is needed?+
Normally there is no need to take oracle backup home , if yours FRA contain baskupset for database files (controlfile,db files),archivelogs ,flashback logs.You should also consider the pfile if you are not using spfile,spfile itself contain in RMAN backupsets.
Consider yours oracle binary backup (i.e cold backup) before any maintainance (patching activity).
Read http://oraware.blogspot.com/2008/07/fra-capacity-planning.html
Are there any backup settings required before backing up files while the database is on, like setting them in backup mode or etc?
yes for online database should be in archivelog mode,if you are using RMAN then there is no any setting required before backup , using RMAN you just have to connect with target database and start online backup using yours own backup script.
Any documentation discussing the backup steps, and what needs to be backed up will be appreciated
http://download.oracle.com/docs/cd/B19306_01/backup.102/b14192/bkup.htmlKhurram -
Is your RAC database working good??
Hi, all.
I have a 2 node RAC database 10.2.0.2.0 on 32bit windows 2003 EE SP1.
I am wondering if your RAC database is working good.
In my case, there are a number of gc related wait events such as gc buffer busy and gc cr request. Thus, I need to restart one node from time to time.( on average, one time per a week)
How about yours??
Thanks and Regards..In my case, there are a number of gc related wait events such as gc buffer busy and gc cr request. Thus, I need to restart one node from time to time.( on average, one time per a week)Seeing excessive/high gc related wait events usually indicates poor interconnect performance and that the interconnect may not have the required bandwidth and is the possible bottleneck. In other words, your data requests via the interconnect are much higher than what it could support without waits. The holding instance may not be able to make the requested block available immediately and thus the wait occurs.
Just curious, how did you decide that these gc waits are causing performance problems and that restarting the nodes would fix it? Are you noticing better performance after restarting the nodes?
Thanks
Chandra -
Hi all,
So the short of it is that my work iMac (bought new with 3 others), every so often, shuts down by itself and then restarts by itself as well with a message saying something along the lines of 'you have shut down your computer because of a problem'. It's like someone pulled the plug, no warning no nothing, just goes black and restarts. This happens as an alternative for then it crashes/freezes with the screen on and i have to force shutdown via power button.
I haven't been able to pin point reasons because there doesn't seem to be any particular trigger. Maybe when I'm using Adobe Creative Suite applications, but then again as a graphic designer, thats what Im always using, and Chrome (with, admitedly, more tabs open than average).
The tech guy has taken it in and supposedly ran some tests but came back with nothing to report, nothing happened with him. His latest theory is maybe a problem with the power supply, like a peak surge or something... I don't know, haven't tested with a surge protector yet. But done the usual permissions repair and disk repair etc and still nothing.
The other 2 iMacs, same specs, have never had these problems.
Searched online and haven't found any solutions and reports to my particular case. So I've decided to turn to this community for any help.
Below are the specs and an example crash report.
Thanks in advance for any help.
Lou
Late 2012 iMac
Processor 2,7 GHz Intel Core i5
Memory 16 GB 1600 MHz DDR3
Graphics NVIDIA GeForce GT 640M 512 MB
Software OS X 10.8.5 (12F45)
Latest crash report ///////////////
Mon Jun 23 16:25:16 2014
panic(cpu 0 caller 0xffffff7f899da8a1): NVRM[0/1:0:0]: Read Error 0x00000144: CFG 0x0fd810de 0x00100406 0xb0000000, BAR0 0x103000000 0xffffff81db65d000 0x0e7180a2, D0, P1/4
Backtrace (CPU 0), Frame : Return Address
0xffffff81c9f0ccc0 : 0xffffff800921d636
0xffffff81c9f0cd30 : 0xffffff7f899da8a1
0xffffff81c9f0cdf0 : 0xffffff7f89aad85a
0xffffff81c9f0ce30 : 0xffffff7f89dc37b7
0xffffff81c9f0ce50 : 0xffffff7f899e1576
0xffffff81c9f0cef0 : 0xffffff7f8997f3ce
0xffffff81c9f0cf10 : 0xffffff8009653298
0xffffff81c9f0cf40 : 0xffffff7f8986828b
0xffffff81c9f0cf50 : 0xffffff7f8b55b533
0xffffff81c9f0cf60 : 0xffffff7f8b56320b
0xffffff81c9f0cf80 : 0xffffff80092b7f7c
0xffffff81c9f0cfd0 : 0xffffff80092cedbb
0xffffff81dadf3890 : 0xffffff7f89b7488b
0xffffff81dadf38b0 : 0xffffff7f89b855ab
0xffffff81dadf3a40 : 0xffffff7f8997b6f7
0xffffff81dadf3ac0 : 0xffffff7f8996759b
0xffffff81dadf3b30 : 0xffffff7f899679fd
0xffffff81dadf3b70 : 0xffffff7f8992da4d
0xffffff81dadf3bc0 : 0xffffff8009670b13
0xffffff81dadf3c20 : 0xffffff800966e74f
0xffffff81dadf3d70 : 0xffffff8009298c21
0xffffff81dadf3e80 : 0xffffff8009220b4d
0xffffff81dadf3eb0 : 0xffffff8009210448
0xffffff81dadf3f00 : 0xffffff800921961b
0xffffff81dadf3f70 : 0xffffff80092a6546
0xffffff81dadf3fb0 : 0xffffff80092cf473
Kernel Extensions in backtrace:
com.apple.iokit.IOPCIFamily(2.8)[2FAEA49C-EA4C-39C6-9203-FC022277A43C]@0xffffff 7f89851000->0xffffff7f89879fff
com.apple.driver.AppleACPIPlatform(1.8)[209B2382-A61F-344C-8BBC-26331B9BA398]@0 xffffff7f8b554000->0xffffff7f8b5adfff
dependency: com.apple.iokit.IOACPIFamily(1.4)[A35915E8-C1B0-3C0F-81DF-5515BC9002FC]@0xfffff f7f8a363000
dependency: com.apple.iokit.IOPCIFamily(2.8)[2FAEA49C-EA4C-39C6-9203-FC022277A43C]@0xffffff 7f89851000
com.apple.iokit.IOGraphicsFamily(2.3.7)[9928306E-3508-3DBC-80A4-D8F1D87650D7]@0 xffffff7f89922000->0xffffff7f89959fff
dependency: com.apple.iokit.IOPCIFamily(2.8)[2FAEA49C-EA4C-39C6-9203-FC022277A43C]@0xffffff 7f89851000
com.apple.iokit.IONDRVSupport(2.3.7)[F16E015E-1ABE-3C40-AC71-BC54F4BE442E]@0xff ffff7f89965000->0xffffff7f89976fff
dependency: com.apple.iokit.IOGraphicsFamily(2.3.7)[9928306E-3508-3DBC-80A4-D8F1D87650D7]@0 xffffff7f89922000
dependency: com.apple.iokit.IOPCIFamily(2.8)[2FAEA49C-EA4C-39C6-9203-FC022277A43C]@0xffffff 7f89851000
com.apple.NVDAResman(8.1.6)[39D35403-42FB-3F08-999C-9866938D6B3A]@0xffffff7f899 79000->0xffffff7f89c1cfff
dependency: com.apple.iokit.IOPCIFamily(2.8)[2FAEA49C-EA4C-39C6-9203-FC022277A43C]@0xffffff 7f89851000
dependency: com.apple.iokit.IONDRVSupport(2.3.7)[F16E015E-1ABE-3C40-AC71-BC54F4BE442E]@0xff ffff7f89965000
dependency: com.apple.iokit.IOGraphicsFamily(2.3.7)[9928306E-3508-3DBC-80A4-D8F1D87650D7]@0 xffffff7f89922000
com.apple.nvidia.gk100hal(8.1.6)[347201ED-EC77-3189-B256-B3403CFCBB06]@0xffffff 7f89c28000->0xffffff7f89f59fff
dependency: com.apple.NVDAResman(8.1.6)[39D35403-42FB-3F08-999C-9866938D6B3A]@0xffffff7f899 79000
dependency: com.apple.iokit.IOPCIFamily(2.8)[2FAEA49C-EA4C-39C6-9203-FC022277A43C]@0xffffff 7f89851000
BSD process name corresponding to current thread: WindowServer
Mac OS version:
12F45
Kernel version:
Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64
Kernel UUID: EA38B02E-2B88-309F-BA68-1DE29F605DD8
Kernel slide: 0x0000000009000000
Kernel text base: 0xffffff8009200000
System model name: iMac13,1 (Mac-00BE6ED71E35EB86)
System uptime in nanoseconds: 24278497409941
last loaded kext at 23378975154171: com.apple.filesystems.cddafs 2.5.1 (addr 0xffffff7f8b71d000, size 24576)
last unloaded kext at 23460702038780: com.apple.filesystems.cddafs 2.5.1 (addr 0xffffff7f8b71d000, size 20480)
loaded kexts:
com.apple.iokit.SCSITaskUserClient 3.5.6
com.apple.filesystems.smbfs 1.8.4
com.apple.filesystems.afpfs 10.0
com.apple.nke.asp_tcp 7.1.0
com.apple.driver.AppleBluetoothMultitouch 75.19
com.apple.driver.AppleHWSensor 1.9.5d0
com.apple.iokit.IOBluetoothSerialManager 4.1.7f2
com.apple.driver.AudioAUUC 1.60
com.apple.driver.AppleMikeyHIDDriver 124
com.apple.driver.ApplePlatformEnabler 2.0.7d2
com.apple.driver.AGPM 100.13.12
com.apple.driver.X86PlatformShim 1.0.0
com.apple.filesystems.autofs 3.0
com.apple.driver.AppleHDA 2.4.7fc4
com.apple.driver.AppleMikeyDriver 2.4.7fc4
com.apple.GeForce 8.1.6
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport 4.1.7f4
com.apple.driver.AppleUpstreamUserClient 3.5.12
com.apple.driver.AppleSMBusPCI 1.0.11d1
com.apple.driver.AppleLPC 1.6.3
com.apple.iokit.IOUserEthernet 1.0.0d1
com.apple.Dont_Steal_Mac_OS_X 7.0.0
com.apple.driver.ApplePolicyControl 3.4.5
com.apple.driver.AppleMCCSControl 1.1.11
com.apple.driver.AppleSMCLMU 2.0.3d0
com.apple.driver.AppleIntelHD4000Graphics 8.1.6
com.apple.driver.AppleIntelFramebufferCapri 8.1.6
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 34
com.apple.driver.XsanFilter 404
com.apple.iokit.IOAHCIBlockStorage 2.3.5
com.apple.driver.AirPort.Brcm4331 615.20.17
com.apple.driver.AppleAHCIPort 2.6.6
com.apple.driver.AppleSDXC 1.4.3
com.apple.iokit.AppleBCM5701Ethernet 3.6.2b4
com.apple.driver.AppleUSBHub 635.4.0
com.apple.driver.AppleUSBEHCI 621.4.6
com.apple.driver.AppleUSBXHCI 635.4.0
com.apple.driver.AppleACPIButtons 1.8
com.apple.driver.AppleRTC 1.5
com.apple.driver.AppleHPET 1.8
com.apple.driver.AppleSMBIOS 1.9
com.apple.driver.AppleACPIEC 1.8
com.apple.driver.AppleAPIC 1.7
com.apple.driver.AppleIntelCPUPowerManagementClient 214.0.0
com.apple.nke.applicationfirewall 4.0.39
com.apple.security.quarantine 2.1
com.apple.driver.AppleIntelCPUPowerManagement 214.0.0
com.apple.iokit.IOSCSIMultimediaCommandsDevice 3.5.6
com.apple.iokit.IOBDStorageFamily 1.7
com.apple.iokit.IODVDStorageFamily 1.7.1
com.apple.iokit.IOCDStorageFamily 1.7.1
com.apple.iokit.IOUSBMassStorageClass 3.5.2
com.apple.security.SecureRemotePassword 1.0
com.apple.driver.AppleMultitouchDriver 237.4
com.apple.driver.AppleBluetoothHIDKeyboard 170.2.4
com.apple.driver.IOBluetoothHIDDriver 4.1.7f2
com.apple.driver.AppleHIDKeyboard 170.2.4
com.apple.driver.AppleHIDMouse 175.8
com.apple.iokit.IOSerialFamily 10.0.6
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 2.4.7fc4
com.apple.iokit.IOAudioFamily 1.9.2fc7
com.apple.kext.OSvKernDSPLib 1.12
com.apple.iokit.IOBluetoothHostControllerUSBTransport 4.1.7f2
com.apple.nvidia.gk100hal 8.1.6
com.apple.NVDAResman 8.1.6
com.apple.driver.X86PlatformPlugin 1.0.0
com.apple.driver.IOPlatformPluginFamily 5.4.1d13
com.apple.iokit.IOSurface 86.0.4
com.apple.iokit.IOBluetoothFamily 4.1.7f2
com.apple.driver.AppleGraphicsControl 3.4.5
com.apple.driver.AppleThunderboltEDMSink 1.2.0
com.apple.driver.AppleSMBusController 1.0.11d1
com.apple.driver.AppleSMC 3.1.5d4
com.apple.driver.AppleHDAController 2.4.7fc4
com.apple.iokit.IOHDAFamily 2.4.7fc4
com.apple.iokit.IOAcceleratorFamily 74.15
com.apple.iokit.IONDRVSupport 2.3.7
com.apple.iokit.IOGraphicsFamily 2.3.7
com.apple.iokit.IOSCSIArchitectureModelFamily 3.5.6
com.apple.driver.AppleThunderboltDPOutAdapter 2.5.0
com.apple.driver.AppleThunderboltDPInAdapter 2.5.0
com.apple.driver.AppleThunderboltDPAdapterFamily 2.5.0
com.apple.driver.AppleThunderboltPCIDownAdapter 1.3.2
com.apple.iokit.IOUSBHIDDriver 623.4.0
com.apple.driver.AppleUSBMergeNub 621.4.6
com.apple.driver.AppleUSBComposite 621.4.0
com.apple.iokit.IO80211Family 530.5
com.apple.iokit.IOAHCIFamily 2.5.1
com.apple.iokit.IOEthernetAVBController 1.0.2b1
com.apple.iokit.IONetworkingFamily 3.0
com.apple.driver.AppleThunderboltNHI 1.9.2
com.apple.iokit.IOThunderboltFamily 2.7.7
com.apple.iokit.IOUSBUserClient 630.4.4
com.apple.iokit.IOUSBFamily 635.4.0
com.apple.driver.AppleEFINVRAM 2.0
com.apple.driver.AppleEFIRuntime 2.0
com.apple.iokit.IOHIDFamily 1.8.1
com.apple.iokit.IOSMBusFamily 1.1
com.apple.security.sandbox 220.3
com.apple.kext.AppleMatch 1.0.0d1
com.apple.security.TMSafetyNet 7
com.apple.driver.DiskImages 345
com.apple.iokit.IOStorageFamily 1.8
com.apple.driver.AppleKeyStore 28.21
com.apple.driver.AppleACPIPlatform 1.8
com.apple.iokit.IOPCIFamily 2.8
com.apple.iokit.IOACPIFamily 1.4
com.apple.kec.corecrypto 1.0The graphics processor is faulty and will have to be replaced.
Make a "Genius" appointment at an Apple Store, or go to another authorized service provider. You may have to leave the machine there for several days.
Print the first page of the panic report and bring it with you.
Back up all data on the internal drive(s) before you hand over your computer to anyone. There are ways to back up a computer that isn't fully functional—ask if you need guidance.
If privacy is a concern, erase the data partition(s) with the option to write zeros* (do this only if you have at least two complete, independent backups, and you know how to restore to an empty drive from any of them.) Don’t erase the recovery partition, if present.
Keeping your confidential data secure during hardware repair
Apple also recommends that you deauthorize a device in the iTunes Store before having it serviced.
*An SSD doesn't need to be zeroed. -
Unable to startup 12c RAC database, can't open spfile in ASM
hello,
I'm testing 12cRAC database on RHEL5 and need your help to troubleshoot and fix the following issue -
DBCA fails at the end of the configuration to create and startup new RAC database with the following errors
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/TDB1/spfileTDB1.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/TDB1/spfileTDB1.ora
ORA-15056: additional error message
ORA-17503: ksfdopn:2 Failed to open file +DATA/TDB1/spfiletdb1.ora
ORA-01017: invalid username/password; logon denied
I also tried executing script (which was created by DBCA) to create the new DB manually and got the same results - it happens during first attempt to start it up using SPFILE after new DB was successfully created (it was started up using pfile)
Clusterware with ASM install was successful, binaries were also installed without issues,
cluster seems healthy, I see correct files within ASM using asmcmd etc ..
got stuck here and need some directions ... very confused by ORA-01017 password error
wonder if anyone had same or similar issues ?
Thank you !Fixed.
Thank you everyone for suggestions and recommendations -
ASM is accessible using asmcmd and I can see all the DB files incuding spfile, they were created by DBCA, alert log showed same information I posted
After extensive troubleshooting and testing the issue was idedntified and fixed - 'dba' group needs to be the primary group for 'oracle' user, not the secondary one. I suspected that something was not right with 'oracle' user access to asm based on that strange password error. There is more I need to check as the issue and fix with swapping groups seems strange. Secondary group is not enough ti have proper rights ??? my first reaction was - seriously ? wtf ?
Anyway - I need to move one now ...
Thank you again !
Maybe you are looking for
-
Why does Numbers say I need a newer version when I am using the latest version
I am getting the error message "You need a newer version of Numbers to open this file". However, this is a file I created yesterday with this version of numbers. What gives ? If I ck my numbers version it says no updates available.
-
Dear All, There is a requirement to add the field "Reference" from the related FI Document in the output generated by KOB1 Report; This field is not among the Hidden fields; Practical suggestions on how this can be achieved will be very well apprecia
-
Hi Please provide me with pointers on how to develop async webservice in JEE5.. Please note that I am interested in building the service and NOT the client.. Thanks Senthil
-
Why error message she trying to install AirPort software 6.0
Why do I get an error message she trying to install AirPort software 6.0 over 5.5.3.
-
Mountain Lion Install Issue - Bootcam partition cant boot up!
After installing Mountain Lion OS, I experienced startup problem with my Bootcam partition. I saw blue screen after rebooting the Mac with error message " process1_initialization_failed " after installation complete. What can I do to recover my Bootc