ASM - arch logs

I am using asm for the arch log storage in a two node cluster. I have set parameter log_archive_format to '%t_%s_%r.arc'. But the logs are not getting generated according to the format. Why is that?
Thanks.

Do you use ASM?
log_archive_format....
LOG_ARCHIVE_FORMAT - If you set the LOG_ARCHIVE_FORMAT to an incomplete ASM filename (such as +dgroupA), Oracle will ignore it.

Similar Messages

Arch log format in asm

I am on single node asm with 10gr3. I changed the arch log format to %t_%s_%r.arc but the files are still getting created as thread_1_seq_90.459.653928775
I have bounced the db already.
SQL> show parameter archive_fo
NAME TYPE VALUE
log_archive_format string %t_%s_%r.arc
SQL>
What else to be setup in order to correct the format?
Thanks.

I believe what you are observing is Alias created by Oracle.
Check logs available at your destination defined by "log_archive_dest_1"
e.g In my case it is +DATA/test
ASMCMD>cd DATA/test
ASMCMD> ls
1_7_653875700.dbf
2_6_653875700.dbf
Now I switch to directory storing alias
ASMCMD>cd ARCHIVELOG
ASMCMD>cd 2008_05_06
ASMCMD>ls
thread_1_seq_7.425.654030021
thread_2_seq_6.363.654040811
Is it clear?
-Amit
http://askoracledba.blogspot.com/

Why don't we keep Oracle Database ARCH logs in RAW devices

Hi All,
Why don't we keep Oracle Database ARCH logs in RAW devices?
How ARCH process is related to Block Device and Character Device ?
Thanks,
Tusar
Edited by: user11987718 on 26-Jun-2012 09:59

Why don't we keep Oracle Database ARCH logs in RAW devices? 1.This is because raw devices do not enable sequential writing of consecutive archive log files.
http://docs.oracle.com/cd/E11882_01/rac.112/e16795/rman.htm#RACAD320
2.with raw you have to specify the exact size of the file at creation time, in contrast with file system, where you can create file without specifying its size, start writing to it, apending new blocks to it until you are done, and thus creating a file of whatever size it needs to be to hold all information.
http://www.dbasupport.com/forums/archive/index.php/t-16429.html
3.raw file systems are difficult to manage because you cannot run normal os commands on them.(as there is no file systems)
How ARCH process is related to Block Device and Character Device ?A block device would read/write bytes in fixed size blocks, as in disk sectors. Character devices read/write 0 or more bytes, in a stream, such as a TTY or a keyboard.
For more details about what is block and character device type :
http://en.wikipedia.org/wiki/Device_file
And now think, how arch process is related to block and character devices.
Regards
Girish Sharma

Arch logging errors (/var/logs)

i got two problems i'd like to solve maybe their cause is the same.
first: is global arch logging. that's my ls -l of /var/log/
-rw-r--r-- 1 root root 0 Apr 21 11:27 acpid.log
-rw-r--r-- 1 root root 0 Apr 7 22:13 acpid.log.1
-rw-r--r-- 1 root root 0 Apr 21 11:27 auth.log
-rw-r--r-- 1 root root 0 Apr 7 22:14 auth.log.1
-rw-r--r-- 1 root root 1092 Nov 7 02:52 boot
-rw------- 1 root root 1152 Mar 17 08:06 btmp
drwxr-xr-x 2 root root 72 Nov 7 16:56 ConsoleKit
-rw-r--r-- 1 root root 0 Apr 7 22:14 crond
-rw-r--r-- 1 root root 0 Apr 21 11:27 daemon.log
-rw-r--r-- 1 root root 0 Apr 7 22:18 daemon.log.1
-rw-r--r-- 1 root root 39494 Apr 22 13:18 dmesg.log
-rw-r--r-- 1 root root 0 Apr 21 11:27 errors.log
-rw-r--r-- 1 root root 0 Apr 7 22:18 errors.log.1
-rw-r--r-- 1 root root 0 Apr 21 11:27 everything.log
-rw-r--r-- 1 root root 0 Apr 7 22:18 everything.log.1
-rw------- 1 root root 32032 Apr 22 12:03 faillog
drwxrwx--T 2 root gdm 1392 Apr 22 13:18 gdm
drwxr-xr-x 2 root root 368 Apr 21 11:27 httpd
-rw-r--r-- 1 root root 0 Apr 21 11:27 kernel.log
-rw-r--r-- 1 root root 0 Apr 7 22:19 kernel.log.1
-rw-r--r-- 1 root root 292292 Apr 22 12:03 lastlog
-rw-r--r-- 1 root root 0 Apr 21 11:27 messages.log
-rw-r--r-- 1 root root 0 Apr 7 22:19 messages.log.1
drwxr-xr-x 2 root root 48 Feb 26 06:56 old
-rw-r--r-- 1 root root 151881 Apr 22 13:42 pacman.log
-rw-r--r-- 1 root root 86 Apr 22 13:18 pm-powersave.log
-rw-r--r-- 1 root root 6049 Apr 20 10:21 pm-suspend.log
-rw-r--r-- 1 root root 0 Apr 21 11:27 syslog.log
-rw-r--r-- 1 root root 0 Apr 7 22:19 syslog.log.1
-rw-r--r-- 1 root root 0 Apr 21 11:27 user.log
-rw-r--r-- 1 root root 0 Apr 7 22:19 user.log.1
-rw-rw-r-- 1 root root 407424 Apr 22 13:29 wtmp
-rw-rw-r-- 1 root root 415488 Apr 7 18:36 wtmp.1
-rw-r--r-- 1 root root 13947 Apr 22 13:18 Xorg.0.log
-rw-r--r-- 1 root root 14486 Apr 22 13:17 Xorg.0.log.old
all files, that have dublicate name with .1 at the end, are always empty and as time goes by files with the same name but with .2, .3, .4 appear but also empty. It looks like arch is trying to create logs but something goes wrong and nothing is logged.
second: i got errors from cron if i do run-cron /etc/cron.daily/ related to log files:
sudo run-cron /etc/cron.daily/
error: syslog-ng:1 duplicate log entry for /var/log/crond.log
error: found error in /var/log/messages.log /var/log/auth.log /var/log/mail.log /var/log/kernel.log /var/log/errors.log /var/log/daemon.log /var/log/user.log /var/log/iptables.log /var/log/everything.log /var/log/syslog.log /var/log/acpid.log /var/log/crond.log /var/log/lpr.log /var/log/uucp.log /var/log/news.log /var/log/ppp.log /var/log/debug.log , skipping
looks like the files r corrupted or something. please help find out the reasons and fix things up.

I think it has to do with the group log having write permissions...
# ls -l /var/log/
total 8240
drwxr-xr-x 2 root root 4096 Mar 31 01:38 ConsoleKit/
drwxr-xr-x 2 root root 4096 Apr 4 03:52 httpd/
drwxr-xr-x 2 http http 4096 Apr 18 03:45 lighttpd/
drwxr-xr-x 2 root root 4096 Feb 25 23:56 old/
-rw-r----- 1 root log 29283 Apr 22 11:05 auth.log
-rw-r----- 1 root log 71395 Apr 18 03:26 auth.log.1
-rw-r----- 1 root log 62225 Apr 11 03:17 auth.log.2
-rw-r----- 1 root log 121945 Apr 4 03:41 auth.log.3
-rw------- 1 root root 0 Mar 29 02:00 btmp
-rw-r----- 1 root log 6622 Apr 22 09:17 crond.log
-rw-r----- 1 root log 10756 Apr 18 03:45 crond.log.1
-rw-r----- 1 root log 10340 Apr 11 03:45 crond.log.2
-rw-r----- 1 root log 8809 Apr 4 03:45 crond.log.3
-rw-r----- 1 root log 63451 Apr 22 11:01 daemon.log
-rw-r----- 1 root log 101060 Apr 18 03:42 daemon.log.1
-rw-r----- 1 root log 103755 Apr 11 03:37 daemon.log.2
-rw-r----- 1 root log 72875 Apr 4 03:36 daemon.log.3
-rw-r--r-- 1 root root 32631 Apr 19 03:27 dmesg.log
-rw-r----- 1 root log 2881 Apr 21 23:22 errors.log
-rw-r----- 1 root log 162980 Apr 17 09:55 errors.log.1
-rw-r----- 1 root log 30491 Apr 9 21:54 errors.log.2
-rw-r----- 1 root log 31213 Apr 3 18:38 errors.log.3
-rw-r----- 1 root log 276636 Apr 22 11:01 everything.log
-rw-r----- 1 root log 1196453 Apr 18 03:45 everything.log.1
-rw-r----- 1 root log 374844 Apr 11 03:45 everything.log.2
-rw-r----- 1 root log 610588 Apr 4 03:45 everything.log.3
-rw------- 1 root root 24144 Apr 22 10:25 faillog
-rw-r----- 1 root log 185643 Apr 22 10:25 kernel.log
-rw-r----- 1 root log 1051728 Apr 18 02:02 kernel.log.1
-rw-r----- 1 root log 250567 Apr 11 03:27 kernel.log.2
-rw-r----- 1 root log 521352 Apr 3 18:38 kernel.log.3
-rw-r--r-- 1 root root 293752 Apr 22 10:25 lastlog
-rw-r----- 1 root log 229278 Apr 22 11:01 messages.log
-rw-r----- 1 root log 959109 Apr 18 03:42 messages.log.1
-rw-r----- 1 root log 309842 Apr 11 03:37 messages.log.2
-rw-r----- 1 root log 489801 Apr 4 03:36 messages.log.3
-rw-r--r-- 1 root root 77695 Apr 22 10:42 pacman.log
-rw-r--r-- 1 root root 86 Apr 19 13:23 pm-powersave.log
-rw-r----- 1 root log 825 Apr 19 03:27 syslog.log
-rw-r----- 1 root log 1387 Apr 15 15:39 syslog.log.1
-rw-r----- 1 root log 738 Apr 9 11:36 syslog.log.2
-rw-r----- 1 root log 1600 Mar 31 00:02 syslog.log.3
-rw-r----- 1 root log 16287 Apr 22 10:15 user.log
-rw-r----- 1 root log 24546 Apr 17 03:45 user.log.1
-rw-r----- 1 root log 626 Apr 11 03:27 user.log.2
-rw-r----- 1 root log 416 Mar 30 04:12 user.log.3
-rw-r----- 1 root log 6622 Apr 22 09:17 uucp.log
-rw-r----- 1 root log 10756 Apr 18 03:45 uucp.log.1
-rw-r----- 1 root log 10340 Apr 11 03:45 uucp.log.2
-rw-r----- 1 root log 8809 Apr 4 03:45 uucp.log.3
-rw-rw-r-- 1 root root 471552 Apr 22 10:25 wtmp
-rw-r--r-- 1 root root 150528 Apr 1 03:21 wtmp.1
-rw-r--r-- 1 root wheel 16308 Apr 22 10:25 Xorg.0.log
-rw-r--r-- 1 root users 23624 Apr 22 10:25 Xorg.0.log.old
Did you chown -R root:root /var/log on Apr 7?

Suggestion for default arch logging behavior

Okay let me throw this out there and see what comes back...
Every Linux distribution I have used (apart from arch) has a boot log file enabled by default, or provides a simple Yes/No flag to turn the feature on.
Arch does not. Moreover, I have searched the forums and posted a question or two myself about the topic. No one seems to know the answer, and a lot of responses seem to indicate that, amongst the arch community, the possibility of a boot log file is treated like: a) something that no one in their right mind would want, or b) something that maybe, might be useful, but that no one can figure out how to implement.
What is this? What is going on? Arch has a bootlogd binary in /sbin/. Presumably there is no reason why it won't actually work, but no one seems to know where to put the call to the binary, or why they would put it there.
This seems like an easy-to-add feature that a developer or moderator (anyone who really knows the system layout well) could make the default for future installs. It just makes a text file that is a) not large, and b) can be very useful at times.
So how about it? Why not make this part of the default arch logging set up? (and of course explain what you did that worked.)

tomk:
Sorry to disclose my ignorance, but I'm not sure how to do that.
What I have been doing though is trying to see in more detail how it is done on a debian system which I have access to. Here is what I've learned so far:
There are not that many references to bootlogd on the system, so it might be possible (for me) to track down what is going on:
root@wave32p:/etc# locate bootlogd
/etc/default/bootlogd
/etc/init.d/bootlogd
/etc/init.d/stop-bootlogd
/etc/init.d/stop-bootlogd-single
/sbin/bootlogd
/usr/share/man/man8/bootlogd.8.gz
1) "/etc/default/bootlogd" must be edited (trivially) such that a "No" becomes a "Yes" -- this seems like just a master switch.
2) the man files are exactly the same on the two systems, and the output of "/sbin/bootlogd -v" is the same on the two systems, however the size of the bootlogd binary itself is not the same on both systems (larger on debian64 system). not sure what to make of that, but it is not what I was hoping to see.
3) the script "/etc/init.d/bootlogd" runs with (eg.) a "start/stop" flag, the same as most "functions" under arch that have scripts associated with them.
4) it would seem that I have to grind my way through the above script if I'm going to make any progress. I'm doing that in my spare time at the moment, though it's a challenge since it's been a few years since I've written bash scripts on a regular basis. FYI, here is the /etc/init.d/bootlogd script verbatim (additional note: the option -r (below) is supported on the arch version of the bootlogd binary, but the -c option does not seem to be...interesting? Here is the man page entry for -c: "Attempt to write to the logfile even if it does not yet exist. Without this option, bootlogd will wait for the logfile to appear before attempting to write to it. This behavior prevents
bootlogd from creating logfiles under mount points."):
#! /bin/sh
### BEGIN INIT INFO
# Provides: bootlogd
# Required-Start: mountdevsubfs
# X-Start-Before: hostname keymap keyboard-setup procps pcmcia hwclock hwclockfirst hdparm hibernate-clean
# Required-Stop:
# Default-Start: S
# Default-Stop:
# Short-Description: Start or stop bootlogd.
# Description: Starts or stops the bootlogd log program
# which logs boot messages.
### END INIT INFO
PATH=/sbin:/bin # No remote fs at start
DAEMON=/sbin/bootlogd
[ -x "$DAEMON" ] || exit 0
NAME=bootlogd
DESC="boot logger"
BOOTLOGD_OPTS="-r -c"
[ -r /etc/default/bootlogd ] && . /etc/default/bootlogd
. /lib/init/vars.sh
. /lib/lsb/init-functions
# Because bootlogd is broken on some systems, we take the special measure
# of requiring it to be enabled by setting an environment variable.
case "$BOOTLOGD_ENABLE" in
[Nn]*)
exit 0
esac
# Previously this script was symlinked as "stop-bootlogd" which, when run
# with the "start" argument, should stop bootlogd. Now stop-bootlogd is
# a distinct script, but for backward compatibility this script continues
# to implement the old behavior.
SCRIPTNAME=${0##*/}
SCRIPTNAME=${SCRIPTNAME#[SK]??}
ACTION="$1"
case "$0" in
*stop-bootlog*)
[ "$ACTION" = start ] && ACTION=stop
esac
case "$ACTION" in
start)
# PATH is set above
log_daemon_msg "Starting $DESC" "$NAME"
if [ -d /proc/1/. ]
then
umask 027
start-stop-daemon --start --quiet --exec $DAEMON -- \
$BOOTLOGD_OPTS
ES=$?
else
$DAEMON $BOOTLOGD_OPTS
ES=$?
fi
log_end_msg $ES
stop)
PATH=/bin:/sbin:/usr/bin:/usr/sbin
log_daemon_msg "Stopping $DESC" "$NAME"
start-stop-daemon --oknodo --stop --quiet --exec $DAEMON
ES=$?
sleep 1
log_end_msg $ES
if [ -f /var/log/boot ] && [ -f /var/log/boot~ ]
then
[ "$VERBOSE" = no ] || log_action_begin_msg "Moving boot log file"
# bootlogd writes to boot, making backup at boot~
cd /var/log && {
chgrp adm boot || :
savelog -q -p -c 5 boot \
&& mv boot.0 boot \
&& mv boot~ boot.0
ES=$?
[ "$VERBOSE" = no ] || log_action_end_msg $ES
fi
restart|force-reload)
/etc/init.d/bootlogd stop
/etc/init.d/bootlogd start
status)
status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload|status}" >&2
exit 3
esac

Arch log problem

Dear all,
There is not much activity in the database. but archive logs are filling the disk.. How can I resolve this issue ?
Kai

Aren't we jumping to conclusions here without having cross-checked if it is the statistics gathering job only which is resulting in large number of archive files?
Its definitely not recommended to disable this job. If you want to disable this, then you need to come up with your own job to gather statistics at regular intervals.
You can change the schedule for this job to run at off-peak times though.

How to apply missing arching logs in a logical stand by database

hi,
I have created logical stand by data base for our production.It was working fine.We have not set the value for standby_archvie_dest and it send the archive files to $ORACLE_HOME/dbs, due to high volume transavtion many files generated and $ORACLE_HOME mount got filled and logical stan by apply stopped working and Db went down as well.
I tried to apply the files once i bought back the instance but after applying one archivie file it stopped applying further.
and logical stand by not working appropriately.
Please let me know that is there is a mechanism to apply the missing logs?
DB version : 10.2.0.5
OS :OEL 5
regards
Manoj

Hi,
Since then the issue happened I have noticed archives are not shipping.
Following are the outputs
SQL> ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-YY HH24:MI:SS';
Session altered.
SQL> COLUMN STATUS FORMAT A60
SQL> SELECT EVENT_TIME, STATUS, EVENT FROM DBA_LOGSTDBY_EVENTS ORDER BY EVENT_TIME, COMMIT_SCN;
EVENT_TIME STATUS
EVENT
18-MAR-12 11:11:35 ORA-16111: log mining and apply setting up
18-MAR-12 22:34:26 ORA-16226: DDL skipped due to lack of support
alter database begin backup
18-MAR-12 22:34:26 ORA-16226: DDL skipped due to lack of support
alter database end backup
EVENT_TIME STATUS
EVENT
18-MAR-12 22:49:25 ORA-16226: DDL skipped due to lack of support
alter database backup controlfile to '/tmp/PCEGYK_control.ctl'
18-MAR-12 22:49:25 ORA-16226: DDL skipped due to lack of support
alter database backup controlfile to trace
18-MAR-12 22:49:25 ORA-16226: DDL skipped due to lack of support
create pfile='/pcegyk/backup/hot_backups/18032012_2234/initPCEGYK.ora_from_spfil
EVENT_TIME STATUS
EVENT
19-MAR-12 00:04:40 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218894 to "SYSADM"
19-MAR-12 00:04:41 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218895 to "SYSADM"
19-MAR-12 00:04:41 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218896 to "SYSADM"
EVENT_TIME STATUS
EVENT
19-MAR-12 00:04:41 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218897 to "SYSADM"
19-MAR-12 00:04:41 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218898 to "SYSADM"
19-MAR-12 00:04:41 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218899 to "SYSADM"
EVENT_TIME STATUS
EVENT
19-MAR-12 00:19:26 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218900 to "SYSADM"
19-MAR-12 00:19:26 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218901 to "SYSADM"
19-MAR-12 00:19:26 ORA-16227: DDL skipped due to missing object
grant select,insert on sys.ora_temp_1_ds_218902 to "SYSADM"
EVENT_TIME STATUS
EVENT
20-MAR-12 03:28:09 ORA-16111: log mining and apply setting up
20-MAR-12 03:31:54 ORA-16128: User initiated stop apply successfully completed
20-MAR-12 03:55:13 ORA-16111: log mining and apply setting up
EVENT_TIME STATUS
EVENT
20-MAR-12 04:17:38 ORA-16128: User initiated stop apply successfully completed
20-MAR-12 04:17:54 ORA-16111: log mining and apply setting up
20-MAR-12 21:20:20 ORA-16111: log mining and apply setting up
21 rows selected.
SQL>
===========================
SQL> SELECT FILE_NAME, SEQUENCE#, FIRST_CHANGE#, NEXT_CHANGE#,TIMESTAMP, DICT_BEGIN, DICT_END, THREAD# FROM DBA_LOGSTDBY_LOG ORDER BY SEQUENCE#;
FILE_NAME
SEQUENCE# FIRST_CHANGE# NEXT_CHANGE# TIMESTAMP DIC DIC THREAD#
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138743_679263487.arc
138743 7.4580E+12 7.4580E+12 19-MAR-12 06:33:16 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138744_679263487.arc
138744 7.4580E+12 7.4580E+12 19-MAR-12 06:36:22 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138745_679263487.arc
138745 7.4580E+12 7.4580E+12 19-MAR-12 06:39:21 NO NO 1
FILE_NAME
SEQUENCE# FIRST_CHANGE# NEXT_CHANGE# TIMESTAMP DIC DIC THREAD#
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138746_679263487.arc
138746 7.4580E+12 7.4580E+12 19-MAR-12 06:41:25 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138747_679263487.arc
138747 7.4580E+12 7.4580E+12 19-MAR-12 06:43:24 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138748_679263487.arc
138748 7.4580E+12 7.4580E+12 19-MAR-12 06:45:21 NO NO 1
FILE_NAME
SEQUENCE# FIRST_CHANGE# NEXT_CHANGE# TIMESTAMP DIC DIC THREAD#
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138749_679263487.arc
138749 7.4580E+12 7.4580E+12 19-MAR-12 06:48:07 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138750_679263487.arc
138750 7.4580E+12 7.4580E+12 19-MAR-12 06:50:19 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138751_679263487.arc
138751 7.4580E+12 7.4580E+12 19-MAR-12 06:52:52 NO NO 1
FILE_NAME
SEQUENCE# FIRST_CHANGE# NEXT_CHANGE# TIMESTAMP DIC DIC THREAD#
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138752_679263487.arc
138752 7.4580E+12 7.4580E+12 19-MAR-12 06:55:32 NO NO 1
/pcegyk/oracle/product/102/dbs/archPCEGYK_1_138805_679263487.arc
138805 7.4580E+12 7.4580E+12 19-MAR-12 15:33:26 NO NO 1
=================
SQL> SELECT APPLIED_SCN, NEWEST_SCN FROM DBA_LOGSTDBY_PROGRESS;
APPLIED_SCN NEWEST_SCN
7.4580E+12 7.4580E+12

Different instructions for disable arch log mode on 11Gr2 RAC server?

Hello all,
I've run into a problem where I've lost my tape drive...and have no sysadmins to help.
I don't want my RAC instances to run our of space and halt, so I'm planning to take them out of archive log mode, and just do exports daily till I can move them or get tape going again.
This is easy enough with a non-clustered instance, but I'm reading around and finding conflicting information for doing it on a RAC system.
In the Oracle® Real Application Clusters Administration and Deployment Guide
11g Release 2 (11.2)...it states in simple terms:
(http://download.oracle.com/docs/cd/E11882_01/rac.112/e16795/rman.htm#i474611)
"In order for redo log files to be archived, the Oracle RAC database must be in ARCHIVELOG mode. You can run the ALTER DATABASE SQL statement to change the archiving mode in Oracle RAC, because the database is mounted by the local instance but not open in any instances. You do not need to modify parameter settings to run this statement."
and thats about it.
I've been researching and found a couple of other non-oracle official guides to this, where they describe a much more involved process, that seems to follow this path:
1. sqlplus into one instance and change the cluster_database=false scope=spfile sid='specific_node_name';
2. Shut down all instances, srvctl stop database -d <instance_name>
3. Startup the instance you changed cluster_database on with sqlplus and startup mount;
4 On this instance you do ALTER DATABASE NOARCHIVELOG;
5. On same instance change the cluster parameter back: alter system set cluster_database=true scope=spfile sid='specific_node_name';
6. Shut down this single instance
7. Start all instances with srvctl start database -d <instance>
I've found references to this at:
http://oracle-dba-yi.blogspot.com/2010/12/enabledisable-archive-log-mode-in-11gr2.html
and
http://www.dba-oracle.com/bk_disable_archive_log_mode.htm
Among other sites. I'm curious why the Oracle documentation on this doesn't mention all these steps?
I'm guessing the longer version is the path I should take, but I wanted to ask here first if this is correct?
I'm on Oracle 11Gr2....hasn't been patched with latest patchset, running on RHEL5, and is a 5 node cluster.
Thank you in advance,
cayenne
Edited by: cayenne on Oct 21, 2011 11:51 AM

Fiedi Z wrote:
There are couple things you need to consider
- export daily is not a backup strategy
- you're risking your enterprise company by disabling archivelog
your company have 5 RAC nodes I assume this is mid-to-large company, question you might ask to yourself Is your company really desperately don't have any available disk space for you to backup to temporary location / server?
However if you still insist and persistent with your strategy then follow the links you have, that is how to disable archivelog in RAC
CheersThank you everyone for the comments.
This is a DEV environment...and they are planning to move this all to a new facility where we won't have the powerouttages and old defunct equipment.
Right now...I do not have drive space to put all of this. I've informed them of the risks of not having point in time recovery. I really don't see any other choice on this....I don't want to run noarchive either...but I've been without tape to move the logs off for days now..and even with low traffic, I'm afraid they will fill and I'll have databases halting.
I think at this point, and again, this is not production data....I'm going to have to go with daily exports and that will have to do me till I can get these servers 'moved' to a new facility soon.
Again, thank you for the comments!!!
cayenne

Question about ASM Online log files

Hi All,
Wondering if this setup is desirable?
SQL> show parameter db_create
NAME TYPE VALUE
db_create_file_dest string +SYS1
db_create_online_log_dest_1 string +SYS1
db_create_online_log_dest_2 string +SYS2
db_create_online_log_dest_3 string +SYS3
db_create_online_log_dest_4 string +SYS4
db_create_online_log_dest_5 string
SQL>
Looks like logs are being sent to 4 DG's....

HI,
DB_CREATE_ONLINE_LOG_DEST_n (where n = 1, 2, 3, ... 5) specifies the default location for Oracle-managed control files and online redo logs. If more than one DB_CREATE_ONLINE_LOG_DEST_n parameter is specified, then the control file or online redo log is multiplexed across the locations of the other DB_CREATE_ONLINE_LOG_DEST_n parameters. One member of each online redo log is created in each location, and one control file is created in each location.
Specifying at least two parameters provides greater fault tolerance for the control files and online redo logs if one of the locations should fail.
If a file system directory is specified as the default location, then the directory must already exist; Oracle does not create it. The directory must have appropriate permissions that allow Oracle to create files in it. Oracle generates unique names for the files, and a file thus created is an Oracle Managed File.
Cheers

Missing arch logs

Hi
Oracle DB Version: 11.1.6
OS Version: Windows Server 2003.
When ever I start the database it is asking for recovery.
SQL> startup
ORACLE instance started.
Total System Global Area 644468736 bytes
Fixed Size 1335108 bytes
Variable Size 176160956 bytes
Database Buffers 461373440 bytes
Redo Buffers 5599232 bytes
Database mounted.
ORA-01113: file 5 needs media recovery
ORA-01110: data file 5: 'D:\SIEBELOWB\OWBDB\OWBDATA01.DBF'
SQL> recover datafile 5
ORA-00279: change 37455422 generated at 03/25/2011 06:56:41 needed for thread 1
ORA-00289: suggestion : C:\ORADB11G_HOME\RDBMS\ARC00210_0711838953.001
ORA-00280: change 37455422 for thread 1 is in sequence #210
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
auto
ORA-00310: archived log contains sequence 218; sequence 210 required
ORA-00334: archived log: 'C:\ORADB11G_HOME\RDBMS\ARC00210_0711838953.001'
The fact is that the database was never in archive log
Any idea or suggestion..
Thanks.

1) Because applying (archived) redo log is the only recovery mechanism available, as it is the very purpose of redo log
2) Because there is a possibility the online redologs are sufficient. Can Oracle know? Of course not.
3) Because no one in their sane mind is running a production database in noarchivelog.
As Tom Kyte has once put it : 'I don't believe a database which is running in noarchivelog, is a production database'
Rest assured, in the next major release, noarchivelog will disappear.
Apparently Oracle knows there are too many careless 'DBA's' out there, who never think about recovery, and run their database in noarchivelog.
There is an old Latin proverb 'Vitula demersa, puteum completur' Just in case you can't Google for it, it means most preventive measures are taken after the accident happened.
You have lost your database, I can not help it. Your only hope is you have an export somewhere.
Sybrand Bakker
Senior Oracle DBA

Applying the arch logs after the full backup

DB version:10GR2
In NOCATALOG mode, I am going to take a full backup (Hot) of DBx (800gb) using the command
run {
allocate channel c1 TYPE DISK connect 'sys/dempo' FORMAT '/u04/bkp/stprod_%U.rbk';
backup as compressed backupset database tag 'full' plus archivelog;
}and restore it into DBy.
But how can i apply the archive logs generated in DBx during the full backup after the full backup?
The restored control file doesn't know about these archived logs. Right?

with controlfile autobackup off, the backup will be 'complete', as backing up a whole database means backing up it's system tablespace(1 st datafile to be exact, which is the part of system tablespace)The SYSTEM datafile (or multiple datafiles in SYSTEM) may be the first or the second or the nth datafile to be backed up. The BackupSet consisting of multiple backup pieces may have other tablespaces backed up after the SYSTEM datafile. Those other tablespaces and possible redo log switches have timestamps which succeed the SYSTEM datafile backup . Therefore, those other tablespaces and redo log switches are not captured in that controlfile backup that was included with the SYSTEM datafile.
(Yes, I know we can use the controlfile backup --- but I am just pointing out that this doesn't meet the OP's requirement that the controlfile be aware of all redologs that have been switched during or after the database backup).
Hemant K Chitale
http://hemantoracledba.blogspot.com

ASM Direct file access (for creating standby database in Standard Edition)

Hi all,
Just really looking for any opinions, experience etc. that people may have about this.
My company are looking to set up a RAC system. They have also asked me to think about creating a standby database for disaster recovery. The first problem they give me here is that they only pay for Standard Edition of Oracle, therefore Dataguard is not an option.
I know that one can setup "manual" standby databases with Standard Edition, automating the shipping of archive logs with hand-written scripts etc. However, as the primary here is RAC, and will be using ASM on the shared storage, I'm not sure that this would be possible (and I'm even less sure that it would be desirable!). I say this because I don't know of a way to access (using only the O/S and/or Oracle) the database files stored in ASM, and even if there was a way, I can't imagine it being a good idea..
Anybody know of a way to tackle it, or care to contribute anything to this? As I say, my opinion is that even if it is possible to get to those files within ASM and hence manipulate them, I'd be worried that it wouldn't be such a good idea. But I'm willing to bet I'm not the first to have thought about it...
Regards,
Ados
Edit 1:
Sorry, I should state that in principal, the basic environment is:
Windows 2003 Server
Oracle 10g R2
I didn't state that originally though, as I'd be keen to hear from anyone who may have attempted this on any O/S and Oracle version.
Edited by: Ados on 16-sep-2009 9:32
Edit 2:
I also realise we could put the archive logs on the local nodes.. but to reinforce the point, I'm talking about having them on the shared storage where - in theory - the management is easier and more effective.
Edited by: Ados on 16-sep-2009 9:35

For anyone interested..
I guess it would be done using RMAN (in fact, I'm sure of it...) in which case, I feel a lot more comfortable doing it!!
For example:
backup as copy archivelog ALL to destination 'c:\temp\';
This way, it shouldn't matter if the arch logs are stored using ASM, we can now get them and "see" them.. and hence "manipulate" them (pass them on to a standby DB, for example).
I'm just waiting to get my hands on a Standard Edition installation where I can test all this.. To try it all out (with 2-node RAC, and a standby server) I don't think I'll get a chance, so still interested in opinions, or if anyone's actually done this.
Regards,
Ados

Alert log: failed to establish dependency between database and diskgroup

Hi, I have a database 11.2.0 with ASM. When I startup an instance I see this error in alert log:
ERROR: failed to establish dependency between database SPBUFOR and diskgroup resource ora.SPBUFOR_FLASH.dg
SPBUFOR_FLASH is a diskgroup where a flash recovery area is located.
Database is opened cleanly but I need to know that everything works fine.
How can I resolve this issue ?

ASM ALERT LOG:
Wed Oct 05 15:35:12 2011
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/oracle/product/11.2.0/grid/dbs/arch
Autotune of undo retention is turned on.
IMODE=BR
ILAT =0
LICENSE_MAX_USERS = 0
SYS auditing is disabled
NOTE: Volume support enabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Automatic Storage Management option.
Using parameter settings in server-side spfile +CSA_DATA/asm/asmparameterfile/registry.253.758208241
System parameters with non-default values:
large_pool_size = 12M
instance_type = "asm"
remote_login_passwordfile= "EXCLUSIVE"
asm_diskgroups = "CSA_FLASH"
asm_diskgroups = "SPBUFOR_DATA"
asm_diskgroups = "SPBUFOR_FLASH"
asm_power_limit = 1
diagnostic_dest = "/u01/app/oracle"
Wed Oct 05 15:35:13 2011
PMON started with pid=2, OS id=24108
Wed Oct 05 15:35:14 2011
VKTM started with pid=3, OS id=24112 at elevated priority
VKTM running at (10)millisec precision with DBRM quantum (100)ms
Wed Oct 05 15:35:14 2011
GEN0 started with pid=4, OS id=24118
Wed Oct 05 15:35:14 2011
DIAG started with pid=5, OS id=24122
Wed Oct 05 15:35:14 2011
PSP0 started with pid=6, OS id=24127
Wed Oct 05 15:35:14 2011
DIA0 started with pid=7, OS id=24131
Wed Oct 05 15:35:14 2011
MMAN started with pid=8, OS id=24135
Wed Oct 05 15:35:14 2011
DBW0 started with pid=9, OS id=24139
Wed Oct 05 15:35:14 2011
LGWR started with pid=10, OS id=24143
Wed Oct 05 15:35:14 2011
CKPT started with pid=11, OS id=24148
Wed Oct 05 15:35:14 2011
SMON started with pid=12, OS id=24157
Wed Oct 05 15:35:14 2011
RBAL started with pid=13, OS id=24163
Wed Oct 05 15:35:14 2011
GMON started with pid=14, OS id=24173
Wed Oct 05 15:35:14 2011
MMON started with pid=15, OS id=24177
Wed Oct 05 15:35:14 2011
MMNL started with pid=16, OS id=24181
ORACLE_BASE from environment = /u01/app/oracle
Wed Oct 05 15:35:15 2011
SQL> ALTER DISKGROUP ALL MOUNT
NOTE: Diskgroups listed in ASM_DISKGROUPS are
CSA_FLASH
SPBUFOR_DATA
SPBUFOR_FLASH
Diskgroup with spfile:CSA_DATA
NOTE: cache registered group CSA_DATA number=1 incarn=0x75fe76df
NOTE: cache began mount (first) of group CSA_DATA number=1 incarn=0x75fe76df
NOTE: cache registered group CSA_FLASH number=2 incarn=0x760e76e0
NOTE: cache began mount (first) of group CSA_FLASH number=2 incarn=0x760e76e0
NOTE: cache registered group SPBUFOR_DATA number=3 incarn=0x760e76e1
NOTE: cache began mount (first) of group SPBUFOR_DATA number=3 incarn=0x760e76e1
NOTE: cache registered group SPBUFOR_FLASH number=4 incarn=0x761e76e2
NOTE: cache began mount (first) of group SPBUFOR_FLASH number=4 incarn=0x761e76e2
NOTE: Loaded library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so
NOTE: Assigning number (1,0) to disk (ORCL:CSA_D1)
NOTE: Assigning number (1,1) to disk (ORCL:CSA_D2)
NOTE: Assigning number (2,0) to disk (ORCL:CSA_F1)
NOTE: Assigning number (2,1) to disk (ORCL:CSA_F2)
NOTE: Assigning number (3,0) to disk (ORCL:SPBUFOR_D1)
NOTE: Assigning number (3,1) to disk (ORCL:SPBUFOR_D2)
NOTE: Assigning number (3,2) to disk (ORCL:SPBUFOR_D3)
NOTE: Assigning number (4,0) to disk (ORCL:SPBUFOR_F1)
NOTE: Assigning number (4,1) to disk (ORCL:SPBUFOR_F2)
NOTE: start heartbeating (grp 1)
kfdp_query(CSA_DATA): 6
kfdp_queryBg(): 6
NOTE: cache opening disk 0 of grp 1: CSA_D1 label:CSA_D1
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache opening disk 1 of grp 1: CSA_D2 label:CSA_D2
NOTE: cache mounting (first) external redundancy group 1/0x75FE76DF (CSA_DATA)
NOTE: cache recovered group 1 to fcn 0.25573
NOTE: LGWR attempting to mount thread 1 for diskgroup 1 (CSA_DATA)
NOTE: LGWR found thread 1 closed at ABA 4.2486
NOTE: LGWR mounted thread 1 for diskgroup 1 (CSA_DATA)
NOTE: LGWR opening thread 1 at fcn 0.25573 ABA 5.2487
NOTE: cache mounting group 1/0x75FE76DF (CSA_DATA) succeeded
NOTE: cache ending mount (success) of group CSA_DATA number=1 incarn=0x75fe76df
NOTE: start heartbeating (grp 2)
kfdp_query(CSA_FLASH): 8
kfdp_queryBg(): 8
NOTE: cache opening disk 0 of grp 2: CSA_F1 label:CSA_F1
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache opening disk 1 of grp 2: CSA_F2 label:CSA_F2
NOTE: cache mounting (first) external redundancy group 2/0x760E76E0 (CSA_FLASH)
NOTE: cache recovered group 2 to fcn 0.49881
NOTE: LGWR attempting to mount thread 1 for diskgroup 2 (CSA_FLASH)
NOTE: LGWR found thread 1 closed at ABA 3.5793
NOTE: LGWR mounted thread 1 for diskgroup 2 (CSA_FLASH)
NOTE: LGWR opening thread 1 at fcn 0.49881 ABA 4.5794
NOTE: cache mounting group 2/0x760E76E0 (CSA_FLASH) succeeded
NOTE: cache ending mount (success) of group CSA_FLASH number=2 incarn=0x760e76e0
NOTE: start heartbeating (grp 3)
kfdp_query(SPBUFOR_DATA): 10
kfdp_queryBg(): 10
NOTE: cache opening disk 0 of grp 3: SPBUFOR_D1 label:SPBUFOR_D1
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache opening disk 1 of grp 3: SPBUFOR_D2 label:SPBUFOR_D2
NOTE: cache opening disk 2 of grp 3: SPBUFOR_D3 label:SPBUFOR_D3
NOTE: cache mounting (first) external redundancy group 3/0x760E76E1 (SPBUFOR_DATA)
NOTE: cache recovered group 3 to fcn 0.317867
NOTE: LGWR attempting to mount thread 1 for diskgroup 3 (SPBUFOR_DATA)
NOTE: LGWR found thread 1 closed at ABA 3.8570
NOTE: LGWR mounted thread 1 for diskgroup 3 (SPBUFOR_DATA)
NOTE: LGWR opening thread 1 at fcn 0.317867 ABA 4.8571
NOTE: cache mounting group 3/0x760E76E1 (SPBUFOR_DATA) succeeded
NOTE: cache ending mount (success) of group SPBUFOR_DATA number=3 incarn=0x760e76e1
NOTE: start heartbeating (grp 4)
kfdp_query(SPBUFOR_FLASH): 12
kfdp_queryBg(): 12
NOTE: cache opening disk 0 of grp 4: SPBUFOR_F1 label:SPBUFOR_F1
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache opening disk 1 of grp 4: SPBUFOR_F2 label:SPBUFOR_F2
NOTE: cache mounting (first) external redundancy group 4/0x761E76E2 (SPBUFOR_FLASH)
NOTE: cache recovered group 4 to fcn 0.16114
NOTE: LGWR attempting to mount thread 1 for diskgroup 4 (SPBUFOR_FLASH)
NOTE: LGWR found thread 1 closed at ABA 2.1922
NOTE: LGWR mounted thread 1 for diskgroup 4 (SPBUFOR_FLASH)
NOTE: LGWR opening thread 1 at fcn 0.16114 ABA 3.1923
NOTE: cache mounting group 4/0x761E76E2 (SPBUFOR_FLASH) succeeded
NOTE: cache ending mount (success) of group SPBUFOR_FLASH number=4 incarn=0x761e76e2
kfdp_query(CSA_DATA): 13
kfdp_queryBg(): 13
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1
SUCCESS: diskgroup CSA_DATA was mounted
kfdp_query(CSA_FLASH): 14
kfdp_queryBg(): 14
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 2
SUCCESS: diskgroup CSA_FLASH was mounted
kfdp_query(SPBUFOR_DATA): 15
kfdp_queryBg(): 15
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 3
SUCCESS: diskgroup SPBUFOR_DATA was mounted
kfdp_query(SPBUFOR_FLASH): 16
kfdp_queryBg(): 16
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 4
SUCCESS: diskgroup SPBUFOR_FLASH was mounted
SUCCESS: ALTER DISKGROUP ALL MOUNT
SQL> ALTER DISKGROUP ALL ENABLE VOLUME ALL
SUCCESS: ALTER DISKGROUP ALL ENABLE VOLUME ALL
NOTE: diskgroup resource ora.CSA_DATA.dg is online
NOTE: diskgroup resource ora.CSA_FLASH.dg is online
NOTE: diskgroup resource ora.SPBUFOR_DATA.dg is online
NOTE: diskgroup resource ora.SPBUFOR_FLASH.dg is online
Wed Oct 05 15:35:44 2011
Starting background process ASMB
Wed Oct 05 15:35:44 2011
ASMB started with pid=18, OS id=24330
Thu Oct 06 11:48:19 2011
SQL> alter diskgroup SPBUFOR_DATA check all
NOTE: starting check of diskgroup SPBUFOR_DATA
kfdp_checkDsk(): 17
kfdp_checkDsk(): 18
Thu Oct 06 11:48:30 2011
kfdp_checkDsk(): 19
SUCCESS: check of diskgroup SPBUFOR_DATA found no errors
SUCCESS: alter diskgroup SPBUFOR_DATA check all
Thu Oct 06 11:48:56 2011
SQL> alter diskgroup SPBUFOR_FLASH check all
NOTE: starting check of diskgroup SPBUFOR_FLASH
kfdp_checkDsk(): 20
kfdp_checkDsk(): 21
SUCCESS: check of diskgroup SPBUFOR_FLASH found no errors
SUCCESS: alter diskgroup SPBUFOR_FLASH check all

RMAN duplicate from RAC with ASM to RAC with ASM

I'm still reading and trying to figure out how to duplicate...and I'm finding that there are some extra considerations you have to work with with doing a RAC system.
Does anyone have any good links to articles/docs that spell out what to do in this scenario?
I'm wanting to duplicate from tape backups, using NO connectivity to the source/target....
My tape backups do include the spfile and control files (autobackup) from the source.
Thank you in advance,
cayenne

damorgan wrote:
Possibly you are confusing a database with a database.
A RAC database is a just a database. What is different is the instances, the clusterware, and the storage: For example ASM.
A backup is a backup is a backup.
Can you be more specific about what you are trying to do, on what hardware, operating system, version, and edition.
RMAN can not create an instance either RAC or stand-alone.Thank you for the reply.
I have my source system, OS = RHEL5, running Oracle 11Gr2...5 node RAC cluster.
I'm doing RMAN backups to tape...hourly arch. logs, Daily incremental lvl=1 backups and weekly incremental lvl=0
None of the tape backups have ever been tested for restoring...and I've never restored a database myself before, total noob here.
I have a test area I've set up. I have a 2 node RAC cluster, running 11Gr2 and OS=RHEL5.
The tape is accessible from both systems.
I am wanting to test the tape backups...and thought the RMAN DUPLICATE process would be the way to go.
I am wanting to NOT connect to the source database...trying to simulate somewhat of a disaster recovery scenario. I'm only wanting to use tape backup, and the test area for the auxiliary instance.
So far what I've seen I need to learn to do is something like:
1. Create a password file for new aux database to be duplicated to
2. Establish Net connectivity...
3. Create initial parameter file for aux instance
Due to a bug in note:334899.1, add this (because of RAC system)
norecovery_through_resetlogs=TRUE
4. Start aux instance NOMOUNT
Everything I'm reading though...is basically doing this from single instance to single instance...and what little info I've seen on doing it from RAC, indicates there are some differences. The "_no_recovery_through_resetlogs=TRUE" is one thing I found...but wondering what else.
Also, so many of the examples I'm finding...are doing the duplication connecting to the target/source...and also doing backups to disk rather than tape...
Right now, I'm at #1...trying to figure what to put into an init pfile...I'm seeing DB_name, which will be the same as the one I'm cloning from.
I'm not sure what else....
I'm wondering if this is necessary..since on the tape backups from the source...I backup the SPFILE...can that not be used somehow in this?
I'm seeing for the init file, examples show that I need to put in entries for control files and redo logs....if the source system was down and gone...how would I know where these were on the old system? This isn't documented anywhere....is there a way to do this if you didn't know.
If not..guess I need to go through all systems and document the layouts of where everything is located.
Also...most examples I'm finding, in addition to being single instance backup and restore/duplication...they are all using regular file systems....not much to go on with using ASM.
Anyway, I'm trying to learn here...and am having trouble finding examples to go from that match my setup.
Thank you in advance,
cayenne

Root.sh fails on 11.2.0.3 clusterware while starting 'ora.asm' resource

Dear all,
I am trying to install clean Oracle 11.2.0.3 grid infrastructure on a two node cluster running on Solaris 5.10.
- Cluster verification was successfully on both nodes; No warning or issues;
- I am using 2 network cards for the public and 2 for the private interconnect;
- OCR is stored on ASM
- Firewall is disabled on both nodes
- SCAN is being configured on the DNS (not added in /etc/hosts)
- GNS is not used
- hosts file is identical (except the primary hostname)
The problem: root.sh fails on the 2nd (remote) node, because it fails to start the "ora.asm" resource. However, the root.sh has completed successfully on the 1st node.. Somehow, root.sh doesn't create +ASM2 instance on the remote (host2) node.
root.sh was executed first on the local node (host1) and after the successful execution was started on the remote (host2) node.
Output from host1 (working):
===================
Adding Clusterware entries to inittab
CRS-2672: Attempting to start 'ora.mdnsd' on 'host1'
CRS-2676: Start of 'ora.mdnsd' on 'host1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'host1'
CRS-2676: Start of 'ora.gpnpd' on 'host1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'host1'
CRS-2672: Attempting to start 'ora.gipcd' on 'host1'
CRS-2676: Start of 'ora.cssdmonitor' on 'host1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'host1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'host1'
CRS-2672: Attempting to start 'ora.diskmon' on 'host1'
CRS-2676: Start of 'ora.diskmon' on 'host1' succeeded
CRS-2676: Start of 'ora.cssd' on 'host1' succeeded
ASM created and started successfully.
Disk Group CRS created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 4373be34efab4f01bf79f6c5362acfd3.
Successful addition of voting disk 7fd725fa4d904f07bf76cecf96791547.
Successful addition of voting disk a9c85297bdd74f3abfd86899205aaf17.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
1. ONLINE 4373be34efab4f01bf79f6c5362acfd3 (/dev/rdsk/c4t600A0B80006E2CC40000C6674E82AA57d0s4) [CRS]
2. ONLINE 7fd725fa4d904f07bf76cecf96791547 (/dev/rdsk/c4t600A0B80006E2CC40000C6694E82AADDd0s4) [CRS]
3. ONLINE a9c85297bdd74f3abfd86899205aaf17 (/dev/rdsk/c4t600A0B80006E2F100000C7744E82AC7Ad0s4) [CRS]
Located 3 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'host1'
CRS-2676: Start of 'ora.asm' on 'host1' succeeded
CRS-2672: Attempting to start 'ora.CRS.dg' on 'host1'
CRS-2676: Start of 'ora.CRS.dg' on 'host1' succeeded
CRS-2672: Attempting to start 'ora.registry.acfs' on 'host1'
CRS-2676: Start of 'ora.registry.acfs' on 'host1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Name Type Target State Host
ora.CRS.dg ora....up.type ONLINE ONLINE host1
ora....ER.lsnr ora....er.type ONLINE ONLINE host1
ora....N1.lsnr ora....er.type ONLINE ONLINE host1
ora....N2.lsnr ora....er.type ONLINE ONLINE host1
ora....N3.lsnr ora....er.type ONLINE ONLINE host1
ora.asm ora.asm.type ONLINE ONLINE host1
ora....SM1.asm application ONLINE ONLINE host1
ora....B1.lsnr application ONLINE ONLINE host1
ora....db1.gsd application OFFLINE OFFLINE
ora....db1.ons application ONLINE ONLINE host1
ora....db1.vip ora....t1.type ONLINE ONLINE host1
ora.cvu ora.cvu.type ONLINE ONLINE host1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE host1
ora.oc4j ora.oc4j.type ONLINE ONLINE host1
ora.ons ora.ons.type ONLINE ONLINE host1
ora....ry.acfs ora....fs.type ONLINE ONLINE host1
ora.scan1.vip ora....ip.type ONLINE ONLINE host1
ora.scan2.vip ora....ip.type ONLINE ONLINE host1
ora.scan3.vip ora....ip.type ONLINE ONLINE host1
Output from host2 (failing):
===================
OLR initialization - successful
Adding Clusterware entries to inittab
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node billdb1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Start of resource "ora.asm" failed
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'host2'
CRS-2676: Start of 'ora.drivers.acfs' on 'host2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'host2'
CRS-5017: The resource action "ora.asm start" encountered the following error:
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
*. For details refer to "(:CLSN00107:)" in "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log".*
CRS-2674: Start of 'ora.asm' on 'host2' failed
CRS-2679: Attempting to clean 'ora.asm' on 'host2'
CRS-2681: Clean of 'ora.asm' on 'host2' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'host2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'host2' succeeded
CRS-4000: Command Start failed, or completed with errors.
Failed to start Oracle Grid Infrastructure stack
Failed to start ASM at /u01/11.2.0/grid/crs/install/crsconfig_lib.pm line 1272.
/u01/11.2.0/grid/perl/bin/perl -I/u01/11.2.0/grid/perl/lib -I/u01/11.2.0/grid/crs/install /u01/11.2.0/grid/crs/install/rootcrs.pl execution failed
Contents of "/u01/11.2.0/grid/cfgtoollogs/crsconfig/rootcrs_host2.log"
=============================================
CRS-2672: Attempting to start 'ora.asm' on 'host2'
CRS-5017: The resource action "ora.asm start" encountered the following error:
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log".
CRS-2674: Start of 'ora.asm' on 'host2' failed
CRS-2679: Attempting to clean 'ora.asm' on 'host2'
CRS-2681: Clean of 'ora.asm' on 'host2' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'host2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'host2' succeeded
CRS-4000: Command Start failed, or completed with errors.
2011-10-24 19:36:54: Failed to start Oracle Grid Infrastructure stack
2011-10-24 19:36:54: ###### Begin DIE Stack Trace ######
2011-10-24 19:36:54: Package File Line Calling
2011-10-24 19:36:54: --------------- -------------------- ---- ----------
2011-10-24 19:36:54: 1: main rootcrs.pl 375 crsconfig_lib::dietrap
2011-10-24 19:36:54: 2: crsconfig_lib crsconfig_lib.pm 1272 main::__ANON__
2011-10-24 19:36:54: 3: crsconfig_lib crsconfig_lib.pm 1171 crsconfig_lib::start_cluster
2011-10-24 19:36:54: 4: main rootcrs.pl 803 crsconfig_lib::perform_start_cluster
2011-10-24 19:36:54: ####### End DIE Stack Trace #######
Shortened output from "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log"
2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] clean {
2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] InstAgent::stop_option stop mode immediate option 1
2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] InstAgent::stop {
2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] InstAgent::stop original reason system do shutdown abort
2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] ConnectionPool::resetConnection s_statusOfConnectionMap 00ab1948
2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] ConnectionPool::resetConnection sid +ASM2 status 2
2011-10-24 19:35:48.728: [ora.asm][9] {0:0:224} [start] Gimh::check OH /u01/11.2.0/grid SID +ASM2
2011-10-24 19:35:48.728: [ora.asm][9] {0:0:224} [start] Gimh::check condition changes to (GIMH_NEXT_NUM) 0,1,7 exists
2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] (:CLSN00006:)AsmAgent::check failed gimh state 0
2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] AsmAgent::check ocrCheck 1 m_OcrOnline 0 m_OcrTimer 0
2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet { entry
2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet procr_get_conf: retval [0] configured [1] local only [0] error buffer []
2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet procr_get_conf: OCR loc [0], Disk Group : [+CRS]
2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet m_ocrDgpSet 015fba90 dgName CRS
2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet ocrret 0 found 1
2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet ocrDgpSet CRS
2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet exit }
2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::ocrDgCheck Entry {
2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] DgpAgent::getConnxn new pool
2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] DgpAgent::getConnxn new pool m_oracleHome:/u01/11.2.0/grid m_oracleSid:+ASM2 m_usrOraEnv:
2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] ConnectionPool::ConnectionPool 2 m_oracleHome:/u01/11.2.0/grid, m_oracleSid:+ASM2, m_usrOraEnv:
2011-10-24 19:35:48.733: [ora.asm][9] {0:0:224} [start] ConnectionPool::addConnection m_oracleHome:/u01/11.2.0/grid m_oracleSid:+ASM2 m_usrOraEnv: pConnxn:
01fcdf10
2011-10-24 19:35:48.733: [ora.asm][9] {0:0:224} [start] Utils::getCrsHome crsHome /u01/11.2.0/grid
2011-10-24 19:35:51.969: [ora.asm][14] {0:0:224} [check] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/u01/11.2.0/grid/bin/oracle)(ARGV0=o
racle+ASM2)(ENVS='ORACLE_HOME=/u01/11.2.0/grid,ORACLE_SID=+ASM2')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM2)))
2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] ConnectionPool::getConnection 260 pConnxn 013e40a0
2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] DgpAgent::getConnxn connected
2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] InstConnection::connectInt: server not attached
2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
SVR4 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] InstConnection::connectInt (2) Exception OCIException
2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] InstConnection:connect:excp OCIException OCI error 1034
2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus excp ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
SVR4 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus asm inst is down or going down
2011-10-24 19:35:52.191: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus dgName CRS ret 1
2011-10-24 19:35:52.191: [ora.asm][14] {0:0:224} [check] (:CLSN00100:)DgpAgent::ocrDgCheck OCR dgName CRS state 1
2011-10-24 19:35:52.192: [ora.asm][14] {0:0:224} [check] ConnectionPool::releaseConnection InstConnection 013e40a0
2011-10-24 19:35:52.192: [ora.asm][14] {0:0:224} [check] AsmAgent::check ocrCheck 2 m_OcrOnline 0 m_OcrTimer 0
2011-10-24 19:35:52.193: [ora.asm][14] {0:0:224} [check] CrsCmd::ClscrsCmdData::stat entity 1 statflag 32 useFilter 0
2011-10-24 19:35:52.197: [ COMMCRS][23]clsc_connect: (1020d39d0) no listener at (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_UI_SOCKET))
Please advice for any workaround or a metalink note.
Thanks in advance!

Thanks for the fast reply!
- Yes, the shared storage is accessible.
- The alert log for the +ASM2 clearly shows that ASM instance has started normally using default parameters and at one point PMON process dumped.
- The system logs just shows that there is an error executing "crswrapexece.pl"
System Log
===================
*Oct 24 19:25:03 host2 root: [ID 702911 user.error] exec /u01/11.2.0/grid/perl/bin/perl -I/u01/11.2.0/grid/perl/lib /u01/11.2.0/grid/bin/crswrapexece.pl /*
u01/11.2.0/grid/crs/install/s_crsconfig_host2_env.txt /u01/11.2.0/grid/bin/ohasd.bin "reboot"
Oct 24 19:26:33 host2 oracleoks: [ID 902884 kern.notice] [Oracle OKS] mallocing log buffer, size=10485760
Oct 24 19:26:33 host2 oracleoks: [ID 714332 kern.notice] [Oracle OKS] log buffer = 0x301780fcb50, size 10485760
Oct 24 19:26:33 host2 oracleoks: [ID 400061 kern.notice] NOTICE: [Oracle OKS] ODLM hash size 16384
Oct 24 19:26:33 host2 oracleoks: [ID 160659 kern.notice] NOTICE: OKSK-00004: Module load succeeded. Build information: (LOW DEBUG) USM_11.2.0.3.0_SOLAR
IS.SPARC64_110803.1 2011/08/11 02:38:30
Oct 24 19:26:33 host2 pseudo: [ID 129642 kern.info] pseudo-device: oracleadvm0
Oct 24 19:26:33 host2 genunix: [ID 936769 kern.info] oracleadvm0 is /pseudo/oracleadvm@0
Oct 24 19:26:33 host2 oracleoks: [ID 141287 kern.notice] NOTICE: ADVMK-00001: Module load succeeded. Build information: (LOW DEBUG) - USM_11.2.0.3.0_SOL
ARIS.SPARC64_110803.1 built on 2011/08/11 02:40:17.
Oct 24 19:26:33 host2 oracleacfs: [ID 202941 kern.notice] NOTICE: [Oracle ACFS] FCB hash size 16384
Oct 24 19:26:33 host2 oracleacfs: [ID 671725 kern.notice] NOTICE: [Oracle ACFS] buffer cache size 511MB (79884 buckets)
Oct 24 19:26:33 host2 oracleacfs: [ID 730054 kern.notice] NOTICE: [Oracle ACFS] DLM hash size 16384
Oct 24 19:26:33 host2 oracleoks: [ID 617314 kern.notice] NOTICE: ACFSK-0037: Module load succeeded. Build information: (LOW DEBUG) USM_11.2.0.3.0_SOLAR
IS.SPARC64_110803.1 2011/08/11 02:42:45
Oct 24 19:26:33 host2 pseudo: [ID 129642 kern.info] pseudo-device: oracleacfs0
Oct 24 19:26:33 host2 genunix: [ID 936769 kern.info] oracleacfs0 is /pseudo/oracleacfs@0
Oct 24 19:26:36 host2 oracleoks: [ID 621795 kern.notice] NOTICE: OKSK-00010: Persistent OKS log opened at /u01/11.2.0/grid/log/host2/acfs/acfs.log.0.
Oct 24 19:31:37 host2 last message repeated 1 time
Oct 24 19:33:05 host2 CLSD: [ID 770310 daemon.notice] The clock on host host2 has been updated by the Cluster Time Synchronization Service to be synchr
onous with the mean cluster time.
ASM alert log
====================================================================
<msg time='2011-10-24T19:35:48.776+01:00' org_id='oracle' comp_id='asm'
client_id='' type='UNKNOWN' level='16'
host_id='host2' host_addr='10.172.16.200' module=''
pid='26406'>
<txt>System state dump requested by (instance=2, osid=26396 (PMON)), summary=[abnormal instance termination].
</txt>
</msg>
<msg time='2011-10-24T19:35:48.778+01:00' org_id='oracle' comp_id='asm'
client_id='' type='UNKNOWN' level='16'
host_id='host2' host_addr='10.172.16.200' module=''
pid='26406'>
<txt>System State dumped to trace file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_26406.trc
</txt>
</msg>
<msg time='2011-10-24T19:35:48.927+01:00' org_id='oracle' comp_id='asm'
type='UNKNOWN' level='16' host_id='host2'
host_addr='10.172.16.200' pid='26470'>
<txt>ORA-1092 : opitsk aborting process
</txt>
</msg>
<msg time='2011-10-24T19:35:49.128+01:00' org_id='oracle' comp_id='asm'
type='UNKNOWN' level='16' host_id='host2'
host_addr='10.172.16.200' pid='26472'>
<txt>ORA-1092 : opitsk aborting process
</txt>
</msg>
Output from "/u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_26406.trc"
REQUEST:system state dump at level 10, requested by (instance=2, osid=26396 (PMON)), summary=[abnormal instance termination].
kjzdattdlm: Can not attach to DLM (LMON up=[TRUE], DB mounted=[FALSE]).
===================================================
SYSTEM STATE (level=10)
Orapids on dead process list: [count = 0]
PROCESS 1:
SO: 0x3df098b50, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x3df098b50, name=process, file=ksu.h LINE:12616 ID:, pg=0
(process) Oracle pid:1, ser:0, calls cur/top: 0x0/0x0
flags : (0x20) PSEUDO
flags2: (0x0), flags3: (0x10)
intr error: 0, call error: 0, sess error: 0, txn error 0
intr queue: empty
ksudlp FALSE at location: 0
(post info) last post received: 0 0 0
last post received-location: No post
last process to post me: none
last post sent: 0 0 0
last post sent-location: No post
last process posted by me: none
(latch info) wait_event=0 bits=0
O/S info: user: , term: , ospid: (DEAD)
OSD pid info: Unix process pid: 0, image: PSEUDO
SO: 0x38000cef0, type: 5, owner: 0x3df098b50, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x0, name=kss parent, file=kss2.h LINE:138 ID:, pg=0
PSO child state object changes :
Dump of memory from 0x00000003DF722AC0 to 0x00000003DF722CC8
3DF722AC0 00000000 00000000 00000000 00000000 [................]
Repeat 31 times
3DF722CC0 00000000 00000000 [........]
PROCESS 2: PMON
SO: 0x3df099bf8, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x3df099bf8, name=process, file=ksu.h LINE:12616 ID:, pg=0
(process) Oracle pid:2, ser:1, calls cur/top: 0x3db6c8d30/0x3db6c8d30
flags : (0xe) SYSTEM
flags2: (0x0), flags3: (0x10)
intr error: 0, call error: 0, sess error: 0, txn error 0
intr queue: empty
ksudlp FALSE at location: 0
(post info) last post received: 0 0 136
last post received-location: kjm.h LINE:1228 ID:kjmdmi: pmon to attach
last process to post me: 3df0a2138 1 6
last post sent: 0 0 137
last post sent-location: kjm.h LINE:1230 ID:kjiath: pmon attached
last process posted by me: 3df0a2138 1 6
(latch info) wait_event=0 bits=0
Process Group: DEFAULT, pseudo proc: 0x3debbbf40
O/S info: user: grid, term: UNKNOWN, ospid: 26396
OSD pid info: Unix process pid: 26396, image: oracle@host2 (PMON)
SO: 0x3d8800c18, type: 30, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x3df099bf8, name=ges process, file=kji.h LINE:3669 ID:, pg=0
GES MSG BUFFERS: st=emp chunk=0x0 hdr=0x0 lnk=0x0 flags=0x0 inc=0
outq=0 sndq=0 opid=0 prmb=0x0
mbg=(0 0) mbg=(0 0) mbg[r]=(0 0)
fmq=(0 0) fmq=(0 0) fmq[r]=(0 0)
mop[s]=0 mop[q]=0 pendq=0 zmbq=0
nonksxp_recvs=0
------------process 3d8800c18--------------------
proc version : 0
Local inst : 2
pid : 26396
lkp_inst : 2
svr_mode : 0
proc state : KJP_FROZEN
Last drm hb acked : 0
flags : x50
ast_rcvd_svrmod : 0
current lock op : 0
Total accesses : 1
Imm. accesses : 0
Locks on ASTQ : 0
Locks Pending AST : 0
Granted locks : 0
AST_Q:
PENDING_Q:
GRANTED_Q:
SO: 0x3d9835198, type: 14, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
proc=0x3df099bf8, name=channel handle, file=ksr2.h LINE:367 ID:, pg=0
(broadcast handle) 3d9835198 flag: (2) ACTIVE SUBSCRIBER,
owner: 3df099bf8 - ospid: 26396
event: 1, last message event: 1,
last message waited event: 1,
next message: 0(0), messages read: 0
channel: (3d9934df8) PMON actions channel [name: 2]
scope: 7, event: 1, last mesage event: 0,
publishers/subscribers: 0/1,
messages published: 0
heuristic msg queue length: 0
SO: 0x3d9835008, type: 14, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
proc=0x3df099bf8, name=channel handle, file=ksr2.h LINE:367 ID:, pg=0
(broadcast handle) 3d9835008 flag: (2) ACTIVE SUBSCRIBER,
owner: 3df099bf8 - ospid: 26396
event: 1, last message event: 1,
last message waited event: 1,
next message: 0(0), messages read: 0
channel: (3d9941e40) scumnt mount lock [name: 157]
scope: 1, event: 12, last mesage event: 0,
publishers/subscribers: 0/12,
messages published: 0
heuristic msg queue length: 0
SO: 0x3de4a2b80, type: 4, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x3df099bf8, name=session, file=ksu.h LINE:12624 ID:, pg=0
(session) sid: 33 ser: 1 trans: 0x0, creator: 0x3df099bf8
flags: (0x51) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
flags2: (0x409) -/-/INC
DID: , short-term DID:
txn branch: 0x0
oct: 0, prv: 0, sql: 0x0, psql: 0x0, user: 0/SYS
ksuxds FALSE at location: 0
service name: SYS$BACKGROUND
Current Wait Stack:
Not in wait; last wait ended 0.666415 sec ago
Wait State:
fixed_waits=0 flags=0x21 boundary=0x0/-1
Session Wait History:
elapsed time of 0.666593 sec since last wait
0: waited for 'pmon timer'
duration=0x12c, =0x0, =0x0
wait_id=63 seq_num=64 snap_id=1
wait times: snap=3.000089 sec, exc=3.000089 sec, total=3.000089 sec
wait times: max=3.000000 sec
wait counts: calls=1 os=1
occurred after 0.002067 sec of elapsed time
1: waited for 'pmon timer'
duration=0x12c, =0x0, =0x0
wait_id=62 seq_num=63 snap_id=1
wait times: snap=3.010111 sec, exc=3.010111 sec, total=3.010111 sec
wait times: max=3.000000 sec
wait counts: calls=1 os=1
occurred after 0.001926 sec of elapsed time
2: waited for 'pmon timer'
duration=0x12c, =0x0, =0x0
wait_id=61 seq_num=62 snap_id=1
wait times: snap=3.125286 sec, exc=3.125286 sec, total=3.125286 sec
wait times: max=3.000000 sec
wait counts: calls=1 os=1
occurred after 0.003361 sec of elapsed time
3: waited for 'pmon timer'
duration=0x12c, =0x0, =0x0
wait_id=60 seq_num=61 snap_id=1
wait times: snap=3.000081 sec, exc=3.000081 sec, total=3.000081 sec
wait times: max=3.000000 sec
wait counts: calls=1 os=1
occurred after 0.002102 sec of elapsed time
4: waited for 'pmon timer'
duration=0x12c, =0x0, =0x0

ASM - arch logs

Similar Messages

Maybe you are looking for