ONLINELOG

Hi,
I have problem with the second online redo log file.
SQL> ALTER DATABASE OPEN;
ALTER DATABASE OPEN;
ERROR at line 1:
ORA-00322: log 2 of thread 1 is not current copy
ORA-00312: online log 2 thread 1:
'D:\ORACLEXE\ORADATA\XE\O1_MF_2_266RQQDC_.LOG'
That happened when I copied files from one folder to another.
Then ALTER DATABASE RENAME FILE …
Database altered, but I tried to open the database until the process of copying files.
Is there any technique to restore and open the database? Unfortunately, there is NO any copy or archive.
I will appreciate any help or ideas.
Thank you
Konstantin

Hi,
The problem was solved.
If there is a problem with the online redo log files.
One is deleted or corrupted.
SQL> SHUTDOWN
CONNECT / AS SYSDBA
STARTUP NOMAUNT;
ALTER DATABASE MOUNT;
RECOVER DATABESE UNTIL CANCEL;
ALTER DATABASE OPEN;
That is "incomplete database restore".
In my case the second group of redo log file was corrupted,
but still on place and visible (normal) from OS.
You must change last statement with ALTER DATABASE OPEN RESTORELOGS
if the log file is badly corrupted (can not open by OS) or deleted.
Oracle creates two new redo log files in this case.
Thank you for your help
Konstantin

Similar Messages

  • What is ONLINELOG for in the FRA

    Hi,
    I have a primary ( PRIMARYP ) and 2 standbys - 1 physical standby ( STANDBYP ) and 1 logical standby ( STANDBYL )
    When I converted STANDBYL from physical to logical, a couple of new directories appeared under my FRA i.e. ONLINELOG and FOREIGN_ARCHIVELOG. Currently ONLINELOG is empty
    My LOG_ARCHIVE_DEST_ID's on my STANDBYL are as such
    log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ONLINE_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=standbyl'
    log_archive_dest_2='SERVICE=primaryp ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=primaryp'
    log_archive_dest_3='SERVICE=standbyp ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=standbyp'
    log_archive_dest_4='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(STANDBY_LOGFILES,STANDBY_ROLE) DB_UNIQUE_NAME=standbyl'
    I can see that DEST_4 created the FOREIGN_ARCHIVELOG and is placing the primary archive logs in it ( so not to possibly overwrite the archive logs of STANDBYL itself, now going into DEST_1 )
    Q1. However I assume that ONLINELOG is being created by DEST_3 and the reason it is empty is because the FAL has been able to get all the archive logs from the primary ?
    Q2. Would I ever expect to see anything under DEST_3 in these circumstances eg -
    If the FAL was unable to get an archive from the Primary but was able to get it from STANDBYP - would the retried archive log go into ONLINELOG or FOREIGN_ARCHIVELOG ?
    Q3. If STANDBYP became primary would in STANDBYL, the archive logs go into ONLINELOGS or FOREIGN_ARCHIVELOG ?
    any clarity appreciated
    thanks, Jim

    The purpose of foreign archive logs is as below.
    "Foreign archived redo logs are received by a logical standby database for a LogMiner session. Unlike normal archived logs, foreign archived logs have a different DBID. For this reason, they cannot be backed up or restored on a logical standby database."
    Check the same here for explanation.
    foreignlogRecordSpecifier
    In your case, as you said dest2 and dest3 are out of scope.
    You can now check which destination is creating logs in the specified location using below.
    SELECT DEST_ID,VALID_TYPE,VALID_ROLE,VALID_NOW FROM V$ARCHIVE_DEST;
    Check Data Guard Scenarios for details.
    I believe you have not set up standby redo log in your DR set up. So dest 4 will not produce any logs.
    Actually your dest 1 is producing the foreign logs for a logminer session.
    Hope I have made somewhat clear
    Thank you!!

  • OnlineLog file , Undo file , Control file management on ASM

    What are the commands to add online logfile, undo tablespace file and control file on ASM(Automatic Storage Manager)

    Duplicate thread
    onlineLog file , Undo file , Control file management on ASM

  • Issue in creation of plant related data at receiving server using BD10

    Hi all,
    This is regarding Material master creation using B10.I am using MATMAS05 message type for sending data from one system to another.Data is sent and received successfully.When i go in mm03 i can see all the views created successfully accept views related to PLANT.Please guide to resolve the issue.
    When i entered into Log-
    1)"The field MBEW-BKLAS is defined as a required field; it does not contain an entry".
    2)"No material master data exists for material AB_08.04.09(30) in plant 4001".
    My segemnt is as follows-
    ZMATMAS05                      matmas05
           E1MARAM                        Master material general data (MARA)
               Z1KLART                        KLART----
    My extention
               E1MARA1                        Additional Fields for E1MARAM
               E1MAKTM                        Master material short texts (MAKT)
               E1MARCM                        Master material C segment (MARC)
                   Z1AUSPM                        E1AUSPMDistribution of Classification:----
    My extention
                   E1MARC1                        Additional Fields for E1MARCM
                   E1MARDM                        Master material warehouse/batch segment (MARD)
                   E1MFHMM                        Master material production resource/tool (MFHM)
                   E1MPGDM                        Master material product group
                   E1MPOPM                        Master material forecast parameter
                   E1MPRWM                        Master material forecast value
                   E1MVEGM                        Master material total consumption
                   E1MVEUM                        Master material unplanned consumption
                   E1MKALM                        Master material production version
               E1MARMM                        Master material units of measure (MARM)
               E1MBEWM                        Master material material valuation (MBEW)
               E1MLGNM                        Master material material data per warehouse number (MLGN)
               E1MVKEM                        Master material sales data (MVKE)
               E1MLANM                        Master material tax classification (MLAN)
               E1MTXHM                        Master material long text header
               E1CUCFG                        CU: Configuration data
           E1UPSLINK                      Reference from Object to Superior UPS
    Thanks.
    Edited by: sanu debu on Apr 27, 2009 7:10 PM

    CREATE CONTROLFILE SET DATABASE "NEWDB" NORESETLOGS ARCHIVELOGAlso when you are setting a new database, the option should be RESETLOGS and not NORESETLOGS.
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\ONLINELOG\O1_MF_2_7FK0XKB8_.LOG
    D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_SYSTEM_7FK0SKN0_.DBFWhy underscore(_) at the end of the datafile name. Any specific reason ?

  • Issue in creation of databse with the help of online user managed backup

    Dear dba's
    i am using oracle 11gR2 database in windows server 2003. database is running in ARCHIVE LOG mode.
    i have taken an online backup of all datafile,controlfile and spfile.Then i crated folders in all the locations as required for new database.
    then i registerd the service of new database named as 'newdb' by
    oradim -NEW -SID newdb
    then i created a password file manually in 'oracle_home\database' location.
    i created a new contolfile named as controlfile_01.ctl. the content of controlfile as follows
    STARTUP NOMOUNT
    CREATE CONTROLFILE SET DATABASE "NEWDB" NORESETLOGS ARCHIVELOG
    MAXLOGFILES 16
    MAXLOGMEMBERS 3
    MAXDATAFILES 100
    MAXINSTANCES 8
    MAXLOGHISTORY 292
    LOGFILE
    GROUP 1 (
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\ONLINELOG\O1_MF_1_7FK0XG7B_.LOG',
    'D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\NEWDB\ONLINELOG\O1_MF_1_7FK0XHWB_.LOG'
    ) SIZE 50M,
    GROUP 2 (
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\ONLINELOG\O1_MF_2_7FK0XKB8_.LOG',
    'D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\NEWDB\ONLINELOG\O1_MF_2_7FK0XM0Z_.LOG'
    ) SIZE 50M,
    GROUP 3 (
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\ONLINELOG\O1_MF_3_7FK0XNOZ_.LOG',
    'D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\NEWDB\ONLINELOG\O1_MF_3_7FK0XOWB_.LOG'
    ) SIZE 50M
    DATAFILE
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_SYSTEM_7FK0SKN0_.DBF',
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_SYSAUX_7FK0SKPG_.DBF',
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_UNDOTBS1_7FK0SKTC_.DBF',
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_USERS_7FK0SKWB_.DBF',
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_EXAMPLE_7FK0Z5LK_.DBF',
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\MARSH.DBF',
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\JOMARSH.DBF'
    CHARACTER SET AL32UTF8
    the control file path was registered in pfile also.
    then i brought the database to nomount stage.
    the problem is when i try to mount database it shows following error. anyone can help me to over come from this issue????????
    SQL> startup pfile='D:\app\Administrator\product\11.1.0\db_1\database\INITnewdb.ora' nomount;
    ORACLE instance started.
    Total System Global Area 535662592 bytes
    Fixed Size 1334380 bytes
    Variable Size 301990804 bytes
    Database Buffers 226492416 bytes
    Redo Buffers 5844992 bytes
    SQL> ALTER DATABASE MOUNT;
    ALTER DATABASE MOUNT
    ERROR at line 1:
    ORA-00205: error in identifying control file, check alert log for more info
    the alert massage is:
    ORA-00210: cannot open the specified control file
    ORA-00202: control file: 'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\CONTROLFILE\CONTROLFILE_01.CTL'
    ORA-27048: skgfifi: file header information is invalid
    OSD-04001: invalid logical block size (OS 1413563730)
    Fri Dec 09 13:11:59 2011
    Checker run found 1 new persistent data failures
    ORA-205 signalled during: ALTER DATABASE MOUNT...
    Thanks & Regards,
    John Marshal.A
    Edited by: 895689 on Dec 8, 2011 11:53 PM

    CREATE CONTROLFILE SET DATABASE "NEWDB" NORESETLOGS ARCHIVELOGAlso when you are setting a new database, the option should be RESETLOGS and not NORESETLOGS.
    'D:\APP\ADMINISTRATOR\ORADATA\NEWDB\ONLINELOG\O1_MF_2_7FK0XKB8_.LOG
    D:\APP\ADMINISTRATOR\ORADATA\NEWDB\DATAFILE\O1_MF_SYSTEM_7FK0SKN0_.DBFWhy underscore(_) at the end of the datafile name. Any specific reason ?

  • RMAN duplicate target using set until telling me it can not find data files

    I have RMAN scripts that I use freqently to clone a database from DB (production) to another (non-prod). These work just fine. I use the set until so that I can tell it at what point I want the new DB to be created from the backups of the source.
    I had a request to go back a few weeks and the backup files (I do compressed backups to disk) were on tape. I had my backup person restore my backup directory for the source DB as it looked on a certain day (May 28). I have an 8 day retention policy and so the backup files that were restored showed files all the way back to 5/20 ( includes the point I want to use in my set until clause).
    However, whenever I try to execute the rman clone, it tells me for each datafile:
    RMAN-06023: no backup or copy of datafile 1 found to restore
    Like I said, I am able to do this for a current backup that is on disk. I moved the backup files to the source db server and it works fine. However, from these files restored from tape it errors.
    Here is the RMAN script:
    spool log to c:\temp\clone_CSPROD_CSPRSUM1.log;
    #connect to catalog <catalog info>
    # target is the source and auxiliary is destination
    #connect target <put in source info here>
    #connect auxiliary /
    run {
    allocate auxiliary channel d1 type disk format 'F:\backups\CSPROD\d1\CSPROD_DATA_%s';
    allocate auxiliary channel d2 type disk format 'F:\backups\CSPROD\d2\CSPROD_DATA_%s';
    allocate auxiliary channel d3 type disk format 'F:\backups\CSPROD\d3\CSPROD_DATA_%s';
    allocate auxiliary channel d4 type disk format 'F:\backups\CSPROD\d4\CSPROD_DATA_%s';
    allocate auxiliary channel a1 type disk format 'F:\backups\CSPROD\a1\CSPROD_arch_%s';
    ##Archivelog number get from sql archive log list command
    #set until sequence 831;
    set until time "to_date('2011-05-25 08:00:00', 'YYYY-MM-DD HH24:MI:SS')";
    duplicate target database to CSPRSUM1 nofilenamecheck
    logfile
    group 1('+DATA/CSPRSUM1/onlinelog/redo1a.log', '+FRA/CSPRSUM1/onlinelog/redo1b.log') size 50m,
    group 2('+DATA/CSPRSUM1/onlinelog/redo2a.log', '+FRA/CSPRSUM1/onlinelog/redo2b.log') size 50m,
    group 3('+DATA/CSPRSUM1/onlinelog/redo3a.log', '+FRA/CSPRSUM1/onlinelog/redo3b.log') size 50m;
    Exit
    Here is the output:
    Spooling started in log file: c:\temp\clone_CSPROD_CSPRSUM1.log
    Recovery Manager11.1.0.7.0
    RMAN> #connect catalog <redacted info>>
    2> # target is the source and auxiliary is destination
    3> #connect target <redacted info>
    4> #connect auxiliary /
    5>
    6> run {
    7> allocate auxiliary channel d1 type disk format 'F:\backups\CSPROD\d1\CSPROD_DATA_%s';
    8> allocate auxiliary channel d2 type disk format 'F:\backups\CSPROD\d2\CSPROD_DATA_%s';
    9> allocate auxiliary channel d3 type disk format 'F:\backups\CSPROD\d3\CSPROD_DATA_%s';
    10> allocate auxiliary channel d4 type disk format 'F:\backups\CSPROD\d4\CSPROD_DATA_%s';
    11> allocate auxiliary channel a1 type disk format 'F:\backups\CSPROD\a1\CSPROD_arch_%s';
    12> ##Archivelog number get from sql archive log list command
    13> #set until sequence 831;
    14> set until time "to_date('2011-05-25 08:00:00', 'YYYY-MM-DD HH24:MI:SS')";
    15> duplicate target database to CSPRSUM1 nofilenamecheck
    16> logfile
    17> group 1('+DATA/CSPRSUM1/onlinelog/redo1a.log', '+FRA/CSPRSUM1/onlinelog/redo1b.log') size 50m,
    18> group 2('+DATA/CSPRSUM1/onlinelog/redo2a.log', '+FRA/CSPRSUM1/onlinelog/redo2b.log') size 50m,
    19> group 3('+DATA/CSPRSUM1/onlinelog/redo3a.log', '+FRA/CSPRSUM1/onlinelog/redo3b.log') size 50m;
    20> }
    starting full resync of recovery catalog
    full resync complete
    allocated channel: d1
    channel d1: SID=534 device type=DISK
    allocated channel: d2
    channel d2: SID=533 device type=DISK
    allocated channel: d3
    channel d3: SID=532 device type=DISK
    allocated channel: d4
    channel d4: SID=531 device type=DISK
    allocated channel: a1
    channel a1: SID=530 device type=DISK
    executing command: SET until clause
    Starting Duplicate Db at 15-JUN-11
    RMAN-05529: WARNING: DB_FILE_NAME_CONVERT resulted in invalid ASM names; names changed to disk group only.
    contents of Memory Script:
    set until scn 260398799;
    set newname for datafile 1 to
    "+data";
    set newname for datafile 2 to
    "+data";
    set newname for datafile 3 to
    "+data";
    set newname for datafile 4 to
    "+data";
    set newname for datafile 5 to
    "+data";
    set newname for datafile 6 to
    "+data";
    set newname for datafile 7 to
    "+data";
    set newname for datafile 8 to
    "+data";
    set newname for datafile 9 to
    "+data";
    set newname for datafile 10 to
    "+data";
    set newname for datafile 11 to
    "+data";
    set newname for datafile 12 to
    "+data";
    set newname for datafile 13 to
    "+data";
    set newname for datafile 14 to
    "+data";
    set newname for datafile 15 to
    "+data";
    set newname for datafile 16 to
    "+data";
    set newname for datafile 17 to
    "+data";
    set newname for datafile 18 to
    "+data";
    set newname for datafile 19 to
    "+data";
    set newname for datafile 20 to
    "+data";
    set newname for datafile 21 to
    "+data";
    set newname for datafile 22 to
    "+data";
    set newname for datafile 23 to
    "+data";
    set newname for datafile 24 to
    "+data";
    set newname for datafile 25 to
    "+data";
    set newname for datafile 26 to
    "+data";
    set newname for datafile 27 to
    "+data";
    set newname for datafile 28 to
    "+data";
    set newname for datafile 29 to
    "+data";
    set newname for datafile 30 to
    "+data";
    set newname for datafile 31 to
    "+data";
    set newname for datafile 32 to
    "+data";
    set newname for datafile 33 to
    "+data";
    set newname for datafile 34 to
    "+data";
    set newname for datafile 35 to
    "+data";
    set newname for datafile 36 to
    "+data";
    set newname for datafile 37 to
    "+data";
    set newname for datafile 38 to
    "+data";
    set newname for datafile 39 to
    "+data";
    set newname for datafile 40 to
    "+data";
    set newname for datafile 41 to
    "+data";
    set newname for datafile 42 to
    "+data";
    set newname for datafile 43 to
    "+data";
    set newname for datafile 44 to
    "+data";
    set newname for datafile 45 to
    "+data";
    set newname for datafile 46 to
    "+data";
    set newname for datafile 47 to
    "+data";
    set newname for datafile 48 to
    "+data";
    set newname for datafile 49 to
    "+data";
    set newname for datafile 50 to
    "+data";
    set newname for datafile 51 to
    "+data";
    set newname for datafile 52 to
    "+data";
    set newname for datafile 53 to
    "+data";
    set newname for datafile 54 to
    "+data";
    set newname for datafile 55 to
    "+data";
    set newname for datafile 56 to
    "+data";
    set newname for datafile 57 to
    "+data";
    set newname for datafile 58 to
    "+data";
    set newname for datafile 59 to
    "+data";
    set newname for datafile 60 to
    "+data";
    set newname for datafile 61 to
    "+data";
    set newname for datafile 62 to
    "+data";
    set newname for datafile 63 to
    "+data";
    set newname for datafile 64 to
    "+data";
    set newname for datafile 65 to
    "+data";
    set newname for datafile 66 to
    "+data";
    set newname for datafile 67 to
    "+data";
    set newname for datafile 68 to
    "+data";
    set newname for datafile 69 to
    "+data";
    set newname for datafile 70 to
    "+data";
    set newname for datafile 71 to
    "+data";
    set newname for datafile 72 to
    "+data";
    set newname for datafile 73 to
    "+data";
    set newname for datafile 74 to
    "+data";
    set newname for datafile 75 to
    "+data";
    set newname for datafile 76 to
    "+data";
    set newname for datafile 77 to
    "+data";
    set newname for datafile 78 to
    "+data";
    set newname for datafile 79 to
    "+data";
    set newname for datafile 80 to
    "+data";
    set newname for datafile 81 to
    "+data";
    set newname for datafile 82 to
    "+data";
    set newname for datafile 83 to
    "+data";
    set newname for datafile 84 to
    "+data";
    set newname for datafile 85 to
    "+data";
    set newname for datafile 86 to
    "+data";
    set newname for datafile 87 to
    "+data";
    set newname for datafile 88 to
    "+data";
    set newname for datafile 89 to
    "+data";
    set newname for datafile 90 to
    "+data";
    set newname for datafile 91 to
    "+data";
    set newname for datafile 92 to
    "+data";
    set newname for datafile 93 to
    "+data";
    set newname for datafile 94 to
    "+data";
    set newname for datafile 95 to
    "+data";
    set newname for datafile 96 to
    "+data";
    set newname for datafile 97 to
    "+data";
    set newname for datafile 98 to
    "+data";
    set newname for datafile 99 to
    "+data";
    set newname for datafile 100 to
    "+data";
    set newname for datafile 101 to
    "+data";
    set newname for datafile 102 to
    "+data";
    set newname for datafile 103 to
    "+data";
    set newname for datafile 104 to
    "+data";
    set newname for datafile 105 to
    "+data";
    set newname for datafile 106 to
    "+data";
    set newname for datafile 107 to
    "+data";
    set newname for datafile 108 to
    "+data";
    set newname for datafile 109 to
    "+data";
    set newname for datafile 110 to
    "+data";
    set newname for datafile 111 to
    "+data";
    set newname for datafile 112 to
    "+data";
    set newname for datafile 113 to
    "+data";
    set newname for datafile 114 to
    "+data";
    set newname for datafile 115 to
    "+data";
    set newname for datafile 116 to
    "+data";
    set newname for datafile 117 to
    "+data";
    set newname for datafile 118 to
    "+data";
    set newname for datafile 119 to
    "+data";
    set newname for datafile 120 to
    "+data";
    set newname for datafile 121 to
    "+data";
    set newname for datafile 122 to
    "+data";
    restore
    clone database
    executing Memory Script
    executing command: SET until clause
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    executing command: SET NEWNAME
    Starting restore at 15-JUN-11
    released channel: d1
    released channel: d2
    released channel: d3
    released channel: d4
    released channel: a1
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of Duplicate Db command at 06/15/2011 11:21:42
    RMAN-01005: not all datafiles have backups that can be recovered to scn 260398799
    RMAN-03015: error occurred in stored script Memory Script
    RMAN-06026: some targets not found - aborting restore
    RMAN-06023: no backup or copy of datafile 122 found to restore
    RMAN-06023: no backup or copy of datafile 121 found to restore
    RMAN-06023: no backup or copy of datafile 120 found to restore
    RMAN-06023: no backup or copy of datafile 119 found to restore
    RMAN-06023: no backup or copy of datafile 118 found to restore
    RMAN-06023: no backup or copy of datafile 117 found to restore
    RMAN-06023: no backup or copy of datafile 116 found to restore
    RMAN-06023: no backup or copy of datafile 115 found to restore
    RMAN-06023: no backup or copy of datafile 114 found to restore
    RMAN-06023: no backup or copy of datafile 113 found to restore
    RMAN-06023: no backup or copy of datafile 112 found to restore
    RMAN-06023: no backup or copy of datafile 111 found to restore
    RMAN-06023: no backup or copy of datafile 110 found to restore
    RMAN-06023: no backup or copy of datafile 109 found to restore
    RMAN-06023: no backup or copy of datafile 108 found to restore
    RMAN-06023: no backup or copy of datafile 107 found to restore
    RMAN-06023: no backup or copy of datafile 106 found to restore
    RMAN-06023: no backup or copy of datafile 105 found to restore
    RMAN-06023: no backup or copy of datafile 104 found to restore
    RMAN-06023: no backup or copy of datafile 103 found to restore
    RMAN-06023: no backup or copy of datafile 102 found to restore
    RMAN-06023: no backup or copy of datafile 101 found to restore
    RMAN-06023: no backup or copy of datafile 100 found to restore
    RMAN-06023: no backup or copy of datafile 99 found to restore
    RMAN-06023: no backup or copy of datafile 98 found to restore
    RMAN-06023: no backup or copy of datafile 97 found to restore
    RMAN-06023: no backup or copy of datafile 96 found to restore
    RMAN-06023: no backup or copy of datafile 95 found to restore
    RMAN-06023: no backup or copy of datafile 94 found to restore
    RMAN-06023: no backup or copy of datafile 93 found to restore
    RMAN-06023: no backup or copy of datafile 92 found to restore
    RMAN-06023: no backup or copy of datafile 91 found to restore
    RMAN-06023: no backup or copy of datafile 90 found to restore
    RMAN-06023: no backup or copy of datafile 89 found to restore
    RMAN-06023: no backup or copy of datafile 88 found to restore
    RMAN-06023: no backup or copy of datafile 87 found to restore
    RMAN-06023: no backup or copy of datafile 86 found to restore
    RMAN-06023: no backup or copy of datafile 85 found to restore
    RMAN-06023: no backup or copy of datafile 84 found to restore
    RMAN-06023: no backup or copy of datafile 83 found to restore
    RMAN-06023: no backup or copy of datafile 82 found to restore
    RMAN-06023: no backup or copy of datafile 81 found to restore
    RMAN-06023: no backup or copy of datafile 80 found to restore
    RMAN-06023: no backup or copy of datafile 79 found to restore
    RMAN-06023: no backup or copy of datafile 78 found to restore
    RMAN-06023: no backup or copy of datafile 77 found to restore
    RMAN-06023: no backup or copy of datafile 76 found to restore
    RMAN-06023: no backup or copy of datafile 75 found to restore
    RMAN-06023: no backup or copy of datafile 74 found to restore
    RMAN-06023: no backup or copy of datafile 73 found to restore
    RMAN-06023: no backup or copy of datafile 72 found to restore
    RMAN-06023: no backup or copy of datafile 71 found to restore
    RMAN-06023: no backup or copy of datafile 70 found to restore
    RMAN-06023: no backup or copy of datafile 69 found to restore
    RMAN-06023: no backup or copy of datafile 68 found to restore
    RMAN-06023: no backup or copy of datafile 67 found to restore
    RMAN-06023: no backup or copy of datafile 66 found to restore
    RMAN-06023: no backup or copy of datafile 65 found to restore
    RMAN-06023: no backup or copy of datafile 64 found to restore
    RMAN-06023: no backup or copy of datafile 63 found to restore
    RMAN-06023: no backup or copy of datafile 62 found to restore
    RMAN-06023: no backup or copy of datafile 61 found to restore
    RMAN-06023: no backup or copy of datafile 60 found to restore
    RMAN-06023: no backup or copy of datafile 59 found to restore
    RMAN-06023: no backup or copy of datafile 58 found to restore
    RMAN-06023: no backup or copy of datafile 57 found to restore
    RMAN-06023: no backup or copy of datafile 56 found to restore
    RMAN-06023: no backup or copy of datafile 55 found to restore
    RMAN-06023: no backup or copy of datafile 54 found to restore
    RMAN-06023: no backup or copy of datafile 53 found to restore
    RMAN-06023: no backup or copy of datafile 52 found to restore
    RMAN-06023: no backup or copy of datafile 51 found to restore
    RMAN-06023: no backup or copy of datafile 50 found to restore
    RMAN-06023: no backup or copy of datafile 49 found to restore
    RMAN-06023: no backup or copy of datafile 48 found to restore
    RMAN-06023: no backup or copy of datafile 47 found to restore
    RMAN-06023: no backup or copy of datafile 46 found to restore
    RMAN-06023: no backup or copy of datafile 45 found to restore
    RMAN-06023: no backup or copy of datafile 44 found to restore
    RMAN-06023: no backup or copy of datafile 43 found to restore
    RMAN-06023: no backup or copy of datafile 42 found to restore
    RMAN-06023: no backup or copy of datafile 41 found to restore
    RMAN-06023: no backup or copy of datafile 40 found to restore
    RMAN-06023: no backup or copy of datafile 39 found to restore
    RMAN-06023: no backup or copy of datafile 38 found to restore
    RMAN-06023: no backup or copy of datafile 37 found to restore
    RMAN-06023: no backup or copy of datafile 36 found to restore
    RMAN-06023: no backup or copy of datafile 35 found to restore
    RMAN-06023: no backup or copy of datafile 34 found to restore
    RMAN-06023: no backup or copy of datafile 33 found to restore
    RMAN-06023: no backup or copy of datafile 32 found to restore
    RMAN-06023: no backup or copy of datafile 31 found to restore
    RMAN-06023: no backup or copy of datafile 30 found to restore
    RMAN-06023: no backup or copy of datafile 29 found to restore
    RMAN-06023: no backup or copy of datafile 28 found to restore
    RMAN-06023: no backup or copy of datafile 27 found to restore
    RMAN-06023: no backup or copy of datafile 26 found to restore
    RMAN-06023: no backup or copy of datafile 25 found to restore
    RMAN-06023: no backup or copy of datafile 24 found to restore
    RMAN-06023: no backup or copy of datafile 23 found to restore
    RMAN-06023: no backup or copy of datafile 22 found to restore
    RMAN-06023: no backup or copy of datafile 21 found to restore
    RMAN-06023: no backup or copy of datafile 20 found to restore
    RMAN-06023: no backup or copy of datafile 19 found to restore
    RMAN-06023: no backup or copy of datafile 18 found to restore
    RMAN-06023: no backup or copy of datafile 17 found to restore
    RMAN-06023: no backup or copy of datafile 16 found to restore
    RMAN-06023: no backup or copy of datafile 15 found to restore
    RMAN-06023: no backup or copy of datafile 14 found to restore
    RMAN-06023: no backup or copy of datafile 13 found to restore
    RMAN-06023: no backup or copy of datafile 12 found to restore
    RMAN-06023: no backup or copy of datafile 11 found to restore
    RMAN-06023: no backup or copy of datafile 10 found to restore
    RMAN-06023: no backup or copy of datafile 9 found to restore
    RMAN-06023: no backup or copy of datafile 8 found to restore
    RMAN-06023: no backup or copy of datafile 7 found to restore
    RMAN-06023: no backup or copy of datafile 6 found to restore
    RMAN-06023: no backup or copy of datafile 5 found to restore
    RMAN-06023: no backup or copy of datafile 4 found to restore
    RMAN-06023: no backup or copy of datafile 3 found to restore
    RMAN-06023: no backup or copy of datafile 2 found to restore
    RMAN-06023: no backup or copy of datafile 1 found to restore
    RMAN> Exit
    Recovery Manager complete.
    Edited by: kerrygm on Jun 15, 2011 11:36 AM

    Maybe making progress.
    I got into RMAN and did a List backup command. In looking at the output, I see that it does have the 5/20 backup showing. Here is part of the output:
    BS Key Type LV Size Device Type Elapsed Time Completion Time
    965689 Full 19.86M DISK 00:00:01 16-JUN-11
    BP Key: 967942 Status: AVAILABLE Compressed: NO Tag: TAG20110616T231223
    Piece Name: F:\BACKUPS\CSPROD\C-368871413-20110616-01
    SPFILE Included: Modification time: 16-JUN-11
    SPFILE db_unique_name: CSPROD
    Control File Included: Ckp SCN: 369596068 Ckp time: 16-JUN-11
    BS Key Size Device Type Elapsed Time Completion Time
    966218 403.84M DISK 00:00:00 20-MAY-11
    BP Key: 967842 Status: AVAILABLE Compressed: YES Tag: TAG20110520T231012
    Piece Name: F:\BACKUPS\CSPROD\A1\CSPROD_ARCH_5844
    List of Archived Logs in backup set 966218
    Thrd Seq Low SCN Low Time Next SCN Next Time
    1 2112 237027300 20-MAY-11 237046947 20-MAY-11
    1 2113 *237046947* 20-MAY-11 237059284 20-MAY-11
    1 2114 237059284 20-MAY-11 237216514 20-MAY-11
    1 2115 237216514 20-MAY-11 237709545 20-MAY-11
    1 2116 237709545 20-MAY-11 237722825 20-MAY-11
    1 2117 237722825 20-MAY-11 237730431 20-MAY-11
    1 2118 237730431 20-MAY-11 237860199 20-MAY-11
    I then changed my rman script to use the *237046947* SCN number that says it is from 5/20. I them kicked off my rman script and got the following result.
    Spooling started in log file: c:\temp\clone_CSPROD_CSPRSUM1.log
    Recovery Manager11.1.0.7.0
    RMAN> #connect catalog 'rmancat/rmancat@rmancat';
    2> # target is the source and auxiliary is destination
    3> #connect target sys/<pwd>@csprod
    4> #connect auxiliary /
    5>
    6> run {
    7> allocate auxiliary channel d1 type disk format 'F:\backups\CSPROD\d1\CSPROD_DATA_%s';
    8> allocate auxiliary channel d2 type disk format 'F:\backups\CSPROD\d2\CSPROD_DATA_%s';
    9> allocate auxiliary channel d3 type disk format 'F:\backups\CSPROD\d3\CSPROD_DATA_%s';
    10> allocate auxiliary channel d4 type disk format 'F:\backups\CSPROD\d4\CSPROD_DATA_%s';
    11> allocate auxiliary channel a1 type disk format 'F:\backups\CSPROD\a1\CSPROD_arch_%s';
    12> ##Archivelog number get from sql archive log list command
    13> set until sequence 237046947;
    14> ##set until time "to_date('2011-05-20 08:00:00', 'YYYY-MM-DD HH24:MI:SS')";
    15> duplicate target database to CSPRSUM1 nofilenamecheck
    16> logfile
    17> group 1('+DATA/CSPRSUM1/onlinelog/redo1a.log', '+FRA/CSPRSUM1/onlinelog/redo1b.log') size 50m,
    18> group 2('+DATA/CSPRSUM1/onlinelog/redo2a.log', '+FRA/CSPRSUM1/onlinelog/redo2b.log') size 50m,
    19> group 3('+DATA/CSPRSUM1/onlinelog/redo3a.log', '+FRA/CSPRSUM1/onlinelog/redo3b.log') size 50m;
    20> }
    allocated channel: d1
    channel d1: SID=532 device type=DISK
    allocated channel: d2
    channel d2: SID=533 device type=DISK
    allocated channel: d3
    channel d3: SID=531 device type=DISK
    allocated channel: d4
    channel d4: SID=555 device type=DISK
    allocated channel: a1
    channel a1: SID=534 device type=DISK
    executing command: SET until clause
    Starting Duplicate Db at 17-JUN-11
    RMAN-05529: WARNING: DB_FILE_NAME_CONVERT resulted in invalid ASM names; names changed to disk group only.
    contents of Memory Script:
    set until scn 373021170;
    set newname for datafile 1 to
    "+data";
    set newname for datafile 2 to
    Starting restore at 17-JUN-11
    channel d1: starting datafile backup set restore
    channel d1: specifying datafile(s) to restore from backup set
    channel d1: restoring datafile 00003 to +DATA
    channel d2: restoring datafile 00122 to +DATA
    channel d2: reading from backup piece F:\BACKUPS\CSPROD\D4\CSPROD_DATA_6149
    channel d3: starting datafile backup set restore
    channel d3: specifying datafile(s) to restore from backup set
    channel d3: restoring datafile 00002 to +DATA
    channel d3: restoring datafile 00020 to +DATA
    channel d4: restoring datafile 00120 to +DATA
    channel d4: reading from backup piece F:\BACKUPS\CSPROD\D2\CSPROD_DATA_6147
    channel d1: piece handle=F:\BACKUPS\CSPROD\D3\CSPROD_DATA_6148 tag=TAG20110616T230013
    channel d1: restored backup piece 1
    channel d1: restore complete, elapsed time: 00:01:15
    channel d4: piece handle=F:\BACKUPS\CSPROD\D2\CSPROD_DATA_6147 tag=TAG20110616T230013
    channel d4: restored backup piece 1
    channel d4: restore complete, elapsed time: 00:03:15
    channel d2: piece handle=F:\BACKUPS\CSPROD\D4\CSPROD_DATA_6149 tag=TAG20110616T230013
    channel d2: restored backup piece 1
    channel d2: restore complete, elapsed time: 00:03:35
    channel d3: piece handle=F:\BACKUPS\CSPROD\D1\CSPROD_DATA_6146 tag=TAG20110616T230013
    channel d3: restored backup piece 1
    channel d3: restore complete, elapsed time: 00:04:55
    Finished restore at 17-JUN-11
    sql statement: CREATE CONTROLFILE REUSE SET DATABASE "CSPRSUM1" RESETLOGS ARCHIVELOG
    MAXLOGFILES 23
    MAXLOGMEMBERS 3
    MAXDATAFILES 1021
    MAXINSTANCES 8
    MAXLOGHISTORY 584
    LOGFILE
    GROUP 1 ( '+DATA/CSPRSUM1/onlinelog/redo1a.log', '+FRA/CSPRSUM1/onlinelog/redo1b.log' ) SIZE 50 M ,
    GROUP 2 ( '+DATA/CSPRSUM1/onlinelog/redo2a.log', '+FRA/CSPRSUM1/onlinelog/redo2b.log' ) SIZE 50 M ,
    GROUP 3 ( '+DATA/CSPRSUM1/onlinelog/redo3a.log', '+FRA/CSPRSUM1/onlinelog/redo3b.log' ) SIZE 50 M
    DATAFILE
    '+DATA/csprsum1/datafile/system.1535.754067379'
    CHARACTER SET WE8MSWIN1252
    contents of Memory Script:
    switch clone datafile all;
    executing Memory Script
    datafile 2 switched to datafile copy
    input datafile copy RECID=1 STAMP=754067676 file name=+DATA/csprsum1/datafile/sysaux.1536.754067379
    datafile 121 switched to datafile copy
    input datafile copy RECID=120 STAMP=754067677 file name=+DATA/csprsum1/datafile/waapp.1653.754067403
    datafile 122 switched to datafile copy
    input datafile copy RECID=121 STAMP=754067677 file name=+DATA/csprsum1/datafile/cu_custom.1549.754067381
    contents of Memory Script:
    set until scn 373021170;
    recover
    clone database
    delete archivelog
    executing Memory Script
    executing command: SET until clause
    Starting recover at 17-JUN-11
    starting media recovery
    Oracle Error:
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01194: file 1 needs more recovery to be consistent
    ORA-01110: data file 1: '+DATA/csprsum1/datafile/system.1535.754067379'
    released channel: d1
    released channel: d2
    released channel: d3
    released channel: d4
    released channel: a1
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of Duplicate Db command at 06/17/2011 15:14:54
    RMAN-03015: error occurred in stored script Memory Script
    RMAN-06053: unable to perform media recovery because of missing log
    RMAN-06025: no backup of archived log for thread 1 with sequence 3818 and starting SCN of 372868482 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3817 and starting SCN of 372685133 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3816 and starting SCN of 372593528 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3815 and starting SCN of 372374385 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3814 and starting SCN of 372325053 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3813 and starting SCN of 372316138 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3812 and starting SCN of 372249619 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3811 and starting SCN of 371775981 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3810 and starting SCN of 371643855 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3809 and starting SCN of 371614442 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3808 and starting SCN of 371432892 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3807 and starting SCN of 371121955 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3806 and starting SCN of 371047786 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3805 and starting SCN of 371029095 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3804 and starting SCN of 371018252 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3803 and starting SCN of 370947755 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3802 and starting SCN of 370857440 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3801 and starting SCN of 370814417 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3800 and starting SCN of 370797061 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3799 and starting SCN of 370756569 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3798 and starting SCN of 370746833 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3797 and starting SCN of 370746693 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3796 and starting SCN of 370746568 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3795 and starting SCN of 370746068 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3794 and starting SCN of 370745451 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3793 and starting SCN of 370733767 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3792 and starting SCN of 370674629 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3791 and starting SCN of 370501026 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3790 and starting SCN of 370498513 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3789 and starting SCN of 370497977 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3788 and starting SCN of 370497635 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3787 and starting SCN of 370497319 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3786 and starting SCN of 370496938 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3785 and starting SCN of 370495428 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3784 and starting SCN of 370491173 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3783 and starting SCN of 370390074 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3782 and starting SCN of 370385574 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3781 and starting SCN of 370385310 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3780 and starting SCN of 370385044 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3779 and starting SCN of 370368486 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3778 and starting SCN of 370333960 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3777 and starting SCN of 370330980 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3776 and starting SCN of 370327876 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3775 and starting SCN of 370174433 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3774 and starting SCN of 370148373 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3773 and starting SCN of 370147821 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3772 and starting SCN of 370147623 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3771 and starting SCN of 370141528 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3770 and starting SCN of 369711271 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3769 and starting SCN of 369621694 found to restore
    RMAN-06025: no backup of archived log for thread 1 with sequence 3768 and starting SCN of 369595614 found to restore
    RMAN> Exit
    Recovery Manager complete.
    What I am not sure is if the SCN I used in my set until sequence command was from 5/20, when I look at the backup pieces that it is reading to restore from, they appear to be from backup files taken from June.
    (e.g channel d1: reading from backup piece F:\BACKUPS\CSPROD\D3\CSPROD_DATA_6148 -- this is a 6/16 backup file).
    Also, as you can see it was not able to do a media recovery.
    Thanks in advance for your help.

  • Database is not starting after RAC installation

    Hi All,
    I have installed 10g R2 2-node RAC on RHEL4 without any problem. But after the installation the database is not starting and i'm not able to connect. It is showing the below error..
    ERROR:
    ORA-01034: ORACLE not available
    ORA-27101: shared memory realm does not exist
    Linux Error: 2: No such file or directory
    Can anybody help what might be the problem?
    Thanks,
    Praveen

    Hi,
    I'm giving the contents of alter log file from the first node...
    Mon Feb 19 16:45:59 2007
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Interface type 1 eth1 10.250.90.0 configured from OCR for use as a cluster interconnect
    Interface type 1 eth0 10.250.90.0 configured from OCR for use as a public interface
    Shared memory segment for instance monitoring created
    Picked latch-free SCN scheme 2
    Autotune of undo retention is turned on.
    IMODE=BR
    ILAT =36
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.2.0.
    System parameters with non-default values:
    processes = 300
    sessions = 335
    sga_max_size = 1073741824
    sga_target = 1073741824
    db_block_size = 8192
    compatible = 10.2.0.2.0
    log_archive_dest_1 = LOCATION=+BSA_DATA1/bsaprod/
    log_archive_format = %t_%s_%r.dbf
    db_file_multiblock_read_count= 16
    cluster_database_instances= 1
    db_create_file_dest = +ORCL_DATA1
    instance_number = 1
    undo_management = AUTO
    undo_tablespace = UNDOTBS1
    remote_login_passwordfile= EXCLUSIVE
    db_domain =
    dispatchers = (PROTOCOL=TCP) (SERVICE=bsaprodXDB)
    remote_listener = LISTENERS_BSAPROD
    job_queue_processes = 10
    background_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/bdump
    user_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/udump
    core_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/cdump
    audit_file_dest = /u01/app/oracle/oracle/admin/bsaprod/adump
    db_name = bsaprod
    open_cursors = 300
    pga_aggregate_target = 262144000
    Cluster communication is configured to use the following interface(s) for this instance
    10.250.90.107
    Mon Feb 19 16:46:00 2007
    cluster interconnect IPC version:Oracle UDP/IP (generic)
    IPC Vendor 1 proto 2
    PMON started with pid=2, OS id=31282
    DIAG started with pid=3, OS id=31284
    PSP0 started with pid=4, OS id=31286
    LMON started with pid=5, OS id=31300
    LMD0 started with pid=6, OS id=31315
    MMAN started with pid=7, OS id=31317
    DBW0 started with pid=8, OS id=31319
    LGWR started with pid=9, OS id=31321
    CKPT started with pid=10, OS id=31323
    SMON started with pid=11, OS id=31325
    RECO started with pid=12, OS id=31327
    CJQ0 started with pid=13, OS id=31329
    MMON started with pid=14, OS id=31331
    Mon Feb 19 16:46:00 2007
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    MMNL started with pid=15, OS id=31333
    Mon Feb 19 16:46:00 2007
    starting up 1 shared server(s) ...
    Mon Feb 19 16:46:00 2007
    lmon registered with NM - instance id 1 (internal mem no 0)
    Mon Feb 19 16:46:01 2007
    Reconfiguration started (old inc 0, new inc 2)
    List of nodes:
    0
    Global Resource Directory frozen
    * allocate domain 0, invalid = TRUE
    Communication channels reestablished
    Master broadcasted resource hash value bitmaps
    Non-local Process blocks cleaned out
    Resources and enqueues cleaned out
    Resources remastered 0
    Set master node info
    Submitted all remote-enqueue requests
    Dwn-cvts replayed, VALBLKs dubious
    All grantable enqueues granted
    Post SMON to start 1st pass IR
    Submitted all GCS remote-cache requests
    Post SMON to start 1st pass IR
    Reconfiguration complete
    Mon Feb 19 16:46:01 2007
    CREATE DATABASE "bsaprod"
    MAXINSTANCES 32
    MAXLOGHISTORY 1
    MAXLOGFILES 192
    MAXLOGMEMBERS 3
    MAXDATAFILES 1024
    DATAFILE SIZE 300M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL
    SYSAUX DATAFILE SIZE 120M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    SMALLFILE DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE SIZE 20M AUTOEXTEND ON NEXT 640K MAXSIZE UNLIMITED
    SMALLFILE UNDO TABLESPACE "UNDOTBS1" DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    CHARACTER SET WE8ISO8859P1
    NATIONAL CHARACTER SET AL16UTF16
    LOGFILE GROUP 1 SIZE 51200K,
    GROUP 2 SIZE 51200K
    USER SYS IDENTIFIED BY *USER SYSTEM IDENTIFIED BY
    Mon Feb 19 16:46:01 2007
    Starting background process ASMB
    ASMB started with pid=19, OS id=31365
    Starting background process RBAL
    RBAL started with pid=20, OS id=31369
    Loaded ASM Library - Generic Linux, version 2.0.2 (KABI_V2) library for asmlib interface
    Mon Feb 19 16:46:05 2007
    SUCCESS: diskgroup ORCL_DATA1 was mounted
    SUCCESS: diskgroup ORCL_DATA1 was dismounted
    Mon Feb 19 16:46:06 2007
    SUCCESS: diskgroup ORCL_DATA1 was mounted
    Mon Feb 19 16:46:06 2007
    Database mounted in Exclusive Mode
    Mon Feb 19 16:46:09 2007
    Successful mount of redo thread 1, with mount id 3309011657
    Assigning activation ID 3309011657 (0xc53b82c9)
    Thread 1 opened at log sequence 1
    Current log# 1 seq# 1 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Successful open of redo thread 1
    Mon Feb 19 16:46:09 2007
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Mon Feb 19 16:46:09 2007
    SMON: enabling cache recovery
    Mon Feb 19 16:46:09 2007
    create tablespace SYSTEM datafile SIZE 300M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL online
    Mon Feb 19 16:46:20 2007
    Completed: create tablespace SYSTEM datafile SIZE 300M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL online
    Mon Feb 19 16:46:20 2007
    create rollback segment SYSTEM tablespace SYSTEM
    storage (initial 50K next 50K)
    Completed: create rollback segment SYSTEM tablespace SYSTEM
    storage (initial 50K next 50K)
    Mon Feb 19 16:46:24 2007
    CREATE SMALLFILE UNDO TABLESPACE UNDOTBS1 DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    Mon Feb 19 16:46:31 2007
    Successfully onlined Undo Tablespace 1.
    Completed: CREATE SMALLFILE UNDO TABLESPACE UNDOTBS1 DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    Mon Feb 19 16:46:31 2007
    create tablespace SYSAUX datafile SIZE 120M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO online
    Completed: create tablespace SYSAUX datafile SIZE 120M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO online
    Mon Feb 19 16:46:36 2007
    CREATE SMALLFILE TEMPORARY TABLESPACE TEMP TEMPFILE SIZE 20M AUTOEXTEND ON NEXT 640K MAXSIZE UNLIMITED
    Completed: CREATE SMALLFILE TEMPORARY TABLESPACE TEMP TEMPFILE SIZE 20M AUTOEXTEND ON NEXT 640K MAXSIZE UNLIMITED
    Mon Feb 19 16:46:36 2007
    ALTER DATABASE DEFAULT TEMPORARY TABLESPACE TEMP
    Completed: ALTER DATABASE DEFAULT TEMPORARY TABLESPACE TEMP
    Mon Feb 19 16:46:36 2007
    ALTER DATABASE DEFAULT TABLESPACE SYSTEM
    Completed: ALTER DATABASE DEFAULT TABLESPACE SYSTEM
    Mon Feb 19 16:46:38 2007
    SMON: enabling tx recovery
    Mon Feb 19 16:46:39 2007
    Threshold validation cannot be done before catproc is loaded.
    Threshold validation cannot be done before catproc is loaded.
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=22, OS id=32303
    Mon Feb 19 16:46:39 2007
    Completed: CREATE DATABASE "bsaprod"
    MAXINSTANCES 32
    MAXLOGHISTORY 1
    MAXLOGFILES 192
    MAXLOGMEMBERS 3
    MAXDATAFILES 1024
    DATAFILE SIZE 300M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL
    SYSAUX DATAFILE SIZE 120M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    SMALLFILE DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE SIZE 20M AUTOEXTEND ON NEXT 640K MAXSIZE UNLIMITED
    SMALLFILE UNDO TABLESPACE "UNDOTBS1" DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    CHARACTER SET WE8ISO8859P1
    NATIONAL CHARACTER SET AL16UTF16
    LOGFILE GROUP 1 SIZE 51200K,
    GROUP 2 SIZE 51200K
    USER SYS IDENTIFIED BY *USER SYSTEM IDENTIFIED BY
    Mon Feb 19 16:46:39 2007
    CREATE SMALLFILE UNDO TABLESPACE "UNDOTBS2" DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    Completed: CREATE SMALLFILE UNDO TABLESPACE "UNDOTBS2" DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    Mon Feb 19 16:46:46 2007
    CREATE SMALLFILE TABLESPACE "USERS" LOGGING DATAFILE SIZE 5M AUTOEXTEND ON NEXT 1280K MAXSIZE UNLIMITED EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO
    Completed: CREATE SMALLFILE TABLESPACE "USERS" LOGGING DATAFILE SIZE 5M AUTOEXTEND ON NEXT 1280K MAXSIZE UNLIMITED EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO
    Mon Feb 19 16:46:46 2007
    ALTER DATABASE DEFAULT TABLESPACE "USERS"
    Completed: ALTER DATABASE DEFAULT TABLESPACE "USERS"
    Mon Feb 19 16:47:05 2007
    Thread 1 advanced to log sequence 2
    Current log# 2 seq# 2 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:47:34 2007
    Thread 1 cannot allocate new log, sequence 3
    Checkpoint not complete
    Current log# 2 seq# 2 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 3
    Current log# 1 seq# 3 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:48:10 2007
    Thread 1 cannot allocate new log, sequence 4
    Checkpoint not complete
    Current log# 1 seq# 3 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 4
    Current log# 2 seq# 4 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:49:05 2007
    Thread 1 cannot allocate new log, sequence 5
    Checkpoint not complete
    Current log# 2 seq# 4 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 5
    Current log# 1 seq# 5 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:50:19 2007
    Thread 1 cannot allocate new log, sequence 6
    Checkpoint not complete
    Current log# 1 seq# 5 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 6
    Current log# 2 seq# 6 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:51:00 2007
    Thread 1 cannot allocate new log, sequence 7
    Checkpoint not complete
    Current log# 2 seq# 6 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 7
    Current log# 1 seq# 7 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:51:54 2007
    Thread 1 cannot allocate new log, sequence 8
    Checkpoint not complete
    Current log# 1 seq# 7 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 8
    Current log# 2 seq# 8 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:55:32 2007
    Thread 1 cannot allocate new log, sequence 9
    Checkpoint not complete
    Current log# 2 seq# 8 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 9
    Current log# 1 seq# 9 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:56:35 2007
    Thread 1 cannot allocate new log, sequence 10
    Checkpoint not complete
    Current log# 1 seq# 9 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 10
    Current log# 2 seq# 10 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:56:52 2007
    Thread 1 cannot allocate new log, sequence 11
    Checkpoint not complete
    Current log# 2 seq# 10 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 11
    Current log# 1 seq# 11 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:57:04 2007
    Thread 1 cannot allocate new log, sequence 12
    Checkpoint not complete
    Current log# 1 seq# 11 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 12
    Current log# 2 seq# 12 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:57:14 2007
    Thread 1 cannot allocate new log, sequence 13
    Checkpoint not complete
    Current log# 2 seq# 12 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 13
    Current log# 1 seq# 13 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:57:31 2007
    Thread 1 cannot allocate new log, sequence 14
    Checkpoint not complete
    Current log# 1 seq# 13 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 14
    Current log# 2 seq# 14 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 16:58:13 2007
    Thread 1 cannot allocate new log, sequence 15
    Checkpoint not complete
    Current log# 2 seq# 14 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 15
    Current log# 1 seq# 15 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 16:59:42 2007
    Thread 1 cannot allocate new log, sequence 16
    Checkpoint not complete
    Current log# 1 seq# 15 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 16
    Current log# 2 seq# 16 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:00:33 2007
    Thread 1 cannot allocate new log, sequence 17
    Checkpoint not complete
    Current log# 2 seq# 16 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 17
    Current log# 1 seq# 17 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:01:49 2007
    Thread 1 cannot allocate new log, sequence 18
    Checkpoint not complete
    Current log# 1 seq# 17 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 18
    Current log# 2 seq# 18 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:02:23 2007
    Thread 1 cannot allocate new log, sequence 19
    Checkpoint not complete
    Current log# 2 seq# 18 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 19
    Current log# 1 seq# 19 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:04:06 2007
    Thread 1 cannot allocate new log, sequence 20
    Checkpoint not complete
    Current log# 1 seq# 19 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 20
    Current log# 2 seq# 20 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:05:22 2007
    Thread 1 cannot allocate new log, sequence 21
    Checkpoint not complete
    Current log# 2 seq# 20 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 21
    Current log# 1 seq# 21 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:06:22 2007
    Thread 1 cannot allocate new log, sequence 22
    Checkpoint not complete
    Current log# 1 seq# 21 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 22
    Current log# 2 seq# 22 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:08:42 2007
    Thread 1 cannot allocate new log, sequence 23
    Checkpoint not complete
    Current log# 2 seq# 22 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 23
    Current log# 1 seq# 23 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:09:43 2007
    Thread 1 cannot allocate new log, sequence 24
    Checkpoint not complete
    Current log# 1 seq# 23 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 24
    Current log# 2 seq# 24 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:10:42 2007
    Thread 1 cannot allocate new log, sequence 25
    Checkpoint not complete
    Current log# 2 seq# 24 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 25
    Current log# 1 seq# 25 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:11:31 2007
    Thread 1 cannot allocate new log, sequence 26
    Checkpoint not complete
    Current log# 1 seq# 25 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 26
    Current log# 2 seq# 26 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:12:01 2007
    Thread 1 cannot allocate new log, sequence 27
    Checkpoint not complete
    Current log# 2 seq# 26 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 27
    Current log# 1 seq# 27 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:13:12 2007
    Thread 1 cannot allocate new log, sequence 28
    Checkpoint not complete
    Current log# 1 seq# 27 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 28
    Current log# 2 seq# 28 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:14:05 2007
    Thread 1 cannot allocate new log, sequence 29
    Checkpoint not complete
    Current log# 2 seq# 28 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 29
    Current log# 1 seq# 29 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:14:57 2007
    Thread 1 cannot allocate new log, sequence 30
    Checkpoint not complete
    Current log# 1 seq# 29 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Thread 1 advanced to log sequence 30
    Current log# 2 seq# 30 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Mon Feb 19 17:16:29 2007
    Thread 1 cannot allocate new log, sequence 31
    Checkpoint not complete
    Current log# 2 seq# 30 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_2.258.614969167
    Thread 1 advanced to log sequence 31
    Current log# 1 seq# 31 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Mon Feb 19 17:17:54 2007
    Starting background process EMN0
    EMN0 started with pid=25, OS id=15651
    Mon Feb 19 17:17:54 2007
    Shutting down instance: further logons disabled
    Mon Feb 19 17:17:55 2007
    Stopping background process QMNC
    Mon Feb 19 17:17:55 2007
    Stopping background process CJQ0
    Mon Feb 19 17:17:57 2007
    Stopping background process MMNL
    Mon Feb 19 17:17:58 2007
    Stopping background process MMON
    Mon Feb 19 17:17:59 2007
    Shutting down instance (immediate)
    License high water mark = 1
    Mon Feb 19 17:17:59 2007
    Stopping Job queue slave processes
    Mon Feb 19 17:17:59 2007
    Job queue slave processes stopped
    All dispatchers and shared servers shutdown
    Mon Feb 19 17:18:01 2007
    ALTER DATABASE CLOSE NORMAL
    Mon Feb 19 17:18:01 2007
    SMON: disabling tx recovery
    SMON: disabling cache recovery
    Mon Feb 19 17:18:03 2007
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    Thread 1 closed at log sequence 31
    Successful close of redo thread 1
    Mon Feb 19 17:18:04 2007
    Completed: ALTER DATABASE CLOSE NORMAL
    Mon Feb 19 17:18:04 2007
    ALTER DATABASE DISMOUNT
    Mon Feb 19 17:18:04 2007
    SUCCESS: diskgroup ORCL_DATA1 was dismounted
    Mon Feb 19 17:18:04 2007
    Completed: ALTER DATABASE DISMOUNT
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    Mon Feb 19 17:18:09 2007
    freeing rdom 0
    Mon Feb 19 17:18:12 2007
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Interface type 1 eth1 10.250.90.0 configured from OCR for use as a cluster interconnect
    Interface type 1 eth0 10.250.90.0 configured from OCR for use as a public interface
    Picked latch-free SCN scheme 2
    Autotune of undo retention is turned on.
    IMODE=BR
    ILAT =36
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.2.0.
    System parameters with non-default values:
    processes = 300
    sessions = 335
    sga_max_size = 1073741824
    sga_target = 1073741824
    control_files = +ORCL_DATA1/bsaprod/controlfile/current.256.614969167
    db_block_size = 8192
    compatible = 10.2.0.2.0
    log_archive_dest_1 = LOCATION=+BSA_DATA1/bsaprod/
    log_archive_format = %t_%s_%r.dbf
    db_file_multiblock_read_count= 16
    cluster_database_instances= 1
    db_create_file_dest = +ORCL_DATA1
    instance_number = 1
    undo_management = AUTO
    undo_tablespace = UNDOTBS1
    remote_login_passwordfile= EXCLUSIVE
    db_domain =
    dispatchers = (PROTOCOL=TCP) (SERVICE=bsaprodXDB)
    remote_listener = LISTENERS_BSAPROD
    job_queue_processes = 10
    background_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/bdump
    user_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/udump
    core_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/cdump
    audit_file_dest = /u01/app/oracle/oracle/admin/bsaprod/adump
    db_name = bsaprod
    open_cursors = 300
    pga_aggregate_target = 262144000
    Cluster communication is configured to use the following interface(s) for this instance
    10.250.90.107
    Mon Feb 19 17:18:13 2007
    cluster interconnect IPC version:Oracle UDP/IP (generic)
    IPC Vendor 1 proto 2
    PMON started with pid=2, OS id=16124
    DIAG started with pid=3, OS id=16126
    PSP0 started with pid=4, OS id=16128
    LMON started with pid=5, OS id=16130
    LMD0 started with pid=6, OS id=16132
    MMAN started with pid=7, OS id=16134
    DBW0 started with pid=8, OS id=16136
    LGWR started with pid=9, OS id=16138
    CKPT started with pid=10, OS id=16140
    SMON started with pid=11, OS id=16142
    RECO started with pid=12, OS id=16144
    CJQ0 started with pid=13, OS id=16146
    MMON started with pid=14, OS id=16148
    Mon Feb 19 17:18:13 2007
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    MMNL started with pid=15, OS id=16150
    Mon Feb 19 17:18:13 2007
    starting up 1 shared server(s) ...
    Mon Feb 19 17:18:13 2007
    lmon registered with NM - instance id 1 (internal mem no 0)
    Mon Feb 19 17:18:14 2007
    Reconfiguration started (old inc 0, new inc 2)
    List of nodes:
    0
    Global Resource Directory frozen
    * allocate domain 0, invalid = TRUE
    Communication channels reestablished
    Master broadcasted resource hash value bitmaps
    Non-local Process blocks cleaned out
    Resources and enqueues cleaned out
    Resources remastered 0
    Set master node info
    Submitted all remote-enqueue requests
    Dwn-cvts replayed, VALBLKs dubious
    All grantable enqueues granted
    Post SMON to start 1st pass IR
    Submitted all GCS remote-cache requests
    Post SMON to start 1st pass IR
    Reconfiguration complete
    Mon Feb 19 17:18:14 2007
    ALTER DATABASE MOUNT
    Mon Feb 19 17:18:14 2007
    Starting background process ASMB
    ASMB started with pid=19, OS id=16182
    Starting background process RBAL
    RBAL started with pid=20, OS id=16186
    Loaded ASM Library - Generic Linux, version 2.0.2 (KABI_V2) library for asmlib interface
    Mon Feb 19 17:18:18 2007
    SUCCESS: diskgroup ORCL_DATA1 was mounted
    Mon Feb 19 17:18:23 2007
    Setting recovery target incarnation to 1
    Mon Feb 19 17:18:23 2007
    Successful mount of redo thread 1, with mount id 3309012054
    Mon Feb 19 17:18:23 2007
    Database mounted in Exclusive Mode
    Completed: ALTER DATABASE MOUNT
    Mon Feb 19 17:18:23 2007
    alter database archivelog
    Completed: alter database archivelog
    Mon Feb 19 17:18:23 2007
    alter database open
    Mon Feb 19 17:18:23 2007
    LGWR: STARTING ARCH PROCESSES
    ARC0 started with pid=22, OS id=16393
    Mon Feb 19 17:18:23 2007
    ARC0: Archival started
    ARC1: Archival started
    LGWR: STARTING ARCH PROCESSES COMPLETE
    ARC1 started with pid=23, OS id=16395
    Mon Feb 19 17:18:23 2007
    Thread 1 opened at log sequence 31
    Current log# 1 seq# 31 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Successful open of redo thread 1
    Mon Feb 19 17:18:23 2007
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Mon Feb 19 17:18:23 2007
    SMON: enabling cache recovery
    Mon Feb 19 17:18:23 2007
    ARC0: STARTING ARCH PROCESSES
    Mon Feb 19 17:18:23 2007
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    Mon Feb 19 17:18:23 2007
    ARC2: Archival started
    ARC0: STARTING ARCH PROCESSES COMPLETE
    ARC0: Becoming the heartbeat ARCH
    ARC2 started with pid=24, OS id=16422
    Mon Feb 19 17:18:23 2007
    Successfully onlined Undo Tablespace 1.
    Mon Feb 19 17:18:23 2007
    SMON: enabling tx recovery
    Mon Feb 19 17:18:23 2007
    Database Characterset is WE8ISO8859P1
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=25, OS id=16449
    Mon Feb 19 17:18:25 2007
    Completed: alter database open
    Mon Feb 19 17:18:25 2007
    ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 3 SIZE 51200K,
    GROUP 4 SIZE 51200K
    Completed: ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 3 SIZE 51200K,
    GROUP 4 SIZE 51200K
    Mon Feb 19 17:18:28 2007
    ALTER DATABASE ENABLE PUBLIC THREAD 2
    Completed: ALTER DATABASE ENABLE PUBLIC THREAD 2
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    Mon Feb 19 17:18:29 2007
    Starting background process EMN0
    EMN0 started with pid=26, OS id=16564
    Mon Feb 19 17:18:29 2007
    Shutting down instance: further logons disabled
    Mon Feb 19 17:18:29 2007
    Stopping background process QMNC
    Mon Feb 19 17:18:29 2007
    Stopping background process CJQ0
    Mon Feb 19 17:18:30 2007
    Stopping background process MMNL
    Mon Feb 19 17:18:31 2007
    Stopping background process MMON
    Mon Feb 19 17:18:32 2007
    Shutting down instance (immediate)
    License high water mark = 3
    Mon Feb 19 17:18:32 2007
    Stopping Job queue slave processes
    Mon Feb 19 17:18:32 2007
    Job queue slave processes stopped
    All dispatchers and shared servers shutdown
    Mon Feb 19 17:18:34 2007
    ALTER DATABASE CLOSE NORMAL
    Mon Feb 19 17:18:34 2007
    SMON: disabling tx recovery
    SMON: disabling cache recovery
    Mon Feb 19 17:18:34 2007
    Shutting down archive processes
    Archiving is disabled
    Mon Feb 19 17:18:39 2007
    ARCH shutting down
    ARC2: Archival stopped
    Mon Feb 19 17:18:44 2007
    ARCH shutting down
    ARC1: Archival stopped
    Mon Feb 19 17:18:49 2007
    ARCH shutting down
    ARC0: Archival stopped
    Mon Feb 19 17:18:50 2007
    Thread 1 closed at log sequence 31
    Successful close of redo thread 1
    Mon Feb 19 17:18:51 2007
    Completed: ALTER DATABASE CLOSE NORMAL
    Mon Feb 19 17:18:51 2007
    ALTER DATABASE DISMOUNT
    Mon Feb 19 17:18:51 2007
    SUCCESS: diskgroup ORCL_DATA1 was dismounted
    Mon Feb 19 17:18:51 2007
    Completed: ALTER DATABASE DISMOUNT
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    Mon Feb 19 17:18:56 2007
    freeing rdom 0
    Mon Feb 19 17:18:59 2007
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Interface type 1 eth1 10.250.90.0 configured from OCR for use as a cluster interconnect
    Interface type 1 eth0 10.250.90.0 configured from OCR for use as a public interface
    Picked latch-free SCN scheme 2
    Autotune of undo retention is turned on.
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.2.0.
    System parameters with non-default values:
    processes = 300
    sessions = 335
    sga_max_size = 1073741824
    spfile = +BSA_DATA1/bsaprod/spfilebsaprod.ora
    sga_target = 1073741824
    control_files = +ORCL_DATA1/bsaprod/controlfile/current.256.614969167
    db_block_size = 8192
    compatible = 10.2.0.2.0
    log_archive_dest_1 = LOCATION=+BSA_DATA1/bsaprod/
    log_archive_format = %t_%s_%r.dbf
    db_file_multiblock_read_count= 16
    cluster_database = TRUE
    cluster_database_instances= 2
    db_create_file_dest = +ORCL_DATA1
    thread = 1
    instance_number = 1
    undo_management = AUTO
    undo_tablespace = UNDOTBS1
    remote_login_passwordfile= EXCLUSIVE
    db_domain =
    dispatchers = (PROTOCOL=TCP) (SERVICE=bsaprodXDB)
    remote_listener = LISTENERS_BSAPROD
    job_queue_processes = 10
    background_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/bdump
    user_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/udump
    core_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/cdump
    audit_file_dest = /u01/app/oracle/oracle/admin/bsaprod/adump
    db_name = bsaprod
    open_cursors = 300
    pga_aggregate_target = 262144000
    Cluster communication is configured to use the following interface(s) for this instance
    10.250.90.107
    Mon Feb 19 17:19:00 2007
    cluster interconnect IPC version:Oracle UDP/IP (generic)
    IPC Vendor 1 proto 2
    PMON started with pid=2, OS id=17373
    DIAG started with pid=3, OS id=17375
    PSP0 started with pid=4, OS id=17377
    LMON started with pid=5, OS id=17379
    LMD0 started with pid=6, OS id=17381
    LMS0 started with pid=7, OS id=17383
    LMS1 started with pid=8, OS id=17387
    MMAN started with pid=9, OS id=17391
    DBW0 started with pid=10, OS id=17393
    LGWR started with pid=11, OS id=17395
    CKPT started with pid=12, OS id=17397
    SMON started with pid=13, OS id=17399
    RECO started with pid=14, OS id=17412
    CJQ0 started with pid=15, OS id=17428
    MMON started with pid=16, OS id=17430
    Mon Feb 19 17:19:00 2007
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    MMNL started with pid=17, OS id=17432
    Mon Feb 19 17:19:00 2007
    starting up 1 shared server(s) ...
    Mon Feb 19 17:19:01 2007
    lmon registered with NM - instance id 1 (internal mem no 0)
    Mon Feb 19 17:19:01 2007
    Reconfiguration started (old inc 0, new inc 2)
    List of nodes:
    0
    Global Resource Directory frozen
    * allocate domain 0, invalid = TRUE
    Communication channels reestablished
    Master broadcasted resource hash value bitmaps
    Non-local Process blocks cleaned out
    Mon Feb 19 17:19:01 2007
    LMS 1: 0 GCS shadows cancelled, 0 closed
    Mon Feb 19 17:19:01 2007
    LMS 0: 0 GCS shadows cancelled, 0 closed
    Set master node info
    Submitted all remote-enqueue requests
    Dwn-cvts replayed, VALBLKs dubious
    All grantable enqueues granted
    Post SMON to start 1st pass IR
    Mon Feb 19 17:19:01 2007
    LMS 1: 0 GCS shadows traversed, 0 replayed
    Mon Feb 19 17:19:01 2007
    LMS 0: 0 GCS shadows traversed, 0 replayed
    Mon Feb 19 17:19:01 2007
    Submitted all GCS remote-cache requests
    Post SMON to start 1st pass IR
    Fix write in gcs resources
    Reconfiguration complete
    LCK0 started with pid=20, OS id=17460
    Mon Feb 19 17:19:02 2007
    ALTER DATABASE MOUNT
    Mon Feb 19 17:19:02 2007
    This instance was first to mount
    Mon Feb 19 17:19:02 2007
    Starting background process ASMB
    ASMB started with pid=22, OS id=17466
    Starting background process RBAL
    RBAL started with pid=23, OS id=17470
    Loaded ASM Library - Generic Linux, version 2.0.2 (KABI_V2) library for asmlib interface
    Mon Feb 19 17:19:06 2007
    SUCCESS: diskgroup ORCL_DATA1 was mounted
    Mon Feb 19 17:19:09 2007
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    Mon Feb 19 17:19:10 2007
    Setting recovery target incarnation to 1
    Mon Feb 19 17:19:10 2007
    Successful mount of redo thread 1, with mount id 3308997510
    Mon Feb 19 17:19:10 2007
    Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
    Completed: ALTER DATABASE MOUNT
    Mon Feb 19 17:19:11 2007
    ALTER DATABASE OPEN
    This instance was first to open
    Picked broadcast on commit scheme to generate SCNs
    Mon Feb 19 17:19:11 2007
    LGWR: STARTING ARCH PROCESSES
    ARC0 started with pid=26, OS id=17679
    Mon Feb 19 17:19:11 2007
    ARC0: Archival started
    ARC1: Archival started
    LGWR: STARTING ARCH PROCESSES COMPLETE
    ARC1 started with pid=27, OS id=17706
    Mon Feb 19 17:19:11 2007
    Thread 1 opened at log sequence 31
    Current log# 1 seq# 31 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Successful open of redo thread 1
    Mon Feb 19 17:19:11 2007
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Mon Feb 19 17:19:11 2007
    ARC0: STARTING ARCH PROCESSES
    Mon Feb 19 17:19:11 2007
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    Mon Feb 19 17:19:11 2007
    SMON: enabling cache recovery
    Mon Feb 19 17:19:11 2007
    ARC2: Archival started
    ARC0: STARTING ARCH PROCESSES COMPLETE
    ARC0: Becoming the heartbeat ARCH
    ARC2 started with pid=28, OS id=17708
    Mon Feb 19 17:19:11 2007
    Successfully onlined Undo Tablespace 1.
    Mon Feb 19 17:19:11 2007
    SMON: enabling tx recovery
    Mon Feb 19 17:19:11 2007
    Database Characterset is WE8ISO8859P1
    Mon Feb 19 17:19:11 2007
    Instance recovery: looking for dead threads
    Instance recovery: lock domain invalid but no dead threads
    Mon Feb 19 17:19:12 2007
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    Mon Feb 19 17:19:12 2007
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=29, OS id=17735
    Mon Feb 19 17:19:13 2007
    Completed: ALTER DATABASE OPEN
    Mon Feb 19 17:20:11 2007
    Shutting down archive processes
    Mon Feb 19 17:20:16 2007
    ARCH shutting down
    ARC2: Archival stopped
    Mon Feb 19 17:23:35 2007
    Starting background process EMN0
    EMN0 started with pid=74, OS id=28350
    Mon Feb 19 17:23:35 2007
    Shutting down instance: further logons disabled
    Mon Feb 19 17:23:36 2007
    Stopping background process QMNC
    Mon Feb 19 17:23:36 2007
    Stopping background process CJQ0
    Mon Feb 19 17:23:38 2007
    Stopping background process MMNL
    Mon Feb 19 17:23:39 2007
    Stopping background process MMON
    Mon Feb 19 17:23:40 2007
    Shutting down instance (immediate)
    License high water mark = 34
    Mon Feb 19 17:23:40 2007
    Stopping Job queue slave processes
    Mon Feb 19 17:23:40 2007
    Job queue slave processes stopped
    All dispatchers and shared servers shutdown
    Mon Feb 19 17:24:07 2007
    ALTER DATABASE CLOSE NORMAL
    Mon Feb 19 17:24:07 2007
    SMON: disabling tx recovery
    SMON: disabling cache recovery
    Mon Feb 19 17:24:08 2007
    Shutting down archive processes
    Archiving is disabled
    Mon Feb 19 17:24:18 2007
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    ARC1: Archiving disabled
    ARCH shutting down
    Mon Feb 19 17:24:18 2007
    ARC1: Archival stopped
    Mon Feb 19 17:24:18 2007
    ARC0: Becoming the heartbeat ARCH
    Mon Feb 19 17:24:18 2007
    ARC0: Archiving disabled
    ARCH shutting down
    ARC0: Archival stopped
    Mon Feb 19 17:24:19 2007
    Thread 1 closed at log sequence 31
    Successful close of redo thread 1
    Mon Feb 19 17:24:19 2007
    Completed: ALTER DATABASE CLOSE NORMAL
    Mon Feb 19 17:24:19 2007
    ALTER DATABASE DISMOUNT
    Mon Feb 19 17:24:19 2007
    SUCCESS: diskgroup ORCL_DATA1 was dismounted
    Mon Feb 19 17:24:19 2007
    Completed: ALTER DATABASE DISMOUNT
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Archive process shutdown avoided: 0 active
    Mon Feb 19 17:24:25 2007
    freeing rdom 0
    Mon Feb 19 17:24:35 2007
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Interface type 1 eth1 10.250.90.0 configured from OCR for use as a cluster interconnect
    Interface type 1 eth0 10.250.90.0 configured from OCR for use as a public interface
    Picked latch-free SCN scheme 2
    Autotune of undo retention is turned on.
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.2.0.
    System parameters with non-default values:
    processes = 300
    sessions = 335
    sga_max_size = 1073741824
    __shared_pool_size = 255852544
    __large_pool_size = 4194304
    __java_pool_size = 4194304
    __streams_pool_size = 0
    spfile = +BSA_DATA1/bsaprod/spfilebsaprod.ora
    sga_target = 1073741824
    control_files = +ORCL_DATA1/bsaprod/controlfile/current.256.614969167
    db_block_size = 8192
    __db_cache_size = 801112064
    compatible = 10.2.0.2.0
    log_archive_dest_1 = LOCATION=+BSA_DATA1/bsaprod/
    log_archive_format = %t_%s_%r.dbf
    db_file_multiblock_read_count= 16
    cluster_database = TRUE
    cluster_database_instances= 2
    db_create_file_dest = +ORCL_DATA1
    thread = 1
    instance_number = 1
    undo_management = AUTO
    undo_tablespace = UNDOTBS1
    remote_login_passwordfile= EXCLUSIVE
    db_domain =
    dispatchers = (PROTOCOL=TCP) (SERVICE=bsaprodXDB)
    remote_listener = LISTENERS_BSAPROD
    job_queue_processes = 10
    background_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/bdump
    user_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/udump
    core_dump_dest = /u01/app/oracle/oracle/admin/bsaprod/cdump
    audit_file_dest = /u01/app/oracle/oracle/admin/bsaprod/adump
    db_name = bsaprod
    open_cursors = 300
    pga_aggregate_target = 262144000
    Cluster communication is configured to use the following interface(s) for this instance
    10.250.90.107
    Mon Feb 19 17:24:35 2007
    cluster interconnect IPC version:Oracle UDP/IP (generic)
    IPC Vendor 1 proto 2
    PMON started with pid=2, OS id=29950
    DIAG started with pid=3, OS id=29952
    PSP0 started with pid=4, OS id=29954
    LMON started with pid=5, OS id=29956
    LMD0 started with pid=6, OS id=29958
    LMS0 started with pid=7, OS id=29960
    LMS1 started with pid=8, OS id=29964
    MMAN started with pid=9, OS id=29968
    DBW0 started with pid=10, OS id=29970
    LGWR started with pid=11, OS id=29972
    CKPT started with pid=12, OS id=29974
    SMON started with pid=13, OS id=29976
    RECO started with pid=14, OS id=29978
    CJQ0 started with pid=15, OS id=29980
    MMON started with pid=16, OS id=29982
    Mon Feb 19 17:24:36 2007
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    MMNL started with pid=17, OS id=29984
    Mon Feb 19 17:24:36 2007
    starting up 1 shared server(s) ...
    Mon Feb 19 17:24:36 2007
    lmon registered with NM - instance id 1 (internal mem no 0)
    Mon Feb 19 17:24:39 2007
    Reconfiguration started (old inc 0, new inc 4)
    List of nodes:
    0 1
    Global Resource Directory frozen
    * allocate domain 0, invalid = TRUE
    Communication channels reestablished
    * domain 0 not valid according to instance 1
    * domain 0 valid = 0 according to instance 1
    Mon Feb 19 17:24:39 2007
    Master broadcasted resource hash value bitmaps
    Non-local Process blocks cleaned out
    Mon Feb 19 17:24:39 2007
    LMS 1: 0 GCS shadows cancelled, 0 closed
    Mon Feb 19 17:24:39 2007
    LMS 0: 0 GCS shadows cancelled, 0 closed
    Set master node info
    Submitted all remote-enqueue requests
    Dwn-cvts replayed, VALBLKs dubious
    All grantable enqueues granted
    Mon Feb 19 17:24:39 2007
    LMS 0: 0 GCS shadows traversed, 0 replayed
    Mon Feb 19 17:24:39 2007
    LMS 1: 0 GCS shadows traversed, 0 replayed
    Mon Feb 19 17:24:39 2007
    Submitted all GCS remote-cache requests
    Post SMON to start 1st pass IR
    Fix write in gcs resources
    Reconfiguration complete
    LCK0 started with pid=20, OS id=30069
    Mon Feb 19 17:24:40 2007
    ALTER DATABASE MOUNT
    Mon Feb 19 17:24:41 2007
    Starting background process ASMB
    ASMB started with pid=22, OS id=30121
    Starting background process RBAL
    RBAL started with pid=23, OS id=30125
    Loaded ASM Library - Generic Linux, version 2.0.2 (KABI_V2) library for asmlib interface
    Mon Feb 19 17:24:45 2007
    SUCCESS: diskgroup ORCL_DATA1 was mounted
    Mon Feb 19 17:24:49 2007
    Setting recovery target incarnation to 1
    Mon Feb 19 17:24:49 2007
    Successful mount of redo thread 1, with mount id 3309017062
    Mon Feb 19 17:24:49 2007
    Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
    Mon Feb 19 17:24:51 2007
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    SUCCESS: diskgroup BSA_DATA1 was mounted
    SUCCESS: diskgroup BSA_DATA1 was dismounted
    Mon Feb 19 17:24:52 2007
    Completed: ALTER DATABASE MOUNT
    Mon Feb 19 17:24:52 2007
    ALTER DATABASE OPEN
    Picked broadcast on commit scheme to generate SCNs
    Mon Feb 19 17:24:52 2007
    LGWR: STARTING ARCH PROCESSES
    ARC0 started with pid=25, OS id=30406
    Mon Feb 19 17:24:53 2007
    ARC0: Archival started
    ARC1: Archival started
    LGWR: STARTING ARCH PROCESSES COMPLETE
    ARC1 started with pid=26, OS id=30422
    Mon Feb 19 17:24:53 2007
    Thread 1 opened at log sequence 31
    Current log# 1 seq# 31 mem# 0: +ORCL_DATA1/bsaprod/onlinelog/group_1.257.614969167
    Successful open of redo thread 1
    Mon Feb 19 17:24:53 2007
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Mon Feb 19 17:24:53 2007
    ARC0: STARTING ARCH PROCESSES
    Mon Feb 19 17:24:53 2007
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    Mon Feb 19 17:24:53 2007
    SMON: enabling cache recovery
    Mon Feb 19 17:24:53 2007
    ARC2: Archival started
    ARC0: STARTING ARCH PROCESSES COMPLETE
    ARC0: Becoming the heartbeat ARCH
    ARC2 started with pid=27, OS id=30436
    Mon Feb 19 17:25:07 2007
    Successfully onlined Undo Tablespace 1.
    Mon Feb 19 17:25:07 2007
    SMON: enabling tx recovery
    Mon Feb 19 17:25:07 2007
    Database Characterset is WE8ISO8859P1
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=29, OS id=30882
    Mon Feb 19 17:26:31 2007
    Completed: ALTER DATABASE OPEN
    Mon Feb 19 17:26:38 2007
    ALTER SYSTEM SET service_names='bsaprod','bsatest' SCOPE=MEMORY SID='bsaprod1';
    Mon Feb 19 17:26:53 2007
    Shutting down archive processes
    Mon Feb 19 17:26:58 2007
    ARCH shutting down
    ARC2: Archival stopped
    Mon Feb 19 17:32:16 2007
    Error: KGXGN aborts the instance (6)
    Mon Feb 19 17:32:16 2007
    Errors in file /u01/app/oracle/oracle/admin/bsaprod/bdump/bsaprod1_lmon_29956.trc:
    ORA-29702: error occurred in Cluster Group Service operation
    LMON: terminating instance due to error 29702
    Mon Feb 19 17:32:17 2007
    System state dump is made for local instance
    System State dumped to trace file /u01/app/oracle/oracle/admin/bsaprod/bdump/bsaprod1_diag_29952.trc
    Mon Feb 19 17:32:18 2007
    Trace dumping is performing id=[cdmp_20070219173217]
    Mon Feb 19 17:32:21 2007
    Instance terminated by LMON, pid = 29956
    and also the output of crs_stat -t in first node is....
    Name Type Target State Host
    ora....d1.inst application ONLINE OFFLINE
    ora....d2.inst application ONLINE OFFLINE
    ora....od1.srv application ONLINE OFFLINE
    ora....od2.srv application ONLINE OFFLINE
    ora....test.cs application ONLINE OFFLINE
    ora.bsaprod.db application ONLINE OFFLINE
    ora....SM1.asm application ONLINE UNKNOWN dsrvbd003
    ora....ER.lsnr application ONLINE UNKNOWN dsrvbd003
    ora....03.lsnr application ONLINE UNKNOWN dsrvbd003
    ora....003.gsd application ONLINE UNKNOWN dsrvbd003
    ora....003.ons application ONLINE UNKNOWN dsrvbd003
    ora....003.vip application ONLINE ONLINE dsrvbd003
    ora....SM2.asm application ONLINE OFFLINE
    ora....04.lsnr application ONLINE OFFLINE
    ora....004.gsd application ONLINE OFFLINE
    ora....004.ons application ONLINE OFFLINE
    ora....004.vip application ONLINE OFFLINE

  • Changing the location of archive log from flash recovery area PLZ HELP!!!

    Hi All,
    My archive log is being stored in flash memory area which got full and the production server went down.
    alert log file details.....
    ORA-19809: limit exceeded for recovery files
    ORA-19804: cannot reclaim 43432960 bytes disk space from 2147483648 limit
    *** 2010-04-25 14:22:49.777 62692 kcrr.c
    ARCH: Error 19809 Creating archive log file to
    '/oracle/product/10.2.0/flash_rec
    overy_area/EDWREP/archivelog/2010_04_25/o1_mf_1_232_%u_.arc'
    *** 2010-04-25 14:22:49.777 60970 kcrr.c
    kcrrfail: dest:10 err:19809 force:0 blast:1I removed the files and started the database,
    Can someone kindly tell me as to how to avoid this problem in future by keeping archive log destination in flash recovery area.
    I want to change the location of archive log files, can someone please guide me as to hiow to do that
    I changed the size of flash recovery area for the time being, but i am afraid it will be full again!!
    SQL> select * from v$flash_recovery_area_usage;
    FILE_TYPE    PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
    CONTROLFILE                   0                         0               0
    ONLINELOG                     0                         0               0
    ARCHIVELOG                99.44                         0              57
    BACKUPPIECE                   0                         0               0
    IMAGECOPY                     0                         0               0
    FLASHBACKLOG                  0                         0               0
    6 rows selected.
    SQL> alter system set DB_RECOVERY_FILE_DEST_SIZE = 4G ;
    System altered.
    SQL> select * from v$flash_recovery_area_usage;
    FILE_TYPE    PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
    CONTROLFILE                   0                         0               0
    ONLINELOG                     0                         0               0
    ARCHIVELOG                49.72                         0              57
    BACKUPPIECE                   0                         0               0
    IMAGECOPY                     0                         0               0
    FLASHBACKLOG                  0                         0               0
    6 rows selected.regards,
    Edited by: user10243788 on Apr 25, 2010 6:12 AM

    user10243788 wrote:
    Hi All,
    My archive log is being stored in flash memory area which got full and the production server went down.
    alert log file details.....
    ORA-19809: limit exceeded for recovery files
    ORA-19804: cannot reclaim 43432960 bytes disk space from 2147483648 limit
    *** 2010-04-25 14:22:49.777 62692 kcrr.c
    ARCH: Error 19809 Creating archive log file to
    '/oracle/product/10.2.0/flash_rec
    overy_area/EDWREP/archivelog/2010_04_25/o1_mf_1_232_%u_.arc'
    *** 2010-04-25 14:22:49.777 60970 kcrr.c
    kcrrfail: dest:10 err:19809 force:0 blast:1I removed the files and started the database,
    Can someone kindly tell me as to how to avoid this problem in future by keeping archive log destination in flash recovery area.
    I want to change the location of archive log files, can someone please guide me as to hiow to do that
    I changed the size of flash recovery area for the time being, but i am afraid it will be full again!!
    SQL> select * from v$flash_recovery_area_usage;
    FILE_TYPE    PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
    CONTROLFILE                   0                         0               0
    ONLINELOG                     0                         0               0
    ARCHIVELOG                99.44                         0              57
    BACKUPPIECE                   0                         0               0
    IMAGECOPY                     0                         0               0
    FLASHBACKLOG                  0                         0               0
    6 rows selected.
    SQL> alter system set DB_RECOVERY_FILE_DEST_SIZE = 4G ;
    System altered.
    SQL> select * from v$flash_recovery_area_usage;
    FILE_TYPE    PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
    CONTROLFILE                   0                         0               0
    ONLINELOG                     0                         0               0
    ARCHIVELOG                49.72                         0              57
    BACKUPPIECE                   0                         0               0
    IMAGECOPY                     0                         0               0
    FLASHBACKLOG                  0                         0               0
    6 rows selected.regards,
    Edited by: user10243788 on Apr 25, 2010 6:12 AMPointing the archive log dest (and/or the FRA) to a new location, or enlarging them, will do no good if you are not performing regular housekeeping on the archivelogs. You will just keep knocking down the same problem over and over.
    If you simply delete the archivelogs at the OS level, the database will never know about it and it will continue to think the destination is full, based on records kept in the control file.
    For regular housekeeping, you need to be doing something similar to this in rman:
    run {
      backup archivelog all not backed up 1 times tag='bkup_vlnxora1_arch';
      delete noprompt archivelog all backed up 1 times to device type disk;
    run {
    delete noprompt obsolete;
    crosscheck archivelog all;
    delete noprompt expired archivelog all;

  • Problem with mmon and freespace

    Hi all, I'm using Oracle 10 XE and i have a problem:
    Oracle located on disc D:\ (size=93GB). The day before yesterday free space on disc is over... I have found in a folder D:\oraclexe\app\oracle\admin\xe\bdump ~100 files xe_mmon_[number].trc in size everyone on 100MB...I delete it.
    For a day the empty space decreases on 10GB.
    What I can make????
    alert.log (some part)
    Thu Jul 31 07:13:14 2008
    ORACLE V10.2.0.1.0 - Production vsnsta=0
    vsnsql=14 vsnxtr=3
    Windows XP Version V5.1 Service Pack 2
    CPU : 2 - type 586
    Process Affinity : 0x00000000
    Memory (Avail/Total): Ph:776M/1006M, Ph+PgF:2293M/2423M, VA:1945M/2047M
    Thu Jul 31 07:13:14 2008
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Picked latch-free SCN scheme 2
    Using LOG_ARCHIVE_DEST_10 parameter default value as USE_DB_RECOVERY_FILE_DEST
    Autotune of undo retention is turned on.
    IMODE=BR
    ILAT =10
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.1.0.
    System parameters with non-default values:
    sessions = 49
    __shared_pool_size = 150994944
    __large_pool_size = 4194304
    __java_pool_size = 4194304
    __streams_pool_size = 0
    spfile = D:\ORACLEXE\APP\ORACLE\PRODUCT\10.2.0\SERVER\DBS\SPFILEXE.ORA
    sga_target = 570425344
    control_files = D:\ORACLEXE\ORADATA\XE\CONTROL.DBF
    __db_cache_size = 406847488
    compatible = 10.2.0.1.0
    db_recovery_file_dest = D:\oraclexe\app\oracle\flash_recovery_area
    db_recovery_file_dest_size= 10737418240
    undo_management = AUTO
    undo_tablespace = UNDO
    remote_login_passwordfile= EXCLUSIVE
    dispatchers = (PROTOCOL=TCP) (SERVICE=XEXDB)
    shared_servers = 4
    job_queue_processes = 4
    audit_file_dest = D:\ORACLEXE\APP\ORACLE\ADMIN\XE\ADUMP
    background_dump_dest = D:\ORACLEXE\APP\ORACLE\ADMIN\XE\BDUMP
    user_dump_dest = D:\ORACLEXE\APP\ORACLE\ADMIN\XE\UDUMP
    core_dump_dest = D:\ORACLEXE\APP\ORACLE\ADMIN\XE\CDUMP
    db_name = XE
    open_cursors = 300
    os_authent_prefix =
    pga_aggregate_target = 188743680
    MMAN started with pid=4, OS id=284
    DBW0 started with pid=5, OS id=456
    LGWR started with pid=6, OS id=620
    CKPT started with pid=7, OS id=984
    SMON started with pid=8, OS id=1260
    RECO started with pid=9, OS id=852
    CJQ0 started with pid=10, OS id=1740
    MMON started with pid=11, OS id=1992
    MMNL started with pid=12, OS id=384
    Thu Jul 31 07:13:16 2008
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    starting up 4 shared server(s) ...
    Oracle Data Guard is not available in this edition of Oracle.
    Thu Jul 31 07:13:17 2008
    alter database mount exclusive
    PSP0 started with pid=3, OS id=1900
    PMON started with pid=2, OS id=1592
    Setting recovery target incarnation to 2
    Thu Jul 31 07:13:21 2008
    Successful mount of redo thread 1, with mount id 2542578765
    Thu Jul 31 07:13:21 2008
    Database mounted in Exclusive Mode
    Completed: alter database mount exclusive
    Thu Jul 31 07:13:22 2008
    alter database open
    Thu Jul 31 07:13:22 2008
    Beginning crash recovery of 1 threads
    Thu Jul 31 07:13:22 2008
    Started redo scan
    Thu Jul 31 07:13:23 2008
    Completed redo scan
    46578 redo blocks read, 3101 data blocks need recovery
    Thu Jul 31 07:13:26 2008
    Started redo application at
    Thread 1: logseq 955, block 97063
    Thu Jul 31 07:13:26 2008
    Recovery of Online Redo Log: Thread 1 Group 2 Seq 955 Reading mem 0
    Mem# 0 errs 0: D:\ORACLEXE\APP\ORACLE\FLASH_RECOVERY_AREA\XE\ONLINELOG\O1_MF_2_3JWMW0BQ_.LOG
    Thu Jul 31 07:13:27 2008
    Recovery of Online Redo Log: Thread 1 Group 1 Seq 956 Reading mem 0
    Mem# 0 errs 0: D:\ORACLEXE\APP\ORACLE\FLASH_RECOVERY_AREA\XE\ONLINELOG\O1_MF_1_3JWMVZ4W_.LOG
    Thu Jul 31 07:13:32 2008
    Completed redo application
    Thu Jul 31 07:13:32 2008
    Completed crash recovery at
    Thread 1: logseq 956, block 42546, scn 41444666
    3101 data blocks read, 3101 data blocks written, 46578 redo blocks read
    Thu Jul 31 07:13:33 2008
    Thread 1 advanced to log sequence 957
    Thread 1 opened at log sequence 957
    Current log# 2 seq# 957 mem# 0: D:\ORACLEXE\APP\ORACLE\FLASH_RECOVERY_AREA\XE\ONLINELOG\O1_MF_2_3JWMW0BQ_.LOG
    Successful open of redo thread 1
    Thu Jul 31 07:13:33 2008
    SMON: enabling cache recovery
    Thu Jul 31 07:13:36 2008
    Successfully onlined Undo Tablespace 1.
    Thu Jul 31 07:13:36 2008
    SMON: enabling tx recovery
    Thu Jul 31 07:13:36 2008
    Database Characterset is CL8MSWIN1251
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=18, OS id=2028
    Thu Jul 31 07:13:43 2008
    Completed: alter database open
    Thu Jul 31 07:13:43 2008
    db_recovery_file_dest_size of 10240 MB is 0.98% used. This is a
    user-specified limit on the amount of space that will be used by this
    database for recovery-related files, and does not reflect the amount of
    space available in the underlying filesystem or ASM diskgroup.
    Thu Jul 31 07:13:45 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:13:47 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:17:53 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:17:54 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:22:55 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:22:56 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:27:58 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:27:59 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:33:00 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1992.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:33:03 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=468
    Thu Jul 31 07:33:06 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:33:08 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:38:09 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:38:10 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:43:09 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:43:10 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:48:12 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:48:13 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:53:14 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_468.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:53:54 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=1096
    Thu Jul 31 07:53:57 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:53:59 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:58:58 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 07:58:59 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:03:58 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:03:59 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:08:58 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:09:00 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:13:59 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1096.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:14:36 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=372
    Thu Jul 31 08:14:39 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:14:41 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:19:40 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:19:42 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:24:42 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:24:44 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:29:42 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:29:44 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:34:43 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_372.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:35:21 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=812
    Thu Jul 31 08:35:24 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:35:27 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:40:27 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:40:29 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:45:28 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:45:30 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:50:30 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:50:32 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:55:31 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_812.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:56:09 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=400
    Thu Jul 31 08:56:12 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 08:56:15 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:01:15 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:01:17 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:06:17 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:06:19 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:11:19 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:11:21 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:16:21 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_400.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:16:27 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=552
    Thu Jul 31 09:16:30 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:16:33 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:21:33 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:21:35 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:26:36 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:26:38 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:31:38 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:31:41 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:36:38 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_552.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:37:15 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=1132
    Thu Jul 31 09:37:18 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:37:21 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:42:21 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:42:24 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:47:21 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:47:24 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:52:24 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:52:27 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:57:24 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1132.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:57:54 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=1692
    Thu Jul 31 09:57:57 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 09:58:00 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:02:58 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:03:01 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:07:58 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:08:01 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:12:59 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:13:02 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:18:00 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1692.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    Thu Jul 31 10:18:39 2008
    Restarting dead background process MMON
    MMON started with pid=11, OS id=1984
    Thu Jul 31 10:18:42 2008
    Errors in file d:\oraclexe\app\oracle\admin\xe\bdump\xe_mmon_1984.trc:
    ORA-00600: internal error code, arguments: [kjhn_post_ha_alert0-862], [], [], [], [], [], [], []
    ....................

    Hi,
    Contact oracle support, raise a SR. Or you can search from metalink lookup tool
    https://metalink.oracle.com/metalink/plsql/f?p=130:14:11851539057398949375::::p14_database_id,p14_docid,p14_show_header,p14_show_help,p14_black_frame,p14_font:NOT,153788.1,1,0,0,helvetica
    regards
    Jafar

  • DataGuard Standby 環境で db が open できない.

    DataGuard Standby 環境で db が open できなくて困っています。
    詳しい方いれば、原因と対処方法を教えていただきたいと思います。
    create した spfile が正確に読み込まれていないことが原因のようなのですが。。
    なぜ読み込めないのかが分からない状態です。
    pfile を読み込んだ起動は可能ですが、spfile を読み込んだ起動の前提条件とは、
    何なのでしょうか。単に整合性が取れていないだけなのか、それとも、、。
    [grid@osaka1 shell]$ asmcmd
    ASMCMD> ls
    DATA/
    FRA/
    ASMCMD> cd data
    ASMCMD> ls
    ASM/
    WEST/
    ASMCMD> ls -l
    Type  Redund  Striped  Time             Sys  Name
                                            Y    ASM/
                                            N    WEST/
    ASMCMD>
    ASMCMD> cd west
    ASMCMD> ls
    CONTROLFILE/
    DATAFILE/
    ONLINELOG/
    PARAMETERFILE/
    TEMPFILE/
    spfilewest.ora
    ASMCMD> ls -l
    Type           Redund  Striped  Time             Sys  Name
                                                     N    CONTROLFILE/
                                                     N    DATAFILE/
                                                     N    ONLINELOG/
                                                     N    PARAMETERFILE/
                                                     N    TEMPFILE/
                                                     N    spfilewest.ora => +DATA/WEST/PARAMETERFILE/spfile.257.824236121
    ASMCMD> pwd
    +data/west
    ASMCMD>
    ASMCMD> cd para*
    ASMCMD> ls -l
    Type           Redund  Striped  Time             Sys  Name
    PARAMETERFILE  MIRROR  COARSE   AUG 23 18:00:00  Y    spfile.257.824236121
    ASMCMD>
    ASMCMD> pwd
    +data/west/PARAMETERFILE
    ASMCMD> quit
    [grid@osaka1 shell]$
    [oracle@osaka1 dbs]$ more initHPYMUSIC.ora
    SPFILE='+DATA/west/spfilewest.ora'
    [oracle@osaka1 dbs]$
    よろしくお願い致します。
    ps.
    ORA-12154 は整合性の問題であるので、それを合わせれば消えると思っています。
    そもそも RAC を前提としていたのですが、それを standalone に置き換えて検証始めた結果、
    こうなってしまっています。
    open できない原因が ORA-12154 だったりして。。
    ■ プライマリの場合
    ○ open するほうは、シンプルに以下だけで open することが確認できる。
       が、db_name を変更した関係で
       「ORA-12154: TNS: 指定された接続識別子を解決できませんでした」
       が出続けている。
       ORA-12154 は db が open できない原因とは無関係かもしれない。
    srvctl stop database -d east -f
    srvctl start database -d east -o open
    srvctl config database -d east
    srvctl status database -d east
    ○ 参考出力
    set linesize 500 pages 0
    col value for a90
    col name for a50
    select name, value
    from v$parameter
    where name in ('db_name','db_unique_name','log_archive_config', 'log_archive_dest_1','log_archive_dest_2',
                   'log_archive_dest_state_1','log_archive_dest_state_2', 'remote_login_passwordfile',
                   'log_archive_format','log_archive_max_processes','fal_server','db_file_name_convert',
                   'log_file_name_convert', 'standby_file_management');
    SQL>
    db_file_name_convert
    log_file_name_convert
    log_archive_dest_1
    log_archive_dest_2            SERVICE=HPYMUSIC SYNC NOAFFIRM VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=west
    log_archive_dest_state_1      enable
    log_archive_dest_state_2      enable
    fal_server
    log_archive_config
    log_archive_format            %t_%s_%r.dbf
    log_archive_max_processes     4
    standby_file_management       AUTO
    remote_login_passwordfile     EXCLUSIVE
    db_name                       HPYMUSIC
    db_unique_name                HPYMUSIC    ← ▼ db_name だけを変更したつもりが db_unique_name も変更されていた
    14行が選択されました。
    [oracle@tokyo1 shell]$ srvctl stop database -d east -f
    [oracle@tokyo1 shell]$ /u01/app/11.2.0/grid/bin/crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS
    Cluster Resources
    ora.east.db
          1        OFFLINE OFFLINE                               Instance Shutdown
    [oracle@tokyo1 shell]$ srvctl start database -d east -o open
    [oracle@tokyo1 shell]$ /u01/app/11.2.0/grid/bin/crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS
    Local Resources
    ora.DATA.dg
                   ONLINE  ONLINE       tokyo1
    ora.FRA.dg
                   ONLINE  ONLINE       tokyo1
    ora.LISTENER.lsnr
                   ONLINE  ONLINE       tokyo1
    ora.asm
                   ONLINE  ONLINE       tokyo1                   Started
    Cluster Resources
    ora.cssd
          1        ONLINE  ONLINE       tokyo1
    ora.diskmon
          1        ONLINE  ONLINE       tokyo1
    ora.east.db
          1        ONLINE  ONLINE       tokyo1                   Open ← ▼
    [oracle@tokyo1 shell]$
    [oracle@tokyo1 shell]$ srvctl config database -d east
    一意のデータベース名: east
    データベース名: east
    Oracleホーム: /u01/app/oracle/product/11.2.0/dbhome_1
    Oracleユーザー: grid
    spfile: +DATA/east/spfileeast.ora
    ドメイン:
    開始オプション: open
    停止オプション: immediate
    データベース・ロール: PRIMARY
    管理ポリシー: AUTOMATIC
    ディスク・グループ: DATA,FRA
    サービス:
    [oracle@tokyo1 shell]$ srvctl status database -d east
    データベースは実行中です。
    Fri Aug 23 19:44:10 2013
    Error 12154 received logging on to the standby
    Errors in file /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_arc2_7579.trc:
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    PING[ARC2]: Heartbeat failed to connect to standby 'HPYMUSIC'. Error is 12154.
    [oracle@tokyo1 dbs]$ pwd
    /u01/app/oracle/product/11.2.0/dbhome_1/dbs
    [oracle@tokyo1 dbs]$
    [oracle@tokyo1 dbs]$
    [oracle@tokyo1 dbs]$ more 2013.08.23_east_pfile.txt
    HPYMUSIC.__db_cache_size=301989888
    HPYMUSIC.__java_pool_size=4194304
    HPYMUSIC.__large_pool_size=8388608
    HPYMUSIC.__pga_aggregate_target=339738624
    HPYMUSIC.__sga_target=503316480
    HPYMUSIC.__shared_io_pool_size=0
    HPYMUSIC.__shared_pool_size=176160768
    HPYMUSIC.__streams_pool_size=0
    *.audit_file_dest='/u01/app/oracle/admin/east/adump'
    *.audit_trail='db'
    *.compatible='11.2.0.0.0'
    *.control_files='+DATA/east/controlfile/current.270.823277705','+FRA/east/controlfile/current.
    260.823277707'
    *.db_block_checking='TRUE'
    *.db_block_checksum='TRUE'
    *.db_block_size=8192
    *.db_create_file_dest='+DATA'
    *.db_domain=''
    *.db_name='HPYMUSIC'
    *.db_recovery_file_dest='+FRA'
    *.db_recovery_file_dest_size=3038773248
    *.diagnostic_dest='/u01/app/oracle'
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=HPYMUSICXDB)'
    *.log_archive_format='%t_%s_%r.dbf'
    *.memory_target=842006528
    *.nls_language='JAPANESE'
    *.nls_territory='JAPAN'
    *.open_cursors=300
    *.processes=150
    *.remote_login_passwordfile='EXCLUSIVE'
    *.standby_file_management='AUTO'
    *.undo_tablespace='UNDOTBS1'
    Fri Aug 23 19:49:38 2013
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Picked latch-free SCN scheme 3
    Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST
    ARCH: Warning; less destinations available than specified
    by LOG_ARCHIVE_MIN_SUCCEED_DEST init.ora parameter
    Autotune of undo retention is turned on.
    IMODE=BR
    ILAT =27
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    Starting up:
    Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options.
    Using parameter settings in server-side spfile /u01/app/oracle/product/11.2.0/dbhome_1/dbs/spfileHPYMUSIC.ora
    System parameters with non-default values:
      processes                = 150
      nls_language             = "JAPANESE"
      nls_territory            = "JAPAN"
      memory_target            = 804M
      control_files            = "+DATA/east/controlfile/current.270.823277705"
      control_files            = "+FRA/east/controlfile/current.260.823277707"
      db_block_checksum        = "TRUE"
      db_block_size            = 8192
      compatible               = "11.2.0.0.0"
      log_archive_dest_2       = "SERVICE=HPYMUSIC SYNC NOAFFIRM VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=west"
      log_archive_format       = "%t_%s_%r.dbf"
      db_create_file_dest      = "+DATA"
      db_recovery_file_dest    = "+FRA"
      db_recovery_file_dest_size= 2898M
      standby_file_management  = "AUTO"
      undo_tablespace          = "UNDOTBS1"
      db_block_checking        = "TRUE"
      remote_login_passwordfile= "EXCLUSIVE"
      db_domain                = ""
      dispatchers              = "(PROTOCOL=TCP) (SERVICE=HPYMUSICXDB)"
      audit_file_dest          = "/u01/app/oracle/admin/east/adump"
      audit_trail              = "DB"
      db_name                  = "HPYMUSIC"
      open_cursors             = 300
      diagnostic_dest          = "/u01/app/oracle"
    Fri Aug 23 19:49:39 2013
    PMON started with pid=2, OS id=8442
    Fri Aug 23 19:49:39 2013
    VKTM started with pid=3, OS id=8444 at elevated priority
    VKTM running at (10)millisec precision with DBRM quantum (100)ms
    Fri Aug 23 19:49:39 2013
    GEN0 started with pid=4, OS id=8448
    Fri Aug 23 19:49:39 2013
    DIAG started with pid=5, OS id=8450
    Fri Aug 23 19:49:39 2013
    DBRM started with pid=6, OS id=8452
    Fri Aug 23 19:49:39 2013
    PSP0 started with pid=7, OS id=8454
    Fri Aug 23 19:49:39 2013
    DIA0 started with pid=8, OS id=8456
    Fri Aug 23 19:49:39 2013
    MMAN started with pid=9, OS id=8458
    Fri Aug 23 19:49:39 2013
    DBW0 started with pid=10, OS id=8460
    Fri Aug 23 19:49:39 2013
    LGWR started with pid=11, OS id=8462
    Fri Aug 23 19:49:39 2013
    CKPT started with pid=12, OS id=8464
    Fri Aug 23 19:49:39 2013
    SMON started with pid=13, OS id=8466
    Fri Aug 23 19:49:39 2013
    RECO started with pid=14, OS id=8468
    Fri Aug 23 19:49:39 2013
    RBAL started with pid=15, OS id=8470
    Fri Aug 23 19:49:39 2013
    ASMB started with pid=16, OS id=8472
    Fri Aug 23 19:49:39 2013
    MMON started with pid=17, OS id=8474
    Fri Aug 23 19:49:39 2013
    MMNL started with pid=18, OS id=8478
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    NOTE: initiating MARK startup
    starting up 1 shared server(s) ...
    Starting background process MARK
    Fri Aug 23 19:49:39 2013
    MARK started with pid=20, OS id=8482
    NOTE: MARK has subscribed
    ORACLE_BASE not set in environment. It is recommended
    that ORACLE_BASE be set in the environment
    Reusing ORACLE_BASE from an earlier startup = /u01/app/oracle
    Fri Aug 23 19:49:39 2013
    ALTER DATABASE MOUNT
    NOTE: Loaded library: System
    SUCCESS: diskgroup DATA was mounted
    ERROR: failed to establish dependency between database HPYMUSIC and diskgroup resource ora.DATA.dg
    SUCCESS: diskgroup FRA was mounted
    ERROR: failed to establish dependency between database HPYMUSIC and diskgroup resource ora.FRA.dg
    Fri Aug 23 19:49:46 2013
    NSS2 started with pid=24, OS id=8572
    Successful mount of redo thread 1, with mount id 2951868947
    Database mounted in Exclusive Mode
    Lost write protection disabled
    Completed: ALTER DATABASE MOUNT
    ALTER DATABASE OPEN
    LGWR: STARTING ARCH PROCESSES
    Fri Aug 23 19:49:47 2013
    ARC0 started with pid=26, OS id=8574
    ARC0: Archival started
    LGWR: STARTING ARCH PROCESSES COMPLETE
    ARC0: STARTING ARCH PROCESSES
    Fri Aug 23 19:49:48 2013
    ARC1 started with pid=27, OS id=8576
    Fri Aug 23 19:49:48 2013
    ARC2 started with pid=28, OS id=8578
    ARC1: Archival started
    ARC2: Archival started
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    ARC2: Becoming the heartbeat ARCH
    Fri Aug 23 19:49:48 2013
    ARC3 started with pid=29, OS id=8580
    LGWR: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
    ARC3: Archival started
    ARC0: STARTING ARCH PROCESSES COMPLETE
    Error 12154 received logging on to the standby
    Fri Aug 23 19:49:51 2013
    Errors in file /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_lgwr_8462.trc:
    ORA-12154: TNS: ?????????????????????
    Error 12154 for archive log file 2 to 'HPYMUSIC'
    LGWR: Failed to archive log 2 thread 1 sequence 8 (12154)
    Thread 1 advanced to log sequence 8 (thread open)
    Thread 1 opened at log sequence 8
      Current log# 2 seq# 8 mem# 0: +DATA/hpymusic/onlinelog/group_2.272.824213887
      Current log# 2 seq# 8 mem# 1: +FRA/hpymusic/onlinelog/group_2.262.824213889
    Successful open of redo thread 1
    Fri Aug 23 19:49:51 2013
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Fri Aug 23 19:49:51 2013
    SMON: enabling cache recovery
    Error 12154 received logging on to the standby
    Errors in file /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_arc2_8578.trc:
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    PING[ARC2]: Heartbeat failed to connect to standby 'HPYMUSIC'. Error is 12154.
    Archived Log entry 7 added for thread 1 sequence 7 ID 0xaff1210d dest 1:
    Error 12154 received logging on to the standby
    Errors in file /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_arc3_8580.trc:
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    FAL[server, ARC3]: Error 12154 creating remote archivelog file 'HPYMUSIC'
    FAL[server, ARC3]: FAL archive failed, see trace file.
    Errors in file /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_arc3_8580.trc:
    ORA-16055: FALリクエストが拒否されました。
    ARCH: FAL archive failed. Archiver continuing
    ORACLE Instance HPYMUSIC - Archival Error. Archiver continuing.
    Successfully onlined Undo Tablespace 2.
    Verifying file header compatibility for 11g tablespace encryption..
    Verifying 11g file header compatibility for tablespace encryption completed
    SMON: enabling tx recovery
    Database Characterset is AL32UTF8
    No Resource Manager plan active
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    Fri Aug 23 19:49:55 2013
    QMNC started with pid=32, OS id=8590
    Completed: ALTER DATABASE OPEN
    Fri Aug 23 19:49:59 2013
    Starting background process CJQ0
    Fri Aug 23 19:49:59 2013
    CJQ0 started with pid=33, OS id=8609
    Fri Aug 23 19:49:59 2013
    db_recovery_file_dest_size of 2898 MB is 6.38% used. This is a
    user-specified limit on the amount of space that will be used by this
    database for recovery-related files, and does not reflect the amount of
    space available in the underlying filesystem or ASM diskgroup.
    [root@tokyo1 app]#
    [root@tokyo1 app]# more /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_arc2_8578.trc
    Trace file /u01/app/oracle/diag/rdbms/hpymusic/HPYMUSIC/trace/HPYMUSIC_arc2_8578.trc
    Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    With the Partitioning, Automatic Storage Management, OLAP, Data Mining
    and Real Application Testing options
    ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1
    System name:    Linux
    Node name:      tokyo1.oracle11g.jp
    Release:        2.6.18-348.12.1.el5
    Version:        #1 SMP Wed Jul 10 05:28:41 EDT 2013
    Machine:        x86_64
    Instance name: HPYMUSIC
    Redo thread mounted by this instance: 1
    Oracle process number: 28
    Unix process pid: 8578, image: [email protected] (ARC2)
    *** 2013-08-23 19:49:51.707
    *** SESSION ID:(15.1) 2013-08-23 19:49:51.707
    *** CLIENT ID:() 2013-08-23 19:49:51.707
    *** SERVICE NAME:() 2013-08-23 19:49:51.707
    *** MODULE NAME:() 2013-08-23 19:49:51.707
    *** ACTION NAME:() 2013-08-23 19:49:51.707
    Redo shipping client performing standby login
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    *** 2013-08-23 19:49:51.972 4132 krsh.c
    Error 12154 received logging on to the standby
    *** 2013-08-23 19:49:51.972 869 krsu.c
    Error 12154 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'HPYMUSIC'
    Error 12154 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'HPYMUSIC'
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    *** 2013-08-23 19:49:51.973 4132 krsh.c
    PING[ARC2]: Heartbeat failed to connect to standby 'HPYMUSIC'. Error is 12154.
    *** 2013-08-23 19:49:51.973 2747 krsi.c
    krsi_dst_fail: dest:2 err:12154 force:0 blast:1
    *** 2013-08-23 19:50:49.816
    Redo shipping client performing standby login
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    *** 2013-08-23 19:50:50.070 4132 krsh.c
    Error 12154 received logging on to the standby
    *** 2013-08-23 19:50:50.070 869 krsu.c
    Error 12154 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'HPYMUSIC'
    Error 12154 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'HPYMUSIC'
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    *** 2013-08-23 19:50:50.070 4132 krsh.c
    PING[ARC2]: Heartbeat failed to connect to standby 'HPYMUSIC'. Error is 12154.
    *** 2013-08-23 19:50:50.070 2747 krsi.c
    krsi_dst_fail: dest:2 err:12154 force:0 blast:1
    *** 2013-08-23 19:51:51.147
    Redo shipping client performing standby login
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    OCIServerAttach failed -1
    .. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    *** 2013-08-23 19:51:51.403 4132 krsh.c
    Error 12154 received logging on to the standby
    *** 2013-08-23 19:51:51.403 869 krsu.c
    Error 12154 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'HPYMUSIC'
    Error 12154 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'HPYMUSIC'
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    *** 2013-08-23 19:51:51.403 4132 krsh.c
    PING[ARC2]: Heartbeat failed to connect to standby 'HPYMUSIC'. Error is 12154.
    *** 2013-08-23 19:51:51.403 2747 krsi.c
    krsi_dst_fail: dest:2 err:12154 force:0 blast:1
    [root@tokyo1 app]#
    [grid@tokyo1 shell]$ ./grid_info_east-x.sh
    + export ORACLE_SID=+ASM
    + ORACLE_SID=+ASM
    + LOGDIR=/home/grid/log
    + PRIMARYDB=east_DGMGRL
    + STANDBYDB=
    + PASSWORD=dataguard
    + mkdir -p /home/grid/log
    ++ date +%y%m%d,%H%M%S
    + echo 'asm info,130823,195709'
    + sqlplus / as sysasm
    SQL*Plus: Release 11.2.0.1.0 Production on 金 8月 23 19:57:09 2013
    Copyright (c) 1982, 2009, Oracle.  All rights reserved.
    Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    With the Automatic Storage Management option
    に接続されました。
    SQL> SQL>
    SYSDATE
    13-08-23
    SQL> SQL> SQL>
    NAME                                 TYPE                VALUE
    asm_diskgroups                       string              FRA
    asm_diskstring                       string              /dev/sd*1
    asm_power_limit                      integer             1
    asm_preferred_read_failure_groups    string
    audit_file_dest                      string              /u01/app/11.2.0/grid/rdbms/audit
    audit_sys_operations                 boolean             FALSE
    audit_syslog_level                   string
    background_core_dump                 string              partial
    background_dump_dest                 string              /u01/app/grid/diag/asm/+asm/+ASM/trace
    cluster_database                     boolean             FALSE
    cluster_database_instances           integer             1
    cluster_interconnects                string
    core_dump_dest                       string              /u01/app/grid/diag/asm/+asm/+ASM/cdump
    db_cache_size                        big integer         0
    db_ultra_safe                        string              OFF
    db_unique_name                       string              +ASM
    diagnostic_dest                      string              /u01/app/grid
    event                                string
    file_mapping                         boolean             FALSE
    filesystemio_options                 string              none
    ifile                                file
    instance_name                        string              +ASM
    instance_number                      integer             1
    instance_type                        string              asm
    large_pool_size                      big integer         12M
    ldap_directory_sysauth               string              no
    listener_networks                    string
    local_listener                       string
    lock_name_space                      string
    lock_sga                             boolean             FALSE
    max_dump_file_size                   string              unlimited
    memory_max_target                    big integer         272M
    memory_target                        big integer         272M
    nls_calendar                         string
    nls_comp                             string              BINARY
    nls_currency                         string
    nls_date_format                      string
    nls_date_language                    string
    nls_dual_currency                    string
    nls_iso_currency                     string
    nls_language                         string              AMERICAN
    nls_length_semantics                 string              BYTE
    nls_nchar_conv_excp                  string              FALSE
    nls_numeric_characters               string
    nls_sort                             string
    nls_territory                        string              AMERICA
    nls_time_format                      string
    nls_time_tz_format                   string
    nls_timestamp_format                 string
    nls_timestamp_tz_format              string
    os_authent_prefix                    string              ops$
    os_roles                             boolean             FALSE
    pga_aggregate_target                 big integer         0
    processes                            integer             100
    remote_listener                      string
    remote_login_passwordfile            string              EXCLUSIVE
    remote_os_authent                    boolean             FALSE
    remote_os_roles                      boolean             FALSE
    service_names                        string              +ASM
    sessions                             integer             172
    sga_max_size                         big integer         272M
    sga_target                           big integer         0
    shadow_core_dump                     string              partial
    shared_pool_reserved_size            big integer         6081740
    shared_pool_size                     big integer         0
    sort_area_size                       integer             65536
    spfile                               string              +DATA/asm/asmparameterfile/registry.253.823204697
    sql_trace                            boolean             FALSE
    statistics_level                     string              TYPICAL
    timed_os_statistics                  integer             0
    timed_statistics                     boolean             TRUE
    trace_enabled                        boolean             TRUE
    user_dump_dest                       string              /u01/app/grid/diag/asm/+asm/+ASM/trace
    workarea_size_policy                 string              AUTO
    ++ date +%y%m%d,%H%M%S
    + echo 'asmcmd info,130823,195709'
    + asmcmd ls -l
    State    Type    Rebal  Name
    MOUNTED  NORMAL  N      DATA/
    MOUNTED  NORMAL  N      FRA/
    + asmcmd ls -l 'data/asm/*'
    Type              Redund  Striped  Time             Sys  Name
    ASMPARAMETERFILE  MIRROR  COARSE   AUG 11 19:00:00  Y    REGISTRY.253.823204697
    + asmcmd ls -l 'data/east/*'
    Type           Redund  Striped  Time             Sys  Name
    +data/east/CONTROLFILE/:
    CONTROLFILE    HIGH    FINE     AUG 12 15:00:00  Y    Current.260.823276231
    CONTROLFILE    HIGH    FINE     AUG 23 19:00:00  Y    Current.270.823277705
    +data/east/DATAFILE/:
    DATAFILE       MIRROR  COARSE   AUG 12 15:00:00  Y    SYSAUX.257.823276133
    DATAFILE       MIRROR  COARSE   AUG 23 19:00:00  Y    SYSAUX.267.823277615
    DATAFILE       MIRROR  COARSE   AUG 12 15:00:00  Y    SYSTEM.256.823276131
    DATAFILE       MIRROR  COARSE   AUG 23 19:00:00  Y    SYSTEM.266.823277615
    DATAFILE       MIRROR  COARSE   AUG 12 15:00:00  Y    UNDOTBS1.258.823276133
    DATAFILE       MIRROR  COARSE   AUG 23 19:00:00  Y    UNDOTBS1.268.823277615
    DATAFILE       MIRROR  COARSE   AUG 12 15:00:00  Y    USERS.259.823276133
    DATAFILE       MIRROR  COARSE   AUG 23 19:00:00  Y    USERS.269.823277615
    +data/east/ONLINELOG/:
    ONLINELOG      MIRROR  COARSE   AUG 12 15:00:00  Y    group_1.261.823276235
    ONLINELOG      MIRROR  COARSE   AUG 12 15:00:00  Y    group_2.262.823276241
    ONLINELOG      MIRROR  COARSE   AUG 12 15:00:00  Y    group_3.263.823276247
    +data/east/PARAMETERFILE/:
    PARAMETERFILE  MIRROR  COARSE   AUG 23 12:00:00  Y    spfile.265.823277967
    +data/east/TEMPFILE/:
    TEMPFILE       MIRROR  COARSE   AUG 12 15:00:00  Y    TEMP.264.823276263
    TEMPFILE       MIRROR  COARSE   AUG 23 19:00:00  Y    TEMP.274.823277733
                                                     N    spfileeast.ora => +DATA/EAST/PARAMETERFILE/spfile.265.823277967
    + asmcmd ls -l 'fra/east/*'
    Type         Redund  Striped  Time             Sys  Name
    +fra/east/ARCHIVELOG/:
                                                   Y    2013_08_12/
                                                   Y    2013_08_15/
                                                   Y    2013_08_19/
                                                   Y    2013_08_22/
                                                   Y    2013_08_23/
    +fra/east/CONTROLFILE/:
    CONTROLFILE  HIGH    FINE     AUG 12 15:00:00  Y    Current.256.823276231
    CONTROLFILE  HIGH    FINE     AUG 23 19:00:00  Y    Current.260.823277707
    +fra/east/ONLINELOG/:
    ONLINELOG    MIRROR  COARSE   AUG 12 15:00:00  Y    group_1.257.823276237
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_10.272.823535727
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_11.273.823535737
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_12.274.823535745
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_13.275.823535757
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_14.276.823535763
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_15.277.823535771
    ONLINELOG    MIRROR  COARSE   AUG 12 15:00:00  Y    group_2.258.823276245
    ONLINELOG    MIRROR  COARSE   AUG 12 15:00:00  Y    group_3.259.823276251
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_7.269.823535685
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_8.270.823535695
    ONLINELOG    MIRROR  COARSE   AUG 23 19:00:00  Y    group_9.271.823535703
    +fra/east/STANDBYLOG/:
                                                   N    standby_group_07.log => +FRA/EAST/ONLINELOG/group_7.269.823535685
                                                   N    standby_group_08.log => +FRA/EAST/ONLINELOG/group_8.270.823535695
                                                   N    standby_group_09.log => +FRA/EAST/ONLINELOG/group_9.271.823535703
                                                   N    standby_group_10.log => +FRA/EAST/ONLINELOG/group_10.272.823535727
                                                   N    standby_group_11.log => +FRA/EAST/ONLINELOG/group_11.273.823535737
                                                   N    standby_group_12.log => +FRA/EAST/ONLINELOG/group_12.274.823535745
                                                   N    standby_group_13.log => +FRA/EAST/ONLINELOG/group_13.275.823535757
                                                   N    standby_group_14.log => +FRA/EAST/ONLINELOG/group_14.276.823535763
                                                   N    standby_group_15.log => +FRA/EAST/ONLINELOG/group_15.277.823535771
    + asmcmd find +data 'group*'
    +data/EAST/ONLINELOG/group_1.261.823276235
    +data/EAST/ONLINELOG/group_2.262.823276241
    +data/EAST/ONLINELOG/group_3.263.823276247
    +data/HPYMUSIC/ONLINELOG/group_1.271.824213881
    +data/HPYMUSIC/ONLINELOG/group_2.272.824213887
    +data/HPYMUSIC/ONLINELOG/group_3.273.824213895
    + asmcmd find +data 'spf*'
    +data/EAST/PARAMETERFILE/spfile.265.823277967
    +data/EAST/spfileeast.ora
    + asmcmd ls -l data/east/CONTROLFILE
    Type         Redund  Striped  Time             Sys  Name
    CONTROLFILE  HIGH    FINE     AUG 12 15:00:00  Y    Current.260.823276231
    CONTROLFILE  HIGH    FINE     AUG 23 19:00:00  Y    Current.270.823277705
    + asmcmd find +fra 'group*'
    +fra/EAST/ONLINELOG/group_1.257.823276237
    +fra/EAST/ONLINELOG/group_10.272.823535727
    +fra/EAST/ONLINELOG/group_11.273.823535737
    +fra/EAST/ONLINELOG/group_12.274.823535745
    +fra/EAST/ONLINELOG/group_13.275.823535757
    +fra/EAST/ONLINELOG/group_14.276.823535763
    +fra/EAST/ONLINELOG/group_15.277.823535771
    +fra/EAST/ONLINELOG/group_2.258.823276245
    +fra/EAST/ONLINELOG/group_3.259.823276251
    +fra/EAST/ONLINELOG/group_7.269.823535685
    +fra/EAST/ONLINELOG/group_8.270.823535695
    +fra/EAST/ONLINELOG/group_9.271.823535703
    +fra/HPYMUSIC/ONLINELOG/group_1.261.824213883
    +fra/HPYMUSIC/ONLINELOG/group_2.262.824213889
    +fra/HPYMUSIC/ONLINELOG/group_3.263.824213897
    + asmcmd find +fra 'spf*'
    + asmcmd ls -l fra/east/CONTROLFILE
    Type         Redund  Striped  Time             Sys  Name
    CONTROLFILE  HIGH    FINE     AUG 12 15:00:00  Y    Current.256.823276231
    CONTROLFILE  HIGH    FINE     AUG 23 19:00:00  Y    Current.260.823277707
    ++ date +%y%m%d,%H%M%S
    + echo END,130823,195712
    [grid@tokyo1 shell]$
    ■ 以下、スタンバイ側 ■ ■ ■ ■ ■ ■ ■
    export ORACLE_SID=HPYMUSIC
    sqlplus / as sysdba
    startup nomount pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt'
    create spfile='+data/west/spfilewest.ora' from pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt';
    srvctl stop database -d west -f
    srvctl start database -d west -o open
    srvctl start database -d west -o mount
    srvctl start database -d west
    startup mount pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt'
    srvctl start database -d west -o open
    srvctl config database -d west
    srvctl status database -d west
    alter database recover managed standby database disconnect from session;
    select name, database_role, open_mode from gv$database;
    srvctl modify database -d west -s open
    ○ spfile を作成する
    export ORACLE_SID=HPYMUSIC
    sqlplus / as sysdba
    startup nomount pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt'
    create spfile='+data/west/spfilewest.ora' from pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt';
    ○ 落とす
    srvctl stop database -d west -f
    ○ open したいが起動しない（ Mounted (Closed) 状態で起動することもある）。
    srvctl start database -d west -o open
    PRCR-1079 : リソースora.west.dbの起動に失敗しました
    CRS-2674: Start of 'ora.west.db' on 'osaka1' failed
    ○ open したいが起動しない（ Mounted (Closed) 状態で起動することもある）。
    srvctl start database -d west -o mount
    PRCR-1079 : リソースora.west.dbの起動に失敗しました
    CRS-2674: Start of 'ora.west.db' on 'osaka1' failed
    ○ open したいが起動しない（ Mounted (Closed) 状態で起動することもある）。
    srvctl start database -d west
    PRCR-1079 : リソースora.west.dbの起動に失敗しました
    CRS-2674: Start of 'ora.west.db' on 'osaka1' failed
    ○ 起動するがエラーあり（ alert_HPYMUSIC.log ）
    startup mount pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt'
    [oracle@osaka1 dbs]$ sqlplus / as sysdba
    SQL*Plus: Release 11.2.0.1.0 Production on 金 8月 23 19:05:35 2013
    Copyright (c) 1982, 2009, Oracle.  All rights reserved.
    アイドル・インスタンスに接続しました。
    SQL> startup mount pfile='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/pfile_for_standby_HPYMUSIC.txt'
    ORACLEインスタンスが起動しました。
    Total System Global Area  839282688 bytes
    Fixed Size                  2217992 bytes
    Variable Size             515901432 bytes
    Database Buffers          314572800 bytes
    Redo Buffers                6590464 bytes
    データベースがマウントされました。
    Error 12154 received logging on to the standby
    FAL[client, ARC3]: Error 12154 connecting to HPYMUSIC for fetching gap sequence
    Errors in file /u01/app/oracle/diag/rdbms/west/HPYMUSIC/trace/HPYMUSIC_arc3_25690.trc:
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    Errors in file /u01/app/oracle/diag/rdbms/west/HPYMUSIC/trace/HPYMUSIC_arc3_25690.trc:
    ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    ○ open にならず Mounted (Closed) としかなってくれない。
    srvctl start database -d west -o open
    [oracle@osaka1 dbs]$ srvctl start database -d west -o open
    [oracle@osaka1 dbs]$ /u01/app/11.2.0/grid/bin/crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS
    Local Resources
    ora.DATA.dg
                   ONLINE  ONLINE       osaka1
    ora.FRA.dg
                   ONLINE  ONLINE       osaka1
    ora.LISTENER.lsnr
                   ONLINE  ONLINE       osaka1
    ora.asm
                   ONLINE  ONLINE       osaka1                   Started
    Cluster Resources
    ora.cssd
          1        ONLINE  ONLINE       osaka1
    ora.diskmon
          1        ONLINE  ONLINE       osaka1
    ora.west.db
          1        ONLINE  INTERMEDIATE osaka1                   Mounted (Closed)
    ○
    srvctl config database -d west
    srvctl status database -d west
    [oracle@osaka1 dbs]$ srvctl config database -d west
    一意のデータベース名: west
    データベース名: HPYMUSIC
    Oracleホーム: /u01/app/oracle/product/11.2.0/dbhome_1
    Oracleユーザー: grid
    spfile: +data/west/spfilewest.ora
    ドメイン:
    開始オプション: open
    停止オプション: immediate
    データベース・ロール: physical_standby
    管理ポリシー: AUTOMATIC
    ディスク・グループ: DATA,FRA
    サービス:
    [oracle@osaka1 dbs]$ srvctl status database -d west
    データベースは実行中です。
    ○ mrp プロセスが起動するが、Read Only ではない。
    alter database recover managed standby database disconnect from session;
    select name, database_role, open_mode from gv$database;
    [oracle@osaka1 dbs]$ sqlplus / as sysdba
    SQL*Plus: Release 11.2.0.1.0 Production on 金 8月 23 19:33:08 2013
    Copyright (c) 1982, 2009, Oracle.  All rights reserved.
    Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    With the Partitioning, Automatic Storage Management, OLAP, Data Mining
    and Real Application Testing options
    に接続されました。
    SQL>
    SQL> alter database recover managed standby database disconnect from session;
    データベースが変更されました。
    SQL> select name, database_role, open_mode from gv$database;
    NAME                        DATABASE_ROLE
    OPEN_MODE
    HPYMUSIC                    PHYSICAL STANDBY
    MOUNTED
    [root@osaka1 app]# ps -ef |egrep -i mrp
    oracle   26269     1  0 19:33 ?        00:00:00 ora_mrp0_HPYMUSIC
    ○ modify しても open にならない。
    srvctl modify database -d west -s open
    [oracle@osaka1 dbs]$ srvctl modify database -d west -s open
    [oracle@osaka1 dbs]$ /u01/app/11.2.0/grid/bin/crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS
    Local Resources
    ora.DATA.dg
                   ONLINE  ONLINE       osaka1
    ora.FRA.dg
                   ONLINE  ONLINE       osaka1
    ora.LISTENER.lsnr
                   ONLINE  ONLINE       osaka1
    ora.asm
                   ONLINE  ONLINE       osaka1                   Started
    Cluster Resources
    ora.cssd
          1        ONLINE  ONLINE       osaka1
    ora.diskmon
          1        ONLINE  ONLINE       osaka1
    ora.west.db
          1        ONLINE  INTERMEDIATE osaka1                   Mounted (Closed)

    standby 側に、アーカイブログファイルを少し適用すれば open できそうに感じます。
    これを解決するためには、以下のエラーを解決するのがよいと思います。
    ・ORA-12154: TNS: 指定された接続識別子を解決できませんでした
    両ノードの tnsnames.ora を確認させて貰えますか？
    この推測が間違えていたら再検討しましょう。

  • ORA-16191: Primary log shipping client not logged on standby.

    Hi,
    Please help me in the following scenario. I have two nodes ASM1 & ASM2 with RHEL4 U5 OS. On node ASM1 there is database ORCL using ASM diskgroups DATA & RECOVER and archive location is on '+RECOVER/orcl/'. On ASM2 node, I have to configure STDBYORCL (standby) database using ASM. I have taken the copy of database ORCL via RMAN, as per maximum availability architecture.
    Then I have ftp'd all to ASM2 and put them on FS /u01/oradata. Have made all necessary changes in primary and standby database pfile and then perform the duplicate database for standby using RMAN in order to put the db files in desired diskgroups. I have mounted the standby database but unfortunately, log transport service is not working and archives are not getting shipped to standby host.
    Here are all configuration details.
    Primary database ORCL pfile:
    [oracle@asm dbs]$ more initorcl.ora
    stdbyorcl.__db_cache_size=251658240
    orcl.__db_cache_size=226492416
    stdbyorcl.__java_pool_size=4194304
    orcl.__java_pool_size=4194304
    stdbyorcl.__large_pool_size=4194304
    orcl.__large_pool_size=4194304
    stdbyorcl.__shared_pool_size=100663296
    orcl.__shared_pool_size=125829120
    stdbyorcl.__streams_pool_size=0
    orcl.__streams_pool_size=0
    *.audit_file_dest='/opt/oracle/admin/orcl/adump'
    *.background_dump_dest='/opt/oracle/admin/orcl/bdump'
    *.compatible='10.2.0.1.0'
    *.control_files='+DATA/orcl/controlfile/current.270.665007729','+RECOVER/orcl/controlfile/current.262.665007731'
    *.core_dump_dest='/opt/oracle/admin/orcl/cdump'
    *.db_block_size=8192
    *.db_create_file_dest='+DATA'
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_name='orcl'
    *.db_recovery_file_dest='+RECOVER'
    *.db_recovery_file_dest_size=3163553792
    *.db_unique_name=orcl
    *.fal_client=orcl
    *.fal_server=stdbyorcl
    *.instance_name='orcl'
    *.job_queue_processes=10
    *.log_archive_config='dg_config=(orcl,stdbyorcl)'
    *.log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST'
    *.log_archive_dest_2='SERVICE=stdbyorcl'
    *.log_archive_dest_state_1='ENABLE'
    *.log_archive_dest_state_2='ENABLE'
    *.log_archive_format='%t_%s_%r.dbf'
    *.open_cursors=300
    *.pga_aggregate_target=121634816
    *.processes=150
    *.remote_login_passwordfile='EXCLUSIVE'
    *.sga_target=364904448
    *.standby_file_management='AUTO'
    *.undo_management='AUTO'
    *.undo_tablespace='UNDOTBS'
    *.user_dump_dest='/opt/oracle/admin/orcl/udump'
    Standby database STDBYORCL pfile:
    [oracle@asm2 dbs]$ more initstdbyorcl.ora
    stdbyorcl.__db_cache_size=251658240
    stdbyorcl.__java_pool_size=4194304
    stdbyorcl.__large_pool_size=4194304
    stdbyorcl.__shared_pool_size=100663296
    stdbyorcl.__streams_pool_size=0
    *.audit_file_dest='/opt/oracle/admin/stdbyorcl/adump'
    *.background_dump_dest='/opt/oracle/admin/stdbyorcl/bdump'
    *.compatible='10.2.0.1.0'
    *.control_files='u01/oradata/stdbyorcl_control01.ctl'#Restore Controlfile
    *.core_dump_dest='/opt/oracle/admin/stdbyorcl/cdump'
    *.db_block_size=8192
    *.db_create_file_dest='/u01/oradata'
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_name='orcl'
    *.db_recovery_file_dest='+RECOVER'
    *.db_recovery_file_dest_size=3163553792
    *.db_unique_name=stdbyorcl
    *.fal_client=stdbyorcl
    *.fal_server=orcl
    *.instance_name='stdbyorcl'
    *.job_queue_processes=10
    *.log_archive_config='dg_config=(orcl,stdbyorcl)'
    *.log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST'
    *.log_archive_dest_2='SERVICE=orcl'
    *.log_archive_dest_state_1='ENABLE'
    *.log_archive_dest_state_2='ENABLE'
    *.log_archive_format='%t_%s_%r.dbf'
    *.log_archive_start=TRUE
    *.open_cursors=300
    *.pga_aggregate_target=121634816
    *.processes=150
    *.remote_login_passwordfile='EXCLUSIVE'
    *.sga_target=364904448
    *.standby_archive_dest='LOCATION=USE_DB_RECOVERY_FILE_DEST'
    *.standby_file_management='AUTO'
    *.undo_management='AUTO'
    *.undo_tablespace='UNDOTBS'
    *.user_dump_dest='/opt/oracle/admin/stdbyorcl/udump'
    db_file_name_convert=('+DATA/ORCL/DATAFILE','/u01/oradata','+RECOVER/ORCL/DATAFILE','/u01/oradata')
    log_file_name_convert=('+DATA/ORCL/ONLINELOG','/u01/oradata','+RECOVER/ORCL/ONLINELOG','/u01/oradata')
    Have configured the tns service on both the hosts and its working absolutely fine.
    <p>
    ASM1
    =====
    [oracle@asm dbs]$ tnsping stdbyorcl
    </p>
    <p>
    TNS Ping Utility for Linux: Version 10.2.0.1.0 - Production on 19-SEP-2008 18:49:00
    </p>
    <p>
    Copyright (c) 1997, 2005, Oracle. All rights reserved.
    </p>
    <p>
    Used parameter files:
    </p>
    <p>
    Used TNSNAMES adapter to resolve the alias
    Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.20)(PORT = 1521))) (CONNECT_DATA = (SID = stdbyorcl) (SERVER = DEDICATED)))
    OK (30 msec)
    ASM2
    =====
    </p>
    <p>
    [oracle@asm2 archive]$ tnsping orcl
    </p>
    <p>
    TNS Ping Utility for Linux: Version 10.2.0.1.0 - Production on 19-SEP-2008 18:48:39
    </p>
    <p>
    Copyright (c) 1997, 2005, Oracle. All rights reserved.
    </p>
    <p>
    Used parameter files:
    </p>
    <p>
    Used TNSNAMES adapter to resolve the alias
    Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.10)(PORT = 1521))) (CONNECT_DATA = (SID = orcl) (SERVER = DEDICATED)))
    OK (30 msec)
    Please guide where I am missing. Thanking you in anticipation.
    Regards,
    Ravish Garg

    Following are the errors I am receiving as per alert log.
    ORCL alert log:
    Thu Sep 25 17:49:14 2008
    ARCH: Possible network disconnect with primary database
    Thu Sep 25 17:49:14 2008
    Error 1031 received logging on to the standby
    Thu Sep 25 17:49:14 2008
    Errors in file /opt/oracle/admin/orcl/bdump/orcl_arc1_4825.trc:
    ORA-01031: insufficient privileges
    FAL[server, ARC1]: Error 1031 creating remote archivelog file 'STDBYORCL'
    FAL[server, ARC1]: FAL archive failed, see trace file.
    Thu Sep 25 17:49:14 2008
    Errors in file /opt/oracle/admin/orcl/bdump/orcl_arc1_4825.trc:
    ORA-16055: FAL request rejected
    ARCH: FAL archive failed. Archiver continuing
    Thu Sep 25 17:49:14 2008
    ORACLE Instance orcl - Archival Error. Archiver continuing.
    Thu Sep 25 17:49:44 2008
    FAL[server]: Fail to queue the whole FAL gap
    GAP - thread 1 sequence 40-40
    DBID 1192788465 branch 665007733
    Thu Sep 25 17:49:46 2008
    Thread 1 advanced to log sequence 48
    Current log# 2 seq# 48 mem# 0: +DATA/orcl/onlinelog/group_2.272.665007735
    Current log# 2 seq# 48 mem# 1: +RECOVER/orcl/onlinelog/group_2.264.665007737
    Thu Sep 25 17:55:43 2008
    Shutting down archive processes
    Thu Sep 25 17:55:48 2008
    ARCH shutting down
    ARC2: Archival stopped
    STDBYORCL alert log:
    ==============
    Thu Sep 25 17:49:27 2008
    Errors in file /opt/oracle/admin/stdbyorcl/bdump/stdbyorcl_arc0_4813.trc:
    ORA-01017: invalid username/password; logon denied
    Thu Sep 25 17:49:27 2008
    Error 1017 received logging on to the standby
    Check that the primary and standby are using a password file
    and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
    and that the SYS password is same in the password files.
    returning error ORA-16191
    It may be necessary to define the DB_ALLOWED_LOGON_VERSION
    initialization parameter to the value "10". Check the
    manual for information on this initialization parameter.
    Thu Sep 25 17:49:27 2008
    Errors in file /opt/oracle/admin/stdbyorcl/bdump/stdbyorcl_arc0_4813.trc:
    ORA-16191: Primary log shipping client not logged on standby
    PING[ARC0]: Heartbeat failed to connect to standby 'orcl'. Error is 16191.
    Thu Sep 25 17:51:38 2008
    FAL[client]: Failed to request gap sequence
    GAP - thread 1 sequence 40-40
    DBID 1192788465 branch 665007733
    FAL[client]: All defined FAL servers have been attempted.
    Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
    parameter is defined to a value that is sufficiently large
    enough to maintain adequate log switch information to resolve
    archivelog gaps.
    Thu Sep 25 17:55:16 2008
    Errors in file /opt/oracle/admin/stdbyorcl/bdump/stdbyorcl_arc0_4813.trc:
    ORA-01017: invalid username/password; logon denied
    Thu Sep 25 17:55:16 2008
    Error 1017 received logging on to the standby
    Check that the primary and standby are using a password file
    and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
    and that the SYS password is same in the password files.
    returning error ORA-16191
    It may be necessary to define the DB_ALLOWED_LOGON_VERSION
    initialization parameter to the value "10". Check the
    manual for information on this initialization parameter.
    Thu Sep 25 17:55:16 2008
    Errors in file /opt/oracle/admin/stdbyorcl/bdump/stdbyorcl_arc0_4813.trc:
    ORA-16191: Primary log shipping client not logged on standby
    PING[ARC0]: Heartbeat failed to connect to standby 'orcl'. Error is 16191.
    Please suggest where I am missing.
    Regards,
    Ravish Garg

  • Error 472 in job queue process + ORA-472: PMON  process terminated with err

    Hi,
    I've an Oracle Database 11g Release 11.1.0.6.0 - 64bit Production With the Real Application Clusters option.
    Yesterday instance on node1 crashed.
    This is the extract of alert log:
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc (incident=195198):
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    Incident details in: /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195198/EVODB1_pmon_9297_i195198.trc
    Sat Jan 22 16:24:47 2011
    Trace dumping is performing id=[cdmp_20110122162447]
    Sat Jan 22 16:24:50 2011
    Sweep Incident[195198]: completed
    Sat Jan 22 16:25:00 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc (incident=195199):
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    Incident details in: /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195199/EVODB1_pmon_9297_i195199.trc
    Sat Jan 22 16:25:10 2011
    Trace dumping is performing id=[cdmp_20110122162510]
    Sat Jan 22 16:25:12 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc (incident=195200):
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    Incident details in: /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195200/EVODB1_pmon_9297_i195200.trc
    Sat Jan 22 16:25:23 2011
    DBRM (ospid: 9310): terminating the instance due to error 472
    Sat Jan 22 16:25:23 2011
    System state dump is made for local instance
    System State dumped to trace file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_diag_16320.trc
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3724_47445977853072)
    Sat Jan 22 16:25:23 2011
    SUCCESS: diskgroup ARCHIVELOG was dismounted
    Sat Jan 22 16:14:22 2011
    DDE: Problem Key 'ORA 600 [17183]' was completely flood controlled (0x6)
    Further messages for this problem key will be suppressed for up to 10 minutes
    Sat Jan 22 16:15:56 2011
    Thread 1 advanced to log sequence 171322
    Current log# 18 seq# 171322 mem# 0: +ONLINELOG/evodb/onlinelog/group_18.264.729348055
    Sat Jan 22 16:15:56 2011
    SUCCESS: diskgroup ARCHIVELOG was mounted
    Sat Jan 22 16:16:15 2011
    SUCCESS: diskgroup ARCHIVELOG was dismounted
    Sat Jan 22 16:20:52 2011
    Thread 1 advanced to log sequence 171323
    Current log# 12 seq# 171323 mem# 0: +ONLINELOG/evodb/onlinelog/group_12.267.729333861
    Sat Jan 22 16:20:53 2011
    SUCCESS: diskgroup ARCHIVELOG was mounted
    Sat Jan 22 16:21:14 2011
    SUCCESS: diskgroup ARCHIVELOG was dismounted
    Sat Jan 22 16:24:23 2011
    DDE: Problem Key 'ORA 600 [17183]' was completely flood controlled (0x2)
    Further messages for this problem key will be suppressed for up to 10 minutes
    Sat Jan 22 16:24:38 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc (incident=195198):
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    Incident details in: /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195198/EVODB1_pmon_9297_i195198.trc
    Sat Jan 22 16:24:47 2011
    Trace dumping is performing id=[cdmp_20110122162447]
    Sat Jan 22 16:24:50 2011
    Sweep Incident[195198]: completed
    Sat Jan 22 16:25:00 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc (incident=195199):
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    Incident details in: /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195199/EVODB1_pmon_9297_i195199.trc
    Sat Jan 22 16:25:10 2011
    Trace dumping is performing id=[cdmp_20110122162510]
    Sat Jan 22 16:25:12 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc (incident=195200):
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    Incident details in: /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195200/EVODB1_pmon_9297_i195200.trc
    Sat Jan 22 16:25:23 2011
    DBRM (ospid: 9310): terminating the instance due to error 472
    Sat Jan 22 16:25:23 2011
    System state dump is made for local instance
    System State dumped to trace file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_diag_16320.trc
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3724_47445977853072)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3650_47673206152336)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (19709_47962498952336)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3715_47239602621584)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3718_47120460606608)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3646_47899813694608)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3671_47700480982160)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3637_47223057443984)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3695_47109627223184)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3679_47262512999568)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3733_46976527121552)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3667_47474016554128)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3669_47585103858832)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3658_47992462539920)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3654_47736794324112)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3686_47295241862288)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3706_47055349939344)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3630_47331091332240)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3713_47194150415504)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3726_47289949199504)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3677_47842172337296)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (19131_47557370236048)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (27091_47096969530512)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3688_47422639070352)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3652_47944422886544)
    Sat Jan 22 16:25:23 2011
    ORA-1092 : opiodr aborting process unknown ospid (3648_47426149817488)
    Sat Jan 22 16:25:24 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j005_9360.trc:
    ORA-00472: PMON  process terminated with error
    ORA-1092 : opidrv aborting process J005 ospid (9360_47420959304848)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (18393_47566684101776)
    Sat Jan 22 16:25:24 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j003_7054.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process J003 ospid (7054_47244451724432)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (19719_47840732109968)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (22184_47575322619024)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (19721_47679094451344)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (30170_47106572815504)
    Sat Jan 22 16:25:24 2011
    ORA-472 : opidrv aborting process S000 ospid (9367_46990649617552)
    Sat Jan 22 16:25:24 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j002_7052.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process J002 ospid (7052_47411695402128)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (18493_47469336738960)
    Sat Jan 22 16:25:24 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j006_10011.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process J006 ospid (10011_47836408102032)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (19714_47958179912848)
    Sat Jan 22 16:25:24 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j007_10289.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process J007 ospid (10289_47356480347280)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opiodr aborting process unknown ospid (9076_47765392362640)
    Sat Jan 22 16:25:24 2011
    ORA-1092 : opitsk aborting process
    Sat Jan 22 16:25:25 2011
    ORA-1092 : opiodr aborting process unknown ospid (22167_47949919177872)
    Sat Jan 22 16:25:25 2011
    ORA-1092 : opiodr aborting process unknown ospid (24320_47408077532304)
    Sat Jan 22 16:25:25 2011
    ORA-1092 : opiodr aborting process unknown ospid (22130_47998542101648)
    Sat Jan 22 16:25:25 2011
    ORA-472 : opidrv aborting process N000 ospid (14774_47466200525968)
    Sat Jan 22 16:25:25 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_q000_14701.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process Q000 ospid (14701_47693688122512)
    Sat Jan 22 16:25:25 2011
    ORA-1092 : opiodr aborting process unknown ospid (18386_47900707175568)
    Sat Jan 22 16:25:25 2011
    ORA-1092 : opiodr aborting process unknown ospid (27553_46921398619280)
    Sat Jan 22 16:25:26 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j001_7050.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process J001 ospid (7050_47290016251024)
    Sat Jan 22 16:25:26 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_o001_5297.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process O001 ospid (5297_47702958673040)
    Sat Jan 22 16:25:26 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_o000_5277.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process O000 ospid (5277_47599668255888)
    Sat Jan 22 16:25:26 2011
    ORA-1092 : opiodr aborting process unknown ospid (19745_47560156560528)
    Sat Jan 22 16:25:26 2011
    ORA-1092 : opiodr aborting process unknown ospid (27044_47388543339664)
    Sat Jan 22 16:25:26 2011
    Errors in file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j004_7061.trc:
    ORA-00472: PMON process terminated with error
    ORA-1092 : opidrv aborting process J004 ospid (7061_47341656689808)
    Sat Jan 22 16:25:27 2011
    ORA-1092 : opiodr aborting process unknown ospid (14431_47662297960112)
    Sat Jan 22 16:25:27 2011
    ORA-1092 : opiodr aborting process unknown ospid (14433_47826608635568)
    Sat Jan 22 16:25:31 2011
    ORA-1092 : opitsk aborting process
    Sat Jan 22 16:25:32 2011
    License high water mark = 269
    Instance terminated by DBRM, pid = 9310
    USER (ospid: 10539): terminating the instance
    Instance terminated by USER, pid = 10539
    Sat Jan 22 16:25:49 2011
    Starting ORACLE instance (normal)
    This is the trace /u01/app/oracle/diag/rdbms/evodb/EVODB1/incident/incdir_195200/EVODB1_pmon_9297_i195200.trc
    (the one regarding the error DBRM (ospid: 9310): terminating the instance due to error 472)
    Oracle Database 11g Release 11.1.0.6.0 - 64bit Production
    With the Real Application Clusters option
    ORACLE_HOME = /u01/app/oracle/product/11.1.0/db1
    System name:    Linux
    Node name:      dcsrv-evodb01
    Release:        2.6.18-164.10.1.el5
    Version:        #1 SMP Wed Dec 30 18:35:28 EST 2009
    Machine:        x86_64
    Instance name: EVODB1
    Redo thread mounted by this instance: 1
    Oracle process number: 2
    Unix process pid: 9297, image: oracle@dcsrv-evodb01 (PMON)
    *** 2011-01-22 16:25:12.984
    *** SESSION ID:(884.1) 2011-01-22 16:25:12.984
    *** CLIENT ID:() 2011-01-22 16:25:12.984
    *** SERVICE NAME:(SYS$BACKGROUND) 2011-01-22 16:25:12.984
    *** MODULE NAME:() 2011-01-22 16:25:12.984
    *** ACTION NAME:() 2011-01-22 16:25:12.984
    Dump continued from file: /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_pmon_9297.trc
    ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []
    +========= Dump for incident 195200 (ORA 600 [17183]) ========+
    ----- Beginning of Customized Incident Dump(s) -----
    And this is the trace /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j005_9360.trc
    (the one regarding the error ORA-00472: PMON process terminated with error)
    Trace file /u01/app/oracle/diag/rdbms/evodb/EVODB1/trace/EVODB1_j005_9360.trc
    Oracle Database 11g Release 11.1.0.6.0 - 64bit Production
    With the Real Application Clusters option
    ORACLE_HOME = /u01/app/oracle/product/11.1.0/db1
    System name:    Linux
    Node name:      dcsrv-evodb01
    Release:        2.6.18-164.10.1.el5
    Version:        #1 SMP Wed Dec 30 18:35:28 EST 2009
    Machine:        x86_64
    Instance name: EVODB1
    Redo thread mounted by this instance: 1
    Oracle process number: 95
    Unix process pid: 9360, image: oracle@dcsrv-evodb01 (J005)
    *** 2011-01-22 16:25:24.057
    *** SESSION ID:(649.45863) 2011-01-22 16:25:24.057
    *** CLIENT ID:() 2011-01-22 16:25:24.057
    *** SERVICE NAME:(SYS$USERS) 2011-01-22 16:25:24.057
    *** MODULE NAME:() 2011-01-22 16:25:24.057
    *** ACTION NAME:() 2011-01-22 16:25:24.057
    error 472 in job queue process
    ORA-00472: PMON  process terminated with error
    My questions are:
    - how can I discover the nature of the ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []?
    - what does error 472 in job queue proces mean?
    - is it normal the the error 472 in job queue proces crash the instance?
    Thanks in advanca for any kind of help!
    Samuel

    >
    how can I discover the nature of the ORA-00600: internal error code, arguments: [17183], [0x14E251C38], [], [], [], [], [], []?
    >
    ORA-00600 should be reported to oracle. Only they know the meaning.
    Regards,
    - wiZ

  • Archive Logs NOT APPLIED but transferred

    Hi Gurus,
    I have configured Primary & Standby databases in same Oracle Home. OS version is OEL 5. Database version is 10.2.0.1. I could get the archive logs in the standby site but they are not getting applied in the standby database. I don't have OLAP installed in my database version. Would this create this issue? However I attached my primary alert log details below for your reference:
    Thu Aug 30 23:55:37 2012
    Starting ORACLE instance (normal)
    Cannot determine all dependent dynamic libraries for /proc/self/exe
    Unable to find dynamic library libocr10.so in search paths
    RPATH = /ade/aime1_build2101/oracle/has/lib/:/ade/aime1_build2101/oracle/lib/:/ade/aime1_build2101/oracle/has/lib/:
    LD_LIBRARY_PATH is not set!
    The default library directories are /lib and /usr/lib
    Unable to find dynamic library libocrb10.so in search paths
    Unable to find dynamic library libocrutl10.so in search paths
    Unable to find dynamic library libocrutl10.so in search paths
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Picked latch-free SCN scheme 2
    Autotune of undo retention is turned on.
    IMODE=BR
    ILAT =18
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.1.0.
    System parameters with non-default values:
    processes = 150
    sga_target = 289406976
    control_files = /home/oracle/oracle/product/10.2.0/db_1/oradata/newprim/control01.ctl, /home/oracle/oracle/product/10.2.0/db_1/oradata/newprim/control02.ctl, /home/oracle/oracle/product/10.2.0/db_1/oradata/newprim/control03.ctl
    db_file_name_convert = /home/oracle/oracle/product/10.2.0/db_1/oradata/newstand, /home/oracle/oracle/product/10.2.0/db_1/oradata/newprim
    log_file_name_convert = /home/oracle/oracle/product/10.2.0/db_1/oradata/newstand, /home/oracle/oracle/product/10.2.0/db_1/oradata/newprim, /home/oracle/oracle/product/10.2.0/db_1/flash_recovery_area/NEWSTAND/onlinelog, /home/oracle/oracle/product/10.2.0/db_1/flash_recovery_area/NEWPRIM/onlinelog
    db_block_size = 8192
    compatible = 10.2.0.1.0
    log_archive_config = DG_CONFIG=(newprim,newstand)
    log_archive_dest_1 = LOCATION=/home/oracle/oracle/product/10.2.0/db_1/oradata/newprim/arch/
    VALID_FOR=(ALL_LOGFILES,ALL_ROLES)
    DB_UNIQUE_NAME=newprim
    log_archive_dest_2 = SERVICE=newstand LGWR ASYNC VALID_FOR=(online_logfiles,primary_role) DB_UNIQUE_NAME=newstand
    log_archive_dest_state_1 = enable
    log_archive_dest_state_2 = enable
    log_archive_max_processes= 30
    log_archive_format = %t_%s_%r.dbf
    fal_client = newprim
    fal_server = newstand
    db_file_multiblock_read_count= 16
    db_recovery_file_dest = /home/oracle/oracle/product/10.2.0/db_1/flash_recovery_area
    db_recovery_file_dest_size= 2147483648
    standby_file_management = AUTO
    undo_management = AUTO
    undo_tablespace = UNDOTBS1
    remote_login_passwordfile= EXCLUSIVE
    db_domain =
    dispatchers = (PROTOCOL=TCP) (SERVICE=newprimXDB)
    job_queue_processes = 10
    background_dump_dest = /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/bdump
    user_dump_dest = /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/udump
    core_dump_dest = /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/cdump
    audit_file_dest = /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/adump
    db_name = newprim
    db_unique_name = newprim
    open_cursors = 300
    pga_aggregate_target = 95420416
    PMON started with pid=2, OS id=28091
    PSP0 started with pid=3, OS id=28093
    MMAN started with pid=4, OS id=28095
    DBW0 started with pid=5, OS id=28097
    LGWR started with pid=6, OS id=28100
    CKPT started with pid=7, OS id=28102
    SMON started with pid=8, OS id=28104
    RECO started with pid=9, OS id=28106
    CJQ0 started with pid=10, OS id=28108
    MMON started with pid=11, OS id=28110
    MMNL started with pid=12, OS id=28112
    Thu Aug 30 23:55:38 2012
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    starting up 1 shared server(s) ...
    Thu Aug 30 23:55:38 2012
    ALTER DATABASE MOUNT
    Thu Aug 30 23:55:42 2012
    Setting recovery target incarnation to 2
    Thu Aug 30 23:55:43 2012
    Successful mount of redo thread 1, with mount id 1090395834
    Thu Aug 30 23:55:43 2012
    Database mounted in Exclusive Mode
    Completed: ALTER DATABASE MOUNT
    Thu Aug 30 23:55:43 2012
    ALTER DATABASE OPEN
    Thu Aug 30 23:55:43 2012
    LGWR: STARTING ARCH PROCESSES
    ARC0 started with pid=16, OS id=28122
    ARC1 started with pid=17, OS id=28124
    ARC2 started with pid=18, OS id=28126
    ARC3 started with pid=19, OS id=28128
    ARC4 started with pid=20, OS id=28133
    ARC5 started with pid=21, OS id=28135
    ARC6 started with pid=22, OS id=28137
    ARC7 started with pid=23, OS id=28139
    ARC8 started with pid=24, OS id=28141
    ARC9 started with pid=25, OS id=28143
    ARCa started with pid=26, OS id=28145
    ARCb started with pid=27, OS id=28147
    ARCc started with pid=28, OS id=28149
    ARCd started with pid=29, OS id=28151
    ARCe started with pid=30, OS id=28153
    ARCf started with pid=31, OS id=28155
    ARCg started with pid=32, OS id=28157
    ARCh started with pid=33, OS id=28159
    ARCi started with pid=34, OS id=28161
    ARCj started with pid=35, OS id=28163
    ARCk started with pid=36, OS id=28165
    ARCl started with pid=37, OS id=28167
    ARCm started with pid=38, OS id=28169
    ARCn started with pid=39, OS id=28171
    ARCo started with pid=40, OS id=28173
    ARCp started with pid=41, OS id=28175
    ARCq started with pid=42, OS id=28177
    ARCr started with pid=43, OS id=28179
    ARCs started with pid=44, OS id=28181
    Thu Aug 30 23:55:44 2012
    ARC0: Archival started
    ARC1: Archival started
    ARC2: Archival started
    ARC3: Archival started
    ARC4: Archival started
    ARC5: Archival started
    ARC6: Archival started
    ARC7: Archival started
    ARC8: Archival started
    ARC9: Archival started
    ARCa: Archival started
    ARCb: Archival started
    ARCc: Archival started
    ARCd: Archival started
    ARCe: Archival started
    ARCf: Archival started
    ARCg: Archival started
    ARCh: Archival started
    ARCi: Archival started
    ARCj: Archival started
    ARCk: Archival started
    ARCl: Archival started
    ARCm: Archival started
    ARCn: Archival started
    ARCo: Archival started
    ARCp: Archival started
    ARCq: Archival started
    ARCr: Archival started
    ARCs: Archival started
    ARCt: Archival started
    LGWR: STARTING ARCH PROCESSES COMPLETE
    ARCt started with pid=45, OS id=28183
    LNS1 started with pid=46, OS id=28185
    Thu Aug 30 23:55:48 2012
    Thread 1 advanced to log sequence 68
    Thu Aug 30 23:55:48 2012
    ARCo: Becoming the 'no FAL' ARCH
    ARCo: Becoming the 'no SRL' ARCH
    Thu Aug 30 23:55:48 2012
    ARCp: Becoming the heartbeat ARCH
    Thu Aug 30 23:55:48 2012
    Thread 1 opened at log sequence 68
    Current log# 1 seq# 68 mem# 0: /home/oracle/oracle/product/10.2.0/db_1/oradata/newprim/redo01.log
    Successful open of redo thread 1
    Thu Aug 30 23:55:48 2012
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Thu Aug 30 23:55:48 2012
    SMON: enabling cache recovery
    Thu Aug 30 23:55:48 2012
    Successfully onlined Undo Tablespace 1.
    Thu Aug 30 23:55:48 2012
    SMON: enabling tx recovery
    Thu Aug 30 23:55:49 2012
    Database Characterset is WE8ISO8859P1
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=47, OS id=28205
    Thu Aug 30 23:55:49 2012
    Error 1034 received logging on to the standby
    Thu Aug 30 23:55:49 2012
    Errors in file /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/bdump/newprim_arc1_28124.trc:
    ORA-01034: ORACLE not available
    FAL[server, ARC1]: Error 1034 creating remote archivelog file 'newstand'
    FAL[server, ARC1]: FAL archive failed, see trace file.
    Thu Aug 30 23:55:49 2012
    Errors in file /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/bdump/newprim_arc1_28124.trc:
    ORA-16055: FAL request rejected
    ARCH: FAL archive failed. Archiver continuing
    Thu Aug 30 23:55:49 2012
    ORACLE Instance newprim - Archival Error. Archiver continuing.
    Thu Aug 30 23:55:49 2012
    db_recovery_file_dest_size of 2048 MB is 9.77% used. This is a
    user-specified limit on the amount of space that will be used by this
    database for recovery-related files, and does not reflect the amount of
    space available in the underlying filesystem or ASM diskgroup.
    Thu Aug 30 23:55:50 2012
    Errors in file /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/udump/newprim_ora_28120.trc:
    ORA-00604: error occurred at recursive SQL level 1
    ORA-12663: Services required by client not available on the server
    ORA-36961: Oracle OLAP is not available.
    ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
    ORA-06512: at line 15
    Thu Aug 30 23:55:50 2012
    Completed: ALTER DATABASE OPEN
    Thu Aug 30 23:56:33 2012
    FAL[server]: Fail to queue the whole FAL gap
    GAP - thread 1 sequence 1-33
    DBID 1090398314 branch 792689455
    Kindly, guide me please..
    -Vimal.

    CKPT: The trace file details are added below for your reference;
    /home/oracle/oracle/product/10.2.0/db_1/admin/newprim/bdump/newprim_arc1_28124.trc
    Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
    With the Partitioning and Data Mining options
    ORACLE_HOME = /home/oracle/oracle/product/10.2.0/db_1
    System name:     Linux
    Node name:     localhost.localdomain
    Release:     2.6.18-8.el5PAE
    Version:     #1 SMP Tue Jun 5 23:39:57 EDT 2007
    Machine:     i686
    Instance name: newprim
    Redo thread mounted by this instance: 1
    Oracle process number: 17
    Unix process pid: 28124, image: [email protected] (ARC1)
    *** SERVICE NAME:() 2012-08-30 23:55:48.314
    *** SESSION ID:(155.1) 2012-08-30 23:55:48.314
    kcrrwkx: nothing to do (start)
    Redo shipping client performing standby login
    OCISessionBegin failed -1
    .. Detailed OCI error val is 1034 and errmsg is 'ORA-01034: ORACLE not available
    *** 2012-08-30 23:55:49.723 60679 kcrr.c
    Error 1034 received logging on to the standby
    Error 1034 connecting to destination LOG_ARCHIVE_DEST_2 standby host 'newstand'
    Error 1034 attaching to destination LOG_ARCHIVE_DEST_2 standby host 'newstand'
    ORA-01034: ORACLE not available
    *** 2012-08-30 23:55:49.723 58941 kcrr.c
    kcrrfail: dest:2 err:1034 force:0 blast:1
    kcrrwkx: unknown error:1034
    ORA-16055: FAL request rejected
    ARCH: Connecting to console port...
    ARCH: Connecting to console port...
    kcrrwkx: nothing to do (end)
    *** 2012-08-31 00:00:43.417
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:05:43.348
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:10:43.280
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:15:43.217
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:20:43.160
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:25:43.092
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:30:43.031
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:35:42.961
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:40:42.890
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:45:42.820
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:50:42.755
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 00:55:42.686
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 01:00:42.631
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 01:05:42.565
    kcrrwkx: nothing to do (start)
    *** 2012-08-31 01:10:42.496
    kcrrwkx: nothing to do (start)
    Mahir: Yes I have my 4 standby redo logs!
    I created the standby manually without using RMAN.
    Hemant: if it asks for even first thread, then obviously it shows nothing is applied on Standby. By the way so it is not called a 'GAP', I think..!
    Thanks.

  • Data Guard Failover after primary site network failure or disconnect.

    Hello Experts:
    I'll try to be clear and specific with my issue:
    Environment:
    Two nodes with NO shared storage (I don't have an Observer running).
    Veritas Cluser Server (VCS) with Data Guar Agent. (I don't use the Broker. Data Guard agent "takes care" of the switchover and failover).
    Two single instance databases, one per node. NO RAC.
    What I'm being able to perform with no issues:
    Manual switch(over) of the primary database by running VCS command "hagrp -switch oraDG_group -to standby_node"
    Automatic fail(over) when primary node is rebooted with "reboot" or "init"
    Automatic fail(over) when primary node is shut down with "shutdown".
    What I'm NOT being able to perform:
    If I manually unplug the network cables from the primary site (all the network, not only the link between primary and standby node so, it's like a server unplug from the energy source).
    Same situation happens if I manually disconnect the server from the power.
    This is the alert logs I have:
    This is the portion of the alert log at Standby site when Real Time Replication is working fine:
    Recovery of Online Redo Log: Thread 1 Group 4 Seq 7 Reading mem 0
      Mem# 0: /u02/oracle/fast_recovery_area/standby_db/onlinelog/o1_mf_4_9c3tk3dy_.log
    At this moment, node1 (Primary) is completely disconnected from the network. SEE at the end when the database (standby which should be converted to PRIMARY) is not getting all the archived logs from the Primary due to the abnormal disconnect from the network:
    Identified End-Of-Redo (failover) for thread 1 sequence 7 at SCN 0xffff.ffffffff
    Incomplete Recovery applied until change 15922544 time 12/23/2013 17:12:48
    Media Recovery Complete (primary_db)
    Terminal Recovery: successful completion
    Forcing ARSCN to IRSCN for TR 0:15922544
    Mon Dec 23 17:13:22 2013
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance primary_db - Archival ErrorAttempt to set limbo arscn 0:15922544 irscn 0:15922544
    ORA-16014: log 4 sequence# 7 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/u02/oracle/fast_recovery_area/standby_db/onlinelog/o1_mf_4_9c3tk3dy_.log'
    Resetting standby activation ID 2071848820 (0x7b7de774)
    Completed:  ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH
    Mon Dec 23 17:13:33 2013
    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH
    Terminal Recovery: applying standby redo logs.
    Terminal Recovery: thread 1 seq# 7 redo required
    Terminal Recovery:
    Recovery of Online Redo Log: Thread 1 Group 4 Seq 7 Reading mem 0
      Mem# 0: /u02/oracle/fast_recovery_area/standby_db/onlinelog/o1_mf_4_9c3tk3dy_.log
    Identified End-Of-Redo (failover) for thread 1 sequence 7 at SCN 0xffff.ffffffff
    Incomplete Recovery applied until change 15922544 time 12/23/2013 17:12:48
    Media Recovery Complete (primary_db)
    Terminal Recovery: successful completion
    Forcing ARSCN to IRSCN for TR 0:15922544
    Mon Dec 23 17:13:22 2013
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance primary_db - Archival ErrorAttempt to set limbo arscn 0:15922544 irscn 0:15922544
    ORA-16014: log 4 sequence# 7 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/u02/oracle/fast_recovery_area/standby_db/onlinelog/o1_mf_4_9c3tk3dy_.log'
    Resetting standby activation ID 2071848820 (0x7b7de774)
    Completed:  ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH
    Mon Dec 23 17:13:33 2013
    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH
    Attempt to do a Terminal Recovery (primary_db)
    Media Recovery Start: Managed Standby Recovery (primary_db)
    started logmerger process
    Mon Dec 23 17:13:33 2013
    Managed Standby Recovery not using Real Time Apply
    Media Recovery failed with error 16157
    Recovery Slave PR00 previously exited with exception 283
    ORA-283 signalled during:  ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH...
    Mon Dec 23 17:13:34 2013
    Shutting down instance (immediate)
    Shutting down instance: further logons disabled
    Stopping background process MMNL
    Stopping background process MMON
    License high water mark = 38
    All dispatchers and shared servers shutdown
    ALTER DATABASE CLOSE NORMAL
    ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
    ALTER DATABASE DISMOUNT
    Shutting down archive processes
    Archiving is disabled
    Mon Dec 23 17:13:38 2013
    Mon Dec 23 17:13:38 2013
    Mon Dec 23 17:13:38 2013
    ARCH shutting downARCH shutting down
    ARCH shutting down
    ARC0: Relinquishing active heartbeat ARCH role
    ARC2: Archival stopped
    ARC0: Archival stopped
    ARC1: Archival stopped
    Completed: ALTER DATABASE DISMOUNT
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Mon Dec 23 17:13:40 2013
    Stopping background process VKTM
    ARCH: Archival disabled due to shutdown: 1089
    Shutting down archive processes
    Archiving is disabled
    Mon Dec 23 17:13:43 2013
    Instance shutdown complete
    Mon Dec 23 17:13:44 2013
    Adjusting the default value of parameter parallel_max_servers
    from 1280 to 470 due to the value of parameter processes (500)
    Starting ORACLE instance (normal)
    ************************ Large Pages Information *******************
    Per process system memlock (soft) limit = 64 KB
    Total Shared Global Region in Large Pages = 0 KB (0%)
    Large Pages used by this instance: 0 (0 KB)
    Large Pages unused system wide = 0 (0 KB)
    Large Pages configured system wide = 0 (0 KB)
    Large Page size = 2048 KB
    RECOMMENDATION:
      Total System Global Area size is 3762 MB. For optimal performance,
      prior to the next instance restart:
      1. Increase the number of unused large pages by
    at least 1881 (page size 2048 KB, total size 3762 MB) system wide to
      get 100% of the System Global Area allocated with large pages
      2. Large pages are automatically locked into physical memory.
    Increase the per process memlock (soft) limit to at least 3770 MB to lock
    100% System Global Area's large pages into physical memory
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Initial number of CPU is 32
    Number of processor cores in the system is 16
    Number of processor sockets in the system is 2
    CELL communication is configured to use 0 interface(s):
    CELL IP affinity details:
        NUMA status: NUMA system w/ 2 process groups
        cellaffinity.ora status: cannot find affinity map at '/etc/oracle/cell/network-config/cellaffinity.ora' (see trace file for details)
    CELL communication will use 1 IP group(s):
        Grp 0:
    Picked latch-free SCN scheme 3
    Autotune of undo retention is turned on.
    IMODE=BR
    ILAT =88
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    NUMA system with 2 nodes detected
    Starting up:
    Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options.
    ORACLE_HOME = /u01/oracle/product/11.2.0.4
    System name:    Linux
    Node name:      node2.localdomain
    Release:        2.6.32-131.0.15.el6.x86_64
    Version:        #1 SMP Tue May 10 15:42:40 EDT 2011
    Machine:        x86_64
    Using parameter settings in server-side spfile /u01/oracle/product/11.2.0.4/dbs/spfileprimary_db.ora
    System parameters with non-default values:
      processes                = 500
      sga_target               = 3760M
      control_files            = "/u02/oracle/orafiles/primary_db/control01.ctl"
      control_files            = "/u01/oracle/fast_recovery_area/primary_db/control02.ctl"
      db_file_name_convert     = "standby_db"
      db_file_name_convert     = "primary_db"
      log_file_name_convert    = "standby_db"
      log_file_name_convert    = "primary_db"
      control_file_record_keep_time= 40
      db_block_size            = 8192
      compatible               = "11.2.0.4.0"
      log_archive_dest_1       = "location=/u02/oracle/archivelogs/primary_db"
      log_archive_dest_2       = "SERVICE=primary_db ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=primary_db"
      log_archive_dest_state_2 = "ENABLE"
      log_archive_min_succeed_dest= 1
      fal_server               = "primary_db"
      log_archive_trace        = 0
      log_archive_config       = "DG_CONFIG=(primary_db,standby_db)"
      log_archive_format       = "%t_%s_%r.dbf"
      log_archive_max_processes= 3
      db_recovery_file_dest    = "/u02/oracle/fast_recovery_area"
      db_recovery_file_dest_size= 30G
      standby_file_management  = "AUTO"
      db_flashback_retention_target= 1440
      undo_tablespace          = "UNDOTBS1"
      remote_login_passwordfile= "EXCLUSIVE"
      db_domain                = ""
      dispatchers              = "(PROTOCOL=TCP) (SERVICE=primary_dbXDB)"
      job_queue_processes      = 0
      audit_file_dest          = "/u01/oracle/admin/primary_db/adump"
      audit_trail              = "DB"
      db_name                  = "primary_db"
      db_unique_name           = "standby_db"
      open_cursors             = 300
      pga_aggregate_target     = 1250M
      dg_broker_start          = FALSE
      diagnostic_dest          = "/u01/oracle"
    Mon Dec 23 17:13:45 2013
    PMON started with pid=2, OS id=29108
    Mon Dec 23 17:13:45 2013
    PSP0 started with pid=3, OS id=29110
    Mon Dec 23 17:13:46 2013
    VKTM started with pid=4, OS id=29125 at elevated priority
    VKTM running at (1)millisec precision with DBRM quantum (100)ms
    Mon Dec 23 17:13:46 2013
    GEN0 started with pid=5, OS id=29129
    Mon Dec 23 17:13:46 2013
    DIAG started with pid=6, OS id=29131
    Mon Dec 23 17:13:46 2013
    DBRM started with pid=7, OS id=29133
    Mon Dec 23 17:13:46 2013
    DIA0 started with pid=8, OS id=29135
    Mon Dec 23 17:13:46 2013
    MMAN started with pid=9, OS id=29137
    Mon Dec 23 17:13:46 2013
    DBW0 started with pid=10, OS id=29139
    Mon Dec 23 17:13:46 2013
    DBW1 started with pid=11, OS id=29141
    Mon Dec 23 17:13:46 2013
    DBW2 started with pid=12, OS id=29143
    Mon Dec 23 17:13:46 2013
    DBW3 started with pid=13, OS id=29145
    Mon Dec 23 17:13:46 2013
    LGWR started with pid=14, OS id=29147
    Mon Dec 23 17:13:46 2013
    CKPT started with pid=15, OS id=29149
    Mon Dec 23 17:13:46 2013
    SMON started with pid=16, OS id=29151
    Mon Dec 23 17:13:46 2013
    RECO started with pid=17, OS id=29153
    Mon Dec 23 17:13:46 2013
    MMON started with pid=18, OS id=29155
    Mon Dec 23 17:13:46 2013
    MMNL started with pid=19, OS id=29157
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    starting up 1 shared server(s) ...
    ORACLE_BASE from environment = /u01/oracle
    Mon Dec 23 17:13:46 2013
    ALTER DATABASE   MOUNT
    ARCH: STARTING ARCH PROCESSES
    Mon Dec 23 17:13:50 2013
    ARC0 started with pid=23, OS id=29210
    ARC0: Archival started
    ARCH: STARTING ARCH PROCESSES COMPLETE
    ARC0: STARTING ARCH PROCESSES
    Successful mount of redo thread 1, with mount id 2071851082
    Mon Dec 23 17:13:51 2013
    ARC1 started with pid=24, OS id=29212
    Allocated 15937344 bytes in shared pool for flashback generation buffer
    Mon Dec 23 17:13:51 2013
    ARC2 started with pid=25, OS id=29214
    Starting background process RVWR
    ARC1: Archival started
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    Mon Dec 23 17:13:51 2013
    RVWR started with pid=26, OS id=29216
    Physical Standby Database mounted.
    Lost write protection disabled
    Completed: ALTER DATABASE   MOUNT
    Mon Dec 23 17:13:51 2013
    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
             USING CURRENT LOGFILE DISCONNECT FROM SESSION
    Attempt to start background Managed Standby Recovery process (primary_db)
    Mon Dec 23 17:13:51 2013
    MRP0 started with pid=27, OS id=29219
    MRP0: Background Managed Standby Recovery process started (primary_db)
    ARC2: Archival started
    ARC0: STARTING ARCH PROCESSES COMPLETE
    ARC2: Becoming the heartbeat ARCH
    ARC2: Becoming the active heartbeat ARCH
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance primary_db - Archival Error
    ORA-16014: log 4 sequence# 7 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/u02/oracle/fast_recovery_area/standby_db/onlinelog/o1_mf_4_9c3tk3dy_.log'
    At this moment, I've lost service and I have to wait until the prmiary server goes up again to receive the missing log.
    This is the rest of the log:
    Fatal NI connect error 12543, connecting to:
    (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=node1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=primary_db)(CID=(PROGRAM=oracle)(HOST=node2.localdomain)(USER=oracle))))
      VERSION INFORMATION:
            TNS for Linux: Version 11.2.0.4.0 - Production
            TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production
      Time: 23-DEC-2013 17:13:52
      Tracing not turned on.
      Tns error struct:
        ns main err code: 12543
    TNS-12543: TNS:destination host unreachable
        ns secondary err code: 12560
        nt main err code: 513
    TNS-00513: Destination host unreachable
        nt secondary err code: 113
        nt OS err code: 0
    Fatal NI connect error 12543, connecting to:
    (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=node1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=primary_db)(CID=(PROGRAM=oracle)(HOST=node2.localdomain)(USER=oracle))))
      VERSION INFORMATION:
            TNS for Linux: Version 11.2.0.4.0 - Production
            TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production
      Time: 23-DEC-2013 17:13:55
      Tracing not turned on.
      Tns error struct:
        ns main err code: 12543
    TNS-12543: TNS:destination host unreachable
        ns secondary err code: 12560
        nt main err code: 513
    TNS-00513: Destination host unreachable
        nt secondary err code: 113
        nt OS err code: 0
    started logmerger process
    Mon Dec 23 17:13:56 2013
    Managed Standby Recovery starting Real Time Apply
    MRP0: Background Media Recovery terminated with error 16157
    Errors in file /u01/oracle/diag/rdbms/standby_db/primary_db/trace/primary_db_pr00_29230.trc:
    ORA-16157: media recovery not allowed following successful FINISH recovery
    Managed Standby Recovery not using Real Time Apply
    Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
             USING CURRENT LOGFILE DISCONNECT FROM SESSION
    Recovery Slave PR00 previously exited with exception 16157
    MRP0: Background Media Recovery process shutdown (primary_db)
    Fatal NI connect error 12543, connecting to:
    (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=node1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=primary_db)(CID=(PROGRAM=oracle)(HOST=node2.localdomain)(USER=oracle))))
      VERSION INFORMATION:
            TNS for Linux: Version 11.2.0.4.0 - Production
            TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production
      Time: 23-DEC-2013 17:13:58
      Tracing not turned on.
      Tns error struct:
        ns main err code: 12543
    TNS-12543: TNS:destination host unreachable
        ns secondary err code: 12560
        nt main err code: 513
    TNS-00513: Destination host unreachable
        nt secondary err code: 113
        nt OS err code: 0
    Mon Dec 23 17:14:01 2013
    Fatal NI connect error 12543, connecting to:
    (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=node1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=primary_db)(CID=(PROGRAM=oracle)(HOST=node2.localdomain)(USER=oracle))))
      VERSION INFORMATION:
            TNS for Linux: Version 11.2.0.4.0 - Production
            TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production
      Time: 23-DEC-2013 17:14:01
      Tracing not turned on.
      Tns error struct:
        ns main err code: 12543
    TNS-12543: TNS:destination host unreachable
        ns secondary err code: 12560
        nt main err code: 513
    TNS-00513: Destination host unreachable
        nt secondary err code: 113
        nt OS err code: 0
    Error 12543 received logging on to the standby
    FAL[client, ARC0]: Error 12543 connecting to primary_db for fetching gap sequence
    Archiver process freed from errors. No longer stopped
    Mon Dec 23 17:15:07 2013
    Using STANDBY_ARCHIVE_DEST parameter default value as /u02/oracle/archivelogs/primary_db
    Mon Dec 23 17:19:51 2013
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance primary_db - Archival Error
    ORA-16014: log 4 sequence# 7 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/u02/oracle/fast_recovery_area/standby_db/onlinelog/o1_mf_4_9c3tk3dy_.log'
    Mon Dec 23 17:26:18 2013
    RFS[1]: Assigned to RFS process 31456
    RFS[1]: No connections allowed during/after terminal recovery.
    Mon Dec 23 17:26:47 2013
    flashback database to scn 15921680
    ORA-16157 signalled during: flashback database to scn 15921680...
    Mon Dec 23 17:27:05 2013
    alter database recover managed standby database using current logfile disconnect
    Attempt to start background Managed Standby Recovery process (primary_db)
    Mon Dec 23 17:27:05 2013
    MRP0 started with pid=28, OS id=31481
    MRP0: Background Managed Standby Recovery process started (primary_db)
    started logmerger process
    Mon Dec 23 17:27:10 2013
    Managed Standby Recovery starting Real Time Apply
    MRP0: Background Media Recovery terminated with error 16157
    Errors in file /u01/oracle/diag/rdbms/standby_db/primary_db/trace/primary_db_pr00_31486.trc:
    ORA-16157: media recovery not allowed following successful FINISH recovery
    Managed Standby Recovery not using Real Time Apply
    Completed: alter database recover managed standby database using current logfile disconnect
    Recovery Slave PR00 previously exited with exception 16157
    MRP0: Background Media Recovery process shutdown (primary_db)
    Mon Dec 23 17:27:18 2013
    RFS[2]: Assigned to RFS process 31492
    RFS[2]: No connections allowed during/after terminal recovery.
    Mon Dec 23 17:28:18 2013
    RFS[3]: Assigned to RFS process 31614
    RFS[3]: No connections allowed during/after terminal recovery.
    Do you have any advice?
    Thanks!
    Alex.

    Hello;
    What's not clear to me in your question at this point:
    What I'm NOT being able to perform:
    If I manually unplug the network cables from the primary site (all the network, not only the link between primary and standby node so, it's like a server unplug from the energy source).
    Same situation happens if I manually disconnect the server from the power.
    This is the alert logs I have:"
    Are you trying a failover to the Standby?
    Please advise.
    Is it possible your "valid_for clause" is set incorrectly?
    Would also review this:
    ORA-16014 and ORA-00312 Messages in Alert.log of Physical Standby
    Best Regards
    mseberg

  • How to delete the foreign archivelogs in a Logical Standby database

    How do I remove the foreign archive logs that are being sent to my logical standby database. I have files in the FRA of ASM going back weeks ago. I thought RMAN would delete them.
    I am doing hot backups of the databases to FRA for both databases. Using ASM, FRA, in a Data Guard environment.
    I am not backing up anything to tape yet.
    The ASM FRA foreign_archivelog directory on the logical standby FRA keeps growing and nothing is get deleted when
    I run the following command every day.
    delete expired backup;
    delete noprompt force obsolete;
    Primary database RMAN settings (Not all of them)
    CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 9 DAYS;
    CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
    CONFIGURE DB_UNIQUE_NAME 'WMRTPRD' CONNECT IDENTIFIER 'WMRTPRD_CWY';
    CONFIGURE DB_UNIQUE_NAME 'WMRTPRD2' CONNECT IDENTIFIER 'WMRTPRD2_CWY';
    CONFIGURE DB_UNIQUE_NAME 'WMRTPRD3' CONNECT IDENTIFIER 'WMRTPRD3_DG';
    CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON ALL STANDBY;
    Logical standby database RMAN setting (not all of them)
    CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 9 DAYS;
    CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
    CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
    How do I cleanup/delete the old ASM foreign_archivelog files?

    OK, the default is TRUE which is what it is now
    from DBA_LOGSTDBY_PARAMETERS
    LOG_AUTO_DELETE     TRUE          SYSTEM     YES
    I am not talking about deleting the Archive logs files for the Logical database that it is creating, but the Standby archive log files being sent to the Logical Database after they have been applied.
    They are in the alert log as follows under RFS LogMiner: Registered logfile
    RFS[1]: Selected log 4 for thread 1 sequence 159 dbid -86802306 branch 763744382
    Thu Jan 12 15:44:57 2012
    *RFS LogMiner: Registered logfile [+FRA/wmrtprd2/foreign_archivelog/wmrtprd/2012_01_12/thread_1_seq_158.322.772386297] to LogM*
    iner session id [1]
    Thu Jan 12 15:44:58 2012
    LOGMINER: Alternate logfile found. Transition to mining archived logfile for session 1 thread 1 sequence 158, +FRA/wmrtprd2/
    foreign_archivelog/wmrtprd/2012_01_12/thread_1_seq_158.322.772386297
    LOGMINER: End mining logfile for session 1 thread 1 sequence 158, +FRA/wmrtprd2/foreign_archivelog/wmrtprd/2012_01_12/threa
    d_1_seq_158.322.772386297
    LOGMINER: Begin mining logfile for session 1 thread 1 sequence 159, +DG1/wmrtprd2/onlinelog/group_4.284.771760923                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

Maybe you are looking for

  • Using union query to return a value to Oracle Type object in a view

    Hi All, I am using type objects inside a view to fetch data. For a particular type, in select statement when i use a "union all", it is throwing oracle error ora-03001 unimplemented feature. But it is working fine when it contains only select stateme

  • Garageband missing loops like Island long

    Hi there. Last year I took music in school. We used the schools macs on apps like garageband and Logic Pro. I decided to buy myself a mac this year and, after installing the addon loops in garageband, I still only have 2000 loops. I am unable to find

  • Duplicates of iTunes

    I just upgraded to itunes plus, and now my library has duplicates and some triplicates of the same songs. On my music list it shows only one listing, my question is: Can I get rid of the duplicates and triplicates from my library without affecting th

  • I would like to ask a few things since i`m new with java

    i would like to know if anyone can help me with a few things: 1)With which method does java get the ip address of the web user that visits the web page in which my applet will be posted? 2)Does java include charts like the ones excel does?Specificly

  • Change the description VFKP-POSTX (VI01)

    Hi all. I need to change the description of the text field of the item cost of transport (VI01) VFKP-POSTX currently describes standard point of departure and arrival point, I need to change that. NOTE SAP tried and found. Can anyone help me? Vladimi