Recover standby database error

Hi
here is the error message i got when i run this command
09:30:35 SYS@MOZAI> alter database recover standby database until cancel;
alter database recover standby database until cancel
ERROR at line 1:
ORA-00279: change 126425376421 generated at 02/26/2012 09:26:00 needed for thread 2
ORA-00289: suggestion : +ARCH
ORA-00280: change 126425376421 for thread 2 is in sequence #312362
Any suggestion is highly apprciated .
Thanks

Hello;
You probably started the recovery ( to apply redo from the Primary )
Cancel this before doing the command you trying :
alter database recover managed standby database cancel;Then issue your command again.
Best Regards
mseberg

Similar Messages

  • Recover Standby Database suggests wrong filename

    Hi,
    I am running Oracle Database 10g Release 10.2.0.3.0 - 64bit Production Standard Edition on Linux version 2.6.9-42.0.8.ELsmp ([email protected]) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3))
    I've created a physical standby database, but since I am running Standard Edition, I am not using the DataGuard features. I use the rsync utility to copy over the archivelogs to the standby database, and I apply them periodically to the standby database.
    The standby database is started this way :
    startup nomount pfile='/u01/oradata/orcl/initorcl.stdby';
    alter database mount standby database;
    And the archives are applied this way :
    recover standby database;
    AUTO
    (AUTO for the command to apply all available archive logs automatically, using the suggested paths and filenames)
    My problem is that once in a while (maybe once every 2-3 weeks), the suggested filename does not have the same format as the rest of the time. I then have to manually specify the correct filename and it goes fine after that.
    Example :
    In this example, you will see that it is first looking for sequence 22907 (o1_mf_1_22907_5n3m1xrf_.arc), then 22908 (o1_mf_1_22908_5n3m4kf0_.arc) [Notice the format of the file name] and then tries to look for sequence 22909, but looks for filename ".o1_mf_1_22909_5n3md1h5_.arc.qXMz5s"
    Mon Jan 4 06:22:01 2010
    ALTER DATABASE RECOVER standby database
    Media Recovery Start
    Managed Standby Recovery not using Real Time Apply
    ORA-279 signalled during: ALTER DATABASE RECOVER standby database ...
    Mon Jan 4 06:22:02 2010
    ALTER DATABASE RECOVER CONTINUE DEFAULT
    Mon Jan 4 06:22:02 2010
    Media Recovery Log /oraarch/oracle/flash_recovery_area/GIGA10G/archivelog/2010_01_04/o1_mf_1_22907_5n3m1xrf_.arc
    Mon Jan 4 06:24:20 2010
    ORA-279 signalled during: ALTER DATABASE RECOVER CONTINUE DEFAULT ...
    Mon Jan 4 06:24:20 2010
    ALTER DATABASE RECOVER CONTINUE DEFAULT
    Mon Jan 4 06:24:20 2010
    Media Recovery Log /oraarch/oracle/flash_recovery_area/GIGA10G/archivelog/2010_01_04/o1_mf_1_22908_5n3m4kf0_.arc
    Mon Jan 4 06:24:46 2010
    ORA-279 signalled during: ALTER DATABASE RECOVER CONTINUE DEFAULT ...
    Mon Jan 4 06:24:46 2010
    ALTER DATABASE RECOVER CONTINUE DEFAULT
    Mon Jan 4 06:24:46 2010
    Media Recovery Log /oraarch/oracle/flash_recovery_area/GIGA10G/archivelog/2010_01_04/.o1_mf_1_22909_5n3md1h5_.arc.qXMz5s
    Errors with log /oraarch/oracle/flash_recovery_area/GIGA10G/archivelog/2010_01_04/.o1_mf_1_22909_5n3md1h5_.arc.qXMz5s
    ORA-308 signalled during: ALTER DATABASE RECOVER CONTINUE DEFAULT ...
    Mon Jan 4 06:24:46 2010
    ALTER DATABASE RECOVER CANCEL
    Mon Jan 4 06:24:46 2010
    Media Recovery Canceled
    Completed: ALTER DATABASE RECOVER CANCEL
    Can someone explain to me why is this happening please ?
    Thanks a lot,
    Mat

    @fjfranken
    Yes. If I manually enter the name of the file for the sequence requested instead of passing the suggested fielname, it works fine.
    And the log_archive_format is not defined, I am using the flash_recovery_area and let Oracle manage this automatically.
    @Hemant K Chitale
    Unfortunately, those sequences are not listed anymore in the V$ARCHIVED_LOG view.
    Well, I think the the problem might be related to files that are being written to at some time...
    Thanks,
    Mat

  • Recover standby database

    Our primary linux 10g db is in standard edition and we would like to manually create a standby database
    After copying the control and datafiles from primary to standby database, started the standby instance ..
    SQL> startup nomount pfile=/path/to/pfile/initSID.standby
    SQL> alter database mount standby database;
    SQL> recover standby database;
    ORA-00279: change 2342934 generated at 8/27/2009 21:10:35 needed for thread 1
    ORA-00289: suggestion : /opt/oracle/arch/SID/1_833_682861383.arc
    ORA-00280: change 2342934 for thread 1 is in sequence #833
    Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
    AUTO
    ORA-00308: cannot open archived log
    '/opt/oracle/arch/SID/1_833_682861383.arc'
    ORA-27037: unable to obtain file status
    Linux Error: 2: No such file or directory
    Additional information: 3
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01195: online backup of file 1 needs more recovery to be consistent
    ORA-01110: data file 1: '/opt/oracle/oradata/SID/system01.dbf'
    How do I resolve this problem? Log sequence 833 is the current log on the primary db, archive log haven't been written yet. When I try to "alter database open read only" on the standby db, I get an error "ORA-01156: recovery in progress may need access to file".
    I then went back to my primary database, which already had another log switch to 834, log 833 is now available, I then moved that archive log file to the standby db. Try to recover standby database again, but still got errors..
    SQL> startup nomount pfile=/path/to/pfile/initSID.standby
    SQL> alter database mount standby database;
    SQL> recover standby database;
    ORA-00279: change 2342934 generated at 8/27/2009 21:10:35 needed for thread 1
    ORA-00289: suggestion : /opt/oracle/arch/SID/1_833_682861383.arc
    ORA-00280: change 2342934 for thread 1 is in sequence #833
    Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
    AUTO
    ORA-00317: file type 0 in header is not log file
    ORA-00334: archived log: '/opt/oracle/arch/SID/1_833_682861383.arc'
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01195: online backup of file 1 needs more recovery to be consistent
    ORA-01110: data file 1: '/opt/oracle/oradata/SID/system01.dbf'
    SQL> alter database open read only;
    ERROR at line 1:
    ORA-16004: backup database requires recovery
    ORA-01195: online backup of file 1 needs more recovery to be consistent
    ORA-01110: data file 1: '/opt/oracle/oradata/SID/system01.dbf'
    Would something please explain this to me? What am I doing wrong? Thanks in advance.

    I copied the missing archive log over to standby db, still got the same error.
    1) SQL> recover standby database;
    ORA-00279: change 2342934 generated at 8/27/2009 21:10:35 needed for thread 1
    ORA-00289: suggestion : /opt/oracle/arch/SID/1_833_682861383.arc
    ORA-00280: change 2342934 for thread 1 is in sequence #833
    Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
    AUTO
    ORA-00317: file type 0 in header is not log file
    ORA-00334: archived log: '/opt/oracle/arch/SID/1_833_682861383.arc'
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01195: online backup of file 1 needs more recovery to be consistent
    ORA-01110: data file 1: '/opt/oracle/oradata/SID/system01.dbf'
    I also tried..
    2) SQL> recover standby database using backup controlfile until cancel;
    ORA-00279: change 2342934 generated at 8/27/2009 21:10:35 needed for thread 1
    ORA-00289: suggestion : /opt/oracle/arch/SID/1_833_682861383.arc
    ORA-00280: change 2342934 for thread 1 is in sequence #833
    Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
    AUTO
    ORA-00317: file type 0 in header is not log file
    ORA-00334: archived log: '/opt/oracle/arch/SID/1_833_682861383.arc'
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01195: online backup of file 1 needs more recovery to be consistent
    ORA-01110: data file 1: '/opt/oracle/oradata/SID/system01.dbf'
    Edited by: user10427867 on Aug 28, 2009 5:56 AM

  • Auto specified any value after ran RECOVER STANDBY DATABASE;

    Dear all,
    When i ran the RECOVER STANDBY DATABASE command.It 's will prompt as
    below :
    SQL> RECOVER STANDBY DATABASE;
    ORA-00279: change 302927 generated at 09/10/2007 14:08:24 needed for thread 1
    ORA-00289: suggestion : D:\ORACLE\ORADATA\DB817\ARCHIVE\DB817T001S00021.ARC
    ORA-00280: change 302927 for thread 1 is in sequence #21
    Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
    Is there anyway to force specified "AUTO" automatically ?
    Because i will to create a scripts file for run automatic .
    Thanks for advance !
    Chara

    ORA-00289: suggestion : D:\ORACLE\ORADATA\DB817\ARCHIVE\DB817T001S00021.ARCFirst check if the required Archive / Redo log is present on your system under the directory "D:\ORACLE\ORADATA\DB817\ARCHIVE".
    For automatic recovery use:
    ALTER DATABASE RECOVER AUTOMATIC STANDBY DATABASE;
    or
    RECOVER AUTOMATIC STANDBY DATABASE;
    Cheers!

  • Physical standby database error

    Hi,
    I try to create physical standby in my local machine using oracle 10g
    version :10.2.0.1.0
    Os: windows xp.
    I followed steps and parameter files
    on primary :-
    sql>alter database force logging;
    orcl.__db_cache_size=155189248
    orcl.__java_pool_size=4194304
    orcl.__large_pool_size=4194304
    orcl.__shared_pool_size=75497472
    orcl.__streams_pool_size=0
    *.audit_file_dest='D:\oracle\product\10.2.0/admin/orcl/adump'
    *.background_dump_dest='D:\oracle\product\10.2.0/admin/orcl/bdump'
    *.compatible='10.2.0.1.0'
    *.control_files='D:\oracle\product\10.2.0\oradata\orcl\control01.ctl','D:\oracl
    e\product\10.2.0\oradata\orcl\control02.ctl','D:\oracle\product\10.2.0\orad
    ata\orcl\control03.ctl'
    *.core_dump_dest='D:\oracle\product\10.2.0/admin/orcl/cdump'
    *.db_block_size=8192
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_name='orcl'
    *.db_unique_name='orcl'
    *.log_archive_config='DG_CONFIG=(orcl,stby)'
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=orclXDB)'
    *.job_queue_processes=10
    *.open_cursors=300
    *.pga_aggregate_target=80740352
    *.processes=150
    *.remote_login_passwordfile='EXCLUSIVE'
    *.sga_target=244318208
    *.undo_management='AUTO'
    *.undo_tablespace='UNDOTBS1'
    *.user_dump_dest='D:\oracle\product\10.2.0/admin/orcl/udump'
    *.log_archive_dest_1='LOCATION=D:\oracle\archive
    VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=orcl'
    *.log_archive_dest_2='SERVICE=stby LGWR ASYNC
    VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=stby'
    *.log_archive_format=%t_%s_%r.arc
    *.log_archive_dest_state_1=ENABLE
    *.log_archive_dest_state_2=ENABLE
    On standby
    orcl.__db_cache_size=155189248
    orcl.__java_pool_size=4194304
    orcl.__large_pool_size=4194304
    orcl.__shared_pool_size=75497472
    orcl.__streams_pool_size=0
    *.audit_file_dest='D:\stby\adump'
    *.background_dump_dest='D:\stby\bdump'
    *.compatible='10.2.0.1.0'
    *.control_files='D:\stby\ctrl.ctl'
    *.core_dump_dest='D:\stby\cdump'
    *.db_block_size=8192
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_name='orcl'
    *.db_unique_name='stby'
    *.log_archive_config='DG_CONFIG=(orcl,stby)'
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=orclXDB)'
    *.job_queue_processes=10
    *.open_cursors=300
    *.pga_aggregate_target=80740352
    *.processes=150
    *.remote_login_passwordfile='NONE'
    *.sga_target=244318208
    *.undo_management='AUTO'
    *.undo_tablespace='UNDOTBS1'
    *.user_dump_dest='D:\stby\udump'
    *.db_file_name_convert='D:\oracle\product\10.2.0\oradata\orcl','D:\stby'
    *.log_file_name_convert='D:\oracle\product\10.2.0\oradata\orcl','D:\stby'
    *.log_archive_dest_1='LOCATION=D:\oracle\archive
    VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=stby'
    *.log_archive_dest_2='SERVICE=orcl LGWR ASYNC
    VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE)
    DB_UNIQUE_NAME=orcl'
    *.standby_archive_dest='D:\stby\archive'
    *.log_archive_dest_state_1=ENABLE
    *.log_archive_dest_state_2=ENABLE
    *.standby_file_management='AUTO'
    *.fal_server='orcl'
    *.fal_client='stby'
    SQL> conn / as sysdba
    Connected to an idle instance.
    SQL> startup pfile=D:\stby\initstby.ora nomount;
    ORACLE instance started.
    Total System Global Area 247463936 bytes
    Fixed Size 1248332 bytes
    Variable Size 79692724 bytes
    Database Buffers 159383552 bytes
    Redo Buffers 7139328 bytes
    SQL> alter database mount standby database;
    Database altered.
    SQL> alter database recover managed standby database disconnect from session;
    Database altered.
    SQL> alter database open read only;
    alter database open read only
    ERROR at line 1:
    ORA-01154: database busy. Open, close, mount, and dismount not allowed now
    SQL> recover managed standby database cancel;
    Media recovery complete.
    SQL> alter database open read only;
    Database altered.
    SQL> select sequence#,first_time,next_time from v$archived_log order by sequence
    no rows selected
    No archive s generated in standby ..
    what i mistake.
    Reagrds
    S.Azar

    Anand... wrote:
    Firstly you can use "*select sequence#,first_time,next_time from v$archived_log order by sequence#;"* command in mount stage itself.No need to open the standby database in ready only mode.
    No rows selected in mount stage also.
    The V$ARCHIVED_LOG fixed view on the physical standby database shows all the archived redo log files received from the primary database.
    sql> select * from v$archived_log;
    no rows selected.
    and No archived logs received from primary database to physical standby . i think here i wrongly configure parameter file.
    sql> RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION; -- > start Redo Apply
    When you do log switch on the primary are the archive logs getting transferred to the standby database???Is there some error in the alert log file in the primary database.
    No..
    >
    [http://advait.wordpress.com/2007/06/12/setting-up-oracle-dataguard-for-10g/]
    Regards
    S.Azar

  • Standby database errors - Alter database open read only

    alter database open read only
    AUDIT_TRAIL initialization parameter is changed to OS, as DB is NOT compatible for database opened with read-only access
    Signalling error 1152 for datafile 1!
    Beginning standby crash recovery.
    Serial Media Recovery started
    Managed Standby Recovery starting Real Time Apply
    Media Recovery Waiting for thread 1 sequence 216
    Mon Dec 20 11:58:18 2010
    Standby crash recovery need archive log for thread 1 sequence 216 to continue.
    Please verify that primary database is transporting redo logs to the standby database.
    Wait timeout: thread 1 sequence 216
    Standby crash recovery aborted due to error 16016.
    Errors in file /u01/app/oracle/diag/rdbms/mdm2/MDM2/trace/MDM2_ora_17442.trc:
    ORA-16016: archived log for thread 1 sequence# 216 unavailable
    Recovery interrupted!
    Completed standby crash recovery.
    Signalling error 1152 for datafile 1!
    Errors in file /u01/app/oracle/diag/rdbms/mdm2/MDM2/trace/MDM2_ora_17442.trc:
    ORA-10458: standby database requires recovery
    ORA-01152: file 1 was not restored from a sufficiently old backup
    ORA-01110: data file 1: '+MDMDG1/mdm2/datafile/system.280.738243341'
    ORA-10458 signalled during: alter database open read only...
    Mon Dec 20 12:13:46 2010
    ALTER DATABASE RECOVER managed standby database using current logfile disconnect
    Attempt to start background Managed Standby Recovery process (MDM2)
    Mon Dec 20 12:13:46 2010
    MRP0 started with pid=23, OS id=18974
    MRP0: Background Managed Standby Recovery process started (MDM2)
    started logmerger process
    Mon Dec 20 12:13:51 2010
    Managed Standby Recovery starting Real Time Apply
    Parallel Media Recovery started with 2 slaves
    Waiting for all non-current ORLs to be archived...
    All non-current ORLs have been archived.
    Media Recovery Waiting for thread 1 sequence 216
    Completed: ALTER DATABASE RECOVER managed standby database using current logfile disconnect
    The above lines are from alert log of standby database.
    Standby standbase
    SQL> alter database open read only;
    alter database open read only
    ERROR at line 1:
    ORA-10458: standby database requires recovery
    ORA-01152: file 1 was not restored from a sufficiently old backup
    ORA-01110: data file 1: '+MDMDG1/mdm2/datafile/system.280.738243341'
    Parameters set on primary are
    log_archive_dest_1 LOCATION=+MDMDG3/MDM1/ARCH VALID_FOR=(ALL_LOGFILES,ALL_ROLE ) DB_UNIQUE_NAME=MDM1
    log_archive_dest_state_1 ENABLE
    log_archive_dest_2 SERVICE=MDM2 SYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=MDM2
    log_archive_dest_state_2 ENABLE
    dg_broker_config_file1 +MDMDG2/mdm/dg_config/dgconfig1_mdm.dat
    dg_broker_config_file2 +MDMDG2/mdm/dg_config/dgconfig2_mdm.dat
    fal_server MDM2
    standby_file_management AUTO
    log_archive_config dg_config=(MDM1,MDM2)
    db_file_name_convert MDM2, MDM1
    ALTER DATABASE SET STANDBY DATABASE TO MAXIMIZE availability ;
    Standby pfile
    *.archive_lag_target=900
    *.audit_file_dest='/u01/app/oracle/admin/MDM2/adump'
    *.audit_trail='db'
    *.compatible='11.2.0.0.0'
    *.control_files='+MDMDG1/MDM2/CONTROLFILE/controlfile01.ctl','+MDMDG2/MDM2/CONTROLFILE/controlfile02.ctl'
    *.db_block_size=8192
    *.db_create_file_dest='+MDMDG1'
    *.db_domain=''
    *.db_file_name_convert='MDM1','MDM2'
    *.db_name='MDM'
    *.db_recovery_file_dest='+MDMDG2'
    *.db_recovery_file_dest_size=10485760000
    *.db_unique_name='MDM2'
    *.dg_broker_config_file1='+MDMDG2/MDM/DG_CONFIG/dgconfig1_MDM.dat'
    *.dg_broker_config_file2='+MDMDG2/MDM/DG_CONFIG/dgconfig2_MDM.dat'
    *.diagnostic_dest='/u01/app/oracle'
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=MDM2XDB)'
    *.fal_server='MDM11','MDM12'
    *.instance_name='MDM2'
    *.log_archive_config='dg_config=(MDM1,MDM2)'
    *.log_archive_dest_1='LOCATION=+MDMDG3/MDM2/ARCH VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=MDM2'
    *.log_archive_dest_2='SERVICE=MDM1 ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=MDM1'
    *.log_archive_dest_state_1='ENABLE'
    *.log_archive_dest_state_2='ENABLE'
    *.log_archive_format='MDM_%t_%s_%r.arc'
    *.log_file_name_convert='MDM1','MDM2'
    *.memory_target=838860800
    *.nls_language='ENGLISH'
    *.nls_territory='UNITED KINGDOM'
    *.open_cursors=300
    *.processes=500
    *.remote_login_passwordfile='exclusive'
    *.sessions=555
    *.standby_file_management='AUTO'
    *.undo_tablespace='UNDOTBS1'
    On standby ASM
    ASMCMD [+] > find * *
    +MDMDG1/ASM/
    +MDMDG1/ASM/ASMPARAMETERFILE/
    +MDMDG1/ASM/ASMPARAMETERFILE/REGISTRY.253.737811541
    +MDMDG1/MDM/
    +MDMDG1/MDM2/
    +MDMDG1/MDM2/CONTROLFILE/
    +MDMDG1/MDM2/CONTROLFILE/controlfile01.ctl
    +MDMDG1/MDM2/CONTROLFILE/current.265.738243333
    +MDMDG1/MDM2/DATAFILE/
    +MDMDG1/MDM2/DATAFILE/CANVAS_POPULARITY_DATA.264.738243343
    +MDMDG1/MDM2/DATAFILE/CANVAS_POPULARITY_IDX.277.738243343
    +MDMDG1/MDM2/DATAFILE/MDM_SRC_DATA.282.738243343
    +MDMDG1/MDM2/DATAFILE/MDM_SRC_IDX.275.738243343
    +MDMDG1/MDM2/DATAFILE/MIPS_MDM_DATA.283.738243341
    +MDMDG1/MDM2/DATAFILE/MIPS_MDM_IDX.276.738243343
    +MDMDG1/MDM2/DATAFILE/SYSAUX.281.738243341
    +MDMDG1/MDM2/DATAFILE/SYSTEM.280.738243341
    +MDMDG1/MDM2/DATAFILE/TEST_TBSP1.273.738243345
    +MDMDG1/MDM2/DATAFILE/TEST_TBSP2.272.738243345
    +MDMDG1/MDM2/DATAFILE/UNDOTBS1.256.738243343
    +MDMDG1/MDM2/DATAFILE/UNDOTBS2.279.738243343
    +MDMDG1/MDM2/DATAFILE/USERS.278.738243347
    +MDMDG1/MDM2/ONLINELOG/
    +MDMDG1/MDM2/ONLINELOG/group_1.259.738243429
    +MDMDG1/MDM2/ONLINELOG/group_2.257.738243431
    +MDMDG1/MDM2/ONLINELOG/group_21.284.738243505
    +MDMDG1/MDM2/ONLINELOG/group_22.261.738243505
    +MDMDG1/MDM2/ONLINELOG/group_23.274.738243505
    +MDMDG1/MDM2/ONLINELOG/group_3.258.738243431
    +MDMDG1/MDM2/ONLINELOG/group_31.262.738243513
    +MDMDG1/MDM2/ONLINELOG/group_32.270.738243513
    +MDMDG1/MDM2/ONLINELOG/group_33.263.738243513
    +MDMDG1/MDM2/ONLINELOG/group_4.260.738243431
    +MDMDG2/MDM/
    +MDMDG2/MDM/DG_CONFIG/
    +MDMDG2/MDM2/
    +MDMDG2/MDM2/AUTOBACKUP/
    +MDMDG2/MDM2/AUTOBACKUP/2010_12_20/
    +MDMDG2/MDM2/AUTOBACKUP/2010_12_20/s_738242861.263.738244155
    +MDMDG2/MDM2/CONTROLFILE/
    +MDMDG2/MDM2/CONTROLFILE/controlfile02.ctl
    +MDMDG2/MDM2/CONTROLFILE/current.271.738243335
    +MDMDG2/MDM2/ONLINELOG/
    +MDMDG2/MDM2/ONLINELOG/group_1.270.738243429
    +MDMDG2/MDM2/ONLINELOG/group_2.269.738243431
    +MDMDG2/MDM2/ONLINELOG/group_21.268.738243505
    +MDMDG2/MDM2/ONLINELOG/group_22.272.738243505
    +MDMDG2/MDM2/ONLINELOG/group_23.262.738243505
    +MDMDG2/MDM2/ONLINELOG/group_3.273.738243431
    +MDMDG2/MDM2/ONLINELOG/group_31.266.738243513
    +MDMDG2/MDM2/ONLINELOG/group_32.265.738243513
    +MDMDG2/MDM2/ONLINELOG/group_33.264.738243513
    +MDMDG2/MDM2/ONLINELOG/group_4.261.738243431
    +MDMDG3/MDM/
    +MDMDG3/MDM/ARCH/
    +MDMDG3/MDM2/
    +MDMDG3/MDM2/ARCH/
    -- Please can I know how to open read only standby database.

    user5846399 wrote:
    ORA-16016: archived log for thread 1 sequence# 216 unavailable
    Recovery interrupted!archived log for thread 1 sequence# 216
    This file is needed for recovery, Find it and move it to the standby database side.

  • Standby database Error in applying archives

    Hi All
    Environment:
    Oracle 8i
    Platform- HP-UX
    I have production database on HPUX box and standby database on anothe HPUX box.
    I am applying archive to standby everyday manually.
    In last week, I have added ney datafile webapp03 in Production database tablespace.
    When I apply archive to standby database. It says archive apply sucessfully but give below warning:
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01195: online backup of file 25 needs more recovery to be consistent
    ORA-01110: data file 25: '/u06/oradata/vestcom/webapp03.dbf'
    Yesterday I have copied datafile from production to standby database.
    But still I am getting same warning:
    Please let me know in detail, I am totally new in this admin field.
    Please help me out for this issue:
    Thanks in Advance.
    R M

    Absolutely. First place the tablespace space in backup mode, then copy over the datafile and then end the backup mode - applying the log files should be fine now.
    HTH
    Thanks
    Chandra

  • Standby database error

    Hi Guys..
    I am using oracle 10.2.0.4.0 on windows 2003 server. I created a physical standby database successfully.
    On standby database, I fired the following query
    SELECT SEQUENCE#, FIRST_TIME, NEXT_TIME FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#
    SEQUENCE# FIRST_TIM NEXT_TIME
    10715 02-AUG-10 02-AUG-10
    10716 02-AUG-10 02-AUG-10
    10717 02-AUG-10 03-AUG-10
    10718 03-AUG-10 03-AUG-10
    10719 03-AUG-10 03-AUG-10
    10720 03-AUG-10 03-AUG-10
    10721 03-AUG-10 03-AUG-10
    10722 03-AUG-10 03-AUG-10
    10723 03-AUG-10 03-AUG-10
    10724 03-AUG-10 03-AUG-10
    10725 03-AUG-10 03-AUG-10
    SEQUENCE# FIRST_TIM NEXT_TIME
    10726 03-AUG-10 03-AUG-10
    10727 03-AUG-10 03-AUG-10
    10728 03-AUG-10 03-AUG-10
    10729 03-AUG-10 03-AUG-10
    I then did alter system switch logfile on primary database
    Then on standby I fired the above query
    SELECT SEQUENCE#, FIRST_TIME, NEXT_TIME FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#
    SEQUENCE# FIRST_TIM NEXT_TIME
    10715 02-AUG-10 02-AUG-10
    10716 02-AUG-10 02-AUG-10
    10717 02-AUG-10 03-AUG-10
    10718 03-AUG-10 03-AUG-10
    10719 03-AUG-10 03-AUG-10
    10720 03-AUG-10 03-AUG-10
    10721 03-AUG-10 03-AUG-10
    10722 03-AUG-10 03-AUG-10
    10723 03-AUG-10 03-AUG-10
    10724 03-AUG-10 03-AUG-10
    10725 03-AUG-10 03-AUG-10
    SEQUENCE# FIRST_TIM NEXT_TIME
    10726 03-AUG-10 03-AUG-10
    10727 03-AUG-10 03-AUG-10
    10728 03-AUG-10 03-AUG-10
    10729 03-AUG-10 03-AUG-10
    *10730 03-AUG-10 03-AUG-10*
    My question is everytime I do alter systyem switch logfile on primary , two trace files are generated on primary database which are:
    stutest_lns1_5764.trc
    Dump file c:\oracle\admin\stutest\bdump\stutest_lns1_5764.trc
    Tue Aug 03 15:50:35 2010
    ORACLE V10.2.0.4.0 - 64bit Production vsnsta=0
    vsnsql=14 vsnxtr=3
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options
    Windows Server 2003 Version V5.2 Service Pack 2
    CPU : 8 - type 8664, 2 Physical Cores
    Process Affinity : 0x0000000000000000
    Memory (Avail/Total): Ph:8414M/16378M, Ph+PgF:10750M/17814M
    Instance name: stutest
    Redo thread mounted by this instance: 1
    Oracle process number: 43
    Windows thread id: 5764, image: ORACLE.EXE (LNS1)
    *** SERVICE NAME:(SYS$BACKGROUND) 2010-08-03 15:50:35.996
    *** SESSION ID:(213.29604) 2010-08-03 15:50:35.996
    *** 2010-08-03 15:50:35.996 60512 kcrr.c
    LNS1: initializing for LGWR communication
    LNS1: connecting to KSR channel
    Success
    LNS1: subscribing to KSR channel
    Success
    *** 2010-08-03 15:50:35.996 60565 kcrr.c
    LNS1: initialized successfully ASYNC=1
    Destination is specified with ASYNC=61440
    Redo shipping client performing standby login
    *** 2010-08-03 15:50:36.043 68145 kcrr.c
    Logged on to standby successfully
    Client logon and security negotiation successful!
    Archiving to destination STUSTDBY ASYNC blocks=20480
    Allocate ASYNC blocks: Previous blocks=0 New blocks=20480
    Log file opened [logno 2]
    *** 2010-08-03 16:00:58.344
    Archiving to destination STUSTDBY ASYNC blocks=20480
    Log file opened [logno 1]
    *** 2010-08-03 16:04:44.383
    Archiving to destination STUSTDBY ASYNC blocks=20480
    Log file opened [logno 3]
    *** 2010-08-03 16:07:36.794
    Archiving to destination STUSTDBY ASYNC blocks=20480
    Log file opened [logno 2]
    stutest_lgwr_4092.trc
    Dump file c:\oracle\admin\stutest\bdump\stutest_lgwr_4092.trc
    Fri Jul 30 17:01:59 2010
    ORACLE V10.2.0.4.0 - 64bit Production vsnsta=0
    vsnsql=14 vsnxtr=3
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options
    Windows Server 2003 Version V5.2 Service Pack 2
    CPU : 8 - type 8664, 2 Physical Cores
    Process Affinity : 0x0000000000000000
    Memory (Avail/Total): Ph:13601M/16378M, Ph+PgF:10927M/17814M
    Instance name: stutest
    Redo thread mounted by this instance: 1
    Oracle process number: 6
    Windows thread id: 4092, image: ORACLE.EXE (LGWR)
    *** 2010-07-30 17:01:59.622
    *** SERVICE NAME:() 2010-07-30 17:01:59.606
    *** SESSION ID:(221.1) 2010-07-30 17:01:59.606
    Maximum redo generation record size = 156160 bytes
    Maximum redo generation change vector size = 150676 bytes
    *** 2010-07-30 22:01:00.446
    Warning: log write time 770ms, size 2KB
    *** 2010-07-30 22:08:37.586
    Warning: log write time 1120ms, size 9KB
    *** 2010-07-30 22:08:41.258
    Warning: log write time 680ms, size 7KB
    *** 2010-07-30 22:09:58.244
    Warning: log write time 660ms, size 1KB
    *** 2010-07-30 22:11:21.388
    Warning: log write time 860ms, size 6KB
    *** 2010-07-30 23:42:11.203
    Warning: log write time 580ms, size 1KB
    *** 2010-07-31 00:04:13.917
    Warning: log write time 980ms, size 8KB
    *** 2010-07-31 00:06:40.422
    Warning: log write time 930ms, size 22KB
    *** 2010-07-31 00:08:11.940
    Warning: log write time 1800ms, size 23KB
    *** 2010-07-31 21:13:33.694
    Warning: log write time 1460ms, size 40KB
    *** 2010-07-31 21:13:39.522
    Warning: log write time 2790ms, size 87KB
    *** 2010-07-31 21:13:46.054
    Warning: log write time 4940ms, size 69KB
    *** 2010-07-31 21:13:50.038
    Warning: log write time 3980ms, size 81KB
    *** 2010-07-31 21:13:55.616
    Warning: log write time 5580ms, size 61KB
    *** 2010-07-31 21:14:01.726
    Warning: log write time 6110ms, size 77KB
    *** 2010-07-31 21:14:08.258
    Warning: log write time 3530ms, size 140KB
    *** 2010-07-31 21:14:19.508
    Warning: log write time 1500ms, size 3KB
    *** 2010-07-31 22:07:49.954
    Warning: log write time 930ms, size 1KB
    *** 2010-07-31 23:16:01.054
    Warning: log write time 1080ms, size 103KB
    *** 2010-07-31 23:17:52.854
    Warning: log write time 1360ms, size 3KB
    *** 2010-07-31 23:23:26.631
    Warning: log write time 710ms, size 2KB
    *** 2010-08-01 00:04:58.476
    Warning: log write time 690ms, size 13KB
    *** 2010-08-01 00:06:51.980
    Warning: log write time 590ms, size 20KB
    *** 2010-08-01 00:07:38.153
    Warning: log write time 1540ms, size 28KB
    *** 2010-08-01 00:08:02.748
    Warning: log write time 770ms, size 64KB
    *** 2010-08-01 00:11:04.628
    Warning: log write time 1140ms, size 43KB
    *** 2010-08-01 17:05:49.886
    Warning: log write time 780ms, size 3KB
    *** 2010-08-01 21:00:46.571
    Warning: log write time 910ms, size 365KB
    *** 2010-08-01 21:09:22.447
    Warning: log write time 2910ms, size 22KB
    *** 2010-08-01 21:09:27.932
    Warning: log write time 4430ms, size 62KB
    *** 2010-08-01 21:09:34.010
    Warning: log write time 6060ms, size 74KB
    *** 2010-08-01 21:09:40.088
    Warning: log write time 3080ms, size 150KB
    *** 2010-08-01 21:09:45.495
    Warning: log write time 2410ms, size 99KB
    *** 2010-08-01 21:09:50.370
    Warning: log write time 4870ms, size 40KB
    *** 2010-08-01 21:09:55.401
    Warning: log write time 5030ms, size 82KB
    *** 2010-08-01 21:09:59.151
    Warning: log write time 3610ms, size 65KB
    *** 2010-08-01 23:05:40.560
    Warning: log write time 720ms, size 4KB
    *** 2010-08-01 23:06:05.030
    Warning: log write time 1670ms, size 21KB
    *** 2010-08-01 23:17:55.537
    Warning: log write time 1540ms, size 116KB
    *** 2010-08-01 23:23:35.891
    Warning: log write time 600ms, size 1KB
    *** 2010-08-02 00:05:52.363
    Warning: log write time 1340ms, size 48KB
    *** 2010-08-02 00:06:43.427
    Warning: log write time 1060ms, size 24KB
    *** 2010-08-02 00:06:50.177
    Warning: log write time 1730ms, size 209KB
    *** 2010-08-02 00:07:08.256
    Warning: log write time 950ms, size 17KB
    *** 2010-08-02 00:11:25.436
    Warning: log write time 1690ms, size 8KB
    *** 2010-08-02 00:12:11.313
    Warning: log write time 850ms, size 45KB
    *** 2010-08-02 00:13:55.754
    Warning: log write time 560ms, size 3KB
    *** 2010-08-02 11:27:52.686
    Warning: log write time 1440ms, size 46KB
    *** 2010-08-02 11:27:58.780
    Warning: log write time 3790ms, size 54KB
    *** 2010-08-02 11:28:02.546
    Warning: log write time 3760ms, size 70KB
    *** 2010-08-02 11:28:08.483
    Warning: log write time 5940ms, size 74KB
    *** 2010-08-02 11:28:13.812
    Warning: log write time 2330ms, size 139KB
    *** 2010-08-02 11:28:18.796
    Warning: log write time 1980ms, size 79KB
    *** 2010-08-02 11:28:25.093
    Warning: log write time 3360ms, size 88KB
    *** 2010-08-02 11:28:29.421
    Warning: log write time 4330ms, size 38KB
    *** 2010-08-02 16:10:45.927
    Warning: log write time 590ms, size 3KB
    *** 2010-08-02 22:08:57.806
    Warning: log write time 700ms, size 37KB
    *** 2010-08-02 23:07:22.262
    Warning: log write time 1750ms, size 234KB
    *** 2010-08-02 23:13:44.993
    Warning: log write time 1410ms, size 17KB
    *** 2010-08-03 00:04:08.387
    Warning: log write time 670ms, size 6KB
    *** 2010-08-03 00:07:44.347
    Warning: log write time 1890ms, size 1024KB
    *** 2010-08-03 00:08:14.395
    Warning: log write time 1680ms, size 1024KB
    *** 2010-08-03 00:08:26.817
    Warning: log write time 570ms, size 36KB
    *** 2010-08-03 00:08:39.427
    Warning: log write time 1540ms, size 1024KB
    *** 2010-08-03 00:08:50.818
    Warning: log write time 1810ms, size 1025KB
    *** 2010-08-03 00:08:58.990
    Warning: log write time 1640ms, size 942KB
    *** 2010-08-03 00:09:21.303
    Warning: log write time 1580ms, size 1024KB
    *** 2010-08-03 00:09:33.710
    Warning: log write time 1790ms, size 1025KB
    *** 2010-08-03 00:09:42.382
    Warning: log write time 1860ms, size 1024KB
    *** 2010-08-03 00:09:50.945
    Warning: log write time 1590ms, size 1024KB
    *** 2010-08-03 10:28:34.753
    Warning: log write time 2330ms, size 47KB
    *** 2010-08-03 10:29:44.440
    Warning: log write time 810ms, size 1KB
    *** 2010-08-03 10:32:27.392
    Warning: log write time 890ms, size 1KB
    *** 2010-08-03 10:36:02.536
    Warning: log write time 2070ms, size 8KB
    *** 2010-08-03 10:44:34.174
    Warning: log write time 740ms, size 7KB
    *** 2010-08-03 12:04:46.687
    Warning: log write time 2850ms, size 86KB
    *** 2010-08-03 12:04:52.656
    Warning: log write time 2970ms, size 85KB
    *** 2010-08-03 12:04:54.547
    Warning: log write time 1810ms, size 45KB
    *** 2010-08-03 14:06:20.827
    Warning: log write time 500ms, size 0KB
    *** 2010-08-03 14:13:46.404
    Warning: log write time 820ms, size 0KB
    *** 2010-08-03 15:50:32.996
    *** 2010-08-03 15:50:32.996 57052 kcrr.c
    Initializing NetServer[LNS1] for dest=STUSTDBY mode ASYNC
    Initializing PGA storage for Netserver communication
    LNSb is not running anymore.
    Check if new Async LNS1 needs to be started..
    .. yes it needs to be started
    Starting LNS1 ...
    Subscribing to KSR Channel [id=1]
    success!
    Indicating recv buffer for KSR Channel [id=1]
    success
    Waiting for LNS1 to initialize itself
    *** 2010-08-03 15:50:36.012 57343 kcrr.c
    Netserver LNS1 [pid 5764] for mode ASYNC has been initialized
    Performing a channel reset to ignore previous responses
    Connecting as publisher to KSR Channel [id=1]
    Successfully started LNS1 [pid 5764] for dest STUSTDBY mode ASYNC ocis=0x00000000144AE5B0
    *** 2010-08-03 15:50:36.012 57846 kcrr.c
    Making upiahm request to LNS1 [pid 5764]: Begin Time is <08/03/2010 15:50:32>. NET_TIMEOUT = <10> seconds
    *** 2010-08-03 16:00:58.266
    *** 2010-08-03 16:00:58.266 57052 kcrr.c
    Initializing NetServer[LNS1] for dest=STUSTDBY mode ASYNC
    LNSb is not running anymore.
    Check if new Async LNS1 needs to be started..
    Previous Async LNS1 [pid 5764] is running .. no need to start another one
    LGWR detected that Async LNS1 [pid 5764] is already running
    *** 2010-08-03 16:04:44.289
    *** 2010-08-03 16:04:44.289 57052 kcrr.c
    Initializing NetServer[LNS1] for dest=STUSTDBY mode ASYNC
    LNSb is not running anymore.
    Check if new Async LNS1 needs to be started..
    Previous Async LNS1 [pid 5764] is running .. no need to start another one
    LGWR detected that Async LNS1 [pid 5764] is already running
    *** 2010-08-03 16:07:36.747
    *** 2010-08-03 16:07:36.747 57052 kcrr.c
    Initializing NetServer[LNS1] for dest=STUSTDBY mode ASYNC
    LNSb is not running anymore.
    Check if new Async LNS1 needs to be started..
    Previous Async LNS1 [pid 5764] is running .. no need to start another one
    LGWR detected that Async LNS1 [pid 5764] is already running
    *** 2010-08-03 16:13:06.633
    Warning: log write time 570ms, size 0KB
    Did I miss something while creating standby.???
    Thanks for the help

    are these databases on same server.
    BTW These are not errors, they are warning ... I have also seen these kind of warning many times. they can be ignored if ur shipping and recovery on standby is Preperly happening.
    Regards
    jasbir

  • Recover standby database apply archivelog slow

    1.recover managed standby database disconnect from session;
    2.view alert file,find apply a achivelog last block need 10m
    3.view metalink MAA - Data Guard Redo Apply and Media Recovery Best Practices 10gR1
    modify database parameter,still is slow
    3.view v$managed_standby
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:49:09
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:56
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:57
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:57
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:58
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:58
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:58
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:59
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:59
    SQL>
    SQL> //
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:51:59
    SQL> //
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:00
    SQL> //
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:00
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:00
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:00
    SQL> //
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:01
    SQL> /
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:01
    SQL> //
    PROCESS SEQUENCE# THREAD# BLOCK# BLOCKS TIME
    MRP0 133538 1 581533 581534 15-APR-2011 10:52:01
    4. dump mrp
    SO: 0xc103c9b58, type: 4, owner: 0xc172b7820, flag: INIT/-/-/0x00
    (session) sid: 1087 trans: (nil), creator: 0xc172b7820, flag: (51) USR/- BSY/-/-/-/-/-
    DID: 0001-0013-00000002, short-term DID: 0000-0000-00000000
    txn branch: (nil)
    oct: 0, prv: 0, sql: (nil), psql: (nil), user: 0/SYS
    service name: SYS$BACKGROUND
    waiting for 'PX Deq: Par Recov Reply' blocking sess=0x(nil) seq=6397 wait_time=0 seconds since wait started=3
    sleeptime/senderid=10010000, passes=19f, =0
    Dumping Session Wait History
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1953964
    sleeptime/senderid=10010000, passes=19e, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954140
    sleeptime/senderid=10010000, passes=19d, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954066
    sleeptime/senderid=10010000, passes=19c, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954065
    sleeptime/senderid=10010000, passes=19b, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954129
    sleeptime/senderid=10010000, passes=19a, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954061
    sleeptime/senderid=10010000, passes=199, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1953991
    sleeptime/senderid=10010000, passes=198, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954123
    sleeptime/senderid=10010000, passes=197, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954120
    sleeptime/senderid=10010000, passes=196, =0
    for 'PX Deq: Par Recov Reply' count=1 wait_time=1954073
    sleeptime/senderid=10010000, passes=195, =0
    5.how to read dump process file
    Edited by: 852786 on 2011-4-16 上午1:46

    SO: 0xbf34d65b0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f4990d0, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000001A-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17426408, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17426418
    SO: 0xbf34d6570, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f499020, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000019-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17426380, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17426390
    SO: 0xbf34d6530, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498f88, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000018-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174262f8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17426308
    SO: 0xbf34d64f0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498ef0, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000017-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17426270, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17426280
    SO: 0xbf34d64b0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498e58, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000016-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174261e8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc174261f8
    SO: 0xbf34d6470, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498dc0, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000015-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17426160, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17426170
    SO: 0xbf34d6430, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498d28, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000014-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174260d8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc174260e8
    SO: 0xbf34d63f0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498c90, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000013-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17426050, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17426060
    SO: 0xbf34d63b0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498bf8, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000012-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425fc8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425fd8
    SO: 0xbf34d6370, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498b60, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000011-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425f40, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425f50
    SO: 0xbf34d6330, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498ac8, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000010-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425eb8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425ec8
    SO: 0xbf34d62f0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498a30, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000000F-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425e30, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425e40
    SO: 0xbf34d62b0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498998, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000000E-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425da8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425db8
    SO: 0xbf34d6270, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498900, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000000D-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425d20, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425d30
    SO: 0xbf34d6230, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498868, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000000C-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425c80, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425c90
    SO: 0xbf34d61f0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f4987d0, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000000B-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425bf8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425c08
    SO: 0xbf34d61b0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498738, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-0000000A-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425b70, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425b80
    SO: 0xbf34d6170, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498688, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000009-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425ae8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425af8
    SO: 0xbf34d6130, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f4985f0, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000008-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425a60, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425a70
    SO: 0xbf34d60f0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498558, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000007-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174259d8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc174259e8
    SO: 0xbf34d60b0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f4984c0, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000006-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425950, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425960
    SO: 0xbf34d6070, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498428, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000005-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174258c8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc174258d8
    SO: 0xbf34d6030, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498390, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000004-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425840, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425850
    SO: 0xbf34d5ff0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f4982f8, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000003-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174257b8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc174257c8
    SO: 0xbf34d5fb0, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f498260, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000002-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425730, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425740
    SO: 0xbf34d5f70, type: 27, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    SO: 0xc0f4981c8, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000001-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc174256a8, mode: SSX, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc174256b8
    SO: 0xc0f498130, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) FS-00000000-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425510, mode: S, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425520
    SO: 0xc0f498000, type: 5, owner: 0xbfce71e80, flag: INIT/-/-/0x00
    (enqueue) MR-00000000-00000000     DID: 0001-0016-00000002
    lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
    res: 0xc17425488, mode: S, lock_flag: 0x0
    own: 0xc0e403c70, sess: 0xc0e403c70, proc: 0xc0c28ae80, prv: 0xc17425498
    SO: 0xc13fe20e8, type: 61, owner: 0xc0e403c70, flag: INIT/-/-/0x00
    Process Queue--kxfpq: 0x0xc13fe20e8, serial: 513, # of qrefs: 16,
    inc: 0
    client 2, detached proc: 0x(nil), QC qref 0x(nil), flags: FEML
    Queue Descriptor--kxfpqd: 0x0xc13fe2140, remote queue addr: 0x0xc1
    3fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fdd200, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdd200, ser: 513, seq: 1123, err
    or: 0
    opp qref: 0x0xc13fdbb50, process: 0x0xc0e2c33e8, bufs: {0x(nil),
    0x0xb3fe124f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fdd310, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdd248, remote queue addr: 0x
    0xc13fdfb40
    instance id: 1, server id: 15, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fe124f8, 0x0xb3fe224f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fe124f8, type: DTA, bufnum: 1
    ser: 513, seq: 1117, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdbb50, from qref: 0x0xc13fdd200, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fe12550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fdd410, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdd410, ser: 513, seq: 936, erro
    r: 0
    opp qref: 0x0xc13fdbd60, process: 0x0xc172b8fd8, bufs: {0x(nil),
    0x0xb3fe024f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fdd520, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdd458, remote queue addr: 0x
    0xc13fdfd98
    instance id: 1, server id: 14, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fe024f8, 0x0xb3fdf24f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fe024f8, type: DTA, bufnum: 1
    ser: 513, seq: 926, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdbd60, from qref: 0x0xc13fdd410, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fe02550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fdd620, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdd620, ser: 513, seq: 1076, err
    or: 0
    opp qref: 0x0xc13fdc5a0, process: 0x0xc0f28aa28, bufs: {0x0xb3fd
    d24f8, 0x0xb3fdc24f8}
    state: 10010, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fdd730, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdd668, remote queue addr: 0x
    0xc13fdfff0
    instance id: 1, server id: 13, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fdd24f8, 0x0xb3fdc24f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fdd24f8, type: DTA, bufnum: 0
    ser: 513, seq: 1069, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdc5a0, from qref: 0x0xc13fdd620, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fdd2550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    Message Buffer--kxfpmh: 0x0xb3fdc24f8, type: DTA, bufnum: 1
    ser: 513, seq: 1076, flags: DIAL, status: RCV, err: 0
    to qref: 0x0xc13fdd620, from qref: 0x0xc13fdc5a0, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fdc2550, remote queue addr:
    0x0xc13fdfff0
    instance id: 1, server id: 13, flags: INIT
    can't dump contents, client unknown
    SO: 0xc13fdda40, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdda40, ser: 513, seq: 1209, err
    or: 0
    opp qref: 0x0xc13fdc7b0, process: 0x0xc102860a8, bufs: {0x(nil),
    0x0xb3fe324f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fddb50, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdda88, remote queue addr: 0x
    0xc13fe0248
    instance id: 1, server id: 12, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fe324f8, 0x0xb3fda24f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fe324f8, type: DTA, bufnum: 1
    ser: 513, seq: 1203, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdc7b0, from qref: 0x0xc13fdda40, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fe32550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fddc50, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fddc50, ser: 513, seq: 822, erro
    r: 0
    opp qref: 0x0xc13fdc180, process: 0x0xc0c28be50, bufs: {0x(nil),
    0x0xb3fd924f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fddd60, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fddc98, remote queue addr: 0x
    0xc13fe04a0
    instance id: 1, server id: 11, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fd924f8, 0x0xb3f069230}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fd924f8, type: DTA, bufnum: 1
    ser: 513, seq: 816, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdc180, from qref: 0x0xc13fddc50, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fd92550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fdde60, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdde60, ser: 513, seq: 829, erro
    r: 0
    opp qref: 0x0xc13fdbf70, process: 0x0xc0d2a44f8, bufs: {0x(nil),
    0x0xb3f456dc8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fddf70, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fddea8, remote queue addr: 0x
    0xc13fe06f8
    instance id: 1, server id: 10, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3f456dc8, 0x0xb3fd724f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3f456dc8, type: DTA, bufnum: 1
    ser: 513, seq: 823, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdbf70, from qref: 0x0xc13fdde60, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3f456e20, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fde070, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fde070, ser: 513, seq: 1321, err
    or: 0
    opp qref: 0x0xc13fdb940, process: 0x0xc0e2c2c00, bufs: {0x(nil),
    0x0xb3fd424f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fde180, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fde0b8, remote queue addr: 0x
    0xc13fe0950
    instance id: 1, server id: 9, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fd424f8, 0x0xb3f854960}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fd424f8, type: DTA, bufnum: 1
    ser: 513, seq: 1315, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdb940, from qref: 0x0xc13fde070, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fd42550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fde490, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fde490, ser: 513, seq: 1002, err
    or: 0
    opp qref: 0x0xc13fdd830, process: 0x0xc172b87f0, bufs: {0x(nil),
    0x0xb3fde24f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fde5a0, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fde4d8, remote queue addr: 0x
    0xc13fe0ba8
    instance id: 1, server id: 8, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fde24f8, 0x0xb3fd224f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fde24f8, type: DTA, bufnum: 1
    ser: 513, seq: 995, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdd830, from qref: 0x0xc13fde490, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fde2550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fde6a0, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fde6a0, ser: 513, seq: 1198, err
    or: 0
    opp qref: 0x0xc13fdcbd0, process: 0x0xc0f28a240, bufs: {0x(nil),
    0x0xb3fd024f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fde7b0, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fde6e8, remote queue addr: 0x
    0xc13fe0e00
    instance id: 1, server id: 7, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fd024f8, 0x0xb3f059230}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3fd024f8, type: DTA, bufnum: 1
    ser: 513, seq: 1191, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdcbd0, from qref: 0x0xc13fde6a0, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3fd02550, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fdeac0, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdeac0, ser: 513, seq: 1080, err
    or: 0
    opp qref: 0x0xc13fdcde0, process: 0x0xc102858c0, bufs: {0x(nil),
    0x0xb3f436dc8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fdebd0, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdeb08, remote queue addr: 0x
    0xc13fe1058
    instance id: 1, server id: 6, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3f436dc8, 0x0xb3fce24f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3f436dc8, type: DTA, bufnum: 1
    ser: 513, seq: 1074, flags: STRE, status: FRE, err: 0
    to qref: 0x0xc13fdcde0, from qref: 0x0xc13fdeac0, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3f436e20, remote queue addr:
    0x0xc13fe20e8
    instance id: 1, server id: 65535, flags: ISQC
    SO: 0xc13fdeee0, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdeee0, ser: 513, seq: 1418, err
    or: 0
    opp qref: 0x0xc13fde280, process: 0x0xc0c28b668, bufs: {0x(nil),
    0x0xb3f824960}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fdeff0, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdef28, remote queue addr: 0x
    0xc13fe12b0
    instance id: 1, server id: 5, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3f824960, 0x0xb3fcc24f8}, sta
    te: 10011
    Message Buffer--kxfpmh: 0x0xb3f824960, type: NUL, bufnum: 1
    ser: 513, seq: 1407, flags: DIAL, status: FRE, err: 0
    to qref: 0x0xc13fdeee0, from qref: 0x0xc13fde280, inc: 0, send
    er:
    Queue Descriptor--kxfpqd: 0x0xb3f8249b8, remote queue addr:
    0x0xc13fe12b0
    instance id: 1, server id: 5, flags: INIT
    SO: 0xc13fdf0f0, type: 62, owner: 0xc13fe20e8, flag: -/-/-/0x00
    Queue Reference--kxfpqr: 0x0xc13fdf0f0, ser: 513, seq: 949, erro
    r: 0
    opp qref: 0x0xc13fdc390, process: 0x0xc0d2a3d10, bufs: {0x(nil),
    0x0xb3fc924f8}
    state: 00000, flags: SMEM, nulls 0, hint 0x1
    latch 0x0xc13fdf200, remote descriptor:
    Queue Descriptor--kxfpqd: 0x0xc13fdf138, remote queue addr: 0x
    0xc13fe1508
    instance id: 1, server id: 4, flags: INIT
    recovery info--opr: 0, bufs: {0x0xb3fc924f8, 0x0xb3fca24f8}, sta
    te: 10011
    ---------------------------------------

  • Recovered standby database, still it is looking for gap sequence

    I have lost archive logs in both primary and standby database(deleted due to low space).
    So folloed the below steps.
    1.Identified minimum SCN from standby database.
    2.Took inremental backup from primary database according this SCN(nnnnn)
    3.Transfer this backup file to standby side
    4. Cataloged database backups and recovered the database.
    But medial recovery is searching for missing archived logs. Please advice.

    957905 wrote:
    I have lost archive logs in both primary and standby database(deleted due to low space).
    So folloed the below steps.
    1.Identified minimum SCN from standby database.
    2.Took inremental backup from primary database according this SCN(nnnnn)
    3.Transfer this backup file to standby side
    4. Cataloged database backups and recovered the database.
    But medial recovery is searching for missing archived logs. Please advice.Probably you have not restored new standby controlfile before performing recovery.

  • Recover standby database after primary failed

    Hi,
    I'm having 11g setup with 2 standby databases.  My scenario is i'm doing failover on one standby[new primary] and converting old primary as a standby.  Question is what is the status of another standby? have to create new standby or can recover using flash option?
    regards,
    jp

    '11g' is not a version, it is a marketing label. You need to post your version in the format <x>.<x>.<x>.<x>
    Yes, I know this is asked much.
    Also your question sadly lacks on details, as in this version of Oracle you can cascade standby databases (standby 1 can cascade to standby 2)
    This begs the simple question:
    Did you try?
    If so, what happened?
    If you didn't try, why didn't you try? What can happen?
    Sybrand Bakker
    Senior Oracle DBA

  • Rman standby database error

    Hi All,
    I am getting the following error when trying to CONFIGURE STANDBY database using rman,db version 10.2.0.2(HP-UX 11.2),library link seems to be fine.
    do i have to relink the library to oracle on the stanby db host too? any suggestion please.
    connect target / catalog rman@xx auxiliary sys@aux
    RMAN> run {
    allocate channel c1 device type 'sbt_tape';
    ALLOCATE AUXILIARY CHANNEL aux1 DEVICE TYPE 'SBT_TAPE';
    duplicate target database for standby dorecover;}2> 3> 4>
    allocated channel: c1
    channel c1: sid=164 instance=WWPODSP1 devtype=SBT_TAPE
    channel c1: Data Protector A.06.00/331
    released channel: c1
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03009: failure of allocate command on aux1 channel at 05/09/2007 16:57:07
    ORA-19554: error allocating device, device type: SBT_TAPE, device name:
    ORA-27211: Failed to load Media Management Library
    Additional information: 11
    RMAN> show all;
    RMAN configuration parameters are:
    CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 45 DAYS;
    CONFIGURE BACKUP OPTIMIZATION ON;
    CONFIGURE DEFAULT DEVICE TYPE TO 'SBT_TAPE';
    CONFIGURE CONTROLFILE AUTOBACKUP ON;
    CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE 'SBT_TAPE' TO 'control_file_%F';
    CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
    CONFIGURE DEVICE TYPE SBT_TAPE PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
    CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
    CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE SBT_TAPE TO 1; # default
    CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
    CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE SBT_TAPE TO 1; # default
    CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
    CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS 'SBT_LIBRARY=/opt/omni/lib/libob2oracle8_64bit.so';
    CONFIGURE MAXSETSIZE TO UNLIMITED; # default
    CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
    CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
    CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
    CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/10.2.0/db_1/dbs/snapcf_WWPODSP1.f'; # default

    sorry here is the exact commands.
    rman TARGET / AUXILIARY sys/xxx@WWPODSS catalog rman@rman
    Recovery Manager: Release 10.2.0.2.0 - Production on Wed May 9 17:29:37 2007
    Copyright (c) 1982, 2005, Oracle. All rights reserved.
    connected to target database: WWPODSP (DBID=669261963)
    recovery catalog database Password:
    connected to recovery catalog database
    connected to auxiliary database: WWPODSP (not mounted)
    RMAN> run
    2> {
    3> duplicate target database for standby dorecover;}
    Starting Duplicate Db at 09-MAY-07
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of Duplicate Db command at 05/09/2007 17:33:35
    ORA-19554: error allocating device, device type: SBT_TAPE, device name:
    ORA-27211: Failed to load Media Management Library
    Additional information: 11
    RMAN>

  • Oracle 9i standby database error

    Hi all,
    I am getting this error on my standby DB while it's archiving a log...and I've even restarted the DB and media recovery.Also, it's not a disk space issue.
    Errors in file f:\oracle\admin\xxxprd\bdump\xxxprd_pmon_5224.trc:
    ORA-16145: archival for thread# 1 sequence# 19113 in progress
    Some recovered datafiles maybe left media fuzzy
    Media recovery may continue but open resetlogs may fail
    Any ideas ?
    Much appreciated

    Hi;
    Please see below notes which could be helpful for your issue
    https://support.oracle.com/CSP/main/article?cmd=show&type=COMMUNITIES&doctype=COMTHREAD&id=75725
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14219/e12700.htm#sthref4916
    OERR: ORA-16145 archival for thread# %s sequence# %s in progress [ID 172871.1]
    Regard
    Helios

  • ERROR in creating a Physical Standby Database

    Hello all,
    I am using a Windows Vista O/S and Oracle 10g Enterprise Edition. Both my primary and standby are on the same host. Created standby instance using oradim.
    I have followed instruction in the documentation and created a manual standby database.
    primary database is 'orcl'
    standby database is 'stby'
    My primary archives is not being copied to the the stby location as confirmed by an error shown:
    SQL> select dest_name, status from v$archive_dest_status
    DEST_NAME
    STATUS
    LOG_ARCHIVE_DEST_1
    VALID
    LOG_ARCHIVE_DEST_2
    ERROR
    Also my standby does not open in read only mode, error:
    SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;
    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION
    ERROR at line 1:
    ORA-01153: an incompatible media recovery is active
    SQL> ALTER DATABASE open read only;
    ALTER DATABASE open read only
    ERROR at line 1:
    ORA-01154: database busy. Open, close, mount, and dismount not allowed now
    I have copied both my prim and stby init.ora here for your reference
    PRIMARY INIT
    ============================================
    orcl.__db_cache_size=394264576
    orcl.__java_pool_size=20971520
    orcl.__large_pool_size=4194304
    orcl.__shared_pool_size=184549376
    orcl.__streams_pool_size=0
    *.audit_file_dest='E:\oracle\product\10.2.0\admin\orcl\adump'
    *.background_dump_dest='E:\oracle\product\10.2.0\admin\orcl\bdump'
    *.compatible='10.2.0.3.0'
    *.control_files='E:\oracle\product\10.2.0\oradata\orcl\control01.ctl'
    *.core_dump_dest='E:\oracle\product\10.2.0\admin\orcl\cdump'
    *.db_block_size=8192
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_name='orcl'
    *.db_unique_name=orcl
    *.db_recovery_file_dest_size=2147483648
    *.db_recovery_file_dest='E:\oracle\product\10.2.0\flash_recovery_area'
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=orclXDB)'
    *.job_queue_processes=10
    *.nls_language='ENGLISH'
    *.nls_territory='UNITED KINGDOM'
    *.open_cursors=300
    *.pga_aggregate_target=203423744
    *.processes=150
    *.remote_login_passwordfile='EXCLUSIVE'
    *.sga_target=611319808
    *.undo_management='AUTO'
    *.undo_tablespace='UNDOTBS1'
    *.user_dump_dest='E:\oracle\product\10.2.0\admin\orcl\udump'
    *.standby_file_management=AUTO
    *.log_archive_dest_1='location=E:\oracle\product\10.2.0\flash_recovery_area\orcl\archivelog valid_for=(all_logfiles,primary_role) db_unique_name=orcl'
    *.log_archive_dest_state_1='ENABLE'
    *.log_archive_dest_2='SERVICE=stby LGWR ASYNC valid_for=(online_logfiles,primary_role) db_unique_name=stby'
    *.log_archive_dest_state_2='ENABLE'
    *.log_archive_config='DG_CONFIG=(orcl,stby)'
    *.db_file_name_convert='orcl','stby'
    *.fal_client='orcl'
    *.fal_server='stby'
    *.db_file_name_convert='E:\oracle\product\10.2.0\oradata\stby', 'E:\oracle\product\10.2.0\oradata\orcl'
    *.log_file_name_convert='E:\oracle\product\10.2.0\oradata\stby', 'E:\oracle\product\10.2.0\oradata\orcl'
    *.standby_file_management=AUTO
    ======================================
    STANDBY INIT.ora
    stby.__db_cache_size=423624704
    stby.__java_pool_size=4194304
    tby.__large_pool_size=4194304
    stby.__large_pool_size=4194304
    stby.__shared_pool_size=171966464
    stby.__streams_pool_size=0
    *.audit_file_dest='E:\oracle\product\10.2.0\admin\stby\adump'
    *.background_dump_dest='E:\oracle\product\10.2.0\admin\stby\bdump'
    *.compatible='10.2.0.3.0'
    *.control_files='E:\oracle\product\10.2.0\oradata\stby\CONTROL_SB01.CTL'
    *.core_dump_dest='E:\oracle\product\10.2.0\admin\stby\cdump'
    *.db_block_size=8192
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_file_name_convert='orcl','stby'
    *.db_name='orcl'
    *.db_recovery_file_dest_size=2147483648
    *.db_recovery_file_dest='E:\oracle\product\10.2.0\flash_recovery_area\STBY'
    *.db_unique_name='stby'
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=orclXDB)'
    *.fal_client='stby'
    *.fal_server='orcl'
    *.job_queue_processes=10
    *.log_archive_config='DG_CONFIG=(orcl,stby)'
    *.log_archive_dest_1='location=E:\oracle\product\10.2.0\flash_recovery_area\stby valid_for=(all_logfiles,primary_role) db_unique_name=stby'
    *.log_archive_dest_2='SERVICE=stby LGWR ASYNC valid_for=(online_logfiles,primary_role) db_unique_name=orcl'
    *.log_archive_dest_state_1='ENABLE'
    *.log_archive_dest_state_2='ENABLE'
    *.log_file_name_convert='E:\oracle\product\10.2.0\oradata\orcl','E:\oracle\product\10.2.0\oradata\stby'
    *.nls_language='ENGLISH'
    *.nls_territory='UNITED KINGDOM'
    *.open_cursors=300
    *.pga_aggregate_target=203423744
    *.processes=150
    *.remote_login_passwordfile='EXCLUSIVE'
    *.sga_target=611319808
    *.standby_archive_dest='USE_DB_RECOVERY_FILE_DEST'
    *.standby_file_management='AUTO'
    *.undo_management='AUTO'
    *.undo_tablespace='UNDOTBS1'
    *.user_dump_dest='E:\oracle\product\10.2.0\admin\stby\udump'
    =================================================
    Any help would be much appreciated. Apologies for my ignorance!
    Thanks!
    null
    null

    OK that solves part of the problem.
    Now this is very strange error I am getting. The path it looks to recover from archive is not right, it should point as per my actual archive log dest. See below.
    Standby:
    =================
    SQL> alter database open;
    alter database open
    ERROR at line 1:
    ORA-16004: backup database requires recovery
    ORA-01152: file 1 was not restored from a sufficiently old backup
    ORA-01110: data file 1: 'E:\ORACLE\PRODUCT\10.2.0\ORADATA\STBY\SYSTEM01.DBF'
    SQL> recover standby database until cancel;
    ORA-00279: change 619067 generated at 06/22/2008 13:20:14 needed for thread 1
    ORA-00289: suggestion :
    E:\ORACLE\PRODUCT\10.2.0\DB_1\RDBMS\ARC00006_0658028298.001
    ORA-00280: change 619067 for thread 1 is in sequence #6
    Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
    ORA-00308: cannot open archived log
    'E:\ORACLE\PRODUCT\10.2.0\DB_1\RDBMS\ARC00006_0658028298.001'
    ORA-27041: unable to open file
    OSD-04002: unable to open file
    O/S-Error: (OS 2) The system cannot find the file specified.
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01152: file 1 was not restored from a sufficiently old backup
    ORA-01110: data file 1: 'E:\ORACLE\PRODUCT\10.2.0\ORADATA\STBY\SYSTEM01.DBF'
    ==================================================
    Why is not pointing to the destination below, where the above archives reside??
    =====================================
    SQL> archive log list
    Database log mode Archive Mode
    Automatic archival Enabled
    Archive destination E:\oracle\product\10.2.0\flash_recovery_area\stby
    Oldest online log sequence 12
    Next log sequence to archive 0
    Current log sequence 14
    =================

  • Error while trying to setup a standby database.

    While tring to setup a standby database (with version 8.1.7.0.0) i am running the command:
    recover managed standby database timeout 20;
    Here is the errors i'm getting:
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01152: file 1 was not restored from a sufficiently old backup
    ORA-01110: data file 1: '/data/syndicate/standby/system/dup75_system01.dbf'
    ORA-16016: archived log for thread 1 sequence# 6518 unavailable
    Although it says that file 1 was not restored from sufficiently old backup I've copied all the datafiles from the primary database at once - using a script.
    The archived log sequence it asks for is non-existant - the last one i have is # 6517.
    Thank you guys in advance for any ideas.
    Greg

    I am having the same problem as you are when I am applying the current logs after successfully mounting standby database as seen below:
    SVRMGR> alter database mount standby database;
    Statement processed.
    SVRMGR> recover standby database ;
    ORA-00279: change 2939586 generated at 07/28/2002 01:09:33 needed for thread 5718
    ORA-00289: suggestion : C:\ORACLE\ORADATA\STANDBY\ARCHIVE\STANDBYT571S00362.ARC
    ORA-00280: change 2939586 for thread 5718 is in sequence #362
    Specify log: {=suggested | filename | AUTO | CANCEL}
    CANCEL
    ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
    ORA-01152: file 1 was not restored from a sufficiently old backup
    ORA-01110: data file 1: 'C:\ORACLE\ORADATA\STANDBY\SYSTEM01.DBF'
    SVRMGR>
    Just before copying the datafiles from the primary database I did the following:
    Connected to the primary database:
    1. alter database create standby controlfile as .....
    2. alter system switch logfile
    3. shutdown normal
    4. took a cold backup (including archived redo logs)
    Copied the data files, redolog files, archived redo logs and standby controlfile to standby database:
    1. made copies of standby controlfile over current standby ones.
    2. startup nomount (the standby database)
    3. alter database mount standby database;
    4. recover standby database;
    That's when it asks for an archived redo log that does'nt exist and when I enter cancel, that's when I get the above error?
    Am I not applying the latest redo logs to the latest cold backup of the primary database?
    Your help would be much appreciated!

Maybe you are looking for

  • No signal Champaign, IL

    Both phones on my account have no signal.  Is anyone else having issues?  I can't even make a call.

  • Problem in SAP-Script

    Hi All, I am creating one report form using SAP Script, it is a replacement of standard SAP Script and which uses standard driver program. I am getting one dump while running SAP Script, it shows like "variable with name 'XXX' has not been used in so

  • 'Missing file' warning.

    Hello Keynote Gurus: I have just recently installed Keynote '08 (version 4.0.2), and when I apply one of my themes, I get the following Document Warnings: Missing File - tilepaperpurple.jpg Missing File - tilepaperred.jpg Missing File - tilepaperred.

  • Document set and list

    Hi, I need to add a list to a document set or link it somehow. I tried on the document set page to add webpart a list which works but dont know how to filter only list items related to the particular document set. Has anyone done something similiar?

  • Oracle Personalization..-How can we open it?

    Hi everybody. Just want to ask. We have problem with the Oracle Customization (user can see any portlet, even which they do not have access to,when they click Add portlet in Customization). So, one of the user ask us to look into Oracle Personalizati