DR Server: Standby Database

Hello All,
I need to Design a DR Server for my Client.
I have decided to go for a hot Standby Database.
I am very new to DB2. Are there any Documents that can help me on designing the procedure for this. If you can let me know the steps/commands in detail that would also be helpful.
I have read the DB2 Operations guide available in SAP Marketplace which gives a high level Strategy.
1. On Source System create a Snapshot of the DB. What exactly is a Snapshot? How to create a Split Image?
2.On the Target (DR Server) create the db after mounting the split image.
Can you please help me in understanding how to aplly/execute this.
Thanks,Bidwan

Hi,
Please see the following documentation which should be of help:
[BACKUP DATABASE command|http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.cmd.doc/doc/r0001933.html]
The default behavior for a snapshot backup is a FULL DATABASE OFFLINE
backup of all paths that make up the database including all containers,
local volume directory, database path (DBPATH), and primary log and
mirror log paths (INCLUDE LOGS is the default for all snapshot backups
unless EXCLUDE LOGS is explicitly stated).
[Backing up and restoring data is faster with snapshot backups|http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.wn.doc/doc/c0052793.html]
[Performing a snapshot backup |http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.ha.doc/doc/t0052799.html]
[Restoring from a snapshot backup image|http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.ha.doc/doc/t0052798.html]
[IBM Redbook "High Availability and Disaster Recovery Options for DB2 on Linux, UNIX, and Windows"|http://www.redbooks.ibm.com/redbooks/pdfs/sg247363.pdf]
Section 14.2 Backup and restore database with snapshot backup
o further guides
http://www.ibm.com/developerworks/tivoli/library/t-db2dptsam/
http://www.redbooks.ibm.com/redbooks/pdfs/sg246423.pdf
Hope this helps,
Paul

Similar Messages

Problem while creating Physical standby database on remote Server

Hi
I try to create a physical standby datbaase using OEM 10gr2, it is working when I create on same server. No issue but when I try to create same on remote server it is giving below warning and process failing. Please let me know what needs to be down in this senario...
It is showing redo log information in v$log and v$log_file but not creating physically on PHYSICAL Standby instance..
I appriciate your help and ur time too..
These messages are from PRIMARY database alert*.log file
There is space for up to 12 standby redo logfiles
Use the following SQL commands on the standby database to create
standby redo logfiles that match the primary database:
ALTER DATABASE ADD STANDBY LOGFILE 'srl1.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl2.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl3.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl4.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl5.f' SIZE 52428800;
WARNING: OMF is enabled on this database. Creating a physical
standby controlfile, when OMF is enabled on the primary
database, requires manual RMAN intervention to resolve OMF
datafile pathnames.
NOTE: Please refer to the RMAN documentation for procedures
describing how to manually resolve OMF datafile pathnames.
Thanks and Regards,
RK

Well, it is simply warning that "Creating a physical
standby controlfile, when OMF is enabled on the primary
database, requires manual RMAN intervention to resolve OMF
datafile pathnames.
NOTE: Please refer to the RMAN documentation for procedures
describing how to manually resolve OMF datafile pathnames"
For instance, the path for the logfile name in "ALTER DATABASE ADD STANDBY LOGFILE 'srl5.f' SIZE 52428800" may need to be changed manually. It is advisable to check the mentioned documentation.

Standby Database on separate server

Hi all,
I'm running Oracle 10g Standard Edition and i'm trying to setup a manual standby database. From what i have read this should be possible w/ Stand Edition. I have it working when both db's are on the same server. Now i need it to work when the db's are on separate linux servers. I have the following set in my init files for the log_archive_dest_# params:
Primary Db init file:
*.log_archive_config='DG_CONFIG'=(PRDDB,PRDDB2)'
prddb.log_archive_dest_1='location="/opt/oracle/product/10.2.0/db_1/flash_recovery_area/PRDDB/"','valid_for(ONLINE_LOGFILE,ALL_ROLES)','db_unique_name=PRDDB'
*.log_archive_dest_state_1=enable
prddb.log_archive_dest_2='service=PRDDB2','valid_for=(online_logfiles,primary_role)','db_unique_name=PRDDB2'
*.log_archive_dest_state_2=enable
Standby Db init file:
*.db_file_name_convert='/opt/oracle/product/10.2.0/db_1/oradata/prddb/','/opt/oracle/product/10.2.0/db_1/oradata/prddb2/'
*.log_file_name_convert='/opt/oracle/product/10.2.0/db_1/flash_recover_area/PRDDB/','/opt/oracle/product/10.2.0/db_1/flash_recovery_area/PRDDB2/','/opt/oracle/product/10.2.0/db_1/oradata/prddb/','/opt/oracle/product/10.2.0/db_1/oradata/prddb2/'
*.log_archive_config='DG_CONFIG=(PRDDB,PRDDB2)'
prddb2.log_archive_dest_1='location="/opt/oracle/product/10.2.0/db_1/flash_recovery_area/PRDDB2/"','valid_for(ALL_LOGFILES,ALL_ROLES)','db_unique_name=PRDDB2'
*.log_archive_dest_state_1=enable
prddb2.log_archive_dest_2='service=PRDDB','valid_for=(online_logfiles,primary_role)','db_unique_name=PRDDB'
*.log_archive_dest_state_2=enable
The problem i'm having is the arch log files aren't being transported to the other server. Do I have to have a common mount on the servers for them to send the log files (setting archive_log_archive_dest_2 to the common mount between servers)?? Currently i don't have a common mount between the 2 servers.
How does oracle transmit these log files? Does it take service name in log_archive_dest_2 and know which server & directory to copy the logs too. When i have this all setup on one server the logs succesfully copy to log_archive_dest_2 but i'm stumped as to how it works with separate servers accross the network. Can anyone point me in the right direction??

Thanks for the info Kuljeet. I was afraid of that. But Standard Edition should be able to run a Manual Standby database correct?? I am now manually copying over the archive logs but they are not automatically getting applied to the standby database. Any tips on what parameter to check in order to get these archive logs applying automatically. I have issued the "alter database recover managed standby database disconnect from session" so the databse should be in recover mode and applying the archive log files.
If i try and define the LOG_ARCHIVE_DEST_2 parameter the database tells me the
"ORA-00439: feature not enabled: Managed Standby". I queried v$option and Managed Standby is set to false.

Standby database archive log apply in production server.

Dear All,
How I apply standby database archive log apply in production server.
Please help me.
Thanks,
Manas

How can i use standby database as primary for that 48 hour ?Perform a switchover (role transitions).
First check if the standby is in sync with the primary database.
Primary database:
sql>select max(sequence#) from v$archived_log; ---> Value AStandby database:
sql>select max(sequence#) from v$archived_log where applied='YES'; -----> Value BCheck if Value B is same as Value A.
If the standby is in sycn with the primary database, then perform the switchover operation (refer the below link)
http://www.articles.freemegazone.com/oracle-switchover-physical-standby-database.php
http://docs.oracle.com/cd/B19306_01/server.102/b14230/sofo.htm
http://www.oracle-base.com/articles/9i/DataGuard.php#DatabaseSwitchover
manas
Handle: manas
Email: [email protected]
Status Level: Newbie
Registered: Jan 24, 2012
Total Posts: 10
Total Questions: 3 (3 unresolved)
Name Manas
Location kolkata Mark your questions as answered if you feel that you have got better answers rather than building up a heap of unanswered questions.

Can we create standby database on same server as primary database

I have Oracle 10g Rel2, OS Windows server 2003. Can I create standby database on same server on which we have our primary database?

Bharat Kaushik wrote:
I have VMware server s/w, do i need to create 2 virtual machines inside the one installation of VMware server? Or do i need to have 2 different installation of VMware server?You could think of VMWare Server in similar terms are Oracle Database Server.
The HOME: The VMWare Server s/w is put into a "VMWare Home" (I use VMWare Workstation, it goes into C:\Program Files\VMware\VMware Workstation)
The INSTANCE: WHen I create a VM Image, I specify the name (like a SID), configuration (like PFILE), and location (like ORACLE_BASE/oradata/ ...)
And just like Oracle with which I can have 2 or more instances running on the host (if there is sufficient memory and other resource), I can have 2 or more VM Images running on my host.
And just like Oracle, if I have insufficient resources, things get slow like molasses in the middle of winter.
Hope your desktop has at least 4GB of OS-addressible RAM and at least dual-core CPU. ALternately ... I hope you have lots of coin for the coffee machine.

Plan to shutdown server at site 1 supporting a standby database

We plan on shutting down all servers at site 1 for 1/2 day.
One server has a standby database.
The production database at site 2 will still be running and available to users.
Is there any special considerations when I startup the standby database?
Is there a special startup command?
Will it automatically apply all the archive redo logs from the production system?
Or is it a manual process?

Hi,
We are not switching over to the standby database.
When the power returns to site 1, we are simply starting up the
standby database, but as I understand it , there are archivelogs that
get pushed over to the standby.
How do we startup the standby after power returns to site 1.

How Auto starting Physical Standby Database ( when Server Restart)

Dear All(s)
I have configure oracle (10.2.0) dataguard on production database, some time server restart due to power disconnectivity, i want to restard dataguard when server restart. i am using Server 2003 32bit. Guide me how i can run following commands when server restart.
SQL> connect/as sysdba
SQL> Startup mount;
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
Thanks in Advance

i want to run these command each time when server restart.Yes, I did understand that... but what's the problem ? can't you schedule a task when the server starts ?

Goldengate between active and standby database server

Hi:
I want to make goldengate connection between active and standby database servers (bidirectional). is it possible?
if yes what steps is required rather than active-active connection.
PLease help!!!
Regards,
Abhishek

What do you mean by "standby" server? If you are referring to a Data Guard standby, how would bidirectional work? That's why one is primary, the other is standby.
If you are doing A-A, your "standby" is just another master in the N-way master architecture, so you already have a bi-di setup, right? Are you looking to replace Streams with GG?

Can standby database configured on VMWare server or not?

HI all,
Is it possible that our primary server is not on vmware server but standby server can be configured on VMWare server?
AFAIK, configuration of both the server should be same.
If our Primary server is not on Vmware server but standby database configured on Vmware server, Will it work or not.
Pls suggest me.....
And another thing is that Is it necessary to have a dedicated connection b/w primary and standby server for RFS process.
pls suggest me....

If our Primary server is not on Vmware server but standby database configured on Vmware server, Will it work or not.Logically speaking, it should not matter but YMMV.
And another thing is that Is it necessary to have a dedicated connection b/w primary and standby server for RFS process.what is RFS process? Could you elaborate?

How to create a standby database in another server?

hi,
I want to create a standby database in another machine, please tell me the steps fot it.
thx.

Can you tell what version of Oracle you are running, because the steps can vary depending on your version.

Issue in configuring Standby Database from Active database in 11g by RMAN

POSTED THE SAME IN DATA GUARD CATEGORY UNABLE TO DELETE
Hi All,
I am facing issue in creating the standby database from Active database using RMAN and getting the below issue after i executed the duplicate command.
Version of Database:11g(11.2.0.1.0)
Operating System:Linux 5
Error:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 12/21/2012 17:26:52
RMAN-03015: error occurred in stored script Memory Script
RMAN-04006: error from auxiliary database: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Primary Database Entries:
Tnsentry:
SONYPRD =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.131)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprd.localdomain)(UR=A)
SONYPRDSTBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.132)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprdstby)(UR=A)
Listner Entry:
SID_LIST_SONYPRD =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtproc)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(PROGRAM = extproc)
(SID_DESC =
(SID_NAME = SONYPRD)
(GLOBAL_DBNAME = SONYPRD)
Auxiliary Details:
Tns Entry:
SONYPRD =
(DESCRIPTION =
# (ADDRESS = (PROTOCOL = TCP)(HOST = oracle11g.localdomain)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.131)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprd.localdomain)
SONYPRDSTBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.132)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprdstby)(UR=A)
Listener Entry in auxiliary:
SID_LIST_SONYPRDSTBY =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = SONYPRDSTBY)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = SONYPRDSTBY)
TNSPING from Primary DB:
[oracle@oracle11g ~]$ tnsping sonyprdstby
TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 17:39:28
Copyright (c) 1997, 2009, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.132)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = sonyprdstby)(UR=A)))
OK (0 msec)
TNSPING from Auxuliary server
[oracle@oracle11gstby ~]$ tnsping sonyprd
TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 17:40:19
Copyright (c) 1997, 2009, Oracle. All rights reserved.
Used parameter files:
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.131)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = sonyprd.localdomain)))
OK (10 msec)
Script Used for duplicate:
run {
allocate channel prmy1 type disk;
allocate channel prmy2 type disk;
allocate channel prmy3 type disk;
allocate channel prmy4 type disk;
allocate auxiliary channel stby type disk;
duplicate target database for standby from active database
spfile
parameter_value_convert 'sonyprd','sonyprdstby'
set db_unique_name='sonyprdstby'
set db_file_name_convert='/sonyprd/','/sonyprdstby/'
set log_file_name_convert='/sonyprd/','/sonyprdstby/'
set control_files='/u01/app/oracle/oradata/control01.ctl'
set log_archive_max_processes='5'
set fal_client='sonyprdstby'
set fal_server='sonyprd'
set standby_file_management='AUTO'
set log_archive_config='dg_config=(sonyprd,sonyprdstby)'
set log_archive_dest_2='service=sonyprd ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=sonyprd'
Tried the script from both Primary and auxiliary but no luck
[oracle@oracle11gstby admin]$ rman target sys/welcome@sonyprd auxiliary sys/*****@sonyprdstby
Recovery Manager: Release 11.2.0.1.0 - Production on Fri Dec 21 17:26:24 2012
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
connected to target database: SONYPRD (DBID=3131093559)
connected to auxiliary database: SONYPRD (not mounted)
Listener Status from primary:
[oracle@oracle11g ~]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 18:08:56
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 20-DEC-2012 17:42:17
Uptime 1 days 0 hr. 26 min. 41 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/localhost/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
Services Summary...
Service "sonyprd.localdomain" has 1 instance(s).
Instance "sonyprd", status READY, has 1 handler(s) for this service...
Service "sonyprdXDB.localdomain" has 1 instance(s).
Instance "sonyprd", status READY, has 1 handler(s) for this service...
The command completed successfully
Listener Status from Standby when database bring to Nomount state:
[oracle@oracle11gstby admin]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 18:11:54
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 21-DEC-2012 16:13:47
Uptime 0 days 1 hr. 58 min. 6 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/oracle11gstby/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle11gstby)(PORT=1521)))
Services Summary...
Service "sonyprdstby" has 1 instance(s).
Instance "sonyprdstby", status BLOCKED, has 1 handler(s) for this service...
The command completed successfully
Please provide any work arounds to proceed further in creating the standby database.
Thanks,
Ram.
Edited by: 895188 on Dec 21, 2012 5:50 PM
Edited by: 895188 on Dec 21, 2012 6:09 PM
Edited by: 895188 on Dec 21, 2012 6:22 PM

Hello;
Script has to be run from auxiliary.
Great details, but what I don't see is how you connect in RMAN.
Example
Start RMAN
$ORACLE_HOME/bin/rman target=sys/@recover2 auxiliary=sys/@reclone
Recovery Manager: Release 11.2.0.2.0 - Production on Wed Feb 22 14:50:31 2012
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
connected to target database: RECOVER2 (DBID=3806912436)
connected to auxiliary database: RECLONE (not mounted)Nothing in your tnsnames or listener jumps out as wrong. I copied them into my text editor and compared against my working ones.
Generally for active duplication I start the future standby on a pfile NOMOUNT.
Auxiliary Instance Initialization Parameters Needed :
DB_NAME
CONTROL_FILES
DB_BLOCK_SIZE
( add extra parameters like DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT instead of using the SET command - clean up standby spfile right after duplication )
Then start RMAN and finally run a single line duplication command.
$ORACLE_HOME/bin/rman target=sys/@primary auxiliary=sys/@standby
RMAN>duplicate target database for standby from active database NOFILENAMECHECK;Best Regards
mseberg

Issue in creating the standby database from Active database using RMAN

Hi All,
I am facing issue in creating the standby database from Active database using RMAN and getting the below issue after i executed the duplicate command.
Version of Database:11g(11.2.0.1.0)
Operating System:Linux 5
Error:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 12/21/2012 17:26:52
RMAN-03015: error occurred in stored script Memory Script
RMAN-04006: error from auxiliary database: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Primary Database Entries:
Tnsentry:
SONYPRD =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.131)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprd.localdomain)(UR=A)
SONYPRDSTBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.132)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprdstby)(UR=A)
Listner Entry:
SID_LIST_SONYPRD =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtproc)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(PROGRAM = extproc)
(SID_DESC =
(SID_NAME = SONYPRD)
(GLOBAL_DBNAME = SONYPRD)
Auxiliary Details:
Tns Entry:
SONYPRD =
(DESCRIPTION =
# (ADDRESS = (PROTOCOL = TCP)(HOST = oracle11g.localdomain)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.131)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprd.localdomain)
SONYPRDSTBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.132)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = sonyprdstby)(UR=A)
Listener Entry in auxiliary:
SID_LIST_SONYPRDSTBY =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = SONYPRDSTBY)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = SONYPRDSTBY)
TNSPING from Primary DB:
[oracle@oracle11g ~]$ tnsping sonyprdstby
TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 17:39:28
Copyright (c) 1997, 2009, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.132)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = sonyprdstby)(UR=A)))
OK (0 msec)
TNSPING from Auxuliary server
[oracle@oracle11gstby ~]$ tnsping sonyprd
TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 17:40:19
Copyright (c) 1997, 2009, Oracle. All rights reserved.
Used parameter files:
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.20.131)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = sonyprd.localdomain)))
OK (10 msec)
Script Used for duplicate:
run {
allocate channel prmy1 type disk;
allocate channel prmy2 type disk;
allocate channel prmy3 type disk;
allocate channel prmy4 type disk;
allocate auxiliary channel stby type disk;
duplicate target database for standby from active database
spfile
parameter_value_convert 'sonyprd','sonyprdstby'
set db_unique_name='sonyprdstby'
set db_file_name_convert='/sonyprd/','/sonyprdstby/'
set log_file_name_convert='/sonyprd/','/sonyprdstby/'
set control_files='/u01/app/oracle/oradata/control01.ctl'
set log_archive_max_processes='5'
set fal_client='sonyprdstby'
set fal_server='sonyprd'
set standby_file_management='AUTO'
set log_archive_config='dg_config=(sonyprd,sonyprdstby)'
set log_archive_dest_2='service=sonyprd ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=sonyprd'
Tried the script from both Primary and auxiliary but no luck
[oracle@oracle11gstby admin]$ rman target sys/welcome@sonyprd auxiliary sys/*****@sonyprdstby
Recovery Manager: Release 11.2.0.1.0 - Production on Fri Dec 21 17:26:24 2012
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
connected to target database: SONYPRD (DBID=3131093559)
connected to auxiliary database: SONYPRD (not mounted)
Listener Status from primary:
[oracle@oracle11g ~]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 18:08:56
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 20-DEC-2012 17:42:17
Uptime 1 days 0 hr. 26 min. 41 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/localhost/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
Services Summary...
Service "sonyprd.localdomain" has 1 instance(s).
Instance "sonyprd", status READY, has 1 handler(s) for this service...
Service "sonyprdXDB.localdomain" has 1 instance(s).
Instance "sonyprd", status READY, has 1 handler(s) for this service...
The command completed successfully
Listener Status from Standby when database bring to Nomount state:
[oracle@oracle11gstby admin]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 21-DEC-2012 18:11:54
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 21-DEC-2012 16:13:47
Uptime 0 days 1 hr. 58 min. 6 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/oracle11gstby/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle11gstby)(PORT=1521)))
Services Summary...
Service "sonyprdstby" has 1 instance(s).
Instance "sonyprdstby", status BLOCKED, has 1 handler(s) for this service...
The command completed successfully
Please provide any work arounds to proceed further in creating the standby database.
Thanks,
Ram.

Pl do not post duplicates - Issue in configuring Standby Database from Active database in 11g by RMAN

Logs are not getting applied in the standby database

Hello,
I have created a physical standby database and in it the logs are not getting applied..
following is an extract of the standby alert log
Wed Sep 05 07:53:59 2012
Media Recovery Log /u01/oracle/oradata/ABC/archives/1_37638_765704228.arc
Error opening /u01/oracle/oradata/ABC/archives/1_37638_765704228.arc
Attempting refetch
Media Recovery Waiting for thread 1 sequence 37638
Fetching gap sequence in thread 1, gap sequence 37638-37643
Wed Sep 05 07:53:59 2012
RFS[46]: Assigned to RFS process 3081
RFS[46]: Allowing overwrite of partial archivelog for thread 1 sequence 37638
RFS[46]: Opened log for thread 1 sequence *37638* dbid 1723205832 branch 765704228
Wed Sep 05 07:55:34 2012
RFS[42]: Possible network disconnect with primary database
However, the archived files are getting copied to the standby server.
I tried registering and recovering the logs but it also failed..
Follows some of the information,
Primary
Oralce 11R2 EE
SQL> select max(sequence#) from v$log where archived='YES';
MAX(SEQUENCE#)
37668
SQL> select DEST_NAME, RECOVERY_MODE,DESTINATION,ARCHIVED_SEQ#,APPLIED_SEQ#, SYNCHRONIZATION_STATUS,SYNCHRONIZED,GAP_STATUS from v$archive_dest_status where DEST_NAME = 'LOG_ARCHIVE_DEST_3';
DEST_NAME RECOVERY_MODE DESTINATION ARCHIVED_SEQ# APPLIED_SEQ# SYNCHRONIZATION_STATUS SYNCHRONIZED GAP_STATUS
LOG_ARCHIVE_DEST_3 MANAGED REAL TIME APPLY ABC 37356 0 CHECK CONFIGURATION NO RESOLVABLE GAP
Standby
Oralce 11R2 EE
SQL> select max(sequence#) from v$archived_log where applied='YES';
MAX(SEQUENCE#)
37637
SQL> select * from v$archive_gap;
no rows selected
SQL> select open_mode, database_role from v$database;
OPEN_MODE DATABASE_ROLE
READ ONLY WITH APPLY PHYSICAL STANDBY
Please help me to troubleshoot this and get the standby in sync..
Thanks a lot..

the results are as follows..
SQL> select process, status, group#, thread#, sequence# from v$managed_standby order by process, group#, thread#, sequence#;
PROCESS STATUS GROUP# THREAD# SEQUENCE#
ARCH CLOSING 1 1 37644
ARCH CLOSING 1 1 37659
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
ARCH CONNECTED N/A 0 0
MRP0 WAIT_FOR_GAP N/A 1 37638
RFS IDLE N/A 0 0
RFS IDLE N/A 0 0
RFS IDLE N/A 0 0
RFS RECEIVING N/A 1 37638
RFS RECEIVING N/A 1 37639
RFS RECEIVING N/A 1 37640
RFS RECEIVING N/A 1 37641
RFS RECEIVING N/A 1 37642
RFS RECEIVING N/A 1 37655
RFS RECEIVING N/A 1 37673
RFS RECEIVING N/A 1 37675
42 rows selected.
SQL>
SQL> select name,value, time_computed from v$dataguard_stats;
NAME VALUE TIME_COMPUTED
transport lag +00 02:44:33 09/05/2012 09:25:58
apply lag +00 03:14:30 09/05/2012 09:25:58
apply finish time +00 00:01:09.974 09/05/2012 09:25:58
estimated startup time 12 09/05/2012 09:25:58
SQL> select timestamp , facility, dest_id, message_num, error_code, message from v$dataguard_status order by timestamp
TIMESTAMP FACILITY DEST_ID MESSAGE_NUM ERROR_CODE MESSAGE
05-SEP-12 Remote File Server 0 60 0 RFS[13]: Assigned to RFS process 2792
05-SEP-12 Remote File Server 0 59 0 RFS[12]: Assigned to RFS process 2790
05-SEP-12 Log Apply Services 0 61 16037 MRP0: Background Media Recovery cancelled with status 16037
05-SEP-12 Log Apply Services 0 62 0 MRP0: Background Media Recovery process shutdown
05-SEP-12 Log Apply Services 0 63 0 Managed Standby Recovery Canceled
05-SEP-12 Log Apply Services 0 64 0 Managed Standby Recovery not using Real Time Apply
05-SEP-12 Log Apply Services 0 65 0 Attempt to start background Managed Standby Recovery process
05-SEP-12 Log Apply Services 0 66 0 MRP0: Background Managed Standby Recovery process started
05-SEP-12 Log Apply Services 0 67 0 Managed Standby Recovery not using Real Time Apply
05-SEP-12 Log Apply Services 0 68 0 Media Recovery Waiting for thread 1 sequence 37638 (in transit)
05-SEP-12 Network Services 0 69 0 RFS[5]: Possible network disconnect with primary database
05-SEP-12 Network Services 0 70 0 RFS[6]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 71 0 RFS[14]: Assigned to RFS process 2829
05-SEP-12 Remote File Server 0 72 0 RFS[15]: Assigned to RFS process 2831
05-SEP-12 Network Services 0 73 0 RFS[9]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 74 0 RFS[16]: Assigned to RFS process 2833
05-SEP-12 Network Services 0 75 0 RFS[1]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 76 0 RFS[17]: Assigned to RFS process 2837
05-SEP-12 Network Services 0 77 0 RFS[3]: Possible network disconnect with primary database
05-SEP-12 Network Services 0 78 0 RFS[2]: Possible network disconnect with primary database
05-SEP-12 Network Services 0 79 0 RFS[7]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 80 0 RFS[18]: Assigned to RFS process 2848
05-SEP-12 Network Services 0 81 0 RFS[16]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 82 0 RFS[19]: Assigned to RFS process 2886
05-SEP-12 Network Services 0 83 0 RFS[19]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 84 0 RFS[20]: Assigned to RFS process 2894
05-SEP-12 Log Apply Services 0 85 16037 MRP0: Background Media Recovery cancelled with status 16037
05-SEP-12 Log Apply Services 0 86 0 MRP0: Background Media Recovery process shutdown
05-SEP-12 Log Apply Services 0 87 0 Managed Standby Recovery Canceled
05-SEP-12 Remote File Server 0 89 0 RFS[22]: Assigned to RFS process 2900
05-SEP-12 Remote File Server 0 88 0 RFS[21]: Assigned to RFS process 2898
05-SEP-12 Remote File Server 0 90 0 RFS[23]: Assigned to RFS process 2902
05-SEP-12 Remote File Server 0 91 0 Primary database is in MAXIMUM PERFORMANCE mode
05-SEP-12 Remote File Server 0 92 0 RFS[24]: Assigned to RFS process 2904
05-SEP-12 Remote File Server 0 93 0 RFS[25]: Assigned to RFS process 2906
05-SEP-12 Log Apply Services 0 94 0 Attempt to start background Managed Standby Recovery process
05-SEP-12 Log Apply Services 0 95 0 MRP0: Background Managed Standby Recovery process started
05-SEP-12 Log Apply Services 0 96 0 Managed Standby Recovery starting Real Time Apply
05-SEP-12 Log Apply Services 0 97 0 Media Recovery Waiting for thread 1 sequence 37638 (in transit)
05-SEP-12 Log Transport Services 0 98 0 ARCa: Beginning to archive thread 1 sequence 37644 (7911979302-7912040568)
05-SEP-12 Log Transport Services 0 99 0 ARCa: Completed archiving thread 1 sequence 37644 (0-0)
05-SEP-12 Network Services 0 100 0 RFS[8]: Possible network disconnect with primary database
05-SEP-12 Log Apply Services 0 101 16037 MRP0: Background Media Recovery cancelled with status 16037
05-SEP-12 Log Apply Services 0 102 0 Managed Standby Recovery not using Real Time Apply
05-SEP-12 Log Apply Services 0 103 0 MRP0: Background Media Recovery process shutdown
05-SEP-12 Log Apply Services 0 104 0 Managed Standby Recovery Canceled
05-SEP-12 Network Services 0 105 0 RFS[20]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 106 0 RFS[26]: Assigned to RFS process 2930
05-SEP-12 Network Services 0 107 0 RFS[24]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 108 0 RFS[27]: Assigned to RFS process 2938
05-SEP-12 Network Services 0 109 0 RFS[14]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 110 0 RFS[28]: Assigned to RFS process 2942
05-SEP-12 Network Services 0 111 0 RFS[15]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 112 0 RFS[29]: Assigned to RFS process 2986
05-SEP-12 Network Services 0 113 0 RFS[17]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 114 0 RFS[30]: Assigned to RFS process 2988
05-SEP-12 Log Apply Services 0 115 0 Attempt to start background Managed Standby Recovery process
05-SEP-12 Log Apply Services 0 116 0 MRP0: Background Managed Standby Recovery process started
05-SEP-12 Log Apply Services 0 117 0 Managed Standby Recovery starting Real Time Apply
05-SEP-12 Log Apply Services 0 118 0 Media Recovery Waiting for thread 1 sequence 37638 (in transit)
05-SEP-12 Network Services 0 119 0 RFS[18]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 120 0 RFS[31]: Assigned to RFS process 3003
05-SEP-12 Network Services 0 121 0 RFS[26]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 122 0 RFS[32]: Assigned to RFS process 3005
05-SEP-12 Network Services 0 123 0 RFS[27]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 124 0 RFS[33]: Assigned to RFS process 3009
05-SEP-12 Remote File Server 0 125 0 RFS[34]: Assigned to RFS process 3012
05-SEP-12 Log Apply Services 0 127 0 Managed Standby Recovery not using Real Time Apply
05-SEP-12 Log Apply Services 0 126 16037 MRP0: Background Media Recovery cancelled with status 16037
05-SEP-12 Log Apply Services 0 128 0 MRP0: Background Media Recovery process shutdown
05-SEP-12 Log Apply Services 0 129 0 Managed Standby Recovery Canceled
05-SEP-12 Network Services 0 130 0 RFS[32]: Possible network disconnect with primary database
05-SEP-12 Log Apply Services 0 131 0 Managed Standby Recovery not using Real Time Apply
05-SEP-12 Log Apply Services 0 132 0 Attempt to start background Managed Standby Recovery process
05-SEP-12 Log Apply Services 0 133 0 MRP0: Background Managed Standby Recovery process started
05-SEP-12 Log Apply Services 0 134 0 Managed Standby Recovery not using Real Time Apply
05-SEP-12 Log Apply Services 0 135 0 Media Recovery Waiting for thread 1 sequence 37638 (in transit)
05-SEP-12 Network Services 0 136 0 RFS[33]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 137 0 RFS[35]: Assigned to RFS process 3033
05-SEP-12 Log Apply Services 0 138 16037 MRP0: Background Media Recovery cancelled with status 16037
05-SEP-12 Log Apply Services 0 139 0 MRP0: Background Media Recovery process shutdown
05-SEP-12 Log Apply Services 0 140 0 Managed Standby Recovery Canceled
05-SEP-12 Remote File Server 0 141 0 RFS[36]: Assigned to RFS process 3047
05-SEP-12 Log Apply Services 0 142 0 Attempt to start background Managed Standby Recovery process
05-SEP-12 Log Apply Services 0 143 0 MRP0: Background Managed Standby Recovery process started
05-SEP-12 Log Apply Services 0 144 0 Managed Standby Recovery starting Real Time Apply
05-SEP-12 Log Apply Services 0 145 0 Media Recovery Waiting for thread 1 sequence 37638 (in transit)
05-SEP-12 Network Services 0 146 0 RFS[35]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 147 0 RFS[37]: Assigned to RFS process 3061
05-SEP-12 Network Services 0 148 0 RFS[36]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 149 0 RFS[38]: Assigned to RFS process 3063
05-SEP-12 Remote File Server 0 150 0 RFS[39]: Assigned to RFS process 3065
05-SEP-12 Network Services 0 151 0 RFS[25]: Possible network disconnect with primary database
05-SEP-12 Network Services 0 152 0 RFS[21]: Possible network disconnect with primary database
05-SEP-12 Remote File Server 0 153 0 Archivelog record exists, but no file is found
05-SEP-12 Remote File Server 0 154 0 RFS[40]: Assigned to RFS process 3067
05-SEP-12 Network Services 0 155 0 RFS[37]: Possible network disconnect with primary database

Error while trying to open physical standby database - (DATA GUARD)

Hi Everyone,
I have problems in opening the database of the physical standby in read- write mode/ read only mode. I have a primary server which is running on 2 node RAC and the standby on a seperate single server being used as DR. I recently got this server and my aim was to isolate the standby server from primary server and perform few test. As it has never been tested even once.
Primary Database spec: (2 Node Rac on ASM)
Oracle Version : 10.2.0.3.0
O/s : HP-UX B.11.23
Standby Database spec: (Single Node)
Oracle Version : 10.2.0.3.0
O/s: HP-UX db01 B.11.23
Error:
alter database recover managed standby database cancel;
Database altered.
SQL> alter database open
2 ;
alter database open
ERROR at line 1:
ORA-16004: backup database requires recovery
ORA-01152: file 1 was not restored from a sufficiently old backup
ORA-01110: data file 1: '+DATA/dprod/datafile/system01.dbf'
Parameters :
log_archive_dest_2 string SERVICE=PROD1 LGWR ASYNC VALID
FOR=(ONLINELOGFILES,PRIMARY_
ROLE) DB_UNIQUE_NAME=PROD
remote_archive_enable string true
fal_client string DPROD
fal_server string PROD1, PROD2
Steps tried so far:
Changed log_archive_dest_2 = DEFER on both the primary nodes
Standby :
startup nomount
alter database mount standby database;
alter database recover managed standby database disconnect;
alter database recover managed standby database cancel;
alter database open/readonly (tried both)
Same error.
On Primary:
SQL> select max(sequence#) from v$log_history;
MAX(SEQUENCE#)
55702
on Standby:
MAX(SEQUENCE#)
33289
Primary Database:
SELECT SEQUENCE#, FIRST_TIME, NEXT_TIME FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#;
SEQUENCE# FIRST_TIME NEXT_TIME
55700 13-JUN-11 13-JUN-11
55700 13-JUN-11 13-JUN-11
55701 13-JUN-11 13-JUN-11
55701 13-JUN-11 13-JUN-11
55702 13-JUN-11 13-JUN-11
60824 rows selected.
Standby Database:
SEQUENCE# FIRST_TIME NEXT_TIME
55698 13-JUN-11 13-JUN-11
55699 13-JUN-11 13-JUN-11
55700 13-JUN-11 13-JUN-11
55701 13-JUN-11 13-JUN-11
15206 rows selected.
Additional Information :
There is a delay of 20 minutes before the logs get applied. which has been intentional set by team.
Any help will be highly appreciated. Thanks in advance
Sadiq

Hi,
Primary Database:
select status,checkpoint_count from v$datafile_header;
STATUS CHECKPOINT_COUNT
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
STATUS CHECKPOINT_COUNT
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 672472065
ONLINE 59736
ONLINE 59736
ONLINE 59736
ONLINE 59736
ONLINE 59736
STATUS CHECKPOINT_COUNT
ONLINE 57717
ONLINE 57717
57 rows selected.
Standby Database;
select status,checkpoint_count from v$datafile_header;
STATUS CHECKPOINT_COUNT
ONLINE 672445072
ONLINE 672445072
ONLINE 672445072
ONLINE 672445072
ONLINE 672445072
ONLINE 672445072
ONLINE 672445072
ONLINE 672445071
ONLINE 672445071
ONLINE 672445071
ONLINE 672445071
STATUS CHECKPOINT_COUNT
ONLINE 672445071
ONLINE 672445071
ONLINE 672445071
ONLINE 672445071
ONLINE 672445071
ONLINE 672445071
ONLINE 32742
ONLINE 32742
ONLINE 32742
ONLINE 32742
ONLINE 32742
STATUS CHECKPOINT_COUNT
ONLINE 30723
ONLINE 30723
57 rows selected.
Archieve log list :
Primary database:
SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence 49110
Next log sequence to archive 49111
Current log sequence 49111
Standby Database:
SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence 49110
Next log sequence to archive 0
Current log sequence 49111
I tried log switch multiple times in primary database i saw that its getting highlighted in standby database.

How to delete archivelog with RMAN in the Primary and Standby database?

Hello,
I am working on Oracle 10gR2.
My question is :
How could I implement a automatized and secure way of:
1) Delete archivelogs on my PRIMARY server only when these logs are successfully TRANSFERED to all my standby databases?
2) Delete archivelogs on my STANDBY server only when these logs are sucessfully APPLIED to the database?
Thanks a lot for your help!

on 10G
CONFIGURE ARCHIVELOG DELETION POLICY TO [CLEAR | NONE | APPLIED ON STANDBY];
http://download.oracle.com/docs/cd/B19306_01/server.102/b14239/rman.htm#CHDBEICE
Coskan Gundogar
http://coskan.wordpress.com

DR Server: Standby Database

Similar Messages

Maybe you are looking for