Archive not applying

Question

Setup: - Oracle 10.2.0.4 EE on HP-UX 11.23
Two weeks ago we got our DG physical standby setup working, initial tests looked good, I could tail both primary and standby alert logs and see results of log switches, everything looked good for about a week. Then ..
One morning I came in and logs weren't shipping. Checked the standby site and it had lost its mounts to the NAS. So far out setup was more proof of concept and I was getting ready to take a week off. So on the primary I set log arch dest 2 status to DEFER. In my absence our SA fixed the problem with mounting the fs and restarted the standby db (he's not a dba and just did a normal startup). I returned to the office this morning and find dest2 status back to ENABLE. But it appears that archivelogs are not applying.
Prior to the crash, as redo was received at the standy, its alert would show
Wed May 25 18:38:11 2011
Primary database is in MAXIMUM PERFORMANCE mode
RFS[39]: Successfully opened standby log 5: '/oradata/ora_redo/BOSTON/sbyredo05a.rdo'
Wed May 25 18:38:17 2011
Media Recovery Waiting for thread 1 sequence 3123 (in transit)
Wed May 25 18:38:17 2011
Recovery of Online Redo Log: Thread 1 Group 5 Seq 3123 Reading mem 0
  Mem# 0: /oradata/ora_redo/BOSTON/sbyredo05a.rdo
  Mem# 1: /archive/ora_redo/BOSTON/sbyredo05b.rdoAfter the standby was restarted, I don't see the above sequence, but instead get
Fri Jun  3 07:27:09 2011
Primary database is in MAXIMUM PERFORMANCE mode
RFS[28]: Successfully opened standby log 4: '/oradata/ora_redo/BOSTON/sbyredo04a.rdo'
Fri Jun  3 07:28:49 2011
Primary database is in MAXIMUM PERFORMANCE mode
RFS[28]: Successfully opened standby log 5: '/oradata/ora_redo/BOSTON/sbyredo05a.rdo'
Fri Jun  3 07:31:15 2011
Primary database is in MAXIMUM PERFORMANCE mode
RFS[28]: Successfully opened standby log 4: '/oradata/ora_redo/BOSTON/sbyredo04a.rdo'
Fri Jun  3 08:39:47 2011
Primary database is in MAXIMUM PERFORMANCE mode
RFS[28]: Successfully opened standby log 5: '/oradata/ora_redo/BOSTON/sbyredo05a.rdo'It appears that I have exceeded control_file_record_keep_time. This showed up in the standby alert
Mon Jun  6 08:25:47 2011
FAL[client]: Failed to request gap sequence
GAP - thread 1 sequence 3126-3224
DBID 2542058214 branch 737468006
FAL[client]: All defined FAL servers have been attempted.
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-------------------------------------------------------------On primary:
SQL> show parameter CONTROL_FILE_RECORD_KEEP_TIME
NAME                                 TYPE        VALUE
control_file_record_keep_time        integer     7
SQL>I could take the "easy" way out and rebuild the standby from a fresh backup of primary, but before doing that would like to learn what I can from this, as I'm still pretty new to managaing a DB setup.

Answer 1

Mon Jun 6 08:25:47 2011
FAL[client]: Failed to request gap sequence
GAP - thread 1 sequence 3126-3224
DBID 2542058214 branch 737468006
FAL[client]: All defined FAL servers have been attempted.
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-------------------------------------------------------------Actually these information from alert logfile regarding CONTROL_FILE_RECORD_KEEP_TIME it shows any Archive gap found on standby even if it set to *30* days. It is very generic information, Nothing to worry,
Fri Jun 3 07:27:09 2011
Primary database is in MAXIMUM PERFORMANCE mode<RFS[28]: Successfully opened standby log 4: '/oradata/ora_redo/BOSTON/sbyredo04a.rdo'
In above information, Whenever a archive ship from primary to standby it will assigned to logfile on standby. This is also very normal behaviour.
I could take the "easy" way out and rebuild the standby from a fresh backup of primary, but before doing that would like to learn what I can from this, as I'm still pretty new to managaing a DB setup.
According to your archive log gap GAP - thread 1 sequence 3126-3224 it is not an big, Why you want to rebuild.?
Check what is the errors on primary alert log file.
Post from PRIMARY:-
show parameter dest_state_2select ds.dest_id id
, ad.status
, ds.database_mode db_mode
, ad.archiver type
, ds.recovery_mode
, ds.protection_mode
, ds.standby_logfile_count "SRLs"
, ds.standby_logfile_active active
, ds.archived_seq#
from v$archive_dest_status ds
, v$archive_dest ad
where ds.dest_id = ad.dest_id
and ad.status != 'INACTIVE'
order by
ds.dest_id
select error_code,timestamp, message from v$dataguard_status where dest_id=2;Thanks.