Oracle physru rolling upgrade problem

Hi, I'm having a problem with the Oracle Physru script provided from MOS note # 949322 I was hoping I could get some help with.
My system contains of two Oracle 11.1.0.7.0, one primary and one physical standby.
The hosts are two Oracle Solaris 10 Sparc x64 machines.
My goal is to update from 11.1.0.7.0 to 11.1.0.7.8 using the script that is provided, however I'm bumping into problems at the step where the script is checking the Apply lag on the logical standby (function "get_apply_lag"). The lag seems to increase, and to be that indicates a problem with the redo apply process. However, when I query the DBA_LOGSTDBY_EVENTS View i get the following:
SQL> SELECT EVENT_TIME, STATUS, EVENT FROM DBA_LOGSTDBY_EVENTS ORDER BY EVENT_TIMESTAMP, COMMIT_SCN;
EVENT_TIME STATUS EVENT
20-SEP-11 16:02:12 ORA-16111: log mining and apply setting up
20-SEP-11 16:02:12 Apply LWM 848622, HWM 848622, SCN 848622
Showing the output from Primary archive dest 2:
SQL> show parameter log_archive_dest_2;
NAME TYPE VALUE
log_archive_dest_2 string service="test08db2", LGWR ASYNC NOAFFIRM delay=0 OPTIONAL compression=DISABLE max_failure=0 max_connections=1 reopen=300 db_unique_name="db_test08db2" net_timeout=30 valid_for=(online_logfile,primary_role)
SQL>  select from v$LOGSTDBY_STATS;*
NAME VALUE
logminer session id 1
number of preparers 1
number of appliers 11
server processes in use 15
maximum SGA for LCR cache (MB) 50
maximum events recorded 2000000000
preserve commit order TRUE
transaction consistency FULL
record skipped errors Y
record skipped DDLs Y
record applied DDLs N
NAME VALUE
record unsupported operations Y
realtime apply Y
apply delay (minutes) 0
coordinator state IDLE
coordinator startup time 20-SEP-11 16:02:11
coordinator uptime (seconds) 727
txns received from logminer 62
txns assigned to apply 29
txns applied 29
txns discarded during restart 33
large txns waiting to be assigned 0
NAME VALUE
rolled back txns mined 4
DDL txns mined 2
CTAS txns mined 0
bytes of redo mined 8195884
bytes paged out 0
pageout time (seconds) 0
bytes checkpointed 709802
checkpoint time (seconds) 0
system idle time (seconds) 479
standby redo logs mined 0
archived logs mined 4
NAME VALUE
gap fetched logs mined 2
standby redo log reuse detected 0
logfile open failures 0
current logfile wait (seconds) 0
total logfile wait (seconds) 0
thread enable mined 0
thread disable mined 0
distinct txns in queue 0
41 rows selected.
SQL> select type, high_scn, status, pid from v$logstdby order by type;
TYPE HIGH_SCN STATUS PID
ANALYZER 850320 ORA-16116: no work available 20702
APPLIER 848808 ORA-16116: no work available 20719
APPLIER 850320 ORA-16116: no work available 20731
APPLIER 850204 ORA-16116: no work available 20727
APPLIER 848895 ORA-16116: no work available 20725
APPLIER 848665 ORA-16116: no work available 20705
APPLIER 848677 ORA-16116: no work available 20709
APPLIER 848728 ORA-16116: no work available 20713
APPLIER 848740 ORA-16116: no work available 20715
APPLIER 848796 ORA-16116: no work available 20717
APPLIER 848842 ORA-16116: no work available 20721
TYPE HIGH_SCN STATUS PID
APPLIER 848854 ORA-16116: no work available 20723
BUILDER 850320 ORA-16116: no work available 20221
COORDINATOR 850326 ORA-16116: no work available 20119
PREPARER 850318 ORA-16116: no work available 20223
READER 850326 ORA-16116: no work available 20217
16 rows selected.
Physru Script Output:_
### Initialize script to either start over or resume execution
Sep 20 14:35:41 2011 [0-1] Identifying rdbms software version
Sep 20 14:35:41 2011 [0-1] database nobilldb is at version 11.1.0.7.0
Sep 20 14:35:42 2011 [0-1] database nobilldb is at version 11.1.0.7.0
Sep 20 14:35:44 2011 [0-1] verifying flashback database is enabled at db_test08db1 and db_test08db2
Sep 20 14:35:44 2011 [0-1] verifying available flashback restore points
Sep 20 14:35:45 2011 [0-1] verifying DG Broker is disabled
Sep 20 14:35:46 2011 [0-1] looking up prior execution history
Sep 20 14:35:46 2011 [0-1] purging script execution state from database db_test08db1
Sep 20 14:35:46 2011 [0-1] purging script execution state from database db_test08db2
Sep 20 14:35:47 2011 [0-1] starting new execution of script
### Stage 1: Backup user environment in case rolling upgrade is aborted
Sep 20 14:35:47 2011 [1-1] stopping media recovery on db_test08db2
Sep 20 14:35:48 2011 [1-1] creating restore point PRUP_0000_0001 on database db_test08db2
Sep 20 14:35:49 2011 [1-1] backing up current control file on db_test08db2
Sep 20 14:35:50 2011 [1-1] created backup control file /opt/oracle/product/11.1.0.7/dbs/PRUP_0001_db_test08db2_f.f
Sep 20 14:35:50 2011 [1-1] creating restore point PRUP_0000_0001 on database db_test08db1
Sep 20 14:35:51 2011 [1-1] backing up current control file on db_test08db1
Sep 20 14:35:52 2011 [1-1] created backup control file /opt/oracle/product/11.1.0.7/dbs/PRUP_0001_db_test08db1_f.f
NOTE: Restore point PRUP_0000_0001 and backup control file PRUP_0001_db_test08db2_f.f
can be used to restore db_test08db2 back to its original state as a
physical standby, in case the rolling upgrade operation needs to be aborted
prior to the first switchover done in Stage 4.
### Stage 2: Create transient logical standby from existing physical standby
Sep 20 14:35:53 2011 [2-1] verifying RAC is disabled at db_test08db2
Sep 20 14:35:53 2011 [2-1] verifying database roles
Sep 20 14:35:54 2011 [2-1] verifying physical standby is mounted
Sep 20 14:35:54 2011 [2-1] verifying database protection mode
Sep 20 14:35:55 2011 [2-1] verifying transient logical standby datatype support
Sep 20 14:36:00 2011 [2-2] starting media recovery on db_test08db2
Sep 20 14:36:11 2011 [2-2] confirming media recovery is running
Sep 20 14:36:12 2011 [2-2] waiting for v$dataguard_stats view to initialize
Sep 20 14:36:13 2011 [2-2] waiting for apply lag on db_test08db2 to fall below 30 seconds
Sep 20 14:36:13 2011 [2-2] apply lag is now less than 30 seconds
Sep 20 14:36:14 2011 [2-2] stopping media recovery on db_test08db2
Sep 20 14:36:15 2011 [2-2] executing dbms_logstdby.build on database db_test08db1
Sep 20 14:36:27 2011 [2-2] converting physical standby into transient logical standby
Sep 20 14:36:52 2011 [2-3] opening database db_test08db2
Sep 20 14:37:28 2011 [2-4] configuring transient logical standby parameters for rolling upgrade
Sep 20 14:37:29 2011 [2-4] starting logical standby on database db_test08db2
Sep 20 14:37:37 2011 [2-4] waiting until logminer dictionary has fully loaded
Sep 20 14:37:58 2011 [2-4] dictionary load 17% complete
Sep 20 14:38:09 2011 [2-4] dictionary load 32% complete
Sep 20 14:38:19 2011 [2-4] dictionary load 43% complete
Sep 20 14:38:30 2011 [2-4] dictionary load 59% complete
Sep 20 14:38:40 2011 [2-4] dictionary load 62% complete
Sep 20 14:38:50 2011 [2-4] dictionary load 70% complete
Sep 20 14:39:01 2011 [2-4] dictionary load 72% complete
Sep 20 14:39:11 2011 [2-4] dictionary load 75% complete
Sep 20 14:40:54 2011 [2-4] dictionary load is complete
Sep 20 14:41:00 2011 [2-4] waiting for v$dataguard_stats view to initialize
Sep 20 14:41:01 2011 [2-4] waiting for apply lag on db_test08db2 to fall below 30 seconds
Sep 20 14:42:02 2011 [2-4] current apply lag: 316
Sep 20 14:42:32 2011 [2-4] current apply lag: 316
Sep 20 14:43:03 2011 [2-4] current apply lag: 376
Sep 20 14:43:33 2011 [2-4] current apply lag: 376
Sep 20 14:44:03 2011 [2-4] current apply lag: 437
Sep 20 14:44:34 2011 [2-4] current apply lag: 437
Sep 20 14:45:04 2011 [2-4] current apply lag: 497
Sep 20 14:45:35 2011 [2-4] current apply lag: 497
Sep 20 14:46:05 2011 [2-4] current apply lag: 558
Sep 20 14:46:36 2011 [2-4] current apply lag: 558
Sep 20 14:47:06 2011 [2-4] current apply lag: 618
I would appreciate any help I could get, I'm stuck =/
Regards,

http://www.oracle.com/technetwork/database/features/availability/maa-wp-11g-upgrades-made-easy-131972.pdf

Similar Messages

  • Rolling Upgrade of Oracle 10gR2 RAC with Physical Standby (DG)

    Hi DBAs,
    I want to do a rolling upgrade of Oracle 10.2.0.3 (2 node RAC+ASM) with Physical Standy Dataguard to Oracle 10.2.0.4 also applying CPU (April2009).
    Please suggest the best way to perform this upgrade and patching with no downtime.
    Thanks
    -Samar-

    Unfortunately rolling upgrade is only possible with a logical standby in place:
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14239/upgrades.htm#CHDBJAGG
    That doesn't help you at this time, but 11g has a way to do it with a physical standby. Even here you need an intermediate logical standby:
    http://download.oracle.com/docs/cd/B28359_01/server.111/b28294/rollup.htm#CHDHCBGA
    Werner

  • Upgrade problem : Oracle Portal 10.1.2 to 10.1.4

    While I'm upgrading Oracle Portal 10.1.2.0.2 to 10.1.4 I got the following errors:
    ERROR: WWU-01013: Upgrade terminated with the following errors:
    ### 2240 : ERROR at line 1:
    ### 2241 : ORA-20000:
    ### 2242 : ORA-06512: at "PORTAL.WWPOF", line 440
    ### 2243 : ORA-06512: at line 60
    ### 2244 : ORA-02273: this unique/primary key is referenced by some foreign keys
    ### 2252 : # ERROR: Altering primary key things_pk to change the column ordering
    ### 2253 : # ERROR: ORA-02273: this unique/primary key is referenced by some foreign keys
    ### 2261 : ### ERROR: Exception Executing upg/10140/presch/upthing1.sql PORTAL
    Do you have any solution for this? Thanks in advance

    A similar issue was reported to Oracle when the upgrade was done on a metadata repository residing in a 9.2.0.7 database. The issue was not resolved however.
    You may want to log a service request to address the issue to Global Customer Support.

  • Scshutdown fails - "Rolling Upgrade in progress"

    Anybody seen this before?
    bash$ sudo scshutdown -g 0 -y
    scshutdown: Cannot shutdown the cluster. Rolling Upgrade in progress.
    bash$
    No one has (with intention) started an upgrade on this cluster.. Its SC3.1 U2 on Solaris 8.

    While you are saying that no one hast started an upgrade, is it possible that this cluster got recently patched with current Sun Cluster patches?
    If so, you might want to check if the follwoing binary exists on your cluster:
    /usr/cluster/bin/scversions
    If it exists, just call it.
    If you see something like:
    # scversions
    Upgrade commit is NOT needed. All versions match.
    then I don't know what your problem is. However, if it says something different, you might want to try:
    # scversions -c
    and see if the problem went away.
    Greets
    Thorsten

  • Rolling upgrade in MAA database with 2 node RAC on each site

    All,
    I have 2 node rac on SITE A, and it has Physical Standby database on SITE B it also 2 node RAC. Currently it is on 10.2.0.4 on AIX systems. We have mission crictical application runing and we can't aford the downtime of 5 to 6 hr to upgrade our CRS,ASM,DB oracle homes to 11G or even 10.2.0.5. I read rolling upgrade document on metalink and also documnet as well, and come up with following sequence.
    SITE A==Primary Site, SITE B == Standby database
    1) Shutdown DB on SITE B
    2) Take the OS Level Backup of Binaries only on SITE B (Both node), I think this is just a Precautionary Measure
    3) Start the DB on SITE B Still Physical Standby database let it synch with primary Site SITE A
    4) Convert SITE B to Logical standby database
    5) let it run for some time and be the two site SYNCH
    6) now stop archivelog shipping to STANDBY SITE SITE B
    7) Shutdown SITE B and apply the patch/upgrade binaries (CRS/ASM/DB)
    8) Start and upgrade the DB on SITE B
    9) Start the logical standby apply (SQL APPLY) and let it catch up with SITE A
    10) Now Switchover to logical standby database on SITE B and ask application to connect to SITE B
    11) Make sure the archivelogs are moving from SITE B to SITE A and SQL Apply is runing.
    12) Now Stop archive log flow from SITE B to SITE A
    13) Stop the database/ASM/CRS on SITE A
    14) upgrade the SITE A binaries (CRS/ASM/DB)
    15) Start and upgrade the DB on SITE A
    16) Start the logical standby apply (SQL APPLY) and let it catch up with SITE B
    17) Now Switchover Back to SITE A and let the application connect to SITE A
    At this point by SITE B is a Logical Standby database
    I think these summaries the steps involved in rolling upgrade, but I can't understand how I can go back to my MAA with physical standby database mode?
    Does anyone has done this? Can anyone give me some tips or comments on my steps??
    Help me if you can... thanks in advance.

    Hi,
    Is it you want to know the OS/Kernel side setting? You must comply with the following kernel requirements to install the clusterware.
    http://docs.oracle.com/cd/B28359_01/install.111/b28259/pre_hpux.htm#BABJHCJI
    Thanks.

  • Database rolling upgrade cpu patch with minimal service outage

    Oracle 11gR2 Primary Instance
    Oracle 11gR2 Active Physical Standby
    Requirement of applying CPU and CVE patches with minimal downtime. I believe that I've been able to apply CPU patch to Active Standby after deferring redo transport and applying patch to active physical standby first and then performing a manual switchover to have the standby become the primary and former-primary become the standby and they apply CPU Patch there and then perform another role transition to its original role.
    Appreciate your experience with primary and active physical standby rolling upgrades/patches. Thanks

    Hello;
    I believe you are over thinking this. All you have to do on the Standby is install the patch. No switchover, no SQL. No nothing.
    The SQL will move from the Primary to the Standby.
    Check the readme for the patch closely or post the patch number if you want to confirm.
    OR
    How do you apply a Patchset,PSU or CPU in a Data Guard Physical Standby configuration [ID 278641.1]
    ( If it helps I had the same thoughts after my first Data Guard setup !! )
    If you are trying to avoid downtime entirely the switchover will take about as long as the patch.
    If you have RAC this is interesting :
    http://javeedkaleem.blogspot.com/2010/05/how-to-apply-oracle-database-patch.html
    As is this :
    http://docs.oracle.com/cd/B28359_01/server.111/b28282/schedule_outage.htm
    Best Regards
    mseberg

  • Rolling upgrade

    Hi all,
    How to know if my Oracle home is shared to install the patcher 10.2.0.3 to 10.2.0.4 in rolling upgrade for Oracle Clusterware?
    Which refers to the shared homes, only CRS Home or Oracle Home and CRS Home?
    I have 3 nodes on RAC.
    SO: Solaris Operating System (SPARC 64-Bit)
    RDBMS: Oracle 10g R2
    Amin

    A shared Oracle home is one where a single Oracle Home is accessed via shared storage from multiple nodes. Contrast this with the case needed for rolling upgrade, where there would be a separate Oracle Home installed on each node in the RAC.
    The CRS_HOME and ORACLE_HOME should be different in any event (they can be under the same Oracle Base).
    Does that help?

  • Java.io.StreamCorruptedException after rolling upgrade

    Hi,
    After removing a first class citizen Class file A in Version2 and performing a rolling upgrade from Version1 to Version2 of our software we find an entry for Class A in replicated cache of Version2. Once we try to read / remove / or overwrite it with null we get a "java.io.StreamCorruptedException".
    Is there a way to get rid of these kind of entries in the cache after rolling upgrade is performed or even prevent Class A to be added into the cache during de-serialization?
    Thanks 

    I'm not quite sure what you mean yet. Are you talking about the server code or the client code? And, can you be a little more specific on where the problem is? I'm sorry, it's just that I've spent quite a while on the code and I haven't found the problem yet. Thanks again.

  • SQL Apply Rolling Upgrade

    Hi,
    I would like to upgrade our Oracle database from release 9i to oracle 10g.
    Anybody have used "SQL Apply Rolling Upgrade" with a SAP application to upgrade the database ?
    Thank's in advanced
    Francesc

    Hi Cristian,
    well you can do / try that, but it is not supported in a SAP environment.
    SQL Apply means Logical Standby database and this is explicitly not supported in a SAP environment - for more details please check SAPnote #105047 (sub point 14 - Oracle Data Guard).
    However from a technical point of view it is possible as none of these restrictions usually apply a SAP environment ( Data Type and DDL Support on a Logical Standby Database ).
    Regards
    Stefan

  • Rolling Upgrades in Rac

    Hi All,
    can you please explain abt rolling upgrades in rac env
    suppose i am having rac setup on 10.2.0.1 if i want to apply patch set 10.2.0.4 is it required downtime
    As per the readme of 10.2.0.4
    1)Stop every thing on node 1 and apply patch set
    2)stop every thing on node 2 and apply patch set
    3)stop the database and run the catupgrd.sql script.
    My question is for running catupgrd script i have to bring down the db so what the meaning of rolling upgrade??
    And in this type of sece which type of upgrade is best one rolling or non-rolling.
    KK
    Edited by: user12061936 on May 20, 2010 2:35 PM

    Hi HTH
    Thank you for clearing my doubts regarding Rolling upgrades and i have gone though the link and i have noticed the following steps
    For Standalone system
    1) Install Oracle 10g to a separate Oracle Home on the same server
    2) Create a 10g database with only the base tablespaces: SYSTEM, SYSAUX, UNDO, and TEMP
    3) On the 9i database, put all your tablespaces into read only mode (write downtime begins)
    4) Perform a transportable tablespace export of all non-system tablespaces (as a sysdba user)
    5) Shut down the 9i database (true downtime begins)
    6) Start up the 10g database
    7) Perform a transportable tablespace import into the 10g database (end true downtime)
    8) Make all your tablespaces read/write (end write downtime)
    The above steps is going to work only for 9i to 10g or can we use it to any version to version.
    BY using TTS is it suggestable method to do the upgrade??
    KK
    Edited by: user12061936 on May 21, 2010 11:41 AM

  • Rolling Upgrades

    Hi,
    what is Rolling Upgrades ?
    And what is the difference with an ordinary Upgrade ?
    Thanks.

    A rolling upgrade is an upgrade performed in a high availabilty environment where downtime is 0 or limited.
    See High Availability Overview: http://download.oracle.com/docs/cd/B19306_01/server.102/b14210/hafeatures.htm#CJAFCCFJ
    Edited by: P. Forstmann on 14 oct. 2009 11:06

  • Rolling upgrade with Data Guard

    I'm interesting if this table of possible upgrade from documentation (http://download.oracle.com/docs/cd/B28359_01/server.111/b28300/preup.htm#i1007814) is true, if I have a plan to do rolling upgrade with Data Guard?

    I thinking no, that table does not appear to have Rolling Upgrade Information.
    If I read the document correctly you can only this with a logical standby database in Data Guard.
    Larry Carpenter's book has some information on this in Chapter 11.
    There is also a separate Data Guard section here where you might find more information.
    Data Guard
    Best Regards
    mseberg

  • Is it possible to delay a rolling upgrade after a single domain is updated?

    Hello,
    I am working on new Azure services for my group. For various reasons, we have decided that a Staging / VIP-Swap deployment is not useful for our group and will be going with a rolling upgrade. What I am interested in, is knowing if there is a way to
    "pause" a rolling upgrade after the first update domain until it is released to complete. Or, for a set time ( 30 minutes, etc ), to allow us the opportunity to essentially have a A-B deployment of our webrole, to observe behavior while receiving
    production traffic before the release has completely deployed.
    Thanks for your help
    Jeff

    hello Jeff,
    Currently, base on my experience, delay a rolling upgrade or "pause" a rolling upgrade after the first update domain until it is released to complete, it may be difficult. I suggest you could can submit a feature suggestion on
    http://www.mygreatwindowsazureidea.com/forums/34192-windows-azure-feature-voting. I suggest you could get more detail about upgrade domain and fault domain (http://blog.toddysm.com/2010/04/upgrade-domains-and-fault-domains-in-windows-azure.html).
    Best Regards,
    Will
    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click
    HERE to participate the survey.

  • DSL and 9.1 to 9.2 upgrade problems?   Please Help!!

    I have a Beige g3 - 300 w/ 160 megs of memory - using Netscape 7.02 and occasionally Gatesmess IE-5. I have had dial up for 5 years and recently upgraded to DSL w/ Linksys WRT54G router connected via ethernet cable. Wifes HP laptop uses the wireless.
    Problem is when connected to the internet, I get a looong -(30 - 40 second) delay between selecting the url and the page loading with a "resolving host" message at the bottom of the page. The modem send light flashes every five seconds or so and once the system locks on the url it loads really fast. With the dialup there was no delay in finding the url but took a long time to load. Result is it takes about the same amount of time to browse now as before. I don't think the winbox is having the same problem.
    To try to resolve the "resolving host" issue I have been trying to upgrade from 9.1 to 9.2. When I download the package from apple.com at the end I get a 127 error which reestablishes itself everytime I click OK til I force quit the System. When I restart the computer, the file is on desktop OS 9.2 update.smi. First I opened the file w/ stuffit 5.2 and tried to install it following the instructions (sharing off, OS 9 extensions only etc) and about 4 minutes into the process I get the Bomb error 10. I have downloaded this three times and upgraded the Stuffit to 7.03 but am having the same problem. In order to get to 9.2.2 you have to upgrade to 9.2.1 first and that is what I am doing.
    Anybody have any ideas either on the "resolving host" issue or the upgrade problem.
    Thanks in advance,
    Beige g3 - 300mghz 160 meg memory   Mac OS 9.1.x  

    Thanks Grant.
    I read the article and it was of some help. Reference TCP/IP control panel settings. I called Commspeed support and they gave me their three current DNS numbers. I typed them per article and nothing - dead. Called them back and they gave me the typical winbox answers "No hable apple" so left to my own devices and considering the stratigies suggested in the article I did two things.
    I opened the TCP/IP control panel and deleted the Name Server Address - As they are supposed to be assigned by the server. There was also a check box that said use 802.3 which was checked. I unchecked it and tried the internet. Blam it was right there. I don't know which stratigy worked as I did them simultaneously.
    Still don't know about upgrading to 9.2.
    Thanks again,
    Bill
    Beige g3 - 300mghz 160 meg memory Mac OS 9.1.x

  • How to find out the date when an oracle database is upgraded

    Hi,
    I would like to know the date when an oracle database is upgraded (for example 10.2.0.4 to 11.2.0.3).
    Can you help me to find out these details.
    Thanks,
    Sarayu

    user13312943 wrote:
    Hi Ed and SB,
    Thanks for your replies. How about this option?
    Let us assume I am interested in checking when my database is upgraded to any 11g version.
    Will this query help?
    select min(first_usage_date)  from  dba_feature_usage_statistics where  version like '11%';
    I understand this may not be a generic solution , but with this idea I may be able to build up a query to get the result.
    Thanks,
    Sarayu
    How is it that you or nobody knows when the DB was upgraded?
    I would have expected a formal testing & upgrade plan.
    Does the organization practice safe software & utilize any source code repository?
    Does everyone suffer from amnesia?
    How could a MAJOR database upgrade slip by everyone who should care about such a detail?

Maybe you are looking for