Restoring Back To 10g After A Failed 11g Upgrade Of a RAC Database

Question

I'm testing this out on a small test RAC database. I successfully upgraded it from 10.2.0.4 to 11.2.0.2 but wanted to test the scenario of having to go back to 10g if the upgrade really hosed up. The first recovery attempt seemed to be successful but after bringing the DB down with srvctl, it failed on the next startup saying it needed to be started in upgrade mode. Something from 11g was still in place or the fact that I was trying to restart a 10g database managed by 11g clusterware was the issue. I tried starting the DB from both 10g and 11g environments and got the same result. Even starting each instance individually got the same result.
In all that I tried, I got the usual incarnation and "until time before reset time" messages. I've been doing this all through RMAN without EM or Grid Ctl. As usual, any docs I found have had just a little information and I have to piece my own instructions together from all of them, not knowing for sure it all steps would apply in my situation.
Can anybody point me to a good doc or other resource that might help me out? Many thanks!

Answer 1

Now I have a different issue with apparently the same problem. I successfully did the restore/recover as before but the thing pukes when I open resetlogs at the end. Something, somewhere is still pointing to 11g but I have no idea where or what has changed since the last time I did this. Maybe I've messed things up by doing this multiple times. Here are my RMAN commands:
RMAN> connect target
RMAN> startup force nomount;
RMAN> RESTORE SPFILE TO '+DATA/jimg/spfilejimg.ora' from '/local/oracle/10.2.0/db_1/dbs/c-2526333028-20110915-01';
RMAN> shutdown immediate;
(in a different session, from command line)
% mv /local/oracle/10.2.0/db_1/dbs/initJIMG1.ora.bak /local/oracle/10.2.0/db_1/dbs/initJIMG1.ora
RMAN> startup force nomount pfile='/local/oracle/10.2.0/db_1/dbs/initJIMG1.ora';
RMAN> restore controlfile from '/local/oracle/10.2.0/db_1/dbs/c-2526333028-20110915-01';
RMAN> alter database mount;
RMAN>
run {
restore database;
recover database;
RMAN> alter database open resetlogs;
The errors I get after resetlogs are:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of alter db command at 09/16/2011 09:24:50
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
ORA-03114: not connected to ORACLE
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of alter db command at 09/16/2011 09:24:50
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option
This is being done on a small, expendable database for practice. I think I've learned enough about what NOT to do to keep me from getting in this jam to begin with. Any thoughts?