Oracle 10.2.0.4 to 11.2.0.2 fails with ORA-03113

Question

This is Oracle upgrade from 10.2.0.2 to Oracle 11.2   (on Windows platform)
Since 10202 to 11 is not SAP supported, we upgrade from 10.2.0.4 first before upgrade to 11
10202 to 10204 upgrade finished fine.  All post upgrade sql files are run without any issues.
During 10204 to 11202 upgrade, at 42% complete the Database Upgrade Assistant gives
ORA-03113: end-of-file on communication channel
Upgrade failed due to running the step "Upgrading Oracle Server"
The upgrade was repeated and it always gets the same error at the same spot.
Once the error occurs, if I try to rerun the upgrade script, it complains that objects already exist.
Looking at the \oracle\cfgtoollogs\dbua\<SID>\upgrade1 directory, I see the log file which shows at the end of the file:
GRANT INSERT ON sys.impdp_stats TO PUBLIC
2  /
Grant succeeded.
GRANT DELETE ON sys.impdp_stats TO PUBLIC
2  /
Grant succeeded.
Rem ===================================
Rem End  User stats table format change
Rem ===================================
Rem==========================================================================
Rem Call script to upgrade type dictionary tables from 8.0 image to
Rem 8.1 image format. If they are already in 8.1 image format, the
Rem script will do nothing.
Rem==========================================================================
set serveroutput on
EXECUTE dbms_objects_utils.upgrade_dict_image;
BEGIN dbms_objects_utils.upgrade_dict_image; END;
ERROR at line 1:
ORA-03113: end-of-file on communication channel
ERROR:
So the problem is at running  dbms_objects_utils.upgrade_dict_image procedure
"This procedure upgrades the type dictionary images from 8.0 to 8.1. The dictionary tables that could be in 8.0 are kottd$, kottb$, kottbx$, kotad$ and kotmd$, which existed in 8.0 or 8.1, when 8.0 compatibility was possible, kotadx$ was created in 9iR2 when we required 8.1 minimum  compatibility. The source code is fully exposed for both package header and body."
/rdbms/admin/dbmsobj.sql
If I try to run that same procedure within SQLPlus (after the upgrade failed), I get the error message ORA-00955: name is already used by an existing object.   This tells me that the upgrade script actually created the newer version of the dictionary images, but for some reason failed to complete.  And so when I re-run the upgrade script, it complains.
We've searched google, SAP Notes, sdn, etc. but havent gotten anywhere.    Help or hints would be greatly appreciated.
Thanks

Answer 1

HI,
Check below details from oracle Metalink.
Curtesy of Oracle Metalink
A1) Errors connecting as SYSDBA / Internal OR on startup nomount
There is something fundamental wrong with the software / environment
if you cannot connect to Server Manager as a DBA user.
The steps here cover errors such as ORA-3113, ORA-12547: TNS:lost contact
or similar errors connecting to Oracle or starting the instance NOMOUNT.
Check the following items:
A1.1) If possible reboot the server disabling any automatic
startup of Oracle before you do so. This may seem drastic
but helps make sure you are working from a consistent
starting point.
A1.2) Check your environment points at the expected ORACLE_HOME
and ORACLE_SID and that TWO_TASK is not set (Unix) or
LOCAL is not set (NT registry).
Check the USER_DUMP_DEST and BACKGROUND_DUMP_DEST and default
trace directories under this environment for any user trace
files or alert log entries generated. These may help indicate
the cause of the problem.
Eg: ORA-600[SKGMINVALID] may indicate a problem with the
shared memory Unix parameters on Unix systems.
Try to show that any trace file / alert log entry you
find is truely related to the "CONNECT" command by re-issuing
the "connect" and checking for a new trace file / alert entry
at the time of the error.
A1.3) Unix only:
Some Unix platforms need LD_LIBRARY_PATH to be set
correctly to resolve any dynamically linked libraries.
As the user with the problem:
% script /tmp/ldd.out
% id
% cd $ORACLE_HOME/bin
% ldd oracle
% exit
If the 'ldd' command does not exist go to the next step below.
Check that all lines listed show a full library file. If there
are any 'not found' lines reported contact Oracle support
with the output of /tmp/ldd.out .
A1.4) Unix only:
Your 'oracle' executable may be corrupt. Relink it thus:
Log in as the 'oracle' user.
% script /tmp/relink.out
% cd $ORACLE_HOME/rdbms/lib
% mv $ORACLE_HOME/bin/oracle $ORACLE_HOME/bin/oracle.dd.mon.yy
% rm -f ./oracle
% make -f ins_rdbms.mk ioracle
% exit
Prior to Oracle7.3 the relink command was:
make -f oracle.mk ioracle
If this reports any errors Oracle support will need to see
the contents of the file /tmp/relink.out .
A1.5) Have you installed the Parallel Server Option ?
ORA-3113 can occur if you have installed the Parallel
Server Option but do NOT have a Distributed Lock Manager
installed or running correctly.
Unix:
If the Parallel Server Option was installed by accident
then it can be de-installed by relinking.
Eg:
Shut down any Oracle instances
% script /tmp/relink.out
% cd $ORACLE_HOME/rdbms/lib
'oracle' should not exist so delete it if it present
% rm -f oracle
% make -f ins_rdbms.mk no_parropt ioracle
% exit
NB: Do NOT deinstall the Parallel Server Option if the
database is using Parallel Server unless both nodes
are shut down otherwise database corruption could occur.
A1.6) If the error is on STARTUP NOMOUNT:
Check the init.ora file used to start the database.
This provides the configuration details used
configure the instance. To help isolate the problem
it may be useful use a very basic init.ora file
when starting the instance. If this works then
parameters can be increased / introduced one at a
time to see if there is a problem with a particular
setting.
A1.7) Check for server side trace files which may give more
indication what the underlying problem is.
See section C for details on how to check
for server trace files.
A1.8) Ensure there is free disk space in:
a. Your USER_DUMP_DEST and BACKGROUND_DUMP_DEST locations
b. Your AUDIT destination (Unix)
The default is $ORACLE_HOME/rdbms/audit
c. Your Oracle Trace directory if Oracle Trace is enabled
See <Note:45482.1>
A2) Errors Mounting the database
Check all the items in A1 first.
If an error occurs when mounting the database there may be problems
with the control-files or data files, or with resources required to
open these files.
A2.1) The location of the control files are specified in the
init.ora file. Try mounting using each controlfile in
turn.
eg: "Shutdown abort",
edit the init.ora to refer to ONE of the controlfiles only,
"startup nomount",
"alter database mount"
Repeat for each controlfile to see if any controlfile works.
A2.2) It is possible to re-create the controlfiles if you know the
location of all datafiles and online logs, or to restore an old
backup controlfile. Always back up the current controlfiles before
restoring any backup copies or issuing a CREATE CONTROLFILE
command.
The steps for this are not documented here.
A2.3) Unix:
Some unix platforms have a 'truss' command (or 'tusc').
If available this can be used to help trace how far Oracle
gets before the error occurs.
Eg:
% truss -o /tmp/truss.out -f svrmgrl
Keep the file /tmp/truss.out safe - Oracle Support MAY need to see it.
Regards,
Venkata S Pagolu
Edited by: Venkata Pagolu on Mar 10, 2012 12:10 PM