SAP not starting after crash: Terminateing With error code 12

Question

Hi,
After system crash SAP is not starting normally. When running startsap under SIDadm puts out: /usr/sap/TST/SYS/exe/run/startdb: Terminateing With error code 12
startdb.log shows that:
/oracle/TST/102_64/bin/sqlplus: error while loading shared libraries: libsqlplus.so: cannot open shares object file: No such file or directory
I have checked - file and directory are present.
However - when manually starting DB under oraSID and then logging in as SIDadm and starting SAP - everything starts normally.
(Allthough SID is has been changed before - system also worked normally after SID change) I have checked userrights and groups , also profile - everything seems to be normal. Any ideas?
Edited by: Kristjan T on Mar 16, 2011 1:16 PM

Answer 1

Hi,
1.what UNIX Flavor are we talking about  -> SuSe
2.What SAP Kernel Release is in place (may be 6.40 non EX2 ?) -> 6.40
3.What release is the DB suposed to have? -> 10.2.0.2.0 - 64bit Prod
4.I see, you are calling an sqlplus from a 10.2 ORACLE_HOME.
Is that the correct HOME the DB is running with ? -> Yes
5.Can you start the DB with
brconnect -c -u / -f dbstart  -> NO
Error is:
:devadm 51> brconnect -c -u / -f dbstart
BR0801I BRCONNECT 7.00 (40)
oracleDEV: error while loading shared libraries: libskgxp10.so: cannot open shared object file: No such file or directory
BR0278E Command output of '/oracle/DEV/102_64/bin/sqlplus /nolog':
/oracle/DEV/102_64/bin/sqlplus: error while loading shared libraries: libsqlplus.so: cannot open shared object file: No such file or directory
BR0280I BRCONNECT time stamp: 2011-05-05 15.01.17
BR0279E Return code from '/oracle/DEV/102_64/bin/sqlplus /nolog': 127
BR0302E SQLPLUS call for database instance DEV failed
BR0303E Determination of Oracle version failed
BR0280I BRCONNECT time stamp: 2011-05-05 15.01.17
BR0804I BRCONNECT terminated with errors
(Different machine but same situation)
6.As already requested, what is the output of:
6.1.ldd sqlplus
:devadm 55> ldd sqlplus
ldd: ./sqlplus: No such file or directory
in addition:
6.2. env | grep -i ora
devadm 56> env | grep -i ora
PATH=/oracle/DEV/102_64/bin:/opt/IBMJava2-amd64-142/bin:.:/home/devadm:/usr/sap/DEV/SYS/exe/run:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/devadm/bin:/usr/games:/opt/gnome/bin:/opt/kde3/bin:/usr/bin/X11:/usr/lib/mit/bin:/usr/lib/mit/sbin
LD_LIBRARY_PATH=/usr/sap/DEV/SYS/exe/run:/oracle/client/10x_64/instantclient
dbms_type=ORA
dbs_ora_tnsname=DEV
dbs_ora_schema=SAPSR3
ORACLE_SID=DEV
ORACLE_BASE=/oracle
TNS_ADMIN=/usr/sap/DEV/SYS/profile/oracle
ORACLE_HOME=/oracle/DEV/102_64
SAPDATA_HOME=/oracle/DEV
6.3 env | grep -i nls
:devadm 57> env | grep -i nls
XNLSPATH=/usr/X11R6/lib/X11/nls
NLS_LANG=AMERICAN_AMERICA.UTF8
6.4 env | grep -i lib
devadm 58> env | grep -i lib
PATH=/oracle/DEV/102_64/bin:/opt/IBMJava2-amd64-142/bin:.:/home/devadm:/usr/sap/DEV/SYS/exe/run:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/devadm/bin:/usr/games:/opt/gnome/bin:/opt/kde3/bin:/usr/bin/X11:/usr/lib/mit/bin:/usr/lib/mit/sbin
XKEYSYMDB=/usr/X11R6/lib/X11/XKeysymDB
XNLSPATH=/usr/X11R6/lib/X11/nls
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig:/opt/kde3/lib64/pkgconfig:/opt/gnome/lib64/pkgconfig:/opt/gnome/lib64/pkgconfig:/opt/gnome/share/pkgconfig
GTK_PATH=/usr/local/lib/gtk-2.0:/opt/gnome/lib/gtk-2.0:/usr/lib/gtk-2.0
GTK_PATH64=/usr/local/lib64/gtk-2.0:/opt/gnome/lib64/gtk-2.0:/usr/lib64/gtk-2.0
DIR_LIBRARY=/usr/sap/DEV/SYS/exe/run
LD_LIBRARY_PATH=/usr/sap/DEV/SYS/exe/run:/oracle/client/10x_64/instantclient
Any ideas???
Br,
Kristjan T.