Instance start up error..

Hi,
first My database was running two nodes cluster then i added another instance(in primary database). it was working fine in primary.then i switch over the primary database and added instance.instance1,instance2 oped successfully
then i tried to open new instance3, it is mounted but i am getting following error when opening
ORA-00600: internal error code, arguments: [3700], [3], [2], [0], [0], [], [], []
Sat Nov 22 23:22:23 2008
LGWR: terminating instance due to error 470
Sat Nov 22 23:22:23 2008
Trace dumping is performing id=[cdmp_20081122232223]
Sat Nov 22 23:22:26 2008
Dump system state for local instance only
Sat Nov 22 23:22:26 2008
Trace dumping is performing id=[cdmp_20081122232226]
Sat Nov 22 23:22:28 2008
Instance terminated by LGWR, pid = 28854
Sun Nov 23 00:00:35 2008
thanks,

Hi Mohammed
Have you got any other error like
ORA-01538: failed to acquire
Please provide us the lines before.
How did you added the 3 nodes ? Are you settings in the control files supporting more than 2 instances? see the MAXINSTANCES.
What is the result of doing
alter database backup controlfile to trace ;
Control the file generated in udump it contains all the information of your control files.
You can change that value with out any downtime.
Note:331067.1
Please post your new posts/questions about RAC cluster in the following section :
Forum Home » Grid Computing » RAC, ASM & Clusterware Installation
Thanks
Edited by: Hub on Nov 22, 2008 10:28 PM

Similar Messages

SAP installation isse - sap instance start error

Hi,
     I am trying to install sap ecc 60u ides. During start instance (22nd step of installation) sap couldn't start .
sapinst log says:
INFO 2007-04-22 09:02:55
State of instance TE2/DVEBMGS00 changed from ABAP: STARTING to ABAP: STARTING.
ERROR 2007-04-22 09:02:55
CJS-30105 Instance TE2/DVEBMGS00 reached state UNKNOWN after having state STARTING. Giving up.
The error in the SAP MMC is, Database: > ORA-01017: invalid username/password; logon denied
After all debugging / changing so many entries in tables etc.
sqlplus / - works fine
sqlplus /@te2 - works fine
Now I got the error: ORA-12154: TNS:could not resolve the connect identifier specified, sap
but sqlplus sapte2/password@te2 gives the above error.
I changed the pw for sapte2 using brconnect and directly inserting into db table.
r3trans -d - works fine as well.
It could connect /@te2 to get the pw for sapte2 and could successfully connect to oracle using sapte2/pw@te2.
I am totally lost and gone mad. Could anyone help? Thanks

Hi,
   The issue is solved and SAP is up and running. The issue was, the sap service user was created in oracle with password. But it should have been created 'as identified by externally'.
Thanks guys for all your help in this.
Regards,
Sundar.

Database Crash - LGWR: terminating instance due to error 227

Hi,
We´re having a problem with our database. The server reboots abnormally and the database shutdown abort, when we bring online the machine the database didn´t open.
Analyzing the logs, we found a corrupt problem. see the alert file:
Corrupt block relative dba: 0x00000003 (file 0, block 3)
Completely zero block found during controlfile block read
LGWR: terminating instance due to error 227
Instance terminated by LGWR, pid = 2080
Dump file E:\ORACLE\PDS\bdump\pdsALRT.LOG
So ... we try to starts the database and he didn´t mount, we only startup nomount. see the log above:
Oracle Server Manager Release 3.1.7.0.0 - Production
Copyright (c) 1997, 1999, Oracle Corporation. All Rights Reserved.
Oracle8i Enterprise Edition Release 8.1.7.0.0 - Production
JServer Release 8.1.7.0.0 - Production
SVRMGR> connect internal
Password:
Connected.
SVRMGR> startup nomount
ORACLE instance started.
Total System Global Area 633632796 bytes
Fixed Size 75804 bytes
Variable Size 167411712 bytes
Database Buffers 466067456 bytes
Redo Buffers 77824 bytes
SVRMGR> alter database mount;
alter database mount
ORA-03113: end-of-file on communication channel
SVRMGR>
We doesn´t mount the database, so we can´t do a recovery.
Anybody knows the resolution of this problem ??
Regards,
Eduardo

We can´t do a recover because database is not mounted.
We don´t have a backup of the controlfile and didn´t make the "alter database backup controlfile to trace"
Thanks,
Eduardo

CKPT : Terminating instance due to Error 472

Hi All,
I got the above error and the system stopped and restarted on the DR side.
Snippet from Alert file before shutdown
===========================
Wed Nov 27 22:39:12 2013
Thread 1 advanced to log sequence 10770
Current log# 2 seq# 10770 mem# 0: /var/opt/oracle/oradata/xxxx/redo02.log
Wed Nov 27 22:39:12 2013
ARC0: Evaluating archive   log 1 thread 1 sequence 10769
ARC0: Beginning to archive log 1 thread 1 sequence 10769
Creating archive destination LOG_ARCHIVE_DEST_1: '/var/opt/oracle/oradata/xxxx/archive/1_10769.dbf'
ARC0: Completed archiving log 1 thread 1 sequence 10769
Wed Nov 27 23:18:37 2013
CKPT: terminating instance due to error 472
Instance terminated by CKPT, pid = 22682
After restart following is snippet
=======================
Wed Nov 27 23:19:19 2013
Starting ORACLE instance (force)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
SCN scheme 3
Using log_archive_dest parameter default value
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 9.2.0.1.0.
System parameters with non-default values:
processes                = 300
timed_statistics         = TRUE
shared_pool_size         = 486539264
large_pool_size          = 436207616
java_pool_size           = 33554432
control_files            = /var/opt/oracle//oradata/xxxx/control01.ctl, /var/opt/oracle//oradata/xxxx/control02.ctl, /var/opt/oracle//oradata/xxxx/control03.ctl
db_block_size            = 4096
db_cache_size            = 1761607680
compatible               = 9.2.0.0.0
log_archive_start        = TRUE
log_archive_dest_1       = LOCATION=/var/opt/oracle//oradata/xxxx/archive
log_archive_format       = %t_%s.dbf
db_file_multiblock_read_count= 8
fast_start_mttr_target   = 300
undo_management          = AUTO
undo_tablespace          = UNDOTBS1
undo_retention           = 900
remote_login_passwordfile= EXCLUSIVE
db_domain                =
instance_name            = xxxx
dispatchers              = (protocol=TCP)
job_queue_processes      = 10
hash_join_enabled        = FALSE
background_dump_dest     = /var/opt/oracle//admin/xxxx/bdump
user_dump_dest           = /var/opt/oracle//admin/xxxx/udump
core_dump_dest           = /var/opt/oracle//admin/xxxx/cdump
sort_area_size           = 524288
db_name                  = xxxx
open_cursors             = 300
star_transformation_enabled= FALSE
query_rewrite_enabled    = FALSE
pga_aggregate_target     = 524288000
aq_tm_processes          = 1
PMON started with pid=2
DBW0 started with pid=3
LGWR started with pid=4
CKPT started with pid=5
SMON started with pid=6
RECO started with pid=7
CJQ0 started with pid=8
QMN0 started with pid=9
Wed Nov 27 23:19:25 2013
starting up 1 shared server(s) ...
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=12
ARC0: Archival started
ARC1 started with pid=13
Wed Nov 27 23:19:25 2013
ARCH: STARTING ARCH PROCESSES COMPLETE
Wed Nov 27 23:19:25 2013
ARC1: Archival started
ARC1: Thread not mounted
Wed Nov 27 23:19:25 2013
ARC0: Thread not mounted
Wed Nov 27 23:19:26 2013
ALTER DATABASE   MOUNT
Wed Nov 27 23:19:32 2013
Successful mount of redo thread 1, with mount id 1572646464.
Wed Nov 27 23:19:32 2013
Database mounted in Exclusive Mode.
Completed: ALTER DATABASE   MOUNT
Wed Nov 27 23:19:32 2013
ALTER DATABASE OPEN
Wed Nov 27 23:19:32 2013
Beginning crash recovery of 1 threads
Wed Nov 27 23:19:32 2013
Started first pass scan
Wed Nov 27 23:19:33 2013
Completed first pass scan
11410 redo blocks read, 798 data blocks need recovery
Wed Nov 27 23:19:33 2013
Started recovery at
Thread 1: logseq 10770, block 3829, scn 0.0
Recovery of Online Redo Log: Thread 1 Group 2 Seq 10770 Reading mem 0
Mem# 0 errs 0: /var/opt/oracle/oradata/xxxx/redo02.log
Wed Nov 27 23:19:33 2013
Ended recovery at
Thread 1: logseq 10770, block 15239, scn 0.1667153707
798 data blocks read, 798 data blocks written, 11410 redo blocks read
Crash recovery completed successfully
==============
Was the CKPT Termination caused by some error in Redo Logs which needed recovery?

Lookup the Error messages documentation.
Error 472 is
"PMON process terminated with error "
CKPT detected that PMON had "died" and so it forcibly terminated the database instance. (The background processes monitor each other and are designed to terminate the database instance if a critical process dies).
Look for a trace file.
PMON died because of a resource issue or a bug or a server issue or because it was killed (by the OS or a user or process with oracle / administrator privileges)
Hemant K Chitale

PMON: terminating instance due to error 28

Hi ,
OS : AIX 5.3
Oracle : 10.2.0.4
Problem :
Database instance crashed .
Alert log shows :
Tue Jun 26 13:21:18 2012
Thread 1 advanced to log sequence 4097 (LGWR switch)
Current log# 1 seq# 4097 mem# 0: /oracle/abcd/oradata1/redo/abcd_redo_g1m1.dbf
Current log# 1 seq# 4097 mem# 1: /oracle/abcd/oradata2/redo/abcd_redo_g1m2.dbf
Tue Jun 26 14:29:56 2012 ---> Instance terminated
Errors in file /oracle/abcd/oratrace/bdump/abcd_pmon_2834484.trc:
ORA-00028: your session has been killed
Tue Jun 26 14:29:56 2012
PMON: terminating instance due to error 28
Termination issued to instance processes. Waiting for the processes to exit
Instance terminated by PMON, pid = 2834484
Tue Jun 26 15:09:00 2012 --> Instance started manually at this point
Starting ORACLE instance (normal)
Trace file shows :
/oracle/abcd/oratrace/bdump/abcd_pmon_2834484.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /oracle/product/10.2.0/ora10gr2
System name: AIX
Node name: a838dae0
Release: 3
Version: 5
Machine: 00047961D600
Instance name: abcd
Redo thread mounted by this instance: 1
Oracle process number: 2
Unix process pid: 2834484, image: oracle@a838dae0 (PMON)
*** 2012-06-26 14:29:56.830
*** SERVICE NAME:(SYS$BACKGROUND) 2012-06-26 14:29:56.801
*** SESSION ID:(225.1) 2012-06-26 14:29:56.801
error 28 detected in background process
ORA-00028: your session has been killed
OS logs does not have any errors when the instance got terminated .
I would like to know what is the cause for this error ? when such errors occurs ?

@pavan , thanks for the advise . I shall work with Unix admin on how best we can trace it out if it occurs next time .
However , I have one more question :
I did a small test to crash the instance by killing the pmon process at OS level . When I did that only the following message got recorded in the alert log :
Wed Jun 27 04:39:26 2012
PSP0: terminating instance due to error 472 --> error code is 472
Instance terminated by PSP0, pid = 4970
But the issue I see in produciton db is something different , the alert log shows :
ORA-00028: your session has been killed
Tue Jun 26 14:29:56 2012
PMON: terminating instance due to error 28 ----> error code is 28
Termination issued to instance processes. Waiting for the processes to exit
Instance terminated by PMON, pid = 2834484
I have started to belive that the PMON was not killed from OS level . I dont think error ORA-00028 gets recorded when you kill pmon from OS level as I could not reproduce in my testing as well.
Anyways , thanks for your help on this.

USER: terminating instance due to error 472 (Oracle 10g)

Hi folks,
We come across a problem that our instance was down suddenly. When checking with the alert log file, it shows that the instance was terminated due to error 472. Did you come across such situation before? Any suggestions would be welcomed.
Following are some information about our Oracle and the log file, trace file.
Oracle Database 10g Release 10.2.0.2.0 - Production
ORACLE_HOME = /app/oracle/lmes
System name: Linux
Node name: lmes-oracle2.ust.hk
Release: 2.4.21-32.ELsmp
Version: #1 SMP Fri Apr 15 21:17:59 EDT 2005
Machine: i686
Alert Log File
Fri Feb 23 09:11:07 2007
Shutting down archive processes
Fri Feb 23 09:11:12 2007
ARCH shutting down
ARC2: Archival stopped
Fri Feb 23 15:16:09 2007
The value (30) of MAXTRANS parameter ignored.
kupprdp: master process DM00 started with pid=32, OS id=1072
to execute - SYS.KUPM$MCP.MAIN('SYS_EXPORT_SCHEMA_01', 'SYSTEM', 'KUPC$C_1_20070223151610', 'KUPC$S_1_20070223151610', 0);
kupprdp: worker process DW01 started with worker id=1, pid=33, OS id=1077
to execute - SYS.KUPW$WORKER.MAIN('SYS_EXPORT_SCHEMA_01', 'SYSTEM');
Fri Feb 23 15:16:28 2007
Thread 1 advanced to log sequence 1309
Current log# 3 seq# 1309 mem# 0: /app/oracle/oradata/elop/redo03.log
Fri Feb 23 15:20:29 2007
The value (30) of MAXTRANS parameter ignored.
kupprdp: master process DM00 started with pid=32, OS id=1526
to execute - SYS.KUPM$MCP.MAIN('SYS_EXPORT_SCHEMA_01', 'SYSTEM', 'KUPC$C_1_20070223152029', 'KUPC$S_1_20070223152029', 0);
kupprdp: worker process DW01 started with worker id=1, pid=33, OS id=1528
to execute - SYS.KUPW$WORKER.MAIN('SYS_EXPORT_SCHEMA_01', 'SYSTEM');
Fri Feb 23 22:15:35 2007
Thread 1 advanced to log sequence 1310
Current log# 1 seq# 1310 mem# 0: /app/oracle/oradata/elop/redo01.log
Fri Feb 23 23:04:10 2007
USER: terminating instance due to error 472
Instance terminated by USER, pid = 7779
Trace File - elop_lgwr_2312.trc
/app/oracle/admin/elop/bdump/elop_lgwr_2312.trc
Oracle Database 10g Release 10.2.0.2.0 - Production
ORACLE_HOME = /app/oracle/lmes
System name: Linux
Node name: lmes-oracle2.ust.hk
Release: 2.4.21-32.ELsmp
Version: #1 SMP Fri Apr 15 21:17:59 EDT 2005
Machine: i686
Instance name: elop
Redo thread mounted by this instance: 1
Oracle process number: 6
Unix process pid: 2312, image: [email protected] (LGWR)
*** SERVICE NAME:() 2007-02-23 09:10:07.372
*** SESSION ID:(331.1) 2007-02-23 09:10:07.372
LGWR: Archivelog for thread 1 sequence 1308 will NOT be compressed
tkcrrsarc: (WARN) Failed to find ARCH for message (message:0x1)
tkcrrpa: (WARN) Failed initial attempt to send ARCH message (message:0x1)
Maximum redo generation record size = 156160 bytes
Maximum redo generation change vector size = 150672 bytes
tkcrrsarc: (WARN) Failed to find ARCH for message (message:0x10)
tkcrrpa: (WARN) Failed initial attempt to send ARCH message (message:0x10)
*** 2007-02-23 15:16:27.116
LGWR: Archivelog for thread 1 sequence 1309 will NOT be compressed
*** 2007-02-23 22:15:35.383
LGWR: Archivelog for thread 1 sequence 1310 will NOT be compressed
Trace File - elop_ora_2336.trc
/app/oracle/admin/elop/udump/elop_ora_2336.trc
Oracle Database 10g Release 10.2.0.2.0 - Production
ORACLE_HOME = /app/oracle/lmes
System name: Linux
Node name: lmes-oracle2.ust.hk
Release: 2.4.21-32.ELsmp
Version: #1 SMP Fri Apr 15 21:17:59 EDT 2005
Machine: i686
Instance name: elop
Redo thread mounted by this instance: 1
Oracle process number: 15
Unix process pid: 2336, image: [email protected]
*** SERVICE NAME:() 2007-02-23 09:10:05.988
*** SESSION ID:(324.3) 2007-02-23 09:10:05.988
Thread 1 checkpoint: logseq 1307, block 2, scn 27684035
cache-low rba: logseq 1307, block 30724
on-disk rba: logseq 1307, block 33205, scn 27699991
start recovery at logseq 1307, block 30724, scn 0
----- Redo read statistics for thread 1 -----
Read rate (ASYNC): 1240Kb in 0.03s => 40.38 Mb/sec
Total physical reads: 4096Kb
Longest record: 21Kb, moves: 0/1975 (0%)
Change moves: 1/24 (4%), moved: 0Mb
Longest LWN: 285Kb, moves: 0/245 (0%), moved: 0Mb
Last redo scn: 0x0000.01a6ab16 (27699990)
----- Recovery Hash Table Statistics ---------
Hash table buckets = 32768
Longest hash chain = 2
Average hash chain = 346/345 = 1.0
Max compares per lookup = 1
Avg compares per lookup = 4783/5281 = 0.9
*** 2007-02-23 09:10:06.132
KCRA: start recovery claims for 346 data blocks
*** 2007-02-23 09:10:07.153
KCRA: blocks processed = 346/346, claimed = 346, eliminated = 0
*** 2007-02-23 09:10:07.154
Recovery of Online Redo Log: Thread 1 Group 1 Seq 1307 Reading mem 0
----- Recovery Hash Table Statistics ---------
Hash table buckets = 32768
Longest hash chain = 2
Average hash chain = 346/345 = 1.0
Max compares per lookup = 2
Avg compares per lookup = 3424/5126 = 0.7
tkcrrsarc: (WARN) Failed to find ARCH for message (message:0x1)
tkcrrpa: (WARN) Failed initial attempt to send ARCH message (message:0x1)
Error in executing triggers on database startup
*** 2007-02-23 09:10:11.557
ksedmp: internal or fatal error
ORA-00604: error occurred at recursive SQL level 1
ORA-12663: Services required by client not available on the server
ORA-36961: Oracle OLAP is not available.
ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
ORA-06512: at line 15
Thanks & Regards,
Liona

Hi,
I have the same error:
I have gridcontrol 10gR3 installed on VM / Linux 4 AS box. This is second time down time:
finally I found the DB tiers is down.
I have the following message:
Private_strands 18 at log switch
Thread 1 advanced to log sequence 215
Current log# 2 seq# 215 mem# 0: /u02/ora/gridcontrol10g/oradata/emrep/redo02.log
Thu Apr 19 14:30:33 2007
Private_strands 18 at log switch
Thread 1 advanced to log sequence 216
Current log# 3 seq# 216 mem# 0: /u02/ora/gridcontrol10g/oradata/emrep/redo03.log
Thu Apr 19 15:42:34 2007
Private_strands 18 at log switch
Thread 1 advanced to log sequence 217
Current log# 4 seq# 217 mem# 0: /u02/ora/gridcontrol10g/oradata/emrep/redo04.log
Thu Apr 19 16:52:05 2007
LGWR: terminating instance due to error 472
Instance terminated by LGWR, pid = 26049
Thu Apr 19 17:27:08 2007
I have no idea why this happen. is it because DB is ona VM machine?
I remember one time whe I took a snapshot on VM server, grid control services are down for a moment, 4 minutes, WebCache seemed cleaned, this page is gone, after 3 minutes, it comes back.
Anyone has the same issue, please share your experience.
Thanks a lot,
Hank
Message was edited by:
Hank@AHM

USER (ospid: 2268): terminating the instance due to error 12853

Good morning all.
I have a tricky situation going on around here....
My Oracle DB was working fine till 5 minutes ago. Once we bounced the server, It stopped working. Here are some details:
ORACLE RDBMS Version: 11.1.0.6.0
C:\Documents and Settings\Administrator>tnsping dwcorp
TNS Ping Utility for 32-bit Windows: Version 11.1.0.6.0 - Production on 12-MAR-2
009 11:46:44
Copyright (c) 1997, 2007, Oracle. All rights reserved.
Used parameter files:
D:\app\Administrator\product\11.1.0\client_2\network\admin\sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)
(HOST = riovermelho)(PORT = 1521))) (CONNECT_DATA = (SERVICE_NAME = dwcorp)))
OK (10 msec)
SQL> conn [sys@dwcorp|mailto:sys@dwcorp] /as sysdba
Enter password:
ERROR:
ORA-12514: TNS:listener does not currently know of service requested in connect
descriptor
Something I consider may cause this issue is that there was a change in the Shared pool size and java pool size without any precautions...
Wed Mar 11 16:03:13 2009
ALTER SYSTEM SET java_pool_size='150M' SCOPE=SPFILE;
ALTER SYSTEM SET shared_pool_size='150M' SCOPE=SPFILE;
Wed Mar 11 16:06:10 2009
ALTER SYSTEM SET shared_pool_size='450M' SCOPE=SPFILE;
ALTER SYSTEM SET java_pool_size='150M' SCOPE=SPFILE;
Wed Mar 11 16:07:03 2009
ALTER SYSTEM SET java_pool_size='150M' SCOPE=SPFILE;
Wed Mar 11 16:14:35 2009
ALTER SYSTEM SET shared_pool_size='800M' SCOPE=SPFILE;
Wed Mar 11 21:24:04 2009
Even after these changes, made yesterday, database was working perfectly fine. Here is the ALERT LOG output, after bounce:
Starting up ORACLE RDBMS Version: 11.1.0.6.0.
Using parameter settings in server-side spfile D:\APP\ADMINISTRATOR\PRODUCT\11.1.0\DB_1\DATABASE\SPFILEDWCORP.ORA
System parameters with non-default values:
processes                = 150
shared_pool_size         = 800M
java_pool_size           = 152M
memory_target            = 1552M
control_files            = "E:\APP\ADMINISTRATOR\ORADATA\DWCORP\CONTROL01.CTL"
control_files            = "E:\APP\ADMINISTRATOR\ORADATA\DWCORP\CONTROL02.CTL"
control_files            = "E:\APP\ADMINISTRATOR\ORADATA\DWCORP\CONTROL03.CTL"
db_block_size            = 8192
compatible               = "11.1.0.0.0"
db_recovery_file_dest    = "D:\app\Administrator\flash_recovery_area"
db_recovery_file_dest_size= 2G
undo_tablespace          = "UNDOTBS1"
remote_login_passwordfile= "EXCLUSIVE"
db_domain                = ""
dispatchers              = "(PROTOCOL=TCP) (SERVICE=dwcorpXDB)"
audit_file_dest          = "D:\APP\ADMINISTRATOR\ADMIN\DWCORP\ADUMP"
audit_trail              = "DB"
db_name                  = "dwcorp"
open_cursors             = 300
star_transformation_enabled= "TRUE"
diagnostic_dest          = "D:\APP\ADMINISTRATOR"
Thu Mar 12 11:44:31 2009
PMON started with pid=2, OS id=3156
Thu Mar 12 11:44:31 2009
VKTM started with pid=3, OS id=1324 at elevated priority
VKTM running at (20)ms precision
Thu Mar 12 11:44:31 2009
DIAG started with pid=4, OS id=3000
Thu Mar 12 11:44:31 2009
DBRM started with pid=5, OS id=2920
Thu Mar 12 11:44:31 2009
PSP0 started with pid=6, OS id=3004
Thu Mar 12 11:44:31 2009
DSKM started with pid=7, OS id=3320
Thu Mar 12 11:44:31 2009
DIA0 started with pid=8, OS id=2072
Thu Mar 12 11:44:31 2009
MMAN started with pid=7, OS id=4068
Thu Mar 12 11:44:31 2009
DBW0 started with pid=9, OS id=2092
Thu Mar 12 11:44:31 2009
LGWR started with pid=10, OS id=1296
Thu Mar 12 11:44:31 2009
CKPT started with pid=11, OS id=1736
Thu Mar 12 11:44:31 2009
SMON started with pid=12, OS id=2816
Thu Mar 12 11:44:31 2009
RECO started with pid=13, OS id=964
Thu Mar 12 11:44:31 2009
MMON started with pid=14, OS id=3040
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Thu Mar 12 11:44:31 2009
MMNL started with pid=15, OS id=1492
starting up 1 shared server(s) ...
Thu Mar 12 11:44:34 2009
Sweep Incident[26443]: completed
Sweep Incident[26442]: completed
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27649):
ORA-04031: unable to allocate 257772 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Incident details in: d:\app\administrator\diag\rdbms\dwcorp\dwcorp\incident\incdir_27649\dwcorp_ora_2268_i27649.trc
Trace dumping is performing id=[cdmp_20090312114438]
Thu Mar 12 11:44:44 2009
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27650):
ORA-04031: unable to allocate 128892 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Incident details in: d:\app\administrator\diag\rdbms\dwcorp\dwcorp\incident\incdir_27650\dwcorp_ora_2268_i27650.trc
Thu Mar 12 11:44:45 2009
Trace dumping is performing id=[cdmp_20090312114445]
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27651):
ORA-04031: unable to allocate 64452 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Incident details in: d:\app\administrator\diag\rdbms\dwcorp\dwcorp\incident\incdir_27651\dwcorp_ora_2268_i27651.trc
Trace dumping is performing id=[cdmp_20090312114451]
Thu Mar 12 11:44:57 2009
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27652):
ORA-04031: unable to allocate 32232 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Incident details in: d:\app\administrator\diag\rdbms\dwcorp\dwcorp\incident\incdir_27652\dwcorp_ora_2268_i27652.trc
Thu Mar 12 11:44:58 2009
Trace dumping is performing id=[cdmp_20090312114458]
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27653):
ORA-04031: unable to allocate 16124 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Incident details in: d:\app\administrator\diag\rdbms\dwcorp\dwcorp\incident\incdir_27653\dwcorp_ora_2268_i27653.trc
Trace dumping is performing id=[cdmp_20090312114504]
Thu Mar 12 11:45:10 2009
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27654):
ORA-04031: unable to allocate 16012 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27655):
ORA-04031: unable to allocate 257772 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Thu Mar 12 11:45:23 2009
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27656):
ORA-04031: unable to allocate 128892 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27657):
ORA-04031: unable to allocate 64452 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Thu Mar 12 11:45:34 2009
Sweep Incident[27657]: completed
Sweep Incident[27656]: completed
Sweep Incident[27655]: completed
Sweep Incident[27654]: completed
Sweep Incident[27653]: completed
Sweep Incident[27652]: completed
Sweep Incident[27651]: completed
Sweep Incident[27650]: completed
Sweep Incident[27649]: completed
Thu Mar 12 11:45:35 2009
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27658):
ORA-04031: unable to allocate 32232 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
Errors in file d:\app\administrator\diag\rdbms\dwcorp\dwcorp\trace\dwcorp_ora_2268.trc (incident=27659):
ORA-04031: unable to allocate 21492 bytes of shared memory ("large pool","unknown object","large pool","PX msg pool")
USER (ospid: 2268): terminating the instance due to error 12853
Instance terminated by USER, pid = 2268Any ideas?

Hi Anantha.
First of all, really appreciate your help.
Completing information regarding my environment, it's a Windows 2003 server (SP2).
I've read this thread you posted. It's been a long time since I was a DBA, till oracle 8i. Now, stuff is completely different, so, i'm kinda lost...
The point is that the database is not up and it fails whenever I try to start it.
I've tried to mannualy recreate my PFILE. So, I've made a copy of the SPFILE, removed special characters and now it looks like this:
dwcorp.__db_cache_size=335544320
dwcorp.__java_pool_size=33554432
dwcorp.__large_pool_size=83886080
dwcorp.__oracle_base='D:\app\Administrator'#ORACLE_BASE set from environment
dwcorp.__pga_aggregate_target=64592281
dwcorp.__sga_target=981467136
dwcorp.__shared_io_pool_size=16777216
dwcorp.__shared_pool_size=57881395
dwcorp.__streams_pool_size=0
*.audit_file_dest='D:\app\Administrator\admin\dwcorp\adump'
*.audit_trail='db'
*.compatible='11.1.0.0.0'
*.control_files='E:\app\Administrator\oradata\dwcorp\control01.ctl','E:\app\Administrator\oradata\dwcorp\control02.ctl','E:\app\Administrator\oradata\dwcorp\control03.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='dwcorp'
*.db_recovery_file_dest='D:\app\Administrator\flash_recovery_area'
*.db_recovery_file_dest_size=2147483648
*.diagnostic_dest='D:\app\Administrator'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=dwcorpXDB)'
*.java_pool_size=157286400
*.memory_target=1622147072
*.open_cursors=300
*.processes=150
*.remote_login_passwordfile='EXCLUSIVE'
*.shared_pool_size=83886080
*.star_transformation_enabled='TRUE'
*.undo_tablespace='UNDOTBS1'I've tried to start it up using ORADIM, since NET START (or either windows services panel) is not working.
C:\Documents and Settings\Administrator>oradim -startup -sid dwcorp -starttype srvc,inst -pfile D:\app\Administrator\product\11.1.0\db_1\database\archive\pfiled
wcorp.ora
ORA-01078: failure in processing system parametersIf you need more information, please, just ask me. My database is down and the whole dev. team is idle......
Another question is, I know I can connect to Oracle down, as an idle instance. But I cannot remember how to do it.
Please, refresh my memory and tell me how to connect to an idle instance, so I could try to manually start database.
As you may notice in my first post, This issue started after I bounced the server. I have done it several times before, but Yesterday, I changed some of the Oracle parameters (SHARED_POOL_SIZE and JAVA_POOL_SIZE).
TIA,
Marcos

Dbw0: terminating instance due to error 472

Running RH linux 6.2, and EE 8.1.6.1.
Am using MTS and have applied oracle's patch
on the ora601 bug.
HOwever I have come across a terminated instance with error 472 on two occasions. Both times I was running a soak test for first 15+ hrs and then 20+ hrs when the database crashed.
Any ideas ?

Alert Log :
Current log# 14 seq# 31016 mem# 0: /db/prime/oradata/onlinelogs/redo14a.log
Sun Apr 19 06:54:26 2009
Thread 1 advanced to log sequence 31017
Current log# 15 seq# 31017 mem# 0: /db/prime/oradata/onlinelogs/redo15a.log
Sun Apr 19 06:59:01 2009
Thread 1 advanced to log sequence 31018
Current log# 11 seq# 31018 mem# 0: /db/prime/oradata/onlinelogs/redo11a.log
Sun Apr 19 07:00:28 2009
DBW0: terminating instance due to error 472
Termination issued to instance processes. Waiting for the processes to exit
Sun Apr 19 07:00:38 2009
Instance termination failed to kill one or more processes
Instance terminated by DBW0, pid = 331960
Sun Apr 19 07:29:04 2009
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
IMODE=BR

DBW0: terminating instance due to error 27061

Hi,
PROD
Db 9.2.0.8
AIX 5.2
Suddenly database crashed. What I found in alert log file and trace file find below.
I started database it started normally but it took much time to start. Suggest me why it occurred and what I need to change so that I could not face again it.
Alert log file:
ARC1: Completed archiving log 2 thread 1 sequence 3653
Mon May 9 21:14:43 2011
Errors in file /vol01/oracle/product/admin/prod/bdump/prod_dbw0_695450.trc
ORA-27061: skgfospo: waiting for async I/Os failed
IBM AIX RISC System/6000 Error: 22: Invalid argument
Mon May 9 21:14:43 2011
DBW0: terminating instance due to error 27061
Instance terminated by DBW0, pid = 695450
Trace file prod_dbw0_695450.trc:
Redo thread mounted by this instance: 1
Oracle process number: 3
Unix process pid: 695450, image: oracle@prod (DBW0)
*** 2011-05-09 21:14:43.089
*** SESSION ID:(2.1) 2011-05-09 21:14:43.028
error 27061 detected in background process
ORA-27061: skgfospo: waiting for async I/Os failed
IBM AIX RISC System/6000 Error: 22: Invalid argument
During database starting find below some of the lines of alert log file:
Completed: ALTER DATABASE MOUNT
Tue May 10 08:46:17 2011
ALTER DATABASE OPEN
Tue May 10 08:46:17 2011
Beginning crash recovery of 1 threads
Tue May 10 08:46:17 2011
Started redo scan
Tue May 10 08:46:32 2011
Completed redo scan
1884170 redo blocks read, 61752 data blocks need recovery
Tue May 10 08:46:45 2011
Started recovery at
Thread 1: logseq 3653, block 1324175, scn 0.0
Tue May 10 08:46:46 2011
Recovery of Online Redo Log: Thread 1 Group 2 Seq 3653 Reading mem 0
Mem# 0 errs 0: /vol01/oracle/product/oradata/prod/redo2a.log
Mem# 1 errs 0:/vol02/oracle/product/oradata/prod//redo2b.log
Tue May 10 08:46:57 2011
Recovery of Online Redo Log: Thread 1 Group 3 Seq 3654 Reading mem 0
Mem# 0 errs 0: /vol01/oracle/product/oradata/prod/redo3a.log
Mem# 1 errs 0: /vol02/oracle/product/oradata/prod/redo3b.log
Tue May 10 08:53:02 2011
Completed redo application
Tue May 10 08:53:13 2011
Ended recovery at
Thread 1: logseq 3654, block 1111196, scn 3.3471125210
61752 data blocks read, 38719 data blocks written, 1884170 redo blocks read
Crash recovery completed successfully
Thanks

Check metalink note 464768.1. The version mentioned in the note is 10.2.0.1 + but I think it applies in your case too.
Confirm it by raising an SR with Oracle Support.

PMON: terminating instance due to error 600

Hi all,
Our instance was terminated by giving following message:
PMON: terminating instance due to error 600
OS: W2k sp4
Part of alert log:
Thu Sep 21 03:00:42 2006
Errors in file d:\oracle\admin\ropprod\udump\ropprod_ora_3388.trc:
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
Thu Sep 21 03:00:43 2006
Errors in file d:\oracle\admin\ropprod\udump\ropprod_ora_3388.trc:
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
Thu Sep 21 03:00:43 2006
Errors in file d:\oracle\admin\ropprod\udump\ropprod_ora_3388.trc:
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
Thu Sep 21 03:00:44 2006
Errors in file d:\oracle\admin\ropprod\udump\ropprod_ora_3388.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
Thu Sep 21 03:00:44 2006
Errors in file d:\oracle\admin\ropprod\bdump\ropprod_pmon_2692.trc:
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
Thu Sep 21 03:01:12 2006
Errors in file d:\oracle\admin\ropprod\bdump\ropprod_pmon_2692.trc:
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
Thu Sep 21 03:01:12 2006
PMON: terminating instance due to error 600
Instance terminated by PMON, pid = 2692
Part of 2692.trc:
Dump file d:\oracle\admin\ropprod\bdump\ropprod_pmon_2692.trc
Thu Sep 21 03:00:44 2006
ORACLE V9.2.0.6.0 - Production vsnsta=0
vsnsql=12 vsnxtr=3
Windows 2000 Version 5.0 Service Pack 4, CPU type 586
Oracle9i Enterprise Edition Release 9.2.0.6.0 - Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.6.0 - Production
Windows 2000 Version 5.0 Service Pack 4, CPU type 586
Instance name: ropprod
Redo thread mounted by this instance: 1
Oracle process number: 2
Windows thread id: 2692, image: ORACLE.EXE
*** 2006-09-21 03:00:44.367
*** SESSION ID:(1.1) 2006-09-21 03:00:44.351
KSSDCH: parent 27D8418C cur 27CE53D8 prev 27D8419C next 27CE53E0 own 00000000 link [27ce53e0,27ce53e0]
SO: 27CE53D8, type: 216, owner: 00000000, flag: INIT/FLST/-/0x50
*** 2006-09-21 03:00:44.367
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [1115], [], [], [], [], [], [], []
I restarted the oracle instance and it`s working fine now but i don`t know why the instance was terminated and how I can avoid it in future.
Pls,send your comments.
Regards,Lukas

Take a look at Metalink. Start with Note:137262.1 ORA-600[1115] SO cleanup. That document has links to several bugs and one might be what you are experiencing.

Terminating instance due to error - CKPT process terminated with error

Hi Folks !
i have a problem with my oracle 10g it works under an debain 3.1. In my alert.log i have see an error-message, the database terminating.
Thu Feb 15 13:01:21 2007
Errors in file /u01/app/oracle/admin/tlp/bdump/tlp_pmon_9689.trc:
ORA-00469: CKPT process terminated with error
Thu Feb 15 13:01:21 2007
PMON: terminating instance due to error 469
Instance terminated by PMON, pid = 9689
A look into the trace file:
/u01/app/oracle/admin/tlp/bdump/tlp_pmon_9689.trc
Oracle Database 10g Release 10.2.0.1.0 - Production
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1
System name: Linux
Node name: hyperion
Release: 2.6.18.1-vs2.1.1-rc48-vserver
Version: #1 SMP PREEMPT Wed Nov 15 16:53:41 CET 2006
Machine: i686
Instance name: tlp
Redo thread mounted by this instance: 1
Oracle process number: 2
Unix process pid: 9689, image: oracle@hyperion (PMON)
*** 2007-02-15 13:01:21.813
*** SERVICE NAME:(SYS$BACKGROUND) 2007-02-15 13:01:21.813
*** SESSION ID:(170.1) 2007-02-15 13:01:21.813
Background process CKPT found dead
Oracle pid = 7
OS pid (from detached process) = 9700
OS pid (from process state) = 9700
dtp = 0x2000c9ac, proc = 0x672b14c4
Dump of memory from 0x2000C9AC to 0x2000C9D8
2000C9A0 00000076 [v...]
2000C9B0 672B14C4 00000000 00000000 54504B43 [..+g........CKPT]
2000C9C0 00000200 000025E4 0A8BA210 00000001 [.....%..........]
2000C9D0 70F2A67F 00040081 [...p....]
Dump of memory from 0x672B14C4 to 0x672B1A78
672B14C0 00000102 00000000 00000000 [............]
672B14D0 00000000 00000000 65621FF8 67825F00 [..........be._.g]
672B14E0 673C0034 678250A4 00000000 67825108 [4.<g.P.g.....Q.g]
672B14F0 67825108 67825EF4 01000601 673AB4A0 [.Q.g.^.g......:g]
672B1500 673C0034 00000007 00000000 00000000 [4.<g............]
672B1510 00000000 674CB40C 674CC04C 00000000 [......LgL.Lg....]
672B1520 00000000 00000000 00000000 00000000 [................]
Repeat 3 times
672B1A70 00000003 00000005 [........]
error 469 detected in background process
ORA-00469: CKPT process terminated with error
What can i do ? Have you any ideas to fix the problem ?
Problem 2:
After the error message i start my server and see in my alert.log:
Starting background process QMNC
QMNC started with pid=17, OS id=17164
Thu Feb 15 14:33:45 2007
Errors in file /u01/app/oracle/admin/tlp/udump/tlp_ora_17122.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-12663: Services required by client not available on the server
ORA-36961: Oracle OLAP is not available.
ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
ORA-06512: at line 15
Thu Feb 15 14:33:45 2007
db_recovery_file_dest_size of 5120 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Thu Feb 15 14:33:46 2007
Completed: ALTER DATABASE OPEN
hyperion:/u01/app/oracle/logs# less /u01/app/oracle/admin/tlp/udump/tlp_ora_17122.trc
Node name: hyperion
Release: 2.6.18.1-vs2.1.1-rc48-vserver
Version: #1 SMP PREEMPT Wed Nov 15 16:53:41 CET 2006
Machine: i686
Instance name: tlp
Redo thread mounted by this instance: 1
Oracle process number: 15
Unix process pid: 17122, image: oracle@hyperion (TNS V1-V3)
*** SERVICE NAME:() 2007-02-15 14:33:39.169
*** SESSION ID:(159.7) 2007-02-15 14:33:39.169
Thread 1 checkpoint: logseq 204, block 3873, scn 6263056
cache-low rba: logseq 204, block 63015
on-disk rba: logseq 204, block 66596, scn 6301195
start recovery at logseq 204, block 63015, scn 0
----- Redo read statistics for thread 1 -----
Read rate (ASYNC): 1790Kb in 0.18s => 9.71 Mb/sec
Total physical reads: 4096Kb
Longest record: 20Kb, moves: 0/2889 (0%)
Change moves: 0/32 (0%), moved: 0Mb
Longest LWN: 299Kb, moves: 0/503 (0%), moved: 0Mb
Last redo scn: 0x0000.0060260a (6301194)
----- Recovery Hash Table Statistics ---------
Hash table buckets = 32768
Longest hash chain = 2
Average hash chain = 477/472 = 1.0
Max compares per lookup = 1
Avg compares per lookup = 8228/8877 = 0.9
*** 2007-02-15 14:33:39.382
KCRA: start recovery claims for 477 data blocks
*** 2007-02-15 14:33:39.398
KCRA: blocks processed = 477/477, claimed = 477, eliminated = 0
*** 2007-02-15 14:33:39.403
Recovery of Online Redo Log: Thread 1 Group 2 Seq 204 Reading mem 0
----- Recovery Hash Table Statistics ---------
Hash table buckets = 32768
Longest hash chain = 2
Average hash chain = 477/472 = 1.0
Max compares per lookup = 2
Avg compares per lookup = 7619/8700 = 0.9
Error in executing triggers on database startup
*** 2007-02-15 14:33:45.811
ksedmp: internal or fatal error
ORA-00604: error occurred at recursive SQL level 1
ORA-12663: Services required by client not available on the server
ORA-36961: Oracle OLAP is not available.
ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
ORA-06512: at line 15
Any ideas ?
Thanks
Blue

Your basic problem seems to be you have a trigger, which wants to use OLAP functionality, but OLAP is not installed or not available for whatever reason
Error in executing triggers on database startup <================
*** 2007-02-15 14:33:45.811
ksedmp: internal or fatal error
ORA-00604: error occurred at recursive SQL level 1
ORA-12663: Services required by client not available on the server
ORA-36961: Oracle OLAP is not available.
ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
ORA-06512: at line 15
So check your database startup triggers.
Werner

Terminating the instance due to error 119

System : Oracle 11gR2 running on Oracle VM.
Issue : Oracle fails to comeup after LISTENER.ORA is removed/corrupted. Any reason?
Actions taken : used *.ora files from backup taken during healthy state and dbstart works fine now.
Resolved but Question : Does dbstart depend on *.ora files availability? What is this error 119 (couldnt find any info on that)?
Thanks.

Dear sb92075,
Thanks for confirming :
listener is NOT required to start or access any local Oracle database.
listener.ora file is NOT required to start or use the listener.
Coming to ...
It is really, Really, REALLY difficult to fix a problem that can not be seen.
As said earlier, issue is fixed. Question is why DBSTART works only when proper ORA files are used.
Anyways, here is the info requested :
[oracle@oraclelinux6 bin]$ dbstart
ORACLE_HOME_LISTNER is not SET, unable to auto-start Oracle Net Listener
Usage: /apps/app/oracle/product/11.2.0/dbhome_1/bin/dbstart ORACLE_HOME
Processing Database instance "orcl": log file /apps/app/oracle/product/11.2.0/dbhome_1/startup.log
[oracle@oraclelinux6 bin]$
From startup log :
SQL*Plus: Release 11.2.0.1.0 Production on Mon Dec 2 18:47:47 2013
Copyright (c) 1982, 2009, Oracle. All rights reserved.
SQL> Connected to an idle instance.
SQL> ORA-00119: invalid specification for system parameter LOCAL_LISTENER
ORA-00132: syntax error or unresolved network name 'LISTENER_ORCL'
SQL> Disconnected
/apps/app/oracle/product/11.2.0/dbhome_1/bin/dbstart: Database instance "orcl" warm started.
[oracle@oraclelinux6 bin]$
From trace file :
*** 2013-12-02 18:47:58.236
USER (ospid: 3950): terminating the instance due to error 119
[oracle@oraclelinux6 trace]$
From alert log :
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options.
Using parameter settings in server-side spfile /apps/app/oracle/product/11.2.0/dbhome_1/dbs/spfileorcl.ora
System parameters with non-default values:
processes                = 150
memory_target            = 1584M
control_files            = "/apps/db/orcl/control01.ctl"
control_files            = "/apps/app/oracle/flash_recovery_area/orcl/control02.ctl"
db_block_size            = 8192
compatible               = "11.2.0.0.0"
db_recovery_file_dest    = "/apps/app/oracle/flash_recovery_area"
db_recovery_file_dest_size= 3882M
undo_tablespace          = "UNDOTBS1"
remote_login_passwordfile= "EXCLUSIVE"
db_domain                = "localdomain"
dispatchers              = "(PROTOCOL=TCP) (SERVICE=orclXDB)"
local_listener           = "LISTENER_ORCL"
audit_file_dest          = "/apps/app/oracle/admin/orcl/adump"
audit_trail              = "DB"
db_name                  = "orcl"
open_cursors             = 300
diagnostic_dest          = "/apps/app/oracle"
USER (ospid: 2999): terminating the instance due to error 119
Instance terminated by USER, pid = 2999
Mon Dec 02 18:47:51 2013
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST
Autotune of undo retention is turned on.
IMODE=BR
ILAT =27
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options.
Using parameter settings in server-side spfile /apps/app/oracle/product/11.2.0/dbhome_1/dbs/spfileorcl.ora
System parameters with non-default values:
processes                = 150
memory_target            = 1584M
control_files            = "/apps/db/orcl/control01.ctl"
control_files            = "/apps/app/oracle/flash_recovery_area/orcl/control02.ctl"
db_block_size            = 8192
compatible               = "11.2.0.0.0"
db_recovery_file_dest    = "/apps/app/oracle/flash_recovery_area"
db_recovery_file_dest_size= 3882M
undo_tablespace          = "UNDOTBS1"
remote_login_passwordfile= "EXCLUSIVE"
db_domain                = "localdomain"
dispatchers              = "(PROTOCOL=TCP) (SERVICE=orclXDB)"
local_listener           = "LISTENER_ORCL"
audit_file_dest          = "/apps/app/oracle/admin/orcl/adump"
audit_trail              = "DB"
db_name                  = "orcl"
open_cursors             = 300
diagnostic_dest          = "/apps/app/oracle"
USER (ospid: 3950): terminating the instance due to error 119
Instance terminated by USER, pid = 3950
[oracle@oraclelinux6 trace]$
[oracle@oraclelinux6 trace]$ df -k
Filesystem                          1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg_oraclelinux6-lv_root 15237120   8088772   5709648 59% /
tmpfs                                 5242880       272   5242608   1% /dev/shm
/dev/sda1                              495844    125202    345042 27% /boot
/dev/sdd                             12385456 10527572   1228740 90% /apps
none                                  4194304       108   4194196   1% /tmp
Linux                               353894396 300370788 53523608 85% /media/sf_Linux
/dev/sr0                                54082     54082         0 100% /media/VBOXADDITIONS_4.2.4_81684
[oracle@oraclelinux6 trace]$ ps -ef | grep oracle
avahi     1513     1 0 18:43 ?        00:00:00 avahi-daemon: running [oraclelinux6.local]
oracle    2354     1 0 18:45 ?        00:00:00 /usr/bin/gnome-keyring-daemon --daemonize --login
oracle    2363 2344 0 18:45 ?        00:00:00 gnome-session
oracle    2371     1 0 18:45 ?        00:00:00 dbus-launch --sh-syntax --exit-with-session
oracle    2372     1 0 18:45 ?        00:00:00 /bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
oracle    2435     1 0 18:45 ?        00:00:00 /usr/bin/VBoxClient --clipboard
oracle    2443     1 0 18:45 ?        00:00:00 /usr/bin/VBoxClient --display
oracle    2448     1 0 18:45 ?        00:00:00 /usr/bin/VBoxClient --seamless
oracle    2453     1 0 18:45 ?        00:00:01 /usr/bin/VBoxClient --draganddrop
oracle    2477     1 0 18:45 ?        00:00:00 /usr/libexec/gconfd-2
oracle    2484     1 0 18:45 ?        00:00:00 /usr/libexec/gnome-settings-daemon
oracle    2491     1 0 18:45 ?        00:00:00 /usr/libexec/gvfsd
oracle    2494     1 0 18:45 ?        00:00:00 seahorse-daemon
oracle    2502 2363 0 18:45 ?        00:00:00 metacity --sm-client-id 1072b0438d14c83ab2138561248658457800000023660026
oracle    2506     1 0 18:45 ?        00:00:00 /usr/bin/pulseaudio --start --log-target=syslog
oracle    2511 2363 0 18:45 ?        00:00:00 gnome-panel --sm-client-id 1072b0438d14c83ab2138561248661659600000023660027
oracle    2519 2506 0 18:45 ?        00:00:00 /usr/libexec/pulse/gconf-helper
oracle    2520 2363 0 18:45 ?        00:00:00 nautilus --sm-client-id 1072b0438d14c83ab2138561248673276200000023660028 --sm-client-state-file /home/oracle/.config/session-state/nautilus-1386038529.state
oracle    2522     1 0 18:45 ?        00:00:00 /usr/libexec/bonobo-activation-server --ac-activate --ior-output-fd=18
oracle    2528     1 0 18:45 ?        00:00:00 /usr/libexec/wnck-applet --oaf-activate-iid=OAFIID:GNOME_Wncklet_Factory --oaf-ior-fd=18
oracle    2530     1 0 18:45 ?        00:00:00 /usr/libexec/trashapplet --oaf-activate-iid=OAFIID:GNOME_Panel_TrashApplet_Factory --oaf-ior-fd=24
oracle    2532     1 0 18:45 ?        00:00:00 /usr/libexec/gvfs-gdu-volume-monitor
oracle    2538     1 0 18:45 ?        00:00:00 /usr/libexec/gvfsd-trash --spawner :1.7 /org/gtk/gvfs/exec_spaw/0
oracle    2544 2363 0 18:45 ?        00:00:00 gnome-volume-control-applet
oracle    2545 2363 0 18:45 ?        00:00:00 /usr/libexec/polkit-gnome-authentication-agent-1
oracle    2546 2363 0 18:45 ?        00:00:00 gpk-update-icon
oracle    2551     1 0 18:45 ?        00:00:00 /usr/libexec/gvfs-gphoto2-volume-monitor
oracle    2605 2363 0 18:45 ?        00:00:00 abrt-applet
oracle    2606 2363 0 18:45 ?        00:00:00 nm-applet --sm-disable
oracle    2607 2363 0 18:45 ?        00:00:00 gnome-power-manager
oracle    2608 2363 0 18:45 ?        00:00:00 bluetooth-applet
oracle    2611     1 0 18:45 ?        00:00:00 /usr/libexec/gvfs-afc-volume-monitor
oracle    2612 2363 0 18:45 ?        00:00:00 /usr/libexec/gdu-notification-daemon
oracle    2614 2363 0 18:45 ?        00:00:00 python /usr/share/system-config-printer/applet.py
oracle    2625     1 0 18:45 ?        00:00:00 gnome-screensaver
oracle    2627     1 0 18:45 ?        00:00:00 /usr/libexec/im-settings-daemon
oracle    2780     1 0 18:45 ?        00:00:00 /usr/libexec/gconf-im-settings-daemon
oracle    2786     1 0 18:45 ?        00:00:00 /usr/libexec/clock-applet --oaf-activate-iid=OAFIID:GNOME_ClockApplet_Factory --oaf-ior-fd=19
oracle    2788     1 0 18:45 ?        00:00:00 /usr/libexec/gdm-user-switch-applet --oaf-activate-iid=OAFIID:GNOME_FastUserSwitchApplet_Factory --oaf-ior-fd=28
oracle    2790     1 0 18:45 ?        00:00:00 /usr/libexec/notification-area-applet --oaf-activate-iid=OAFIID:GNOME_NotificationAreaApplet_Factory --oaf-ior-fd=34
oracle    2792     1 0 18:45 ?        00:00:00 /usr/bin/gnote --panel-applet --oaf-activate-iid=OAFIID:GnoteApplet_Factory --oaf-ior-fd=40
oracle    2837     1 0 18:45 ?        00:00:00 /usr/libexec/gvfsd-burn --spawner :1.7 /org/gtk/gvfs/exec_spaw/1
oracle    2839     1 0 18:45 ?        00:00:00 /usr/libexec/gvfsd-metadata
oracle    3863     1 0 18:46 ?        00:00:01 gnome-terminal
oracle    3864 3863 0 18:46 ?        00:00:00 gnome-pty-helper
oracle    3865 3863 0 18:46 pts/0    00:00:00 bash
oracle    3881 3863 0 18:46 pts/1    00:00:00 bash
oracle    3896 3863 0 18:46 pts/2    00:00:00 bash
oracle    4012 3896 0 18:54 pts/2    00:00:00 ps -ef
oracle    4013 3896 0 18:54 pts/2    00:00:00 grep oracle
[oracle@oraclelinux6 trace]$

PMON (ospid: 8143): terminating the instance due to error 472

Hi Friends,
My Environment_
OS : oel4u5
Apps Version : R12.1.1
DB Version : *11.1.0.7*
When i am trying to start my ebiz 11g database the db started and its automatically going down because of PMON termination. When am looking in to the alert log files its showing the below ora-600 errors,
Resuming block recovery (PMON) for file 4 block 180627
Block recovery from logseq 125, block 70 to scn 10132192035480
Recovery of Online Redo Log: Thread 1 Group 2 Seq 125 Reading mem 0
Mem# 0: /ebiz/oracle/db/apps_st/data/log2.dbf
Block recovery completed at rba 125.281.16, scn 2359.364184220
Errors in file /ebiz/oracle/db/tech_st/11.1.0/admin/TEST05_erptest05/diag/rdbms/test05/TEST05/trace/TEST05_pmon_8143.trc (incident=49647):
ORA-00600: internal error code, arguments: [4194], [41], [37], [], [], [], [], [], [], [], [], []
Errors in file /ebiz/oracle/db/tech_st/11.1.0/admin/TEST05_erptest05/diag/rdbms/test05/TEST05/trace/TEST05_pmon_8143.trc:
ORA-00600: internal error code, arguments: [4194], [41], [37], [], [], [], [], [], [], [], [], []
PMON (ospid: 8143): terminating the instance due to error 472
Instance terminated by PMON, pid = 8143
Following error which i found in the trace file /ebiz/oracle/db/tech_st/11.1.0/admin/TEST05_erptest05/diag/rdbms/test05/TEST05/trace/TEST05_pmon_8143.trc
(call) sess: cur 5f1efd10, rec 5f1e5ff0, usr 5f1efd10; depth: 0
ksudlc FALSE at location: 6
(k2g table)
error 472 detected in background process
ORA-00600: internal error code, arguments: [4194], [41], [37], [], [], [], [], [], [], [], [], []
*** 2011-11-27 20:19:33.113
PMON (ospid: 8143): terminating the instance due to error 472
Kindly Suggest me,
Thanks
Athish

Hi Helios,
We identified that the error was in undo corruption and We followed the below steps to over come above mention error.
Steps as followed,
Step 1:
SQL> SELECT name,value FROM v$parameter WHERE name IN ('undo_management','undo_tablespace');
NAME VALUE
undo_management MANUAL
undo_tablespace UNDO_TBS
Step2:
SQL> select FILE_NAME, TABLESPACE_NAME from dba_data_files where TABLESPACE_NAME like 'UNDO%';
FILE_NAME                         TABLESPACE_NAME
/ebiz/oracle/db/apps_st/data/undotbs_02.dbf     UNDO_TBS
/ebiz/oracle/db/apps_st/data/undotbs_01.dbf     UNDO_TBS
Step 3: Create a new undo tablespace
SQL> create UNDO tablespace UNDOTBS datafile '/ebiz/oracle/db/apps_st/data/undotbs01.dbf' size 1024m REUSE AUTOEXTEND ON NEXT 4096K MAXSIZE 1024M;
Tablespace created.
Step 4:
SQL> ALTER SYSTEM SET undo_tablespace = 'UNDOTBS' scope=spfile;
System altered.
Step 5: set old undo tablespace offine mode and drop
SQL> ALTER TABLESPACE UNDO_TBS offline;
Tablespace altered.
SQL> drop tablespace UNDO_TBS including contents and datafiles;
Tablespace dropped.
Step 6:
Rebounced the db services
Step 7: Changed the undo management parameter to AUTO
SQL> alter system set undo_management='AUTO' scope=spfile;
System altered.
SQL> SELECT name,value FROM v$parameter WHERE name IN ('undo_management','undo_tablespace');
NAME VALUE
undo_management AUTO
undo_tablespace UNDOTBS
Now the database is up and running with no issue and we cant find any ora error in the alert log file,
ThaNks
Athish

One of out DB while starting gives error ORA-03113

One of out DB while starting gives error ORA-03113. Need some inputs to resolve this issue.
SQL> startup
ORACLE instance started.
Total System Global Area 603979776 bytes
Fixed Size                  1250380 bytes
Variable Size             163580852 bytes
Database Buffers          432013312 bytes
Redo Buffers                7135232 bytes
Database mounted.
ORA-03113: end-of-file on communication channelRegards,
Lokanath

Please find the dts.
Mon Jan 09 16:12:41 2012
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
starting up 1 shared server(s) ...
Mon Jan 09 16:12:41 2012
alter database mount exclusive
Mon Jan 09 16:12:48 2012
Setting recovery target incarnation to 2
Mon Jan 09 16:12:49 2012
Successful mount of redo thread 1, with mount id 3125175737
Mon Jan 09 16:12:49 2012
Database mounted in Exclusive Mode
Completed: alter database mount exclusive
Mon Jan 09 16:12:49 2012
alter database open
Mon Jan 09 16:12:51 2012
Rolling back half complete log switch of thread 1
Mon Jan 09 16:12:52 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_lgwr_5024.trc:
ORA-00202: control file: 'D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL01.CTL'
ORA-27091: unable to queue I/O
ORA-27069: attempt to do I/O beyond the range of the file
OSD-04026: Invalid parameter passed. (OS 432)
Mon Jan 09 16:12:56 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_lgwr_5024.trc:
ORA-00204: error in reading (block 432, # blocks 1) of control file
ORA-00202: control file: 'D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL01.CTL'
ORA-27091: unable to queue I/O
ORA-27069: attempt to do I/O beyond the range of the file
OSD-04026: Invalid parameter passed. (OS 432)
LGWR: terminating instance due to error 204
Mon Jan 09 16:12:58 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_dbw0_5264.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:00 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_pmon_3784.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:01 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_psp0_4388.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:01 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_mman_4828.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:01 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_ckpt_2604.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:01 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_reco_4492.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:02 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_smon_4484.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 16:13:03 2012
Instance terminated by LGWR, pid = 5024
Mon Jan 09 16:15:01 2012
WARNING: inbound connection timed out (ORA-3136)
Mon Jan 09 16:29:35 2012
WARNING: inbound connection timed out (ORA-3136)
Mon Jan 09 16:55:01 2012
WARNING: inbound connection timed out (ORA-3136)
Mon Jan 09 17:06:50 2012
WARNING: inbound connection timed out (ORA-3136)
Mon Jan 09 17:07:52 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_10 parameter default value as USE_DB_RECOVERY_FILE_DEST
Autotune of undo retention is turned on.
IMODE=BR
ILAT =18
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
processes                = 150
__shared_pool_size       = 155189248
__large_pool_size        = 4194304
__java_pool_size         = 4194304
__streams_pool_size      = 0
sga_target               = 603979776
control_files            = D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL01.CTL, D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL02.CTL, D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL03.CTL
db_block_size            = 8192
__db_cache_size          = 432013312
compatible               = 10.2.0.1.0
db_file_multiblock_read_count= 16
db_recovery_file_dest    = D:\oracle\product\10.2.0/flash_recovery_area
db_recovery_file_dest_size= 2147483648
undo_management          = AUTO
undo_tablespace          = UNDOTBS1
remote_login_passwordfile= EXCLUSIVE
db_domain                =
dispatchers              = (PROTOCOL=TCP) (SERVICE=ecuprogXDB)
job_queue_processes      = 10
audit_file_dest          = D:\ORACLE\PRODUCT\10.2.0\ADMIN\ECUPROG\ADUMP
background_dump_dest     = D:\ORACLE\PRODUCT\10.2.0\ADMIN\ECUPROG\BDUMP
user_dump_dest           = D:\ORACLE\PRODUCT\10.2.0\ADMIN\ECUPROG\UDUMP
core_dump_dest           = D:\ORACLE\PRODUCT\10.2.0\ADMIN\ECUPROG\CDUMP
db_name                  = ecuprog
open_cursors             = 300
pga_aggregate_target     = 201326592
PMON started with pid=2, OS id=4832
PSP0 started with pid=3, OS id=4208
MMAN started with pid=4, OS id=3216
DBW0 started with pid=5, OS id=4020
LGWR started with pid=6, OS id=4728
CKPT started with pid=7, OS id=4860
SMON started with pid=8, OS id=2932
RECO started with pid=9, OS id=3772
CJQ0 started with pid=10, OS id=5244
MMON started with pid=11, OS id=4568
MMNL started with pid=12, OS id=5828
Mon Jan 09 17:07:53 2012
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
starting up 1 shared server(s) ...
Mon Jan 09 17:07:53 2012
ALTER DATABASE   MOUNT
Mon Jan 09 17:07:58 2012
Setting recovery target incarnation to 2
Mon Jan 09 17:07:59 2012
Successful mount of redo thread 1, with mount id 3125157033
Mon Jan 09 17:07:59 2012
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Mon Jan 09 17:07:59 2012
ALTER DATABASE OPEN
Mon Jan 09 17:08:01 2012
Rolling back half complete log switch of thread 1
Mon Jan 09 17:08:02 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_lgwr_4728.trc:
ORA-00202: control file: 'D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL01.CTL'
ORA-27091: unable to queue I/O
ORA-27069: attempt to do I/O beyond the range of the file
OSD-04026: Invalid parameter passed. (OS 432)
Mon Jan 09 17:08:05 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_lgwr_4728.trc:
ORA-00204: error in reading (block 432, # blocks 1) of control file
ORA-00202: control file: 'D:\ORACLE\PRODUCT\10.2.0\ORADATA\ECUPROG\CONTROL01.CTL'
ORA-27091: unable to queue I/O
ORA-27069: attempt to do I/O beyond the range of the file
OSD-04026: Invalid parameter passed. (OS 432)
LGWR: terminating instance due to error 204
Mon Jan 09 17:08:06 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_dbw0_4020.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 17:08:07 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_pmon_4832.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 17:08:07 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_reco_3772.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 17:08:07 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_smon_2932.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 17:08:07 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_ckpt_4860.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 17:08:09 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_psp0_4208.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Mon Jan 09 17:08:09 2012
Errors in file d:\oracle\product\10.2.0\admin\ecuprog\bdump\ecuprog_mman_3216.trc:
ORA-00204: error in reading (block , # blocks ) of control file
Instance terminated by LGWR, pid = 4728

DR Server not started generating errors

Dear All
i want to start dr server
but it generating error
SQL*Plus: Release 10.2.0.2.0 - Production on Wed Oct 22 09:58:18 2008
Copyright (c) 1982, 2005, Oracle. All Rights Reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 1526726656 bytes
Fixed Size                  2053752 bytes
Variable Size             771754376 bytes
Database Buffers          738197504 bytes
Redo Buffers               14721024 bytes
Database mounted.
ORA-01589: must use RESETLOGS or NORESETLOGS option for database open
SQL> shutdown
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.
Total System Global Area 1526726656 bytes
Fixed Size                  2053752 bytes
Variable Size             771754376 bytes
Database Buffers          738197504 bytes
Redo Buffers               14721024 bytes
Database mounted.
SQL> select OPEN_RESETLOGS from v$database;
OPEN_RESETL
REQUIRED
SQL> alter database open resetlogs;
alter database open resetlogs
ERROR at line 1:
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'G:\ORACLE\PRD\SAPDATA1\SYSTEM_1\SYSTEM.DATA1'
SQL>
Please tell how can i resolve that issue........ i want to up DR Immediatly
regards

Hi
execute the below commands in mount mode
SQL>recover database;
then
SQL> alter database open resetlogs;
Raj

Instance start up error..

Similar Messages

Maybe you are looking for