Huge user trace files being generated under udump

last few days one of my database is generating huge number of user trace files under udump. I tried to investigate in alert.log file. But as I am not very expert to analyze the alert.log file I could not find the cause behind it. Only the ORA-00600: internal error code is found there. But it is a generic error . Please help.

Thank you again. I got the clue from your reply.
The error message is pasted below in short.
ORA-00600: internal error code, arguments: [12333], [0], [0], [0], [], [], [], []
Current SQL statement for this session:
BEGIN :1 := TESTDBA.LOCAL_SYSDATE_SFT_FNC( 1,:2,:3 ); END;
And it is obvious that the function had some problem recently. We need to investigate on that. Now my request is
Please guide me how to check in metalink with the value '12333' which came as argument of ORA-00600 error. Here my job was easy as I got the problem function name . By I want to know the art of analysing the error message in alert.log file.
Also tell me which places I should check regularly as a responsibility of a DBA ?
What are the common errors in Alert log file? Where I should be cautioned ? Where I should ignore. Give me some tips please. Do not restrict your discussion on Alret.log only . Also please cover all important areas. If you think I am taking much of your time then please send me some link or source of PDF etc.

Similar Messages

Numerous trace files are generating every minute causing space issue

Hi All,
numerous trace files are generating every minute <SID>_<PID>_APPSPERF01.trc format.
entry in trace file will be like..
EXEC #10:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,plh=1734896627,tim=1339571764486430
WAIT #10: nam='SQL*Net message to client' ela= 6 driver id=1952673792 #bytes=1 p3=0 obj#=34562 tim=1339571764491273
FETCH #10:c=0,e=0,p=0,cr=2,cu=0,mis=0,r=1,dep=0,og=1,plh=1734896627,tim=1339571764486430
WAIT #10: nam='SQL*Net message from client' ela= 277 driver id=1952673792 #bytes=1 p3=0 obj#=34562 tim=1339571764491806
EXEC #11:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,plh=2638510909,tim=1339571764486430
FETCH #11:c=0,e=0,p=0,cr=9,cu=0,mis=0,r=0,dep=0,og=1,plh=2638510909,tim=1339571764486430
WAIT #11: nam='SQL*Net message to client' ela= 6 driver id=1952673792 #bytes=1 p3=0 obj#=34562 tim=1339571764493265
*** 2012-06-13 03:16:14.496
WAIT #11: nam='SQL*Net message from client' ela= 10003326 driver id=1952673792 #bytes=1 p3=0 obj#=34562 tim=1339571774496705
BINDS #10:
Bind#0
oacdty=01 mxl=32(21) mxlc=00 mal=00 scl=00 pre=00
oacflg=00 fl2=1000001 frm=01 csi=871 siz=2064 off=0
kxsbbbfp=2b8ec799df38 bln=32 avl=03 flg=05
value="535"
Bind#1
oacdty=01 mxl=32(21) mxlc=00 mal=00 scl=00 pre=00
oacflg=00 fl2=1000001 frm=01 csi=871 siz=0 off=32
kxsbbbfp=2b8ec799df58 bln=32 avl=04 flg=01
value="1003"
SQL> show parameter trace
NAME TYPE VALUE
tracefiles_public boolean TRUE
log_archive_trace integer 0
sec_protocol_error_trace_action string TRACE
sql_trace boolean FALSE
trace_enabled boolean TRUE
tracefile_identifier string
Profile options like "FND:Debug Log Enabled" and "Utilities:SQL Trace" are set to No
Can some one help me to stop these trace generation.
is there any way to find the cause for these trace?
Thanks in adv...

Hi;
Please check who enable trace. Please see:
How to audit users who enabled traces?
check concurrent programs first
*from the screen
*F11, then select the trace, then Ctrl+F11
Concurrent > program > define
open the form, press F11 (query mode), select the trace, then (ctrl + f11) this should return all concurrent programs which have trace enabled
Regard
Helios

Sqlnet.ora trace files getting generated even after turning off tracing

Hi,
I have recently added the following parameters to the sqlnet.ora file.
TRACE_LEVEL_SERVER=16
TRACE_FILE_SERVER=SERVER
TRACE_DIRECTORY_SERVER=/ftpland/trace
TRACE_TIMESTAMP_SERVER=on
Even after removing these enteries from the sqlnet.ora I still see the trave files being generated.
#####enable tracing###################
#Trace_level_server=0
#Trace_filelen_server=1000
#Trace_fileno_server=1
#Trace_timestamp_server=on
#Trace_directory_server=/opt/oracle/product/10.2.0/network/trace
#Diag_adr_enabled=off
AUTOMATIC_IPC = ON
TRACE_LEVEL_CLIENT = OFF
SQLNET.EXPIRE_TIME = 10
NAMES.DEFAULT_DOMAIN = bsca.eds.com
NAME.DEFAULT_ZONE = bsca.eds.com
SQLNET.CRYPTO_SEED = "232166927-1713903352"
NAMES.DIRECTORY_PATH = (ONAMES,TNSNAMES)
NAMES.PREFERRED_SERVERS =
      (ADDRESS_LIST =
        (ADDRESS =
          (COMMUNITY = TCP.bsca.eds.com)
          (PROTOCOL = TCP)
          (Host = oraclenames1.bsca.eds.com)
          (Port = 1575)
        (ADDRESS =
          (COMMUNITY = TCP.bsca.eds.com)
          (PROTOCOL = TCP)
          (Host = oraclenames2.bsca.eds.com)
          (Port = 1575)
NAME.PREFERRED_SERVERS =
      (ADDRESS_LIST =
        (ADDRESS =
          (COMMUNITY = TCP.bsca.eds.com)
          (PROTOCOL = TCP)
          (Host = oraclenames1.bsca.eds.com)
          (Port = 1575)
(ADDRESS =
          (COMMUNITY = TCP.bsca.eds.com)
          (PROTOCOL = TCP)
          (Host = oraclenames2.bsca.eds.com)
          (Port = 1575)
BEQUEATH_DETACH=YES
Regards,
VN

SID_LIST_LISTENER =
(SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME =ROSDMP.bsca.eds.com)
      (ORACLE_HOME = /opt/oracle/product/10.2.0)
      (SID_NAME = ROSDMP)
TRACE_LEVEL_LISTENER=16
I believe, this is the reason, you are seeing trace files even after disabling it in sqlnet.ora

Trace File STAT generated on J2EE

Hi everyone,
I am having problem with trace file called STAT. This file is being generated but I Don´t now by who.
They are being recorded on the file system sapdata3.
The system is J2EE SP15, XI 3.0 with Oracle Database!
Thanks

Hi Ricardo,
The stat files being generated by the CCMS Agent. Check
the link for more
http://help.sap.com/saphelp_erp2005/helpdata/en/c7/69bcc3f36611d3a6510000e835363f/frameset.htm
Regards
Srikishan

Trace file getting generated

Hi,
We have PI 7.1 installed in landscape. Some trace files getting generated at
server\sapmnt\SID\DVEBMGS01\j2ee\cluster\server0\jrfc07500_06148.trc and utilizing more disk space aroung 1 GB.
Could you please let me know from where trace can be disabled
thanks

Hi Yash,
Please find the details on this link:
http://help.sap.com/saphelp_nw04/Helpdata/EN/f6/daea401675752ae10000000a155106/content.htm
name
jrfc*.trc
location
directory j2ee/cluster/server* or defined path
how to switch on and off set JVM options for the server via the Config Tool and restart server: -Djrfc.trace=0/1, -Djco.trace_path=[defined_path]
Kindly let us know if this resolves the problem.
Thanks.
Regards,
Shweta

How to read/understand an user Trace Files

Dear All,
I am new to this forum, could any help me how to read/understand User Trace File
Thanks,
Kannan Pasupathi
Edited by: 835519 on Feb 10, 2011 2:40 AM

Hi,
http://hoopercharles.wordpress.com/2009/12/01/10046-extended-sql-trace-interpretation/
http://hoopercharles.wordpress.com/2010/01/26/10046-extended-sql-trace-interpretation-2/
http://hoopercharles.wordpress.com/2010/09/06/10046-extended-sql-trace-interpretation-3/
HTH
Anand

Migration from windows to Linux - JAR files not generating under JAVA_TOP

Hi DBA's
Jar files not generating under JAVA_TOP after migration from windows to linux.
I followed workaround but same in result.
Workaround
Run this command. It will give the missing file
adjava -mx512m -nojit oracle.apps.ad.jri.adjcopy -masterArchive $JAVA_TOP -sync -reportfile javatopfiles.lst
Then run this command
adjava -mx512m -nojit oracle.apps.ad.jri.adjcopy -masterArchive JAVA_TOP -sync -mode APPLY
I have checked in google.
Best Regards,
Suresh Gelda

Suresh,
I can find the following error in the log file:
JarSigner error output:
Exception in thread "main" java.lang.NoClassDefFoundError: sun/security/tools/JarSigner
Caused by: java.lang.ClassNotFoundException: sun.security.tools.JarSigner
     at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
     at java.security.AccessController.doPrivileged(Native Method)
     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
     at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: sun.security.tools.JarSigner. Program will exit.Please review the following notes for suggested solutions:
Note: 393306.1 - Generate Jar File causes NoClassDefFoundError sun/security/tools/JarSigner
https://metalink2.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=393306.1
Note: 269054.1 - Running adadmin to Force Regenerate of the Java Files Fails
https://metalink2.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=269054.1

System State dumped to trace file e:\oracle\admin\udump\rbidev_ora_2864.trc

System State dumped to trace file e:\oracle\admin\udump\rbidev_ora_2864.trc
What should i do to resolve this issue?
Database : 10.2.0.2.0
OS version : Microsoft Windows 2003 Enterprise Server Service Pack 2 ( 32-bit)

Ankit,
These errors come from multiple reasons.Most of them areeither due to some bugs which are in Oracle code.Now I am sure you would agree that Oraclewont publish the details of the code so that dbas can do something about the errors.Moreover,just because Oracle wants dbas to work upon these errors and troubleshoot them,oracle has made lookup tools in Metalink specifically so that dbas can look there and work upon these errors.
You are sayig that Oracle has not published error information.There is so much about working of Oracle which we don't know and chances are very less that will ever come to know.
Just my 2 cents to make the point clear.
Aman....

Trace file not Generated.

Hey Guys,
I am using oracle 10g and trying to generate a trace file.
Using sql_Plus I can see that timed_statistics is set to true, max_dump_file_size is set to unlimited, and user_dump_dest is set to C:\ORACLE\PRODUCT\10.2.0\ADMIN\ORCL\DUMP.
In Oracle SQL Developer I am running the script:
alter session set sql_trace = true;
... PL/SQL block that I want to trace...
alter session set sql_trace = false;
After this sql runs without error there is no file on my computer at the user_dump_dest. In fact the path under user_dump_dest does not even exist. Do I have to create the path first? Am I looking in the wrong location? Am I doing something wrong when setting sql_trace?
Thanks,
Ian

In fact the path under user_dump_dest does not even exist.Perhaps this is the source of the mystery.
If Oracle has no place to actually write the file, how should it proceed?
as a proof of concept do the following
ALTER SESSION SET SQL_TRACE=TRUE;
show parameter dump
SELECT SYSDATE FROM DUAL;
ALTER SESSION SET SQL_TRACE=FALSE;
now find the trace file
post results from session above
Edited by: sb92075 on Jun 1, 2010 4:45 PM

Session trace files not generated

Hai all,
Am tracing a session as follows :
EXEC DBMS_SUPPORT.start_trace_in_session(sid=>2166, serial=>30629, waits=>TRUE, binds=>FALSE);
EXEC DBMS_SUPPORT.stop_trace_in_session(sid=>2166, serial=>30629);
but,no trace files are getting generated ?.. in udump dir..
Any idea ?
Kai

Likely a version and/or platform specific bug.
However, according to you this information is not important, or too much cumbersome to type, so no one can help you, even if you were told to include this several time.
If you want free support from volunteers, at least provide information to work in.
Or consider not posting at all.
Sybrand Bakker
Senior Oracle DBA

Where is the user trace file?

I have issued the statement below to enable sql trace in session 260 ,and it execute successfully.but i can not find where the trace file in directory $ORACLE_BASE/admin/testcs/udump (testcs is my database name).
SQL> execute dbms_system.set_sql_trace_in_session(260,34880,true);
PL/SQL procedure successfully completed.
where is the trace file?
By the way,how can i find the information about the package DBMS_SYSTEM which is not in ORACLE references?

Did you issue some queries after turning tracing on? The trace file might not be created until there is some activity in the session.
I know that DBMS_SYSTEM is documented in the O'Reilly Press book "Oracle Built-in Packages" by Feuerstein, Dye & Beresniewicz.
However for tracing I think that Oracle expects you to use the dbms_support package:
execute sys.dbms_support.start_trace_in_session -
(sid => &sid, serial => &serial, waits => true, binds => true)Or for Oracle 7.3.4 or 8.0 you can use oradebug. Example:
select pid, spid, username from v$process;
PID SPID USERNAME
8 25807 oracle
SVRMGR> oradebug setorapid 8
-- or -
SVRMGR> oradebug setospid 25807
-- turn on
SVRMGR> oradebug event 10046 trace name context forever, level 12
-- turn off
SVRMGR> oradebug event 10046 trace name context off

How to proceed further once the explain plan and trace files are generated?

Hi Friends,
I need to improve the performance of on of the views that i am working on.
As suggested in the thread - http://forums.oracle.com/forums/thread.jspa?threadID=863295&tstart=0 , i gave generated the explain plan and the trace file.
From the explain plan, we can see the expensive operations for the query.
Can any one please tell, how to proceed further from here on i.e. how to make this expensive operations less expensive?
For ex: FULL TABLE SCAN might be an expensive operation when the table has indexes.In such cases, how can we avoid such operations to make query faster?
Regards,
Sreekanth Munagala.

Hi Veena,
An earlier post by you regarding P45 is as below
Starter report P45(3) / P46 efiling for UK
from my understanding though i have not worked on GB Payroll you have said that you deleted IT 65 details of leaver,however there must be clusters generated in system from where the earlier data needs to be deleted and may be that is why you are facing the issue.
In Indian payroll when we execute text file for efiling of tax after challan mapping all the data compiles and sits in PCL cluster and therefore we are unable to generate form 16 with proper output,here we delete the clusters and rerun again the mappings and then check form 16.
Hope this might help you,Experts have suggested you earlier also,they may correct me for this.
Salil

Huge nsa trace files in 11.2

Dear all,
11.2.2 rac one node on windows 2008. We are using Physical standby for this too. We have huge generation of trace files in the rdbms/trace directory with the below name
medtdw_2_nsa2_7644
contents of the file are :
Trace file C:\APP\ORAADMIN\diag\rdbms\medtdw\medtdw_2\trace\medtdw_2_nsa2_7644.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
Windows NT Version V6.1 Service Pack 1
CPU                 : 16 - type 8664, 8 Physical Cores
Process Affinity    : 0x0x0000000000000000
Memory (Avail/Total): Ph:8426M/16373M, Ph+PgF:23293M/32745M
Instance name: medtdw_2
Redo thread mounted by this instance: 2
Oracle process number: 61
Windows thread id: 7644, image: ORACLE.EXE (NSA2)
*** 2012-02-24 10:08:11.265
*** SESSION ID:(642.3) 2012-02-24 10:08:11.265
*** CLIENT ID:() 2012-02-24 10:08:11.265
*** SERVICE NAME:() 2012-02-24 10:08:11.265
*** MODULE NAME:() 2012-02-24 10:08:11.265
*** ACTION NAME:() 2012-02-24 10:08:11.265
*** 2012-02-24 10:08:11.265 862 krsw.c
NSA2: initializing for LGWR communication
NSA2: connecting to KSR channel
Success
NSA2: subscribing to KSR channel
Success
*** 2012-02-24 10:08:11.265 907 krsw.c
NSA2: initialized successfully ASYNC=1
*** 2012-02-24 10:08:24.260
NSA2: receiving message on its channel
No NET_TIMEOUT has been specified - will use default network timeout
...Successfully Received Message from KSR
Received message opcodes(11,11) and status=0
*** 2012-02-24 10:08:24.260 1277 krsw.c
Message 1 has 1 parts
Destination is specified with ASYNC=61440
*** 2012-02-24 10:08:24.260 7291 krsw.c
krsw_log_switch_wait: log switch in progress
*** 2012-02-24 10:08:24.369 2116 krsa.c
Requesting CFTXN 0x0000000055D2D548 noenqueue (last PID:7656) [krsw.c:5185] IX0
*** 2012-02-24 10:08:28.316
*** 2012-02-24 10:08:28.316 2133 krsa.c
Started    CFTXN 0x0000000055D2D548 noenqueue (PID:7644) [krsw.c:5185] IX+
Retrieving log 3
*** 2012-02-24 10:08:28.316 5212 krsw.c
Sending online log thread 2 seq 422 [logfile 3] to standby
Opening logfile [logno 3]
*** 2012-02-24 10:08:28.331 4654 krsw.c
Acquiring Async LNS enqueue thread 2 seq 422 dest 2 mode 4
*** 2012-02-24 10:08:28.331 4674 krsw.c
Acquired Async LNS enqueue thread 2 seq 422 dest 2 mode 4 [id1 131074 id2 422]
Clearing FARC_FINAL/FARC_EOR by Lgwr archive NotiFYer routine
krsd_init_sdips: called from [krsl.c:915] with function 5
Log switch from thread 2 sequence 421 to sequence 422
Changing dest 1 PDI dstvno from 0 to 2 [krsd.c:6072]
Changing [krsd.c:6097] dest 1 stavno from 0 to 2
Setting dest 1 KCRRDROP [krsd.c:6107]
Changing dest 2 PDI dstvno from 0 to 2 [krsd.c:6072]
Changing [krsd.c:6097] dest 2 stavno from 0 to 2
Setting dest 2 KCRRDROP [krsd.c:6107]
Changing dest 3 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 3 stavno from 0 to 1
Setting dest 3 KCRRDROP [krsd.c:6107]
Changing dest 4 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 4 stavno from 0 to 1
Setting dest 4 KCRRDROP [krsd.c:6107]
Changing dest 5 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 5 stavno from 0 to 1
Setting dest 5 KCRRDROP [krsd.c:6107]
Changing dest 6 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 6 stavno from 0 to 1
Setting dest 6 KCRRDROP [krsd.c:6107]
Changing dest 7 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 7 stavno from 0 to 1
Setting dest 7 KCRRDROP [krsd.c:6107]
Changing dest 8 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 8 stavno from 0 to 1
Setting dest 8 KCRRDROP [krsd.c:6107]
Changing dest 9 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 9 stavno from 0 to 1
Setting dest 9 KCRRDROP [krsd.c:6107]
Changing dest 10 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 10 stavno from 0 to 1
Setting dest 10 KCRRDROP [krsd.c:6107]
Changing dest 11 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 11 stavno from 0 to 1
Setting dest 11 KCRRDROP [krsd.c:6107]
Changing dest 12 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 12 stavno from 0 to 1
Setting dest 12 KCRRDROP [krsd.c:6107]
Changing dest 13 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 13 stavno from 0 to 1
Setting dest 13 KCRRDROP [krsd.c:6107]
Changing dest 14 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 14 stavno from 0 to 1
Setting dest 14 KCRRDROP [krsd.c:6107]
Changing dest 15 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 15 stavno from 0 to 1trace files are getting very huge and reaching 80Gb too.. what are these files ? any clue ?
Kai

Handle: KaiS
Email: [email protected]
Status Level: Newbie (10)
Registered: Nov 20, 2007
Total Posts: 1,353
Total Questions: 574 (434 unresolved)
You have very bad profile status, please bring down the number of pending thread and mark then answered if possible...Please read following
https://forums.oracle.com/forums/ann.jspa?annID=718
Now coming backup to your problem, it seems that you have enabled the tracing of archive..Please read following metalink document which applies to your case...
How To Avoid Many Trace Files With KCRRDROP [ID 1150056.1]
set log_archive_trace=0, if its current value is higher...
Bye

Gather Schema Statistics produces HUGE oracle trace file

Hi
I am running gather schema stats on my E-Biz 12.1.3 / 11.2.0.3 system
Each week we run Gather Schema Stats
I have noticed that when the job completes a 3g trace fuile is produced under the Oracle RDBMS home
I have tried un-checking the box when the COnc Request is submitted to DO NOT SAVE OUTPUT FILE but it is still procuced.
I dont want it and do not understand why it is produced. Anyone out there have a resolution?

I have noticed that when the job completes a 3g trace fuile is produced under the Oracle RDBMS homeWhat is the file name/location? What is the contents of this trace file?
I have tried un-checking the box when the COnc Request is submitted to DO NOT SAVE OUTPUT FILE but it is still procuced.Do you have any debug/trace enabled on HRMS schema?
I dont want it and do not understand why it is produced. Anyone out there have a resolution?Was this working before? If yes, any changes been done since then?
What are the parameters you use to run "Gather Schema Statistics" concurrent program?
Thanks,
Hussein

Trace files being truncated after 18mb

I have enabled trace file debugging for my application and after the trace file has reached approx 18mb, it stops writing to the file. Im also doing logging and the log file is usually around 80mb. Is there some sort of maximum file size restriction? How do I fix this?

You gave almost no details but as a general statement there are no limits on file sizes, at least with the size of files you're dealing with. Some O/S's and file systems might have an issue after 2GB.
What is your application? What kind of logging does it use (i.e. log4j, jdk logging, etc). What O/S are you on? How much free disk space do you have?

Huge user trace files being generated under udump

Similar Messages

Maybe you are looking for