Huge nsa trace files in 11.2

Dear all,
11.2.2 rac one node on windows 2008. We are using Physical standby for this too. We have huge generation of trace files in the rdbms/trace directory with the below name
medtdw_2_nsa2_7644
contents of the file are :
Trace file C:\APP\ORAADMIN\diag\rdbms\medtdw\medtdw_2\trace\medtdw_2_nsa2_7644.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
Windows NT Version V6.1 Service Pack 1
CPU                 : 16 - type 8664, 8 Physical Cores
Process Affinity    : 0x0x0000000000000000
Memory (Avail/Total): Ph:8426M/16373M, Ph+PgF:23293M/32745M
Instance name: medtdw_2
Redo thread mounted by this instance: 2
Oracle process number: 61
Windows thread id: 7644, image: ORACLE.EXE (NSA2)
*** 2012-02-24 10:08:11.265
*** SESSION ID:(642.3) 2012-02-24 10:08:11.265
*** CLIENT ID:() 2012-02-24 10:08:11.265
*** SERVICE NAME:() 2012-02-24 10:08:11.265
*** MODULE NAME:() 2012-02-24 10:08:11.265
*** ACTION NAME:() 2012-02-24 10:08:11.265
*** 2012-02-24 10:08:11.265 862 krsw.c
NSA2: initializing for LGWR communication
NSA2: connecting to KSR channel
Success
NSA2: subscribing to KSR channel
Success
*** 2012-02-24 10:08:11.265 907 krsw.c
NSA2: initialized successfully ASYNC=1
*** 2012-02-24 10:08:24.260
NSA2: receiving message on its channel
No NET_TIMEOUT has been specified - will use default network timeout
...Successfully Received Message from KSR
Received message opcodes(11,11) and status=0
*** 2012-02-24 10:08:24.260 1277 krsw.c
Message 1 has 1 parts
Destination is specified with ASYNC=61440
*** 2012-02-24 10:08:24.260 7291 krsw.c
krsw_log_switch_wait: log switch in progress
*** 2012-02-24 10:08:24.369 2116 krsa.c
Requesting CFTXN 0x0000000055D2D548 noenqueue (last PID:7656) [krsw.c:5185] IX0
*** 2012-02-24 10:08:28.316
*** 2012-02-24 10:08:28.316 2133 krsa.c
Started    CFTXN 0x0000000055D2D548 noenqueue (PID:7644) [krsw.c:5185] IX+
Retrieving log 3
*** 2012-02-24 10:08:28.316 5212 krsw.c
Sending online log thread 2 seq 422 [logfile 3] to standby
Opening logfile [logno 3]
*** 2012-02-24 10:08:28.331 4654 krsw.c
Acquiring Async LNS enqueue thread 2 seq 422 dest 2 mode 4
*** 2012-02-24 10:08:28.331 4674 krsw.c
Acquired Async LNS enqueue thread 2 seq 422 dest 2 mode 4 [id1 131074 id2 422]
Clearing FARC_FINAL/FARC_EOR by Lgwr archive NotiFYer routine
krsd_init_sdips: called from [krsl.c:915] with function 5
Log switch from thread 2 sequence 421 to sequence 422
Changing dest 1 PDI dstvno from 0 to 2 [krsd.c:6072]
Changing [krsd.c:6097] dest 1 stavno from 0 to 2
Setting dest 1 KCRRDROP [krsd.c:6107]
Changing dest 2 PDI dstvno from 0 to 2 [krsd.c:6072]
Changing [krsd.c:6097] dest 2 stavno from 0 to 2
Setting dest 2 KCRRDROP [krsd.c:6107]
Changing dest 3 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 3 stavno from 0 to 1
Setting dest 3 KCRRDROP [krsd.c:6107]
Changing dest 4 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 4 stavno from 0 to 1
Setting dest 4 KCRRDROP [krsd.c:6107]
Changing dest 5 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 5 stavno from 0 to 1
Setting dest 5 KCRRDROP [krsd.c:6107]
Changing dest 6 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 6 stavno from 0 to 1
Setting dest 6 KCRRDROP [krsd.c:6107]
Changing dest 7 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 7 stavno from 0 to 1
Setting dest 7 KCRRDROP [krsd.c:6107]
Changing dest 8 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 8 stavno from 0 to 1
Setting dest 8 KCRRDROP [krsd.c:6107]
Changing dest 9 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 9 stavno from 0 to 1
Setting dest 9 KCRRDROP [krsd.c:6107]
Changing dest 10 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 10 stavno from 0 to 1
Setting dest 10 KCRRDROP [krsd.c:6107]
Changing dest 11 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 11 stavno from 0 to 1
Setting dest 11 KCRRDROP [krsd.c:6107]
Changing dest 12 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 12 stavno from 0 to 1
Setting dest 12 KCRRDROP [krsd.c:6107]
Changing dest 13 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 13 stavno from 0 to 1
Setting dest 13 KCRRDROP [krsd.c:6107]
Changing dest 14 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 14 stavno from 0 to 1
Setting dest 14 KCRRDROP [krsd.c:6107]
Changing dest 15 PDI dstvno from 0 to 1 [krsd.c:6072]
Changing [krsd.c:6097] dest 15 stavno from 0 to 1trace files are getting very huge and reaching 80Gb too.. what are these files ? any clue ?
Kai

Handle: KaiS
Email: [email protected]
Status Level: Newbie (10)
Registered: Nov 20, 2007
Total Posts: 1,353
Total Questions: 574 (434 unresolved)
You have very bad profile status, please bring down the number of pending thread and mark then answered if possible...Please read following
https://forums.oracle.com/forums/ann.jspa?annID=718
Now coming backup to your problem, it seems that you have enabled the tracing of archive..Please read following metalink document which applies to your case...
How To Avoid Many Trace Files With KCRRDROP [ID 1150056.1]
set log_archive_trace=0, if its current value is higher...
Bye

Similar Messages

  • Huge user trace files being generated under udump

    last few days one of my database is generating huge number of user trace files under udump. I tried to investigate in alert.log file. But as I am not very expert to analyze the alert.log file I could not find the cause behind it. Only the ORA-00600: internal error code is found there. But it is a generic error . Please help.

    Thank you again. I got the clue from your reply.
    The error message is pasted below in short.
    ORA-00600: internal error code, arguments: [12333], [0], [0], [0], [], [], [], []
    Current SQL statement for this session:
    BEGIN :1 := TESTDBA.LOCAL_SYSDATE_SFT_FNC( 1,:2,:3 ); END;
    And it is obvious that the function had some problem recently. We need to investigate on that. Now my request is
    Please guide me how to check in metalink with the value '12333' which came as argument of ORA-00600 error. Here my job was easy as I got the problem function name . By I want to know the art of analysing the error message in alert.log file.
    Also tell me which places I should check regularly as a responsibility of a DBA ?
    What are the common errors in Alert log file? Where I should be cautioned ? Where I should ignore. Give me some tips please. Do not restrict your discussion on Alret.log only . Also please cover all important areas. If you think I am taking much of your time then please send me some link or source of PDF etc.

  • Gather Schema Statistics produces HUGE oracle trace file

    Hi
    I am running gather schema stats on my E-Biz 12.1.3 / 11.2.0.3 system
    Each week we run Gather Schema Stats
    I have noticed that when the job completes a 3g trace fuile is produced under the Oracle RDBMS home
    I have tried un-checking the box when the COnc Request is submitted to DO NOT SAVE OUTPUT FILE but it is still procuced.
    I dont want it and do not understand why it is produced. Anyone out there have a resolution?

    I have noticed that when the job completes a 3g trace fuile is produced under the Oracle RDBMS homeWhat is the file name/location? What is the contents of this trace file?
    I have tried un-checking the box when the COnc Request is submitted to DO NOT SAVE OUTPUT FILE but it is still procuced.Do you have any debug/trace enabled on HRMS schema?
    I dont want it and do not understand why it is produced. Anyone out there have a resolution?Was this working before? If yes, any changes been done since then?
    What are the parameters you use to run "Gather Schema Statistics" concurrent program?
    Thanks,
    Hussein

  • Trace file size is huge

    HI Experts,
    In my production database one trace file alone getting created in huge size, could you please make me aware of the reason behind this and solution to this.
    Path:/oracle/admin/proddb/bdump
    proddb_j000_17828.trc /5G/
    Regards,
    Salai

    You can limit trace file size using MAX_DUMP_FILE_SIZE db parameter if you have a free space problem too.
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/initparams116.htm#REFRN10103
    But of course this wont solve your original issue. Try to analyze content of the trace file and try to solve the base issue.

  • SQL*Net message from client - huge wait in trace file

    Dear All,
    I am facing a performance issue in a particular operation ( which was completed in 21 Minutes earlier). Now the same operation takes more than 35 Minutes. I took a trace for those session ( 10046 level 12 trace ) and found Lot of waits in SQL*Net message from client.
    Elapsed times include waiting on following events:
    Event waited on Times Max. Wait Total Waited
    ---------------------------------------- Waited ---------- ------------
    SQLNet message from client 611927 10.00 1121.35*
    I copied only the highest wait in the tkprof output.
    And I found from the tkprof and even in raw trace file this event waits more time after excuting
    SELECT sysdate AS SERVERDATE from dual;
    Elapsed times include waiting on following events:
    Event waited on Times Max. Wait Total Waited
    ---------------------------------------- Waited ---------- ------------
    SQL*Net message to client 115 0.00 0.00
    SQLNet message from client 115 10.00 724.52*
    Please help me to find out why this wait taking long time, especially on the above query..
    Regards,
    Vinodh

    Vinodh Kumar wrote:
    Hi,
    This is what available in the trace file
    PARSING IN CURSOR #2 len=38 dep=0 uid=60 oct=3 lid=60 tim=7052598842 hv=3788189359 ad='7d844fa0'
    *"SELECT sysdate AS SERVERDATE FROM dual"*
    END OF STMT
    PARSE #2:c=0,e=12,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=7052598839
    BINDS #2:
    EXEC #2:c=0,e=42,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=7052599002
    WAIT #2: nam='SQL*Net message to client' ela= 1 driver id=1952673792 #bytes=1 p3=0 obj#=-1 tim=7052599058
    FETCH #2:c=0,e=15,p=0,cr=0,cu=0,mis=0,r=1,dep=0,og=1,tim=7052599110
    *** 2012-01-02 17:07:30.364
    WAIT #2: nam='SQL*Net message from client' *" ela= 10007957"* driver id=1952673792 #bytes=1 p3=0 obj#=-1 tim=7062607120Please find the last line WAIT -- in the complete trace after executing this query
    In awr report , this query taken less than a sec for more than 2000 executions.
    Regards,
    VinodhGood idea to check the raw trace file. It is important to notice that this particular wait event appears after the fetch of the result from the database. The client receives the SYSDATE from the database server, and then the client performs some sort of action for about 10 seconds before submitting its next request to the database. The SQL statement that immediately follows and immediately preceeds this section of the trace file might provide clues regarding what caused the delay, and where that delay resides in the client side code. Maybe a creative developer added a "sleep for 10 seconds" routine when intending to sleep for 10ms? Maybe the client CPU is close to 100% utilization?
    Charles Hooper
    http://hoopercharles.wordpress.com/
    IT Manager/Oracle DBA
    K&M Machine-Fabricating, Inc.

  • Trace file in different location

    Dear all,
    11g on Solaris 10.
    My oracle home (dump location) located in a file system of 10GB.. I need to trace a session which will generate huge trace file. Is there anyway I can have the trace file generated in a different location where I have enough space to allow the growth of the trace file.
    Thanks
    Kai

    Please refer:
    http://www.oracle.com/technology/oramag/oracle/09-mar/o29asktom.html?_template=/ocom/print
    Edited by: SKU on 19-Dec-2009 05:10

  • Best practice to reduce size of BIA trace files

    Hi,
    I saw alert on BIA monitor says 'check size of trace files'. Most of my trace files are above 20MB. I clicked on details it says "Check the size of your trace files. Remove or move the trace files with the memory usage that is too high or trace files that are no longer needed."
    I would like to reduce them these trace files but not sure what is the safetest way to do it. Any suggestion would be appreciated!
    Thanks.
    Mimosa

    Mimosa,
    Let's be clear here first. The tracing set via sm50 is for tracing on the ABAP side of BI not the BIA.
    Yes, it is safe to move/delete TrexAlertServer.trc, TrexIndexServer.trc, etc from the OS level. You can also right click the individual trace when you enter the "Trace" tab in the TREX Admin Tool (python) and I believe there is options to delete them there but it is certaintly OKAY to do this on the OS level. They are simply recreated when new traces are generated.
    I would recommend that you simply .zip the files and move the .zip files to another folder in case SAP support may need them to analyze an issue. As long as they aren't huge, and if hard disk space permits, this shouldnt be an issue. After this you then will need to delete the trace file. It is important that if a trace file has an open handle registered to it then it wont let you delete/move it. Therefore it might be a good idea to do this task when system activity is low or non-existent.
    2 things also to check:
    1. Make sure the python trace is not on.
    2. In the python TREXAdmin Tool, check the Alerts tab and click "Alert Server Configuration". Make sure the trace level is set to "error".
    Hope that helps. As always check the TOM for any concerns:
    https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/46e11c10-0e01-0010-1182-b02db2e8bafb
    Edited by: Mike Bestvina on Apr 1, 2008 3:59 AM - revised some statements to be more clear

  • Unusuall TREX  Trace file getting created..

    Dear All
    we have installed TREX and  in TREX server its getting crating a tarace file TrexQueueServerAlert_myportalci.trc size of  more than 30GB..
    following is the extract of that file ..waht can be the reason for this huge trace file
    Regards
    Buddhike
    [2700] 2008-12-17 16:34:47.158 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [2700] 2008-12-17 16:34:47.158 e Qidx_publi QueueDocStore::hasDocument(01235) : : DocIDMissing
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [4960] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [3772] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [6052] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [6052] 2008-12-17 16:34:51.111 e Qidx_publi DocStateStore.cpp(00570) : DocStateStore::getDocument(UDIV): udiv: 756, result: 4501
    [6052] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [6052] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::hasDocument(01235) : : DocIDMissing
    [6052] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [4960] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [6052] 2008-12-17 16:34:51.111 e Qidx_publi Queue.cpp(04093) : Queue::preprocessMsg: preprocessing doc:  not found
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi DocStateStore.cpp(00570) : DocStateStore::getDocument(UDIV): udiv: 756, result: 4501
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::hasDocument(01235) : : DocIDMissing
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [3772] 2008-12-17 16:34:51.111 e Qidx_publi QueueDocStore::getDocID(01189) : : DocNotFound
    [5296] 2008-12-17 16:34:51.111 e Qidx_publi Queue.cpp(04093) : Queue::preprocessMsg: preprocessing doc:  not found
    [3772] 2008-12-17 16:34:51.111 e Qidx_publi DocStateStore.cpp(00570) : DocStateStore::getDocument(UDIV): udiv: 756,

    Dear Michell
    Thnx for your post..how can i change the trace levels in TREX..?
    which trace level should i keep?
    Regards
    Buddhike

  • Trace Files 11g (Space Issue)

    friends, we are running 11g (11.1.0.7.0) database in our environment. Currently our OS slice is facing space related issues due to huge number of trace files generation. Please share any helpful note/idea.
    Regards,
    Irfan Ahmad

    please see following information against trace related parameters:
    log_archive_trace     3     0     0
    sql_trace     1     FALSE     FALSE
    sec_protocol_error_trace_action     2     TRACE     TRACE
    tracefile_identifier     2          
    trace_enabled     1     TRUE     TRUE
    any idea.
    regards

  • SQL*NET waits in trace file

    Hi All,
    There is a long running query, i generated trace file for this request. In trace file i found that there are huge waits on SQL*Net message from client
    The below is the trace file output:
    Elapsed times include waiting on following events:
    Event waited on Times Waited Max. Wait Total Waited
    SQL*Net message to client 16 0.00 0.00
    SQL*Net more data to client 17 0.00 0.00
    db file sequential read 1450 0.02 4.26
    SQL*Net message from client 16 1414.20 2702.84
    How to resolve this waits from SQL*NET message from client? I checked the network connection, there is no delays in network.
    Any inputs on this issue will be appreciated.

    As Satish indicated, the "SQL*Net message from client" wait is an event which indicates that the database server was waiting for the next request from the client computer, and not an indication that the query needs to be tuned. Manually review the trace file. At one point in the trace file, you will see this wait event with an ela= value which begins with 14142 - please post to this thread that line from the trace file along with the 20 lines before that line and the 20 lines after that line. You may just have a long wait on this event at the beginning, and another long wait on this event at the end of the query.
    Charles Hooper
    IT Manager/Oracle DBA
    K&M Machine-Fabricating, Inc.

  • Create trace file on specific location

    Hi all,
    I'm working with a 11gR2 Enterprise DB on Linux RedHat 5
    Now I need to create a trace file. This trace will be huge and I need to create it on an specific folder on my system because the udump fs could be small for it. These are the commands I'll use:
    oradebug setospid 3165
    oradebug unlimit
    oradebug event 10046 trace name context forever, level 12
    If I use the "oradebug tracefile_name" I'll get the actual trace file name but, How can I say the specific location where I need to create it?
    Best regards,
    dbajug

    dbajug wrote:
    Hi all,
    I'm working with a 11gR2 Enterprise DB on Linux RedHat 5
    Now I need to create a trace file. This trace will be huge and I need to create it on an specific folder on my system because the udump fs could be small for it. These are the commands I'll use:
    oradebug setospid 3165
    oradebug unlimit
    oradebug event 10046 trace name context forever, level 12
    If I use the "oradebug tracefile_name" I'll get the actual trace file name but, How can I say the specific location where I need to create it?
    Best regards,
    dbajugi dont know of any method that will allow you to do this for a single file, but you can use below workaround since the tracefile name is predictable
    lets say your database name is ORCL
    the ospid is 3165, so the tracefile will be created as udump/ORCL_ora_3165.trc
    before starting the trace, you can pre-create it at different location and point a symlink on it, e.g
    touch /lot_of_space_here/trace.trc
    ln -s /lot_of_space_here/trace.trc udump/ORCL_ora_3165.trc
    and now start tracing

  • How to stop the generation of the trace files relate to the particular user

    Dear All,
    Please advice:
    Is it possible to stop the trace file created by a user for a particular job (sql/exp)
    For example :
    Trace files create whenever the user run the sql/exp command from his application server.
    This will created a huge files and caused the file system getting full (not in bdump but it is generation in application user home directory).
    Is there any method to stop the generation of the trace files relate to the particular user base on that job.
    thanks,
    Shanoj

    Dear SHANOJ
    This trigger will created for that specific user by dba or person who have sysdba privs. it will triggered at the login time of that user and alter the sqltrace.
    for example we have write one trigger in our database for every ddl operation in database.
    CREATE OR REPLACE TRIGGER "SYS"."SYS$DDLTRIG" after DDL on database
    declare
    v_machinename VARCHAR2(100);
    v_osuser VARCHAR2(100);
    v_dbuser VARCHAR2(100);
    v_host VARCHAR2(100);
    v_program VARCHAR2(100);
    v_instance VARCHAR2(100);
    v_date VARCHAR2(100);
    begin
    select to_char(sysdate, 'mm-dd-yyyy HH24:MI:SS'),user,sys_context('userenv','OS_
    USER'),sys_context('userenv','HOST'), (select instance_name from v$instance),(se
    lect program from v$session where sid = (select distinct sid from v$mystat)),(se
    lect machine from v$session where sid = (select distinct sid from v$mystat)) int
    o v_date,v_dbuser,v_osuser,v_host,v_instance,v_program,v_machinename from dual;
    IF ora_sysevent = 'DROP' or ora_sysevent = 'REVOKE' or ora_sysevent = 'GRANT' or
    ora_sysevent = 'CREATE' or ora_sysevent = 'TRUNCATE' or ora_sysevent = 'ALTER'
    Then
    insert into ddl_log_new(v_date,v_dbuser,v_osuser,v_host,v_instance,v_program,v_m
    achinename,V_EVENT,V_OBJECT_NAME,V_OBJECT_TYPE,V_OBJECT_OWNER) values (v_date,v_
    dbuser,v_osuser,v_host,v_instance,v_program,v_machinename,ora_sysevent,ora_dict_
    obj_name,ora_dict_obj_type,ora_dict_obj_owner);
    END IF;
    END;
    i hope above example will helpful for u,
    Regards
    Ambrish
    Edited by: ambrishr24 on Dec 5, 2012 5:02 PM
    Edited by: ambrishr24 on Dec 5, 2012 5:05 PM
    Edited by: ambrishr24 on Dec 5, 2012 5:20 PM

  • Trace file woes

    I deleted a trace file that grew too large on a pc that is offline. Has been offline since 2012. It uses software to make badges and to take pictures for those badges. The software - I think, is backed up locally with SQL 2005. I know NOTHING about SQL. 
    I assumed, because it is a log file, and I didn't know about shrink, etc... since it was alog file, I assumed that the data that was growing to 460 gigs was over two years worth of data... but:
    By deleting this log.. could I have lost everything? Pictures, etc... It's running windows 7 and has an older image on it. This is just a log file, right?
    Win 7, Identicard/Premisys, SQL server... machine is offline. 
    Please ask me what you need for clarification... I feel I made a huge mistake.

    Did you delete a Profiler trace file or a transaction log file for your database (usually TLog file has a .LDF extension but it can be anything)?
    If you deleted the TLog file:
    If you have a valid backup of the DB, then you can restore the needed DB.
    If you don't have a valid backup but the SQL machine was shutdown cleanly (not an abrupt shutdown) and there were no transactions that need to be rolled back/forward, you may have a chance to rebuild the transaction log file and access the DB again (use
    this as a last resort to access the DB). Run DBCC CHECKDB after rebuilding the TLog to make sure DB is not corrupted
    Satish Kartan www.sqlfood.com

  • Capping dev_ms trace file size

    hi - i'm wondering if anyone knows a way to help.  we have been asked by SAP to run our msg server at an elevated trace level (trace level 3 - we set it from SMMS).  this writes out a huge amt of data to the dev_ms trace file.
    the default trace file (dev*) size, per the rdisp/TRACE_LOGGING param default is "on, 10m" (i.e. 10mb).
    that applies to "all" the dev* trace files obviously.
    w/ our elevated dev_ms trace set, it's overwrapping in ~3 min ...so between dev_ms.old and dev_ms, we never have more than ~10 min of logs kept at any one time.
    the pt of us running at this elevated dev_ms trace level is so we can capture (save off) the trace file and send to SAP the next time our msg server crashes.
    our SAP file system mountpoint /usr/sap/<SID> is limited in size...and by setting rdisp/TRACE_LOGGING value to a higher value it affects all the dev* files, not just the 1 file i really care about increasing the capped file size on (dev_ms).
    **QUESTION*:  does any one know a way i could keep dev_ms capped at a large value like 100mb yet keep all the other dev files at the normal 10mb default?  thanks in advance

    1.  Increase rdisp/TRACE_LOGGING to 100MB.
    2.  Set (SM51) > Select All Processes > Menu > Process > Trace > Active Components > Uncheck everything and set trace level to 1.
    3.  Menu > Process > Trace > Dispatcher > Change Trace Level > Set to 2
    Wouldn't this essentially just increase dev_ms to 100MB while leaving other dev* trace files to not log anything?

  • Abnormal trace file generation.

    I am facing this problem for some time now ….. i am having trouble with trace files on only one node ….
    SYSTEM DETAILS
    i have a 2 node production server …
    IBM power 7 P750 servers
    oracle version 11.2.0.3 RAC
    RMAN not yet configured but logged in once to delete archiving logs
    The issue is that on node 2 the trace files are generating on a huge level and i have to manually delete them every 2 hours other wise the space is filling up fast. And to add to my misery the trace files are generating in two separate locations
    One location is the default oracle location folder which has a space of 200 gb and it is some how manageable by adrci
    The main issue with me is that some files are getting created in the '/var' folder under '/tmp/oradiag_oracle/../../..' forlder also
    and this /var folder is only 5 gb
    if i ignore it for one hour the space reaches 100%
    i am trying to find what traces are enabled on the system so i can disable any unwanted traces.
    Can anyone help me by telling me how to check for enable tracing events and how to disable them manually
    Thank you .

    i would like to add a couple of points here.
    1. these trace files have no mention in my original alert log file. However there is log file in the trace folder in the /var filesystem where only this error is logged.
    2. here is the complete trace file output.
    Dump file /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1_i3877017.trc
    Dump continued from file: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1.trc
    ORA-07445: exception encountered: core dump [ksmpclrpga()+23296] [SIGSEGV] [ADDR:0x109A1BD28] [PC:0x101E2E540] [Invalid permissions for mapped object] []
    ========= Dump for incident 3877017 (ORA 7445) ========
    ----- Beginning of Customized Incident Dump(s) -----
    ----- End of Customized Incident Dump(s) -----
    ----- START Event Driven Actions Dump ----
    ---- END Event Driven Actions Dump ----
    ----- START DDE Actions Dump -----
    Executing SYNC actions
    Executing ASYNC actions
    ----- START DDE Action: 'dumpFrameContext' (Sync) -----
    ----- START Frame Context DUMP -----
    Status: 0
    <Empty>
    ----- END Frame Context DUMP -----
    ----- END DDE Action: 'dumpFrameContext' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'dumpBuckets' (Sync) -----
    Trace Bucket Dump Begin: default trace bucket
    TIME(*=approx):SEQ:COMPONENT:FILE@LINE: DATA
    Trace Bucket Dump End: default trace bucket
    ----- END DDE Action: 'dumpBuckets' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'dumpDiagCtx' (Sync) -----
    ------- Diag Context Dump -------
    ----- General DiagCtx Dump -----
    Serial#: 3
    Status: 3
    Flags: 0x15
    Usage type: 0
    Diag Fixed SGA information:
    Status: 2
    Flags: 0x0
    Incident Flag: UNSET
    Heap: 0x1106c96f0
    Flood Control information:
    Initialized: NO
    Enabled: NO
    Diag init parameters:
    Flags: 0x1
    Base Flag: 0x1
    Options: 0x86
    ADR option: 1
    ADR Base: '/var/tmp/oradiag_oracle'
    Product Type: 4
    Product ID: 'user_oracle'
    Instance ID: 'host_1545611351_80'
    Flood Control Gate: 0x0
    Flood Control Ctx size: 0
    ADR default lib ID: 0
    UTS in mem trace enabled: TRUE
    UTS default bucket size: 4294967295
    UTS default bucket: 0x0
    Incident ID cache cbf: 0x0
    Incident ID cache pop init: FALSE
    UTS default trace file attr:
    Flags: 0x0
    Prefix: ''
    PIDStr: ''
    Suffix: ''
    Notify cbk: 0x0
    Size limits: 0
    Size cbk: 0x0
    DDE alive update cbk: 0x0
    DDE alive start cbk: 0x0
    DDE alive clear cbk: 0x0
    Custom Flood Control cbk: 0x0
    ----- END General DiagCtx Dump -----
    ----- DDE Diagnostic Information Dump -----
    Depth: 1
    DDE flags: 0x0
    Heap: 0x1106e84b8
    Incident Context pointer in diag: 0x11083ea98
    Incident ID Cache: 0x1106f96d8 (PGA)
    Invocation Context #: 0
    ----- Invocation Context Dump -----
    Address: 0x1106f02f8
    Phase: 3
    flags: 0x10E0005
    Incident ID: 3877017
    Error Descriptor: ORA-7445 [ksmpclrpga()+23296] [SIGSEGV] [ADDR:0x109A1BD28] [PC:0x101E2E540] [Invalid permissions for mapped object] [] [] [] [] [] [] []
    Error class: 0
    Problem Key # of args: 0
    Number of actions: 9
    ----- Incident Context Dump -----
    Address: 0x11083ea98
    Incident ID: 3877017
    Problem Key: ORA 7445
    Error: ORA-7445 [ksmpclrpga()+23296] [SIGSEGV] [ADDR:0x109A1BD28] [PC:0x101E2E540] [Invalid permissions for mapped object] [] [] [] [] [] [] []
    [00]: dbgexExplicitEndInc [diag_dde]
    [01]: dbgeEndDDEInvocationImpl [diag_dde]
    [02]: dbgeEndDDEInvocation [diag_dde]
    [03]: ssexhd []
    [04]: 47f8 []<-- Signaling
    [05]: opidcl []
    [06]: opidrv []
    [07]: sou2o []
    [08]: opimai_real []
    [09]: ssthrdmain []
    [10]: main []
    [11]: __start []
    MD [00]: 'Client ProcId'='[email protected]_1' (0x0)
    Impact 0:
    Impact 1:
    Impact 2:
    Impact 3:
    Derived Impact:
    ----- END Incident Context Dump -----
    ----- END Invocation Context Dump -----
    Invocation Context #: 1
    ----- Invocation Context Dump -----
    Address: 0x1106f24b0
    Phase: 0
    flags: 0x0
    Incident ID: 0
    Error Descriptor:
    Error class: 0
    Problem Key # of args: 0
    Number of actions: 0
    ----- END Invocation Context Dump -----
    ----- END DDE Diagnostic Information Dump -----
    ----- UTS Diagnostic Information Dump (addr=0x11087f670) -----
    flags_dbgtc=0x44 seq_ptr_dbgtc=0x11068c248
    def_bucket_dbgtc=0x1106d6858 gid_dbgtc=20
    defTraceFile_dbgtc=0x1106e8f60 fstack_dbgtc=0x11087f600
    lastEpochTsOutputted_dbgtc=412011185194127 lastRecWriteDumpMs_dbgtc=0
    recWriteDepth_dbgtc=0 numRecWriteDumps_dbgtc=0
    ==== Dumping UTS file=0x1106e8f60 desc=default ====
    Type=2 flags=0 err=0 cbk=0x11036c140 sctx=0x0
    --- Dumping UTS file (ADR type) file=0x1106e8f60 ---
    File name: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1.trc
    flags=0x1106e8f60 needInit=515 nfyMasked=0 bytes=0
    attrib: flags=0x0 pref=ora pid=24838300_1 suffix=
    limit=18446744073709551615 nfy=0x0
    --- Dumping file stack addr=0x11087f600 cnt=1 ---
    Pos=0 fileP=0x1106f0ba8
    ==== Dumping UTS file=0x1106f0ba8 desc=stack ====
    Type=2 flags=0 err=0 cbk=0x11036c140 sctx=0x0
    --- Dumping UTS file (ADR type) file=0x1106f0ba8 ---
    File name: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1_i3877017.trc
    flags=0x1106f0ba8 needInit=555 nfyMasked=0 bytes=0
    attrib: flags=0x0 pref=ora pid=24838300_1 suffix=i3877017
    limit=18446744073709551615 nfy=0x0
    ----- END UTS Diagnostic Information Dump -----
    ----- Events Diagnostic Information Dump -----
    Active Fast Chk Desc in DiagCtx is 0x0
    Active Event group is Default Group
    --- Dump of Event Group ---
    Event Group desc : Default/Active Event Group
    Located at Address 0x11087f5b0
    Event group id 1
    Event group version 1
    Event group heap 0x1106c9470
    Fast Check Descriptor is NULL
    ----- END Events Diagnostic Information Dump -----
    ----- ADR Diagnostic Information Dump -----
    ADR Flags: 0x1e01
    Retries: 1
    ADR Home #0
    ADR Id: 1688299656
    Create Id: 3016083015
    Base Flag: 0x1
    Product Type: 4
    Product Name: clients
    Product Id: user_oracle
    Instance Id: host_1545611351_80
    Base Directory: /var/tmp/oradiag_oracle
    Home Directory: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80
    Relative Path: diag/clients/user_oracle/host_1545611351_80/
    Home Loc #0: 0
    Test Action: 0x0, 0
    ----- ADS Open Files Dump -----
    ################ Open Stream File: 0 ################
    PathFile: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1.trc
    OpFlag: 10, Status: 1, MagicBeg: 2153609765, MagicNum: 3593058129
    Stream Access
    ----- ADS Stream Desc Dump -----
    fd: 23
    ----- END ADS Stream Desc Dump -----
    File Name Fragment: 51_80/trace/ora_24838300_1.trc
    ################ Open Stream File: 1 ################
    PathFile: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1.trm
    OpFlag: 136, Status: 1, MagicBeg: 2153609765, MagicNum: 3593058129
    Stream Access
    ----- ADS Stream Desc Dump -----
    fd: 27
    ----- END ADS Stream Desc Dump -----
    File Name Fragment: 51_80/trace/ora_24838300_1.trm
    ################ Open Stream File: 2 ################
    PathFile: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1_i3877017.trc
    OpFlag: 10, Status: 1, MagicBeg: 2153609765, MagicNum: 3593058129
    Stream Access
    ----- ADS Stream Desc Dump -----
    fd: 28
    ----- END ADS Stream Desc Dump -----
    File Name Fragment: ce/ora_24838300_1_i3877017.trc
    ################ Open Stream File: 3 ################
    PathFile: /var/tmp/oradiag_oracle/diag/clients/user_oracle/host_1545611351_80/trace/ora_24838300_1_i3877017.trm
    OpFlag: 136, Status: 1, MagicBeg: 2153609765, MagicNum: 3593058129
    Stream Access
    ----- ADS Stream Desc Dump -----
    fd: 29
    ----- END ADS Stream Desc Dump -----
    File Name Fragment: ce/ora_24838300_1_i3877017.trm
    ----- END ADS Open Files Dump -----
    ----- END ADR Diagnostic Information Dump -----
    ----- ADL Diagnostic Information Dump -----
    ADL Flags: 0x4
    Timestamp: -1569355021
    Host Name: sihibmn2
    NW Address: 172.25.25.64
    ----- END ADL Diagnostic Information Dump -----
    ----- Diag Context Memory -----
    1106C93A0 00000001 00000000 00000000 00000000 [................]
    1106C93B0 00000003 00000001 03000000 00000000 [................]
    1106C93C0 00000001 106DF1E0 00000001 10693A98 [.....m.......i:.]
    1106C93D0 00000000 00000000 00000000 00000000 [................]
    1106C93E0 00000001 106F4670 00000001 106F8830 [.....oFp.....o.0]
    1106C93F0 00000001 106CB008 00000001 1087F360 [.....l.........`]
    1106C9400 00000001 1087EB88 00000001 1087EC78 [...............x]
    1106C9410 00000001 1087ED40 00000001 1087EE3F [.......@.......?]
    1106C9420 00000001 106CB2A8 00000001 106CB3A7 [.....l.......l..]
    1106C9430 00000001 106CBA60 00000001 106CBB5F [.....l.`.....l._]
    1106C9440 00000001 106CA250 00000001 106CA34F [.....l.P.....l.O]
    1106C9450 00000001 106CAA08 00000001 106CAB07 [.....l.......l..]
    1106C9460 00000001 1087F670 00000001 1087DAE0 [.......p........]
    1106C9470 00000001 106C9500 00001040 00000000 [.....l.....@....]
    1106C9480 00000000 00000000 00000001 108522B0 [..............".]
    1106C9490 00000001 106F8810 00000000 00000000 [.....o..........]
    1106C94A0 00000000 00000000 00030103 00000000 [................]
    1106C94B0 00000000 00000000 000008C0 64696167 [............diag]
    1106C94C0 20706761 00000000 00000020 7FFF7FFF [ pga....... ....]
    1106C94D0 7FFF7FFF 00000000 00000000 00000000 [................]
    1106C94E0 00000000 00000000 00000001 1087EB48 [...............H]
    1106C94F0 00000001 106F8F30 00000000 00000000 [.....o.0........]
    1106C9500 00000000 00000000 000020C0 00000000 [.......... .....]
    1106C9510 00000000 00000000 00000001 10854368 [..............Ch]
    1106C9520 00000001 1087DA40 00000000 00000000 [.......@........]
    1106C9530 00000000 00000000 00021002 00000000 [................]
    1106C9540 00000000 00000000 000020C0 70676120 [.......... .pga ]
    1106C9550 68656170 00000000 00000060 00007FFF [heap.......`....]
    1106C9560 7FFF7FFF 00000000 00000000 00000000 [................]
    1106C9570 00000000 00000118 00000001 1087DA20 [............... ]
    1106C9580 00000001 106D67B8 00000000 00000218 [.....mg.........]
    1106C9590 00000001 106C9590 00000001 106C9590 [.....l.......l..]
    1106C95A0 00000000 00000418 00000001 106C95A8 [.............l..]
    1106C95B0 00000001 106C95A8 00000000 00000818 [.....l..........]
    1106C95C0 00000001 106C95C0 00000001 106C95C0 [.....l.......l..]
    1106C95D0 00000000 00001018 00000001 106C95D8 [.............l..]
    1106C95E0 00000001 106C95D8 00000000 00002018 [.....l........ .]
    1106C95F0 00000001 106C95F0 00000001 106C95F0 [.....l.......l..]
    1106C9600 00000000 00004018 00000001 106C9608 [[email protected]..]
    1106C9610 00000001 106C9608 00000000 00008018 [.....l..........]
    1106C9620 00000001 106C9620 00000001 106C9620 [.....l. .....l. ]
    1106C9630 00000000 00010018 00000001 106C9638 [.............l.8]
    1106C9640 00000001 106C9638 00000000 00020018 [.....l.8........]
    1106C9650 00000001 106C9650 00000001 106C9650 [.....l.P.....l.P]
    1106C9660 00000000 00040018 00000001 106C9668 [.............l.h]
    1106C9670 00000001 106C9668 00000000 00080018 [.....l.h........]
    1106C9680 00000001 106C9680 00000001 106C9680 [.....l.......l..]
    1106C9690 00000000 00100018 00000001 106C9698 [.............l..]
    1106C96A0 00000001 106C9698 00000000 00200018 [.....l....... ..]
    1106C96B0 00000001 106C96B0 00000001 106C96B0 [.....l.......l..]
    1106C96C0 00000000 00400018 00000001 106C96C8 [[email protected]..]
    1106C96D0 00000001 106C96C8 00000000 00800018 [.....l..........]
    1106C96E0 00000001 106C96E0 00000001 106C96E0 [.....l.......l..]
    1106C96F0 00000000 00000000 000020C0 00000000 [.......... .....]
    1106C9700 00000000 00000000 00000001 106EA3F8 [.............n..]
    1106C9710 00000001 106EA458 00000001 106C98E0 [.....n.X.....l..]
    1106C9720 00000000 00000000 00021002 00000000 [................]
    1106C9730 00000000 00000000 00002058 73676120 [.......... Xsga ]
    1106C9740 68656170 00000000 00000020 00107FFF [heap....... ....]
    1106C9750 7FFF7FFF 00000000 00000000 00000000 [................]
    1106C9760 00000000 00000118 00000001 106EA438 [.............n.8]
    1106C9770 00000001 106EA438 00000000 00000218 [.....n.8........]
    1106C9780 00000001 106C9780 00000001 106C9780 [.....l.......l..]
    1106C9790 00000000 00000418 00000001 106C9798 [.............l..]
    1106C97A0 00000001 106C9798 00000000 00000818 [.....l..........]
    1106C97B0 00000001 106C97B0 00000001 106C97B0 [.....l.......l..]
    1106C97C0 00000000 00001018 00000001 106C97C8 [.............l..]
    1106C97D0 00000001 106C97C8 00000000 00002018 [.....l........ .]
    1106C97E0 00000001 106C97E0 00000001 106C97E0 [.....l.......l..]
    1106C97F0 00000000 00004018 00000001 106C97F8 [[email protected]..]
    1106C9800 00000001 106C97F8 00000000 00008018 [.....l..........]
    1106C9810 00000001 106C9810 00000001 106C9810 [.....l.......l..]
    1106C9820 00000000 00010018 00000001 106C9828 [.............l.(]
    1106C9830 00000001 106C9828 00000000 00020018 [.....l.(........]
    1106C9840 00000001 106C9840 00000001 106C9840 [[email protected].@]
    1106C9850 00000000 00040018 00000001 106C9858 [.............l.X]
    1106C9860 00000001 106C9858 00000000 00080018 [.....l.X........]
    1106C9870 00000001 106C9870 00000001 106C9870 [.....l.p.....l.p]
    1106C9880 00000000 00100018 00000001 106C9888 [.............l..]
    1106C9890 00000001 106C9888 00000000 00200018 [.....l....... ..]
    1106C98A0 00000001 106C98A0 00000001 106C98A0 [.....l.......l..]
    1106C98B0 00000000 00400018 00000001 106C98B8 [[email protected]..]
    1106C98C0 00000001 106C98B8 00000000 00800018 [.....l..........]
    1106C98D0 00000001 106C98D0 00000001 106C98D0 [.....l.......l..]
    1106C98E0 00000001 106C98F0 00000001 106C98F0 [.....l.......l..]
    1106C98F0 00000001 106C98E0 00000001 106C98E0 [.....l.......l..]
    1106C9900 00000000 00000000 00000000 00000000 [................]
    1106C9910 00000000 00000000 00000000 7FFFFFFF [................]
    1106C9920 00000000 00000000 00000000 00000000 [................]
    Repeat 2 times
    1106C9950 00000000 00000000 00000001 106C9958 [.............l.X]
    1106C9960 00000001 106C9958 00000000 00000000 [.....l.X........]
    1106C9970 00000000 00000000 00000000 00000000 [................]
    Repeat 63 times
    1106C9D70 00000000 00000000 00000001 106C9D78 [.............l.x]
    1106C9D80 00000001 106C9D78 00000000 00000000 [.....l.x........]
    1106C9D90 00000001 106C9D90 00000001 106C9D90 [.....l.......l..]
    1106C9DA0 00000000 00000000 00000001 106C9DA8 [.............l..]
    1106C9DB0 00000001 106C9DA8 00000000 00000000 [.....l..........]
    1106C9DC0 00000001 106C9DC0 00000001 106C9DC0 [.....l.......l..]
    1106C9DD0 00000000 00000000 00000001 106C9DD8 [.............l..]
    1106C9DE0 00000001 106C9DD8 00000000 00000000 [.....l..........]
    1106C9DF0 00000001 106C9DF0 00000001 106C9DF0 [.....l.......l..]
    1106C9E00 00000000 00000000 00000001 106C9E08 [.............l..]
    1106C9E10 00000001 106C9E08 00000000 00000000 [.....l..........]
    1106C9E20 00000001 106C9E20 00000001 106C9E20 [.....l. .....l. ]
    1106C9E30 00000000 00000000 00000001 106C9E38 [.............l.8]
    1106C9E40 00000001 106C9E38 00000000 00000000 [.....l.8........]
    1106C9E50 00000001 106C9E50 00000001 106C9E50 [.....l.P.....l.P]
    1106C9E60 00000000 00000000 00000001 106C9E68 [.............l.h]
    1106C9E70 00000001 106C9E68 00000000 00000000 [.....l.h........]
    1106C9E80 00000001 106C9E80 00000001 106C9E80 [.....l.......l..]
    1106C9E90 00000000 00000000 00000001 106C9E98 [.............l..]
    1106C9EA0 00000001 106C9E98 00000000 00000000 [.....l..........]
    1106C9EB0 00000001 106C9EB0 00000001 106C9EB0 [.....l.......l..]
    1106C9EC0 00000000 00000000 00000001 106C9EC8 [.............l..]
    1106C9ED0 00000001 106C9EC8 00000000 00000000 [.....l..........]
    1106C9EE0 00000001 106C9EE0 00000001 106C9EE0 [.....l.......l..]
    1106C9EF0 00000000 00000000 00000001 106C9EF8 [.............l..]
    1106C9F00 00000001 106C9EF8 00000000 00000000 [.....l..........]
    1106C9F10 00000000 00000000 00000000 00000000 [................]
    1106C9F20 00000001 106C4D10 00000001 106EFF88 [.....lM......n..]
    1106C9F30 00000001 106EA478 00000000 00000000 [.....n.x........]
    1106C9F40 00000001 106E34E0 00000001 106C8F88 [.....n4......l..]
    1106C9F50 00000001 106C8EE8 00000015 00000000 [.....l..........]
    1106C9F60 00000001 106E8558 00000001 106F02F8 [.....n.X.....o..]
    1106C9F70 00000001 106F24B0 00000001 00000000 [.....o$.........]
    1106C9F80 00000001 1087F8E8 00000001 106E8640 [.............n.@]
    1106C9F90 00000000 00000000 00000001 1083EA98 [................]
    1106C9FA0 00000001 1087F5B0 00000001 1087F5B0 [................]
    1106C9FB0 00000001 106C93A8 00000000 00000001 [.....l..........]
    1106C9FC0 00000000 00000000 00000000 00000000 [................]
    1106C9FD0 00000003 00000000 00000001 1036C230 [.............6.0]
    1106C9FE0 00000001 1036C380 00000001 106EC538 [.....6.......n.8]
    1106C9FF0 00000001 106DF1E0 00000000 00000000 [.....m..........]
    1106CA000 00000000 00000000 00000000 00000000 [................]
    Repeat 1 times
    ----- Diag Context Init Params Memory -----
    1106EFF80 00000001 00000001 [........]
    1106EFF90 2F766172 2F746D70 2F6F7261 64696167 [var/tmp/oradiag]
    1106EFFA0 5F6F7261 636C6500 00000000 00000000 [_oracle.........]
    1106EFFB0 00000000 00000000 00000000 00000000 [................]
    Repeat 20 times
    1106F0100 00000004 75736572 5F6F7261 636C6500 [....user_oracle.]
    1106F0110 00000000 00000000 00000000 00000000 [................]
    1106F0120 00000068 6F73745F 31353435 36313133 [...host_15456113]
    1106F0130 35315F38 30000000 00000000 00000000 [51_80...........]
    1106F0140 00000000 00000000 00000000 00000000 [................]
    1106F0150 00000000 00000086 01000000 00000000 [................]
    1106F0160 00000000 00000000 00000000 00000000 [................]
    Repeat 1 times
    1106F0180 00000000 00000001 FFFFFFFF 00000000 [................]
    1106F0190 00000000 00000000 00000000 00000000 [................]
    Repeat 14 times
    1106F0280 00000000 00000000 [........]
    ----- Diag Thread Local Root Dump -----
    Sequence#: 4
    Flags: 0x0
    Diag Pointers by type:
    [0]: 0x1106c93a0
    [1]: 0x0
    [2]: 0x0
    Library References:
    [00]: ID=0 (0x1106c8f08):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [01]: ID=0 (0x1106c8f18):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [02]: ID=2 (0x1106c8f28):
    Precedence: 2
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [03]: ID=3 (0x1106c8f38):
    Precedence: 3
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [04]: ID=0 (0x1106c8f48):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [05]: ID=0 (0x1106c8f58):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [06]: ID=0 (0x1106c8f68):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [07]: ID=0 (0x1106c8f78):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [08]: ID=8 (0x1106c8f88):
    Precedence: 8
    Reference Count: 1
    Usage type: 0
    Usage flags: 0x0
    [09]: ID=0 (0x1106c8f98):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [10]: ID=0 (0x1106c8fa8):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [11]: ID=0 (0x1106c8fb8):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [12]: ID=0 (0x1106c8fc8):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [13]: ID=0 (0x1106c8fd8):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [14]: ID=0 (0x1106c8fe8):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [15]: ID=0 (0x1106c8ff8):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [16]: ID=0 (0x1106c9008):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [17]: ID=0 (0x1106c9018):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [18]: ID=0 (0x1106c9028):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [19]: ID=0 (0x1106c9038):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    [20]: ID=0 (0x1106c9048):
    Precedence: 0
    Reference Count: 0
    Usage type: 0
    Usage flags: 0x0
    ----- Diag Thread Local Root Memory -----
    1106C8EE0 3B9EF5C8 00000000 [;.......]
    1106C8EF0 00000001 106C93A0 00000000 00000000 [.....l..........]
    1106C8F00 00000000 00000000 00000000 00000000 [................]
    Repeat 2 times
    1106C8F30 00000002 02000000 00000000 00000000 [................]
    1106C8F40 00000003 03000000 00000000 00000000 [................]
    1106C8F50 00000000 00000000 00000000 00000000 [................]
    Repeat 2 times
    1106C8F80 00000000 00000000 00000001 106EFF88 [.............n..]
    1106C8F90 00000008 08010000 00000000 00000000 [................]
    1106C8FA0 00000000 00000000 00000000 00000000 [................]
    Repeat 10 times
    1106C9050 00000000 00000000 00000004 00000000 [................]
    ----- END Diag Thread Local Root Dump -----
    ----- Diag Context Flood Control Memory -----
    1106EA480 00000000 00000000 [........]
    1106EA490 00000000 00000000 00000000 00000000 [................]
    Repeat 5 times
    1106EA4F0 00000000 00000000 [........]
    ------- END Diag Context Dump -------
    ----- END DDE Action: 'dumpDiagCtx' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'dumpGenralConfiguration' (Sync) -----
    ========= General Configuration ========
    --------- GC Source: [generic GC] --------
    [0000]: item 0=ORACLE_HOME = /u0/app/oracle/product/11.2.0/dbhome_1
    [0000]: item 1=System name:     AIX
    [0000]: item 2=Node name:     sihibmn2
    [0000]: item 3=Release:     1
    [0000]: item 4=Version:     7
    [0000]: item 5=Machine:     00F7694C4C00
    --------- GC Source: [unittest_GC] --------
    --------- GC Source: [kernel_GC] --------
    [0003]: banner=Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
    With the Partitioning, Real Application Clusters, OLAP, Data Mining
    and Real Application Testing options
    [0003]: product_str=Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
    [0003]: product_version=11.2.0.3.0
    [0003]: installed_feature_1=Partitioning
    [0003]: installed_feature_2=Real Application Clusters
    [0003]: installed_feature_3=OLAP
    [0003]: installed_feature_4=Data Mining
    [0003]: installed_feature_5=Real Application Testing
    [0003]: installed_features_num=5
    [0003]: instance_name=SIH2
    [0001]: os_name=AIX
    [0001]: number_core_cpu=4
    --------- GC Source: [Non_Default_Kernel_Parameters] --------
    --------- GC Source: [Visible_Default_Kernel_Parameters] --------
    ========= End of General Configuration ========
    ----- END DDE Action: 'dumpGenralConfiguration' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'dumpKernelDiagState' (Sync) -----
    ------- Kernel Diag Dump -------
    dbkcBSExt: 0
    dbkedDefDump info:
    Internal err count: 0
    Error Flags: 0x0
    Exception: FALSE
    No Bootstrapping info (SGA is not mapped)
    ------- END Kernel Diag Dump -------
    ----- END DDE Action: 'dumpKernelDiagState' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'xdb_dump_buckets' (Sync) -----
    ----- END DDE Action: 'xdb_dump_buckets' (FAILURE, 0 csec) -----
    ----- START DDE Action: 'dumpKGERing' (Sync) -----
    ----- END DDE Action: 'dumpKGERing' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'dumpKGEState' (Sync) -----
    kgepgtfr 0x11083b260
    kgepgtba 0x110841a90
    kgepgter 0
    kgepgpar kgepgbpa 0x1106dfaa4
    kgepgepa 0x1106e02a4
    kgepgtfd 4
    kgepgdmc 0
    kgepgflg 0x0
    kgepg_stkgfr 0x0
    kgepgkgsmp 0x1106df1e0
    kgepgspm 4
    kgepg_ba_set_in_eh 0x0
    kgepg_kgecatch_set_in_eh_ba 0x0
    kge_ba_set_in_eh_funcloc 0x0
    kge_ba_set_in_eh_fileloc 0x0
    ------------------- start error stack dump with barriers
    <error barrier> at 0x110841a90
    ------------------- end error stack dump with barriers
    ----- END DDE Action: 'dumpKGEState' (SUCCESS, 0 csec) -----
    ----- START DDE Action: 'kpuActionDefault' (Sync) -----
    Begin OCI Current State Dump
    End OCI Current State Dump
    Begin OCI Call Context Dump
    End OCI Call Context Dump
    Begin Process state dump.
    ttcdrvdmplocation: msg-0 ln-0 reporting 0
    HST is NULL or no two task connection
    End Process state dump.
    ----- END DDE Action: 'kpuActionDefault' (SUCCESS, 0 csec) -----
    ----- END DDE Actions Dump (total 0 csec) -----
    End of Incident Dump
    Any advices please.
    Thank you.

Maybe you are looking for