Top wait events in awr

hi,
We are using 11.2.0.3.0 on solaris 10 facing slow performance, following are the Wait Events in AWR report, need assistance to overcome it. Also if any specific document to analyze AWR report and to pin point the performance bottleneck.
Foreground Wait Events
Avg
%Time Total Wait
wait
Waits   % DB
Event                        
Waits -outs   Time (s)
(ms)
/txn   time
direct path read           
308,729
0
21,191 
69
58.0   39.5
db file sequential read    
208,754

3,742 
18
39.2
7.0
cursor: pin S           
19,541,899

2,561  
0  3,668.5
4.8
Background Wait Events
Avg
%Time Total Wait
wait
Waits   % bg
Event                        
Waits -outs   Time (s)
(ms)
/txn   time
log file parallel write     
26,479
0   
942 
36 
5.0   40.3
db file parallel write     
216,823
0   
809  
4
40.7   34.6
control file sequential re  
11,673
0    
56  

2.2
2.4
control file parallel writ   
6,280
0    
35  

1.2
1.5
direct path read               
534
0    
26 
49 
0.1
1.1

You need to identify if you are excessively running Parallel Query -- too many queries being parallelised and doing direct path reads bypassing the buffer cache.
In 11gR2, you might also find full table scans of large tables becoming direct path reads.
See this thread :  https://forums.oracle.com/thread/2552571
Hemant K Chitale

Similar Messages

  • Oracle RAC 9i LMD library cache lock top wait event

    We are experiencing the library cache lock as our top wait event. Even thought the box is currently idle, The Global Enqueue Service Daemon (LMD) is taking up CPU cycles. The background process is also logging to trace "skgxpdocon: warning outstanding accept handle count has reached new high water mark 245000".
    Any help would be appreciated.
    Thanks

    There is a new patch for this - check out p4673610 on metalink. We have also experience the problem in 9.2.0.8.

  • Top 5 wait events in AWR Repprt

    Hi,
    The following is top 5 wait event in my AWR reports...
    Whenever I take reports this are always top 5 events
    Top 5 Timed Events
    =============================================================================================================
    Event                
    CPU time           
    Waits               4,717
    % Total Call Time     62.0
    log file sync           
    Waits                64,963           
    Time(s)               1,362           
    Avg Wait(ms)          21                
    % Total Call Time     17.9           
    Wait Class          Commit
    log file parallel write     
    Waits               63,485           
    Time(s)               1,004           
    Avg Wait(ms)          16                
    % Total Call Time     13.2           
    Wait Class          System I/O
    enq: TX - row lock contention
    Waits               348           
    Time(s)               984           
    Avg Wait(ms)          2,828                
    % Total Call Time     12.9           
    Wait Class          Application
    db file parallel write      
    Waits               29,305           
    Time(s)               561           
    Avg Wait(ms)          19                
    % Total Call Time     7.4           
    Wait Class          System I/O
    ------------------------------------------------------------------------------------------------------------

    Start with Performance Tuning Guide
    10.2.3 Table of Wait Events and Potential Causes

  • Top Wait Events

    hi gurus,
    3 node rac 10.2.0.4 serving a packaged application.
    Top 5 timed events in awr shown as
    Event=CPU time
         Waits=
    Time(s)=1,950
    Avg Wait(ms)
    % Total Call Time=45.3     
    Wait Class
    Event=gc cr multi block request
         Waits= 6,551,055
    Time(s)= 1,396
    Avg Wait(ms)=0
    % Total Call Time=38.9          
    Wait Class= Cluster
    Event=db file scattered read
         Waits= 186,295
    Time(s)= 719
    Avg Wait(ms)=4
    % Total Call Time=18.2          
    Wait Class= User I/O
    Event=db file parallel read
         Waits= 43,383
    Time(s)= 241
    Avg Wait(ms)=6
    % Total Call Time= 5.9               
    Wait Class= User I/O
    Event=      log file sync
         Waits= 71,064
    Time(s)= 83
    Avg Wait(ms)=1
    % Total Call Time= 3.1               
    Wait Class= Commit
    db_block_size=8KB
    db_file_multiblock_read_count = default setting of 128
    question:
    are the high wait values of gc cr multi block request and db file scattered read are due to db_file_multiblock_read_count?
    if that's the case, is there a way to find optimum value for db_file_multiblock_read_count?
    or any other findings please?
    experts, appreciate your valuable help
    thanks in advance,
    charles

    user570138 wrote:
    there are queries going for the full table scan with outer joins (milliion of records). those are the same sqls at the top of "sql order by cluster time"in awr with high CPU utilization.
    any way to fine tune the instance to reduce the "gc cr multi block request"
    apart from changing the code as the code belongs to a package based application please?
    Do you have a performance problem ?
    You are doing some large tablescans; these are (probably) the root cause of the gc cr multiblock read, the db file scattered reads, and the CPU, but if the queries are necessary and the execution paths are the best that can be done then maybe you just have to recognise that the resource you're using is reasonable for the queries you have to run.
    Otherwise
    <ul>
    (a) can you find a more efficient access path for any of these queries
    (b) can you make sure that all these queries run on the same node so that you get some benefit from node-affinity (possibly the object(s) will be remasted to that single node) and reduce the interconnect traffic.
    </ul>
    Regards
    Jonathan Lewis
    http://jonathanlewis.wordpress.com
    http://www.jlcomp.demon.co.uk
    To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
    "Science is more than a body of knowledge; it is a way of thinking"
    Carl Sagan                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

  • Top Wait Events Query is needed

    Hi,
    I hope I'm asking this question in right place.
    I need a script and its output should give me the top 5 wait events in last 1 hour for an instance.

    986330 wrote:
    Hi,
    I hope I'm asking this question in right place.
    I need a script and its output should give me the top 5 wait events in last 1 hour for an instance.
    which Top 5? Top number of Waits? Top Total time Waited? Top Avg Wait Time?
    why don't you just run AWR report?

  • RAC specific Wait events in AWR

    DB version: 10.2.0.4
    OS : Solaris x86
    What are the most frequent RAC specific Wait events that appear in AWR reports ?

    Hi Pete,
    This depends on your environment. You can identify them as follows:
    Monitoring Oracle RAC Statistics and Wait Events
    http://download.oracle.com/docs/cd/E11882_01/rac.112/e16795/monitor.htm#i1010220
    ASH Report for Oracle RAC: Top Cluster Events
    The ASH report Top Cluster Events section is part of the Top Events report that is specific to Oracle RAC. The Top Cluster Events report lists events that account for the highest percentage of session activity in the cluster wait class event along with the instance number of the affected instances. You can use this information to identify which events and instances caused a high percentage of cluster wait events.
    http://download.oracle.com/docs/cd/E11882_01/rac.112/e16795/monitor.htm
    Regards,
    Levi Pereira
    <font size="1" color="red">Please close your thread when you get the solution to your problem.</font><br>
    <font size="1" color="red">Mark the replies answered "helpful" answer and/or "correct" answer that will help others with same problem.</font><br>
    <font size="1" color="red">Thanks for doing your part to make this community as valuable as possible for everyone!</font><br>

  • About wait events of awr

    is there anyone who can helip to advise me the meaning of
    1.SQL*Net break/reset to client
    2. Streams AQ: waiting for messages in the queue
    3. wait for unread message on broadcast channel
    we are processing a performence test , tks for your kind advise

    You can ignore these events as they are IDLE events, not WAIT events. In short, they mean user is connected but is not doing anything.

  • Tuning top wait events

    hi,
    why does the following wait event occurs.how to tune these wait events
    control file parallel write and direct path load
    With Regards
    Boo

    "control file parallel write" event occurs while the session is writing physical blocks to all control files.
    This happens when:
    * The session starts a control file transaction (to make sure that the control files
    are up to date in case the session crashes before committing the control file
    transaction)
    * The session commits a transaction to a control file
    * Changing a generic entry in the control file, the new value is being written to all
    control files
    The wait time is the time it takes to finish all writes to all control files
    To reduce this wait event, you can decrease the number of your control files (if this number too high) or place your control files to faster disks.
    "direct path load" event occurs when a session waits for completion of direct load operations to database files. For example, if you are using SQL*Loader direct path load operation or import dump file made in direct mode.
    To reduce this wait event, you can also place your control files to faster disks.

  • Enq: TX - row lock contention in TOP 5 wait event

    DB version:11.1.0.7.0
    I am having enq: TX - row lock contention in top 5 wait event.
    AWR analyze period - 9-10(pm). During this time only one sql loader is running to insert the data. No other job are running. So there is no chance of other session blocking this session. is there any chance of row lock contention happen by same session.
    SQL> SELECT INDEX_NAME,INDEX_TYPE,UNIQUENESS FROM DBA_INDEXES WHERE TABLE_NAME='DATA_DATA';
    INDEX_NAME INDEX_TYPE UNIQUENES
    CIDX      BITMAP NONUNIQUE
    VIDX           BITMAP NONUNIQUE
    Thanks.

    SQL> SELECT INDEX_NAME,INDEX_TYPE,UNIQUENESS FROM DBA_INDEXES WHERE TABLE_NAME='DATA_DATA';
    INDEX_NAME INDEX_TYPE UNIQUENES
    CIDX BITMAP NONUNIQUE
    VIDX BITMAP NONUNIQUEYou have bitmap indexes here on a table being inserted into. Bitmap Indexes are another source of lock(and deadlock) in OLTP application. You said that the SQLloader was the unique active program but may be you are also triggering another procedure after the load. Procedure in which you might be using also automomous transactions and so on...
    Check first if your table is subject to DML operation in a a multi-user concurrent accesss and in which case you have to get rid of those bitmap indexes
    http://hourim.wordpress.com/2011/03/14/deadlock-%e2%80%93-part-1-bitmap-index/
    Best regards
    Mohamed Houri
    www.hourim.wordpress.com

  • Top 5 wait event

    I need some guidance on my AWR 5 top wait events
    I have 10gR2 on Solaris 9.
    The top 5 events in my AWR (ran hourly) always contain the following: (not necessary in order)
    CPU time
    control file parallel write
    dbfile parallel write
    log file parallel write
    log file sync.
    Is this an indication of an undersized log buffer ?
    My value for log buffer is 14,258,176
    I have 4 CPUs
    I'd appreciate any help

    Hi!
    I do have the same problem and trying to figure it out
    Top 5 Timed Events Avg %Total
    ~~~~~~~~~~~~~~~~~~ wait Call
    Event Waits Time (s) (ms) Time Wait Class
    CPU time 932 71.3
    reliable message 2,828 509 180 38.9 Other
    control file parallel write 8,759 300 34 23.0 System I/O
    db file parallel write 19,813 238 12 18.2 System I/O
    control file sequential read 65,435 193 3 14.7 System I/O
    share with me, your thoughts
    Ravi

  • How do I the interpret "Disk file operations I/O" wait event?

    I have a large and very busy batch database. All of a sudden the "Disk file operations I/O" wait event is in the top 5 in AWR.
    The manual page isn't very helpful:
    http://download.oracle.com/docs/cd/E11882_01/server.112/e17110/waitevents003.htm#insertedID40
    Disk file operations I/O
    This event is used to wait for disk file operations (for example, open, close, seek, and resize). It is also used for miscellaneous I/O operations such as block dumps and password file accesses.
    So here is my question: What exactly is going on when I see this wait event? Why doesn't it show up as one of the other I/O events? Can I make it go away? Should I make it go away?
    DR

    sb92075 wrote:
    All of a sudden the "Disk file operations I/O" wait event is in the top 5 in AWR.Top wait event
    In EVERY Top Wait Event list, one wait event will ALWAYS be on top as #1; by definition of the list.
    Simply because any item, even #1, appears on this list does not mean this is a problem & needs to be fixed.
    If the Top Wait Event accounts for only 5 seconds out of a 1 hour sample,
    then reducing it to ZERO won't measurably improve overall application performance.
    The actual Time Waited is required to determine if it is a problem or not.It's taking 20% of time in a 15 minute sample. Anything that takes 20% of deserves to be understood....So: What actually causes it?
    DR

  • What is ges reusing os pid wait event

    What is wait event "ges reusing os pid". In our RAC environment it is one of the top wait events. How to minimze it.

    This is a wait event in Oracle 10g for Global Enqueue Services (ges) waiting on an operating system process id (os pid).
    How to resolve this issue? I checked the bug list on Metalink and there is a patch set for the issue that may help.
    Question: what version and patch release are you running for Oracle RAC?
    Also, you probably want to tune your public network and private interconnects between the nodes in your Oracle RAC cluster.
    Regards,
    Ben Prusinski
    http://oracle-magician.blogspot.com/

  • CXPACKET Wait events

    Hi,
    I am facing wait event CXPACKET in top wait events. we are using SQL Server 2012. the DOP is set to 1. 
    Please suggest how to resolve this
    REgards

    Hi,
    I am facing wait event CXPACKET in top wait events. we are using SQL Server 2012. the DOP is set to 1. 
    Any reaosn why DOP is set to 1.Have you tested this scenario.Instaed of first moving for resolving CXPACKET wait which is not a acctual reason it is just a symptom .It happens when various parallel threads are waiting to synchronize after doing the
    task.What is other major wait stats you can see.Can you paste output of below query here
    --By Jnathan Kehayias
    SELECT TOP 10
    wait_type ,
    max_wait_time_ms wait_time_ms ,
    signal_wait_time_ms ,
    wait_time_ms - signal_wait_time_ms AS resource_wait_time_ms ,
    100.0 * wait_time_ms / SUM(wait_time_ms) OVER ( )
    AS percent_total_waits ,
    100.0 * signal_wait_time_ms / SUM(signal_wait_time_ms) OVER ( )
    AS percent_total_signal_waits ,
    100.0 * ( wait_time_ms - signal_wait_time_ms )
    / SUM(wait_time_ms) OVER ( ) AS percent_total_resource_waits
    FROM sys.dm_os_wait_stats
    WHERE wait_time_ms > 0 -- remove zero wait_time
    AND wait_type NOT IN -- filter out additional irrelevant waits
    ( 'SLEEP_TASK', 'BROKER_TASK_STOP', 'BROKER_TO_FLUSH',
    'SQLTRACE_BUFFER_FLUSH','CLR_AUTO_EVENT', 'CLR_MANUAL_EVENT',
    'LAZYWRITER_SLEEP', 'SLEEP_SYSTEMTASK', 'SLEEP_BPOOL_FLUSH',
    'BROKER_EVENTHANDLER', 'XE_DISPATCHER_WAIT', 'FT_IFTSHC_MUTEX',
    'CHECKPOINT_QUEUE', 'FT_IFTS_SCHEDULER_IDLE_WAIT',
    'BROKER_TRANSMITTER', 'FT_IFTSHC_MUTEX', 'KSOURCE_WAKEUP',
    'LOGMGR_QUEUE', 'ONDEMAND_TASK_QUEUE',
    'REQUEST_FOR_DEADLOCK_SEARCH', 'XE_TIMER_EVENT', 'BAD_PAGE_PROCESS',
    'DBMIRROR_EVENTS_QUEUE', 'BROKER_RECEIVE_WAITFOR',
    'PREEMPTIVE_OS_GETPROCADDRESS', 'PREEMPTIVE_OS_AUTHENTICATIONOPS',
    'WAITFOR', 'DISPATCHER_QUEUE_SEMAPHORE', 'XE_DISPATCHER_JOIN',
    'RESOURCE_QUEUE' )
    ORDER BY wait_time_ms DESC
    Please mark this reply as the answer or vote as helpful, as appropriate, to make it useful for other readers

  • Awr report showing "Undo segment recovery" in top 1st wait event.

    Hi all.
    Today evening oracle.exe is hitting 100% cpu in windows server 2003.
    In the awr report "undo segment recovery" listed in the top 5 wait event (1st place) and
    also in the enterprise manager it shows the details like,
    ACTION 1:
    Action Investigate the cause for high "undo segment recovery" waits. Refer to Oracle's "Database Reference" for the description of this wait event. Use given SQL for further investigation.
    Rationale The SQL statement with SQL_ID "0x63ctfjb1m1j" was found waiting for "undo segment recovery" wait event.
    SQL Text UPDATE PF_SubjectVEChapterPage SET NeedsRecalcState = NULL, NeedsUnsignState = ...
    SQL ID 0x63ctfjb1m1j
    Rationale The SQL statement with SQL_ID "0x6uvufcw5umh" was found waiting for "undo segment recovery" wait event.
    SQL Text
    SQL ID 0x6uvufcw5umh
    Rationale The SQL statement with SQL_ID "2dvmt5mhr3m10" was found waiting for "undo segment recovery" wait event.
    SQL Text UPDATE PF_SubjectVEChapterPage SET NeedsRecalcState = NULL, NeedsUnsignState = ...
    SQL ID 2dvmt5mhr3m10
    Rationale The SQL statement with SQL_ID "gx5pummu20jzb" was found waiting for "undo segment recovery" wait event.
    SQL Text UPDATE PF_SubjectVEChapterPage SET NeedsRecalcState = NULL, NeedsUnsignState = ...
    SQL ID gx5pummu20jzb
    Rationale The SQL statement with SQL_ID "1rxk3vt41zg1u" was found waiting for "undo segment recovery" wait event.
    SQL Text
    SQL ID 1rxk3vt41zg1u
    ACTION 2:
    Investigate the cause for high "undo segment recovery" waits in Module "dllhost.exe".
    ACTION 3:
    Investigate the cause for high "undo segment recovery" waits in Service "SYS$USERS".
    I'm not sure what action i need to take exactly.Please provide your valuable suggestions to proceed further.
    Thanks, Muhammed Thameem.

    http://download.oracle.com/docs/cd/A97630_01/server.920/a96536/apa5.htm
    "undo segment recovery
    PMON is rolling back a dead transaction. The wait continues until rollback finishes.
    Wait Time: 3 seconds
    Parameters:
    segment# -> The ID of the rollback segment that contains the transaction that is being rolled back
    tx flags -> The transaction flags (options) set for the transaction that is being rolled back?

  • Interpret DB CPUwait event (top 5 wait event AWR)

    Hi,
    Can anyone tell me how to read the table below especially the "DB CPU" section,
    Is it right to say that 41.71% of time was consumed waiting for CPU?? this is urgent
    Event                     Waits           Time(s)      Avg wait (ms)      % DB time      Wait Class
    db file sequential read      300,835      1,483           5           58.42           User I/O
    DB CPU                     1,059                     41.71
    reliable message           9,499           18           2           0.71           Other
    PX Deq: Slave Session Stats      6,506           11           2           0.43           Other
    gc cr grant 2-way           26,218           6           0           0.25           Cluster

    user589420 wrote:
    Hi,
    Can anyone tell me how to read the table below especially the "DB CPU" section,
    Is it right to say that 41.71% of time was consumed waiting for CPU?? this is urgent
    Event                           Waits  Time(s)  Avg wait (ms)  % DB time  Wait Class
    db file sequential read       300,835    1,483              5      58.42  User I/O
    DB CPU                                   1,059                     41.71
    reliable message                9,499       18              2       0.71  Other
    PX Deq: Slave Session Stats     6,506       11              2       0.43  Other
    gc cr grant 2-way              26,218        6              0       0.25  Cluster
    When posting information to the forum that includes critical spaces, like the above, use a { code } tag (without spaces) before and after the information.
    I do not understand why this question is an urgent problem.
    It is incorrect to state that 41.71% of the time was consumed waiting for the CPU. When an Oracle process is running on the CPU, it is officially not waiting. It causes a bit of confusion having the CPU time consumed listed among the top 5 wait events, but as long as you understand why it is in the top 5 list, it almost makes sense for it to be included.
    The DB CPU statistic is listed as 1,059 seconds. If the duration of this report is 1 hour, that is 3,600 seconds of total time. If there is a single CPU in the server, there are 3,600 CPU seconds available in the time period, indicating that the server's CPU on average was 29.4% busy. If there were 12 CPUs in the server, there were 43,200 CPU seconds available in the time period, indicating that on average the CPUs were 2.5% busy. Does this mean that there was a problem, or was this OK, or is there not enough information? Just because on average the CPUs are not busy, that does not mean that there were not periods of intense CPU competion, where in fact there was a temporary shortage of available CPU time for processing.
    The DB Time statistic is supposed to be an indication of work performed by the instance on behalf of the user sessions. It is the accumulation of CPU time consumed by foreground sessions plus the accumulated sum of all non-idle wait events consumed by foreground sessions. Blog articles that might be of interest to you:
    http://hoopercharles.wordpress.com/2010/01/13/working-with-oracle-time-model-data/
    http://hoopercharles.wordpress.com/2010/02/05/faulty-quotes-6-cpu-utilization/
    Charles Hooper
    Co-author of "Expert Oracle Practices: Oracle Database Administration from the Oak Table"
    http://hoopercharles.wordpress.com/
    IT Manager/Oracle DBA
    K&M Machine-Fabricating, Inc.

Maybe you are looking for