Rac wait events

Hi ,
Could you please tell me the wait events in RAC
Please let me know the causes and the solution to fix this issue.
Thanks
Rangarajbk

Please read documentation for the same,
http://docs.oracle.com/cd/E11882_01/rac.112/e16795/monitor.htm#CFAGAAGD
Aman....

Similar Messages

  • Oracle RAC Wait events

    Sun OS 10
    Oracle 10.2.0.5
    We we are running 2 node RAC and we frequently seeing the following waits in the top 5 wait event
    cr request retry
    gcs log flush sync
    Couldn't locate these events in the database reference
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents.htm
    Thanks
    Saravanan

    gcs log flush sync is similar to log file sync in standalone:
    from - http://orainternals.files.wordpress.com/2010/02/riyaj_advanced_rac_troubleshooting_rmoug_2010_ppt.pdf (you might have more luck opening this one)
    Gcs log flush sync
    - But, if the instances crash right after the block is transferred to
    other node, how does RAC maintain consistency?
    -Actually, before sending a current mode block LMS process will
    request LGWR for a log flush.
    - Until LGWR sends a signal back to LMS process, LMS process
    will wait on ‘gcs log flush’ event.
    - CR block transfer might need log flush if the block was
    considered “busy”.
    - One of the busy condition is that if the block was constructed by
    applying undo records.
    cr request retry in some cases means that the message was lost and re-requested... this is tied to interconnect - either udp issues (like truncated udp packets or packets sent out of order), the session was lost on the other node, or the node restarted quickly... could also mean your nic might be flaky or something happening on the switch. If this is a big concern then you'll need to have someone look at the flow on in the interconnects as this is specific to cache fusion.

  • RAC specific Wait events in AWR

    DB version: 10.2.0.4
    OS : Solaris x86
    What are the most frequent RAC specific Wait events that appear in AWR reports ?

    Hi Pete,
    This depends on your environment. You can identify them as follows:
    Monitoring Oracle RAC Statistics and Wait Events
    http://download.oracle.com/docs/cd/E11882_01/rac.112/e16795/monitor.htm#i1010220
    ASH Report for Oracle RAC: Top Cluster Events
    The ASH report Top Cluster Events section is part of the Top Events report that is specific to Oracle RAC. The Top Cluster Events report lists events that account for the highest percentage of session activity in the cluster wait class event along with the instance number of the affected instances. You can use this information to identify which events and instances caused a high percentage of cluster wait events.
    http://download.oracle.com/docs/cd/E11882_01/rac.112/e16795/monitor.htm
    Regards,
    Levi Pereira
    <font size="1" color="red">Please close your thread when you get the solution to your problem.</font><br>
    <font size="1" color="red">Mark the replies answered "helpful" answer and/or "correct" answer that will help others with same problem.</font><br>
    <font size="1" color="red">Thanks for doing your part to make this community as valuable as possible for everyone!</font><br>

  • Oracle RAC 9i LMD library cache lock top wait event

    We are experiencing the library cache lock as our top wait event. Even thought the box is currently idle, The Global Enqueue Service Daemon (LMD) is taking up CPU cycles. The background process is also logging to trace "skgxpdocon: warning outstanding accept handle count has reached new high water mark 245000".
    Any help would be appreciated.
    Thanks

    There is a new patch for this - check out p4673610 on metalink. We have also experience the problem in 9.2.0.8.

  • Need help to analysis "foreground and background wait events" on statspack report for oracle database 11.2.0.4 on AIX

    Hi: I'm analyzing this STATSPACK report: it is "volume test" on our UAT server, so most input is from 'bind variables'.  Our shared pool is well utilized in oracle.  Oracle redo logs is not appropriately configured on this server, as in 'Top 5 wait events' there are 2 for redos.
    I need to know what else information can be dig-out from 'foreground wait events' & 'background wait events', and what can assist us to better understanding, in combination of 'Top 5 wait event's, that how the server/test went?  it could be overwelming No. of wait events, so appreciate any helpful diagnostic or analysis.  Database is oracle 11.2.0.4 upgraded from 11.2.0.3, on IBM AIX power system 64bit, level 6.x
    STATSPACK report for
    Database    DB Id    Instance     Inst Num  Startup Time   Release     RAC
    ~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
    700000XXX   XXX              1 22-Apr-15 12:12 11.2.0.4.0  NO
    Host Name             Platform                CPUs Cores Sockets   Memory (G)
    ~~~~ ---------------- ---------------------- ----- ----- ------- ------------
         dXXXX_XXX    AIX-Based Systems (64-     2     1       0         16.0
    Snapshot       Snap Id     Snap Time      Sessions Curs/Sess Comment
    ~~~~~~~~    ---------- ------------------ -------- --------- ------------------
    Begin Snap:       5635 22-Apr-15 13:00:02      114       4.6
      End Snap:       5636 22-Apr-15 14:00:01      128       8.8
       Elapsed:      59.98 (mins) Av Act Sess:       0.6
       DB time:      35.98 (mins)      DB CPU:      19.43 (mins)
    Cache Sizes            Begin        End
    ~~~~~~~~~~~       ---------- ----------
        Buffer Cache:     2,064M              Std Block Size:         8K
         Shared Pool:     3,072M                  Log Buffer:    13,632K
    Load Profile              Per Second    Per Transaction    Per Exec    Per Call
    ~~~~~~~~~~~~      ------------------  ----------------- ----------- -----------
          DB time(s):                0.6                0.0        0.00        0.00
           DB CPU(s):                0.3                0.0        0.00        0.00
           Redo size:          458,720.6            8,755.7
       Logical reads:           12,874.2              245.7
       Block changes:            1,356.4               25.9
      Physical reads:                6.6                0.1
    Physical writes:               61.8                1.2
          User calls:            2,033.7               38.8
              Parses:              286.5                5.5
         Hard parses:                0.5                0.0
    W/A MB processed:                1.7                0.0
              Logons:                1.2                0.0
            Executes:              801.1               15.3
           Rollbacks:                6.1                0.1
        Transactions:               52.4
    Instance Efficiency Indicators
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                Buffer Nowait %:  100.00       Redo NoWait %:  100.00
                Buffer  Hit   %:   99.98  Optimal W/A Exec %:  100.00
                Library Hit   %:   99.77        Soft Parse %:   99.82
             Execute to Parse %:   64.24         Latch Hit %:   99.98
    Parse CPU to Parse Elapsd %:   53.15     % Non-Parse CPU:   98.03
    Shared Pool Statistics        Begin   End
                 Memory Usage %:   10.50   12.79
        % SQL with executions>1:   69.98   78.37
      % Memory for SQL w/exec>1:   70.22   81.96
    Top 5 Timed Events                                                    Avg %Total
    ~~~~~~~~~~~~~~~~~~                                                   wait   Call
    Event                                            Waits    Time (s)   (ms)   Time
    CPU time                                                       847          50.2
    enq: TX - row lock contention                    4,480         434     97   25.8
    log file sync                                  284,169         185      1   11.0
    log file parallel write                        299,537         164      1    9.7
    log file sequential read                           698          16     24    1.0
    Host CPU  (CPUs: 2  Cores: 1  Sockets: 0)
    ~~~~~~~~              Load Average
                          Begin     End      User  System    Idle     WIO     WCPU
                           1.16    1.84     19.28   14.51   66.21    1.20   82.01
    Instance CPU
    ~~~~~~~~~~~~                                       % Time (seconds)
                         Host: Total time (s):                  7,193.8
                      Host: Busy CPU time (s):                  2,430.7
                       % of time Host is Busy:      33.8
                 Instance: Total CPU time (s):                  1,203.1
              % of Busy CPU used for Instance:      49.5
            Instance: Total Database time (s):                  2,426.4
      %DB time waiting for CPU (Resource Mgr):       0.0
    Memory Statistics                       Begin          End
    ~~~~~~~~~~~~~~~~~                ------------ ------------
                      Host Mem (MB):     16,384.0     16,384.0
                       SGA use (MB):      7,136.0      7,136.0
                       PGA use (MB):        282.5        361.4
        % Host Mem used for SGA+PGA:         45.3         45.8
    Foreground Wait Events  DB/Inst: XXXXXs  Snaps: 5635-5636
    -> Only events with Total Wait Time (s) >= .001 are shown
    -> ordered by Total Wait Time desc, Waits desc (idle events last)
                                                                 Avg          %Total
                                              %Tim Total Wait   wait    Waits   Call
    Event                               Waits  out   Time (s)   (ms)     /txn   Time
    enq: TX - row lock contentio        4,480    0        434     97      0.0   25.8
    log file sync                     284,167    0        185      1      1.5   11.0
    Disk file operations I/O            8,741    0          4      0      0.0     .2
    direct path write                  13,247    0          3      0      0.1     .2
    db file sequential read             6,058    0          1      0      0.0     .1
    buffer busy waits                   1,800    0          1      1      0.0     .1
    SQL*Net more data to client        29,161    0          1      0      0.2     .1
    direct path read                    7,696    0          1      0      0.0     .0
    db file scattered read                316    0          1      2      0.0     .0
    latch: shared pool                    144    0          0      2      0.0     .0
    CSS initialization                     30    0          0      3      0.0     .0
    cursor: pin S                          10    0          0      9      0.0     .0
    row cache lock                         41    0          0      2      0.0     .0
    latch: row cache objects               19    0          0      3      0.0     .0
    log file switch (private str            8    0          0      7      0.0     .0
    library cache: mutex X                 28    0          0      2      0.0     .0
    latch: cache buffers chains            54    0          0      1      0.0     .0
    latch free                            290    0          0      0      0.0     .0
    control file sequential read        1,568    0          0      0      0.0     .0
    log file switch (checkpoint             4    0          0      6      0.0     .0
    direct path sync                        8    0          0      3      0.0     .0
    latch: redo allocation                 60    0          0      0      0.0     .0
    SQL*Net break/reset to clien           34    0          0      1      0.0     .0
    latch: enqueue hash chains             45    0          0      0      0.0     .0
    latch: cache buffers lru cha            7    0          0      2      0.0     .0
    latch: session allocation               5    0          0      1      0.0     .0
    latch: object queue header o            6    0          0      1      0.0     .0
    ASM file metadata operation            30    0          0      0      0.0     .0
    latch: In memory undo latch            15    0          0      0      0.0     .0
    latch: undo global data                 8    0          0      0      0.0     .0
    SQL*Net message from client     6,362,536    0    278,225     44     33.7
    jobq slave wait                     7,270  100      3,635    500      0.0
    SQL*Net more data from clien        7,976    0         15      2      0.0
    SQL*Net message to client       6,362,544    0          8      0     33.7
    Background Wait Events  DB/Inst: XXXXXs  Snaps: 5635-5636
    -> Only events with Total Wait Time (s) >= .001 are shown
    -> ordered by Total Wait Time desc, Waits desc (idle events last)
                                                                 Avg          %Total
                                              %Tim Total Wait   wait    Waits   Call
    Event                               Waits  out   Time (s)   (ms)     /txn   Time
    log file parallel write           299,537    0        164      1      1.6    9.7
    log file sequential read              698    0         16     24      0.0    1.0
    db file parallel write              9,556    0         13      1      0.1     .8
    os thread startup                     146    0         10     70      0.0     .6
    control file parallel write         2,037    0          2      1      0.0     .1
    Log archive I/O                        35    0          1     30      0.0     .1
    LGWR wait for redo copy             2,447    0          0      0      0.0     .0
    db file async I/O submit            9,556    0          0      0      0.1     .0
    db file sequential read               145    0          0      2      0.0     .0
    Disk file operations I/O              349    0          0      0      0.0     .0
    db file scattered read                 30    0          0      4      0.0     .0
    control file sequential read        5,837    0          0      0      0.0     .0
    ADR block file read                    19    0          0      4      0.0     .0
    ADR block file write                    5    0          0     15      0.0     .0
    direct path write                      14    0          0      2      0.0     .0
    direct path read                        3    0          0      7      0.0     .0
    latch: shared pool                      3    0          0      6      0.0     .0
    log file single write                  56    0          0      0      0.0     .0
    latch: redo allocation                 53    0          0      0      0.0     .0
    latch: active service list              1    0          0      3      0.0     .0
    latch free                             11    0          0      0      0.0     .0
    rdbms ipc message                 314,523    5     57,189    182      1.7
    Space Manager: slave idle wa        4,086   88     18,996   4649      0.0
    DIAG idle wait                      7,185  100      7,186   1000      0.0
    Streams AQ: waiting for time            2   50      4,909 ######      0.0
    Streams AQ: qmn slave idle w          129    0      3,612  28002      0.0
    Streams AQ: qmn coordinator           258   50      3,612  14001      0.0
    smon timer                             43    2      3,605  83839      0.0
    pmon timer                          1,199   99      3,596   2999      0.0
    SQL*Net message from client        17,019    0         31      2      0.1
    SQL*Net message to client          12,762    0          0      0      0.1
    class slave wait                       28    0          0      0      0.0
    thank you very much!

    Hi: just know it now: it is a large amount of 'concurrent transaction' designed in this "Volume Test" - to simulate large incoming transaction volme, so I guess wait in eq:TX - row is expected.
    The fact: (1) redo logs at uat server is known to not well-tune for configurations (2) volume test slow 5%, however data amount in its test is kept the same by each time import  production data, by the team. So why it slowed 5% this year?
    The wait histogram is pasted below, any one interest to take a look?  any ideas?
    Wait Event Histogram  DB/Inst: XXXX/XXXX  Snaps: 5635-5636
    -> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
    -> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
    -> % of Waits - value: .0 indicates value was <.05%, null is truly 0
    -> Ordered by Event (idle events last)
                               Total ----------------- % of Waits ------------------
    Event                      Waits  <1ms  <2ms  <4ms  <8ms <16ms <32ms  <=1s   >1s
    ADR block file read          19   26.3   5.3  10.5  57.9
    ADR block file write          5                     40.0        60.0
    ADR file lock                 6  100.0
    ARCH wait for archivelog l   14  100.0
    ASM file metadata operatio   30  100.0
    CSS initialization           30              100.0
    Disk file operations I/O   9090   97.2   1.4    .6    .4    .2    .1    .1
    LGWR wait for redo copy    2447   98.5    .5    .4    .2    .2    .2    .1
    Log archive I/O              35   40.0         8.6  25.7   2.9        22.9
    SQL*Net break/reset to cli   34   85.3   8.8         5.9
    SQL*Net more data to clien   29K  99.9    .0    .0    .0          .0    .0
    buffer busy waits          1800   96.8    .7    .7    .6    .3    .4    .5
    control file parallel writ 2037   90.7   5.0   2.1    .8   1.0    .3    .1
    control file sequential re 7405  100.0                      .0
    cursor: pin S                10   10.0                    90.0
    db file async I/O submit   9556   99.9    .0                .0          .0
    db file parallel read         1  100.0
    db file parallel write     9556   62.0  32.4   1.7    .8   1.5   1.3    .1
    db file scattered read      345   72.8   3.8   2.3  11.6   9.0    .6
    db file sequential read    6199   97.2    .2    .3   1.6    .7    .0    .0
    direct path read           7699   99.1    .4    .2    .1    .1    .0
    direct path sync              8   25.0  37.5  12.5  25.0
    direct path write            13K  97.8    .9    .5    .4    .3    .1    .0
    enq: TX - row lock content 4480     .4    .7   1.3   3.0   6.8  12.3  75.4    .1
    latch free                  301   98.3    .3    .7    .7
    latch: In memory undo latc   15   93.3   6.7
    latch: active service list    1              100.0
    latch: cache buffers chain   55   94.5                     3.6   1.8
    latch: cache buffers lru c    9   88.9                    11.1
    latch: call allocation        6  100.0
    latch: checkpoint queue la    3  100.0
    latch: enqueue hash chains   45   97.8                     2.2
    latch: messages               4  100.0
    latch: object queue header    7   85.7        14.3
    latch: redo allocation      113   97.3               1.8    .9
    latch: row cache objects     19   89.5                           5.3   5.3
    latch: session allocation     5   80.0              20.0
    latch: shared pool          147   90.5   1.4   2.7   1.4    .7   1.4   2.0
    latch: undo global data       8  100.0
    library cache: mutex X       28   89.3         3.6         3.6         3.6
    log file parallel write     299K  95.6   2.6   1.0    .4    .3    .2    .0
    log file sequential read    698   29.5    .1               4.6  46.8  18.9
    log file single write        56  100.0
    log file switch (checkpoin    4               25.0  50.0  25.0
    log file switch (private s    8         12.5        37.5  50.0
    log file sync               284K  93.3   3.7   1.4    .7    .5    .3    .1
    os thread startup           146                                      100.0
    row cache lock               41   85.4   9.8               2.4         2.4
    DIAG idle wait             7184                                      100.0
    SQL*Net message from clien 6379K  86.6   5.1   2.9   1.3    .7    .3   2.8    .3
    SQL*Net message to client  6375K 100.0    .0    .0    .0    .0    .0    .0
    Wait Event Histogram  DB/Inst: XXXX/xxxx  Snaps: 5635-5636
    -> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
    -> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
    -> % of Waits - value: .0 indicates value was <.05%, null is truly 0
    -> Ordered by Event (idle events last)
                               Total ----------------- % of Waits ------------------
    Event                      Waits  <1ms  <2ms  <4ms  <8ms <16ms <32ms  <=1s   >1s
    SQL*Net more data from cli 7976   99.7    .1    .1    .0                      .1
    Space Manager: slave idle  4086     .1    .2    .0    .0    .3         3.2  96.1
    Streams AQ: qmn coordinato  258   49.2                .8                    50.0
    Streams AQ: qmn slave idle  129                                            100.0
    Streams AQ: waiting for ti    2   50.0                                      50.0
    class slave wait             28   92.9   3.6   3.6
    jobq slave wait            7270     .0                               100.0
    pmon timer                 1199                                            100.0
    rdbms ipc message           314K  10.3   7.3  39.7  15.4  10.6   5.3   8.2   3.3
    smon timer                   43                                            100.0

  • Find the Wait events for an objects.

    Hi Gurus.
    I need you help to find out the wait events for a particular table and index when batch export job running from the application. I have done some work around to to find out, but i am not able to figure out the issue or what are the wait events. i am working on linux 4node rac 11gR2.
    Below are the steps i have done.
    1. find object_id for a table finding the wait events to that object_id.
    SYS@ccmintpt1 SQL>select data_object_id from dba_objects where object_name='ACCOUNT_DETAIL_DENORM';
    DATA_OBJECT_ID
    82646
    88518
    84184
    SYS@ccmintpt1 SQL>select w.sid,w.event,w.wait_class,w.wait_class# from gv$session s,gv$session_wait w where s.row_wait_obj#=82646;
    no rows selected
    SYS@ccmintpt1 SQL>select w.sid,w.event,w.wait_class,w.wait_class# from gv$session s,gv$session_wait w where s.row_wait_obj#=88518;
    no rows selected
    SYS@ccmintpt1 SQL>select w.sid,w.event,w.wait_class,w.wait_class# from gv$session s,gv$session_wait w where s.row_wait_obj#=84184;
    no rows selected
    2. find the object id for the index of that table and finding the wait events to that object_id.
    SYS@ccmintpt1 SQL>select data_object_id from dba_objects where object_name='XAK2ACCOUNT_DETAIL_DENORM';
    DATA_OBJECT_ID
    88655
    SID EVENT WAIT_CLASS WAIT_CLASS#
    1 rdbms ipc message Idle 6
    2 heartbeat monitor sleep Idle 6
    3 rdbms ipc message Idle 6
    4 wait for unread message on broadcast channel Idle 6
    6 SQL*Net message from client Idle 6
    7 SQL*Net message from client Idle 6
    8 SQL*Net message from client Idle 6
    9 SQL*Net message from client Idle 6
    10 SQL*Net message from client Idle 6
    11 SQL*Net message from client Idle 6
    12 SQL*Net message from client Idle 6
    13 SQL*Net message from client Idle 6
    1876 SQL*Net message from client Idle 6
    1877 VKTM Logical Idle Wait Idle 6
    1878 rdbms ipc message Idle 6
    1879 rdbms ipc message Idle 6
    1880 rdbms ipc message Idle 6
    1881 SQL*Net message from client Idle 6
    3200 SQL*Net message from client Idle 6
    3210 SQL*Net message from client Idle 6
    3212 db file sequential read User I/O 8
    3213 SQL*Net message from client Idle 6
    3214 SQL*Net message from client Idle 6
    3216 SQL*Net message from client Idle 6
    3751 rdbms ipc message Idle 6
    3752 gcs remote message Idle 6
    3754 rdbms ipc message Idle 6
    3756 SQL*Net message from client Idle 6
    3757 SQL*Net message from client Idle 6
    1326 SQL*Net message from client Idle 6
    1876 Streams AQ: waiting for time management or cleanup tasks Idle 6
    1877 VKTM Logical Idle Wait Idle 6
    1878 rdbms ipc message Idle 6
    1879 rdbms ipc message Idle 6
    1880 rdbms ipc message Idle 6
    1882 SQL*Net message from client Idle 6
    3751 rdbms ipc message Idle 6
    3752 gcs remote message Idle 6
    3754 rdbms ipc message Idle 6
    3756 rdbms ipc message Idle 6
    3758 SQL*Net message from client Idle 6
    3759 PX Deq: Execution Msg Idle 6
    3200 SQL*Net message from client Idle 6
    3210 SQL*Net message from client Idle 6
    3212 db file sequential read User I/O 8
    3213 SQL*Net message from client Idle 6
    3214 SQL*Net message from client Idle 6
    3216 SQL*Net message from client Idle 6
    3751 rdbms ipc message Idle 6
    3752 gcs remote message Idle 6
    3754 rdbms ipc message Idle 6
    3756 SQL*Net message from client Idle 6
    3757 SQL*Net message from client Idle 6
    3786 SQL*Net message from client Idle 6
    3787 SQL*Net message from client Idle 6
    1 heartbeat monitor sleep Idle 6
    2 wait for unread message on broadcast channel Idle 6
    314 rdbms ipc message Idle 6
    315 Streams AQ: qmn coordinator idle wait Idle 6
    627 pmon timer Idle 6
    628 rdbms ipc message Idle 6
    629 Streams AQ: waiting for messages in the queue Idle 6
    940 Streams AQ: waiting for time management or cleanup tasks Idle 6
    941 VKTM Logical Idle Wait Idle 6
    942 rdbms ipc message Idle 6
    1253 rdbms ipc message Idle 6
    1254 rdbms ipc message Idle 6
    1256 Space Manager: slave idle wait Idle 6
    1566 DIAG idle wait Idle 6
    1567 rdbms ipc message Idle 6
    1568 Streams AQ: qmn slave idle wait Idle 6
    1879 rdbms ipc message Idle 6
    1880 smon timer Idle 6
    1881 rdbms ipc message Idle 6
    2192 PING Idle 6
    2193 rdbms ipc message Idle 6
    2194 rdbms ipc message Idle 6
    2505 rdbms ipc message Idle 6
    2506 rdbms ipc message Idle 6
    2818 rdbms ipc message Idle 6
    2819 rdbms ipc message Idle 6
    2821 PX Deq: reap credit Other 0
    3131 DIAG idle wait Idle 6
    3132 jobq slave wait Idle 6
    3444 rdbms ipc message Idle 6
    3445 jobq slave wait Idle 6
    3757 ges remote message Idle 6
    3758 rdbms ipc message Idle 6
    3759 VKRM Idle Idle 6
    4070 gcs remote message Idle 6
    4072 rdbms ipc message Idle 6
    4383 gcs remote message Idle 6
    4385 PX Deq: Execution Msg Idle 6
    4696 rdbms ipc message Idle 6
    4697 rdbms ipc message Idle 6
    1 rdbms ipc message Idle 6
    2 heartbeat monitor sleep Idle 6
    3 rdbms ipc message Idle 6
    4 wait for unread message on broadcast channel Idle 6
    6 SQL*Net message from client Idle 6
    From this output, what are the waits events i need to take into consideration.
    Is this the right approach to follow?
    Please let me know.
    Thanks in advance.

    user13162661 wrote:
    Hi Gurus.
    I need you help to find out the wait events for a particular table and index when batch export job running from the application. I have done some work around to to find out, but i am not able to figure out the issue or what are the wait events. i am working on linux 4node rac 11gR2.
    Below are the steps i have done.
    1. find object_id for a table finding the wait events to that object_id.
    SYS@ccmintpt1 SQL>select data_object_id from dba_objects where object_name='ACCOUNT_DETAIL_DENORM';
    DATA_OBJECT_ID
    82646
    88518
    84184
    SYS@ccmintpt1 SQL>select w.sid,w.event,w.wait_class,w.wait_class# from gv$session s,gv$session_wait w where s.row_wait_obj#=82646;
    no rows selected
    SYS@ccmintpt1 SQL>select w.sid,w.event,w.wait_class,w.wait_class# from gv$session s,gv$session_wait w where s.row_wait_obj#=88518;
    no rows selected
    SYS@ccmintpt1 SQL>select w.sid,w.event,w.wait_class,w.wait_class# from gv$session s,gv$session_wait w where s.row_wait_obj#=84184;
    no rows selected
    2. find the object id for the index of that table and finding the wait events to that object_id.
    SYS@ccmintpt1 SQL>select data_object_id from dba_objects where object_name='XAK2ACCOUNT_DETAIL_DENORM';
    DATA_OBJECT_ID
    88655
    SID EVENT WAIT_CLASS WAIT_CLASS#
    1 rdbms ipc message Idle 6
    2 heartbeat monitor sleep Idle 6
    3 rdbms ipc message Idle 6
    4 wait for unread message on broadcast channel Idle 6
    6 SQL*Net message from client Idle 6
    7 SQL*Net message from client Idle 6
    8 SQL*Net message from client Idle 6
    9 SQL*Net message from client Idle 6
    10 SQL*Net message from client Idle 6
    11 SQL*Net message from client Idle 6
    12 SQL*Net message from client Idle 6
    13 SQL*Net message from client Idle 6
    1876 SQL*Net message from client Idle 6
    1877 VKTM Logical Idle Wait Idle 6
    1878 rdbms ipc message Idle 6
    1879 rdbms ipc message Idle 6
    1880 rdbms ipc message Idle 6
    1881 SQL*Net message from client Idle 6
    3200 SQL*Net message from client Idle 6
    3210 SQL*Net message from client Idle 6
    3212 db file sequential read User I/O 8
    3213 SQL*Net message from client Idle 6
    3214 SQL*Net message from client Idle 6
    3216 SQL*Net message from client Idle 6
    3751 rdbms ipc message Idle 6
    3752 gcs remote message Idle 6
    3754 rdbms ipc message Idle 6
    3756 SQL*Net message from client Idle 6
    3757 SQL*Net message from client Idle 6
    1326 SQL*Net message from client Idle 6
    1876 Streams AQ: waiting for time management or cleanup tasks Idle 6
    1877 VKTM Logical Idle Wait Idle 6
    1878 rdbms ipc message Idle 6
    1879 rdbms ipc message Idle 6
    1880 rdbms ipc message Idle 6
    1882 SQL*Net message from client Idle 6
    3751 rdbms ipc message Idle 6
    3752 gcs remote message Idle 6
    3754 rdbms ipc message Idle 6
    3756 rdbms ipc message Idle 6
    3758 SQL*Net message from client Idle 6
    3759 PX Deq: Execution Msg Idle 6
    3200 SQL*Net message from client Idle 6
    3210 SQL*Net message from client Idle 6
    3212 db file sequential read User I/O 8
    3213 SQL*Net message from client Idle 6
    3214 SQL*Net message from client Idle 6
    3216 SQL*Net message from client Idle 6
    3751 rdbms ipc message Idle 6
    3752 gcs remote message Idle 6
    3754 rdbms ipc message Idle 6
    3756 SQL*Net message from client Idle 6
    3757 SQL*Net message from client Idle 6
    3786 SQL*Net message from client Idle 6
    3787 SQL*Net message from client Idle 6
    1 heartbeat monitor sleep Idle 6
    2 wait for unread message on broadcast channel Idle 6
    314 rdbms ipc message Idle 6
    315 Streams AQ: qmn coordinator idle wait Idle 6
    627 pmon timer Idle 6
    628 rdbms ipc message Idle 6
    629 Streams AQ: waiting for messages in the queue Idle 6
    940 Streams AQ: waiting for time management or cleanup tasks Idle 6
    941 VKTM Logical Idle Wait Idle 6
    942 rdbms ipc message Idle 6
    1253 rdbms ipc message Idle 6
    1254 rdbms ipc message Idle 6
    1256 Space Manager: slave idle wait Idle 6
    1566 DIAG idle wait Idle 6
    1567 rdbms ipc message Idle 6
    1568 Streams AQ: qmn slave idle wait Idle 6
    1879 rdbms ipc message Idle 6
    1880 smon timer Idle 6
    1881 rdbms ipc message Idle 6
    2192 PING Idle 6
    2193 rdbms ipc message Idle 6
    2194 rdbms ipc message Idle 6
    2505 rdbms ipc message Idle 6
    2506 rdbms ipc message Idle 6
    2818 rdbms ipc message Idle 6
    2819 rdbms ipc message Idle 6
    2821 PX Deq: reap credit Other 0
    3131 DIAG idle wait Idle 6
    3132 jobq slave wait Idle 6
    3444 rdbms ipc message Idle 6
    3445 jobq slave wait Idle 6
    3757 ges remote message Idle 6
    3758 rdbms ipc message Idle 6
    3759 VKRM Idle Idle 6
    4070 gcs remote message Idle 6
    4072 rdbms ipc message Idle 6
    4383 gcs remote message Idle 6
    4385 PX Deq: Execution Msg Idle 6
    4696 rdbms ipc message Idle 6
    4697 rdbms ipc message Idle 6
    1 rdbms ipc message Idle 6
    2 heartbeat monitor sleep Idle 6
    3 rdbms ipc message Idle 6
    4 wait for unread message on broadcast channel Idle 6
    6 SQL*Net message from client Idle 6
    From this output, what are the waits events i need to take into consideration.
    Is this the right approach to follow?
    Please let me know.
    Thanks in advance.what actual problem are you trying to solve?
    post SQL & results that have you concerned about wait events

  • What is "KJC: Wait for msg .." wait event in 10g??

    Hi, all.
    The database is 2 node RAC database (10.2.0.2.0)
    on 32-bit windows 2003 EE SP1.
    I found "KJC: Wait for msg sends to complete" wait event in
    "Top 5 Timed Event" Section from AWR report.
    What is "KJC: Wait for msg sends to complete" wait event??
    The following is from UDUMP.
    Dump file d:\oracle\product\10.2.0\admin\rac\udump\rac2_ora_5656.trc
    Mon Sep 24 00:04:40 2007
    ORACLE V10.2.0.2.0 - Production vsnsta=0
    vsnsql=14 vsnxtr=3
    Oracle Database 10g Enterprise Edition Release 10.2.0.2.0 - Production
    With the Real Application Clusters, OLAP and Data Mining options
    Windows Server 2003 Version V5.2 Service Pack 1
    CPU : 4 - type 586, 2 Physical Cores
    Process Affinity : 0x00000000
    Memory (Avail/Total): Ph:5278M/8190M, Ph+PgF:6596M/10041M, VA:316M/2047M
    Instance name: rac2
    Redo thread mounted by this instance: 2
    Oracle process number: 64
    Windows thread id: 5656, image: ORACLE.EXE (SHAD)
    *** 2007-09-24 00:04:40.156
    *** ACTION NAME:() 2007-09-24 00:04:40.156
    *** MODULE NAME:(OEM.SystemPool) 2007-09-24 00:04:40.156
    *** SERVICE NAME:(RAC.world) 2007-09-24 00:04:40.156
    *** CLIENT ID:() 2007-09-24 00:04:40.156
    *** SESSION ID:(486.53) 2007-09-24 00:04:40.156
    IPCSendMsg: could not initiate send on conn 0x5b0d3e98 to node [rac1 : 696 : 3996 : 359937], err 10054
    IPCGetRequestInfo: failed a request rqh(0x5b060db8), type(6), status(2), bytes(0)
    Thanks and Regards.
    Message was edited by:
    user507290
    Message was edited by:
    user507290

    This might have something to do with bug 5075434 - Small performance overhead in RAC (waits for "KJC: Wait for msg sends to complete").
    Check metalink for further details.

  • Wait events

    Hi ! I have the following wait events in my top timed and I don't know who originated them:
    Wait Event Wait Time Summary Avg Wait Time (ms)
    I# Class Event Waits %Timeouts Total(s) Avg(ms) %DB time Avg Min Max Std Dev Cnt
    * DB CPU N/A N/A 59,651.48 N/A 45.87 2
    User I/O db file sequential read 4,369,213 0.0 20,831.46 4.8 16.02 4.72 4.29 5.14 0.60 2
    Other enq: CF - contention 155,822 3.9 10,390.74 66.7 7.99 68.62 60.31 76.94 11.76 2
    System I/O RMAN backup & recovery I/O 87,205 0.0 5,477.09 62.8 9.15 62.81 62.81 62.81 1
    Cluster gc current block 2-way 2,914,457 0.0 4,811.61 1.7 3.70 1.67 1.60 1.74 0.10 2
    System I/O control file sequential read 3,038,672 0.0 3,762.66 1.2 2.89 1.24 1.22 1.27 0.04 2
    Concurrenc os thread startup 2,842 0.0 3,695.14 1300.2 2.84 1311.83 1143.07 1480.59 238.66 2
    System I/O log file parallel write 1,341,907 0.0 2,530.17 1.9 1.95 1.88 1.88 1.89 0.01 2
    Other reliable message 471,495 0.1 2,388.01 5.1 1.84 5.08 4.12 6.03 1.35 2
    Concurrenc row cache lock 3,135,774 0.0 2,224.53 0.7 1.71 0.72 0.68 0.75 0.05 2
    1 DB CPU N/A N/A 22,584.30 N/A 37.75
    User I/O db file sequential read 2,451,215 0.0
    System I/O RMAN backup & recovery I/O 87,205 0.0
    Other enq: CF - contention 59,735 5.3
    Cluster gc current block 2-way 1,803,542 0.0
    System I/O control file sequential read 1,831,180 0.0
    Concurrenc os thread startup 1,323 0.0
    System I/O log file parallel write 727,883 0.0
    Cluster gc cr multi block request 523,744 0.0
    Concurrenc row cache lock 1,830,913 0.0
    2 DB CPU N/A N/A
    User I/O db file sequential read 1,917,998 0.0
    Other enq: CF - contention 96,087 3.0
    Cluster gc current block 2-way 1,110,915 0.0
    Concurrenc os thread startup 1,519 0.0
    System I/O control file sequential read 1,207,492 0.0
    User I/O direct path read 404,587 0.0
    Other reliable message 233,033 0.1
    System I/O log file parallel write 614,024 0.0
    System I/O control file parallel write 128,905 0.0
    Those are the most worrying events:
    enq: CF - contention
    I/O control file sequential read
    Concurrenc os thread startup
    I have been investigating and I wonder what is wrong to get Concurrenc os thread startup. According to one blog, os thread should be always related with network issues...
    The awr snapshot is from my production window day.
    Rac 11.2.0.3 two nodes on Solaris Sparc 10.

    I have studied the ASH in the problematic period and I have found that there are some full scans:
    Summary of All User Input
    Format         : TEXT
    DB Id          : 2752323407
    Inst num       : 1
    Begin time     : 08-Feb-13 09:30:00
    End time       : 08-Feb-13 10:20:00
    Slot width     : Default
    Report targets : 0
    Report name    : ashrpt_1_0208_1020.txt
    ASH Report For dbp/dbp1
    DB Name         DB Id    Instance     Inst Num Release     RAC Host
    dbp           2752323407 dbp1                1 11.2.0.3.0  YES host-dbp-1
    CPUs           SGA Size       Buffer Cache        Shared Pool    ASH Buffer Size
      16     12,651M (100%)    10,048M (79.4%)     1,921M (15.2%)       32.0M (0.3%)
              Analysis Begin Time:   08-Feb-13 09:30:00
                Analysis End Time:   08-Feb-13 10:20:00
                     Elapsed Time:        50.0 (mins)
                Begin Data Source:   DBA_HIST_ACTIVE_SESS_HISTORY
                                     in AWR snapshot 5100
                  End Data Source:   DBA_HIST_ACTIVE_SESS_HISTORY
                                     in AWR snapshot 5101
                                      + V$ACTIVE_SESSION_HISTORY
                     Sample Count:      10,069
          Average Active Sessions:       33.56
      Avg. Active Session per CPU:        2.10
                    Report Target:   None specified
    Top User Events                     DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                                                                   Avg Active
    Event                               Event Class        % Event   Sessions
    library cache lock                  Concurrency          43.73      14.68
    cursor: pin S wait on X             Concurrency          18.61       6.25
    CPU + Wait for CPU                  CPU                  15.77       5.29
    reliable message                    Other                 5.88       1.97
    enq: KO - fast object checkpoint    Application           3.48       1.17
    Top Background Events               DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                                                                   Avg Active
    Event                               Event Class     % Activity   Sessions
    CPU + Wait for CPU                  CPU                   1.25       0.42
    Top Cluster Events                  DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    Event                          % Event Remote I % Activity
    gc current block 2-way            1.72        2       1.72
    gc cr grant 2-way                 1.58      N/A       1.07
    Top Event P1/P2/P3 Values           DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    Event                          % Event  P1 Value, P2 Value, P3 Value % Activity
    Parameter 1                Parameter 2                Parameter 3
    library cache lock               43.75 "29115227816","29218763456","       1.22
    handle address             lock address               100*mode+namespace
                                           "29115227816","28694732944","       1.20
                                           "29115227816","28812373936","       1.17
    cursor: pin S wait on X          18.61 "1497800770","3934190043136",       1.54
    idn                        value                      where
                                           "1497800770","7773890805760",       1.15
    reliable message                  6.07 "30432532808","30354909248","       0.13
    channel context            channel handle             broadcast message
    enq: KO - fast object checkpoi    3.49      "1263468550","65640","1"       0.52
    name|mode                  2                          0
    db file sequential read           1.81               "1","25220","1"       0.01
    file#                      block#                     blocks
    Top Service/Module                  DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    Service        Module                   % Activity Action               % Action
    dbp_DVEBMGS11  CL_SQL_STATEMENT========      86.80 383                     86.80
    dbp_D10_0066   CL_SQL_STATEMENT========       6.28 383                      3.34
                                                       104                      2.94
    dbp_D10_0064   CL_SQL_STATEMENT========       2.40 383                      2.39
    SYS$BACKGROUND UNNAMED                        1.51 UNNAMED                  1.51
    Top Client IDs                      DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                      No data exists for this section of the report.
    Top SQL Command Types               DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    -> 'Distinct SQLIDs' is the count of the distinct number of SQLIDs
          with the given SQL Command Type found over all the ASH samples
          in the analysis period
                                               Distinct            Avg Active
    SQL Command Type                             SQLIDs % Activity   Sessions
    SELECT                                          485      94.56      31.74
    ALTER TABLE                                     220       2.89       0.97
    Top Phases of Execution             DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                                              Avg Active
    Phase of Execution             % Activity   Sessions
    Parse                               67.50      22.66
    SQL Execution                       30.46      10.22
    Hard Parse                           5.37       1.80
    Top Remote Instances                DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    Wait Class           % Wait  Remote I % Activity
    Cluster                 5.22        2       3.90
                                      N/A       1.27
    Top SQL with Top Events     DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                                                            Sampled #
                     SQL ID             Planhash        of Executions     % Activity
    Event                          % Event Top Row Source                    % RwSrc
              350v06jcnd822                  N/A                    0          18.03
    library cache lock                9.41 ** Row Source Not Available **       9.41
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
                                             N/A                    0          18.03
    cursor: pin S wait on X           8.62 ** Row Source Not Available **       8.62
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
              48unmtd983uz6                  N/A                    0          16.75
    library cache lock               12.87 ** Row Source Not Available **      12.87
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
                                             N/A                    0          16.75
    cursor: pin S wait on X           3.88 ** Row Source Not Available **       3.88
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
              350v06jcnd822           2426825131                    0          15.49
    library cache lock                9.74 ** Row Source Not Available **       9.74
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
    cursor: pin S wait on X           4.14 ** Row Source Not Available **       4.14
    CPU + Wait for CPU                1.61 SELECT STATEMENT                     1.58
              48unmtd983uz6           3511339786                    0          14.98
    library cache lock               11.50 ** Row Source Not Available **      11.50
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
    cursor: pin S wait on X           1.97 ** Row Source Not Available **       1.97
    CPU + Wait for CPU                1.51 SELECT STATEMENT                     1.42
              07tcvyb6frtkx           2929764020                    1           1.87
    gc cr grant 2-way                 0.80 TABLE ACCESS - BY USER ROWID         0.75
    SELECT "D3"."SID_0SHIP_TO" AS "SID" FROM "/BIC/FZ99IC035" "F" JOIN "/BIC/DZ99IC
    0352" "D2" ON "F" . "KEY_Z99IC0352" = "D2" . "DIMID" JOIN "/BI0/XMATERIAL" "X9"
    ON "D2" . "SID_0MATERIAL" = "X9" . "SID" JOIN "/BIC/DZ99IC0355" "D5" ON "F" .
    "KEY_Z99IC0355" = "D5" . "DIMID" JOIN "/BIC/DZ99IC0353" "D3" ON "F" . "KEY_Z99
    Top SQL with Top Events     DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                                                            Sampled #
                     SQL ID             Planhash        of Executions     % Activity
    Event                          % Event Top Row Source                    % RwSrc
    Top SQL with Top Row Sources        DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                                                            Sampled #
                     SQL ID             PlanHash        of Executions     % Activity
    Row Source                               % RwSrc Top Event               % Event
              350v06jcnd822                  N/A                    0          18.03
    ** Row Source Not Available **             18.03 library cache lock         9.41
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
              48unmtd983uz6                  N/A                    0          16.75
    ** Row Source Not Available **             16.75 library cache lock        12.87
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
              350v06jcnd822           2426825131                    0          15.49
    ** Row Source Not Available **             13.91 library cache lock         9.74
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
    SELECT STATEMENT                            1.58 CPU + Wait for CPU         1.58
              48unmtd983uz6           3511339786                    0          14.98
    ** Row Source Not Available **             13.56 library cache lock        11.50
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
    SELECT STATEMENT                            1.42 CPU + Wait for CPU         1.42
              07tcvyb6frtkx           2929764020                    1           1.87
    TABLE ACCESS - BY USER ROWID                1.59 gc cr grant 2-way          0.75
    SELECT "D3"."SID_0SHIP_TO" AS "SID" FROM "/BIC/FZ99IC035" "F" JOIN "/BIC/DZ99IC
    0352" "D2" ON "F" . "KEY_Z99IC0352" = "D2" . "DIMID" JOIN "/BI0/XMATERIAL" "X9"
    ON "D2" . "SID_0MATERIAL" = "X9" . "SID" JOIN "/BIC/DZ99IC0355" "D5" ON "F" .
    "KEY_Z99IC0355" = "D5" . "DIMID" JOIN "/BIC/DZ99IC0353" "D3" ON "F" . "KEY_Z99
    Top SQL using literals              DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    -> FORCE_MATCHING_SIGNATURE is used to identify SQL statements that are
          identical except for their use of literals.
    -> Please refer to the Oracle Database Reference to understand how
          the FORCE_MATCHING_SIGNATURE is derived.
                                         # of Sampled
    FORCE_MATCHING_SIGNATURE % Activity SQL Versions
    Example SQL 1
    Example SQL 2
          1021017294885722791       2.89          218
    0htvt0stu1vtq
    SELECT COUNT(*) FROM "/BIC/FZ99IC003" WHERE "KEY_Z99IC003P" = :A0
    0htvt0stu1vtq
    Top Parsing Module/Action           DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    Module                         Action                           % Activ Event
    CL_SQL_STATEMENT============== 383                                67.25 library
                                   383                                      cursor:
                                   383                                      CPU + Wa
    Top Sessions running PQs            DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    -> This section aggregates all the work done by the PQ slaves into
          the session issuing the parallel query.
    Sid,Srl# (Inst) % Activity SQL ID        Event                          % Event
    User                 Program
      1506,   19(1)      33.57 350v06jcnd822 library cache lock               19.15
    UserID:
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
                                             cursor: pin S wait on X          12.76
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
                                             CPU + Wait for CPU                1.61
    SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
    "DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
    ___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
      2255, 1067(1)      31.78 48unmtd983uz6 library cache lock               24.37
    UserID:
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
                                             cursor: pin S wait on X           5.85
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
                                             CPU + Wait for CPU                1.51
    SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
    3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
    YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
    AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
    Top DB Objects                      DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    -> With respect to Application, Cluster, User I/O and buffer busy waits only.
          Object ID % Activity Event                             % Event
    Object Name (Type)                                    Tablespace
           13661539       2.45 gc buffer busy acquire               0.87
    SAPSR3./BIC/EZ99IC013 (TABLE)                         PSAPSR3SSD
    Top DB Files                        DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    -> With respect to Cluster and User I/O events only.
            File ID % Activity Event                             % Event
    File Name                                             Tablespace
                 53       3.60 gc current block 2-way               0.98
    +dbp_DATA/dbp_2/datafile/psapsr3ssd.315.805562113     PSAPSR3SSD
    Top Latches                         DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
                      No data exists for this section of the report.
    Activity Over Time                  DB/Inst: dbp/dbp1  (Feb 08 09:30 to 10:20)
    -> Analysis period is divided into smaller time slots
    -> Top 3 events are reported in each of those slots
    -> 'Slot Count' shows the number of ASH samples in that slot
    -> 'Event Count' shows the number of ASH samples waiting for
       that event in that slot
    -> '% Event' is 'Event Count' over all ASH samples in the analysis period
                             Slot                                   Event
    Slot Time (Duration)    Count Event                             Count % Event
    09:30:00   (5.0 min)      260 gc buffer busy acquire               43    0.43
                                  reliable message                     34    0.34
                                  CPU + Wait for CPU                   29    0.29
    09:35:00   (5.0 min)      303 CPU + Wait for CPU                   76    0.75
                                  db file sequential read              40    0.40
                                  gc buffer busy acquire               39    0.39
    09:40:00   (5.0 min)      366 CPU + Wait for CPU                  209    2.08
                                  db file sequential read              26    0.26
                                  gc current block 2-way               22    0.22
    09:45:00   (5.0 min)      511 CPU + Wait for CPU                  249    2.47
                                  cursor: pin S wait on X              93    0.92
                                  reliable message                     45    0.45
    09:50:00   (5.0 min)    2,245 cursor: pin S wait on X           1,442   14.32
                                  library cache lock                  407    4.04
                                  reliable message                    112    1.11
    09:55:00   (5.0 min)    2,037 library cache lock                1,378   13.69
                                  cursor: pin S wait on X             297    2.95
                                  CPU + Wait for CPU                  125    1.24
    10:00:00   (5.0 min)    1,823 library cache lock                1,371   13.62
                                  CPU + Wait for CPU                  263    2.61
                                  reliable message                     72    0.72
    10:05:00   (5.0 min)    1,273 library cache lock                  866    8.60
                                  CPU + Wait for CPU                  155    1.54
                                  reliable message                     96    0.95
    10:10:00   (5.0 min)      798 library cache lock                  350    3.48
                                  CPU + Wait for CPU                  287    2.85
                                  reliable message                     54    0.54
    10:15:00   (5.0 min)      436 CPU + Wait for CPU                  200    1.99
                                  reliable message                     61    0.61
                                  enq: KO - fast object checkpoi       42    0.42
              -------------------------------------------------------------Problems are always on instance 1.
    The queries are different each day, the top sql with performance problem changes the sql_id and I cant attack them or apply a sql profile or tune them because they only execute during a period.
    Any idea?
    :(

  • What is "reliable message" wait event in 10g??

    Hi, all.
    The database is 2 node RAC 10.2.0.2.0 on 32-bit windows 2003 EE SP1.
    From time to time, I see "reliable message" wait event on the
    "Top 5 Timed Events" section from AWR report.
    The wait class is "Other".
    There seems to be no manuals and html links explaining "reliable message"
    wait event.
    Thanks and Regards.
    Message was edited by:
    user507290
    Message was edited by:
    user507290

    Metalink Doc 461052.1 is a fairly lightweight walk-through of a system resource profiling tool called LTOM, the "Lite Onboard Monitor". It has an appendix in which it simply points out "The following events were considered idle and ignored... reliable message..."
    It also says that if you want to include something for wait analysis, just take it out of the list of idle and ignored events... so it's not exactly saying that the thing is an idle event, just that it was treated as such for the purposes of writing that document but that you might think differently.
    On the other hand, this page:
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents003.htm
    ...is 10g Release 2's description of "wait events of greatest interest"... and "reliable message" isn't on it! From which one concludes, I suppose, that it isn't of great interest!

  • RE: Looking for some database statistics and wait event statistics

    Hi:
    We are using Oracle 10g RAC on Sun Solaris 5.9.
    1) In GV$SYSSTAT, we cannot find the following database statistics:
         'global cache converts',
         'global cache convert time',
         'global cache gets',
         'global cache get time',
         'global cache convert timeouts',
         'global cache defers'
    2) In GV$SYSTEM_EVENT, we cannot find the following wait event:
         'buffer busy global CR'
    For 1) and 2), could you please tell us where to retrieve the above information,
    or suggest a reasonable alternative?
    Have these stats been removed in Oracle 10g?
    Have they been renamed? To what?
    3) We currently compute the number of additional redo logs that can be accommodated in the free space of the archive directory using the formula:
    (arhive directory free space / redo log file size)
    However, using ASM we find that the free space for a ASM diskgroup is being reported as zero in the table v$asm_diskgroup if the diskgroup is mounted manually. How would one go about retrieving the free space for a logical volume?
    Thanks

    Hi:
    We are using Oracle 10g RAC on Sun Solaris 5.9.
    1) In GV$SYSSTAT, we cannot find the following database statistics:
         'global cache converts',
         'global cache convert time',
         'global cache gets',
         'global cache get time',
         'global cache convert timeouts',
         'global cache defers'
    2) In GV$SYSTEM_EVENT, we cannot find the following wait event:
         'buffer busy global CR'
    For 1) and 2), could you please tell us where to retrieve the above information,
    or suggest a reasonable alternative?
    Have these stats been removed in Oracle 10g?
    Have they been renamed? To what?
    3) We currently compute the number of additional redo logs that can be accommodated in the free space of the archive directory using the formula:
    (arhive directory free space / redo log file size)
    However, using ASM we find that the free space for a ASM diskgroup is being reported as zero in the table v$asm_diskgroup if the diskgroup is mounted manually. How would one go about retrieving the free space for a logical volume?
    Thanks

  • (  name-service call wait   ) event   is amoung the top 5 wait events

    10.2.0.3 2node- RAC
    When I check the wait event in the active session wait event the sessions can be see that they are blocked by the LMON process. The wait event appears only on one node.
    It is very hard to find information on the net or on the metalink.
    How could I overcome this wait event ?
    select program, type from v$session
         where sid in
         (select blocking_session from v$active_session_history
                   where event like 'name-service%'
         and rownum < 1000 )
    oracle@dbokyanus1 (LMON) , BACKGROUND
    Top 5 Timed Events Avg %Total
    ~~~~~~~~~~~~~~~~~~ wait Call
    Event Waits Time (s) (ms) Time
    db file sequential read 4,331,507 15,218 *4* *31.2*
    CPU time *8* *17.5*
    log file sync 1,462,968 5,513 *4* *11.3*
    name-service call wait 72,058 4,545 *63* *9.3*
    SQL*Net message from dblink 4,197 4,047 *964* *8.3*

    oceanic-815 wrote:
    10.2.0.3 2node- RAC
    When I check the wait event in the active session wait event the sessions can be see that they are blocked by the LMON process. The wait event appears only on one node.
    It is very hard to find information on the net or on the metalink.
    If I use Google to find information about this
    I ran into this page.
    There is indeed not much to find about this wait-event, other than there is probably some sort of network communication problem between the two nodes.
    Maybe the best solution is to raise a S/R at Oracle support

  • What is "reliable message" wait event?

    Hi, all.
    What is "reliable message" wait event?
    The wait class is "Other", not "Idle".
    From time to time, I can see "reliable message" wait event
    in the Top 5 Timed Events section from AWR reports.
    In addition, what is "CPU time" in the Top 5 Timed Events section?
    Thanks and Regards.
    Message was edited by:
    user507290

    Dear Yas.
    Thanks for your reply.
    The database is 2 node RAC database (10.2.0.2.0) on 32 bit windows 2003 EE SP1.
    Does "AQ" mean Streams AQ???
    Can I disable Streams AQ facility??
    Thanks and Regards.
    Message was edited by:
    user507290
    Message was edited by:
    user507290

  • A new wait event

    Hi all,
    We have a 2 node RAC 10.2.0.3 database.From quite sometime we are experiencing "transaction" wait event apart from "enq:TX".
    What is this "transaction wait event"? How does it differ from enq:TX row lock contention event. I have searched a lot but couldn't find any proper document to describe it.
    Has any one encountered this event before?
    Thanks.

    Yasir Hashmi wrote:
    Hi all,
    We have a 2 node RAC 10.2.0.3 database.From quite sometime we are experiencing "transaction" wait event apart from "enq:TX".
    What is this "transaction wait event"? How does it differ from enq:TX row lock contention event. I have searched a lot but couldn't find any proper document to describe it.
    Has any one encountered this event before?
    Thanks.what SQL produces this event?
    against which object type does this wait occur?
    do as below so we can know complete Oracle version & OS name.
    Post via COPY & PASTE complete results of
    SELECT * from v$version;

  • What do the wait events 'gc cr failure' and 'cr request retry' mean?

    I'm trying to troubleshoot an issue for a customer. Environment is Oracle 10.2.0.4 (64-bit) on Redhat 5. Two node RAC cluster. The 10046 trace file shows lots of 'gc current block 2-way' waits but also a few 'gc cr failure' and 'cr request retry' waits. The 'cr request retry' waits take about 0.9 seconds each. I cannot find much if any information on these two wait events. Any help is much appreciated.
    Thanks!

    Hi,
    Also, you might need to check the protocol is being used for the interconnect communication.
    here are the steps just in case:
    $ORACLE_HOME/bin/sqlplus / as sysdba
    oradebug setmypid
    oradebug unlimit
    oradebug ipc
    oradebug TRACEFILE_NAME
    Review the file that the last oradebug gave you back.
    Metalink note 181489.1 provides some handy steps to analyze your situation. (also contains the latest supported protocols for IPC)
    Hope this helps
    Regards,
    Jozsef

  • What is ges reusing os pid wait event

    What is wait event "ges reusing os pid". In our RAC environment it is one of the top wait events. How to minimze it.

    This is a wait event in Oracle 10g for Global Enqueue Services (ges) waiting on an operating system process id (os pid).
    How to resolve this issue? I checked the bug list on Metalink and there is a patch set for the issue that may help.
    Question: what version and patch release are you running for Oracle RAC?
    Also, you probably want to tune your public network and private interconnects between the nodes in your Oracle RAC cluster.
    Regards,
    Ben Prusinski
    http://oracle-magician.blogspot.com/

Maybe you are looking for

  • Photoshop CS2

    since Adobe Photoshop CS2 was installed I had so many troubles. It's very slow, and it takes up to 10 minutes to pen an image, as I'm working in huge files. Moreover, it keeps haning and crashing, and collapses out of the sudden without any error mes

  • Error: 130200 upon updating Acrobat 9.0 Pro

    Hi all, Acrobat 9.0 Pro keeps telling me that there is an update available. When trying to install the update there is an error message appearing: (translated from german): screenshot see below "Update Error An error has occured. Please try again to

  • Using java beans in jsp

    Hi, I need to do this, can any one help me. In the signup form, i have some text frields like this HTML <form action=http://localhost:8080/servlet/Signup.jsp"> <input type=text name=name> <input type=text name=phone> <input type=submit value="Submit"

  • Order of the mp3 files

    How the MP3 files could be sort by folder- and filenames in player (and played in that order)? Now they are sort by track number in "albums" (player makes two different albums with same name to one album) and albums are in alphabetical order. I have

  • Error message -2 and -12345

    Hello out there, I've seen other people ask about these error messages on encoding with DVDSP4 but never seen a satisfactory answer. On the record: does anyone know what these error messages mean, and even more importantly, how does one resolve the i