CPU Time Wait Event

I am running perfstat in Oracle9i Enterprise Edition Release 9.2.0.4.0. When I create a perfstat report using the standard scripts I see a wait event called "CPU time." I don't understand what this means. The OEM server monitoring tool does not indicate that the CPU is being used more than 50%, often more like 20%. The top 5 wait events is below.
What does "CPU time" time mean?
the following top 5 wait events:
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % Total
Event Waits Time (s) Ela Time
CPU time 2,200 61.88
db file sequential read 113,042 639 17.96
control file parallel write 9,345 471 13.24
log file sync 4,722 122 3.44
db file scattered read 13,375 67 1.87

Hi,
Below is part of my Statspack
```````
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 13805 26-Mar-08 08:00:03 85 173.2
End Snap: 13806 26-Mar-08 09:00:05 98 154.4
Elapsed: 60.03 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 1,024M Std Block Size: 8K
Shared Pool Size: 400M Log Buffer: 2,048K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 183,878.75 16,898.36
Logical reads: 61,602.61 5,661.25
Block changes: 1,255.54 115.38
Physical reads: 7,370.56 677.35
Physical writes: 55.64 5.11
User calls: 341.11 31.35
Parses: 56.26 5.17
Hard parses: 0.32 0.03
Sorts: 56.43 5.19
Logons: 0.80 0.07
Executes: 712.67 65.49
Transactions: 10.88
% Blocks changed per Read: 2.04 Recursive Call %: 82.56
Rollback per transaction %: 9.29 Rows per Sort: 1043.63
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.98 Redo NoWait %: 100.00
Buffer Hit %: 88.06 In-memory Sort %: 100.00
Library Hit %: 99.96 Soft Parse %: 99.42
Execute to Parse %: 92.11 Latch Hit %: 99.95
Parse CPU to Parse Elapsd %: 69.60 % Non-Parse CPU: 99.64
Shared Pool Statistics Begin End
Memory Usage %: 89.25 89.26
% SQL with executions>1: 40.51 39.68
% Memory for SQL w/exec>1: 29.05 29.45
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % Total
Event Waits Time (s) Ela Time
CPU time 9,272 43.73
db file sequential read 5,203,772 7,027 33.14
db file scattered read 2,198,721 1,547 7.30
log file parallel write 20,935 1,126 5.31
log file sync 10,277 759 3.58
``````````````````
From my observation I found that it seems CPU is bottneck on this Server. I have gathered few SQLs to find out whether any scope for tuning but still I believe that by reducing the cache size also I can divide the load or can say transfer the load from CPU to I/O. Am I correct or do I need to look over else ?
Thanks
MJ

Similar Messages

  • Top 5 wait events in AWR Repprt

    Hi,
    The following is top 5 wait event in my AWR reports...
    Whenever I take reports this are always top 5 events
    Top 5 Timed Events
    =============================================================================================================
    Event                
    CPU time           
    Waits               4,717
    % Total Call Time     62.0
    log file sync           
    Waits                64,963           
    Time(s)               1,362           
    Avg Wait(ms)          21                
    % Total Call Time     17.9           
    Wait Class          Commit
    log file parallel write     
    Waits               63,485           
    Time(s)               1,004           
    Avg Wait(ms)          16                
    % Total Call Time     13.2           
    Wait Class          System I/O
    enq: TX - row lock contention
    Waits               348           
    Time(s)               984           
    Avg Wait(ms)          2,828                
    % Total Call Time     12.9           
    Wait Class          Application
    db file parallel write      
    Waits               29,305           
    Time(s)               561           
    Avg Wait(ms)          19                
    % Total Call Time     7.4           
    Wait Class          System I/O
    ------------------------------------------------------------------------------------------------------------

    Start with Performance Tuning Guide
    10.2.3 Table of Wait Events and Potential Causes

  • Top Wait Events

    hi gurus,
    3 node rac 10.2.0.4 serving a packaged application.
    Top 5 timed events in awr shown as
    Event=CPU time
         Waits=
    Time(s)=1,950
    Avg Wait(ms)
    % Total Call Time=45.3     
    Wait Class
    Event=gc cr multi block request
         Waits= 6,551,055
    Time(s)= 1,396
    Avg Wait(ms)=0
    % Total Call Time=38.9          
    Wait Class= Cluster
    Event=db file scattered read
         Waits= 186,295
    Time(s)= 719
    Avg Wait(ms)=4
    % Total Call Time=18.2          
    Wait Class= User I/O
    Event=db file parallel read
         Waits= 43,383
    Time(s)= 241
    Avg Wait(ms)=6
    % Total Call Time= 5.9               
    Wait Class= User I/O
    Event=      log file sync
         Waits= 71,064
    Time(s)= 83
    Avg Wait(ms)=1
    % Total Call Time= 3.1               
    Wait Class= Commit
    db_block_size=8KB
    db_file_multiblock_read_count = default setting of 128
    question:
    are the high wait values of gc cr multi block request and db file scattered read are due to db_file_multiblock_read_count?
    if that's the case, is there a way to find optimum value for db_file_multiblock_read_count?
    or any other findings please?
    experts, appreciate your valuable help
    thanks in advance,
    charles

    user570138 wrote:
    there are queries going for the full table scan with outer joins (milliion of records). those are the same sqls at the top of "sql order by cluster time"in awr with high CPU utilization.
    any way to fine tune the instance to reduce the "gc cr multi block request"
    apart from changing the code as the code belongs to a package based application please?
    Do you have a performance problem ?
    You are doing some large tablescans; these are (probably) the root cause of the gc cr multiblock read, the db file scattered reads, and the CPU, but if the queries are necessary and the execution paths are the best that can be done then maybe you just have to recognise that the resource you're using is reasonable for the queries you have to run.
    Otherwise
    <ul>
    (a) can you find a more efficient access path for any of these queries
    (b) can you make sure that all these queries run on the same node so that you get some benefit from node-affinity (possibly the object(s) will be remasted to that single node) and reduce the interconnect traffic.
    </ul>
    Regards
    Jonathan Lewis
    http://jonathanlewis.wordpress.com
    http://www.jlcomp.demon.co.uk
    To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
    "Science is more than a body of knowledge; it is a way of thinking"
    Carl Sagan                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

  • CPU Time in Top 5 Timed Events

    Hi,
    We have a 2 node RAC database(10.2.0.3) on Solaris 10.
    There is performance issue with CMRO application R12.
    In database I see CPU time consistently as the top wait event in the Top 5 Timed Events.
    This is mostl followed by db file sequential read.
    For one of the days:
    Top 5 Timed Events
    Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
    CPU time 8,383 82.8
    db file sequential read 173,417 838 5 8.3 User I/O
    SQL*Net break/reset to client 26,015 651 25 6.4 Application
    enq: TX - row lock contention 1,063 356 335 3.5 Application
    gcs log flush sync 37,747 88 2 .9 Other
    For other day:
    Top 5 Timed Events
    Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
    CPU time 25,286 62.0
    db file sequential read 2,644,332 8,267 3 20.3 User I/O
    gc buffer busy 1,358,725 3,830 3 9.4 Cluster
    read by other session 438,494 1,169 3 2.9 User I/O
    SQL*Net more data to client 19,423 879 45 2.2 Network
    Any idea of the of the bottleneck?
    Thanks

    8 CPUs, load average 4, runqueue 0 and usage 30-35%
    Does this indicate any issue with system resourcesNO. Not at all.
    However a poor schema design or inefficient SQL execution can mean that a query that should do 100 'consistent gets' is doing 10,000 'consistent gets' -- in the buffer cache, consuming CPU and not waiting for I/O. This is a scenario where you have idle CPU but CPU usage is inefficient. (Thus, for example, adding more CPUs will not help your users at all).
    So you should look at the queries and see if queries can be improved.
    If, on the other hand, users are not complaining of performance and all response times are within expectations, than you have no issue at all.
    Hemant K Chitale

  • Understanding statspack report(CPU time in top time events)

    Hi,
    I am using oracle 9.2.0.8 RAC on SUN solaris platform.I am trying to understand my DB statistics using the below statspack report.Can you please coment on the below report
    My quetions/thoughts are:
    1) CPU time is in the top timed events,Is that eman some need to do with CPU increase.Was CPU bottleneck?
    2) Parse CPU to Parse Elapsd %: 80.28 .Is this means I am hard parsing most of the time.How can identify which queries doing more hard parses.what is mean by% Non-Parse CPU: 98.76
    3) Memory Usage %: 96.25 96.64.It seems to be there is too much memory usage.Can you elaborate this usage about what could be the reasons for this to happen
    4) global cache cr request is coming in the top wait evetns and top timed events.Is there some issue with RAC?
    5) can you please explain about 5 CR Blocks Served (RAC) and 5 CU Blocks Served (RAC) and Top 5 ITL Waits per
    Your help is appreciated!!
    Load Profile
    ~~~~~~~~~~~~ Per Second Per Transaction
    Redo size: 2,101,521.49 18,932.15
    Logical reads: 91,525.82 824.54
    Block changes: 6,720.68 60.55
    Physical reads: 5,644.92 50.85
    Physical writes: 464.97 4.19
    User calls: 922.79 8.31
    Parses: 342.37 3.08
    Hard parses: 1.52 0.01
    Sorts: 324.18 2.92
    Logons: 2.66 0.02
    Executes: 2,131.75 19.20
    Transactions: 111.00
    % Blocks changed per Read: 7.34 Recursive Call %: 78.48
    Rollback per transaction %: 22.43 Rows per Sort: 15.89
    Instance Efficiency Percentages (Target 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Buffer Nowait %: 99.66 Redo NoWait %: 100.00
    Buffer Hit %: 93.86 In-memory Sort %: 100.00
    Library Hit %: 99.95 Soft Parse %: 99.56
    Execute to Parse %: 83.94 Latch Hit %: 99.79
    Parse CPU to Parse Elapsd %: 80.28 % Non-Parse CPU: 98.76
    Shared Pool Statistics Begin End
    Memory Usage %: 96.25 96.64
    % SQL with executions>1: 34.19 32.67
    % Memory for SQL w/exec>1: 39.87 40.47
    Top 5 Timed Events
    ~~~~~~~~~~~~~~~~~~ % Total
    Event Waits Time (s) Ela Time
    CPU time 10,406 42.54
    db file sequential read 1,707,372 4,282 17.51
    global cache cr request 2,566,822 2,369 9.68
    db file scattered read 1,109,892 1,719 7.03
    SQL*Net break/reset to client 17,287 1,348 5.51
    Wait Events for DB: Instance:
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc (idle events last)
    Avg
    Total Wait wait Waits
    Event Waits Timeouts Time (s) (ms) /txn
    db file sequential read 1,707,372 0 4,282 3 8.5
    global cache cr request 2,566,822 3,356 2,369 1 12.8
    db file scattered read 1,109,892 0 1,719 2 5.5
    SQL*Net break/reset to clien 17,287 0 1,348 78 0.1
    buffer busy waits 312,198 11 1,082 3 1.6
    Message was edited by:
    user509266

    This statspack taken for 30 minutes interval.We have 16 CPU's.We never got ORA-4031 errors.It means you have 16 * 30 * 60 = 28,800 seconds CPU available during the interval but you only used 10,406. So you don't have a CPU problem.
    For Statspack documentation, you can have a look to <ORACLE_HOME>/rdbms/admin/spdoc.txt, Metalink note 228913.1, Jonathan Lewis Scratchpad, books commended by Rajesh Kumar Yogi and also to http://www.oracle.com/technology/deploy/performance/index.html

  • CPU wait events on ADDM report

    Hello,
    My Oracle version is:
    Connected to Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 Yesterday I was taking a look on an ADDM report and spot the following:
       Rationale
          SQL statement with SQL_ID "0mgk8gx9hj71d" was executed 777 times and had
          an average elapsed time of 42 seconds.
       Rationale
          Waiting for event "resmgr:cpu quantum" in wait class "Scheduler"
          accounted for 34% of the database time spent in processing the SQL
          statement with SQL_ID "0mgk8gx9hj71d".After that, I started looking for how ADDM could know that the SQL_ID "0mgk8gx9hj71d" waited 34% on "resmgr:cpu quantum" event. No lucky with that...
    The only wait event information related to a given SQL_ID I've found on v$active_session_history (or the AWR persisted table for it), but in the ASH there is no information about CPU wait events like "cpu quantum". When the session is waiting for CPU, there is no event related in v$ash.
    So, my question is: where ADDM got the information that the SQL waited 34% of the time on "resmgr:cpu quantum"?
    Thanks,
    Heitor Kirsten

    Hi,
    Is a session waiting for CPU resources ("res:cpu quantum") considered as an active session ? Maybe not.
    I guess (I made no test) that this maybe the reason why this kind of wait is not shown in the active session history .
    Regards
    Maurice

  • Statspack : Top 5 Timed Events - CPU time

    Hi,
    I just get some statspack reports on my 10.2.0.2 database (HP-UX 11i).
    I'm just surprise about the CPU time into the Top-5 events.
    Top 5 Timed Events                                                    Avg %Total
    ~~~~~~~~~~~~~~~~~~                                                   wait   Call
    Event                                            Waits    Time (s)   (ms)   Time
    CPU time 4,263 97.3
    latch: cache buffers chains                    197,925          42      0    1.0
    log file parallel write                          8,982          22      2     .5
    log file sync                                    8,620          22      3     .5
    wait list latch free                               399           7     17     .2What does CPU time here ? Is it a problem here ?
    Thanks in advance for your lights.

    Hi,
    it seems that your database experiences a high SQL workload. The section of statspack report called "Top SQLs by Buffer Gets" might give you an idea what SQLs caused this CPU workload.

  • 100% CPU, wait event : latch shared pool

    I have a store procedure, run in one of database, it hangs in a "create table ... as select ..." statement.
    the wait event is : latch shared pool, and CPU is up to 100%, it has run over few hours and seems hang.
    Same stored procedure run on others enviroment, never seen such problem, even run on the same data size or even much much bigger data size.
    This procedure has been used more than 2 years, never see such problem in any others enviroment. it only happend in this new setup enviroment.
    however, in this enviroment, if I try to reduce data to be very very small, I was able to see procudure complate in 10 sec.
    I suspect parameter, for example, I changed shared_pool_size from 40MB to 150 MB, re-start database and re-run, still see the same problem here.
    Could anybody suggest any thing I can look into?
    Thanks

    jjzz wrote:
    I have a store procedure, run in one of database, it hangs in a "create table ... as select ..." statement.
    the wait event is : latch shared pool, and CPU is up to 100%, it has run over few hours and seems hang.
    If it's running at 100% CPU, it's not waiting.
    Does v$session_wait (or even v$session since you seem to be running 10g) tell you that the session is *"waiting"*, or is it simply noting that your last wait was on the shared pool latch ?
    If the latter, then you probably have some SQL in the procedure that has changed its execution plan to become much more CPU intensive - perhaps because of a small change in the data volume, data distribution, or statistics.
    First step - find out what SQL statements are executing, and see how much work they are doing. You could query v$session for that session a few times and check what the sql_id and sql_child number are, also prev_sql_id and prev_child_number. If these stay constant, one or other may give you the guilty SQL statement. If not check v$open_cursor for the session.
    Regards
    Jonathan Lewis
    http://jonathanlewis.wordpress.com
    http://www.jlcomp.demon.co.uk
    "Science is more than a body of knowledge; it is a way of thinking"
    Carl Sagan

  • DB CPU wait event is high in AWR

    Hello Experts,
    Could you please tell me what are the causes of increasing DB CPU wait event ? I mentioned below which i know. please guide me
    1. When Buffer cache is more than required then DB CPU wait event occur.
    Regards,
    Sachin

    Sachin.Ichake wrote:
    Currently in my database DB CPU has taken 90% DB time . in accordance to resolve it I will gonna follow steps
    1. Tune the query which has taken more cpu
    2. Decrease Buffer cache size by referring buffer cache advisory.Solve what? You must understand that DB CPU is not shown as a Wait Event but as a Timed Event and so are the other events that are shown in the Top 5 Timed Events category. This is an indication of how much you have used in the comparison of the total DB Time but not necessarily , it's an issue as to do anything in the system, you would need to burn the CPU only. You need to check that how much total CPU time you have with you and then compare it with your DB CPU seconds. In addition to this, you also need to check the CPU consumption from the o/s commands like Top etc. Combining all of such information only would be able to help to understand that whether any tuning needs to be done or not.
    Post here the AWR/Statspack report. That would give more clear picture of the things.
    Aman....

  • Wait Time / Timeout Time in waiting events

    Hi,
    From manual, I came to know that some of waiting events having wait time or time out duration. I have not understood, what happens if it has reached timeout. What happens next ? ( Will you explain in more detail ). It saysl it will renew wait event. ( I have not understood what does mean "RENEW", Or what steps will be taken by Oracle, what happens if again reached timeout ) ?
    Thanks for clearing my doubt in detail.
    regards
    pjp

    Please read the FAQ and learn how to enclose your listings in tags so we can read them.
    I can't help you at this time because I can not read what you posted.                                                                                                                                                                                                                                                                                                                                               

  • CPU TIME and DB CPU events under TOP 5 TIMED FOREGROUND EVENTS section in AWR report

    Is there any difference between "CPU TIME" event and "DB CPU" event when shown in AWR report under section "TOP 5 TIMED FOREGROUND EVENTS"?
    As per my understanding of both these terms they indicate the same thing. But then if it is so then why have different names?
    I searched around but the only relevant discussion that I found was as under, which didn't really cleared the doubt.
    https://forums.oracle.com/message/2571255#2571255

    In the article that you have mentioned - Jonathan Lewis gives you a very clear explanation. CPU Time is updated at the end of a query. DB CPU is updated every few seconds.
    So the CPU Time may be less than DB CPU if there is a long running query that did not complete during the snapshot that you are reporting for. Conversly CPU Time may be larger than DB CPU if there is a long running query that spanned multiple snapshots but completed in the snapshot that you are reporting for.

  • Thread CPU time: does it include the time while thread is waiting

    Hi,
    My question is regarding the getCurrentThreadCpuTime() supported by the ThreadMXBean.
    I wonder if the returned time includes the time while the thread is waiting, i.e. after the thread calls wait() and before getting a notify() or a notifyAll().
    Thanks,
    Nuha

    CPU time is intended to reflect actual time executing on a CPU. If block for a lock/wait/sleep or other blocking operation like I/O then you don't consume CPU while blocked.
    On Solaris it is implemented using gethrvtime()
    If a VM used a busy-wait for something rather than blocking (not likely with wait()) then that's consuming CPU time.

  • Need help to analysis "foreground and background wait events" on statspack report for oracle database 11.2.0.4 on AIX

    Hi: I'm analyzing this STATSPACK report: it is "volume test" on our UAT server, so most input is from 'bind variables'.  Our shared pool is well utilized in oracle.  Oracle redo logs is not appropriately configured on this server, as in 'Top 5 wait events' there are 2 for redos.
    I need to know what else information can be dig-out from 'foreground wait events' & 'background wait events', and what can assist us to better understanding, in combination of 'Top 5 wait event's, that how the server/test went?  it could be overwelming No. of wait events, so appreciate any helpful diagnostic or analysis.  Database is oracle 11.2.0.4 upgraded from 11.2.0.3, on IBM AIX power system 64bit, level 6.x
    STATSPACK report for
    Database    DB Id    Instance     Inst Num  Startup Time   Release     RAC
    ~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
    700000XXX   XXX              1 22-Apr-15 12:12 11.2.0.4.0  NO
    Host Name             Platform                CPUs Cores Sockets   Memory (G)
    ~~~~ ---------------- ---------------------- ----- ----- ------- ------------
         dXXXX_XXX    AIX-Based Systems (64-     2     1       0         16.0
    Snapshot       Snap Id     Snap Time      Sessions Curs/Sess Comment
    ~~~~~~~~    ---------- ------------------ -------- --------- ------------------
    Begin Snap:       5635 22-Apr-15 13:00:02      114       4.6
      End Snap:       5636 22-Apr-15 14:00:01      128       8.8
       Elapsed:      59.98 (mins) Av Act Sess:       0.6
       DB time:      35.98 (mins)      DB CPU:      19.43 (mins)
    Cache Sizes            Begin        End
    ~~~~~~~~~~~       ---------- ----------
        Buffer Cache:     2,064M              Std Block Size:         8K
         Shared Pool:     3,072M                  Log Buffer:    13,632K
    Load Profile              Per Second    Per Transaction    Per Exec    Per Call
    ~~~~~~~~~~~~      ------------------  ----------------- ----------- -----------
          DB time(s):                0.6                0.0        0.00        0.00
           DB CPU(s):                0.3                0.0        0.00        0.00
           Redo size:          458,720.6            8,755.7
       Logical reads:           12,874.2              245.7
       Block changes:            1,356.4               25.9
      Physical reads:                6.6                0.1
    Physical writes:               61.8                1.2
          User calls:            2,033.7               38.8
              Parses:              286.5                5.5
         Hard parses:                0.5                0.0
    W/A MB processed:                1.7                0.0
              Logons:                1.2                0.0
            Executes:              801.1               15.3
           Rollbacks:                6.1                0.1
        Transactions:               52.4
    Instance Efficiency Indicators
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                Buffer Nowait %:  100.00       Redo NoWait %:  100.00
                Buffer  Hit   %:   99.98  Optimal W/A Exec %:  100.00
                Library Hit   %:   99.77        Soft Parse %:   99.82
             Execute to Parse %:   64.24         Latch Hit %:   99.98
    Parse CPU to Parse Elapsd %:   53.15     % Non-Parse CPU:   98.03
    Shared Pool Statistics        Begin   End
                 Memory Usage %:   10.50   12.79
        % SQL with executions>1:   69.98   78.37
      % Memory for SQL w/exec>1:   70.22   81.96
    Top 5 Timed Events                                                    Avg %Total
    ~~~~~~~~~~~~~~~~~~                                                   wait   Call
    Event                                            Waits    Time (s)   (ms)   Time
    CPU time                                                       847          50.2
    enq: TX - row lock contention                    4,480         434     97   25.8
    log file sync                                  284,169         185      1   11.0
    log file parallel write                        299,537         164      1    9.7
    log file sequential read                           698          16     24    1.0
    Host CPU  (CPUs: 2  Cores: 1  Sockets: 0)
    ~~~~~~~~              Load Average
                          Begin     End      User  System    Idle     WIO     WCPU
                           1.16    1.84     19.28   14.51   66.21    1.20   82.01
    Instance CPU
    ~~~~~~~~~~~~                                       % Time (seconds)
                         Host: Total time (s):                  7,193.8
                      Host: Busy CPU time (s):                  2,430.7
                       % of time Host is Busy:      33.8
                 Instance: Total CPU time (s):                  1,203.1
              % of Busy CPU used for Instance:      49.5
            Instance: Total Database time (s):                  2,426.4
      %DB time waiting for CPU (Resource Mgr):       0.0
    Memory Statistics                       Begin          End
    ~~~~~~~~~~~~~~~~~                ------------ ------------
                      Host Mem (MB):     16,384.0     16,384.0
                       SGA use (MB):      7,136.0      7,136.0
                       PGA use (MB):        282.5        361.4
        % Host Mem used for SGA+PGA:         45.3         45.8
    Foreground Wait Events  DB/Inst: XXXXXs  Snaps: 5635-5636
    -> Only events with Total Wait Time (s) >= .001 are shown
    -> ordered by Total Wait Time desc, Waits desc (idle events last)
                                                                 Avg          %Total
                                              %Tim Total Wait   wait    Waits   Call
    Event                               Waits  out   Time (s)   (ms)     /txn   Time
    enq: TX - row lock contentio        4,480    0        434     97      0.0   25.8
    log file sync                     284,167    0        185      1      1.5   11.0
    Disk file operations I/O            8,741    0          4      0      0.0     .2
    direct path write                  13,247    0          3      0      0.1     .2
    db file sequential read             6,058    0          1      0      0.0     .1
    buffer busy waits                   1,800    0          1      1      0.0     .1
    SQL*Net more data to client        29,161    0          1      0      0.2     .1
    direct path read                    7,696    0          1      0      0.0     .0
    db file scattered read                316    0          1      2      0.0     .0
    latch: shared pool                    144    0          0      2      0.0     .0
    CSS initialization                     30    0          0      3      0.0     .0
    cursor: pin S                          10    0          0      9      0.0     .0
    row cache lock                         41    0          0      2      0.0     .0
    latch: row cache objects               19    0          0      3      0.0     .0
    log file switch (private str            8    0          0      7      0.0     .0
    library cache: mutex X                 28    0          0      2      0.0     .0
    latch: cache buffers chains            54    0          0      1      0.0     .0
    latch free                            290    0          0      0      0.0     .0
    control file sequential read        1,568    0          0      0      0.0     .0
    log file switch (checkpoint             4    0          0      6      0.0     .0
    direct path sync                        8    0          0      3      0.0     .0
    latch: redo allocation                 60    0          0      0      0.0     .0
    SQL*Net break/reset to clien           34    0          0      1      0.0     .0
    latch: enqueue hash chains             45    0          0      0      0.0     .0
    latch: cache buffers lru cha            7    0          0      2      0.0     .0
    latch: session allocation               5    0          0      1      0.0     .0
    latch: object queue header o            6    0          0      1      0.0     .0
    ASM file metadata operation            30    0          0      0      0.0     .0
    latch: In memory undo latch            15    0          0      0      0.0     .0
    latch: undo global data                 8    0          0      0      0.0     .0
    SQL*Net message from client     6,362,536    0    278,225     44     33.7
    jobq slave wait                     7,270  100      3,635    500      0.0
    SQL*Net more data from clien        7,976    0         15      2      0.0
    SQL*Net message to client       6,362,544    0          8      0     33.7
    Background Wait Events  DB/Inst: XXXXXs  Snaps: 5635-5636
    -> Only events with Total Wait Time (s) >= .001 are shown
    -> ordered by Total Wait Time desc, Waits desc (idle events last)
                                                                 Avg          %Total
                                              %Tim Total Wait   wait    Waits   Call
    Event                               Waits  out   Time (s)   (ms)     /txn   Time
    log file parallel write           299,537    0        164      1      1.6    9.7
    log file sequential read              698    0         16     24      0.0    1.0
    db file parallel write              9,556    0         13      1      0.1     .8
    os thread startup                     146    0         10     70      0.0     .6
    control file parallel write         2,037    0          2      1      0.0     .1
    Log archive I/O                        35    0          1     30      0.0     .1
    LGWR wait for redo copy             2,447    0          0      0      0.0     .0
    db file async I/O submit            9,556    0          0      0      0.1     .0
    db file sequential read               145    0          0      2      0.0     .0
    Disk file operations I/O              349    0          0      0      0.0     .0
    db file scattered read                 30    0          0      4      0.0     .0
    control file sequential read        5,837    0          0      0      0.0     .0
    ADR block file read                    19    0          0      4      0.0     .0
    ADR block file write                    5    0          0     15      0.0     .0
    direct path write                      14    0          0      2      0.0     .0
    direct path read                        3    0          0      7      0.0     .0
    latch: shared pool                      3    0          0      6      0.0     .0
    log file single write                  56    0          0      0      0.0     .0
    latch: redo allocation                 53    0          0      0      0.0     .0
    latch: active service list              1    0          0      3      0.0     .0
    latch free                             11    0          0      0      0.0     .0
    rdbms ipc message                 314,523    5     57,189    182      1.7
    Space Manager: slave idle wa        4,086   88     18,996   4649      0.0
    DIAG idle wait                      7,185  100      7,186   1000      0.0
    Streams AQ: waiting for time            2   50      4,909 ######      0.0
    Streams AQ: qmn slave idle w          129    0      3,612  28002      0.0
    Streams AQ: qmn coordinator           258   50      3,612  14001      0.0
    smon timer                             43    2      3,605  83839      0.0
    pmon timer                          1,199   99      3,596   2999      0.0
    SQL*Net message from client        17,019    0         31      2      0.1
    SQL*Net message to client          12,762    0          0      0      0.1
    class slave wait                       28    0          0      0      0.0
    thank you very much!

    Hi: just know it now: it is a large amount of 'concurrent transaction' designed in this "Volume Test" - to simulate large incoming transaction volme, so I guess wait in eq:TX - row is expected.
    The fact: (1) redo logs at uat server is known to not well-tune for configurations (2) volume test slow 5%, however data amount in its test is kept the same by each time import  production data, by the team. So why it slowed 5% this year?
    The wait histogram is pasted below, any one interest to take a look?  any ideas?
    Wait Event Histogram  DB/Inst: XXXX/XXXX  Snaps: 5635-5636
    -> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
    -> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
    -> % of Waits - value: .0 indicates value was <.05%, null is truly 0
    -> Ordered by Event (idle events last)
                               Total ----------------- % of Waits ------------------
    Event                      Waits  <1ms  <2ms  <4ms  <8ms <16ms <32ms  <=1s   >1s
    ADR block file read          19   26.3   5.3  10.5  57.9
    ADR block file write          5                     40.0        60.0
    ADR file lock                 6  100.0
    ARCH wait for archivelog l   14  100.0
    ASM file metadata operatio   30  100.0
    CSS initialization           30              100.0
    Disk file operations I/O   9090   97.2   1.4    .6    .4    .2    .1    .1
    LGWR wait for redo copy    2447   98.5    .5    .4    .2    .2    .2    .1
    Log archive I/O              35   40.0         8.6  25.7   2.9        22.9
    SQL*Net break/reset to cli   34   85.3   8.8         5.9
    SQL*Net more data to clien   29K  99.9    .0    .0    .0          .0    .0
    buffer busy waits          1800   96.8    .7    .7    .6    .3    .4    .5
    control file parallel writ 2037   90.7   5.0   2.1    .8   1.0    .3    .1
    control file sequential re 7405  100.0                      .0
    cursor: pin S                10   10.0                    90.0
    db file async I/O submit   9556   99.9    .0                .0          .0
    db file parallel read         1  100.0
    db file parallel write     9556   62.0  32.4   1.7    .8   1.5   1.3    .1
    db file scattered read      345   72.8   3.8   2.3  11.6   9.0    .6
    db file sequential read    6199   97.2    .2    .3   1.6    .7    .0    .0
    direct path read           7699   99.1    .4    .2    .1    .1    .0
    direct path sync              8   25.0  37.5  12.5  25.0
    direct path write            13K  97.8    .9    .5    .4    .3    .1    .0
    enq: TX - row lock content 4480     .4    .7   1.3   3.0   6.8  12.3  75.4    .1
    latch free                  301   98.3    .3    .7    .7
    latch: In memory undo latc   15   93.3   6.7
    latch: active service list    1              100.0
    latch: cache buffers chain   55   94.5                     3.6   1.8
    latch: cache buffers lru c    9   88.9                    11.1
    latch: call allocation        6  100.0
    latch: checkpoint queue la    3  100.0
    latch: enqueue hash chains   45   97.8                     2.2
    latch: messages               4  100.0
    latch: object queue header    7   85.7        14.3
    latch: redo allocation      113   97.3               1.8    .9
    latch: row cache objects     19   89.5                           5.3   5.3
    latch: session allocation     5   80.0              20.0
    latch: shared pool          147   90.5   1.4   2.7   1.4    .7   1.4   2.0
    latch: undo global data       8  100.0
    library cache: mutex X       28   89.3         3.6         3.6         3.6
    log file parallel write     299K  95.6   2.6   1.0    .4    .3    .2    .0
    log file sequential read    698   29.5    .1               4.6  46.8  18.9
    log file single write        56  100.0
    log file switch (checkpoin    4               25.0  50.0  25.0
    log file switch (private s    8         12.5        37.5  50.0
    log file sync               284K  93.3   3.7   1.4    .7    .5    .3    .1
    os thread startup           146                                      100.0
    row cache lock               41   85.4   9.8               2.4         2.4
    DIAG idle wait             7184                                      100.0
    SQL*Net message from clien 6379K  86.6   5.1   2.9   1.3    .7    .3   2.8    .3
    SQL*Net message to client  6375K 100.0    .0    .0    .0    .0    .0    .0
    Wait Event Histogram  DB/Inst: XXXX/xxxx  Snaps: 5635-5636
    -> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
    -> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
    -> % of Waits - value: .0 indicates value was <.05%, null is truly 0
    -> Ordered by Event (idle events last)
                               Total ----------------- % of Waits ------------------
    Event                      Waits  <1ms  <2ms  <4ms  <8ms <16ms <32ms  <=1s   >1s
    SQL*Net more data from cli 7976   99.7    .1    .1    .0                      .1
    Space Manager: slave idle  4086     .1    .2    .0    .0    .3         3.2  96.1
    Streams AQ: qmn coordinato  258   49.2                .8                    50.0
    Streams AQ: qmn slave idle  129                                            100.0
    Streams AQ: waiting for ti    2   50.0                                      50.0
    class slave wait             28   92.9   3.6   3.6
    jobq slave wait            7270     .0                               100.0
    pmon timer                 1199                                            100.0
    rdbms ipc message           314K  10.3   7.3  39.7  15.4  10.6   5.3   8.2   3.3
    smon timer                   43                                            100.0

  • Performance Degradated  Possibly due to CPU Time

    Hi Gurus,
    There is a utility in our application with which we can upload an excel sheet containing data and schedule the timing of the job, now when the job is executed, each row in the excel sheet leads to dml operations on multiple tables finally leading to generation of a transaction no. Now at the start around 100-120 transaction nos were generated which goes down drastically to around 30-35 after 6-7 hours. AWR report at the two instances shows that CPU time has decreased considerably in the 2nd case.
    I would like you experts to check the awr reports and suggest me the probable reason for the decrease in performance.
    Brief AWR Report When Performance was OK
    Snap Id Snap Time Sessions Curs/Sess
    Begin Snap: 2151 14-Dec-10 16:32:57 26 3.7
    End Snap: 2152 14-Dec-10 17:31:04 40 16.7
    Elapsed: 58.13 (mins)
    DB Time: 55.37 (mins)
    Cache Sizes
    ~~~~~~~~~~~ Begin End
    Buffer Cache: 436M 444M Std Block Size: 8K
    Shared Pool Size: 120M 120M Log Buffer: 6,968K
    Load Profile
    ~~~~~~~~~~~~ Per Second Per Transaction
    Redo size: 27,541.56 1,747.07
    Logical reads: 49,830.97 3,160.97
    Block changes: 181.79 11.53
    Physical reads: 1,270.12 80.57
    Physical writes: 2.81 0.18
    User calls: 119.95 7.61
    Parses: 200.94 12.75
    Hard parses: 29.29 1.86
    Sorts: 91.80 5.82
    Logons: 0.03 0.00
    Executes: 457.16 29.00
    Transactions: 15.76
    % Blocks changed per Read: 0.36 Recursive Call %: 96.36
    Rollback per transaction %: 0.01 Rows per Sort: 270.64
    Instance Efficiency Percentages (Target 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Buffer Nowait %: 100.00 Redo NoWait %: 100.00
    Buffer Hit %: 97.45 In-memory Sort %: 100.00
    Library Hit %: 90.18 Soft Parse %: 85.42
    Execute to Parse %: 56.05 Latch Hit %: 100.00
    Parse CPU to Parse Elapsd %: 98.04 % Non-Parse CPU: 94.98
    Shared Pool Statistics Begin End
    Memory Usage %: 72.65 84.55
    % SQL with executions>1: 71.49 75.08
    % Memory for SQL w/exec>1: 84.79 85.25
    Top 5 Timed Events Avg %Total
    ~~~~~~~~~~~~~~~~~~ wait Call
    Event Waits Time (s) (ms) Time Wait Class
    CPU time 2,541 76.5
    db file scattered read 284,992 410 1 12.3 User I/O
    log file parallel write 31,188 145 5 4.4 System I/O
    TCP Socket (KGAS) 24 131 5459 3.9 Network
    log file sync 8,617 46 5 1.4 Commit
    Time Model Statistics DB/Inst: ABCTEST/abctest Snaps: 2151-2152
    -> Total time in database user-calls (DB Time): 3322.4s
    -> Statistics including the word "background" measure background process
    time, and so do not contribute to the DB time statistic
    -> Ordered by % or DB time desc, Statistic name
    Statistic Name Time (s) % of DB Time
    sql execute elapsed time 3,176.8 95.6
    DB CPU 2,541.1 76.5
    PL/SQL execution elapsed time 288.5 8.7
    parse time elapsed 278.7 8.4
    hard parse elapsed time 254.6 7.7
    PL/SQL compilation elapsed time 28.9 .9
    failed parse elapsed time 4.9 .1
    hard parse (sharing criteria) elapsed time 1.3 .0
    sequence load elapsed time 1.1 .0
    repeated bind elapsed time 1.1 .0
    connection management call elapsed time 0.7 .0
    hard parse (bind mismatch) elapsed time 0.3 .0
    DB time 3,322.4 N/A
    background elapsed time 197.1 N/A
    background cpu time 5.6 N/A
    Wait Class DB/Inst: ABCTEST/abctest Snaps: 2151-2152
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc
    Avg
    %Time Total Wait wait Waits
    Wait Class Waits -outs Time (s) (ms) /txn
    User I/O 292,720 .0 427 1 5.3
    System I/O 37,408 .0 190 5 0.7
    Network 272,062 .0 132 0 4.9
    Commit 8,617 .0 46 5 0.2
    Configuration 4 .0 2 593 0.0
    Application 3,212 .0 0 0 0.1
    Other 280 .4 0 0 0.0
    Concurrency 247 .0 0 0 0.0
    Wait Events DB/Inst: ABCTEST/abctest Snaps: 2151-2152
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc (idle events last)
    Avg
    %Time Total Wait wait Waits
    Event Waits -outs Time (s) (ms) /txn
    db file scattered read 284,992 .0 410 1 5.2
    log file parallel write 31,188 .0 145 5 0.6
    TCP Socket (KGAS) 24 .0 131 5459 0.0
    log file sync 8,617 .0 46 5 0.2
    db file parallel write 4,215 .0 29 7 0.1
    db file sequential read 7,634 .0 16 2 0.1
    control file parallel write 1,202 .0 16 13 0.0
    Streams AQ: enqueue blocked 1 .0 2 2055 0.0
    control file sequential read 795 .0 1 1 0.0
    Data file init write 48 .0 0 9 0.0
    SQL*Net message to client 266,802 .0 0 0 4.9
    log file switch completion 3 .0 0 106 0.0
    SQL*Net break/reset to clien 3,212 .0 0 0 0.1
    SQL*Net more data to client 4,789 .0 0 0 0.1
    direct path write 23 .0 0 3 0.0
    rdbms ipc reply 67 .0 0 1 0.0
    kksfbc child completion 1 100.0 0 47 0.0
    latch: shared pool 213 .0 0 0 0.0
    latch: library cache 26 .0 0 1 0.0
    log file single write 4 .0 0 7 0.0
    log file sequential read 4 .0 0 5 0.0
    db file single write 3 .0 0 5 0.0
    os thread startup 3 .0 0 4 0.0
    enq: JS - queue lock 4 .0 0 3 0.0
    LGWR wait for redo copy 207 .0 0 0 0.0
    library cache pin 1 .0 0 6 0.0
    SQL*Net more data from clien 447 .0 0 0 0.0
    library cache load lock 1 .0 0 2 0.0
    latch: cache buffers chains 1 .0 0 0 0.0
    latch: row cache objects 1 .0 0 0 0.0
    direct path read 20 .0 0 0 0.0
    latch free 1 .0 0 0 0.0
    cursor: mutex S 1 .0 0 0 0.0
    SQL*Net message from client 266,789 .0 64,143 240 4.9
    Streams AQ: qmn slave idle w 124 .0 3,488 28127 0.0
    Streams AQ: qmn coordinator 257 51.4 3,488 13571 0.0
    virtual circuit status 116 100.0 3,480 29999 0.0
    Streams AQ: waiting for time 5 60.0 745 148902 0.0
    jobq slave wait 52 96.2 155 2987 0.0
    PL/SQL lock timer 16 100.0 16 995 0.0
    class slave wait 1 100.0 5 4995 0.0
    Background Wait Events DB/Inst: ABCTEST/abctest Snaps: 2151-2152
    -> ordered by wait time desc, waits desc (idle events last)
    Avg
    %Time Total Wait wait Waits
    Event Waits -outs Time (s) (ms) /txn
    log file parallel write 31,188 .0 145 5 0.6
    db file parallel write 4,215 .0 29 7 0.1
    control file parallel write 1,193 .0 16 13 0.0
    Streams AQ: enqueue blocked 1 .0 2 2055 0.0
    control file sequential read 691 .0 0 1 0.0
    db file sequential read 66 .0 0 5 0.0
    direct path write 23 .0 0 3 0.0
    log file single write 4 .0 0 7 0.0
    log file sequential read 4 .0 0 5 0.0
    events in waitclass Other 211 .0 0 0 0.0
    os thread startup 3 .0 0 4 0.0
    db file scattered read 1 .0 0 13 0.0
    latch: shared pool 5 .0 0 0 0.0
    direct path read 20 .0 0 0 0.0
    latch: library cache 1 .0 0 0 0.0
    rdbms ipc message 34,411 32.3 30,621 890 0.6
    Streams AQ: qmn slave idle w 124 .0 3,488 28127 0.0
    Streams AQ: qmn coordinator 257 51.4 3,488 13571 0.0
    pmon timer 1,235 100.0 3,486 2822 0.0
    smon timer 19 47.4 3,460 182099 0.0
    Streams AQ: waiting for time 5 60.0 745 148902 0.0
    class slave wait 1 100.0 5 4995 0.0
    Operating System Statistics DB/Inst: ABCTEST/abctest Snaps: 2151-2152
    Statistic Total
    AVG_BUSY_TIME 81,951
    AVG_IDLE_TIME 266,698
    AVG_SYS_TIME 10,482
    AVG_USER_TIME 71,389
    BUSY_TIME 328,163
    IDLE_TIME 1,067,144
    SYS_TIME 42,281
    USER_TIME 285,882
    RSRC_MGR_CPU_WAIT_TIME 0
    VM_IN_BYTES 1,625,600,000
    VM_OUT_BYTES 145,162,240
    PHYSICAL_MEMORY_BYTES 3,755,851,776
    NUM_CPUS 4
    NUM_CPU_CORES 1
    Brief AWR Report When Performance* Deteriorated.
    Snap Id Snap Time Sessions Curs/Sess
    Begin Snap: 2168 15-Dec-10 08:31:05 32 18.4
    End Snap: 2169 15-Dec-10 09:30:56 32 18.3
    Elapsed: 59.85 (mins)
    DB Time: 17.97 (mins)
    Cache Sizes
    ~~~~~~~~~~~ Begin End
    Buffer Cache: 448M 448M Std Block Size: 8K
    Shared Pool Size: 116M 116M Log Buffer: 6,968K
    Load Profile
    ~~~~~~~~~~~~ Per Second Per Transaction
    Redo size: 10,503.58 1,792.02
    Logical reads: 17,583.21 2,999.87
    Block changes: 68.60 11.70
    Physical reads: 472.37 80.59
    Physical writes: 1.54 0.26
    User calls: 39.12 6.67
    Parses: 53.32 9.10
    Hard parses: 7.99 1.36
    Sorts: 13.84 2.36
    Logons: 0.00 0.00
    Executes: 130.30 22.23
    Transactions: 5.86
    % Blocks changed per Read: 0.39 Recursive Call %: 94.39
    Rollback per transaction %: 0.00 Rows per Sort: 691.64
    Instance Efficiency Percentages (Target 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Buffer Nowait %: 100.00 Redo NoWait %: 100.00
    Buffer Hit %: 97.31 In-memory Sort %: 100.00
    Library Hit %: 92.41 Soft Parse %: 85.02
    Execute to Parse %: 59.08 Latch Hit %: 100.00
    Parse CPU to Parse Elapsd %: 100.28 % Non-Parse CPU: 95.35
    Shared Pool Statistics Begin End
    Memory Usage %: 88.40 88.48
    % SQL with executions>1: 76.15 80.48
    % Memory for SQL w/exec>1: 86.82 88.85
    Top 5 Timed Events Avg %Total
    ~~~~~~~~~~~~~~~~~~ wait Call
    Event Waits Time (s) (ms) Time Wait Class
    CPU time 918 85.1
    db file scattered read 113,003 127 1 11.7 User I/O
    log file parallel write 11,978 52 4 4.8 System I/O
    db file parallel write 3,089 16 5 1.4 System I/O
    control file parallel write 1,217 15 13 1.4 System I/O
    Time Model Statistics DB/Inst: ABCTEST/abctest Snaps: 2168-2169
    -> Total time in database user-calls (DB Time): 1078.1s
    -> Statistics including the word "background" measure background process
    time, and so do not contribute to the DB time statistic
    -> Ordered by % or DB time desc, Statistic name
    Statistic Name Time (s) % of DB Time
    sql execute elapsed time 1,032.1 95.7
    DB CPU 917.6 85.1
    parse time elapsed 71.8 6.7
    hard parse elapsed time 52.4 4.9
    PL/SQL execution elapsed time 7.2 .7
    PL/SQL compilation elapsed time 6.2 .6
    failed parse elapsed time 1.8 .2
    sequence load elapsed time 0.4 .0
    repeated bind elapsed time 0.3 .0
    connection management call elapsed time 0.1 .0
    hard parse (sharing criteria) elapsed time 0.0 .0
    hard parse (bind mismatch) elapsed time 0.0 .0
    DB time 1,078.1 N/A
    background elapsed time 89.4 N/A
    background cpu time 6.4 N/A
    Wait Class DB/Inst: ABCTEST/abctest Snaps: 2168-2169
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc
    Avg
    %Time Total Wait wait Waits
    Wait Class Waits -outs Time (s) (ms) /txn
    User I/O 122,810 .0 133 1 5.8
    System I/O 17,013 .0 83 5 0.8
    Commit 3,129 .0 14 5 0.1
    Network 90,186 .0 0 0 4.3
    Configuration 2 .0 0 63 0.0
    Application 1,120 .0 0 0 0.1
    Other 112 .0 0 0 0.0
    Concurrency 2 .0 0 6 0.0
    Wait Events DB/Inst: ABCTEST/abctest Snaps: 2168-2169
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc (idle events last)
    Avg
    %Time Total Wait wait Waits
    Event Waits -outs Time (s) (ms) /txn
    db file scattered read 113,003 .0 127 1 5.4
    log file parallel write 11,978 .0 52 4 0.6
    db file parallel write 3,089 .0 16 5 0.1
    control file parallel write 1,217 .0 15 13 0.1
    log file sync 3,129 .0 14 5 0.1
    db file sequential read 9,753 .0 6 1 0.5
    control file sequential read 725 .0 0 0 0.0
    Data file init write 32 .0 0 7 0.0
    SQL*Net message to client 88,906 .0 0 0 4.2
    log file switch completion 2 .0 0 63 0.0
    SQL*Net break/reset to clien 1,120 .0 0 0 0.1
    rdbms ipc reply 4 .0 0 8 0.0
    direct path write 10 .0 0 3 0.0
    SQL*Net more data to client 1,120 .0 0 0 0.1
    db file single write 2 .0 0 6 0.0
    os thread startup 2 .0 0 6 0.0
    log file single write 2 .0 0 4 0.0
    log file sequential read 2 .0 0 3 0.0
    SQL*Net more data from clien 160 .0 0 0 0.0
    LGWR wait for redo copy 108 .0 0 0 0.0
    direct path read 10 .0 0 0 0.0
    SQL*Net message from client 88,906 .0 55,500 624 4.2
    virtual circuit status 120 100.0 3,588 29900 0.0
    Streams AQ: qmn slave idle w 127 .0 3,550 27949 0.0
    Streams AQ: qmn coordinator 260 51.2 3,550 13652 0.0
    class slave wait 2 100.0 10 4994 0.0
    SGA: MMAN sleep for componen 9 22.2 0 4 0.0
    Background Wait Events DB/Inst: ABCTEST/abctest Snaps: 2168-2169
    -> ordered by wait time desc, waits desc (idle events last)
    Avg
    %Time Total Wait wait Waits
    Event Waits -outs Time (s) (ms) /txn
    log file parallel write 11,978 .0 52 4 0.6
    db file parallel write 3,089 .0 16 5 0.1
    control file parallel write 1,211 .0 15 13 0.1
    db file scattered read 175 .0 0 1 0.0
    control file sequential read 33 .0 0 2 0.0
    db file sequential read 53 .0 0 1 0.0
    direct path write 10 .0 0 3 0.0
    os thread startup 2 .0 0 6 0.0
    log file single write 2 .0 0 4 0.0
    log file sequential read 2 .0 0 3 0.0
    events in waitclass Other 108 .0 0 0 0.0
    direct path read 10 .0 0 0 0.0
    rdbms ipc message 19,991 57.4 31,320 1567 0.9
    pmon timer 1,208 100.0 3,590 2972 0.1
    Streams AQ: qmn slave idle w 127 .0 3,550 27949 0.0
    Streams AQ: qmn coordinator 260 51.2 3,550 13652 0.0
    smon timer 12 100.0 3,302 275149 0.0
    SGA: MMAN sleep for componen 9 22.2 0 4 0.0
    Operating System Statistics DB/Inst: ABCTEST/abctest Snaps: 2168-2169
    Statistic Total
    AVG_BUSY_TIME 30,152
    AVG_IDLE_TIME 328,781
    AVG_SYS_TIME 4,312
    AVG_USER_TIME 25,757
    BUSY_TIME 120,981
    IDLE_TIME 1,315,433
    SYS_TIME 17,612
    USER_TIME 103,369
    RSRC_MGR_CPU_WAIT_TIME 0
    VM_IN_BYTES 353,361,920
    VM_OUT_BYTES 163,041,280
    PHYSICAL_MEMORY_BYTES 3,755,851,776
    NUM_CPUS 4
    NUM_CPU_CORES 1
    Request you to help me.
    Thanks in Advance,
    Rajesh

    Hi CKPT,
    Thanks for your reply.
    The main finding that I have got from addm report (in both the cases i.e when performance was good initially vis a vis when performance deteriorated is the same -
    FINDING 1: 100% impact (3234 seconds)
    Significant virtual memory paging was detected on the host operating system.
    RECOMMENDATION 1: Host Configuration, 100% benefit (3234 seconds)
    ACTION: Host operating system was experiencing significant paging but no
    particular root cause could be detected. Investigate processes that
    do not belong to this instance running on the host that are consuming
    significant amount of virtual memory. Also consider adding more
    physical memory to the host.
    I still am unable to find out the reasons ... pls help.
    Thanks
    Rajesh

  • Wait events - how to read it

    Hi frnds,
    As, I'm beginner to performance tuning I dont know
    What action do i need to take?
    I mean how to read the output which I given below.
    this is the output suffering buffer busy waits.
    Could anyone please tell me
    CLASS TOTAL_WAITS TOTAL_TIME
    data block 93303 58711
    unused 0 0
    system undo header 12 232
    undo header 7847 6636
    3rd level bmb 0 0
    save undo header 0 0
    bitmap index block 0 0
    file header block 0 0
    free list 0 0
    undo block 68 207
    segment header 422 399
    extent map 0 0
    2nd level bmb 0 0
    system undo block 0 0
    sort block 0 0
    save undo block 0 0
    1st level bmb 1 17
    bitmap block 0 0
    Thanks, Muhammed Thameem. S

    Hello,
    "Buffer busy waits" is contention for a buffer (representing a specific
    version of a database block) within the Buffer Cache. So, in essence
    it is block contention and thus it is most likely something to do with
    the design of the tables and indexes supporting the application. A
    built-in bottleneck. On indexes, it could be the age-old problem of
    insertions into an index on a column with a monotonically-ascending
    data value (i.e. timestamps or sequence numbers) which tends to cause
    contention on the highest leaf node of the index. On tables, it might
    have to do with many concurrent insertions into a table in a
    freelist-managed tablespace where the table has only one freelist. It
    could also be due to a home-grown implementation of sequence-number
    generators (i.e. small table with one row, one column in which contains
    the "last value" of a sequence, etc) which lots of people use to avoid
    not being "portable across databases" which they think means not using
    Oracle sequences (yadda yadda yadda).
    I'd look for any SQL statement in the "SQL sorted by Elapsed Time"
    section of the AWR report which exhibits high elapsed time but
    relatively low CPU time, indicating a lot of wait time. Of course,
    there are something like 800 possible wait events in current releases
    of Oracle, of which "buffer busy waits" is only one, so this is just
    inference and not a direct causal connection to your problem. But,
    once I find such statements I'd check to see if they are
    accessing/manipulating tables within the CUBS_DATA tablespace, and then
    use "select * from table(dbms_xplan.display_awr('sql-id'))" to
    get the execution plan(s), and then look for something ineffective
    within the execution plan. You might find the script "sqlhistory.sql" helpful
    here as well, to get a "historical perspective" on the execution of the
    SQL statements over time, in case the buffer busy waits peaked at some
    point in the past
    Please refer to:
    http://www.pubbs.net/201003/oracle/51925-understanding-awr-buffer-waits.html
    Also
    http://www.remote-dba.net/oracle_10g_tuning/t_buffer_busy_waits.htm
    kind regards
    Mohamed

Maybe you are looking for

  • I installed Adobe Acrobat Pro XI on my Windows 7 Ultimate (64 bit) machine. It won't start.

    I installed Adobe Acrobat Pro XI on my Windows 7 Ultimate (64 bit) machine. I double click the desktop icon and double click on a PDF file and the hour glass (circle) appears for about 5 seconds but no Adobe Acrobat.

  • Extreme Slow Down In Speeds During Peak Hours

    1. Old LJU Single Master Socket, using new microfilter 2. Quite line test comes back silent 3. On an Unlimited Usage package 4. This problem has been occuring for the past couple of weeks now

  • 06 IMovie question on resizing a clip

    How do I resize a video clip from 600 to uner 100m? It's not even a minute long. I have Imovie, streemclip and quicktime softward. thanks

  • Zip files related to startCD

    Hi, Can you please help me what are the zip files in 12.1.1 for Linux86-64 related to startCD. I think these are related to startCD. B53824-01_1of4.zip B53824-01_2of4.zip B53824-01_3of4.zip B53824-01_4of4.zip Thanks, Kavitha

  • Deploy to Apache in 9i Rel2

    Hi, I'm learning Jdeveloper and tried the tutorials for creating JSP/BC4J. I substituted the OE schema for one of my own and ran the tutorials. I created pages just fine, and ran them in the self-contained server that you use to test with. However, I