What is "ges enter server mode" wait event..

I cannot find 'ges enter server mode' wait event definition.
What is it?

10g onwards, Instance recovery is done in two phases. First phase scans the blocks to be recoverd and applied from rdo log files and the second phase actually does that.
In a RAC instance the during the instance recovery, first pass scan can be delayed by 300ms-1.5s waiting on GRD (Global resource directory). During that time SMAON enters the wait event which is called "ges enter server mode" .

Similar Messages

  • What is ges reusing os pid wait event

    What is wait event "ges reusing os pid". In our RAC environment it is one of the top wait events. How to minimze it.

    This is a wait event in Oracle 10g for Global Enqueue Services (ges) waiting on an operating system process id (os pid).
    How to resolve this issue? I checked the bug list on Metalink and there is a patch set for the issue that may help.
    Question: what version and patch release are you running for Oracle RAC?
    Also, you probably want to tune your public network and private interconnects between the nodes in your Oracle RAC cluster.
    Regards,
    Ben Prusinski
    http://oracle-magician.blogspot.com/

  • What's operation can cause these wait event?

    Hi all,
    what's operation can cause "library cache lock" and "library cache pin"?

    So surely you went first to the [url http://docs.oracle.com/cd/E11882_01/server.112/e16638/instance_tune.htm#PFGRF94529]documentation. Right?

  • What is "KJC: Wait for msg .." wait event in 10g??

    Hi, all.
    The database is 2 node RAC database (10.2.0.2.0)
    on 32-bit windows 2003 EE SP1.
    I found "KJC: Wait for msg sends to complete" wait event in
    "Top 5 Timed Event" Section from AWR report.
    What is "KJC: Wait for msg sends to complete" wait event??
    The following is from UDUMP.
    Dump file d:\oracle\product\10.2.0\admin\rac\udump\rac2_ora_5656.trc
    Mon Sep 24 00:04:40 2007
    ORACLE V10.2.0.2.0 - Production vsnsta=0
    vsnsql=14 vsnxtr=3
    Oracle Database 10g Enterprise Edition Release 10.2.0.2.0 - Production
    With the Real Application Clusters, OLAP and Data Mining options
    Windows Server 2003 Version V5.2 Service Pack 1
    CPU : 4 - type 586, 2 Physical Cores
    Process Affinity : 0x00000000
    Memory (Avail/Total): Ph:5278M/8190M, Ph+PgF:6596M/10041M, VA:316M/2047M
    Instance name: rac2
    Redo thread mounted by this instance: 2
    Oracle process number: 64
    Windows thread id: 5656, image: ORACLE.EXE (SHAD)
    *** 2007-09-24 00:04:40.156
    *** ACTION NAME:() 2007-09-24 00:04:40.156
    *** MODULE NAME:(OEM.SystemPool) 2007-09-24 00:04:40.156
    *** SERVICE NAME:(RAC.world) 2007-09-24 00:04:40.156
    *** CLIENT ID:() 2007-09-24 00:04:40.156
    *** SESSION ID:(486.53) 2007-09-24 00:04:40.156
    IPCSendMsg: could not initiate send on conn 0x5b0d3e98 to node [rac1 : 696 : 3996 : 359937], err 10054
    IPCGetRequestInfo: failed a request rqh(0x5b060db8), type(6), status(2), bytes(0)
    Thanks and Regards.
    Message was edited by:
    user507290
    Message was edited by:
    user507290

    This might have something to do with bug 5075434 - Small performance overhead in RAC (waits for "KJC: Wait for msg sends to complete").
    Check metalink for further details.

  • Enter query - what was changed (entered) in enter query mode

    Hello,
    I need to know what criteria (values and items) an user entered (typed) when my form is in enter query mode.
    Thanks

    You can read about Pre-Query trigger in the Forms on-line help:
    About controlling queries with Pre-Query and Post-Query triggers
    The Pre-Query and Post-Query triggers allow control over query processing. They can be defined at the form or block level. Most often, attach them to specific blocks to control the query functionality of those blocks.
    The Pre-Query trigger fires just before Form Builder issues the SELECT statement to the database, after the operator has defined the example record by entering query criteria in Enter Query mode.
    Inside a Pre-Query trigger, the example record defined by the query criteria is the current record. This means that trigger code can read and set the values of items in the example record using standard :block_name.item_name syntax.
    A Pre-Query trigger can be used to disallow query conditions that might be invalid. When a form is in Enter Query mode, normal validation is suspended and no validation triggers fire as they do in Normal mode. The Pre-Query trigger thus allows you to verify that any values entered by the operator are valid query conditions.
    When invalid query conditions have been entered, you can abort the query by raising the FORM_TRIGGER_FAILURE built-in exception in the Pre-Query trigger.
    You can also call SET_BLOCK_PROPERTY to modify the block's WHERE and ORDER BY clauses from within the Pre-Query trigger, to further restrict or order the records the query will retrieve.
    I hope this will help you.
    Helena

  • What is "reliable message" wait event in 10g??

    Hi, all.
    The database is 2 node RAC 10.2.0.2.0 on 32-bit windows 2003 EE SP1.
    From time to time, I see "reliable message" wait event on the
    "Top 5 Timed Events" section from AWR report.
    The wait class is "Other".
    There seems to be no manuals and html links explaining "reliable message"
    wait event.
    Thanks and Regards.
    Message was edited by:
    user507290
    Message was edited by:
    user507290

    Metalink Doc 461052.1 is a fairly lightweight walk-through of a system resource profiling tool called LTOM, the "Lite Onboard Monitor". It has an appendix in which it simply points out "The following events were considered idle and ignored... reliable message..."
    It also says that if you want to include something for wait analysis, just take it out of the list of idle and ignored events... so it's not exactly saying that the thing is an idle event, just that it was treated as such for the purposes of writing that document but that you might think differently.
    On the other hand, this page:
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents003.htm
    ...is 10g Release 2's description of "wait events of greatest interest"... and "reliable message" isn't on it! From which one concludes, I suppose, that it isn't of great interest!

  • What is "reliable message" wait event?

    Hi, all.
    What is "reliable message" wait event?
    The wait class is "Other", not "Idle".
    From time to time, I can see "reliable message" wait event
    in the Top 5 Timed Events section from AWR reports.
    In addition, what is "CPU time" in the Top 5 Timed Events section?
    Thanks and Regards.
    Message was edited by:
    user507290

    Dear Yas.
    Thanks for your reply.
    The database is 2 node RAC database (10.2.0.2.0) on 32 bit windows 2003 EE SP1.
    Does "AQ" mean Streams AQ???
    Can I disable Streams AQ facility??
    Thanks and Regards.
    Message was edited by:
    user507290
    Message was edited by:
    user507290

  • The remote server returned an error: 227 Entering Passive Mode (201,199,193,165,154,52).

    Hi I have a problem 
    The remote server returned an error: 227 Entering Passive Mode (201,199,193,165,154,52).
    I dont knoe how to solve it 

    Hi,
    Please see the following links and try the methods to check if they are helpful:
    http://geekswithblogs.net/hmloo/archive/2012/10/16/the-remote-server-returned-an-error-227-entering-passive-mode.aspx
    http://social.msdn.microsoft.com/Forums/en-US/0128e595-c8e2-4f5e-9426-fd93eb510cab/the-remote-server-returned-an-error-227-entering-passive-mode-67228534212130?forum=netfxnetcom
    Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.
    Then, it is not a Microsoft Office issue, you'd better post your question to MSDN forum:
    http://social.msdn.microsoft.com/Forums/en-US/home?forum=netfxnetcom
    Regards,
    George Zhao
    TechNet Community Support

  • What's wrong from this wait event

    Please,
    Below are two tables showing respectively database wait event by wait class and session wait event by wait class.
    1. Database wait event by wait class
    WAIT TOTAL PCT TIME PCT
    CLASS WAITS WAITS WAITED_SECS TIME
    Application 7427 .08 1572.45 76.29
    User I/O 50416 .57 193.24 9.38
    Network 8714874 98.66 177.67 8.62
    System I/O 48169 .55 85.98 4.17
    2. Session wait event by wait class
    SID USER WAIT TOTAL TIME_
    NAME CLASS WAITS WAITED_SECS
    318 PMS1000 Application 110 321.64
    259 PMS1000 Application 81 212.8
    318 PMS1000 Network 541943 31.8
    259 PMS1000 Network 258368 17.76
    258 PMS1000 Network 132774 9.34
    318 PMS1000 User I/O 1392 7.49
    Top Events found:
    CPU + WAIT for CPU
    ROW lock contention
    SQL*Net more data from/to client
    Question:
    What may cause the application wait_class to be at the top?, event though the row lock contention was found ?
    I also think the system may sufering from a network bottleneck, I also thinking to set SDU parameter, but the network is 1Gb speed, and I don't know if this can help.
    Does someone give me some clue to pinpoint what is going wrong wiht the above stats?
    thanks enough

    user552326
    I've used the "code" tags to make your first section of data more readable:
    WAIT               TOTAL       PCT        TIME             PCT
    CLASS              WAITS       WAITS      WAITED_SECS      TIME
    Application              7427        .08           1572.45      76.29
    User I/O                50416        .57            193.24       9.38
    Network               8714874      98.66            177.67       8.62
    System I/O              48169        .55             85.98       4.17 If you want to know what events belong to each wait class you can query v$event_name:
    select wait_class, name
    from v$event_name
    order by wait_class, nameThe events in class "Application" are:
    SQL*Net break/reset to client
    SQL*Net break/reset to dblink
    Streams capture: filter callback waiting for ruleset
    Streams: apply reader waiting for DDL to apply
    Wait for Table Lock
    enq: KO - fast object checkpoint
    enq: PW - flush prewarm buffers
    enq: RO - contention
    enq: RO - fast object reuse
    enq: TM - contention
    enq: TX - row lock contention
    enq: UL - contention
    As you can see, this is consistent with your comment about the top event being "row lock contention". The implication of the name given to this wait class is that it is your application design that is causing the problem. Your biggest problem is that your code allows your users to lock each other out.
    Looking at the summary numbers, the time spent on waiting for other users to get out of the way is a very large fraction of your total wait - until you deal with that, problems relating to I/O and network appear to be pretty irrelevant. Having said that, you seem to do a very large number of round-trips on the network - so maybe the amount of time you are losing is not hugely significant compared to the amount of work you are getting done. (You didn't actually tell us how long it took or how many concurrent users, to accumulate this wait time).
    Regards
    Jonathan Lewis
    http://jonathanlewis.wordpress.com
    http://www.jlcomp.demon.co.uk

  • HT201407 i Perform this , but stil problem occur, could not activate, what can i do. it say server is temporary unavialable. Actually i update my iphone 3gs from 4.1 to 6.1.3 ...   What can i do, reply iam waiting for answer.

    i Perform this , but stil problem occur, could not activate, what can i do. it say server is temporary unavialable. Actually i update my iphone 3gs from 4.1 to 6.1.3 ...   What can i do, reply iam waiting for answer.

    Is your phone jailbroken by any chance?

  • Server won't finish booting...can't enter safe mode...won't read install

    My G4 XServe starts to boot, I see the Apple, then it shifts to the Starting Mac.... screen, and then it just stops on an empty blue screen. I see the server going through and checking each of the four hard drives, system activity lights do their thing, but the blue screen remains.
    I cannot get the server to boot from the install CD or DVD (using 10.4.8 and have the 10.4 disks). It will not let me set startup mode by holding down the system identifier button when powering up. I can't get it to accept holding down the C or D keys on the keyboard when starting. When I hold down the shift key it goes to the console and asks me to log in there instead of entering safe mode.
    Any ideas or suggestions?

    Are you able to start in Verbose mode?
    Hold down "apple" and "v" at startup. This will take you through a logging bootup and into the GUI. Try and see if you can mount it to another machine via firewire disk mode. From there you can check it with Disk Utility or Diskwarrior (provided it mounts).
    Or try start in single user mode "apple" and "s" and when you get to the prompt "#" type "fsck -f"
    See if any volume repairs are conducted. If they complete or if it works you type "reboot" when done and the server should restart.
    Good luck

  • What do the wait events 'gc cr failure' and 'cr request retry' mean?

    I'm trying to troubleshoot an issue for a customer. Environment is Oracle 10.2.0.4 (64-bit) on Redhat 5. Two node RAC cluster. The 10046 trace file shows lots of 'gc current block 2-way' waits but also a few 'gc cr failure' and 'cr request retry' waits. The 'cr request retry' waits take about 0.9 seconds each. I cannot find much if any information on these two wait events. Any help is much appreciated.
    Thanks!

    Hi,
    Also, you might need to check the protocol is being used for the interconnect communication.
    here are the steps just in case:
    $ORACLE_HOME/bin/sqlplus / as sysdba
    oradebug setmypid
    oradebug unlimit
    oradebug ipc
    oradebug TRACEFILE_NAME
    Review the file that the last oradebug gave you back.
    Metalink note 181489.1 provides some handy steps to analyze your situation. (also contains the latest supported protocols for IPC)
    Hope this helps
    Regards,
    Jozsef

  • Shared and Dedicated server modes

    Hi,
    We have been having some issues with connections and listeners. Please find here... Link [Post1|http://forums.oracle.com/forums/message.jspa?messageID=3665455] and Link [Post2|http://forums.oracle.com/forums/message.jspa?messageID=3677914].
    Despite this help from this forum, I still face some problem configuring listener in both Shared and Dedicated server modes.
    Pls find the below the problem...
    Two listeners namely, LSNR1 and LSNR2 have been started.
    listener.ora:
    LSNR2 =
      (DESCRIPTION_LIST =
        (DESCRIPTION =
          (ADDRESS = (PROTOCOL = TCP)(HOST = dbdev.website.org)(PORT = 1522))
          (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0))
    SID_LIST_LSNR2  =
      (SID_LIST =
        (SID_DESC =
          (SID_NAME = TESTDB)
          (ORACLE_HOME = /u02/app/oracle)
    LSNR1 =
      (DESCRIPTION_LIST =
        (DESCRIPTION =
          (ADDRESS = (PROTOCOL = TCP)(HOST = dbdev.website.org)(PORT = 1521))
          (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0))
    SID_LIST_LSNR1 =
      (SID_LIST =
        (SID_DESC =
          (SID_NAME = TESTDB)
          (ORACLE_HOME = /u02/app/oracle)
      )LSNR1 works with both Shared and Dedicated configurations, whereas LSNR2 neither DEDICATED nor SHARED configurations...
    Here is the TNS names entry at the client (for LSNR2)...
    serv2=
    (DESCRIPTION =
        (ADDRESS_LIST =
          (ADDRESS = (PROTOCOL = TCP)(HOST = xxx.xxx.xxx.xxx)(PORT = 1522))
        (CONNECT_DATA =
          (SERVICE_NAME=DEDICATED)
      )When I try to connect using this, it says
    + ORA-12514: TNS: Listener does not know of service requested in connect descriptor +
    Pls help.
    Thanks,
    Aswin.

    Aswin,
    I didnt see the other posts but the error is fairly simple,
    >
    ORA-12514:
    TNS:listener does not currently know of service requested in connect descriptor
    Cause:     The listener received a request to establish a connection to a database or other service. The connect descriptor received by the listener specified a service name for a service (usually a database service) that either has not yet dynamically registered with the listener or has not been statically configured for the listener. This may be a temporary condition such as after the listener has started, but before the database instance has registered with the listener.
    Action:     
    - Wait a moment and try to connect a second time.
    - Check which services are currently known by the listener by executing: lsnrctl services <listener name>
    - Check that the SERVICE_NAME parameter in the connect descriptor of the net service name used specifies a service known by the listener.
    - If an easy connect naming connect identifier was used, check that the service name specified is a service known by the listener.
    - Check for an event in the listener.log file.
    >
    So check what you are passing in the connection string.
    HTH
    Aman....

  • Tuning row lock contention wait events

    Hello everyone,
    Working on 10g/windows
    Top 5 events
    EVENT TOTAL_WAITS TIME_WAITED AVG_MS PERCENT
    CPU 9462339 48
    enq: TX - row lock contention 12531 3660728 2921.34 18
    control file parallel write 1300731 3088079 23.74 16
    log file parallel write 1510503 1264080 8.37 6
    log file sync 1072553 968007 9.03 5
    Distribution of row lock wait during the last 4 days in the database server
    END_INTERVAL_TIME TOTAL_WAITS TIME_WAITED_MICRO AVG_WAIT_MS
    2008-04-01 16:00:58 909 2721008230 2993.41
    2008-04-01 15:00:27 50 149941140 2998.82
    2008-03-31 12:00:42 193 575595397 2982.36
    2008-03-29 23:00:13 172 513058700 2982.9
    2008-03-29 22:00:37 164 483940046 2950.85
    2008-03-27 22:00:35 565 1667120838 2950.66
    2008-03-26 18:00:59 348 1042918982 2996.89
    My analysis:
    It's obvious that the row lock contention wait time is huge, and this direct me to find out SQL stmt, causing this.
    all the SQL statement was SELECT ....... FOR UPDATE stmt.
    I was also able to find out locked tables.
    My tuning idea:
    1. I'm thinking to reorganize hot tables as well as their indexes, but by instinct it seems to not give so much value to avoid the huge row lock wait time.
    2. I'm also seeing if I can reduce the number of rows per block, by increasing PCTFREE and diminishing PCTUSED, so the contention will spread over many blocks instead of one heavy block.
    Question
    As SQL stmt related to those locked tables are select ... for update, how could I tune this kind of stmt?
    Does someone have other idea to come up with this row lock contention?
    Tanks for your effort and help

    Taking another look at your suggested function based index, it depends on the data type of the DEV.POS_FOLIO_ID.POS_FOLIO_ID column. If the column is defined as a number, and it is a primary key, there will already be a usable index on that column.
    Yesterday, I wrote this: "Once I understood why or how the sessions were trying to insert duplicate primary key values, I would try to determine why the average number of seconds for the wait event is almost 3 seconds (maybe a timeout)."
    After fixing the formatting of the top 5 wait events (total duration unknown):
    EVENT                        TOTAL_WAITS  TIME_WAITED   AVG_MS PERCENT
    CPU                                         94,623.39             48
    enq: TX - row lock contention     12,531    36,607.28  2921.34    18
    control file parallel write    1,300,731    30,880.79    23.74    16
    log file parallel write        1,510,503    12,640.80     8.37     6
    log file sync                  1,072,553     9,680.07     9.03     512,531 * 3 second time out = 37,593 seconds = 10.44 hours.
    What if the reason for the 3 second average wait time is due to a timeout. I performed a little experiment... I changed a row in a test table and then made a pot of coffee.
    In session 1:
    CREATE TABLE T1 (
      C1 NUMBER(10),
      C2 NUMBER(10),
      PRIMARY KEY (C1));
    INSERT INTO T1
    SELECT
      ROWNUM,
      ROWNUM*10
    FROM
      DUAL
    CONNECT BY
      LEVEL<=1000000;
    COMMIT;I now have a test table with 1,000,000 rows. I start monitoring the changes in the wait events roughly every 60 seconds, and V$SESSION_WAIT and V$LOCK roughly 4 times per second.
    Back in session 1:
    UPDATE
      T1
    SET
      C1=-C1
    WHERE
      C1<=100;I have now modified the first 100 rows that were inserted into the table, time to make the pot of coffee.
    In session 2, I try to insert a row with a primary key value of -10:
    INSERT INTO T1 VALUES (
      -10,
      10);Session 2 hangs.
    If I take the third 60 second snap of the system wide wait events as the zero point, and the 11th snap as the end point. There were 149 waits on ENQ: TX - ROW LOCK CONTENTION, 148 time outs, 446.62 seconds of total time in the wait event, with an average wait time of 2.997450 seconds.
    Rolling down to the session level wait events, SID 208 (my session 2) had 149 waits on ENQ: TX - ROW LOCK CONTENTION, for a total time of 446.61 seconds with an average wait time of 2.997383 seconds. All of the 149 waits and the wait time was in this one session that was locked up for the full duration of this time period because session 1 was making a pot of coffee.
    Rolling down to V$SESSION_WAIT (sampled roughly 4 times per second): At the start of the third time interval, SID 208 has been in the ENQ: TX - ROW LOCK CONTENTION wait event for 39 seconds and is actively waiting trying to execute SQL with a hash value of 1001532423, the wait object is -1, wait file is 0, wait block is 0, wait row is 0, P1 is 1415053316, P2 is 196646, P3 is 4754.
    At the end of the 11th time interval: , SID 208 has been in the ENQ: TX - ROW LOCK CONTENTION wait event for 483 seconds and is actively waiting trying to execute SQL with a hash value of 1001532423, the wait object is -1, wait file is 0, wait block is 0, wait row is 0, P1 is 1415053316, P2 is 196646, P3 is 4754.
    Rolling down to V$LOCK (sampled roughly 4 times per second): I see that SID 214 (session 1) is blocking SID 208 (session 2). SID 214 has a TX lock in mode 6 with ID1 of 196646 and ID2 of 4754. SID 208 is requesting a TX lock in mode 4 with ID1 of 196646 and ID2 of 4754.
    So, it seems that I need a faster coffee pot rather than an additional index on my table. It could be that the above process would have found that the application associated with SID 214 was abandoned or crashed and for some reason the lock was not released for a long period of time, a little less than 10.44 hours in your case.
    Charles Hooper
    IT Manager/Oracle DBA
    K&M Machine-Fabricating, Inc.

  • Need help to analysis "foreground and background wait events" on statspack report for oracle database 11.2.0.4 on AIX

    Hi: I'm analyzing this STATSPACK report: it is "volume test" on our UAT server, so most input is from 'bind variables'.  Our shared pool is well utilized in oracle.  Oracle redo logs is not appropriately configured on this server, as in 'Top 5 wait events' there are 2 for redos.
    I need to know what else information can be dig-out from 'foreground wait events' & 'background wait events', and what can assist us to better understanding, in combination of 'Top 5 wait event's, that how the server/test went?  it could be overwelming No. of wait events, so appreciate any helpful diagnostic or analysis.  Database is oracle 11.2.0.4 upgraded from 11.2.0.3, on IBM AIX power system 64bit, level 6.x
    STATSPACK report for
    Database    DB Id    Instance     Inst Num  Startup Time   Release     RAC
    ~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
    700000XXX   XXX              1 22-Apr-15 12:12 11.2.0.4.0  NO
    Host Name             Platform                CPUs Cores Sockets   Memory (G)
    ~~~~ ---------------- ---------------------- ----- ----- ------- ------------
         dXXXX_XXX    AIX-Based Systems (64-     2     1       0         16.0
    Snapshot       Snap Id     Snap Time      Sessions Curs/Sess Comment
    ~~~~~~~~    ---------- ------------------ -------- --------- ------------------
    Begin Snap:       5635 22-Apr-15 13:00:02      114       4.6
      End Snap:       5636 22-Apr-15 14:00:01      128       8.8
       Elapsed:      59.98 (mins) Av Act Sess:       0.6
       DB time:      35.98 (mins)      DB CPU:      19.43 (mins)
    Cache Sizes            Begin        End
    ~~~~~~~~~~~       ---------- ----------
        Buffer Cache:     2,064M              Std Block Size:         8K
         Shared Pool:     3,072M                  Log Buffer:    13,632K
    Load Profile              Per Second    Per Transaction    Per Exec    Per Call
    ~~~~~~~~~~~~      ------------------  ----------------- ----------- -----------
          DB time(s):                0.6                0.0        0.00        0.00
           DB CPU(s):                0.3                0.0        0.00        0.00
           Redo size:          458,720.6            8,755.7
       Logical reads:           12,874.2              245.7
       Block changes:            1,356.4               25.9
      Physical reads:                6.6                0.1
    Physical writes:               61.8                1.2
          User calls:            2,033.7               38.8
              Parses:              286.5                5.5
         Hard parses:                0.5                0.0
    W/A MB processed:                1.7                0.0
              Logons:                1.2                0.0
            Executes:              801.1               15.3
           Rollbacks:                6.1                0.1
        Transactions:               52.4
    Instance Efficiency Indicators
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                Buffer Nowait %:  100.00       Redo NoWait %:  100.00
                Buffer  Hit   %:   99.98  Optimal W/A Exec %:  100.00
                Library Hit   %:   99.77        Soft Parse %:   99.82
             Execute to Parse %:   64.24         Latch Hit %:   99.98
    Parse CPU to Parse Elapsd %:   53.15     % Non-Parse CPU:   98.03
    Shared Pool Statistics        Begin   End
                 Memory Usage %:   10.50   12.79
        % SQL with executions>1:   69.98   78.37
      % Memory for SQL w/exec>1:   70.22   81.96
    Top 5 Timed Events                                                    Avg %Total
    ~~~~~~~~~~~~~~~~~~                                                   wait   Call
    Event                                            Waits    Time (s)   (ms)   Time
    CPU time                                                       847          50.2
    enq: TX - row lock contention                    4,480         434     97   25.8
    log file sync                                  284,169         185      1   11.0
    log file parallel write                        299,537         164      1    9.7
    log file sequential read                           698          16     24    1.0
    Host CPU  (CPUs: 2  Cores: 1  Sockets: 0)
    ~~~~~~~~              Load Average
                          Begin     End      User  System    Idle     WIO     WCPU
                           1.16    1.84     19.28   14.51   66.21    1.20   82.01
    Instance CPU
    ~~~~~~~~~~~~                                       % Time (seconds)
                         Host: Total time (s):                  7,193.8
                      Host: Busy CPU time (s):                  2,430.7
                       % of time Host is Busy:      33.8
                 Instance: Total CPU time (s):                  1,203.1
              % of Busy CPU used for Instance:      49.5
            Instance: Total Database time (s):                  2,426.4
      %DB time waiting for CPU (Resource Mgr):       0.0
    Memory Statistics                       Begin          End
    ~~~~~~~~~~~~~~~~~                ------------ ------------
                      Host Mem (MB):     16,384.0     16,384.0
                       SGA use (MB):      7,136.0      7,136.0
                       PGA use (MB):        282.5        361.4
        % Host Mem used for SGA+PGA:         45.3         45.8
    Foreground Wait Events  DB/Inst: XXXXXs  Snaps: 5635-5636
    -> Only events with Total Wait Time (s) >= .001 are shown
    -> ordered by Total Wait Time desc, Waits desc (idle events last)
                                                                 Avg          %Total
                                              %Tim Total Wait   wait    Waits   Call
    Event                               Waits  out   Time (s)   (ms)     /txn   Time
    enq: TX - row lock contentio        4,480    0        434     97      0.0   25.8
    log file sync                     284,167    0        185      1      1.5   11.0
    Disk file operations I/O            8,741    0          4      0      0.0     .2
    direct path write                  13,247    0          3      0      0.1     .2
    db file sequential read             6,058    0          1      0      0.0     .1
    buffer busy waits                   1,800    0          1      1      0.0     .1
    SQL*Net more data to client        29,161    0          1      0      0.2     .1
    direct path read                    7,696    0          1      0      0.0     .0
    db file scattered read                316    0          1      2      0.0     .0
    latch: shared pool                    144    0          0      2      0.0     .0
    CSS initialization                     30    0          0      3      0.0     .0
    cursor: pin S                          10    0          0      9      0.0     .0
    row cache lock                         41    0          0      2      0.0     .0
    latch: row cache objects               19    0          0      3      0.0     .0
    log file switch (private str            8    0          0      7      0.0     .0
    library cache: mutex X                 28    0          0      2      0.0     .0
    latch: cache buffers chains            54    0          0      1      0.0     .0
    latch free                            290    0          0      0      0.0     .0
    control file sequential read        1,568    0          0      0      0.0     .0
    log file switch (checkpoint             4    0          0      6      0.0     .0
    direct path sync                        8    0          0      3      0.0     .0
    latch: redo allocation                 60    0          0      0      0.0     .0
    SQL*Net break/reset to clien           34    0          0      1      0.0     .0
    latch: enqueue hash chains             45    0          0      0      0.0     .0
    latch: cache buffers lru cha            7    0          0      2      0.0     .0
    latch: session allocation               5    0          0      1      0.0     .0
    latch: object queue header o            6    0          0      1      0.0     .0
    ASM file metadata operation            30    0          0      0      0.0     .0
    latch: In memory undo latch            15    0          0      0      0.0     .0
    latch: undo global data                 8    0          0      0      0.0     .0
    SQL*Net message from client     6,362,536    0    278,225     44     33.7
    jobq slave wait                     7,270  100      3,635    500      0.0
    SQL*Net more data from clien        7,976    0         15      2      0.0
    SQL*Net message to client       6,362,544    0          8      0     33.7
    Background Wait Events  DB/Inst: XXXXXs  Snaps: 5635-5636
    -> Only events with Total Wait Time (s) >= .001 are shown
    -> ordered by Total Wait Time desc, Waits desc (idle events last)
                                                                 Avg          %Total
                                              %Tim Total Wait   wait    Waits   Call
    Event                               Waits  out   Time (s)   (ms)     /txn   Time
    log file parallel write           299,537    0        164      1      1.6    9.7
    log file sequential read              698    0         16     24      0.0    1.0
    db file parallel write              9,556    0         13      1      0.1     .8
    os thread startup                     146    0         10     70      0.0     .6
    control file parallel write         2,037    0          2      1      0.0     .1
    Log archive I/O                        35    0          1     30      0.0     .1
    LGWR wait for redo copy             2,447    0          0      0      0.0     .0
    db file async I/O submit            9,556    0          0      0      0.1     .0
    db file sequential read               145    0          0      2      0.0     .0
    Disk file operations I/O              349    0          0      0      0.0     .0
    db file scattered read                 30    0          0      4      0.0     .0
    control file sequential read        5,837    0          0      0      0.0     .0
    ADR block file read                    19    0          0      4      0.0     .0
    ADR block file write                    5    0          0     15      0.0     .0
    direct path write                      14    0          0      2      0.0     .0
    direct path read                        3    0          0      7      0.0     .0
    latch: shared pool                      3    0          0      6      0.0     .0
    log file single write                  56    0          0      0      0.0     .0
    latch: redo allocation                 53    0          0      0      0.0     .0
    latch: active service list              1    0          0      3      0.0     .0
    latch free                             11    0          0      0      0.0     .0
    rdbms ipc message                 314,523    5     57,189    182      1.7
    Space Manager: slave idle wa        4,086   88     18,996   4649      0.0
    DIAG idle wait                      7,185  100      7,186   1000      0.0
    Streams AQ: waiting for time            2   50      4,909 ######      0.0
    Streams AQ: qmn slave idle w          129    0      3,612  28002      0.0
    Streams AQ: qmn coordinator           258   50      3,612  14001      0.0
    smon timer                             43    2      3,605  83839      0.0
    pmon timer                          1,199   99      3,596   2999      0.0
    SQL*Net message from client        17,019    0         31      2      0.1
    SQL*Net message to client          12,762    0          0      0      0.1
    class slave wait                       28    0          0      0      0.0
    thank you very much!

    Hi: just know it now: it is a large amount of 'concurrent transaction' designed in this "Volume Test" - to simulate large incoming transaction volme, so I guess wait in eq:TX - row is expected.
    The fact: (1) redo logs at uat server is known to not well-tune for configurations (2) volume test slow 5%, however data amount in its test is kept the same by each time import  production data, by the team. So why it slowed 5% this year?
    The wait histogram is pasted below, any one interest to take a look?  any ideas?
    Wait Event Histogram  DB/Inst: XXXX/XXXX  Snaps: 5635-5636
    -> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
    -> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
    -> % of Waits - value: .0 indicates value was <.05%, null is truly 0
    -> Ordered by Event (idle events last)
                               Total ----------------- % of Waits ------------------
    Event                      Waits  <1ms  <2ms  <4ms  <8ms <16ms <32ms  <=1s   >1s
    ADR block file read          19   26.3   5.3  10.5  57.9
    ADR block file write          5                     40.0        60.0
    ADR file lock                 6  100.0
    ARCH wait for archivelog l   14  100.0
    ASM file metadata operatio   30  100.0
    CSS initialization           30              100.0
    Disk file operations I/O   9090   97.2   1.4    .6    .4    .2    .1    .1
    LGWR wait for redo copy    2447   98.5    .5    .4    .2    .2    .2    .1
    Log archive I/O              35   40.0         8.6  25.7   2.9        22.9
    SQL*Net break/reset to cli   34   85.3   8.8         5.9
    SQL*Net more data to clien   29K  99.9    .0    .0    .0          .0    .0
    buffer busy waits          1800   96.8    .7    .7    .6    .3    .4    .5
    control file parallel writ 2037   90.7   5.0   2.1    .8   1.0    .3    .1
    control file sequential re 7405  100.0                      .0
    cursor: pin S                10   10.0                    90.0
    db file async I/O submit   9556   99.9    .0                .0          .0
    db file parallel read         1  100.0
    db file parallel write     9556   62.0  32.4   1.7    .8   1.5   1.3    .1
    db file scattered read      345   72.8   3.8   2.3  11.6   9.0    .6
    db file sequential read    6199   97.2    .2    .3   1.6    .7    .0    .0
    direct path read           7699   99.1    .4    .2    .1    .1    .0
    direct path sync              8   25.0  37.5  12.5  25.0
    direct path write            13K  97.8    .9    .5    .4    .3    .1    .0
    enq: TX - row lock content 4480     .4    .7   1.3   3.0   6.8  12.3  75.4    .1
    latch free                  301   98.3    .3    .7    .7
    latch: In memory undo latc   15   93.3   6.7
    latch: active service list    1              100.0
    latch: cache buffers chain   55   94.5                     3.6   1.8
    latch: cache buffers lru c    9   88.9                    11.1
    latch: call allocation        6  100.0
    latch: checkpoint queue la    3  100.0
    latch: enqueue hash chains   45   97.8                     2.2
    latch: messages               4  100.0
    latch: object queue header    7   85.7        14.3
    latch: redo allocation      113   97.3               1.8    .9
    latch: row cache objects     19   89.5                           5.3   5.3
    latch: session allocation     5   80.0              20.0
    latch: shared pool          147   90.5   1.4   2.7   1.4    .7   1.4   2.0
    latch: undo global data       8  100.0
    library cache: mutex X       28   89.3         3.6         3.6         3.6
    log file parallel write     299K  95.6   2.6   1.0    .4    .3    .2    .0
    log file sequential read    698   29.5    .1               4.6  46.8  18.9
    log file single write        56  100.0
    log file switch (checkpoin    4               25.0  50.0  25.0
    log file switch (private s    8         12.5        37.5  50.0
    log file sync               284K  93.3   3.7   1.4    .7    .5    .3    .1
    os thread startup           146                                      100.0
    row cache lock               41   85.4   9.8               2.4         2.4
    DIAG idle wait             7184                                      100.0
    SQL*Net message from clien 6379K  86.6   5.1   2.9   1.3    .7    .3   2.8    .3
    SQL*Net message to client  6375K 100.0    .0    .0    .0    .0    .0    .0
    Wait Event Histogram  DB/Inst: XXXX/xxxx  Snaps: 5635-5636
    -> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
    -> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
    -> % of Waits - value: .0 indicates value was <.05%, null is truly 0
    -> Ordered by Event (idle events last)
                               Total ----------------- % of Waits ------------------
    Event                      Waits  <1ms  <2ms  <4ms  <8ms <16ms <32ms  <=1s   >1s
    SQL*Net more data from cli 7976   99.7    .1    .1    .0                      .1
    Space Manager: slave idle  4086     .1    .2    .0    .0    .3         3.2  96.1
    Streams AQ: qmn coordinato  258   49.2                .8                    50.0
    Streams AQ: qmn slave idle  129                                            100.0
    Streams AQ: waiting for ti    2   50.0                                      50.0
    class slave wait             28   92.9   3.6   3.6
    jobq slave wait            7270     .0                               100.0
    pmon timer                 1199                                            100.0
    rdbms ipc message           314K  10.3   7.3  39.7  15.4  10.6   5.3   8.2   3.3
    smon timer                   43                                            100.0

Maybe you are looking for