High enqueue wait

The oracle version is 9.2.0.7. The total memory of machine is 16g. I change sga max size from 2g to 7g. I changed the db cache size from 1.5g to 6g because i was getting law buffer hit.I increased processes parameter to 1000 from 700 and sessions parameter to 1000 from 700.Now I am seeing the high enqueue wait in the database like 90% in statspack.
What could be the reason for this

There are no rows return from those queries
But i got this
select * from v$enqueue_stat where cum_wait_time>0
order by cum_wait_time desc;
INST_ID EQ TOTAL_REQ# TOTAL_WAIT# SUCC_REQ#
FAILED_REQ# CUM_WAIT_TIME
1 TX 310179 8293 310175
4 76761677
TC 130 25 130 0
526
US 21407 7 21407 0
97
SQ 14093 217 14093 0
78
HW 9835 13 9835 0
40
CF 8901 2 8901 0
29Well, the result says that your enqueue waits were "TX" type.
Followings are genercal cases of TX lock contention:
- Row lock contention - Row update confliction, PK confliction, Bitmap Index confliction, ...
- ITL entry contention
- Index split contention
- Misc
Unfortunately, TX lock contention generally has no relationship with buffer cache size. I don't think your increased enqueue wait was caused by bigger buffer cache. TX lock contention means that multiple transactions are modifying/accessing same data.
You need to capture V$SESSION_WAIT, V$SESSION and V$SQL(as i said) just when the enqueue wait happens. More info will tell you more reasonable explanation.

Similar Messages

  • High Enqueue Waits in Statspack Report

    Hi Everybody,
    Oracle Version:Oracle9i Enterprise Edition Release 9.2.0.7.0 - 64bit Production
    OS:Solaris 64 bit
    Statspack Report is showing High Enqueue Waits
    Here is a Snapshot.....
    STATSPACK report for
    DB Name DB Id Instance Inst Num Release Cluster Host
    XXXXX 434917312 XXXXX 1 9.2.0.7.0 NO INgenius1
    Snap Id Snap Time Sessions Curs/Sess Comment
    Begin Snap: 1064 24-Sep-08 06:00:01 1,333 19.6
    End Snap: 1065 24-Sep-08 07:00:01 1,344 19.7
    Elapsed: 60.00 (mins)
    Cache Sizes (end)
    ~~~~~~~~~~~~~~~~~
    Buffer Cache: 1,152M Std Block Size: 8K
    Shared Pool Size: 752M Log Buffer: 1,536K
    Load Profile
    ~~~~~~~~~~~~ Per Second Per Transaction
    Redo size: 27,771.53 1,196.23
    Logical reads: 777.20 33.48
    Block changes: 180.58 7.78
    Physical reads: 33.25 1.43
    Physical writes: 7.51 0.32
    User calls: 76.89 3.31
    Parses: 23.29 1.00
    Hard parses: 0.27 0.01
    Sorts: 0.42 0.02
    Logons: 0.05 0.00
    Executes: 88.69 3.82
    Transactions: 23.22
    % Blocks changed per Read: 23.23 Recursive Call %: 61.50
    Rollback per transaction %: 0.00 Rows per Sort: 738.76
    Instance Efficiency Percentages (Target 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Buffer Nowait %: 100.00 Redo NoWait %: 100.00
    Buffer Hit %: 96.90 In-memory Sort %: 99.67
    Library Hit %: 99.76 Soft Parse %: 98.84
    Execute to Parse %: 73.74 Latch Hit %: 99.98
    Parse CPU to Parse Elapsd %: 94.38 % Non-Parse CPU: 94.45
    Shared Pool Statistics Begin End
    Memory Usage %: 86.56 87.54
    % SQL with executions>1: 50.57 63.10
    % Memory for SQL w/exec>1: 10.02 14.63
    Top 5 Timed Events
    ~~~~~~~~~~~~~~~~~~ % Total
    Event Waits Time (s) Ela Time
    enqueue 19,553 57,405 98.39
    db file sequential read 44,161 384 .66
    CPU time 333 .57
    log file parallel write 166,602 82 .14
    log file sync 67,683 71 .12
    Wait Events for DB: XXXXX Instance: XXXXX Snaps: 1064 - 1065
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc (idle events last)
    Avg
    Total Wait wait Waits
    Event Waits Timeouts Time (s) (ms) /txn
    enqueue 19,553 19,540 57,405 2936 0.2
    db file sequential read 44,161 0 384 9 0.5
    log file parallel write 166,602 0 82 0 2.0
    log file sync 67,683 0 71 1 0.8
    db file scattered read 6,676 0 54 8 0.1
    db file parallel write 1,135 0 7 6 0.0
    direct path read 1,117 0 3 3 0.0
    SQL*Net more data to client 3,932 0 2 0 0.0
    control file parallel write 1,200 0 2 1 0.0
    control file sequential read 1,389 0 1 1 0.0
    PX Deq: Execute Reply 112 0 0 4 0.0
    direct path write 752 0 0 1 0.0
    db file parallel read 9 0 0 42 0.0
    Background Wait Events for DB: XXXXX Instance: XXXXX Snaps: 1064 - 10
    -> ordered by wait time desc, waits desc (idle events last)
    Avg Wt Wait
    Eq Requests Succ Gets Failed Gets Waits Time (ms) Time (s)
    TC 25 24 0 4 32.00 0
    TX 84,615 84,605 0 3 8.33 0
    HW 118 118 0 2 2.00 0
    PS 29 25 4 2 1.00 0
    Here frm Statspack Report we can see that enqueue type- TX is taking up most of the resources.......
    I want to find out which sql statements are causing this high enqueue waits......
    Any Help Appreciated....
    Regards,
    Prosenjit Mukherjee

    Hi All,
    Here is the Statspack Report..........
    STATSPACK report for
    DB Name         DB Id    Instance     Inst Num Release     Cluster Host
    XXXXX          434917312 XXXXX               1 9.2.0.7.0   NO      XXXXXxxx1
                  Snap Id     Snap Time      Sessions Curs/Sess Comment
    Begin Snap:      1064 24-Sep-08 06:00:01    1,333      19.6
      End Snap:      1065 24-Sep-08 07:00:01    1,344      19.7
       Elapsed:               60.00 (mins)
    Cache Sizes (end)
    ~~~~~~~~~~~~~~~~~
                   Buffer Cache:     1,152M      Std Block Size:          8K
               Shared Pool Size:       752M          Log Buffer:      1,536K
    Load Profile
    ~~~~~~~~~~~~                            Per Second       Per Transaction
                      Redo size:             27,771.53              1,196.23
                  Logical reads:                777.20                 33.48
                  Block changes:                180.58                  7.78
                 Physical reads:                 33.25                  1.43
                Physical writes:                  7.51                  0.32
                     User calls:                 76.89                  3.31
                         Parses:                 23.29                  1.00
                    Hard parses:                  0.27                  0.01
                          Sorts:                  0.42                  0.02
                         Logons:                  0.05                  0.00
                       Executes:                 88.69                  3.82
                   Transactions:                 23.22
      % Blocks changed per Read:   23.23    Recursive Call %:     61.50
    Rollback per transaction %:    0.00       Rows per Sort:    738.76
    Instance Efficiency Percentages (Target 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                Buffer Nowait %:  100.00       Redo NoWait %:    100.00
                Buffer  Hit   %:   96.90    In-memory Sort %:     99.67
                Library Hit   %:   99.76        Soft Parse %:     98.84
             Execute to Parse %:   73.74         Latch Hit %:     99.98
    Parse CPU to Parse Elapsd %:   94.38     % Non-Parse CPU:     94.45
    Shared Pool Statistics        Begin   End
                 Memory Usage %:   86.56   87.54
        % SQL with executions>1:   50.57   63.10
      % Memory for SQL w/exec>1:   10.02   14.63
    Top 5 Timed Events
    ~~~~~~~~~~~~~~~~~~                                                     % Total
    Event                                               Waits    Time (s) Ela Time
    enqueue                                            19,553      57,405    98.39
    db file sequential read                            44,161         384      .66
    CPU time                                                          333      .57
    log file parallel write                           166,602          82      .14
    log file sync                                      67,683          71      .12
    Wait Events for DB: XXXXX  Instance: XXXXX  Snaps:      1064 -     1065
    -> s  - second
    -> cs - centisecond -     100th of a second
    -> ms - millisecond -    1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc (idle events last)
                                                                       Avg
                                                         Total Wait   wait    Waits
    Event                               Waits   Timeouts   Time (s)   (ms)     /txn
    enqueue                            19,553     19,540     57,405   2936      0.2
    db file sequential read            44,161          0        384      9      0.5
    log file parallel write           166,602          0         82      0      2.0
    log file sync                      67,683          0         71      1      0.8
    db file scattered read              6,676          0         54      8      0.1
    db file parallel write              1,135          0          7      6      0.0
    direct path read                    1,117          0          3      3      0.0
    SQL*Net more data to client         3,932          0          2      0      0.0
    control file parallel write         1,200          0          2      1      0.0
    control file sequential read        1,389          0          1      1      0.0
    PX Deq: Execute Reply                 112          0          0      4      0.0
    direct path write                     752          0          0      1      0.0
    db file parallel read                   9          0          0     42      0.0
    process startup                         6          0          0     44      0.0
    SQL*Net break/reset to clien          296          0          0      1      0.0
    PX Deq: Signal ACK                      3          1          0     33      0.0
    latch free                             17          4          0      3      0.0
    LGWR wait for redo copy               286          0          0      0      0.0
    PX Deq: Join ACK                        3          0          0      2      0.0
    PX Deq: Parse Reply                     4          0          0      1      0.0
    buffer busy waits                      12          0          0      0      0.0
    PX Deq Credit: need buffer              4          0          0      0      0.0
    PX Deq: Table Q Sample                  1          0          0      0      0.0
    SQL*Net message from client       204,628          0  4,620,111  22578      2.4
    virtual circuit status                120        120      3,516  29297      0.0
    PX Idle Wait                          753        749      1,470   1953      0.0
    SQL*Net more data from clien       20,540          0          3      0      0.2
    PX Deq: Execution Msg                 128          0          2     13      0.0
    SQL*Net message to client         204,628          0          0      0      2.4
    Background Wait Events for DB: XXXXX  Instance: XXXXX  Snaps:      1064 -     10
    -> ordered by wait time desc, waits desc (idle events last)
                                                                       Avg
                                                         Total Wait   wait    Waits
    Event                               Waits   Timeouts   Time (s)   (ms)     /txn
    log file parallel write           166,634          0         82      0      2.0
    db file parallel write              1,134          0          8      7      0.0
    control file parallel write         1,200          0          2      1      0.0
    control file sequential read        1,327          0          1      1      0.0
    db file scattered read                 44          0          0     10      0.0
    db file sequential read                25          0          0      3      0.0
    direct path read                       23          0          0      3      0.0
    rdbms ipc reply                        67          0          0      0      0.0
    LGWR wait for redo copy               286          0          0      0      0.0
    direct path write                      23          0          0      0      0.0
    buffer busy waits                       2          0          0      0      0.0
    rdbms ipc message                  86,933      3,437     20,199    232      1.0
    pmon timer                          1,194      1,194      3,504   2935      0.0
    smon timer                             17          8      3,239 ######      0.0
    Enqueue activity for DB: XXXXX  Instance: XXXXX  Snaps:      1064 -     1065
    -> Enqueue stats gathered prior to 9i should not be compared with 9i data
    -> ordered by Wait Time desc, Waits desc
                                                            Avg Wt         Wait
    Eq     Requests    Succ Gets Failed Gets       Waits   Time (ms)     Time (s)
    TC           25           24           0           4         32.00            0
    TX       84,615       84,605           0           3          8.33            0
    HW          118          118           0           2          2.00            0
    PS           29           25           4           2          1.00            0
    End of Report (this is not the entire report,only posted part of it,getting error page when trying to post the entire report)
    Regards,
    Prosenjit Mukherjee

  • 10.2.0.3 high concurrency wait event

    I have a new 64bit windows 10g 10.2.0.3 VM server doing nothing but spinning its wheels. I intalled Oracle on it Friday and when I checked it tonight I see it is getting high concurrency wait events. Looks like every 10 minutes concurrency goes up to about 35.0 (?seconds) and then goes back to 0 - up and down every few minutes. Just enough to hit the warning threashold.
    I need this machine for a peoplesoft install on Monday morning.
    Anyone have any ideas what this could be? I kinda know what concurrency is but not sure if it could be from hardware or software.
    Didn't see any patches/bugs but I will continue to look.
    Help, Kathie

    What are the wait events as described in v$system_wait and v$session_event?

  • High roll wait time, gui time and processing time

    Hi all,
    We are having a performance issue with the system at the moment. After running ST03N and STAD, we have determined that we are having abnormally high roll-wait, GUI and processing times.
    Here are some results from STAD:
    CPU time: 2956ms
    Processing time: 4417ms
    Roll wait time: 1319ms
    GUI Time: 2859ms
    Can someone direct me towards finding the problem?
    If you need more information please let me know. I would post a screenshot if I was able to.
    Thanks,
    HL

    Hi,
    Regarding Performance Of SAP system it will not be possible to Come to conclusion with the one Single data.
    GUI time is basically the time spent in front end i.e Your Network load or such time
    Coming to CPU time Can you see ST06 what is the CPU Utilization of your system?
    Roll Wait time is the time Spent in buffer if you can share us the OS and Memory configuration.
    We can Try to tune the Roll memory ( extended memory tuning) so that we can try to reduce the Roll time.
    If possible try to find the Top Dialog Response Time Transaction Is it Standard or for Z Programs.
    https://cw.sdn.sap.com/cw/docs/DOC-14387
    Go through this link you might get clear idea of performance.
    Thanks & Regards,
    Balaji.S

  • High Database Waits of type "other"

    I justed added cpu patch 19 to my 11.1.0.7 windows 2008 database. Now I have a large amount of database waiting of type "other". See below.
    Investigate the cause for high "unspecified wait event" waits. Refer to Oracle's "Database Reference" for the description of this wait event.
    Action Investigate the cause for high "unspecified wait event" waits in Service "SYS$USERS".
    Action Investigate the cause for high "unspecified wait event" waits in Module "DBMS_SCHEDULER"
    Could stale statistics cause this and if so what is the correct way to gather system statistics. Do I unlock sys schema , gather data dictionary stats, and then lock sys schema?
    Other thoughts what might cause this?
    Thanks,
    Kathie

    Hi,
    You should run the following.
    DBMS_STATS.GATHER_SYSTEM_STATS (
    gathering_mode VARCHAR2 DEFAULT 'NOWORKLOAD',
    interval INTEGER DEFAULT NULL,
    stattab VARCHAR2 DEFAULT NULL,
    statid VARCHAR2 DEFAULT NULL,
    statown VARCHAR2 DEFAULT NULL);
    dbms_stats.gather_system_stats(gathering_mode=>'start');
    dbms_stats.gather_system_stats(gathering_mode=>'stop');
    System statistics will start gathering afetr you run the command with start option. System stats gathering will stop and may show them locked after you run stop.
    NOWORKLOAD: Will capture characteristics of the I/O system. Gathering may take a few minutes and depends on the size of the database. During this period Oracle will estimate the average read seek time and transfer speed for the I/O system. This mode is suitable for the all workloads. Oracle recommends to run GATHER_SYSTEM_STATS ('noworkload') after creation of the database and tablespaces. To fine tune system statistics for the workload use 'START' and 'STOP' or 'INTERVAL' options. If you gather both 'NOWORKLOAD' and workload specific (statistics collected using 'INTERVAL' or 'START' and 'STOP' ), the workload statistics will be used by optimizer. Collected components: cpuspeednw, ioseektim, iotfrspeed.
    INTERVAL: Captures system activity during a specified interval. This works in combination with the interval parameter. You should provide an interval value in minutes, after which system statistics are created or updated in the dictionary or stattab. You can use GATHER_SYSTEM_STATS (gathering_mode=>'STOP') to stop gathering earlier than scheduled. Collected components: maxthr, slavethr, cpuspeed, sreadtim, mreadtim, mbrc.
    START | STOP: Captures system activity during specified start and stop times and refreshes the dictionary or stattab with statistics for the elapsed period. Interval value is ignored. Collected components: maxthr, slavethr, cpuspeed, sreadtim, mreadtim, mbrc.
    interval
    Time, in minutes, to gather statistics. This parameter applies only when gathering_mode='INTERVAL'.
    Regards

  • Enqueue waits

    HI all,
    9.0.1
    The users are encountering lots of enqueue waits while running an application.The application updates 2 tables in a single pl/sql block.
    select * from v$lock;
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    6A4BCD30 6A4BCD40          2 MR        202          0          4          0
          7819          0
    6A4BCCE4 6A4BCCF4          2 MR        201          0          4          0
          7819          0
    6A4BCC98 6A4BCCA8          2 MR         12          0          4          0
          7819          0
    6A4BCC4C 6A4BCC5C          2 MR         11          0          4          0
          7819          0
    6A4BCC00 6A4BCC10          2 MR         10          0          4          0
          7819          0
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    6A4BCBB4 6A4BCBC4          2 MR          9          0          4          0
          7819          0
    6A4BCB68 6A4BCB78          2 MR          8          0          4          0
          7819          0
    6A4BCB1C 6A4BCB2C          2 MR          7          0          4          0
          7819          0
    6A4BCAD0 6A4BCAE0          2 MR          6          0          4          0
          7819          0
    6A4BCA84 6A4BCA94          2 MR          5          0          4          0
          7819          0
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    6A4BCA38 6A4BCA48          2 MR          4          0          4          0
          7819          0
    6A4BC9EC 6A4BC9FC          2 MR          3          0          4          0
          7819          0
    6A4BC9A0 6A4BC9B0          2 MR          2          0          4          0
          7819          0
    6A4BC954 6A4BC964          2 MR          1          0          4          0
          7819          0
    6A4BC870 6A4BC880          3 RT          1          0          6          0
          7821          0
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    6A4BC8BC 6A4BC8CC          4 KT       3317          0          4          0
          7817          0
    6A4BC7D8 6A4BC7E8          5 TS         11          1          3          0
          7037          0
    B5CA4AC8 B5CA4BB0          8 TX     524333       1430          6          0
          2079          0
    6ABBE34C 6ABBE360          8 TM      32282          0          3          0
          2079          0
    6ABBE244 6ABBE258          8 TM      32296          0          3          0
          2079          0
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    6A4BC908 6A4BC918          8 TX     196637       1411          0          6
          2079          0
    B5CA4AC8 B5CA4BB0         16 TX     196637       1411          6          0
          2227          1
    6ABBE2C8 6ABBE2DC         16 TM      32282          0          3          0
          2227          0
    6ABBE13C 6ABBE150         16 TM      32296          0          3          0
          2227          0
    6A4BC824 6A4BC834         16 TX     131104       1439          0          6
          2227          0
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    B5CA4AC8 B5CA4BB0         18 TX     327709       1446          6          0
          1864          0
    6ABBE664 6ABBE678         18 TM      32282          0          3          0
          1864          0
    6ABBE0B8 6ABBE0CC         18 TM      32296          0          3          0
          1864          0
    6A4BCE14 6A4BCE24         18 TX     196637       1411          0          6
          1864          0
    B5CA4AC8 B5CA4BB0         24 TX     131104       1439          6          0
          2920          1
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    B5CA4AC8 B5CA4BB0         27 TX     262167       1433          6          0
          2002          0
    6ABBE3D0 6ABBE3E4         27 TM      32282          0          3          0
          2002          0
    6ABBE1C0 6ABBE1D4         27 TM      32296          0          3          0
          2002          0
    6A4BCD7C 6A4BCD8C         27 TX     196637       1411          0          6
          2002          0
    B5CA4AC8 B5CA4BB0         29 TX      65573       1409          6          0
          1924          0
    .................................................................Press Enter Key
    ADDR     KADDR           SID TY        ID1        ID2      LMODE    REQUEST
         CTIME      BLOCK
    6ABBE5E0 6ABBE5F4         29 TM      32282          0          3          0
          1924          0
    6ABBE454 6ABBE468         29 TM      32296          0          3          0
          1924          0
    6A4BCDC8 6A4BCDD8         29 TX     196637       1411          0          6
          1924          0
    B5CA4AC8 B5CA4BB0         35 CF          0          3          6          0
             2          0
    6A4BCE60 6A4BCE70         35 TX     589858       1474          1          0
             2          0How to resolve the issue?

    SQL> SELECT DECODE(request,0,'Holder: ','Waiter: ') ||
              sid sess, id1, id2, lmode, request, type
       FROM V$LOCK
    WHERE (id1, id2, type) IN (SELECT id1, id2, type FROM V$LOCK WHERE request > 0)   ORDER BY id1, request;  2    3    4    5
    .................................................................Press Enter Key
    SESS                                                    ID1        ID2
         LMODE    REQUEST TY
    Holder: 24                                           131104       1439
             6          0 TX
    Waiter: 16                                           131104       1439
             0          6 TX
    Holder: 16                                           196637       1411
             6          0 TX
    Waiter: 8                                            196637       1411
             0          6 TX
    Waiter: 29                                           196637       1411
             0          6 TX
    .................................................................Press Enter Key
    SESS                                                    ID1        ID2
         LMODE    REQUEST TY
    Waiter: 35                                           196637       1411
             0          6 TX
    Waiter: 27                                           196637       1411
             0          6 TX
    Waiter: 17                                           196637       1411
             0          6 TX
    Waiter: 18                                           196637       1411
             0          6 TXThe users says that they are updating separate rows ie every user is updating different rows.So no concurrent cupdation is going on in any row.
    Edited by: MYH on Jan 23, 2009 11:02 PM

  • Redo log space requests and Enqueue Waits

    Hi all,
    I am seeing an increase on the Enqueue Waits and Redo Log Space Request from 58, 274 to 192, 1245 in two weeks time respectively.
    The DB is a production database and runs on an HP cluster with 4X1G ram and 550mghz cpu.
    There are four Redo Log files with 200M (2 members each)which I have increased to 400M over this past weekend.
    I have included below the memory structure details:
    Redo Log Summary
    Total System Global Area 1646094824 bytes
    Fixed Size 104936 bytes
    Variable Size 408989696 bytes
    Database Buffers 1228800000 bytes
    Redo Buffers 8200192 bytes
    My question is that, who do I stop it from growing further and passing the 1:5000 ratio ?
    At the moment the ratio is in the range of 1:186194.
    Your input is much appreciated.
    Cheers,
    Seyoum.

    Here is some information from Oracle's Peformance Tuning Guide.
    The V$SYSSTAT statistic redo log space requests indicates how many times a server process had to wait for space in the online redo log, not for space in the redo log buffer. A significant value for this statistic and the wait events should be used as an indication that checkpoints, DBWR, or archiver activity should be tuned, not LGWR. Increasing the size of log buffer does not help.

  • Trying to diagnose high "Network Wait" in my application

    Hello,
    I developed a simple standalone application that starts 4 threads. Essentially each thread is a database write operation by calling a stateless session bean (SLSB). The SLSB has a method that make use of Toplink Essentials JPA which is responsible of persisting the objects in the database. I am using an application diagnostic tool. This tool is showing high "Network Wait". For example 50% of the time is being spent on "java.net.SocketException" and finally in the following call "java.net.SocketInputStream -> socketRead0".
    1. What is the reason for SocketException?
    2. Could it because threads are trying to write at the same time?
    I would appreciate if you could share with me any thoughts.
    Thanks,
    Mustafa

    50% of the time is being spent on "java.net.SocketException" and finally in the following call "java.net.SocketInputStream -> socketRead0". That means that 50% of the time there is nothing to read. You would expect at least that, more like 90% IMO.
    1. What is the reason for SocketException?What SocketException? Did you catch one? What was the text? Where was it thrown from?
    2. Could it because threads are trying to write at the same time?Could what be because ...?

  • High roll wait time

    For my below crm transaction for a perticular user the response time is ~18000 ms out of which 15130 is roll time. Memory used is 16 MB.
    "BSPWDWCC_SLS_HOME  SAPMHTTP  /sap/bc/bsp/sap/CRM_UI_PORTAL/B"
    Roll time
    Out                 5  ms
    In                  5  ms
    Wait           15,130  ms
    Where is causing this roll wait ? What memory is not sufficient ? What to look at ?
    In st02
    roll buffer max used is 25% ,
    page buffer max used is 10%.
    EM max used 50%.
    No swaps for above.
    My user memory settings are as below:
    ztta/roll_area                              9000000
    ztta/roll_first                             1
    ztta/roll_extension                         4000000000
    thank you
    Laxmi

    Thank you Shyam,Krishna.
    I checked another similar instance again today with high wait time.  Most of the time was wait time. There were NO RFC RECORD. just http record with most of it being on the server (Remote exec. time ~ 16 534 ms).
    Now i assume calling time = gate way time + remote exec time. So as most of it was on the server I can rule out network.
    I checked st03 and rfc times were really small. Also we have plenty of wps .. ~ 90. So not sure why still the high wait time. Also stad does not give any RFC record. If this was rfc will it now show up ?
    I have given below the details. Am I missing some thing ?
    Response time           16,546 ms
    Wait for work process          5 ms
    Processing time            1,155 ms
    Load time                     28 ms
    Generating time                0 ms
    Roll (in+wait) time       15,016 ms
    Database request time        341 ms
    Enqueue time                   0 ms
    Roll time   Out                 5  ms
                 In                  3  ms
                 Wait           15,013  ms
    NO RFC SUB RECORD
    HTTP Records
    as Server
    Number      Connections                            1
                       Destinations                           1
                       Calls                                       1
    Time     Calling                            16,540   ms
                Remote execution          16,534   ms
                Idle                                 219   ms
    HTTP Server connection record
    Connection-Id          4B7CAADEF5DA00A7E10080000A68F456
    Communication step     Send/Receive
    Timestamp              20100218 155216 EST
    Host                   sappc003
    Port                   8031
    Path                   /sap/bc/bsp/sap/crm_ui_frame/BSPWDApplication.do
    Status phrase          OK
    Status code              200
    Protocol               HTTP
    Method                 GET

  • High PREEMPTIVE_SP_SERVER_DIAGNOSTICS wait time in sql server 2014 server

    I am seeing high waittimes for PREEMPTIVE_SP_SERVER_DIAGNOSTICS and the server is runninf with high cpu.
    What could have triggered this high waittime, how can this be fixed,
    WaitType Wait_S
    Resource_S Signal_S
    WaitCount Percentage
    AvgWait_S AvgRes_S
    AvgSig_S
    PREEMPTIVE_SP_SERVER_DIAGNOSTICS 1655043.72
    1655043.72 0.00
    4 39.75
    413760.9305 413760.9305
    0.0000
    PREEMPTIVE_OS_PIPEOPS 935157.21
    935157.21 0.00
    343445 22.46
    2.7229 2.7229
    0.0000
    CXPACKET 259912.58
    250608.73 9303.85
    25115286 6.24
    0.0103 0.0100
    0.0004
    PAGEIOLATCH_SH 206350.14
    205563.18 786.96
    37649420 4.96
    0.0055 0.0055
    0.0000
    SOS_SCHEDULER_YIELD 139263.91
    2430.27 136833.63
    293382681 3.35
    0.0005 0.0000
    0.0005

    Hello,
    I remember I read a user complaining about high waits on PREEMPTIVE_SP_SERVER_DIAGNOSTICS when he installed CU5 for SQL
    Server 2012 SP1, but in your scenario you have a SQL Server 2014 with CU1 (build 2342). I am treating this as a bug and that is the reason I would like you to consider applying CU5, the latest cumulative update, to see if that solves the issue.
    http://support.microsoft.com/kb/3011055/en-us
    This is the first time I read about this happening on SQL Server 2014 though.
    I would like to suggest you to create a Connect item too, to see if the SQL Server team can suggest you another way to
    treat this issue.
    https://connect.microsoft.com/sqlServer/
    Hope this helps.
    Regards,
    Alberto Morillo
    SQLCoffee.com

  • Resolving PS enqueue waits

    Hi All,
    I find the following in statspack report.
    Enqueue activity for DB: FCRLIVE Instance: fcrlive Snaps: 8318 -8319
    -> Enqueue stats gathered prior to 9i should not be compared with 9i data
    -> ordered by Wait Time desc, Waits desc
    Avg Wt Wait
    Eq Requests Succ Gets Failed Gets Waits Time (ms) Time (s)
    PS 33,012 19,048 13,964 2,876 .00 0
    Kindly denote how do we interprete this and how do we resolve this??
    Regards
    Vijay

    Vijay Salian wrote:
    Hi
    Below are the wait events.I more concerned about Failed Gets in Enqueue -PS part which shows 15,187.What does that denote??Secondly how do we resolve it??
    Event                                               Waits      Time (s) Ela Time
    SQL*Net message from dblink             8,190          12,641    88.42
    db file scattered read                         389,290         858     6.00
    db file sequential read                         125,058         515     3.60
    PX Deq Credit: send blkd                      50,441         117      .82
    PX Deq: Execute Reply                        21,396          71       .50Avg Wt Wait
    Eq Requests Succ Gets Failed Gets Waits Time (ms) Time (s)
    PS 35,991 20,804 *15,187* 3,141 .00 0
    Regards
    vijayCheck your "Instance statistics" for the number of parallel queries that have been downgraded - (search for downgraded) and show us the results.
    Looking at your Top 5 you seem to spend all your time waiting on a remote database - so if you have specific queries that are slower than expected it might something to do with the remote database rather than the behaviour of local PQ slaves. Note - a parallel query that goes distributed HAS to go through a serial step to get remote data - which may make problems with parallel slaves insignificant.
    Regards
    Jonathan Lewis

  • High CPU Waits but medium CPU Usage

    Hi All,
    I'm currently checking one SQL Server after our users complained about poor application performance.
    In the Data Collection Report, I can see that most of the waits come from the CPU. However, the average CPU load is just between 30% and 50%. What could be the cause for this?
    The queries which generate most of the CPU wait time are SELECTs to a view with around 30 columns (decimals) and 30.000 row and the INSERTS to the tables behind the view. Since the physical ready are 0, I guess the results come all from memory.
    How could I reduce the CPU waits? Thanks! 
    Greetings,
      Theodor

    >Could you explain why you think that showing CPU utilization as waits is a *good* thing? 
    In performance analysis you care about reducing the elapsed time.  Including CPU use as a wait gives you a unified view of the relative importance of CPU and the other waits.  Otherwise you can't know whether to focus on reducing, say lock waits,
    IO waits or CPU use. 
    > I don't see anything to fix!
    Yes SOS_SCHEDULER_YEILD and Signal Wait Time are both fine.  Those indicate global CPU pressure, ie not having an available CPU to do the work that's ready to be done.
    Here the problem is CPU Consumed, which is simply queries chewing up CPU.  So look at the queries, find the queries that account for the most CPU and analyze them.  Looking for instance for unnecessary table scans on tables in memory, inefficient
    joins, running moderately expensive queries too frequently.
    I really can't follow that.  A wait is when you really are losing elapsed time for external reasons.  CPU or reads are what your code really needs to do.  Very different categories and should not be added, and that's why they
    are tracked separately inside of SQL Server.
    OP is asking about 9 millisecond transactions.  In most systems I work on, that's already very quick.  I've worked on maybe one system in the last few years where the average transaction was faster than that because of the way it was architected
    (we called it "chatty"), most OLTP systems I've worked on in recent years are in the 1000++ millisecond ranges and we consider that excellent performance given what we ask of it.  If that is reported as a "wait", I could make no sense out of such a report. 
    As you point out, the DMV report is much more useful.
    Fortunately we can break out the numbers and separate the waits from the uses, but I think that SQL Server waits report is highly misleading.
    Josh

  • High "CPU + Wait for CPU" event on pl/sql execute operation

    I am executing a pl/sql in 256 parallel sessions, on 11G r2 DB (RAC 2 nodes), on a 42core IBM P7 Machine.
    PL/sql function opens a cursor on a huge table with around 20M rows and does further processing.
    Work-load is equally divided into 256 parallel sessions. 256 parallel sessions are opened by a middle-ware application and each session processes data based on an identifier (there are 256 distinct identifier values).
    PL/sql function is comprised of some SQL operations, on which i am experiencing some cluster waits. But CPU + wait for CPU event on pl/sql execute operation is close to 80% of the total execution time.
    Top user events:
    Event Event Class % Event Avg Active Sessions
    CPU + Wait for CPU CPU 80.88 98.15
    gc current block 2-way Cluster 3.33 4.05
    gc cr block busy Cluster 2.01 2.44
    gc cr block 2-way Cluster 1.97 2.39
    db file sequential read User I/O 1.81 2.20
    Top SQL command type:
    SQL Command Type Distinct SQLIDs % Activity Avg Active Sessions
    PL/SQL EXECUTE 3 60.99 74.02
    SELECT 66 12.90 15.65
    INSERT 24 9.89 12.01
    UPDATE 9 6.00 7.28
    DELETE 2 1.33 1.61Rest of the SQL queries seem to be very optimum, but waits on pl/sql execute operation are causing very low tps.
    Can anybody give me some heads-up about where to and what to look for to resolve?
    Please let me know if any extra information is required.

    AWR report :
    Header
    DB      Name      DB Id           Instance      Inst num      Startup Time           Release RAC
    FCR           1304316316      fcrypp1      1                01-12ÔÂ-12 04:12      11.2.0.2.0 YES
    Host           Name Platform                     CPUs      Cores      Sockets Memory (GB)
    z4ci2011      AIX-Based Systems (64-bit)      168      42        320.00
                   Snap Id      Snap Time                     Sessions      Cursors/Session
    Begin Snap: 40650           01-12ÔÂ-12 06:40:03      1203           5.8
    End Snap:      40669           01-12ÔÂ-12 09:50:01      1122           5.3
    Elapsed:        189.96 (mins)    
    DB Time:        22,251.65 (mins)
    Load profile
    Per Second           Per Transaction      Per Exec      Per Call
    DB Time(s):           117.1                19.5                     0.00           3.89
    DB CPU(s):                16.1                2.7                     0.00           0.53
    Redo size:                12,759,186.3      2,126,361.0    
    Logical reads:           181,875.9           30,310.2    
    Block changes:           54,515.5           9,085.2    
    Physical reads:      1,340.3           223.4    
    Physical writes:      8,788.9           1,464.7    
    User calls:           30.1                5.0    
    Parses:                26.5                4.4    
    Hard parses:           0.4                0.1    
    W/A MB processed:      8.5                1.4    
    Logons:                0.1                0.0    
    Executes:                41,176.0           6,862.1    
    Rollbacks:                1.9                0.3    
    Transactions:           6.0      
    Time model statistics
    Statistic Name                                             Time (s)          % of DB Time
    sql execute elapsed time                              1,334,935.55     99.99
    PL/SQL execution elapsed time                         1,180,376.60     88.41
    DB CPU                                                       182,904.44          13.7
    repeated bind elapsed time                              292.83               0.02
    sequence load elapsed time                              279.75               0.02
    parse time elapsed                                        87.4               0.01
    hard parse elapsed time                                   22.52               0
    failed parse elapsed time                              5.12               0
    connection management call elapsed time               4.61               0
    PL/SQL compilation elapsed time                         1.91               0
    hard parse (sharing criteria) elapsed time          0.49               0
    hard parse (bind mismatch) elapsed time               0.39               0
    inbound PL/SQL rpc elapsed time                         0.1     0
    DB time                                                       1,335,099.30     
    background elapsed time                                   33,298.67     
    background cpu time                                        11,692.76     
    Operating System Statistics
    Statistic Value End Value
    AVG_BUSY_TIME 202,428  
    AVG_IDLE_TIME 936,397  
    AVG_IOWAIT_TIME 4,124  
    AVG_SYS_TIME 84,480  
    AVG_USER_TIME 117,573  
    BUSY_TIME 34,074,303  
    IDLE_TIME 157,378,825  
    IOWAIT_TIME 755,368  
    SYS_TIME 14,256,010  
    USER_TIME 19,818,293  
    LOAD 21 10
    OS_CPU_WAIT_TIME 23,770,800  
    VM_IN_BYTES 20,496  
    VM_OUT_BYTES 2,086,940,520  
    PHYSICAL_MEMORY_BYTES 343,597,383,680  
    NUM_CPUS 168  
    NUM_CPU_CORES 42  
    NUM_LCPUS 168  
    NUM_VCPUS 42  
    GLOBAL_RECEIVE_SIZE_MAX 41,943,040  
    GLOBAL_SEND_SIZE_MAX 41,943,040  
    TCP_RECEIVE_SIZE_DEFAULT 16,384  
    TCP_RECEIVE_SIZE_MAX 9,223,372,036,854,775,807  
    TCP_RECEIVE_SIZE_MIN 4,096  
    TCP_SEND_SIZE_DEFAULT 16,384  
    TCP_SEND_SIZE_MAX 9,223,372,036,854,775,807  
    TCP_SEND_SIZE_MIN 4,096
    SQL ordered by CPU Time
    CPU Time (s)      Executions       CPU per Exec (s) %Total      Elapsed Time (s)      %CPU      %IO      SQL Id SQL Module SQL Text
    180,330.13           127                1,419.92                98.59      1,326,401.03           13.60      1.08      04kt8u64udphu    BEGIN :1 := ap_ch_eod_shell_en...
    8,025.48           9,868,469           0.00                     4.39      10,809.88                74.24      9.21      arnkbsnzhha77 ch_txn_shell_115  SELECT * FROM CH_ACCT_MAST WHE...
    6,117.64           9,873,495           0.00                     3.34      8,557.64                71.49      7.14      8qcryvj294s79 ch_eod_shell_138  UPDATE CH_ACCT_MAST_PAR SET DA...
    4,614.71           3,185,313           0.00                     2.52      11,130.77                41.46      11.88      b75wwkxw34x2k ch_eod_shell_228  INSERT INTO CH_TMP_XF_GL_TXNS ...
    4,374.29           9,866,217           0.00                     2.39      5,876.00                74.44      37.88      g22p493ra2zr5 ch_eod_shell_248  UPDATE CH_ACCT_MAST SET BAL_LA...
    3,514.57           14,026,451           0.00                     1.92      6,274.60                56.01      29.55      7bwhphfnnuqpr ch_eod_shell_59  INSERT INTO CH_ACCT_INT_BREAKU...
    3,253.36           3,185,706           0.00                     1.78      3,875.42                83.95      9.20      9dq134q9btxq8 ch_eod_shell_74  INSERT INTO CH_ST_CAP_INPUT_TX...
    3,131.64           9,875,603           0.00                     1.71      5,338.43                58.66      15.55      6xhwk1b37rh1t ch_txn_shell_143  UPDATE CH_ACCT_ATTRIBUTES SET ...
    2,954.15           9,878,718           0.00                     1.62      5,692.88                51.89      13.22      b4at7uq2hw6r7 ch_sweepin_shell  SELECT TRIM(A.COD_PKG) FROM RP...
    2,572.01           9,867,277           0.00                     1.41      4,605.88                55.84      12.58      54ud0a8tuwwbc ch_txn_shell_17  SELECT * FROM CH_ACCT_ATTRIBUT...
    1,941.29           19,730,455           0.00                     1.06      5,580.38                34.79      7.02      dx5kng8qu560t ch_txn_shell_59  UPDATE CH_ACTIONS_DUE SET COD_...
    1,846.01           9,875,239           0.00                     1.01      4,737.66                38.96      12.55      af7f92f13rmy4 ch_txn_shell_85  INSERT INTO CH_ACTIONS_DUE (CO...

  • How to Reduce Clusetering Factor on Table?

    I am seeing a very high clustering factor on an SDO geometry table in our 10g RAC DB on our Linux boxes. This slow performance is repeateable on othe r Linux as well as Solaris DBs for the same table. Inserts go in at a rate of 44 milliseconds per insert and we only have about 27000 rows in the table. After viewing a VERY slow insert of about 600 records into this same table, I saw the clustering factor in OEM. The clustering factor is nearly identical to the # rows in the table indicating that useability of the index is fairly low now. I have referenced Metalink Tech Note 223117.1 and, while it affirms what I've seen, I am still trying to determine how to reduce the Clustering Factor. The excerpt on how to do this is below:
    "The only method to affect the clustering factor is to sort and then store the rows in the table in the same order as in they appear in the index. Exporting rows and putting them back in the same order that they appeared originally will have no affect. Remember that ordering the rows to suit one index may have detrimental effects on the choice of other indexes."
    Sounds great, but how does one actually go about storing the rows in the table in the same order as they appear in the index?
    We have tried placing our commits after the last insert as well as after every insert and the results are fairly neglible. We also have a column of type SDE.ST_GEOMETRY in the table and are wondering if this might also be an issue. Thanks in advance for any help.
    Matt Sauter

    Joel is right that the clustering factor is going to have absolutely no effect on the speed of inserts. The clustering factor is merely one, purely statistical, factor the optimiser makes use of to determine how to perform a SELECT statement (i.e., do I bother to use this index or not for row retrieval). It's got nothing to do with the efficiency of inserts.
    If I were you, I'd be looking at factors such as excessive disk I/O taking place for other reasons, inadequate buffer cache and/or enqueue and locking issues instead.
    If you're committing after every insert, for example, then redo will have to be flushed (a commit is about the only foreground wait event -i.e., one that you get to experience in real time- that Oracle has, so a commit after every insert's really not a smart idea). If your redo logs are stored on, say, the worst-performing disk you could buy that's also doing duty as a fileserver's main hard disk, then LGWR will be twiddling its thumbs a lot! You say you've tested this, and that's fine... I'm just saying, it's one theoretical possibility in these sorts of situations. You still want to make sure you're not suffering any log writer-related waits, all the same.
    Similarly, if you're performing huge reads on a (perhaps completely separate) table that is causing the buffer cache to be wiped every second or so, then getting access to your table so your inserts can take place could be problematic. Check if you've got any database writer waits, for example: they are usally a good sign of general I/O bottlenecks.
    Finally, you're on a RAC... so if the blocks of the table you're writing to are in memory over on another instance, and they have to be shipped to your instance, you could have high enqueue waits whilst that shipment is taking place. Maybe your interconnect is not up to the job? Maybe it's faulty, even, with significant packet loss along the way? Even worse if someone's decided to switch off cache fusion transfer for the datafiles invoved (for then block shipment happens by writing them to disk in one instance and reading from disk in the other). RAC adds a whole new level of complexity to things, so good luck tracking that lot down!!
    Also, maybe you're using Freelists and Freelist groups rather than ASSM, so perhaps you're fighting for access to the freelist with whatever else is happening on your database at the time...
    You get the idea: this could be a result of activity taking place on the server for reasons completely unconnected with your insert. It could be a feature of Spatial (with which not many people will be familiar, so good luck if so!) It could be a result of the way your RAC is configured. It could be any number of things... but I'd be willing to bet quite a bit that it's got sod-all to do with the clustering factor!
    You'll need to monitor the insert using a tool like Insider or Toad so you can see if waits and so on happen, more or less in real time -or start using the built-in tools like Statspack or AWR to analyze your workload after it's completed- to work out what your best fix is likely to be.

  • Exremely high (98%) roll wait time

    I am running a ECC 6.0 EHP4 server in solaris 5.10 OS, Oracle 10.2.0.4.
    My users experience extremely high roll wait time when they execute transactions like PA20/PA30. It took about 20 minutes for it to load.
    I ran stad and found out the majority of the portion is wasted at roll wait time. How do we know what caused the wait time?
    e.g. user execute TCODE PA20 and wait for the hourglass for about 20 minutes before the record is shown
    Other transactions ran smoothly.
    Your help is appreciated.

    After a RFC trace, it showed this RFC statement having very high duration:
    RFC Statement
    Function module name      HTTP_GET
    Source IP Address         a.b.c.d
    Source Server             ourSECCServer
    Destination IP Address    a.b.c.d (same ip as source)
    Destination Server        ourSECCServer
    Client/Server             Client
    Conversation ID
    RFC Trace Rec. Status     4
    Sent Bytes                9.476
    Received Bytes            3.353
    Total Sent Bytes
    Total Received Bytes
    ABAP program name         SAPLSFTP
    RFC Time                  224881.681
    Any ideas?

Maybe you are looking for

  • Auto creation of Purchase order by heuristic

    Hello Experts, In my project, Heuristic is creating "Purchase orders" automatically, which should have been created by planners only. Sometimes, it creates it without even creating "Purchase requisitions" and sometimes with PurReq. Has any one faced

  • Problem with the Matrix State

    Hi Guys, I am facing a problem with the use of the matrix. I am not able to select cells from the matrix with a click  although I set the SelectMode = ms_Auto. does someone know how to solve this problem ?? Thanks Bop

  • Aperture/iPhoto issue

    some of my images in Aperture/iPhoto are appearing pixelated.  any idea on how to get the issue fixed

  • Fonts are activated despite being off in Font Book

    I opened a Word doc in MS Word 2008 and, all of a sudden, started asking me in a dialog box: "Microsoft Word want to use the font... "bla bla" ... that was downloaded from the internet. Allow Microsoft Word to use this font? And then buttons to eithe

  • Encoding query

    Hi, I want to write an app which allows someone to describe how to parse an incoming byte stream into a number of strings. The user can describe the length of each string and how it's encoded. What I'm trying to figure out is - depending on the encod