Enq: PS – contention wait event
Hi Guys,
I am running Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production on RHEL5.
I changed my db parameter PARALLEL_MAX_SERVERS from 10 to 20. Now, most of my queries are hanging due to this enq: PS – contention wait event. I looked at the req_dgree from the V$PX_SESSION and it is too high.
I reduced the PARALLEL_MAX_SERVERS back to 10 and still the queries are still hanging and the req_dgree still too high.
How can I fix this situation?
Thanks in advance....
user11979518 wrote:
Hi Guys,
I am running Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production on RHEL5.
I changed my db parameter PARALLEL_MAX_SERVERS from 10 to 20. Now, most of my queries are hanging due to this enq: PS – contention wait event. I looked at the req_dgree from the V$PX_SESSION and it is too high.
I reduced the PARALLEL_MAX_SERVERS back to 10 and still the queries are still hanging and the req_dgree still too high.
How can I fix this situation?do use PARALLEL.
PARALLEL sometimes make SQL slower;.
PARALLEL is NOT magic solution to all/every performance problem.
more is not always better; as you have now learned.
How many cores/CPU exist in this DB Server system?
Similar Messages
-
Enq: ST - contention wait event
Hi,
At one of our production database I am getting enq: ST - contention wait event at the Top 5 Timed Event section of the AWR report. I have checked with Metalink (Doc ID:974863.1), where I have been suggested to migrate to LMT. But all the tablespaces at the database are in LMT. It would be really very helpful if someone can explain the occurance of such wait event and the way this can be eliminated in future.
Environment:
Database Version: 10.2.0.3
OS: Solaris 10
Thanks for your valuable time.
Regards,
JoyI suggest reviewing the following document:
http://www.perfvision.com/papers/09_enqueues.ppt
Specifically, look at slide 61 and its notes section.
Take a look at the output of the following SQL statement:
SELECT
SUBSTR(TABLESPACE_NAME,1,20) NAME,
EXTENT_MANAGEMENT,
ALLOCATION_TYPE,
INITIAL_EXTENT,
NEXT_EXTENT
FROM
DBA_TABLESPACES
ORDER BY
EXTENT_MANAGEMENT;Is everything listed as LOCAL? For those tablespaces with a UNIFORM ALLOCATION_TYPE, what is the value of INITIAL_EXTENT?
Charles Hooper
Co-author of "Expert Oracle Practices: Oracle Database Administration from the Oak Table"
http://hoopercharles.wordpress.com/
IT Manager/Oracle DBA
K&M Machine-Fabricating, Inc. -
Tuning row lock contention wait events
Hello everyone,
Working on 10g/windows
Top 5 events
EVENT TOTAL_WAITS TIME_WAITED AVG_MS PERCENT
CPU 9462339 48
enq: TX - row lock contention 12531 3660728 2921.34 18
control file parallel write 1300731 3088079 23.74 16
log file parallel write 1510503 1264080 8.37 6
log file sync 1072553 968007 9.03 5
Distribution of row lock wait during the last 4 days in the database server
END_INTERVAL_TIME TOTAL_WAITS TIME_WAITED_MICRO AVG_WAIT_MS
2008-04-01 16:00:58 909 2721008230 2993.41
2008-04-01 15:00:27 50 149941140 2998.82
2008-03-31 12:00:42 193 575595397 2982.36
2008-03-29 23:00:13 172 513058700 2982.9
2008-03-29 22:00:37 164 483940046 2950.85
2008-03-27 22:00:35 565 1667120838 2950.66
2008-03-26 18:00:59 348 1042918982 2996.89
My analysis:
It's obvious that the row lock contention wait time is huge, and this direct me to find out SQL stmt, causing this.
all the SQL statement was SELECT ....... FOR UPDATE stmt.
I was also able to find out locked tables.
My tuning idea:
1. I'm thinking to reorganize hot tables as well as their indexes, but by instinct it seems to not give so much value to avoid the huge row lock wait time.
2. I'm also seeing if I can reduce the number of rows per block, by increasing PCTFREE and diminishing PCTUSED, so the contention will spread over many blocks instead of one heavy block.
Question
As SQL stmt related to those locked tables are select ... for update, how could I tune this kind of stmt?
Does someone have other idea to come up with this row lock contention?
Tanks for your effort and helpTaking another look at your suggested function based index, it depends on the data type of the DEV.POS_FOLIO_ID.POS_FOLIO_ID column. If the column is defined as a number, and it is a primary key, there will already be a usable index on that column.
Yesterday, I wrote this: "Once I understood why or how the sessions were trying to insert duplicate primary key values, I would try to determine why the average number of seconds for the wait event is almost 3 seconds (maybe a timeout)."
After fixing the formatting of the top 5 wait events (total duration unknown):
EVENT TOTAL_WAITS TIME_WAITED AVG_MS PERCENT
CPU 94,623.39 48
enq: TX - row lock contention 12,531 36,607.28 2921.34 18
control file parallel write 1,300,731 30,880.79 23.74 16
log file parallel write 1,510,503 12,640.80 8.37 6
log file sync 1,072,553 9,680.07 9.03 512,531 * 3 second time out = 37,593 seconds = 10.44 hours.
What if the reason for the 3 second average wait time is due to a timeout. I performed a little experiment... I changed a row in a test table and then made a pot of coffee.
In session 1:
CREATE TABLE T1 (
C1 NUMBER(10),
C2 NUMBER(10),
PRIMARY KEY (C1));
INSERT INTO T1
SELECT
ROWNUM,
ROWNUM*10
FROM
DUAL
CONNECT BY
LEVEL<=1000000;
COMMIT;I now have a test table with 1,000,000 rows. I start monitoring the changes in the wait events roughly every 60 seconds, and V$SESSION_WAIT and V$LOCK roughly 4 times per second.
Back in session 1:
UPDATE
T1
SET
C1=-C1
WHERE
C1<=100;I have now modified the first 100 rows that were inserted into the table, time to make the pot of coffee.
In session 2, I try to insert a row with a primary key value of -10:
INSERT INTO T1 VALUES (
-10,
10);Session 2 hangs.
If I take the third 60 second snap of the system wide wait events as the zero point, and the 11th snap as the end point. There were 149 waits on ENQ: TX - ROW LOCK CONTENTION, 148 time outs, 446.62 seconds of total time in the wait event, with an average wait time of 2.997450 seconds.
Rolling down to the session level wait events, SID 208 (my session 2) had 149 waits on ENQ: TX - ROW LOCK CONTENTION, for a total time of 446.61 seconds with an average wait time of 2.997383 seconds. All of the 149 waits and the wait time was in this one session that was locked up for the full duration of this time period because session 1 was making a pot of coffee.
Rolling down to V$SESSION_WAIT (sampled roughly 4 times per second): At the start of the third time interval, SID 208 has been in the ENQ: TX - ROW LOCK CONTENTION wait event for 39 seconds and is actively waiting trying to execute SQL with a hash value of 1001532423, the wait object is -1, wait file is 0, wait block is 0, wait row is 0, P1 is 1415053316, P2 is 196646, P3 is 4754.
At the end of the 11th time interval: , SID 208 has been in the ENQ: TX - ROW LOCK CONTENTION wait event for 483 seconds and is actively waiting trying to execute SQL with a hash value of 1001532423, the wait object is -1, wait file is 0, wait block is 0, wait row is 0, P1 is 1415053316, P2 is 196646, P3 is 4754.
Rolling down to V$LOCK (sampled roughly 4 times per second): I see that SID 214 (session 1) is blocking SID 208 (session 2). SID 214 has a TX lock in mode 6 with ID1 of 196646 and ID2 of 4754. SID 208 is requesting a TX lock in mode 4 with ID1 of 196646 and ID2 of 4754.
So, it seems that I need a faster coffee pot rather than an additional index on my table. It could be that the above process would have found that the application associated with SID 214 was abandoned or crashed and for some reason the lock was not released for a long period of time, a little less than 10.44 hours in your case.
Charles Hooper
IT Manager/Oracle DBA
K&M Machine-Fabricating, Inc. -
Enq: TX - row lock contention wait event
Hi,
I would like to find which DML query has not given COMMIT or ROLLBACK after the execution. Because one of the development database have more table locks and developer reported that their session was hanging. I referred AWR report also and more timed waits occurred in the enq: TX - row lock contention. I need to trace which DML query has not commit or rollback.
Please help me to solve the issue.
Database version: 11.2.0.1.0
Foreground Wait Events
Event
Waits
%Time -outs
Total Wait Time (s)
Avg wait (ms)
Waits /txn
% DB time
enq: TX - row lock contention
320
0
72,047
225147
0.20
99.53
log file sync
547
0
14
26
0.35
0.02
library cache lock
13
0
11
843
0.01
0.02
SQL*Net break/reset to client
1,080
0
2
1
0.69
0.00
SQL*Net message to client
659,006
0
1
0
421.63
0.00
direct path sync
3
0
1
299
0.00
0.00
SQL*Net more data from client
5,541
0
1
0
3.55
0.00
db file scattered read
554
0
0
1
0.35
0.00
SQL*Net more data to client
14,975
0
0
0
9.58
0.00
db file sequential read
2,817
0
0
0
1.80
0.00
ADR block file read
4
0
0
43
0.00
0.00
enq: CR - block range reuse ckpt
2
0
0
71
0.00
0.00
asynch descriptor resize
38,073
100
0
0
24.36
0.00
latch: shared pool
61
0
0
1
0.04
0.00
control file sequential read
6,900
0
0
0
4.41
0.00
Disk file operations I/O
550
0
0
0
0.35
0.00
cursor: pin S
1
0
0
8
0.00
0.00
direct path write temp
34
0
0
0
0.02
0.00
library cache: mutex X
5
0
0
1
0.00
0.00
latch: In memory undo latch
2
0
0
1
0.00
0.00
buffer busy waits
14
0
0
0
0.01
0.00
SQL*Net message from client
658,990
0
294,847
447
421.62
jobq slave wait
669
99
333
497
0.43
PL/SQL lock timer
1
100
1
998
0.00Oracle does not and cannot tell you from historical views (e.g. AWR) which DMLs have not COMMITed or ROLLBACKed. A Transaction ends with a COMMIT or ROLLBACK. The transaction could have a million (or more) DML statements with a million (or more) SELECT statements between the first DML and the COMMIT / ROLLBACK.
Even identifying such DMLs in real time is close to impossible. Because the session holding the lock may have issued a dozen or a million subsequent SQL statements while other sessions are waiting for the lock. You can only identify the session that is the lock holder (the BLOCKING_SESSION in V$SESSION).
If you have tracing enabled for all sessions, then you could review the trace file for the BLOCKING_SESSION to identify the DML(s) the session has executed.
Hemant K Chitale -
Wait event enq: JD - contention
Hi All,
In the top-5 wait events of the AWR-report i encounter a enq: JD contention wait event (wait class Other).
I only figured out that JD means Job Queue Date.
Can anybody tell me more about this wait event?
Thanks in advance,
Jeroen van SchaijkHi All,
I have additional information:
This wait event synchronizes dates between job queue coordinator and slave processes.
It appears that this wait event takes more than 10% of the total wait time!
Does anybody know how to deal with this wait event?
Thanks in advance,
Jeroen van Schaijk -
Enq: MN - contention with log miner
One of our customers is hitting enq: MN - contention wait event.
UKJA@ukja102> exec print_table('select * from v$lock_type where type = ''MN'');
TYPE : MN
NAME : LogMiner
ID1_TAG : session ID
ID2_TAG : 0
IS_USER : NO
DESCRIPTION : Synchronizes updates to the LogMiner dictionary
and prevents multiple instances from preparing the same LogMiner session
-----------------The situation is as following
- Batch job is generating massive redo
- Frequent log file switching occurs during this job
- Multiple sessions are mining archive logs due to some business requirement
- From time to time, one session holds MN lock in excluisve mode and other sessions wait for the ML lock to be released.
Holding session and waiting sessions are executing same SQL statement like following.
SELECT SCN, SQL_REDO, SEG_OWNER, SEG_NAME, OPERATION_CODE, CSF,DATA_OBJ#,
XIDUSN || '_' || XIDSLT || '_' || XIDSQN) AS XID, ROW_ID, ROLLBACK, TIMESTAMP FROM V$LOGMNR_CONTENTS
WHERE (OPERATION_CODE IN (7, 36)
OR ( ( ROLLBACK = 0 OR (ROLLBACK = 1 AND CSF = 0) ) AND ( OPERATION_CODE IN (1, 2, 3)
AND ((SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE1')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE2')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE3')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE4')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE5')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE6')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE7')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE8')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE9')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE10')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE'11)
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE12')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE13')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE14')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE15')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE16')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE17')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE18')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE19')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE20')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE21')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE22')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE23')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE24')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE25')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE26')
OR (SEG_OWNER = 'TEST1' AND TABLE_NAME = 'TABLE27')))))But my experiments show that normal log miner operations do not need MN lock thus no MN lock contention is reproduceable. Same under the situation of very frequent log file switching.
Does anyone have experience and/or information on this lock?
================================
Dion Cho - Oracle Performance Storyteller
http://dioncho.wordpress.com (english)
http://ukja.tistory.com (korean)
================================My first test case was flawed, so posted wrong info. :(
Further investigation shows that at certain steps of typical log miner operations need MN lock in exclusive mode. For example, dbms_logmnr.start_logmnr procedure needs exclusive MN lock.
Excerpt from 10704 event trace file.
*** 2009-05-19 13:57:53.812
ksqgtl *** MN-00000000-00000000 mode=6 flags=0x21 timeout=600 ***
ksqgtl: no transaction
ksqgtl: use existing ksusetxn DID
ksqgtl:
ksqlkdid: 0001-0016-00000014
*** 2009-05-19 13:57:53.828
*** ksudidTrace: ksqgtl
ksusesdi: 0000-0000-00000000
ksusetxn: 0001-0016-00000014
ksqgtl: RETURNS 0
*** 2009-05-19 13:57:53.828
ksqrcl: MN,0,0
ksqrcl: returns 0Starting log mining operation would require modification on log miner dictionary.
This means that multiple sessions can't start log mining concurrently, but once they've started successfully other types of jobs can be done concurrently.
Any operation that should access the log miner dictionary would require MN lock. I would contact the customer who reported this problem and let them have more investigation.
================================
Dion Cho - Oracle Performance Storyteller
http://dioncho.wordpress.com (english)
http://ukja.tistory.com (korean)
================================ -
I have problem with enq: TM – contention wait events.
From ASH report in Top Event P1/P2/P3 I find problematic object – table T1. It has two columns C1 and C2. There are PK and FK constraints on these two columns.Table T1 has one UNIQUE INDEX - index_name(C1, C2).
With this:
SELECT * FROM (
SELECT c.table_name, cc.column_name, cc.position column_position
FROM user_constraints c, user_cons_columns cc
WHERE c.constraint_name = cc.constraint_name
AND c.constraint_type = 'R'
MINUS
SELECT i.table_name, ic.column_name, ic.column_position
FROM user_indexes i, user_ind_columns ic
WHERE i.index_name = ic.index_name
+)+
ORDER BY table_name, column_position;
I find unindexed Foreign Key constraints, and show me that no index on C2 column.
Can I create new index on this C2 column: CREATE INDEX index_name_c2 ON T1(C2) because I have index on C1 and C2 columns ?
In ASH report in Top DB Objects I see another indexes and table with enq: TM – contention event. Whey Oracle show me this objects?Based on what you've written I see nothing that should stop you from creating an index but I would recommend you do so in dev or test before production.
As to why Oracle is showing it to you? Likely because it thinks there is a potential issue there.
Remember always that the best designed database in the world is going to have a top 5 or top 10 list too. The only way to not have events reported is with shutdown abort. -
Enq: TX - row lock contention in TOP 5 wait event
DB version:11.1.0.7.0
I am having enq: TX - row lock contention in top 5 wait event.
AWR analyze period - 9-10(pm). During this time only one sql loader is running to insert the data. No other job are running. So there is no chance of other session blocking this session. is there any chance of row lock contention happen by same session.
SQL> SELECT INDEX_NAME,INDEX_TYPE,UNIQUENESS FROM DBA_INDEXES WHERE TABLE_NAME='DATA_DATA';
INDEX_NAME INDEX_TYPE UNIQUENES
CIDX BITMAP NONUNIQUE
VIDX BITMAP NONUNIQUE
Thanks.SQL> SELECT INDEX_NAME,INDEX_TYPE,UNIQUENESS FROM DBA_INDEXES WHERE TABLE_NAME='DATA_DATA';
INDEX_NAME INDEX_TYPE UNIQUENES
CIDX BITMAP NONUNIQUE
VIDX BITMAP NONUNIQUEYou have bitmap indexes here on a table being inserted into. Bitmap Indexes are another source of lock(and deadlock) in OLTP application. You said that the SQLloader was the unique active program but may be you are also triggering another procedure after the load. Procedure in which you might be using also automomous transactions and so on...
Check first if your table is subject to DML operation in a a multi-user concurrent accesss and in which case you have to get rid of those bitmap indexes
http://hourim.wordpress.com/2011/03/14/deadlock-%e2%80%93-part-1-bitmap-index/
Best regards
Mohamed Houri
www.hourim.wordpress.com -
What´s event "enq: FU - Contention"
what is the meaning of the following waiting event "enq: FU - Contention"
http://www.lmgtfy.com/?q=oracle+enq+FU+Contention
-
hi,
Please could someone point me to some doc explaining enq events like:
enq: AT - contention and how to interpret the name|mode parameter for such events (select event#,name,parameter1 p1, parameter2 p2, parameter3 p3 from v$event_name where name='enq: AT - contention')
Thanks for ur time,
RUbenIn addition,
UKJA@ukja102> @lock_type at
UKJA@ukja102> set serveroutput on
UKJA@ukja102>
UKJA@ukja102> exec print_table('select * from v$lock_type where type = upper(''&1'')');
TYPE : AT
NAME : Alter Tablespace
ID1_TAG : 0
ID2_TAG : 0
IS_USER : NO
DESCRIPTION : Serializes 'alter tablespace' operations
-----------------Dion Cho -
Hi: I'm analyzing this STATSPACK report: it is "volume test" on our UAT server, so most input is from 'bind variables'. Our shared pool is well utilized in oracle. Oracle redo logs is not appropriately configured on this server, as in 'Top 5 wait events' there are 2 for redos.
I need to know what else information can be dig-out from 'foreground wait events' & 'background wait events', and what can assist us to better understanding, in combination of 'Top 5 wait event's, that how the server/test went? it could be overwelming No. of wait events, so appreciate any helpful diagnostic or analysis. Database is oracle 11.2.0.4 upgraded from 11.2.0.3, on IBM AIX power system 64bit, level 6.x
STATSPACK report for
Database DB Id Instance Inst Num Startup Time Release RAC
~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
700000XXX XXX 1 22-Apr-15 12:12 11.2.0.4.0 NO
Host Name Platform CPUs Cores Sockets Memory (G)
~~~~ ---------------- ---------------------- ----- ----- ------- ------------
dXXXX_XXX AIX-Based Systems (64- 2 1 0 16.0
Snapshot Snap Id Snap Time Sessions Curs/Sess Comment
~~~~~~~~ ---------- ------------------ -------- --------- ------------------
Begin Snap: 5635 22-Apr-15 13:00:02 114 4.6
End Snap: 5636 22-Apr-15 14:00:01 128 8.8
Elapsed: 59.98 (mins) Av Act Sess: 0.6
DB time: 35.98 (mins) DB CPU: 19.43 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 2,064M Std Block Size: 8K
Shared Pool: 3,072M Log Buffer: 13,632K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ ------------------ ----------------- ----------- -----------
DB time(s): 0.6 0.0 0.00 0.00
DB CPU(s): 0.3 0.0 0.00 0.00
Redo size: 458,720.6 8,755.7
Logical reads: 12,874.2 245.7
Block changes: 1,356.4 25.9
Physical reads: 6.6 0.1
Physical writes: 61.8 1.2
User calls: 2,033.7 38.8
Parses: 286.5 5.5
Hard parses: 0.5 0.0
W/A MB processed: 1.7 0.0
Logons: 1.2 0.0
Executes: 801.1 15.3
Rollbacks: 6.1 0.1
Transactions: 52.4
Instance Efficiency Indicators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.98 Optimal W/A Exec %: 100.00
Library Hit %: 99.77 Soft Parse %: 99.82
Execute to Parse %: 64.24 Latch Hit %: 99.98
Parse CPU to Parse Elapsd %: 53.15 % Non-Parse CPU: 98.03
Shared Pool Statistics Begin End
Memory Usage %: 10.50 12.79
% SQL with executions>1: 69.98 78.37
% Memory for SQL w/exec>1: 70.22 81.96
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
CPU time 847 50.2
enq: TX - row lock contention 4,480 434 97 25.8
log file sync 284,169 185 1 11.0
log file parallel write 299,537 164 1 9.7
log file sequential read 698 16 24 1.0
Host CPU (CPUs: 2 Cores: 1 Sockets: 0)
~~~~~~~~ Load Average
Begin End User System Idle WIO WCPU
1.16 1.84 19.28 14.51 66.21 1.20 82.01
Instance CPU
~~~~~~~~~~~~ % Time (seconds)
Host: Total time (s): 7,193.8
Host: Busy CPU time (s): 2,430.7
% of time Host is Busy: 33.8
Instance: Total CPU time (s): 1,203.1
% of Busy CPU used for Instance: 49.5
Instance: Total Database time (s): 2,426.4
%DB time waiting for CPU (Resource Mgr): 0.0
Memory Statistics Begin End
~~~~~~~~~~~~~~~~~ ------------ ------------
Host Mem (MB): 16,384.0 16,384.0
SGA use (MB): 7,136.0 7,136.0
PGA use (MB): 282.5 361.4
% Host Mem used for SGA+PGA: 45.3 45.8
Foreground Wait Events DB/Inst: XXXXXs Snaps: 5635-5636
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
enq: TX - row lock contentio 4,480 0 434 97 0.0 25.8
log file sync 284,167 0 185 1 1.5 11.0
Disk file operations I/O 8,741 0 4 0 0.0 .2
direct path write 13,247 0 3 0 0.1 .2
db file sequential read 6,058 0 1 0 0.0 .1
buffer busy waits 1,800 0 1 1 0.0 .1
SQL*Net more data to client 29,161 0 1 0 0.2 .1
direct path read 7,696 0 1 0 0.0 .0
db file scattered read 316 0 1 2 0.0 .0
latch: shared pool 144 0 0 2 0.0 .0
CSS initialization 30 0 0 3 0.0 .0
cursor: pin S 10 0 0 9 0.0 .0
row cache lock 41 0 0 2 0.0 .0
latch: row cache objects 19 0 0 3 0.0 .0
log file switch (private str 8 0 0 7 0.0 .0
library cache: mutex X 28 0 0 2 0.0 .0
latch: cache buffers chains 54 0 0 1 0.0 .0
latch free 290 0 0 0 0.0 .0
control file sequential read 1,568 0 0 0 0.0 .0
log file switch (checkpoint 4 0 0 6 0.0 .0
direct path sync 8 0 0 3 0.0 .0
latch: redo allocation 60 0 0 0 0.0 .0
SQL*Net break/reset to clien 34 0 0 1 0.0 .0
latch: enqueue hash chains 45 0 0 0 0.0 .0
latch: cache buffers lru cha 7 0 0 2 0.0 .0
latch: session allocation 5 0 0 1 0.0 .0
latch: object queue header o 6 0 0 1 0.0 .0
ASM file metadata operation 30 0 0 0 0.0 .0
latch: In memory undo latch 15 0 0 0 0.0 .0
latch: undo global data 8 0 0 0 0.0 .0
SQL*Net message from client 6,362,536 0 278,225 44 33.7
jobq slave wait 7,270 100 3,635 500 0.0
SQL*Net more data from clien 7,976 0 15 2 0.0
SQL*Net message to client 6,362,544 0 8 0 33.7
Background Wait Events DB/Inst: XXXXXs Snaps: 5635-5636
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
log file parallel write 299,537 0 164 1 1.6 9.7
log file sequential read 698 0 16 24 0.0 1.0
db file parallel write 9,556 0 13 1 0.1 .8
os thread startup 146 0 10 70 0.0 .6
control file parallel write 2,037 0 2 1 0.0 .1
Log archive I/O 35 0 1 30 0.0 .1
LGWR wait for redo copy 2,447 0 0 0 0.0 .0
db file async I/O submit 9,556 0 0 0 0.1 .0
db file sequential read 145 0 0 2 0.0 .0
Disk file operations I/O 349 0 0 0 0.0 .0
db file scattered read 30 0 0 4 0.0 .0
control file sequential read 5,837 0 0 0 0.0 .0
ADR block file read 19 0 0 4 0.0 .0
ADR block file write 5 0 0 15 0.0 .0
direct path write 14 0 0 2 0.0 .0
direct path read 3 0 0 7 0.0 .0
latch: shared pool 3 0 0 6 0.0 .0
log file single write 56 0 0 0 0.0 .0
latch: redo allocation 53 0 0 0 0.0 .0
latch: active service list 1 0 0 3 0.0 .0
latch free 11 0 0 0 0.0 .0
rdbms ipc message 314,523 5 57,189 182 1.7
Space Manager: slave idle wa 4,086 88 18,996 4649 0.0
DIAG idle wait 7,185 100 7,186 1000 0.0
Streams AQ: waiting for time 2 50 4,909 ###### 0.0
Streams AQ: qmn slave idle w 129 0 3,612 28002 0.0
Streams AQ: qmn coordinator 258 50 3,612 14001 0.0
smon timer 43 2 3,605 83839 0.0
pmon timer 1,199 99 3,596 2999 0.0
SQL*Net message from client 17,019 0 31 2 0.1
SQL*Net message to client 12,762 0 0 0 0.1
class slave wait 28 0 0 0 0.0
thank you very much!Hi: just know it now: it is a large amount of 'concurrent transaction' designed in this "Volume Test" - to simulate large incoming transaction volme, so I guess wait in eq:TX - row is expected.
The fact: (1) redo logs at uat server is known to not well-tune for configurations (2) volume test slow 5%, however data amount in its test is kept the same by each time import production data, by the team. So why it slowed 5% this year?
The wait histogram is pasted below, any one interest to take a look? any ideas?
Wait Event Histogram DB/Inst: XXXX/XXXX Snaps: 5635-5636
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
ADR block file read 19 26.3 5.3 10.5 57.9
ADR block file write 5 40.0 60.0
ADR file lock 6 100.0
ARCH wait for archivelog l 14 100.0
ASM file metadata operatio 30 100.0
CSS initialization 30 100.0
Disk file operations I/O 9090 97.2 1.4 .6 .4 .2 .1 .1
LGWR wait for redo copy 2447 98.5 .5 .4 .2 .2 .2 .1
Log archive I/O 35 40.0 8.6 25.7 2.9 22.9
SQL*Net break/reset to cli 34 85.3 8.8 5.9
SQL*Net more data to clien 29K 99.9 .0 .0 .0 .0 .0
buffer busy waits 1800 96.8 .7 .7 .6 .3 .4 .5
control file parallel writ 2037 90.7 5.0 2.1 .8 1.0 .3 .1
control file sequential re 7405 100.0 .0
cursor: pin S 10 10.0 90.0
db file async I/O submit 9556 99.9 .0 .0 .0
db file parallel read 1 100.0
db file parallel write 9556 62.0 32.4 1.7 .8 1.5 1.3 .1
db file scattered read 345 72.8 3.8 2.3 11.6 9.0 .6
db file sequential read 6199 97.2 .2 .3 1.6 .7 .0 .0
direct path read 7699 99.1 .4 .2 .1 .1 .0
direct path sync 8 25.0 37.5 12.5 25.0
direct path write 13K 97.8 .9 .5 .4 .3 .1 .0
enq: TX - row lock content 4480 .4 .7 1.3 3.0 6.8 12.3 75.4 .1
latch free 301 98.3 .3 .7 .7
latch: In memory undo latc 15 93.3 6.7
latch: active service list 1 100.0
latch: cache buffers chain 55 94.5 3.6 1.8
latch: cache buffers lru c 9 88.9 11.1
latch: call allocation 6 100.0
latch: checkpoint queue la 3 100.0
latch: enqueue hash chains 45 97.8 2.2
latch: messages 4 100.0
latch: object queue header 7 85.7 14.3
latch: redo allocation 113 97.3 1.8 .9
latch: row cache objects 19 89.5 5.3 5.3
latch: session allocation 5 80.0 20.0
latch: shared pool 147 90.5 1.4 2.7 1.4 .7 1.4 2.0
latch: undo global data 8 100.0
library cache: mutex X 28 89.3 3.6 3.6 3.6
log file parallel write 299K 95.6 2.6 1.0 .4 .3 .2 .0
log file sequential read 698 29.5 .1 4.6 46.8 18.9
log file single write 56 100.0
log file switch (checkpoin 4 25.0 50.0 25.0
log file switch (private s 8 12.5 37.5 50.0
log file sync 284K 93.3 3.7 1.4 .7 .5 .3 .1
os thread startup 146 100.0
row cache lock 41 85.4 9.8 2.4 2.4
DIAG idle wait 7184 100.0
SQL*Net message from clien 6379K 86.6 5.1 2.9 1.3 .7 .3 2.8 .3
SQL*Net message to client 6375K 100.0 .0 .0 .0 .0 .0 .0
Wait Event Histogram DB/Inst: XXXX/xxxx Snaps: 5635-5636
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
SQL*Net more data from cli 7976 99.7 .1 .1 .0 .1
Space Manager: slave idle 4086 .1 .2 .0 .0 .3 3.2 96.1
Streams AQ: qmn coordinato 258 49.2 .8 50.0
Streams AQ: qmn slave idle 129 100.0
Streams AQ: waiting for ti 2 50.0 50.0
class slave wait 28 92.9 3.6 3.6
jobq slave wait 7270 .0 100.0
pmon timer 1199 100.0
rdbms ipc message 314K 10.3 7.3 39.7 15.4 10.6 5.3 8.2 3.3
smon timer 43 100.0 -
hi
in my spreport i am getting this enq problem pls tell me how to solve this.
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
buffer busy waits 1,480,241 1,024,132 692 64.5
enq: HW - contention 142,929 389,756 2727 24.5
free buffer waits 5,245,113 99,454 19 6.3
enq: US - contention 12,172 19,131 1572 1.2
db file parallel write 5,228 10,397 1989 .7
log_buffer=25m
checkpointtimeout=0
checkpointinterval=0
thanks.Total System Global Area 599785472 bytes
Fixed Size 1980328 bytes
Variable Size 146802776 bytes
Database Buffers 423624704 bytes
Redo Buffers 27377664 bytes
using 10gr2
simultaniously 150 user will affect the database.In this database we have create 150 tables.the procedure is.after creating this table using afterinsertion procedure to insert the data in table.We were using 150 procedure to insert 150 tables.
declare
counter NUMBER;
sql_string VARCHAR2(2000);
BEGIN FOR counter IN 1..150 LOOP
sql_string := 'CREATE TABLE emp'||counter||'
(id varchar2(10) primary key,
col_a varchar2(10),
col_b date,
col_c varchar2(10),
col_d varchar2(10),
col_e varchar2(10),
col_f varchar2(10),
col_g varchar2(10),
col_h date,
col_i varchar2(10),
col_j varchar2(10),
col_k date,
col_l varchar2(10),
col_m varchar2(10),
col_n varchar2(10),
col_o varchar2(10),
col_p varchar2(10),
col_q date,
col_r varchar2(10),
col_s varchar2(10),
col_t varchar2(10),
col_u varchar2(10),
col_v char(20),
col_w varchar2(10),
col_x date,
col_y varchar2(10),
col_z varchar2(10),
col_aa varchar2(10),
col_bb varchar2(10),
col_cc char(20),
col_dd varchar2(10),
col_ee date,
col_ff varchar2(10),
col_gg varchar2(10),
col_hh varchar2(10),
col_ii varchar2(10),
col_jj char(10),
col_kk varchar2(10),
col_ll date,
col_mm varchar2(10),
col_nn varchar2(10),
col_oo varchar2(10),
col_pp varchar2(10),
col_qq char(10),
col_rr varchar2(10),
col_ss date,
col_tt varchar2(10),
col_uu varchar2(10),
col_vv varchar2(10),
col_ww varchar2(10),
col_xx char(12)
) TABLESPACE employee
PCTFREE 10
INITRANS 10
MAXTRANS 255
STORAGE
INITIAL 50m
next 50M
MAXEXTENTS UNLIMITED
EXECUTE IMMEDIATE sql_string;
END LOOP;
END;
CREATE OR REPLACE procedure afterinsertion1(n in number)
is
a number;
cc number;
BEGIN
cc:=1;
FOR innercounter IN 1 .. n LOOP
INSERT INTO emp1(id, col_a ,col_b,col_c ,col_d ,col_e, col_f ,col_g ,col_h ,col_i ,col_j ,col_k,col_l,col_m,col_n,col_o,col_p,
col_q,col_r,col_s,col_t,col_u,col_v,col_w,col_x,col_y,col_z,col_aa,col_bb,col_cc,col_dd,col_ee,col_ff,col_gg,
col_hh,col_ii,col_jj,col_kk,col_ll,col_mm,col_nn,col_oo,col_pp,col_qq,col_rr,col_ss,col_tt,col_uu,col_vv,col_ww,
col_xx) VALUES (id.nextval,'col_a','11-JAN-07','col_c','col_d','col_e','col_f','col_g','11-JAN-07' ,'col_i' ,'col_j',
'11-JAN-07','col_l','col_m','col_n','col_o','col_p','11-JAN-07','col_r','col_s','col_t','col_u','col_v','col_w',
'11-JAN-07','col_y','col_z','col_aaa','COL_BB','col_cc','col_dd','11-JAN-07','col_ff','col_gg','col_h','col_ii','col_jj',
'col_kk','11-JAN-07','col_mm','col_nn','col_o','col_pp','col_qq','col_rr','11-JAN-07','col_tt','col_uu',
'col_v','col_ww','col_xx');
cc:=cc+1;
if cc > 3 then
commit;
cc:=0;
end if;
END LOOP;
END;
/ -
Performance problem on wait event PX Deq: Execute Reply
Hi everybody
I encounter some performance problem, I've made a tkprof on a select statement and I saw that more than 95% of the elapsed time is due to event PX Deq: Execute Reply.
This request is not CPU or paging consuming. What is this event and how could I reduce it ? Could it be a disk problem ?
Thanks a lot, best regards
Greg
Here is a sample of my tkprof:
call count cpu elapsed disk query current rows
Parse 1 0.03 0.03 0 0 0 0
Execute 1 0.22 2.16 68 177 12 0
Fetch 2 0.17 511.97 38 40 0 1
total 4 0.42 514.16 106 217 12 1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 38
Rows Row Source Operation
1 PX COORDINATOR (cr=202 pr=103 pw=0 time=513984636 us)
0 PX SEND QC (RANDOM) :TQ10003 (cr=0 pr=0 pw=0 time=0 us)
0 HASH GROUP BY (cr=0 pr=0 pw=0 time=0 us)
0 PX RECEIVE (cr=0 pr=0 pw=0 time=0 us)
0 PX SEND HASH :TQ10002 (cr=0 pr=0 pw=0 time=0 us)
0 HASH GROUP BY (cr=0 pr=0 pw=0 time=0 us)
0 HASH JOIN (cr=0 pr=0 pw=0 time=0 us)
0 BUFFER SORT (cr=0 pr=0 pw=0 time=0 us)
0 PX RECEIVE (cr=0 pr=0 pw=0 time=0 us)
0 PX SEND BROADCAST :TQ10000 (cr=0 pr=0 pw=0 time=0 us)
473 TABLE ACCESS FULL DIM_CALL_DISTANCE (cr=8 pr=7 pw=0 time=27259 us)
0 HASH JOIN (cr=0 pr=0 pw=0 time=0 us)
0 BUFFER SORT (cr=0 pr=0 pw=0 time=0 us)
0 PX RECEIVE (cr=0 pr=0 pw=0 time=0 us)
0 PX SEND BROADCAST :TQ10001 (cr=0 pr=0 pw=0 time=0 us)
4 TABLE ACCESS FULL DIM_AUDIT_CALL (cr=32 pr=31 pw=0 time=35037 us)
0 PX BLOCK ITERATOR PARTITION: 1 16 (cr=0 pr=0 pw=0 time=0 us)
0 TABLE ACCESS FULL FACT_CALL PARTITION: 1 48 (cr=0 pr=0 pw=0 time=0 us)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 67 0.05 0.95
os thread startup 4 0.21 0.80
PX Deq: Join ACK 4 0.00 0.00
PX Deq: Parse Reply 3 0.13 0.17
SQL*Net message to client 2 0.00 0.00
PX Deq: Execute Reply 304 1.96 511.68
db file scattered read 6 0.01 0.03
PX qref latch 12 0.00 0.00
SQL*Net message from client 2 94.93 94.94
PX Deq: Signal ACK 6 0.10 0.11
enq: PS - contention 1 0.00 0.00
********************************************************************************PX Deq: Execute Reply is an idle event associated with Parallel Query. Are your tables partitioned or have a degree greater then 1?
The tables appear to be small in size. The overhead associated with parallel query generally hinders response time on queries involving small tables. -
Hi ! I have the following wait events in my top timed and I don't know who originated them:
Wait Event Wait Time Summary Avg Wait Time (ms)
I# Class Event Waits %Timeouts Total(s) Avg(ms) %DB time Avg Min Max Std Dev Cnt
* DB CPU N/A N/A 59,651.48 N/A 45.87 2
User I/O db file sequential read 4,369,213 0.0 20,831.46 4.8 16.02 4.72 4.29 5.14 0.60 2
Other enq: CF - contention 155,822 3.9 10,390.74 66.7 7.99 68.62 60.31 76.94 11.76 2
System I/O RMAN backup & recovery I/O 87,205 0.0 5,477.09 62.8 9.15 62.81 62.81 62.81 1
Cluster gc current block 2-way 2,914,457 0.0 4,811.61 1.7 3.70 1.67 1.60 1.74 0.10 2
System I/O control file sequential read 3,038,672 0.0 3,762.66 1.2 2.89 1.24 1.22 1.27 0.04 2
Concurrenc os thread startup 2,842 0.0 3,695.14 1300.2 2.84 1311.83 1143.07 1480.59 238.66 2
System I/O log file parallel write 1,341,907 0.0 2,530.17 1.9 1.95 1.88 1.88 1.89 0.01 2
Other reliable message 471,495 0.1 2,388.01 5.1 1.84 5.08 4.12 6.03 1.35 2
Concurrenc row cache lock 3,135,774 0.0 2,224.53 0.7 1.71 0.72 0.68 0.75 0.05 2
1 DB CPU N/A N/A 22,584.30 N/A 37.75
User I/O db file sequential read 2,451,215 0.0
System I/O RMAN backup & recovery I/O 87,205 0.0
Other enq: CF - contention 59,735 5.3
Cluster gc current block 2-way 1,803,542 0.0
System I/O control file sequential read 1,831,180 0.0
Concurrenc os thread startup 1,323 0.0
System I/O log file parallel write 727,883 0.0
Cluster gc cr multi block request 523,744 0.0
Concurrenc row cache lock 1,830,913 0.0
2 DB CPU N/A N/A
User I/O db file sequential read 1,917,998 0.0
Other enq: CF - contention 96,087 3.0
Cluster gc current block 2-way 1,110,915 0.0
Concurrenc os thread startup 1,519 0.0
System I/O control file sequential read 1,207,492 0.0
User I/O direct path read 404,587 0.0
Other reliable message 233,033 0.1
System I/O log file parallel write 614,024 0.0
System I/O control file parallel write 128,905 0.0
Those are the most worrying events:
enq: CF - contention
I/O control file sequential read
Concurrenc os thread startup
I have been investigating and I wonder what is wrong to get Concurrenc os thread startup. According to one blog, os thread should be always related with network issues...
The awr snapshot is from my production window day.
Rac 11.2.0.3 two nodes on Solaris Sparc 10.I have studied the ASH in the problematic period and I have found that there are some full scans:
Summary of All User Input
Format : TEXT
DB Id : 2752323407
Inst num : 1
Begin time : 08-Feb-13 09:30:00
End time : 08-Feb-13 10:20:00
Slot width : Default
Report targets : 0
Report name : ashrpt_1_0208_1020.txt
ASH Report For dbp/dbp1
DB Name DB Id Instance Inst Num Release RAC Host
dbp 2752323407 dbp1 1 11.2.0.3.0 YES host-dbp-1
CPUs SGA Size Buffer Cache Shared Pool ASH Buffer Size
16 12,651M (100%) 10,048M (79.4%) 1,921M (15.2%) 32.0M (0.3%)
Analysis Begin Time: 08-Feb-13 09:30:00
Analysis End Time: 08-Feb-13 10:20:00
Elapsed Time: 50.0 (mins)
Begin Data Source: DBA_HIST_ACTIVE_SESS_HISTORY
in AWR snapshot 5100
End Data Source: DBA_HIST_ACTIVE_SESS_HISTORY
in AWR snapshot 5101
+ V$ACTIVE_SESSION_HISTORY
Sample Count: 10,069
Average Active Sessions: 33.56
Avg. Active Session per CPU: 2.10
Report Target: None specified
Top User Events DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Avg Active
Event Event Class % Event Sessions
library cache lock Concurrency 43.73 14.68
cursor: pin S wait on X Concurrency 18.61 6.25
CPU + Wait for CPU CPU 15.77 5.29
reliable message Other 5.88 1.97
enq: KO - fast object checkpoint Application 3.48 1.17
Top Background Events DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Avg Active
Event Event Class % Activity Sessions
CPU + Wait for CPU CPU 1.25 0.42
Top Cluster Events DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Event % Event Remote I % Activity
gc current block 2-way 1.72 2 1.72
gc cr grant 2-way 1.58 N/A 1.07
Top Event P1/P2/P3 Values DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Event % Event P1 Value, P2 Value, P3 Value % Activity
Parameter 1 Parameter 2 Parameter 3
library cache lock 43.75 "29115227816","29218763456"," 1.22
handle address lock address 100*mode+namespace
"29115227816","28694732944"," 1.20
"29115227816","28812373936"," 1.17
cursor: pin S wait on X 18.61 "1497800770","3934190043136", 1.54
idn value where
"1497800770","7773890805760", 1.15
reliable message 6.07 "30432532808","30354909248"," 0.13
channel context channel handle broadcast message
enq: KO - fast object checkpoi 3.49 "1263468550","65640","1" 0.52
name|mode 2 0
db file sequential read 1.81 "1","25220","1" 0.01
file# block# blocks
Top Service/Module DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Service Module % Activity Action % Action
dbp_DVEBMGS11 CL_SQL_STATEMENT======== 86.80 383 86.80
dbp_D10_0066 CL_SQL_STATEMENT======== 6.28 383 3.34
104 2.94
dbp_D10_0064 CL_SQL_STATEMENT======== 2.40 383 2.39
SYS$BACKGROUND UNNAMED 1.51 UNNAMED 1.51
Top Client IDs DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
No data exists for this section of the report.
Top SQL Command Types DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
-> 'Distinct SQLIDs' is the count of the distinct number of SQLIDs
with the given SQL Command Type found over all the ASH samples
in the analysis period
Distinct Avg Active
SQL Command Type SQLIDs % Activity Sessions
SELECT 485 94.56 31.74
ALTER TABLE 220 2.89 0.97
Top Phases of Execution DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Avg Active
Phase of Execution % Activity Sessions
Parse 67.50 22.66
SQL Execution 30.46 10.22
Hard Parse 5.37 1.80
Top Remote Instances DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Wait Class % Wait Remote I % Activity
Cluster 5.22 2 3.90
N/A 1.27
Top SQL with Top Events DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Sampled #
SQL ID Planhash of Executions % Activity
Event % Event Top Row Source % RwSrc
350v06jcnd822 N/A 0 18.03
library cache lock 9.41 ** Row Source Not Available ** 9.41
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
N/A 0 18.03
cursor: pin S wait on X 8.62 ** Row Source Not Available ** 8.62
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
48unmtd983uz6 N/A 0 16.75
library cache lock 12.87 ** Row Source Not Available ** 12.87
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
N/A 0 16.75
cursor: pin S wait on X 3.88 ** Row Source Not Available ** 3.88
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
350v06jcnd822 2426825131 0 15.49
library cache lock 9.74 ** Row Source Not Available ** 9.74
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
cursor: pin S wait on X 4.14 ** Row Source Not Available ** 4.14
CPU + Wait for CPU 1.61 SELECT STATEMENT 1.58
48unmtd983uz6 3511339786 0 14.98
library cache lock 11.50 ** Row Source Not Available ** 11.50
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
cursor: pin S wait on X 1.97 ** Row Source Not Available ** 1.97
CPU + Wait for CPU 1.51 SELECT STATEMENT 1.42
07tcvyb6frtkx 2929764020 1 1.87
gc cr grant 2-way 0.80 TABLE ACCESS - BY USER ROWID 0.75
SELECT "D3"."SID_0SHIP_TO" AS "SID" FROM "/BIC/FZ99IC035" "F" JOIN "/BIC/DZ99IC
0352" "D2" ON "F" . "KEY_Z99IC0352" = "D2" . "DIMID" JOIN "/BI0/XMATERIAL" "X9"
ON "D2" . "SID_0MATERIAL" = "X9" . "SID" JOIN "/BIC/DZ99IC0355" "D5" ON "F" .
"KEY_Z99IC0355" = "D5" . "DIMID" JOIN "/BIC/DZ99IC0353" "D3" ON "F" . "KEY_Z99
Top SQL with Top Events DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Sampled #
SQL ID Planhash of Executions % Activity
Event % Event Top Row Source % RwSrc
Top SQL with Top Row Sources DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Sampled #
SQL ID PlanHash of Executions % Activity
Row Source % RwSrc Top Event % Event
350v06jcnd822 N/A 0 18.03
** Row Source Not Available ** 18.03 library cache lock 9.41
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
48unmtd983uz6 N/A 0 16.75
** Row Source Not Available ** 16.75 library cache lock 12.87
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
350v06jcnd822 2426825131 0 15.49
** Row Source Not Available ** 13.91 library cache lock 9.74
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
SELECT STATEMENT 1.58 CPU + Wait for CPU 1.58
48unmtd983uz6 3511339786 0 14.98
** Row Source Not Available ** 13.56 library cache lock 11.50
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
SELECT STATEMENT 1.42 CPU + Wait for CPU 1.42
07tcvyb6frtkx 2929764020 1 1.87
TABLE ACCESS - BY USER ROWID 1.59 gc cr grant 2-way 0.75
SELECT "D3"."SID_0SHIP_TO" AS "SID" FROM "/BIC/FZ99IC035" "F" JOIN "/BIC/DZ99IC
0352" "D2" ON "F" . "KEY_Z99IC0352" = "D2" . "DIMID" JOIN "/BI0/XMATERIAL" "X9"
ON "D2" . "SID_0MATERIAL" = "X9" . "SID" JOIN "/BIC/DZ99IC0355" "D5" ON "F" .
"KEY_Z99IC0355" = "D5" . "DIMID" JOIN "/BIC/DZ99IC0353" "D3" ON "F" . "KEY_Z99
Top SQL using literals DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
-> FORCE_MATCHING_SIGNATURE is used to identify SQL statements that are
identical except for their use of literals.
-> Please refer to the Oracle Database Reference to understand how
the FORCE_MATCHING_SIGNATURE is derived.
# of Sampled
FORCE_MATCHING_SIGNATURE % Activity SQL Versions
Example SQL 1
Example SQL 2
1021017294885722791 2.89 218
0htvt0stu1vtq
SELECT COUNT(*) FROM "/BIC/FZ99IC003" WHERE "KEY_Z99IC003P" = :A0
0htvt0stu1vtq
Top Parsing Module/Action DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
Module Action % Activ Event
CL_SQL_STATEMENT============== 383 67.25 library
383 cursor:
383 CPU + Wa
Top Sessions running PQs DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
-> This section aggregates all the work done by the PQ slaves into
the session issuing the parallel query.
Sid,Srl# (Inst) % Activity SQL ID Event % Event
User Program
1506, 19(1) 33.57 350v06jcnd822 library cache lock 19.15
UserID:
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
cursor: pin S wait on X 12.76
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
CPU + Wait for CPU 1.61
SELECT "DT"."SID_0CALMONTH" AS "S____048" ,"D3"."SID_0MATERIAL" AS "S____006" ,
"DU"."SID_0UNIT" AS "S____023" ,"DT"."SID_0CALDAY" AS "S____021" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" ,"X33"."S__Z99GRMAT" AS "S_
___4443" , SUM ( "F"."QUANTITY" ) AS "Z____1299" , COUNT( * ) AS "Z____016"
2255, 1067(1) 31.78 48unmtd983uz6 library cache lock 24.37
UserID:
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
cursor: pin S wait on X 5.85
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
CPU + Wait for CPU 1.51
SELECT "DT"."SID_0CALDAY" AS "S____021" ,"DT"."SID_0CALMONTH" AS "S____048" ,"D
3"."SID_0MATERIAL" AS "S____006" ,"DU"."SID_0UNIT" AS "S____023" ,"D2"."SID_0MET
YPE" AS "S____1342" ,"D2"."SID_0VTYPE" AS "S____504" , SUM ( "F"."QUANTITY" )
AS "Z____1299" , COUNT( * ) AS "Z____016" FROM "/BIC/FZ99IC114" "F" JOIN "/BIC
Top DB Objects DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
-> With respect to Application, Cluster, User I/O and buffer busy waits only.
Object ID % Activity Event % Event
Object Name (Type) Tablespace
13661539 2.45 gc buffer busy acquire 0.87
SAPSR3./BIC/EZ99IC013 (TABLE) PSAPSR3SSD
Top DB Files DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
-> With respect to Cluster and User I/O events only.
File ID % Activity Event % Event
File Name Tablespace
53 3.60 gc current block 2-way 0.98
+dbp_DATA/dbp_2/datafile/psapsr3ssd.315.805562113 PSAPSR3SSD
Top Latches DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
No data exists for this section of the report.
Activity Over Time DB/Inst: dbp/dbp1 (Feb 08 09:30 to 10:20)
-> Analysis period is divided into smaller time slots
-> Top 3 events are reported in each of those slots
-> 'Slot Count' shows the number of ASH samples in that slot
-> 'Event Count' shows the number of ASH samples waiting for
that event in that slot
-> '% Event' is 'Event Count' over all ASH samples in the analysis period
Slot Event
Slot Time (Duration) Count Event Count % Event
09:30:00 (5.0 min) 260 gc buffer busy acquire 43 0.43
reliable message 34 0.34
CPU + Wait for CPU 29 0.29
09:35:00 (5.0 min) 303 CPU + Wait for CPU 76 0.75
db file sequential read 40 0.40
gc buffer busy acquire 39 0.39
09:40:00 (5.0 min) 366 CPU + Wait for CPU 209 2.08
db file sequential read 26 0.26
gc current block 2-way 22 0.22
09:45:00 (5.0 min) 511 CPU + Wait for CPU 249 2.47
cursor: pin S wait on X 93 0.92
reliable message 45 0.45
09:50:00 (5.0 min) 2,245 cursor: pin S wait on X 1,442 14.32
library cache lock 407 4.04
reliable message 112 1.11
09:55:00 (5.0 min) 2,037 library cache lock 1,378 13.69
cursor: pin S wait on X 297 2.95
CPU + Wait for CPU 125 1.24
10:00:00 (5.0 min) 1,823 library cache lock 1,371 13.62
CPU + Wait for CPU 263 2.61
reliable message 72 0.72
10:05:00 (5.0 min) 1,273 library cache lock 866 8.60
CPU + Wait for CPU 155 1.54
reliable message 96 0.95
10:10:00 (5.0 min) 798 library cache lock 350 3.48
CPU + Wait for CPU 287 2.85
reliable message 54 0.54
10:15:00 (5.0 min) 436 CPU + Wait for CPU 200 1.99
reliable message 61 0.61
enq: KO - fast object checkpoi 42 0.42
-------------------------------------------------------------Problems are always on instance 1.
The queries are different each day, the top sql with performance problem changes the sql_id and I cant attack them or apply a sql profile or tune them because they only execute during a period.
Any idea?
:( -
Hello SAP Community,
I start by mentioning a few details about the system I'll be talking about in this subject:
- SAP NetWeaver 7.0
- Oracle Database 10.2g
I was reading the following Note: "Note 618868 - FAQ: Oracle performance", in order to try to understand what's causing the oracle database to have slow performance.
While reading section 3 "How can I determine whether the general database performance can be optimized?" I found out that the ratio of "Busy wait time to CPU time" is away above the recommended 60:40 value. I'm getting a 94:6 ratio. This value was calculated using the query:
SELECT
ROUND((STM1.VALUE - STM2.VALUE) / 1000000) "BUSY WAIT TIME (S)",
ROUND(STM2.VALUE / 1000000) "CPU TIME (S)",
ROUND((STM1.VALUE - STM2.VALUE) / STM1.VALUE * 100) || ' : ' ||
ROUND(STM2.VALUE / STM1.VALUE * 100) RATIO
FROM V$SYS_TIME_MODEL STM1, V$SYS_TIME_MODEL STM2
WHERE STM1.STAT_NAME = 'DB time' AND STM2.STAT_NAME = 'DB CPU';
With such high values, SAP recommends to improve system performance doing some "wait event tuning".
Can someone give me some directions about this subject? Some guides specific to this subject would be nice. Any further information about my system you may require, please ask me.
Thanks in advance.
Best regards,
Daniel GarridoHello again,
Before I did any changes to the Oracle's parameters I checked the Note 619188 - FAQ: Oracle wait events, to understand what could be causing such high event wait time.
With the query:
SELECT EVENT, TOTAL_WAITS, TIME_WAITED, AVG_MS,
ROUND(RATIO_TO_REPORT(TIME_WAITED) OVER () * 100) PERCENT
FROM (SELECT SUBSTR(EVENT, 1, 30) EVENT, TOTAL_WAITS, TIME_WAITED,
ROUND(TIME_WAITED_MICRO / TOTAL_WAITS / 1000, 2) AVG_MS
FROM V$SYSTEM_EVENT
WHERE WAIT_CLASS NOT IN ('Idle', 'System I/O')
UNION
SELECT 'CPU' EVENT, NULL, VALUE, NULL
FROM V$SYSSTAT
WHERE STATISTIC# = 12
ORDER BY 3 DESC)
WHERE ROWNUM <=10;
I got the non-idle events that took more time in my system and the result was:
Result of the SELECT statement
EVENT
TOTAL_WAITS
TIME_WAITED
AVG_MS
PERCENT
log file switch (archiving nee
578.686
57.850.863
999.69
80
buffer busy waits
712.163
6.420.932
90.16
9
CPU
0
2.791.238
4
db file sequential read
4.005.546
1.746.442
4.36
2
log file sync
10.176.490
1.577.177
1.55
2
enq: TX - row lock contention
854.451
642.955
7.52
1
db file scattered read
1.055.533
621.332
5.89
1
enq: CF - contention
210.085
246.910
11.75
0
read by other session
561.558
119.910
2.14
0
log file switch completion
10.777
85.843
79.65
0
So most of the TIME_WAITED for wait events was because of the "log file switch (archiving needed)", after reading what could cause such wait event, I understood this was related with a problem I previously had in the server, where the archiving folder was with no space left. (Meanwhile the backup of the archives is being done and so the folder is being cleaned on a daily basis).
Thank you all for your help!
Maybe you are looking for
-
Row limit in Dashboard Prompts - OBIEE 11g?
Hi, In dashboard prompts values are shown as sets of 256 and I know it can be customizable but my question is..."Is there any limit on records to extract as we have it in Analysis"? If yes, what is the max limit and where to configure this? Thanks in
-
In the last few months I've seen what may be a change in the way the text cursor moves in text fields, particularly in Mail. In Mail I'm composing a reply to an e-mail message. I've typed some text that occupies two lines. The cursor is in the third,
-
how do i get into my ipad if i do not know passcode and i dont want to erase the data i have on it
-
Changing the namespace prefix in OSB body content
I would like to change the namespace prefix of the namespace in the content of the Body in OSB (Replacing g: with dg:) For example: <g:Information> <g:firtName>John<g:firstName> <nc:date>1970-01-01<nc:date> </g:Information> should become <dg:Informat
-
Print text vertically in SAPScript
Hi, How to print text vertically in SAPScript? Can we use controls for it? Please help its urgent. Thanks, Madhura