Cursor: pin S 事件导致系统性能严重下降
环境：db：10.2.0.4.0 system：hp-ux 11.31
近期系统在访问高峰时，性能严重下降，分析awr得到信息为
Elapsed: 60.20 (mins)
DB Time: 1,916.95 (mins) 系统负担非常严重
Library Hit %: 99.60 Soft Parse %: 99.44
Execute to Parse %: -22.89 重解析现象很严重
Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
CPU time 26,661 23.2
cursor: pin S 5,346,339 11,727 2 10.2 Other
latch: cache buffers chains 25,963 5,205 200 4.5 Concurrency
db file scattered read 710,601 2,698 4 2.3 User I/O
latch: library cache 6,743 2,087 309 1.8 Concurrenc
cursor: pin S事件为等待时间第一， 一般判读为系统并发sql语句过多，由于没有之前正常状态的awr报告作为对比，所以对于这个系统问题的根源和解决办法不是很清楚，对于应用方面，减少并发sql语句，绑定变量等，在数据库方面调整ession_cached_cursors参数，增加sharepool空间，由于水平有限只能知道这么多，希望得到更加深入的分析。谢谢
附部分awr（如何上传完整报告？）WORKLOAD REPOSITORY report for
DB Name DB Id Instance Inst num Release RAC Host
ORCL 1183953527 orcl 1 10.2.0.4.0 NO HP-UX-1
Snap Id Snap Time Sessions Cursors/Session
Begin Snap: 36825 14-8ÔÂ -12 09:00:16 1426 1.5
End Snap: 36826 14-8ÔÂ -12 10:00:29 1630 1.4
Elapsed: 60.20 (mins)
DB Time: 1,916.95 (mins)
Report Summary
Cache Sizes
Begin End
Buffer Cache: 9,216M 9,216M Std Block Size: 8K
Shared Pool Size: 6,144M 6,144M Log Buffer: 14,348K
Load Profile
Per Second Per Transaction
Redo size: 85,077.91 5,632.74
Logical reads: 323,798.90 21,437.69
Block changes: 489.98 32.44
Physical reads: 4,190.38 277.43
Physical writes: 36.21 2.40
User calls: 7,273.13 481.53
Parses: 1,499.45 99.27
Hard parses: 8.39 0.56
Sorts: 150.73 9.98
Logons: 1.72 0.11
Executes: 1,220.19 80.78
Transactions: 15.10
% Blocks changed per Read: 0.15 Recursive Call %: 9.01
Rollback per transaction %: 12.68 Rows per Sort: 118.78
Instance Efficiency Percentages (Target 100%)
Buffer Nowait %: 99.99 Redo NoWait %: 99.99
Buffer Hit %: 98.71 In-memory Sort %: 100.00
Library Hit %: 99.60 Soft Parse %: 99.44
Execute to Parse %: -22.89 Latch Hit %: 99.95
Parse CPU to Parse Elapsd %: 6.61 % Non-Parse CPU: 94.79
Shared Pool Statistics
Begin End
Memory Usage %: 92.12 92.19
% SQL with executions>1: 90.33 87.07
% Memory for SQL w/exec>1: 90.37 88.32
Top 5 Timed Events
Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
CPU time 26,661 23.2
cursor: pin S 5,346,339 11,727 2 10.2 Other
latch: cache buffers chains 25,963 5,205 200 4.5 Concurrency
db file scattered read 710,601 2,698 4 2.3 User I/O
latch: library cache 6,743 2,087 309 1.8 Concurrency
Time Model Statistics
Total time in database user-calls (DB Time): 115017.1s
Statistics including the word "background" measure background process time, and so do not contribute to the DB time statistic
Ordered by % or DB time desc, Statistic name
Statistic Name Time (s) % of DB Time
sql execute elapsed time 89,472.10 77.79
DB CPU 26,660.94 23.18
parse time elapsed 20,759.09 18.05
PL/SQL execution elapsed time 322.11 0.28
hard parse elapsed time 256.97 0.22
connection management call elapsed time 90.04 0.08
failed parse elapsed time 44.23 0.04
hard parse (sharing criteria) elapsed time 27.70 0.02
sequence load elapsed time 13.43 0.01
PL/SQL compilation elapsed time 10.03 0.01
hard parse (bind mismatch) elapsed time 2.82 0.00
repeated bind elapsed time 0.10 0.00
inbound PL/SQL rpc elapsed time 0.05 0.00
DB time 115,017.09
background elapsed time 451.46
background cpu time 47.64
Back to Wait Events Statistics
Back to Top
Wait Class
s - second
cs - centisecond - 100th of a second
ms - millisecond - 1000th of a second
us - microsecond - 1000000th of a second
ordered by wait time desc, waits desc
Wait Class Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn
Other 5,359,517 0.06 12,985 2 98.23
Concurrency 41,787 7.97 7,899 189 0.77
User I/O 1,541,553 0.00 4,385 3 28.25
Network 28,263,892 0.00 515 0 518.03
Commit 31,974 1.02 489 15 0.59
Application 7,028 1.04 232 33 0.13
System I/O 41,367 0.00 118 3 0.76
Configuration 48 0.00 8 177 0.00
Back to Wait Events Statistics
Back to Top
Wait Events
s - second
cs - centisecond - 100th of a second
ms - millisecond - 1000th of a second
us - microsecond - 1000000th of a second
ordered by wait time desc, waits desc (idle events last)
Event Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn
cursor: pin S 5,346,339 0.00 11,727 2 97.99
latch: cache buffers chains 25,963 0.00 5,205 200 0.48
db file scattered read 710,601 0.00 2,698 4 13.02
latch: library cache 6,743 0.00 2,087 309 0.12
db file sequential read 657,512 0.00 1,191 2 12.05
latch free 7,839 0.10 973 124 0.14
read by other session 153,818 0.04 489 3 2.82
log file sync 31,974 1.02 489 15 0.59
SQL*Net more data to client 2,454,333 0.00 308 0 44.98
enq: TX - row lock contention 82 89.02 227 2762 0.00
latch: library cache lock 968 0.00 211 218 0.02
latch: session allocation 1,369 0.00 196 143 0.03
latch: shared pool 1,801 0.00 177 98 0.03
SQL*Net message to client 25,709,776 0.00 130 0 471.22
latch: row cache objects 2,772 0.00 113 41 0.05
log file parallel write 33,654 0.00 105 3 0.62
SQL*Net more data from client 99,783 0.00 77 1 1.83
buffer busy waits 198 24.24 64 325 0.00
latch: cache buffers lru chain 204 0.00 35 173 0.00
cursor: pin S wait on X 3,284 99.97 32 10 0.06
LGWR wait for redo copy 3,493 91.24 32 9 0.06
latch: object queue header operation 91 0.00 8 91 0.00
latch: In memory undo latch 43 0.00 8 191 0.00
db file parallel read 530 0.00 7 14 0.01
enq: SQ - contention 8 0.00 5 670 0.00
log file sequential read 400 0.00 5 13 0.01
SQL*Net break/reset to client 6,942 0.00 5 1 0.13
Streams AQ: qmn coordinator waiting for slave to start 1 100.00 5 4889 0.00
control file parallel write 1,844 0.00 5 2 0.03
latch: enqueue hash chains 55 0.00 4 76 0.00
Log archive I/O 344 0.00 3 9 0.01
wait list latch free 104 0.00 2 20 0.00
latch: undo global data 9 0.00 2 191 0.00
log file switch (checkpoint incomplete) 11 0.00 2 150 0.00
log file switch completion 28 0.00 1 53 0.00
os thread startup 11 0.00 1 132 0.00
control file sequential read 5,093 0.00 0 0 0.09
latch: redo allocation 4 0.00 0 68 0.00
enq: RO - fast object reuse 4 0.00 0 59 0.00
local write wait 50 0.00 0 2 0.00
latch: library cache pin 1 0.00 0 95 0.00
latch: cache buffer handles 1 0.00 0 80 0.00
enq: CF - contention 2 0.00 0 19 0.00
direct path read temp 18,314 0.00 0 0 0.34
log file single write 32 0.00 0 1 0.00
enq: TX - index contention 3 0.00 0 5 0.00
direct path write temp 25 0.00 0 0 0.00
reliable message 4 0.00 0 0 0.00
direct path read 350 0.00 0 0 0.01
Background Wait Events
ordered by wait time desc, waits desc (idle events last)
Event Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn
events in waitclass Other 2,441 130.93 140 57 0.04
log file parallel write 33,654 0.00 105 3 0.62
log file sequential read 400 0.00 5 13 0.01
control file parallel write 1,844 0.00 5 2 0.03
buffer busy waits 4 75.00 3 832 0.00
Log archive I/O 344 0.00 3 9 0.01
os thread startup 11 0.00 1 132 0.00
latch: library cache 3 0.00 1 220 0.00
control file sequential read 4,390 0.00 0 0 0.08
latch: shared pool 1 0.00 0 309 0.00
db file sequential read 82 0.00 0 4 0.00
db file scattered read 56 0.00 0 4 0.00
log file single write 32 0.00 0 1 0.00
latch: library cache lock 1 0.00 0 26 0.00
direct path read 344 0.00 0 0 0.01
direct path write 343 0.00 0 0 0.01
latch: In memory undo latch 1 0.00 0 0 0.00
Back to Wait Events Statistics
Back to Top
Operating System Statistics
Statistic Total
AVG_BUSY_TIME 355,404
AVG_IDLE_TIME 5,624
AVG_IOWAIT_TIME 1,350
AVG_SYS_TIME 24,626
AVG_USER_TIME 330,677
BUSY_TIME 2,844,143
IDLE_TIME 45,639
IOWAIT_TIME 11,363
SYS_TIME 197,893
USER_TIME 2,646,250
LOAD 8
OS_CPU_WAIT_TIME ###############
RSRC_MGR_CPU_WAIT_TIME 0
VM_IN_BYTES 0
VM_OUT_BYTES 0
PHYSICAL_MEMORY_BYTES 51,504,857,088
NUM_CPUS 8
NUM_CPU_SOCKETS 8
Back to Wait Events Statistics
Back to Top
Service Statistics
ordered by DB Time
Service Name DB Time (s) DB CPU (s) Physical Reads Logical Reads
orcl 108,264.60 24,956.90 12,796,071 1,110,466,840
SYS$USERS 6,748.20 1,707.20 2,350,206 59,196,825
SYS$BACKGROUND 0.00 0.00 1,188 41,361
Back to Wait Events Statistics
Back to Top
Service Wait Class Stats
Wait Class info for services in the Service Statistics section.
Total Waits and Time Waited displayed for the following wait classes: User I/O, Concurrency, Administrative, Network
Time Waited (Wt Time) in centisecond (100th of a second)
Service Name User I/O Total Wts User I/O Wt Time Concurcy Total Wts Concurcy Wt Time Admin Total Wts Admin Wt Time Network Total Wts Network Wt Time
orcl 1354220 373545 40097 757402 0 0 26777629 50042
SYS$USERS 186115 64780 1563 27129 0 0 1456144 1238
SYS$BACKGROUND 1221 183 23 584 0 0 0 0
Back to Wait Events Statistics
Back to Top
SQL ordered by Elapsed Time
Resources reported for PL/SQL code includes the resources used by all SQL statements called by the code.
% Total DB Time is the Elapsed Time of the SQL statement divided into the Total Database Time multiplied by 100
Elapsed Time (s) CPU Time (s) Executions Elap per Exec (s) % Total DB Time SQL Id SQL Module SQL Text
17,352 1,151 282,032 0.06 15.09 58nqcgq93qdpp w3wp.exe SELECT sysdate FROM dual
13,412 3,344 3,516 3.81 11.66 gpz6zsfjvvgx7 obilling.exe select sum ( costs ) , sum ( ...
9,176 2,771 393 23.35 7.98 ajp712h079d57 dosage.exe select DIAG_DESC from outp_mr ...
8,351 2,411 3,163 2.64 7.26 9skkhx9qfmzjy outpdoct.exe SELECT "OUTP_TREAT_REC"."VIS...
6,278 2,042 2,598 2.42 5.46 f9nfgywd1dzbc outpdoct.exe SELECT :"SYS_B_0" del_indica...
3,406 1,070 729 4.67 2.96 3fgaurmna0hx9 obilling.exe SELECT "OUTP_PRESC_T"."VISIT...
3,228 867 9,221 0.35 2.81 3vy04kpknkqgk w3wp.exe select T1.REMIND_DATETIME, T1....
2,802 514 214 13.09 2.44 f4q13jvt6n8r8 mzcx.exe SELECT DISTINCT "OUTPBILL"."...
2,406 533 725 3.32 2.09 87mazvgqfas1z obilling.exe SELECT "OUTP_TREAT_REC_T"."V...
2,340 535 724 3.23 2.03 f4xrxypn2cpkm obilling.exe SELECT "OUTP_PRESC_T"."VISIT...
2,303 781 15,143 0.15 2.00 89yw129xmhchc w3wp.exe select a.PATIENT_ID, a.VISIT_I...
1,572 449 4,748 0.33 1.37 0nwg8cxxzcb0m w3wp.exe select T1.REMIND_DATETIME, T1....
1,258 135 44 28.58 1.09 cnuj5dstf2559 outpdoct.exe SELECT "DOCT_DRUG_PRESC_DE...
Back to SQL Statistics
Back to Top
SQL ordered by CPU Time
Resources reported for PL/SQL code includes the resources used by all SQL statements called by the code.
% Total DB Time is the Elapsed Time of the SQL statement divided into the Total Database Time multiplied by 100
CPU Time (s) Elapsed Time (s) Executions CPU per Exec (s) % Total DB Time SQL Id SQL Module SQL Text
3,344 13,412 3,516 0.95 11.66 gpz6zsfjvvgx7 obilling.exe select sum ( costs ) , sum ( ...
2,771 9,176 393 7.05 7.98 ajp712h079d57 dosage.exe select DIAG_DESC from outp_mr ...
2,411 8,351 3,163 0.76 7.26 9skkhx9qfmzjy outpdoct.exe SELECT "OUTP_TREAT_REC"."VIS...
2,042 6,278 2,598 0.79 5.46 f9nfgywd1dzbc outpdoct.exe SELECT :"SYS_B_0" del_indica...
1,151 17,352 282,032 0.00 15.09 58nqcgq93qdpp w3wp.exe SELECT sysdate FROM dual
1,070 3,406 729 1.47 2.96 3fgaurmna0hx9 obilling.exe SELECT "OUTP_PRESC_T"."VISIT...
867 3,228 9,221 0.09 2.81 3vy04kpknkqgk w3wp.exe select T1.REMIND_DATETIME, T1....
781 2,303 15,143 0.05 2.00 89yw129xmhchc w3wp.exe select a.PATIENT_ID, a.VISIT_I...
540 825 1,790 0.30 0.72 cw74b4gm99xd5 ORACLE.EXE SELECT "A4"."PATIENT_ID", TO_C...
535 2,340 724 0.74 2.03 f4xrxypn2cpkm obilling.exe SELECT "OUTP_PRESC_T"."VISIT...
533 2,406 725 0.73 2.09 87mazvgqfas1z obilling.exe SELECT "OUTP_TREAT_REC_T"."V...
514 2,802 214 2.40 2.44 f4q13jvt6n8r8 mzcx.exe SELECT DISTINCT "OUTPBILL"."...
449 1,572 4,748 0.09 1.37 0nwg8cxxzcb0m w3wp.exe select T1.REMIND_DATETIME, T1....
135 1,258 44 3.06 1.09 cnuj5dstf2559 outpdoct.exe SELECT "DOCT_DRUG_PRESC_DE...
Back to SQL Statistics
Back to Top
SQL ordered by Gets
Resources reported for PL/SQL code includes the resources used by all SQL statements called by the code.
Total Buffer Gets: 1,169,640,302
Captured SQL account for 71.4% of Total
Buffer Gets Executions Gets per Exec %Total CPU Time (s) Elapsed Time (s) SQL Id SQL Module SQL Text
158,325,351 3,516 45,029.96 13.54 3344.37 13412.17 gpz6zsfjvvgx7 obilling.exe select sum ( costs ) , sum ( ...
142,486,001 3,163 45,047.74 12.18 2410.79 8351.24 9skkhx9qfmzjy outpdoct.exe SELECT "OUTP_TREAT_REC"."VIS...
107,954,245 2,598 41,552.83 9.23 2041.61 6278.17 f9nfgywd1dzbc outpdoct.exe SELECT :"SYS_B_0" del_indica...
54,640,906 729 74,953.23 4.67 1069.81 3405.85 3fgaurmna0hx9 obilling.exe SELECT "OUTP_PRESC_T"."VISIT...
54,464,191 15,143 3,596.66 4.66 781.43 2302.60 89yw129xmhchc w3wp.exe select a.PATIENT_ID, a.VISIT_I...
52,300,799 9,221 5,671.92 4.47 866.73 3228.09 3vy04kpknkqgk w3wp.exe select T1.REMIND_DATETIME, T1....
36,350,381 1,790 20,307.48 3.11 539.85 825.10 cw74b4gm99xd5 ORACLE.EXE SELECT "A4"."PATIENT_ID", TO_C...
28,097,571 1,132 24,821.18 2.40 105.62 304.33 44cpsg5hgwtv3 outpdoct.exe SELECT "CLINIC_MASTER"."NAME...
27,329,179 724 37,747.48 2.34 534.54 2339.79 f4xrxypn2cpkm obilling.exe SELECT "OUTP_PRESC_T"."VISIT...
27,276,384 725 37,622.60 2.33 532.84 2405.61 87mazvgqfas1z obilling.exe SELECT "OUTP_TREAT_REC_T"."V...
26,913,555 4,748 5,668.40 2.30 449.02 1571.69 0nwg8cxxzcb0m w3wp.exe select T1.REMIND_DATETIME, T1....
25,335,084 214 118,388.24 2.17 514.25 2802.12 f4q13jvt6n8r8 mzcx.exe SELECT DISTINCT "OUTPBILL"."...
12,756,154 4 3,189,038.50 1.09 73.99 592.57 4v3jxnrj4duq0 ºǫ́ÅäÒ©ÐÂ2012.07.31.exe SELECT "DRUG_PRESC_MASTER"....
Back to SQL Statistics
Back to Top
SQL ordered by Reads
Total Disk Reads: 15,136,657
Captured SQL account for 81.4% of Total
Physical Reads Executions Reads per Exec %Total CPU Time (s) Elapsed Time (s) SQL Id SQL Module SQL Text
6,716,909 214 31,387.43 44.38 514.25 2802.12 f4q13jvt6n8r8 mzcx.exe SELECT DISTINCT "OUTPBILL"."...
3,642,877 44 82,792.66 24.07 134.64 1257.56 cnuj5dstf2559 outpdoct.exe SELECT "DOCT_DRUG_PRESC_DE...
1,707,395 12 142,282.92 11.28 56.42 470.82 bkkm26gd9ydjp prescent.exe SELECT "DRUG_PRESC_MASTER"....
208,221 4 52,055.25 1.38 73.99 592.57 4v3jxnrj4duq0 ºǫ́ÅäÒ©ÐÂ2012.07.31.exe SELECT "DRUG_PRESC_MASTER"....
137,856 0 0.91 383.09 1041.12 b1y7hxw5s572b ORACLE.EXE SELECT /*+ OPAQUE_TRANSFORM */...
45,210 1 45,210.00 0.30 136.24 617.92 9k2mtm738ffbk ORACLE.EXE SELECT "A1"."PATIENT_ID", "A1"...
21,692 0 0.14 44.82 288.67 5w6u29xnk192r ORACLE.EXE SELECT "A1"."TEST_NO", "A1"."I...
2,078 17,010 0.12 0.01 6.78 13.35 3ts97u1my91n4 testprn.exe SELECT "LAB"."LAB_TEST_ITEM...
539 1,732 0.31 0.00 124.85 289.44 0a96nnbzvb78j sjcj.exe select diag_desc , doctor fro...
224 14,499 0.02 0.00 1.75 2.81 adr24vcznhjhz C:\Documents and Settings\yj.ÐÄÄÚʵÑéÊÒ1.000\×À? SELECT COSTS , BILLING_INDICA...
Back to SQL Statistics
Back to Top
SQL ordered by Executions
Total Executions: 4,407,615
Captured SQL account for 43.0% of Total
Executions Rows Processed Rows per Exec CPU per Exec (s) Elap per Exec (s) SQL Id SQL Module SQL Text
324,063 324,042 1.00 0.00 0.00 1jzhjx524wyfm presdisp.exe select TRADE_PRICE , retail_p...
282,032 281,745 1.00 0.00 0.06 58nqcgq93qdpp w3wp.exe SELECT sysdate FROM dual
197,067 197,006 1.00 0.00 0.00 f0wzs9nc663bn obilling.exe select sysdate from dual
162,033 112,966 0.70 0.00 0.00 ak8wcxnbv0mq7 presdisp.exe select location from drug_stoc...
162,032 162,023 1.00 0.00 0.00 g982zswg09n71 presdisp.exe SELECT sum ( DRUG_STOCK.QUANTI...
152,651 1,296 0.01 0.00 0.00 ctc7t080mcrpr PACSVR.exe Select exam_master.*, exam_ma...
89,187 89,186 1.00 0.00 0.00 054n8y7sfgsjc anesmgr.exe select 1 from dual
53,088 24,076 0.45 0.00 0.00 fhzdwv4sgvurz ORACLE.EXE SELECT "A1"."END_DATE_TIME", S...
43,074 991 0.02 0.00 0.00 8kj11spfu1w5k ORACLE.EXE SELECT DISTINCT "A1"."LOG_DATE...
40,962 4,803 0.12 0.00 0.00 664crkjw8n47y doctws.exe select dept_code , ward_code ...
Back to SQL Statistics
Back to Top
SQL ordered by Parse Calls
Total Parse Calls: 5,416,394
Captured SQL account for 41.3% of Total
Parse Calls Executions % Total Parses SQL Id SQL Module SQL Text
457,941 152,651 8.45 ctc7t080mcrpr PACSVR.exe Select exam_master.*, exam_ma...
324,064 324,063 5.98 1jzhjx524wyfm presdisp.exe select TRADE_PRICE , retail_p...
281,992 282,032 5.21 58nqcgq93qdpp w3wp.exe SELECT sysdate FROM dual
197,067 197,067 3.64 f0wzs9nc663bn obilling.exe select sysdate from dual
162,033 162,033 2.99 ak8wcxnbv0mq7 presdisp.exe select location from drug_stoc...
162,032 162,032 2.99 g982zswg09n71 presdisp.exe SELECT sum ( DRUG_STOCK.QUANTI...
113,451 0 2.09 gbztx1wqkp1zw ORACLE.EXE SELECT * FROM "OPERATION_MASTE...
109,344 0 2.02 1gk864rffswjx ORACLE.EXE SELECT * FROM "PAT_VISIT"
104,957 0 1.94 dw9ggd0h86cw4 ORACLE.EXE SELECT * FROM "ADT_LOG"
89,187 89,187 1.65 054n8y7sfgsjc anesmgr.exe select 1 from dual
Back to SQL Statistics
Back to Top
SQL ordered by Sharable Memory
Only Statements with Sharable Memory greater than 1048576 are displayed
Sharable Mem (b) Executions % Total SQL Id SQL Module SQL Text
14,285,976 0.22 998p6b7d1q9bz ** SQL Text Not Available **
14,265,528 0.22 dsxd21s5bw5h3 ** SQL Text Not Available **
4,621,517 45 0.07 18h8s79av5v45 w3wp.exe select T1.REMIND_DATETIME, T1....
3,949,845 20 0.06 bn69cjprha24w w3wp.exe select T1.REMIND_DATETIME, T1....
3,407,101 41 0.05 5syt1sugfkg7c w3wp.exe select T1.REMIND_DATETIME, T1....
2,623,013 65 0.04 gh6391dujc994 w3wp.exe select T1.REMIND_DATETIME, T1....
2,314,741 30 0.04 49husy8txxw6h w3wp.exe select T1.REMIND_DATETIME, T1....
2,184,421 87 0.03 5a77mhab8fmq4 w3wp.exe select T1.REMIND_DATETIME, T1....
2,029,197 24 0.03 57wp7yk0ajprt w3wp.exe select T1.REMIND_DATETIME, T1....
2,028,837 5 0.03 csd4pjvrfpxry w3wp.exe select T1.REMIND_DATETIME, T1....
2,024,581 44 0.03 890zsd9cju097 w3wp.exe select T1.REMIND_DATETIME, T1....
1,925,949 39 0.03 cw4y41z3vnz8y w3wp.exe select T1.REMIND_DATETIME, T1....
1,793,389 71 0.03 gwx893aphkzw8 w3wp.exe select T1.REMIND_DATETIME, T1....
1,747,565 19 0.03 4p0vbj8qba0sj w3wp.exe select T1.REMIND_DATETIME, T1....
1,747,253 75 0.03 6hhka0kpch5bq w3wp.exe select T1.REMIND_DATETIME, T1....
1,746,061 60 0.03 cqbyt6m5m9f6g w3wp.exe select T1.REMIND_DATETIME, T1....
1,745,909 47 0.03 aatwks74mtqkx w3wp.exe select T1.REMIND_DATETIME, T1....
1,718,005 20 0.03 cmws54r2xv332 w3wp.exe select T1.REMIND_DATETIME, T1....
1,709,493 24 0.03 71zxrhv747290 w3wp.exe select T1.REMIND_DATETIME, T1....
1,265,905 1,585 0.02 a7c5z25vb0kvz obilling.exe SELECT COLUMN_NAME, DATA_TYPE...
1,132,605 30 0.02 9s6btksh172dz w3wp.exe select T1.REMIND_DATETIME, T1....
1,115,157 30 0.02 9s6btksh172dz w3wp.exe select T1.REMIND_DATETIME, T1....
1,055,029 105 0.02 da0pcbq6qxxqx w3wp.exe select T1.REMIND_DATETIME, T1....
Back to SQL Statistics
Back to Top
SQL ordered by Version Count
Only Statements with Version Count greater than 20 are displayed
Version Count Executions SQL Id SQL Module SQL Text
698 998p6b7d1q9bz ** SQL Text Not Available **
697 dsxd21s5bw5h3 ** SQL Text Not Available **
57 192 9mrhw217pc809 DOCTWS.EXE select MR_CODE , mr_attr , m...
43 1,307 53mkjqn5tdptj examapt.exe SELECT to_char ( last_updt_dat...
32 125 399071urkdx7n admit.exe SELECT to_char ( last_updt_dat...
26 564 14fz7s749tx24 prescent.exe Select drug_spec , batch_no , ...
Back to SQL Statistics
Back to Top
Complete List of SQL Text
SQL Id SQL Text
054n8y7sfgsjc select 1 from dual
0a96nnbzvb78j select diag_desc , doctor from outp_mr where patient_id =:1 and visit_date like visit_date and visit_no =:2
0nwg8cxxzcb0m select T1.REMIND_DATETIME, T1.START_DATETIME, T1.VISIT_DATE, T1.REMIND_TYPE, T1.REMIND_ID, T1.DELETE_FLAG, T1.REPEAT_INTERVAL, T1.VISIT_ID, T1.VISIT_NO, T1.DELETE_USER_ID, T1.DELETE_USER_NAME, T1.PATIENT_ID, T1.REMIND_DEPICT, T1.REMIND_INDEX, T2.REMIND_TYPE_NAME, T1.REMIND_USER_ID, T1.REMIND_USER_NAME from REMIND T1, REMIND_TYPE_DICT T2 where T1.REMIND_TYPE=T2.REMIND_TYPE_CODE(+) and T1.Remind_Type in (:"SYS_B_0") and T1.Delete_Flag=:"SYS_B_1" and T1.Remind_User_Name=:"SYS_B_2" order by T1.Remind_Id
14fz7s749tx24 Select drug_spec , batch_no , quantity , supply_indicator from drug_stock where drug_code =:1 and firm_id =:2 and package_spec =:3 and package_units =:4 and storage =:5 order by batch_no DESC
18h8s79av5v45
sql--省略
Back to SQL Statistics
Back to Top
SGA breakdown difference
ordered by Pool, Name
N/A value for Begin MB or End MB indicates the size of that Pool/Name was insignificant, or zero in that snapshot
Pool Name Begin MB End MB % Diff
java free memory 26.47 26.47 0.00
java joxlod exec hp 5.34 5.34 0.00
large free memory 32.00 32.00 0.00
shared CCursor 870.37 871.80 0.16
shared Cursor Stats 65.30 65.30 0.00
shared PCursor 506.48 505.60 -0.17
shared db_block_hash_buckets 90.00 90.00 0.00
shared free memory 484.07 480.10 -0.82
shared kglsim object batch 90.30 90.30 0.00
shared library cache 352.64 351.46 -0.33
shared sql area 3,290.12 3,291.57 0.04
streams free memory 16.00 16.00 0.00
buffer_cache 9,216.00 9,216.00 0.00
fixed_sga 1.99 1.99 0.00
log_buffer 14.01 14.01 0.00
Back to Memory Statistics
Back to Top
Resource Limit Stats
only rows with Current or Maximum Utilization > 80% of Limit are shown
ordered by resource name
Resource Name Current Utilization Maximum Utilization Initial Allocation Limit
processes 1,631 1,824 2000 2000
sessions 1,639 1,884 2205 2205
Back to Top
init.ora Parameters
Parameter Name Begin value End value (if different)
audit_file_dest /oracle/product/10.2.0/admin/orcl/adump
background_dump_dest /oracle/product/10.2.0/admin/orcl/bdump
compatible 10.2.0.1.0
control_files /oradata/orcl/control01.ctl, /oradata/orcl/control02.ctl, /oradata/orcl/control03.ctl, /oralog/orcl/control04.ctl
core_dump_dest /oracle/product/10.2.0/admin/orcl/cdump
cursor_sharing FORCE
db_block_size 8192
db_cache_size 9663676416
db_domain
db_file_multiblock_read_count 32
db_name orcl
db_recovery_file_dest /oracle/product/10.2.0/flash_recovery_area
db_recovery_file_dest_size 2147483648
java_pool_size 33554432
job_queue_processes 10
large_pool_size 33554432
log_archive_dest_1 LOCATION=/oralog/arch
nls_language SIMPLIFIED CHINESE
nls_length_semantics CHAR
nls_territory CHINA
open_cursors 300
pga_aggregate_target 2147483648
processes 2000
remote_login_passwordfile EXCLUSIVE
sessions 2205
shared_pool_size 6442450944
streams_pool_size 16777216
undo_management AUTO
undo_tablespace UNDOTBS1
user_dump_dest /oracle/product/10.2.0/admin/orcl/udump
Back to Top
End of Report
焦点在于解析：
Parses: 1,499.45 99.27
Hard parses: 8.39 0.56
Execute to Parse %: -22.89每秒软解析1500次 ，Execute to Parse 解析是负值 ， 这说明 执行解析比非常差。
以下SQL parse次数频繁：
SQL ordered by Parse Calls
Total Parse Calls: 5,416,394
Captured SQL account for 41.3% of Total
Parse Calls Executions % Total Parses SQL Id SQL Module SQL Text
457,941 152,651 8.45 ctc7t080mcrpr PACSVR.exe Select exam_master.*, exam_ma...
324,064 324,063 5.98 1jzhjx524wyfm presdisp.exe select TRADE_PRICE , retail_p...SQL ID ctc7t080mcrpr 1jzhjx524wyfm
ctc7t080mcrpr 实际执行只有 152,651 ，解析倒有457,941次
建议分析以上语句为何产生了大量软解析
Advice:
1. 分析SQL语句，减少软解析的数量
2. 利用open cursor、session cached cursors等技术减少软解析
3. 考虑设置_kks_use_mutex_pin=FALSE 禁用10g中的MUTEX PIN CURSOR特性
*<font color="red" size="2" face="courier">如果觉得本回复有意义，请点击本条回复右手边的Correct按钮，谢谢！</font>*
Maclean Liu
Oracle Database Administrator
Oracle Certified 10g/11g Master
www.askmaclean.com
Edited by: Liu Maclean on 2012-8-23 下午11:48
Similar Messages
-
Performance problem - event : cursor: pin S wait on X
Hi,
Bellow is 17 min awr report , of oracle PeopleSoft DB on 10204 instance on HPUX machine.
During this time the customers complained on poor performance.
There were 4,104.23 execution per second and 3,784.95 parses
which mean that almost any statment was parsed. since the Soft Parse %= 99.77
its seems that most of the parses are soft parse.
During those 17 min , the DB Time = 721.74 min and the "Top 5 Timed Events"
shows : "cursor: pin S wait on X" at the top of the Timed Events
Attached some details for the awr report
Could you please suggest where to focus ?
Thanks
WORKLOAD REPOSITORY report for
DB Name DB Id Instance Inst Num Release RAC Host
xxxx 2993006132 xxxx 1 10.2.0.4.0 NO xxxx
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 18085 25-Mar-10 10:30:41 286 14.9
End Snap: 18086 25-Mar-10 10:48:39 301 15.1
Elapsed: 17.96 (mins)
DB Time: 721.74 (mins)
Cache Sizes
~~~~~~~~~~~ Begin End
Buffer Cache: 4,448M 4,368M Std Block Size: 8K
Shared Pool Size: 2,736M 2,816M Log Buffer: 2,080K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 3,831,000.13 271,096.84
Logical reads: 164,733.47 11,657.20
Block changes: 17,757.42 1,256.59
Physical reads: 885.19 62.64
Physical writes: 504.92 35.73
User calls: 5,775.09 408.67
Parses: 3,784.95 267.84
Hard parses: 8.55 0.60
Sorts: 212.37 15.03
Logons: 0.77 0.05
Executes: 4,104.23 290.43
Transactions: 14.13
% Blocks changed per Read: 10.78 Recursive Call %: 24.14
Rollback per transaction %: 0.18 Rows per Sort: 57.86
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.98 Redo NoWait %: 99.97
Buffer Hit %: 99.47 In-memory Sort %: 100.00
Library Hit %: 99.73 Soft Parse %: 99.77
Execute to Parse %: 7.78 Latch Hit %: 99.77
Parse CPU to Parse Elapsd %: 3.06 % Non-Parse CPU: 89.23
Shared Pool Statistics Begin End
Memory Usage %: 34.44 34.78
% SQL with executions>1: 76.52 60.40
% Memory for SQL w/exec>1: 73.75 99.18
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
cursor: pin S wait on X 1,378,354 13,462 10 31.1 Concurrenc
db file sequential read 878,684 8,779 10 20.3 User I/O
CPU time 4,998 11.5
local write wait 2,692 2,442 907 5.6 User I/O
cursor: pin S 1,932,830 2,270 1 5.2 Other
Time Model Statistics DB/Inst: xxxx/xxxx Snaps: 18085-18086
Statistic Name Time (s) % of DB Time
sql execute elapsed time 21,690.6 50.1
parse time elapsed 17,504.9 40.4
DB CPU 4,998.0 11.5
hard parse elapsed time 372.1 .9
connection management call elapsed time 183.9 .4
sequence load elapsed time 125.8 .3
PL/SQL execution elapsed time 89.2 .2
PL/SQL compilation elapsed time 9.2 .0
inbound PL/SQL rpc elapsed time 5.5 .0
hard parse (sharing criteria) elapsed time 5.5 .0
hard parse (bind mismatch) elapsed time 0.5 .0
failed parse elapsed time 0.1 .0
repeated bind elapsed time 0.0 .0
DB time 43,304.1 N/A
background elapsed time 3,742.3 N/A
background cpu time 114.8 N/A
Avg
%Time Total Wait wait Waits
Wait Class Waits -outs Time (s) (ms) /txn
Concurrency 1,413,633 97.5 14,283 10 92.8
User I/O 925,010 .3 11,485 12 60.7
Other 1,984,969 .2 2,858 1 130.3
Application 1,342 46.4 1,873 1396 0.1
Configuration 12,116 63.6 1,857 153 0.8
System I/O 582,094 .0 1,444 2 38.2
Commit 17,253 .6 1,057 61 1.1
Network 6,180,701 .0 68 0 405.9
Wait Events DB/Inst: xxxx/xxxx Snaps: 18085-18086
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
cursor: pin S wait on X 1,378,354 100.0 13,462 10 90.5
db file sequential read 878,684 .0 8,779 10 57.7
local write wait 2,692 91.2 2,442 907 0.2
cursor: pin S 1,932,830 .0 2,270 1 126.9
log file switch (checkpoint 2,669 49.1 1,510 566 0.2
enq: RO - fast object reuse 542 86.5 1,398 2580 0.0
log file sync 17,253 .6 1,057 61 1.1
control file sequential read 450,043 .0 579 1 29.6
log file parallel write 17,903 .0 558 31 1.2
enq: TX - row lock contentio 295 52.2 475 1610 0.0
buffer busy waits 7,338 4.4 348 47 0.5
buffer exterminate 322 92.5 302 938 0.0
read by other session 24,694 .0 183 7 1.6
library cache lock 59 94.9 167 2825 0.0
log file sequential read 109,494 .0 161 1 7.2
latch: cache buffers chains 18,662 .0 149 8 1.2
log buffer space 2,493 .0 139 56 0.2
Log archive I/O 3,592 .0 131 37 0.2
free buffer waits 6,420 99.1 130 20 0.4
latch free 42,812 .0 121 3 2.8
Streams capture: waiting for 845 6.0 106 125 0.1
latch: library cache 2,074 .0 96 46 0.1
db file scattered read 12,437 .0 80 6 0.8
enq: SQ - contention 150 14.0 71 471 0.0
SQL*Net more data from clien 331,961 .0 41 0 21.8
latch: shared pool 320 .0 32 100 0.0
LGWR wait for redo copy 5,307 49.1 29 5 0.3
SQL*Net more data to client 254,217 .0 17 0 16.7
control file parallel write 1,038 .0 15 14 0.1
latch: library cache lock 477 .4 14 29 0.0
latch: row cache objects 6,013 .0 10 2 0.4
SQL*Net message to client 5,587,878 .0 10 0 366.9
latch: redo allocation 1,274 .0 9 7 0.1
log file switch completion 62 .0 6 92 0.0
Streams AQ: qmn coordinator 1 100.0 5 4882 0.0
latch: cache buffers lru cha 434 .0 4 9 0.0
block change tracking buffer 111 .0 4 35 0.0
wait list latch free 135 .0 3 21 0.0
enq: TX - index contention 132 .0 2 17 0.0
latch: session allocation 139 .0 2 14 0.0
latch: object queue header o 379 .0 2 4 0.0
row cache lock 15 .0 2 107 0.0
latch: redo copy 56 .0 1 17 0.0
latch: library cache pin 184 .0 1 5 0.0
write complete waits 14 28.6 1 51 0.0
latch: redo writing 251 .0 1 3 0.0
enq: MN - contention 3 .0 1 206 0.0
enq: CF - contention 16 .0 0 23 0.0
log file single write 24 .0 0 13 0.0
os thread startup 3 .0 0 102 0.0
reliable message 66 .0 0 4 0.0
enq: JS - queue lock 2 .0 0 136 0.0
latch: cache buffer handles 46 .0 0 5 0.0
buffer deadlock 65 100.0 0 4 0.0
latch: undo global data 73 .0 0 3 0.0
change tracking file synchro 24 .0 0 6 0.0
change tracking file synchro 30 .0 0 3 0.0
kksfbc child completion 2 100.0 0 52 0.0
SQL*Net break/reset to clien 505 .0 0 0 0.0
db file parallel read 3 .0 0 30 0.0
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
SQL*Net more data from dblin 127 .0 0 0 0.0
SQL*Net more data to dblink 319 .0 0 0 0.0
latch: enqueue hash chains 20 .0 0 2 0.0
latch: checkpoint queue latc 5 .0 0 5 0.0
SQL*Net message to dblink 6,199 .0 0 0 0.4
enq: TX - allocate ITL entry 1 .0 0 22 0.0
direct path read 5,316 .0 0 0 0.3
latch: messages 24 .0 0 1 0.0
enq: US - contention 3 .0 0 4 0.0
direct path write 1,178 .0 0 0 0.1
rdbms ipc reply 1 .0 0 1 0.0
library cache load lock 2 .0 0 0 0.0
direct path write temp 3 .0 0 0 0.0
direct path read temp 3 .0 0 0 0.0
SQL*Net message from client 5,587,890 .0 135,002 24 366.9
wait for unread message on b 7,809 21.8 3,139 402 0.5
LogMiner: client waiting for 262,604 .1 3,021 12 17.2
LogMiner: wakeup event for b 1,405,104 2.4 2,917 2 92.3
Streams AQ: qmn slave idle w 489 .0 2,650 5420 0.0
LogMiner: wakeup event for p 123,723 32.1 2,453 20 8.1
Streams AQ: waiting for time 9 55.6 1,790 198928 0.0
LogMiner: reader waiting for 45,193 51.3 1,526 34 3.0
Streams AQ: waiting for mess 297 99.3 1,052 3542 0.0
Streams AQ: qmn coordinator 470 33.8 1,050 2233 0.0
Streams AQ: delete acknowled 405 32.3 1,049 2591 0.0
jobq slave wait 379 77.8 958 2529 0.0
LogMiner: wakeup event for r 16,591 10.6 125 8 1.1
SGA: MMAN sleep for componen 3,928 99.3 35 9 0.3
SQL*Net message from dblink 6,199 .0 31 5 0.4
single-task message 108 .0 8 74 0.0
class slave wait 3 .0 0 0 0.0
Background Wait Events DB/Inst: xxxx/xxxx Snaps: 18085-18086
-> ordered by wait time desc, waits desc (idle events last)
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
log file parallel write 17,916 .0 558 31 1.2
Log archive I/O 3,592 .0 131 37 0.2
log file sequential read 3,636 .0 47 13 0.2
events in waitclass Other 6,149 42.4 40 7 0.4
log file switch (checkpoint 30 53.3 19 619 0.0
control file parallel write 1,038 .0 15 14 0.1
db file sequential read 1,166 .0 6 5 0.1
control file sequential read 2,986 .0 6 2 0.2
latch: shared pool 4 .0 4 917 0.0
latch: library cache 5 .0 3 646 0.0
free buffer waits 160 98.8 2 10 0.0
buffer busy waits 2 .0 1 404 0.0
latch: redo writing 19 .0 0 23 0.0
log file single write 24 .0 0 13 0.0
os thread startup 3 .0 0 102 0.0
log buffer space 7 .0 0 35 0.0
latch: cache buffers chains 16 .0 0 8 0.0
log file switch completion 1 .0 0 71 0.0
latch: library cache lock 3 66.7 0 11 0.0
latch: redo copy 1 .0 0 20 0.0
direct path read 5,316 .0 0 0 0.3
latch: row cache objects 3 .0 0 1 0.0
direct path write 1,174 .0 0 0 0.1
latch: library cache pin 3 .0 0 0 0.0
rdbms ipc message 20,401 24.2 11,112 545 1.3
Streams AQ: qmn slave idle w 489 .0 2,650 5420 0.0
Streams AQ: waiting for time 9 55.6 1,790 198928 0.0
pmon timer 379 94.5 1,050 2771 0.0
Streams AQ: delete acknowled 406 32.3 1,050 2586 0.0
Streams AQ: qmn coordinator 470 33.8 1,050 2233 0.0
smon timer 146 .0 1,039 7118 0.0
SGA: MMAN sleep for componen 3,928 99.3 35 9 0.3
Operating System Statistics DB/Inst: xxxx/xxxx Snaps: 18085-18086
Statistic Total
AVG_BUSY_TIME 68,992
AVG_IDLE_TIME 37,988
AVG_IOWAIT_TIME 28,529
AVG_SYS_TIME 11,748
AVG_USER_TIME 57,214
BUSY_TIME 552,209
IDLE_TIME 304,181
IOWAIT_TIME 228,489
SYS_TIME 94,253
USER_TIME 457,956
LOAD 2
OS_CPU_WAIT_TIME 147,872,604,500
RSRC_MGR_CPU_WAIT_TIME 0
VM_IN_BYTES 49,152
VM_OUT_BYTES 0
PHYSICAL_MEMORY_BYTES 25,630,269,440
NUM_CPUS 8
NUM_CPU_SOCKETS 8mbobak wrote:
So, this is a parsing related wait. You already mentioned that you're doing lots of parsing, mostly soft. Do you have session_cursor_cache parameter set to a reasonable value? 10g, I believe the default is 50, which is probably not a bad starting point. You may get additional benefits with moderate increases, perhaps to 100-200 range. It can be costly to do so, but can the extra parsing be addressed in the application? Is there anything you can do to reduce parsing in the application? When the problem occurs, how is the CPU consumption on the box? Are the CPUs pegged? Are you bottlenecked on CPU resources? Finally, there are bugs around 10.2.0.x and mutexes, so, you may want to open an SR w/ Oracle support, and determine if the root cause is actually a bug.
Mark,
I think you might read a little more into the stats than you have done - averaging etc. notwithstanding.
There are 8.55 "hard" parses per second - which in 17.96 minutes is about 9,500 hard parses - and there are 1.3M pin S wait on X: which is about 130 per hard parse (and 1.9M pin S). So the average statistics might be showing an interesting impact on individual actions.
The waits on "local write wait" are worth nothing. There are various reasons for this, one of which is the segment header block writes and index root block writes when you truncate a table - which could also be a cause of the "enq: RO - fast object reuse" waits in the body of the report.
Truncating tables tends to invalidate cursors and cause hard parsing.
So I would look for code that is popular, executed from a number of sessions, and truncates tables.
There were some bugs in this area relating to global temporay tables - but they should have been fixed in 10.2.0.4.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
There is a +"Preview"+ tab at the top of the text entry panel. Use this to check what your message will look like before you post the message. If it looks a complete mess you're unlikely to get a response. (Click on the +"Plain text"+ tab if you want to edit the text to tidy it up.)
+"Science is more than a body of knowledge; it is a way of thinking"+
+Carl Sagan+ -
100 %CPU utilizationis , cache buffers chains and cursor: pin S
Hi every one ,
we have incident causing system response very slow with very bad response time, below top 5 wait events from AWR (RAC database)
Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
latch: cache buffers chains 122,492 198,139 1,618 16.8 Concurrency
gc buffer busy 119,903 83,248 694 7.1 Cluster
cursor: pin S 18,674,280 72,651 4 6.2 Other
log file sync 639,867 66,673 104 5.7
Commit latch free 143,519 54,239 378 4.6 Other
Oracle support clearly identified the issue with latch cache buffer chains as SQL statement executed around 35000 times which is too high based on execution plan . and they suggest to tune SQL statements .
my question is cursor: pin S wait on X and library cache lock related ot it is just a symptoms , and is document 742599.1 applicable to us or not as we have 10.2.0.5 (suggest disable automatic memory management)
As I know high CPU utilization as result of latch: cache buffers chains , the cursor Pin S Wait should not .
Thank you in advanceHi,
All these 4 top events (excluding log file sync) are quite unusual and in your case, if all these are comming atop, these quite well be related. So, you can't say that cursor pin s wait on x should not be dealt saperatly, but, still you can try out suggestion in the note. First find out from v$sgasta about current allocation of shared pool, then after disabling automatic memory management, increase shared_pool significantly as compared to current value, and then monito the system
Definitely you should tune your SQL also, as suggested by support.
Salman -
先library cache pin还是先library cache lock??
Question from Oracler:
session1 给test 建主键
session2 select test 出现library cache lock
session3 select test 出现library cache pin
不是说先获得library cache lock再library cache pin吗
session1以exclusive模式获得 library cache lock
session2 以shared模式请求 library cache lock ，session1未释放，所以session2 wait
那session3 什么解释呢？？？？as maclean answer:
SQL> select * from V$version;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
PL/SQL Release 10.2.0.1.0 - Production
CORE 10.2.0.1.0 Production
TNS for Linux: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production
session A SQL> alter table maclean add a10 char(2000) default 'maclean';
session B: select 1 from maclean where rownum=1; ==> hang here !
session C: select 1 from maclean where rownum=1; ==> SAME SQL, hang here !
SQL> select event from v$session where event like 'library%';
EVENT
library cache lock
library cache pin
session 4:
SQL> oradebug setmypid;
Statement processed.
SQL> oradebug dump systemstate 266;
Statement processed.
SQL> oradebug tracefile_name;
/s01/admin/G10R21/udump/g10r21_ora_6208.trc
SO: 0x84f5b4a8, type: 4, owner: 0x84e5d4f8, flag: INIT/-/-/0x00
(session) sid: 142 trans: (nil), creator: 0x84e5d4f8, flag: (41) USR/- BSY/-/-/-/-/-
DID: 0001-0010-00000027, short-term DID: 0000-0000-00000000
txn branch: (nil)
oct: 3, prv: 0, sql: 0x7bf10088, psql: 0x7bf582f0, user: 0/SYS
O/S info: user: oracle, term: pts/1, ospid: 6159, machine: vrh8.oracle.com
program: [email protected] (TNS V1-V3)
application name: [email protected] (TNS V1-V3), hash value=0
waiting for 'library cache lock' blocking sess=0x(nil) seq=23 wait_time=0 seconds since wait started=17
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
Dumping Session Wait History
for 'library cache lock' count=1 wait_time=2149666
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930643
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930300
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930715
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930545
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2929985
session 142 is B waiting for library cache lock
the lock handle address is 7c3a5560
SO: 0x8003b350, type: 53, owner: 0x84f98ba0, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=8003b350 handle=7c3a5560 request=S
call pin=(nil) session pin=(nil) hpc=0005 hlc=0000
htl=0x8003b3d0[0x7d034330,0x7d034330] htb=0x7d034330 ssga=0x7e9f2168
user=84f5b4a8 session=84f5b4a8 count=0 flags=RES/[0010] savepoint=0x1f9
LIBRARY OBJECT HANDLE: handle=7c3a5560 mutex=0x7c3a5690(0)
name=SYS.MACLEAN
hash=458787ae49fd6f284ccb04a892b38231 timestamp=02-09-2012 21:32:36
namespace=TABL flags=KGHP/TIM/SML/[02000000]
kkkk-dddd-llll=0000-0701-0701 lock=X pin=X latch#=3 hpc=0006 hlc=0004
lwt=0x7c3a5608[0x8003b380,0x8003b380] ltm=0x7c3a5618[0x7c3a5618,0x7c3a5618]
pwt=0x7c3a55d0[0x7c3a55d0,0x7c3a55d0] ptm=0x7c3a55e0[0x7c3a55e0,0x7c3a55e0]
ref=0x7c3a5638[0x7c3a5638,0x7c3a5638] lnd=0x7c3a5650[0x7bf75a18,0x7bf90650]
LIBRARY OBJECT: object=7c1dec60
type=TABL flags=EXS/LOC/UPD[0905] pflags=[0000] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change whr
0 7c3a54a0 7c1ded78 I/P/A/-/- 0 NONE 00
8 7c1de7f0 7e33ed48 I/P/A/-/- 1 UPDATE 00
9 7c1de8c0 7bf109e8 I/P/A/-/- 1 NONE 00
10 7c1de948 7bf10600 I/P/A/-/- 1 NONE 00
关于session B的 library cache lock ， 其原因是 add column 的session A 以 X mode lock SYS.MACLEAN, X mode pin SYS.MACLEAN 且不释放， 所以session B的 library cache lock不用多解释
session C:
SO: 0x84f5dd18, type: 4, owner: 0x84e5dce0, flag: INIT/-/-/0x00
(session) sid: 144 trans: (nil), creator: 0x84e5dce0, flag: (41) USR/- BSY/-/-/-/-/-
DID: 0001-0011-0000000A, short-term DID: 0000-0000-00000000
txn branch: (nil)
oct: 3, prv: 0, sql: 0x7bf10088, psql: 0x7bf582f0, user: 0/SYS
O/S info: user: oracle, term: pts/2, ospid: 6183, machine: vrh8.oracle.com
program: [email protected] (TNS V1-V3)
application name: [email protected] (TNS V1-V3), hash value=0
waiting for 'library cache pin' blocking sess=0x(nil) seq=19 wait_time=0 seconds since wait started=17
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
Dumping Session Wait History
for 'library cache pin' count=1 wait_time=2568684
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2930677
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2929805
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2931420
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2930258
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
session 144 is session C , waiting for library cache pin
handle address 7bf46e40=> 指向 一个 child cursor namespace=CRSR， 而这个child cursor已经被 session B pin住了：
SO: 0x7f03f890, type: 54, owner: 0x84f5dd18, flag: INIT/-/-/0x00
LIBRARY OBJECT PIN: pin=7f03f890 handle=7bf46e40 request=S lock=0
user=84f5dd18 session=84f5dd18 count=0 mask=0000 savepoint=0x3f flags=[00]
SO: 0x7ec4cc80, type: 53, owner: 0x84f5dd18, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=7ec4cc80 handle=7bf46e40 mode=N
call pin=(nil) session pin=(nil) hpc=0000 hlc=0000
htl=0x7ec4cd00[0x7e449348,0x80c35108] htb=0x80c35108 ssga=0x80c34ff0
user=84f5dd18 session=84f5dd18 count=1 flags=[0000] savepoint=0x0
LIBRARY OBJECT HANDLE: handle=7bf46e40 mutex=0x7bf46f70(0)
namespace=CRSR flags=RON/KGHP/PN0/[10010000]
kkkk-dddd-llll=0000-0001-0000 lock=N pin=X latch#=3 hpc=0004 hlc=0004
lwt=0x7bf46ee8[0x7bf46ee8,0x7bf46ee8] ltm=0x7bf46ef8[0x7bf46ef8,0x7bf46ef8]
pwt=0x7bf46eb0[0x7f03f8c0,0x7f03f8c0] ptm=0x7bf46ec0[0x7bf46ec0,0x7bf46ec0]
ref=0x7bf46f18[0x7bf7bfe0,0x7bf7bfe0] lnd=0x7bf46f30[0x7bf46f30,0x7bf46f30]
LIBRARY OBJECT: object=7bf29018
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
READ ONLY DEPENDENCIES: count=1 size=16
DATA BLOCKS:
data# heap pointer status pins change whr
0 7bf20060 7bf28ba8 I/P/A/-/- 0 NONE 00
6 7bf77a20 7bf776f8 I/P/A/-/- 1 NONE 00
我们可以找到 上面这个child cursor 的 parent cursor :
SO: 0x7d03b620, type: 53, owner: 0x84f5b4a8, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=7d03b620 handle=7bf10088 mode=N
call pin=(nil) session pin=(nil) hpc=0000 hlc=0000
htl=0x7d03b6a0[0x7d034030,0x7c03c9f8] htb=0x7d034030 ssga=0x7e9f2168
user=84f5b4a8 session=84f5b4a8 count=1 flags=[0000] savepoint=0x1f7
LIBRARY OBJECT HANDLE: handle=7bf10088 mutex=0x7bf101b8(0)
name=select 1 from maclean where rownum=1
hash=324793c639b13d0954bd5421eaed6701 timestamp=03-08-2012 02:29:24
namespace=CRSR flags=RON/KGHP/TIM/KEP/PN0/SML/DBN/[12010044]
kkkk-dddd-llll=0001-0001-0001 lock=N pin=0 latch#=3 hpc=0004 hlc=0004
lwt=0x7bf10130[0x7bf10130,0x7bf10130] ltm=0x7bf10140[0x7bf10140,0x7bf10140]
pwt=0x7bf100f8[0x7bf100f8,0x7bf100f8] ptm=0x7bf10108[0x7bf10108,0x7bf10108]
ref=0x7bf10160[0x7bf10160,0x7bf10160] lnd=0x7bf10178[0x82f4f2f8,0x7bf4d608]
LIBRARY OBJECT: object=7bf7c8a8
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
CHILDREN: size=16
child# table reference handle
0 7bf7c370 7bf7bfe0 7bf46e40 => 只有一个child handle 7bf46e40
DATA BLOCKS:
data# heap pointer status pins change whr
0 7bf2a428 7bf7c9c0 I/P/A/-/- 0 NONE 00即在session B parse SQL "select 1 from maclean where rownum=1" 的时候， 会生成一个child cursor 并 X mode pin住这个child cursor ， 而session C 同时发起 一样的SQL语句 "select 1 from maclean where rownum=1" 时 需要 share 这个child cursor ， 即以 S mode pin 这个child cursor ， 但是session B 还没有完成 optimize 没有生成完整的child cursor ， 需要等待 session A 释放 library cache lock才能 完成， 所以 session C 要等 session B build child cursor ， 此时session C等" library cache pin" ；
如果 session C 执行的是不一样的SQL，那么 session C 不share 同一个child cursor ， session C 会wait for library cache lock.
since 10.2.0.3 "_kks_use_mutex_pin"=TRUE or 11g 开始 使用mutex 保护cursor pin ，所以 session C 若执行 与session B 一样的SQL，那么 wiat for cursor pin S on X
SQL> select * from V$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
session A:
SQL> alter table test add t11 char(2000) default 'maclean';
session B:
SQL> select * from test where rownum=1;
session C:
SQL> select * from test where rownum=1;
SQL> select event from v$session where wait_class='Concurrency';
EVENT
cursor: pin S wait on X
library cache lock
SQL> oradebug setmypid;
Statement processed.
SQL> oradebug dump systemstate 266;
Statement processed.
session C:
SO: 0x9e2256b8, type: 4, owner: 0x9e59a1c0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x9e59a1c0, name=session, file=ksu.h LINE:12624, pg=0
(session) sid: 179 ser: 41307 trans: (nil), creator: 0x9e59a1c0
flags: (0x41) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
flags2: (0x40009) -/-/INC
DID: , short-term DID:
txn branch: (nil)
oct: 3, prv: 0, sql: 0x956e18b8, psql: 0x956e18b8, user: 0/SYS
ksuxds FALSE at location: 0
service name: SYS$USERS
client details:
O/S info: user: oracle, term: pts/3, ospid: 26823
machine: vrh1.oracle.com program: [email protected] (TNS V1-V3)
application name: [email protected] (TNS V1-V3), hash value=1481565533
Current Wait Stack:
0: waiting for 'cursor: pin S wait on X'
idn=0xe76d0d8c, value=0xca00000000, where=0x500000000
wait_id=17 seq_num=18 snap_id=1
wait times: snap=12.671273 sec, exc=12.671273 sec, total=12.671273 sec
wait times: max=infinite, heur=12.671273 sec
wait counts: calls=1148 os=1148
in_wait=1 iflags=0x15b2
There is at least one session blocking this session.
Dumping 1 direct blocker(s):
inst: 1, sid: 202, ser: 15511
Dumping final blocker:
inst: 1, sid: 9, ser: 1
Wait State:
fixed_waits=0 flags=0x22 boundary=(nil)/-1
idn=0xe76d0d8c=> 这个是mutex的标示
KGX Atomic Operation Log 0x94aa8ca8
Mutex 0x8a328978(202, 0) idn e76d0d8c oper GET_SHRD
Cursor Pin uid 179 efd 0 whr 5 slp 1148
opr=2 pso=0x8b5a8c48 flg=0
pcs=0x8a3288e0 nxt=(nil) flg=35 cld=1 hd=0x93d4bbb8 par=0x8a328048
ct=1 hsh=0 unp=(nil) unn=0 hvl=8a328ef0 nhv=1 ses=0x9e1e0ea0
hep=0x8a328978 flg=80 ld=1 ob=0x939a30b0 ptr=0x935e0348 fex=0x935df6f0
这个mutex的 oper是 GET_SHRD 即 pin S 它指向 0x93d4bbb8 是一个child cursor
以下是parent cursor:
SO: 0x957fa9d8, type: 78, owner: 0x9e1e0ea0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x9e593da0, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8547, pg=0
LibraryObjectLock: Address=0x957fa9d8 Handle=0x956e18b8 Mode=N CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0
User=0x9e1e0ea0 Session=0x9e1e0ea0 ReferenceCount=1 Flags=CNB/[0001] SavepointNum=4f5864b8
LibraryHandle: Address=0x956e18b8 Hash=e76d0d8c LockMode=N PinMode=0 LoadLockMode=0 Status=VALD
ObjectName: Name=select * from test where rownum=1
FullHashValue=7e277fabf95d7c80e8924ed6e76d0d8c Namespace=SQL AREA(00) Type=CURSOR(00) Identifier=3882683788 OwnerIdn=0
Statistics: InvalidationCount=1 ExecutionCount=2 LoadCount=3 ActiveLocks=2 TotalLockCount=4 TotalPinCount=1
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=1 BucketInUse=3 HandleInUse=3 HandleReferenceCount=0
Concurrency: DependencyMutex=0x956e1968(0, 2, 0, 0) Mutex=0x956e19e8(0, 45, 0, 0)
Flags=RON/PIN/TIM/PN0/DBN/[10012841]
WaitersLists:
Lock=0x956e1948[0x956e1948,0x956e1948]
Pin=0x956e1928[0x956e1928,0x956e1928]
LoadLock=0x956e19a0[0x956e19a0,0x956e19a0]
Timestamp: Current=03-08-2012 02:45:45
HandleReference: Address=0x956e1a78 Handle=(nil) Flags=[00]
LibraryObject: Address=0x8a327fa8 HeapMask=0000-0001-0001-0000 Flags=EXS[0000] Flags2=[0000] PublicFlags=[0000]
ChildTable: size='16'
Child: id='0' Table=0x8a328e58 Reference=0x8a3288b8 Handle=0x956db988
Child: id='1' Table=0x8a328e58 Reference=0x8a328b80 Handle=0x93d4bbb8
NamespaceDump:
Parent Cursor: sql_id=fj4kfuvmqu3cc parent=0x8a328048 maxchild=2 plk=y ppn=n
但是很可惜 0x93d4bbb8 这个 child cursor 被 session B pin住了：
SO: 0x957fa8d8, type: 78, owner: 0x9e1e0ea0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x9e593da0, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8547, pg=0
LibraryObjectLock: Address=0x957fa8d8 Handle=0x93d4bbb8 Mode=N CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0
ClusterLock=0x8f1945f8 Context=0x7fd379518308 User=0x9e1e0ea0 Session=0x9e1e0ea0 ReferenceCount=1
Flags=CBK/[0020] SavepointNum=0
LibraryHandle: Address=0x93d4bbb8 Hash=0 LockMode=N PinMode=X LoadLockMode=0 Status=VALD
Name: Namespace=SQL AREA(00) Type=CURSOR(00)
Statistics: InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=2 TotalPinCount=3
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0
Concurrency: DependencyMutex=0x93d4bc68(0, 0, 0, 0) Mutex=0x956e19e8(0, 45, 0, 0)
Flags=RON/PIN/PN0/EXP/CHD/[10012111]
WaitersLists:
Lock=0x93d4bc48[0x93d4bc48,0x93d4bc48]
Pin=0x93d4bc28[0x93d4bc28,0x93d4bc28]
LoadLock=0x93d4bca0[0x93d4bca0,0x93d4bca0]
LibraryObject: Address=0x939a30b0 HeapMask=0000-0001-0001-0000 Flags=EXS[0000] Flags2=[0000] PublicFlags=[0000]
DataBlocks:
Block: #='0' name=KGLH0^e76d0d8c pins=0 Change=NONE
Heap=0x93d1a808 Pointer=0x939a3150 Extent=0x939a3030 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=1.546875 Size=4.000000 LoadTime=4385736620
Block: #='6' name=SQLA^e76d0d8c pins=0 Change=NONE
Heap=0x8a328a20 Pointer=0x935e0348 Extent=0x935df6f0 Flags=I/-/P/A/-/E
FreedLocation=0 Alloc=0.000000 Size=0.000000 LoadTime=0
NamespaceDump:
Child Cursor: Heap0=0x939a3150 Heap6=0x935e0348 Heap0 Load Time=03-08-2012 02:50:16 Heap6 Load
PinMode=X
保持这个 X mode pin的是另外一个 mutex ， 这个mutex的 oper是 LONG_EXCL
Time=03-08-2012 02:50:16 ----------------------------------------
KGX Atomic Operation Log 0x8f1945f8
Mutex 0x8a328978(202, 0) idn e76d0d8c oper LONG_EXCL
Cursor Pin uid 202 efd 0 whr 1 slp 0
opr=3 pso=0x957fa8d8 flg=0
pcs=0x8a3288e0 nxt=(nil) flg=35 cld=1 hd=0x93d4bbb8 par=0x8a328048
ct=1 hsh=0 unp=(nil) unn=0 hvl=8a328ef0 nhv=1 ses=0x9e1e0ea0
hep=0x8a328978 flg=80 ld=1 ob=0x939a30b0 ptr=0x935e0348 fex=0x935df6f0 -
Update Statement Simply hanged but doing db file sequential read
Hi,
Last night we had issue with one of the prod server where we updating one of table which contains large number records in millions.Same identical machine completed in1 hour and other box never completed but doing db file sequential read but in the long ops the last statement it was done 20:16 after that nothing is happening but i ran few trace on that user.
/u01/app/oracle/admin/SURV2/udump/surv2_ora_10048.trc
Oracle Database 10g Release 10.2.0.4.0 - Production
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db
System name: SunOS
Node name: prdfa001
Release: 5.10
Version: Generic_139556-08
Machine: i86pc
Instance name: SURV2
Redo thread mounted by this instance: 1
Oracle process number: 18
Unix process pid: 10048, image: oracle@prdfa001
*** 2010-09-09 23:37:07.484
*** ACTION NAME:() 2010-09-09 23:37:07.473
*** MODULE NAME:(SQL*Plus) 2010-09-09 23:37:07.473
*** SERVICE NAME:(SURV2) 2010-09-09 23:37:07.473
*** SESSION ID:(289.54) 2010-09-09 23:37:07.473
Received ORADEBUG command 'unlimit' from process Unix process pid: 3983, image:
*** 2010-09-09 23:37:20.315
Received ORADEBUG command 'event 10046 trace name context forever, level 12' from process Unix process pid: 3983, image:
WAIT #7: nam='db file sequential read' ela= 11160 file#=13 block#=2252349 blocks=1 obj#=166421 tim=12499462835161
WAIT #7: nam='db file sequential read' ela= 2857 file#=13 block#=2249751 blocks=1 obj#=166421 tim=12499462838137
WAIT #7: nam='db file sequential read' ela= 3810 file#=13 block#=2251361 blocks=1 obj#=166421 tim=12499462842048
WAIT #7: nam='db file sequential read' ela= 4459 file#=13 block#=2247059 blocks=1 obj#=166421 tim=12499462846564
WAIT #7: nam='db file sequential read' ela= 2841 file#=13 block#=2247507 blocks=1 obj#=166421 tim=12499462849468
WAIT #7: nam='db file sequential read' ela= 427 file#=13 block#=2247568 blocks=1 obj#=166421 tim=12499462850032
WAIT #7: nam='db file sequential read' ela= 1187 file#=13 block#=2248264 blocks=1 obj#=166421 tim=12499462851327
WAIT #7: nam='db file sequential read' ela= 2687 file#=13 block#=2250707 blocks=1 obj#=166421 tim=12499462854178
WAIT #7: nam='db file sequential read' ela= 3657 file#=13 block#=2249697 blocks=1 obj#=166421 tim=12499462857896
WAIT #7: nam='db file sequential read' ela= 4139 file#=13 block#=2247074 blocks=1 obj#=166421 tim=12499462862093
WAIT #7: nam='db file sequential read' ela= 4180 file#=47 block#=3649690 blocks=1 obj#=166421 tim=12499509270445
WAIT #7: nam='db file sequential read' ela= 4802 file#=47 block#=3649309 blocks=1 obj#=166421 tim=12499509275327
WAIT #7: nam='db file sequential read' ela= 2459 file#=47 block#=3652697 blocks=1 obj#=166421 tim=12499509277859
WAIT #7: nam='db file sequential read' ela= 4015 file#=47 block#=3652826 blocks=1 obj#=166421 tim=12499509281948
WAIT #7: nam='db file sequential read' ela= 2248 file#=47 block#=3651610 blocks=1 obj#=166421 tim=12499509284269
WAIT #7: nam='db file sequential read' ela= 4824 file#=47 block#=3654297 blocks=1 obj#=166421 tim=12499509289166
WAIT #7: nam='db file sequential read' ela= 2008 file#=47 block#=3652312 blocks=1 obj#=166421 tim=12499509291248
WAIT #7: nam='db file sequential read' ela= 1925 file#=47 block#=3654490 blocks=1 obj#=166421 tim=12499509293246
WAIT #7: nam='db file sequential read' ela= 2859 file#=47 block#=3648458 blocks=1 obj#=166421 tim=12499509296178
WAIT #7: nam='db file sequential read' ela= 1740 file#=47 block#=3648212 blocks=1 obj#=166421 tim=12499509297991
WAIT #7: nam='db file sequential read' ela= 2566 file#=47 block#=3648411 blocks=1 obj#=166421 tim=12499509300631
WAIT #7: nam='db file sequential read' ela= 50772 file#=5 block#=480749 blocks=1 obj#=166421 tim=12499509351477
WAIT #7: nam='db file sequential read' ela= 12928 file#=5 block#=477177 blocks=1 obj#=166421 tim=12499509364482
WAIT #7: nam='db file sequential read' ela= 11116 file#=5 block#=479412 blocks=1 obj#=166421 tim=12499509375672
WAIT #7: nam='db file sequential read' ela= 4803 file#=5 block#=483440 blocks=1 obj#=166421 tim=12499509380549
WAIT #7: nam='db file sequential read' ela= 6900 file#=5 block#=481454 blocks=1 obj#=166421 tim=12499509387522
Received ORADEBUG command 'event 10046 trace name context off' from process Unix process pid: 3983, image:
/u01/app/oracle/admin/SURV2/udump/surv2_ora_1545.trc
Oracle Database 10g Release 10.2.0.4.0 - Production
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db
System name: SunOS
Node name: prdfa001
Release: 5.10
Version: Generic_139556-08
Machine: i86pc
Instance name: SURV2
Redo thread mounted by this instance: 1
Oracle process number: 22
Unix process pid: 1545, image: oracle@prdfa001 (TNS V1-V3)
*** ACTION NAME:() 2010-09-09 23:20:13.485
*** MODULE NAME:(sqlplus@prdfa001 (TNS V1-V3)) 2010-09-09 23:20:13.485
*** SERVICE NAME:(SYS$USERS) 2010-09-09 23:20:13.485
*** SESSION ID:(290.697) 2010-09-09 23:20:13.485
===================================================
SYSTEM STATE
System global information:
processes: base 47819b480, size 300, cleanup 4781a5638
allocation: free sessions 47f1d6148, free calls 0
control alloc errors: 0 (process), 0 (session), 0 (call)
PMON latch cleanup depth: 0
seconds since PMON's last scan for dead processes: 20
system statistics:
1171 logons cumulative
19 logons current
89219 opened cursors cumulative
86 opened cursors current
15095069 user commits
5 user rollbacks
58632904 user calls
44023255 recursive calls
224311 recursive cpu usage
201424173 session logical reads
0 session stored procedure space
901812 CPU used when call started
995437 CPU used by this session
6814196 DB time
0 cluster wait time
22542300822 concurrency wait time
3095 application wait time
16479074661 user I/O wait time
1284052668 session connect time
1284067190 process last non-idle time
189018343568 session uga memory
1249667216 session uga memory max
26059216 messages sent
26059220 messages received
239739 background timeouts
162399896 session pga memory
189662872 session pga memory max
4 enqueue timeouts
901146 enqueue waits
0 enqueue deadlocks
32122711 enqueue requests
17819 enqueue conversions
32122676 enqueue releases
0 global enqueue gets sync
0 global enqueue gets async
0 global enqueue get time
0 global enqueue releases
2865667 physical read total IO requests
262620 physical read total multi block requests
270093476864 physical read total bytes
select SYS_CONTEXT('USERENV', 'SERVER_HOST'), SYS_CONTEXT('USERENV', 'DB_UNIQUE_NAME'), SYS_CONTEXT('USERENV', 'INSTANCE_NAME'), SYS_CONTEXT('USERENV', 'SERVICE_NAME'), INSTANCE_NUMBER, STARTUP_TIME, SYS_CONTEXT('USERENV', 'DB_DOMAIN') from v$instance where INSTANCE_NAME=SYS_CONTEXT('USERENV', 'INSTANCE_NAME')
hash=550c95f3d0cfa8290e60ea8382d3a2ca timestamp=09-09-2010 04:24:19
namespace=CRSR flags=RON/KGHP/TIM/PN0/LRG/KST/DBN/MTX/[100100d1]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=9 hpc=0582 hlc=0582
lwt=47df576e8[47df576e8,47df576e8] ltm=47df576f8[47df576f8,47df576f8]
pwt=47df576b0[47df576b0,47df576b0] ptm=47df576c0[47df576c0,47df576c0]
ref=47df57718[47df57718,47df57718] lnd=47df57730[47df57730,47df57730]
LIBRARY OBJECT: object=471ee1d38
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
CHILDREN: size=16
child# table reference handle
0 471ee1800 471ee1470 47df7dce0
DATA BLOCKS:
data# heap pointer status pins change whr
0 47df7de48 471ee1e50 I/P/A/-/- 0 NONE 00
SO: 473691d60, type: 53, owner: 47924e810, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=473691d60 handle=47bb22fa0 mode=N
call pin=0 session pin=0 hpc=0000 hlc=0000
htl=473691de0[4735dbcb8,476cfbf58] htb=476cfbf58 ssga=476cfb6a0
user=47924e810 session=47f2310f0 count=1 flags=[0000] savepoint=0x0
LIBRARY OBJECT HANDLE: handle=47bb22fa0 mtx=47bb230d0(0) cdp=0
namespace=CRSR flags=RON/KGHP/PN0/EXP/[10010100]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=3 hpc=fd84 hlc=fd84
lwt=47bb23048[47bb23048,47bb23048] ltm=47bb23058[47bb23058,47bb23058]
pwt=47bb23010[47bb23010,47bb23010] ptm=47bb23020[47bb23020,47bb23020]
ref=47bb23078[472f8de18,472f8de18] lnd=47bb23090[47bb23090,47bb23090]
LIBRARY OBJECT: object=472f8d9d8
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
DEPENDENCIES: count=1 size=16
AUTHORIZATIONS: count=1 size=16 minimum entrysize=16
ACCESSES: count=1 size=16
TRANSLATIONS: count=1 size=16
DATA BLOCKS:
data# heap pointer status pins change whr
0 47bb22ee0 472f8daf0 I/P/A/-/- 0 NONE 00
6 472f8e508 46be86250 I/-/A/-/E 0 NONE 00
SO: 4735dbc38, type: 53, owner: 47924e810, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=4735dbc38 handle=47bb231c8 mode=N
call pin=0 session pin=0 hpc=0000 hlc=0000
htl=4735dbcb8[476cfbf58,473691de0] htb=476cfbf58 ssga=476cfb6a0
user=47924e810 session=47f2310f0 count=1 flags=[0000] savepoint=0x4c894f8b
LIBRARY OBJECT HANDLE: handle=47bb231c8 mtx=47bb232f8(1) cdp=1
name=select value$ from props$ where name = 'GLOBAL_DB_NAME'
hash=4bb432d65c5a391a42a5c3fa74472c7a timestamp=09-09-2010 04:24:12
namespace=CRSR flags=RON/KGHP/TIM/PN0/SML/KST/DBN/MTX/[120100d0]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=3 hpc=0584 hlc=0584
lwt=47bb23270[47bb23270,47bb23270] ltm=47bb23280[47bb23280,47bb23280]
pwt=47bb23238[47bb23238,47bb23238] ptm=47bb23248[47bb23248,47bb23248]
ref=47bb232a0[47bb232a0,47bb232a0] lnd=47bb232b8[47bb232b8,47bb232b8]
LIBRARY OBJECT: object=472f8e6e0
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
CHILDREN: size=16
child# table reference handle
0 472f8e1a8 472f8de18 47bb22fa0
DATA BLOCKS:
data# heap pointer status pins change whr
0 47bb23108 472f8e7f8 I/P/A/-/- 0 NONE 00
SO: 473644348, type: 53, owner: 47924e810, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=473644348 handle=47bbde418 mode=N
call pin=0 session pin=0 hpc=0000 hlc=0000
htl=4736443c8[476cfc0b8,476cfc0b8] htb=476cfc0b8 ssga=476cfb6a0
user=47924e810 session=47924e810 count=1 flags=[0000] savepoint=0x4c894f8b
LIBRARY OBJECT HANDLE: handle=47bbde418 mtx=47bbde548(0) cdp=0
name=ALTER SESSION SET TIME_ZONE='+02:00'
hash=3878dff8839e71e3dd05a2e75fbd6390 timestamp=09-09-2010 04:24:04
namespace=CRSR flags=RON/KGHP/TIM/PN0/SML/DBN/[12010040]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=11 hpc=04e8 hlc=04e8
lwt=47bbde4c0[47bbde4c0,47bbde4c0] ltm=47bbde4d0[47bbde4d0,47bbde4d0]
pwt=47bbde488[47bbde488,47bbde488] ptm=47bbde498[47bbde498,47bbde498]
ref=47bbde4f0[47bbde4f0,47bbde4f0] lnd=47bbde508[47bbde508,47bbde508]
LIBRARY OBJECT: object=472fffc08
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change whr
0 47bbde320 472fffd20 I/P/A/-/- 0 NONE 00
SO: 47aecf9e8, type: 41, owner: 47924e810, flag: INIT/-/-/0x00
(dummy) nxc=0, nlb=0
SO: 47f290540, type: 11, owner: 4781a7dc0, flag: INIT/-/-/0x00
(broadcast handle) flag: (2) ACTIVE SUBSCRIBER, owner: 4781a7dc0,
event: 1132, last message event: 1132,
last message waited event: 1132, next message: 0(0), messages read: 0
channel: (47a2df4f8) system events broadcast channel
scope: 2, event: 1132, last mesage event: 18,
publishers/subscribers: 0/17,
messages published: 1
SO: 47826b228, type: 3, owner: 4781a7dc0, flag: INIT/-/-/0x00
(call) sess: cur 47924e810, rec 0, usr 47924e810; depth: 0
SO: 476c52968, type: 16, owner: 4781a7dc0, flag: INIT/-/-/0x00
(osp req holder)
PSEUDO PROCESS for group DEFAULT:
SO: 47a1eb7d0, type: 2, owner: 0, flag: INIT/-/-/0x00
(process) Oracle pid=0, calls cur/top: 0/0, flag: (20) PSEUDO
int error: 0, call error: 0, sess error: 0, txn error 0
(post info) last post received: 0 0 0
last post received-location: No post
last process to post me: none
last post sent: 0 0 0
last post sent-location: No post
last process posted by me: none
(latch info) wait_event=0 bits=0
Process Group: DEFAULT, pseudo proc: 47a1eb7d0
O/S info: user: , term: , ospid: (DEAD)
OSD pid info: Unix process pid: 0, image: PSEUDO
Dump of memory from 0x00000004791BF538 to 0x00000004791BF740
4791BF530 00000000 00000000 [........]
4791BF540 00000000 00000000 00000000 00000000 [................]
Repeat 31 times
NO DETACHED BRANCHES.
NO DETACHED NETWORK CONNECTIONS.
CLEANUP STATE OBJECTS:
SO: 47f0cd038, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: instance enqueue anchor state
latch: 0x380009890
SO: 4782cf080, type: 5, owner: 47f0cd038, flag: INIT/-/-/0x00
(enqueue) TA-00000006-00000001 DID: 0001-000F-0000000B
lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
res: 0x47a28d020, mode: X, lock_flag: 0x0
own: 0x0, sess: 0x0, prv: 0x47a28d030
SO: 47f0cd098, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: switchable channel handle anch
latch: 0x38000ac98
SO: 47f28f868, type: 11, owner: 47f0cd098, flag: INIT/-/-/0x00
(broadcast handle) flag: (c2) ACTIVE SUBSCRIBER, owner: 0,
event: 1, last message event: 1,
last message waited event: 1, next message: 0(0), messages read: 0
channel: (47a2e4190) KPON channel
scope: 2, event: 1, last mesage event: 0,
publishers/subscribers: 0/1,
messages published: 0
SO: 47f0cd0f8, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: TT shared object cleanup SO
latch: 0x38001c6b8
SO: 47f0cd158, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: SS shared object cleanup SO
latch: 0x38001cd48
END OF SYSTEM STATE
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
db file sequential read 2,347,652 9,215 4 64.5 User I/O
db file scattered read 245,687 4,199 17 29.4 User I/O
CPU time 974 6.8
db file parallel write 50,082 408 8 2.9 System I/O
log file parallel write 6,963 52 7 0.4 System I/O
Time Model Statistics DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> Total time in database user-calls (DB Time): 14286.4s
-> Statistics including the word "background" measure background process
time, and so do not contribute to the DB time statistic
-> Ordered by % or DB time desc, Statistic name
Statistic Name Time (s) % of DB Time
sql execute elapsed time 14,280.3 100.0
DB CPU 974.5 6.8
PL/SQL execution elapsed time 531.8 3.7
parse time elapsed 30.5 .2
hard parse elapsed time 27.1 .2
connection management call elapsed time 14.9 .1
hard parse (sharing criteria) elapsed time 3.4 .0
hard parse (bind mismatch) elapsed time 3.1 .0
PL/SQL compilation elapsed time 2.4 .0
failed parse elapsed time 0.0 .0
repeated bind elapsed time 0.0 .0
sequence load elapsed time 0.0 .0
DB time 14,286.4 N/A
background elapsed time 670.2 N/A
background cpu time 186.1 N/A
Wait Class DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc
Avg
%Time Total Wait wait Waits
Wait Class Waits -outs Time (s) (ms) /txn
User I/O 2,593,484 .0 13,415 5 150.0
System I/O 87,506 .0 515 6 5.1
Other 839 11.4 6 7 0.0
Commit 3,225 .1 6 2 0.2
Concurrency 1,033 .0 5 5 0.1
Configuration 2,514 99.4 0 0 0.1
Network 47,559 .0 0 0 2.8
Application 7 .0 0 0 0.0
Wait Events DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
db file sequential read 2,347,652 .0 9,215 4 135.8
db file scattered read 245,687 .0 4,199 17 14.2
db file parallel write 50,082 .0 408 8 2.9
log file parallel write 6,963 .0 52 7 0.4
control file parallel write 6,203 .0 44 7 0.4
control file sequential read 24,242 .0 11 0 1.4
log file sync 3,225 .1 6 2 0.2
latch free 84 .0 4 47 0.0
os thread startup 25 .0 3 120 0.0
latch: session allocation 39 .0 1 33 0.0
db file parallel read 12 .0 1 92 0.0
enq: TX - index contention 186 .0 1 3 0.0
latch: shared pool 47 .0 1 11 0.0
LGWR wait for redo copy 319 3.1 0 1 0.0
library cache load lock 2 .0 0 172 0.0
buffer busy waits 590 .0 0 0 0.0
log file switch completion 6 .0 0 29 0.0
SGA: allocation forcing comp 11 54.5 0 14 0.0
latch: library cache lock 50 .0 0 3 0.0
read by other session 38 .0 0 4 0.0
direct path read 42 .0 0 3 0.0
SQL*Net message to client 44,807 .0 0 0 2.6
rdbms ipc reply 207 .0 0 0 0.0
SQL*Net more data from clien 1,014 .0 0 0 0.1
latch: cache buffers chains 24 .0 0 1 0.0
latch: library cache 29 .0 0 1 0.0
log file sequential read 8 .0 0 3 0.0
direct path write 50 .0 0 0 0.0
SQL*Net more data to client 398 .0 0 0 0.0
latch: object queue header o 12 .0 0 1 0.0
latch: In memory undo latch 78 .0 0 0 0.0
undo segment extension 2,507 99.7 0 0 0.1
latch: cache buffers lru cha 4 .0 0 1 0.0
log file single write 8 .0 0 0 0.0
local write wait 3 .0 0 1 0.0
enq: RO - fast object reuse 3 .0 0 1 0.0
buffer deadlock 87 92.0 0 0 0.0
enq: JS - queue lock 1 .0 0 1 0.0
cursor: pin S 70 .0 0 0 0.0
latch: row cache objects 2 .0 0 1 0.0
SQL*Net message to dblink 1,338 .0 0 0 0.1
latch: checkpoint queue latc 2 .0 0 0 0.0
reliable message 3 .0 0 0 0.0
log buffer space 1 .0 0 1 0.0
SQL*Net break/reset to clien 4 .0 0 0 0.0
SQL*Net more data from dblin 2 .0 0 0 0.0
SQL*Net message from client 44,949 .0 155,701 3464 2.6
virtual circuit status 621 100.0 18,156 29237 0.0
Streams AQ: qmn slave idle w 664 .0 18,127 27299 0.0
Streams AQ: qmn coordinator 1,339 50.4 18,099 13517 0.1
Streams AQ: waiting for time 12 100.0 8,741 728394 0.0
jobq slave wait 130 100.0 380 2927 0.0
PL/SQL lock timer 1 100.0 1 978 0.0
SQL*Net message from dblink 1,338 .0 0 0 0.1
single-task message 1 .0 0 38 0.0
class slave wait 11 .0 0 1 0.0
SQL ordered by Elapsed Time DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> Resources reported for PL/SQL code includes the resources used by all SQL
statements called by the code.
-> % Total DB Time is the Elapsed Time of the SQL statement divided
into the Total Database Time multiplied by 100
Elapsed CPU Elap per % Total
Time (s) Time (s) Executions Exec (s) DB Time SQL Id
13,664 906 0 N/A 95.6 gr2cx6athc5j5
Module: SQL*Plus
BEGIN DBMS_OUTPUT.PUT_LINE(equiduct.eod(NULL,NULL)); END;
8,792 195 0 N/A 61.5 986fzxtzr52u5
Module: SQL*Plus
UPDATE TIBEX_ORDER SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"SYS_B_1"
2,524 368 1 2524.1 17.7 c4uf0x6hdgnwq
Module: SQL*Plus
UPDATE TIBEX_FIXSESSIONSTATE SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"
SYS_B_1"
1,414 177 1 1414.4 9.9 cbg09ma34kq8w
Module: SQL*Plus
SELECT count(*) FROM TIBEX_ORDER WHERE INSTRUMENTID=:"SYS_B_0"
742 137 1 742.2 5.2 g0sg6v994wssq
Module: SQL*Plus
SELECT count(*) FROM TIBEX_FIXSESSIONSTATE WHERE INSTRUMENTID=:"SYS_B_0"
274 11 1 274.2 1.9 6mcpb06rctk0x
Module: DBMS_SCHEDULER
call dbms_space.auto_space_advisor_job_proc ( )
264 8 27 9.8 1.8 8szmwam7fysa3
Module: DBMS_SCHEDULER
insert into wri$_adv_objspace_trend_data select timepoint, space_usage, space_a
lloc, quality from table(dbms_space.object_growth_trend(:1, :2, :3, :4, NULL, N
ULL, NULL, 'FALSE', :5, 'FALSE'))
99 1 1 99.4 0.7 1z0x41f66nvjr
Module: SQL*Plus
UPDATE TIBEX_INSTRUMENTADMIN SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"
SYS_B_1"
21 10 1 21.5 0.2 bbc1ck8594kvj
Module: SQL*Plus
UPDATE TIBEX_INSTRUMENTDAILYHIST SET ADJOPEN=NVL(ADJOPEN,OPEN), ADJHIGH=NVL(ADJH
IGH,HIGH), ADJLOW=NVL(ADJLOW,LOW), ADJMID=NVL(ADJMID,MID), ADJCLOSE=NVL(ADJCLOSE
,CLOSE), ADJVOLUME=NVL(ADJVOLUME,VOLUME), ADJCLOSINGBID=NVL(ADJCLOSINGBID,CLOSIN
GBID), ADJCLOSINGOFFER=NVL(ADJCLOSINGOFFER,CLOSINGOFFER)
12 0 1 12.5 0.1 6xm9p9uy5kaap
Module: SQL*Plus
SELECT count(*) FROM TIBEX_INSTRUMENTSTATE WHERE INSTRUMENTID=:"SYS_B_0"
SQL ordered by CPU Time DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> Resources reported for PL/SQL code includes the resources used by all SQL
statements called by the code.
-> % Total DB Time is the Elapsed Time of the SQL statement divided
into the Total Database Time multiplied by 100
CPU Elapsed CPU per % Total
Time (s) Time (s) Executions Exec (s) DB Time SQL Id
906 13,664 0 N/A 95.6 gr2cx6athc5j5
Module: SQL*Plus
BEGIN DBMS_OUTPUT.PUT_LINE(equiduct.eod(NULL,NULL)); END;
368 2,524 1 367.51 17.7 c4uf0x6hdgnwq
Module: SQL*Plus
UPDATE TIBEX_FIXSESSIONSTATE SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"
SYS_B_1"
195 8,792 0 N/A 61.5 986fzxtzr52u5
Module: SQL*Plus
UPDATE TIBEX_ORDER SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"SYS_B_1"
177 1,414 1 176.93 9.9 cbg09ma34kq8w
Module: SQL*Plus
SELECT count(*) FROM TIBEX_ORDER WHERE INSTRUMENTID=:"SYS_B_0"
137 742 1 137.38 5.2 g0sg6v994wssq
Module: SQL*Plus
SELECT count(*) FROM TIBEX_FIXSESSIONSTATE WHERE INSTRUMENTID=:"SYS_B_0"
11 274 1 10.82 1.9 6mcpb06rctk0x
Module: DBMS_SCHEDULER
call dbms_space.auto_space_advisor_job_proc ( )
10 21 1 9.65 0.2 bbc1ck8594kvjEdited by: NM on 10-Sep-2010 07:39Hi,
Last night we had issue with one of the prod server where we updating one of table which contains large number records in millions.Same identical machine completed in1 hour and other box never completed but doing db file sequential read but in the long ops the last statement it was done 20:16 after that nothing is happening but i ran few trace on that user.
/u01/app/oracle/admin/SURV2/udump/surv2_ora_10048.trc
Oracle Database 10g Release 10.2.0.4.0 - Production
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db
System name: SunOS
Node name: prdfa001
Release: 5.10
Version: Generic_139556-08
Machine: i86pc
Instance name: SURV2
Redo thread mounted by this instance: 1
Oracle process number: 18
Unix process pid: 10048, image: oracle@prdfa001
*** 2010-09-09 23:37:07.484
*** ACTION NAME:() 2010-09-09 23:37:07.473
*** MODULE NAME:(SQL*Plus) 2010-09-09 23:37:07.473
*** SERVICE NAME:(SURV2) 2010-09-09 23:37:07.473
*** SESSION ID:(289.54) 2010-09-09 23:37:07.473
Received ORADEBUG command 'unlimit' from process Unix process pid: 3983, image:
*** 2010-09-09 23:37:20.315
Received ORADEBUG command 'event 10046 trace name context forever, level 12' from process Unix process pid: 3983, image:
WAIT #7: nam='db file sequential read' ela= 11160 file#=13 block#=2252349 blocks=1 obj#=166421 tim=12499462835161
WAIT #7: nam='db file sequential read' ela= 2857 file#=13 block#=2249751 blocks=1 obj#=166421 tim=12499462838137
WAIT #7: nam='db file sequential read' ela= 3810 file#=13 block#=2251361 blocks=1 obj#=166421 tim=12499462842048
WAIT #7: nam='db file sequential read' ela= 4459 file#=13 block#=2247059 blocks=1 obj#=166421 tim=12499462846564
WAIT #7: nam='db file sequential read' ela= 2841 file#=13 block#=2247507 blocks=1 obj#=166421 tim=12499462849468
WAIT #7: nam='db file sequential read' ela= 427 file#=13 block#=2247568 blocks=1 obj#=166421 tim=12499462850032
WAIT #7: nam='db file sequential read' ela= 1187 file#=13 block#=2248264 blocks=1 obj#=166421 tim=12499462851327
WAIT #7: nam='db file sequential read' ela= 2687 file#=13 block#=2250707 blocks=1 obj#=166421 tim=12499462854178
WAIT #7: nam='db file sequential read' ela= 3657 file#=13 block#=2249697 blocks=1 obj#=166421 tim=12499462857896
WAIT #7: nam='db file sequential read' ela= 4139 file#=13 block#=2247074 blocks=1 obj#=166421 tim=12499462862093
WAIT #7: nam='db file sequential read' ela= 4180 file#=47 block#=3649690 blocks=1 obj#=166421 tim=12499509270445
WAIT #7: nam='db file sequential read' ela= 4802 file#=47 block#=3649309 blocks=1 obj#=166421 tim=12499509275327
WAIT #7: nam='db file sequential read' ela= 2459 file#=47 block#=3652697 blocks=1 obj#=166421 tim=12499509277859
WAIT #7: nam='db file sequential read' ela= 4015 file#=47 block#=3652826 blocks=1 obj#=166421 tim=12499509281948
WAIT #7: nam='db file sequential read' ela= 2248 file#=47 block#=3651610 blocks=1 obj#=166421 tim=12499509284269
WAIT #7: nam='db file sequential read' ela= 4824 file#=47 block#=3654297 blocks=1 obj#=166421 tim=12499509289166
WAIT #7: nam='db file sequential read' ela= 2008 file#=47 block#=3652312 blocks=1 obj#=166421 tim=12499509291248
WAIT #7: nam='db file sequential read' ela= 1925 file#=47 block#=3654490 blocks=1 obj#=166421 tim=12499509293246
WAIT #7: nam='db file sequential read' ela= 2859 file#=47 block#=3648458 blocks=1 obj#=166421 tim=12499509296178
WAIT #7: nam='db file sequential read' ela= 1740 file#=47 block#=3648212 blocks=1 obj#=166421 tim=12499509297991
WAIT #7: nam='db file sequential read' ela= 2566 file#=47 block#=3648411 blocks=1 obj#=166421 tim=12499509300631
WAIT #7: nam='db file sequential read' ela= 50772 file#=5 block#=480749 blocks=1 obj#=166421 tim=12499509351477
WAIT #7: nam='db file sequential read' ela= 12928 file#=5 block#=477177 blocks=1 obj#=166421 tim=12499509364482
WAIT #7: nam='db file sequential read' ela= 11116 file#=5 block#=479412 blocks=1 obj#=166421 tim=12499509375672
WAIT #7: nam='db file sequential read' ela= 4803 file#=5 block#=483440 blocks=1 obj#=166421 tim=12499509380549
WAIT #7: nam='db file sequential read' ela= 6900 file#=5 block#=481454 blocks=1 obj#=166421 tim=12499509387522
Received ORADEBUG command 'event 10046 trace name context off' from process Unix process pid: 3983, image:
/u01/app/oracle/admin/SURV2/udump/surv2_ora_1545.trc
Oracle Database 10g Release 10.2.0.4.0 - Production
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db
System name: SunOS
Node name: prdfa001
Release: 5.10
Version: Generic_139556-08
Machine: i86pc
Instance name: SURV2
Redo thread mounted by this instance: 1
Oracle process number: 22
Unix process pid: 1545, image: oracle@prdfa001 (TNS V1-V3)
*** ACTION NAME:() 2010-09-09 23:20:13.485
*** MODULE NAME:(sqlplus@prdfa001 (TNS V1-V3)) 2010-09-09 23:20:13.485
*** SERVICE NAME:(SYS$USERS) 2010-09-09 23:20:13.485
*** SESSION ID:(290.697) 2010-09-09 23:20:13.485
===================================================
SYSTEM STATE
System global information:
processes: base 47819b480, size 300, cleanup 4781a5638
allocation: free sessions 47f1d6148, free calls 0
control alloc errors: 0 (process), 0 (session), 0 (call)
PMON latch cleanup depth: 0
seconds since PMON's last scan for dead processes: 20
system statistics:
1171 logons cumulative
19 logons current
89219 opened cursors cumulative
86 opened cursors current
15095069 user commits
5 user rollbacks
58632904 user calls
44023255 recursive calls
224311 recursive cpu usage
201424173 session logical reads
0 session stored procedure space
901812 CPU used when call started
995437 CPU used by this session
6814196 DB time
0 cluster wait time
22542300822 concurrency wait time
3095 application wait time
16479074661 user I/O wait time
1284052668 session connect time
1284067190 process last non-idle time
189018343568 session uga memory
1249667216 session uga memory max
26059216 messages sent
26059220 messages received
239739 background timeouts
162399896 session pga memory
189662872 session pga memory max
4 enqueue timeouts
901146 enqueue waits
0 enqueue deadlocks
32122711 enqueue requests
17819 enqueue conversions
32122676 enqueue releases
0 global enqueue gets sync
0 global enqueue gets async
0 global enqueue get time
0 global enqueue releases
2865667 physical read total IO requests
262620 physical read total multi block requests
270093476864 physical read total bytes
select SYS_CONTEXT('USERENV', 'SERVER_HOST'), SYS_CONTEXT('USERENV', 'DB_UNIQUE_NAME'), SYS_CONTEXT('USERENV', 'INSTANCE_NAME'), SYS_CONTEXT('USERENV', 'SERVICE_NAME'), INSTANCE_NUMBER, STARTUP_TIME, SYS_CONTEXT('USERENV', 'DB_DOMAIN') from v$instance where INSTANCE_NAME=SYS_CONTEXT('USERENV', 'INSTANCE_NAME')
hash=550c95f3d0cfa8290e60ea8382d3a2ca timestamp=09-09-2010 04:24:19
namespace=CRSR flags=RON/KGHP/TIM/PN0/LRG/KST/DBN/MTX/[100100d1]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=9 hpc=0582 hlc=0582
lwt=47df576e8[47df576e8,47df576e8] ltm=47df576f8[47df576f8,47df576f8]
pwt=47df576b0[47df576b0,47df576b0] ptm=47df576c0[47df576c0,47df576c0]
ref=47df57718[47df57718,47df57718] lnd=47df57730[47df57730,47df57730]
LIBRARY OBJECT: object=471ee1d38
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
CHILDREN: size=16
child# table reference handle
0 471ee1800 471ee1470 47df7dce0
DATA BLOCKS:
data# heap pointer status pins change whr
0 47df7de48 471ee1e50 I/P/A/-/- 0 NONE 00
SO: 473691d60, type: 53, owner: 47924e810, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=473691d60 handle=47bb22fa0 mode=N
call pin=0 session pin=0 hpc=0000 hlc=0000
htl=473691de0[4735dbcb8,476cfbf58] htb=476cfbf58 ssga=476cfb6a0
user=47924e810 session=47f2310f0 count=1 flags=[0000] savepoint=0x0
LIBRARY OBJECT HANDLE: handle=47bb22fa0 mtx=47bb230d0(0) cdp=0
namespace=CRSR flags=RON/KGHP/PN0/EXP/[10010100]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=3 hpc=fd84 hlc=fd84
lwt=47bb23048[47bb23048,47bb23048] ltm=47bb23058[47bb23058,47bb23058]
pwt=47bb23010[47bb23010,47bb23010] ptm=47bb23020[47bb23020,47bb23020]
ref=47bb23078[472f8de18,472f8de18] lnd=47bb23090[47bb23090,47bb23090]
LIBRARY OBJECT: object=472f8d9d8
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
DEPENDENCIES: count=1 size=16
AUTHORIZATIONS: count=1 size=16 minimum entrysize=16
ACCESSES: count=1 size=16
TRANSLATIONS: count=1 size=16
DATA BLOCKS:
data# heap pointer status pins change whr
0 47bb22ee0 472f8daf0 I/P/A/-/- 0 NONE 00
6 472f8e508 46be86250 I/-/A/-/E 0 NONE 00
SO: 4735dbc38, type: 53, owner: 47924e810, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=4735dbc38 handle=47bb231c8 mode=N
call pin=0 session pin=0 hpc=0000 hlc=0000
htl=4735dbcb8[476cfbf58,473691de0] htb=476cfbf58 ssga=476cfb6a0
user=47924e810 session=47f2310f0 count=1 flags=[0000] savepoint=0x4c894f8b
LIBRARY OBJECT HANDLE: handle=47bb231c8 mtx=47bb232f8(1) cdp=1
name=select value$ from props$ where name = 'GLOBAL_DB_NAME'
hash=4bb432d65c5a391a42a5c3fa74472c7a timestamp=09-09-2010 04:24:12
namespace=CRSR flags=RON/KGHP/TIM/PN0/SML/KST/DBN/MTX/[120100d0]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=3 hpc=0584 hlc=0584
lwt=47bb23270[47bb23270,47bb23270] ltm=47bb23280[47bb23280,47bb23280]
pwt=47bb23238[47bb23238,47bb23238] ptm=47bb23248[47bb23248,47bb23248]
ref=47bb232a0[47bb232a0,47bb232a0] lnd=47bb232b8[47bb232b8,47bb232b8]
LIBRARY OBJECT: object=472f8e6e0
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
CHILDREN: size=16
child# table reference handle
0 472f8e1a8 472f8de18 47bb22fa0
DATA BLOCKS:
data# heap pointer status pins change whr
0 47bb23108 472f8e7f8 I/P/A/-/- 0 NONE 00
SO: 473644348, type: 53, owner: 47924e810, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=473644348 handle=47bbde418 mode=N
call pin=0 session pin=0 hpc=0000 hlc=0000
htl=4736443c8[476cfc0b8,476cfc0b8] htb=476cfc0b8 ssga=476cfb6a0
user=47924e810 session=47924e810 count=1 flags=[0000] savepoint=0x4c894f8b
LIBRARY OBJECT HANDLE: handle=47bbde418 mtx=47bbde548(0) cdp=0
name=ALTER SESSION SET TIME_ZONE='+02:00'
hash=3878dff8839e71e3dd05a2e75fbd6390 timestamp=09-09-2010 04:24:04
namespace=CRSR flags=RON/KGHP/TIM/PN0/SML/DBN/[12010040]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch#=11 hpc=04e8 hlc=04e8
lwt=47bbde4c0[47bbde4c0,47bbde4c0] ltm=47bbde4d0[47bbde4d0,47bbde4d0]
pwt=47bbde488[47bbde488,47bbde488] ptm=47bbde498[47bbde498,47bbde498]
ref=47bbde4f0[47bbde4f0,47bbde4f0] lnd=47bbde508[47bbde508,47bbde508]
LIBRARY OBJECT: object=472fffc08
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change whr
0 47bbde320 472fffd20 I/P/A/-/- 0 NONE 00
SO: 47aecf9e8, type: 41, owner: 47924e810, flag: INIT/-/-/0x00
(dummy) nxc=0, nlb=0
SO: 47f290540, type: 11, owner: 4781a7dc0, flag: INIT/-/-/0x00
(broadcast handle) flag: (2) ACTIVE SUBSCRIBER, owner: 4781a7dc0,
event: 1132, last message event: 1132,
last message waited event: 1132, next message: 0(0), messages read: 0
channel: (47a2df4f8) system events broadcast channel
scope: 2, event: 1132, last mesage event: 18,
publishers/subscribers: 0/17,
messages published: 1
SO: 47826b228, type: 3, owner: 4781a7dc0, flag: INIT/-/-/0x00
(call) sess: cur 47924e810, rec 0, usr 47924e810; depth: 0
SO: 476c52968, type: 16, owner: 4781a7dc0, flag: INIT/-/-/0x00
(osp req holder)
PSEUDO PROCESS for group DEFAULT:
SO: 47a1eb7d0, type: 2, owner: 0, flag: INIT/-/-/0x00
(process) Oracle pid=0, calls cur/top: 0/0, flag: (20) PSEUDO
int error: 0, call error: 0, sess error: 0, txn error 0
(post info) last post received: 0 0 0
last post received-location: No post
last process to post me: none
last post sent: 0 0 0
last post sent-location: No post
last process posted by me: none
(latch info) wait_event=0 bits=0
Process Group: DEFAULT, pseudo proc: 47a1eb7d0
O/S info: user: , term: , ospid: (DEAD)
OSD pid info: Unix process pid: 0, image: PSEUDO
Dump of memory from 0x00000004791BF538 to 0x00000004791BF740
4791BF530 00000000 00000000 [........]
4791BF540 00000000 00000000 00000000 00000000 [................]
Repeat 31 times
NO DETACHED BRANCHES.
NO DETACHED NETWORK CONNECTIONS.
CLEANUP STATE OBJECTS:
SO: 47f0cd038, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: instance enqueue anchor state
latch: 0x380009890
SO: 4782cf080, type: 5, owner: 47f0cd038, flag: INIT/-/-/0x00
(enqueue) TA-00000006-00000001 DID: 0001-000F-0000000B
lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 res_flag: 0x2
res: 0x47a28d020, mode: X, lock_flag: 0x0
own: 0x0, sess: 0x0, prv: 0x47a28d030
SO: 47f0cd098, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: switchable channel handle anch
latch: 0x38000ac98
SO: 47f28f868, type: 11, owner: 47f0cd098, flag: INIT/-/-/0x00
(broadcast handle) flag: (c2) ACTIVE SUBSCRIBER, owner: 0,
event: 1, last message event: 1,
last message waited event: 1, next message: 0(0), messages read: 0
channel: (47a2e4190) KPON channel
scope: 2, event: 1, last mesage event: 0,
publishers/subscribers: 0/1,
messages published: 0
SO: 47f0cd0f8, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: TT shared object cleanup SO
latch: 0x38001c6b8
SO: 47f0cd158, type: 1, owner: 0, flag: INIT/-/-/0x00
(cleanup state object) description: SS shared object cleanup SO
latch: 0x38001cd48
END OF SYSTEM STATE
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
db file sequential read 2,347,652 9,215 4 64.5 User I/O
db file scattered read 245,687 4,199 17 29.4 User I/O
CPU time 974 6.8
db file parallel write 50,082 408 8 2.9 System I/O
log file parallel write 6,963 52 7 0.4 System I/O
Time Model Statistics DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> Total time in database user-calls (DB Time): 14286.4s
-> Statistics including the word "background" measure background process
time, and so do not contribute to the DB time statistic
-> Ordered by % or DB time desc, Statistic name
Statistic Name Time (s) % of DB Time
sql execute elapsed time 14,280.3 100.0
DB CPU 974.5 6.8
PL/SQL execution elapsed time 531.8 3.7
parse time elapsed 30.5 .2
hard parse elapsed time 27.1 .2
connection management call elapsed time 14.9 .1
hard parse (sharing criteria) elapsed time 3.4 .0
hard parse (bind mismatch) elapsed time 3.1 .0
PL/SQL compilation elapsed time 2.4 .0
failed parse elapsed time 0.0 .0
repeated bind elapsed time 0.0 .0
sequence load elapsed time 0.0 .0
DB time 14,286.4 N/A
background elapsed time 670.2 N/A
background cpu time 186.1 N/A
Wait Class DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc
Avg
%Time Total Wait wait Waits
Wait Class Waits -outs Time (s) (ms) /txn
User I/O 2,593,484 .0 13,415 5 150.0
System I/O 87,506 .0 515 6 5.1
Other 839 11.4 6 7 0.0
Commit 3,225 .1 6 2 0.2
Concurrency 1,033 .0 5 5 0.1
Configuration 2,514 99.4 0 0 0.1
Network 47,559 .0 0 0 2.8
Application 7 .0 0 0 0.0
Wait Events DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
db file sequential read 2,347,652 .0 9,215 4 135.8
db file scattered read 245,687 .0 4,199 17 14.2
db file parallel write 50,082 .0 408 8 2.9
log file parallel write 6,963 .0 52 7 0.4
control file parallel write 6,203 .0 44 7 0.4
control file sequential read 24,242 .0 11 0 1.4
log file sync 3,225 .1 6 2 0.2
latch free 84 .0 4 47 0.0
os thread startup 25 .0 3 120 0.0
latch: session allocation 39 .0 1 33 0.0
db file parallel read 12 .0 1 92 0.0
enq: TX - index contention 186 .0 1 3 0.0
latch: shared pool 47 .0 1 11 0.0
LGWR wait for redo copy 319 3.1 0 1 0.0
library cache load lock 2 .0 0 172 0.0
buffer busy waits 590 .0 0 0 0.0
log file switch completion 6 .0 0 29 0.0
SGA: allocation forcing comp 11 54.5 0 14 0.0
latch: library cache lock 50 .0 0 3 0.0
read by other session 38 .0 0 4 0.0
direct path read 42 .0 0 3 0.0
SQL*Net message to client 44,807 .0 0 0 2.6
rdbms ipc reply 207 .0 0 0 0.0
SQL*Net more data from clien 1,014 .0 0 0 0.1
latch: cache buffers chains 24 .0 0 1 0.0
latch: library cache 29 .0 0 1 0.0
log file sequential read 8 .0 0 3 0.0
direct path write 50 .0 0 0 0.0
SQL*Net more data to client 398 .0 0 0 0.0
latch: object queue header o 12 .0 0 1 0.0
latch: In memory undo latch 78 .0 0 0 0.0
undo segment extension 2,507 99.7 0 0 0.1
latch: cache buffers lru cha 4 .0 0 1 0.0
log file single write 8 .0 0 0 0.0
local write wait 3 .0 0 1 0.0
enq: RO - fast object reuse 3 .0 0 1 0.0
buffer deadlock 87 92.0 0 0 0.0
enq: JS - queue lock 1 .0 0 1 0.0
cursor: pin S 70 .0 0 0 0.0
latch: row cache objects 2 .0 0 1 0.0
SQL*Net message to dblink 1,338 .0 0 0 0.1
latch: checkpoint queue latc 2 .0 0 0 0.0
reliable message 3 .0 0 0 0.0
log buffer space 1 .0 0 1 0.0
SQL*Net break/reset to clien 4 .0 0 0 0.0
SQL*Net more data from dblin 2 .0 0 0 0.0
SQL*Net message from client 44,949 .0 155,701 3464 2.6
virtual circuit status 621 100.0 18,156 29237 0.0
Streams AQ: qmn slave idle w 664 .0 18,127 27299 0.0
Streams AQ: qmn coordinator 1,339 50.4 18,099 13517 0.1
Streams AQ: waiting for time 12 100.0 8,741 728394 0.0
jobq slave wait 130 100.0 380 2927 0.0
PL/SQL lock timer 1 100.0 1 978 0.0
SQL*Net message from dblink 1,338 .0 0 0 0.1
single-task message 1 .0 0 38 0.0
class slave wait 11 .0 0 1 0.0
SQL ordered by Elapsed Time DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> Resources reported for PL/SQL code includes the resources used by all SQL
statements called by the code.
-> % Total DB Time is the Elapsed Time of the SQL statement divided
into the Total Database Time multiplied by 100
Elapsed CPU Elap per % Total
Time (s) Time (s) Executions Exec (s) DB Time SQL Id
13,664 906 0 N/A 95.6 gr2cx6athc5j5
Module: SQL*Plus
BEGIN DBMS_OUTPUT.PUT_LINE(equiduct.eod(NULL,NULL)); END;
8,792 195 0 N/A 61.5 986fzxtzr52u5
Module: SQL*Plus
UPDATE TIBEX_ORDER SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"SYS_B_1"
2,524 368 1 2524.1 17.7 c4uf0x6hdgnwq
Module: SQL*Plus
UPDATE TIBEX_FIXSESSIONSTATE SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"
SYS_B_1"
1,414 177 1 1414.4 9.9 cbg09ma34kq8w
Module: SQL*Plus
SELECT count(*) FROM TIBEX_ORDER WHERE INSTRUMENTID=:"SYS_B_0"
742 137 1 742.2 5.2 g0sg6v994wssq
Module: SQL*Plus
SELECT count(*) FROM TIBEX_FIXSESSIONSTATE WHERE INSTRUMENTID=:"SYS_B_0"
274 11 1 274.2 1.9 6mcpb06rctk0x
Module: DBMS_SCHEDULER
call dbms_space.auto_space_advisor_job_proc ( )
264 8 27 9.8 1.8 8szmwam7fysa3
Module: DBMS_SCHEDULER
insert into wri$_adv_objspace_trend_data select timepoint, space_usage, space_a
lloc, quality from table(dbms_space.object_growth_trend(:1, :2, :3, :4, NULL, N
ULL, NULL, 'FALSE', :5, 'FALSE'))
99 1 1 99.4 0.7 1z0x41f66nvjr
Module: SQL*Plus
UPDATE TIBEX_INSTRUMENTADMIN SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"
SYS_B_1"
21 10 1 21.5 0.2 bbc1ck8594kvj
Module: SQL*Plus
UPDATE TIBEX_INSTRUMENTDAILYHIST SET ADJOPEN=NVL(ADJOPEN,OPEN), ADJHIGH=NVL(ADJH
IGH,HIGH), ADJLOW=NVL(ADJLOW,LOW), ADJMID=NVL(ADJMID,MID), ADJCLOSE=NVL(ADJCLOSE
,CLOSE), ADJVOLUME=NVL(ADJVOLUME,VOLUME), ADJCLOSINGBID=NVL(ADJCLOSINGBID,CLOSIN
GBID), ADJCLOSINGOFFER=NVL(ADJCLOSINGOFFER,CLOSINGOFFER)
12 0 1 12.5 0.1 6xm9p9uy5kaap
Module: SQL*Plus
SELECT count(*) FROM TIBEX_INSTRUMENTSTATE WHERE INSTRUMENTID=:"SYS_B_0"
SQL ordered by CPU Time DB/Inst: SURV2/SURV2 Snaps: 19172-19178
-> Resources reported for PL/SQL code includes the resources used by all SQL
statements called by the code.
-> % Total DB Time is the Elapsed Time of the SQL statement divided
into the Total Database Time multiplied by 100
CPU Elapsed CPU per % Total
Time (s) Time (s) Executions Exec (s) DB Time SQL Id
906 13,664 0 N/A 95.6 gr2cx6athc5j5
Module: SQL*Plus
BEGIN DBMS_OUTPUT.PUT_LINE(equiduct.eod(NULL,NULL)); END;
368 2,524 1 367.51 17.7 c4uf0x6hdgnwq
Module: SQL*Plus
UPDATE TIBEX_FIXSESSIONSTATE SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"
SYS_B_1"
195 8,792 0 N/A 61.5 986fzxtzr52u5
Module: SQL*Plus
UPDATE TIBEX_ORDER SET INSTRUMENTID=:"SYS_B_0" WHERE INSTRUMENTID=:"SYS_B_1"
177 1,414 1 176.93 9.9 cbg09ma34kq8w
Module: SQL*Plus
SELECT count(*) FROM TIBEX_ORDER WHERE INSTRUMENTID=:"SYS_B_0"
137 742 1 137.38 5.2 g0sg6v994wssq
Module: SQL*Plus
SELECT count(*) FROM TIBEX_FIXSESSIONSTATE WHERE INSTRUMENTID=:"SYS_B_0"
11 274 1 10.82 1.9 6mcpb06rctk0x
Module: DBMS_SCHEDULER
call dbms_space.auto_space_advisor_job_proc ( )
10 21 1 9.65 0.2 bbc1ck8594kvjEdited by: NM on 10-Sep-2010 07:39 -
Open_cursors and session_cached_cursors!!
Hi, all.
I have a 2 node RAC database (10.2.0.2.0) on windows2003 EE SP1.
Recently, I am getting warnings related to "library cache lock" and "cursor: pin S wait on X" wait event.
The recommendation from ADDM findigs is as follows.
-- increase open_cursors
-- increase session_cached_cursors
The above parameters are dynamic ?? or do I need to restart the instances??
Thanks and Regards.set linesize 121
SELECT name, isses_modifiable, issys_modifiable
FROM gv$parameter
WHERE ....; -
Performance issue showing read by other session Event
Hi All,
we are having a severe performance issue in my database when we are running batch jobs.
This was a new database(11.2.0.2) and we are testing the performance by running some batch jobs. These batch jobs included some inserts and updates.
I am seeing read by other session in top 5 timed events and cache buffers chains in Latch Miss Sources section.
Please help me to solve this out.
Inst Num Startup Time Release RAC
1 27-Feb-12 09:03 11.2.0.2.0 NO
Platform CPUs Cores Sockets Memory(GB)
Linux x86 64-bit 8 8 8 48.00
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 5605 29-Feb-12 03:00:27 63 4.5
End Snap: 5614 29-Feb-12 12:00:47 63 4.3
Elapsed: 540.32 (mins)
DB Time: 1,774.23 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 1,952M 1,952M Std Block Size: 16K
Shared Pool Size: 1,024M 1,024M Log Buffer: 18,868K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ --------------- --------------- ---------- ----------
DB Time(s): 3.3 0.8 0.02 0.05
DB CPU(s): 1.1 0.3 0.01 0.02
Redo size: 55,763.8 13,849.3
Logical reads: 23,906.6 5,937.4
Block changes: 325.7 80.9
Physical reads: 665.6 165.3
Physical writes: 40.4 10.0
User calls: 60.7 15.1
Parses: 10.6 2.6
Hard parses: 1.1 0.3
W/A MB processed: 0.6 0.2
Logons: 0.1 0.0
Executes: 151.2 37.6
Rollbacks: 0.0 0.0
Transactions: 4.0
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.94 Redo NoWait %: 100.00
Buffer Hit %: 97.90 In-memory Sort %: 100.00
Library Hit %: 98.06 Soft Parse %: 90.16
Execute to Parse %: 92.96 Latch Hit %: 100.00
Parse CPU to Parse Elapsd %: 76.71 % Non-Parse CPU: 98.57
Shared Pool Statistics Begin End
Memory Usage %: 89.38 87.96
% SQL with executions>1: 97.14 95.15
% Memory for SQL w/exec>1: 96.05 92.46
Top 5 Timed Foreground Events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Avg
wait % DB
Event Waits Time(s) (ms) time Wait Class
db file sequential read 14,092,706 65,613 5 61.6 User I/O
DB CPU 34,819 32.7
read by other session 308,534 1,260 4 1.2 User I/O
direct path read 97,454 987 10 .9 User I/O
db file scattered read 71,870 910 13 .9 User I/O
Host CPU (CPUs: 8 Cores: 8 Sockets: 8)
~~~~~~~~ Load Average
Begin End %User %System %WIO %Idle
0.43 0.36 13.7 0.6 9.7 85.7
Instance CPU
~~~~~~~~~~~~
% of total CPU for Instance: 13.5
% of busy CPU for Instance: 94.2
%DB time waiting for CPU - Resource Mgr: 0.0
Memory Statistics
~~~~~~~~~~~~~~~~~ Begin End
Host Mem (MB): 49,152.0 49,152.0
SGA use (MB): 3,072.0 3,072.0
PGA use (MB): 506.5 629.1
% Host Mem used for SGA+PGA: 7.28 7.53
Time Model Statistics
-> Total time in database user-calls (DB Time): 106453.8s
-> Statistics including the word "background" measure background process
time, and so do not contribute to the DB time statistic
-> Ordered by % or DB time desc, Statistic name
Statistic Name Time (s) % of DB Time
sql execute elapsed time 105,531.1 99.1
DB CPU 34,818.8 32.7
parse time elapsed 714.7 .7
hard parse elapsed time 684.8 .6
PL/SQL execution elapsed time 161.9 .2
PL/SQL compilation elapsed time 44.2 .0
connection management call elapsed time 16.9 .0
hard parse (sharing criteria) elapsed time 10.2 .0
hard parse (bind mismatch) elapsed time 9.4 .0
sequence load elapsed time 2.9 .0
repeated bind elapsed time 0.5 .0
failed parse elapsed time 0.0 .0
DB time 106,453.8
background elapsed time 1,753.9
background cpu time 61.7
Operating System Statistics
-> *TIME statistic values are diffed.
All others display actual values. End Value is displayed if different
-> ordered by statistic type (CPU Use, Virtual Memory, Hardware Config), Name
Statistic Value End Value
BUSY_TIME 3,704,415
IDLE_TIME 22,203,740
IOWAIT_TIME 2,517,864
NICE_TIME 3
SYS_TIME 145,696
USER_TIME 3,557,758
LOAD 0 0
RSRC_MGR_CPU_WAIT_TIME 0
VM_IN_BYTES 358,813,045,760
VM_OUT_BYTES 29,514,830,848
PHYSICAL_MEMORY_BYTES 51,539,607,552
NUM_CPUS 8
NUM_CPU_CORES 8
NUM_CPU_SOCKETS 8
GLOBAL_RECEIVE_SIZE_MAX 4,194,304
GLOBAL_SEND_SIZE_MAX 1,048,586
TCP_RECEIVE_SIZE_DEFAULT 87,380
TCP_RECEIVE_SIZE_MAX 4,194,304
TCP_RECEIVE_SIZE_MIN 4,096
TCP_SEND_SIZE_DEFAULT 16,384
TCP_SEND_SIZE_MAX 4,194,304
TCP_SEND_SIZE_MIN 4,096
Operating System Statistics -
Snap Time Load %busy %user %sys %idle %iowait
29-Feb 03:00:27 0.4 N/A N/A N/A N/A N/A
29-Feb 04:00:35 1.4 11.9 11.2 0.6 88.1 14.3
29-Feb 05:00:41 1.7 13.8 13.2 0.6 86.2 15.8
29-Feb 06:00:48 1.5 14.0 13.5 0.6 86.0 12.3
29-Feb 07:01:00 1.8 16.3 15.8 0.5 83.7 10.4
29-Feb 08:00:12 2.6 23.2 22.5 0.6 76.8 12.6
29-Feb 09:00:26 1.3 16.6 16.0 0.5 83.4 5.7
29-Feb 10:00:33 1.2 13.8 13.3 0.5 86.2 2.0
29-Feb 11:00:43 1.3 14.5 14.0 0.5 85.5 3.8
29-Feb 12:00:47 0.4 4.9 4.2 0.7 95.1 10.6
Foreground Wait Class
-> s - second, ms - millisecond - 1000th of a second
-> ordered by wait time desc, waits desc
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
-> Captured Time accounts for 97.9% of Total DB time 106,453.79 (s)
-> Total FG Wait Time: 69,415.64 (s) DB CPU time: 34,818.79 (s)
Avg
%Time Total Wait wait
Wait Class Waits -outs Time (s) (ms) %DB time
User I/O 14,693,843 0 69,222 5 65.0
DB CPU 34,819 32.7
Commit 40,629 0 119 3 0.1
System I/O 26,504 0 57 2 0.1
Network 1,945,010 0 11 0 0.0
Other 125,200 99 4 0 0.0
Application 2,673 0 2 1 0.0
Concurrency 3,059 0 1 0 0.0
Configuration 31 19 0 15 0.0
Foreground Wait Events
-> s - second, ms - millisecond - 1000th of a second
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by wait time desc, waits desc (idle events last)
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Avg
%Time Total Wait wait Waits % DB
Event Waits -outs Time (s) (ms) /txn time
db file sequential read 14,092,706 0 65,613 5 108.0 61.6
read by other session 308,534 0 1,260 4 2.4 1.2
direct path read 97,454 0 987 10 0.7 .9
db file scattered read 71,870 0 910 13 0.6 .9
db file parallel read 35,001 0 372 11 0.3 .3
log file sync 40,629 0 119 3 0.3 .1
control file sequential re 26,504 0 57 2 0.2 .1
direct path read temp 14,499 0 49 3 0.1 .0
direct path write temp 9,186 0 28 3 0.1 .0
SQL*Net message to client 1,923,973 0 5 0 14.7 .0
SQL*Net message from dblin 1,056 0 5 5 0.0 .0
Disk file operations I/O 8,848 0 2 0 0.1 .0
ASM file metadata operatio 36 0 2 54 0.0 .0
SQL*Net break/reset to cli 2,636 0 1 1 0.0 .0
ADR block file read 472 0 1 1 0.0 .0
os thread startup 8 0 1 74 0.0 .0
SQL*Net more data to clien 17,656 0 1 0 0.1 .0
asynch descriptor resize 123,852 100 0 0 0.9 .0
local write wait 110 0 0 4 0.0 .0
utl_file I/O 55,635 0 0 0 0.4 .0
log file switch (private s 8 0 0 52 0.0 .0
cursor: pin S wait on X 2 0 0 142 0.0 .0
enq: KO - fast object chec 13 0 0 20 0.0 .0
PX Deq: Slave Session Stat 248 0 0 1 0.0 .0
enq: RO - fast object reus 18 0 0 11 0.0 .0
latch: cache buffers chain 2,511 0 0 0 0.0 .0
latch: shared pool 195 0 0 1 0.0 .0
CSS initialization 12 0 0 8 0.0 .0
PX qref latch 54 100 0 2 0.0 .0
SQL*Net more data from cli 995 0 0 0 0.0 .0
SQL*Net more data from dbl 300 0 0 0 0.0 .0
kksfbc child completion 1 100 0 56 0.0 .0
library cache: mutex X 244 0 0 0 0.0 .0
PX Deq: Signal ACK RSG 124 0 0 0 0.0 .0
undo segment extension 6 100 0 7 0.0 .0
PX Deq: Signal ACK EXT 124 0 0 0 0.0 .0
library cache load lock 3 0 0 9 0.0 .0
ADR block file write 45 0 0 1 0.0 .0
CSS operation: action 12 0 0 2 0.0 .0
reliable message 28 0 0 1 0.0 .0
CSS operation: query 72 0 0 0 0.0 .0
latch: row cache objects 14 0 0 1 0.0 .0
enq: SQ - contention 17 0 0 0 0.0 .0
latch free 32 0 0 0 0.0 .0
buffer busy waits 52 0 0 0 0.0 .0
enq: PS - contention 16 0 0 0 0.0 .0
enq: TX - row lock content 6 0 0 1 0.0 .0
SQL*Net message to dblink 1,018 0 0 0 0.0 .0
cursor: pin S 23 0 0 0 0.0 .0
latch: cache buffers lru c 8 0 0 0 0.0 .0
SQL*Net message from clien 1,923,970 0 944,508 491 14.7
jobq slave wait 66,732 100 33,334 500 0.5
Streams AQ: waiting for me 6,481 100 32,412 5001 0.0
wait for unread message on 32,858 98 32,411 986 0.3
PX Deq: Execution Msg 1,448 0 190 131 0.0
PX Deq: Execute Reply 1,196 0 74 62 0.0
HS message to agent 228 0 4 19 0.0
single-task message 42 0 4 97 0.0
PX Deq Credit: send blkd 904 0 2 3 0.0
PX Deq Credit: need buffer 205 0 1 3 0.0
Foreground Wait Events
-> s - second, ms - millisecond - 1000th of a second
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by wait time desc, waits desc (idle events last)
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Avg
%Time Total Wait wait Waits % DB
Event Waits -outs Time (s) (ms) /txn time
PX Deq: Table Q Normal 4,291 0 1 0 0.0
PX Deq: Join ACK 124 0 0 1 0.0
PX Deq: Parse Reply 124 0 0 0 0.0
KSV master wait 256 0 0 0 0.0
Latch Miss Sources
-> only latches with sleeps are shown
-> ordered by name, sleeps desc
NoWait Waiter
Latch Name Where Misses Sleeps Sleeps
ASM map operation freeli kffmTranslate2 0 2 0
DML lock allocation ktadmc 0 2 0
FOB s.o list latch ksfd_allfob 0 2 2
In memory undo latch ktiFlushMe 0 5 0
In memory undo latch ktichg: child 0 3 0
PC and Classifier lists No latch 0 6 0
Real-time plan statistic keswxAddNewPlanEntry 0 20 20
SQL memory manager worka qesmmIRegisterWorkArea:1 0 1 1
active service list kswslogon: session logout 0 23 12
active service list kswssetsvc: PX session swi 0 6 1
active service list kswsite: service iterator 0 1 0
archive process latch kcrrgpll 0 3 3
cache buffers chains kcbgtcr_2 0 1,746 573
cache buffers chains kcbgtcr: fast path (cr pin 0 1,024 2,126
cache buffers chains kcbgcur_2 0 60 8
cache buffers chains kcbchg1: kslbegin: bufs no 0 16 3
cache buffers chains kcbgtcr: fast path 0 14 20
cache buffers chains kcbzibmlt: multi-block rea 0 10 0
cache buffers chains kcbrls_2 0 9 53
cache buffers chains kcbgtcr: kslbegin shared 0 8 1
cache buffers chains kcbrls_1 0 7 84
cache buffers chains kcbgtcr: kslbegin excl 0 6 14
cache buffers chains kcbnew: new latch again 0 6 0
cache buffers chains kcbzgb: scan from tail. no 0 6 0
cache buffers chains kcbzwb 0 5 8
cache buffers chains kcbgcur: fast path (shr) 0 3 0
cache buffers chains kcbget: pin buffer 0 3 0
cache buffers chains kcbzhngcbk2_1 0 1 0
cache buffers lru chain kcbzgws 0 19 0
cache buffers lru chain kcbo_link_q 0 3 0
call allocation ksuxds 0 14 10
call allocation ksudlp: top call 0 2 3
enqueue hash chains ksqgtl3 0 2 1
enqueue hash chains ksqrcl 0 1 2
enqueues ksqgel: create enqueue 0 1 0
object queue header oper kcbo_unlink_q 0 5 2
object queue header oper kcbo_sw_buf 0 2 0
object queue header oper kcbo_link_q 0 1 2
object queue header oper kcbo_switch_cq 0 1 2
object queue header oper kcbo_switch_mq_bg 0 1 4
parallel query alloc buf kxfpbalo 0 1 1
process allocation ksucrp:1 0 2 0
process queue reference kxfpqrsnd 0 1 0
qmn task queue latch kwqmnmvtsks: delay to read 0 1 0
redo allocation kcrfw_redo_gen: redo alloc 0 17 0
row cache objects kqreqd: reget 0 6 0
row cache objects kqrpre: find obj 0 6 13
row cache objects kqrso 0 2 0
row cache objects kqreqd 0 1 2
row cache objects kqrpre: init complete 0 1 1
shared pool kghalo 0 199 106
shared pool kghupr1 0 39 109
shared pool kghfre 0 18 19
shared pool kghalp 0 7 29
space background task la ktsj_grab_task 0 21 27
Mutex Sleep Summary
-> ordered by number of sleeps desc
Wait
Mutex Type Location Sleeps Time (ms)
Library Cache kglhdgn2 106 338 12
Library Cache kgllkc1 57 259 10
Library Cache kgllkdl1 85 123 21
Cursor Pin kkslce [KKSCHLPIN2] 70 286
Library Cache kglget2 2 31 1
Library Cache kglhdgn1 62 31 2
Library Cache kglpin1 4 26 1
Library Cache kglpnal1 90 18 0
Library Cache kglpndl1 95 15 2
Library Cache kgllldl2 112 6 0
Library Cache kglini1 32 1 0
-------------------------------------------------------------Thanks in advance.Hi,
Thanks for reply.
I provided one hour report.
Inst Num Startup Time Release RAC
1 27-Feb-12 09:03 11.2.0.2.0 NO
Platform CPUs Cores Sockets Memory(GB)
Linux x86 64-bit 8 8 8 48.00
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 5606 29-Feb-12 04:00:35 63 3.7
End Snap: 5607 29-Feb-12 05:00:41 63 3.6
Elapsed: 60.11 (mins)
DB Time: 382.67 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 1,952M 1,952M Std Block Size: 16K
Shared Pool Size: 1,024M 1,024M Log Buffer: 18,868K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ --------------- --------------- ---------- ----------
DB Time(s): 6.4 0.8 0.03 0.03
DB CPU(s): 1.0 0.1 0.00 0.00
Redo size: 84,539.3 10,425.6
Logical reads: 23,345.6 2,879.1
Block changes: 386.5 47.7
Physical reads: 1,605.0 197.9
Physical writes: 7.1 0.9
User calls: 233.9 28.9
Parses: 4.0 0.5
Hard parses: 0.1 0.0
W/A MB processed: 0.1 0.0
Logons: 0.1 0.0
Executes: 210.9 26.0
Rollbacks: 0.0 0.0
Transactions: 8.1
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.62 Redo NoWait %: 100.00
Buffer Hit %: 95.57 In-memory Sort %: 100.00
Library Hit %: 99.90 Soft Parse %: 98.68
Execute to Parse %: 98.10 Latch Hit %: 99.99
Parse CPU to Parse Elapsd %: 32.08 % Non-Parse CPU: 99.90
Shared Pool Statistics Begin End
Memory Usage %: 89.25 89.45
% SQL with executions>1: 96.79 97.52
% Memory for SQL w/exec>1: 95.67 96.56
Top 5 Timed Foreground Events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Avg
wait % DB
Event Waits Time(s) (ms) time Wait Class
db file sequential read 3,054,464 17,002 6 74.0 User I/O
DB CPU 3,748 16.3
read by other session 199,603 796 4 3.5 User I/O
direct path read 46,301 439 9 1.9 User I/O
db file scattered read 21,113 269 13 1.2 User I/O
Host CPU (CPUs: 8 Cores: 8 Sockets: 8)
~~~~~~~~ Load Average
Begin End %User %System %WIO %Idle
1.45 1.67 13.2 0.6 15.8 86.2
Instance CPU
~~~~~~~~~~~~
% of total CPU for Instance: 13.0
% of busy CPU for Instance: 94.7
%DB time waiting for CPU - Resource Mgr: 0.0
Memory Statistics
~~~~~~~~~~~~~~~~~ Begin End
Host Mem (MB): 49,152.0 49,152.0
SGA use (MB): 3,072.0 3,072.0
PGA use (MB): 513.5 467.7
% Host Mem used for SGA+PGA: 7.29 7.20
Time Model Statistics
-> Total time in database user-calls (DB Time): 22960.5s
-> Statistics including the word "background" measure background process
time, and so do not contribute to the DB time statistic
-> Ordered by % or DB time desc, Statistic name
Statistic Name Time (s) % of DB Time
sql execute elapsed time 22,835.9 99.5
DB CPU 3,748.4 16.3
parse time elapsed 15.4 .1
hard parse elapsed time 14.3 .1
PL/SQL execution elapsed time 7.5 .0
PL/SQL compilation elapsed time 6.0 .0
connection management call elapsed time 1.6 .0
sequence load elapsed time 0.4 .0
hard parse (sharing criteria) elapsed time 0.0 .0
repeated bind elapsed time 0.0 .0
failed parse elapsed time 0.0 .0
DB time 22,960.5
background elapsed time 238.1
background cpu time 4.9
Operating System Statistics
-> *TIME statistic values are diffed.
All others display actual values. End Value is displayed if different
-> ordered by statistic type (CPU Use, Virtual Memory, Hardware Config), Name
Statistic Value End Value
BUSY_TIME 396,506
IDLE_TIME 2,483,725
IOWAIT_TIME 455,495
NICE_TIME 0
SYS_TIME 16,163
USER_TIME 380,052
LOAD 1 2
RSRC_MGR_CPU_WAIT_TIME 0
VM_IN_BYTES 95,646,943,232
VM_OUT_BYTES 1,686,059,008
PHYSICAL_MEMORY_BYTES 51,539,607,552
NUM_CPUS 8
NUM_CPU_CORES 8
NUM_CPU_SOCKETS 8
GLOBAL_RECEIVE_SIZE_MAX 4,194,304
GLOBAL_SEND_SIZE_MAX 1,048,586
TCP_RECEIVE_SIZE_DEFAULT 87,380
TCP_RECEIVE_SIZE_MAX 4,194,304
TCP_RECEIVE_SIZE_MIN 4,096
TCP_SEND_SIZE_DEFAULT 16,384
TCP_SEND_SIZE_MAX 4,194,304
TCP_SEND_SIZE_MIN 4,096
Operating System Statistics -
Snap Time Load %busy %user %sys %idle %iowait
29-Feb 04:00:35 1.4 N/A N/A N/A N/A N/A
29-Feb 05:00:41 1.7 13.8 13.2 0.6 86.2 15.8
Foreground Wait Class
-> s - second, ms - millisecond - 1000th of a second
-> ordered by wait time desc, waits desc
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
-> Captured Time accounts for 97.6% of Total DB time 22,960.46 (s)
-> Total FG Wait Time: 18,651.75 (s) DB CPU time: 3,748.35 (s)
Avg
%Time Total Wait wait
Wait Class Waits -outs Time (s) (ms) %DB time
User I/O 3,327,253 0 18,576 6 80.9
DB CPU 3,748 16.3
Commit 23,882 0 69 3 0.3
System I/O 1,035 0 3 3 0.0
Network 842,393 0 2 0 0.0
Other 10,120 99 0 0 0.0
Configuration 3 0 0 58 0.0
Application 264 0 0 1 0.0
Concurrency 1,482 0 0 0 0.0
Foreground Wait Events
-> s - second, ms - millisecond - 1000th of a second
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by wait time desc, waits desc (idle events last)
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Avg
%Time Total Wait wait Waits % DB
Event Waits -outs Time (s) (ms) /txn time
db file sequential read 3,054,464 0 17,002 6 104.5 74.0
read by other session 199,603 0 796 4 6.8 3.5
direct path read 46,301 0 439 9 1.6 1.9
db file scattered read 21,113 0 269 13 0.7 1.2
log file sync 23,882 0 69 3 0.8 .3
db file parallel read 4,727 0 68 14 0.2 .3
control file sequential re 1,035 0 3 3 0.0 .0
SQL*Net message to client 840,792 0 2 0 28.8 .0
direct path read temp 95 0 2 18 0.0 .0
local write wait 79 0 0 4 0.0 .0
Disk file operations I/O 870 0 0 0 0.0 .0
ASM file metadata operatio 4 0 0 50 0.0 .0
log file switch (private s 3 0 0 58 0.0 .0
ADR block file read 36 0 0 3 0.0 .0
enq: RO - fast object reus 5 0 0 16 0.0 .0
latch: cache buffers chain 1,465 0 0 0 0.1 .0
SQL*Net break/reset to cli 256 0 0 0 0.0 .0
asynch descriptor resize 10,059 100 0 0 0.3 .0
SQL*Net more data to clien 1,510 0 0 0 0.1 .0
enq: KO - fast object chec 3 0 0 8 0.0 .0
SQL*Net more data from cli 91 0 0 0 0.0 .0
latch: shared pool 14 0 0 0 0.0 .0
ADR block file write 5 0 0 1 0.0 .0
reliable message 8 0 0 0 0.0 .0
direct path write temp 1 0 0 2 0.0 .0
SQL*Net message from clien 840,794 0 68,885 82 28.8
jobq slave wait 7,365 100 3,679 499 0.3
Streams AQ: waiting for me 721 100 3,605 5000 0.0
wait for unread message on 3,648 98 3,603 988 0.1
KSV master wait 20 0 0 0 0.0
Background Wait Events
-> ordered by wait time desc, waits desc (idle events last)
-> Only events with Total Wait Time (s) >= .001 are shown
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Avg
%Time Total Wait wait Waits % bg
Event Waits -outs Time (s) (ms) /txn time
log file parallel write 29,353 0 83 3 1.0 34.8
db file parallel write 5,753 0 17 3 0.2 6.9
db file sequential read 1,638 0 15 9 0.1 6.1
control file sequential re 5,142 0 13 2 0.2 5.4
os thread startup 140 0 8 58 0.0 3.4
control file parallel writ 1,440 0 8 6 0.0 3.4
log file sequential read 304 0 8 26 0.0 3.3
db file scattered read 214 0 2 9 0.0 .8
ASM file metadata operatio 1,199 0 1 1 0.0 .3
direct path write 35 0 0 6 0.0 .1
direct path read 41 0 0 5 0.0 .1
kfk: async disk IO 6 0 0 9 0.0 .0
Disk file operations I/O 1,266 0 0 0 0.0 .0
ADR block file read 16 0 0 2 0.0 .0
read by other session 3 0 0 8 0.0 .0
Log archive I/O 2 0 0 10 0.0 .0
log file sync 3 0 0 5 0.0 .0
asynch descriptor resize 341 100 0 0 0.0 .0
CSS initialization 1 0 0 6 0.0 .0
log file single write 4 0 0 1 0.0 .0
latch: redo allocation 3 0 0 1 0.0 .0
ADR block file write 5 0 0 1 0.0 .0
LGWR wait for redo copy 45 0 0 0 0.0 .0
CSS operation: query 6 0 0 0 0.0 .0
CSS operation: action 1 0 0 1 0.0 .0
SQL*Net message to client 420 0 0 0 0.0 .0
rdbms ipc message 47,816 39 61,046 1277 1.6
DIAG idle wait 7,200 100 7,200 1000 0.2
Space Manager: slave idle 1,146 98 5,674 4951 0.0
class slave wait 284 0 3,983 14026 0.0
dispatcher timer 61 100 3,660 60006 0.0
Streams AQ: qmn coordinato 258 50 3,613 14003 0.0
Streams AQ: qmn slave idle 130 0 3,613 27789 0.0
Streams AQ: waiting for ti 7 71 3,608 515430 0.0
wait for unread message on 3,605 100 3,606 1000 0.1
pmon timer 1,201 100 3,604 3001 0.0
smon timer 15 73 3,603 240207 0.0
ASM background timer 754 0 3,602 4777 0.0
shared server idle wait 120 100 3,601 30006 0.0
SQL*Net message from clien 554 0 4 7 0.0
KSV master wait 101 0 0 2 0.0
Wait Event Histogram
-> Units for Total Waits column: K is 1000, M is 1000000, G is 1000000000
-> % of Waits: value of .0 indicates value was <.05%; value of null is truly 0
-> % of Waits: column heading of <=1s is truly <1024ms, >1s is truly >=1024ms
-> Ordered by Event (idle events last)
% of Waits
Total
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
ADR block file read 52 73.1 1.9 9.6 13.5 1.9
ADR block file write 10 100.0
ADR file lock 12 100.0
ARCH wait for archivelog l 3 100.0
ASM file metadata operatio 1203 97.3 .5 .7 .3 .2 .9
CSS initialization 1 100.0
CSS operation: action 1 100.0
CSS operation: query 6 83.3 16.7
Disk file operations I/O 2118 95.4 4.5 .1
LGWR wait for redo copy 45 100.0
Log archive I/O 2 100.0
SQL*Net break/reset to cli 256 99.6 .4
SQL*Net message to client 839.9 100.0 .0
SQL*Net more data from cli 91 100.0
SQL*Net more data to clien 1503 100.0
asynch descriptor resize 10.4K 100.0
buffer busy waits 2 100.0
control file parallel writ 1440 5.7 35.1 24.0 16.3 12.0 5.5 1.5
control file sequential re 6177 69.4 7.5 5.9 8.1 7.1 1.7 .3
db file parallel read 4727 1.7 3.2 3.2 10.1 46.6 33.3 1.8
db file parallel write 5755 42.3 21.3 18.6 11.2 4.6 1.4 .5
db file scattered read 21.5K 8.4 4.3 11.9 18.9 26.3 25.3 4.9
db file sequential read 3053. 28.7 15.1 11.1 17.9 21.5 5.4 .3 .0
direct path read 46.3K 9.9 8.8 18.5 21.7 22.8 15.7 2.7
direct path read temp 95 9.5 9.5 23.2 49.5 8.4
direct path write 35 11.4 31.4 17.1 22.9 11.4 2.9 2.9
direct path write temp 1 100.0
enq: KO - fast object chec 3 66.7 33.3
enq: RO - fast object reus 5 20.0 20.0 20.0 20.0 20.0
kfk: async disk IO 6 50.0 16.7 16.7 16.7
latch free 3 100.0
latch: cache buffers chain 1465 100.0
latch: cache buffers lru c 1 100.0
latch: object queue header 2 100.0
latch: redo allocation 3 33.3 33.3 33.3
latch: row cache objects 2 100.0
latch: shared pool 15 93.3 6.7
local write wait 79 35.4 34.2 21.5 8.9
log file parallel write 29.4K 47.8 21.7 11.9 9.9 6.8 1.6 .3
log file sequential read 304 6.3 3.0 3.6 10.2 23.4 24.3 29.3
log file single write 4 25.0 75.0
log file switch (private s 3 100.0
log file sync 23.9K 40.9 28.0 12.9 9.7 6.7 1.5 .3
os thread startup 140 100.0
read by other session 199.6 37.1 19.9 12.9 13.1 13.8 3.1 .2
reliable message 8 100.0
ASM background timer 755 2.9 .4 .1 .1 .3 .1 .3 95.8
DIAG idle wait 7196 100.0
KSV master wait 121 88.4 2.5 3.3 2.5 .8 .8 1.7
SQL*Net message from clien 840.1 97.1 1.8 .5 .2 .2 .1 .0 .1
Space Manager: slave idle 1147 .1 .5 99.4
Streams AQ: qmn coordinato 258 49.6 .4 50.0
Streams AQ: qmn slave idle 130 .8 99.2
Streams AQ: waiting for me 721 100.0
Streams AQ: waiting for ti 7 28.6 42.9 28.6
class slave wait 283 39.9 2.5 2.5 3.5 4.9 9.2 15.2 22.3
dispatcher timer 60 100.0
jobq slave wait 7360 .0 .0 .0 99.9
pmon timer 1201 100.0
rdbms ipc message 47.8K 2.7 31.6 17.4 1.1 1.1 .9 20.9 24.3
Wait Event Histogram DB/Inst: I2KPROD/I2KPROD Snaps: 5606-5607
-> Units for Total Waits column: K is 1000, M is 1000000, G is 1000000000
-> % of Waits: value of .0 indicates value was <.05%; value of null is truly 0
-> % of Waits: column heading of <=1s is truly <1024ms, >1s is truly >=1024ms
-> Ordered by Event (idle events last)
% of Waits
Total
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
shared server idle wait 120 100.0
smon timer 16 6.3 93.8
wait for unread message on 7250 .1 99.9
Latch Miss Sources
-> only latches with sleeps are shown
-> ordered by name, sleeps desc
NoWait Waiter
Latch Name Where Misses Sleeps Sleeps
In memory undo latch ktichg: child 0 1 0
active service list kswslogon: session logout 0 2 0
cache buffers chains kcbgtcr_2 0 1,123 483
cache buffers chains kcbgtcr: fast path (cr pin 0 496 1,131
cache buffers chains kcbrls_2 0 5 6
cache buffers chains kcbgcur_2 0 4 0
cache buffers chains kcbgtcr: fast path 0 3 1
cache buffers chains kcbzwb 0 2 4
cache buffers chains kcbchg1: kslbegin: bufs no 0 1 0
cache buffers chains kcbnew: new latch again 0 1 0
cache buffers chains kcbrls_1 0 1 6
cache buffers chains kcbzgb: scan from tail. no 0 1 0
cache buffers lru chain kcbzgws 0 1 0
object queue header oper kcbo_switch_cq 0 1 0
object queue header oper kcbo_switch_mq_bg 0 1 2
redo allocation kcrfw_redo_gen: redo alloc 0 3 0
row cache objects kqrpre: find obj 0 1 1
row cache objects kqrso 0 1 0
shared pool kghalo 0 13 3
shared pool kghupr1 0 4 15
shared pool kghalp 0 1 0
space background task la ktsj_grab_task 0 2 2
------------------------------------------------------------- -
Hi: I'm analyzing this STATSPACK report: it is "volume test" on our UAT server, so most input is from 'bind variables'. Our shared pool is well utilized in oracle. Oracle redo logs is not appropriately configured on this server, as in 'Top 5 wait events' there are 2 for redos.
I need to know what else information can be dig-out from 'foreground wait events' & 'background wait events', and what can assist us to better understanding, in combination of 'Top 5 wait event's, that how the server/test went? it could be overwelming No. of wait events, so appreciate any helpful diagnostic or analysis. Database is oracle 11.2.0.4 upgraded from 11.2.0.3, on IBM AIX power system 64bit, level 6.x
STATSPACK report for
Database DB Id Instance Inst Num Startup Time Release RAC
~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
700000XXX XXX 1 22-Apr-15 12:12 11.2.0.4.0 NO
Host Name Platform CPUs Cores Sockets Memory (G)
~~~~ ---------------- ---------------------- ----- ----- ------- ------------
dXXXX_XXX AIX-Based Systems (64- 2 1 0 16.0
Snapshot Snap Id Snap Time Sessions Curs/Sess Comment
~~~~~~~~ ---------- ------------------ -------- --------- ------------------
Begin Snap: 5635 22-Apr-15 13:00:02 114 4.6
End Snap: 5636 22-Apr-15 14:00:01 128 8.8
Elapsed: 59.98 (mins) Av Act Sess: 0.6
DB time: 35.98 (mins) DB CPU: 19.43 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 2,064M Std Block Size: 8K
Shared Pool: 3,072M Log Buffer: 13,632K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ ------------------ ----------------- ----------- -----------
DB time(s): 0.6 0.0 0.00 0.00
DB CPU(s): 0.3 0.0 0.00 0.00
Redo size: 458,720.6 8,755.7
Logical reads: 12,874.2 245.7
Block changes: 1,356.4 25.9
Physical reads: 6.6 0.1
Physical writes: 61.8 1.2
User calls: 2,033.7 38.8
Parses: 286.5 5.5
Hard parses: 0.5 0.0
W/A MB processed: 1.7 0.0
Logons: 1.2 0.0
Executes: 801.1 15.3
Rollbacks: 6.1 0.1
Transactions: 52.4
Instance Efficiency Indicators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.98 Optimal W/A Exec %: 100.00
Library Hit %: 99.77 Soft Parse %: 99.82
Execute to Parse %: 64.24 Latch Hit %: 99.98
Parse CPU to Parse Elapsd %: 53.15 % Non-Parse CPU: 98.03
Shared Pool Statistics Begin End
Memory Usage %: 10.50 12.79
% SQL with executions>1: 69.98 78.37
% Memory for SQL w/exec>1: 70.22 81.96
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
CPU time 847 50.2
enq: TX - row lock contention 4,480 434 97 25.8
log file sync 284,169 185 1 11.0
log file parallel write 299,537 164 1 9.7
log file sequential read 698 16 24 1.0
Host CPU (CPUs: 2 Cores: 1 Sockets: 0)
~~~~~~~~ Load Average
Begin End User System Idle WIO WCPU
1.16 1.84 19.28 14.51 66.21 1.20 82.01
Instance CPU
~~~~~~~~~~~~ % Time (seconds)
Host: Total time (s): 7,193.8
Host: Busy CPU time (s): 2,430.7
% of time Host is Busy: 33.8
Instance: Total CPU time (s): 1,203.1
% of Busy CPU used for Instance: 49.5
Instance: Total Database time (s): 2,426.4
%DB time waiting for CPU (Resource Mgr): 0.0
Memory Statistics Begin End
~~~~~~~~~~~~~~~~~ ------------ ------------
Host Mem (MB): 16,384.0 16,384.0
SGA use (MB): 7,136.0 7,136.0
PGA use (MB): 282.5 361.4
% Host Mem used for SGA+PGA: 45.3 45.8
Foreground Wait Events DB/Inst: XXXXXs Snaps: 5635-5636
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
enq: TX - row lock contentio 4,480 0 434 97 0.0 25.8
log file sync 284,167 0 185 1 1.5 11.0
Disk file operations I/O 8,741 0 4 0 0.0 .2
direct path write 13,247 0 3 0 0.1 .2
db file sequential read 6,058 0 1 0 0.0 .1
buffer busy waits 1,800 0 1 1 0.0 .1
SQL*Net more data to client 29,161 0 1 0 0.2 .1
direct path read 7,696 0 1 0 0.0 .0
db file scattered read 316 0 1 2 0.0 .0
latch: shared pool 144 0 0 2 0.0 .0
CSS initialization 30 0 0 3 0.0 .0
cursor: pin S 10 0 0 9 0.0 .0
row cache lock 41 0 0 2 0.0 .0
latch: row cache objects 19 0 0 3 0.0 .0
log file switch (private str 8 0 0 7 0.0 .0
library cache: mutex X 28 0 0 2 0.0 .0
latch: cache buffers chains 54 0 0 1 0.0 .0
latch free 290 0 0 0 0.0 .0
control file sequential read 1,568 0 0 0 0.0 .0
log file switch (checkpoint 4 0 0 6 0.0 .0
direct path sync 8 0 0 3 0.0 .0
latch: redo allocation 60 0 0 0 0.0 .0
SQL*Net break/reset to clien 34 0 0 1 0.0 .0
latch: enqueue hash chains 45 0 0 0 0.0 .0
latch: cache buffers lru cha 7 0 0 2 0.0 .0
latch: session allocation 5 0 0 1 0.0 .0
latch: object queue header o 6 0 0 1 0.0 .0
ASM file metadata operation 30 0 0 0 0.0 .0
latch: In memory undo latch 15 0 0 0 0.0 .0
latch: undo global data 8 0 0 0 0.0 .0
SQL*Net message from client 6,362,536 0 278,225 44 33.7
jobq slave wait 7,270 100 3,635 500 0.0
SQL*Net more data from clien 7,976 0 15 2 0.0
SQL*Net message to client 6,362,544 0 8 0 33.7
Background Wait Events DB/Inst: XXXXXs Snaps: 5635-5636
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
log file parallel write 299,537 0 164 1 1.6 9.7
log file sequential read 698 0 16 24 0.0 1.0
db file parallel write 9,556 0 13 1 0.1 .8
os thread startup 146 0 10 70 0.0 .6
control file parallel write 2,037 0 2 1 0.0 .1
Log archive I/O 35 0 1 30 0.0 .1
LGWR wait for redo copy 2,447 0 0 0 0.0 .0
db file async I/O submit 9,556 0 0 0 0.1 .0
db file sequential read 145 0 0 2 0.0 .0
Disk file operations I/O 349 0 0 0 0.0 .0
db file scattered read 30 0 0 4 0.0 .0
control file sequential read 5,837 0 0 0 0.0 .0
ADR block file read 19 0 0 4 0.0 .0
ADR block file write 5 0 0 15 0.0 .0
direct path write 14 0 0 2 0.0 .0
direct path read 3 0 0 7 0.0 .0
latch: shared pool 3 0 0 6 0.0 .0
log file single write 56 0 0 0 0.0 .0
latch: redo allocation 53 0 0 0 0.0 .0
latch: active service list 1 0 0 3 0.0 .0
latch free 11 0 0 0 0.0 .0
rdbms ipc message 314,523 5 57,189 182 1.7
Space Manager: slave idle wa 4,086 88 18,996 4649 0.0
DIAG idle wait 7,185 100 7,186 1000 0.0
Streams AQ: waiting for time 2 50 4,909 ###### 0.0
Streams AQ: qmn slave idle w 129 0 3,612 28002 0.0
Streams AQ: qmn coordinator 258 50 3,612 14001 0.0
smon timer 43 2 3,605 83839 0.0
pmon timer 1,199 99 3,596 2999 0.0
SQL*Net message from client 17,019 0 31 2 0.1
SQL*Net message to client 12,762 0 0 0 0.1
class slave wait 28 0 0 0 0.0
thank you very much!Hi: just know it now: it is a large amount of 'concurrent transaction' designed in this "Volume Test" - to simulate large incoming transaction volme, so I guess wait in eq:TX - row is expected.
The fact: (1) redo logs at uat server is known to not well-tune for configurations (2) volume test slow 5%, however data amount in its test is kept the same by each time import production data, by the team. So why it slowed 5% this year?
The wait histogram is pasted below, any one interest to take a look? any ideas?
Wait Event Histogram DB/Inst: XXXX/XXXX Snaps: 5635-5636
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
ADR block file read 19 26.3 5.3 10.5 57.9
ADR block file write 5 40.0 60.0
ADR file lock 6 100.0
ARCH wait for archivelog l 14 100.0
ASM file metadata operatio 30 100.0
CSS initialization 30 100.0
Disk file operations I/O 9090 97.2 1.4 .6 .4 .2 .1 .1
LGWR wait for redo copy 2447 98.5 .5 .4 .2 .2 .2 .1
Log archive I/O 35 40.0 8.6 25.7 2.9 22.9
SQL*Net break/reset to cli 34 85.3 8.8 5.9
SQL*Net more data to clien 29K 99.9 .0 .0 .0 .0 .0
buffer busy waits 1800 96.8 .7 .7 .6 .3 .4 .5
control file parallel writ 2037 90.7 5.0 2.1 .8 1.0 .3 .1
control file sequential re 7405 100.0 .0
cursor: pin S 10 10.0 90.0
db file async I/O submit 9556 99.9 .0 .0 .0
db file parallel read 1 100.0
db file parallel write 9556 62.0 32.4 1.7 .8 1.5 1.3 .1
db file scattered read 345 72.8 3.8 2.3 11.6 9.0 .6
db file sequential read 6199 97.2 .2 .3 1.6 .7 .0 .0
direct path read 7699 99.1 .4 .2 .1 .1 .0
direct path sync 8 25.0 37.5 12.5 25.0
direct path write 13K 97.8 .9 .5 .4 .3 .1 .0
enq: TX - row lock content 4480 .4 .7 1.3 3.0 6.8 12.3 75.4 .1
latch free 301 98.3 .3 .7 .7
latch: In memory undo latc 15 93.3 6.7
latch: active service list 1 100.0
latch: cache buffers chain 55 94.5 3.6 1.8
latch: cache buffers lru c 9 88.9 11.1
latch: call allocation 6 100.0
latch: checkpoint queue la 3 100.0
latch: enqueue hash chains 45 97.8 2.2
latch: messages 4 100.0
latch: object queue header 7 85.7 14.3
latch: redo allocation 113 97.3 1.8 .9
latch: row cache objects 19 89.5 5.3 5.3
latch: session allocation 5 80.0 20.0
latch: shared pool 147 90.5 1.4 2.7 1.4 .7 1.4 2.0
latch: undo global data 8 100.0
library cache: mutex X 28 89.3 3.6 3.6 3.6
log file parallel write 299K 95.6 2.6 1.0 .4 .3 .2 .0
log file sequential read 698 29.5 .1 4.6 46.8 18.9
log file single write 56 100.0
log file switch (checkpoin 4 25.0 50.0 25.0
log file switch (private s 8 12.5 37.5 50.0
log file sync 284K 93.3 3.7 1.4 .7 .5 .3 .1
os thread startup 146 100.0
row cache lock 41 85.4 9.8 2.4 2.4
DIAG idle wait 7184 100.0
SQL*Net message from clien 6379K 86.6 5.1 2.9 1.3 .7 .3 2.8 .3
SQL*Net message to client 6375K 100.0 .0 .0 .0 .0 .0 .0
Wait Event Histogram DB/Inst: XXXX/xxxx Snaps: 5635-5636
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
SQL*Net more data from cli 7976 99.7 .1 .1 .0 .1
Space Manager: slave idle 4086 .1 .2 .0 .0 .3 3.2 96.1
Streams AQ: qmn coordinato 258 49.2 .8 50.0
Streams AQ: qmn slave idle 129 100.0
Streams AQ: waiting for ti 2 50.0 50.0
class slave wait 28 92.9 3.6 3.6
jobq slave wait 7270 .0 100.0
pmon timer 1199 100.0
rdbms ipc message 314K 10.3 7.3 39.7 15.4 10.6 5.3 8.2 3.3
smon timer 43 100.0 -
How to find where is bottleneck oracle 11.2
I am running oracle 11gr2 on windows server 2008 R2 having ram 20gb.
I am taking statspack report on database and following is it:
i am not getting where is actual bottleneck is and some time there is problem regarding procedure hangup which having business logic fetch by cursor.
i am generate statistics every day using :
begin
dbms_stats.gather_schema_stats(ownname => 'MFG',cascade => TRUE,no_invalidate => FALSE);
end;
and also using:
analyze table abc compute statistics;
stats pack report:
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ ------------------ ----------------- ----------- -----------
DB time(s): 2.0 0.8 0.00 0.01
DB CPU(s): 1.5 0.6 0.00 0.00
Redo size: 68,274.8 28,441.6
Logical reads: 83,672.4 34,855.8
Block changes: 633.0 263.7
Physical reads: 2,763.0 1,151.0
Physical writes: 37.4 15.6
User calls: 379.8 158.2
Parses: 342.6 142.7
Hard parses: 0.3 0.1
W/A MB processed: 9.9 4.1
Logons: 34.0 14.2
Executes: 2,702.3 1,125.7
Rollbacks: 0.0 0.0
Transactions: 2.4
Instance Efficiency Indicators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 98.06 Optimal W/A Exec %: 99.99
Library Hit %: 100.03 Soft Parse %: 99.90
Execute to Parse %: 87.32 Latch Hit %: 99.97
Parse CPU to Parse Elapsd %: 91.04 % Non-Parse CPU: 99.19
Shared Pool Statistics Begin End
Memory Usage %: 68.94 69.12
% SQL with executions>1: 64.57 65.59
% Memory for SQL w/exec>1: 81.76 82.22
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
CPU time 2,194 71.4
db file sequential read 131,781 323 2 10.5
db file scattered read 89,404 206 2 6.7
Disk file operations I/O 91,788 176 2 5.7
direct path read 17,001 89 5 2.9
Host CPU (CPUs: 8 Cores: 4 Sockets: 1)
~~~~~~~~ Load Average
Begin End User System Idle WIO WCPU
16.82 5.62 77.56
Instance CPU
~~~~~~~~~~~~ % Time (seconds)
Host: Total time (s): 15,156.8
Host: Busy CPU time (s): 3,401.3
% of time Host is Busy: 22.4
Instance: Total CPU time (s): 2,827.1
% of Busy CPU used for Instance: 83.1
Instance: Total Database time (s): 3,753.3
%DB time waiting for CPU (Resource Mgr): 0.0
Memory Statistics Begin End
~~~~~~~~~~~~~~~~~ ------------ ------------
Host Mem (MB): 20,468.5 20,468.5
SGA use (MB): 11,022.6 11,022.6
PGA use (MB): 779.3 807.4
% Host Mem used for SGA+PGA: 57.7 57.8
Time Model System Stats DB/Inst: ORACLE/oracle Snaps: 511-512
-> Ordered by % of DB time desc, Statistic name
Statistic Time (s) % DB time
sql execute elapsed time 2,912.2 78.8
DB CPU 2,825.0 76.4
connection management call elapsed 343.6 9.3
PL/SQL execution elapsed time 56.6 1.5
parse time elapsed 42.8 1.2
hard parse elapsed time 25.1 .7
PL/SQL compilation elapsed time 1.1 .0
repeated bind elapsed time 1.0 .0
inbound PL/SQL rpc elapsed time 0.7 .0
hard parse (sharing criteria) elaps 0.5 .0
sequence load elapsed time 0.1 .0
failed parse elapsed time 0.0 .0
hard parse (bind mismatch) elapsed 0.0 .0
DB time 3,697.2
background elapsed time 56.1
background cpu time 2.1
Foreground Wait Events DB/Inst: ORACLE/oracle Snaps: 511-512
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
db file sequential read 131,324 0 321 2 28.9 10.5
db file scattered read 89,376 0 206 2 19.6 6.7
Disk file operations I/O 91,705 0 176 2 20.2 5.7
direct path read 16,992 0 89 5 3.7 2.9
log file sync 5,064 0 16 3 1.1 .5
db file parallel read 1,575 0 14 9 0.3 .4
enq: KO - fast object checkp 8 0 5 591 0.0 .2
control file sequential read 4,457 0 2 0 1.0 .1
direct path write temp 1,635 0 2 1 0.4 .1
SQL*Net more data to client 14,776 0 1 0 3.2 .0
SQL*Net message from dblink 603 0 0 1 0.1 .0
ADR block file read 91 0 0 4 0.0 .0
direct path read temp 713 0 0 0 0.2 .0
SQL*Net break/reset to clien 152 0 0 0 0.0 .0
asynch descriptor resize 8,239 100 0 0 1.8 .0
library cache: mutex X 1,238 0 0 0 0.3 .0
SQL*Net more data from dblin 345 0 0 0 0.1 .0
ADR block file write 5 0 0 0 0.0 .0
latch free 66 0 0 0 0.0 .0
direct path write 4 0 0 0 0.0 .0
cursor: pin S 10 0 0 0 0.0 .0
SQL*Net message from client 526,480 0 238,770 454 115.7
jobq slave wait 3,954 100 2,034 514 0.9
wait for unread message on b 1,896 98 1,894 999 0.4
Streams AQ: waiting for mess 379 100 1,892 4993 0.1
SQL*Net more data from clien 11,597 0 4 0 2.5
single-task message 29 0 1 21 0.0
SQL*Net message to client 526,478 0 1 0 115.7
Background Wait Events DB/Inst: ORACLE/oracle Snaps: 511-512
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
log file parallel write 7,020 0 16 2 1.5 .5
db file parallel write 5,529 0 14 3 1.2 .5
control file sequential read 5,966 0 5 1 1.3 .2
control file parallel write 1,618 0 3 2 0.4 .1
log file sequential read 66 0 3 50 0.0 .1
SQL*Net more data to client 64,218 0 2 0 14.1 .1
db file sequential read 457 0 2 4 0.1 .1
os thread startup 66 0 1 8 0.0 .0
Disk file operations I/O 83 0 0 1 0.0 .0
asynch descriptor resize 64,343 100 0 0 14.1 .0
direct path read 9 0 0 6 0.0 .0
db file scattered read 28 0 0 2 0.0 .0
rdbms ipc reply 8 0 0 0 0.0 .0
LGWR wait for redo copy 57 0 0 0 0.0 .0
log file single write 8 0 0 0 0.0 .0
db file single write 1 0 0 2 0.0 .0
rdbms ipc message 14,376 51 31,809 2213 3.2
DIAG idle wait 3,738 100 3,788 1013 0.8
smon timer 12 33 2,077 ###### 0.0
dispatcher timer 32 100 1,920 60012 0.0
Streams AQ: qmn coordinator 136 50 1,905 14009 0.0
Streams AQ: qmn slave idle w 68 0 1,905 28017 0.0
pmon timer 2,099 30 1,896 903 0.5
Space Manager: slave idle wa 381 98 1,894 4970 0.1
shared server idle wait 63 100 1,891 30014 0.0
SQL*Net message from client 257,103 0 1,741 7 56.5
SQL*Net more data from clien 64,218 0 88 1 14.1
SQL*Net message to client 192,827 0 0 0 42.4
-------------------------------------------------------------Now , I am perform only analyzed method for statics.
but even though dbms_stats is new method and cover all limit of analyzed method why should you not prefer it?..
second thing we don't have licence for performance tuning so,AWR does not work . I am using stats pack utility and some query to find top sql on load time which are as below:
high memory consumed:
110125 19483 5.65
begin CHECK_EMP_ISSUE(P_FOR_COMP=>:P_FOR_COMP, P_FOR_TRANS_DATE=>:P_FOR_TRANS_DATE, P_FOR_EMP_CODE=>:P_FOR_EMP_CODE, P_FOR_DEPT_CODE=>:P_FOR_DEPT_CODE, P_FOR_PROCESS=>:P_FOR_PROCESS, P_FOR_KAPAN_NO=>:P_FOR_KAPAN_NO, P_FOR_PACKET_NO=>:P_FOR_PACKET_NO, P_FOR_PACKET_ID=>:P_FOR_PACKET_ID, P_FOR_ISSUE_TYPE=>:P_FOR_ISSUE_TYPE, P_FOR_SHIFT_NO=>:P_FOR_SHIFT_NO, P_FOR_MC_CODE=>:P_FOR_MC_CODE, P_FOR_CHECK_PREF_EMP=>:P_FOR_CHECK_PREF_EMP, P_FOR_DUMMY=>:P_FOR_DUMMY, P_FOR_MSG=>:P_FOR_MSG, P_FOR_WARNING=>:P_FOR_WARNING, P_FOR_MSG_VALUE=>:P_FOR_MSG_VALUE, VREC=>:VREC); end; high i/o consumed:
EVENT WAIT_CLASS USER_IO_WAIT_TIME SQL_TEXT
asynch descriptor resize Other 32454263 SELECT /*+ result_cache */ COMP_CODE,KAPAN_GROUP,VKAPAN_GROUP,SEQ_NO,PACKET_ID,CHILD_ID,INW_DATE, VKAPAN_NO,KAPAN_NO,VPACKET_NO,PACKET_NO,SUB_ID,STONE_TYPE,PCS,WGT,CUR_WGT,P_SEQ_NO,L_SEQ_NO, STAGE,STATUS,PACKET_TYPE,:B1 DEPT_CODE,EMP_CODE,PROCESS,CLV_END_DATE,MFG_END_DATE,TRUNC(SYSDATE)-TRUNC(INW_DATE)VDAYS FROM ( SELECT /*+ leading(a) use_hash(b)*/ :B3 COMP_CODE,A.KAPAN_GROUP,A.VKAPAN_GROUP,:B1 DEPT_CODE,A.SEQ_NO,A.PACKET_ID,A.CHILD_ID,A.INW_DATE, A.VKAPAN_NO,A.KAPAN_NO,A.VPACKET_NO,A.PACKET_NO,A.SUB_ID,A.STONE_TYPE,A.PCS PCS,A.WGT,A.CUR_WGT,A.P_SEQ_NO,A.L_SEQ_NO, -
Slow down Database after upgrading to 10.2.0.5
Hi
I am having performance problems after upgrading to 10.2.0.5
At the beginning I thought the problem was sga too smalle ( ini: 598M , now 1408M ) but even after recreating the database with the new value the problem remains.
I am sending reports so that someone could give me an idea.
Thanks in advance!
DETAILED ADDM REPORT FOR TASK 'TASK_240' WITH ID 240
Analysis Period: 22-JUN-2011 from 08:34:06 to 16:00:13
Database ID/Instance: 2462860799/1
Database/Instance Names: DXT/DXT
Host Name: thoracle
Database Version: 10.2.0.5.0
Snapshot Range: from 71 to 78
Database Time: 6726 seconds
Average Database Load: .3 active sessions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FINDING 1: 38% impact (2540 seconds)
SQL statements consuming significant database time were found.
RECOMMENDATION 1: SQL Tuning, 26% benefit (1763 seconds)
ACTION: Investigate the SQL statement with SQL_ID "30rku9qg2y30j" for
possible performance improvements.
RELEVANT OBJECT: SQL statement with SQL_ID 30rku9qg2y30j and
PLAN_HASH 2734400036
select a.owner, a.object_name, INSTR(a.object_type, :"SYS_B_00"),
:"SYS_B_01" from sys.all_objects a where a.object_type IN
(:"SYS_B_02",:"SYS_B_03") and a.status = :"SYS_B_04" and a.owner
like:"SYS_B_05"escape:"SYS_B_06" and a.object_name
like:"SYS_B_07"escape:"SYS_B_08" union all select c.owner,
c.synonym_name, INSTR(a.object_type, :"SYS_B_09"), :"SYS_B_10" from
sys.all_objects a, sys.all_synonyms c where c.table_owner = a.owner
and c.table_name = a.object_name and a.object_type IN
(:"SYS_B_11",:"SYS_B_12") and a.status = :"SYS_B_13" and c.owner
like:"SYS_B_14"escape:"SYS_B_15" and c.synonym_name
like:"SYS_B_16"escape:"SYS_B_17" union all select distinct b.owner,
CONCAT(b.package_name, :"SYS_B_18" || b.object_name),
min(b.position), max(b.overload) from sys.all_arguments b where
b.package_name IS NOT NULL and b.owner
like:"SYS_B_19"escape:"SYS_B_20" and b.package_name
like:"SYS_B_21"escape:"SYS_B_22" group by b.owner,
CONCAT(b.package_name, :"SYS_B_23" || b.object_name) union all select
distinct c.owner, CONCAT(c.synonym_name, :"SYS_B_24" ||
b.object_name), min(b.position), max(b.overload) from
sys.all_arguments b, sys.all_synonyms c where c.table_owner = b.owner
and c.table_name = b.package_name and b.package_name IS NOT NULL and
c.owner like:"SYS_B_25"escape:"SYS_B_26" and c.synonym_name
like:"SYS_B_27"escape:"SYS_B_28" group by c.owner,
CONCAT(c.synonym_name, :"SYS_B_29" || b.object_name) union all select
distinct c.owner, c.synonym_name, min(b.position), max(b.overload)
from sys.all_arguments b, sys.all_synonyms c where c.owner = b.owner
and c.table_owner=b.package_name and c.table_name=b.object_name and
c.owner like:"SYS_B_30"escape:"SYS_B_31" and c.synonym_name
like:"SYS_B_32"escape:"SYS_B_33" group by c.owner, c.synonym_name
RATIONALE: SQL statement with SQL_ID "30rku9qg2y30j" was executed 12270
times and had an average elapsed time of 0.036 seconds.
RATIONALE: Waiting for event "cursor: pin S wait on X" in wait class
"Concurrency" accounted for 7% of the database time spent in
processing the SQL statement with SQL_ID "30rku9qg2y30j".
RECOMMENDATION 2: SQL Tuning, 23% benefit (1550 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"7yv1ba0c8y86t".
RELEVANT OBJECT: SQL statement with SQL_ID 7yv1ba0c8y86t and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
RATIONALE: SQL statement with SQL_ID "7yv1ba0c8y86t" was executed 47
times and had an average elapsed time of 32 seconds.
RECOMMENDATION 3: SQL Tuning, 14% benefit (926 seconds)
ACTION: Use bigger fetch arrays while fetching results from the SELECT
statement with SQL_ID "7yv1ba0c8y86t".
RELEVANT OBJECT: SQL statement with SQL_ID 7yv1ba0c8y86t and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
FINDING 2: 37% impact (2508 seconds)
Time spent on the CPU by the instance was responsible for a substantial part
of database time.
RECOMMENDATION 1: SQL Tuning, 26% benefit (1763 seconds)
ACTION: Investigate the SQL statement with SQL_ID "30rku9qg2y30j" for
possible performance improvements.
RELEVANT OBJECT: SQL statement with SQL_ID 30rku9qg2y30j and
PLAN_HASH 2734400036
select a.owner, a.object_name, INSTR(a.object_type, :"SYS_B_00"),
:"SYS_B_01" from sys.all_objects a where a.object_type IN
(:"SYS_B_02",:"SYS_B_03") and a.status = :"SYS_B_04" and a.owner
like:"SYS_B_05"escape:"SYS_B_06" and a.object_name
like:"SYS_B_07"escape:"SYS_B_08" union all select c.owner,
c.synonym_name, INSTR(a.object_type, :"SYS_B_09"), :"SYS_B_10" from
sys.all_objects a, sys.all_synonyms c where c.table_owner = a.owner
and c.table_name = a.object_name and a.object_type IN
(:"SYS_B_11",:"SYS_B_12") and a.status = :"SYS_B_13" and c.owner
like:"SYS_B_14"escape:"SYS_B_15" and c.synonym_name
like:"SYS_B_16"escape:"SYS_B_17" union all select distinct b.owner,
CONCAT(b.package_name, :"SYS_B_18" || b.object_name),
min(b.position), max(b.overload) from sys.all_arguments b where
b.package_name IS NOT NULL and b.owner
like:"SYS_B_19"escape:"SYS_B_20" and b.package_name
like:"SYS_B_21"escape:"SYS_B_22" group by b.owner,
CONCAT(b.package_name, :"SYS_B_23" || b.object_name) union all select
distinct c.owner, CONCAT(c.synonym_name, :"SYS_B_24" ||
b.object_name), min(b.position), max(b.overload) from
sys.all_arguments b, sys.all_synonyms c where c.table_owner = b.owner
and c.table_name = b.package_name and b.package_name IS NOT NULL and
c.owner like:"SYS_B_25"escape:"SYS_B_26" and c.synonym_name
like:"SYS_B_27"escape:"SYS_B_28" group by c.owner,
CONCAT(c.synonym_name, :"SYS_B_29" || b.object_name) union all select
distinct c.owner, c.synonym_name, min(b.position), max(b.overload)
from sys.all_arguments b, sys.all_synonyms c where c.owner = b.owner
and c.table_owner=b.package_name and c.table_name=b.object_name and
c.owner like:"SYS_B_30"escape:"SYS_B_31" and c.synonym_name
like:"SYS_B_32"escape:"SYS_B_33" group by c.owner, c.synonym_name
RATIONALE: SQL statement with SQL_ID "30rku9qg2y30j" was executed 12270
times and had an average elapsed time of 0.036 seconds.
RATIONALE: Waiting for event "cursor: pin S wait on X" in wait class
"Concurrency" accounted for 7% of the database time spent in
processing the SQL statement with SQL_ID "30rku9qg2y30j".
RATIONALE: Average CPU used per execution was 0.036 seconds.
RECOMMENDATION 2: SQL Tuning, 23% benefit (1550 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"7yv1ba0c8y86t".
RELEVANT OBJECT: SQL statement with SQL_ID 7yv1ba0c8y86t and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
RATIONALE: SQL statement with SQL_ID "7yv1ba0c8y86t" was executed 47
times and had an average elapsed time of 32 seconds.
RATIONALE: Average CPU used per execution was 32 seconds.
RECOMMENDATION 3: SQL Tuning, 5.8% benefit (390 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"cbtd2nt52qn1c".
RELEVANT OBJECT: SQL statement with SQL_ID cbtd2nt52qn1c and
PLAN_HASH 2897530229
Select DAE_.ROWID, DAE_.*, HAE_.*
From THPR.GACCENTRYD DAE_, THPR.GACCENTRY HAE_ Where ((HAE_.TYP_0(+)
= DAE_.TYP_0) AND (HAE_.NUM_0(+) = DAE_.NUM_0))
And HAE_.CPY_0 = :1 And HAE_.ACCDAT_0 >= :2 And HAE_.ACCDAT_0 <= :3
And DAE_.ACC_0 = :4 And HAE_.FCY_0 >= :5 And HAE_.FCY_0 <= :6
Order by DAE_.BPR_0,DAE_.CUR_0,DAE_.ACC_0
RATIONALE: SQL statement with SQL_ID "cbtd2nt52qn1c" was executed 12980
times and had an average elapsed time of 0.03 seconds.
RATIONALE: Average CPU used per execution was 0.029 seconds.
RECOMMENDATION 4: SQL Tuning, 2.1% benefit (138 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"33t7fszkr29gy".
RELEVANT OBJECT: SQL statement with SQL_ID 33t7fszkr29gy and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
(((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7)))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
RATIONALE: SQL statement with SQL_ID "33t7fszkr29gy" was executed 1
times and had an average elapsed time of 136 seconds.
RATIONALE: Average CPU used per execution was 138 seconds.
FINDING 3: 15% impact (1008 seconds)
SQL statements with the same text were not shared because of cursor
environment mismatch. This resulted in additional hard parses which were
consuming significant database time.
RECOMMENDATION 1: Application Analysis, 15% benefit (1008 seconds)
ACTION: Look for top reason for cursor environment mismatch in
V$SQL_SHARED_CURSOR.
ADDITIONAL INFORMATION:
Common causes of environment mismatch are session NLS settings, SQL
trace settings and optimizer parameters.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Hard parsing of SQL statements was consuming significant
database time. (20% impact [1336 seconds])
SYMPTOM: Contention for latches related to the shared pool was
consuming significant database time. (2% impact [135
seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of
database time.
SYMPTOM: Wait class "Concurrency" was consuming significant
database time. (2.3% impact [154 seconds])
FINDING 4: 8.5% impact (570 seconds)
Wait class "User I/O" was consuming significant database time.
NO RECOMMENDATIONS AVAILABLE
ADDITIONAL INFORMATION:
Waits for I/O to temporary tablespaces were not consuming significant
database time.
The throughput of the I/O subsystem was not significantly lower than
expected.
FINDING 5: 5.3% impact (355 seconds)
The SGA was inadequately sized, causing additional I/O or hard parses.
RECOMMENDATION 1: DB Configuration, 3.2% benefit (215 seconds)
ACTION: Increase the size of the SGA by setting the parameter
"sga_target" to 1740 M.
ADDITIONAL INFORMATION:
The value of parameter "sga_target" was "1392 M" during the analysis
period.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Hard parsing of SQL statements was consuming significant
database time. (20% impact [1336 seconds])
SYMPTOM: Contention for latches related to the shared pool was
consuming significant database time. (2% impact [135
seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of
database time.
SYMPTOM: Wait class "Concurrency" was consuming significant
database time. (2.3% impact [154 seconds])
SYMPTOM: Wait class "User I/O" was consuming significant database time.
(8.5% impact [570 seconds])
INFO: Waits for I/O to temporary tablespaces were not consuming
significant database time.
The throughput of the I/O subsystem was not significantly lower
than expected.
FINDING 6: 4.2% impact (281 seconds)
Cursors were getting invalidated due to DDL operations. This resulted in
additional hard parses which were consuming significant database time.
RECOMMENDATION 1: Application Analysis, 4.2% benefit (281 seconds)
ACTION: Investigate appropriateness of DDL operations.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Hard parsing of SQL statements was consuming significant
database time. (20% impact [1336 seconds])
SYMPTOM: Contention for latches related to the shared pool was
consuming significant database time. (2% impact [135
seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of
database time.
SYMPTOM: Wait class "Concurrency" was consuming significant
database time. (2.3% impact [154 seconds])
FINDING 7: 4% impact (266 seconds)
Waits on event "log file sync" while performing COMMIT and ROLLBACK operations
were consuming significant database time.
RECOMMENDATION 1: Host Configuration, 4% benefit (266 seconds)
ACTION: Investigate the possibility of improving the performance of I/O
to the online redo log files.
RATIONALE: The average size of writes to the online redo log files was
26 K and the average time per write was 2 milliseconds.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Wait class "Commit" was consuming significant database time.
(4% impact [266 seconds])
FINDING 8: 2.9% impact (192 seconds)
Soft parsing of SQL statements was consuming significant database time.
RECOMMENDATION 1: Application Analysis, 2.9% benefit (192 seconds)
ACTION: Investigate application logic to keep open the frequently used
cursors. Note that cursors are closed by both cursor close calls and
session disconnects.
RECOMMENDATION 2: DB Configuration, 2.9% benefit (192 seconds)
ACTION: Consider increasing the maximum number of open cursors a session
can have by increasing the value of parameter "open_cursors".
ACTION: Consider increasing the session cursor cache size by increasing
the value of parameter "session_cached_cursors".
RATIONALE: The value of parameter "open_cursors" was "800" during the
analysis period.
RATIONALE: The value of parameter "session_cached_cursors" was "20"
during the analysis period.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Contention for latches related to the shared pool was consuming
significant database time. (2% impact [135 seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of database
time.
SYMPTOM: Wait class "Concurrency" was consuming significant database
time. (2.3% impact [154 seconds])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ADDITIONAL INFORMATION
Wait class "Application" was not consuming significant database time.
Wait class "Configuration" was not consuming significant database time.
Wait class "Network" was not consuming significant database time.
Session connect and disconnect calls were not consuming significant database
time.
The database's maintenance windows were active during 100% of the analysis
period.
The analysis of I/O performance is based on the default assumption that the
average read time for one database block is 10000 micro-seconds.
An explanation of the terminology used in this report is available when you
run the report with the 'ALL' level of detail.user12023161 wrote:
I have upgraded 10.2.0.3.0 to 10.2.0.5.0 and facing same issue. The database is slow in general after upgrade compared to 10.2.0.3.0.Try setting OPTIMIZER_FEATURE_ENABLE parameter to 10.2.0.3.
Refer following link:
http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/initparams142.htm -
Any general tips on getting better performance out of multi table insert?
I have been struggling with coding a multi table insert which is the first time I ever use one and my Oracle skills are pretty poor in general so now that the query is built and works fine I am sad to see its quite slow.
I have checked numerous articles on optimizing but the things I try dont seem to get me much better performance.
First let me describe my scenario to see if you agree that my performance is slow...
its an insert all command, which ends up inserting into 5 separate tables, conditionally (at least 4 inserts, sometimes 5 but the fifth is the smallest table). Some stats on these tables as follows:
Source table: 5.3M rows, ~150 columns wide. Parallel degree 4. everything else default.
Target table 1: 0 rows, 27 columns wide. Parallel 4. everything else default.
Target table 2: 0 rows, 63 columns wide. Parallel 4. default.
Target table 3: 0 rows, 33 columns wide. Parallel 4. default.
Target table 4: 0 rows, 9 columns wide. Parallel 4. default.
Target table 5: 0 rows, 13 columns wide. Parallel 4. default.
The parallelism is just about the only customization I myself have done. Why 4? I dont know it's pretty arbitrary to be honest.
Indexes?
Table 1 has 3 index + PK.
Table 2 has 0 index + FK + PK.
Table 3 has 4 index + FK + PK
Table 4 has 3 index + FK + PK
Table 5 has 4 index + FK + PK
None of the indexes are anything crazy, maybe 3 or 4 of all of them are on multiple columns, 2-3 max. The rest are on single columns.
The query itself looks something like this:
insert /*+ append */ all
when 1=1 then
into table1 (...) values (...)
into table2 (...) values (...)
when a=b then
into table3 (...) values (...)
when a=c then
into table3 (...) values (...)
when p=q then
into table4(...) values (...)
when x=y then
into table5(...) values (...)
select .... from source_table
Hints I tried are with append, without append, and parallel (though adding parallel seemed to make the query behave in serial, according to my session browser).
Now for the performance:
It does about 8,000 rows per minute on table1. So that means it should also have that much in table2, table3 and table4, and then a subset of that in table5.
Does that seem normal or am I expecting too much?
I find articles talking about millions of rows per minute... Obviously i dont think I can achieve that much... but maybe 30k or so on each table is a reasonable goal?
If it seems my performance is slow, what else do you think I should try? Is there any information I may try to get to see if maybe its a poorly configured database for this?
P.S. Is it possible I can run this so that it commits every x rows or something? I had the heartbreaking event of a network issue giving me this sudden "ora-25402: transaction must roll back" after it was running for 3.5 hours. So I lost all the progress it made... have to start over. plus i wonder if the sheer amount of data being queued for commit/rollback is causing some of the problem?
Edited by: trant on Jun 27, 2011 9:29 PMLooks like there are about 54 sessions on my database, 7 of the sessions belong to me (2 taken by TOAD and 4 by my parallel slave sessions and 1 by the master of those 4)
In v$session_event there are 546 rows, if i filter it to the SIDs of my current session and order my micro_wait_time desc:
510 events in waitclass Other 30670 9161 329759 10.75 196 3297590639 1736664284 1893977003 0 Other
512 events in waitclass Other 32428 10920 329728 10.17 196 3297276553 1736664284 1893977003 0 Other
243 events in waitclass Other 21513 5 329594 15.32 196 3295935977 1736664284 1893977003 0 Other
223 events in waitclass Other 21570 52 329590 15.28 196 3295898897 1736664284 1893977003 0 Other
241 row cache lock 1273669 0 42137 0.03 267 421374408 1714089451 3875070507 4 Concurrency
241 events in waitclass Other 614793 0 34266 0.06 12 342660764 1736664284 1893977003 0 Other
241 db file sequential read 13323 0 3948 0.3 13 39475015 2652584166 1740759767 8 User I/O
241 SQL*Net message from client 7 0 1608 229.65 1566 16075283 1421975091 2723168908 6 Idle
241 log file switch completion 83 0 459 5.54 73 4594763 3834950329 3290255840 2 Configuration
241 gc current grant 2-way 5023 0 159 0.03 0 1591377 2685450749 3871361733 11 Cluster
241 os thread startup 4 0 55 13.82 26 552895 86156091 3875070507 4 Concurrency
241 enq: HW - contention 574 0 38 0.07 0 378395 1645217925 3290255840 2 Configuration
512 PX Deq: Execution Msg 3 0 28 9.45 28 283374 98582416 2723168908 6 Idle
243 PX Deq: Execution Msg 3 0 27 9.1 27 272983 98582416 2723168908 6 Idle
223 PX Deq: Execution Msg 3 0 25 8.26 24 247673 98582416 2723168908 6 Idle
510 PX Deq: Execution Msg 3 0 24 7.86 23 235777 98582416 2723168908 6 Idle
243 PX Deq Credit: need buffer 1 0 17 17.2 17 171964 2267953574 2723168908 6 Idle
223 PX Deq Credit: need buffer 1 0 16 15.92 16 159230 2267953574 2723168908 6 Idle
512 PX Deq Credit: need buffer 1 0 16 15.84 16 158420 2267953574 2723168908 6 Idle
510 direct path read 360 0 15 0.04 4 153411 3926164927 1740759767 8 User I/O
243 direct path read 352 0 13 0.04 6 134188 3926164927 1740759767 8 User I/O
223 direct path read 359 0 13 0.04 5 129859 3926164927 1740759767 8 User I/O
241 PX Deq: Execute Reply 6 0 13 2.12 10 127246 2599037852 2723168908 6 Idle
510 PX Deq Credit: need buffer 1 0 12 12.28 12 122777 2267953574 2723168908 6 Idle
512 direct path read 351 0 12 0.03 5 121579 3926164927 1740759767 8 User I/O
241 PX Deq: Parse Reply 7 0 9 1.28 6 89348 4255662421 2723168908 6 Idle
241 SQL*Net break/reset to client 2 0 6 2.91 6 58253 1963888671 4217450380 1 Application
241 log file sync 1 0 5 5.14 5 51417 1328744198 3386400367 5 Commit
510 cursor: pin S wait on X 3 2 2 0.83 1 24922 1729366244 3875070507 4 Concurrency
512 cursor: pin S wait on X 2 2 2 1.07 1 21407 1729366244 3875070507 4 Concurrency
243 cursor: pin S wait on X 2 2 2 1.06 1 21251 1729366244 3875070507 4 Concurrency
241 library cache lock 29 0 1 0.05 0 13228 916468430 3875070507 4 Concurrency
241 PX Deq: Join ACK 4 0 0 0.07 0 2789 4205438796 2723168908 6 Idle
241 SQL*Net more data from client 6 0 0 0.04 0 2474 3530226808 2000153315 7 Network
241 gc current block 2-way 5 0 0 0.04 0 2090 111015833 3871361733 11 Cluster
241 enq: KO - fast object checkpoint 4 0 0 0.04 0 1735 4205197519 4217450380 1 Application
241 gc current grant busy 4 0 0 0.03 0 1337 2277737081 3871361733 11 Cluster
241 gc cr block 2-way 1 0 0 0.06 0 586 737661873 3871361733 11 Cluster
223 db file sequential read 1 0 0 0.05 0 461 2652584166 1740759767 8 User I/O
223 gc current block 2-way 1 0 0 0.05 0 452 111015833 3871361733 11 Cluster
241 latch: row cache objects 2 0 0 0.02 0 434 1117386924 3875070507 4 Concurrency
241 enq: TM - contention 1 0 0 0.04 0 379 668627480 4217450380 1 Application
512 PX Deq: Msg Fragment 4 0 0 0.01 0 269 77145095 2723168908 6 Idle
241 latch: library cache 3 0 0 0.01 0 243 589947255 3875070507 4 Concurrency
510 PX Deq: Msg Fragment 3 0 0 0.01 0 215 77145095 2723168908 6 Idle
223 PX Deq: Msg Fragment 4 0 0 0 0 145 77145095 2723168908 6 Idle
241 buffer busy waits 1 0 0 0.01 0 142 2161531084 3875070507 4 Concurrency
243 PX Deq: Msg Fragment 2 0 0 0 0 84 77145095 2723168908 6 Idle
241 latch: cache buffers chains 4 0 0 0 0 73 2779959231 3875070507 4 Concurrency
241 SQL*Net message to client 7 0 0 0 0 51 2067390145 2000153315 7 Network
(yikes, is there a way to wrap that in equivalent of other forums' tag?)
v$session_wait;
223 835 PX Deq Credit: send blkd sleeptime/senderid 268697599 000000001003FFFF passes 1 0000000000000001 qref 0 00 1893977003 0 Other 0 10 WAITING
241 22819 row cache lock cache id 13 000000000000000D mode 0 00 request 5 0000000000000005 3875070507 4 Concurrency -1 0 WAITED SHORT TIME
243 747 PX Deq Credit: send blkd sleeptime/senderid 268697599 000000001003FFFF passes 1 0000000000000001 qref 0 00 1893977003 0 Other 0 7 WAITING
510 10729 PX Deq Credit: send blkd sleeptime/senderid 268697599 000000001003FFFF passes 1 0000000000000001 qref 0 00 1893977003 0 Other 0 2 WAITING
512 12718 PX Deq Credit: send blkd sleeptime/senderid 268697599 000000001003FFFF passes 1 0000000000000001 qref 0 00 1893977003 0 Other 0 4 WAITING
v$sess_io:
223 0 5779 5741 0 0
241 38773810 2544298 15107 27274891 0
243 0 5702 5688 0 0
510 0 5729 5724 0 0
512 0 5682 5678 0 0 -
Performance crippled with audit_trail=db in 12.1.0.2 CDB/PDB
Performance tanked as soon as I set audit_trail=db, and all I see in EM Express is buffer busy waits. Tracked it to object sys.aud$ with 100% of buffer busy waits. I have been using Oracle auditing for years and never see this type of impact. Anyone else see this?
Oracle 12.1.0.2 on 64-bit Oracle Linux 6.5 with kernel 3.8.13-44.el6uek.x86_64.
Created container database using DBCA with one pluggable database.
Ran benchmarks to baseline performance. Very good.
set audit_trail=db, bounced the database, logged into the PDB and enabled audit on select, insert, update, delete on all my application tables.
Transactions per minute dropped 95% and write I/O throughput dropped 75%.
I collected AWR reports with and without audit_trail set. These are shown below.
Each benchmark is 5 minutes with 2 minute ramp-up time, and 40 users no key/think time.
BEFORE SETTING AUDIT_TRAIL (IT IS UNSET). Buffer Busy Waits is only 1.2 seconds total.
Top 10 Foreground Events by Total Wait Time
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Total Wait Wait % DB Wait
Event Waits Time (sec) Avg(ms) time Class
log file sync 2,500,409 6224.2 2.49 54.9 Commit
DB CPU 5166.3 45.6
enq: TX - row lock contention 16,155 14.3 0.88 .1 Applicat
library cache: mutex X 83,598 13.6 0.16 .1 Concurre
db file sequential read 87,145 10.4 0.12 .1 User I/O
SQL*Net message to client 2,739,811 8.3 0.00 .1 Network
cursor: pin S 3,631 3.9 1.07 .0 Concurre
latch: In memory undo latch 86,392 2.9 0.03 .0 Concurre
buffer busy waits 45,740 1.2 0.03 .0 Concurre
cursor: mutex X 1,004 .9 0.94 .0 Concurre
NOW RE-RUN SAME BENCHMARK WITH AUDIT_TRAIL = DB. Buffer Busy Waits over 8000 seconds!
Top 10 Foreground Events by Total Wait Time
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Total Wait Wait % DB Wait
Event Waits Time (sec) Avg(ms) time Class
buffer busy waits 12,024,693 8002.1 0.67 68.1 Concurre
DB CPU 3253.5 27.7
log file sync 151,536 255.6 1.69 2.2 Commit
enq: TX - row lock contention 5,549 199.6 35.98 1.7 Applicat
enq: TX - contention 959 110.3 114.98 .9 Other
enq: HW - contention 177,020 95.9 0.54 .8 Configur
Data file init write 2,232 1.9 0.84 .0 User I/O
db file scattered read 19,037 1.7 0.09 .0 User I/O
db file sequential read 19,363 1.4 0.07 .0 User I/O
library cache: mutex X 36,060 1.2 0.03 .0 Concurre
Buffer Wait Statistics DB/Inst: NORM/NORM Snaps: 60-61
-> ordered by wait time desc, waits desc
Class Waits Total Wait Time (s) Avg Time (ms)
data block 14,204,039 7,922 1
segment header 1,288,059 22 0
undo header 286,303 2 0
1st level bmb 1 0 0
Segments by Buffer Busy Waits DB/Inst: NORM/NORM Snaps: 60-61
-> % of Capture shows % of Buffer Busy Waits for each top segment compared
-> with total Buffer Busy Waits for all segments captured by the Snapshot
-> When ** MISSING ** occurs, some of the object attributes may not be available
Tablespace
Owner Name
Buffer
Subobject Obj. Busy % of
Object Name Name Type Obj# Dataobj# Waits Capture
SYS SYSTEM
AUD$ TABLE 499 499 15,496,081 100.00Hi,
Happened to me that parameter values of a workspace were defaulting to the values in <configdir>/defaults.xml instead of the ones in the <workspace>,xml.
When you setup a database (e.g..java -jar ords.war setup --database mydb), you will have mydb.xml, mydb_al.xml and mydb_rt.xml under the <configdir>/ords created for APEX_PUBLIC_USER, APEX_LISTENER and APEX_REST_PUBLIC_USER, respectively. When a config entry is not present in mydb_al.xml or mydb_rt.xml, one would expect that it would be read from mydb.xml. However, that is not what happens. It reads from defaults.xml under the configdir.
That is the case for db connection parameters as well.
So, inspect all three xmls and make sure entries with
key="db.hostname"
key="db.password"
key="db.port"
key="db.servicename"
exist in all three xmls.
P.S. That should be the case for other parameters as well, but haven't verified.
Ege -
Can you please provide me the suggestion on how to reduce the number of buffer busy waits for the below query?
Please find the query where the buffer busy waits is taking 11091 and 13160 seconds.
INSERT INTO RPM_CLEARANCE (CLEARANCE_ID, CLEARANCE_DISPLAY_ID, STATE,
REASON_CODE, CLEARANCE_RESET_ID, RESET_IND, ITEM, ZONE_ID, LOCATION,
ZONE_NODE_TYPE, EFFECTIVE_DATE, OUT_OF_STOCK_DATE, RESET_DATE, CHANGE_TYPE,
CHANGE_AMOUNT, CHANGE_PERCENT, CHANGE_CURRENCY, VENDOR_FUNDED_IND,
CREATE_DATE, CREATE_ID, APPROVAL_DATE, APPROVAL_ID, TSL_EVENT_REF,
TSL_MARKDOWN_REF, TSL_EVENT_PHASE, TSL_COVER_GROUP, TSL_END_DATE,
TSL_EVENT_POS_IND, TSL_EVENT_SEL_IND, TSL_HOPOS_TEMPLATE_ID )
VALUES
(:B23 , 'reset:'||:B22 , :B21 , :B20 , NULL, '1', :B19 , :B18 , :B17 , :B16 ,
:B15 , :B15 - 1, NULL, :B14 , :B13 , :B12 , :B11 , '0', :B10 , :B9 , :B10 ,
:B9 , :B8 , :B7 , :B6 , :B5 , :B4 , :B3 , :B2 , :B1 )
call count cpu elapsed disk query current rows
Parse 1849 0.06 0.05 0 0 0 0
Execute 1895091 2539.06 2606.23 32 119694 31084693 1895091
Fetch 0 0.00 0.00 0 0 0 0
total 1896940 2539.12 2606.28 32 119694 31084693 1895091
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 202 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 32 0.01 0.22
buffer busy waits 11091 0.02 0.29
latch: cache buffers chains 30 0.00 0.00
enq: TX - index contention 563 0.01 0.11
log file switch completion 15 0.06 0.42
latch: In memory undo latch 7 0.00 0.00
buffer deadlock 88 0.00 0.00
cursor: pin S 59 0.00 0.00
cursor: pin S wait on X 1 0.00 0.00
INSERT INTO RPM_CLEARANCE (CLEARANCE_ID, CLEARANCE_DISPLAY_ID, STATE,
REASON_CODE, CLEARANCE_RESET_ID, RESET_IND, ITEM, ZONE_ID, LOCATION,
ZONE_NODE_TYPE, EFFECTIVE_DATE, OUT_OF_STOCK_DATE, RESET_DATE, CHANGE_TYPE,
CHANGE_AMOUNT, CHANGE_PERCENT, CHANGE_CURRENCY, VENDOR_FUNDED_IND,
CREATE_DATE, CREATE_ID, APPROVAL_DATE, APPROVAL_ID, TSL_EVENT_REF,
TSL_MARKDOWN_REF, TSL_EVENT_PHASE, TSL_COVER_GROUP, TSL_END_DATE,
TSL_EVENT_POS_IND, TSL_EVENT_SEL_IND, TSL_HOPOS_TEMPLATE_ID )
VALUES
(:B23 , 'reset:'||:B22 , :B21 , :B20 , NULL, '1', :B19 , :B18 , :B17 , :B16 ,
:B15 , :B15 - 1, NULL, :B14 , :B13 , :B12 , :B11 , '0', :B10 , :B9 , :B10 ,
:B9 , :B8 , :B7 , :B6 , :B5 , :B4 , :B3 , :B2 , :B1 )
call count cpu elapsed disk query current rows
Parse 575 0.02 0.01 0 0 0 0
Execute 1066687 1460.30 1478.77 0 121065 17556349 1066687
Fetch 0 0.00 0.00 0 0 0 0
total 1067262 1460.32 1478.79 0 121065 17556349 1066687
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 202 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
latch: cache buffers chains 28 0.01 0.01
buffer busy waits 13160 0.07 0.46
enq: TX - index contention 522 0.00 0.09
buffer deadlock 108 0.00 0.00
latch: In memory undo latch 17 0.00 0.00
cursor: pin S 8 0.00 0.00
log file switch completion 2 0.05 0.07
cursor: pin S wait on X 1 0.00 0.00
********************************************************************************For reducing buffer busy waits you need to increase DB_CACHE_SIZE parameter value.
Try the query below
14:40:23 SQL> select name,size_for_estimate,size_factor,estd_physical_reads,buffers_for_estimate fro
m v$db_cache_advice order by 4;
NAME SIZE_FOR_ESTIMATE SIZE_FACTOR ESTD_PHYSICAL_READS BUFFERS_FOR_ESTIMATE
DEFAULT 960 1.875 252983 119760
DEFAULT 912 1.7813 268597 113772
DEFAULT 864 1.6875 269415 107784
DEFAULT 816 1.5938 269673 101796
DEFAULT 768 1.5 270449 95808
DEFAULT 720 1.4063 270923 89820
DEFAULT 672 1.3125 272107 83832
DEFAULT 624 1.2188 276651 77844
DEFAULT 576 1.125 282272 71856
DEFAULT 528 1.0313 308869 65868
DEFAULT 512 1 334346 63872
DEFAULT 480 .9375 411617 59880
DEFAULT 432 .8438 467955 53892
DEFAULT 384 .75 520223 47904
DEFAULT 336 .6563 575829 41916
DEFAULT 288 .5625 628226 35928
DEFAULT 240 .4688 670286 29940
DEFAULT 192 .375 725289 23952
DEFAULT 144 .2813 784512 17964
DEFAULT 96 .1875 921481 11976
DEFAULT 48 .0938 1948144 5988
21 rows selected.You will get a result like above. SIZE_FOR_ESTIMATE column shows values in MB.
Then from the result, choose a size for the buffer cache in which there is minimal physical reads but according to the avaliability of your physical memory and set it. -
Slow query performance in Oracle 10.2.0.3
Hi,
We have Oracle 10.2.0.3 installed on RHEL 5(64 bit).We have two queries out of which one is a query using select while other query is using a insert.First we executed insert query which inserts 10000 rows in a table and then select query on this table.This works fine in one thread.But when we do samething in 10 threads, at that time INSERT is fine but select is taking very long time for 10 threads.Any bug related to parallel execution of queries for SELECT in 10.2.0.3?Any suggestion??
Thanks in advance.
Regards,
RJ.Justin,
We have a same queries for INSERT and Select in 10 manual sessions outof which select query is taking more time to execute.Please refer to WAITs given below.No there is no bottleneck as far as hardware is concerned because we tested it on different configuration of servers.
Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
CPU time 52 93.2
latch: cache buffers chains 45,542 6 0 10.7 Concurrency
log file parallel write 2,107 3 1 5.2 System I/O
log file sync 805 2 2 3.5 Commit
latch: session allocation 5,116 1 0 2.6 Other Wait Events
• s - second
• cs - centisecond - 100th of a second
• ms - millisecond - 1000th of a second
• us - microsecond - 1000000th of a second
• ordered by wait time desc, waits desc (idle events last)
Event Waits %Time-outs Total Wait Time (s) Avg wait (ms) Waits /txn
latch: cache buffers chains 45,542 0.00 6 0 22.99
log file parallel write 2,107 0.00 3 1 1.06
log file sync 805 0.00 2 2 0.41
latch: session allocation 5,116 0.00 1 0 2.58
buffer busy waits 20,482 0.00 1 0 10.34
db file sequential read 157 0.00 1 4 0.08
control file parallel write 1,330 0.00 0 0 0.67
wait list latch free 39 0.00 0 10 0.02
enq: TX - index contention 632 0.00 0 0 0.32
latch free 996 0.00 0 0 0.50
SQL*Net break/reset to client 1,738 0.00 0 0 0.88
SQL*Net message to client 108,947 0.00 0 0 55.00
os thread startup 2 0.00 0 19 0.00
cursor: pin S wait on X 3 100.00 0 11 0.00
latch: In memory undo latch 136 0.00 0 0 0.07
log file switch completion 4 0.00 0 7 0.00
latch: shared pool 119 0.00 0 0 0.06
latch: undo global data 121 0.00 0 0 0.06
buffer deadlock 238 99.58 0 0 0.12
control file sequential read 1,735 0.00 0 0 0.88
SQL*Net more data to client 506 0.00 0 0 0.26
log file single write 2 0.00 0 2 0.00
SQL*Net more data from client 269 0.00 0 0 0.14
reliable message 12 0.00 0 0 0.01
LGWR wait for redo copy 26 0.00 0 0 0.01
rdbms ipc reply 6 0.00 0 0 0.00
latch: library cache 7 0.00 0 0 0.00
latch: redo allocation 2 0.00 0 0 0.00
enq: RO - fast object reuse 2 0.00 0 0 0.00
direct path write 21 0.00 0 0 0.01
cursor: pin S 1 0.00 0 0 0.00
log file sequential read 2 0.00 0 0 0.00
direct path read 8 0.00 0 0 0.00
SQL*Net message from client 108,949 0.00 43,397 398 55.00
jobq slave wait 14,527 49.56 35,159 2420 7.33
Streams AQ: qmn slave idle wait 246 0.00 3,524 14326 0.12
Streams AQ:qmn coordinator-
idle wait 451 45.45 3,524 7814 0.23
wait for unread message on -
broadcast channel 3,597 100.00 3,516 978 1.82
virtual circuit status 120 100.00 3,516 29298 0.06
class slave wait 2 0.00 0 0 0.00 Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv
Message was edited by:
RJiv -
STATSPACK REPORT (BUFFER HIT RATIO)
my statspack report shows that my buffer ration is 83%...what factors i need to look to imporve the buffer hit ratio. Thanks
I deleted because i realized that i took the statspack report of 1 day period.
Below is the Statspack report of 1 hour. Can you please let me know if i still need to increase database buffer cache?
STATSPACK report for
Database DB Id Instance Inst Num Startup Time Release RAC
~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
4254163 TEST1 1 28-Jun-07 23:30 10.2.0.3.0 NO
Host Name: Linux3 Num CPUs: 2 Phys Memory (MB): 7,968
~~~~
Snapshot Snap Id Snap Time Sessions Curs/Sess Comment
~~~~~~~~ ---------- ------------------ -------- --------- -------------------
Begin Snap: 32 03-Jul-07 11:59:13 23 11.0
End Snap: 42 03-Jul-07 14:07:33 26 11.3
Elapsed: 128.33 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 100M Std Block Size: 8K
Shared Pool Size: 100M Log Buffer: 33,823K
Load Profile Per Second Per Transaction
~~~~~~~~~~~~ --------------- ---------------
Redo size: 1,259.57 8,598.13
Logical reads: 148.39 1,012.92
Block changes: 6.41 43.76
Physical reads: 41.91 286.09
Physical writes: 0.73 5.02
User calls: 15.66 106.91
Parses: 4.07 27.77
Hard parses: 0.27 1.85
Sorts: 1.70 11.61
Logons: 0.01 0.07
Executes: 9.59 65.47
Transactions: 0.15
% Blocks changed per Read: 4.32 Recursive Call %: 83.09
Rollback per transaction %: 6.03 Rows per Sort: 11.39
Instance Efficiency Percentages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 71.77 In-memory Sort %: 100.00
Library Hit %: 93.15 Soft Parse %: 93.34
Execute to Parse %: 57.58 Latch Hit %: 100.00
Parse CPU to Parse Elapsd %: 97.12 % Non-Parse CPU: 86.74
Shared Pool Statistics Begin End
Memory Usage %: 91.37 92.38
% SQL with executions>1: 77.55 80.43
% Memory for SQL w/exec>1: 83.11 84.69
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
CPU time 132 48.3
db file sequential read 89,745 91 1 33.4
db file scattered read 29,289 35 1 13.0
control file parallel write 2,558 6 2 2.1
log file parallel write 2,294 3 1 1.0
Host CPU (CPUs: 2)
~~~~~~~~ Load Average
Begin End User System Idle WIO WCPU
0.11 0.11 2.26 2.65 95.09 0.90 0.24
Instance CPU
~~~~~~~~~~~~
% of total CPU for Instance: 1.06
% of busy CPU for Instance: 21.63
%DB time waiting for CPU - Resource Mgr:
Memory Statistics Begin End
~~~~~~~~~~~~~~~~~ ------------ ------------
Host Mem (MB): 7,967.6 7,967.6
SGA use (MB): 316.0 316.0
PGA use (MB): 57.8 62.6
% Host Mem used for SGA+PGA: 4.7 4.8
Time Model System Stats DB/Inst: TEST1/TEST1 Snaps: 32-42
-> Ordered by % of DB time desc, Statistic name
Statistic Time (s) % of DB time
sql execute elapsed time 212.3 92.7
DB CPU 124.2 54.2
parse time elapsed 21.6 9.4
hard parse elapsed time 19.7 8.6
PL/SQL execution elapsed time 4.3 1.9
hard parse (sharing criteria) elaps 1.4 .6
connection management call elapsed 1.4 .6
PL/SQL compilation elapsed time 1.2 .5
repeated bind elapsed time 0.1 .0
hard parse (bind mismatch) elapsed 0.1 .0
sequence load elapsed time 0.0 .0
DB time 228.9
background elapsed time 48.2
background cpu time 39.3
Wait Events DB/Inst: TEST1/TEST1 Snaps: 32-42
-> s - second, cs - centisecond, ms - millisecond, us - microsecond
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
db file sequential read 89,745 0 91 1 79.6
db file scattered read 29,289 0 35 1 26.0
control file parallel write 2,558 0 6 2 2.3
log file parallel write 2,294 0 3 1 2.0
db file parallel write 2,179 0 3 1 1.9
log file sync 1,089 0 2 2 1.0
os thread startup 7 0 1 120 0.0
latch free 3 0 0 89 0.0
SQL*Net break/reset to client 640 0 0 0 0.6
direct path read 140 0 0 1 0.1
control file sequential read 3,599 0 0 0 3.2
SQL*Net more data to client 2,121 0 0 0 1.9
db file parallel read 49 0 0 1 0.0
cursor: pin S wait on X 2 100 0 16 0.0
read by other session 4 0 0 5 0.0
direct path write 24 0 0 0 0.0
latch: shared pool 1 0 0 2 0.0
SQL*Net message from client 120,211 0 47,282 393 106.6
wait for unread message on broadc 7,631 100 7,517 985 6.8
Streams AQ: waiting for messages 1,540 100 7,512 4878 1.4
Streams AQ: qmn slave idle wait 275 0 7,508 27302 0.2
Streams AQ: qmn coordinator idle 554 51 7,508 13553 0.5
Streams AQ: waiting for time mana 25 52 6,643 ###### 0.0
SQL*Net message to client 120,215 0 0 0 106.6
class slave wait 7 0 0 1 0.0
SQL*Net more data from client 146 0 0 0 0.1
Background Wait Events DB/Inst: TEST1/TEST1 Snaps: 32-42
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
control file parallel write 2,557 0 6 2 2.3
log file parallel write 2,290 0 3 1 2.0
db file parallel write 2,179 0 3 1 1.9
os thread startup 7 0 1 120 0.0
db file sequential read 1,456 0 1 0 1.3
db file scattered read 25 0 0 8 0.0
control file sequential read 156 0 0 0 0.1
latch: shared pool 1 0 0 2 0.0
rdbms ipc message 25,017 92 59,496 2378 22.2
pmon timer 2,576 100 7,513 2917 2.3
Streams AQ: qmn slave idle wait 275 0 7,508 27302 0.2
Streams AQ: qmn coordinator idle 554 51 7,508 13553 0.5
smon timer 26 96 7,148 ###### 0.0
Streams AQ: waiting for time mana 25 52 6,643 ###### 0.0
Wait Event Histogram DB/Inst: TEST1/TEST1 Snaps: 32-42
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
LGWR wait for redo copy 7 100.0
SQL*Net break/reset to cli 640 99.2 .6 .2
SQL*Net more data to clien 2121 100.0
control file parallel writ 2558 84.2 12.0 .7 1.4 1.5 .2
control file sequential re 3599 99.9 .1
cursor: pin S wait on X 2 100.0
db file parallel read 49 93.9 2.0 4.1
db file parallel write 2179 68.2 19.9 6.8 4.0 .9 .1 .1
db file scattered read 29K 90.7 6.0 .5 .5 .6 .8 .9
db file sequential read 89K 89.4 2.8 1.3 3.6 1.5 .7 .6
direct path read 140 87.1 2.9 .7 1.4 7.1 .7
direct path write 24 100.0
latch free 3 100.0
latch: messages 1 100.0
latch: shared pool 1 100.0
log file parallel write 2294 77.4 17.3 2.0 1.3 1.1 .8 .2
log file sync 1089 62.4 28.8 3.3 1.7 2.5 1.1 .2
os thread startup 7 100.0
read by other session 4 50.0 25.0 25.0
SQL*Net message from clien 120K 95.2 1.6 .9 .3 .1 .2 .1 1.7
SQL*Net message to client 120K 100.0
SQL*Net more data from cli 146 100.0
Streams AQ: qmn coordinato 554 49.1 .2 .2 50.5
Streams AQ: qmn slave idle 275 100.0
Streams AQ: waiting for me 1540 .2 99.8
Streams AQ: waiting for ti 25 36.0 16.0 48.0
class slave wait 7 85.7 14.3
pmon timer 2577 .5 .1 .1 99.3
rdbms ipc message 25K 2.3 1.3 1.4 .4 .4 .3 32.1 61.8
smon timer 26 100.0
wait for unread message on 7631 .0 .0 100.0 .0
Maybe you are looking for
-
I cannot find the Firefox Start page with the Sync Set Up Link
I'm trying to get the Sync code on my android device in order to retrieve all my bookmarks from my main computer to the tablet pc. The directions on the Mozilla support site say to "Tap the Set up Firefox Sync button on the Firefox Start page." Well
-
How do you reinstall OS X Mountain Lion?
I have had a problem and I was wondering how you take off Mountain lion and reinstall it onto the OS?
-
i know how to embed a single video from youtube in muse, but how do i go about creating a video playlist of several videos?
-
I can't find a discussion area dedicated to Quicktime or Darwin Streaming Server technologies, so since I am working on a MacBook Pro I'll put my initial question here... if anyone wants to direct me to a more appropriate forum, please do! The proble
-
How much cost a keyboard backlight replacement ?
Hi, I just spill coffee on my MacBook pro (mid 2010). I was lucky, everything working fine, except the keyboard backlight :/ How much is it to replace it in a apple store ? Thanks