100% CPU, wait event : latch shared pool
I have a store procedure, run in one of database, it hangs in a "create table ... as select ..." statement.
the wait event is : latch shared pool, and CPU is up to 100%, it has run over few hours and seems hang.
Same stored procedure run on others enviroment, never seen such problem, even run on the same data size or even much much bigger data size.
This procedure has been used more than 2 years, never see such problem in any others enviroment. it only happend in this new setup enviroment.
however, in this enviroment, if I try to reduce data to be very very small, I was able to see procudure complate in 10 sec.
I suspect parameter, for example, I changed shared_pool_size from 40MB to 150 MB, re-start database and re-run, still see the same problem here.
Could anybody suggest any thing I can look into?
Thanks
jjzz wrote:
I have a store procedure, run in one of database, it hangs in a "create table ... as select ..." statement.
the wait event is : latch shared pool, and CPU is up to 100%, it has run over few hours and seems hang.
If it's running at 100% CPU, it's not waiting.
Does v$session_wait (or even v$session since you seem to be running 10g) tell you that the session is *"waiting"*, or is it simply noting that your last wait was on the shared pool latch ?
If the latter, then you probably have some SQL in the procedure that has changed its execution plan to become much more CPU intensive - perhaps because of a small change in the data volume, data distribution, or statistics.
First step - find out what SQL statements are executing, and see how much work they are doing. You could query v$session for that session a few times and check what the sql_id and sql_child number are, also prev_sql_id and prev_child_number. If these stay constant, one or other may give you the guilty SQL statement. If not check v$open_cursor for the session.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
"Science is more than a body of knowledge; it is a way of thinking"
Carl Sagan
Similar Messages
-
CPU wait events on ADDM report
Hello,
My Oracle version is:
Connected to Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 Yesterday I was taking a look on an ADDM report and spot the following:
Rationale
SQL statement with SQL_ID "0mgk8gx9hj71d" was executed 777 times and had
an average elapsed time of 42 seconds.
Rationale
Waiting for event "resmgr:cpu quantum" in wait class "Scheduler"
accounted for 34% of the database time spent in processing the SQL
statement with SQL_ID "0mgk8gx9hj71d".After that, I started looking for how ADDM could know that the SQL_ID "0mgk8gx9hj71d" waited 34% on "resmgr:cpu quantum" event. No lucky with that...
The only wait event information related to a given SQL_ID I've found on v$active_session_history (or the AWR persisted table for it), but in the ASH there is no information about CPU wait events like "cpu quantum". When the session is waiting for CPU, there is no event related in v$ash.
So, my question is: where ADDM got the information that the SQL waited 34% of the time on "resmgr:cpu quantum"?
Thanks,
Heitor KirstenHi,
Is a session waiting for CPU resources ("res:cpu quantum") considered as an active session ? Maybe not.
I guess (I made no test) that this maybe the reason why this kind of wait is not shown in the active session history .
Regards
Maurice -
DB CPU wait event is high in AWR
Hello Experts,
Could you please tell me what are the causes of increasing DB CPU wait event ? I mentioned below which i know. please guide me
1. When Buffer cache is more than required then DB CPU wait event occur.
Regards,
SachinSachin.Ichake wrote:
Currently in my database DB CPU has taken 90% DB time . in accordance to resolve it I will gonna follow steps
1. Tune the query which has taken more cpu
2. Decrease Buffer cache size by referring buffer cache advisory.Solve what? You must understand that DB CPU is not shown as a Wait Event but as a Timed Event and so are the other events that are shown in the Top 5 Timed Events category. This is an indication of how much you have used in the comparison of the total DB Time but not necessarily , it's an issue as to do anything in the system, you would need to burn the CPU only. You need to check that how much total CPU time you have with you and then compare it with your DB CPU seconds. In addition to this, you also need to check the CPU consumption from the o/s commands like Top etc. Combining all of such information only would be able to help to understand that whether any tuning needs to be done or not.
Post here the AWR/Statspack report. That would give more clear picture of the things.
Aman.... -
Hi: I'm analyzing this STATSPACK report: it is "volume test" on our UAT server, so most input is from 'bind variables'. Our shared pool is well utilized in oracle. Oracle redo logs is not appropriately configured on this server, as in 'Top 5 wait events' there are 2 for redos.
I need to know what else information can be dig-out from 'foreground wait events' & 'background wait events', and what can assist us to better understanding, in combination of 'Top 5 wait event's, that how the server/test went? it could be overwelming No. of wait events, so appreciate any helpful diagnostic or analysis. Database is oracle 11.2.0.4 upgraded from 11.2.0.3, on IBM AIX power system 64bit, level 6.x
STATSPACK report for
Database DB Id Instance Inst Num Startup Time Release RAC
~~~~~~~~ ----------- ------------ -------- --------------- ----------- ---
700000XXX XXX 1 22-Apr-15 12:12 11.2.0.4.0 NO
Host Name Platform CPUs Cores Sockets Memory (G)
~~~~ ---------------- ---------------------- ----- ----- ------- ------------
dXXXX_XXX AIX-Based Systems (64- 2 1 0 16.0
Snapshot Snap Id Snap Time Sessions Curs/Sess Comment
~~~~~~~~ ---------- ------------------ -------- --------- ------------------
Begin Snap: 5635 22-Apr-15 13:00:02 114 4.6
End Snap: 5636 22-Apr-15 14:00:01 128 8.8
Elapsed: 59.98 (mins) Av Act Sess: 0.6
DB time: 35.98 (mins) DB CPU: 19.43 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 2,064M Std Block Size: 8K
Shared Pool: 3,072M Log Buffer: 13,632K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ ------------------ ----------------- ----------- -----------
DB time(s): 0.6 0.0 0.00 0.00
DB CPU(s): 0.3 0.0 0.00 0.00
Redo size: 458,720.6 8,755.7
Logical reads: 12,874.2 245.7
Block changes: 1,356.4 25.9
Physical reads: 6.6 0.1
Physical writes: 61.8 1.2
User calls: 2,033.7 38.8
Parses: 286.5 5.5
Hard parses: 0.5 0.0
W/A MB processed: 1.7 0.0
Logons: 1.2 0.0
Executes: 801.1 15.3
Rollbacks: 6.1 0.1
Transactions: 52.4
Instance Efficiency Indicators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.98 Optimal W/A Exec %: 100.00
Library Hit %: 99.77 Soft Parse %: 99.82
Execute to Parse %: 64.24 Latch Hit %: 99.98
Parse CPU to Parse Elapsd %: 53.15 % Non-Parse CPU: 98.03
Shared Pool Statistics Begin End
Memory Usage %: 10.50 12.79
% SQL with executions>1: 69.98 78.37
% Memory for SQL w/exec>1: 70.22 81.96
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
CPU time 847 50.2
enq: TX - row lock contention 4,480 434 97 25.8
log file sync 284,169 185 1 11.0
log file parallel write 299,537 164 1 9.7
log file sequential read 698 16 24 1.0
Host CPU (CPUs: 2 Cores: 1 Sockets: 0)
~~~~~~~~ Load Average
Begin End User System Idle WIO WCPU
1.16 1.84 19.28 14.51 66.21 1.20 82.01
Instance CPU
~~~~~~~~~~~~ % Time (seconds)
Host: Total time (s): 7,193.8
Host: Busy CPU time (s): 2,430.7
% of time Host is Busy: 33.8
Instance: Total CPU time (s): 1,203.1
% of Busy CPU used for Instance: 49.5
Instance: Total Database time (s): 2,426.4
%DB time waiting for CPU (Resource Mgr): 0.0
Memory Statistics Begin End
~~~~~~~~~~~~~~~~~ ------------ ------------
Host Mem (MB): 16,384.0 16,384.0
SGA use (MB): 7,136.0 7,136.0
PGA use (MB): 282.5 361.4
% Host Mem used for SGA+PGA: 45.3 45.8
Foreground Wait Events DB/Inst: XXXXXs Snaps: 5635-5636
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
enq: TX - row lock contentio 4,480 0 434 97 0.0 25.8
log file sync 284,167 0 185 1 1.5 11.0
Disk file operations I/O 8,741 0 4 0 0.0 .2
direct path write 13,247 0 3 0 0.1 .2
db file sequential read 6,058 0 1 0 0.0 .1
buffer busy waits 1,800 0 1 1 0.0 .1
SQL*Net more data to client 29,161 0 1 0 0.2 .1
direct path read 7,696 0 1 0 0.0 .0
db file scattered read 316 0 1 2 0.0 .0
latch: shared pool 144 0 0 2 0.0 .0
CSS initialization 30 0 0 3 0.0 .0
cursor: pin S 10 0 0 9 0.0 .0
row cache lock 41 0 0 2 0.0 .0
latch: row cache objects 19 0 0 3 0.0 .0
log file switch (private str 8 0 0 7 0.0 .0
library cache: mutex X 28 0 0 2 0.0 .0
latch: cache buffers chains 54 0 0 1 0.0 .0
latch free 290 0 0 0 0.0 .0
control file sequential read 1,568 0 0 0 0.0 .0
log file switch (checkpoint 4 0 0 6 0.0 .0
direct path sync 8 0 0 3 0.0 .0
latch: redo allocation 60 0 0 0 0.0 .0
SQL*Net break/reset to clien 34 0 0 1 0.0 .0
latch: enqueue hash chains 45 0 0 0 0.0 .0
latch: cache buffers lru cha 7 0 0 2 0.0 .0
latch: session allocation 5 0 0 1 0.0 .0
latch: object queue header o 6 0 0 1 0.0 .0
ASM file metadata operation 30 0 0 0 0.0 .0
latch: In memory undo latch 15 0 0 0 0.0 .0
latch: undo global data 8 0 0 0 0.0 .0
SQL*Net message from client 6,362,536 0 278,225 44 33.7
jobq slave wait 7,270 100 3,635 500 0.0
SQL*Net more data from clien 7,976 0 15 2 0.0
SQL*Net message to client 6,362,544 0 8 0 33.7
Background Wait Events DB/Inst: XXXXXs Snaps: 5635-5636
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
log file parallel write 299,537 0 164 1 1.6 9.7
log file sequential read 698 0 16 24 0.0 1.0
db file parallel write 9,556 0 13 1 0.1 .8
os thread startup 146 0 10 70 0.0 .6
control file parallel write 2,037 0 2 1 0.0 .1
Log archive I/O 35 0 1 30 0.0 .1
LGWR wait for redo copy 2,447 0 0 0 0.0 .0
db file async I/O submit 9,556 0 0 0 0.1 .0
db file sequential read 145 0 0 2 0.0 .0
Disk file operations I/O 349 0 0 0 0.0 .0
db file scattered read 30 0 0 4 0.0 .0
control file sequential read 5,837 0 0 0 0.0 .0
ADR block file read 19 0 0 4 0.0 .0
ADR block file write 5 0 0 15 0.0 .0
direct path write 14 0 0 2 0.0 .0
direct path read 3 0 0 7 0.0 .0
latch: shared pool 3 0 0 6 0.0 .0
log file single write 56 0 0 0 0.0 .0
latch: redo allocation 53 0 0 0 0.0 .0
latch: active service list 1 0 0 3 0.0 .0
latch free 11 0 0 0 0.0 .0
rdbms ipc message 314,523 5 57,189 182 1.7
Space Manager: slave idle wa 4,086 88 18,996 4649 0.0
DIAG idle wait 7,185 100 7,186 1000 0.0
Streams AQ: waiting for time 2 50 4,909 ###### 0.0
Streams AQ: qmn slave idle w 129 0 3,612 28002 0.0
Streams AQ: qmn coordinator 258 50 3,612 14001 0.0
smon timer 43 2 3,605 83839 0.0
pmon timer 1,199 99 3,596 2999 0.0
SQL*Net message from client 17,019 0 31 2 0.1
SQL*Net message to client 12,762 0 0 0 0.1
class slave wait 28 0 0 0 0.0
thank you very much!Hi: just know it now: it is a large amount of 'concurrent transaction' designed in this "Volume Test" - to simulate large incoming transaction volme, so I guess wait in eq:TX - row is expected.
The fact: (1) redo logs at uat server is known to not well-tune for configurations (2) volume test slow 5%, however data amount in its test is kept the same by each time import production data, by the team. So why it slowed 5% this year?
The wait histogram is pasted below, any one interest to take a look? any ideas?
Wait Event Histogram DB/Inst: XXXX/XXXX Snaps: 5635-5636
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
ADR block file read 19 26.3 5.3 10.5 57.9
ADR block file write 5 40.0 60.0
ADR file lock 6 100.0
ARCH wait for archivelog l 14 100.0
ASM file metadata operatio 30 100.0
CSS initialization 30 100.0
Disk file operations I/O 9090 97.2 1.4 .6 .4 .2 .1 .1
LGWR wait for redo copy 2447 98.5 .5 .4 .2 .2 .2 .1
Log archive I/O 35 40.0 8.6 25.7 2.9 22.9
SQL*Net break/reset to cli 34 85.3 8.8 5.9
SQL*Net more data to clien 29K 99.9 .0 .0 .0 .0 .0
buffer busy waits 1800 96.8 .7 .7 .6 .3 .4 .5
control file parallel writ 2037 90.7 5.0 2.1 .8 1.0 .3 .1
control file sequential re 7405 100.0 .0
cursor: pin S 10 10.0 90.0
db file async I/O submit 9556 99.9 .0 .0 .0
db file parallel read 1 100.0
db file parallel write 9556 62.0 32.4 1.7 .8 1.5 1.3 .1
db file scattered read 345 72.8 3.8 2.3 11.6 9.0 .6
db file sequential read 6199 97.2 .2 .3 1.6 .7 .0 .0
direct path read 7699 99.1 .4 .2 .1 .1 .0
direct path sync 8 25.0 37.5 12.5 25.0
direct path write 13K 97.8 .9 .5 .4 .3 .1 .0
enq: TX - row lock content 4480 .4 .7 1.3 3.0 6.8 12.3 75.4 .1
latch free 301 98.3 .3 .7 .7
latch: In memory undo latc 15 93.3 6.7
latch: active service list 1 100.0
latch: cache buffers chain 55 94.5 3.6 1.8
latch: cache buffers lru c 9 88.9 11.1
latch: call allocation 6 100.0
latch: checkpoint queue la 3 100.0
latch: enqueue hash chains 45 97.8 2.2
latch: messages 4 100.0
latch: object queue header 7 85.7 14.3
latch: redo allocation 113 97.3 1.8 .9
latch: row cache objects 19 89.5 5.3 5.3
latch: session allocation 5 80.0 20.0
latch: shared pool 147 90.5 1.4 2.7 1.4 .7 1.4 2.0
latch: undo global data 8 100.0
library cache: mutex X 28 89.3 3.6 3.6 3.6
log file parallel write 299K 95.6 2.6 1.0 .4 .3 .2 .0
log file sequential read 698 29.5 .1 4.6 46.8 18.9
log file single write 56 100.0
log file switch (checkpoin 4 25.0 50.0 25.0
log file switch (private s 8 12.5 37.5 50.0
log file sync 284K 93.3 3.7 1.4 .7 .5 .3 .1
os thread startup 146 100.0
row cache lock 41 85.4 9.8 2.4 2.4
DIAG idle wait 7184 100.0
SQL*Net message from clien 6379K 86.6 5.1 2.9 1.3 .7 .3 2.8 .3
SQL*Net message to client 6375K 100.0 .0 .0 .0 .0 .0 .0
Wait Event Histogram DB/Inst: XXXX/xxxx Snaps: 5635-5636
-> Total Waits - units: K is 1000, M is 1000000, G is 1000000000
-> % of Waits - column heading: <=1s is truly <1024ms, >1s is truly >=1024ms
-> % of Waits - value: .0 indicates value was <.05%, null is truly 0
-> Ordered by Event (idle events last)
Total ----------------- % of Waits ------------------
Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
SQL*Net more data from cli 7976 99.7 .1 .1 .0 .1
Space Manager: slave idle 4086 .1 .2 .0 .0 .3 3.2 96.1
Streams AQ: qmn coordinato 258 49.2 .8 50.0
Streams AQ: qmn slave idle 129 100.0
Streams AQ: waiting for ti 2 50.0 50.0
class slave wait 28 92.9 3.6 3.6
jobq slave wait 7270 .0 100.0
pmon timer 1199 100.0
rdbms ipc message 314K 10.3 7.3 39.7 15.4 10.6 5.3 8.2 3.3
smon timer 43 100.0 -
How to set the correct shared pool size and db_buffer_cache using awr
Hi All,
I want to how to set the correct size for shared_pool_size and db_cache_size using shared pool advisory and buffer pool advisory of awr report. I have paste the shared and buffer pool advisory of awr report.
Shared Pool Advisory
* SP: Shared Pool Est LC: Estimated Library Cache Factr: Factor
* Note there is often a 1:Many correlation between a single logical object in the Library Cache, and the physical number of memory objects associated with it. Therefore comparing the number of Lib Cache objects (e.g. in v$librarycache), with the number of Lib Cache Memory Objects is invalid.
Shared Pool Size(M) SP Size Factr Est LC Size (M) Est LC Mem Obj Est LC Time Saved (s) Est LC Time Saved Factr Est LC Load Time (s) Est LC Load Time Factr Est LC Mem Obj Hits (K)
4,096 1.00 471 25,153 184,206 1.00 149 1.00 9,069
4,736 1.16 511 27,328 184,206 1.00 149 1.00 9,766
5,248 1.28 511 27,346 184,206 1.00 149 1.00 9,766
5,760 1.41 511 27,346 184,206 1.00 149 1.00 9,766
6,272 1.53 511 27,346 184,206 1.00 149 1.00 9,766
6,784 1.66 511 27,346 184,206 1.00 149 1.00 9,766
7,296 1.78 511 27,346 184,206 1.00 149 1.00 9,766
7,808 1.91 511 27,346 184,206 1.00 149 1.00 9,766
8,320 2.03 511 27,346 184,206 1.00 149 1.00 9,766
Buffer Pool Advisory
* Only rows with estimated physical reads >0 are displayed
* ordered by Block Size, Buffers For Estimate
P Size for Est (M) Size Factor Buffers (thousands) Est Phys Read Factor Estimated Phys Reads (thousands) Est Phys Read Time Est %DBtime for Rds
D 4,096 0.10 485 1.02 1,002 1 0.00
D 8,192 0.20 970 1.00 987 1 0.00
D 12,288 0.30 1,454 1.00 987 1 0.00
D 16,384 0.40 1,939 1.00 987 1 0.00
D 20,480 0.50 2,424 1.00 987 1 0.00
D 24,576 0.60 2,909 1.00 987 1 0.00
D 28,672 0.70 3,394 1.00 987 1 0.00
D 32,768 0.80 3,878 1.00 987 1 0.00
D 36,864 0.90 4,363 1.00 987 1 0.00
D 40,960 1.00 4,848 1.00 987 1 0.00
D 45,056 1.10 5,333 1.00 987 1 0.00
D 49,152 1.20 5,818 1.00 987 1 0.00
D 53,248 1.30 6,302 1.00 987 1 0.00
D 57,344 1.40 6,787 1.00 987 1 0.00
D 61,440 1.50 7,272 1.00 987 1 0.00
D 65,536 1.60 7,757 1.00 987 1 0.00
D 69,632 1.70 8,242 1.00 987 1 0.00
D 73,728 1.80 8,726 1.00 987 1 0.00
D 77,824 1.90 9,211 1.00 987 1 0.00
D 81,920 2.00 9,696 1.00 987 1 0.00
My shared pool size is 4gb and db_cache_size is 40Gb.
Please help me in configuring the correct size for this.
Thanks and Regards,Hi ,
Actually batch load is taking too much time.
Please find below the 1 hr awr report
Snap Id Snap Time Sessions Cursors/Session
Begin Snap: 6557 27-Nov-11 16:00:06 126 1.3
End Snap: 6558 27-Nov-11 17:00:17 130 1.6
Elapsed: 60.17 (mins)
DB Time: 34.00 (mins)
Report Summary
Cache Sizes
Begin End
Buffer Cache: 40,960M 40,960M Std Block Size: 8K
Shared Pool Size: 4,096M 4,096M Log Buffer: 25,908K
Load Profile
Per Second Per Transaction Per Exec Per Call
DB Time(s): 0.6 1.4 0.00 0.07
DB CPU(s): 0.5 1.2 0.00 0.06
Redo size: 281,296.9 698,483.4
Logical reads: 20,545.6 51,016.4
Block changes: 1,879.5 4,667.0
Physical reads: 123.7 307.2
Physical writes: 66.4 164.8
User calls: 8.2 20.4
Parses: 309.4 768.4
Hard parses: 8.5 21.2
W/A MB processed: 1.7 4.3
Logons: 0.7 1.6
Executes: 1,235.9 3,068.7
Rollbacks: 0.0 0.0
Transactions: 0.4
Instance Efficiency Percentages (Target 100%)
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.66 In-memory Sort %: 100.00
Library Hit %: 99.19 Soft Parse %: 97.25
Execute to Parse %: 74.96 Latch Hit %: 99.97
Parse CPU to Parse Elapsd %: 92.41 % Non-Parse CPU: 98.65
Shared Pool Statistics
Begin End
Memory Usage %: 80.33 82.01
% SQL with executions>1: 90.90 86.48
% Memory for SQL w/exec>1: 90.10 86.89
Top 5 Timed Foreground Events
Event Waits Time(s) Avg wait (ms) % DB time Wait Class
DB CPU 1,789 87.72
db file sequential read 27,531 50 2 2.45 User I/O
db file scattered read 26,322 30 1 1.47 User I/O
row cache lock 1,798 20 11 0.96 Concurrency
OJVM: Generic 36 15 421 0.74 Other
Host CPU (CPUs: 24 Cores: 12 Sockets: )
Load Average Begin Load Average End %User %System %WIO %Idle
0.58 1.50 2.8 0.7 0.1 96.6
Instance CPU
%Total CPU %Busy CPU %DB time waiting for CPU (Resource Manager)
2.2 63.6 0.0
Memory Statistics
Begin End
Host Mem (MB): 131,072.0 131,072.0
SGA use (MB): 50,971.4 50,971.4
PGA use (MB): 545.5 1,066.3
% Host Mem used for SGA+PGA: 39.30 39.70
RAC Statistics
Begin End
Number of Instances: 2 2
Global Cache Load Profile
Per Second Per Transaction
Global Cache blocks received: 3.09 7.68
Global Cache blocks served: 1.86 4.62
GCS/GES messages received: 78.64 195.27
GCS/GES messages sent: 53.82 133.65
DBWR Fusion writes: 0.52 1.30
Estd Interconnect traffic (KB) 65.50
Global Cache Efficiency Percentages (Target local+remote 100%)
Buffer access - local cache %: 99.65
Buffer access - remote cache %: 0.02
Buffer access - disk %: 0.34
Global Cache and Enqueue Services - Workload Characteristics
Avg global enqueue get time (ms): 0.0
Avg global cache cr block receive time (ms): 1.7
Avg global cache current block receive time (ms): 1.0
Avg global cache cr block build time (ms): 0.0
Avg global cache cr block send time (ms): 0.0
Global cache log flushes for cr blocks served %: 1.4
Avg global cache cr block flush time (ms): 0.9
Avg global cache current block pin time (ms): 0.0
Avg global cache current block send time (ms): 0.0
Global cache log flushes for current blocks served %: 0.1
Avg global cache current block flush time (ms): 0.0
Global Cache and Enqueue Services - Messaging Statistics
Avg message sent queue time (ms): 0.0
Avg message sent queue time on ksxp (ms): 0.4
Avg message received queue time (ms): 0.5
Avg GCS message process time (ms): 0.0
Avg GES message process time (ms): 0.0
% of direct sent messages: 79.13
% of indirect sent messages: 17.10
% of flow controlled messages: 3.77
Cluster Interconnect
Begin End
Interface IP Address Pub Source IP Pub Src
en9 10.51.10.61 N Oracle Cluster Repository
Main Report
* Report Summary
* Wait Events Statistics
* SQL Statistics
* Instance Activity Statistics
* IO Stats
* Buffer Pool Statistics
* Advisory Statistics
* Wait Statistics
* Undo Statistics
* Latch Statistics
* Segment Statistics
* Dictionary Cache Statistics
* Library Cache Statistics
* Memory Statistics
* Streams Statistics
* Resource Limit Statistics
* Shared Server Statistics
* init.ora Parameters
More RAC Statistics
* RAC Report Summary
* Global Messaging Statistics
* Global CR Served Stats
* Global CURRENT Served Stats
* Global Cache Transfer Stats
* Interconnect Stats
* Dynamic Remastering Statistics
Back to Top
Statistic Name Time (s) % of DB Time
sql execute elapsed time 1,925.20 94.38
DB CPU 1,789.38 87.72
connection management call elapsed time 99.65 4.89
PL/SQL execution elapsed time 89.81 4.40
parse time elapsed 46.32 2.27
hard parse elapsed time 25.01 1.23
Java execution elapsed time 21.24 1.04
PL/SQL compilation elapsed time 11.92 0.58
failed parse elapsed time 9.37 0.46
hard parse (sharing criteria) elapsed time 8.71 0.43
sequence load elapsed time 0.06 0.00
repeated bind elapsed time 0.02 0.00
hard parse (bind mismatch) elapsed time 0.01 0.00
DB time 2,039.77
background elapsed time 122.00
background cpu time 113.42
Statistic Value End Value
NUM_LCPUS 0
NUM_VCPUS 0
AVG_BUSY_TIME 12,339
AVG_IDLE_TIME 348,838
AVG_IOWAIT_TIME 221
AVG_SYS_TIME 2,274
AVG_USER_TIME 9,944
BUSY_TIME 299,090
IDLE_TIME 8,375,051
IOWAIT_TIME 6,820
SYS_TIME 57,512
USER_TIME 241,578
LOAD 1 2
OS_CPU_WAIT_TIME 312,200
PHYSICAL_MEMORY_BYTES 137,438,953,472
NUM_CPUS 24
NUM_CPU_CORES 12
GLOBAL_RECEIVE_SIZE_MAX 1,310,720
GLOBAL_SEND_SIZE_MAX 1,310,720
TCP_RECEIVE_SIZE_DEFAULT 16,384
TCP_RECEIVE_SIZE_MAX 9,223,372,036,854,775,807
TCP_RECEIVE_SIZE_MIN 4,096
TCP_SEND_SIZE_DEFAULT 16,384
TCP_SEND_SIZE_MAX 9,223,372,036,854,775,807
TCP_SEND_SIZE_MIN 4,096
Back to Wait Events Statistics
Back to Top
Operating System Statistics - Detail
Snap Time Load %busy %user %sys %idle %iowait
27-Nov 16:00:06 0.58
27-Nov 17:00:17 1.50 3.45 2.79 0.66 96.55 0.08
Back to Wait Events Statistics
Back to Top
Foreground Wait Class
* s - second, ms - millisecond - 1000th of a second
* ordered by wait time desc, waits desc
* %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
* Captured Time accounts for 95.7% of Total DB time 2,039.77 (s)
* Total FG Wait Time: 163.14 (s) DB CPU time: 1,789.38 (s)
Wait Class Waits %Time -outs Total Wait Time (s) Avg wait (ms) %DB time
DB CPU 1,789 87.72
User I/O 61,229 0 92 1 4.49
Other 102,743 40 31 0 1.50
Concurrency 3,169 10 24 7 1.16
Cluster 58,920 0 11 0 0.52
System I/O 45,407 0 6 0 0.29
Configuration 107 7 1 5 0.03
Commit 383 0 0 1 0.01
Network 15,275 0 0 0 0.00
Application 52 8 0 0 0.00
Back to Wait Events Statistics
Back to Top
Foreground Wait Events
* s - second, ms - millisecond - 1000th of a second
* Only events with Total Wait Time (s) >= .001 are shown
* ordered by wait time desc, waits desc (idle events last)
* %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Event Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn % DB time
db file sequential read 27,531 0 50 2 18.93 2.45
db file scattered read 26,322 0 30 1 18.10 1.47
row cache lock 1,798 0 20 11 1.24 0.96
OJVM: Generic 36 42 15 421 0.02 0.74
db file parallel read 394 0 7 19 0.27 0.36
control file sequential read 22,248 0 6 0 15.30 0.28
reliable message 4,439 0 4 1 3.05 0.18
gc current grant busy 7,597 0 3 0 5.22 0.16
PX Deq: Slave Session Stats 2,661 0 3 1 1.83 0.16
DFS lock handle 3,208 0 3 1 2.21 0.16
direct path write temp 4,842 0 3 1 3.33 0.15
library cache load lock 39 0 3 72 0.03 0.14
gc cr multi block request 37,008 0 3 0 25.45 0.14
IPC send completion sync 5,451 0 2 0 3.75 0.10
gc cr block 2-way 4,669 0 2 0 3.21 0.09
enq: PS - contention 3,183 33 1 0 2.19 0.06
gc cr grant 2-way 5,151 0 1 0 3.54 0.06
direct path read temp 1,722 0 1 1 1.18 0.05
gc current block 2-way 1,807 0 1 0 1.24 0.03
os thread startup 6 0 1 108 0.00 0.03
name-service call wait 12 0 1 47 0.01 0.03
PX Deq: Signal ACK RSG 2,046 50 0 0 1.41 0.02
log file switch completion 3 0 0 149 0.00 0.02
rdbms ipc reply 3,610 0 0 0 2.48 0.02
gc current grant 2-way 1,432 0 0 0 0.98 0.02
library cache pin 903 32 0 0 0.62 0.02
PX Deq: reap credit 35,815 100 0 0 24.63 0.01
log file sync 383 0 0 1 0.26 0.01
Disk file operations I/O 405 0 0 0 0.28 0.01
library cache lock 418 3 0 0 0.29 0.01
kfk: async disk IO 23,159 0 0 0 15.93 0.01
gc current block busy 4 0 0 35 0.00 0.01
gc current multi block request 1,206 0 0 0 0.83 0.01
ges message buffer allocation 38,526 0 0 0 26.50 0.00
enq: FB - contention 131 0 0 0 0.09 0.00
undo segment extension 8 100 0 6 0.01 0.00
CSS initialization 8 0 0 6 0.01 0.00
SQL*Net message to client 14,600 0 0 0 10.04 0.00
enq: HW - contention 96 0 0 0 0.07 0.00
CSS operation: action 8 0 0 4 0.01 0.00
gc cr block busy 33 0 0 1 0.02 0.00
latch free 30 0 0 1 0.02 0.00
enq: TM - contention 49 6 0 0 0.03 0.00
enq: JQ - contention 19 100 0 1 0.01 0.00
SQL*Net more data to client 666 0 0 0 0.46 0.00
asynch descriptor resize 3,179 100 0 0 2.19 0.00
latch: shared pool 3 0 0 3 0.00 0.00
CSS operation: query 24 0 0 0 0.02 0.00
PX Deq: Signal ACK EXT 72 0 0 0 0.05 0.00
KJC: Wait for msg sends to complete 269 0 0 0 0.19 0.00
latch: object queue header operation 4 0 0 1 0.00 0.00
gc cr block congested 5 0 0 0 0.00 0.00
utl_file I/O 11 0 0 0 0.01 0.00
enq: TO - contention 3 33 0 0 0.00 0.00
SQL*Net message from client 14,600 0 219,478 15033 10.04
jobq slave wait 7,726 100 3,856 499 5.31
PX Deq: Execution Msg 10,556 19 50 5 7.26
PX Deq: Execute Reply 2,946 31 27 9 2.03
PX Deq: Parse Reply 3,157 35 3 1 2.17
PX Deq: Join ACK 2,976 28 2 1 2.05
PX Deq Credit: send blkd 7 14 0 4 0.00
Back to Wait Events Statistics
Back to Top
Background Wait Events
* ordered by wait time desc, waits desc (idle events last)
* Only events with Total Wait Time (s) >= .001 are shown
* %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Event Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn % bg time
os thread startup 140 0 13 90 0.10 10.35
db file parallel write 8,233 0 6 1 5.66 5.08
log file parallel write 3,906 0 6 1 2.69 4.62
log file sequential read 350 0 5 16 0.24 4.49
control file sequential read 13,737 0 5 0 9.45 3.72
DFS lock handle 2,990 27 2 1 2.06 1.43
db file sequential read 921 0 2 2 0.63 1.39
SQL*Net break/reset to client 18 0 1 81 0.01 1.19
control file parallel write 2,455 0 1 1 1.69 1.12
ges lms sync during dynamic remastering and reconfig 24 100 1 50 0.02 0.98
library cache load lock 35 0 1 24 0.02 0.68
ASM file metadata operation 3,483 0 1 0 2.40 0.65
enq: CO - master slave det 1,203 100 1 0 0.83 0.46
kjbdrmcvtq lmon drm quiesce: ping completion 9 0 1 62 0.01 0.46
enq: WF - contention 11 0 0 35 0.01 0.31
CGS wait for IPC msg 32,702 100 0 0 22.49 0.19
gc object scan 28,788 100 0 0 19.80 0.15
row cache lock 535 0 0 0 0.37 0.14
library cache pin 370 55 0 0 0.25 0.12
ksxr poll remote instances 19,119 100 0 0 13.15 0.11
name-service call wait 6 0 0 19 0.00 0.10
gc current block 2-way 304 0 0 0 0.21 0.09
gc cr block 2-way 267 0 0 0 0.18 0.08
gc cr grant 2-way 355 0 0 0 0.24 0.08
ges LMON to get to FTDONE 3 100 0 24 0.00 0.06
enq: CF - contention 145 76 0 0 0.10 0.05
PX Deq: reap credit 8,842 100 0 0 6.08 0.05
reliable message 126 0 0 0 0.09 0.05
db file scattered read 19 0 0 3 0.01 0.05
library cache lock 162 1 0 0 0.11 0.04
latch: shared pool 2 0 0 27 0.00 0.04
Disk file operations I/O 504 0 0 0 0.35 0.04
gc current grant busy 148 0 0 0 0.10 0.04
gcs log flush sync 84 0 0 1 0.06 0.04
ges message buffer allocation 24,934 0 0 0 17.15 0.02
enq: CR - block range reuse ckpt 83 0 0 0 0.06 0.02
latch free 22 0 0 1 0.02 0.02
CSS operation: action 13 0 0 2 0.01 0.02
CSS initialization 4 0 0 6 0.00 0.02
direct path read 1 0 0 21 0.00 0.02
rdbms ipc reply 153 0 0 0 0.11 0.01
db file parallel read 2 0 0 8 0.00 0.01
direct path write 5 0 0 3 0.00 0.01
gc current multi block request 49 0 0 0 0.03 0.01
gc current block busy 5 0 0 2 0.00 0.01
enq: PS - contention 24 50 0 0 0.02 0.01
gc cr multi block request 54 0 0 0 0.04 0.01
ges generic event 1 100 0 10 0.00 0.01
gc current grant 2-way 35 0 0 0 0.02 0.01
kfk: async disk IO 183 0 0 0 0.13 0.01
Log archive I/O 3 0 0 2 0.00 0.01
gc buffer busy acquire 2 0 0 3 0.00 0.00
LGWR wait for redo copy 123 0 0 0 0.08 0.00
IPC send completion sync 18 0 0 0 0.01 0.00
enq: TA - contention 11 0 0 0 0.01 0.00
read by other session 2 0 0 2 0.00 0.00
enq: TM - contention 9 89 0 0 0.01 0.00
latch: ges resource hash list 135 0 0 0 0.09 0.00
PX Deq: Slave Session Stats 12 0 0 0 0.01 0.00
KJC: Wait for msg sends to complete 89 0 0 0 0.06 0.00
enq: TD - KTF dump entries 8 0 0 0 0.01 0.00
enq: US - contention 7 0 0 0 0.00 0.00
CSS operation: query 12 0 0 0 0.01 0.00
enq: TK - Auto Task Serialization 6 100 0 0 0.00 0.00
PX Deq: Signal ACK RSG 24 50 0 0 0.02 0.00
log file single write 6 0 0 0 0.00 0.00
enq: WL - contention 2 100 0 1 0.00 0.00
ADR block file read 13 0 0 0 0.01 0.00
ADR block file write 5 0 0 0 0.00 0.00
latch: object queue header operation 1 0 0 1 0.00 0.00
gc cr block busy 1 0 0 1 0.00 0.00
rdbms ipc message 103,276 67 126,259 1223 71.03
PX Idle Wait 6,467 67 12,719 1967 4.45
wait for unread message on broadcast channel 7,240 100 7,221 997 4.98
gcs remote message 218,809 84 7,213 33 150.49
DIAG idle wait 203,228 95 7,185 35 139.77
shared server idle wait 121 100 3,630 30000 0.08
ASM background timer 3,343 0 3,611 1080 2.30
Space Manager: slave idle wait 723 100 3,610 4993 0.50
heartbeat monitor sleep 722 100 3,610 5000 0.50
ges remote message 73,089 52 3,609 49 50.27
dispatcher timer 66 88 3,608 54660 0.05
pmon timer 1,474 82 3,607 2447 1.01
PING 1,487 19 3,607 2426 1.02
Streams AQ: qmn slave idle wait 125 0 3,594 28754 0.09
Streams AQ: qmn coordinator idle wait 250 50 3,594 14377 0.17
smon timer 18 50 3,505 194740 0.01
JOX Jit Process Sleep 73 100 976 13370 0.05
class slave wait 56 0 605 10806 0.04
KSV master wait 2,215 98 1 0 1.52
SQL*Net message from client 109 0 0 2 0.07
PX Deq: Parse Reply 27 44 0 1 0.02
PX Deq: Join ACK 30 40 0 1 0.02
PX Deq: Execute Reply 20 30 0 0 0.01
Streams AQ: RAC qmn coordinator idle wait 259 100 0 0 0.18
Back to Wait Events Statistics
Back to Top
Wait Event Histogram
* Units for Total Waits column: K is 1000, M is 1000000, G is 1000000000
* % of Waits: value of .0 indicates value was <.05%; value of null is truly 0
* % of Waits: column heading of <=1s is truly <1024ms, >1s is truly >=1024ms
* Ordered by Event (idle events last)
% of Waits
Event Total Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s
ADR block file read 13 100.0
ADR block file write 5 100.0
ADR file lock 6 100.0
ARCH wait for archivelog lock 3 100.0
ASM file metadata operation 3483 99.6 .1 .1 .2
CGS wait for IPC msg 32.7K 100.0
CSS initialization 12 50.0 50.0
CSS operation: action 21 28.6 9.5 61.9
CSS operation: query 36 86.1 5.6 8.3
DFS lock handle 6198 98.6 1.2 .1 .1
Disk file operations I/O 909 95.7 3.6 .7
IPC send completion sync 5469 99.9 .1 .0 .0
KJC: Wait for msg sends to complete 313 100.0
LGWR wait for redo copy 122 100.0
Log archive I/O 3 66.7 33.3
OJVM: Generic 36 55.6 44.4
PX Deq: Signal ACK EXT 72 98.6 1.4
PX Deq: Signal ACK RSG 2070 99.7 .0 .1 .0 .1
PX Deq: Slave Session Stats 2673 99.7 .2 .1 .0
PX Deq: reap credit 44.7K 100.0
SQL*Net break/reset to client 20 95.0 5.0
SQL*Net message to client 14.7K 100.0
SQL*Net more data from client 32 100.0
SQL*Net more data to client 689 100.0
asynch descriptor resize 3387 100.0
buffer busy waits 2 100.0
control file parallel write 2455 96.6 2.2 .6 .6 .1
control file sequential read 36K 99.4 .3 .1 .1 .1 .1 .0
db file parallel read 397 8.8 .8 5.5 12.6 17.4 46.3 8.6
db file parallel write 8233 85.4 10.3 2.3 1.4 .4 .1
db file scattered read 26.3K 79.2 1.5 8.2 10.5 .6 .1 .0
db file sequential read 28.4K 60.2 3.3 18.0 18.1 .3 .1 .0
db file single write 2 100.0
direct path read 2 50.0 50.0
direct path read temp 1722 95.8 2.8 .1 .5 .8 .1
direct path write 6 83.3 16.7
direct path write temp 4842 96.3 2.7 .5 .2 .0 .0 .2
enq: AF - task serialization 1 100.0
enq: CF - contention 145 99.3 .7
enq: CO - master slave det 1203 98.9 .8 .2
enq: CR - block range reuse ckpt 83 100.0
enq: DR - contention 2 100.0
enq: FB - contention 131 100.0
enq: HW - contention 97 100.0
enq: JQ - contention 19 89.5 10.5
enq: JS - job run lock - synchronize 3 100.0
enq: MD - contention 1 100.0
enq: MW - contention 2 100.0
enq: PS - contention 3207 99.5 .4 .1
enq: TA - contention 11 100.0
enq: TD - KTF dump entries 8 100.0
enq: TK - Auto Task Serialization 6 100.0
enq: TM - contention 58 100.0
enq: TO - contention 3 100.0
enq: TQ - DDL contention 1 100.0
enq: TS - contention 1 100.0
enq: UL - contention 1 100.0
enq: US - contention 7 100.0
enq: WF - contention 11 81.8 18.2
enq: WL - contention 2 50.0 50.0
gc buffer busy acquire 2 50.0 50.0
gc cr block 2-way 4934 99.9 .1 .0 .0
gc cr block busy 35 68.6 31.4
gc cr block congested 6 100.0
gc cr disk read 2 100.0
gc cr grant 2-way 4824 100.0 .0
gc cr grant congested 2 100.0
gc cr multi block request 37.1K 99.8 .2 .0 .0 .0 .0 .0
gc current block 2-way 2134 99.9 .0 .0
gc current block busy 7 14.3 14.3 14.3 28.6 28.6
gc current block congested 2 100.0
gc current grant 2-way 1337 99.9 .1
gc current grant busy 7123 99.2 .2 .2 .0 .0 .3 .1
gc current grant congested 2 100.0
gc current multi block request 1260 99.8 .2
gc object scan 28.8K 100.0
gcs log flush sync 65 95.4 3.1 1.5
ges LMON to get to FTDONE 3 100.0
ges generic event 1 100.0
ges inquiry response 2 100.0
ges lms sync during dynamic remastering and reconfig 24 16.7 29.2 54.2
ges message buffer allocation 63.1K 100.0
kfk: async disk IO 23.3K 100.0 .0 .0
kjbdrmcvtq lmon drm quiesce: ping completion 9 11.1 88.9
ksxr poll remote instances 19.1K 100.0
latch free 52 59.6 40.4
latch: call allocation 2 100.0
latch: gc element 1 100.0
latch: gcs resource hash 1 100.0
latch: ges resource hash list 135 100.0
latch: object queue header operation 5 40.0 40.0 20.0
latch: shared pool 5 40.0 20.0 20.0 20.0
library cache load lock 74 9.5 5.4 8.1 17.6 10.8 13.5 35.1
library cache lock 493 99.2 .4 .4
library cache pin 1186 98.4 .3 1.2 .1
library cache: mutex X 6 100.0
log file parallel write 3897 72.9 1.5 17.1 7.5 .6 .3 .1
log file sequential read 350 4.6 3.1 59.4 30.0 2.9
log file single write 6 100.0
log file switch completion 3 33.3 66.7
log file sync 385 90.4 3.6 4.7 .8 .5
name-service call wait 18 5.6 5.6 5.6 16.7 44.4 22.2
os thread startup 146 100.0
rdbms ipc reply 3763 99.7 .3
read by other session 2 50.0 50.0
reliable message 4565 99.7 .2 .0 .0 .1
row cache lock 2334 99.3 .2 .1 .1 .3
undo segment extension 8 50.0 37.5 12.5
utl_file I/O 11 100.0
ASM background timer 3343 57.0 .3 .1 .1 .1 21.1 21.4
DIAG idle wait 203.2K 3.4 .2 .4 18.0 41.4 14.8 21.8
JOX Jit Process Sleep 73 2.7 97.3
KSV master wait 2213 99.4 .1 .2 .3
PING 1487 81.0 19.0
PX Deq Credit: send blkd 7 57.1 14.3 14.3 14.3
PX Deq: Execute Reply 2966 59.8 .8 9.5 5.6 10.2 2.6 11.4
PX Deq: Execution Msg 10.6K 72.4 12.1 2.6 2.5 .1 5.6 4.6 .0
PX Deq: Join ACK 3006 77.9 22.1 .1
PX Deq: Parse Reply 3184 67.1 31.1 1.6 .2
PX Idle Wait 6466 .2 8.7 4.3 4.8 .3 .1 5.0 76.6
SQL*Net message from client 14.7K 72.4 2.8 .8 .5 .9 .4 2.8 19.3
Space Manager: slave idle wait 722 100.0
Streams AQ: RAC qmn coordinator idle wait 259 100.0
Streams AQ: qmn coordinator idle wait 250 50.0 50.0
Streams AQ: qmn slave idle wait 125 100.0
class slave wait 55 67.3 7.3 1.8 5.5 1.8 7.3 9.1
dispatcher timer 66 6.1 93.9
gcs remote message 218.6K 7.7 1.8 1.2 1.6 1.7 15.7 70.3
ges remote message 72.9K 29.7 5.1 2.7 2.2 1.5 4.0 54.7
heartbeat monitor sleep 722 100.0
jobq slave wait 7725 .1 .0 99.9
pmon timer 1474 18.4 81.6
rdbms ipc message 103.3K 20.7 2.7 1.5 1.3 .9 .7 40.7 31.6
shared server idle wait 121 100.0
smon timer 18 100.0
wait for unread message on broadcast channel 7238 .3 99.7
Back to Wait Events Statistics
Back to Top
Wait Event Histogram Detail (64 msec to 2 sec)
* Units for Total Waits column: K is 1000, M is 1000000, G is 1000000000
* Units for % of Total Waits: ms is milliseconds s is 1024 milliseconds (approximately 1 second)
* % of Total Waits: total waits for all wait classes, including Idle
* % of Total Waits: value of .0 indicates value was <.05%; value of null is truly 0
* Ordered by Event (only non-idle events are displayed)
% of Total Waits
Event Waits 64ms to 2s <32ms <64ms <1/8s <1/4s <1/2s <1s <2s >=2s
ASM file metadata operation 6 99.8 .1 .1
DFS lock handle 6 99.9 .1 .0
OJVM: Generic 16 55.6 2.8 41.7
PX Deq: Signal ACK RSG 3 99.9 .0 .1
PX Deq: Slave Session Stats 3 99.9 .0 .0 .0
SQL*Net break/reset to client 1 95.0 5.0
control file sequential read 1 100.0 .0
db file parallel read 34 91.4 8.6
db file scattered read 4 100.0 .0 .0
db file sequential read 6 100.0 .0 .0 .0
direct path write temp 11 99.8 .1 .1 .0
enq: WF - contention 2 81.8 18.2
gc cr block 2-way 1 100.0 .0
gc cr multi block request 1 100.0 .0
gc current block 2-way 1 100.0 .0
gc current block busy 2 71.4 28.6
gc current grant busy 8 99.9 .0 .1
ges lms sync during dynamic remastering and reconfig 13 45.8 20.8 33.3
kjbdrmcvtq lmon drm quiesce: ping completion 8 11.1 11.1 77.8
latch: shared pool 1 80.0 20.0
library cache load lock 26 64.9 14.9 12.2 4.1 4.1
log file parallel write 2 99.9 .0 .0
log file sequential read 10 97.1 2.0 .6 .3
log file switch completion 2 33.3 66.7
name-service call wait 4 77.8 22.2
os thread startup 146 100.0
reliable message 4 99.9 .0 .1
row cache lock 2 99.7 .0 .0 .3
Back to Wait Events Statistics
Back to Top
Wait Event Histogram Detail (4 sec to 2 min)
* Units for Total Waits column: K is 1000, M is 1000000, G is 1000000000
* Units for % of Total Waits: s is 1024 milliseconds (approximately 1 second) m is 64*1024 milliseconds (approximately 67 seconds or 1.1 minutes)
* % of Total Waits: total waits for all wait classes, including Idle
* % of Total Waits: value of .0 indicates value was <.05%; value of null is truly 0
* Ordered by Event (only non-idle events are displayed)
% of Total Waits
Event Waits 4s to 2m <2s <4s <8s <16s <32s < 1m < 2m >=2m
row cache lock 6 99.7 .3
Back to Wait Events Statistics
Back to Top
Wait Event Histogram Detail (4 min to 1 hr)
No data exists for this section of the report.
Back to Wait Events Statistics
Back to Top
Service Statistics
* ordered by DB Time
Service Name DB Time (s) DB CPU (s) Physical Reads (K) Logical Reads (K)
ubshost 1,934 1,744 445 73,633
SYS$USERS 105 45 1 404
SYS$BACKGROUND 0 0 1 128
ubshostXDB 0 0 0 0
Back to Wait Events Statistics
Back to Top
Service Wait Class Stats
* Wait Class info for services in the Service Statistics section.
* Total Waits and Time Waited displayed for the following wait classes: User I/O, Concurrency, Administrative, Network
* Time Waited (Wt Time) in seconds
Service Name User I/O Total Wts User I/O Wt Time Concurcy Total Wts Concurcy Wt Time Admin Total Wts Admin Wt Time Network Total Wts Network Wt Time
ubshost 60232 90 2644 4 0 0 13302 0
SYS$USERS 997 2 525 19 0 0 1973 0
SYS$BACKGROUND 1456 2 1258 14 0 0 0 0
I am not able to paste the whole awr report. I have paste some of the sections of awr report.
Please help.
Thanks and Regards, -
Enq: TX - row lock contention wait event
Hi,
I would like to find which DML query has not given COMMIT or ROLLBACK after the execution. Because one of the development database have more table locks and developer reported that their session was hanging. I referred AWR report also and more timed waits occurred in the enq: TX - row lock contention. I need to trace which DML query has not commit or rollback.
Please help me to solve the issue.
Database version: 11.2.0.1.0
Foreground Wait Events
Event
Waits
%Time -outs
Total Wait Time (s)
Avg wait (ms)
Waits /txn
% DB time
enq: TX - row lock contention
320
0
72,047
225147
0.20
99.53
log file sync
547
0
14
26
0.35
0.02
library cache lock
13
0
11
843
0.01
0.02
SQL*Net break/reset to client
1,080
0
2
1
0.69
0.00
SQL*Net message to client
659,006
0
1
0
421.63
0.00
direct path sync
3
0
1
299
0.00
0.00
SQL*Net more data from client
5,541
0
1
0
3.55
0.00
db file scattered read
554
0
0
1
0.35
0.00
SQL*Net more data to client
14,975
0
0
0
9.58
0.00
db file sequential read
2,817
0
0
0
1.80
0.00
ADR block file read
4
0
0
43
0.00
0.00
enq: CR - block range reuse ckpt
2
0
0
71
0.00
0.00
asynch descriptor resize
38,073
100
0
0
24.36
0.00
latch: shared pool
61
0
0
1
0.04
0.00
control file sequential read
6,900
0
0
0
4.41
0.00
Disk file operations I/O
550
0
0
0
0.35
0.00
cursor: pin S
1
0
0
8
0.00
0.00
direct path write temp
34
0
0
0
0.02
0.00
library cache: mutex X
5
0
0
1
0.00
0.00
latch: In memory undo latch
2
0
0
1
0.00
0.00
buffer busy waits
14
0
0
0
0.01
0.00
SQL*Net message from client
658,990
0
294,847
447
421.62
jobq slave wait
669
99
333
497
0.43
PL/SQL lock timer
1
100
1
998
0.00Oracle does not and cannot tell you from historical views (e.g. AWR) which DMLs have not COMMITed or ROLLBACKed. A Transaction ends with a COMMIT or ROLLBACK. The transaction could have a million (or more) DML statements with a million (or more) SELECT statements between the first DML and the COMMIT / ROLLBACK.
Even identifying such DMLs in real time is close to impossible. Because the session holding the lock may have issued a dozen or a million subsequent SQL statements while other sessions are waiting for the lock. You can only identify the session that is the lock holder (the BLOCKING_SESSION in V$SESSION).
If you have tracing enabled for all sessions, then you could review the trace file for the BLOCKING_SESSION to identify the DML(s) the session has executed.
Hemant K Chitale -
My wait events - can anyone see a problem?
hi,
this is what i have, can anyone see a problem?
thanks
EVENT TOTAL_WAITS PCT_WAITS TIME_WAIT_SEC PCT_TIME_WAITED TOTAL_TIMEOUTS PCT_TIMEOUTS AVERAGE_WAIT_SEC
Streams AQ: qmn slave idle wait 148147 .3 4051461.88 38.04 3478 .07 27.35
Streams AQ: qmn coordinator idle wa 291006 .59 3962890.53 37.21 148370 3.13 13.62
it
Streams AQ: waiting for time manage 948 0 2021434.2 18.98 948 .02 2132.31
ment or cleanup tasks
control file parallel write 1292057 2.64 266839.64 2.51 0 0 .21
log file parallel write 28433394 58.02 134658.55 1.26 0 0 0
db file sequential read 8307195 16.95 69830.07 .66 0 0 .01
free buffer waits 3117839 6.36 43374.04 .41 3106093 65.55 .01
log buffer space 55520 .11 20810.2 .2 20235 .43 .37
db file scattered read 583604 1.19 18169.58 .17 0 0 .03
write complete waits 17946 .04 17536.66 .16 17941 .38 .98
log file sync 282268 .58 10005.35 .09 9369 .2 .04
enq: RO - fast object reuse 26602 .05 6623.44 .06 2171 .05 .25
enq: CF - contention 1839 0 5178.14 .05 1723 .04 2.82
Streams AQ: qmn coordinator waiting 999 0 4311.01 .04 883 .02 4.32
for slave to start
buffer busy waits 32464 .07 3898.51 .04 3950 .08 .12
control file sequential read 2199199 4.49 3558.34 .03 0 0 0
SGA: MMAN sleep for component shrin 234330 .48 2523.65 .02 234216 4.94 .01
k
buffer exterminate 1583 0 1539.72 .01 1573 .03 .97
library cache pin 317 0 927.71 .01 316 .01 2.93
enq: CI - contention 1829 0 570.84 .01 159 0 .31
log file switch completion 1658 0 517.18 0 425 .01 .31
enq: TX - row lock contention 257 0 438.8 0 149 0 1.71
read by other session 27269 .06 355.17 0 52 0 .01
os thread startup 3869 .01 338.67 0 98 0 .09
latch: shared pool 760 0 285.87 0 0 0 .38
latch: row cache objects 664 0 250 0 0 0 .38
Data file init write 16324 .03 231.59 0 0 0 .01
reliable message 19189 .04 218.16 0 170 0 .01
latch: library cache 483 0 172.51 0 0 0 .36
SQL*Net message from dblink 1143086 2.33 128.69 0 0 0 0
latch free 6091 .01 121.1 0 0 0 .02
library cache load lock 90 0 89.48 0 18 0 .99
log file single write 1894 0 69.76 0 0 0 .04
cursor: pin S wait on X 5183 .01 55.87 0 5165 .11 .01
local write wait 6732 .01 42.58 0 2 0 .01
log file switch (checkpoint incompl 95 0 42.11 0 30 0 .44
ete)
row cache lock 119 0 30.96 0 10 0 .26
SQL*Net more data from dblink 17198 .04 25.92 0 0 0 0
log file switch (private strand flu 69 0 17.54 0 5 0 .25
sh incomplete)
enq: HW - contention 180 0 16.53 0 5 0 .09
enq: PR - contention 9 0 14.5 0 2 0 1.61
enq: JS - queue lock 51 0 12.36 0 0 0 .24
SQL*Net more data to client 48311 .1 11.66 0 0 0 0
enq: TM - contention 12 0 10.66 0 3 0 .89
class slave wait 3128 .01 7.03 0 1 0 0
JS coord start wait 68 0 6.42 0 68 0 .09
direct path write 92712 .19 6.06 0 0 0 0
control file heartbeat 1 0 3.91 0 1 0 3.91
PX Deq: Par Recov Execute 100 0 3.8 0 0 0 .04
log file sequential read 1900 0 2.88 0 0 0 0
single-task message 50 0 2.61 0 0 0 .05
enq: TX - contention 11 0 2.38 0 0 0 .22
undo segment extension 1181001 2.41 1.95 0 1180981 24.92 0
db file single write 165 0 1.3 0 0 0 .01
enq: TX - index contention 97 0 1.27 0 0 0 .01
LGWR wait for redo copy 20840 .04 .66 0 0 0 0
JS kgl get object wait 8 0 .63 0 8 0 .08
SQL*Net message to dblink 1143086 2.33 .55 0 0 0 0
kksfbc child completion 14 0 .55 0 11 0 .04
direct path read temp 217237 .44 .41 0 0 0 0
latch: cache buffers chains 2138 0 .37 0 0 0 0
latch: messages 1245 0 .27 0 0 0 0
latch: redo writing 786 0 .15 0 0 0 0
PX Deq: Par Recov Reply 65 0 .09 0 0 0 0
latch: checkpoint queue latch 171 0 .09 0 0 0 0
latch: redo allocation 1029 0 .08 0 0 0 0
latch: cache buffers lru chain 268 0 .07 0 0 0 0
SGA: allocation forcing component g 5 0 .05 0 2 0 .01
rowth
db file parallel read 83 0 .04 0 0 0 0
latch: In memory undo latch 558 0 .04 0 0 0 0
latch: object queue header operatio 338 0 .04 0 0 0 0
n
direct path read 5042 .01 .03 0 0 0 0
PX Deque wait 7 0 .02 0 0 0 0
direct path write temp 4691 .01 .02 0 0 0 0
enq: SQ - contention 1 0 .02 0 0 0 .02
latch: session allocation 190 0 .02 0 0 0 0
PX Deq: Join ACK 15 0 .01 0 0 0 0
cursor: pin S 894 0 .01 0 0 0 0
enq: TX - allocate ITL entry 37 0 .01 0 0 0 0
kkdlgon 15 0 .01 0 0 0 0
latch: enqueue hash chains 37 0 .01 0 0 0 0
library cache lock 1 0 .01 0 0 0 .01
Log archive I/O 1 0 0 0 0 0 0
PX Deq: Par Recov Change Vector 2 0 0 0 0 0 0
PX Deq: Signal ACK 3 0 0 0 0 0 0
PX Deq: Test for msg 1 0 0 0 0 0 0
PX qref latch 1 0 0 0 1 0 0
SQL*Net break/reset to dblink 5 0 0 0 0 0 0
SQL*Net more data to dblink 1 0 0 0 0 0 0
buffer deadlock 27 0 0 0 27 0 0
checkpoint completed 4 0 0 0 0 0 0
cursor: mutex S 3 0 0 0 0 0 0
cursor: mutex X 1 0 0 0 0 0 0
enq: JS - q mem clnup lck 1 0 0 0 0 0 0
enq: PS - contention 2 0 0 0 0 0 0
enq: US - contention 1 0 0 0 0 0 0
instance state change 2 0 0 0 0 0 0
latch: library cache lock 4 0 0 0 0 0 0
latch: library cache pin 1 0 0 0 0 0 0
latch: object queue header heap 8 0 0 0 0 0 0
latch: undo global data 3 0 0 0 0 0 0
recovery read 39 0 0 0 0 0 0Hi,
If its for a week than I won't bother. Probably you should try to get teh same report for these wait events in a much smaller period , like 20-30minutes of time period when your db is fully operational. If still at that time the wait events,these or any other, shoot up to high wait times, things can be investigated more deeply.
HTH
Aman.... -
"shared pool free memory" include "SHARED_POOL_RESERVED_SIZE" area??
Hi, all.
"shared pool free memory" from v$sgastat include "SHARED_POOL_RESERVED_SIZE" ??
For example,
select * from v$sgastat
where pool ='shared pool'
and name like 'free memory'
assuming that the result of the above query is about 100Megabytes
and "SHARED_POOL_RESERVED_SIZE" is 50Megabytes,
"100 M free memory" in shared pool includes 50M (reserved area)??
Thanks and Regards.
Message was edited by:
user507290Shortly after the database starts up, some of the 'shared_pool_reserved_size' will probably be in use, although quite a lot of it may still be free; so you cannot say (directly) how much of the "free memory" belongs in the reserved area and how much comes from the rest of the shared pool.
However, there is a view called v$shared_pool_reserved that tells you how much of the reserved area is currently free (and gives various other statisics about the pool's use). There are some versions of Oracle where the definition of this view is wrong, though - possibly in the lower 9i versions.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk -
When I run as root or normal User
zenity --notification --text="test"
The zenity process takes 100% Cpu time
System:
Arch 32bit gnome 2.22
Last edited by brickwedde.eu (2008-05-06 11:26:00)I GOT IT... I think.
zenity --notification actually calls libnotify! I was trying out a script and suddenly realized something, because by chance dbus decided to take a break. Take a look at these two code fragments.
$ zenity --info --text "hello"
$ _
$ zenity --notification --text "hello"
libnotify-Message: Unable to get session bus: Failed to connect to socket /tmp/dbus-jmMdFXbznS: Connection refused
<hang>
<100% CPU usage>
I've figured out why: it's basic socket programming gone wrong. The notification part of zenity calls libnotify, tries to connect to libnotify's socket, and fails, using 100% CPU waiting for the connection to... end a stream that never started.
A possible solution is to just use notify-send.
-dav7
Last edited by dav7 (2008-05-09 00:27:30) -
Sessions with Event "class slave wait" taking 100% CPU
Hi experts,
I have an issue in an 11.2.0.2 RAC database with 2 standby nodes that are driving me nuts.
All started when I've been told there are 2 Oracle processes consuming too much CPU in the environment.
I found 2 processes which takes 100% CPU each. These processes are Oracle processes into the DB and checking them out although they are registered in v$process and v$session, there is nothing related to them in v$bgprocess so I could not find out which oracle processes started these 2 OS processes.
The view v$session shows them ACTIVE in the event "class slave wait". There aren't too much information about this event, at least I failed to find...
Using dbms_monitor and dbms_system to create trace events did not create any tracefile. Only ORADEBUG was able to create events but with nearly no information inside the trace files.
Questions I need to have answered or at least hints that I can follow to find the answer:
1) what is causing the CPUs to be consumed at 100%? Which process?
2) Why does these processes are using that much of CPU?
3) What can be done to safely get over it?
Honestly I don't know where else to look at except trying to get some help. Could someone give me a hand please?
BR,
Lauro Ojeda
Edited by: LauroOjeda on 26/01/2013 06:44Hi Pal, thanks for your reply!
Here are the answers:
1. If it is RAC there is no such construct as a "standby node" and you say you have two of them. Please be specific ... is this RAC or Data Guard or a combination? Or do you have a three node cluster with all services pointing to only a single server wasting all of the resources of the other three?
A: This is a combination of them. We have 2 nodes in a RAC environment shipping logs to two physical standby databases in another site.
2. Two processes can not be at 100% of CPU any more than I can be 200% awake. Please show us how you arrived at this conclusion, on what hardware and operating system, and include a formatted (read the FAQ to learn how) extract showing what you are seeing.
A: top in Linux shows two oracle processes consuming 100% (or nearly of it) of 1 CPU each. This is a 16 cores server (each primary node) so 2 of these are working on full capacity to service the described processes.
3. What processes? Name them and again run a SQL statement and post the output so we can see what you are looking at.
A: Like I said before I failed to find which background process they belongs to as there are no indications of them in v$bgprocess and in v$session/v$process either. I can see them in v$process and v$session but again, there are no indications of for which bg process they belong to.
4. Is anything slow? Are there any problems with the system or are you only reacting to two numbers you think are too high?
A: No, system is not slow because of that but the client wants to investigate and have it sorted as it is not normal.
Additionally I found a bug in metalink which seems to be the culprit, but I'm unsure yet. This is Bug 12929268 : HIGH CPU ON ORA_O00N PROCESS
Is any other information that I may provide you?
Thanks for help!
Lauro Ojeda -
I am running perfstat in Oracle9i Enterprise Edition Release 9.2.0.4.0. When I create a perfstat report using the standard scripts I see a wait event called "CPU time." I don't understand what this means. The OEM server monitoring tool does not indicate that the CPU is being used more than 50%, often more like 20%. The top 5 wait events is below.
What does "CPU time" time mean?
the following top 5 wait events:
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % Total
Event Waits Time (s) Ela Time
CPU time 2,200 61.88
db file sequential read 113,042 639 17.96
control file parallel write 9,345 471 13.24
log file sync 4,722 122 3.44
db file scattered read 13,375 67 1.87Hi,
Below is part of my Statspack
```````
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 13805 26-Mar-08 08:00:03 85 173.2
End Snap: 13806 26-Mar-08 09:00:05 98 154.4
Elapsed: 60.03 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 1,024M Std Block Size: 8K
Shared Pool Size: 400M Log Buffer: 2,048K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 183,878.75 16,898.36
Logical reads: 61,602.61 5,661.25
Block changes: 1,255.54 115.38
Physical reads: 7,370.56 677.35
Physical writes: 55.64 5.11
User calls: 341.11 31.35
Parses: 56.26 5.17
Hard parses: 0.32 0.03
Sorts: 56.43 5.19
Logons: 0.80 0.07
Executes: 712.67 65.49
Transactions: 10.88
% Blocks changed per Read: 2.04 Recursive Call %: 82.56
Rollback per transaction %: 9.29 Rows per Sort: 1043.63
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.98 Redo NoWait %: 100.00
Buffer Hit %: 88.06 In-memory Sort %: 100.00
Library Hit %: 99.96 Soft Parse %: 99.42
Execute to Parse %: 92.11 Latch Hit %: 99.95
Parse CPU to Parse Elapsd %: 69.60 % Non-Parse CPU: 99.64
Shared Pool Statistics Begin End
Memory Usage %: 89.25 89.26
% SQL with executions>1: 40.51 39.68
% Memory for SQL w/exec>1: 29.05 29.45
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % Total
Event Waits Time (s) Ela Time
CPU time 9,272 43.73
db file sequential read 5,203,772 7,027 33.14
db file scattered read 2,198,721 1,547 7.30
log file parallel write 20,935 1,126 5.31
log file sync 10,277 759 3.58
``````````````````
From my observation I found that it seems CPU is bottneck on this Server. I have gathered few SQLs to find out whether any scope for tuning but still I believe that by reducing the cache size also I can divide the load or can say transfer the load from CPU to I/O. Am I correct or do I need to look over else ?
Thanks
MJ -
Contention for latches related to the shared pool was consuming significant
We are having performance issue on our database. When I look at the AWR, I see that there is a contention for latches. Below is the AWR Report.
ADDM Report for Task 'ADDM:1775307360_12808'
Analysis Period
AWR snapshot range from 12807 to 12808.
Time period starts at 10-MAY-11 01.00.15 PM
Time period ends at 10-MAY-11 02.00.23 PM
Analysis Target
Database 'ADVFDWP' with DB ID 1775307360.
Database version 11.1.0.7.0.
ADDM performed an analysis of all instances.
Activity During the Analysis Period
Total database time was 27827 seconds.
The average number of active sessions was 7.71.
Summary of Findings
Description Active Sessions Recommendations
Percent of Activity
1 Shared Pool Latches 6.43 | 83.42 0
2 Top SQL by DB Time 2.41 | 31.24 3
3 "Concurrency" Wait Class 2.18 | 28.22 0
4 PL/SQL Execution 1.53 | 19.86 1
5 "User I/O" wait Class 1.33 | 17.24 0
6 Hard Parse 1.24 | 16.14 0
7 Undersized Buffer Cache .83 | 10.73 0
8 CPU Usage .7 | 9.02 0
9 Top SQL By I/O .31 | 4.04 1
10 Top Segments by I/O .24 | 3.12 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Findings and Recommendations
Finding 1: Shared Pool Latches
Impact is 6.43 active sessions, 83.42% of total activity.
Contention for latches related to the shared pool was consuming significant
database time in some instances.
Instances that were significantly affected by this finding:
Number Name Percent Impact ADDM Task Name
1 ADVFDWP1 99.31 ADDM:1775307360_1_12808
Check the ADDM analysis of affected instances for recommendations.
Finding 2: Top SQL by DB Time
Impact is 2.41 active sessions, 31.24% of total activity.
SQL statements consuming significant database time were found.
Recommendation 1: SQL Tuning
Estimated benefit is 1.07 active sessions, 13.82% of total activity.
Action
Run SQL Tuning Advisor on the SQL statement with SQL_ID "fdk73nhpt93a5".
Related Object
SQL statement with SQL_ID fdk73nhpt93a5.
INSERT INTO SFCDM.F_LOAN_PTFL_MOL_SNPSHT SELECT * FROM
F_LOAN_PTFL_MOL_SNPSHT_STG
Recommendation 2: SQL Tuning
Estimated benefit is 1 active sessions, 12.96% of total activity.
Action
Tune the PL/SQL block with SQL_ID "7nvgzsgy9ydn9". Refer to the "Tuning
PL/SQL Applications" chapter of Oracle's "PL/SQL User's Guide and
Reference".
Related Object
SQL statement with SQL_ID 7nvgzsgy9ydn9.
begin
insert into SFCDM.F_LOAN_PTFL_MOL_SNPSHT select * from
F_LOAN_PTFL_MOL_SNPSHT_STG;
end;
Recommendation 3: SQL Tuning
Estimated benefit is .4 active sessions, 5.2% of total activity.
Action
Investigate the SQL statement with SQL_ID "fcvfq2gzmxu0t" for possible
performance improvements.
Related Object
SQL statement with SQL_ID fcvfq2gzmxu0t.
select
a11.DT_YR_MO DT_YR_MO,
a11.IND_SCRTZD IND_SCRTZD,
a13.CD_LNSTAT CD_LNSTAT_INTGRTD,
sum(a11.CNT_LOAN) WJXBFS1,
sum(a11.AMT_PART_EOP_UPB) WJXBFS2,
sum(a11.AMT_LST_VLD_PART_UPB) WJXBFS3
from
SFCDM.F_LOAN_PTFL_MOL_SNPSHT
a11
join
SFCDM.D_DETD_LNSTAT_CURR
a12
on
(a11.ID_CYCL_CLOS_DETD_LNSTAT_SRGT = a12.ID_DETD_LNSTAT_SRGT)
join
SFCDM.D_LNSTAT_CD
a13
on
(a12.ID_LNSTAT_CD_SRGT = a13.ID_LNSTAT_CD_SRGT)
join
SFCDM.D_LOAN_CHARTC_CURR_MINI
a14
on
(a11.ID_LOAN_CHARTC_SRGT = a14.ID_LOAN_CHARTC_SRGT)
where
(a11.DT_YR_MO in (201103)
and a14.CD_SFCRM_LOAN_BUS_LI not in ('L', 'T', 'W')
and a13.CD_LNSTAT in (14)
and not exists
(select * from SFCDM.F_LOAN_PTFL_MOL_SNPSHT s
where s.id_loan_syst_gend = a11.id_loan_syst_gend
and s.dt_yr_moIt is worth checking the actual size of the shared pool e.g.
select pool,sum(bytes)/1024/1024/1024 from v$sgastat group by pool;
the parameters you ahve posted suggest you have set a minimum but no maximum, so it could very large.
Next up is looking for unhared SQL i.e.
select column1 from some_table where column2='A_VALUE';
select column1 from some_table where column2='Another_Value';
where the code should be using binds instead of literals for security and performance reasons, a simple way to find this is to look in v$sql for sql having the same plan_hash_value but different sql_Ids and compare the sql_fulltext of each statement.
Also a possibility is sql with many child cursors, this is trickier as the cause may vary and may not be easy to fix. check th econtents of v$sql for sql that have high values in the child_number column anmd investigate the contents of v$sql_shared_cursor for the reason there are multiple child cursors.
Chris -
High "CPU + Wait for CPU" event on pl/sql execute operation
I am executing a pl/sql in 256 parallel sessions, on 11G r2 DB (RAC 2 nodes), on a 42core IBM P7 Machine.
PL/sql function opens a cursor on a huge table with around 20M rows and does further processing.
Work-load is equally divided into 256 parallel sessions. 256 parallel sessions are opened by a middle-ware application and each session processes data based on an identifier (there are 256 distinct identifier values).
PL/sql function is comprised of some SQL operations, on which i am experiencing some cluster waits. But CPU + wait for CPU event on pl/sql execute operation is close to 80% of the total execution time.
Top user events:
Event Event Class % Event Avg Active Sessions
CPU + Wait for CPU CPU 80.88 98.15
gc current block 2-way Cluster 3.33 4.05
gc cr block busy Cluster 2.01 2.44
gc cr block 2-way Cluster 1.97 2.39
db file sequential read User I/O 1.81 2.20
Top SQL command type:
SQL Command Type Distinct SQLIDs % Activity Avg Active Sessions
PL/SQL EXECUTE 3 60.99 74.02
SELECT 66 12.90 15.65
INSERT 24 9.89 12.01
UPDATE 9 6.00 7.28
DELETE 2 1.33 1.61Rest of the SQL queries seem to be very optimum, but waits on pl/sql execute operation are causing very low tps.
Can anybody give me some heads-up about where to and what to look for to resolve?
Please let me know if any extra information is required.AWR report :
Header
DB Name DB Id Instance Inst num Startup Time Release RAC
FCR 1304316316 fcrypp1 1 01-12ÔÂ-12 04:12 11.2.0.2.0 YES
Host Name Platform CPUs Cores Sockets Memory (GB)
z4ci2011 AIX-Based Systems (64-bit) 168 42 320.00
Snap Id Snap Time Sessions Cursors/Session
Begin Snap: 40650 01-12ÔÂ-12 06:40:03 1203 5.8
End Snap: 40669 01-12ÔÂ-12 09:50:01 1122 5.3
Elapsed: 189.96 (mins)
DB Time: 22,251.65 (mins)
Load profile
Per Second Per Transaction Per Exec Per Call
DB Time(s): 117.1 19.5 0.00 3.89
DB CPU(s): 16.1 2.7 0.00 0.53
Redo size: 12,759,186.3 2,126,361.0
Logical reads: 181,875.9 30,310.2
Block changes: 54,515.5 9,085.2
Physical reads: 1,340.3 223.4
Physical writes: 8,788.9 1,464.7
User calls: 30.1 5.0
Parses: 26.5 4.4
Hard parses: 0.4 0.1
W/A MB processed: 8.5 1.4
Logons: 0.1 0.0
Executes: 41,176.0 6,862.1
Rollbacks: 1.9 0.3
Transactions: 6.0
Time model statistics
Statistic Name Time (s) % of DB Time
sql execute elapsed time 1,334,935.55 99.99
PL/SQL execution elapsed time 1,180,376.60 88.41
DB CPU 182,904.44 13.7
repeated bind elapsed time 292.83 0.02
sequence load elapsed time 279.75 0.02
parse time elapsed 87.4 0.01
hard parse elapsed time 22.52 0
failed parse elapsed time 5.12 0
connection management call elapsed time 4.61 0
PL/SQL compilation elapsed time 1.91 0
hard parse (sharing criteria) elapsed time 0.49 0
hard parse (bind mismatch) elapsed time 0.39 0
inbound PL/SQL rpc elapsed time 0.1 0
DB time 1,335,099.30
background elapsed time 33,298.67
background cpu time 11,692.76
Operating System Statistics
Statistic Value End Value
AVG_BUSY_TIME 202,428
AVG_IDLE_TIME 936,397
AVG_IOWAIT_TIME 4,124
AVG_SYS_TIME 84,480
AVG_USER_TIME 117,573
BUSY_TIME 34,074,303
IDLE_TIME 157,378,825
IOWAIT_TIME 755,368
SYS_TIME 14,256,010
USER_TIME 19,818,293
LOAD 21 10
OS_CPU_WAIT_TIME 23,770,800
VM_IN_BYTES 20,496
VM_OUT_BYTES 2,086,940,520
PHYSICAL_MEMORY_BYTES 343,597,383,680
NUM_CPUS 168
NUM_CPU_CORES 42
NUM_LCPUS 168
NUM_VCPUS 42
GLOBAL_RECEIVE_SIZE_MAX 41,943,040
GLOBAL_SEND_SIZE_MAX 41,943,040
TCP_RECEIVE_SIZE_DEFAULT 16,384
TCP_RECEIVE_SIZE_MAX 9,223,372,036,854,775,807
TCP_RECEIVE_SIZE_MIN 4,096
TCP_SEND_SIZE_DEFAULT 16,384
TCP_SEND_SIZE_MAX 9,223,372,036,854,775,807
TCP_SEND_SIZE_MIN 4,096
SQL ordered by CPU Time
CPU Time (s) Executions CPU per Exec (s) %Total Elapsed Time (s) %CPU %IO SQL Id SQL Module SQL Text
180,330.13 127 1,419.92 98.59 1,326,401.03 13.60 1.08 04kt8u64udphu BEGIN :1 := ap_ch_eod_shell_en...
8,025.48 9,868,469 0.00 4.39 10,809.88 74.24 9.21 arnkbsnzhha77 ch_txn_shell_115 SELECT * FROM CH_ACCT_MAST WHE...
6,117.64 9,873,495 0.00 3.34 8,557.64 71.49 7.14 8qcryvj294s79 ch_eod_shell_138 UPDATE CH_ACCT_MAST_PAR SET DA...
4,614.71 3,185,313 0.00 2.52 11,130.77 41.46 11.88 b75wwkxw34x2k ch_eod_shell_228 INSERT INTO CH_TMP_XF_GL_TXNS ...
4,374.29 9,866,217 0.00 2.39 5,876.00 74.44 37.88 g22p493ra2zr5 ch_eod_shell_248 UPDATE CH_ACCT_MAST SET BAL_LA...
3,514.57 14,026,451 0.00 1.92 6,274.60 56.01 29.55 7bwhphfnnuqpr ch_eod_shell_59 INSERT INTO CH_ACCT_INT_BREAKU...
3,253.36 3,185,706 0.00 1.78 3,875.42 83.95 9.20 9dq134q9btxq8 ch_eod_shell_74 INSERT INTO CH_ST_CAP_INPUT_TX...
3,131.64 9,875,603 0.00 1.71 5,338.43 58.66 15.55 6xhwk1b37rh1t ch_txn_shell_143 UPDATE CH_ACCT_ATTRIBUTES SET ...
2,954.15 9,878,718 0.00 1.62 5,692.88 51.89 13.22 b4at7uq2hw6r7 ch_sweepin_shell SELECT TRIM(A.COD_PKG) FROM RP...
2,572.01 9,867,277 0.00 1.41 4,605.88 55.84 12.58 54ud0a8tuwwbc ch_txn_shell_17 SELECT * FROM CH_ACCT_ATTRIBUT...
1,941.29 19,730,455 0.00 1.06 5,580.38 34.79 7.02 dx5kng8qu560t ch_txn_shell_59 UPDATE CH_ACTIONS_DUE SET COD_...
1,846.01 9,875,239 0.00 1.01 4,737.66 38.96 12.55 af7f92f13rmy4 ch_txn_shell_85 INSERT INTO CH_ACTIONS_DUE (CO... -
Reducing shared pool latch contention
I see a lot of shared pool latch contention on my 11gR2 database because there are over 100 identical schemas that run identical SQL. This causes lots of child cursors to be created. Scheduled jobs kick off hourly for every schema at the same time and that produces contention for shared pool latches for the same parent cursor.
I want to reduce that contention. Here's my question...
If I make the SQL text unique by either
(a) changing the object referenbes from "TableName" to "SchemaName.TableName", or
(b) adding a comment to the SQL that includes the SchemaName
...will that cause a separate cursor to be created in the shared pool?
I know in older versions even a change in a comment would cause Oracle to treat it as a separate statement. Is that still true in 11gR2?
FWIW cursor_sharing is set to EXACT.Chuck1958 wrote:
If I make the SQL text unique by either
(a) changing the object referenbes from "TableName" to "SchemaName.TableName", or
(b) adding a comment to the SQL that includes the SchemaName
...will that cause a separate cursor to be created in the shared pool?
I know in older versions even a change in a comment would cause Oracle to treat it as a separate statement. Is that still true in 11gR2?
Yes:
SQL> select * from v$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for Linux: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
SQL> show parameter cursor_sharing
NAME TYPE VALUE
cursor_sharing string EXACT
SQL> select * /* schema1 */ from t where x=0;
no rows selected
SQL> select * /* schema2 */ from t where x=0;
no rows selected
SQL> select sql_id, child_number, sql_text
2 from v$sql
3 where sql_text like '%/* schema%';
SQL_ID CHILD_NUMBER SQL_TEXT
4d03bxnvbw4ac 0 select * /* schema2 */ from t where x=0
76xtkkpvfqx1h 0 select * /* schema1 */ from t where x=0
b3zjvv0cfbhrx 0 select sql_id, child_number, sql_text fr
om v$sql where sql_text like '%/* schema
%' -
Latch: session allocation in top 5 wait events
Hi,
This wait event session is coming in the top 5 timed events
How to proceed for solving this issue?
This database is only being used for migration activities currently,which means a lot of imports going on.
PFB the Top 5 Timed Foreground Events
++++++++++++++++++
Event Waits Time(s) Avg wait (ms) % DB time Wait Class
DB CPU 8,437 57.95
latch: session allocation 2,035,326 3,671 2 25.22 Other
wait list latch free 243,511 2,448 10 16.81 Other
direct path write 504,262 363 1 2.49 User I/O
log file sync 39,396 156 4 1.07 Commit
++++++++++++++++++
Cheers,
Kunwaruser9131570 wrote:
This database is only being used for migration activities currently,which means a lot of imports going on.
PFB the Top 5 Timed Foreground Events
++++++++++++++++++
Event Waits Time(s) Avg wait (ms) % DB time Wait Class
DB CPU 8,437 57.95
latch: session allocation 2,035,326 3,671 2 25.22 Other
wait list latch free 243,511 2,448 10 16.81 Other
direct path write 504,262 363 1 2.49 User I/O
log file sync 39,396 156 4 1.07 Commit
++++++++++++++++++
A little more information would help - you could start with the Load Profile section, and the summary of the number of sessions, CPU, and memory. Also OSSTATs might be helpful and the Time Model stats. The version of Oracle might also be relevant.
See the note below about using the "code" tag and preview option.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
There is a +"Preview"+ tab at the top of the text entry panel. Use this to check what your message will look like before you post the message. If it looks a complete mess you're unlikely to get a response. (Click on the +"Plain text"+ tab if you want to edit the text to tidy it up.)
+"Science is more than a body of knowledge; it is a way of thinking"+
+Carl Sagan+
Maybe you are looking for
-
How to access a variable from inside another symbol
So i did this tutorial, Leveraging Independent Symbol Timelines created by Eliane...it's rockin, btw. ;-) All's well but now i have a symbol on the stage called mc-home. inside of mc-home is a button called btn-go. On the stage in composition ready
-
I suddenly can't connect to printer on wireless home network.
I have an Officjet Pro L7680 and we just had a new cable company installation in our house. We have 1 desktop and 2 laptops. All three share this printer and we had no problems. Since the install, 1 laptop won't connect to the printer. The cable co.
-
A problem i encountered was i plugged my ipod into the comp after the upd8 and it said it needed to be updated it reinstalled itunes and theres was a problem with quicktime it didnt update and now itunes wont open, i downloaded it again clicked the f
-
HT1918 how to add a rescue email adress to recieve my reset security question ..i dont have one
how to add a rescue email adress to recieve my reset security question ..i dont have one
-
How to load file with 500M into noncontiguous memory and visit them
My data file consist of a serial of 2d images. Each image is 174*130*2 butes and the total images in the file is up to 11000, which takes almost 500M bytes. I hope to load them into memory in one turn since I need to visit either image many times in