High library cache load lock waits in AWR
Hi All,
Today i faced a significant performance problem related to shared pool. I made some observations, thought it would be a nice idea to share them with Oracle experts. Please feel free to add your observations/recommendations and correct me where i am wrong.
Here are the excerpts from AWR report created for the problem timing. Database server is on 10.2.0.3 and running with 2*16 configuration. DB cache size is 4,000M and shared pool size is of 3008M.
Snap Id Snap Time Sessions Cursors/Session
Begin Snap: 9994 29-Jun-09 10:00:07 672 66.3
End Snap: 10001 29-Jun-09 17:00:49 651 64.4
Elapsed: 420.70 (mins)
DB Time: 4,045.34 (mins) -- Very poor response time visible from difference between DB time and elapsed time.
Load Profile
Per Second Per Transaction
Redo size: 248,954.70 23,511.82
Logical reads: 116,107.04 10,965.40
Block changes: 1,357.13 128.17
Physical reads: 125.49 11.85
Physical writes: 51.49 4.86
User calls: 224.69 21.22
Parses: 235.22 22.21
Hard parses: 4.83 0.46
Sorts: 102.94 9.72
Logons: 1.12 0.11
Executes: 821.11 77.55
Transactions: 10.59 -- User calls and Parse count are almost same, means most of the calls are for parse. Most of the parses are soft. Per transaction 22 parses are very high figure.
-- Not much disk I/O activity. Most of the reads are being satisfy from memory.
Instance Efficiency
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.92 In-memory Sort %: 100.00
Library Hit %: 98.92 Soft Parse %: 97.95
Execute to Parse %: 71.35 Latch Hit %: 99.98
Parse CPU to Parse Elapsd %: 16.82 % Non-Parse CPU: 91.41 -- Low execute to parse ratio denotes CPU is significantly busy in parsing. Soft Parse% showing, most of the parse are soft parses. It means we should concentrate on soft parsing activity.
-- Parse CPU to Parse Elapsed % is quite low, means some bottleneck is there related to parsing. It could be a side-effect of huge parsing pressure. Like CPU cycles are not available.
Shared Pool Statistics
Begin End
Memory Usage %: 81.01 81.92
% SQL with executions>1: 88.51 86.93
% Memory for SQL w/exec>1: 86.16 86.76 -- Shared Pool memory seems ok (in 80% range)
-- 88% of the SQLs are repeating ones. It's a good sign.
Top 5 Timed Events
Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
library cache load lock 24,243 64,286 2,652 26.5 Concurrency
db file sequential read 1,580,769 42,267 27 17.4 User I/O
CPU time 33,039 13.6
latch: library cache 53,013 29,194 551 12.0 Concurrency
db file scattered read 151,669 13,550 89 5.6 User I/O Problem-1: Contention on Library cache: May be due to under-sized shared pool, incorrect parameters, poor application design, But since we already observed that most of the parses are soft parses and shared pool usgae in 80%, seems problem related to holding cursors. open_cursors/session_cached_cursors are red flags.
Problem-2: User I/O, may be due to poor SQLs, I/O sub-system, or poor physical design (wrong indexes are being used as DB file seq reads)
Wait Class
Wait Class Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn
Concurrency 170,577 44.58 109,020 639 0.64
User I/O 2,001,978 0.00 59,662 30 7.49
System I/O 564,771 0.00 8,069 14 2.11
Application 145,106 1.25 6,352 44 0.54
Commit 176,671 0.37 4,528 26 0.66
Other 27,557 6.31 2,532 92 0.10
Network 6,862,704 0.00 696 0 25.68
Configuration 3,858 3.71 141 37 0.01
Wait Events
Event Waits %Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn
library cache load lock 24,243 83.95 64,286 2652 0.09
db file sequential read 1,580,769 0.00 42,267 27 5.91
latch: library cache 53,013 0.00 29,194 551 0.20
db file scattered read 151,669 0.00 13,550 89 0.57
latch: shared pool 25,403 0.00 12,969 511 0.10
log file sync 176,671 0.37 4,528 26 0.66
enq: TM - contention 1,455 90.93 3,975 2732 0.01 Instance Activity Stats
opened cursors cumulative 5,290,760 209.60 19.80
parse count (failures) 6,181 0.24 0.02
parse count (hard) 121,841 4.83 0.46
parse count (total) 5,937,336 235.22 22.21
parse time cpu 283,787 11.24 1.06
parse time elapsed 1,687,096 66.84 6.31 Latch Activity
library cache 85,042,375 0.15 0.43 29194 304,831 7.16
library cache load lock 257,089 0.00 1.20 0 69,065 0.00
library cache lock 41,467,300 0.02 0.07 6 2,714 0.07
library cache lock allocation 730,422 0.00 0.44 0 0
library cache pin 28,453,986 0.01 0.16 8 167 0.00
library cache pin allocation 509,000 0.00 0.38 0 0 Init.ora parameters
cursor_sharing= EXACT
open_cursors= 3000
session_cached_cursors= 0
-- open_cursors value is too high. I have checked that maximum usage by a single session is 12%.
-- session_cached_cursors are 0 causing soft parsing. 500/600 is good number to start with.
cursor_sharing exact may cause hard parses. But here, hard parsing is comparatively small, we can ignore this.
From v$librarycache
NAMESPACE GETS GETHITS GETHITRATIO PINS PINHITRATIO RELOADS INVALIDATIONS
SQL AREA 162827 25127 .154317159 748901435 .999153087 107941 81886-- high invalidation count due to DDL like activities.
-- high reloads due to small library cache.
-- hit ratio too small.
-- Need to pin frequently executed objects into library cache.
P.S. Same question asked on Oracle_L, but due to formatting reasons, pasing duplicate contents here.
Regards,
Neeraj Bhatia
Edited by: Neeraj.Bhatia2 on Jul 13, 2009 6:51 AM
Thanks Charles. I really appreciate your efforts to diagnose the issue.
I agree with you performance issue is caused by soft parsing, which can be solved by holding cursors (session_cached_cursors). It may be due to oversized shared pool, which is causing delay in searching child cursors.
My second thought is, there is large number of reloads, which can be due to under-sized shared pool, if invalidation activities are not going (CBO statistics collection, DDL etc), cursors are being flushed frequently.
CPU utilization is continuously high (above 90%). Pasting additional information from same AWR report.
Namespace Get Requests Pct Miss Pin Requests Pct Miss Reloads Invalidations
BODY 225,345 0.76 4,965,541 0.15 5,533 0
CLUSTER 1,278 1.41 2,542 1.73 26 0
INDEX 5,982 9.31 13,922 7.35 258 0
SQL AREA 141,465 54.10 27,831,235 1.21 69,863 19,085 Latch Miss Sources
Latch Name Where NoWait Misses Sleeps Waiter Sleeps
library cache lock kgllkdl: child: no lock handle 0 8,250 5,792 Time Model Statistics
Statistic Name Time (s) % of DB Time
sql execute elapsed time 206,979.31 85.27
PL/SQL execution elapsed time 94,651.78 39.00
DB CPU 33,039.29 13.61
parse time elapsed 22,635.47 9.33
inbound PL/SQL rpc elapsed time 14,763.48 6.08
hard parse elapsed time 14,136.77 5.82
connection management call elapsed time 1,625.07 0.67
PL/SQL compilation elapsed time 760.76 0.31
repeated bind elapsed time 664.81 0.27
hard parse (sharing criteria) elapsed time 500.11 0.21
Java execution elapsed time 252.95 0.10
failed parse elapsed time 167.23 0.07
hard parse (bind mismatch) elapsed time 124.11 0.05
sequence load elapsed time 23.34 0.01
DB time 242,720.12
background elapsed time 11,645.52
background cpu time 247.25 According to this DB CPU is 65% utilization (DB CPU + Background CPU / Total Available CPU seconds). While at the same time DB host was 95% utilized (confirmed from DBA_HIST_SYSMETRIC_SUMMARY).
Operating System Statistics
Statistic Total
BUSY_TIME 3,586,030
IDLE_TIME 1,545,064
IOWAIT_TIME 22,237
NICE_TIME 0
SYS_TIME 197,661
USER_TIME 3,319,452
LOAD 11
RSRC_MGR_CPU_WAIT_TIME 0
PHYSICAL_MEMORY_BYTES 867,180
NUM_CPUS 2
Similar Messages
-
ROW CACHE ENQUEUE LOCK/ibrary cache load lock leads to database hung
(lowercase, curly brackets, no spaces)
We faced database hung on 3 node 11i erp 9i rac database.
We saw the library cache load lock timed out events reported in alert log.
Then few ora-600 and later ROW CACHE ENQUEUE LOCK timed out event. Eventually database was hung and we had to bounce the services .
we created support sr 7845542.992 for RCA.
The support says to increase shared pool size to avoid shared pool fragmentation and avoid reload ,additionaly to upgrade to 10g database.
I am not covinced adding additional pool size would solve this or upgrade to 10 .furthermore even 10g has such issues reported.
I saw couple of bugs mentioned such issue can happen due deadlock of session holding latches .
kindly let me know your view on issue
If required i can attach statspack for more information. (lowercase, curly brackets, no spaces)Many Thanks, i was keen to have your update .
There are 8 cpus on each node . Reloads very high during time period ,but normally there are not high reloads.
Statspack details for 3 nodes
STATSPACK report for
DB Name DB Id Instance Inst Num Release Cluster Host
PROD 21184234 PROD1 1 9.2.0.8.0 YES npi-or-db-p-
11.npi.corp
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 149817 30-Oct-09 13:00:09 574 #########
End Snap: 149837 30-Oct-09 14:00:17 602 #########
Elapsed: 60.13 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 8,192M Std Block Size: 8K
Shared Pool Size: 1,024M Log Buffer: 10,240K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 122,414.93 11,449.13
Logical reads: 69,550.76 6,504.89
Block changes: 928.41 86.83
Physical reads: 196.24 18.35
Physical writes: 28.65 2.68
User calls: 343.97 32.17
Parses: 558.61 52.25
Hard parses: 43.48 4.07
Sorts: 467.24 43.70
Logons: 0.63 0.06
Executes: 2,046.99 191.45
Transactions: 10.69
% Blocks changed per Read: 1.33 Recursive Call %: 97.59
Rollback per transaction %: 5.07 Rows per Sort: 15.85
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.72 In-memory Sort %: 100.00
Library Hit %: 96.79 Soft Parse %: 92.22
Execute to Parse %: 72.71 Latch Hit %: 99.77
Parse CPU to Parse Elapsd %: 60.10 % Non-Parse CPU: 78.07
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
db file sequential read 249,234 0 1,537 6 6.5
db file scattered read 61,776 0 769 12 1.6
row cache lock 780,098 10 566 1 20.2
library cache lock 697,849 157 432 1 18.1
latch free 127,926 4,715 387 3 3.3
global cache cr request 370,770 3,091 309 1 9.6
PL/SQL lock timer 59 58 112 1903 0.0
wait for scn from all nodes 303,572 18 103 0 7.9
library cache pin 26,231 2 100 4 0.7
global cache null to x 17,717 716 92 5 0.5
buffer busy waits 5,388 18 74 14 0.1
db file parallel read 5,245 0 69 13 0.1
log file sync 20,407 29 66 3 0.5
enqueue 52,200 70 60 1 1.4
buffer busy global CR 4,845 33 55 11 0.1
CGS wait for IPC msg 412,512 407,106 50 0 10.7
ksxr poll remote instances 1,279,565 483,046 48 0 33.2
log file parallel write 160,040 0 42 0 4.1
library cache load lock 1,491 2 29 20 0.0
global cache open x 19,507 344 28 1 0.5
buffer busy global cache 957 0 22 23 0.0
global cache s to x 16,516 180 20 1 0.4
db file parallel write 11,120 0 12 1 0.3
log file sequential read 618 0 11 18 0.0
DFS lock handle 23,768 0 10 0 0.6
control file sequential read 8,563 0 4 0 0.2
KJC: Wait for msg sends to c 1,549 57 4 3 0.0
lock escalate retry 76 76 4 52 0.0
SQL*Net break/reset to clien 12,546 0 3 0 0.3
SQL*Net more data to client 85,773 0 3 0 2.2
control file parallel write 1,265 0 2 1 0.0
global cache null to s 648 23 1 2 0.0
global cache busy 200 0 1 5 0.0
global cache open s 1,493 28 1 1 0.0
log file switch completion 12 0 1 61 0.0
PX Deq Credit: send blkd 161 70 1 4 0.0
kksfbc child completion 119 118 1 5 0.0
PX Deq: reap credit 5,948 5,456 0 0 0.2
PX Deq: Execute Reply 83 29 0 3 0.0
process startup 8 0 0 25 0.0
LGWR wait for redo copy 992 12 0 0 0.0
IPC send completion sync 450 450 0 0 0.0
PX Deq: Parse Reply 100 28 0 1 0.0
undo segment extension 10,380 10,372 0 0 0.3
PX Deq: Join ACK 146 65 0 1 0.0
buffer deadlock 222 221 0 0 0.0
async disk IO 1,179 0 0 0 0.0
wait list latch free 2 0 0 16 0.0
PX Deq: Msg Fragment 112 28 0 0 0.0
Library Cache Activity for DB: PROD Instance: PROD1 Snaps: 149817 -149837
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 116,007 1.1 133,347 19.9 24,338 0
CLUSTER 4,224 0.6 5,131 1.0 0 0
INDEX 15,048 24.1 13,798 26.4 2 0
JAVA DATA 82 0.0 692 39.6 136 0
JAVA RESOURCE 66 39.4 206 25.2 12 0
PIPE 1,140 0.5 1,160 0.5 0 0
SQL AREA 1,197,908 12.6 13,517,660 1.5 111,833 73
TABLE/PROCEDURE 3,847,439 0.8 4,230,265 7.9 142,200 0
TRIGGER 8,444 2.4 8,657 18.5 1,274 0
GES Lock GES Pin GES Pin GES Inval GES Invali-
Namespace Requests Requests Releases Requests dations
BODY 1 1,234 1,258 985 0
CLUSTER 3,222 25 25 25 0
INDEX 13,792 3,641 3,631 3,629 0
JAVA DATA 0 0 0 0 0
JAVA RESOURCE 0 26 25 0 0
PIPE 0 0 0 0 0
SQL AREA 0 0 0 0 0
TABLE/PROCEDURE 857,137 13,130 13,264 10,762 0
TRIGGER 0 200 202 200 0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
STATSPACK report for
DB Name DB Id Instance Inst Num Release Cluster Host
PROD 21184234 PROD2 2 9.2.0.8.0 YES npi-or-db-p-
12.npi.corp
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 149847 30-Oct-09 14:00:05 493 #########
End Snap: 149857 30-Oct-09 15:00:02 432 #########
Elapsed: 59.95 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 8,192M Std Block Size: 8K
Shared Pool Size: 1,024M Log Buffer: 10,240K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 71,853.44 32,058.65
Logical reads: 273,904.84 122,207.36
Block changes: 889.13 396.70
Physical reads: 40.40 18.03
Physical writes: 20.97 9.35
User calls: 153.74 68.60
Parses: 66.19 29.53
Hard parses: 2.66 1.19
Sorts: 25.70 11.47
Logons: 0.16 0.07
Executes: 726.41 324.10
Transactions: 2.24
% Blocks changed per Read: 0.32 Recursive Call %: 92.41
Rollback per transaction %: 4.84 Rows per Sort: 193.55
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 99.99
Buffer Hit %: 99.99 In-memory Sort %: 100.00
Library Hit %: 99.35 Soft Parse %: 95.97
Execute to Parse %: 90.89 Latch Hit %: 99.99
Parse CPU to Parse Elapsd %: 36.55 % Non-Parse CPU: 98.28
Wait Events for DB: PROD Instance: PROD2 Snaps: 149847 -149857
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
enqueue 65,823 33,667 90,459 1374 8.2
row cache lock 38,996 560 1,795 46 4.8
PX Deq Credit: send blkd 522 499 1,223 2344 0.1
PX Deq: Parse Reply 466 416 987 2117 0.1
db file sequential read 50,130 0 421 8 6.2
library cache lock 78,842 172 210 3 9.8
db file scattered read 6,904 0 152 22 0.9
global cache cr request 84,801 575 113 1 10.5
latch free 8,096 736 65 8 1.0
log file sync 5,676 27 41 7 0.7
wait for scn from all nodes 18,891 10 24 1 2.3
CGS wait for IPC msg 394,678 392,142 21 0 49.0
library cache pin 1,339 0 17 13 0.2
global cache null to x 2,145 48 16 8 0.3
global cache s to x 3,242 32 16 5 0.4
buffer busy waits 366 10 15 40 0.0
ksxr poll remote instances 70,990 31,295 14 0 8.8
db file parallel read 359 0 11 31 0.0
global cache open x 2,708 55 10 4 0.3
async disk IO 3,474 0 8 2 0.4
global cache open s 3,470 10 6 2 0.4
log file parallel write 13,076 0 5 0 1.6
global cache busy 58 40 5 90 0.0
PL/SQL lock timer 1 1 5 4877 0.0
DFS lock handle 3,362 0 5 1 0.4
log file sequential read 412 0 4 10 0.1
db file parallel write 2,774 0 3 1 0.3
library cache load lock 59 0 3 58 0.0
buffer busy global CR 722 0 3 4 0.1
control file sequential read 6,398 0 3 0 0.8
SQL*Net break/reset to clien 16,078 0 2 0 2.0
name-service call wait 26 0 2 67 0.0
control file parallel write 1,248 0 2 1 0.2
process startup 24 0 1 49 0.0
KJC: Wait for msg sends to c 3,491 4 1 0 0.4
SQL*Net more data to client 23,724 0 1 0 2.9
buffer busy global cache 23 0 0 19 0.0
global cache null to s 114 0 0 4 0.0
PX Deq: reap credit 5,646 5,509 0 0 0.7
log file switch completion 4 0 0 58 0.0
lock escalate retry 54 54 0 1 0.0
IPC send completion sync 119 118 0 0 0.0
direct path read 2,820 0 0 0 0.3
direct path read (lob) 3,632 0 0 0 0.5
PX Deq: Join ACK 88 37 0 0 0.0
direct path write 2,470 0 0 0 0.3
kksfbc child completion 6 6 0 6 0.0
buffer deadlock 3 3 0 11 0.0
global cache quiesce wait 4 4 0 8 0.0
Library Cache Activity for DB: PROD Instance: PROD2 Snaps: 149847 -149857
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 27,353 0.5 28,091 6.5 1,643 0
CLUSTER 203 1.0 269 1.5 0 0
INDEX 526 9.9 271 19.9 0 0
JAVA DATA 18 0.0 120 6.7 4 0
JAVA RESOURCE 20 45.0 56 26.8 3 0
JAVA SOURCE 1 100.0 1 100.0 0 0
PIPE 999 0.4 1,043 0.4 0 0
SQL AREA 131,793 7.6 3,406,577 0.4 7,012 0
TABLE/PROCEDURE 926,987 0.2 1,907,993 1.0 8,845 0
TRIGGER 1,519 0.1 1,532 4.9 69 0
GES Lock GES Pin GES Pin GES Inval GES Invali-
Namespace Requests Requests Releases Requests dations
BODY 1 129 277 117 0
CLUSTER 168 2 2 2 0
INDEX 271 52 56 52 0
JAVA DATA 0 0 0 0 0
JAVA RESOURCE 0 9 6 0 0
JAVA SOURCE 0 1 1 1 0
PIPE 0 0 0 0 0
SQL AREA 0 0 0 0 0
TABLE/PROCEDURE 89,523 764 868 460 0
TRIGGER 0 2 14 2 0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
DB Name DB Id Instance Inst Num Release Cluster Host
PROD 21184234 PROD3 3 9.2.0.8.0 YES npi-or-db-p-
13.npi.corp
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 149808 30-Oct-09 14:00:00 31 #########
End Snap: 149809 30-Oct-09 15:00:02 34 11,831.4
Elapsed: 60.03 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 8,192M Std Block Size: 8K
Shared Pool Size: 1,024M Log Buffer: 10,240K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 1,518.14 36,700.35
Logical reads: 1,333.43 32,235.02
Block changes: 5.09 123.01
Physical reads: 54.31 1,312.88
Physical writes: 3.91 94.44
User calls: 1.46 35.40
Parses: 2.24 54.21
Hard parses: 0.04 0.93
Sorts: 0.84 20.28
Logons: 0.06 1.45
Executes: 3.11 75.23
Transactions: 0.04
% Blocks changed per Read: 0.38 Recursive Call %: 94.31
Rollback per transaction %: 45.64 Rows per Sort: 215.97
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.99 Redo NoWait %: 100.00
Buffer Hit %: 96.21 In-memory Sort %: 100.00
Library Hit %: 99.07 Soft Parse %: 98.29
Execute to Parse %: 27.94 Latch Hit %: 99.98
Parse CPU to Parse Elapsd %: 69.88 % Non-Parse CPU: 97.92
Wait Events for DB: PROD Instance: PROD3 Snaps: 149808 -149809
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
enqueue 19,510 7,472 15,509 795 130.9
PX Deq: Parse Reply 1,152 1,071 2,577 2237 7.7
row cache lock 2,202 518 1,579 717 14.8
db file scattered read 31,556 0 354 11 211.8
db file sequential read 17,272 0 67 4 115.9
db file parallel read 1,722 0 34 20 11.6
global cache cr request 53,754 91 32 1 360.8
wait for scn from all nodes 1,897 13 10 5 12.7
CGS wait for IPC msg 403,358 401,478 10 0 2,707.1
DFS lock handle 4,753 0 8 2 31.9
direct path read 1,248 0 6 5 8.4
PX Deq: Execute Reply 110 38 6 51 0.7
global cache open s 160 10 5 31 1.1
control file sequential read 6,442 0 3 0 43.2
name-service call wait 26 0 2 78 0.2
latch free 129 109 2 13 0.9
KJC: Wait for msg sends to c 153 24 1 9 1.0
control file parallel write 1,245 0 1 1 8.4
buffer busy waits 199 0 1 6 1.3
process startup 20 0 1 44 0.1
global cache null to x 74 2 1 9 0.5
global cache null to s 19 0 1 29 0.1
global cache open x 268 1 1 2 1.8
library cache lock 1,150 0 0 0 7.7
PX Deq: Join ACK 129 48 0 3 0.9
log file parallel write 1,157 0 0 0 7.8
async disk IO 219 0 0 1 1.5
direct path write 1,024 0 0 0 6.9
ksxr poll remote instances 6,740 4,595 0 0 45.2
PX Deq: reap credit 6,580 6,511 0 0 44.2
buffer busy global CR 73 0 0 2 0.5
log file sequential read 11 0 0 10 0.1
log file sync 100 0 0 1 0.7
global cache s to x 282 2 0 0 1.9
db file parallel write 95 0 0 1 0.6
library cache pin 142 0 0 0 1.0
SQL*Net break/reset to clien 28 0 0 1 0.2
IPC send completion sync 81 81 0 0 0.5
PX Deq: Signal ACK 32 14 0 1 0.2
PX Deq Credit: send blkd 3 1 0 7 0.0
SQL*Net more data to client 841 0 0 0 5.6
PX Deq: Msg Fragment 37 17 0 0 0.2
log file single write 4 0 0 1 0.0
db file single write 1 0 0 1 0.0
SQL*Net message from client 4,213 0 13,673 3246 28.3
gcs remote message 214,784 75,745 7,016 33 1,441.5
wakeup time manager 233 233 6,812 29237 1.6
PX Idle Wait 2,338 2,294 5,686 2432 15.7
PX Deq: Execution Msg 2,151 1,979 4,796 2229 14.4
Library Cache Activity for DB: PROD Instance: PROD3 Snaps: 149808 -149809
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 1,290 0.0 1,290 0.0 0 0
CLUSTER 18 0.0 8 0.0 0 0
SQL AREA 4,893 2.0 36,371 0.5 2 0
TABLE/PROCEDURE 1,555 3.9 3,834 4.9 71 0
TRIGGER 286 0.0 286 0.0 0 0
GES Lock GES Pin GES Pin GES Inval GES Invali-
Namespace Requests Requests Releases Requests dations
BODY 1 0 0 0 0
CLUSTER 4 0 0 0 0
SQL AREA 0 0 0 0 0
TABLE/PROCEDURE 863 224 42 42 0
TRIGGER 0 0 0 0 0
------------------------------------------------------------- -
High wait on Library Cache Lock while doing ALTER TABLE EXCHANGE PARTITION
We are observing a very high wait time on "Library cache lock" while performing Exchange partition.
Here we go
ALTER TABLE PSALESREG EXCHANGE PARTITION P123
WITH TABLE PBKPSALESREF WITHOUT VALIDATION
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.11 *6684.73* 2 9 2 0
Fetch 0 0.00 0.00 0 0 0 0
total 2 0.11 6684.73 2 9 2 0
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
library cache lock 2274 3.12 *6681.32*
Is it a bug? is anyone there who experienced the same issue?
RgdsMaurice Muller wrote:
Hi,
As far as I remember a exchange partition can only be done when no other query is accessing the table.
So you should check if any other queries are executed against the table PSALESREG while you do the exchange partition. Maurice,
queries won't block the exchange operation but continue to read from the "original", exchanged segment; this "cross-DDL" read consistency is a feature that has been introduced a long time ago.
But any kind of (long-running) DML should effectively prevent the exchange operation. Still I would assume that this shouldn't show up as "library cache lock", but you would get an "ORA-00054: resource busy" error.
Regards,
Randolf
Oracle related stuff blog:
http://oracle-randolf.blogspot.com/
SQLTools++ for Oracle (Open source Oracle GUI for Windows):
http://www.sqltools-plusplus.org:7676/
http://sourceforge.net/projects/sqlt-pp/ -
Hi,
I am experience hanging in my database. I am using 10.2.0.3 on Solaris 5.10.
I am having automatic processes that drop the schema and recreate the schema and upload the new data. Since the last few days i am facing in hanging while dropping the schema. I checked and found "Library cache lock" event that caused other processes also in the hanged state. I am trying to find out the reason but not able to crack.
Can some one advice me?
ThanksHi,
There exists an concurrency problem with respect to the object - resource utilization.
- One client can prevent other clients from accessing the same object
- The client can maintain a dependency for a long time (for example, no other client can change the object)
check the from v$session_wait (query taken from the http://www.dba-oracle.com/m_library_cache_pin.htm - for your reference)
select decode(lob.kglobtyp, 0, 'NEXT OBJECT', 1, 'INDEX', 2, 'TABLE', 3, 'CLUSTER',
4, 'VIEW', 5, 'SYNONYM', 6, 'SEQUENCE',
7, 'PROCEDURE', 8, 'FUNCTION', 9, 'PACKAGE',
11, 'PACKAGE BODY', 12, 'TRIGGER',
13, 'TYPE', 14, 'TYPE BODY',
19, 'TABLE PARTITION', 20, 'INDEX PARTITION', 21, 'LOB',
22, 'LIBRARY', 23, 'DIRECTORY', 24, 'QUEUE',
28, 'JAVA SOURCE', 29, 'JAVA CLASS', 30, 'JAVA RESOURCE',
32, 'INDEXTYPE', 33, 'OPERATOR',
34, 'TABLE SUBPARTITION', 35, 'INDEX SUBPARTITION',
40, 'LOB PARTITION', 41, 'LOB SUBPARTITION',
42, 'MATERIALIZED VIEW',
43, 'DIMENSION',
44, 'CONTEXT', 46, 'RULE SET', 47, 'RESOURCE PLAN',
48, 'CONSUMER GROUP',
51, 'SUBSCRIPTION', 52, 'LOCATION',
55, 'XML SCHEMA', 56, 'JAVA DATA',
57, 'SECURITY PROFILE', 59, 'RULE',
62, 'EVALUATION CONTEXT',
'UNDEFINED') object_type,
lob.KGLNAOBJ object_name,
pn.KGLPNMOD lock_mode_held,
pn.KGLPNREQ lock_mode_requested,
ses.sid,
ses.serial#,
ses.username
FROM
x$kglpn pn,
v$session ses,
x$kglob lob,
v$session_wait vsw
WHERE
pn.KGLPNUSE = ses.saddr and
pn.KGLPNHDL = lob.KGLHDADR
and lob.kglhdadr = vsw.p1raw
and vsw.event = 'library cache pin'
order by lock_mode_held desc
Resolution might - you need to check the time it takes (the script to drop the schema) and it loads - either doing from parallel sessions or in serial order. I doubt it, till the schema get dropped completely it should not be accessible to any other applications - either objects too, that makes the rise to concurrency Issue with respect to library cache waits - the other sessions are trying access the object.
- Pavan Kumar N
- ORACLE 9i/10g - OCP
http://www.oracleinternals.blogspot.com -
Oracle RAC 9i LMD library cache lock top wait event
We are experiencing the library cache lock as our top wait event. Even thought the box is currently idle, The Global Enqueue Service Daemon (LMD) is taking up CPU cycles. The background process is also logging to trace "skgxpdocon: warning outstanding accept handle count has reached new high water mark 245000".
Any help would be appreciated.
ThanksThere is a new patch for this - check out p4673610 on metalink. We have also experience the problem in 9.2.0.8.
-
A killed session is still waiting for "library cache lock" for hours
Hello everybody
Today on my environment I killed a session while compiling a procedure that was taking too much time. Now, after more than a couple of hours that session is still there waiting for "library cache lock". I really can't understand why it's taking that much. Can anyone give me some hints about this. I already read the docs to find something to explain this case but I haven't been able to find the source of this lock.
Down here I past a description of what that procedure has been doing.
Processing ...
SELECT saddr, SID, command,taddr,lockwait,status,action
FROM v$session
WHERE SID = 636
Query finished, retrieving results...
SADDR SID COMMAND TADDR LOCKWAIT STATUS ACTION
C00000023C7A2438 636 24 C000000238FB1650 KILLED Main session
1 row(s) retrieved
Processing ...
SELECT *
FROM v$session_wait
WHERE SID = 636
Query finished, retrieving results...
SID SEQ# EVENT P1TEXT P1 P1RAW P2TEXT P2 P2RAW P3TEXT P3 P3RAW WAIT_CLASS_ID WAIT_CLASS# WAIT_CLASS WAIT_TIME SECONDS_IN_WAIT STATE
636 247 library cache lock handle address 13835058063333980008 C0000001DFECF768 lock address 13835058064378433728 C00000021E2E10C0 100*mode+namespace 301 000000000000012D 3875070507 4 Concurrency 38 6089 WAITED KNOWN TIME
1 row(s) retrieved
Processing ...
select *
from v$session_event
where sid = 636
Query finished, retrieving results...
SID EVENT TOTAL_WAITS TOTAL_TIMEOUTS TIME_WAITED AVERAGE_WAIT MAX_WAIT TIME_WAITED_MICRO EVENT_ID WAIT_CLASS_ID WAIT_CLASS# WAIT_CLASS
636 db file sequential read 3 0 4 1,5 3 44994 2652584166 1740759767 8 User I/O
636 library cache lock 58 57 16737 288,56 293 167366653 916468430 3875070507 4 Concurrency
636 SQL*Net message to client 94 0 0 0 0 46 2067390145 2000153315 7 Network
636 SQL*Net more data to client 25 0 0 0 0 294 554161347 2000153315 7 Network
636 SQL*Net message from client 94 0 2848 30,3 2246 28480966 1421975091 2723168908 6 Idle
636 SQL*Net more data from client 28 0 0 0,02 0 4471 3530226808 2000153315 7 Network
636 SQL*Net break/reset to client 2 0 0 0,01 0 194 1963888671 4217450380 1 Application
7 row(s) retrieved Thanks
Bye Alessandro
Edited by: Alessandro Rossi on 11-nov-2008 11.37Alessandro Rossi wrote:
At the end I decided to kill the server process and now everything goes fine. i don't know why but the problem seems associated to the client application used to compile the procedure, because every time the developer tried to compile it using plslqldeveloper 7.14 through "edit-procedure" from his host the behavior was always like I just described before while the compilation of that procedure with other tools or from other hosts runs just fine.
Actually I haven't been able to find the reason for this, there was no view, among the ones I queried, telling me who was holding the resources, other than the blocked session, and it seems very hard that such a behavior may depend on the client tool used.
Thanks anywayAlessandro,
you may want to check the settings of this particular PL/SQL Developer installation. If I remember correctly then there are several settings in PL/SQL Developer related to how the tool connects to the database and compiles objects, e.g. "Safe compilation (using temporary compilation object)", "Background compilation", "Multi-Session" support etc. that might lead to situations where the tool could block itself for instance.
Regards,
Randolf
Oracle related stuff blog:
http://oracle-randolf.blogspot.com/
SQLTools++ for Oracle (Open source Oracle GUI for Windows):
http://www.sqltools-plusplus.org:7676/
http://sourceforge.net/projects/sqlt-pp/ -
High CPU load with Library Cache Pin wit event.
Oracle8i Enterprise Edition Release 8.1.6.0.0, 64 bit - Production
HP-UX B.11.00 U 9000/800 605309363 unlimited-user license
Currently CPU load is 100% with very less sessions in the database.
But i am finding Library Cache Pin wait event in top.
If i query v$latchholder i am not getting any rows..and in v$lacth_children
SQL> select SID,EVENT,WAIT_TIME,SECONDS_IN_WAIT,STATE from v$session_wait where EVENT='library cache pin';
SID EVENT WAIT_TIME SECONDS_IN_WAIT STATE
8 library cache pin 0 863 WAITING
90 library cache pin 0 3093 WAITING
89 library cache pin 0 2566 WAITING
57 library cache pin 0 27384 WAITING
54 library cache pin 0 21029 WAITING
53 library cache pin 0 7840 WAITING
50 library cache pin 0 2620 WAITING
47 library cache pin 0 6031 WAITING
46 library cache pin 0 1241 WAITING
41 library cache pin 0 15637 WAITING
145 library cache pin 0 910 WAITING
133 library cache pin 0 5124 WAITING
111 library cache pin 0 15077 WAITING
98 library cache pin 0 6563 WAITING
94 library cache pin 0 2088 WAITING
93 library cache pin 0 4592 WAITING
92 library cache pin 0 14705 WAITING
91 library cache pin 0 14798 WAITING
36 library cache pin 0 1533 WAITING
33 library cache pin 0 1491 WAITING
28 library cache pin 0 13970 WAITING
25 library cache pin 0 7630 WAITING
11 library cache pin 0 12169 WAITING
12 library cache pin 0 2352 WAITING
14 library cache pin 0 19748 WAITING
SQL> select addr,name from v$latch where name='library cache';
ADDR NAME
C00000006971D460 library cache
SQL> select latch#,name from v$latch_children where name='library cache';
LATCH# NAME
105 library cache
105 library cache
105 library cache
105 library cache
105 library cachePlease help me out to find actual cause of these latches and fix ?
-Yasserbump
No-one?
The problem seems to have 'resolved itself' over the weekend incidentally (although I don't believe anything truly resolves itself in this manner - something must have changed). -
Oracle 10.2.0.4 library cache contention,every event wait 64s! strange
my database suddently very slow in a few seconds.After a while, all become normal.
In awr report,I find no special sql.
I paste two trouble time awr report here.
If you have any more information for troubleshooting ,I will paste if you require.
Thanks.
AWR report1:
WORKLOAD REPOSITORY report for
DB Name DB Id Instance Inst Num Release RAC Host
DB 3594421410 db2 2 10.2.0.4.0 YES db2
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 2342 17-Mar-09 21:30:23 521 1.1
End Snap: 2344 17-Mar-09 22:30:25 498 1.0
Elapsed: 60.03 (mins)
DB Time: 750.95 (mins)
Cache Sizes
~~~~~~~~~~~ Begin End
Buffer Cache: 3,504M 3,504M Std Block Size: 8K
Shared Pool Size: 1,200M 1,200M Log Buffer: 14,340K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 18,182.64 2,754.51
Logical reads: 326.19 49.41
Block changes: 123.54 18.72
Physical reads: 21.33 3.23
Physical writes: 7.29 1.10
User calls: 178.46 27.03
Parses: 62.04 9.40
Hard parses: 0.09 0.01
Sorts: 2.00 0.30
Logons: 0.20 0.03
Executes: 63.12 9.56
Transactions: 6.60
% Blocks changed per Read: 37.87 Recursive Call %: 17.93
Rollback per transaction %: 14.26 Rows per Sort: 44.63
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.99 Redo NoWait %: 99.95
Buffer Hit %: 93.46 In-memory Sort %: 100.00
Library Hit %: 99.52 Soft Parse %: 99.86
Execute to Parse %: 1.70 Latch Hit %: 99.97
Parse CPU to Parse Elapsd %: 0.82 % Non-Parse CPU: 98.60
Shared Pool Statistics Begin End
Memory Usage %: 78.60 78.77
% SQL with executions>1: 99.38 99.27
% Memory for SQL w/exec>1: 99.00 98.78
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
latch: library cache 736 47,078 63965 104.5 Concurrenc
CPU time 236 0.5
rdbms ipc reply 360 219 608 0.5 Other
log file sync 20,469 137 7 0.3 Commit
gc cr block 2-way 35,641 102 3 0.2 Cluster
AWR report 2:
DB Name DB Id Instance Inst Num Release RAC Host
db 3594421410 db2 2 10.2.0.4.0 YES yt-db2
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 2364 18-Mar-09 08:30:29 497 1.1
End Snap: 2365 18-Mar-09 08:42:28 511 1.0
Elapsed: 11.99 (mins)
DB Time: 277.14 (mins)
Cache Sizes
~~~~~~~~~~~ Begin End
Buffer Cache: 3,504M 3,504M Std Block Size: 8K
Shared Pool Size: 1,200M 1,200M Log Buffer: 14,340K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 23,306.73 2,556.74
Logical reads: 337.31 37.00
Block changes: 159.74 17.52
Physical reads: 0.72 0.08
Physical writes: 9.74 1.07
User calls: 274.07 30.06
Parses: 95.29 10.45
Hard parses: 0.04 0.00
Sorts: 2.52 0.28
Logons: 0.19 0.02
Executes: 95.71 10.50
Transactions: 9.12
% Blocks changed per Read: 47.36 Recursive Call %: 9.14
Rollback per transaction %: 11.13 Rows per Sort: 32.48
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 98.81 Redo NoWait %: 100.00
Buffer Hit %: 99.79 In-memory Sort %: 100.00
Library Hit %: 99.86 Soft Parse %: 99.95
Execute to Parse %: 0.44 Latch Hit %: 99.94
Parse CPU to Parse Elapsd %: 40.91 % Non-Parse CPU: 81.90
Shared Pool Statistics Begin End
Memory Usage %: 79.91 80.01
% SQL with executions>1: 99.52 99.52
% Memory for SQL w/exec>1: 98.83 98.77
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
latch free 176 4,391 24950 26.4 Other
gc buffer busy 3,726 3,335 895 20.1 Cluster
gc cr multi block request 4,165 2,204 529 13.3 Cluster
gc current grant busy 3,938 1,798 457 10.8 Cluster
latch: cache buffers chains 124 1,548 12487 9.3 Concurrenc
^LRAC Statistics DB/Inst: db/db2 Snaps: 2364-2365
Begin End
Number of Instances: 2 2
Global Cache Load Profile
~~~~~~~~~~~~~~~~~~~~~~~~~ Per Second Per Transaction
Global Cache blocks received: 19.36 2.12
Global Cache blocks served: 19.39 2.13
GCS/GES messages received: 63.76 6.99
GCS/GES messages sent: 63.64 6.98
DBWR Fusion writes: 2.58 0.28
Estd Interconnect traffic (KB) 334.84
Edited by: gaoyafang on 2009-3-18 上午12:46my database suddently very slow in a few seconds.After a while, all become normal.
In awr report,I find no special sql.
I paste two trouble time awr report here.
If you have any more information for troubleshooting ,I will paste if you require.
Thanks.
AWR report1:
WORKLOAD REPOSITORY report for
DB Name DB Id Instance Inst Num Release RAC Host
DB 3594421410 db2 2 10.2.0.4.0 YES db2
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 2342 17-Mar-09 21:30:23 521 1.1
End Snap: 2344 17-Mar-09 22:30:25 498 1.0
Elapsed: 60.03 (mins)
DB Time: 750.95 (mins)
Cache Sizes
~~~~~~~~~~~ Begin End
Buffer Cache: 3,504M 3,504M Std Block Size: 8K
Shared Pool Size: 1,200M 1,200M Log Buffer: 14,340K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 18,182.64 2,754.51
Logical reads: 326.19 49.41
Block changes: 123.54 18.72
Physical reads: 21.33 3.23
Physical writes: 7.29 1.10
User calls: 178.46 27.03
Parses: 62.04 9.40
Hard parses: 0.09 0.01
Sorts: 2.00 0.30
Logons: 0.20 0.03
Executes: 63.12 9.56
Transactions: 6.60
% Blocks changed per Read: 37.87 Recursive Call %: 17.93
Rollback per transaction %: 14.26 Rows per Sort: 44.63
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.99 Redo NoWait %: 99.95
Buffer Hit %: 93.46 In-memory Sort %: 100.00
Library Hit %: 99.52 Soft Parse %: 99.86
Execute to Parse %: 1.70 Latch Hit %: 99.97
Parse CPU to Parse Elapsd %: 0.82 % Non-Parse CPU: 98.60
Shared Pool Statistics Begin End
Memory Usage %: 78.60 78.77
% SQL with executions>1: 99.38 99.27
% Memory for SQL w/exec>1: 99.00 98.78
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
latch: library cache 736 47,078 63965 104.5 Concurrenc
CPU time 236 0.5
rdbms ipc reply 360 219 608 0.5 Other
log file sync 20,469 137 7 0.3 Commit
gc cr block 2-way 35,641 102 3 0.2 Cluster
AWR report 2:
DB Name DB Id Instance Inst Num Release RAC Host
db 3594421410 db2 2 10.2.0.4.0 YES yt-db2
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 2364 18-Mar-09 08:30:29 497 1.1
End Snap: 2365 18-Mar-09 08:42:28 511 1.0
Elapsed: 11.99 (mins)
DB Time: 277.14 (mins)
Cache Sizes
~~~~~~~~~~~ Begin End
Buffer Cache: 3,504M 3,504M Std Block Size: 8K
Shared Pool Size: 1,200M 1,200M Log Buffer: 14,340K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 23,306.73 2,556.74
Logical reads: 337.31 37.00
Block changes: 159.74 17.52
Physical reads: 0.72 0.08
Physical writes: 9.74 1.07
User calls: 274.07 30.06
Parses: 95.29 10.45
Hard parses: 0.04 0.00
Sorts: 2.52 0.28
Logons: 0.19 0.02
Executes: 95.71 10.50
Transactions: 9.12
% Blocks changed per Read: 47.36 Recursive Call %: 9.14
Rollback per transaction %: 11.13 Rows per Sort: 32.48
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 98.81 Redo NoWait %: 100.00
Buffer Hit %: 99.79 In-memory Sort %: 100.00
Library Hit %: 99.86 Soft Parse %: 99.95
Execute to Parse %: 0.44 Latch Hit %: 99.94
Parse CPU to Parse Elapsd %: 40.91 % Non-Parse CPU: 81.90
Shared Pool Statistics Begin End
Memory Usage %: 79.91 80.01
% SQL with executions>1: 99.52 99.52
% Memory for SQL w/exec>1: 98.83 98.77
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
latch free 176 4,391 24950 26.4 Other
gc buffer busy 3,726 3,335 895 20.1 Cluster
gc cr multi block request 4,165 2,204 529 13.3 Cluster
gc current grant busy 3,938 1,798 457 10.8 Cluster
latch: cache buffers chains 124 1,548 12487 9.3 Concurrenc
^LRAC Statistics DB/Inst: db/db2 Snaps: 2364-2365
Begin End
Number of Instances: 2 2
Global Cache Load Profile
~~~~~~~~~~~~~~~~~~~~~~~~~ Per Second Per Transaction
Global Cache blocks received: 19.36 2.12
Global Cache blocks served: 19.39 2.13
GCS/GES messages received: 63.76 6.99
GCS/GES messages sent: 63.64 6.98
DBWR Fusion writes: 2.58 0.28
Estd Interconnect traffic (KB) 334.84
Edited by: gaoyafang on 2009-3-18 上午12:46 -
How to resolve Library Cache Pin waits?
Hai All,
I grant a select privilage on a table to another user. Then my session hangs. after 5 minutes a message present that is 'ORA-04021: timeout occurred while waiting to lock object....' etc.
I query the v$session_wait and see a Library Cache pin wait present in my session. I copy the p1raw column from here and put into some x$ tables and v$session then I can identify one session using a application using that table. I kill that session . then my grant privilage query works..
So what is library cache pin waits.. How to identify in application...How to resolve it? it resolve in database level or application level?
Please Help..
ShijuMost common cause of library cache pin wait event are
-- Not use binding variables in the query
-- Some one keeps modify the schema objects
library cache pin
This event manages library cache concurrency. Pinning an object causes the heaps to be loaded into memory. If a client wants to modify or examine the object, the client must acquire a pin after the lock.
Wait Time: 3 seconds (1 second for PMON)
Parameter Description
handle address Address of the object being loaded
pin address Address of the load lock being used. This is not the same thing as a latch or an enqueue, it is basically a State Object.
mode Indicates which data pieces of the object that needs to be loaded
namespace See "namespace" -
Performance issues; waited too long for a row cache enqueue lock!
hi Experts,
OS: Oracle Solaris on SPARC (64-bit)
DB version:
SQL> select * from V$VERSION;
BANNER
Oracle Database 11g Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for Solaris: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
SQL>We have seen 100% CPU usage and high database load, so I checked the instance and have seen there were many blocking sessions and more than 71 sessions running the same select ;
elect tablespace_name as tbsname from (select tablespace_name,sum(bytes)/1024/1024 free_mb,0 total_mb,0 max_mb from dba_free_space group by tablespace_name union select tablespace_name, 0 current_mb,sum(bytes)/1024/1024 total_mb, sum(decode(maxbytes, 0, bytes, maxbytes))/1024/1024 max_mb from dba_data_files group by tablespace_name) group by tablespace_name having round((sum(total_mb)-sum(free_mb))/sum(max_mb)*100) > 95 Blocking sessions are running queries like this;
SELECT * from MYTABLE WHERE MYCOL=:1 FOR UPDATE;This select queries are coming from a cron job running every 10 minutes to check the tablespaces; so I first killed (kill -9 pid) those select statements so the load and CPU decreased to 13% of CPU usage. Blocking sessions still there and I didn't killed them waiting for app guys confirmation... after few hours and the CPU usage never went down the 13%; I have seen many errors;
WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=...System State dumped to trace file .....trcAfter that , we decided to restart the DB to release the locks!
I would like to understand why during loads we were no able to run those select statements, statspack schedule snapshot reports were not able to finish, also automatic
database statistics... why 5 for update statements locked the whole DB?user12035575 wrote:
SELECT FOR UPDATE will only lock the table row until the transaction is completed.
"WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK" happens when it needs to acquire a lock on data dictionary. Did you check the trace file associated with the statement?The trace file is too long, which information I need to focus more? -
How to detect the object or session in a database with library cache lock
Hi Everyone,
We've been experiencing frequent and high time waits for this event: library cache lock.
- What causes this?
- Is there a way to detect the object being locked?
(this doesnt show up in V$LOCK)
- Is there a way to detect the session that is causing the lock? aka blocking session?
(I can detect the objects being blocked, they show up in v$session)
thanks,Similar post is here, maybe that helps already:
library cache lock
And You can read this as well:
http://www.ixora.com.au/q+a/0101/19235723.htm -
先library cache pin还是先library cache lock??
Question from Oracler:
session1 给test 建主键
session2 select test 出现library cache lock
session3 select test 出现library cache pin
不是说先获得library cache lock再library cache pin吗
session1以exclusive模式获得 library cache lock
session2 以shared模式请求 library cache lock ，session1未释放，所以session2 wait
那session3 什么解释呢？？？？as maclean answer:
SQL> select * from V$version;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
PL/SQL Release 10.2.0.1.0 - Production
CORE 10.2.0.1.0 Production
TNS for Linux: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production
session A SQL> alter table maclean add a10 char(2000) default 'maclean';
session B: select 1 from maclean where rownum=1; ==> hang here !
session C: select 1 from maclean where rownum=1; ==> SAME SQL, hang here !
SQL> select event from v$session where event like 'library%';
EVENT
library cache lock
library cache pin
session 4:
SQL> oradebug setmypid;
Statement processed.
SQL> oradebug dump systemstate 266;
Statement processed.
SQL> oradebug tracefile_name;
/s01/admin/G10R21/udump/g10r21_ora_6208.trc
SO: 0x84f5b4a8, type: 4, owner: 0x84e5d4f8, flag: INIT/-/-/0x00
(session) sid: 142 trans: (nil), creator: 0x84e5d4f8, flag: (41) USR/- BSY/-/-/-/-/-
DID: 0001-0010-00000027, short-term DID: 0000-0000-00000000
txn branch: (nil)
oct: 3, prv: 0, sql: 0x7bf10088, psql: 0x7bf582f0, user: 0/SYS
O/S info: user: oracle, term: pts/1, ospid: 6159, machine: vrh8.oracle.com
program: [email protected] (TNS V1-V3)
application name: [email protected] (TNS V1-V3), hash value=0
waiting for 'library cache lock' blocking sess=0x(nil) seq=23 wait_time=0 seconds since wait started=17
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
Dumping Session Wait History
for 'library cache lock' count=1 wait_time=2149666
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930643
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930300
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930715
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2930545
handle address=7c3a5560, lock address=8003b350, 100*mode+namespace=c9
for 'library cache lock' count=1 wait_time=2929985
session 142 is B waiting for library cache lock
the lock handle address is 7c3a5560
SO: 0x8003b350, type: 53, owner: 0x84f98ba0, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=8003b350 handle=7c3a5560 request=S
call pin=(nil) session pin=(nil) hpc=0005 hlc=0000
htl=0x8003b3d0[0x7d034330,0x7d034330] htb=0x7d034330 ssga=0x7e9f2168
user=84f5b4a8 session=84f5b4a8 count=0 flags=RES/[0010] savepoint=0x1f9
LIBRARY OBJECT HANDLE: handle=7c3a5560 mutex=0x7c3a5690(0)
name=SYS.MACLEAN
hash=458787ae49fd6f284ccb04a892b38231 timestamp=02-09-2012 21:32:36
namespace=TABL flags=KGHP/TIM/SML/[02000000]
kkkk-dddd-llll=0000-0701-0701 lock=X pin=X latch#=3 hpc=0006 hlc=0004
lwt=0x7c3a5608[0x8003b380,0x8003b380] ltm=0x7c3a5618[0x7c3a5618,0x7c3a5618]
pwt=0x7c3a55d0[0x7c3a55d0,0x7c3a55d0] ptm=0x7c3a55e0[0x7c3a55e0,0x7c3a55e0]
ref=0x7c3a5638[0x7c3a5638,0x7c3a5638] lnd=0x7c3a5650[0x7bf75a18,0x7bf90650]
LIBRARY OBJECT: object=7c1dec60
type=TABL flags=EXS/LOC/UPD[0905] pflags=[0000] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change whr
0 7c3a54a0 7c1ded78 I/P/A/-/- 0 NONE 00
8 7c1de7f0 7e33ed48 I/P/A/-/- 1 UPDATE 00
9 7c1de8c0 7bf109e8 I/P/A/-/- 1 NONE 00
10 7c1de948 7bf10600 I/P/A/-/- 1 NONE 00
关于session B的 library cache lock ， 其原因是 add column 的session A 以 X mode lock SYS.MACLEAN, X mode pin SYS.MACLEAN 且不释放， 所以session B的 library cache lock不用多解释
session C:
SO: 0x84f5dd18, type: 4, owner: 0x84e5dce0, flag: INIT/-/-/0x00
(session) sid: 144 trans: (nil), creator: 0x84e5dce0, flag: (41) USR/- BSY/-/-/-/-/-
DID: 0001-0011-0000000A, short-term DID: 0000-0000-00000000
txn branch: (nil)
oct: 3, prv: 0, sql: 0x7bf10088, psql: 0x7bf582f0, user: 0/SYS
O/S info: user: oracle, term: pts/2, ospid: 6183, machine: vrh8.oracle.com
program: [email protected] (TNS V1-V3)
application name: [email protected] (TNS V1-V3), hash value=0
waiting for 'library cache pin' blocking sess=0x(nil) seq=19 wait_time=0 seconds since wait started=17
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
Dumping Session Wait History
for 'library cache pin' count=1 wait_time=2568684
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2930677
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2929805
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2931420
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
for 'library cache pin' count=1 wait_time=2930258
handle address=7bf46e40, pin address=7f03f890, 100*mode+namespace=c8
session 144 is session C , waiting for library cache pin
handle address 7bf46e40=> 指向 一个 child cursor namespace=CRSR， 而这个child cursor已经被 session B pin住了：
SO: 0x7f03f890, type: 54, owner: 0x84f5dd18, flag: INIT/-/-/0x00
LIBRARY OBJECT PIN: pin=7f03f890 handle=7bf46e40 request=S lock=0
user=84f5dd18 session=84f5dd18 count=0 mask=0000 savepoint=0x3f flags=[00]
SO: 0x7ec4cc80, type: 53, owner: 0x84f5dd18, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=7ec4cc80 handle=7bf46e40 mode=N
call pin=(nil) session pin=(nil) hpc=0000 hlc=0000
htl=0x7ec4cd00[0x7e449348,0x80c35108] htb=0x80c35108 ssga=0x80c34ff0
user=84f5dd18 session=84f5dd18 count=1 flags=[0000] savepoint=0x0
LIBRARY OBJECT HANDLE: handle=7bf46e40 mutex=0x7bf46f70(0)
namespace=CRSR flags=RON/KGHP/PN0/[10010000]
kkkk-dddd-llll=0000-0001-0000 lock=N pin=X latch#=3 hpc=0004 hlc=0004
lwt=0x7bf46ee8[0x7bf46ee8,0x7bf46ee8] ltm=0x7bf46ef8[0x7bf46ef8,0x7bf46ef8]
pwt=0x7bf46eb0[0x7f03f8c0,0x7f03f8c0] ptm=0x7bf46ec0[0x7bf46ec0,0x7bf46ec0]
ref=0x7bf46f18[0x7bf7bfe0,0x7bf7bfe0] lnd=0x7bf46f30[0x7bf46f30,0x7bf46f30]
LIBRARY OBJECT: object=7bf29018
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
READ ONLY DEPENDENCIES: count=1 size=16
DATA BLOCKS:
data# heap pointer status pins change whr
0 7bf20060 7bf28ba8 I/P/A/-/- 0 NONE 00
6 7bf77a20 7bf776f8 I/P/A/-/- 1 NONE 00
我们可以找到 上面这个child cursor 的 parent cursor :
SO: 0x7d03b620, type: 53, owner: 0x84f5b4a8, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=7d03b620 handle=7bf10088 mode=N
call pin=(nil) session pin=(nil) hpc=0000 hlc=0000
htl=0x7d03b6a0[0x7d034030,0x7c03c9f8] htb=0x7d034030 ssga=0x7e9f2168
user=84f5b4a8 session=84f5b4a8 count=1 flags=[0000] savepoint=0x1f7
LIBRARY OBJECT HANDLE: handle=7bf10088 mutex=0x7bf101b8(0)
name=select 1 from maclean where rownum=1
hash=324793c639b13d0954bd5421eaed6701 timestamp=03-08-2012 02:29:24
namespace=CRSR flags=RON/KGHP/TIM/KEP/PN0/SML/DBN/[12010044]
kkkk-dddd-llll=0001-0001-0001 lock=N pin=0 latch#=3 hpc=0004 hlc=0004
lwt=0x7bf10130[0x7bf10130,0x7bf10130] ltm=0x7bf10140[0x7bf10140,0x7bf10140]
pwt=0x7bf100f8[0x7bf100f8,0x7bf100f8] ptm=0x7bf10108[0x7bf10108,0x7bf10108]
ref=0x7bf10160[0x7bf10160,0x7bf10160] lnd=0x7bf10178[0x82f4f2f8,0x7bf4d608]
LIBRARY OBJECT: object=7bf7c8a8
type=CRSR flags=EXS[0001] pflags=[0000] status=VALD load=0
CHILDREN: size=16
child# table reference handle
0 7bf7c370 7bf7bfe0 7bf46e40 => 只有一个child handle 7bf46e40
DATA BLOCKS:
data# heap pointer status pins change whr
0 7bf2a428 7bf7c9c0 I/P/A/-/- 0 NONE 00即在session B parse SQL "select 1 from maclean where rownum=1" 的时候， 会生成一个child cursor 并 X mode pin住这个child cursor ， 而session C 同时发起 一样的SQL语句 "select 1 from maclean where rownum=1" 时 需要 share 这个child cursor ， 即以 S mode pin 这个child cursor ， 但是session B 还没有完成 optimize 没有生成完整的child cursor ， 需要等待 session A 释放 library cache lock才能 完成， 所以 session C 要等 session B build child cursor ， 此时session C等" library cache pin" ；
如果 session C 执行的是不一样的SQL，那么 session C 不share 同一个child cursor ， session C 会wait for library cache lock.
since 10.2.0.3 "_kks_use_mutex_pin"=TRUE or 11g 开始 使用mutex 保护cursor pin ，所以 session C 若执行 与session B 一样的SQL，那么 wiat for cursor pin S on X
SQL> select * from V$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
session A:
SQL> alter table test add t11 char(2000) default 'maclean';
session B:
SQL> select * from test where rownum=1;
session C:
SQL> select * from test where rownum=1;
SQL> select event from v$session where wait_class='Concurrency';
EVENT
cursor: pin S wait on X
library cache lock
SQL> oradebug setmypid;
Statement processed.
SQL> oradebug dump systemstate 266;
Statement processed.
session C:
SO: 0x9e2256b8, type: 4, owner: 0x9e59a1c0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x9e59a1c0, name=session, file=ksu.h LINE:12624, pg=0
(session) sid: 179 ser: 41307 trans: (nil), creator: 0x9e59a1c0
flags: (0x41) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
flags2: (0x40009) -/-/INC
DID: , short-term DID:
txn branch: (nil)
oct: 3, prv: 0, sql: 0x956e18b8, psql: 0x956e18b8, user: 0/SYS
ksuxds FALSE at location: 0
service name: SYS$USERS
client details:
O/S info: user: oracle, term: pts/3, ospid: 26823
machine: vrh1.oracle.com program: [email protected] (TNS V1-V3)
application name: [email protected] (TNS V1-V3), hash value=1481565533
Current Wait Stack:
0: waiting for 'cursor: pin S wait on X'
idn=0xe76d0d8c, value=0xca00000000, where=0x500000000
wait_id=17 seq_num=18 snap_id=1
wait times: snap=12.671273 sec, exc=12.671273 sec, total=12.671273 sec
wait times: max=infinite, heur=12.671273 sec
wait counts: calls=1148 os=1148
in_wait=1 iflags=0x15b2
There is at least one session blocking this session.
Dumping 1 direct blocker(s):
inst: 1, sid: 202, ser: 15511
Dumping final blocker:
inst: 1, sid: 9, ser: 1
Wait State:
fixed_waits=0 flags=0x22 boundary=(nil)/-1
idn=0xe76d0d8c=> 这个是mutex的标示
KGX Atomic Operation Log 0x94aa8ca8
Mutex 0x8a328978(202, 0) idn e76d0d8c oper GET_SHRD
Cursor Pin uid 179 efd 0 whr 5 slp 1148
opr=2 pso=0x8b5a8c48 flg=0
pcs=0x8a3288e0 nxt=(nil) flg=35 cld=1 hd=0x93d4bbb8 par=0x8a328048
ct=1 hsh=0 unp=(nil) unn=0 hvl=8a328ef0 nhv=1 ses=0x9e1e0ea0
hep=0x8a328978 flg=80 ld=1 ob=0x939a30b0 ptr=0x935e0348 fex=0x935df6f0
这个mutex的 oper是 GET_SHRD 即 pin S 它指向 0x93d4bbb8 是一个child cursor
以下是parent cursor:
SO: 0x957fa9d8, type: 78, owner: 0x9e1e0ea0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x9e593da0, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8547, pg=0
LibraryObjectLock: Address=0x957fa9d8 Handle=0x956e18b8 Mode=N CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0
User=0x9e1e0ea0 Session=0x9e1e0ea0 ReferenceCount=1 Flags=CNB/[0001] SavepointNum=4f5864b8
LibraryHandle: Address=0x956e18b8 Hash=e76d0d8c LockMode=N PinMode=0 LoadLockMode=0 Status=VALD
ObjectName: Name=select * from test where rownum=1
FullHashValue=7e277fabf95d7c80e8924ed6e76d0d8c Namespace=SQL AREA(00) Type=CURSOR(00) Identifier=3882683788 OwnerIdn=0
Statistics: InvalidationCount=1 ExecutionCount=2 LoadCount=3 ActiveLocks=2 TotalLockCount=4 TotalPinCount=1
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=1 BucketInUse=3 HandleInUse=3 HandleReferenceCount=0
Concurrency: DependencyMutex=0x956e1968(0, 2, 0, 0) Mutex=0x956e19e8(0, 45, 0, 0)
Flags=RON/PIN/TIM/PN0/DBN/[10012841]
WaitersLists:
Lock=0x956e1948[0x956e1948,0x956e1948]
Pin=0x956e1928[0x956e1928,0x956e1928]
LoadLock=0x956e19a0[0x956e19a0,0x956e19a0]
Timestamp: Current=03-08-2012 02:45:45
HandleReference: Address=0x956e1a78 Handle=(nil) Flags=[00]
LibraryObject: Address=0x8a327fa8 HeapMask=0000-0001-0001-0000 Flags=EXS[0000] Flags2=[0000] PublicFlags=[0000]
ChildTable: size='16'
Child: id='0' Table=0x8a328e58 Reference=0x8a3288b8 Handle=0x956db988
Child: id='1' Table=0x8a328e58 Reference=0x8a328b80 Handle=0x93d4bbb8
NamespaceDump:
Parent Cursor: sql_id=fj4kfuvmqu3cc parent=0x8a328048 maxchild=2 plk=y ppn=n
但是很可惜 0x93d4bbb8 这个 child cursor 被 session B pin住了：
SO: 0x957fa8d8, type: 78, owner: 0x9e1e0ea0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x9e593da0, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8547, pg=0
LibraryObjectLock: Address=0x957fa8d8 Handle=0x93d4bbb8 Mode=N CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0
ClusterLock=0x8f1945f8 Context=0x7fd379518308 User=0x9e1e0ea0 Session=0x9e1e0ea0 ReferenceCount=1
Flags=CBK/[0020] SavepointNum=0
LibraryHandle: Address=0x93d4bbb8 Hash=0 LockMode=N PinMode=X LoadLockMode=0 Status=VALD
Name: Namespace=SQL AREA(00) Type=CURSOR(00)
Statistics: InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=2 TotalPinCount=3
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0
Concurrency: DependencyMutex=0x93d4bc68(0, 0, 0, 0) Mutex=0x956e19e8(0, 45, 0, 0)
Flags=RON/PIN/PN0/EXP/CHD/[10012111]
WaitersLists:
Lock=0x93d4bc48[0x93d4bc48,0x93d4bc48]
Pin=0x93d4bc28[0x93d4bc28,0x93d4bc28]
LoadLock=0x93d4bca0[0x93d4bca0,0x93d4bca0]
LibraryObject: Address=0x939a30b0 HeapMask=0000-0001-0001-0000 Flags=EXS[0000] Flags2=[0000] PublicFlags=[0000]
DataBlocks:
Block: #='0' name=KGLH0^e76d0d8c pins=0 Change=NONE
Heap=0x93d1a808 Pointer=0x939a3150 Extent=0x939a3030 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=1.546875 Size=4.000000 LoadTime=4385736620
Block: #='6' name=SQLA^e76d0d8c pins=0 Change=NONE
Heap=0x8a328a20 Pointer=0x935e0348 Extent=0x935df6f0 Flags=I/-/P/A/-/E
FreedLocation=0 Alloc=0.000000 Size=0.000000 LoadTime=0
NamespaceDump:
Child Cursor: Heap0=0x939a3150 Heap6=0x935e0348 Heap0 Load Time=03-08-2012 02:50:16 Heap6 Load
PinMode=X
保持这个 X mode pin的是另外一个 mutex ， 这个mutex的 oper是 LONG_EXCL
Time=03-08-2012 02:50:16 ----------------------------------------
KGX Atomic Operation Log 0x8f1945f8
Mutex 0x8a328978(202, 0) idn e76d0d8c oper LONG_EXCL
Cursor Pin uid 202 efd 0 whr 1 slp 0
opr=3 pso=0x957fa8d8 flg=0
pcs=0x8a3288e0 nxt=(nil) flg=35 cld=1 hd=0x93d4bbb8 par=0x8a328048
ct=1 hsh=0 unp=(nil) unn=0 hvl=8a328ef0 nhv=1 ses=0x9e1e0ea0
hep=0x8a328978 flg=80 ld=1 ob=0x939a30b0 ptr=0x935e0348 fex=0x935df6f0 -
Hi there,
My database has a lot of contention in 'library cache lock', more than 90% of all events.
I'm trying to find the query that is waiting for this event, but the sql_id in v$session not exist in v$sql.
SQL> select sql_id, count(*)
2 from gv$session
3 where EVENT ='library cache lock'
4 group by sql_id;
SQL_ID COUNT(*)
4gd6b1r53yt88 49
47
SQL> select *
2 from gv$sql
3 where sql_id = '4gd6b1r53yt88';
no rows selected
The oracle version is 10.2.0.3 RAC.
Can somebody help me?
Thanks,
EversonHi rjaf
Following this post, I got the table SYS.KOTTD$.
I think this is some kind of Oracle internal problem, but I don't know what. -
Library cache lock in statspack report
Hi,
We have some production issues on my database from long back. So i had taken snapshot and applied in one of analyzer tool.
In the TOP 5 Events they mentioned that library cache lock is 94.46 % , i am afraiding by seeing this number. And in there analyzing report they didn't populate the recommendations.
Please suggest me why the library cache lock showing 94.46%.
And i am also adding the other statistics as well.
Please go though and suggest me into right way.
Top 5 Events
Event Percentage of Total Timed Events
library cache lock 94.46%
PX Deq: Table Q Normal 1.51%
db file sequential read 1.23%
direct path read 1.06%
PX Deq Credit: send blkd .55%
Wait Events
Event Waits Wait Time (s) Avg Wait (ms) Waits/txn
library cache lock 45,777 137,044.09 2994 10.6
PX Deq: Table Q Normal 1,263,860 2,196.88 2 292.2
db file sequential read 80,826 1,780.39 22 18.7
direct path read 244,968 1,543.45 6 56.6
PX Deq Credit: send blkd 3,200,312 794.01 0 739.8
PX Deq: Execute Reply 529 570.82 1079 0.1
db file scattered read 42,356 524.47 12 9.8
library cache pin 93 286.44 3080 0.0
PX qref latch 96 90.05 938 0.0
log file sync 4,685 71.9 15 1.1
Instance Activity Stats
Statistic Total per Second per Trans
SQL*Net roundtrips to/from client 277,935 975.2 64.3
consistent gets 8,112,050 28,463.3 1,875.2
db block changes 446,986 1,568.4 103.3
execute count 31,369 110.1 7.3
parse count (hard) 65 0.2 0.0
parse count (total) 30,083 105.6 7.0
physical reads 3,041,562 10,672.2 703.1
physical reads direct 2,624,892 9,210.2 606.8
physical writes 151,245 530.7 35.0
physical writes direct 77,161 270.7 17.8
redo writes 4,933 17.3 1.1
session cursor cache hits 17,868 62.7 4.1
sorts (disk) 2 0.0 0.0
sorts (memory) 2,006 7.0 0.5
table fetch continued row 72,024 252.7 16.7
table scans (long tables) 323 1.1 0.1
table scans (short tables) 3,371 11.8 0.8
8 Recommendations:You need to figure out what session is holding the blocking library cache lock.
Here's a script that will help you do that:
select decode(lob.kglobtyp, 0, 'NEXT OBJECT', 1, 'INDEX', 2, 'TABLE', 3, 'CLUSTER',
4, 'VIEW', 5, 'SYNONYM', 6, 'SEQUENCE',
7, 'PROCEDURE', 8, 'FUNCTION', 9, 'PACKAGE',
11, 'PACKAGE BODY', 12, 'TRIGGER',
13, 'TYPE', 14, 'TYPE BODY',
19, 'TABLE PARTITION', 20, 'INDEX PARTITION', 21, 'LOB',
22, 'LIBRARY', 23, 'DIRECTORY', 24, 'QUEUE',
28, 'JAVA SOURCE', 29, 'JAVA CLASS', 30, 'JAVA RESOURCE',
32, 'INDEXTYPE', 33, 'OPERATOR',
34, 'TABLE SUBPARTITION', 35, 'INDEX SUBPARTITION',
40, 'LOB PARTITION', 41, 'LOB SUBPARTITION',
42, 'MATERIALIZED VIEW',
43, 'DIMENSION',
44, 'CONTEXT', 46, 'RULE SET', 47, 'RESOURCE PLAN',
48, 'CONSUMER GROUP',
51, 'SUBSCRIPTION', 52, 'LOCATION',
55, 'XML SCHEMA', 56, 'JAVA DATA',
57, 'SECURITY PROFILE', 59, 'RULE',
62, 'EVALUATION CONTEXT',
'UNDEFINED') object_type,
lob.KGLNAOBJ object_name,
lk.KGLLKMOD lock_mode_held, lk.KGLLKREQ lock_mode_requested,
ses.sid,
ses.serial#,
ses.username
FROM
x$kgllk lk,
v$session ses,
x$kglob lob,
v$session_wait vsw
WHERE
lk.KGLLKUSE = ses.saddr and
lk.KGLLKHDL = lob.KGLHDADR
and lob.kglhdadr = vsw.p1raw
and vsw.event = 'library cache lock'
order by lock_mode_held desc
-Mark -
Sessions hangs with library cache lock
Der all,
11.1.0.7 rac on solaris 10
Our workflow session yesterday hanged on a particular step the session was waiting on a library cache lock (by using query select event,p1,p2 from v$session where sid=<my_sid>;)
when I checked the blocking session , using the note 122793.1 and http://oracle-study-notes.blogspot.com/2009/05/resolving-library-cache-lock-issue.html and http://oracle-study-notes.blogspot.com/2009/05/find-session-holding-library-cache-lock.html .
I found that
SQL> SELECT SID,USERNAME,TERMINAL,PROGRAM FROM V$SESSION
2 WHERE SADDR in
3 (SELECT KGLLKSES FROM X$KGLLK LOCK_A
4 WHERE KGLLKREQ > 0
5 AND EXISTS (SELECT LOCK_B.KGLLKHDL FROM X$KGLLK LOCK_B
WHERE KGLLKSES = '&SADDR_OF_BLKING_SESS'
6 7 AND LOCK_A.KGLLKHDL = LOCK_B.KGLLKHDL
8 AND KGLLKREQ = 0)
9 );
Enter value for saddr_of_blking_sess: 0000000770E494E0
old 6: WHERE KGLLKSES = '&SADDR_OF_BLKING_SESS'
new 6: WHERE KGLLKSES = '0000000770E494E0'
SID USERNAME TERMINAL PROGRAM
817 SYS UNKNOWN oracle@tabsdb07
(J002)
828 SYS UNKNOWN oracle@tabsdb07
(J001)after killing the session, the library cache locks still remained.when I ran trace on the session
select /*+ all_rows ordered */ A.rowid, :1, :2, :3
from
"DBMRPT"."DBM_BIAUTO_SUSP" A , "DBMRPT"."DBM_CDR_FILE_HEAD" B where(
"A"."CDR_TYPE" is not null and "A"."FILE_ID" is not null) and(
"B"."CDR_TYPE" (+)= "A"."CDR_TYPE" and "B"."FILE_ID" (+)= "A"."FILE_ID")
and( "B"."CDR_TYPE" is null or "B"."FILE_ID" is null)
call count cpu elapsed disk query current rows
Parse 1 0.01 0.01 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
total 2 0.01 0.01 0 0 0 0
Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: SYS (recursive depth: 3)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
row cache lock 5 0.00 0.00
db file sequential read 295932636 0.07 5066.63
gc cr grant 2-way 727813 0.02 233.95
latch: gc element 80 0.00 0.00
latch: gcs resource hash 870 0.00 0.00
latch free 2 0.00 0.00
gc remaster 9 2.00 12.91
gcs drm freeze in enter server mode 9 0.54 2.08
latch: object queue header operation 66 0.00 0.05
latch: cache buffers chains 15 0.03 0.20
resmgr:internal state change 63 0.10 5.30
latch: cache buffers lru chain 1260 0.00 0.01
********************************************************************************Please guide
Kai
all this time sql_id for the session remanined in the sql :
ALTER TABLE DBMRPT.DBM_BIAUTO_SUSP ENABLE CONSTRAINT DBS1_DCFH_FK ..hi..
Go through [http://orainternals.wordpress.com/2009/06/02/library-cache-lock-and-library-cache-pin-waits/]
Anand
Maybe you are looking for
-
How to change data element of IDOC segment
HI Experts, I have copied a segment E1ISU_MEASUREMENT_MEA to ZE1ISU_MEASUREMENT_MEA. Now i want to change data element of one of the field of the segment. Please guide me how i can do it? is there any disadvantages of doing this? can we do it easily?
-
Use Apple TV to back-up iTunes?
I am being forced to reformat my computer due to a virus. Do I need to back-up all my movies and songs on to a hard drive, or will I be able to use Apple TV to send all my media back to my computer once it is reformatted? Currently, my entire library
-
DI API Connection freezes to death
Hi, I deployed a simple app which connect to SAP oCompany.Connect() and it freezes there forever. Is there anything I could do to know what's wrong ? Is there any in depth documentation on what a connection to sap involve in term of security and oth
-
Moving lightroom library onto new PC
Hi there I have an extensive lightroom library on my laptop and have just bought a new PC operating Windows Vista as my laptop was out of hard disk. I have installed Lightroom 1.3 onto the new PC and now want to move my existing library in. I have ba
-
HT5569 cannot connect to home wifi!
i,ve reset my router and my iphone slowly finds the wifi, however when I tap to choose this network it doesnt ask me a password