Understanding DFS lock handle
Hi all,
does anyone can help understanding the DFS lock handle Event ?.
The reference does not consult clear information why this can happen
http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents003.htm#sthref3033
Thanks
*T
Actually it does. It discusses the lock is a global lock, so cluster-wide.
It also discusses communication with the DLM, an acronym which you can look up and which means Distributed Lock Manager.
So no doubt DFS means Distrihbuted File System.
Sybrand Bakker
Senior Oracle DBA
Similar Messages
-
What means DFS lock handle?
It is in TOP 5 Wait Events of Oracle 10G database running on Windows.
ThanksWhy are you asking a basic docs question here?
I went to tahiti.oracle.com and put in your subject and it immediately
returned this link:
http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents003.htm#sthref4395
Please check the docs before asking for help. -
RAC processes (10.1.0.3) spinning forever on a DFS lock handle. Any SELECT on a specific table runs into this wait state and waits forever (until killed).
Potentially bug 3520103, but the causes described in Metalink Note 3520103.8 is not applicable in our case.
Metalink provides very little usable data about this bug. It would seem that we have now run into this problem twice in so many months on 10.1.0.3. Upgrade to 10.2 is planned but a few month away still.
The suggest workaround in Note 3520103.8 says "Restart one instance". This fails to work and does not make much sense to me either. Surely the ASM instance owning the unreleased DFS lock should be restarted and not just simply any (random) instance in the cluster?
Is it perhaps possible to identify these locks using SQL against an ASM instance?
Current workarounds I have are
- restarting the entire cluster (a very large hammer approach I don't like)
- dropping and re-creating the table in question (not always feasible and does it cause the dfs lock to be released, or does it instead merely hides the problem?)
If anyone has any details to share about this DFS lock handle problem, potential causes (other than what Metalink thinks), alternative workarounds, etc, I will appreciate it.
Thanks.Hi Bkboudreau,
If I am not misunderstand your question, you are finding the method for iSCSI target servers synchronize iSCSI data.
Replication of data between sites is very important in a multi-site cluster, and is accomplished in different ways by different hardware vendors. You can choose the hardware
storage vendor solution when you want to synchronize data between two shared storage.
You cannot use the feature in Windows Server 2008 called Distributed File System Replication (DFS-R) as your data replication method in a multi-site cluster. DFS-R only performs
its data replication after a file is closed. This works well for files such as documents, presentations, or spreadsheets, but it will not work for files that are held open, such as databases or virtual machines. You must choose a replication option other than
DFS-R.
The related KB:
Requirements and Recommendations for a Multi-Site Failover Cluster
http://technet.microsoft.com/zh-cn/library/dd197575(v=ws.10).aspx
The similar thread:
how to sync the data between the two iSCSI target server
http://social.technet.microsoft.com/Forums/windowsserver/en-US/067ba543-369c-4d22-9f00-1f41d7aedc00/how-to-sync-the-data-between-the-two-iscsi-target-server?forum=winserverClustering
Hope this helps.
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place. -
Livecache lock handler not available in APO
Hello Experts,
We are getting the error "Livecache lock handler not available" error when we are running planning jobs in SAP APO.
We are using parallel processing profiles for DP background processing jobs (Macro calculations) and I am getting this error only when a job is run with PP profile.
Below are the parameters of the parallel processing profiles:
Application: DP background processing
No of parallel processes: 2
Block Size: 500
The error message is written in the spool generated for the background job and the job fails. Eventually the process chain fails in which the job is included.
Please provide some pointers if anyone has come across the same error.
Regards,
RaghuHi JB,
Thanks for the note, Basis check the solution in the note and system seems to have met the requirements.
But one thing is likely to increase the MaxSQLLocks in the system as the demand of the locks is more than the availability. So basis is looking at this dimension to increase the number of locks in the system.
Will update soon in case of any conclusion.
Regards,
Raghu -
Live cache lock handler not available
Hi we are frequently getting this error Live cache lock handler not available and this causes our process chains to fail.
Can anyone please explain in details what this error refers to and direct to any good links for error resolution.Hello,
1. As SAP customer, you could create the SAP message to the component BC-DB-LCA and get SAP support.
2. If you set the flag 'activate livecache lock' in tcode /SAPAPO/MSDP_ADMIN for your planning area on SCM system, please run the connection test for the LEA connection in /ndb59 to check the LEA connection settings.
The liveCache installation guide has additional information about LEA connection.
Regards, Natalia Khlopina -
Can't understand how Finder handles photos!
I would like to understand how finder handles photos because I switched from windows recently and can't understand some things.
First example, renaming lots of photos (without using iPhoto). I already have a automator action for renaming files, but if I have lots of photos with the numbers wrong, I wanted to organize the photos by date created and then rename them and get them in order. The problem is finder renames the files but with some order he chooses, and not the one I wanted. I know this happens because I select all the photos and he decides this way, but how can I do this my way!?
The second question is kinda related. If I open lots of photos in finder (using command+a and command+o), why do the photos that are open in preview are opened in an order finder chooses, even if I arrange my photos in different orders (kind, name, date, etc)?
I'm finding handling files in finder a little too much work to do, it was supposed to be a simple thing, just renaming or opening photos, because despite all his many weaknesses, windows worked kinda well with files, like pictures...Part of it depends on what criterion you have chosen to arrange the files in the Finder window: Name, Date Modified, Date Created, Size, Kind, Label. If by Name, I think the ones with numbers are ordered before letters.
-
Livecache lock handler not available
Hello Experts,
We are getting the error "Livecache lock handler not available" error when we are running planning jobs in SAP APO.
We are using parallel processing profiles for DP background processing jobs (Macro calculations) and I am getting this error only when a job is run with PP profile.
Below are the parameters of the parallel processing profiles:
Application: DP background processing
No of parallel processes: 2
Block Size: 500
The error message is written in the spool generated for the background job and the job fails. Eventually the process chain fails in which the job is included.
Please provide some pointers if anyone has come across the same error.
Regards,
RaghuHi JB,
Thanks for the note, Basis check the solution in the note and system seems to have met the requirements.
But one thing is likely to increase the MaxSQLLocks in the system as the demand of the locks is more than the availability. So basis is looking at this dimension to increase the number of locks in the system.
Will update soon in case of any conclusion.
Regards,
Raghu -
Hi, I have a big problem... after setting up SSL (only some changes in strust and adding parameter in rz10) I cannot use Sap Logon anymore.. Log in is ok, but when I try to run any transaction I see only error:
'Unknown error calling profile editor'
and in syslog after restart I have a lot of errors like that:
'SAP-Basis System: Error in Central Lock Handler'
please help.after system restart I see this when calling any transaction:
http://img441.imageshack.us/img441/5152/errorso.jpg
ok, I found backup files of profile in sap directory and replace it with actual. After restart of whole machine it started working.
Edited by: pw.skiba on Jun 17, 2010 1:35 PM -
Understanding how Error handling works in transformation
Hi Experts,
I am trying to achieve error handling in Start and End Routine by writing the error log to the monitor and also raising the exception in case a match is not found.
I understand the code below achieves what I am looking for, but I want to know how this works
if sy-subrc ne 0.
monitor_rec-msgid = 'BI'.
monitor_rec-msgty = 'E'.
monitor_rec-msgno = '001'.
monitor_rec-msgv1 = 'No entry for ZTYPE in source '.
monitor_rec-msgv2 = .
append monitor_rec to MONITOR.
raise exception type CX_RSROUT_ABORT.
endif.
Can someone please explain how the above code works?
I am not sure how these messages get written to the monitor and how to check these error records. Also if a load is scheduled in the process chains, does the DTP error log show the exception CX_RSROUT_ABORT?proper exception handling is one of the most important thing to consider in your pl/sql code.
One basic principle is to trap/handle only those exceptions that you expect. In general this means you should NOT use the when other execption (only at the outermost call structure)
I guess this is oracle FORMS pl/sql, so we have to be careful with some assupmtions.
But this should work. See also how indenting the ode helps to improve readability. Here in the forum you need to use {<b></b>code} tages before and after your code to preserve indention and to enable basic highlighting
untested
/* fetch the primary account */
begin
select customer_code into cust from primary_account where pc.customer_code=:import_can.consignee_code;
exception
when no_data_found then
j:='Customer ('||:import_can.consignee_code||') not found. Account is not created for this consignee ';
raise_application_error(-20001,j); /* might be raise form_trigger_error here or something like that */
when too_many_rows then
j:='Customer ('||:import_can.consignee_code||') has several Accounts!';
raise_application_error(-20001,j);
end;
begin
select customer_code into cust from cust_details where customer_code=:import_can.consignee_code;
exception
when no_data_found then
h:='consignee_code is not exist in customer code';
raise_application_error(-20001,h);
end;
COMMIT_FORM;
:GLOBAL.ADDCAN:=0;
SA:=SHOW_ALERT('SAVE');
ENAB; -
Understanding XML file handling in ODI
Hi People,
I've just completed a simple Interface to upload an XML file into an Oracle 10g database table. The interface copes well with both inserts and updates, so I'm very pleased about that. ;)
This exercise has raised some questions though. I tried an experiment, which involved changing an element value in the XML file. I wanted to see if this change would be reflected in the file's associated Source Table in ODI. Unfortunately, it wasn't; despite me clicking the REFRESH button about 4 times. I guess my first question is:
*1. How can I ensure a change to an element value in the XML file is reflected in ODI, after reverse engineering the associated data store from the file? Is this possible?*
Moving on, I'm trying to understand the architecture ODI creates in the background, in order to support a source datasource/table that represents the XML file. I have created the following Topology to support the file:
!http://img2.imageshack.us/img2/6895/odiscr375.jpg!
As you can see, I've associated the XML file with a physical schema called ODI_XML. This represents a schema owned by my Oracle Database.
Now, when I check the XML file data in ODI Designer, I notice that it's based on this query:
select * from ODI_XML.TRADE
I've checked the ODI_XML database schema, and there is neither a table nor any other objects named TRADE. My next question is therefore:
*2. Was I correct to assign a database schema (named ODI_XML) to my XML Data Server, or should this have been something else? If so, what?*
Thanks in advance for any assistance.
JamesSince you are using the Variable ,when you right click and view datastore its missing the required path of the file, as value of the variable is not determined , to do so you need to either refresh or set the variable and call the interface or other ODI objects and in loop keep reading XML files one by one and that should work.
-
Hi
I am running a little test that runs two threads that update the same table. Each thread tries to update several documents in the within a single transaction. The update is done by retrieving the document, modifying it, adding the updated document and deleting the existing document.
I have enable dead lock detection and when a dead lock exception is thrown I abort the current transaction.
However after a few iteration, deleteDocument causes a core dump and DBXML prints to stderr : "Previous deadlock return not resolved".
Is there anything else to resolve when a dead lock occurs other than abort the transaction?
This is the relevant stack trace:
#0 0x00421780 in Dbc::get () from /u/yoava/dbxml-2.2.13/install/lib/libdb_cxx-4.3.so
(gdb) where
#0 0x00421780 in Dbc::get () from /u/yoava/dbxml-2.2.13/install/lib/libdb_cxx-4.3.so
#1 0x00934289 in DbXml::SyntaxDatabase::updateStatistics (this=0x2000001c,
context=@0x8641c34, key=@0x8628070, statistics=@0x8628088) at Cursor.hpp:48
#2 0x00901d10 in DbXml::StatisticsWriteCache::updateContainer (this=0x8641bcc,
context=@0x8641c34, container=Internal: global symbol `Container' found in Container.cpp psymtab but not in symtab.
Container may be an inlined function, or may be a template function
(if a template, try specifying an instantiation: Container<type>).
) at /usr/include/c++/3.2.3/bits/stl_tree.h:202
#3 0x0093bacf in DbXml::KeyStash::updateIndex (this=0x8634df8, context=@0x8641c34,
container=0x8630e68) at KeyStash.cpp:210
#4 0x008c72b7 in DbXml::Container::deleteDocument (this=0x8630e68, txn=0x86417f8,
document=@0x8671f38, context=Internal: global symbol `UpdateContext' found in UpdateContext.cpp psymtab but not in symtab.
UpdateContext may be an inlined function, or may be a template function
(if a template, try specifying an instantiation: UpdateContext<type>).
) at Container.cpp:679
#5 0x008d3475 in DeleteDocumentFunctor2::method (this=0x2000001c, container=@0x8630e68,
txn=Internal: global symbol `Transaction' found in Transaction.cpp psymtab but not in symtab.
Transaction may be an inlined function, or may be a template function
(if a template, try specifying an instantiation: Transaction<type>).
) at TransactedContainer.cpp:121
#6 0x008d3149 in DbXml::TransactedContainer::transactedMethod (this=0x8630e68,
txn=0x3d71c8, flags=0, f=@0xb176b470) at TransactedContainer.cpp:217
#7 0x008d2fe8 in DbXml::TransactedContainer::deleteDocument (this=0x8630e68,
txn=0x86417f8, document=Internal: global symbol `Document' found in Document.cpp psymtab but not in symtab.
Document may be an inlined function, or may be a template function
(if a template, try specifying an instantiation: Document<type>).
) at TransactedContainer.cpp:26
#8 0x00906940 in DbXml::XmlContainer::deleteDocument (this=0xbfff8c94, txn=@0xb176b5a0,
document=Internal: global symbol `XmlDocument' found in XmlDocument.cpp psymtab but not in symtab.
XmlDocument may be an inlined function, or may be a template function
(if a template, try specifying an instantiation: XmlDocument<type>).
) at /u/yoava/dbxml-2.2.13/dbxml/include/dbxml/XmlDocument.hpp:72
#9 0x0804a5c0 in DoUpdates (arg=0xbfff8c90) at dbxml_test_6.cpp:99
#10 0x003dedec in start_thread () from /lib/tls/libpthread.so.0
#11 0x0037ea2a in clone () from /lib/tls/libc.so.6
thanksI have applied them. When applying patch 6 I got an error:
compile3.server 46% patch < patch.2.2.13.6
patch.2.2.13.6: No such file or directory.
compile3.server 47% patch < patch.2.2.13.6
(Stripping trailing CRs from patch.)
can't find file to patch at input line 3
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
|*** NsEventGenerator.cpp.orig Thu Dec 8 14:50:50 2005
|--- dbxml/src/dbxml/nodeStore/NsEventGenerator.cppThu Sep 28 17:24:57 2006
File to patch: dbxml/src/dbxml/DbWrapper.hpp
patching file dbxml/src/dbxml/DbWrapper.hpp
Hunk #1 FAILED at 357.
1 out of 1 hunk FAILED -- saving rejects to file dbxml/src/dbxml/DbWrapper.hpp.rej
However this patch does not seem to be lock related. -
Understanding on Exception handling???
Hi all
I am doing some bug correction on exception and encounter some problem and questions. Hope someone can help clear my doubts pls. Thanks.
Q1. I have an exception handling in method A whereby it throw an exception of type ERROR class. I try to catch this ERROR in method B with MyException class which extends from EXCEPTION class. I realise this cannot be done. When I change MyException class to Throwable class, I can catch ERROR thrown. Is it correct that I can only catch exception using superclass? Coz in this case MyException is not a superclass of ERROR class?
public class MyClass() throws MyException
void method B()
try{
method A();
}catch(MyException e){
system.println.out();
void method A()
try{
}catch(IndexOutOfBoundsException e){
throw new Error("string text');
Q2. Is it true that I cannot put methodA in MyClass because it throws ERROR class but MyClass throws only MyException?Uh.. I think what you just said is called a
bug, which need to be caught.No ... bugs can cause exceptions, but there are several occasions when a exception can occur that have nothing to do with bugs.
If for example the hard disk is full when you want to save your file that can hardly be called a bug, but rather an unlucky occurance: an exception to the rule that normally there is enough space left on the drive.
Also: if somewhere a router shuts down, because it was hit by a lightning an consequently your connection is interrupted then that's not a bug (at least not in your program), but an exception that you must handle.
But as I said: bugs can cause exceptions: If you allocate a buffer of 1024 bytes and try to write to its 1025th byte then an ArrayIndexOutOfBoundsException is thrown. In this case you generally don't want to catch the exception, but rather prevent it from beeing thrown. -
ROW CACHE ENQUEUE LOCK/ibrary cache load lock leads to database hung
(lowercase, curly brackets, no spaces)
We faced database hung on 3 node 11i erp 9i rac database.
We saw the library cache load lock timed out events reported in alert log.
Then few ora-600 and later ROW CACHE ENQUEUE LOCK timed out event. Eventually database was hung and we had to bounce the services .
we created support sr 7845542.992 for RCA.
The support says to increase shared pool size to avoid shared pool fragmentation and avoid reload ,additionaly to upgrade to 10g database.
I am not covinced adding additional pool size would solve this or upgrade to 10 .furthermore even 10g has such issues reported.
I saw couple of bugs mentioned such issue can happen due deadlock of session holding latches .
kindly let me know your view on issue
If required i can attach statspack for more information. (lowercase, curly brackets, no spaces)Many Thanks, i was keen to have your update .
There are 8 cpus on each node . Reloads very high during time period ,but normally there are not high reloads.
Statspack details for 3 nodes
STATSPACK report for
DB Name DB Id Instance Inst Num Release Cluster Host
PROD 21184234 PROD1 1 9.2.0.8.0 YES npi-or-db-p-
11.npi.corp
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 149817 30-Oct-09 13:00:09 574 #########
End Snap: 149837 30-Oct-09 14:00:17 602 #########
Elapsed: 60.13 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 8,192M Std Block Size: 8K
Shared Pool Size: 1,024M Log Buffer: 10,240K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 122,414.93 11,449.13
Logical reads: 69,550.76 6,504.89
Block changes: 928.41 86.83
Physical reads: 196.24 18.35
Physical writes: 28.65 2.68
User calls: 343.97 32.17
Parses: 558.61 52.25
Hard parses: 43.48 4.07
Sorts: 467.24 43.70
Logons: 0.63 0.06
Executes: 2,046.99 191.45
Transactions: 10.69
% Blocks changed per Read: 1.33 Recursive Call %: 97.59
Rollback per transaction %: 5.07 Rows per Sort: 15.85
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 99.72 In-memory Sort %: 100.00
Library Hit %: 96.79 Soft Parse %: 92.22
Execute to Parse %: 72.71 Latch Hit %: 99.77
Parse CPU to Parse Elapsd %: 60.10 % Non-Parse CPU: 78.07
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
db file sequential read 249,234 0 1,537 6 6.5
db file scattered read 61,776 0 769 12 1.6
row cache lock 780,098 10 566 1 20.2
library cache lock 697,849 157 432 1 18.1
latch free 127,926 4,715 387 3 3.3
global cache cr request 370,770 3,091 309 1 9.6
PL/SQL lock timer 59 58 112 1903 0.0
wait for scn from all nodes 303,572 18 103 0 7.9
library cache pin 26,231 2 100 4 0.7
global cache null to x 17,717 716 92 5 0.5
buffer busy waits 5,388 18 74 14 0.1
db file parallel read 5,245 0 69 13 0.1
log file sync 20,407 29 66 3 0.5
enqueue 52,200 70 60 1 1.4
buffer busy global CR 4,845 33 55 11 0.1
CGS wait for IPC msg 412,512 407,106 50 0 10.7
ksxr poll remote instances 1,279,565 483,046 48 0 33.2
log file parallel write 160,040 0 42 0 4.1
library cache load lock 1,491 2 29 20 0.0
global cache open x 19,507 344 28 1 0.5
buffer busy global cache 957 0 22 23 0.0
global cache s to x 16,516 180 20 1 0.4
db file parallel write 11,120 0 12 1 0.3
log file sequential read 618 0 11 18 0.0
DFS lock handle 23,768 0 10 0 0.6
control file sequential read 8,563 0 4 0 0.2
KJC: Wait for msg sends to c 1,549 57 4 3 0.0
lock escalate retry 76 76 4 52 0.0
SQL*Net break/reset to clien 12,546 0 3 0 0.3
SQL*Net more data to client 85,773 0 3 0 2.2
control file parallel write 1,265 0 2 1 0.0
global cache null to s 648 23 1 2 0.0
global cache busy 200 0 1 5 0.0
global cache open s 1,493 28 1 1 0.0
log file switch completion 12 0 1 61 0.0
PX Deq Credit: send blkd 161 70 1 4 0.0
kksfbc child completion 119 118 1 5 0.0
PX Deq: reap credit 5,948 5,456 0 0 0.2
PX Deq: Execute Reply 83 29 0 3 0.0
process startup 8 0 0 25 0.0
LGWR wait for redo copy 992 12 0 0 0.0
IPC send completion sync 450 450 0 0 0.0
PX Deq: Parse Reply 100 28 0 1 0.0
undo segment extension 10,380 10,372 0 0 0.3
PX Deq: Join ACK 146 65 0 1 0.0
buffer deadlock 222 221 0 0 0.0
async disk IO 1,179 0 0 0 0.0
wait list latch free 2 0 0 16 0.0
PX Deq: Msg Fragment 112 28 0 0 0.0
Library Cache Activity for DB: PROD Instance: PROD1 Snaps: 149817 -149837
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 116,007 1.1 133,347 19.9 24,338 0
CLUSTER 4,224 0.6 5,131 1.0 0 0
INDEX 15,048 24.1 13,798 26.4 2 0
JAVA DATA 82 0.0 692 39.6 136 0
JAVA RESOURCE 66 39.4 206 25.2 12 0
PIPE 1,140 0.5 1,160 0.5 0 0
SQL AREA 1,197,908 12.6 13,517,660 1.5 111,833 73
TABLE/PROCEDURE 3,847,439 0.8 4,230,265 7.9 142,200 0
TRIGGER 8,444 2.4 8,657 18.5 1,274 0
GES Lock GES Pin GES Pin GES Inval GES Invali-
Namespace Requests Requests Releases Requests dations
BODY 1 1,234 1,258 985 0
CLUSTER 3,222 25 25 25 0
INDEX 13,792 3,641 3,631 3,629 0
JAVA DATA 0 0 0 0 0
JAVA RESOURCE 0 26 25 0 0
PIPE 0 0 0 0 0
SQL AREA 0 0 0 0 0
TABLE/PROCEDURE 857,137 13,130 13,264 10,762 0
TRIGGER 0 200 202 200 0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
STATSPACK report for
DB Name DB Id Instance Inst Num Release Cluster Host
PROD 21184234 PROD2 2 9.2.0.8.0 YES npi-or-db-p-
12.npi.corp
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 149847 30-Oct-09 14:00:05 493 #########
End Snap: 149857 30-Oct-09 15:00:02 432 #########
Elapsed: 59.95 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 8,192M Std Block Size: 8K
Shared Pool Size: 1,024M Log Buffer: 10,240K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 71,853.44 32,058.65
Logical reads: 273,904.84 122,207.36
Block changes: 889.13 396.70
Physical reads: 40.40 18.03
Physical writes: 20.97 9.35
User calls: 153.74 68.60
Parses: 66.19 29.53
Hard parses: 2.66 1.19
Sorts: 25.70 11.47
Logons: 0.16 0.07
Executes: 726.41 324.10
Transactions: 2.24
% Blocks changed per Read: 0.32 Recursive Call %: 92.41
Rollback per transaction %: 4.84 Rows per Sort: 193.55
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 99.99
Buffer Hit %: 99.99 In-memory Sort %: 100.00
Library Hit %: 99.35 Soft Parse %: 95.97
Execute to Parse %: 90.89 Latch Hit %: 99.99
Parse CPU to Parse Elapsd %: 36.55 % Non-Parse CPU: 98.28
Wait Events for DB: PROD Instance: PROD2 Snaps: 149847 -149857
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
enqueue 65,823 33,667 90,459 1374 8.2
row cache lock 38,996 560 1,795 46 4.8
PX Deq Credit: send blkd 522 499 1,223 2344 0.1
PX Deq: Parse Reply 466 416 987 2117 0.1
db file sequential read 50,130 0 421 8 6.2
library cache lock 78,842 172 210 3 9.8
db file scattered read 6,904 0 152 22 0.9
global cache cr request 84,801 575 113 1 10.5
latch free 8,096 736 65 8 1.0
log file sync 5,676 27 41 7 0.7
wait for scn from all nodes 18,891 10 24 1 2.3
CGS wait for IPC msg 394,678 392,142 21 0 49.0
library cache pin 1,339 0 17 13 0.2
global cache null to x 2,145 48 16 8 0.3
global cache s to x 3,242 32 16 5 0.4
buffer busy waits 366 10 15 40 0.0
ksxr poll remote instances 70,990 31,295 14 0 8.8
db file parallel read 359 0 11 31 0.0
global cache open x 2,708 55 10 4 0.3
async disk IO 3,474 0 8 2 0.4
global cache open s 3,470 10 6 2 0.4
log file parallel write 13,076 0 5 0 1.6
global cache busy 58 40 5 90 0.0
PL/SQL lock timer 1 1 5 4877 0.0
DFS lock handle 3,362 0 5 1 0.4
log file sequential read 412 0 4 10 0.1
db file parallel write 2,774 0 3 1 0.3
library cache load lock 59 0 3 58 0.0
buffer busy global CR 722 0 3 4 0.1
control file sequential read 6,398 0 3 0 0.8
SQL*Net break/reset to clien 16,078 0 2 0 2.0
name-service call wait 26 0 2 67 0.0
control file parallel write 1,248 0 2 1 0.2
process startup 24 0 1 49 0.0
KJC: Wait for msg sends to c 3,491 4 1 0 0.4
SQL*Net more data to client 23,724 0 1 0 2.9
buffer busy global cache 23 0 0 19 0.0
global cache null to s 114 0 0 4 0.0
PX Deq: reap credit 5,646 5,509 0 0 0.7
log file switch completion 4 0 0 58 0.0
lock escalate retry 54 54 0 1 0.0
IPC send completion sync 119 118 0 0 0.0
direct path read 2,820 0 0 0 0.3
direct path read (lob) 3,632 0 0 0 0.5
PX Deq: Join ACK 88 37 0 0 0.0
direct path write 2,470 0 0 0 0.3
kksfbc child completion 6 6 0 6 0.0
buffer deadlock 3 3 0 11 0.0
global cache quiesce wait 4 4 0 8 0.0
Library Cache Activity for DB: PROD Instance: PROD2 Snaps: 149847 -149857
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 27,353 0.5 28,091 6.5 1,643 0
CLUSTER 203 1.0 269 1.5 0 0
INDEX 526 9.9 271 19.9 0 0
JAVA DATA 18 0.0 120 6.7 4 0
JAVA RESOURCE 20 45.0 56 26.8 3 0
JAVA SOURCE 1 100.0 1 100.0 0 0
PIPE 999 0.4 1,043 0.4 0 0
SQL AREA 131,793 7.6 3,406,577 0.4 7,012 0
TABLE/PROCEDURE 926,987 0.2 1,907,993 1.0 8,845 0
TRIGGER 1,519 0.1 1,532 4.9 69 0
GES Lock GES Pin GES Pin GES Inval GES Invali-
Namespace Requests Requests Releases Requests dations
BODY 1 129 277 117 0
CLUSTER 168 2 2 2 0
INDEX 271 52 56 52 0
JAVA DATA 0 0 0 0 0
JAVA RESOURCE 0 9 6 0 0
JAVA SOURCE 0 1 1 1 0
PIPE 0 0 0 0 0
SQL AREA 0 0 0 0 0
TABLE/PROCEDURE 89,523 764 868 460 0
TRIGGER 0 2 14 2 0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
DB Name DB Id Instance Inst Num Release Cluster Host
PROD 21184234 PROD3 3 9.2.0.8.0 YES npi-or-db-p-
13.npi.corp
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 149808 30-Oct-09 14:00:00 31 #########
End Snap: 149809 30-Oct-09 15:00:02 34 11,831.4
Elapsed: 60.03 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 8,192M Std Block Size: 8K
Shared Pool Size: 1,024M Log Buffer: 10,240K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 1,518.14 36,700.35
Logical reads: 1,333.43 32,235.02
Block changes: 5.09 123.01
Physical reads: 54.31 1,312.88
Physical writes: 3.91 94.44
User calls: 1.46 35.40
Parses: 2.24 54.21
Hard parses: 0.04 0.93
Sorts: 0.84 20.28
Logons: 0.06 1.45
Executes: 3.11 75.23
Transactions: 0.04
% Blocks changed per Read: 0.38 Recursive Call %: 94.31
Rollback per transaction %: 45.64 Rows per Sort: 215.97
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.99 Redo NoWait %: 100.00
Buffer Hit %: 96.21 In-memory Sort %: 100.00
Library Hit %: 99.07 Soft Parse %: 98.29
Execute to Parse %: 27.94 Latch Hit %: 99.98
Parse CPU to Parse Elapsd %: 69.88 % Non-Parse CPU: 97.92
Wait Events for DB: PROD Instance: PROD3 Snaps: 149808 -149809
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
enqueue 19,510 7,472 15,509 795 130.9
PX Deq: Parse Reply 1,152 1,071 2,577 2237 7.7
row cache lock 2,202 518 1,579 717 14.8
db file scattered read 31,556 0 354 11 211.8
db file sequential read 17,272 0 67 4 115.9
db file parallel read 1,722 0 34 20 11.6
global cache cr request 53,754 91 32 1 360.8
wait for scn from all nodes 1,897 13 10 5 12.7
CGS wait for IPC msg 403,358 401,478 10 0 2,707.1
DFS lock handle 4,753 0 8 2 31.9
direct path read 1,248 0 6 5 8.4
PX Deq: Execute Reply 110 38 6 51 0.7
global cache open s 160 10 5 31 1.1
control file sequential read 6,442 0 3 0 43.2
name-service call wait 26 0 2 78 0.2
latch free 129 109 2 13 0.9
KJC: Wait for msg sends to c 153 24 1 9 1.0
control file parallel write 1,245 0 1 1 8.4
buffer busy waits 199 0 1 6 1.3
process startup 20 0 1 44 0.1
global cache null to x 74 2 1 9 0.5
global cache null to s 19 0 1 29 0.1
global cache open x 268 1 1 2 1.8
library cache lock 1,150 0 0 0 7.7
PX Deq: Join ACK 129 48 0 3 0.9
log file parallel write 1,157 0 0 0 7.8
async disk IO 219 0 0 1 1.5
direct path write 1,024 0 0 0 6.9
ksxr poll remote instances 6,740 4,595 0 0 45.2
PX Deq: reap credit 6,580 6,511 0 0 44.2
buffer busy global CR 73 0 0 2 0.5
log file sequential read 11 0 0 10 0.1
log file sync 100 0 0 1 0.7
global cache s to x 282 2 0 0 1.9
db file parallel write 95 0 0 1 0.6
library cache pin 142 0 0 0 1.0
SQL*Net break/reset to clien 28 0 0 1 0.2
IPC send completion sync 81 81 0 0 0.5
PX Deq: Signal ACK 32 14 0 1 0.2
PX Deq Credit: send blkd 3 1 0 7 0.0
SQL*Net more data to client 841 0 0 0 5.6
PX Deq: Msg Fragment 37 17 0 0 0.2
log file single write 4 0 0 1 0.0
db file single write 1 0 0 1 0.0
SQL*Net message from client 4,213 0 13,673 3246 28.3
gcs remote message 214,784 75,745 7,016 33 1,441.5
wakeup time manager 233 233 6,812 29237 1.6
PX Idle Wait 2,338 2,294 5,686 2432 15.7
PX Deq: Execution Msg 2,151 1,979 4,796 2229 14.4
Library Cache Activity for DB: PROD Instance: PROD3 Snaps: 149808 -149809
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 1,290 0.0 1,290 0.0 0 0
CLUSTER 18 0.0 8 0.0 0 0
SQL AREA 4,893 2.0 36,371 0.5 2 0
TABLE/PROCEDURE 1,555 3.9 3,834 4.9 71 0
TRIGGER 286 0.0 286 0.0 0 0
GES Lock GES Pin GES Pin GES Inval GES Invali-
Namespace Requests Requests Releases Requests dations
BODY 1 0 0 0 0
CLUSTER 4 0 0 0 0
SQL AREA 0 0 0 0 0
TABLE/PROCEDURE 863 224 42 42 0
TRIGGER 0 0 0 0 0
------------------------------------------------------------- -
Automatic Parallelism causes Merge statement to take longer.
We have a problem in a new project as part of the ETL load into the Oracle datawarehouse we perform a merge statement to update rows in a global temporary table then load
the results into a permanant table, when testing with automatic parallel execution enabled the plan changes and the merge never finishes and consumes vast amounts of resources.
The database version is:-
Database version :11.2.0.3
OS: redhat 64bit
three node rac 20 cores per node
when executing serially the query response is typically similar to the following:
MERGE /*+ gather_plan_statistics no_parallel */ INTO T_GTTCHARGEVALUES USING
(SELECT
CASTACCOUNTID,
CHARGESCHEME,
MAX(CUMULATIVEVALUE) AS MAXMONTHVALUE,
MAX(CUMULATIVECOUNT) AS MAXMONTHCOUNT
FROM
V_CACHARGESALL
WHERE
CHARGEDATE >= TRUNC(TO_DATE(:B1,'YYYY-MM-DD'),'MM')
AND CHARGEDATE < TO_DATE(:B1,'YYYY-MM-DD')
GROUP BY
CASTACCOUNTID,
CHARGESCHEME
HAVING MAX(CUMULATIVECOUNT) IS NOT NULL ) MTOTAL
ON
(T_GTTCHARGEVALUES.CASTACCOUNTID=MTOTAL.CASTACCOUNTID AND
T_GTTCHARGEVALUES.CHARGESCHEME=MTOTAL.CHARGESCHEME)
WHEN MATCHED
THEN UPDATE SET
CUMULATIVEVALUE=CUMULATIVEVALUE+MTOTAL.MAXMONTHVALUE ,
CUMULATIVECOUNT=CUMULATIVECOUNT+MTOTAL.MAXMONTHCOUNT;
1448340 rows merged.
select * from table(dbms_xplan.display_cursor(null,null,'ALLSTATS LAST'));
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | Used-Mem |
| 0 | MERGE STATEMENT | | 1 | | 0 |00:03:08.43 | 2095K| 186K| | | |
| 1 | MERGE | T_GTTCHARGEVALUES | 1 | | 0 |00:03:08.43 | 2095K| 186K| | | |
| 2 | VIEW | | 1 | | 1448K|00:02:53.14 | 619K| 177K| | | |
|* 3 | HASH JOIN | | 1 | 1 | 1448K|00:02:52.70 | 619K| 177K| 812K| 812K| 1218K (0)|
| 4 | VIEW | | 1 | 1 | 203 |00:02:51.26 | 608K| 177K| | | |
|* 5 | FILTER | | 1 | | 203 |00:02:51.26 | 608K| 177K| | | |
| 6 | SORT GROUP BY | | 1 | 1 | 480 |00:02:51.26 | 608K| 177K| 73728 | 73728 | |
|* 7 | FILTER | | 1 | | 21M|00:02:56.04 | 608K| 177K| | | |
| 8 | PARTITION RANGE ITERATOR| | 1 | 392K| 21M|00:02:51.32 | 608K| 177K| | | |
|* 9 | TABLE ACCESS FULL | T_CACHARGES | 24 | 392K| 21M|00:02:47.48 | 608K| 177K| | | |
| 10 | TABLE ACCESS FULL | T_GTTCHARGEVALUES | 1 | 1451K| 1451K|00:00:00.48 | 10980 | 0 | | | |
Predicate Information (identified by operation id):
3 - access("T_GTTCHARGEVALUES"."CASTACCOUNTID"="MTOTAL"."CASTACCOUNTID" AND "T_GTTCHARGEVALUES"."CHARGESCHEME"="MTOTAL"."CHARGESCHEME")
5 - filter(MAX("CUMULATIVECOUNT") IS NOT NULL)
7 - filter(TRUNC(TO_DATE(:B1,'YYYY-MM-DD'),'fmmm')<TO_DATE(:B1,'YYYY-MM-DD'))
9 - filter(("LOGICALLYDELETED"=0 AND "CHARGEDATE">=TRUNC(TO_DATE(:B1,'YYYY-MM-DD'),'fmmm') AND "CHARGEDATE"<TO_DATE(:B1,'YYYY-MM-DD')))removing the no_parallel hint results in the following, (this is pulled from the sql monitoring report and editied to remove the lines relating to individual parallel servers)
I understand that the query is considered for parallel execution due to the estimated length of time it will run for and although the degree of parallleism seems excessive
it is the default maximum for the server configuration, what we are tryig to understand is which statistics could be inacurate or missing and could cause this kind of problem.
In this case we can add the no_parallel hint in the etl package as a workaround but would really liek to identify the root cause to avoid similar problems elsewhere.
SQL Monitoring Report
SQL Text
MERGE INTO T_GTTCHARGEVALUES USING (SELECT CASTACCOUNTID, CHARGESCHEME, MAX(CUMULATIVEVALUE) AS MAXMONTHVALUE,
MAX(CUMULATIVECOUNT) AS MAXMONTHCOUNT FROM V_CACHARGESALL WHERE CHARGEDATE >= TRUNC(TO_DATE(:B1,'YYYY-MM-DD'),'MM')
AND CHARGEDATE < to_date(:B1,'YYYY-MM-DD')
GROUP BY CASTACCOUNTID, CHARGESCHEME HAVING MAX(CUMULATIVECOUNT) IS NOT NULL ) MTOTAL
ON (T_GTTCHARGEVALUES.CASTACCOUNTID=MTOTAL.CASTACCOUNTID AND
T_GTTCHARGEVALUES.CHARGESCHEME=MTOTAL.CHARGESCHEME) WHEN MATCHED THEN UPDATE SET
CUMULATIVEVALUE=CUMULATIVEVALUE+MTOTAL.MAXMONTHVALUE ,
CUMULATIVECOUNT=CUMULATIVECOUNT+MTOTAL.MAXMONTHCOUNT
Error: ORA-1013
ORA-01013: user requested cancel of current operation
Global Information
Status : DONE (ERROR)
Instance ID : 1
Session : XXXX(2815:12369)
SQL ID : 70kzttjbyyspt
SQL Execution ID : 16777216
Execution Started : 04/27/2012 09:43:27
First Refresh Time : 04/27/2012 09:43:27
Last Refresh Time : 04/27/2012 09:48:43
Duration : 316s
Module/Action : SQL*Plus/-
Service : SYS$USERS
Program : sqlplus@XXXX (TNS V1-V3)
Binds
========================================================================================================================
| Name | Position | Type | Value |
========================================================================================================================
| :B1 | 1 | VARCHAR2(32) | 2012-04-25 |
========================================================================================================================
Global Stats
====================================================================================================================
| Elapsed | Queuing | Cpu | IO | Application | Concurrency | Cluster | Other | Buffer | Read | Read |
| Time(s) | Time(s) | Time(s) | Waits(s) | Waits(s) | Waits(s) | Waits(s) | Waits(s) | Gets | Reqs | Bytes |
====================================================================================================================
| 7555 | 0.00 | 4290 | 2812 | 0.08 | 27 | 183 | 243 | 3M | 294K | 7GB |
====================================================================================================================
SQL Plan Monitoring Details (Plan Hash Value=323941584)
==========================================================================================================================================================================================================
| Id | Operation | Name | Rows | Cost | Time | Start | Execs | Rows | Read | Read | Mem | Activity | Activity Detail |
| | | | (Estim) | | Active(s) | Active | | (Actual) | Reqs | Bytes | (Max) | (%) | (# samples) |
==========================================================================================================================================================================================================
| 0 | MERGE STATEMENT | | | | | | 1 | | | | | | |
| 1 | MERGE | T_GTTCHARGEVALUES | | | | | 1 | | | | | | |
| 2 | PX COORDINATOR | | | | 57 | +1 | 481 | 0 | 317 | 5MB | | 4.05 | latch: shared pool (40) |
| | | | | | | | | | | | | | os thread startup (17) |
| | | | | | | | | | | | | | Cpu (7) |
| | | | | | | | | | | | | | DFS lock handle (36) |
| | | | | | | | | | | | | | SGA: allocation forcing component growth (14) |
| | | | | | | | | | | | | | latch: parallel query alloc buffer (200) |
| 3 | PX SEND QC (RANDOM) | :TQ10003 | 1 | 19054 | | | | | | | | | |
| 4 | VIEW | | | | | | | | | | | | |
| 5 | FILTER | | | | | | | | | | | | |
| 6 | SORT GROUP BY | | 1 | 19054 | | | | | | | | | |
| 7 | PX RECEIVE | | 1 | 19054 | | | | | | | | | |
| 8 | PX SEND HASH | :TQ10002 | 1 | 19054 | | | 240 | | | | | | |
| 9 | SORT GROUP BY | | 1 | 19054 | 246 | +70 | 240 | 0 | | | 228M | 49.32 | Cpu (3821) |
| 10 | FILTER | | | | 245 | +71 | 240 | 3G | | | | 0.08 | Cpu (6) |
| 11 | HASH JOIN | | 1 | 19054 | 259 | +57 | 240 | 3G | | | 276M | 4.31 | Cpu (334) |
| 12 | PX RECEIVE | | 1M | 5 | 259 | +57 | 240 | 1M | | | | 0.04 | Cpu (3) |
| 13 | PX SEND HASH | :TQ10000 | 1M | 5 | 6 | +56 | 240 | 1M | | | | 0.01 | Cpu (1) |
| 14 | PX BLOCK ITERATOR | | 1M | 5 | 6 | +56 | 240 | 1M | | | | 0.03 | Cpu (1) |
| | | | | | | | | | | | | | PX Deq: reap credit (1) |
| 15 | TABLE ACCESS FULL | T_GTTCHARGEVALUES | 1M | 5 | 7 | +55 | 5486 | 1M | 5487 | 86MB | | 2.31 | gc cr grant 2-way (3) |
| | | | | | | | | | | | | | gc current block lost (7) |
| | | | | | | | | | | | | | Cpu (7) |
| | | | | | | | | | | | | | db file sequential read (162) |
| 16 | PX RECEIVE | | 78M | 19047 | 255 | +61 | 240 | 801K | | | | 0.03 | IPC send completion sync (2) |
| 17 | PX SEND HASH | :TQ10001 | 78M | 19047 | 250 | +66 | 240 | 3M | | | | 0.06 | Cpu (5) |
| 18 | PX BLOCK ITERATOR | | 78M | 19047 | 250 | +66 | 240 | 4M | | | | | |
| 19 | TABLE ACCESS FULL | T_CACHARGES | 78M | 19047 | 254 | +62 | 1016 | 4M | 288K | 6GB | | 37.69 | gc buffer busy acquire (104) |
| | | | | | | | | | | | | | gc cr block 2-way (1) |
| | | | | | | | | | | | | | gc cr block lost (9) |
| | | | | | | | | | | | | | gc cr grant 2-way (14) |
| | | | | | | | | | | | | | gc cr multi block request (1) |
| | | | | | | | | | | | | | gc current block 2-way (3) |
| | | | | | | | | | | | | | gc current block 3-way (2) |
| | | | | | | | | | | | | | gc current block busy (1) |
| | | | | | | | | | | | | | gc current grant busy (2) |
| | | | | | | | | | | | | | Cpu (58) |
| | | | | | | | | | | | | | latch: gc element (1) |
| | | | | | | | | | | | | | db file parallel read (26) |
| | | | | | | | | | | | | | db file scattered read (207) |
| | | | | | | | | | | | | | db file sequential read (2433) |
| | | | | | | | | | | | | | direct path read (1) |
| | | | | | | | | | | | | | read by other session (57) |
==========================================================================================================================================================================================================
Parallel Execution Details (DOP=240 , Servers Allocated=480)
Instances : 3chris_c wrote:
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | Used-Mem |
|* 9 | TABLE ACCESS FULL | T_CACHARGES | 24 | 392K| 21M|00:02:47.48 | 608K| 177K| | | |
Based on the discrepancy between the estimated number of rows and the actual, and the below posted bind value of 2012-04-25 i'd first be checking if the statistics on T_CACHARGES are up to date.
As a reference
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:4399338600346902127
So that would be my first avenue of exploration.
Cheers, -
Here's is the contents of v$system_event..
Is this
EVENT TOTAL_WAITS TIME_WAITED AVERAGE_WAIT
enq: TX - index contention 40564851 214701526 5.29
enq: TX - row lock contention 188846 12454614 65.95
enq: SQ - contention 141971 70568 0.5
cause for concern?
EVENT TOTAL_WAITS TIME_WAITED AVERAGE_WAIT
SQL*Net message to client 6015051449 607254 0
SQL*Net message from client 6015048542 178177969892 29.62
gcs remote message 2948555287 2633481757 0.89
CGS wait for IPC msg 1517805027 634397 0
db file sequential read 1500615188 816364485 0.54
ges remote message 1247679701 1407300224 1.13
gc cr multi block request 778432813 9913464 0.01
gc current block 2-way 747852637 38030616 0.05
db file scattered read 709428365 460939295 0.65
rdbms ipc message 708473316 37650068633 53.14
gc buffer busy acquire 671285134 1033621285 1.54
PX Deq: reap credit 667784615 484449 0
gcs log flush sync 592376026 171712257 0.29
gc cr block 2-way 530861847 19607062 0.04
library cache pin 437937120 15126237 0.03
log file sync 379523272 797193932 2.1
DIAG idle wait 359607166 2822108755 7.85
log file parallel write 351225436 259263769 0.74
LNS ASYNC end of log 350170653 1398410516 3.99
LNS wait on SENDREQ 321652621 3209301 0.01
PX qref latch 297396661 94308 0
read by other session 289140108 148440270 0.51
buffer deadlock 163505781 983055 0.01
gc current block busy 119223825 467716658 3.92
PX Deq: Table Q Normal 117332841 23574867 0.2
ksxr poll remote instances 110480324 90333 0
buffer busy waits 106938153 19933900 0.19
direct path read 93429599 108427028 1.16
SQL*Net more data from client 86471785 23026529 0.27
gc current grant busy 84978157 28215346 0.33
control file sequential read 82646297 23694583 0.29
PX Deq Credit: send blkd 78641669 9569299 0.12
latch: cache buffers chains 74218671 690277 0.01
gc current grant 2-way 72557796 1920419 0.03
library cache: mutex X 71106697 75993 0
DFS lock handle 70722498 2716407 0.04
gc cr grant 2-way 64558237 1633004 0.03
PX Deq: Execution Msg 61706261 314222076 5.09
gc cr block busy 61469863 119850802 1.95
library cache lock 52428649 3773354 0.07
PX Deq: Slave Session Stats 48040224 1886805 0.04
db file parallel read 46415188 118467902 2.55
IPC send completion sync 46250594 965101 0.02
enq: TX - index contention 40564851 214701526 5.29
PX Deq: Execute Reply 39689685 17243721 0.43
gc buffer busy release 36976909 242714774 6.56
SQL*Net more data to client 36627952 44167 0
PX Deq: Msg Fragment 30501244 343397 0.01
rdbms ipc reply 29725302 1352370 0.05
RMAN backup & recovery I/O 28824547 37722662 1.31
reliable message 27892263 3082134 0.11
PX Idle Wait 27356097 4651277341 170.03
ASM file metadata operation 25098749 8850323 0.35
gc object scan 22705857 7485 0
db file parallel write 19896252 52152606 2.62
latch: ges resource hash list 19336183 427451 0.02
enq: PS - contention 19143961 707455 0.04
PX Deq: Parse Reply 19093356 895799 0.05
gc cr disk read 17816846 448909 0.03
ASM background timer 16101806 1383957874 85.95
PX Deq: Slave Join Frag 16044789 233149 0.01
wait for unread message on broadcast channel 15056320 1413552546 93.88
cursor: mutex X 13435193 24140 0
KJC: Wait for msg sends to complete 13268497 11397 0
PX Deq: Signal ACK RSG 13214824 101941 0.01
KSV master wait 13206286 4235645 0.32
direct path read temp 12617694 5487608 0.43
PX Deq Credit: need buffer 11675868 879967 0.08
row cache lock 11536185 398216 0.03
PX Deq Credit: Session Stats 9480862 78910 0.01
SQL*Net message to dblink 9312894 1538 0
SQL*Net message from dblink 9312894 6279631 0.67
control file parallel write 7760982 11854435 1.53
pmon timer 7558889 1412576090 186.88
PX Deq: Join ACK 7548816 498931 0.07
gc current multi block request 6035173 155898 0.03
PING 5706961 1413230267 247.63
enq: XR - database force logging 4662671 198813 0.04
class slave wait 4561877 7097429006 1555.81
Streams AQ: waiting for messages in the queue 4495828 1543411682 343.3
SQL*Net more data from dblink 3696582 444575 0.12
LGWR wait for redo copy 3655353 17840 0
log file sequential read 3387305 6610414 1.95
Log archive I/O 2990486 276772 0.09
SQL*Net break/reset to client 2971976 2385935 0.8
direct path write temp 2839390 2522114 0.89
Space Manager: slave idle wait 2827526 1412987186 499.73
latch: shared pool 2808517 298150 0.11
latch: gc element 2421717 24688 0.01
SGA: MMAN sleep for component shrink 2336447 2458094 1.05
latch: enqueue hash chains 2279645 15435 0.01
latch free 2089418 78732 0.04
gc current split 2044784 1864009 0.91
PX Deq: Signal ACK EXT 1976164 19263 0.01
enq: FB - contention 1473469 61036 0.04
cursor: pin S wait on X 1313129 1464789 1.12
SQL*Net more data to dblink 1232891 986 0
Streams AQ: RAC qmn coordinator idle wait 1211300 788 0
enq: HW - contention 1175390 2077008 1.77
latch: session allocation 1167768 21883 0.02
Streams AQ: qmn coordinator idle wait 1144699 1412546634 1233.99
Streams AQ: qmn slave idle wait 1031585 2227183681 2158.99
lock deadlock retry 962937 4698 0
enq: CF - contention 956154 609647 0.64
latch: cache buffers lru chain 902764 37552 0.04
latch: object queue header operation 817911 27717 0.03
global enqueue expand wait 768633 654105 0.85
Data file init write 756191 329758 0.44
latch: gcs resource hash 647021 4147 0.01
local write wait 603007 286191 0.47
latch: row cache objects 599358 6453 0.01
ges lmd/lmses to freeze in rcfg - mrcvr 481759 156345 0.32
shared server idle wait 471190 1413238589 2999.3
enq: RF - DG Broker Current File ID 469833 23209 0.05
smon timer 432383 1411851085 3265.28
SGA: allocation forcing component growth 363333 379008 1.04
gc current retry 341104 1121252 3.29
enq: RF - synch: DG Broker metadata 319143 588290 1.84
enq: PG - contention 313659 14830 0.05
enq: TT - contention 260134 11207172 43.08
enq: KO - fast object checkpoint 236745 820808 3.47
dispatcher timer 236637 1413242481 5972.2
direct path write 231382 191008 0.83
cursor: pin S 229011 394 0
Streams AQ: waiting for time management or cleanup tasks 199981 1413148548 7066.41
enq: TX - row lock contention 188846 12454614 65.95
enq: TX - allocate ITL entry 153703 54252 0.35
enq: SQ - contention 141971 70568 0.5
ksdxexeother 141885 56 0
latch: redo allocation 138912 1858 0.01
recovery area: computing applied logs 126415 45925 0.36
gc current block congested 126318 21768 0.17
resmgr:cpu quantum 123074 151384 1.23
jobq slave wait 120678 35574221 294.79
Datapump dump file I/O 90431 9127 0.1
ges inquiry response 89402 4041 0.05
os thread startup 83809 222586 2.66
cr request retry 80062 71896 0.9
PX Deq: Table Q Sample 79665 133402 1.67
gc cr block congested 79026 14792 0.19
gc cr failure 77521 25019 0.32
enq: WF - contention 73983 825388 11.16
enq: TQ - TM contention 72871 3319 0.05
lock escalate retry 65714 1574 0.02
buffer exterminate 59775 64919 1.09
fbar timer 47136 1413183353 29980.98
log file switch completion 46911 452097 9.64
recovery area: computing obsolete files 45699 8547 0.19
enq: US - contention 40401 8805 0.22
enq: TM - contention 39149 5435032 138.83
library cache load lock 36311 382575 10.54
kjbdrmcvtq lmon drm quiesce: ping completion 31668 47443 1.5
enq: TD - KTF dump entries 31468 1424 0.05
enq: RO - fast object reuse 28422 31772 1.12
parallel recovery slave wait for change 27558 3163 0.11
name-service call wait 23694 181533 7.66
control file single write 22375 1624 0.07
kksfbc child completion 21239 106926 5.03
PX Deq: Table Q qref 19325 245 0.01
enq: TX - contention 18805 113253 6.02
latch: messages 17203 181 0.01
enq: RS - prevent file delete 16913 1013 0.06
enq: RS - prevent aging list update 15682 642 0.04
PX Deq: Table Q Get Keys 14322 42935 3
gc current grant congested 14292 2192 0.15
cursor: mutex S 13285 8 0
log file single write 13164 5371 0.41
latch: undo global data 12649 178 0.01
kksfbc research 11894 12680 1.07
parallel recovery slave idle wait 11193 5872 0.52
wait list latch free 11026 11794 1.07
enq: CT - state 11001 417 0.04
latch: checkpoint queue latch 10526 132 0.01
enq: PE - contention 10506 1139 0.11
ARCH wait on SENDREQ 9957 216480 21.74
gc cr grant congested 9465 1584 0.17
wait for scn ack 9377 3155 0.34
enq: TA - contention 8856 324 0.04
log buffer space 8777 89323 10.18
enq: TK - Auto Task Serialization 8542 343 0.04
enq: DR - contention 7842 323 0.04
process diagnostic dump 7707 2072 0.27
JOX Jit Process Sleep 7612 11286431 1482.72
enq: TC - contention 7357 340817 46.33
ges global resource directory to be frozen 7140 12299 1.72
enq: CO - master slave det 6850 312 0.05
enq: JS - job run lock - synchronize 6704 397 0.06
gcs drm freeze in enter server mode 6542 40742 6.23
enq: TS - contention 5959 89332 14.99
ARCH wait for archivelog lock 5600 36 0.01
PX Nsq: PQ load info query 5377 104798 19.49
db file single write 5373 3452 0.64
gc remaster 5315 50625 9.52
latch: parallel query alloc buffer 4939 1906 0.39
enq: TO - contention 4799 143 0.03
enq: AF - task serialization 4395 161 0.04
enq: PI - contention 4251 163 0.04
ges2 LMON to wake up LMD - mrcvr 4210 28 0.01
enq: DL - contention 3889 239 0.06
kjctssqmg: quick message send wait 3408 22 0.01
LNS wait on DETACH 3275 741 0.23
ksfd: async disk IO 3274 1 0
LNS wait on ATTACH 3273 51940 15.87
ARCH wait on DETACH 3231 714 0.22
ARCH wait on ATTACH 3226 43238 13.4
enq: BR - file shrink 2787 116 0.04
write complete waits 2631 1070 0.41
enq: MD - contention 2596 67 0.03
enq: WL - contention 2198 266518 121.25
single-task message 2098 25896 12.34
enq: OD - Serializing DDLs 2054 66 0.03
resmgr:internal state change 2001 14735 7.36
ARCH wait on c/f tx acquire 2 1751 175230 100.07
enq: WR - contention 1636 69 0.04
latch: cache buffer handles 1610 29 0.02
statement suspended, wait error to be cleared 1497 22626 15.11
Streams AQ: qmn coordinator waiting for slave to start 1214 678966 559.28
enq: PD - contention 1182 33 0.03
JS kgl get object wait 1096 4922 4.49
undo segment extension 1070 10065 9.41
PL/SQL lock timer 949 8739819 9209.5
enq: AE - lock 937 28 0.03
LGWR-LNS wait on channel 832 913 1.1
ges DFS hang analysis phase 2 acks 816 495 0.61
latch: redo writing 729 9 0.01
gc quiesce 665 564 0.85
enq: JS - queue lock 482 2111 4.38
PX Deq: Test for credit 442 13 0.03
enq: SS - contention 386 274 0.71
recovery area: computing dropped files 328 1400 4.27
recovery area: computing backed up files 328 496 1.51
ksdxexeotherwait 279 10592 37.97
log switch/archive 250 137570 550.28
gc domain validation 223 39964 179.21
auto-sqltune: wait graph update 195 96514 494.95
wait for a undo record 170 1214 7.14
parallel recovery coord send blocked 168 4 0.02
enq: JS - wdw op 168 3741 22.27
enq: KT - contention 165 5 0.03
switch logfile command 156 6290 40.32
gcs resource directory to be unfrozen 149 12839 86.17
Data Guard Broker Wait 139 10906 78.46
enq: SK - contention 129 4 0.03
enq: JS - job recov lock 128 4 0.03
gc cr block lost 125 6772 54.17
virtual circuit wait 122 3 0.03
ges LMON to get to FTDONE 100 187 1.87
enq: CU - contention 80 242 3.02
enq: JQ - contention 78 7 0.09
cursor: pin X 73 83 1.14
parallel recovery coord wait for reply 70 510 7.29
PX Deq: Txn Recovery Start 67 3436 51.29
SQL*Net break/reset to dblink 60 0 0
gc current block lost 57 2869 50.33
ges LMD suspend for testing event 51 709 13.89
inactive session 46 4550 98.91
recovery read 45 5 0.11
JS kill job wait 41 3548 86.53
enq: AS - service activation 40 1 0.03
enq: TL - contention 35 2 0.05
enq: UL - contention 34 524 15.42
gcs enter server mode 33 1559 47.24
wait for stopper event to be increased 30 218 7.27
enq: TQ - DDL contention 24 300 12.52
enq: MR - contention 21 1 0.03
ges reconfiguration to start 20 54 2.72
ges enter server mode 20 502 25.08
buffer latch 18 1337 74.26
enq: SR - contention 18 1 0.05
Streams: RAC waiting for inter instance ack 18 3748 208.21
enq: PR - contention 17 46 2.72
kupp process wait 16 166 10.39
checkpoint completed 15 3678 245.19
PX Deque wait 14 68 4.87
enq: BF - allocation contention 14 1 0.08
enq: XL - fault extent map 14 51 3.66
enq: FU - contention 14 17 1.18
enq: TH - metric threshold evaluation 13 114 8.78
enq: MW - contention 12 0 0.04
enq: DD - contention 10 0 0.04
process terminate 8 41 5.08
ges cgs registration 8 151 18.9
buffer resize 7 0 0
ktm: instance recovery 7 698 99.66
LNS wait on LGWR 6 0 0
ASM background starting 6 381 63.43
gc cr block 3-way 5 0 0.08
enq: PV - syncstart 5 9 1.74
Global transaction acquire instance locks 4 4 1.09
enq: RS - read alert level 4 0 0.04
LGWR wait on LNS 3 0 0
gc recovery 3 540 179.85
Streams AQ: enqueue blocked on low memory 3 544 181.2
DBWR range invalidation sync 3 17 5.83
enq: DM - contention 3 0 0.03
enq: RF - FSFO Observer Heartbeat 3 0 0.03
enq: JS - q mem clnup lck 3 0 0
DG Broker configuration file I/O 2 0 0
enq: RC - Result Cache: Contention 2 493 246.6
enq: KM - contention 2 0 0.03
enq: RT - contention 2 0 0.04
instance state change 2 0 0.12
kkdlgon 2 10 5.11
enq: TQ - INI contention 2 292 146.07
enq: JS - contention 2 0 0
ARCH wait for netserver start 1 400 400.02
log file switch (checkpoint incomplete) 1 3 3.44
JS coord start wait 1 50 50.09
ges lmd and pmon to attach 1 1 1.26
wait for tmc2 to complete 1 3 3.03
control file heartbeat 1 400 400.02
enq: SW - contention 1 0 0.04
enq: PW - perwarm status in dbw0 1 0 0.09
enq: FS - contention 1 0 0.04
enq: XR - quiesce database 1 0 0.04
enq: RS - write alert level 1 0 0.02
enq: CN - race with init 1 0 0.03
enq: FE - contention 1 4 3.77
Wait for shrink lock2 1 10 10.03
enq: IA - contention 1 0 0.02
enq: RF - atomicity 1 0 0.05
enq: RF - synchronization: aifo master 1 0 0.02
enq: RF - RF - Database Automatic Disable 1 0 0.06
enq: WP - contention 1 0 0.02
enq: TB - SQL Tuning Base Cache Load 1 0 0.05
enq: JS - evt notify 1 0 0.02Edited by: steffi on Mar 20, 2011 12:21 AM
Edited by: steffi on Mar 20, 2011 8:18 AM
Edited by: steffi on Mar 20, 2011 8:19 AMText can be formatted by tagging the beginning and end of the block of text with the code ta
\Formatted text goes here.
\Example:
This is formatted.When cutting and pasting text such as execution plans, excerpts from AWR reports, etc, it will maintain spacing and formatting, and make for much easier reading.
As to the content, well, dumping the contents of v$system_event is pretty close to useless.
As to the first three events you listed, 'enq: TX - index contention', 'enq: TX - row lock contention', 'enq: SQ - contention', well, all of those are easily tunable.
First, for 'enq: SQ - contention', check your sequences. Do you have any NOCACHE sequences? Or sequences with small caches?
As for 'enq: TX - row lock contention', well that's fairly self-explanatory. You have multiple sessions trying to lock the same row in the same table at the same time.
Last, 'enq: TX - index contention', that's non-row level contention on an index. For example, if you have a unique index, insert a row w/ column value 1, but don't commit, then try to insert that same value from another session.
But, before you do any of that, I think you need to clearly understand where the bottlenecks are. Try taking some AWR snapshots, about 5 minutes apart, when you're having performance problems. Look at the AWR report for that 5 minute period. In particular, look at your Top 5 timed events.
Hope that helps,
-Mark
Maybe you are looking for
-
Wrong File(parent, child) behaviour on Windows when child starts with one s
I've sent following problem report: A DESCRIPTION OF THE PROBLEM : On Windows when relative path starts from one slash it should be resolved from current disk (not current directory). This behaviour should be preserved for Files constructed with cons
-
Combining Two Internal Tables??
How do you combine two internal tables(not by append statement)...if any one knws please let me know..thanks in advance..
-
User manual for Premiere Elements 4
Is there a link for a complete user manual for Premiere Elements 4? I found a link for the Photoshop Elements 6 user manual (which was not shipped with 6.0.) Wondering if there's one for Premiere 4
-
i followed instructions on line and reset the pixels the wrong way and now the font is so tiny that I cannot even see it to adjust. I tried to just reinstall firefox, but that did not work
-
Songs appearing on multiple albums
iTunes v7.6.1.9 I think I probably know the answer to this but if someone can confirm it I can sleep easy. I have over 5500 tracks loaded into iTunes and there are a number of duplicate songs that appear on more than album. Is there a way of telling