DEADLOCK DETECTED == wwv_flow_data

Question

I was hoping (praying) someone could shed further light on why we are experiencing deadlocks in our APEX business applications. We have a number of large organisations using our business suite of apex applications (all LIVE - searches, inserts, updates, deletes) and we are currently struggling to find out what's causing deadlocks that are being generated (these are severely affecting the server performance etc). I know it's difficult to summarise what's causing without a reproducible example but unfortunately we also cannot reproduce it at our developement centre. It is currently only occuring at customer sites on a consistent basis.
Does anyone have any idea how we can go about tracing the cause of these Deadlocks ?
Why are they occurring - there is no specific dml against apex dictionary tables (api's are alwasy used to do things like set session state, clear cache etc)
All we currently have is the generated trace file and some Deadlock examples are below. Why is it always the 'wwv_flow_data' table ?
Also, the majority of them seem to be updating an item to an apex URL (bind vars separated by colons) ie :B6 || ':' || :B5 || ':' || :B4 || ':' || :B3
An example is:
*** 2009-09-29 10:50:24.760
*** ACTION NAME:(PAGE 169) 2009-09-29 10:50:24.759
*** MODULE NAME:(APEX:APPLICATION 147) 2009-09-29 10:50:24.759
*** SERVICE NAME:(SYS$USERS) 2009-09-29 10:50:24.759
*** CLIENT ID:(GHANZANFARJ:2261912940872031) 2009-09-29 10:50:24.759
*** SESSION ID:(4947.3819) 2009-09-29 10:50:24.759
DEADLOCK DETECTED ( ORA-00060 )
[Transaction Deadlock]
The following deadlock is not an ORACLE error. It is a
deadlock due to user error in the design of an application
or from issuing incorrect ad-hoc SQL. The following
information may aid in determining the deadlock:
Deadlock graph:
---------Blocker(s)-------- ---------Waiter(s)---------
Resource Name process session holds waits process session holds waits
TX-0089002a-00032b7e 1008 4947 X 710 3585 X
TX-01240010-00023f5f 710 3585 X 1008 4947 X
session 4947: DID 0001-03F0-00000028     session 3585: DID 0001-02C6-00000087
session 3585: DID 0001-02C6-00000087     session 4947: DID 0001-03F0-00000028
Rows waited on:
Session 3585: obj - rowid = 0004AC08 - AABKwIAAaAAAEX1ABZ
(dictionary objn - 306184, file - 26, block - 17909, slot - 89)
Session 4947: obj - rowid = 0004AC08 - AABKwIAAaAAAEX1ABX
(dictionary objn - 306184, file - 26, block - 17909, slot - 87)
Information on the OTHER waiting sessions:
Session 3585:
pid=710 serial=6190 audsid=169143612 user: 6970/<none>
O/S info: user: oracle, term: , ospid: 2113768, machine: rbisprodapp
program: httpd@rbisprodapp (TNS V1-V3)
client info: GHANZANFARJ
application name: APEX:APPLICATION 147, hash value=3432819319
action name: PAGE 169, hash value=1337406124
Current SQL Statement:
UPDATE WWV_FLOW_DATA SET ITEM_VALUE = :B6 || ':' || :B5 || ':' || :B4 || ':' || :B3 WHERE FLOW_INSTANCE = :B2 AND ITEM_ID = :B1
End of information on OTHER waiting sessions.
Current SQL statement for this session:
DELETE FROM WWV_FLOW_DATA WHERE FLOW_INSTANCE = :B1 AND ITEM_ID IN (SELECT ID FROM WWV_FLOW_PAGE_PLUGS WHERE FLOW_ID = :B3 AND PAGE_ID = :B2 AND PLUG_SOURCE_TYPE IN ( 'SIMPLE_CHART', 'UPDATABLE_SQL_QUERY', 'DBMSSQL_CURSOR', 'FUNCTION_RETURNING_DBMSSQL_CURSOR', 'FUNCTION_RETURNING_SQL_QUERY_CACHED', 'FUNCTION_RETURNING_SQL_QUERY', 'STRUCTURED_QUERY', 'SQL_QUERY', 'DYNAMIC_QUERY'))
----- PL/SQL Call Stack -----
object line object
handle number name
7000003679ac978 227 package body FLOWS_030100.WWV_FLOW_DISP_PAGE_PLUGS
70000050e85efb8 10283 package body FLOWS_030100.WWV_FLOW
7000004340cf8c0 255 procedure FLOWS_030100.F
70000039feafba8 31 anonymous block

Answer 1

This sounds like you are missing some commits in your DML.
Deadlocks normally only occur in Oracle when two sessions try to lock the same rows for update.
Session1: locks row A;
Session2: locks row B;
Session1: tries to lock row B;-- Waiting because of lock by session2
Session2: tries to lock row A; -- Waiting because of lock by session1 / Deadlock
This situation is highly unuasual for an Oracle application and is (almost) always a flaw in application design.
You should try to figure out which DML is being triggered by the two locking sessions. This maybe on an APEX page, but could also be in a database trigger or procedure.
To rule other things out: check which SESSION_ID's are being used by the application at the time of the deadlock. Maybe your users are using some bookmarks that point to the same SESSION_ID...