Lock conflicts

Question

I'm quite a newbie when it comes to using Berkeley DB. I am using BDB
dbxml.
I have an application that fetches data from an external source and
writes that into dbxml. The data fetch takes a long time, so I start a
new thread to do this. All writes into dbxml for that batch are then
done in that same new thread.
When I have a few batches running at the same time, after variable
amounts of time, the application will hang. When this happens my only
option is to run db_recover.
The problem happens when I have more than one of these: multiple writes
in multiple threads. I am protecting everything with transactions, but
am getting lock conflicts.
The output of db_stat -CA includes this:
1 Total number of locks not immediately available due to
conflicts
80000275 READ 28 HELD RefData page
3105
80000275 WRITE 55 HELD RefData page
3105
80000276 READ 1 WAIT RefData page
3105
Unfortunately, I don't know how to interpret this. How do I tell where
the conflict is happening? What sort of pattern should I be following
to prevent this from happening?
I don't know what information would help make the above clearer, but am happy to provide more.
Many thanks,
PC

Answer 1

After reading some more of the manual, I see the following paragraph:
"This has important implications. If a single thread of control opens two cursors, uses a combination of cursor and non-cursor operations, or begins two separate transactions, the operations are performed on behalf of different lockers. Conflicts that arise between these different lockers may not cause actual deadlocks, but can, in fact, permanently block the thread of control. For example, assume that an application creates a cursor and uses it to read record A. Now, assume a second cursor is opened, and the application attempts to write record A using the second cursor. Unfortunately, the first cursor has a read lock, so the second cursor cannot obtain its write lock. However, that read lock is held by the same thread of control, so the read lock can never be released if we block waiting for the write lock. This might appear to be a deadlock from the application's perspective, but Berkeley DB cannot identify it as such because it has no knowledge of which lockers belong to which threads of control. For this reason, application designers are encouraged to close cursors as soon as they are done with them."
I am wondering if this is what is happening as db_deadlock is not returning anything.
In dbxml, how is a cursor created?
In order to make modifications to nodes, first I have to select those nodes. I then have to modify those nodes. Presumably, a select will create a read lock, and a modify will then create a write lock? In my application i have a function "setValues" that does this- first select nodes, then modify the selected nodes in some way. I am wondering if two instances of setValues in separate threads are causing the problem.
I see that I have a read lock waiting on a write lock. How do I tell which reads and writes these are happening on?
Thanks,
PC