The DB_KEYEMPTY return code ?

Question

Hi,
We are using a Berkeley DB queue in a transactional database. Our application is designed with some threads inserting records into the queue and a single thread reading records from the queue. Our concern is around the reading of the queue where we use the "Db::get" method to read records using a given recno key:
     rtn = m_queueDb->get(m_dbTxn, &m_queueDbKey, &m_queueDbData, DB_READ_COMMITTED);
Note, we are intentionally not using DB_CONSUME/DB_CONSUME_WAIT as we may want to read the record again at a later date. We also have maintenance tasks to clean-up (consume) records when we are certain the records are no longer needed in the queue.
We would like someone explain the scenarios in which the "DB_KEYEMPTY" code is returned for the "Db::get" method:
The documentation appears to be a little unclear around the exact circumstances that cause the DB_KEYEMPTY code to be returned:
http://www.sleepycat.com/docs/api_cxx/db_get.html
Since we use the "Db::get" method with an incrementing recno key to our retrieve records in the queue, we assume that it is possible to get a DB_KEYEMPTY return code for records that were part of a transaction which was rolled back (i.e. these would be implicit records we should skip over). Is that a fair assumption?
Also, what else might cause a DB_KEYEMPTY code to be returned, because we seem to be getting this even when nothing is rolled back?
Thanks,
Atiq.

Answer 1

Hi Atiq,
Since the reader and writers are in separate threads,
there is always the chance that the reader thread may
be reading the queue while a writer is adding records
in an open transaction. Due to the nature of our
application, it is not appropriate for us to
co-ordinate our worker threads such that we suspend
the writer threads while the reader thread attempts
to read the queue. We were expecting that the
DB_READ_COMMITTED isolation level would be sufficient
to take care of this.The readers threads will only see data that had been committed by the writer threads. If your application is under the constraint of having the threads work so that neither of them can suspend and wait for the other one, maybe you should consider relaxing the isolation degree, and use DB_READ_UNCOMMITTED (this flag should be specified when opening the database "DB->open", as well as when creating the transaction "DB_ENV->txn_begin").
Of course, this means that the readers threads can get data that could disappear at a future moment, due to aborting a writing transaction.
> DB_KEYEMPTY could be thrown in case you are
trying to retrieve a record
inserted in a transaction that hasn't been yetcommitted (or a record inserted
through using an aborted transaction).
OK. The statement above outlines two different
scenarios:
(1) DB_KEYEMPTY can be thrown to the reader thread
while trying to retrieve a record inserted in an open
transaction (i.e. not yet committed). Is this true
while using the DB_READ_COMMITTED flag? If so, in
this particular case we should wait and then trying
reading the record again in the expectancy that it
will be committed soon.Yes this is the behavior when using the second isolation degree, DB_READ_COMMITTED. The reader thread should begin working after the writing transaction had been committed (estimate how long you should delay starting your readers threads).
(2) DB_KEYEMPTY can be thrown to the reader thread
while trying to retrieve a record inserted in an
aborted transaction. In this particular case I assume
it is sensible to skip this record?Records inserted in an aborted transaction will never be seen by the reader thread, since they hadn't been committed. If you were using DB_READ_UNCOMMITTED, these records would be seen by the reader threads, before being dropped.
> You may also step into this exception if the
specified key exists, but wasn't
explicitly created through your application or waslater deleted.
The statement "wasn't explicitly created through your
application" is a little unclear to me. Can you
please clarify further what this means?It's possible to insert a record that has a specific number which is greater that any other record numbers existing in the Recno (or Queue) database. Fictive records will then be created with record numbers between the newly inserted record number and the first record number smaller than this one (e.g.: your database has 48 records; the greatest record number in the database than, is 48; let's say that you insert a record with a specific record number of 52; implicit records are created, with record numbers 49 to 51).
If you try to retrieve any of these " fictive" records, DB_KEYEMPTY will be thrown, because for these records, data doesn't exist.
Thank you, best regards,
Andrei Costache
Oracle Support Services