Repeated Opening of database in a Txn causes Logging region out of memory

Question

Hi
BDB 4.6.21
When I open and close a single database file repeatedly, it causes the error message "Logging region out of memory; you may need to increase its size". I have set the 65KB default size for set_lg_regionmax. Is there any work around for solving this issue, other than increasing the value for the set_lg_regionmax. Even if we set it to a higher value, we cannot predict how the clients of BDB will opens a file and closes a database file. Following is a stand alone program, using which one can reproduce the scenario.
void main()
const int SUCCESS = 0;
ULONG uEnvFlags = DB_CREATE | DB_INIT_MPOOL | DB_INIT_LOG | DB_INIT_TXN |DB_INIT_LOCK | DB_THREAD;// |
//DB_RECOVER;
LPCSTR lpctszHome = "D:\\Nisam\\Temp";
int nReturn = 0;
DbEnv* pEnv = new DbEnv( DB_CXX_NO_EXCEPTIONS );
nReturn = pEnv->set_thread_count( 20 );
nReturn = pEnv->open( lpctszHome, uEnvFlags, 0 );
if( SUCCESS != nReturn )
return 0;
DbTxn* pTxn = 0;
char szBuff[MAX_PATH];
UINT uDbFlags = DB_CREATE | DB_THREAD;
Db* pDb = 0;
Db Database( pEnv, 0 );
lstrcpy( szBuff, "DBbbbbbbbbbbbbbbbbbbbbbbbbbb________0" );// some long name
// First create the database
nReturn = Database.open( pTxn, szBuff, 0, DB_BTREE, uDbFlags, 0 );
nReturn = Database.close( 0 );
for( int nCounter = 0; 10000 > nCounter; ++nCounter )
// Now repeatedly open and close the above created database
pEnv->txn_begin( pTxn, &pTxn, 0 );
Db Database( pEnv, 0 );
nReturn = Database.open( pTxn, szBuff, 0, DB_BTREE, uDbFlags, 0 );
if( SUCCESS != nReturn )
// when the count reaches 435, the error occurs
pTxn->abort();
pDb->close( 0 );
pEnv->close( 0 );
return 0;
pTxn->abort();
pTxn = 0;
Database.close( 0 );
By the way, following is the content of my DB_CONFIG file
set_tx_max 1000
set_lk_max_lockers 10000
set_lk_max_locks 100000
set_lk_max_objects 100000
set_lock_timeout 20000
set_lg_bsize 1048576
set_lg_max 10485760
#log region: 66KB
set_lg_regionmax 67584
set_cachesize 0 8388608 1
Thanks are Regards
Nisam

Answer 1

Hi Nisam,
I was able to reproduce the problem using Berkeley DB 4.6.21. The problem is with releasing the FNAME structure in certain cases involving abort Transactions. In a situation where you have continuous (in a loop) transactional (open, abort, close) of databases you will notice (as you did) that the log region size needs to be increased (set_lg_regionmax).
This problem was identified and reproduced yesterday (thanks for letting us know about this) and is reported as SR #15953. It will be fixed in the next release of Berkeley DB and is currently in code review/regression testing. I have a patch that you can apply to Berkeley DB 4.6 and have confirmed that your test program runs with the patch applied. If you send me email at (Ron dot Cohen at Oracle) I’ll send the patch to you.
As you noticed, commiting the transaction will run cleanly without error. You could do that (with the suggestiion DB_TXN_NOSYNC below) but you may not even need transactions for this.
I want to expand a bit on my recommendation that you not abort transactions in the manner that you are doing (though with the patch you can certainly do that). First, the open/close database is a heavyweight operation. Typically you create/open your databases and keep them open for the life of the application (or a long time).
You also mentioned, that you noticed commits may have taken a longer time. We can talk about that (if you email me), but you could consider using the DB_TXN_NOSYNC flag losing durability. Make sure that this suggestion will work with your application requirements.
Even if you have (create/open/get/commit/abort) that should not need transactions for a single get operation. For that case, there would be no logging for the open and close therefore this sequence would be faster. This was a code snippet so what you have in your application may be a lot more complicated and justify what you have done. But the simple test case above should not require a transaction since you are doing a single atomic get.
I hope this helps!
Ron Cohen
Oracle Corporation