__db_tas_mutex_lock() ???
what's __db_tas_mutex_lock() mean?
after my program run fro servral days, it seems to stop doing anying with allmost 100% cpu usage.
i use gdb to attach it, and i get the following results:
$gdb --pid=22323
info thread
thread .... in __db_tas_mutex_lock() from /usr/lib64/libdb_cxx-4.8.so
and db_stat seems to print nothing unusual.
is that means a deadlock happend?
i configured my db using
dbenv->set_lk_detect(DB_SET_LOCK_OLDEST)
and i tried
$ db_deadlock -h envhome -a o
$ db_deadlock -h envhome
and nothing happended.
i have no idea what's wrong with it now. please someone help me.
thank you
Edited by: 863811 on 2011-6-6 上午2:27
what's __db_tas_mutex_lock() mean? __db_tas_mutex_lock is called when we need to acquire a lock on a mutex. Look it up in the BDB 4.8.26 tree; the method is in db-4.8.26/mutex/mut_tas.c, line 244. If you are using 4.8.26, please try to switch to the latest 4.8 patch release, which is 4.8.30; see the Berkeley DB downloads page.
after my program run fro servral days, it seems to stop doing anying with allmost 100% cpu usage.
i use gdb to attach it, and i get the following results:
$gdb --pid=22323
info thread
thread .... in __db_tas_mutex_lock() from /usr/lib64/libdb_cxx-4.8.so
and db_stat seems to print nothing unusual.
is that means a deadlock happend? Have you examined the locking statistics? Collect them using db_stat -Co or -CA and look for any deadlocks or potential self-deadlocks.
See the following section in the Berkeley DB Programmer's Reference Guide:
Deadlock detection, Deadlock debugging.
i configured my db using
dbenv->set_lk_detect(DB_SET_LOCK_OLDEST)
and i tried
$ db_deadlock -h envhome -a o
$ db_deadlock -h envhome
and nothing happended.
i have no idea what's wrong with it now. please someone help me.
thank you
Edited by: 863811 on 2011-6-6 上午2:27If you still this issue, attach to the process, get the stacks of the threads in the process, collect the locking statistics with db_stat -CA, the CPU utilization data using top, sar or mpstat (if on SMP - multiple CPU - system) and post them here. A short guide on finding out Linux CPU utilization is here.
If you need to limit the CPU usage per process, you could use cpulimit.
It would be useful if you could reproduce this with debug and diagnostic enabled libraries so that we have the process id and thread ids in the locking statistics.
Regards,
Andrei
Similar Messages
-
Possible deadlock: transactions, cursors, and sequences...
Hello,
After making recent changes to our database in response to a previous issue (bug found and patched in BDB; local code fixed by removing both DB_TXN_SNAPSHOT and DB_MULTIVERSION) we've come across a possible deadlock.
BACKGROUND:
This database has been running for over a year in production and I've never seen a hard deadlock. We use the default deadlock detection engine internal to BDB, transactions, and our code supports the processing of deadlocks and subsequent retries and final abandonment when necessary. The transactions in question involve cursors and sequences; we're using cursors to flip through entries in an existing database, and should no match be found for an update, we insert a new record. Before doing so, we grab the "next ID" (primary key) from a sequence we have (attached, as all sequences are, to its own different DB, done on advice from online docs: "For this reason, it is often preferable for sequence objects to be stored in their own database.") and finally insert the new record.
This is a 64-bit Linux machine. There were 4 operational threads at the time; all were waiting on pthread conditions. As I understand it, deadlocks should have been internally detected and returning DB_LOCK_DEADLOCK to us somewhere? Since we're using the default lock detection engine and firing it constantly, we should not require an Nth thread to monitor the lock tables and manually reject deadlocked transactions, etc?
I wasn't sure what to do when the deadlock occured, but I found some forum posts referencing a db_stat -Co and ran it, along with grabbing a core dump which I still have available. I've reverted the DB binary to older DB_MULTIVERSION code as I work on figuring this out, but if there's something else crucial I should have done, I can run the new code again and wait for another deadlock to happen to run additional diagnostics.
Any ideas or assistance is appreciated. Thank you.
DEADLOCK INFORMATION:
THREAD 2: Attempting to 'get' an asset.
(gdb) bt
#0 0x00002aec07e27496 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00002aec078ddeed in __db_pthread_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#2 0x00002aec078dda8b in __db_tas_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#3 0x00002aec0795b8f1 in __lock_get_internal () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#4 0x00002aec0795bc42 in __lock_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#5 0x00002aec07987d94 in __db_lget () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#6 0x00002aec07914625 in __ham_get_meta () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#7 0x00002aec07908d4b in __hamc_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#8 0x00002aec07979e8a in __dbc_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#9 0x00002aec0797aa0d in __dbc_pget () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#10 0x00002aec0798654b in __dbc_pget_pp () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#11 0x00002aec078d4dd7 in Dbc::get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
THREAD 5: Attempting to 'set' an asset.
(gdb) bt
#0 0x00002aec07e27496 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00002aec078ddeed in __db_pthread_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#2 0x00002aec078dda8b in __db_tas_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#3 0x00002aec079d0c45 in __seq_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#4 0x00002aec078dd02e in DbSequence::get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
THREAD 6: Attempting to 'set' an asset.
(gdb) bt
#0 0x00002aec07e27496 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00002aec078ddeed in __db_pthread_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#2 0x00002aec078dda8b in __db_tas_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#3 0x00002aec0795b8f1 in __lock_get_internal () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#4 0x00002aec0795bc42 in __lock_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#5 0x00002aec07987d94 in __db_lget () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#6 0x00002aec079893ee in __db_new () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#7 0x00002aec0798bb7e in __db_poff () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#8 0x00002aec07918eb3 in __ham_add_el () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#9 0x00002aec07907fe1 in __hamc_put () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#10 0x00002aec0797b6e5 in __dbc_put () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#11 0x00002aec0796ecde in __db_put () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#12 0x00002aec07985e6c in __db_put_pp () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#13 0x00002aec078d395c in Db::put () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
THREAD 7: Attempting to 'set' an asset.
(gdb) bt
#0 0x00002aec07e27496 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00002aec078ddeed in __db_pthread_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#2 0x00002aec078dda8b in __db_tas_mutex_lock () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#3 0x00002aec0795b8f1 in __lock_get_internal () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#4 0x00002aec0795bc42 in __lock_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#5 0x00002aec07987d94 in __db_lget () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#6 0x00002aec07915b6a in __ham_lock_bucket () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#7 0x00002aec07915dc7 in __ham_get_cpage () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#8 0x00002aec079075f9 in __ham_lookup () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#9 0x00002aec07908f9f in __hamc_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#10 0x00002aec07979e8a in __dbc_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#11 0x00002aec07984045 in __db_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#12 0x00002aec079d0374 in __seq_update () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#13 0x00002aec079d0c29 in __seq_get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
#14 0x00002aec078dd02e in DbSequence::get () from /usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock REGINFO information:
Lock Region type
5 Region ID
env/__db.005 Region name
0x2b8dbff65000 Original region address
0x2b8dbff65000 Region address
0x2b8dbff65138 Region primary address
0 Region maximum allocation
0 Region allocated
Region allocations: 225009 allocations, 0 failures, 0 frees, 1 longest
REGION_JOIN_OK Region flags
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by object:
Locker Mode Count Status ----------------- Object ---------------
808cfad0 READ 1 HELD db1 page 0
808cfad3 READ 2 HELD db1 page 0
808cfad5 READ 1 HELD db1 page 0
33 READ 1 HELD db1 handle 0
22 READ 1 HELD db2 handle 0
18 READ 1 HELD db3 handle 0
808cfad5 READ 1 HELD db1 page 3906
808cfad3 READ 1 HELD db1 page 6300
808cfad3 WRITE 1 HELD db1 page 6300
808cfad0 READ 1 HELD db1 page 8272
808cfabe WRITE 2 HELD db4 page 82387
808cfad3 WRITE 1 HELD db4 page 42879
808cfad3 WRITE 1 HELD db4 page 42878
808cfad3 WRITE 1 HELD db4 page 42877
808cfad3 WRITE 1 HELD db4 page 42874
808cfad3 WRITE 1 HELD db4 page 42873
808cfad3 WRITE 1 HELD db4 page 42872
808cfad3 WRITE 1 HELD db4 page 42901
808cfad3 WRITE 1 HELD db4 page 42897
808cfad3 WRITE 1 HELD db4 page 42882
808cfad3 WRITE 1 HELD db4 page 42881
808cfad3 WRITE 1 HELD db4 page 42880
808cfad3 WRITE 1 HELD db4 page 42894
808cfad3 WRITE 1 HELD db4 page 43797
1a READ 1 HELD db3sequence handle 0
1c READ 1 HELD db5 handle 0
20 READ 1 HELD db6 handle 0
24 READ 1 HELD db7 handle 0
808cfad3 READ 13 HELD db4 page 0
808cfabe READ 2 HELD db4 page 0
808cfabe WRITE 1 WAIT db4 page 0
808cfad5 READ 1 WAIT db4 page 0
26 READ 1 HELD db4 handle 0
808cfabe READ 2 HELD db4sequence page 0
808cfad0 READ 1 HELD db4sequence page 0
28 READ 1 HELD db4sequence handle 0
808cfabe READ 3 HELD db8 page 0
2a READ 1 HELD db8 handle 0
808cfabe READ 4 HELD db9 page 0
808cfad0 READ 1 HELD db9 page 0
808cfad3 READ 14 HELD db9 page 0
808cfad5 READ 1 HELD db9 page 0
808cfabe READ 1 HELD db4sequence page 2
808cfabe WRITE 1 HELD db4sequence page 2
808cfad0 READ 1 WAIT db4sequence page 2
2e READ 1 HELD db9 handle 0
808cfabe WRITE 2 HELD db9 page 1
808cfad3 READ 2 HELD db1sequence page 0
35 READ 1 HELD db1sequence handle 0
808cfad3 READ 1 HELD db1sequence page 2
808cfad3 WRITE 1 HELD db1sequence page 2
808cfad5 READ 1 HELD db9 page 2833
808cfad0 WRITE 1 HELD db9 page 7946
808cfad3 WRITE 14 HELD db9 page 8250
808cfabe WRITE 3 HELD db8 page 13301What else could be causing this in terms of the application having a resource locked? I can say that there are no other running threads doing anything related to BDB at all - they are all in similar "no work, sleep until we get some" calls, with the exception being the main thread which is sitting in sigwait(). What types of things could the application be doing that would prevent all 4 BDB threads from being able to obtain mutexes that are internal to them and not accessible to the application?
Other thoughts:
* On a fifth thread, from time to time, txn_checkpoint() is called. Could this have been left in an unclean state?
* If DB_RMW is used incorrectly, could lock order be compromised? We are not using CDS, so do we need to specify DB_WRITECURSOR to our db->cursor() calls? We do not; we only provide DB_RMW to pget() calls at present.
* Why is the thread attempting to call sequence->get also deadlocked? The sequence is in its own database - is it waiting on a more global "locker manager" mutex at a high level?
As I don't see how anything we do can directly control BDB's locking strategies, my only thought is that we're making a programming error to force BDB to lock things in an incorrect order in a way that prevents deadlock detection from occuring. Is this possible? Mainly the only thing changing here was our replacing DB_TXN_SNAPSHOT with the appropriate DB_RMW flags when needed, which is why I'm thinking we did something wrong here, but I'm not sure what.
I'll continue investigation, but any ideas you have in terms of appropriate directions would be helpful. I'll also work on reproducing this if I can by working backwards from the stack information. Thanks.
Later thought: Why is 808cfad3 not waiting on anything even though stack clearly shows it (thread 5, I'm guessing) in pthread_cond_wait? Can a transaction enter a wait state without showing up in db_stat output?
Thanks!
Edited by: user10542315 on Sep 11, 2009 1:21 PM
Edited by: user10542315 on Sep 11, 2009 2:52 PM -
Is there a mutex error that caused the AB-BA lock?
Hi,
I have a multiple-threaded program based on BerkeleyDB 4.8.30, and transaction environment,
it's coded by ANSI-C language, all threads are using the same DB_ENV variable.
some times the program encountered some errors that looked like
Thread A held mutex M1 and was waiting for mutex M2,
Thread B held mutex M2 and was waiting for mutex M1.
so these 2 threads were waiting for each other infinitely.
some back trace info:
Thread A
#0 0x00c78410 in __kernel_vsyscall ()
#1 0x00c52c05 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x080da6eb in __db_pthread_mutex_lock (env=0x9934e40, mutex=21)
at /harper/build/p1/lib/db-4.8.30/dist/../mutex/mut_pthread.c:318
#3 0x080d9e11 in __db_tas_mutex_lock_int (env=0x9934e40, mutex=21, nowait=0)
at /harper/build/p1/lib/db-4.8.30/dist/../mutex/mut_tas.c:218
#4 0x080d9ba4 in __db_tas_mutex_lock (env=0x9934e40, mutex=21)
at /harper/build/p1/lib/db-4.8.30/dist/../mutex/mut_tas.c:248
#5 0x080c1b55 in __memp_mf_sync (dbmp=0x99351d0, mfp=0xb5652c44, locked=0)
at /harper/build/p1/lib/db-4.8.30/dist/../mp/mp_sync.c:813
#6 0x080b431f in __memp_mf_discard (dbmp=0x99351d0, mfp=0xb5652c44)
at /harper/build/p1/lib/db-4.8.30/dist/../mp/mp_fopen.c:1000
#7 0x080b4016 in __memp_fclose (dbmfp=0xa6292e00, flags=0)
at /harper/build/p1/lib/db-4.8.30/dist/../mp/mp_fopen.c:925
#8 0x08186359 in __db_refresh (dbp=0xa69543b8, txn=0x0, flags=0, deferred_closep=0x92c1b4, reuse=0)
at /harper/build/p1/lib/db-4.8.30/dist/../db/db.c:962
#9 0x08185d37 in __db_close (dbp=0xa69543b8, txn=0x0, flags=0)
at /harper/build/p1/lib/db-4.8.30/dist/../db/db.c:695
#10 0x081a2041 in __db_close_pp (dbp=0xa69543b8, flags=0)
at /harper/build/p1/lib/db-4.8.30/dist/../db/db_iface.c:253
#11 0x0804eb9f in close_database (dbase=0xa68acef8)
at /harper/build/p1/usr.bin/hdb/dba.c:1737
Thread B
#0 0x00c78410 in __kernel_vsyscall ()
#1 0x00c52c05 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x080da6eb in __db_pthread_mutex_lock (env=0x9934e40, mutex=5)
at /harper/build/p1/lib/db-4.8.30/dist/../mutex/mut_pthread.c:318
#3 0x080d9e11 in __db_tas_mutex_lock_int (env=0x9934e40, mutex=5, nowait=0)
at /harper/build/p1/lib/db-4.8.30/dist/../mutex/mut_tas.c:218
#4 0x080d9ba4 in __db_tas_mutex_lock (env=0x9934e40, mutex=5)
at /harper/build/p1/lib/db-4.8.30/dist/../mutex/mut_tas.c:248
#5 0x080b6b16 in __memp_nameop (env=0x9934e40, fileid=0xa626fa8c "\253\001q\020\v", <incomplete sequence \375>,
newname=0x26eed8c "/usr/store/ilanrg/var.db",
fullold=0xa641da30 "/usr/store/ilanrg/801e9017.7ca2518",
fullnew=0xa60f1920 "/usr/store/ilanrg/var.db", inmem=0)
at /harper/build/p1/lib/db-4.8.30/dist/../mp/mp_method.c:722
#6 0x0809b498 in __fop_rename (env=0x9934e40, txn=0xa6204588,
oldname=0xa69aeb98 "/usr/store/ilanrg/801e9017.7ca2518",
newname=0x26eed8c "/usr/store/ilanrg/var.db", dirp=0xa626fa84,
fid=0xa626fa8c "\253\001q\020\v", <incomplete sequence \375>, appname=DB_APP_DATA, with_undo=1, flags=0)
at /harper/build/p1/lib/db-4.8.30/dist/../fileops/fop_basic.c:312
#7 0x0809dd52 in __fop_file_setup (dbp=0xa626fa50, ip=0x0, txn=0xa69a27d8,
name=0x26eed8c "/usr/store/ilanrg/var.db", mode=420, flags=17, retidp=0x26ee678)
at /harper/build/p1/lib/db-4.8.30/dist/../fileops/fop_util.c:609
#8 0x08077adc in __db_open (dbp=0xa626fa50, ip=0x0, txn=0xa69a27d8,
fname=0x26eed8c "/usr/store/ilanrg/var.db", dname=0x0, type=DB_BTREE, flags=17, mode=420, meta_pgno=0)
at /harper/build/p1/lib/db-4.8.30/dist/../db/db_open.c:177
#9 0x081a3fd6 in __db_open_pp (dbp=0xa626fa50, txn=0xa69a27d8, fname=0x26eed8c "/usr/store/ilanrg/var.db",
dname=0x0, type=DB_BTREE, flags=17, mode=420)
at /harper/build/p1/lib/db-4.8.30/dist/../db/db_iface.c:1147
#10 0x0804febe in open_database (path=0x26eed8c "/usr/store/ilanrg/var.db", create=273, mode=420,
res=0xa58aee00, am=DB_BTREE, truncate=0) at /harper/build/p1/usr.bin/hdb/dba.c:1633
Thread A was performing DB's close, while Thread B was performaing DB's open.
Regarding some code inspection about these functions: __memp_mf_discard, __memp_mf_sync and __memp_nameop
seemed Thread A held the region lock related to ((REGINFO *) 0x9935218) and trying to acquire to lock related to ((DB_MPOOL_HASH *) 0xb5500510)
while thread B held the lock related to ((DB_MPOOL_HASH *) 0xb5500510), and trying to acquire to region lock related to ((REGINFO *) 0x9935218)
Have you encountered the similar issues?Thanks Mike, I'm waiting for the newer version of these docs.
On the other hand, do you prefer the modifications in the source code file: db-4.8.30 (or latest BDB version)/mp/mp_method.c, procedure __memp_nameop:
nearby the label 'err:' the original implement is as follow:
/* Delete the memory we no longer need. */
err: if (p != NULL) {
MPOOL_REGION_LOCK(env, &dbmp->reginfo[0]);
__memp_free(&dbmp->reginfo[0], p);
MPOOL_REGION_UNLOCK(env, &dbmp->reginfo[0]);
/* If we have buckets locked, unlock them when done moving files. */
if (locked == 1) {
MUTEX_UNLOCK(env, hp->mtx_hash);
if (nhp != NULL && nhp != hp)
MUTEX_UNLOCK(env, nhp->mtx_hash);
and the code modifications is below:
err:
/* If we have buckets locked, unlock them when done moving files. */
if (locked == 1) {
MUTEX_UNLOCK(env, hp->mtx_hash); //Unlock here!
if (nhp != NULL && nhp != hp)
MUTEX_UNLOCK(env, nhp->mtx_hash);
if (p != NULL) {
MPOOL_REGION_LOCK(env, &dbmp->reginfo[0]);
__memp_free(&dbmp->reginfo[0], p);
MPOOL_REGION_UNLOCK(env, &dbmp->reginfo[0]);
as you saw, the modification unlocked the mutex 'hp->mtx_hash' first, then was trying to lock the mutex &dbmp->reginfo[0],
and seems it has been working on my systems for close to a year.
How about this modification?
Thanks,
Harper -
BDB is deadlock , env init :
set_lk_detect isn't action. Why??
/* Open and configure an environment. */
int env_init(DB_ENV dbenv, const char home)
u_int32_t flags;
int ret;
ret = dbenv->set_cachesize(dbenv, 0, M_DB_ENV_CACHESIZE, 0);
ret |= dbenv->set_flags(dbenv, DB_TXN_NOSYNC, 1);
ret |= dbenv->set_flags(dbenv, DB_AUTO_COMMIT, 1);
ret |= dbenv->set_flags(dbenv, DB_LOG_INMEMORY, 1);
ret |= dbenv->set_lg_bsize(dbenv, 512*1024);
ret |= dbenv->set_timeout(dbenv, 1*1000*1000, DB_SET_LOCK_TIMEOUT);
ret |= dbenv->set_lk_detect(dbenv, DB_LOCK_MINLOCKS);
flags = DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL |
DB_INIT_REP | DB_INIT_TXN | DB_RECOVER | DB_THREAD|DB_PRIVATE;
if ((ret = dbenv->open(dbenv, home, flags, 0)) != 0)
dbenv->err(dbenv, ret, "can't open environment");
return (ret);
db_stat "-C co"
env_lock_stat
db_stat "-C co"
show :
8000360b READ 1 HELD /ram/uplinkhis15.db.db page 1
8000360d WRITE 1 WAIT /ram/uplinkhis15.db.db page 1
8000360e READ 1 WAIT /ram/uplinkhis15.db.db page 1
8000360d WRITE 3 HELD /ram/uplinkhis15.db.db page 2
8000360b READ 1 PENDING /ram/uplinkhis15.db.db page 46
vxWorks-Shell#tt perf
4854f0 vxTaskEntry +68 : ipcModuleRecvTaskEntry (3)
683b04 ipcModuleRecvTaskEntry+190: perfCoreIpcCallback (909a4c0)
25d550 perfCoreIpcCallback+dc : perf15mHandAll ()
25d654 perf15mHandAll +18 : perfUplink15mHandle ()
25d850 perfUplink15mHandle+190: DBApiCondCount (2007, 8, b835a80, b83572c)
2c0650 DBApiCondCount +1c4: __db_c_get_pp (be7f8e8, b834c20, b834c40, 1b)
3dd950 __db_c_get_pp +12c: __db_c_get (be7f8e8, b834c20, b834c40, 1b)
3cdc3c __db_c_get +6e0: 425828 ()
425828 __bam_c_dup +ab0: 429e68 (90a5280, 0, b834c20, 1b, b834ad0)
42a9b8 __bam_c_rget +19b4: __bam_search (90a5280)
3a2e84 __bam_search +120: __bam_get_root (90a5280, 1, 1, 101, b8349c4)
3a28c0 __bam_get_root +108: __db_lget (90a5280, 0, 1, 1, 0, b834900)
3e3714 __db_lget +3cc: __lock_get (9971970, 8000360e, 0, 90a52f8, 1, b83490
0)
3102d0 __lock_get +e0 : __lock_get_internal (97d8930, 8000360e, 0, 90a52f8,
1, f4240, b834900)
3117e4 __lock_get_internal+149c: __db_tas_mutex_lock (9971970, 349)
348808 __db_tas_mutex_lock+2fc: __os_sleep (9971970, 0, 2710)
34be78 __os_sleep +ec : select ()
value = 0 = 0x0
vxWorks-Shell#tt CLI_T
4854f0 vxTaskEntry +68 : cli_telnet_thread ()
665abc cli_telnet_thread+7c : thread_call (90e9ef8)
66bfd8 thread_call +4c : 671568 ()
671568 vty_read +450: 670f24 (be878c8)
670f9c vty_clear_buf +4e0: vty_command (be878c8, be847b0)
66ebec vty_command +54 : cmd_execute_command (be773e0, be878c8, 0, 0)
6614a0 cmd_execute_command+130: 660ec0 (be773e0, be878c8, 0)
66134c cmd_complete_command+570: 6bbb04 (&no_perf_uplink_if_cmd, be878c8, 1, 90e
9ab8)
6bbb6c perfCheckSampleTime+24e8: perfDisableUplink (1010000)
25e5ac perfDisableUplink+c0 : perfDeleteUplinkIf (1, 1)
260e90 perfDeleteUplinkIf+21c: DBApiDelSingle (2007, 90e9758)
2bef74 DBApiDelSingle +178: __db_del_pp (970ddb0, 90b8690, 90e89c8, 0)
3d9ad4 __db_del_pp +284: __db_del (970ddb0, 90b8690, 90e89c8, 0)
3c2048 __db_del +308: __db_c_close (90911b8)
3cc73c __db_c_close +238: 423b84 ()
423b84 __bam_c_refresh+7a8: 42ad00 (90911b8)
42b0c4 __bam_c_rget +20c0: __bam_search (90911b8)
3a2e84 __bam_search +120: __bam_get_root (90911b8, 1, 0, 40000, 90e877c)
3a28c0 __bam_get_root +108: __db_lget (90911b8, 0, 1, 2, 0, 90e86b8)
3e3714 __db_lget +3cc: __lock_get (9971970, 8000360d, 0, 9091230, 2, 90e86b
8)
3102d0 __lock_get +e0 : __lock_get_internal (97d8930, 8000360d, 0, 9091230,
2, f4240, 90e86b8)
3117e4 __lock_get_internal+149c: __db_tas_mutex_lock (9971970, 347)
348808 __db_tas_mutex_lock+2fc: __os_sleep (9971970, 0, 2710)
34be78 __os_sleep +ec : select ()
b41d0 select +168: semTake ()
f0e88 semTake +13c: semBTake ()
value = 0 = 0x0
vxWorks-Shell#tt tSnmpd
4854f0 vxTaskEntry +68 : e1adc (eeeeeeee)
e1b10 snmpdInit +1a4: snmpIoMain ()
e31a8 snmpIoMain +b8 : e31c4 ()
e3324 snmpIoMain +234: snmpdPktProcess ()
e1be0 snmpdPktProcess+28 : Process_Rcvd_SNMP_Packet_Async ()
48b4a4 Process_Rcvd_SNMP_Packet_Async+e0 : 48b0a4 ()
48b3a8 process_packet_two+3b4: process_packet_two ()
48b060 process_packet_two+6c : SNMP_Process_Next_PDU ()
490bf8 SNMP_Process_Next_PDU+324: eponPerfHis15minUplinkEntry_next (3, 3, 90b838
c, 9091450, 9091340)
45e210 eponPerfHis15minUplinkEntry_next+50 : eponPerfHis15minUplinkEntry_int_nex
t (3, 90b838c, be91860)
47bb38 eponPerfHis15minUplinkEntry_int_next+238: perfGetUplinkHistory15StatNext
(be91860)
6b6a80 perfGetUplinkHistory15StatNext+28 : DBApiQueryNext (2007, be91860, be9186
0)
2bfb9c DBApiQueryNext +1ec: __db_c_get_pp (970d7b0, be90cf0, be90d10, 19)
3dd950 __db_c_get_pp +12c: __db_c_get (970d7b0, be90cf0, be90d10, 19)
3cdc3c __db_c_get +6e0: 4257e8 ()
4257e8 __bam_c_dup +a70: 429e68 (90bd268, 0, be90cf0, 19, be90ba0)
42a9b8 __bam_c_rget +19b4: __bam_search (90bd268)
3a2e84 __bam_search +120: __bam_get_root (90bd268, 1, 1, 581, be90a94)
3a28c0 __bam_get_root +108: __db_lget (90bd268, 0, 1, 1, 0, be909d0)
3e3714 __db_lget +3cc: __lock_get (9971970, 8000360f, 0, 90bd2e0, 1, be909d
0)
3102d0 __lock_get +e0 : __lock_get_internal (97d8930, 8000360f, 0, 90bd2e0,
1, f4240, be909d0)
3117e4 __lock_get_internal+149c: __db_tas_mutex_lock (9971970, 344)
value = 0 = 0x0
vxWorks-Shell#
vxWorks-Shell#
vxWorks-Shell#
vxWorks-Shell#env_lock_stat ---->lock_stat_print
182 Last allocated locker ID
0x7fffffff Current maximum unused locker ID
9 Number of lock modes
1000 Maximum number of locks possible
1000 Maximum number of lockers possible
1000 Maximum number of lock objects possible
76 Number of current locks
82 Maximum number of locks at any one time
185 Number of current lockers
185 Maximum number of lockers at any one time
74 Number of current lock objects
81 Maximum number of lock objects at any one time
37040 Total number of locks requested
36961 Total number of locks released
0 Total number of locks upgraded
72 Total number of locks downgraded
72 Lock requests not available due to conflicts, for which we waited
1 Lock requests not available due to conflicts, for which we did not wait
0 Number of deadlocks
1000000 Lock timeout value
0 Number of locks that have timed out
0 Transaction timeout value
0 Number of transactions that have timed out
344KB The size of the lock region
29 The number of region locks that required waiting (0%)
value = 0 = 0x0
vxWorks-Shell#Hello,
There are three basic ways to perform deadlock detection.
http://www.oracle.com/technology/documentation/berkeley-db/db/ref/lock/dead.html
http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/env_set_lk_detect.html
http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/lock_detect.html
http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/env_set_timeout.html
1. use the DbEnv::set_lk_detect method to automatically run the
deadlock detector whenever there is a conflict over a lock. With
this method you specify what lock request(s) should be rejected.
With this choice the lock table will be walked as soon as a lock
request blocks) hence an application should not have to wait
on a lock before discovering that deadlock occurred.
2. use the DbEnv::lock_detect method which runs a single iteration
of the BDB deadlock detector. In this case use a thread or external
process to perform deadlock detection. Your application will run
deadlock detection at some interval. The application may have to
wait to be notified of a deadlock, but you will not incur the
overhead of walking the lock table every time a lock request is blocked.
3. use DBenv::set_timeout to configure the locking subsystem to time out
any lock not released within a specified amount of time. Note that lock
timeouts are only checked when a lock request is blocked or when deadlock
detection is otherwise performed. Therefore, a lock can have timed out
and still be held for some length of time until BDB has a reason to
examine its locking tables.
So for your question on DbEnv::lock_detect, yes this needs to be
run in a thread or external process after some specific time interval,
like a few seconds to a minute.
For your initial question, please clarify in a bit more detail what
the problem is. The output from db_stat -Co does not look to show a
deadlock.
Thanks,
Sandra -
Opening a db environment and preventing corruption?
Greetings,
Sorry for the long post, but I’m having some problems and I’m not sure where to even begin. I also have a deadline looming and many people looking to me for fixes.
I’m working on a project and we’re trying to use the BDB for looking up data (read only tables). Things run fine when testing one process, but in production there will need to be 20 or more processes, and we cannot keep any of them running very long due to environment file corruption. Also, the processes do have some known memory leaks and crash occasionally, this problem cannot be corrected at this time and must be worked around (currently the processes are all terminated and restarted nightly).
1. How should the environment be set up / opened per process? Currently each process has code that calculates the environment cache size (based on calculations from the docs) and opens the databases (currently 15 .db files). Is this the correct method? Should every process perform all the environment settings, or should an existing environment be checked for first somehow? Every process has code that calls:
db_env_create()
set_cachesize(<size calculated to be large enough for all .db files>)
open(DB_CREATE | DB_THREAD | DB_INIT_MPOOL)
Each database is opened with DB_RDONLY | DB_THREAD options.
2. The environment cache size is calculated to be large enough to hold all the .db files in memory (currently about 1.08G), will that RAM be shared between processes, or does each process need the physical RAM available?
3. I have noticed that processes compiled with debugging symbols modify the environment such that when you try to run a program compiled with optimizations (and no debugging symbols), the program will hang trying to open the environment. Is this to be expected?
4. Once the environment files are created (currently any process that starts will create the __db.* files if they do not exist), it is possible to mark them read-only in an attempt to prevent corruption? We are only using the database files for read-only access.
Once the environment files are corrupt, there are generally two types of problems we encounter. One is when a process is trying to start and fails because the environment or a database cannot be opened:
In bdbdb_open(), DB->open(bdb_ru_rule.db) failed: Not enough space
unable to allocate memory for mutex; resize mutex region
The other problem is that a currently running process will just hang. This is a stack trace:
(dbx) where
current thread: t@1
[1] ___lwp_mutex_timedlock(0xf5a36028, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfe241314
[2] __db_pthread_mutex_lock(0x10bd08, 0x2d798, 0x2118000, 0x1c8, 0x10c190, 0x0), at 0xfdd11fac
[3] __db_tas_mutex_lock(0xf6de0000, 0x2d798, 0xf5a36050, 0x4000, 0xafc00000, 0xb02806c0), at 0xfdd11d14
[4] __memp_fget(0x0, 0xffbfd4d0, 0x0, 0x34036000, 0xced56bc0, 0x0), at 0xfde3f7ec
[5] __bam_get_root(0x10ccb8, 0x0, 0x1, 0x0, 0xffbfd5b8, 0x0), at 0xfdd2c318
[6] __bam_search(0x10ccb8, 0x0, 0x10ce24, 0x10ce24, 0x1, 0x0), at 0xfdd2c8c4
[7] __bamc_search(0x10ccb8, 0x0, 0xffbfd8e8, 0x1a, 0xffbfd6a4, 0x13), at 0xfdd1a420
[8] __bamc_get(0x10ccb8, 0xffbfd8e8, 0xffbfd8cc, 0x1a, 0xffbfd758, 0xfdd15b08), at 0xfdd16104
[9] __dbc_get(0x10ccb8, 0xffbfd8e8, 0xffbfd8cc, 0x1a, 0x14, 0x0), at 0xfdde145c
[10] __db_get(0x10c900, 0x50000000, 0xffbfd8e8, 0xffbfd8cc, 0x1a, 0x10c938), at 0xfddee648
[11] __db_get_pp(0x10bd08, 0x0, 0xffbfd8e8, 0xffbfd8cc, 0x8000000, 0x10c880), at 0xfddee38c
=>[12] bdb_lookup(i_bdb = 1, vp_key = 0xffbfd9a0, vp_data = 0xffbfd98c), line 334 in "bdb_base.c"
Any insights into these problems would be greatly appreciated!
Thanks,
MatthewHi Matthew,
I will try to answer your questions.
1. How should the environment be set up / opened
per process?
As documented in the flags section here:
http://www.oracle.com/technology/documentation/berkele
y-db/db/api_c/env_open.html
You should open the process you want to create the
environment with all of the configuration settings.
Then each subsequent environment open should be done
with no flags set.
Is there a way to detect if the environment is already opened so the first process that starts can open the environment with flags?
What about the settings done with function calls between env->create() and env->open(), like setting the cache size? Should those be skipped as well if another process already has the environment open?
What if the __db* files already exist, then does the first process still need to open the environment with flags set, or are the environment's settings stored in the __db* files directly?
3. I have noticed that processes compiled with
debugging symbols modify the environment such that
when you try to run a program compiled with
optimizations (and no debugging symbols), the program
will hang trying to open the environment. Is this to
be expected?
There is a slight distinction here. There are two
different flags (called slightly different things
depending on your platform).
Environments are not compatible if they have
different diagnostic settings. Diagnostic mode
(--enable-diagnostic using configure, or DIAGNOSTIC
preprocessor define in a Visual Studio project).
You can do a debug build without diagnostics enabled.
On systems using configure, just build without
--enable-diagnostic. On Windows, alter the Debug
project build files to remove the DIAGNOSTIC
pre-processor define.
Sorry for not being more clear, I was referring to the application I'm writing, not compiling the BDB. For example, if I compile with -g (Solaris CC) and run the process, then recompile with -xO4 (and remove -g), then the process will not be able to open the environment and will either terminate with an error or hang in the mutex lock.
>
5. In bdbdb_open(), DB->open(bdb_ru_rule.db)
failed: Not enough space
unable to allocate memory for mutex; resize mutex
region
If you are encountering this problem, you should read
the reference guide section on configuring Locking.
http://www.oracle.com/technology/documentation/berkele
y-db/db/ref/lock/max.html
6. [2] __db_pthread_mutex_lock(0x10bd08, 0x2d798,
0x2118000, 0x1c8, 0x10c190, 0x0), at 0xfdd11fac
Are you running deadlock detection? Is it possible
that a process has died while holding locks open on
the database?
I currently do not have deadlock detection enabled for two reasons:
1. All the databases are opened as read-only, so why is any locking happeneing? In the C-API docs for env->open(), the DB_INIT_LOCK description specifically says: "If all threads are accessing the database(s) read-only, locking is unnecessary."
2. When I find processes that have failed waiting on some mutex, running db_stat does not indicate that any processes are deadlocked and running db_deadlock hangs.
Thanks,
Matthew -
Concurrent Data Store (CDB) application hangs when it shouldn't
My application hangs when trying to open a concurrent data store (CDB) database for reading:
#0 0x0000003ad860b309 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib64/libpthread.so.0
#1 0x00007ffff7ce67de in __db_pthread_mutex_lock (env=0x610960, mutex=100)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../mutex/mut_pthread.c:318
#2 0x00007ffff7ce5ea5 in __db_tas_mutex_lock_int (env=0x610960, mutex=100,
nowait=0)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../mutex/mut_tas.c:218
#3 0x00007ffff7ce5c43 in __db_tas_mutex_lock (env=0x610960, mutex=100)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../mutex/mut_tas.c:248
#4 0x00007ffff7d3715b in __lock_id (env=0x610960, idp=0x0, lkp=0x610e88)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../lock/lock_id.c:68
#5 0x00007ffff7da1b4d in __fop_file_setup (dbp=0x610df0, ip=0x0, txn=0x0,
name=0x40b050 "registry.db", mode=0, flags=1024, retidp=0x7fffffffdd94)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../fileops/fop_util.c:243
#6 0x00007ffff7d70c8e in __db_open (dbp=0x610df0, ip=0x0, txn=0x0,
fname=0x40b050 "registry.db", dname=0x0, type=DB_BTREE, flags=1024,
mode=0, meta_pgno=0)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../db/db_open.c:176
#7 0x00007ffff7d673b2 in __db_open_pp (dbp=0x610df0, txn=0x0,
fname=0x40b050 "registry.db", dname=0x0, type=DB_BTREE, flags=1024, mode=0)
at /home/steve/ldm/package/src/Berkeley-DB/dist/../db/db_iface.c:1146
I suspect that the database environment believes that another process has the database open for writing. This cannot be the case, however, as all applications that access the database do so via an interface library I wrote that registers a termination function via the atexit() system-call to ensure that both the DB and DB_ENV handles are properly closed -- and all previously-executed applications terminated normally.
The interface library opens the database like this (apparently, this forum doesn't support indentation, sorry):
int status;
Backend* backend = (Backend*)malloc(sizeof(Backend));
if (NULL == backend) {
else {
DB_ENV* env;
if (status = db_env_create(&env, 0)) {
else {
if (status = env->open(env, path,
DB_CREATE | DB_INIT_CDB | DB_INIT_MPOOL, 0)) {
else {
DB* db;
if (status = db_create(&db, env, 0)) {
else {
if (status = db->open(db, NULL, DB_FILENAME, NULL,
DB_BTREE, forWriting ? DB_CREATE : DB_RDONLY, 0)) {
else {
backend->db = db;
} /* "db" opened */
if (status)
db->close(db, 0);
} /* "db" allocated */
if (status) {
env->close(env, 0);
env = NULL;
} /* "env" opened */
if (status && NULL != env)
env->close(env, 0);
} /* "env" allocated */
if (status)
free(backend);
} /* "backend" allocated */
This code encounters no errors.
The interface library also registers the following code to be executed when any process that uses the interface library exits:
if (NULL != backend) {
DB* db = backend->db;
DB_ENV* env = db->get_env(db);
if (db->close(db, 0)) {
else {
if (env->close(env, 0)) {
else {
/* database properly closed */
As I indicated, all previously-executed processes that use the interface library terminated normally.
I'm using version 4.8.24.NC of Berkeley DB on the following platform:
$ uname -a
Linux gilda.unidata.ucar.edu 2.6.27.41-170.2.117.fc10.x86_64 #1 SMP Thu Dec 10 10:36:29 EST 2009 x86_64 x86_64 x86_64 GNU/Linux
Any ideas?Bogdan,
That can't be it. I'm using a structured programming style in which the successful initialization of a cursor is ultimately followed by a closing of the cursor. There's only one place where the code does this and it's obvious that the cursor gets released.
I've also read the CDB section.
--Steve Emmerson -
Db- verify of standalone database file
I need help with database verification for a 4.5.20 CDS application running on VxWorks.
I create an environment, and create ~30 databases within that environment, which all exist within one single file. I recently discovered that since I create a set of database file handles at startup, which are shared between several VxWorks tasks, that I need to specify the DB_THREAD flag to both the environment open, and database open calls. This works great and our concurrency problems were eliminated.
The environment is configured with the following flags:
(DB_INIT_CDB | DB_INIT_MPOOL | DB_CREATE | DB_SYSTEM_MEM | DB_THREAD)
However, now my database verifcation is no longer working correctly.
Here is my original code, where I create the verification database handle within the database environment which I later open the databases within:
if ((ret = db_create(&dbp_verify, dbenv_starchoice, 0)) != 0)
{goto err;}
if ((ret = dbp_verify->verify(dbp_verify, DB_STARCHOICE_FILENAME, NULL, NULL, 0)) != 0)
if ((ret = db_destroy()) != 0)
{goto err;}
Until I set the DB_THREAD flag for the environment and databases, this code worked just fine. However, with DB_THREAD flag set the db->verify routine NEVER returns.
So, I modified the code so that I create the dbp_verify database handle outside of the database environment by setting the second parameter of db_create to NULL. Now when the verification runs I get the following error:
S_iosLib_DRIVER_GLUT
I suspect that this is because it has no way to know where to create the temporary files needed for the verify, where previously it used the "temp" directory I specified when configuring the database environment.
Can anyone tell me how to correctly configure the database handle I use for the verifcation, so that I can verify it as a standalone? Or, tell me why when the environment is configured with DB_THREAD the verification never returns??
Thanks!I have used two different Berkeley builds. For the first one, db->verify had not returned after having let my box run overnight. When I did a task trace in the morning on the task that had called db->verify, I got the following:
0x80053670 vxTaskEntry +0x14 : Neptune::OsTask::entryHelper(Neptune::OsTask*) (
0x830f6a34 Neptune::OsTask::entryHelper(Neptune::OsTask*)+0x80 : Neptune::OsHelp
erTask::entry() ()
0x830f4a10 Neptune::OsHelperTask::entry()+0x40 : OpenTVTask::entry() ()
0x83defba8 OpenTVTask::entry()+0x80 : o_db_vsk_init ()
0x83e3df08 o_db_vsk_init+0x14c: db_startup ()
0x83e1bfac db_startup +0x44 : db_database_init ()
0x83e1a1a0 db_database_init+0x94 : 0x83a28a18 ()
0x83a28a40 __db_verify_pp+0x20 : __db_verify_internal ()
0x83a28b40 __db_verify_internal+0xf4 : 0x83a28ca8 ()
0x83a28f48 __db_verify_internal+0x4fc: 0x83a299fc ()
0x83a29b18 __db_verify_internal+0x10cc: 0x83a2a7bc ()
0x83a2a9b8 __db_vrfy_meta+0x6c0: __bam_vrfy_structure ()
0x839fdf38 __bam_vrfy_structure+0x1a0: __bam_vrfy_subtree ()
0x839fe918 __bam_vrfy_subtree+0x870: __bam_vrfy_subtree ()
0x839fe320 __bam_vrfy_subtree+0x278: __db_vrfy_ovfl_structure ()
0x83a2d9c4 __db_vrfy_ovfl_structure+0x138: __db_vrfy_pgset_inc ()
0x83a2cde4 __db_vrfy_pgset_inc+0xc4 : __db_put ()
0x83a058d0 __db_put +0x40 : __db_cursor ()
0x83a13360 __db_cursor +0x124: __lock_get ()
0x83a59c98 __lock_get +0xa0 : __lock_get_internal ()
0x83a5a84c __lock_get_internal+0xb5c: __db_tas_mutex_lock ()
0x83a7b5d8 __db_tas_mutex_lock+0x114: 0x83a7d994 ()
0x83a7da50 __os_sleep +0xb0 : select ()
0x80110ef0 select +0x578: semTake ()
0x801996b4 semTake +0x160: semBTake ()
value = 0 = 0x0
Then we discovered there was a known bug in VxWorks due to the use of select(), which is used by os_sleep(). So, one of our developers modified os_sleep() based on both Wind River's and Oracle's recommendations so that os_sleep() does not use select.
When I used this new build, db->verify still does not return, but when I do a stack trace on the task I don't get anything. tt simply returns value = 0 = 0x0. It is like the task just stopped running though it's status is READY, not PEND.
Does this information help? -
Hello,
We have experienced a kind a deadlock, which could only be solved by a "db_recover". IMHO this
happens by a parallel insert. We always insert after the last value (database already open):
- environment->txn_begin( DNV, NULL, &transaction, DB_TXN_WRITE_NOSYNC );
- database->cursor( database, transaction, &cursor, 0 );
- cursor->c_get( cursor, &last_key, &unused, DB_LAST );
- cursor->c_close( cursor );
- database->put( database, transaction, &key, &data, DB_NOOVERWRITE );
- transaction->commit( transaction, 0 );
During inserts we have quite much of reads (all DB_READ_UNCOMMITTED).
Once a minute a checkpoint writes the dirty data to the database (a mutex prevents
that more than one checkpoint is called at any time).
Sorry, I forgot to make a coredump, so we have only the glibc `backtrace()`.
Thanks
================================================================================
[0xb80d1400]
[0xb80d1424]
/lib/tls/i686/cmov/libc.so.6(__sched_yield+0xc) [0xb775d9dc]
/lib/tls/i686/cmov/libpthread.so.0(pthread_yield+0x14) [0xb7bcde84]
/usr/local/lib/libdb-4.8.so(__os_yield+0x81) [0xb808d731]
/usr/local/lib/libdb-4.8.so [0xb7f2090b]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7f206da]
/usr/local/lib/libdb-4.8.so(__memp_fget+0x9cf) [0xb807478b]
/usr/local/lib/libdb-4.8.so(__bam_search+0x15a0) [0xb7f49995]
/usr/local/lib/libdb-4.8.so [0xb7f31acb]
/usr/local/lib/libdb-4.8.so [0xb7f2f85e]
/usr/local/lib/libdb-4.8.so(__dbc_iput+0x261) [0xb801008e]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x198) [0xb800fe20]
/usr/local/lib/libdb-4.8.so [0xb800fae9]
/usr/local/lib/libdb-4.8.so [0xb800eb93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb800fdac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7ffdbe1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb801e25c]
================================================================================
[0xb7faf400]
[0xb7faf422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7aaf0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7dff27d]
/usr/local/lib/libdb-4.8.so [0xb7dfe951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7dfe6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7ec3b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7ec2246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7f027e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7e25de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7e264e0]
/usr/local/lib/libdb-4.8.so [0xb7e0facb]
/usr/local/lib/libdb-4.8.so [0xb7e0b031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7eec1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7eebc83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7eeedd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7efe227]
================================================================================
[0xb7ff5400]
[0xb7ff5422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7af50e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7e4527d]
/usr/local/lib/libdb-4.8.so [0xb7e44951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7e446da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7f09b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7f08246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7f487e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7e6bde5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7e6c4e0]
/usr/local/lib/libdb-4.8.so [0xb7e55acb]
/usr/local/lib/libdb-4.8.so [0xb7e51031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7f321d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7f31c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7f34dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7f44227]
================================================================================
[0xb80ca400]
[0xb80ca422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7bca0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7f1a27d]
/usr/local/lib/libdb-4.8.so [0xb7f19951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7f196da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7fdeb2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7fdd246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb801d7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7f40de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7f414e0]
/usr/local/lib/libdb-4.8.so [0xb7f2aacb]
/usr/local/lib/libdb-4.8.so [0xb7f26031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb80071d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb8006c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb8009dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb8019227]
================================================================================
[0xb7f0e400]
[0xb7f0e422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a0e0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7d5e27d]
/usr/local/lib/libdb-4.8.so [0xb7d5d951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7d5d6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e22b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e21246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7e617e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7d84de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7d854e0]
/usr/local/lib/libdb-4.8.so [0xb7d6eacb]
/usr/local/lib/libdb-4.8.so [0xb7d6a031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7e4b1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7e4ac83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7e4ddd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7e5d227]
================================================================================
[0xb7f21400]
[0xb7f21422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a210e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7d7127d]
/usr/local/lib/libdb-4.8.so [0xb7d70951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7d706da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e35b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e34246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7e747e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7d97de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7d984e0]
/usr/local/lib/libdb-4.8.so [0xb7d81acb]
/usr/local/lib/libdb-4.8.so [0xb7d7d031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7e5e1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7e5dc83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7e60dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7e70227]
================================================================================
[0xb804d400]
[0xb804d422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7b4d0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7e9d27d]
/usr/local/lib/libdb-4.8.so [0xb7e9c951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7e9c6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7f61b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7f60246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7fa07e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x530) [0xb7ec4200]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7ec44e0]
/usr/local/lib/libdb-4.8.so [0xb7eadacb]
/usr/local/lib/libdb-4.8.so [0xb7ea9031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7f8a1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7f89c83]
/usr/local/lib/libdb-4.8.so [0xb7f8b97a]
/usr/local/lib/libdb-4.8.so [0xb7f8ab93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb7f8bdac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7f79be1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb7f9a25c]
================================================================================
[0xb8020400]
[0xb8020422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7b200e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7e7027d]
/usr/local/lib/libdb-4.8.so [0xb7e6f951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7e6f6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7f34b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7f33246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7f737e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7e96de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7e974e0]
/usr/local/lib/libdb-4.8.so [0xb7e80acb]
/usr/local/lib/libdb-4.8.so [0xb7e7c031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7f5d1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7f5cc83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7f5fdd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7f6f227]
================================================================================
[0xb8054400]
[0xb8054422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7b540e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7ea427d]
/usr/local/lib/libdb-4.8.so [0xb7ea3951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7ea36da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7f68b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7f67246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7fa77e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7ecade5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7ecb4e0]
/usr/local/lib/libdb-4.8.so [0xb7eb4acb]
/usr/local/lib/libdb-4.8.so [0xb7eb0031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7f911d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7f90c83]
/usr/local/lib/libdb-4.8.so [0xb7f9297a]
/usr/local/lib/libdb-4.8.so [0xb7f91b93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb7f92dac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7f80be1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb7fa125c]
================================================================================
[0xb7f79400]
[0xb7f79422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a790e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7dc927d]
/usr/local/lib/libdb-4.8.so [0xb7dc8951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7dc86da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e8db2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e8c246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7ecc7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7defde5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7df04e0]
/usr/local/lib/libdb-4.8.so [0xb7dd9acb]
/usr/local/lib/libdb-4.8.so [0xb7dd5031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7eb61d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7eb5c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7eb8dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7ec8227]
================================================================================
[0xb7fba400]
[0xb7fba422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7aba0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7e0a27d]
/usr/local/lib/libdb-4.8.so [0xb7e09951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7e096da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7eceb2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7ecd246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7f0d7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7e30de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7e314e0]
/usr/local/lib/libdb-4.8.so [0xb7e1aacb]
/usr/local/lib/libdb-4.8.so [0xb7e16031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7ef71d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7ef6c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7ef9dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7f09227]
================================================================================
[0xb80aa400]
[0xb80aa422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7baa0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7efa27d]
/usr/local/lib/libdb-4.8.so [0xb7ef9951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7ef96da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7fbeb2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7fbd246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7ffd7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7f20de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7f214e0]
/usr/local/lib/libdb-4.8.so [0xb7f0aacb]
/usr/local/lib/libdb-4.8.so [0xb7f06031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7fe71d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7fe6c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7fe9dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7ff9227]
================================================================================
[0xb80ae400]
[0xb80ae422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7bae0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7efe27d]
/usr/local/lib/libdb-4.8.so [0xb7efd951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7efd6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7fc2b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7fc1246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb80017e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7f24de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7f254e0]
/usr/local/lib/libdb-4.8.so [0xb7f0eacb]
/usr/local/lib/libdb-4.8.so [0xb7f0a031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7feb1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7feac83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7feddd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7ffd227]
================================================================================
[0xb7f9d400]
[0xb7f9d422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a9d0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7ded27d]
/usr/local/lib/libdb-4.8.so [0xb7dec951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7dec6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7eb1b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7eb0246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7ef07e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7e13de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7e144e0]
/usr/local/lib/libdb-4.8.so [0xb7dfdacb]
/usr/local/lib/libdb-4.8.so [0xb7df9031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7eda1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7ed9c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7edcdd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7eec227]
================================================================================
[0xb7f04400]
[0xb7f04422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a040e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7d5427d]
/usr/local/lib/libdb-4.8.so [0xb7d53951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7d536da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e18b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e17246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7e577e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x530) [0xb7d7b200]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7d7b4e0]
/usr/local/lib/libdb-4.8.so [0xb7d64acb]
/usr/local/lib/libdb-4.8.so [0xb7d60031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7e411d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7e40c83]
/usr/local/lib/libdb-4.8.so [0xb7e4297a]
/usr/local/lib/libdb-4.8.so [0xb7e41b93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb7e42dac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7e30be1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb7e5125c]
================================================================================
[0xb80d8400]
[0xb80d8422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7bd80e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7f2827d]
/usr/local/lib/libdb-4.8.so [0xb7f27951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7f276da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7fecb2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7feb246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb802b7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7f4ede5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7f4f4e0]
/usr/local/lib/libdb-4.8.so [0xb7f38acb]
/usr/local/lib/libdb-4.8.so [0xb7f34031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb80151d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb8014c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb8017dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb8027227]
================================================================================
[0xb80af400]
[0xb80af422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7baf0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7eff27d]
/usr/local/lib/libdb-4.8.so [0xb7efe951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7efe6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7fc3b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7fc2246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb80027e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x530) [0xb7f26200]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7f264e0]
/usr/local/lib/libdb-4.8.so [0xb7f0facb]
/usr/local/lib/libdb-4.8.so [0xb7f0b031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7fec1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7febc83]
/usr/local/lib/libdb-4.8.so [0xb7fed97a]
/usr/local/lib/libdb-4.8.so [0xb7fecb93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb7feddac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7fdbbe1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb7ffc25c]
================================================================================
[0xb8066400]
[0xb8066422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7b660e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7eb627d]
/usr/local/lib/libdb-4.8.so [0xb7eb5951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7eb56da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7f7ab2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7f79246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7fb97e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7edcde5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7edd4e0]
/usr/local/lib/libdb-4.8.so [0xb7ec6acb]
/usr/local/lib/libdb-4.8.so [0xb7ec2031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7fa31d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7fa2c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7fa5dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7fb5227]
================================================================================
[0xb7f86400]
[0xb7f86424]
/lib/tls/i686/cmov/libc.so.6(__select+0x2d) [0xb762877d]
/usr/local/lib/libdb-4.8.so [0xb7f42783]
/usr/local/lib/libdb-4.8.so(__os_yield+0x7a) [0xb7f4272a]
/usr/local/lib/libdb-4.8.so(__memp_sync_int+0x545) [0xb7f3ce57]
/usr/local/lib/libdb-4.8.so(__txn_checkpoint+0x3ab) [0xb7f55926]
/usr/local/lib/libdb-4.8.so(__txn_checkpoint_pp+0x1b6) [0xb7f55532]
================================================================================
[0xb7f6e400]
[0xb7f6e422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a6e0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7dbe27d]
/usr/local/lib/libdb-4.8.so [0xb7dbd951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7dbd6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e82b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e81246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7ec17e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7de4de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7de54e0]
/usr/local/lib/libdb-4.8.so [0xb7dceacb]
/usr/local/lib/libdb-4.8.so [0xb7dca031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7eab1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7eaac83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7eaddd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7ebd227]
================================================================================
[0xb7f5e400]
[0xb7f5e422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a5e0e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7dae27d]
/usr/local/lib/libdb-4.8.so [0xb7dad951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7dad6da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e72b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e71246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7eb17e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7dd4de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7dd54e0]
/usr/local/lib/libdb-4.8.so [0xb7dbeacb]
/usr/local/lib/libdb-4.8.so [0xb7dba031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7e9b1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7e9ac83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7e9ddd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7ead227]
================================================================================
[0xb80d1400]
[0xb80d1424]
/lib/tls/i686/cmov/libc.so.6(__sched_yield+0xc) [0xb775d9dc]
/lib/tls/i686/cmov/libpthread.so.0(pthread_yield+0x14) [0xb7bcde84]
/usr/local/lib/libdb-4.8.so(__os_yield+0x81) [0xb808d731]
/usr/local/lib/libdb-4.8.so [0xb7f2090b]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7f206da]
/usr/local/lib/libdb-4.8.so(__memp_fget+0x9cf) [0xb807478b]
/usr/local/lib/libdb-4.8.so(__bam_search+0x15a0) [0xb7f49995]
/usr/local/lib/libdb-4.8.so [0xb7f31acb]
/usr/local/lib/libdb-4.8.so [0xb7f2f85e]
/usr/local/lib/libdb-4.8.so(__dbc_iput+0x261) [0xb801008e]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x198) [0xb800fe20]
/usr/local/lib/libdb-4.8.so [0xb800fae9]
/usr/local/lib/libdb-4.8.so [0xb800eb93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb800fdac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7ffdbe1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb801e25c]
================================================================================
[0xb7fa6400]
[0xb7fa6422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7aa60e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7df627d]
/usr/local/lib/libdb-4.8.so [0xb7df5951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7df56da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7ebab2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7eb9246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7ef97e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7e1cde5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7e1d4e0]
/usr/local/lib/libdb-4.8.so [0xb7e06acb]
/usr/local/lib/libdb-4.8.so [0xb7e02031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7ee31d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7ee2c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7ee5dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7ef5227]
================================================================================
[0xb7f77400]
[0xb7f77422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a770e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7dc727d]
/usr/local/lib/libdb-4.8.so [0xb7dc6951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7dc66da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e8bb2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e8a246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7eca7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x530) [0xb7dee200]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7dee4e0]
/usr/local/lib/libdb-4.8.so [0xb7dd7acb]
/usr/local/lib/libdb-4.8.so [0xb7dd3031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7eb41d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7eb3c83]
/usr/local/lib/libdb-4.8.so [0xb7eb597a]
/usr/local/lib/libdb-4.8.so [0xb7eb4b93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb7eb5dac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7ea3be1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb7ec425c]
================================================================================
[0xb8051400]
[0xb8051422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7b510e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7ea127d]
/usr/local/lib/libdb-4.8.so [0xb7ea0c3b]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_readlock+0x20) [0xb7ea0a42]
/usr/local/lib/libdb-4.8.so(__memp_fget+0xa6b) [0xb7ff4827]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x7a5) [0xb7f8e434]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7f8dc83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7f90dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7fa0227]
================================================================================
[0xb7ef2400]
[0xb7ef2422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb79f20e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7d4227d]
/usr/local/lib/libdb-4.8.so [0xb7d41951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7d416da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e06b2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e05246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7e457e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7d68de5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7d694e0]
/usr/local/lib/libdb-4.8.so [0xb7d52acb]
/usr/local/lib/libdb-4.8.so [0xb7d4e031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7e2f1d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7e2ec83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7e31dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7e41227]
================================================================================
[0xb7f47400]
[0xb7f47422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7a470e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7d9727d]
/usr/local/lib/libdb-4.8.so [0xb7d96951]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7d966da]
/usr/local/lib/libdb-4.8.so(__lock_get_internal+0x14df) [0xb7e5bb2a]
/usr/local/lib/libdb-4.8.so(__lock_get+0xc0) [0xb7e5a246]
/usr/local/lib/libdb-4.8.so(__db_lget+0x2dc) [0xb7e9a7e3]
/usr/local/lib/libdb-4.8.so(__bam_get_root+0x115) [0xb7dbdde5]
/usr/local/lib/libdb-4.8.so(__bam_search+0xeb) [0xb7dbe4e0]
/usr/local/lib/libdb-4.8.so [0xb7da7acb]
/usr/local/lib/libdb-4.8.so [0xb7da3031]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x548) [0xb7e841d7]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7e83c83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7e86dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7e96227]
================================================================================
[0xb80d1400]
[0xb80d1424]
/lib/tls/i686/cmov/libc.so.6(__sched_yield+0xc) [0xb775d9dc]
/lib/tls/i686/cmov/libpthread.so.0(pthread_yield+0x14) [0xb7bcde84]
/usr/local/lib/libdb-4.8.so(__os_yield+0x81) [0xb808d731]
/usr/local/lib/libdb-4.8.so [0xb7f2090b]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_lock+0x20) [0xb7f206da]
/usr/local/lib/libdb-4.8.so(__memp_fget+0x9cf) [0xb807478b]
/usr/local/lib/libdb-4.8.so(__bam_search+0x15a0) [0xb7f49995]
/usr/local/lib/libdb-4.8.so [0xb7f31acb]
/usr/local/lib/libdb-4.8.so [0xb7f2f85e]
/usr/local/lib/libdb-4.8.so(__dbc_iput+0x261) [0xb801008e]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x198) [0xb800fe20]
/usr/local/lib/libdb-4.8.so [0xb800fae9]
/usr/local/lib/libdb-4.8.so [0xb800eb93]
/usr/local/lib/libdb-4.8.so(__dbc_put+0x124) [0xb800fdac]
/usr/local/lib/libdb-4.8.so(__db_put+0x615) [0xb7ffdbe1]
/usr/local/lib/libdb-4.8.so(__db_put_pp+0x25e) [0xb801e25c]
================================================================================
[0xb8051400]
[0xb8051422]
/lib/tls/i686/cmov/libpthread.so.0(pthread_cond_wait+0x95) [0xb7b510e5]
/usr/local/lib/libdb-4.8.so(__db_pthread_mutex_lock+0x1c5) [0xb7ea127d]
/usr/local/lib/libdb-4.8.so [0xb7ea0c3b]
/usr/local/lib/libdb-4.8.so(__db_tas_mutex_readlock+0x20) [0xb7ea0a42]
/usr/local/lib/libdb-4.8.so(__memp_fget+0xa6b) [0xb7ff4827]
/usr/local/lib/libdb-4.8.so(__dbc_iget+0x7a5) [0xb7f8e434]
/usr/local/lib/libdb-4.8.so(__dbc_get+0xa0) [0xb7f8dc83]
/usr/local/lib/libdb-4.8.so(__dbc_pget+0x349) [0xb7f90dd7]
/usr/local/lib/libdb-4.8.so(__dbc_pget_pp+0x148) [0xb7fa0227]Hi Josef,
Could you try reproducing this with debug and diagnostics enabled libraries and collect the locking statistics at the moment you see this issue? (use the enable-debug and enable-diagnostic options when building)
This will help getting the process IDs/thread IDs in the locking statistics. If the environment region files (__db.NNN files) are still around, collect the locking statistics (db_stat -C o)
I assume deadlock handling is correctly implemented. Were there any errors reported (i.e. DB_LOCK_DEADLOCK)?
To enhance error reporting in the application see:
Error support
Run-time error information
Regards,
Andrei -
We are using BDB XML.And we are calling
delete DbXml::XmlManager *
We got a hung, no any return or error.
stack trace as below:
(gdb) bt
warning: Lowest section in system-supplied DSO at 0xffffe000 is .hash at ffffe0b4
0x4f66e071 in pthread_cond_wait () from /lib/libpthread.so.0
(gdb) bt
#0 0x4f66e071 in pthread_cond_wait () from /lib/libpthread.so.0
#1 0x4f2130bd in pthread_cond_wait () from /lib/libc.so.6
#2 0xf6b0e918 in __db_pthread_mutex_lock () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#3 0xf6b0e39d in __db_tas_mutex_lock () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#4 0xf6baf1d9 in __lock_id_free () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#5 0xf6bc9ba3 in __dbc_destroy () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#6 0xf6bbfd25 in __db_refresh () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#7 0xf6bbfa5b in __db_close () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#8 0xf6bd6d66 in __db_close_pp () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#9 0xf6b02984 in Db::close () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdb_cxx-4.6.so
#10 0xf68f3d25 in DbXml::DbWrapper::close () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#11 0xf68f7d60 in DbXml::DbWrapper::~DbWrapper () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#12 0xf68f88f8 in DbXml::IndexDatabase::~IndexDatabase () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#13 0xf695e8e8 in DbXml::SyntaxDatabase::~SyntaxDatabase () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#14 0xf68cfbac in DbXml::Container::~Container () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#15 0xf68c746c in DbXml::Container::release () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#16 0xf692caf2 in DbXml::XmlContainer::~XmlContainer () from /usr/app/capc/CL_Agent/LTE/thirdparty/lib/libdbxml-2.4.so
#17 0xf777f051 in std::_Rb_tree<std::string, std::pair<std::string const, DbXml::XmlContainer>, std::_Select1st<std::pair<std::string const, DbXml::XmlContainer> >, std::less<std::string>, std::allocator<std::pair<std::string const, DbXml::XmlContainer> > >::_M_erase ()
from /usr/app/capc/CL_Agent/LTE/agent/lib/libDataAccess.so
#18 0xf7777b8e in ma::BDBXml::terminate () from /usr/app/capc/CL_Agent/LTE/agent/lib/libDataAccess.so
#19 0xf7a96dd4 in ma::ConfigQueryTool::queryTerminate () from /usr/app/capc/CL_Agent/LTE/agent/lib/libcla_get_cfg.so
#20 0xf7ac6863 in queryTerminate () from /usr/app/capc/CL_Agent/LTE/agent/lib/libcla_get_cfg.so
#21 0xf7f75a7a in CLA_NMI_Handle_reset () at CLA_NMI.c:3872
#22 0xf7f70e6c in clanmiCCSDispatchThread (arg=0x0) at CLA_NMI.c:1575
#23 0x4f669e49 in ?? () from /lib/libpthread.so.0
#24 0x00000000 in ?? ()
(gdb)
Could anybody tell me how to resolve this issue?
Thanks.Code is like below. and we have two process A, B which are using this class/function. During we hit this issue, only one process is running, this process wants to termiate() and then restart. But terminate() hung there. no return any more.
* @brief The pointer to an XmlManager.
DbXml::XmlManager * m_manager_;
void BDBXml::terminate()
m_containers_.clear();
if(m_manager_ == NULL)
return;
if(m_manager_->getDbEnv() != NULL)
try
delete m_manager_->getDbEnv();
catch(...)
CL_LOG_ERROR("Delete DbEnv failed.");
delete m_manager_;
m_manager_ = NULL;
Edited by: user646849 on Sep 23, 2010 5:31 PM -
Dear devels
db version 4.8.30
I have met DbEnv::dbremove hang.
Could you please advise what can be a reason?
backtrace:
(gdb) bt
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x00007f3c516c191b in __db_pthread_mutex_lock () from /usr/lib64/libdb_cxx-4.8.so
#2 0x00007f3c516c1052 in __db_tas_mutex_lock () from /usr/lib64/libdb_cxx-4.8.so
#3 0x00007f3c51754a6a in __lock_get_internal () from /usr/lib64/libdb_cxx-4.8.so
#4 0x00007f3c517559d5 in __lock_vec () from /usr/lib64/libdb_cxx-4.8.so
#5 0x00007f3c517b0ec7 in __fop_lock_handle () from /usr/lib64/libdb_cxx-4.8.so
#6 0x00007f3c517b11b6 in __fop_remove_setup () from /usr/lib64/libdb_cxx-4.8.so
#7 0x00007f3c5179393f in __db_rename_int () from /usr/lib64/libdb_cxx-4.8.so
#8 0x00007f3c5179309a in __db_remove_int () from /usr/lib64/libdb_cxx-4.8.so
#9 0x00007f3c517936b1 in __env_dbremove_pp () from /usr/lib64/libdb_cxx-4.8.so
#10 0x00007f3c516b84db in DbEnv::dbremove(DbTxn*, char const*, char const*, unsigned int) () from /usr/lib64/libdb_cxx-4.8.so
Appreciate in advance.please describe what is going on in your environment. Do you have multiple processes, do you have processes with multiple threads.
From what is minimally described, it looks like some other process/thread died and it had an open handle to the db file you want
to remove. We cannot get the lock on the file handle so we are sitting waiting. I would suggest you remove all the processes from
the environment, run recovery, and then do the DbEnv::dbremove.
thanks
mike
Maybe you are looking for
-
Linux, Solaris 8-10 Software Update Library Inactive
Hello All. In the Navigation pane , in libraries , when i open Linux, Solaris 8-10 Software Update Library , the options on the left such as Upload config file , Upload local action and others are disabled. I cannot click on them and perform an actio
-
Web browsers crashing and using up RAM on new Macbook Pro
Hey everyone, Recently I bought a Mid 2014 Macbook Pro to replace my old 2011 model that was destroyed by liquid damage. Though I like this new computer, I seem to always have an issue with it running low on memory, ESPECIALLY when using web browsers
-
When an RFC calls fails I get the standard Rails page with the dump; how to detect and handle rfc errors (like subrc check); thanks
-
ITunes 7.4.2 slows my computer to molasses.
After upgrading, it takes a full minute to open iTunes. Not just iTunes though, it takes thiry seconds to open Internet Explorer and near triple the load time of all other applications. I was able to get rid of the lag for other programs by disabling
-
I know this question has been asked before, but management want me to query if we are missing any changes in the newer versions This issue is with what management calls signature stacking at the end of the email thread. Is there a switch in 2012 or 2