Envp- open hangs after crash

Question

Hello All,
If I repeatedly kill a test app that opens and closes a BDB databaseate, I eventually get into a state where envp->open() freezes and never returns. I accept that open() may throw an exception and I may need to call Open() again with the DB_RECOVER flag, but I can not allow it to hang indefinitely. It is not an option for me to call open() with the DB_RECOVER flag every time I open the database because this must be process and thread safe.
Can anyone offer some assistance for how to get around this problem? The only solution I could think of was to spawn a new thread from which to call envp->open, and kill that thread if it did not complete in time, but that is hardly an acceptable solution.
I am using BDB version 4.7.25.
I am opening the environment with the following code:
     const int MAX_THREAD_COUNT = 30;
     const int MAX_LOGFILE_SIZE = 2097152;
     DbEnv* envp = NULL;
     u_int32_t envFlags =
     DB_CREATE | // Create the environment if it does not exist
     DB_INIT_LOCK | // Initialize the locking subsystem
     DB_INIT_TXN | // Initialize the transactional subsystem.
     DB_INIT_MPOOL | // Initialize the memory pool (in-memory cache)
     DB_THREAD; // Cause the environment to be free-threaded
     // Add recover option
     if (dbRecoverAttempt > 0)
          envFlags = envFlags | DB_RECOVER;
     // Create and open the environment
     envp = new DbEnv(0);
     envp->set_lk_detect(DB_LOCK_YOUNGEST);
     envp->set_thread_count(MAX_THREAD_COUNT);
     envp->set_isalive(&is_alive);
     envp->set_lg_max(MAX_LOGFILE_SIZE);
     envp->set_flags(DB_LOG_AUTO_REMOVE, 1);
     envp->open(dbDir, envFlags, 0/*callback*/);
Regards,
Todd Kobus

Answer 1

Hello Michael,
We have implemented a monitor thread that calls failchk(), but the monitor thread is blocked during environment open and close operations. Should I be calling failchk() immediately prior to calling envp->close() to limit the potential for a deadlocked close operation?
Update: I tried adding a call to failchk() immediately prior to my call to envp->close(). I managed to get a hang from within failchk() on my first test run.
Occasionally, I get the following output when calling envp->open() on an environment that has recently died within a different process:
unable to join the environment
Recovery function for LSN 2 194808 failed on forward pass
PANIC: No such file or directory
unable to join the environment
Followed by an "Attempted to read or write protected memory. This is often an indication that..." exception
Regards,
Todd