Bug #18104 still exists

Question

Hello,
I am quite disappointed about BDB 5. One of the most nasty bugs (#18104) still
exists. As soon as you want to use the set_thread_count() feature, BDB will not
free the thread-control-block. If you connect/disconnect, BDB will stop working
quite early. You can re-produce this issue with any `db_*` tool.
The testcase:
#include <stdlib.h>
#include <db.h>
#define DIRECTORY "/tmp/db"
int main()
    /* open environment */
    DB_ENV *handle;
    int status = db_env_create( &handle, 0 );
    status = handle->set_cachesize( handle, 0, 32*1024*1024, 0 );
    status = handle->set_thread_count( handle, 0x100 );
    status = handle->open( handle, DIRECTORY, DB_CREATE, 0644 );
    if( status != 0 )
        return 1;
    /* raise error "Unable to allocate thread control block" */
    for( ;; )
        system( "db_stat -e -h " DIRECTORY " | grep process/thread | wc" );
    return 0;
}

Answer 1

Hi Sandra,
Hi Greg,
Our architecture is quite simple: a cron-job runs ENV->failchk() and ENV->txn_checkpoint()
once a minute (which should be fine, shouldn't it?)
The problem, neither ENV->close() nor ENV->failchk() removes outdated items from the
thread-control-block. One would expect, that the PID is added to the block as soon as the
process connects and removed when the process disconnects. But it is not.
If the block runs out of memory, BDB requires a recovery (see the down below testcase).
Usually ENV->failchk() cleans the thread-control-block, as soon as there are more items in
the block than "thread_count". Today three times it did not even remove them, if there were
more than "thread_count" items in the block - but this happend sporadic so it was difficult
to create a testcase.
Once ENV->failchk() cleans the thread-control-block, BDB works as expected. If a process
connects, the PID is added to the block, and as soon as the process disconnects, the PID is
removed from the thread-control-block. You can test this by looping the first loop 1090 times.
The testcase:
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <db.h>
#include <errno.h>
#define DIRECTORY "/tmp/db"
static int bdb_is_alive( DB_ENV *dbenv, pid_t pid, db_threadid_t tid, u_int32_t flags )
    int status;
    status = kill( pid, 0 );
    if( status != -1 )
        return 1;
    if( errno == ESRCH )
        return 0;
    exit( 1 );  /* should never happen */
int main()
    system( "mkdir -p " DIRECTORY );
    system( "db_recover -h " DIRECTORY );
    system( "rm -rfv " DIRECTORY "/*" );
    /* open environment */
    int status, i;
    DB_ENV *handle;
    status = db_env_create( &handle, 0 );
    status = handle->set_cachesize( handle, 0, 32*1024*1024, 0 );
    status = handle->set_thread_count( handle, 1091 );
    status = handle->set_isalive( handle, bdb_is_alive );
    status = handle->open( handle, DIRECTORY, DB_CREATE, 0644 );
    if( status != 0 )
        return 1;   /* should never happen */
    status = handle->failchk( handle, 0 );
    status = handle->close( handle, 0 );
    /* Now start workers (`db_stat`) + failchks() */
    for( i=0; i < 1089; ++i )       /* thread_count - 2 */
        fprintf(stderr,"Number of items in the thread-control-block: "); fflush(stdout);
        system( "db_stat -e -h " DIRECTORY " | grep process/thread | wc -l" );
        status = db_env_create( &handle, 0 );
        status = handle->set_isalive( handle, bdb_is_alive );
        status = handle->open( handle, DIRECTORY, DB_CREATE, 0644 );
        if( status != 0 )
            return 1;   /* should never happen */
        status = handle->failchk( handle, 0 );      /* strange - does not remove the process-entries created by `db_stat` */
        if( status != 0 )
            return 1;   /* should never happen */
        status = handle->close( handle, 0 );
    /* raise error: "Unable to allocate thread control block" */
    for( i=1; i < 422; ++i )
        fprintf(stderr,"Number of items in the thread-control-block: "); fflush(stdout);
        system( "db_stat -e -h " DIRECTORY " | grep process/thread | wc -l" );
    /* You should see on stderr:
            db_stat: Unable to allocate thread control block
            db_stat: Unable to allocate thread control block
            db_stat: DB_ENV->open: /tmp/db: Cannot allocate memory
    fprintf(stderr,"BDB is dead now. Only `db_recover` helps...\n%s\n",db_version(NULL,NULL,NULL));
    return 0;
}