Log files can't be removed automatically in HA environment

Question

Hi BDB experts,
I am writing db HA application based on bdb version 4.6.21. Two daemons run on two machines, one as master which will read/write db, one as client/backup will only read db. There is one thread in master daemon that run checkpoint every 1 second: dbenv->txn_checkpoint(dbenv, 1, 1, 0), and dbenv->log_archive(dbenv, NULL, DB_ARCH_REMOVE) will be called after runnng checkpoint each time. The env was open with flag: DB_CREATE | DB_INIT_TXN |  DB_INIT_LOCK | DB_INIT_LOG | DB_REGISTER | DB_RECOVER | DB_INIT_MPOOL | DB_THREAD  | DB_INIT_REP;   Autoremove flag was set by: envp->set_flags(uid_dbs.envp, DB_LOG_AUTOREMOVE, 1) before open env.
I found this thread https://forums.oracle.com/message/10945602#10945602 which discussed about non-ha environment, and I tested my code in a non-ha env without DB_INIT_REP, it worked. However in HA env those log files were never removed. Could you help on this issue? Does the client need to run checkpoint? May there be a bdb bug?
Thanks,
Min

Answer 1

There is one thread in master daemon that run checkpoint every 1 second: dbenv->txn_checkpoint(dbenv, 1, 1, 0), and dbenv->log_archive(dbenv, NULL, DB_ARCH_REMOVE) will be called after runnng checkpoint each time. The env was open with flag: DB_CREATE | DB_INIT_TXN |  DB_INIT_LOCK | DB_INIT_LOG | DB_REGISTER | DB_RECOVER | DB_INIT_MPOOL | DB_THREAD  | DB_INIT_REP;   Autoremove flag was set by: envp->set_flags(uid_dbs.envp, DB_LOG_AUTOREMOVE, 1) before open env.
I am not saying that this is causing a problem, but doing the DB_ENV->log_archive(DB_ARCH_REMOVE) in your thread and setting DB_ENV->set_flags(DB_LOG_AUTOREMOVE) is redundant. In your thread, you control the timing. The DB_ENV->set_flags(DB_LOG_AUTOREMOVE) option checks for and removes unneeded log files when we create a new log file.
Did you see in the documentation for DB_ENV->set_flags(DB_LOG_AUTOREMOVE) that we don't recommend doing automatic log file removal with replication? Although this warning is not repeated in DB_ENV->log_archive(DB_ARCH_REMOVE), it also applies to this option. You should reconsider using this option, particularly if it is possible that your client could go down for a long time.
But this is only a warning and automatic log removal should work. My first thought here is to ask whether your client has recently gone through a sync? Internally, we block archiving on the master during some parts of a client sync to improve the chances that we will keep around all logs needed by the syncing client. We block archiving for up to 30 seconds after the client sync.
I found this thread https://forums.oracle.com/message/10945602#10945602 which discussed about non-ha environment, and I tested my code in a non-ha env without DB_INIT_REP, it worked. However in HA env those log files were never removed.
This thread is discussing a different issue. The reason for our warning in BDB 4.6 against using automatic log removal with replication is that it doesn't take into account all the sites in your replication group, so we could remove a log from the master that a client still needs.
We added replication group-aware automatic log removal in BDB 5.3 Replication Manager, and this discussion is about a change of behavior from this addition. With this addition, we no longer need to recommend against using automatic log removal with replication in BDB 5.3 and later releases.
Could you help on this issue? Does the client need to run checkpoint? May there be a bdb bug?
I'm not sure the client needs to run its own checkpoints because it performs checkpoints when it receives checkpoint log records from the master.
But none of the log removal options on the master does anything to remove logs on the client. You will need to perform steps to archive logs separately on the client and the master.
Paula Bingham
Oracle