Partial backup question

Question

Hallo developers.
We are testing the partial backup procedure and we have some doubts.
We would like to know how the partial backup is made consistent; if during the operation of backup (the copy of the jdb files) the application writes to the last jdb file after a commit is there any problem of consistency?
In the guide is written clearly that you can copy the files during database operations: when you open the environment from the backup is the new environment reconstructed from the last chekpoint?
Kind regards
Massimo

Answer 1

Hi Massimo,
We would like to know how the partial backup is made
consistent; if during the operation of backup (the
copy of the jdb files) the application writes to the
last jdb file after a commit is there any problem of
consistency?JE uses a log based storage system. Once a record is written to the log, it is never overwritten again -- inserts, updates, and deletes are only ever appended to the log. In addition, there are never any "holes" in the log file. Therefore, if you copy log files from the environment directory in lexicographic order (essentially numerical hex order), and you copy the sectors in those files in ascending order, you will always have a consistent backup.
For example, if you have files 000000ab.jdb through 000000af.jdb and you copy them in order (ab, ac, ad, ae, af) you will have a consistent backup snapshot of your environment. If you want to do an incrental backup on top of that backup, then you should copy any files that have been modified since the last backup was started. In this example, if 000000af had been modified (as it probably would have been since it was the last file, but probably not completely full when the backup finished), you would copy that file and any ones after that (e.g. b0, b1, ...). When you restore, you would take the most recent (and most full) copy of af (as well as all the other files).
When the recovered databases are opened, JE runs a recovery (as usual) and makes the transactional state consistent. So if your backup included records from a transaction had not yet committed, and the backup did not copy the commit record, that transaction would be rolled back at recovery time.
Now what causes a problem is if the cleaner is running. I won't go into a lengthy discussion of it here, but you can read about it at
http://www.oracle.com/technology/documentation/berkeley-db/je/index.html
under the javadoc for com.sleepycat.je.util.DbBackup.
The net result is that you should use the DbBackup utility class for your backups.
In the guide is written clearly that you can copy the
files during database operations: when you open the
environment from the backup is the new environment
reconstructed from the last chekpoint? Correct. Because the log file is append only, when you copy files you will see a consistent, although possibly not most-recent, state of the database. By the way, a checkpoint is really just a performance improvement -- it is possible that there are no checkpoints in the database to recover from, but JE will still be able to recover the Environment.
I hope this is useful.
Regards,
Charles Lamb