Core + JNI binding cannot re-open database with log in mem option

Question

I am doing some tests with BDB Core using the JNI interface and I have so far found that the speed is much faster with the setLogInMemory(true) option set on the database. I know this puts durability at risk but assuming there is no crash and I successfully close the database I am not able to re-open the database.
I am opening the DB with code like the following:
public void setupEventDB(File envHome)
throws DatabaseException, FileNotFoundException {
EnvironmentConfig myEnvConfig = new EnvironmentConfig();
myEnvConfig.setTransactional(_transactional);
myEnvConfig.setAllowCreate(_allowCreate);
myEnvConfig.setInitializeCache(true);
myEnvConfig.setLogBufferSize(100000000);
myEnvConfig.setLogInMemory(true); //Causes all logging to go to memory
myEnvConfig.setPrivate(true);
// Open the environment
myEnv = new Environment(envHome, myEnvConfig);
DatabaseConfig myDbConfig = new DatabaseConfig();
myDbConfig.setReadOnly(_readOnly);
myDbConfig.setTransactional(_transactional);
myDbConfig.setAllowCreate(_allowCreate);
myDbConfig.setType(DatabaseType.BTREE);
// Now open, or create and open, our databases
// Open the vendors and inventory databases
eventDb = myEnv.openDatabase(null,
"eventDB.db",
"eventDB",
myDbConfig);
My basic test is the following:
1. Open database
2. Insert 10 million records
3. Walk cursor across all records
4. Close database
5. Re open database
6. Walk cursor again
7. Close database
Basically when I get to step 5 I get the following error:
file unknown has LSN 18889/65067, past end of log at 1/28
Commonly caused by moving a database from one database environment
to another without clearing the database LSNs, or by removing all of
the log files from a database environment
/u02/DB/0/eventDB.db: unexpected file type or format
Exception: java.lang.IllegalArgumentException: Invalid argument: the log files from a database environment
/u02/DB/0/eventDB.db: unexpected file type or format
java.lang.IllegalArgumentException: Invalid argument: the log files from a database environment
/u02/DB/0/eventDB.db: unexpected file type or format
at com.sleepycat.db.internal.db_javaJNI.Db_open(Native Method)
at com.sleepycat.db.internal.Db.open(Db.java:404)
at com.sleepycat.db.DatabaseConfig.openDatabase(DatabaseConfig.java:1990)
at com.sleepycat.db.Environment.openDatabase(Environment.java:314)
at db.MyDbEnv.setupEventDB(MyDbEnv.java:139)
at db.InsertTool.run(InsertTool.java:94)
at db.InsertTool.main(InsertTool.java:61)
Exception in thread "main" java.lang.NullPointerException
at db.MyDbEnv.close(MyDbEnv.java:170)
at db.InsertTool.main(InsertTool.java:70)
My cursor code looks like the following:
private void walkCursor() throws DatabaseException {
System.out.println("Walking cursor");
Cursor cursor = myDbEnv.getEventDB().openCursor(null, null);
long count = 0;
long beforeCursor = System.currentTimeMillis();
PnRealTimeObj obj = new PnRealTimeObj();
PnRealTimeObj temp = null;
while(cursor.getNext(theKey, theData, null) == OperationStatus.SUCCESS){
count ++;
temp = (PnRealTimeObj) obj.entryToObject(theData);
if(count != temp.getMsgId()){
System.out.println("Count: "+count+" record: "+temp);
cursor.close();
long afterCursor = System.currentTimeMillis();
System.out.println("End Scan Found : "+count+" records. Time delta: "+(afterCursor - beforeCursor)+ " ms");
Interested in how you can get a DB with log in memory == true to open? I have tried this by starting a different program afterwords to just walk the cursor and that didn't work either. I am assuming it is complaining because there is no log but there isn't supposed to be a log with log in mem == true. I am assuming everything should get flushed to the DB when you close the database so it should still work?
Thanks,
Jared

Answer 1

To answer your questions I was closing the DB and then closing the environment when I ran into this problem.
I however was not using the remove call from the API it isn't clear to me when one should call remove and when one should call close? It appears that both put the environment in a terminal state and seems that remove uses slightly more forceful language and talks about removing environment files on disk.
I did however as an update to this receive an answer from oracle from an email thread I had with them and was told that I should try the resetLogSequenceNumber command. I tried doing this between the DB close and the environment close and this seemed to do the trick. My code looked like the following:
eventDb.close();
                    long logResetStart = System.currentTimeMillis();
                    File[] dbfiles = _envHome.listFiles(new FilenameFilter(){
                         @Override
                         public boolean accept(File arg0, String arg1) {
                              if(arg1.endsWith(".db")){
                                   return true;
                              }else{
                                   return false;
                    for(File dbfile:dbfiles){
                         System.out.println("Resetting log sequence number in: "+dbfile.getAbsolutePath());
                         myEnv.resetLogSequenceNumber(dbfile.getAbsolutePath(), false);
                    long logResetEnd = System.currentTimeMillis();
                    System.out.println("Elapsed time for log reset: "+(logResetEnd-logResetStart));
=============
As a side note it looks like the reset log sequence number call is fairly heavy weight operation it took on the order of 22 seconds to complete on my box.