Deletion strategy

Question

I'm considering using BDB JE for building an application that needs to store a large amount of events per day (in the order of 100,000,000). The record structure is pretty small (I assume it will be less of 100B) but I was wondering what would be the best strategy for adding such a large amount of records to the database and then removing them after given amount of time according to the DB aging/archiving policy. What I'm thinking of is a sort of partitioning by creating a separate DB for each day and remove it from the environment when I'm done with it. Suggestions?
Regards.
Antonio.

Answer 1

Hi Antonio,
You could have a logical grouping of databases, for example, an event database each month and then simply delete the oldest database when you add the new one.
You could do this by creating a db, and then calling Environment.deleteDatabase or truncateDatabase. This approach lets you administer the db within a single env, in the normal way, and access your db in the normal way. But when you call deleteDatabase and removeDatabase, the log cleaner has to remove all that data from the intermingled files.
A faster approach is to put each droppable set of dbs into their own environment. When the time to delete comes, you just rm the whole database, so it's really fast. You skip all of the log cleaning work. But the downside is that
The db is in a separate environment; you can't use txns across environments, if that's a concern.
Each environment has its own cache, so the app has to take on the burden of deciding how much cache to allot to each env.
You may also find this forum entry useful:
Multiple envs in one process?
We are listening to the fact that folks want a single cache across multiple envs, but don't have that available yet.
Regards,
Ron Cohen
Oracle Corporation