New Log Files Not 100% Utilization

Question

I just recently deployed my web app with new configurations to require my BDB to have at least 65% disk utilization (up from the default 50%) and 25% minimum file utilization (up from the default 5%). On start of my app, I temporarily coded an env.cleanLog() to force a clean of the logs to bring the DB up to these utilization parameters.
I've deployed the app, and its been chugging away at the clean process now for about 2.5 hours. It is generating a new 100 MB log file approximately every minute as it is attempting to compact the DB. However, at 2.5 hours, I just ran DbSpace and the DB utilization is only up to 51%. Perhaps most surprising are the following two facts:
1) The newly created log files from the cleaner are nowhere near 100% utilization
2) Some of the newly created log files have already now been cleaned (deleted) themselves.
There are 0 updates happening to the BDB while this process is running. My understanding was that when the cleaner removed old files and created new files, only good/valid data is written to the new files. However, this doesn't seem to be the case. Can someone enlighten me? I'm happy to read docs if someone can point me in the right direction.
Thanks.

Answer 1

Sorry, somehow I missed that you weren't doing any updates, even though you said so. I apologize.
However, your cache size is much too small for your data set. Running DbCacheSize with a rough approximation of your data set size gives:
~/je.cvs$ java -jar build/lib/je.jar DbCacheSize -records 50000000 -key 30 -data 300
Inputs: records=50000000 keySize=30 dataSize=300 nodeMax=128 binMax=128 density=80% overhead=10%
=== Cache Sizing Summary ===
   Cache Size       Btree Size    Description
  3,543,168,702    3,188,851,832  Minimum, internal nodes only
  3,963,943,955    3,567,549,560  Maximum, internal nodes only
22,209,835,368   19,988,851,832  Minimum, internal nodes and leaf nodes
22,630,610,622   20,367,549,560  Maximum, internal nodes and leaf nodes
=== Memory Usage by Btree Level ===
Minimum Bytes    Maximum Bytes      Nodes    Level
  3,157,713,227    3,532,713,035     488,281    1
     30,834,656       34,496,480       4,768    2
        297,482          332,810          46    3
          6,467            7,235           1    4Apparently when you started, the cleaner was "behind" -- utilization was low. Without enough cache to hold the internal nodes -- 3.5GB according to DbCacheSize -- the cleaner will either take a very long time or never be able to clean up to 50% utilization.
Do you really only have that much memory available, or is this just a dev environment issue?
--mark