"Not enough space" DatabaseException while using DatabaseEnvironment

Question

I want to create some database in a DatabaseEnvironment which will be store a large data, such as 10TB. But I always get an DatabaseException with message "Not enough space",  of course there are enough space to store these data in the driver. It throws a database exception while put data and the database size reach a size(or 125% size) specified by CacheInfo, even though I close and then reopen the database environment and database. In addition,  it will throw an exception when I give the param "gbytes" a value larger than 1 in CacheInfo,  such as  "new CacheInfo(2, 0, 1)". The OS is windows 8. Why?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Diagnostics;
using BerkeleyDB;
namespace TestDatabaseEnvironment
    class Program
        static void Main(string[] args)
            string Dir = @"D:\Temp2\";
            string Name = "TestEnv";
            if (Directory.Exists(Dir))
                Directory.Delete(Dir, true);
            DatabaseEnvironmentConfig envConfig = new DatabaseEnvironmentConfig();
            envConfig.Create = true;
            envConfig.DataDirs.Add(Dir);
            envConfig.CreationDir = Dir;
            envConfig.MPoolSystemCfg = new MPoolConfig();
            //envConfig.MPoolSystemCfg.CacheSize = new CacheInfo(2, 0, 1);
            envConfig.MPoolSystemCfg.CacheSize = new CacheInfo((uint)0, (uint)64 * 1024 * 1024, 1);
            Console.WriteLine("CacheSize:" +
                envConfig.MPoolSystemCfg.CacheSize.Gigabytes.ToString() + "GB and " +
                envConfig.MPoolSystemCfg.CacheSize.Bytes.ToString() + "Bytes");
            envConfig.UseMPool = true;
            envConfig.ErrorPrefix = Name;
            if (!Directory.Exists(Dir))
                Directory.CreateDirectory(Dir);
            DatabaseEnvironment Env = DatabaseEnvironment.Open(Dir, envConfig);
            HashDatabaseConfig cfg = new HashDatabaseConfig();
            cfg.Env = Env;
            cfg.Creation = CreatePolicy.IF_NEEDED;
            HashDatabase database1 = HashDatabase.Open(null, "db1", cfg);
            HashDatabase database2 = HashDatabase.Open(null, "db2", cfg);
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (int i = 0; i < 100000000; i++)
                DatabaseEntry dbKey = new DatabaseEntry(BitConverter.GetBytes(i));
                DatabaseEntry dbValue = new DatabaseEntry(Encoding.UTF8.GetBytes(new string('A', 10 * 1024) + i.ToString()));
                database1.Put(dbKey, dbValue);
                database2.Put(dbKey, dbValue);
                if (i % 100 == 0)
                    Console.WriteLine(i.ToString() + ":" + sw.Elapsed.TotalSeconds.ToString());
            Console.WriteLine("TimeSpan:" + sw.Elapsed.TotalSeconds.ToString());
            database1.Close();
            database2.Close();
            Env.Close();

Answer 1

Hi,
I have run your program on both 32 and 64 bit Windows, but there is no error of "Not enough space". The program is able to inserts the records until it fills up the disk. I have tried running it with CacheInfo(0, 64 * 1024 * 1024, 1) and CacheInfo(1, 64 * 1024 * 1024, 1).
For the cachesize problem, I can reproduce the failure when the program requests a single 2GB cache. But this is normal.
32-bit Windows hardly ever has a 2GB contiguous free region in its address map. There are .DLLs and data segments mapped scattered all over the address space, include just below 2GB. Since the most data that a 32-bit application can possibly have is 3 GB (and even that is unusual), there is no single 2GB space for a mapping.
This is not a BDB bug. If you really need 2GB cache, you could try several smaller regions to get closer to 2GB.
Moreover, if you pass the null db file name to HashDatabase.Open(). The database will be created in-memory. That means the database stays inside the environment cache only. So if the database grows bigger than the cache, it will return the "Not enough space" error.
If there is any question, please let me know.
Regards,
Cindy Zeng