DBLoad utility: (huge) difference between Windows and Linux

Question

Hello
I have a backfile of 55M entries to load. I have prepared it under Linux (Dual Core machine + 4GB - Mandriva distribution - java 1.6). When I run DbLoad, here is the (partial) log:
java -Xmx2048m -jar /home/pd51444/jNplMatchBackfile_Main_0.2/lib/je-3.3.69.jar DbLoad -h . -s group.db -f group.db.txt -v
loaded 2817812 records 63275 ms - % completed: 5
loaded 5624893 records 141023 ms - % completed: 10
loaded 8462602 records 962019 ms - % completed: 15
loaded 11285385 records 2401566 ms - % completed: 20
loaded 14094928 records 786746 ms - % completed: 25
loaded 16914275 records 8965457 ms - % completed: 30
loaded 19741557 records 15766560 ms - % completed: 35
loaded 22567310 records 2226015 ms - % completed: 40
loaded 25376363 records 19662455 ms - % completed: 45
Then I copied the exact same file on my Windows laptop (dual core + 2GB - XP - java 1.6), and DbLoad goes much faster:
C:\nplmatch\db>java -Xmx1024m -jar ..\jar\je-3.3.69.jar DbLoad -h . -s group.db -f group.db.txt -v
Load start: Thu Oct 02 10:33:23 CEST 2008
loaded 2817812 records 59876 ms - % completed: 5
loaded 5624893 records 69283 ms - % completed: 10
loaded 8462602 records 77470 ms - % completed: 15
loaded 11285385 records 69688 ms - % completed: 20
loaded 14094928 records 62716 ms - % completed: 25
loaded 16914275 records 59122 ms - % completed: 30
loaded 19741557 records 63200 ms - % completed: 35
loaded 22567310 records 58654 ms - % completed: 40
loaded 25376363 records 61482 ms - % completed: 45
loaded 28197663 records 58889 ms - % completed: 50
loaded 31019453 records 55937 ms - % completed: 55
loaded 33839878 records 62045 ms - % completed: 60
loaded 36664839 records 65749 ms - % completed: 65
loaded 39498035 records 100718 ms - % completed: 70
loaded 42302599 records 99733 ms - % completed: 75
loaded 45125268 records 96000 ms - % completed: 80
loaded 47947180 records 92749 ms - % completed: 85
loaded 50755655 records 85485 ms - % completed: 90
loaded 53578015 records 96240 ms - % completed: 95
Load end: Thu Oct 02 10:57:36 CEST 2008
Also I use the same je.properties file on both platforms.
Any idea where this performance problem comes from?
Thanks in advance
Best regards
Philippe

Answer 1

Hello Phillipe,
Nothing jumps out at me, but you might try the following:
. Check the status of disk write caches. On linux, the cache may be disabled and it may be enabled on windows.
. Does the windows machine have an SSD?
. Run with -verbose:gc to see if the Linux is being held up by full GC's.
. Use top or some other utility to see if something else is running on the linux box.
. Take some random c-\'s to get some stack traces to see what is going on when things get slow.
. Check on the JVM ergonomics. You may be getting a server or client JVM without knowing. Force it one way or the other with -server or -client.
Here is a (future) FAQ entry regarding disk write caches (poorly formatted at the moment):
During my system testing I pulled the power cord on my server to make the ultimate test of JE's durability claims. I am using commitSync() for my transactions, but some of the data that JE said it had committeed was not on disk when the system came back up. What gives?
Quoting the Berkeley DB Reference Guide:
Many disk drives contain onboard caches. Some of these
drives include battery-backup or other functionality that guarantees that all
cached data will be completely written if the power fails. These drives can
offer substantial performance improvements over drives without caching support.
However, some caching drives rely on capacitors or other mechanisms that
guarantee only that the write of the current sector will complete. These
drives can endanger your database and potentially cause corruption of your
data.
To avoid losing your data, make sure the caching on your disk
drives is properly configured so the drive will never report that data has
been written unless the data is guaranteed to be written in the face of a
power failure. Many times, this means that write-caching on the disk drive
must be disabled.
Some operating systems enable the disk write cache by default.
If you need true durability in the face of a power failure, then you should
verify that the disk write cache is disabled or that you have some alternative
means of ensuring durability (e.g. nvram, battery backup, an Uninterruptible
Power Supply (UPS), Solid State Disk (SSD), etc.) Some disk drives may
actually require changing hardware jumpers to enable/disable the write cache.
You can check the status of the write cache using the
hdparm utility on Linux and the format utility on Solaris.
On Windows, use the Windows Explorer. Right click on
the disk drive that you want to check, select Properties,
click on the Hardware tab, select the desired disk, click
on the Properties button, click on
Policies and verify the cache setting with the check box.
On Windows Server 2003 you generally disable the write caching
from within the RAID controller software (OEM specific).