Slow secondary indexes creation

Question

I have a database table where each record is a fixed length of 16 bytes and the key length is 4 bytes.
The table has ~3G records, giving file size of 61GB.
The access method is DB_QUEUE.
I'm using the 5.1.25 version.
I'm trying to create two secondary indexes using DB_HASH as access method.
I set the pagesize to 65K, cachesize to 65M.
When I start to build the indexes I see that the creation time of the indexes will take me something like 3 days which I think is reasonable. But after a day the performances are dropping drastically, giving me approximately creation time of 1 month.
What should I do in order to boost performance on such case?
Edited by: 868434 on Jun 26, 2011 2:07 AM

Answer 1

Hi,
I set the pagesize to 65K, cachesize to 65M.You probably mean 64KB; see the DB->set_pagesize() documentation.
When I start to build the indexes I see that the creation time of the indexes will take me something like 3 days which I think is reasonable. But after a day the performances are dropping drastically, giving me approximately creation time of 1 month. Are you recreating the secondary indexes on a regular basis? If so, what is the reason for this?
Once you open, associate and populate the secondary (using the DB_CREATE flag for the associate() call), you should keep the secondary databases around and not remove them just to recreate them again. Whenever the primary database is updated, the appropriate updates are performed in the secondaries as well to reflect the changes; see Secondary indexes documentation section.
Opening (creating a new database file handle) and populating a secondary database on open are potentially very expensive operations; see the subheading on DB_CREATE in the DB->associate() documentation section.
What should I do in order to boost performance on such case?The best approach would be to not remove and recreate the secondary indexes regularly, but rather create, populate and keep them around as long as the application runs.
If however you need to recreate them again and again, and given that they use the Hash access method, here are some suggestions:
1. If you have a support contract and can access MOS (My Oracle Support) have a look over Doc ID 463613.1 and Doc ID 463507.1 notes in the KM repository (they discuss tuning Hash databases' configuration for speeding up insertion times).
If you cannot access them, let me know and I can make a summary with further guidance.
2. Review the Hash access method specific configuration doc section and try to experiment with appropriately calculated values for the density / page fill factor, hash table size (estimated no of elements that will be stored in the database), and with a hash function that evenly distributes the (secondary) keys across buckets (and eventually with a Hash key comparison function that sorts the keys appropriately respecting their structure/format, data type and size).
3. I assume that this is happening in a transactional environment. If so, it might worth trying to use a transaction handle in the DB->associate() call, which has been opened with the DB_TXN_WRITE_NOSYNC or DB_TXN_NOSYNC flag (see DB_ENV->txn_begin()).
Some general tuning suggestions, that could be used are also given in the following documentation sections: Access method tuning, and Transaction tuning.
Regards,
Andrei