Join query problem

Question

Hi all,
I'm new to Berkeley DB and apologise if the answer to my question has been covered elsewhere.
I've been using the 'Getting Started Guide' (BerkeleyDB-Core-Cxx-GSG.pdf) to familiarise myself with the C++ API. The document has been vastly useful, but it has left me stranded on the subject of join queries. I have used the example in the guide (p 57) carefully to construct my join query between two secondary databases, but I get a segmentation fault that doesn't seem to originate from within my code. The GDB backtrace shows:
(gdb) backtrace
#0 0x007fbffb in __db_c_count () from /nfs/acari/dta/bdb/lib/libdb_cxx-4.4.so
#1 0x00807aef in __db_join_cmp () from /nfs/acari/dta/bdb/lib/libdb_cxx-4.4.so
#2 0x0013c1af in msort_with_tmp () from /lib/libc.so.6
#3 0x0013c0a7 in msort_with_tmp () from /lib/libc.so.6
#4 0x0013c360 in qsort () from /lib/libc.so.6
#5 0x00806de6 in __db_join () from /nfs/acari/dta/bdb/lib/libdb_cxx-4.4.so
#6 0x00804384 in __db_join_pp () from /nfs/acari/dta/bdb/lib/libdb_cxx-4.4.so
#7 0x0079070b in Db::join () from /nfs/acari/dta/bdb/lib/libdb_cxx-4.4.so
#8 0x0804a9fe in show_join ()
#9 0x0804a165 in main ()
The code that I have written to perform the join query looks like:
int show_join(MyDb &itemnameSDB, MyDb &catnameSDB,
     std::string &itemName, std::string &categoryName)
std::cout << "Have item : " << itemName << " and category : "
     << categoryName << std::endl;
// Position cursor at item
int ret;
Dbc *item_curs;
Dbt key, data;
try {
itemnameSDB.getDb().cursor(NULL, &item_curs, 0);
char * c_item = (char *)itemName.c_str();
key.set_data(c_item);
key.set_size(strlen(c_item) + 1);
if ((ret = item_curs->get(&key, &data, DB_SET)) != 0)
     std::cout << "Did not find any records matching item ["
          << c_item << "]" << std::endl;
catch(DbException &e) {        
itemnameSDB.getDb().err(e.get_errno(), "Error!");
} catch(std::exception &e) {
itemnameSDB.getDb().errx("Error! %s", e.what());
// Position cursor at category
Dbc *category_curs;
try {
catnameSDB.getDb().cursor(NULL, &category_curs, 0);
char c_category = (char )categoryName.c_str();
key.set_data(c_category);
key.set_size(strlen(c_category) + 1);
if ((ret = category_curs->get(&key, &data, DB_SET)) != 0)
     std::cout << "Did not find any records matching category ["
          << c_category << "]" << std::endl;
catch(DbException &e) {        
catnameSDB.getDb().err(e.get_errno(), "Error!");
} catch(std::exception &e) {
catnameSDB.getDb().errx("Error! %s", e.what());
// Set up an array of cursors ready for the join
Dbc *carray[3];
carray[0] = item_curs;
carray[1] = category_curs;
carray[3] = NULL;
// Perform the join
Dbc *join_curs;
try {
if ((ret = itemnameSDB.getDb().join(carray, &join_curs, 0)) != 0)
     std::cout << "Successful query results should go here." << std::endl;
catch(DbException &e) {        
itemnameSDB.getDb().err(e.get_errno(), "Error!");
} catch(std::exception &e) {
itemnameSDB.getDb().errx("Error! %s", e.what());
// Iterate through results using the join cursor
while ((ret = join_curs->get(&key, &data, 0)) == 0)
std::cout << "Iterating through cursors" << std::endl;
// If we exited the loop because we ran out of records,
// then it has completed successfully.
if (ret == DB_NOTFOUND)
item_curs->close();
category_curs->close();
join_curs->close();
return(0);
The seg fault occurs at the line in the final try/catch block where the Db.join() call is made.
It seems highly likely that I am making a simple mistake due to inexperience (both with Berkeley DB and C++) and am hoping that the problem glares out at someone with deeper knowledge.
I'm running this under linux if this makes any difference.
Many thanks for reading this far,
Dan

Answer 1

Hi Keith,
The following test program isn't pretty, but should produce the seg fault that I'm seeing. Much of the code is copy-and-pasted from the C++ API guide. It will need some input data to run - and presumably create the correct error. Save the following as ./small_inventory.txt (this is also just hacked from the guide):
Oranges#OranfruiRu6Ghr#0.71#451#fruits#TriCounty Produce
Spinach#SpinvegeVcqXL6#0.11#708#vegetables#TriCounty Produce
Banana Split#Banadessfif758#11.07#14#desserts#The Baking Pan
Thanks for your help,
Dan
Code follows:
#include <db_cxx.h>
#include <iostream>
#include <fstream>
#include <cstdlib>
class InventoryData
public:
inline void setPrice(double price) {price_ = price;}
inline void setQuantity(long quantity) {quantity_ = quantity;}
inline void setCategory(std::string &category) {category_ = category;}
inline void setName(std::string &name) {name_ = name;}
inline void setVendor(std::string &vendor) {vendor_ = vendor;}
inline void setSKU(std::string &sku) {sku_ = sku;}
inline double& getPrice() {return(price_);}
inline long& getQuantity() {return(quantity_);}
inline std::string& getCategory() {return(category_);}
inline std::string& getName() {return(name_);}
inline std::string& getVendor() {return(vendor_);}
inline std::string& getSKU() {return(sku_);}
/* Initialize our data members */
void clear()
price_ = 0.0;
quantity_ = 0;
category_ = "";
name_ = "";
vendor_ = "";
sku_ = "";
// Default constructor
InventoryData() { clear(); }
// Constructor from a void *
// For use with the data returned from a bdb get
InventoryData(void *buffer)
char buf = (char )buffer;
price_ = *((double *)buf);
bufLen_ = sizeof(double);
quantity_ = *((long *)(buf + bufLen_));
bufLen_ += sizeof(long);
name_ = buf + bufLen_;
bufLen_ += name_.size() + 1;
sku_ = buf + bufLen_;
bufLen_ += sku_.size() + 1;
category_ = buf + bufLen_;
bufLen_ += category_.size() + 1;
vendor_ = buf + bufLen_;
bufLen_ += vendor_.size() + 1;
* Marshalls this classes data members into a single
* contiguous memory location for the purpose of storing
* the data in a database.
char *
getBuffer()
// Zero out the buffer
memset(databuf_, 0, 500);
* Now pack the data into a single contiguous memory location for
* storage.
bufLen_ = 0;
int dataLen = 0;
     dataLen = sizeof(double);
     memcpy(databuf_, &price_, dataLen);
     bufLen_ += dataLen;
     dataLen = sizeof(long);
     memcpy(databuf_ + bufLen_, &quantity_, dataLen);
     bufLen_ += dataLen;
packString(databuf_, name_);
packString(databuf_, sku_);
packString(databuf_, category_);
packString(databuf_, vendor_);
return (databuf_);
* Returns the size of the buffer. Used for storing
* the buffer in a database.
inline size_t getBufferSize() { return (bufLen_); }
/* Utility function used to show the contents of this class */
void
show() {
std::cout << "\nName: " << name_ << std::endl;
std::cout << " SKU: " << sku_ << std::endl;
std::cout << " Price: " << price_ << std::endl;
std::cout << " Quantity: " << quantity_ << std::endl;
std::cout << " Category: " << category_ << std::endl;
std::cout << " Vendor: " << vendor_ << std::endl;
private:
* Utility function that appends a char * to the end of
* the buffer.
void
packString(char *buffer, std::string &theString)
size_t string_size = theString.size() + 1;
memcpy(buffer+bufLen_, theString.c_str(), string_size);
bufLen_ += string_size;
/* Data members */
std::string category_, name_, vendor_, sku_;
double price_;
long quantity_;
size_t bufLen_;
char databuf_[500];
//Forward declarations
void loadDB(Db &, std::string &);
int get_item_name(Db dbp, const Dbt pkey, const Dbt pdata, Dbt skey);
int get_category_name(Db dbp, const Dbt pkey, const Dbt pdata, Dbt skey);
int show_join(Db &item_index, Db &category_index,
     std::string &itemName, std::string &categoryName);
int main (){
Db primary_database(NULL, 0); // Primary
Db item_index(NULL, 0); // Secondary
Db category_index(NULL, 0); // Secondary
// Open the primary database
primary_database.open(NULL,
               "inventorydb.db",
               NULL,
               DB_BTREE,
               DB_CREATE,
               0);
/* // Setup the secondary to use sorted duplicates.
// This is often desireable for secondary databases.
item_index.set_flags(DB_DUPSORT);
category_index.set_flags(DB_DUPSORT);
// Open secondary databases
item_index.open(NULL,
          "itemname.sdb",
          NULL,
          DB_BTREE,
          DB_CREATE,
          0);
category_index.open(NULL,
          "categoryname.sdb",
          NULL,
          DB_BTREE,
          DB_CREATE,
          0);
// Associate the primary and the secondary dbs
primary_database.associate(NULL,
               &item_index,
               get_item_name,
               0);
primary_database.associate(NULL,
               &category_index,
               get_category_name,
               0);
// Load database
std::string input_file = "./small_inventory.txt";
try {
loadDB(primary_database, input_file);
} catch(DbException &e) {
std::cerr << "Error loading databases. " << std::endl;
std::cerr << e.what() << std::endl;
return (e.get_errno());
} catch(std::exception &e) {
std::cerr << "Error loading databases. " << std::endl;
std::cerr << e.what() << std::endl;
return (-1);
// Perform join query
std::string itemName = "Spinach";
std::string categoryName = "vegetables";
show_join(item_index, category_index, itemName, categoryName);
// Close dbs
item_index.close(0);
category_index.close(0);
primary_database.close(0);
return(0);
} // End main
// Used to locate the first pound sign (a field delimiter)
// in the input string.
size_t
getNextPound(std::string &theString, std::string &substring)
size_t pos = theString.find("#");
substring.assign(theString, 0, pos);
theString.assign(theString, pos + 1, theString.size());
return (pos);
// Loads the contents of the inventory.txt file into a database
void
loadDB(Db &inventoryDB, std::string &inventoryFile)
InventoryData inventoryData;
std::string substring;
size_t nextPound;
std::ifstream inFile(inventoryFile.c_str(), std::ios::in);
if ( !inFile )
std::cerr << "Could not open file '" << inventoryFile
<< "'. Giving up." << std::endl;
throw std::exception();
while (!inFile.eof())
inventoryData.clear();
std::string stringBuf;
std::getline(inFile, stringBuf);
// Now parse the line
if (!stringBuf.empty())
nextPound = getNextPound(stringBuf, substring);
inventoryData.setName(substring);
nextPound = getNextPound(stringBuf, substring);
inventoryData.setSKU(substring);
nextPound = getNextPound(stringBuf, substring);
inventoryData.setPrice(strtod(substring.c_str(), 0));
nextPound = getNextPound(stringBuf, substring);
inventoryData.setQuantity(strtol(substring.c_str(), 0, 10));
nextPound = getNextPound(stringBuf, substring);
inventoryData.setCategory(substring);
nextPound = getNextPound(stringBuf, substring);
inventoryData.setVendor(substring);
void buff = (void )inventoryData.getSKU().c_str();
size_t size = inventoryData.getSKU().size()+1;
Dbt key(buff, (u_int32_t)size);
buff = inventoryData.getBuffer();
size = inventoryData.getBufferSize();
Dbt data(buff, (u_int32_t)size);
inventoryDB.put(NULL, &key, &data, 0);
inFile.close();
int
get_item_name(Db dbp, const Dbt pkey, const Dbt pdata, Dbt skey)
* First, obtain the buffer location where we placed the item's name. In
* this example, the item's name is located in the primary data. It is the
* first string in the buffer after the price (a double) and the quantity
* (a long).
u_int32_t offset = sizeof(double) + sizeof(long);
char itemname = (char )pdata->get_data() + offset;
// unused
(void)pkey;
* If the offset is beyond the end of the data, then there was a problem
* with the buffer contained in pdata, or there's a programming error in
* how the buffer is marshalled/unmarshalled. This should never happen!
if (offset > pdata->get_size()) {
dbp->errx("get_item_name: buffer sizes do not match!");
// When we return non-zero, the index record is not added/updated.
return (-1);
/* Now set the secondary key's data to be the item name */
skey->set_data(itemname);
skey->set_size((u_int32_t)strlen(itemname) + 1);
return (0);
int
get_category_name(Db dbp, const Dbt pkey, const Dbt pdata, Dbt skey)
* First, obtain the buffer location where we placed the item's name. In
* this example, the item's name is located in the primary data. It is the
* first string in the buffer after the price (a double) and the quantity
* (a long).
u_int32_t offset = sizeof(double) + sizeof(long);
char itemname = (char )pdata->get_data() + offset;
offset += strlen(itemname) + 1;
char sku = (char )pdata->get_data() + offset;
offset += strlen(sku) + 1;
char category = (char )pdata->get_data() + offset;
// unused
(void)pkey;
* If the offset is beyond the end of the data, then there was a problem
* with the buffer contained in pdata, or there's a programming error in
* how the buffer is marshalled/unmarshalled. This should never happen!
if (offset > pdata->get_size()) {
dbp->errx("get_item_name: buffer sizes do not match!");
// When we return non-zero, the index record is not added/updated.
return (-1);
/* Now set the secondary key's data to be the item name */
skey->set_data(category);
skey->set_size((u_int32_t)strlen(category) + 1);
return (0);
int
show_join(Db &itemnameSDB, Db &catnameSDB,
     std::string &itemName, std::string &categoryName)
std::cout << "Have item : " << itemName << " and category : "
     << categoryName << std::endl;
// Position cursor at item
int ret;
Dbc *item_curs;
Dbt key, data;
try {
itemnameSDB.cursor(NULL, &item_curs, 0);
char * c_item = (char *)itemName.c_str();
key.set_data(c_item);
key.set_size(strlen(c_item) + 1);
if ((ret = item_curs->get(&key, &data, DB_SET)) != 0)
     std::cout << "Did not find any records matching item ["
          << c_item << "]" << std::endl;
// while (ret != DB_NOTFOUND)
//      printf("Database record --\n");
//     std::cout << "Key : " << (char *)key.get_data() << std::endl;
//      ret = item_curs->get(&key, &data, DB_NEXT_DUP);
catch(DbException &e) {        
itemnameSDB.err(e.get_errno(), "Error!");
} catch(std::exception &e) {
itemnameSDB.errx("Error! %s", e.what());
// Position cursor at category
Dbc *category_curs;
try {
catnameSDB.cursor(NULL, &category_curs, 0);
char c_category = (char )categoryName.c_str();
key.set_data(c_category);
key.set_size(strlen(c_category) + 1);
if ((ret = category_curs->get(&key, &data, DB_SET)) != 0)
     std::cout << "Did not find any records matching category ["
          << c_category << "]" << std::endl;
//!! Debug, print everything
// Dbt temp_key, temp_data;
// while ((ret = category_curs->get(&temp_key, &temp_data, DB_NEXT)) == 0) {        
// std::cout << "Key : " << (char *)temp_key.get_data() << std::endl;
catch(DbException &e) {        
catnameSDB.err(e.get_errno(), "Error!");
} catch(std::exception &e) {
catnameSDB.errx("Error! %s", e.what());
// Set up an array of cursors ready for the join
Dbc *carray[3];
carray[0] = item_curs;
carray[1] = category_curs;
carray[3] = NULL;
// Perform the join
Dbc *join_curs;
try {
if ((ret = itemnameSDB.join(carray, &join_curs, 0)) != 0)
     std::cout << "Successful query results should go here." << std::endl;
catch(DbException &e) {        
itemnameSDB.err(e.get_errno(), "Error[3]!");
} catch(std::exception &e) {
itemnameSDB.errx("Error! %s", e.what());
// Iterate through results using the join cursor
while ((ret = join_curs->get(&key, &data, 0)) == 0)
std::cout << "Iterating through cursors" << std::endl;
// If we exited the loop because we ran out of records,
// then it has completed successfully.
if (ret == DB_NOTFOUND)
item_curs->close();
category_curs->close();
join_curs->close();
return(0);
}