Problems with environments and transactional mode in the Python API

Question

Hello everyone,
I have been having problems with the Python API, and I wonder if anyone can comment? I am using DB XML version 2.2.13, with python 2.3 (RedHat linux WS 4.0) and python 2.4 (SuSE 10.1) with identical results. I started with a simple example:
#!/usr/bin/python
"""XPath example from http://www.w3schools.com/xpath/default.asp
ported to DbXML
from bsddb3.db import *
from dbxml import *
books = """<?xml version="1.0" encoding="ISO-8859-1"?>
<bookstore>
  [xml data omitted for brevity]
</bookstore>"""
if __name__ == "__main__":
    conFlags = DB_CREATE | DB_NOMMAP
    myMgr = XmlManager()
    myMgr.setDefaultContainerFlags(conFlags)
    myMgr.setDefaultContainerType( XmlContainer.NodeContainer)
    uc = myMgr.createUpdateContext()
    container = myMgr.createContainer("books.dbxml")
    container.putDocument("books.xml", books, uc)So far, so good - this works, and creates a container books.dbxml that I can open and query. Next, I try to instantiate the XmlManager using an environment:
if __name__ == "__main__":
    envFlags = DB_CREATE | DB_PRIVATE
    conFlags = DB_CREATE | DB_EXCL | DB_NOMMAP
    myEnv = DBEnv()
    myEnv.open("/home/pkeller/dbxml_tests", envFlags, 0)
    myMgr = XmlManager(myEnv, DBXML_ADOPT_DBENV)
    myMgr.setDefaultContainerFlags(conFlags)
    myMgr.setDefaultContainerType( XmlContainer.NodeContainer)
    uc = myMgr.createUpdateContext()
    container = myMgr.createContainer("books.dbxml")
    container.putDocument("books.xml", books, uc)This fails with the following output:
Traceback (most recent call last):
  File "w3school_xpath_c2.py", line 60, in ?
    container = myMgr.createContainer("books.dbxml")
  File "/scratch_bernoulli/pkeller/dbxml/install/lib/python2.3/site-packages/dbxml.py", line 125, in createContainer
    def createContainer(*args): return dbxml.XmlManagercreateContainer(*args)
RuntimeError: Error: Invalid argument
Segmentation faultA bit odd - I can't find anything in the docs about the required arguments to XmlManager.createContainer being different if an environment has been used explicitly.
Anyway, I persevered (my aim being to use transactional mode in Python). Changing the environment and container flags like so:
    envFlags = DB_CREATE  | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_PRIVATE
    conFlags = DB_CREATE  | DB_EXCL | DBXML_TRANSACTIONAL | DB_NOMMAPThis change allowed the script to run, outputting the single classic line:
Segmentation fault(the C++ API docs at http://www.sleepycat.com/xmldocs/gsg_xml/cxx/transaction.html#initializetransaction are incorrect by the way - there is no flag DB_TRANSACTIONAL).
In spite of the segfault, the file "books.dbxml" was produced and could be queried by another application. A transaction log file "log.0000000001" was also written (10485760 bytes long).
Running this last example again with "python -v" showed that the segfault was being produced during the python's cleanup phase:
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
Segmentation faultCan anyone clarify what is going on? I had been hoping to deploy DB XML as an alternative to something more admin-heavy, but with this behaviour that would be hard to justify. A dirty exit from Python means that no-one will consider trusting their data to this engine (or am I doing something wrong?).
Regards,
Peter.

Answer 1

Peter,
Your first failure -- invalid argument -- was due to an invalid combination of
flags to DBEnv.open(). Along with DB_CREATE you need at least DB_INIT_MPOOL.
As you can see, it worked once you added more flags.
The second, cleanup problem (segmentation fault) is the result of out of order
destruction of objects. This is fixed in the next release (later this year), but
you can work around it by ensuring that your XmlContainer and XmlManager
objects are deleted/cleaned up before the DBEnv object.
Python will cleanup in reverse order of creation, unless you perform explicit
deletions, which is also fine (e.g. del container, del myMgr).
Explicit deletion is always safe.
Regards,
George