SIGSEGV, com.sleepycat.dbxml.dbxml_javaJNI.XmlInputStream_change_ownership

Question

Hello,
I'm experimenting crashes on adding a document, on RHEL5 64bit with both Sun JDK 1.5 and 1.6; and on RHEL4 32bit with JDK 1.5
Selinux is disabled on both RHEL.
The essential part of the jvm error is:
# An unexpected error has been detected by Java Runtime Environment:
# SIGSEGV (0xb) at pc=0x00002aaaaaf2124a, pid=22759, tid=1102199104
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.6.0_01-b06 mixed mode)
# Problematic frame:
# V [libjvm.so+0x35124a]
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j com.sleepycat.dbxml.dbxml_javaJNI.XmlInputStream_change_ownership(Lcom/sleepycat/dbxml/XmlInputStream;JZ)V+0
j com.sleepycat.dbxml.XmlInputStream.swigReleaseOwnership()V+11
j com.sleepycat.dbxml.XmlDocument.setContentAsXmlInputStream(Lcom/sleepycat/dbxml/XmlInputStream;)V+12
j uk.ac.open.kmi.watson.crawler.dbxml.DbXMLDriver.processFile(Ljava/io/File;)V+169
j uk.ac.open.kmi.watson.crawler.processor.DbXMLWriter.innerProcess(Lorg/archive/crawler/datamodel/CrawlURI;)V+241
j org.archive.crawler.framework.Processor.process(Lorg/archive/crawler/datamodel/CrawlURI;)V+50
j org.archive.crawler.framework.ToeThread.processCrawlUri()V+127
j org.archive.crawler.framework.ToeThread.run()V+97
v ~StubRoutines::call_stub
In java I'm adding a document as:
XmlInputStream theStream = myManager.createLocalFileInputStream("/opt/watson/test.xml");
myDoc.setContentAsXmlInputStream(theStream);
and I have the above issues, if I drop XmlInputStream and just set the content as:
myDoc.setContent("<foo />");
Everything works, even on 50 threads.
Anybody encountered similar issues?
Thank you.
Laurian
Message was edited by:
user555161

Answer 1

Hi,
This is the output of the tests, java file put after:
MAC OS X
$ java -version
java version "1.5.0_07"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_07-154)
Java HotSpot(TM) Client VM (build 1.5.0_07-87, mixed mode, sharing)
RUN 1
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1
Dictionary - crawler - Define new name 1 -> {name,http://www.sleepycat.com/2002/dbxml}
Dictionary - crawler - Define new name 2 -> {root,http://www.sleepycat.com/2002/dbxml}
Dictionary - crawler - Define new name 3 -> dbxml
Dictionary - crawler - Define new name 4 -> http://www.sleepycat.com/2002/dbxml
Dictionary - crawler - Define new name 5 -> xml
Dictionary - crawler - Define new name 6 -> xmlns
Dictionary - crawler - Define new name 7 -> xsi
Dictionary - crawler - Define new name 8 -> xsd
Dictionary - crawler - Define new name 9 -> xs
Dictionary - crawler - Define new name 10 -> xlink
Dictionary - crawler - Define new name 11 -> xhtml
Dictionary - crawler - Define new name 12 -> http://www.w3.org/XML/1998/namespace/
Dictionary - crawler - Define new name 13 -> http://www.w3.org/XML/2000/xmlns/
Dictionary - crawler - Define new name 14 -> http://www.w3.org/2000/xmlns/
Dictionary - crawler - Define new name 15 -> http://www.w3.org/1999/xlink
Dictionary - crawler - Define new name 16 -> http://www.w3.org/2001/XMLSchema-instance
Dictionary - crawler - Define new name 17 -> http://www.w3.org/2001/XMLSchema
Dictionary - crawler - Define new name 18 -> http://www.w3.org/1999/xhtml
Dictionary - crawler - Define new name 19 -> id
Dictionary - crawler - Define new name 20 -> name
Container - crawler - Document storage container opened.
DB opened/created!
args:
test1 - for ading XML from a file (path as argument 2)
test2 - for ading XML from a string
test3 - for ading RDF/XML, namespace issue
Container - crawler - container closed.
RUN 2 - test 1
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test1 /opt/watson/test.xml
Container - crawler - Document storage container opened.
Container - crawler - Adding document: foo1176934467482
Dictionary - crawler - Define new name 51 -> http://maven.apache.org/POM/4.0.0
Dictionary - crawler - Define new name 52 -> {created,http://watson.kmi.open.ac.uk/crawler#}
Indexer - crawler - add unique-node-metadata-equality-string, key={Size=12 Hex=5a01666f6f31313736393334343637343832}, data={Size=3 Hex=000200}
Container - crawler - Added document successfully: foo1176934467482
Container - crawler - container closed.
java(1497,0x1804a00) malloc: *** Deallocation of a pointer not malloced: 0x333a00; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug
RUN 3 - test 2
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test2
Container - crawler - Document storage container opened.
Container - crawler - Adding document: bar1176934723078
Indexer - crawler - add unique-node-metadata-equality-string, key={Size=12 Hex=5a0162617231313736393334373233303738}, data={Size=3 Hex=002000}
Container - crawler - Added document successfully: bar1176934723078
Container - crawler - container closed.
RUN 4 - test 3
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test3
Container - crawler - Document storage container opened.
Container - crawler - Adding document: bar1176934775465
Dictionary - crawler - Define new name 53 -> http://www.w3.org/1999/02/22-rdf-syntax-ns#
Dictionary - crawler - Define new name 54 -> http://www.recshop.fake/cd#
Indexer - crawler - add unique-node-metadata-equality-string, key={Size=12 Hex=5a0162617231313736393334373735343635}, data={Size=3 Hex=002a00}
Container - crawler - Added document successfully: bar1176934775465
Container - crawler - container closed.
==============================
RHEL 5
$ java -version
java version "1.6.0_01"
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_01-b06, mixed mode)
RUN 1
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1
Dictionary - crawler - Define new name 1 -> {name,http://www.sleepycat.com/2002/dbxml}
Dictionary - crawler - Define new name 2 -> {root,http://www.sleepycat.com/2002/dbxml}
Dictionary - crawler - Define new name 3 -> dbxml
Dictionary - crawler - Define new name 4 -> http://www.sleepycat.com/2002/dbxml
Dictionary - crawler - Define new name 5 -> xml
Dictionary - crawler - Define new name 6 -> xmlns
Dictionary - crawler - Define new name 7 -> xsi
Dictionary - crawler - Define new name 8 -> xsd
Dictionary - crawler - Define new name 9 -> xs
Dictionary - crawler - Define new name 10 -> xlink
Dictionary - crawler - Define new name 11 -> xhtml
Dictionary - crawler - Define new name 12 -> http://www.w3.org/XML/1998/namespace/
Dictionary - crawler - Define new name 13 -> http://www.w3.org/XML/2000/xmlns/
Dictionary - crawler - Define new name 14 -> http://www.w3.org/2000/xmlns/
Dictionary - crawler - Define new name 15 -> http://www.w3.org/1999/xlink
Dictionary - crawler - Define new name 16 -> http://www.w3.org/2001/XMLSchema-instance
Dictionary - crawler - Define new name 17 -> http://www.w3.org/2001/XMLSchema
Dictionary - crawler - Define new name 18 -> http://www.w3.org/1999/xhtml
Dictionary - crawler - Define new name 19 -> id
Dictionary - crawler - Define new name 20 -> name
Container - crawler - Document storage container opened.
DB opened/created!
args:
test1 - for ading XML from a file (path as argument 2)
test2 - for ading XML from a string
test3 - for ading RDF/XML, namespace issue
Container - crawler - container closed.
RUN 2 - test 1
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test1 /opt/watson/test.xml
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test1 /opt/watson/test.xml
Container - crawler - Document storage container opened.
Container - crawler - Adding document: foo1176938860942
Dictionary - crawler - Define new name 51 -> http://maven.apache.org/POM/4.0.0
Dictionary - crawler - Define new name 52 -> {created,http://watson.kmi.open.ac.uk/crawler#}
Indexer - crawler - add unique-node-metadata-equality-string, key={Size=12 Hex=5a01666f6f31313736393338383630393432}, data={Size=3 Hex=000200}
Container - crawler - Added document successfully: foo1176938860942
Container - crawler - container closed.
RUNNING AGAIN
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test1 /opt/watson/test.xml
Container - crawler - Document storage container opened.
# An unexpected error has been detected by Java Runtime Environment:
# SIGSEGV (0xb) at pc=0x00002aaaaaf2124a, pid=6114, tid=1075882304
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.6.0_01-b06 mixed mode)
# Problematic frame:
# V [libjvm.so+0x35124a]
# An error report file with more information is saved as hs_err_pid6114.log
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
Aborted
In hs_err_pid6114.log:
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j com.sleepycat.dbxml.dbxml_javaJNI.XmlInputStream_change_ownership(Lcom/sleepycat/dbxml/XmlInputStream;JZ)V+0
j com.sleepycat.dbxml.XmlInputStream.swigReleaseOwnership()V+11
j com.sleepycat.dbxml.XmlDocument.setContentAsXmlInputStream(Lcom/sleepycat/dbxml/XmlInputStream;)V+12
j Test1.test1(Ljava/lang/String;)V+150
j Test1.main([Ljava/lang/String;)V+32
v ~StubRoutines::call_stub
RUN 3 - test 2
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test2
Container - crawler - Document storage container opened.
Container - crawler - Adding document: bar1176939020231
Indexer - crawler - add unique-node-metadata-equality-string, key={Size=12 Hex=5a0162617231313736393339303230323331}, data={Size=3 Hex=000700}
Container - crawler - Added document successfully: bar1176939020231
Container - crawler - container closed.
RUN 4 - test 3
$ java -Djava.library.path=/opt/watson/usr/dbxml/lib -cp build/classes:../../dbxml/lib/db.jar:../../dbxml/lib/dbxml.jar Test1 test3
Container - crawler - Document storage container opened.
Container - crawler - Adding document: bar1176939053875
Indexer - crawler - add unique-node-metadata-equality-string, key={Size=12 Hex=5a0162617231313736393339303533383735}, data={Size=3 Hex=001100}
Container - crawler - Added document successfully: bar1176939053875
Container - crawler - container closed.
JAVA FILE:
import com.sleepycat.db.Environment;
import com.sleepycat.db.EnvironmentConfig;
import com.sleepycat.db.LockDetectMode;
import com.sleepycat.db.Transaction;
import com.sleepycat.db.TransactionConfig;
import com.sleepycat.dbxml.XmlContainer;
import com.sleepycat.dbxml.XmlContainerConfig;
import com.sleepycat.dbxml.XmlDocument;
import com.sleepycat.dbxml.XmlException;
import com.sleepycat.dbxml.XmlInputStream;
import com.sleepycat.dbxml.XmlManager;
import com.sleepycat.dbxml.XmlManagerConfig;
import com.sleepycat.dbxml.XmlTransaction;
import com.sleepycat.dbxml.XmlUpdateContext;
import com.sleepycat.dbxml.XmlValue;
import java.io.File;
import java.text.SimpleDateFormat;
import java.util.GregorianCalendar;
import java.util.TimeZone;
* Test1.java
* Created on April 18, 2007, 10:43 PM
* To change this template, choose Tools | Template Manager
* and open the template in the editor.
* @author lg3388
public class Test1 {
public static final File TMP = new File("/tmp");
public static final File ENV = new File(TMP, "/foo");
static {
ENV.mkdirs();
public static final String container = "crawler";
private Environment myEnv;
private EnvironmentConfig envConf;
private XmlManager myManager;
private XmlContainer myContainer;
public void initialTasks() throws Exception {
EnvironmentConfig envConf = new EnvironmentConfig();
envConf.setErrorStream(System.err);
envConf.setAllowCreate(true);
//envConf.setRunRecovery(true);
envConf.setJoinEnvironment(true);
envConf.setInitializeLocking(true);
envConf.setInitializeLogging(true);
envConf.setInitializeCache(true);
envConf.setTransactional(true);
envConf.setLockDetectMode(LockDetectMode.DEFAULT);
envConf.setPrivate(false);
envConf.setTxnMaxActive(64);
envConf.setTxnTimeout(60*1000000);
envConf.setThreaded(true);
myEnv = new Environment(ENV, envConf);
XmlManagerConfig managerConfig = new XmlManagerConfig();
managerConfig.setAdoptEnvironment(true);
myManager = new XmlManager(myEnv, managerConfig);
myManager.setLogLevel(XmlManager.LEVEL_ALL, true);
myManager.setLogCategory(XmlManager.CATEGORY_ALL, true);
XmlContainerConfig containerConf = new XmlContainerConfig();
containerConf.setNodeContainer(false);
containerConf.setAllowValidation(false);
containerConf.setTransactional(true);
containerConf.setAllowCreate(true);
if ( myManager.existsContainer(container) != 0 ) {
myContainer = myManager.openContainer(container, containerConf);
} else {
myContainer = myManager.createContainer(container, containerConf);
public void finalTasks() throws Exception {
myContainer.close();
public void test1(String fileName) throws Exception {
// UTC unix timestamp
GregorianCalendar time = new GregorianCalendar(TimeZone.getTimeZone("UTC"));
long stamp = time.getTimeInMillis();
Transaction tn = null;
TransactionConfig tc = new TransactionConfig();
//tc.setNoWait(true);
tn = myEnv.beginTransaction(null, tc);//
XmlTransaction txn = null;
try {
txn = myManager.createTransaction(tn);
XmlUpdateContext theContext = myManager.createUpdateContext();
String URI = "http://watson.kmi.open.ac.uk/crawler#";
String attrName = "created";
XmlValue attrValue = new XmlValue(XmlValue.DATE_TIME, new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'").format(time.getTime()));
// @@@@@ THIS IS NOT IN TEST 2
// Get the input stream.
XmlInputStream theStream = myManager.createLocalFileInputStream(fileName);
// Get an XmlDocument
XmlDocument myDoc = myManager.createDocument();
// Set the document's name
myDoc.setName("foo" + Long.toString(stamp));
// @@@@@ THIS IS NOT IN TEST 2
// Set the content
myDoc.setContentAsXmlInputStream(theStream);
// Set the metadata
myDoc.setMetaData(URI, attrName, attrValue);
// Put the document into the container
myContainer.putDocument(txn, myDoc, theContext, null);
//free resources?
myDoc.delete();
txn.commit();
// free resources?
txn.delete();
} catch (XmlException e) {
e.printStackTrace();
if (txn != null) txn.abort();
public void test2(String content) throws Exception {
// UTC unix timestamp
GregorianCalendar time = new GregorianCalendar(TimeZone.getTimeZone("UTC"));
long stamp = time.getTimeInMillis();
Transaction tn = null;
TransactionConfig tc = new TransactionConfig();
//tc.setNoWait(true);
tn = myEnv.beginTransaction(null, tc);//
XmlTransaction txn = null;
try {
txn = myManager.createTransaction(tn);
XmlUpdateContext theContext = myManager.createUpdateContext();
String URI = "http://watson.kmi.open.ac.uk/crawler#";
String attrName = "created";
XmlValue attrValue = new XmlValue(XmlValue.DATE_TIME, new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'").format(time.getTime()));
// Get an XmlDocument
XmlDocument myDoc = myManager.createDocument();
// Set the document's name
myDoc.setName("bar" + Long.toString(stamp));
// @@@@ THIS IS THE DIFF TO TEST 1
// Set the content
myDoc.setContent(content);
// Set the metadata
myDoc.setMetaData(URI, attrName, attrValue);
// Put the document into the container
myContainer.putDocument(txn, myDoc, theContext, null);
//free resources?
myDoc.delete();
txn.commit();
// free resources?
txn.delete();
} catch (XmlException e) {
e.printStackTrace();
if (txn != null) txn.abort();
public static void main(String[] args) throws Exception {
Test1 test = new Test1();
test.initialTasks();
if (args.length > 0) {
if ("test1".equals(args[0])) test.test1(args[1]);
if ("test2".equals(args[0])) test.test2("<bar />");
if ("test3".equals(args[0])) test.test2(
"<rdf:RDF"
+ " xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' "
+ "xmlns:cd='http://www.recshop.fake/cd#'> "
+ "<rdf:Description"
+ " rdf:about='http://www.recshop.fake/cd/Empire Burlesque'>"
+ " <cd:artist>Bob Dylan</cd:artist>"
+ " <cd:country>USA</cd:country>"
+ " <cd:company>Columbia</cd:company>"
+ " <cd:price>10.90</cd:price>"
+ " <cd:year>1985</cd:year>"
+ "</rdf:Description></rdf:RDF>"
} else {
System.out.println("DB opened/created!");
System.out.println("args:");
System.out.println("test1 - for ading XML from a file (path as argument 2)");
System.out.println("test2 - for ading XML from a string");
System.out.println("test3 - for ading RDF/XML, namespace issue");
test.finalTasks();
}