SAXParser.parse(resder, ...) ???
Hi;
Is there any way to pass SAXParser.parse() a Reader instead of an InputStream? I thought we were not supposed to use InputStreams anymore.
thanks - dave
Is there any way to pass SAXParser.parse() a Reader instead of an InputStream? The SAX parser should implement a mechanisim to check the character encoding declared in the the XML prefix is compatible with the streamed octet data. If you use Reader for untrusted sources (ie pretty well anything other than StringReader, where the character encoding is well defined), you circumvent that error check.
I thought we were not supposed to use InputStreams anymore.Part of the role of an XML parser in dealing with the character encoding, so it is appropriate to use the stream that provide octets which encode characters rather than a reader that provides characters.
Pete
Similar Messages
-
Problems with using SAXParser.parse with an InputStream
Hello:
I have successfully run the echo01 program from the XML tutorial. I made the following changes to parse the output from a servlet which generates an XML document.
URL recordedSensor = new URL("http://redbd01:8010/servlet/RecordedSensor?vin=1M000000000000001");
URLConnection rsConn = recordedSensor.openConnection();
// Parse the input
SAXParser saxParser = factory.newSAXParser();
saxParser.parse( rsConn.getInputStream(), handler);
I get the following errors when I try to run the code.
org.xml.sax.SAXParseException: java.lang.NullPointerException
at org.apache.crimson.parser.Parser2.parseInternal(Parser2.java:524)
at org.apache.crimson.parser.Parser2.parse(Parser2.java:305)
at org.apache.crimson.parser.XMLReaderImpl.parse(XMLReaderImpl.java:442)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:393)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:191)
at ServletInterface.main(ServletInterface.java:29)
I think I have the problem narrowed down to the handlers accessing the class variable out which is a Writer the emit() method uses to output the document. When I replace out.write in emit() with System.out.println() the code works.
I've tried accessing other class variables in the handler methods with similar results.
Anybody encountered similar problems? And what is the solution.
Thanks in advance,
GaryI have the same problem, if you find a solution let me know. I've double checked my objects to make sure their not null, I've checked my constructors in the extended default handler, not really sure where else to look, its such a simple program....
-
All validation error from saxParser.parse
I am getting just the 1st error but I want all the errors in the 1st pass itself.
Attaching the code for reference also.
?would really appreciate you suggestion.
==code---
public class EDITExtractSchema {
static Document document;
public void validateXMLVOusingXSD(String str)
try {
//create a SchemaFactory capable of understanding W3C XML Schemas (WXS)
SchemaFactory factory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
//Set the error handler to receive any error during Schema Compilation
XMLErrorHandler errorHandler = new XMLErrorHandler();
factory.setErrorHandler(errorHandler);
//set the resource resolver to customize resource resolution
//factory.setResourceResolver( new MyLSResourceResolver());
// load a WXS schema, represented by a Schema instance
File schemaFile = new File("C:/Temp/project2-CorrVO/XSDs/TaxExtract5498DetailVO.xsd");
Schema schema = factory.newSchema(new StreamSource(schemaFile));
SAXParserFactory spf = SAXParserFactory.newInstance();
//spf.setNamespaceAware(true);
//spf.setValidating(true);
spf.setFeature("http://apache.org/xml/features/validation/schema-full-checking", true);
//Just set the Schema instance on SAXParserFactory
spf.setSchema(schema);
//Obtain the SAXParser instance
SAXParser saxParser = spf.newSAXParser();
//parser will parse the XML document but validate it using Schema instance
//saxParser.parse(new File(str), myHandler);
saxParser.parse(new File(str),new MyDefaulyHandler());
}catch(ParserConfigurationException e) {
System.out.println("ParserConfigurationException-" + e);
}catch (SAXException e) {
System.out.println("SAXException-" + e);
catch (Exception e) {
System.out.println("Exception-" + e);
/*// output the errors XML
XMLWriter writer = null;
try {
writer = new XMLWriter(OutputFormat.createPrettyPrint());
} catch (UnsupportedEncodingException e) {
System.out.println("validate error" + e);
e.printStackTrace();
try {
writer.write(errorHandler.geterrors());
} catch (IOException e) {
System.out.println("validate error" + e);
e.printStackTrace();
//implement error handler
public static class XMLErrorHandler implements ErrorHandler {
private boolean valid = true;
public void reset() {
// Assume document is valid until proven otherwise
valid = true;
public boolean isValid() {
return valid;
public void warning(SAXParseException exception) throws SAXException{
System.out.println("Warning: " + exception.getMessage());
System.out.println(" at line " + exception.getLineNumber()
+ ", column " + exception.getColumnNumber());
System.out.println(" in entity " + exception.getSystemId());
valid = true;
public void error(SAXParseException exception) throws SAXException{
System.out.println("Error: " + exception.getMessage());
System.out.println(" at line " + exception.getLineNumber()
+ ", column " + exception.getColumnNumber());
System.out.println(" in entity " + exception.getSystemId());
valid = true;
/* // output the errors XML
XMLWriter writer = null;
try {
writer = new XMLWriter(OutputFormat.createPrettyPrint());
} catch (UnsupportedEncodingException e) {
System.out.println("validate error" + e);
e.printStackTrace();
try {
writer.write((Object)exception);
} catch (IOException e) {
System.out.println("validate error" + e);
e.printStackTrace();
public void fatalError(SAXParseException exception) throws SAXException {
System.out.println("Fatal Error: " + exception.getMessage());
System.out.println(" at line " + exception.getLineNumber()
+ ", column " + exception.getColumnNumber());
System.out.println(" in entity " + exception.getSystemId());
valid = true;
public static class MyDefaulyHandler extends DefaultHandler {
private boolean valid = true;
public void reset() {
// Assume document is valid until proven otherwise
valid = true;
public boolean isValid() {
return valid;
public void warning(SAXParseException exception) throws SAXException{
System.out.println("Warning: " + exception.getMessage());
System.out.println(" at line " + exception.getLineNumber()
+ ", column " + exception.getColumnNumber());
System.out.println(" in entity " + exception.getSystemId());
valid = true;
public void error(SAXParseException exception) throws SAXException{
System.out.println("Error: " + exception.getMessage());
System.out.println(" at line " + exception.getLineNumber()
+ ", column " + exception.getColumnNumber());
System.out.println(" in entity " + exception.getSystemId());
valid = true;
/* // output the errors XML
XMLWriter writer = null;
try {
writer = new XMLWriter(OutputFormat.createPrettyPrint());
} catch (UnsupportedEncodingException e) {
System.out.println("validate error" + e);
e.printStackTrace();
try {
writer.write((Object)exception);
} catch (IOException e) {
System.out.println("validate error" + e);
e.printStackTrace();
public void fatalError(SAXParseException exception) throws SAXException {
System.out.println("Fatal Error: " + exception.getMessage());
System.out.println(" at line " + exception.getLineNumber()
+ ", column " + exception.getColumnNumber());
System.out.println(" in entity " + exception.getSystemId());
valid = true;
public static void main(String[] args) {
EDITExtractSchema schemaClass = new EDITExtractSchema();
try {
schemaClass.validateXMLVOusingXSD("C:/Temp/project2-CorrVO/XSDs/TaxExtract5498DetailVO.xml");
}catch (Exception e) {
System.out.println("xml file read error Exception-" + e);
--====
Regards,
TapasNo really, it will report all the error if you turn on the validation and implement the error hander methods(error()).
ALSO,
My XSD had "<xs:sequence>" which was causing the problem.
We had to change it to "<xs:all>". Now all the errors are getting reported.
thx! -
SaxParser.parse(sok.getInputStream(), new handler()) [clarification please]
this code works, however i am not sure how/why?
i am interested in how Strings/characters are sent over tcp
streams:
here is sender:
Socket sok = new Socket(server, port);
OutputStreamWriter streamWriter = new OutputStreamWriter(sok.getOutputStream());
streamWriter.write(xmlString, 0, xmlString.length());i am ok with this.
i see that the OutputStreamWriter does "something" about converting the
socket's OutputStream to characters.
here is receiver:
SAXParser parser = saxFactory.newSAXParser();
Socket sok = serverSok.accept();
parser.parse(sok.getInputStream(), new MyHandler());see, i am plugging:
sok.getInputStream()
directly into the sax parser without providing context
that those bytes are characters (and what character encodings ).
so, what is going on? i might imagine using:
ObjectOutputStream.appendClass();
ObjectInputStream.resolveClass();to provide context that the byte stream are
characters of the "x" character set.
however, i am not using any
ObjectStreams.
if this is a ridiculous question, and the answer is
in the api somehow, i am sorry. i am not good
at always understanding the api.then my fundamental confusion is the difference between
java.io.Reader;
java.io.InputStream;
consider the set of polymorphic method signatures for the
SAXParser parse () methods.
parser.parse(sok.getInputStream(), new MyHandler());clearly invokes:
public void parse(InputStream is, DefaultHandler dh);however i don't see any method signature that matches:
parser.parse(new InputStreamReader(sok.getInputStream(), myCharSet), new MyHandler());for example, in the api, there is no:
public void parse(Reader r, DefaultHandler dh);this has been very helpful.
i will Google the interchangability of Reader/InputStream tomorrow, however any
further guidance would be welcomed. the java.io heirarchy confuses me. -
Javax.xml.parsers.SAXParser parser() error
Hi,
I've got some code that works fine in the Forte for Java IDE but just will not work when running as a stand-alone Windows application. I think it's to do with the filename having spaces in the pathname?
Any help would be great!
Regards.
Gary Revell
[email protected]
[email protected]
I've attached the code and the error(s) I get
------------------CODE FRAGMENT------------------
// loadCE.java
// Created on 25 April 2001, 15:43
// @author RevellG
// @version V1.0
// This class is used to load the Standard Call Reporting XML document
// and to allow the easy interrogation of the contents. The data can
// then be used to populate fields in the Call Reporting GUI.
import java.util.Vector;
import java.io.*;
import java.util.*;
import java.util.Collections;
import org.xml.sax.*;
import javax.xml.parsers.SAXParserFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParser;
public class loadCallElement extends Object
private static loadCall loadCall = null;
private static boolean DEBUG = false;
// Creates new loadCE
public loadCallElement(boolean debugFlag )
this.DEBUG = debugFlag;
public void parseDoc( String docName ) throws Throwable
// Use the default (non-validating) parser
SAXParserFactory factory = null;
try
// Parse the input
factory = SAXParserFactory.newInstance();
SAXParser saxParser = factory.newSAXParser();
loadCall = new loadCall(DEBUG);
saxParser.parse( new File( docName ), loadCall );
catch (Throwable t)
// catch ( Exception t )
System.err.println( t.getMessage() +" - <"+docName+">");
t.printStackTrace ();
loadCall.ceTree.clear();
throw t;
------------------Error and Stack Trace---------------
E:\Support\Call Logs\bin>java -classpath .\SCM_GUI.jar;%CLASSPATH%;%XML_HOME%\jaxp.jar;%XML_HOME%\parser.jar M
anagerInterface > g.g
CWD /Telecom/Support/Call Logs/Logs/Open/BBC PICS/001$revellg$qwerty.xml: 550 /Telecom/Support/Call Logs/Logs/
Open/BBC PICS/001$revellg$qwerty.xml: The system cannot find the path specified.
- <\\Reoclu1\Telecom\Support\Call Logs\Logs\Open\BBC PICS\001$revellg$qwerty.xml>
java.io.FileNotFoundException: CWD /Telecom/Support/Call Logs/Logs/Open/BBC PICS/001$revellg$qwerty.xml: 550 /
Telecom/Support/Call Logs/Logs/Open/BBC PICS/001$revellg$qwerty.xml: The system cannot find the path specified
at sun.net.ftp.FtpClient.readReply(Unknown Source)
at sun.net.ftp.FtpClient.issueCommand(Unknown Source)
at sun.net.ftp.FtpClient.issueCommandCheck(Unknown Source)
at sun.net.ftp.FtpClient.cd(Unknown Source)
at sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at com.sun.xml.parser.InputEntity.init(InputEntity.java:141)
at com.sun.xml.parser.Parser.parseInternal(Parser.java:463)
at com.sun.xml.parser.Parser.parse(Parser.java:284)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:155)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:126)
at loadCallElement.parseDoc(loadCallElement.java:52)
at SCMView.create(SCMView.java, Compiled Code)
at EventController$5.actionPerformed(EventController.java:116)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$ForwardActionEvents.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)Thanks Jim,
I've installed JDK V1.3.1 and the problem HAS gone away..... However now JDK V1.3.1 won't install on other Win 2K machines.....
Gary. -
Hello,
I'm trying to parse with SAX a XML file coming from remote device, but i Tomcat shows me error message as:
Apache Tomcat/4.0.4 - HTTP Status 404 -
http:\localhost:8080\fieldgate\output1.xml (Syntax file name or directory or volume is wrong)
code is:
<jsp:useBean id="parser" scope="page" class="beans.xml.sax.SAXParserBean" />
<% Collection elements = parser.parse("http://localhost:8080/fieldgate/output1.xml");
Iterator it = elements.iterator();
%>
.........Someone can help me please?
Thx
RiccardoOk this code is not mine but is taken from a manual:
package beans.xml.sax;
import java.io.FileReader;
import org.xml.sax.Attributes;
import java.io.IOException;
import java.util.Vector;
import org.xml.sax.Attributes;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.xml.sax.XMLReader;
import org.xml.sax.helpers.DefaultHandler;
import org.apache.xerces.parsers.SAXParser;
public class SAXParserBean extends DefaultHandler {
private Vector vector = new Vector();
private SAXElement currentElement = null;
private String elementText;
public Vector parse(String filename) throws SAXException,
IOException {
XMLReader xmlReader = new SAXParser();
FileReader fileReader = new FileReader(filename);
xmlReader.setContentHandler(this);
xmlReader.parse(new InputSource(fileReader));
return vector;
public void startElement(String uri, String localName,
String qName, Attributes attrs) {
currentElement = new SAXElement(uri,localName,qName,attrs);
vector.addElement(currentElement);
elementText = new String();
public void characters(char[] chars, int start, int length) {
if(currentElement != null && elementText != null) {
String value = new String(chars, start, length);
elementText += value;
public void endElement(String uri, String localName,
String qName) {
if(currentElement != null && elementText != null)
currentElement.setValue(elementText.trim());
currentElement = null;
} -
SAXParser.parse never returns!
I'm using sockets to transfer xml data. When I feed my InputSource (containing the xml document) into the SAXPaser.parse method I can see the events being fired correctly (ie: startDocument, characters, startElement,etc) apart from the endDocument event. The parse method is executing all my code but it never returns! What should cause it to return? Do I need to close the socket connection or should it recognise when it has reached the final closing element?
I couldn't find that exact implementation but I looked into the crimson version and it reads until it hits an EOF. It's a reasonable guess that the xerces version does the same. I think it has to read in the whole file becasue it cannot tell whether the document is well-formed otherwise.
-
SaxParser error parsing xml encoded string
I'm trying to parse an XML string with Java's SaxParser. The program
fails at the end of an element or at the beginning of a new element.
Is my XML string okay?
<?xml version="1.0" encoding="utf-8"
?><DTYPE>OVL</DTYPE><DTYPE>IMG</DTYPE>. . .
If fails after </DTYPE> and before the next <DTYPE>
The Sax Parser works fine on files.
Here is the code I use to send the string to the parser :
// Parse the input
SAXParser saxParser = factory.newSAXParser();
InputStream is = new ByteArrayInputStream(stringToParse.getBytes());
saxParser.parse( is, handler );
Here is the error message I get :
org.xml.sax.SAXParseException: Inadmissible sign at the document end<
at org.apache.crimson.parser.Parser2.fatal(Parser2.java:3376)
at org.apache.crimson.parser.Parser2.fatal(Parser2.java:3370)
at org.apache.crimson.parser.Parser2.parseInternal(Parser2.java:673)
at org.apache.crimson.parser.Parser2.parse(Parser2.java:337)
at
org.apache.crimson.parser.XMLReaderImpl.parse(XMLReaderImpl.java:448)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:345)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:143)
at EchoSaxParser.parse(EchoSaxParser.java:51)
at StringParser.parse(StringParser.java:23)
at Servant.stringparse(Servant.java:30)
at TEST._TestImplBase._invoke(_TestImplBase.java:43)
at
com.sun.corba.se.internal.corba.ServerDelegate.dispatch(ServerDelegate.java�:353)
at com.sun.corba.se.internal.iiop.ORB.process(ORB.java:280)
at
com.sun.corba.se.internal.iiop.RequestProcessor.process(RequestProcessor.ja�va:81)
at
com.sun.corba.se.internal.orbutil.ThreadPool$PooledThread.run(ThreadPool.ja�va:106)
Any help would be greatly appreciated!
Thank you!<?xml version="1.0" encoding="utf-8" ?><DTYPE>IMG</DTYPE><DTYPE>REF</DTYPE>Above is a short XML string that causes the problem.
It's all one line.
I also tried adding System.getProperty("line.separator") add the
end of each tag so I get :
<?xml version="1.0" encoding="utf-8" ?>
<DTYPE>IMG</DTYPE>
<DTYPE>REF</DTYPE>But same error. It still crashes after the first <DTYPE></DTYPE> tag.
Any help greatly appreciated! -
Recommended way to abort parsing in SAXParser
Hi,
I'm having a usecase where I'm parsing an XML document using SAXParser. In this usecase I'm interested in just finding out the values of few nodes, and I want to stop/abort the SAXParser parser as soon as I have found those values.
One way I could do it was to throw the SAXException to make the parser abort at the point when I have found the node values I was looking for. Second way I tried was to call reset() method on the SAXParser instance. The SAXException approach is not a clean one and for the second one I'm not sure about its performance usefulness.
I'll appreiate if someone could share with me the recommended way to stop/abort the parsing in such a situation.
RegardsThrowing an Exception seems reasonable
-
I've got my program ready to handle XML, but instead of parsing a file, I want to parse just a string.
saxParser.parse( ???, handler);
what should I pass in there?You should putnew InputSource(new StringReader(yourString))in there.
-
How to validate XML against XSD and parse/save in one step using SAXParser?
How to validate XML against XSD and parse/save in one step using SAXParser?
I currently have an XML file and XSD. The XML file specifies the location of the XSD. In Java code I create a SAXParser with parameters indicating that it needs to validate the XML. However, SAXParser.parse does not validate the XML, but it does call my handler functions which save the elements/attributes in memory as it is read. On the other hand, XMLReader.parse does validate the XML against the XSD, but does not save the document in memory.
My code can call XMLReader.parse to validate the XML followed by SAXParser.parse to save the XML document in memory. But this sound inefficient. Besides, while a valid document is being parsed by XMLReader, it can be changed to be invalid and saved, and XMLReader.parse would be looking at the original file and would think that the file is OK, and then SAXParser.parse would parse the document without errors.
<Book xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="book.xsd" name="MyBook">
<Chapter name="First Chapter"/>
<Chapter name="Second Chapter">
<Section number="1"/>
<Section number="2"/>
</Chapter>
</Book>
<?xml version="1.0" encoding="ISO-8859-1" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Book">
<xs:complexType>
<xs:sequence>
<xs:element name="Chapter" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="Section" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="xnumber"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="name"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="name"/>
</xs:complexType>
</xs:element>
</xs:schema>
public class SAXXMLParserTest
public static void main(String[] args)
try
SAXParserFactory factory = SAXParserFactory.newInstance();
factory.setNamespaceAware(true);
factory.setValidating(true);
SAXParser parser = factory.newSAXParser();
parser.setProperty("http://java.sun.com/xml/jaxp/properties/schemaLanguage",
"http://www.w3.org/2001/XMLSchema");
BookHandler handler = new BookHandler();
XMLReader reader = parser.getXMLReader();
reader.setErrorHandler(handler);
parser.parse("xmltest.dat", handler); // does not throw validation error
Book book = handler.getBook();
System.out.println(book);
reader.parse("xmltest.dat"); // throws validation error because of 'xnumber' in the XSD
public class Book extends Element
private String name;
private List<Chapter> chapters = new ArrayList<Chapter>();
public Book(String name)
this.name = name;
public void addChapter(Chapter chapter)
chapters.add(chapter);
public String toString()
StringBuilder builder = new StringBuilder();
builder.append("<Book name=\"").append(name).append("\">\n");
for (Chapter chapter: chapters)
builder.append(chapter.toString());
builder.append("</Book>\n");
return builder.toString();
public static class BookHandler extends DefaultHandler
private Stack<Element> root = null;
private Book book = null;
public void startDocument()
root = new Stack<Element>();
public void startElement(String uri, String localName, String qName, Attributes attributes) throws SAXException
if (qName.equals("Book"))
String name = attributes.getValue("name");
root.push(new Book(name));
else if (qName.equals("Chapter"))
String name = attributes.getValue("name");
Chapter child = new Chapter(name);
((Book)root.peek()).addChapter(child);
root.push(child);
else if (qName.equals("Section"))
Integer number = Integer.parseInt(attributes.getValue("number"));
Section child = new Section(number);
((Chapter)root.peek()).addSection(child);
root.push(child);
public void endElement(String uri, String localName, String qName) throws SAXException
Element finished = root.pop();
if (root.size() == 0)
book = (Book) finished;
public Book getBook()
return book;
public void error(SAXParseException e)
System.out.println(e.getMessage());
public void fatalError(SAXParseException e)
error(e);
public void warning(SAXParseException e)
error(e);
public class Chapter extends Element
public static class Section extends Element
private Integer number;
public Section(Integer number)
this.number = number;
public String toString()
StringBuilder builder = new StringBuilder();
builder.append("<Section number=\"").append(number).append("\"/>\n");
return builder.toString();
private String name;
private List<Section> sections = null;
public Chapter(String name)
this.name = name;
public void addSection(Section section)
if (sections == null)
sections = new ArrayList<Section>();
sections.add(section);
public String toString()
StringBuilder builder = new StringBuilder();
builder.append("<Chapter name=\"").append(name).append("\">\n");
if (sections != null)
for (Section section: sections)
builder.append(section.toString());
builder.append("</Chapter>\n");
return builder.toString();
}Edited by: sn72 on Oct 28, 2008 1:16 PMHave you looked at the XML DB FAQ thread (second post) in this forum? It has some examples for validating XML against schemas.
-
Need suggestion about a fast XML parser
I have a program which needs to parse lots of XML files of sizes varing from few MBs to hundreds of MBs. It needs to parse the file in one-pass, and for which the SAX approach is best suited. I initially tried coding this program using the SAX parser provided by Java library, but it takes way-way too much time to parse the files. I then googled out Piccolo XML Parser, but it throws ArrayIndexOutOfBound exception. Furthermore, sometimes XML files could terminated prematurely.
Can you suggest any SAX XML parser for this job?I am pasting here a trimmed version of my code..... sorry this is not the simplified and complete in itself version as the norm is.
public Vector<String> search(String dumpfile, String srfor, int type, long size, boolean showTTH, boolean phpSerialize, String hubname) {
Vector<String> results = new Vector<String>();
try {
SAXParserFactory factory = SAXParserFactory.newInstance();
SAXParser parser = factory.newSAXParser();
BufferedInputStream ubin = new BufferedInputStream(new FileInputStream(dumpfile));
ubin.read(new byte[2]); //To discard the starting BZ flag.
CBZip2InputStream bin = new CBZip2InputStream(ubin);
String line = null;
int c=0;
line = "";
while((c=bin.read())!='\n' && c!=-1) line = line + (char) c;
String dnt = line;
if (c==-1){// (line == null){
dnt="";
return null;
line = "";
while((c=bin.read())!='\n' && c!=-1) line = line + (char) c;
if(c==-1)// (line == null)
return null;
if (!phpSerialize)
results.add("Dump's Date and Time stamp: " + dnt + "\n==========================\n"+"hubname: " + line);
else
results.add("$" + dnt +"\n"+"|" + line);
if(hubname!=null && !line.trim().toLowerCase().contains(hubname.toLowerCase().subSequence(0, hubname.length()))){
results.add("No hits.");
return results;
while((c=bin.read())!='\n' && c!=-1);
FilelistHandler handler = new FilelistHandler(srfor,type,size,showTTH,phpSerialize,hubname,results);
try{
parser.parse(bin, handler);
}catch(org.xml.sax.SAXParseException saxe){
saxe.printStackTrace();
if(results.size()==1){
results.add("No hits.");
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
System.err.println("Line: "+e.getLineNumber()+"; Col:"+e.getColumnNumber());
} catch (SAXException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return results;
private class FilelistHandler extends DefaultHandler {
static final int USER = 0;
static final int FILE = 1;
static final int DIR = 2;
static final int ANY = 3;
static final int UNKNOWN = 4;
private Vector<String> results;
Vector<String> dirs = new Vector<String>();
private String srfor;
private int type;
private int level = -1;
private long size;
private boolean showTTH;
private boolean phpSerialize;
private String hubname;
private String currentUser;
private String currentIP;
public FilelistHandler(String Srfor, int Type, long Size, boolean ShowTTH, boolean PhpSerialize, String Hubname, Vector<String> res) {
srfor = Srfor.trim().toLowerCase();
type = Type;
size = Size;
showTTH = ShowTTH;
phpSerialize = PhpSerialize;
hubname = Hubname;
results = res;
public void startElement(String uri, String lname, String qname, Attributes attrs) throws SAXException {
String result = null;
long currsize = 0L;
String TTH = "";
String value = "";
int currType = UNKNOWN;
if (qname.equalsIgnoreCase("Directory")) {
value = attrs.getValue("Name");
dirs.add(attrs.getValue("Name"));
currType = DIR;
} else if (qname.equalsIgnoreCase("user")) {
currentUser = attrs.getValue("username");
currentIP = attrs.getValue("ip");
currType = USER;
} else if (qname.equalsIgnoreCase("File")) {
value = attrs.getValue("Name");
currsize = Long.parseLong(attrs.getValue("Size"));
TTH = attrs.getValue("TTH");
currType = FILE;
if (currType == FILE || currType == DIR) {
// Searching.
boolean found = false;
result = currentUser + ":" + currentIP + ":" + (currType == FILE && showTTH ? TTH + ":" : "") + getPwd(dirs);
if (result.trim().toLowerCase().contains(srfor.subSequence(0, srfor.length())) || currType == FILE && TTH.equalsIgnoreCase(srfor)) {
found = true;
if (type != ANY && currType != type)
found = false;
if (size >= 0 && currType == FILE)
if ((size == 0 && currsize != size) || (size != 0 && (((double) (Math.abs(currsize - size))) / size) > 0.1))
found = false;
if (found) {
if (!phpSerialize) {
if (currType == FILE)
result = result + "/" + value;
} else {
int index = 0;
result = serializeEntity(index++, currType == FILE ? "f" : "d");
result = result + serializeEntity(index++, currentUser);
result = result + serializeEntity(index++, currentIP);
if (currType == FILE && showTTH)
result = result + serializeEntity(index++, TTH);
result = result + serializeEntity(index++, getPwd(dirs));
if (currType == FILE)
result = result + serializeEntity(index++, value);
result = "a:" + index + ":{" + result + "}";
results.add(result); // ADDING THE RESULT.
public void endElement(String uri, String lname, String qname) throws SAXException {
if (qname.equalsIgnoreCase("Directory")) {
dirs.remove(dirs.size() - 1);
private String getPwd(Vector<String> dirs) {
String pwd = "";
for (String dir : dirs) {
pwd = pwd + "/" + dir;
return pwd;
private String serializeEntity(int index, String s) {
return "i:" + index + ";s:" + s.length() + ":\"" + s + "\";";
public Vector<String> getParsedData() {
return results;
} -
XML Parsing in a Schedule Job (OIM 11g R1)
Hi,
I am writing a lookup recon - schedule task in OIM 11g R1 to connect to a webservice and get the lookup attribute values. Now, I am able to fetch the values in a xml format. I am facing an issue while trying to parse the xml values (tried both as a string and from a file).
I have tried using both the SAX and the DOM parser. PFA the snippet for the DOM Parser:
DocumentBuilderFactory builderFactory =
DocumentBuilderFactory.newInstance();
DocumentBuilder builder = null;
try {
builder = builderFactory.newDocumentBuilder();
} catch (ParserConfigurationException e) {
e.printStackTrace();
org.w3c.dom.Document doc =
(org.w3c.dom.Document)builder.parse(file);
System.out.println("Parsed the xml");
org.w3c.dom.NodeList nodes =
doc.getElementsByTagName("entry");
System.out.println("Total Profiles fetched:: " +
nodes.getLength());
This is giving the nodes.getLength() as 0. Though the code runs fine in the Java Client (directly from eclipse)
The snippet for the SAX method is :
SAXParserFactory factory = SAXParserFactory.newInstance();
SAXParser saxParser = factory.newSAXParser();
DefaultHandler handler = new DefaultHandler() {
boolean bfname = false;
boolean blname = false;
boolean bnname = false;
boolean bsalary = false;
public void startElement(String uri, String localName,
String qName,
Attributes attributes) throws SAXException {
// System.out.println("Start Element :" + qName);
if (qName.equalsIgnoreCase("d:Name")) {
blname = true;
public void endElement(String uri, String localName,
String qName) throws SAXException {
// System.out.println("End Element :" + qName);
public void characters(char[] ch, int start,
int length) throws SAXException {
if (blname) {
System.out.println("d:Name : " +
new String(ch, start, length));
blname = false;
saxParser.parse("home/oracle/UserProfiles.xml", handler);
This is throwing a java.net.malformedurlexception: no protocol: /home/oracle/userprofiles.xml exception. I tried to pass it as "file://home/oracle/UserProfiles.xml" but then is gives a
java.net.ConnectException: Connection refused Even this code is running fine in the Java Client when ran directly. It seems there is some weblogic configurations I am missing for the XML parsing?
Please do help, if anyone has faced any issues parsing an xml in a Schedule job. Will highly appreciate your inputs.
Thanks,
Anuj.Hi,
I am writing a lookup recon - schedule task in OIM 11g R1 to connect to a webservice and get the lookup attribute values. Now, I am able to fetch the values in a xml format. I am facing an issue while trying to parse the xml values (tried both as a string and from a file).
I have tried using both the SAX and the DOM parser. PFA the snippet for the DOM Parser:
DocumentBuilderFactory builderFactory =
DocumentBuilderFactory.newInstance();
DocumentBuilder builder = null;
try {
builder = builderFactory.newDocumentBuilder();
} catch (ParserConfigurationException e) {
e.printStackTrace();
org.w3c.dom.Document doc =
(org.w3c.dom.Document)builder.parse(file);
System.out.println("Parsed the xml");
org.w3c.dom.NodeList nodes =
doc.getElementsByTagName("entry");
System.out.println("Total Profiles fetched:: " +
nodes.getLength());
This is giving the nodes.getLength() as 0. Though the code runs fine in the Java Client (directly from eclipse)
The snippet for the SAX method is :
SAXParserFactory factory = SAXParserFactory.newInstance();
SAXParser saxParser = factory.newSAXParser();
DefaultHandler handler = new DefaultHandler() {
boolean bfname = false;
boolean blname = false;
boolean bnname = false;
boolean bsalary = false;
public void startElement(String uri, String localName,
String qName,
Attributes attributes) throws SAXException {
// System.out.println("Start Element :" + qName);
if (qName.equalsIgnoreCase("d:Name")) {
blname = true;
public void endElement(String uri, String localName,
String qName) throws SAXException {
// System.out.println("End Element :" + qName);
public void characters(char[] ch, int start,
int length) throws SAXException {
if (blname) {
System.out.println("d:Name : " +
new String(ch, start, length));
blname = false;
saxParser.parse("home/oracle/UserProfiles.xml", handler);
This is throwing a java.net.malformedurlexception: no protocol: /home/oracle/userprofiles.xml exception. I tried to pass it as "file://home/oracle/UserProfiles.xml" but then is gives a
java.net.ConnectException: Connection refused Even this code is running fine in the Java Client when ran directly. It seems there is some weblogic configurations I am missing for the XML parsing?
Please do help, if anyone has faced any issues parsing an xml in a Schedule job. Will highly appreciate your inputs.
Thanks,
Anuj. -
JavaMapping in PI 7.1 Error:Unable to display tree view; Error when parsing
hi,
i get by testing in PI 7.1 (operation mapping) this ERROR:
"Unable to display tree view; Error when parsing an XML document (Content is not allowed in prolog.)"
this is my java-programm-code:
import java.io.FileInputStream;
import java.io.FileOutputStream;
import com.sap.aii.mapping.api.StreamTransformation;
import java.io.*;
import java.util.Map;
import javax.xml.parsers.*;
import org.xml.sax.*;
import org.xml.sax.helpers.*;
/*IMPORT statement imports the specified classes and its methods into the program */
/Every Java mapping program must implement the interface StreamTransformation and its methods execute() and setParameter() and extend the class DefaultHandler./
public class Mapping extends DefaultHandler implements StreamTransformation {
Below is the declaration for all the variables we are going to use in the
subsequent methods.
private Map map;
private OutputStream out;
private boolean input1 = false;
private boolean input2 = false;
private int number1;
private int number2;
private int addvalue;
private int mulvalue;
private int subvalue;
String lineEnd = System.getProperty("line.separator");
setParamater() method is used to store the mapping object in the variable
"map"
public void setParameter(Map param) {
map = param;
public void execute(InputStream in, OutputStream out)
throws com.sap.aii.mapping.api.StreamTransformationException {
DefaultHandler handler = this;
SAXParserFactory factory = SAXParserFactory.newInstance();
try {
SAXParser saxParser = factory.newSAXParser();
this.out = out;
saxParser.parse(in, handler);
} catch (Throwable t) {
t.printStackTrace();
As seen above execute() method has two parameters "in" of type
InputStream and "out" of type OutputStream. First we get a new instance
of SAXParserFactory and from this one we create a new Instance of
SAXParser. To the Parse Method of SaxParser, we pass two parameters,
inputstream "in" and the class variable "handler".
Method "write" is a user defined method, which is used to write the
string "s" to the outpurstream "out".
private void write(String s) throws SAXException {
try {
out.write(s.getBytes());
out.flush();
} catch (IOException e) {
throw new SAXException("I/O error", e);
public void startDocument() throws SAXException {
write("");
write(lineEnd);
write("");
write(lineEnd);
public void endDocument() throws SAXException {
write("");
try {
out.flush();
} catch (IOException e) {
throw new SAXException("I/O error", e);
public void startElement(String namespaceURI, String sName, String qName,
Attributes attrs) throws SAXException {
String eName = sName;
if ("".equals(eName))
eName = qName;
if (eName.equals("NUMBER1"))
input1 = true;
if (eName.equals("NUMBER2"))
input2 = true;
public void endElement(String namespaceURI, String sName, String qName)
throws SAXException {
String eName = sName;
if ("".equals(eName))
eName = qName;
if (eName.equals("NUMBER1"))
input1 = false;
if (eName.equals("NUMBER2"))
input2 = false;
public void characters(char[] chars, int startIndex, int endIndex)
throws SAXException {
String dataString = new String(chars, startIndex, endIndex).trim();
if (input1) {
try {
number1 = Integer.parseInt(dataString);
} catch (NumberFormatException nfe) {
if (input2) {
number2 = Integer.parseInt(dataString);
if (input2 == true) {
addvalue = number1 + number2;
mulvalue = number1 * number2;
subvalue = number1 - number2;
write("" + addvalue + "");
write(lineEnd);
write("" + mulvalue + "");
write(lineEnd);
write("" + subvalue + "");
write(lineEnd);
in developer studio 7.1 i dont get error.
this happens by testing the mapping-programm in ESR.
can somebody help me please?Make sure that the xml created out after the java mapping is a valid xml with only one root node.
Regards,
Prateek -
hi,
i am assuming the problem is with sax parser but i cant be sure. I am parsing a xml file (about 1.4MB) with some data in it. the parser i have created reads the xml file correctly for the most part but when at some point the
"public void characters(char buf[], int offset, int len) throws SAXException"
function stops working correctly....i.e it doesnt fully read read the data between the "<start>" and "</start>" element. say it reads about 100 id's correctly---for 101 ID it does this. This is just an example. Since, the problem might be with how :
"public void characters(char buf[], int offset, int len) throws SAXException"
function is reading the data i was wondering if anybody else had encountered this problem or please let me know if i need to change something in the code: here's a part of the code :
Bascially i have created three classes to enter data into three mysql tables and as i parse the data i fill up the columns by matching the column header with the tagName.
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.*;
import java.io.*;
import java.util.ArrayList;
import java.lang.Object;
import org.xml.sax.*;
import org.xml.sax.helpers.DefaultHandler;
import java.util.*;
import javax.xml.parsers.SAXParserFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParser;
public class Echo03 extends DefaultHandler
StringBuffer textBuffer;
int issuedValue, prodValue;
OrdHeader header = new OrdHeader();
OrdDetail detail = new OrdDetail();
Member memInfo = new Member();
//new addition to store the dynamic value of the products
TestOrdheader prod = new TestOrdheader();
int counter;
String tag, newTag;
SetValue setVal = new SetValue();
String test;
public static void main(String argv[])
if (argv.length != 1) {
System.err.println("Usage: cmd filename");
System.exit(1);
// Use an instance of ourselves as the SAX event handler
DefaultHandler handler = new Echo03();
// Use the default (non-validating) parser
SAXParserFactory factory = SAXParserFactory.newInstance();
try {
// Set up output stream
out = new OutputStreamWriter(System.out, "UTF8");
// Parse the input
SAXParser saxParser = factory.newSAXParser();
saxParser.parse( new File(argv[0]), handler);
} catch (Throwable t) {
t.printStackTrace();
System.exit(0);
static private Writer out;
private String indentString = " "; // Amount to indent
private int indentLevel = 0;
//===========================================================
// SAX DocumentHandler methods
//===========================================================
public void startDocument()
throws SAXException
nl();
nl();
emit("START DOCUMENT");
nl();
emit("<?xml version='1.0' encoding='UTF-8'?>");
header.assign();
public void endDocument()
throws SAXException
nl(); emit("END DOCUMENT");
try {
nl();
out.flush();
} catch (IOException e) {
throw new SAXException("I/O error", e);
public void startElement(String namespaceURI,
String lName, // local name
String qName, // qualified name
Attributes attrs)
throws SAXException
indentLevel++;
nl(); //emit("ELEMENT: ");
String eName = lName; // element name
if ("".equals(eName)) eName = qName; // namespaceAware = false
if (qName.equals("Billing")){
issuedValue = 1;
}else if (qName.equals("Shipping")){
issuedValue = 2;
}else if (qName.equals("ShippingTotal")){
issuedValue = 3;
//check to see if "Product" is the name of the element thats coming next
if (qName.equals("Product")){
if (issuedValue != 3){
prodValue = 1;
prod.addCounter();
}else{
prodValue = 0;
tag = eName;
if (attrs != null) {
for (int i = 0; i < attrs.getLength(); i++) {
String aName = attrs.getLocalName(i); // Attr name
if ("".equals(aName)) aName = attrs.getQName(i);
nl();
emit(" ATTR: ");
emit(aName);
emit("\t\"");
emit(attrs.getValue(i));
emit("\"");
if (attrs.getLength() > 0) nl();
public void endElement(String namespaceURI,
String sName, // simple name
String qName // qualified name
throws SAXException
nl();
String eName = sName; // element name
if ("".equals(eName)){
eName = qName; // not namespaceAware
if ("Order".equals(eName)){
//enter into database
databaseEnter();
textBuffer = null;
indentLevel--;
public void characters(char buf[], int offset, int len)
throws SAXException
nl();
try {
String s = new String(buf, offset, len);
if (!s.trim().equals("")){
settag(tag, s);
s = null;
}catch (NullPointerException E){
System.out.println("Null pointer Exception:"+E);
//===========================================================
// Utility Methods ...
//===========================================================
// Wrap I/O exceptions in SAX exceptions, to
// suit handler signature requirements
private void emit(String s)
throws SAXException
try {
out.write(s);
out.flush();
} catch (IOException e) {
throw new SAXException("I/O error", e);
// Start a new line
// and indent the next line appropriately
private void nl()
throws SAXException
String lineEnd = System.getProperty("line.separator");
try {
out.write(lineEnd);
for (int i=0; i < indentLevel; i++) out.write(indentString);
} catch (IOException e) {
throw new SAXException("I/O error", e);
===================================================================
///User defined methods
===================================================================
private String strsplit(String splitstr){
String delimiter = new String("=");
String[] value = splitstr.split(delimiter);
value[1] = value[1].replace(':', ' ');
return value[1];
public void settag(String tag, String s){
String pp_transid = null, pp_respmsg = null,pp_authid = null, pp_avs = null, pp_avszip = null;
if ((tag.equals("OrderDate")) || (tag.equals("OrderProcessingInfo"))){
if (tag.equals("OrderDate")){
StringTokenizer st = new StringTokenizer(s);
String orddate = st.nextToken();
String ordtime = st.nextToken();
header.put("ordDate", orddate);
header.put("ordTime", ordtime);
}else if (tag.equals("OrderProcessingInfo")){
StringTokenizer st1 = new StringTokenizer(s);
int tokenCount = 1;
while (tokenCount <= st1.countTokens()){
switch(tokenCount){
case 1:
String extra = st1.nextToken();
break;
case 2:
String Opp_transid = st1.nextToken();
pp_transid = strsplit(Opp_transid);
break;
case 3:
String Opp_respmsg = st1.nextToken();
pp_respmsg = strsplit(Opp_respmsg);
break;
case 4:
String Opp_authid = st1.nextToken();
pp_authid = strsplit(Opp_authid);
break;
case 5:
String Opp_avs = st1.nextToken();
pp_avs = strsplit(Opp_avs);
break;
case 6:
String Opp_avszip = st1.nextToken();
pp_avszip = strsplit(Opp_avszip);
break;
tokenCount++;
header.put("pp_transid", pp_transid);
header.put("pp_respmsg", pp_respmsg);
header.put("pp_authid", pp_authid);
header.put("pp_avs", pp_avs);
header.put("pp_avszip", pp_avszip);
}else{
newTag = new String(setVal.set_name(tag, issuedValue));
header.put(newTag, s);
//detail.put(newTag, s);
prod.put(newTag, s);
memInfo.put(newTag,s);
//Check to see-- if we should add this product to the database or not
boolean check = prod.checkValid(newTag, prodValue);
if (check){
prod.addValues(s);
setVal.clearMod();
==================================================================
Here's the error that i get:
java.util.NoSuchElementException
at org.apache.crimson.parser.Parser2.parseInternal(Parser2.java:691)
at org.apache.crimson.parser.Parser2.parse(Parser2.java:337)
at org.apache.crimson.parser.XMLReaderImpl.parse(XMLReaderImpl.java:448)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:345)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:281)
at Echo03.main(Echo03.java:47)I haven't gone through your code but I also had a similar error....and the exception in my was because of an "&" instead of the entity reference & in one of the element values. I use a non-validating parser but if you use a validating one then this might not be the reason for your exception.
Maybe you are looking for
-
Can I send a link to a photo book so family members can choose to order a copy from Apple?
I woud like to share the photo book that I created so that family members in other states can consider purchasing a copy from I Photo. Is there a way they can view it?
-
Script Errors in TOC from RoboHelp 9
Hello, all. I can usually find the answer to my problems by searching existing threads, but I can't seem to find a solution this time. I recently upgraded from RoboHelp 7 to 9.0.2.271. I have published WebHelp locally and then checked files in to a T
-
HT1657 Std Def. on New Apple TV
Can I rent movies in std. def. on new apple tv ?
-
hi all, i am running this command at command prompt like below java filename argument. ex: java validateuser user1 it works fine at command prompt. and the same command i want to run in jsp so i tried like this Runtime x = Runtime.getRuntime().exec("
-
Hello, I have written an application to show video on J2ME devices. The application shows video properly. If the video is huge it takes time to download all video and then plays it. How can I stream part of video of a huge video and we show the downl