How to handler 200MB - 600MB XML file

I came into a problem to handle very big XML files using DOM or SAX. Is there any method I can build XMLDocument for such big files?

You cannot use DOM to load such a large file unless you can free up 1 to 3 gigabytes of heap memory. You could write special-purpose programs that used SAX to parse it as long as you were careful not to accumulate data in memory. You may want to seriously consider storing the data in some other format.

Similar Messages

How do I create individual xml files from the parsed data output of a xml file?

I have written a program (DOM Parser) that parses data from a XMl File. I would like to create an individual file with the corresponding name for each set of data parsed from the xml document. If the parsed output is Single, Double, Triple, I would like to create an individual xml file (Single.xml, Double.xml, Triple.xml)with those corresponding names. How do I create the xml files and give each file the name of my parsed data output? Thanks in advance for your help.
import java.io.IOException;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
public class MyDomParser {
public static void main(String[] args) {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
try {
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse("ENtemplate.xml");
doc.normalize();
NodeList rootNodes = doc.getElementsByTagName("templates");
Node rootNode = rootNodes.item(0);
Element rootElement = (Element) rootNode;
NodeList templateList = rootElement.getElementsByTagName("template");
for(int i=0; i < templateList.getLength(); i++) {
Node theTemplate = templateList.item(i);
Element templateElement = (Element) theTemplate;
System.out.println("Template" + ": " +templateElement.getAttribute("name")+ ".xml");
} catch (ParserConfigurationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SAXException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();

Ive posted the new code but now I'm getting a FileAlreadyExistException error. How do I handle this exception error correctly in my code?
import java.io.IOException;
import java.nio.file.FileAlreadyExistsException;
import java.nio.file.Files;
import java.nio.file.Paths;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
public class MyDomParser {
public static void main(String[] args) {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
try {
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse("ENtemplate.xml");
doc.normalize();
NodeList rootNodes = doc.getElementsByTagName("templates");
Node rootNode = rootNodes.item(0);
Element rootElement = (Element) rootNode;
NodeList templateList = rootElement.getElementsByTagName("template");
for(int i=0; i < templateList.getLength(); i++) {
Node theTemplate = templateList.item(i);
Element templateElement = (Element) theTemplate;
System.out.println(templateElement.getAttribute("name")+ ".xml");
for(int i=0; i < templateList.getLength(); i++) {
Node theTemplate = templateList.item(i);
Element templateElement = (Element) theTemplate;
String fileName = templateElement.getAttribute("name") + ".xml";
Files.createFile(Paths.get(fileName));
System.out.println("File" + ":" + fileName + ".xml created");
} catch (ParserConfigurationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SAXException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();

How to retrieve value from xml file

hi all,
can somebody pls tell me how to retrieve value from xml file using SAXParser.
I want to retrieve value of only one tag and have to perform some validation with that value.
it's urgent .
pls help me out
thnx in adv.
ritu

hi shanu,
the pbm is solved, now i m able to access XXX no. in action class & i m able to validate it. The only thing which i want to know is it ok to declare static ArrayList as i have done in this code. i mean will it affect the performance or functionality of the system.
pls have a look at the following code snippet.
public class XMLValidator {
static ArrayList strXXX = new ArrayList();
public void validate(){
factory.setValidating(true);
parser = factory.newSAXParser();
//all factory code is here only
parser.parse(xmlURI, new XMLErrorHandler());
public void setXXX(String pstrXXX){
strUpn.add(pstrXXX);
public ArrayList getXXX(){
return strXXX;
class XMLErrorHandler extends DefaultHandler {
String tagName = "";
String tagValue = "";
String applicationRefNo = "";
String XXXValue ="";
String XXXNo = "";
XMLValidator objXmlValidator = new XMLValidator();
public void startElement(String uri, String name, String qName, Attributes atts) {
tagName = qName;
public void characters(char ch[], int start, int length) {
if ("Reference".equals(tagName)) {
tagValue = new String(ch, start, length).trim();
if (tagValue.length() > 0) {
RefNo = new String(ch, start, length);
if ("XXX".equals(tagName)) {
XXXValue = new String(ch, start, length).trim();
if (XXXValue.length() > 0) {
XXXNo = new String(ch, start, length);
public void endElement(String uri, String localName, String qName) throws SAXException {
if(qName.equalsIgnoreCase("XXX")) {
objXmlValidator.setXXX(XXXNo);
thnx & Regards,
ritu

How I can create a XML file from java Aplication

How I can create a XML file from java Aplication
whith have a the following structure
<users>
<user>
<login>anyName</login>
<password>xxxx</password>
</user>
</users>
the password label must be encripted
accept any suggestion

Let us assume you have all the data from the jsp form in an java bean object..
Now you want a xml file. This can be acheived in 2 ways
1. Write it into a file using java.io classes. Say you have a class with name
write("<name>"+obj.getName+</name>);
bingo you have a flat file with the xml
2. Use data binding to do the trick
will recommend JiBx and Castor for the 2nd option
Regards,
Rajagopal

How to Read and Generate XML file from java code.

hi guys,
how to read the xml file (Condition :we know only DTD or Shema only).
How to Generate the new xml file ?(using Shema )
And one more how directly Generate the xml from DB?
Pleas with code or any URL

Using XMLbeans you can generate Java objects from an XSD schema (perhaps DTDs aswell)
Then you can create an instance of the Document object and ask it to write itself.
This will create an XML document complient to the schema.
XMLBeans generates a "type" safe DOM where you can only ever have a structure compilent to you schema.
matfud

How to handle errors in a file at sender side?

Hi
I have done a file to proxy scenario.
I know how to handle errors on proxy.
But on sender side when picking the file if one the record have worng fomat its throwing mapping error and its not processing any record..
I wanted to process the records which have right format and data and all remaining recrods which have wromg format should be send back to the sender as file.
How to do this.
How to handle error in sender file.
Regards
Sowmya

Hello Sowmya,
In your scenario Three ways you can validate the data.
1) Before the data reaches into SAP system ie in XI system during Mapping or before mapping i,e in Adapter Module in the Sender side
2) this option, is in the receiver applications side. ie. Validations will be taken care in the SAP system i.e in ABAP server proxy code.
3)Through BPM, If Mapping Exception Occures then through exception Branch you can send bad formate file to sender again.
Generally, it is prefer to more business critical validations in the Application System ie Receiver Application System (ABAP Server Proxy)
In this, you can have more flexibility of the validations as you are validating some of the SAP payroll informations as Personal ID etc.
Based on the complexity and flexibility of the requirement, you can either do this in the XI (if XI, ie Sender Adapter Module or Mapping) or in the ABAP proxy
Thanks'
Sunil Singh

How I could generate an XML file from a report in version 4.0B

Good morning,
How could I generate an XML file from a report? Please note that I am using version 4.0B
I don't have access to
Billy Vital

Hi,
In the Class CL_XML_DOCUMENT,
we have a method EXPORT_TO_FILE to download an XML file.

How To check validity of XML File

can any one tell me how to find whether an XML file is valid or not before beginning to parse the XML file
Thanks in advance

Xerces will do that for qou, if you activate validation (and, of course, a DTD is specified in the file). Read the documentation of Xerces at xml.apache.org

How to remove Unicode from XML file

I get following error when unmarshal xml:
[java] org.xml.sax.SAXParseException: An invalid XML character (Unicode: 0x15) was found in the element content of the document.
Anyone know how to remove Unicode from xml file? Can I remove the unicode by rebuild the file?
Thanks

These sort of error usually occur when you're using a different character encoding to read the file than the one you wrote it with. Perhaps if you were to post the problem section of the file and/or the code that created it in the first place.

How to handle the java.policy file ?

Can somebody tell me how to handle the java.policy file?
I always get java.net.SocketExceptions and java.security.AccessControlExceptions while connecting to an appserver from an applet.
What do I have to write in the java.policy file, where do I have to place it and do I have to call it in some way form my applet?
Thanks in advance.
don call

The java.policy file goes in your jre installation directory in .../jre/lib/security (there should be one there already).
I used it to allow otherwise restricted permissions for an applet using javax.comm. Add something like the following to the file:
grant codeBase "URL:http://yourDomainName/rootDirectoryOfYourApp/*" {
permission java.security.AllPermission;
This will give the applet downloaded from your site all permissions. You might want to give only certain permissions, I don't know.
Teri

How to parse contents from XML file in Java

Hi All,
I have a scenario like this . I have one xml file with key value pairs of ( name , URL ) . I have retrieved contents from XML file , now I want to parse these contents and store in a bean object.
How to parse Contents of XML file??
Thanks in advance,
Rajendra.

Hi All,
I have a scenario like this . I have one xml file with key value pairs of ( name , URL ) . I have retrieved contents from XML file , now I want to parse these contents and store in a bean object.
How to parse Contents of XML file??
Thanks in advance,
Rajendra.

How to config the web.xml file, when I use Richfaces + RI 1.2?

Hi there:
I want to use Richfaces + RI 1.2 to build a project. I don`t know how to config the web.xml file.
By the way, my web server is Tomcat 6.0, my JDK's version is 6u6. I don`t want to use the facelets.
thanks.
lxm

just add this before *</web-app>*
<context-param>
     <param-name>org.richfaces.SKIN</param-name>
     <param-value>blueSky</param-value>
</context-param>
<filter>
     <display-name>RichFaces Filter</display-name>
     <filter-name>richfaces</filter-name>
     <filter-class>org.ajax4jsf.Filter</filter-class>
</filter>
<filter-mapping>
     <filter-name>richfaces</filter-name>
     <servlet-name>Faces Servlet</servlet-name>
     <dispatcher>REQUEST</dispatcher>
     <dispatcher>FORWARD</dispatcher>
     <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

How to extract data from XML file with JavaScript

HI All
I am new to this group.
Can anybody help me regarding XML.
I want to know How to extract data from XML file with JavaScript.
And also how to use API for XML
regards
Nagaraju

This is a Java forum.
JavaScript is something entirely different than Java, even though the names are similar.
Try another website with forums about JavaScript.
For example here: http://www.webdeveloper.com/forum/forumdisplay.php?s=&forumid=3

How can (parse) i use XML file with missing EndTag

hi,
i have an application which writes an "XML file".
another application should read that XML file, build an DOM and
access the nodes with xpath.
my problem. if the first application is not finished there are tags
missing. e.g. </xml>. but the seconds application cannot wait until the first application finishes it task.
if i now read the XML file the parser cannot load it because the end tags are missing.
my question:
how can i deactivate the check or how can i read the XML file and access it via XPath (my application is using at the moment XPath to access the nodes and i dont want to change that)
as parser i am using XERCES
alex

As far as I know, you can't do this - xml must be well formed (this is sort of a bedrock of xml). There may be some work around's, but I'm not aware of any - and they would most likely be hacks.

How do I upload an XML file to salesforce using BULK API?

Hi There,
Please let me know how do we upload an XML file to salesforce using Bulk API?
Thanks,
ET

Hi,
I think that this is a more SalesForce.com question and think you will have more chance looking at SOAP API Developer's Guide for salesforce. Sending a SOAP request from the API Server is very straight forward and there are several tutorials and well documented about this.
Cheers,
Stefan

How to handler 200MB - 600MB XML file

Similar Messages

Maybe you are looking for