Transforming large XML using xalan

Hi,
we need to transform a large xml (50 MB) to a flat file. Our java program throws out of memory exception. any pointer will be helpful in this regard.
Thanks
Sabir

for this kind of monster XSLT, i have quited JAXP for using James Clark's XP/XT - which offers 100% compatibility with JAXP's XSLs (unlike Saxon that doesn't understand my XSLs).

Similar Messages

  • Loading, processing and transforming Large XML Files

    Hi all,
    I realize this may have been asked before, but searching the history of the forum isn't easy, considering it's not always a safe bet which words to use on the search.
    Here's the situation. We're trying to load and manipulate large XML files of up to 100MB in size.
    The difference from what we have in our hands to other related issues posted is that the XML isn't big because it has a largly branched tree of data, but rather because it includes large base64-encoded files in the xml itself. The size of the 'clean' xml is relatively small (a few hundred bytes to some kilobytes).
    We had to deal with transferring the xml to our application using a webservice, loading the xml to memory in order to read values from it, and now we also need to transform the xml to a different format.
    We solved the webservice issue using XFire.
    We solved the loading of the xml using JAXB. Nevertheless, we use string manipulations to 'cut' the xml before we load it to memory - otherwise we get OutOfMemory errors. We don't need to load the whole XML to memory, but I really hate this solution because of the 'unorthodox' manipulation of the xml (i.e. the cutting of it).
    Now we need to deal with the transofmation of those XMLs, but obviously we can't cut it down this time. We have little experience writing XSL, but no experience on how to use Java to use the XSL files. We're looking for suggestions on how to do it most efficiently.
    The biggest problem we encounter is the OutOfMemory errors.
    So I ask several questions in one post:
    1. Is there a better way to transfer the large files using a webservice?
    2. Is there a better way to load and manipulate the large XML files?
    3. What's the best way for us to transform those large XMLs?
    4. Are we missing something in terms of memory management? Is there a better way to control it? We really are struggling there.
    I assume this is an important piece of information: We currently use JDK 1.4.2, and cannot upgrade to 1.5.
    Thanks for the help.

    I think there may be a way to do it.
    First, for low RAM needs, nothing beats SAX. as the first processor of the data. With SAX, you control the memory use since SAX only processes one "chunk" of the file at a time. You supply a class with methods named startElement, endElement, and characters. It calls the startElement method when it finds a new element. It calls the characters method when it wants to pass you some or all of the text between the start and end tags. It calls endElement to signal that passing characters is over, and to let you get ready for the next element. So, if your characters method did nothing with the base-64 data, you could see the XML go by with low memory needs.
    Since we know in your case that the characters will process large chunks of data, you can expect many calls as SAX calls your code. The only workable solution is to use a StringBuffer to accumulate the data. When the endElement is called, you can decode the base-64 data and keep it somewhere. The most efficient way to do this is to have one StringBuffer for the class handling the SAX calls. Instantiate it with a big enough size to hold the largest of your binary data streams. In the startElement, you can set the length of the StringBuilder to zero and reuse it over and over.
    You did not say what you wanted to do with the XML data once you have processed it. SAX is nice from a memory perspective, but it makes you do all the work of storing the data. Unless you build a structured set of classes "on the fly" nothing is kept. There is a way to pass the output of one SAX pass into a DOM processor (without the binary data, in this case) and then you would wind up with a nice tree object with the rest of your data and a group of binary data objects. I've never done the SAX/DOM combo, but it is called a SAXFilter, and you should be able to google an example.
    So, the bottom line is that is is very possible to do what you want, but it will take some careful design on your part.
    Dave Patterson

  • Transform Large XML files with XSL

    HELP, LARGE XML FILES
    I have got 30 - 50 MB large xml file, and i would like to transform it
    with xslt, i tried but i have got OutOfMemory Exception.
    I tried to find out the solution on JAVA site, but i didn't find it.
    I can not displit my xml file. I hope for some help.
    I tried really everything.
    Thanks a lot

    What is your machine configuration ?
    The above 2 suggestions would help, but it does depend on how your software is written.
    Please post more info about your environment and object design.
    Chintan

  • Transforming LARGE xml file

    Hi All,
    I am trying to transform a 230 meg xml document and it is taking very long too complete. I am using the TransformerFactory etc in the xalan.jar with the new release of the java extensions. Anyone out there have any ideas/advice that they can give me on how to step this transformation up?

    I think there are only two ways to speed things up besides buying more ram. Either seperate the file in smaller units or filter out unused elements before you transform it.

  • Problem transforming XML/XSL Using Xalan

    Hi, I am using the xalan apis to transform XML/XSL in a servlet..My servlet code is like...
    response.setContentType("text/html; charset=UTF-8");
    // Output goes in the response stream.
    PrintWriter out = response.getWriter();
    boolean environmentOK = (new org.apache.xalan.xslt.EnvironmentCheck()).checkEnvironment (out);
    try
    TransformerFactory tFactory = TransformerFactory.newInstance();
    // Get the XML input document and the stylesheet.
    Source xmlSource = new StreamSource("D:\\DATA\\jrun\\test\\test.xml");
    Source xslSource = new StreamSource("D:\\DATA\\jrun\\test\\test.xsl");
    // Generate the transformer.
    Transformer transformer = tFactory.newTransformer(xslSource);
    // Perform the transformation, sending the output to the response.
    transformer.transform(xmlSource, new StreamResult(out));
    I am getting the error ..
    javax.xml.transform.TransformerConfigurationException: javax.xml.transform.TransformerException: org.xml.sax.SAXException: Namespace not supported by SAXParserjavax.xml.transform.TransformerConfigurationException
    That's why I have added the line..
    boolean environmentOK = (new org.apache.xalan.xslt.EnvironmentCheck()).checkEnvironment (out);
    But that gives me the following error diagnostic information...
    ERROR.version.DOM=ERROR attempting to load DOM level 2 class: java.lang.NoSuchMethodException
    I have put all the 3 required classes, xml-apis.jar, xercesImpl.jar and xalan.jar in my application servers classpath..
    Please let me know what I am doing wrong..and how this can be fixed.
    Thanks
    Surajit

    hey Surajit
    Your code is unable to create a transformer from a factory instance.
    I would check why it is not creating the Transformer.On my win32 system i usee two forward slashes..to identify the file path
    D://xyz//abc....instead of D:\\...i wonder if thats one reason why your code is not able to create a transformer based on the StreamSource u create for the file paths.
    Code i use..that works
    javax.xml.transform.TransformerFactory tFactory =
                                                                javax.xml.transform.TransformerFactory.newInstance();
                                                      javax.xml.transform.Source xmlSource = new javax.xml.transform.stream.StreamSource                              (xmlInputSource);
                                                 javax.xml.transform.Source xslSource = new javax.xml.transform.stream.StreamSource
                                       (xslInputSource);
                                                                                                        // Generate the transformer.
                                                                javax.xml.transform.Transformer transformer = tFactory.newTransformer(xslSource);
                                                                                                   // Perform the transformation, sending the output to the response.
                                                                transformer.transform(xmlSource,new javax.xml.transform.stream.StreamResult(out));
                                                                                                        I used jaxp-1.2 with tomcat 4.1.12
    Hope that helps.

  • Transforming XML using XSLT more than once

    Hi,
    I am trying to transform an XML Document object in Java. However, it requires 2 transforms as each of them is complicated and needs to be generic enough for use by different XML strings. So I have a single XML string and two XSLT files.
    Currently, I am using the Java transformer to perform both the transforms one after another. However, I was wondering if there is a function within XSLT or Java that would allow performing the second transform on the result of the first transform without having to resort to multiple calls from within Java.
    Thanks a lot.
    Jay Badiyani

    http://xml.apache.org/xalan-j/samples.html#usexmlfilters

  • Problems during Transformation using Xalan

    Hi,
    i'm using Xalan 2.6.0 to transform xml and xsl into HTML.
    This works fine but i get errors during Transformation but the HTML File is build.
    The Error is:
    [Error] version.xsl:9:80: Element type "xsl:stylesheet" must be declared.
    [Error] version.xsl:11:33: Element type "xsl:include" must be declared.
    [Error] version.xsl:20:3: Element type "xsl:output" must be declared.
    [Error] version.xsl:21:25: Element type "xsl:template" must be declared.
    [Error] version.xsl:22:7: Element type "html" must be declared.
    [Error] version.xsl:23:34: Element type "xsl:call-template" must be declared.
    [Error] version.xsl:24:79: Element type "xsl:with-param" must be declared.
    [Error] version.xsl:26:7: Element type "body" must be declared.
    [Error] version.xsl:27:83: Element type "table" must be declared.
    [Error] version.xsl:28:7: Element type "tr" must be declared.
    [Error] version.xsl:29:34: Element type "th" must be declared.
    [Error] version.xsl:32:64: Element type "table" must be declared.
    [Error] version.xsl:33:7: Element type "tr" must be declared.
    . and so on
    The header of the XSL File looks like:
    <?xml version="1.0" encoding="iso-8859-1"?>
    <!DOCTYPE xsl:stylesheet [
    <!ENTITY amp "&#038;">
    <!ENTITY nbsp "&#160;">
    <!ENTITY uuml "&#252;">
    <!ENTITY Auml "&#196;">
    ]>
    <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:include href="global.xsl"/>
    <!-- Template f�r root Element-->
    <xsl:output
    method="html"
    doctype-public="-//W3C//DTD HTML 4.01 Transitional//EN"
    encoding="ISO-8859-1"
    version="4.01"
    indent="yes"
    />
    <xsl:template match="/">
    Can somebody help me with this problem ?
    Thanks.
    Jens

    When I remove the DOCTYPE it works but is this a Xalan Problem or is the using of a doctype declaration in a XSL Document generally forbidden ?
    Because without Doctype i cant use characters such as "�", "�", and so on.

  • Detecting transform errors when using org.xml.sax.XMLFilter

    I am using javax.xml.transform.sax.SAXTransformerFactory.newXMLFilter to transform xml against a pipeline of stylesheets (See function testPipelineTransform in the example below). The process works okay, excepting that I cannot figure out how to detect errors in the transformation process.
    I would like to achieve the same result for the pipeline process as I would for a single transformation using javax.xml.transform.Transformer.setErrorListener. The example below demonstrates how the call to javax.xml.transform.Transformer.setErrorListener does not generate the same result when using org.xml.sax.XMLFilter as it does when using a single transformation.
    import javax.xml.parsers.ParserConfigurationException;
    import javax.xml.parsers.SAXParser;
    import javax.xml.parsers.SAXParserFactory;
    import org.xml.sax.SAXException;
    import org.xml.sax.InputSource;
    import org.xml.sax.XMLReader;
    import org.xml.sax.XMLFilter;
    import javax.xml.transform.ErrorListener;
    import javax.xml.transform.Source;
    import javax.xml.transform.Transformer;
    import javax.xml.transform.TransformerException;
    import javax.xml.transform.TransformerFactory;
    import javax.xml.transform.TransformerConfigurationException;
    import javax.xml.transform.sax.SAXTransformerFactory;
    import javax.xml.transform.sax.SAXSource;
    import javax.xml.transform.stream.StreamSource;
    import javax.xml.transform.stream.StreamResult;
    import java.io.*;
    public class FilterChain {
        static private final String newline = System.getProperty("line.separator");
         public static void main(String argv[]) {
              System.out.println("Testing pipeline transform");
              testPipelineTransform();
              System.out.println("Testing single transform");
              testXsltSingleTransform();
         } // main
         static private void testPipelineTransform() {
              try {
                   // Set up the input stream
                   BufferedInputStream bis = new BufferedInputStream(
                             new ByteArrayInputStream(getExample().getBytes()));
                   InputSource input = new InputSource(bis);
                   // Set up to read the input file
                   SAXParserFactory spf = SAXParserFactory.newInstance();
                   SAXParser parser = spf.newSAXParser();
                   XMLReader reader = parser.getXMLReader();
                   // Create the filters
                   SAXTransformerFactory stf = (SAXTransformerFactory) TransformerFactory
                             .newInstance();
                   XMLFilter filter1 = stf.newXMLFilter(new StreamSource(new StringReader(getStylesheet1())));
                   XMLFilter filter2 = stf.newXMLFilter(new StreamSource(new StringReader(getStylesheet2())));
                   // Wire the output of the reader to filter1
                   // and the output of filter1 to filter2
                   filter1.setParent(reader);
                   filter2.setParent(filter1);
                   // Set up the output stream
                   StreamResult result = new StreamResult(System.out);
                   // Set up the transformer to process the SAX events generated
                   // by the last filter in the chain
                   Transformer transformer = stf.newTransformer();
                   transformer.setErrorListener(new ErrorListener() {
                        public void error(TransformerException te)
                                  throws TransformerException {
                             System.out.println("Transform generated Transform Error");
                             System.out.println(te.getMessage());
                        public void fatalError(TransformerException te)
                                  throws TransformerException {
                             System.out
                                       .println("Transform generated Transform Fatal Error");
                             System.out.println(te.getMessage());
                        public void warning(TransformerException te)
                                  throws TransformerException {
                             System.out.println("Transform generated Transform Warning");
                             System.out.println(te.getMessage());
                   SAXSource transformSource = new SAXSource(filter2, input);
                   transformer.transform(transformSource, result);
              } catch (TransformerConfigurationException tce) {
                   // Error generated by the parser
                   System.out.println("\n** Transformer Factory error");
                   System.out.println("   " + tce.getMessage());
                   // Use the contained exception, if any
                   Throwable x = tce;
                   if (tce.getException() != null)
                        x = tce.getException();
                   x.printStackTrace();
              } catch (TransformerException te) {
                   // Error generated by the parser
                   System.out.println("\n** Transformation error");
                   System.out.println("   " + te.getMessage());
                   // Use the contained exception, if any
                   Throwable x = te;
                   if (te.getException() != null)
                        x = te.getException();
                   x.printStackTrace();
              } catch (SAXException sxe) {
                   // Error generated by this application
                   // (or a parser-initialization error)
                   Exception x = sxe;
                   if (sxe.getException() != null)
                        x = sxe.getException();
                   x.printStackTrace();
              } catch (ParserConfigurationException pce) {
                   // Parser with specified options can't be built
                   pce.printStackTrace();
         static private void testXsltSingleTransform() {
              try {
                   BufferedInputStream bis = new BufferedInputStream(
                             new ByteArrayInputStream(getExample().getBytes()));
                   // Set up the output stream
                   StreamResult result = new StreamResult(System.out);
                   InputSource input = new InputSource(bis);
                   TransformerFactory factory = TransformerFactory.newInstance();
                   Source source = new StreamSource(new StringReader(getStylesheet2()));
                   Transformer stylesheet = factory.newTransformer(source);
                   stylesheet.setErrorListener(new ErrorListener() {
                        public void error(TransformerException te)
                                  throws TransformerException {
                             System.out.println("Transform generated Transform Error");
                             System.out.println(te.getMessage());
                        public void fatalError(TransformerException te)
                                  throws TransformerException {
                             System.out
                                       .println("Transform generated Transform Fatal Error");
                             System.out.println(te.getMessage());
                        public void warning(TransformerException te)
                                  throws TransformerException {
                             System.out.println("Transform generated Transform Warning");
                             System.out.println(te.getMessage());
                   SAXSource transformSource = new SAXSource(input);
                   stylesheet.transform(transformSource, result);
              } catch (Exception exc) {
                   exc.printStackTrace();
         private static String getStylesheet1() {
              return
                     "<?xml version='1.0' encoding='ISO-8859-1'?>" + newline
                   + "<xsl:stylesheet" + newline
                   + "xmlns:xsl='http://www.w3.org/1999/XSL/Transform'" + newline
                   + "version='1.0'>" + newline
                   + "<xsl:output method='xml'/>" + newline
                   + "<xsl:template match='/'>" + newline
                   + "<DontCareAboutContent></DontCareAboutContent>" + newline
                   + "</xsl:template>" + newline
                   + "</xsl:stylesheet>" + newline;
         private static String getStylesheet2() {
              return
                     "<?xml version='1.0' encoding='ISO-8859-1'?>" + newline
                   + "<xsl:stylesheet" + newline
                   + "xmlns:xsl='http://www.w3.org/1999/XSL/Transform'" + newline
                   + "version='1.0'>" + newline
                   + "<xsl:output method='html'/>" + newline
                   + "<xsl:template match='/'>" + newline
                  + "<html><body>" + newline
                 + "<xsl:message>" + newline
                 + "Error Message for the xslt processor" + newline
                 + "</xsl:message>" + newline
                 + "Dont care about the xslt content," + newline
                 + "The only significant part is the xsl:message element" + newline
                 + "which results in a error to be handled by the xslt" + newline
                 + "processor" + newline
                   + "</body></html>" + newline
                   + "</xsl:template>" + newline
                   + "</xsl:stylesheet>" + newline;
         private static String getExample() {
              return
                     "<?xml version='1.0' encoding='ISO-8859-1'?>" + newline
                   + "<DontCareAboutContent>" + newline
                   + "</DontCareAboutContent>" + newline;
    }

    I made the following change which solves the problem but the sollution is tightly coupled to Xalan. If anyone has any ideas, I would still like to find a way to achieve the desired result using JAXP API's.
                   XMLFilter filter1 = stf.newXMLFilter(new StreamSource(new StringReader(getStylesheet1())));
                   if (filter1 instanceof org.apache.xalan.transformer.TrAXFilter) {
                        ((org.apache.xalan.transformer.TrAXFilter)filter1).getTransformer().setErrorListener(...);
    ...

  • Best way to Transform one XML to another XML using SSIS

    I am using Altova Mapforce tool to transform one XML into another XML, but now we are planning to use any other best way of using XSLT transformation. Could you guys shed me some light on different approaches and which one would be best based on time and
    cost constraint.
    In Altova Map force Tool, due to below reasons we wanted to avoid that,
    Resource cannot work parallely in single XSLT
    Maintanence is very difficult
    Licensed version of tool
    When the XML size grows, new resourse are feeling difficult to edit the XSLT etc.,
    Related to .net approaches would be best.
    Can we create XSLT by using SSIS to transfrom one xml into another xml format.
    In .net code we can do, but feel that it is time consuming due to large size of XML.
    Any other best way for handling XSLT transformation or any tool

    XSLT can be applied to an XML file fed as a source, but it does not resolve #1. I by the way do not understand this point, technically.
    For SSIS to go live in prod you need to deplete a SQL Server license (CPU + CALs).
    The time to transform in SSIS will be equal to doing in .net code as the whole SSIS is coded in .Net, too.
    Here, in this section of the forum we may not have the needed expertise in XSLT or XML transformations.
    In my IT life, XML transformations were done in Java if on a *NIX box. On Windows a C# .net app webservice did it. The most interesting implementation, and if you are a heavy users of XML is to use a dedicated database as Base X, it has an XLSLT transform
    moduleL http://docs.basex.org/wiki/XSLT_Module
    Arthur
    MyBlog
    Twitter

  • JSP XML file parsing XSLT using Xalan

    Hi all
    I have created an XML file "view_campaign.xml" using JSP as shown in a code below and i wanna know how i should proceed to parse the XML file and so i can display this XML as the XSLT file i created.
    <%@ page import="java.sql.*" %>
    <%@ page import="java.io.*" %>
    <%
    // Identify a carriage return character for each output line
    int iLf = 10;
    char cLf = (char)iLf;
    // Create a new empty binary file, which will content XML output
    File outputFile = new File("C:\\WebContent\\view_campaigns.xml");
    //outputFile.createNewFile();
    FileWriter outfile = new FileWriter(outputFile);
    // the header for XML file
    outfile.write("<?xml version='1.0' encoding='ISO-8859-1'?>"+cLf);
    try {
         // Define connection string and make a connection to database
         //DriverManager.registerDriver (new org.apache.derby.jdbc.ClientDriver());
         Connection conn = DriverManager.getConnection("jdbc:derby://localhost:1527/sample","app","app");
         Statement stat = conn.createStatement();
         // Create a recordset
         ResultSet rset = stat.executeQuery("Select * From campagn");
         // Expecting at least one record
         if( !rset.next() ) {
              throw new IllegalArgumentException("No data found for the campaigns table");
         outfile.write("<campaigns>"+cLf);
         outfile.write("<campaign>"+cLf);
         outfile.write("<campaign_id>" + rset.getString("campagn_id") +"</campaign_id>"+cLf);
         outfile.write("<campaign_name>" + rset.getString("campagn_name") +"</campaign_name>"+cLf);
         outfile.write("<campaign_type>" + rset.getString("campagn_type") +"</campaign_type>"+cLf);
         outfile.write("<client>" + rset.getString("client_name") +"</client>"+cLf);
         outfile.write("<positions>" + rset.getString("positions_nbr") +"</positions>"+cLf);
         outfile.write("<begin>" + rset.getString("campagn_beginning_date") +"</begin>"+cLf);
         outfile.write("<close>" + rset.getString("campagn_ending_date") +"</close>"+cLf);
         outfile.write("</campaign>"+cLf);
         // Parse our recordset
    // Parse our recordset
         while(rset.next()) {
              outfile.write("<campaign>"+cLf);
              outfile.write("<campaign_id>" + rset.getString("campagn_id") +"</campaign_id>"+cLf);
              outfile.write("<campaign_name>" + rset.getString("campagn_name") +"</campaign_name>"+cLf);
              outfile.write("<campaign_type>" + rset.getString("campagn_type") +"</campaign_type>"+cLf);
              outfile.write("<client>" + rset.getString("client_name") +"</client>"+cLf);
              outfile.write("<positions>" + rset.getString("positions_nbr") +"</positions>"+cLf);
              outfile.write("<begin>" + rset.getString("campagn_beginning_date") +"</begin>"+cLf);
              outfile.write("<close>" + rset.getString("campagn_ending_date") +"</close>"+cLf);
              outfile.write("</campaign>"+cLf);
         outfile.write("</campaigns>"+cLf);
         // Everything must be closed
         rset.close();
         stat.close();
         conn.close();
         outfile.close();
    catch( Exception er ) {
    %>////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
    this is my .XSL file
    <?xml version="1.0" encoding="iso-8859-1" ?>
    - <!--  DWXMLSource="view_campaigns.xml"
      -->
      <!DOCTYPE xsl:stylesheet (View Source for full doctype...)>
    - <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
      <xsl:output method="html" encoding="iso-8859-1" doctype-public="-//W3C//DTD XHTML 1.0 Transitional//EN" doctype-system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" />
    - <xsl:template match="/">
    - <html xmlns="http://www.w3.org/1999/xhtml">
    - <head>
      <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
      <title>Gestion des campagnes</title>
      </head>
    - <body>
      Gestion des campagnes
    - <table border="1">
    - <tr bgcolor="#9acd32">
      <th align="left">Code</th>
      <th align="left">Nom</th>
      <th align="left">Type</th>
      <th align="left">Client</th>
      <th align="left">Nombre de positions</th>
      <th align="left">Date d'ouverture</th>
      <th align="left">Date de cl�ture</th>
      </tr>
    - <xsl:for-each select="campaigns/campaign">
    - <tr>
    - <td>
      <xsl:value-of select="campaign_id" />
      </td>
    - <td>
      <xsl:value-of select="campaign_name" />
      </td>
    - <td>
      <xsl:value-of select="campaign_type" />
      </td>
    - <td>
      <xsl:value-of select="client" />
      </td>
    - <td>
      <xsl:value-of select="positions" />
      </td>
    - <td>
      <xsl:value-of select="begin" />
      </td>
    - <td>
      <xsl:value-of select="close" />
      </td>
      </tr>
      </xsl:for-each>
      </table>
      </body>
      </html>
      </xsl:template>
      </xsl:stylesheet>I would be greatful that u answer my question what i should do have any exemple case study.

    Hi,
    Try this code
    JspWriter out = pageContext.getOut(); // Get JSP output writter
          javax.xml.transform.TransformerFactory tFactory = javax.xml.transform.TransformerFactory.newInstance(); //Instantiate a TransformerFactory.           
          String realPath = "c:/applyXsl.xsl";
          java.io.File file = new java.io.File(realPath); // crearte a file object for given XSL.
          // Use the TransformerFactory to process the stylesheet Source and  generate a Transformer.           
          javax.xml.transform.Transformer transformer = tFactory.newTransformer(new javax.xml.transform.stream.StreamSource(file));
          java.io.StringReader inputStream = new java.io.StringReader("c:/xmlFile.xml"); // create an input stream for given XML doc
          java.io.ByteArrayOutputStream obj = new java.io.ByteArrayOutputStream(); // Create an output stream for XSL applied XML doc.
          // 3. Use the Transformer to transform an XML Source and send the output to a Result object.
          transformer.transform(new javax.xml.transform.stream.StreamSource(inputStream), new javax.xml.transform.stream.StreamResult(obj));
          String outputString = obj.toString(); // get the XSL applied applied XML document for print
          out.println(outputString); // print the XSL applied XML in to JSP.
    however you need xercesImpl.jar  and xml-apis.jar files  to run this program.
    Regards,
    Ananth.P

  • Reading large XML file using a file event generator and a JPD process

    I am using a FileEventGenerator and a JPD Subscription process to read a large XML file. The large XML file basically contains repeated XML elements. My understanding is that the file subscription method reads the whole file in memory which causes lots of problem for huge file size like 1MB. Is there a way to read the file size-wise or is there a way to read chunks of data from a large size file..or any other alternative. I would like to process the file in a loop iteration by iteration.

    Hitejain,
    Here are a couple of pointers you could try. One is that the file event generator has a pass by reference (filename) functionality which you could use so that you could do the following inside of your process.
    1) Read file name from the reference
    2) Move the file to a processed directory (so it doesn't get picked up again. Note: I don't know how the embedded archive methods of the file event generator plays with pass by reference.
    3) Open a stream to the file.
    4) Use a SAX or SAX - DOM combined approach to parse your XML while managing the memory usage inside of your process
    There is another possibility which might fit your needs and it is related to the RawData object that BEA provides. If I understand it correctly provides wrapping functionality around a stream object, but depending on your parsing methods might just postpone the problem.
    Hope this helps
    Chris Falling
    Stormforge Software

  • Loading Large XML files  using plsql

    I have a process where there is a need to load large xml files (i.e. easily over 500k or more) into Oracle via an interface. Preference would be to use plsql or some plsql based utility if possible. I am looking for any suggestions on the best method to accomplish this. Currently running on 9.2.0.6. Thanks in advance.

    I have a process where there is a need to load large xml files (i.e. easily over 500k or more) into Oracle via an interface. Preference would be to use plsql or some plsql based utility if possible. I am looking for any suggestions on the best method to accomplish this. Currently running on 9.2.0.6. Thanks in advance.

  • Parsing large xml file and display using swing

    Hi all,
    I want to read a large xml file and display graphically in swing as a tree structure.
    I implemented it and works fine for files of 5MB size after increasing the jvm heap size (-Xmx). If the file size is larger than 5MB it throws out of memory error. I'm creating a custom datastructure from the xml and I'm using sax parsing.
    After displaying the datastructure, the user could do some operation on this, like search etc.
    Can any of you suggest a method, to support larger files ? What I'm looking for is create the datastructure in file system, rather than in memory.
    Any other tips for memory management would be greatly appreciated
    Thanks in Advance.
    Nisha

    Use a memory-mapped file?
    http://javaalmanac.com/egs/java.nio/CreateMemMap.html

  • Parsing the large XML ( 1GB) using SAX PARSER

    We have very large XMLs being generated by processes. These XMLs should be validated first and then parsed next. We have implementation that works for small files. My question is
    how can we validate and parse large XMLs with SAX parser?

    The same way as parsing a small XML file, no? Why don't you try it? Then if you have problems that you can't solve, ask about them here.

  • Is there a way to import large XML files into HANA efficiently are their any data services provided to do this?

    1. Is there a way to import large XML files into HANA efficiently?
    2. Will it process it node by node or the entire file at a time?
    3. Are there any data services provided to do this?
    This for a project use case i also have an requirement to process bulk XML files, suggest me to accomplish this task

    Hi Patrick,
         I am addressing the similar issue. "Getting data from huge XMLs into Hana."
    Using Odata services can we handle huge data (i.e create schema/load into Hana) On-the-fly ?
    In my scenario,
    I get a folder of different complex XML files which are to be loaded into Hana database.
    Then I gotta transform & cleanse the data.
    Can I use oData services to transform and cleanse the data ?
    If so, how can I create oData services dynamically ?
    Any help is highly appreciated.
    Thank you.
    Regards,
    Alekhya

Maybe you are looking for

  • Considering switching to Mac

    Hello everyone, i have been a PC/Windows user all of my life, and am now very seriously considering switching over to mac. However, i have some questions first that i cant seem to find a direct answer to, and apologize in advance for asking things th

  • SORT butttons on custom screen for custom fields

    Hi all, I have a dedveloped a custom screen in which there are 5-6 fields.Nowi need to put Sort buttons below these fields so that I can sort these fields. How do i do this???Is there std FM i can use to achieve the sort functionality?

  • Problem trying to log on to my video library system

    hello. i am trying to log on to the video library system that i just developed. the LogOn program works to a certain extent (the appropriate programs are shown below). when i try to log onto my system time. a JOptionPane.ERROR_MESSAGE message appears

  • How to stop light fading when reading a book o iPad

    Whenever I am reading a book on my iPad the light keeps dimming, I have to touch the screen to increase the level. Is there a way to stop this

  • Can't find the application program in itunes

    Can't find the application program in itunes!! Can charge, see a picture, is unable to find application in ITUNES