XML PL/SQL Parser Out of memory

Im parsing a 20M file and recieving the following result.
I have tried to cut the file size down and have found a small
version that will parse successfully.
We have tried adjusting
the ulimit -d 2097152
ulimit -s 32768
and also set java_pool_size = 41943040 in init.ora
none of these seemed to be enough to enable parsing of the file.
The following are 2 executions of the parser using 2 different
file sizes.
Thanks,
Steve
BEGIN
domsample('/u01/app/oracle/xmlparser/samp','dan55.xml','errors.tx
t'); END;
ERROR at line 1:
ORA-29554: unhandled Java out of memory condition
BEGIN
domsample('/u01/app/oracle/xmlparser/samp','dan60.xml','errors.tx
t'); END;
ERROR at line 1:
ORA-29532: Java call terminated by uncaught Java exception:
java.lang.OutOfMemoryError
ORA-06512: at "PHASE2.XMLPARSERCOVER", line 0
ORA-06512: at "PHASE2.XMLPARSER", line 118
ORA-06512: at "PHASE2.DOMSAMPLE", line 84
ORA-06512: at line 1
null

Steve (guest) wrote:
: Oracle XML Team wrote:
: : Steve Coffman (guest) wrote:
: : : Im parsing a 20M file and recieving the following result.
: : : I have tried to cut the file size down and have found a
small
: : : version that will parse successfully.
: : : We have tried adjusting
: : : the ulimit -d 2097152
: : : ulimit -s 32768
: : : and also set java_pool_size = 41943040 in init.ora
: : : none of these seemed to be enough to enable parsing of the
: : file.
: : : The following are 2 executions of the parser using 2
: different
: : : file sizes.
: : : Thanks,
: : : Steve
: : : BEGIN
: : : domsample
: : ('/u01/app/oracle/xmlparser/samp','dan55.xml','errors.tx
: : : t'); END;
: : : ERROR at line 1:
: : : ORA-29554: unhandled Java out of memory condition
: : : BEGIN
: : : domsample
: : ('/u01/app/oracle/xmlparser/samp','dan60.xml','errors.tx
: : : t'); END;
: : : ERROR at line 1:
: : : ORA-29532: Java call terminated by uncaught Java exception:
: : : java.lang.OutOfMemoryError
: : : ORA-06512: at "PHASE2.XMLPARSERCOVER", line 0
: : : ORA-06512: at "PHASE2.XMLPARSER", line 118
: : : ORA-06512: at "PHASE2.DOMSAMPLE", line 84
: : : ORA-06512: at line 1
: : On what OS and with how much installed memory are you
running?
: : Oracle XML Team
: : http://technet.oracle.com
: : Oracle Technology Network
: The server is a
: Digital UNIX V4.0E (Rev. 1091)
: with 2Gig Ram
Oracle version 8.1.5
null

Similar Messages

XML PL/SQL Parser Demo Error ORA-29516

I'm trying to get the XML Plsql demo to work on the following
system:
Sun Solaris 8
Oracle 8.1.7.0.0 64 bit
using the $ORACLE_HOME/xdk/plsql/demo files
I created an oracle user "xmldba" , granted javauserpriv to
xmldba, granted javasyspriv to xmldba. I have run
$ORACLE_HOME/xdk/plsql/parser/bin/load.sql. Then I try the
demo -
oracle8$ sqlplus xmldba/xmldba
SQL*Plus: Release 8.1.7.0.0 - Production on Fri Nov 16 11:07:44
2001
(c) Copyright 2000 Oracle Corporation. All rights reserved.
Connected to:
Oracle8i Enterprise Edition Release 8.1.7.0.0 - 64bit Production
With the Partitioning option
JServer Release 8.1.7.0.0 - 64bit Production
SQL> @domsample
Procedure created.
No errors.
SQL> execute domsample
('/home/oracle8/app/oracle/product/8.1.7/xdk/plsql/demo','family.
xml','errors.txt');
BEGIN domsample
('/home/oracle8/app/oracle/product/8.1.7/xdk/plsql/demo','family.
xml','errors.txt'); END;
ERROR at line 1:
ORA-29516: Aurora assertion failure: Assertion failure at
joncomp.c:127
jtc_active_clint_init_ncomp_slots(oracle/xml/parser/v2/DTD, 0)
returned 0
ORA-06512: at "XMLDBA.XMLPARSERCOVER", line 0
ORA-06512: at "XMLDBA.XMLPARSER", line 57
ORA-06512: at "XMLDBA.DOMSAMPLE", line 72
ORA-06512: at line 1
Anyone have any ideas on what I might be missing or what could
cause the problem?
Thanks,
Rhea

These errors were due to not running initjvm etc. I have resolved all the jvm sqlj related problems now .
I am still seeing resolver errors . Opened a new thread for those errors.
Please respond on those
Thanks
null

OracleXML SQL utility out of memory

when selecting from a latge table.
What can I do to fix this:
java.lang.OutOfMemoryError
I would appreciate any input.

You can use the XMLGEN.getXML() to retrieve the XML as a CLOB instead of as an in-memory DOM tree.
The next release of XML SQL Utility supports stream output and SAX2 events as output as well, which are both ways of circumventing the memory usage of a DOM tree for large datasets.

XML PL/SQL parser on Oracle 8.1.6

I am trying to loadjava xmlparserv2.jar and plsql.jar . Receiving the following errors .
What is the issue ?
$ loadjava -user drug/drug -r -v plsql.jar
initialization complete
loading : oracle/xml/parser/plsql/XMLDOMImplCover
creating : oracle/xml/parser/plsql/XMLDOMImplCover
Error while retrieving errors for oracle/xml/parser/plsql/XMLDOMImplCover
ORA-00904: invalid column name
loading : oracle/xml/parser/plsql/XMLDocumentCover
creating : oracle/xml/parser/plsql/XMLDocumentCover
Error while retrieving errors for oracle/xml/parser/plsql/XMLDocumentCover
ORA-00904: invalid column name
loading : oracle/xml/parser/plsql/XSLStylesheetCover
creating : oracle/xml/parser/plsql/XSLStylesheetCover
Error while retrieving errors for oracle/xml/parser/plsql/XSLStylesheetCover
ORA-00904: invalid column name
loading : oracle/xml/parser/plsql/XMLNodeListCover
creating : oracle/xml/parser/plsql/XMLNodeListCover
Error while retrieving errors for oracle/xml/parser/plsql/XMLNodeListCover
ORA-00904: invalid column name
loading : oracle/xml/parser/plsql/XMLEntityCover
creating : oracle/xml/parser/plsql/XMLEntityCover
Error while retrieving errors for oracle/xml/parser/plsql/XMLEntityCover
ORA-00904: invalid column name
loading : oracle/xml/parser/plsql/XMLNodeCover
creating : oracle/xml/parser/plsql/XMLNodeCover
Error while retrieving errors for oracle/xml/parser/plsql/XMLNodeCover
ORA-00904: invalid column name
I appreciate any help to resolve these issues
Sree
null

These errors were due to not running initjvm etc. I have resolved all the jvm sqlj related problems now .
I am still seeing resolver errors . Opened a new thread for those errors.
Please respond on those
Thanks
null

XML -- PL/SQL parser

I am trying to read a XML document using PL/SQL into the database ( to update or insert records). Could you point me in the right direction as far as documentation and may be samples. I have never used XML before and not really familiar with this technology.
Appreciate any help.
Thanks,
Mujahid.

Take a look at OTN's XML Home page. It includes links to Oracle's XML Developer's Kit (XDK) for PL/SQL.
Regards,
-rh
null

Performance degradation in pl/sql parsing

We are trying to use xml pl/sql parser and noticed performance degradation as we run multiple times. We zeroed into the following clause:
doc := xmlparser.getDocument(p);
The first time the procedure is run the elapsed time at sqlplus is something like .45sec, but as we run repeatedly in the same session the elapsed time keeps on increasing by .02 seconds. If we log out and start fresh, we start again from .45sec.
We noticed similar degradation with
p := xmlparser.newParser;
but we got around by making the 'p' variable as package variable, initializing it once and using the same for all invocations.
Any suggestions?
Thank you.

Can I enhance the PL/SQL code for better performance ? Probably you can enhance it.
or, is this OK to take so long to process these many rows? It should take a few minutes, not several hours.
But please provide some more details like your database version etc.
I suggest to TRACE the session that executes the PL/SQL code, with WAIT events, so you'll see where and on what time is spent, you'll identify your 'problem statements very quickly' (after you or your DBA have TKPROF'ed the trace file).
SQL> alter session set events '10046 trace name context forever, level 12';
SQL> execute your PL/SQL code here
SQL> exitWill give you a .trc file in your udump directory on the server.
http://www.oracle-base.com/articles/10g/SQLTrace10046TrcsessAndTkprof10g.php
Also this informative thread can give you more ideas:
HOW TO: Post a SQL statement tuning request - template posting
as well as doing a search on 10046 at AskTom, http://asktom.oracle.com will give you more examples.
and reading Oracle's Performance Tuning Guide: http://www.oracle.com/pls/db102/to_toc?pathname=server.102%2Fb14211%2Ftoc.htm&remark=portal+%28Getting+Started%29

SAX XMLReader out of Memory!

I've been stuck on this one for a few hours now, and can't seem to be able to resolve the issue. I'm trying to parse a 50MB+ file using the apache (or 1.4 built in) SAX parser and keep running out of memory. Here are some details:
- one big outer tag
- about 1000 smaller inner tags
- one value over 200K per inner tag
About 10 or so of the innter tags parse just fine, but on the 11th, "Out Of Memory" exception!
Has anyone seen this before? (Is anyone else crazy enough to try parsing such a large file?)

i parse 100MB+ on a regular basis.
1) Are you creating objects while you are parsing?
you should parse and use the information and get rid of the info.
2) What parser are you using...some SAX parser use a buffer to help improve speed for small xml doc, but -run out of memory- for large doc 20MB+.
3) are you saving everything??? the hold 50MB in local/global variables?

Connecting through java to HSQLDB: Out of Memory

Through my java program i am connecting to hsqldb which has .data file of approx 6.5 GB and getting the error java.sql.SqlException: out of memory (am not getting anything else)
have increased jvm option -Xms256m -Xmx1024m -XX:PermSize=64M -XX:MaxPermSize=2000M
What else can be tried...??? A complete Stacktrace for the same
exception in thread HSQL timer
java.lang.OutofMemoryError: Java Heap Space
Besides that also got this on netbeans console but seems this is something else... and not an error
Mar 19, 2009 4:55:36 AM sun.awt.X11.XToolkit processException
WARNING: Exception on Toolkit thread
java.lang.OutOfMemoryError: Java heap space
Edited by: shubham on Mar 19, 2009 1:57 AM
Edited by: shubham on Mar 19, 2009 3:31 AM

Well The issue was somewhere else due to it reading too much of data. It ran but my program is very slow (approx 2-3 hrs).
I am worried about the speed...
is it because of the large amount of data.. 23,355,459 records..?? or there could be some issue in programming probably memory leak etc..??
-> reads db
-> for each record set
-> perform different operation/ writes to file

PL/SQL parser installation error

I tried to install the XML PL/SQL parser against Oracle 8.1.6 with JServer Release 8.1.6.0.0 and I received a large number of errors about invalid column names including the following:
Error while resolving class oracle/xml/parser/v2/FromDescendantAttributes
ORA-00904: invalid column name
Error while resolving class oracle/xml/parser/v2/XSLNode
ORA-00904: invalid column name
Error while resolving class oracle/xml/parser/v2/UnaryExpr
ORA-00904: invalid column name
Any suggestions?
Thanks

Hi Steven:
Yep...I just checked our install and there are no java classes in user_objects. We had the init*.ora file shared_pool_size set too low (as indicated under $ORACLE_HOME/javavm/doc/readme.txt).
So I'll get our DBA to try it again.
Thanks,
Derek
<BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by Steven Muench ([email protected]):
This likely means that the database JavaVM is not properly installed in your Oracle8i database. You need to run the initjvm.sql script as detailed in the JServer documentation before any 'loadjava'
commands will work.<HR></BLOCKQUOTE>
null

Oracle XML CLOB out of memory error

I am running several of Oracle's XML tools
to store large XML documents in Oracle.
I am successful with a 3 MB file, but not
with a 20 MB file. I get an "out of memory"
exception in LobPlsqlUtil.class.
The CLOB is accessed through the method
CLOB.getCharacterStream(), which is then
handed off to an XMLParser class. The
exact exception is below:
Any ideas? Thanks, Rich
==========================================
Unhandled exception breakpoint occurred at line 135 in file [D:\programs\jdev\jdbc\lib\oracle8.1.7\classes12.zip]\oracle\sql\LobPlsqlUtil.class: java.lang.OutOfMemoryError.
==============================

I have some more information to add to the
original post. I found that if I access the
CLOB very fast then I can get all 20 MB of
data from the CLOB. If I access it slowly
(meaning that I pass CLOB.getCharacterStream
to a parser) then it fails.
I wrote my own InputStream as a wrapper
around CLOB, using the CLOB.getChars()
function. If I pass this input stream to
a parser (one of Oracles, or Xerces) and
get the data chunks on demand, it still
fails. If, instead, I read all the CLOB
data as fast as possible, and buffer it
locally before passing it on to the parser,
then I get all 20 MB. Go figure!!
Also, when the CLOB reading fails, it doesn't
help to get a new ResultSet, or to close and
open a new Connection and try to start where
it left off.

Large Pdf using XML XSL - Out of Memory Error

Hi Friends.
I am trying to generate a PDF from XML, XSL and FO in java. It works fine if the PDF to be generated is small.
But if the PDF to be generated is big, then it throws "Out of Memory" error. Can some one please give me some pointers about the possible reasons for this errors. Thanks for your help.
RM
Code:
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
import org.xml.sax.InputSource;
import org.xml.sax.XMLReader;
import org.apache.fop.apps.Driver;
import org.apache.fop.apps.Version;
import org.apache.fop.apps.XSLTInputHandler;
import org.apache.fop.messaging.MessageHandler;
import org.apache.avalon.framework.logger.ConsoleLogger;
import org.apache.avalon.framework.logger.Logger;
public class PdfServlet extends HttpServlet {
public static final String FO_REQUEST_PARAM = "fo";
public static final String XML_REQUEST_PARAM = "xml";
public static final String XSL_REQUEST_PARAM = "xsl";
Logger log = null;
     Com_BUtil myBu = new Com_BUtil();
public void doGet(HttpServletRequest request,
HttpServletResponse response) throws ServletException {
if(log == null) {
     log = new ConsoleLogger(ConsoleLogger.LEVEL_WARN);
     MessageHandler.setScreenLogger(log);
try {
String foParam = request.getParameter(FO_REQUEST_PARAM);
String xmlParam = myBu.getConfigVal("filePath") +"/"+request.getParameter(XML_REQUEST_PARAM);
String xslParam = myBu.SERVERROOT + "/jsp/servlet/"+request.getParameter(XSL_REQUEST_PARAM)+".xsl";
     if((xmlParam != null) && (xslParam != null)) {
XSLTInputHandler input = new XSLTInputHandler(new File(xmlParam), new File(xslParam));
renderXML(input, response);
} else {
PrintWriter out = response.getWriter();
out.println("<html><head><title>Error</title></head>\n"+
"<body><h1>PdfServlet Error</h1><h3>No 'fo' "+
"request param given.</body></html>");
} catch (ServletException ex) {
throw ex;
catch (Exception ex) {
throw new ServletException(ex);
public void renderXML(XSLTInputHandler input,
HttpServletResponse response) throws ServletException {
try {
ByteArrayOutputStream out = new ByteArrayOutputStream();
response.setContentType("application/pdf");
Driver driver = new Driver();
driver.setLogger(log);
driver.setRenderer(Driver.RENDER_PDF);
driver.setOutputStream(out);
driver.render(input.getParser(), input.getInputSource());
byte[] content = out.toByteArray();
response.setContentLength(content.length);
response.getOutputStream().write(content);
response.getOutputStream().flush();
} catch (Exception ex) {
throw new ServletException(ex);
* creates a SAX parser, using the value of org.xml.sax.parser
* defaulting to org.apache.xerces.parsers.SAXParser
* @return the created SAX parser
static XMLReader createParser() throws ServletException {
String parserClassName = System.getProperty("org.xml.sax.parser");
if (parserClassName == null) {
parserClassName = "org.apache.xerces.parsers.SAXParser";
try {
return (XMLReader) Class.forName(
parserClassName).newInstance();
} catch (Exception e) {
throw new ServletException(e);

Hi,
I did try that initially. After executing the command I get this message.
C:\>java -Xms128M -Xmx256M
Usage: java [-options] class [args...]
(to execute a class)
or java -jar [-options] jarfile [args...]
(to execute a jar file)
where options include:
-cp -classpath <directories and zip/jar files separated by ;>
set search path for application classes and resources
-D<name>=<value>
set a system property
-verbose[:class|gc|jni]
enable verbose output
-version print product version and exit
-showversion print product version and continue
-? -help print this help message
-X print help on non-standard options
Thanks for your help.
RM

Out of memory error - from parsing a "fixed width file"

This may be fairly simple for someone out there but I am trying to write a simple program that can go through a "fixed width" flat txt file and parse it to be comma dilmeted.
I use a xml file with data dictionary specifications to do the work. I do this because there are over 430 fields that need to be parsed from a fixed width with close to 250,000 lines I can read the xml file fine to get the width dimensions but when I try to apply the parsing instructions, I get an out of memory error.
I am hoping it is an error with code and not the large files. If it is the latter, does anyone out there know some techniques for getting at this data?
Here is the code
   import java.io.*;
   import org.w3c.dom.Document;
   import org.w3c.dom.*;
   import javax.xml.parsers.DocumentBuilderFactory;
   import javax.xml.parsers.DocumentBuilder;
   import org.xml.sax.SAXException;
   import org.xml.sax.SAXParseException;
    public class FixedWidthConverter{
      String[] fieldNameArray;
      String[] fieldTypeArray;
      String[] fieldSizeArray;
       public static void main(String args []){
         FixedWidthConverter fwc = new FixedWidthConverter();
         fwc.go();
         fwc.loadFixedWidthFile();
        //System.exit (0);
      }//end of main
       public void go(){
         try {
            DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
            DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
            Document doc = docBuilder.parse (new File("files/dic.xml"));
            // normalize text representation            doc.getDocumentElement ().normalize ();
            System.out.println ("Root element of the doc is " +
                 doc.getDocumentElement().getNodeName());
            NodeList listOfFields = doc.getElementsByTagName("FIELD");
            int totalFields = listOfFields.getLength();
            System.out.println("Total no of fields : " + totalFields);
            String[] fldNameArray = new String[totalFields];
            String[] fldTypeArray = new String[totalFields];
            String[] fldSizeArray = new String[totalFields];
            for(int s=0; s<listOfFields.getLength() ; s++){
               Node firstFieldNode = listOfFields.item(s);
               if(firstFieldNode.getNodeType() == Node.ELEMENT_NODE){
                  Element firstFieldElement = (Element)firstFieldNode;
                  NodeList firstFieldNMList = firstFieldElement.getElementsByTagName("FIELD_NM");
                  Element firstFieldNMElement = (Element)firstFieldNMList.item(0);
                  NodeList textFNList = firstFieldNMElement.getChildNodes();
                  //System.out.println("Field Name : " +
                           //((Node)textFNList.item(0)).getNodeValue().trim());
                  //loads values into an array
                  //fldNameArray[s] = ((Node)textFNList.item(0)).getNodeValue().trim();
                  NodeList typeList = firstFieldElement.getElementsByTagName("TYPE");
                  Element typeElement = (Element)typeList.item(0);
                  NodeList textTypList = typeElement.getChildNodes();
                  //System.out.println("Field Type : " +
                           //((Node)textTypList.item(0)).getNodeValue().trim());
                  //loads values into an array
                  //fldTypeArray[s] = ((Node)textTypList.item(0)).getNodeValue().trim();
                  NodeList sizeList = firstFieldElement.getElementsByTagName("SIZE");
                  Element sizeElement = (Element)sizeList.item(0);
                  NodeList textSizeList = sizeElement.getChildNodes();
                  //System.out.println("Field Size : " +
                           //((Node)textSizeList.item(0)).getNodeValue().trim());
                  //loads values into an array
                  fldSizeArray[s] = ((Node)textSizeList.item(0)).getNodeValue().trim();
               }//end of if clause
            }//end of for loop with s var
            //setFldNameArray(fldNameArray);
            //setFldTypeArray(fldTypeArray);
            setFldSizeArray(fldSizeArray);
             catch (SAXParseException err) {
               System.out.println ("** Parsing error" + ", line "
                  + err.getLineNumber () + ", uri " + err.getSystemId ());
               System.out.println(" " + err.getMessage ());
             catch (SAXException e) {
               Exception x = e.getException ();
               ((x == null) ? e : x).printStackTrace ();
             catch (Throwable t) {
               t.printStackTrace ();
      }//end go();
       public void setFldNameArray(String[] s){
         fieldNameArray = s;
      }//end setFldNameArray
       public void setFldTypeArray(String[] s){
         fieldTypeArray = s;
      }//end setFldTypeArray
       public void setFldSizeArray(String[] s){
         fieldSizeArray = s;
      }//end setFldSizeArray
       public String[] getFldNameArray(){
         return fieldNameArray;
      }//end setFldNameArray
       public String[] getFldTypeArray(){
         return fieldTypeArray;
      }//end setFldTypeArray
       public String[] getFldSizeArray(){
         return fieldSizeArray;
      }//end setFldSizeArray
       public int getNumLines(){
         int countLines = 0;
         try {
                //File must be in same director and be the name of the string below
            BufferedReader in = new BufferedReader(new FileReader("files/FLAT.txt"));
            String str;
            while ((str = in.readLine()) != null) {
               countLines++;
            in.close();
             catch (IOException e) {}
         return countLines;
      }//end of getNumLines
       public void loadFixedWidthFile(){
         int c = getNumLines();
         int i = 0;
         String[] lineProcessed = new String[c];
         String chars;
         try {
                //File must be in same director and be the name of the string below
            BufferedReader in = new BufferedReader(new FileReader("files/FLAT.txt"));
            String str;
            while ((str = in.readLine()) != null) {
               //System.out.println(str.length());
               lineProcessed[i] = parseThatLine(str);
               i++;
            in.close();
             catch (IOException e) {}
            //write out the lineProcess[] array to another file
         writeThatFile(lineProcessed);
      }//end loadFixedWidthFile()
       public void writeThatFile(String[] s){
         try {
            BufferedWriter out = new BufferedWriter(new FileWriter("files/outfilename.txt"));
            for(int i = 0; i < s.length -1; i++){
               out.write(s);
}//end for loop
out.close();
catch (IOException e) {}
}//end writeThatFile
public String parseThatLine(String s){
int start = 0;
int end = 0;
String parsedLine = "";
int numChars = getFldSizeArray().length;
//Print number of lines for testing
//System.out.println(numChars);
String[] oArray = getFldSizeArray();
//String chars = oArray[0];
//System.out.println(chars.length());
//oArray
for(int i = 0; i < numChars -1; i++ ){
if(i == 0){
start = 0;
end = end + Integer.parseInt(oArray[i])-1;
else
start = end;
end = end + Integer.parseInt(oArray[i]);
parsedLine = parsedLine + s.substring(start, end) + "~";
}//end for loop
return parsedLine;
}//End of parseThatLine
I have tried to illeminate as many arrays as I can thinking that was chewing up the memory but to no avail.
Any thoughts or ideas?
Message was edited by:
SaipanMan2005

You should not keep a String array of all the lines of the file read.
Instead for each line read, parse it, then write the parsed line in the other file:      public void loadFixedWidthFile() {
         BufferedReader in = null;
         BufferedWriter out = null;
         try {
            //File must be in same director and be the name of the string below
            in = new BufferedReader(new FileReader("files/FLAT.txt"));
            out = new BufferedWriter(new FileWriter("files/outfilename.txt"));
            String str;
            while ((str = in.readLine()) != null) {
               //System.out.println(str.length());
               str = parseThatLine(str);
               //write out the parsed str to another file
               out.write(str);
         catch (IOException e) {
            e.printStackTrace(); // At least print the exception - never swallow an exception
         finally { // Use a finally block to be sure of closing the files even when exception occurs
            try { in.close(); }
            catch (Exception e) {}
            try { out.close(); }
            catch (Exception e) {}
      }//end loadFixedWidthFile()Regards

Generating large amounts of XML without running out of memory

Hi there,
I need some advice from the experienced xdb users around here. I´m trying to map large amounts of data inside the DB (Oracle 11.2.0.1.0) and by large I mean files up to several GB. I compared the "low level" mapping via PL/SQL in combination with ExtractValue/XMLQuery with the elegant XML View Mapping and the best performance gave me the View Mapping by using the XMLTABLE XQuery PATH constructs. So now I have a View that lies on several BINARY XMLTYPE Columns (where the XML files are stored) for the mapping and another view which lies above this Mapping View and constructs the nested XML result document via XMLELEMENT(),XMLAGG() etc. Example Code for better understanding:
CREATE OR REPLACE VIEW MAPPING AS
SELECT type, (...) FROM XMLTYPE_BINARY, XMLTABLE ('/ROOT/ITEM' passing xml
     COLUMNS
      type       VARCHAR2(50)          PATH 'for $x in .
                                                            let $one := substring($x/b012,1,1)
                                                            let $two := substring($x/b012,1,2)
                                                            return
                                                                if ($one eq "A")
                                                                  then "A"
                                                                else if ($one eq "B" and not($two eq "BJ"))
                                                                  then "AA"
                                                                else if (...)
CREATE OR REPLACE VIEW RESULT AS
select XMLELEMENT("RESULTDOC",
                 (SELECT XMLAGG(
                         XMLELEMENT("ITEM",
                                      XMLFOREST(
                                           type "ITEMTYPE",
) as RESULTDOC FROM MAPPING;
----------------------------------------------------------------------------------------------------------------------------Now all I want to do is materialize this document by inserting it into a XMLTYPE table/column.
insert into bla select * from RESULT;
Sounds pretty easy but can´t get it to work, the DB seems to load a full DOM representation into the RAM every time I perform a select, insert into or use the xmlgen tool. This Representation takes more than 1 GB for a 200 MB XML file and eventually I´m running out of memory with an
ORA-19202: Error occurred in XML PROCESSING
ORA-04030: out of process memory
My question is how can I get the result document into the table without memory exhaustion. I thought the db would be smart enough to generate some kind of serialization/datastream to perform this task without loading everything into the RAM.
Best regards

The file import is performed via jdbc, clob and binary storage is possible up to several GB, the OR storage gives me the ORA-22813 when loading files with more than 100 MB. I use a plain prepared statement:
        File f = new File( path );
     PreparedStatement pstmt = CON.prepareStatement( "insert into " + table + " values ('" + id + "', XMLTYPE(?) )" );
     pstmt.setClob( 1, new FileReader(f) , (int)f.length() );
     pstmt.executeUpdate();
     pstmt.close(); DB version is 11.2.0.1.0 as mentioned in the initial post.
But this isn´t my main problem, the above one is, I prefer using binary xmltype anyway, much easier to index. Anyone an idea how to get the large document from the view into a xmltype table?

SharePoint 2013 Search - Zip - Parser server ran out of memory - Processing this item failed because of a IFilter parser error

Moving content databases from 2010 to 2013 August CU. Have 7 databases attached and ready to go, all the content is crawled successfully except zip files. Getting errors such as
Processing this item failed because of a IFilter parser error. ( Error parsing document 'http://sharepoint/file1.zip'. Error loading IFilter for extension '.zip' (Error code is 0x80CB4204). The function encountered an unknown error.; ; SearchID = 7A541F21-1CD3-4300-A95C-7E2A67B2563C
Processing this item failed because the parser server ran out of memory. ( Error parsing document 'http://sharepoint/file2.zip'. Document failed to be processed. It probably crashed the server.; ; SearchID = 91B5D685-1C1A-4C43-9505-DA5414E40169 )
SharePoint 2013 in a single instance out-of-the-box. Didn't install custom iFilters as 2013 supports zip. No other extensions have this issue. Range in file size from 60-90MB per zip. They contain mp3 files. I can download and unzip the file as needed.
Should I care that the index isn't being populated with these items since they contain no metadata? I am thinking I should just omit these from the crawl.

This issue came back up for me as my results aren't displaying since this data is not part of the search index.
Curious if anyone knows of a way to increase the parser server memory in SharePoint 2013 search?
http://sharepoint/materials-ca/HPSActiveCDs/Votrevieprofessionnelleetvotrecarrireenregistrement.zip
Processing this item failed because the parser server ran out of memory. ( Error parsing document 'http://sharepoint/materials-ca/HPSActiveCDs/Votrevieprofessionnelleetvotrecarrireenregistrement.zip'. Document failed to be processed. It probably crashed the
server.; ; SearchID = 097AE4B0-9EB0-4AEC-AECE-AEFA631D4AA6 )
http://sharepoint/materials-ca/HPSActiveCDs/Travaillerauseindunequipemultignrationnelle.zip
Processing this item failed because of a IFilter parser error. ( Error parsing document 'http://sharepoint/materials-ca/HPSActiveCDs/Travaillerauseindunequipemultignrationnelle.zip'. Error loading IFilter for extension '.zip' (Error code is 0x80CB4204). The
function encountered an unknown error.; ; SearchID = 4A0C99B1-CF44-4C8B-A6FF-E42309F97B72 )

Out of memory Error while querying SQL Script based Calculation View

Hi All,
I wanted to test the performance of Graphical and SQL Script based Calculation views.
Created Graphical (CA_GRPH) and SQL Script (CA_SQL) Calculation views.
Analytic View (AN_GRPH) for both Calculation views are the same which is Graphical based (90 Attributes and 5 Measures)
In Analytic View data foundation I have a Fact table which has 1.5 Billion records and 9 Dimension Tables –collectively 500 million records (7 Attribute Views). 9 Referential joins with cardinality N:1 and 1 Referential join with cardinality N:N.
I wanted to keep (CA_GRPH) and (CA_SQL) as a base Calculation views and leverage those to create various calculation views (Will be creating different Calc views for respective Business segments)
In order to test this I have created below calc views on top of base calc views.
Graphical Based: Created (CA_GRAPH_XYZ) by having CA_GRPH in projection with 30 Calculated Columns. – This retrieves data in 13 secs
SQL Script Based: Created (CA_GRPH_ABC) by having CA_SQL in projection view with 30 calculated columns – This errors out after 1.50 mins.
Could not execute 'SELECT "COLUMN_A","COLUMN _B"," COLUMN _C"," COLUMN _D", SUM("COLUMN _REVENUE") AS ...' in 1:50.480 minutes .
SAP DBTech JDBC: [2048]: column store error: [2048] column store error: search table error: [1000002] Error executing physical plan: exception 1000002:
ltt/impl/memory.cpp:63
Out of memory ; $size$=1507711; $name$=ihm; $type$=pool; $inuse_count$=170104; $allocated_size$=219215007925
exception 1000002:
Any suggestion / help in fixing this issue will be greatly appreciated.
Regards,
Av

Hi Raj,
Thanks for your time, please find edited snap hot of Analytic View (AN_GRPH) below,
Calculation view(CA_SQL)
        /********* Begin Procedure Script ************/
BEGIN
       var_out =
     SELECT
            "COLUMN_1"
            "COLUMN_2",
            "COLUMN_84",
            "COLUMN_85;",
            SUM("REVN") AS "REVN",
            SUM("MGN") AS "MGN",
            SUM("ORD_QTY") AS "ORD_QTY",
            SUM("SYS_QTY1") AS "SYS_QTY1",
            SUM("SYS_QTY") AS "SYS_QTY"
FROM
      "_SYS_BIC"."XYZ/AN_GRPH"
GROUP BY
            "COLUMN_1"
            "COLUMN_2",
            "COLUMN_84",
            "COLUMN_85";
END
/********* End Procedure Script ************/
Later i have built one more Calculation view(CA_GRPH_ABC) using (CA_SQL)in projection. i have 30 calculated measures in this final calc view. this final calc view is throwing above mentioned error.
Not sure if i can use SQL script based calc view in graphical based calc views?
Regards,
AV

XML PL/SQL Parser Out of memory

Similar Messages

Maybe you are looking for