Slow extraction in big XML-Files with PL/SQL

Question

Hello,
i have a performance problem with the extraction from attributes in big XML Files. I tested with a size of ~ 30 mb.
The XML file is a response of a webservice. This response include some metadata of a document and the document itself. The document is inline embedded with a Base64 conversion.  Here is an example of a XML File i want to analyse:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <ns2:GetDocumentByIDResponse xmlns:ns2="***">
         <ArchivedDocument>
            <ArchivedDocumentDescription version="1" currentVersion="true" documentClassName="Allgemeines Dokument" csbDocumentID="***">
               <Metadata archiveDate="2013-08-01+02:00" documentID="123">
                  <Descriptor type="Integer" name="fachlicheId">
                     <Value>123<Value>
                  </Descriptor>
                  <Descriptor type="String" name="user">
                     <Value>***</Value>
                  </Descriptor>
                  <InternalDescriptor type="Date" ID="DocumentDate">
                     <Value>2013-08-01+02:00</Value>
                  </InternalDescriptor>
                  <!-- Here some more InternalDescriptor Nodes -->
               </Metadata>
               <RepresentationDescription default="true" description="Description" documentPartCount="1" mimeType="application/octet-stream">
                  <DocumentPartDescription fileName="20mb.test" mimeType="application/octet-stream" length="20971520 " documentPartNumber="0" hashValue=""/>
               </RepresentationDescription>
            </ArchivedDocumentDescription>
            <DocumentPart mimeType="application/octet-stream" length="20971520 " documentPartNumber="0" representationNumber="0">
               <Data fileName="20mb.test">
                  <BinaryData>
                    <!-- Here is the BASE64 converted document -->
                  </BinaryData>
               </Data>
            </DocumentPart>
         </ArchivedDocument>
      </ns2:GetDocumentByIDResponse>
   </soap:Body>
</soap:Envelope>
Now i want to extract the filename and the Base64 converted document from this XML response.
For the extraction of the filename i use the following command:
v_filename := apex_web_service.parse_xml(v_xml, '//ArchivedDocument/ArchivedDocumentDescription/RepresentationDescription/DocumentPartDescription/@fileName');
For the extraction of the binary data i use the following command:
v_clob := apex_web_service.parse_xml_clob(v_xml, '//ArchivedDocument/DocumentPart/Data/BinaryData/text()');
My problem is the performance of this extraction. Here i created some summary of the start and end time for the commands:
Start Time
End Time
Difference
Command
10.09.13 - 15:46:11,402668000
10.09.13 - 15:47:21,407895000
00:01:10,005227
v_filename_bcm := apex_web_service.parse_xml(v_xml, '//ArchivedDocument/ArchivedDocumentDescription/RepresentationDescription/DocumentPartDescription/@fileName');
10.09.13 - 15:47:21,407895000
10.09.13 - 15:47:22,336786000
00:00:00,928891
v_clob := apex_web_service.parse_xml_clob(v_xml, '//ArchivedDocument/DocumentPart/Data/BinaryData/text()');
As you can see the extraction of the filename is slower then the document extraction. For the Extraction of the filename i need ~01
I wonder about it and started some tests.
I tried to use an exact - non dynamic - filename. So i have this commands:
v_filename := '20mb_1.test';
v_clob := apex_web_service.parse_xml_clob(v_xml, '//ArchivedDocument/DocumentPart/Data/BinaryData/text()');
Under this Conditions the time for the document extraction soar. You can see this in the following table:
Start Time
End Time
Difference
Command
10.09.13 - 16:02:33,212035000
10.09.13 - 16:02:33,212542000
00:00:00,000507
v_filename_bcm := '20mb_1.test';
10.09.13 - 16:02:33,212542000
10.09.13 - 16:03:40,342396000
00:01:07,129854
v_clob := apex_web_service.parse_xml_clob(v_xml, '//ArchivedDocument/DocumentPart/Data/BinaryData/text()');
So i'm looking for a faster extraction out of the xml file. Do you have any ideas? If you need more informations, please ask me.
Thank you,
Matthias
PS: I use the Oracle 11.2.0.2.0

Answer 1

Although using an XML schema is a good advice for an XML-centric application, I think it's a little overkill in this situation.
Here are two approaches you can test :
Using the DOM interface over your XMLType variable, for example :
DECLARE
  v_xml    xmltype := xmltype('<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> 
       <soap:Body> 
          <ns2:GetDocumentByIDResponse xmlns:ns2="***"> 
             <ArchivedDocument> 
                <ArchivedDocumentDescription version="1" currentVersion="true" documentClassName="Allgemeines Dokument" csbDocumentID="***"> 
                   <Metadata archiveDate="2013-08-01+02:00" documentID="123"> 
                      <Descriptor type="Integer" name="fachlicheId"> 
                         <Value>123</Value> 
                      </Descriptor> 
                      <Descriptor type="String" name="user"> 
                         <Value>***</Value> 
                      </Descriptor> 
                      <InternalDescriptor type="Date" ID="DocumentDate"> 
                         <Value>2013-08-01+02:00</Value> 
                      </InternalDescriptor> 
                      <!-- Here some more InternalDescriptor Nodes --> 
                   </Metadata> 
                   <RepresentationDescription default="true" description="Description" documentPartCount="1" mimeType="application/octet-stream"> 
                      <DocumentPartDescription fileName="20mb.test" mimeType="application/octet-stream" length="20971520 " documentPartNumber="0" hashValue=""/> 
                   </RepresentationDescription> 
                </ArchivedDocumentDescription> 
                <DocumentPart mimeType="application/octet-stream" length="20971520 " documentPartNumber="0" representationNumber="0"> 
                   <Data fileName="20mb.test"> 
                      <BinaryData> 
                        ABC123 
                      </BinaryData> 
                   </Data> 
                </DocumentPart> 
             </ArchivedDocument> 
          </ns2:GetDocumentByIDResponse> 
       </soap:Body> 
    </soap:Envelope>');
  domDoc    dbms_xmldom.DOMDocument;
  docNode   dbms_xmldom.DOMNode;
  node      dbms_xmldom.DOMNode;
  nsmap     varchar2(2000) := 'xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="***"';
  xpath_pfx varchar2(2000) := '/soap:Envelope/soap:Body/ns2:GetDocumentByIDResponse/';
  istream   sys.utl_characterinputstream;
  buf       varchar2(32767);
  numRead   pls_integer := 1;
  filename       varchar2(30);
  base64clob     clob;
BEGIN
  domDoc := dbms_xmldom.newDOMDocument(v_xml);
  docNode := dbms_xmldom.makeNode(domdoc);
  filename := dbms_xslprocessor.valueOf(
                docNode
              , xpath_pfx || 'ArchivedDocument/ArchivedDocumentDescription/RepresentationDescription/DocumentPartDescription/@fileName'
              , nsmap
  node := dbms_xslprocessor.selectSingleNode(
            docNode
          , xpath_pfx || 'ArchivedDocument/DocumentPart/Data/BinaryData/text()'
          , nsmap
  --create an input stream to read the node content :
  istream := dbms_xmldom.getNodeValueAsCharacterStream(node);
  dbms_lob.createtemporary(base64clob, false);
  -- read the content in 32k chunk and append data to the CLOB :
  loop
    istream.read(buf, numRead);
    exit when numRead = 0;
    dbms_lob.writeappend(base64clob, numRead, buf);
  end loop;
  -- free resources :
  istream.close();
  dbms_xmldom.freeDocument(domDoc);
END;
Using a temporary XMLType storage (binary XML) :
create table tmp_xml of xmltype
xmltype store as securefile binary xml;
insert into tmp_xml values( v_xml );
select x.*
from tmp_xml t
   , xmltable(
       xmlnamespaces(
         'http://schemas.xmlsoap.org/soap/envelope/' as "soap"
       , '***' as "ns2"
     , '/soap:Envelope/soap:Body/ns2:GetDocumentByIDResponse/ArchivedDocument/DocumentPart/Data'
       passing t.object_value
       columns filename    varchar2(30) path '@fileName'
             , base64clob  clob         path 'BinaryData'
     ) x