Loading XML tables.

Question

Hi,
I have managed to successfully register xml schema and create the xml tables. I now need to load the data into these tables. I am now working on different scenarios of loading the XML data into these tables.
I successfully experimented with the SQL/PLSQL options as well as SQL*Loader. I am reluctant to try the C and JAVA based interfaces because, frankly that is a completely new field for me and will present a huge learning curve that I won’t be able to pull off in short time I need to put this application in production.
My scenario, I have about 250 XML tables which I’m going to populate from XML data files containing thousands of records. I need the lode process to be automatic such that I can do scheduled runs.
The problem I have is SQL*Loader, and I am strictly going by the example given in the Oracle documentation, is that it requires for me to put ‘0’ for every record in the file. Here’s what my sample control file looks like,
LOAD DATA
INFILE *
INTO TABLE "MCF_Custom_Form_Date_Rg" APPEND
xmltype(xmldata)
FIELDS
xmldata LOBFILE (CONSTANT sampleload1.xml) TERMINATED BY '[XML]'
BEGINDATA
0
0
0
This method works if I have finite number of records which I’ll need to count before hand and add a ‘0’ for each record. That’s not an optimal solution because I will not know the number of records before hand. Is there any way of changing this behavior?
I am kind of stumped. What are my other alternatives? I read some where about SAX load utility. Where can I find more documentation about it? Can I run automated loading programs using it?
Any help on this would be greatly appreciated.
Regards,
AVJ.

Answer 1

Anand, you might want to try something using xmldb repository since it allows you to do the FTP operations for the xml files and FTP operations can be scheduled.
First check of your database already accepts FTP operations.
From the command prompt type
FTP
FTP> OPEN <DATABASE SERVER NAME> 2100
Next it asks for a user which you can give database user and password.
Now the lets consider this example schema.
<!-- edited with XML Spy v4.0 U (http://www.xmlspy.com) by Mark (Drake) -->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xdb="http://xmlns.oracle.com/xdb" version="1.0">
     <xs:element name="PurchaseOrder" type="PurchaseOrderType" xdb:defaultTable="PURCHASEORDER"/>
     <xs:complexType name="PurchaseOrderType" xdb:SQLType="XDBPO_TYPE">
          <xs:sequence>
               <xs:element ref="Reference"/>
               <xs:element name="Actions" type="ActionsType" xdb:SQLName="ACTIONS"/>
               <xs:element name="Reject" type="RejectType" minOccurs="0" xdb:SQLName="REJECTION"/>
               <xs:element ref="Requestor"/>
               <xs:element ref="User"/>
               <xs:element ref="CostCenter"/>
               <xs:element name="ShippingInstructions" type="ShippingInstructionsType" xdb:SQLName="SHIPPINGINSTRUCTIONS"/>
               <xs:element ref="SpecialInstructions"/>
               <xs:element name="LineItems" type="LineItemsType" xdb:SQLName="LINEITEMS"/>
          </xs:sequence>
     </xs:complexType>
     <xs:complexType name="LineItemsType" xdb:SQLType="XDBPO_LINEITEMS_TYPE">
          <xs:sequence>
               <xs:element name="LineItem" type="LineItemType" maxOccurs="unbounded" xdb:SQLName="LINEITEM" xdb:SQLCollType="XDBPO_LINEITEM_COLLECTION"/>
          </xs:sequence>
     </xs:complexType>
     <xs:complexType name="LineItemType" xdb:SQLType="XDBPO_LINEITEM_TYPE">
          <xs:sequence>
               <xs:element ref="Description"/>
               <xs:element ref="Part"/>
          </xs:sequence>
          <xs:attribute name="ItemNumber" type="xs:integer" xdb:SQLName="ITEMNUMBER" xdb:SQLType="NUMBER"/>
     </xs:complexType>
     <xs:element name="Reference" xdb:SQLName="REFERENCE" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="18"/>
                    <xs:maxLength value="30"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="Part" xdb:SQLType="XDBPO_PART_TYPE" xdb:defaultTable="">
          <xs:complexType>
               <xs:attribute name="Id" xdb:SQLName="PARTNO" xdb:SQLType="VARCHAR2">
                    <xs:simpleType>
                         <xs:restriction base="xs:string">
                              <xs:minLength value="12"/>
                              <xs:maxLength value="14"/>
                         </xs:restriction>
                    </xs:simpleType>
               </xs:attribute>
               <xs:attribute name="Quantity" type="money" xdb:SQLName="Quantity" xdb:SQLType="NUMBER"/>
               <xs:attribute name="UnitPrice" type="quantity" xdb:SQLName="UNITPRICE" xdb:SQLType="NUMBER"/>
          </xs:complexType>
     </xs:element>
     <xs:complexType name="ActionsType" xdb:SQLType="XDBPO_ACTIONS_TYPE">
          <xs:sequence>
               <xs:element name="Action" maxOccurs="4" xdb:SQLName="ACTION" xdb:SQLCollType="XDBPO_ACTION_COLLECTION">
                    <xs:complexType xdb:SQLType="XDBPO_ACTION_TYPE">
                         <xs:sequence>
                              <xs:element ref="User"/>
                              <xs:element ref="Date" minOccurs="0"/>
                         </xs:sequence>
                    </xs:complexType>
               </xs:element>
          </xs:sequence>
     </xs:complexType>
     <xs:complexType name="RejectType" xdb:SQLType="XDBPO_REJECTION_TYPE">
          <xs:all>
               <xs:element ref="User" minOccurs="0"/>
               <xs:element ref="Date" minOccurs="0"/>
               <xs:element ref="Comments" minOccurs="0"/>
          </xs:all>
     </xs:complexType>
     <xs:complexType name="ShippingInstructionsType" xdb:SQLType="XDBPO_SHIPINSTRUCTIONS_TYPE">
          <xs:sequence>
               <xs:element ref="name"/>
               <xs:element ref="address"/>
               <xs:element ref="telephone"/>
          </xs:sequence>
     </xs:complexType>
     <xs:simpleType name="money">
          <xs:restriction base="xs:decimal">
               <xs:fractionDigits value="2"/>
               <xs:totalDigits value="12"/>
          </xs:restriction>
     </xs:simpleType>
     <xs:simpleType name="quantity">
          <xs:restriction base="xs:decimal">
               <xs:fractionDigits value="4"/>
               <xs:totalDigits value="8"/>
          </xs:restriction>
     </xs:simpleType>
     <xs:element name="User" xdb:SQLName="USERID" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="10"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="Requestor" xdb:SQLName="REQUESTOR" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="0"/>
                    <xs:maxLength value="128"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="CostCenter" xdb:SQLName="COSTCENTER" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="4"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="Vendor" xdb:SQLName="VENDOR" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="0"/>
                    <xs:maxLength value="20"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="PONumber" xdb:SQLName="PONUMBER" xdb:SQLType="NUMBER" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:integer"/>
          </xs:simpleType>
     </xs:element>
     <xs:element name="SpecialInstructions" xdb:SQLName="SPECIALINSTRUCTIONS" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="0"/>
                    <xs:maxLength value="2048"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="name" xdb:SQLName="SHIPTONAME" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="20"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="address" xdb:SQLName="ADDRESS" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="256"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="telephone" xdb:SQLName="PHONE" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="24"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="Date" type="xs:date" xdb:SQLName="DATEACTIONED" xdb:SQLType="DATE" xdb:defaultTable=""/>
     <xs:element name="Comments" xdb:SQLName="COMMENTS" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="2048"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
     <xs:element name="Description" xdb:SQLName="DESCRIPTION" xdb:SQLType="VARCHAR2" xdb:defaultTable="">
          <xs:simpleType>
               <xs:restriction base="xs:string">
                    <xs:minLength value="1"/>
                    <xs:maxLength value="256"/>
               </xs:restriction>
          </xs:simpleType>
     </xs:element>
</xs:schema>
Here you can check at the top there is defaultTable="PURCHASEORDER". So after the registration of this schema you will find a table PURCHASEORDER in your database. This table is created only if the GENTABLE parameter value was set to TRUE during schema registration.
Now you can FTP the xml-instance file. Note during the registration of the xml schema, you would have given a URL to identify this schema uniquely in the database.
This URL should be present in the xml-instance file as part of noNamespaceSchemaLocation tag.
Now for FTP operations you need couple of things like a folder path on the reposity. see the documentation on how create the folder on the XMLDB repository.
Now FTP the xml instance file to the folder. Automatically using the value specified in the noNamespaceSchemaLocation tag, Oracle recognizes that this instance file belongs to a particular XML schema which has a defaultTable PURCHASEORDER. It will push this xml file automatically into this table.
you can check that using select count(*) from purchaseorder after the FTP operation.
You can automate this process of FTP operations.
refer documentation XMLDB Developers Guide, it has detail information about repository and how to query your files in the repository.
good luck.