Extract parent and child data from an XML sequence?

Question

I'm very new to XML, so please excuse me if I use incorrect terminology here. I've been given a schema definition (.xsd), and several sample xml files from which I need to extract data and insert it into relational tables. I've made some progress in that I've been able to hack enough to modify and register the schema and get the XML into the database both through the registered schema and into a separate table with an XMLType column. I've also been able to use the isSchemaValid function to validate the XML against the schema definition, and I've been able to extract some of the simple data elements via the extractValue function. My problem is with extracting some of the nested data - that is, extracting parent node elements and child node elements together where there is more than one value in the child node. BTW. I'm on Oracle 10gR2.
Following are the .xsd and and sample XML. Specifically, I need to select the NPI value from the PROVIDER node, and associate it with each of the CODE values from the SPECIALTIES node (which is part of the PROVIDER node).
Here's the .xsd (sorry, it's very long). The only modifications I've made to the schema definition are to add the xdb namespace and the "xdb:defaultTable="HMOSCHEMA" snippet:
<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xdb="http://xmlns.oracle.com/xdb" version="1.0">
<xsd:element name="HMO" xdb:defaultTable="HMOSCHEMA">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="HMO_GENERAL_INFORMATION" />
<xsd:element ref="PROVIDERS" />
<xsd:element ref="FACILITIES" minOccurs="0"
maxOccurs="1" />
</xsd:sequence>
<xsd:attribute name="DATE" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="HMO_GENERAL_INFORMATION">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="MCO_ID_LIST" minOccurs="1" maxOccurs="1" />
<xsd:element ref="HMO_WEBSITE_ADDRESS" minOccurs="1"
maxOccurs="1" />
<xsd:element ref="HMO_SPECIAL_SERVICES" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="HMO_DAILY_CONTACT_EMAIL_ADDRESS"
minOccurs="1" maxOccurs="1" />
<xsd:element ref="HMO_MONTHLY_CONTACT_EMAIL_ADDRESS"
minOccurs="1" maxOccurs="1" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="PROVIDERS">
<xsd:complexType>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" ref="PROVIDER" />
</xsd:sequence>
<xsd:attribute name="RECORDS" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="PROVIDER">
<xsd:complexType>
<xsd:sequence minOccurs="1" maxOccurs="unbounded">
<xsd:element ref="NPI" minOccurs="1" maxOccurs="1" />
<xsd:element ref="FIRST_NAME" minOccurs="1"
maxOccurs="1" />
<xsd:element ref="LAST_NAME" minOccurs="1"
maxOccurs="1" />
<xsd:element ref="MIDDLE_INITIAL" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="GENDER" minOccurs="1" maxOccurs="1" />
<xsd:element ref="LINE_1_ADDRESS" minOccurs="1"
maxOccurs="1" />
<xsd:element ref="LINE_2_ADDRESS" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="CITY" minOccurs="1" maxOccurs="1" />
<xsd:element ref="STATE" minOccurs="1" maxOccurs="1" />
<xsd:element ref="ZIP" minOccurs="1" maxOccurs="1" />
<xsd:element ref="COUNTY" minOccurs="0" maxOccurs="1" />
<xsd:element ref="PHONE_NUMBER" minOccurs="1"
maxOccurs="1" />
<xsd:element ref="CLINIC_NAME" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="CLINIC_TYPE" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="CLINIC_NPI" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="PROVIDER_OFFICE_HOURS_TENTHS"
minOccurs="0" maxOccurs="1" />
<xsd:element ref="MAXIMUM_PATIENTS" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="ACCEPT_NEW_PATIENT" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="PRIMARY_CARE_INDICATOR" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="FOREIGN_LANGUAGES" minOccurs="0"
maxOccurs="1" />
<xsd:element ref="SPECIALTIES" minOccurs="1"
maxOccurs="1" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="SPECIALTIES">
<xsd:complexType>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" name="CODE"
minOccurs="1">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{3}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="FACILITIES">
<xsd:complexType>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" ref="FACILITY_NUM"
minOccurs="0" />
</xsd:sequence>
<xsd:attribute name="RECORDS" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="FOREIGN_LANGUAGES">
<xsd:complexType>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" name="CODE"
minOccurs="0">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[A-Z]{3}" />
<xsd:pattern value="[a-z]{3}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="HMO_SPECIAL_SERVICES">
<xsd:complexType>
<xsd:sequence>
<xsd:element maxOccurs="unbounded" name="CODE"
minOccurs="0">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="MCO_ID_LIST">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="MCO_ID" minOccurs="1" maxOccurs="unbounded">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="[0-9]{3}" />
<xsd:pattern value="[0-9]{4}" />
<xsd:pattern value="[0-9]{5}" />
<xsd:pattern value="[0-9]{6}" />
<xsd:pattern value="[0-9]{7}" />
<xsd:pattern value="[0-9]{8}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="HMO_WEBSITE_ADDRESS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:maxLength value="100" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="HMO_DAILY_CONTACT_EMAIL_ADDRESS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:maxLength value="50" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="HMO_MONTHLY_CONTACT_EMAIL_ADDRESS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:maxLength value="50" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="NPI">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="[0-9]{3}" />
<xsd:pattern value="[0-9]{4}" />
<xsd:pattern value="[0-9]{5}" />
<xsd:pattern value="[0-9]{6}" />
<xsd:pattern value="[0-9]{7}" />
<xsd:pattern value="[0-9]{8}" />
<xsd:pattern value="[0-9]{9}" />
<xsd:pattern value="[0-9]{10}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="FIRST_NAME">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="LAST_NAME">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="MIDDLE_INITIAL">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="0" />
<xsd:maxLength value="1" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="GENDER">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="0" />
<xsd:maxLength value="1" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="LINE_1_ADDRESS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="LINE_2_ADDRESS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="0" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="CITY">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="1" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="STATE">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[a-zA-Z][a-zA-Z]" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="ZIP">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{5}" />
<xsd:pattern value="[0-9]{9}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="COUNTY">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="[0-9]{3}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="PHONE_NUMBER">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{10}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="CLINIC_NAME">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="0" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="CLINIC_TYPE">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:minLength value="0" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="CLINIC_NPI">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="[0-9]{3}" />
<xsd:pattern value="[0-9]{4}" />
<xsd:pattern value="[0-9]{5}" />
<xsd:pattern value="[0-9]{6}" />
<xsd:pattern value="[0-9]{7}" />
<xsd:pattern value="[0-9]{8}" />
<xsd:pattern value="[0-9]{9}" />
<xsd:pattern value="[0-9]{10}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="PROVIDER_OFFICE_HOURS_TENTHS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="MAXIMUM_PATIENTS">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="[0-9]{3}" />
<xsd:pattern value="[0-9]{4}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="ACCEPT_NEW_PATIENT">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[A-Z]{1}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="PRIMARY_CARE_INDICATOR">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[A-Z]{1}" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="FACILITY_NUM">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:pattern value="[0-9]{1}" />
<xsd:pattern value="[0-9]{2}" />
<xsd:pattern value="[0-9]{3}" />
<xsd:pattern value="[0-9]{4}" />
<xsd:pattern value="[0-9]{5}" />
<xsd:pattern value="[0-9]{6}" />
<xsd:pattern value="[0-9]{7}" />
<xsd:pattern value="[0-9]{8}" />
<xsd:pattern value="[0-9]{9}" />
<xsd:pattern value="[0-9]{10}" />
<xsd:pattern value="" />
<xsd:whiteSpace value="collapse" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:schema>
Here's a sample of the XML (significantly shortened, and names and IDs changed to protect the innocent). BTW, the XML validates cleanly against the schema definition:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<HMO DATE="03/26/2010">
<HMO_GENERAL_INFORMATION>
<MCO_ID_LIST>
<MCO_ID>11111111</MCO_ID>
</MCO_ID_LIST>
<HMO_WEBSITE_ADDRESS>HTTP://WWW.SOMEHMONAME.COM</HMO_WEBSITE_ADDRESS>
<HMO_SPECIAL_SERVICES>
<CODE>00</CODE>
</HMO_SPECIAL_SERVICES>
<HMO_DAILY_CONTACT_EMAIL_ADDRESS>somename@someurl</HMO_DAILY_CONTACT_EMAIL_ADDRESS>
<HMO_MONTHLY_CONTACT_EMAIL_ADDRESS>somename@someurl</HMO_MONTHLY_CONTACT_EMAIL_ADDRESS>
</HMO_GENERAL_INFORMATION>
<PROVIDERS RECORDS="1011">
<PROVIDER>
<NPI>1111111111</NPI>
<FIRST_NAME>JOHN</FIRST_NAME>
<LAST_NAME>DOE</LAST_NAME>
<MIDDLE_INITIAL>A</MIDDLE_INITIAL>
<GENDER>M</GENDER>
<LINE_1_ADDRESS>1111 Some Rd</LINE_1_ADDRESS>
<LINE_2_ADDRESS> </LINE_2_ADDRESS>
<CITY>NOWHERE</CITY>
<STATE>YN</STATE>
<ZIP>12345</ZIP>
<COUNTY>13</COUNTY>
<PHONE_NUMBER>1111111111</PHONE_NUMBER>
<CLINIC_NAME>SOME CLINIC - CORPORATE OFFICE</CLINIC_NAME>
<CLINIC_NPI/>
<ACCEPT_NEW_PATIENT>Y</ACCEPT_NEW_PATIENT>
<PRIMARY_CARE_INDICATOR>N</PRIMARY_CARE_INDICATOR>
<FOREIGN_LANGUAGES>
<CODE> </CODE>
</FOREIGN_LANGUAGES>
<SPECIALTIES>
<CODE>170</CODE>
</SPECIALTIES>
</PROVIDER>
<PROVIDER>
<NPI>2222222222</NPI>
<FIRST_NAME>JANE</FIRST_NAME>
<LAST_NAME>DOE</LAST_NAME>
<MIDDLE_INITIAL>J</MIDDLE_INITIAL>
<GENDER>F</GENDER>
<LINE_1_ADDRESS>2222 SOMEOTHER ST</LINE_1_ADDRESS>
<LINE_2_ADDRESS> </LINE_2_ADDRESS>
<CITY>MYTOWN</CITY>
<STATE>YN</STATE>
<ZIP>12345</ZIP>
<COUNTY>13</COUNTY>
<PHONE_NUMBER>2222222222</PHONE_NUMBER>
<CLINIC_NAME>SOME CLINIC - MYTOWN</CLINIC_NAME>
<CLINIC_NPI/>
<ACCEPT_NEW_PATIENT>Y</ACCEPT_NEW_PATIENT>
<PRIMARY_CARE_INDICATOR>N</PRIMARY_CARE_INDICATOR>
<FOREIGN_LANGUAGES>
<CODE> </CODE>
</FOREIGN_LANGUAGES>
<SPECIALTIES>
<CODE>322</CODE>
<CODE>329</CODE>
</SPECIALTIES>
</PROVIDER>
</PROVIDERS>
<FACILITIES RECORDS="2">
<FACILITY_NUM>1000000001</FACILITY_NUM>
<FACILITY_NUM>2000000002</FACILITY_NUM>
</FACILITIES>
</HMO>
As you can see, the SPECIALTIES node can contain more than one CODE value, si I can't use extractValue to retrieve them. Again, I need to extract the NPI value from the PROVIIDER node, and associate it with each of the CODE values from the SPECIALTIES node. So, what I would need to extract from this sample XML would be:
NPI CODE
1111111111 170
2222222222 322
2222222222 329
This seems like it should be a pretty routine requirement, but I've tried everything I can think of (that is, everything I've learned to this point), and have not yet found a way............
Ideally, the solution to extracting this data would utilize SQL (not PL/SQL, or Java, or XSL, etc) with XML extraction functions (e.g. extract, extractValue, XMLSequence, XMLTable, etc).
Can anyone provide me with an example of how that iterative specialy code data could be extracted? Do I need to modify the .xsd to facilitate this extraction (so that registration of the schema results in other objects in the database)?
Any suggestions or examples would be greatly appreciated.
Thanks in advance, Dan
P.S. - I hope the XML displays OK when I post this question!

Answer 1

Hi,
Thanks for your detailed sample, it's very helpful.
This seems like it should be a pretty routine requirementIndeed, that's a classic (once you've got the method).
So, here's how you can do it :
SELECT x1.npi, x2.code
FROM t,
     XMLTable(
      '/HMO/PROVIDERS/PROVIDER'
      passing t.doc
      columns
        npi number path 'NPI',
        specialties xmltype path 'SPECIALTIES'
     ) x1,
     XMLTable(
      'SPECIALTIES/CODE'
      passing x1.specialties
      columns
        code number path '.'
     ) x2
;Assuming the XML is stored in column T.DOC.
As you see, for each NPI, the trick is to pass the entire SPECIALTIES node (kind of a join) to another nested XMLTable so that each CODE could be extracted as well.
There's another method using only one XMLTable and a little more complex XQuery. I'll try to post it soon.
HTH.
Edit : 2nd method
SELECT x.*
FROM t,
     XMLTable(
      'for $i in /HMO/PROVIDERS/PROVIDER/SPECIALTIES/CODE
       return element e
        attribute npi { $i/../../NPI },
        $i/text()
      passing t.doc
      columns
        npi  number path '@npi',
        code number path '.'
     ) x
;Edited by: odie_63 on 27 mai 2010 18:43