Load XML Document/Data into relational Table
Hi All,
I have a requirement to load data in an XML file into an oracle db and subsequently split the contents into columns in an oracle table. This has to be done on the command line from a unix box which connects to a remote oracle DB.
Can some one give me a step by step procedure to do this or point me in the right direction.
thank you
Hi,
There can be many different way we can achieve this. Mainly depending on size of XML, complexity of XML and your use of that data (e.g. you just want to insert XML into relational or you want to write something and the generate new xmls later on).
If you go to Oracle online documentation -> Oracle XML DB Developer's guide, first 3-4 chapters will give you what you need.
Regards
Similar Messages
-
OWB11gR2 - simple and easy way to load XML formatted data into db tables?
Hi,
we're currently trying to load table data stored in XML files into our datawarehouse using OWB 11gR2.
However, we're finding this is not quite as trivial as loading flat files...
Most postings on this forum points to the blog-entry title "Leveraging XDB" found here (http://blogs.oracle.com/warehousebuilder/2007/09/leveraging_xdb.html).
This blog also references the zip-file owb_xml_etl_utils.zip, which seems to have disappeared from it's original location and can now be found on sourceforge.
Anyway, the solution described is for OWB 10g, and when trying to import experts from the zip-file etc. we end up not being able to run the "Create ETL from XSD" expert, as the 11gR2 client is different from the 10g and does not have the Experts menu et.al.
Also, this solution was published over 3 years ago, and it seems rather strange that importing XML-formatted data should still be so cumbersome in the newer warehouse builder releases.
The OWB 11gR2 documentation is very sparse (or rather - quite empty) on how to load XML data, all it has is a few lines on "XML Transformations", giving no clue as to how one goes about loading data.
Is this really the state of things? Or are we missing some vital information here?
We'd have thought that with 11g-releases, loading XML-data would be rather simple, quick and painless?
Is there somewhere besides the blog mentioned above where we can find simple and to the point guidelines for OWB 11gR2 on how to load XML-formatted data into Oracle tables?
Regards,
-Haakon-Yes it is possible to use SQL*Loader to parse and load XML, but that is not what it was designed for and so is not recommended. You also don't need to register a schema, just to load/store/parse XML in the DB either.
So where does that leave you?
Some options
{thread:id=410714} (see page 2)
{thread:id=1090681}
{thread:id=1070213}
Those talk some about storage options and reading in XML from disk and parsing XML. They should also give you options to consider. Without knowing more about your requirements for the effort, it is difficult to give specific advice. Maybe your 7-8 tables don't exist and so using Object Relational Storage for the XML would be the best solution as you can query/update tables that Oracle creates based off the schema associated to the XML. Maybe an External Table definition works better for reading the XML into the system because this process will happen just once. Maybe using WebDAV makes more sense for loading XML to be parsed (I don't have much experience with this, just know it is possible from what I've read on the forums). Also, your version makes a difference as you have different options available depending upon the version of Oracle.
Hope all that helps as a starter.
Edited by: A_Non on Jul 8, 2010 4:31 PM
A great example, see the answers by mdrake in {thread:id=1096784} -
Date insertion from large XML document (clob) into relation table very slow
Hi Everybody!
I'm working with Oracle 9.2.0.5 on Microsoft Windows Server 2003 Enterprise Edition.
The server (a test server) is a Pentium 4 2.8 GHz, 1GB of RAM.
I use a procedure called PARITOP_TRAITERXMLRESULTMASSE to insert the data contained in the pXMLDOC clob parameter in the table pTABLENAME. (You can see the format of the XML document below). The first step on this procedure is to verify that the XML document is not empty. If not, the procedure needs to add a node in the document, in every <ROW> tag. This added node is named âRST_IDâ. Itâs the foreign key of each record. I can retrieve the value of each <RST_ID> node in an other table in which the data has been previously added (by the calling procedure). When each of the <ROW> elements has been treated, the PARITOP_INSERTXML procedure is called. This procedure uses DBMS_XMLSAVE.INSERTXML to insert the data contained in the XML document in the specified table.
(Below, you can see the code of my procedures.)
With this information, can you tell me why this treatment is very very very slow with a large XML document and how I can improve it?
Thank you for your help!
Anne-Marie
CREATE OR REPLACE PROCEDURE "PARITOP_TRAITERXMLRESULTMASSE" (
pPRC_ID IN PARITOP_PARC.PRC_ID%TYPE,
pRST_MONDE IN PARITOP_RESULTAT.RST_MONDE%TYPE,
pXMLDOC IN CLOB,
pTABLENAME IN VARCHAR2)
AS
Objectif :Insérer le contenu du XML passé en paramètre (pXMLDOC) à la table passée en paramètre (pTABLENAME)
La table passée en paramètre doit être une table ayant comme clé étrangère le champs "RST_ID" .
(Le noeud RST_ID est donc ajouté à tous les document XML. Ce rst_id est
déterminé à partir de la table PARITOP_RESULTAT grâce à pPRC_ID et
pRstMonde fournis en paramètre)
result_doc CLOB;
XMLDOMDOC XDB.DBMS_XMLDOM.DOMDOCUMENT;
NODE_ROWSET DBMS_XMLDOM.DOMNODE;
NODE_ROW DBMS_XMLDOM.DOMNODE;
vUE_ID PARITOP_RESULTAT.UE_ID%TYPE;
vRST_ID PARITOP_RESULTAT.RST_ID%TYPE;
nodeList DBMS_XMLDOM.DOMNODELIST;
BEGIN
BEGIN
vUE_ID := 0;
vRST_ID := 0;
XMLDOMDOC := DBMS_XMLDOM.NEWDOMDOCUMENT(pXMLDOC);
IF NOT GESTXML_PKG.FN_PARITOP_DOCUMENT_IS_NULL(XMLDOMDOC) THEN
NODE_ROWSET := DBMS_XMLDOM.item(DBMS_XMLDOM.GETCHILDNODES (DBMS_XMLDOM.MAKENODE(XMLDOMDOC)),0);
for i in 0..dbms_xmldom.getLength(DBMS_XMLDOM.getchildnodes(NODE_ROWSET))-1 loop
NODE_ROW := DBMS_XMLDOM.ITEM(DBMS_XMLDOM.GETCHILDNODES(NODE_ROWSET), i) ;
nodeList := DBMS_XMLDOM.GETELEMENTSBYTAGNAME(DBMS_XMLDOM.makeelement(NODE_ROW) , 'UE_ID');
IF vUE_ID <> DBMS_XMLDOM.GETNODEVALUE(DBMS_XMLDOM.GETFIRSTCHILD(DBMS_XMLDOM.ITEM(nodeList, 0))) THEN
vUE_ID := DBMS_XMLDOM.GETNODEVALUE(DBMS_XMLDOM.GETFIRSTCHILD(DBMS_XMLDOM.ITEM(nodeList, 0)));
--on ramasse le rst_id
SELECT RST_ID INTO vRST_ID
FROM PARITOP_RESULTAT RST
WHERE RST.PRC_ID = pPRC_ID
AND RST.UE_ID = vUE_ID
AND RST.RST_MONDE = pRST_MONDE
AND RST_A_SUPPRIMER = 0;
END IF;
GESTXML_PKG.PARITOP_ADDNODETOROW(XMLDOMDOC, NODE_ROW, 'RST_ID', vRST_ID);
end loop;
RESULT_DOC := ' '; --Ã garder, pour ne pas que ca fasse d'erreur lors du WriteToClob.
dbms_xmldom.writeToClob(DBMS_XMLDOM.MAKENODE(XMLDOMDOC), RESULT_DOC);
--Insertion du document XML dans la table "tableName"
GESTXML_PKG.PARITOP_INSERTXML(RESULT_DOC, pTABLENAME);
DBMS_XMLDOM.FREEDOCUMENT( XMLDOMDOC);
end if;
EXCEPTION
[â¦exception treatementâ¦]
END;
END;
The format of a XML clob is :
<ROWSET>
<ROW>
<PRC_ID>193</PRC_ID>
<UE_ID>8781</UE_ID>
<VEN_ID>6223</VEN_ID>
<RST_MONDE>0</RST_MONDE>
<CMP_SELMAN>0</CMP_SELMAN>
<CMP_INDICESELECTION>92.307692307692307</CMP_INDICESELECTION>
<CMP_PVRES>94900</CMP_PVRES>
<CMP_PVAJUSTE>72678.017699115066</CMP_PVAJUSTE>
<CMP_PVAJUSTEMIN>72678.017699115095</CMP_PVAJUSTEMIN>
<CMP_PVAJUSTEMAX>72678.017699115037</CMP_PVAJUSTEMAX>
<CMP_PV>148000</CMP_PV>
<CMP_VALROLE>129400</CMP_VALROLE>
<CMP_PVRESECART>4790</CMP_PVRESECART>
<CMP_PVRHAB>101778.01769911509</CMP_PVRHAB>
<CMP_UTILISE>1</CMP_UTILISE>
<CMP_TVM>1</CMP_TVM>
<CMP_PVA>148000</CMP_PVA>
</ROW>
<ROW>
<PRC_ID>193</PRC_ID>
<UE_ID>8781</UE_ID>
<VEN_ID>6235</VEN_ID>
<RST_MONDE>0</RST_MONDE>
<CMP_SELMAN>0</CMP_SELMAN>
<CMP_INDICESELECTION>76.92307692307692</CMP_INDICESELECTION>
<CMP_PVRES>117800</CMP_PVRES>
<CMP_PVAJUSTE>118080</CMP_PVAJUSTE>
<CMP_PVAJUSTEMIN>118080</CMP_PVAJUSTEMIN>
<CMP_PVAJUSTEMAX>118080</CMP_PVAJUSTEMAX>
<CMP_PV>172000</CMP_PV>
<CMP_VALROLE>134800</CMP_VALROLE>
<CMP_PVRESECART>0</CMP_PVRESECART>
<CMP_PVRHAB>147180</CMP_PVRHAB>
<CMP_UTILISE>1</CMP_UTILISE>
<CMP_TVM>1</CMP_TVM>
<CMP_PVA>172000</CMP_PVA>
</ROW>
</ROWSET>
PARITOP_COMPARABLE TABLE :
RST_ID NUMBER(10) NOT NULL,
VEN_ID NUMBER(10) NOT NULL,
CMP_SELMAN NUMBER(1) NOT NULL,
CMP_UTILISE NUMBER(1) NOT NULL,
CMP_INDICESELECTION FLOAT(53) NOT NULL,
CMP_PVRES FLOAT(53) NULL,
CMP_PVAJUSTE FLOAT(53) NULL,
CMP_PVRHAB FLOAT(53) NULL,
CMP_TVM FLOAT(53) NULL
ROCEDURE PARITOP_INSERTXML (xmlDoc IN clob, tableName IN VARCHAR2)
AS
insCtx DBMS_XMLSave.ctxType;
rowss number;
BEGIN
--permet d'insérer les champs du XML dans la table passée en paramètre.
--il suffit que les champs XML aient le même nom que les champs de la table
BEGIN
insCtx := DBMS_XMLSave.newContext(tableName); -- get context handle
DBMS_XMLSAVE.SETDATEFORMAT( insCtx, 'yyyy-MM-dd HH:mm:ss');--attention, case sensitive
DBMS_XMLSAVE.setIgnoreCase(insCtx, 1);
rowss := DBMS_XMLSAVE.INSERTXML(insCtx , xmlDoc);
DBMS_XMLSave.closeContext(insCtx);
EXCEPTION
[â¦]
END;
END;
PROCEDURE PARITOP_ADDNODETOROW (
XMLDOMDOC DBMS_XMLDOM.DOMDOCUMENT,
NODE_ROW dbms_xmldom.DOMNode,
NOM_NOEUD VARCHAR2,
VALEUR_NOEUD VARCHAR2)
AS
--PERMET D'AJOUTER UN NOEUD AVEC 1 SEULE VALEUR DANS une ROW D'UN XML.
--UTILE SURTOUT POUR LES CLÃS ÃTRANGÃRES
domElemAInserer DBMS_XMLDOM.DOMELEMENT;
NODE dbms_xmldom.DOMNode;
NODE_TMP dbms_xmldom.DOMNode;
BEGIN
domElemAInserer := DBMS_XMLDOM.createElement(XMLDOMDOC, NOM_NOEUD) ;
NODE := DBMS_XMLDOM.MAKENODE(domElemAInserer); --cast
NODE := DBMS_XMLDOM.APPENDCHILD(NODE_ROW,NODE);
NODE_TMP := DBMS_XMLDOM.MAKENODE(DBMS_XMLDOM.CREATETEXTNODE(XMLDOMDOC, VALEUR_NOEUD ) );
NODE := DBMS_XMLDOM.APPENDCHILD(NODE,NODE_TMP );
END;Hi Everybody!
I'm working with Oracle 9.2.0.5 on Microsoft Windows Server 2003 Enterprise Edition.
The server (a test server) is a Pentium 4 2.8 GHz, 1GB of RAM.
I use a procedure called PARITOP_TRAITERXMLRESULTMASSE to insert the data contained in the pXMLDOC clob parameter in the table pTABLENAME. (You can see the format of the XML document below). The first step on this procedure is to verify that the XML document is not empty. If not, the procedure needs to add a node in the document, in every <ROW> tag. This added node is named âRST_IDâ. Itâs the foreign key of each record. I can retrieve the value of each <RST_ID> node in an other table in which the data has been previously added (by the calling procedure). When each of the <ROW> elements has been treated, the PARITOP_INSERTXML procedure is called. This procedure uses DBMS_XMLSAVE.INSERTXML to insert the data contained in the XML document in the specified table.
(Below, you can see the code of my procedures.)
With this information, can you tell me why this treatment is very very very slow with a large XML document and how I can improve it?
Thank you for your help!
Anne-Marie
CREATE OR REPLACE PROCEDURE "PARITOP_TRAITERXMLRESULTMASSE" (
pPRC_ID IN PARITOP_PARC.PRC_ID%TYPE,
pRST_MONDE IN PARITOP_RESULTAT.RST_MONDE%TYPE,
pXMLDOC IN CLOB,
pTABLENAME IN VARCHAR2)
AS
Objectif :Insérer le contenu du XML passé en paramètre (pXMLDOC) à la table passée en paramètre (pTABLENAME)
La table passée en paramètre doit être une table ayant comme clé étrangère le champs "RST_ID" .
(Le noeud RST_ID est donc ajouté à tous les document XML. Ce rst_id est
déterminé à partir de la table PARITOP_RESULTAT grâce à pPRC_ID et
pRstMonde fournis en paramètre)
result_doc CLOB;
XMLDOMDOC XDB.DBMS_XMLDOM.DOMDOCUMENT;
NODE_ROWSET DBMS_XMLDOM.DOMNODE;
NODE_ROW DBMS_XMLDOM.DOMNODE;
vUE_ID PARITOP_RESULTAT.UE_ID%TYPE;
vRST_ID PARITOP_RESULTAT.RST_ID%TYPE;
nodeList DBMS_XMLDOM.DOMNODELIST;
BEGIN
BEGIN
vUE_ID := 0;
vRST_ID := 0;
XMLDOMDOC := DBMS_XMLDOM.NEWDOMDOCUMENT(pXMLDOC);
IF NOT GESTXML_PKG.FN_PARITOP_DOCUMENT_IS_NULL(XMLDOMDOC) THEN
NODE_ROWSET := DBMS_XMLDOM.item(DBMS_XMLDOM.GETCHILDNODES (DBMS_XMLDOM.MAKENODE(XMLDOMDOC)),0);
for i in 0..dbms_xmldom.getLength(DBMS_XMLDOM.getchildnodes(NODE_ROWSET))-1 loop
NODE_ROW := DBMS_XMLDOM.ITEM(DBMS_XMLDOM.GETCHILDNODES(NODE_ROWSET), i) ;
nodeList := DBMS_XMLDOM.GETELEMENTSBYTAGNAME(DBMS_XMLDOM.makeelement(NODE_ROW) , 'UE_ID');
IF vUE_ID <> DBMS_XMLDOM.GETNODEVALUE(DBMS_XMLDOM.GETFIRSTCHILD(DBMS_XMLDOM.ITEM(nodeList, 0))) THEN
vUE_ID := DBMS_XMLDOM.GETNODEVALUE(DBMS_XMLDOM.GETFIRSTCHILD(DBMS_XMLDOM.ITEM(nodeList, 0)));
--on ramasse le rst_id
SELECT RST_ID INTO vRST_ID
FROM PARITOP_RESULTAT RST
WHERE RST.PRC_ID = pPRC_ID
AND RST.UE_ID = vUE_ID
AND RST.RST_MONDE = pRST_MONDE
AND RST_A_SUPPRIMER = 0;
END IF;
GESTXML_PKG.PARITOP_ADDNODETOROW(XMLDOMDOC, NODE_ROW, 'RST_ID', vRST_ID);
end loop;
RESULT_DOC := ' '; --Ã garder, pour ne pas que ca fasse d'erreur lors du WriteToClob.
dbms_xmldom.writeToClob(DBMS_XMLDOM.MAKENODE(XMLDOMDOC), RESULT_DOC);
--Insertion du document XML dans la table "tableName"
GESTXML_PKG.PARITOP_INSERTXML(RESULT_DOC, pTABLENAME);
DBMS_XMLDOM.FREEDOCUMENT( XMLDOMDOC);
end if;
EXCEPTION
[â¦exception treatementâ¦]
END;
END;
The format of a XML clob is :
<ROWSET>
<ROW>
<PRC_ID>193</PRC_ID>
<UE_ID>8781</UE_ID>
<VEN_ID>6223</VEN_ID>
<RST_MONDE>0</RST_MONDE>
<CMP_SELMAN>0</CMP_SELMAN>
<CMP_INDICESELECTION>92.307692307692307</CMP_INDICESELECTION>
<CMP_PVRES>94900</CMP_PVRES>
<CMP_PVAJUSTE>72678.017699115066</CMP_PVAJUSTE>
<CMP_PVAJUSTEMIN>72678.017699115095</CMP_PVAJUSTEMIN>
<CMP_PVAJUSTEMAX>72678.017699115037</CMP_PVAJUSTEMAX>
<CMP_PV>148000</CMP_PV>
<CMP_VALROLE>129400</CMP_VALROLE>
<CMP_PVRESECART>4790</CMP_PVRESECART>
<CMP_PVRHAB>101778.01769911509</CMP_PVRHAB>
<CMP_UTILISE>1</CMP_UTILISE>
<CMP_TVM>1</CMP_TVM>
<CMP_PVA>148000</CMP_PVA>
</ROW>
<ROW>
<PRC_ID>193</PRC_ID>
<UE_ID>8781</UE_ID>
<VEN_ID>6235</VEN_ID>
<RST_MONDE>0</RST_MONDE>
<CMP_SELMAN>0</CMP_SELMAN>
<CMP_INDICESELECTION>76.92307692307692</CMP_INDICESELECTION>
<CMP_PVRES>117800</CMP_PVRES>
<CMP_PVAJUSTE>118080</CMP_PVAJUSTE>
<CMP_PVAJUSTEMIN>118080</CMP_PVAJUSTEMIN>
<CMP_PVAJUSTEMAX>118080</CMP_PVAJUSTEMAX>
<CMP_PV>172000</CMP_PV>
<CMP_VALROLE>134800</CMP_VALROLE>
<CMP_PVRESECART>0</CMP_PVRESECART>
<CMP_PVRHAB>147180</CMP_PVRHAB>
<CMP_UTILISE>1</CMP_UTILISE>
<CMP_TVM>1</CMP_TVM>
<CMP_PVA>172000</CMP_PVA>
</ROW>
</ROWSET>
PARITOP_COMPARABLE TABLE :
RST_ID NUMBER(10) NOT NULL,
VEN_ID NUMBER(10) NOT NULL,
CMP_SELMAN NUMBER(1) NOT NULL,
CMP_UTILISE NUMBER(1) NOT NULL,
CMP_INDICESELECTION FLOAT(53) NOT NULL,
CMP_PVRES FLOAT(53) NULL,
CMP_PVAJUSTE FLOAT(53) NULL,
CMP_PVRHAB FLOAT(53) NULL,
CMP_TVM FLOAT(53) NULL
ROCEDURE PARITOP_INSERTXML (xmlDoc IN clob, tableName IN VARCHAR2)
AS
insCtx DBMS_XMLSave.ctxType;
rowss number;
BEGIN
--permet d'insérer les champs du XML dans la table passée en paramètre.
--il suffit que les champs XML aient le même nom que les champs de la table
BEGIN
insCtx := DBMS_XMLSave.newContext(tableName); -- get context handle
DBMS_XMLSAVE.SETDATEFORMAT( insCtx, 'yyyy-MM-dd HH:mm:ss');--attention, case sensitive
DBMS_XMLSAVE.setIgnoreCase(insCtx, 1);
rowss := DBMS_XMLSAVE.INSERTXML(insCtx , xmlDoc);
DBMS_XMLSave.closeContext(insCtx);
EXCEPTION
[â¦]
END;
END;
PROCEDURE PARITOP_ADDNODETOROW (
XMLDOMDOC DBMS_XMLDOM.DOMDOCUMENT,
NODE_ROW dbms_xmldom.DOMNode,
NOM_NOEUD VARCHAR2,
VALEUR_NOEUD VARCHAR2)
AS
--PERMET D'AJOUTER UN NOEUD AVEC 1 SEULE VALEUR DANS une ROW D'UN XML.
--UTILE SURTOUT POUR LES CLÃS ÃTRANGÃRES
domElemAInserer DBMS_XMLDOM.DOMELEMENT;
NODE dbms_xmldom.DOMNode;
NODE_TMP dbms_xmldom.DOMNode;
BEGIN
domElemAInserer := DBMS_XMLDOM.createElement(XMLDOMDOC, NOM_NOEUD) ;
NODE := DBMS_XMLDOM.MAKENODE(domElemAInserer); --cast
NODE := DBMS_XMLDOM.APPENDCHILD(NODE_ROW,NODE);
NODE_TMP := DBMS_XMLDOM.MAKENODE(DBMS_XMLDOM.CREATETEXTNODE(XMLDOMDOC, VALEUR_NOEUD ) );
NODE := DBMS_XMLDOM.APPENDCHILD(NODE,NODE_TMP );
END; -
How to load Matrix report data into basic table data using ODI
Hi,
How to load Matrix report data into basic table data using oracle Data Integrator ?
Requirement Description:
Following is the matrix report data:
JOB DEPT10 DEPT20
ANALYST 6000
CLERK 1300 1900 Need to convert it into below format:
JOB Dept Salary
ANALYST DEPT10
ANALYST DEPT20 6000
CLERK DEPT10 1300
CLERK DEPT20 1900
Thanks for your help in advance. Let me know if any further explanation is required.Your list seems to be a little restrictive, you can do a lot more with ODI procedures.
If you create new procedure, and add a step. In the 'command on source' tab set you technology and schema as per your source database. Use the unpivot functionality as described in the link, please, rather than using 'SELECT *' use the appropriate column names and alias them for eg:
SELECT job as job,
deptsal as deptsal,
saldesc as saledesc
FROM pivoted_data
UNPIVOT (
deptsal --<-- unpivot_clause
FOR saldesc --<-- unpivot_for_clause
IN (d10_sal, d20_sal, d30_sal, d40_sal) --<-- unpivot_in_clause
Then in your 'command on target' tab set the technology and schema to your target db, then put your INSERT statement for eg:
INSERT INTO job_sales
(job,
deptsal,
saledesc
VALUES
:job,
:deptsal,
:saledesc
Therefore you are using bind variables from source to load data into target.
Obviously if the source and target table are in the same database, then you can have it all in one statement in the 'command on target' as
INSERT INTO job_sales
(job,
deptsal,
saledesc
SELECT job as job,
deptsal as deptsal,
saldesc as saledesc
FROM pivoted_data
UNPIVOT (
deptsal --<-- unpivot_clause
FOR saldesc --<-- unpivot_for_clause
IN (d10_sal, d20_sal, d30_sal, d40_sal) --<-- unpivot_in_clause
also set the log counter as 'Insert' on the tab where your INSERT statement is, so you know how many rows you insert into the table.
Hope this helps.
BUT remember that this feature only came out in Oracle 11g. -
Most efficient way to load XML file data into tables
I have a complex XML file running into MBs. I want to load it's data into 7-8 tables.
Which way will be better:
1) Use SQL Loader to actually load directly into the 7-8 tables directly by modifying the control card.
Is this really possible and feasible? I am not even sure about it
2) Load data as XML Type in a table and register it. Then extract from there to load into various tables.
Please help. I have to find the most efficient way of doing it.
Regards,
SudhirYes it is possible to use SQL*Loader to parse and load XML, but that is not what it was designed for and so is not recommended. You also don't need to register a schema, just to load/store/parse XML in the DB either.
So where does that leave you?
Some options
{thread:id=410714} (see page 2)
{thread:id=1090681}
{thread:id=1070213}
Those talk some about storage options and reading in XML from disk and parsing XML. They should also give you options to consider. Without knowing more about your requirements for the effort, it is difficult to give specific advice. Maybe your 7-8 tables don't exist and so using Object Relational Storage for the XML would be the best solution as you can query/update tables that Oracle creates based off the schema associated to the XML. Maybe an External Table definition works better for reading the XML into the system because this process will happen just once. Maybe using WebDAV makes more sense for loading XML to be parsed (I don't have much experience with this, just know it is possible from what I've read on the forums). Also, your version makes a difference as you have different options available depending upon the version of Oracle.
Hope all that helps as a starter.
Edited by: A_Non on Jul 8, 2010 4:31 PM
A great example, see the answers by mdrake in {thread:id=1096784} -
Loading XML Data into Relational Table
Hello,
I receive an XML file generated from another tool (on Windows), I am trying to create a Linux shell script that will gather the needed XML file from my Linux database server, then have Oracle use this file to load the XML data into a relational table. This activity & data will be needed on an ongoing basis.
I have tried this two ways. First, I loaded the XML document into the database and tried to extract the data directly from the document, but that is not working. Now I want to try to read the data directly from the file on the server via select, however I am not getting any data returned. In the Select statement below, I am simply trying to query the data to see what is returned for my testing.
Create Table ci_results_table (transactionID Varchar2(100), //transactionID should be PrimaryKey but was getting NULL value errors during insert test, so removed PK
message Varchar2(200),
ci Varchar2(50),
processeddate xmltype,
status Varchar2(50),
sourcefile VarChar2(100));
select x.*
from XMLTable(
'TSPLoadResults/Results'
PASSING xmltype(bfilename('CMDB_DEVADHOCRESULTS_DIR','LoadResults-HP_146.results.xml'), nls_charset_id('AL32UTF8'))
COLUMNS
transactionID Varchar2(100) PATH 'TransactionID',
Result XMLType PATH 'Result',
Message Varchar2(200) PATH 'Message',
PrimaryKey Varchar2(50) PATH 'PrimaryKey',
ProcessedDate date PATH 'ProcessedDate',
Status Varchar2(50) PATH 'Status',
SourceFile VarChar2(100) PATH 'SourceFileName'
) x
Eventually I will need to build on this to limit the data returned to those records where SourceFileName is like 'HPDS%' and insert what is returned in to the ci_results_table. Attached is a sample Results XML file I am trying to load, It is named "ResultsTransformedtoUnix" because I used dos2Unix to convert it to Unix which may be right or wrong. (The output file I send out has to be transformed to DOS format before the other application can read it). Original file (before Unix conversion) named in script is also attached.
Please help. Thank you!Hi,
I see some wrong things in your query.
1) The obvious one, explaining why you're not getting any data : there's a typo in the XQuery expression, it's "Result" not "Results"
2) ProcessedDate cannot be extracted as a date (at least not directly) since it actually represents a timestamp, use TIMESTAMP WITH TIME ZONE datatype and cast back to DATE in the SELECT clause
3) transactionID is an attribute, it must be accessed with '@' (or 'attribute::' axis)
4) If the file encoding is truly ISO-8859-1 as the prolog suggests, then do not use AL32UTF8 but the corresponding charset name : WE8ISO8859P1
Here's the working query :
select x.transactionID
, x.Message
, x.Primarykey
, cast(x.ProcessedDate as date) ProcessDate
, x.Status
, x.SourceFile
from XMLTable(
'/TSPLoadResults/Result'
PASSING xmltype(bfilename('XML_DIR','LoadResults-HP_146.results.xml'), nls_charset_id('WE8ISO8859P1'))
COLUMNS
transactionID Varchar2(100) PATH '@transactionID',
Message Varchar2(200) PATH 'Message',
PrimaryKey Varchar2(50) PATH 'PrimaryKey',
ProcessedDate timestamp with time zone PATH 'ProcessedDate',
Status Varchar2(50) PATH 'Status',
SourceFile VarChar2(100) PATH 'SourceFileName'
) x
Using this query directly over the file will only perform decently (for large files) on 11.2.0.4 and onwards.
On prior versions, first load the file in a (temporary) XMLType column with Binary XML storage and SELECT from there.
because I used dos2Unix to convert it to Unix which may be right or wrong.
This conversion shouldn't be necessary. -
Upload XML file data into internal table
Hi,
I need help from you.
I am uploading XML file from Application server.
1.In application server under directory it may have several XML files and i need to upload all the XML files when we gave only directory name. So, what is the function module for that logic . If possible code also.
2.Once we get the files into some internal table we need to upload data from that files to internal table.
So what is the logic for that , if possible code also.
I am giving sample directory:
For example in this directory assume 3 xml files are there.
I need to upload those files and data from that in to internal table.
Thanks & Regards
VSRBelow code will give you an idea.
TYPE-POOLS: ixml. "iXML Library Types
*TABLES : rbkp.
TYPE DECLERATIION
TYPES: BEGIN OF type_tabpo,
ebeln TYPE ekko-ebeln, "PO document number
ebelp TYPE ekpo-ebelp, "PO line item
END OF type_tabpo.
TYPES: BEGIN OF type_ekbe,
belnr TYPE rbkp-belnr, "Invoice document
gjahr TYPE rbkp-gjahr, "fiscal year
END OF type_ekbe.
TYPES: BEGIN OF type_invoice,
belnr TYPE rbkp-belnr, "PO document number
gjahr TYPE rbkp-gjahr, "Fiscal Year
rbstat TYPE rbkp-rbstat, "invoice status
END OF type_invoice.
TYPES: BEGIN OF t_xml_line, "Structure for holding XML data
data(256) TYPE x,
END OF t_xml_line.
INTERNAL TABLE DECLERATIION
DATA: gi_tabpo TYPE STANDARD TABLE OF type_tabpo,
gi_ekbe TYPE STANDARD TABLE OF type_ekbe,
gi_invoice TYPE STANDARD TABLE OF type_invoice,
gi_bapiret2 TYPE STANDARD TABLE OF bapiret2.
DATA: l_ixml TYPE REF TO if_ixml,
l_streamfactory TYPE REF TO if_ixml_stream_factory.
DATA: l_xml_table TYPE TABLE OF t_xml_line, " XML Table of the structure
l_xml_line TYPE t_xml_line, " Record of structure t_xml_line
l_xml_table_size TYPE i. " XML table size
DATA: l_filename TYPE string.
WORK AREA DECLARATION
DATA: gw_tabpo TYPE type_tabpo,
gw_ekbe TYPE type_ekbe,
gw_invoice TYPE type_invoice,
gw_bapiret2 TYPE bapiret2.
BEGIN OF SELECTION SCREEN
SELECTION-SCREEN BEGIN OF BLOCK blk1 WITH FRAME TITLE text-001.
PARAMETERS: p_file TYPE pathintern LOWER CASE DEFAULT '/usr/sap/tmp/'.
Validation of XML file: Only DTD included in XML document is supported
SELECTION-SCREEN END OF BLOCK blk1.
SELECTION SCREEN VALIDATION
AT SELECTION-SCREEN.
To validate p_file is not initial
PERFORM sub_validate_file.
PERFORM sub_validate_path.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
Request for filename for xml file from the application server
PERFORM sub_get_filename_appl USING p_file.
START OF SELECTION SCREEN
START-OF-SELECTION.
PERFORM sub_fetch_po_details.
PERFORM sub_get_invoice.
PERFORM sub_rel_invoice.
END OF SELECTION SCREEN
END-OF-SELECTION.
*& Form sub_validate_file
To Validate the file
FORM sub_validate_file .
IF p_file IS INITIAL.
MESSAGE e000. "specify the file path
ENDIF.
ENDFORM. " sub_validate_file
*& Form sub_get_filename_appl
form sub_get_filename_appl USING l_fname TYPE any.
DATA: l_fname TYPE filename-fileintern. " File name
*GET THE FILENAME FROM THE APPLICATION SERVER
CALL FUNCTION '/SAPDMC/LSM_F4_SERVER_FILE'
EXPORTING
directory = l_fname
filemask = '*'
IMPORTING
serverfile = l_fname
EXCEPTIONS
canceled_by_user = 1
OTHERS = 2.
IF sy-subrc 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
ENDFORM. " sub_get_filename_appl
*& Form sub_fetch_po_details
To fetch the PO details from the application server
Format of file is XML
FORM sub_fetch_po_details .
TYPE DECLERATIION
l_ixml = cl_ixml=>create( ).
Creating a stream factory
l_streamfactory = l_ixml->create_stream_factory( ).
PERFORM get_xml_table.
LOOP AT gi_tabpo INTO gw_tabpo.
WRITE:/ gw_tabpo.
ENDLOOP.
ENDFORM. " sub_fetch_po_details
*& Form get_xml_table
Read from the xml file
FORM get_xml_table .
Local variable declarations
DATA: l_len TYPE i,
l_len2 TYPE i,
l_tab TYPE tsfixml,
l_content TYPE string,
l_str1 TYPE string,
c_conv TYPE REF TO cl_abap_conv_in_ce,
l_itab TYPE TABLE OF string.
l_filename = p_file.
code to upload data from application server
OPEN DATASET l_filename FOR INPUT IN BINARY MODE.
IF sy-subrc 0.
WRITE:/ 'invalid file path'.
ENDIF.
DO.
READ DATASET l_filename INTO l_xml_line.
IF sy-subrc EQ 0.
APPEND l_xml_line TO l_xml_table.
ELSE.
EXIT.
ENDIF.
ENDDO.
CLOSE DATASET l_filename.
code to find the table size
DESCRIBE TABLE l_xml_table.
l_xml_table_size = ( sy-tleng ) * ( sy-tfill ).
*code to convert hexadecimal to XML
LOOP AT l_xml_table INTO l_xml_line.
c_conv = cl_abap_conv_in_ce=>create( input = l_xml_line-data
replacement
= space ).
c_conv->read( IMPORTING data = l_content len = l_len ).
CONCATENATE l_str1 l_content INTO l_str1.
ENDLOOP.
l_str1 = l_str1+0(l_xml_table_size).
SPLIT l_str1 AT cl_abap_char_utilities=>cr_lf INTO TABLE l_itab.
LOOP AT l_itab INTO l_str1.
REPLACE ALL OCCURRENCES OF cl_abap_char_utilities=>horizontal_tab IN
l_str1 WITH space.
ENDLOOP.
CALL TRANSFORMATION ('ID') " code to put in internal table
SOURCE XML l_str1
RESULT tab = gi_tabpo[].
ENDFORM. " get_xml_table -
Load XML Data into relational Table
Hi All,
I have a requirement to load data in an XML file into columns in an oracle table. This has to be done on the command line from a unix box which connects to a remote oracle DB.
Can some one give me a step by step procedure to do this (preferably using sqlloader) or point me in the right direction.
thank youHi,
Read "Oracle XML DB Developer's guide", Chapters 2,3,4,5.
That will give you very good idea.
The reason, I am suggesting reading is, there is not "one" way to do it. There can be many good ways, depending on the size, complexity of the XML and also your intended use of that XML data.
Regards -
Reading XML file from specific location&Storing xmldata into related tables
I am new to xml.
My requirement is,
1) Get the xml file from specified location (C:\xmlfiles\ xmldata.xml)
2)Convert xml data in clob data.
3)store the data into related table.
and vice-versa.
What i did,
a) I got the data from related tables and converted into xmlformat using SQLX
b)converted this resultset into clob data and stored as xml file in specific location.
It uses more then ten tables.
All this help i got from AskTom site , thanks for that.
Now i have to do vise-versa, i.e. i have to perform 1), 2), 3) steps........
Please tell me proper steps to acheive it.
Thanks in advance for giving your precious time to solve my issue.Have you read the"XMLDB FAQ" in this Forum?
You could use stuff like:
create or replace directory xmldir as C:\xmlfiles'
declare
xmldata xmltype;
begin
xmldata := xmltype(bfilename('XMLDIR','xmldata.xml'),nls_charset_id('AL32UTF8'));
-- etc, etc your code --
-- --> convert to clob by using for instance getclobval() function
-- --> then insert the data in your relational table with CLOB column
-- etc, etc your code --
end;
/ -
How to insert data into a table from an xml document
using the XmlSql Utility, how do I insert data into a table from an xml document, using the sqlplus prompt.
if i use the xmlgen.insertXML(....)
requires a CLOB file, which i dont have, only the xml doc.
Cant i insert directly from the doc to the table?
the xmlgen examples I have seen first convert a table to a CLOB xmlString and then insert it into another table.
Isnt there any other way?Your question is little perplexing.
If you're using XML SQL Utility from
the commandline, just use putXML.
java OracleXML putXML
null -
Hi,
Iam new to the xml,
can u please anyone help me how to write procedure to load the data into a table using xml as input parameter to a procedure and xml file is as shown below which is input to me.
<?xml version="1.0"?>
<DiseaseCodes>
<Entity><dcode>0</dcode><ddesc>(I87)Other disorders of veins - postphlebitic syndrome</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity>
<Entity><dcode>0</dcode><ddesc>(J04)Acute laryngitis and tracheitis</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity>
<Entity><dcode>0</dcode><ddesc>(J17*)Pneumonia in other diseases - whooping cough</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity>
</DiseaseCodes>.
Regards,
vikram.here is the your XML parse in 11g :
select *
from xmltable('//Entity' passing xmltype
'<?xml version="1.0"?>
<DiseaseCodes>
<Entity><dcode>0</dcode><ddesc>(I87)Other disorders of veins - postphlebitic syndrome</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity>
<Entity><dcode>0</dcode><ddesc>(J04)Acute laryngitis and tracheitis</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity>
<Entity><dcode>0</dcode><ddesc>(J17*)Pneumonia in other diseases - whooping cough</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity>
</DiseaseCodes>
') columns
"dcode" varchar2(4000) path '/Entity/dcode',
"ddesc" varchar2(4000) path '/Entity/ddesc',
"reauthflag" varchar2(4000) path '/Entity/reauthflag'
dcode ddesc reauthflag
0 (I87)Other disorders of veins - postphlebitic syndrome 0
0 (J04)Acute laryngitis and tracheitis 0
0 (J17*)Pneumonia in other diseases - whooping cough 0
SQL>
Using this parser you can create procedure as
SQL> create or replace procedure myXMLParse(x clob) as
2 begin
3 insert into MyXmlTable
4 select *
5 from xmltable('//Entity' passing xmltype(x) columns "dcode"
6 varchar2(4000) path '/Entity/dcode',
7 "ddesc" varchar2(4000) path '/Entity/ddesc',
8 "reauthflag" varchar2(4000) path '/Entity/reauthflag');
9 commit;
10 end;
11
12 /
Procedure created
SQL>
SQL>
SQL> exec myXMLParse('<?xml version="1.0"?><DiseaseCodes><Entity><dcode>0</dcode><ddesc>(I87)Other disorders of veins - postphlebitic syndrome</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity><Entity><dcode>0</dcode><ddesc>(J04)Acute laryngitis and tracheitis</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity><Entity><dcode>0</dcode><ddesc>(J17*)Pneumonia in other diseases - whooping cough</ddesc><claimid>34543></claimid><reauthflag>0</reauthflag></Entity></DiseaseCodes>');
PL/SQL procedure successfully completed
SQL> select * from MYXMLTABLE;
dcode ddesc reauthflag
0 (I87)Other disorders of veins - postphlebitic syndrome 0
0 (J04)Acute laryngitis and tracheitis 0
0 (J17*)Pneumonia in other diseases - whooping cough 0
SQL>
SQL>
Ramin Hashimzade -
Want to load an xml document directly into an Oracle table
From the Oracle XML Applications book pg: 106/107
there is an example where it inserts into a Oracle table from the xml_documents table. What I would like to do is insert directly from the xml document and was able to but not able to get rid of the inserting into xml_documents section.
CREATE OR REPLACE PROCEDURE insertXmlFile( dir VARCHAR2,
file VARCHAR2,
myTable varchar2,
name VARCHAR2 := NULL) IS
theBFile BFILE;
theCLob CLOB;
theDocName VARCHAR2(200) := NVL(name,file);
BEGIN
-- (1) Insert a new row into xml_documents with an empty CLOB, and
-- (2) Retrieve the empty CLOB into a variable with RETURNING..INTO
INSERT INTO xml_documents(XML_DOCUMENT_NM,XML_DOCUMENT_TX)
VALUES('DELETE ME',empty_clob())
RETURNING XML_DOCUMENT_TX INTO theCLob;
-- (3) Get a BFile handle to the external file
theBFile := BFileName(dir,file);
-- (4) Open the file
dbms_lob.fileOpen(theBFile);
-- (5) Copy the contents of the BFile into the empty CLOB
dbms_lob.loadFromFile(dest_lob => theCLob,
src_lob => theBFile,
amount => dbms_lob.getLength(theBFile));
-- (5a) Insert the rows into the proper table
insProc(theClob, myTable);
-- (6) Close the file and commit
dbms_lob.fileClose(theBFile);
delete from xml_documents
where xml_document_nm = 'DELETE ME';
COMMIT;
END;
execute insertXmlFile ('XMLFILES','emp.xml','emp')Yes I would like to insert directly from an emp.xml file and load it directly into an emp table and not have to load the document in the xml_documents table and then insert into the emp table like I am doing in this procedure.
-
How to extract data from xml and insert into Oracle table
Hi,
I have a large xml file. which will have hundreds of the following transaction tags having column names and there values.
There is a table one of the schema with coulums "actualCostRate","billRate"....etc.
I need to extract the values of these columns and insert into the table
<Transaction actualCostRate="0" billRate="0" chargeable="1" clientID="NikuUK" chargeCode="LCOCD1" externalID="L-RESCODE_UK1-PROJ_UK_CNT_GBP-37289-8" importStatus="N" projectID="TESTPROJ" resourceID="admin" transactionDate="2002-02-12" transactionType="L" units="11" taskID="5017601" inputTypeCode="SALES" groupId="123" voucherNumber="ABCVDD" transactionClass="ABCD"/>
<Transaction actualCostRate="0" billRate="0" chargeable="1" clientID="NikuEU" chargeCode="LCOCD1" externalID="L-RESCODE_US1-PROJ_EU_STD2-37291-4" importStatus="N" projectID="TESTPROJ" resourceID="admin" transactionDate="2002-02-04" transactionType="L" units="4" taskID="5017601" inputTypeCode="SALES" groupId="124" voucherNumber="EEE222" transactionClass="DEFG"/>Re: Insert from XML to relational table
http://www.google.ae/search?hl=ar&q=extract+data+from+xml+and+insert+into+Oracle+table+&btnG=%D8%A8%D8%AD%D8%AB+Google&meta= -
How to store XML data into Oracle Table
I had trouble to store XML data into Oracle Table with XDK (Oracle 8.1.7 ). The error is:
C:\XDK_Java_9_2\xdk\demo\java\Test>java testInsert Dept.xml
<Line 1, Column 1>: XML-0108: (Fatal Error) Start of root element expected.
Exception in thread "main" oracle.xml.sql.OracleXMLSQLException: Start of root element expected.
at oracle.xml.sql.dml.OracleXMLSave.saveXML(OracleXMLSave.java:2263)
at oracle.xml.sql.dml.OracleXMLSave.insertXML(OracleXMLSave.java:1333)
at testInsert.main(testInsert.java:8)
Here is my xml file:
<?xml version = '1.0'?>
<ROWSET>
<ROW num="1">
<DEPTNO>10</DEPTNO>
<DNAME>ACCOUNTING</DNAME>
<LOC>NEW YORK</LOC>
</ROW>
<ROW num="2">
<DEPTNO>20</DEPTNO>
<DNAME>RESEARCH</DNAME>
<LOC>DALLAS</LOC>
</ROW>
<ROW num="3">
<DEPTNO>30</DEPTNO>
<DNAME>SALES</DNAME>
<LOC>CHICAGO</LOC>
</ROW>
<ROW num="4">
<DEPTNO>40</DEPTNO>
<DNAME>OPERATIONS</DNAME>
<LOC>BOSTON</LOC>
</ROW>
</ROWSET>
and here is structure of table:
Name Null? Type
DEPTNO NOT NULL NUMBER(2)
DNAME VARCHAR2(14)
LOC VARCHAR2(13)
and here is my Java Code:
import java.sql.*;
import oracle.xml.sql.dml.OracleXMLSave;
public class testInsert{
public static void main(String[] args) throws SQLException{
Connection conn = getConnection();
OracleXMLSave sav = new OracleXMLSave(conn,"scott.tmp_dept");
sav.insertXML(args[0]);
sav.close();
conn.close();
private static Connection getConnection()throws SQLException{
DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
Connection conn = DriverManager.getConnection("jdbc:oracle:thin:@amt-ebdev01:1521:mydept","scott","tiger");
return conn;
Could you help me ? Thanks !The problem is that you need to pass avalid URL , Document...
Please try this code instead:
import java.net.*;
import java.sql.*;
import java.io.*;
import oracle.xml.sql.dml.OracleXMLSave;
public class testInsert
public static void main(String[] args) throws SQLException{
Connection conn = getConnection();
OracleXMLSave sav = new OracleXMLSave(conn,"scott.temp_dept");
URL url = createURL(args[0]);
sav.insertXML(url);
sav.close();
conn.close();
private static Connection getConnection()throws SQLException{
DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
Connection conn = DriverManager.getConnection("jdbc:oracle:thin:@dlsun1982:1521:jwxdk9i","scott","tiger");
return conn;
// Helper method to create a URL from a file name
static URL createURL(String fileName)
URL url = null;
try
url = new URL(fileName);
catch (MalformedURLException ex)
File f = new File(fileName);
try
String path = f.getAbsolutePath();
// This is a bunch of weird code that is required to
// make a valid URL on the Windows platform, due
// to inconsistencies in what getAbsolutePath returns.
String fs = System.getProperty("file.separator");
if (fs.length() == 1)
char sep = fs.charAt(0);
if (sep != '/')
path = path.replace(sep, '/');
if (path.charAt(0) != '/')
path = '/' + path;
path = "file://" + path;
url = new URL(path);
catch (MalformedURLException e)
System.out.println("Cannot create url for: " + fileName);
System.exit(0);
return url; -
Insert data into oracle table from XML file
I need to insert data into oracle table from XML file
If anybody handled this type of scenario, Please let me know how to insert data into oracle table from XML file
Thanks in advanceThe XML DB forum provides the best support for XML topics related to Oracle.
Here's the FAQ on that forum:
XML DB FAQ
where there are plenty of examples of shredding XML into Oracle tables and such like. ;)
Maybe you are looking for
-
How can I reach a person with Verizon Wireless
Due to Hurricane Sandy I have a bill payment problem. I called my bank this morning to see if they were open. They were. I drove to the bank to make a deposit and they had already closed so I could not make the deposit. Tomorrow is a wait and see
-
Can not open PDF-files from the internet...
Hi, I migrade my data to my new mac. But the old user xxx is now yyy. On the old imac I can download PDF-Files with the mozilla firefox. There was a copy from the pdf at the desktop! Now with the new mac mini and the new user, I get an error, because
-
After Effects Crashes when trying to replace footage
I recently got a new PC and the new computer has drive letters assigned to removeable drives that used to point to where my files were stored. When I try to replace the footage, After Effects crashes and does not give me any option to send a crash r
-
How to convert an anlog signal having frequency range 40MHZ~800MHZ, to digital
I have an analog signal with frequency range 40MHZ~800MHZ. This is to be converted to Digital Signal and will be given to LPT port of the computer. From there this will be captured and compared with a pre-stored data. Please guide me to do this.
-
In itunes it says i have 904 items but when i check setting in the ipod it says 883 songs 24 videos and 173 photos. I know the photos aren't included in the items, but im wpondering y the numbers don't match up no matter how much i update??