Load different xml docs to same database table

I have 2 different xml documents that contains the information about rows which should be loaded into a database table.
File 1:
<customer>
<name>Company1</name>
<city>Helsinki</city>
</customer>
File 2:
<client>
<clientname>Company2</clientname>
<clientcity>Helsinki</clientcity>
</client>
The data in the 2 different files should be read and inserted as a row into the same database table.
Instead of creating a java program for each xml file to insert the content into the database, I would like to first "translate" the content of the xml files and create a file for each xml file which uses the same tags, so that my java programs only reads the new "translated" files and inserts the data into the database.
This means that when I receive a new xml file with different tags but information about a row that should be inserted into the same table, I just have to "translate" this file, and use the same java program to insert the new file data into the database.
I would like to know which technology should be used to perform this task.

XSLT is quite commonly used for this.
Or, for more a formal approach, google 'xml "architectural forms"'
Pete

Similar Messages

Loading an XML file into a database table.

What is the convenient way to parse XML data present in a data file in server?

Hi;
Please check:
http://riteshkk2000.blogspot.com/2012/01/loadimport-xml-file-into-database-table .html
http://docs.oracle.com/cd/E18283_01/appdev.112/e16659/xdb03usg.htm#BABIFADB
http://www.oracle.com/technetwork/articles/quinlan-xml-095823.html
Regard
Helios

How to use Xpath to query elements loaded from different xml docs?

I have a big DBLP XML doc (95mb), to load into the container, i splitted the doc into many small xml docs by the publication types, such as articles, books, inproceedings, etc.
I tried to use xpath query such as
query 'collection()/dblp[article/author]/book[title]'
I got empty answers.
but if i use XQuery such as
query 'let $r:=collection()/dblp,$a :=$r/inproceedings/author
for $l2 in $r/book
where $a and $l2/title
return <result> {$l2}{$a}</result>'
i got answers. However, if i remove {$a} from the return clause, i still got empty answers.
is there any problem with my Xpath query and flwor Xquery?
regards,
xiaoying

Hi John,
Following is the procedure i have used to load the xml docs (after the indexes have been created):
private static void loadXmlFiles(File path2DbEnv,String theContainer, File file) throws Throwable {
     //Open a container in the db environment
          XmlManager theMgr = null;
          XmlContainer openedContainer = null;
          myDbEnv env;
          try {
          env = new myDbEnv(path2DbEnv);
     theMgr = new XmlManager(env.getEnvironment(), new XmlManagerConfig());
     //use node container
     theMgr.setDefaultContainerType(XmlContainer.NodeContainer);
               try{
               openedContainer = theMgr.createContainer(theContainer);
               }catch (XmlException e){
                    //if the container alreay exist, then open it;
                    openedContainer = theMgr.openContainer(theContainer);
          //Get an update context.
          XmlUpdateContext updateContext = theMgr.createUpdateContext();
               String theFile = file.toString();
               String docName=file.getName();
          //Get the input stream.
          XmlInputStream theStream = theMgr.createLocalFileInputStream(theFile);
          System.out.println("Adding " + theFile + " to container " +
                         theContainer);
     //     Do the actual put
          openedContainer.putDocument(docName, // The document's name
          theStream, // The actual document.
          updateContext, // The update context
     //     (required).
          null); // XmlDocumentConfig object
          System.out.println("done.");
          //XmlException extends DatabaseException, which in turn extends Exception.
          // Catching Exception catches them all.
          } catch (XmlException e){
               System.err.println("Error loading files into container " + theContainer);
               System.err.println(" Message: " + e.getMessage());
               //In the event of an error, we abort the operation
               // The database is left in the same state as it was in before
               // we started this operation.
               throw e;
          finally {
          cleanup(theMgr, openedContainer);
Unfortunately, i waited for around 20 hours 148mb doc was not loaded; so i gave up.
Could you please give me some suggestions on how to load the large doc efficiently ? thanks.
regards,
xiaoying

Performance problems loading an XML file into oracle database

Hello ODI Guru's,
I am trying to load and XML file into the database after doing simple business validations. But the interface takes hours to complete.
1. The XML files are large in size >200 Mb. We have an XSD file for the schema definition instead of a DTD.
2. We used the external database feature for loading these files in database.
The following configuration was used in the XML Data Server:
jdbc:snps:xml?f=D:\CustomerMasterData1\CustomerMasterInitialLoad1.xml&d=D:\CustomerMasterData1\CustomerMasterInitialLoad1.xsd&re=initialLoad&s=CM&db_props=oracle&ro=true
3. Now we reverse engineer the XML files and created models using ODI Designer
4. Similar thing was done for the target i.e. an Oracle database table as well.
5. Next we created a simple interface with one-to-one mapping from the XSD schema to the Oracle database table and executed the interface. This execution takes more than one hour to complete.
6. We are running ODI client on Windows XP Professional SP2.
7. The Oracle database server(Oracle 10g 10.2.0.3) for the target schema as well as the ODI master and work repositories are on the same machine.
8. I tried changing the following properties but it is not making much visible difference:
use_prepared_statements=Y
use_batch_update=Y
batch_update_size=510
commit_periodically=Y
num_inserts_before_commit=30000
I have another problem that when I set batch_update_size to value greater that 510 I get the following error:
java.sql.SQLException: class org.xml.sax.SAXException
class java.lang.ArrayIndexOutOfBoundsException said -32413
at com.sunopsis.jdbc.driver.xml.v.a(v.java)
The main concern is why should the interface taking so long to execute.
Please send suggestions to resolve the problem.
Thanks in advance,
Best Regards,
Nikunj

Approximately how many rows are you trying to insert?
One of the techniques which I found improved performance for this scenario was to extract from the xml to a flat file, then to use SQL*LOADER or external tables to load the data into Oracle.

Techinical issue loading an XML file in to database

hi,
i am trying to load an XML file into Oracle database using SSIS 2005. the XML file is huge with 4000+ columns and also has many sub catogoreys. if i try to load the file using XSD it is taking lot of space to read the entire file upto 10GB even if the data row in the file is only one. I am only loading 60 columns out of the 4000+ columns. please provide me any alternatives that you can think of.
Thanks in advance.
Deb

Stew Ashton wrote:
I think Oracle's support for loading XML data to relational tables could be improved. I haven't found a scalable solution for very large XML files that doesn't involve breaking the file up into little parts or writing / copying a good bit of code.XML DB provides two efficient ways to load XML data relationally :
- Object-Relational storage and FTP : the file is send to the XDB repository and inserted into the default XMLType table declared during schema registration. The underlying storage structures (collections, nested tables) are loaded directly while the data source is read.
Then it's just a matter of building views over the XMLType table to present the data relationally.
We can process several GBs with this method.
- Binary XML and streaming XPath evaluation : binary XML is a post-parsed format which is optimized for streaming processing, it's available since 11.1 and now the default starting with 11.2.0.2.
and you are supposed to parse the whole thing and return any errors before actually doing anything.That's the DOM view of things.
XML DB doesn't use DOM when documents are first stored as XMLType (binary or OR storage), or when it does, for example if a functional evaluation of XQuery is required, then a "lazy DOM" model is used where only the needed part(s) of the XML is (are) retrieved.

Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the rows? Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.

Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the records?
Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.

The Oracle documentation has a good overview of the options available
Generating XML Data from the Database
Without knowing your version, I just picked 11.2, so you made need to look for that chapter in the documentation for your version to find applicable information.
You can also find some information in XML DB FAQ

A query while importing an XML file into a Database Table

Hi,
I am Creating an ODI Project to import an XML file into a Database Table.With the help of the following link
http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_11g/odi_project_xml-to-table/odi_project_xml-to-table.htm
I am facing a problem while creating Physical Schema for the XML Source Model.
For the
Schema(Schema)
and
Schema(Work Schema) field they have selected GEO_D.
What is GEO_D here??
or
What I have to select here?
1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)
or
2)my target servers schema
Please tell me what I'll do??
Thanks

and
Schema(Work Schema) field they have selected GEO_D.
What is GEO_D here??This is the schema name which is specified in the XML file .
What I have to select here?
1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)Yes
2)my target servers schema
Please tell me what I'll do??
Thanks

Mix source data with different granularity into the same fact table?

I have two transaction tables "Incident (157 columns)" and "Unit (70 Colums)". For every "Incident" that happens there could be one or more records in the "Unit" table.
As part of my data mart design, I have merged both the tables into a single Fact "Incident Fact (227 Columns)" and inserted the records from both the tables with a join condition between them [incident.IN_NUM = Unit.IN_NUM].
Is this correct, is my question? or am I mixing source data with different granularity in the same fact table. Appreciate your help.
Best Regards
Bees

Bees,
Are the measures from 'Incident' , repeated for a given incident where it has more than one record in the Unit table ? If so, then the sum(indicent.measure) will give an incorrect result, no ?
What requirement is there to physically merge the tables together outside of OBIEE? With OBIEE you could have one logical 'fact' table to present to report users, which sourced from seperate Incidents and Units tables and would stop the incorrect aggregations occuring. A common modelling piece in the same way would be Order Headers and Order Lines, quite common in OBIEE to have a logical 'Orders' fact which contained both Order header measures and Order line measures, this translates to your Incidents -> Units relationship.
To do what I've mentioned, is relatively straight forward, you need a 'Dim - Incident' with two levels, Incident and Unit, mapp the unique identifiers in as the level keys and then use these levels to set the content levels correctly in your 2 logical tables sources for logical 'Fact' , ie Incidents LTS at incident level, Units LTS as units level.
Hope this helps, let us know if you get stuck.
Cheers
Alastair

How to load text file data to Oracle Database table?

By using Oracle Forms, how to load text file data to Oracle Database table?

Metalink note 33247.1 explains how to use text_io as suggested by Robin to read the file into a Multi-Row block. However, that article was written for forms 4.5 and uses CREATE_RECORD in a loop. There was another article, 91513.1 describing the more elegant method of 'querying' the file into the block by transactional triggers. Unfortunately this more recent article has disappeared without trace and Oracle deny its existence. I know it existed as I have a printed copy in front of me, and very useful it is too.

Error While Loading XMl Doc into Oracle Database 10g

Hi all,
I have a task that , I have to make a utillity by which we can load XML Doc into a Table. While searching on Internet i found following Procedure on ASK Tom
CREATE OR REPLACE
procedure insert_xml_emps(
p_directory in varchar2, p_filename in varchar2, vtableName in varchar2 )
as
v_filelocator bfile;
v_cloblocator clob;
l_ctx dbms_xmlsave.ctxType;
l_rows number;
begin
dbms_lob.createtemporary(v_cloblocator,true);
v_filelocator := bfilename(p_directory, p_filename);
dbms_lob.open(v_filelocator, dbms_lob.file_readonly);
DBMS_LOB.LOADFROMFILE(v_cloblocator, v_filelocator,
dbms_lob.getlength(v_filelocator));
l_ctx := dbms_xmlsave.newContext(vTableName);
l_rows := dbms_xmlsave.insertxml(l_ctx,v_cloblocator);
dbms_xmlsave.closeContext(l_ctx);
dbms_output.put_line(l_rows || ' rows inserted...');
dbms_lob.close(v_filelocator);
DBMS_LOB.FREETEMPORARY(v_cloblocator);
end ;
when i try to run this procedure
BEGIN
insert_xml_emps('XML_LOAD','load.xml','IBSCOLYTD');
END;
it gaves me following Error
ORA-29532: java call terminated by uncaught java exception : Oracle.xml.sql.OracleXMLSQLException:No
rows to modify-- the row enclosing tag missing. Specify the correct row enclosing tag.
ORA-06512: at "SYS.DBMS_XMLSAVE", line 115
ORA-06512: at "EXT_TEST.INSERT_XML_EMPS", line 18
ORA-06512: at line 2
Can anyone describe me this error
Thanks.
Best Regards.

SQL> /* Creating Your table */
SQL> CREATE TABLE IBSCOLYTD
2 (
3 ACTNOI VARCHAR2 (8),
4 MEMONOI NUMBER (7,0),
5 MEMODTEI DATE,
6 AMOUNTI NUMBER (8,0),
7 BRCDSI NUMBER (4,0),
8 TYPEI NUMBER (4,0),
9 TRANSMONI NUMBER (6,0)
10 );
Table created.
SQL> CREATE OR REPLACE PROCEDURE insert_xml_emps(p_directory in varchar2,
2                                              p_filename in varchar2,
3                                              vtableName in varchar2) as
4    v_filelocator    BFILE;
5    v_cloblocator    CLOB;
6    l_ctx            DBMS_XMLSTORE.CTXTYPE;
7    l_rows           NUMBER;
8    v_amount_to_load NUMBER;
9    dest_offset      NUMBER := 1;
10    src_offset       NUMBER := 1;
11    lang_context     NUMBER := DBMS_LOB.DEFAULT_LANG_CTX;
12    warning          NUMBER;
13 BEGIN
14    dbms_lob.createtemporary(v_cloblocator, true);
15    v_filelocator := bfilename(p_directory, p_filename);
16    dbms_lob.open(v_filelocator, dbms_lob.file_readonly);
17    v_amount_to_load := DBMS_LOB.getlength(v_filelocator);
18    --- ***This line is changed*** ---
19    DBMS_LOB.LOADCLOBFROMFILE(v_cloblocator,
20                              v_filelocator,
21                              v_amount_to_load,
22                              dest_offset,
23                              src_offset,
24                              0,
25                              lang_context,
26                              warning);
27
28    l_ctx := DBMS_XMLSTORE.newContext(vTableName);
29    DBMS_XMLSTORE.setRowTag(l_ctx, 'ROWSET');
30    DBMS_XMLSTORE.setRowTag(l_ctx, 'IBSCOLYTD');
31    -- clear the update settings
32    DBMS_XMLStore.clearUpdateColumnList(l_ctx);
33    -- set the columns to be updated as a list of values
34    DBMS_XMLStore.setUpdateColumn(l_ctx, 'ACTNOI');
35    DBMS_XMLStore.setUpdateColumn(l_ctx, 'MEMONOI');
36    DBMS_XMLStore.setUpdatecolumn(l_ctx, 'MEMODTEI');
37    DBMS_XMLStore.setUpdatecolumn(l_ctx, 'AMOUNTI');
38    DBMS_XMLStore.setUpdatecolumn(l_ctx, 'BRCDSI');
39    DBMS_XMLStore.setUpdatecolumn(l_ctx, 'TYPEI');
40    DBMS_XMLStore.setUpdatecolumn(l_ctx, 'TRANSMONI');
41    -- Now insert the doc.
42    l_rows := DBMS_XMLSTORE.insertxml(l_ctx, v_cloblocator);
43    DBMS_XMLSTORE.closeContext(l_ctx);
44    dbms_output.put_line(l_rows || ' rows inserted...');
45    dbms_lob.close(v_filelocator);
46    DBMS_LOB.FREETEMPORARY(v_cloblocator);
47 END;
48 /
Procedure created.
SQL> BEGIN
2 insert_xml_emps('TEST_DIR','load.xml','IBSCOLYTD');
3 END;
4 /
PL/SQL procedure successfully completed.
SQL> SELECT * FROM ibscolytd;
ACTNOI      MEMONOI MEMODTEI     AMOUNTI     BRCDSI      TYPEI TRANSMONI
28004125     251942 05-SEP-92        400        513          1          0
28004125     251943 04-OCT-92        400        513          1          0
SQL>

SQL Loader problem while loading records from txt file to database table.

I am getting following error while loading records from flat txt file into database table with the help of 'sqlldr' command. I have executed catldr.sql from RDBMS folder but it is still showing same error. I am setting DIRECT = TRUE while issuing sqlldr command. If I try with DIRECT = FALSE then it works fine. Database is Oracle 8i.
SQL*Loader-951: Error calling once/load initialization
ORA-24329: invalid character set identifier
F1 Please.

Hello,
Direct path load, can only be used with SQL*Loader and Database have the same version.
Care to tell the database version and sql*loader version you are using.
-Sri

Problem Loading Large XML Doc

I'm running 10.2.0.3 on a Linux box and I'm having problems loading a large XML document (about 100 MB). In the past, I would simply load the XML file into a XMLType column like this:
INSERT INTO foo VALUES (XMLType(bfilename('XMLDIR', 'test.xml'). nls_charset_id('AL32UTF8')));
But when I try this with a large file, it runs for 10 minutes and then returns an ORA-03113. I'm assuming the file is just too large for this technique. I spoke to Mark Drake when I was at OpenWorld and he suggested I use Oracle XML DB, so I created and registered a schema and tried using sqlldr to load the doc, but it ran for 2 1/2 hours before returning:
Parse Error on row 1 in table FOO
OCI-31038: Invalid integer value: "129"
I tried simplifying both the XML file and schema to just the following:
<schedules>
<s s="2009-09-21T04:00:00" d="21600" p="335975" c="19672"/>
<s s="2009-09-21T04:00:00" d="21600" p="335975" c="15387"/>
<s s="2009-09-21T04:00:00" d="25200" p="335975" c="5256"/>
<s s="2009-09-21T04:00:00" d="86400" p="335975" c="26198">
<k id="5" v="2009-09-21 09:00:00.000"/>
<k id="6" v="2009-09-22 03:59:59.000"/>
<k id="26" v="0.00"/><k id="27" v="US"/>
</s>
<s s="2009-09-21T04:00:00" d="21600" p="335975" c="11678"/>
<s s="2009-09-21T04:00:00" d="21600" p="335975" c="26697"/>
<s s="2009-09-21T04:00:00" d="21600" p="335975" c="25343"/>
<s s="2009-09-21T04:00:00" d="21600" p="335975" c="25269"/>
<s s="2009-09-21T04:00:00" d="86400" p="335975" c="26200">
<k id="5" v="2009-09-21 09:00:00.000"/>
<k id="6" v="2009-09-22 03:59:59.000"/>
<k id="26" v="0.00"/><k id="27" v="US"/>
</s>
</schedules>
<?xml version="1.0" encoding="UTF-8"?>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
     xmlns:xdb="http://xmlns.oracle.com/xdb"
    version="1.0"
    xdb:storeVarrayAsTable="true">
     <xs:element name="schedules" xdb:defaultTable="SCHEDULES">
          <xs:complexType xdb:SQLType="SCHEDULES_T">
               <xs:sequence>
                    <xs:element ref="s" maxOccurs="unbounded"/>
               </xs:sequence>
          </xs:complexType>
     </xs:element>
     <xs:element name="s">
          <xs:complexType>
               <xs:choice minOccurs="0">
                    <xs:element ref="f" maxOccurs="unbounded"/>
                    <xs:element ref="k" maxOccurs="unbounded"/>
               </xs:choice>
               <xs:attribute name="s" use="required" type="xs:dateTime"/>
               <xs:attribute name="p" use="required" type="xs:int"/>
               <xs:attribute name="d" use="required" type="xs:int"/>
               <xs:attribute name="c" use="required" type="xs:short"/>
          </xs:complexType>
     </xs:element>
     <xs:element name="k">
          <xs:complexType>
               <xs:attribute name="v" use="required" type="xs:string"/>
               <xs:attribute name="id" use="required" type="xs:byte"/>
          </xs:complexType>
     </xs:element>
     <xs:element name="f">
          <xs:complexType>
               <xs:attribute name="id" use="required" type="xs:byte"/>
          </xs:complexType>
     </xs:element>
</xs:schema>
Keep in mind both the actual XML file and corresponding XSD is much more complex, but this particular section is about 70 MB so I wanted to see if I could just load that. I used the following sqlldr script:
LOAD DATA
INFILE *
INTO TABLE schedules_tmp TRUNCATE
XMLType(xml_doc)(
     lobfn FILLER CHAR TERMINATED by ',',
     xml_doc LOBFILE(lobfn) TERMINATED BY EOF
BEGINDATA
/tmp/schedules.xmlThis worked fine on a small doc - loaded correctly and I could query it fine - but when I tried using the 70 MB file it ran for a couple of hours before dying with a memory problem.
So what am I doing wrong? Is there a better way to load a large file?
Thanks for the help.
Pete
Edited by: mdrake on Nov 9, 2009 8:46 PM

Mark:
Answers to your questions:
Do you use direct load ? -> Yes. I tried again using UNRECOVERABLE LOAD DATA to try to speed up performance, but it still ran for a couple of hours before dying.
Which DB Release are you working with ? -> 10.2.0.3
Can you see if you can upload via FTP ? -> I added noNamespaceSchemaLocation to my XML file and ftp'd it to my XML directory, but it wasn't recognized. Is there something else I have to do?
The change for unsignedInt should have code rid of the issue with the value 129. Did it ? -> I didn't try it again on the whole XML file (I'm just working with the schedules section), so I haven't verified this.
I'm still stumped as to why sqlldr takes so long. I could write something to parse the XML file into a flat file and then use sqlldr to load it into a relational table, and the load would only take a few minutes. But then I wouldn't be using XML DB which I thought would be faster. What am I doing wrong?
Pete

Data Loading into BW ( from excel to database table)

Hi Guys
I have an excel sheet which has the fields:
--> Soldto(3.1), Soldto(4.6), Shipto, Billto, Payer, SalesOrg, AAG.
I also have a database table /BIC/AZSTXREF00 in BW QA (SE16), which has fields:
--> /BIC/ZSOLD_TO(3.1), /BIC/ZBTOPRTY, PAYER, SHIP_TO, SOLD_TO(4.6), BILLTOPRTY, ACCNT_ASG, RECORDMODE.
The fields are in this order in the respective places.
Please note that the fields are not in the same order in the two places. But they are equal in number.
How can I delete the previous data in the database table in BW, and then load the fresh data from my excel sheet.
Thanks.

Hi Chintai,
/BIC/AZSTXREF00 seems to be the Active Table of ZSTXREF ODS. If this is the situation you can delete all ODS content from RSA1 --> right mouse click on ODS --> Delete Data.
For the upload: right mouse click on ODS --> Show Data Flow --> Take the name of the InfoSource for this ODS --> go in InfoSources tree --> enter in Transfer Roules --> Modify mapping beetween fields of MS-Excel file and you Transfer Structure --> Activate --> Upload.
Ciao.
Riccardo.

How to load a XML file into the database

Hi,
I've always only loaded data into the database by using SQL-Loader and the data format was Excel or ASCII
Now I have to load a XML.
How can I do?
The company where I work has Oracle vers. 8i (don't laugh, please)
Thanks in advance!

Hi,
Tough job especially if the XML data is complex. The have been some similar question in the forum:
Using SQL Loader to load an XML File -- use 1 field's data for many records
SQL Loader to upload XML file
Hope they help.
Regards,
Sujoy

Load different xml structure in single flash file

Hi to all,
How can i call different xml structures in single file..
actually i am looking for 2 different menu galleries in single
file.. first menu contains.. some 4 to 5 buttons.. done with this..
in other flash file..having some of 4-5 buttons.. and these
buttons.. to show in main flash file.. can i load 2 different swf
files in single main file and with different xml structure please
help me on this.. thanks in advance.
Best Regards
Satish Kumar Rajula

yes, you can use one xml instance with an if-statement, but
it's easier and cleaner to use different xml instances and
different parsing code for each xml file loaded that has different
structure.

Load different xml docs to same database table

Similar Messages

Maybe you are looking for