Best way to extract XML data to DB

Hello,
In our work we need to extract data from XML documents into a database. Here are some extra notes:
+ There is no constant schema for the XML documents.
+ Some processing on the XML files is required.
+ The XML documents are very big.
+ The database is Oracle 8i or 9i (Enterprise Edition both) and our framework is NET.
Our questions are:
1. What is the best way to extract the data into the database under the above circumstances ?
2. In case there was a constant schema for the XML documents, would there be a better way ?
3. Is writing the data to text files first, and then loading it via SQL-Loader is an effective way ?
Any thoughts would be welcome. Thanks.

Hi Nicolas,
The answer depends on your actual storage method (binary, OR, CLOB?), and db version.
You can try XMLTable, it might be better in this case :
SELECT x.elem1, x.elem2, ... , x.elem20
FROM your_table t
   , XMLTable(
      '/an/xpath/to/extract'
      passing t.myxmltype
      columns elem1  varchar2(30) path 'elem1'
            , elem2  varchar2(30) path 'elem2'
            , elem20 varchar2(30) path 'elem20'
     ) x
;

Similar Messages

  • Best way to extract XML value wiith an xpath

    Hello,
    I wonder what is the best way to extract text value from XmlType with an xpath.
    I need to insert a row inside a table where the row's data come from xpath extractions of an XmlType. I do a lot of (approximative 20) :
    EXTRACTVALUE(var.myxmltype , '/an/xpath/to/extract/elem1').
    EXTRACTVALUE(var.myxmltype , '/an/xpath/to/extract/elemI').
    EXTRACTVALUE(var.myxmltype , '/an/xpath/to/extract/elem20').
    inside the insert statement
    Is this way is the best or is there a more optimal way ?
    For example extracting the node '/an/xpath/to/extract/' and sarting from this node extracting "elem1", ... , "elemI", "elemN" children.
    Thanks for your help,
    Regards,
    Nicolas

    Hi Nicolas,
    The answer depends on your actual storage method (binary, OR, CLOB?), and db version.
    You can try XMLTable, it might be better in this case :
    SELECT x.elem1, x.elem2, ... , x.elem20
    FROM your_table t
       , XMLTable(
          '/an/xpath/to/extract'
          passing t.myxmltype
          columns elem1  varchar2(30) path 'elem1'
                , elem2  varchar2(30) path 'elem2'
                , elem20 varchar2(30) path 'elem20'
         ) x
    ;

  • Best way to handle XML data

    I was hoping someone could tell me if I'm doing something stupid or a much harder way then needed.
    I have some data that is stored in an xml file. Probably 150 items (will stay around this number) with less then a dozen attributes per item.
    The name of each of these will be loaded into a list on a jsp. When clicking on an item the details will be displayed on the page below it.
    Right now my plan is to:
    !. read the file in on application startup and store the items in a collection of beans in the session, or maybe at application scope.
    2. Then in the jsp use AJAX and pass the items ID to a servlet.
    3. The servlet pulls the collection out of session or application scope and find the correct item and sends it back as xml.
    4. Then either parse the xml or use JSON to convert it to an object.
    5. Populate the fields on the jsp.
    I'm just worried that I'm doing something incredibly inefficient or stupid.
    With only 150 items would I be better served just loading all the items into a multi dimensional javascript array on loading the page and just get rid of AJAX all together?

    Comments are shown below preceeded by *******:
    I was hoping someone could tell me if I'm doing something stupid or a much harder way then needed.
    **** Yes, I think its much harder than it needs to be.
    I have some data that is stored in an xml file. Probably 150 items (will stay around this number) with less then a dozen attributes per item.
    ****** Normally, such data is stored in a database table (no xml involved). However, you can store them in a file if you want.
    ****** If you store it in a file, I suggest not using xml. Its an advanced topic. I suggest using a tab delimited file (you can use a text editor to create it).
    The name of each of these will be loaded into a list on a jsp. When clicking on an item the details will be displayed on the page below it.
    ***** ok, so far
    Right now my plan is to:
    !. read the file in on application startup and store the items in a collection of beans in the session, or maybe at application scope.
    ****** Your options are:
    ***** request scope: need to read the file each time the user needs the data. Not efficient.
    ****** session scope: data lasts as long as user is logged on. Good idea.
    ****** application scope: data lasts after the user logs off. Bad idea. takes up memory permidently. Must remove it from application scope when last user logs off.
    ****** Store it in a database: best idea, but takes time to learn.
    2. Then in the jsp use AJAX and pass the items ID to a servlet.
    ****** Dont use AJAX, its an advanced topic. Learn JSP and XHTML first.
    3. The servlet pulls the collection out of session or application scope and find the correct item and sends it back as xml.
    ****** send it back via request.setAttribute() and read it on the JSP via <useBean>. xml is not the way to go.
    4. Then either parse the xml or use JSON to convert it to an object.
    ****** dont use xml.
    5. Populate the fields on the jsp.
    ***** Yes. Note JSP should not contain business logic. Business logic should be done in a servlet. The servlet should put data in request scope
    for the JSP to populate itself. The data objects put in request scope should contain just data, no business logic functionality.
    I'm just worried that I'm doing something incredibly inefficient or stupid.
    ***** Best way to learn is to try stuff out and make your next project better than the previous.
    With only 150 items would I be better served just loading all the items into a multi dimensional javascript array on loading the page and just get rid of AJAX all together?
    ******** Javascript is an advanced topic. Its main purpose on the JSP is to handle simple onclick events from html tags and to do some basic client side validation.
    ***** I believe a book on JSP would be very helpful.

  • Whats the best way to extract hex data from control code a Intel format

    Does anyone know an easy way to extract the green set of data from this EEPROM dump?
    :020000040000FA
    :10420000110022003300440055006600770088004A
    :104210009900AA00BB00CC00EE00FF00FF00FF00E9
    :10422000FF00FF00FF00FF00FF00FF00FF00FF0096
    :10423000FF00FF00FF00FF00FF00FF00FF00FF0086
    :10424000FF00FF00FF00FF00FF00FF00FF00FF0076
    :10425000FF00FF00FF00FF00FF00FF00FF00FF0066
    :10426000FF00FF00FF00FF00FF00FF00FF00FF0056
    :10427000FF00FF00FF00FF00FF00FF00FF00FF0046
    :10428000FF00FF00FF00FF00FF00FF00FF00FF0036
    :10429000FF00FF00FF00FF00FF00FF00FF00FF0026
    :1042A000FF00FF00FF00FF00FF00FF00FF00FF0016
    :1042B000FF00FF00FF00FF00FF00FF00FF00FF0006
    :1042C000FF00FF00FF00FF00FF00FF00FF00FF00F6
    :1042D000FF00FF00FF00FF00FF00FF00FF00FF00E6
    :1042E000FF00FF00FF00FF00FF00FF00FF00FF00D6
     :10430000FF00FF00FF00FF00FF00FF00FF00FF00B5
    :10431000FF00FF00FF00FF00FF00FF00FF00FF00A5
    :10432000FF00FF00FF00FF00FF00FF00FF00FF0095
    :10433000FF00FF00FF00FF00FF00FF00FF00FF0085
     :10434000FF00FF00FF00FF00FF00FF00FF00FF0075
     :10435000FF00FF00FF00FF00FF00FF00FF00FF0065
    :10436000FF00FF00FF00FF00FF00FF00FF00FF0055
    :10437000FF00FF00FF00FF00FF00FF00FF00FF0045
    :10438000FF00FF00FF00FF00FF00FF00FF00FF0035
    :10439000FF00FF00FF00FF00FF00FF00FF00FF0025
    :1043A000FF00FF00FF00FF00FF00FF00FF00FF0015
     :1043B000FF00FF00FF00FF00FF00FF00FF00FF0005
    :1043C000FF00FF00FF00FF00FF00FF00FF00FF00F5
     :1043D000FF00FF00FF00FF00FF00FF00FF00FF00E5
     :1043E000FF00FF00FF00FF00FF00FF00FF00FF00D5
     :1043F000FF00FF00FF00FF00FF00FF00FF00FF00C5
     :02401200FF3F6E
     :00000001FF

    rolfk wrote:
    crossrulz,
    Intel Hex Format is a text format describing binary data, usually used for chip programming devices. Besides records for the actual data itself it also allows for addressing records, such that you only need to have the data in the file that needs to be written to the chip, even if the various areas are all over the address range of the chip. Since it is a well known format for hardware developers, some people coming here assume that everybody knows what it is. Of course many true softies may never have heard of it.
    rolfk, thanks for the information.  I was not aware of this.  I guess I am a true softie even with my EE background.
    There are only two ways to tell somebody thanks: Kudos and Marked Solutions
    Unofficial Forum Rules and Guidelines

  • What is the best way to extract/consume HR data to and from Peoplesoft?

    I have to do integration between Peoplesoft and Oracle ERP to send HR data from Peoplesoft to Oracle HRMS. I will use Oracle Fusion Middleware for building the interface. I don't have much experience in working with Peoplesoft tools, like PeopleTool, App Engine, Integration Broker, Component Interface and Oracle adapter for Peoplesoft. Can somebody please let me know what is the most popular or recommended or best way to extract HR data from Peoplesoft.
    The second interface will consume some HR data from Oracle. Once again I need to know different ways to insert data into Peoplesoft.
    I appreciate if you can share your experience for the above two scenarios.

    If you plan on buying a apple laptop you have two options. You can use time machine or pull the data off of it directly and put it onto the external. I advise the second choice. This way you do not have stuff that you dont need filling up the space on your new drive. Get your applications, music, documents, photos, videos and perhaps downloads and desktop; Put all of these on your external. And then reinstall osx onto your drive.
    *** If buying a windows machine next check the format of the drive in disk utility. You will need a program called Paragon NTFS to format the drive window compatibility. You have to pay for the free version but can download a trail version for free.
    http://www.paragon-software.com/home/ntfs-mac/
    Did i leave anything unanswered?

  • Best way to extract data from archived cube

    Hello Experts,
    Can anyone tell me best way to extract data from archived cube.
    Basically I am trying to pull all the data from archived cube and then load it into another brand new infoprovider which is in different box.
    Also I need to extract all the master data for all infoobjects.
    I have two options in my mind:
    1) Use open hub destination
    or
    2) Infoprovider>display data>select the fields and download the data.
    Is it really possible to extract data using option (2) if records are too high and then load it into another infoprovider in new system.
    Please suggest me the pros and cons for the two options.
    Thanks for your time in advance.

    Hello Reddy,
    Thanks a lot for your quick reply.
    Actually in my case I am trying to extract archived infocube data and then load it into new infoprovider which is in different system. If I have connectivity I can simply export data source from archived infocube and then reload into new infoprovider.
    But there is no connectivity between those two systems (where archived cube is and new infoprovider) and so I am left with the two options I mentioned.
    1) Use Open Hub
    or
    2) Extract data manually from infoprovider into excel.
    Can anyone let me know which of the two options is the best and also I doubt on how to use excel in extracting data as excel have limit of no.of records 65536
    Thanks
    Edited by: saptrain on Mar 12, 2010 6:13 AM

  • SD reporting- best way to extract SD contracts data

    Hi All,
    I want to know what is the best way of extracting the SD contracts data from the VEDA table & the technical objects attached to the contracts(which are in tables like SER02, OBJK etc..). Is it possible to enhance the 2LIS_11_VAITM datasource with these fields? or there is a better way of doing this? Is there a standard datasource that can be used?
    Any helpful information will be appreciated.
    Thanks,
    Shilpa

    You can create a User Exit in CMOD for the 2LIS_11_VAITM DataSource to get the data. There should be a 1:1 (equi or inner join) or 0:1 (outer join) relationship of the data that you want to include, so that you don't create duplicate data.
    Another option would be to create generic DataSources and join the data in BW.

  • What is the best way to export the data out of BW into a flat file on the S

    Hi All,
    We are BW 7.01 (EHP 1, Service Pack Level 7).
    As part of our BW project scope for our current release, we will be developing certain reports in BW, and for certain reports, the existing legacy reporting system based out of MS Access and the old version of Business Objects Release 2 would be used, with the needed data supplied from the BW system.
    What is the best way to export the data out of BW into a flat file on the Server on regular intervals using a process chain?
    Thanks in advance,
    - Shashi

    Hello Shashi,
    some comments:
    1) An "open hub license" is required for all processes that extract data from BW to a non-SAP system (including APD). Please check with your SAP Account Executive for details.
    2) The limitation of 16 key fields is only valid when using open hub for extracting to a DB table. There's no such limitation when writing files.
    3) Open hub is the recommended solution since it's the easiest to implement, no programming is required, and you don't have to worry much about scaling with higher data volumes (APD and CRM BAPI are quite different in all of these aspects).
    For completeness, here's the most recent documentation which also lists other options:
    http://help.sap.com/saphelp_nw73/helpdata/en/0a/0212b4335542a5ae2ecf9a51fbfc96/frameset.htm
    Regards,
    Marc
    SAP Customer Solution Adoption (CSA)

  • Best way to "page" through data?

    If I have a XM, something like
    <people>
    <person>
    <firstname>john</firstname>
    <lastname>smith</lastname>
    </person>
    <person>
    <firstname>robert</firstname>
    <lastname>walker</lastname>
    </person>
    </people>
    And I create some bound fields to display this, what's the best way to "page" the data?
    I know I can create an XML variable of the data and then access person[0].firstname, but I need to have a button or other ui element that would choose person 0,1,2,3 etc.
    Normally I's use a list, but this application calls for a static screen where only the data changes when buttons are chosen.

    I use this method(this is simplified) as it doesn't require any sort of data services, you can have change events etc for editing/deleteing. the slider is just for poc, a prev/next button could just as easily do.
    <?xml version="1.0" encoding="utf-8"?>
    <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
    xmlns:s="library://ns.adobe.com/flex/spark"
    xmlns:mx="library://ns.adobe.com/flex/halo" minWidth="800" minHeight="400" creationComplete="initApp()" width="800" height="400">
    <fx:Declarations>
    <!-- Place non-visual elements (e.g., services, value objects) here -->
    </fx:Declarations>
    <fx:Script>
    <![CDATA[
    import mx.collections.ArrayCollection;
    [Bindable] private var Arr:ArrayCollection=new ArrayCollection();
    private var Sel:int = 0;
    protected function initApp(): void
    Arr.addItem({id:1,name:"John",surname:"Robertson",age:30});
    Arr.addItem({id:2,name:"Peter",surname:"Williams",age:35});
    Arr.addItem({id:3,name:"Jane",surname:"Brown",age:23});
    Arr.addItem({id:4,name:"Rebecca",surname:"Smith",age:42});
    Arr.addItem({id:5,name:"Susan",surname:"Reynolds",age:25});
    Arr.addItem({id:6,name:"Michael",surname:"Royce",age:23});
    Arr.addItem({id:7,name:"Jack",surname:"Jones",age:22});
    Arr.addItem({id:8,name:"Pete",surname:"Young",age:50});
    Arr.addItem({id:9,name:"Robert",surname:"Peters",age:39});
    hs.maximum = Arr.length-1;
    hs.value = 0;
    updateList();
    protected function updateList():void
    Sel = hs.value;
    surname.text= Arr[Sel].surname;
    firstname.text= Arr[Sel].name;
    age.text= Arr[Sel].age;
    ]]>
    </fx:Script>
    <s:Group width="497" height="114" horizontalCenter="0" verticalCenter="0">
    <s:TextInput id="surname" x="46" y="45"/>
    <s:TextInput id="firstname" x="182" y="45"/>
    <s:TextInput id="age" x="318" y="45"/>
    <s:HSlider id="hs" x="62" y="96" width="200" minimum="0" maximum="1" liveDragging="true" change="updateList()" value="-1"/>
      </s:Group>
    </s:Application>
    David

  • Best way to extract this info?

    Hello:
    I have information stored in xmltype as follows:
    <INFO>
    <ASG>
    <LMT id="A1">
    <LMT_VAL>7</LMT_VAL>
    </LMT>
    <LMT id="B1">
    <LMT_VAL>6</LMT_VAL>
    </LMT>
    </ASG>
    </INFO>
    The above info may be dynamic. Example, for some types, There may be additional LMT element with a value of C1 and D1. So what is the best way to extract all of them (A1, B1, C1, D1) - even though in some cases only A1 and B1 may be present?
    Do I have to extract like:
    select ...
    where
    t1.xml.col.extract('/INFO/ASG/LMT/LMT_VAL/@id').getStringVal() = 'A1'
    Do I have a condition like this where ... = A1
    Any help appreciated.

    Hi,
    one left round bracket is missing
    TABLE(xmlsequence(extract (<b> (</b><---missing SELECT xml_column FROM your_table)....If you have more then one row in the table it's not working!
    SQL> CREATE TABLE test_xml(
      2   id NUMBER
      3  ,xml_col XMLType)
      4  /
    Table created.
    SQL> INSERT INTO test_xml VALUES(1,xmltype('<INFO>
      2  <ASG>
      3  <LMT id="A1">
      4  <LMT_VAL>7</LMT_VAL>
      5  </LMT>
      6  <LMT id="B1">
      7  <LMT_VAL>6</LMT_VAL>
      8  </LMT>
      9  </ASG>
    10  </INFO>'))
    11  /
    1 row created.
    SQL> INSERT INTO test_xml VALUES(2,xmltype('<INFO>
      2  <ASG>
      3  <LMT id="A1">
      4  <LMT_VAL>7</LMT_VAL>
      5  </LMT>
      6  <LMT id="B1">
      7  <LMT_VAL>6</LMT_VAL>
      8  </LMT>
      9  </ASG>
    10  </INFO>'))
    11  /
    1 row created.
    SQL> SELECT extractvalue(column_value, '/LMT/@id') new_id,
      2         extractvalue(column_value,'/LMT/LMT_VAL') new_value
      3  FROM TABLE(xmlsequence(extract ((SELECT xml_col FROM test_xml), '/INFO/ASG/LMT')));
    FROM TABLE(xmlsequence(extract ((SELECT xml_col FROM test_xml), '/INFO/ASG/LMT')))
    ERROR at line 3:
    ORA-01427: single-row subquery returns more than one row
    SQL> spool off;

  • I am moving from PC to Mac.  My PC has two internal drives and I have a 3Tb external.  What is best way to move the data from the internal drives to Mac and the best way to make the external drive read write without losing data

    I am moving from PC to Mac.  My PC has two internal drives and I have a 3Tb external.  What is best way to move the data from the internal drives to Mac and the best way to make the external drive read write without losing data

    Paragon even has non-destriuctive conversion utility if you do want to change drive.
    Hard to imagine using 3TB that isn't NTFS. Mac uses GPT for default partition type as well as HFS+
    www.paragon-software.com
    Some general Apple Help www.apple.com/support/
    Also,
    Mac OS X Help
    http://www.apple.com/support/macbasics/
    Isolating Issues in Mac OS
    http://support.apple.com/kb/TS1388
    https://www.apple.com/support/osx/
    https://www.apple.com/support/quickassist/
    http://www.apple.com/support/mac101/help/
    http://www.apple.com/support/mac101/tour/
    Get Help with your Product
    http://docs.info.apple.com/article.html?artnum=304725
    Apple Mac App Store
    https://discussions.apple.com/community/mac_app_store/using_mac_apple_store
    How to Buy Mac OS X Mountain Lion/Lion
    http://www.apple.com/osx/how-to-upgrade/
    TimeMachine 101
    https://support.apple.com/kb/HT1427
    http://www.apple.com/support/timemachine
    Mac OS X Community
    https://discussions.apple.com/community/mac_os

  • What is the best way to mimic the data from production to other server?

    Hi,
    here we user streams and advanced replication to send the data for 90% of tables from production to another production database server. if one goes down can use another one. is there any other best option rather using the streams and replication? we are having lot of problems with streams these days they keep break and get calls.
    I heard about data guard but dont know what is use of it? please advice the best way to replicate the data.
    Thanks a lot.....

    RAC, Data Guard. The first one is active-active, that is, you have two or more nodes accessing the same database on shared storage and you get both HA and load balancing. The second is active-passive (unless you're on 11.2 with Active Standby or Snapshot Standby), that is one database is primary and the other is standby, which you normally cannot query or modify, but to which you can quickly switch in case primary fails. There's also Logical Standby - it's based on Streams and generally looks like what you seem to be using now (sort of.) But it definitely has issues. You can also take a look at GoldenGate or SharePlex.

  • Iphone 3GS was dropped and will not turn on or respond to USB connection.  Is there any way to extract the data off when it is not responding so that I can restore to my new phone?

    The phone is physically intact, but you can see the LCD is cracked behond the glass. I have tried plugging it in, tried putting it on the charger, and resetting it using the off and home buttons, bit response. I don't need the phone to work, but I do need to be bale to back it up in order to restore to the new phone.  Is there any way to extract the data or back up the hard drive without repairing the phone?  Just curious what my options are before I spend cash trying to fix the phone and find out there was another way to get the data.  Any thoughts here or should a good repair shop being able to create a backup somehow?

    Im not going to be much help because I don't know of anything else to besides take it somewhere that fixes electronics and see if they can get it off some how maybe if you take it to an apple store they can do something to get your information.

  • Best Way to port the data from one DB to another DB using Biztalk

    Hi,
    please suggest best way to move the data from one db to another DB using biztalk.
    Currently I am doing like that, for each transaction(getting from different source tables) through receive port, and do some mapping (some custom logic for data mapping), then insert to target normalized tables(multiple tables) and back to update the status
    of transaction in source table in sourceDB. It is processing one by one.
    How/best we we can do it using  bulk transfer and update the status. Since it has more than 10000 transaction per call.
    Thanks,
    Vinoth

    Hi Vinoth,
    For SQL Bulk inserts you can always use SQL Bulk Load
    adapter.
    http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalksyn/archive/2005/10/23/processing-a-large-flat-file-message-with-biztalk-and-the-sqlbulkinsert-adapter.aspx
    However, even though a SQL Bulk Load adapter can efficiently insert a large amount of data into SQL you are still stuck with the issues of transmitting the
    MessageBox database and the memory issues of dealing with really large messages.
    I would personally suggest you to use SSIS, as you have mentioned that records have to be processed in specific time of day as opposed to when the
    records are available.
    Please refer to this link to get more information about SSIS: http://msdn.microsoft.com/en-us/library/ms141026.aspx
    If you have any more questions related to SSIS, please ask it in
    SSIS 
    forum and you will get specific support.
    Rachit

  • I am giving away a computer, what is the best way to wipe out data prior to depature

    i am giving away a computer, what is the best way to wipe out data prior to depature

    Did the Mac come with two grey disks when new? If so, use disk one to erase the drive using Disk Utility and then re-install the OS from the same disk. Once installed, quit and when the new owner boots they can set it up as a new out-of-the-box Mac when they boot it up. The grey disks need to be passed on with the computer.
    If you need detailed instructions on how to erase and re-install please post back.
    If the Mac came with Lion or Mountain Lion installed the above process can be done using the Recovery HD as since Lion no restore disks are supplied with the Mac.
    The terms of the licence state that a Mac should be sold/passed on with the OS installed that was on the machine when new (or words to that effect).

Maybe you are looking for

  • Albums and Albums Categories

    I have Photoshop 8.  I am attempting to create album categories but can not seem to do it.  In trying to create them , I click on create albums and then create album categories.  I enter my new album category name,  I never get a choice of anything t

  • My files were deleted by cc from the cloud and from my mac

    after trying to sync a pc to my creative cloud folder, almost all of my files were deleted from the cloud, and I cant acces the archive folder on the web.. i needed to restore all the files from my trash manually and now I hope that all the files are

  • MV 11gPr1 - variable pie chart issue

    Hi, Trying out advanced style - variable pie chart - we faced some issue - if data source contains bigger numbers (e.g. 100000,200000....) something causes that radius of all pies is the same. with smaller values (10,20...) the pie radius is variable

  • Program paths under linux

    Hallo, I need in my application to launch specific programs (common ones like firefox, adobe etc..). I could find the following posting, which gives the path of the installed software by passing the software name (like firefox.exe), under windows. My

  • 6220 classic slow message writing problem - need h...

    hi everyone, i updated my phone's version from 4 to 5.15, then i reiliezed that i was writing the message very slowly, for example; you click the start button on your computer and it opens the menu after 10 second when your computer is very busy, i a