Querying wide (multi-record case) table data in oracle 10g

I've got a set of "wide" (multi-record case data format) tables similar to the format described in Oracle's "Oracle Data Mining Concepts" book in chapter 2 (http://download-west.oracle.com/docs/cd/B14117_01/datamine.101/b10698/2data.htm#1010394).
I like the flexibility of this format, especially since it can handle thousands of attributes. But the main problem I have with this format is in determining how to write efficient, complex queries against the data.
I've included some sample data below to demonstrate my problem. I've got two tables, one with people and the other with the attributes and values of age, income, height and weight. If I try to generate a complex query against this data, I seem to either have to rely on multisets and functions, or on a large number of self joins. Neither of which seems to scale well or perform well for large amounts of data. I know I could also add a pipelined TABLE function to filter the data more, but none of these options seem like great solutions. Does anyone have any ideas for a better way to form complex well performing queries against this type of data layout?
For example, with my sample data below I'd like to select everyone whose (income = 2000) or (income = 50000 and weight = 210), but the only way I see of doing this is with a query like the following (which won't take advantage of any indexes on the data):
select * from people_attributes_view
where find_eq_attr(AttributeValue('income',2000),attribute_values) is not null
or (find_eq_attr(AttributeValue('income',50000),attribute_values) is not null
and
find_eq_attr(AttributeValue('weight',210),attribute_values) is not null);
Any help is greatly appreciated.
Thanks.
-- My full example starts here ----------------
-- Create the sample tables and data
create table people (id int, name varchar(30));
create table attributes (id int, attribute varchar(10), value NUMBER);
insert into people values (1,'tom');
insert into people values (2,'jerry');
insert into people values (3,'spike');
commit;
insert into attributes values (1,'age',23);
insert into attributes values (1,'income',1000);
insert into attributes values (1,'height',5);
insert into attributes values (1,'weight',120);
insert into attributes values (2,'age',20);
insert into attributes values (2,'income',2000);
insert into attributes values (3,'age',30);
insert into attributes values (3,'income',50000);
insert into attributes values (3,'weight',210);
commit;
-- Create some types, functions and views for the search
CREATE OR REPLACE TYPE AttributeValue AS OBJECT
(attribute varchar(30),
value NUMBER);
CREATE OR REPLACE TYPE AttributeValues AS TABLE OF AttributeValue;
create or replace function find_eq_attr (val AttributeValue, vals AttributeValues) RETURN AttributeValue IS
begin
for i in vals.FIRST .. vals.LAST
LOOP
if ((val.attribute = vals(i).attribute) and
(val.value = vals(i).value)) then
return vals(i);
end if;
END LOOP;
return null;
end;
create or replace view people_attributes_view as
select p.name, p.id,
cast(multiset(select attribute,value from attributes a where a.id = p.id)
as AttributeValues) attribute_values
from people p;
-- Search for everyone whose (income = 2000) or (income = 50000 and weight = 210)
select * from people_attributes_view
where find_eq_attr(AttributeValue('income',2000),attribute_values) is not null
or (find_eq_attr(AttributeValue('income',50000),attribute_values) is not null
and
find_eq_attr(AttributeValue('weight',210),attribute_values) is not null);

I like the flexibility of this format, especially
since it can handle thousands of attributes. But the
main problem I have with this format is in
determining how to write efficient, complex queries
against the data. Can't be done. The flexibility is achieved at the cost of simplicity and performance, it is a trade off.
See this thread for a full discussion on the subject.
http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:10678084117056

Similar Messages

  • How to transfer data between different tables in an Oracle 10g databse?

    I have to do the following: there are 5 database tables in our Oracle 10g database that store certain data. Then there are 2 new tables where we want the data to go to. My task is to somehow transfer all the data that is in the first 5 tables and insert it in the 2 new ones. All tables are in the same database. The challenge lies in the fact that the structures of the tables are very dissimilar so it won't be a matter of a simple INSERT SQL query.
    The data access tier of our application is developed in JAVA. We use EJB. So one way I am thinking of doing the above mentioned data transfer is to simply code it in the JAVA backed. But that might be to slow, there are millions of records in those first 5 tables. What is the best way to do something like that? Perhaps a custom (Perl?) script or some such?
    Thanks.

    The problem is that the way the data is stored in the old tables you cannot write a query that would return it in such a format that you could immediately insert it in the new table without some formatting. Let me illustrate with an example. To pull the data from the old tables I use this query:
    SELECT
         SXML_DOCUMENT_DATA.VALUE As DocumentValue,
    SXML_ELEMENT.NAME As ElementName
    FROM
         SXML_DOCUMENT,
         SXML_DOCUMENT_DATA,
         SXML_DOCUMENT_DETAIL,
         SXML_ELEMENT,
         SXML_TYPE
    WHERE
         SXML_TYPE.XML_TYPE_KEY = SXML_DOCUMENT.XML_TYPE_KEY
    AND     SXML_DOCUMENT.XML_DOCUMENT_KEY = SXML_DOCUMENT_DETAIL.XML_DOCUMENT_KEY
    AND     SXML_DOCUMENT_DETAIL.XML_DOCUMENT_DETAIL_KEY = SXML_DOCUMENT_DATA.XML_DOCUMENT_DETAIL_KEY
    AND     SXML_ELEMENT.XML_ELEMENT_KEY = SXML_DOCUMENT_DATA.XML_ELEMENT_KEY
    AND     SXML_TYPE.NAME = 'DA_UNIT_COMMITMENT'
    AND     (SXML_ELEMENT.NAME = 'resource'
         OR SXML_ELEMENT.NAME = 'resourceType'
         OR SXML_ELEMENT.NAME = 'commitmentType'
         OR SXML_ELEMENT.NAME = 'startTime'
         OR SXML_ELEMENT.NAME = 'endTime'
         OR SXML_ELEMENT.NAME = 'schedulingCoordinator')
    ORDER BY
         SXML_DOCUMENT.NAME,
         SXML_DOCUMENT_DETAIL.XML_DOCUMENT_DETAIL_KEY,
         SXML_DOCUMENT_DATA.XML_DOCUMENT_DATA_KEY,
         SXML_ELEMENT.NAME;
    The results from the SQL query above look like this:
    DOCUMENTVALUE | ELEMENTNAME
    1 | ALAMIT_7_UNIT_1 | resource
    2 | GEN | resourceType
    3 | BRS8 | schedulingCoordinator
    4 | IFM | commitmentType
    5 | 2008-07-29T18:00:00:00 | startTime
    6 | 2008-07-30T00:00:00 | endTime
    7 | ALAMIT_7_UNIT_1 | resource
    8 | GEN | resourceType
    9 | BRS8 | schedulingCoordinator
    10 | IFM | commitmentType
    11 | 2008-07-29T00:00:00 | startTime
    12 | 2008-07-29T04:00:00 | endTime
    and so on. The type of data repeats every 6 records. And the values of each 6 records corresponds to 1 row in the new table, which looks like this:
    schedulingCoordinator | resource | resourceType | commitmentType | startDate | endDate
    1| 1 | 27 | GEN | IFM | 2008-07-29T18:00:00:00 | 2008-07-30T00:00:00
    2| 1 | 27 | GEN | | 2008-07-29T00:00:00 | 2008-07-29T04:00:00
    So hopefully now you see the challenge. It is not as simple as writing a SQL query that returns result rows corresponding 1 to 1 to a row in the new table. Somehow I need to take the first 6 result rows from that big SQL query and put them in the first row of the new table. Then the next 6 and put them in the second row and so on. And also, we are talking about millions of records. What happens if I process the first 2 million and then some error occurs? Do I start from the beginning again? One idea I have is to process the data in chunks, say process 500 results, commit, process another 500, commit, and so on.
    Edited by: 799984 on Oct 11, 2010 9:08 AM

  • Give me the sql query which calculte the table size in oracle 10g ecc 6.0

    Hi expert,
    Please  give me the sql query which calculte the table size in oracle 10g ecc 6.0.
    Regards

    Orkun Gedik wrote:
    select segment_name, sum(bytes)/(1024*1024) from dba_segments where segment_name = '<TABLE_NAME>' group by segment_name;
    Hi,
    This delivers possibly wrong data in MCOD installations.
    Depending on Oracle Version and Patchlevel dba_segments does not always have the correct data,
    at any time esp. for indexes right after being rebuild parallel (Even in DB02 because it is using USER_SEGMENTS).
    Takes a day to get the data back in line (never found out, who did the correction at night, could be RSCOLL00 ?).
    Use above statement with "OWNER = " in WHERE for MCOD or connect as schema owner and use USER_SEGMENTS.
    Use with
    segment_name LIKE '<TABLE_NAME>%'
    if you like to see the related indexes as well.
    For partitioned objects, a join from dba_tables / dba_indexes to dba_tab_partitions/dba_ind_partitions to dba_segments
    might be needed, esp. for hash partitioned tables, depending on how they have been created ( partition names SYS_xxxx).
    Volker

  • What is the easiest way to export all tables data from Oracle to MS SQL Server?

    Hello MS,
    I would like to export all tables from Oracle 11.2 to MS SQL Server 2012 R1.
    Using the tool "Microsoft SQL Server Migration Assistant v6.0 for Oracle" did not work for me because there are too many warnings and errors regarding the schema creation (MS cannot know it because they are not the schema designer). My idea is
    to leave/skip the schema creation to the application designer/supplier and instead concentrate on the Oracle data export and MS SQL data import.
    What is the easiest way to export all tables data from Oracle to MS SQL Server quickly?
    Is it:
    - the „MS SQL Import and Export Data“ Tool
    - the “MS SQL Integration Services” Tool
    - not Oracle dump *.dmp format because it is a propritery binary format
    - flat file *.csv (delimited format)
    Thanks!

    Hi lingodingo,
    If you want to directly export all tables from Oracle database to SQL Server, I suggest you use SQL Server Import and Export Wizard. Because you just need to follow the wizard with GUI, this is the easiest way.
    If you want to make some modification for the tables‘ data before loading to SQL Server, I suggest you use SQL Server Integration Services package. For more details, please refer to the following similar thread:
    https://social.msdn.microsoft.com/Forums/sqlserver/en-US/38b2bed2-9d4e-48d4-a33f-1d9eed1c062d/flat-file-to-sql-server?forum=sqldatamining
    Thanks,
    Katherine Xiong
    Katherine Xiong
    TechNet Community Support

  • Error in import table data using oracle datapump

    i am trying to import table data using oracle datapump
    CREATE TABLE emp_xt (
    ID NUMBER,
    NAME VARCHAR2(30)
    ORGANIZATION EXTERNAL (
    TYPE ORACLE_DATAPUMP
    DEFAULT DIRECTORY backup
    LOCATION ('a.dmp')
    it return the following error
    ORA-29913: error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    KUP-04084: The ORACLE_DATAPUMP access driver does not support the ROWID column.
    ORA-06512: at "SYS.ORACLE_DATAPUMP", line 19
    please help me

    .dmp file generated from exp command file not from oracle_datapump

  • How to find the count of tables going for fts(full table scan in oracle 10g

    HI
    how to find the count of tables going for fts(full table scan) in oracle 10g
    regards

    Hi,
    Why do you want to 'find' those tables?
    Do you want to 'avoid FTS' on those tables?
    You provide little information here. (Perhaps you just migrated from 9i and having problems with certain queries now?)
    FTS is sometimes the fastest way to retrieve data, and sometimes an index scan is.
    http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:9422487749968
    There's no 'FTS view' available, if you want to know what happens on your DB you need, like Anand already said, to trace sessions that 'worry you'.

  • Is there any method to export the data from oracle 10g in any format !!!!!

    is there any methods to export the data from oracle 10g in the format of excel or csc or txt or anyother. i already downloaded the sql developer tool but it only exports the data of upto one lakh rows ,but i have two tables of more than 3 lakh and 10 lakh rows respectively.for this i used 'where' command to break the file through sql developer but that didnt work
    plz help me out if any thing possible
    i need to again import this data into microsoft sql if there is any direct method plz let me know
    its urgent
    Message was edited by:
    user628031
    Message was edited by:
    user628031

    Take a look at DUMP_CSV function by Tom Kyte.

  • Problem in importing data in oracle 10g from 9i backup

    Hi ,
    Am trying to import data in oracle 10g..but it does not importing constraints, and stop doing anything by the msg.
    About to enable constraints.....
    after this msg it does not work.
    plz help,
    Regards,
    Neha

    Hi Neha,
    You have a lot of options. I'll supose you can test first, and you're using an export file.
    First forget about the data, think in your database structure. So you can do this:
    exp userid=... file=exp_norows.dmp full=y rows=n statistics=none
    Now you have the structure of your database. Create the database, in 10g, empty, check you have all filesystem, etc. for you new datafiles. And do the import with parameter ignore=y, you'll see a lot off errors on SYSTEM schema, don't worry about it.
    Now you have the structure of you database in 10g. You can take a look in the import log file, looking for some possible errors come from your schemas. so check the objects, and check the schemas.
    Bear in mind things like CONNECT role in 9i it's different in 10g, you can't create db_links in 10g with "IDENTIFIED BY VALUES" because you'll get an ORA-00600, etc.
    Well, fix all problems on source database (9i) and repeate this procedure until you don't recieve any more errors (not a SYSTEM schema errors).
    When you have everything solved, you can try to import the data.
    I hope, this method can help you. If you have any problem do not hesitate to ask me.
    John Ospino Rivas

  • How to delete the recyclebin data  in oracle 10g

    i create one table.then i drop the that table.this table will be kept in recyclebin in oracle10g .how to remove table permanentley in oracle 10g.
    i used drop table table name and select * from tab;it will come two tables like
    BIN$kLM/ilgxTUil64ZsNG0l7A==$0.
    how to drop table permanentley.please give me steps.

    Hello;
    Use the PURGE statement to remove a table or index from your recycle bin and release all of the space associated with the object, or to remove the entire recycle bin, or to remove part of all of a dropped tablespace from the recycle bin.
    Remove a File From Your Recycle Bin: Example
    The following statement removes the table test from the recycle bin. If more than one version of test resides in the recycle bin, Oracle Database removes the version that has been there the longest:
    PURGE TABLE test;
    To determine system-generated name of the table you want removed from your recycle bin, issue a SELECT statement on your recycle bin. Using that object name, you can remove the table by issuing a statement similar to the following statement. (The system-generated name will differ from the one shown in the example.)
    PURGE TABLE RB$$33750$TABLE$0;
    Remove the Contents of Your Recycle Bin: Example
    To remove the entire contents of your recycle bin, issue the following statement:
    PURGE RECYCLEBIN;

  • Migration using data pump Oracle 10g - Oracle 11g

    HI all,
    1)
    Right now I`m using Oracle 11g. I have a plan to import data from Oracle 10g. I would like to know if its possible to import data which was exported by datapump on Oracle 10g ?
    Should I convert somehow expdp output from Oracle 10g to Oracle 11g format ?
    2)
    Next question is. If I will use expdp to create dump of complete database. May I use *.dmp to import selected users ? Or only full database can be restored ?

    Yes, you can import 10g dump into an 11g database.
    Maybe you should take some time and read the section on datapump in the fine Oracle® [Database Utilities|http://download.oracle.com/docs/cd/B28359_01/server.111/b28319/dp_import.htm#i1007324] manual.
    :p

  • Getting all the CRM on Demand Data in Oracle 10g database

    Hello Everyone,
    The requirement is to get all the module data into Oracle 10g database.
    Currently i am using the background worker in .NET to span through all the modules, but recently experiencing performance issues due to data volume.
    I want to use .NET as the programming language.
    Kindly suggest any other methids / techniques.
    Thanks in advance.
    Regards.

    Thanks for your reply.
    We are using CRMOD webservices.
    We want the full OD data to be refreshed overnight and we havve used .NET to build the EXE, the same we have configured as a task.
    Could you please shed some more light as to what would be the best practices to do this.
    Regards.

  • How to migrate MS SQL Server OLAP data to Oracle 10g OLAP

    Hi,
    Anyone has any idea on whether migration workbench can migrate my MS SQL Server OLAP data to Oracle 10g (with OLAP option) ?
    Best Regards,
    Ian Ho

    Ian,
    What exactly do you mean by SQL*Server OLAP ? Is it just a 'normal' SQL*Server database which is used for an OLAP application ?
    The migration workbench migrates schema objects, triggers, and stored procedures and is not concerned with the application that uses those objects.
    And the same with the Oracle OLAP database. Once the data etc has been moved into the database you can then use it for whichever application you want.
    If you have concerns apart from moving the actual data then please give us more details.
    Regards,
    Mike

  • Plz help to to transfer excel or acess data to Oracle 10g database

    Plz help me to transfer excel or acess data to Oracle 10g database.
    I have excel and acess sheet data here i wanna transfer it to oracle database. Plz guide me how to this
    Message was edited by:
    user582212

    There are several ways, such as ODBC or migration workbench, which is also mentioned in the current article on sqldeveloper on OTN, here's the link. If you use this forum's search function, you should find several threads regarding this topic.
    C.
    Message was edited by:
    cd

  • Query about multi record block

    Abdetu
    Please clarify this
    I have a multi record group on the canvas, which displays 5 records at a time. The block is based on a table. All the items, except 3 items are displayed items.Now in the block I have a list item, in which I have two values "PO" and "TSF".. If user selects PO, associated text items will get enabled and user will be entering the values .. Same is the case with TSF .. And at the end user selects Ok button to save the record to the database
    Now while fetching the records (based on query) ..once the records are fetched, control will be on PO/TSF list box, now the user can either change PO to TSF or TSF to PO ..or he/she may opt not to change also.. If user changes PO to TSF, only that particular record's TSF fields should get enabled .. The problem what I am facing is, when the control comes to PO/TSF list box.. since its multi record block, all the record's PO/TSF list box are getting enabled
    In the sense, if control is in the 1st record of multi record block, only those fields should get enabled, once the user navigates to the text item which is the last of that particular record, then 2nd record should get enabled....
    Can I know the solution
    Regards
    Message was edited by:
    Seshu
    Message was edited by:
    Seshu

    Hello Seshu
    ur case explanation should be in points 1,2,etc as possible leaving extra lines and spaces in between .. and the case should be more clear and summarized like:
    1- I have multi-records data block.
    2- I have a static list item with 2 values eg. (a and b).
    3- Depending on selecting from the list some items r enabled.etc.
    Kindly note that, this will facilitate ur problem to be more readable,recognizable and save time and effort of someone who wants to help u.
    Here comes the confusion:
    Now while fetching the records (based on query)
    The problem what I am facing is, when the control comes to PO/TSF list box.. >since its multi record block, all the record's PO/TSF list box are getting enabled i think with the piece of code i have just posted to u this can't be happened since it disable all the record's PO/TSF.
    if this didn't solve ur problem i suggest either u post ur code to help u or u try to put some validations with :
    if statment inside the loop depending on the list item's values which will enable or disable related items to the list.
    Hope this help u,
    If any Questions pls let me know..
    Regards,
    Abdetu..

  • Multi-record / spread table...moveable fields in designer generated form

    Good morning all;
    I hope you are doing good...
    Imagine a multi-record block layout with it's overflow property set to spread-table to accomodate 10 varchar2(25) fields. Now, of the hundreds of users working with this generated form, not one of them wants to see the fields in the same order on the screen.
    What i would like to do is generate the form from designer so that a user could 'drag'n drop' let say, field #10 to the field #2 spot on the screen. Something like windows explorer where you can move the 'Size' column around so it is viewable without having to use the spread-table bar at the bottom of the screen...
    Any idea on how to generate this form from designer?
    Thank you and wish me luck!

    I don't know a solution, but I think a solution could be easier found if it was a read-only block. You did not specify that. Then maybe you could dynamically change the query behind the block.
    Good luck, Paul.

Maybe you are looking for

  • App for 1st gen iPod Touch

    Is there an ereader that will work with a 1st gen ipod touch running ios 3.1.3?  If yes, where can I find & how do I download onto the ipod?

  • Datasources missing in R/3

    Hi all we transported datasources from R/3 DEV to R/3QAU. On BI side we have done dataloading. We observed that QAU refresh&backup has beeen done on Aug 12th 2011. Now In quality system, we couldn't find any datasources in RSA5 and RSA6. Also on BI s

  • Upgradation

    Hi I am currently in a upgradation project...SAP R/3 4.6 to  ECC 6.0 can anyone help me in preparing the tfunctionalest case scenarios for  SD which are to be followed during upgrade... thanks prasad

  • How to open Excel FIles

    I am a novice When I open excel file  I found some problems. using System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using System.Drawing; using System.Linq; using System.Text; using System.Threading.Tasks; usi

  • Address book contact

    Hi I do not understand why I am not seeing a contact in my Skype suggested contact when I saved his number then why it is not showing as address book contact