Selecting Records from 125 million record table to insert into smaller table

Oracle 11g
I have a large table of 125 million records - t3_universe.  This table never gets updated or altered once loaded,  but holds data that we receive from a lead company.
I need to select records from this large table that fit certain demographic criteria and insert those into a smaller table - T3_Leads -  that will be updated with regard to when the lead is mailed and for other relevant information.
My question is what is the best (fastest) approach to select records from this 125 million record table to insert into the smaller table.  I have tried a variety of things - views, materialized views, direct insert into smaller table...I think I am probably missing other approaches.
My current attempt has been to create a View using the query that selects the records as shown below.  Then use a second query that inserts into T3_Leads from this View V_Market.  This is very slow. Can I just use an Insert Into T3_Leads with this query - it did not seem to work with the WITH clause?    My Index on the large table is t3_universe_composite and includes zip_code, address_key, household_key. 
CREATE VIEW V_Market  as
WITH got_pairs    AS  
     SELECT /*+ INDEX_FFS(t3_universe t3_universe_composite) */  l.zip_code, l.zip_plus_4, l.p1_givenname, l.surname, l.address, l.city, l.state, l.household_key, l.hh_type as l_hh_type, l.address_key, l.narrowband_income, l.p1_ms, l.p1_gender, l.p1_exact_age, l.p1_personkey, e.hh_type as filler_data, 1.p1_seq_no, l.p2_seq_no 
     ,      ROW_NUMBER () OVER ( PARTITION BY  l.address_key 
                                  ORDER BY      l.hh_verification_date  DESC 
                  ) AS r_num   
     FROM   t3_universe  e   
     JOIN   t3_universe  l  ON   
            l.address_key  = e.address_key
            AND l.zip_code = e.zip_code
          AND   l.p1_gender != e.p1_gender
             AND   l.household_key != e.household_key         
             AND  l.hh_verification_date  >= e.hh_verification_date 
  SELECT  * 
  FROM  got_pairs
  where l_hh_type !=1 and l_hh_type !=2 and filler_data != 1 and filler_data != 2 and zip_code in (select * from M_mansfield_02048) and p1_exact_age BETWEEN 25 and 70 and narrowband_income >= '8' and r_num = 1
Then
INSERT INTO T3_leads(zip, zip4, firstname, lastname, address, city, state, household_key, hh_type, address_key, income, relationship_status, gender, age, person_key, filler_data, p1_seq_no, p2_seq_no)
select zip_code, zip_plus_4, p1_givenname, surname, address, city, state, household_key, l_hh_type, address_key, narrowband_income, p1_ms, p1_gender, p1_exact_age, p1_personkey, filler_data, p1_seq_no, p2_seq_no
from V_Market;

I had no trouble creating the view exactly as you posted it.  However, be careful here:
and zip_code in (select * from M_mansfield_02048)
You should name the column explicitly rather than select *.  (do you really have separate tables for different zip codes?)
About the performance, it's hard to tell because you haven't posted anything we can use, like explain plans or traces but simply encapsulating your query into a view is not likely to make it any faster.
Depending on the size of the subset of rows you're selecting, the /*+ INDEX hint may be doing your more harm than good.

Similar Messages

  • How to select data from 3rd row of Excel to insert into Sql server table using ssis

    Hi,
    Iam having Excel files with headers in first two rows , i want two skip that two rows and select data from 3rd row to insert into Sql Server table using ssis.3rd row is having column names.

                                                         CUSTOMER DETAILS
                         REGION
    COL1        COL2        COL3       COL4           COL5          COL6          COL7
           COL8          COL9          COL10            COL11      
    1            XXX            yyyy         zzzz
    2            XXX            yyyy        zzzzz
    3           XXX            yyyy          zzzzz
    4          XXX             yyyy          zzzzz
    First two rows having cells merged and with headings in excel , i want two skip the first two rows and select the data from 3rd row and insert into sql server using ssis
    Set range within Excel command as per below
    See
    http://www.joellipman.com/articles/microsoft/sql-server/ssis/646-ssis-skip-rows-in-excel-source-file.html
    Please Mark This As Answer if it solved your issue
    Please Mark This As Helpful if it helps to solve your issue
    Visakh
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • Records are not inserted into MTL_TXN_REQUEST_HEADERS,MTL_TXN_REQUEST_LINES

    Hi all,
    I am doing pick release thru API.
    For that , I have run following API successfully but my problem is that records are not inserted into MTL_TXN_REQUEST_HEADERS , MTL_TXN_REQUEST_LINES
    tables so that I cannot generate the move order. Can anyone suggent the valuable tips for that ?
    1. WSH_DELIVERIES_PUB.create_update_delivery
    2. WSH_DELIVERY_DETAILS_PUB.Detail_to_Delivery
    3. WSH_PICKING_BATCHES_PUB.CREATE_BATCH
    4. WSH_PICKING_BATCHES_PUB.RELEASE_BATCH
    5. WSH_DELIVERIES_PUB.Delivery_Action ( for p_action_code = 'PICK-RELEASE' )
    regards with thanks in advance
    sanjay

    Hi All,
    To create a Move Order Header & Line Record in MTL_TXN_REQUEST_HEADERS and MTL_TXN_REQUEST_LINES tables , I have run an seeded
    procedure : CreateMoveOrderHeader & CreateMoveOrderLines .
    The procedure : CreateMoveOrderHeader was run successfully but when I am going to run procedure : CreateMoveOrderLines , it is giving
    an error .
    Following Error text I am receiving at the time of running of procedure : CreateMoveOrderLines .
    Initialized applications context: 5707 50138 660
    *==========================================================*
    Calling INV_MOVE_ORDER_PUB.Create_Move_Order_Lines API
    *==========================================================*
    Return Status: E
    Error Message :INV
    *==========================================================*
    From the above error message, no specific error is received . Only " Error Message : INV " is received hence it is very difficult to judge & resolve an error .
    Can anyone guide me that what is an error ? For what reason I am getting an error ?
    regrads
    Sanjay

  • Record is not inserting into the Table thourgh forms 10g

    Hi All,
    I have the form built in 10g(10.1.2.0.2).
    Basically the form has 2 blocks.
    Block 1 with only one item, where we enter some value and hit enter(this will navigate to block2 and execute the query).
    Block 2(tabular) will fetch the records. Now this block2 have 3 columns(caseid, userid, date).
    Now when I insert a new record, I just need to enter the caseid only. And userid and date have to be automatically populated.
    I can populate the USERID and DATE fields.
    And when I enter some value in caseid item of block 2, and then do Control+S(to insert the record and Save the transaction),
    i get the message saying FRM-40400: Transaction complete: 1 records applied and saved.
    But when I query again for the same, I dont se the record inserted into the table.
    Why is this happening?
    Help please...

    @ Inol
    There is no promary or foriegn key relation ship. The form is fetching records from just one table. As I said block1 has col1. and Block2 has col2,3,4.
    @Andreas
    Yes I did select * from table, select count(*) from table in SQL*Plus. Nothing is inserted.
    And one thing I have to tall here is, I have a ON-INSERT trigger in block2.
    The code in ON-INSERT trigger is
    --  :BLOCK2.DATE := TO_DATE(:DATE,'DD-MON-YYYY') ;
    /* commented since I was populating date by assigning $$DATETIME$$ to the Initial value property of the data item. */
    :BLOCK2.SOURCE_CASE := :BLOCK1.SOURCE_CASEID;Forgot to tell you,
    Since I was inserting the record from block2(3 columns), to the table which has 4 columns, there is another item in block 2 which has the visible property No.
    So the block 2 has 4 columns. And the hidden column will hold the value that is in the item of block1. This is what i'm pushing in the ON-INSERT trigger.
    Edited by: Charan on Sep 20, 2011 1:51 PM

  • How to select data from a PL/SQL table

    Hi,
    I am selecting data from database after doing some screening i want to store it in a PL/SQL table (temporary area) and pass it to oracle reports.
    Is there any way to select the data from a PL/SQL table as a cursor. Or is there any other way of holding the temporary data and then pass it back as a cursor.
    Regards
    Kamal

    A PL/SQL "table" is anything but a table. Whoever came up with this term in PL/SQL to describe what is known as dynamic arrays (the correct programming terminology that existed since the 70's if not earlier and what is used in all other programming languages I'm familiar with)... well, several descriptions come to mind and none of them are complimentary.
    You cannot "select" from a PL/SQL dynamic array as it is not a table within the Oracle context of tables.
    Thus you need to convert (cast) a PL/SQL dynamic array into a temporary Oracle data set/table in order to select from it. This is in general a Bad Idea (tm). Oracle tables and SQL and concurrency controls and all that are especially designed for processing data. PL/SQL arrays is a very simplistic data structure with very limited usage. Why would you want to use that in SQL via a SELECT statement when you can use Oracle tables (or proper temp tables) instead? Besides that, it is also slow to cast a dynamic PL/SQL array into an Oracle SQL data set structure (context switching, copying of memory, etc).
    The proper way to use PL/SQL to generate data sets for use via the SQL engine is pipelined table functions.
    This is not to say that you should never use PL/SQL arrays and casting in SQL.. simply that you need to make sure that this is the correct and scalable way to do it. And that will also always be an exception to the rule when you do.

  • Select data from 4 or more tables

    Hi All,
    I got stUck up in a problem in Oracle.
    I have 4 tables say t1,t2,t3,t4
    t1 has c1 is a primary key,c2,c3
    t2 has c1 is a foreign key ,c4,c5
    t3 has c1 is a foreign key,c6,c7
    t4 has c1 is a foreign key,c8,c9
    I want to retreive the data from all these tables at a time.i.e. c1,c2,c3,c4,c5,c6,c7,c8,c9 such that
    it doesnot result in duplicacy od records.
    that is the record should be grouped if it has same value
    FOR EXAMPLE
    a query
    "select company.company_name,import_from,export_to from company,importfrom,exportto
    where company.company_name=exportto.company_name
    and
    company.company_name=importfrom.company_name"
    will retreive
    COMPANY_NAME iMPORT_FROM EXPORT_TO
    IMG CANADA INDIA
    IMG CANADA ITALY
    IMG CANADA BRAZIL
    IMG CANADA CHINA
    IMG CANADA SOUTH AFRICA
    IMG MEXICO INDIA
    IMG MEXICO ITALY
    IMG MEXICO BRAZIL
    IMG MEXICO CHINA
    IMG MEXICO SOUTH AFRICA
    AND I WANT
    COMPANY_NAME iMPORT_FROM EXPORT_TO
    IMG null INDIA
    IMG null ITALY
    IMG NULL BRAZIL
    IMG NULL CHINA
    IMG NULL SOUTH AFRICA
    IMG CANADA NULL
    IMG MEXICO NULL
    PLEASE REPLY SOON
    Manoj

    Try this query.
    select c.company_name, e.exp, null
    from company c, export_to e
    where c.company_name = e.company_name
    union
    select c.company_name, null, i.imp
    from company c, import_from i
    where c.company_name = i.company_name
    If you have 4 tables, you'll need 3 queries with union statement.

  • Delete from 95 million rows table ...

    Hi folks, need to delete from a 95 millions rows regular table, what should be my best options, have tried CTAS using parallel, but it failed after 1+ hrs ... it was due to bad query, but checking is there any other way to achieve this.
    Thanks in advance.

    user8604530 wrote:
    Hi folks, need to delete from a 95 millions rows regular table, what should be my best options, have tried CTAS using parallel, but it failed after 1+ hrs ... it was due to bad query, but checking is there any other way to achieve this.
    Thanks in advance.how many rows in the table BEFORE the DELETE?
    how many rows in the table AFTER the DELETE?
    How do I ask a question on the forums?
    SQL and PL/SQL FAQ
    Handle:     user8604530
    Status Level:     Newbie
    Registered:     Mar 10, 2010
    Total Posts:     64
    Total Questions:     26 (22 unresolved)
    I extend to you my condolences since you rarely get your questions answered.

  • How to select data from all HR Master Tables(PA0000 to PA****)

    Hi Gurus!
    My requirement is to pick the changedon(AEDTM) , changedby(UNAME) and reason for change(PREAS)  from all tables (pa0000 to pa****)of HR  in which he made any change during a specific date.
    Now if it is specific for two or three tables then inner join can solve it. but now it is the matter of picking data from maximum 9999 tables for which inner join will not be accurate.
    plz suggest me any appropriate way to solve it out. or there is any table which store  the changes in All HR Master Tables.
    Thanking in Advance.
    Tahir Naqqash

    Hi,
    it is very tidious to get it in one go, as u know hr tables contains huge data.
    You can chack my wiki in https://wiki.sdn.sap.com/wiki/x/XYC0Aw .
    Here you need to make the following modification.
    1. in the selection screen set on date range with select option.
    2. in the select query, change select * by ur specified fields.
    3. change dynamic internal table and ALV as per ur required fields
    In this way you have to put the table name and the date range and will get the data.
    for repeation u can call the same job in background again and again store data in a seperate internal table and display finaly ..... but huge chance of getting 'Time Out ' error in that case for comparetively large day gap.
    Regards,
    Anirban

  • Record not getting inserted into R/3 table.

    Hi Experts,
    I am trying to save some information in R/3 using BAPI. I have written codes like as shown below:
    Code in View
    public void onActionInsertRecord(com.sap.tc.webdynpro.progmodel.api.IWDCustomEvent wdEvent )
        //@@begin onActionInsertRecord(ServerEvent)
        try {          wdThis.wdGetTimesheetCompController().executeBapi_Catimesheetmgr_Insert_Input();
                   wdThis.wdGetTimesheetCompController().executeBapi_Transaction_Commit_Input();
        } catch (Exception ex)
        {wdComponentAPI.getMessageManager().reportException(ex.getMessage(), false);
        //@@end
    Code in Component Controller
    public void executeBapi_Catimesheetmgr_Insert_Input( )
        //@@begin executeBapi_Catimesheetmgr_Insert_Input()
        IWDMessageManager manager = wdComponentAPI.getMessageManager();
        try
        {   Bapicats1 xx = new Bapicats1();
           xx.setWbs_Element(wdContext.currentWorklistElement().getRec_Wbs());
           xx.setEmployeenumber(wdContext.currentZbapi_Cat_Stech00_Getdetail_InputElement().getEmployeenumber());
           xx.setCatshours(new BigDecimal ("1"));
           xx.setWorkdate(wdContext.currentOutputElement().getDate_From());
           wdContext.currentBapi_Catimesheetmgr_Insert_InputElement().setProfile(wdContext.currentZbapi_Cat_Stech00_Getdetail_InputElement().getProfile());
              wdContext.currentBapi_Catimesheetmgr_Insert_InputElement().modelObject().execute();
              wdContext.nodeOutputnew().invalidate();
        catch(WDDynamicRFCExecuteException e)
          manager.reportException(e.getMessage(), false);
        //@@end
    My records are  not getting saved in R/3 tables. Can you please let me know what is going wrong here. The BAPI input parameters values are correctly getting populated in Component Controller.
    I have also created error message table in View. This error message table is mapped to output of Bapi_Catimesheetmgr_Insert. This table is not getting populated.
    Due to this reason, I believe that the values of input parameters of BAPI are not getting passed to R/3. However, I am not sure where the problem lies. Can you please help me in resolving the issue?
    Regards,
    Brian

    Hi Sumit,
    I am not getting any exception. Using SE37, I can save the transaction. Can you please let me know what is going wrong here.
    Regards,
    Sumit

  • Insert selected records

    Hi,
    I have a requirement as follows
    I need to have a report (say in page1 on table1) with the checkboxes in order to select the required records.
    When I click on next button, I should be able to display these selected records only to the next page(say page2) .
    Page2 has submit button and when I click on submit these selected records should be inserted into a table(say table2).
    I am able to create a report with checkboxes but not understanding how to collect these selected records to the next page and insert them into database table.
    Kindly advice me how this can be achieved.
    Thanks in Advance.
    Sarvani.

    Sarvani,
    here is an example on how to select checked rows:
    http://htmldb.oracle.com/pls/otn/f?p=31517:95
    You should use an array to collect the picked records - item with concatenated primary
    keys. After that, you may use the example here:
    http://htmldb.oracle.com/pls/otn/f?p=31517:84
    to insert those records into a table.
    Denes Kubicek
    http://deneskubicek.blogspot.com/
    http://htmldb.oracle.com/pls/otn/f?p=31517:1
    -------------------------------------------------------------------

  • How can I find out the place from where a record is being inserted in table

    Hi,
      I want to find out the exact code statement from the SAP code from where record is being inserted into a SAP table. I tried ST05 SQL trace but it seems to only give the SQL statement and not the program from where it is being fired (unless you guys can show me a way to find it from the trace)
      I m trying to see that when sales order is changed in VA02, from which piece of code in SAP the tables CDHDR and CDPOS get update. Also I tried using SE30 but it doesnt seem to give the results properly. Please help.
    Thanks for reading

    Hello Srikrishna,
    Did you try the following?
    In your trace with ST05:
    i) Find the line that you are interested, right-click and select 'Display ABAP Source'.
    OR
    ii) Go to Trace List menu and click 'Summarize Trace by SQL Statement'. Then here, select the line you want and again right click to see 'Display ABAP Source'.
    Regards,
    Guven.

  • Deleting selected record in Dynamic Internal table

    Hi Friends,
    I want to delete selected row from a dynamic internal table. The selected entry should be deleted from Adobe and as well as Webdynpro abap.
    Kindly help me how to solve this problem.
    Thanks in advance.
    Regards,
    Phani.

    Hi Matthias,
    Thanks a lot for responding.
    In my adobe i kept a check box to delete the record. It should delete the selected ones. But right now, its deleting one by one.
    I have coded following script for delete button:
    IT_TIME_SHEET --> is my internal table
    G_ROW_STATUS is the field to track the action performed.
    var tlength = xfa.resolveNodes("IT_TIME_SHEET.DATA[*]").length;
    for ( var i=0; i<tlength; i++
    if(xfa.resolveNode("IT_TIME_SHEET.DATA["i"].FLAG").rawValue ==
    1 )
    IT_TIME_SHEET.DATA.instanceManager.removeInstance(i);
    G_ROW_STATUS.rawValue = "DELETE";
    In the Webdynpro abap following code I wrote:
        row_count = 1.
        FIELD-SYMBOLS <wa> TYPE zshr_time_sheet_time.
        LOOP AT it_time_sheet ASSIGNING <wa>.
          <wa>-srl_no = row_count.
          row_count = row_count + 1.
        ENDLOOP.
        last_row = <wa>-srl_no.
          DELETE it_time_sheet WHERE srl_no EQ last_row.
    Suppose if two records checkboxes are selected I can loop at the internal table IT_TIME_SHEET in webdynpro and delete where checkbox is selected.
    But problem is once user deletes records, adobe is deleting only one record and if i delete two records from webdynpro it looks like something wrong for the user.
    Please help me how to solve this problem.
    Regards,
    Phani.

  • Record not inserting into the table through Forms 10g

    Hi all,
    I have created a form in 10g(10.1.2.0.2) based on just one table that has 4 columns(col1, col2, col3, col4).
    Here col1, col2 and col3 are VARCHAR2 and col4 is date and all the columns are not null columns(There are no primary and foriegn key constrains, which means duplicates are allowed).
    My form contains 2 blocks where block 1 has one text item (col1) and 3 buttons (Delete, Save, Exit).
    And block2 is a database block and has col2,col3,col4 which are in tabluar layout frame displaying 10 records.
    When the form is opened the cursor has to be in block1.col1 for querrying. Here i enter a value in col1, and then when I click on col2 in the block2, then I put execute_query in new_block_instance of block2, which displays the records.
    The block2 properties are not updatable, insertable and query is allowed.
    Everything is working good until here. But here in the block2 when I want to insert another record into the table, by navigating all the way down to the last empty record and entering the new values for col2, col3 and col4 And then Ctrl+S will display the message "*FRM-40400: Transaction complete: 1 record applied and saved.*" But actually the record is not inserted into the table.
    I also disabled the col4 by setting the Enabled property to No, since while inserting new record the date have to be populated into it and it shouldnt be changed by the user. And im populating the sysdate into the new record by setting Intial Value property to *$$DATE$$*.
    And another requirement which I could not work arround here is that, the col3 also should be populated with the username of the user while inserting.
    please help me...

    Hi Sarah,
    I do not want to update the existing record. So I kept Udate Allowed to No in property palette for the items in block2.
    Do I have to do this property at block level also?
    I'm inserting a new record here.
    Edited by: Charan on Sep 19, 2011 8:48 AM

  • Record not inserting into sap table with connector framework ?

    here is the code, but record not being inserting into the table ... but same piece of code working fine while updating ... the record ...
    try {
    interaction = connection.createInteractionEx();
    IInteractionSpec interactionSpec = interaction.getInteractionSpec();
    String functionName = "Z_XYZ";
    interactionSpec.setPropertyValue("Name", functionName);
    String writingTable = "MYTABLE";
    RecordFactory rf = interaction.getRecordFactory();
    MappedRecord importParams = rf.createMappedRecord("input");
    importParams.put("ATTR1", "VALUE1");
    importParams.put("ATTR2", "VALUE2");
    IFunction function = connection.getFunctionsMetaData().getFunction(functionName);
    IStructureFactory sf = interaction.retrieveStructureFactory();
    IRecordSet table = (IRecordSet) sf.getStructure(function.getParameter(writingTable).getStructure());
    table.insertRow();
    table.setString("ATNAME", "VALUE");
    table.setString("ATWRT", "VALUE");
    importParams.put(writingTable, table);
    MappedRecord output = (MappedRecord) interaction.execute(interactionSpec, importParams);
    } catch (Exception e) {
    any idea?
    than ks
    MMK

    Hi Mohan,
    Does a creation through SE37 with the same input work?
    Yoav.

  • How Select data from four tables

    Hi Sdners,
    How select data from four or more tables please give me any suggestion. iam learner please help me
    Advance wishes
    sreenu
    Moderator Message: I've already locked 2 threads of yours. Please read the Rules of Engagement of this forum and abide by it.
    Edited by: kishan P on Sep 5, 2010 9:12 PM

    I think there was a sample of using CURSOR in the XSQL docs.Check section:
    Using the CURSOR Operator for Nested Rowsets
    null

Maybe you are looking for

  • Simple button to play symbol....?

    I have a symbol "NA". It is an animated map that scales up and then back in place when closed. I have added a button to the main stage to play this symbol with the following code on the "click" event: sym.getSymbol("NA").play(); What am I missing? So

  • Help: consuming an external web service with user name token

    Hello Together! I need to consume an external web service secured with WS-UserNameToken. The way, how did I do it: 1. I generated a web service consumer (proxy) in SE80 from the wsdl file 2. I created logical port for the consumer in SOAMANAGER 3. I

  • How to upload a PDF file in iCloud

    Hi everybody, I would like to know how to upload a PDF file in iCloud. I want to share some PDF files with my iPhone, MacBook and iMac. Thank you a lot!

  • Reviewing Slides On Wrong Answer?

    When in questions is there a way of reviewing previous slides when a wrong answer is given; for example: A standard presentation of 50 slides is viewed and then candidates are taken to the question slides If question 1 is answered correctly it moves

  • Connecting iPhone 3G to Macbook with dead battery

    Battery died. Using Magcord for now. iPhone won't connect to download music. Is a new battery necessary to do that??