Find out duplicate rows in a flat file before using sqlldr

Hello i want to import via sqlldr from a flat file to a table inside my data base. My flat file has unfortunately some duplicate copies inside. So, i defined my upload table with two primary keys- date and time(and sometimes there are more than one row with the same time and date inside the flat file). These primaries are important to me because i want to use them for later tables and i can't use the direct path and parallel method by using primaries.
Is there any tool which can find duplicate copies before i use sqlldr. But, the special case here is, that the rows not really duplicated but date and time rows twice. And for my interest it isn't necessary whether there are different values in the second row of the same date and time. The file contains data which is monitored every second and that's enough.
It would be nice if someone could help me
cheers

I simply upload from sqlldr to staging tables first.
The staging tables allow duplicates then I do what I need to do in regards to duplicates or missing data during the transfer from the staging tables to the real tables.
The staging tables are also global temporary tables so I don't have to worry about cleaning them up or multiple sessions trampling each other.
I have also used an external table on the datafile instead of sqlldr, this allows me to get rid of the staging table. But that is only good for very small datasets in the file being loaded.

Similar Messages

  • Is there a way to find out whether all of my music files have been imported to iTunes?

    Is there a way to find  out whether all of my music files on the computer have been imported to iTunes?
    I want to consolidate my music files to tidy it all up into one place, and then delete the originals. I don't want to delete any music until I know that it's all been imported.
    Thanks.

    Try iTunes Folder Watch and point it all the places you store media. When you're happy everything has been imported you can consolidate.
    See also this post on Duplicates which includes a link to a backup tip.
    tt2

  • ADF table: How to find out which rows were modified by the user

    Hi,
    I am using ADF table to display data that can be modified by the user (ReadOnly = false in the input text). I need to find out which rows were modified by the user, and only update (persist) those rows in the database.
    Is there an easy way to find out which rows were modified by the user.
    I am NOT particularly concerned with which columns were modified as I would update the entire row (if any data in that row is modified).
    I am using Oracle JDeveloper version 11.1.1.3.
    Thanks,
    Ash K

    Thanks Timo.
    I am using ADF Data Control (which are created from Web Service proxy) to display/update the data.
    In the backing bean, the operation that is invoked at the time persisting data is as follows:
    public void commitOperation {
    BindingContainer bindings = getBindings();
    OperationBinding operationBinding = bindings.getOperationBinding("operationA");
    DCIteratorBinding dciter = (DCIteratorBinding)bindings.get("someIterator");
    for(int i=0; i<dciter.getViewObject().getEstimatedRowCount(); i++)
    Row row=dciter.getRowAtRangeIndex(i);
    SomeObject someObject= new SomeObject ();
    someObject.setAttr1(row.getAttribute("Attrbute1").toString());
    someObject.setAttr2(row.getAttribute("Attribute2").toString());
    objectList.add(someObject);
    operationBinding.getParamsMap().put("param1", objectList);
    ResultObject result = (ResultObject) operationBinding.execute();
    Inside the for loop (or any other place in the backing bean), I would like to figure out if the row was modified by the user. Please let me know if how to get that. If ADF provides this automatically, that would be GREAT.
    Thanks,
    Ash K

  • How to find out the rows inserted between a time period.

    Hi,
    Please help me to solve this.
    Table - emp.
    Colmns - empno(Primary Key),ename, mgr
    How to find out the rows inserted between a time period.
    For eg:- Between 02-Oct-2006 1 PM and 03-Oct-2006 2 PM.
    regards,
    Mathew.

    Hi,
    Maybe work:
    For each row, ORA_ROWSCN returns the conservative upper bound system change number (SCN) of the most recent change to the row. This pseudocolumn is useful for determining approximately when a row was last updated. It is not absolutely precise, because Oracle tracks SCNs by transaction committed for the block in which the row resides
    e.g.:
    SGMS@ORACLE10> create table test(cod number);
    Table created.
    SGMS@ORACLE10> insert into test values (1);
    1 row created.
    SGMS@ORACLE10> insert into test values (2);
    1 row created.
    SGMS@ORACLE10> commit;
    Commit complete.
    SGMS@ORACLE10> insert into test values (3);
    1 row created.
    SGMS@ORACLE10> commit;
    Commit complete.
    SGMS@ORACLE10> select SCN_TO_TIMESTAMP(ora_rowscn),ora_rowscn,cod from test;
    SCN_TO_TIMESTAMP(ORA_ROWSCN)       ORA_ROWSCN        COD
    06/11/06 08:56:56,000000000         727707205          1
    06/11/06 08:56:56,000000000         727707205          2
    06/11/06 08:57:05,000000000         727707210          3Cheers

  • How to find out when a folder or a file has been renamed?

    I need to find out when a folder and some files have been renamed.
    This happened when Mac OS X was 10.6 (Leopard Snow), now the Max OS X is Lion.
    I use standard HFS+ filesystem format.
    In Lion apparently there is a file property "Date Added" that change if a file is renamed, but the same property seems to be unused (e.g. -/-) before Lion.
    I have Time Machine backups that help me to know that rename happened within a certain week, but I was looking for something more accurate (at least day accuracy).
    If file system metadata can't help, I'm wondering if there is some log that track the first time spotlight indexed a specific file.
    Thank you
    /Andrea

    Maybe you could keep tabs of the file size. If the file size grows (which it will do each time the writing process flushes that files output buffer), then the file was written to.
    What I'd like to know is if there is some way to monitor the entire drive for changes and respond to stuff like that. I know that the Win32 API provides services that monitor the file system for changes, but I don't know if other OS's do... or if Java provides an interface to those features.

  • What is the best way to export the data out of BW into a flat file on the S

    Hi All,
    We are BW 7.01 (EHP 1, Service Pack Level 7).
    As part of our BW project scope for our current release, we will be developing certain reports in BW, and for certain reports, the existing legacy reporting system based out of MS Access and the old version of Business Objects Release 2 would be used, with the needed data supplied from the BW system.
    What is the best way to export the data out of BW into a flat file on the Server on regular intervals using a process chain?
    Thanks in advance,
    - Shashi

    Hello Shashi,
    some comments:
    1) An "open hub license" is required for all processes that extract data from BW to a non-SAP system (including APD). Please check with your SAP Account Executive for details.
    2) The limitation of 16 key fields is only valid when using open hub for extracting to a DB table. There's no such limitation when writing files.
    3) Open hub is the recommended solution since it's the easiest to implement, no programming is required, and you don't have to worry much about scaling with higher data volumes (APD and CRM BAPI are quite different in all of these aspects).
    For completeness, here's the most recent documentation which also lists other options:
    http://help.sap.com/saphelp_nw73/helpdata/en/0a/0212b4335542a5ae2ecf9a51fbfc96/frameset.htm
    Regards,
    Marc
    SAP Customer Solution Adoption (CSA)

  • ALV event ON_CLICK - find out selected row?

    I have an ALV table and in one of the columns I display a linkToAction in each cell.
    When the link is clicked, i go to the ON_CLICK event handler and try to find out, which row is selected.
    For this, I can take the attribute "INDEX" of the parameter "R_PARAM", and read the context with the given index.
    METHOD alv_event_on_click .
       DATA lo_nd_my_node    TYPE REF TO if_wd_context_node.
       DATA lt_my_table          TYPE wd_this->elements_my_table.
       DATA ls_my_structure    TYPE wd_this->element_my_struture.
    *  click on prduct standard name link
       IF r_param->attribute = 'NAME_OF_THE_COLUMN'.
           lo_nd_my_node = wd_context->get_child_node( name = wd_this->wdctx_my_node ).
           lo_nd_my_node->get_static_attributes_table( IMPORTING table = lt_my_table ). "this is not sorted!!!!
           lo_nd_my_node->set_lead_selection_index( index = r_param->index ).
           READ TABLE lt_my_table INTO ls_my_structure INDEX r_param->index. "lt_my_table table might be sorted differently!!!!
           IF sy-subrc = 0.
             wd_this->do_something_with( ls_my_strcture ).
           ENDIF.
       ENDIF.
    ENDMETHOD.
    However, this does not work when I sort the ALV before!
    For a strange reason, the table that I get from the context is not sorted the same way as the data in the ALV.
    What can I do?

    Hi Daniel,
    I don't know if the external mapping messes things up.
    Perhaps you could try the following:
    lo_nd_my_node->get_static_attributes(
      EXPORTING
        index                = r_param->index
      IMPORTING
        static_attributes = ls_my_structure ).
    It could be that passing the data towards your internal table doesn't take over the sort.
    If you try debugging and check out the contents of the node, are they sorted correctly or not?
    Regards and good luck.
    Jos

  • How can i find out how many users can login as sysdba using password file

    can any one please tell me how can i find out how many users can login as sysdba using password file
    please reply
    asif

    SYS@db102 SQL> select * from v$pwfile_users;
    USERNAME                       SYSDB SYSOP
    SYS                            TRUE  TRUE
    SYS@db102 SQL>                                  

  • How to write the query to find the duplicate rows?

    Hi,
    How will I find the duplicate rows in the table?
    I have a table called ITEM
    Select item_no,mfg_id from item
    Item Mfg_ID
    178 06278
    178 06589
    238 5489
    289 7895
    289 7596
    Now I want to find only the duplicate rows i.e.,178 & 289 is coming twice.
    How will get the all duplicate rows in the table?

    nihar2 wrote:
    Convert to
    SELECT it.item,
         im.mfg_item,
         im.manufacturer,
         im.description
    FROM (SELECT it.item,
         im.mfg_item,
         im.manufacturer,
         im.description,
         count(it.item) over(partition by it.item order by it.item) cnt
         FROM item it,item_mfg im
         WHERE it.u##item=im.u##item)
    WHERE cnt > 1
    But I tried It show not valid identifier.You can't use the table aliases from the sub-query...
    SELECT item,
           mfg_item,
           manufacturer,
           description 
    FROM (SELECT it.item,
           im.mfg_item,
           im.manufacturer,
           im.description,
           count(it.item) over(partition by it.item order by it.item) cnt
      FROM item it,item_mfg im
      WHERE it.u##item=im.u##item)
    WHERE cnt > 1

  • How to find out if SQL execution plan is changed proactively using job/grid

    Hello,
    Can you help me on How to find out if SQL execution plan is changed proactively using job/grid control?
    Thank you..
    -

    The answers so far are supposed to show ways how to see that a plan changed after the fact - that is not really proactive but that was the question.
    A way to see a plan change proactively would be to create a SQL Plan Baseline for the SQL statement with the 'good old' plan and then watch DBA_SQL_PLAN_BASELINES for new rows with that SQL_HANDLE which would indicate that a new execution plan was computed - although not yet used.
    Kind regards
    Uwe Hesse
    "Don't believe it, test it!"
    http://uhesse.com

  • Is there any way to find out wn was a table deleted apart from using audit

    Is there any way to find out wn was a table deleted apart from using audit
    Oracle version 9.2.0.7

    SQL> SHOW USER
    USER is "SCOTT"
    SQL> CREATE TABLE test_delete AS SELECT * FROM emp;
    Table created.
    SQL> DELETE FROM test_delete;
    14 rows deleted.
    SQL> COMMIT;
    Commit complete.
    SQL> conn sys@orclsb as sysdba
    Enter password: ******
    Connected.
    SQL> ALTER SESSION SET nls_date_format='DD-MON-RRRR HH24:MI:SS';
    Session altered.
    SQL> BEGIN
      2     DBMS_LOGMNR.START_LOGMNR (
      3           starttime => '02-FEB-2011 17:51:00',
      4           endtime   => '02-FEB-2011 17:54:00',
      5           options   => dbms_logmnr.DICT_FROM_ONLINE_CATALOG+
      6                        dbms_logmnr.continuous_mine +
      7                        dbms_logmnr.no_sql_delimiter +
      8                        dbms_logmnr.print_pretty_sql
      9                                  );
    10  END;
    11  /
    PL/SQL procedure successfully completed.
    SQL> set line 10000
    SQL> SELECT scn ,timestamp
      2         ,commit_timestamp, username
      3  FROM v$logmnr_contents
      4  WHERE
      5   username = 'SCOTT'
      6  AND  TABLE_NAME='TEST_DELETE';
           SCN TIMESTAMP            COMMIT_TIMESTAMP     USERNAME
       2309736 02-FEB-2011 17:53:20                      SCOTT
       2309739 02-FEB-2011 17:53:20                      SCOTT
       2309769 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
           SCN TIMESTAMP            COMMIT_TIMESTAMP     USERNAME
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
       2309774 02-FEB-2011 17:53:59                      SCOTT
    17 rows selected.
    SQL>

  • Process chains from the flat file by using filezilla client version in  BI

    Hi experts,
    please let me know how to create the process chains from  flat file by using filezilla client version.
    so far, I didn't work with file zilla FTP. can anybody give detailed step by step procedure to find the flat files and download them and creating process chains from that flat file.
    Thanks & Regards,
    Babu..

    Hi,
    Check these:----
    Process chain configuration for Flat file loading
    http://wiki.sdn.sap.com/wiki/display/BI/Howtowriteroutinetofetchcurrentday%27sfilename
    Regards,
    Suman

  • How can I find out the screen size of the users moniter using the Acrobat SDK?

    How can I find out the screen size of the users moniter using the Acrobat SDK? I need to know how much sreen real estate that is available on the users moniter. Is there some call that I can make from the SDK to discover the maximun X and Y coordinates?
    Thanks,
    Gregory

    Currently, I am testing on multiple moniters and it is defaulting to the moniter designated as the #1 moniter. For our purposes, this is acceptable. Once the two documents have loaded, the user can move and re-size at will.
    Gregory

  • Export table data in a flat file without using FL

    Hi,
    I am looking for options where I can export table data into a flat file without using FL(File Layout) i.e., by using App Engine only.
    Please share your experience if you did anything as this
    Thanks

    A simple way to export any record (table/view) to an csv fiel, is to create a rowset and loop through all record fields, like below example code
    Local Rowset &RS;
    Local Record &Rec;
    Local File &MYFILE;
    Local string &FileName, &strRecName, &Line, &Seperator, &Value;
    Local number &numRow, &numField;
    &FileName = "c:\temp\test.csv";
    &strRecName = "PSOPRDEFN";
    &Seperator = ";";
    &RS = CreateRowset(@("Record." | &strRecName));
    &RS.Fill();
    &MYFILE = GetFile(&FileName, "W", %FilePath_Absolute);
    If &MYFILE.IsOpen Then
       For &numRow = 1 To &RS.ActiveRowCount
          &Rec = &RS(&numRow).GetRecord(@("RECORD." | &strRecName));
          For &numField = 1 To &Rec.FieldCount
             &Value = String(&Rec.GetField(&numField).Value);
             If &numField = 1 Then
                &Line = &Value;
             Else
                &Line = &Line | &Seperator | &Value;
             End-If;
          End-For;
          &MYFILE.WriteLine(&Line);
       End-For;
    End-If;
    &MYFILE.Close(); You can of course create an application class for generic calling this piece of code.
    Hope it helps.
    Note:
    Do not come complaining to me on performance issues ;)

  • I need to find out how much wifi data my apps are using. I have a very limited amount of wifi data, and I am exceeding my monthly allowance. Apparently, even apps I think are not open are sending/receiving data through the wifi and using up my allowance.

    I need to find out how much wifi data my apps are using. I am on a very limited amount of WiFi data each month, which I am regularly exceeding. I have been told to work out which of my apps is using the data. Also, I think I have closed an app by double clicking the home button, then swiping the app up - is this the way to close it, or will it still be sending/receiving data?

    Go into your Settings : General : and turn off background refresh for your apps.  In Settings : Mail  turn Fetch new data to OFF and Load Remote Images to OFF.  This will mean that Mail will only check for messages when you actually use it, and all your advertising junk mail won't have all the images in it.
    Turn off push notifications every chance you get.
    Make sure you are actually quitting apps:  to quit apps press the Home button twice and you should see a bunch of smaller screen images for every open app.  To quit the app swipe from the screen image (not the icon) upward off the top of the iPad.  You can swipe left and right to see more open apps, but there must be no left-right movement on the screen when you swipe upward to close the app.
    Turn off your internet connection when you do not need it.  The easiest way to do this is to swipe up from the bottom of you screen to get the control centre, and then touch the airplane to turn on airplane mode.  You can repeat this sequence to turn it back on again when you need it.  Most especially turn airplane mode on whenever you are sleeping your iPad for long periods.  This will save battery life too.  OR actually turn your iPad off - which means holding the power key down for several seconds until the red swipe bar appears, and then swipe to turn it off.  If you go this route, note that it will take longer to turn on then it takes to wake from sleep.

Maybe you are looking for