Duplicate records in a collection

Hi Experts,
Just now I've seen a thread related to finding duplicate records in a collection. I understand that it is not advisable to sort/filter data in a collection.
(https://forums.oracle.com/thread/2584168)
Just for curiosity I tried to display duplicate records in a collection. Please Please .. this is just for practice purpose only. Below is the rough code which I wrote.
I'm aware of one way - can be handled effectively by passing data into a global temporary table and display the duplicate/unique records.
Can you please let me know if there is any other efficient wayto do this.
declare
  type emp_rec is record ( ename varchar2(40), empno number);
  l_emp_rec emp_rec; 
  type emp_tab is table of l_emp_rec%type index by binary_integer;
  l_emp_tab emp_tab;
  l_dup_tab emp_tab;
  l_cnt number;
  n number :=1;
begin
-- Assigning values to Associative array
  l_emp_tab(1).ename := 'suri';
  l_emp_tab(1).empno := 1;
  l_emp_tab(2).ename := 'surya';
  l_emp_tab(2).empno := 2;
  l_emp_tab(3).ename := 'suri';
  l_emp_tab(3).empno := 1;
-- Comparing collection for duplicate records
for i in l_emp_tab.first..l_emp_tab.last
loop
    l_cnt :=0;  
for j in l_emp_tab.first..l_emp_tab.last 
    loop      
       if l_emp_tab(i).empno  =  l_emp_tab(j).empno and l_emp_tab(i).ename  =  l_emp_tab(j).ename then
           l_cnt := l_cnt+1;          
               if l_cnt >=2 then
                  l_dup_tab(n):= l_emp_tab(i);
               end if;
       end if;                   
    end loop;  
end loop;
-- Displaying duplicate records
for i in l_dup_tab.first..l_dup_tab.last
loop
   dbms_output.put_line(l_dup_tab(i).ename||'  '||l_dup_tab(i).empno);
end loop;
end;
Cheers,
Suri

Dunno if this is either easier or more efficient but it is different.  The biggest disadvantage to this technique is that you have extraneous database objects (a table) to keep track of.  The advantage is that you can use SQL to perform the difference checks easily.
Create 2 global temporary tables with the structure you need, load them, and use set operators (UNION [ALL], INTERSECT, MINUS) to find the differences.  Or, create 1 GTT with an extra column identifying the set and use the extra column to identify the set records you need.

Similar Messages

  • Delete duplicate records

    Hi everyone,
    I am writing a statement to delete duplicate records in my data.
    For example:
    select identnr,datum,klokin,klokuit,count(datum)
    from fus_timesheets
    group by identnr,datum,klokin,klokuit
    having count(datum) > 1; Above code gives following result:
    IDENTNR                DATUM                     KLOKIN     KLOKUIT    COUNT(DATUM)          
    10376                  14/09/2009                                                     2                     
    10376                  16/09/2009                                                     3                     
    10376                  15/09/2009                                                    16    What i want to do is delete duplicate records, meaning that count(datum) will have a value of 1.
    I have written following code to delete duplicate records :
    declare
    type tst_type is record
        (identnr number
        ,datum varchar2(15)
        ,klokin varchar2(10)
        ,klokuit varchar2(10)
        ,aantal  number(3));
        type type_coll_tst
        is table of tst_type;
        t_tst type_coll_tst;
    begin
    select identnr,datum,klokin,klokuit,count(datum)
    bulk collect into t_tst
    from fus_timesheets
    group by identnr,datum,klokin,klokuit
    having count(datum) > 1;
    for i in 1..t_tst.count loop
    dbms_output.put_line(t_tst(i).identnr ||' '|| t_tst(i).datum||' '||t_tst(i).aantal);
    end loop;
    for i in 1..t_tst.count loop
    delete from fus_timesheets
    where identnr = t_tst(i).identnr
    and klokin = t_tst(i).klokin
    and klokuit = t_tst(i).klokuit
    and datum = t_tst(i).datum
    and rownum < t_tst(i).aantal;
    end loop;
    end;My delete statement is not working good.
    Can someone please help me with the delete part?
    Thanks,
    Diana

    Manage correctly null values:
    for i in 1..t_tst.count loop
    delete from fus_timesheets
    where identnr = t_tst(i).identnr
    and (klokin = t_tst(i).klokin or (t_tst(i).klokin is null and klokin is null))
    and (klokuit = t_tst(i).klokuit or (t_tst(i).klokuit is null and klokuit is null))
    and datum = t_tst(i).datum
    and rownum < t_tst(i).aantal;
    end loop;Max

  • Check duplicate record in array

    suppose i have an array
    int a[]={1,2,3,4,4,5,6,7,7};
    how should i check duplicate record and remove them?
    thanks...

    Write yourself some pseudo-code:
    // Loop though all the elements in the array from 0 to length-2.
    // For each element i, loop through following elements from i+1 to length-1
    // if the following element is equal to the current one, do something with it.
    "do something" can mean either one of two things:
    (1) Remove it (VERY messy) or
    (2) Create a new array and only add in the non-duplicate elements.
    The easiest thing of all would be to create a List and then create a Set from that. Why code all this logic when somebody has already done it for you? Look at the java.util.Arrays and java.util.Collections classes. Use what's been given to you.
    %

  • Duplicate records returned by db adapter

    Hi,
    We have a DB Adapter design as following.
    From a J2EE application, an invoking procedure at the adapter is triggered. This invokes the DB adapter which inturn invokes the implemetned procedure which
    returns a VARRAY.
    J2EE application is deployed on the AS 10g Protal side.
    The implemented procedure further invokes a legacy procedure which returns a Refcursor.Inside this implemetned procedure, the cursor is fetched into a
    Recortype and each element of the record type is added to an Object (of type MPLUS_MPLUS_BankGurDet_OAI_V1). Then each of this object is added to a VARRAY and this VARRAY is returned to the invoking procedure.
    The problem we face is that, in the returned the varray, there is always a duplicate of the last recored. That is if we have , say 5 records, then when we manipulate the VARRAY, there will 6 recored and the sixth record will be a duplicate of the 5th one.
    Why is this happening. I have attached the code I used and log file. This is happening for all cases where VARRAY is returned. I have attached only one set
    here.The log file attached clearly show the duplicate records.
    -- All these is defined at the Legacy system
    -- Following is a object definition used by the implemented procedure to return a collection of objects.
    TYPE MPLUS_MPLUS_BankGurDet_OAI_V1 IS OBJECT (
    AG_CODE VARCHAR2(1000),
    AMT_NO NUMBER,
    BANK VARCHAR2(1000),
    VAL_FROM DATE,
    VAL_TO DATE,
    GUAR_NO VARCHAR2(1000)
    -- Following is the defintion of Varray returned by the implemented procedure.
    TYPE MPLUS_MPLUS_BankGurDet_OAI_Arr IS VARRAY(1000) OF MPLUS_MPLUS_BankGurDet_OAI_V1;
    -- This is the implemented procedure called by adapter. This returns a Varray of the above object.(MPLUS_MPLUS_BankGurDet_OAI_V1).
    PACKAGE Mplus AS
    TYPE MY_TYPE IS REF CURSOR;
    PROCEDURE imp_bankGurDet_OAI_V1(
    i_CUSTOMER_CODE IN LONG,
    i_CURRENT_DATE IN DATE,
    o_RESULT OUT MPLUS_MPLUS_BankGurDet_OAI_Arr
    END Mplus ;
    PACKAGE BODY Mplus AS
    PROCEDURE imp_bankGurDet_OAI_V1(
    i_CUSTOMER_CODE IN LONG,
    i_CURRENT_DATE IN DATE,
    o_RESULT OUT MPLUS_MPLUS_BankGurDet_OAI_Arr
    AS
    Type BANK IS RECORD(agencyname VARCHAR2(1000),Amout NUMBER,BankName VARCHAR2(1000),validfrom date , validto date,guarenteeno varchar2(1000));
    RECS MY_TYPE; -- Ref cursor
    BANKDETAILS_REC BANK ;
    BANKDETAILS_OBJ MPLUS_MPLUS_BANKGURDET_OAI_V1;
    i Number;
    dummy number;
    BEGIN
    WF_MP_ADVT.SP_ADV_AGE_BANK_GRTY(i_CUSTOMER_CODE,i_CURRENT_DATE,RECS);
    dummy:= 0;
    BANKDETAILS_OBJ:=new MPLUS_MPLUS_BankGurDet_OAI_V1('',0,'',sysdate,sysdate,'');
    o_RESULT:=MPLUS_MPLUS_BankGurDet_OAI_Arr();
    i:=1;
    LOOP
    fetch RECS INTO BANKDETAILS_REC;
    BANKDETAILS_OBJ.AG_CODE:=BANKDETAILS_REC.agencyname;
    BANKDETAILS_OBJ.AMT_NO:=BANKDETAILS_REC.Amout;
    BANKDETAILS_OBJ.BANK:=BANKDETAILS_REC.BankName;
    BANKDETAILS_OBJ.VAL_FROM:=to_date(BANKDETAILS_REC.validfrom,'dd-mon-yyyy');
    BANKDETAILS_OBJ.VAL_TO:=to_date(BANKDETAILS_REC.validto,'dd-mon-yyyy');
    BANKDETAILS_OBJ.GUAR_NO:=BANKDETAILS_REC.guarenteeno;
    o_RESULT.EXTEND;
    o_RESULT(i):=BANKDETAILS_OBJ;
    i:=i+1;
    EXIT WHEN RECS%NOTFOUND;
    END LOOP;
    END imp_bankGurDet_OAI_V1;
    END Mplus ;
    -- The following is a legacy procedure which return a refcursor which is iterated by the implemented procedure to create object and then this object is added to VARRAY in the.
    PACKAGE WF_MP_ADVT AS
    PROCEDURE SP_ADV_AGE_BANK_GRTY(
              Customer_Code      IN     CHAR,
    curdate in date,
              RESULT          OUT     Record_Type);
    END;
    PACKAGE BODY WF_MP_ADVT AS
    PROCEDURE SP_ADV_AGE_BANK_GRTY(
              Customer_Code      IN     CHAR,
    curdate in date,
              RESULT          OUT     Record_Type) IS
    BEGIN
    -- OPEN RESULT FOR SELECT AG_CODE,AMT_NO,BANK,to_char(VAL_FROM,'dd-mm-yyyy'),to_char(VAL_TO,'dd-mm-yyyy'),GUAR_NO FROM WF_MP_ADVT_BANKGAR WHERE AG_CODE=CUSTOMER_CODE;
    --OPEN RESULT FOR SELECT * FROM WF_MP_ADVT_BANKGAR WHERE AG_CODE=CUSTOMER_CODE;
    OPEN RESULT FOR SELECT AG_CODE,AMT_NO,BANK,to_char(VAL_FROM,'dd-mon-yyyy'),to_char(VAL_TO,'dd-mon-yyyy'),GUAR_NO FROM WF_MP_ADVT_BANKGAR WHERE AG_CODE=CUSTOMER_CODE;
    -- null;
    END SP_ADV_AGE_BANK_GRTY;
    END;
    The log file is as following
    Mplus.bankGurDet:OAI/V1,OAI/V1,false,1
    CUSTOMER_CODE: 1
    CURRENT_DATE: Fri Oct 29 00:00:00 GMT+05:30 2004
    Tue Nov 02 14:55:10 GMT+05:30 2004: InMessageTransformer: got a message for processing.
    Mplus.bankGurDet:OAI/V1,OAI/V1,false,1
    CUSTOMER_CODE: 1
    CURRENT_DATE: Fri Oct 29 00:00:00 GMT+05:30 2004
    Tue Nov 02 14:55:11 GMT+05:30 2004: Inbound Transform Engine: beginning to transform message.
    Tue Nov 02 14:55:11 GMT+05:30 2004: MessageTransformer: Successfully created destination message object with necessary header information but no attribute objects yet.
    Tue Nov 02 14:55:11 GMT+05:30 2004: InMessageTransformer: got a message for processing.
    Mplus.bankGurDet:OAI/V1,OAI/V1,false,1
    CUSTOMER_CODE: 1
    CURRENT_DATE: Fri Oct 29 00:00:00 GMT+05:30 2004
    Tue Nov 02 14:56:14 GMT+05:30 2004: db_bridge_writer_1 wrote the message to the database successfully.
    Tue Nov 02 14:56:14 GMT+05:30 2004: Agent: sending reply message.
    Mplus.bankGurDet:OAI/V1,OAI/V1,true,2
    RESULT[0]
    AG_CODE: 1
    AMT_NO: 200000.0
    BANK: STATE BANK OF INDIA
    VAL_FROM: 0004-04-01
    VAL_TO: 0005-03-31
    GUAR_NO: SBI01
    RESULT[1]
    AG_CODE: 1
    AMT_NO: 200000.0
    BANK: HDFC BANK
    VAL_FROM: 0004-04-01
    VAL_TO: 0005-03-31
    GUAR_NO: SBI01
    RESULT[2]
    AG_CODE: 1
    AMT_NO: 200000.0
    BANK: HDFC BANK
    VAL_FROM: 0004-04-01
    VAL_TO: 0005-03-31
    GUAR_NO: SBI01

    Vinod,
    As far as I can see the problem with duplicating the last record is a result of the EXIT condition in the loop in procedure 'imp_bankGurDet_OAI_V1'. Your loop looks as follows:
    LOOP
    FETCH recs INTO bankdetails_rec;
    bankdetails_obj.ag_code := bankdetails_rec.agencyname;
    bankdetails_obj.amt_no := bankdetails_rec.amout;
    bankdetails_obj.bank := bankdetails_rec.bankname;
    bankdetails_obj.val_from := TO_DATE(bankdetails_rec.validfrom,'DD-MON-YYYY');
    bankdetails_obj.val_to := TO_DATE(bankdetails_rec.validto,'DD-MON-YYYY');
    bankdetails_obj.guar_no := bankdetails_rec.guarenteeno;
    o_result.EXTEND;
    o_result(i):= bankdetails_obj;
    i:=i+1;
    EXIT WHEN recs%NOTFOUND;
    END LOOP;
    The problem is that checking for recs%NOTFOUND at the end of the loop results in going through the loop one more time even though you can't fetch another row from the recs cursor. The solution is to put the EXIT condition right after the FETCH statement. You now exit the loop if you can't fetch another row without assigning the last fetched record to bankdetails_obj again:
    LOOP
    FETCH recs INTO bankdetails_rec;
    EXIT WHEN recs%NOTFOUND;
    bankdetails_obj.ag_code := bankdetails_rec.agencyname;
    bankdetails_obj.amt_no := bankdetails_rec.amout;
    bankdetails_obj.bank := bankdetails_rec.bankname;
    bankdetails_obj.val_from := TO_DATE(bankdetails_rec.validfrom,'DD-MON-YYYY');
    bankdetails_obj.val_to := TO_DATE(bankdetails_rec.validto,'DD-MON-YYYY');
    bankdetails_obj.guar_no := bankdetails_rec.guarenteeno;
    o_result.EXTEND;
    o_result(i):= bankdetails_obj;
    i:=i+1;
    END LOOP;
    CLOSE recs;
    You also might want to consider to close the ref cursor after exiting the loop... 'too many open cursors' is not a good exception to get. ;-)
    Hope this is helpful.
    Thanks,
    Markus

  • Help....Duplicate records found in loading customer attributes!!!!

    Hi ALL,
    we have a full update master data with attributes job that  is failed with this error message: (note the processing is PSA and then Data targets)
    1 duplicate record found. 66 recordings used in table /BIC/PZMKE_CUST RSDMD 199     
    1 duplicate record found. 0 recordings used in table /BIC/PZTSA_CUST RSDMD 199     
    1 duplicate record found. 0 recordings used in table /BIC/PZTSA_CUST RSDMD 199     
    1 duplicate record found. 0 recordings used in table /BIC/XZTSA_CUST RSDMD 199     
    1 duplicate record found. 66 recordings used in table /BIC/XZMKE_CUST RSDMD 199     
    our datasource is 0CUSTOMER_ATTR i tried to use the transaction rso2 to get more information about this datasource to know where i can find the original data in R/3 but when i execute i got this message:
    DataSource 0CUSTOMER_ATTR  is extracted using functional module MDEX_CUSTOMER_MD
    Can you help me please what should i do to correct and reload or to find the duplicate data to tell the person on R/3 system to delete the duplicate.
    Thanks
    Bilal

    Hi Bilal,
    Could you try the following pls.
    Only PSA and Update Subsequently into Datatargets in the Processing tab.
    In Update tab use Error Handling , mark "Valid Records Update, Reporting Possible(request green).
    If my assumptions are right, this should collect the duplicate records in another request, from where you could get the data that needs to be corrected.
    To resolve this issue, just load using ONLY PSA and Update Subsequently into Datatargets with Ignore duplicate Records checked.
    Cheers,
    Praveen.

  • Duplicate records for Dynamic Query in WLS 7

    I am getting duplicate records back when I run a dynamic query. If I run the same
    query via a finder method, I get the correct results.
    Here's the query:
    SELECT DISTINCT OBJECT(a) FROM Company AS a, b IN a.userRole WHERE b.userId = ?1
    AND a.deptId IN ('1', '2', '3')
    Company EJB maps to LU_Company (look up table for company)
    Company to UserRole is a one-to-many relationship.
    The number of duplicates is equal to the number of records I have for the same company_id
    in User_Role table.
    The ids for the object I should get back are 1, 2, 3.
    Instead I get back 1, 1, 2, 2, 2, 2, 2, 3, 3, 3
    In the database, company_id 1 occurs twice in user_role table; company_id 2 occurs
    five times.
    Any ideas of why the dynamic query has this problem whereas the same query for a
    finder method works fine?
    Thanks for your help.

    Thanks Greg, that worked!
    "Greg Nyberg" <greg.nyberg.at.objectpartners.com> wrote:
    You can supply properties on the query request, perhaps there is a
    SQL_SELECT_DISTINCT property:
    Properties p = new Properties();
    p.setProperty("GROUP_NAME", "fieldgroup");
    p.setProperty("INCLUDE_UPDATES", "true");
    p.setProperty("SQL_SELECT_DISTINCT", "true");
    Collection people = myQuery.find(
    "SELECT OBJECT(o) FROM PersonCMPEJB o WHERE o.lastName = 'Smith'", p);
    -Greg
    Check out my WebLogic 6.1 Workbook for O'Reilly EJB Third Edition
    www.amazon.com/exec/obidos/ASIN/1931822468 or www.titan-books.com
    "Mike" <[email protected]> wrote in message
    news:[email protected]...
    Matt,
    <sql-select-distinct> needs to be set in <weblogic-query> element. I amgenerating
    a dynamic query from within a local client. So I don't have a finderdefined for
    this method. (I don't want to define a finder for this method). How doyou suggest
    I can use this? Thanks again for your help.
    Matthew Shinn <[email protected]> wrote:
    Hi Mike,
    I just filed a bug report for this (CR079471). As a work around, you
    can
    use the
    setSQLSelectDistinct flag so the database will weed out the duplicates.
    The only thing to
    look out for when using this flag is that Oracle will not allow the use
    of 'SELECT DISTINCT'
    in conjunction with a 'FOR UPDATE' clause, thus,<sql-select-distinct>True
    CANNOT be used if
    any Bean in the calling chain has a method with <transaction-isolation>
    set to
    <isolation-level>TRANSACTION_READ_COMMITTED_FOR_UPDATE. Sorry for theinconvenience.
    - Matt
    Mike wrote:
    I am getting duplicate records back when I run a dynamic query. If
    I
    run
    the same
    query via a finder method, I get the correct results.
    Here's the query:
    SELECT DISTINCT OBJECT(a) FROM Company AS a, b IN a.userRole WHERE
    b.userId
    = ?1
    AND a.deptId IN ('1', '2', '3')
    Company EJB maps to LU_Company (look up table for company)
    Company to UserRole is a one-to-many relationship.
    The number of duplicates is equal to the number of records I have forthe same company_id
    in User_Role table.
    The ids for the object I should get back are 1, 2, 3.
    Instead I get back 1, 1, 2, 2, 2, 2, 2, 3, 3, 3
    In the database, company_id 1 occurs twice in user_role table;
    company_id
    2 occurs
    five times.
    Any ideas of why the dynamic query has this problem whereas the same
    query
    for a
    finder method works fine?
    Thanks for your help.

  • How to suppress duplicate records in rtf templates

    Hi All,
    I am facing issue with payment reason comments in check template.
    we are displaying payment reason comments. Now the issue is while making batch payment we are getting multiple payment reason comments from multiple invoices with the same name and it doesn't looks good. You can see payment reason comments under tail number text field in the template.
    If you provide any xml syntax to suppress duplicate records for showing distinct payment reason comments.
    Attached screen shot, template and xml file for your reference.
    Thanks,
    Sagar.

    I have CRXI, so the instructions are for this release
    you can create a formula, I called it cust_Matches
    if = previous () then 'true' else 'false'
    IN your GH2 section, right click the field, select format field, select the common tab (far left at the top)
    Select the x/2 to the right of Supress  in the formula field type in
    {@Cust_Matches} = 'true'
    Now every time the {@Cust_Matches} is true, the CustID should be supressed,
    do the same with the other fields you wish to hide.  Ie Address, City, etc.

  • Duplicate records to DSO

    Hello Friends,
    we have an issue with the Duplicate records in to DSO let me enplain the senarion
    The Heder and Details data is loaded to saperate DSO's
    and these 2 DSO's data shuld get merged in the third one,
    the Key fields in
    DSO 1 : DWRECID, 0AC_DOC_NO
    DSO 2 : DWRECID , DWPOSNR
    DSO 3 will fetch data from these the above 2
    Key Fields are : ]
    DWTSLO,
    DWRECID,
    DWEDAT ,
    AC_DOC_NO
    DWPOSNR,
    0CUSTOMER
    Now the data shuld be merge in to a single record in the 3 rd dso
    DSO 1  do not have the DWPOSNR object in its data fields also.
    we even have start routine  data from DSO 1 to populate  some values in the result fields from dso2 ,
    Please provide if you have any inputs to merge the data record wise.
    and also give me all the posibilites or options we have to over write " apart from mappings " the data ,

    Hi,
    You should go for creating an Infoset instead of creating third DSO.
    In that DSO provide the Keys of DSOs and the Common records with those keys will be merged  in that Infoset.
    Hope It Helps.
    Regards
    Praeon

  • How to create duplicate records in end routines

    Hi
    Key fields in DSO are:
    Plant
    Storage Location
    MRP Area
    Material
    Changed Date
    Data Fields:
    Safety Stocky
    Service Level
    MRP Type
    Counter_1 (In flow Key figure)
    Counter_2 (Out flow Key Figure)
    n_ctr  (Non Cumulative Key Figure)
    For every record that comes in, we need to create a dupicate record. For the original record, we need to make the Counter_1 as 1 and Counter_2 as 0. For the duplicate record, we need to update Changed_Date to today's date and rest of the values will remain as is and update the counter_1 as 0 and counter_2 as -1. Where is the best place to write this code in DSO. IS it End
    routine?
    please let me know some bais cidea of code.

    Hi Uday,
    I have same situation like Suneel and have written your logic in End routine DSO as follows:
    DATA: l_t_duplicate_records TYPE TABLE OF TYS_TG_1,
          l_w_duplicate_record TYPE TYS_TG_1.
    LOOP AT RESULT_PACKAGE ASSIGNING <result_fields>.
        MOVE-CORRESPONDING <result_fields> TO l_w_duplicate_record.
        <result_fields>-/BIC/ZPP_ICNT = 1.
        <result_fields>-/BIC/ZPP_OCNT = 0.
        l_w_duplicate_record-CH_ON = sy-datum.
        l_w_duplicate_record-/BIC/ZPP_ICNT = 0.
        l_w_duplicate_record-/BIC/ZPP_OCNT = -1.
        APPEND l_w_duplicate_record TO  l_t_duplicate_records.
    ENDLOOP.
    APPEND LINES OF l_t_duplicate_records TO RESULT_PACKAGE.
    I am getting below error:
    Duplicate data record detected (DS ZPP_O01 , data package: 000001 , data record: 4 )     RSODSO_UPDATE     19     
    i have different requirement for date. Actually my requirement is to populate the CH_ON date as mentioned below:
    sort the records based on the key and get the latest CH_ON value with unique Plant,sloc, material combination and populate
    that CH_ON value for duplicate record.
    Please help me to resolve this issue.
    Thanks,
    Ganga

  • USE of PREVIOUS command to eliminate duplicate records in counter formula

    i'm trying to create a counter formula to count the number of documents paid over 30 days.  to do this i have to subtract the InvDate from the PayDate.   and then create a counter based on this value.  if {days to pay} is greater than 30 then 1 else 0.
    then sum the {days to pay} field to each group.   groups are company, month, and supplier.
    becuase invoices can have multiple payments and payments can have multiple invoices. there is no way around having duplicate records for the field. 
    so my counter is distorted by by the duplicate records and my percentage of payments over 30 days formula will not be accurate do to these duplicates.
    I've tried Distinct Count based on this formula  if {days to pay} is greater than 30 then . and it works except that is counts 0.00 has a distinct records so my total is off 1 for summaries with a record that (days to pay} is less than or equal to 30.
    if i subract 1 from the formula then it will be inaccurate for summaries with no records over 30 days.
    so i'm come to this.
    if Previous() do not equal
    then
      if {day to days} greater than 30
      then 1
      else 0.00
    else 0.00
    but it doesn't work.  i've sorted the detail section by
    does anyone have any knowledge or success using the PREVIOUS command in a report?
    Edited by: Fred Ebbett on Feb 11, 2010 5:41 PM

    So, you have to include all data and not just use the selection criteria 'PayDate-InvDate>30'?
    You will need to create a running total on the RPDOC ID, one for each section you need to show a count for, evaluating for your >30 day formula. 
    I don't understand why you're telling the formula to return 0.00 in your if statement.
    In order to get percentages you'll need to use the distinct count (possibly running totals again but this time no formula). Then in each section you'd need a formula that divides the two running totals.
    I may not have my head around the concept since you stated "invoices can have multiple payments and payments can have multiple invoices".  So, invoice A can have payments 1, 2 and 3.  And Payment 4 can be associated with invoice B and C?  Ugh.  Still though, you're evaluating every row of data.  If you're focus is the invoices that took longer than 30 days to be paid...I'd group on the invoice number, put the "if 'PayDate-InvDate>30' then 1 else 0" formula in the detail, do a sum on it in the group footer and base my running total on the sum being >0 to do a distinct count of invoices.
    Hope this points you in the right direction.
    Eric

  • Duplicate records problem

    Hi everyone,
    I'm having a a little difficulty resolving a problem with a repeating field causing duplication of data in a report I'm working on, and was hoping someone on here can suggest something to help!
    My report is designed to detail library issues during a particular period, categorised by the language of the item issued. My problem is that on the sql database that out library management system uses, it is possible for an item to have more than one language listed against it (some books will be in more than one language). When I list the loan records excluding the language data field, I get a list of distinct loan records. Bringing the language data into the report causes the loan record to repeat for each language associated with it, so if a book is both in English and French, it will cause the loan record to appear like this:
    LOAN RECORD NO.     LANGUAGE CODE
      123456                             ENG
      123456                             FRE
    So, although the loan only occurred once I have two instances of it in my report.
    I am only interested in the language that appears first and I can exclude duplicated records from the report page. I can also count only the distinct records to get an accurate overall total. My problem is that when I group the loan records by language code (I really need to do this as there are millions of loan records held in the database) the distinct count stops being a solution, as when placed at this group level it only excludes duplicates in the respective group level it's placed in. So my report would display something like this:
    ENG     1
    FRE      1
    A distinct count of the whole report would give the correct total of 1, but a cumulative total of the figures calculated at the language code group level would total 2, and be incorrect. I've encountered similar results when using Running Totals evaluating on a formula that excludes repeated loan record no.s from the count, but again when I group on the language code this goes out of the window.
    I need to find a way of grouping the loan records by language with a total count of loan records alongside each grouping that accurately reflects how many loans of that language took place.
    Is this possible using a calculation formula when there are repeating fields, or do I need to find a way of merging the repeating language fields into one field so that the report would appear like:
    LOAN RECORD     LANGUAGE CODE
      123456                      ENG, FRE
    Any suggestions would be greatly appreciated, as aside from this repeating language data there are quite a few other repeating database fields on the system that it would be nice to report on!
    Thanks!

    if you create a group by loan
    then create a group by language
    place the values in the group(loan id in the loan header)
    you should only see the loan id 1x.
    place the language in the language group you should only see that one time
    a group header returns the 1st value of a unique id....
    then in order to calculate avoiding the duplicates
    use manual running totals
    create a set for each summary you want- make sure each set has a different variable name
    MANUAL RUNNING TOTALS
    RESET
    The reset formula is placed in a group header report header to reset the summary to zero for each unique record it groups by.
    whileprintingrecords;
    Numbervar  X := 0;
    CALCULATION
    The calculation is placed adjacent to the field or formula that is being calculated.
    (if there are duplicate values; create a group on the field that is being calculated on. If there are not duplicate records, the detail section is used.
    whileprintingrecords;
    Numbervar  X := x + ; ( or formula)
    DISPLAY
    The display is the sum of what is being calculated. This is placed in a group, page or report footer. (generally placed in the group footer of the group header where the reset is placed.)
    whileprintingrecords;
    Numbervar  X;
    X

  • How to find out duplicate record contained in a flat file

    Hi Experts,
    For my project I have written a program for flat file upload.
    Requirement 1
    In the flat file there may be some duplicate record like:
    Field1   Field2
    11        test1
    11        test2
    12        test3
    13        test4
    Field1 is primary key.
    Can you please let me know how I can find out the duplicate record.
    Requirement 2
    The flat file contains the header row as shown above
    Field1   Field2
    How our program can skip this record and start reading / inserting records from row no 2 ie
    11        test1
    onwards.
    Thanks
    S
    FORM upload1.
    DATA : wf_title TYPE string,
    lt_filetab TYPE filetable,
    l_separator TYPE char01,
    l_action TYPE i,
    l_count TYPE i,
    ls_filetab TYPE file_table,
    wf_delemt TYPE rollname,
    wa_fieldcat TYPE lvc_s_fcat,
    tb_fieldcat TYPE lvc_t_fcat,
    rows_read TYPE i,
    p_error TYPE char01,
    l_file TYPE string.
    DATA: wf_object(30) TYPE c,
    wf_tablnm TYPE rsdchkview.
    wf_object = 'myprogram'.
    DATA i TYPE i.
    DATA:
    lr_mdmt TYPE REF TO cl_rsdmd_mdmt,
    lr_mdmtr TYPE REF TO cl_rsdmd_mdmtr,
    lt_idocstate TYPE rsarr_t_idocstate,
    lv_subrc TYPE sysubrc.
    TYPES : BEGIN OF test_struc,
    /bic/myprogram TYPE /bic/oimyprogram,
    txtmd TYPE rstxtmd,
    END OF test_struc.
    DATA : tb_assum TYPE TABLE OF /bic/pmyprogram.
    DATA: wa_ztext TYPE /bic/tmyprogram,
    myprogram_temp TYPE ziott_assum,
    wa_myprogram TYPE /bic/pmyprogram.
    DATA : test_upload TYPE STANDARD TABLE OF test_struc,
    wa2 TYPE test_struc.
    DATA : wa_test_upload TYPE test_struc,
    ztable_data TYPE TABLE OF /bic/pmyprogram,
    ztable_text TYPE TABLE OF /bic/tmyprogram,
    wa_upld_text TYPE /bic/tmyprogram,
    wa_upld_data TYPE /bic/pmyprogram,
    t_assum TYPE ziott_assum.
    DATA : wa1 LIKE test_upload.
    wf_title = text-026.
    CALL METHOD cl_gui_frontend_services=>file_open_dialog
    EXPORTING
    window_title = wf_title
    default_extension = 'txt'
    file_filter = 'Tab delimited Text Files (*.txt)'
    CHANGING
    file_table = lt_filetab
    rc = l_count
    user_action = l_action
    EXCEPTIONS
    file_open_dialog_failed = 1
    cntl_error = 2
    OTHERS = 3. "#EC NOTEXT
    IF sy-subrc 0.
    EXIT.
    ENDIF.
    LOOP AT lt_filetab INTO ls_filetab.
    l_file = ls_filetab.
    ENDLOOP.
    CHECK l_action = 0.
    IF l_file IS INITIAL.
    EXIT.
    ENDIF.
    l_separator = 'X'.
    wa_fieldcat-fieldname = 'test'.
    wa_fieldcat-dd_roll = wf_delemt.
    APPEND wa_fieldcat TO tb_fieldcat.
    CALL FUNCTION 'MESSAGES_INITIALIZE'.
    CLEAR wa_test_upload.
    Upload file from front-end (PC)
    File format is tab-delimited ASCII
    CALL FUNCTION 'GUI_UPLOAD'
    EXPORTING
    filename = l_file
    has_field_separator = l_separator
    TABLES
    data_tab = i_mara
    data_tab = test_upload
    EXCEPTIONS
    file_open_error = 1
    file_read_error = 2
    no_batch = 3
    gui_refuse_filetransfer = 4
    invalid_type = 5
    no_authority = 6
    unknown_error = 7
    bad_data_format = 8
    header_not_allowed = 9
    separator_not_allowed = 10
    header_too_long = 11
    unknown_dp_error = 12
    access_denied = 13
    dp_out_of_memory = 14
    disk_full = 15
    dp_timeout = 16
    OTHERS = 17.
    IF sy-subrc 0.
    EXIT.
    ELSE.
    CALL FUNCTION 'MESSAGES_INITIALIZE'.
    IF test_upload IS NOT INITIAL.
    DESCRIBE TABLE test_upload LINES rows_read.
    CLEAR : wa_test_upload,wa_upld_data.
    LOOP AT test_upload INTO wa_test_upload.
    CLEAR : p_error.
    rows_read = sy-tabix.
    IF wa_test_upload-/bic/myprogram IS INITIAL.
    p_error = 'X'.
    MESSAGE s153 WITH wa_test_upload-/bic/myprogram sy-tabix.
    CONTINUE.
    ELSE.
    TRANSLATE wa_test_upload-/bic/myprogram TO UPPER CASE.
    wa_upld_text-txtmd = wa_test_upload-txtmd.
    wa_upld_text-txtsh = wa_test_upload-txtmd.
    wa_upld_text-langu = sy-langu.
    wa_upld_data-chrt_accts = 'xyz1'.
    wa_upld_data-co_area = '12'.
    wa_upld_data-/bic/zxyzbcsg = 'Iy'.
    wa_upld_data-objvers = 'A'.
    wa_upld_data-changed = 'I'.
    wa_upld_data-/bic/zass_mdl = 'rrr'.
    wa_upld_data-/bic/zass_typ = 'I'.
    wa_upld_data-/bic/zdriver = 'yyy'.
    wa_upld_text-langu = sy-langu.
    MOVE-CORRESPONDING wa_test_upload TO wa_upld_data.
    MOVE-CORRESPONDING wa_test_upload TO wa_upld_text.
    APPEND wa_upld_data TO ztable_data.
    APPEND wa_upld_text TO ztable_text.
    ENDIF.
    ENDLOOP.
    DELETE ADJACENT DUPLICATES FROM ztable_data.
    DELETE ADJACENT DUPLICATES FROM ztable_text.
    IF ztable_data IS NOT INITIAL.
    CALL METHOD cl_rsdmd_mdmt=>factory
    EXPORTING
    i_chabasnm = 'myprogram'
    IMPORTING
    e_r_mdmt = lr_mdmt
    EXCEPTIONS
    invalid_iobjnm = 1
    OTHERS = 2.
    CALL FUNCTION 'MESSAGES_INITIALIZE'.
    **Lock the Infoobject to update
    CALL FUNCTION 'RSDG_IOBJ_ENQUEUE'
    EXPORTING
    i_objnm = wf_object
    i_scope = '1'
    i_msgty = rs_c_error
    EXCEPTIONS
    foreign_lock = 1
    sys_failure = 2.
    IF sy-subrc = 1.
    MESSAGE i107(zddd_rr) WITH wf_object sy-msgv2.
    EXIT.
    ELSEIF sy-subrc = 2.
    MESSAGE i108(zddd_rr) WITH wf_object.
    EXIT.
    ENDIF.
    *****Update Master Table
    IF ztable_data IS NOT INITIAL.
    CALL FUNCTION 'RSDMD_WRITE_ATTRIBUTES_TEXTS'
    EXPORTING
    i_iobjnm = 'myprogram'
    i_tabclass = 'M'
    I_T_ATTR = lt_attr
    TABLES
    i_t_table = ztable_data
    EXCEPTIONS
    attribute_name_error = 1
    iobj_not_found = 2
    generate_program_error = 3
    OTHERS = 4.
    IF sy-subrc 0.
    CALL FUNCTION 'MESSAGE_STORE'
    EXPORTING
    arbgb = 'zddd_rr'
    msgty = 'E'
    txtnr = '054'
    msgv1 = text-033
    EXCEPTIONS
    OTHERS = 3.
    MESSAGE e054(zddd_rr) WITH 'myprogram'.
    ELSE.
    CALL FUNCTION 'MESSAGE_STORE'
    EXPORTING
    arbgb = 'zddd_rr'
    msgty = 'S'
    txtnr = '053'
    msgv1 = text-033
    EXCEPTIONS
    OTHERS = 3.
    ENDIF.
    *endif.
    *****update Text Table
    IF ztable_text IS NOT INITIAL.
    CALL FUNCTION 'RSDMD_WRITE_ATTRIBUTES_TEXTS'
    EXPORTING
    i_iobjnm = 'myprogram'
    i_tabclass = 'T'
    TABLES
    i_t_table = ztable_text
    EXCEPTIONS
    attribute_name_error = 1
    iobj_not_found = 2
    generate_program_error = 3
    OTHERS = 4.
    IF sy-subrc 0.
    CALL FUNCTION 'MESSAGE_STORE'
    EXPORTING
    arbgb = 'zddd_rr'
    msgty = 'E'
    txtnr = '055'
    msgv1 = text-033
    EXCEPTIONS
    OTHERS = 3.
    ENDIF.
    ENDIF.
    ELSE.
    MESSAGE s178(zddd_rr).
    ENDIF.
    ENDIF.
    COMMIT WORK.
    CALL FUNCTION 'RSD_CHKTAB_GET_FOR_CHA_BAS'
    EXPORTING
    i_chabasnm = 'myprogram'
    IMPORTING
    e_chktab = wf_tablnm
    EXCEPTIONS
    name_error = 1.
    IF sy-subrc 0.
    MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
    WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
    ENDIF.
    ****Release locks on Infoobject
    CALL FUNCTION 'RSDG_IOBJ_DEQUEUE'
    EXPORTING
    i_objnm = 'myprogram'
    i_scope = '1'.
    ENDIF.
    ENDIF.
    PERFORM data_selection .
    PERFORM update_alv_grid_display.
    CALL FUNCTION 'MESSAGES_SHOW'.
    ENDFORM.

    Can you please let me know how I can find out the duplicate record.
    you need to split the records from flat file structure into your internal table ans use a delete ADJACENT duplicates comparing fields
    split flat_str into wa_f1 wa_f2 wa_f2 at tab_space.

  • Check for duplicate record in SQL database before doing INSERT

    Hey guys,
           This is part powershell app doing a SQL insert. BUt my question really relates to the SQL insert. I need to do a check of the database PRIOR to doing the insert to check for duplicate records and if it exists then that record needs
    to be overwritten. I'm not sure how to accomplish this task. My back end is a SQL 2000 Server. I'm piping the data into my insert statement from a powershell FileSystemWatcher app. In my scenario here if the file dumped into a directory starts with I it gets
    written to a SQL database otherwise it gets written to an Access Table. I know silly, but thats the environment im in. haha.
    Any help is appreciated.
    Thanks in Advance
    Rich T.
    #### DEFINE WATCH FOLDERS AND DEFAULT FILE EXTENSION TO WATCH FOR ####
                $cofa_folder = '\\cpsfs001\Data_pvs\TestCofA'
                $bulk_folder = '\\cpsfs001\PVS\Subsidiary\Nolwood\McWood\POD'
                $filter = '*.tif'
                $cofa = New-Object IO.FileSystemWatcher $cofa_folder, $filter -Property @{ IncludeSubdirectories = $false; EnableRaisingEvents= $true; NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite' }
                $bulk = New-Object IO.FileSystemWatcher $bulk_folder, $filter -Property @{ IncludeSubdirectories = $false; EnableRaisingEvents= $true; NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite' }
    #### CERTIFICATE OF ANALYSIS AND PACKAGE SHIPPER PROCESSING ####
                Register-ObjectEvent $cofa Created -SourceIdentifier COFA/PACKAGE -Action {
           $name = $Event.SourceEventArgs.Name
           $changeType = $Event.SourceEventArgs.ChangeType
           $timeStamp = $Event.TimeGenerated
    #### CERTIFICATE OF ANALYSIS PROCESS BEGINS ####
                $test=$name.StartsWith("I")
         if ($test -eq $true) {
                $pos = $name.IndexOf(".")
           $left=$name.substring(0,$pos)
           $pos = $left.IndexOf("L")
           $tempItem=$left.substring(0,$pos)
           $lot = $left.Substring($pos + 1)
           $item=$tempItem.Substring(1)
                Write-Host "in_item_key $item in_lot_key $lot imgfilename $name in_cofa_crtdt $timestamp"  -fore green
                Out-File -FilePath c:\OutputLogs\CofA.csv -Append -InputObject "in_item_key $item in_lot_key $lot imgfilename $name in_cofa_crtdt $timestamp"
                start-sleep -s 5
                $conn = New-Object System.Data.SqlClient.SqlConnection("Data Source=PVSNTDB33; Initial Catalog=adagecopy_daily; Integrated Security=TRUE")
                $conn.Open()
                $insert_stmt = "INSERT INTO in_cofa_pvs (in_item_key, in_lot_key, imgfileName, in_cofa_crtdt) VALUES ('$item','$lot','$name','$timestamp')"
                $cmd = $conn.CreateCommand()
                $cmd.CommandText = $insert_stmt
                $cmd.ExecuteNonQuery()
                $conn.Close()
    #### PACKAGE SHIPPER PROCESS BEGINS ####
              elseif ($test -eq $false) {
                $pos = $name.IndexOf(".")
           $left=$name.substring(0,$pos)
           $pos = $left.IndexOf("O")
           $tempItem=$left.substring(0,$pos)
           $order = $left.Substring($pos + 1)
           $shipid=$tempItem.Substring(1)
                Write-Host "so_hdr_key $order so_ship_key $shipid imgfilename $name in_cofa_crtdt $timestamp"  -fore green
                Out-File -FilePath c:\OutputLogs\PackageShipper.csv -Append -InputObject "so_hdr_key $order so_ship_key $shipid imgfilename $name in_cofa_crtdt $timestamp"
    Rich Thompson

    Hi
    Since SQL Server 2000 has been out of support, I recommend you to upgrade the SQL Server 2000 to a higher version, such as SQL Server 2005 or SQL Server 2008.
    According to your description, you can try the following methods to check duplicate record in SQL Server.
    1. You can use
    RAISERROR to check the duplicate record, if exists then RAISERROR unless insert accordingly, code block is given below:
    IF EXISTS (SELECT 1 FROM TableName AS t
    WHERE t.Column1 = @ Column1
    AND t.Column2 = @ Column2)
    BEGIN
    RAISERROR(‘Duplicate records’,18,1)
    END
    ELSE
    BEGIN
    INSERT INTO TableName (Column1, Column2, Column3)
    SELECT @ Column1, @ Column2, @ Column3
    END
    2. Also you can create UNIQUE INDEX or UNIQUE CONSTRAINT on the column of a table, when you try to INSERT a value that conflicts with the INDEX/CONSTRAINT, an exception will be thrown. 
    Add the unique index:
    CREATE UNIQUE INDEX Unique_Index_name ON TableName(ColumnName)
    Add the unique constraint:
    ALTER TABLE TableName
    ADD CONSTRAINT Unique_Contraint_Name
    UNIQUE (ColumnName)
    Thanks
    Lydia Zhang

  • How to get rid of duplicate records generated frm hierarchical cube in sql?

    Hi All,
    database version 10gR2.
    I am trying to aggregated data for two hierarchical dimensions, specifically organization and products.
    I am using one ROLLUP for each dimension, which would be two ROLLUP in GROUP BY clause to do the aggregation for every level of organization and product that are in included in the hierarchy.
    the troubling part is that that products that have data in corresponding fact table are not always located at the lowest level (which is 6) of the product hierarchy.
    e.g.
    product_id                               level
    0/01/0101/010102/01010201    5                           -->01010201, at level 5 , has data in fact table
    0/01/0101/010103                   4                           -->010103, at level 4, has data in fact table as well
    0/02/0201/020102/02010203/0201020304/020102030405              6   --> at level 6,(lowest level) and has data in fact table     we have a flat product hierarchy stored in table as below:
    prod_id  up_code_1 up_code_2 up_code_3   up_code_4   up_code_5 up_code_6
    01010201     0     01     0101     010102     01010201      NULL
    010103     0     01     0101     010103     null             nulldue to the NULL in product in level 6 for 01010201, when i run the query below, one duplicate record will be generated.
    for 010103, there will be 2 duplicate records, and for 020102030405 will be none.
    Encounter the same issue with the organizational dimension.
    currently, I am using DISTINCT to get rid of the duplicate records, but I don`t feel right to do it this way.
    So, I wonder if there is a more formal and standard way to do this?
    select distinct ORG_ID, DAY_ID,  TRADE_TYPE_ID, cust_id, PRODUCT_ID, QUANTITY_UNIT, COST_UNIT, SOURCE_ID,
          CONTRACT_AMOUNT, CONTRACT_COST, SALE_AMOUNT,SALE_COST, ACTUAL_AMOUNT, ACTUAL_COST, TRADE_COUNT
    from (     
    select  coalesce(UP_ORG_ID_6, UP_ORG_ID_5, UP_ORG_ID_4, UP_ORG_ID_3, UP_ORG_ID_2, UP_ORG_ID_1) as ORG_ID,
          a.day_id as day_id,        
          a.TRADE_TYPE_ID as TRADE_TYPE_ID,
          a.CUST_ID,
          coalesce(UP_CODE_6, UP_CODE_5, UP_CODE_4, UP_CODE_3, UP_CODE_2, UP_CODE_1) as product_id,
          QUANTITY_UNIT,
          COST_UNIT,
          A.SOURCE_ID as SOURCE_ID,
          SUM(CONTRACT_AMOUNT) as CONTRACT_AMOUNT,
          SUM(CONTRACT_COST) as CONTRACT_COST,
          SUM(SALE_AMOUNT) as SALE_AMOUNT,
          SUM(SALE_COST) as SALE_COST,
          SUM(ACTUAL_AMOUNT) as ACTUAL_AMOUNT,
          SUM(ACTUAL_COST) as ACTUAL_COST,
          SUM(TRADE_COUNT) as TRADE_COUNT     
    from DM_F_LO_SALE_DAY a, DM_D_ALL_ORG_FLAT B, DM_D_ALL_PROD_FLAT D --, DM_D_LO_CUST E
    where a.ORG_ID=B.ORG_ID
          and a.PRODUCT_ID=D.CODE
    group by rollup(UP_ORG_ID_1, UP_ORG_ID_2, UP_ORG_ID_3, UP_ORG_ID_4, UP_ORG_ID_5, UP_ORG_ID_6),
          a.TRADE_TYPE_ID,
          a.day_id,
          A.CUST_ID,
          rollup(UP_CODE_1, UP_CODE_2, UP_CODE_3, UP_CODE_4, UP_CODE_5, UP_CODE_6),
          a.QUANTITY_UNIT,
          a.COST_UNIT,
          a.SOURCE_ID );Note, GROUPING_ID seems not help, at least i didn`t find it useful in this scenario.
    any recommendation, links or ideas would be highly appreciated as always.
    Thanks

    anyone ever encounter this kind of problems?
    any thought would be appreciated.
    thanks

  • How to delete Duplicate records in IT2006

    Dear Experts
    We have a situation like where we have duplicate records with same start and end dates in IT2006. This is because of the incorrect configuration which we have corrected now, but we need to do  a clean-up for the existing duplicate records. Any idea on how to clean it?  I ran report RPTKOK00 to find these duplicates but I could not delete the duplicate/inconsistenct record using report RPTBPC10 or HNZUPTC0, i Could only delete the deductions happened in the record.
    Is there any standard report/any other means of deleting the duplicate records created in IT2006?
    Thanks in advance for all your help.
    Regards
    Vignesh.

    You could probably use se16n to identify the duplicates and create the list of quotas to delete, and you could probably use t-code lsmw to write up a script to delete them, but be aware that you can't delete a Quota if it's been deducted from.
    You'd have to delete the Absence/Attendance first, then delete the Quota, then recreate the Absence/Attendance.

Maybe you are looking for