Deleting duplicate from list

What is the easy and fastest way to delete duplicate from
list.
My dynamic list is pretty small. Need to loop through and
delete duplicate and give unique item list.
We don't have a CF function yet?.

What do I think? Well - I think your code has a few problems:
1) It's not very re-usable
2) It's not very easy to read (variables are named poorly)
3) It's introduces unecessary usage of complex data types
(structs)
4) Functionally, it does not preserve the sort order of the
original list.
To elaborate on #4 above....once you have your list values
stored as "keys" in a struct, you lose the original order of the
list. Keys in a struct are completely unsorted. A call to
StructKeyList will return a list of keys, with no guarantee on the
order of the keys returned. The order in which the keys are
inserted makes no difference. Once all of the keys are defined in a
struct, they are all essentially un-ordered indexes.
If order is not important to you for this particular usage,
than you shouldn't have anything to worry about. Functionally, your
code WILL work. But my honest opinion is that there are many
"better" ways to approach this (see the example UDF I attached
previously).
Please bear in mind I'm not trying to be mean here, just
offering some constructive criticism.

Similar Messages

Deleting Duplicates from a table

Its a huge table with 52 fields and 30k rows. I need to delete the duplicates based on one of the fields. GROUP BY is taking a lot of time. Is there a quicker way to delete the duplicates using SQL.
Thanks.

How many duplicates have you got? Do you have even a vague idea? 1%? 20%? 90%?
One way would be to add a unique constraint on the column in question. This will fail, of course, but you can use the EXCEPTIONS INTO clause to find all the ROWIDs which have duplicate values. You can then choose to delete those rows using a variant on teh query already posted. You may need to run %ORACLE_HOME%\rdbms\admin\utlexcptn.sql to build the EXCEPTIONS table first.
This may seem like some unnecessary work, but the most effective way of deleting duplicates from a table is to have relational integrity constraints in place which prevent you having duplicates in the first place. To paraphrase Tim Gorman, you can't get faster than zero work!
Cheers, APC

Delete Duplicates from internal table with object references

Hi
How can I delete duplicates from an internal table in ABAP OO based on the value of one of the attributes?
I have created a method, with the following code:
LOOP AT me->business_document_lines INTO l_add_line.
    CREATE OBJECT ot_line_owner
      EXPORTING
        i_user      = l_add_line->add_line_data-line_owner
        i_busdoc = me->business_document_id.
      APPEND ot_line_owner TO e_line_owners.
ENDLOOP.
e_line_owners are defined as a table containing only object references.
One of the attribute of the object in the table is called USER. And I would like to do a "delete ADJACENT DUPLICATES FROM e_line_owners", based on that attribute.
How can do this?
Regards,
Morten Nielsen

Hello Morten
Assuming that the instance attribute is <b>public </b>you could try to use the following coding:
SORT e_line_owners BY table_line->user.
DELETE ADJACENT DUPLICATES FROM e_line_owners
    COMPARING table_line->user.
However, I am not really sure (cannot test myself) whether <b>TABLE_LINE</b> can be used together with SORT and DELETE.
Alternative solution:
DATA:
     ld_idx    TYPE sy-tabix.
LOOP AT e_line_owners INTO ls_line.
    ld_idx = syst-tabix + 1.
    LOOP AT e_line_owners TRANSPORTING NO FIELDS FROM ld_idx
                   WHERE ( table_line->user = ls_line->user ).
      DELETE e_line_owners INDEX syst-tabix.
    ENDLOOP.
ENDLOOP.
Regards
Uwe

How to choose in Delete Duplicates from internal table?

Now I need to delete Duplicates from internal table,
So at first I sort
than I delete duplicate
Sort itab1 BY Company_Code Asset_No Capital_Date.
      DELETE ADJACENT DUPLICATES FROM itab1 COMPARING Company_Code Asset_No Capital_Date
Company_Code
Asset_No
Capital_Date
Remark
BC35
1515593
20021225
Helen
BC35
1515593
20021225
Common Asset
BC35
1515594
20030109
Judy
BC35
1515594
20030109
Common Asset
But here comes my problem~If I want to delete the Common Asset in Remark Column,how I let it choose the right one to do it?

Hi Jack
Try the below coding..
Report zsamp.
types: begin of t_tab,
        comp_code(4) type c,
        ***_no(7) type n,
        cap_date type d,
        remark type string,
        end of t_tab.
data: i_tab type TABLE OF t_tab,
       w_tab type t_tab.
w_tab-comp_code = 'BC35'.
w_tab-***_no = '1515593'.
w_tab-cap_date = '20021225'.
w_tab-remark = 'Helen'.
append w_tab to i_tab.
w_tab-comp_code = 'BC35'.
w_tab-***_no = '1515593'.
w_tab-cap_date = '20021225'.
w_tab-remark = 'Common Asset'.
append w_tab to i_tab.
w_tab-comp_code = 'BC35'.
w_tab-***_no = '1515594'.
w_tab-cap_date = '20030109'.
w_tab-remark = 'Judy'.
append w_tab to i_tab.
w_tab-comp_code = 'BC35'.
w_tab-***_no = '1515594'.
w_tab-cap_date = '20030109'.
w_tab-remark = 'Common Asset'.
append w_tab to i_tab.
sort i_tab by remark.
delete ADJACENT DUPLICATES FROM i_tab COMPARING remark.

How do i delete duplicates from my music library. After updating my iTunes the "Show Duplicates" option does not appear in the menu

How do i delete duplicates from my music library. After updating my iTunes the "Show Duplicates" option does not appear in the menu

If you are on Version 11.0.1 rather than 11 Apple put it back in
View > Show Duplicates
Option + View > Show Exact Duplicates
so if you are on 11 upgrade to 11.0.1 and you will have it

HT2905 I want to delete duplicates from my iTunes music but cannot see "Display Exact Duplicates" in File menu.

Hi,
I want to delete duplicates from my iTunes music but and iTunes help says to click on "Display Exact Duplicates" in File men but I don't appear to have this function. Where else can I look?

Thanks Kappy. I had just found it but thanks reply. Next question is: Is there a quick way to then delete the duplicates without having to select each one and delete?

Deleting Duplicate from ITAB without sorting????

Hi,
A challenging and interesting problem please help. I want to delete duplicates from an ITAB without sorting (so cant use delete adjacent duplicates)
data: begin of dpp occurs 0,
        val type i,
        end of dpp.
        dpp-val = 13.
        append dpp.
        dpp-val = 15.
        append dpp.
        dpp-val = 26.
        append dpp.
        dpp-val = 15.
        append dpp.
        dpp-val = 27
        append dpp.
        dpp-val = 15.
        append dpp.
As you see 15 is duplicated in DPP,,,how can duplicated 15 entries be deleted without sorting
                   VAL
     13
     15
     26
     15
     27
     15
thhnx
Edited by: Salman Akram on Oct 12, 2010 3:54 PM

Hi,
Loop through your DPP itab then append to another. try this:
DATA: BEGIN OF dpp OCCURS 0,
val TYPE i,
END OF dpp.
dpp-val = 13.
APPEND dpp.
dpp-val = 15.
APPEND dpp.
dpp-val = 26.
APPEND dpp.
dpp-val = 15.
APPEND dpp.
dpp-val = 27.
APPEND dpp.
dpp-val = 15.
APPEND dpp.
DATA: BEGIN OF dpp1 OCCURS 0.
        INCLUDE STRUCTURE dpp.
DATA: END OF dpp1.
LOOP AT dpp.
READ TABLE dpp1 WITH KEY val = dpp-val.
IF sy-subrc NE 0.
    APPEND dpp TO dpp1.
ELSE.
    CONTINUE.
ENDIF.
ENDLOOP.
REFRESH dpp.
dpp[] = dpp1[].
thanks.

How do I delete duplicates from my photo library. I am sure I let the duplicates happen but do not know how to get rid of them.

How do I delete duplicates from my photo library? I am sure I let the duplicates happen but do not know how to get rid of them.

For dealing with duplicates in iPhoto check out Duplicate Annihilator

How to delete duplicates from iTunes but not hard drive

I'm running iTunes 10 and have my music stored on both my NAS drive and backup drive. Each track appears twice on iTunes: once as lossless for streaming and and once as compressed AAC files for synching to ipods and iphones.
For some reason, when my PC is turned on after being turned on, the linkstation cannon be 'seen' by iTunes, and therefore if I try to play a track, 'the original file cannot be found'. If I use the 'add folder to library' option in the 'file' dropdown menu, the tracks are located again.
Unfortunately, I the last time I did this it resulted in 4 copies of each song on itunes; 2 lossless and 2 AAC.
I have 2 questions:
Firstly is there an easy way to delete the duplicates from my iTunes library so that I am left with only 1 lossless and one AAC copy, but not deleing the files from my NAS and backup drive?
Is there a way I can just store my music as lossless, but convert it easily to aa more compressed format for synching to iphone and ipod?
Thanks!

You can try my new DeDupe script. It should be able to get rid of the redundant copies while keeping one copy of each track in each format. See this thread for background.
I've not used the feature, but iTunes can downsample to 128k AAC on the fly as it syncs your devices. Of course you might find the process is too slow for your needs or find that 128k is too much compression.
tt2

Deleting duplicates from a table ,who's size is 386 GB

Need to delete duplicate records from the table.Table contains 33 columns out of them only PK_NUM is the primary key columns. As PK_NUM contains unique records we need to consider either min/max value.
Sample data :
PK_NUM
Name
AGE
1
ABC
20
2
PQR
25
3
ABC
20
Expected data should contains only 2 records:
PK_NUM
Name
AGE
1
ABC
20
2
PQR
25
*1 can be replaced by 3 ,vice versa.
Size of table : 386 GB
Total records in the table : 1766799022
Distinct records in the table : 69237983(Row distinct with out Primary key)
Duplicate records in the table : 1697561039(Row duplicates without primary key)
Column details :
4 : Date data type
4 : Number data type
1 : Char data type
24: Varchar2 data type
DB details : Oracle Database 11g EE::11.2.0.2.0 ::64bit Production
My plan here is to
Pull distinct records and store it in a back up table.(ie by using insert into select)
Truncate existing table and move records from back up to existing.
As data size is huge ,
Want to know what is the optimized sql for retrieving the distinct records
Any estimate on how much it will take to complete (insert into select) and to truncate the existing table.
Please do let me know ,if there is any other best way to achieve this.My ultimate goal is to remove the duplicates.

As data size is huge ,
Want to know what is the optimized sql for retrieving the distinct records
Any estimate on how much it will take to complete (insert into select) and to truncate the existing table.
@ 1. - Your best chance seems to be (should require a single FTS only)
create backup_table as
select pk,name,age,a_date,a_string,a_number, ...
from (select pk,name,age,a_date,a_string,a_number, ...
               row_number() over (partition by name,age order by a_date) rn
          from big_table
where rn = 1
@ 2. - Having statistics in place and (at least nearly) up to date explain plan should return an appropriate estimate
Regards
Etbin
select pk,name,age,a_date,a_string,a_number
from (select pk,name,age,a_date,a_string,a_number,
               row_number() over (partition by name,age order by a_date) rn
          from big_table
where rn = 1
Operation
Options
Object
Rows
Time
Cost
Bytes
Filter
Predicates *
Access
Predicates
SELECT STATEMENT
13,044
1
30
53,023,860
VIEW
13,044
1
30
53,023,860
"RN" = 1
WINDOW
SORT PUSHED RANK
13,044
1
30
495,672
ROW_NUMBER() OVER ( PARTITION BY "NAME","AGE" ORDER BY "A_DATE")< = 1
TABLE ACCESS
STORAGE FULL
BIG_TABLE
13,044
1
26
495,672
select pk,name,age,a_date,a_string,a_number
from big_table
where pk in (select min(pk) keep (dense_rank first order by a_date)
                from big_table
               group by name,age
Operation
Options
Object
Rows
Time
Cost
Bytes
Filter
Predicates *
Access
Predicates
SELECT STATEMENT
6,000
1
52
306,000
HASH JOIN
6,000
1
52
306,000
"PK" = "$kkqu_col_1"
VIEW
VW_NSO_1
6,000
1
27
78,000
HASH
UNIQUE
6,000
1
27
126,000
SORT
GROUP BY
6,000
1
27
126,000
TABLE ACCESS
STORAGE FULL
BIG_TABLE
13,044
1
23
273,924
TABLE ACCESS
STORAGE FULL
BIG_TABLE
13,044
1
24
495,672
Message was edited by: Etbin

Delete duplicates from query output

Hello,
I would like to delete duplicated records before the output of a SAP query
I tried writing in SQ02 in additional coding, under "Free coding":
data %dtab type standard table of /1BCDWB/IQ000000001053 with header line.
sort %dtab by vbeln posnr.
DELETE ADJACENT DUPLICATES FROM %dtab.
In this way I get the short dump when running the query: "Statement is not accessible.".
If I declare %dtab in the additional coding under "DATA", when I run the query I get the error "%DTAB" has already been declared.".
If I don't declare %dtab, I can't generate the infoset.
Do you have any suggestion about the way of obtaining this result?
Thanks in advance for your support.
Best regards,
Andrea

Hello,
I didn't find any solution to this problem. I choosed another way: I asked an ABAPer to develop a report to get the same information I wanted to get with the query, as in the ABAP report there's more "space to move".
Best regards,
Andrea

Delete duplicate from internal table

HI Abapers,
I have a query on how to remove the duplicates from an internal table
My internal table data is as follows :
Cno    Catg1 Catg2
01       0         1000
01      2000         0
I want to get only one record as
01   2000 1000
How to get the result.
I tried sorted by cno and used delete duplicates but it was not helpful.
Is there any other alternative to get this done
Please help me.
Regards,
Priya

check it out with delete adjacent duplicate records
Deleting Adjacent Duplicate Entries
To delete adjacent duplicate entries use the following statement:
DELETE ADJACENT DUPLICATE ENTRIES FROM <itab>
                                  [COMPARING <f1> <f 2> ...
                                             |ALL FIELDS].
The system deletes all adjacent duplicate entries from the internal table <itab>. Entries are duplicate if they fulfill one of the following compare criteria:
Without the COMPARING addition, the contents of the key fields of the table must be identical in both lines.
If you use the addition COMPARING <f1> <f 2> ... the contents of the specified fields <f 1 > <f 2 > ... must be identical in both lines. You can also specify a field <f i > dynamically as the contents of a field <n i > in the form (<n i >). If <n i > is empty when the statement is executed, it is ignored. You can restrict the search to partial fields by specifying offset and length.
If you use the addition COMPARING ALL FIELDS the contents of all fields of both lines must be identical.
You can use this statement to delete all duplicate entries from an internal table if the table is sorted by the specified compare criterion.
If at least one line is deleted, the system sets SY-SUBRC to 0, otherwise to 4.

Deleting Apps from List

1) I tried a few apps on my iPhone and iPad that I didn't want, and uninstalled them. On iTunes they still show in the apps list, with a button to install. I'd like to permanently delete them from that app list. Hos can I do that?
2) I've got a bunch of movies and TV shows on DVD I ripped and put them on my iMac. How can I get them to my iPad?

Figured it all out. All done.

How do I delete duplicates from my iPod?

I found the duplicates in the library and deleted them. However, they didn't delete from my iPod, even after seleting Update songs on iPod. I have it set to automatically update. So now the library doesn't think there are duplicates but they are still on my iPod. How can I delete them from my iPod?

Yamipod has a feature for removing duplicates from the iPod. I've no personal experience of it but you can read the documentation and decide for yourself if it would be of use to you: YamiPod - Duplicate Songs

How can i delete duplicates from my itunes account?

At "View" I chose "Show duplicates."
Now what do I do to delete duplicates quickly without having to delete each one separately?
THanks

Hello, singinheart.
Thank you for the question. Since you have already identified the duplicate media you just have a couple more steps to go. This article outlines the remainder of the steps in order to mass delete the duplicates.
How to find and remove duplicate items in your iTunes library
http://support.apple.com/kb/ht2905
Cheers,
Jason H.

Deleting duplicate from list

Similar Messages

Maybe you are looking for