Delete duplicate rows -- based on 4 columns -- how?

I asked this question on how to delete duplicates recently and received this suggestion which works well -- except --
With CTE AS
SELECT *, ROW_NUMBER() OVER (Partition by fld1 order by fld1) RowNum
FROM #tmpA
DELETE From CTE
WHERE RowNum > 1
-- the actual table I need to delete duplicate rows on is based on 4 columns. The following table contains 14,462 rows of which 14,348 are distinct -- based on the following 4 colums. Below is an image of a sample of the data contained in the
table for my question and to the right of that data is the column structures (data types). Is it possible to do something like the above example suggestion with the table in the image below? How to do that? I need to delete rows so that 14462
goes down to 14348. If I only reference one column for the delete -- this would delete like 7000+ rows. I only need to remove 114 rows.
Rich P

Add the other 3 columns to the partition.
Jason Long

Similar Messages

Deleting duplicate rows based on three columns in Oracle 8i

Hi all,
The database we use is Oracle 8i.
The query below raises the too_many_rows exception when I launch an application. So I want to delete the duplicated rows :
select polarisation_1, polarisation_2
into v_pol1_tech, v_pol2_tech
from v_cfh_lien_element
where nom_lien = p_nom_lien
AND num_canal_1 = p_num_canal_1
AND freq_emise_1 = p_freq_emise_1;
Notice that with many possible values of the parameters p_nom_lien, p_num_canal_1 and p_freq_emise_1 then the exception is raised.
So how to delete generally the duplicated rows based on the three columns "nom_lien" , "num_canal_1" and "freq_emise_1" ?
Thank you very much indeed.

Check the other thread with same question deleting duplicate rows based on three columns in Oracle 8i

Deleting duplicate entries based on one column

Guys,
I have two internal tables i_tab1 and i_tab2.
I want to copy contents of i_tab1 into i_tab2 while removing any duplicate entry.
i am comparing on just one column in i_tab1.

Hi,
Try this way
LOOP AT itab2 INTO wa_itab2.
READ TABLE itab1 INTO wa_itab1 WITH KEY field = wa_itab1-field BINARY SEARCH.
IF sy-subrc = 0.
code
MODIFY itab2 INDEX sy-tabix FROM wa_itab2 TRANSPORTING field. " field is modified field
ENDIF.
ENDLOOP.
Regards
Bala Krishna

How to delete duplicate rows in oracle and retaining last duplicate rows

Hi,
I'm having millions of records in the table .Some duplicate values/rows are inserted in it.
I just want to delete duplicate rows but also want to retain the last duplicate row.
For eg if one record is found three times ,i want to delete first and second record and retain
the third one i.e the last inserted one.
Regards
Paramdeep Singh

user13310594 wrote:
Hi,
I'm having millions of records in the table .Some duplicate values/rows are inserted in it.
I just want to delete duplicate rows but also want to retain the last duplicate row.
For eg if one record is found three times ,i want to delete first and second record and retain
the third one i.e the last inserted one.Hi Paramdeep,
To start with, since you do not wish to keep multiple rows with same values, why allow them to get inserted in the first place?
Wouldn't it be easier to block duplicate rows from inserting by creating a Unique constraint on Column(s) that define a row as duplicate, then rather deleting duplicate rows periodically?
For deleting duplicate rows, there are N number of techniques available. Always remember, you need to have a rigid criteria that marks row as duplicate.
[url http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:15258974323143]this is one recomended by tom for large tables.
Vivek L

How to Delete duplicates rows without using rowid/distinct.

How to delete duplicates rows present in a table without using rowid or even distinct.

How about:
SQL> SELECT * FROM t1;
         A          B
         1          2
         2          3
         1          2
         4          4
         4          4
         4          4
SQL> DELETE FROM t1
2 WHERE (a, b) IN (SELECT a, b FROM t1
3                   GROUP BY a, b
4                   HAVING COUNT(*) > 1) and
5        rownum = 1;
1 row deleted.
SQL> /
1 row deleted.
SQL> /
1 row deleted.
SQL> /
0 rows deleted.
SQL> SELECT * FROM t1;
         A          B
         2          3
         1          2
         4          4Although, if I was asked a similar question with all those restrictions, my first response would be along the lines of: Is this question indicative of the way I will have to work if I join this company? If so, there is no point answering because I wouldn't touch this job with a ten foot pole.

How to delete duplicate row in certain case

Hi all,
I need you help to delete my duplicate row.
Sample Data in table (table1)
 
Timein
DateIn
locationin useridin
status
08:20:00 01/09/2007
0001
U01
1
08:10:30 01/09/2007
0004
U01
1
07:20:00 01/09/2007
0006
U01
1
08:14:00 01/09/2007
0001
U02
1
06:10:30 01/09/2007
0004
U02
1
08:10:10 01/09/2007
0006
U02
1
 
I need to delete the row in table1 and leave the minimum timein in
that table by useridin.
The last output in table1 is like below.
Timein
DateIn
locationin useridin
status
07:20:00 01/09/2007
0006
U01
1
06:10:30 01/09/2007
0004
U02
1
Can i used below sql to output the result;


delete
from
table1 t1 
where
t1.rowid
>


(
select
min(t2.rowID)
from
table1 t2

where
t1.Datein
= t2.Datein


and
t1.Status
=
t2.Status

and
t1.UserIdin
=
t2.UserIdin
order by t2.timein);
 
Thanks
in advanced

Hi Rob,
Just trying to chip in little bit!
Created Table As follows same as you, only two rows with duplicatem user, datein, and timein:
create table table1
as
select '08:20:00' timein, date '2007-09-01' datein, '0001' locationin, 'U01' useridin, 1 status from dual union all
select '08:10:30', date '2007-09-01', '0004', 'U01', 1 from dual union all
select '07:20:00', date '2007-09-01', '0006', 'U01', 1 from dual union all
select '08:14:00', date '2007-09-01', '0001', 'U02', 1 from dual union all
select '06:10:30', date '2007-09-01', '0004', 'U02', 1 from dual union all
select '06:10:30', date '2007-09-01', '0001', 'U02', 1 from dual
If I run your delete statement result row will be as follows:
SQL> select * from table1;
TIMEIN DATEIN LOCA USE STATUS
07:20:00 01-SEP-07 0006 U01 1
06:10:30 01-SEP-07 0004 U02 1
06:10:30 01-SEP-07 0001 U02 1
Here problem is that its not able to delete duplicate rows having same timein and timeout.
So I changed you query as follows:
DELETE FROM tab1
WHERE rowid NOT IN (
 SELECT MIN(rowid)
 FROM tab1
 WHERE (UseridIn, TO_DATE(TO_CHAR(DateIn,'YYYYMMDD') || TimeIn,'YYYYMMDDHH24:MI:SS')) IN
 ( SELECT UseridIn, MIN(TO_DATE(TO_CHAR(DateIn,'YYYYMMDD') || timein,'YYYYMMDDHH24:MI:SS'))
 FROM tab1
 GROUP BY UseridIn)
 GROUP BY UseridIn
If I run your delete statement with little bit of changes, the result row was as follows:
SQL> select * from table1;
TIMEIN DATEIN LOCA USE STATUS
07:20:00 01-SEP-07 0006 U01 1
06:10:30 01-SEP-07 0004 U02 1
Regards,
Raj

Change array size and delete duplicate row

Hello,
I am running a program that does a for loop 3 times. In each loop, data is collected for x and y points (10 in the example provided). How can I output the array with the 30 points in a simple 2D array (size 2x30)?
Also, from the array, how do I delete duplicate rows that contain the same x-value?
Thanks,
hiNi.
Solved!
Go to Solution.
Attachments:
array size.vi ‏10 KB

Hello,
THANK YOU for solving my first issue! Attached is the data set that contains the duplicate entries for the x-axis values. The y-axis value may be different, but not the x. I would like to delete the 2nd row that contains the same x-value.
Here is simple example of my input and desired output:
IN:
28.512000 -115.310532
28.812000 -86.937660
29.112000 -116.877052
29.412000 -87.223625
29.412000 -111.349045
30.012000 -86.510780
30.312000 -112.048187
OUT:
28.512000 -115.310532
28.812000 -86.937660
29.112000 -116.877052
29.412000 -87.223625
30.012000 -86.510780
30.312000 -112.048187
I am thinking of comparig the last x-axis value to the current one, and if it is the same, then not parse that row. The attached VI is as far as I got.
Any help will be greatly appreciated.
Thanks,
hiNi
Attachments:
delete duplicate row.vi ‏8 KB
data.txt ‏1 KB

Delete Duplicate rows in Data Package based on selection

Hello experts,
I have the data coming from Oracle using DB_Connect. The data has duplicate order no. I need to delete the duplicate rows in the data package based selection before updating the data target.
I am thinking of writing this in updaterule start routine. I would greatly appreciate any sample code on selective deletion of data package rows.
Thanks a lot!
Sri

Nagesh,
Thanks for your reply.
Do I need write this in Update Start routine?
Another thing the order number field is coming from source system. Currently we are not mapping/assingning to InfoObject. In other words we are not seding order no to data target.
In my query can I write something like this...
DELETE ADJACENT DUPLICATES FROM DATA_PACKAGE COMPARING DATA_Package-OrderNo
Thanks a lot!
Sri

How to delete duplicate rows in a table

hai,
i have a table which consists of hunderds of records and contains duplicate records also. how can i delete duplicate records.
thanks
ravi

This maybe a litle bit fast for deletion:-
delete from emp where rowid in (
2 SELECT rowid FROM emp
3 group by rowid,empno,ename,job
4 minus
5 SELECT min(rowid) FROM emp
6 group by empno,ename,job);

Selecting duplicate rows based on a combination of columns as key

Hi ,
I have a table with 5 columns.
Code ID S_DATE E_DATE Name
1 23 01012001 null ABC
1 09 01012001 null XYZ
2 81 04022007 null TVU
1 43 03092008 null XXX
Now, I need write a select query to fetch the duplicate rows from the above table having the combination of (Code, S_DATE,E_DATE) as the key.
So from the above example, I need to get Row1 and Row2 as output (but not Row3 as it has a different S_DATE)
Thanks in advance for your suggestions.
Thanks
Edited by: thotaramesh on Mar 9, 2009 4:54 PM

On XE;
WITH sample_data AS (
   SELECT 1 code,23 ID, '01012001' s_date, null e_date, 'ABC' NAME FROM dual UNION ALL
   SELECT 1, 09, '01012001', null, 'XYZ' FROM dual UNION ALL
   SELECT 2, 81, '04022007', null, 'TVU' FROM dual UNION ALL
   SELECT 1, 43, '03092008', null, 'XXX' FROM dual)
SELECT code, ID, s_date, e_date, NAME
FROM (
   SELECT
      sample_data.*,
      COUNT(*) over (PARTITION BY code, s_date, e_date) dups
   FROM sample_data)
WHERE dups > 1;
      CODE         ID S_DATE   E_DATE NAME
         1         23 01012001        ABC
         1          9 01012001        XYZ

How not show duplicate rows based on one field

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0
Hello
I have a query that looks for certain scripts that did not run on a particular day compared to what ran the same day a week ago. We want to include the start_datetime and end_datetime but when I add it to the select statement it brings up all instances of the jobs that run multiple times during the day. Is there a way to exclude the extra rows based on the script_name field?
SELECT instances.script_name,
                         instances.instance_name,
                         REGEXP_REPLACE(master.description,
                                        chr(49814), -- em-dash
                                        '-') description
                                        --instances.start_datetime
                  FROM   xxcar.xxcar_abat_instances Instances,
                         xxcar.xxcar_abatch_master Master
                  WHERE 1 = 1
                  AND    TRUNC(start_datetime) = TRUNC(TO_DATE(:p_StartDate, 'YYYY/MM/DD HH24:MI:SS')) - (:p_NumOfWeeks * 7)
                  AND    Instances.SCRIPT_NAME = Master.SCRIPT_NAME (+)
                  MINUS
                  SELECT script_name,
                         instance_name,
                         NULL
                         --NULL
                  FROM   xxcar.xxcar_abat_instances
                  WHERE 1 = 1
                  AND    TRUNC(start_datetime) = TRUNC(TO_DATE(:p_StartDate, 'YYYY/MM/DD HH24:MI:SS'))

MINUS does a set operation - removing rows from the first set that exactly match the second set.
When you add columns to the first set, you want a more restricted filtering - try a multi-column NOT IN:
To remove multiple runs, group and get min/max
SELECT instances.script_name,
                         instances.instance_name,
                         REGEXP_REPLACE(master.description,
                                        chr(49814), -- em-dash
                                        '-') description,
                         min(instances.start_datetime) start_datetime,
                         min(instances.end_datetime) end_datetime
                  FROM   xxcar.xxcar_abat_instances Instances,
                         xxcar.xxcar_abatch_master Master
                  WHERE 1 = 1
                  AND    TRUNC(start_datetime) = TRUNC(TO_DATE(:p_StartDate, 'YYYY/MM/DD HH24:MI:SS')) - (:p_NumOfWeeks * 7)
                  AND    Instances.SCRIPT_NAME = Master.SCRIPT_NAME (+)
                  AND (script_name, instance_name) NOT IN
                ( SELECT script_name,
                         instance_name
                  FROM   xxcar.xxcar_abat_instances
                  WHERE 1 = 1
                  AND    TRUNC(start_datetime) = TRUNC(TO_DATE(:p_StartDate, 'YYYY/MM/DD HH24:MI:SS'))
group by instances.script_name, instances.instance_name, master.descriptionYou didn't give table definitions, and the query has schemas in it, so I didn't test it.
regards,
David

SQL Server 2014 - Delete duplicate rows in a table except one

Hi all,
I have a table that contains duplicate rows. Using grouping I can easily identify those rows.
As of now, I count the grouped records and copy one of the group in a temporary table together with its count of elements. Than I loop over this table and delete N - 1 rows of each group from the original table. Unfortunately this is done with two cursors
and WHILE loops.
Is anybody out there who could give some hints how to handle deletion of duplicated rows leaving one in place using pure SQL? I tried a full delete of all elements of a group and then an INSERT of one row to recreate a row again. But this is not
very elegant and close to using cursors.
Regards Uwe

Hi all,
based on input below from Uri, I have developed a little more general solution for removal of duplicates. I did that, because I'm sure that for many beginners in SQL it is easier specialize a proposal by removing parts of it than adding new functionality.
Here is my code.
-- Remove duplicates from a table and document the deleted rows
DECLARE @MyTable Table (Col1 int, Col2 int, Col3 int, Col4 int);
DECLARE @OutTable Table (Col1 int, Col2 int, Col3 int, Col4 int);
INSERT INTO @MyTable VALUES
(1, 10, 100, 7),
(2, 20, 200, 2),
(2, 20, 200, 1),
(3, 30, 300, 3),
(1, 10, 200, 4),
(1, 10, 100, 5),
(1, 10, 100, 6);
WITH cte (RN, Col1, Col2, Col3, Col4)
AS (SELECT ROW_NUMBER() OVER (PARTITION BY Col1, Col2, Col3 ORDER BY (SELECT 0)) AS RN, Col1, Col2, Col3, Col4
FROM @MyTable)
DELETE FROM cte
OUTPUT DELETED.Col1, DELETED.Col2, DELETED.Col3, DELETED.Col4
INTO @OutTable (Col1, Col2, Col3, Col4)
WHERE RN > 1;
SELECT * FROM @OutTable;
And this is the result:
The MS 139444 article Satish mentioned is focussed on duplicate PKs removal. Thanks for the hint.
Thanks to you all for your rapid response.
Regards Uwe

Delete duplicate records based on condition

Hi Friends,
I am scratching my head as how to select one record from a group of duplicate records based upon column condition.
Let's say I have a table with following data :
ID   START_DATE   END_DATE    ITEM_ID     MULT    RETAIL            |                      RETAIL / MULT
1     10/17/2008   1/1/2009     83     3     7                 |                            2.3333
2     10/17/2008   1/1/2009     83     2     4                 |                            2
3     10/17/2008   1/1/2009     83     2     4                 |                            2
4     10/31/2008   1/1/2009     89     3     6                 |                            2
5     10/31/2008   1/1/2009     89     4     10                |                            2.5
6     10/31/2008   1/1/2009     89     4     10                |                            2.5
7     10/31/2008   1/1/2009     89     6     6                 |                            1
8     10/17/2008   10/23/2008     124     3     6                 |                            2From the above records the rule to identify duplicates is based on START_DATE,+END_DATE+,+ITEM_ID+.
Hence the duplicate sets are {1,2,3} and {4,5,6,7}.
Now I want to keep one record from each duplicate set which has lowest value for retail/mult(retail divided by mult) and delete rest.
So from the above table data, for duplicate set {1,2,3}, the min(retail/mult) is 2. But records 2 & 3 have same value i.e. 2
In that case pick either of those records and delete the records 1,2 (or 3).
All this while it was pretty straight forward for which I was using the below delete statement.
DELETE FROM table_x a
      WHERE ROWID >
               (SELECT MIN (ROWID)
                  FROM table_x b
                 WHERE a.ID = b.ID
                   AND a.start_date = b.start_date
                   AND a.end_date = b.end_date
                   AND a.item_id = b.item_id);Due to sudden requirement changes I need to change my SQL.
So, experts please throw some light on how to get away from this hurdle.
Thanks,
Raj.

Well, it was my mistake that I forgot to mention one more point in my earlier post.
Sentinel,
Your UPDATE perfectly works if I am updating only NEW_ID column.
But I have to update the STATUS_ID as well for these duplicate records.
ID   START_DATE   END_DATE    ITEM_ID     MULT    RETAIL    NEW_ID   STATUS_ID |   RETAIL / MULT
1     10/17/2008   1/1/2009     83     3     7         2         1      |     2.3333
2     10/17/2008   1/1/2009     83     2     4                                |     2
3     10/17/2008   1/1/2009     83     2     4           2         1      |     2
4     10/31/2008   1/1/2009     89     3     6           7         1      |     2
5     10/31/2008   1/1/2009     89     4     10          7         1      |     2.5
6     10/31/2008   1/1/2009     89     4     10          7         1      |     2.5
7     10/31/2008   1/1/2009     89     6     6                            |     1
8     10/17/2008   10/23/2008     124     3     6                            |     2So if I have to update the status_id then there must be a where clause in the update statement.
WHERE ROW_NUM = 1
AND t2.id != t1.id
AND t2.START_DATE = t1.START_DATE
AND t2.END_DATE = t1.END_DATE
AND t2.ITEM_ID = t1.ITEM_IDInfact the entire where_ clause in the inner select statement must be in the update where clause, which makes it totally impossible as T2 is persistent only with in the first select statement.
Any thoughts please ?
I appreciate your efforts.
Definitely this is a very good learning curve. In all my experience I was always writing straight forward Update statements but not like this one. Very interesting.
Thanks,
Raj.

Deleting Duplicate Rows in a list

Hey folks, I've scowered around a bit for the answer to this and can't for the life of my figure it out.
I've got a list of ~2,000-3,000 words in the following format:
Fact
Fiction
Funny
Funny
Funny
Funky
etc etc. I am looking to make numbers delete all the duplicate rows such that the above list would become:
Fact
Fiction
Funny
Funky
All of these words are in column A on a separate sheet in a numbers document I'm using to run an experiment. Is there a built in command or something that would do this? (I'm a COMPLETE beginner at using Numbers as a heads up)
Any help with this would be GREATLY appreciated.

Teaghue wrote:
Never mind! I found an old old post explaining that this can't be done in numbers, so I just did it in excel.
Perfectly wrong !
The way to achieve the described goal was described several times in this forum.
*_You just didn't search carefully !_*
Searching for delet AND duplicate return several threads.
Here are two of them.
I didn't made a typo. I used delet so that it retrieve delete as well as deleting
http://discussions.apple.com/thread.jspa?messageID=12992492
http://discussions.apple.com/thread.jspa?messageID=11559125
Yvan KOENIG (VALLAURIS, France) mercredi 2 mars 2011 17:59:40

Deleting a row based on a field in the table

Hi,
How do I programatically in Java delete a row in a table based on a specific field in a table? For example, I want to delete a specific row by a zip code.
thank you

try this
public void deleteSpecificRow(){
                          // 1. Access the binding container
               DCBindingContainer bc = (DCBindingContainer)getBindings();
              // 2. Find a named iterator binding
               DCIteratorBinding iter =
                 (DCIteratorBinding)bc.findIteratorBinding("YourIteratorName");
               //4. create RowSetIterator Object
               RowSetIterator rsi = iter.getViewObject().createRowSetIterator(null);
                rsi.reset();
               //delete all rows
                while (rsi.hasNext())
                    Row current=rsi.next();
             //check your condition here, if yes remove the record
            current.remove();
                    rsi.next();
   rsi.closeRowSetIterator();
}Edited by: M.Jabr on Nov 24, 2011 5:59 PM

Delete duplicate rows -- based on 4 columns -- how?

Similar Messages

Maybe you are looking for