Generalise Delete Duplicate Rows Query work in any Database

Hello Friends,
I need generalize SQL Query for removal of duplicate data from a table.
Condition: It will work in both the database Oracle as well as SQL Server 2005.
Waiting for your valuable response on the above issue.
Thanks/Regards,
Bhavin Patel
Please reply to this post...

Thanks, Nimish for your reply..._
I know how to delete duplicate values entry from Oracle database.
I'm using below method quite often when to delete duplicate value from the Oracle database.::
SELECT * FROM dept do
WHERE do.ROWID NOT IN (SELECT MIN(di.ROWID) FROM dept di WHERE di.deptno = do.deptno)
WITH T1 AS
(SELECT dept.*,ROW_NUMBER() OVER(PARTITION BY deptno,dname ORDER BY deptno) RNUM FROM dept)
SELECT * FROM T1 WHERE RNUM > 1
Actually in interview this question asked to me, I've given Oracle process, but unable to give solution for SQL Server or DB2

Similar Messages

How to delete duplicate row in certain case

Hi all,
I need you help to delete my duplicate row.
Sample Data in table (table1)
 
Timein
DateIn
locationin useridin
status
08:20:00 01/09/2007
0001
U01
1
08:10:30 01/09/2007
0004
U01
1
07:20:00 01/09/2007
0006
U01
1
08:14:00 01/09/2007
0001
U02
1
06:10:30 01/09/2007
0004
U02
1
08:10:10 01/09/2007
0006
U02
1
 
I need to delete the row in table1 and leave the minimum timein in
that table by useridin.
The last output in table1 is like below.
Timein
DateIn
locationin useridin
status
07:20:00 01/09/2007
0006
U01
1
06:10:30 01/09/2007
0004
U02
1
Can i used below sql to output the result;


delete
from
table1 t1 
where
t1.rowid
>


(
select
min(t2.rowID)
from
table1 t2

where
t1.Datein
= t2.Datein


and
t1.Status
=
t2.Status

and
t1.UserIdin
=
t2.UserIdin
order by t2.timein);
 
Thanks
in advanced

Hi Rob,
Just trying to chip in little bit!
Created Table As follows same as you, only two rows with duplicatem user, datein, and timein:
create table table1
as
select '08:20:00' timein, date '2007-09-01' datein, '0001' locationin, 'U01' useridin, 1 status from dual union all
select '08:10:30', date '2007-09-01', '0004', 'U01', 1 from dual union all
select '07:20:00', date '2007-09-01', '0006', 'U01', 1 from dual union all
select '08:14:00', date '2007-09-01', '0001', 'U02', 1 from dual union all
select '06:10:30', date '2007-09-01', '0004', 'U02', 1 from dual union all
select '06:10:30', date '2007-09-01', '0001', 'U02', 1 from dual
If I run your delete statement result row will be as follows:
SQL> select * from table1;
TIMEIN DATEIN LOCA USE STATUS
07:20:00 01-SEP-07 0006 U01 1
06:10:30 01-SEP-07 0004 U02 1
06:10:30 01-SEP-07 0001 U02 1
Here problem is that its not able to delete duplicate rows having same timein and timeout.
So I changed you query as follows:
DELETE FROM tab1
WHERE rowid NOT IN (
 SELECT MIN(rowid)
 FROM tab1
 WHERE (UseridIn, TO_DATE(TO_CHAR(DateIn,'YYYYMMDD') || TimeIn,'YYYYMMDDHH24:MI:SS')) IN
 ( SELECT UseridIn, MIN(TO_DATE(TO_CHAR(DateIn,'YYYYMMDD') || timein,'YYYYMMDDHH24:MI:SS'))
 FROM tab1
 GROUP BY UseridIn)
 GROUP BY UseridIn
If I run your delete statement with little bit of changes, the result row was as follows:
SQL> select * from table1;
TIMEIN DATEIN LOCA USE STATUS
07:20:00 01-SEP-07 0006 U01 1
06:10:30 01-SEP-07 0004 U02 1
Regards,
Raj

Deleting duplicate rows based on three columns in Oracle 8i

Hi all,
The database we use is Oracle 8i.
The query below raises the too_many_rows exception when I launch an application. So I want to delete the duplicated rows :
select polarisation_1, polarisation_2
into v_pol1_tech, v_pol2_tech
from v_cfh_lien_element
where nom_lien = p_nom_lien
AND num_canal_1 = p_num_canal_1
AND freq_emise_1 = p_freq_emise_1;
Notice that with many possible values of the parameters p_nom_lien, p_num_canal_1 and p_freq_emise_1 then the exception is raised.
So how to delete generally the duplicated rows based on the three columns "nom_lien" , "num_canal_1" and "freq_emise_1" ?
Thank you very much indeed.

Check the other thread with same question deleting duplicate rows based on three columns in Oracle 8i

Change array size and delete duplicate row

Hello,
I am running a program that does a for loop 3 times. In each loop, data is collected for x and y points (10 in the example provided). How can I output the array with the 30 points in a simple 2D array (size 2x30)?
Also, from the array, how do I delete duplicate rows that contain the same x-value?
Thanks,
hiNi.
Solved!
Go to Solution.
Attachments:
array size.vi ‏10 KB

Hello,
THANK YOU for solving my first issue! Attached is the data set that contains the duplicate entries for the x-axis values. The y-axis value may be different, but not the x. I would like to delete the 2nd row that contains the same x-value.
Here is simple example of my input and desired output:
IN:
28.512000 -115.310532
28.812000 -86.937660
29.112000 -116.877052
29.412000 -87.223625
29.412000 -111.349045
30.012000 -86.510780
30.312000 -112.048187
OUT:
28.512000 -115.310532
28.812000 -86.937660
29.112000 -116.877052
29.412000 -87.223625
30.012000 -86.510780
30.312000 -112.048187
I am thinking of comparig the last x-axis value to the current one, and if it is the same, then not parse that row. The attached VI is as far as I got.
Any help will be greatly appreciated.
Thanks,
hiNi
Attachments:
delete duplicate row.vi ‏8 KB
data.txt ‏1 KB

Delete duplicate rows -- based on 4 columns -- how?

I asked this question on how to delete duplicates recently and received this suggestion which works well -- except --
With CTE AS
SELECT *, ROW_NUMBER() OVER (Partition by fld1 order by fld1) RowNum
FROM #tmpA
DELETE From CTE
WHERE RowNum > 1
-- the actual table I need to delete duplicate rows on is based on 4 columns. The following table contains 14,462 rows of which 14,348 are distinct -- based on the following 4 colums. Below is an image of a sample of the data contained in the
table for my question and to the right of that data is the column structures (data types). Is it possible to do something like the above example suggestion with the table in the image below? How to do that? I need to delete rows so that 14462
goes down to 14348. If I only reference one column for the delete -- this would delete like 7000+ rows. I only need to remove 114 rows.
Rich P

Add the other 3 columns to the partition.
Jason Long

How to Delete duplicates rows without using rowid/distinct.

How to delete duplicates rows present in a table without using rowid or even distinct.

How about:
SQL> SELECT * FROM t1;
         A          B
         1          2
         2          3
         1          2
         4          4
         4          4
         4          4
SQL> DELETE FROM t1
2 WHERE (a, b) IN (SELECT a, b FROM t1
3                   GROUP BY a, b
4                   HAVING COUNT(*) > 1) and
5        rownum = 1;
1 row deleted.
SQL> /
1 row deleted.
SQL> /
1 row deleted.
SQL> /
0 rows deleted.
SQL> SELECT * FROM t1;
         A          B
         2          3
         1          2
         4          4Although, if I was asked a similar question with all those restrictions, my first response would be along the lines of: Is this question indicative of the way I will have to work if I join this company? If so, there is no point answering because I wouldn't touch this job with a ten foot pole.

How to delete duplicate rows in oracle and retaining last duplicate rows

Hi,
I'm having millions of records in the table .Some duplicate values/rows are inserted in it.
I just want to delete duplicate rows but also want to retain the last duplicate row.
For eg if one record is found three times ,i want to delete first and second record and retain
the third one i.e the last inserted one.
Regards
Paramdeep Singh

user13310594 wrote:
Hi,
I'm having millions of records in the table .Some duplicate values/rows are inserted in it.
I just want to delete duplicate rows but also want to retain the last duplicate row.
For eg if one record is found three times ,i want to delete first and second record and retain
the third one i.e the last inserted one.Hi Paramdeep,
To start with, since you do not wish to keep multiple rows with same values, why allow them to get inserted in the first place?
Wouldn't it be easier to block duplicate rows from inserting by creating a Unique constraint on Column(s) that define a row as duplicate, then rather deleting duplicate rows periodically?
For deleting duplicate rows, there are N number of techniques available. Always remember, you need to have a rigid criteria that marks row as duplicate.
[url http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:15258974323143]this is one recomended by tom for large tables.
Vivek L

Would like to change the "Retrieve duplicate rows" Query Prop to unchecked

Currently the Query Property "Retrieve duplicate rows" is checked (selected) by default.
For our users, the typical case would be to NOT retrieve duplicate rows.
Thus, we would like to change the default setting for the "Retrieve duplicate rows" query property to
unchecked (not selected).
Is there a way to do this?
Thanks in advance.

Is there a way to do this?
No. Not easily/out of the box.
The only way I can think off to do this would be to use the Enterprise and Webi SDKs.

Delete Duplicate rows in Data Package based on selection

Hello experts,
I have the data coming from Oracle using DB_Connect. The data has duplicate order no. I need to delete the duplicate rows in the data package based selection before updating the data target.
I am thinking of writing this in updaterule start routine. I would greatly appreciate any sample code on selective deletion of data package rows.
Thanks a lot!
Sri

Nagesh,
Thanks for your reply.
Do I need write this in Update Start routine?
Another thing the order number field is coming from source system. Currently we are not mapping/assingning to InfoObject. In other words we are not seding order no to data target.
In my query can I write something like this...
DELETE ADJACENT DUPLICATES FROM DATA_PACKAGE COMPARING DATA_Package-OrderNo
Thanks a lot!
Sri

Deleting Duplicate Rows in a list

Hey folks, I've scowered around a bit for the answer to this and can't for the life of my figure it out.
I've got a list of ~2,000-3,000 words in the following format:
Fact
Fiction
Funny
Funny
Funny
Funky
etc etc. I am looking to make numbers delete all the duplicate rows such that the above list would become:
Fact
Fiction
Funny
Funky
All of these words are in column A on a separate sheet in a numbers document I'm using to run an experiment. Is there a built in command or something that would do this? (I'm a COMPLETE beginner at using Numbers as a heads up)
Any help with this would be GREATLY appreciated.

Teaghue wrote:
Never mind! I found an old old post explaining that this can't be done in numbers, so I just did it in excel.
Perfectly wrong !
The way to achieve the described goal was described several times in this forum.
*_You just didn't search carefully !_*
Searching for delet AND duplicate return several threads.
Here are two of them.
I didn't made a typo. I used delet so that it retrieve delete as well as deleting
http://discussions.apple.com/thread.jspa?messageID=12992492
http://discussions.apple.com/thread.jspa?messageID=11559125
Yvan KOENIG (VALLAURIS, France) mercredi 2 mars 2011 17:59:40

How to make "Duplicate Row" button work with LOV

Hi,
I am using JDeveloper version: Studio Edition Version 10.1.3.1.0.3984
JHeadstart version: 10.1.3.1 Release 10.1.3.1.26
We have a VO where there are 2 EOs - one is Updatable and one is Reference.
We are using Association for the Reference EO.
In the page we have a LOV and from the LOV we are redirecting some fields including some fields for the Reference EO (to show only).
Everything works fine.
But the problem is, - we checked the "Show Duplicate Row Button?" option in the Application Definition file.
when we click the "Duplicate Row" button then we get the error:
javax.faces.el.EvaluationException: oracle.jbo.ReadOnlyAttrException: JBO-27008: Attribute set for UserId in view object Userpermission_under_Personalinfo_View_for_Owner failed
the attribute UserId is from Reference EO in the VO.
Can anybody suggest how we can make the duplicate work in this case?
And one more thing - If we check the "Show Duplicate Row Button?" option in JHeadstart Application Definition file, it shows the "Duplicate Row" button in the table layout. Is there any way so that we can put the "Duplicate Row" button in the form layout or do we have to do it using a .vm file modification ?
Any suggestion would be appreciated.
Thanks
Syed Jabbar
University of Windsor
Windsor, ON, Canada

Syed,
This is fixed in a later JHeadstart 10.1.3 release.
If you don't want to upgrade, you need to override the duplicate row method in JhsCollectionModel as follows:
public void duplicateRow(ActionEvent event)
DCIteratorBinding ib = getRangeBinding().getIteratorBinding();
int rangeSize = ib.getRangeSize();
int rowsInRange = ib.getAllRowsInRange().length;
int insertPos = 0;
if (!isNewRowsAtTop())
insertPos = rowsInRange < rangeSize? rowsInRange: rangeSize - 1;
Row currentRow = ib.getCurrentRow();
Row row = ib.getRowSetIterator().createRow();
row.setNewRowState(Row.STATUS_NEW);
ib.getRowSetIterator().insertRowAtRangeIndex(insertPos, row);
AttributeDef[] attrs = ib.getAttributeDefs();
for (int i = 0; i < attrs.length; i++)
if (!attrs.isPrimaryKey() &&
(attrs[i].getUpdateableFlag() == AttributeDef.UPDATEABLE ||
attrs[i].getUpdateableFlag() ==
AttributeDef.UPDATEABLE_WHILE_NEW))
String attrName = attrs[i].getName();
row.setAttribute(attrName, currentRow.getAttribute(attrName));
ib.setCurrentRowIndexInRange(insertPos);
discloseRow(row, true);
Steven Davelaar,
JHeadstart team.

SQL Server 2014 - Delete duplicate rows in a table except one

Hi all,
I have a table that contains duplicate rows. Using grouping I can easily identify those rows.
As of now, I count the grouped records and copy one of the group in a temporary table together with its count of elements. Than I loop over this table and delete N - 1 rows of each group from the original table. Unfortunately this is done with two cursors
and WHILE loops.
Is anybody out there who could give some hints how to handle deletion of duplicated rows leaving one in place using pure SQL? I tried a full delete of all elements of a group and then an INSERT of one row to recreate a row again. But this is not
very elegant and close to using cursors.
Regards Uwe

Hi all,
based on input below from Uri, I have developed a little more general solution for removal of duplicates. I did that, because I'm sure that for many beginners in SQL it is easier specialize a proposal by removing parts of it than adding new functionality.
Here is my code.
-- Remove duplicates from a table and document the deleted rows
DECLARE @MyTable Table (Col1 int, Col2 int, Col3 int, Col4 int);
DECLARE @OutTable Table (Col1 int, Col2 int, Col3 int, Col4 int);
INSERT INTO @MyTable VALUES
(1, 10, 100, 7),
(2, 20, 200, 2),
(2, 20, 200, 1),
(3, 30, 300, 3),
(1, 10, 200, 4),
(1, 10, 100, 5),
(1, 10, 100, 6);
WITH cte (RN, Col1, Col2, Col3, Col4)
AS (SELECT ROW_NUMBER() OVER (PARTITION BY Col1, Col2, Col3 ORDER BY (SELECT 0)) AS RN, Col1, Col2, Col3, Col4
FROM @MyTable)
DELETE FROM cte
OUTPUT DELETED.Col1, DELETED.Col2, DELETED.Col3, DELETED.Col4
INTO @OutTable (Col1, Col2, Col3, Col4)
WHERE RN > 1;
SELECT * FROM @OutTable;
And this is the result:
The MS 139444 article Satish mentioned is focussed on duplicate PKs removal. Thanks for the hint.
Thanks to you all for your rapid response.
Regards Uwe

How to delete duplicate rows in a table

hai,
i have a table which consists of hunderds of records and contains duplicate records also. how can i delete duplicate records.
thanks
ravi

This maybe a litle bit fast for deletion:-
delete from emp where rowid in (
2 SELECT rowid FROM emp
3 group by rowid,empno,ename,job
4 minus
5 SELECT min(rowid) FROM emp
6 group by empno,ename,job);

XML Query works on one database and not working on others with errLPX-00241

I have one query as below
SELECT EXTRACTVALUE
(VALUE (xml_node),
'/inspectionActivityArtifact/ns2:landfillTypes/ns2:type[position()=1]',
'xmlns:ns2="http://dwm.sw.xmlNamespace/InspectionActivity"'
) AS TEST
FROM artifacts a,TABLE (XMLSEQUENCE ((XMLTYPE (NVL (artifact_data, '<empty></empty>'))))) xml_node
this works well on other databases but not working on one database. Everything is same configuration and OS and everything.
I can work on other XML parsing which is working fine but not this query.
Any idea ?
Viral Dave

post versions and patch levels for both databases eg 10.2.0.1.0 and the error message you are getting.

Delete duplicates from query output

Hello,
I would like to delete duplicated records before the output of a SAP query
I tried writing in SQ02 in additional coding, under "Free coding":
data %dtab type standard table of /1BCDWB/IQ000000001053 with header line.
sort %dtab by vbeln posnr.
DELETE ADJACENT DUPLICATES FROM %dtab.
In this way I get the short dump when running the query: "Statement is not accessible.".
If I declare %dtab in the additional coding under "DATA", when I run the query I get the error "%DTAB" has already been declared.".
If I don't declare %dtab, I can't generate the infoset.
Do you have any suggestion about the way of obtaining this result?
Thanks in advance for your support.
Best regards,
Andrea

Hello,
I didn't find any solution to this problem. I choosed another way: I asked an ABAPer to develop a report to get the same information I wanted to get with the query, as in the ABAP report there's more "space to move".
Best regards,
Andrea

Generalise Delete Duplicate Rows Query work in any Database

Similar Messages

Maybe you are looking for