Delete duplicate records based on condition
Hi Friends,
I am scratching my head as how to select one record from a group of duplicate records based upon column condition.
Let's say I have a table with following data :
ID START_DATE END_DATE ITEM_ID MULT RETAIL | RETAIL / MULT
1 10/17/2008 1/1/2009 83 3 7 | 2.3333
2 10/17/2008 1/1/2009 83 2 4 | 2
3 10/17/2008 1/1/2009 83 2 4 | 2
4 10/31/2008 1/1/2009 89 3 6 | 2
5 10/31/2008 1/1/2009 89 4 10 | 2.5
6 10/31/2008 1/1/2009 89 4 10 | 2.5
7 10/31/2008 1/1/2009 89 6 6 | 1
8 10/17/2008 10/23/2008 124 3 6 | 2From the above records the rule to identify duplicates is based on START_DATE,+END_DATE+,+ITEM_ID+.
Hence the duplicate sets are {1,2,3} and {4,5,6,7}.
Now I want to keep one record from each duplicate set which has lowest value for retail/mult(retail divided by mult) and delete rest.
So from the above table data, for duplicate set {1,2,3}, the min(retail/mult) is 2. But records 2 & 3 have same value i.e. 2
In that case pick either of those records and delete the records 1,2 (or 3).
All this while it was pretty straight forward for which I was using the below delete statement.
DELETE FROM table_x a
WHERE ROWID >
(SELECT MIN (ROWID)
FROM table_x b
WHERE a.ID = b.ID
AND a.start_date = b.start_date
AND a.end_date = b.end_date
AND a.item_id = b.item_id);Due to sudden requirement changes I need to change my SQL.
So, experts please throw some light on how to get away from this hurdle.
Thanks,
Raj.
Well, it was my mistake that I forgot to mention one more point in my earlier post.
Sentinel,
Your UPDATE perfectly works if I am updating only NEW_ID column.
But I have to update the STATUS_ID as well for these duplicate records.
ID START_DATE END_DATE ITEM_ID MULT RETAIL NEW_ID STATUS_ID | RETAIL / MULT
1 10/17/2008 1/1/2009 83 3 7 2 1 | 2.3333
2 10/17/2008 1/1/2009 83 2 4 | 2
3 10/17/2008 1/1/2009 83 2 4 2 1 | 2
4 10/31/2008 1/1/2009 89 3 6 7 1 | 2
5 10/31/2008 1/1/2009 89 4 10 7 1 | 2.5
6 10/31/2008 1/1/2009 89 4 10 7 1 | 2.5
7 10/31/2008 1/1/2009 89 6 6 | 1
8 10/17/2008 10/23/2008 124 3 6 | 2So if I have to update the status_id then there must be a where clause in the update statement.
WHERE ROW_NUM = 1
AND t2.id != t1.id
AND t2.START_DATE = t1.START_DATE
AND t2.END_DATE = t1.END_DATE
AND t2.ITEM_ID = t1.ITEM_IDInfact the entire where_ clause in the inner select statement must be in the update where clause, which makes it totally impossible as T2 is persistent only with in the first select statement.
Any thoughts please ?
I appreciate your efforts.
Definitely this is a very good learning curve. In all my experience I was always writing straight forward Update statements but not like this one. Very interesting.
Thanks,
Raj.
Similar Messages
-
How to delete duplicate records in 10 G.
how to delete duplicate records in 10 G.
--Here is one way to do it using a second table
create table temp1
(col1 char(1));
--Table created.
insert into temp1 (col1) values('A');
insert into temp1 (col1) values('B');
insert into temp1 (col1) values('B');
--1 row created.
--1 row created.
--1 row created.
create table temp2 as select distinct * from temp1;
--Table created.
--now you have a second table with no duplicates
--truncate your old table
truncate table temp1;
--Table truncated.
--and reload it with data from the new table
insert into temp1 select * from temp2;
--2 rows created.
--then drop the temp2 table
drop table temp2 -
Deleting duplicate rows based on three columns in Oracle 8i
Hi all,
The database we use is Oracle 8i.
The query below raises the too_many_rows exception when I launch an application. So I want to delete the duplicated rows :
select polarisation_1, polarisation_2
into v_pol1_tech, v_pol2_tech
from v_cfh_lien_element
where nom_lien = p_nom_lien
AND num_canal_1 = p_num_canal_1
AND freq_emise_1 = p_freq_emise_1;
Notice that with many possible values of the parameters p_nom_lien, p_num_canal_1 and p_freq_emise_1 then the exception is raised.
So how to delete generally the duplicated rows based on the three columns "nom_lien" , "num_canal_1" and "freq_emise_1" ?
Thank you very much indeed.Check the other thread with same question deleting duplicate rows based on three columns in Oracle 8i
-
Hi everyone,
I am writing a statement to delete duplicate records in my data.
For example:
select identnr,datum,klokin,klokuit,count(datum)
from fus_timesheets
group by identnr,datum,klokin,klokuit
having count(datum) > 1; Above code gives following result:
IDENTNR DATUM KLOKIN KLOKUIT COUNT(DATUM)
10376 14/09/2009 2
10376 16/09/2009 3
10376 15/09/2009 16 What i want to do is delete duplicate records, meaning that count(datum) will have a value of 1.
I have written following code to delete duplicate records :
declare
type tst_type is record
(identnr number
,datum varchar2(15)
,klokin varchar2(10)
,klokuit varchar2(10)
,aantal number(3));
type type_coll_tst
is table of tst_type;
t_tst type_coll_tst;
begin
select identnr,datum,klokin,klokuit,count(datum)
bulk collect into t_tst
from fus_timesheets
group by identnr,datum,klokin,klokuit
having count(datum) > 1;
for i in 1..t_tst.count loop
dbms_output.put_line(t_tst(i).identnr ||' '|| t_tst(i).datum||' '||t_tst(i).aantal);
end loop;
for i in 1..t_tst.count loop
delete from fus_timesheets
where identnr = t_tst(i).identnr
and klokin = t_tst(i).klokin
and klokuit = t_tst(i).klokuit
and datum = t_tst(i).datum
and rownum < t_tst(i).aantal;
end loop;
end;My delete statement is not working good.
Can someone please help me with the delete part?
Thanks,
DianaManage correctly null values:
for i in 1..t_tst.count loop
delete from fus_timesheets
where identnr = t_tst(i).identnr
and (klokin = t_tst(i).klokin or (t_tst(i).klokin is null and klokin is null))
and (klokuit = t_tst(i).klokuit or (t_tst(i).klokuit is null and klokuit is null))
and datum = t_tst(i).datum
and rownum < t_tst(i).aantal;
end loop;Max -
Hi,
Can I know is there any way to delete the duplicate recors in the internal table?
The code i'm using seem not working.Thanks.
SELECT *
INTO CORRESPONDING FIELDS
OF TABLE i_pa0009
FROM pa0001 INNER JOIN pa0009
ON pa0001pernr = pa0009pernr
WHERE bukrs IN s_ccode AND
bankn NE ''.
DELETE ADJACENT DUPLICATES FROM i_pa0009 COMPARING ALL FIELDS .
or
How can I move the company code to the i_PA0009 table with all i_PA0009 data?
SELECT bukrs pernr INTO TABLE i_tmp
FROM pa0001
WHERE bukrs IN s_ccode.
IF NOT i_tmp[] IS INITIAL.
DELETE ADJACENT DUPLICATES FROM i_comp.
SELECT * INTO CORRESPONDING FIELDS OF TABLE i_pa0009
FROM pa0009
FOR ALL ENTRIES IN i_tmp
WHERE pernr EQ i_tmp-pernr AND
bankn NE ''.
endif.Hi,
Yes, I would like to delete duplicate records in i_PA0009.
But It seem it not successful to delete all as I saw there's still remain duplicate records during the debugging.
Below here is the code i tried.
SELECT *
INTO CORRESPONDING FIELDS
OF TABLE i_p0009
FROM pa0001 INNER JOIN pa0009
ON pa0001pernr = pa0009pernr
WHERE bukrs IN s_ccode AND
bankn NE space.
SORT i_p0009 BY pernr.
DELETE ADJACENT DUPLICATES FROM i_p0009 COMPARING pernr endda begda.
Second approach. Could anyone advise on below how i can move the bukrs from i_tmp to the i_PA0009 table with the i_PA0009 data as well.
SELECT bukrs pernr INTO TABLE i_tmp
FROM pa0001
WHERE bukrs IN s_ccode.
IF NOT i_tmp[] IS INITIAL.
DELETE ADJACENT DUPLICATES FROM i_comp.
SELECT * INTO CORRESPONDING FIELDS OF TABLE i_pa0009
FROM pa0009
FOR ALL ENTRIES IN i_tmp
WHERE pernr EQ i_tmp-pernr AND
bankn NE space.
endif.
Edited by: Blue Sky on Feb 27, 2009 11:23 AM -
Deleting Duplicate Records -EIM Import Account & Contact
Hi,
Can anyone give me the query to delete duplicate records both from legacy as well as from the tables imported.Try this..
DELETE
FROM table
WHERE ROWID NOT IN(SELECT MAX(ROWID)
FROM table
GROUP BY column)
For which column u have duplicate rows that column u have to give in group by clause so except max of rowid it will delete all records. -
Delete Duplicate Record leaving 1 record
Hi friends
I want to delete a record from table where there is duplicate which i am able to do it , but i want while deleting it should leave 1 record.
for ex
1111
1111
2222
2222
it should delete all duplicate and leave the unique one.
1111
2222
sandyHere are some ideas:
AskTom: Deleting duplicate records
AskTom: Deleting duplicate records without using rowid and rownum
AskTom: to remove duplicate records -
Delete duplicate records issue
Dear experts,
i did one development 1 month back for send mail.
In that i fetch email id of F_cord from one table(zcoordinator) based on some condition.now they want to changes in that object.
now two new fields added in that table ven_cord and HOD, both have some mail id.
As per their requirement , i need to fetch the ven_cord mail id and HOD mail id from that same table for priviously fetched F_cord.
And I need to delete duplicate ven_cord from tha table.
So for this should i need to create another structure or any ohther ways to solve the requirement.
\Plase suggest ?
Kin Regards,
Ajit@sanjeev,
I do't have create a separate structure for fetch these two fields ?
As per you, I need to add these two new fields to that previous structure which I created for F_CORD, Wright ?
And during fetch statement I need to add these two new fields. wright ?
Then how to delete the duplicate ven_cord for the perticular F_CORD.
Please suggst ?
I can't paste the code here ?
Regards,
Ajit -
To delete duplicate records from internal table
hi friends,
i have to delete records from internal table based on following criterion.
total fields are 7.
out of which if 4 fields are same and 5th field is different,then both records must be deleted.
in case all five fields are same,the program should do nothing.
for example.
if there are 3 records as follows
a1 b1 c1 d1 e1 f g
a1 b1 c1 d1 e2 w r
a1 b1 c1 d1 e1 j l
then first two records should be deleted as four fields are same but fifth(e) field differs.
but third record should remain as it is evenif first five fields are same for first and third record.
values of last two fields need not to be consider for deleting the records.LOOP AT ITAB.
V_FILED5 = ITAB-F5. "to compare later
V_TABIX = SY-TABIX. "used to delete if condition not matches
READ TABLE ITAB WITH KEY F1 = ITAB-F1
F2 = ITAB-F2
F3 = ITAB-F3
F4 = ITAB-F4.
IF SY-SUBRC = 0.
IF ITAB-F5 <> V_FIELD5.
*--both the records to be deleted,as Field5 is different.
DELETE ITAB INDEX SY-TABIX. "deletes that record
DELETE ITAB INDEX V_TABIX. "deletes the current record
ENDIF.
ENDIF.
ENDLOOP.
Message was edited by: Srikanth Kidambi
added comments
Message was edited by: Srikanth Kidambi -
How to delete Duplicate records in IT2006
Dear Experts
We have a situation like where we have duplicate records with same start and end dates in IT2006. This is because of the incorrect configuration which we have corrected now, but we need to do a clean-up for the existing duplicate records. Any idea on how to clean it? I ran report RPTKOK00 to find these duplicates but I could not delete the duplicate/inconsistenct record using report RPTBPC10 or HNZUPTC0, i Could only delete the deductions happened in the record.
Is there any standard report/any other means of deleting the duplicate records created in IT2006?
Thanks in advance for all your help.
Regards
Vignesh.You could probably use se16n to identify the duplicates and create the list of quotas to delete, and you could probably use t-code lsmw to write up a script to delete them, but be aware that you can't delete a Quota if it's been deducted from.
You'd have to delete the Absence/Attendance first, then delete the Quota, then recreate the Absence/Attendance. -
How to delete duplicate records in cube
Hi,
can u help me how to delete the duplicate records in my cube
and tell me some predifined cubes and data sourcess for MM and SD modulesHi Anne,
about "duplicate records" could you be more precise?.
The must be at least one different Characteristic to distinguish one record from the other (at least Request ID). In order to delete Data from InfoCubes (selectively) use ABAP Report RSDRD_DELETE_FACTS (be carefull it does not request any confirmation as in RSA1 ...).
About MM and SD Cubes see RSA1 -> Business Content -> InfoProvider by InfoAreas. See also for MetadataRepository about the same InfoProviders.
About DataSources just execute TCode LBWE in you source sys: there you see all LO-Cockipt Extrators.
Hope it helps (and if so remember reward points)
GFV -
How to delete duplicate records in all tables of the database
I would like to be able to delete all duplicate records in all the tables of the database. Many of the tables have LONG columns.
Thanks.Hello
To delete duplicates from an individual table you can use a construct like:
DELETE FROM
table_a del_tab
WHERE
del_tab.ROWID <> (SELECT
MAX(dups_tab.ROWID)
FROM
table_a dups_tab
WHERE
dups_tab.col1 = del_tab.col1
AND
dups_tab.col2 = del_tab.col2
)You can then apply this to any table you want. The only differences will be the columns that you join on in the sub query. If you want to look for duplicated data in the long columns themselves, I'm pretty sure you're going to need to do some PL/SQL coding or maybe convert them to blobs or something.
HTH
David -
How to delete duplicate record in Query report
Hi Experts,
I had created an infoset and query in my sap, but I want to delete some duplicate records before the list out put.Please can we add some further codes in the Extras code to delete duplicates? And how do it? Would you please give me a simple brief.
JoeHi,
You can try to restrict in the filter area in query designer with the values for characteristic which gives correct
result.
But still i would suggest that in the cube you keep not the duplicate records as this is not your requirement and giving
you wrong result.
So you can reload the correct records in the cube inorder to avoid such problems even in future.
Regards,
Amit -
Delete duplicate rows -- based on 4 columns -- how?
I asked this question on how to delete duplicates recently and received this suggestion which works well -- except --
With CTE AS
SELECT *, ROW_NUMBER() OVER (Partition by fld1 order by fld1) RowNum
FROM #tmpA
DELETE From CTE
WHERE RowNum > 1
-- the actual table I need to delete duplicate rows on is based on 4 columns. The following table contains 14,462 rows of which 14,348 are distinct -- based on the following 4 colums. Below is an image of a sample of the data contained in the
table for my question and to the right of that data is the column structures (data types). Is it possible to do something like the above example suggestion with the table in the image below? How to do that? I need to delete rows so that 14462
goes down to 14348. If I only reference one column for the delete -- this would delete like 7000+ rows. I only need to remove 114 rows.
Rich PAdd the other 3 columns to the partition.
Jason Long -
How to delete Duplicate records from the informatica target using mapping?
Hi, I have a scenario like below: In my mapping I have a source, which may containg unique records or duplicate records. Source and target are different tables. I have a target in my mapping which contains duplicate records. Using Mapping I have to delete the duplicate records in the target table. Target table does not contain any surrogate key. We can use target table as a look up table, but target table cannot be used as source in the mapping. We cannot use post SQL.
Hi All, I have multiple flat files which i need to load in a single table.I did that using indirect option at session level.But need to dig out on how to populate substring of header in name column in target table. i have two columns Id and Name. in all input file I have only one column 'id' with header like H|ABCD|Date. I need to populate target like below example. File 1 File2 H|ABCD|Date. H|EFGH|Date.1 42 5 3 6 Target tale: Id Name1 ABCD2 ABCD3 ABCD4 EFGH5 EFGH6 EFGH can anyone help on what should be the logic to get this data in a table in informatica.
Maybe you are looking for
-
I have upgraded to 10.6.8 Snow Leopard from Leopard on my macbook pro and now cannot read and write pages. Pages in app store is for 10.10 and above. I don't have a pages application. Where can I get pages for snow leopard?
-
Insert byte[] into postgres text-field
Hi, i made some tries on using postgresql and jdbc, i noticed that for some reason the postgres jdbc.driver does not convert text from the db correct to UNICODE. The db is in ISO-8859-1. So, i tried to write a workaround to this and was successfull f
-
Hi all, In my prod alert log I found that there was some sql query which were running for a long time and get out with some ORA error. I need to know the whole SQL query that was running. Is there any table (in backend) through which I can find out w
-
Keyword list length in 2.1
Hi all... not sure if Ive done something wrong here, but have imported catalog from 1.3 to 2.1 and a weird thing happened to the keyword list. I can only go down about halfway with my scrolling.... get to the letter 'I' and no further. If I delete a
-
How to activate FIN_LOC_CI_8 in EHP6
Hi, I am trying to implement note 738919 on EHP 6. For this I need to first activate business function FIN_LOC_CI_8 and EA-FIN, when I am trying to activate these business function I am getting message 'no activation necessary'. Please guide me how t