Trying to omit duplicate rows in a UNION
I am trying to omit duplicate rows from my union’ed join of 2 tables but am running into a problem which I am hoping someone can help me with. The following SQL works, but doesn’t omit the dups.
SELECT service_code, description_txt, effective_date, ROW_NUMBER() OVER (PARTITION BY service_code
ORDER BY effective_date DESC) AS rn
FROM
(SELECT s.service_code, s.effective_date, s.description_txt
FROM state_code_desc s
WHERE s.expire_date IS NULL
AND s.code_type='H'
AND s.coverage_type='W'
AND s.geo_area_code='USA'
AND s.state_abbr='GE'
UNION
SELECT f.service_code, f.effective_date, f.description_txt
FROM fed_code_desc f
WHERE f.code_type='H'
AND f.expire_date IS NULL
ORDER BY service_code
When I try to add the WHERE RN=1 immediately before the ORDER by service_code (last line of the SQL), I get the following error:
ORA-00904 – “RN” Invalid identifier
Can someone out there help me out, please?
Thanks!
Janet
P.S.-I'm running Oracle 9.1
Just a precesion,
It is not a question of analytic functions at all
In fact, column aliases are not usable in tbe same
query in the WHERE clause.
SQL> SELECT deptno,COUNT(*) OVER (PARTITION BY deptno) cnt
2 FROM emp;
DEPTNO CNT
10 2
10 2
20 2
20 2
30 5
30 5
30 5
30 5
30 5
9 rows selected.
SQL> SELECT deptno,COUNT(*) OVER (PARTITION BY deptno) cnt
2 FROM emp
3 WHERE cnt=1;
WHERE cnt=1
ERROR at line 3:
ORA-00904: "CNT": invalid identifier
SQL> SELECT deptno,sal*100 s,COUNT(*) OVER (PARTITION BY deptno) cnt
2 FROM emp
3 WHERE s=100;
WHERE s=100
ERROR at line 3:
ORA-00904: "S": invalid identifier
SQL> SELECT deptno,sal*100 s,COUNT(*) OVER (PARTITION BY deptno) cnt
2 FROM emp
3 WHERE (sal*100)=100;
no rows selected
SQL> SELECT deptno,sal*100 s,COUNT(*) OVER (PARTITION BY deptno) cnt
2 FROM emp
3 HAVING COUNT(*) OVER (PARTITION BY deptno)=2;
HAVING COUNT(*) OVER (PARTITION BY deptno)=2
ERROR at line 3:
ORA-30483: window functions are not allowed hereKhurram
Similar Messages
-
How to delete duplicate row in certain case
Hi all,</p>
<p>I need you help to delete my duplicate row.</p>
<p>Sample Data in table (<b>table1</b>)</p>
<p style="margin-top: 0; margin-bottom: 0"> </p>
<p style="margin-top: 0; margin-bottom: 0"><b>Timein
DateIn
locationin useridin
status</b></p>
<p style="margin-top: 0; margin-bottom: 0">08:20:00 01/09/2007
0001
U01
1</p>
<p style="margin-top: 0; margin-bottom: 0">08:10:30 01/09/2007
0004
U01
1</p>
<p style="margin-top: 0; margin-bottom: 0">07:20:00 01/09/2007
0006
U01
1</p>
<p style="margin-top: 0; margin-bottom: 0">08:14:00 01/09/2007
0001
U02
1</p>
<p style="margin-top: 0; margin-bottom: 0">06:10:30 01/09/2007
0004
U02
1</p>
<p style="margin-top: 0; margin-bottom: 0">08:10:10 01/09/2007
0006
U02
1</p>
<p> </p>
<p>I need to delete the row in table1 and leave the minimum <b>timein</b> in
that table by <b>useridin</b>.</p>
<p>The last output in table1 is like below.</p>
<p style="margin-top: 0; margin-bottom: 0"><b>Timein
DateIn
locationin useridin
status</b></p>
<p style="margin-top: 0; margin-bottom: 0">07:20:00 01/09/2007
0006
U01
1</p>
<p style="margin-top: 0; margin-bottom: 0">06:10:30 01/09/2007
0004
U02
1</p>
<p>Can i used below sql to output the result;</p>
<font FACE="Courier" SIZE="2">
<p style="margin-top: 0; margin-bottom: 0"></font>
<font FACE="Courier" SIZE="2" COLOR="#0000ff">delete</font><font FACE="Courier" SIZE="2">
</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">from</font><font FACE="Courier" SIZE="2">
<font color="#808000">table1</font> t1 </font>
<font FACE="Courier" SIZE="2" COLOR="#0000ff">where</font><font FACE="Courier" SIZE="2">
t1</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font><font FACE="Courier" SIZE="2" COLOR="#ff0000">rowid</font><font FACE="Courier" SIZE="2">
</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">></p>
</font><font FACE="Courier" SIZE="2">
<p style="margin-top: 0; margin-bottom: 0"></font>
<font FACE="Courier" SIZE="2" COLOR="#0000ff">(</font><font FACE="Courier" SIZE="2">
</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">select</font><font FACE="Courier" SIZE="2">
</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">min(</font><font FACE="Courier" SIZE="2">t2</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font><font FACE="Courier" SIZE="2" COLOR="#ff0000">rowID</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">)</font><font FACE="Courier" SIZE="2">
</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">from</font><font FACE="Courier" SIZE="2">
<font color="#808000">table1</font> t2</p>
<p style="margin-top: 0; margin-bottom: 0"></font>
<font FACE="Courier" SIZE="2" COLOR="#0000ff">where</font><font FACE="Courier" SIZE="2">
t1</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font><font face="Courier" size="2">Datein
= t2.Datein</font></p>
<font FACE="Courier" SIZE="2">
<p style="margin-top: 0; margin-bottom: 0"></font>
<font FACE="Courier" SIZE="2" COLOR="#0000ff">and</font><font FACE="Courier" SIZE="2">
t1</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font><font FACE="Courier" SIZE="2">Status
</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">=</font><font FACE="Courier" SIZE="2">
t2</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font><font FACE="Courier" SIZE="2">Status</font></p>
<p style="margin-top: 0; margin-bottom: 0">
<font FACE="Courier" SIZE="2" COLOR="#0000ff">and</font><font FACE="Courier" SIZE="2">
t1</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font>UserIdin
<font FACE="Courier" SIZE="2" COLOR="#0000ff">=</font><font FACE="Courier" SIZE="2">
t2</font><font FACE="Courier" SIZE="2" COLOR="#0000ff">.</font>UserIdin
<font FACE="Courier" SIZE="2" COLOR="#0000ff">order by t2.timein);</font></p>
<p style="margin-top: 0; margin-bottom: 0"> </p>
<p style="margin-top: 0; margin-bottom: 0"><font face="Courier" size="2">Thanks
in advancedHi Rob,
Just trying to chip in little bit!
Created Table As follows same as you, only two rows with duplicatem user, datein, and timein:
create table table1
as
select '08:20:00' timein, date '2007-09-01' datein, '0001' locationin, 'U01' useridin, 1 status from dual union all
select '08:10:30', date '2007-09-01', '0004', 'U01', 1 from dual union all
select '07:20:00', date '2007-09-01', '0006', 'U01', 1 from dual union all
select '08:14:00', date '2007-09-01', '0001', 'U02', 1 from dual union all
select '06:10:30', date '2007-09-01', '0004', 'U02', 1 from dual union all
select '06:10:30', date '2007-09-01', '0001', 'U02', 1 from dual
If I run your delete statement result row will be as follows:
SQL> select * from table1;
TIMEIN DATEIN LOCA USE STATUS
07:20:00 01-SEP-07 0006 U01 1
06:10:30 01-SEP-07 0004 U02 1
06:10:30 01-SEP-07 0001 U02 1
Here problem is that its not able to delete duplicate rows having same timein and timeout.
So I changed you query as follows:
DELETE FROM tab1
WHERE rowid NOT IN (
SELECT MIN(rowid)
FROM tab1
WHERE (UseridIn, TO_DATE(TO_CHAR(DateIn,'YYYYMMDD') || TimeIn,'YYYYMMDDHH24:MI:SS')) IN
( SELECT UseridIn, MIN(TO_DATE(TO_CHAR(DateIn,'YYYYMMDD') || timein,'YYYYMMDDHH24:MI:SS'))
FROM tab1
GROUP BY UseridIn)
GROUP BY UseridIn
If I run your delete statement with little bit of changes, the result row was as follows:
SQL> select * from table1;
TIMEIN DATEIN LOCA USE STATUS
07:20:00 01-SEP-07 0006 U01 1
06:10:30 01-SEP-07 0004 U02 1
Regards,
Raj -
I have five tables(A,B,C,D,E), and I am trying to check for duplicate rows in all the tables, I tried using inner join, but the first three join did not return any table. is there another way.the tables only have two columns , the email(different values)
and authour (which is the same person in the table)
when i tried
select * from TABLE A
INNER JOIN TABLE B
ON TABLEA. EMAIL = TABLEB.EMAIL
INNER JOIN TABLEC
ON TABLEA.EMAIL=TABLEC.EMAIL
it came back with no result, the other way I am thinking is to unioun all all the tables and try to use count and group by ,but this will only show me the duplicates and not the authours
please any other wayit came back with no result, the other way I am thinking is to unioun all all the tables and try to use count and group by (but I could not insert my result in a new table)
please any other way
I dont understand your point here...Are you trying the below?
Create Table T1(name varchar(50),Email Varchar(50))
Insert into T1 Values('SQL','[email protected]'),('.NET','[email protected]')
Create Table T2(name varchar(50),Email Varchar(50))
Insert into T2 Values('Server','[email protected]'),('BizTalk','[email protected]')
Create Table T3(name varchar(50),Email Varchar(50))
Insert into T2 Values('Sql','[email protected]'),('server','[email protected]')
;With cte as
(Select * From T1
Union All
Select * From T2
Union All
Select * From T3)
Select name,email, count(1) From cte Group by name , email having count(1)>1
Drop table T1,T2,T3
I was able to insert the union product in a table, and i used this
SELECT EMAIL,AUTHOR,COUNT(EMAIL) AS AMOUNT FROM ALLEMAIL
GROUP BY AUTHOR,EMAIL
ORDER BY AMOUNT DESC, email desc
but its showing the email and the count but its appearing like the duplicate is only associated with one authour -
Merge stmt trying to insert duplicate while the ON clause is based on index
Hi,
I am using a merge statement to insert record in a table. The table has an unique index based on four columns. The merge statement matches on all four column (a.column1 = b.column1 and a.column2 = b.column2 ... etc.) in the ON clause. If a match is found then it does nothing if not found corresponding record is inserted.
But for some reason it is trying to insert duplicate and I am getting unique constraint violated exception.
The database is huge and the source table contains more than 200,000 records.
Any idea how it can happen?How can it happen?
Easily if there is a duplicate in the data you want to merge in, bearing in mind that this is a set-based operation with the two sets of data evaluated using the standard read consistency model at the start of the query.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> DROP TABLE t1;
Table dropped.
SQL>
SQL> CREATE TABLE t1
2 (col1 NUMBER PRIMARY KEY);
Table created.
SQL>
SQL> MERGE
2 INTO t1
3 USING (SELECT 1 x
4 FROM DUAL
5 UNION ALL
6 SELECT 1
7 FROM DUAL) x
8 ON (x.x = t1.col1)
9 WHEN NOT MATCHED THEN
10 INSERT
11 VALUES
12 ( x.x);
MERGE
ERROR at line 1:
ORA-00001: unique constraint (ROS.SYS_C0088944) violated
SQL> rollback;
Rollback complete.
SQL> MERGE
2 INTO t1
3 USING (SELECT DISTINCT x
4 FROM (
5 SELECT 1 x
6 FROM DUAL
7 UNION ALL
8 SELECT 1
9 FROM DUAL)) x
10 ON (x.x = t1.col1)
11 WHEN NOT MATCHED THEN
12 INSERT
13 VALUES
14 ( x.x);
1 row merged.
SQL>Edited by: DomBrooks on Jan 15, 2010 9:36 AM -
Selecting duplicate rows based on a combination of columns as key
Hi ,
I have a table with 5 columns.
Code ID S_DATE E_DATE Name
1 23 01012001 null ABC
1 09 01012001 null XYZ
2 81 04022007 null TVU
1 43 03092008 null XXX
Now, I need write a select query to fetch the duplicate rows from the above table having the combination of (Code, S_DATE,E_DATE) as the key.
So from the above example, I need to get Row1 and Row2 as output (but not Row3 as it has a different S_DATE)
Thanks in advance for your suggestions.
Thanks
Edited by: thotaramesh on Mar 9, 2009 4:54 PMOn XE;
WITH sample_data AS (
SELECT 1 code,23 ID, '01012001' s_date, null e_date, 'ABC' NAME FROM dual UNION ALL
SELECT 1, 09, '01012001', null, 'XYZ' FROM dual UNION ALL
SELECT 2, 81, '04022007', null, 'TVU' FROM dual UNION ALL
SELECT 1, 43, '03092008', null, 'XXX' FROM dual)
SELECT code, ID, s_date, e_date, NAME
FROM (
SELECT
sample_data.*,
COUNT(*) over (PARTITION BY code, s_date, e_date) dups
FROM sample_data)
WHERE dups > 1;
CODE ID S_DATE E_DATE NAME
1 23 01012001 ABC
1 9 01012001 XYZ -
To avoid duplicate rows in oracle 8i
Can you suggest how to prevent duplicate rows while in situations where multiple refreshes from a single export file using Oracle 8.1.7?
Do i need to specify Commit=Y, Constraints= N and IGNORE=N while importing data?
Your suggestion would be appreciated.
Thanks in advance.PK or not pk, if you set ignore=n, the import will be abort in case of table already exists, so you won't have any duplicate rows.
SQL> create table mytable as select * from all_users union all select * from all_users;
Table created.
SQL> select count(*) from mytable;
COUNT(*)
14
SQL> quit
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> exp P97RPPRD/P97RPPRD tables=mytable log=test.log file=test.dmp
Export: Release 9.2.0.4.0 - Production on Tue Nov 14 13:32:40 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
Export done in UTF8 character set and AL16UTF16 NCHAR character set
About to export specified tables via Conventional Path ...
. . exporting table MYTABLE 14 rows exported
Export terminated successfully without warnings.
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> sqlplus P97RPPRD/P97RPPRD
SQL*Plus: Release 9.2.0.4.0 - Production on Tue Nov 14 13:32:48 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to:
Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
SQL> drop table mytable;
Table dropped.
SQL> quit
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> imp P97RPPRD/P97RPPRD tables=mytable log=test.log file=test.dmp
Import: Release 9.2.0.4.0 - Production on Tue Nov 14 13:33:07 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
Export file created by EXPORT:V09.02.00 via conventional path
import done in UTF8 character set and AL16UTF16 NCHAR character set
. importing P97RPPRD's objects into P97RPPRD
. . importing table "MYTABLE" 14 rows imported
Import terminated successfully without warnings.
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> imp P97RPPRD/P97RPPRD tables=mytable log=test.log file=test.dmp
Import: Release 9.2.0.4.0 - Production on Tue Nov 14 13:33:46 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
Export file created by EXPORT:V09.02.00 via conventional path
import done in UTF8 character set and AL16UTF16 NCHAR character set
. importing P97RPPRD's objects into P97RPPRD
IMP-00015: following statement failed because the object already exists:
"CREATE TABLE "MYTABLE" ("USERNAME" VARCHAR2(30), "USER_ID" NUMBER, "CREATED"
"" DATE) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 6553"
"6 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 0 FREELISTS 1 FREELI"
"ST GROUPS 1 BUFFER_POOL DEFAULT) LOGGING NOCOMPRESS"
Import terminated successfully with warnings.
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> sqlplus P97RPPRD/P97RPPRD
SQL*Plus: Release 9.2.0.4.0 - Production on Tue Nov 14 13:34:13 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to:
Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
SQL> select count(*) from mytable;
COUNT(*)
14
SQL>Now, if you haven't pk and set ignore=y, you will have duplicate lines, for sure :
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> imp P97RPPRD/P97RPPRD tables=mytable log=test.log file=test.dmp ignore=y
Import: Release 9.2.0.4.0 - Production on Tue Nov 14 13:37:33 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
Export file created by EXPORT:V09.02.00 via conventional path
import done in UTF8 character set and AL16UTF16 NCHAR character set
. importing P97RPPRD's objects into P97RPPRD
. . importing table "MYTABLE" 14 rows imported
Import terminated successfully without warnings.
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> sqlplus P97RPPRD/P97RPPRD
SQL*Plus: Release 9.2.0.4.0 - Production on Tue Nov 14 13:37:57 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to:
Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
SQL> select count(*) from mytable;
COUNT(*)
28
SQL> Now, test with pk and ignore=y, import doesn't insert the duplicate key, so you won't have any duplicate rows :
SQL> drop table mytable;
Table dropped.
SQL> create table mytable as select * from all_users;
Table created.
SQL> alter table mytable add (primary key (username));
Table altered.
SQL> insert into mytable select * from all_users;
insert into mytable select * from all_users
ERROR at line 1:
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
SQL> quit
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> exp P97RPPRD/P97RPPRD tables=mytable log=test.log file=test.dmp
Export: Release 9.2.0.4.0 - Production on Tue Nov 14 13:41:39 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
Export done in UTF8 character set and AL16UTF16 NCHAR character set
About to export specified tables via Conventional Path ...
. . exporting table MYTABLE 7 rows exported
Export terminated successfully without warnings.
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> imp P97RPPRD/P97RPPRD tables=mytable log=test.log file=test.dmp ignore=y
Import: Release 9.2.0.4.0 - Production on Tue Nov 14 13:41:49 2006
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.4.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.4.0 - Production
Export file created by EXPORT:V09.02.00 via conventional path
import done in UTF8 character set and AL16UTF16 NCHAR character set
. importing P97RPPRD's objects into P97RPPRD
. . importing table "MYTABLE"
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 SYS
Column 2 0
Column 3 14-NOV-2005:16:14:52
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 SYSTEM
Column 2 5
Column 3 14-NOV-2005:16:14:52
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 OUTLN
Column 2 11
Column 3 14-NOV-2005:16:15:05
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 DBSNMP
Column 2 19
Column 3 14-NOV-2005:16:24:23
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 PS
Column 2 21
Column 3 14-NOV-2005:16:36:41
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 P97RPPRD
Column 2 24
Column 3 14-NOV-2005:16:53:41
IMP-00019: row rejected due to ORACLE error 1
IMP-00003: ORACLE error 1 encountered
ORA-00001: unique constraint (P97RPPRD.SYS_C001422815) violated
Column 1 PEOPLE
Column 2 25
Column 3 14-NOV-2005:16:57:09 0 rows imported
Import terminated successfully with warnings.
[data/ora/adm/DBA_ORACLE i05c09 : oracle : P97RPPRD ]> HTH,
Nicolas. -
Deleting BOTH duplicate rows in Excel
Hi everyone,
I have just been introduced to Applescript and I was hoping to automate some things at my workplace. One of the first activities that I have been trying to figure out involves Excel 2008, which is running on a G5 (Latest version of Mac OS X updated just today). It involves deleting both duplicates of records--file numbers--in a long one column list (there may soon be thousands of these records piling up, with many duplicates.)
Someone over at MacScripter ("mikerickson" @ http://macscripter.net/viewtopic.php?id=32340) posted the following code for using Excel autofilter to delete only one instance of each duplicate row:
*tell application "Microsoft Excel"*
*set myMessySheet to get worksheet "Sheet1" of workbook "Workbook2.xls"*
*set myMessyRange to get current region of range "A1" of myMessySheet*
*set myCleanRange to get resize (get offset (myMessyRange) column offset ((count of columns of myMessyRange) + 1)) row size 1*
*advanced filter myMessyRange action filter copy copy to range myCleanRange with unique*
*-- optional deletes dirty range*
*delete range (entire column of (get resize myMessyRange column size ((count of columns of myMessyRange) + 1))) shift shift to left*
*end tell*
This code works great when I copy and paste in Applescript Editor. Now, I need to find all duplicates in my data and delete both of the numbers, leaving only the single instance records/numbers. There are even instances where a number shows up three or more times; it would be great to get rid of those as well.
Thank you in advance to anyone who can help me with this.I think you're going to have a hard time doing this because the original script used Excel to do the heavy lifting (i.e. filter the duplicates). The keyword with unique in the advanced filter is what's doing that.
Off hand, I don't see an equivalent option for the filter that finds singletons in the list. Without that you'll need to iterate through the list manually, excluding the duplicate entries. -
SQL Server 2014 - Delete duplicate rows in a table except one
Hi all,
I have a table that contains duplicate rows. Using grouping I can easily identify those rows.
As of now, I count the grouped records and copy one of the group in a temporary table together with its count of elements. Than I loop over this table and delete N - 1 rows of each group from the original table. Unfortunately this is done with two cursors
and WHILE loops.
Is anybody out there who could give some hints how to handle deletion of duplicated rows leaving one in place using pure SQL? I tried a full delete of all elements of a group and then an INSERT of one row to recreate a row again. But this is not
very elegant and close to using cursors.
Regards UweHi all,
based on input below from Uri, I have developed a little more general solution for removal of duplicates. I did that, because I'm sure that for many beginners in SQL it is easier specialize a proposal by removing parts of it than adding new functionality.
Here is my code.
-- Remove duplicates from a table and document the deleted rows
DECLARE @MyTable Table (Col1 int, Col2 int, Col3 int, Col4 int);
DECLARE @OutTable Table (Col1 int, Col2 int, Col3 int, Col4 int);
INSERT INTO @MyTable VALUES
(1, 10, 100, 7),
(2, 20, 200, 2),
(2, 20, 200, 1),
(3, 30, 300, 3),
(1, 10, 200, 4),
(1, 10, 100, 5),
(1, 10, 100, 6);
WITH cte (RN, Col1, Col2, Col3, Col4)
AS (SELECT ROW_NUMBER() OVER (PARTITION BY Col1, Col2, Col3 ORDER BY (SELECT 0)) AS RN, Col1, Col2, Col3, Col4
FROM @MyTable)
DELETE FROM cte
OUTPUT DELETED.Col1, DELETED.Col2, DELETED.Col3, DELETED.Col4
INTO @OutTable (Col1, Col2, Col3, Col4)
WHERE RN > 1;
SELECT * FROM @OutTable;
And this is the result:
The MS 139444 article Satish mentioned is focussed on duplicate PKs removal. Thanks for the hint.
Thanks to you all for your rapid response.
Regards Uwe -
How to Create primary key index with duplicate rows.
Hi All,
While rebuilding an index on a table , I am getting error that there are duplicate rows in a table.
Searching out the reason led me to an interesting observation.
Please follow.
SELECT * FROM user_ind_columns WHERE table_name='SERVICE_STATUS';
INDEX_NAME TABLE_NAME COLUMN_NAME COLUMN_POSITION COLUMN_LENGTH CHAR_LENGTH DESCEND
SERVICE_STATUS_PK SERVICE_STATUS SUBSCR_NO_RESETS 2 22 0 ASC
SERVICE_STATUS_PK SERVICE_STATUS STATUS_TYPE_ID 3 22 0 ASC
SERVICE_STATUS_PK SERVICE_STATUS ACTIVE_DT 4 7 0 ASC
SERVICE_STATUS_PK SERVICE_STATUS SUBSCR_NO 1 22 0 ASC
SELECT index_name,index_type,table_name,table_type,uniqueness, status,partitioned FROM user_indexes WHERE index_name='SERVICE_STATUS_PK';
INDEX_NAME INDEX_TYPE TABLE_NAME TABLE_TYPE UNIQUENESS STATUS PARTITIONED
SERVICE_STATUS_PK NORMAL SERVICE_STATUS TABLE UNIQUE VALID NO
SELECT constraint_name ,constraint_type,table_name,status,DEFERRABLE,DEFERRED,validated,index_name
FROM user_constraints WHERE constraint_name='SERVICE_STATUS_PK';
CONSTRAINT_NAME CONSTRAINT_TYPE TABLE_NAME STATUS DEFERRABLE DEFERRED VALIDATED INDEX_NAME
SERVICE_STATUS_PK P SERVICE_STATUS ENABLED NOT DEFERRABLE IMMEDIATE VALIDATED SERVICE_STATUS_PK
1. Using index scan:
SELECT COUNT (*)
FROM (SELECT subscr_no, active_dt, status_type_id, subscr_no_resets
FROM service_status
GROUP BY subscr_no, active_dt, status_type_id, subscr_no_resets
HAVING COUNT (*) > 1) ;
no rows returned
Explain plan:
Operation OBJECT Name ROWS Bytes Cost OBJECT Node IN/OUT PStart PStop
SELECT STATEMENT Optimizer MODE=CHOOSE 519 K 14756
FILTER
SORT GROUP BY NOSORT 519 K 7 M 14756
INDEX FULL SCAN ARBOR.SERVICE_STATUS_PK 10 M 158 M 49184
2. Using Full scan:
SELECT COUNT (*)
FROM (SELECT /*+ full(s) */ subscr_no, active_dt, status_type_id, subscr_no_resets
FROM service_status s
GROUP BY subscr_no, active_dt, status_type_id, subscr_no_resets
HAVING COUNT (*) > 1) ;
71054 rows returned.
Explain Plan:
Operation Object Name Rows Bytes Cost Object Node In/Out PStart PStop
SELECT STATEMENT Optimizer Mode=CHOOSE 1 24123
SORT AGGREGATE 1
VIEW 519 K 24123
FILTER
SORT GROUP BY 519 K 7 M 24123
TABLE ACCESS FULL ARBOR.SERVICE_STATUS 10 M 158 M 4234
Index SERVICE_STATUS_PK is a unique and composite primary key VALID index. And the constraint is ENABLED and VALIDATED still having duplicate rows in table.
How it is possible?
Is it an Oracle soft Bug??
Regards,
Saket Bansalsaket bansal wrote:
Values are inserted as single rows inserts through an GUI interface.And you still claim to have over 71K duplicate records, without the GUI getting any kind of errors?
That does not add up and can only be explained by a "bug".
I tried inserting a duplicate record but failed.
SQL> insert into service_status (select * from service_status where rownum <2);
insert into service_status (select * from service_status where rownum <2)
ERROR at line 1:
ORA-00001: unique constraint (ARBOR.SERVICE_STATUS_PK) violatedAre you really sure there is no other way data in this table is populated/manipulated in bulk? -
How to find duplicate row in sql query?
Hi All,
Please solve my query, find duplicate row and how to count its. your suggestion would be greatly appreciated.You can use group by and having.
SQL> WITH t
2 AS (SELECT LEVEL id
3 FROM DUAL
4 CONNECT BY LEVEL <= 5
5 UNION ALL
6 SELECT LEVEL + 2
7 FROM DUAL
8 CONNECT BY LEVEL <= 3)
9 SELECT *
10 FROM t;
ID
1
2
3
4
5
3
4
5
8 rows selected.
SQL> WITH t
2 AS (SELECT LEVEL id
3 FROM DUAL
4 CONNECT BY LEVEL <= 5
5 UNION ALL
6 SELECT LEVEL + 2
7 FROM DUAL
8 CONNECT BY LEVEL <= 3)
9 SELECT id, COUNT (*)
10 FROM t
11 GROUP BY id
12 HAVING COUNT (*) > 1;
ID COUNT(*)
3 2
4 2
5 2
SQL> -
ABAP / Query to Identify Duplicate Rows in Cube
Dear Experts,
We have a situation were some of our Cubes (due to compression and varying levels of forceful reloads) now contain duplicate rows.
What I need to know is :-
1) Is there a way to identify duplicate rows where one of the characteristics are different but all key figures are identical.
2) If so what is easier to achieve, ABAP routine/program or Query
3) If ABAP suggestions on how to code such
4) If query same.
What I need it to do is tell me which ClaimNo record (Primary Key) has duplicates and what characteristic has caused it.
I know I am asking for a lot but I really need to get this resolved as it's causing mayhem and trying to pinpoint these records is both time consuming and painful. What we are looking to do with the records is establish how they became duplicated so we can prevent this happening in the future.
Your help as always much appreciated.
Regards
Craig
Message was edited by: Craig ArmsteadHi Craig,
My previous answer can find out what all cubes and data targets have been loaded based on a request.
Actually for your query. The following information will be surely useful.
tables: /BIC/**(source ) , /BIC**(target)
parameter : fieldname like /BIC/****-fieldname ( In ur case the it can be primary key or Duplicate entry )
data: itab_source like /BIC/*** occurs 0 with header line,
itab_destination like /BIC/*** occurs 0 with header line.
data: wa_itab_destination like line of itab_destination.
select *
from /BIC/*****
into corresponding fields of table itab_source.
where fieldname = fieldname.
******Include your piece of code which is for deleting records
Delete adjacent duplicates from itab_source comparing characteristic ( i.e duplicate characteristic you specified)
****Use this to delete the ODS Data before writing into it
call function 'RSDRI_ODSO_DELETE_RFC'
exporting
i_odsobject = 'ODS Name'
i_delete_all = 'X'.
if sy-subrc = 0.
loop at itab_source.
move-corresponding itab_source to itab_destination.
append itab_destination.
endloop.
modify /BIC/*** from table itab_destination[]. target being written from itab.
commit work.
endif.
Please Reward points if this helps really.
Thanks,
Srinivas. -
Duplicate Rows In Oracle Pipelined Table Functions
Hi fellow oracle users,
I am trying to create an Oracle piplined table function that contains duplicate records. Whenever I try to pipe the same record twice, the duplicate record does not show up in the resulting pipelined table.
Here's a sample piece of SQL:
/* Type declarations */
TYPE MY_RECORD IS RECORD(
MY_NUM INTEGER
TYPE MY_TABLE IS TABLE OF MY_RECORD;
/* Pipelined function declaration */
FUNCTION MY_FUNCTION RETURN MY_TABLE PIPELINED IS
V_RECORD MY_RECORD;
BEGIN
-- insert first record
V_RECORD.MY_NUM = 1;
PIPE ROW (V_RECORD);
-- insert second duplicate record
V_RECORD.MY_NUM = 1;
PIPE ROW (V_RECORD);
-- return piplined table
RETURN;
END;
/* Statement to query pipelined function */
SELECT * FROM TABLE( MY_FUNCTION ); -- for some reason this only returns one record instead of two
I am trying to get the duplicate row to show up in the select statement. Any help would be greatly appreciated.Can you provide actual output from an SQL*Plus prompt trying this? I don't see the same behavior
SQL> SELECT * FROM V$VERSION;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
CORE 10.2.0.4.0 Production
TNS for 64-bit Windows: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production
SQL> CREATE TYPE MY_RECORD IS OBJECT(MY_NUM INTEGER);
2 /
Type created.
SQL> CREATE TYPE MY_TABLE IS TABLE OF MY_RECORD;
2 /
Type created.
SQL> CREATE OR REPLACE FUNCTION MY_FUNCTION
2 RETURN MY_TABLE
3 PIPELINED
4 AS
5 V_RECORD MY_RECORD;
6 BEGIN
7 V_RECORD.MY_NUM := 1;
8 PIPE ROW(V_RECORD);
9
10 V_RECORD.MY_NUM := 1;
11 PIPE ROW(V_RECORD);
12
13 RETURN;
14 END;
15 /
Function created.
SQL> SELECT * FROM TABLE(MY_FUNCTION);
MY_NUM
1
1 -
Duplicate rows displayed in list
I am using ADF, Struts and JSP.
On occasion, some of the pages that simply display a list of attibutes in a VO display duplicate rows - these views reference just one EO and in the majority of cases all of the default settings are used.
There seems to be no consistency as to what causes this, at first it would appear after a new row had been inserted but then is would just appear when listing the detilas.
If you refresh the list (event invokes executeQuery() to refresh the data from the DB) then the duplicate row goes away.
This happens on my development machine (using JDev 10.1.2) and on the Application Server deployment.
Has anyone esle seen this?
Does anyone know why this might be happening and what I can do to solve this problem?
Thanks
AlanMarcos - thanks for the reply - this is the type of answer I was looking for since I'm not able to find anywhere in my code that I'm inserting a new row.
However, I've tried this approach and am still experiencing the issue. I wanted to check that I had the right items checked / not checked. I don't have an item that says Disconnect Application Module 'unmanaged Release' - mine just says 'Release'. Here are my options (Jdev 10.1.3):
(default unchecked) Failover Transaction State Upon Managed Release
(default unchecked) Disconnect Application Module Upon Release
(default checked) Support Dynamic JDBC Credentials
(default checked) Reset Non-Transcational State Upon Unmanaged Release
(default checked) Enable Application Module Pooing
I've tried multiple combinations of checking / un-checking options (including having everything checked except the 2nd item above) and still get the same issue. I'm going to read up in the developer guide to see if I can understand these better.
By the way, can you let me know which passivate state options you have set for your View Object that was showing the duplicate row but is now working? For example, right-click on VO, select edit - what items are checked on the Tuning tab?
#1 Fill Last Page of Rows when Paging through Rowset
#2 Passivate State
#2a Including all Transient Values
#3 Retain View Link Accessor
Thanks -
Loading a flat table with duplicate rows in SQL server
Hi,
I'm trying to load a flat table with different levels that has duplicate rows. When I'm loading it from my source SQL server enviornment to target SQL server environment.. I can only load 63 rows out of the 1225 rows.. This is happenning because i had to define a primary key on the couple of columns..
When I just try to load it without a primary key, I get an error that PK needs to be defined for load to happen..
My table structure looks as follows -
Lvl1 Lvl2 Lvl3 Lvl4 AccountID AccountDesc
How do i load all rows of data in my target table using ODI?\
Please helpwhirlpool wrote:
Hi,
I'm trying to load a flat table What is a flat table ? Are you talking about FACT table ?
When I'm loading it from my source SQL server enviornment to target SQL server environment.. I can only load 63 rows out of the 1225 rows.. This is happenning because i had to define a primary key on the couple of columns..
When I just try to load it without a primary key, I get an error that PK needs to be defined for load to happen..
Which IKM is in use ? I can not remember an IKM which needs a PK . Incremental Update IKM needs a Update key which can be a PK or UK at database level or ODI level.
My table structure looks as follows -
Lvl1 Lvl2 Lvl3 Lvl4 AccountID AccountDesc
How do i load all rows of data in my target table using ODI?\
If you not bother about PK at target then you can go for SQL Control Append to load your target table.
Thanks,
Sutirtha -
Merge inserting duplicate rows
Hi,
I first check if a particular dept_no exists in the table, if not insert into the table. I am getting duplicate rows inserted.
This is the table data before Merge.
CUST_ID FIRST_NAME LAST_NAME DEPT_NO SALARY
1 Dan Morgan 10 100000
2 Jack Cline 20 100000
3 Helen Lofstrom 20 50000This is the table data after Merge.
3 rows merged.
CUST_ID FIRST_NAME LAST_NAME DEPT_NO SALARY
1 Dan Morgan 10 100000
2 Jack Cline 20 100000
3 Helen Lofstrom 20 50000
4 Ram Bharad 30 2000
4 Ram Bharad 30 2000
4 Ram Bharad 30 2000
6 rows selected.This is the code to merge
MERGE INTO customer c
USING ( SELECT cust_id,
first_name,
last_name,
dept_no,
salary
FROM customer ) e
ON ( c.dept_no = 30 )
WHEN MATCHED THEN
UPDATE SET
c.cust_id = 4,
c.first_name = 'Ram',
c.last_name = 'Bharad',
c.salary = 2000
WHEN NOT MATCHED THEN
INSERT
(c.cust_id,c.first_name, c.last_name,c.dept_no,c.salary)
VALUES
(4,'Ram','Bharad',30,2000);Shouldn't the above code insert once, and update twice?
ThanksYou have done it wrong...
Try like this
SQL> create table customer(cust_id number,first_name varchar2(20), last_name varchar2(20), dept_no number, salary number)
2 /
Table created.
SQL> insert into customer
2 select 1, 'Dan','Morgan',10,100000 from dual
3 union all
4 select 2, 'Jack','Cline',20,100000 from dual
5 union all
6 select 3, 'Helen','Lofstrom',20,50000 from dual
7 /
3 rows created.
SQL> commit
2 /
Commit complete.
SQL> select * from customer
2 /
CUST_ID FIRST_NAME LAST_NAME DEPT_NO SALARY
1 Dan Morgan 10 100000
2 Jack Cline 20 100000
3 Helen Lofstrom 20 50000
SQL> MERGE INTO customer c
2 USING ( SELECT 4 cust_id,
3 'Ram' first_name,
4 'Bharad' last_name,
5 30 dept_no,
6 2000 salary
7 FROM dual ) e
8 ON ( c.dept_no = e.dept_no )
9 WHEN MATCHED THEN
10 UPDATE SET
11 c.cust_id = e.cust_id,
12 c.first_name = e.first_name,
13 c.last_name = e.last_name,
14 c.salary = e.salary
15 WHEN NOT MATCHED THEN
16 INSERT
17 (c.cust_id,c.first_name, c.last_name,c.dept_no,c.salary)
18 VALUES
19 (e.cust_id,e.first_name, e.last_name,e.dept_no,e.salary);
1 row merged.
SQL> select * from customer
2 /
CUST_ID FIRST_NAME LAST_NAME DEPT_NO SALARY
1 Dan Morgan 10 100000
2 Jack Cline 20 100000
3 Helen Lofstrom 20 50000
4 Ram Bharad 30 2000but beware of the fact that you are joining based on dept_no so multiple rows can get updated with same cust_id. see below.
SQL> rollback
2 /
Rollback complete.
SQL> MERGE INTO customer c
2 USING ( SELECT 4 cust_id,
3 'Ram' first_name,
4 'Bharad' last_name,
5 20 dept_no,
6 2000 salary
7 FROM dual ) e
8 ON ( c.dept_no = e.dept_no )
9 WHEN MATCHED THEN
10 UPDATE SET
11 c.cust_id = e.cust_id,
12 c.first_name = e.first_name,
13 c.last_name = e.last_name,
14 c.salary = e.salary
15 WHEN NOT MATCHED THEN
16 INSERT
17 (c.cust_id,c.first_name, c.last_name,c.dept_no,c.salary)
18 VALUES
19 (e.cust_id,e.first_name, e.last_name,e.dept_no,e.salary);
2 rows merged.
SQL> select * from customer
2 /
CUST_ID FIRST_NAME LAST_NAME DEPT_NO SALARY
1 Dan Morgan 10 100000
4 Ram Bharad 20 2000
4 Ram Bharad 20 2000Thanks,
Karthick.
Maybe you are looking for
-
When I try to save an emailed-to-me document to an existing file on my hard drive by using the little "search" window the recipient file is inactive and does not open to recieve the document I want to save (nothing happens when I click on it and it's
-
I am working as web developer in india...One of my cient need this for his website... Awaiting for valuable replay
-
Dynamic Hyperlink in Publisher Report.
Can anyone suggest a way to build a Hyperlink Dynamicaly in a Report based on web site returned with the data?
-
Contact sheet images are grayed out
I have made an album of black and white images and want to print them as a series of contact sheets. The sheets look good when customized, but when I tell it to print, all but the first two of the previewed images are grayed out and not worth printin
-
Adobe authorization error 2004, what do i do?
Error 2004, authorization too many times. Now what?