Script to make a non-unique column unique?

Question

Hi all,
Currently I have a table that contains nothing but ID's (the rest of the table is to be populated at a later date).
In this table there are around 500,000,000 ID's, of which only about 35,000,000 are unique.
I need to run a script that will find all duplicate entries of an ID and delete them (only the duplicates). Only one copy of each ID should be in the table so that I can make the column the PK.
I can't think of how to do this. Any ideas?
Thanks,
fakelvis

Answer 1

Hello
There's a couple of ways you could do this, but I think creating a new table with the distinct list of ids, truncating the original and re-inserting the distinct may be the most optimal...if I understand correctly.
SQL> --Set up some test data to use with method 1 CTAS + Truncate
SQL> CREATE TABLE dt_test_big_tab AS SELECT object_id FROM dba_objects WHERE object_id IS NOT NUL
  2  /
Table created.
SQL> INSERT INTO dt_test_big_tab SELECT * from dt_test_big_tab
  2  /
36210 rows created.
SQL> /
72420 rows created.
SQL> /
144840 rows created.
SQL> /
289680 rows created.
SQL> /
579360 rows created.
SQL> /
1158720 rows created.
SQL> commit;
Commit complete.
SQL> SELECT COUNT(*) from dt_test_big_tab
  2  /
  COUNT(*)
   2317440
SQL> SELECT COUNT(DISTINCT object_id) from dt_test_big_tab
  2  /
COUNT(DISTINCTOBJECT_ID)
                   36210
SQL> --Set up a copy of the test data to use with method 2 - Delete duplicates
SQL> CREATE TABLE dt_test_big_tab_2 as select * from dt_test_big_tab
  2  /
Table created.
SQL> set timi on
SQL> --Method one, ctas + truncate
SQL> CREATE TABLE dt_test_unique AS SELECT DISTINCT object_id FROM dt_test_big_tab
  2  /
Table created.
Elapsed: 00:00:02.07
SQL> SELECT COUNT(*) from dt_test_unique
  2  /
  COUNT(*)
     36210
Elapsed: 00:00:00.00
SQL> SELECT COUNT(DISTINCT object_id) from dt_test_unique
  2  /
COUNT(DISTINCTOBJECT_ID)
                   36210
Elapsed: 00:00:00.00
SQL> TRUNCATE TABLE dt_test_big_tab
  2  /
Table truncated.
Elapsed: 00:00:01.02
SQL> INSERT INTO dt_test_big_tab(object_id) SELECT object_id FROM dt_test_unique
  2  /
36210 rows created.
Elapsed: 00:00:00.00
SQL> SELECT COUNT(*) from dt_test_big_tab
  2  /
  COUNT(*)
     36210
Elapsed: 00:00:00.01
SQL> SELECT COUNT(DISTINCT object_id) from dt_test_big_tab
  2  /
COUNT(DISTINCTOBJECT_ID)
                   36210
Elapsed: 00:00:00.00
SQL> --Method 2, delete duplicates
SQL> DELETE
  2  FROM
  3     dt_test_big_tab_2
  4  WHERE
  5     rowid IN(SELECT
  6                             del_rowid
  7                     FROM
  8                             (SELECT
  9                                     rowid del_rowid,
10                                     ROW_NUMBER() OVER(partition by object_id ORDER BY object_id) rn
11                             FROM
12                                     dt_test_big_tab_2
13                             )
14                     WHERE
15                                     rn > 1
16                             );
2281230 rows deleted.
Elapsed: 00:05:39.06
SQL> select count(*) from dt_test_big_tab_2;
  COUNT(*)
     36210
Elapsed: 00:00:00.03
SQL> select count(distinct object_id) from dt_test_big_tab_2;
COUNT(DISTINCTOBJECT_ID)
                   36210
Elapsed: 00:00:00.00
SQL>HTH
David