Need to test if a column have unique values or not

Question

Hi all,
in ETL process I need to check if some cols have unique values or not using sql or plsql.
Suppose we need to load a big file data with external table initially and then test if values
on one or more columns have unique values  in order to proceed with ETL process.
What is the faster test I can execute to verify that a column have unique values or not?
It's better for the ETL performance, use:
a. techniques regard constraints like described on Ask tom forum
"ENABLE NOVALIDATE validating existing data"
(Ask Tom "ENABLE NOVALIDATE validating existing da...")
b. "simply" query on the data?
like this:
select count(count(*)) distinct_count,
         sum(count(*)) total_count,
         sum(case when count(*) = 1 then 1 else null end) non_distinct_groups,
         sum(case when count(*) > 1 then 1 else null end) distinct_groups
from hr.employees a
group by A.JOB_ID
c. use analytics function?
d. use some feature directly on external table?
Bye in advance

Answer 1

Here is the example to handling the errrs using LOG_ERRORS into concept. You will check this and let me know if any doubt you have
DATAFILE:-
1000,ANN,ZZ105
1001,KARTHI,ZZ106
1002,PRAVEEN,ZZ109
1002,PARTHA,ZZ107
1003,SATHYA,ZZ108
1000,ANN,ZZ105
----- Original Table With unique constraints
SQL> CREATE TABLE tab_uniqtest(student_id     NUMBER(10) UNIQUE,
                          student_name   VARCHAR2(15),
                                                  course_name    VARCHAR2(15)
  2    3    4
Table created.
----- External table
SQL> CREATE TABLE tab_extuniqtest(student_id     NUMBER(10),
  2                               student_name   VARCHAR2(15),
  3                                                  course_name    VARCHAR2(15)
  4                              )
  5  ORGANIZATION EXTERNAL
  6  (
  7  DEFAULT DIRECTORY ann_dir
  8  ACCESS PARAMETERS
  9  (
10    RECORDS DELIMITED BY NEWLINE
11    BADFILE 'tabextuniqtest_badfile.txt'
12    LOGFILE 'tabextuniqtest_logfile.txt'
13    FIELDS TERMINATED BY ','
14    MISSING FIELD VALUES ARE NULL
15    REJECT ROWS WITH ALL NULL FIELDS
16    (student_id,student_name,course_name)
17  )
18  LOCATION ('unique_check.csv')
19  )
20  REJECT LIMIT UNLIMITED;
Table created.
---- Error logging table to log the errors
SQL> CREATE TABLE dmlerrlog_uniqtest(ORA_ERR_NUMBER$     NUMBER ,
  2                                 ORA_ERR_MESG$       VARCHAR2(2000),
  3                                 ORA_ERR_ROWID$      ROWID,
  4                                 ORA_ERR_OPTYP$      VARCHAR2(2),
  5                                 ORA_ERR_TAG$        VARCHAR2(4000),
  6                                 inserted_dt         VARCHAR2(50) DEFAULT TO_CHAR(SYSDATE,'YYYY-MM-DD'),
  7                                 student_id              VARCHAR2(10)
  8                                  );
Table created.
---- Procedure to insert from external table
SQL> CREATE OR REPLACE PROCEDURE proc_uniqtest
  2  AS
  3  v_errcnt NUMBER;
  4  BEGIN
  5      INSERT INTO tab_uniqtest
  6      SELECT * FROM tab_extuniqtest
  7      LOG ERRORS INTO dmlerrlog_uniqtest('PROC_UNIQTEST@TAB_UNIQTEST') REJECT LIMIT UNLIMITED;
  8      SELECT COUNT(1) into v_errcnt
  9      FROM dmlerrlog_uniqtest
10      WHERE ORA_ERR_TAG$ = 'PROC_UNIQTEST@TAB_UNIQTEST';
11       IF(v_errcnt > 0) THEN
12       ROLLBACK;
13      ELSE
14        COMMIT;
15       END IF;
16      DBMS_OUTPUT.PUT_LINE ( 'Procedure PROC_UNIQTEST is completed with ' || v_errcnt || ' errors') ;
17  EXCEPTION
18   WHEN OTHERS THEN
19    RAISE;
20  END proc_uniqtest;
21  /
Procedure created.
SQL> SET SERVEROUTPUT ON
SQL> EXEC proc_uniqtest;
Procedure PROC_UNIQTEST is completed with 2 errors
PL/SQL procedure successfully completed.
SQL> SELECT STUDENT_ID,ORA_ERR_MESG$ FROM dmlerrlog_uniqtest;
STUDENT_ID                     ORA_ERR_MESG$
1002                           ORA-00001: unique constraint (
                               SCOTT.SYS_C0037530) violated
1000                           ORA-00001: unique constraint (
                               SCOTT.SYS_C0037530) violated