SQL Loader Inserting Log File Statistics to a table

Question

Hello.
I'm contemplating how to approach gathering the statistics from the SQL Loader log file to insert them into a table. I've approached this from a Korn Shell Script perspective previously, but now that I'm working in a Windows environment and my peers aren't keen about batch files and scripting I thought I'd attempt to use SQL Loader itself to read the log file and insert one or more records into a table that tracks data uploads. Has anyone created a control file that accomplishes this?
My current environment:
Windows 2003 Server
SQL*Loader: Release 10.2.0.1.0
Thanks,
Luke

Answer 1

Hello.
Learned a little about inserting into multiple tables with delimited records. Here is my current tested control file:
LOAD DATA
APPEND
INTO TABLE upload_log
WHEN (1:12) = 'SQL*Loader: '
FIELDS TERMINATED BY WHITESPACE
TRAILING NULLCOLS
(  upload_log_id   RECNUM
, filler_field_0  FILLER
, filler_field_1  FILLER
, filler_field_2  FILLER
, filler_field_3  FILLER
, filler_field_4  FILLER
, filler_field_5  FILLER
, day_of_week
, month
, day_of_month
, time_of_day
, year
, log_started_on          "TO_DATE((:month ||' '|| :day_of_month ||' '|| :time_of_day ||' '|| :year), 'Mon DD HH24:MI:SS YYYY')"
INTO TABLE upload_log
WHEN (1:11) = 'Data File: '
FIELDS TERMINATED BY ':'
(  upload_log_id    RECNUM
, filler_field_0   FILLER  POSITION(1)
, input_file_name          "TRIM(:input_file_name)"
INTO TABLE upload_log
WHEN (1:6) = 'Table '
FIELDS TERMINATED BY WHITESPACE
(  upload_log_id   RECNUM
, filler_field_0  FILLER  POSITION(1)
, table_name              "RTRIM(:table_name, ',')"
INTO TABLE upload_rejects
WHEN (1:7) = 'Record '
FIELDS TERMINATED BY ':'
(  upload_rejects_id  RECNUM
, record_number      POSITION(1)  "TO_NUMBER(SUBSTR(:record_number,8,20))"
, reason
INTO TABLE upload_rejects
WHEN (1:4) = 'ORA-'
FIELDS TERMINATED BY ':'
(  upload_rejects_id  RECNUM
, error_code         POSITION(1)
, error_desc
INTO TABLE upload_log
WHEN (1:22) = 'Total logical records '
FIELDS TERMINATED BY WHITESPACE
(  upload_log_id      RECNUM
, filler_field_0     FILLER  POSITION(1)
, filler_field_1     FILLER
, filler_field_2     FILLER
, action                     "RTRIM(:action, ':')"
, number_of_records
INTO TABLE upload_log
WHEN (1:13) = 'Run began on '
FIELDS TERMINATED BY WHITESPACE
TRAILING NULLCOLS
(  upload_log_id   RECNUM
, filler_field_0  FILLER  POSITION(1)
, filler_field_1  FILLER
, filler_field_2  FILLER
, day_of_week
, month
, day_of_month
, time_of_day
, year
, run_began_on            "TO_DATE((:month ||' '|| :day_of_month ||' '|| :time_of_day ||' '|| :year), 'Mon DD HH24:MI:SS YYYY')"
INTO TABLE upload_log
WHEN (1:13) = 'Run ended on '
FIELDS TERMINATED BY WHITESPACE
TRAILING NULLCOLS
(  upload_log_id   RECNUM
, filler_field_0  FILLER  POSITION(1)
, filler_field_1  FILLER
, filler_field_2  FILLER
, day_of_week
, month
, day_of_month
, time_of_day
, year
, run_ended_on            "TO_DATE((:month ||' '|| :day_of_month ||' '|| :time_of_day ||' '|| :year), 'Mon DD HH24:MI:SS YYYY')"
INTO TABLE upload_log
WHEN (1:18) = 'Elapsed time was: '
FIELDS TERMINATED BY ':'
(  upload_log_id   RECNUM
, filler_field_0  FILLER  POSITION(1)
, filler_field_1  FILLER
, filler_field_2  FILLER
, elapsed_time
INTO TABLE upload_log
WHEN (1:14) = 'CPU time was: '
FIELDS TERMINATED BY ':'
(  upload_log_id   RECNUM
, filler_field_0  FILLER  POSITION(1)
, filler_field_1  FILLER
, filler_field_2  FILLER
, cpu_time
)Here are the basic table create scripts:
TRUNCATE TABLE upload_log;
DROP TABLE upload_log;
CREATE TABLE upload_log
(  upload_log_id      INTEGER
, day_of_week        VARCHAR2(  3)
, month              VARCHAR2(  3)
, day_of_month       INTEGER
, time_of_day        VARCHAR2(  8)
, year               INTEGER
, log_started_on     DATE
, input_file_name    VARCHAR2(255)
, table_name         VARCHAR2( 30)
, action             VARCHAR2( 10)
, number_of_records  INTEGER
, run_began_on       DATE
, run_ended_on       DATE
, elapsed_time       VARCHAR2(  8)
, cpu_time           VARCHAR2(  8)
TRUNCATE TABLE upload_rejects;
DROP TABLE upload_rejects;
CREATE TABLE upload_rejects
(  upload_rejects_id  INTEGER
, record_number      INTEGER
, reason             VARCHAR2(255)
, error_code         VARCHAR2(  9)
, error_desc         VARCHAR2(255)
);Now, if I could only insert a single record to the upload_log table (per table logged); adding separate columns for skipped, read, rejected, discarded quantities. Any advice on how to use SQL Loader to do this (writing a procedure would be fairly simple, but I'd like to perform all of the work in one place if at all possible)?
Thanks,
Luke
Edited by: Luke Mackey on Nov 12, 2009 4:28 PM