Parse file text file

Question

Hi,
I like read Apache error log using external table.
For this I did create directory APACHE_LOG and grant read permission to it for my schema.
Then external table
CREATE TABLE EXT_APACHE_ERROR_LOG
    "TEXT" VARCHAR2(2000 BYTE)
  ORGANIZATION EXTERNAL
    TYPE ORACLE_LOADER DEFAULT DIRECTORY "APACHE_LOG" ACCESS PARAMETERS ( records delimited BY newline nobadfile nodiscardfile nologfile ) LOCATION ( "APACHE_LOG":'error.log' )
  REJECT LIMIT UNLIMITED;Log format is like below
[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration: /export/home/live/ap/htdocs/testI have try read file using query
SELECT REGEXP_SUBSTR(text, '[^\[]+(.*?)')      AS mesg_date,
  REGEXP_SUBSTR(text, '\[([a-z]*?)\]')         AS mesg_type,
  REGEXP_SUBSTR(text, '\[client ([0-9\.]*)\]') AS client,
  REGEXP_SUBSTR(text, '\] (.*)$')              AS mesg
FROM ext_apache_error_log;Output it is not quite what I like have as I'm bad with regexp.
I like have result in 4 columns like
MESG_DATE               MESG_TYPE     CLIENT          MESG
Wed Oct 11 14:32:52 2000     error          127.0.01     client denied by server configuration: /export/home/live/ap/htdocs/testSo I need bit help
What is best method parse that log format?
Can I have those 4 columns some way already to external table?
Thanks advance
Regards,
Jari

Answer 1

Hi,
I just noticed that log have lines where is no data for third column (client).
This query demonstrate situation
WITH test_data AS
  (SELECT '[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration: /export/home/live/ap/htdocs/test' text
  FROM dual
  UNION ALL
  SELECT '[Wed Oct 11 14:32:52 2000] [notice] SIGUSR1 received.  Doing graceful restart' text
  FROM dual
SELECT REGEXP_replace(text, '^\[(.*?)\].*$', '\1')   AS mesg_date,
  REGEXP_SUBSTR(text, '\[(.*?)\]',1,2)               AS mesg_type,
  REGEXP_SUBSTR(text, '\[(.*?)\]',1,3)               AS client,
  REGEXP_replace(text, '^(\[.*?\]\s){3}(.*)$', '\2') AS mesg
FROM test_data ;In case there is no client info whole line from file is in fourth column (mesg).
What could be best way handle this case so that client column is null but only the message should shown in mesg column ?
Thanks advance
Regards,
Jari
Edited by: jarola on Aug 31, 2011 8:24 PM
I did come to this query
WITH test_data AS
  (SELECT '[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration: /export/home/live/ap/htdocs/test' text
  FROM dual
  UNION ALL
  SELECT '[Wed Oct 11 14:32:52 2000] [notice] SIGUSR1 received.  Doing graceful restart' text
  FROM dual
SELECT TO_DATE(REGEXP_REPLACE(text, '^\[(.*?)\].*$', '\1'),'Dy Mon DD HH24:MI:SS YYYY', 'NLS_DATE_LANGUAGE = AMERICAN') AS mesg_date,
  REGEXP_SUBSTR(text, '\[(.*?)\]',1,2)                                                                                  AS mesg_type,
  REGEXP_SUBSTR(text, '\[(.*?)\]',1,3)                                                                                  AS client,
  REGEXP_REPLACE(REGEXP_REPLACE(text, '^(\[.*?\]\s){3}(.*)$','\2'), '^(\[.*?\]\s){2}(.*)$','\2')                        AS mesg
FROM test_data ;It seems work. If somebody have better query please post.
And also if there is in good way combine expression that remove opening bracket and closing bracket from MESG_TYPE and CLIENT columns.
Thanks