External Table, Handling Delimited and Special Character in file

Question

Hi ,
I have created one external table with these option
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY ***************************************
ACCESS PARAMETERS
( RECORDS DELIMITED BY NEWLINE
SKIP 0
FIELDS TERMINATED BY '|'
OPTIONALLY ENCLOSED BY '"'
MISSING FIELD VALUES ARE NULL                                          
LOCATION
( 'test_feed.csv'
Now problem is these are coming as valid.
anupam|anupam2
anupam"test|anupam"test2
"anupam|test3"|test3
anupam""""test5|test5
anupam"|test7
but these are not coming as valid
"anupam"test4"|test4    --> Case when we have quotes in the filed but still have quotes in it. I guess in this case we can send the filed expect closing double quotes.
"anupam|test6   --> In case field is starting with double quotes then it's failing
"anupam"test8|test8"|test8 --> In case one filed contains both pipe ( |) and double quotes then we are sending it enclosed in double quotes. But thats failing the job.
Can you suggest what is the best way to handle such scenario? ( One restriction though. The file is used by other system - Netezza as well, which can't take more than one character long delimited :'( )

Answer 1

One approach is to define the external table a ONE column table (with single field on the file). This way each line will come in as a row in the external table. Of course you have to build "parsing logic" on top of that.
DROP TABLE xtern_table;
CREATE TABLE xtern_table
    c1 VARCHAR2(4000)
  organization external
    type ORACLE_LOADER DEFAULT directory xtern_data_dir
    ACCESS PARAMETERS (
        RECORDS DELIMITED BY NEWLINE
        FIELDS TERMINATED BY '~'   ---- <<<<<<<< Use a field terminator as a character that is not found in the file
        MISSING FIELD VALUES ARE NULL
        ( c1 CHAR(4000)
     ) location ('mycsv.csv')
> desc xtern_table
desc xtern_table
Name Null Type          
C1        VARCHAR2(4000)
> column c1 format A40
> select * from xtern_table
C1                                    
anupam|anupam2                          
anupam"test|anupam"test2                
"anupam|test3"|test3                    
anupam""""test5|test5                   
anupam"|test7                           
"anupam"test4"|test4                    
"anupam|test6                           
"anupam"test8|test8"|test8              
8 rows selected
Ideally, it will be good t have an incoming source file with predictable format.
Hope this helps.