External table with dos ^M char

I have created an external table in my Oracle 9i running on Solaris. The table is based on a flatfile that sources from NT. The very last column of each line has ^M character (I think its the Dos newline char ). So if I query the last column of my external table I'lll get :
select last_col
FROM my_ext_tab
LAST_
^M
D^M
A^M
but what I really want to return is
LAST_
<- "NULL"
D
A
I tried to dos2unix the file, and I get
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
help please
thank you

How is the file getting from NT to Unix? It sounds like you're using binary-mode FTP to transfer an ASCII file. If you use ascii-mode FTP, that should solve the problem.
It's probably possible to solve the problem exclusively on the Oracle side, but fixing the file in the first place will probably be more robust and straightforward.
Justin
Distributed Database Consulting, Inc.
www.ddbcinc.com

Similar Messages

Can we bind a single external table with multiple files in OWB 11g?

Hi,
I wanted to ask if it is possible to bind an external table with multiple source files at same or different locations? Or an external table has to be bound to a single source file and a single location.
Thanks in advance,
Ann.
Edited by: Ann on Oct 8, 2010 9:38 AM

Hi Ann,
Can you please help me out by telling me the steps to accomplish this. Right click on the external table in project tree, from the menu choose Configure,
then in opened Configuration Properties dialog window right clock on Data Files node and choose from menu Create -
you will get new record for file - specify Data File Name property
Also link from OWB user guide
http://download.oracle.com/docs/cd/B28359_01/owb.111/b31278/ref_def_flatfiles.htm#i1126304
Regards,
Oleg

How to get the bound object of an external table with OMB

Hello,
I try to find it but without success. So may be one of you, know this secret information.
I want to synchronize with the help of an OMB script my external tables.
OMBSYNCHRONIZE FLAT_FILE '/MY_PROJECT/MY_FLAT_FILE_MODULE/MY_FLAT_FILE' RECORD 'MY_RECORD' TO EXTERNAL_TABLE 'MY_EXTERNAL_TABLE' \
USE (RECONCILE_STRATEGY 'REPLACE', MATCHING_STRATEGY 'MATCH_BY_OBJECT_ID')For this purpose, I need to know the bound object. When you synchronize with the help of the GUI, you see it easily.
Example:
/MY_PROJECT/MY_FLAT_FILE_MODULE/MY_FLAT_FILE/MY_RECORDI searched in the properties of the external table and I don't see a BOUND OBJECT property.
OMBDESCRIBE CLASS_DEFINITION 'EXTERNAL_TABLE' GET PROPERTY_DEFINITIONS
BAD_FILE_LOCATION BAD_FILE_NAME BUSINESS_NAME DATA_FILES DEPLOYABLE DESCRIPTION
DISCARD_FILE_LOCATION DISCARD_FILE_NAME ENDIAN GENERATE_ERROR_TABLE_ONLY GENERAT
ION_COMMENTS LOAD_NULLS_WHEN_MISSING_VALUES LOG_FILE_LOCATION LOG_FILE_NAME NLS_
CHARACTERSET NUMBER_OF_REJECTS_ALLOWED PARALLEL_ACCESS_DRIVERS PARALLEL_ACCESS_M
ODE REJECTS_ARE_UNLIMITED SHADOW_TABLESPACE SHADOW_TABLE_NAME STRING_SIZES_IN TR
IM UOIDThen I try the BOUND_OBJECT cached properties of a mapping operator but no luck.
OMBRETRIEVE EXTERNAL_TABLE 'MY_EXTERNAL_TABLE' GET BOUND_OBJECT
OMB00001: Encountered BOUND_OBJECT at line: 1, column: 56. Was expecting one of:
"PROPERTIES" ...
    "REF" ...
    "REFERENCE" ...
    "COLUMN" ...
    "DEFAULT_LOCATION" ...
    "FLAT_FILE" ...
    "RECORD" ...
    "COLUMNS" ...
    "DATA_FILES" ...
    "DATA_RULE_USAGES" ...I have already the file and the record with the FLAT_FILE and RECORD properties
OMBRETRIEVE EXTERNAL_TABLE 'MY_EXTERNAL_TABLE' GET FLAT_FILE
OMBRETRIEVE EXTERNAL_TABLE 'MY_EXTERNAL_TABLE' GET RECORDBut how can I get the flat file module:
MY_FLAT_FILE_MODULEDoes anybody know how OWB retrieve this information ?
Thanks in advance and good day
Nico
Edited by: gerardnico on Jan 13, 2010 12:08 PM
Change get the location by get the flat_file_module

Yes, Oleg. It's what's worried me.
The BOUND_OBJECT property of a table operator is not in the API.
OMBDESCRIBE CLASS_DEFINITION 'TABLE_OPERATOR' GET PROPERTY_DEFINITIONS
ADVANCED_MATCH_BY_CONSTRAINT AUTOMATIC_HINTS_ENABLED BOUND_NAME BUSINESS_NAME CO
NFLICT_RESOLUTION DATABASE_FILE_NAME DATABASE_LINK DATA_COLLECTION_FREQUENCY DAT
A_RULES DB_LOCATION DEBUG_BOUND_NAME DEBUG_DB_LOCATION DESCRIPTION DIRECT ENABLE
_CONSTRAINTS ERROR_SELECT_FILTER ERROR_SELECT_ROLL_UP ERROR_TABLE_NAME EVALUATE_
CHECK_CONSTRAINTS EXCEPTIONS_TABLE_NAME EXTRACTION_HINT IS_TEMP_STAGE_TABLE JOIN
RANK KEYS_READONLY LOADING_HINT LOADING_TYPE MATCH_BY_CONSTRAINT OPTIMIZE_MERGE
PARTITION_NAME PEL_ENABLED PRIMARY_SOURCE RECORDS_TO_SKIP REPLACE_DATA ROW_COUNT
ROW_COUNT_ENABLED SCHEMA SINGLEROW SORTED_INDEXES_CLAUSE SUBPARTITION_NAME TARG
ET_FILTER_FOR_DELETE TARGET_FILTER_FOR_UPDATE TARGET_LOAD_ORDER TEMP_STAGE_TABLE
_EXTRA_DDL_CLAUSES TEST_DATA_COLUMN_LIST TEST_DATA_WHERE_CLAUSE TRAILING_NULLCOL
S TRUNCATE_ERROR_TABLE UOID USE_LCR_APIThen I was wondering if anyone knew a little bit the same hidden property but to get the flat file source object of an external table.
Cheers
Nico

Error while calling an procedure using an external table with C#.

Hello,
I am developping a scheduler application with Visual Studio 2010 (C#) to start my PL/SQL procedures.
Everything works fine with every procedure but one who is reading the content of an external table.
Strange thing is when i launch the same procedure with Toad, i have no problem, but when i launch it with my C# code :
            OracleCommand cmdMET = new OracleCommand();
            cmdMET.CommandText = "STG_AE.M_MET_S_EXT_DEFECT";
            cmdMET.CommandType = CommandType.StoredProcedure;
            cmdMET.Connection = con;
            //OracleParameter retvalMET = new OracleParameter("retval", OracleDbType.Varchar2, 50);
            //retvalMET.Direction = ParameterDirection.ReturnValue;
            //cmdMET.Parameters.Add(retvalMET);
            cmdMET.ExecuteNonQuery();
I've got this error :
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-01722: invalid number
ORA-06512: at "STG_AE.M_MET_S_EXT_DEFECT", line 8
ORA-06512: at "STG_AE.MET_SRC", line 10
ORA-06512: at "STG_AE.MET", line 14
ORA-06512: at line 1
I can't figure out why it works when i launch it with Toad and why i get this error when launching it with C#... Any advice would be great !
Thanks!

You're right!
Adding
string sql = "ALTER SESSION SET NLS_NUMERIC_CHARACTERS = \",.\"";
OracleCommand cmd = new OracleCommand(sql, con);
cmd.CommandType = CommandType.Text;
OracleDataReader dr = cmd.ExecuteReader();
solved the problem !
Thanks a lot =)

External table with Date Format

I have external table and in the file date format is YYYY-MM-DD.
i want to insert date format mask in the create table command .
the bleow command is getting created . But it is giving error when i use select command.
CREATE TABLE XADV.XADV_test_EXT_TABLE
BPO_START_DATE Date
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY XCRM_DASHBOARD
ACCESS PARAMETERS
( RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY '|^|'
BPO_START_DATE date 'YYYY-MM-DD'
LOCATION (XCRM_DASHBOARD:'test.txt')
REJECT LIMIT UNLIMITED
NOPARALLEL
NOMONITORING;
the error i am getting is in select * from XADV_test_EXT_TABLE
RA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "minussign": expecting one of: "column, enclosed, exit, (, ltrim, lrtrim, ldrtrim, missing, notrim, optionally, rtrim, reject"
KUP-01007: at line 3 column 3
ORA-06512: at "SYS.ORACLE_LOADER", line 19
regards
Mani

shoblock wrote:
use double quotes instead of single quotesHas nothing to do with the quotes. Issue is field list must be enclosed in parents:
SQL> CREATE TABLE XADV_test_EXT_TABLE
2 (
3 BPO_START_DATE Date
4 )
5 ORGANIZATION EXTERNAL
6 (
7   TYPE ORACLE_LOADER
8   DEFAULT DIRECTORY TMP
9   ACCESS PARAMETERS
10   (
11    RECORDS DELIMITED BY NEWLINE
12    FIELDS TERMINATED BY ','
13     BPO_START_DATE date "YYYY-MM-DD"
14   )
15 LOCATION (TMP:'test.txt')
16 )
17 REJECT LIMIT UNLIMITED
18 NOPARALLEL
19 NOMONITORING
20 /
Table created.
SQL> select * from XADV_test_EXT_TABLE
2 /
select * from XADV_test_EXT_TABLE
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "identifier": expecting one of: "column,
enclosed, exit, (, ltrim, lrtrim, ldrtrim, missing, notrim, optionally, rtrim,
reject"
KUP-01008: the bad identifier was: BPO_START_DATE
KUP-01007: at line 3 column 4
ORA-06512: at "SYS.ORACLE_LOADER", line 19
SQL> drop table XADV_test_EXT_TABLE;
Table dropped.
SQL> CREATE TABLE XADV_test_EXT_TABLE
2 (
3 BPO_START_DATE Date
4 )
5 ORGANIZATION EXTERNAL
6 (
7   TYPE ORACLE_LOADER
8   DEFAULT DIRECTORY TMP
9   ACCESS PARAMETERS
10   (
11    RECORDS DELIMITED BY NEWLINE
12    FIELDS TERMINATED BY ','
13    (
14     BPO_START_DATE date 'YYYY-MM-DD'
15    )
16   )
17 LOCATION (TMP:'test.txt')
18 )
19 REJECT LIMIT UNLIMITED
20 NOPARALLEL
21 NOMONITORING
22 /
Table created.
SQL> select * from XADV_test_EXT_TABLE
2 /
BPO_START
19-NOV-08
SQL> SY.

External tables with user defined fuction help

Can any one help me
I have problem with using user defined function while creating the External table.
The user defined function Just takes an input empno and returns one number.
CREATE TABLE EMPXT_1(empno NUMBER(4)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER
DEFAULT DIRECTORY TEST_DIR
ACCESS PARAMETERS(
RECORDS FIXED 4
STRING SIZES ARE IN BYTES
NOBADFILE
NODISCARDFILE
LOGFILE '211205.KNU.LOG'
FIELDS
NOTRIM
(empno POSITION(1) integer external(4) p_1(:empno)
LOCATION
('empxt_1.dat')
REJECT LIMIT UNLIMITED
NOPARALLEL
content of expxt_1.dat
7849
SELECT * from EMPXT_1;
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "double-quoted-string": expecting one of:
"comma, defaultif, nullif, optionally, ), terminated"
KUP-01007: at line 8 column 49
ORA-06512: at "SYS.ORACLE_LOADER", line 14
ORA-06512: at line 1

hi,
may be u have do these
for ORA-29913 error
1. you have to grant read and write to a directory.
for other errors do check this document
http://www.lc.leidenuniv.nl/awcourse/oracle/server.920/a96525/kupus.htm
hope this may help u
Trinath Somanchi

How can I use external tables with directories not on Oracle host?

Oracle 11.2.0.2
We have developers using C# to populate global temporary tables (two one header and detail with 1 to many relationbship between these two tables). The process of using C# was blowing up memory on Windows.
As an alternative I requested them to create two CSV files and I can use sqlldr to load the detail CSV file of over 1 million rows under 15 seconds. It came to my mind that I could use external tables for this purpose with getting read of header and trailer from the CSV files.
The issue I have is that I am not a DBA so I do not have access to OS host that Oracle instance is running. sqlldr is a client side tool, whereas external tables are server side. I was wondering if there is an alternative in 11g to use external tables without creating directories on the OS host that Oracle runs on? Are there other alternatives?
Thanks

905989 wrote:
Oracle 11.2.0.2
We have developers using C# to populate global temporary tables (two one header and detail with 1 to many relationbship between these two tables). The process of using C# was blowing up memory on Windows.
As an alternative I requested them to create two CSV files and I can use sqlldr to load the detail CSV file of over 1 million rows under 15 seconds. It came to my mind that I could use external tables for this purpose with getting read of header and trailer from the CSV files.
The issue I have is that I am not a DBA so I do not have access to OS host that Oracle instance is running. sqlldr is a client side tool, whereas external tables are server side. I was wondering if there is an alternative in 11g to use external tables without creating directories on the OS host that Oracle runs on? Are there other alternatives?
Thanks
no other alternative

Deploying external tables with ombplus

Hi,
we try to deploy our projects with ombplus. The script breaks at the first external table when we do:
OMBCREATE TRANSIENT DEPLOYMENT_ACTION_PLAN 'SVR_ET'
foreach ext_tab [OMBLIST EXTERNAL_TABLES] {
set exttab_path [concat $svr_modul$ext_tab]
OMBALTER DEPLOYMENT_ACTION_PLAN 'SVR_ET' \
ADD ACTION '$ext_tab' SET PROPERTIES (OPERATION) VALUES ('CREATE') \
SET REFERENCE EXTERNAL_TABLE '$exttab_path'}
OMBDEPLOY DEPLOYMENT_ACTION_PLAN 'SVR_ET'
There is one external table that give an "VLD-0917" - error details are:
java.lang.NullPointerException.
Beforehand we did the Connectors with such a loop.
Any ideas?
Michael

Hi Michael,
Try to make it simple first time without a loop. When you have that fixed you try to do it with a loop.
The example below, witout a loop, works for me.
/JZ
--First script deploy to file
set OMBLOG {c:\tmp_jz\omb_logfile_deploy_test_to_file.log}
OMBCONNECT owb_o/owb_o@localhost:1521:owbdb USE REPOSITORY 'owbdb'
OMBCC 'DATAWAREHOUSE'
OMBCONNECT CONTROL_CENTER owb_o/owb_o@localhost:1521:dw \
USE REPOSITORY 'owb_o'
OMBCREATE TRANSIENT DEPLOYMENT_ACTION_PLAN 'DEPLOY_TEST' \
ADD ACTION 'ST_ET_TABELL' \
SET PROPERTIES(OPERATION) VALUES ('CREATE') \
SET REFERENCE EXTERNAL_TABLE '/DATAWAREHOUSE/TEST/ET_TABLE'
OMBDEPLOY DEPLOYMENT_ACTION_PLAN 'DEPLOY_TEST' \
AS SPECIFICATION TO 'c:\\tmp_jz\\deploy_test_to_file.xml'
OMBDISCONNECT CONTROL_CENTER
OMBDISCONNECT
exit
--Second script deploy from file to control center
set OMBLOG {omb_logfile_deploy_test_to_cc.log}
OMBCONNECT CONTROL_CENTER owb_ow/owb_o@localhost:dw \
USE REPOSITORY 'owb_owner'
OMBDEPLOY SPECIFICATION FROM 'deploy_test.xml'
OMBDISCONNECT CONTROL_CENTER
exit

External table with empty file

Hi,
My db version: Oracle 11g
I have an empty csv file.
I created a external table for the empty csv file.
When I run:
select count(*) from externaltblname;
It returns 1.
It should return 0 right.
In the definition, I specified "SKIP 1"
But still it returns 1.
When I use this external table to load into a target table. It loads a single row with null values.
How to fix this. Please advice.

What works for me is the following (t_ext points to an empty csv):
SQL> select count(field) from t_ext;
COUNT(FIELD)
           1
1 row selected.
SQL> select ascii(field) from t_ext;
ASCII(FIELD)
          13
1 row selected.
SQL> select count(replace(field, chr(13))) from t_ext;
COUNT(REPLACE(FIELD,CHR(13)))
                            0
1 row selected.

Uploading data from an external table with a where clause

Hi,
How do insert data from an external table in to a table already created in the database where person_id = person-_id on the external table?
Example:
External table
XX_EXTERNAL_TBL (Person_id, emp_number, emp_name)
Internal table
XX_SQL_LOADER_TEST (Person_id, emp_number, emp_name)
and person id already exists on the internal table (only want the last to columns to import where the person ids are the same)
Thanks

INSERT
INTO XX_SQL_LOADER_TEST
SELECT *
    FROM XX_EXTERNAL_TBL
    WHERE Person_id IN (
                        SELECT Person_id
                          FROM XX_SQL_LOADER_TEST
/SY.

JDBC and creation of external tables with positions

Hello,
this might be a trick question or I am probably running around in circles with this one. I know that it is not possible to call pl/sql command ("set define off") using a jdbc connection. Correct me on this one if I am wrong ;)
So the point why I want to that is because I want to execute the following stmt using a CallableStatement:
prompt
prompt: CREATE TABLE "TEST_IN"."TEST_RECEIPT_EXTERNAL" ...
prompt
CREATE TABLE "TEST"."TEST_RECEIPT_EXTERNAL"
("APPLE" VARCHAR2(8 CHAR)
,"PIE" VARCHAR2(1 CHAR)
,"RECEIPT" VARCHAR2(58 CHAR)
ORGANIZATION EXTERNAL
(TYPE ORACLE_LOADER
DEFAULT DIRECTORY "TEST_IN"
ACCESS PARAMETERS
(records delimited by newline
fields (
APPLE position(1: 8) CHAR(8),
PIE position(9: 9) CHAR(1),
RECEIPT position(10: 67) CHAR(58), )
LOCATION
("TEST_IN":'FLIP.FLOP.19760104170600'))
REJECT LIMIT UNLIMITED
PARALLEL 1
which gives me back the expected error:
Missing IN or OUT parameter at index:: 1
Any thoughts on this one would be very helpful - thanks in advance,
Sascha

Perfect - did not thought that the behavior of Statement, CallableStatement and PreparedStatement is different. In retrospect and thinking about the use cases of the implementations I probably should have thought about that my self.
Thanks a lot - problem solved :)

DATE fields and LOG files in context with external tables

I am facing two problems when dealing with the external tables feature in Oracle 9i.
I created an External Table with some fileds with the DATE data type . There were no issues during the creation part. But when i query the table, the DATE fields are not properly selected though the data is there in the files. Is there any ideas to deal with this ?
My next question is regarding the log files. The contents in the log file seems to be growing when querying the external tables. Is there a way to control this behaviour?
Suggestions / Advices on the above two issues are welcome.
Thanks
Lakshminarayanan

Hi
If you have date datatypes than:
select
greatest(TABCASER1.CASERRECIEVEDDATE, EVCASERS.FINALEVDATES, EVCASERS.PUBLICATIONDATE, EVCASERS.PUBLICATIONDATE, TABCASER.COMPAREACCEPDATE)
from TABCASER, TABCASER1, EVCASERS
where ...-- join and other conditions
1. greatest is good enough
2. to_date creates date dataype from string with the format of format string ('mm/dd/yyyy')
3. decode(a, b, c, d) is a function: if a = b than return c else d. NULL means that there is no data in the cell of the table.
6. to format the date for display use to_char function with format modell as in the to_date function.
Ott Karesz
http://www.trendo-kft.hu

XML File in External Table - OS error permission denied.

Hi.
10g R2, Red Hat Linux
I'm using the article (see below, taken from http://www.dbazine.com/olc/olc-articles/scardina1 by Mark Scardina) to create an external table where I'd store my XML file.
So, I
1. Created a directory xmlfile_dir
2. Granted access to needed db user
3. Created the table
CREATE TABLE relayxml_xt (doc CLOB)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER
DEFAULT DIRECTORY xmlfile_dir
ACCESS PARAMETERS
FIELDS (lobfn CHAR TERMINATED BY ',')
COLUMN TRANSFORMS (doc FROM lobfile (lobfn))
LOCATION ('xml.dat')
REJECT LIMIT UNLIMITED;
4. mv relay.xml /xmlfile_dir/xml.dat
When I run SELECT * FROM relayxml_xt I get this:
Error starting at line 1 in command:
select * from relayxml_xt
Error report:
SQL Error: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04063: unable to open log file RELAYXML_XT_28773.log
OS error Permission denied
ORA-06512: at "SYS.ORACLE_LOADER", line 19
29913. 00000 - "error in executing %s callout"
*Cause:    The execution of the specified callout caused an error.
*Action:   Examine the error messages take appropriate action.
What am I doing wrong?
Thanks,
Using External Tables
Introduced in Oracle9i, Oracle’s external table feature offers a solution to define a table in the database while leaving the data stored outside of the database. Prior to Oracle Database 10g, external tables can be used only as read-only tables. In other words, if you create an external table for XML files, these files can be queries and the table can be joined with other tables. However, no DML operations, such as INSERT, UPDATE, and DELETE, are allowed on the external tables.
Note: In Oracle Database 10g , by using the ORACLE_DATAPUMP driver instead of the default ORACLE_DRIVER, you can write to external tables. In Oracle Database 10g, you can define VARCHAR2 and CLOB columns in external tables to store XML documents. The following example shows how you can create an external table with a CLOB column to store the XML documents. First, you need to create a DIRECTORY to read the data files:
CREATE DIRECTORY data_file_dir AS 'D:\xmlbook\Examples\Chapter9\src\xml';
GRANT READ, WRITE ON DIRECTORY data_file_dir TO demo;
Then, you can use this DIRECTORY to define an external table:
CREATE TABLE customer_xt (doc CLOB)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER
DEFAULT DIRECTORY data_file_dir
ACCESS PARAMETERS
FIELDS (lobfn CHAR TERMINATED BY ',')
COLUMN TRANSFORMS (doc FROM lobfile (lobfn))
LOCATION ('xml.dat')
REJECT LIMIT UNLIMITED;
The xml.dat file follows:
customer1.xml
customer2.xml
If you describe the table, you can see the following definition:
SQL> DESC customer_xt;
Name Null? Type
DOC CLOB
Then, you can query the XML document as follows:
SELECT XMLType(doc).extract('/Customer/EMAIL')
FROM customer_xt;
Though the query requires run-time XMLType creation and XPath evaluation, this approach is useful when applications just need a few queries on the XML data and don’t want to upload the XML data into database. In Oracle Database 10g, you cannot create external tables that contain pre-defined XMLType column types.
Message was edited by:
vi2167

Your don't have the proper operating system privileges. Be sure that you (=oracle OS user / the OS Linux user that is starting the database) are allowed have read privs on the path and/or file.
for example...
chown -Rf /xxxxxxx/xxxx/etc
ls -l file.xml
file.xml    oracle:oinstall    rw-rw-rw

External Table, Handling Delimited and Special Character in file

Hi ,
I have created one external table with these option
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY ***************************************
ACCESS PARAMETERS
( RECORDS DELIMITED BY NEWLINE
SKIP 0
FIELDS TERMINATED BY '|'
OPTIONALLY ENCLOSED BY '"'
MISSING FIELD VALUES ARE NULL
LOCATION
( 'test_feed.csv'
Now problem is these are coming as valid.
anupam|anupam2
anupam"test|anupam"test2
"anupam|test3"|test3
anupam""""test5|test5
anupam"|test7
but these are not coming as valid
"anupam"test4"|test4    --> Case when we have quotes in the filed but still have quotes in it. I guess in this case we can send the filed expect closing double quotes.
"anupam|test6   --> In case field is starting with double quotes then it's failing
"anupam"test8|test8"|test8 --> In case one filed contains both pipe ( |) and double quotes then we are sending it enclosed in double quotes. But thats failing the job.
Can you suggest what is the best way to handle such scenario? ( One restriction though. The file is used by other system - Netezza as well, which can't take more than one character long delimited :'( )

One approach is to define the external table a ONE column table (with single field on the file). This way each line will come in as a row in the external table. Of course you have to build "parsing logic" on top of that.
DROP TABLE xtern_table;
CREATE TABLE xtern_table
    c1 VARCHAR2(4000)
organization external
    type ORACLE_LOADER DEFAULT directory xtern_data_dir
    ACCESS PARAMETERS (
        RECORDS DELIMITED BY NEWLINE
        FIELDS TERMINATED BY '~'   ---- <<<<<<<< Use a field terminator as a character that is not found in the file
        MISSING FIELD VALUES ARE NULL
        ( c1 CHAR(4000)
     ) location ('mycsv.csv')
> desc xtern_table
desc xtern_table
Name Null Type
C1        VARCHAR2(4000)
> column c1 format A40
> select * from xtern_table
C1
anupam|anupam2
anupam"test|anupam"test2
"anupam|test3"|test3
anupam""""test5|test5
anupam"|test7
"anupam"test4"|test4
"anupam|test6
"anupam"test8|test8"|test8
8 rows selected
Ideally, it will be good t have an incoming source file with predictable format.
Hope this helps.

Csv file processing using external table

Dear All,
Our database is oracle 10g r2 and OS is solaris
We would receive csv files to a particular directory on server each day.
File Format look like:
H00,SOURCE_NAME,FILE_CREATED_DATE
RECORD_TYPE,EMP_ID,EMP_NAME,EMP_DOB(DDMMYYYY),EMP_HIRE_DATE(DDMMYYYY),EMP_LOCATION
T00,RECORD_COUNT
EMPLOYEE TABLE STRUCTURE
EMP_ID                   NOT NULL    NUMBER ,
EMP_NAME            NOT NULL    VARCHAR2(10) ,
EMP_DOB                                  DATE,
EMP_HIRE_DATE   NOT NULL     DATE,
EMP_LOCATION                VARCHAR2(80)
Sample File:
H00,ABC,21092013
"R01",1,"EMP1","14021986","06072010","LOC1"
"R01",20000000000,"EMP2","14021-987","06072011",""
,***,"EMPPPPPPPPPPP3","14021988","060**012","LOC2"
"R01",4,4,"14021989","06072013",
T00,4
we need to validate each record excluding header and trailer for:
DATATYPE, LENGTH,OPTIONALITY, and other date validations such as EMP_HIRE_DATE can not be less than EMP_DOB
In case of any data errors we need to send a response file for corresponding source file.
we have predefined error codes to be sent in the response file.
ERR001    EMP_ID can not be null
ERR002    EMP_ID exceeds 10 digits
ERR003    EMP_ID is not a number
ERR004    EMP_NAME   has to be text
ERR005    EMP_NAME length can not exceed 10
ERR006    EMP_NAME   can not be null
ERR007    EMP_DOB is not a date
ERR008    EMP_DOB is not in ddmmyyyy format
ERR009    EMP_HIRE_DATE is not a date
ERR010    EMP_HIRE_DATE is not in ddmmyyyy format
ERR011    EMP_HIRE_DATE can not be null
ERR012    EMP_LOCATION    has to be text
ERR013    EMP_LOCATION   length can not exceed 80
ERR014    EMP_HIRE_DATE can not be less than EMP_DOB
ERR015    Field missing in the record
ERR016    More number of fields than allowed
1.Do I need to create external table before processing each file.(EMP1.txt,EMP2.txt)?
2.How to generate these error codes in case of respective failure scenarios and to log into an exception table?
3.response file needs to have entire record and a concatination of all the error codes in the next line.
4.what would be a better approach among
creating an external table with all char(2000) fields and writing a select statement
such as select * from ext_table where (emp id is not null and length(emp_id)<=10 and....to select only proper data);
or creating the external table to be same as employee table and creating a bad file? if this is the preferred how can I generate the custom error codes?
Could you please help me in achieving this!
Warm Regards,
Shankar.

You can do a one-time creation of an external table. After that, you can either overwrite the existing text file or alter the location. In the example below I have split your original sample file into two files, in order to demonstrate altering the location. You can make the external table all varchar2 fields as large as they need to be to accommodate all possible data. If you then create a staging table and rejects table of the same structure, you can create a trigger on the staging table such that, when you insert from the external table to the staging table, it inserts into either the employee table or rejects table with the concatenated error list. If you want this in a file, then you can just spool a select from the rejects table or use some other method such as utl_file. The following is a partial example. Another alternative would be to use SQL*Loader to load directly into the staging table without an external table.
SCOTT@orcl12c> HOST TYPE emp1.txt
H00,ABC,21092013
"R01",1,"EMP1","14021986","06072010","LOC1"
"R01",20000000000,"EMP2","14021-987","06072011",""
T00,4
SCOTT@orcl12c> HOST TYPE emp2.txt
H00,ABC,21092013
,***,"EMPPPPPPPPPPP3","14021988","060**012","LOC2"
"R01",4,4,"14021989","06072013",
T00,4
SCOTT@orcl12c> CREATE OR REPLACE DIRECTORY my_dir AS 'c:\my_oracle_files'
2 /
Directory created.
SCOTT@orcl12c> CREATE TABLE external_table
2    (record_type       VARCHAR2(10),
3     emp_id            VARCHAR2(11),
4     emp_name          VARCHAR2(14),
5     emp_dob           VARCHAR2(10),
6     emp_hire_date     VARCHAR2(10),
7     emp_location      VARCHAR2(80))
8 ORGANIZATION external
9    (TYPE oracle_loader
10     DEFAULT DIRECTORY my_dir
11     ACCESS PARAMETERS
12       (RECORDS DELIMITED BY NEWLINE
13        LOAD WHEN ((1: 3) != "H00" AND (1:3) != 'T00')
14        LOGFILE 'test.log'
15        FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"' LDRTRIM
16        MISSING FIELD VALUES ARE NULL
17        REJECT ROWS WITH ALL NULL FIELDS
18          (record_type, emp_id, emp_name, emp_dob, emp_hire_date, emp_location))
19     LOCATION ('emp1.txt'))
20 /
Table created.
SCOTT@orcl12c> CREATE TABLE staging
2    (record_type       VARCHAR2(10),
3     emp_id            VARCHAR2(11),
4     emp_name          VARCHAR2(14),
5     emp_dob           VARCHAR2(10),
6     emp_hire_date     VARCHAR2(10),
7     emp_location      VARCHAR2(80))
8 /
Table created.
SCOTT@orcl12c> CREATE TABLE employee
2    (emp_id          NUMBER       NOT NULL,
3     emp_name        VARCHAR2(10) NOT NULL,
4     emp_dob         DATE,
5     emp_hire_date   DATE,
6     emp_location    VARCHAR2(80))
7 /
Table created.
SCOTT@orcl12c> CREATE TABLE rejects
2    (record_type       VARCHAR2(10),
3     emp_id            VARCHAR2(11),
4     emp_name          VARCHAR2(14),
5     emp_dob           VARCHAR2(10),
6     emp_hire_date     VARCHAR2(10),
7     emp_location      VARCHAR2(80),
8     error_codes       VARCHAR2(4000))
9 /
Table created.
SCOTT@orcl12c> CREATE OR REPLACE TRIGGER staging_air
2    AFTER INSERT ON staging
3    FOR EACH ROW
4 DECLARE
5    v_rejects         NUMBER := 0;
6    v_error_codes VARCHAR2(4000);
7    v_num          NUMBER;
8    v_dob          DATE;
9    v_hire         DATE;
10 BEGIN
11    IF :NEW.emp_id IS NULL THEN
12      v_rejects := v_rejects + 1;
13      v_error_codes := v_error_codes || ',' || 'ERR001';
14    ELSIF LENGTH (:NEW.emp_id) > 10 THEN
15      v_rejects := v_rejects + 1;
16      v_error_codes := v_error_codes || ',' || 'ERR002';
17    END IF;
18    BEGIN
19      v_num := TO_NUMBER (:NEW.emp_id);
20    EXCEPTION
21      WHEN value_error THEN
22        v_rejects := v_rejects + 1;
23        v_error_codes := v_error_codes || ',' || 'ERR003';
24    END;
25    IF :NEW.emp_name IS NULL THEN
26      v_rejects := v_rejects + 1;
27      v_error_codes := v_error_codes || ',' || 'ERR006';
28    ELSIF LENGTH (:NEW.emp_name) > 10 THEN
29      v_rejects := v_rejects + 1;
30      v_error_codes := v_error_codes || ',' || 'ERR005';
31    END IF;
32    BEGIN
33      v_dob := TO_DATE (:NEW.emp_dob, 'ddmmyyyy');
34    EXCEPTION
35      WHEN OTHERS THEN
36        v_rejects := v_rejects + 1;
37        v_error_codes := v_error_codes || ',' || 'ERR008';
38    END;
39    BEGIN
40      IF :NEW.emp_hire_date IS NULL THEN
41        v_rejects := v_rejects + 1;
42        v_error_codes := v_error_codes || ',' || 'ERR011';
43      ELSE
44        v_hire := TO_DATE (:NEW.emp_hire_date, 'ddmmyyyy');
45      END IF;
46    EXCEPTION
47      WHEN OTHERS THEN
48        v_rejects := v_rejects + 1;
49        v_error_codes := v_error_codes || ',' || 'ERR010';
50    END;
51    IF LENGTH (:NEW.emp_location) > 80 THEN
52      v_rejects := v_rejects + 1;
53      v_error_codes := v_error_codes || ',' || 'ERR013';
54    END IF;
55    IF v_hire IS NOT NULL AND v_dob IS NOT NULL AND v_hire < v_dob THEN
56        v_rejects := v_rejects + 1;
57        v_error_codes := v_error_codes || ',' || 'ERR014';
58    END IF;
59    IF :NEW.emp_id IS NULL OR :NEW.emp_name IS NULL OR :NEW.emp_dob IS NULL
60       OR :NEW.emp_hire_date IS NULL OR :NEW.emp_location IS NULL THEN
61        v_rejects := v_rejects + 1;
62        v_error_codes := v_error_codes || ',' || 'ERR015';
63    END IF;
64    IF v_rejects = 0 THEN
65      INSERT INTO employee (emp_id, emp_name, emp_dob, emp_hire_date, emp_location)
66      VALUES (:NEW.emp_id, :NEW.emp_name,
67              TO_DATE (:NEW.emp_dob, 'ddmmyyyy'), TO_DATE (:NEW.emp_hire_date, 'ddmmyyyy'),
68              :NEW.emp_location);
69    ELSE
70      v_error_codes := LTRIM (v_error_codes, ',');
71      INSERT INTO rejects
72        (record_type,
73         emp_id, emp_name, emp_dob, emp_hire_date, emp_location,
74         error_codes)
75      VALUES
76        (:NEW.record_type,
77         :NEW.emp_id, :NEW.emp_name, :NEW.emp_dob, :NEW.emp_hire_date, :NEW.emp_location,
78         v_error_codes);
79    END IF;
80 END staging_air;
81 /
Trigger created.
SCOTT@orcl12c> SHOW ERRORS
No errors.
SCOTT@orcl12c> INSERT INTO staging SELECT * FROM external_table
2 /
2 rows created.
SCOTT@orcl12c> ALTER TABLE external_table LOCATION ('emp2.txt')
2 /
Table altered.
SCOTT@orcl12c> INSERT INTO staging SELECT * FROM external_table
2 /
2 rows created.
SCOTT@orcl12c> SELECT * FROM employee
2 /
    EMP_ID EMP_NAME   EMP_DOB         EMP_HIRE_DATE   EMP_LOCATION
         1 EMP1       Fri 14-Feb-1986 Tue 06-Jul-2010 LOC1
1 row selected.
SCOTT@orcl12c> COLUMN error_codes NEWLINE
SCOTT@orcl12c> SELECT * FROM rejects
2 /
RECORD_TYP EMP_ID      EMP_NAME       EMP_DOB    EMP_HIRE_D EMP_LOCATION
ERROR_CODES
R01        20000000000 EMP2           14021-987 06072011
ERR002,ERR008,ERR015
           ***         EMPPPPPPPPPPP3 14021988   060**012   LOC2
ERR003,ERR005,ERR010
R01        4           4              14021989   06072013
ERR015
3 rows selected.

External table with dos ^M char

Similar Messages

Maybe you are looking for