Substr in external table definition

Hi Guys,
I have an external table where I am loading a txt file. The issue being, in the txt file I have a column with a variable length. This column houses the application/website link that is not of a fixed length. My external table field width is 100, but the txt file value is more than that. Now, I want to only insert this field for the 100 characters.
I tried using substr as shown in the example below, but it fails. Any idea what could be the alternative or if I am doing it right?
Here's my external table properties (access parameters) and I am trying to use 'substr' on field4
records delimited BY NEWLINE
badfile BAD_DIR:'email_events%a_%p.bad'
LOGFILE LOG_DIR:'email_events%a_%p.log'
fields terminated by '\t'
missing field values are null
event_type,
cm_timestamp,
issue_id,
cm_uid,
field1,
field2,
field3,
field4 "substr(:field4,1,100)",
field5,
field6,
field7,
field8,
field9,
field10,
field11,
field12,
field13,
field14
)

Just set your external table definition to query the full thing and then when you come to insert that data to a real table from the external table you can apply the substr in that query.

Similar Messages

How to use substr in external table defnition.

Hi All,
Im using oracle 11g. I have an external table which is reading data from a file. For one of the column, i need to get only the first 250 characters. My external table defnition looks like this
create table tbl_substr
( col1 varchar2(20),
col2 varchar2(250)
organization external
( type oracle_loader
default directory XXXX
access parameters (
records delimited by newline
FIELDS TERMINATED BY '|'
missing field values are null
( col1 ,
col2 "substr(:col2,1,250)"
) ) location ('file.txt') )
reject limit unlimited
But this defnition gives an error when i do select * from tbl_substr
I want to use substr in external table defnition its self and not in SELECT. Also i dont want to crete a view to solve this. If anyone has done this please help.

You need to play with COLUMN_TRANSFORMS
http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/et_params.htm#sthref1792
BTW, i too got it from Google. I was not aware about this :)
Amardeep Sidhu

UPPER() in external table definition

Hi,
Is it possible to apply the UPPER() function to character columns within an external table definition using the ORACLE_LOADER driver?
I've tried myself and have looked through the documentation, and it seems like this is not possible ... just wanted to double-check before moving forward with a different solution.
Thanks.
- k r
Message was edited by:
kenny r.

I've never seen an example of doing it nor do I see anywhere in the syntax that lends itself to such manipulation.
Consider creating a view.

LTRIM in an External Table Definition.

Hi,
We're using Oracle 11.2.
Seeing as there's little documentation on the Web about non-general External Table definitions, I want to put this question out there.
How do I put an Ltrim function into an External table definition?

Thanks,
That's one way would be to use a view on an external table definition but I know there's way to so it in the definition of the external table.
       MISSING FIELD VALUES ARE NULL
            REJECT ROWS WITH ALL NULL FIELDS
        CLAIM_NUMBER                  CHAR,
PATIENT_ZIP_CODE              CHAR NULLIF "PATIENT_ZIP_CODE"= '!',

UPPER function in external table definition ... ?

Greetings
Is there a way to specify a SQL function, like UPPER, in an external table column definition?
I know I can do it with SQL*Loader in the control file but I haven't found a way to do it in OWB?
I realize there are ways outside of the external table using a view, etc. but I'd like to reference the external table directly without having to create yet another object.
Thanks very much!!
-gary

Greetings
Is there a way to specify a SQL function, like UPPER, in an external table column definition?
I know I can do it with SQL*Loader in the control file but I haven't found a way to do it in OWB?
I realize there are ways outside of the external table using a view, etc. but I'd like to reference the external table directly without having to create yet another object.
Thanks very much!!
-gary

How to use lpad function in external table definition?

i need to modify one column of .csv file using lpad and want to insert it in external table.
lpad(AMT,3,'0')
Please send the code

Hi,
Please read SQL and PL/SQL FAQ
Additionally when you put some code or output please enclose it between two lines starting with {noformat}{noformat}
i.e.:
{noformat}{noformat}
SELECT ...
{noformat}{noformat}
The information you posted are quite poor. You don't expect to have an answer with such few details.
Anyway external table are read only.
Regards.
Al

KUP-01005 error in external table definition

Hello,
I'm facing an issue that I found it has been rose in the past but that never had an answer.
I'm using an external table to load a file on the database.
This file format is CSV (actually using ";" as delimiter) and it has header and trailer records that has the same format of data records, but contain field names in it.
E.g.:
FIELD_NAME1;FIELD_NAME2;...
I want to discard this records (for the header is easy, I can use SKIP 1 parameter) and the way I was trying to do this was using the following condition:
LOAD WHEN ( FIELD_NAME1 != "FIELD_NAME1" )
The problem arises when a record has the first field empty (hence NULL): the record is discarded.
Therefore I tried modifing the condition in this way:
LOAD WHEN ( ( FIELD_NAME1 != "FIELD_NAME1" ) OR ( ( FIELD_NAME1 = NULL ) )
but in this case I get the following error:
KUP-01005 : syntax error : found "null": expecting on of "blanks, double-quoted-string, hexprefix, identifier, single-quoted-string"
I've doublechecked Oracle reference and the NULL keyword is admitted - morover it is stated that a NULL value can be tested only against NULL value, otherwise the comparison returns false.
Can anyone help me solving this issue? Is it a bug in SQLLOADER Driver used by External Tables or an error in Oracle Reference?
Regards

It's an other option that I've already tried, but it doesn't work - and it is correct because in External Table statement I've specified to trim fields (LRTRIM), so there's no fields containing blanks (they're simply empty).
However, my main concern is that I tried to use a statement that should be correct - according to Oracle reference.
Instead I get an error regarding the syntax of the statement...
I think that I can workaround this problem - but it is a workaround, not a real solution... :-|

How read the flat data by external table definition. Please help me. Thanks

----Following is my code--
create or replace procedure AGE_UPLOAD_SER_RCD1 is
check_drop PLS_INTEGER;
begin
SELECT COUNT(*) INTO check_drop FROM USER_TABLES WHERE TABLE_NAME='EXT_SERVICES_RECEIVED_UPLOAD';
IF (check_drop <> 0) THEN
EXECUTE IMMEDIATE 'DROP Table EXT_SERVICES_RECEIVED_UPLOAD';
END IF;
EXECUTE IMMEDIATE 'CREATE Table EXT_SERVICES_RECEIVED_UPLOAD
ALIEN_NUMBER number(9),
SOCIAL_SECURITY_NUMBER number(9),
INTAKE_DATE          date,
CLOSURE_DATE          date,
CLOSURE_REASON          varchar(200)
ORGANIZATION external
TYPE oracle_loader
DEFAULT DIRECTORY UPLOAD_STAGE_AREA_AE
ACCESS parameters
RECORDS DELIMITED BY ''|''
BADFILE ''ext_services_received.bad''
DISCARDFILE ''ext_services_received.dis''
LOGFILE ''ext_services_received.log''
SKIP 1
FIELDS TERMINATED BY '','' OPTIONALLY ENCLOSED BY ''"''
LOCATION (''AGE_SERVICES_RECEIVED2.TXT'')
REJECT LIMIT unlimited';
end AGE_UPLOAD_SER_RCD1;
--Problem is Unable to read the data-----
---Following is the error log ----
LOG file opened at 07/19/06 15:57:23
Field Definitions for table EXT_SERVICES_RECEIVED_UPLOAD
Record format DELIMITED, delimited by |
Data in file has same endianness as the platform
Rows with all null fields are accepted
Fields in Data Source:
ALIEN_NUMBER CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
SOCIAL_SECURITY_NUMBER CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
INTAKE_DATE CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
CLOSURE_DATE CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
CLOSURE_REASON CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
KUP-04021: field formatting error for field SOCIAL_SECURITY_NUMBER
KUP-04023: field start is after end of record
KUP-04101: record 7 rejected in file /data/External_Files/Data_Uploads/DB_Area/Adult_Ed/AGE_SERVICES_RECEIVED2.TXT
error processing column INTAKE_DATE in row 2 for datafile /data/External_Files/Data_Uploads/DB_Area/Adult_Ed/AGE_SERVICES_RECEIVED2.TXT
ORA-01843: not a valid month
error processing column INTAKE_DATE in row 3 for datafile /data/External_Files/Data_Uploads/DB_Area/Adult_Ed/AGE_SERVICES_RECEIVED2.TXT
ORA-01843: not a valid month
error processing column INTAKE_DATE in row 4 for datafile /data/External_Files/Data_Uploads/DB_Area/Adult_Ed/AGE_SERVICES_RECEIVED2.TXT
ORA-01843: not a valid month
error processing column INTAKE_DATE in row 5 for datafile /data/External_Files/Data_Uploads/DB_Area/Adult_Ed/AGE_SERVICES_RECEIVED2.TXT
ORA-01843: not a valid month
error processing column INTAKE_DATE in row 6 for datafile /data/External_Files/Data_Uploads/DB_Area/Adult_Ed/AGE_SERVICES_RECEIVED2.TXT
ORA-01843: not a valid month

What does the data look like for INTAKE_DATE field in the data file?

Error while selecting date from external table

Hello all,
I am getting the follwing error while selecting data from external table. Any idea why?
SQL> CREATE TABLE SE2_EXT (SE_REF_NO VARCHAR2(255),
2        SE_CUST_ID NUMBER(38),
3        SE_TRAN_AMT_LCY FLOAT(126),
4        SE_REVERSAL_MARKER VARCHAR2(255))
5 ORGANIZATION EXTERNAL (
6    TYPE ORACLE_LOADER
7    DEFAULT DIRECTORY ext_tables
8    ACCESS PARAMETERS (
9      RECORDS DELIMITED BY NEWLINE
10      FIELDS TERMINATED BY ','
11      MISSING FIELD VALUES ARE NULL
12      (
13        country_code      CHAR(5),
14        country_name      CHAR(50),
15        country_language CHAR(50)
16      )
17    )
18    LOCATION ('SE2.csv')
19 )
20 PARALLEL 5
21 REJECT LIMIT UNLIMITED;
Table created.
SQL> select * from se2_ext;
SQL> select count(*) from se2_ext;
select count(*) from se2_ext
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04043: table column not found in external source: SE_REF_NO
ORA-06512: at "SYS.ORACLE_LOADER", line 19

It would appear that you external table definition and the external data file data do not match up. Post a few input records so someone can duplicate the problem and determine the fix.
HTH -- Mark D Powell --

External Table Help

I have created an external table but when i try to select data it seems to be empty, where could be the problem?
Thanks

The log file and bad file are part of the external table definition. I'm not where where Oracle Warehouse Builder (OWB) would put those files-- they may be in the same directory as the data file. You could check the external table definition to be sure, though.
Justin
Distributed Database Consulting, Inc.
http://www.ddbcinc.com/askDDBC

External table error in windows environment

Can i create external table from directory which is on network drive not on local server where oracle is intsalled. I am receiving follwoing error when I try to query external table poinittng to directory object on network drive in windows. Thanks
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04063: unable to open log file BRIDGIES_EXTERNAL_4564_4844.log
OS error The system cannot find the file specified.
ORA-06512: at "SYS.ORACLE_LOADER", line 19

I suspect not. Copy the file to a local disk location, point the directory or a directory to it, update the external table definition if necessary, repeat. If this run works then you have your answer.
HTH -- Mark D Powell --

"File" tab in external table editor replaced by "Access Parameters"?

Hi,
I have a few external tables where the "File" tab in the external table editor is replaced by an "Access Parameters" tab.
Does anybody know what may have caused this or how to repair it?
Thanks,
Ed

Hi Ed
I have had that happen many times but have not figured out the root cause. The fix I used was pretty basic. Delete the external table and redefine it.
This has happened to me mostly when I move an external table definition from our TEST repository to PROD. It seems to lose the file definition it was originally attached to.
Since my external tables look exactly like my flat file sources it takes only a few seconds to recreate the external table.
I didn't spend any more time trying to figure out the 'why'.
Good luck.
-gary

Headers appear in external table

I have defined several external tables over text files that have been built in the same way - in excel, then saved as a csv file.
Most of them work fine but there is one that always returns the column headers as data. I cant understand how this happens for this one file when they are all built in the the same way. I have checked the ddl for the external tables and they are equivalent.
I realise I could set it to skip 1 row but I am loathe to do that because if this is some sort of bug then it may be resolved later resulting in the first row of data being lost - a problem that may not become obvious until found by end users.
I have tried rebuilding from scratch and get the same result.
Has anyone else faced this situation?

I don't know what the exact trick is that OWB does,
but when you sample the file you use for the external
table and tell OWB to use the first row for the
header names, it won't use the first line as a data
line, even though there's nothing to be found in the
external table definition that indicates that.Hi,
that's no trick. If you sample the file and use the first row for column names the first row only disapears in the wizard. The resulting external table definition doesn't contain a skip row statement in this case automatically. The reason why you don't recognize this is because per default the loader writes no bad files. So the header row is normally ignored (because it doesn't match the structure) and you don't find it in the external table. If you will configure your external tables based on a file with an header and a skip parameter = 0 to produce a bad file (Bad File Name = file.bad) you will find the header row in this bad file then.
The only right way ist to configure the skip parameter - not to repress the resulting errors.
In this special file I guess the header matches the structure of the external table, so it isn't ignored but loaded into the external table.
Regards,
Detlef

How to remove carraige return from the field while loading external table

I am facing an issue of not getting rid of carraige returns present in the fileds of the source .csv file while loading the records into external table.
I had tried using LRTRIM, but it does not help.
The error I am getting is:
KUP-04021: field formatting error for field POPULATION_DESCRIPTION
KUP-04037: terminator not found
I am pasting one record out of the .csv file which is causing this error as below:
"Business Card Accounts
",123,7 BizCard - Gamers,Control,"Business Card Accounts
",75270,75271
You can see the carraige return in the 1st field as well as 5th field. Filed are separated by commas & eclosed by double quotes.
Could anybody help on this please?

Can you copy the file to an external table only version, and then dos2unix it?
Alternatively, you can use the ACCESS PARAMETERS area in your external table definition to set that your RECORDS DELIMITED BY carriage returns, instead of the default.
Check out the example here http://www.psoug.org/reference/externaltab.html

Use of External tables to load XML data.

Hi,
I have used external table definitions to load various XML files to the database, usually splitting the XML into seperate records - 1 per major element tag, and using PL/SQL to parse out a primary key to store in a relational table with all of the XML relevant to that primary key value stored as an XMLTYPE coumn in a row of the table. This has worked fine for XML with a single major entity (element tag) .
However, I now have an XML file that contains two "major" elements (both children of the root) that I would like to split out and store in seperate tables.
The XML file is of the following basic format:-
<drugs>
<drug>drug 1...</drug>
<drug>drug 2...</drug>
<partners>
<partner>partner 1</partner>
<partner>partner 2</partner>
</partners>
</drugs>
The problem is there are around 18000 elements of the first type, followed by several thousand of the 2nd type. I can create two seperate external tables - one for each element type, but how do I get the external table for the 2nd to ignore all the elements of the first type? My external table definition is :-
CREATE TABLE DRUGBANK_OWNER.DRUGBANK_PARTNERS_XML_EXTERNAL
DRUGBANK_XML CLOB
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY DRUGBANK_DIR
ACCESS PARAMETERS
( records delimited by "</partner>" SKIP 100000
characterset al32utf8
badfile extlogs:'drugbank_partners_xml.bad'
logfile extlogs:'drugbank_partners_xml.log'
discardfile extlogs:'drugbank_partners_xml.dis'
READSIZE 52428800
fields
drugbank_xml CHAR(50000000) terminated by '</partners>'
LOCATION (DRUGBANK_DIR:'drugbank.xml')
REJECT LIMIT UNLIMITED
PARALLEL ( DEGREE 8 INSTANCES 1 )
NOMONITORING;
The problem is that before the first <partners> element the 1800 or so <drugs> elements cause a data cartrdige error:-
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-29400: data cartridge error
KUP-04020: found record longer than buffer size supported, 52428800
This happens regardless of the value of the SKIP or the size of the drugbank_xml field.
I have tried using an OR on the "records delimited by" access parameter, to 'delimit by "</partner>" OR "</drug>"', with the intention of filtering out the <drug> elements but this leads to a syntax error.
Anyone ever tried anything similar and got it to work?
Any other suggestions?
Thanks,
Sid.

No, the content inside quotes is spanned across multiple lines....there are line breaks after every html tag.
"What's the error message you are getting?"
Iam not getting any error while selecting from external table , but I am getting tose rows in BAD file and log file has the following entries
KUP-04021: field formatting error for field TKBS_DSCN
KUP-04036: second enclosing delimiter not found
Message was edited by:
user627610

Substr in external table definition

Similar Messages

Maybe you are looking for