SQL LOADER USING EXTRNAL TABLE

I have .csv file having around 70k records
in which fields are delimited by tab and
enclosed in double quotes but double quotes may be part of data.
and records are delimited by newline.
After creating external table when I issue SELECT statment
select count(*) from proTxt ;
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04020: found record longer than buffer size supported, 524288, in C:\Program Files\Apache Software Foundation\Tomcat
5.5\webapps\tmTest\upload\product\Data\output09_1.txt
ORA-06512: at "SYS.ORACLE_LOADER", line 19
Following is the create table statement:
CREATE TABLE proTxt (PRO_CODE VARCHAR2(30),
PRO_DESC VARCHAR2(500),
PUR_PRICE VARCHAR2(20),
SALE_PRICE VARCHAR2(20)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER
DEFAULT DIRECTORY DAT_DIR
ACCESS PARAMETERS
records delimited by NEWLINE SKIP 1
badfile BAD_DIR:'proTxt%a_%p.bad'
logfile LOG_DIR:'proTxt%a_%p.log'
fields terminated by X'9' OPTIONALLY ENCLOSED BY '"' AND '"'
missing field values are null
( PRO_CODE,
PRO_DESC,
PUR_PRICE,
SALE_PRICE
LOCATION ('output09_1.txt')
PARALLEL 4
REJECT LIMIT UNLIMITED;
record size is not large.
Log file :
LOG file opened at 12/05/12 20:25:40
KUP-04020: found record longer than buffer size supported, 524288, in C:\Program Files\Apache Software Foundation\Tomcat 5.5\webapps\tmTest\upload\product\Data\output09_1.txt
KUP-04053: record number 2
data file
PRO_CODE     PRO_DESC     PUR_PRICE     SALE_PRICE
"0000336658"     "BEARING"     "Rs.0.00"     "Rs.0.00"
"0000790028"     "SEAL"     "Rs.76.00"     "Rs.90.00"
"0000790118"     "SPRING"     "Rs.24.00"     "Rs.28.00"
"0000792284"     "F.BRK.CAL.W/O PA"     "Rs.2,627.00"     "Rs.3,100.00"
"0000792285"     "F.BRK.CAL.W/O PA"     "Rs.2,627.00"     "Rs.3,100.00"
"0005896322"     "PISTON, RING"     "Rs.5,000.00"     "Rs.5,900.00"
"0005896323"     "PISTONS, RINGS AND P"     "Rs.17,755.00"     "Rs.20,951.00"
"0005896559"     "PISTON, RINGS AND PI"     "Rs.5,000.00"     "Rs.5,900.00"

Hi,
when i used
records delimited by *'\r'*
then 4226 record written to table
but enclosed charcter double quotes["] were also written and
there is some space between charcters
" 0 0 0 0 3 3 6 6 5 8 " " B E A R I N G " " R s . 0 . 0 0 " " R s . 0 . 0 0 "
" 0 0 0 0 8 5 6 7 0 7 " " P L U G " " R s . 0 . 0 0 " " R s . 0 . 0 0 "
Definitely this is "External Table with Flatfile Moved Across Platforms" issue.
when I opened .csv file in excel and saved as tab delimited it works fine.
But I do not know plateform of data file.
How to know the CHARACTERSET of data file
Log file
Field Definitions for table PROTXT
Record format DELIMITED, delimited by
Data in file has same endianness as the platform
Rows with all null fields are accepted
Fields in Data Source:
PRO_CODE CHAR (255)
Terminated by "9"
Enclosed by """ and """
Trim whitespace same as SQL Loader
PRO_DESC CHAR (255)
Terminated by "9"
Enclosed by """ and """
Trim whitespace same as SQL Loader
PUR_PRICE CHAR (255)
Terminated by "9"
Enclosed by """ and """
Trim whitespace same as SQL Loader
SALE_PRICE CHAR (255)
Terminated by "9"
Enclosed by """ and """
Trim whitespace same as SQL Loader

Similar Messages

SQL *Loader and External Table

Hi,
Can anyone tell me the difference between SQL* Loader and External table?
What are the conditions under we can use SQL * Loader and External Table.
Thanx

External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
Justin

SQL Loader versus External Table

If anyone has worked on external tables please let me know your views.

user637544 wrote:
for sqlldr i follow the following approach
1. truncate table
2. call sqlldr and load records in staging table
3. check for bad records
how do i tuncate, call external table and check bad records for external table.As part of the SQL*Loader control file you can tell it to truncate the table as part of the process before it loads in the data.
The key differences between SQL*Loader and External Tables are:
1. SQL*Loader is an external utility run from the o/s command line, whereas External tables are usable from within SQL and PL/SQL.
2. SQL*Loader can only load data up into the database, whereas External tables can allow you to read files and write files (10g onwards)
3. SQL*Loader has limited control on skipping rows etc. whereas External tables can be queried using SQL and has all the flexibility that SQL can offer.
4. SQL*Loader requires connection information to the database, whereas External tables are already part of the database
5. SQL*Loader can only change the source filename through dynamic o/s level scripts, whereas External tables can have the filename changed using an ALTER TABLE ... LOCATION ... command.
If you want to truncate your staging table before loading it with data from the external table, then you will just have to do that as a seperate step, because the external table is not associated with other tables. Simply truncate the other table and then query the data from the external table into the staging table.
External tables offer a lot more flexibility than using SQL*Loader.

Sql loader or external table

Hi all good morning.
can we write a stored procedure that loads a file into the table using sql loader?
can Java call the above stored procedure?
regards
raj
Edited by: user10887630 on Apr 23, 2009 6:18 AM

Are you saying the files themselves can't be located on the server? or just that the process for loading the files can't be located on the server.
If the files themselves cannot reside on the server then you won't be able to use SQL*Loader or External tables from within stored procedures. It would require some clever Java code type stuff to get across to a.n.other machine where the files are stored and get the data.
Typically it is normal for such files to be located on the server. What reasons do the DBA's give for not wanting them there?
DBA's say that application specific files cannot be maintained on the Database serverSo what are all the database data files then? They hold data from the applications. ;)

Loading millions of rows using SQL*loader to a table with constraints

I have a table with constraints and I need to load millions of rows in it using SQL*Loader.
What is the best way to do this, means what SQL*Loader options to use, for getting the best loading performance and how to deal with constraints?
Regards

- check if your table has check constraints (like column not null)
if you trust the data in the file you have to load you can disable this constrainst and after the loader enable this constrainst.
- Check if you can modify the table and place it in nologging mode (generate less redo but ONLY is SOME Conditions)
Hope it helps
Rui Madaleno

Export table format SQL loader using spool

Hi
In enviroment I only access for to do SELECT , then I want to export some tables using SELECT using spool of data
But I wanted export in format Sql Loader for to import other environment
Is possible to do It ?

Depends where the NULL is.
SQL> SELECT * FROM t ORDER BY 1;
        ID DESCR      DT
         1 One
         2            30-MAY-2007
         4 Four       30-MAY-2007
           Three      30-MAY-2007
SQL> SELECT id||',"'||descr||'",'||TO_CHAR(dt,'dd-mon-yyyy') output
2 FROM t
3 ORDER BY 1;
OUTPUT
,"Three",30-may-2007
1,"One",
2,"",30-may-2007
4,"Four",30-may-2007John

Sql*loader and nested tables

I'm having trouble loading a nested table via sqlldr in Oracle 10g and was hoping someone could point me in the right direction. I keep getting the following error:
SQL*Loader-403: Referenced column not present in table mynamespace.mytable
Here's an overview of my type and table definitions:
create type mynamespace.myinfo as object
i_name varchar2(64),
i_desc varchar2(255)
create TYPE mynamespace.myinfotbl as TABLE of mynamespace.myinfo;
create table mynamespace.mytable
Info mynamespace.myinfotbl,
note varchar2(255)
NESTED TABLE Info STORE AS mytable_nested_tab;
My control file looks like this:
load data
infile 'mydatafile.csv'
insert into table mynamespace.mytable
fields terminated by ',' optionally enclosed by '"'
TRAILING NULLCOLS
Info nested table count(6)
Info column object
i_name char(64),
i_desc char(255)
note
Example mydatafile.csv would be something like:
lvl1,des1,lvl2,des2,lvl3,des3,lvl4,des4,lvl5,des5,lvl6,des6,a test data set
I can't figure out why sqlldr keeps rejecting this control file. I'm using 'direct=false' in my .par file.
Any hints?

I just noticed that my email is wrong. If you can help, plese send email to [email protected]
thanks.

SQL Loader AND Temporary Tables

Hi :
Have a Couple of Questions..
(1) Is SQL Loader not part of the ORACLE DB 9.X?? Coz I could not find that exe.
(2) Can we have the Data file on the Client Machine?? OR is it mandatory that the data file should be on the Server to load the data?

SQL loader is part of Oracle9i and you can use its driver to create external table.Moreover for external tables datafiles should be on server.

Sql loader using position and functions

Hi all, i need help loading some data in my table using sql loader. consider the following
CREATE TABLE er
a1 NUMBER,
a2 number,
a3 VARCHAR2(100),
a4 VARCHAR2(100),
a5 VARCHAR2(100),
a6 VARCHAR2(100),
a7 VARCHAR2(100),
a8 VARCHAR2(100)
OPTIONS (BINDSIZE=20548000, READSIZE=20548000, STREAMSIZE=20548000, DATE_CACHE=25000, SKIP=0)
LOAD DATA
INTO TABLE er
APPEND
TRAILING NULLCOLS
a1            POSITION(0001:0021)               ,
a2            POSITION(0022:0042)       "DECODE(SUBSTR(:a2,1,3),'***',NULL,:a2)"      ,
a3            POSITION(0043:0053)       ,
a4            POSITION(0054:0064)          ,
a5            POSITION(0065:0075)           ,
a6            POSITION(0076:0086)       ,
a7            POSITION(0087:0093)      "DECODE(SUBSTR(:a7,1,3),'***',NULL,:a7)"
BEGIN
                 0.00 ******************** X          X          X          *X          ****
END;if you look at the data, some fields have a lot of * and some has af few such as ****. i want to load this data into a table and when a field contain all * as a value, i want to set it to null. if a field contain a * and alphanumeric then that value should be load as it is.
in the example above, ******************** should be set to null and **** should also be set to null. notice that there is a field with X. since this field contain alpha numeric, it should be loaded into the table as is. the only time field should be set to null is when the value contain all .
somebody in this forum suggest using decode but it looks like it is not working and i get error when it reads second field and try to insert into a2 number column.
is there any way to use regular expression to find out if a field contain all *. also i want to trim each field since they might contain leading spaces.
can some one help with this using the sqlloader ctl and data above?

You can include regular expression in you SQL*Loader control file.
An example can be found here:
http://www.morganslibrary.org/reference/sqlloader.html
Demos 7 and 8 using the UPPER and DECODE functions to illustrate how to do it.

Sql Loader Vs External table? Which one is preffered?

Hello guru,
We trying to load the data into about 160 tables. Here are the following things we are doing currently,
1. We get the flat file in .txt format
2. We are writing the control file for each flat file for thier respective tables
3.And we are using sql loader command to load this data into the table.
I was wondering if this process is cumbersome, and i m not sure if external table could be simple in loading the tables compared to what we are doing ?And i have not used external tables, so wanted to know which on eis better ? So any idea is greatly appriciated !
FYI.. Version :- Oracle 11g
Thank you so much!

Thanks for you reply justin !
I found the below example for loading the data into external table...
CREATE OR REPLACE DIRECTORY dat_dir AS 'C:\Oradata\Data';
CREATE OR REPLACE DIRECTORY log_dir AS 'C:\Oradata\Log';
CREATE OR REPLACE DIRECTORY bad_dir AS 'C:\Oradata\Bad';
GRANT READ ON DIRECTORY dat_dir TO scott;
GRANT WRITE ON DIRECTORY log_dir TO scott;
GRANT WRITE ON DIRECTORY bad_dir TO scott;
CREATE TABLE revext (person      VARCHAR2(20),
                     rev_jan     NUMBER(4),
                     rev_feb     NUMBER(4),
                     rev_mar     NUMBER(4),
                     rev_apr     NUMBER(4),
                     rev_mai     NUMBER(4),
                     rev_jun     NUMBER(4),
                     rev_jul     NUMBER(4),
                     rev_aug     NUMBER(4),
                     rev_sep     NUMBER(4),
                     rev_oct     NUMBER(4),
                     rev_nov     NUMBER(4),
                     rev_dez     NUMBER(4))
ORGANIZATION EXTERNAL
   TYPE ORACLE_LOADER
   DEFAULT DIRECTORY dat_dir
   ACCESS PARAMETERS
     records delimited by newline
     badfile bad_dir:'revext%a_%p.bad'
     logfile log_dir:'revext%a_%p.log'
     fields terminated by ','
     missing field values are null
     ( person,
       rev_jan,
       rev_feb,
       rev_mar,
       rev_apr,
       rev_mai,
       rev_jun,
       rev_jul,
       rev_aug,
       rev_sep,
       rev_oct,
       rev_nov,
       rev_dez
   LOCATION ('revext.dat')
PARALLEL 4
REJECT LIMIT UNLIMITED;
CREATE TABLE revenue (
    person       VARCHAR2(20),
    month        VARCHAR2(3),
    revenue      NUMBER,
    CONSTRAINT revenue_pk PRIMARY KEY (person,month));
INSERT INTO revenue (person,month,revenue)
   SELECT person,'Jan',rev_jan
   FROM revext--but currently we are using sql loader, our data looks like this
1119Smith      01/01/1982AXYZ corporation xyz corp
1111collen      01/01/1990AABC corporation abc corp
     and control file is like this
INTO TABLE "XYZ_tbl"
   ID                     POSITION(01:05)        CHAR                  "DECODE(RTRIM(:ID), NULL, 'NA', :ID)"       ,
   Name                POSITION(06:15)        CHAR                  "DECODE(RTRIM(:NAME), NULL, 'NA', :Name)"   ,
   Act_dt              POSITION(16:25)        DATE                  "MM/DD/YYYY" NULLIF ACT_DT=BLANKS
My question is, can i use the options like NULLIF/ DECODE or changing datatypes when loading or functions like REPLACE / TO_DATE in External tables ? Any idea? Any example code or SQL will great
Thank you so much!

SQL*Loader or external table for load a MSG (email) file

Hi there!
I'm looking for a way to load an email in a Oracle DB.
I mean, not all the email's body in a column, but to "parse" it in a multi column/table fashion.
Is it possible to do with a sql*loader script or an external table?
I think it is not possible, and that I must switch to XML DB.
Any idea?
Thanks,
Antonio

Hello,
Why don't you just load the entire MSG (email) as clob into one email_body column or whatever column name you want to use.
To load data upto 32k, you can use varchar2(32656) but its not a good idea to load clob in that manner because it's very inconsistent as length can
vary resulting in string literal too long. So you have 2 choices now, first you have to use either procedure or anonymous block to load clob data.
First Method -- I loaded alert.log successfully and you can imagine how big this file can be (5MB in my test case)
CREATE OR REPLACE DIRECTORY DIR AS '/mydirectory/logs';
DECLARE
   clob_data   CLOB;
   clob_file   BFILE;
BEGIN
   INSERT INTO t1clob
   VALUES (EMPTY_CLOB ())
   RETURNING clob_text INTO clob_data;
   clob_file   := BFILENAME ('DIR', 'wwalert_dss.log');
   DBMS_LOB.fileopen (clob_file);
   DBMS_LOB.loadfromfile (clob_data,
                          clob_file,
                          DBMS_LOB.getlength (clob_file)
   DBMS_LOB.fileclose (clob_file);
   COMMIT;
END;Second Method: Use of Sqlldr
Example of controlfile
LOAD DATA
INFILE alert.log "STR '|\n'"
REPLACE INTO table t1clob
   clob_text char(30000000)
)Hope this helps

Sql loader master detail tables

Hi
I am trying to load master detail tables using sql loader. I have a sequence in the master table as primary key and have to load primary key in detail table. is there any way i can query up the primary key by name in the control file. please whats the best way to do this.
can i query up the primary key in the master table by the name in detail table which is jan09.
e.g
master data
jan09,description
detail data
jan09,'test1','100,'y'
LOAD DATA
INFILE 'test.data'
insert
INTO TABLE master_table
fields terminated by ',' optionally enclosed by '"' trailing nullcols
(id position(1:2) "myseq.nextval",
code char,
creation_date date sysdate,
last_update_date date sysdate,
into TABLE BDETAILS
fields terminated by ',' optionally enclosed by '"' trailing nullcols
(id number "select id from master_table where code=" || :code
name char,
amt decimal,
flag char
)

Hey Here is what you need.
You will have to understand and test this properly.
It took me quiet some time before i could get to this stage
Here is the test data with all the commands.
As far the ID in the detail table is concerned you have no option but to update that column once the loading is complete.
1) create Table
CREATE TABLE master_table(ID NUMBER,code VARCHAR2(50),creation DATE,last_update DATE);
CREATE TABLE bdetails(ID NUMBER,NAME VARCHAR2(100),amt NUMBER,flag VARCHAR2(10));
ALTER TABLE master_table ADD (chc VARCHAR2(10));2) Create Sequence
CREATE SEQUENCE testseq INCREMENT BY 1;3) CTL FILE
LOAD DATA
INFILE 'c:\sqlldr\test.txt' /*path of data file */
append
INTO TABLE master_table
when (1) ='M' /*to check if the record in the data file is a master record */
fields terminated by ',' optionally enclosed by '"' trailing nullcols
(id expression "testseq.nextval", /*select next value from sequence */
mcol1 filler, /*FILLER keyword is sused to skip first column from data file */
chc,
code ,
creation "sysdate",
last_update "sysdate"
Into TABLE BDETAILS
when (1) ='D' /*to check if the record in the data file is a Detail record */
fields terminated by ',' optionally enclosed by '"' trailing nullcols
(id expression "testseq.currval", /* UNFORTUNATELY THIS PICKS UP THE LAST ID OF THE MASTER RECORD SESSION STUFF GOING ON HERE*/
col1 filler position(1:2), /* specifying this POSITION(1:2) is very very important as you need to reset the cursor to start of the line for the next record */
name ,
amt ,
flag )4) SQLLDR Command
sqlldr scott/tiger data=c:\sqlldr\test.txt control=c:\sqlldr\1.ctl
5) data file
M,jan09,description
D,test1,100,y
M,feb09,description1
D,test2,200,x
M,mar09,description2
D,test3,200,xHope this helps you and give you a lead how to proceed.
Cheers!!!
Bhushan

Sql loader using sql statement

Hi All ,
May i know any way used sql loader refer sql statement e.g "select a from test"
import data into other table ?
if you have any sample or link , please let me know , thanks a lot

If you are using the database steps, the answer is yes. I've done it by creating a data source that uses the Microsoft Text Driver in the ODBC administrator. The Open Database step is then configured like any other ODBC database. In the .csv file, I have the first line define the column names (i.e. column1, column2) and the SQL statement I use is "SELECT * FROM junk.txt". I haven't tried anything with a WHERE clause but that should work as well.

Sql*loader using JSP

wht is the way of using sqlldr in JSP script to load data Oracle DB from a csv file.
i know how we do sqlldr in cmd.
can we provide the sqlldr command to system by using jsp if yes how?

SQL*Loader is a command-line utility. In theory, since Java can call out to the operating system, you could invoke SQL*Loader on the application server to load data into Oracle. It would probably be more appropriate, though, to copy the CSV to the database server and make use of an external table in the database to do the load.
Justin

Sql loader utl_file & external table

can any one let me know the differences between.
1.sql loader
2.utl_file
3.external table
Regards.
Asif.

To expand on Aron's answer....
SQL*Loader - An operating system utility which uses control files (which you create) to load data files onto database tables.
UTL_FILE - A database package which can be used for reading and writing files in any format you care to design programmatically.
External Table - The latest thing which can be used instead of SQL*Loader. This is done from the database end, by creating a table as an external table and pointing it at the source file on the operating system. It also allows information similar to that put in the SQL*Loader control files to be specified against the table. By querying against the table you are in fact querying against the source file. There are some limitation compared to regular database tables such as there is no ability to write to the external table.
;)

SQL LOADER USING EXTRNAL TABLE

Similar Messages

Maybe you are looking for