SQL Loader: ignore data field

Hi,
in a SQL Loader control file, how do you specify that you do not wish to load a particular field?
E.g. if I wanted to ignore the second field in the data file and load the first and third fields into a table, how would I modify the control file below?
Thanks.
load data
infile 'example.dat'
into table example
fields terminated by ',' optionally enclosed by '"'
col1 char,
ignore char,
col2 char
)

User FILLER keyword:
col1 char,
ignore filler char,
col2 char
)

Similar Messages

SQL*Loader Sequential Data File Record Processing?

If I use the conventional path will SQL*Loader process a data file sequentially from top to bottom? I have a file comprised of header and detail records with no value found in the detail records that can be used to relate to the header records. The only option is to derive a header value via a sequence (nextval) and then populate the detail records with the same value pulled from the same sequence (currval). But for this to work SQL*Loader must process the file in the exact same sequence that the data has been written to the data file. I've read through the 11g Oracle® Database Utilities SQL*Loader sections looking for proof that this is what will happen but haven't found this information and I don't want to assume that SQL*Loader will always process the data file records sequentially.
Thank you

Oracle Support responded with the following statement.
"Yes, SQL*LOADER process data file from top to bottom.
This was touched in the note below:
SQL*Loader - How to Load a Single Logical Record from Physical Records which Include Linefeeds (Doc ID 160093.1)"
Jason

SQL Loader error while loading a date field

Hi,
I am getting the below error while I am trying to load a table with a date field using SQL Loader
Record 1: Rejected - Error on table RPT_HOST_USAGE, column USAGE_TIMESTAMP.
ORA-01861: literal does not match format string
My input file is as below
<code>
Host_Usage_ID,Host_ID,Technology_ID,Environment_ID,Usage_Timestamp,Avg_CPU_Pct,Avg_Memory_MB,CPU_Spike
1,12,1,8,'2009-08-01 00:00:00',0.000000000,23875.000000000,0.000000000
<code>
My Loader.ctl is
<code>
OPTIONS (SKIP=1)
load data
infile 'C:\rpt_Host_Usage.txt'
into table RPT_HOST_USAGE
fields terminated by ","
HOST_USAGE_ID,
HOST_ID,
TECHNOLOGY_ID,
ENVIRONMENT_ID,
USAGE_TIMESTAMP,
AVG_CPU_PCT,
AVG_MEMORY_MB,
CPU_SPIKE
<code>
I have tried options like USAGE_TIMESTAMP TO_DATE(USAGE_TIMESTAMP,'YYYY-MM-DD HH24:MI:SS') but didn't work...
Can you please tell me how to resolve this issue?
Any pointers on this will be helpful
Thanks
Mahesh

I went back and looked at some of my old *.ctl files and I did something simlilar to what you mentioned and it worked for me, but I had surrounded the option in double-quotes and included a colon in front of the item. Example:
TECHNOLOGY_ID,
ENVIRONMENT_ID,
USAGE_TIMESTAMP "to_date(:USAGE_TIMESTAMP,'YYYY-MM-DD HH24:MI:SS')",
AVG_CPU_PCT,
AVG_MEMORY_MB,
....

SQL Loader - Set a field to a CONSTANT and the data is date

Hi.
I have a question on SQL Loader.
I want to load a field(field name is TIME_IN).
In the Oracle table, this field type is DATE.
I my csv file, I do not put any value for this field.
I intend to use CONSTANT variable to this field.
In my control file, following is the command for above field.
Hoever, when I run the SQLLoader, it will return me error message which is INVALID MONTH.
I hope somebody can help and advise me on this matter.
Thanks.
INTO TABLE eqreceival_temp
FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"'
EQ_NO,
SIZE_TYPE,
CUSTOMER,
CONSTRUCTION,
QUALITY,
CONDITION,
DATE_IN DATE "DD/MM/YYYY",
TIME_IN CONSTANT '01/01/1999',
PARK_LOC CONSTANT 'CP',
STOCK CONSTANT 'S',
DISCHARGE_PORT CONSTANT 'MYPEN',
CTYPE CONSTANT 'P',
ROAD CONSTANT 'Y',
GATE CONSTANT 'Y',
SHIPMENT CONSTANT 'Y'
)

use to_data function when u r inserting....
or
give simply SYSDATE --- AND FIRST CHECK
--YOU MAY REMOVE CONSTANT KEY WORD.

SQL Loader - CSV Data file with carraige returns and line fields

Hi,
I have a CSV data file with occasional carraige returns and line feeds in between, which throws my SQL loader script off. Sql loader, takes the characters following the carraige return as a new record and gives me error. Is there a way I could handle carraige returns and linefeeds in SQL Loader.
Please help. Thank you for your time.
This is my Sql Loader script.
load data
infile 'D:\Documents and Settings\user1\My Documents\infile.csv' "str '\r\n'"
append
into table MYSCHEMA.TABLE1
fields terminated by ','
OPTIONALLY ENCLOSED BY '"'
trailing nullcols
( NAME CHAR(4000),
field2 FILLER,
field3 FILLER,
TEST DEPT CHAR(4000)
)

You can "regexp_replace" the columns for special characters

BI UDI data load conflict using MS SQL Server and date fields

Hi BW Experts!
We have found some unexpected problems when we are trying to perform a data extraction process from an SQL database (non-SAP application).
We are using the BI UDI tool (JDBC Driver/Connector).    The basic data extraction works properly, however we have some specific problems to work with SQL Server fields defined as Date Time.
The JDBC driver automatically intermediate the communication and translate these fields splitting the original SQL date time field into two separated fields with suffix _D (for date) and _T (for time).
It works perfect for extraction, but it doesnt work well when we try to restrict the data selection based on these fields.
When we put a date selection into the infopackage data selection area (we already have tried several date formats), the system simply ignores these selection parameters and send a SQL statement without WHERE clause to DBMS.
Please, anybody has experienced anything like this or know somethings that could help us?
This is a standard limitation from SAP BI UDI?
Thanks in advance and best regards,

Hi Matt and Thomas
Many thanks for your replies.
Yes I have to do the join. Even worse I have to write an Aggregate query like this!
Select o.F1,o.F2, m.F1,m.F2, count(*)
from
table@oracleServer o,
table@microsoftSQLServer m
where o.F1=m.F1
group by o.F1,o.F2, m.F1,m.F2;
These are 2 different systems (hardware & software) which actually do the same job, they produce transactions for a common business entity. Kind of, both sell tickets for the same ABC ferry company.
I can not put them side by side on the dashboard, as I need to aggregate the data and present it maybe in a Oracle BI Report for Accounting and Financial reconciliation purposes.
The truth is, as it is. I can't change it, is too late.
I have to device a way to harvest both systems as they are. I don't want to move around data. I thought looking at Oracle BI I could write SQL against multiple Data Sources. I am struggling to find a way. As it seems that it can "SQL query" 1 data source at a time.
I have been hinted on another forum (OracleTURK) to use Oracle Transparent Gateways and Generic Connectivity. http://download.oracle.com/docs/cd/B19306_01/server.102/b14232/toc.htm
Shame, I have to pay licenses for OWB and Oracle Transparent Gateways. I thought DB vendors can do better. Why do I have to pay to access the other 50% of the market.
I can understand the performance implications this might have. Because of it I might even be forced at the end to move data (ETL) into a separate database, slice it into partitions and manage it that way. Pitty because currenlty I only need one report out of these systems but seems like will be dear to get.
Thank you for all your help.
Kubilay

Using sql load insert multiple fields data into a single column in database

Hi ,
I have my log file in sun OS box something like this
=======
(07/29/2009 00:02:24.467) 367518 (07/29/2009 00:02:26.214) 949384011
(07/29/2009 00:02:26.236) 3675 (07/29/2009 00:02:28.207) 949395117
(07/29/2009 00:02:28.240) 337710 (07/29/2009 00:02:30.621) 949400864
=============
I am trying to insert the data into oracle data base as follows.
=============================
column1 : (07/29/2009 00:02:24.467)
column2 : 367518
column3 : (07/29/2009 00:02:26.214)
column4 : 949384011
===========================
Can anyone help me with the control file format?
someone suggested me the code below.
==========
LOAD DATA
INFILE 'D:\work\load.txt'
INTO TABLE sample
(col1 POSITION(02:24) char,
col2 POSITION(27:32) INTEGER EXTERNAL,
col3 POSITION(35:57) CHAR,
col4 POSITION(60:68) INTEGER EXTERNAL
===========
but this works only for the fixed length data? Please help

user11744904 wrote:
Hi ,
I have my log file in sun OS box something like this
=======
(07/29/2009 00:02:24.467) 367518 (07/29/2009 00:02:26.214) 949384011
(07/29/2009 00:02:26.236) 3675 (07/29/2009 00:02:28.207) 949395117
(07/29/2009 00:02:28.240) 337710 (07/29/2009 00:02:30.621) 949400864
=============
I am trying to insert the data into oracle data base as follows.
=============================
column1 : (07/29/2009 00:02:24.467)
column2 : 367518
column3 : (07/29/2009 00:02:26.214)
column4 : 949384011
===========================
Can anyone help me with the control file format?
someone suggested me the code below.
==========
LOAD DATA
INFILE 'D:\work\load.txt'
INTO TABLE sample
(col1 POSITION(02:24) char,
col2 POSITION(27:32) INTEGER EXTERNAL,
col3 POSITION(35:57) CHAR,
col4 POSITION(60:68) INTEGER EXTERNAL
===========
but this works only for the fixed length data? Please helpIs the requirement to load all data in a single column or multiple columns? The thread subject and body are conflicting.

SQL*Loader and data enclosed by single quotes

Hello,
I need to migrate some data from Sybase to Oracle. Can't use the Migration Workbench because this is Sybase's SQL Anywhere product. We are therefore trying to load via SQL*Loader.
Sybase outputs its data into text files and encloses it with single quotes, like:
'ABC123',
'UR94LL',
'7YUHII'
We are running into a problem because we cannot figure out how to use the OPTIONALLY ENCLOSED BY parameter with single quotes. If we just use the FIELDS TERMINATED BY parameter, then the entire string, including the quotation marks, gets loaded into the Oracle table. This causes bad records to be created when the data is exactly the length of the column width - the two quotation markss make the field longer than is allowed in the table. Plus - the data shouldn't have quotes around it in normal situations.
We can do 'Find and Replace' to replace the single quotation marks with double quotations, but some of the Sybase tables are huge, and we'd like to avoid having to open and edit them. However, if this is the only way to go, then we'll have to use it. I just wondered whether anyone had run into this before and been able to solve it.
Thanks.
-melissa

We are running into a problem because we cannot figure out how to use the OPTIONALLY ENCLOSED BY parameter with single quotesTry
OPTIONALLY ENCLOSED BY X'27'http://download-uk.oracle.com/docs/cd/B10501_01/server.920/a96652/ch06.htm#1015083

Sql*loader and sdo_geometry field

Is there a way to load longitude/latitude coordinates from a flat file directly into an sdo_geometry field with sql*loader? If so, does anyone have an example? I just want to create point features.
Thanks,
David

David,
Section 4.1.2 of the Spatial User Guide has examples for this:
LOAD DATA
INFILE *
TRUNCATE
CONTINUEIF NEXT(1:1) = '#'
INTO TABLE POINT
FIELDS TERMINATED BY '|'
TRAILING NULLCOLS (
GID INTEGER EXTERNAL,
GEOMETRY COLUMN OBJECT
SDO_GTYPE INTEGER EXTERNAL,
SDO_POINT COLUMN OBJECT
(X FLOAT EXTERNAL,
Y FLOAT EXTERNAL)
BEGINDATA
1| 2001| -122.4215| 37.7862|
2| 2001| -122.4019| 37.8052|
3| 2001| -122.426| 37.803|
4| 2001| -122.4171| 37.8034|
5| 2001| -122.416151| 37.8027228|
You need to add the SRID field if you want to add the SRID to the geometry.
Or you can do that with SQL after the data is loaded into the DB.
siva

SQL Loader error on Fields terminated and optionally enclosed

When you have fields terminated and optionally enclosed, you should be able to have the enclosure characters within the data by doubling them.
So if I have fields terminated by '|' optionally enclosed by '"' and my data has a double quote in it, I can get SQL Loader to take it by replacing the double quote with two double quotes.
But this doesn't always work. For some reason the following string gets the error "no terminator found after TERMINATED and ENCLOSED field"
|"TRO2 ldj?=)(/&%ç*""aàéè_:;!àèé,jpdjbdbd"|
Apparently having the double quote immediately after an asterisk makes SQL Loader not see the second double quote. If I edit it and remove the two double quotes it loads OK. Also if I change the string to
|"TRO2 ldj?=)(/&%ç*aà""éè_:;!àèé,jpdjbdbd"|
Moving the double quote away from the asterisk it loads OK.
Anybody encounter this problem before?

Oracle version is 11.1.0.7.0
OS is Windows 6.1.7601
I can't remove the "optionally enclosed by" clause because we're getting the data from another system. Fortunately this is test data and this particular combination of characters is unlikely to occur in production data.

SQL Loader deletes data, after reporting a file not found error

I have several control files beginning:
LOAD DATA
INFILE <dataFile>
REPLACE INTO TABLE <tableName>
FIELDS TERMINATED BY '<separator>'
When running SQL Loader, in the case of one particular control file, if the file referenced does not exist, SQL Loader first reports that the file could not be found, but then proceeds to delete all the data in the table in which the import was meant to take place. The corresponding log file reveals that the file could not be found, but also states that 0 records were loaded and 0 records were skipped.
In the case of all other control files, if the file is not found, the log files simply report this exception but do not show any statisitcs about the number of records loaded/skipped nor does SQL Loader delete the data in any of the referenced tables. This is obviously the expected behaviour.
Why is SQL Loader deleting the data referenced by one particular control file, even though this file does not exist and the corresponding log file has correctly picked up on this?

in the ressource name box of your file model, when you push the search button ("...") do you see the file ?
Cause the problem can occur when you write directly the path without selectionning the file with the assistant.
Try this.
I think too that that you can't see the data by right clicking and selectionning View Data ?
Let me know the avancement...

Sql loader maximum data file size..?

Hi - I wrote sql loader script runs through shell script which will import data into table from CSV file. CSV file size is around 700MB. I am using Oracle 10g with Sun Solaris 5 environment.
My question is, is there any maximum data file size. The following code from my shell script.
SQLLDR=
DB_USER=
DB_PASS=
DB_SID=
controlFile=
dataFile=
logFileName=
badFile=
${SQLLDR} userid=$DB_USER"/"$DB_PASS"@"$DB_SID \
          control=$controlFile \
          data=$dataFile \
          log=$logFileName \
          bad=$badFile \
          direct=true \
          silent=all \
          errors=5000Here is my control file code
LOAD DATA
APPEND
INTO TABLE KEY_HISTORY_TBL
WHEN OLD_KEY <> ''
AND NEW_KEY <> ''
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
        OLD_KEY "LTRIM(RTRIM(:OLD_KEY))",
        NEW_KEY "LTRIM(RTRIM(:NEW_KEY))",
        SYS_DATE "SYSTIMESTAMP",
        STATUS CONSTANT 'C'
)Thanks,
-Soma
Edited by: user4587490 on Jun 15, 2011 10:17 AM
Edited by: user4587490 on Jun 15, 2011 11:16 AM

Hello Soma.
How many records exist in your 700 MB CSV file? How many do you expect to process in 10 minutes? You may want to consider performing a set of simple unit tests with 1) 1 record, 2) 1,000 records, 3) 100 MB filesize, etc. to #1 validate that your shell script and control file syntax function as expected (including the writing of log files, etc.), and #2 gauge how long the processing will take for the full file.
Hope this helps,
Luke
Please mark the answer as helpful or answered if it is so. If not, provide additional details.
Always try to provide actual or sample statements and the full text of errors along with error code to help the forum members help you better.

SQL Loader - Conditional Data Loading

Hello Friends,
I am trying to use SQL Loader to load the following records in to the Emp table (columns being - empno, ename, job, mgr, hiredate, sal, comm, deptno ).
7876,ADAMS,CLERK,7788,"23-MAY-87",1100,,20
7900,JAMES,CLERK,7698,"03-DEC-81",950,,30
7902,FORD,ANALYST,7566,"03-DEC-81",3000,,20
7934,MILLER,CLERK,7782,"23-JAN-82",1300,,10
I want to put a condition that the data should be loaded only when the 5th field in the record (i.e, the Hiredate field) should be greater than "01-JAN-82". So with this condition, the only records to be loaded are the 1st, and the 4th one... the other two records have the hiredates being less than "01-JAN-82".
load data
infile 'c:\mydata.csv'
into table emp1
when .....
fields terminated by "," optionally enclosed by '"'
( empno, ename, job, mgr, hiredate, sal, comm, deptno )
I am unable to put the when condition in the control file above and would be grateful if you kindly can give me some pointer as to how to achieve this.
Thanks and Regards.
Edited by: user645883 on Dec 29, 2008 12:45 AM

Hello Friends,
Thanks for your replies.
The examples only show the when clauses where we can only use "=" or "<>" in the clause. I am trying to see if we can use ">=" or "<=" in the when clause to load the conditional data. Also trying to find out if date fields can be compared in the when clause. I am unable to do so till now and it throws syntax errors everytime
I am able to test the when with string literals and with "=" and "<>" conditions though.
Please help.
Thanks & Regards.

SQL Loader with date formatting

Hi,
I'm trying to get sql loader to insert a date into a column. After much browsing, reading trial and error I still get an array of errors.
I'm using oracle XE
my control file looks like this
LOAD DATA
INFILE 'posmeters/meters.csv'
INTO TABLE position_meters
FIELDS terminated by ","
ID CONSTANT '0',
POSITION_ID,
DATETIME DATE "DD/MM/YYYY HH24:MI:SS",
CASH_IN,
CASH_OUT,
NOTES_IN,
CHANGE_OUT,
WINNINGS,
VTP,
REFILL,
TOKEN_IN,
TOKEN_OUT,
ELEC_PAY,
ELEC_CREDIT,
REMOTE_PAY,
REMOTE_CREDIT,
INSERT_TS EXPRESSION "TO_CHAR(SYSDATE, 'DD/MM/YYYY HH24:MI:SS')",
FIFTY_PND,
TWENTY_PND,
TEN_PND,
FIVE_PND,
TWO_PND,
ONE_PND,
FIFTY_P,
TWENTY_P,
TEN_P,
FIVE_P
It is the DATETIME field which gives me grief. I have a test data file that looks like this
0,1010,29/09/2011 10:23:24,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24
and my table is defined as follows
ID NUMBER NOT NULL,
POSITION_ID NUMBER,
DATETIME TIMESTAMP(6) DEFAULT localTIMESTAMP NOT NULL,
CASH_IN NUMBER,
CASH_OUT NUMBER,
NOTES_IN NUMBER,
CHANGE_OUT NUMBER,
WINNINGS NUMBER,
VTP NUMBER,
REFILL NUMBER,
TOKEN_IN NUMBER DEFAULT (0) NOT NULL,
TOKEN_OUT NUMBER DEFAULT (0) NOT NULL,
ELEC_PAY NUMBER DEFAULT (0) NOT NULL,
ELEC_CREDIT NUMBER DEFAULT (0) NOT NULL,
REMOTE_PAY NUMBER DEFAULT (0) NOT NULL,
REMOTE_CREDIT NUMBER DEFAULT (0) NOT NULL,
INSERT_TS TIMESTAMP(6) DEFAULT (localtimestamp) NOT NULL,
FIFTY_PND NUMBER DEFAULT 0,
TWENTY_PND NUMBER DEFAULT 0,
TEN_PND NUMBER DEFAULT 0,
FIVE_PND NUMBER DEFAULT 0,
TWO_PND NUMBER DEFAULT 0,
ONE_PND NUMBER DEFAULT 0,
FIFTY_P NUMBER DEFAULT 0,
TWENTY_P NUMBER DEFAULT 0,
TEN_P NUMBER DEFAULT 0,
FIVE_P NUMBER DEFAULT 0
I have tried defining the control file with
DATETIME DATE "DD/MM/YYYY HH24:MI:SS",
DATETIME EXPRESSION "TO_DATE(:DATETIME, 'DD/MM/YYYY HH24:MI:SS')",
DATETIME EXPRESSION "TO_TIMESTAMP(:DATETIME, 'DD/MM/YYYY HH24:MI:SS')",
I get errors such as
Record 1: Rejected - Error on table "SITE_MAIN"."POSITION_METERS", column DATETIME.
ORA-01861: literal does not match format string
SQL*Loader-291: Invalid bind variable DATETIME in SQL string for column DATETIME.
any help would greatfully appreciated.

It seems that the problem was caused by the constant at the beginning of the record and had nothing to do with date formats.
My control file now looks like this
LOAD DATA
INFILE 'posmeters/meters.csv'
APPEND INTO TABLE position_meters
FIELDS terminated by ","
POSITION_ID          ,
DATETIME      date "DD/MM/YYYY HH24:MI:SS",
CASH_IN          ,
CASH_OUT          ,
NOTES_IN          ,
CHANGE_OUT          ,
WINNINGS          ,
VTP               ,
REFILL          ,
TOKEN_IN          ,
TOKEN_OUT          ,
ELEC_PAY          ,
ELEC_CREDIT          ,
REMOTE_PAY          ,
REMOTE_CREDIT     ,
INSERT_TS      "TO_TIMESTAMP(SYSDATE, 'DD/MM/YYYY HH24:MI:SS')",
FIFTY_PND          ,
TWENTY_PND          ,
TEN_PND          ,
FIVE_PND          ,
TWO_PND          ,
ONE_PND          ,
FIFTY_P          ,
TWENTY_P          ,
TEN_P          ,
FIVE_P
all is good :o)

SQL Loader, CLOB, delimited fields

Hello.
I have to load using SQL Loader data from csv file into table, which one field is CLOB type.
Here is how ctl file initially looked like:
UNRECOVERABLE
LOAD DATA
INFILE '.\csv_files\TSH_DGRA.csv'
BADFILE '.\bad_files\TSH_DGRA.bad'
DISCARDFILE '.\dsc_files\TSH_DGRA.dsc'
APPEND
INTO TABLE TSH_DGRA
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
ID_OBJ_TSHD,
PR_ZOOM_TSHD,
PR_GRID_TSHD,
PR_ELMGR_TSHD CHAR(4000) OPTIONALLY ENCLOSED BY '<clob>' AND '</clob>',
PR_ALRMGR_TSHD CHAR(4000) OPTIONALLY ENCLOSED BY '<clob>' AND '</clob>'
Problems are fields PR_ELMGR_TSHD and PR_ALRMGR_TSHD (CLOBs in table TSH_DGRA). Until data which should be loaded into CLOB fields are under 4000 characters long, it works fine, but what should I do if I want to load data which are longer than 4000 characters?
If found on Link:[http://download.oracle.com/docs/cd/B14117_01/server.101/b10825/ldr_loading.htm#i1006803] which one sentence said that:
"SQL*Loader defaults to 255 bytes when moving CLOB data, but a value of up to 2 gigabytes can be specified. For a delimited field, if a length is specified, that length is used as a maximum. If no maximum is specified, it defaults to 255 bytes. For a CHAR field that is delimited and is also greater than 255 bytes, you must specify a maximum length. See CHAR for more information about the CHAR datatype."
So, my question is, how to specify "up to 2gb" as text said? I can not use CHAR datatype because it is limited to 4000 characters. And I have to load about 60000 characters. I also can not use technique where all data for every CLOB field are in separate files.

Just specify the maximum expected size:
PR_ELMGR_TSHD CHAR(100000) OPTIONALLY ENCLOSED BY '<clob>' AND '</clob>',
PR_ALRMGR_TSHD CHAR(1000000) OPTIONALLY ENCLOSED BY '<clob>' AND '</clob>'
The CHAR(1000000) will allow SQLLDR to handle up to 1000000 bytes of input text.

SQL Loader: ignore data field

Similar Messages

Maybe you are looking for