SQL*Loader with ZONED EXTERNAL in a delimited file
I have a text file delimited with '|' and numeric data with trailing signs, e.g somedata|12.00-|13.01|42.01-
My table columns are defined as number(9,2) and I defined the datatype in the control file as zoned external. However, I get the dreaded ORA-01722: invalid number error.
Does anyone have any suggestions?
As Bravid says, you can't do that from within SQL*Loader, unless the control files are dynamically generated by a script and the filename included as the default value for that particular column as a hardcoded literal.
It's easier if you use external tables as the table definition contains the "location" which is the list of filenames it is accessing, and the value can be queried as well as being dynamically changed (though if you're dynamically changing it, then you'll programatically know the filename anyway. hint hint!)
;)
Similar Messages
-
Problem using SQL Loader with ODI
Hi,
I am having problems using SQL Loader with ODI. I am trying to fill an oracle table with data from a txt file. At first I had used "File to SQL" LKM, but due to the size of the source txt file (700MB), I decided to use "File to Oracle (SQLLDR)" LKM.
The error that appears in myFile.txt.log is: "SQL*Loader-101: Invalid argument for username/password"
I think that the problem could be in the definition of the data server (Physical architecutre in topology), because I have left blank Host, user and password.
Is this the problem? What host and user should I use? With "File to SQL" works fine living this blank, but takes to much time.
Thanks in advanceI tried to use your code, but I couldn´t make it work (I don´t know Jython). I think the problem could be with the use of quotes
Here is what I wrote:
import os
retVal = os.system(r'sqlldr control=E:\Public\TXTODI\PROFITA2/Profita2Final.txt.ctl log=E:\Public\TXTODI\PROFITA2/Profita2Final.txt.log userid=MYUSER/myPassword @ mySID')
if retVal == 1 or retVal > 2:
raise 'SQLLDR failed. Please check the for details '
And the error message is:
org.apache.bsf.BSFException: exception from Jython:
Traceback (innermost last):
File "<string>", line 5, in ?
SQLLDR failed. Please check the for details
at org.apache.bsf.engines.jython.JythonEngine.exec(JythonEngine.java:146)
at com.sunopsis.dwg.codeinterpretor.k.a(k.java)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.scripting(SnpSessTaskSql.java)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.execScriptingOrders(SnpSessTaskSql.java)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.execScriptingOrders(SnpSessTaskSql.java)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTaskTrt(SnpSessTaskSql.java)
at com.sunopsis.dwg.dbobj.SnpSessTaskSqlC.treatTaskTrt(SnpSessTaskSqlC.java)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java)
at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
at com.sunopsis.dwg.cmd.e.i(e.java)
at com.sunopsis.dwg.cmd.h.y(h.java)
at com.sunopsis.dwg.cmd.e.run(e.java)
at java.lang.Thread.run(Unknown Source) -
Sql Loader with TSN environment variable.
Sql loader command works fine if I have TSN entry in my system,Can I use SQL Loader with in any other way ie by use of some environment variable???
What are the different ways I can use sql ldr ???952189 wrote:
Sql loader command works fine if I have TSN entry in my system,Can I use SQL Loader with in any other way ie by use of some environment variable???
What are the different ways I can use sql ldr ???
>Sql loader command works fine if I have TSN entry in my system,Can I use SQL Loader with in any other way ie by use of some environment variable???
What are the different ways I can use sql ldr ???
if sqlldr is run on DB Server system, then no tnsnames.ora file is required. -
How to use SQL loader with DBF fixed format record
Hi everybody!
My situation is that: I want to use SQL loader with Foxpro DBF format, it similar to case 2 study (Fixed format record) but DBF file has header, how can I tell SQL loader skip header.
Thank you in advanceAnother option is to apply SQL operators to fields
LOAD DATA
INFILE *
APPEND
INTO TABLE emp
FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"' (
empno,
ename,
job,
mgr,
hiredate DATE(20) "DD-Month-YYYY",
sal,
comm,
deptno CHAR TERMINATED BY ':',
projno,
loadseq "my_seq.nextval")This is a modified control file of Case Study 3 which originally demonstrated the use of the Sequence parameter -
SQL Loader and foreign characters in the data file problem
Hello,
I have run into an issue which I can't find an answer for. When I run SQL Loader, one of my control files is used to get file content (LOBFILE) and one of the fields in the data file has a path to that file. The control file looks like:
LOAD DATA
INFILE 'PLACE_HOLDER.dat'
INTO TABLE iceberg.rpt_document_core APPEND
FIELDS TERMINATED BY ','
doc_core_id "iceberg.seq_rpt_document_core.nextval",
-- created_date POSITION(1) date "yyyy-mm-dd:hh24:mi:ss",
created_date date "yyyy-mm-dd:hh24:mi:ss",
document_size,
hash,
body_format,
is_generic_doc,
is_legacy_doc,
external_filename FILLER char(275) ENCLOSED by '"',
body LOBFILE(external_filename) terminated by EOF
A sample data file looks like:
0,2012-10-22:10:09:35,21,BB51344DD2127002118E286A197ECD4A,text,N,N,"E:\tmp\misc_files\index_testers\foreign\شیمیایی.txt"
0,2012-10-22:10:09:35,17,CF85BE76B1E20704180534E19D363CF8,text,N,N,"E:\tmp\misc_files\index_testers\foreign\ลอบวางระเบิด.txt"
0,2012-10-22:10:09:35,23552,47DB382558D69F170227AA18179FD0F0,binary,N,N,"E:\tmp\misc_files\index_testers\foreign\leesburgis_á_ñ_é_í_ó_ú_¿_¡_ü_99.doc"
0,2012-10-22:10:09:35,17,83FCA0377445B60CE422DE8994900A79,binary,N,N,"E:\tmp\misc_files\index_testers\foreign\làm thế nào bạn làm ngày hôm nay"
The problem is that whan I run this, SQL Loader throws an error that it can't find the file. It appears that it can't interpret the foreign characters in a way that allows it to find that path. I have tried adding a CHARACTERSET (using AL32UTF8 or UTF8) value in the control file but that only has some success with Western languages, not the ones listed above. Also, there is no set of defined languages that could be found in the data file. It essentaially could be any language.
Does anyone know if there is a way to somehow get SQL Loader to "understand" the file system paths when a folder and/or file name could be in some other langauge?
Thanks for any thoughts - PeterThanks for the reply Harry. If I try to open the file in various text editors like Wordpad, Notepad, GVIM, andTextpad, they all display the foreign characters differently. Only Notepad comes close to displaying the characters properly. I have a C# app that will read the file and display the contents and it renders it fine. If you look at the directory of files in Windows Explorer, they all are displayed properly. So it seems things like .Net and Windows have some mechanism to understand the characters in order to render them properly. Other applications, again like Wordpad, do not know how to render them properly. It would seem that whatever SQL Loader is using to "read" the data files also is not rendering the characters properly which prevents it from finding the directory path to the file. If I add "CHARACTERSET AL32UTF8" in the control file, all is fine when dealing with Western langauges (ex, German, Spanish) but not for the Eastern languages (ex. Thai, Chinese). So .... telling SQL Loader to use a characterset seems to work, but not in all cases. The AL32UTF8 is the characterset that the Oracle database was created with. I have not had any luck if I try to set the CHARACTERSET to whatever the Thai character set is, for example. There problem there though is that even if that did work, I can't target specific lagauages because the data could come from anywhere. It's like I need some sort of global "super set" characterset to use. It seems like the CHARACTERSET is the right track to follow but I am not sure, and even if it is, is there a way to handle all languages.
Thanks - Peter -
SQL Loader with decimal commas
Dear All,
I am loading data for a German client and the delivered files contain numeric data with decimal commas. For example 123,5 instead of 123.5
I've been googling like mad on this one and there is a lot of chatter about setting NLS_LANG and/or NLS_NUMERIC_CHARACTERS but none of this seems to work in the SQL Loader environment.
I imagine that I will end up cobbling together a complicated thing with TO_NUMBER or with another function but I really can't believe that SQL Loader doesn't have a simple switch to flip from point to comma.
Or have I overlooked something?
Any tips would be a great help.
Regards,
Alan SearleI am on a client's PC and don't have access to administrative settings.
And also, I cannot be sure that whoever runs the load function in the future will check this.
It is therefore important that I find a way to set this in SQL Loader without any other dependencies.
I think I will probably end up processing it as a string and then translating it to numeric. But this seems like a sledge hammer to crack a nut.
Regards and thanks,
Alan -
SQL*Loader with multiple files
Gurus,
I search the documentation and this forum and haven't found a solution to my issue yet...
I am not expert of SQL*Loader. I have used SQL*Loader to copy from one file to a table many times. But I have not copied multiple files into one table especially with different names.
More specifically....
I need to load data from multiple files into a table. But the file names will be different each time. A file will be created every hour. The file name will consist of the root file name appended by a time stamp. For example, a file created on 10/07/2010 at 2:15 P.M. would be filea100720101415.txt while a file created on 10/08/2010 at 8:15 A.M. would be filea100820100815.txt. All the files will be in one directory.How can I load the data from the files using SQL*Loader?
My database: Oracle 10g Release 2
Operating System: Windows 2003 Server
Please assist.
Robertsect55 wrote:
Gurus,
I search the documentation and this forum and haven't found a solution to my issue yet...
I am not expert of SQL*Loader. I have used SQL*Loader to copy from one file to a table many times. But I have not copied multiple files into one table especially with different names.
More specifically....
I need to load data from multiple files into a table. But the file names will be different each time. A file will be created every hour. The file name will consist of the root file name appended by a time stamp. For example, a file created on 10/07/2010 at 2:15 P.M. would be filea100720101415.txt while a file created on 10/08/2010 at 8:15 A.M. would be filea100820100815.txt. All the files will be in one directory.How can I load the data from the files using SQL*Loader?
My database: Oracle 10g Release 2
Operating System: Windows 2003 Server
Please assist.
RobertToo bad this isn't in *nix, where you get a powerful shell scripting capability.
That said, here is the core of the solution .... you will also need a way to identify files that have been processed vs. new ones. Maybe rename them, maybe move them. But with this sample you can see the basics. From there it is really an issue of DOS scripting, which would better be found by googling around a bit.
cd c:\loadfiles
FOR %%datfile IN (*.txt) DO SQLLDR CONTROL=sample.ctl, LOG=sample.log, BAD=baz.bad, DATA=%%datfileTry googling "dos scripting language". You'll find lots of tutorials and ideas on "advanced" (well, as advanced as DOS gets) techniques to solve your problem.
Edited by: EdStevens on Dec 1, 2010 5:03 PM -
Hello,
I have got some problemes with sql loader syntax.
I'm trying to import some data into my table. And I want that the sysdate is import in my table automatically.
This is my char.ctl file :
LOAD DATA
INFILE *
INTO TABLE CHANGE_REQUEST
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(CHAR_ID,DATE_CREATION SYSDATE,STATUS,ABSTRACT,USER_ID_FK,SYSTEM_ID_FK,PRIORITY)
BEGINDATA
1,PENDING,Need to have the firsname when listing users,1,Medium
But I still get the error : ORA-00947: not enough values
I have try different line like :
1, ,PENDING,Need to have the firsname when listing users,1,Medium
or
1,SYSDATE,PENDING,Need to have the firsname when listing users,1,Medium
But nothing is working.
Where is the mistake?
Regards.Another way, wich does not require an alter table is something like this:
LOAD DATA
INFILE *
INTO TABLE CHANGE_REQUEST
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
(CHAR_ID,
stATUS,
ABSTRACT,
uSER_ID_FK,SYSTEM_ID_FK,PRIORITY,
DATE_CREATION "sysdate"
BEGINDATA
1,PENDING,Print issue,1,1,High
I putted DATE_CREATION in last position only for example.
Hope this can help someone.
Bye bye.
Antonio -
SQL Loader with date formatting
Hi,
I'm trying to get sql loader to insert a date into a column. After much browsing, reading trial and error I still get an array of errors.
I'm using oracle XE
my control file looks like this
LOAD DATA
INFILE 'posmeters/meters.csv'
INTO TABLE position_meters
FIELDS terminated by ","
ID CONSTANT '0',
POSITION_ID,
DATETIME DATE "DD/MM/YYYY HH24:MI:SS",
CASH_IN,
CASH_OUT,
NOTES_IN,
CHANGE_OUT,
WINNINGS,
VTP,
REFILL,
TOKEN_IN,
TOKEN_OUT,
ELEC_PAY,
ELEC_CREDIT,
REMOTE_PAY,
REMOTE_CREDIT,
INSERT_TS EXPRESSION "TO_CHAR(SYSDATE, 'DD/MM/YYYY HH24:MI:SS')",
FIFTY_PND,
TWENTY_PND,
TEN_PND,
FIVE_PND,
TWO_PND,
ONE_PND,
FIFTY_P,
TWENTY_P,
TEN_P,
FIVE_P
It is the DATETIME field which gives me grief. I have a test data file that looks like this
0,1010,29/09/2011 10:23:24,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24
and my table is defined as follows
ID NUMBER NOT NULL,
POSITION_ID NUMBER,
DATETIME TIMESTAMP(6) DEFAULT localTIMESTAMP NOT NULL,
CASH_IN NUMBER,
CASH_OUT NUMBER,
NOTES_IN NUMBER,
CHANGE_OUT NUMBER,
WINNINGS NUMBER,
VTP NUMBER,
REFILL NUMBER,
TOKEN_IN NUMBER DEFAULT (0) NOT NULL,
TOKEN_OUT NUMBER DEFAULT (0) NOT NULL,
ELEC_PAY NUMBER DEFAULT (0) NOT NULL,
ELEC_CREDIT NUMBER DEFAULT (0) NOT NULL,
REMOTE_PAY NUMBER DEFAULT (0) NOT NULL,
REMOTE_CREDIT NUMBER DEFAULT (0) NOT NULL,
INSERT_TS TIMESTAMP(6) DEFAULT (localtimestamp) NOT NULL,
FIFTY_PND NUMBER DEFAULT 0,
TWENTY_PND NUMBER DEFAULT 0,
TEN_PND NUMBER DEFAULT 0,
FIVE_PND NUMBER DEFAULT 0,
TWO_PND NUMBER DEFAULT 0,
ONE_PND NUMBER DEFAULT 0,
FIFTY_P NUMBER DEFAULT 0,
TWENTY_P NUMBER DEFAULT 0,
TEN_P NUMBER DEFAULT 0,
FIVE_P NUMBER DEFAULT 0
I have tried defining the control file with
DATETIME DATE "DD/MM/YYYY HH24:MI:SS",
DATETIME EXPRESSION "TO_DATE(:DATETIME, 'DD/MM/YYYY HH24:MI:SS')",
DATETIME EXPRESSION "TO_TIMESTAMP(:DATETIME, 'DD/MM/YYYY HH24:MI:SS')",
I get errors such as
Record 1: Rejected - Error on table "SITE_MAIN"."POSITION_METERS", column DATETIME.
ORA-01861: literal does not match format string
SQL*Loader-291: Invalid bind variable DATETIME in SQL string for column DATETIME.
any help would greatfully appreciated.It seems that the problem was caused by the constant at the beginning of the record and had nothing to do with date formats.
My control file now looks like this
LOAD DATA
INFILE 'posmeters/meters.csv'
APPEND INTO TABLE position_meters
FIELDS terminated by ","
POSITION_ID ,
DATETIME date "DD/MM/YYYY HH24:MI:SS",
CASH_IN ,
CASH_OUT ,
NOTES_IN ,
CHANGE_OUT ,
WINNINGS ,
VTP ,
REFILL ,
TOKEN_IN ,
TOKEN_OUT ,
ELEC_PAY ,
ELEC_CREDIT ,
REMOTE_PAY ,
REMOTE_CREDIT ,
INSERT_TS "TO_TIMESTAMP(SYSDATE, 'DD/MM/YYYY HH24:MI:SS')",
FIFTY_PND ,
TWENTY_PND ,
TEN_PND ,
FIVE_PND ,
TWO_PND ,
ONE_PND ,
FIFTY_P ,
TWENTY_P ,
TEN_P ,
FIVE_P
all is good :o) -
Problems using SQL*Loader with Oracle SQL Developer
I have been using TOAD and able to import large (milllions of rows of data) in various file formats into a table in an Oracle database. My company recently decided not to renew any more TOAD licenses and go with Oracle SQL Developer. The Oracle database is on a corporate server and I access the database via Oracle client locally on my machine. Oracle SQL Developer and TOAD are local on my desktop and connected through TNSnames using the Windows XP platform. I have no issues with using SQL*Loader via the import wizard in TOAD to import the data in these large files into an Oracle table and producing a log file. Loading the same files via SQL*Loader in SQL Developer, freezes up my machine and I cannot get it to produce a log file. Please help!
I am using SQL Developer version 3.0.04. Yes, I have tried it with a smaller file with no success. What is odd is that the log file is not even created. What is created is a .bat file a control file and a .sh file but no log file. The steps that I take:
1.Right click on the table I want to import to or go to actions
2. Import Data
3. Find file to import
4. Data Preview - All fields entered according to file
5. Import Method - SQL Loader utility
6. Column Definitions - Mapped
7. Options - Directory of files set
8. Finish
With the above steps I was not able to import 255 rows of data. No log file was produced so I don't know why it is failing.
thanks.
Edited by: user3261987 on Apr 16, 2012 1:23 PM -
How to Import data via SQL Loader with characterset UTF16 little endian?
Hello,
I'm importing data from text file into one of my table which contains blob column.
I've specified following in my control file.
-----Control file-------
LOAD DATA
CHARACTERSET UTF16
BYTEORDER LITTLE
INFILE './DataFiles/Data.txt'
BADFILE './Logs/Data.bad'
INTO TABLE temp_blob truncate
FIELDS TERMINATED BY " "
TRAILING NULLCOLS
(GROUP_BLOB,CODE)
Problem:
SQL Loader always importing data via big endian. Is there any method available using which we can convert these data to little endian?
ThanksA new preference has been added to customize the import delimiter in main code line. This should be available as part of future release.
-
Good morning,
I have a *.CVS file containing data like this:
A123456789,Ah Tong Station,Jalan Dungun
I would like insert the data into a table that returns 1 row for each value
id outlet_name addr_1A123456789 Ah Tong Station Jalan Dungun
etc
In the stored procedure, SQL loader, I used
Insert into XXXXX values
(SUBSTR(input_buffer, 1, 10),
SUBSTR(input_buffer, 11, 60),
SUBSTR(input_buffer, 61, 110);
but the problem is that insertion of the outlet_name and addr_1 will not follow the subcripts that provided. Instead in gave an output like that:
id outlet_name addr_1
A123456789 ,Ah Tong Station,Jalan Dungun NULL
Hope to get some advice asap. Thanks. :)
PaulineWhy not use the SQL*Loader proper to load this in to your table? It will be significantly faster than anything you could do in PL/SQL, especially if your volumes increase. And it will take care of the parsing for you, as long as you tell it the field delimiter. You've already seen that you can't rely on fixed widths when using delimited data.
Given this data...
A123456789,Ah Tong Station,Jalan Dungun
A234,Some Station,Some Place...the following SQL*Loader control file ( stored in a file called mytable.ctl ) will load your table ( note I've used truncate - you could also append )...
load data
infile 'mydata.csv'
truncate
into table mytable
fields terminated by ','
( id
, outlet_name
, addr_1
)This is invoked using the following:-
sqlldr userid/password@tns control=mytable.ctlThe results are:-
SQL> select * from mytable;
ID OUTLET_NAME ADDR_1
A123456789 Ah Tong Station Jalan Dungun
A234 Some Station Some PlaceYou will find this much easier. Of course, you could always avoid the database altogether if you don't need to actually store this data. You could just parse the source .csv ( if you are on UNIX this will be really easy using an awk one-liner ) and write it to an output file.
Anyway, hope this helps.
Regards
Adrian -
Oracle 11G Direct SQL Load with Data Guard
Does SQL Loader in direct mode always bypass the writing of redo logs ?
If the database has force logging on, will SQL Loader in direct mode bypass the writing of redo logs ?
Is there a way to run SQL Loader in direct mode that will create redo logs that can be applied by Data Guard to the backup database ?846797 wrote:
Does SQL Loader in direct mode always bypass the writing of redo logs ?
If the database has force logging on, will SQL Loader in direct mode bypass the writing of redo logs ?
Is there a way to run SQL Loader in direct mode that will create redo logs that can be applied by Data Guard to the backup database ?In case of data guard setup , redo logs will always be generated. -
SQL * Loader with Spanish data!
Hi All,
I have a requirement to load Spanish data into a table. I am using SQL*Loader control file to load the data.
Everything is fine, but the Spanish charecters are not inserting as they have to. It is inserting some junk charecters.
How can I solve this issue?
Thanks in advance.Hello,
Are you using init.ora (pfile) or spfile? Anyway you can do this
sqlplus "/as sysdba"
sql> create pfile='/location_to_pfile/init.ora' from spfile;
File Created
Modify init.ora file and add parameter to support spanish characters
sql>shutdown immediate;
sql>startup pfile='/path_to_pfile/init.ora'; -- YOu should be able mount and open and successfully
Test your data using sqlldr and if it is doing what you are expected it do then.
sql>shutdown immediate;
sql>create spfile from pfile='/path_to_pfile/init.ora';
File Created
sql>startup;
sql>show parameter you_parameter_nameor you can try this to
sqlplus "/as sysdba"
sql> alter system set parameter_name=value scope=both sid='*';
or
sql>alter system set paramter_name=value sid='*';
sql> show parameter parameter_name;
Test your data using sqlldr Regards -
Hi All,
I have 20 text files which have large amount of data. I need to load these data into oracle db. Each of these text files having header as column name which are same in order. Now my manager ask me to load these data in database like that last column should be the file name, so that he can track from which text file data is from. Can anybody help me to do this using sql loader.
Example:
temp1.txt temp2.txt
col1 col2 col1 col2
118 'B' 100 'J'
245 'D' 110 'K'
160 'A' 200 'N'
Now i need to load data into a database table from these text file as-------
Table Name: Temp
col1 col2 FileName
118 'B' temp1
245 'D' temp1
160 'A' temp1
100 'J' temp2
110 'K' temp2
200 'N' temp2you can load from multiple input files provided that the files use the same record format by repeating the INFILE clause. Here is an example:
load data
infile file1.dat
infile file2.dat
infile file3.dat
fields terminated by "," optionally enclosed by '"' trailing nullcols
( column1,
column2 )
Maybe you are looking for
-
How to perform multiple selection from dropdown list
Hi, I have developed one application in which i have taken one dropdownbyindex and entered some values in it, now i need to have multiple selection of entries, can any one please suggest how to do these Regards, Prasanna
-
All the imported images are gone
I've made an animation in Adope Edge Animate with a lot of images imported from the same directory folder as the animation. But the next time i had started up the animation, all the images are gone. The animation is saying they are still there, like
-
IPOD CANNOT BE UPDATED ON PC --- HELP!!!
I've got a problem when I try to transfer new songs from itunes onto my ipod shuffle on my pc. The whole thing seems to be working ok, until I get a message saying "Ipod cannot be updated--an unknown error occurred (-36)". What should I do? I've down
-
i wan to change my mouse coordinate to x->0, y ->0, possible?
-
Multiple session handling in the Web UI
Hi all, According to the SAP help documentation, a user can have multiple sessions open in the Web UI. However after a certain period, inactive sessions are terminated. This is certainly the behaviour that we witness on our system, the second inactiv