SQL*Loader v/s PERL

Hi All,
Anyone used PERL Script to load data into Oracle (replace SQL*Loader). If so, pl post your samples.
Thanks in advance
Regards
r@m@

Unless you need to do some complex manipulation of data between the text file and the Oracle table, I would use SQLLoader every time.
Based on somewhat limited experience with utl_file, I would guess that SQLLoader would be 3 to 4 times faster run normally, and probavly 6 to 8 times faster running in direct load mode.

Similar Messages

SQL LOADER / INFILE filename as variable in .ctl file

I stumbled over several threads in the OTN-forums regarding this problem, but neither was it finaly solved nor did I find a FAQ that answered my question. Soooo:
We get several datafiles from several sources and process them via SQL Loader and store 'em in the DB.
This is done via a CHRON job and a PERL skript, for all datafiles in a specific directory.
We need the information which file on which date generated the data INSIDE the DB as well.
So I want to store the filename || SYSDATE combination as well.
I know, I could parse the .ctl file and replace a key-string with the actual filename and so have it in the input.
But this seems a bit dirty to me. Isn't there some way, i.e. a keyword or variable for the infile-filename within the SQLLoader that I can access in the .ctl file? Something like:
INTO TABLE processed_files
FIELDS TERMINATED BY ';'
WHEN LOWER(LTRIM(RTRIM(hdr_ftr))) = 'ftr' -- FOOTER??
(hdr_ftr VARCHAR2(100),
source INFILE||' on '||TO_CHAR(SYSDATE, 'MM/DD/YYYY'),
realm VARCHAR2(100),
version VARCHAR2(20)
I would be greatfull if you?d share your wisdom with me. ;-))
Oliver

I passed this quite similar to 'Ask Tom' and got the advice to put the .ctl's content as a string variable into a Shell skript.
This shell skript (which had to be written anyway to loop over the datafiles an subsequently call the sqlldr) should then replace the INFILE parameter and the CONSTANTs for the filenames and generate a 'temporarry' .ctl before calling sqlldr!
That's it, no better and safer way!

SQL-Loader and Pipes

I have to insert 300.000.000 datasets (generated dynamically by a seperate process) into an Oracle 8.1.6 database. I'd like to do this via SQL-Loader but since Linux (kernel 2.2 at least) supports only files up to 2Gigs and I don't have the necessary space on my machine to generate a large amount of ASCII files I'd like to ask whether there is a possibility to use an ordinary pipe in combination with SQL-Loader. I tried to use a named pipe but all my attempts failed.
I don't like to use Perl and the DBI module since this would be far so slow (would take about one week I guess).
Hans

External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
Justin

Define variable in SQL Loader Control File

Hi,
I have an input file where the first line is the header record, followed by the detail records. For the processing, I do not need to store the fields of the header record but I need a date field from this header record and store in as part of the detail record in an oracle record row.
Is it possible to define a variable within the sql loader control file to store the value that I need in memory then use it when I do the sql insert by the sql loader?
Thanks for any advice.

Not sure that you can. But if your on unix/linux/mac its easy enough to write a shell script to populates the variables in a template file that you can then use as the ctl file. The perl template toolkit could be an option for that as well

SQL*Loader-704: error..OCIServerAttach...ORA-12535 Oracle 9.2 ..Solaris 9

Hello.
We run an application with an Oracle backend. Everything seems to work. There arer some Perl scripts we run to update the database. One does not work. We get.
SQL*Loader-704: Internal error: ulconnect: OCIServerAttach [0]
ORA-12535: TNS:operation timed out
We are using Solaris 9
Oracle 9.2i
Any help would be appreciated.
Thanks
John

Have you checked the sqlnet.log file for an indication of the underlying error? You can also enable network tracing in the sqlnet.ora file, by setting trace_level_client=16.

Using SQL Loader in more than one table

Hi all,
I have a new question for those who have used SQL Loader. I
have never used it and I just know that I need a control file to
tell SQL Loader what is my flat file layout and what table the
information goes to. My problem is: my flat file has information
that goes to two tables in my schema. Those files are very big
(aprox 280Mb) and I would like to read them just once. Can I do
this with SQL Loader?
Other question, is that the fastest way to import data from
flat files becouse I am using PERL and it takes aprox. 9 hours
to import 10 of those files. I could use UTL_FILE to read it but
I heard that SQL Loader was better.
Thanks for your cooperation
(Robocop)
Marcelo Lopes
Rio de Janeiro - Brazil

SQL*Loader is the fastest way to load, particularly in direct parallel mode, and can certainly load to multiple tables.
>
My advice would be to have a look at the examples given in the Oracle Utilities guide, there is one for loading to multiple
tables, which I have pasted below.
>
-- Loads EMP records from first 23 characters
-- Creates and loads PROJ records for each PROJNO listed
-- for each employee
LOAD DATA
INFILE ’ulcase5.dat’
BADFILE ’ulcase5.bad’
DISCARDFILE ’ulcase5.dsc’
REPLACE
INTO TABLE emp
(empno POSITION(1:4) INTEGER EXTERNAL,
ename POSITION(6:15) CHAR,
deptno POSITION(17:18) CHAR,
mgr POSITION(20:23) INTEGER EXTERNAL)
INTO TABLE proj
-- PROJ has two columns, both not null: EMPNO and PROJNO
WHEN projno != ’ ’
(empno POSITION(1:4) INTEGER EXTERNAL,
projno POSITION(25:27) INTEGER EXTERNAL) -- 1st proj
INTO TABLE proj
WHEN projno != ’ ’
(empno POSITION(1:4) INTEGER EXTERNAL,
projno POSITION(29:31 INTEGER EXTERNAL) -- 2nd proj
INTO TABLE proj
WHEN projno != ’ ’
(empno POSITION(1:4) INTEGER EXTERNAL,
projno POSITION(33:35) INTEGER EXTERNAL) -- 3rd proj
see the documentation for a complete explanation of the configuration.
Thanks, I will read it.

SQL Loader from Forms 10

Hello!
How can i call to SQL Loader from my application developed by Forms 10?

My application is doing something similar.
Given that the iAS server can read from the data source (where the flat file for the sqlldr to load), all you need is to write a shell script that is invoked from within the forms application (using the host command). The shell script (I use Perl) will do invoking the sqlldr and also other checks, etc
hope this helps

Pre parse terrible SQL Loader data file

Long story, but I'm on Oracle EBS 11.5.10.2 which means i'm stuck with an SQL Loader from the dark ages (8.06 - yes, I know and yes, it's still supported under EBS).
Anyhow. I have a vendor sending very annoying data. For example:
\|"Fan-Hou" dessert|
\|"Donald is cool | yes”|
\|"Fred “is” cool | yes”|
Due to the limitations of such an old SQL Loader, I need to end up with:
\|"Fan-Hou dessert"|
\|"Donald is cool | yes”|
\|"Fred “is” cool | yes”|
So you can see that really it's only the first row which is causing the issue, I need to shift the quote out to the end of the string (just before the |). The other two lines are just for reference, to show that I get data like that too, and I need to be able to keep that as is.
I have a rather un-elegant solution solution using perl but I'm sure someone out there knows a clever way to achieve the same result. Before you ask - no, the vendor won't change the data.
Edited by: user13007502 on 07-May-2013 01:35

There are probably various ways you can do that. I would try the SED command.
The idea is the following:
1. Replace all double quotes of words that start and end with a space with a special character.
2. Remove all all double quotes.
3. Add double quotes to the beginning and end.
4. Reconstruct double quotes of words according to 1.
For instance:
$ cat testfile
|"Fan-Hou" dessert|
|"Donald is cool | yes"|
|"Fred "is" cool | yes"|
Replace double quotes with {} for words that start and end with a space:
sed -i 's: "$.*$" : {\1} :g' testfile
$ cat testfile
|"Fan-Hou" dessert|
|"Donald is cool | yes"|
|"Fred {is} cool | yes"|
Remove all double quotes
$ sed -i 's:"::g' testfile
$ cat testfile
|Fan-Hou dessert|
|Donald is cool | yes|
|Fred {is} cool | yes|
Replace all lines that begin and end with | with |" and "|.
$ sed -i 's:^|:|":g' testfile
$ sed -i 's:$.*$|:\1"|:g' testfile
$ cat testfile
|"Fan-Hou dessert"|
|"Donald is cool | yes"|
|"Fred {is} cool | yes"|
Finally replace { and } with double qoutes:
sed -i 's:{:":g' testfile
sed -i 's:}:":g' testfile
Here is the result:
# cat testfile
|"Fan-Hou dessert"|
|"Donald is cool | yes"|
|"Fred "is" cool | yes"|
{code}

Need help:Scripting on windows and Sql Loader

Hi
I need some help regarding script work on windows machines i.e., how to write a script,the file extension,how to run the script etc,how to schedule it...
I also need the doc regarding sql loader(complete details.....)
Thanks in advance...
bye

The sqlldr utility is fully documented in the Oracle version# Utilities manual for your version of Oracle. You can find the documentation sets here on OTN for download or you can use the online versions at http://tahiti.oracle.com
There is a product called Script Works and it is documented by the vendor. Otherwise the details depend on the scripting language/tool you use: DOS, Perl, etc....
HTH -- Mark D Powell --

SQL*Loader-524 using sqlldr

I am getting the following error while running sqlldr80 on 8.0.6.3.1 version.
SQL*Loader-524: partial record found at end of datafile
I followed metalink note 1058890.6 that tells about the bug 735710.
The problem gets resolved when I give a line feed end of datafile.
My problem is the whole system has to be automated where I cannot manually provide a new line in datafile that is automatically placed in a directory from where sqlldr reads it.
Please advice.

You can simply process the file by sed on Unix (on Windows you can implement cygwin to add Unix functionality) or by Perl (works on both Windows and Unix) to add that extra linefeed automatically.
Which obviously is a workaround. You would better upgrade from Oracle 8.0.6, which was desupported many, many years ago.
Sybrand Bakker
Senior Oracle DBA

SQL*Loader: Skipping input files fields

There were several postings here addressing an issue of skipping fields from the input file when using SQL*Loader. Most suggestions were to use FILLER fields.
Is there any other way? My input file (over which I have no control) has literally hundreds of fields, most of them blanks. To write a control file with this many dummy fields will be difficult (I can write a perl script to do it, I know, I know...).
Thanks for any suggestions.

Hi, I think in your case the best tool for you use is pl/sql. Cause have function called Utl_file, there you have more control to do this type of load, and you can combine another functions.
Paulo Sergio

How to load a default value in to a column when using sql loader

Im trying to load from a flat file using sql loader.
for 1 column i need to update using a default value
how to go about this?

Hi!
try this code --
LOAD DATA
   INFILE 'sample.dat'
   REPLACE
   INTO TABLE emp
   empno   POSITION(01:04) INTEGER EXTERNAL NULLIF empno=BLANKS,
   ename   POSITION(06:15) CHAR,
   job         POSITION(17:25) CHAR,
   mgr       POSITION(27:30) INTEGER EXTERNAL NULLIF mgr=BLANKS,
   sal        POSITION(32:39) DECIMAL EXTERNAL NULLIF sal=BLANKS,
   comm   POSITION(41:48) DECIMAL EXTERNAL DEFAULTIF comm = 100,
   deptno POSITION(50:51) INTEGER EXTERNAL NULLIF deptno=BLANKS,
   hiredate POSITION(52:62) CONSTANT SYSDATE
)-hope this will solve ur purpose.
Regards.
Satyaki De.

How can we tell if SQL*Loader is working on a TABLE?

We have a process that requires comparing batches with LDAP information. Instead of using an LDAP lookup tool, we get a nightly directory file, and import the two COLUMNs we want via SQL*Loader (REPLACE) into an IOT. Out of three cases, two just check the first COLUMN, and the third needs the second COLUMN as well.
We did not think of using External TABLEs, because we cannot store files on the DB server itself.
The question arises, what to do while the file is being imported. The file is just under 300M, so it takes a minute or so to replace all the data. We found SQL*Loader waits until a transaction is finished before starting, but a query against the TABLE only waits while it is actually importing the data. At the beginning of SQL*Loader's process, however, a query against the TABLE returns no rows.
The solution we are trying right now is, to have the process that starts SQL*Loader flip a flag in another TABLE denoting that it is unavailable. When it is done, it flips it back, and notes the date. Then, the process that queries the information, exits if the flag is currently 'N'.
The problem, is, what if SQL*Loader starts inbetween the check of the flag, and the query against the TABLE. How do we guarantee that it is still not being imported.
I can think of three solutions:
1) LOCK the ldap information TABLE before checking the flag.
2) LOCK the record that the process starting SQL*Loader flips.
3) Add a clause to the query against the TABLE checks that there are records in the TABLE (AND EXISTS(SELECT * FROM ldap_information).
The problem with 3) is that the process has already tagged the batches (via a COLUMN). It could, technically reset them afterwards, but that seems a bit backwards.

Just out of curiosity, are you aware that Oracle supplies a DBMS_LDAP package for pulling information from LDAP sources? It would obviously be relatively easy to have a single transaction that deletes the existing data, loads the new data via DBMS_LDAP, and commits, which would get around the problem you're having with SQL*Loader truncating the table.
You could also have SQL*Loader load the data into a staging table and then have a second process either MERGE the changes from the staging table into the real table (again in a transactionally consistent manner) or just delete and insert the data.
Justin

Loading two tables at same time with SQL Loader

I have two tables I would like to populate from a file C:\my_data_file.txt.
Many of the columns I am loading into both tables but there are a handful of columns I do not want. The first column I do not want for either table. My problem is how I can direct SQL Loader to go back to the first column and skip over it. I had tried using POSITION(1) and FILLER for the first column while loading the second table but I got THE following error message:
SQL*Loader-350: Syntax error at line 65
Expecting "," or ")" found keyword Filler
col_a Poistion(1) FILLER INTEGER EXTERNALMy control file looks like the following:
LOAD DATA
INFILE 'C:\my_data_file.txt'
BADFILE 'C:\my_data_file.txt'
DISCARDFILE 'C:\my_data_file.txt'
TRUNCATE INTO TABLE table_one
WHEN (specific conditions)
FIELDS TERMINATED BY ' '
TRAILING NULLCOLS
col_a FILLER INTEGER EXTERNAL,
col_b INTEGER EXTERNAL,
col_g FILLER CHAR,
col_h CHAR,
col_date DATE "yyyy-mm-dd"
INTO TABLE table_two
WHEN (specific conditions)
FIELDS TERMINATED BY ' '
TRAILING NULLCOLS
col_a POSITION(1) FILLER INTEGER EXTERNAL,
col_b INTEGER EXTERNAL,
col_g FILLER CHAR,
col_h CHAR,
col_date DATE "yyyy-mm-dd"
)

Try adapting this for your scenario.
tables for the test
create table test1 ( fld1 varchar2(20), fld2 integer, fld3 varchar2(20) );
create table test2 ( fld1 varchar2(20), fld2 integer, fld3 varchar2(20) );
control file
LOAD DATA
INFILE "test.txt"
INTO TABLE user.test1 TRUNCATE
WHEN RECID = '1'
FIELDS TERMINATED BY ' '
recid filler integer external,
fld1 char,
fld2 integer external,
fld3 char
INTO TABLE user.test2 TRUNCATE
WHEN RECID <> '1'
FIELDS TERMINATED BY ' '
recid filler position(1) integer external,
fld1 char,
fld2 integer external,
fld3 char
data for loading [text.txt]
1 AAAAA 11111 IIIII
2 BBBBB 22222 JJJJJ
1 CCCCC 33333 KKKKK
2 DDDDD 44444 LLLLL
1 EEEEE 55555 MMMMM
2 FFFFF 66666 NNNNN
1 GGGGG 77777 OOOOO
2 HHHHH 88888 PPPPP
HTH
RK

Creating SQL-Loader script for more than one table at a time

Hi,
I am using OMWB 2.0.2.0.0 with Oracle 8.1.7 and Sybase 11.9.
It looks like I can create SQL-Loader scripts for all the tables
or for one table at a time. If I want to create SQL-Loader
scripts for 5-6 tables, I have to either create script for all
the tables and then delete the unwanted tables or create the
scripts for one table at a time and then merge them.
Is there a simple way to create migration scripts for more than
one but not all tables at a time?
Thanks,
Prashant Rane

No there is no multi-select for creating SQL-Loader scripts.
You can either create them separately or create them all and
then discard the one you do not need.

SQL*Loader v/s PERL

Similar Messages

Maybe you are looking for