2 sql loader questions

Hello,
I am going to be performing a data load with sql loader on approx 60 tables. For each table I have one csv and one ctl file. I'd like to run a file that will batch all the ctl files into one... is this possible? So that the person who has to load all this data into dev, test and prod doesn't have to go through the tedious process of running all the ctl files independantly.
My second question is about the log file. Can I specify in the ctl file where the log file will go? I'm specifying my infile and badfile, but I can't find the syntax to specify the log.
TIA,
Janice

Hi Janice,
Following is for specifing log file
$ORACLE_HOME/bin/sqlload userid=$ORACLE_PW control=$LDRCTL data=$complete_file_critical_parts discard=$LDRDSC bad=$LDRBAD log=$LDRLOG errors=10000000 skip=1
Prashant

Similar Messages

Sql*loader question

Hi Experts,
In a flat file 50000 records are there.
In that one record is having problem so that from that record
the remaing records are not loading to oracle tables.
Could you please help me to load the remaing records,
after the record which is having problem.
The data is loading using sql*loader
Please help me .
Thanks in advance.

It looks ok!
But it will only work when table HR_DTLS is empty.
If you want to append rows in a non empty table you should change it to
LOAD DATA
INFILE 'c:\test.txt'
APPEND
INTO TABLE hr_dtls
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
(sid,sname)
{code}
If you want to define a file for bad records (that can't be insertet for any reasons) you can change it to
{code}
LOAD DATA
INFILE 'c:\test.txt'
BADFILE 'c:\test.bad'
APPEND
INTO TABLE hr_dtls
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
(sid,sname)

SQL*LOADER Question. Please help.

Hi List,
Below are my control and data files:(both files are stored on /Disk05)
terminal.ctl file:
LOAD DATA
INFILE = Disk05 erminal.dat
INTO TABLE TERMINAL_POSITION
FIELDS TERMINATED BY , ENCLOSED BY
TERMINAL_POSITION_ID VARCHAR2 TERMINATED BY , ENCLOSED BY ,
TERMINAL_POSITION_NAME VARCHAR2 TERMINATED BY ,
ENCLOSED BY ,
TERMINAL_POSITION_DESCRIPTION VARCHAR2 TERMINATED BY ,
ENCLOSED BY
terminal.dat file:
A,Axial,Terminals extend from both ends in the direction of the major
axis of a cylindrical or elliptical package.
B,Bottom,Terminals extend from the bottom of the package.
D,Dual,Terminals are on opposite sides of a square or rectangular
package or located in two parallel rows.
for any reason, when entering
sqlldr username/password control = /Disk05/terminal.ctl log
=/Disk05/terminal.log data = /Disk05/terminal.dat
an error message shows:
segmentation fault.
Alternatively, when issuing
sqlldr userid = username/password control = /Disk05/terminal.ctl log
=/Disk05/terminal.log
data = /Disk05/terminal.dat
an error message shows:
lf iopn failed for file (terminal.log)
Please note that: I did create terminal.log that is stored on /Disk05
as well. Please anyone point out what did I do wrong. Your help is
greatly appreciated.
Thanks
Trang

Hi,
Thanks to your help,finally, I successfully loaded the data into the database without any errors. Just wanted to mention that I have an error in my control file. The line infile = '/Disk05/terminal.dat' should be infile '/Disk05/terminal.dat' # no equal sign (=) in between.
Again, Thanks alot.
Trang

SQL*loader questions

If i use oracle functions like trim, upper in my control file then can i use DIRECT method
If i have specified DIRECT=TRUE, then what will happen if my target table has a trigger? Will it fail or use Conventional path? If it fails, how can i come to know?
If i have not specified DIRECT=TRUE, then will Oracle use DIRECT PATH if there are no triggers or foreign keys on my target table?
If I use INSERT option for loading data but my target table is not empty, then what error will be thrown
For date type field we specify the date mask. Are there any other data types for which we need to specify the format?
Thanks

Hello,
If i use oracle functions like trim, upper in my control file then can i use DIRECT method
Yes you can use
If i have specified DIRECT=TRUE, then what will happen if my target table has a trigger? Will it fail or use Conventional path? If it fails, how can i come to know?
YOur trigger wil be disabled if you have direct=true and wont' be used
If i have not specified DIRECT=TRUE, then will Oracle use DIRECT PATH if there are no triggers or foreign keys on my target table?
Unless you specified direct=true on command line or script, conventional path will be used
If I use INSERT option for loading data but my target table is not empty, then what error will be thrown
It will insert unless you have unless you have unique constraint on table and it will fail for those records and default error limit is 50. Other options are APPEND, TRUNCATE, APPEND
For date type field we specify the date mask. Are there any other data types for which we need to specify the format?
Timestamp and use of newline but that goes in infile option
Regards
Edited by: OrionNet on Mar 4, 2009 11:26 PM

Dummy SQL LOADER question

Hi ,
That might be pretty simple.
File format check.
How to check if records have more fields then defined in the control file?
When I load pipe delimeted file into the ORACLE table using sqlldr,
for records where number of fields less then defined in control file I'm getting an error, which is right.
Now, for records where number of fields is more then defined in control file IT IS LOADED OK, with no errors!
Is this right ?
And how can I get records where number of fields is more then defined in control file to the bad or discard file?
Thanks Leo.
Edited by: user10243745 on Oct 20, 2008 8:16 AM

Initially read the file using UTL_FILE, or some other tool, that can read line-by-line and count them.
But I most be honest and tell you that if you are getting invalid files my instinct would be to set a very low number for DISCARDMAX and reject the entire file back to its originating source.
If they can not consistently get you valid files then you should not be loading them.

Help with sql loader charsets

I was wondering if anyone could assist me with a sql loader question. I am trying to use it to load some hungarian data. It always turns the Ő character into an upside down question mark. I understand that it is because the database doesn't understand what that character is. Can someone help me find out how to resolve that? The database is UTF-8. If I paste the character in to the application, it takes it. I have tried all kinds of values for the CHARSET parameter in the .ctl file, but nothing seems to work....

I believe we are using iso 8859-1.
I am having a difficult time thinking that it could be we don't have the appropriate installations - because if we paste it in via the oracle application we can see it in a form. If we sql load that same character and look at it in an oracle form, we see the question marks.
i've looked at many of the documents on tahiti already, and found some helpful stuff, but no explanation for the problem i am having so far :)

Need suggestions for imporving data load performance via SQL Loader

Hi,
Our requirement is to load 512 (1 GB each) files in Oracle database.
We are using SQL loaders to load files into the DB (A partitioned table) and have tried almost all the possible options that come with sql loaders (Direct load path, parallel=true, multithreading=true, unrecoverable)
As the tables is growing bigger in size, each file load time is increasing (It started with 5 minutes per file and has reached 2 hours per 3 files now and is increasing with every batch- Note we are loading 3 files concurrently on the target table using the parallel = true oprion of sql loader)
Questions 1:
My problem is that somehow multithreading is not working for us (we have multi CPU server and have enabled multithreading=true). Could it be something to do with DB setting which might be hindering the data load to be done in multiple threads?
Question 2:
Would gathering stats on the target table and it's partitions help improve load performance ? I'm not sure if stats improve DML's, they would definitely improve sql queries. Any thoughts?
Question 3:
What would be the best strategy to gather stats on this table (which would end up having 512 GB data) ?
Question 4:
Do you think insertions in a partitioned table (with growing sizes) would have poor performance as compared to a non-partitioned table ?
Any other suggestions to improve performace are most welcome !!
Thanks,
Sachin
Edited by: Sachin Tiwari on Mar 13, 2013 6:29 AM

2 hours to load just 3 GB of data seems unreasonable regardless of the SQL Loader settings. It seems likely to me that the problem is not with SQL Loader but somewhere else.
Have you generated a Statspack/ AWR/ ASH report to see where all that time is being spent? Are there triggers on the table? Are there bitmap indexes?
Is your table partitioned in a way that is designed to improve the efficiency of loads so that all the data from one file goes into one partition? Or is data from each file getting inserted into many different partitions.
Justin

Dumb question re Sql Loader in XP Client

Hi all,
I have a rather silly question. Previously I've worked exclusively in a Unix environment, but now find myself in XP.
I've done the full install of the Windows client for XP, but SQL*Loader is missing. I've RTFM'd, but can't find the necessary section. How do I get sql*loader installed in the XP environment?
Thanks (and apologies) in advance

IF you are talking of the 9i version, there should be the three database install disks (that contain both the database and the client tools). When you run the installer from the first disk from this set, you will have the option of installing the client tools (including SQL*Loader).
http://www.oracle.com/technology/software/products/database/oracle10g/index.html
http://www.oracle.com/technology/software/products/oracle9i/htdocs/winsoft.html

SQL LOADER LPAD CONTROL FILE QUESTION

Hi, inthe flat file
company_cd is "1" or "01"
and center_cd is 3 digits... ex: "493"
in the table the userid coulmn should be 6 digits
currently i am getting this as userid in the table
"010493" is right, because company_cd is "01"
"10493" is not right, because company_cd is "1"
if company_cd is 2 digits(01) i am getting 6 digits userid which is OK
but when company_cd is singile digit(1) i am getting 5 digits userid
I NEED TO LPAD with 0 in the front when company_cd is "1"any suggetions ???????
***********This is the code i am using currently in the CTL file for userid**********
,USERID "CONCAT(substr(trim(:company_cd),1,2),lpad(trim(:center_cd),4,0))"
.......Thank You..........
Edited by: phani_Marella on Aug 28, 2012 11:12 AM

Now where does company 'coz' come from all of a sudden?
I'm sure you read {message:id=9360002} , hence my confusion.
Anyway, the SQL*Loader forum is @ Export/Import/SQL Loader & External Tables

Question about 'direct' parameter in sql loader utility

Hi eveyone,
Today I was asked what the advantages and disadvantages of 'direct' parameter of sql loader.
In my opinion, if swithing direct on, oracle use direct path load.
And it has no difference from the append hind in insert statement.
The core merits are less redo hence faster.
The main shortcoming is since little redo generated, we have to ask DBA to back up the data according to the rules immediately after the loading.
Does my answer make sense or not? Plesae help to point out the important things I missed.
best regards,
Leon

Hi,
Direct path Export is much faster than conventional path Export because the data is read from disk into the buffer cache and rows are transferred directly to the Export client.
Some conditions to use DIRECT parameter are,
1. The Export parameter BUFFER applies only to conventional path Exports. For direct path Export, use the RECORDLENGTH parameter to specify the size of the buffer that Export uses for writing to the export file.
2. You cannot use direct path when exporting in tablespace mode (TRANSPORT_TABLESPACES=Y).
3. The QUERY parameter cannot be specified in a direct path Export.
Thanks

SQL Loader newbie's question.

Hi all,
I have a delimiter file, e.g. data.dat, that contains the titles of each of the columns on the first row. Subsequent rows contain real column information/data. I didn't know how to create a control file, e.g. data.ctl, to create the table, its columns, and its data.
Currently, I got to create it manually, e.g.: table, columns, and used sql*loader to load in the data. Is there a better way to do this? Thank you very much for your help.

Yes, Sir. My external data is arranged as followed, e.g.:
"x","y","z"
"1","2","3"
"2","3","4"
"3","4","5"
where x, y, and z are the titles of the columns, not the data. My data are "1","2",...
Thank you very much for your response.

Question for SQL Loader / IMP /EXPORT

hi all,
i just describe my example first : -
TABLE A have five columns.
a_column,b_column,c_column,d_column,e_column.
and have 1000 records.
i want to remove c_column & e_column and add f_column.
on TABLE A. which method (SQL Loader or IMP/EXP) is good suggestion for export & import data before take above action.
boris

Export/Import is the most appropiate solution for the task that you want to carry out because SQL*Loader is designed to load data into the database from flat files.
Regarding how to drop the two columns , you can do it
directly if you are working in 9i setting both columns as
unused after drop unused columns in the table.
Complete reference for those utilities:
http://download-uk.oracle.com/docs/cd/B10501_01/server.920/a96652.pdf
Joel P�rez

Sql loader - skip record question

I am running Oracle 9 and using sql loader to import text file into table. Can sql loader skips the record which contain blank line or carriage return? Do I need to set up with options? Please advise me how. Thanks.

http://docs.oracle.com/cd/B10500_01/server.920/a96652/ch05.htm
http://www.orafaq.com/wiki/SQL*Loader_FAQ

How can we tell if SQL*Loader is working on a TABLE?

We have a process that requires comparing batches with LDAP information. Instead of using an LDAP lookup tool, we get a nightly directory file, and import the two COLUMNs we want via SQL*Loader (REPLACE) into an IOT. Out of three cases, two just check the first COLUMN, and the third needs the second COLUMN as well.
We did not think of using External TABLEs, because we cannot store files on the DB server itself.
The question arises, what to do while the file is being imported. The file is just under 300M, so it takes a minute or so to replace all the data. We found SQL*Loader waits until a transaction is finished before starting, but a query against the TABLE only waits while it is actually importing the data. At the beginning of SQL*Loader's process, however, a query against the TABLE returns no rows.
The solution we are trying right now is, to have the process that starts SQL*Loader flip a flag in another TABLE denoting that it is unavailable. When it is done, it flips it back, and notes the date. Then, the process that queries the information, exits if the flag is currently 'N'.
The problem, is, what if SQL*Loader starts inbetween the check of the flag, and the query against the TABLE. How do we guarantee that it is still not being imported.
I can think of three solutions:
1) LOCK the ldap information TABLE before checking the flag.
2) LOCK the record that the process starting SQL*Loader flips.
3) Add a clause to the query against the TABLE checks that there are records in the TABLE (AND EXISTS(SELECT * FROM ldap_information).
The problem with 3) is that the process has already tagged the batches (via a COLUMN). It could, technically reset them afterwards, but that seems a bit backwards.

Just out of curiosity, are you aware that Oracle supplies a DBMS_LDAP package for pulling information from LDAP sources? It would obviously be relatively easy to have a single transaction that deletes the existing data, loads the new data via DBMS_LDAP, and commits, which would get around the problem you're having with SQL*Loader truncating the table.
You could also have SQL*Loader load the data into a staging table and then have a second process either MERGE the changes from the staging table into the real table (again in a transactionally consistent manner) or just delete and insert the data.
Justin

How do i map one field to another in control file via SQL Loader

Can someone please reply back to this question
Hi,
I have a flat file (student.dat delimiter %~| ) using control file (student.ctl) through sql loader. Here are the details.
student.dat
student_id, student_firstname, gender, student_lastName, student_newId
101%~|abc%~|F %~|xyz%~|110%~|
Corresponding table
Student (
Student_ID,
Student_FN,
Gender,
Student_LN
Question:
How do i map student_newId field to student_id field in STUDENT DB table so that new id should be inserted in student_id column. How do i specify the mapping in control file. I dont want to create a new column in student table. Please let me know the best way to do this.
Can someone please reply back to this question.
My approach:
In control file i will sepecify the below, Is this a best approach?. Do we have any othe way?
STUDENT_ID *(:STUDENT_NEWID)*,
STUDENT_FN,
GENDER,
STUDENT_LNAME,
STUDENT_NEWID BOUNDFILLER
Thanks
Sunil
Edited by: 993112 on Mar 13, 2013 12:28 AM
Edited by: 993112 on Mar 13, 2013 12:30 AM
Edited by: 993112 on Mar 13, 2013 12:31 AM
Edited by: 993112 on Mar 18, 2013 2:52 AM

OK, ok...
Here is the sample data:
101%~|abc%~|F %~|xyz%~|110%~|
102%~|def%~|M %~|pqr%~|120%~|
103%~|ghi%~|M %~|stu%~|130%~|
104%~|jkl%~|F %~|vwx%~|140%~|
105%~|mno%~|F %~|yza%~|150%~|Here is the control file:
LOAD DATA
INFILE student.dat
TRUNCATE INTO TABLE STUDENT
FIELDS TERMINATED BY '%~|' TRAILING NULLCOLS
student_old FILLER
, student_fn
, gender
, student_ln
, student_id
)And here is the execution:
SQL> CREATE TABLE student
2 (
3    student_id   NUMBER
4 , student_fn   VARCHAR2 (10)
5 , gender       VARCHAR2 (2)
6 , student_ln   VARCHAR2 (10)
7 );
Table created.
SQL>
SQL> !sqlldr / control=student.ctl
SQL*Loader: Release 11.2.0.3.0 - Production on Tue Mar 19 14:37:31 2013
Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 5
SQL> select * from student;
STUDENT_ID STUDENT_FN                     GENDER STUDENT_LN
       110 abc                            F      xyz
       120 def                            M      pqr
       130 ghi                            M      stu
       140 jkl                            F      vwx
       150 mno                            F      yza
SQL>:p

2 sql loader questions

Similar Messages

Maybe you are looking for