SQL*Loader to insert data file name during load

I'd like to use a single control file to load data from different files (at different times) to the same table. I'd like this table to have a column to hold the name of the file the data came from. Is there a way for SQL*Loader to automatically do this? (I.e., as opposed to running an update query separately.) I can edit the control file before each load to set a CONSTANT to hold the new data file name, but I'd like it to pick this up automatically.
Thanks for any help.
-- Harvey

Hello Harvey.
I've previously attempted to store a value into a global/local OS variable and use this within a SQL*Loader control file (Unix OS and Oracle versions between 7.3.4 and 10g). I was unsuccessful in each attempt and approach I could imagine. It was very easy to use a sed script to make a copy of the control file, changing a string within it to do this however.
Do you really want to store a file name on each and every record? Perhaps an alternative would be to use a relational model. Create a file upload log table that would store the file name and an upload # and then have the SQL*Loader control file call a function that would read that table for the most recent upload #. You'll save some disk space too.
Hope this helps,
Luke

Similar Messages

Insert data file name into table from sql loader

Hi All,
I have a requirement to insert the data file name dynamically into table using sql loader.
Example:
sqlldr userid=username/passwword@host_string control=test_ctl.ctl data=test_data.dat
test_ctl.ctl
LOAD DATA
FILED TERMINATED BY ','
INTO TABLE test
(empid number,
ename varchar2(20),
file_name varchar2(20) ---------- This should be the data file name which can be dynamic (coming from parameter)
test_data.dat
1,test
2,hello
3,world
4,end
Please help..
Thanks in advance.
Regards
Anuj

you'll probably have to write your control file on the fly, using a .bat or .sh file
rem ===== file : test.bat ========
rem
rem ============== in pseudo speak =============
rem
rem
echo LOAD DATA > test.ctl
echo FILED TERMINATED BY ',' >> test.ctl
echo INTO TABLE test >> test.ctl
echo (empid number, >> test.ctl
echo ename varchar2(20), >> test.ctl
echo file_name constant %1% >> test.ctl
echo ) >> test.ctl
rem
rem
rem
sqlldr userid=username/passwword@host_string control=test.ctl data=test_data.dat
rem =============== end of file test.bat =======================
http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_field_list.htm#i1008664

How to store data file name in one of the columns of staging table

My requirement is to load data from .dat file to oracle staging table. I have done following steps:
1. Created control file and stored in bin directory.
2. Created data file and stored in bin directory.
3. Registered a concurrent program with execution method as SQL*Loader.
4. Added the concurrent program to request group.
I am passing the file name as a parameter to concurrent program. When I am running the program, the data is getting loaded to the staging table correctly.
Now I want to store the filename (which is passed as a parameter) in one of the columns of staging table. I tried different ways found through Google, but none of them worked. I am using the below control file:
OPTIONS (SKIP = 1)
LOAD DATA
INFILE '&1'
APPEND INTO TABLE XXCISCO_SO_INTF_STG_TB
FIELDS TERMINATED BY ","
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
COUNTRY_NAME
,COUNTRY_CODE
,ORDER_CATEGORY
,ORDER_NUMBER
,RECORD_ID "XXCISCO_SO_INTF_STG_TB_S.NEXTVAL"
,FILE_NAME CONSTANT "&1"
,REQUEST_ID "fnd_global.conc_request_id"
,LAST_UPDATED_BY "FND_GLOBAL.USER_ID"
,LAST_UPDATE_DATE SYSDATE
,CREATED_BY "FND_GLOBAL.USER_ID"
,CREATION_DATE SYSDATE
,INTERFACE_STATUS CONSTANT "N"
,RECORD_STATUS CONSTANT "N"
I want to store file name in the column FILE_NAME stated above. I tried with and without constant using "$1", "&1", ":$1", ":&1", &1, $1..... but none of them worked. Please suggest me the solution for this.
Thanks,
Abhay

Pl post details of OS, database and EBS versions. There is no easy way to achieve this.
Pl see previous threads on this topic
SQL*Loader to insert data file name during load
Sql Loader with new column
HTH
Srini

Possibility of capturing data file name in SQL * Loader

Hi,
I have a requirement to capture the data file name in the staging table, is there a way that i can capture it in SQL * Loader or any other way of doing it.
Need experts suggestion please.
Thanks,
Genoo

Hi Genoo.
how do we capture the file name and stores in the temporary table
You may use the above command mentioned in my previous post (if Linux) to populate the Test.csv file with the available file name in the directory, i.e:
ls /some/path/*.dat | xargs -n1 basename > /home/oracle/Test.csv
1. Ensure to first load the Test.csv file as for eg:
1,aaa
2,bbb
3,ccc
2. Create a control file to load these records into temporary tables,for eg:
load data
infile '/home/oracle/Test.csv'
into table file_name_upload
fields terminated by ","
( id, file_name )
3. Create the respective table in the database:
create table file_name_upload
id number,
file_name varchar2(20)
4. Load the data into temporary table
sqlldr test/test control=/home/oracle/sqlldr_test.ctl
Please refer notes:
SQL*Loader - How To Load A Date Column With Fractions Of Second (Doc ID 1276259.1)
Script To Generate SQL*Loader Control File (Doc ID 1019523.6)
SQL*Loader performance tips (Doc ID 28631.1)
How To use the Sequence Function of SQL*Loader (Doc ID 1058895.6)
How to Get Data from Existing Table to Flat File Usable by SQL*Loader (Doc ID 123852.1)
Also see link:
10 Oracle SQLLDR Command Examples (Oracle SQL*Loader Tutorial)
Thanks &
Best regards,

Sql loader:loading data file name to one of the tabe column.

Hi,
I want to load the data file name also to one of the table column.
Table structure:
File_Name varchar2
Name varchar2
Age number
Data File Format:
Vijay|24
Anandh|24
Senthil|26
In the above eg how to write a control file to load the data file name to the File_Name column of the table.
Note:
The shell script will first pick up the data files to be loaded and calls the control file.
Thanks,
Vijay.

From a unix load perspective this is what we have done :
Create a control file (say load_data.ctl) something like
LOAD DATA
append
Into table file_name_age
File_Name constant '<FILENAME>',
Name CHAR TERMINATED BY '|'
Age INTEGER EXTERNAL TERMINATED BY '|' )
in a unix loop do the following
for i in `ls $LOADDIR` ; do
#Replace <FILENAME> with actual filename and create a temp ctl file
sed s/\<FILENAME\>/${1}/g < ${CTLDIR}/load_data.ctl > ${TMPDIR}/load_data_${1}.ctl
#load using temp file
sqlldr / CONTROL=${TMPDIR}/load_data_${1}.ctl
DATA=$i
#remove temp
rm ${TMPDIR}/load_data_${1}.ctl
done
Where
${TMPDIR}=your temp directory
${CTLDIR}=where your main directory for ctl files
$LOADDIR=Your data directory

How to insert the file name to the start and the end of a SQL file by a powershell script

I just began to learn powershell script and I have a problem that I cannot figure out. I have a lot of .sql files. I need to insert its file name to the beginning of the file (After USE statement) and the end of the file (before the last END statement). Is
there anybody knowing how to do this?
Thanks so much
Regards

Actually, I want to add a insert SQL statement to the beginning of the file and the end of the file. Only the file name is a variable and different among
different files. All the other sql statements are the same and can be hard coded. Hopefully it helps –
The problem I think is that I want to add the same thing (except the file name) to hundreds of sql files and only the file name is different. I do not want
to add it manually and want to use a script to add it –

How to load thousands of data files?

My database is installed on windows, and I have thousands of data files that I need to load. Trying to figure out how to do it without manually editing my CTL file every time I load one data file.
Can you specifiy your control file to load directories instead of just a file?
Or can some looping script be written that makes the changes?
Thanks!
Message was edited by:
user447781

Don't list the data file in the CTL file ... use the "data=" option on the command line to pass in the data file.
A wrapper BAT file should be easy to create/generate.
D:\>sqlldr
SQL*Loader: Release 9.2.0.6.0 - Production on Mon Aug 29 13:52:54 2
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Usage: SQLLDR keyword=value [,keyword=value,...]
Valid Keywords:
    userid -- ORACLE username/password
   control -- Control file name
       log -- Log file name
       bad -- Bad file name
      data -- Data file name
   discard -- Discard file name
discardmax -- Number of discards to allow          (Default all)
...

Looping multiple excel files at a time in ssis and create a new column to insert the files names

Hi Friend,
i have one requirement.
I got around 200 excel files with date as file name like 120101.xls, 120102.xls (YYMMDD).
I am able to loop all the files and insert data but i am unable to load the files names using ssis.
Please can any one help to fix this using script task or any other code..
It is very urgent

Thank You Vaibhav,
I want to insert the file name along with the other columns of data from excel.
like this deriving a new column FILE_NAME
Your ForEach Loop task must be returning the entire file path, I suppose.
1. Get the file name from your file path.
New variable: FileName
Expression : RIGHT(@[User::FilePath],FINDSTRING(REVERSE(@[User::FilePath]),"\\",1)-1)
2. Inside Data flow, after Excel source use Dervied Column and use above @FileName
3. At your OLEDB destinition, use MyFileName.
-Vaibhav Chaudhari

SQLLoader: Uploading data file name along with data

I am using Oracle SQLLoader to upload my data from CSV files into a single table. My current CTL file is as follows:
LOAD DATA
INFILE 'trace_metals100.csv'
INFILE 'trace_metals101.csv'
APPEND
INTO TABLE TRACE_METALS
FIELDS TERMINATED BY ','
(STATION_NO, METAL_NAME, METAL_WT)
Just for my record I want to upload the data file name also along with the data into a fourth column. How should I do it using SQLLoader??
Regards,
Moiz

Do two loads using two separate control files:
LOAD DATA
INFILE 'trace_metals100.csv'
APPEND
INTO TABLE trace_metals
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
(station_no,
metal_name,
metal_wt,
fourth_column_name CONSTANT trace_metals100.csv)
LOAD DATA
INFILE 'trace_metals101.csv'
APPEND
INTO TABLE trace_metals
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
(station_no,
metal_name,
metal_wt,
fourth_column_name CONSTANT trace_metals101.csv)

Date, File name, User Stamp

I've looked for hours for a script that will work in Illustrator CS4 that will put a text block on the page identifying:
Date, File Name and the User's name on the document (art board)
I am not versed in scripting. Can anyone help me out?

Hi Y'all This post is where scripting all started for me. At the time I didn't know anything and Muppet Mark kindly helped me out. Since then I have learned Applescript and written my slug script in that. This has the advantage of being able to access system information. Here it is:
-- captures the user's first name from the system for use in dialog boxes
set myName to (long user name of (system info))
set sp to (offset of " " in myName)
set myFirstName to text 1 thru (sp - 1) of myName
-- captures the user's short name from the system for use in the slug
set myShortName to (short user name of (system info))
set myInitials to text 1 thru 2 of myShortName
tell application "Adobe Illustrator" to set myDocCount to count of documents
if myDocCount > 0 then
    tell application "Adobe Illustrator"
        try
            set myPath to file path of document 1 as alias -- if the document is not saved this will cause an error
        on error
            display dialog "You haven't saved document yet " & myFirstName & "." buttons {"Cancel", "Save"} default button 2
            -- brings up Save As dialog box
            tell application "System Events" to tell process "Adobe Illustrator"
                click menu item "Save As..." of menu 1 of menu bar item "File" of menu bar 1
                tell application "Finder"
                    set filesavepath to "/Volumes/Server/Folder" -- put your own file path in here
                end tell
                delay 0.2
                tell window "Save As"
                    keystroke "g" using {command down, shift down}
                    repeat until exists sheet 1
                        delay 0.5
                    end repeat
                    tell sheet 1
                        keystroke filesavepath
                        click button "Go"
                    end tell
                end tell
            end tell
            -- end Save As dialog box
            return
        end try
        set myFile to name of document 1
        set myFolder to myPath as string
        -- this section looks at the file info to get the file version information
        set myFileVersion to my getVersion(myPath)
        set AppleScript's text item delimiters to "Created"
        set myFileVersion to text item 1 of myFileVersion
        if myFileVersion is "Saved As v.15 " then
            set myFileVersion to "Illustrator CS5"
        else if myFileVersion is "Saved As v.14 " then
            set myFileVersion to "Illustrator CS4"
        else if myFileVersion is "Saved As v.13 " then
            set myFileVersion to "Illustrator CS3"
        else if myFileVersion is "Saved As v.12 " then
            set myFileVersion to "Illustrator CS2"
        end if
        set AppleScript's text item delimiters to {""}
        -- end of file info
        set AppleScript's text item delimiters to ":"
        set clientFolder to text item 3 of myFolder -- always picks out the client folder on our server
        set AppleScript's text item delimiters to {""}
        if clientFolder contains "_" then
            set clientFolder to my cleanName(clientFolder) -- to format the client names properly
        else
            set clientFolder to clientFolder
        end if
        -- find the current date. This is the date when the file was last edited but this method enables you to run the script before saving
        tell application "Finder"
            set myDate to (current date) as string
        end tell
        set myDate to (word 2 of myDate) & " " & (text 1 thru 3 of word 3 of myDate) & " " & (word 4 of myDate) as string
        -- end find date
        -- checks for the correctly named frames and then populates with the correct information
        if exists (text frame "titleblock-software" of document 1) then
            set contents of text frame "titleblock-software" of document 1 to myFileVersion
        end if
        if exists (text frame "titleblock-client" of document 1) then
            set contents of text frame "titleblock-client" of document 1 to clientFolder
        end if
        if exists (text frame "titleblock-file" of document 1) then
            set contents of text frame "titleblock-file" of document 1 to myFile
        end if
        if exists (text frame "titleblock-path" of document 1) then
            set contents of text frame "titleblock-path" of document 1 to myPath as string
        end if
        if exists (text frame "titleblock-date" of document 1) then
            set contents of text frame "titleblock-date" of document 1 to myDate
        end if
        if exists (text frame "titleblock-editor" of document 1) then
            set contents of text frame "titleblock-editor" of document 1 to myInitials
        end if
        if exists (text frame "titleblock-version" of document 1) then
            set versionCount to get contents of text frame "titleblock-version" of document 1
            set versionCount to (versionCount + 1)
            set contents of text frame "titleblock-version" of document 1 to versionCount
        end if
        -- end populating named frames
    end tell
end if
-- this handler is to get the version of the file
on getVersion(added_item)
    set myFileVersion to long version of (info for added_item)
end getVersion
-- this handler converts every 'odd' character to an underscore, modify as needed
on cleanName(newName)
    set chars to every character of newName
    repeat with i from 1 to length of chars
        if item i of chars as text is equal to "_" then
            set item i of chars to " "
        end if
    end repeat
    return every item of chars as string
end cleanName

Quicken data file names are mysteriously being change on my iDisk

within the last couple of days my Quicken data file names are mysteriously being change on my iDisk which seem to make them inaccesable. Not sure why this is happening

within the last couple of days my Quicken data file names are mysteriously being change on my iDisk which seem to make them inaccesable. Not sure why this is happening

Feature Request: Ultrabeat: Load Sample auto copy file name to Drum Voice

Feature Request: Ultrabeat: Load Sample auto copy file name to Drum Voice
I wish that when I load a sample into a drum voice, that it would automatically copy the name of the sound file I'm importing to the drum voice minus the extension. If I were importing BD01.wav I'd like for the drum voice to display BD01. As it is, I have to import the file, then name or rename the drum voice to BD01. When importing large quntities, this becomes a giant pain, and I'm looking for anything programmatic that could save me a load of time. When I think of all the time I'm going to waste...and if you multiply that by all the number of users...that's a lot of wasted time.
I wish of course that the program could suck up and autoload all of the sound files within a directory, names too, until the voices were full. Then I could just fine tune and kick out ones I don't want or want in different order. Not a big deal for users who are importing from all kinds of different sources, but say for impoting libraries like NI's Battery, it's a big deal.
Maybe these could be choices in the load sample menu.
powerbook G4 Mac OS X (10.4.4)
pb g4 Mac OS X (10.4.4)

Microsoft pulled the plug on DirectSound HAL in Vista due to lack of hardware support. The SAL exists for
software compatibility, good intention but bad performance as there is no longer a direct path from
DirectSound to audio drivers.
People choose DirectSound for its Direct-ness
but there is no reason to choose it now.
DirectX SDK was integrated
with WIndows SDK which ships with Visual Studio 2012 so you don't need additional downloads for
WASAPI and XAudio 2.
Just look up the documentation for headers/librarieslike every other Windows API you use. If you have a hard time to find the documentation then you need to go to search engines and find search engine tutorials.
There are plenty of samples for both WASAPI and XAudio 2, both in the Windows SDK and online. The DirectX team has some suggestions on which to use on their team blog.
You can find experts for those APIs at the Windows Desktop Pro-Audio Application Development forum and the
Audio/XACT forum on MSDN (link left out for you to practice your search skills). Again, it is fine to ask for hints but don't ask the whole
solution. You are competing with others who just need a hint to finish work here. The time spent on working on your assignment would be better spent on helping on giving hints to others.
Visual C++ MVP

Need to generate multiple error files with rule file names during parallel data load

Hi,
Is there a way that MAXL could generate multiple error files during parallel data load?
import database AsoSamp.Sample data
connect as TBC identified by 'password'
using multiple rules_file 'rule1' , 'rule2'
to load_buffer_block starting with buffer_id 100
on error write to "error.txt";
I want to get error files as this - rule1.err, rule2.err (Error files with rule file name included). Is this possible in MAXL?
I even faced a situation , If i hard code the error file name like above, its giving me error file names as error1.err and error2.err. Is there any solution for this?
Thanks,
DS

Are you saying that if you specify the error file as "error.txt" Essbase actually produces multiple error files and appends a number?
Tim.
Yes its appending the way i said.
Out of interest, though - why do you want to do this? The load rules must be set up to select different 'chunks' of input data; is it impossible to tell which rule an error record came from if they are all in the same file?
I have like 6 - 7 rule files using which the data will be pulled from SQL and loaded into Essbase. I dont say its impossible to track the error record.
Regardless, the only way I can think of to have total control of the error file name is to use the 'manual' parallel load approach. Set up a script to call multiple instances of MaxL, each performing a single load to a different buffer. Then commit them all together. This gives you most of the parallel load benefit, albeit with more complex scripting.
Even i had the same thought of calling multiple instances of a Maxl using a shell script. Could you please elaborate on this process? What sort of complexity is involved in this approach.? Did anyone tried it before?
Thanks,
DS

SQL Loader - Field in data file exceeds maximum length

Dear All,
I have a file which has more than 4000 characters in a field and I wish to load the data in a table with field length = 4000. but I receive error as
Field in data file exceeds maximum lengthThe below given are the scripts and ctl file
Table creation script:
CREATE TABLE "TEST_TAB"
    "STR" VARCHAR2(4000 BYTE),
    "STR2" VARCHAR2(4000 BYTE),
    "STR3" VARCHAR2(4000 BYTE)
);Control file:
LOAD DATA
INFILE 'C:\table_export.txt'
APPEND INTO TABLE TEST_TAB
FIELDS TERMINATED BY '|'
TRAILING NULLCOLS
( STR CHAR(4000) "SUBSTR(:STR,1,4000)" ,
STR2 CHAR(4000) "SUBSTR(:STR2,1,4000)" ,
STR3 CHAR(4000) "SUBSTR(:STR3,1,4000)"
)Log:
SQL*Loader: Release 10.2.0.1.0 - Production on Mon Jul 26 16:06:25 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Control File:   C:\TEST_TAB.CTL
Data File:      C:\table_export.txt
Bad File:     C:\TEST_TAB.BAD
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 0
Bind array:     64 rows, maximum of 256000 bytes
Continuation:    none specified
Path used:      Conventional
Table TEST_TAB, loaded from every logical record.
Insert option in effect for this table: APPEND
TRAILING NULLCOLS option in effect
   Column Name                  Position   Len Term Encl Datatype
STR                                 FIRST 4000   |       CHARACTER
    SQL string for column : "SUBSTR(:STR,1,4000)"
STR2                                 NEXT 4000   |       CHARACTER
    SQL string for column : "SUBSTR(:STR2,1,4000)"
STR3                                 NEXT 4000   |       CHARACTER
    SQL string for column : "SUBSTR(:STR3,1,4000)"
value used for ROWS parameter changed from 64 to 21
Record 1: Rejected - Error on table TEST_TAB, column STR.
Field in data file exceeds maximum length
MAXIMUM ERROR COUNT EXCEEDED - Above statistics reflect partial run.
Table TEST_TAB:
0 Rows successfully loaded.
1 Row not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
Space allocated for bind array:                 252126 bytes(21 rows)
Read   buffer bytes: 1048576
Total logical records skipped:          0
Total logical records read:             1
Total logical records rejected:         1
Total logical records discarded:        0
Run began on Mon Jul 26 16:06:25 2010
Run ended on Mon Jul 26 16:06:25 2010
Elapsed time was:     00:00:00.22
CPU time was:         00:00:00.15Please suggest a way to get it done.
Thanks for reading the post!
*009*

Hi Toni,
Thanks for the reply.
Do you mean this?
CREATE TABLE "TEST"."TEST_TAB"
    "STR" VARCHAR2(4001),
    "STR2" VARCHAR2(4001),
    "STR3" VARCHAR2(4001)
);However this does not work as the error would be:
Error at Command Line:8 Column:20
Error report:
SQL Error: ORA-00910: specified length too long for its datatype
00910. 00000 - "specified length too long for its datatype"
*Cause:    for datatypes CHAR and RAW, the length specified was > 2000;
           otherwise, the length specified was > 4000.
*Action:   use a shorter length or switch to a datatype permitting a
           longer length such as a VARCHAR2, LONG CHAR, or LONG RAW*009*
Edited by: 009 on Jul 28, 2010 6:15 AM

Using SQL Loader to insert data based on conditionally statements

I would like to use sql loader to insert records from a single text file into an Oracle database table. Sounds easy but the problem is that I need to check the existence of data in another table (Table A) before data can be loaded into Table B.
Table A has two columns: dept_no (primary key value) and dept_name (char)
Table B has five columns: emp_no (primary key value), dept_no (foreign key value to Table A), employee (char), job_title (char) and salary (number)
Text File looks like this:
Finance Jones President 10000
HR Smith Admin 2000
HelpDesk Jenkins Technician 3000
Problem:
1. I need sql loader to insert records into Table B by first checking if the first field in the file is in Table A.
2. If value exists get it's dept_no value.
3. If value doesn't exist discard record ( I might want to have sql loader insert a new record for this into Table A)
4. Using value from #2, insert the value in Table B for dept_no column.
5. Also assign a sequence value for the emp_no value in Table B.
Any guidance is greatly appreciated.
Thanks,

Hello,
I am not sure this is possible with SQL loader. I would rather use an external table based on your file.
Then, I would use SQL to load your data into your table, based on your conditions.
Your request is not very complicated, writing the SQL to do that will be really more simple than trying to do it with SQL loader.
Hope it will help.
Regards,
Sylvie

SQL*Loader to insert data file name during load

Similar Messages

Maybe you are looking for