Sql loader:loading data file name to one of the tabe column.
Hi,
I want to load the data file name also to one of the table column.
Table structure:
File_Name varchar2
Name varchar2
Age number
Data File Format:
Vijay|24
Anandh|24
Senthil|26
In the above eg how to write a control file to load the data file name to the File_Name column of the table.
Note:
The shell script will first pick up the data files to be loaded and calls the control file.
Thanks,
Vijay.
From a unix load perspective this is what we have done :
Create a control file (say load_data.ctl) something like
LOAD DATA
append
Into table file_name_age
File_Name constant '<FILENAME>',
Name CHAR TERMINATED BY '|'
Age INTEGER EXTERNAL TERMINATED BY '|' )
in a unix loop do the following
for i in `ls $LOADDIR` ; do
#Replace <FILENAME> with actual filename and create a temp ctl file
sed s/\<FILENAME\>/${1}/g < ${CTLDIR}/load_data.ctl > ${TMPDIR}/load_data_${1}.ctl
#load using temp file
sqlldr / CONTROL=${TMPDIR}/load_data_${1}.ctl
DATA=$i
#remove temp
rm ${TMPDIR}/load_data_${1}.ctl
done
Where
${TMPDIR}=your temp directory
${CTLDIR}=where your main directory for ctl files
$LOADDIR=Your data directory
Similar Messages
-
How to store data file name in one of the columns of staging table
My requirement is to load data from .dat file to oracle staging table. I have done following steps:
1. Created control file and stored in bin directory.
2. Created data file and stored in bin directory.
3. Registered a concurrent program with execution method as SQL*Loader.
4. Added the concurrent program to request group.
I am passing the file name as a parameter to concurrent program. When I am running the program, the data is getting loaded to the staging table correctly.
Now I want to store the filename (which is passed as a parameter) in one of the columns of staging table. I tried different ways found through Google, but none of them worked. I am using the below control file:
OPTIONS (SKIP = 1)
LOAD DATA
INFILE '&1'
APPEND INTO TABLE XXCISCO_SO_INTF_STG_TB
FIELDS TERMINATED BY ","
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
COUNTRY_NAME
,COUNTRY_CODE
,ORDER_CATEGORY
,ORDER_NUMBER
,RECORD_ID "XXCISCO_SO_INTF_STG_TB_S.NEXTVAL"
,FILE_NAME CONSTANT "&1"
,REQUEST_ID "fnd_global.conc_request_id"
,LAST_UPDATED_BY "FND_GLOBAL.USER_ID"
,LAST_UPDATE_DATE SYSDATE
,CREATED_BY "FND_GLOBAL.USER_ID"
,CREATION_DATE SYSDATE
,INTERFACE_STATUS CONSTANT "N"
,RECORD_STATUS CONSTANT "N"
I want to store file name in the column FILE_NAME stated above. I tried with and without constant using "$1", "&1", ":$1", ":&1", &1, $1..... but none of them worked. Please suggest me the solution for this.
Thanks,
AbhayPl post details of OS, database and EBS versions. There is no easy way to achieve this.
Pl see previous threads on this topic
SQL*Loader to insert data file name during load
Sql Loader with new column
HTH
Srini -
Create sql loader data file dynamically
Hi,
I want a sample program/approach which is used to create a sql loader data file.
The program will read table name as i/p and will use
select stmt will column list derived from user_tab_columns from data dictionary
assuming multiple clob columns in the column list.
Thanks
ManojI 'm writing clob and other columns to a sql loader dat file.
Below sample code for writing clob column is giving file write error.
How can I write multiple clobs to dat file so that control file will handle it correctly
offset NUMBER := 1;
chunk VARCHAR2(32000);
chunk_size NUMBER := 32000;
WHILE( offset < dbms_lob.getlength(l_rec_type.narrative) )
LOOP
chunk := dbms_lob.substr(l_rec_type.narrative, chunk_size, offset );
utl_file.put( l_file_handle, chunk );
utl_file.fflush(l_file_handle);
offset := offset + chunk_size;
END LOOP;
utl_file.new_line(l_file_handle); -
How to get the Output File Name as One of the Field Value From Payload
Hi All,
I want to get the Output file name as one of the Field value from payload.
Example:
Source XML
<?xml version="1.0" encoding="UTF-8" ?>
- <ns0:MT_TEST xmlns:ns0="http://sample.com">
- <Header>
<NAME>Bopanna</NAME>
</Header>
</ns0:MT_TEST>
I want to get the Output file name as " Bopanna.xml"
Please suggest me on this.
Regards
BopannaHi,
There are couple of links already available for this. Just for info see the below details,
The Output file name could be used from the field value of payload. For this you need to use the UDF DynamicFile name with below code,
// Description: Function to create dynamic Filename
DynamicConfiguration conf = (DynamicConfiguration) container.getTransformationParameters().get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File" , "FileName");
conf.put(key,a);
return "";
DynamicConfiguration conf = (DynamicConfiguration) container.getTransformationParameters().get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File" , "FileName");
conf.put(key,a);
return "";
With this udf map it with the MessageType as
(File Name field from Payload) > DynamicFileConfiguration>MTReceiver
Thanks
Swarup -
Skipping fields in SQL*LOADER data file
I have a data file that has more fields than the target table does. How can I write a SQL*LOADER control file to skip some fields in the middle of the text line?
nullIf you don't want to define input fields by position, the simplest way I think is to use FILLER fields.
Quoted from SQL*Loader doc:
"Specifying Filler Fields
Filler fields have names but they are not loaded into the table. However, filler fields can be used as arguments to init_specs (for example, NULLIF and DEFAULTIF) as well as to directives (for example, SID, OID, REF, BFILE). Also, filler fields can occur anyplace in the data file. They can be inside of the field list for an object or inside the definition of a VARRAY.
See SQL*Loader DDL Behavior and Restrictions for more information on filler fields and their use.
A sample filler field specification looks as follows:
field_1_count FILLER char,
Ex:
Regards,
Zoltan -
How to get the file name in one of the Xml node field?
Hi Experts,
i have one requirement with file name in xml file.
i need to get <b>the file name of file</b> in one of the node in xml content of that file.
how can i achieve that req?
treat this as Very Urgent.Hi Bheem,
Explain your scenario in detail..
>>i need to get the file name of file in one of the node in xml content of that file.
1. Create an node <b>Filename</b> in the target datatype.
2. Check the Option Filename in the Adapter Specific Message Attributes in the sender file CC.
3. Use the following UDF and map it to the Filename field in the target message type.
public String getFileName(Container container)
DynamicConfiguration conf = (DynamicConfiguration) container.getTransformationParameters().get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File","FileName");
String ourSourceFileName = conf.get(key);
return ourSourceFileName;
Regards
San -
SQL*Loader .dat file
Hello I am trying to migrate large quantities of data from our Access database to Oracle. I managed to generate my SQL*Loader scripts in the OMWB directory but when I try to run the .bat file a text doc is generated with the following message:
SQL*Loader-128: unable to begin a session
ORA-01017: Invalid username/password: logon denied
I have tried to match the username/password in both my OEM an OMWB with no success. Please does anybody know where I am going and how it can be rectified. Or should I recreate the user again via OMWB?
Thank you
[email protected]Pilton,
I assume the problem with the username and password which you are using to run.
And one more thing that check the session previleges on the user side.
Hope it will help you.
Ramesh
Hello I am trying to migrate large quantities of data from our Access database to Oracle. I managed to generate my SQL*Loader scripts in the OMWB directory but when I try to run the .bat file a text doc is generated with the following message:
SQL*Loader-128: unable to begin a session
ORA-01017: Invalid username/password: logon denied
I have tried to match the username/password in both my OEM an OMWB with no success. Please does anybody know where I am going and how it can be rectified. Or should I recreate the user again via OMWB?
Thank you
[email protected] -
Pre parse terrible SQL Loader data file
Long story, but I'm on Oracle EBS 11.5.10.2 which means i'm stuck with an SQL Loader from the dark ages (8.06 - yes, I know and yes, it's still supported under EBS).
Anyhow. I have a vendor sending very annoying data. For example:
\|"Fan-Hou" dessert|
\|"Donald is cool | yes”|
\|"Fred “is” cool | yes”|
Due to the limitations of such an old SQL Loader, I need to end up with:
\|"Fan-Hou dessert"|
\|"Donald is cool | yes”|
\|"Fred “is” cool | yes”|
So you can see that really it's only the first row which is causing the issue, I need to shift the quote out to the end of the string (just before the |). The other two lines are just for reference, to show that I get data like that too, and I need to be able to keep that as is.
I have a rather un-elegant solution solution using perl but I'm sure someone out there knows a clever way to achieve the same result. Before you ask - no, the vendor won't change the data.
Edited by: user13007502 on 07-May-2013 01:35There are probably various ways you can do that. I would try the SED command.
The idea is the following:
1. Replace all double quotes of words that start and end with a space with a special character.
2. Remove all all double quotes.
3. Add double quotes to the beginning and end.
4. Reconstruct double quotes of words according to 1.
For instance:
$ cat testfile
|"Fan-Hou" dessert|
|"Donald is cool | yes"|
|"Fred "is" cool | yes"|
Replace double quotes with {} for words that start and end with a space:
sed -i 's: "\(.*\)" : {\1} :g' testfile
$ cat testfile
|"Fan-Hou" dessert|
|"Donald is cool | yes"|
|"Fred {is} cool | yes"|
Remove all double quotes
$ sed -i 's:"::g' testfile
$ cat testfile
|Fan-Hou dessert|
|Donald is cool | yes|
|Fred {is} cool | yes|
Replace all lines that begin and end with | with |" and "|.
$ sed -i 's:^|:|":g' testfile
$ sed -i 's:\(.*\)|:\1"|:g' testfile
$ cat testfile
|"Fan-Hou dessert"|
|"Donald is cool | yes"|
|"Fred {is} cool | yes"|
Finally replace { and } with double qoutes:
sed -i 's:{:":g' testfile
sed -i 's:}:":g' testfile
Here is the result:
# cat testfile
|"Fan-Hou dessert"|
|"Donald is cool | yes"|
|"Fred "is" cool | yes"|
{code} -
Even if one of the open tabs contain a .pdf file opened in it, then I can't scroll the pages down in any other tab. Irrespective of which tab is active, the pages in the pdf get scrolled. :(
I've tried to reinstall the Firefox, but that hasn't helped. In fact the problem continues even after I've done a complete system format.My workaround for this problem is to open the tabbed pdf document into a new Firefox window (Right-Click tab --> Move to New Window) which unlocks scrolling capabilities for all tabs in your original window.
-
SQLLoader: Uploading data file name along with data
I am using Oracle SQLLoader to upload my data from CSV files into a single table. My current CTL file is as follows:
LOAD DATA
INFILE 'trace_metals100.csv'
INFILE 'trace_metals101.csv'
APPEND
INTO TABLE TRACE_METALS
FIELDS TERMINATED BY ','
(STATION_NO, METAL_NAME, METAL_WT)
Just for my record I want to upload the data file name also along with the data into a fourth column. How should I do it using SQLLoader??
Regards,
MoizDo two loads using two separate control files:
LOAD DATA
INFILE 'trace_metals100.csv'
APPEND
INTO TABLE trace_metals
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
(station_no,
metal_name,
metal_wt,
fourth_column_name CONSTANT trace_metals100.csv)
LOAD DATA
INFILE 'trace_metals101.csv'
APPEND
INTO TABLE trace_metals
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
(station_no,
metal_name,
metal_wt,
fourth_column_name CONSTANT trace_metals101.csv) -
Insert data file name into table from sql loader
Hi All,
I have a requirement to insert the data file name dynamically into table using sql loader.
Example:
sqlldr userid=username/passwword@host_string control=test_ctl.ctl data=test_data.dat
test_ctl.ctl
LOAD DATA
FILED TERMINATED BY ','
INTO TABLE test
(empid number,
ename varchar2(20),
file_name varchar2(20) ---------- This should be the data file name which can be dynamic (coming from parameter)
test_data.dat
1,test
2,hello
3,world
4,end
Please help..
Thanks in advance.
Regards
Anujyou'll probably have to write your control file on the fly, using a .bat or .sh file
rem ===== file : test.bat ========
rem
rem ============== in pseudo speak =============
rem
rem
echo LOAD DATA > test.ctl
echo FILED TERMINATED BY ',' >> test.ctl
echo INTO TABLE test >> test.ctl
echo (empid number, >> test.ctl
echo ename varchar2(20), >> test.ctl
echo file_name constant %1% >> test.ctl
echo ) >> test.ctl
rem
rem
rem
sqlldr userid=username/passwword@host_string control=test.ctl data=test_data.dat
rem =============== end of file test.bat =======================
http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_field_list.htm#i1008664 -
Possibility of capturing data file name in SQL * Loader
Hi,
I have a requirement to capture the data file name in the staging table, is there a way that i can capture it in SQL * Loader or any other way of doing it.
Need experts suggestion please.
Thanks,
GenooHi Genoo.
how do we capture the file name and stores in the temporary table
You may use the above command mentioned in my previous post (if Linux) to populate the Test.csv file with the available file name in the directory, i.e:
ls /some/path/*.dat | xargs -n1 basename > /home/oracle/Test.csv
1. Ensure to first load the Test.csv file as for eg:
1,aaa
2,bbb
3,ccc
2. Create a control file to load these records into temporary tables,for eg:
load data
infile '/home/oracle/Test.csv'
into table file_name_upload
fields terminated by ","
( id, file_name )
3. Create the respective table in the database:
create table file_name_upload
id number,
file_name varchar2(20)
4. Load the data into temporary table
sqlldr test/test control=/home/oracle/sqlldr_test.ctl
Please refer notes:
SQL*Loader - How To Load A Date Column With Fractions Of Second (Doc ID 1276259.1)
Script To Generate SQL*Loader Control File (Doc ID 1019523.6)
SQL*Loader performance tips (Doc ID 28631.1)
How To use the Sequence Function of SQL*Loader (Doc ID 1058895.6)
How to Get Data from Existing Table to Flat File Usable by SQL*Loader (Doc ID 123852.1)
Also see link:
10 Oracle SQLLDR Command Examples (Oracle SQL*Loader Tutorial)
Thanks &
Best regards, -
SQL*Loader to insert data file name during load
I'd like to use a single control file to load data from different files (at different times) to the same table. I'd like this table to have a column to hold the name of the file the data came from. Is there a way for SQL*Loader to automatically do this? (I.e., as opposed to running an update query separately.) I can edit the control file before each load to set a CONSTANT to hold the new data file name, but I'd like it to pick this up automatically.
Thanks for any help.
-- HarveyHello Harvey.
I've previously attempted to store a value into a global/local OS variable and use this within a SQL*Loader control file (Unix OS and Oracle versions between 7.3.4 and 10g). I was unsuccessful in each attempt and approach I could imagine. It was very easy to use a sed script to make a copy of the control file, changing a string within it to do this however.
Do you really want to store a file name on each and every record? Perhaps an alternative would be to use a relational model. Create a file upload log table that would store the file name and an upload # and then have the SQL*Loader control file call a function that would read that table for the most recent upload #. You'll save some disk space too.
Hope this helps,
Luke -
Passing file name dynamically to sql loader ctl file
Hi,
I am new to scripting and I have a complex requirement involving writing a script.
Requierment:
I need to upload a CSV file from a FTP server into oracle table using SQL Loader. The file name resembles like APF0912312359.csv represents 31-DEC-2009 23:59 and there will be multiple files in same day indicated by differnt timestamp as its filename. Once I load a file using SQL loader, I need to move the file to another directory for archival.
Since the sql loader ctl file has a fixed file name, I would like to know how I can pass the file name dynamically to ctl file to load a new file every time it runs.
Any help is greatly appreciated..
Bye
Sudheer
Edited by: user2138419 on Oct 28, 2009 4:08 PMI agree with Pradeep in regards to declaring the file names on the command line. However, I do have a concern regarding presenting the password on the command line as any user that can issue a ps (assuming Unix ~= Linux here) would be able to read it while the sqlldr command is running.
Unfortunately, you may not always have the option of declaring the files on the command line. I was recently challenged with this in a Windows 2003 Server environment running SQL*Loader: Release 10.2.0.1.0. In this environment I found that I am able to set a variable file name in a calling batch file and use that value in the control file successfully. Just to demonstrate the approach:
Batch file:
set IN_FILE=’c:\inbound\load_me.txt’
sqlldr user/pswd@db PARFILE=’c:\parameters.txt’
Parameter file:
errors=500000
rows=50000
control=%CTL_FILE%
bad=%BAD_FILE%
discard=%DSC_FILE%
log=%LOG_FILE%
Control file:
LOAD DATA
INFILE ‘%IN_FILE%’
INSERT
INTO TABLE table_to_be_loaded
I’m really interested to see if this would work on Unix.
-Luke -
Parameters / Variables in SQL Loader Control File
Here's my problem, I have created SQL Loader control file (ctl) to load a file of employees. This loader program is registered in Oracle Applications. What I want is to pass a parameter derived from the profile like the user name of the one who executed the loader program and pass it to the loader program so that the user name info is included in the loading
example
input data
123456,John,Smith
654321,Jane,Doe
loaded data
123456,John,Smith,03-JUL-2009,USER13
654321,Jane,Doe,03-JUL-2009,USER13
the sysdate is easy because it can be defined in the column but what I really want to achieve is a parameter passed to the ctl file.
Thanks in advance.Dear user!
Please have a look at this thread.
{thread:id=915277}
In the last three posts I explain how to use a shellscript with AWK and SED to pass parameters to a controlfile.
Please feel free to post again if you have some questions regarding my explanatory notes.
Yours sincerely
Florian W.
Maybe you are looking for
-
We have a Canon imageRunner 5020 networked here. Several computers cannot print to it; it appears that Accounting is not available for the printer drivers even though Accounting (Dept ID) is enabled on the computer. Several Windows machines print jus
-
I can load Pages on my IMac when I log in as a guest but not when I go into my normal main log in. Its a new Imac and Apple transferred all my apps over from my Mac Book Pro. Many thanks Phil
-
I am not allowed to use upgrade on this computer ???
Itunes 7.6 keeps telling me, that I am not allowed to use the Jan-update on my computer ? ? ? It does not allow me to sync the upgrade to my ipod touch FW 1.1.3 eventhough my Vista-PC is activated to Apple. Need help - thanks
-
I had a CC 2014 student subscription and want to renew but what will it cost?
I want to renew my Creative Cloud student subscription but in the "update payment details" all I can see is a figure of $24.95..is it a month to month subscription? I am surprised there isn't more information. I am an Australian customer
-
Help I am having trouble updating Ipod's
Help I am having trouble updating our Ipod's & Iphone. I have followed the troubleshooting on Itunes error page, but still not working. I don't have an error number just this message - "There was a problem downloading the software for the iphone "A