Loading from text file using Sql Loader

I need to load data from a text file into Oracle table. The file has long strings of text in the following format:
12342||||||Lots and lots of text with all kinds of characters including
^&*!#%#^@ etc.xxxxxxxxxxxxxxxxxxxxxxx
yyyyyyyyyyyyyyyyyyyyyyyyytrrrrrrrrrrrrrrrrrrr
uuuuuuuuuuuuuuuuuuurtgggggggggggggggg.||||||||
45356|||||||||||again lots and lots of text.uuuuuudccccccccccccccccccccd
gyhjjjjjjjjjjjjjjjjjjjjjjjjkkkkkkkkkkkkklllllllllllnmmmmmmmmmmmmnaaa|||||||.
There are pipes within the text as well. On the above example, the line starting with 12342 is an entire record that needs to be loaded into a CLOB column. The next record would be the 45356 one. Therefore, all records have a bunch of pipes at the end, so the only way to know where a new record starts is to see where the next number is after all the ending pipes. The only other thing I know is that there are a fixed number of pipes in each record.
Does anyone have any ideas on how I can load the data into the table either using sql loader or any other utility? Any input would be greatly appreciated. Thanks.

STFF [url http://forums.oracle.com/forums/thread.jspa?messageID=1773678&#1764219]Sqlldr processing of records with embedded newline and delimiter

Similar Messages

Not loading from flat file using SQL*Loader

Hi,
I am trying to load from an excel file.
first i converted excel file into csv file and save it as as dat file.
in the excel file one column is salary and the data is like $100,000
while converting xls to csv the salary is changed to "$100,000 " (with quotes and a space after the amount)
after the last digit it will put a space.
in the control file of sql*loader i had given
salary "to_number('L999,999')"
my problem is the space after the salary in the dat file.---> "$100,000 "
what changes i have to make in the to_number function which is in the control file.
Please guide me.
Thanks & Regards
Salih KM
Message was edited by:
kmsalih

Thanks a lot Jens Petersen
It's is loading ..........
MI means miniute.
am i correct.
but i didn't get the logic behind that.
can u please explain that.
Thanks & Regards
Salih KM

Issue while loading a csv file using sql*loader...

Hi,
I am loading a csv file using sql*loader.
On the number columns where there is data populated in them, decimal number/integers, the row errors out on the error -
ORA-01722: invalid number
I tried checking the value picking from the excel,
and found the chr(13),chr(32),chr(10) values characters on the value.
ex: select length('0.21') from dual is giving a value of 7.
When i checked each character as
select ascii(substr('0.21',5,1) from dual is returning a value 9...etc.
I tried the following command....
"to_number(trim(replace(replace(replace(replace(:std_cost_price_scala,chr(9),''),chr(32),''),chr(13),''),chr(10),'')))",
to remove all the non-number special characters. But still facing the error.
Please let me know, any solution for this error.
Thanks in advance.
Kiran

control file:
OPTIONS (ROWS=1, ERRORS=10000)
LOAD DATA
CHARACTERSET WE8ISO8859P1
INFILE '$Xx_TOP/bin/ITEMS.csv'
APPEND INTO TABLE XXINF.ITEMS_STAGE
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' TRAILING NULLCOLS
ItemNum                    "trim(replace(replace(:ItemNum,chr(9),''),chr(13),''))",
cross_ref_old_item_num               "trim(replace(replace(:cross_ref_old_item_num,chr(9),''),chr(13),''))",
Mas_description               "trim(replace(replace(:Mas_description,chr(9),''),chr(13),''))",
Mas_long_description               "trim(replace(replace(:Mas_long_description,chr(9),''),chr(13),''))",
Org_description               "trim(replace(replace(:Org_description,chr(9),''),chr(13),''))",
Org_long_description               "trim(replace(replace(:Org_long_description,chr(9),''),chr(13),''))",
user_item_type                    "trim(replace(replace(:user_item_type,chr(9),''),chr(13),''))",
organization_code               "trim(replace(replace(:organization_code,chr(9),''),chr(13),''))",
primary_uom_code               "trim(replace(replace(:primary_uom_code,chr(9),''),chr(13),''))",
inv_default_item_status          "trim(replace(replace(:inv_default_item_status,chr(9),''),chr(13),''))",
inventory_item_flag               "trim(replace(replace(:inventory_item_flag,chr(9),''),chr(13),''))",
stock_enabled_flag               "trim(replace(replace(:stock_enabled_flag,chr(9),''),chr(13),''))",
mtl_transactions_enabled_flag          "trim(replace(replace(:mtl_transactions_enabled_flag,chr(9),''),chr(13),''))",
revision_qty_control_code          "trim(replace(replace(:revision_qty_control_code,chr(9),''),chr(13),''))",
reservable_type               "trim(replace(replace(:reservable_type,chr(9),''),chr(13),''))",
check_shortages_flag               "trim(replace(replace(:check_shortages_flag,chr(9),''),chr(13),''))",
shelf_life_code               "trim(replace(replace(replace(replace(:shelf_life_code,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
shelf_life_days               "trim(replace(replace(replace(replace(:shelf_life_days,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
lot_control_code               "trim(replace(replace(:lot_control_code,chr(9),''),chr(13),''))",
auto_lot_alpha_prefix               "trim(replace(replace(:auto_lot_alpha_prefix,chr(9),''),chr(13),''))",
start_auto_lot_number               "trim(replace(replace(:start_auto_lot_number,chr(9),''),chr(13),''))",
negative_measurement_error          "trim(replace(replace(replace(replace(:negative_measurement_error,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
positive_measurement_error          "trim(replace(replace(replace(replace(:positive_measurement_error,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
serial_number_control_code          "trim(replace(replace(:serial_number_control_code,chr(9),''),chr(13),''))",
auto_serial_alpha_prefix          "trim(replace(replace(:auto_serial_alpha_prefix,chr(9),''),chr(13),''))",
start_auto_serial_number          "trim(replace(replace(:start_auto_serial_number,chr(9),''),chr(13),''))",
location_control_code               "trim(replace(replace(:location_control_code,chr(9),''),chr(13),''))",
restrict_subinventories_code          "trim(replace(replace(:restrict_subinventories_code,chr(9),''),chr(13),''))",
restrict_locators_code               "trim(replace(replace(:restrict_locators_code,chr(9),''),chr(13),''))",
bom_enabled_flag               "trim(replace(replace(:bom_enabled_flag,chr(9),''),chr(13),''))",
costing_enabled_flag               "trim(replace(replace(:costing_enabled_flag,chr(9),''),chr(13),''))",
inventory_asset_flag               "trim(replace(replace(:inventory_asset_flag,chr(9),''),chr(13),''))",
default_include_in_rollup_flag          "trim(replace(replace(:default_include_in_rollup_flag,chr(9),''),chr(13),''))",
cost_of_goods_sold_account          "trim(replace(replace(:cost_of_goods_sold_account,chr(9),''),chr(13),''))",
std_lot_size                    "trim(replace(replace(replace(replace(:std_lot_size,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
sales_account                    "trim(replace(replace(:sales_account,chr(9),''),chr(13),''))",
purchasing_item_flag               "trim(replace(replace(:purchasing_item_flag,chr(9),''),chr(13),''))",
purchasing_enabled_flag          "trim(replace(replace(:purchasing_enabled_flag,chr(9),''),chr(13),''))",
must_use_approved_vendor_flag          "trim(replace(replace(:must_use_approved_vendor_flag,chr(9),''),chr(13),''))",
allow_item_desc_update_flag          "trim(replace(replace(:allow_item_desc_update_flag,chr(9),''),chr(13),''))",
rfq_required_flag               "trim(replace(replace(:rfq_required_flag,chr(9),''),chr(13),''))",
buyer_name                    "trim(replace(replace(:buyer_name,chr(9),''),chr(13),''))",
list_price_per_unit               "trim(replace(replace(replace(replace(:list_price_per_unit,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
taxable_flag                    "trim(replace(replace(:taxable_flag,chr(9),''),chr(13),''))",
purchasing_tax_code               "trim(replace(replace(:purchasing_tax_code,chr(9),''),chr(13),''))",
receipt_required_flag               "trim(replace(replace(:receipt_required_flag,chr(9),''),chr(13),''))",
inspection_required_flag          "trim(replace(replace(:inspection_required_flag,chr(9),''),chr(13),''))",
price_tolerance_percent          "trim(replace(replace(replace(replace(:price_tolerance_percent,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
expense_account               "trim(replace(replace(:expense_account,chr(9),''),chr(13),''))",
allow_substitute_receipts_flag          "trim(replace(replace(:allow_substitute_receipts_flag,chr(9),''),chr(13),''))",
allow_unordered_receipts_flag          "trim(replace(replace(:allow_unordered_receipts_flag,chr(9),''),chr(13),''))",
receiving_routing_code               "trim(replace(replace(:receiving_routing_code,chr(9),''),chr(13),''))",
inventory_planning_code          "trim(replace(replace(:inventory_planning_code,chr(9),''),chr(13),''))",
min_minmax_quantity               "trim(replace(replace(replace(replace(:min_minmax_quantity,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
max_minmax_quantity               "trim(replace(replace(replace(replace(:max_minmax_quantity,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
planning_make_buy_code               "trim(replace(replace(:planning_make_buy_code,chr(9),''),chr(13),''))",
source_type                    "trim(replace(replace(:source_type,chr(9),''),chr(13),''))",
mrp_safety_stock_code               "trim(replace(replace(:mrp_safety_stock_code,chr(9),''),chr(13),''))",
material_cost                    "trim(replace(replace(:material_cost,chr(9),''),chr(13),''))",
mrp_planning_code               "trim(replace(replace(:mrp_planning_code,chr(9),''),chr(13),''))",
customer_order_enabled_flag          "trim(replace(replace(:customer_order_enabled_flag,chr(9),''),chr(13),''))",
customer_order_flag               "trim(replace(replace(:customer_order_flag,chr(9),''),chr(13),''))",
shippable_item_flag               "trim(replace(replace(:shippable_item_flag,chr(9),''),chr(13),''))",
internal_order_flag               "trim(replace(replace(:internal_order_flag,chr(9),''),chr(13),''))",
internal_order_enabled_flag          "trim(replace(replace(:internal_order_enabled_flag,chr(9),''),chr(13),''))",
invoice_enabled_flag               "trim(replace(replace(:invoice_enabled_flag,chr(9),''),chr(13),''))",
invoiceable_item_flag               "trim(replace(replace(:invoiceable_item_flag,chr(9),''),chr(13),''))",
cross_ref_ean_code               "trim(replace(replace(:cross_ref_ean_code,chr(9),''),chr(13),''))",
category_set_intrastat               "trim(replace(replace(:category_set_intrastat,chr(9),''),chr(13),''))",
CustomCode                    "trim(replace(replace(:CustomCode,chr(9),''),chr(13),''))",
net_weight                    "trim(replace(replace(replace(replace(:net_weight,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
production_speed               "trim(replace(replace(:production_speed,chr(9),''),chr(13),''))",
LABEL                         "trim(replace(replace(:LABEL,chr(9),''),chr(13),''))",
comment1_org_level               "trim(replace(replace(:comment1_org_level,chr(9),''),chr(13),''))",
comment2_org_level               "trim(replace(replace(:comment2_org_level,chr(9),''),chr(13),''))",
std_cost_price_scala               "to_number(trim(replace(replace(replace(replace(:std_cost_price_scala,chr(9),''),chr(32),''),chr(13),''),chr(10),'')))",
supply_type                    "trim(replace(replace(:supply_type,chr(9),''),chr(13),''))",
subinventory_code               "trim(replace(replace(:subinventory_code,chr(9),''),chr(13),''))",
preprocessing_lead_time          "trim(replace(replace(replace(replace(:preprocessing_lead_time,chr(9),''),chr(32),''),chr(13),''),chr(10),''))",
processing_lead_time                "trim(replace(replace(replace(replace(:processing_lead_time,chr(9),''),chr(32),''),chr(13),''),chr(10),''))",
wip_supply_locator               "trim(replace(replace(:wip_supply_locator,chr(9),''),chr(13),''))"
Sample data from csv file.
"9901-0001-35","390000","JMKL16 Pipe bend 16 mm","","JMKL16 Putkikaari 16 mm","","AI","FJE","Ea","","","","","","","","","","","","","","","","","","","","","","","","","21-21100-22200-00000-00000-00-00000-00000","0","21-11100-22110-00000-00000-00-00000-00000","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","0.1","Pull","AFTER PROD","","","Locator for Production"
The load errors out on especially two columns :
1) std_cost_price_scala
2) list_price_per_unit
both are number columns.
And when there is data being provided on them. It errors out. But, if they are holding null values, the records go through fine.
Message was edited by:
KK28

How to Load Arabic Data from flat file using SQL Loader ?

Hi All,
We need to load Arabic data from an xls file to Oracle database, Request you to provide a very good note/step to achieve the same.
Below are the database parameters used
NLS_CHARACTERSET AR8ISO8859P6
nls_language american
DB version:-10g release 2
OS: rhel 5
Thanks in advance,
Satish

Try to save your XLS file into CSV format and set either NLS_LANG to the right value or use SQL*Loader control file parameter CHARACTERSET.
See http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_control_file.htm#i1005287

Problem loading XML-file using SQL*Loader

Hello,
I'm using 9.2 and tryin to load a XML-file using SQL*Loader.
Loader control-file:
LOAD DATA
INFILE *
INTO TABLE BATCH_TABLE TRUNCATE
FIELDS TERMINATED BY ','
FILENAME char(255),
XML_DATA LOBFILE (FILENAME) TERMINATED BY EOF
BEGINDATA
data.xml
The BATCH_TABLE is created as:
CREATE TABLE BATCH_TABLE (
FILENAME VARCHAR2 (50),
XML_DATA SYS.XMLTYPE ) ;
And the data.xml contains the following lines:
<?xml version="2.0" encoding="UTF-8"?>
<!DOCTYPE databatch SYSTEM "databatch.dtd">
<batch>
<record>
<data>
<type>10</type>
</data>
</record>
<record>
<data>
<type>20</type>
</data>
</record>
</batch>
However, the sqlldr gives me an error:
Record 1: Rejected - Error on table BATCH_TABLE, column XML_DATA.
ORA-21700: object does not exist or is marked for delete
ORA-06512: at "SYS.XMLTYPE", line 0
ORA-06512: at line 1
If I remove the first two lines
"<?xml version="2.0" encoding="UTF-8"?>"
and
"<!DOCTYPE databatch SYSTEM "databatch.dtd">"
from data.xml everything works, and the contentents of data.xml are loaded into the table.
Any idea what I'm missing here? Likely the problem is with special characters.
Thanks in advance,

I'm able to load your file just by removing the second line <!DOCTYPE databatch SYSTEM "databatch.dtd">. I dont have your dtd file, so skipped that line. Can you check if it's problem with ur DTD?

Need suggestions on loading 5000+ files using sql loader

Hi Guys,
I'm writing a shell script to load more than 5000 files using sql loader.
My intention is to load the files in parallel. When I checked the maximum number of sessions in v$parameter, it is around 700.
Before starting the data load, programmatically I am getting the number of current sessions and maximum number of sessions and keeping free 200 sessions without using them (max. no. of sessions minus 200 ) and utilizing the remaining ~300 sessions to load the files in parallel.
Also I am using a "wait" option to make the shell to wait until the 300 concurrent sql loader process to complete and moving further.
Is there any way to make it more efficient? Also is it possible to reduce the wait time without hard coding the seconds (For Example: If any of those 300 sessions becomes free, assign the next file to the job queue and so on..)
Please share your thoughts on this.
Thanks.

Manohar wrote:
I'm writing a shell script to load more than 5000 files using sql loader.
My intention is to load the files in parallel. When I checked the maximum number of sessions in v$parameter, it is around 700. Concurrent load you mean? Parallel processing implies take a workload, breaking that up into smaller workloads, and doing that in parallel. This is what the Parallel Query feature does in Oracle.
SQL*Loader does not do that for you. It uses a single session to load a single file. To make it run in parallel, requires manually starting multiple loader sessions and perform concurrent loads.
Have a look at Parallel Data Loading Models in the Oracle® Database Utilities guide. It goes into detail on how to perform concurrent loads. But you need to parallelise that workload yourself (as explained in the manual).
Before starting the data load, programmatically I am getting the number of current sessions and maximum number of sessions and keeping free 200 sessions without using them (max. no. of sessions minus 200 ) and utilizing the remaining ~300 sessions to load the files in parallel.
Also I am using a "wait" option to make the shell to wait until the 300 concurrent sql loader process to complete and moving further.
Is there any way to make it more efficient? Also is it possible to reduce the wait time without hard coding the seconds (For Example: If any of those 300 sessions becomes free, assign the next file to the job queue and so on..)Consider doing it the way that Parallel Query does (as I've mentioned above). Take the workload (all files). Break the workload up into smaller sub-workloads (e.g. 50 files to be loaded by a process). Start a 100 processes in parallel and provide each one with a sub-workload to do (100 processes each loading 50 odd files).
This is a lot easier to manage than starting for example a 5000 load processes and then trying some kind of delay method to ensure that not all hit the database at the same time.
I'm loading about 100+ files (3+ million rows) every 60 seconds 24x7 using SQL*Loader. Oracle is quite scalable and SQL*Loader quite capable.

Loading XML file using sql*loader (10g)

Hi Everyone....
I have a major problem. I have a 10g database and I need to use sql loader to load a large XML file into the database. This file contains records for many, many customers. This will be done several times and the number of records will vary. I want to use sql loader and load to a staging table, BUT SPLIT THE RECORDS OUT and add a sequence. I am having 2 problems.
In the 10g documentation, when you want to split the records you use the BEGINDATA clause and indicate something (like a 0) for each instance of a record. Well my first file has 3,722 records in it. How do I know the number of records in each XML file?
The second problem is that because this is XML, I thought that I could use ENCLOSED BY. But the start tag has an attribute /counter in it, sql*loader doesnt recognize the starting tag. (i.e.: start tag is: </CustomerRec number=1>
end tag is: </CustomerRec> )
So, I used TERMINATED BY '</CustomerRec>'. This will split out the records, but it will NOT include the ending tag of </CustomerRec> and when I use extract, I receive an XML parsing error.
I have tried to get the ending tag using CONTINUEIF and SKIP. But those options are for "records" and not lines.
Does anyone have any ideas for the 2 problems above. I am at my wits end and I need to finish this ASAP. Any help would be appreciated.
Thank you!

Sorry.... here is an example of what my control file looks like. At the end, I have 92 "0", but remember, I have 3722 records in this first file.
LOAD DATA (SKIP 1)
INFILE *
INTO TABLE RETURN_DATA_STG
TRUNCATE
XMLType(xmldata)
FIELDS
(fill FILLER CHAR(1),
loadseq SEQUENCE(MAX,1),
xmldata LOBFILE (CONSTANT F001AB.XML)
TERMINATED BY '</ExtractObject>'
------ ":xmldata||'</ExtractObject>'"
BEGINDATA
0
0
0
0
0
0

Loading Data Error while using SQL Loader

Hi All,
I am facing another issue while loading the data from text file to oracle db.
I have a table in which data is loaded from multiple files. What i want the when i load the data i want to load the filename as well in the table in front of the record. the script which works fine when i load data in windows OS but i am getting error in Unix OS. My sql loader script is as follows.
source $HOME/. .bash_profile
export LOGFILE=/NSN/rawfiles/scripts/PAYMOBILE_LOAD/load.log
ORACLE_ACCESS=cdr/cdr123_awcc@$ORACLE_SID
FILES=`ls /NSN/rawfiles/bkp/PAYMOBILE/JAN2013/*.txt`;
for file in ${FILES[@]}
do
filename=`expr substr "$file" 43 50`
echo $filename
sqlldr $ORACLE_ACCESS control=/NSN/rawfiles/scripts/PAYMOBILE/CAT.ctl log=$LOGFILE data=/NSN/rawfiles/bkp/PAYMOBILE/JAN2013/$filename skip=0 direct=true
cd
mv /NSN/rawfiles/bkp/PAYMOBILE/JAN2013/$filename /NSN/backup/PAYMOBILE/JAN2013/
done
Control File for the same which i run on unix is :
Load data
INFILE "AR_101_1011_01-01-2013-01-32.txt"
Append
into table CDR.PAY_MOBILE_012013
fields terminated by "|" optionally enclosed by " "
trailing nullcols
VERSION, TICKETTYPE, GMT_TIME Date 'YYYY-MM-DD HH24:MI:SS' , LOCAL_TIME Date 'YYYY-MM-DD HH24:MI:SS', TRANSACID, EXTERNAL_TRANSACTION_ID, MEDIUM, ALIASCATEGORY, SERVICE, OPERATION, ACTORID,
ALIASNAME, SENDERACTORID, SENDERALIASNAME, RECIPIENTACTORID, RECIPIENTALIASNAME, THIRDACTORID, THIRDALIASNAME, ERRORHRN, CLASS, SNE,
MASTERSNE, VOUCHERCATEGORYID, VOUCHERSCENARIOID, EXPIRYDATE Date 'YYYY-MM-DD HH24:MI:SS', UNITVALUEID, CREDIT, VAT, IERROR, OPERSTATE, OPERERROR, CREATION_DATE Date 'YYYY-MM-DD HH24:MI:SS',
LAST_UPDATE Date 'YYYY-MM-DD HH24:MI:SS', BRANDID, CREDIT_PERIOD, CREDIT_EXPIRY_DATE Date 'YYYY-MM-DD HH24:MI:SS', SENDER_ACCOUNT, SENDER_TRANSAC, SENDER_ACCOUNT_TYPE, RECIPIENT_ACCOUNT,
RECIPIENT_TRANSAC, RECIPIENT_ACCOUNT_TYPE, RECIPIENT_MEDIUM, TIMESTAMP Date 'YYYY-MM-DD HH24:MI:SS', SENDERBILLINGTYPE, RECIPIENTBILLINGTYPE
Control file which works fine in windows is as follows:
Load data
     Append
     into table CDR.PAY_MOBILE_012013
     fields terminated by "|" optionally enclosed by " "
     trailing nullcols
VERSION, TICKETTYPE, GMT_TIME Date 'YYYY-MM-DD HH24:MI:SS' , LOCAL_TIME Date 'YYYY-MM-DD HH24:MI:SS', TRANSACID, EXTERNAL_TRANSACTION_ID, MEDIUM, ALIASCATEGORY, SERVICE, OPERATION, ACTORID,
ALIASNAME, SENDERACTORID, SENDERALIASNAME, RECIPIENTACTORID, RECIPIENTALIASNAME, THIRDACTORID, THIRDALIASNAME, ERRORHRN, CLASS, SNE,
MASTERSNE, VOUCHERCATEGORYID, VOUCHERSCENARIOID, EXPIRYDATE Date 'YYYY-MM-DD HH24:MI:SS', UNITVALUEID, CREDIT, VAT, IERROR, OPERSTATE, OPERERROR, CREATION_DATE Date 'YYYY-MM-DD HH24:MI:SS',
LAST_UPDATE Date 'YYYY-MM-DD HH24:MI:SS', BRANDID, CREDIT_PERIOD, CREDIT_EXPIRY_DATE Date 'YYYY-MM-DD HH24:MI:SS', SENDER_ACCOUNT, SENDER_TRANSAC, SENDER_ACCOUNT_TYPE, RECIPIENT_ACCOUNT,
RECIPIENT_TRANSAC, RECIPIENT_ACCOUNT_TYPE, RECIPIENT_MEDIUM, TIMESTAMP Date 'YYYY-MM-DD HH24:MI:SS', SENDERBILLINGTYPE, RECIPIENTBILLINGTYPE, FILENAME CONSTANT "AR_101_1011_01-12-2012-23-32.txt"
and the sql loader script which works fine in windows is as follows:
cd K:\paymobilefiles\DEC2012\1
for %%f in (*.txt) do (K:\paymobilefiles\DEC2012\1\LOAD\REPTEXT K:\paymobilefiles\DEC2012\1\LOAD\TEST.ctl "%%f">K:\paymobilefiles\DEC2012\1\LOAD\MYMOBILE.ctl
sqlldr userid=cdr/cdr123_awcc@tsiindia control=K:\paymobilefiles\DEC2012\1\LOAD\MYMOBILE.ctl rows=50000 bindsize=20000000 readsize=20000000 data=%%f log=K:\paymobilefiles\DEC2012\1\LOAD\MYMOBILE.log
move %%f K:\paymobilefiles\DEC2012\backup\1
when using windows based scripts i am getting filename in the filename column but when running the above mentioned scripts i am not able to get the filenames in the filename column.
Can anyone help over the same please

HI,
There is Difference in Control files , no INFILE & FILENAME CONSTANT values
Load data
Append
into table CDR.PAY_MOBILE_012013
fields terminated by "|" optionally enclosed by " "
trailing nullcols
VERSION, TICKETTYPE, GMT_TIME Date 'YYYY-MM-DD HH24:MI:SS' , LOCAL_TIME Date 'YYYY-MM-DD HH24:MI:SS', TRANSACID, EXTERNAL_TRANSACTION_ID, MEDIUM, ALIASCATEGORY, SERVICE, OPERATION, ACTORID,
ALIASNAME, SENDERACTORID, SENDERALIASNAME, RECIPIENTACTORID, RECIPIENTALIASNAME, THIRDACTORID, THIRDALIASNAME, ERRORHRN, CLASS, SNE,
MASTERSNE, VOUCHERCATEGORYID, VOUCHERSCENARIOID, EXPIRYDATE Date 'YYYY-MM-DD HH24:MI:SS', UNITVALUEID, CREDIT, VAT, IERROR, OPERSTATE, OPERERROR, CREATION_DATE Date 'YYYY-MM-DD HH24:MI:SS',
LAST_UPDATE Date 'YYYY-MM-DD HH24:MI:SS', BRANDID, CREDIT_PERIOD, CREDIT_EXPIRY_DATE Date 'YYYY-MM-DD HH24:MI:SS', SENDER_ACCOUNT, SENDER_TRANSAC, SENDER_ACCOUNT_TYPE, RECIPIENT_ACCOUNT,
RECIPIENT_TRANSAC, RECIPIENT_ACCOUNT_TYPE, RECIPIENT_MEDIUM, TIMESTAMP Date 'YYYY-MM-DD HH24:MI:SS', SENDERBILLINGTYPE, RECIPIENTBILLINGTYPE, FILENAME CONSTANT "AR_101_1011_01-12-2012-23-32.txt"
)Thanks,
Ajay More
http://moreajays.blogspot.com

Loading leap year date using SQL*Loader

Hello,
I have a problem loading a date '29/02/2000' using SQL*Loader. This date is on a leap year. I'm getting an error message from SQL*Loader as 'ORA-01839: date not valid for month specified'. My colleague and I have tried using various date functions to convert the data into date, but no luck.
I would appreciate any helps,
Bruce

Thanks for your help, I found the bug on my control file. I was using the RTRIM function to remove bad timestamp such as '29/02/2000 0:00:00'. So instead of using this statement:
LOG_DATE DATE "DD/MM/RRRR" "RTRIM(:LOG_DATE,'0:00:00')"
I was using the statement below with a space before the '0:00:00' string literal, with the intention to remove a space also:
LOG_DATE DATE "DD/MM/RRRR" "RTRIM(:LOG_DATE,' 0:00:00')"
Well, it turned out that if there was a space before the string literal, RTRIM function would trim the matching string plus any '0' characters from the right, including the '000' that belongs to '2000'. Thus, the error.
Thanks again,
Bruce

Loading millions of rows using SQL*loader to a table with constraints

I have a table with constraints and I need to load millions of rows in it using SQL*Loader.
What is the best way to do this, means what SQL*Loader options to use, for getting the best loading performance and how to deal with constraints?
Regards

- check if your table has check constraints (like column not null)
if you trust the data in the file you have to load you can disable this constrainst and after the loader enable this constrainst.
- Check if you can modify the table and place it in nologging mode (generate less redo but ONLY is SOME Conditions)
Hope it helps
Rui Madaleno

Loading data with dates using SQL*Loader

Dear everyone
I am currently trying to load some data containing dates using SQL*Loader.
For termination of fields I have been using ^ because I have some book titles which contain " and ' as part of their title. I found that the TO_DATE function did not seem to work using ^ instead of ". Would I be correct? I think the Oracle manual says that " must be used.
After some Web research I eventually amended my control file to as follows:
load data
infile 'h:\insert_statements\22_insert_into_SCAN_FILE_INFO.txt'
REPLACE
into table SCAN_FILE_INFO
fields terminated by "," optionally enclosed by '^'
TRAILING NULLCOLS
(scan_id, scan_filename
file_format_id
orig_scanning_resolution_dpi
scanner_id, scanner_operator_id
scanning_date "TO_DATE (:scanning_date, 'YYYY-MM-DD')"
original_map_publication_id
reprint_publication_id)
A simple line of data is as follow:
280001, ^1910 - London^, 270001, 400, 250001, 260001, "TO_DATE('2007-06-06', 'YYYY-MM-DD')", 200019,
The final column being null.
However when I attempt that I get the following error message:
Record 1: Rejected - Error on table SCAN_FILE_INFO, column SCANNING_DATE.
ORA-01841: (full) year must be between -4713 and +9999, and not be 0
If I change the scanning_date part to:
scanning_date "EXPRESSION TO_DATE (:scanning_date, 'YYYY-MM-DD')",
or
scanning_date "CONSTANT TO_DATE (:scanning_date, 'YYYY-MM-DD')",
I get the error message:
Record 1: Rejected - Error on table SCAN_FILE_INFO, column SCANNING_DATE.
ORA-00917: missing comma
As soon as I do the following:
scanning_date "EXPRESSION, TO_DATE (:scanning_date, 'YYYY-MM-DD')",
or
scanning_date "CONSTANT, TO_DATE (:scanning_date, 'YYYY-MM-DD')",
I get too many values error message:
Record 1: Rejected - Error on table SCAN_FILE_INFO.
ORA-00913: too many values
I also tested out scanning_date DATE "YYYY-MM-DD", but that just gave the same ORA-01841 error message as above.
I must be doing something very simple which is wrong but I cannot figure it out.
Kind regards
Tim

And why do you have scanning date as "TO_DATE('2007-06-06', 'YYYY-MM-DD')" in your infile? All you need is 2007-06-06. If you can not change infile generation code, use:
load data
infile 'h:\insert_statements\22_insert_into_SCAN_FILE_INFO.txt'
REPLACE
into table SCAN_FILE_INFO
fields terminated by "," optionally enclosed by '^'
TRAILING NULLCOLS
(scan_id, scan_filename
file_format_id
orig_scanning_resolution_dpi
scanner_id, scanner_operator_id
scanning_date "TO_DATE(REPLACE(REPLACE(:scanning_date,'TO_DATE('),'''YYYY-MM-DD'')'), 'YYYY-MM-DD')"
original_map_publication_id
reprint_publication_id)SY.

Uploading into database table from text file using tab (GUI_UPLOAD)

i have small doubt
i have 3 fiels in text file using tab as separator
i need to update into database table 'ZABPSP_01'
from 's.txt' located in local disk.
My code is below.
Please let me know the correction.
Awaiting for ur response.
Thanks in advance
REPORT ZABPSPPRG_02.
TABLES: LFA1,MARA,KNA1,ZABPSP_01.
DATA: begin of itab occurs 0,
IKUNNR type zabpsp_01-kunnr,
IMATNR type zabpsp_01-matnr,
IADRNR type zabpsp_01-adrnr.
DATA:END OF ITAB.
DATA: FILENAME1 TYPE STRING.
FILENAME1 = 'C:/s.txt'.
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
    FILENAME                      = FILENAME1
FILETYPE                      = 'ASC'
   HAS_FIELD_SEPARATOR           = 'X'
HEADER_LENGTH                 = 0
READ_BY_LINE                  = 'X'
DAT_MODE                      = ' '
CODEPAGE                      = ' '
IGNORE_CERR                   = ABAP_TRUE
REPLACEMENT                   = '#'
CHECK_BOM                     = ' '
VIRUS_SCAN_PROFILE            =
NO_AUTH_CHECK                 = ' '
IMPORTING
FILELENGTH                    =
HEADER                        =
TABLES
    DATA_TAB                     = itab.
EXCEPTIONS
FILE_OPEN_ERROR               = 1
FILE_READ_ERROR               = 2
NO_BATCH                      = 3
GUI_REFUSE_FILETRANSFER       = 4
INVALID_TYPE                  = 5
NO_AUTHORITY                  = 6
UNKNOWN_ERROR                 = 7
BAD_DATA_FORMAT               = 8
HEADER_NOT_ALLOWED            = 9
SEPARATOR_NOT_ALLOWED         = 10
HEADER_TOO_LONG               = 11
UNKNOWN_DP_ERROR              = 12
ACCESS_DENIED                 = 13
DP_OUT_OF_MEMORY              = 14
DISK_FULL                     = 15
DP_TIMEOUT                    = 16
OTHERS                        = 17
IF SY-SUBRC <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
IF sy-subrc EQ 0.
zabpsp_01-kunnr = ITAB-IKUNNR.
zabpsp_01-matnr = ITAB-IMATNR.
zabpsp_01-adrnr = ITAB-IADRNR.
WRITE : / ' UPLOAD SUCCESS ' .
ENDIF.
\[subject changed, don't write everything in upper case!\]
Edited by: Jan Stallkamp on Aug 6, 2008 2:39 PM

Hi,
After upload modify the code like below. Also change the file name as some one suggested already. If u are still facing problems then check in debug mode what is happening after FM call.
CALL GUI_UPLOAD FM.
IF sy-subrc EQ 0
IF NOT itab[] IS INITIAL.
MODIFY ZABPSP_01 FROM TABLE itab.
WRITE : / ' UPLOAD SUCCESS ' .
ELSE.
WRITE 'No data in file'.
ENDIF.
ELSE.
WRITE 'Upload failure'.
ENDIF.
Thanks,
Vinod.

Load multiple records in 1 table from 1 line in text file w/sql loader

hi guys,
quick question, perhaps someone can help. searched around and didn't see this question asked before, and can't find any answer in SQL Loader faqs or docs.
i know i can extract multiple logical records from a single physical record in the input file as two logical records, and then use two into tables clauses to load the data into the table. see oracle 9i sql loader control file reference chapter 5 to see what i am talking about.
but my question follows:
cust_id amount1_val amount1_qual amount2_val amount2_qual amount3_val amount3_qual
123 1500.35 TA 230.34 VZ 3045.50 TW
basically i want to use one sql loader statement to load these 3 records into 1 table. the issue for me is that i need to re-use the cust_id for all 3 records as the key, along with the qualifier code. the example in the Oracle docs only works for data where the logical records are completely separate -- no shared column values.
i'm sure this is possible, perhaps using some :cust_id type parameter for the 2nd and 3rd records, or something, but i just don't have enough knowledge/experience with sql loader to know what to do. appreciate any help.
wayne

Hi wayne,
I found an example on what exactly you were looking for from an SQL*Loader documentation. Please see if it of some help to you
EXAMPLE
The control file is ULCASE5.CTL.
1234 BAKER 10 9999 101
1234 JOKER 10 9999 777
2664 YOUNG 20 2893 425
5321 OTOOLE 10 9999 321
2134 FARMER 20 4555 236
2414 LITTLE 20 5634 236
6542 LEE 10 4532 102
2849 EDDS xx 4555
4532 PERKINS 10 9999 40
1244 HUNT 11 3452 665
123 DOOLITTLE 12 9940
1453 MACDONALD 25 5532
In the above datafile
Column1 - Empno
Column2 - ENAME
Column3 - Depno.
Column4 - MGR
Column5 - Proj no.
-- Loads EMP records from first 23 characters
-- Creates and loads PROJ records for each PROJNO listed
-- for each employee
LOAD DATA
INFILE 'ulcase5.dat'
BADFILE 'ulcase5.bad'
DISCARDFILE 'ulcase5.dsc'
1) REPLACE
2) INTO TABLE emp
(empno POSITION(1:4) INTEGER EXTERNAL,
ename POSITION(6:15) CHAR,
deptno POSITION(17:18) CHAR,
mgr POSITION(20:23) INTEGER EXTERNAL)
2) INTO TABLE proj
(empno POSITION(1:4) INTEGER EXTERNAL,
3) projno POSITION(25:27) INTEGER EXTERNAL) -- 1st proj
Notes:
REPLACE specifies that if there is data in the tables to be loaded (EMP and PROJ), SQL*loader should delete the data before loading new rows.
Multiple INTO clauses load two tables, EMP and PROJ. The same set of records is processed three times, using different combinations of columns each time to load table PROJ.
Regards,
Murali Mohan

How to skip the first row in Text file (in Sql Loader)

Hi All,
How to say the control file to skip the first row of the text file..
i just going this example
this is my text file:
01308201222455038130820122245503813082012224550382
1090358 474661834012245503813082012 0075 0 00000000000 000000000 00000130820120000000000000000 136.84 -833.3911082012 000000 000 009035847466183 090358 47466183 0015007514300000970001430000097700BH1150274792012081320120811201208122012-08-11-22.45.50.38369899999.0075LIQJGL17
Control file:
OPTIONS
DIRECT = TRUE ,
PARALLEL = FALSE ,
SKIP = 1,
ERRORS = 0
UNRECOVERABLE
LOAD DATA
INFILE <"FILE_DIR">
BADFILE <"FILE_DIR">
INSERT
INTO TABLE DAILY_TRANSACTION
WHEN (1:1)='D'
TRAILING NULLCOLS
Above the Eaxmple text file i need to Skip the first row which is
01308201222455038130820122245503813082012224550382
please can anyone suggest me is this correct?
if not please correct me please
since two days i been serching but i didn't get any thing
Thank's
Edited by: Lavanya on 24-Sep-2012 00:51

Lavanya wrote:
Hi Jeneesh,
Thank you for your prompt replay
I didn't try to load this control file
And i need one more question
I have Text file in my Machine which Contains very huge data so i was wrote one control file for that, and i got FTP connection also as well as.
My question is firstly I need to PUT the Text file Into Unix Server isn't?
so when i trying to put the text file into unix it's giving an error
ftp>put c:/abc.txt
Not connectedYou haven't succesfully made your FTP connection to the server.
e.g.
c:\>ftp
ftp> o testserver
Connected to testserver.mycompany.com.
220 testserver FTP server ready.
User (testserver.mycompany.com:(none)): myuser
331 Password required for myuser.
Password:
230 User myuser logged in.
ftp>then you can use FTP to put your data to the server. Remember, if you are transferring a text file from a windows based operating system to a unix server you need to put it in ASCII mode...
ftp> ascii
200 Type set to A.
ftp>which will automatically convert the windows CR/LF pairs to the single LF character used by unix.

CSV FILES DOESN'T LOAD WITH RIGHT DATA USING SQL LOADER

Hi pals, I have the following information in csv file:
MEXICO,Seretide_Q110,2010_SEE_01,Sales Line,OBJECTIVE,MEXICO,Q110,11/01/2010,02/04/2010,Activo,,,MEXICO
MEXICO,Seretide_Q210,2010_SEE_02,Sales Line,OBJECTIVE,MEXICO,Q210,05/04/2010,25/06/2010,Activo,,,MEXICO
When I use SQLLOADER the data is loaded as follow:*
EXICO,Seretide_Q110,2010_SEE_01,Sales Line,OBJECTIVE,MEXICO,Q110,11/01/2010,02/04/2010,Activo,,,MEXICO
And for the next data in a csv file too:
MX_001,MEXICO,ASMA,20105912,Not Verified,General,,RH469364,RH469364,Change Request,,,,,,,Y,MEXICO,RH469364
MX_002,MEXICO,ASMA,30094612,Verified,General,,LCS1405,LCS1405,Change Request,,,,,,,Y,MEXICO,LCS1405
the data is loaded as follow:
X_001,MEXICO,ASMA,20105912,Not Verified,General,,RH469364,RH469364,Change Request,,,,,,,Y,MEXICO,RH469364
X_002,MEXICO,ASMA,30094612,Verified,General,,LCS1405,LCS1405,Change Request,,,,,,,Y,MEXICO,LCS1405
I mean the first character is truncated and this bug happens with all my data. Any suggestion? I really hope you can help me.
Edited by: user11260938 on 11/06/2009 02:17 PM
Edited by: Mariots on 12/06/2009 09:37 AM
Edited by: Mariots on 12/06/2009 09:37 AM

Your table and view don't make sense so I created a "dummy" table to match your .ctl file.
SQL> create table CCI_SRC_MX
2 (ORG_BU               varchar2(30)
3 ,name                 varchar2(30)
4 ,src_num              varchar2(30)
5 ,src_cd               varchar2(30)
6 ,sub_type             varchar2(30)
7 ,period_bu            varchar2(30)
8 ,period_name          varchar2(30)
9 ,prog_start_dt        date
10 ,prog_end_dt          date
11 ,status_cd            varchar2(30)
12 ,X_ACTUALS_CALC_DATE date
13 ,X_ACTUAL_UPDATE_SRC varchar2(30)
14 ,prod_bu              varchar2(30)
15 ,ROW_ID               NUMBER(15,0)
16 ,IF_ROW_STAT          VARCHAR2(90)
17 ,JOB_ID               NUMBER(15,0)
18 );
Table created.
SQL> create sequence GSK_GENERAL_SEQ;
Sequence created.I simplified your .ctl file and moved all the constant and sequence stuff to the end. I also changed the format masks to match the dates in your data.
LOAD DATA
INFILE 'SBSLSLT.txt'
BADFILE 'SBSLSLT.bad'
DISCARDFILE 'SBSLSLT.dis'
APPEND
INTO TABLE CCI_SRC_MX
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(ORG_BU
,NAME
,SRC_NUM
,SRC_CD
,SUB_TYPE
,PERIOD_BU
,PERIOD_NAME
,PROG_START_DT          DATE 'dd/mm/yyyy'
,PROG_END_DT            DATE 'dd/mm/yyyy'
,STATUS_CD
,X_ACTUALS_CALC_DATE    DATE 'dd/mm/yyyy'
,X_ACTUAL_UPDATE_SRC
,PROD_BU
,row_id                 "GSK_GENERAL_SEQ.nextval"
,if_row_stat            CONSTANT 'UPLOADED'
,job_id                 constant 36889106
{code}
When I run SQL Loader, I get this:
{code}
SQL> select * from CCI_SRC_MX;
ORG_BU NAME           SRC_NUM      SRC_CD      SUB_TYPE   PERIOD_BU PERIOD_NAME PROG_START_DT        PROG_END_DT          STATUS_CD PROD_BU ROW_ID IF_ROW_STAT    JOB_ID
MEXICO Seretide_Q110 2010_SEE_01 Sales Line OBJECTIVE MEXICO     Q110         11-JAN-2010 00:00:00 02-APR-2010 00:00:00 Activo     MEXICO        1 UPLOADED     36889106
MEXICO Seretide_Q210 2010_SEE_02 Sales Line OBJECTIVE MEXICO     Q210         05-APR-2010 00:00:00 25-JUN-2010 00:00:00 Activo     MEXICO        2 UPLOADED     36889106
{code}

Loading from text file using Sql Loader

Similar Messages

Maybe you are looking for