Using SQL*Loader to load a .csv file having multiple CLOBs

Oracle 8.1.5 on Solaris 2.6
I want to use SQL*Loader to load a .CSV file that has 4 inline CLOB columns. I shall attempt to give some information about the problem:
1. The CLOBs are not delimited in the field level and could themselves contain commas.
2. I cannot get the data file in any other format.
Can anybody help me out with this? While loading LOB in predetermined size fields, is there a limit on the size?
TIA.
-Murali

Thanks for the article link. The article states "...the loader can load only XMLType tables, not columns." Is this still the case with 10g R2? If so, what is the best way to workaround this problem? I am migrating data from a Sybase table that contains a TEXT column (among others) to an Oracle table that contains an XMLType column. How do you recommend I accomplish this task?
- Ron

Similar Messages

SQL loader with *.csv file

Good morning,
I have a *.CVS file containing data like this:
A123456789,Ah Tong Station,Jalan Dungun
I would like insert the data into a table that returns 1 row for each value
id outlet_name addr_1A123456789 Ah Tong Station Jalan Dungun
etc
In the stored procedure, SQL loader, I used
Insert into XXXXX values
(SUBSTR(input_buffer, 1, 10),
SUBSTR(input_buffer, 11, 60),
SUBSTR(input_buffer, 61, 110);
but the problem is that insertion of the outlet_name and addr_1 will not follow the subcripts that provided. Instead in gave an output like that:
id outlet_name addr_1
A123456789 ,Ah Tong Station,Jalan Dungun NULL
Hope to get some advice asap. Thanks. :)
Pauline

Why not use the SQL*Loader proper to load this in to your table? It will be significantly faster than anything you could do in PL/SQL, especially if your volumes increase. And it will take care of the parsing for you, as long as you tell it the field delimiter. You've already seen that you can't rely on fixed widths when using delimited data.
Given this data...
A123456789,Ah Tong Station,Jalan Dungun
A234,Some Station,Some Place...the following SQL*Loader control file ( stored in a file called mytable.ctl ) will load your table ( note I've used truncate - you could also append )...
load data
infile 'mydata.csv'
truncate
into table mytable
fields terminated by ','
( id
, outlet_name
, addr_1
)This is invoked using the following:-
sqlldr userid/password@tns control=mytable.ctlThe results are:-
SQL> select * from mytable;
ID                             OUTLET_NAME                    ADDR_1
A123456789                     Ah Tong Station                Jalan Dungun
A234                           Some Station                   Some PlaceYou will find this much easier. Of course, you could always avoid the database altogether if you don't need to actually store this data. You could just parse the source .csv ( if you are on UNIX this will be really easy using an awk one-liner ) and write it to an output file.
Anyway, hope this helps.
Regards
Adrian

Issue while loading a csv file using sql*loader...

Hi,
I am loading a csv file using sql*loader.
On the number columns where there is data populated in them, decimal number/integers, the row errors out on the error -
ORA-01722: invalid number
I tried checking the value picking from the excel,
and found the chr(13),chr(32),chr(10) values characters on the value.
ex: select length('0.21') from dual is giving a value of 7.
When i checked each character as
select ascii(substr('0.21',5,1) from dual is returning a value 9...etc.
I tried the following command....
"to_number(trim(replace(replace(replace(replace(:std_cost_price_scala,chr(9),''),chr(32),''),chr(13),''),chr(10),'')))",
to remove all the non-number special characters. But still facing the error.
Please let me know, any solution for this error.
Thanks in advance.
Kiran

control file:
OPTIONS (ROWS=1, ERRORS=10000)
LOAD DATA
CHARACTERSET WE8ISO8859P1
INFILE '$Xx_TOP/bin/ITEMS.csv'
APPEND INTO TABLE XXINF.ITEMS_STAGE
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' TRAILING NULLCOLS
ItemNum                    "trim(replace(replace(:ItemNum,chr(9),''),chr(13),''))",
cross_ref_old_item_num               "trim(replace(replace(:cross_ref_old_item_num,chr(9),''),chr(13),''))",
Mas_description               "trim(replace(replace(:Mas_description,chr(9),''),chr(13),''))",
Mas_long_description               "trim(replace(replace(:Mas_long_description,chr(9),''),chr(13),''))",
Org_description               "trim(replace(replace(:Org_description,chr(9),''),chr(13),''))",
Org_long_description               "trim(replace(replace(:Org_long_description,chr(9),''),chr(13),''))",
user_item_type                    "trim(replace(replace(:user_item_type,chr(9),''),chr(13),''))",
organization_code               "trim(replace(replace(:organization_code,chr(9),''),chr(13),''))",
primary_uom_code               "trim(replace(replace(:primary_uom_code,chr(9),''),chr(13),''))",
inv_default_item_status          "trim(replace(replace(:inv_default_item_status,chr(9),''),chr(13),''))",
inventory_item_flag               "trim(replace(replace(:inventory_item_flag,chr(9),''),chr(13),''))",
stock_enabled_flag               "trim(replace(replace(:stock_enabled_flag,chr(9),''),chr(13),''))",
mtl_transactions_enabled_flag          "trim(replace(replace(:mtl_transactions_enabled_flag,chr(9),''),chr(13),''))",
revision_qty_control_code          "trim(replace(replace(:revision_qty_control_code,chr(9),''),chr(13),''))",
reservable_type               "trim(replace(replace(:reservable_type,chr(9),''),chr(13),''))",
check_shortages_flag               "trim(replace(replace(:check_shortages_flag,chr(9),''),chr(13),''))",
shelf_life_code               "trim(replace(replace(replace(replace(:shelf_life_code,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
shelf_life_days               "trim(replace(replace(replace(replace(:shelf_life_days,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
lot_control_code               "trim(replace(replace(:lot_control_code,chr(9),''),chr(13),''))",
auto_lot_alpha_prefix               "trim(replace(replace(:auto_lot_alpha_prefix,chr(9),''),chr(13),''))",
start_auto_lot_number               "trim(replace(replace(:start_auto_lot_number,chr(9),''),chr(13),''))",
negative_measurement_error          "trim(replace(replace(replace(replace(:negative_measurement_error,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
positive_measurement_error          "trim(replace(replace(replace(replace(:positive_measurement_error,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
serial_number_control_code          "trim(replace(replace(:serial_number_control_code,chr(9),''),chr(13),''))",
auto_serial_alpha_prefix          "trim(replace(replace(:auto_serial_alpha_prefix,chr(9),''),chr(13),''))",
start_auto_serial_number          "trim(replace(replace(:start_auto_serial_number,chr(9),''),chr(13),''))",
location_control_code               "trim(replace(replace(:location_control_code,chr(9),''),chr(13),''))",
restrict_subinventories_code          "trim(replace(replace(:restrict_subinventories_code,chr(9),''),chr(13),''))",
restrict_locators_code               "trim(replace(replace(:restrict_locators_code,chr(9),''),chr(13),''))",
bom_enabled_flag               "trim(replace(replace(:bom_enabled_flag,chr(9),''),chr(13),''))",
costing_enabled_flag               "trim(replace(replace(:costing_enabled_flag,chr(9),''),chr(13),''))",
inventory_asset_flag               "trim(replace(replace(:inventory_asset_flag,chr(9),''),chr(13),''))",
default_include_in_rollup_flag          "trim(replace(replace(:default_include_in_rollup_flag,chr(9),''),chr(13),''))",
cost_of_goods_sold_account          "trim(replace(replace(:cost_of_goods_sold_account,chr(9),''),chr(13),''))",
std_lot_size                    "trim(replace(replace(replace(replace(:std_lot_size,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
sales_account                    "trim(replace(replace(:sales_account,chr(9),''),chr(13),''))",
purchasing_item_flag               "trim(replace(replace(:purchasing_item_flag,chr(9),''),chr(13),''))",
purchasing_enabled_flag          "trim(replace(replace(:purchasing_enabled_flag,chr(9),''),chr(13),''))",
must_use_approved_vendor_flag          "trim(replace(replace(:must_use_approved_vendor_flag,chr(9),''),chr(13),''))",
allow_item_desc_update_flag          "trim(replace(replace(:allow_item_desc_update_flag,chr(9),''),chr(13),''))",
rfq_required_flag               "trim(replace(replace(:rfq_required_flag,chr(9),''),chr(13),''))",
buyer_name                    "trim(replace(replace(:buyer_name,chr(9),''),chr(13),''))",
list_price_per_unit               "trim(replace(replace(replace(replace(:list_price_per_unit,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
taxable_flag                    "trim(replace(replace(:taxable_flag,chr(9),''),chr(13),''))",
purchasing_tax_code               "trim(replace(replace(:purchasing_tax_code,chr(9),''),chr(13),''))",
receipt_required_flag               "trim(replace(replace(:receipt_required_flag,chr(9),''),chr(13),''))",
inspection_required_flag          "trim(replace(replace(:inspection_required_flag,chr(9),''),chr(13),''))",
price_tolerance_percent          "trim(replace(replace(replace(replace(:price_tolerance_percent,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
expense_account               "trim(replace(replace(:expense_account,chr(9),''),chr(13),''))",
allow_substitute_receipts_flag          "trim(replace(replace(:allow_substitute_receipts_flag,chr(9),''),chr(13),''))",
allow_unordered_receipts_flag          "trim(replace(replace(:allow_unordered_receipts_flag,chr(9),''),chr(13),''))",
receiving_routing_code               "trim(replace(replace(:receiving_routing_code,chr(9),''),chr(13),''))",
inventory_planning_code          "trim(replace(replace(:inventory_planning_code,chr(9),''),chr(13),''))",
min_minmax_quantity               "trim(replace(replace(replace(replace(:min_minmax_quantity,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
max_minmax_quantity               "trim(replace(replace(replace(replace(:max_minmax_quantity,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
planning_make_buy_code               "trim(replace(replace(:planning_make_buy_code,chr(9),''),chr(13),''))",
source_type                    "trim(replace(replace(:source_type,chr(9),''),chr(13),''))",
mrp_safety_stock_code               "trim(replace(replace(:mrp_safety_stock_code,chr(9),''),chr(13),''))",
material_cost                    "trim(replace(replace(:material_cost,chr(9),''),chr(13),''))",
mrp_planning_code               "trim(replace(replace(:mrp_planning_code,chr(9),''),chr(13),''))",
customer_order_enabled_flag          "trim(replace(replace(:customer_order_enabled_flag,chr(9),''),chr(13),''))",
customer_order_flag               "trim(replace(replace(:customer_order_flag,chr(9),''),chr(13),''))",
shippable_item_flag               "trim(replace(replace(:shippable_item_flag,chr(9),''),chr(13),''))",
internal_order_flag               "trim(replace(replace(:internal_order_flag,chr(9),''),chr(13),''))",
internal_order_enabled_flag          "trim(replace(replace(:internal_order_enabled_flag,chr(9),''),chr(13),''))",
invoice_enabled_flag               "trim(replace(replace(:invoice_enabled_flag,chr(9),''),chr(13),''))",
invoiceable_item_flag               "trim(replace(replace(:invoiceable_item_flag,chr(9),''),chr(13),''))",
cross_ref_ean_code               "trim(replace(replace(:cross_ref_ean_code,chr(9),''),chr(13),''))",
category_set_intrastat               "trim(replace(replace(:category_set_intrastat,chr(9),''),chr(13),''))",
CustomCode                    "trim(replace(replace(:CustomCode,chr(9),''),chr(13),''))",
net_weight                    "trim(replace(replace(replace(replace(:net_weight,chr(9),''),chr(13),''),chr(32),''),chr(10),''))",
production_speed               "trim(replace(replace(:production_speed,chr(9),''),chr(13),''))",
LABEL                         "trim(replace(replace(:LABEL,chr(9),''),chr(13),''))",
comment1_org_level               "trim(replace(replace(:comment1_org_level,chr(9),''),chr(13),''))",
comment2_org_level               "trim(replace(replace(:comment2_org_level,chr(9),''),chr(13),''))",
std_cost_price_scala               "to_number(trim(replace(replace(replace(replace(:std_cost_price_scala,chr(9),''),chr(32),''),chr(13),''),chr(10),'')))",
supply_type                    "trim(replace(replace(:supply_type,chr(9),''),chr(13),''))",
subinventory_code               "trim(replace(replace(:subinventory_code,chr(9),''),chr(13),''))",
preprocessing_lead_time          "trim(replace(replace(replace(replace(:preprocessing_lead_time,chr(9),''),chr(32),''),chr(13),''),chr(10),''))",
processing_lead_time                "trim(replace(replace(replace(replace(:processing_lead_time,chr(9),''),chr(32),''),chr(13),''),chr(10),''))",
wip_supply_locator               "trim(replace(replace(:wip_supply_locator,chr(9),''),chr(13),''))"
Sample data from csv file.
"9901-0001-35","390000","JMKL16 Pipe bend 16 mm","","JMKL16 Putkikaari 16 mm","","AI","FJE","Ea","","","","","","","","","","","","","","","","","","","","","","","","","21-21100-22200-00000-00000-00-00000-00000","0","21-11100-22110-00000-00000-00-00000-00000","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","0.1","Pull","AFTER PROD","","","Locator for Production"
The load errors out on especially two columns :
1) std_cost_price_scala
2) list_price_per_unit
both are number columns.
And when there is data being provided on them. It errors out. But, if they are holding null values, the records go through fine.
Message was edited by:
KK28

How to Load Arabic Data from flat file using SQL Loader ?

Hi All,
We need to load Arabic data from an xls file to Oracle database, Request you to provide a very good note/step to achieve the same.
Below are the database parameters used
NLS_CHARACTERSET AR8ISO8859P6
nls_language american
DB version:-10g release 2
OS: rhel 5
Thanks in advance,
Satish

Try to save your XLS file into CSV format and set either NLS_LANG to the right value or use SQL*Loader control file parameter CHARACTERSET.
See http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_control_file.htm#i1005287

Loading a CSV file to Sql Server

Hi,
How can i load a csv file, which is in a shared location itn osql using bods.
Where can i create the data source pointing to csv file

Hi,
Use fileformat to load CSV file into SQL server, below the details
A file format defines a connection to a file. Therefore, you use a file format to connect to source or target data when the data is stored in a file rather than a database table. The object library stores file format templates that you use to define specific file formats as sources and targets in data flows.
To work with file formats, perform the following tasks:
• Create a file format template that defines the structure for a file.
• Create a specific source or target file format in a data flow. The source or target file format is based on a template and specifies connection information such as the file name.
File format objects can describe files of the following types:
• Delimited: Characters such as commas or tabs separate each field.
• Fixed width: You specify the column width.
• SAP transport: Use to define data transport objects in SAP application data flows.
• Unstructured text: Use to read one or more files of unstructured text from a directory.
• Unstructured binary: Use to read one or more binary documents from a directory.
More details, refer the designer guide
http://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_designer_en.pdf

Load XML File into temporary tables using sql loader

Hi All,
I have an XML file as below. I need to insert the contents into a temporary staging table using sql loader. Please advice how I need to do that.
For example Portfolios should go into a seperate table, and all the tags inside it should be populated in the columns of the table.
Family should go into a seperate table and all the tags inside it should be populated in the columns of the table.
Similarly offer, Products etc.
- <ABSProductCatalog xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
- <ProductSalesHierachy>
- <Portfolios>
- <Portfolio productCode="P1">
<Attribute name="CatalogProductName" value="Access" />
<Attribute name="Status" value="Active" />
</Portfolio>
- <Portfolio productCode="P2">
<Attribute name="CatalogProductName" value="Data" />
<Attribute name="Status" value="Active" />
</Portfolio>
- <Portfolio productCode="P3">
<Attribute name="CatalogProductName" value="Voice" />
<Attribute name="Status" value="Active" />
</Portfolio>
- <Portfolio productCode="P4">
<Attribute name="CatalogProductName" value="Wireless" />
<Attribute name="Status" value="Active" />
</Portfolio>
</Portfolios>
- <Families>
- <Family productCode="F1">
<Attribute name="CatalogProductName" value="Internet Access Services" />
<Attribute name="Status" value="Active" />
- <ParentHierarchy>
<Item productCode="P1" modelType="Portfolio" />
</ParentHierarchy>
</Family>
- <Family productCode="F2">
<Attribute name="CatalogProductName" value="Local Access Services" />
<Attribute name="Status" value="Active" />
- <ParentHierarchy>
<Item productCode="P2" modelType="Portfolio" />
</ParentHierarchy>
</Family>
</Families>
- <SubFamilies>
- <SubFamily productCode="SF1">
<Attribute name="CatalogProductName" value="Business Internet service" />
<Attribute name="Status" value="Active" />
- <ParentHierarchy>
<Item productCode="F1" modelType="Family" />
</ParentHierarchy>
</SubFamily>
</SubFamilies>
- <ProductRefs>
- <ProductRef productCode="WSP1" modelType="Wireline Sales Product">
<ActiveFlag>Y</ActiveFlag>
- <ProductHierarchy>
<SalesHierarchy family="F1" subFamily="SF1" portfolio="P1" primary="Y" />
<SalesHierarchy family="F2" portfolio="P2" primary="N" />
<FinancialHierarchy quotaBucket="Voice" strategicProdCategory="Local Voice" />
</ProductHierarchy>
</ProductRef>
- <ProductRef productCode="MSP2" modelType="Handset">
<ActiveFlag>Y</ActiveFlag>
- <ProductHierarchy>
<SalesHierarchy portfolio="P4" primary="Y" />
</ProductHierarchy>
</ProductRef>
</ProductRefs>
</ProductSalesHierachy>
- <Offers>
- <Offer productCode="ABN">
<OfferName>ABN</OfferName>
<OfferDescription>ABN Description</OfferDescription>
- <Segments>
<Segment>SCG</Segment>
<Segment>PCG</Segment>
</Segments>
<OfferUpdateDate>2009-11-20</OfferUpdateDate>
<ActiveFlag>Y</ActiveFlag>
</Offer>
- <Offer productCode="OneNet">
<OfferName>OneNet</OfferName>
<OfferDescription>OneNet Description</OfferDescription>
- <Segments>
<Segment>SCG</Segment>
<Segment>PCG</Segment>
<Segment>PCG2</Segment>
</Segments>
<OfferUpdateDate>2009-11-20</OfferUpdateDate>
<ActiveFlag>Y</ActiveFlag>
</Offer>
</Offers>
- <Products>
- <Product productCode="WSP1" modelType="Wireline Sales Product">
<ProductName>AT&T High Speed Internet</ProductName>
<ProductDescription>High Speed Internet</ProductDescription>
<LegacyCoProdIndicator>SBC</LegacyCoProdIndicator>
<RevenueCBLCode>1234B</RevenueCBLCode>
<VolumeCBLCode>4567A</VolumeCBLCode>
<SAARTServiceIDCode>S1234</SAARTServiceIDCode>
<MarginPercentRequired>Y</MarginPercentRequired>
<PercentIntl>%234</PercentIntl>
<UOM>Each</UOM>
<PriceType>OneTime</PriceType>
<ProductStatus>Active</ProductStatus>
<Compensable>Y</Compensable>
<Jurisdiction>Everywhere</Jurisdiction>
<ActiveFlag>Y</ActiveFlag>
- <Availabilities>
<Availability>SE</Availability>
<Availability>E</Availability>
</Availabilities>
- <Segments>
<Segment>SCG</Segment>
<Segment>PCG</Segment>
</Segments>
<VDIndicator>Voice</VDIndicator>
<PSOCCode>PSOC 1</PSOCCode>
<USBilled>Y</USBilled>
<MOWBilled>N</MOWBilled>
<ProductStartDate>2009-11-20</ProductStartDate>
<ProductUpdateDate>2009-11-20</ProductUpdateDate>
<ProductEndDate>2010-11-20</ProductEndDate>
- <AliasNames>
<AliasName>AT&T HSI</AliasName>
<AliasName>AT&T Fast Internet</AliasName>
</AliasNames>
- <OfferTypes>
<OfferType productCode="ABN" endDate="2009-11-20" />
<OfferType productCode="OneNet" />
</OfferTypes>
- <DynamicAttributes>
- <DynamicAttribute dataType="String" defaultValue="2.5 Mbps" name="Speed">
<AttrValue>1.5 Mbps</AttrValue>
<AttrValue>2.5 Mbps</AttrValue>
<AttrValue>3.5 Mbps</AttrValue>
</DynamicAttribute>
- <DynamicAttribute dataType="String" name="TransportType">
<AttrValue>T1</AttrValue>
</DynamicAttribute>
</DynamicAttributes>
</Product>
- <Product productCode="MSP2" modelType="Handset">
<ProductName>Blackberry Bold</ProductName>
<ProductDescription>Blackberry Bold Phone</ProductDescription>
<LegacyCoProdIndicator />
<RevenueCBLCode />
<VolumeCBLCode />
<SAARTServiceIDCode />
<MarginPercentRequired />
<PercentIntl />
<UOM>Each</UOM>
<PriceType />
<ProductStatus>Active</ProductStatus>
<Compensable />
<Jurisdiction />
<ActiveFlag>Y</ActiveFlag>
- <Availabilities>
<Availability />
</Availabilities>
- <Segments>
<Segment>SCG</Segment>
<Segment>PCG</Segment>
</Segments>
<VDIndicator>Voice</VDIndicator>
<PSOCCode />
<USBilled />
<MOWBilled />
<ProductStartDate>2009-11-20</ProductStartDate>
<ProductUpdateDate>2009-11-20</ProductUpdateDate>
- <AliasNames>
<AliasName />
</AliasNames>
- <OfferTypes>
<OfferType productCode="ABN" />
</OfferTypes>
- <DynamicAttributes>
- <DynamicAttribute dataType="String" name="StlmntContractType">
<AttrValue />
</DynamicAttribute>
- <DynamicAttribute dataType="String" name="BMG 2 year price">
<AttrValue>20</AttrValue>
</DynamicAttribute>
- <DynamicAttribute dataType="String" name="MSRP">
<AttrValue>40</AttrValue>
</DynamicAttribute>
- <DynamicAttribute dataType="String" name="BMGAvailableType">
<AttrValue />
</DynamicAttribute>
- <DynamicAttribute dataType="String" name="ProductId">
<AttrValue>123456</AttrValue>
</DynamicAttribute>
- <DynamicAttribute dataType="String" name="modelSource">
<AttrValue>product</AttrValue>
</DynamicAttribute>
</DynamicAttributes>
</Product>
</Products>
<CatalogChanged>Y</CatalogChanged>
</ABSProductCatalog>

Two options that come to mind. Others exist.
#1 - {thread:id=474031}, which is basically storing the XML in an Object Relational structure for parsing
#2 - Dump the XML into either an XMLType based table or column and use SQL (with XMLTable) to create a view that parses the data. This would be the same as the view shown in the above post.
Don't use sql*loader to parse the XML. I was trying to find a post from mdrake about that but couldn't. In short, sql*loader was not build as an XML parser so don't try to use it that way.

How to Load Multiple Files in Oracle Database using Sql Loader

Hi All,
I want to import multiple files in my DB using Sql*Loader. Please tell me the Syntax, how can I import multiple files using one Control File?
Thanks & Regards,
Imran

Hi,
You might get a good response to your post in the forum dedicated to data movement , including SQL*Loader . You can find it here Export/Import/SQL Loader & External Tables
Regards,

Loading XML file using sql*loader (10g)

Hi Everyone....
I have a major problem. I have a 10g database and I need to use sql loader to load a large XML file into the database. This file contains records for many, many customers. This will be done several times and the number of records will vary. I want to use sql loader and load to a staging table, BUT SPLIT THE RECORDS OUT and add a sequence. I am having 2 problems.
In the 10g documentation, when you want to split the records you use the BEGINDATA clause and indicate something (like a 0) for each instance of a record. Well my first file has 3,722 records in it. How do I know the number of records in each XML file?
The second problem is that because this is XML, I thought that I could use ENCLOSED BY. But the start tag has an attribute /counter in it, sql*loader doesnt recognize the starting tag. (i.e.: start tag is: </CustomerRec number=1>
end tag is: </CustomerRec> )
So, I used TERMINATED BY '</CustomerRec>'. This will split out the records, but it will NOT include the ending tag of </CustomerRec> and when I use extract, I receive an XML parsing error.
I have tried to get the ending tag using CONTINUEIF and SKIP. But those options are for "records" and not lines.
Does anyone have any ideas for the 2 problems above. I am at my wits end and I need to finish this ASAP. Any help would be appreciated.
Thank you!

Sorry.... here is an example of what my control file looks like. At the end, I have 92 "0", but remember, I have 3722 records in this first file.
LOAD DATA (SKIP 1)
INFILE *
INTO TABLE RETURN_DATA_STG
TRUNCATE
XMLType(xmldata)
FIELDS
(fill FILLER CHAR(1),
loadseq SEQUENCE(MAX,1),
xmldata LOBFILE (CONSTANT F001AB.XML)
TERMINATED BY '</ExtractObject>'
------ ":xmldata||'</ExtractObject>'"
BEGINDATA
0
0
0
0
0
0

Using SQL*Loader and UTL_FILE to load and unload large files(i.e PDF,DOCs)

Problem : Load PDF or similiar files( stored at operating system) into an oracle table using SQl*Loader .
and than Unload the files back from oracle tables to prevoius format.
I 've used SQL*LOADER .... " sqlldr " command as :
" sqlldr scott/[email protected] control=c:\sqlldr\control.ctl log=c:\any.txt "
Control file is written as :
LOAD DATA
INFILE 'c:\sqlldr\r_sqlldr.txt'
REPLACE
INTO table r_sqlldr
Fields terminated by ','
id sequence (max,1) ,
fname char(20),
data LOBFILE(fname) terminated by EOF )
It loads files ( Pdf, Image and more...) that are mentioned in file r_sqlldr.txt into oracle table r_sqlldr
Text file ( used as source ) is written as :
c:\kalam.pdf,
c:\CTSlogo1.bmp
c:\any1.txt
after this load ....i used UTL_FILE to unload data and write procedure like ...
CREATE OR REPLACE PROCEDURE R_UTL AS
l_file UTL_FILE.FILE_TYPE;
l_buffer RAW(32767);
l_amount BINARY_INTEGER ;
l_pos INTEGER := 1;
l_blob BLOB;
l_blob_len INTEGER;
BEGIN
SELECT data
INTO l_blob
FROM r_sqlldr
where id= 1;
l_blob_len := DBMS_LOB.GETLENGTH(l_blob);
DBMS_OUTPUT.PUT_LINE('blob length : ' || l_blob_len);
IF (l_blob_len < 32767) THEN
l_amount :=l_blob_len;
ELSE
l_amount := 32767;
END IF;
DBMS_LOB.OPEN(l_blob, DBMS_LOB.LOB_READONLY);
l_file := UTL_FILE.FOPEN('DBDIR1','Kalam_out.pdf','w', 32767);
DBMS_OUTPUT.PUT_LINE('File opened');
WHILE l_pos < l_blob_len LOOP
DBMS_LOB.READ (l_blob, l_amount, l_pos, l_buffer);
DBMS_OUTPUT.PUT_LINE('Blob read');
l_pos := l_pos + l_amount;
UTL_FILE.PUT_RAW(l_file, l_buffer, TRUE);
DBMS_OUTPUT.PUT_LINE('writing to file');
UTL_FILE.FFLUSH(l_file);
UTL_FILE.NEW_LINE(l_file);
END LOOP;
UTL_FILE.FFLUSH(l_file);
UTL_FILE.FCLOSE(l_file);
DBMS_OUTPUT.PUT_LINE('File closed');
DBMS_LOB.CLOSE(l_blob);
EXCEPTION
WHEN OTHERS THEN
IF UTL_FILE.IS_OPEN(l_file) THEN
UTL_FILE.FCLOSE(l_file);
END IF;
DBMS_OUTPUT.PUT_LINE('Its working at last');
END R_UTL;
This loads data from r_sqlldr table (BOLBS) to files on operating system ,,,
-> Same procedure with minor changes is used to unload other similar files like Images and text files.
In above example : Loading : 3 files 1) Kalam.pdf 2) CTSlogo1.bmp 3) any1.txt are loaded into oracle table r_sqlldr 's 3 rows respectively.
file names into fname column and corresponding data into data ( BLOB) column.
Unload : And than these files are loaded back into their previous format to operating system using UTL_FILE feature of oracle.
so PROBLEM IS : Actual capacity (size ) of these files is getting unloaded back but with quality decreased. And PDF file doesnt even view its data. means size is almot equal to source file but data are lost when i open it.....
and for images .... imgaes are getting loaded an unloaded but with colors changed ....
Also features ( like FFLUSH ) of Oracle 've been used but it never worked
ANY SUGGESTIONS OR aLTERNATE SOLUTION TO LOAD AND UNLOAD PDFs through Oracle ARE REQUESTED.
------------------------------------------------------------------------------------------------------------------------

Thanks Justin ...for a quick response ...
well ... i am loading data into BLOB only and using SQL*Loader ...
I've never used dbms_lob.loadFromFile to do the loads ...
i 've opend a file on network and than used dbms_lob.read and
UTL_FILE.PUT_RAW to read and write data into target file.
actually ...my process is working fine with text files but not with PDF and IMAGES ...
and your doubt of ..."Is the data the proper length after reading it in?" ..m not getting wat r you asking ...but ... i think regarding data length ..there is no problem... except ... source PDF length is 90.4 kb ..and Target is 90.8 kb..
thats it...
So Request u to add some more help ......or should i provide some more details ??

Problem loading XML-file using SQL*Loader

Hello,
I'm using 9.2 and tryin to load a XML-file using SQL*Loader.
Loader control-file:
LOAD DATA
INFILE *
INTO TABLE BATCH_TABLE TRUNCATE
FIELDS TERMINATED BY ','
FILENAME char(255),
XML_DATA LOBFILE (FILENAME) TERMINATED BY EOF
BEGINDATA
data.xml
The BATCH_TABLE is created as:
CREATE TABLE BATCH_TABLE (
FILENAME VARCHAR2 (50),
XML_DATA SYS.XMLTYPE ) ;
And the data.xml contains the following lines:
<?xml version="2.0" encoding="UTF-8"?>
<!DOCTYPE databatch SYSTEM "databatch.dtd">
<batch>
<record>
<data>
<type>10</type>
</data>
</record>
<record>
<data>
<type>20</type>
</data>
</record>
</batch>
However, the sqlldr gives me an error:
Record 1: Rejected - Error on table BATCH_TABLE, column XML_DATA.
ORA-21700: object does not exist or is marked for delete
ORA-06512: at "SYS.XMLTYPE", line 0
ORA-06512: at line 1
If I remove the first two lines
"<?xml version="2.0" encoding="UTF-8"?>"
and
"<!DOCTYPE databatch SYSTEM "databatch.dtd">"
from data.xml everything works, and the contentents of data.xml are loaded into the table.
Any idea what I'm missing here? Likely the problem is with special characters.
Thanks in advance,

I'm able to load your file just by removing the second line <!DOCTYPE databatch SYSTEM "databatch.dtd">. I dont have your dtd file, so skipped that line. Can you check if it's problem with ur DTD?

Loading from text file using Sql Loader

I need to load data from a text file into Oracle table. The file has long strings of text in the following format:
12342||||||Lots and lots of text with all kinds of characters including
^&*!#%#^@ etc.xxxxxxxxxxxxxxxxxxxxxxx
yyyyyyyyyyyyyyyyyyyyyyyyytrrrrrrrrrrrrrrrrrrr
uuuuuuuuuuuuuuuuuuurtgggggggggggggggg.||||||||
45356|||||||||||again lots and lots of text.uuuuuudccccccccccccccccccccd
gyhjjjjjjjjjjjjjjjjjjjjjjjjkkkkkkkkkkkkklllllllllllnmmmmmmmmmmmmnaaa|||||||.
There are pipes within the text as well. On the above example, the line starting with 12342 is an entire record that needs to be loaded into a CLOB column. The next record would be the 45356 one. Therefore, all records have a bunch of pipes at the end, so the only way to know where a new record starts is to see where the next number is after all the ending pipes. The only other thing I know is that there are a fixed number of pipes in each record.
Does anyone have any ideas on how I can load the data into the table either using sql loader or any other utility? Any input would be greatly appreciated. Thanks.

STFF [url http://forums.oracle.com/forums/thread.jspa?messageID=1773678&#1764219]Sqlldr processing of records with embedded newline and delimiter

Need suggestions on loading 5000+ files using sql loader

Hi Guys,
I'm writing a shell script to load more than 5000 files using sql loader.
My intention is to load the files in parallel. When I checked the maximum number of sessions in v$parameter, it is around 700.
Before starting the data load, programmatically I am getting the number of current sessions and maximum number of sessions and keeping free 200 sessions without using them (max. no. of sessions minus 200 ) and utilizing the remaining ~300 sessions to load the files in parallel.
Also I am using a "wait" option to make the shell to wait until the 300 concurrent sql loader process to complete and moving further.
Is there any way to make it more efficient? Also is it possible to reduce the wait time without hard coding the seconds (For Example: If any of those 300 sessions becomes free, assign the next file to the job queue and so on..)
Please share your thoughts on this.
Thanks.

Manohar wrote:
I'm writing a shell script to load more than 5000 files using sql loader.
My intention is to load the files in parallel. When I checked the maximum number of sessions in v$parameter, it is around 700. Concurrent load you mean? Parallel processing implies take a workload, breaking that up into smaller workloads, and doing that in parallel. This is what the Parallel Query feature does in Oracle.
SQL*Loader does not do that for you. It uses a single session to load a single file. To make it run in parallel, requires manually starting multiple loader sessions and perform concurrent loads.
Have a look at Parallel Data Loading Models in the Oracle® Database Utilities guide. It goes into detail on how to perform concurrent loads. But you need to parallelise that workload yourself (as explained in the manual).
Before starting the data load, programmatically I am getting the number of current sessions and maximum number of sessions and keeping free 200 sessions without using them (max. no. of sessions minus 200 ) and utilizing the remaining ~300 sessions to load the files in parallel.
Also I am using a "wait" option to make the shell to wait until the 300 concurrent sql loader process to complete and moving further.
Is there any way to make it more efficient? Also is it possible to reduce the wait time without hard coding the seconds (For Example: If any of those 300 sessions becomes free, assign the next file to the job queue and so on..)Consider doing it the way that Parallel Query does (as I've mentioned above). Take the workload (all files). Break the workload up into smaller sub-workloads (e.g. 50 files to be loaded by a process). Start a 100 processes in parallel and provide each one with a sub-workload to do (100 processes each loading 50 odd files).
This is a lot easier to manage than starting for example a 5000 load processes and then trying some kind of delay method to ensure that not all hit the database at the same time.
I'm loading about 100+ files (3+ million rows) every 60 seconds 24x7 using SQL*Loader. Oracle is quite scalable and SQL*Loader quite capable.

Error loading local CSV file into external table

Hello,
I am trying to load a CSV file located on my C:\ drive on a WIndows XP system into an 'external table'. Everyting used to work correctly when using Oracle XE (iinstalled also locally on my WIndows system).
However, once I am trynig to load the same file into a Oracle 11g R2 database on UNIX, I get the following errr:
ORA-29913: Error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
error opening file ...\...\...nnnnn.log
Please let me know if I can achieve the same functionality I had with Oracle XP.
(Note: I cannot use SQL*Loader approach, I am invoking Oracle stored procedures from VB.Net that are attempting to load data into external tables).
Regards,
M.R.

user7047382 wrote:
Hello,
I am trying to load a CSV file located on my C:\ drive on a WIndows XP system into an 'external table'. Everyting used to work correctly when using Oracle XE (iinstalled also locally on my WIndows system).
However, once I am trynig to load the same file into a Oracle 11g R2 database on UNIX, I get the following errr:
ORA-29913: Error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
error opening file ...\...\...nnnnn.log
Please let me know if I can achieve the same functionality I had with Oracle XP.
(Note: I cannot use SQL*Loader approach, I am invoking Oracle stored procedures from VB.Net that are attempting to load data into external tables).
Regards,
M.R.So your database is on a unix box, but the file is on your Windows desktop machine? So .... how is it you are making your file (on your desktop) visible to your database (on a unix box)???????

How to load a default value in to a column when using sql loader

Im trying to load from a flat file using sql loader.
for 1 column i need to update using a default value
how to go about this?

Hi!
try this code --
LOAD DATA
   INFILE 'sample.dat'
   REPLACE
   INTO TABLE emp
   empno   POSITION(01:04) INTEGER EXTERNAL NULLIF empno=BLANKS,
   ename   POSITION(06:15) CHAR,
   job         POSITION(17:25) CHAR,
   mgr       POSITION(27:30) INTEGER EXTERNAL NULLIF mgr=BLANKS,
   sal        POSITION(32:39) DECIMAL EXTERNAL NULLIF sal=BLANKS,
   comm   POSITION(41:48) DECIMAL EXTERNAL DEFAULTIF comm = 100,
   deptno POSITION(50:51) INTEGER EXTERNAL NULLIF deptno=BLANKS,
   hiredate POSITION(52:62) CONSTANT SYSDATE
)-hope this will solve ur purpose.
Regards.
Satyaki De.

Error in loading data using SQL loader

I am getting a error like ‘SQL*Loader -350 syntax error of illegal combination of non-alphanumeric characters’ while loading a file using SQL loader in RHEL. The command used to run SQL*Loader is:
Sqlldr userid=<username>/<password> control =data.ctl
The control file, data.ctl is :
LOAD data
infile '/home/oraprod/data.txt'
append into table test
empid terminated by ',',
fname terminated by ',',
lname terminated by ',',
salary terminated by whitespace
The data.txt file is:
1,Kaushal,halani,5000
2,Chetan,halani,1000
I hope, my question is clear.
Please revert with the reply to my query.
Regards

Replace ''{" by "(" in your control file
LOAD data
infile 'c:\data.txt'
append into table emp_t
empid terminated by ',',
fname terminated by ',',
lname terminated by ',',
salary terminated by whitespace
C:\>sqlldr user/pwd@database control=c.ctl
SQL*Loader: Release 10.2.0.3.0 - Production on Wed Nov 13 10:10:24 2013
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Commit point reached - logical record count 1
Commit point reached - logical record count 2
SQL> select * from emp_t;
     EMPID FNAME                LNAME                    SALARY
         1 Kaushal              halani                     5000
         2 Chetan               halani                     1000
Best regards
Mohamed Houri

Using SQL*Loader to load a .csv file having multiple CLOBs

Similar Messages

Maybe you are looking for