CSV to Oracle Table

Hi
In BLT I am FTPing a CSV file with approx 7000 lines, then loading it with "TextLoader" and parsing with "FlatFileParser" then stepping through with a repeater running an SQL insert.
It all works OK, but takes about half an hour, I was wondering if anyone had any better suggestions?
Thanks
Nick
PS Here is a sample of the CSV file
"Ident","Make","Model","Description","Serial","LastCal","NextCal","DueDate","StatusChanged","CalStatus","Live","Job"
"xxx1706","AGILENT","34401A","DIGITAL MULTIMETER","US3612","2003-09-12","2003-09-12","2003-09-14","N","ok","N","10410"
"yyy1724","AGILENT","8482H","POWER SENSOR","MY4109","2001-04-18","2002-03-18","2009-03-21","N","so-so","N","1048"
My repeater strips out the header row ([LineNumber > 1]), and my insert is just
INSERT into mytable VALUES ([Param.1])
The only other thing I have to do is a string replace on each line as follows
stringreplace( stringreplace( stringreplace( Repeater_0.Output{/Row/Line} , "'" , "" ) , "\"" , "'" ) , "'None'" , "''" )
to first of all strip out single quotes (found in the description field), then replace double quotes with single quotes, then replace empty date fields (in the csv file as 'None') with an empty string.

Nick:
I would use SQL*Loader also. Info at:
http://www.orafaq.com/faqloadr.htm
Any time you are doing "bulk inserts", there are special DB techniques that can avoid substantial overhead (e.g. don't commit each row, grab a connection once, etc.). This automatically happens with SQL*Loader.
Alternatively, you might be able to run some PL/SQL with the "LOAD DATA" syntax (and specifying your CSV file with INFILE). Google for examples. Also, there are many performance options that can make a HUGE difference in load speed.
You could theoretically write a custom BLS action to do this also.
Rick

Similar Messages

Conversion of csv to oracle tables in oracle 8i

hello friends,
i have a csv file ready with me which i have created by preprocessing a text file,,,
Now i want o
load this csv file into precreated oracle tables or tables created on the fly anyway's...
suggest something that works on oracle 8i as well
please do the needful...
hoping for a quick reply

SQL*Loader is available in 8i too
Try the example as demonstrated by Paul M in your other thread here
conversion of csv in flat file to oracle table format
For more information about SQL*Loader see the documentation here
http://download-west.oracle.com/docs/cd/A87860_01/doc/server.817/a76955/part2.htm#435958

" (quotes) is column values when loading data from csv to oracle table

I am loading a data from csv file to an oracle table.
After the data is loaded , lets suppose the value in one column is X.
when i write a query to fetch data like
select * from table where col='X' then it gives no output.
On investigating it was found that the value is stored in col as "X".
This also happens when i copy paste the value from column to some text editor ....
i want to remove these double quotes , and also want to know why these are coming...
Any suggestions guys ?
Thanks
Using Oracle 10g

These quotes are part of your data file. Most CSV-parsers remove those quotes.
If you are using an external table you can remove them by something like this for your access parameters
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER
ACCESS PARAMETERS
    RECORDS DELIMITED BY NEWLINE
    FIELDS TERMINATED BY ','
    OPTIONALLY ENCLOSED BY '"'
But if you want to remove them from your existing data use this
update your_table
set col1 = trim( '"' from col1 )
Note that in some CSV-files wich use an enclosing character the enclosing character is itself is used as an escape character to use the enclosing character inside a field.
For example a field might be "Some text with ""this"" enclosed"
To correct these fields you might use another update
update your_table
set col1 = replace( col1, '""', '"' )

Conversion of flat files in csv to oracle table format:"error encountered"

hello friends
thanks for replying
i have written the following code
create table ext_table_csv (
i Number,
n Varchar2(20),
m Varchar2(20)
organization external (
type oracle_loader
default directory ext_dir
access parameters (
records delimited by newline
fields terminated by ','
missing field values are null
location ('file.csv')
reject limit unlimited;
After writing this code i got an error saying u dont have sufficient privileges
what does this exactly mean ?????
Please help me out with this
i also got an error saying that[b] the directory does not exist
Please help me out with the same
Regards,
Mahesh

possibly that you don't have priveleges to create a table. also the command you are trying is applicable only to server side not client side. it should be run to where your database is installed.
you can also check if the existing directory exist
select *
from all_directories
where directory_name = 'EXT_DIR'if the directory does not exist you have to ask your database administrator to create the EXT_DIR.

Loading data from .csv file into Oracle Table

Hi,
I have a requirement where I need to populate data from .csv file into oracle table.
Is there any mechanism so that i can follow the same?
Any help will be fruitful.
Thanks and regards

You can use Sql Loader or External tables for your requirement
Missed Karthick's post ...alredy there :)
Edited by: Rajneesh Kumar on Dec 4, 2008 10:54 AM

How to load the data from .csv file to oracle table???

Hi,
I am using oracle 10g , plsql developer. Can anyone help me in how to load the data from .csv file to oracle table. The table is already created with the required columns. The .csv file is having about 10lakh records. Is it possible to load 10lakh records. can any one please tell me how to proceed.
Thanks in advance

981145 wrote:
Can you tell more about sql * loader??? how to know that utility is available for me or not??? I am using oracle 10g database and plsql developer???SQL*Loader is part of the Oracle client. If you have a developer installation you should normally have it on your client.
the command is
sqlldrType it and see if you have it installed.
Have a look also at the FAQ link posted by Marwin.
There are plenty of examples also on the web.
Regards.
Al

Dynamically creating oracle table with csv file as source

Hi,
We have a requirment..TO create a dynamic external table.. Whenever the data or number of columns change in the CSV file the table should be replaced with current data and current number of columns...as we are naive experienced people in oracle please give us a clear solution.. We have tried with a code already ..But getting some errors. Code given below..
thank you
we have executed this code by changing the schema name and table name ..Remaining everything same ...
Assume the following:
- Oracle User and Schema name is ALLEXPERTS
- Database name is EXPERTS
- The directory object is file_dir
- CSV file directory is /export/home/log
- The csv file name is ALLEXPERTS_CSV.log
- The table name is all_experts_tbl
1. Create a directory object in Oracle. The directory will point to the directory where the file located.
conn sys/{password}@EXPERTS as sysdba;
CREATE OR REPLACE DIRECTORY file_dir AS '/export/home/log';
2. Grant the directory privilege to the user
GRANT READ ON DIRECTORY file_dir TO ALLEXPERTS;
3. Create the table
Connect as ALLEXPERTS user
create table ALLEXPERTS.all_experts_tbl
(txt_line varchar2(512))
organization external
(type ORACLE_LOADER
default directory file_dir
access parameters (records delimited by newline
fields
(txt_line char(512)))
location ('ALLEXPERTS_CSV.log')
This will create a table that links the data to a file. Now you can treat this file as a regular table where you can use SELECT statement to retrieve the data.
PL/SQL to create the data (PSEUDO code)
CREATE OR REPLACE PROCEDURE new_proc IS
-- Setup the cursor
CURSOR c_main IS SELECT *
FROM allexperts.all_experts_tbl;
CURSOR c_first_row IS ALLEXPERTS_CSV.logSELECT *
FROM allexperts.all_experts_tbl
WHERE ROWNUM = 1;
-- Declare Variable
l_delimiter_count NUMBER;
l_temp_counter NUMBER:=1;
l_current_row VARCHAR2(100);
l_create_statements VARCHAR2(1000);
BEGIN
-- Get the first row
-- Open the c_first_row and fetch the data into l_current_row
-- Count the number of delimiter l_current_row and set the l_delimiter_count
OPEN c_first_row;
FETCH c_first_row INTO l_current_row;
CLOSE c_first_row;
l_delimiter_count := number of delimiter in l_current_row;
-- Create the table with the right number of columns
l_create_statements := 'CREATE TABLE csv_table ( ';
WHILE l_temp_counter <= l_delimiter_count
LOOP
l_create_statement := l_create_statement || 'COL' || l_temp_counter || ' VARCHAR2(100)'
l_temp_counter := l_temp_counter + 1;
IF l_temp_counter <=l_delimiter_count THEN
l_create_statement := l_create_statement || ',';
END IF;
END;
l_create_statement := l_create_statement || ')';
EXECUTE IMMEDIATE l_create_statement;
-- Open the c_main to parse all the rows and insert into the table
WHILE rec IN c_main
LOOP
-- Loop thru all the records and parse them
-- Insert the data into the table created above
END LOOP;

The initial table is showing errors and the procedure is created with compilation errors
After executing the create table i am getting the following errors
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "identifier": expecting one of: "badfile,
byteordermark, characterset, column, data, delimited, discardfile,
disable_directory_link_check, exit, fields, fixed, load, logfile, language,
nodiscardfile, nobadfile, nologfile, date_cache, processing, readsize, string,
skip, territory, varia"
KUP-01008: the bad identifier was: deli
KUP-01007: at line 1 column 9
ORA-06512: at "SYS.ORACLE_LOADER", line 19

How to upload above20000 records csv files into oracle table,Oracle APEX3.2

Can any one help me how to CSV upload more than 20,000 records using APEX 3.2 upload process.i am using regular upload process using BOLB file
SELECT blob_content,id,filename into v_blob_data,v_file_id,v_file_name
FROM apex_application_files
WHERE last_updated = (select max(last_updated)
from apex_application_files WHERE UPDATED_BY = :APP_USER)
AND id = (select max(id) from apex_application_files where updated_by = :APP_USER);
I tried to upload but my page getting time out. my application best working up to 1000 records. after that its getting timed out.Each record is storing 2 secornds in the oracle table.So 1000 records it taking 7 minuts after that APEX upload webpage getting timed out
please help me with source how to speed upload csv file process or help another best with with source example.
Thanks,
Sant.
Edited by: 994152 on Mar 15, 2013 5:38 AM

See this posting:
Internet Explorer Cannot Display
There, I provided a couple of links on this particular issue. You need to change the timeout on your application server.
Denes Kubicek
http://deneskubicek.blogspot.com/
http://www.apress.com/9781430235125
http://apex.oracle.com/pls/apex/f?p=31517:1
http://www.amazon.de/Oracle-APEX-XE-Praxis/dp/3826655494
-------------------------------------------------------------------

How to read a CSV file and Insert data into an Oracle Table

Hi All,
I have a Clob file as a in parameter in my PROC. . File is comma separated.need procedure that would parse this CLOB variable and populate in oracle table .
Please let me some suggestions on this.
Thanks,
Chandra R

jeneesh wrote:
And, please don't "hijack" 5 year old thread..Better start a new one..I've just split it off to a thread of it's own. ;)
@OP,
I have a Clob file as a in parameter in my PROC. . File is comma separated.need procedure that would parse this CLOB variable and populate in oracle table .You don't have a "clob file" as there's no such thing. CLOB is a datatype for storing large character based objects. A file is something on the operating system's filesystem.
So, why have you stored comma seperated data in a CLOB?
Where did this data come from? If it came from a file, why didn't you use SQL*Loader or, even better, External Tables to read and parse the data into structured format when populating the database with it?
If you really do have to parse a CLOB of data to pull out the comma seperated values, then you're going to have to write something yourself to do that, reading "lines" by looking for the newline character(s), and then breaking up the "lines" into the component data by looking for commas within it, using normal string functions such as INSTR and SUBSTR or, if necessary, REGEXP_INSTR and REGEXP_SUBSTR. If you have string data that contains commas but uses double quotes around the string, then you'll also have the added complexity of ignoring commas within such string data.
Like I say... it's much easier with SQL*Loader of External Tables as these are designed to parse such CSV type data.

I need to import data from a CSV file to an Oracle table

I need to import data from a CSV file to an Oracle table. I'd prefer to use either SQL Developer or SQL Plus code.
As an example, my target database is HH910TS2, server is ADDb0001, my dB login is em/em, the Oracle table is AE1 and the CSV file is AECSV.
Any ideas / help ?

And just for clarity, it's good to get your head around some basic concepts...
user635625 wrote:
I need to import data from a CSV file to an Oracle table. I'd prefer to use either SQL Developer or SQL Plus code.SQL Developer is a GUI front end that submits code to the database and displays the results. It does not have any code of it's own (although it may have some "commands" that are SQL Developer specific)
SQL*Plus is another front end (character based rather than GUI) that submits code to the database and displays the results. It also does not have code of it's own although there are SQL*Plus commands for use only in the SQL*Plus environment.
The "code" that you are referring to is either SQL or PL/SQL, so you shouldn't limit yourself to thinking it has to be for SQL Developer or SQL*Plus. There are many front end tools that can all deal with the same SQL and/or PL/SQL code. Focus on the SQL and/or PL/SQL side of your coding and don't concern yourself with limitations of what tool you are using. We often see people on here who don't recognise these differences and then ask why their code isn't working when they've put SQL*Plus commands inside their PL/SQL code. ;)

Import excel/csv data to oracle table problem

hello sir,
i am using application importing csv/excel data to oracle table which is developed by vikas and that i want but when i add csv file and upload. My cvs file will displayed but when i click on create table button then it gives error like ...
1) invalid datatype. vikas link as :
http://htmldb.oracle.com/pls/otn/wwv_flow_custom_auth_std.login_page?p_flow_page=38131:1:1287783765705596
i search in forum for solution i got following link but it also gives me error
Re: File Browse, File Upload
still i am trying but i dont get solution . plz help me!!! give me steps ......
Report on table created
says:
failed to parse SQL query:
ORA-00942: table or view does not exist

Hi
Firstly, are you trying to upload your data into a new table or an existing table? If it's a new table, what name are you using?
Secondly, it may help if we could see a sample of the csv

How to import data from excel or csv files to Oracle table

hello everybody,
I am new here and new in Oracle. I would like to know the steps how to import data from excel or csv files to Oracle table.
Let say I already have table inside the Oracle. Then my user give me the sets of data inside the Excel Worksheet.
So, how can I import the excel data into Oracle table.
Thank you in advance.
cheers,
shima

Even easier. Download JDeveloper 11G from this site.
Set up the database connection, right click on the table, select Import->Excel and specify your file to load it. On the import pop-up, you must view and update each tab indicating Columns, Data Types, and DML.
Columns -- move the selected columns that you want to load to the box on the right
Data Types -- select column name from second column to which the data for each column of the import file should load
DML -- click this tab to generate the INSERT SQL
Once done click 'Insert'

SQL loaded not loading data from csv format to oracle table.

Hi,
I trying to load data from flat files to oracle table,it not loading decimal values.Only character type of data is loaded and number format is null.
CONTROL FILE:
LOAD DATA
INFILE cost.csv
BADFILE consolidate.bad
DISCARDFILE Sybase_inventory.dis
INSERT
INTO TABLE FIT_UNIX_NT_SERVER_COSTS
FIELDS TERMINATED BY ',' optionally enclosed by '"'
TRAILING NULLCOLS
HOST_NM,
SERVICE_9071_DOLLAR FLOAT,
SERVICE_9310_DOLLAR FLOAT,
SERVICE_9700_DOLLAR FLOAT,
SERVICE_9701_DOLLAR FLOAT,
SERVICE_9710_DOLLAR FLOAT,
SERVICE_9711_DOLLAR FLOAT,
SERVICE_9712_DOLLAR FLOAT,
SERVICE_9713_DOLLAR FLOAT,
SERVICE_9720_DOLLAR FLOAT,
SERVICE_9721_DOLLAR FLOAT,
SERVICE_9730_DOLLAR FLOAT,
SERVICE_9731_DOLLAR FLOAT,
SERVICE_9750_DOLLAR FLOAT,
SERVICE_9751_DOLLAR FLOAT,
GRAND_TOTAL FLOAT
In table FLOAT are replaced by number(20,20)
SAmple i/p from csv:
ABOS12,122.46,,1315.00,,1400.00,,,,,,,,1855.62,,4693.07
Only abos12 is loaded and rest of the number are not loaded.
Thanks.

Hi,
Thanks for your reply.
Its throwing error.
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Bind array: 64 rows, maximum of 256000 bytes
Continuation: none specified
Path used: Conventional
Table FIT_UNIX_NT_SERVER_COSTS, loaded from every logical record.
Insert option in effect for this table: INSERT
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
HOST_NM FIRST 255 , O(") CHARACTER
SERVICE_9071_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9310_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9700_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9701_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9710_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9711_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9712_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9713_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9720_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9721_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9730_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9731_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9750_DOLLAR NEXT 255 , O(") CHARACTER
SERVICE_9751_DOLLAR NEXT 255 , O(") CHARACTER
GRAND_TOTAL NEXT 255 , O(") CHARACTER
value used for ROWS parameter changed from 64 to 62
Record 1: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 2: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 3: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 4: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 5: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 6: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 7: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 8: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 9: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 10: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Record 11: Rejected - Error on table FIT_UNIX_NT_SERVER_COSTS, column GRAND_TOTAL.
ORA-01722: invalid number
Please help me on it.
Thanks,

Conversion of csv in flat file to oracle table format

hi friends
I have written a C code for conversion of flat file data to Comma seperated values
Now i have to put this .csv file into oracle tables.
Please suggest me a solution to this so that i can put this values into oracle tables
Regards,
Mahesh

You can use Sql*Loader (sqlldr) :
TEST@db102 SQL> create table test (a varchar2(30), b varchar2(30), c varchar2(30));
Table created.
TEST@db102 SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
$ cat test.csv
aaa,bbb,ccc
ddd,eee,fff
111,222,333
$ cat test.ctl
load data
infile 'test.csv'
replace into table test
fields terminated by ','
(a char, b char, c char)
$ sqlldr test/test control=test.ctl
SQL*Loader: Release 10.2.0.1.0 - Production on Wed Mar 8 15:42:42 2006
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Commit point reached - logical record count 3
$ sqlplus test/test
SQL*Plus: Release 10.2.0.1.0 - Production on Wed Mar 8 15:42:48 2006
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
TEST@db102 SQL> select * from test;
A                    B                    C
aaa                  bbb                  ccc
ddd                  eee                  fff
111                  222                  333
TEST@db102 SQL>                                                                                                             But probably there's no need to convert the flat file to csv...

Load data from Oracle table,CSV to Hyperion Essbase

Hi All,
when i am executing interface(load data into hyperion essbase from oracle table and csv files),
I am getting below error.
com.sunopsis.tools.core.exception.SnpsSimpleMessageException: Error during task interpretation
Task:1
java.lang.Exception: BeanShell script error: Sourced file: inline evaluation of: ``out.print("drop table TEMP_BIAPPS.A3$") ; out.print(thread) ; out.print("_0Combi . . . '' : Undefined argument: thread : at Line: 2 : in file: inline evaluation of: ``out.print("drop table TEMP_BIAPPS.A3$") ; out.print(thread) ; out.print("_0Combi . . . '' : ( thread )
BSF info: Drop work table at line: 0 column: columnNo
     at com.sunopsis.dwg.codeinterpretor.a.a(a.java)
     at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
     at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
     at com.sunopsis.dwg.cmd.e.k(e.java)
     at com.sunopsis.dwg.cmd.g.A(g.java)
     at com.sunopsis.dwg.cmd.e.run(e.java)
     at java.lang.Thread.run(Unknown Source)
Text:drop table TEMP_BIAPPS.A3$<?=thread?>_0Combination1 A
     at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
     at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
     at com.sunopsis.dwg.cmd.e.k(e.java)
     at com.sunopsis.dwg.cmd.g.A(g.java)
     at com.sunopsis.dwg.cmd.e.run(e.java)
     at java.lang.Thread.run(Unknown Source)
Can any one help me out?
Edited by: 791206 on Nov 17, 2010 11:16 PM
Edited by: maulik vadodariya on Nov 17, 2010 11:16 PM
Edited by: maulik vadodariya on Nov 17, 2010 11:17 PM
帖子经 791206编辑过

Hi All,
It was staging area problem.
So,I created new schema(User/Password in Oracle Database),and give staging area to that newly created schema(Defination Tab of Interface).
Also,in Flow tab,I selected LKM File to SQL. and problem resolved.
Thanks,
Maulik G. Vadodariya
Edited by: Maulik Vadodariya on 22 Nov, 2010 2:36 PM

CSV to Oracle Table

Similar Messages

Maybe you are looking for