About External Tables

Hello Everybody
I have a flat file (text file). I have to upload certain data into the database. I dont want to use sqlloader but instead external tables. I did some research myself :)
http://download.oracle.com/docs/cd/B19306_01/server.102/b14231/tables.htm#sthref2446
I have read the document first according to forums etiquette :) but here is my question the directories are being placed on the server but I have them on client.
Just to give you all further details; the text file comes from another software and now I have to upload the data from those files into the database.
I know someone might say that I can manually move them from client to server but I am not looking for that. Is there a way that I could link up those directories from server to client or anything similar?
I am working on Oracle 10g Database 10.2.0.2 and windows enviroment if that can also help to answer :)
- Amy De Cay

No need to be sorry at all for a first question yours is very well written (I wish other first time posters would read this).
You can't have remote external jobs (where DBMS_SCHEDULER can call on OS script that resides on another machine) until 11g and that involves installing an agent on the client.
I didn't realise that you wanted to automate this process, that makes for a different answer. I was working on the assumption that the end user on the client would be able to invoke SQL*Loader and you could wrap this with whatever you desired (.cmd script, --.nyet-- .net, c flat c#, Victoria Bitter VB).
if you want to automate this I would be looking at mapping the UNC drive. Sybrand is right about using a Domain Admin account to run the Oracle service, but it might be considered overkill by your possibly unfriendly Windows BOFH sysadmin. However it is the easiest way to solve the problem.

Similar Messages

Can we create external table from an existing database table ?

Hi everyone,
As i understand, its possible to create a new table based on an existing table without copying any values from it, using the following command;
CREATE TABLE newtable AS
(SELECT * FROM oldtable WHERE 1=2);
I would like to know whether we can do a similar thing with external tables. That is to create an external table with the same columns as of an existing table.
Thanks in advance for your answers.

You need to clear conception about external table.
http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/utility.htm#sthref1800
http://download.oracle.com/docs/cd/B19306_01/server.102/b14231/tables.htm#sthref2448

What is an External Table all about ?

Here is another question that i faced while solving the aptitude paper of a software company and i am confused with the term external table :(
EMPDET is an external table containing the columns EMPNO and ENAME. Which command would work in relation to the EMPDET table?
A. UPDATE empdet
SET ename = 'Anton'
WHERE empno = 21354;
B. DELETE FROM empdet
WHERE ename LIKE 'LT%';
C. CREATE VIEW empvu
AS
SELECT * FROM empdept;
D. CREATE INDEX empdet_idx
ON empdet(empno);
The correct answer is C.
But please someone help me in understanding it
Please help
Edited by: gOMzY on Aug 30, 2010 7:15 AM

gOMzY wrote:
Here is another question that i faced while solving the aptitude paper of a software company and i am confused with the term external table :(
EMPDET is an external table containing the columns EMPNO and ENAME. Which command would work in relation to the EMPDET table?It depends which driver was used to create the External Table. Usually it is not possible to WRITE into external tables. Only to select from them. But newer database versions and the proper driver will allow you to write to this table. There are two drivers. the ORACLE_LOADER which only allows to select. And the ORACLE_DATAPUMP which allows to insert (unload) data.
>
A. UPDATE empdet
SET ename = 'Anton'
WHERE empno = 21354;Wrong with both drivers
>
B. DELETE FROM empdet
WHERE ename LIKE 'LT%';Wrong with both drivers
>
C. CREATE VIEW empvu
AS
SELECT * FROM empdept;Possible with both drivers
>
D. CREATE INDEX empdet_idx
ON empdet(empno);Not possible. Since the file can change , the database would not notice that. Therefore any kind of index would point to the wrong information. Therefore it is not allowed to create an index on it.
>
The correct answer is C.makes sense.
See also: http://download.oracle.com/docs/cd/B19306_01/server.102/b14231/tables.htm#sthref2448

Facing Many Problems About Creating Directory and an External Table

Question:
The weird thing is if you look at question 10-b in page 3-41, it says:
(page 3-41 "Oracle Database 10g SQL Fund. II Vol.1")
Merge the data in the EMP_DATA table created in the last lab into the data in the emp_hist table. Assume
that the data in external EMP_DATA table matches the EMP_HIST table, update the email column
of the EMP_HIST table to match the EMP_DATA table row. If a row in the EMP_DATA table does not
match, insert into the EMP_HIST tables. Rows are considered matching when the employee's first and
last name are identical.
To me, this question is constructed wrongly. First of all in the last lab we have not been asked to create EMP_DATA. Secondly, EMP_DATA is empty.
Thirdly, this question asks us to merge into EMP_HIST table while EMP_DATA is empty.
EMP_HIST table currently has copied data from employees table. EMP_HIST structure:
FIRST_NAME VARCHAR2(20)
LAST_NAME NOT NULL VARCHAR2(25)
EMAIL NOT NULL VARCHAR2(45)
Anway, i did the merge as following:
merge into emp_hist e
using emp_data d
on (e.first_name = d.first_name)
when matched then
update set
e.last_name = d.last_name,
e.email = d.email
when not matched then
insert values (d.first_name, d.last_name, d.email);
I get this error:
Error report:
SQL Error: ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file emp.dat in EMP_DIR not found
ORA-06512: at "SYS.ORACLE_LOADER", line 19
29913. 00000 - "error in executing %s callout"
*Cause:    The execution of the specified callout caused an error.
*Action:   Examine the error messages take appropriate action.
On the other hand, i said let me try this:
merge into emp_data d
using emp_hist e
on (d.first_name = e.first_name)
when matched then
update set
d.last_name = e.last_name,
d.email = e.email
when not matched then
insert values (e.first_name, e.last_name, e.email);
I get this error because external table is final once its created as far as i know:
Error report:
SQL Error: ORA-30657: operation not supported on external organized table
30657.0000 - "operation not supported on external organized table"
*Cause:    User attempted on operation on an external table which is
not supported.
*Action:   Don't do that!
I do not know what to do. I did my best, please help.
Edited by: user11164565 on Jul 27, 2009 2:43 AM

user11164565 wrote:
NOTE: I did my best, i did all what i can do, but the problem persists. Please help
I will mention all the steps i did clearly....
I gave scott the following grants:
grant create any directory to scott;
grant read on directory emp_dir to scott;
1. Created a directory and its been created successfully:
create or replace directory emp_dir
as 'F:\emp_dir';
Then i did the following just to make sure my directory is recognized:
SELECT *
FROM dba_directories;
I found the drive amongst the results...
OWNER DIRECTORY_NAME
DIRECTORY_PATH
SYS EMP_DIR
F:\emp_dir
SYS SUBDIR
D:\oracle\product\10.2.0\db_1\demo\schema\order_entry\/2002/Sep
SYS XMLDIR
D:\oracle\product\10.2.0\db_1\demo\schema\order_entry\
2. I created an external table emp_data (the script is given by the text book): done successfully
drop table emp_data;
CREATE TABLE emp_data
(first_name VARCHAR2(20)
,last_name VARCHAR2(20)
, email VARCHAR2(30)
ORGANIZATION EXTERNAL
TYPE oracle_loader
DEFAULT DIRECTORY emp_dir
ACCESS PARAMETERS
RECORDS DELIMITED BY NEWLINE CHARACTERSET US7ASCII
NOBADFILE
NOLOGFILE
FIELDS
( first_name POSITION ( 1:20) CHAR
, last_name POSITION (22:41) CHAR
, email POSITION (43:72) CHAR )
LOCATION ('emp.dat') ) ;
3. I went to F:\ drive to see if emp_dir folder exist or not! I did not see it. I checked hidden files, nothing there. Anyway, i ignored it and did step 4.
<snip>
"Anyway, I ignored it . . . "
and hence the rest of your problems. I did not see in the steps you recounted that you acually created a directory ("folder") named "\emp_dir" on your f: drive. Nothing you create within the database will actually create that directory on the OS. Createing a directory in Oracle, createing an external table in Oracle, will only create pointers to objects that Oracle will simply assume actually exists.

How to spilt files when using DATA UNLOAD (External Table, 10g)?

Hi,
I am runnin 10gR2 and need to export partitions by using data_pump driver
via dmp files (External Tables).
Now as requierment the created files can not exceed 2GB mark.
Some of partitions are larger than that.
How could I split the partition so I could be able to create files smaller than 2GB? Is there any parameter I am not aware of or do I need to do SELECT COUNT(*) FROM source_table PARTITION(partiton_01);
and than to work with ROWNUM?
This example working fine for all partitions samller than 2GB:
CREATE TABLE partiton_01_tbl
2 ORGANIZATION EXTERNAL
3 (
4 TYPE ORACLE_DATAPUMP
5 DEFAULT DIRECTORY def_dir1
6 LOCATION ('inv_xt1.dmp')
7 )
8 PARALLEL 3
9 AS SELECT * FROM source_table PARTITION(partiton_01);

You could specify multiple destination files in the LOCATION parameter (the number of files should match the degree of parallelism specified). I am not aware of an option that would allow the external table to automatically add new data files as the partition size increased, so you'd likely have to do some sort computation about the expected size of the dump file in order to figure out how many files to create.
Justin

How to use substr in external table defnition.

Hi All,
Im using oracle 11g. I have an external table which is reading data from a file. For one of the column, i need to get only the first 250 characters. My external table defnition looks like this
create table tbl_substr
( col1 varchar2(20),
col2 varchar2(250)
organization external
( type oracle_loader
default directory XXXX
access parameters (
records delimited by newline
FIELDS TERMINATED BY '|'
missing field values are null
( col1 ,
col2 "substr(:col2,1,250)"
) ) location ('file.txt') )
reject limit unlimited
But this defnition gives an error when i do select * from tbl_substr
I want to use substr in external table defnition its self and not in SELECT. Also i dont want to crete a view to solve this. If anyone has done this please help.

You need to play with COLUMN_TRANSFORMS
http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/et_params.htm#sthref1792
BTW, i too got it from Google. I was not aware about this :)
Amardeep Sidhu

External Table Authentication in OBIEE 11g

Hi ,
I have a security table, which contains userid,displayname,group . I have imported Security table in Physical Layer. I'm creating session variables based on condition.
When am trying to logging into analytic s getting an error, invalid username and password . I'm using 11.1.1.6.0 version
How to handle external table authentication in OBIEE 11g version.
Regards,
Malli

Hi fiaz,
That links talks about 10g version.
Step1: We have imported a secutiry table in Physical layer.
Step2: Creating a session variable by selecting initilazation block.
Select user_name,group from security_table where user_id=':USER' and pwd=':password';
step3: created DISPLAYNAME,GROUP & USER VARIABLES in edit target window
After these modifications i was trying to logging with new user, which is there in security table.
I am getting an error that is invalid user or password.
Is there any other changes does it required here.
Regards,
Malli
Edited by: user10675696 on Dec 26, 2012 9:39 PM

Need help in Creating External Tables

Hi All,
I have a flat file containing numberic data, CLOB data, and also date columns data. I have to load this flat file data into staging server table using External tables. I have to write a stored procedure in such way creating exteranl tables dynamically. My question or need help from you people is that how to define the external table to load the CLOB data from flat file.
Thanks,
Sankar

The LOCATION clause of an external table specifies the file or files to be read when the table is queried. You can change it to refer to different files without dropping and recreating the table and invalidating all dependent code.
Regarding DBMS_SQL, yes you can do it the hard way if you prefer.
Good point about CLOB columns in external tables. Quite possibly they are not supported, but I would have to check the manuals and try some examples.

Unix permission problem for external table in oracle 10g, sun solaris

Hello All,
I'm facing a problem in accessing external table which has stumped me a bit.
What I'm looking for is to use a external table with restricted permission to Others(770) on unix.
Would appreciate if someone helps me out here.
Here are the steps:
1.create directory ext_tab_dir1 as '/home/ravi/test'
2.grant read,write on directory ext_tab_dir1 to scott
3.CREATE TABLE scott.emp_load1(employee_number CHAR(5))
ORGANIZATION EXTERNAL (
type oracle_loader
default directory ext_tab_dir1
access parameters (
records delimited by newline
fields terminated by ' '
optionally enclosed by '"'
missing field values are null
location ('info.dat')
Now I have added unix user oracle to unix group myDbGroup and provided myDbGroup read/write/exec permission on directory /home/ravi/test.
info.dat has been placed in /home/ravi/test.
#pwd
/home/ravi
#ls -l
drwxrwx--- 2 ravi myDbGroup 512 Mar 7 17:35 test
I have manually logged in as user oracle and successfully created a sample file in /home/ravi/test.
-rwxrwx--- 1 ravi myDbGroup 13 Mar 7 17:33 info.dat
-rw-r--r-- 1 oracle oinstall 0 Mar 7 18:05 sampleFile
I then connect to the db using sqlplus as ravi and do a
#select * from scott.emp_load1
I get the following error
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04063: unable to open log file emp_load1_18567.log
OS error Permission denied
ORA-06512: at "SYS.ORACLE_LOADER", line 19
After this, I gave full permission to /home/ravi/test
drwxrwxrwx 2 ravi myDbGroup 512 Mar 7 17:35 test
#select * from scott.emp_load1
And this was successful.
It created a log file in /home/ravi/test
-rwxrwx--- 1 ravi myDbGroup 13 Mar 7 17:33 info.dat
-rw-r--r-- 1 oracle oinstall 0 Mar 7 18:05 sampleFile
-rw-r--r-- 1 oracle oinstall 0 Mar 7 18:05 emp_load1_18567.log
Now what stumped me is the owner and group owner of the file emp_load1_18567.log.
It is same as the sampleFile which I created manually!!
From this it can be deducted oracle is not using user id oracle while reading/writing to the unix directory but somehow assigning user id oracle as owner to the log file at the end.
If someone has encountered this problem earlier or has some info about this...pls share.
Regards,
Ravinandan

Thanks for the reply.
I have earlier checked this with NOLOGFILE option.
But no luck.
I would like to add one more detail(which I missed earlier) about the problem.
If I give 750 access to /home/ravi(That is read/exec to Group and none perm to Others),
I will get the same error(KUP-04063: unable to open log file emp_load1_18567.log).
This obviously means oracle is not even able to access the directory(Although unix user oracle has access via the group perms).

External table Authenticaittion

hi all
In external table authenticatio, how to authenticate passwords.
I have usernames and encrypted pwds in an oracle table. How to verify them at user login.
If my doubt is a childish one, plz excuse me
Thanks in Advance

Hi Anand,
When you use external tables, you pretty much have complete control over the process. All you have to do is make sure you encript the password that is passed in and match it with the stored encrypted password. For example suppose I have the following table
OBIEE_SECURITY:
USER PASSWORD
joe jk4A9M#920
Then in an initialization block, when I set the USER variable, I'll set it with a custom query like:
SELECT USER
FROM OBIEE_SECURITY
WHERE USER = ':USER'
AND PASSWORD = ENCRYPT(':PASSWORD')
John Minkjan has an excellent blog about general External Table authentication. See it here
Hope this helps!
Best regards,
-Joe

External Tables preprocessor in 11.2 as a tool for DBAs

I'm really really excited about new 11.2 external tables preprocessor feature. Oracle documentation show examples based on compressed files and how to load uncompress them on a fly and load them to oracle.
Although that is interesting use of external tables preprocessor, I think what is more interesting is the fact that it can be used outside normal ETL and can be great tool for DBAs who run scripts in unix, now they can execute the scripts from oracle environment (e.g. oracle apex) and read the results of the script very easy way using select * from ... (until now I was using JAVA for executing unix scripts from Oracle environment, this is easier way for sure)
I posted few very simple samples on my blog http://jiri.wordpress.com/2010/01/19/no-more-unix-scripts-in-11-2/

Hi,
I agree the external tables preprocessor is a great feature.
BUT, I have a problem when I use it with Oracle APEX 4.1.
It gives ORA-29913: error in executing ODCIEXTTABLEFETCH callout
Any ideas?
Thanks

IGNORE COMMENTS IN EXTERNAL TABLE FILE

I currently have a partitioned table, split by month going back some 7 years.
The business have now agreed to archive off some of the data but to have it
available if required. I therefore intend to select month by month, the data
to a flat file and make use of external tables to provide the data for the
business if required. Then to remove the months partition thus releaseing
space back to the database. Each months data is only about 100bytes long, but
there could be about 15million rows.
I can successfully create the external table and read the data.
However I would like to place a header in the spooled file but the only method
I have found to ignore the header is to use the "SKIP" parameter. I would
rather not use this as it hard codes the number of lines I reserve for a
header.
Is there another method to add a comment to a external tables file and have the
selects on the file ignore the comments?
Could I use "LOAD WHEN ( column1 != '--' )" as each comment line begins with a
double dash?
Also, is there a limit to the size of an external tables file?

When I added the lines to attempt to
exclude the Header lines to be:
RECORDS DELIMITED BY newline
NOBADFILE
NODISCARDFILE
NOLOGFILE
SKIP 0
LOAD WHEN ( cvt_seq_num LIKE '--%' )
FIELDS
TERMINATED BY ","
OPTIONALLY ENCLOSED BY '"'
MISSING FIELD VALUES ARE NULL
REJECT ROWS WITH ALL NULL FIELDS
( cvt_seq_num INTEGER EXTERNAL(12)
I got:
select * from old_cvt_external
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPENcallout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "identifier": expecting one of: "equal,notequal"
KUP-01008: the bad identifier was: LIKE
KUP-01007: at line 6 column 33
ORA-06512: at "SYS.ORACLE_LOADER", line 14
ORA-06512: at line 1
When I moved the line I got:
RECORDS DELIMITED BY newline
NOBADFILE
NODISCARDFILE
NOLOGFILE
SKIP 0
FIELDS
TERMINATED BY ","
OPTIONALLY ENCLOSED BY '"'
MISSING FIELD VALUES ARE NULL
REJECT ROWS WITH ALL NULL FIELDS
LOAD WHEN ( cvt_seq_num LIKE '--%' )
I got:
select * from old_cvt_external
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "load": expecting one of: "exit, ("
KUP-01007: at line 11 column 11
ORA-06512: at "SYS.ORACLE_LOADER", line 14
ORA-06512: at line 1
So it appears that the "LOAD WHEN" does not work.
Can you please suggest anything else?

IGNORE COMMENTS IN EXTERNAL TABLE

I currently have a partitioned table, split by month going back some 7 years.
The business have now agreed to archive off some of the data but to have it
available if required. I therefore intend to select month by month, the data
to a flat file and make use of external tables to provide the data for the
business if required. Then to remove the months partition thus releaseing
space back to the database. Each months data is only about 100bytes long, but
there could be about 15million rows.
I can successfully create the external table and read the data.
However I would like to place a header in the spooled file but the only method
I have found to ignore the header is to use the "SKIP" parameter. I would
rather not use this as it hard codes the number of lines I reserve for a
header.
Is there another method to add a comment to a external tables file and have the
selects on the file ignore the comments?
Could I use "LOAD WHEN ( column1 != '--' )" as each comment line begins with a
double dash?
Also, is there a limit to the size of an external tables file?
Thanks
JD

Hi Gurus..
any help will be highly appreciated.

Oracle 9i External Tables

Helllo Everyone.
I am new to external tables concept and I have only used using flat files.
Is there a complete reference on on external tables somewhere?
Also, instead of .csv or text based datafiles, can I use XML as data source for these external tables? If so, how?
Thanks,
R

The link to the documentation has been already posted.
About the xml as a possible datasource, in Oracle9i only the ORACLE_LOADER driver is supported, it allows you to read data using SQL*Loader, that is to say only plain text files.
In Oracle10g a new driver has been added: ORACLE_DATAPUMP, it allows you to read with an external table a dump file created from other Oracle tables using the same ORACLE_DATAPUMP driver.
Unfortunately the ORACLE_DATAPUMP driver doesn't support the XMLTYPE datatype:
SQL> desc tbl_xml
Nome                                      Nullo?   Tipo
XML                                                PUBLIC.XMLTYPE
SQL> CREATE TABLE xml_ext
2    ORGANIZATION EXTERNAL
3    (
4      TYPE ORACLE_DATAPUMP
5      DEFAULT DIRECTORY DATA_PUMP_DIR
6      LOCATION ('xml_ext.dmp')
7    )
8    AS SELECT * FROM tbl_xml;
CREATE TABLE xml_ext
ERRORE alla riga 1:
ORA-30656: tipo di colonna non supportato in una tabella organizzata esterna
SQL>Max
[My Italian Oracle blog|http://oracleitalia.wordpress.com/2009/12/18/table-elimination-oppure-join-elimination-lottimizzatore-si-libera-della-zavorra/]

Sql Loader Vs External table? Which one is preffered?

Hello guru,
We trying to load the data into about 160 tables. Here are the following things we are doing currently,
1. We get the flat file in .txt format
2. We are writing the control file for each flat file for thier respective tables
3.And we are using sql loader command to load this data into the table.
I was wondering if this process is cumbersome, and i m not sure if external table could be simple in loading the tables compared to what we are doing ?And i have not used external tables, so wanted to know which on eis better ? So any idea is greatly appriciated !
FYI.. Version :- Oracle 11g
Thank you so much!

Thanks for you reply justin !
I found the below example for loading the data into external table...
CREATE OR REPLACE DIRECTORY dat_dir AS 'C:\Oradata\Data';
CREATE OR REPLACE DIRECTORY log_dir AS 'C:\Oradata\Log';
CREATE OR REPLACE DIRECTORY bad_dir AS 'C:\Oradata\Bad';
GRANT READ ON DIRECTORY dat_dir TO scott;
GRANT WRITE ON DIRECTORY log_dir TO scott;
GRANT WRITE ON DIRECTORY bad_dir TO scott;
CREATE TABLE revext (person      VARCHAR2(20),
                     rev_jan     NUMBER(4),
                     rev_feb     NUMBER(4),
                     rev_mar     NUMBER(4),
                     rev_apr     NUMBER(4),
                     rev_mai     NUMBER(4),
                     rev_jun     NUMBER(4),
                     rev_jul     NUMBER(4),
                     rev_aug     NUMBER(4),
                     rev_sep     NUMBER(4),
                     rev_oct     NUMBER(4),
                     rev_nov     NUMBER(4),
                     rev_dez     NUMBER(4))
ORGANIZATION EXTERNAL
   TYPE ORACLE_LOADER
   DEFAULT DIRECTORY dat_dir
   ACCESS PARAMETERS
     records delimited by newline
     badfile bad_dir:'revext%a_%p.bad'
     logfile log_dir:'revext%a_%p.log'
     fields terminated by ','
     missing field values are null
     ( person,
       rev_jan,
       rev_feb,
       rev_mar,
       rev_apr,
       rev_mai,
       rev_jun,
       rev_jul,
       rev_aug,
       rev_sep,
       rev_oct,
       rev_nov,
       rev_dez
   LOCATION ('revext.dat')
PARALLEL 4
REJECT LIMIT UNLIMITED;
CREATE TABLE revenue (
    person       VARCHAR2(20),
    month        VARCHAR2(3),
    revenue      NUMBER,
    CONSTRAINT revenue_pk PRIMARY KEY (person,month));
INSERT INTO revenue (person,month,revenue)
   SELECT person,'Jan',rev_jan
   FROM revext--but currently we are using sql loader, our data looks like this
1119Smith      01/01/1982AXYZ corporation xyz corp
1111collen      01/01/1990AABC corporation abc corp
     and control file is like this
INTO TABLE "XYZ_tbl"
   ID                     POSITION(01:05)        CHAR                  "DECODE(RTRIM(:ID), NULL, 'NA', :ID)"       ,
   Name                POSITION(06:15)        CHAR                  "DECODE(RTRIM(:NAME), NULL, 'NA', :Name)"   ,
   Act_dt              POSITION(16:25)        DATE                  "MM/DD/YYYY" NULLIF ACT_DT=BLANKS
My question is, can i use the options like NULLIF/ DECODE or changing datatypes when loading or functions like REPLACE / TO_DATE in External tables ? Any idea? Any example code or SQL will great
Thank you so much!

About External Tables

Similar Messages

Maybe you are looking for