External Tables preprocessor in 11.2 as a tool for DBAs

I'm really really excited about new 11.2 external tables preprocessor feature. Oracle documentation show examples based on compressed files and how to load uncompress them on a fly and load them to oracle.
Although that is interesting use of external tables preprocessor, I think what is more interesting is the fact that it can be used outside normal ETL and can be great tool for DBAs who run scripts in unix, now they can execute the scripts from oracle environment (e.g. oracle apex) and read the results of the script very easy way using select * from ... (until now I was using JAVA for executing unix scripts from Oracle environment, this is easier way for sure)
I posted few very simple samples on my blog http://jiri.wordpress.com/2010/01/19/no-more-unix-scripts-in-11-2/

Hi,
I agree the external tables preprocessor is a great feature.
BUT, I have a problem when I use it with Oracle APEX 4.1.
It gives ORA-29913: error in executing ODCIEXTTABLEFETCH callout
Any ideas?
Thanks

Similar Messages

External Table Preprocessor

Dear Experts,
I was trying to explore the "preprocessor" command in external table, but I was not able to succeed.
I just followed the steps mentioned in, latest Oracle Magazine.
http://www.oracle.com/technetwork/issue-archive/2012/12-nov/o62asktom-1867739.html
But finally when I do the select from table df, I am getting the following error.
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-29400: data cartridge error
KUP-04095: preprocessor command /export/csrmgr/ripple_dms/run_df.sh encountered error "error during exec: errno is 2
PS: The output of v$version;
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
"CORE     11.2.0.2.0     Production"
TNS for IBM/AIX RISC System/6000: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
Could anyone help me to resolve this issue.

I'm not able to test Tom's example at the minute.
However, that new feature was discussed in the March/April 2011 edition of Oracle Magazine too.
Oracle Magazine March/April 2011 article on new feature in 11gR2 called the external table preprocessor.
by Arup Nanda
CHANGE IN PROCESS
Now suppose that with your external table and its text file in place, the input file for the external table is compressed to reduce the volume of data transmitted across the network. Although compression helps the network bandwidth utilization, it creates a challenge for the ETL process. The file must be uncompressed before its contents can be accessed by the external table.
Rather than uncompress the file in a separate process, you can use the preprocessor feature of Oracle Database 11g Release 2 with external tables to uncompress the file inline. And you will notneed to change the ETL process.
To use the preprocessor feature, first you need to create a preprocessor program. The external table expects input in a text format but not necessarily in a file. The external table does not need to read a file; rather, it expects to get the file contents “fed” to it. So the preprocessor program must stream the input data directly to the external table—and not
create another input file. The input to the pre-processor will be a compressed file, and the output will be the uncompressed contents.
The following is the code for your new preprocessor program, named preprocess.bat:
@echo off
C:\oracle\product\11.2.0\dbhome_1\BIN\unzip.exe -qc %1The first line, @echo off, suppresses the output of the command in a Windows environment. The remaining code calls the unzip.exe utility located in the Oracle home. The utility needs an input file, which is the first (and only) parameter passed to it, shown as %1. The options q and c tell the utility to uncompress quietly (q) without producing extraneous output such as “Inflating” or “%age inflated” and match filenames case-insensitively (c), respectively.
Next you need to create the directory object where this preprocessor program is located. Logging in as SYS, issue
create directory execdir as 'c:\tools';And now grant EXECUTE permissions on the directory to the ETLADMIN user:
grant execute on directory execdir to etladmin;Finally, create the new external table:
create table indata1
cust_id number,
cust_name varchar2(20),
credit_limit number(10)
organization external
type oracle_loader
default directory etl_dir
access parameters
    records delimited by newline
    preprocessor execdir:'preprocess.bat'
    fields terminated by ","
location ('indata1.txt')
/It calls the preprocess.bat executable in the directory specified by EXECDIR before the external table accesses the indata1.txt file in the location specified by the ETL_DIR directory. Remember, indata1.txt is now a compressed file. So, in effect, the external table reads not the actual specified input file but rather the output of preprocess.bat, which is the uncompressed data from the indata1.txt file.
If you select from the external table now, the output will be similar to that of the earlier select * from indata1; query. The preprocessor passed the uncompressed contents of the indata1.txt (compressed) file on to the external table. There was no need to uncompress the file first—saving significant time and the intermediate space required and making it unnecessary to change the ETL process.
This inline preprocessing unzip example uses a script, but that is not always necessary. An executable can be used instead. For example, in Linux you can use /bin/gunzip. However, the utility can’t accept any parameters. So if you pass parameters (as in this article’s example), you must use a script.
Security Concerns
=================
The EXECUTE privilege on a directory is a new feature introduced in Oracle Database 11g Release 2. It enables the DBA to grant EXECUTE permissions only for certain directories and only to certain users. Without WRITE privileges, users will not be able to
update the executables inside a directory to insert malicious code, but users will be able to execute the “approved” code accessible in a single location. The DBA can put all the necessary preprocessor tools into a single directory and grant EXECUTE privileges there to the users who may need them. And, of course, the executables and data should be in different directories.
Preprocessing also requires some special precautions on the part of the DBA. Because the executables called by preprocessing programs will be executed under the privileges of the Oracle software owner and malicious executable code can cause a lot of damage, the DBA should be extremely careful in monitoring executables for potentially harmful code.
The directory containing the preprocessor executables needs to be accessible to the Oracle software owner for EXECUTE operations only, not for WRITE activity. Therefore, as an added precaution, the system administrator can remove WRITE access to that directory from all users, including the Oracle software owner. This significantly reduces the chance of damage by malicious code.
Other Uses
==========
Compression is not the only use for inline preprocessing, although it certainly is the most widely used. You can, for example, use this preprocessing technique to show the output of a program as an external table. Consider, for instance, the dir command in Windows for listing the contents of a directory. How would you like to get the output as a table so that you can apply predicates?
Getting and using this output is quite simple with the preprocessor functionality. Remember, the preprocessor does not actually need a file but, rather, requires the output of the preprocessor program. You can write a preprocessor program to send the
output of a dir command. The new preprocessor program, named preproc_dir.bat, has only the following two lines:
@echo off
dirYou will also need a file for the external table. The contents of the file are irrelevant, so you can use any file that the Oracle software owner can read in a directory to which that owner has read access. For this example, the file is dirfile.txt, and although the contents of the file are immaterial, the file must exist, because the external table will access it. Listing 1 shows how to create the table.
Because the dir command displays output in a prespecified manner, the external table easily parses it by reading the fields located in specific positions. For example, positions 1 through 10 display the date, 11 through 20 display the time, and so on. The dir command produces some heading and preliminary information that the external table has to ignore, so there is a skip 5 clause in Listing 1 that skips the first five lines of the output. The last few lines of the output show how many files and directories are present and how much free space remains. This output must be skipped as well, so the external table displays records only when the date column has a value.
Listing 1 also shows the result of a query against the external table. Because the MOD_DT column is of the date datatype, you can also apply a WHERE condition to select a specified set of records.
Code Listing 1:
===============
create table dir_tab
mod_dt date,
mod_time char(10),
file_type char(10),
file_size char(10),
file_name char(40)
organization external
type oracle_loader
default directory etl_dir
access parameters
    records delimited by newline
    preprocessor execdir:'preproc_dir.bat'
    skip 5
    load when (mod_dt != blanks)
    fields
      mod_dt position (01:10) DATE mask "mm/dd/yyyy",
      mod_time position (11:20),
      file_type position (21:29),
      file_size position (30:38),
      file_name position (39:80)
location ('dirfile.txt')
reject limit unlimited
-- select from this table
SQL> select * from dir_tab;
MOD_DT        MOD_TIME       FILE_TYPE       FILE_SIZE      FILE_NAME
16-DEC-10     10:12 AM       <DIR> .
16-DEC-10     10:12 AM       <DIR> ..
22-MAY-10     09:57 PM       <DIR> archive
22-MAY-10     10:27 PM                                2,048 hc_alap112.dat
05-DEC-10     07:07 PM                                   36 indata1.txt
22-DEC-05     04:07 AM                               31,744 oradba.exe
16-DEC-10     09:58 AM                                1,123 oradim.log
28-SEP-10     12:41 PM                                1,536 PWDALAP112.ora
16-DEC-10     09:58 AM                                2,560 SPFILEALAP112.ORA
9 rows selected.
-- select a file not updated in last 1 year
SQL> select * from dir_tab where mod_dt < sysdate - 365;
MOD_DT        MOD_TIME       FILE_TYPE       FILE_SIZE      FILE_NAME
22-DEC-05     04:07 AM                               31,744 oradba.exe

Oracle 11g - External Table/Remote File Issue?

Oracle 11g - External Table/Remote File Issue?
=============================
I hope this is the right forum for this issue, if not let me, where to go.
We are using Oracle 11g (11.2.0.1.0) on (Platform : solaris[tm] oe (64-bit)), Sql Developer 3.0.04
We are not allowed to put files on the CSV file system (Server A), where DB instance is running. We are able place CSV files on another server(Server B), where DB instance is not running.
We are trying to use oracle external table to load text files in .CSV format.
How do we create a Directory (Create Directory) on Server A DB to point to File system of Server B?
Is it feasible?
Any idea?
Thanks in helping.

The Solaris DBA should be able to mount the filesystem for you. Either that or you have to get creative transferring the file like this;
http://www.linkedin.com/groups/Getting-creative-external-table-preprocessor-140609.S.50474382?qid=ba673ce4-c4bb-40c5-8367-52bd2a2dfc80&trk=group_search_item_list-0-b-ttl&goback=%2Egmp_140609
Cheers
David

External Tables Date Error

I get error while loading data into Oracle 11g R2 with EXTERNAL TABLES.
error processing column DATE_M in row 1 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01847: day of month must be between 1 and last day of month
error processing column DATE_M in row 2 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 3 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 4 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01847: day of month must be between 1 and last day of month
error processing column DATE_M in row 5 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 6 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 7 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 8 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 9 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month
error processing column DATE_M in row 10 for datafile C:\app\S\admin\orcl\dpdump\TABLE_EXT.txt
ORA-01843: not a valid month-----
CREATE TABLE TABLE_EXT
   ( "COMPANY" VARCHAR2(101),
"COMPANY_VN" VARCHAR2(15),
"IL" VARCHAR2(17),
"TERMINAL" VARCHAR2(8),
"T_NO" VARCHAR2(15),
"NAME" VARCHAR2(108),
"SNAME" VARCHAR2(50),
"REF_NO" VARCHAR2(23),
"AMOUNT" NUMBER(15,2),
"DATE_M" DATE,
"TIME" VARCHAR2(11),
"TEL_NO" VARCHAR2(25),
"ADDRESS" VARCHAR2(50)
   ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY data_pump_dir
ACCESS PARAMETERS (
    RECORDS DELIMITED BY NEWLINE
    FIELDS TERMINATED BY '|'
    MISSING FIELD VALUES ARE NULL
COMPANY,
COMPANY_VN,
IL,
TERMINAL,
T_NO,
NAME,
SNAME,
REF_NO,
AMOUNT decimal ,
DATE_M CHAR date_format DATE mask "dd.mm.yyyy" ,
TIME,
TEL_NO,
ADDRESS
LOCATION ('TABLE_EXT.txt')
REJECT LIMIT 10------
Sample Data:
CITY HOSPITAL|04680072124|CITY|00614860|47746244218|JOHN|WHITE|172871|420,12|21.08.2011|14:26|0806422627784|06
CITY HOSPITAL|04680072124|CITY|00614847|14274017676|BRAD|BROWN|448127|810,00|22.08.2011|11:04|0806427488476|06
CITY HOSPITAL|04680072124|CITY|00614842|16218778886|PETER|BALSON|862626|12,00|24.08.2011|14:16|0806062177008|06
CITY HOSPITAL|04680072124|CITY|00614846|14607666866|GEORGE|LOUIS|688811|40,10|24.08.2011|08:48|0806424172468|06
CITY HOSPITAL|04680072124|CITY|00614846|14607666866|GEORGE|LOUIS|460481|42,64|24.08.2011|08:47|0806424172468|06
CITY HOSPITAL|04680072124|CITY|00614860|18460662462|JR|TEPE|404622|44,16|22.04.2011|20:08|0806446446866|06
CITY HOSPITAL|04680072124|CITY|00614840|47207688618|BARRY|HRAN|402886|42,40|27.08.2011|11:12|0806478768007|06
CITY HOSPITAL|04680072124|CITY|00614847|42161048612|TOM|HIGGS|148640|12,00|06.06.2011|08:18|0806068076700|06
CITY HOSPITAL|04680072124|CITY|00614846|42161048612|TOM|HIGGS|208847|12,00|06.06.2011|08:46|0806068076700|06-----
NLS_DATE_FORMAT : DD/MM/RRRR-----
NOTE:
I tried :
TARIH CHAR date_format DATE mask "DD/MM/RRRR" , same error.
Edited by: 904386 on Jan 24, 2012 6:42 AM

I have no experience with external tables but my understanding is that they are described the same was as you would describe a file to be loaded by SQL Loader. When ever I create a sql loader control file to load a date column I use syntax like this:
DATE_M                                             "TO_DATE(:DATE_M,'dd.mm.yyyy')",Not sure if that will work for external tables or not.

Create an external table based on an XML file

Hi ,
I am trying to create an external table based on XML file .Is it possible to do in OWB ?If Yes .Can someone please let me know the same .
I was successfully able to create an external table based on a flat file which is of the extension *.txt.
I was even able to load an XML file to a normal
table .How do we do it to an external table?
Thanks,
Manjula

Hello Manjula,
Is it possible for you to spare sometime to list down the steps to be followed for creating External Table using flat file *.txt?
I am trying to do it but not getting through it.
Thanks in Advance,
Ripesh

Selective load in external table

Hi all,
I need to load data into my external table based on conditions. If i use the 'load when' im able to load the data into the table but my requirement is to load data into different columns of the same table based on the conditional "when" clause.
This is equivalent to wht we do in .ctl file using loader..Th scenario being:
LOAD DATA
INFILE '/m018/applmgr/ERP1/modap/1.0.0/bin/BNKSTMT260508.txt'
APPEND
INTO TABLE
MODAP_IN_BNK_RECON_TBL
WHEN record_type = '01'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(RECORD_TYPE POSITION(1)
,SENDER_IDENTI
,RECEIVER_IDENTI
     ,FILE_CREATION_DATE DATE "YYMMDD" NULLIF FILE_CREATION_DATE=BLANKS
     ,FILE_CREATION_TIME
     ,FILE_SEQ_NO
     ,RECORD_LENGTH
     ,BLOCKING_FACTOR
     ,VERSION_NO
INTO TABLE
MODAP_IN_BNK_RECON_TBL
WHEN record_type = '02'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(RECORD_TYPE POSITION(1) "TRIM(:RECORD_TYPE)"
,RECEIVER_IDENTI "TRIM(:RECEIVER_IDENTI)"
,ORIGINATOR_IDENTI "TRIM(:ORIGINATOR_IDENTI)"
     ,GROUP_STATUS "TRIM(:GROUP_STATUS)"
     ,AS_OF_DATE DATE "YYMMDD" NULLIF AS_OF_DATE=BLANKS
     ,AS_OF_TIME "TRIM(:AS_OF_TIME)"
     ,CURRENCY_CODE "TRIM(:CURRENCY_CODE)"
I want to replicate the same in external table. Please suggest me a suitable solution for this at the earliest.
Regards,
Balaji V

Here is a sample I used:
DROP TABLE myschema.mytable;
CREATE TABLE myschema.mytable (
"ID" VARCHAR2(12),
"RESP_DOB" DATE,
"RESP_FAM_REL" VARCHAR2(5),
"PET_FIRST_NAME" VARCHAR2(11),
"PET_MIDDLE_NAME" VARCHAR2(11),
"PET_LAST_NAME" VARCHAR2(17),
"PET_SUFFIX" VARCHAR2(1)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY "MY_DIRECTORY"
ACCESS PARAMETERS(
RECORDS DELIMITED BY NEWLINE
BADFILE 'the_BAD.TXT'
DISCARDFILE 'the_DISCARD.TXT'
LOGFILE 'the_LOG.TXT'
FIELDS
MISSING FIELD VALUES ARE NULL
ID POSITION(1:12),
RESP_DOB POSITION(98:105) DATE MASK "mmddyyyy",
RESP_FAM_REL POSITION(106:110) NULLIF RESP_FAM_REL=".",
PET_FIRST_NAME POSITION(111:121) NULLIF PET_FIRST_NAME=".",
PET_MIDDLE_NAME POSITION(122:132) NULLIF PET_MIDDLE_NAME=".",
PET_LAST_NAME POSITION(133:149) NULLIF PET_LAST_NAME=".",
PET_SUFFIX POSITION(150:150) NULLIF PET_SUFFIX="."
LOCATION ('the_file.txt')
);

External Table Problem

Hi Guys,
Need your help regarding external tables, I'm using Oracle 10g. I have tried using external tables and almost all of them works except for one. It has a null data on the last field.
Kindly check what am I missing.
CREATE TABLE X_MHC_FCSDOC2PROV_02
DOCTOR_ID VARCHAR2(100 BYTE),
PROVIDER_ID VARCHAR2(100 BYTE),
JOIN_DATE VARCHAR2(100 BYTE),
EXIT_DATE VARCHAR2(100 BYTE),
ROOM_NUMBER VARCHAR2(100 BYTE),
AGREEMENT_NUMBER VARCHAR2(100 BYTE),
CP1_NAME VARCHAR2(100 BYTE),
CP1_BUSINESS_TEL VARCHAR2(100 BYTE),
CLINIC_HOUR_MON VARCHAR2(100 BYTE),
CLINIC_HOUR_TUE VARCHAR2(100 BYTE),
CLINIC_HOUR_WED VARCHAR2(100 BYTE),
CLINIC_HOUR_THU VARCHAR2(100 BYTE),
CLINIC_HOUR_FRI VARCHAR2(100 BYTE),
CLINIC_HOUR_SAT VARCHAR2(100 BYTE),
CLINIC_HOUR_SUN VARCHAR2(100 BYTE)
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY MEDILINK_DATA
ACCESS PARAMETERS
records delimited by "\n"
fields missing field values are null
badfile medilink_data_log:'X_MHC_FCSDOC2PROV_02.bad'
LOGFILE medilink_data_log:'X_MHC_FCSDOC2PROV_02.log'
discardfile medilink_data_log:'X_MHC_FCSDOC2PROV_02.dsc'
fields terminated BY '|'
reject ROWS WITH ALL NULL fields
LOCATION (MEDILINK_DATA:'medlnk-docToProv.txt')
REJECT LIMIT UNLIMITED
select * from x_mhc_fcsdoc2prov_02
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "badfile": expecting one of: "column, exit, (, reject"
KUP-01007: at line 3 column 1
ORA-06512: at "SYS.ORACLE_LOADER", line 19
Here is sample of the data.
medlnk-docToProv.txt
DR026556|HO008051|11/29/2006 0:0:0|12/31/9999 0:0:0||10095|ANICETA R FUENTENEGRA |2552821|||||||
DR026556|HO001322|11/29/2006 0:0:0|12/31/9999 0:0:0||10095|ANECITA FUENTENEGRA |2341520|1400 - 1730|1400 - 1730|1400 - 1730||1400 - 1730||
* The last field is null
Your help is very much appreciated. I've been researching for days now on how to resolve this issue but still I can't figure it out.
Thanks.
Best Regards,
Earl Del Rosario

There was syntax problem.
Try this
CREATE TABLE X_MHC_FCSDOC2PROV_02
DOCTOR_ID VARCHAR2(100 BYTE),
PROVIDER_ID VARCHAR2(100 BYTE),
JOIN_DATE VARCHAR2(100 BYTE),
EXIT_DATE VARCHAR2(100 BYTE),
ROOM_NUMBER VARCHAR2(100 BYTE),
AGREEMENT_NUMBER VARCHAR2(100 BYTE),
CP1_NAME VARCHAR2(100 BYTE),
CP1_BUSINESS_TEL VARCHAR2(100 BYTE),
CLINIC_HOUR_MON VARCHAR2(100 BYTE),
CLINIC_HOUR_TUE VARCHAR2(100 BYTE),
CLINIC_HOUR_WED VARCHAR2(100 BYTE),
CLINIC_HOUR_THU VARCHAR2(100 BYTE),
CLINIC_HOUR_FRI VARCHAR2(100 BYTE),
CLINIC_HOUR_SAT VARCHAR2(100 BYTE),
CLINIC_HOUR_SUN VARCHAR2(100 BYTE)
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY EXT_DIR
ACCESS PARAMETERS
RECORDS DELIMITED BY NEWLINE
badfile EXT_DIR:'X_MHC_FCSDOC2PROV_02.bad'
LOGFILE EXT_DIR:'X_MHC_FCSDOC2PROV_02.log'
discardfile EXT_DIR:'X_MHC_FCSDOC2PROV_02.dsc'
FIELDS TERMINATED BY '|'
MISSING FIELD VALUES ARE NULL
reject ROWS WITH ALL NULL fields )
LOCATION (EXT_DIR:'medlnk-docToProv.txt')
REJECT LIMIT UNLIMITED
Details of directory object EXT_DIR is as belows
SQL> SElect * from all_directories where directory_name = 'EXT_DIR';
OWNER DIRECTORY_NAME
DIRECTORY_PATH
SYS EXT_DIR
c:\

How Can We Achieve External Table Authentication

Hai Guru's
I want Know The External Table Authentication By Step By Step
Thanks

Hi Friend,
For External table authentication you can follow this link:
http://obieeblog.wordpress.com/2009/06/18/obiee-security-enforcement-%E2%80%93-external-database-table-authorization/
Thanks
Don

11g hybrid authentication / authorization: WLS plus external table

I've implemented external table authentication / authorization in 11g. Now I'd like to add a twist.
I have an external table containing users B, C, and D. That external table contains all of the columns I need for authentication (including a clear text password) and for authorization (roles, log level, a dynamic table name, and so forth). I have authentication in one initialization block, authorization in another. Everything works fine. I can log in as B, C, or D and see exactly what I'm supposed to see, based on the ROLES.
The clear text passwords are generally not a problem, because this is a training instance and almost all of the passwords are the same. However, I want to add a user whose password should not be held in clear text. For that reason, I'd like to add that user into WLS. I've done that, and I'm able to log in to OBIEE. After confirming that I could log in to OBIEE with user A from the WLS, I added User A to the external table, left its password field blank, and filled in the other columns (roles, loglevel, etc...) that I need to assign into session variables.
Here's the problem: the authorization init block properly assigns ALL session variables for users B, C, and D. It assigns all session varaibles EXCEPT the ROLES variable for user A. I've confirmed this by creating an Answers analysis that shows me the values of the session variables. The ROLES session variable for user A shows "authenticated-role;BIConsumer;AuthenticatedUser". For all other users (those who are authenticated using the clear text passwords in the external table) the ROLES variable is populated correctly, based on the values in the ROLES column in the external table. In short, the authorization init block is properly assigning the ROLES session variable only for those users that were authenticated using the authentication init block, but is assigning all other session variables correctly for all users, even the one in WLS.
Here's my authentication init block code:
select bi_user
from bi_auth_ldap
where bi_user = ':USER'
and bi_user_pwd = ':PASSWORD'
Here's the authorization init block code:
select roles, bi_user_name, to_number(loglevel,0), channel_tbl
from bi_auth_ldap
where bi_user = ':USER'
(returned results are assigned into ROLES, DISPLAYNAME, LOGLEVEL, and CHANNEL_TBL session variables, respectively)
It feels like the ROLES session variable is populated in conjuction with the user logging on and being authenticated via WLS, and that the initialization block isn't able to overwrite that variable. Can an OBIEE developer confirm that for us, please? Once set in WLS, is it not possible to overwrite the ROLES session variable with SQL from an initialization block? If it IS possible, can you post some code that will accomplish it?
Thanks!

It occurs to me that Oracle's support model is a fantastic way to make money. Let's see, I wonder if I could become a billionaire doing this:
Create some software. Sell that software. Then, charge customers several thousand MORE dollars, year after year, plus about $60 per bug, so that they have the right to report MY bugs to me. Yeah, that's the ticket - people PAYING for the right to report bugs to me. Oh, and if more than one person reports the same bug, I get to keep ALL of the money from ALL of them.
Let's summarize, make sure I haven't missed something: You buy my software, you PAY ME additionally to report MY bugs to me, I don't necessarily have to fix the bugs (but I keep your money whether I fix it or not), and I can collect multiple times from different people who report the same bug.
Sweeeeeeet.........
Billionaire Acres, here I come!

Error: while Selecting External table

Hi everybody,
When i Select an external table i am getting this error. The file is like this:
229|1|506460|SIGROUP |4890|100|0|0|10:31:01|2007/12/17|M009|20191395001|L|B|12|CLIENT|INE547A01012|10:31:00|
229|1|506460|SIGROUP |4900|900|0|0|10:31:01|2007/12/17|M009|20191395001|L|B|13|CLIENT|INE547A01012|10:31:00|
229|1|500407|SWARAJENG |21400|300|0|0|10:33:28|2007/12/17|OWN|20191397001|L|B|154|OWN|INE277A01016|10:33:28|
I had created the Table like this:
SQL> CREATE TABLE TEMP_SAUDA
2 (S_A VARCHAR2(20),
3 S_TYPE VARCHAR2(20),
4 S_CO VARCHAR2(20),
5 S_CONAME VARCHAR2(40),
6 S_RATE NUMBER,
7 S_QTY NUMBER,
8 S_G NUMBER,
9 S_H NUMBER,
10 S_TIME TIMESTAMP WITH TIME ZONE,
11 S_DATE DATE,
12 S_PCODE VARCHAR2(20),
13 S_SETNO VARCHAR2(20),
14 S_M VARCHAR2(20),
15 S_N VARCHAR2(20),
16 S_O VARCHAR2(20),
17 S_CLIENTOWN VARCHAR2(10),
18 S_ISIN VARCHAR2(12),
19 S_ORDER_TIME TIMESTAMP WITH TIME ZONE
20 )
21 ORGANIZATION EXTERNAL
22 (TYPE oracle_loader
23 DEFAULT DIRECTORY BSE17122007
24 ACCESS PARAMETERS
25 (RECORDS DELIMITED BY NEWLINE
26 FIELDS
27 (
28 S_A CHAR(20),
29 S_TYPE CHAR(20),
30 S_CO CHAR(20),
31 S_CONAME CHAR(20),
32 S_RATE CHAR(20),
33 S_QTY CHAR(20),
34 S_G CHAR(20),
35 S_H CHAR(20),
36 S_TIME CHAR(35) date_format TIMESTAMP WITH TIMEZONE mask "DD-MON-RR HH.MI.SSXFF AM TZH:TZM
37 S_DATE CHAR(22) date_format DATE mask "mm/dd/yyyy hh:mi:ss ",
38 S_PCODE CHAR(20),
39 S_SETNO CHAR(20),
40 S_M CHAR(20),
41 S_N CHAR(20),
42 S_O CHAR(20),
43 S_CLIENTOWN CHAR(20),
44 S_ISIN CHAR(20),
45 S_ORDER_TIME date_format TIMESTAMP WITH TIMEZONE mask "DD-MON-RR HH.MI.SSXFF AM TZH:TZM"
46 )
47 )
48 location (BSE17122007:'BR171207.DAT')
49 )
50 ;
Table created.
SQL> SELECT * FROM TEMP_SAUDA;
SELECT * FROM TEMP_SAUDA
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "date_format": expecting one of: "binary_double,
binary_float, comma, char, date, defaultif, decimal, double, float, integer, (,
nullif, oracle_date, oracle_number, position, raw, recnum, ), unsigned,
varrawc, varchar, varraw, varcharc, zoned"
KUP-01007: at line 21 column 14
ORA-06512: at "SYS.ORACLE_LOADER", line 19
Is there any mistake in this table creation.
what i have to declare to the time format if the format in the file id hh:mm:ss
Thank u...!
Ravi

The output you posted is completely wrong, I could not even create the table without errors.
Try with this.
CREATE TABLE TEMP_SAUDA
(S_A VARCHAR2(20),
S_TYPE VARCHAR2(20),
S_CO VARCHAR2(20),
S_CONAME VARCHAR2(40),
S_RATE NUMBER,
S_QTY NUMBER,
S_G NUMBER,
S_H NUMBER,
S_TIME TIMESTAMP WITH TIME ZONE,
S_DATE DATE,
S_PCODE VARCHAR2(20),
S_SETNO VARCHAR2(20),
S_M VARCHAR2(20),
S_N VARCHAR2(20),
S_O VARCHAR2(20),
S_CLIENTOWN VARCHAR2(10),
S_ISIN VARCHAR2(12),
S_ORDER_TIME TIMESTAMP WITH TIME ZONE
ORGANIZATION EXTERNAL
(TYPE oracle_loader
DEFAULT DIRECTORY BSE17122007
ACCESS PARAMETERS
(RECORDS DELIMITED BY NEWLINE
FIELDS terminated by "|"
S_A CHAR(20),
S_TYPE CHAR(20),
S_CO CHAR(20),
S_CONAME CHAR(20),
S_RATE CHAR(20),
S_QTY CHAR(20),
S_G CHAR(20),
S_H CHAR(20),
S_TIME CHAR(8) date_format TIMESTAMP WITH TIMEZONE mask "HH.MI.SSXFF AM TZH:TZM",
S_DATE CHAR(10) date_format DATE mask "yyyy/mm/dd",
S_PCODE CHAR(20),
S_SETNO CHAR(20),
S_M CHAR(20),
S_N CHAR(20),
S_O CHAR(20),
S_CLIENTOWN CHAR(20),
S_ISIN CHAR(20),
S_ORDER_TIME char(8) date_format TIMESTAMP WITH TIMEZONE mask "HH.MI.SSXFF AM TZH:TZM"
location (BSE17122007:'BR171207.DAT')
;With this you get:
SQL> col s_time format a40
SQL> col s_date format a40
SQL> col s_order_time format a40
SQL> r
1* select s_time,s_date,s_order_time from temp_sauda
S_TIME                                   S_DATE                                   S_ORDER_TIME
01-JAN-08 10.31.01.000000 AM +00:00      17.DEC.2007 00:00:00                     01-JAN-08 10.31.00.000000 AM +00:00
01-JAN-08 10.31.01.000000 AM +00:00      17.DEC.2007 00:00:00                     01-JAN-08 10.31.00.000000 AM +00:00
01-JAN-08 10.33.28.000000 AM +00:00      17.DEC.2007 00:00:00                     01-JAN-08 10.33.28.000000 AM +00:00Be aware that your file does not contain date information for the time fields, so as you see above it is defaulted to 01-JAN-08 for the S_TIME and S_ORDER_TIME column.

Error while creating external table

Hi i tried to create external table. The table is created but while selecting that table it is throwing below errors
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file Countries1.txt in EXT_TABLES not found
ORA-06512: at "SYS.ORACLE_LOADER", line 19I've created temp directory in window under oracle directory " C:\oracle\product\10.2.0\temp"
In the temp directory i've a text file countries1.txt
the text file has the below information
ENG,England,English
SCO,Scotland,English
IRE,Ireland,English
WAL,Wales,WelshI've connected to system user and created one directory and granted the read and write permissions to user SCOTT.
SQL> create or replace directory ext_tables as 'C:\oracle\product\10.2.0\temp\';
Directory created.
SQL> grant read,write on directory ext_tables to scott;
Grant succeeded.The creation of external table query is
CREATE TABLE countries_ext (
country_code      VARCHAR2(5),
country_name      VARCHAR2(50),
country_language VARCHAR2(50)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY ext_tables
ACCESS PARAMETERS (
    RECORDS DELIMITED BY NEWLINE
    FIELDS TERMINATED BY ','
    MISSING FIELD VALUES ARE NULL
      country_code      CHAR(5),
      country_name      CHAR(50),
      country_language CHAR(50)
LOCATION ('Countries1.txt')
PARALLEL 5
REJECT LIMIT UNLIMITED;And the error is
SQL> select *from countries_ext;
select *from countries_ext
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file Countries1.txt in EXT_TABLES not found
ORA-06512: at "SYS.ORACLE_LOADER", line 19
SQL> Please help me in this

You are missing something. Most probably the file does not exists in your specified path. This is working in my 10.2.0.3
Step1: Check the file is actually there.
C:\oracle\product\10.2.0>mkdir temp
C:\oracle\product\10.2.0>cd temp
C:\oracle\product\10.2.0\temp>dir
Volume in drive C is C_Drive
Volume Serial Number is 8A93-1441
Directory of C:\oracle\product\10.2.0\temp
07/30/2011 12:00 PM    <DIR>          .
07/30/2011 12:00 PM    <DIR>          ..
07/30/2011 12:00 PM                79 countries1.txt
               1 File(s)             79 bytes
               2 Dir(s) 50,110,582,784 bytes free
C:\oracle\product\10.2.0\temp>type countries1.txt
ENG,England,English
SCO,Scotland,English
IRE,Ireland,English
WAL,Wales,Welsh
C:\oracle\product\10.2.0\temp>Step 2: Creating the directory object.
SQL> show user
USER is "SYS"
SQL> SELECT * FROM v$version;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Prod
PL/SQL Release 10.2.0.3.0 - Production
CORE    10.2.0.3.0      Production
TNS for 32-bit Windows: Version 10.2.0.3.0 - Production
NLSRTL Version 10.2.0.3.0 - Production
SQL> create or replace directory ext_tables as 'C:\oracle\product\10.2.0\temp';
Directory created.
SQL> grant read,write on directory ext_tables to scott;
Grant succeeded.
SQL>Step 3: Table definition.
C:\>sqlplus scott@orclsb/tiger
SQL*Plus: Release 10.1.0.4.2 - Production on Sat Jul 30 12:04:24 2011
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production
With the Partitioning, OLAP and Data Mining options
SQL> CREATE TABLE countries_ext (
2    country_code      VARCHAR2(5),
3    country_name      VARCHAR2(50),
4    country_language VARCHAR2(50)
5 )
6 ORGANIZATION EXTERNAL (
7    TYPE ORACLE_LOADER
8    DEFAULT DIRECTORY ext_tables
9    ACCESS PARAMETERS (
10      RECORDS DELIMITED BY NEWLINE
11      FIELDS TERMINATED BY ','
12      MISSING FIELD VALUES ARE NULL
13      (
14        country_code      CHAR(5),
15        country_name      CHAR(50),
16        country_language CHAR(50)
17      )
18    )
19    LOCATION ('Countries1.txt')
20 )
21 PARALLEL 5
22 REJECT LIMIT UNLIMITED;
Table created.
SQL> SELECT * FROM countries_ext;
COUNT COUNTRY_NAME
COUNTRY_LANGUAGE
ENG   England
English
SCO   Scotland
English
IRE   Ireland
English
COUNT COUNTRY_NAME
COUNTRY_LANGUAGE
WAL   Wales
Welsh

Error while selecting date from external table

Hello all,
I am getting the follwing error while selecting data from external table. Any idea why?
SQL> CREATE TABLE SE2_EXT (SE_REF_NO VARCHAR2(255),
2        SE_CUST_ID NUMBER(38),
3        SE_TRAN_AMT_LCY FLOAT(126),
4        SE_REVERSAL_MARKER VARCHAR2(255))
5 ORGANIZATION EXTERNAL (
6    TYPE ORACLE_LOADER
7    DEFAULT DIRECTORY ext_tables
8    ACCESS PARAMETERS (
9      RECORDS DELIMITED BY NEWLINE
10      FIELDS TERMINATED BY ','
11      MISSING FIELD VALUES ARE NULL
12      (
13        country_code      CHAR(5),
14        country_name      CHAR(50),
15        country_language CHAR(50)
16      )
17    )
18    LOCATION ('SE2.csv')
19 )
20 PARALLEL 5
21 REJECT LIMIT UNLIMITED;
Table created.
SQL> select * from se2_ext;
SQL> select count(*) from se2_ext;
select count(*) from se2_ext
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04043: table column not found in external source: SE_REF_NO
ORA-06512: at "SYS.ORACLE_LOADER", line 19

It would appear that you external table definition and the external data file data do not match up. Post a few input records so someone can duplicate the problem and determine the fix.
HTH -- Mark D Powell --

Error while querying the external tables in 9i

i am working on a project on Oracle 9iR2 on Linux AS 3.0, i am in urgent need to port my data in text files for using in the DB
I have created an external table using the following script. While executing a query it gives the following error. Pls Hlp
create table ap_info_ex
     (ser_no number(7),
     no_of_sps number(1),
     hpy_liq_mm number(2),
     rmp_rnk_cd number(1),
     ir number(3),
     dps_flg varchar2(1))
organization external
(type oracle_loader
     default directory aps_jul
     access parameters (records delimited by newline
                    fields terminated by ","
                    (sno char,
                    no_sps char,
                    hpy_mm char,
                    rmp_rnk char,
                    ir char,
                    dpsflg char))
location ('info.txt'));
Table has been created successfully
but when i execute a query say
select * from ap_info_ex;
THIS IS THE ERROR MESSAGE I AM GETTING
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04063: unable to open log file AP_INFO_EX_6151.log
OS error Permission denied
ORA-06512: at "SYS.ORACLE_LOADER", line 14
ORA-06512: at line 1
hope you would help me. i am greateful for that.

Arun,
Here are the things you could do.
1. You may want to specify a logfile in your access parameters using the command 'logfile'.
For example,
logfile 'info.log'. You could create the file in advance and provide appropriate permissions to the operating system user(In unix, I use the commands 'touch' and 'chmod').
2. You could specify the command
'NOLOGFILE' in your access parameters.
3. Provide permissions for the user to create files in the folder associated with the directory object 'aps_jul'.

Join large external external Tables (best practice ?)

Hi there,
I have three external tables in a master-detail relation: table A (10000000 rows) is master of table B (20000000 rows), Table B is master of table C (100000000 rows). Can you tell my the best way:
- directly join the external tables, or
- copy the external tables into tables, create an index, and join them
What is more efficient, and why ?
Thanks for your help and ideas.
Gerhard.

In general, if the joins you are doing can benefit from indexes, you will want to copy the data to database tables. If the joins will end up doing full table scans anyway, it will matter far less.
For data integrity, you will likely also want to be able to enforce foreign key constraints.
Justin
Distributed Database Consulting, Inc.
http://www.ddbcinc.com/askDDBC

How to import external table, which exist in export dump file.

My export dump file has one external table. While i stated importing into my developement instance , I am getting the error "ORA-00911: invalid character".
The original definition of the extenal table is as given below
CREATE TABLE EXT_TABLE_EV02_PRICEMARTDATA
EGORDERNUMBER VARCHAR2(255 BYTE),
EGINVOICENUMBER VARCHAR2(255 BYTE),
EGLINEITEMNUMBER VARCHAR2(255 BYTE),
EGUID VARCHAR2(255 BYTE),
EGBRAND VARCHAR2(255 BYTE),
EGPRODUCTLINE VARCHAR2(255 BYTE),
EGPRODUCTGROUP VARCHAR2(255 BYTE),
EGPRODUCTSUBGROUP VARCHAR2(255 BYTE),
EGMARKETCLASS VARCHAR2(255 BYTE),
EGSKU VARCHAR2(255 BYTE),
EGDISCOUNTGROUP VARCHAR2(255 BYTE),
EGREGION VARCHAR2(255 BYTE),
EGAREA VARCHAR2(255 BYTE),
EGSALESREP VARCHAR2(255 BYTE),
EGDISTRIBUTORCODE VARCHAR2(255 BYTE),
EGDISTRIBUTOR VARCHAR2(255 BYTE),
EGECMTIER VARCHAR2(255 BYTE),
EGECM VARCHAR2(255 BYTE),
EGSOLATIER VARCHAR2(255 BYTE),
EGSOLA VARCHAR2(255 BYTE),
EGTRANSACTIONTYPE VARCHAR2(255 BYTE),
EGQUOTENUMBER VARCHAR2(255 BYTE),
EGACCOUNTTYPE VARCHAR2(255 BYTE),
EGFINANCIALENTITY VARCHAR2(255 BYTE),
C25 VARCHAR2(255 BYTE),
EGFINANCIALENTITYCODE VARCHAR2(255 BYTE),
C27 VARCHAR2(255 BYTE),
EGBUYINGGROUP VARCHAR2(255 BYTE),
QTY NUMBER,
EGTRXDATE DATE,
EGLISTPRICE NUMBER,
EGUOM NUMBER,
EGUNITLISTPRICE NUMBER,
EGMULTIPLIER NUMBER,
EGUNITDISCOUNT NUMBER,
EGCUSTOMERNETPRICE NUMBER,
EGFREIGHTOUTBOUNDCHARGES NUMBER,
EGMINIMUMORDERCHARGES NUMBER,
EGRESTOCKINGCHARGES NUMBER,
EGINVOICEPRICE NUMBER,
EGCOMMISSIONS NUMBER,
EGCASHDISCOUNTS NUMBER,
EGBUYINGGROUPREBATES NUMBER,
EGINCENTIVEREBATES NUMBER,
EGRETURNS NUMBER,
EGOTHERCREDITS NUMBER,
EGCOOP NUMBER,
EGPOCKETPRICE NUMBER,
EGFREIGHTCOSTS NUMBER,
EGJOURNALBILLINGCOSTS NUMBER,
EGMINIMUMORDERCOSTS NUMBER,
EGORDERENTRYCOSTS NUMBER,
EGRESTOCKINGCOSTSWAREHOUSE NUMBER,
EGRETURNSCOSTADMIN NUMBER,
EGMATERIALCOSTS NUMBER,
EGLABORCOSTS NUMBER,
EGOVERHEADCOSTS NUMBER,
EGPRICEADMINISTRATIONCOSTS NUMBER,
EGSHORTPAYMENTCOSTS NUMBER,
EGTERMCOSTS NUMBER,
EGPOCKETMARGIN NUMBER,
EGPOCKETMARGINGP NUMBER,
EGWEIGHTEDAVEMULTIPLIER NUMBER
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY EV02_PRICEMARTDATA_CSV_CON
ACCESS PARAMETERS
LOCATION (EV02_PRICEMARTDATA_CSV_CON:'VPA.csv')
REJECT LIMIT UNLIMITED
NOPARALLEL
NOMONITORING;
While importing , when i seen the log file , it is failing the create the external table. Getting the error "ORA-00911: invalid character".
Can some one suggest how to import external tables
Addressing this issue will be highly appriciated.
Naveen

Hi Srinath,
When i observed the create table syntax of external table from import dump log file, it show few lines as below. I could not understand these special characters. And create table definationis failing with special character viz ORA-00911: invalid character
ACCESS PARAMETERS
LOCATION (EV02_PRICEMARTDATA_CSV_CON:'VPA.csv').
I even observed the create table DDL from TOAD. It is same as i mentioned earlier
Naveen

External Tables preprocessor in 11.2 as a tool for DBAs

Similar Messages

Maybe you are looking for