External table and newline character
I have an external table which is just a csv file, records delimited by newline and fields terminated by ','. However, the last column has a newline character appended to it. Is there an option in the external table syntax which will trim this?
Try setting 'REJECT ROWS WITH ALL NULL FIELDS'
http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96652/ch12.htm#1011062
Similar Messages
-
Request: PL/SQL, External Table and SQL Loader
I see lately Questions asked in SQL and PL/SQL forum regarding SQL Loader and External Table are being moved to {forum:id=732}.
Being an PL/SQL developer for some time now i feel External Table (and if i may, SQL Loader and DBMS_DATAPUMP) are very much an integral part of a PL/SQL development and the question related to these topics are well suited in SQL and PL/SQL forum. Even in SQL and PL/SQL FAQ we have exclusive content that discuss on these topics {message:id=9360007}
So i would like to request the moderators to consider not moving such questions out of the SQL and PL/SQL forum.
Thanks,
Karthick.Karthick_Arp wrote:
I see lately Questions asked in SQL and PL/SQL forum regarding SQL Loader and External Table are being moved to {forum:id=732}.
Being an PL/SQL developer for some time now i feel External Table (and if i may, SQL Loader and DBMS_DATAPUMP) are very much an integral part of a PL/SQL development and the question related to these topics are well suited in SQL and PL/SQL forum. Even in SQL and PL/SQL FAQ we have exclusive content that discuss on these topics {message:id=9360007}
So i would like to request the moderators to consider not moving such questions out of the SQL and PL/SQL forum.
Thanks,
Karthick.Not sure which moderators are moving them... cos it ain't me. I'm quite happy to leave those there. -
SQL LOADER , EXTERNAL TABLE and ODBS DATA SOURCE
hello
Can any body help loading data from dbase file .dbt to an oracle 10g table.
I tried last day with SQL LOADER, EXTERNAL table , and ODBC data source.
Why all of these utilities still failing to solve my problem ?
Is there an efficient way to reach this goal ?
Thanks in advanceexport the dbase data file to text file,
then you have choice of using either sql loader or external table option to use.
regards -
External Table and Direct path load
Hi,
I was just playing with Oracle sql loader and external table features. few things which i ovserved are that data loading through direct path method of sqlloader is much faster and takes much less hard disk space than what external table method takes. here are my stats which i found while data loading:
For Direct Path: -
# OF RECORDS.............TIME...................SOURCE FILE SIZE...................DATAFILE SIZE(.dbf)
478849..........................00:00:43.53...................108,638 KB...................142,088 KB
957697..........................00:01:08.81...................217,365 KB...................258,568 KB
1915393..........................00:02:54.43...................434,729 KB...................509,448 KB
For External Table: -
# OF RECORDS..........TIME...................SOURCE FILE SIZE...................DATAFILE SIZE(.dbf)
478849..........................00:02:51.03...................108,638 KB...................966,408 KB
957697..........................00:08:05.32...................217,365 KB...................1,930,248 KB
1915393..........................00:17:16.31...................434,729 KB...................3,860,488 KB
1915393..........................00:23:17.05...................434,729 KB...................3,927,048 KB
(With PARALLEL)
i used same files for testing and all other conditions are similar also. In my case datafile is autoextendable, hence, as par requirement its size is automatically increased and hard disk space is reduced thus.The issue is that, is this an expected behaviour? and why in case of external tables such a large hard disk space is used when compared to direct path method? Performance of external table load is also very bad when compared to direct path load.
one more thing is that while using external table load with PARALLEL option, ideally, it should take less time. But what i actually get is more than what the time was without PARALLEL option.
In both the cases i am loading data from the same file to the same table (once using direct path and once using external table). before every fresh loading i truncate my internal table into which data was loaded.
any views??
Deep
Message was edited by:
DeepThanx to all for your suggestions.
John, my scripts are as follows:
for external table:
CREATE TABLE LOG_TBL_LOAD
(COL1 CHAR(20), COL2 CHAR(2), COL3 CHAR(20), COL4 CHAR(400),
COL5 CHAR(20), COL6 CHAR(400), COL7 CHAR(20), COL8 CHAR(20),
COL9 CHAR(400), COL10 CHAR(400))
ORGANIZATION EXTERNAL
(TYPE ORACLE_LOADER
DEFAULT DIRECTORY EXT_TAB_DIR
ACCESS PARAMETERS
(RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY WHITESPACE OPTIONALLY ENCLOSED BY '"' MISSING FIELD VALUES ARE NULL
LOCATION ('LOGZ3.DAT')
REJECT LIMIT 10;
for loading i did:
INSERT INTO LOG_TBL (COL1, COL2, COL3, COL4,COL5, COL6,
COL7, COL8, COL9, COL10)
(SELECT COL1, COL2, COL3, COL4, COL5, COL6, COL7, COL8,
COL9, COL10 FROM LOG_TBL_load_1);
for direct path my control file is like this:
OPTIONS (
DIRECT = TRUE
LOAD DATA
INFILE 'F:\DATAFILES\LOGZ3.DAT' "str '\n'"
INTO TABLE LOG_TBL
APPEND
FIELDS TERMINATED BY WHITESPACE OPTIONALLY ENCLOSED BY '"'
(COL1 CHAR(20),
COL2 CHAR(2),
COL3 CHAR(20),
COL4 CHAR(400),
COL5 CHAR(20),
COL6 CHAR(400),
COL7 CHAR(20),
COL8 CHAR(20),
COL9 CHAR(400),
COL10 CHAR(400))
and ya, i have used same table in both the situations. after loading once i used to truncate my table, LOG_TBL. i used the same source file, LOGZ3.DAT.
my tablespace USERS is loaclly managed.
thanks -
Hi.
I have this XML-structure:
<order>
<orderhead>
<orderno>123</orderno>
<orderline>
<name>name_of_what_i_order</name>
</orderline>
<orderline>
<name>name_of_what_i_order</name>
</orderline>
</orderhead>
</order>
How can I get the <orderhead><orderno> when I'm making an external table for the <ordeline> elements? I use 'records delimited by "</orderline>"' to get all the orderlines in one tablerow each. But how can I insert the orderno in each of these rows?Hi,
You need to practice or test by youself. Fortunately, I have a posting at http://www.oraclepoint.com/topic.php?filename=98&extra=page%3D1, which is a similar case I've done couple of years ago.
What I did are:
1. formating XML
2. Load to oracle table by using SQL*Loader
3. Parse elements one by one from entire xml large object from within oracle table
4. insert the parsed elements to oracle table
Hope it helps. By the way, please register and then log in at oraclepoint.com to get the solution package becuase it's only available for members.
R.Wang
http://www.oraclepoint.com -
External table and discarded rows
I need to load a table mapping an external table.
Sometimes some rows are discarded due to constraints violation or bad values in the file.
How is it possible to identify the discarded rows?
Is it possible to automatically create a file containing the
discarded records?
Thanks, Gio.Before specify a location, you have to create/define directories in the database and set permissions on them.
Add the location(s) to OWB and register them.
Right click on the external table, choose CONFGURE.
Navigate to ACCESS SPECIFICATION. set the locations here. -
All,
Is it possible to specify a URL as the location of a file for an external table? (e.g. http://www.somesite.com/somedir/somefile.csv)
For some organizations, getting files onto the actual Oracle server is a difficult process because they tend to be managed externally, often by an entirely different group from the ones doing the DBA work. NFS, FTP drop-zones, etc. all seem to be considered security risks. Being able to pull directly across a protocol would be a very nice feature.You need to specify a directory object from where the file is being read by external table. May be you can try creating a directory object which maps to the directory in the network. It may work if the directory is accessible from the server.
-
External table and error: ORA-01036: illegal variable name/number
using the following script I get the referenced error:
create table dol_sch_c_part2
(DLN NUMBER(14) ,
PAGE_ID VARCHAR2(20),
PAGE_SEQ VARCHAR2(20),
PAGE_ROW_NUM VARCHAR2(20),
ROW_NUM VARCHAR2(20),
IMAGE_FORM_ID VARCHAR2(20),
PROVIDER_TERM_01_NAME VARCHAR2(35),
PROVIDER_TERM_01_EIN VARCHAR2(9) ,
PROVIDER_TERM_01_POSITION VARCHAR2(25),
PROVIDER_TERM_01_STR_ADDRESS VARCHAR2(35),
PROVIDER_TERM_01_CITY VARCHAR2(20),
PROVIDER_TERM_01_STATE VARCHAR2(2),
PROVIDER_TERM_01_ZIP_CODE VARCHAR2(9),
PROVIDER_TERM_01_PHONE_NUM VARCHAR2(10),
PROVIDER_TERM_01_TEXT VARCHAR2(250)
ORGANIZATION EXTERNAL
( type oracle_loader
default directory data_dir
access parameters
RECORDS FIXED 510
FIELDS
DLN(1:14) char(14),
PAGE_ID(15:34) char(20) NULLIF PAGE_ID=BLANKS,
PAGE_SEQ(35:54) char(20) NULLIF PAGE_SEQ=BLANKS,
PAGE_ROW_NUM(55:74) char(20) NULLIF PAGE_ROW_NUM=BLANKS,
ROW_NUM(75:94) char(20) NULLIF ROW_NUM=BLANKS,
IMAGE_FORM_ID(95:114) char(20) NULLIF IMAGE_FORM_ID=BLANKS,
PROVIDER_TERM_01_NAME(115:149) char(35) NULLIF PROVIDER_TERM_01_NAME=BLANKS,
PROVIDER_TERM_01_EIN(150:158) char(9) NULLIF PROVIDER_TERM_01_EIN=BLANKS,
PROVIDER_TERM_01_POSITION(159:183) char(25) NULLIF PROVIDER_TERM_01_POSITION=BLANKS,
PROVIDER_TERM_01_STR_ADDRESS(184:218) char(35) NULLIF PROVIDER_TERM_01_STR_ADDRESS=BLANKS,
PROVIDER_TERM_01_CITY(219:238) char(20) NULLIF PROVIDER_TERM_01_CITY=BLANKS,
PROVIDER_TERM_01_STATE(239:240) char(2) NULLIF PROVIDER_TERM_01_STATE=BLANKS,
PROVIDER_TERM_01_ZIP_CODE(241:249) char(9) NULLIF PROVIDER_TERM_01_ZIP_CODE=BLANKS,
PROVIDER_TERM_01_PHONE_NUM(250:259) char(10) NULLIF PROVIDER_TERM_01_PHONE_NUM=BLANKS,
PROVIDER_TERM_01_TEXT(260:509) char(250) NULLIF PROVIDER_TERM_01_TEXT=BLANKS
location ('f_dol_sch_c_part2.txt')
I've tried it with and without the char(*) in the fields section - no change
thanks*** SCRIPT START : Session:GLEN_SELF@RAMBO(8) 8/31/2006 12:16:48 PM ***
Processing ...
create or replace directory data_dir as 'c:\dol\'
create table dol_sch_c_part2
(DLN NUMBER(14) ,
PAGE_ID VARCHAR2(20),
PAGE_SEQ VARCHAR2(20),
PAGE_ROW_NUM VARCHAR2(20),
ROW_NUM VARCHAR2(20),
IMAGE_FORM_ID VARCHAR2(20),
PROVIDER_TERM_01_NAME VARCHAR2(35),
PROVIDER_TERM_01_EIN VARCHAR2(9),
PROVIDER_TERM_01_POSITION VARCHAR2(25),
PROVIDER_TERM_01_STR_ADDRESS VARCHAR2(35),
PROVIDER_TERM_01_CITY VARCHAR2(20),
PROVIDER_TERM_01_STATE VARCHAR2(2),
PROVIDER_TERM_01_ZIP_CODE VARCHAR2(9),
PROVIDER_TERM_01_PHONE_NUM VARCHAR2(10),
PROVIDER_TERM_01_TEXT VARCHAR2(250)
ORGANIZATION EXTERNAL
( type oracle_loader
default directory data_dir
logfile 'ext_tab.log'
access parameters
RECORDS FIXED 510
FIELDS
DLN(1:14),
PAGE_ID(15:34),
PAGE_SEQ(35:54),
PAGE_ROW_NUM(55:74),
ROW_NUM(75:94),
IMAGE_FORM_ID(95:114),
PROVIDER_TERM_01_NAME(115:149),
PROVIDER_TERM_01_EIN(150:158),
PROVIDER_TERM_01_POSITION(159:183),
PROVIDER_TERM_01_STR_ADDRESS(184:218),
PROVIDER_TERM_01_CITY(219:238),
PROVIDER_TERM_01_STATE(239:240),
PROVIDER_TERM_01_ZIP_CODE(241:249),
PROVIDER_TERM_01_PHONE_NUM(250:259),
PROVIDER_TERM_01_TEXT(260:509)
location ('f_dol_sch_c_part2.txt')
create or replace directory data_dir as 'c:\dol\'
ORA-01036: illegal variable name/number
*** Script stopped due to error ***
*** SCRIPT END : Session:GLEN_SELF@RAMBO(8) 8/31/2006 12:16:49 PM *** -
Import over network link fails when table and special character.
I am trying to do an import over the network link. Below is the command I am using to do the import. In the parameter file the table name has "$" sign. Database is on AIX.
impdp "'/ as sysdba'" PARFILE=par_aml_aud_import.txt LOGFILE=test.log
PARFILE:
JOB_NAME=MONTHLY_XYZ_IMPORT
NETWORK_LINK=XYZ_IMPORT_LNK
DIRECTORY=XYZ_AUD_IMPORT
TABLE_EXISTS_ACTION=TRUNCATE
REMAP_SCHEMA=XYZ:XYZ_AUDIT
REMAP_TABLESPACE=XYZ_DATA:XYZ_AUDIT_DATA
REMAP_TABLESPACE=XYZ_INDEX:XYZ_AUDIT_INDEX
EXCLUDE=GRANT
TABLES=XYZ.DR$ABC_EFG_CTC1$N, DR$MNO_PQR_CTC1$N
I have tested this import with following options.
on the command line TABLES=\"XYZ\".\"DR\$ABC_EFG_CTC1\$N\" ---- (IN THE PARFILE)
on the command line TABLES=\"XYZ\".\"DR\$ABC_EFG_CTC1\$N\" ---- (WITHOUT USING PARFILE)
Still my import fails with below error.
======================
ORA-39166: Object ."DR$ABC_EFG_CTC1$N" was not found.
ORA-39166: Object ."DR$MNO_PQR_CTC1$N" was not found.
ORA-39166: Object XYZ was not found.
ORA-31655: no data or metadata objects selected for job
Job "SYS"."SYS_IMPORT_TABLE_01" completed with 3 error(s) at 07:46:30
I had more than 20 tables to import. All the other tables get imported except these tables which have special character. I have confirmed that these tables exists in source database.
Please advice.
Thanks
ShellyFirst hint: don't use as sysdba
http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/dp_import.htm#sthref243
Instead, create a user with (no more than necessary) export and/or import rights
user7414922 wrote:
on the command line TABLES=\"XYZ\".\"DR\$ABC_EFG_CTC1\$N\" ---- (WITHOUT USING PARFILE)Try something like tables=xyz.table\$name
or
tables='xyz.table$name'
TABLES=XYZ.DR$ABC_EFG_CTC1$N, DR$MNO_PQR_CTC1$NSecond table should probably have schema qualifier as well. -
External table and data load order
I'm unsing Oracle 10g exterbal table feature to load a text file to the database
1- What is the default order of loading the text file into the Oracle table
2- How to ensure or to change the this default behavior1- What is the default order of loading the text file into the Oracle tableTop (row#1) to bottom
2- How to ensure or to change the this default behavioruse ORDER BY clause -
Problem with a character in an external table
I have Unix Sun OS 5.8
I received 1 file with data. The problem is that this file has a character ' ' in the end file then the charge has an error. The character in the end file is in hexadecimal: 0A, 1A.
Then I have an External table and I put the the clause LOAD WHEN (COMARC != ' ') but when I edit the external table I see LOAD WHEN (COMARC != '.').
And the last record with the ' ' character doesnât filter.
Note: the character isn't a space ' '. It is the hexadecimal: 0A, 1A
What I have to put in the LOAD WHEN (COMARC != '??????????') to filter this character?I put LOAD WHEN (COMARC != x'0A1A') but I receive the error:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "single-quoted-string": expecting one of: "and, not, or, )"
KUP-01007: at line 3 column 26
ORA-06512: at "SYS.ORACLE_LOADER", line 14
ORA-06512: at line 1
If I put:
LOAD WHEN (COMARC != ' ') for example I don't received any error!
What happens?
Thanks! -
External table log and bad files
Hi, i have defined the following access parameters for my external table and set REJECT LIMIT to 0:
ACCESS PARAMETERS
RECORDS DELIMITED by NEWLINE
BADFILE BAD_DIR:'CARDS.bad'
LOGFILE LOG_DIR:'CARDS.log'
NODISCARDFILE
FIELDS TERMINATED BY ","
OPTIONALLY ENCLOSED BY '"'
READSIZE 1048576
LRTRIM
MISSING FIELD VALUES ARE NULL
REJECT ROWS WITH ALL NULL FIELDS
I want to know everytime i query the external table, my log file and bad file will be overwritten or appended? i want them to be overwritten.Hi,
Yeah, well, be in for a surprise here:
"The external tables feature is a complement to existing SQL*Loader functionality."
http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/utility.htm#sthref1800
http://www.oracle.com/pls/db102/search?remark=quick_search&word=external+table&tab_id=&format=ranked
Have you actually read/tried anything?
You can download an XE DB for free and play with that. -
Subject area security validating users and groups from external table
Hi all.
I don't have practice to put question here, but there is one problem, that seems don't work correctly in OBIEE.
I'm trying to put users in groups within external table and this works fine.
I put security on the subject area level like this:
SA1 -> GroupA allow, Everyone not allow
SA2 -> GroupB allow, Everyone not allow
External table:
User----------Group
A---------GroupA;GroupB
B---------GroupB
Users A, B and GroupA, GroupB exists in the RPD, but I didn't put users inside them, I want this from table.
From the init block, external table I'm taking users and join them in the group. Same name users and groups are also in the presentation service.
When i connect with user A i don't see any subject area, when go to My Account i see in Group Membership/GroupA and GroupB, so it's readed from the external table.
Why in this case the subject area permission is not working?
It works if I explicitly put users in groups, in the RPD.
I have read this blog entry http://kpipartners.blogspot.com/2009/07/groups-webgroups-and-delivers.html and it is said that this works, but I'm interested how.
What should we have in the presentation part, administration, Manage Privileges -> Access within Oracle BI Answers option for those two subject area?
This doesn't work or something is missing:
Re: Security on Subject Areas
Regards
Goran
http://108obiee.blogspot.comWhat should we have in the presentation part, administration, Manage Privileges -> Access within Oracle BI Answers option for those two subject area?Yes, you should remove Everyone and add the relevant groups to each Subject Area. You don't need to set privileges in the RPD, in fact that's probably why it doesn't work for you. Leave your RPD Presentation Catalog as "Everyone" = Read as you will controlling access from the Presentation Services and it should work.
-
How to ''give'' error for this case of an EXTERNAL TABLE?
Our external table routine works fine:
-- We have a csv file with 2 cols.
-- When we create the table referring the csv it works fine.
-- Even if the csv has more the 2 cols, the ET command only takes the 2 cols and it works fine.
-- Now, users are saying that if the csv has more than 2 cols, the ET command should give an error
I went through the command but cannot find any clause which will do this.
Is there any other way or workaround?
CREATE TABLE <table_name> (
<column_definitions>)
ORGANIZATION EXTERNAL
(TYPE oracle_loader
DEFAULT DIRECTORY <oracle_directory_object_name>
ACCESS PARAMETERS (
RECORDS DELIMITED BY newline
BADFILE <file_name>
DISCARDFILE <file_name>
LOGFILE <file_name>
[READSIZE <bytes>]
[SKIP <number_of_rows>
FIELDS TERMINATED BY '<terminator>'
REJECT ROWS WITH ALL NULL FIELDS
MISSING FIELD VALUES ARE NULL
(<column_name_list>))\
LOCATION ('<file_name>'))
[PARALLEL]
REJECT LIMIT <UNLIMITED | integer>;
Is it possible to use the READSIZE?
Edited by: Channa on Sep 23, 2010 2:28 AM-- Now, users are saying that if the csv has more than 2 cols, the ET command should give an error
I went through the command but cannot find any clause which will do this.
Is there any other way or workaround?I looked at Serverprocess' sql*loader script and did not see how that would answer your question - how to raise an error if the file has more than 2 columns. If I missed something can Serverprocess explain?
I can't think of a direct way to do this with your external table either, but there may be indirect ways. Some brainstorming ideas of perhaps dubious usefulness follow.
Placing a view over the external table can limit results to the first two columns but won't raise an error.
A pipelined function can read the external table, check for data where there shouldn't be any, and raise an exception when you find data in columns where there should not be any.
Similarly, you could ditch the external table and use utl_file to read the file, manually parsing and checking the data. LOTS more work but more control on your end. External tables are much easer to use :(
Or, first load the external table into a work table before the "real" select. Check the work table for the offending data programatically and raise an error if data is where it should not be. You could keep the existing external table and not have to do a lot of recoding.
Or, also load the data into an otherwise unneeded global temporary table first. Use a trigger on the load to look for the unwanted data and raise an error if offending data is there
These ideas are boiling down to variations on validating the data before you use it.
Good luck! -
How to transform data while populating an external table
Hi Folks,
I have to populate the first four fields of my external table from the data in a csv file. While the last field should have the "SYSDATE". My code to create an external table (and to populate the first four fields) is as follows
create table T_XT
(PNODE VARCHAR2(10),
NODE VARCHAR2(15),
SUB_NODE VARCHAR2(12),
NODE_COUNT NUMBER,
CREATE_DATE DATE
organization external
(type oracle_loader
default DIRECTORY topology
access parameters
(records delimited by NEWLINE
characterset US7ASCII
badfile 'TOPOLOGY':'Modem_Count.bad'
discardfile 'TOPOLOGY':'Modem_Count.dis'
logfile 'TOPOLOGY':'Modem_Count.log'
fields terminated by ','
optionally enclosed by "'"
location ('Modem_Count.csv')
reject limit unlimited parallel;
Can somebody please let me know where this data transformation (for the last field) should be specified ? And the syntax to do so.
Thanks in advance
rogers42Hi,
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:6790030213850#59876418675556
You'd rather just:
select pnode,
node,
sub_node,
node_count,
sysdate
from t_xt;
Maybe you are looking for
-
How to read tables and fields from a transaction screen.
hi all, i am having problem in reading fields and tables from a screen, could any one help me in mapping tables from a structure as well. it will be very help full if u had any documentation. Thanking u kiran
-
Business Partner in SRM gets Centrally Blocked
Hi SRM Gurus, We are in now SRM 5.0 Version. I have noticed that the Business Partner for a user is centrally blocked and this happens automatically. User gets the error "Partner xxxxx is blocked", while creating the Shopping Cart. This will be resol
-
Mp4 iMovie - is it possible without extra steps?
I can't import mp4 clips in to iMovie. I can see them listed, but they are grayed out and I can't select them. I also can't seem to get a definite answer as to whether or not iMovie will import mp4s without jumping through a bunch of hoops. I'd lo
-
I can't find iTunes U/podcast files in iTunes OR Windows Explorer.
There are some podcasts (mostly iTunes U) on my 5th gen iPod that I cannot delete. They show up in my iPod's menus, but they do not show up in any section of iTunes. I have used Windows to show hidden files and deleted some of them off my iPod throug
-
How can I programmatically control stylesheet slection in TestSTand?
I would like to have several style sheets available for report generation in TestStand 4.1.1. How can I programatticaly select which style sheet is selected when running a TS sequence?