SQL Loader/External Table multiple record delimiters

Hi every one.
I have a strange problem, I have an external csv file which i wish to deal with (external tables or sql loader). This csv is totally not organized in structure and it contains records that are mixed together, meaning that some records are delimited by newline characters. So in short, i want to know if I will be able to load the data in this csv separating records by newline character and another character? So is that possible to have multiple record delimiters specified in the same ctl file?

abohsin,
I think using the Stream record format would be helful in your case. Please explore that.
Using stream record option, instead of the default new line, you can specify a user defined record delimiter.
Check this link.
http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_control_file.htm#i1005509
Here is what I did. Not the complete answer, but it might be helpful.
Replace all delimiters witha standard delimiters (in unix)
sed s/HEAD,/**DLM**/g < test.dat >test2.dat
sed s/TAIL,/**DLM**/g < test2.dat >test3.dat
create table t(
  TEXT varchar2(100)
and Use that delimiter as the standard delimiter.
load data
infile "test3.dat" "str '**DLM**'"
into table T
TRUNCATE
fields terminated by 'XXXXX' optionally enclosed by '"'
TEXT
sql> select * from t;
TEXT
1111,2222,
4444,5555,
4444
1111,3333,
8888,6666,
5555
{code}
You should also replace new line charecters with '***DLM***'.
Thanks,
Rajesh.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

Similar Messages

  • SQL LOADER , EXTERNAL  TABLE and ODBS DATA SOURCE

    hello
    Can any body help loading data from dbase file .dbt to an oracle 10g table.
    I tried last day with SQL LOADER, EXTERNAL table , and ODBC data source.
    Why all of these utilities still failing to solve my problem ?
    Is there an efficient way to reach this goal ?
    Thanks in advance

    export the dbase data file to text file,
    then you have choice of using either sql loader or external table option to use.
    regards

  • Comparison of Data Loading techniques - Sql Loader & External Tables

    Below are 2 techniques using which the data can be loaded from Flat files to oracle tables.
    1)     SQL Loader:
    a.     Place the flat file( .txt or .csv) on the desired Location.
    b.     Create a control file
    Load Data
    Infile "Mytextfile.txt" (-- file containing table data , specify paths correctly, it could be .csv as well)
    Append or Truncate (-- based on requirement) into oracle tablename
    Separated by "," (or the delimiter we use in input file) optionally enclosed by
    (Field1, field2, field3 etc)
    c.     Now run sqlldr utility of oracle on sql command prompt as
    sqlldr username/password .CTL filename
    d.     The data can be verified by selecting the data from the table.
    Select * from oracle_table;
    2)     External Table:
    a.     Place the flat file (.txt or .csv) on the desired location.
    abc.csv
    1,one,first
    2,two,second
    3,three,third
    4,four,fourth
    b.     Create a directory
    create or replace directory ext_dir as '/home/rene/ext_dir'; -- path where the source file is kept
    c.     After granting appropriate permissions to the user, we can create external table like below.
    create table ext_table_csv (
    i Number,
    n Varchar2(20),
    m Varchar2(20)
    organization external (
    type oracle_loader
    default directory ext_dir
    access parameters (
    records delimited by newline
    fields terminated by ','
    missing field values are null
    location ('file.csv')
    reject limit unlimited;
    d.     Verify data by selecting it from the external table now
    select * from ext_table_csv;
    External tables feature is a complement to existing SQL*Loader functionality.
    It allows you to –
    •     Access data in external sources as if it were in a table in the database.
    •     Merge a flat file with an existing table in one statement.
    •     Sort a flat file on the way into a table you want compressed nicely
    •     Do a parallel direct path load -- without splitting up the input file, writing
    Shortcomings:
    •     External tables are read-only.
    •     No data manipulation language (DML) operations or index creation is allowed on an external table.
    Using Sql Loader You can –
    •     Load the data from a stored procedure or trigger (insert is not sqlldr)
    •     Do multi-table inserts
    •     Flow the data through a pipelined plsql function for cleansing/transformation
    Comparison for data loading
    To make the loading operation faster, the degree of parallelism can be set to any number, e.g 4
    So, when you created the external table, the database will divide the file to be read by four processes running in parallel. This parallelism happens automatically, with no additional effort on your part, and is really quite convenient. To parallelize this load using SQL*Loader, you would have had to manually divide your input file into multiple smaller files.
    Conclusion:
    SQL*Loader may be the better choice in data loading situations that require additional indexing of the staging table. However, we can always copy the data from external tables to Oracle Tables using DB links.

    Please let me know your views on this.

  • Oracle SQL Loader - External Table

    This is my code:-
    CREATE TABLE GLO_CUST_EXT
    (SOURCE_ID CHAR(3),
    BUS_DT CHAR(8),
    TRAN_CD CHAR(1),
    CUST_CD NUMBER(10),
    MNEMONIC CHAR(10),
    SHORT_NM CHAR(255))
    ORGANIZATION external
    (TYPE oracle_loader
    DEFAULT DIRECTORY Source_dir
    ACCESS PARAMETERS
    (RECORDS DELIMITED BY NEWLINE Skip 1
    badfile bad_dir:'GLO_CUST%a_%p.bad'
    LOGFILE log_dir:'GLO_CUST%a_%p.log'
    FIELDS TERMINATED BY '|'
    (SOURCE_ID ,
    BUS_DT ,
    TRAN_Cd ,
    CUST_CD,
    MNEMONIC,
    SHORT_NM))
    LOCATION ('GBCUS.txt'))
    REJECT LIMIT 0;
    Question:
    Would like how to load the data by selected field?
    Example:
    I want to all these fields (SOURCE_ID , BUS_DT , TRAN_CD ,
    CUST_CD, SHORT_NM) except the MNEMONIC. Thank you.

    Mohan Nair,
    This is my code:
    CREATE TABLE GLO_CUST_EXT5
    (SOURCE_ID CHAR(3),
    BUS_DT CHAR(8),
    TRAN_CD CHAR(1),
    CUST_CD NUMBER(10),
    MNEMONIC CHAR(10),
    SHORT_NM CHAR(255))
    ORGANIZATION external
    (TYPE oracle_loader
    DEFAULT DIRECTORY Source_dir
    ACCESS PARAMETERS
    (RECORDS DELIMITED BY NEWLINE Skip 1
    badfile bad_dir:'GLO_CUST%a_%p.bad'
    LOGFILE log_dir:'GLO_CUST%a_%p.log'
    FIELDS TERMINATED BY '|'
    (SOURCE_ID ,
    BUS_DT ,
    TRAN_Cd ,
    CUST_CD,
    MNEMONIC FILLER,
    SHORT_NM))
    LOCATION ('GBCUS.txt'))
    REJECT LIMIT 0;
    I'm using Oracle 10g. I execute this code using TOAD version 8.5.0.50.
    I can create the external table, but when i used simple select statement
    (select * from GLO_CUST_EXT5), i got this error:
    ORA-29913: error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    KUP-00554: error encountered while parsing access parameters
    KUP-01005: syntax error: found "identifier": expecting one of: "binary_double, binary_float, comma, char, date, defaultif, decimal, double, float, integer, (, nullif, oracle_date, oracle_number, position, raw, recnum, ), unsigned, varrawc, varchar, varraw, varcharc, zoned"
    KUP-01008: the bad identifier was: FILLER
    KUP-01007: at line 9 column 10
    ORA-06512: at "S
    Can you help me solve this problem? Thank you.

  • Fields terminated by (SQL loader, external table) question?

    Hello.
    I have a txt file which looks like:
    Columns:
    A..........B.........C...........D.........E..............F.............G...........H
    739.......P.........0002......05........25012006..25012006..5...........data group
    . = space
    There are different number of spaces between columns.
    What must i use in FIELDS TERMINATED BY to import this?
    Thanks.

    So, don't use FIELDS TERMINATED BY, but, as Ino suggested, fixed format, something like
    LOAD DATA
    TRUNCATE INTO TABLE <table name>
    (a position(1:10),
    b position(11:20),
    c position(21:30),
    d position(31:40),
    e position(41:48) date "ddmmyyyy",
    f position(51:58) date "ddmmyyyy",
    g position(61:72),
    h position(73:92))

  • How to resolve error when Loading External Table?

    I’m getting the following errors when attempting to load External Table -- I've changed the file extension from .csv to .txt to resolve ORA-29913 but error re-occurred. See syntax of External Table below.
    Thanks,
    Carol-Ann
    SQL> desc OWB_COUNTY_TIMEZONE_EXT;
    Name Null? Type
    STATE_CODE VARCHAR2(2)
    COUNTY_CODE VARCHAR2(3)
    TIME_ZONE VARCHAR2(1)
    SQL> select count(*) from OWB_COUNTY_TIMEZONE_EXT;
    select count(*) from OWB_COUNTY_TIMEZONE_EXT
    ERROR at line 1:
    ORA-29913: error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    KUP-04063: unable to open log file owb_county_timezone.log
    OS error Permission denied
    ORA-06512: at "SYS.ORACLE_LOADER", line 14
    ORA-06512: at line 1
    ++++++++++++++++++++++++++++++++++++++++++++++
    Synatx of External Table:
    WHENEVER SQLERROR EXIT FAILURE;
    CREATE TABLE "OWB_COUNTY_TIMEZONE_EXT"
    "STATE_CODE" VARCHAR2(2),
    "COUNTY_CODE" VARCHAR2(3),
    "TIME_ZONE" VARCHAR2(1))
    ORGANIZATION EXTERNAL (
    TYPE ORACLE_LOADER
    DEFAULT DIRECTORY AIMQRYD_AIMP_LOC_FF_MODULE_LOC
    ACCESS PARAMETERS (
    RECORDS DELIMITED BY NEWLINE
    CHARACTERSET WE8MSWIN1252
    STRING SIZES ARE IN BYTES
    BADFILE 'owb_county_timezone'
    DISCARDFILE 'owb_county_timezone'
    LOGFILE 'owb_county_timezone'
    FIELDS
    TERMINATED BY ','
    OPTIONALLY ENCLOSED BY '"' AND '"'
    NOTRIM
    MISSING FIELD VALUES ARE NULL
    "STATE_CODE" ,
    "COUNTY_CODE" ,
    "TIME_ZONE"
    LOCATION (
    AIMQRYD_AIMP_LOC_FF_MODULE_LOC:'county_timezone_comma.txt'
    REJECT LIMIT UNLIMITED
    NOPARALLEL

    Hi Carol-Ann,
    The key issue here is
    "KUP-04063: unable to open log file owb_county_timezone.log
    OS error Permission denied".
    Looks like you don't have sufficient system priviliges on Unix.
    This is wat AskTom mentions about it:
    "the directory must exist on the SERVER.
    the concerned user is the "oracle software owner" as far as the OS is concerned.
    oracle must have read write access to this directory
    and the directory must exist on the SERVER (database server) itself."
    Hope this helps.
    Cheers, Patrick

  • SQL and External table question

    Hello averyone,
    I have a file to be read as an external table part of it is below:
    ISO-10303-21;
    HEADER;
    FILE_DESCRIPTION((''),'2;1');
    FILE_NAME('BRACKET','2005-07-08T',('broilo'),(''),
    'PRO/ENGINEER BY PARAMETRIC TECHNOLOGY CORPORATION, 2004400',
    'PRO/ENGINEER BY PARAMETRIC TECHNOLOGY CORPORATION, 2004400','');
    FILE_SCHEMA(('CONFIG_CONTROL_DESIGN'));
    ENDSEC;
    DATA;
    #5=CARTESIAN_POINT('',(5.5E0,5.5E0,-5.1E1));
    #6=DIRECTION('',(0.E0,0.E0,1.E0));
    #7=DIRECTION('',(-1.E0,0.E0,0.E0));
    #8=AXIS2_PLACEMENT_3D('',#5,#6,#7);
    The first question is: how to ignore the lines until the DATA; line or SQL already does it for me?
    The second question is: since the fields of interest are separated by commas and the first field does not interest me (it is solved with a varchar2) how can I read the following fields as numbers ignoring the (,# and ) characters please?
    Thanks for any help.
    Sincerely yours,
    André Luiz

    The SKIP option can be used with SQL*Loader to skip a certain number of lines before starting to load. Off hand I cannot see any easy way to load the data in the format given. The format does not resemble a typical CVS format. You can look at the test cases provided for SQ*Loader in the Oracle® Database Utilities guide - or simply write PL/SQL code to load this data manually using UTL_FILE.

  • How to remove carraige return from the field while loading external table

    I am facing an issue of not getting rid of carraige returns present in the fileds of the source .csv file while loading the records into external table.
    I had tried using LRTRIM, but it does not help.
    The error I am getting is:
    KUP-04021: field formatting error for field POPULATION_DESCRIPTION
    KUP-04037: terminator not found
    I am pasting one record out of the .csv file which is causing this error as below:
    "Business Card Accounts
    ",123,7 BizCard - Gamers,Control,"Business Card Accounts
    ",75270,75271
    You can see the carraige return in the 1st field as well as 5th field. Filed are separated by commas & eclosed by double quotes.
    Could anybody help on this please?

    Can you copy the file to an external table only version, and then dos2unix it?
    Alternatively, you can use the ACCESS PARAMETERS area in your external table definition to set that your RECORDS DELIMITED BY carriage returns, instead of the default.
    Check out the example here http://www.psoug.org/reference/externaltab.html

  • SQL Loader, nested tables and default values

    Is there a way to specify a default value for a nested table entry when SQL*Loader encounters a 'null' value?
    I want to avoid this:
    Record 5: Rejected - Error on table LEVEL_DESC, column LEVELS.
    NULL nested table element is not allowed

    Use the NULLIF parameter in your control file for the nested table objects.
    e.g
    LOAD DATA
    INFILE 'level_data.dat'
    INTO TABLE LEVEL
    (LEVEL_ID POSITION (01:05) CHAR
    LEVEL_NAME POSITION (07:20)
    LEVEL_DESC COLUMN OBJECT
    (LEVELS POSITION (22:25) CHAR NULLIF LEVEL_DESC.LEVELS=BLNAKS,
    ... ))

  • SQL Loader: Total number of record read

    Hello,
    When we use SQL loader to load a data file, the total number of records read is output to the SQL Loader log. Is there a way to output this number to an oracle table?
    Regards

    slsam01 wrote:
    Thank you Hans.
    Can you provide me more information about loading SQL loader log file (e.g., control file) and point me to an example of loading SQL loader log files to Oracle table?
    Regards,Oh joy, oh bliss. A documentation request.
    For Oracle 10g, http://download.oracle.com/docs/cd/B19306_01/server.102/b14215/part_ldr.htm#i436326 provides a few pages (including examples called Case Studies)

  • External table max record length check

    This question is regarding fixed length input file.
    I am checking to see if it is possible to add a check to verify the max length of a record when defining an external table.
    For example, if I am expecting a fixed length file with max record/row length of 100, is it possible to reject lines in the
    file that are more than 100 characters long?
    Thanks.

    What you can do is something like:
    DROP TABLE TBL_EXT
    CREATE TABLE TBL_EXT(
                         VAL       VARCHAR2(4),
                         INDICATOR VARCHAR2(1)
      ORGANIZATION EXTERNAL(
                            TYPE ORACLE_LOADER
                            DEFAULT DIRECTORY TEMP
                            ACCESS PARAMETERS (
                                               FIELDS TERMINATED BY ','
                                               OPTIONALLY ENCLOSED BY '"'
                                               MISSING FIELD VALUES ARE NULL
                                               REJECT ROWS WITH ALL NULL FIELDS
                                                VAL       POSITION(1:4),
                                                INDICATOR POSITION(5:5) CHAR NOTRIM
                            LOCATION ('tbl_ext.txt')
    SELECT  *
      FROM  TBL_EXT
      WHERE INDICATOR IS NULL
    {code}
    Now tbl_ext.txt:
    {code}
    X
    XX
    XXX
    XXXX
    XXXX    XXXXXX
    XXXX    <-- this line has trailing spaces
    XXX
    XX
    X
    {code}
    Now:
    {code}
    SQL> CREATE TABLE TBL_EXT(
      2                       VAL       VARCHAR2(4),
      3                       INDICATOR VARCHAR2(1)
      4                      )
      5    ORGANIZATION EXTERNAL(
      6                          TYPE ORACLE_LOADER
      7                          DEFAULT DIRECTORY TEMP
      8                          ACCESS PARAMETERS (
      9                                             FIELDS TERMINATED BY ','
    10                                             OPTIONALLY ENCLOSED BY '"'
    11                                             MISSING FIELD VALUES ARE NULL
    12                                             REJECT ROWS WITH ALL NULL FIELDS
    13                                             (
    14                                              VAL       POSITION(1:4),
    15                                              INDICATOR POSITION(5:5) CHAR NOTRIM
    16                                             )
    17                                            )
    18                          LOCATION ('tbl_ext.txt')
    19                         )
    20  /
    Table created.
    SQL> SELECT  *
      2    FROM  TBL_EXT
      3    WHERE INDICATOR IS NULL
      4  /
    VAL  I
    X
    XX
    XXX
    XXXX
    XXX
    XX
    X
    7 rows selected.
    SQL> As you can see, lines 'XXXX XXXXXX' and 'XXXX    ' were not selected since WHERE INDICATOR IS NULL for these rows results in FALSE.
    SY.

  • How  change NLS_NUMERIC_CHARACTERS parameter for load external table

    Hi,
    I use this version:
    OWB 11gR2
    Database 11gR2
    Parameter NLS_NUMERIC_CHARACTERS Database ., Instance ,.
    When I created database with wizard and in this moment I don't set spanish language, later I changed this parameters in instance parameters.
    Now I want load data from a file to external table, but I've an error when I try load data with decimal point.
    why does it use the database parameter instead of instance parameter?
    Is possible to change this parameter?
    Cheers
    Marisol

    At this moment , this is not possible . You can see metalink note ID 268906.1.
    It says:
    Currently, external tables always use the setting of NLS_NUMERIC_CHARACTERS
    +at the database level.+
    Cheers
    Marisol

  • Failed to parse SQL Query - External Tables [Apex 4.0.2]

    Greetings experts -
    Has anyone encountered errors when creating a report in Apex that sources from an external table within the database? I'm using the 4.0.2 version that is packaged with the 11g XE edition on 64bit CentOS.
    For example, I might run:
    SELECT NULL LINK,
    COL1 LABEL,
    COL2 VALUE
    FROM MYTAB;
    Where MYTAB is an external table sitting on top of a comma-separated file. When I go to create the page, I'm prompted with the "Failed to parse SQL" dialogue.
    I noticed that if I did CTAS on the external table, and referenced the CTAS table, my page ran without problem!
    Any ideas? Is this a known "limitation" of Apex?
    Thanks,
    CJ

    Chiedu,
    Please try removing all declarative validations on this tabular form, and see if it works as expected then. There are some limitations on the type of views and joins that are supported by tabular forms when using the new declarative validations, i.e. you'll need a key preserved table in your view or join.
    Regards,
    Marc

  • SQL Loader and table views

    I am having a problem updating a large partitioned table through it's appropriate view. Here is the loader file
    LOAD DATA               
    INSERT INTO TABLE test.example               
         (      PROGRAM POSITION(1:4) CHAR,
              DWING_NUMBER     POSITION(6:32) CHAR,
              LAST_ISSUED_REVISION     POSITION(34:35) CHAR,
              TEMP_NUMBER     POSITION(37:43) CHAR,
              CONFIGURATION_ITEM     POSITION(45:45) CHAR,
              LOCATION_CODE     POSITION(47:47) CHAR
    It generates this error:
    SQL*Loader-951: Error calling once/load initialization
    ORA-26018: Column PROGRAM in table test.example does not exist
    There are a few considerations that go along with this, and I am developing a strong disdain for the 2 dba's that are involved. First, this view accesses a large partitioned table, all according to "PROGRAM". Secondly, PROGRAM is called PROG_ID in the main table. I didn't think this was an issue at first, but it seems like it might be part of the problem I'm having. We're running Oracle 9.2.0 here. Any helpful replies will be greatly appreciated. Thanks.

    This is what the view looks like:
    Column / Data Type / Null? / Updatable
    PROGRAM VARCHAR(10) N y
    DWING_NUMBER VARCHAR(28) N y
    LAST_ISSUED_REVISION VARCHAR(2) Y y
    TEMP_NUMBER VARCHAR(7) Y y
    CONFIGURATION_ITEM VARCHAR(1) Y y
    LOCATION_CODE VARCHAR(1) Y y
    The main table that this view grabs data from is identical, except for the fact that PROGAM is PROG_ID. I'm fairly new to doing database work, so if the answer is trivial please spare my self esteem.
    formatting sucks
    vagrantgringo

  • SQL*Loader Sequential Data File Record Processing?

    If I use the conventional path will SQL*Loader process a data file sequentially from top to bottom?  I have a file comprised of header and detail records with no value found in the detail records that can be used to relate to the header records.  The only option is to derive a header value via a sequence (nextval) and then populate the detail records with the same value pulled from the same sequence (currval).  But for this to work SQL*Loader must process the file in the exact same sequence that the data has been written to the data file.  I've read through the 11g Oracle® Database Utilities SQL*Loader sections looking for proof that this is what will happen but haven't found this information and I don't want to assume that SQL*Loader will always process the data file records sequentially.
    Thank you

    Oracle Support responded with the following statement.
    "Yes, SQL*LOADER process data file from top to bottom.
    This was touched in the note below:
    SQL*Loader - How to Load a Single Logical Record from Physical Records which Include Linefeeds (Doc ID 160093.1)"
    Jason

Maybe you are looking for

  • ERROR

    dear all, i am having a peculiar error. while creating po thru me21n everyone must have seen a hold tab comes just after other purchase order tab. now whenever i am trying to make a service po thru me21n with cost center K and item category as D with

  • Clicking a link opens a smaller window, not a maximized window, how can I fix this?

    When I click links that open in a new window, that new window is not maximized, like the browser window that I used to click the link. How can I fix this? I prefer using new windows instead of new tabs, and I have been doing this forever. I had turne

  • Flex 4.5.1 in Adobe Air on Playbook or Desktop

    I have been trying to figure this out and I am stuck. I have tried the SquigglyUIExample and it works perfect in html using mx so the dictionary files and libs should be in the proper folders. I am trying to use the following code in a view container

  • Update to 2.0.2

    I saw on Apple's page that 2.0.2 was available, so I plugged my phone in and hit update. iTunes told me that 2.0.1 was the latest version. I thought this was odd so I clicked it again: the same message. I double checked the page to see if I had the f

  • Value Help Error - Portal Runtime Error

    Hey, I am getting the error when i click on any Value help what is the problem The Error that shown  on the popup up or screen is Portal Runtime Error An exception occurred while processing your request Exception id: 11:45_03/04/11_0002_3832651 See t