SQL Loader and record order

I am using sqlloader utility to load data from a CSV file in to a table.
My .ctl file looks as below
------- 8< -------
options (errors=5,SILENT=(HEADER, FEEDBACK),direct=true)
load data
infile "mytest.csv"
discardmax 0
into table owneruser.MY_TABLE
fields terminated by "," optionally enclosed by "##"
(ID, ID1, VAL, VAL2)
------- 8< -------
sqlldr tool is run with this ctl file by another database user who has sufficient privileges to insert this data in to mytest.csv has about 400000 entries each entry maps to one row in MY_TABLE. Before loading data using the sqlldr MY_TABLE is truncated.
In mytest.csv, the value for ID field is a number starting at 1 which keeps incrementing by 1 for the next entry. The records are ordered by ID in the csv file.
After loading the data using sqlldr, when we query MY_TABLE (select * from MY_TABLE), so far the records are returned in same order in which they were inserted (i.e. ordered by ID). But off late they are not being returned in random order. This happens only on one database instance. On other test instances the the resultset is ordered. I agree that the only way the order can be guaranteed is by using the ORDER BY clause.
But, I was wondering why this has worked even when ORDER BY is not used.
This is the only way in which MY_TABLE is manipulated. Rest all use it only for querying.
ID is the primary key column in MY_TABLE and there is an index on (ID, ID1).
Thanks in advance.
S

There are any number of reasons that the data would be coming back in a different order since you're not using an ORDER BY. My guess is that the most likely reason is that you have one or more extents in your table that is physically before another extent that it is logically after, in which case a full scan would read that extent first. You may also be seeing differences in how ASSM happens to choose which block to insert into, in the use of parallelism, etc.
Justin

Similar Messages

  • SQL Loader and Insert Into Performance Difference

    Hello All,
    Im in a situation to measure performance difference between SQL Loader and Insert into. Say there 10000 records in a flat file and I want to load it into a staging table.
    I know that if I use PL/SQL UTL_FILE to do this job performance will degrade(dont ask me why im going for UTL_FILE instead of SQL Loader). But I dont know how much. Can anybody tell me the performance difference in % (like 20% will decrease) in case of 10000 records.
    Thanks,
    Kannan.

    Kannan B wrote:
    Do not confuse the topic, as I told im not going to use External tables. This post is to speak the performance difference between SQL Loader and Simple Insert Statement.I don't think people are confusing the topic.
    External tables are a superior means of reading a file as it doesn't require any command line calls or external control files to be set up. All that is needed is a single external table definition created in a similar way to creating any other table (just with the additional external table information obviously). It also eliminates the need to have a 'staging' table on the database to load the data into as the data can just be queried as needed directly from the file, and if the file changes, so does the data seen through the external table automatically without the need to re-run any SQL*Loader process again.
    Who told you not to use External Tables? Do they know what they are talking about? Can they give a valid reason why external tables are not to be used?
    IMO, if you're considering SQL*Loader, you should be considering External tables as a better alternative.

  • HELP: SQL*LOADER AND Ref Column

    Hallo,
    I have already posted and I really need help and don't come further with this
    I have the following problem. I have 2 tables which I created the following way:
    CREATE TYPE gemark_schluessel_t AS OBJECT(
    gemark_id NUMBER(8),
    gemark_schl NUMBER(4),
    gemark_name VARCHAR2(45)
    CREATE TABLE gemark_schluessel_tab OF gemark_schluessel_t(
    constraint pk_gemark PRIMARY KEY(gemark_id)
    CREATE TYPE flurstueck_t AS OBJECT(
    flst_id NUMBER(8),
    flst_nr_zaehler NUMBER(4),
    flst_nr_nenner NUMBER(4),
    zusatz VARCHAR2(2),
    flur_nr NUMBER(2),
    gemark_schluessel REF gemark_schluessel_t,
    flaeche SDO_GEOMETRY
    CREATE TABLE flurstuecke_tab OF flurstueck_t(
    constraint pk_flst PRIMARY KEY(flst_id),
    constraint uq_flst UNIQUE(flst_nr_zaehler,flst_nr_nenner,zusatz,flur_nr),
    flst_nr_zaehler NOT NULL,
    flur_nr NOT NULL,
    gemark_schluessel REFERENCES gemark_schluessel_tab
    Now I have data in the gemark_schluessel_tab which looks like this (a sample):
    1 101 Borna
    2 102 Draisdorf
    Now I wanna load data in my flurstuecke_tab with SQL*Loader and there I have problems with my ref column gemark_schluessel.
    One data record looks like this in my file (it is without geometry)
    1|97|7||1|1|
    If I wanna load my data record, it does not work. The reference (the system generated OID) should be taken from gemark_schluessel_tab.
    LOAD DATA
    INFILE *
    TRUNCATE
    CONTINUEIF NEXT(1:1) = '#'
    INTO TABLE FLURSTUECKE_TAB
    FIELDS TERMINATED BY '|'
    TRAILING NULLCOLS (
    flst_id,
    flst_nr_zaehler,
    flst_nr_nenner,
    zusatz,
    flur_nr,
    gemark_schluessel REF(CONSTANT 'GEMARK_SCHLUESSEL_TAB',GEMARK_ID),
    gemark_id FILLER
    BEGINDATA
    1|97|7||1|1|
    Is there a error I made?
    Thanks in advance
    Tig

    multiple duplicate threads:
    to call an oracle procedure and sql loader in an unix script
    Re: Can some one help he sql loader issue.

  • Import and process larger data with SQL*Loader and Java resource

    Hello,
    I have a project to import data from a text file in a schedule. A lager data, with nearly 20,000 record/1 hours.
    After that, we have to analysis the data, and export the results into a another database.
    I research about SQL*Loader and Java resource to do these task. But I have no experiment about that.
    I'm afraid of the huge data, Oracle could be slowdown or the session in Java Resource application could be timeout.
    Please tell me some advice about the solution.
    Thank you very much.

    With '?' mark i mean " How i can link this COL1 with column in csv file ? "
    Attilio

  • SQL *Loader and External Table

    Hi,
    Can anyone tell me the difference between SQL* Loader and External table?
    What are the conditions under we can use SQL * Loader and External Table.
    Thanx

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

  • Help in calling sql loader and an oracle procedure in a script

    Hi Guru's
    please help me in writing an unix script which will call sql loader and also an oracle procedure..
    i wrote an script which is as follows.
    !/bin/sh
    clear
    #export ORACLE_SID='HOBS2'
    sqlldr USERID=load/ps94mfo16 CONTROL=test_nica.ctl LOG=test_nica.log
    retcode=`echo $?`
    case "$retcode" in
    0) echo "SQL*Loader execution successful" ;;
    1) echo "SQL*Loader execution exited with EX_FAIL, see logfile" ;;
    2) echo "SQL*Loader execution exited with EX_WARN, see logfile" ;;
    3) echo "SQL*Loader execution encountered a fatal error" ;;
    *) echo "unknown return code";;
    esac
    sqlplus USERID=load/ps94mfo16 << EOF
    EXEC DO_TEST_SHELL_SCRIPT
    it is loading the data in to an oracle table
    but the procedure is not executed..
    any valuable suggestion is highly appriciated..
    Cheers

    multiple duplicate threads:
    to call an oracle procedure and sql loader in an unix script
    Re: Can some one help he sql loader issue.

  • Using SQL*Loader and UTL_FILE to load and unload large files(i.e PDF,DOCs)

    Problem : Load PDF or similiar files( stored at operating system) into an oracle table using SQl*Loader .
    and than Unload the files back from oracle tables to prevoius format.
    I 've used SQL*LOADER .... " sqlldr " command as :
    " sqlldr scott/[email protected] control=c:\sqlldr\control.ctl log=c:\any.txt "
    Control file is written as :
    LOAD DATA
    INFILE 'c:\sqlldr\r_sqlldr.txt'
    REPLACE
    INTO table r_sqlldr
    Fields terminated by ','
    id sequence (max,1) ,
    fname char(20),
    data LOBFILE(fname) terminated by EOF )
    It loads files ( Pdf, Image and more...) that are mentioned in file r_sqlldr.txt into oracle table r_sqlldr
    Text file ( used as source ) is written as :
    c:\kalam.pdf,
    c:\CTSlogo1.bmp
    c:\any1.txt
    after this load ....i used UTL_FILE to unload data and write procedure like ...
    CREATE OR REPLACE PROCEDURE R_UTL AS
    l_file UTL_FILE.FILE_TYPE;
    l_buffer RAW(32767);
    l_amount BINARY_INTEGER ;
    l_pos INTEGER := 1;
    l_blob BLOB;
    l_blob_len INTEGER;
    BEGIN
    SELECT data
    INTO l_blob
    FROM r_sqlldr
    where id= 1;
    l_blob_len := DBMS_LOB.GETLENGTH(l_blob);
    DBMS_OUTPUT.PUT_LINE('blob length : ' || l_blob_len);
    IF (l_blob_len < 32767) THEN
    l_amount :=l_blob_len;
    ELSE
    l_amount := 32767;
    END IF;
    DBMS_LOB.OPEN(l_blob, DBMS_LOB.LOB_READONLY);
    l_file := UTL_FILE.FOPEN('DBDIR1','Kalam_out.pdf','w', 32767);
    DBMS_OUTPUT.PUT_LINE('File opened');
    WHILE l_pos < l_blob_len LOOP
    DBMS_LOB.READ (l_blob, l_amount, l_pos, l_buffer);
    DBMS_OUTPUT.PUT_LINE('Blob read');
    l_pos := l_pos + l_amount;
    UTL_FILE.PUT_RAW(l_file, l_buffer, TRUE);
    DBMS_OUTPUT.PUT_LINE('writing to file');
    UTL_FILE.FFLUSH(l_file);
    UTL_FILE.NEW_LINE(l_file);
    END LOOP;
    UTL_FILE.FFLUSH(l_file);
    UTL_FILE.FCLOSE(l_file);
    DBMS_OUTPUT.PUT_LINE('File closed');
    DBMS_LOB.CLOSE(l_blob);
    EXCEPTION
    WHEN OTHERS THEN
    IF UTL_FILE.IS_OPEN(l_file) THEN
    UTL_FILE.FCLOSE(l_file);
    END IF;
    DBMS_OUTPUT.PUT_LINE('Its working at last');
    END R_UTL;
    This loads data from r_sqlldr table (BOLBS) to files on operating system ,,,
    -> Same procedure with minor changes is used to unload other similar files like Images and text files.
    In above example : Loading : 3 files 1) Kalam.pdf 2) CTSlogo1.bmp 3) any1.txt are loaded into oracle table r_sqlldr 's 3 rows respectively.
    file names into fname column and corresponding data into data ( BLOB) column.
    Unload : And than these files are loaded back into their previous format to operating system using UTL_FILE feature of oracle.
    so PROBLEM IS : Actual capacity (size ) of these files is getting unloaded back but with quality decreased. And PDF file doesnt even view its data. means size is almot equal to source file but data are lost when i open it.....
    and for images .... imgaes are getting loaded an unloaded but with colors changed ....
    Also features ( like FFLUSH ) of Oracle 've been used but it never worked
    ANY SUGGESTIONS OR aLTERNATE SOLUTION TO LOAD AND UNLOAD PDFs through Oracle ARE REQUESTED.
    ------------------------------------------------------------------------------------------------------------------------

    Thanks Justin ...for a quick response ...
    well ... i am loading data into BLOB only and using SQL*Loader ...
    I've never used dbms_lob.loadFromFile to do the loads ...
    i 've opend a file on network and than used dbms_lob.read and
    UTL_FILE.PUT_RAW to read and write data into target file.
    actually ...my process is working fine with text files but not with PDF and IMAGES ...
    and your doubt of ..."Is the data the proper length after reading it in?" ..m not getting wat r you asking ...but ... i think regarding data length ..there is no problem... except ... source PDF length is 90.4 kb ..and Target is 90.8 kb..
    thats it...
    So Request u to add some more help ......or should i provide some more details ??

  • Is there any difference in Oracle 9i SQL Loader and Oracle 10g SQL Loader

    Hi
    Can anyone tell me whether is there any difference in Oracle 9i SQL Loader and Oracle 10g SQL Loader?
    I am upgrading the 9i db to 10g and wanted to run the 9i SQL Loader control files on upgraded 10g db. So please let me know is there any difference which I need to consider any modifications in the control files..
    Thank you in advance
    Adi

    answered

  • SQL*Loader and multiple files

    Hello, am tasked with loading tables with 21+ million rows. Will SQL*Loader perform better with one large file or many smaller files? Is there any ideal file size for optimal performance? Thank you
    David

    Don, when I tried to loada 21M row table using direct, I get the following messages:
    Record 2373: Rejected - Error on table STAGE_CUSTOMER.
    ORA-03113: end-of-file on communication channel
    SQL*Loader-926: OCI error while uldlfca:OCIDirPathColArrayLoadStream for table STAGE_CUSTOMER
    SQL*Loader-2026: the load was aborted because SQL Loader cannot continue.
    SQL*Loader-925: Error while uldlgs: OCIStmtExecute (ptc_hp)
    ORA-03114: not connected to ORACLE
    SQL*Loader-925: Error while uldlgs: OCIStmtFetch (ptc_hp)
    ORA-24338: statement handle not executed
    Here is the SQL*Loader log:
    SQL*Loader: Release 9.2.0.1.0 - Production on Thu Apr 26 15:38:29 2007
    Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
    Control File: stage_customer.ctl
    Character Set UTF8 specified for all input.
    First primary datafile stage_Customer_20070301.csv has a
    utf8 byte order mark in it.
    Data File: stage_Customer_20070301.csv
    Bad File: stage_Customer_20070301.bad
    Discard File: none specified
    (Allow all discards)
    Number to load: ALL
    Number to skip: 0
    Errors allowed: 1000
    Continuation: none specified
    Path used: Direct
    Silent options: FEEDBACK
    Table STAGE_CUSTOMER, loaded from every logical record.
    Insert option in effect for this table: APPEND
    TRAILING NULLCOLS option in effect
    Column Name Position Len Term Encl Datatype
    ROW_ID SEQUENCE (MAX, 1)
    CUSTOMER_ACCT_NUM FIRST * , O(") CHARACTER
    IS_DELETED NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN UPPER(:Is_Deleted) IN ('FALSE','0') THEN 0 ELSE 1 END"
    NAME_PREFIX NEXT * , O(") CHARACTER
    FIRST_NAME NEXT * , O(") CHARACTER
    MIDDLE_NAME NEXT * , O(") CHARACTER
    LAST_NAME NEXT * , O(") CHARACTER
    NAME_SUFFIX NEXT * , O(") CHARACTER
    NICK_NAME NEXT * , O(") CHARACTER
    ALT_FIRST_NAME NEXT * , O(") CHARACTER
    ALT_LAST_NAME NEXT * , O(") CHARACTER
    MARKETING_SOURCE_ID NEXT * , O(") CHARACTER
    HOME_PHONE NEXT * , O(") CHARACTER
    WORK_PHONE NEXT * , O(") CHARACTER
    MOBILE_PHONE NEXT * , O(") CHARACTER
    ALTERNATE_PHONE NEXT * , O(") CHARACTER
    EMAIL_ADDR NEXT * , O(") CHARACTER
    ALT_EMAIL_ADDR NEXT * , O(") CHARACTER
    BIRTH_DATE NEXT * , O(") CHARACTER
    SQL string for column : "TRUNC(TO_DATE(:Birth_Date, 'MM/DD/YYYY HH24:MI:SS'))"
    SALES_CHANNEL_ID NEXT * , O(") CHARACTER
    SQL string for column : "decode(:Sales_Channel_id,NULL,NULL,NULL)"
    ASSOCIATE_NUMBER NEXT * , O(") CHARACTER
    ALT_ASSOCIATE_NUMBER NEXT * , O(") CHARACTER
    UPDATE_LOCATION_CD NEXT * , O(") CHARACTER
    UPDATE_DATE NEXT * , O(") CHARACTER
    SQL string for column : "TRUNC(TO_DATE(:Update_Date, 'MM/DD/YYYY HH24:MI:SS'))"
    CUSTOMER_LOGIN_NAME NEXT * , O(") CHARACTER
    DISCOUNT_CD NEXT * , O(") CHARACTER
    DISCOUNT_PERCENT NEXT * , O(") CHARACTER
    BUSINESS_NAME NEXT * , O(") CHARACTER
    POS_TAX_FLAG NEXT * , O(") CHARACTER
    POS_TAX_PROMPT NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN UPPER(:POS_TAX_PROMPT) IN ('FALSE','0') THEN 0 ELSE 1 END"
    POS_DEFAULT_TAX_ID NEXT * , O(") CHARACTER
    POS_TAX_ID_EXPIRATION_DATE NEXT * , O(") CHARACTER
    POS_AUTHORIZED_USER_FLAG NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN UPPER(:POS_AUTHORIZED_USER_FLAG) IN ('FALSE','0') THEN 0 ELSE 1 END"
    POS_ALLOW_PURCHASE_ORDER_FLAG NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN UPPER(:POS_ALLOW_PURCHASE_ORDER_FLAG) IN ('FALSE','0') THEN 0 ELSE 1 END"
    ADDRESS1 NEXT * , O(") CHARACTER
    ADDRESS2 NEXT * , O(") CHARACTER
    ADDRESS3 NEXT * , O(") CHARACTER
    CITY NEXT * , O(") CHARACTER
    STATE_CD NEXT * , O(") CHARACTER
    POSTAL_CD NEXT * , O(") CHARACTER
    COUNTRY_CD NEXT * , O(") CHARACTER
    ALLOW_UPDATE NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN UPPER(:ALLOW_UPDATE) IN ('FALSE','0') THEN 0 ELSE 1 END"
    ACTION_CODE NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN UPPER(:action_code) IN ('FALSE','0') THEN 0 ELSE 1 END"
    ACCOUNT_TYPE_ID NEXT * , O(") CHARACTER
    LOCALE_CD NEXT * , O(") CHARACTER
    SQL string for column : "CASE WHEN :Locale_CD IS NOT NULL AND :Locale_CD LIKE '__-__' THEN :Locale_CD ELSE 'en-US' END"
    IS_READY_FOR_PROCESSING CONSTANT
    Value is '1'
    IS_BUSINESS CONSTANT
    Value is '0'
    HAD_ERRORS CONSTANT
    Value is '0'
    Record 2373: Rejected - Error on table STAGE_CUSTOMER.
    ORA-03113: end-of-file on communication channel
    SQL*Loader-926: OCI error while uldlfca:OCIDirPathColArrayLoadStream for table STAGE_CUSTOMER
    SQL*Loader-2026: the load was aborted because SQL Loader cannot continue.
    SQL*Loader-925: Error while uldlgs: OCIStmtExecute (ptc_hp)
    ORA-03114: not connected to ORACLE
    SQL*Loader-925: Error while uldlgs: OCIStmtFetch (ptc_hp)
    ORA-24338: statement handle not executed
    Table STAGE_CUSTOMER:
    0 Rows successfully loaded.
    1 Row not loaded due to data errors.
    0 Rows not loaded because all WHEN clauses were failed.
    0 Rows not loaded because all fields were null.
    Bind array size not used in direct path.
    Column array rows : 5000
    Stream buffer bytes: 256000
    Read buffer bytes: 1048576
    Total logical records skipped: 0
    Total logical records read: 3469
    Total logical records rejected: 1
    Total logical records discarded: 0
    Direct path multithreading optimization is disabled
    Run began on Thu Apr 26 15:38:29 2007
    Run ended on Thu Apr 26 15:38:30 2007
    Elapsed time was: 00:00:01.18
    CPU time was: 00:00:00.32

  • Problem using SQL-LOADER and Unique Identifiers

    I'm trying to load a fixed-length records file containing people names and phone numbers. Data is specified as follows
    Toni Tomas66666666669999999999
    Jose Luis 33333333330000000000
    Notice that a maximum of 2 numbers can follow a person name, and 0000000000 means "no number specified".
    I want to assign a unique identifier to people (instead of using the NAME field as a Primary Key) using an Oracle Sequence. I did that, but I don't know
    how to assign the same id to each number.
    Considering the 2 previous lines, desired result should be:
    PEOPLE
    ======
    1     Toni Tomas
    2     Jose Luis
    TEL_NUMBERS
    ===========
    1     6666666666
    1     9999999999
    2     3333333333
    In order to achieve that, my Control File looks like this
    LOAD DATA
    INFILE phonenumbers.txt
    INTO TABLE people
         personID "mySequenceName.nextval", --an Oracle sequence
         name POSITION(1:10) CHAR
    INTO TABLE tel_numbers
    WHEN phonenumber !='0000000000'
         personID !!!DON'T KNOW HOW TO REFERENCE THE SAME ID!!!!
         phonenumber POSITION(11:20) CHAR
    INTO TABLE tel_numbers
    WHEN phonenumber !='0000000000'
         personID !!!DON'T KNOW HOW TO REFERENCE THE SAME ID!!!!
         phonenumber POSITION(21:30) CHAR
    I tried lots of things, but anyone works:
    a) reference the ID using something like ":\"people.personID\" (or similar aproaches)
    b) using a BEFORE INSERT TRIGGER getting the CURRVAL value of the Sequence. This solution
    does not work because it seems that all people is loaded before any telephone number. Hence,
    all phone numbers are associated, wrongly, to the last person in the data file.
    Does anyone know how can I solve this issue?
    Help would be appreciated. Thank you.

    Hi V Garcia.
    Information within the file is correct. Each line represents a COMPLETE record (Part of the line represents parent information and the rest is children data). As you can see in my first message, you can have more than one detail for a given master (i.e. two phone numbers):
    Toni Tomas66666666669999999999
    (10 chars for the name, 10 for each phone number. Thus, 2 children records to be created)
    With the solution given by Sreekanth Reddy Bandi (use of CURRVALUE within the SQL-Loader Control File), not all the details are linked to the parent record on the DB tables. It seems SLQ-Loader gets crazy when there is such amount of information.

  • SQL*Loader and binary

    i am trying to load binary data using SQL*Loader. given a table created a la:
    create table oofa ( data raw(500) );
    and a control file (running on x86 linux):
    LOAD DATA
    INFILE *
    APPEND INTO TABLE OOFA
    ( DATA VARRAW(500) )
    BEGINDATA
    X'05003132333435'
    i get an error "variable length field exceeds maximum length" and no record loaded. i tried using X'00053132333435' just in case i'd muffed the big- little-endian stuff, but the same thing happens.
    anyone successfully loaded binary data as above? if so, how'd you do it?
    also, is there a way to make SQL*Loader more verbose?
    thanks.

    i am trying to load binary data using SQL*Loader. given a table created a la:
    create table oofa ( data raw(500) );
    and a control file (running on x86 linux):
    LOAD DATA
    INFILE *
    APPEND INTO TABLE OOFA
    ( DATA VARRAW(500) )
    BEGINDATA
    X'05003132333435'
    i get an error "variable length field exceeds maximum length" and no record loaded. i tried using X'00053132333435' just in case i'd muffed the big- little-endian stuff, but the same thing happens.
    anyone successfully loaded binary data as above? if so, how'd you do it?
    also, is there a way to make SQL*Loader more verbose?
    thanks.

  • SQL*Loader and binary data

    i have a C routine that builds SQL*Loader input files. the input files contain multiple records, with a couple of integer columns and a raw(1400) field. the control file specifies a record separator of '|', which seems weird (having text in the middle of raw data) but also is somewhat co-operative (see below).
    so i basically write each integer to the file, then a short (2-byte) length value for the raw field, then the raw field. then the '|' separator.
    i've noticed that if the size of raw field is 400 bytes or less, everything works fine, i get the correct number of records in the database.
    unfortunately, with a size of 401 or more, SQL*Loader parses the thing into twice as many records as it should. so if i've written 3 records to my input data file, with each record's raw field at 400 bytes or less i get 3 records loaded. but any with a raw field of 401+, i get two records for each.
    any ideas why? and how to correct this? also, any ideas on a better way to do this? all the examples of large data in the online doc and the o'reilly book favor showing examples with large character data, which does me not much good.
    tia. john

    i have a C routine that builds SQL*Loader input files. the input files contain multiple records, with a couple of integer columns and a raw(1400) field. the control file specifies a record separator of '|', which seems weird (having text in the middle of raw data) but also is somewhat co-operative (see below).
    so i basically write each integer to the file, then a short (2-byte) length value for the raw field, then the raw field. then the '|' separator.
    i've noticed that if the size of raw field is 400 bytes or less, everything works fine, i get the correct number of records in the database.
    unfortunately, with a size of 401 or more, SQL*Loader parses the thing into twice as many records as it should. so if i've written 3 records to my input data file, with each record's raw field at 400 bytes or less i get 3 records loaded. but any with a raw field of 401+, i get two records for each.
    any ideas why? and how to correct this? also, any ideas on a better way to do this? all the examples of large data in the online doc and the o'reilly book favor showing examples with large character data, which does me not much good.
    tia. john

  • Sql Loader and carriage returns

    I am currently trying to use sql loader to load data from flat files that was extracted from sybase using bcp and delimited with pipes. There are text and varchar columns that contain carriage return line feeds. I want to preserve these, but I can not load them into Oracle 8.05 using sql loader as it interprets them as end of record indicators. Does anyone have a way to solve this problem? Any assistance would be appreciated.
    Thanks in advance,
    Jignesh

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

  • SQL*Loader and CONTINUEIF clause

    Hi,
    I used SQL*Loader to load in a table some files coming from main frame environnement.
    The length of the physical record is 1662 characters. For some lines of the file, the characters CR (X'0C') could appear anywhere in the position 1158 to 1165 inclusively. I just want to know how to tell to SQL*Loader to concatenate the next record by using CONTINUEIF clause.
    I made a simple test. I checked only the position 1165 and it works. Here is the "ctl":
    OPTIONS (ERRORS=999999)
    LOAD DATA
    infile 'T:\RéalDelisle\AGR.PBC1.SCOPY001.de test avec x25 aux positions 1158 a  1165.txt'
    BADFILE 'D:\Sinpac\Bad\sinpac_YT.bad'
    DISCARDFILE 'D:\Sinpac\Disc\sinpac_YT.dsc'
    APPEND
    CONTINUEIF THIS PRESERVE (1165)=X'0C'
    INTO TABLE AGRMDHD1.Sinpac_YT
    +(+
    CODEFICHIER             constant  "D501.A0405",
    C001_CP12               position (1:12) ,
    C001_record             position (1:1662),
    C001_record_1157        position (1:1157),
    C001_record_1166        position (1166:1662),
    +          C001_Mont_subv_adm position (1644:1652) "TO_NUMBER(:c001_mont_subv_adm,'999999D99', 'NLS_NUMERIC_CHARACTERS = ''. '' ')",+
    C001_Mont_subv_adm_nouv position (1654:1662) "TO_NUMBER(:c001_mont_subv_adm_nouv,'999999D99', 'NLS_NUMERIC_CHARACTERS = ''. '' ')"
    +)+
    But, I want to do the test for the position 1158 to 1165 (the character X'0C' could appear in one of those positions). I tried this but it didn't work (I've got a syntax error):
    OPTIONS (ERRORS=999999)
    LOAD DATA
    infile 'T:\RéalDelisle\AGR.PBC1.SCOPY001.de test avec x25 aux positions 1158 a  1165.txt'
    BADFILE 'D:\Sinpac\Bad\sinpac_YT.bad'
    DISCARDFILE 'D:\Sinpac\Disc\sinpac_YT.dsc'
    APPEND
    CONTINUEIF THIS PRESERVE (1158)=X'0C'
    CONTINUEIF THIS PRESERVE (1150)=X'0C'
    CONTINUEIF THIS PRESERVE (1160)=X'0C'
    CONTINUEIF THIS PRESERVE (1161)=X'0C'
    CONTINUEIF THIS PRESERVE (1162)=X'0C'
    CONTINUEIF THIS PRESERVE (1163)=X'0C'
    CONTINUEIF THIS PRESERVE (1164)=X'0C'
    CONTINUEIF THIS PRESERVE (1165)=X'0C'
    INTO TABLE AGRMDHD1.Sinpac_YT
    +(+
    CODEFICHIER             constant  "D501.A0405",
    C001_CP12               position (1:12) ,
    C001_record             position (1:1662),
    C001_record_1157        position (1:1157),
    C001_record_1166        position (1166:1662),
    +          C001_Mont_subv_adm position (1644:1652) "TO_NUMBER(:c001_mont_subv_adm,'999999D99', 'NLS_NUMERIC_CHARACTERS = ''. '' ')",+
    C001_Mont_subv_adm_nouv position (1654:1662) "TO_NUMBER(:c001_mont_subv_adm_nouv,'999999D99', 'NLS_NUMERIC_CHARACTERS = ''. '' ')"
    +)+
    I looked in the documentation but I found nothing about the way to do it.
    Does anybody have an idea?
    Thank's

    If a record contains CR (X'0C'), then is the CR (X'0C') always the last character of the record? If so, I think you'd use LAST (it's all in the Docs.).
    If not, then I don't think SQL*Loader will do what you want. Instead, I think you'd need to manipulate the data before it reaches SQL*Loader. You use Windows, so what about installing CygWin? It's a winner, really, and you can then do lovely sed,vi & awk stuff on your data - and it's very fast, too.
    Regards - Don Lewis

  • SQL Loader and the bad file

    I use SQL Loader to read a file into a table. In my tests a lot of rows are rejected and written to the bad file. But there are no error messages.
    Is there a way to find out why the row was rejected.

    The Bad File
    The bad file contains records that were rejected, either by SQL*Loader or by the Oracle database server. Some of the possible reasons for rejection are discussed in the next sections.
    SQL*Loader Rejects
    Datafile records are rejected by SQL*Loader when the input format is invalid. For example, if the second enclosure delimiter is missing, or if a delimited field exceeds its maximum length, SQL*Loader rejects the record. Rejected records are placed in the bad file.
    Oracle Rejects
    After a datafile record is accepted for processing by SQL*Loader, it is sent to the Oracle database server for insertion into a table as a row. If the Oracle database server determines that the row is valid, then the row is inserted into the table. If the row is determined to be invalid, then the record is rejected and SQL*Loader puts it in the bad file. The row may be invalid, for example, because a key is not unique, because a required field is null, or because the field contains invalid data for the Oracle datatype.
    YOU WILL get more information
    sqlldr help=y
    also send your control file syntax
    kuljeet pal singh

Maybe you are looking for