SQL Loader and Batch ID

Hi All,
In our application, we are allowing user to upload data using excel sheet in UI.
We are using PHP script in UI and using SQL Loader to load data from excel sheet to temp_table.
The temp_table has a primary key.
Here my question is , Is there any way to put some batch id for every upload in that table in automatic way ?
so that we can easily extract the data by using batch id
we are using Oracle 11g.

All that does is load a constant value, in which case you might as well just use constant 815 in your control file. If you want to automatically increment the value for each batch, then you need to use a different method.
Please see the example below. Prior to each data load, it loads the next value of the sequence into a separate table, then selects that value during the data load. Note that a SQL*Loader expression that uses select must be enclosed within parentheses within the double quotes.
SCOTT@orcl_11gR2> host type test1.dat
1 Prod1
2 Prod2
3 Prod3
4 Prod4
5 Prod5
SCOTT@orcl_11gR2> host type test2.dat
6 Prod6
7 Prod7
8 Prod8
SCOTT@orcl_11gR2> host type batch.ctl
options(load=1)
load data
replace
into table batch_tab
(batch_id expression "test_seq.nextval")
SCOTT@orcl_11gR2> host type data.ctl
load data
append
into table temp_table
fields terminated by whitespace
trailing nullcols
(p_id,
p_name,
batch_id expression "(select batch_id from batch_tab)")
SCOTT@orcl_11gR2> create table temp_table
  2    (p_id      number primary key,
  3     p_name    varchar2(6),
  4     batch_id  number)
  5  /
Table created.
SCOTT@orcl_11gR2> create sequence test_seq
  2  /
Sequence created.
SCOTT@orcl_11gR2> create table batch_tab
  2    (batch_id  number)
  3  /
Table created.
SCOTT@orcl_11gR2> -- first load:
SCOTT@orcl_11gR2> host sqlldr scott/tiger control=batch.ctl log=batch1.log
SQL*Loader: Release 11.2.0.1.0 - Production on Fri Apr 19 17:16:33 2013
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
Commit point reached - logical record count 1
SCOTT@orcl_11gR2> host sqlldr scott/tiger control=data.ctl data=test1.dat log=test1.log
SQL*Loader: Release 11.2.0.1.0 - Production on Fri Apr 19 17:16:33 2013
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
Commit point reached - logical record count 5
SCOTT@orcl_11gR2> select * from batch_tab
  2  /
  BATCH_ID
         1
1 row selected.
SCOTT@orcl_11gR2> select * from temp_table
  2  /
      P_ID P_NAME   BATCH_ID
         1 Prod1           1
         2 Prod2           1
         3 Prod3           1
         4 Prod4           1
         5 Prod5           1
5 rows selected.
SCOTT@orcl_11gR2> -- second load:
SCOTT@orcl_11gR2> host sqlldr scott/tiger control=batch.ctl log=batch2.log
SQL*Loader: Release 11.2.0.1.0 - Production on Fri Apr 19 17:16:33 2013
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
Commit point reached - logical record count 1
SCOTT@orcl_11gR2> host sqlldr scott/tiger control=data.ctl data=test2.dat log=test2.log
SQL*Loader: Release 11.2.0.1.0 - Production on Fri Apr 19 17:16:33 2013
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
Commit point reached - logical record count 3
SCOTT@orcl_11gR2> select * from batch_tab
  2  /
  BATCH_ID
         2
1 row selected.
SCOTT@orcl_11gR2> select * from temp_table
  2  /
      P_ID P_NAME   BATCH_ID
         1 Prod1           1
         2 Prod2           1
         3 Prod3           1
         4 Prod4           1
         5 Prod5           1
         6 Prod6           2
         7 Prod7           2
         8 Prod8           2
8 rows selected.

Similar Messages

  • SQL *Loader and External Table

    Hi,
    Can anyone tell me the difference between SQL* Loader and External table?
    What are the conditions under we can use SQL * Loader and External Table.
    Thanx

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

  • SQL Loader and Insert Into Performance Difference

    Hello All,
    Im in a situation to measure performance difference between SQL Loader and Insert into. Say there 10000 records in a flat file and I want to load it into a staging table.
    I know that if I use PL/SQL UTL_FILE to do this job performance will degrade(dont ask me why im going for UTL_FILE instead of SQL Loader). But I dont know how much. Can anybody tell me the performance difference in % (like 20% will decrease) in case of 10000 records.
    Thanks,
    Kannan.

    Kannan B wrote:
    Do not confuse the topic, as I told im not going to use External tables. This post is to speak the performance difference between SQL Loader and Simple Insert Statement.I don't think people are confusing the topic.
    External tables are a superior means of reading a file as it doesn't require any command line calls or external control files to be set up. All that is needed is a single external table definition created in a similar way to creating any other table (just with the additional external table information obviously). It also eliminates the need to have a 'staging' table on the database to load the data into as the data can just be queried as needed directly from the file, and if the file changes, so does the data seen through the external table automatically without the need to re-run any SQL*Loader process again.
    Who told you not to use External Tables? Do they know what they are talking about? Can they give a valid reason why external tables are not to be used?
    IMO, if you're considering SQL*Loader, you should be considering External tables as a better alternative.

  • Help in calling sql loader and an oracle procedure in a script

    Hi Guru's
    please help me in writing an unix script which will call sql loader and also an oracle procedure..
    i wrote an script which is as follows.
    !/bin/sh
    clear
    #export ORACLE_SID='HOBS2'
    sqlldr USERID=load/ps94mfo16 CONTROL=test_nica.ctl LOG=test_nica.log
    retcode=`echo $?`
    case "$retcode" in
    0) echo "SQL*Loader execution successful" ;;
    1) echo "SQL*Loader execution exited with EX_FAIL, see logfile" ;;
    2) echo "SQL*Loader execution exited with EX_WARN, see logfile" ;;
    3) echo "SQL*Loader execution encountered a fatal error" ;;
    *) echo "unknown return code";;
    esac
    sqlplus USERID=load/ps94mfo16 << EOF
    EXEC DO_TEST_SHELL_SCRIPT
    it is loading the data in to an oracle table
    but the procedure is not executed..
    any valuable suggestion is highly appriciated..
    Cheers

    multiple duplicate threads:
    to call an oracle procedure and sql loader in an unix script
    Re: Can some one help he sql loader issue.

  • HELP: SQL*LOADER AND Ref Column

    Hallo,
    I have already posted and I really need help and don't come further with this
    I have the following problem. I have 2 tables which I created the following way:
    CREATE TYPE gemark_schluessel_t AS OBJECT(
    gemark_id NUMBER(8),
    gemark_schl NUMBER(4),
    gemark_name VARCHAR2(45)
    CREATE TABLE gemark_schluessel_tab OF gemark_schluessel_t(
    constraint pk_gemark PRIMARY KEY(gemark_id)
    CREATE TYPE flurstueck_t AS OBJECT(
    flst_id NUMBER(8),
    flst_nr_zaehler NUMBER(4),
    flst_nr_nenner NUMBER(4),
    zusatz VARCHAR2(2),
    flur_nr NUMBER(2),
    gemark_schluessel REF gemark_schluessel_t,
    flaeche SDO_GEOMETRY
    CREATE TABLE flurstuecke_tab OF flurstueck_t(
    constraint pk_flst PRIMARY KEY(flst_id),
    constraint uq_flst UNIQUE(flst_nr_zaehler,flst_nr_nenner,zusatz,flur_nr),
    flst_nr_zaehler NOT NULL,
    flur_nr NOT NULL,
    gemark_schluessel REFERENCES gemark_schluessel_tab
    Now I have data in the gemark_schluessel_tab which looks like this (a sample):
    1 101 Borna
    2 102 Draisdorf
    Now I wanna load data in my flurstuecke_tab with SQL*Loader and there I have problems with my ref column gemark_schluessel.
    One data record looks like this in my file (it is without geometry)
    1|97|7||1|1|
    If I wanna load my data record, it does not work. The reference (the system generated OID) should be taken from gemark_schluessel_tab.
    LOAD DATA
    INFILE *
    TRUNCATE
    CONTINUEIF NEXT(1:1) = '#'
    INTO TABLE FLURSTUECKE_TAB
    FIELDS TERMINATED BY '|'
    TRAILING NULLCOLS (
    flst_id,
    flst_nr_zaehler,
    flst_nr_nenner,
    zusatz,
    flur_nr,
    gemark_schluessel REF(CONSTANT 'GEMARK_SCHLUESSEL_TAB',GEMARK_ID),
    gemark_id FILLER
    BEGINDATA
    1|97|7||1|1|
    Is there a error I made?
    Thanks in advance
    Tig

    multiple duplicate threads:
    to call an oracle procedure and sql loader in an unix script
    Re: Can some one help he sql loader issue.

  • Using SQL*Loader and UTL_FILE to load and unload large files(i.e PDF,DOCs)

    Problem : Load PDF or similiar files( stored at operating system) into an oracle table using SQl*Loader .
    and than Unload the files back from oracle tables to prevoius format.
    I 've used SQL*LOADER .... " sqlldr " command as :
    " sqlldr scott/[email protected] control=c:\sqlldr\control.ctl log=c:\any.txt "
    Control file is written as :
    LOAD DATA
    INFILE 'c:\sqlldr\r_sqlldr.txt'
    REPLACE
    INTO table r_sqlldr
    Fields terminated by ','
    id sequence (max,1) ,
    fname char(20),
    data LOBFILE(fname) terminated by EOF )
    It loads files ( Pdf, Image and more...) that are mentioned in file r_sqlldr.txt into oracle table r_sqlldr
    Text file ( used as source ) is written as :
    c:\kalam.pdf,
    c:\CTSlogo1.bmp
    c:\any1.txt
    after this load ....i used UTL_FILE to unload data and write procedure like ...
    CREATE OR REPLACE PROCEDURE R_UTL AS
    l_file UTL_FILE.FILE_TYPE;
    l_buffer RAW(32767);
    l_amount BINARY_INTEGER ;
    l_pos INTEGER := 1;
    l_blob BLOB;
    l_blob_len INTEGER;
    BEGIN
    SELECT data
    INTO l_blob
    FROM r_sqlldr
    where id= 1;
    l_blob_len := DBMS_LOB.GETLENGTH(l_blob);
    DBMS_OUTPUT.PUT_LINE('blob length : ' || l_blob_len);
    IF (l_blob_len < 32767) THEN
    l_amount :=l_blob_len;
    ELSE
    l_amount := 32767;
    END IF;
    DBMS_LOB.OPEN(l_blob, DBMS_LOB.LOB_READONLY);
    l_file := UTL_FILE.FOPEN('DBDIR1','Kalam_out.pdf','w', 32767);
    DBMS_OUTPUT.PUT_LINE('File opened');
    WHILE l_pos < l_blob_len LOOP
    DBMS_LOB.READ (l_blob, l_amount, l_pos, l_buffer);
    DBMS_OUTPUT.PUT_LINE('Blob read');
    l_pos := l_pos + l_amount;
    UTL_FILE.PUT_RAW(l_file, l_buffer, TRUE);
    DBMS_OUTPUT.PUT_LINE('writing to file');
    UTL_FILE.FFLUSH(l_file);
    UTL_FILE.NEW_LINE(l_file);
    END LOOP;
    UTL_FILE.FFLUSH(l_file);
    UTL_FILE.FCLOSE(l_file);
    DBMS_OUTPUT.PUT_LINE('File closed');
    DBMS_LOB.CLOSE(l_blob);
    EXCEPTION
    WHEN OTHERS THEN
    IF UTL_FILE.IS_OPEN(l_file) THEN
    UTL_FILE.FCLOSE(l_file);
    END IF;
    DBMS_OUTPUT.PUT_LINE('Its working at last');
    END R_UTL;
    This loads data from r_sqlldr table (BOLBS) to files on operating system ,,,
    -> Same procedure with minor changes is used to unload other similar files like Images and text files.
    In above example : Loading : 3 files 1) Kalam.pdf 2) CTSlogo1.bmp 3) any1.txt are loaded into oracle table r_sqlldr 's 3 rows respectively.
    file names into fname column and corresponding data into data ( BLOB) column.
    Unload : And than these files are loaded back into their previous format to operating system using UTL_FILE feature of oracle.
    so PROBLEM IS : Actual capacity (size ) of these files is getting unloaded back but with quality decreased. And PDF file doesnt even view its data. means size is almot equal to source file but data are lost when i open it.....
    and for images .... imgaes are getting loaded an unloaded but with colors changed ....
    Also features ( like FFLUSH ) of Oracle 've been used but it never worked
    ANY SUGGESTIONS OR aLTERNATE SOLUTION TO LOAD AND UNLOAD PDFs through Oracle ARE REQUESTED.
    ------------------------------------------------------------------------------------------------------------------------

    Thanks Justin ...for a quick response ...
    well ... i am loading data into BLOB only and using SQL*Loader ...
    I've never used dbms_lob.loadFromFile to do the loads ...
    i 've opend a file on network and than used dbms_lob.read and
    UTL_FILE.PUT_RAW to read and write data into target file.
    actually ...my process is working fine with text files but not with PDF and IMAGES ...
    and your doubt of ..."Is the data the proper length after reading it in?" ..m not getting wat r you asking ...but ... i think regarding data length ..there is no problem... except ... source PDF length is 90.4 kb ..and Target is 90.8 kb..
    thats it...
    So Request u to add some more help ......or should i provide some more details ??

  • Is there any difference in Oracle 9i SQL Loader and Oracle 10g SQL Loader

    Hi
    Can anyone tell me whether is there any difference in Oracle 9i SQL Loader and Oracle 10g SQL Loader?
    I am upgrading the 9i db to 10g and wanted to run the 9i SQL Loader control files on upgraded 10g db. So please let me know is there any difference which I need to consider any modifications in the control files..
    Thank you in advance
    Adi

    answered

  • Import and process larger data with SQL*Loader and Java resource

    Hello,
    I have a project to import data from a text file in a schedule. A lager data, with nearly 20,000 record/1 hours.
    After that, we have to analysis the data, and export the results into a another database.
    I research about SQL*Loader and Java resource to do these task. But I have no experiment about that.
    I'm afraid of the huge data, Oracle could be slowdown or the session in Java Resource application could be timeout.
    Please tell me some advice about the solution.
    Thank you very much.

    With '?' mark i mean " How i can link this COL1 with column in csv file ? "
    Attilio

  • Sql Loader and a batch id

    Hello
    I have a loading table which as a primary key. We insert into load_ctl table which has a load_ctl_id and then sql load a csv file into a table called load_table.
    Now when we have to process the load_table we work with the primary key from the control table load_ctl_id to know which load to process. How can I get the load_ctl_id into my load_table. The csv file does not contain it.

    What full version of Oracle?
    How do you currently generate the control table load_ctl_id?
    Do you have to be concerned with concurrent load jobs?
    What tool are you using to perform the load (sqlldr)?
    If you already have a way of gerating the load_ctl_id and placing it into the control table and you do not need to worry about concurrent jobs you could use a before insert trigger to insert the same maximum load_ctl_id (assuming sequence or date stamp) into the load table with each row insert. Or leave the column null during the load and then immediately after the load update each column.
    If you have to worry about concurrent load processes where each would be a different batch number then how you currently create the load_ctl_id value is more important to the solution since you have to make sure two concurrently running sessions would in fact grap two different batch ids.
    HTH -- Mark D Powell --

  • Sql Loader and carriage returns

    I am currently trying to use sql loader to load data from flat files that was extracted from sybase using bcp and delimited with pipes. There are text and varchar columns that contain carriage return line feeds. I want to preserve these, but I can not load them into Oracle 8.05 using sql loader as it interprets them as end of record indicators. Does anyone have a way to solve this problem? Any assistance would be appreciated.
    Thanks in advance,
    Jignesh

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

  • SQL-Loader and Pipes

    I have to insert 300.000.000 datasets (generated dynamically by a seperate process) into an Oracle 8.1.6 database. I'd like to do this via SQL-Loader but since Linux (kernel 2.2 at least) supports only files up to 2Gigs and I don't have the necessary space on my machine to generate a large amount of ASCII files I'd like to ask whether there is a possibility to use an ordinary pipe in combination with SQL-Loader. I tried to use a named pipe but all my attempts failed.
    I don't like to use Perl and the DBI module since this would be far so slow (would take about one week I guess).
    Hans

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

  • Sql loader and Store Prrocedures

    Hi!!
    I ned load text files with sql loader first at temporal tables,later on put this in tables with Stored Procedures in one data base Oracle 8.1.5.
    My sqlldr version is 8.1.6 and not4s compatible with 8.1.5 of the data base server.
    How can i load the files?
    Thank4s

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

  • Sql loader and bulk data

    hi,
    I want to insert 100,000 records daily in a table for the first month and then in next month these records are going to be replaced by new updated records.
    there might be few addition and deletion in the previous records also.
    actually its consumer data so there might be few consumer who have withdrawn the utility and there will be some more consumer added in the database.
    but almost 99% of the previous month data have to be updated/replaced with the fresh month data.
    For instance, what i have in my mind is that i will use sql loader to load data for the first month and then i will delete the previous data using sqlPlus and load the fresh month data using sql loader again.
    1. Is this ok ? or there is some better solution to this.
    2. I have heard of external files, are they feasible in my scenario?
    3. I have planned that i will make scripts for sqlPlus and Loader and use them in batch files. (OS windows 2003 server, Oracle 9i database). is there some better choice to make all the procedure automatic?
    looking for your suggestions
    nadeem ameer

    I would suggest u use External tables since its more flexible then
    sqlloader & is a better option.
    For using external tables
    1)u will have to create a directory first
    2)Generally creation od directory is done by sys,hence after creating the directory
    privileges read & write to be provided to user .
    3)Creation of external tables.
    4) Now use the table as a normal table to insert ,update delete in
    ur table.
    U can get more information from
    http://www.oracle-base.com/articles/9i/SQLNewFeatures9i.php#ExternalTables
    Create Directory <directory_name> as <Directory path where file be present>
    Grant read,write on directory <directory_name> to <username>
    CREATE TABLE <table_name>
    (<column names>)
    ORGANIZATION EXTERNAL
    TYPE ORACLE_LOADER
    DEFAULT DIRECTORY ,directory_name>
    ACCESS PARAMETERS
    RECORDS DELIMITED BY NEWLINE
    FIELDS TERMINATED BY ','
    LOCATION (<filename>)
    PARALLEL 5
    REJECT LIMIT 200;
    Hope this helps.

  • Sql*loader in batch script

    Hi,
    I am calling a sqlloader and a procedure within a loop inside a batch script.
    Sometimes, The sqlloader fails and subsequent procedure fails, I am not able move to the next iteration.
    How to catch oracle errors in batch script
    Thx.

    hi,
    My batch script is something like this.
    I want to catch errors when
    1. sql*loader throws error
    2. when my pl/sql throws error.
    I welcome any suggestions to improve this code.
    d:
    echo cd Data\Scripts
    for %%f in (D:\Data\DELIVERY_NOTE\*.csv) Do (
    echo conn config/config;
    echo delete from SQLLOAD_DELIVERY_STAGE;
    )| sqlplus -s /nolog
    sqlldr USERID=config/config control=D:\Data\Scripts\loaddata_del.ctl skip=2 data=%%f
    echo conn config/config;
    echo exec DELIVERY_NOTE_PKG.Populate_Del_Note_stage;
    echo exec DELIVERY_NOTE_PKG.PROCESS_DELIVERY_NOTE;
    )| sqlplus -s /nolog
    move %%f D:\Data\Processed_Delivery\
    cd D:\Data\processed_Delivery\
    rename *.csv *.csv.processed
    cd d:\Data\scripts\
    echo #############
    echo PROCESSED THE FILE %%f AT %data% %time%
    echo ############
    )

  • Sql*Loader and Conc. Mgr.

    I would like for our users to submit a request (in GL) that will run sql*loader to load a file of their choosing to an oracle table. Oracle tells me I must use their consulting arm for this kind of question. Does anyone have any ideas on how to do this?

    External tables are accessible from SQL, which generally simplifies life if the data files are physically located on the database server since you don't have to coordinate a call to an external SQL*Loader script with other PL/SQL processing. Under the covers, external tables are normally just invoking SQL*Loader.
    SQL*Loader is more appropriate if the data files are on a different server or if it is easier to call an executable rather than calling PL/SQL (i.e. if you have a batch file that runs on a server other than the database server that wants to FTP a data file from a FTP server and then load the data into Oracle).
    Justin

Maybe you are looking for