Dummy SQL LOADER question

Hi ,
That might be pretty simple.
File format check.
How to check if records have more fields then defined in the control file?
When I load pipe delimeted file into the ORACLE table using sqlldr,
for records where number of fields less then defined in control file I'm getting an error, which is right.
Now, for records where number of fields is more then defined in control file IT IS LOADED OK, with no errors!
Is this right ?
And how can I get records where number of fields is more then defined in control file to the bad or discard file?
Thanks Leo.
Edited by: user10243745 on Oct 20, 2008 8:16 AM

Initially read the file using UTL_FILE, or some other tool, that can read line-by-line and count them.
But I most be honest and tell you that if you are getting invalid files my instinct would be to set a very low number for DISCARDMAX and reject the entire file back to its originating source.
If they can not consistently get you valid files then you should not be loading them.

Similar Messages

  • 2 sql loader questions

    Hello,
    I am going to be performing a data load with sql loader on approx 60 tables. For each table I have one csv and one ctl file. I'd like to run a file that will batch all the ctl files into one... is this possible? So that the person who has to load all this data into dev, test and prod doesn't have to go through the tedious process of running all the ctl files independantly.
    My second question is about the log file. Can I specify in the ctl file where the log file will go? I'm specifying my infile and badfile, but I can't find the syntax to specify the log.
    TIA,
    Janice

    Hi Janice,
    Following is for specifing log file
    $ORACLE_HOME/bin/sqlload userid=$ORACLE_PW control=$LDRCTL data=$complete_file_critical_parts discard=$LDRDSC bad=$LDRBAD log=$LDRLOG errors=10000000 skip=1
    Prashant

  • Sql*loader question

    Hi Experts,
    In a flat file 50000 records are there.
    In that one record is having problem so that from that record
    the remaing records are not loading to oracle tables.
    Could you please help me to load the remaing records,
    after the record which is having problem.
    The data is loading using sql*loader
    Please help me .
    Thanks in advance.

    It looks ok!
    But it will only work when table HR_DTLS is empty.
    If you want to append rows in a non empty table you should change it to
    LOAD DATA
    INFILE 'c:\test.txt'
    APPEND
    INTO TABLE hr_dtls
    FIELDS TERMINATED BY ','
    OPTIONALLY ENCLOSED BY '"'
    (sid,sname)
    {code}
    If you want to define a file for bad records (that can't be insertet for any reasons) you can change it to
    {code}
    LOAD DATA
    INFILE 'c:\test.txt'
    BADFILE 'c:\test.bad'
    APPEND
    INTO TABLE hr_dtls
    FIELDS TERMINATED BY ','
    OPTIONALLY ENCLOSED BY '"'
    (sid,sname)

  • SQL*LOADER Question.  Please help.

    Hi List,
    Below are my control and data files:(both files are stored on /Disk05)
    terminal.ctl file:
    LOAD DATA
    INFILE = Disk05 erminal.dat
    INTO TABLE TERMINAL_POSITION
    FIELDS TERMINATED BY , ENCLOSED BY
    TERMINAL_POSITION_ID VARCHAR2 TERMINATED BY , ENCLOSED BY ,
    TERMINAL_POSITION_NAME VARCHAR2 TERMINATED BY ,
    ENCLOSED BY ,
    TERMINAL_POSITION_DESCRIPTION VARCHAR2 TERMINATED BY ,
    ENCLOSED BY
    terminal.dat file:
    A,Axial,Terminals extend from both ends in the direction of the major
    axis of a cylindrical or elliptical package.
    B,Bottom,Terminals extend from the bottom of the package.
    D,Dual,Terminals are on opposite sides of a square or rectangular
    package or located in two parallel rows.
    for any reason, when entering
    sqlldr username/password control = /Disk05/terminal.ctl log
    =/Disk05/terminal.log data = /Disk05/terminal.dat
    an error message shows:
    segmentation fault.
    Alternatively, when issuing
    sqlldr userid = username/password control = /Disk05/terminal.ctl log
    =/Disk05/terminal.log
    data = /Disk05/terminal.dat
    an error message shows:
    lf iopn failed for file (terminal.log)
    Please note that: I did create terminal.log that is stored on /Disk05
    as well. Please anyone point out what did I do wrong. Your help is
    greatly appreciated.
    Thanks
    Trang

    Hi,
    Thanks to your help,finally, I successfully loaded the data into the database without any errors. Just wanted to mention that I have an error in my control file. The line infile = '/Disk05/terminal.dat' should be infile '/Disk05/terminal.dat' # no equal sign (=) in between.
    Again, Thanks alot.
    Trang

  • SQL*loader questions

    If i use oracle functions like trim, upper in my control file then can i use DIRECT method
    If i have specified DIRECT=TRUE, then what will happen if my target table has a trigger? Will it fail or use Conventional path? If it fails, how can i come to know?
    If i have not specified DIRECT=TRUE, then will Oracle use DIRECT PATH if there are no triggers or foreign keys on my target table?
    If I use INSERT option for loading data but my target table is not empty, then what error will be thrown
    For date type field we specify the date mask. Are there any other data types for which we need to specify the format?
    Thanks

    Hello,
    If i use oracle functions like trim, upper in my control file then can i use DIRECT method
    Yes you can use
    If i have specified DIRECT=TRUE, then what will happen if my target table has a trigger? Will it fail or use Conventional path? If it fails, how can i come to know?
    YOur trigger wil be disabled if you have direct=true and wont' be used
    If i have not specified DIRECT=TRUE, then will Oracle use DIRECT PATH if there are no triggers or foreign keys on my target table?
    Unless you specified direct=true on command line or script, conventional path will be used
    If I use INSERT option for loading data but my target table is not empty, then what error will be thrown
    It will insert unless you have unless you have unique constraint on table and it will fail for those records and default error limit is 50. Other options are APPEND, TRUNCATE, APPEND
    For date type field we specify the date mask. Are there any other data types for which we need to specify the format?
    Timestamp and use of newline but that goes in infile option
    Regards
    Edited by: OrionNet on Mar 4, 2009 11:26 PM

  • Help with sql loader charsets

    I was wondering if anyone could assist me with a sql loader question. I am trying to use it to load some hungarian data. It always turns the Ő character into an upside down question mark. I understand that it is because the database doesn't understand what that character is. Can someone help me find out how to resolve that? The database is UTF-8. If I paste the character in to the application, it takes it. I have tried all kinds of values for the CHARSET parameter in the .ctl file, but nothing seems to work....

    I believe we are using iso 8859-1.
    I am having a difficult time thinking that it could be we don't have the appropriate installations - because if we paste it in via the oracle application we can see it in a form. If we sql load that same character and look at it in an oracle form, we see the question marks.
    i've looked at many of the documents on tahiti already, and found some helpful stuff, but no explanation for the problem i am having so far :)

  • Need suggestions for imporving data load performance via SQL Loader

    Hi,
    Our requirement is to load 512 (1 GB each) files in Oracle database.
    We are using SQL loaders to load files into the DB (A partitioned table) and have tried almost all the possible options that come with sql loaders (Direct load path, parallel=true, multithreading=true, unrecoverable)
    As the tables is growing bigger in size, each file load time is increasing (It started with 5 minutes per file and has reached 2 hours per 3 files now and is increasing with every batch- Note we are loading 3 files concurrently on the target table using the parallel = true oprion of sql loader)
    Questions 1:
    My problem is that somehow multithreading is not working for us (we have multi CPU server and have enabled multithreading=true). Could it be something to do with DB setting which might be hindering the data load to be done in multiple threads?
    Question 2:
    Would gathering stats on the target table and it's partitions help improve load performance ? I'm not sure if stats improve DML's, they would definitely improve sql queries. Any thoughts?
    Question 3:
    What would be the best strategy to gather stats on this table (which would end up having 512 GB data) ?
    Question 4:
    Do you think insertions in a partitioned table (with growing sizes) would have poor performance as compared to a non-partitioned table ?
    Any other suggestions to improve performace are most welcome !!
    Thanks,
    Sachin
    Edited by: Sachin Tiwari on Mar 13, 2013 6:29 AM

    2 hours to load just 3 GB of data seems unreasonable regardless of the SQL Loader settings. It seems likely to me that the problem is not with SQL Loader but somewhere else.
    Have you generated a Statspack/ AWR/ ASH report to see where all that time is being spent? Are there triggers on the table? Are there bitmap indexes?
    Is your table partitioned in a way that is designed to improve the efficiency of loads so that all the data from one file goes into one partition? Or is data from each file getting inserted into many different partitions.
    Justin

  • Dumb question re Sql Loader in XP Client

    Hi all,
    I have a rather silly question. Previously I've worked exclusively in a Unix environment, but now find myself in XP.
    I've done the full install of the Windows client for XP, but SQL*Loader is missing. I've RTFM'd, but can't find the necessary section. How do I get sql*loader installed in the XP environment?
    Thanks (and apologies) in advance

    IF you are talking of the 9i version, there should be the three database install disks (that contain both the database and the client tools). When you run the installer from the first disk from this set, you will have the option of installing the client tools (including SQL*Loader).
    http://www.oracle.com/technology/software/products/database/oracle10g/index.html
    http://www.oracle.com/technology/software/products/oracle9i/htdocs/winsoft.html

  • SQL LOADER LPAD CONTROL FILE QUESTION

    Hi, inthe flat file
    company_cd is "1" or "01"
    and center_cd is 3 digits... ex: "493"
    in the table the userid coulmn should be 6 digits
    currently i am getting this as userid in the table
    "010493" is right, because company_cd is "01"
    "10493" is not right, because company_cd is "1"
    if company_cd is 2 digits(01) i am getting 6 digits userid which is OK
    but when company_cd is singile digit(1) i am getting 5 digits userid
    I NEED TO LPAD with 0 in the front when company_cd is "1"any suggetions ???????
    ***********This is the code i am using currently in the CTL file for userid**********
    ,USERID "CONCAT(substr(trim(:company_cd),1,2),lpad(trim(:center_cd),4,0))"
    .......Thank You..........
    Edited by: phani_Marella on Aug 28, 2012 11:12 AM

    Now where does company 'coz' come from all of a sudden?
    I'm sure you read {message:id=9360002} , hence my confusion.
    Anyway, the SQL*Loader forum is @ Export/Import/SQL Loader & External Tables

  • Question about 'direct' parameter in sql loader utility

    Hi eveyone,
    Today I was asked what the advantages and disadvantages of 'direct' parameter of sql loader.
    In my opinion, if swithing direct on, oracle use direct path load.
    And it has no difference from the append hind in insert statement.
    The core merits are less redo hence faster.
    The main shortcoming is since little redo generated, we have to ask DBA to back up the data according to the rules immediately after the loading.
    Does my answer make sense or not? Plesae help to point out the important things I missed.
    best regards,
    Leon

    Hi,
    Direct path Export is much faster than conventional path Export because the data is read from disk into the buffer cache and rows are transferred directly to the Export client.
    Some conditions to use DIRECT parameter are,
    1. The Export parameter BUFFER applies only to conventional path Exports. For direct path Export, use the RECORDLENGTH parameter to specify the size of the buffer that Export uses for writing to the export file.
    2. You cannot use direct path when exporting in tablespace mode (TRANSPORT_TABLESPACES=Y).
    3. The QUERY parameter cannot be specified in a direct path Export.
    Thanks

  • SQL Loader newbie's question.

    Hi all,
    I have a delimiter file, e.g. data.dat, that contains the titles of each of the columns on the first row. Subsequent rows contain real column information/data. I didn't know how to create a control file, e.g. data.ctl, to create the table, its columns, and its data.
    Currently, I got to create it manually, e.g.: table, columns, and used sql*loader to load in the data. Is there a better way to do this? Thank you very much for your help.

    Yes, Sir. My external data is arranged as followed, e.g.:
    "x","y","z"
    "1","2","3"
    "2","3","4"
    "3","4","5"
    where x, y, and z are the titles of the columns, not the data. My data are "1","2",...
    Thank you very much for your response.

  • Question for SQL Loader / IMP /EXPORT

    hi all,
    i just describe my example first : -
    TABLE A have five columns.
    a_column,b_column,c_column,d_column,e_column.
    and have 1000 records.
    i want to remove c_column & e_column and add f_column.
    on TABLE A. which method (SQL Loader or IMP/EXP) is good suggestion for export & import data before take above action.
    boris

    Export/Import is the most appropiate solution for the task that you want to carry out because SQL*Loader is designed to load data into the database from flat files.
    Regarding how to drop the two columns , you can do it
    directly if you are working in 9i setting both columns as
    unused after drop unused columns in the table.
    Complete reference for those utilities:
    http://download-uk.oracle.com/docs/cd/B10501_01/server.920/a96652.pdf
    Joel P�rez

  • Sql loader - skip record question

    I am running Oracle 9 and using sql loader to import text file into table. Can sql loader skips the record which contain blank line or carriage return? Do I need to set up with options? Please advise me how. Thanks.

    http://docs.oracle.com/cd/B10500_01/server.920/a96652/ch05.htm
    http://www.orafaq.com/wiki/SQL*Loader_FAQ

  • Problem in SQL Loader

    Hi Experts,
    i'm using SQL Loader for loading data from an XML file into the DB , my control file was some thing like that :
    load data
    infile 'D:data.xml' "str '</dataNode>'"
    replace
    into table MY_TABLE
    where dataNode is the records' separator, this was working fine, what i'm trying to do now is writing all the parameters to a PARFILE , then passing only the PARFILE to the sql loader like the following :
    sqlldr PARFILE=myParaFile.par
    where myParaFile looks like that :
    userid=xxx/xxx
    control=xxx.ctl
    log=xxx.log
    data=D:\data.xml
    the problem now that i have removed the INFILE clause from the control file , and i have put the "data" parameter insetad on the PARFILE , the question now is Where shall i write "str '</dataNode>'" to tell the SQL Loader that the input data is in stream format and use </dataNode> as records' separator
    I really appreciate your help.

    My XML File:
    <dataNode>
    <ProductID>1</ProductID>
    <Type>Phone</Type>
    </dataNode>
    <dataNode>
    <ProductID>2</ProductID>
    <Type>Sim</Type>
    </dataNode>
    My Control File :
    load data
    infile 'D:data.xml' "str '</dataNode>'"
    replace
    into table MY_TABLE
    dummy filler terminated by "<dataNode>",
    ProductID enclosed by "<ProductID>" and "</ProductID>",
    Type enclosed by "<Type>" and "</Type>"
    )

  • Load Xml Data in oracle (SQL LOADER)

    i have written a sql loader script given below....
    load data
    infile 'contact1.xml' "str '</CONTACT>'"
    truncate
    into table XmlTest
    dummy filler terminated by ">",
    Location_id enclosed by "<Location_id>" and "</Location_id>",
    fname enclosed by "<fname>" and "</fname>",
    lname enclosed by "<lname>" and "</lname>"
    the content of the contact1.xml is given below.....
    <CONTACT>
    <Location_id>1</Location_id>
    <fname>Robert</fname>
    <lname>Jones</lname>
    </CONTACT>
    <CONTACT>
    <Location_id>1</Location_id>
    <fname>Robert</fname>
    <lname>Jones</lname>
    </CONTACT>
    the script works fine for the given xml... but if one of the tag in the xml is changed to below syntax....
    <CONTACT Location_id="1">
    <fname>Robert</fname>
    <lname>Jones</lname>
    </CONTACT>
    <CONTACT Location_id="1">
    <fname>Robert</fname>
    <lname>Jones</lname>
    </CONTACT>
    what modification is required in the script to read such xml...Kindly Help!!

    And in what way does this relate to SQL and PL/SQL?
    There is a demo of loading XML in Morgan's Library under SQL*Loader.
    But as your question is off-topic here ... if the demo does not help you please continue this by starting a thread in the correct forum: Database General.
    Thanks.

Maybe you are looking for