Ordering bulk inserts

The last time I looked for this was back in Oracle 7 so forgive me for trying again. I have an application that transmits large quantities of data between different database instances. This was originally designed (by someone else) using array inserts. The developer relied on the usual chance that the rows when inserted will be in the same order as when they were selected (bad choice). I know that there is not any "guaranteed" relationship between the order the rows were stored in the source database and the order they will be stored in the receiving database. What I need is a way to ensure that the rows go through a trigger on the receiving database in the same order they were selected from the source database. Anything new that might be of use. We are on 9.0.2. Just to be clear, the order the rows are physically stored in the database is not what I am talking about. I could care less about that. I just want to control the order they are processed through the receiving trigger.

I was afraid that might be the answer still. It all still comes back to a bad design to start out with. There is a sequence number assigned when the source row is inserted, but it isn't passed to the second table. There is a batch number created on the secondary table when the rows are inserted from the first table. They also populate another sequence on the incoming table and order by that sequence when processing the records. If they had passed the original sequence from the source table to the destination table as a column of data, and then ordered by the batch number and then the source table sequence, I wouldn't be having this problem. Unfortunately this is a very tightly regulated environment and it would take an act of congress to get a change of that scope pushed through.
Thanks
Chip

Similar Messages

BULK INSERT from a text (.csv) file - read only specific columns.

I am using Microsoft SQL 2005, I need to do a BULK INSERT from a .csv I just downloaded from paypal. I can't edit some of the columns that are given in the report. I am trying to load specific columns from the file.
bulk insert Orders
FROM 'C:\Users\*******\Desktop\DownloadURL123.csv'
   WITH
              FIELDTERMINATOR = ',',
                FIRSTROW = 2,
                ROWTERMINATOR = '\n'
So where would I state what column names (from row #1 on the .csv file) would be used into what specific column in the table.
I saw this on one of the sites which seemed to guide me towards the answer, but I failed.. here you go, it might help you:
FORMATFILE [ = 'format_file_path' ]
Specifies the full path of a format file. A format file describes the data file that contains stored responses created using the bcp utility on the same table or view. The format file should be used in cases in which:
The data file contains greater or fewer columns than the table or view.
The columns are in a different order.
The column delimiters vary.
There are other changes in the data format. Format files are usually created by using the bcp utility and modified with a text editor as needed. For more information, see bcp Utility.

Date, Time, Time Zone, Name, Type, Status, Currency, Gross, Fee, Net, From Email Address, To Email Address, Transaction ID, Item Title, Item ID, Buyer ID, Item URL, Closing Date, Reference Txn ID, Receipt ID,
"04/22/07", "12:00:21", "PDT", "Test", "Payment Received", "Cleared", "USD", "321", "2.32", "3213', "[email protected]", "[email protected]", "", "testing", "392302", "jdal32", "http://ddd.com", "04/22/03", "", "",
"04/22/07", "12:00:21", "PDT", "Test", "Payment Received", "Cleared", "USD", "321", "2.32", "3213', "[email protected]", "[email protected]", "", "testing", "392932930302", "jejsl32", "http://ddd.com", "04/22/03", "", "",
Do you need more than 2 rows? I did not include all the columns from the actual csv file but most of it, I am planning on taking to the first table these specfic columns: date, to email address, transaction ID, item title, item ID, buyer ID, item URL.
The other table, I don't have any values from here because I did not list them, but if you do this for me I could probably figure the other table out.
Thank you very much.

How to use BULK INSERT for a data from a cursor?

Oracle 10G enterprise edition.
I tried to Bulk insert datas returning from a cursor, its returning error.
PLS-00302: component 'LAST' must be declared
I need some help to use the Bulk INSERT here.Can any one help me to specify what error i have made?
CREATE OR REPLACE PROCEDURE HOT_ADMIN.get_search_keyword_stats_prc
IS
CURSOR c_get_scenarios
IS
SELECT a.*,ROWNUM rnum
FROM (
SELECT TRUNC(r.search_date) sdate,
r.search_hits hits,
r.search_type stype,
r.search_qualification qual,
r.search_location loc,
r.search_town stown,
r.search_postcode pcode,
r.search_college college,
r.search_colname colname,
r.search_text text,
r.affiliate_id affiliate,
r.search_study_mode smode,
r.location_hint hint,
r.search_posttown ptown,
COUNT(1) cnt
FROM w_search_headers r
WHERE search_text IS NOT NULL
AND NVL(search_type,' ') <> 'C'
AND TRUNC(search_date)= TO_DATE(TO_CHAR(SYSDATE-1,'DD-MON-RRRR'))
GROUP BY TRUNC(r.search_date),
r.search_hits,
r.search_type,
r.search_qualification,
r.search_location,
r.search_town,
r.search_postcode,
r.search_college,
r.search_colname,
r.search_text,
r.affiliate_id,
r.search_study_mode,
r.location_hint,
r.search_posttown
ORDER BY cnt desc
) a
WHERE ROWNUM <=1000;
lc_get_data c_get_scenarios%ROWTYPE;
BEGIN
OPEN c_get_scenarios;
FETCH c_get_scenarios into lc_get_data;
CLOSE c_get_scenarios;
FORALL i IN 1..lc_get_data.last
INSERT INTO W_SEARCH_SCENARIO_STATS VALUES ( i.sdate,
i.hits,
i.stype,
i.qual,
i.loc,
i.stown,
i.pcode,
i.college,
i.colname,
i.text,
i.affiliate,
i.smode,
i.hint,
i.ptown,
i.cnt
COMMIT;
END;

This isn't what you asked, but I've generally found it helpful to list the columns in an INSERT statement before the values. It is of course optional, but useful for reference when looking at the statement later

HOW to USE 'LONG RAW' in Bulk Insertion using OCI

Hi,
I need to do bulk insertion of LONG RAW data into a table using the OCI.
To the OCIBindByPos API what should I specify in the field size(value_sz). As different records can have different lenght for this LONG RAW data what value should I provide.
Thanks,
Tuhin
sword OCIBindByPos ( OCIStmt *stmtp,
OCIBind **bindpp,
OCIError *errhp,
ub4 position,
dvoid *valuep,
sb4 value_sz,
ub2 dty,
dvoid *indp,
ub2 *alenp,
ub2 *rcodep,
ub4 maxarr_len,
ub4 *curelep,
ub4 mode );

ORA-00997: illegal use of LONG datatype
Cause: A value of datatype LONG was used in a function or in a DISTINCT, WHERE, CONNECT BY, GROUP BY, or ORDER BY clause. A LONG value can only be used in a SELECT clause.
Action: Remove the LONG value from the function or clause
Are you using the column anywhere else but in the SELECT?

FDM bulk insert error

Hello,
i'm trying to bulk insert data into planning and i get an error. As i can understand it seems related to some bad settings regarding decimal separator. I'm from italy where default decimal separator is ",". As far as i can check everything is set on english including Oracle Client (... sqlldr).
As you can see in datafile.log i get oracle ORA-01722 error.
Any ideas?
Thank you in advance,
Daniele
S.O: windows2008
FDM Version: 11.1.2.1
REPOSITORY: Oracle 11.2.0.1.0 - 64bit
Here are log files:
admin.log:
** Begin FDM Runtime Error Log Entry [2011-09-07 16:50:18] **
ERROR:
Code............................................. 4003
Description...................................... Oracle (SQL-Loader) data load failed, please see processing log for details!
Procedure........................................ clsImpProcessMgr.fLoadAndProcessFile
Component........................................ upsWObjectsDM
Version.......................................... 1112
Thread........................................... 4040
IDENTIFICATION:
User............................................. admin
Computer Name.................................... IPEREPMA00
App Name......................................... FDMAuchan
Client App....................................... WebClient
CONNECTION:
Provider......................................... ORAOLEDB.ORACLE
Data Server......................................
Database Name.................................... HYPTEST
Trusted Connect.................................. False
Connect Status.. Connection Open
GLOBALS:
Location......................................... LOADACTSAP
Location ID...................................... 750
Location Seg..................................... 4
Category......................................... SAP
Category ID...................................... 13
Period........................................... Jan - 2011
Period ID........................................ 1/31/2011
POV Local........................................ False
Language......................................... 1033
User Level....................................... 1
All Partitions................................... True
Is Auditor....................................... False
datafile.log
+2011-09-07 16:50:16+
User ID...........     admin
Location..........     LOADACTSAP
Source File.......     D:\Oracle\Middleware\EPMSystem11R1\products\FinancialDataQuality\SharedComponents\APPS\FDMAuchan\Inbox\LOADACTSAP\SAP_1101_Prova.txt
Processing Codes:
BLANK............. Line is blank or empty.
ESD............... Excluded String Detected, SKIP Field value was found.
NN................ Non-Numeric, Amount field contains non numeric characters.
RFM............... Required Field Missing.
TC................ Type Conversion, Amount field could be converted to a number.
ZP................ Zero Suppress, Amount field contains a 0 value and zero suppress is ON.
Create Output File Start: [2011-09-07 16:50:16]
+[Blank] -      +
Excluded Record Count..............0
Blank Record Count.................1
Total Records Bypassed.............1
Valid Records......................477
Total Records Processed............478
Begin Oracle (SQL-Loader) Process (477): [2011-09-07 16:50:16]
Oracle (SQL-Loader) Log File Contents:
SQL*Loader: Release 11.2.0.1.0 - Production on Mer Set 7 16:50:16 2011
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Control File:   C:\Users\INSTAL~1\AppData\Local\Temp\tWadmin154840402517.ctl
Character Set UTF16 specified for all input.
Using character length semantics.
First primary datafile C:\Users\INSTAL~1\AppData\Local\Temp\tWadmin154840402517.tmp has a
little endian byte order mark in it.
Data File:      C:\Users\INSTAL~1\AppData\Local\Temp\tWadmin154840402517.tmp
Bad File:     C:\Users\INSTAL~1\AppData\Local\Temp\tWadmin154840402517.bad
Discard File: none specified
+(Allow all discards)+
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Continuation:    none specified
Path used:      Direct
Load is UNRECOVERABLE; invalidation redo is produced.
Table TWADMIN154840402517, loaded from every logical record.
Insert option in effect for this table: APPEND
TRAILING NULLCOLS option in effect
Column Name                  Position   Len Term Encl Datatype
DATAKEY                                                   SEQUENCE (1, 1)
PARTITIONKEY                        FIRST     *           CHARACTER
Terminator string : '~|'
CATKEY                               NEXT     *           CHARACTER
Terminator string : '~|'
PERIODKEY                            NEXT     *           DATE YYYYMMDD
Terminator string : '~|'
DATAVIEW                             NEXT     *           CHARACTER
Terminator string : '~|'
ACCOUNT                              NEXT     *           CHARACTER
Terminator string : '~|'
ENTITY                               NEXT     *           CHARACTER
Terminator string : '~|'
ICP                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD1                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD2                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD3                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD4                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD5                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD6                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD7                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD8                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD9                                  NEXT     *           CHARACTER
Terminator string : '~|'
UD10                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD11                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD12                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD13                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD14                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD15                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD16                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD17                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD18                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD19                                 NEXT     *           CHARACTER
Terminator string : '~|'
UD20                                 NEXT     *           CHARACTER
Terminator string : '~|'
DESC1                                NEXT     *           CHARACTER
Terminator string : '~|'
DESC2                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR1                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR2                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR3                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR4                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR5                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR6                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR7                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR8                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR9                                NEXT     *           CHARACTER
Terminator string : '~|'
ATTR10                               NEXT     *           CHARACTER
Terminator string : '~|'
ATTR11                               NEXT     *           CHARACTER
Terminator string : '~|'
ATTR12                               NEXT     *           CHARACTER
Terminator string : '~|'
ATTR13                               NEXT     *           CHARACTER
Terminator string : '~|'
ATTR14                               NEXT     *           CHARACTER
Terminator string : '~|'
MEMOKEY                              NEXT     *           CHARACTER
Terminator string : '~|'
AMOUNT                               NEXT     *           CHARACTER
Terminator string : '~|'
CALCACCTTYPE                                              CONSTANT
Value is '9'
CHANGESIGN                                                CONSTANT
Value is '0'
AMOUNTX                                                   CONSTANT
Value is '0'
ACCOUNTR                                                  CONSTANT
Value is '0'
ACCOUNTF                                                  CONSTANT
Value is '0'
ENTITYR                                                   CONSTANT
Value is '0'
ENTITYF                                                   CONSTANT
Value is '0'
ICPR                                                      CONSTANT
Value is '0'
ICPF                                                      CONSTANT
Value is '0'
UD1R                                                      CONSTANT
Value is '0'
UD1F                                                      CONSTANT
Value is '0'
UD2R                                                      CONSTANT
Value is '0'
UD2F                                                      CONSTANT
Value is '0'
UD3R                                                      CONSTANT
Value is '0'
UD3F                                                      CONSTANT
Value is '0'
UD4R                                                      CONSTANT
Value is '0'
UD4F                                                      CONSTANT
Value is '0'
UD5R                                                      CONSTANT
Value is '0'
UD5F                                                      CONSTANT
Value is '0'
UD6R                                                      CONSTANT
Value is '0'
UD6F                                                      CONSTANT
Value is '0'
UD7R                                                      CONSTANT
Value is '0'
UD7F                                                      CONSTANT
Value is '0'
UD8R                                                      CONSTANT
Value is '0'
UD8F                                                      CONSTANT
Value is '0'
UD9R                                                      CONSTANT
Value is '0'
UD9F                                                      CONSTANT
Value is '0'
UD10R                                                     CONSTANT
Value is '0'
UD10F                                                     CONSTANT
Value is '0'
UD11R                                                     CONSTANT
Value is '0'
UD11F                                                     CONSTANT
Value is '0'
UD12R                                                     CONSTANT
Value is '0'
UD12F                                                     CONSTANT
Value is '0'
UD13R                                                     CONSTANT
Value is '0'
UD13F                                                     CONSTANT
Value is '0'
UD14R                                                     CONSTANT
Value is '0'
UD14F                                                     CONSTANT
Value is '0'
UD15R                                                     CONSTANT
Value is '0'
UD15F                                                     CONSTANT
Value is '0'
UD16R                                                     CONSTANT
Value is '0'
UD16F                                                     CONSTANT
Value is '0'
UD17R                                                     CONSTANT
Value is '0'
UD17F                                                     CONSTANT
Value is '0'
UD18R                                                     CONSTANT
Value is '0'
UD18F                                                     CONSTANT
Value is '0'
UD19R                                                     CONSTANT
Value is '0'
UD19F                                                     CONSTANT
Value is '0'
UD20R                                                     CONSTANT
Value is '0'
UD20F                                                     CONSTANT
Value is '0'
ARCHIVEID                                                 CONSTANT
Value is '774'
HASMEMOITEM                                               CONSTANT
Value is '0'
STATICDATAKEY                                             CONSTANT
Value is '0'
Referential Integrity Constraint/Trigger Information:
NULL, UNIQUE, and PRIMARY KEY constraints are unaffected.
Trigger FDMAUCHAN."TWADMIN154840402517_AK" was disabled before the load.
Record 1: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 2: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 3: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 4: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 5: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 6: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 7: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 8: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 9: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 10: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 11: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 12: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 13: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 14: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 15: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 16: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 17: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 18: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 19: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 20: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 21: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 22: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 23: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 24: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 25: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 26: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 27: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 28: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 29: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 30: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 31: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 32: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 33: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 34: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 35: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 36: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 37: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 38: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 39: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 40: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 41: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 42: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 43: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 44: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 45: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 46: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 47: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 48: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 49: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 50: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
Record 51: Rejected - Error on table TWADMIN154840402517, column AMOUNT.
ORA-01722: numero non valido
FDMAUCHAN."TWADMIN154840402517_AK" was re-enabled.
MAXIMUM ERROR COUNT EXCEEDED - Above statistics reflect partial run.
Table TWADMIN154840402517:
+0 Rows successfully loaded.+
+51 Rows not loaded due to data errors.+
+0 Rows not loaded because all WHEN clauses were failed.+
+0 Rows not loaded because all fields were null.+
Bind array size not used in direct path.
Column array rows :    3000
Stream buffer bytes: 256000
Read   buffer bytes:18576000
Total logical records skipped:          0
Total logical records rejected:        51
Total logical records discarded:        0
Total stream buffers loaded by SQL*Loader main thread:        1
Total stream buffers loaded by SQL*Loader load thread:        0
Run began on Mer Set 07 16:50:16 2011
Run ended on Mer Set 07 16:50:17 2011
Elapsed time was:     00:00:00.56
CPU time was:         00:00:00.24
Oracle (SQL-Loader) Bad File Contents:
+750~|13~|20110131~|YTD~|701306~|1854~|~|153~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711200~|0355~|~|190~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711260~|1822~|~|119~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711260~|1858~|~|142~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711261~|3601~|~|160~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711302~|5308~|~|999~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711383~|1748~|~|010~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711383~|1857~|~|010~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711458~|1608~|~|191~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|0355~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|0392~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|1565~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|1784~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|1823~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|1848~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711468~|1853~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711605~|2644~|~|555~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711606~|3601~|~|555~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711641~|2389~|~|999~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|711655~|2649~|~|116~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721330~|1161~|~|155~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721330~|1852~|~|010~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721330~|3539~|~|999~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721330~|5158~|~|999~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721340~|3473~|~|190~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721341~|4263~|~|999~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|721341~|5617~|~|999~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|730001~|3898~|~|010~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|730003~|0355~|~|192~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|730003~|0392~|~|140~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|730003~|1565~|~|142~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|730003~|1795~|~|134~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|730003~|1848~|~|126~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|732000~|1823~|~|130~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|734001~|3601~|~|151~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|761202~|1748~|~|180~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|761205~|5309~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|761206~|1608~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|761206~|1823~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|761210~|1857~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|761210~|7347~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H11468~|1854~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H11468~|2644~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H11468~|5309~|~|181~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H11473~|1784~|~|191~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H11605~|2644~|~|151~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H21330~|1161~|~|123~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H21330~|1161~|~|156~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H21340~|3898~|~|154~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H30000~|1854~|~|177~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+750~|13~|20110131~|YTD~|H30000~|1854~|~|188~|SAP~|DEF_ATT~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|~|0~|100.00+
+[RDMS Bulk Load Error Begin]+
+     Message:      (4003) - Oracle (SQL-Loader) data load failed, please see processing log for details!+
+     See Bulk Load File:      C:\Users\INSTAL~1\AppData\Local\Temp\tWadmin154840402517.tmp+
+[RDMS Bulk Load Error End]+

Hi Daniele,
Can i know the steps to resolve this issue.
I am also facing the same issue. Below is the error message.
[RDMS Bulk Load Error Begin]
Message: (4003) - Oracle (SQL-Loader) data load failed, please see processing log for details!
See Bulk Load File: C:\temp\tWthinad824889961810.tmp
[RDMS Bulk Load Error End]
I am using the Essbase as the Target adaptor. Your reply will be very helpful for me.
Thanks
Dhinesh Kumar T

Query on Bulk Insert

I am using bulk insert to load data into some tables.What limit i should specify for commit to be done, in order to have the best possible performance.

I don't think there is any fixed sort of rule is there to identify the limit numbers.
As Morgan suggested 2500 or the mentioned number for your purpose.
Kindly check the following links ->
http://www.psoug.org/reference/array_processing.html
http://www.oracle.com/technology/oramag/oracle/08-mar/o28plsql.html
http://www.dba-oracle.com/plsql/t_plsql_limit_clause.htm
Regards.
Satyaki De.

Max limit for bulk insert.

Hi friends,
We have 100 Million records in a table on production database. We are planning to move the data to test environment for performance tuning of the databse and queries.
We have created a DB Link between these DB's to transfer the data between the database.
We are planning to move the data in the following fashion:
insert into tab1(.....)
select * from tab1@prod_db_link;
do we have any limit for bulk insert?

> We are planning to move the data to test environment for performance tuning of the databse and
queries.
Flawed premise. What makes you think that the test environment will match the production with a 100% degree of accuracy!? - as that is what you need in order to
a) identify the performance bottlenecks
b) solve these
You are very much mistaken if you think you can identify actual performance issues happening on prod by mucking about on a test environment that just happens to have the same data volume.
Even data volume alone is meaningless as a copy of the data is logical - any potential issues with pctfree and pctused will not be reflected. Any potential hot spots on disk will not be reflected. The copy will not have the same number of segments and extents. Etc. Etc.
And this is just the data.. never mind numerous other issues ranging from actual hardware and operating system to Oracle instance configuration and production load and processing.

Bulk Insert Error message

I have tried to enable bulk insert on a location. SQL* Loader is installed and appears to work, but I get an error when it tries to load the temp table into my Oracle database. Here's the output from the load file log:
2011-05-12-10:43:09
User ID...........     xxx
Location..........     xxx
Source File.......     xxx
Processing Codes:
BLANK............. Line is blank or empty.
ESD............... Excluded String Detected, SKIP Field value was found.
NN................ Non-Numeric, Amount field contains non numeric characters.
RFM............... Required Field Missing.
TC................ Type Conversion, Amount field could be converted to a number.
ZP................ Zero Suppress, Amount field contains a 0 value and zero suppress is ON.
Create Output File Start: [2011-05-12-10:43:10]
Begin Oracle (SQL-Loader) Process (42386): [2011-05-12-10:43:17]
Oracle (SQL-Loader) Log File Contents:
SQL*Loader: Release 10.2.0.1.0 - Production on Thu May 12 10:43:17 2011
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Control File: C:\WINDOWS\TEMP\tWrobert358876967263.ctl
Character Set UTF16 specified for all input.
Using character length semantics.
First primary datafile C:\WINDOWS\TEMP\tWrobert358876967263.tmp has a
little endian byte order mark in it.
Data File: C:\WINDOWS\TEMP\tWrobert358876967263.tmp
Bad File: C:\WINDOWS\TEMP\tWrobert358876967263.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Continuation: none specified
Path used: Direct
Load is UNRECOVERABLE; invalidation redo is produced.
Table TWROBERT358876967263, loaded from every logical record.
Insert option in effect for this table: APPEND
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
DATAKEY SEQUENCE (1, 1)
PARTITIONKEY FIRST * CHARACTER
Terminator string : '~|'
CATKEY NEXT * CHARACTER
Terminator string : '~|'
PERIODKEY NEXT * DATE YYYYMMDD
Terminator string : '~|'
DATAVIEW NEXT * CHARACTER
Terminator string : '~|'
ACCOUNT NEXT * CHARACTER
Terminator string : '~|'
ENTITY NEXT * CHARACTER
Terminator string : '~|'
ICP NEXT * CHARACTER
Terminator string : '~|'
UD1 NEXT * CHARACTER
Terminator string : '~|'
UD2 NEXT * CHARACTER
Terminator string : '~|'
UD3 NEXT * CHARACTER
Terminator string : '~|'
UD4 NEXT * CHARACTER
Terminator string : '~|'
UD5 NEXT * CHARACTER
Terminator string : '~|'
UD6 NEXT * CHARACTER
Terminator string : '~|'
UD7 NEXT * CHARACTER
Terminator string : '~|'
UD8 NEXT * CHARACTER
Terminator string : '~|'
UD9 NEXT * CHARACTER
Terminator string : '~|'
UD10 NEXT * CHARACTER
Terminator string : '~|'
UD11 NEXT * CHARACTER
Terminator string : '~|'
UD12 NEXT * CHARACTER
Terminator string : '~|'
UD13 NEXT * CHARACTER
Terminator string : '~|'
UD14 NEXT * CHARACTER
Terminator string : '~|'
UD15 NEXT * CHARACTER
Terminator string : '~|'
UD16 NEXT * CHARACTER
Terminator string : '~|'
UD17 NEXT * CHARACTER
Terminator string : '~|'
UD18 NEXT * CHARACTER
Terminator string : '~|'
UD19 NEXT * CHARACTER
Terminator string : '~|'
UD20 NEXT * CHARACTER
Terminator string : '~|'
DESC1 NEXT * CHARACTER
Terminator string : '~|'
DESC2 NEXT * CHARACTER
Terminator string : '~|'
ATTR1 NEXT * CHARACTER
Terminator string : '~|'
ATTR2 NEXT * CHARACTER
Terminator string : '~|'
ATTR3 NEXT * CHARACTER
Terminator string : '~|'
ATTR4 NEXT * CHARACTER
Terminator string : '~|'
ATTR5 NEXT * CHARACTER
Terminator string : '~|'
ATTR6 NEXT * CHARACTER
Terminator string : '~|'
ATTR7 NEXT * CHARACTER
Terminator string : '~|'
ATTR8 NEXT * CHARACTER
Terminator string : '~|'
ATTR9 NEXT * CHARACTER
Terminator string : '~|'
ATTR10 NEXT * CHARACTER
Terminator string : '~|'
ATTR11 NEXT * CHARACTER
Terminator string : '~|'
ATTR12 NEXT * CHARACTER
Terminator string : '~|'
ATTR13 NEXT * CHARACTER
Terminator string : '~|'
ATTR14 NEXT * CHARACTER
Terminator string : '~|'
MEMOKEY NEXT * CHARACTER
Terminator string : '~|'
AMOUNT NEXT * CHARACTER
Terminator string : '~|'
CALCACCTTYPE CONSTANT
Value is '9'
CHANGESIGN CONSTANT
Value is '0'
AMOUNTX CONSTANT
Value is '0'
ACCOUNTR CONSTANT
Value is '0'
ACCOUNTF CONSTANT
Value is '0'
ENTITYR CONSTANT
Value is '0'
ENTITYF CONSTANT
Value is '0'
ICPR CONSTANT
Value is '0'
ICPF CONSTANT
Value is '0'
UD1R CONSTANT
Value is '0'
UD1F CONSTANT
Value is '0'
UD2R CONSTANT
Value is '0'
UD2F CONSTANT
Value is '0'
UD3R CONSTANT
Value is '0'
UD3F CONSTANT
Value is '0'
UD4R CONSTANT
Value is '0'
UD4F CONSTANT
Value is '0'
UD5R CONSTANT
Value is '0'
UD5F CONSTANT
Value is '0'
UD6R CONSTANT
Value is '0'
UD6F CONSTANT
Value is '0'
UD7R CONSTANT
Value is '0'
UD7F CONSTANT
Value is '0'
UD8R CONSTANT
Value is '0'
UD8F CONSTANT
Value is '0'
UD9R CONSTANT
Value is '0'
UD9F CONSTANT
Value is '0'
UD10R CONSTANT
Value is '0'
UD10F CONSTANT
Value is '0'
UD11R CONSTANT
Value is '0'
UD11F CONSTANT
Value is '0'
UD12R CONSTANT
Value is '0'
UD12F CONSTANT
Value is '0'
UD13R CONSTANT
Value is '0'
UD13F CONSTANT
Value is '0'
UD14R CONSTANT
Value is '0'
UD14F CONSTANT
Value is '0'
UD15R CONSTANT
Value is '0'
UD15F CONSTANT
Value is '0'
UD16R CONSTANT
Value is '0'
UD16F CONSTANT
Value is '0'
UD17R CONSTANT
Value is '0'
UD17F CONSTANT
Value is '0'
UD18R CONSTANT
Value is '0'
UD18F CONSTANT
Value is '0'
UD19R CONSTANT
Value is '0'
UD19F CONSTANT
Value is '0'
UD20R CONSTANT
Value is '0'
UD20F CONSTANT
Value is '0'
ARCHIVEID CONSTANT
Value is '2260'
HASMEMOITEM CONSTANT
Value is '0'
STATICDATAKEY CONSTANT
Value is '0'
Referential Integrity Constraint/Trigger Information:
NULL, UNIQUE, and PRIMARY KEY constraints are unaffected.
Trigger FDMAPP2."TWROBERT358876967263_AK" was disabled before the load.
SQL*Loader-951: Error calling once/load initialization
ORA-00942: table or view does not exist
Processing Complete... [2011-05-12-10:43:18]

Hi rjgideon,
At the very end it says "ORA-00942: table or view does not exist". Looks like SQL*Loader is trying to insert data into a table that doesn't exist.
Take a look at the control file: C:\WINDOWS\TEMP\tWrobert358876967263.ctl
You might find a hint in there.
Regards,
Matt

How can I debug a Bulk Insert error?

I'm loading a bunch of files into SQL server. All work fine, but one keeps erroring out on me. All files should be exactly the same in structure, but they have different dates, and other different financial metrics, but the structure and field
names should be exactly the same. Nevertheless, one keeps konking out, and throwing this error.
Msg 4832, Level 16, State 1, Line 1
Bulk load: An unexpected end of file was encountered in the data file.
Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.
Msg 7330, Level 16, State 2, Line 1
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
The ROWTERMINATOR should be CRLF, and when you look at it in Notepad++ that's what it looks like, but it must be something else, because I keep getting errors here. I tried the good old: ROWTERMINATOR='0x0a'
That works on all files, but one, so there's something funky going on here, and I need to see what SQL Server is really doing.
Is there some way to print out a log, or look at a log somewhere?
Thanks!!
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.

The first thing to try is to see if BCP likes the file. BCP and BULK INSERT adhere to the same spec, but they are different implementations, but there are subtle differences.
There is an ERRORFILE option, but it more helps when there is bad data.
You can also use the BATCHSIZE option to see how many records in the file it swallows, before things go bad. FIRSTROW and LASTROW can also help.
All in all, it can be quite tedious find that single row where things are different - and where BULK INSERT loses sync entirely. Keep in mind that it reads fields on by one, and it there is one field terminator to few on a line, it will consume the line
feed at the end of the line as data.
Erland Sommarskog, SQL Server MVP, [email protected]

ODBC, bulk inserts and dynamic SQL

I am writing an application running on Windows NT 4 and using the oracle ODBC driver (8.01.05.00, that inserts many rows at a time (10000+) into an oracle 8i database.
At present, I am using a stored procedure to insert each row into the database. The stored procedure uses dynamic SQL because I can only determine the table and field names at run time.
Due to the large number of records, it tends to take a while to perform all the inserts. I have tried a number of solutions such as using batches of SQL statements (e.g. "INSERT...;INSERT...;INSERT..."), but the oracle ODBC driver only seems act on the first statement in the batch.
I have also considered using the FOR ALL statement and SQL*Loader utility.
My problem with FOR ALL is that I'm not sure it works on dynamic SQL statements and even if it did, how do I pass an array of statements to the stored procedure.
I ruled out SQL* Loader because I could not find a way to invoke it it from an ODBC statement. Secondly, it requires the spawining of a new process.
What I am really after is something similar the the SQL Server (forgive me!) BULK INSERT statement where you can simply create an input file with all the records you want to insert, and pass it along in an ODBC statement such as "BULK INSERT <filename>".
Any ideas??
null

Hi,
I faced this same situation years ago (Oracle 7.2!) and had the following alternatives.
1) Use a 3rd party tool such as Sagent or CA Info pump (very pricey $$$)
2) Use VisualC++ and OCI to hook into the array insert routines (there are examples of these in the Oracle Home).
3) Use SQL*Loader (the best performance, but no real control of what's happening).
I ended up using (2) and used the Rouge Wave dbtools.h++ library to speed up the development.
These days, I would also suggest you take a look at Perl on NT (www.activestate.com) and the DBlib modules at www.perl.org. I believe they will also do bulk loading.
Your problem is that your program is using Oracle ODBC, when you should be using Oracle OCI for best performance.
null

Bulk inserts and dynamic SQL

I am writing an application running on Windows NT 4 and using the oracle ODBC driver (8.01.05.00, that inserts many rows at a time (10000+) into an oracle 8i database.
At present, I am using a stored procedure to insert each row into the database. The stored procedure uses dynamic SQL because I can only determine the table and field names at run time.
Due to the large number of records, it tends to take a while to perform all the inserts. I have tried a number of solutions such as using batches of SQL statements (e.g. "INSERT...;INSERT...;INSERT..."), but the oracle ODBC driver only seems act on the first statement in the batch.
I have also considered using the FOR ALL statement and SQL*Loader utility.
My problem with FOR ALL is that I'm not sure it works on dynamic SQL statements and even if it did, how do I pass an array of statements to the stored procedure.
I ruled out SQL* Loader because I could not find a way to invoke it it from an ODBC statement. Secondly, it requires the spawining of a new process.
What I am really after is something similar the the SQL Server (forgive me!) BULK INSERT statement where you can simply create an input file with all the records you want to insert, and pass it along in an ODBC statement such as "BULK INSERT <filename>".
Any ideas??
null

Hi,
I faced this same situation years ago (Oracle 7.2!) and had the following alternatives.
1) Use a 3rd party tool such as Sagent or CA Info pump (very pricey $$$)
2) Use VisualC++ and OCI to hook into the array insert routines (there are examples of these in the Oracle Home).
3) Use SQL*Loader (the best performance, but no real control of what's happening).
I ended up using (2) and used the Rouge Wave dbtools.h++ library to speed up the development.
These days, I would also suggest you take a look at Perl on NT (www.activestate.com) and the DBlib modules at www.perl.org. I believe they will also do bulk loading.
Your problem is that your program is using Oracle ODBC, when you should be using Oracle OCI for best performance.
null

How to get current month from filename and bulk insert from text file into table?

I set up some dynamic SQL to help my bulk copy data from a text file to a table. This works fine for files that come in every day; I get the previous day’s data, based on the file name that’s placed
in the folder. That’s why I’m using the ‘-1’. The dates will look like this: '20140131', so I'm using type 112.
declare @fullpath1 varchar(1000)
select @fullpath1 = '''\\system.local\ms\london\FTP\' + convert(varchar, getdate()-1, 112) + '_INDEXPRICES_EOM.SPC'''
declare @cmd1 nvarchar(1000)
print (@cmd1)
select @cmd1 = 'bulk insert [dbo].[SB_Monthly] from ' + @fullpath1 + ' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec (@cmd1)
I think the syntax will be somewhat similar to this:
YEAR(date_column)=YEAR(getdate()) AND MONTH(date_column)=MONTH(getdate())
I’m not totally sure how to incorporate that into my current syntax.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.

I tried a couple versions of this.
Declare @StartDate Date, @EndDate Date
Select @StartDate = convert(varchar, getdate()-28, 112), @EndDate = convert(varchar, getdate()-1, 112)
BEGIN
declare @fullpath1 varchar(1000)
select @fullpath1 = '''\\ms\london\FTP\' + ''' between ''' + Convert(Varchar(10), @StartDate, 101) + ''' and ''' + Convert(Varchar(10), @EndDate, 101) + '''_SP.SPC'''
declare @cmd1 nvarchar(1000)
print (@cmd1)
select @cmd1 = 'bulk insert [dbo].[SPBMI_Monthly] from ' + @fullpath1 + ' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec (@cmd1)
END
Here’s the string:
bulk insert [dbo].[SPBMI_Monthly] from '\\ms\london\FTP\' between '02/03/2014' and '03/02/2014'_SP.SPC' with (FIELDTERMINATOR = '\t', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR='0x0a')
The error message I keep getting is:
Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'between'.
Msg 319, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
I feel like I’m already pushing this thing to the limit.
Maybe this last part isn’t possible.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.

SSIS BULK INSERT unsing UNC inside of ForEach Loop Container Failed could not be opened. Operating system error code 5(Access is denied.)

Hi,
I am trying to figure out how to fix my problem
Error: Could not be opened. Operating system error code 5(Access is denied.)
Process Description:
Target Database Server Reside on different Server in the Network
SSIS Package runs from a Remote Server
SSIS Package use a ForEachLoop Container to loop into a directory to do Bulk Insert
SSIS Package use variables to specified the share location of the files using UNC like this
\\server\files
Database Service accounts under the Database is runing it has full permission on the share drive were the files reside.
In the Execution Results tab shows the prepare SQL statement for the BULK insert and I can run the same exact the bulk insert in SSMS without errors, from the Database Server and from the server were SSIS package is executed.
I am on a dead end and I don’t want to re-write SSIS to use Data Flow Task because is not flexible to update when metadata of the table changed.
Below post it has almost the same situation:
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/8de13e74-709a-43a5-8be2-034b764ca44f/problem-with-bulk-insert-task-in-foreach-loop?forum=sqlintegrationservices

Insteresting how I fixed the issue, Adding the Application Name into the SQL OLAP Connection String Fixed the issue. I am not sure why SQL Server wasn't able to open the file remotely without this.

[Forum FAQ] How to use multiple field terminators in BULK INSERT or BCP command line

Introduction
Some people want to know if we can have multiple field terminators in BULK INSERT or BCP commands, and how to implement multiple field terminators in BULK INSERT or BCP commands.
Solution
For character data fields, optional terminating characters allow you to mark the end of each field in a data file with a field terminator, as well as the end of each row with a row terminator. If a terminator character occurs within the data, it is interpreted
as a terminator, not as data, and the data after that character is interpreted and belongs to the next field or record. I have done a test, if you use BULK INSERT or BCP commands and set the multiple field terminators, you can refer to the following command.
In Windows command line,
bcp <Databasename.schema.tablename> out “<path>” –c –t –r –T
For example, you can export data from the Department table with bcp command and use the comma and colon (,:) as one field terminator.
bcp AdventureWorks.HumanResources.Department out C:\myDepartment.txt -c -t ,: -r \n –T
The txt file as follows:
However, if you want to bcp by using multiple field terminators the same as the following command, which will still use the last terminator defined by default.
bcp AdventureWorks.HumanResources.Department in C:\myDepartment.txt -c -t , -r \n -t: –T
The txt file as follows:
When multiple field terminators means multiple fields, you use the below comma separated format,
column1,,column2,,,column3
In this occasion, you only separate 3 fields (column1, column2 and column3). In fact, after testing, there will be 6 fields here. That is the significance of a field terminator (comma in this case).
Meanwhile, using BULK INSERT to import the data of the data file into the SQL table, if you specify terminator for BULK import, you can only set multiple characters as one terminator in the BULK INSERT statement.
USE <testdatabase>;
GO
BULK INSERT <your table> FROM ‘<Path>’
WITH (
DATAFILETYPE = ' char/native/ widechar /widenative',
FIELDTERMINATOR = ' field_terminator',
For example, using BULK INSERT to import the data of C:\myDepartment.txt data file into the DepartmentTest table, the field terminator (,:) must be declared in the statement.
In SQL Server Management Studio Query Editor:
BULK INSERT AdventureWorks.HumanResources.DepartmentTest FROM ‘C:\myDepartment.txt’
WITH (
DATAFILETYPE = ‘char',
FIELDTERMINATOR = ‘,:’,
The new table contains like as follows:
We could not declare multiple field terminators (, and :) in the Query statement, as the following format, a duplicate error will occur.
In SQL Server Management Studio Query Editor:
BULK INSERT AdventureWorks.HumanResources.DepartmentTest FROM ‘C:\myDepartment.txt’
WITH (
DATAFILETYPE = ‘char',
FIELDTERMINATOR = ‘,’,
FIELDTERMINATOR = ‘:’
However, if you want to use a data file with fewer or more fields, we can implement via setting extra field length to 0 for fewer fields or omitting or skipping more fields during the bulk copy procedure.
More Information
For more information about filed terminators, you can review the following article.
http://technet.microsoft.com/en-us/library/aa196735(v=sql.80).aspx
http://social.technet.microsoft.com/Forums/en-US/d2fa4b1e-3bd4-4379-bc30-389202a99ae2/multiple-field-terminators-in-bulk-insert-or-bcp?forum=sqlgetsta
http://technet.microsoft.com/en-us/library/ms191485.aspx
http://technet.microsoft.com/en-us/library/aa173858(v=sql.80).aspx
http://technet.microsoft.com/en-us/library/aa173842(v=sql.80).aspx
Applies to
SQL Server 2012
SQL Server 2008R2
SQL Server 2005
SQL Server 2000
Please click to vote if the post helps you. This can be beneficial to other community members reading the thread.

Thanks,
Is this a supported scenario, or does it use unsupported features?
For example, can we call exec [ReportServer].dbo.AddEvent @EventType='TimedSubscription', @EventData='b64ce7ec-d598-45cd-bbc2-ea202e0c129d'
in a supported way?
Thanks! Josh

Bulk Insert into a Table from CSV file

I have a CSV file with 1000 records and i have to insert those records into a table.
I tried for Bulk Insert command and Load data infile command but it throws error.
Am using Oracle 10g Express Edition.
I want to achieve it thru query command and not by plsql procedures
Please send me query syntax for this problem. . . .
Thanks in Advance,
Hariharan ST.

Hi
If you create an external table that points to your csv file you will then be able populate your table from a query.
See: http://www.astral-consultancy.co.uk/cgi-bin/hunbug/doco.cgi?11210
Hope this helps

Ordering bulk inserts

Similar Messages

Maybe you are looking for