Bulk loads

I've read that Drop and recreate an index before bulk loads.What does it mean bulk loads here.
Thanks in advance

bulk loads means, you have lot of inserts to your table/s. While doing so, if there are any indexes on the table, it would be better to make drop and recreate those indexes. But, it is not highly recommend. You can make your index unusable while bulk loading.

Similar Messages

SSRS 2005 report: Cannot bulk load Operating system error code 5(Access is denied.)

I built a SSRS 2005 report, which calls a stored proc on SQL Server 2005. The proc contains following code:
CREATE TABLE #promo (promo VARCHAR(1000))
BULK
INSERT #promo
FROM '\\aseposretail\c$\nz\promo_names.txt'
WITH
--FIELDTERMINATOR = '',
ROWTERMINATOR = '\n'
SELECT * from #promo
It's ok when I manually execute the proc in SSMS.
When I try to run the report from BIDS I got following error:
*Cannot bulk load because the file "\aseposretail\c$\nz\promo_names.txt" could not be opened. Operating system error code 5(Access is denied.).*
Note: I have gooled a bit and see many questions on this but they are not relevant because I CAN run the code no problem in SSMS. It's the SSRS having the issue. I know little about the security of SSRS.

I'm having the same type of issue. I can bulk load the same file into the same table on the same server using the same login on one workstation, but not on another. I get this error:
Msg 4861, Level 16, State 1, Line 1
Cannot bulk load because the file "\\xxx\abc.txt" could not be opened. Operating system error code 5(Access is denied.).
I've checked SQL client versions and they are the same, I've also set the client connection to TCP/IP only in the SQL Server Configuration Manager. Still this one workstation is getting the error. Since the same login is being used on both workstations and it works on one but not the other, the issue is not a permissions issue. I can also have another user login into the bad workstation and have the bulk load fail, but when they log into their regular workstation it works fine. Any ideas on what the client configuration issue is? These are the version numbers for Management Studio:
Microsoft SQL Server Management Studio 9.00.3042.00
Microsoft Analysis Services Client Tools 2005.090.3042.00
Microsoft Data Access Components (MDAC) 2000.085.1132.00 (xpsp.080413-0852)
Microsoft MSXML 2.6 3.0 5.0 6.0
Microsoft Internet Explorer 6.0.2900.5512
Microsoft .NET Framework 2.0.50727.1433
Operating System 5.1.2600
Thanks,
MWise

How to UPDATE a big table in Oracle via Bulk Load

Hi all,
in a datastore target as Oracle 11g, I have a big table having 300milions of record; the structure is One integer key + 10 columns attributes .
In IQ Source i have the same table with the same size ; the structure is One integer key + 1 column attributes .
What i need to do is to UPDATE that single field in Oracle from the values stored in IQ .
Any idea on how to organize efficiently the dataflow and the target writing mode ? bulk load ? api ?
thank you
Maurizio

Hi,
You cannot do bulk load when you need to UPDATE a field. Because all a bulk load does is add records to your table.
Since you have to UPDATE a field, i would suggest to go for SCD with
source > TC > MO > KG >target
Arun

Error while running bulk load utility for account data with CSV file

Hi All,
I'm trying to run the bulk load utility for account data using CSV but i'm getting following error...
ERROR ==> The number of CSV files provided as input does not match with the number of account tables.
Thanks in advance........

Please check your child table.
http://docs.oracle.com/cd/E28389_01/doc.1111/e14309/bulkload.htm#CHDCGGDA
-kuldeep

Bulk Load option doesn't work

Hi Experts,
I am trying to load data to HFM using Bulk load option but it doesnt work. When I Change the option to SQL insert, the loading is successful. The logs say that the temp file is missing. But when I go to the lspecified location , I see the control file and the tmp file. What am I missing to have bulk load working?Here's the log entry.
2009-08-19-18:48:29
User ID...........     kannan
Location..........     KTEST
Source File.......     \\Hyuisprd\Applications\FDM\CRHDATALD1\Inbox\OMG\HFM July2009.txt
Processing Codes:
BLANK............. Line is blank or empty.
ESD............... Excluded String Detected, SKIP Field value was found.
NN................ Non-Numeric, Amount field contains non numeric characters.
RFM............... Required Field Missing.
TC................ Type Conversion, Amount field could be converted to a number.
ZP................ Zero Suppress, Amount field contains a 0 value and zero suppress is ON.
Create Output File Start: [2009-08-19-18:48:29]
[TC] - [Amount=NN]     Batch Month File Created: 07/2009
[TC] - [Amount=NN]     Date File Created: 8/6/2009
[TC] - [Amount=NN]     Time File Created: 08:19:06
[Blank] -
Excluded Record Count.............. 3
Blank Record Count................. 1
Total Records Bypassed............. 4
Valid Records...................... 106093
Total Records Processed............ 106097
Begin Oracle (SQL-Loader) Process (106093): [2009-08-19-18:48:41]
[RDMS Bulk Load Error Begin]
     Message:      (53) - File not found
     See Bulk Load File:      C:\DOCUME~1\fdmuser\LOCALS~1\Temp\tWkannan30327607466.tmp
[RDMS Bulk Load Error End]
Thanks
Kannan.

Hi Experts,
I am facing the data import error while importing data from .csv file to FDM-HFM application.
2011-08-29 16:19:56
User ID...........     admin
Location..........     ALBA
Source File.......     C:\u10\epm\DEV\epm_home\EPMSystem11R1\products\FinancialDataQuality\FDMApplication\BMHCFDMHFM\Inbox\ALBA\BMHC_Alba_Dec_2011.csv
Processing Codes:
BLANK............. Line is blank or empty.
ESD............... Excluded String Detected, SKIP Field value was found.
NN................ Non-Numeric, Amount field contains non numeric characters.
RFM............... Required Field Missing.
TC................ Type Conversion, Amount field could be converted to a number.
ZP................ Zero Suppress, Amount field contains a 0 value and zero suppress is ON.
Create Output File Start: [2011-08-29 16:19:56]
[ESD] ( ) Inter Co,Cash and bank balances,A113000,Actual,Alba,Dec,2011,MOF,MOF,,YTD,Input_Default,[NONE],[NONE],[NONE],1
[ESD] ( ) Inter Co,"Trade receivable, prepayments and other assets",HFM128101,Actual,Alba,Dec,2011,MOF,MOF,,YTD,Input_Default,[NONE],[NONE],[NONE],35
[ESD] ( ) Inter Co,Inventories ,HFM170003,Actual,Alba,Dec,2011,MOF,MOF,,YTD,Input_Default,[NONE],[NONE],[NONE],69
[ESD] ( ) Inter Co,Financial assets carried at fair value through P&L,HFM241001,Actual,Alba,Dec,2011,MOF,MOF,,YTD,Input_Default,[NONE],[NONE],[NONE],103
[Blank] -
Excluded Record Count..............4
Blank Record Count.................1
Total Records Bypassed.............5
Valid Records......................0
Total Records Processed............5
Begin SQL Insert Load Process (0): [2011-08-29 16:19:56]
Processing Complete... [2011-08-29 16:19:56]
Please help me solve the issue.
Regards,
Sudhir Sinha

Bulk Insert Task Cannot bulk load because the file could not be opened.operating system error error code 3(The system cannot find the path specified.)

Following error i am getting after i chnaged the Path in Config File from
\\vs01\d$\\Deployment\Files\temp.txt
to
C:\Deployment\Files\temp.txt
[Bulk Insert Task] Error: An error occurred with the following error message: "Cannot bulk load because the file "C:\Deployment\Files\temp.txt" could not be opened. Operating system error code 3(The system cannot find the path specified.).".

I think i know whats going on. The Bulk Insert task runs by executing sql command (bulk insert) internally from the target sql server to load the file. This means that the SQL Server Agent of the target sql server should have permissions on the file you trying to load. This also means that you need to use UNC path instead to specify the file path (if the target server in on different machine)
Also from BOL (see section Usage Considerations - last bullet point)
http://msdn.microsoft.com/en-us/library/ms141239.aspx
* Only members of the sysadmin fixed server role can run a package that contains a Bulk Insert task.
Make sure you take care of this as well.
HTH
~Mukti
Mukti

Issue with Bulk Load Post Process Scheduled Task

Hello,
I successfully loaded users in OIM using the bulk load utility. I also have LDAP sync ON. The documentation says to run the Bulk Load Post Process scheduled task to push the loaded users in OIM into LDAP.
This works if we run the Bulk Load Post Process Scheduled Task right away after the run the bulk load.
If some time had passed and we go back to run the Bulk Load Post Process Scheduled Task, some of the users loaded through the bulk load utility are not created in our LDAP system. This created an off-sync situation between OIM and our LDAP.
I tried to use the usr_key as a parameter to the Bulk Load Post Process Scheduled Task without success.
Is there a way to force the re-evaluation of these users so they would get created in LDAP?
Thanks
Khanh

The scheduled task carries out post-processing activities on the users imported through the bulk load utility.

Issue with Bulk Load Post Process

Hi,
I ran bulk load command line utility to create users in OIM. I had 5 records in my csv file. Out of which 2 users were successfully created in OIM and for rest i got exception because users already existed. After that if i run bulk load post process for LDAP sync and generate the password and send notification. It is not working even for successfully created users. Ideally it should sync successfully created users. However if there is no exception i during bulk load command line utility then LDAP sync work fine through bulk load post process.Any idea how to resolve this issue and sync the user in OID which were successfully created. Urgent help would be appreciated.

The scheduled task carries out post-processing activities on the users imported through the bulk load utility.

Bulk loading BLOBs using PL/SQL - is it possible?

Hi -
Does anyone have a good reference article or example of how I can bulk load BLOBs (videos, images, audio, office docs/pdf) into the database using PL/SQL?
Every example I've ever seen in PL/SQL for loading BLOBs does a commit; after each file loaded ... which doesn't seem very scalable.
Can we pass in an array of BLOBs from the application, into PL/SQL and loop through that array and then issue a commit after the loop terminates?
Any advice or help is appreciated. Thanks
LJ

It is easy enough to modify the example to commit every N files. If you are loading large amounts of media, I think that you will find that the time to load the media is far greater than the time spent in SQL statements doing inserts or retrieves. Thus, I would not expect to see any significant benefit to changing the example to use PL/SQL collection types in order to do bulk row operations.
If your goal is high performance bulk load of binary content then I would suggest that you look to use Sqlldr. A PL/SQL program loading from BFILEs is limited to loading files that are accessible from the database server file system. Sqlldr can do this but it can also load data from a remote client. Sqlldr has parameters to control batching of operations.
See section 7.3 of the Oracle Multimedia DICOM Developer's Guide for the example Loading DICOM Content Using the SQL*Loader Utility. You will need to adapt this example to the other Multimedia objects (ORDImage, ORDAudio .. etc) but the basic concepts are the same.
Once the binary content is loaded into the database, you will need a to write a program to loop over the new content and initialize the Multimedia objects (extract attributes). The example in 7.3 contains a sample program that does this for the ORDDicom object.

Bulk load option Data Services Oracle

Hello,
I'm trying to use the option "Bulk load" but it doesn't work.
Error message : "DBS-070301Oracle <DB_INFO>> error message for operation <OCIDirPathPrepare>: <ORA-00942: table or view does not exist"
Is it necessary to affect dba grant for the oracle user that is declared in the Datastore?
The version is : Data Services XI 3.1
The database is : Oracle9i Enterprise Edition Release 9.2.0.7.0
Thx for your answer

this also happens in case there is a mismatch in Oracle Client and Server Version, like Oracle 9i client and 10g server
what is you Oracle Server version ?
you can also check similar post in DI BOB forum
http://www.forumtopics.com/busobj/viewtopic.php?t=122242

Notifications are not being sent when Bulk Load is done

Hi All,
I have OIM 11g setup on my machine. I use the bulk load utility for loading the user data. Now in my OIM setup, the notifications are being sent for all stuff like Reset Password. New account creation and other. However when I bulk load the users, notifications are not sent to their mail ids. I am running the scheduled job "Bulk load Post Process" which is necessary so that the users are synced to the LDAP repository. I have the LDAP Sync option checked and also the Notifications option set to yes in this scheduled job. Though the users are loaded successfully and are synced properly, the notifications are not sent. Can some one please guide me as to what could be the problem here?
Thanks,
$id

The code is probably only called in the Event method of the event handler that sends the notification. You can check the mds files and find the notification you are looking for and then use a code decompiler to find the class that is called. You can then use this code as a sample, or write your own notification code and create an event handler that runs in the BulkEvent.
And on another note there is also this System Configuration Variable: Recon.SEND_NOTIFICATION which is set to FALSE by default.
-Kevin

Bulk loading of Customer data into Application

Hi Guys,
I am going on with the development of Teleservice module on a new instance.
Now i need to migrate the data on the old instance to the new instance.
Please let me know if i have to use only APIs to create the customer into Apps or whether i can bulk load into the seeded tables directly.
This has to include even Service Requests data also.
Please let me know if there is any integration violation if we go with bulk loading of data directly.

You donot need to develop a code for loading customer data anymore. Oracle has provided the BUlk IMport functionality in 11.5.8 for importing the customer infromation (using Oracle Customers Online/Oracle Data Libraian modules). If you would like to create accounts in addition to customer parties, you will have to use TCA V2 apis or customer interface program. For migrating the service requests, i guess the only option is to use APIs. HTH, Venit

PL/SQL Bulk Loading

Hello,
I have one question regarding bulk loading. I did lot of bulk loading.
But my requirement is to call function which will do some DML operation and give ref key so that i can insert to fact table.
Because i can't use DML function in select statement. (which will give error). otherway is using autonomous transaction. which i tried working but performance is very slow.
How to call this function inside bulk loading process.
Help !!
xx_f is function which is using autonmous transction,
See my sample code
declare
cursor c1 is select a,b,c from xx;
type l_a is table of xx.a%type;
type l_b is table of xx.b%type;
type l_c is table of xx.c%type;
v_a l_a;
v_b l_b;
v_c l_c;
begin
open c1;
loop
fetch c1 bulk collect into v_a,v_b,v_c limit 1000;
exit when c1%notfound;
begin
forall i in 1..v_a.count
insert into xxyy
(a,b,c) values (xx_f(v_a(i),xx_f(v_b(i),xx_f(v_c(i));
commit;
end bulkload;
end loop;
close c1;
end;
I just want to call xx_f function without autonoumous transaction.
but with bulk loading. Please let me if you need more details
Thanks
yreddyr

Can you show the code for xx_f? Does it do DML, or just transformations on the columns?
Depending on what it does, an alternative could be something like:
DECLARE
   CURSOR c1 IS
      SELECT xx_f(a), xx_f(b), xx_f(c) FROM xx;
   TYPE l_a IS TABLE OF whatever xx_f returns;
   TYPE l_b IS TABLE OF whatever xx_f returns;
   TYPE l_c IS TABLE OF whatever xx_f returns;
   v_a l_a;
   v_b l_b;
   v_c l_c;
BEGIN
   OPEN c1;
   LOOP
      FETCH c1 BULK COLLECT INTO v_a, v_b, v_c LIMIT 1000;
      BEGIN
         FORALL i IN 1..v_a.COUNT
            INSERT INTO xxyy (a, b, c)
            VALUES (v_a(i), v_b(i), v_c(i));
      END;
      EXIT WHEN c1%NOTFOUND;
   END LOOP;
   CLOSE c1;
END;John

Using API to run Catalog Bulk Load - Items & Price Lists concurrent prog

Hi everyone. I want to be able to run the concurrent program "Catalog Bulk Load - Items & Price Lists" for iProcurement. I have been able to run concurrent programs in the past using the fnd_request.submit_request API. But I seem to be having problems with the item loading concurrent program. for one thing, the program is stuck on phase code P (pending) status.
When I run the same concurrent program using the iProcurement Administration page it runs ok.
Has anyone been able to run this program through the backend? If so, any help is appreciated.
Thanks

Hello S.P,
Basically this is what I am trying to achieve.
1. Create a staging table. The columns available for it are category_name, item_number, item_description, supplier, supplier_site, price, uom and currency.
So basically the user can load item details into the database from an excel sheet.
2. use the utl_file api, create an xml file called item_load.xml using the data in the staging table. this will create the xml file used to load items in iprocurement and save it in the database directory /var/tmp/iprocurement This part works great.
3. use the api fnd_request.submit_request to submit the concurrent program 'Catalog Bulk Load - Items & Price Lists'. This is where I am stuck. The process simply says pending or comes up with an error saying:
oracle.apps.fnd.cp.request.FileAccessException: File /var/tmp/iprocurement is not accessable from node/machine moon1.oando-plc.com.
I'm wondering if anyone has used my approach to load items before and if so, have they been successful?
Thank you

CBO madness after bulk loading

This is an extension of my other recent posts, but I felt it deserved it's own space.
I have a table of telephone call records, one row for each telephone call made or received by a customer. Our production table has a 10-field PK that I want to destroy. In my development version, the PK for this table is a compound key on LOC, CUST_NO, YEAR, MONTH, and SEQ_NO. LOC is a char(3), the rest are numbers.
After a bulk load into a new partition of this table, a query with these 5 fields in the where clause chooses a second index. That second index includes LOC, YEAR, MONTH, and two other fields not in the PK nor in the query. The production instance does the same thing, and I was certain that having the 5-field PK would be the magic bullet.
Oracle SQL Developer's autotrace shows a "Filter Predicates" on CUST_NO and SEQ_NO, and then the indexed range scan on the other 3 fields in the second index. Still noteworthy is that query on just LOC, CUST_NO, YEAR and MONTH does use the PK.
Here are the steps I've taken to test this:
1. Truncate the partition in question
2. Drop old PK constraint/index
3. Create new PK constraint/index
4. Gather table stats with cascade=>TRUE
5. Bulk load data (in this case, 1.96 million rows) into empty partition
6. autotrace select query
7. Write to dizwell in tears
This table also has two other partitions for past two cycles, each with around 30 million row.
Yes, gathering table stats again makes things behave as expected, but that takes a fair bit of time. For the meantime we've put an index hint in the application query that was suffering the most.

First, the CBO doesn't actually choose a full table
scan, it chooses to use a second index.Depending on the query, of course. If the CBO thinks a partition is empty, I would suspect that it would find it most efficient to scan the smallest index, and the second index, with fewer columns, would be expected to be smaller. If it thinks they are equally costly, I believe it will use the one that was created first, though I wouldn't want to depend on that sort of failure.
I've lowered the sample percentage to 10% and set
CASCADE to FALSE and it still take 45 minutes in
production. The staging table was something I was
considering.
Are statistics included in partition exchange? I've
asked that question before but never saw an answer.Yes, partition-level statistics will be included. Table-level statistics will be automatically adjusted. From the SQL Reference
http://download-east.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_3001.htm#i2131250
"All statistics of the table and partition are exchanged, including table, column, index statistics, and histograms. Oracle Database recalculates the aggregate statistics of the table receiving the new partition."
You could also just explicitly set table-level statistics, assuming you don't need too many histograms, possibly gathering statistics for real later on.
Justin

How to improve performance for Azure Table Storage bulk loads

Hello all,
Would appreciate your help as we are facing a challenge.
We are tried to bulk load Azure table storage. We have a file that contains nearly 2 million rows.
We would need to reach a point where we could bulk load 100000-150000 entries per minute. Currently, it takes more than 10 hours to process the file..
We have tried Parallel.Foreach but it doesn't help. Today I discovered Partitioning in PLINQ. Would that be the way to go??
Any ideas? I have spent nearly two days in trying to optimize it using PLINQ, but still I am not sure what is the best thing to do.
Kindly, note that we shouldn't be using SQL/Azure SQL for this.
I would really appreciate your help.
Thanks

I'd think you're just pooling the parallel connections to Azure, if you do it on one system. You'd also have a bottleneck of round trip time from you, through the internet to Azure and back again.
You could speed it up by moving the data file to the cloud and process it with a Cloud worker role. That way you'd be in the datacenter (which is a much faster, more optimized network.)
Or, if that's not fast enough - if you can split the data so multiple WorkerRoles could each process part of the file, you can use the VM's scale to put enough machines to it that it gets done quickly.
Darin R.

Bulk loads

Similar Messages

Maybe you are looking for