Parallel loading

Hello BW Experts,
In the performance tuning document there is a reference to the 'Parallelization when loading data' document, but the link http://service.sap.com/~sapidb/011000358700012379892000E
but this link does not seem to work. do you know what the correct link or if anyone has this document / sap note number please send it to [email protected]
Thanks,
BWer

Hi BWer,
Here are some:
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/64e5eb90-0201-0010-b0b2-c87d7125091a
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/1955ba90-0201-0010-d3aa-8b2a4ef6bbb2
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/2b79ba90-0201-0010-1b9a-fa13a8f38127
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/6f0ad290-0201-0010-fe9d-fda6f439b595
Oss note : 130253
Hope this helps...

Similar Messages

SQL*Loader-971: parallel load option not allowed when loading lob columns

Hi,
I am trying to load a table, which has a VARRAY column, using DIRECT=TRUE and PARALLEL=TRUE through
Sql *Loader 10.2.0.4.0
OS: Sun Solaris 10 SPARC 64-bit,
Database: Oracle Database 10g Enterprise Edition Release 10.2.0.4.0
The following error recieved:
SQL*Loader-971: parallel load option not allowed when loading lob columns
Please help me to resolve..
Thanks and regrds
Anji

user8836881 wrote:
Hi,
I am trying to load a table, which has a VARRAY column, using DIRECT=TRUE and PARALLEL=TRUE through
Sql *Loader 10.2.0.4.0
OS: Sun Solaris 10 SPARC 64-bit,
Database: Oracle Database 10g Enterprise Edition Release 10.2.0.4.0
The following error recieved:
SQL*Loader-971: parallel load option not allowed when loading lob columns
Please help me to resolve..
Thanks and regrds
Anjihttp://tinyurl.com/yhxdhnt

PARALLEL LOAD - SPACE USAGE

제품 : ORACLE SERVER
작성날짜 : 2004-08-13
SCOPE
8.1.6 이상의 Standard Edition 부터 지원이 되는 기능입니다.
Subject: Parallel Load - space usage.
I have put together a short summary of how Parallel Load handles
storage allocation in 7.1 which I am going to send to a customer.
This could also quickly turn into a bulletin or PR entry
(if one doesn't already exist).
I'd appreciate if people could review and give corrections/enhancements.
Here is how storage allocation works with parallel loads.
For each parallel loader session, Oracle
1) creates a temporary segment with the INITIAL and NEXT
from the table that is being loaded, and in the datafile
designated by the FILE option.
2) When a load is complete, the final extent used by that
parallel load is trimmed back to the last block which it
used.
3) Lastly, the extents belonging to the temporary segments
are then converted to extents belonging to the table
being loaded.
This creates a couple of problems:
* If your INITIAL and NEXT are different for the table,
then you will end up with multiple extents sized the
same as INITIAL, which doesn't follow general table
growth patterns.
* Extent sizes are not predictable for the last extent of
each load. This can lead to tablespace fragmentation.
What to do about this?
* Size INITIAL and NEXT to be the same.
This will eliminate the first problem listed.
* Oracle 7.2 will have a feature in to specify the storage
parameters used by the parallel load (bug 187319):
OPTIONS (STORAGE=(MINEXTENTS n
MAXEXTENTS n
INITIAL n K
NEXT n K
PCTINCREASE n))
* Enhancement Request 236552 has been filed to allow the
user to select no trimming of the extents.

Hi Karen,
The number of workprocess (Dialog/Background) could be reduced also by selecting the loading options in InfoPackage.
In InfoPackage Processing tab, there are various loading options,
PSA & Then into data target (Package by Package) - This option uses only one process at a time. One data package is lifted from the Source System by a background process to PSA, and then the same process lifts it from PSA to datatarget.
PSA & data target in parallel (Package by Package) - This option uses two process at a time. One data package is lifted from the Source System by a background process to PSA, and then another process lifts it from PSA to datatarget. The first process same time, gets busy in lifting another package from Source System to PSA.
The first option is good, but takes more time to load data than the second one, so there is a tradeoff.
Also it is good if you use another process in process chain to activate the ODS instead of setting it up on automatic activation.
Please let me know if you need any more help.
Regards,
Pankaj

Write optimized parallel loads ?

Hello BI gurus,
Is parallel loads possible for write optimized ODS ?
if possible what are the limitations of it.
Please let me know your opinion.
Thanks in advance,
BWer

In write optimized DSOs the technical key is generated automatically when loading data, and the standard keys are not really required. If you still have those standard Keys , they are called "semantic keys". Those semantic keys can "bundled" into semantic groups in the DTP.
http://help.sap.com/saphelp_nw2004s/helpdata/en/b6/de1c42128a5733e10000000a155106/frameset.htm
Hope it helps,
Gili

Does ODI support parallel loading

Does ODI support parallel loading? For example, I have a source table and a target table for a mapping. In ODI can I specify a number of threads so the mapping can be executed in parallel module? (besides use hint in SQL)

Hi
You can't execute 2 interfaces in parallel, but you can create a package, and call it asynchroneously from an other one.

Parallel load of InfoProvider master Data

Hi, I have a Master Data object configured as an infoprovider. I am trying to load the object from 2 different sources in parallel in my process chain. This seemed to work in my dev environement, but one of the loads is failing because of master data lock.
There are no duplicate characteristics between the 2 sources so no conflicts exist there. Is it just that I cannot load 2 sources into the master data infoprovider at the same time?
ThankS!

Hi Kenneth,
On the following (parallel) processes lock situations can occur:
- Hierarchy and attribute change runs is locking temporarily parts of a BW system
- Loading master data for the same InfoObject (risk when loading in parallel master data out of several source systems)
( Can be avoided (if wanted) by applying note 421419)
- Rolling up data of the same InfoCube
- Selective deletion on InfoCube and parallel loading
- Activation or deletion of ODS object data and loading it
Best regards,
Eugene

Multiple parallel loads into one table with TABLE LOCK option

Hi everyone:
We have several SSIS packages where each package has the basic design of having one sequence container. Inside each sequence container can be anywhere from 2 - 9 data flow tasks where for each data is selected from a different table but all 2-9 tasks do
an OLEDB fast load (with table lock option checked) into the same single destination table.
The number of rows were pulling from the various sources might be anywhere from 5 up to 100,000.
Right now this doesn't seem to be causing any issues, but I wanted to check to see if this set up (since we're doing a review) could potentially cause problems down the road?
We seem to think that each parallel task will acquire its data as normal, and just won't be able to insert until one of the other parallel tasks completes its fast load. To us, that's no big deal as we're at least able to acquire the data in parallel.
What are everyone's thoughts?
Thanks!

>>>We seem to think that each parallel task will acquire its data as normal, and just won't be able to insert
until one >>>of the other parallel tasks completes its fast load.
That is correct observation.
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence

Parallel loading of flat files into BPC 7.0 NW - Help required

Hi All
Can I load multiple flat files in parallel into BPC using IMPORT process chain.
If yes , How ?
Regards
AK

Please import SAP Note [1507226|https://service.sap.com/sap/support/notes/1507226] titled "FileService should support running DM package simultaneously" which is contained in BPC 7.0 NW SP10 and BPC 7.5 NW SP05.
Best regards,
[Jeffrey Holdeman|http://wiki.sdn.sap.com/wiki/display/profile/Jeffrey+Holdeman]
SAP Labs, LLC
BusinessObjects Division
Americas Applications Regional Implementation Group (RIG)

Parallel loading of Transaction Data

Hi,
We are trying to load transaction data into Inventory Valuation cube (0COPC_C04).
This gets data from multiple datasources. My question is, Can the different infopackages (different infosources) be scheduled in parallel into the same cube .
This is transaction data. We are loading manually in production based on some selection criteria.
Thanks

Hi,
For the first part, yes your understanding is correct. Delete all cube indexes before you start any loads and then recreate them after all loading has been completed. This is to increase load performance. Make sure that you recreate the indexes otherwise you'll have problems in querying.
For the second part, yes, as long as the selections are different and the data loads are from different source systems, you can start another data load in parallel to the one already in progress. This again will help you load faster and will reduce load times..
Cheers,
Kedar

Parallel loading in DSO but with sequential start

i have three source DSO A,B,C which is getting loaded to target D.
i want the three loading A->D,B->D,C->D to happen in parallel but want A->D to start first then B->D and then C->D.
Kindly let me know how to do this in process chain.

Hi,
Add all the three DTPs to a process chain in parallel branches. Use the delay option which is available in the process chain. Right click on the process variant and set the time delay in seconds. In this way you can set the delay by the required time.
Hope this helps.
Regards,
Shyam.

Parallel load

I am trying to test functionality. Can anybody please point me to the right set of documentation?

You just need to modify your Essbase.cfg file by adding two lines, like:DLTHREADSWRITE 2DLTHREADSPREPARE 2The above lines would tell Essbase to use two threads when reading data (prepare) and two threads when writing to disk (write).

Sql Loader - Parallel Direct Path Loading

Hi,
I want to load a few million records into a table. I read from OTN site that we can make use of Parallel Direct Loading Option. As given in the site, I split my source file into two and tried loading the file from two different sqlldr sessions simultaniously. I used two seperate control files for each session. The sessions started first is getting completed successfully. But the session which started second is giving the following error.
Will anyone Pls help to sort out the problem.
Error
"SQL*Loader-951: Error calling once/load initialization
ORA-00604: error occurred at recursive SQL level 1
ORA-00054: resource busy and acquire with NOWAIT specified "
I'm using oracle 9.0.1.1.1.
The Options I tried with both the sessions are :
Direct=true parallel=true.
loading method is append.
Thanks in Advance,
Tom

I've got a similar problem.
I'm running Informatica which runs SQL*Loader via OCI.
With direct & parallel my Indexes give me ORA-26002 - quite understandable.
As its all wrapped up, I cannot say SKIP_INDEX_MAINTENANCE directly.
1) Is there a way to give SQL*Loader some Default-Parameters? I read something about a File shrept.lst but I cannot find a reference in the Oracle documentation.
Something useful down this road?
2) when I omit parallel, what am I losing? I load one file into one partition of one table. I read the SQL*Loader Documentation but didn't get the message of the 'Intrasegment Parallel Loading' Paragraph. Is this using parallel? Or is it just enabling to use parallel (if there will be a second SQL*Loader)? If its all about 'enabling' I can easily omit this - I know there won't be a second SQL*Loader for this partition.
regards,
Kathrin

SQL Loader Parallel Mode

Hi,
     I have similar issue where i have requirement to load 270 million record per day into single table(having No constraints & Indexes), where every CTL file contain 37000 records.
I have machine having 16 CPU and 2 thread per CPU is set.I am using the PARALLEL=TRUE, MULTITHREADING=TRUE,DIRECT=TRUE option in sql loader.
E.g:- OPTIONS( ERRORS=100000, SILENT=all, MULTITHREADING=TRUE, DIRECT=TRUE, PARALLEL=TRUE, SKIP_INDEX_MAINTENANCE=TRUE,streamsize=1048576, readsize=1048576, columnarrayrows=8000 )
Also enable the PARALLEL degree and set the value to 32.
When i am running sqlloader 4 session with above configuration it was taking total 4-5 seconds to load 4 CTL files having 37000 records per file. Well for initial 50 million records sql loader behave normally to load the CTL files in 4-5 seconds, but after 50 million records in table, time taken to process the 4 CTL files was gradually increasing to 40 to 70 seconds and it was still increasing as the number of records more and more in table.
I don't know why sqlloader behave like this after 50 million record in table.
Below is the parallel parameter set on the machine
SQL> show parameter parallel;
NAME                                 TYPE        VALUE
fast_start_parallel_rollback         string      LOW
parallel_adaptive_multi_user         boolean     TRUE
parallel_automatic_tuning            boolean     FALSE
parallel_degree_limit                string      CPU
parallel_degree_policy               string      MANUAL
parallel_execution_message_size      integer     16384
parallel_force_local                 boolean     FALSE
parallel_instance_group              string
parallel_io_cap_enabled              boolean     FALSE
parallel_max_servers                 integer     80
parallel_min_percent                 integer     0
NAME                                 TYPE        VALUE
parallel_min_servers                 integer     0
parallel_min_time_threshold          string      AUTO
parallel_server                      boolean     FALSE
parallel_server_instances            integer     1
parallel_servers_target              integer     32
parallel_threads_per_cpu             integer     2
recovery_parallelism                 integer     0
Kindly repl on the above query

I've got a similar problem.
I'm running Informatica which runs SQL*Loader via OCI.
With direct & parallel my Indexes give me ORA-26002 - quite understandable.
As its all wrapped up, I cannot say SKIP_INDEX_MAINTENANCE directly.
1) Is there a way to give SQL*Loader some Default-Parameters? I read something about a File shrept.lst but I cannot find a reference in the Oracle documentation.
Something useful down this road?
2) when I omit parallel, what am I losing? I load one file into one partition of one table. I read the SQL*Loader Documentation but didn't get the message of the 'Intrasegment Parallel Loading' Paragraph. Is this using parallel? Or is it just enabling to use parallel (if there will be a second SQL*Loader)? If its all about 'enabling' I can easily omit this - I know there won't be a second SQL*Loader for this partition.
regards,
Kathrin

Need to generate multiple error files with rule file names during parallel data load

Hi,
Is there a way that MAXL could generate multiple error files during parallel data load?
import database AsoSamp.Sample data
connect as TBC identified by 'password'
using multiple rules_file 'rule1' , 'rule2'
to load_buffer_block starting with buffer_id 100
on error write to "error.txt";
I want to get error files as this - rule1.err, rule2.err (Error files with rule file name included). Is this possible in MAXL?
I even faced a situation , If i hard code the error file name like above, its giving me error file names as error1.err and error2.err. Is there any solution for this?
Thanks,
DS

Are you saying that if you specify the error file as "error.txt" Essbase actually produces multiple error files and appends a number?
Tim.
Yes its appending the way i said.
Out of interest, though - why do you want to do this? The load rules must be set up to select different 'chunks' of input data; is it impossible to tell which rule an error record came from if they are all in the same file?
I have like 6 - 7 rule files using which the data will be pulled from SQL and loaded into Essbase. I dont say its impossible to track the error record.
Regardless, the only way I can think of to have total control of the error file name is to use the 'manual' parallel load approach. Set up a script to call multiple instances of MaxL, each performing a single load to a different buffer. Then commit them all together. This gives you most of the parallel load benefit, albeit with more complex scripting.
Even i had the same thought of calling multiple instances of a Maxl using a shell script. Could you please elaborate on this process? What sort of complexity is involved in this approach.? Did anyone tried it before?
Thanks,
DS

Performance issue - Loading and Calculating

Hi, I am having 5 GB of data. It is taking 1 hr to load and 30 min to calculate. I did the following things to improve the performance. 1) Sort the data and loading them in the order of largest sparse first, followed by smallest and dense 2) Enabled parallel load, gave 6 threads for prepare and 4 for writing. 3) Increased data file cache as 400MB and data cache as 50MB, then index cache as 100MB. 4) Calculation only for 4 dimensions, out of 9. In that 2 are dense and 2 are sparse. 5) Calculation with parallel calculation having 3 threads and CALCTASKDIMS as 2. But i am not getting any improvements. While doing the calculation i got following message in the logs. I feel that CALCTASKDIM is not working [Fri Jan 6 22:01:54 2006]Local/tcm2006/tcm2006/biraprd/Info(1012679) Calculation task schedule [2870,173,33,10,4,1] [Fri Jan 6 22:01:54 2006]Local/tcm2006/tcm2006/biraprd/Info(1012680) Parallelizing using [1] task dimensions. Usage of Calculator cache caused reduction in task dimensions [Fri Jan 6 22:33:54 2006]Local/tcm2006/tcm2006/biraprd/Info(1012681) Empty tasks [2434,115,24,10,2,0] Can any one help me what the above log message is telling and what are the other things to be done to imrpove the performance. Regards prsan 

its not the problem with ur calc task dim. Calculation task schedule [2870,173,33,10,4,1] indicates that ur parell calc can start with 2870calculations in parallel, after which 173 can be performed inparallel then 33 ,10,4 and 1. Empty tasks [2434,115,24,10,2,0] means these manytasks dont need any calculation- either because there is no data orthey are marked clean due to intelligent calc. the problem lies with your cal cache setting. try increaing thecal cache settings in ur cfg file and use calcache high setting inyour calc. hope this works

Parallel loading

Similar Messages

Maybe you are looking for