ODS Partitioning ?

Hi -
I am looking for guidelines regarding ODS/DSO Partitioning.
We are dealing with the LO extractor which brings billing conditions into BW/BI (We are running BI 7). We expect very large volumes of data from this extractor.
We plan to implement a design the follows EDW standards. First level write-optimized DSO, second level standard ODS, and finally into a cube. Most of our transformation will occur between the first and second level DSO - including any filtering.
As we are working out the details of our design, we are trying to determine if we should utilize logical partioning for the first and second level DSO Objects.
Are there any guidelines around maximum ODS Size? Does anyone have reccomendations regarding ODS Partitioning?
I have searched the forum and found plenty of information on logical partitioning of Cubes but have found nothing regarding ODS objects.
Points will be awarded.... Thanks for your help.

Wond -
Thanks, your answer is helpful. I guess in terms of loading from the ODS - I will be loading a delta to the cube so the actual load will come from the change log - it will not be by selection. The change log wont be large so, except for the initial load - it should manageable.
Regarding activiation. When activation occurs, determining if an existing record exists would be done via the semantic key... correct? There would be an index on the key of the ODS - so the search would not be a sequential read through the entire table - it would be and index search. So, would activation really suffer as you suggested?
Lets say I decide to partition it - what number of records or size per ODS would I want to achieve... again, I would look for a guideline from SAP or others from experience to say... If you are going to go throug the work to partition your ODS you want to keep them under X records or X Size.
Any ideas?
Some points awarded.... some points remain!! Thanks.

Similar Messages

ODS partition

Hello,
i just want to know that is it possible to make the partition of the dso.
if yes then how to do this, and what is happening in database level.

Hi Viral,
You can use Multi dimensional clustering in DSO which is similar to partitioning in Cubes.
To do this Double click on the DSO in the modeling view of the administration workbench.
Go to Extras'---> DB Performance -
> Clustering.
Multidimensional clustering (MDC) physically organizes a table in blocks. Each block only contains data records with the same values in the MDC dimensions.
Multi-dimensional clustering can significantly improve the performance of database queries. If the queries contain restrictions on MDC dimensions, only the blocks of the table that contain the corresponding dimension values must be read. For the data access block indexes are used, which are considerably smaller than conventional row indexes and can therefore be searched more quickly.
Best regards,
Sunmit.

Partition of ODS

hi
can u explain how to partition ODS and steps to do that.
regards,
manju

Hi,
ODS partitioning is not possible.
But can partition the Active tables
Check this for more,
http://help.sap.com/saphelp_nw04/helpdata/en/e6/bb01580c9411d5b2df0050da4c74dc/frameset.htm
As well this thread also,
Partioning of ODS object
Hope this helps.........

Partition ods

Hi,
We have 4 ODS (one for month), and then a multiprovider to join them. The records of each ods are around 200.000.000, we are thinking about partitioning the active table. When the month changes, we want to delete one ODS (month - 3), and then we are going to load the new month in this ODS.
How can we partition the ods from BW?
The partitions must be recreated when the new month comes, this can be automatic?
Thanks
Victoria

Partitioning could be done only before the fill of data to ODS
After filling data repartitioning is possible only in version BI 7.x ( repartitioning is partitioning after filling data in ODS)
If you want to partition, you can create 4 partition based on months in the active table after emptying out the ODS.Partition cannot be automatic instead you can create partition based on month for 4 months while creating partition. So ODS will store data accordingly in the active table.

0FIAR_C03 load

Hi All,
The cube 0FIAR_C03 gets data from DSO 0FIAR_O03
Three extractors(0FI_AR_4,0FI_AR_3,0FIAR_O03) feeds data into the DSO 0FIAR_O03
We have a huge volume of data coming into DSO.
If we build three more dsos on top of it for logical data partitioning, will there be nay pblem.
Can anyone of one explain any problem in getting data from the three datatsources.
Thanks in advance

Three extractors(0FI_AR_4,0FI_AR_3,0FIAR_O03) feeds data into the DSO 0FIAR_O03.
0FIAR_O03 feeds 0FIAR_C03 and 0FIAR_O03 is fed by 0FI_AR_4,0FI_AR_3 .
Why are you using both 0FI_AR_3 & 0FI_AR_4 ?
This DataSource replaces DataSource 0FI_AR_3 (Accounts Receivable: Line Items) and uses the same extraction structure. The migration of DataSource 0FI_AR_3 to new DataSource 0FI_AR_4 is described in OSS note 410797.

Source http://help.sap.com/saphelp_nw2004s/helpdata/en/bc/02f93abb013b0de10000000a114084/content.htm
It should be only datasource instead of 3 feeding 0FIAR_O03.
For ODS Performance Optimization you may look at
http://help.sap.com/saphelp_nw04/helpdata/en/e6/bb01580c9411d5b2df0050da4c74dc/content.htm
Partioning of ODS object
ODS Partitioning ?

How to create DB partitioning in active data tables for ods?

hi all,
Can anyone let me know how to create DB partitioning in active data tables for ods. if any docs pls share with me at my email id : [email protected]
regds
haritha

Haritha,
The following steps will briefly explain you to improve the performance in terms of DB partitioning as well as loading. Please find the same,
transaction RSCUSTA2,
oss note 120253 565725 670208
and remove 'bex reporting' setting in ods if that ods not used for reporting.
hope this helps.
565725
Symptom
This note contains recommendations for improving the load performance of ODS objects in Business Information Warehouse Release 3.0B and 3.1 Content.
Other terms
Business Information Warehouse, ODS object, BW, RSCUSTA2, RSADMINA
Solution
To obtain a good load performance for ODS objects, we recommend that you note the following:
1. Activating data in the ODS object
In the Implementation Guide in the BW Customizing, you can implement different settings under Business Information Warehouse -> General BW settings -> Settings for the ODS object that will improve performance when you activate data in the ODS object.
1. Creating SIDs
The creation of SIDs is time-consuming and may be avoided in the following cases:
a) You should not set the indicator for BEx Reporting if you are only using the ODS object as a data store.Otherwise, SIDs are created for all new characteristic values by setting this indicator.
b) If you are using line items (for example, document number, time stamp and so on) as characteristics in the ODS object, you should mark these as 'Attribute only' in the characteristics maintenance.
SIDs are created at the same time if parallel activation is activated (see above).They are then created using the same number of parallel processes as those set for the activation. However:if you specify a server group or a special server in the Customizing, these specifications only apply to activation and not the creation of SIDs.The creation of SIDs runs on the application server on which the batch job is also running.
1. DB partitioning on the table for active data (technical name:
The process of deleting data from the ODS object may be accelerated by partitioning on the database level.Select the characteristic after which you want deletion to occur as a partitioning criterion.For more details on partitioning database tables, see the database documentation (DBMS CD).Partitioning is supported with the following databases:Oracle, DB2/390, Informix.
1. Indexing
Selection criteria should be used for queries on ODS objects.The existing primary index is used if the key fields are specified.As a result, the characteristic that is accessed more frequently should be left justified.If the key fields are only partially specified in the selection criteria (recognizable in the SQL trace), the query runtime may be optimized by creating additional indexes.You can create these secondary indexes in the ODS object maintenance.
1. Loading unique data records
If you only load unique data records (that is, data records with a one-time key combination) into the ODS object, the load performance will improve if you set the 'Unique data record' indicator in the ODS object maintenance.
Hope this helps..
****Assign Points****
Thanks,
Gattu

Logical partitioning of an ODS - when / what size e.g. 100 Mio records?

Hi Folks,
we got an ODS/DSO with about 80 fields and we are currently planning futher rollouts which will lead to an overall volume in the DSO about 100 Mio records.
I wonder if this volume for a DSO is still fine to do reporting and loading / activation with good performance?
Or is there a "rule of thumb" to have let's say only 50 Mio reocrds in a ODS and then go for a logical partion approach in larger scenario
50 Mio -> Region EMEA, APJ
50 Mio -> Region AMS
Thanks for you inputs in advance,
Axel

100 Mo records is not such a big DSO. You should not encounter problems for loading and/or activating your DSO.
You may encounter performance problems with reporting functionnalities but it will depend on the reporting you do. And anyway, if you really want to do reporting on this data, why don't you put this level of detail in the cube (logically partitionned or not).
You can logically partitionned any kind of infoProvider, but I've never seen this for DSO (I'd rather partition the upper levels and have a DSO with all data).
Regards,
Fred

Logical vs. Physical Partitioning

Hi,
In a discussion of logical partition the author pointed out that
if a query that needs data from all 5 years would then automatically (you can control this) be split into 5 separate queries, one against each cube, running at the same time. The system automatically merges the results from the 5 queries into a single result set.
1. Can you direct me on how to control this as the author indicated?
2. Also, the author noted that
Physical Partitioning - I believe only Oracle and Informix currently support Range partitioning.
a) Does it mean that BW does not use physical partitioning?
b) Where do we indicate physical or logical partitioning as an option in BW?
c) Or, when we talk about dimension table, etc. is there always an underlining database such as Oravle, Infomix, etc in BW? If so, what does BW use?
3. For physical partitions, I read that the cube needs to be empty before it can be partitioned. What about logical partition?
4. Finally, what are the underlying criteria to decide on logical or physical partitioning or both
Thanks

. Can you direct me on how to control this as the author indicated?
You make this setting RSRT.
2. Also, the author noted that
Physical Partitioning - I believe only Oracle and Informix currently support Range partitioning.
DB2 also support partiioning. Also the current relese of SQL server support partitioning.
b) Where do we indicate physical or logical partitioning as an option in BW?
Physical parittions are set up in the cube change option. When you are in the cube change mode, on the themenu, chose extras - performance / DB parameters - parttitios.
Now a screen will pop giving the time characteristics in the cube and choose the characteristic (s) that you wish to chose and confirm the entries - then you will get another small pop up where you set the no of parittions.
Also, pl note that you can partition only on fiscalyear and fiscal period and not on other time characteritsitcs.
Logical partitions: Logical paritions are nothinb but spilting the cube into numerous cubes of smaller sizes. You combine all these cubes by means of multi provider. For example, if you have 1000 cost centers , you may want to split into cubes based on the cost cenree numbers and combine them into a multi provider.
No more setting is required.
c) Or, when we talk about dimension table, etc. is there always an underlining database such as Oravle, Infomix, etc in BW? If so, what does BW use?
Dimension tables / fact tables/ ODS tables /master data tables are all database tables. Which ever database you use and when yu activate these objects, the tables are created in the underlying database.
3. For physical partitions, I read that the cube needs to be empty before it can be partitioned. What about logical partition?
Logical partiton can be done any time.
4. Finally, what are the underlying criteria to decide on logical or physical partitioning or both
The underlying criteria is facotrs such as :
(a) the no of years of history tou wish to view in reports.
(b) te no ofyears you will hold the data in BW before archiving.
(c) othe performance matters related tro sizing.
Ravi Thothadri

ODS to CUBE loading - taking too much time

Hi Experts,
I am loading data from R/3(4.7) to BW (3.5).
I am loading with option --> PSA and then Data Target (ODS ).
I have a selection criteria in Infopackage while loading from standard Datasource to ODS.
It takes me 20 mins to load 300K records.
But, from ODS to Infocube ( update method: Data Target Only), it is taking 8 hours.
The data packet size in Infopackage is 20,000 ( same for ODS and Infocube).
I also tried changing the data packet size, tried with full load , load with initialization,..
I tried scheduling it as a background job too.
I do not have any selection criteria in the infopackage from ODS to Cube.
Please let me know how can I decrease this loading time from ODS to Infocube.

Hi,
To improve the data load performance
1. If they are full loads then try to see if you make them delta loads.
2. Check if there are complex routines/transformations being performed in any layer. In that case see if you can optimize those codes with the help of an abaper.
3. Ensure that you are following the standard procedures in the chain like deleting Indices/secondary Indices before loading etc.
4. Check whether the system processes are free when this load is running
5. try making the load as parallel as possible if the load is happening serially. Remove PSA if not needed.
6. When the load is not getiing processed due to huge volume of data, or more number of records per data packet, Please try the below option.
1) Reduce the IDOC size to 8000 and number of data packets per IDOC as 10. This can be done in info package settings.
2) Run the load only to PSA.
3) Once the load is succesfull , then push the data to targets.
In this way you can overcome this issue.
Ensure the data packet sizing and also the number range buffering, PSA Partition size, upload sequence i.e, always load master data first, perform change run and then transaction data loads.
Check this doc on BW data load perfomance optimization
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/1955ba90-0201-0010-d3aa-8b2a4ef6bbb2
BI Performance Tuning
FAQ - The Future of SAP NetWeaver Business Intelligence in the Light of the NetWeaver BI&Business Objects Roadmap
Thanks,
JituK

Logical partitioning, pass-through layer, query pruning

Hi,
I am dealing with performance guidelines for BW and encountered few interesting topics, which however I do not fully undestand.
1. Mainetance of logical partitioning.
Let's assume logical partitioning is performed on year. Does it mean that every year or so it is necessary to create additional cube/transformation and modify multiprovider? Is there any automatic procedure by SAP that supports creation of new objects, or it is fully manual?
2.Pass- though layer.
There are very few information about this basic concept. Anyway:
- is pass through DSO write optimized one? Does it store only one loading - last one? Is it deleted after lading is sucessfully finished (or before new load starts)? And - does this deletion do not destroy delta mechanism? Is the DSO replacing PSAfunctionally (i.e. PSA can be deleted every load as well)?
3. Query pruning
Does this happen automatically on DB level, or additional developments with exits variables, steering tables and FMs is required?
4. DSOs for master data loads
What is the benefit of using full MD extraction and DSO delta insetad of MD delta extraction?
Thanks,
Marcin

1. Mainetance of logical partitioning.
Let's assume logical partitioning is performed on year. Does it mean that every year or so it is necessary to create additional cube/transformation and modify multiprovider? Is there any automatic procedure by SAP that supports creation of new objects, or it is fully manual?
Logical partitioning is when you have separate ODS / Cubes for separate Years etc ....
There is no automated way - however if you want to you can physically partition the cubes using time periods and extend them regularly using the repartitioning options provided.
2.Pass- though layer.
There are very few information about this basic concept. Anyway:
- is pass through DSO write optimized one? Does it store only one loading - last one? Is it deleted after lading is sucessfully finished (or before new load starts)? And - does this deletion do not destroy delta mechanism? Is the DSO replacing PSAfunctionally (i.e. PSA can be deleted every load as well)?
Usually a pass through layer is used to
1. Ensure data consistency
2. Possibly use Deltas
3. Additional transformations
In a write optimized DSo - the request ID is key and hence delta is based on request ID. If you do not have any additional transformations - then a Write optimized DSO is essentially like your PSA.
3. Query pruning
Does this happen automatically on DB level, or additional developments with exits variables, steering tables and FMs is required?
The query pruning - depends on the run based and cost based optimizers within the DB and not much control over how well you can influence the execution of a query other than havin up to date statistics , building aggregates etc etc.
4. DSOs for master data loads
What is the benefit of using full MD extraction and DSO delta insetad of MD delta extraction?
It depends more on the data volumes and also the number of transformation required...
If you have multiple levels of transformations - use a DSO or if you have very high data volumes and want to identify changed records - then use a DSO.

To delete a request from ODS

Hello all,
I am not able to delete the request from ODS .
It gives the error message :no request found in the change log.
how shlould i delete the request??
thnx in advance.
Regards,
Dhanya.

Hello,
thnx sriram n balaji!
i checked my infosource through the prgm:RSAR_PSA_CLEANUP_DIRECTORY
the following message appears in the display log:
PSA: 8ZFIADS11_CA
Version 000 technical name /BIC/B0000043000
Partition 0003
Request Ratio:[Deleted:Total]->1 :2
Records Ratio:[Deleted:Total]->753943 :1507886 ,50 % logically deleted
Partition 0004
Request Ratio:[Deleted:Total]->0 :1
Records Ratio:[Deleted:Total]->0 :753943 ,0 % logically deleted
what should i infer from this ?
can anyone please tell me the step by step procedure to delete the request.
Regards,
Dhanya.

How to get the Change Log Table name of the ODS from the system table

Hi,
I have a list of ODSs in the system and I am interested in finding the corresponding Change Log Table and the number of records in that table.
Can any one please tell me the name of the system table where this information is stored.
I dont want to get in this information manully (from the Contents tab of the ODS maintenane).
Regards,
Harikiran Gunnala

Hi
Go to SE16
Give the table name as "RSTSODS".
For User application field give CHANGELOG as input
and execute. You will get all the list of Changelog tables Along with below mentioned data
PSA
Version
To
Valid from
Object Status
Technical ODS name
Program Name
Maint Program
Object Name
Last changed by
UTC Time Stamp in Short Form
Partition number
Regards
Raj

How to store as single record data coming from different ods to cube

Hi All,
we have the scenario like .
Same contract values are uploaded from 3 ods to cube.In the cube this information is stored as 3 different records.
Is there any option having same contract no and same fiscal period values should be stored as single record in cube.
Thanks in advance.
Regards,
Shradda.

Hi Shradda,
On Performance Side .....consider below points
1. Indexes on Cube ( That delete indexes before load and create them after Load ).
2. Aggregate Design ( Decision on Base Aggregates, roll up Hierarchy , BW Statistics etc ).
3. Partition of InfoCube ( Basically decision on No. of Partition )
4. Data Package Size ( Always try to have larger Data Package so that preaggreation will reduce the no. of Data Records ).
Best is Service.sap.com on BI page u will find material for performance . THat will help you .
To assign points on the left of screen u will get radio buttons for assigning points for each person who has responded .
Regards,
Vijay

How to select on an ODS with index to consume less internal memory?

Hi all,
I want to load an ODS B from ODS A with an enhanced communication structure. The enhanced fields are populated through look up on a third ODS C based on joining condition with certain fields of the data_package (structure of ODS A).
The entire code is written in the start routine from A to B and the look up on C is done by a select statement on C for all entries in data_package ...etc. However this ODS C has huge data and when we are loading the data from A to B and hitting this select in start routine it is giving us a short dump due to shortage of extendable memory space.
To solve this problem we had built an index on ODS C with the fields used in where clause of the select statement but the problem persists. Can anybody tell me how can we take the benefit of the index so created in the select statement on ODS C in start routine ? Because it is the internal memory and not the query size that is giving the trouble is there any way to partition ODS C and use select on the partitions separately? Should an internal table of type ODS C be declared as sorted table with unique key instead of standard table ?
FYI, the index fields we have added in ODS maintenance screen are checked for unique.

I guess the short dump you are encountering is due to the size of the internal table you are filling up from ODS C not the index. Check the where clause on the select statement. You can reduce the data package size. Try using selections on the infopackages from A to B to limit the data... you can try other options to split the data into multiple (serial sessions) based on the business rules & logic in place at your implementation.
Default installation has a limitation of 2GB on the size of internal sessions considering signed 32bit variable on 32 bit o/s systems. Now that most of the unix based systems are 64bit and with unsigned 32bit variables you can use 4GB or more.
Check the note 548845 for more info
Gopal
Pls assign points if you find the response helpful !

Difference in number of records in GROUP BY and PARTITION BY

Hi Experts
If I run the following query I got 997 records by using GROUP BY.
SELECT c.ins_no, b.pd_date,a.project_id,
a.tech_no
FROM mis.tranche_balance a,
FMSRPT.fund_reporting_period b,
ods.proj_info_lookup c,
ods.institution d
WHERE a.su_date = b.pd_date
AND a.project_id = c.project_id
AND c.ins_no = d.ins_no
AND d.sif_code LIKE 'P%'
AND d.sif_code <> 'P-DA'
AND a.date_stamp >='01-JAN-2011'
AND pd_date='31-MAR-2011'
GROUP BY c.ins_no,
b.pd_date,
a.project_id,
a.tech_no;
I want to show the extra columns a.date_stamp and a.su_date
in the out put so that I have used PARTITION BY in the second query but I got 1079 records.
SELECT c.ins_no, b.pd_date,a.date_stamp,a.su_date, a.project_id,
a.tech_no,
COUNT(*) OVER(PARTITION BY c.ins_no,
b.pd_date,
a.project_id,
a.tech_no)c
FROM mis.tranche_balance a,
FMSRPT.fund_reporting_period b,
ods.proj_info_lookup c,
ods.institution d
WHERE a.su_date = b.pd_date
AND a.project_id = c.project_id
AND c.ins_no = d.ins_no
AND d.sif_code LIKE 'P%'
AND d.sif_code <> 'P-DA'
AND a.date_stamp >='01-JAN-2011'
AND pd_date='31-MAR-2011'
Please help me why I got 1079 records.
And also please help me how to show the two extra columns in the out put whcich are not used in
GROUP BY clause.
Thanks in advance.

Hi,
user9077483 wrote:
Hi Experts
If I run the following query I got 997 records by using GROUP BY. ...Let''s call this "Query 1", and the number of rows it returns "N1".
The results tell you that there are 997 distinct combinations of the GROUP BY columns (c.ins_no, b.pd_date, a.project_id, a.tech_no).
I want to show the extra columns a.date_stamp and a.su_date
in the out put so that I have used PARTITION BY in the second query but I got 1079 records. ...Let's call the query without the GROUP BY "Query 2", and the number of rows it returns "N2".
Please help me why I got 1079 records.Because there are 1079 rows that meet all the GROUP BY conditions. Query 2 has nothing to with distinct values in any columns. You would expect that N2 to be at least as high as N1, but it's not surprising that the N2 is higher than N1.
And also please help me how to show the two extra columns in the out put whcich are not used in
GROUP BY clause.Doesn't Query 2 show those two columns already? If the Query 2 is not producing the results you want, then what results do you want?
Post a little sample data (CREATE TABLE and INSERT statements, relevant columns only) for all the tables involved, and the results you want from that data.
Explain, using specific examples, how you get those results from that data. Point out a couple of places where Query 2 is not doing what you want.
Always say which version of Oracle you're using.

ODS Partitioning ?

Similar Messages

Maybe you are looking for