Improving initial load performance.

Hi ,
Please let me know the setup and prerequisite required for running parallel request so as to fasten the connection object download .
I need to download connection object and Point of Delivery from ISU to CRM. Is there any other way to improve the performance.
Regards,
Rahul

Hello,
May you please tell us more about your scenario? Because using the connection object ID may not be easy to start many request in parallel, as this field is alphanumeric if I remember well... meaning that a range between 1 and 2 will include 10, 11, 100, etc.
That's why within a migration process SAP introduced a new concept (via table ECRM_TEMP_OBJ) to replicate into CRM only those connection objects that are not already there. This is explained page 12 of the cookbook. Futhermore, as far as replication performance is concerned, I highly recommend to read those OSS notes carefully (which are valid for ISU technical objects as well):
Note 350176 - CRM/EBP: Performance improvement during exchange of data
Note 426159 - Adapter: Running requests in parallel
Regards,
Nicolas Busson.

Similar Messages

Improve data load performance using ABAP code

Hi all,
I want to improve my load performance using ABAP code, how to do this?. If i writing ABAP code in SE38 how i can call
in BW side? if give sample code to improve load performance it will be usefull. please guide me.

There are several points that can improve performance of your ABAP code:
1. Avoid using SELECT...ENDSELECT... construct and use SELECT ... INTO TABLE.
2. Use WHERE clause in your SELECT statement to restrict the volume of data retrieved.
3. Use FOR ALL ENTRIES in your SELECT statement to retrieve the matching records at one shot.
4.Avoid using nested SELECT and SELECT statements within LOOPs.
5. Avoid using INTO CORRESPONDING FIELDS OF. Instead use INTO TABLE.
6. Avoid using SELECT * and select only the required fields from the table.
7. Avoid Executing a SELECT multiple times in the program.
8. Avoid nested loops when working with large internal tables.
9.Whenever using READ TABLE use BINARY SEARCH addition to speed up the search.
10. Use FIELD-SYMBOLS instead of a work area when there are more than 200 entries in an internal table where some fields are being manipulated.
11. Use MOVE with individual variable/field moves instead of MOVE-CORRESPONDING.
12. Use CASE instead of IF/ENDIF whenever possible.
13. Runtime transaction code se30 can be used to measure the application performance.
14. Transaction code st05 can be used to analyse the SQL trace and measure the performance of the select statements of the program.
15. Start routines can be used when transformation is needed in the data package level. Field/individual routines can be used for a simple formula or calculation. End routines are used when you wish to populate data not present in the source but present in the target.
16. Always use a WHERE clause for DELETE statement. To delete records for multiple values, use SELECT-OPTIONS.
17. Always use 'IS INITIAL' instead of equal to '' because null for a character is '' but '0' for an integer.
Hope it helps.

Initial Load Performance Decrease

Hi colleagues,
We noticed a huge decrease initial load performance after installing an
application in the PDA.
Our first test we downloaded one data object with nearly 6.6Mb that
corresponds to 30.000 registries with eight fields each. Initial Load
to PDA took only 2 minutes.
We performed a second test with same PDA after a reinstallation and
new device ID. The difference here is that we installed an MI
application related to same data object. Same amount of data was sent
to the PDA. It took 3 hours to download it.
In third test we change the application in order not to have the
related data object assigned to it. In this case, download took 2
minutes again.
In other words, if we have an application with the data object
assigned, it results in a huge decrease in initial load.
In both cases we use direct connection to our LAN.
Here goes our PDA specs:
- Windows Mobile 6 Classic
- Processor: Marvell PXA310 de 624MHz
- 64MB RAM, 256MB flash ROM (190MB available to user)
Any similar experiences?
Thanks.
Edited by: Renato Petrulis on Jun 1, 2010 4:15 PM

I am confused on downloading a data object with no application.
I thought you can only download data if it is associated to a Mobile Component, I guess you just assign the DMSCV manually?
In any case, I have only experienced scenario two when we were downloading application with mobile component with no packaging of messages. we had maybe a few thousand records to download and process and it would take an hour or more.
When we enabled packaging, it would take 15-30 minutes.
Then I went to Create Setup Package because it was just simpler to install the application and data together with no corruption or failure of DMSCV not going operational and not sending data etc... plus it was a faster download using either FTP or ActiveSync to transfer the install files.

How to improve the load performance while using Datasources for the Invoice

HI All,
How to improve the load performance while using Datasources for the Invoice . Actually my invoice load (Appx. 0.4 M records) is taking very long time nearly ~16 to 18 hrs to update data from R/3 to 0ASA_DS01.
If I load through flat file it will load with in ~20 Min for the same amount of data.
Please suggest how to improve load performance.
PS: I have done the Inpo package settings as per the OSS note.
Regads
Srininivasarao.Namburi.

Hi Srinivas,
Please refer to my blog posting [/people/divyesh.jain/blog/2010/07/20/package-size-in-spend-performance-management-extraction|/people/divyesh.jain/blog/2010/07/20/package-size-in-spend-performance-management-extraction] which gives the details about the package size setting for extractors. I am sure that will be helpful in your case.
Thanks,
Divyesh
Edited by: Divyesh Jain on Jul 20, 2010 8:47 PM

How to improve query & loading performance.

Hi All,
How to improve query & loading performance.
Thanks in advance.
Rgrds
shoba

Hi Shoba
There are lot of things to improve the query and loading performance.
please refer oss note :557870 : Frequently asked questions on query performance
also refer to
weblogs:
/people/prakash.darji/blog/2006/01/27/query-creation-checklist
/people/prakash.darji/blog/2006/01/26/query-optimization
performance docs on query
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/3f66ba90-0201-0010-ac8d-b61d8fd9abe9
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/cccad390-0201-0010-5093-fd9ec8157802
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/ce7fb368-0601-0010-64ba-fadc985a1f94
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/c8c4d794-0501-0010-a693-918a17e663cc
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/064fed90-0201-0010-13ae-b16fa4dab695
This is the oss notes of FAQ on query performance
1. What kind of tools are available to monitor the overall Query Performance?
1. BW Statistics
2. BW Workload Analysis in ST03N (Use Export Mode!)
3. Content of Table RSDDSTAT
2. Do I have to do something to enable such tools?
Yes, you need to turn on the BW Statistics:
RSA1, choose Tools -> BW statistics for InfoCubes
(Choose OLAP and WHM for your relevant Cubes)
3. What kind of tools is available to analyze a specific query in detail?
1. Transaction RSRT
2. Transaction RSRTRACE
4. Do I have an overall query performance problem?
i. Use ST03N -> BW System load values to recognize the problem. Use the number given in table 'Reporting - InfoCubes:Share of total time (s)' to check if one of the columns %OLAP, %DB, %Frontend shows a high number in all Info Cubes.
ii. You need to run ST03N in expert mode to get these values
5. What can I do if the database proportion is high for all queries?
Check:
1. If the database statistic strategy is set up properly for your DB platform (above all for the BW specific tables)
2. If database parameter set up accords with SAP Notes and SAP Services (EarlyWatch)
3. If Buffers, I/O, CPU, memory on the database server are exhausted?
4. If Cube compression is used regularly
5. If Database partitioning is used (not available on all DB platforms)
6. What can I do if the OLAP proportion is high for all queries?
Check:
1. If the CPUs on the application server are exhausted
2. If the SAP R/3 memory set up is done properly (use TX ST02 to find bottlenecks)
3. If the read mode of the queries is unfavourable (RSRREPDIR, RSDDSTAT, Customizing default)
7. What can I do if the client proportion is high for all queries?
Check whether most of your clients are connected via a WAN connection and the amount of data which is transferred is rather high.
8. Where can I get specific runtime information for one query?
1. Again you can use ST03N -> BW System Load
2. Depending on the time frame you select, you get historical data or current data.
3. To get to a specific query you need to drill down using the InfoCube name
4. Use Aggregation Query to get more runtime information about a single query. Use tab All data to get to the details. (DB, OLAP, and Frontend time, plus Select/ Transferred records, plus number of cells and formats)
9. What kind of query performance problems can I recognize using ST03N
values for a specific query?
(Use Details to get the runtime segments)
1. High Database Runtime
2. High OLAP Runtime
3. High Frontend Runtime
10. What can I do if a query has a high database runtime?
1. Check if an aggregate is suitable (use All data to get values "selected records to transferred records", a high number here would be an indicator for query performance improvement using an aggregate)
2. o Check if database statistics are update to data for the Cube/Aggregate, use TX RSRV output (use database check for statistics and indexes)
3. Check if the read mode of the query is unfavourable - Recommended (H)
11. What can I do if a query has a high OLAP runtime?
1. Check if a high number of Cells transferred to the OLAP (use "All data" to get value "No. of Cells")
2. Use RSRT technical Information to check if any extra OLAP-processing is necessary (Stock Query, Exception Aggregation, Calc. before Aggregation, Virtual Char. Key Figures, Attributes in Calculated Key Figs, Time-dependent Currency Translation) together with a high number of records transferred.
3. Check if a user exit Usage is involved in the OLAP runtime?
4. Check if large hierarchies are used and the entry hierarchy level is as deep as possible. This limits the levels of the hierarchy that must be processed. Use SE16 on the inclusion tables and use the List of Value feature on the column successor and predecessor to see which entry level of the hierarchy is used.
5. Check if a proper index on the inclusion table exist
12. What can I do if a query has a high frontend runtime?
1. Check if a very high number of cells and formatting are transferred to the Frontend (use "All data" to get value "No. of Cells") which cause high network and frontend (processing) runtime.
2. Check if frontend PC are within the recommendation (RAM, CPU MHz)
3. Check if the bandwidth for WAN connection is sufficient
and the some threads:
how can i increse query performance other than creating aggregates
How to improve query performance ?
Query performance - bench marking
may be helpful
Regards
C.S.Ramesh
[email protected]

How to Improve DSO loading performance

Hello,
I have a DSO having 3 infosources. This DSO is Generic means based on generic Data Sources. Daily we have a full upload (last 2 months data). Initially it was taking around 55 mins to load the data but now a days all are taking 2.5 Hrs daily.
Can u please tell me how can i improve the performance in other word how can i reduce the time.
Please give some solution or document to resolve this.
amit

Hi,
Genearl tips you can try to improve the data load performance
1. If they are full loads then try to see if you make them delta loads.
2. Check if there are complex routines/transformations being performed in any layer. In that case see if you can optimize those codes with the help of an abaper.
3. Ensure that you are following the standard procedures in the chain like deleting Indices/secondary Indices before loading etc.
4. Check whether the system processes are free when this load is running
5. Try making the load as parallel as possible if the load is happening serially. Remove PSA if not needed.
6. Goto manage ODS -> activate -> activate in parallel -> increase the number of processes from there.for direct access try TCode RSODSO_SETTINGS
7. Remove Bex Reporting check box in ODS if not required.
Ensure the data packet sizing and also the number range buffering, PSA Partition size, upload sequence i.e, always load master data first, perform change run and then transaction data loads.
Use InfoPackages with disjoint selection criteria to parallelize the data export.
Complex database selections can be split to several less complex requests.
Check this doc on BW data load perfomance optimization
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/1955ba90-0201-0010-d3aa-8b2a4ef6bbb2
BI Performance Tuning
Business Intelligence Journal Improving Query Performance in Data Warehouses
http://www.tdwi.org/Publications/BIJournal/display.aspx?ID=7891
Achieving BI Query Performance Building Business Intelligence
http://www.dmreview.com/issues/20051001/1038109-1.html
SAP Business Intelligence Accelerator : A High - Performance Analytic Engine for SAP Ne tWeaver Business Intelligence
http://www.sap.com/platform/netweaver/pdf/BWP_AR_IDC_BI_Accelerator.pdf
BI Performance Audit
http://www.xtivia.com/downloads/Xtivia_BIT_Performance%20Audit.pdf
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/10564d5c-cf00-2a10-7b87-c94e38267742
ODS Query Performance
Thanks,
JituK

To improve data load performance

Hi,
The data is getting loaded into the cube. Here there are no routines in update rules and transfer rules. Direct mapping is done to the infoobjects.
But there is an ABAP routine written for 0CALDAY in the infopackage . Other than the below code , there is no abap code written anywhere. For 77 lac records it is taking more than 10 hrs to load. Any possible solutions for improving the data load performance.
DATA: L_IDX LIKE SY-TABIX.
DATA: ZDATE LIKE SY-DATUM.
DATA: ZDD(2) TYPE N.
READ TABLE L_T_RANGE WITH KEY
       FIELDNAME = 'CALDAY'.
L_IDX = SY-TABIX.
*+1 montn
ZDATE = SY-DATUM.
IF ZDATE+4(2) = '12'.
    ZDATE0(4) = ZDATE0(4) + 1.
    ZDATE+4(2) = '01'.
    ZDATE+6(2) = '01'.
    L_T_RANGE-LOW = ZDATE.
ELSE.
    ZDATE4(2) = ZDATE4(2) + 1.
    ZDATE+6(2) = '01'.
    L_T_RANGE-LOW = ZDATE.
ENDIF.
*+3 montn
ZDATE = SY-DATUM.
IF ZDATE+4(2) => '10'.
    ZDATE0(4) = ZDATE0(4) + 1.
    ZDATE4(2) = ZDATE4(2) + 3 - 12.
    ZDATE+6(2) = '01'.
ELSE.
    ZDATE4(2) = ZDATE4(2) + 3.
    ZDATE+6(2) = '01'.
ENDIF.
CALL FUNCTION 'FIMA_END_OF_MONTH_DETERMINE'
    EXPORTING
      I_DATE                   = ZDATE
    IMPORTING
      E_DAYS_OF_MONTH          = ZDD.
ZDATE+6(2) = ZDD.
L_T_RANGE-HIGH = ZDATE.
L_T_RANGE-SIGN = 'I'.
L_T_RANGE-OPTION = 'BT'.
MODIFY L_T_RANGE INDEX L_IDX.
P_SUBRC = 0.
Thanks,
rani

i dont think this filter routine is causing the issue...
please implement performance impovement methods..
FAQ - The Future of SAP NetWeaver Business Intelligence in the Light of the NetWeaver BI&Business Objects Roadmap

Extractor Designing to improve the Load performance.

Hi all,
I am extracting the data from MM application for this i am using the LO 2lis_02_itm extractor and i had enhanced it with 32 field and its happering my data load performance.
Could u pls let me know, how can i improve the data load performance.
Do i need to create the different Generic Extractors instead of enhancing the LO.
The DSO is also having many fields in it. Should i split it into 2 and create the Multiprovider for reporting.
Regards
KK

Hello,
my suggestion would be to create another generic DS for the logcal set of fields to be required in BI.
then you can load then seperately to different DSOs and then to single IC or to two IC and use MP to report on them.
Further you can check the below links:
Extraction-Enhancement-Performance problem
Increase dataload performance
Dataload Performance
Performance Enhancement for Custom Data Extractor
Regards,
Dhanya

How to improve class-loading performance for missing classes

Hi,
do you have any ideas how we can improve the class-loading performance on a weblogic 12c running on jrockit? We spend about 20-30 ms per request in class-loading due to the way hibernate criteria API works (it tries to load quite a lot of missing classes). The only thing I came up with was to invert the class-loading using the weblogic descriptor for those missing classes (which are actually no real classes but some parts of a generated JPQL) which does improve performance a bit.
Thanks
Dimo

Hi,
do you have any ideas how we can improve the class-loading performance on a weblogic 12c running on jrockit? We spend about 20-30 ms per request in class-loading due to the way hibernate criteria API works (it tries to load quite a lot of missing classes). The only thing I came up with was to invert the class-loading using the weblogic descriptor for those missing classes (which are actually no real classes but some parts of a generated JPQL) which does improve performance a bit.
Thanks
Dimo

How to improve the load performance

can any body tell me how to improve the load performance

Hi,
for all loads: improve your ABAP code in routines.
for master data load:
- load master data attributes before the charateristic itself
- switch number range buffering on for initial loads
for transactional loads:
- load all your master data IObjs prior loading your cube / ODS
- depending on the ratio No.Records loaded / No.Records in Cube F fact table, drop / recreate indexes (if ration is mor than 40/50%
- switch on number range buffering for dimensions with high number of records for initial loads
- switch on number range buffering on master data IObjs which aren't loaded via master data (SIDs always created while transactional loads; eg document, item....)
these recommendations are just some among others like system tuning, DB parameters...
hope this helps...
Olivier.

Initial Load performs deletion TWICE!!

Hi All,
I face a very peculiar issue. I started an initial load on a codition object. On the R/3 there are about 3 million records. The load starts
1)First it deletes all the records in CRM(count bcomes 0)
2) Then it starts inserting the new records ( the records get inserted and the count reaches 3 million)
in R3AM1 this adapter object(DNL_COND_A006) status changes to "DONE"!!
Now comes the problem
There are still some queue entries which again starts deleting the entries from the condition table and the
count starts reducing and the record count becomes 0 agai n in the conditio table!
Then it again starts inserting and the entire load stops after insertin 1.9 million records! Thsi isvery strange.Any pointers will be helpful
I also checked whether the mappin module is maintained twice in CRM but that is also not the case. Since the initial load takes more than a day i checked whether there are any jobs scheduled but there are no jobs scheduled also.
I am really confused as to why 2 times deletion should happen. Any pointers will be highly appreciated.
Thanks,
Abishek

Hi Abishek,
This is really strange and I do not have any clue. What I can suggest is that before you start the load of DNL_COND_A006, load the CNDALL & CND object again. Some time CNDALL resolve this kind of issues.
Good luck.
Vikash.

Golden Gate Initial Load - Performance Problem

Hello,
I'm using the fastest method of initial load. Direct Bulk Load with additional parameters:
BULKLOAD NOLOGGING PARALLEL SKIPALLINDEXES
Unfortunatelly the load of a big Table 734 billions rows (around 30 GB) takes about 7 hours. The same table loaded with normal INSERT Statement in parallel via DB-Link takes 1 hour 20 minutes.
Why does it take so long using Golden Gate? Am I missing something?
I've also noticed that the load time with and without PARALLEL parameter for BULKLOAD is almost the same.
Regards
Pawel

Hi Bobby,
It's Extract / Replicat using SQL Loader.
Created with following commands
ADD EXTRACT initial-load_Extract, SOURCEISTABLE
ADD REPLICAT initial-load_Replicat, SPECIALRUN
The Extract parameter file:
USERIDALIAS {:GGEXTADM}
RMTHOST {:EXT_RMTHOST}, MGRPORT {:REP_MGR_PORT}
RMTTASK replicat, GROUP {:REP_INIT_NAME}_0
TABLE Schema.Table_name;
The Replicat parameter file:
REPLICAT {:REP_INIT_NAME}_0
SETENV (ORACLE_SID='{:REPLICAT_SID}')
USERIDALIAS {:GGREPADM}
BULKLOAD NOLOGGING NOPARALLEL SKIPALLINDEXES
ASSUMETARGETDEFS
MAP Schema.Table_name, TARGET Schema.Table_tgt_name,
COLMAP(USEDEFAULTS),
KEYCOLS(PKEY),
INSERTAPPEND;
Regards,
Pawel

How to improve this load performance

Hi Experts,
There is load to infoobject which is full and having selections on sales document (4 ranges )
AUART Sales Document Type (10) and PSTYV Sales document item category (12)
The zdatasource is based on view which is built on three tables VBAK, VBAP and VBKD
what are the possibilities of improving the performance , Please help
Thanks in Advance,
Nitya

Why you had created a Generic data source for tables VBAP<VBAK<VBKD
You have standard data source
2LIS_11_VAITM -
> VBAP, VBUP, VBAK, VBKD, VBAJP, T001, VBUK, PRPS.
Make use of standard data source if at all there are any fields needed other than provided by SAP enhance the data source.
VIEW is nothing but combination of tables with keys in the tables. it will degrade the performance.... try to check the view design once again....
Are you running the info object full load for the first time? then it will take time...
Try running the delta from next load for info object....
Regards
KP

How do I improve package loading performance in BIDS when I open a package which is imported from another place?

[This is just for sharing information for those common asked questions collected from forums. If you have any better way or feedback, please directly reply in this thread so that community users can benefit from your answers.]
Question:
I have a SSIS package which has some execution problem in production environment and I need to check it and perform debugging on my local machine, however when I add the package into my SSIS project and open it in BIDS, my BIDS hangs for a long time.
Answer:
This issue is usually caused by the fact that your SSIS package is complex with many tasks including many connection managers. When you open the SSIS package in BIDS, by default the validation process on each task will start. Since your SSIS package is copied
from another place, the connections may be invalid. In this case, the validation process will spend a lot of time and cause the BIDS not working during this time.
To resolve this issue, you can do as follows:
1.Before you add the SSIS package, select your SSIS project first, then click the SSIS menu, and change your SSIS project to Offline mode by checking "Work Offline".
2.Add your SSIS package to the project and change each Task's DelayValidation property to True.
3.After these steps, you can modify the connections to be valid based on your local test environment.
Please remember to mark the replies as answers if they help and unmark them if they provide no help

Hello Charles,
My workmate am so happy to hear this solution I told him. Really thanks for your detail solution.
Best regards,
Challen Fu
Challen Fu
TechNet Community Support

Loading performance of the infocube & ODS ?

Hi Experts,
Do we need to turn off the aggregates on the infocubes before loading so that it will decrease the loading time or it doesn't matter at all, I mean if we have aggregates created on the infocube..Is that gonna effect in anyway to the loading of the cube ? Also please let me know few tips to increase the loading performance of a cube/ods. Some of them are
1. delete index and create index after loading.
2. run paralled processes.
3. compression of the infocube , how does the compression of an infocube decrease the loading time ?
Please throw some light on the loading performance of the cube/ods.
Thanks,

Hi Daniel,
Aggregates will not affect the data loading. Aggregates are just the views similar to InfoCube.
As you mentioned some performance tuning options while loading data:
Compression is just like archiving the InfoCube data. Once compressed, data cannot be decompressed. So need to ensure the data is correct b4 Compressing. When you compress the data, you will have some free space available, which will improve data loading performance.
Other than the above options:
1.If you have routines written at the transformation level, just check whether it is tuned properly.
2.PSA partition size: In transaction RSCUSTV6 the size of each PSA partition can be defined. This size defines the number of records that must be exceeded to create a new PSA partition. One request is contained in one partition, even if its size exceeds the user-defined PSA size; several packages can be stored within one partition.
The PSA is partitioned to enable fast deletion (DDL statement DROP PARTITION). Packages are not deleted physically until all packages in the same partition can be deleted.
3. Export Datasource:The Export DataSource (or Data Mart interface) enables the data population of InfoCubes and ODS Objects out of other InfoCubes.
The read operations of the export DataSource are single threaded (i.e. sequential). Note that during the read operations u2013 dependent on the complexity of the source InfoCube u2013 the initial time before data is retrieved (i.e. parsing, reading, sorting) can be significant.
The posting to a subsequent DataTarget can be parallelized by ROIDOCPRMS settings for the u201Cmyselfu201D system. But note that several DataTargets cannot be populated in parallel; there is only parallelism within one DataTarget.
Hope it helps!!!
Thanks,
Lavanya.

Improving initial load performance.

Similar Messages

Maybe you are looking for