Data Load slow on some days

Dear SDN Team,
I have a load into ODS which sometimes takes 5 minutes to finish but on some days takes >40 minutes. (Though number of records are same)
Problem is not in R/3 but in BW itself.
In the details tab I notice the following:
The time stamp at 'Tranfer (Idocs and TRFC) shows 5:05
The time stamp at 'processing datapacket' shows 5:50
Kindly help me as this is really annoying.
Best Regards,
Krishna.
Edited by: Siegfried Szameitat on Oct 29, 2008 12:39 PM
I deleted your offer of points as it is against the rules.

Hi,
1)You can check with just a click on the last step before the first data package in the details tab.
This will create a displa of time below
once you click on the step it will give you a time and if you click on a step below it will give you another time in the same box.
This is how I determin the time elapsed between different steps in the manage tab.
2)To find the job search for job begning with BI* and user ALEREMOTE.
You will get a long list just try to see the tart time for both and he one which matches with the start of the job in BW and may be slightly after that.
You will have to go inside job log of each job to see the data source and the one whch contains your data source is the job you want.
Thanks
Ajeet

Similar Messages

EIS data load slow

Greetings all,
Windows Server 2003 SP2. SQL Server 2000 SP3. Hyperion 9.3.1.
I am running an EIS data load, and the load time is progressively longer depending on the number of records loaded.
For example, when doing a data load of 1,000,000 records, it takes 2 minutes to load the first 500,000 records but an additional 10 minutes to load the next 500,000 records.
Is there any reason for this, and can I do anything about it?
Regards,
Paul.

Since non-Streaming mode is the only mode that EIS leverages to load data, it's not surprising that- Data loads are slow.
If you want to continue using EIS for data loads, I believe- All you can do is- To refine/tune your User-defined SQL.
You've mentioned about just few million records. However, in the volume-intensive environments where records are in the order of hundreds/thousands of millions, the majority would think of using EIS only for Metadata load & deferring data load to the flat file data extracts, which is way faster. At the same time, you'd have to consider if the extraction time required is not too much that it defeats the very purpose.
- Natesh

Data loading slow in BI server

hi all,
We have installed our BI production server recentlly. while loading data BI functional consultants complain of poor speed. The process which took only few seconds in development and quality system is taking several minutes in the production system even though the production server is a complete dedicated server for the BI production with 32GB RAM. being a basis person Please help us in identifying the issue in the production server which is causing the speed to be so slow in the prod server as the prod server would be used by users who wouldnt like to wait for so long. and what can i do to improve the speed of the servers.
thanks,
Priya

loading from PSA takes always less time.
However, is quite generic a time load issue, certainly the production system will have more data so it is not always possible to use development and test as reference.
Anyway, i think you should focus on the tuning (using the SBIW)
For example
on R3 you have to customize the dat aload.
on BW for datamart.
So, in R3
maximal package size in kB (default 10,000 KB)
maximum package size in number of records (Maximum memory consumption per data package is around 2Max. lines1000
Byte)
Frequency of Info IDocs. Default: 1. Recommended: 5 - 10
Max proc. (= maximum number of dialog workprocesses which were
allocated for this dataload for processing extracted data in BW)
in BW
values recomended
Recommended values:
max size: 20,000 50,000 (kbyte)
max lines: 50,000 100,000 (Data lines)
frequency: 10
max proc.: 2 5
there are a lot of parameters to set, but i suggest you to proceed stepbystep.
ciao

Related Data Loading

Hi,
I am new to BI, I am going to work support project. Mainly monitoring the process chain & data loading.
So please anyone tell me , what things i have to keep in mind for data loading and tell some real time scenarios and Issues regarding data loading please....
<removed by moderator>.
<removed by moderator>.
Thanks & Regards,
San
Edited by: Siegfried Szameitat on Nov 17, 2008 12:27 PM

Hi,
Take a look at the links below.
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/08f1b622-0c01-0010-618c-cb41e12c72be
Data load errors - basic checks
Process Chain Errors
Regards.

Direct vs Indirect data load from ODS - InfoCube

We know that there are two options to load data from ODS to InfoCube:
1. Direct load:
If the ODS setting is checked with the option of "Update data targets from ODS object automatically", then when loading data to ODS, then the data also can be loaded to the InfoCube, in this way, only one InfoPackage is needed to load data to the ODS, then the InfoCube can be filled with data automatically.
2. Indirect load:
If the ODS setting is NOT checked with the option of "Update data targets from ODS object automatically", then not only the InfoPackage to load the data to the ODS is needed, but also the InfoPackage to load data from the exported datasource of the ODS to the InfoCube is needed to be created in order to load the data from ODS to the InfoCube.
I wonder what are the pro and con between the above two loading methods from ODS to InfoCube.
Welcome to every BW expert inputs!

HI Kevin,
Direct Loads or rather Automated Loads are usually used where u need to load data automatically into the final data target. And it has no dependencies.This process has less of maintenance involved and u need to execute a single infopackage to send data to the final data target.
But Indirect Loads are usually used when u have a number of dependencies and this ODs object is one of the object in a process chain. So if u require the this ODS data load wait till some other event has been executed , then indirect loads are used through process chains.
Regards,
JAsprit

Can't Delete Data Load Request

Hi All,
During the initial data load there was some error and the data load in Manage ODS is shown as RED, also 12 records are seen as uploaded, which can be seen in New Data Tab. This request is also seen as available for reporting but with RED Light. Now, we want to delete this request but we can't, the reason is beyond our knowledge. Can some help how to delete this request, so that we can load data again. Below is wht we could get
Message type__________ E (Error)
Message class_________ RSMPC (Messages for Process Chains of Generic Processes)
We didn't use Process Chain for uploading the initial data
Thanks & Regards
Vikas

Hi,
Go to the info package
select the "Sheduler" menu
Delete the initial upload data and try the initial load again
regards
Happy Tony

I have installed Photoshop element 12 trail version in my system and i worked also for some days but it given error that "cannot load default brushes because scratch disk is full " but scratch is 158GB free

Hi,
I have installed Photoshop element 12 trail version in my system and i worked also for some days but it given error that "cannot load default brushes because scratch disk is full " but scratch is 158GB free why whats the problem am not getting.

That thread helped for a time. But now it seems as though my computer is back to slow again. But, some of the jargon is over my head as well. Not sure I knew quoite everything they were talking about. But thanks a lot for that, it helped I probably just need to reread it and follow the instructions again. Thanks so much.

Very slow performance in every area after massive data load

Hi,
I'm new to Siebel. I had a call from customer saying that virtually every aspect of the application (login, etc) is slow after they did a massive data loading ~ around 15GB of data.
Could you please help to point out what would be the best practice for this massive data loading exercise? All the table statistics are up to date.
Anyone encountered this kind of problem before?

Hello,
Siebel CRM is a highly customizable customer relationship management solution. There are number of customizations (scripting, workflow, web services,...) and integrations (custom c++, java, ERP system,...) that can cause Siebel performance issues.
Germain Monitoring v1.8.5 can help you -clean-up- all your siebel performance issues (5 min after installation, which can take between 4hours and 10 days whether it is to be used against your siebel dev/qa or prod environment) and then monitor your siebel production system at every layer of your infrastructure, at the siebel user click & back-end transaction levels and either solve or identify the root-cause of siebel performance issues, 24x7.
Germain Monitoring Software (currently version 1.8.5) helps siebel customers 1)faster solve siebel performance issues introduced by customizations and 2)effectively solve siebel performance issues before business is impacted once siebel is on production.
Customers like NetApp, J.M Smucker, Alltel/Verizon,...have saved hundred of thousands of dollars using Germain Monitoring software.
Let us know whether you would like to discuss this further...good luck w/ these issues,
Regards,
Yannick Germain
GERMAIN SOFTWARE LLC
Siebel Performance Software
21 Columbus Avenue, Suite 221
San Francisco, CA 94111, USA
Cell: +1-415-606-3420
Fax: +1-415-651-9683
[email protected]
http://www.germainsoftware.com

Last day of previous month for data load

Hi,
I have to load data from the previous month into the psa and then into an infocube. I was wondering as to how to get the last of the previous month to write a code in ABAP. I will be writing the code at the infopackage level in the data selection. I could load data from the 1st of the previous month to the 1st of the current month. This will be an additional load of 30,000 records for the 1st of every month, since I will be loading 30,000 records everyday, I was wondering if I could limit the load from the 1st of every previous month to the last day of that month. This will be a repetitive loading.
DATA: CURR_MM(2) TYPE N,
CURR_YYYY(4) TYPE N,
CURR_DD(2) TYPE N,
PREV_MM(2) TYPE N,
PREV_YYYY(4) TYPE N,
PREV_DD(2) TYPE N,
YYYY_MM(6),
YYYY_MM1(6),
DATE LIKE SY-DATUM.
DATE = SY-DATUM.
CURR_YYYY = DATE+0(4).
CURR_MM = DATE+4(2).
CURR_DD = DATE+6(2).
PREV_DD = 1.
IF CURR_MM = '01'.
PREV_MM = '12'.
PREV_YYYY = CURR_YYYY - 1.
ELSE.
PREV_MM = CURR_MM - 1.
PREV_YYYY = CURR_YYYY.
ENDIF.
concatenate PREV_YYYY PREV_MM PREV_DD into YYYY_MM.
concatenate CURR_YYYY PREV_MM PREV_DD into YYYY_MM1.
read table l_t_range with key
fieldname = 'BLDAT'.
l_idx = sy-tabix.
l_t_range-low = YYYY_MM.
l_t_range-high = YYYY_MM1.
l_t_range-sign = 'I'.
l_t_range-option = 'BT'.
modify l_t_range index l_idx.
p_subrc = 0.
Mind you this code will load data from 1st of the previous month to the 1st of current month. I just don't want to load that extra "1st day" of current month data as I have 30,000 records everyday.
Say for example, I want to load data from 1st Mar to 31st Mar or 1st Feb to 28thFeb. How should I modify the above code.
Is there a formula to get the last date of the previous month. That's all I need. This would solve the problem.

try this routine. it will return a range from 1st day to end of the month.
DATA: l_s_range TYPE rsr_s_rangesid,
          E_T_RANGE TYPE RSR_T_RANGESID.
DATA: year(4) TYPE n,
      month(2) TYPE n,
      day(2) TYPE n,
    ld_keydate TYPE sydatum,
      ld_lastday TYPE sydatum.
REFRESH e_t_range.
CLEAR l_s_range.
year = sy-datum(4).
month = sy-datum+4(2).
*Months with 31 days in year
IF month = '01' OR
     month = '03' OR
     month = '05' OR
     month = '07' OR
     month = '08' OR
     month = '10' OR
     month ='12'.
    day = '31'.
ENDIF.
*check for leap year: provoking sy-sybrc <> 0
IF month = '02'.
    day = '29'.
    MOVE:   '02'        TO ld_keydate+4(2),
            year        TO ld_keydate(4),
            day         TO ld_keydate+6(2).
    CALL FUNCTION 'DATE_CHECK_PLAUSIBILITY'
      EXPORTING
        date   = ld_keydate
      EXCEPTIONS
        OTHERS = 1.
    IF sy-subrc <> 0.
      day = '28'.
    ENDIF.
ENDIF.
*months with 31 days in year
IF month = '04' OR
     month = '06' OR
     month = '09' OR
     month = '11'.
    day = '30'.
ENDIF.
MOVE: year              TO ld_lastday(4),
        month             TO ld_lastday+4(2),
        day               TO ld_lastday+6(2).
l_s_range-low = sy-datum.
l_s_range-high = ld_lastday.
l_s_range-sign = 'I'.
l_s_range-opt = 'BT'.
APPEND l_s_range TO e_t_range.

4.2.3/.4 Data load wizard - slow when loading large files

Hi,
I am using the data load wizard to load csv files into an existing table. It works fine with small files up to a few thousand rows. When loading 20k rows or more the loading process becomes very slow. The table has a single numeric column for primary key.
The primary key is declared at "shared components" -> logic -> "data load tables" and is recognized as "pk(number)" with "case sensitve" set to "No".
While loading data, these configuration leads to the execution of the following query for each row:
select 1 from "KLAUS"."PD_IF_CSV_ROW" where upper("PK") = upper(:uk_1)
which can be found in the v$sql view while loading.
It makes the loading process slow, because of the upper function no index can be used.
It seems that the setting of "case sensitive" is not evaluated.
Dropping the numeric index for the primary key and using a function based index does not help.
Explain plan shows an implicit "to_char" conversion:
UPPER(TO_CHAR(PK)=UPPER(:UK_1)
This is missing in the query but maybe it is necessary for the function based index to work.
Please provide a solution or workaround for the data load wizard to work with large files in an acceptable amount of time.
Best regards
Klaus

Nevertheless, a bulk loading process is what I really like to have as part of the wizard.
If all of the CSV files are identical:
use the Excel2Collection plugin ( - Process Type Plugin - EXCEL2COLLECTIONS )
create a VIEW on the collection (makes it easier elsewhere)
create a procedure (in a Package) to bulk process it.
The most important thing is to have, somewhere in the Package (ie your code that is not part of APEX), information that clearly states which columns in the Collection map to which columns in the table, view, and the variables (APEX_APPLICATION.g_fxx()) used for Tabular Forms.
MK

How to add some days to the current system date

can anyone help how to add some days to the current date i.e if today is 3-12-2007 and if v add 15 more days then the output should be 18-12-2007

RajeshChandan wrote:
First of all thanks a lot ,and coming to the question i dont mean to change the sysdate instead i want to write a prg. where after entering the current date it should effect with the no.of days that i have given but not the system date .if u can answer please reply for thisAh, I see. In that case, you can use an instance of GregorianCalendar and use it's add(...) method to add days, months, years etc. to it.
http://java.sun.com/javase/6/docs/api/java/util/GregorianCalendar.html
Good luck.

How to identify the maximum data loads done for a particular day?

Hi all,
There is huge volume of data been loaded on a last monday, and then data was deleted from the cubes on the same day. And hence i needs to see which are all the cubes which was loaded wtih lot of records for a particular day,
i happened to look at rsmo, i am unable to see the ods nor the cube data loads,
were do i seet it?
Thanks

See if the table RSSELDONE helps. This will give you the recent data load details. Based on those loads , you can search the targets.
And also check table TBTCO which will give the latest job details. You will have to analyze the same jobs to know what loads were done . Give a selection for date.

Since some days my MacBookPro 2011 is slow on start up. It takes 3 minutes from the gingle to the Password. What to do???

Since some days my MacBookPro is slow on start up. It takes 3 minutes from the gingle to the password. The operation system is OS X Lion v. 10.7.4. What can I do???

in system Preferences > users and groups>admin>log in items
What is there?

Data load happening terribly slow

Hi all,
I had opened up quality server and made some change(removed its compunding attribute) at definition level to an infoobject present in the InfoCube.
The infoobject,IC and Update rules are all active.
when i scheduled the load now
ther is a huge variation in the data load speed.
Before:
3Hrs- 60 Lakhs
Now:
3hrs-10 Lakhs.
Also i m observing in SM66 that the process are all reading NRIV Table
Can anyone throw some insght to this scenario??
Any useful input will be rewarded!
Regards
Dhanya.

1. yes, select main memory. how many entries do you expect in this dimension?
In other terms, how many different combinations of characteristics calues included in your DIM are to be posted?
As I first guess, you should enter there 50'000 but please let me know the cardinality of this dimension.
2. the fact to have or not master data doesn't apply. Your fact table is booked with DIMIDs as keys of the table. Every time you book a record, the system will check in the dimension tables if this combination of characteristics values have already one record in their DIM, if yes, fine, nothing to do . If a new combination comes, the system will have to add a record in the dimension, thus it will first look for the next number range value (= DIMID).
In adition, the system will create master data IDs as well (even if there is no master data). In each Dimension table you'll find the corresponding master data SIDs for each of the IObjs belonging to the Dimension.
Tha's why filling empty cubes takes much more time than loading a cube with data already. That's also why the more you load data the less time it takes.
Please also make sure that all your F table indexes are dropped. (manage cube, performance tab, delete indexes prior loading).
this will help considerably initial loads...
Understanding these concepts of BW datawarehousing are of paramount importance in order to set the system up properly.
Message was edited by:
Olivier Cora

Data load is very slow

Hi Experts,
I am working on CRM Analytic. I am loading address data from extractor 0BP_DEF_ADDRESS_ATTR to Business partner, with 19 lakhs records. When I execute the DTP it is taking 3 to 4 days to complete the load.
Please provide me solution so that my data load will become fast.
With Regards,
Avenai

Hi,
Increase the Number of parallel processes.
in order to increase parallel processes --> from menu of DTP -> goto -> "settings for batch manager" -> increase the Number of parallel processes (by default it will be 3 increase it to 6).
Increase the Datapacket size in the DTp -extraction tab.
Do you have any routines used in the transfromations? if yes try to debug the code where its taking time, find n try to fine tune the code with the help of ABAP person.
Below option may also be one of the reasons while using CRM data sources...
The data source consists of lots of fields which are not used or mapped in the transformation ... try to hide those fields or you can create a copy of your data source using BWA1 transaction in CRM system.
Regards
KP

Data Load slow on some days

Similar Messages

Maybe you are looking for