Query performance: DSO or Cube?

For a new report (two queries) we will receive a few 100.000 service messages pushed into a PSA. These messages have various timestamps for different statuses of the items that come in. The client would like to report on very short term (5-15 minutes near real-time) on a subset of the data, and selecting on the date and time.
The idea is as follows. From the PSA we load a subset of the fields to a DSO on real-time basis, because the business has a need for operational reporting on very short basis. This DSO should only have data for max a few days, so the size won't grow over a million. Different messages on the same unique key of the item can come in. On this DSO, we build a simple query.
The other DSO, which gets the data from the PSA with a regular non-real-time DTP (or maybe also real-time? since it should refresh every few hours) contains more fields and has several timestamps on which the client wants to apply selections. We want to build a more complex query on this DSO. This DSo will accumulate to millions of records on a weekly basis.
Question is now: should we apply the use of a query directly on the large DSO (with indices on the selection fields), or build a cube on top? Many fields are timestamps and therefore not suitable as a dimension (maybe line item). However, there are some characteristics that do not have many different values (like 0/1 for true/false selections). Only data from the same PSA is used, no derivations from other data or the use of master data is involved. The timestamp for the
I'm wondering if a cube has advantages over a DSO here. I haven't found clear documentation explaining if it's better to use a DSO or cube. Any ideas?
Edited by: Zephania Wilder on Jan 18, 2010 5:54 PM

Zephania,
The answer to your question depends on how the report is run. I didn't understand clearly, but it looks like you said the client will filter using multiple timestamp fields.
If you can determine what the selection screen filter fields are and if you can make sure to make those field mandatory, then you can just create index on those fields and just report from DSO.
If you can't determine the search criteria, want to give full freedom to the users, but users don't mind waiting when the database performs full table scans, then you may report from DSO. Basis may not like that, though.
If you want to give full freedom to users, and the report should run quickly, then you have to build an InfoCube and report from that.

Similar Messages

  • Query performance on Inventory Cube

    Hi All,
            I have a query on Inventory Cube with non cumulative key figures, when I ran a query with them its taking 60 to 70 minutes. When I ran the same query by removing non cumulatives its displaing results in 25 seconds. Is there any way we can improve query performance which is effected  by non cumulative keyfigures.
        I have checked the performance related tools like RSRV on cube and master data no errors, in RSRT> execute debug the more time query consumes in data manager, ST03> DB and data manager time and also unassigned time is more.
        I know that query consumes time because of non cumulative keyfigures as it need to perform calculations on fly but its taking lot more than that. I apprecate your inputs to this query in advance.
      I will reward points.
    Regards
    Satish Reddy

    Hi Anil,
        Its nice to see you. We have compressed the cube with marker update and we are using only two infosources to the cube(BF and UM). As there are 150 queries on that cube I don't want to build aggregate especially for that query. I also treid DB stats refresh, there is a process chain to delete and recreate indexes, analysed cube and master data in RSRV etc. it didn't really helped me. Would you please suggest any good solution for this. I apprecaite it in advance.
    When i check in Application log in Cube Manage it is displayed that Mass Upsert of Markers update so I assumed that markers are updated.
    Regards
    Satish Arra.

  • MDX query performance on ASO cube with dynamic members

    We have an ASO cube and we are using MDX queries to extract data from that cube. We are doing some performance testing on the MDX data extract.
    Recently we made around 15-20 account dimension members dynamic in the ASO cube, and it is taking around 1 and a half hour for the query to run on an empty cube. Earlier the query was running in 1 minute on the empty cube when there were no dynamic members in the cube.
    Am not clear why it takes so much time to extract data from MDX on an empty cube when there is nothing to extract. The performance has also degraded while extracting data on the cube with data in it.
    Does dynamic members in the outline affect the MDX performance? Is there a way to exclude dynamic members from the MDX extract?
    I appreciate any insights on this issue.

    I guess it depends on what the formulas of those members in the dynamic hierarchy are doing.
    As an extreme example, I can write a member formula to count every unique member combination in the cube and assign it to multiple members, regardless if I have any data in the database or not, that function is going to resolve itself when you query it and it is going to take a lot of time. You are probably somewhere in between there and a simple function that doesn't require any over head. So without seeing the MDX it is hard to say what about it might be causing an issue.
    As far as excluding members there are various function in MDX to narrow down the set you are querying
    Filter(), Contains(), Except(), Is(), Subset(), UDA(), etc.
    Keep in mind you did not make members dynamic, you made a hierarchy dynamic, that is not the same thing and it does impact the way Essbase internally optimizes the database based on Stored vs dynamic hierarchies. So that alone can have an impact as well.

  • Query performance on virtual cube

    Hello,
    We have this requirement wherein we are switching a infocube with a virtual one to reduce BWA space. As part of this activity I am carrying out performance testing on existing queries. For most queries I can see an increase of around 20% which is acceptable. But for one query, we are running into issue where the query is getting timed out after hogging a lot of memory. On debugging I could see that this happens inside RSDRC_INFOPROV_READ ->run_query_handler wherein it ends up doing sequential read on SID table (0BILL_NUM). Strangely bill_num is neither part of the rows/free characteristics nor is it used anyplace else (filters,restrictions). Any pointers to what might be going wrong will be highly appreciated. Thanks.
    Regards,
    Dhrubo

    Hi,
    In this case try to find where more time is utilized by using the ST03.
    if there is more time consuming in front end, rearrange the query by using less in the row and using more free chars and filter areas. using the variable also will work.
    use reporting agent and schedule the query in background to fill the cache and rerun the query.
    Reg,
    Vishwa

  • Query Performance Issues on a cube sized 64GB.

    Hi,
    We have a non-time based cube whose size is 64 GB . Effectively, I can't use time dimension for partitioning. The transaction table has ~ 850 million records. We have 20+ dimensions among which 2 of the dimensions have 50 million records.
    I have equally distributed the fact table records among 60 partitions. Each partition size is around 900 MB.
    The processing of the cube is not an issue as it completes in 3.5 hours. The issue is with the query performance of the cube.
    When an MDX query is submitted, unfortunately, in majority of the cases the storage engine has to scan all the partitions (as our cube  is not time dependent and we can't find a suitable dimension that will fit the bill to partition measure group based
    on it.)
    I'm aware of the cache warming and  usage based aggregation(UBO) techniques.
    However, the cube is available for users to perform adhoc queries and hence the benefits of cache warming and UBO may cease to contribute to the performance gain as there is a high probability that each user may look at the data from different perspectives
    (especially when we have 20 + dimensions) as day(s) progress.
    Also, we have 15 + average calculations (calculated measures) in the cube. So, the storage engine sends all the granular data that the formula engine might have requested (could be millions of rows) and then perform the average calculation.
    A look at the profiler suggested that considerable amount of time has been spent by storage engine to gather the records (from 60 partitions).
    FYI - Our server has RAM 32 GB and 8 cores  and it is exclusive to Analysis Services.
    I would appreciate comments from anyone who has worked on a large cube that is not time dependent and the steps they took to improve the adhoc query performance for the users.
    Thanks
    CoolP

    Hello CoolP,
    Here is a good articles regarding how to tuning query performance in SSAS, please see:
    Analysis Services Query Performance Top 10 Best Practices:
    http://technet.microsoft.com/en-us/library/cc966527.aspx
    Hope you can find some helpful clues to tuning your SSAS Server query performance. Moreover, there are two ways to improve the query response time for an increasing number of end-users:
    Adding more power to the existing server (scale up)
    Distributing the load among several small servers (scale out)
    For detail information, please see:
    http://technet.microsoft.com/en-us/library/cc966449.aspx
    Regards,
    Elvis Long
    TechNet Community Support

  • Impact of real time cube on query performance and OLAP cache

    Hi:
    We have actual and plan cubes both setup as real time cubes (only plan cube is being planned against, not actual cube) and both cubes are compressed once a day.
    We are planning on implementing BIA accelerator and have questions related to query performance optimization:
    1/ Is there are any query performance benefits in changing the actual cube to be a basic cube (using program SAP_CONVERT_NORMAL_TRANS) if the F table is fully compressed
    2/ Can OLAP cache be leveraged for the queries run against the real time cubes e.g. actual cubes
    3/ What is the impact on BIA of having the actual cube as real time (whetehr or not there is data being loaded/planned during the day in that cube)
    Thank you in advance,
    Catherine

    1) Is there are any query performance benefits in changing the actual cube to be a basic cube (using program SAP_CONVERT_NORMAL_TRANS) if the F table is fully compressed
    From the performance point of view, tha actual cubes i.e std cubes are relatively better.
    2) Yes OLAP cache can be leveraged for bringing up the plan query but all the calculations are done in the planning buffer.
    3) Not sure.

  • Building a new Cube Vs Restricted Key figure in Query - Performance issue

    Hi,
    I have a requirement to create  a OPEX restricted key figure in Query. The problem is that the key figure should be restricted to about 30 GL Accounts and almost 300 Cost centers.
    I do not know if this might cause performance issue in the query. At the moment, I am thinking of creating a new OPEX cube and load only those 30 GL Accounts, 300 cost  centers and Amount. and  include OPEX  in multiprovider in order to get OPEX
    amount in the report.
    whats the best solution - creating OPEX restricted key figure or OPEX cube ?
    thanks,
    Bhat

    I think you should go for cube as all the restrcited key figure are calculated at OLAP runtime so it will definitely affect the query performance.There are a lot of costcenter for which you have to restrict it so definitely during the runtime of query it will take a lot of time to fetch tha data from infoprovider.Its better that you create a cube with the restrictions and include it in MP.It will definitely save a lot of time during query execution
    Edited by: shyam agarwal on Feb 29, 2012 10:26 AM

  • Source system - DSO-1 - DSO-2 - Cube (Performance improvement)

    Hello ppl,
    If I have a data flow like written below:
    Source system --> DSO-1 --> DSO-2 --> Cube
    Situation is:
    1.     Data load from source system to DSO-1 is full and can not be changed to delta itu2019s the restriction.
    2.     Amount of data flowing in DSO-1 is in millions daily (because it is full and growing).
    3.     Currently all DTPs in between source system and cube are maintained as full.
    Requirement is to improve performance, is it possible to make DTPs as Delta after DSO-1. Since I am deleting all records from DSO-1 and its PSA daily not sure if it will work? If yes how ?
    Any suggessionsu2026
    Regards,
    Akshay

    DTPs  delta is -->request based delta from write optimized DSO.The DTP should have the delta option selected.
    DTPs delta is -->request based delta only also for standard DSO.
    You will find initialization option in DTP on the execution tab(select DTP as type delta in extraction tab).
    Now since you will be doing init from DSO so system will also give you option to select table of DSO for extraction:
    i.e. Active/Changelog.
    You can select Active if you are using Write optimized DSO and Changelog if you are using standard DSO for DSO 1.
    Now coming bact to your case,using write optimized DSO will definately enhance the performance  as you don't have to activate the request..but I have recently had a very bad experience with write optimized DSO regarding delta.
    So I dont' trust write optimized DSO for pulling delta from it.
    if you want to use it ,test it as much possible before using it . I would still go with standard(the beauty of changelog table).
    Now regarding DTP ,once you load the the data from the DSO you can load init information even after the load.Else load the first request from dtp w/o data transfer for init and then run deltas..both will work fine.Its a similar option present in infopackages called "init without data transfer".
    Refer this thread ,a really nice discussion in this :
    BI7: "Delta Init. Extraction From..." under DTP Extraction tab?

  • Query performance on remote/virtual cube

    All:
    We are on BW 3.5.
    Can anyone suggest anything about improving query performance on remote/virtual cubes? Analysis shows query performance is sufferring at the database level.
    I am looking for some advise beyond hardware and database parameters. It seems current hardware and database parametrs work fine with basis cubes.
    Another solution is datamart.But can anything be done before/without going to datamart?
    Any help will be appreciated.
    Thanks

    Hi,
    In this case try to find where more time is utilized by using the ST03.
    if there is more time consuming in front end, rearrange the query by using less in the row and using more free chars and filter areas. using the variable also will work.
    use reporting agent and schedule the query in background to fill the cache and rerun the query.
    Reg,
    Vishwa

  • Query performance on Multiprovider(Remote Cube)

    Hi All,
    I have to increase the query performance for a report which built on Multi provider.
    This multiprovide designed from several remote cubes,but for this report data will bring through one remote cube from R/3.
    In filter i had one remote cube, which bring data from R/3.
    Now in ST03 the stats are like
    %init Time - 0, %DB time - 0, %OLAP time - 16.67, %Front end - 83.33.
    Now i have to improve the %Front end lapsed time.
    Could you please guide me.
    Thanks
    Srinivas

    Hi Srinivas,
    Please see this document
    https://websmp105.sap-ag.de/~sapidb/011000358700001394912002
    And this Discussion Thread
    Re: Deactivate Hierarchy symbols in excel
    See whether this is helpful in case of Remote Cubes.
    Thanks
    CK

  • How to improve query performance using infoset

    I create one infoset that including 4 char.and 3 DSO which all are time-dependent.When query run, system show very poor perfomance, sometimes no data show in BEX anayzer. In this case I have to close BEX analyzer at first and then open it again, after that it show real results. It seems very strange. Does anybody has experience on infoset performance improvement. pls info, thanks!

    Hi
    As info set itself doesn't have any data so it improves Performance
    also go through the below tips.
    Find the query Run-time
    where to find the query Run-time ?
    557870 'FAQ BW Query Performance'
    130696 - Performance trace in BW
    This info may be helpful.
    General tips
    Using aggregates and compression.
    Using less and complex cell definitions if possible.
    1. Avoid using too many nav. attr
    2. Avoid RKF and CKF
    3. Many chars in row.
    By using T-codes ST03 or ST03N
    Go to transaction ST03 > switch to expert mode > from left side menu > and there in system load history and distribution for a particular day > check query execution time.
    Statistical Records Part 4: How to read ST03N datasets from DB in NW2004
    How to read ST03N datasets from DB
    Try table rsddstats to get the statistics
    Using cache memory will decrease the loading time of the report.
    Run reporting agent at night and sending results to email. This will ensure use of OLAP cache. So later report execution will retrieve the result faster from the OLAP cache.
    Also try
    1. Use different parameters in ST03 to see the two important parameters aggregation ratio and records transferred to F/E to DB selected.
    2. Use the program SAP_INFOCUBE_DESIGNS (Performance of BW infocubes) to see the aggregation ratio for the cube. If the cube does not appear in the list of this report, try to run RSRV checks on the cube and aggregates.
    Go to SE38 > Run the program SAP_INFOCUBE_DESIGNS
    It will shown dimension Vs Fact tables Size in percent.If you mean speed of queries on a cube as performance metric of cube,measure query runtime.
    3. To check the performance of the aggregates,see the columns valuation and usage in aggregates.
    Open the Aggregates...and observe VALUATION and USAGE columns.
    "---" sign is the valuation of the aggregate. You can say -3 is the valuation of the aggregate design and usage. ++ means that its compression is good and access is also more (in effect, performance is good). If you check its compression ratio, it must be good. -- means the compression ratio is not so good and access is also not so good (performance is not so good).The more is the positives...more is useful the aggregate and more it satisfies the number of queries. The greater the number of minus signs, the worse the evaluation of the aggregate. The larger the number of plus signs, the better the evaluation of the aggregate.
    if "-----" then it means it just an overhead. Aggregate can potentially be deleted and "+++++" means Aggregate is potentially very useful.
    In valuation column,if there are more positive sign it means that the aggregate performance is good and it is useful to have this aggregate.But if it has more negative sign it means we need not better use that aggregate.
    In usage column,we will come to know how far the aggregate has been used in query.
    Thus we can check the performance of the aggregate.
    Refer.
    http://help.sap.com/saphelp_nw70/helpdata/en/b8/23813b310c4a0ee10000000a114084/content.htm
    http://help.sap.com/saphelp_nw70/helpdata/en/60/f0fb411e255f24e10000000a1550b0/frameset.htm
    performance ISSUE related to AGGREGATE
    Note 356732 - Performance Tuning for Queries with Aggregates
    Note 166433 - Options for finding aggregates (find optimal aggregates for an InfoCube)
    4. Run your query in RSRT and run the query in the debug mode. Select "Display Aggregates Found" and "Do not use cache" in the debug mode. This will tell you if it hit any aggregates while running. If it does not show any aggregates, you might want to redesign your aggregates for the query.
    Also your query performance can depend upon criteria and since you have given selection only on one infoprovider...just check if you are selecting huge amount of data in the report
    Check for the query read mode in RSRT.(whether its A,X or H)..advisable read mode is X.
    5. In BI 7 statistics need to be activated for ST03 and BI admin cockpit to work.
    By implementing BW Statistics Business Content - you need to install, feed data and through ready made reports which for analysis.
    http://help.sap.com/saphelp_nw70/helpdata/en/26/4bc0417951d117e10000000a155106/frameset.htm
    /people/vikash.agrawal/blog/2006/04/17/query-performance-150-is-aggregates-the-way-out-for-me
    https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/1955ba90-0201-0010-d3aa-8b2a4ef6bbb2
    https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/ce7fb368-0601-0010-64ba-fadc985a1f94
    http://help.sap.com/saphelp_nw04/helpdata/en/c1/0dbf65e04311d286d6006008b32e84/frameset.htm
    You can go to T-Code DB20 which gives you all the performance related information like
    Partitions
    Databases
    Schemas
    Buffer Pools
    Tablespaces etc
    use tool RSDDK_CHECK_AGGREGATE in se38 to check for the corrupt aggregates
    If aggregates contain incorrect data, you must regenerate them.
    202469 - Using aggregate check tool
    Note 646402 - Programs for checking aggregates (as of BW 3.0B SP15)
    You can find out whether an aggregate is usefull or useless you can find out through a proccess of checking the tables RSDDSTATAGGRDEF*
    Run the query in RSRT with statistics execute and come back you will get STATUID... copy this and check in the table...
    This gives you exactly which infoobjects it's hitting, if any one of the object is missing it's useless aggregate.
    6
    Check SE11 > table RSDDAGGRDIR . You can find the last callup in the table.
    Generate Report in RSRT
    https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/cccad390-0201-0010-5093-fd9ec8157802
    https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/4c0ab590-0201-0010-bd9a-8332d8b4f09c
    Business Intelligence Journal Improving Query Performance in Data Warehouses
    http://www.tdwi.org/Publications/BIJournal/display.aspx?ID=7891
    Achieving BI Query Performance Building Business Intelligence
    http://www.dmreview.com/issues/20051001/1038109-1.html
    Assign points if useful
    Cheers
    SM

  • Problem with query performance

    Hi All,
    While loading data if we run the report, will it make any difference in the query performance? I have a cube with 230 million records  I am running the delta which has got another 230 million records. Now I am trying to run the queries on the cube, all those timed out.
    But I have a DSO with the 480 million the same queries are running good on the DSO. But I wanted to run the reports on the cube not on DSO what can I do? Is the data load giving any problem in the query runtime??
    Pleas advise me what to do?
    Regards
    Kiran

    Hi,
    My load got finished, Now i created indexes and tried to run the query, every query on this cube times out.. but on the top of DSO we have Same queries those queries are running.. what could be the reason how can i go ahead to improve the query performance plzzz advise me
    Kiran
    Edited by: kiran kumar on Mar 14, 2009 1:28 AM

  • New field added to cube, delta DTP from DSO to cube is failing

    Dear all,
    Scenerio in BI 7.0 is;
    data source -delta IPs-> DSO -
    delta DTP---> Cube.
    Data load using proces chain daily.
    We added new field to Cube --> transformation from DSO to cube is active, transport was successful.
    Now, delta from DSO to cube is failing.
    Error is: Dereferencing of the NULL reference,
    Error while extracting from source <DSO name>
    Inconsistent input parameter (parameter: Fieldname, value DATAPAKID)
    my conclusion, system is unable to load delta due to new field. And it wants us to initialize it again ( am i right ?)
    Do I have only one choice of deleting data from cube & perform init dtp again ? or any other way ?
    Thanks in advance!
    Regards,
    Akshay Harshe

    Hi Durgesh / Murli,
    Thanks for quick response.
    @ durgesh: we have mapped existing DSO field to a new field in cube. So yes in DTP I can see the field in filter. So I have to do re-init.
    @ Murli: everything is active.
    Actully there are further complications as the cube has many more sources, so wanted to avoid seletive deletion.
    Regards,
    Akshay

  • Unable to load data from DSO to Cube

    Good morning all,
    I was trying to load data from DSO to Cube for validation. Before loading the new data, I deleted all from DSO and Cube. They contain no request at all. Cube has "Delta Update". First DSO was loaded 138,300 records successfully. Then Activated the DSO. The last when I clicked Exectue (DSO --> Cube), it loaded 0 record. I was able to load the data yesterday. What might be the reasons for this situation?
    Thank you so much!

    Hi BI User,
    For loading delta upload into the data target, there should be an initialization request in the data target with same selection criteria.
    So..first do the initialization and perform delta upload into the cube.
    Regards,
    Subhashini.

  • Semantic Partitioning Delta issue while load data from DSO to Cube -BW 7.3

    Hi All,
    We have created the semantic partion with the help of BADI to perform Partitions. Everthing looks good,
    first time i have loaded the data it is full update.
    Second time i init the data load. I pulled the delta  from DSO to Cube . DSO is standard where as Cube is partition with the help of semantic partition. What i can see is the records which are updated in Latest delta are shown into the report rest all are ignored by the system.
    I tried compression but still it did not worked.
    Can some has face this kind
    Thanks

    Yaseen & Soujanya,
    It is very hard to guess the problem with the amount of information you have provided.
    - What do you mean by cube is not being loaded? No records are extracted from DSO? Is the load completing with 0 records?
    - How is data loaded from DSO to InfoCube? Full? Are you first loading to PSA or not?
    - Is there data already in the InfoCube?
    - Is there change log data for DSO or did someone delete all the PSA data?
    Sincere there are so many reasons for the behavior you are witnessing, your best option is to approach your resident expert.
    Good luck.
    Sudhi Karkada
    <a href="http://main.nationalmssociety.org/site/TR/Bike/TXHBikeEvents?px=5888378&pg=personal&fr_id=10222">Biking for MS Relief</a>

Maybe you are looking for