Will block size effect the calc script performance?

Hi Experts,
I have a cube called RCI_LA:RCI_LA, now I have created calc scripts and working fine. But those calc scripts are taking too much time than expected (normally it should not take more than 15 min but those are taking nearly 1 hr or more some calc scripts.)
In database properties I found that block size is 155896 B i.e. 152.KB but this size should be 8 to 100 KB & Block density is 0.72%
If block size exceeds more than 100 KB will it impact the performance of Calc scripts?
I think answer to the above question is “yes”. In this case what should I need to do to improve calc scripts performance?
Could you please share your experience here with me to come out of this problem?
Thanks in advance.
Ram

I believe Sandeep was trying to say "Dynamic" rather than "Intelligent".
The ideal block size is a factor in all calcs, but the contributing reasons are many (The main three are CPU caching, Data I/O overhead, Index I/O overhead).
Generally speaking, the ideal block size is achieved when you can minimize the combination of Data I/O overhead and Index I/O overhead. For this reason a block size that is too large will incur too much Data I/O, while a block size that is too small will incur too much Index I/O. If your Index file is small, increasing your block size may help, although the commonly acceptible block size is between 8K and 64K in size, this is just a guideline.
In other words, if you test it with something right in the middle and your index file is tiny, you might want to test it with a smaller block size. If your index file is very large (i.e. 400 MB or more), you may want to increase the block size and retest.
Ways to increase/decrease it are also many. Obviously, changing the dense/sparse settings is the main way, but there are some considerations that make this a touchy process. Other ways are to use dynamic calc in the dense dimensions. I say start at the top of your smallest dense dimension and keep the number of DIMENSIONS that you use D-C on limited. Using D-C members in a dense dimension does NOT increase the index file, so it could be considered a "free" reduction in block size -- the penulty is paid on the retrieve side (there is no free ride).

Similar Messages

Outline Order, Calc Script Performance, Substitution Variables

Hi All,
I am currently looking in to the performance side.
This is mainly about the calculation script performance.
There are lot of questions in my mind and as it is said you can get the results only by testing.
1. Outline order should be from least sparse to most sparse
(other reason : to accomodate as many sparse members in to calculator cache) correct me if I am wrong
2. Is Index entry created based on the outline order. For example I have outline order as Scenarios, Products, Markets then does my index entry be like scenario -> Products -> Markets ?
3. Does this order has to match with the order of members in FIX Statement of calculation script?
4. I have 3 sparse dimensions. P (150 members), M (8 members), V (20 members).
I use substitution variables for these three in the calculation script. And these three are the mandotary things in my calculation script. Now when I see the fix statement, these three are the first 3 parameters of the fix statemtn and since I am fixing on a specific member, placing these three members as the first 3 sparse dimensions in the outline, ill it improve performance?
In one way, I can say that a member from P, M,V becomes my key for the data.
Theoritically if I think, may be it will...but in practical terms I don't see any of such thing.. Correct me If my thinking is wrong.
One more thing, I have a calc script with say around 10 FIX statements and this P,M,V is being used in every FIX statemnts. Since my entire calculation will be only on one P, one M, one V. Can I put everything in one FIX at the beginning and exclude it from remaining FIX statememts?
5. I have a lot of cross dimensional operations in my calc scripts for accounts dimension (500 + ) members.
Is there a way to reduce these?
6. My cube statistics..
Cube size : 80 GB +
Block Size : 18 KB (Approx)
Block density : 0.03 . This is what I am more worried about. This really hurts me.
This is one of the reason why my calculation time is > 7 hours and some times it is horrible when there is huge amount of data (it takes aound 20 + hours) for calculation.
I would be looking forward for your suggestions.
It would be really apprecialble if It is Ok to share your contact number so that I can get in touch with you. That could be of great help from your side.

I have provided some answers below:
There are lot of questions in my mind and as it is said you can get the results only by testing.
----------------------------You are absolutely right here but it helps to understand the underlying principles and best practices as you seem to understand.
1. Outline order should be from least sparse to most sparse
(other reason : to accomodate as many sparse members in to calculator cache) correct me if I am wrong
----------------------------This is one reason but another is to manage disk I/O during calculations. Especially when performing the intial calculation of a cube, the order of sparse dimensions from smallest to largest will measurably affect your calc times. There is another consideration here though. The smallest to largest (or least to most) sparse dimension argument assumes single threading of the calculations. You can gain improvements in calc time by multi-threading. Essbase will be able to make more effective use of multi-threading if the non-aggregating sparse dimensions are at the end of the outline.
2. Is Index entry created based on the outline order. For example I have outline order as Scenarios, Products, Markets then does my index entry be like scenario -> Products -> Markets ?
----------------------------Index entry or block numbering is indeed based on outline order. However, you do not have to put the members in a cross-dimensional expression in the same order.
3. Does this order has to match with the order of members in FIX Statement of calculation script?
----------------------------No it does not.
4. I have 3 sparse dimensions. P (150 members), M (8 members), V (20 members).
I use substitution variables for these three in the calculation script. And these three are the mandotary things in my calculation script. Now when I see the fix statement, these three are the first 3 parameters of the fix statemtn and since I am fixing on a specific member, placing these three members as the first 3 sparse dimensions in the outline, ill it improve performance?
--------------------------This will not necessarily improve performance in and of itself.
In one way, I can say that a member from P, M,V becomes my key for the data.
Theoritically if I think, may be it will...but in practical terms I don't see any of such thing.. Correct me If my thinking is wrong.
One more thing, I have a calc script with say around 10 FIX statements and this P,M,V is being used in every FIX statemnts. Since my entire calculation will be only on one P, one M, one V. Can I put everything in one FIX at the beginning and exclude it from remaining FIX statememts?
--------------------------You would be well advised to do this and it would almost certainly improve performance. WARNING: There may be a reason for the multiple fix statements. Each fix statement is one pass on all of the blocks of the cube. If the calculation requires certain operations to happen before others, you may have to live with the multiple fix statements. A common example of this would be calculating totals in one pass and then allocating those totals in another pass. The allocation often cannot properly happen in one pass.
5. I have a lot of cross dimensional operations in my calc scripts for accounts dimension (500 + ) members.
Is there a way to reduce these?
-------------------------Without knowing more about the application, there is no way of knowing. Knowledge is power. You may want to look into taking the Calculate Databases class. It is a two day class that could help you gain a better understanding of the underlying calculation principles of Essbase.
6. My cube statistics..
Cube size : 80 GB +
Block Size : 18 KB (Approx)
Block density : 0.03 . This is what I am more worried about. This really hurts me.
This is one of the reason why my calculation time is > 7 hours and some times it is horrible when there is huge amount of data (it takes aound 20 + hours) for calculation.
------------------------Your cube size is large and block density is quite low but there are too many other factors to consider to simply say that you should make changes based solely on these parameters. Too often we get focused on block density and ignore other factors. (To use an analogy from current events, this would be like making a decision on which car to buy solely based on gas mileage. You could do that but then how do you fit all four kids into the sub-compact you just bought?)
Hope this helps.
Brian

Essbase calc script performance issues

Hi,
I have essbase 9.3 running on Sun solaris 4 CPU, 16 GB server. The calc script "calc all" takes ~3 hrs to complete.
This is the calc script.
/ESS_LOCALE English_UnitedStates.US-ASCII@Binary
SET UPDATECALC OFF;
SET CALCPARALLEL 4;
SET CALCTASKDIMS 2;
CALC ALL;
We don't have to calc all dim, but even if we
But even with specific dim we get the same timing. Below is the script
SET UPDATECALC OFF;
SET CALCPARALLEL 4;
SET CALCTASKDIMS 2;
FIX ("Y2009", "Actual");
CALC DIM("Data Source","Intercompany","LegalEntity","Site","Department","Entity");
ENDFIX
The ess00001.ind is 700 Mb and ess00001.pag is 2.1 GB.
In Admin services, this is what I see for caches
1) Index cache size is 1 GB for this DB
2) Index cache current value is 1gb
3) Datafile cache setting is 1.5 GB
4) Datafile cache current value is 0 (?? not sure why??)
5) Data cache setting 4.1 GB
6) Index page setting 8 kb
please help ...
Thanks
Moe

Moe,
I'm guessing you inherited this thing, else you would know why the cache settings are what they are, but here are some thoughts:
Caches:
3) Datafile cache setting is 1.5 GB
4) Datafile cache current value is 0 (?? not sure why??)You're running the database in Buffered I/O, so the data file cache is ignored.
1) Index cache size is 1 GB for this DB
2) Index cache current value is 1gb You have consumed all of the cache -- I'm a little confused, as you state your .ind file to be 700 megabytes -- generally the index cache consumption doesn't go beyond the .ind file size. When you look at your hit ratio statistics in EAS, does it show a 1 against the index cache? If yes, then you don't need to look any further as that's as good as it's going to get.
5) Data cache setting 4.1 GBUnless you're using MEMSCALINGFACTOR, I don't think Essbase is actually addressing all of the memory you've assigned. What are you showing as actually used? In any case, having a data cache almost twice as big as the .pag files is a waste as it's way too large.
Easy, off the cuff suggestions without knowing more about your db:
1) Try AGG instead of CALC DIM for sparse dimensions.
2) Try turning off (yes, turning off, you'd be surprised) parallel calc, and benchmark it. It will probably be slower, but it's nice to know.
3) Dimension order? Modified hourglass?
4) Tried defragmenting the database and benchmarking the performance?
5) What is your block size? Big? Small?
6) I think you are not calculating your Accounts/Measures dimension in your calc? If you are, and it's dense, could you make those Accounts dynamic calc -- dropping a dimension from the calc can be huge.
I'm sure there will be other suggestions -- these are the easiest.
Regards,
Cameron Lackpour

Calc Script performance

 Hello, A customer have a cube that is taking a lot longer to calculate after each new load. The cube have 7 dimensions, monthly data from Jan 2005 on, 20 GB of data. It's taking around 14 hours to calculate it, but if you load the data on an identical cube with no data, it is calculated in less than 2 hours. The calc scripts include a FIX on a dense dimension, as shown below: Fix( &CurrentYear, &CurrentMonth, Actual, Local) <--- sparse dims Fix (@IDescendants("REVENUE"), "Qtd VP Interna") <--- dense dim members (Accounts) Calc Dim (Presidencia, Product); <--- sparse dims EndFix EndFix The question is: since FIXing on a dense dimension causes all data blocks to be touched, is the inner FIX causing a scan in all data blocks of the database, even if the outer FIX refers to sparse dims only? And during the calc process, the Windows performance monitor shows very little CPU activity and only accasionally a disk reading... And since Calc Dim is not allowed within an IF command, is there another way to obtain that consolidation? Thanks in advance! 

 Hello Gary! I agree that calculating a new month's data in an empty cube should be faster than calculation the same data in a cube that already have 16 months of data, but I think that it's taking much longer that expected. I expected it to be 50% slower, but not 700% ! I even recreated the production cube from scratch, loading and calculating one month at a time, in 2 different servers. The results are always the same: the new calc time is a lot longer the previous one. And when I use the windows' performance monitor to compare the server's behavior between the calcs of the empty cube an the production one, you can see that the server is either acessing the hard disk or is calculating 100% of the time for the empty cube, but the graphs for the production cube indicates very low disk access and CPU activities. It seems to be waiting for something... I have already made many configuration changes, such as resizing the Index, data and data-file caches (I'm using direct i/o), number of lock blocks, compression mode among others, but the performance gains obtained for the calc in the empty cube is not reflected for the production cube, maybe because it's (apparently) doing nothing most of the time... Is there a trace I can use to check what the ESSBASE is doing during the calc? I have used the MSG Detail but this didn't help. Thank you for your help!

What is the procedure/code in VBA for passing the calc scripts dynamically based on the selection.

Hello Gurus,
I want to know what is the procedure/code in VBA for passing the calc scripts dynamically based on the selection.
For example:
X=EssVCalculate("Sheetname","Calc_Script name",True)
In the above code instead of the *"Calc_Script name"* I want a script which is called dynamically and the values are calculated accordingly.
Thanks in advance
Saurabh

Hi Todd,
This is the situation:
I have a calc script in Essbase which I can call to perform the calculations on the current sheet that is retrieved. I want calculation for the following formulas:
x = EssVCalculate("Sheet2", "CalcBC", False)
CalcBC is my calc script which is present in Essbase
So instead of passing the above script I want to pass the conditions dynamically in the VBA code . I don't want to mention the script name directly in EssVCalculate option
For example:
I have three drop down menus from which I would select three different( zero level )members. It would then retrieve the data for that particular values in the excel sheet and now when I click on Calculate button it should calculate the script dynamically.
I don't know how calc scripts can be executed dynamically in the VBA code itself.
Thanks in Advance
Saurabh

Why does the calc script screen freezes after executing the script

Hello Everyone:
When a calc script gets executed, the calc Script in the ASS freezes even though the calc script has finished it´s execution. Does anybody knows why? How can i prevent these to happen?
Hope someone could help me, thanks in advanced
Best Regards

if you execute the calc script in foreground, console will freezes and it will not allow any thing to do.
Once script is completed, console will be freed.
You can execute the calc script in back ground so that you can continue your work and you can track the progress in the sessions.

Can not run the calc scripts by mxl, if the first character is Number

I found that if you create the calc scrpt like 1abc.csc. You can not run it by mxl. I tried for times but do not know the reason for that. I found that you can use number in the name of the calc scripts(like abc1.csc, a1bc.csc), but not for the first character(like 1abc.csc).
Below is my mxl scripts:
×××××××××××××××××××××××××××××××××××
login admin identified by password on localhost;
spool on to 'C:\abc.log' ;
execute calculation sample.basic.1abc;
spool off;
logout;
×××××××××××××××××××××××××××××××××××
I write a bat file below:
×××××××××××××××××××××××××××××××××××
D:\Hyperion\AnalyticServices\bin\essmsh C:\abc.mxl
×××××××××××××××××××××××××××××××××××
I can run the scripts if I change the name "1abc" to "abc1"or "a1bc".
Of course I can run this calc in EAS, no matter the name is "1abc" or "abc1".
Anybody find the same problem with me?

I agree, if ever there was a "best practice" to developing scripts (especially when using variables) it is to enclose in single quotes. I actually use double quote, single quote.
execute calculation "'$4'"."'$5'"."'$6'";

My Essbase db corrupting when i terminate the calc script why?

Hi All,
i got the problem with essbase db..It is corrupting the data when i terminating the any running calc script.
So that every time i m taking the back up for that...
Please let me know how to avoid this...and why it is corrupting ?...
Is anything wrong with my Essbase?
Regards,
Prabhas

If by "corrupting" you mean that essbase crashes and you can't restart the db then yes this is unusual and something is wrong with the environment. On the other hand, if you mean that the data is not the same as it was prior to starting the script then no there is nothing wrong. Essbase does not have a rollback feature that undo's the calc script processed up to the point where you killed it.
Dave

Changing layout size of the Interactive Script in the Appl. Area

Hi Experts,
I need to change the standard size of the layout of the interactive script WinClient in the Apppl. Area. The screen has more space to increase the layout of the view of the Interactive Script but i don't know how to make this change.
Thanks for your help

hi!
create a variable window and write this code there
/: POSITION XORIGIN '2' CM YORIGIN '10' CM
/: SIZE WIDTH '18.05' CM HEIGHT '18' CM
/: BOX FRAME 10 TW
/: POSITION XORIGIN '2.05' CM YORIGIN '10.05' CM
/: BOX WIDTH '18' CM HEIGHT '1' CM INTENSITY 15
/: POSITION XORIGIN '3.5' CM YORIGIN '10' CM
/: BOX FRAME 10 TW WIDTH '0' TW HEIGHT '18' CM
/: POSITION XORIGIN '9.2' CM YORIGIN '10' CM
/: BOX FRAME 10 TW WIDTH '0' TW HEIGHT '18' CM
/: POSITION XORIGIN '11.5' CM YORIGIN '10' CM
/: BOX FRAME 10 TW WIDTH '0' TW HEIGHT '18' CM
then create a main window with same height and width as that of the varaible window (overlapping the variable window on to it).
now on looping in the main table and displaying the data boxes will come it self....
here the main window dimensions are
MAIN 00 Main Window 2,00 CM 10,00 CM 18,00 CM 18,00 CM
[then with the help of data element call the box
/: BOX WIDTH '18' CM HEIGHT '1' CM FRAME 10 TW
in loop where write_form is called....]
u will get the box with the data printed in the form......
keeping same for the variable window....
and try it....
this will be done....

Using VB API in VBA to edit the Calc Scripts

Guys,
Does anyone of you have VBA source code/examples utilizing Essbase API for the following operation.
I want to copy Calc Script from Server to the client, edit the Script using VBA and copy back the Script to the Server. Any kind of info is appreciated.
Thanks in advance
HYPuser

I posted some real sample code a while back for a completely different function (member renames).
Renaming Existing Member Names
So while you'll have to write the calc modification code yourself (or ask Glenn his rate ;-)), this at least shows you how the initialization / login / termination of the API would work. That's probably the most confusing aspect of using the VB API for the first time anyway.
All that said, if this is from Excel and the user will already be logged in via the Excel Add-In, I believe that you can make life much easier and grab the hCtx directly without any initialization / login/ termination worries. I probably should have done that myself in the example I gave. See Tim Tow's posts in this thread:
API
EDIT: Two things I just noticed - first, the code sample I gave even has an example of polling for ProcState, so that gets you closer than I thought! Second, remembered that the line that reads...
While ProcState.State ESB_STATE_DONE
...should actually have a 'less than / greater than' symbol between ProcState.State and ESB_STATE_DONE. Can't get the forum to display them correctly, even in code tags...
Edited by: TimG on May 16, 2011 2:55 PM

Fixing the calc script in FDM Planning adaptor

Hi All,
I have run into a issue with the load action in the planning adaptor ES9x-G4-A. By default, when the adaptor loads data into essbase it fixes using the script
                                   '*** CALC SCRIPT ***
                                        'NOTE: CALC SCRIPT CAN BE MODIFIED For Each CUSTOMER
                                        'Clear the intersection of (Category, Period, Entity)
                                        strClear = "Fix(""" & strTCat & """,@Idescendants(""" & strCurEnt & """))" & vbcrlf
                                        strClear = strClear & "CLEARDATA """ & strTPer(0) & """;" & vbcrlf
                                        strClear = strClear & "EndFix"
The issue with this is that it doesn't fix on year, meaning that data loaded with a replace will wipe the same period across every year. Does anyone know what the year variable is to set the fix correctly? I can't find it in any of the documentation, for 9.3.1 or 9.2
Thanks.
KMacintosh

Hi,
{color:#000000}Try updating the load script to be :-
{color}{color:#ff0000}strTargYear = API.POVMgr.fPeriodKey(CStr(strPer(0))).strTargetYear
'*** CALC SCRIPT ***
'NOTE: CALC SCRIPT CAN BE MODIFIED For Each CUSTOMER
'Clear the intersection of (Category, Period, Entity)
strClear = "Fix(""" & strTargYear & """,""" & strTCat & """,@Idescendants(""" & strCurEnt & """))" & vbcrlf
strClear = strClear & "CLEARDATA """ & strTper(0) & """;" & vbcrlf
strClear = strClear & "EndFix"
Cheers
John
[http://john-goodwin.blogspot.com/]{color}

Calc script & performance issues

Hi All we have a calc script which used to take only 10 mint every day. But today it is taking long time 4 hrs,,,stil running..if i cancel that calc operation what is the impact on database?. Earlier all users are hapy with speed...but suddenly every one got pissed off with speed...it is taking long time to retrieve data..Quickly what are the parameters I need to check?thanks in adavance..

If you are using committed access then you can safely cancel thecalculation. All data will be reverted back to what it was beforecalculation. However if you are using uncommitted access it isrecommended not to cancel any running operation.If you want to eliminate fragmentation, just export your level 0data and import again, and then do a calc all.Doing so will remove any fragmentation. It is recommended to dothis once in a while like 2 months to get rid offragmentation.

HT2559 Help with setting raid block size after the fact

I screwed up and created my raid 1 with block size set at 32. I need 256....it won't let me change it? What do I do? Do I delete and re-configure it?

thanks for the reply. I am editing huge photo files (HDR Pano's) off the drive. Doesn't that mean I need 256? Anyway, when I go to erase it, it says "Deleting a mirrored RAID set changes each of its slices into a partition that contains a complete copy of the data from the deleted RAID set". Is that a problem?

Pls check the calc script requirment

Pls check this calc reqirement and do the needful help
we have got 700000/- INR in your bank You can spend the money but there are rules of spending.
1. we can start withdrawing Rs 1000 only in each month.
2. The Money can be withdrawn only from November 2011
3. This facility is only valid upto next 7 months.
At the end of the seventh month your bank Pass book must show:
Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July
1 1 1 1 1 1 1
(1 = 1000 Rupees as required)
Note : Year changes here after December. Take care of this in logic.
You can think of standard dimensions a planning application has :
1. Accounts
2. Period
3. Scenario
4. Version
5. Entity
6. Year
7. And Many more if you wish

GG what do you think is the cause of the error of this code below which i used for testing.....
declare
v_alltables varchar2(10000);
begin
for rec in (select TABLE_NAME from all_tables where owner = 'APPS' AND ROWNUM =1)
loop
execute immediate 'SELECT DBMS_METADATA.GET_DDL( '|| '''TABLE'',' ||rec.table_name||',''APPS'') FROM DUAL' into v_alltables;
dbms_output.put_line(v_alltables);
end loop;
end;
it gives this error message when run:
Error report:
ORA-00904: "ADX_PRE_AUTOCONFIG": invalid identifier
ORA-06512: at line 7
00904. 00000 - "%s: invalid identifier"But when i change the rec.table_name into '''ADX_PRE_AUTOCONFIG''' the code works properly.
so i decided to have this code and it works.
SET SERVEROUTPUT ON
declare
v_alltables varchar2(10000);
begin
for rec in (select TABLE_NAME from all_tables where owner = 'APPS' AND ROWNUM =1)
loop
execute immediate 'SELECT DBMS_METADATA.GET_DDL( '|| '''TABLE'',' ||''''||rec.table_name||''''||',''APPS'') FROM DUAL' into v_alltables;
dbms_output.put_line(v_alltables);
end loop;
end;for you newbie, this must also work
SET SERVEROUTPUT ON
declare
v_alltables varchar2(10000);
begin
for i in (select table_name from all_tables where rownum<2)
loop
execute immediate 'SELECT DBMS_METADATA.GET_DDL( '|| '''TABLE'',' ||''''||i.table_name||''''||',''TRNG8305_GC'') FROM DUAL' into v_alltables;
dbms_output.put_line(v_alltables);
end loop;
end;hope it helps.....

Why the Calc Script runs 15 hours?

Every week, we empty the BSO essbase CUBE, and reload with new data from flat file and make a Calculation. It takes 15 hours to finish, that's too long a time, how to improve the running speed? What is the most possible reason for the bottleneck?

Hi,
1. One has to take a look at your outline , calculation scripts and also other DB settings to recommend any changes for improvement.
2. There are many parameters like parallel calculation, caches, cfg settings..etc which play a role in it.
3. We never know , it might be due to hardware issue sometimes.
Get essbase consultants to fix this issue .
Sandeep Reddy Enti
HCC
http://hyperionconsultancy.com/

Will block size effect the calc script performance?

Similar Messages

Maybe you are looking for