Essbase RAM Optimization/Tuning

Question

Hello,
I have a BSO application that is taking a heavy toll on RAM (starts with 5-6gb goes up to 13 GB and stays there until application is stopped). CPU performance is unaffected and stays less than 20%. There is a full load that takes place every 2hrs which clears, loads and calcs all the data. Sometime calculations take a long time (over an hour) that are usually done in 2-3 mins.
I have the below stats, can someone please let me know, what might be cause (or one of) from here?
Thanks
Dimension
Type
Members
Stored Members
Account
Dense
1778
1587
Period
Dense
190
14
Type
Sparse
9
6
Year
Sparse
7
6
Business Unit
Sparse
45
43
Scenario
Sparse
31
13
Product Code
Sparse
103
59
Department
Sparse
185
162
Operating Unit
Sparse
1194
836
Stats
Statistics
Number of
Number of existing blocks
2,100,034
Block Size
17724 (B)
Potential Number of Blocks
464,511,290,880
Existing level 0 blocks
63,500
Existing upper level blocks
2,036,534
Block Density
.83
Compression ratio
.01
Average Clustering Ratio
.48
Hit Ratio on index cache
1.00
Hit Ratio on data cache
.42
Hit Ratio non data file cache
.89
File sizes
File
size
Data file - ess00001.pag
2,097,144 kb
Data file - ess00002.pag
2,097,144 kb
Data file - ess00003.pag
736,008 kb
Index file - ess00001.ind
184,024 kb
Cache
Size
Index Cache
4,194,303
Data File Cache
4,194,303
Data Cache
4,194,303
.cfg file
NETDELAY 2000
NETRETRYCOUNT 1500
CALCLOCKBLOCKHIGH 1000000
CALCLOCKBLOCKDEFAULT 1000
CALCLOCKBLOCKLOW 100
DIRECTIO TRUE
CALCCACHEHIGH  200000000
CALCCACHEDEFAULT 1000000
CALCCACHELOW  500000
PARCALCMULTIPLEBITMAPMEMOPT TRUE
MULTIPLEBITMAPMEMCHECK TRUE
CALCPARALLEL 4
CALCTASKDIMS 3
CALCPARALLEL  4
CALCTASKDIMS 3
CALCPARALLEL  4

Answer 1

Those cache sizes look to have been set by brute force rather than testing.  The index cache is needlessly high - once it's large enough to contain the entire index there is no benefit to going higher.
The data file cache does not need to be set at all unless you're using Direct I/O instead of Buffered I/O (it probably doesn't make any difference, but I wouldn't leave it set to 4GB anyway).
Are you saying that memory usage climbs to 13GB without any activity (i.e. just start and wait)?
CALCCACHEHIGH is also at the absolute maximum permissible - if you're doing parallel calc and specifying SET CACHE HIGH you grab that much memory for every thread.
EDIT: Actually, I see now that you are using Direct I/O. Still, I'd 'start again' following the DBAG recommendations as a starting point.  Also, do you know why you have multiple  CALCPARALLEL lines in your Essbase.cfg file?  Again, the 'tuning' seems to have been the brute force 'set it all as high as it goes' approach rather than iterative testing to see where you actually get a benefit.