BIA Rev. 7.00.53.00, DBMAN 305, RSD_TREX 100, unloading temp indexes...

Question

Hi BIA team,
we have several questions regarding some error situations which we get when we execute many parallel queries against the BIA.
On BI side we get this 2 errors:
DBMAN 305 Error reading the data of InfoProvider YREP_BC_1$X YREP_BC_1$X
RSD_TREX 100 Error executing physical plan: general error (no f 2999
On BIA side in the TrexRFCServerAlert trace at the same time we found this type of errors:
1. TREXRfcSvr   trexAggregate.cpp(01016) : TREX_RFC_AGGREGATE_SP5 rc: 2999 (Error executing physical plan: general error (no further information available);BwPopBuildIndexUnion pop29(setAliasIndex('pcs_yrep_bc_1-union_of_pcs_bi0_0805006369_pcs_bi0_0805006377en'),addIndexForUnion('pcs_bi0:0805006369en'),addIndexForUnion('pcs_bi0:0805006377en')),in executor::Executor in cube: pcs_yrep_bc_1).
2. TREXRfcSvr   trexAggregate.cpp(01016) : TREX_RFC_AGGREGATE_SP5 rc: 2007 (Error creating physical plan: Index does not exist;index=pcs_bi0:0805095572 in cube: pcs_yrep_bc_1).
We have observed that this types of errors are present if the BIA has the overloaded warning and unloading temp. indexes...
Now the Questions:
1. Why is BIA unloading indexes which are needed for the current query execution, why not deleting only old ones ?
2. After 10000 query executions we have about 22000 temp indexes and in the Reorg tab there is a warning "Standard Deviation of Join/OLAP Indexes 10147 MB, 1625 MB, 84 %, reorg yes", should we schedule a daily job for the cleanup ?
5. What exactly does "Standard Deviation of Join/OLAP Indexes" in BIA world mean ?
3. Were are the temp indexes stored, Shared Memory ?
4. Currently we have phys. 284 GB Ram and 95 MB shared memory assigned, what would be the ideal value for Shared Memory ?
Thanks a lot in advance!
Roman Doubrava

Answer 1

I believe you can dismiss the reorg message, it should take care of itself.
Make sure your BW statistics are running regularly, lack of it may contribute to this.
Looks like you need more memory ( by deleting data or adding blades). Before than I would do a reorg and re-start BWA to clean up temp indexes -there is one memory for data and temp indexes  -.
I would check if you have APD queries and definitely exclude them from BWA.
If none works, I would check the statistics for the last 90 days to see what cubes and what queries went to BWA and how many times. Based on the facts that you can either eliminate the least used cubes, based on the nr of recordes being fetched you can exclude certain queries from BWA maybe schedule them overnight, you can exclude BATCH/Information Broadcasting queries, user aleremote, etc  
Cheers
Tansu