Performance Tuning Data Load for ASO cube

Question

Hi,
Anyone can help how to fine tune data load on ASO cube.
We have ASO cube which load around 110 million records from a total of 20 data files.
18 of the data files has 4 million records each and the last two has around 18 million records.
On average, to load 4 million records it took 130 seconds.
The data file has 157 data column representing period dimension.
With BSO cube, sorting the data file normally help. But with ASO, it does not seem to have
any impact. Any suggestion how to improve the data load performance for ASO cube?
Thanks,
Lian

Answer 1

Yes TimG it sure looks identical - except for the last BSO reference.
Well nevermind as long as those that count remember where the words come from.
To the Original Poster and to 960127 (come on create a profile already will you?):
The sort order WILL matter IF you are using a compression dimension. In this case the compression dimension acts just like a BSO Dense dimension. If you load part of it in one record then when the next record comes along it has to be added to the already existing part. The ASO "load buffer" is really a file named <dbname.dat> that is built in your temp tablespace.
The most recent x records that can fit in the ASO cache are still retained on the disk drive in the cache. So if the record is still there it will not have to be reread from the disk drive. So you could (instead of sorting) create an ASO cache as large as your final dat file. Then the record would already still be on the disk.
BUT WAIT BEFORE YOU GO RAISING YOUR ASO CACHE. All operating systems use memory mapped IO therefore even if it is not in the cache it will likely still be in on the disk in "Standby" memory (the dark blue memory as seen in Resource Monitor) this will continue until the system runs out of "Free" memory (light blue in resource monitor).
So in conclusion if your system still has Free memory there is no need (in a data load) to increase your ASO cache. And if you are out of Free memory then all you will do is slow down the other applications running on your system by increasing ASO Cache during a data load - so don't do it.
Finally, if you have enough memory so that the entire data file fits in StandBY + Free memory then don't bother to sort it first. But if you do not have enough then sort it.
Of course you have 20 data files so I hope that you do not have compression members spread out amongst these files!!!
Finally, you did not say if you were using parallel load threads. If you need to have 20 files read up on having parrallel load buffers and parallel load scripts. that will make it faster.
But if you do not really need 20 files and just broke them up to load parallel then create one single file and raise your DLTHREADSPREPARE and DLTHREADSWRITE settings. Heck these will help even if you do go parallel and really help if you don't but still keep 20 separate files.