Auto estimate percent and histograms or small est percent no histograms by default?

Question

In the past I've used custom written statistics gathering scripts that by default gather statistics on large tables with a small estimate percentage and FOR ALL COLUMNS SIZE 1.  They allow the estimate percentage to be set higher on individual tables and allow me to choose individual columns to have histograms with the maximum number of buckets.  The nice thing about this approach was that the statistics ran very efficiently by default and they could be dialed up as needed to tune individual queries.
But, with 11g you can set preferences at the table level so that the automatic stats job or even a manual run of gather_table_stats will use your settings.  Also, there is some special sampling algorithm associated with auto_sample_size that you miss out on if you manually set the estimate percentage.  So, I could change my approach to query tuning and statistics gathering to use AUTO_SAMPLE_SIZE and FOR ALL COLUMNS SIZE AUTO by default and then override these if needed to tune a query or if needed to make the statistics run in an acceptable length of time.
I work in an largish company with a team of about 10 DBAs and a bunch of developers and over 100 Oracle databases so we can't really carefully keep track of each system.  Having the defaults be less resource intensive saves me the grief of having stats jobs run too long, but it requires intervention when a query needs to be tuned.  Also, with my custom scripts I get a lot of hassles from people wanting to know why I'm not using the Oracle default settings so that is a negative.  But I've seen the default settings, even on 11.2, run much too long on large tables.  Typically our systems start small and then data grows and grows until things start breaking so the auto stats may seem fine at first, but eventually they will start running too long.  Also, when the stats jobs run forever or with the automatic jobs don't finish in the window I get complaints about the stats taking too long or not being updated.  So, either direction has its pros and cons as far as I can tell.
So, my question, fueled by a post work day Sierra Nevada Pale Ale, is whether anyone else has this same dillema and how you have resolved it.  I appreciate any input on this subject.
- Bobby

Answer 1

Hi Bobby,
> Also, when the stats jobs run forever or with the automatic jobs don't finish in the window I get complaints about the stats taking too long or not being updated
Complaints about "not updated" statistics are very common, but they don't need to be "up-to-date" ... they just need to be representative ... a very important difference
> I think it comes down to estimate percentage and histograms.  If you can't get the percentage and histograms you want to complete in time you have to dial them back but then the stats aren't as good.
Be aware of how histograms are gathered under-the-hood for each column. There are some nice improvements with AUTO_SAMPLE_SIZE (Blog: How does AUTO_SAMPLE...). However if you want to know more about the NDV improvements or the synopses - please check out the paper of Amit Poddar One Pass Distinct Sampling.
Personally i usually start with "FOR ALL COLUMNS SIZE 1" and AUTO_SAMPLE_SIZE (from the scratch) and change only the table preferences that need to be adjusted (like histograms due to data skew, etc.). The other basic stuff works pretty well in most cases and so i usually don't need to put much effort in it (you don't need to fix something, that is not broke). There also issues with aggressive sampling and CBO behavior changes (like fix_control 5483301:off/on), but this needs to be cross-checked from case to case of course.
Regards
Stefan