Datatype best practice and plan cardinality

Question

Hi,
I have a scenario where I need to store the data in the format YYYYMM (e.g. 201001 which means January, 2010).
I am trying to evaluate what is the most appropriate datatype to store this kind of data. I am comparing 2 options, NUMBER and DATE.
As the data is essentially a component of oracle date datatype and experts like Tom Kyte have proved (with examples) that using right
datatype is better for optimizer. So I was expecting that using DATE datatype will yield (at least) similar (if not better) cardinality estimates
than using NUMBER datatype. However, my tests show that when using DATE the cardinality estimates are way off from actuals whereas
using NUMBER the cardinality estimates are much closer to actuals.
My questions are:
1) What should be the most appropriate datatype used to store YYYYMM data?
2) Why does using DATE datatype yield estimates that are way off from actuals than using NUMBER datatype?
SQL> select * from V$VERSION ;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Prod
PL/SQL Release 10.2.0.1.0 - Production
CORE     10.2.0.1.0     Production
TNS for Linux: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production
SQL>  create table a nologging as select to_number(to_char(add_months(to_date('200101','YYYYMM'),level - 1), 'YYYYMM')) id from dual connect by level <= 289 ;
Table created.
SQL> create table b (id number) ;
Table created.
SQL> begin
  2  for i in 1..8192
  3  loop
  4     insert into b select * from a ;
  5  end loop;
  6  commit;
  7  end;
  8  /
PL/SQL procedure successfully completed.
SQL> alter table a add dt date ;
Table altered.
SQL> alter table b add dt date ;
Table altered.
SQL> select to_date(200101, 'YYYYMM') from dual ;
TO_DATE(2
01-JAN-01
SQL> update a set dt = to_date(id, 'YYYYMM') ;
289 rows updated.
SQL> update b set dt = to_date(id, 'YYYYMM') ;
2367488 rows updated.
SQL> commit ;
Commit complete.
SQL> exec dbms_stats.gather_table_stats(user, 'A', estimate_percent=>NULL) ;
PL/SQL procedure successfully completed.
SQL> exec dbms_stats.gather_table_stats(user, 'B', estimate_percent=>NULL) ;    
SQL> explain plan for select count(*) from b where id between 200810 and 200903 ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation        | Name | Rows  | Bytes | Cost (%CPU)| Time       |
|   0 | SELECT STATEMENT   |       |     1 |     5 |   824   (4)| 00:00:10 |
|   1 |  SORT AGGREGATE    |       |     1 |     5 |            |       |
|*  2 |   TABLE ACCESS FULL| B       | 46604 |   227K|   824   (4)| 00:00:10 |
Predicate Information (identified by operation id):
PLAN_TABLE_OUTPUT
   2 - filter("ID"<=200903 AND "ID">=200810)
14 rows selected.
SQL> explain plan for select count(*) from b where dt between to_date(200810, 'YYYYMM') and to_date(200903, 'YYYYMM') ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation        | Name | Rows  | Bytes | Cost (%CPU)| Time       |
|   0 | SELECT STATEMENT   |       |     1 |     5 |   825   (4)| 00:00:10 |
|   1 |  SORT AGGREGATE    |       |     1 |     5 |            |       |
|*  2 |   TABLE ACCESS FULL| B       |  5919 | 29595 |   825   (4)| 00:00:10 |
Predicate Information (identified by operation id):
PLAN_TABLE_OUTPUT
   2 - filter("DT">=TO_DATE('2008-10-01 00:00:00', 'yyyy-mm-dd
           hh24:mi:ss') AND "DT"<=TO_DATE('2009-03-01 00:00:00', 'yyyy-mm-dd
           hh24:mi:ss'))
16 rows selected.

Answer 1

Charles,
Thanks for your response.
I did not think of the possibilitty of histograms. When I ran the tests on 10.2.0.4, I could get the results as you have shown.
So I thought it might be due to some bug in 10.2.0.1. But interestingly, when I ran the test after collecting statistics using 'FOR ALL COLUMNS SIZE 1'
option, I got the cardinalities that match my 10.2.0.1 results (where METHOD_OPT was default i.e. 'FOR ALL COLUMNS SIZE AUTO').
So I carried out the tests again on 10.2.0.1 but the results did not look consistent to me. When there were no histograms on DATE column, the cardinality
was quite close to actuals but when I collected stats using 'FOR ALL COLUMNS SIZE SKEWONLY', it generated histograms on DATE column but
the cardinality was not quite close to actuals.
So I am bit confused about whether this is due to a bug or due to combined effect of optimizer's "intelligence" while collecting statistics using default option
values and the way table is queried (COL_USAGE$ data).
Here is my test:
SQL> select * from v$version ;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Prod
PL/SQL Release 10.2.0.1.0 - Production
CORE     10.2.0.1.0     Production
TNS for Linux: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production
SQL> exec dbms_stats.delete_table_stats(user, 'B') ;
PL/SQL procedure successfully completed.
SQL> select column_name, num_distinct, num_buckets, histogram from user_tab_col_statistics where table_name = 'B' ;
no rows selected
SQL> exec dbms_stats.gather_table_stats(user, 'B') ;
PL/SQL procedure successfully completed.
SQL> select column_name, num_distinct, num_buckets, histogram from user_tab_col_statistics where table_name = 'B' ;
COLUMN_NAME                    NUM_DISTINCT NUM_BUCKETS HISTOGRAM
ID                                      289         254 HEIGHT BALANCED
DT                                      289         254 HEIGHT BALANCED
SQL> explain plan for select count(*) from b where b.id between 200810 and 200903 ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |     1 |     5 |  3691   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| B    | 38218 |   186K|  3691   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."ID"<=200903 AND "B"."ID">=200810)
14 rows selected.
SQL> explain plan for select count(*) from b where b.dt between to_date(200810, 'YYYYMM') and to_date(200903, 'YYYYMM') ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |     1 |     8 |  3693   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |      |     1 |     8 |            |          |
|*  2 |   TABLE ACCESS FULL| B    | 38218 |   298K|  3693   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."DT"<=TO_DATE('2009-03-01 00:00:00', 'yyyy-mm-dd
              hh24:mi:ss') AND "B"."DT">=TO_DATE('2008-10-01 00:00:00', 'yyyy-mm-dd
              hh24:mi:ss'))
16 rows selected.
SQL> connect sys as sysdba ;
Connected.
SQL> delete from sys.col_usage$ where obj# in (select object_id from all_objects where owner = 'HR' and object_name in ('A','B')) ;
4 rows deleted.
SQL> commit ;
Commit complete.
SQL> connect hr/hr ;
Connected.
SQL> set serveroutput on size 10000
SQL> exec dbms_stats.delete_table_stats(user, 'B') ;
PL/SQL procedure successfully completed.
SQL> exec dbms_stats.gather_table_stats(user, 'B') ;
PL/SQL procedure successfully completed.
SQL> select column_name, num_distinct, num_buckets, histogram from user_tab_col_statistics where table_name = 'B' ;
COLUMN_NAME                    NUM_DISTINCT NUM_BUCKETS HISTOGRAM
ID                                      289           1 NONE
DT                                      289           1 NONE
SQL> explain plan for select count(*) from b where b.id between 200810 and 200903 ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |     1 |     5 |  3691   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| B    |   110K|   541K|  3691   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."ID"<=200903 AND "B"."ID">=200810)
14 rows selected.
SQL> explain plan for select count(*) from b where b.dt between to_date(200810, 'YYYYMM') and to_date(200903, 'YYYYMM') ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |     1 |     8 |  3693   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |      |     1 |     8 |            |          |
|*  2 |   TABLE ACCESS FULL| B    | 58680 |   458K|  3693   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."DT"<=TO_DATE('2009-03-01 00:00:00', 'yyyy-mm-dd
              hh24:mi:ss') AND "B"."DT">=TO_DATE('2008-10-01 00:00:00', 'yyyy-mm-dd
              hh24:mi:ss'))
16 rows selected.
SQL> exec dbms_stats.gather_table_stats(user, 'B') ;
PL/SQL procedure successfully completed.
SQL> select column_name, num_distinct, num_buckets, histogram from user_tab_col_statistics where table_name = 'B' ;
COLUMN_NAME                    NUM_DISTINCT NUM_BUCKETS HISTOGRAM
ID                                      289         254 HEIGHT BALANCED
DT                                      289           1 NONE
SQL> explain plan for select count(*) from b where b.id between 200810 and 200903 ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |     1 |     5 |  3690   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| B    | 46303 |   226K|  3690   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."ID"<=200903 AND "B"."ID">=200810)
14 rows selected.
SQL> explain plan for select count(*) from b where b.dt between to_date(200810, 'YYYYMM') and to_date(200903, 'YYYYMM') ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |     1 |     8 |  3692   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |      |     1 |     8 |            |          |
|*  2 |   TABLE ACCESS FULL| B    | 56797 |   443K|  3692   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."DT"<=TO_DATE('2009-03-01 00:00:00', 'yyyy-mm-dd
              hh24:mi:ss') AND "B"."DT">=TO_DATE('2008-10-01 00:00:00', 'yyyy-mm-dd
              hh24:mi:ss'))
16 rows selected.
SQL> exec dbms_stats.gather_table_stats(user, 'B') ;
PL/SQL procedure successfully completed.
SQL> select column_name, num_distinct, num_buckets, histogram from user_tab_col_statistics where table_name = 'B' ;
COLUMN_NAME                    NUM_DISTINCT NUM_BUCKETS HISTOGRAM
ID                                      289         254 HEIGHT BALANCED
DT                                      289           1 NONE
SQL> exec dbms_stats.gather_table_stats(user, 'B', method_opt=>'FOR ALL COLUMNS SIZE SKEWONLY') ;
PL/SQL procedure successfully completed.
SQL> select column_name, num_distinct, num_buckets, histogram from user_tab_col_statistics where table_name = 'B' ;
COLUMN_NAME                 NUM_DISTINCT NUM_BUCKETS HISTOGRAM
ID                         289         254 HEIGHT BALANCED
DT                         289         254 HEIGHT BALANCED
SQL> explain plan for select count(*) from b where b.dt between to_date(200810, 'YYYYMM') and to_date(200903, 'YYYYMM') ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation        | Name | Rows  | Bytes | Cost (%CPU)| Time       |
|   0 | SELECT STATEMENT   |       |     1 |     8 |  3692   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |       |     1 |     8 |            |       |
|*  2 |   TABLE ACCESS FULL| B       | 27862 |   217K|  3692   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("B"."DT"<=TO_DATE('2009-03-01 00:00:00', 'yyyy-mm-dd
           hh24:mi:ss') AND "B"."DT">=TO_DATE('2008-10-01 00:00:00', 'yyyy-mm-dd
           hh24:mi:ss'))
16 rows selected.
SQL> explain plan for select count(*) from b where id between 200810 and 200903 ;
Explained.
SQL> select * from table(dbms_xplan.display) ;
PLAN_TABLE_OUTPUT
Plan hash value: 749587668
| Id  | Operation        | Name | Rows  | Bytes | Cost (%CPU)| Time       |
|   0 | SELECT STATEMENT   |       |     1 |     5 |  3690   (1)| 00:00:45 |
|   1 |  SORT AGGREGATE    |       |     1 |     5 |            |       |
|*  2 |   TABLE ACCESS FULL| B       | 32505 |   158K|  3690   (1)| 00:00:45 |
Predicate Information (identified by operation id):
   2 - filter("ID"<=200903 AND "ID">=200810)
14 rows selected.