Question about the Query Optimizer

Question

For an Excercise during my database lecture the following table
CREATE TABLE TASKS
    "ID" NUMBER NOT NULL ENABLE,
    "START_DATE" DATE,
    "END_DATE" DATE,
    "DESCRIPTION" VARCHAR2(50 BYTE)
) ;with about 1.5 Million Entries were given. In addition there was the following Query:
SELECT START_DATE, COUNT(START_DATE) FROM TASKS
GROUP BY START_DATE
ORDER BY START_DATE;And the Index:
create index blub on Tasks (start_date asc);The main excercise was to speed up queries with indexes. Because all data is accessed the optimizer ignores the index and just did a full table scan.
Here the QEP:
| Id  | Operation          | Name  | Rows  | Bytes | Cost (%CPU)| Time     |                                                                                                                                                                                                                                
|   0 | SELECT STATEMENT   |       |  9343 | 74744 |  3423   (6)| 00:00:42 |                                                                                                                                                                                                                                
|   1 |  SORT GROUP BY     |       |  9343 | 74744 |  3423   (6)| 00:00:42 |                                                                                                                                                                                                                                
|   2 |   TABLE ACCESS FULL| TASKS |  1981K|    15M|  3276   (2)| 00:00:40 |                                                                                                                                                                                                                                
----------------------------------------------------------------------------Then we tried to force him to do the index with this query:
ALTER SESSION SET OPTIMIZER_MODE = FIRST_ROWS_1;
SELECT /* + INDEX (TASKS BLUB) */ START_DATE, COUNT(START_DATE) FROM TASKS
GROUP BY START_DATE
ORDER BY START_DATE;but again it ignored the index. The optimizer guide states clearly, that each time you access all data from a table it has to do a full scan.
So we tricked him in doing a fast index scan with this query:
create or replace function bla
return date deterministic is
  ret date;
begin
  select MIN(start_date) into ret from Tasks;
  return ret;
end bla;
ALTER SESSION SET OPTIMIZER_MODE = FIRST_ROWS_1;
SELECT /* + INDEX (TASKS BLUB) */ START_DATE, COUNT(START_DATE) FROM TASKS
where start_date >= bla
GROUP BY START_DATE
ORDER BY START_DATE; now we got the following QEP:
| Id  | Operation            | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                                                                                                                                                                               
|   0 | SELECT STATEMENT     |      |     1 |     8 |     3   (0)| 00:00:01 |                                                                                                                                                                                                                               
|   1 |  SORT GROUP BY NOSORT|      |     1 |     8 |     3   (0)| 00:00:01 |                                                                                                                                                                                                                               
|*  2 |   INDEX RANGE SCAN   | BLUB |     1 |     8 |     3   (0)| 00:00:01 |                                                                                                                                                                                                                               
----------------------------------------------------------------------------- So it do use the index.
Now to my two questions:
1. Why should it always do a full scan (because the optimizer documentation answer is a little bit unsatisfying)?
2. After looking at the difference between the costs (FS: 3276 IR: 3) and the time the system needs (FS: 9,6 sec IR: 4,45) why does the optimizer refuse the clearly better plan?
Thanks in advance,
Kai Gödde
Edited by: Kai Gödde on May 30, 2011 6:54 PM
Edited by: Kai Gödde on May 30, 2011 6:56 PM

Answer 1

John Spencer mentioned already the most important point (the role of NULL values) but here are some minor additions:
* with the additional NOT NULL condition Oracle will do an INDEX FAST FULL SCAN (a multiblock scan of the index segment, similar to a FULL TABLE SCAN)
* with the additional NOT NULL condition you don't need a hint and Oracle will choose the IFFS access
* if the index was a bitmap index you would not need a NOT NULL condition because Oracle stores NULL-values in bitmap indexes (but you should not define bitmap indexes in an OLTP system because they bring massive locking issues)
* if the index would contain a second value the additional NOT NULL condition would also be needless
CREATE TABLE TASKS
    "ID" NUMBER NOT NULL ENABLE,
    "START_DATE" DATE,
    "END_DATE" DATE,
    "DESCRIPTION" VARCHAR2(50 BYTE)
insert into tasks
select rownum
     , case when mod(rownum, 5) = 0 then null else sysdate - round(rownum/100) end
     , case when mod(rownum, 4) = 0 then null else sysdate - round(rownum/50) end
     , lpad('*', 50 , '*')
  from dual
connect by level <= 1500000;
exec dbms_stats.gather_table_stats(user, 'TASKS')
-- the IFFS access without a hint:
create index blub on Tasks (start_date);
SELECT START_DATE
     , COUNT(START_DATE)
  FROM TASKS
WHERE START_DATE IS NOT NULL
GROUP BY START_DATE
ORDER BY START_DATE;
| Id  | Operation             | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT      |      | 15001 |   102K|  1871  (16)| 00:00:10 |
|   1 |  SORT GROUP BY        |      | 15001 |   102K|  1871  (16)| 00:00:10 |
|*  2 |   INDEX FAST FULL SCAN| BLUB |  1200K|  8203K|  1631   (3)| 00:00:09 |
Statistiken
          0  recursive calls
          0  db block gets
       3198  consistent gets
          0  physical reads
          0  redo size
     378627  bytes sent via SQL*Net to client
      11520  bytes received via SQL*Net from client
       1002  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
      15001  rows processed
-- the bitmap index
create bitmap index blub on tasks(START_DATE);
SELECT START_DATE
     , COUNT(START_DATE)
  FROM TASKS
GROUP BY START_DATE
ORDER BY START_DATE
| Id  | Operation                    | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT             |      | 15001 |   102K|   132   (1)| 00:00:01 |
|   1 |  SORT GROUP BY NOSORT        |      | 15001 |   102K|   132   (1)| 00:00:01 |
|   2 |   BITMAP CONVERSION TO ROWIDS|      |  1500K|    10M|   132   (1)| 00:00:01 |
|   3 |    BITMAP INDEX FULL SCAN    | BLUB |       |       |            |          |
Statistiken
          1  recursive calls
          0  db block gets
       1126  consistent gets
          0  physical reads
          0  redo size
     378682  bytes sent via SQL*Net to client
      11520  bytes received via SQL*Net from client
       1002  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      15002  rows processed
-- the composite index
create index blub on tasks(START_DATE, 0);
SELECT START_DATE
     , COUNT(START_DATE)
  FROM TASKS
GROUP BY START_DATE
ORDER BY START_DATE;
| Id  | Operation             | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT      |      | 15001 |   102K|  2400  (15)| 00:00:12 |
|   1 |  SORT GROUP BY        |      | 15001 |   102K|  2400  (15)| 00:00:12 |
|   2 |   INDEX FAST FULL SCAN| BLUB |  1500K|    10M|  2095   (3)| 00:00:11 |
Statistiken
          0  recursive calls
          0  db block gets
       4120  consistent gets
          0  physical reads
          0  redo size
     378682  bytes sent via SQL*Net to client
      11520  bytes received via SQL*Net from client
       1002  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
      15002  rows processedRegards
Martin Preiss