Scaleability with Functions in SQL queries

Question

Hi,
In one of our applications we have many views that use a packaged function in the where clause to filter data. This function uses a SYS_CONTEXT() to set and get values. There are couple of issues while using this approach:
1/ The deterministic function doesn't allow any scability with PQ-server.
2/ Another issue with this function and also the SYS_CONTEXT-function, they manuipulate the estimated CBO-statistics.
  CREATE TABLE TAB_I
  COLUMN1 NUMBER(16, 0) NOT NULL
, COLUMN2 VARCHAR2(20)
, CONSTRAINT TAB_I_PK PRIMARY KEY
    COLUMN1
  ENABLE
CREATE TABLE TAB_V
    I_COL1     NUMBER(16,0) NOT NULL ENABLE,
    VERSION_ID NUMBER(16,0) NOT NULL ENABLE,
    CRE_DATIM TIMESTAMP (6) NOT NULL ENABLE,
    TERM_DATIM TIMESTAMP (6) NOT NULL ENABLE,
    VERSION_VALID_FROM DATE NOT NULL ENABLE,
    VERSION_VALID_TILL DATE NOT NULL ENABLE,
    CONSTRAINT TAB_V_PK PRIMARY KEY (I_COL1, VERSION_ID) USING INDEX NOCOMPRESS LOGGING ENABLE,
    CONSTRAINT COL1_FK FOREIGN KEY (I_COL1) REFERENCES TAB_I (COLUMN1) ENABLE
CREATE OR REPLACE
PACKAGE      app_bitemporal_rules IS
FUNCTION f_knowledge_time RETURN TIMESTAMP DETERMINISTIC;
END app_bitemporal_rules;
create or replace
PACKAGE BODY      app_bitemporal_rules IS
FUNCTION f_knowledge_time RETURN TIMESTAMP DETERMINISTIC IS
BEGIN
     RETURN TO_TIMESTAMP(SYS_CONTEXT ('APP_USR_CTX', 'KNOWLEDGE_TIME'),'DD.MM.YYYY HH24.MI.SSXFF');
END f_knowledge_time;
END app_bitemporal_rules;
explain plan for select *
FROM tab_i
JOIN tab_v
ON tab_i.column1 = tab_v.i_col1
AND           app_bitemporal_rules.f_knowledge_time BETWEEN tab_v.CRE_DATIM AND tab_v.TERM_DATIM
where tab_i.column1 = 11111;
select * from table(dbms_xplan.display);
Plan hash value: 621902595
| Id  | Operation                    | Name     | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT             |          |     1 |    95 |     5   (0)| 00:00:06 |
|   1 |  NESTED LOOPS                |          |     1 |    95 |     5   (0)| 00:00:06 |
|   2 |   TABLE ACCESS BY INDEX ROWID| TAB_I    |     1 |    25 |     1   (0)| 00:00:02 |
|*  3 |    INDEX UNIQUE SCAN         | TAB_I_PK |     1 |       |     1   (0)| 00:00:02 |
|*  4 |   TABLE ACCESS FULL          | TAB_V    |     1 |    70 |     4   (0)| 00:00:05 |
Predicate Information (identified by operation id):
   3 - access("TAB_I"."COLUMN1"=11111)
   4 - filter("TAB_V"."I_COL1"=11111 AND
              "TAB_V"."CRE_DATIM"<="APP_BITEMPORAL_RULES"."F_KNOWLEDGE_TIME"() AND
              "TAB_V"."TERM_DATIM">="APP_BITEMPORAL_RULES"."F_KNOWLEDGE_TIME"())
Note
   - 'PLAN_TABLE' is old version
   - dynamic sampling used for this statement (level=2)
explain plan for select *
FROM tab_i
JOIN tab_v
ON tab_i.column1 = tab_v.i_col1
AND           '10-OCT-2011' BETWEEN tab_v.CRE_DATIM AND tab_v.TERM_DATIM
where tab_i.column1 = 11111;
select * from table(dbms_xplan.display);  
Plan hash value: 621902595
| Id  | Operation                    | Name     | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT             |          |   256 | 24320 |     5   (0)| 00:00:06 |
|   1 |  NESTED LOOPS                |          |   256 | 24320 |     5   (0)| 00:00:06 |
|   2 |   TABLE ACCESS BY INDEX ROWID| TAB_I    |     1 |    25 |     1   (0)| 00:00:02 |
|*  3 |    INDEX UNIQUE SCAN         | TAB_I_PK |     1 |       |     1   (0)| 00:00:02 |
|*  4 |   TABLE ACCESS FULL          | TAB_V    |   256 | 17920 |     4   (0)| 00:00:05 |
Predicate Information (identified by operation id):
   3 - access("TAB_I"."COLUMN1"=11111)
   4 - filter("TAB_V"."I_COL1"=11111 AND "TAB_V"."CRE_DATIM"<=TIMESTAMP'
              2011-10-10 00:00:00.000000000' AND "TAB_V"."TERM_DATIM">=TIMESTAMP' 2011-10-10
              00:00:00.000000000')
Note
   - 'PLAN_TABLE' is old version
   - dynamic sampling used for this statement (level=2)   As can be seen in the second plan the cardinality has been guessed correctly, but not in the first case.
I have also tried with:
ASSOCIATE STATISTICS WITH packages app_bitemporal_rules DEFAULT COST (1000000/*246919*/,1000,0) DEFAULT SELECTIVITY 50;
But, this just leads to a increased cost, but no change in cardinality.
The (1) problem gets solved if I directly use "TO_TIMESTAMP(SYS_CONTEXT ('APP_USR_CTX', 'KNOWLEDGE_TIME'),'DD.MM.YYYY HH24.MI.SSXFF')" in the where clause. But am not able to find a solution for the (2) issue.
Can you please help.
Regards,
Vikram R

Answer 1

Hi Vikram,
On the subject of using [url http://download.oracle.com/docs/cd/E11882_01/server.112/e26088/statements_4006.htm#i2115932]ASSOCIATE STATISTICS, having done a little investigation on 11.2.0.2, I'm having trouble adjusting selectivity via "associate statististics ... default selectivity" but no problems with adjusting default cost.
I've also tried to do the same using an interface type and am running into other issues.
It's not functionality that I'm overly familiar with as I try to avoid/eliminate using functions in predicates.
Further analysis/investigation required.
Including test case of what I've done so far in case anyone else wants to chip in.
SQL> select * from v$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
CORE    11.2.0.2.0      Production
TNS for Linux: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
SQL> drop table t1;
Table dropped.
SQL>
SQL> create table t1
  2  as
  3  select rownum col1
  4  from   dual
  5  connect by rownum <= 100000;
Table created.
SQL>
SQL> exec dbms_stats.gather_table_stats(USER,'T1');
PL/SQL procedure successfully completed.
SQL>
SQL> create or replace function f1
  2  return number
  3  as
  4  begin
  5   return 1;
  6  end;
  7  /
Function created.
SQL>
SQL> create or replace function f2 (
  2   i_col1 in number
  3  )
  4  return number
  5  as
  6  begin
  7   return 1;
  8  end;
  9  /
Function created.
SQL> Created one table with 100000 rows.
Two functions - one without arguments, one with (for later).
With no associations:
SQL> select * from user_associations;
no rows selected
SQL> Run a statement that uses the function:
SQL> select count(*) from t1 where col1 >= f1;
  COUNT(*)
    100000
SQL> select * from table(dbms_xplan.display_cursor);
PLAN_TABLE_OUTPUT
SQL_ID  gm7ppkbzut114, child number 0
select count(*) from t1 where col1 >= f1
Plan hash value: 3724264953
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |       |       |   139 (100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| T1   |  5000 | 25000 |   139  (62)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - filter("COL1">="F1"())
19 rows selected.
SQL> Shows that default selectivity of 5% for an equality predicate against function.
Let's try to adjust the selectivity using associate statistics - the argument for selectivity should be a percentage between 0 and 100:
(turning off cardinality feedback for clarity/simplicity)
SQL> alter session set "_optimizer_use_feedback" = false;
Session altered.
SQL>
SQL> ASSOCIATE STATISTICS WITH FUNCTIONS f1 default selectivity 100;
Statistics associated.
SQL> select count(*) from t1 where col1 >= f1;
  COUNT(*)
    100000
SQL> select * from table(dbms_xplan.display_cursor);
PLAN_TABLE_OUTPUT
SQL_ID  gm7ppkbzut114, child number 1
select count(*) from t1 where col1 >= f1
Plan hash value: 3724264953
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |       |       |   139 (100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| T1   |  5000 | 25000 |   139  (62)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - filter("COL1">="F1"())
19 rows selected.
SQL> Didn't make any difference to selectivity.
An excerpt from a 10053 trace file had the following:
** Performing dynamic sampling initial checks. **
** Dynamic sampling initial checks returning FALSE.
  No statistics type defined for function F1
  No default cost defined for function F1So, crucially what's missing here is a clause saying:
No default selectivity defined for function F1But there's no other information that I could see to indicate why it should be discarded.
Moving on, adjusting the cost does happen:
SQL>exec spflush;
PL/SQL procedure successfully completed.
SQL> disassociate statistics from functions f1;
Statistics disassociated.
SQL>
SQL> ASSOCIATE STATISTICS WITH FUNCTIONS f1 default selectivity 100 default cost (100,5,0);
Statistics associated.
SQL> select count(*) from t1 where col1 >= f1;
  COUNT(*)
    100000
SQL> select * from table(dbms_xplan.display_cursor);
PLAN_TABLE_OUTPUT
SQL_ID  gm7ppkbzut114, child number 0
select count(*) from t1 where col1 >= f1
Plan hash value: 3724264953
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |       |       |   500K(100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| T1   |  5000 | 25000 |   500K  (1)| 00:41:41 |
Predicate Information (identified by operation id):
   2 - filter("COL1">="F1"())
19 rows selected.
SQL> And we see the following in a 10053:
  No statistics type defined for function F1
  Default costs for function F1 CPU: 100, I/O: 5So, confirmation that default costs for function were found and applied but nothing else about selectivity again.
I wondered whether the lack of arguments for function F1 made any difference, hence function F2.
Didn't seem to:
Vanilla:
SQL> select count(*) from t1 where col1 >= f2(col1);
  COUNT(*)
    100000
SQL>
SQL> select * from table(dbms_xplan.display_cursor);
PLAN_TABLE_OUTPUT
SQL_ID  2wxw32wadgc1v, child number 0
select count(*) from t1 where col1 >= f2(col1)
Plan hash value: 3724264953
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |       |       |   139 (100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| T1   |  5000 | 25000 |   139  (62)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - filter("COL1">="F2"("COL1"))
19 rows selected.
SQL> Plus association:
SQL>exec spflush;
PL/SQL procedure successfully completed.
SQL>
SQL> associate statistics with functions f2 default selectivity 90 default cost (100,5,0);
Statistics associated.
SQL> select count(*) from t1 where col1 >= f2(col1);
  COUNT(*)
    100000
SQL>
SQL> select * from table(dbms_xplan.display_cursor);
PLAN_TABLE_OUTPUT
SQL_ID  2wxw32wadgc1v, child number 0
select count(*) from t1 where col1 >= f2(col1)
Plan hash value: 3724264953
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |       |       |   500K(100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| T1   |  5000 | 25000 |   500K  (1)| 00:41:41 |
Predicate Information (identified by operation id):
   2 - filter("COL1">="F2"("COL1"))
19 rows selected.
SQL> Just to confirm associations:
SQL> select * from user_associations;
OBJECT_OWNER                   OBJECT_NAME                    COLUMN_NAME                    OBJECT_TY
STATSTYPE_SCHEMA               STATSTYPE_NAME                 DEF_SELECTIVITY DEF_CPU_COST DEF_IO_COST DEF_NET_COST
INTERFACE_VERSION MAINTENANCE_TY
RIMS                           F2                                                            FUNCTION
                                                                           90          100           5
                0 USER_MANAGED
RIMS                           F1                                                            FUNCTION
                                                                          100          100           5
                0 USER_MANAGED
SQL> So.... started thinking about whether using an interface type would help?
SQL> CREATE OR REPLACE TYPE test_stats_ot AS OBJECT
  2  (dummy_attribute NUMBER
  3  ,STATIC FUNCTION ODCIGetInterfaces (
  4     ifclist                OUT SYS.ODCIObjectList
  5   ) RETURN NUMBER
  6  ,STATIC FUNCTION ODCIStatsSelectivity (
  7      pred                   IN  SYS.ODCIPredInfo,
  8      sel                    OUT NUMBER,
  9      args                   IN  SYS.ODCIArgDescList,
10      strt                   IN  NUMBER,
11      stop                   IN  NUMBER,
12      --i_col1                 in  NUMBER,
13      env                    IN  SYS.ODCIEnv
14   ) RETURN NUMBER
15  --,STATIC FUNCTION ODCIStatsFunctionCost (
16  --    func                   IN  SYS.ODCIPredInfo,
17  --    cost                   OUT SYS.ODCICost,
18  --    args                   IN  SYS.ODCIArgDescList,
19  --    i_col1                 in  NUMBER,
20  --    env                    IN  SYS.ODCIEnv
21  -- ) RETURN NUMBER
22  );
23  /
Type created.
SQL> CREATE OR REPLACE TYPE BODY test_stats_ot
  2  AS
  3   STATIC FUNCTION ODCIGetInterfaces (
  4    ifclist                OUT SYS.ODCIObjectList
  5   ) RETURN NUMBER
  6   IS
  7   BEGIN
  8    ifclist := sys.odciobjectlist(sys.odciobject('SYS','ODCISTATS2'));
  9    RETURN odciconst.success;
10   END;
11   STATIC FUNCTION ODCIStatsSelectivity
12   (pred                   IN  SYS.ODCIPredInfo,
13    sel                    OUT NUMBER,
14    args                   IN  SYS.ODCIArgDescList,
15    strt                   IN  NUMBER,
16    stop                   IN  NUMBER,
17    --i_col1                 in  NUMBER,
18    env                    IN  SYS.ODCIEnv)
19   RETURN NUMBER
20   IS
21   BEGIN
22     sel := 90;
23     RETURN odciconst.success;
24   END;
25  -- STATIC FUNCTION ODCIStatsFunctionCost (
26  --  func                   IN  SYS.ODCIPredInfo,
27  --  cost                   OUT SYS.ODCICost,
28  --  args                   IN  SYS.ODCIArgDescList,
29  --  i_col1                 in  NUMBER,
30  --  env                    IN  SYS.ODCIEnv
31  -- ) RETURN NUMBER
32  -- IS
33  -- BEGIN
34  --  cost := sys.ODCICost(10000,5,0,0);
35  --  RETURN odciconst.success;
36  -- END;
37  END;
38  /
Type body created.
SQL> But this approach is not happy - perhaps not liking the function with no arguments?
SQL> disassociate statistics from functions f1;
Statistics disassociated.
SQL> ASSOCIATE STATISTICS WITH FUNCTIONS f1 USING test_stats_ot;
Statistics associated.
SQL> select count(*) from t1 where col1 >= f1;
select count(*) from t1 where col1 >= f1
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-06550: line 12, column 22:
PLS-00103: Encountered the symbol "ÀÄ" when expecting one of the following:
) , * & = - + < / > at in is mod remainder not rem =>
<an exponent (**)> <> or != or ~= >= <= <> and or like like2
like4 likec between || multiset member submultiset
SQL> So, back to F2 again (uncommenting argument i_col1 in ODCIStatsSelectivity):
SQL> disassociate statistics from functions f1;
Statistics disassociated.
SQL> disassociate statistics from functions f2;
Statistics disassociated.
SQL> ASSOCIATE STATISTICS WITH FUNCTIONS f2 USING test_stats_ot;
Statistics associated.
SQL> select count(*) from t1 where col1 >= f2(col1);
  COUNT(*)
    100000
SQL> select * from table(dbms_xplan.display_cursor);
PLAN_TABLE_OUTPUT
SQL_ID  2wxw32wadgc1v, child number 0
select count(*) from t1 where col1 >= f2(col1)
Plan hash value: 3724264953
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT   |      |       |       |   139 (100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |     5 |            |          |
|*  2 |   TABLE ACCESS FULL| T1   |  5000 | 25000 |   139  (62)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - filter("COL1">="F2"("COL1"))
19 rows selected.
SQL> Nothing obviously happening.
You'll note also in my interface type implementation that I commented out a declaration of ODCIStatsFunctionCost.
This post is probably already too long already so I've skipped some of the detail.
But when ODCIStatsFunctionCost was used with function F2, I presume I've made a mistake in the implementation because I had an error in the 10053 trace as follows:
  Calling user-defined function cost function...
    predicate: "RIMS"."F2"("T1"."COL1")
  declare
     cost sys.ODCICost := sys.ODCICost(NULL, NULL, NULL, NULL);
     arg0 NUMBER := null;
    begin
      :1 := "RIMS"."TEST_STATS_OT".ODCIStatsFunctionCost(
                     sys.ODCIFuncInfo('RIMS',
                            'F2',
                            NULL,
                            1),
                     cost,
                     sys.ODCIARGDESCLIST(sys.ODCIARGDESC(2, 'T1', 'RIMS', '"COL1"', NULL, NULL, NULL))
                     , arg0,
                     sys.ODCIENV(:5,:6,:7,:8));
      if cost.CPUCost IS NULL then
        :2 := -1.0;
      else
        :2 := cost.CPUCost;
      end if;
      if cost.IOCost IS NULL then
        :3 := -1.0;
      else
        :3 := cost.IOCost;
      end if;
      if cost.NetworkCost IS NULL then
        :4 := -1.0;
      else
        :4 := cost.NetworkCost;
      end if;
      exception
        when others then
          raise;
    end;
ODCIEnv Bind :5 Value 0
ODCIEnv Bind :6 Value 0
ODCIEnv Bind :7 Value 0
ODCIEnv Bind :8 Value 4
  ORA-6550 received when calling RIMS.TEST_STATS_OT.ODCIStatsFunctionCost -- method ignoredThere was never any such feedback about ODCIStatsSelectivity.
So, in summary, more questions than answers.
I'll try to have another look later.