Connect BY Performance Prob- [10.2.0.4 - No Additonal Patch Sets deployed]

Question

Hi All,
Having trouble with a connect-by performance.
All columns are indexed, so cannot see why this is taking so long to retrieve a 135 row data-set (105,000 rows data set).
For some reason the explain plan insists on Full Table Scans:
SELECT STATEMENT ALL_ROWS
Cost: 1,121 Bytes: 3,390,384 Cardinality: 70,633 CPU Cost: 203,799,167 IO Cost: 1,085                                         
     16 HASH JOIN Cost: 1,121 Bytes: 3,390,384 Cardinality: 70,633 CPU Cost: 203,799,167 IO Cost: 1,085                                    
          1 TABLE ACCESS FULL TABLE BANKREC.test_table Cost: 140 Bytes: 632,259 Cardinality: 70,251 CPU Cost: 42,653,073 IO Cost: 132                               
          15 VIEW BANKREC. Cost: 665 Bytes: 3,870,321 Cardinality: 99,239 CPU Cost: 124,660,113 IO Cost: 643                               
               14 CONNECT BY WITH FILTERING                          
                    6 VIEW BANKREC. Cost: 665 Bytes: 7,740,642 Cardinality: 99,239 CPU Cost: 124,660,113 IO Cost: 643                     
                         5 SORT UNIQUE Cost: 665 Bytes: 1,488,570 Cardinality: 99,239 CPU Cost: 119,007,497 IO Cost: 641                
                              4 UNION-ALL           
                                   2 TABLE ACCESS FULL TABLE BANKREC.test_table Cost: 137 Bytes: 1,488,570 Cardinality: 99,238 CPU Cost: 31,045,760 IO Cost: 132      
                                   3 FAST DUAL Cost: 2 Cardinality: 1 CPU Cost: 7,271 IO Cost: 2      
                    13 HASH JOIN                     
                         7 CONNECT BY PUMP                
                         12 VIEW BANKREC. Cost: 665 Bytes: 2,580,214 Cardinality: 99,239 CPU Cost: 124,660,113 IO Cost: 643                
                              11 SORT UNIQUE Cost: 665 Bytes: 1,488,570 Cardinality: 99,239 CPU Cost: 119,007,497 IO Cost: 641           
                                   10 UNION-ALL      
                                        8 TABLE ACCESS FULL TABLE BANKREC.test_table Cost: 137 Bytes: 1,488,570 Cardinality: 99,238 CPU Cost: 31,045,760 IO Cost: 132
                                        9 FAST DUAL Cost: 2 Cardinality: 1 CPU Cost: 7,271 IO Cost: 2
The query data being returned is based upon- 20-level depth / up to 10-items at any given single depth hierarchy.
The problem I have is this is taking 2-3 secs to return any given groups data back.
The query:
select * from connect_by_testv4
where acct_group = 1110777 /*Pick anything but 0 - as 0 returns all acct_ids which are in a group.
Select a acct_group which exists*/
The view:
CREATE OR REPLACE FORCE VIEW CONNECT_BY_TESTv4
(ACCT_GROUP, ACCT_ID)
AS
with account_groups
as (select connect_by_root ba.acct_id orig_id, ba.acct_id, ba.acct_group
from (select acct_id, acct_group, acct_type /*We add record 0 as this is the top level of the hierarchy*/
from test_table b1
union
select 0, null,4 from dual) ba
start with (ba.acct_type =4) /*acct_type =4- Group: List only the groups then find the children*/
connect by ba.acct_group = prior ba.acct_id
select ag.orig_id acct_group, ag.acct_id
from account_groups ag
join test_table unio
on ag.acct_id = unio.acct_id
where unio.acct_type in (0,1,2) /*List all items which are not groups, or deleted - acct_type = 3=deleted*/
The table structure:
CREATE TABLE test_table
acct_id INTEGER NOT NULL ,
field1 VARCHAR2(255) NOT NULL ,
field2 VARCHAR2(255) NOT NULL ,
field3 INTEGER NULL ,
acct_group INTEGER NULL ,
field5 CHAR(3) NOT NULL ,
acct_type INTEGER NOT NULL ,
field6 INTEGER NULL ,
CONSTRAINT test_table_pk PRIMARY KEY (acct_id)
CREATE INDEX test_table_if3 ON test_table
acct_group ASC
CREATE INDEX test_table_if4 ON test_table
acct_type ASC
Anyone have any ideas how to make this faster?
(The equivilant query on a different competitors RDBMS takes 0.5 sec to return this same data - on a machine with the same IO power)
It does seem a little odd that the CPU resource required is: CPU Cost: 203,799,167 ?
I have tested with an IOT base table - no real perf. diff benefits either.
Test Data can be sent (In a CSV file - 3Mb) - email: [email protected].
Thanks in advance,
Dan.

Answer 1

Hi All,
is the magic number 4 in :
select 0, null,4 from dual) ba
start with (ba.acct_type =4) /*acct_type =4- Group: List only the groups then find the children*/
-- Can I add this data to the base table data - unfortunatly not, without making the application break.
--The union part will always be 0 - acct_id, acct_type = 4, group_id = NULL [This will always be the case, never changing]
1) Yes UNION /UNION ALL is being problematic. Table has 105,000 rows.
2) There is an index on ACCT_ID/ACCT_GROUP/ACCT_TYPE - all of which are individual indexes.
3) ACCT_ID - is a PK Index (Apologies for not including this in the post)
4) ACCOUNT_GROUPS is a Common Table Expression (CTE): http://www.dba-oracle.com/t_with_clause.htm
5) Indexes are present. Why oh Why cannot oracle join to a CONNECT BY when given input parameters on the view definition instead of to the materialised data-set ?
Materialised view - cannot be considered this is an enterprise edition feature mostly.
When running the same type of query on a different RDBMS System, the following:
select * from <view_name> where acct_group = X
This results in the query inserting acct_group into the middle of the CTE / connect-by expression - so it does a fast lookup inside the Connect-by-clause limiting the results set to the perform the outer joins. This other RDBMS SYstem only takes about 180 ms to perform this query, where the oracle engine takes 2-3 seconds (as it materialises the CTE fully and does not insert the acct_group in the middle of the connect-by expression).
This query only works efficiently in Oracle when converted to a User Defined Function - e.g. specifying the @acct_group as an input variable:
with account_groups
as (select connect_by_root ba.acct_id orig_id, ba.acct_id, ba.acct_group
from (select acct_id, acct_group, acct_type
from bs_accts b1
union
select 0, null,4 from dual) ba
start with (ba.acct_type =4 and acct_id = @acct_group)
connect by ba.acct_group = prior ba.acct_id
The performance of the view/UDF should be the same - I am happy to say oracles RDBMS engine in this case is generating BAD Plans for a connect by and failing badly in optimising the query being asked of it.
If people want test data: email: [email protected]
So to be absolutly clear, the following performs: (Returns data in about 422 ms)--
with account_groups
as (select connect_by_root ba.acct_id orig_id, ba.acct_id, ba.acct_group
from (select acct_id, acct_group, acct_type
from bs_accts b1
) ba
start with (ba.acct_type =4 and ba.acct_id = 1000001) /*UDF - uses @acct_group variable here*/
connect by ba.acct_group = prior ba.acct_id
select ag.orig_id acct_group, ag.acct_id
from account_groups ag
join bs_accts unio
on ag.acct_id = unio.acct_id
where unio.acct_type in (0,1,2)
and ag.orig_id = 1000001;
Where the following: (Returns data in about 2 seconds)
with account_groups
as (select connect_by_root ba.acct_id orig_id, ba.acct_id, ba.acct_group
from (select acct_id, acct_group, acct_type
from bs_accts b1
) ba
start with (ba.acct_type =4)
connect by ba.acct_group = prior ba.acct_id
select ag.orig_id acct_group, ag.acct_id
from account_groups ag
join bs_accts unio
on ag.acct_id = unio.acct_id
where unio.acct_type in (0,1,2)
and ag.orig_id =1000001 ;
* The above query generates a poor plan. Due to what I believe is an oracle engine optimisation deficiency.
Edited by: user626167 on 01-Sep-2008 06:08