Hash Join over NL
My query reads like:
create table XYZ nologging as
select [] from
driving_table 1800K rows,
lkp1 235K
LKP2 160K
lkp3 80K
lkp4 4K
lkp5 4K
lkp6 4K
where [Driving_Table joins with each of the Lookups to Get lookup Keys]..as many rows in result set as in LArge_Driving_Table..all inner joins
And I keep getting NEsted Loops all over.
Optimizer settigs:
-CHOOSE
-hash_area_enabled=true
-hash_area_size =31M (I tried increasing it to 250 M i.e. greater than size of driving table), but does not help
- I have tried to increase optimize_increase_cost_adj to 10000, does not help
-Have tried using USE_HASH. Dones not work , but maybe I am not setting it correctly
AM I hitting bug 2502657 ? Any pointers on using USE_HASH correctly ?
- Version 9.2.0.3
- All tables have been analyzed
CTAS Query:
create table temp_delete as
select CSU.CUSTOMER_SITE_USE_CODE CUST_SITE_CUSTOMER_SITE__1,
CSU.CUSTOMER_SITE_USE_KEY CUST_SITE_CUSTOMER_SITE_,
7 DAILY_LOAD_FL,
ENTR_DT.DAY_KEY ENTR_DT_DAY_KEY,
ACT_DT.DAY_KEY ACT_DT_DAY_KEY,
INV.ED_QTY INVOICE_ED_QTY,
INV.ED_INVOICE_NO INVOICE_ED_INVOICE_NO,
nvl( ITM.ITM_KEY ,-1) ITM_KEY,
-- to_number( INV.ED_SO_LINE ) ITM_LN,
1 PRCS_DT_KEY,
-1 RELS_DT_KEY,
PROM_DT.DAY_KEY PROM_DT_DAY_KEY,
-1 SHIP_DT_KEY,
CSU.CUSTOMER_SALES_CODE_NUMBER CUST_SITE_CUSTOMER_SALES_1,
CSU.CUSTOMER_SALES_CODE_KEY CUST_SITE_CUSTOMER_SALES,
INV.ED_VALUE INVOICE_ED_VALUE,
ORD.SLSORD_GK SLSORD_WH_SLSORD_GK,
1 SLSORD_STAT_KEY,
INV.ED_TOTAL_COST INVOICE_ED_TOTAL_COST,
case
when ACT_DT.OFFSET_DAY = 0
then 'Y'
else NULL
end TODAY_FLG,
INV.ED_HED_CD INVOICE_ED_HED_CD
FROM
INVOICE_EU_ST INV, 1800 K
SLSORD_WH ORD, 235 K,
CUST_SITE_USE_WH CSU, 83 K
ITM_WH ITM, 155 K
TIME_WH ACT_DT, 4K
TIME_WH ENTR_DT, 4K
TIME_WH PROM_DT 4K
WHERE
INV.SRC_ID =7 and
INV.ED_ACTIVITY_DATE = ACT_DT.DAY_CHAR and
INV.ED_LINE_ENTRY_DATE = ENTR_DT.DAY_CHAR and
INV.ED_LINE_PROMISE_DATE= PROM_DT.DAY_CHAR and
INV.ED_SALES_ORDER =ORD.SLSORD_ID and
ORD.SRC_ID = 7 and
INV.ED_CUST_ID=CSU.CUSTOMER_SITE_USE_CODE and
CSU.SOURCE_ID=7 and
INV.ORGANIZATION_ID=ITM.ORGANIZATION_ID and
INV.ITM_CD =ITM.ITM_CD and
ITM.SRC_ID=7
PLAN_TABLE_OUTPUT
| Id | Operation | Name | Rows
| 0 | CREATE TABLE STATEMENT | | 4
| 1 | LOAD AS SELECT | |
| 2 | NESTED LOOPS | | 4
| 3 | NESTED LOOPS | | 4
| 4 | NESTED LOOPS | | 4
| 5 | NESTED LOOPS | | 4
| 6 | NESTED LOOPS | | 4
| 7 | NESTED LOOPS | | 8
|* 8 | TABLE ACCESS FULL | SLSORD_WH | 1
|* 9 | TABLE ACCESS BY INDEX ROWID | INVOICE_EU_ST | 8
|* 10 | INDEX RANGE SCAN | I2 | 8
| 11 | TABLE ACCESS BY GLOBAL INDEX ROWID| ITM_WH | 1
|* 12 | INDEX UNIQUE SCAN | ITM_WH_UK | 2
| 13 | TABLE ACCESS BY GLOBAL INDEX ROWID | CUST_SITE_USE_WH | 1
|* 14 | INDEX UNIQUE SCAN | CUST_SITE_USE_WH_UK | 3
| 15 | TABLE ACCESS BY INDEX ROWID | TIME_WH | 1
|* 16 | INDEX UNIQUE SCAN | TIME_CAL_UK | 1
| 17 | TABLE ACCESS BY INDEX ROWID | TIME_WH | 1
|* 18 | INDEX UNIQUE SCAN | TIME_CAL_UK | 1
| 19 | TABLE ACCESS BY INDEX ROWID | TIME_WH | 1
|* 20 | INDEX UNIQUE SCAN | TIME_CAL_UK | 1
Predicate Information (identified by operation id):
8 - filter("ORD"."SRC_ID"=7)
9 - filter("INV"."SRC_ID"=7)
10 - access("INV"."ED_SALES_ORDER"="ORD"."SLSORD_ID")
12 - access("ITM"."SRC_ID"=7 AND "INV"."ORGANIZATION_ID"="ITM"."ORGANIZATION_I
14 - access("INV"."ED_CUST_ID"="CSU"."CUSTOMER_SITE_USE_CODE" AND "CSU"."SOURC
16 - access("INV"."ED_LINE_PROMISE_DATE"="PROM_DT"."DAY_CHAR")
18 - access("INV"."ED_LINE_ENTRY_DATE"="ENTR_DT"."DAY_CHAR")
20 - access("INV"."ED_ACTIVITY_DATE"="ACT_DT"."DAY_CHAR")
Note: cpu costing is off
40 rows selected
Similar Messages
-
Why optimizer prefers nested loop over hash join?
What do I look for if I want to find out why the server prefers a nested loop over hash join?
The server is 10.2.0.4.0.
The query is:
SELECT p.*
FROM t1 p, t2 d
WHERE d.emplid = p.id_psoft
AND p.flag_processed = 'N'
AND p.desc_pool = :b1
AND NOT d.name LIKE '%DUPLICATE%'
AND ROWNUM < 2tkprof output is:
Production
call count cpu elapsed disk query current rows
Parse 1 0.01 0.00 0 0 4 0
Execute 1 0.00 0.01 0 4 0 0
Fetch 1 228.83 223.48 0 4264533 0 1
total 3 228.84 223.50 0 4264537 4 1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 108 (SANJEEV)
Rows Row Source Operation
1 COUNT STOPKEY (cr=4264533 pr=0 pw=0 time=223484076 us)
1 NESTED LOOPS (cr=4264533 pr=0 pw=0 time=223484031 us)
10401 TABLE ACCESS FULL T1 (cr=192 pr=0 pw=0 time=228969 us)
1 TABLE ACCESS FULL T2 (cr=4264341 pr=0 pw=0 time=223182508 us)Development
call count cpu elapsed disk query current rows
Parse 1 0.01 0.00 0 0 0 0
Execute 1 0.00 0.01 0 4 0 0
Fetch 1 0.05 0.03 0 512 0 1
total 3 0.06 0.06 0 516 0 1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 113 (SANJEEV)
Rows Row Source Operation
1 COUNT STOPKEY (cr=512 pr=0 pw=0 time=38876 us)
1 HASH JOIN (cr=512 pr=0 pw=0 time=38846 us)
51 TABLE ACCESS FULL T2 (cr=492 pr=0 pw=0 time=30230 us)
861 TABLE ACCESS FULL T1 (cr=20 pr=0 pw=0 time=2746 us)sanjeevchauhan wrote:
What do I look for if I want to find out why the server prefers a nested loop over hash join?
The server is 10.2.0.4.0.
The query is:
SELECT p.*
FROM t1 p, t2 d
WHERE d.emplid = p.id_psoft
AND p.flag_processed = 'N'
AND p.desc_pool = :b1
AND NOT d.name LIKE '%DUPLICATE%'
AND ROWNUM < 2
You've got already some suggestions, but the most straightforward way is to run the unhinted statement in both environments and then force the join and access methods you would like to see using hints, in your case probably "USE_HASH(P D)" in your production environment and "FULL(P) FULL(D) USE_NL(P D)" in your development environment should be sufficient to see the costs and estimates returned by the optimizer when using the alternate access and join patterns.
This give you a first indication why the optimizer thinks that the chosen access path seems to be cheaper than the obviously less efficient plan selected in production.
As already mentioned by Hemant using bind variables complicates things a bit since EXPLAIN PLAN is not reliable due to bind variable peeking performed when executing the statement, but not when explaining.
Since you're already on 10g you can get the actual execution plan used for all four variants using DBMS_XPLAN.DISPLAY_CURSOR which tells you more than the TKPROF output in the "Row Source Operation" section regarding the estimates and costs assigned.
Of course the result of your whole exercise might be highly dependent on the actual bind variable value used.
By the way, your statement is questionable in principle since you're querying for the first row of an indeterministic result set. It's not deterministic since you've defined no particular order so depending on the way Oracle executes the statement and the physical storage of your data this query might return different results on different runs.
This is either an indication of a bad design (If the query is supposed to return exactly one row then you don't need the ROWNUM restriction) or an incorrect attempt of a Top 1 query which requires you to specify somehow an order, either by adding a ORDER BY to the statement and wrapping it into an inline view, or e.g. using some analytic functions that allow you specify a RANK by a defined ORDER.
This is an example of how a deterministic Top N query could look like:
SELECT
FROM
SELECT p.*
FROM t1 p, t2 d
WHERE d.emplid = p.id_psoft
AND p.flag_processed = 'N'
AND p.desc_pool = :b1
AND NOT d.name LIKE '%DUPLICATE%'
ORDER BY <order_criteria>
WHERE ROWNUM <= 1;Regards,
Randolf
Oracle related stuff blog:
http://oracle-randolf.blogspot.com/
SQLTools++ for Oracle (Open source Oracle GUI for Windows):
http://www.sqltools-plusplus.org:7676/
http://sourceforge.net/projects/sqlt-pp/ -
Simultaneous hash joins of the same large table with many small ones?
Hello
I've got a typical data warehousing scenario where a HUGE_FACT table is to be joined with numerous very small lookup/dimension tables for data enrichment. Joins with these small lookup tables are mutually independent, which means that the result of any of these joins is not needed to perform another join.
So this is a typical scenario for a hash join: the lookup table is converted into a hashed map in RAM memory, fits there without drama cause it's small and a single pass over the HUGE_FACT suffices to get the results.
Problem is, so far as I can see it in the query plan, these hash joins are not executed simultaneously but one after another, which renders Oracle to do the full scan of the HUGE_FACT (or any intermediary enriched form of it) as many times as there are joins.
Questions:
- is my interpretation correct that the mentioned joins are sequential, not simultaneous?
- if this is the case, is there any possibility to force Oracle to perform these joins simultaneously (building more than one hashed map in memory and doing the single pass over the HUGE_FACT while looking up in all of these hashed maps for matches)? If so, how to do it?
Please note that the parallel execution of a single join at a time is not the matter of the question.
Database version is 10.2.
Thank you very much in advance for any response.user13176880 wrote:
Questions:
- is my interpretation correct that the mentioned joins are sequential, not simultaneous?Correct. But why do you think this is an issue? Because of this:
which renders Oracle to do the full scan of the HUGE_FACT (or any intermediary enriched form of it) as many times as there are joins.That is (should not be) true. Oracle does one pass of the big table, and then sequentually joins to each of the hashmaps (of each of the smaller tables).
If you show us the execution plan, we can be sure of this.
- if this is the case, is there any possibility to force Oracle to perform these joins simultaneously (building more than one hashed map in memory and doing the single pass over the HUGE_FACT while looking up in all of these hashed maps for matches)? If so, how to do it?Yes there is. But again you should not need to resort to such a solution. What you can do is use subquery factoring (WITH clause) in conjunction with the MATERIALIZE hint to first construct the cartesian join of all of the smaller (dimension) tables. And then join the big table to that. -
Seeking advice on a heavy hash join
We have to self-join a 190 million row table 160 times. On our production 9i database we give "RULE" hint and it finishes in about an hour using nested loop join, On our test 10g database, since "RULE" is not available any more. The optimiser chooses to use hash join (160 levels). And query never finishes in a reasonable time. We have gradually increase the hash_area_size from 8M to 512M, thinking that will help. But apparently it does not. Can anyone provide suggestions? Thanks.
There is an approach using Analytics Functions that may be useful here. The idea is to get all the data values required with 1 reference to Tab2 rather than 160, and let Analytics do the work of building the result set.
The goal here is to 'never do multiple references to the same table where 1 reference can suffice (usually with the aid of Analytics)'.
There are 2 variations depending on whether the ID values here (0,10,20,30,...) always follow an ascending pattern or if they don't. The second example is more generic and will also cover the ascending case.
name ID val ID val ID val ID val ID val
name1 0 1 10 1 20 1 30 1 40 1
-- Performance Summary - This demo used a max value of 1600 vs 15000
for a net number of rows of 2.5 million versus 225 million.
Any of the test views ( replacing 5,10, or 160 tab2 references) using the Analytics function used at most 4099 consistent gets. The original approach used 20553 consistent gets for 5 tab2 references, 41016 consistent gets for 10 tab2 references. This was in 10g, doing the hash join. I did try alter session set optimizer_mode = rule (just for test purposes) and that resulted in 46462 consistent gets for the 10 tab2 reference view while it did a merge join operation.
Autotrace for the Analytics version to replace the original 160 table sql.
JT(147)@JTDB10G>select * from analytics_2_joins;
1600 rows selected.
Statistics
0 recursive calls
0 db block gets
4099 consistent gets
3710 physical reads
0 redo size
1112904 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1600 rows processed
-- Minimal examples:
--As always, test thoroughly before using in production.
select name,
id0, value value0,
id1, value value1,
id2, value value2,
id3, value value3,
id4, value value4
from ( select name, id, value,
lead(id,0 ) over(partition by name, value order by rn ) id0 ,
lead(id,1 ) over(partition by name, value order by rn ) id1 ,
lead(id,2 ) over(partition by name, value order by rn ) id2 ,
lead(id,3 ) over(partition by name, value order by rn ) id3 ,
lead(id,4 ) over(partition by name, value order by rn ) id4 ,
rn
from( select tab1.name, i0.id, i0.tab1value value,
(row_number() over (partition by tab1value order by i0.id )) -1 rn
from tab1, tab2 i0 where tab1.name='name1' and i0.tab1value=tab1.value
and i0.id in (0,10,20,30,40)
)) where rn = 0;
-- execute a verions of the smaller analytics approch with bind variables
-- referencing the binds within the numbered in-line view is needed only if
-- the id0, id1, id2 values do not follow the ascending pattern shown in the
-- example. This will handle case where id0 = 30, id2 =20, id4 = 40 , etc.
variable l_id0 number;
variable l_id1 number;
variable l_id2 number;
variable l_id3 number;
variable l_id4 number;
exec :l_id1 := 0;
exec :l_id3 := 10;
exec :l_id2 := 20;
exec :l_id0 := 30;
exec :l_id4 := 40;
select name, bind_rn,
id0, value value0,
id1, value value1,
id2, value value2,
id3, value value3,
id4, value value4
from ( select name, id, value,
lead(id,0 ) over(partition by name, value order by bind_rn ) id0 ,
lead(id,1 ) over(partition by name, value order by bind_rn ) id1 ,
lead(id,2 ) over(partition by name, value order by bind_rn ) id2 ,
lead(id,3 ) over(partition by name, value order by bind_rn ) id3 ,
lead(id,4 ) over(partition by name, value order by bind_rn ) id4 ,
bind_rn
from( select tab1.name, i0.id, i0.tab1value value, bind_rn
from tab1, tab2 i0,
(select 0 bind_rn, :l_id0 arg_value from dual union
select 1 , :l_id1 from dual union
select 2 , :l_id2 from dual union
select 3 , :l_id3 from dual union
select 4 , :l_id4 from dual ) table_of_args
where tab1.name='name1' and i0.tab1value=tab1.value
-- and i0.id in (0,10,20,30,40)
and i0.id = table_of_args.arg_value
)) where bind_rn = 0;
-- Full Test Case
-- table setup
drop table tab1;
drop table tab2;
create table tab1(name varchar2(100), value number) pctfree 0;
create table tab2(id number, tab1value number) pctfree 0;
begin
for x in 0 .. 1600 loop
for y in 1 .. 1600 loop
insert into tab1 values ('name' || x, y);
end loop;
end loop;
end;
-- 15000 results in 225,000,000
-- 1600 results in 2,560,000
begin
for x in 0 .. 1600 loop
for y in 1 .. 1600 loop
insert into tab2 values (x, y);
end loop;
end loop;
end;
commit;
CREATE BITMAP INDEX NAME_BITMAP ON TAB1(NAME);
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME => 'JTOMMANEY',TABNAME => 'TAB1', -
estimate_percent => 20, CASCADE => TRUE);
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME => 'JTOMMANEY',TABNAME => 'TAB2', -
estimate_percent => 20, CASCADE => TRUE);
alter session set optimizer_mode = 'RULE';
-- set up some views both the original approach, and the analytis approach
create view original_5_tab2_tables_join as
select tab1.name name,
i0.id id0, i0.tab1value value0,
i1.id id1, i1.tab1value value1,
i2.id id2, i2.tab1value value2,
i3.id id3, i3.tab1value value3,
i4.id id4, i4.tab1value value4
from tab1,
tab2 i0, tab2 i1, tab2 i2, tab2 i3, tab2 i4
where tab1.name='name1'
and (i0.id=0 and i0.tab1value=tab1.value)
and (i1.id=10 and i1.tab1value=tab1.value)
and (i2.id=20 and i2.tab1value=tab1.value)
and (i3.id=30 and i3.tab1value=tab1.value)
and (i4.id=40 and i4.tab1value=tab1.value);
create view replace_5_tab2_joins as
select name,
id0, value value0,
id1, value value1,
id2, value value2,
id3, value value3,
id4, value value4
from ( select name, id, value,
lead(id,0 ) over(partition by name, value order by rn ) id0 ,
lead(id,1 ) over(partition by name, value order by rn ) id1 ,
lead(id,2 ) over(partition by name, value order by rn ) id2 ,
lead(id,3 ) over(partition by name, value order by rn ) id3 ,
lead(id,4 ) over(partition by name, value order by rn ) id4 ,
rn from( select tab1.name, i0.id, i0.tab1value value,
(row_number() over (partition by tab1value order by i0.id )) -1 rn
from tab1, tab2 i0 where tab1.name='name1' and i0.tab1value=tab1.value
and i0.id in (0,10,20,30,40)
)) where rn = 0;
create view original_10_tab2_tables_join as
select tab1.name name,
i0.id id0, i0.tab1value value0,
i1.id id1, i1.tab1value value1,
i2.id id2, i2.tab1value value2,
i3.id id3, i3.tab1value value3,
i4.id id4, i4.tab1value value4,
i5.id id5, i5.tab1value value5,
i6.id id6, i6.tab1value value6,
i7.id id7, i7.tab1value value7,
i8.id id8, i8.tab1value value8,
i9.id id9, i9.tab1value value9
from tab1,
tab2 i0, tab2 i1, tab2 i2, tab2 i3, tab2 i4,
tab2 i5, tab2 i6, tab2 i7, tab2 i8, tab2 i9
where tab1.name='name1'
and (i0.id=0 and i0.tab1value=tab1.value)
and (i1.id=10 and i1.tab1value=tab1.value)
and (i2.id=20 and i2.tab1value=tab1.value)
and (i3.id=30 and i3.tab1value=tab1.value)
and (i4.id=40 and i4.tab1value=tab1.value)
and (i5.id=50 and i5.tab1value=tab1.value)
and (i6.id=60 and i6.tab1value=tab1.value)
and (i7.id=70 and i7.tab1value=tab1.value)
and (i8.id=80 and i8.tab1value=tab1.value)
and (i9.id=90 and i9.tab1value=tab1.value);
create view replace_10_tab2_joins as
select name,
id0, value value0,
id1, value value1,
id2, value value2,
id3, value value3,
id4, value value4,
id5, value value5,
id6, value value6,
id7, value value7,
id8, value value8,
id9, value value9
from ( select name, id, value,
lead(id,0 ) over(partition by name, value order by rn ) id0 ,
lead(id,1 ) over(partition by name, value order by rn ) id1 ,
lead(id,2 ) over(partition by name, value order by rn ) id2 ,
lead(id,3 ) over(partition by name, value order by rn ) id3 ,
lead(id,4 ) over(partition by name, value order by rn ) id4 ,
lead(id,5 ) over(partition by name, value order by rn ) id5 ,
lead(id,6 ) over(partition by name, value order by rn ) id6 ,
lead(id,7 ) over(partition by name, value order by rn ) id7 ,
lead(id,8 ) over(partition by name, value order by rn ) id8 ,
lead(id,9 ) over(partition by name, value order by rn ) id9 ,
rn from( select tab1.name, i0.id, i0.tab1value value,
(row_number() over (partition by tab1value order by i0.id )) -1 rn
from tab1, tab2 i0 where tab1.name='name1' and i0.tab1value=tab1.value
and i0.id in (0,10,20,30,40,50,60,70,80,90)
)) where rn = 0;
-- set up some views both the original approach, and the analytics approach
spool cr_v1.sql may need to clean up heading, linefeed from created file
begin
dbms_output.put_line('create or replace view original_160_joins as select /*+ rule */ tab1.name ');
for x in 0 .. 160 loop
dbms_output.put_line( ',i' || x || '.id id' || x || ' ,i' || x || '.tab1value value' || x ) ;
end loop;
dbms_output.put_line('from tab1' );
for x in 0 .. 160 loop
dbms_output.put_line( ',tab2 i' || x ) ;
end loop;
dbms_output.put_line(' where tab1.name = ''name1''' );
for x in 0 .. 160 loop
dbms_output.put_line( ' and i' || x || '.id=' || (x * 10) || ' and i' || x || '.tab1value=tab1.value ' ) ;
end loop;
dbms_output.put_line( ' ;');
end;
--spool off
--@cr_v1.sql
spool cr_v2.sql may need to clean up heading, linefeed from created file
begin
dbms_output.put_line('create or replace view analytics_2_joins as select name ');
for x in 0 .. 160 loop
dbms_output.put_line( ',id' || x || ', value value' || x ) ;
end loop;
dbms_output.put_line('from ( select name, id, value ' );
for x in 0 .. 160 loop
dbms_output.put_line( ',lead(id,' || x || ') over(partition by name, value order by rn ) id' || x ) ;
end loop;
dbms_output.put_line(' , rn from( select tab1.name, i0.id, i0.tab1value value, ');
dbms_output.put_line(' (row_number() over (partition by tab1value order by i0.id )) -1 rn ');
dbms_output.put_line(' from tab1, tab2 i0 where tab1.name=''name1'' and i0.tab1value=tab1.value and i0.id in ( ');
for x in 0 .. 159 loop
dbms_output.put_line( (x * 10) || ',' ) ;
end loop;
dbms_output.put_line( ' 1600))) where rn = 0;');
end;
--spool off
--@cr_v2.sql
-- We now have 6 views established
-- Original Approach Analytics Approach w/ 1 tab2 reference
-- 5 tab2s original_5_tab2_tables_join replace_5_tab2_joins
-- 10 tab2s original_10_tab2_tables_join replace_10_tab2_joins
--160 tab2s original_160_joins analytics_2_joins
-- plus we will use call the version with bind variables, but not from a view.
-- Data validation:
select 'orig_minus_new: ' || count(*) from
( select * from original_5_tab2_tables_join minus select * from replace_5_tab2_joins ) union
select 'new_minus_orig: ' || count(*) from
( select * from replace_5_tab2_joins minus select * from original_5_tab2_tables_join );
select 'orig_minus_new: ' || count(*) from
( select * from original_10_tab2_tables_join minus select * from replace_10_tab2_joins ) union
select 'new_minus_orig: ' || count(*) from
( select * from replace_10_tab2_joins minus select * from original_10_tab2_tables_join );
select 'orig_minus_new: ' || count(*) from
( select * from original_160_joins minus select * from analytics_2_joins );
select 'new_minus_orig: ' || count(*) from
( select * from analytics_2_joins minus select * from original_160_joins );
-- Performance test
alter session set workarea_size_policy=manual ;
alter session set sort_area_size = 64000000;
alter session set hash_area_size = 64000000;
set autotrace traceonly stat
select * from original_5_tab2_tables_join;
select * from replace_5_tab2_joins;
select * from original_10_tab2_tables_join;
select * from replace_10_tab2_joins;
select * from analytics_2_joins;
--select * from original_160_joins;
-- execute a verions of the smaller analytics approch with bind variables
-- referencing the binds within the numbered in-line view is needed only if
-- the id0, id1, id2 values do not follow the ascending pattern shown in the
-- example. This will handle case where id0 = 30, id2 =20, id4 = 40 , etc.
variable l_id0 number;
variable l_id1 number;
variable l_id2 number;
variable l_id3 number;
variable l_id4 number;
exec :l_id1 := 0;
exec :l_id3 := 10;
exec :l_id2 := 20;
exec :l_id0 := 30;
exec :l_id4 := 40;
select name, bind_rn,
id0, value value0,
id1, value value1,
id2, value value2,
id3, value value3,
id4, value value4
from ( select name, id, value,
lead(id,0 ) over(partition by name, value order by bind_rn ) id0 ,
lead(id,1 ) over(partition by name, value order by bind_rn ) id1 ,
lead(id,2 ) over(partition by name, value order by bind_rn ) id2 ,
lead(id,3 ) over(partition by name, value order by bind_rn ) id3 ,
lead(id,4 ) over(partition by name, value order by bind_rn ) id4 ,
bind_rn
from( select tab1.name, i0.id, i0.tab1value value, bind_rn
from tab1, tab2 i0,
(select 0 bind_rn, :l_id0 arg_value from dual union
select 1 , :l_id1 from dual union
select 2 , :l_id2 from dual union
select 3 , :l_id3 from dual union
select 4 , :l_id4 from dual ) table_of_args
where tab1.name='name1' and i0.tab1value=tab1.value
-- and i0.id in (0,10,20,30,40)
and i0.id = table_of_args.arg_value
)) where bind_rn = 0;
JT(147)@JTDB10G>select * from original_5_tab2_tables_join;
1600 rows selected.
Statistics
8 recursive calls
2 db block gets
20553 consistent gets
18555 physical reads
0 redo size
52052 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1600 rows processed
JT(147)@JTDB10G>select * from replace_5_tab2_joins;
1600 rows selected.
Statistics
8 recursive calls
2 db block gets
4101 consistent gets
3710 physical reads
0 redo size
52052 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1600 rows processed
JT(147)@JTDB10G>select * from original_10_tab2_tables_join;
1600 rows selected.
Statistics
0 recursive calls
0 db block gets
41016 consistent gets
37115 physical reads
0 redo size
85636 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1600 rows processed
JT(147)@JTDB10G>select * from replace_10_tab2_joins;
1600 rows selected.
Statistics
0 recursive calls
0 db block gets
4099 consistent gets
3710 physical reads
0 redo size
85636 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1600 rows processed
JT(147)@JTDB10G>select * from analytics_2_joins;
1600 rows selected.
Statistics
0 recursive calls
0 db block gets
4099 consistent gets
3710 physical reads
0 redo size
1112904 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1600 rows processed
JT(147)@JTDB10G>--select * from original_160_joins;
JT(147)@JTDB10G>
JT(147)@JTDB10G>----------------------------------------------------------------------------------------
JT(147)@JTDB10G>-- execute a verions of the smaller analytics approch with bind variables
JT(147)@JTDB10G>-- referencing the binds within the numbered in-line view is needed only if
JT(147)@JTDB10G>-- the id0, id1, id2 values do not follow the ascending pattern shown in the
JT(147)@JTDB10G>-- example. This will handle case where id0 = 30, id2 =20, id4 = 40 , etc.
JT(147)@JTDB10G>----------------------------------------------------------------------------------------
JT(147)@JTDB10G>
JT(147)@JTDB10G>variable l_id0 number;
JT(147)@JTDB10G>variable l_id1 number;
JT(147)@JTDB10G>variable l_id2 number;
JT(147)@JTDB10G>variable l_id3 number;
JT(147)@JTDB10G>variable l_id4 number;
JT(147)@JTDB10G>
JT(147)@JTDB10G>exec :l_id1 := 0;
PL/SQL procedure successfully completed.
JT(147)@JTDB10G>exec :l_id3 := 10;
PL/SQL procedure successfully completed.
JT(147)@JTDB10G>exec :l_id2 := 20;
PL/SQL procedure successfully completed.
JT(147)@JTDB10G>exec :l_id0 := 30;
PL/SQL procedure successfully completed.
JT(147)@JTDB10G>exec :l_id4 := 40;
PL/SQL procedure successfully completed.
JT(147)@JTDB10G>
JT(147)@JTDB10G>select name, bind_rn,
2 id0, value value0,
3 id1, value value1,
4 id2, value value2,
5 id3, value value3,
6 id4, value value4
7 from ( select name, id, value,
8 lead(id,0 ) over(partition by name, value order by bind_rn ) id0 ,
9 lead(id,1 ) over(partition by name, value order by bind_rn ) id1 ,
10 lead(id,2 ) over(partition by name, value order by bind_rn ) id2 ,
11 lead(id,3 ) over(partition by name, value order by bind_rn ) id3 ,
12 lead(id,4 ) over(partition by name, value order by bind_rn ) id4 ,
13 bind_rn
14 from( select tab1.name, i0.id, i0.tab1value value, bind_rn
15 from tab1, tab2 i0,
16 (select 0 bind_rn, :l_id0 arg_value from dual union
17 select 1 , :l_id1 from dual union
18 select 2 , :l_id2 from dual union
19 select 3 , :l_id3 from dual union
20 select 4 , :l_id4 from dual ) table_of_args
21 where tab1.name='name1' and i0.tab1value=tab1.value
22 -- and i0.id in (0,10,20,30,40)
23 and i0.id = table_of_args.arg_value
24 )) where bind_rn = 0;
1600 rows selected.
Statistics
1 recursive calls
0 db block gets
4099 consistent gets
3707 physical reads
0 redo size
52111 bytes sent via SQL*Net to client
1250 bytes received via SQL*Net from client
81 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1600 rows processed -
Performance Tuning - remove hash join
Hi Every one,
Can some one help in tuning below query, i have hash join taking around 84%,
SELECT
PlanId
,ReplacementPlanId
FROM
( SELECT
pl.PlanId
,xpl.PlanId ReplacementPlanId
,ROW_NUMBER() OVER(PARTITION BY pl.PlanId ORDER BY xpl.PlanId) RN
FROM [dbo].[Plan] pl
JOIN [dbo].[Plan] xpl
ON pl.PPlanId = xpl.PPlanId
AND pl.Name = xpl.Name
WHERE
pl.SDate > (CONVERT(CHAR(10),DATEADD(M,-12,GETDATE()),120)) AND
xpl.Live = 1
AND pl.TypeId = 7
AND xpl.TypeId = 7
) p
WHERE RN = 1
Thanks in advanceCan you show an execution plan of the query? Is that possible to rewrite the query as
Sorry cannot test it right now
SELECT
PlanId
,ReplacementPlanId
FROM
(SELECT PlanId,Name,SDate,TypeId FROM [dbo].[Plan]) pl
CROSS APPLY
SELECT TOP (1) TypeId ,Live FROM [Plan] xpl
WHERE pl.PPlanId = xpl.PPlanId
AND pl.Name = xpl.Name
) AS Der
WHERE pl.SDate > (CONVERT(CHAR(10),DATEADD(M,-12,GETDATE()),120)) AND
Der.Live = 1
AND pl.TypeId = 7
AND Der.TypeId = 7
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence -
I have an index on CAT_MAP_ID column of STM_RPT_ITEM_PH6_MV but I don't know why it's not using nested look join for 21 rows returned (outer) to join 641k rows in STM_RPT_ITEM_PH6_MV table. I think that's the reason this query is consuming very high TEMP (87GB), can anyone help me explain this:
| Id | Operation | Name | E-Rows | OMem | 1Mem | Used-Mem | Used-Tmp|
| 0 | INSERT STATEMENT | | | | | | |
| 1 | SORT GROUP BY | | 1 | 131M| 3585K| 90M (1)| 118K|
|* 2 | VIEW | | 1 | | | | |
| 3 | SORT UNIQUE | | 1 | 261M| 4982K| 90M (1)| |
|* 4 | WINDOW SORT PUSHED RANK | | 1 | 511M| 6878K| 172M (1)| |
| 5 | NESTED LOOPS OUTER | | 1 | | | | |
| 6 | NESTED LOOPS | | 1 | | | | |
*|* 7 | HASH JOIN | | 1 | 2047M| 75M| 470M (1)| 87M|*
| 8 | MAT_VIEW ACCESS BY INDEX ROWID| BSC_RPT_REBSUM_STRUCT_LI_MV | 21 | | | | |
| 9 | NESTED LOOPS | | 3960 | | | | |
| 10 | NESTED LOOPS | | 185 | | | | |
|* 11 | HASH JOIN | | 2638 | 1465K| 902K| 4966K (0)| |
|* 12 | HASH JOIN | | 4193 | 841K| 841K| 4992K (0)| |
| 13 | TABLE ACCESS FULL | BSC_RPT_REBSUM_TEMP | 4193 | | | | |
| 14 | TABLE ACCESS FULL | BSC_RPT_REBSUM_S1 | 83548 | | | | |
| 15 | MAT_VIEW ACCESS FULL | BSC_RPT_REBSUM_MEM_MV | 52555 | | | | |
|* 16 | TABLE ACCESS BY INDEX ROWID| MN_BUCKET_LINE | 1 | | | | |
|* 17 | INDEX RANGE SCAN | REBATEPAYMENTGRP | 1460 | | | | |
|* 18 | INDEX RANGE SCAN | BSC_RPT_STRUCT_LI_MV_IDX1 | 22 | | | | |
| 19 | MAT_VIEW ACCESS FULL | BSC_RPT_ITEM_PH6_MV *| 641K|* | | | |
|* 20 | INDEX UNIQUE SCAN | MN_10291_PK | 1 | | | | |
| 21 | TABLE ACCESS BY INDEX ROWID | BSC_SPLIT_PMT | 1 | | | | |
|* 22 | INDEX RANGE SCAN | BSC_904200_IDX1 | 1 | | | | |
Predicate Information (identified by operation id):
2 - filter("RPT"."RNK"=1)
4 - filter(ROW_NUMBER() OVER ( PARTITION BY "BUCK_LI"."BUCKET_LINE_ID" ORDER BY
INTERNAL_FUNCTION("ITEM_VW"."BSC_PHLEVEL") DESC )<=1)
7 - access("LIMV"."CAT_MAP_ID"="ITEM_VW"."CAT_MAP_ID" AND "ITEM_VW"."ITEM_ID"="BUCK_LI"."SALE_ITEM_ID")
11 - access("MEM_MV"."PRC_PROGRAM_ID"="S1"."PRC_PROGRAM_ID" AND
"MEM_MV"."PARENT_MEMBER_ID"="S1"."MEMBER_ID_CUST")
12 - access("TEMP"."CONTRACT_ID"="S1"."CONTRACT_ID" AND "TEMP"."REBATE_PMT_ID"="S1"."REBATE_PMT_ID")
16 - filter(("BUCK_LI"."PMT_BENEFIT_ID" IS NOT NULL AND "BUCK_LI"."INCLUSION_TYPE"='INC' AND
"MEM_MV"."CHILD_MEMBER_ID"="BUCK_LI"."SALE_CONTRACTED_CUST_ID"))
17 - access("TEMP"."REBATE_PMT_ID"="BUCK_LI"."REBATE_PMT_ID")
18 - access("LIMV"."PRC_PROGRAM_ID"="S1"."PRC_PROGRAM_ID")
20 - access("PRC"."PRC_PROGRAM_ID"="S1"."PRC_PROGRAM_ID")
22 - access("TEMP"."REBATE_PMT_ID"="SPLIT"."REBATE_PMT_ID")
filter("SPLIT"."REBATE_PMT_ID" IS NOT NULL)Note
- dynamic sampling used for this statement
- Warning: basic plan statistics not available. These are only collected when:
* hint 'gather_plan_statistics' is used for the statement or
* parameter 'statistics_level' is set to 'ALL', at session or system level
Query Is:
INSERT INTO MDK_TAB1
(sale_id, struct_doc_id, pdflg, member_id_cust, paid_end_date, payee, map_name, STM_phlevel, STM_phcode, STM_old_material_num, sold_to_cust_id, member_grp_name,
prc_program_id, STM_r_af_type, rebate_pmt_id_num, rebate_pmt_id, sale_inv_qty, sales, earn_rebate_amt)
SELECT rpt.sale_id, rpt.struct_doc_id, rpt.pdflg, rpt.member_id_cust, rpt.paid_end_date, rpt.payee, rpt.map_name, rpt.STM_phlevel,
rpt.STM_phcode, rpt.STM_old_material_num, rpt.sold_to_cust_id, rpt.member_grp_name, rpt.prc_program_id, rpt.STM_r_af_type, rpt.rebate_pmt_id_num,
rpt.rebate_pmt_id, rpt.sales_units, rpt.sales, SUM (rpt.earn_rebate_amt) earn_rebate_amt
FROM (SELECT DISTINCT buck_li.sale_id, temp.contract_id struct_doc_id,
temp.pdflg, s1.member_id_cust,
temp.paid_end_date, SPLIT.payee,
item_vw.map_name, item_vw.STM_phlevel,
item_vw.STM_phcode, item_vw.STM_old_material_num,
buck_li.sale_contracted_cust_id sold_to_cust_id,
mem_mv.child_name member_grp_name, s1.prc_program_id, s1.STM_r_af_type,
s1.rebate_pmt_id_num, s1.rebate_pmt_id,
buck_li.sale_inv_qty AS sales_units,
NULLIF (buck_li.sale_ext_amt, 0) sales,
NULLIF
(buck_li.STM_payment_amount, 0) earn_rebate_amt,
ROW_NUMBER () OVER (PARTITION BY buck_li.bucket_line_id ORDER BY item_vw.STM_phlevel DESC)
rnk
FROM STM_rpt_rebsum_s1 s1,
mn_bucket_line buck_li,
STM_rpt_item_ph6_mv item_vw,
STM_rpt_rebsum_struct_li_mv limv,
STM_rpt_rebsum_mem_mv mem_mv,
STM_rpt_rebsum_temp temp,
STM_split_pmt SPLIT,
mn_prc_program prc
WHERE temp.contract_id = s1.contract_id
AND temp.rebate_pmt_id = s1.rebate_pmt_id
AND temp.rebate_pmt_id = SPLIT.rebate_pmt_id(+)
AND temp.rebate_pmt_id = buck_li.rebate_pmt_id
AND limv.prc_program_id = s1.prc_program_id
AND prc.prc_program_id = s1.prc_program_id
AND mem_mv.prc_program_id = s1.prc_program_id
AND mem_mv.parent_member_id = s1.member_id_cust
AND *limv.cat_map_id = item_vw.cat_map_id*
AND item_vw.item_id = buck_li.sale_item_id
AND mem_mv.child_member_id = buck_li.sale_contracted_cust_id
AND buck_li.pmt_benefit_id IS NOT NULL
AND buck_li.inclusion_type = 'INC') rpt
WHERE rpt.rnk = 1
GROUP BY rpt.sale_id,
rpt.struct_doc_id,
rpt.pdflg,
rpt.member_id_cust,
rpt.paid_end_date,
rpt.payee,
rpt.map_name,
rpt.STM_phlevel,
rpt.STM_phcode,
rpt.STM_old_material_num,
rpt.sold_to_cust_id,
rpt.member_grp_name,
rpt.prc_program_id,
rpt.STM_r_af_type,
rpt.rebate_pmt_id_num,
rpt.rebate_pmt_id,
rpt.sales_units,
rpt.sales;Edited by: 988590 on Feb 21, 2013 10:33 AMSorry for messing with the code thingy, it's the first timer's mistake, here is the formatted version:
| Id | Operation | Name | E-Rows | OMem | 1Mem | Used-Mem | Used-Tmp|
| 0 | INSERT STATEMENT | | | | | | |
| 1 | SORT GROUP BY | | 1 | 133M| 3616K| 90M (1)| 120K|
|* 2 | VIEW | | 1 | | | | |
| 3 | SORT UNIQUE | | 1 | 266M| 5026K| 90M (1)| |
|* 4 | WINDOW SORT PUSHED RANK | | 1 | 520M| 6933K| 172M (1)| |
| 5 | NESTED LOOPS OUTER | | 1 | | | | |
| 6 | NESTED LOOPS | | 1 | | | | |
|* 7 | HASH JOIN | | 1 | 2047M| 120M| 419M (1)| 93M|
| 8 | MAT_VIEW ACCESS BY INDEX ROWID| BSC_RPT_REBSUM_STRUCT_LI_MV | 21 | | | | |
| 9 | NESTED LOOPS | | 5468 | | | | |
| 10 | NESTED LOOPS | | 255 | | | | |
|* 11 | HASH JOIN | | 2645 | 1465K| 902K| 3386K (0)| |
|* 12 | HASH JOIN | | 4202 | 841K| 841K| 5115K (0)| |
| 13 | TABLE ACCESS FULL | BSC_RPT_REBSUM_TEMP | 4202 | | | | |
| 14 | TABLE ACCESS FULL | BSC_RPT_REBSUM_S1 | 83564 | | | | |
| 15 | MAT_VIEW ACCESS FULL | BSC_RPT_REBSUM_MEM_MV | 52596 | | | | |
|* 16 | TABLE ACCESS BY INDEX ROWID| MN_BUCKET_LINE | 1 | | | | |
|* 17 | INDEX RANGE SCAN | REBATEPAYMENTGRP | 1465 | | | | |
|* 18 | INDEX RANGE SCAN | BSC_RPT_STRUCT_LI_MV_IDX1 | 22 | | | | |
| 19 | MAT_VIEW ACCESS FULL | BSC_RPT_ITEM_PH6_MV | 641K| | | | |
|* 20 | INDEX UNIQUE SCAN | MN_10291_PK | 1 | | | | |
| 21 | TABLE ACCESS BY INDEX ROWID | BSC_SPLIT_PMT | 1 | | | | |
|* 22 | INDEX RANGE SCAN | BSC_904200_IDX1 | 1 | | | | |
Predicate Information (identified by operation id):
2 - filter("RPT"."RNK"=1)
4 - filter(ROW_NUMBER() OVER ( PARTITION BY "BUCK_LI"."BUCKET_LINE_ID" ORDER BY
INTERNAL_FUNCTION("ITEM_VW"."BSC_PHLEVEL") DESC )<=1)
7 - access("LIMV"."CAT_MAP_ID"="ITEM_VW"."CAT_MAP_ID" AND "ITEM_VW"."ITEM_ID"="BUCK_LI"."SALE_ITEM_ID")
11 - access("MEM_MV"."PRC_PROGRAM_ID"="S1"."PRC_PROGRAM_ID" AND
"MEM_MV"."PARENT_MEMBER_ID"="S1"."MEMBER_ID_CUST")
12 - access("TEMP"."CONTRACT_ID"="S1"."CONTRACT_ID" AND "TEMP"."REBATE_PMT_ID"="S1"."REBATE_PMT_ID")
16 - filter(("BUCK_LI"."PMT_BENEFIT_ID" IS NOT NULL AND "BUCK_LI"."INCLUSION_TYPE"='INC' AND
"MEM_MV"."CHILD_MEMBER_ID"="BUCK_LI"."SALE_CONTRACTED_CUST_ID"))
17 - access("TEMP"."REBATE_PMT_ID"="BUCK_LI"."REBATE_PMT_ID")
18 - access("LIMV"."PRC_PROGRAM_ID"="S1"."PRC_PROGRAM_ID")
20 - access("PRC"."PRC_PROGRAM_ID"="S1"."PRC_PROGRAM_ID")
22 - access("TEMP"."REBATE_PMT_ID"="SPLIT"."REBATE_PMT_ID")
filter("SPLIT"."REBATE_PMT_ID" IS NOT NULL)
Note
- dynamic sampling used for this statement
- Warning: basic plan statistics not available. These are only collected when:
* hint 'gather_plan_statistics' is used for the statement or
* parameter 'statistics_level' is set to 'ALL', at session or system level
INSERT INTO BSC_RPT_REBSUM_S3_TEMP
(sale_id, struct_doc_id, pdflg, member_id_cust, paid_end_date, payee, map_name, bsc_phlevel, bsc_phcode, bsc_old_material_num, sold_to_cust_id, member_grp_name,
prc_program_id, bsc_r_af_type, rebate_pmt_id_num, rebate_pmt_id, sale_inv_qty, sales, earn_rebate_amt)
SELECT rpt.sale_id, rpt.struct_doc_id, rpt.pdflg, rpt.member_id_cust, rpt.paid_end_date, rpt.payee, rpt.map_name, rpt.bsc_phlevel,
rpt.bsc_phcode, rpt.bsc_old_material_num, rpt.sold_to_cust_id, rpt.member_grp_name, rpt.prc_program_id, rpt.bsc_r_af_type, rpt.rebate_pmt_id_num,
rpt.rebate_pmt_id, rpt.sales_units, rpt.sales, SUM (rpt.earn_rebate_amt) earn_rebate_amt
FROM (SELECT DISTINCT buck_li.sale_id, temp.contract_id struct_doc_id,
temp.pdflg, s1.member_id_cust,
temp.paid_end_date, SPLIT.payee,
item_vw.map_name, item_vw.bsc_phlevel,
item_vw.bsc_phcode, item_vw.bsc_old_material_num,
buck_li.sale_contracted_cust_id sold_to_cust_id,
mem_mv.child_name member_grp_name, s1.prc_program_id, s1.bsc_r_af_type,
s1.rebate_pmt_id_num, s1.rebate_pmt_id,
buck_li.sale_inv_qty AS sales_units,
NULLIF (buck_li.sale_ext_amt, 0) sales,
NULLIF
(buck_li.bsc_payment_amount, 0) earn_rebate_amt,
ROW_NUMBER () OVER (PARTITION BY buck_li.bucket_line_id ORDER BY item_vw.bsc_phlevel DESC)
rnk
FROM bsc_rpt_rebsum_s1 s1,
mn_bucket_line buck_li,
bsc_rpt_item_ph6_mv item_vw,
bsc_rpt_rebsum_struct_li_mv limv,
bsc_rpt_rebsum_mem_mv mem_mv,
bsc_rpt_rebsum_temp temp,
bsc_split_pmt SPLIT,
mn_prc_program prc
WHERE temp.contract_id = s1.contract_id
AND temp.rebate_pmt_id = s1.rebate_pmt_id
AND temp.rebate_pmt_id = SPLIT.rebate_pmt_id(+)
AND temp.rebate_pmt_id = buck_li.rebate_pmt_id
AND limv.prc_program_id = s1.prc_program_id
AND prc.prc_program_id = s1.prc_program_id
AND mem_mv.prc_program_id = s1.prc_program_id
AND mem_mv.parent_member_id = s1.member_id_cust
AND limv.cat_map_id = item_vw.cat_map_id
AND item_vw.item_id = buck_li.sale_item_id
AND mem_mv.child_member_id = buck_li.sale_contracted_cust_id
AND buck_li.pmt_benefit_id IS NOT NULL
AND buck_li.inclusion_type = 'INC') rpt
WHERE rpt.rnk = 1
GROUP BY rpt.sale_id,
rpt.struct_doc_id,
rpt.pdflg,
rpt.member_id_cust,
rpt.paid_end_date,
rpt.payee,
rpt.map_name,
rpt.bsc_phlevel,
rpt.bsc_phcode,
rpt.bsc_old_material_num,
rpt.sold_to_cust_id,
rpt.member_grp_name,
rpt.prc_program_id,
rpt.bsc_r_af_type,
rpt.rebate_pmt_id_num,
rpt.rebate_pmt_id,
rpt.sales_units,
rpt.sales;
COLUMN_NAME NUM_DISTINCT NUM_NULLS NUM_BUCKETS SAMPLE_SIZE HISTOGRAM
BSC_OLD_MATERIAL_NUM 55394 583842 1 57907 NONE
BSC_NUM_SHEETS 28 0 1 641691 NONE
BSC_PHCODE 69997 207552 1 434160 NONE
BSC_PHLEVEL 6 207552 1 434160 NONE
BSC_PROFIT_CENTER 10 0 1 641691 NONE
BSC_COST 14966 42821 1 598874 NONE
MAP_NAME 61277 0 1 641691 NONE
CAT_MAP_ID 72241 0 1 641691 NONE
ITEM_ID 64299 0 1 641691 NONE Still don't get it why 21 rows hasing with 641k - it looks a bad idea, maybe I should build histograms here?
Any idea. -
Hash join, how to know how much of pga is using?
Good Day everybody...
Is there a way to know if i have enough pga for a given hash join??'
so far i have PGA_AGGREGATE_TARGET = 200MB
WORK AREA SIZE POLICY = AUTO
The advisor from EM looks good, but i know that the query im interested on its expensive.
so reworking the question...
¿how can i get the propper ammount of bytes needed by a given hash join?Its complicated - There are pages of stuff in the manual, histograms, forecast histograms.
But I think you can get a long way with:
SELECT name profile, cnt, decode(total, 0, 0, round(cnt*100/total)) percentage
FROM (SELECT name, value cnt, (sum(value) over ()) total
FROM V$SYSSTAT
WHERE name like 'workarea exec%');
The output of this query might look like the following:
PROFILE CNT PERCENTAGE
workarea executions - optimal 5395 95
workarea executions - onepass 284 5
workarea executions - multipass 0 0
If all executions are optimal, who can ask for more?
Also:
V$PGA_TARGET_ADVICE
This view predicts how the statistics cache hit percentage and over allocation count in V$PGASTAT will be impacted if you change the value of the initialization parameter PGA_AGGREGATE_TARGET. Example 14-8 shows a typical query of this view:
Example 14-8 Querying V$PGA_TARGET_ADVICE
SELECT round(PGA_TARGET_FOR_ESTIMATE/1024/1024) target_mb,
ESTD_PGA_CACHE_HIT_PERCENTAGE cache_hit_perc,
ESTD_OVERALLOC_COUNT
FROM V$PGA_TARGET_ADVICE;
The output of this query might look like the following:
TARGET_MB CACHE_HIT_PERC ESTD_OVERALLOC_COUNT
63 23 367
125 24 30
250 30 3
375 39 0
500 58 0
600 59 0
700 59 0
800 60 0
900 60 0
1000 61 0
1500 67 0
2000 76 0
3000 83 0
4000 85 0
As far as I understand if, if the overallocation count is nonzero, Oracle will override your pga_aggregate_target anyway. (Having said that, dont forget, this parameter is a target, not a hard & fast limit) -
Hi
I would like to know if there is difference between bind join end hash join.
For example If I write sql code in My query tool (in Data Federator Query server administrator XI 3.0) it is traduced in a hashjoin(....);If I set system parameters in right way to produce bind join, hashjoin(...) is present again!
how do i understand the difference?
byeHi,
this is the WebIntelligence forum, but the tool you are describing is Data Federator (Enterprise Information Management)
You will have more luck over here: Data Services and Data Quality
Regards,
H -
I've been asked to check if HASH JOIN is more suitable than NESTED LOOP(which CBO chose by default) for the following query.
SELECT CM_DETAILS.TASK_ID FROM GEN_TYPE, CM_DETAILS WHERE ( ( ( ( ( CM_DETAILS.STAT_CODE < 8 ) AND ( GEN_TYPE.TASK_ID = CM_DETAILS.TASK_ID ) ) AND ( GEN_TYPE.DEST_LOCN_ID = 5 ) ) AND ( GEN_TYPE.COM_ID = 7 ) ) AND ( ( ( CM_DETAILS.CASE_NO = 1 ) OR ( CM_DETAILS.CASE_NO = 3 ) ) OR ( CM_DETAILS.CASE_NO = 9 ) ) )
Both GEN_TYPE and CM_DETAILS tables have over 330,000 rows.
Version: 10g R2
Any thoughts?As gintsp gave you very nice tip but there is initialization parameter " OPTIMIZER_INDEX_COST_ADJ" which cause what path to be chose for CBO,but usually expert says for changing init paramter setting should be at last resort.
It has default value of 100 which indicates to the CBO that indexed access is 100% as costly (i.e., equally costly) as FULL table scan access.
SQL> column plan_plus_exp format a100
SQL> set linesize 1000
SQL> SET AUTOTRACE TRACEONLY
SQL> SELECT e.ename,d.dname
2 FROM emp e,dept d
3 WHERE e.deptno=d.deptno
4 /
14 rows selected.
Execution Plan
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=7 Card=10 Bytes=320)
1 0 HASH JOIN (Cost=7 Card=10 Bytes=320)
2 1 TABLE ACCESS (FULL) OF 'DEPT' (TABLE) (Cost=3 Card=5 Bytes=90)
3 1 TABLE ACCESS (FULL) OF 'EMP' (TABLE) (Cost=3 Card=14 Bytes=196)
Statistics
672 recursive calls
0 db block gets
151 consistent gets
27 physical reads
0 redo size
793 bytes sent via SQL*Net to client
508 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
15 sorts (memory)
0 sorts (disk)
14 rows processed
SQL> /
14 rows selected.
Execution Plan
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=7 Card=10 Bytes=320)
1 0 HASH JOIN (Cost=7 Card=10 Bytes=320)
2 1 TABLE ACCESS (FULL) OF 'DEPT' (TABLE) (Cost=3 Card=5 Bytes=90)
3 1 TABLE ACCESS (FULL) OF 'EMP' (TABLE) (Cost=3 Card=14 Bytes=196)
Statistics
0 recursive calls
0 db block gets
15 consistent gets
0 physical reads
0 redo size
793 bytes sent via SQL*Net to client
508 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
14 rows processed
SQL> SHOW PARAMETER optimizer
NAME TYPE VALUE
optimizer_dynamic_sampling integer 2
optimizer_features_enable string 10.1.0
optimizer_index_caching integer 0
optimizer_index_cost_adj integer 100<--------
optimizer_mode string ALL_ROWS
SQL> ALTER SESSION SET optimizer_index_cost_adj=35
2 /
Session altered.
SQL> SELECT e.ename,d.dname
2 FROM emp e,dept d
3 WHERE e.deptno=d.deptno
4 /
14 rows selected.
Execution Plan
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=5 Card=10 Bytes=320)
1 0 MERGE JOIN (Cost=5 Card=10 Bytes=320)
2 1 TABLE ACCESS (BY INDEX ROWID) OF 'DEPT' (TABLE) (Cost=1 Card=5 Bytes=90)
3 2 INDEX (FULL SCAN) OF 'DEPT_PRIMARY_KEY' (INDEX (UNIQUE)) (Cost=1 Card=5)
4 1 SORT (JOIN) (Cost=4 Card=14 Bytes=196)
5 4 TABLE ACCESS (FULL) OF 'EMP' (TABLE) (Cost=3 Card=14 Bytes=196)
Statistics
1 recursive calls
0 db block gets
11 consistent gets
1 physical reads
0 redo size
733 bytes sent via SQL*Net to client
508 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
14 rows processed
SQL> /
14 rows selected.
Execution Plan
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=5 Card=10 Bytes=320)
1 0 MERGE JOIN (Cost=5 Card=10 Bytes=320)
2 1 TABLE ACCESS (BY INDEX ROWID) OF 'DEPT' (TABLE) (Cost=1 Card=5 Bytes=90)
3 2 INDEX (FULL SCAN) OF 'DEPT_PRIMARY_KEY' (INDEX (UNIQUE)) (Cost=1 Card=5)
4 1 SORT (JOIN) (Cost=4 Card=14 Bytes=196)
5 4 TABLE ACCESS (FULL) OF 'EMP' (TABLE) (Cost=3 Card=14 Bytes=196)
Statistics
0 recursive calls
0 db block gets
11 consistent gets
0 physical reads
0 redo size
733 bytes sent via SQL*Net to client
508 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
14 rows processedKhurram -
HASH JOIN Issue - query performance issue
Hello friends:
I have a nested loop long query. When i execute this on Oracle 9i, the results comes back within no time (less than a sec) but the same query, same schema, the query takes over 20-30secs. Looking at the execution plan (pasted below) - the Oracle 10g is using HASH JOIN compared to NESTED LOOPS (Oracle 9i). The cost on 9i is much less than 10g due to the hash join form 10g --
Here's the explain plan for -- How do i make 10g plan to use the one from 9i--the nested loop?
Oracle 10g:
SORT(AGGREGATE) 1 33
HASH JOIN 25394 2082 68706
MAT_VIEW ACCESS(FULL) GMDB.NODE_ATTRIBUTES ANALYZED 17051 2082 56214
VIEW 8318 1714180 10285080
FILTER
CONNECT BY(WITH FILTERING)
Oracle 9i--
SORT(AGGREGATE) 1 40
NESTED LOOPS 166 65 2600
VIEW 36 65 845
FILTERjust noticed the "CONNECT BY" in the plan. connect by queries are special cases, should you should spell that out loud and clear for us next time. anyway, lots of problems in v10 with connect by:
metalink Note:394358.1
CONNECT BY SQL uses Full Table Scan In 10.2.0.2 - Used Index In earlier versions
solution:
alter session set "_optimizer_connect_by_cost_based"=false;
and bugs related to connect by fixed in the 10.2.0.2 release (other than the bug from the note bug, which was introduced):
metalink Note:358749.1
and
Note:3956141.8
OR filter in CONNECT BY query may be applied at table level (v 10.1.0.3) -
Merge joins turned into expensive hash joins!?
Hello.....
working with two tables....
Table A is where I am bringing data into.....
Table B is where the data resides.
Table B has 32million rows...
I was using just a quick.....select b.whatever from table.a, table.b where primary key and b.table=a.table.....
I could grab a few rows at a time and it would use a merge join and take 2-3 seconds.
Now....(after NO changes) it hangs. In production it is still fast and uses merge joins, but I compared the activity to that of test and it's using hash joins! The hidden parameter _use merge join will force a merge, but then none of my other hash joins (where I want them) will work.
Optimizer dorked up? Whats the scoop?hmm, well your replies are just as ingenius. . .here's a cookie.
10.2.0.4.0 and i put the relevent sql in there enough to ask why over a dblink (not that it matters) why this behavior would all of the sudden change.
Again, simple matter of creating a nested loop over a dblink to grab a few rows from table b to insert them into table a. It used to merge join, now it's hash joining. Just wondering :-) -
Tuning SQL with heavy hash join
Hi all,
My database is 11.1. Rac 4 node.
The sql is so simple.
create table leon_12345
nologging
as
SELECT *
FROM OOOOOOOOOO a
FULL OUTER JOIN PPPPPPPPPPP b ON a.PRMTN_SKID =
b.PRMTN_SKID
AND a.PROD_SKID =
b.PROD_SKID
AND a.MTH_SKID =
b.MTH_SKID
AND a.PRMTD_CATEG_ID =
b.PRMTD_CATEG_ID
AND a.PRMTN_FACT_ID =
b.PRMTN_FACT_IDSince these two tables have DOP settings, the exection plan looks like:
| Id | Operation | Name | E-Rows | OMem | 1Mem | Used-Mem |
| 0 | CREATE TABLE STATEMENT | | | | | |
| 1 | LOAD AS SELECT | | | 256K| 256K| |
| 2 | PX COORDINATOR | | | | | |
| 3 | PX SEND QC (RANDOM) | :TQ10002 | 74M| | | |
| 4 | VIEW | VW_FOJ_0 | 74M| | | |
|* 5 | HASH JOIN FULL OUTER BUFFERED| | 74M| 1926M| 26M| |
| 6 | PX RECEIVE | | 12M| | | |
| 7 | PX SEND HASH | :TQ10000 | 12M| | | |
| 8 | PX BLOCK ITERATOR | | 12M| | | |
|* 9 | TABLE ACCESS FULL | PPPPPPPPPPP | 12M| | | |
| 10 | PX RECEIVE | | 74M| | | |
| 11 | PX SEND HASH | :TQ10001 | 74M| | | |
| 12 | PX BLOCK ITERATOR | | 74M| | | |
|* 13 | TABLE ACCESS FULL | OOOOOOOOOO | 74M| | | |
-----------------------------------------------------------------------------------------------------------I have noticed 8 slave sessions were up for this sql.(9 sessions totally of course)
With execution in parallel, the SQL still runs slowly. I want to know whether the parallel exectuion has some problem here.
Because after running several minutes, I saw four sessions became inactive and only 4 slave sessions were active.
Whether the workload was not distributed evenly or something else? Can I get some ideas on the direction of tuning this sql.
Best regards,
LeonLeon,
There is not much to tune about this SQL. There might be something to tune in your database configuration, so I'd move this question over to the Database General forum.
And while doing that, please be sure to include trace information and wait event information.
Regards,
Rob. -
Query Degradation--Hash Join Degraded
Hi All,
I found one query degradation issue.I am on 10.2.0.3.0 (Sun OS) with optimizer_mode=ALL_ROWS.
This is a dataware house db.
All 3 tables involved are parition tables (with daily partitions).Partitions are created in advance and ELT jobs loads bulk data into daily partitions.
I have checked that CBO is not using local indexes-created on them which i believe,is appropriate because when i used INDEX HINT, elapsed time increses.
I checked giving index hint for all tables one by one but dint get any performance improvement.
Partitions are daily loaded and after loading,partition-level stats are gathered with dbms_stats.
We are collecting stats at partition level(granularity=>'PARTITION').Even after collecting global stats,there is no change in access pattern.Stats gather command is given below.
PROCEDURE gather_table_part_stats(i_owner_name,i_table_name,i_part_name,i_estimate:= DBMS_STATS.AUTO_SAMPLE_SIZE, i_invalidate IN VARCHAR2 := 'Y',i_debug:= 'N')
Only SOT_KEYMAP.IPK_SOT_KEYMAP is GLOBAL.Rest all indexes are LOCAL.
Earlier,we were having BIND PEEKING issue,which i fixed but introducing NO_INVALIDATE=>FALSE in stats gather job.
Here,Partition_name (20090219) is being passed through bind variables.
SELECT a.sotrelstg_sot_ud sotcrct_sot_ud,
b.sotkey_ud sotcrct_orig_sot_ud, a.ROWID stage_rowid
FROM (SELECT sotrelstg_sot_ud, sotrelstg_sys_ud,
sotrelstg_orig_sys_ord_id, sotrelstg_orig_sys_ord_vseq
FROM sot_rel_stage
WHERE sotrelstg_trd_date_ymd_part = '20090219'
AND sotrelstg_crct_proc_stat_cd = 'N'
AND sotrelstg_sot_ud NOT IN(
SELECT sotcrct_sot_ud
FROM sot_correct
WHERE sotcrct_trd_date_ymd_part ='20090219')) a,
(SELECT MAX(sotkey_ud) sotkey_ud, sotkey_sys_ud,
sotkey_sys_ord_id, sotkey_sys_ord_vseq,
sotkey_trd_date_ymd_part
FROM sot_keymap
WHERE sotkey_trd_date_ymd_part = '20090219'
AND sotkey_iud_cd = 'I'
--not to select logical deleted rows
GROUP BY sotkey_trd_date_ymd_part,
sotkey_sys_ud,
sotkey_sys_ord_id,
sotkey_sys_ord_vseq) b
WHERE a.sotrelstg_sys_ud = b.sotkey_sys_ud
AND a.sotrelstg_orig_sys_ord_id = b.sotkey_sys_ord_id
AND NVL(a.sotrelstg_orig_sys_ord_vseq, 1) = NVL(b.sotkey_sys_ord_vseq, 1);
During normal business hr, i found that query takes 5-7 min(which is also not acceptable), but during high load business hr,it is taking 30-50 min.
I found that most of the time it is spending on HASH JOIN (direct path write temp).We have sufficient RAM (64 GB total/41 GB available).
Below is the execution plan i got during normal business hr.
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | Writes | OMem | 1Mem | Used-Mem | Used-Tmp|
| 1 | HASH GROUP BY | | 1 | 1 | 7844K|00:05:28.78 | 16M| 217K| 35969 | | | | |
|* 2 | HASH JOIN | | 1 | 1 | 9977K|00:04:34.02 | 16M| 202K| 20779 | 580M| 10M| 563M (1)| 650K|
| 3 | NESTED LOOPS ANTI | | 1 | 6 | 7855K|00:01:26.41 | 16M| 1149 | 0 | | | | |
| 4 | PARTITION RANGE SINGLE| | 1 | 258K| 8183K|00:00:16.37 | 25576 | 1149 | 0 | | | | |
|* 5 | TABLE ACCESS FULL | SOT_REL_STAGE | 1 | 258K| 8183K|00:00:16.37 | 25576 | 1149 | 0 | | | | |
| 6 | PARTITION RANGE SINGLE| | 8183K| 326K| 327K|00:01:10.53 | 16M| 0 | 0 | | | | |
|* 7 | INDEX RANGE SCAN | IDXL_SOTCRCT_SOT_UD | 8183K| 326K| 327K|00:00:53.37 | 16M| 0 | 0 | | | | |
| 8 | PARTITION RANGE SINGLE | | 1 | 846K| 14M|00:02:06.36 | 289K| 180K| 0 | | | | |
|* 9 | TABLE ACCESS FULL | SOT_KEYMAP | 1 | 846K| 14M|00:01:52.32 | 289K| 180K| 0 | | | | |
I will attached the same for high load business hr once query gives results.It is still executing for last 50 mins.
INDEX STATS (INDEXES ARE LOCAL INDEXES)
TABLE_NAME INDEX_NAME COLUMN_NAME COLUMN_POSITION NUM_ROWS DISTINCT_KEYS CLUSTERING_FACTOR
SOT_REL_STAGE IDXL_SOTRELSTG_SOT_UD SOTRELSTG_SOT_UD 1 25461560 25461560 184180
SOT_REL_STAGE SOTRELSTG_TRD_DATE 2 25461560 25461560 184180
_YMD_PART
TABLE_NAME INDEX_NAME COLUMN_NAME COLUMN_POSITION NUM_ROWS DISTINCT_KEYS CLUSTERING_FACTOR
SOT_KEYMAP IDXL_SOTKEY_ENTORDSYS_UD SOTKEY_ENTRY_ORD_S 1 1012306940 3 38308680
YS_UD
SOT_KEYMAP IDXL_SOTKEY_HASH SOTKEY_HASH 1 1049582320 1049582320 1049579520
SOT_KEYMAP SOTKEY_TRD_DATE_YM 2 1049582320 1049582320 1049579520
D_PART
SOT_KEYMAP IDXL_SOTKEY_SOM_ORD SOTKEY_SOM_UD 1 1023998560 268949136 559414840
SOT_KEYMAP SOTKEY_SYS_ORD_ID 2 1023998560 268949136 559414840
SOT_KEYMAP IPK_SOT_KEYMAP SOTKEY_UD 1 1030369480 1015378900 24226580
TABLE_NAME INDEX_NAME COLUMN_NAME COLUMN_POSITION NUM_ROWS DISTINCT_KEYS CLUSTERING_FACTOR
SOT_CORRECT IDXL_SOTCRCT_SOT_UD SOTCRCT_SOT_UD 1 412484756 412484756 411710982
SOT_CORRECT SOTCRCT_TRD_DATE_Y 2 412484756 412484756 411710982
MD_PART
INDEX partiton stas (from dba_ind_partitions)
INDEX_NAME PARTITION_NAME STATUS BLEVEL LEAF_BLOCKS DISTINCT_KEYS CLUSTERING_FACTOR NUM_ROWS SAMPLE_SIZE LAST_ANALYZ GLO
IDXL_SOTCRCT_SOT_UD P20090219 USABLE 1 372 327879 216663 327879 327879 20-Feb-2009 YES
IDXL_SOTKEY_ENTORDSYS_UD P20090219 USABLE 2 2910 3 36618 856229 856229 19-Feb-2009 YES
IDXL_SOTKEY_HASH P20090219 USABLE 2 7783 853956 853914 853956 119705 19-Feb-2009 YES
IDXL_SOTKEY_SOM_ORD P20090219 USABLE 2 6411 531492 157147 799758 132610 19-Feb-2009 YES
IDXL_SOTRELSTG_SOT_UD P20090219 USABLE 2 13897 9682052 45867 9682052 794958 20-Feb-2009 YESThanks in advance.
Bhavik DesaiHi Randolf,
Thanks for the time you spent on this issue.I appreciate it.
Please see my comments below:
1. You've mentioned several times that you're passing the partition name as bind variable, but you're obviously testing the statement with literals rather than bind
variables. So your tests obviously don't reflect what is going to happen in case of the actual execution. The cardinality estimates are potentially quite different when
using bind variables for the partition key.
Yes.I intentionaly used literals in my tests.I found couple of times that plan used by the application and plan generated by AUTOTRACE+EXPLAIN PLAN command...is same and
caused hrly elapsed time.
As i pointed out earlier,last month we solved couple of bind peeking issue by intproducing NO_VALIDATE=>FALSE in stats gather procedure,which we execute just after data
load into such daily partitions and before start of jobs which executes this query.
Execution plans From AWR (with parallelism on at table level DEGREE>1)-->This plan is one which CBO has used when degradation occured.This plan is used most of the times.
ELAPSED_TIME_DELTA BUFFER_GETS_DELTA DISK_READS_DELTA CURSOR(SELECT*FROMTA
1918506000 46154275 918 CURSOR STATEMENT : 4
CURSOR STATEMENT : 4
PLAN_TABLE_OUTPUT
SQL_ID 39708a3azmks7
SELECT A.SOTRELSTG_SOT_UD SOTCRCT_SOT_UD, B.SOTKEY_UD SOTCRCT_ORIG_SOT_UD, A.ROWID STAGE_ROWID FROM (SELECT SOTRELSTG_SOT_UD,
SOTRELSTG_SYS_UD, SOTRELSTG_ORIG_SYS_ORD_ID, SOTRELSTG_ORIG_SYS_ORD_VSEQ FROM SOT_REL_STAGE WHERE SOTRELSTG_TRD_DATE_YMD_PART = :B1 AND
SOTRELSTG_CRCT_PROC_STAT_CD = 'N' AND SOTRELSTG_SOT_UD NOT IN( SELECT SOTCRCT_SOT_UD FROM SOT_CORRECT WHERE SOTCRCT_TRD_DATE_YMD_PART =
:B1 )) A, (SELECT MAX(SOTKEY_UD) SOTKEY_UD, SOTKEY_SYS_UD, SOTKEY_SYS_ORD_ID, SOTKEY_SYS_ORD_VSEQ, SOTKEY_TRD_DATE_YMD_PART FROM
SOT_KEYMAP WHERE SOTKEY_TRD_DATE_YMD_PART = :B1 AND SOTKEY_IUD_CD = 'I' GROUP BY SOTKEY_TRD_DATE_YMD_PART, SOTKEY_SYS_UD,
SOTKEY_SYS_ORD_ID, SOTKEY_SYS_ORD_VSEQ) B WHERE A.SOTRELSTG_SYS_UD = B.SOTKEY_SYS_UD AND A.SOTRELSTG_ORIG_SYS_ORD_ID =
B.SOTKEY_SYS_ORD_ID AND NVL(A.SOTRELSTG_ORIG_SYS_ORD_VSEQ, 1) = NVL(B.SOTKEY_SYS_ORD_VSEQ, 1)
Plan hash value: 1213870831
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | TQ |IN-OUT| PQ Distrib |
| 0 | SELECT STATEMENT | | | | 19655 (100)| | | | | | |
| 1 | PX COORDINATOR | | | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10003 | 1 | 116 | 19655 (1)| 00:05:54 | | | Q1,03 | P->S | QC (RAND) |
| 3 | HASH GROUP BY | | 1 | 116 | 19655 (1)| 00:05:54 | | | Q1,03 | PCWP | |
| 4 | PX RECEIVE | | 1 | 116 | 19655 (1)| 00:05:54 | | | Q1,03 | PCWP | |
| 5 | PX SEND HASH | :TQ10002 | 1 | 116 | 19655 (1)| 00:05:54 | | | Q1,02 | P->P | HASH |
| 6 | HASH GROUP BY | | 1 | 116 | 19655 (1)| 00:05:54 | | | Q1,02 | PCWP | |
| 7 | NESTED LOOPS ANTI | | 1 | 116 | 19654 (1)| 00:05:54 | | | Q1,02 | PCWP | |
| 8 | HASH JOIN | | 1 | 102 | 19654 (1)| 00:05:54 | | | Q1,02 | PCWP | |
| 9 | PX JOIN FILTER CREATE| :BF0000 | 13M| 664M| 2427 (3)| 00:00:44 | | | Q1,02 | PCWP | |
| 10 | PX RECEIVE | | 13M| 664M| 2427 (3)| 00:00:44 | | | Q1,02 | PCWP | |
| 11 | PX SEND HASH | :TQ10000 | 13M| 664M| 2427 (3)| 00:00:44 | | | Q1,00 | P->P | HASH |
| 12 | PX BLOCK ITERATOR | | 13M| 664M| 2427 (3)| 00:00:44 | KEY | KEY | Q1,00 | PCWC | |
| 13 | TABLE ACCESS FULL| SOT_REL_STAGE | 13M| 664M| 2427 (3)| 00:00:44 | KEY | KEY | Q1,00 | PCWP | |
| 14 | PX RECEIVE | | 27M| 1270M| 17209 (1)| 00:05:10 | | | Q1,02 | PCWP | |
| 15 | PX SEND HASH | :TQ10001 | 27M| 1270M| 17209 (1)| 00:05:10 | | | Q1,01 | P->P | HASH |
| 16 | PX JOIN FILTER USE | :BF0000 | 27M| 1270M| 17209 (1)| 00:05:10 | | | Q1,01 | PCWP | |
| 17 | PX BLOCK ITERATOR | | 27M| 1270M| 17209 (1)| 00:05:10 | KEY | KEY | Q1,01 | PCWC | |
| 18 | TABLE ACCESS FULL| SOT_KEYMAP | 27M| 1270M| 17209 (1)| 00:05:10 | KEY | KEY | Q1,01 | PCWP | |
| 19 | PARTITION RANGE SINGLE| | 16185 | 221K| 0 (0)| | KEY | KEY | Q1,02 | PCWP | |
| 20 | INDEX RANGE SCAN | IDXL_SOTCRCT_SOT_UD | 16185 | 221K| 0 (0)| | KEY | KEY | Q1,02 | PCWP | |
Other Execution plan from AWR
ELAPSED_TIME_DELTA BUFFER_GETS_DELTA DISK_READS_DELTA CURSOR(SELECT*FROMTA
1053251381 0 2925 CURSOR STATEMENT : 4
CURSOR STATEMENT : 4
PLAN_TABLE_OUTPUT
SQL_ID 39708a3azmks7
SELECT A.SOTRELSTG_SOT_UD SOTCRCT_SOT_UD, B.SOTKEY_UD SOTCRCT_ORIG_SOT_UD, A.ROWID STAGE_ROWID FROM (SELECT SOTRELSTG_SOT_UD,
SOTRELSTG_SYS_UD, SOTRELSTG_ORIG_SYS_ORD_ID, SOTRELSTG_ORIG_SYS_ORD_VSEQ FROM SOT_REL_STAGE WHERE SOTRELSTG_TRD_DATE_YMD_PART = :B1 AND
SOTRELSTG_CRCT_PROC_STAT_CD = 'N' AND SOTRELSTG_SOT_UD NOT IN( SELECT SOTCRCT_SOT_UD FROM SOT_CORRECT WHERE SOTCRCT_TRD_DATE_YMD_PART =
:B1 )) A, (SELECT MAX(SOTKEY_UD) SOTKEY_UD, SOTKEY_SYS_UD, SOTKEY_SYS_ORD_ID, SOTKEY_SYS_ORD_VSEQ, SOTKEY_TRD_DATE_YMD_PART FROM
SOT_KEYMAP WHERE SOTKEY_TRD_DATE_YMD_PART = :B1 AND SOTKEY_IUD_CD = 'I' GROUP BY SOTKEY_TRD_DATE_YMD_PART, SOTKEY_SYS_UD,
SOTKEY_SYS_ORD_ID, SOTKEY_SYS_ORD_VSEQ) B WHERE A.SOTRELSTG_SYS_UD = B.SOTKEY_SYS_UD AND A.SOTRELSTG_ORIG_SYS_ORD_ID =
B.SOTKEY_SYS_ORD_ID AND NVL(A.SOTRELSTG_ORIG_SYS_ORD_VSEQ, 1) = NVL(B.SOTKEY_SYS_ORD_VSEQ, 1)
Plan hash value: 3434900850
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | TQ |IN-OUT| PQ Distrib |
| 0 | SELECT STATEMENT | | | | 1830 (100)| | | | | | |
| 1 | PX COORDINATOR | | | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10003 | 1 | 131 | 1830 (2)| 00:00:33 | | | Q1,03 | P->S | QC (RAND) |
| 3 | HASH GROUP BY | | 1 | 131 | 1830 (2)| 00:00:33 | | | Q1,03 | PCWP | |
| 4 | PX RECEIVE | | 1 | 131 | 1830 (2)| 00:00:33 | | | Q1,03 | PCWP | |
| 5 | PX SEND HASH | :TQ10002 | 1 | 131 | 1830 (2)| 00:00:33 | | | Q1,02 | P->P | HASH |
| 6 | HASH GROUP BY | | 1 | 131 | 1830 (2)| 00:00:33 | | | Q1,02 | PCWP | |
| 7 | NESTED LOOPS ANTI | | 1 | 131 | 1829 (2)| 00:00:33 | | | Q1,02 | PCWP | |
| 8 | HASH JOIN | | 1 | 117 | 1829 (2)| 00:00:33 | | | Q1,02 | PCWP | |
| 9 | PX JOIN FILTER CREATE| :BF0000 | 1010K| 50M| 694 (1)| 00:00:13 | | | Q1,02 | PCWP | |
| 10 | PX RECEIVE | | 1010K| 50M| 694 (1)| 00:00:13 | | | Q1,02 | PCWP | |
| 11 | PX SEND HASH | :TQ10000 | 1010K| 50M| 694 (1)| 00:00:13 | | | Q1,00 | P->P | HASH |
| 12 | PX BLOCK ITERATOR | | 1010K| 50M| 694 (1)| 00:00:13 | KEY | KEY | Q1,00 | PCWC | |
| 13 | TABLE ACCESS FULL| SOT_KEYMAP | 1010K| 50M| 694 (1)| 00:00:13 | KEY | KEY | Q1,00 | PCWP | |
| 14 | PX RECEIVE | | 11M| 688M| 1129 (3)| 00:00:21 | | | Q1,02 | PCWP | |
| 15 | PX SEND HASH | :TQ10001 | 11M| 688M| 1129 (3)| 00:00:21 | | | Q1,01 | P->P | HASH |
| 16 | PX JOIN FILTER USE | :BF0000 | 11M| 688M| 1129 (3)| 00:00:21 | | | Q1,01 | PCWP | |
| 17 | PX BLOCK ITERATOR | | 11M| 688M| 1129 (3)| 00:00:21 | KEY | KEY | Q1,01 | PCWC | |
| 18 | TABLE ACCESS FULL| SOT_REL_STAGE | 11M| 688M| 1129 (3)| 00:00:21 | KEY | KEY | Q1,01 | PCWP | |
| 19 | PARTITION RANGE SINGLE| | 5209 | 72926 | 0 (0)| | KEY | KEY | Q1,02 | PCWP | |
| 20 | INDEX RANGE SCAN | IDXL_SOTCRCT_SOT_UD | 5209 | 72926 | 0 (0)| | KEY | KEY | Q1,02 | PCWP | |
EXECUTION PLAN AFTER SETTING DEGREE=1 (It was also degraded)
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Pstart| Pstop |
| 0 | SELECT STATEMENT | | 1 | 129 | | 42336 (2)| 00:12:43 | | |
| 1 | HASH GROUP BY | | 1 | 129 | | 42336 (2)| 00:12:43 | | |
| 2 | NESTED LOOPS ANTI | | 1 | 129 | | 42335 (2)| 00:12:43 | | |
|* 3 | HASH JOIN | | 1 | 115 | 51M| 42334 (2)| 00:12:43 | | |
| 4 | PARTITION RANGE SINGLE| | 846K| 41M| | 8241 (1)| 00:02:29 | 81 | 81 |
|* 5 | TABLE ACCESS FULL | SOT_KEYMAP | 846K| 41M| | 8241 (1)| 00:02:29 | 81 | 81 |
| 6 | PARTITION RANGE SINGLE| | 8161K| 490M| | 12664 (3)| 00:03:48 | 81 | 81 |
|* 7 | TABLE ACCESS FULL | SOT_REL_STAGE | 8161K| 490M| | 12664 (3)| 00:03:48 | 81 | 81 |
| 8 | PARTITION RANGE SINGLE | | 6525K| 87M| | 1 (0)| 00:00:01 | 81 | 81 |
|* 9 | INDEX RANGE SCAN | IDXL_SOTCRCT_SOT_UD | 6525K| 87M| | 1 (0)| 00:00:01 | 81 | 81 |
Predicate Information (identified by operation id):
3 - access("SOTRELSTG_SYS_UD"="SOTKEY_SYS_UD" AND "SOTRELSTG_ORIG_SYS_ORD_ID"="SOTKEY_SYS_ORD_ID" AND
NVL("SOTRELSTG_ORIG_SYS_ORD_VSEQ",1)=NVL("SOTKEY_SYS_ORD_VSEQ",1))
5 - filter("SOTKEY_TRD_DATE_YMD_PART"=20090219 AND "SOTKEY_IUD_CD"='I')
7 - filter("SOTRELSTG_CRCT_PROC_STAT_CD"='N' AND "SOTRELSTG_TRD_DATE_YMD_PART"=20090219)
9 - access("SOTRELSTG_SOT_UD"="SOTCRCT_SOT_UD" AND "SOTCRCT_TRD_DATE_YMD_PART"=20090219)2. Why are you passing the partition name as bind variable? A statement executing 5 mins. best, > 2 hours worst obviously doesn't suffer from hard parsing issues and
doesn't need to (shouldn't) share execution plans therefore. So I strongly suggest to use literals instead of bind variables. This also solves any potential issues caused
by bind variable peeking.
This is a custom application which uses bind variables to extract data from daily partitions.So,daily automated data extract from daily paritions after load and ELT process.
Here,Value of bind variable is being passed through a procedure parameter.It would be bit difficult to use literals in such application.
3. All your posted plans suffer from bad cardinality estimates. The NO_MERGE hint suggested by Timur only caused a (significant) damage limitation by obviously reducing
the row source size by the group by operation before joining, but still the optimizer is way off, apart from the obviously wrong join order (larger row set first) in
particular the NESTED LOOP operation is causing the main troubles due to excessive logical I/O, as already pointed out by Timur.
Can i ask for alternatives to NESTED LOOP?
4. Your PLAN_TABLE seems to be old (you should see a corresponding note at the bottom of the DBMS_XPLAN.DISPLAY output), because none of the operations have a
filter/access predicate information attached. Since your main issue are the bad cardinality estimates, I strongly suggest to drop any existing PLAN_TABLEs in any non-Oracle
owned schemas because 10g already provides one in the SYS schema (GTT PLAN_TABLE$) exposed via a public synonym, so that the EXPLAIN PLAN information provides the
"Predicate Information" section below the plan covering the "Filter/Access" predicates.
Please post a revised explain plan output including this crucial information so that we get a clue why the cardinality estimates are way off.
I have dropped the old plan.Got above execution plan(listed above in first point) with PREDICATE information.
"As already mentioned the usage of bind variables for the partition name makes this issue potentially worse."
Is there any workaround without replacing bind variable.I am on 10g so 11g's feature will not help !!!
How are you gathering the statistics daily, can you post the exact command(s) used?
gather_table_part_stats(i_owner_name,i_table_name,i_part_name,i_estimate:= DBMS_STATS.AUTO_SAMPLE_SIZE, i_invalidate IN VARCHAR2 := 'Y',i_debug:= 'N')
Thanks & Regards,
Bhavik Desai -
End of file on communication channel when doing a (hash) join
I'm having a problem with a simple join between two tables (hash join, from the explain), one containing about 650k rows, the other 730k.
The query runs fine until I add to the select list a geometry field (oracle spatial). In that case I receive an end-of-file error, which seems to be caused by a memory saturation. I'm not sure about this, as I can't understand the trace very well...
I've increased the pga_aggregate_target and the pgamax_size but this have little effects on the problem...
Any hint? Could it be caused by the geometry field size?
thanks,
giovanniThanks for the quick reply. Here it is what you asked:
select from v$version;*
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Prod
PL/SQL Release 10.2.0.4.0 - Production
"CORE 10.2.0.4.0 Production"
TNS for 32-bit Windows: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production
an excerpt from the trace file:
*** ACTION NAME:() 2010-04-16 12:48:17.796
*** MODULE NAME:(SQL*Plus) 2010-04-16 12:48:17.796
*** SERVICE NAME:(ABRUZZO) 2010-04-16 12:48:17.796
*** SESSION ID:(149.10) 2010-04-16 12:48:17.796
*** 2010-04-16 12:48:17.796
ksedmp: internal or fatal error
ORA-07445: trovata eccezione: dump della memoria [ACCESS_VIOLATION] [_kokekd2m+24] [PC:0x11CF9E0] [ADDR:0xBA8C193C] [UNABLE_TO_READ] []
Current SQL statement for this session:
select g.cr375_idobj,g.cr375_geometria,t.cr374_codicectr,t.cr374_scarpt_cont
from DBTI.Cr374_Scarpata t, DBTI.Cr375g_a_Scarp g
where t.cr374_idobj=g.cr375_idobj
check trace file c:\oracle\product\10.2.0\db_1\rdbms\trace\abruzzo_ora_0.trc for preloading .sym file messages
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
_kokekd2m+24 00000000
kokeq2iro+178 CALLrel kokekd2m+0 99384C8 9937624 BA8C18D8
9968588 3660D954 3A52AC00
__VInfreq__rworupo+ CALLrel _kokeq2iro+0 9968218 A05C810 EA7
321
_kxhrUnpack+71 CALLreg 00000000 99693C8 A05C810 0 9 0
qerhjWalkHashBucke CALLrel kxhrUnpack+0 9967DD0 99693C8 9 A05C810 0
t+210
__PGOSF352__qerhjIn CALLrel _qerhjWalkHashBucke A05CED4 99693C0 8
nerProbeHashTable+4 t+0
78
_kdstf0000101km+230 CALLreg 00000000
kdsttgr+1263 CALLrel kdstf0000101km+0 -
Setting PGA_AGGREGATE_TARGET to resolve Hash Join issues?
I'm running some fairly big queries against our data warehouse, 10s millions of rows in some of the tables we are joining. The response times we are getting vary but its not unusual for these to run for many hours (8 and 12 for a couple of the more recent ones). Looking at the long ops I see that it can take 30000 seconds to do a hash join. Full Table Scans are reasonably fast, the cost of the query is pretty good but the hash/sort/merge operations kill the queries every time.
As a developer with access to Google I am obviously well qualified to be telling our DBAs what they need to do to improve the performance of my query - and I stumbled upon the PGA_AGGREGATE_TARGET parameter. Its pretty low in my current environment, 50Mb until I go them to up it to 200Mb (they wouldn't go bigger). But from what I've been reading I think this needs to be alot bigger, the sizes of some of my tables are huge (and often I'll be joining entire datasets and grouping/aggregate them).
But am I looking at the wrong thing, some comments I've read suggest its limited to 200Mb anyway (a hidden parameter?) and that we'd be better off setting manual AREASIZE parameters, is this the case?
Any advice (knowing that I've not supplied explain plans, real sizings etc), but a point in the right direction would be greatly appreciated.
Cheers
RichardHi Richard,
Could you give some more details related to your problem:
Which DB version is exactly used ?
Which OS is installed on the DB Server?
Is it a 32 bit or 64 bit OS?
How large is your SGA?
How much free memory (RAM) do you have on your server?
Is the query executed in parallel?
But even without these information I can clearly say that 50MB is far too small.
By default I would recommend, for a DWH, to set the PGA twice as large as the SGA.
You may also have a look at the following documentation:
http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96533/memory.htm#49321
http://www.pythian.com/documents/Working_with_Automatic_PGA.ppt
http://www.miracleltd.com/msdbf2007/js.ppt
http://www.vldb.org/conf/2002/S29P03.pdf
http://www.sloug.org/presentations/If_Memory_Richmond_Shee.ppt
Regards
Maurice
Maybe you are looking for
-
I have a password in my iphone3 and my niece have entered password wrongly too many times. What shhould i have to do?
-
Where have all the plug-ins gone?
I re-installed InDesign CS3 after my hard drive was wiped clean with the re-installation of Windows XP, and I'm finding that some files won't open. The error messages state that some plug-ins are missing and that the program and/or the plug-ins sho
-
OWB version does not change with Oracle database 11.1.0.7 patch
Hi, This is Kumar and I have just completed applying Oracle database 11.1.0.7 patch (p6890831_111070_Linux-x86-64.zip) on top of Oracle 11.1.0.6 database on RedHat Linux machine. After successful installation, I have issues below statement to check t
-
Unassinged gl account's in FSV
Hello, In FSV, few gl account's are not assigned in FSV later on i have assinged gl accounts to respective group in FSE2.The Problem is in Development everything is working fine were as coming to PRD,when i'm generating a report S_PL0_86000028 FInanc
-
Changing Variant Parameters Programmatically
Hi Guys/Dolls I've developed a program that has a user selection date range on it and I have saved this program as a variant. I submit this program has a batch job on a monthly basis but need to change the dates on the variant each month to be the st