Merge joins turned into expensive hash joins!?
Hello.....
working with two tables....
Table A is where I am bringing data into.....
Table B is where the data resides.
Table B has 32million rows...
I was using just a quick.....select b.whatever from table.a, table.b where primary key and b.table=a.table.....
I could grab a few rows at a time and it would use a merge join and take 2-3 seconds.
Now....(after NO changes) it hangs. In production it is still fast and uses merge joins, but I compared the activity to that of test and it's using hash joins! The hidden parameter _use merge join will force a merge, but then none of my other hash joins (where I want them) will work.
Optimizer dorked up? Whats the scoop?
hmm, well your replies are just as ingenius. . .here's a cookie.
10.2.0.4.0 and i put the relevent sql in there enough to ask why over a dblink (not that it matters) why this behavior would all of the sudden change.
Again, simple matter of creating a nested loop over a dblink to grab a few rows from table b to insert them into table a. It used to merge join, now it's hash joining. Just wondering :-)
Similar Messages
-
How can I merge 2 tracks into one track with iTunes 11?
Thanks for your reply, Jim. I imported a CD that I had burned previously with the 2 tracks of sound effects that I had downloaded from the internet (MP3 files), but I did not see an option to join the tracks when I imported the CD. In the upper right corner there were three buttons: Options, CD Info, and Import CD. When I click on Options, the drop-down menu has two choices: "Get Track Names" and "Submit CD Track Names". I cannot find an option to Join Tracks. Help!
-
We are on ASE 15.0.3/EBF 21284 ESD#4.3 working on a application with over 3000 stored procedures.
Our server optimization goal is allrows_mix.
The Merge-Join is giving us problems. When a query uses Merge-Join, it usually take an order of magnitude longer to run than if we force it
to use the other types of joins - nested-loop, n-ary-nested-loop, or hash-join.
The query plan shows sorting on worktables leading into the merge-join.
I know I can disable it with "set merge_join off", or "set plan optgoal allrows_oltp", but I'd rather not if I can fix the problem instead.
Question: Are there configuration options that would help merge-join?
I've done variations of this:
sp_configure "number of sort buffers", 32000
I've also done variations of this in the proc:
set parallel_degree 5
set scan_parallel_degree 4
When I run the following command, I see sort buffer starvation:
1> sp_monitorconfig "sort buffers"
2> go
Usage information at date and time: Apr 24 2014 2:31PM.
Name Num_free Num_active Pct_act Max_Used Reuse_cnt
number of sort buffers 0 82045 100.00 82045 0
(1 row affected)
(return status = 0)
Maybe there are other configuration option to help merge-joins? Any ideas?
Thanks.Well, I'm gonna have to emphatically disagree with your comment ...
"regressing back to allrows_oltp setting to solve your performance problems should not be encouraged"
For *EVERY* client I've worked with on migrating from ASE 12.5.x to ASE 15.x ... they all had the same objective ... get through the migration as quickly as possible and do not degrade the performance of our database queries. Unfortunately for every client I've worked with ... ASE 15.x, and the default of allrows_mix, did just the opposite, ie, migrations took much longer than expected/planned due primarily to huge performance degradation across their SQL inventory.
For most of my clients merge joins were rarely, if ever, used in ASE 12.5.x. And since hash joins never existed, that leaves us with using nested loop joins in ASE 15.x in an attempt to stay as close to ASE 12.5.x in terms of performance.
NOTE: No, I don't consider compatibility mode as a solution as this requires you go through 2 migrations ... once to compatibility mode ... and eventually once to get off of compatibility mode.
Now, can merge joins improve the performance of *some* queries? Absolutely, but in practice ... especially with the first 4-5 years of ASE 15.x releases ... merge joins caused more headaches and performance degradation than they were worth. I've seen too many clients spend huge amounts of time trying to re-write code to work with merge joins, often failing and having to 'regress back' to nested loop joins in the end.
Unfortunately a) Sybase delivered ASE 15.x with allows_mix as the default and b) most companies didn't have enough migration experience to understand the pitfalls of trying to run all of their queries under the default of allrows_mix. This meant that many companies were left having to 'regress back' to alternative solutions (eg, allrows_oltp, compat mode, don't migrate, move to another RDBMS) to address the performance degradation introduced with ASE 15.x and the default setting of allrows_mix. -
Facing Merge Join Cartersian in the explain plan after adding gl periods
Hi All
I have added gl periods table to the below query , checked the explain plan and it shows merge join cartesian. This query is taking long time to fetch the results.
Need help ASAP , Please let me know where i am going wrong . Any suggestions will be appreciated.
SELECT gljh.period_name, gljh.ledger_id, gljh.je_source,
glcc.segment2,
SUM ( NVL (gljl.accounted_dr, 0)
- NVL (gljl.accounted_cr, 0)
) total_amt,
gljh.currency_code
FROM gl_je_headers gljh,
gl_je_lines gljl,
gl_code_combinations glcc,
gl_periods gps
WHERE 1=1
AND gljh.period_name = gps.period_name
AND gljl.period_name = gps.period_name
AND gps.period_set_name = 'MCD_MONTH_'
AND gps.start_date >= :p_from_date
AND gps.start_date <= :p_to_date
AND gljh.ledger_id = :p_ledger_id
AND gljh.je_header_id = gljl.je_header_id
AND gljl.code_combination_id = glcc.code_combination_id
AND glcc.segment2 = '10007'--get_segment2_rec.flex_value
AND gljh.currency_code <> 'STAT'
GROUP BY gljh.je_source,
gljh.period_name,
glcc.segment2,
gljh.ledger_id,
gljh.currency_code
HAVING SUM ( NVL (gljl.accounted_dr, 0)
- NVL (gljl.accounted_cr, 0)
) <> 0;
Plan
SELECT STATEMENT ALL_ROWSCost: 73,146 Bytes: 2,266 Cardinality: 22
15 FILTER
14 HASH GROUP BY Cost: 73,146 Bytes: 2,266 Cardinality: 22
13 FILTER
12 NESTED LOOPS Cost: 73,145 Bytes: 61,079 Cardinality: 593
9 NESTED LOOPS Cost: 31,603 Bytes: 1,452,780 Cardinality: 20,754
6 MERGE JOIN CARTESIAN Cost: 2,108 Bytes: 394,181 Cardinality: 9,167
2 TABLE ACCESS BY INDEX ROWID TABLE GL.GL_PERIODS Cost: 4 Bytes: 31 Cardinality: 1
1 INDEX RANGE SCAN INDEX (UNIQUE) GL.GL_PERIODS_U2 Cost: 1 Cardinality: 64
5 BUFFER SORT Cost: 2,104 Bytes: 683,988 Cardinality: 56,999
4 TABLE ACCESS BY INDEX ROWID TABLE GL.GL_CODE_COMBINATIONS Cost: 2,104 Bytes: 683,988 Cardinality: 56,999
3 INDEX RANGE SCAN INDEX GL.GL_CODE_COMBINATIONS_N2 Cost: 155 Cardinality: 56,999
8 TABLE ACCESS BY INDEX ROWID TABLE GL.GL_JE_LINES Cost: 18 Bytes: 54 Cardinality: 2
7 INDEX RANGE SCAN INDEX GL.GL_JE_LINES_N1 Cost: 3 Cardinality: 37
11 TABLE ACCESS BY INDEX ROWID TABLE GL.GL_JE_HEADERS Cost: 2 Bytes: 33 Cardinality: 1
10 INDEX UNIQUE SCAN INDEX (UNIQUE) GL.GL_JE_HEADERS_U1 Cost: 1 Cardinality: 1
Thanks
ChandraLots of things come into play when you're tuning a query.
An (unformatted) execution plan isn't enough.
Tuning takes time and understanding how (a lot of) things work, there is no ASAP in the world of tuning.
Please post other important details, like your database version, optimizer settings, how/when are table statistics gathered etc.
So, read the following informative threads (and please take your time, this really is important stuff), and adust your thread as needed.
That way you'll have a bigger chance of getting help that makes sense...
Your DBA should/ought to be able to help you in this as well.
Re: HOW TO: Post a SQL statement tuning request - template posting
http://oracle-randolf.blogspot.com/2009/02/basic-sql-statement-performance.html -
hi can any one please help me understand why this simple query is doing merge cartesion join , but not equi join
i have two table with column name id on both the tables.
CREATE TABLE id_a (ID VARCHAR2(10), description VARCHAR2(100));
CREATE TABLE id_b (ID VARCHAR2(10), long_description VARCHAR2(100),short_description VARCHAR2(100));
/* Formatted on 2010/01/13 12:16 (Formatter Plus v4.8.8) */
SET DEFINE OFF;
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0001'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0002'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0003'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0004'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0005'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0006'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0007'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0008'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0009'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0010'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0011'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0012'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0013'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0014'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0015'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0016'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0017'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0018'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0019'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0020'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0021'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0022'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0023'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0024'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0025'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0026'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0027'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0028'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0029'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0030'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0031'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0032'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0033'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0034'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0035'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0036'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0037'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC0038'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC8000'
INSERT INTO id_a
(ID, description
VALUES ('10ACCC', '10ACCC9998'
COMMIT ;
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10ACCC', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('20LNDM', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10NQWR', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('40KKRA ', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10VUIU', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10NCMM', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10EAQL', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10BLGP', NULL, NULL);
Insert into ID_B
(ID, LONG_DESCRIPTION, SHORT_DESCRIPTION)
Values
('10LWGV', NULL, NULL);
commit;
select a.description from id_a a , id_b b where
a.id = '10ACCC'
and a.id = b.id ;these are the stats that accurately reflect the data. if i use exist clause for each table i can avoid the merge cartesion, but is it a better way to rewrite the query. please suggest.
And one more thing is that i read the article http://jonathanlewis.wordpress.com/2006/12/13/cartesian-merge-join/ in this article it clearly says that i need to set the parameter
optimizertransitivity_retain to true . but how do i do that?
SQL> show parameter user_dump_dest
NAME TYPE VALUE
user_dump_dest string /u01/app/oracle/admin/dnsprx/u
dump
SQL> show parameter optimizer
NAME TYPE VALUE
optimizer_dynamic_sampling integer 2
optimizer_features_enable string 10.1.0.5
optimizer_index_caching integer 0
optimizer_index_cost_adj integer 100
optimizer_mode string ALL_ROWS
SQL> show parameter db_file_multi
NAME TYPE VALUE
db_file_multiblock_read_count integer 16
SQL> show parameter db_block_size
NAME TYPE VALUE
db_block_size integer 8192
SQL> show parameter cursor_sharing
NAME TYPE VALUE
cursor_sharing string EXACT
SQL> column sname format a20
SQL> column pname format a20
SQL> column pval2 format a20
SQL>
SQL> select
2 sname
3 , pname
4 , pval1
5 , pval2
6 from
7 sys.aux_stats$;
SNAME PNAME PVAL1 PVAL2
SYSSTATS_INFO STATUS COMPLETED
SYSSTATS_INFO DSTART 02-21-2006 12:14
SYSSTATS_INFO DSTOP 02-21-2006 12:14
SYSSTATS_INFO FLAGS 1
SYSSTATS_MAIN CPUSPEEDNW 604.316547
SYSSTATS_MAIN IOSEEKTIM 10
SYSSTATS_MAIN IOTFRSPEED 4096
SYSSTATS_MAIN SREADTIM
SYSSTATS_MAIN MREADTIM
SYSSTATS_MAIN CPUSPEED
SYSSTATS_MAIN MBRC
SYSSTATS_MAIN MAXTHR
SYSSTATS_MAIN SLAVETHR
13 rows selected.
Elapsed: 00:00:00.09
SQL>
SQL> --============================================================================
SQL> ----------------- put your sql statement here---------------------------------
SQL> explain plan for
2 SELECT DISTINCT b.sub_nasp_id
3 FROM r_ref_cust_status_unmnpip_nasp a,
4 ref_nasp_eref b,
5 ref_garm_naspid c
6 WHERE a.nasp_id = '10ACCC'
7 AND a.nasp_id = b.nasp_id
8 AND a.nasp_id = SUBSTR (c.ID, 1, 6)
9 AND NVL (INSTR ('1', NVL (c.sensitivity_level, '@!')), 1) != 0;
Explained.
Elapsed: 00:00:00.07
SQL>
SQL>
SQL> ---------------------- end of sql statement ---------------------------------
SQL> --============================================================================
SQL>
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
Plan hash value: 3987722552
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 6 | 192 | 3578 (2)| 00:00:43 |
| 1 | SORT UNIQUE | | 6 | 192 | 3578 (2)| 00:00:43 |
| 2 | MERGE JOIN CARTESIAN | | 3105 | 99360 | 3577 (2)| 00:00:43 |
| 3 | MERGE JOIN CARTESIAN | | 528 | 8448 | 1991 (4)| 00:00:24 |
|* 4 | TABLE ACCESS FULL | REF_GARM_NASPID | 528 | 4752 | 1990 (3)| 00:00:24 |
| 5 | BUFFER SORT | | 1 | 7 | 1 (100)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | UNMNPIP_NASP_INDX | 1 | 7 | 0 (0)| 00:00:01 |
| 7 | BUFFER SORT | | 6 | 96 | 3577 (2)| 00:00:43 |
| 8 | TABLE ACCESS BY INDEX ROWID| REF_NASP_EREF | 6 | 96 | 3 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | REF_NASP_EREF_INDX | 6 | | 2 (0)| 00:00:01 |
Predicate Information (identified by operation id):
4 - filter(SUBSTR("C"."ID",1,6)='10ACCC' AND
NVL(INSTR('1',NVL("C"."SENSITIVITY_LEVEL",'@!')),1)<>0)
6 - access("A"."NASP_ID"='10ACCC')
9 - access("B"."NASP_ID"='10ACCC')
24 rows selected.
Elapsed: 00:00:00.23
SQL> rollback;
Rollback complete.
Elapsed: 00:00:00.04
SQL>
SQL> -- rem Set the ARRAYSIZE according to your application
SQL> set autotrace traceonly arraysize 100
SQL>
SQL> alter session set tracefile_identifier = 'mytrace1';
Session altered.
Elapsed: 00:00:00.04
SQL> alter session set events '10046 trace name context forever, level 8';
Session altered.
Elapsed: 00:00:00.04
SQL>
SQL> --============================================================================
SQL> ----------------- put your sql statement here---------------------------------
SQL> explain plan for
2 SELECT DISTINCT b.sub_nasp_id
3 FROM r_ref_cust_status_unmnpip_nasp a,
4 ref_nasp_eref b,
5 ref_garm_naspid c
6 WHERE a.nasp_id = '10ACCC'
7 AND a.nasp_id = b.nasp_id
8 AND a.nasp_id = SUBSTR (c.ID, 1, 6)
9 AND NVL (INSTR ('1', NVL (c.sensitivity_level, '@!')), 1) != 0;
Explained.
Elapsed: 00:00:00.06
SQL>
SQL> ---------------------- end of sql statement ---------------------------------
SQL> --============================================================================
SQL>
SQL>
SQL> ----------------------------- Step 2 ------------------------------------------
SQL> --============================================================================
SQL>
SQL> -- if you're on 10g or later
SQL> -- get the row source statistics
SQL> -- if the SQL is sensitive
SQL> -- don't switch on the ECHO
SQL> -- set echo off
SQL>
SQL> set echo on
SQL>
SQL> ---set timing on trimspool on linesize 250 pagesize 999
SQL> set arraysize 100 termout off
SQL> -- alter system flush buffer_cache;
SQL> -- spool off
SQL>
SQL> -- put your statement here
SQL> -- use the GATHER_PLAN_STATISTICS hint
SQL> -- if you're not using STATISTICS_LEVEL = ALL
SQL>
SQL> SELECT /*+ gather_plan_statistics */ DISTINCT b.sub_nasp_id
2 FROM r_ref_cust_status_unmnpip_nasp a,
3 ref_nasp_eref b,
4 ref_garm_naspid c
5 WHERE a.nasp_id = '10ACCC'
6 AND a.nasp_id = b.nasp_id
7 AND a.nasp_id = SUBSTR (c.ID, 1, 6)
8 AND NVL (INSTR ('1', NVL (c.sensitivity_level, '@!')), 1) != 0;
40 rows selected.
Elapsed: 00:00:04.65
Execution Plan
Plan hash value: 3987722552
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 6 | 192 | 3578 (2)| 00:00:43 |
| 1 | SORT UNIQUE | | 6 | 192 | 3578 (2)| 00:00:43 |
| 2 | MERGE JOIN CARTESIAN | | 3105 | 99360 | 3577 (2)| 00:00:43 |
| 3 | MERGE JOIN CARTESIAN | | 528 | 8448 | 1991 (4)| 00:00:24 |
|* 4 | TABLE ACCESS FULL | REF_GARM_NASPID | 528 | 4752 | 1990 (3)| 00:00:24 |
| 5 | BUFFER SORT | | 1 | 7 | 1 (100)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | UNMNPIP_NASP_INDX | 1 | 7 | 0 (0)| 00:00:01 |
| 7 | BUFFER SORT | | 6 | 96 | 3577 (2)| 00:00:43 |
| 8 | TABLE ACCESS BY INDEX ROWID| REF_NASP_EREF | 6 | 96 | 3 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | REF_NASP_EREF_INDX | 6 | | 2 (0)| 00:00:01 |
Predicate Information (identified by operation id):
4 - filter(SUBSTR("C"."ID",1,6)='10ACCC' AND
NVL(INSTR('1',NVL("C"."SENSITIVITY_LEVEL",'@!')),1)<>0)
6 - access("A"."NASP_ID"='10ACCC')
9 - access("B"."NASP_ID"='10ACCC')
Statistics
1 recursive calls
0 db block gets
8836 consistent gets
5736 physical reads
0 redo size
855 bytes sent via SQL*Net to client
243 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
3 sorts (memory)
0 sorts (disk)
40 rows processed
SQL> set termout on
SQL> select * from table(dbms_xplan.display_cursor(null, null, 'ALLSTATS LAST'));
Elapsed: 00:00:00.26
Execution Plan
Plan hash value: 3602215112
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 8168 | 16336 | 24 (0)| 00:00:01 |
| 1 | COLLECTION ITERATOR PICKLER FETCH| DISPLAY_CURSOR | | | | |
Statistics
11 recursive calls
0 db block gets
108 consistent gets
0 physical reads
0 redo size
304 bytes sent via SQL*Net to client
243 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed
SQL>
SQL>
SQL>
SQL>
SQL>
SQL> disconnect
Disconnected from Oracle Database 10g Enterprise Edition Release 10.1.0.5.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> spool off -
Why is LOWER function producing a cartesian merge join, when UPPER doesn't?
Hi there,
I have an odd scenario that I would like to understand correctly...
We have a query that is taking a long time to run on one of our databases, further investigation of the explain plan showed that the query was in fact producing a Cartesian merge join even though there is clearly join criteria specified. I know that the optimiser can and will do this if it is a more efficient way of producing the results, however in this scenario it is producing the Cartesian merge on two unrelated tables and seemingly ignoring the Join condition...
*** ORIGINAL QUERY ***
SELECT count(*)
FROM srs_sce sce,
srs_scj scj,
men_mre mre,
srs_mst mst,
cam_smo cam,
ins_spr spr,
men_mua mua,
temp_webct_users u
WHERE sce.sce_scjc = scj.scj_code
AND sce.sce_stuc = mre.mre_code
AND mst.mst_code = mre.mre_mstc
AND mre.mre_mrcc = 'STU'
AND mst.mst_code = mua.mua_mstc
AND cam.ayr_code = sce.sce_ayrc
AND cam.spr_code = scj.scj_sprc
AND spr.spr_code = scj.scj_sprc
-- Ignored Join Condition
AND LOWER(mua.mua_extu) = LOWER(u.login)
AND SUBSTR (sce.sce_ayrc, 1, 4) = '2008'
AND sce.sce_stac IN ('RCE', 'RLL', 'RPD', 'RIN', 'RSAS', 'RHL_R', 'RCO', 'RCI', 'RCA');
*** CARTESIAN EXPLAIN PLAN ***
SELECT STATEMENT CHOOSECost: 83
20 NESTED LOOPS Cost: 83 Bytes: 176 Cardinality: 1
18 NESTED LOOPS Cost: 82 Bytes: 148 Cardinality: 1
15 NESTED LOOPS Cost: 80 Bytes: 134 Cardinality: 1
13 NESTED LOOPS Cost: 79 Bytes: 123 Cardinality: 1
10 NESTED LOOPS Cost: 78 Bytes: 98 Cardinality: 1
7 NESTED LOOPS Cost: 77 Bytes: 74 Cardinality: 1
NOTE: The Cartesian product is performed on the men_mre & temp_webct_users tables not the men_mua mua & temp_webct_users tables specified in the join condition.
4 MERGE JOIN CARTESIAN Cost: 74 Bytes: 32 Cardinality: 1
1 TABLE ACCESS FULL EXETER.TEMP_WEBCT_USERS Cost: 3 Bytes: 6 Cardinality: 1
3 BUFFER SORT Cost: 71 Bytes: 1,340,508 Cardinality: 51,558
2 TABLE ACCESS FULL SIPR.MEN_MRE Cost: 71 Bytes: 1,340,508 Cardinality: 51,558
6 TABLE ACCESS BY INDEX ROWID SIPR.SRS_SCE Cost: 3 Bytes: 42 Cardinality: 1
5 INDEX RANGE SCAN SIPR.SRS_SCEI3 Cost: 2 Cardinality: 3
9 TABLE ACCESS BY INDEX ROWID SIPR.SRS_SCJ Cost: 1 Bytes: 24 Cardinality: 1
8 INDEX UNIQUE SCAN SIPR.SRS_SCJP1 Cardinality: 1
12 TABLE ACCESS BY INDEX ROWID SIPR.INS_SPR Cost: 1 Bytes: 25 Cardinality: 1
11 INDEX UNIQUE SCAN SIPR.INS_SPRP1 Cardinality: 1
14 INDEX UNIQUE SCAN SIPR.SRS_MSTP1 Cost: 1 Bytes: 11 Cardinality: 1
17 TABLE ACCESS BY INDEX ROWID SIPR.MEN_MUA Cost: 2 Bytes: 14 Cardinality: 1
16 INDEX RANGE SCAN SIPR.MEN_MUAI3 Cost: 2 Cardinality: 1
19 INDEX RANGE SCAN SIPR.CAM_SMOP1 Cost: 2 Bytes: 28 Cardinality: 1 After speaking with data experts I realised one of the fields being LOWERed for the join condition generally always had uppercase values so I tried modifying the query to use the UPPER function rather than the LOWER one originally used, in this scenario the query executed in seconds and the Cartesian merge had been eradicated which by all accounts is a good result.
*** WORKING QUERY ***
SELECT count(*)
FROM srs_sce sce,
srs_scj scj,
men_mre mre,
srs_mst mst,
cam_smo cam,
ins_spr spr,
men_mua mua,
temp_webct_users u
WHERE sce.sce_scjc = scj.scj_code
AND sce.sce_stuc = mre.mre_code
AND mst.mst_code = mre.mre_mstc
AND mre.mre_mrcc = 'STU'
AND mst.mst_code = mua.mua_mstc
AND cam.ayr_code = sce.sce_ayrc
AND cam.spr_code = scj.scj_sprc
AND spr.spr_code = scj.scj_sprc
-- Working Join Condition
AND UPPER(mua.mua_extu) = UPPER(u.login)
AND SUBSTR (sce.sce_ayrc, 1, 4) = '2008'
AND sce.sce_stac IN ('RCE', 'RLL', 'RPD', 'RIN', 'RSAS', 'RHL_R', 'RCO', 'RCI', 'RCA');
*** WORKING EXPLAIN PLAN ***
SELECT STATEMENT CHOOSECost: 13
20 SORT AGGREGATE Bytes: 146 Cardinality: 1
19 NESTED LOOPS Cost: 13 Bytes: 146 Cardinality: 1
17 NESTED LOOPS Cost: 12 Bytes: 134 Cardinality: 1
15 NESTED LOOPS Cost: 11 Bytes: 115 Cardinality: 1
12 NESTED LOOPS Cost: 10 Bytes: 91 Cardinality: 1
9 NESTED LOOPS Cost: 7 Bytes: 57 Cardinality: 1
6 NESTED LOOPS Cost: 6 Bytes: 31 Cardinality: 1
4 NESTED LOOPS Cost: 5 Bytes: 20 Cardinality: 1
1 TABLE ACCESS FULL EXETER.TEMP_WEBCT_USERS Cost: 3 Bytes: 6 Cardinality: 1
3 TABLE ACCESS BY INDEX ROWID SIPR.MEN_MUA Cost: 2 Bytes: 42 Cardinality: 3
2 INDEX RANGE SCAN EXETER.TEST Cost: 1 Cardinality: 1
5 INDEX UNIQUE SCAN SIPR.SRS_MSTP1 Cost: 1 Bytes: 11 Cardinality: 1
8 TABLE ACCESS BY INDEX ROWID SIPR.MEN_MRE Cost: 2 Bytes: 26 Cardinality: 1
7 INDEX RANGE SCAN SIPR.MEN_MREI2 Cost: 2 Cardinality: 1
11 TABLE ACCESS BY INDEX ROWID SIPR.SRS_SCE Cost: 3 Bytes: 34 Cardinality: 1
10 INDEX RANGE SCAN SIPR.SRS_SCEI3 Cost: 2 Cardinality: 3
14 TABLE ACCESS BY INDEX ROWID SIPR.SRS_SCJ Cost: 1 Bytes: 24 Cardinality: 1
13 INDEX UNIQUE SCAN SIPR.SRS_SCJP1 Cardinality: 1
16 INDEX RANGE SCAN SIPR.CAM_SMOP1 Cost: 2 Bytes: 19 Cardinality: 1
18 INDEX UNIQUE SCAN SIPR.INS_SPRP1 Bytes: 12 Cardinality: 1 *** RESULT ***
COUNT(*)
83299I am still struggling to understand why this would have worked as to my knowledge the LOWER & UPPER functions are similar enough in function and regardless of that why would one version cause the optimiser to effectively ignore a join condition.
If anyone can shed any light on this for me it would be very much appreciated.
Regards,
Kieron
Edited by: Kieron_Bird on Nov 19, 2008 6:09 AM
Edited by: Kieron_Bird on Nov 19, 2008 6:41 AMMy mistake on the predicate information, was in a rush to run off to a meeting when I posted the entry...
*** UPPER Version of the Explain Plan ***
| Id | Operation | Name | Rows | Bytes | Cost | TQ |IN-OUT| PQ Distrib |
| 0 | SELECT STATEMENT | | 1 | 146 | 736 | | | |
| 1 | SORT AGGREGATE | | 1 | 146 | | | | |
| 2 | SORT AGGREGATE | | 1 | 146 | | 86,10 | P->S | QC (RAND) |
|* 3 | HASH JOIN | | 241 | 35186 | 736 | 86,10 | PCWP | |
|* 4 | HASH JOIN | | 774 | 105K| 733 | 86,09 | P->P | HASH |
|* 5 | HASH JOIN | | 12608 | 1489K| 642 | 86,08 | P->P | BROADCAST |
| 6 | NESTED LOOPS | | 14657 | 1531K| 491 | 86,07 | P->P | HASH |
|* 7 | HASH JOIN | | 14657 | 1359K| 490 | 86,07 | PCWP | |
|* 8 | HASH JOIN | | 14371 | 996K| 418 | 86,06 | P->P | HASH |
|* 9 | TABLE ACCESS FULL | SRS_SCE | 3211 | 106K| 317 | 86,00 | S->P | BROADCAST |
|* 10 | HASH JOIN | | 52025 | 1879K| 101 | 86,06 | PCWP | |
|* 11 | TABLE ACCESS FULL | MEN_MRE | 51622 | 1310K| 71 | 86,01 | S->P | HASH |
| 12 | INDEX FAST FULL SCAN| SRS_MSTP1 | 383K| 4119K| 30 | 86,05 | P->P | HASH |
| 13 | TABLE ACCESS FULL | SRS_SCJ | 114K| 2672K| 72 | 86,02 | S->P | HASH |
|* 14 | INDEX UNIQUE SCAN | INS_SPRP1 | 1 | 12 | | 86,07 | PCWP | |
| 15 | TABLE ACCESS FULL | MEN_MUA | 312K| 4268K| 151 | 86,03 | S->P | HASH |
| 16 | INDEX FAST FULL SCAN | CAM_SMOP1 | 527K| 9796K| 91 | 86,09 | PCWP | |
| 17 | TABLE ACCESS FULL | TEMP_WEBCT_USERS | 33276 | 194K| 3 | 86,04 | S->P | HASH |
Predicate Information (identified by operation id):
3 - access(UPPER("MUA"."MUA_EXTU")=UPPER("U"."LOGIN"))
4 - access("CAM"."AYR_CODE"="SCE"."SCE_AYRC" AND "CAM"."SPR_CODE"="SCJ"."SCJ_SPRC")
5 - access("MST"."MST_CODE"="MUA"."MUA_MSTC")
7 - access("SCE"."SCE_SCJC"="SCJ"."SCJ_CODE")
8 - access("SCE"."SCE_STUC"="MRE"."MRE_CODE")
9 - filter(SUBSTR("SCE"."SCE_AYRC",1,4)='2008' AND ("SCE"."SCE_STAC"='RCA' OR "SCE"."SCE_STAC"='RCE' OR
"SCE"."SCE_STAC"='RCI' OR "SCE"."SCE_STAC"='RCO' OR "SCE"."SCE_STAC"='RHL_R' OR "SCE"."SCE_STAC"='RIN' OR
"SCE"."SCE_STAC"='RLL' OR "SCE"."SCE_STAC"='RPD' OR "SCE"."SCE_STAC"='RSAS'))
10 - access("MST"."MST_CODE"="MRE"."MRE_MSTC")
11 - filter("MRE"."MRE_MRCC"='STU')
14 - access("SPR"."SPR_CODE"="SCJ"."SCJ_SPRC")
Note: cpu costing is off
40 rows selected.*** LOWER Version of the Explain Plan ***
| Id | Operation | Name | Rows | Bytes | Cost | TQ |IN-OUT| PQ Distrib |
| 0 | SELECT STATEMENT | | 1 | 146 | 736 | | | |
| 1 | SORT AGGREGATE | | 1 | 146 | | | | |
| 2 | SORT AGGREGATE | | 1 | 146 | | 88,10 | P->S | QC (RAND) |
|* 3 | HASH JOIN | | 257K| 35M| 736 | 88,10 | PCWP | |
|* 4 | HASH JOIN | | 774 | 105K| 733 | 88,09 | P->P | HASH |
|* 5 | HASH JOIN | | 12608 | 1489K| 642 | 88,08 | P->P | BROADCAST |
| 6 | NESTED LOOPS | | 14657 | 1531K| 491 | 88,07 | P->P | HASH |
|* 7 | HASH JOIN | | 14657 | 1359K| 490 | 88,07 | PCWP | |
|* 8 | HASH JOIN | | 14371 | 996K| 418 | 88,06 | P->P | HASH |
|* 9 | TABLE ACCESS FULL | SRS_SCE | 3211 | 106K| 317 | 88,00 | S->P | BROADCAST |
|* 10 | HASH JOIN | | 52025 | 1879K| 101 | 88,06 | PCWP | |
|* 11 | TABLE ACCESS FULL | MEN_MRE | 51622 | 1310K| 71 | 88,01 | S->P | HASH |
| 12 | INDEX FAST FULL SCAN| SRS_MSTP1 | 383K| 4119K| 30 | 88,05 | P->P | HASH |
| 13 | TABLE ACCESS FULL | SRS_SCJ | 114K| 2672K| 72 | 88,02 | S->P | HASH |
|* 14 | INDEX UNIQUE SCAN | INS_SPRP1 | 1 | 12 | | 88,07 | PCWP | |
| 15 | TABLE ACCESS FULL | MEN_MUA | 312K| 4268K| 151 | 88,03 | S->P | HASH |
| 16 | INDEX FAST FULL SCAN | CAM_SMOP1 | 527K| 9796K| 91 | 88,09 | PCWP | |
| 17 | TABLE ACCESS FULL | TEMP_WEBCT_USERS | 33276 | 194K| 3 | 88,04 | S->P | HASH |
Predicate Information (identified by operation id):
3 - access(LOWER("MUA"."MUA_EXTU")=LOWER("U"."LOGIN"))
4 - access("CAM"."AYR_CODE"="SCE"."SCE_AYRC" AND "CAM"."SPR_CODE"="SCJ"."SCJ_SPRC")
5 - access("MST"."MST_CODE"="MUA"."MUA_MSTC")
7 - access("SCE"."SCE_SCJC"="SCJ"."SCJ_CODE")
8 - access("SCE"."SCE_STUC"="MRE"."MRE_CODE")
9 - filter(SUBSTR("SCE"."SCE_AYRC",1,4)='2008' AND ("SCE"."SCE_STAC"='RCA' OR "SCE"."SCE_STAC"='RCE' OR
"SCE"."SCE_STAC"='RCI' OR "SCE"."SCE_STAC"='RCO' OR "SCE"."SCE_STAC"='RHL_R' OR "SCE"."SCE_STAC"='RIN' OR
"SCE"."SCE_STAC"='RLL' OR "SCE"."SCE_STAC"='RPD' OR "SCE"."SCE_STAC"='RSAS'))
10 - access("MST"."MST_CODE"="MRE"."MRE_MSTC")
11 - filter("MRE"."MRE_MRCC"='STU')
14 - access("SPR"."SPR_CODE"="SCJ"."SCJ_SPRC")
Note: cpu costing is off
40 rows selected.As you state something has obviously changed, but nothing obvious has been changed.
We gather statistics via...
exec dbms_stats.gather_schema_stats(ownname => 'USERNAME', estimate_percent => DBMS_STATS.AUTO_SAMPLE_SIZE , degree => 4, granularity => ''ALL'', cascade => TRUE);
We run a script nightly which works out which indexes require a rebuild and rebuild those only it doesn;t just rebuild all indexes.
It would be nice to be able to use the 10g statistics history, but on this instance we aren't yet at that version, hopefully we will be there soon though.
Hope this helps,
Kieron -
Group by causing merge join cartesian?
Hi All,
Database Version: Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bi
Database Version: PL/SQL Release 10.2.0.5.0 - ProductionI've written a query which runs in under a second and produces a few hundred rows (as you can see below). I only wanted to find the number of distinct incident ID's returned so I put a group by at the end and it ran for ever.
I produced an explain plan (as shown below) for the query with the group by in it and it is trying to perform a cartesian join!
WTF is going on? I've never known a group by statement to do that before.
basic structure of the tables is as follows:
SQL> desc answer_master
Name Null? Type
ANSWER_ID NOT NULL NUMBER(12)
INCIDENT_ID NOT NULL NUMBER(12)
PLAN_ID NOT NULL NUMBER(12)
QUESTION_ID NOT NULL NUMBER(12)
SQL> desc question_master
Name Null? Type
QUESTION_PLAN_ID NOT NULL NUMBER(12)
QUESTION_ID NOT NULL NUMBER(12)
QUESTION_TYPE NOT NULL NUMBER(2)
QUESTION_TEXT NOT NULL VARCHAR2(255)
QUESTION_PARENT_ID NUMBER(12)the offending SQL statement
SQL> explain plan for
2 select incident_id
3 from answer_master am
4 where exists (select 'x'
5 from answer_master am1
6 where question_id in (select question_id
7 from question_master qms
8 where question_plan_id = 1477
9 and question_parent_id = 69067
10 and substr(question_text,-3) = 'PDF')
11 and am1.incident_id = am.incident_id)
12 and exists (select 'x'
13 from answer_master am1
14 where question_id in (select question_id
15 from question_master qms
16 where question_plan_id = 1477
17 and question_parent_id = 69067
18 and substr(question_text,-3) != 'PDF')
19 and am1.incident_id = am.incident_id);
Explained.
Elapsed: 00:00:00.01
SQL> set linesize 132;
SQL> select plan_table_output from table(dbms_xplan.display('plan_table',null,'serial'));
PLAN_TABLE_OUTPUT
Plan hash value: 1710257923
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 32 | 171 (6)| 00:00:01 |
|* 1 | HASH JOIN RIGHT SEMI | | 1 | 32 | 171 (6)| 00:00:01 |
| 2 | VIEW | VW_SQ_2 | 71 | 923 | 70 (5)| 00:00:01 |
| 3 | NESTED LOOPS | | 71 | 3266 | 70 (5)| 00:00:01 |
|* 4 | TABLE ACCESS BY INDEX ROWID | QUESTION_MASTER | 1 | 35 | 2 (0)| 00:00:01 |
|* 5 | INDEX RANGE SCAN | QUESTION_MASTER_IX1 | 3 | | 1 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | ANSWER_MASTER_QID_IX2 | 18731 | 201K| 68 (5)| 00:00:01 |
| 7 | NESTED LOOPS | | 304 | 5776 | 100 (5)| 00:00:01 |
| 8 | VIEW | VW_SQ_1 | 14 | 182 | 70 (5)| 00:00:01 |
| 9 | HASH UNIQUE | | 14 | 644 | | |
| 10 | NESTED LOOPS | | 14 | 644 | 70 (5)| 00:00:01 |
|* 11 | TABLE ACCESS BY INDEX ROWID| QUESTION_MASTER | 1 | 35 | 2 (0)| 00:00:01 |
|* 12 | INDEX RANGE SCAN | QUESTION_MASTER_IX1 | 3 | | 1 (0)| 00:00:01 |
|* 13 | INDEX RANGE SCAN | ANSWER_MASTER_QID_IX2 | 18731 | 201K| 68 (5)| 00:00:01 |
|* 14 | INDEX RANGE SCAN | ANSWER_MASTER_QID_IX3 | 22 | 132 | 2 (0)| 00:00:01 |
Predicate Information (identified by operation id):
1 - access("ITEM_2"="AM"."INCIDENT_ID")
4 - filter("QUESTION_PLAN_ID"=1477 AND SUBSTR("QUESTION_TEXT",-3)!='PDF')
5 - access("QUESTION_PARENT_ID"=69067)
6 - access("QUESTION_ID"="QUESTION_ID")
11 - filter("QUESTION_PLAN_ID"=1477 AND SUBSTR("QUESTION_TEXT",-3)='PDF')
12 - access("QUESTION_PARENT_ID"=69067)
13 - access("QUESTION_ID"="QUESTION_ID")
14 - access("ITEM_1"="AM"."INCIDENT_ID")
33 rows selected.
Elapsed: 00:00:00.06
SQL> explain plan for
2 select incident_id
3 from answer_master am
4 where exists (select 'x'
5 from answer_master am1
6 where question_id in (select question_id
7 from question_master qms
8 where question_plan_id = 1477
9 and question_parent_id = 69067
10 and substr(question_text,-3) = 'PDF')
11 and am1.incident_id = am.incident_id)
12 and exists (select 'x'
13 from answer_master am1
14 where question_id in (select question_id
15 from question_master qms
16 where question_plan_id = 1477
17 and question_parent_id = 69067
18 and substr(question_text,-3) != 'PDF')
19 and am1.incident_id = am.incident_id)
20 group by incident_id;
Explained.
Elapsed: 00:00:00.00
SQL> select plan_table_output from table(dbms_xplan.display('plan_table',null,'serial'));
PLAN_TABLE_OUTPUT
Plan hash value: 1433543102
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 98 | 77 (6)| 00:00:01 |
| 1 | HASH GROUP BY | | 1 | 98 | 77 (6)| 00:00:01 |
| 2 | NESTED LOOPS | | 1 | 98 | 76 (4)| 00:00:01 |
| 3 | NESTED LOOPS | | 1 | 87 | 74 (5)| 00:00:01 |
| 4 | NESTED LOOPS | | 1 | 81 | 72 (5)| 00:00:01 |
| 5 | MERGE JOIN CARTESIAN | | 1 | 70 | 4 (0)| 00:00:01 |
|* 6 | TABLE ACCESS BY INDEX ROWID | QUESTION_MASTER | 1 | 35 | 2 (0)| 00:00:01 |
|* 7 | INDEX RANGE SCAN | QUESTION_MASTER_IX1 | 3 | | 1 (0)| 00:00:01 |
| 8 | BUFFER SORT | | 1 | 35 | 2 (0)| 00:00:01 |
|* 9 | TABLE ACCESS BY INDEX ROWID| QUESTION_MASTER | 1 | 35 | 2 (0)| 00:00:01 |
|* 10 | INDEX RANGE SCAN | QUESTION_MASTER_IX1 | 3 | | 1 (0)| 00:00:01 |
|* 11 | INDEX RANGE SCAN | ANSWER_MASTER_QID_IX2 | 18731 | 201K| 68 (5)| 00:00:01 |
|* 12 | INDEX RANGE SCAN | ANSWER_MASTER_QID_IX3 | 22 | 132 | 2 (0)| 00:00:01 |
|* 13 | INDEX RANGE SCAN | ANSWER_MASTER_QID_IX2 | 1 | 11 | 2 (0)| 00:00:01 |
Predicate Information (identified by operation id):
6 - filter("QUESTION_PLAN_ID"=1477 AND SUBSTR("QUESTION_TEXT",-3)='PDF')
7 - access("QUESTION_PARENT_ID"=69067)
9 - filter("QUESTION_PLAN_ID"=1477 AND SUBSTR("QUESTION_TEXT",-3)!='PDF')
10 - access("QUESTION_PARENT_ID"=69067)
11 - access("QUESTION_ID"="QUESTION_ID")
12 - access("AM1"."INCIDENT_ID"="AM"."INCIDENT_ID")
13 - access("QUESTION_ID"="QUESTION_ID" AND "AM1"."INCIDENT_ID"="AM"."INCIDENT_ID")
31 rows selected.
Elapsed: 00:00:00.00
SQL> spool offedit: changing greater-than/less-than signs to != so it displays on forum
Edited by: WhiteHat on Feb 11, 2011 9:37 AMThe "Cartesian" product is nothing "wrong", if (!) the optimizer is right and the number or rows it's picking for that part of the execution plan is really 1. And that can be true, as ORACLE also changed from HASH join to NL. So that NL might be the cause of your performance issue not the Cartesian.
The performance of IN, EXISTS and SUBQUERIES got better over the versions, but I still see no reason in most cases to use them for good readable and performing code.
What you do think about this:
WITH my_selection As
(SELECT incident_id
from question_master qms
where question_plan_id = 1477
and question_parent_id = 69067
and (substr(question_text,-3) = 'PDF'
or and substr(question_text,-3) != 'PDF')
GROUP BY incident_id
HAVING COUNT(*) >1)
SELECT am.incident_id
FROM answer_master am
INNER JOIN my_selection s
ON am.incident_id = s.incident_id
group by am.incident_id; -
Merge Join Cartesian performing bad on 10g
Hello All,
I have a merge join cartesian query that took seconds in 8i. Once the database was upgraded to 10g, the same query is taking hours. In looking at the explain plans, it looks like the access paths are identical.
Has anyone encountered this issue?
Thanks.Sometimes the cost-based optimizer will do cartesian joins to optimize a query. Seeing the word "cartesian" in a query plan is still a major red flag to my eyes. Also, performance changes between versions are usually expected, though most people complain about 9i to 10g results these days :)
Are you sure you need to be doing a cartesian join at all? As Fly pointed out they have a bad reputation, and yours must be running badly as per your post. I personally get more efficient results from hash joins over sort merges these days so that's something you can look at.
Try variations on your query to generate execution plans to see if you can get better results. Don't forget to actually time results since plan statistics have been known to be out-of-touch with actual run performance.
Good luck! -
Hi All,
I have a problem in writing the below query. The below query always shows an MERGE JOIN CARTESIAN in the explain plan due to which the query execution takes lot of time(close to 2 hour for 1000 records).
I'm using "Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi". Please advice if I need to provide more information on this.
Select to_char(a.amnt_date,'DD-MON-YYYY') amnt_date,
SUM(a.AMNT_CON) amount
from amount a,
(SELECT area_no
FROM (SELECT *
FROM area_usage
WHERE area_hier_no = '201064')
CONNECT BY area_par_no = PRIOR area_no
START WITH area_no in (SELECT area_no
FROM area
WHERE area_code in ('EQ'))) b,
(SELECT acc_no
FROM (SELECT *
FROM acc_usage
WHERE acc_hier_no = '1')
CONNECT BY acc_par_no = PRIOR acc_no
START WITH acc_no = 202917) c -- Account Hierarchy
where a.area_no = b.area_no
AND a.acc_no = c.acc_no
GROUP BY a.amnt_date-----
PLAN_TABLE_OUTPUT
PLAN_TABLE_OUTPUT
Plan hash value: 2492729134
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
| 0 | SELECT STATEMENT | | 692 | 22144 | 3327 (32)| 00:00:40 | | |
| 1 | HASH GROUP BY | | 692 | 22144 | 3327 (32)| 00:00:40 | | |
| 2 | TABLE ACCESS BY LOCAL INDEX ROWID | AMOUNT | 138 | 2898 | 3326 (32)| 00:00:40 | | |
| 3 | NESTED LOOPS | | 692 | 22144 | 3326 (32)| 00:00:40 | | |
| 4 | MERGE JOIN CARTESIAN | | 5 | 55 | 13 (0)| 00:00:01 | | |
| 5 | VIEW | | 1 | 5 | 5 (0)| 00:00:01 | | |
|* 6 | CONNECT BY WITH FILTERING | | | | | | | |
| 7 | TABLE ACCESS BY INDEX ROWID | ACC_USAGE | 1 | 24 | 2 (0)| 00:00:01 | | |
|* 8 | INDEX UNIQUE SCAN | ACU_PK | 1 | | 1 (0)| 00:00:01 | | |
| 9 | NESTED LOOPS | | | | | | | |
| 10 | CONNECT BY PUMP | | | | | | | |
|* 11 | TABLE ACCESS BY INDEX ROWID | ACC_USAGE | 1 | 14 | 5 (0)| 00:00:01 | | |
|* 12 | INDEX RANGE SCAN | ACU_AH_HAVE_AS_PARENT_FK_I | 4 | | 1 (0)| 00:00:01 | | |
| 13 | BUFFER SORT | | 5 | 30 | 13 (0)| 00:00:01 | | |
| 14 | VIEW | | 5 | 30 | 8 (0)| 00:00:01 | | |
|* 15 | CONNECT BY WITH FILTERING | | | | | | | |
|* 16 | TABLE ACCESS BY INDEX ROWID | AREA_USAGE | | | | | | |
| 17 | NESTED LOOPS | | 1 | 32 | 2 (0)| 00:00:01 | | |
| 18 | TABLE ACCESS BY INDEX ROWID| AREA | 1 | 21 | 2 (0)| 00:00:01 | | |
|* 19 | INDEX UNIQUE SCAN | AR_AR2_UK | 1 | | 1 (0)| 00:00:01 | | |
|* 20 | INDEX UNIQUE SCAN | AU_PK | 8608 | 94688 | 0 (0)| 00:00:01 | | |
| 21 | NESTED LOOPS | | | | | | | |
| 22 | CONNECT BY PUMP | | | | | | | |
|* 23 | TABLE ACCESS BY INDEX ROWID | AREA_USAGE | 5 | 85 | 8 (0)| 00:00:01 | | |
|* 24 | INDEX RANGE SCAN | ARU_APN_FKI | 8 | | 1 (0)| 00:00:01 | | |
| 25 | PARTITION RANGE ALL | | | | | | 1 | 60 |
| 26 | BITMAP CONVERSION TO ROWIDS | | | | | | | |
| 27 | BITMAP AND | | | | | | | |
| 28 | BITMAP CONVERSION FROM ROWIDS | | | | | | | |
|* 29 | INDEX RANGE SCAN | A_ARH_FK_I | 194K| | 134 (7)| 00:00:02 | 1 | 60 |
| 30 | BITMAP CONVERSION FROM ROWIDS | | | | | | | |
|* 31 | INDEX RANGE SCAN | A_AH_FK_I | 194K| | 488 (41)| 00:00:06 | 1 | 60 |
Predicate Information (identified by operation id):
6 - access("ACC_USAGE"."ACC_PAR_NO"=PRIOR "ACC_USAGE"."ACC_NO")
8 - access("ACC_USAGE"."ACC_NO"=202917 AND "ACC_HIER_NO"=1)
11 - filter("ACC_HIER_NO"=1)
12 - access("ACC_USAGE"."ACC_PAR_NO"=PRIOR "ACC_USAGE"."ACC_NO")
15 - access("AREA_USAGE"."AREA_PAR_NO"=PRIOR "AREA_USAGE"."AREA_NO")
16 - filter("AREA_HIER_NO"=201064)
19 - access("AREA_CODE"='EQ')
20 - access("AREA_HIER_NO"=201064 AND "AREA_USAGE"."AREA_NO"="AREA_NO")
23 - filter("AREA_HIER_NO"=201064)
24 - access("AREA_USAGE"."AREA_PAR_NO"=PRIOR "AREA_USAGE"."AREA_NO")
29 - access("A"."AREA_NO"="B"."AREA_NO")
31 - access("A"."ACC_NO"="C"."ACC_NO")
54 rows selected.
Edited by: Harinath Arasu on Oct 14, 2010 5:31 AMThat's a very small difference, and I wouldn't expect that to cause huge shifts in an execution plan. Either way to gather statistics you would use the DBMS_STATS package.
By the way, your "formatted" execution plan isn't exactly readable in your post.
Looking at your query a little more closely I do recommend you remove the SELECT * from the subqueries. Something like this may suffice:
Select to_char(a.amnt_date,'DD-MON-YYYY') amnt_date,
SUM(a.AMNT_CON) amount
from amount a,
(SELECT area_no
FROM (SELECT area_no
, area_par_no
FROM area_usage
WHERE area_hier_no = '201064')
CONNECT BY area_par_no = PRIOR area_no
START WITH area_no in (SELECT area_no
FROM area
WHERE area_code in ('EQ'))) b,
(SELECT acc_no
FROM (SELECT acc_no
, acc_par_no
FROM acc_usage
WHERE acc_hier_no = '1')
CONNECT BY acc_par_no = PRIOR acc_no
START WITH acc_no = 202917) c -- Account Hierarchy
where a.area_no = b.area_no
AND a.acc_no = c.acc_no
GROUP BY a.amnt_dateUsing a SELECT * and not subsequently using all the columns is hiding valuable information from Oracle. If you provide only the columns used Oracle may be able to find a better execution plan.
Granted I don't know the table structures so you could actually be using all the columns. I still recommend the change because I think it makes the code easier to understand. -
Is merge join cartesian more cpu intensibe than nested loop ?
Hi,
just wonderning which access method is more cpu intensive , lets supposed we got 2 the same row sources and doing joing via merge join cartesian and next case is nested loop .
I know NL can be cpu intensive because of tight loop access , but what abour MJC ?
I can see bufferd sort but not sure is that cpu friendly ?
Regards
GregGHi,
I think in your case it's more accurate to compare a NESTED LOOP (NL) to a MERGE JOIN (MJ), because CARTESIAN MERGE JOIN is a rather special case of MJ.
Merge join sorts its inputs before combining them, and it could be efficient when one or both of inputs are already sorted.
Regarding your question (which is more CPU intensive):
1) if MERGE JOIN involves disk spills, then CPU is probably irrelevant, because disk operations are much more expensive
2) the amount of work to combine rowsources via a MJ depends on how well they are aligned with respect to each other, so I don't think it can be expressed via a simple formula.
For nested loops, the situation is much more simple: you don't need to do any special work do combine the rowsource, so the cost is just the sum of the cost acquiring the outer rowsource plus the number of iterations times the cost of one iteration. If the data is read from disk, then CPU probably won't matter much, if most of reads are logical ones than CPU becomes of a factor (it's hard to tell how much work CPU will have to do per one logical read because there are two many factors here -- how many columns are accessed, how they are located within the block, are there any expensive math functions applied to any of them etc.)
Best regards,
Nikolay -
Why Data Sort is required for Merge Join Transformation
hi,
In my understanding Merge Join works very similar to T SQL joins. But I am trying to understand why Sort is a compulsory requirement? I have looked into following article but not helpful
http://technet.microsoft.com/en-us/library/ms137653(v=sql.105).aspx
aamertaimor
aamertaimorMerge Join is going to walk through two sets in the order that you gave in your input or using the Sort transformation. So, it loads one record from one input and one record from the second input. If the keys match, it will output the row with information
from both inputs. If the left input has a value that is less than the value in right input, then either the row from the left input is sent out with no right input information (if there is an outer join) or if the join is an inner join, the
Merge Join component will get the next row from Left input. If the right input has a value that is less than the value that in the left input, then if there is a full outer join, output the right input with no left input information otherwise, get the
next row from the right input.
Here is the beauty, SSIS only needs to retain a couple rows in memory.
What if Microsoft decided that there is no requirement for sorting? Then in order for the Merge join to work is that it would load all of the rows from one input into memory and then the Merge Join would look up the row in memory. That means
a large amount of memory would be needed.
By the way, if you want Merge Join behavior without the sort, use a Lookup component.
Russel Loski, MCT, MCSE Data Platform/Business Intelligence. Twitter: @sqlmovers; blog: www.sqlmovers.com -
SQL Server 2014 - ColumnStore index Clustered with Merge Join
Hi All,
I would like to know in SQL Server 2014, the new feature "Clustered Columnstore". I had inserted 9 millions of records in to a clustered columnstore table.
Then i do a merge join query to insert and update to another table. The Merge join query is as below :
MERGE [dbo].[Temp_FactSales] AS TARGET
USING [dbo].[FactSales_9m] AS SOURCE
ON
TARGET.SALESID = SOURCE.SALESID
WHEN NOT MATCHED BY TARGET
THEN INSERT
[listid],
[sellerid],
[buyerid],
[eventid],
[dateid],
[qtysold],
[pricepaid],
[commission],
[saletime]
VALUES
SOURCE.[listid],
SOURCE.[sellerid],
SOURCE.[buyerid],
SOURCE.[eventid],
SOURCE.[dateid],
SOURCE.[qtysold],
SOURCE.[pricepaid],
SOURCE.[commission],
SOURCE.[saletime]
WHEN MATCHED THEN
UPDATE
SET
TARGET.[listid] = SOURCE.[listid],
TARGET.[sellerid] = SOURCE.[sellerid],
TARGET.[buyerid] = SOURCE.[buyerid],
TARGET.[eventid] = SOURCE.[eventid],
TARGET.[dateid] = SOURCE.[dateid],
TARGET.[qtysold] = SOURCE.[qtysold],
TARGET.[pricepaid] = SOURCE.[pricepaid],
TARGET.[commission] = SOURCE.[commission],
TARGET.[saletime] = SOURCE.[saletime]
The total execute time is about 10 mins. However if i removed the Columnstore index in the table which using normal primary key. Then the execution time is less than 1 mins.
My question is when using "merge join", is this the behavior for columnstore index ? As from what i read, Columnstore index is great performance in read the data but not doing "Insert, update and Delete"?
Hope anyone can help.
Thanks a lot.Hi TTL1964,
How's everything going?
If there is any progress from your side, feel free to share it in the forum.
Thanks.
Tracy Cai
TechNet Community Support -
Hi,
I have a query which is using MERGE JOIN CARTESIAN (see step 4 and 11 in execution plan bellow) and i believe this is the culprit and query is never returning. Stats are up to date on all tables. This query involves two remote databases which are accessed using VIEWs from local database. Is there any hint which i can use here to avoid merge join Cartesian and test. Thanks. Also pasting query if any one can have any idea.
Select distinct
substr(Revenue.FKDAT,5,2) || '/' || substr(Revenue.FKDAT,7,2) || '/' || substr(Revenue.FKDAT,1,4) INVOICE_DATE
, Revenue.VBELN INVOICE_NR
, Revenue.ARKTX CUSTDEVICE
, (case Revenue.ZU_LOTID
when null then hstg_partinfo_cust_dw.CUSTPARTOUT
when '' then hstg_partinfo_cust_dw.CUSTPARTOUT
Else
(case decode(trim(zodss1b2.biztype),'TNR-E330-BAKE','TNR','')
when 'TNR' then
(case when (select CUSTPARTOUT from hstg_partinfo_cust_dw Where lotid = 'U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9) and stage = 'T998-TFIN' and rownum = 1)
is null then (select CUSTPARTOUT from hstg_partinfo_cust_dw Where lotid = (select parentid from hstg_partinfo_cust_dw where lotid = 'U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9) and stage = 'E330-BAKE' and rownum = 1) and stage = 'T998-TFIN' and rownum = 1)
else (select CUSTPARTOUT from hstg_partinfo_cust_dw Where lotid = 'U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9) and stage = 'T998-TFIN' and rownum = 1)
end)
else hstg_partinfo_cust_dw.CUSTPARTOUT
end)
end) CUSTPARTOUT
, Revenue.PONUM DO_NR
, Revenue.BSTKD PO_NR
, Revenue.FKIMG GD_QTY
, Revenue.zkbetr GD_PRICE
, Revenue.NETWR GD_AMT
, '' REJ_QTY
, '' REJ_PRICE
, '' REJ_AMT
, Revenue.ZU_LOTID LOTTYPE
,(case Revenue.ZU_LOTID
when null then substr(Revenue.ZU_PRTID,1,instr(Revenue.ZU_PRTID,'.') -1)
when '' then substr(Revenue.ZU_PRTID,1,instr(Revenue.ZU_PRTID,'.') -1)
Else
(case decode(trim(zodss1b2.biztype),'TNR-E330-BAKE','TNR','')
when 'TNR' then
(case when (select substr(PARTID,1,instr(PARTID,'.') -1) from hstg_partinfo_cust_dw Where lotid = 'U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9) and stage = 'T998-TFIN' and rownum = 1)
is null then (select substr(PARTID,1,instr(PARTID,'.') -1) from hstg_partinfo_cust_dw Where lotid = (select parentid from hstg_partinfo_cust_dw where lotid = 'U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9) and stage = 'E330-BAKE' and rownum = 1) and stage = 'T998-TFIN' and rownum = 1)
else (select substr(PARTID,1,instr(PARTID,'.') -1) from hstg_partinfo_cust_dw Where lotid = 'U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9) and stage = 'T998-TFIN' and rownum = 1)
end)
else substr(Revenue.ZU_PRTID,1,instr(Revenue.ZU_PRTID,'.') -1)
end)
end) PARTID
, '' TESTER
, substr(zodss1b2.BSTDK,5,2) || '/' || substr(zodss1b2.BSTDK,7,2) || '/' || substr(zodss1b2.BSTDK,1,4) DO_DATE
, hstg_partinfo_cust_dw.package package
, hstg_partinfo_cust_dw.pincount pincount
, (select c.catg07 from ops$ods_adm.catg c where c.partid=revenue.zu_prtid) as package_size
, (select c.catg09 from ops$ods_adm.catg c where c.partid=revenue.zu_prtid) as wafer_size
, decode(trim(zodss1b2.biztype),'TNR-E330-BAKE','TNR','') BIZTYPE
--from revenue, zodss1b2,dim_partinfo
from revenue, zodss1b2,hstg_partinfo_cust_dw
where Revenue.Remarks <> 'CANCELLED'
and Revenue.FKART in ('ZUF2')
and Revenue.KUNNR in ('BC1001','BC1002','BC1003')
and Revenue.FKDAT like '201008%'
and Revenue.FKIMG > 0
and Revenue.kpein > 0
and Revenue.ZU_LOTID = zodss1b2.charg
and Revenue.MATNR = zodss1b2.matnr
--and Revenue.ZU_PRTID = dim_partinfo.partid
and hstg_partinfo_cust_dw.lotid='U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9)
Union all
Select distinct
substr(Revenue.FKDAT,5,2) || '/' || substr(Revenue.FKDAT,7,2) || '/' || substr(Revenue.FKDAT,1,4) INVOICE_DATE
, Revenue.VBELN INVOICE_NR
, Revenue.ARKTX CUSTDEVICE
, hstg_partinfo_cust_dw.CUSTPARTOUT CUSTPARTOUT
, Revenue.PONUM DO_NR
, Revenue.BSTKD PO_NR
, Revenue.FKIMG GD_QTY
, Revenue.zkbetr GD_PRICE
, Revenue.NETWR GD_AMT
, '' REJ_QTY
, '' REJ_PRICE
, '' REJ_AMT
, Revenue.ZU_LOTID LOTTYPE
, substr(Revenue.ZU_PRTID,1,instr(Revenue.ZU_PRTID,'.') -1) PARTID
, '' TESTER
, substr(zodsscrap.BSTDK,5,2) || '/' || substr(zodsscrap.BSTDK,7,2) || '/' || substr(zodsscrap.BSTDK,1,4) DO_DATE
, hstg_partinfo_cust_dw.package package
, hstg_partinfo_cust_dw.pincount pincount
, (select c.catg07 from ops$ods_adm.catg c where c.partid=revenue.zu_prtid) as package_size
, (select c.catg09 from ops$ods_adm.catg c where c.partid=revenue.zu_prtid) as wafer_size
--from revenue, zodsscrap,dim_partinfo
from revenue, zodsscrap,hstg_partinfo_cust_dw
where Revenue.Remarks <> 'CANCELLED'
and Revenue.FKART in ('ZUF2')
and Revenue.KUNNR in ('BC1001','BC1002','BC1003')
and Revenue.FKDAT like '201008%'
and Revenue.FKIMG > 0
and Revenue.kpein > 0
and Revenue.ZU_LOTID = zodsscrap.charg
--and Revenue.ZU_PRTID = dim_partinfo.partid
and hstg_partinfo_cust_dw.lotid='U' || substr(Revenue.ZU_LOTID,2,7) || '.' || substr(Revenue.ZU_LOTID,9);hstg_partinfo_cust_dw is a view which is on remote database USGDWDBP and is queried using synonym in the local database.
revenue is also a view which is in another remote database which is queried using synonym in local database.
ODS Plan
| Id | Operation | Name | Rows | Bytes | Cost | Inst |IN-OUT|
| 0 | SELECT STATEMENT | | 2 | 8532 | 999 | | |
| 1 | UNION-ALL | | | | | | |
| 2 | SORT UNIQUE | | 1 | 4277 | 985 | | |
| 3 | NESTED LOOPS | | 1 | 4277 | 983 | | |
| 4 | MERGE JOIN CARTESIAN| | 1 | 4107 | 980 | | |
| 5 | REMOTE | | 1 | 4076 | 2 | USGDW~ | R->S |
| 6 | BUFFER SORT | | 918K| 27M| 978 | | |
| 7 | TABLE ACCESS FULL | ZODSS1B2 | 918K| 27M| 978 | | |
| 8 | REMOTE | | 1 | 170 | 3 | SAPP0~ | R->S |
| 9 | SORT UNIQUE | | 1 | 4255 | 14 | | |
| 10 | NESTED LOOPS | | 1 | 4255 | 12 | | |
| 11 | MERGE JOIN CARTESIAN| | 1 | 4096 | 9 | | |
| 12 | REMOTE | | 1 | 4076 | 2 | USGDW~ | R->S |
| 13 | BUFFER SORT | | 1755 | 35100 | 7 | | |
| 14 | TABLE ACCESS FULL | ZODSSCRAP | 1755 | 35100 | 7 | | |
| 15 | REMOTE | | 1 | 159 | 3 | SAPP0~ | R->S |
---------------------------------------------------------------------------------------Thanks in advance
SalmanDear Salman Qureshi,
First of all please see Jonathan Lewis' post about the merge join cartesian;
http://jonathanlewis.wordpress.com/2006/12/13/cartesian-merge-join/
You may also check the Thomas Kyte's thread;
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:4105951726381
You merge join cartesian cost is so low that the impact will not be that high.
Yes, there are ways to avoid the merge join cartesian. First you have to check the where conditions. Merge join cartesian steps are happening because of the bad SQL algorithm OR sometimes Oracle needs to have a merge join cartesian. At the end, it is up to your SQL.
After than that, you can also use the ORDERED CBO hint to avoid the merge join cartesian.
Please add below line after the SELECT clause and see the new execution plan;
/*+ ORDERED */I hope your problem will be fixed when you show the CBO to use the ORDERED Oracle hint.
About the ORDERED Oracle hint, please check it out;
http://download.oracle.com/docs/cd/B14117_01/server.101/b10752/hintsref.htm#5555
ORDERED
The ORDERED hint causes Oracle to join tables in the order in which they appear in the FROM clause.
If you omit the ORDERED hint from a SQL statement performing a join, then the optimizer chooses the order in which to join the tables. You might want to use the ORDERED hint to specify a join order if you know something about the number of rows selected from each table that the optimizer does not. Such information lets you choose an inner and outer table better than the optimizer could.Hope That Helps.
Ogan -
I have query whch when I do showplan ...
It show merge join is happening at client place and
not at mine....
Can anybody suggest as what is reason behind this
merge join to occur.
Merge join should not occur at any cost...
It is slowing down my performance of Query and taking lot of time...Impossible to tell without viewing the query. Please post the 'offending' statement.
-
hi all,
In my project am using table type varibale as paramter with the columns of id salary to the table. In my procedure i have used merge join with the passed table type variable with the existing table.Insert the value given id does not match,update the value
to the id if matched this is the condition to my merge join. I passed the below table as parameter. Every id is new so i passed the id as -1. My problem is, when i insert the below table with my procedure it is not maintainig order. For example during insert
600 should be first,300 should be second and 800 should be third row, but am not getting like that it always save ascending order 300,600,800 respectively. Am i facing this issue since am using merge join?if so can any one help me to maintain order during
insert.
My table type variable paramter value has been given below
id salary
-1 600
-1 300
-1 800For example during insert 600 should be first,300 should be second and 800 should be third row,
Can you tell me on what's the rule that determines the order of insertion?
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page
Maybe you are looking for
-
Create a JSP client to access Web Services
Hi, I am working on a JSP client in WL 8.1 to access a non-WebLogic web service, with it's WSDL URL. I am trying to run the sample code from the WebLogic web services web site in my JSP but am running into errors with the Service class from JAX-RPC.
-
Suddenly the ADS stopped working after near to a month period. When trying to execute the program FP_PDF_TEST_00 , i am getting the following error SOAP runtime exception: CSoapExceptionTransport : (100101). As per the this discussion in the thread a
-
When overwritting to an existing file , what would happen it the existing file has a bigger size than the data written in it ? I haven't tried it right now , but i would know how this is done in Java . Thanks in advance for any valuable answer.
-
Sales order number not showin in ATP sccreen
Hello All Can you please answer my quesiton regarding ATP, i created a sales order, system performed ATP check and saved. In MD04 screen, i see the sales order number as a demand, but when i navigate to ATP quantities screen from
-
Hello, I have two select statements as Query 1 : Select col1,col2,col3,col4,value_a from table_1 Query 2: Select col1,col2,col3,col4,value_b from table_2 I want to join these two select statement to get below result. col1 col2 col3 col4 value_a value