UNION ALL destroys performance
Strangely, we have 2 queries, which when they run independently, run pretty fast (2 seconds for one, instantaneous for the other), but when they're concatenated into a UNION ALL query with no ORDER BY clause, it takes 45 seconds for it to run.
We tried pushing the SQL into a WITH clause, and some other subquery rewrites, but no perfomance boost. Has anyone seen this before?
version 10.1.0.4
--=Chuck
Here are the results from DBMS_XPLAN. The prior explain plan was copied & pasted from TOAD.
SQL> explain plan for SELECT rownum, RUN_DATE.RUN_DATE,
2 RUN_TIME.RUN_TIME,
3 cc_report_work.report_name,
4 cc_report_work.line_sequence,
5 PURGE_DATE.PURGE_DATE,
6 NULL AS LAD_GROUP,
7 NULL AS NP_REASON,
8 SUBSTR (cc_report_work.print_line, 11, 8) AS FILENR,
9 SUBSTR (cc_report_work.print_line, 21, 27) AS CLAIMANT,
10 SUBSTR (cc_report_work.print_line, 48, 10) AS LAD,
11 SUBSTR (cc_report_work.print_line, 62, 11) AS SSN
12 FROM claims.cc_report_work,
13 (SELECT report_name, SUBSTR (print_line, 7, 8) AS RUN_DATE
14 FROM claims.cc_report_work
15 WHERE report_name = 'CC2518B' AND line_sequence = 8) RUN_DATE,
16 (SELECT report_name, SUBSTR (print_line, 124, 5) AS RUN_TIME
17 FROM claims.cc_report_work
18 WHERE report_name = 'CC2518A' AND line_sequence = 2) RUN_TIME,
19 (SELECT DISTINCT
20 report_name, SUBSTR (print_line, 66, 10) AS PURGE_DATE
21 FROM claims.cc_report_work
22 WHERE cc_report_work.report_name = 'CC2518B'
23 AND SUBSTR (cc_report_work.print_line, 54, 11) =
24 'Purged Thru') PURGE_DATE
25 WHERE cc_report_work.report_name = 'CC2518B'
26 AND SUBSTR (cc_report_work.print_line, 14, 1) = ' '
27 AND TRIM (cc_report_work.print_line) <>
28 'M A N U A L P U R G E L I S T I N G'
29 AND cc_report_work.report_name = RUN_DATE.report_name
30 AND cc_report_work.report_name = PURGE_DATE.report_name;
Explained.
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
Plan hash value: 3196380662
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|Time |
PLAN_TABLE_OUTPUT
| 0 | SELECT STATEMENT | | 1 | 206 | 16903 (2)|00:03:23 |
| 1 | COUNT | | | | | |
| 2 | MERGE JOIN CARTESIAN | | 1 | 206 | 16903 (2)|00:03:23 |
|* 3 | HASH JOIN | | 1 | 192 | 12674 (2)|00:02:33 |
| 4 | MERGE JOIN CARTESIAN| | 1 | 128 | 8445 (2)|00:01:42 |
|* 5 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4222 (2)|00:00:51 |
| 6 | BUFFER SORT | | 1 | 64 | 4222 (2)|00:00:51 |
|* 7 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4222 (2)|00:00:51 |
|* 8 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4229 (2)|00:00:51 |
| 9 | BUFFER SORT | | 1 | 14 | 12675 (2)|00:02:33 |
| 10 | VIEW | | 1 | 14 | 4230 (2)|00:00:51 |
| 11 | SORT UNIQUE | | 1 | 62 | 4230 (2)|00:00:51 |
|* 12 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 62 | 4229 (2)|00:00:51 |
Predicate Information (identified by operation id):
PLAN_TABLE_OUTPUT
3 - access("CC_REPORT_WORK"."REPORT_NAME"="REPORT_NAME")
5 - filter("REPORT_NAME"='CC2518B' AND "LINE_SEQUENCE"=8)
7 - filter("REPORT_NAME"='CC2518A' AND "LINE_SEQUENCE"=2)
8 - filter("CC_REPORT_WORK"."REPORT_NAME"='CC2518B' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",14,1)=' ' AND
TRIM("CC_REPORT_WORK"."PRINT_LINE")<>'M A N U A L P U R G E L I S T I N G')
12 - filter("CC_REPORT_WORK"."REPORT_NAME"='CC2518B' AND
PLAN_TABLE_OUTPUT
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",54,11)='Purged Thru')
31 rows selected.
SQL> explain plan for SELECT rownum, RUN_DATE.RUN_DATE,
2 RUN_TIME.RUN_TIME,
3 cc_report_work.report_name,
4 cc_report_work.line_sequence,
5 PURGE_DATE.PURGE_DATE,
6 LAD_GROUP.LAD_GROUP,
7 NP_REASON.NP_REASON,
8 SUBSTR (cc_report_work.print_line, 11, 8) AS FILENR,
9 SUBSTR (cc_report_work.print_line, 21, 31) AS CLAIMANT,
10 SUBSTR (cc_report_work.print_line, 52, 10) AS LAD,
11 SUBSTR (cc_report_work.print_line, 70, 11) AS SSN
12 FROM claims.cc_report_work,
13 (SELECT DISTINCT
14 report_name, SUBSTR (print_line, 7, 8) AS RUN_DATE
15 FROM claims.cc_report_work
16 WHERE report_name = 'CC2518C'
17 AND SUBSTR (print_line, 7, 8) LIKE '%/%/%') RUN_DATE,
18 (SELECT report_name, SUBSTR (print_line, 124, 5) AS RUN_TIME
19 FROM claims.cc_report_work
20 WHERE report_name = 'CC2518A' AND line_sequence = 2) RUN_TIME,
21 (SELECT DISTINCT
22 report_name, SUBSTR (print_line, 67, 10) AS PURGE_DATE
23 FROM claims.cc_report_work
24 WHERE report_name = 'CC2518C'
25 AND SUBSTR (print_line, 37, 29) =
26 'Claims with No Activity since') PURGE_DATE,
27 (SELECT TRIM (print_line) AS LAD_GROUP, page_advance
28 FROM claims.cc_report_work
29 WHERE report_name = 'CC2518C'
30 AND SUBSTR (print_line, 11, 13) = 'Last Activity')
31 LAD_GROUP,
32 (SELECT TRIM (print_line) AS NP_REASON,
33 page_advance,
34 line_sequence,
35 LEAD (page_advance, 1, NULL)
36 OVER (ORDER BY page_advance, line_sequence)
37 AS NEXT_PAGE,
38 LEAD (line_sequence, 1, NULL)
39 OVER (ORDER BY page_advance, line_sequence)
40 AS NEXT_LINE
41 FROM claims.cc_report_work
42 WHERE cc_report_work.report_name = 'CC2518C'
43 AND SUBSTR (cc_report_work.print_line, 11, 20) =
44 'Reason for Non-Purge') NP_REASON
45 WHERE cc_report_work.report_name = 'CC2518C'
46 AND cc_report_work.page_advance = LAD_GROUP.page_advance
47 AND cc_report_work.report_name = PURGE_DATE.report_name
48 AND cc_report_work.report_name = RUN_DATE.report_name
49 AND SUBSTR (cc_report_work.print_line, 14, 1) = '-'
50 AND ( (cc_report_work.page_advance = NP_REASON.page_advance
51 AND cc_report_work.line_sequence BETWEEN NP_REASON.line_sequence
52 AND NP_REASON.NEXT_LINE)
53 OR ( cc_report_work.page_advance = NP_REASON.page_advance
54 AND NP_REASON.NEXT_PAGE IS NULL
55 AND NP_REASON.NEXT_LINE IS NULL
56 AND cc_report_work.line_sequence > NP_REASON.line_sequence));
Explained.
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
Plan hash value: 1773249683
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
PLAN_TABLE_OUTPUT
| 0 | SELECT STATEMENT | | 1 | 381 | 25361 (2)| 00:05:05 |
| 1 | COUNT | | | || |
| 2 | NESTED LOOPS | | 1 | 381 | 25361 (2)| 00:05:05 |
| 3 | MERGE JOIN CARTESIAN | | 1 | 221 | 21131 (2)| 00:04:14 |
| 4 | MERGE JOIN CARTESIAN | | 1 | 207 | 16902 (2)| 00:03:23 |
|* 5 | HASH JOIN | | 1 | 194 | 12672 (2)| 00:02:33 |
| 6 | MERGE JOIN CARTESIAN| | 1 | 128 | 8447 (2)| 00:01:42 |
|* 7 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4222 (2)| 00:00:51 |
| 8 | BUFFER SORT | | 1 | 64 | 4225 (2)| 00:00:51 |
|* 9 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4225 (2)| 00:00:51 |
|* 10 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 66 | 4225 (2)| 00:00:51 |
| 11 | BUFFER SORT | | 1 | 13 | 12677 (2)| 00:02:33 |
| 12 | VIEW | | 1 | 13 | 4230 (2)| 00:00:51 |
| 13 | SORT UNIQUE | | 1 | 62 | 4230 (2)| 00:00:51 |
|* 14 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 62 | 4229 (2)| 00:00:51 |
| 15 | BUFFER SORT | | 1 | 14 | 16902 (2)| 00:03:23 |
| 16 | VIEW | | 1 | 14 | 4230 (2)| 00:00:51 |
| 17 | SORT UNIQUE | | 1 | 62 | 4230 (2)| 00:00:51 |
|* 18 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 62 | 4229 (2)| 00:00:51 |
|* 19 | VIEW | | 1 | 160 | 4230 (2)| 00:00:51 |
| 20 | WINDOW SORT | | 1 | 66 | 4230 (2)| 00:00:51 |
|* 21 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 66 | 4229 (2)| 00:00:51 |
Predicate Information (identified by operation id):
5 - access("CC_REPORT_WORK"."PAGE_ADVANCE"="PAGE_ADVANCE")
7 - filter("REPORT_NAME"='CC2518A' AND "LINE_SEQUENCE"=2)
9 - filter("PAGE_ADVANCE" IS NOT NULL AND "REPORT_NAME"='CC2518C' AND
SUBSTR("PRINT_LINE",11,13)='Last Activity')
10 - filter("CC_REPORT_WORK"."PAGE_ADVANCE" IS NOT NULL AND
"CC_REPORT_WORK"."REPORT_NAME"='CC2518C' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",14,1)='-')
14 - filter("REPORT_NAME"='CC2518C' AND SUBSTR("PRINT_LINE",7,8) LIKE '%/%/%')
18 - filter("REPORT_NAME"='CC2518C' AND SUBSTR("PRINT_LINE",37,29)='Claims with No Activity since')
19 - filter("CC_REPORT_WORK"."PAGE_ADVANCE"="NP_REASON"."PAGE_ADVANCE" AND
"CC_REPORT_WORK"."PAGE_ADVANCE" IS NOT NULL AND
"CC_REPORT_WORK"."LINE_SEQUENCE">="NP_REASON"."LINE_SEQUENCE" AND
"CC_REPORT_WORK"."LINE_SEQUENCE"<="NP_REASON"."NEXT_LINE" OR
"CC_REPORT_WORK"."PAGE_ADVANCE"="NP_REASON"."PAGE_ADVANCE" AND
"CC_REPORT_WORK"."PAGE_ADVANCE" IS NOT NULL AND "NP_REASON"."NEXT_PAGE" IS NULL AND
"NP_REASON"."NEXT_LINE" IS NULL AND "CC_REPORT_WORK"."LINE_SEQUENCE">"NP_REASON"."LINE_SEQUENCE")
21 - filter("CC_REPORT_WORK"."REPORT_NAME"='CC2518C' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",11,20)='Reason for Non-Purge')
52 rows selected.
SQL> explain plan for SELECT rownum, RUN_DATE.RUN_DATE,
2 RUN_TIME.RUN_TIME,
3 cc_report_work.report_name,
4 cc_report_work.line_sequence,
5 PURGE_DATE.PURGE_DATE,
6 NULL AS LAD_GROUP,
7 NULL AS NP_REASON,
8 SUBSTR (cc_report_work.print_line, 11, 8) AS FILENR,
9 SUBSTR (cc_report_work.print_line, 21, 27) AS CLAIMANT,
10 SUBSTR (cc_report_work.print_line, 48, 10) AS LAD,
11 SUBSTR (cc_report_work.print_line, 62, 11) AS SSN
12 FROM claims.cc_report_work,
13 (SELECT report_name, SUBSTR (print_line, 7, 8) AS RUN_DATE
14 FROM claims.cc_report_work
15 WHERE report_name = 'CC2518B' AND line_sequence = 8) RUN_DATE,
16 (SELECT report_name, SUBSTR (print_line, 124, 5) AS RUN_TIME
17 FROM claims.cc_report_work
18 WHERE report_name = 'CC2518A' AND line_sequence = 2) RUN_TIME,
19 (SELECT DISTINCT
20 report_name, SUBSTR (print_line, 66, 10) AS PURGE_DATE
21 FROM claims.cc_report_work
22 WHERE cc_report_work.report_name = 'CC2518B'
23 AND SUBSTR (cc_report_work.print_line, 54, 11) =
24 'Purged Thru') PURGE_DATE
25 WHERE cc_report_work.report_name = 'CC2518B'
26 AND SUBSTR (cc_report_work.print_line, 14, 1) = ' '
27 AND TRIM (cc_report_work.print_line) <>
28 'M A N U A L P U R G E L I S T I N G'
29 AND cc_report_work.report_name = RUN_DATE.report_name
30 AND cc_report_work.report_name = PURGE_DATE.report_name
31 UNION ALL
32 SELECT rownum, RUN_DATE.RUN_DATE,
33 RUN_TIME.RUN_TIME,
34 cc_report_work.report_name,
35 cc_report_work.line_sequence,
36 PURGE_DATE.PURGE_DATE,
37 LAD_GROUP.LAD_GROUP,
38 NP_REASON.NP_REASON,
39 SUBSTR (cc_report_work.print_line, 11, 8) AS FILENR,
40 SUBSTR (cc_report_work.print_line, 21, 31) AS CLAIMANT,
41 SUBSTR (cc_report_work.print_line, 52, 10) AS LAD,
42 SUBSTR (cc_report_work.print_line, 70, 11) AS SSN
43 FROM claims.cc_report_work,
44 (SELECT DISTINCT
45 report_name, SUBSTR (print_line, 7, 8) AS RUN_DATE
46 FROM claims.cc_report_work
47 WHERE report_name = 'CC2518C'
48 AND SUBSTR (print_line, 7, 8) LIKE '%/%/%') RUN_DATE,
49 (SELECT report_name, SUBSTR (print_line, 124, 5) AS RUN_TIME
50 FROM claims.cc_report_work
51 WHERE report_name = 'CC2518A' AND line_sequence = 2) RUN_TIME,
52 (SELECT DISTINCT
53 report_name, SUBSTR (print_line, 67, 10) AS PURGE_DATE
54 FROM claims.cc_report_work
55 WHERE report_name = 'CC2518C'
56 AND SUBSTR (print_line, 37, 29) =
57 'Claims with No Activity since') PURGE_DATE,
58 (SELECT TRIM (print_line) AS LAD_GROUP, page_advance
59 FROM claims.cc_report_work
60 WHERE report_name = 'CC2518C'
61 AND SUBSTR (print_line, 11, 13) = 'Last Activity')
62 LAD_GROUP,
63 (SELECT TRIM (print_line) AS NP_REASON,
64 page_advance,
65 line_sequence,
66 LEAD (page_advance, 1, NULL)
67 OVER (ORDER BY page_advance, line_sequence)
68 AS NEXT_PAGE,
69 LEAD (line_sequence, 1, NULL)
70 OVER (ORDER BY page_advance, line_sequence)
71 AS NEXT_LINE
72 FROM claims.cc_report_work
73 WHERE cc_report_work.report_name = 'CC2518C'
74 AND SUBSTR (cc_report_work.print_line, 11, 20) =
75 'Reason for Non-Purge') NP_REASON
76 WHERE cc_report_work.report_name = 'CC2518C'
77 AND cc_report_work.page_advance = LAD_GROUP.page_advance
78 AND cc_report_work.report_name = PURGE_DATE.report_name
79 AND cc_report_work.report_name = RUN_DATE.report_name
80 AND SUBSTR (cc_report_work.print_line, 14, 1) = '-'
81 AND ( (cc_report_work.page_advance = NP_REASON.page_advance
82 AND cc_report_work.line_sequence BETWEEN NP_REASON.line_sequence
83 AND NP_REASON.NEXT_LINE)
84 OR ( cc_report_work.page_advance = NP_REASON.page_advance
85 AND NP_REASON.NEXT_PAGE IS NULL
86 AND NP_REASON.NEXT_LINE IS NULL
87 AND cc_report_work.line_sequence > NP_REASON.line_sequence));
Explained.
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
Plan hash value: 3197798840
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
PLAN_TABLE_OUTPUT
| 0 | SELECT STATEMENT | | 2 | 609 | 42264 (61)| 00:08:28 |
| 1 | UNION-ALL | | | | | |
| 2 | COUNT | | | | | |
| 3 | MERGE JOIN CARTESIAN | | 1 | 206 | 16903 (2)| 00:03:23 |
|* 4 | HASH JOIN | | 1 | 192 | 12674 (2)| 00:02:33 |
| 5 | MERGE JOIN CARTESIAN | | 1 | 128 | 8445 (2)| 00:01:42 |
|* 6 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4222 (2)| 00:00:51 |
| 7 | BUFFER SORT | | 1 | 64 | 4222 (2)| 00:00:51 |
|* 8 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4222 (2)| 00:00:51 |
|* 9 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4229 (2)| 00:00:51 |
| 10 | BUFFER SORT | | 1 | 14 | 12675 (2)| 00:02:33 |
| 11 | VIEW | | 1 | 14 | 4230 (2)| 00:00:51 |
| 12 | SORT UNIQUE | | 1 | 62 | 4230 (2)| 00:00:51 |
|* 13 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 62 | 4229 (2)| 00:00:51 |
| 14 | COUNT | | | | | |
| 15 | NESTED LOOPS | | 1 | 403 | 25361 (2)| 00:05:05 |
| 16 | MERGE JOIN CARTESIAN | | 1 | 221 | 21131 (2)| 00:04:14 |
| 17 | MERGE JOIN CARTESIAN | | 1 | 207 | 16902 (2)| 00:03:23 |
|* 18 | HASH JOIN | | 1 | 194 | 12672 (2)| 00:02:33 |
| 19 | MERGE JOIN CARTESIAN| | 1 | 128 | 8447 (2)| 00:01:42 |
|* 20 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4222 (2)| 00:00:51 |
| 21 | BUFFER SORT | | 1 | 64 | 4225 (2)| 00:00:51 |
|* 22 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 64 | 4225 (2)| 00:00:51 |
|* 23 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 66 | 4225 (2)| 00:00:51 |
| 24 | BUFFER SORT | | 1 | 13 | 12677 (2)| 00:02:33 |
| 25 | VIEW | | 1 | 13 | 4230 (2)| 00:00:51 |
| 26 | SORT UNIQUE | | 1 | 62 | 4230 (2)| 00:00:51 |
|* 27 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 62 | 4229 (2)| 00:00:51 |
| 28 | BUFFER SORT | | 1 | 14 | 16902 (2)| 00:03:23 |
| 29 | VIEW | | 1 | 14 | 4230 (2)| 00:00:51 |
| 30 | SORT UNIQUE | | 1 | 62 | 4230 (2)| 00:00:51 |
|* 31 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 62 | 4229 (2)| 00:00:51 |
|* 32 | VIEW | | 1 | 182 | 4230 (2)| 00:00:51 |
| 33 | WINDOW SORT | | 1 | 66 | 4230 (2)| 00:00:51 |
|* 34 | TABLE ACCESS FULL | CC_REPORT_WORK | 1 | 66 | 4229 (2)| 00:00:51 |
Predicate Information (identified by operation id):
4 - access("CC_REPORT_WORK"."REPORT_NAME"="REPORT_NAME")
6 - filter("REPORT_NAME"='CC2518B' AND "LINE_SEQUENCE"=8)
8 - filter("REPORT_NAME"='CC2518A' AND "LINE_SEQUENCE"=2) 9 - filter("CC_REPORT_WORK"."REPORT_NAME"='CC2518B' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",14,1)=' ' AND
TRIM("CC_REPORT_WORK"."PRINT_LINE")<>'M A N U A L P U R G E LI S T I N G')
13 - filter("CC_REPORT_WORK"."REPORT_NAME"='CC2518B' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",54,11)='Purged Thru')
18 - access("CC_REPORT_WORK"."PAGE_ADVANCE"="PAGE_ADVANCE")
20 - filter("REPORT_NAME"='CC2518A' AND "LINE_SEQUENCE"=2)
22 - filter("PAGE_ADVANCE" IS NOT NULL AND "REPORT_NAME"='CC2518C' AND
SUBSTR("PRINT_LINE",11,13)='Last Activity')
23 - filter("CC_REPORT_WORK"."PAGE_ADVANCE" IS NOT NULL AND
"CC_REPORT_WORK"."REPORT_NAME"='CC2518C' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",14,1)='-')
27 - filter("REPORT_NAME"='CC2518C' AND SUBSTR("PRINT_LINE",7,8) LIKE '%/%/%')
31 - filter("REPORT_NAME"='CC2518C' AND SUBSTR("PRINT_LINE",37,29)='Claims with No
Activity since')
32 - filter("CC_REPORT_WORK"."PAGE_ADVANCE"="NP_REASON"."PAGE_ADVANCE" AND
"CC_REPORT_WORK"."PAGE_ADVANCE" IS NOT NULL AND
"CC_REPORT_WORK"."LINE_SEQUENCE">="NP_REASON"."LINE_SEQUENCE" AND
"CC_REPORT_WORK"."LINE_SEQUENCE"<="NP_REASON"."NEXT_LINE" OR
"CC_REPORT_WORK"."PAGE_ADVANCE"="NP_REASON"."PAGE_ADVANCE" AND
"CC_REPORT_WORK"."PAGE_ADVANCE" IS NOT NULL AND "NP_REASON"."NEXT_PAGE" IS NULL AND
"NP_REASON"."NEXT_LINE" IS NULL AND "CC_REPORT_WORK"."LINE_SEQUENCE">"NP_REASON"."LINE_SEQUENCE")
34 - filter("CC_REPORT_WORK"."REPORT_NAME"='CC2518C' AND
SUBSTR("CC_REPORT_WORK"."PRINT_LINE",11,20)='Reason for Non-Purge')
73 rows selected.--=cf
Similar Messages
-
Improve performance with union all
Hello there,
Oracle Database 11g Release 11.1.0.7.0 - 64bit Production
PL/SQL Release 11.1.0.7.0 - Production
SQL> show parameter optimizer
ORA-00942: Tabel of view bestaat niet. (Does not exist)I have the following query using the following input variables
- id
- startdate
- enddate
The query has the following format
- assume that the number of columns are the same
- t1 != t3 and t2 != t4
select ct.*
from
select t1.*
from tabel1 t1
join tabel2 t2
on t2.key = t1.key
union all
select t3.*
from tabel3 t3
join tabel4 t4
on t4.key = t3.key
where ct.id = :id
and ct.date >= :startdate
and ct.date < :enddate
order by ct.dateIt is performing really slow, after the first read it performs fast.
I tried the following thing, which was actually even slower!
with t1c as
select t1.*
from tabel1 t1
join tabel2 t2
on t2.key = t1.key
where t1.id = :id
and t1.date >= :startdate
and t1.date < :enddate
t2c as
select t3.*
from tabel3 t3
join tabel4 t4
on t4.key = t3.key
where t3.id = :id
and t3.date >= :startdate
and t3.date < :enddate
select ct.*
from
select *
from t1c
union all
select *
from t2c
order by ct.dateSo in words, I have an 'union all' construction reading from different tables with matching columns 'id' and 'date'.
How can I improve this? Can it be improved? If you do not know the answer, but maybe a suggestion, I will be happy aswell!!!
Thanks in advance!
Kind regards,
Metroickha>
So in words, I have an 'union all' construction reading from different tables with matching columns 'id' and 'date'.
How can I improve this? Can it be improved? If you do not know the answer, but maybe a suggestion, I will be happy aswell!!!
>
If you want to improve on what Oracle is doing you first need to know 'what Oracle is doing'.
Post the execution plans for the query that show what Oracle is doing.
Also post the DDL for the tables and indexes and the record counts for the tables and ID/DATE predicates. -
Performance Issue in UNION ALL....
Hi,
I have a performance issue in a query using select.. union all .
There are few select statements which are joined by union all.
I need to re-frame the query such that the records from each individual select
statement can be appended into a collection variable.
I need to avoid the use of UNION ALL.
Can anyone please help me out in this issue?
Please find the select statement below.
SELECT *
FROM v_gfd1_asset_allocation
WHERE NVL(value,0) <> 0
AND scheme = p_scheme_id
UNION ALL
SELECT *
FROM v_gfd1_capitalization_alloc
WHERE NVL(value,0) <> 0
AND scheme = p_scheme_id
UNION ALL
SELECT *
FROM v_gfd1_currency_allocation
WHERE NVL(value,0) <> 0
AND scheme = p_scheme_id
Thanks,
SanthoshThe UNION ALL takes the result set of one query, appends the result set of the next, and so forth. No sorting or filtering takes place. So, as each query has to be executed to completion, I do not think there is another way - that is what union all is for.
You would probably do well to examine each query's access path. Would an index on NVL( value, 0 ), scheme help, for instance? Understanding the data distribution in the table and how it affects the execution plan would help you improve query performance. -
Hello
i have 2 schemas S1 and S2 containing complex relational tables. The tables in each schema are related to each other via foreign key relationships. i made views for feature tables in both the schemas , querying column values from their related tables.
the data structure in both the schemas are exactly the same. Dut to management reasons we have to split them in 2 schemas. S1 contain data for region A and S2 contains data from region B. Now the client wants to see a combined data from region A & B.
we are planning to create another schema S3 and make views combining views from S1 and S2 in both schemas (V1 in S1 + V1 in S2) using UNION ALL.
Does UNION ALL will make use of the indexes we already built for parent tables in S1 and S2? Will there be a performance degradation using this approach? What can be the best approach? Our client needs to see real time data....
regards
samSince union does an extra sort it has a performance difference compared to union all.
SQL> select user from dual union select user from dual ;
USER
HR
SQL> select user from dual union all select user from dual ;
USER
HR
HRİf there is up to date and appropriate object statistics Oracle's Cost Based Optimizer will choose best access path, join method and join order depending on your query. Only exceptions are hints, outlines and sql profiles since they stabilize the execution plan.
For further commenting please post your oracle version, query's test results and its statistics taken from sql*plus timing and autotrace options - http://www.bhatipoglu.com/entry/17/oracle-performance-analysis-tracing-and-performance-evaluation -
UNION ALL and UNION performance issue
Hi All,
I am trying to figure out the data for which only receive transaction has been done and further processing is pending. These transactions include all PO, RMA , ISO etc...
I have to use UNION ALL in this case as for RMA and ISO, details which i want are not able to gather in a single query.
But query is taking a lot of time ...may be around 30..mins in UNION ALL while 6 to 7 mins in UNION.
To get all records I must have to use UNION ALL...
So kindly suggest the solution for this problem
Thanks
Sachin
Query is given below...
SELECT /* + FIRST_ROWS */ DECODE(rsl.SOURCE_DOCUMENT_CODE,'REQ',(SELECT org1.ORGANIZATION_NAME
FROM org_organization_definitions org1
WHERE org1.ORGANIZATION_ID =
rsl.FROM_ORGANIZATION_ID)) Vendor_Name
,rsh.RECEIPT_NUM Receipt_Number
,TO_CHAR(rt3.TRANSACTION_DATE,'Mon-DD-YYYY HH:MM:SS') Receipt_Date_and_Time
,msi.SEGMENT1 Part_Number
,msi.DESCRIPTION Part_Name
,rt3.QUANTITY Quantity
,rt3.UNIT_OF_MEASURE UOM
,NULL ASL_Status
--for ISO no asl flag ASL Flag
,TO_CHAR(TRUNC((((86400*(SYSDATE-rt3.TRANSACTION_DATE))/60)/60)/24))|| ' Days ' || TO_CHAR(TRUNC(((86400*(SYSDATE-rt3.TRANSACTION_DATE))/60)/60)-24*(TRUNC((((86400*(SYSDATE-rt3.TRANSACTION_DATE))/60)/60)/24)))|| ' Hours' Days_and_hours_passed
,DECODE(
NVL(msi.max_minmax_quantity,0) ,
0 , 0 ,
(NVL(msi.max_minmax_quantity,0) -
NVL(inmohqd.onhand,0))
* 100
/ NVL(msi.max_minmax_quantity,0)
) gap_percent
FROM rcv_transactions rt3
,rcv_shipment_headers rsh
,rcv_shipment_lines rsl
,mtl_system_items msi
,org_organization_definitions org
--,MTL_ONHAND_QUANTITIES_DETAIL moqhd
,(SELECT NVL(SUM(primary_transaction_quantity),0) onhand,INVENTORY_ITEM_ID item_id,ORGANIZATION_ID organization_id
FROM mtl_onhand_quantities_detail
WHERE SUBINVENTORY_CODE NOT IN ('Wip_SF','Wip_Int','Reject','Scrap','FG Trading','FG')
GROUP BY INVENTORY_ITEM_ID, ORGANIZATION_ID) inmohqd
WHERE inmohqd.item_id(+) = msi.INVENTORY_ITEM_ID
AND inmohqd.organization_id(+) = msi.ORGANIZATION_ID
--AND inmoqhd.SUBINVENTORY_CODE NOT IN ('Wip_SF','Wip_Int','Reject','Scrap','FG Trading','FG')
AND msi.INVENTORY_ITEM_ID = rsl.ITEM_ID
AND rsh.SHIPMENT_HEADER_ID = rsl.SHIPMENT_HEADER_ID
AND org.ORGANIZATION_ID = rt3.ORGANIZATION_ID
AND msi.ORGANIZATION_ID = rt3.ORGANIZATION_ID
AND rsh.SHIPMENT_HEADER_ID = rt3.SHIPMENT_HEADER_ID
AND rsl.SHIPMENT_HEADER_ID = rt3.SHIPMENT_HEADER_ID
AND rsl.SHIPMENT_LINE_ID = rt3.SHIPMENT_LINE_ID
AND rt3.PO_HEADER_ID IS NULL
AND TRUNC(rt3.TRANSACTION_DATE) <= TRUNC(p_tilldate)
AND rsl.TO_ORGANIZATION_ID = p_organization_id
AND rsh.ORGANIZATION_ID = p_organization_id
AND CONCAT(TRIM(rt3.SHIPMENT_HEADER_ID),TRIM(rt3.SHIPMENT_LINE_ID)) IN
SELECT CONCAT(TRIM(rt1.SHIPMENT_HEADER_ID),TRIM(rt1.SHIPMENT_LINE_ID))
FROM rcv_transactions rt1
WHERE NOT EXISTS(
SELECT 1
FROM rcv_transactions rt2
WHERE rt2.TRANSACTION_TYPE <> 'RECEIVE'
AND rt1.SHIPMENT_HEADER_ID = rt2.SHIPMENT_HEADER_ID
AND rt1.SHIPMENT_LINE_ID = rt2.SHIPMENT_LINE_ID
AND rt2.ORGANIZATION_ID = p_organization_id
UNION
SELECT /* + FIRST_ROWS */ pv.VENDOR_NAME Vendor_Name
,rsh.RECEIPT_NUM Receipt_Number
,TO_CHAR(rt.TRANSACTION_DATE,'Mon-DD-YYYY HH:MM:SS') Receipt_Date_and_Time
,msi.SEGMENT1 Part_Number
,msi.DESCRIPTION Part_Name
,rt.QUANTITY Quantity
,rt.UNIT_OF_MEASURE UOM
--start 001
,NVL((SELECT DISTINCT DECODE (ASL_STATUS_ID,1,'New',2,'Approved','To be checked')
FROM po_approved_supplier_list pasl
WHERE pasl.item_id=rsl.ITEM_ID
AND pasl.VENDOR_ID(+) = pv.VENDOR_ID
AND pasl.VENDOR_SITE_ID(+) = pvs.VENDOR_SITE_ID),'No_data') ASL_Status
--end 001
,TO_CHAR(TRUNC((((86400*(SYSDATE-rt.TRANSACTION_DATE))/60)/60)/24))|| ' Days ' || TO_CHAR(TRUNC(((86400*(SYSDATE-rt.TRANSACTION_DATE))/60)/60)-24*(TRUNC((((86400*(SYSDATE-rt.TRANSACTION_DATE))/60)/60)/24)))|| ' Hours' Days_and_hours_passed ,DECODE(
NVL(msi.max_minmax_quantity,0) ,
0 , 0 ,
(NVL(msi.max_minmax_quantity,0) -
NVL(inmohqd.onhand,0))
* 100
/ NVL(msi.max_minmax_quantity,0)
) gap_percent
FROM rcv_transactions rt
,po_vendors pv
,po_vendor_sites_all pvs
,rcv_shipment_headers rsh
,rcv_shipment_lines rsl
,mtl_system_items msi
,org_organization_definitions org
--,mtl_onhand_quantities_detail moqhd
,(SELECT NVL(SUM(primary_transaction_quantity),0) onhand,INVENTORY_ITEM_ID item_id,ORGANIZATION_ID organization_id
FROM mtl_onhand_quantities_detail
WHERE SUBINVENTORY_CODE NOT IN ('Wip_SF','Wip_Int','Reject','Scrap','FG Trading','FG')
GROUP BY INVENTORY_ITEM_ID, ORGANIZATION_ID) inmohqd
WHERE inmohqd.item_id(+) = msi.INVENTORY_ITEM_ID
AND inmohqd.ORGANIZATION_ID(+) = msi.ORGANIZATION_ID
--AND inmoqhd.SUBINVENTORY_CODE NOT IN ('Wip_SF','Wip_Int','Reject','Scrap','FG Trading','FG')
AND msi.INVENTORY_ITEM_ID = rsl.ITEM_ID
AND rsh.SHIPMENT_HEADER_ID = rsl.SHIPMENT_HEADER_ID
AND pv.VENDOR_ID = pvs.VENDOR_ID
AND org.ORGANIZATION_ID = rt.ORGANIZATION_ID
AND msi.ORGANIZATION_ID = rt.ORGANIZATION_ID
AND pvs.VENDOR_SITE_ID = rt.VENDOR_SITE_ID
AND pv.VENDOR_ID = rt.VENDOR_ID
AND rsh.SHIPMENT_HEADER_ID = rt.SHIPMENT_HEADER_ID
AND rsl.SHIPMENT_HEADER_ID = rt.SHIPMENT_HEADER_ID
AND rsl.SHIPMENT_LINE_ID = rt.SHIPMENT_LINE_ID
AND TRUNC(rt.TRANSACTION_DATE) <= TRUNC(p_tilldate)
AND rsl.TO_ORGANIZATION_ID = p_organization_id
AND CONCAT(TRIM(rt.SHIPMENT_HEADER_ID),TRIM(rt.SHIPMENT_LINE_ID)) IN
SELECT CONCAT(TRIM(rt1.SHIPMENT_HEADER_ID),TRIM(rt1.SHIPMENT_LINE_ID))
FROM RCV_TRANSACTIONS rt1
WHERE rt1.TRANSACTION_TYPE = 'RECEIVE'
AND rt1.DESTINATION_TYPE_CODE = 'RECEIVING'
AND rt1.PO_HEADER_ID IS NOT NULL
AND NOT EXISTS(
SELECT 1
FROM RCV_TRANSACTIONS rt2
WHERE rt2.SHIPMENT_HEADER_ID = rt1.SHIPMENT_HEADER_ID
AND rt2.SHIPMENT_LINE_ID = rt1.SHIPMENT_LINE_ID
AND rt2.TRANSACTION_TYPE <> 'RECEIVE'
)In this case, for selected columns, all data is same for one of the RMA with more than one line. So UNION will skip one of the records. However, shipment line id are different for both records, so by selecting it in select list is solving the problem and so no need to use UNION ALL. But, anyhow UNION ALL is better than UNION in performance as it does not require to sort. Then why I am facing this problem...
Kindly suggest
Regards,
Sachin -
Update..Union All Vs multiple individual Updates -- performance?
Hi All,
I have a procedure which fetches data from a remote database. It performs multiple updates on a single table. Will it be any better in terms of performance if I have all the updates combined into one update using a UNION ALL.
Please advice in terms of performance or any other options of improving performance..
INSERT INTO BALAN (
deal_id, projection_period, deposit, spread
SELECT DEALID, 1, nvl(to_char(sum(avg_deposit) / count(avg_deposit), '99999999999.99'), 0) deposit,
nvl(to_char(sum(avg_float)/ count(avg_float), '99999999999.99'), 0) spread
FROM BILLING.CAP_ANST1@LINK_BILLN01D a
WHERE account = AACCOUNT
AND group_ind = GROUPID
AND stamp = v_stamp_date
UNION ALL
SELECT DEALID, 3, nvl(to_char(sum(avg_deposit) / count(avg_deposit), '99999999999.99'), 0) deposit,
nvl(to_char(sum(avg_float)/ count(avg_float), '99999999999.99'), 0) spread
FROM BILLING.CAP_ANST1@LINK_BILLN01D
WHERE account = AACCOUNT
AND group_ind = GROUPID
AND stamp >= v_stamp_date_minus_2
UNION ALL
SELECT DEALID, 6, nvl(to_char(sum(avg_deposit) / count(avg_deposit), '99999999999.99'), 0) deposit,
nvl(to_char(sum(avg_float)/ count(avg_float), '99999999999.99'), 0) spread
FROM BILLING.CAP_ANST1@LINK_BILLN01D
WHERE account = AACCOUNT
AND group_ind = GROUPID
AND stamp >= v_stamp_date_minus_5
UNION ALL
SELECT DEALID, 12, nvl(to_char(sum(avg_deposit) / count(avg_deposit), '99999999999.99'), 0) deposit,
nvl(TO_CHAR(SUM(avg_float)/ COUNT(avg_float), '99999999999.99'), 0) spread
FROM billing.CAP_ANST1@LINK_BILLN01D a
WHERE account = AACCOUNT
AND group_ind = GROUPID
AND stamp >= v_stamp_date_minus_11;
---------------- OR---------------------------------
INSERT INTO IDM.balan
(DEAL_id, PROJECTION_PERIOD, deposit, spread)
SELECT DEALID, 1, nvl(SUM (avg_deposit) / COUNT (avg_deposit), 0) deposit,
nvl(SUM (avg_float) / COUNT (avg_float),0) spread
FROM BILLING.CAP_ANST1@LINK_BILLN01D a
WHERE aaccount = a.ACCOUNT
AND a.group_ind = groupid
AND stamp = (SELECT to_date(value, 'MM/DD/YYYY')
FROM APP_ENV
WHERE name = 'CUR_STAMP');
INSERT INTO IDM.balan
(DEAL_id, projection_period, deposit, spread)
SELECT DEALID, 3, nvl( TO_CHAR(SUM (avg_deposit) / COUNT (avg_deposit), '99999999999.99'),0) deposit,
nvl( TO_CHAR(SUM (avg_float)/ COUNT (avg_float), '99999999999.99'),0) spread
FROM BILLING.CAP_ANST1@LINK_BILLN01D a
WHERE aaccount = a.ACCOUNT
AND a.group_ind = groupid
AND stamp >= (SELECT ADD_MONTHS (to_date(value, 'MM/DD/YYYY'), -2)
FROM APP_ENV
WHERE name = 'CUR_STAMP');
INSERT INTO IDM.balan
(DEAL_ID, projection_period, deposit, spread)
SELECT DEALID, 6,
nvl( TO_CHAR(SUM (avg_deposit) / COUNT (avg_deposit), '99999999999.99'),0) deposit,
nvl( TO_CHAR(SUM (avg_float)/ COUNT (avg_float), '99999999999.99'),0) spread
FROM billing.CAP_ANST1@LINK_BILLN01D a
WHERE a.ACCOUNT = aaccount
AND a.group_ind = groupid
AND stamp >= (SELECT ADD_MONTHS (to_date(value, 'MM/DD/YYYY'), -5)
FROM APP_ENV
WHERE name = 'CUR_STAMP');
INSERT INTO IDM.balan
(DEAL_ID, projection_period, deposit, spread)
SELECT DEALID, 12, nvl( TO_CHAR(SUM (avg_deposit) / COUNT (avg_deposit), '99999999999.99'),0) deposit,
nvl( TO_CHAR(SUM (avg_float)/ COUNT (avg_float), '99999999999.99'),0) spread
FROM billing.CAP_ANST1@LINK_BILLN01D a
WHERE aaccount = a.ACCOUNT
AND a.group_ind = groupid
AND stamp >= (SELECT ADD_MONTHS (to_date(value, 'MM/DD/YYYY'), -11)
FROM APP_ENV
WHERE name = 'CUR_STAMP');
Thanks in advance!
Appreciate your help!
Thanks
BobHi Bob,
Maybe I misunderstood, but couldn't you just:
insert into balan(deal_id
,projection_period
,deposit
,spread)
select dealid
,case
when stamp = v_stamp_date then 1
when stamp >= v_stamp_date_minus_2 then 3
when stamp >= v_stamp_date_minus_5 then 6
when stamp >= v_stamp_date_minus_11 then 12
end
projection_period
,nvl(to_char(sum(avg_deposit) / count(avg_deposit), '99999999999.99')
,0)
deposit
,nvl(to_char(sum(avg_float) / count(avg_float), '99999999999.99'), 0)
spread
from billing.cap_anst1@link_billn01d a
where account = aaccount
and group_ind = groupid
and stamp between v_stamp_date_minus_11 and v_stamp;Regards
Peter -
Performing UNION/UNION ALL in views
I have two databases (DB1 AND DB2) in same server. I have to create two views (V1 AND V2) respectively in DB1 and DB2. And I have to make an union/union all operation in these views (V1 and V2). Is this possible.???.
Or can I create a single view instead of two views in such a way that the single view contains an union/union all operation between the data taken from two DBs (DB1 and DB2).?
Any ideas or suggestions on these questions. Your help is well appreciated.
Thanks in advance.Or can I create a single view instead of two views in such a way that the single view contains an union/union all operation between the data taken from two DBs (DB1 and DB2).?
Make sure you do very thorough testing if you plan to use UNION since that requires a sort operation. And that means that the data from one query has to be sent to the other server before it can be sorted.
So a query issued on DB1 might sent the DB1 data to DB2 and then send the sorted result back to DB1. But the same query issued on DB2 would already have the DB2 data so after the DB1 data is sent to DB2 and sorted the query is done.
That is, in one case the entire result set may be sent over the network from one server to the other instead of just one table. -
Very slow performance with UNION and UNION ALL
I am returning three datasets as one with a UNION ALL between each of three SQL statements. The first one returns either 3 or 4 rows, the second one returns about 10 rows and the third one returns one row. The SQL statements are a little complex but they're all working on very small datasets ( < 1000 rows ) so when I run each of the three SQL statements by themselves, they run to completion and return data in less than 1 second.
However.
When I join the three statements together with a UNION ALL statement between them, the statement runs until I manually abort it. I've run it for up to two hours and it shows no signs of either 1) erroring or 2) running to completion. And, I've tried replacing the UNION ALL with UNION and I get the identical results - nothing returned, no error message.
I've checked the obvious -- same column names, same number of columns, etc. I think that if there were something obvious like that, I would get an error message as soon as I attempted to execute the statements.
I will be happy to post the code if you'd like, but I'm wondering if anyone has experienced these or similar symptoms when joining together SQL statements with UNIONs and if so what you did to get around it.
Thanks in advance,
CarlOr I guess you could use subquery factoring (WITH clause) and MATERIALIZE hints to create the temp tables inline, e.g.
WITH table1 AS
( SELECT /*+ materialize */ somecolumns FROM sometable )
, table2 AS
( SELECT /*+ materialize */ somecolumns FROM othertable )
, table2 AS
( SELECT /*+ materialize */ somecolumns FROM anothertable )
SELECT somecolumns FROM table1
UNION ALL
SELECT somecolumns FROM table2
UNION ALL
SELECT somecolumns FROM table3It might also be valuable to investigate exactly what is happening and why, as this might suggest a way to address the root cause. -
How can I join/Full outer join two different columns instead of union all?
Hi,
I have a scenario as the following:
I am performing set operations using obiee 11g where I want to take values from two different criteria. Howwver, I dont want union to take place, instead i want join to take place to see all the columns in the output.
For that, I tried changing the sql in advanced tab and tried to put full outer join instead of union all but its not allowing me to change.
How can I achieve it? please help.
Thanks.Hi,
My problem is that I am unable to modify the sql in advanced tab. Probably due to some security reason,it's restricting me to change.
Can you suggest me a way to change it?
Thanks.. -
Report which concatenates 13 views with union all running slowly
Oracle 8.1.7 windows 2000 server
I am trying to improve the performance of a report which is comprised of 13 views.
When I run each of the views individually, the total run time for the views is less than 5 minutes. When I run the report, it takes 28 minutes.
Can anyone suggest why the extra time is being taken?
To reiterate:
select a,b from c; (executes in 10 seconds)
select d, e from f (executes in 3 seconds)
select x, y from z (executes in 1 minute)
total runs time = 5 minutes
However,
select a,b from c
union all
select d, e from f
union all
select x, y from z (executes in 28 minutes)
The execution plans do not change between the report and the indiividual views. Views are being concatenated with union all so no sorting is taking place
Many thanks,
Jason Parker.
Edited by: jclparker on Feb 18, 2009 4:26 AM
Edited by: jclparker on Feb 18, 2009 4:30 AMCould you post the execution plan? Please use formatting tags to save the white space while posting the plan.
-
Need sql query to remove duplicates using UNION ALL clause
Hi,
I have a sql query which has UNION clause.But the UNION clause is causing some performance issues.
To overcome that I have used UNION ALL to improve performance but its returning duplicates.
Kindly anyone send a sample SQL query where my primary objective is used to use UNION ALL clause and to consider unique rows (elimating duplicate
ones)
Any help will be needful for me
Thanks and Regardswhy not UNION? :(
another way also use MINUS
SQL>
SQL> with t as
2 (
3 select 1 if from dual union all
4 select 2 if from dual union all
5 select 1 if from dual union all
6 select 3 if from dual union all
7 select 3 if from dual
8 )
9 ,t2 as
10 (
11 select 1 if from dual union all
12 select 2 if from dual union all
13 select 3 if from dual union all
14 select 4 if from dual union all
15 select 5 if from dual
16 )
17 (select if from t
18 union all
19 select if from t2)
20 /
IF
1
2
1
3
3
1
2
3
4
5
10 rows selected
SQL> so
SQL>
SQL> with t as
2 (
3 select 1 if from dual union all
4 select 2 if from dual union all
5 select 1 if from dual union all
6 select 3 if from dual union all
7 select 3 if from dual
8 )
9 ,t2 as
10 (
11 select 1 if from dual union all
12 select 2 if from dual union all
13 select 3 if from dual union all
14 select 4 if from dual union all
15 select 5 if from dual
16 )
17 (select if from t
18 union all
19 select if from t2)
20 minus
21 select -99 from dual
22 /
IF
1
2
3
4
5
SQL> -
SQL Union all VS SSIS Union All
I have 4 source SQL queries which is pointing different db's in the same server. All 4 queries results need to stored in single SQLServer table of different server. which one is best approach mentioned in below ?
1) Writing SP using union all option with all four source queries and use this sp in single data flow task[single source and destination tasks].
2) 4 source tasks to execute the 4 sql queries and then use Union all transformation to club the result and then one destination.
3) 4 different data flow task. Each data flow task point the different souurce queries and the same destination.
Please suggest in all aspects.Prabu,
You are right in your observations from your previous post. Based on my experience, number 2 would be the best option.
First of all, it might be best to parallelize the extraction because the time taken to read from all sources equals the time to read from the source with the highest volume of data, whereas reading from sources serially would result in a sum of the time
taken to read from each of them individually.
Secondly, BULK LOADS with table locks are very fast, but you can only have one process writting to the same destination at the same time, because a table level lock is placed on the object. This will probably offset the performance gains of a parallel insert
into the same destination, but it is worth testing.
IF, and only IF, your process uses a lot of CPU time for some reason (transformations are performed), then there is a chance option number 3 will outperform option number 2 because CPU workload would generally be split between multiple processors, but I
dont think this is your case.
Just because there are clouds in the sky it doesn't mean it isn't blue. Some people would disagree. -
Hi,
From Oracle 9i performance guide:
"Concatenation is useful for statements with different conditions combined with an OR clause"
"If a query contains a WHERE clause with multiple conditions combined with OR operators, then the optimizer transforms it into an equivalent compound query that uses the UNION ALL set operator, if this makes the query execute more efficiently"
Can anyone tell me when concatenation is better than union and vice versa?skde wrote:
Hi,
From Oracle 9i performance guide:
"Concatenation is useful for statements with different conditions combined with an OR clause"
"If a query contains a WHERE clause with multiple conditions combined with OR operators, then the optimizer transforms it into an equivalent compound query that uses the UNION ALL set operator, if this makes the query execute more efficiently"
Can anyone tell me when concatenation is better than union and vice versa?Concatenation is not exactly the same as UNION ALL (the ALL is important, by the way), as it has to eliminate repetitions of data. You may rewrite an 'OR' query as a UNION ALL, concatenation is simply Oracle's strategy for doing this. There's an example on my blog which might help you understand the issues: http://jonathanlewis.wordpress.com/2007/02/26/subquery-with-or/
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
There is a +"Preview"+ tab at the top of the text entry panel. Use this to check what your message will look like before you post the message. If it looks a complete mess you're unlikely to get a response. (Click on the +"Plain text"+ tab if you want to edit the text to tidy it up.)
+"I believe in evidence. I believe in observation, measurement, and reasoning, confirmed by independent observers. I'll believe anything, no matter how wild and ridiculous, if there is evidence for it. The wilder and more ridiculous something is, however, the firmer and more solid the evidence will have to be."+
Isaac Asimov -
Excluding slow table access in a UNION ALL view
Hi,
I have a view which unions three tables together.
One component of the view requires a table scan, as 90% of the records are required.
This view is then used in another outer select where these records are actually not required.
So I tagged each component with a code and excluded that in the outer select. However it still appears to access the table.
Is there any way I can exclude a component of the UNION ALL or do I need to explicitly split them?
example:
SELECT * FROM (
SELECT 'A' Q, JANUARY F FROM ENORMOUS.TABLE WHERE KEY = 'Non Selective Key' UNION ALL
SELECT 'B' Q, 1 F FROM DUAL UNION ALL
SELECT 'C' Q, 1 F FROM DUAL
) A
WHERE Q = 'B'
When I run the query plan without the WHERE it performs the table scan
When I include the WHERE it still performs the table scan but with a FILTER NULL IS NOT NULL afterwards.
So it appears that it is doing the table scan regardless and then throwing the records away - is that correct?
Any thoughts appreciated. I would prefer not the split this view out if possible as it is used everywhere.In summary my question is: is The Oracle query planner smart enough to exclude a component from a load of stacked UNION ALL queries?
given this query:
CREATE VIEW TEST AS
SELECT Q, F
FROM
SELECT 'A' Q, JANUARY F FROM ENORMOUS.TABLE WHERE KEY = 'Non Selective Key' UNION ALL
SELECT 'B' Q, 1 F FROM DUAL UNION ALL
SELECT 'C' Q, 1 F FROM DUAL
) A;
-- 1. This one selects from all tables, including a table scan on the enormous table
SELECT * FROM TEST;
-- 2. This one selects from all tables, including a table scan on the enormous table
-- However the query plan has a FILTER after the table scan. Does it exclude this work?
SELECT * FROM TEST WHERE Q = 'B';When I run the query plan without the WHERE it performs the table scan
When I include the WHERE the query plan indicates it's doing the table scan but with a FILTER NULL IS NOT NULL afterwards.
So it appears that it is doing the table scan regardless and then throwing the records away - is that correct?
Any thoughts appreciated. I would prefer not the split this view out if possible as it is used everywhere.
Oracle version:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
CORE 11.2.0.2.0 Production
TNS for 64-bit Windows: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
Plan output for 1:
SELECT STATEMENT Optimizer Mode=ALL_ROWS (Cost=7166 Card=2 M Bytes=41 M)
1 VIEW (Cost=7166 Card=2 M Bytes=41 M)
2 1 UNION-ALL
3 2 TABLE ACCESS FULL PLANNING.BF_GEN_STATS_TRAN (Cost=7162 Card=2 M Bytes=72 M)
4 2 FAST DUAL (Cost=2 Card=1)
5 2 FAST DUAL (Cost=2 Card=1)
Plan output for 2:
SELECT STATEMENT Optimizer Mode=ALL_ROWS (Cost=2 Card=3 Bytes=48)
1 VIEW (Cost=2 Card=3 Bytes=48)
2 1 UNION-ALL
3 2 FILTER
4 3 TABLE ACCESS FULL PLANNING.BF_GEN_STATS_TRAN (Cost=7162 Card=2 M Bytes=72 M)
5 2 FAST DUAL (Cost=2 Card=1)
6 2 FILTER
7 6 FAST DUAL (Cost=2 Card=1) -
UNION Subquery Factoring performance
Hi,
We have 2 'with' clauses, each ones works fine, and the performance is good.
If I try to join the 2 'with' queries together, either using union all (which is what we really want), or an inner join.
Its a 9.2.0.5 database, and all the statistics are up to date!
Thanks for any help.
HannahThe query is:
with
allele_number_one AS
(SELECT hbo1.hla_test_id,
hbo1.hla_test_name,
hbo1.hla_broad_order,
'1' test_specific_order,
stm1.sample_id,
hbo1.hla_left,
hbo1.hla_right,
hbo1.hla_class,
hbo1.hla_locus,
stm1.test_class,
stm1.homozygous,
'allele one'
FROM sample_test_map stm1,
hla_broad_order hbo1
-- WHERE stm1.sample_id = 293602
where hbo1.hla_test_id = stm1.hla_test_id
AND stm1.test_class = 1
AND hbo1.hla_class = 4
AND NOT EXISTS
(SELECT 'f'
FROM sample_test_map stm2,
hla_broad_order hbo2
WHERE stm2.sample_id = stm1.sample_id
AND hbo2.hla_test_id = stm2.hla_test_id
AND hbo2.hla_class = hbo1.hla_class
AND hbo2.hla_test_id <> hbo1.hla_test_id
AND hbo2.hla_locus = hbo1.hla_locus
AND hbo2.hla_broad_order < hbo1.hla_broad_order
AND stm2.test_class = 1)
-- if they are children of homozygous parents, only report one
AND NOT EXISTS
(SELECT 'f'
FROM sample_test_map stm3,
hla_broad_order hbo3
WHERE stm3.sample_id = stm1.sample_id
AND hbo3.hla_test_id = stm3.hla_test_id
AND hbo3.hla_class = hbo1.hla_class
AND hbo3.hla_test_id < hbo1.hla_test_id
AND hbo3.hla_locus = hbo1.hla_locus
AND stm3.test_class = stm1.test_class
AND hbo3.hla_broad_order = hbo1.hla_broad_order)),
allele_number_two AS
(SELECT hbo1.hla_test_id,
hbo1.hla_test_name,
hbo1.hla_broad_order,
'2' test_specific_order,
stm1.sample_id,
hbo1.hla_left,
hbo1.hla_right,
hbo1.hla_class,
hbo1.hla_locus,
stm1.test_class,
stm1.homozygous,
'allele two'
FROM sample_test_map stm1,
hla_broad_order hbo1
-- WHERE stm1.sample_id = 293602
WHERE hbo1.hla_test_id = stm1.hla_test_id
AND stm1.test_class = 1
AND hbo1.hla_class = 4
-- and hasn't been determined as a 'left' one
AND NOT EXISTS
(SELECT 'f'
FROM allele_number_one an1
WHERE an1.hla_test_id = stm1.hla_test_id
AND an1.sample_id = stm1.sample_id
AND an1.hla_broad_order = hbo1.hla_broad_order
AND an1.test_class = stm1.test_class
AND an1.homozygous = stm1.homozygous)
select * from allele_number_one
where sample_id = 293602
union all
select * from allele_number_two
where sample_id = 293602And the explain:
SELECT STATEMENT CHOOSE 12 4 467
TEMP TABLE TRANSFORMATION
UNION-ALL
VIEW 2 1 164
TABLE ACCESS(FULL) SYS.SYS_TEMP_0FD9D6778_3099A94B ANALYZED 2 1 36
HASH JOIN(ANTI) 10 3 303
HASH JOIN 7 3 108
INDEX(RANGE SCAN) NTXD_TEST.SAMPLE_TEST_MAP_PK ANALYZED 3 3 33 TABLE ACCESS(FULL) NTXD_TEST.HLA_BROAD_ORDER ANALYZED 3 832 20800
VIEW 2 1 65
TABLE ACCESS(FULL) SYS.SYS_TEMP_0FD9D6778_3099A94B ANALYZED 2 1 36
Maybe you are looking for
-
Report to delet production orders
Hi guys, help needed! Does anybody of you know a report to delete production orders completely? Such on as the RMMMDE00, which deletes all material of the current client. Hoping for an answer, Barbara
-
Hi Friends, I am developing a small web application using jsp, spring, and tomcat 5.0. In my application i need to to up-load one file at add page. it also contains some form fields. In servlet i am using the multipartformdatarequest. its work fine w
-
Quickest way to transfer iMovie 09 Files from Macbook Pro to iMac
Hello: We recently got an iMac and I would like to transfer my iMovie 09 files from my macbook pro to the iMac which also has iMovie 09. Can you suggest the quickest and easiest way to do this? I am assumign this can be done since they are both iMovi
-
ISpeak Protocol timer callback error
I'm having trouble with an XI production system that has been reporting an error since 4 days every couple of minutes in the DefaultTrace of one J2EE server node. The error is: #1.5 #[...]#com.sap.aii.af.protocol.ispeak.services.timer.impl.ISPTimerEv
-
Recommended Structure for Large Files
I am working at re-familiarizing myself with Oracle development and management so please forgive my ignorance on some of these topics or questions. I will be working with a client who is planning a large-scale database for what is called "Flow Cytome