Distinct clause and query performance
Friends,
I have a query which returns results in 40 seconds without distinct clause and when I add distinct clause it takes over 2 hours.
I have verified following -
1. indexes/table statistics are up to date.
2. columns that are used in where clause but are not indexed have upto date column statistics
Any idea what could be the reason, explain plan shows that distinct clause has a very expensive cost.
Thanks
Query and explain plan is below
SELECT
DISTINCT -- with distinct 2hrs + and without 40 seconds
quote_by_dst.qte_hdr_stat_cd, quote_by_dst.qte_ln_cond_cd,
product.prod_nm, product.prod_id,
cs_ship_by_dst.bto_ds_cac_ownr_ud,
quote_by_dst.qte_csup_csup_am, cs_ship_by_dst.bto_ds_cac_nm,
product.spl_sht_nm,
product.prod_blg_un_fac_um
|| ' '
|| product.prod_blg_um
|| ' '
|| product.prod_stck_um,
product.prod_blg_um, quote_by_dst.qte_ln_brk_1_blg_uom_am,
quote_by_dst.qte_csup_avg_cst_am,
quote_by_dst.qte_csup_rev_gm_pct_am,
quote_by_dst.qte_csup_avg_cst_am, cs_ship_by_dst.bto_id,
cs_ship_by_dst.bto_ds_cac_cd,
cs_ship_by_dst.bto_ds_cac_cd
|| product.prod_id
|| cs_ship_by_dst.bto_id
FROM infowhse.quote_by_dst4 quote_by_dst,
infowhse.product,
infowhse.cs_ship_by_dst4 cs_ship_by_dst,
infowhse.department
WHERE (quote_by_dst.dpt_cd = department.dpt_cd)
AND (quote_by_dst.cus_dpt_id = cs_ship_by_dst.cus_dpt_id)
AND (product.prod_id = quote_by_dst.prod_id)
AND ( ( quote_by_dst.qte_ln_cond_cd = 'E'
OR quote_by_dst.qte_ln_cond_cd = 'C'
AND quote_by_dst.qte_hdr_stat_cd = 'A'
AND ((cs_ship_by_dst.bto_cust_type_cd) = '01')
AND cs_ship_by_dst.bto_ds_cac_ownr_ud = 'EHOC'
AND department.dpt_cd > '0.00'
)Explain plan
Plan
SELECT STATEMENT CHOOSECost: 911,832,256 Bytes: 433,941,639,459 Cardinality: 2,729,192,701
15 SORT UNIQUE Cost: 911,832,256 Bytes: 433,941,639,459 Cardinality: 2,729,192,701
14 NESTED LOOPS Cost: 68,705 Bytes: 433,941,639,459 Cardinality: 2,729,192,701
12 HASH JOIN Cost: 68,705 Bytes: 425,754,061,356 Cardinality: 2,729,192,701
1 INDEX FAST FULL SCAN NON-UNIQUE INFOWHSE.DST_SEC_DST_SEC_DST_CD_IX Cost: 25 Bytes: 922,700 Cardinality: 184,540
11 HASH JOIN Cost: 16,179 Bytes: 1,199,209,082 Cardinality: 7,941,782
2 INDEX FAST FULL SCAN NON-UNIQUE INFOWHSE.DST_SEC_DST_SEC_DST_CD_IX Cost: 25 Bytes: 922,700 Cardinality: 184,540
10 HASH JOIN Cost: 15,879 Bytes: 3,374,060 Cardinality: 23,110
8 HASH JOIN Cost: 15,200 Bytes: 2,981,190 Cardinality: 23,110
6 HASH JOIN Cost: 13,113 Bytes: 1,779,470 Cardinality: 23,110
3 TABLE ACCESS FULL INFOWHSE.CUSTOMER_SHIP Cost: 5,640 Bytes: 42,372 Cardinality: 1,177
5 PARTITION RANGE ALL Partition #: 11 Partitions accessed #1 - #12
4 TABLE ACCESS FULL INFOWHSE.QUOTE Cost: 7,328 Bytes: 38,826,590 Cardinality: 946,990 Partition #: 11 Partitions accessed #1 - #12
7 TABLE ACCESS FULL INFOWHSE.PRODUCT Cost: 1,542 Bytes: 9,246,640 Cardinality: 177,820
9 INDEX FAST FULL SCAN NON-UNIQUE INFOWHSE.CUST_SHIP_SLSDST_DTP_SICALL_IX Cost: 185 Bytes: 9,878,411 Cardinality: 581,083
13 INDEX UNIQUE SCAN UNIQUE INFOWHSE.DEPARTMENT_PK Bytes: 3 Cardinality: 1
This might be more useful.
Query is still running.
There is heavy wait time for scattered file read.
Results from
SELECT * FROM V$SESSION_WAIT WHERE SID = 48;
SID SEQ# EVENT P1TEXT P1 P1RAW P2TEXT P2 P2RAW P3TEXT P3 P3RAW WAIT_TIME SECONDS_IN_WAIT STATE
48 6865 db file scattered read file# 108 000000000000006C block# 1593370000000000026E69 blocks 32 0000000000000020 2 30 WAITED KNOWN TIME
SELECT * FROM V$SESSION_EVENT WHERE SID = 48;
SID EVENT TOTAL_WAITS TOTAL_TIMEOUTS TIME_WAITED AVERAGE_WAIT MAX_WAIT TIME_WAITED_MICRO
48 log file sync 1 0 0 0 0 563
48 db file sequential read 11 0 0 0 0 243
48 db file scattered read 6820 0 330 0 7 3296557
48 SQL*Net message to client 19 0 0 0 0 23
48 SQL*Net message from client 18 0 128 7 127 1281912 Sorry for long post.
Similar Messages
-
I found out issue distinct clause in query ,refcursor not returning rows
URGENT
hi the following procedure returns records but when i add distinct clause to (open v_refcursor for select )
i.e open v_refcursor for select distinct column1,column2 .......... in the procedure the procedure is not returning records. please help what is the issue here?
CREATE OR REPLACE procedure proc_shared_report3 (in_cust_id varchar2,in_user_array user_tab , in_acct_array acct_tab,in_report_pvlg_hier varchar2,in_fmt_pvlg_hier varchar2,v_refcursor OUT sys_refcursor)
is
BEGIN
if in_acct_array.count>0 and in_fmt_pvlg_hier is not null and in_cust_id is not null and in_user_array.count>0 and in_report_pvlg_hier is not null then
dbms_output.put_line('all are not null');
*open v_refcursor for select usr_id* from vw_get_user_profile where usr_id in (
SELECT USR_ID FROM VW_GET_ACCOUNTS WHERE USR_ID IN (SELECT distinct up1.usr_id FROM vw_get_user_privileges up1,vw_get_user_privileges up2 WHERE
up1.usr_id=up2.usr_id and
up1.product_hierarchy=in_report_pvlg_hier and up2.PRODUCT_HIERARCHY=in_fmt_pvlg_hier AND up1.usr_id in (select usr_id from vw_get_user_for_customer where
cust_id=in_cust_id)
and up1.usr_id Member Of in_user_array) AND acct_nb Member of in_acct_array);
/* if account list is null and rest all not null */
elsif (in_acct_array is null or in_acct_array IS EMPTY or in_acct_array.count = 0) and in_fmt_pvlg_hier is not null and in_cust_id is not null and in_user_array.count>0 and in_report_pvlg_hier is not null then
dbms_output.put_line('acc is null and rest are not null');
*open v_refcursor for select usr_id,usr_type_cd* from vw_get_user_profile where usr_id in(SELECT distinct up1.usr_id FROM vw_get_user_privileges up1,vw_get_user_privileges up2 WHERE
up1.usr_id=up2.usr_id and
up1.product_hierarchy=in_report_pvlg_hier and up2.PRODUCT_HIERARCHY=in_fmt_pvlg_hier AND up1.usr_id in (select usr_id from vw_get_user_for_customer where
cust_id=in_cust_id)
and up1.usr_id Member Of in_user_array);
/* if account list is null and format pvlg hierarchy is null */
elsif (in_acct_array is null or in_acct_array IS EMPTY or in_acct_array.count = 0) and in_fmt_pvlg_hier is null and in_cust_id is not null and in_user_array.count>0 and in_report_pvlg_hier is not null
then
dbms_output.put_line('acc is null and format is null null');
*open v_refcursor for select usr_id,tmzon_cd* from vw_get_user_profile where usr_id in (
(SELECT distinct usr_id FROM vw_get_user_privileges WHERE product_hierarchy=in_report_pvlg_hier AND usr_id in (select usr_id from vw_get_user_for_customer where
cust_id=in_cust_id)
and usr_id Member Of in_user_array)) ;
/* if account list is not null and format pvlg hierarchy is null */
else -- If i get only one privilege and all other inputs then
dbms_output.put_line('acc list is not null and format pvlg is null');
*open v_refcursor for select usr_id,prod_shrt_nm* from vw_get_user_profile where usr_id in (
SELECT USR_ID FROM VW_GET_ACCOUNTS WHERE USR_ID IN (SELECT distinct usr_id FROM vw_get_user_privileges WHERE
product_hierarchy=in_report_pvlg_hier AND usr_id in (select usr_id from vw_get_user_for_customer where
cust_id=in_cust_id) and usr_id Member Of in_user_array ) AND acct_nb Member of in_acct_array);
END IF;
END proc_shared_report3;
/Edited by: raj_fresher on Aug 12, 2009 8:50 AM
Edited by: raj_fresher on Aug 12, 2009 1:40 PMits like this ...... ?
You have to understand that without any testdata/ a reproducable testcase, all I can do is asking you to test some alternatives, I'm just guessing things from what I can make of your example, and hope you'll repost back any differences in a clear and concisive way.
What I didn't understand is why you're not just joining your tables/views instead of using inner queries.
Unfortunatly you're not providing any feedback on that, on the other hand you've narrowed down your problem to a specific part of your query? But you're not sharing why/what makes you decide to take that turn.
But: you're still using inner queries in your reply.
So, sorry, I cannot proceed/say anything useful unless you answer my previous post.
Because, there are still many other questions/doubts: I don't understand why you're not aliasing your columns properly, for example.
I don't know why you've suddenly switched to VW_GET_USER_cust_sa?
I don't know if you've tried the straight join instead of nesting queries?
And if you did, what's the problem?
Please read and understand this, so you'll get your answers by the speed of light the next time you've got a question or problem:
http://tkyte.blogspot.com/2005/06/how-to-ask-questions.html
And don't refrain from using that tag ;) -
Order of tables in the form clause influences query performance? (ora 9i)
Hi,
I have a SQL select with 5 tables in the 'from' clause. On Oracle 9i (and 8i too) if I user a certain table order (bigger tables first, smaller tables last) the query executes fast. If I change the order, then it executes in slow motion. I use cost based optimizing.
I thought it was just Oracle 8i where I had to take into account such things when writing a query. Does some one know the cause for this? I
Regards,
Tamas SzecsyAnd, even in 10GR2, if the optimizer_mode is changed to CHOOSE (the default is ALL_ROWS), the optimizer defaults to RULE based:
SQL> show parameter optimizer_mode
NAME TYPE VALUE
optimizer_mode string ALL_ROWS
SQL> alter session set optimizer_mode = choose ;
Session altered.
SQL> show parameter optimizer_mode
NAME TYPE VALUE
optimizer_mode string CHOOSE
SQL> analyze table emp delete statistics ;
Table analyzed.
SQL> analyze table dept delete statistics ;
Table analyzed.
SQL> set autotrace traceonly explain
SQL> select e.*, d.* from emp e, dept d where d.deptno = e.deptno (+) ;
Execution Plan
Plan hash value: 748780195
| Id | Operation | Name |
| 0 | SELECT STATEMENT | |
| 1 | NESTED LOOPS OUTER | |
| 2 | TABLE ACCESS FULL | DEPT |
| 3 | TABLE ACCESS BY INDEX ROWID| EMP |
|* 4 | INDEX RANGE SCAN | IDX_EMP_DEPTNO |
Predicate Information (identified by operation id):
4 - access("D"."DEPTNO"="E"."DEPTNO"(+))
Note
- rule based optimizer used (consider using cbo)
SQL> analyze table emp compute statistics for table for all indexes for all indexed columns ;
Table analyzed.
SQL> analyze table dept compute statistics for table for all indexes for all indexed columns ;
Table analyzed.
SQL> select e.*, d.* from emp e, dept d where d.deptno = e.deptno (+) ;
Execution Plan
Plan hash value: 748780195
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 14 | 826 | 4 (0)| 00:00:01 |
| 1 | NESTED LOOPS OUTER | | 14 | 826 | 4 (0)| 00:00:01 |
| 2 | TABLE ACCESS FULL | DEPT | 4 | 76 | 3 (0)| 00:00:01 |
| 3 | TABLE ACCESS BY INDEX ROWID| EMP | 4 | 160 | 1 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | IDX_EMP_DEPTNO | 5 | | 0 (0)| 00:00:01 |
Predicate Information (identified by operation id):
4 - access("D"."DEPTNO"="E"."DEPTNO"(+))
SQL> alter session set optimizer_mode = rule ;
Session altered.
SQL> select e.*, d.* from emp e, dept d where d.deptno = e.deptno (+) ;
Execution Plan
Plan hash value: 748780195
| Id | Operation | Name |
| 0 | SELECT STATEMENT | |
| 1 | NESTED LOOPS OUTER | |
| 2 | TABLE ACCESS FULL | DEPT |
| 3 | TABLE ACCESS BY INDEX ROWID| EMP |
|* 4 | INDEX RANGE SCAN | IDX_EMP_DEPTNO |
Predicate Information (identified by operation id):
4 - access("D"."DEPTNO"="E"."DEPTNO"(+))
Note
- rule based optimizer used (consider using cbo)
SQL> disconnect
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
SQL> -
Compression and query performance in data warehouses
Hi,
Using Oracle 11.2.0.3 have a large fact table with bitmap indexes to the asscoiated dimensions.
Understand bitmap indexes are compressed by default so assume cannot further compress them.
Is this correct?
Wish to try compress the large fact table to see if this will reduce the i/o on reads and therfore give performance benefits.
ETL speed fine just want to increase the report performance.
Thoughts - anyone seen significant gains in data warehouse report performance with compression.
Also, current PCTFREE on table 10%.
As only insert into tabel considering making this 1% to imporve report performance.
Thoughts?
ThanksFirst of all:
Table Compression and Bitmap Indexes
To use table compression on partitioned tables with bitmap indexes, you must do the following before you introduce the compression attribute for the first time:
Mark bitmap indexes unusable.
Set the compression attribute.
Rebuild the indexes.
The first time you make a compressed partition part of an existing, fully uncompressed partitioned table, you must either drop all existing bitmap indexes or mark them UNUSABLE before adding a compressed partition. This must be done irrespective of whether any partition contains any data. It is also independent of the operation that causes one or more compressed partitions to become part of the table. This does not apply to a partitioned table having B-tree indexes only.
This rebuilding of the bitmap index structures is necessary to accommodate the potentially higher number of rows stored for each data block with table compression enabled. Enabling table compression must be done only for the first time. All subsequent operations, whether they affect compressed or uncompressed partitions, or change the compression attribute, behave identically for uncompressed, partially compressed, or fully compressed partitioned tables.
To avoid the recreation of any bitmap index structure, Oracle recommends creating every partitioned table with at least one compressed partition whenever you plan to partially or fully compress the partitioned table in the future. This compressed partition can stay empty or even can be dropped after the partition table creation.
Having a partitioned table with compressed partitions can lead to slightly larger bitmap index structures for the uncompressed partitions. The bitmap index structures for the compressed partitions, however, are usually smaller than the appropriate bitmap index structure before table compression. This highly depends on the achieved compression rates. -
Large XML and Query performance
This problem came to me from a Developer and she claims XML query on XMLType field is very slow when using large XML and is there any alternates. Details are below:
=============
Query:
select attributepool_id, attributepool_name, vintage , p.attributepool.extract('//attributepool/segmentationsystem/id/text()').getStringVal() ,
p.attributepool.extract('//attributepool/datasource/id/text()').getStringVal()
from saved_attributepools p
where user_id = 'CLPROFILE2' and vintage = 'SPRING_2003' order by attributepool_name
Table name: saved_attributepools
Space: ecommerce
A Column Name: attributepool
attributepool Column Type: XmlType
One of xml contains 4Mbytes: CORE LIFESTLY
When we try to get the data against this row, query is taking longer.
conn ecommerce@ecom3 --> 82 seconds (table has 65 rows)
conn ecommerce@oradev--> 34 seconds (table has only 4 rows)
We think that;
Oracle parse the entire XML document and load this document into an 'in-memory' DOM structure before executing the specified xpaths.
Adding INDEX into XmlType won't help as we don't use whereclasue against XmlType for this case.
We don't know 10g has solution for this or not.
Any suggestion will be greatly appreciated.This problem came to me from a Developer and she claims XML query on XMLType field is very slow when using large XML and is there any alternates. Details are below:
=============
Query:
select attributepool_id, attributepool_name, vintage , p.attributepool.extract('//attributepool/segmentationsystem/id/text()').getStringVal() ,
p.attributepool.extract('//attributepool/datasource/id/text()').getStringVal()
from saved_attributepools p
where user_id = 'CLPROFILE2' and vintage = 'SPRING_2003' order by attributepool_name
Table name: saved_attributepools
Space: ecommerce
A Column Name: attributepool
attributepool Column Type: XmlType
One of xml contains 4Mbytes: CORE LIFESTLY
When we try to get the data against this row, query is taking longer.
conn ecommerce@ecom3 --> 82 seconds (table has 65 rows)
conn ecommerce@oradev--> 34 seconds (table has only 4 rows)
We think that;
Oracle parse the entire XML document and load this document into an 'in-memory' DOM structure before executing the specified xpaths.
Adding INDEX into XmlType won't help as we don't use whereclasue against XmlType for this case.
We don't know 10g has solution for this or not.
Any suggestion will be greatly appreciated. -
Avoid distinct clause in the query
hi i have a query from a single table using distinct clause and has columns (a,b,c,d,e,f,g,h,i)
eg: select distinct a,b ,c,d from table
where a<>0 and b<>0 and c<>0 and d<>0 and e=o and f=0 and g=0 and h=0 and i=0 ;
when i see the execution plan with out distinct... it performs a bit faster.
how do i remove distinct clause and make the query perform better
this particular query is used as inline view in my vieiw . it is used in from clause .
please suggest
second question.... iam getting bitmap index to rowid conversion ...is this good? does it hamper performance . how to avoid this?
regards
raj
Edited by: raj_fresher on Jul 17, 2009 7:48 AMHi Raj
what is e in the query ?It is the column name you provided yourself in your own unformatted example.
After 400+ posts I thought you would understand that it's not that hard to put this exact tag => (yes, 6 characters, 4 letters between 2 curly brackets) before and after your code examples or sample data or explain plans.
That way you maintain formatting and improve readability for us poor readers by magnitudes.
But, I'm sure you'll succeed as well as all the others who can.
I always look on the bright side.
Example:
Earlier this week another Raj also finally got enlightened about it, and his life has since then changed forever!
+He drives a Bentley now, you know+ ;) -
Problem with performance of a query having order by, distinct clause
Hi,
I have a problem with queries having order by, distinct clause.
While its executing its taking lot of time. With DBMS_PROFILER identified the queries taking long time.
The table is having approximately 70 million rows.
Problem -1
select * from table_name order by col1;
select distinct col1,col2 from table_name;
Here i am having 2 solutions request to let me know whether i am right if not suggest me right solution.
Solution1:
Max parallel servers is 8.
select /* + parallel(table_name,8) */ * from table_name order by col1;
select /* + parallel(table_name,8) */ distinct col1, col2 from table_name ;
Solution-2:
select /* + first_rows */ * from table_name order by col1;
select /* + first_rows */ distinct col1, col2 from table_name ;
Problem-2
I am having a query with where condition on columns.
Select * from table_name where col1='value1' and col2!='value2';
Index created on col1 and col2.
As we no that not equal won't use index as it is a composite index it should use the lead column. but its not using the index.
Should i forcibly use index with hint or suggest me better solution.
Any help really appreciated.
Thanks in advanceunique wrote:
The table is having approximately 70 million rows.
select * from table_name order by col1;Do you really want 70,000,000 rows from your table without any restrictions ? And furthermore ordered output ? I honestly understand the slowness of that query.
Here i am having 2 solutions request to let me know whether i am right if not suggest me right solution.Who knows if you choosed the right solution. I would suggest to reconsider your query and the need of 70,000,000 returned rows.
Problem-2
I am having a query with where condition on columns.
Select * from table_name where col1='value1' and col2!='value2';Please, provide the explain plan, eventually the tkprof output could also help. And tables descirption AND indexes description.
And OS and Oracle version.
Nicolas. -
Tunning the Query with Distinct Clause
Hi All,
I have the below query that returns 28113657 records
select src_Wc_id, osp_id, src_osp_id
from osp_complements o1
where exists (select 1 from wc_crossref wc
where wc.src_wc_id = o1.SRC_WC_ID
and wc.state = 'CA')
This query executes within a second...
But when i include a DISTINCT clause in the select statement, it takes more time ... (more than 20 mins)
I am trying to get it tunned. Please advice me with your knowledge to get it done
Thanks for your time
Kannan.Retrieving distinct rows requires a sort of all returned rows. 20 - 3 = ~17 mins for sorting 28 mln rows looks too much. You need to tune your instance in order to speed up sort operation. The amount of memory dedicated to sorts is controlled by PGA_AGGREGATE_TARGET parameter. If it's set to 0 (not recommended) then SORT_AREA_SIZE is used. The process of PGA tuning is quite complex and described in PGA Memory Management chapter of Performance Tuning Guide.
There is a workaround which allows to bypass sort operation, but it requires proper index and proper access by that index. The idea is that rows rertrieved via index are automatically ordered by indexed columns. If that and only that columns (possibly - in the same order as in the index, I don't know) are selected using DISTINCT then sort is not actually performed. Rows are already sorted due to access via index.
Hope this will help you.
Regards,
Dima -
Query performance using distinct
Greetings! We're on Oracle 8.1.7, Solaris 2.8.
I have a query that utilizes a different access path if I use the word distinct in the select from this view. Here is our query:
SELECT
DISTINCT SETID,
VENDOR_ID,
VENDOR_NAME_SHORT,
AR_NUM,
NAME1,
ADDRESS1,
ADDRESS2,
CITY
FROM
PS_VENDOR_VW
WHERE
SETID LIKE 'MNSA' AND
NAME1='FUN' ORDER BY NAME1, SETID, VENDOR_ID
The view SQL is:
SELECT /*+ FIRST_ROWS */ a.setid
, a.vendor_name_short
, a.name1
, c.address1
, c.address2
, c.city
, a.vendor_id
FROM SYSADM.ps_vendor a
SELECT /*+ INDEX_ASC(B PSAVENDOR_ADDR) */
FROM SYSADM.ps_vendor_addr b
WHERE b.effdt = (
SELECT MAX(effdt)
FROM SYSADM.ps_vendor_addr
WHERE setid = b.setid
AND vendor_id = b.vendor_id
AND address_seq_num = b.address_seq_num
AND effdt <= sysdate)) c
WHERE a.setid = c.setid (+)
AND a.vendor_id = c.vendor_id (+)
AND a.prim_addr_seq_num=c.address_seq_num (+)
This query does an index full scan on an index for ps_vendor_addr. It takes 2+ minutes to run. Now, if I remove the word distinct, it uses an index range scan and "view pushed predicate". It runs in 2 seconds.
I've tried with and without the first_rows hint in the view. If I leave off the INDEX_ASC hint then it does a full table scan of table ps_vendor_addr. It refuses to do a range scan with the hint. Can anybody tell me how I can get the 'distinct' query tuned?
2 minutes may not seem like a lot but when online users run the query many times a day, it is very frustrating.
Thanks! I hope I provided enough info.Thomas:
The different behaviours you are seeing are a result of the DISTINCT in the query. This causes a sort to be performed, and will influence the way that the CBO will execute the query. (You do know that you are using the Cost Based Optimizer because of the hints, and that you should analyze the tables and indexes?) You need to be able to re-write the view to avoid the need for the DISTINCT in the query.
Without knowing the meaning of the fields, it is really hard to say anything meaningful, but my guess is that it is the correlated sub-query that is ultimately causing the need for the DISTINCT. Is the combination of set_id,vendor_id and address_seq_num truly unique, or is the address_seq_num just a sequence.
For example in one of my databases, I have a table with INDV_ID, EFF_DT, EMPSTAT_SEQ. The empstat_seq field is just there to allow for more than one thing happening on the same day. The way we query this table is:
SELECT *
FROM empstat_t a
WHERE indv_id = :emp_id and
TO_CHAR(eff_dt,'yyyymmdd')||TO_CHAR(empstat_seq,'009') =
(SELECT MAX(TO_CHAR(eff_dt,'yyyymmdd')||TO_CHAR(empstat_seq,'009')
FROM empstat_t
WHERE a.indv_id = indv_id);Could something similar work in your case?
If not, assuming your statistics are up to date, I would also look at creating the view without hints to see what the optimizer comes up with on its own. It may be better than you think.
TTFN
John -
How to update this query and avoid performance issue?
Hi, guys:
I wonder how to update the following query to make it weekend day aware. My boss want the query to consider business days only. Below is just a portion of the query:
select count(distinct cmv.invoicekey ) total ,'3' as type, 'VALID CALL DATE' as Category
FROM cbwp_mv2 cmv
where cmv.colresponse=1
And Trunc(cmv.Invdate) Between (Trunc(Sysdate)-1)-39 And (Trunc(Sysdate)-1)-37
And Trunc(cmv.Whendate) Between cmv.Invdate+37 And cmv.Invdate+39the CBWP_MV2 is a materialized view to tune query. This query is written for a data warehouse application, the CBWP_MV2 will be updated every day evening. My boss wants the condition in the query to consider only business days, for example, if (Trunc(Sysdate)-1)-39 falls in weekend, I need to move the range begins from next coming business day, if (Trunc(Sysdate)-1)-37 falls in weekend, I need to move the range ends from next coming business day. but I should always keep the range within 3 business days. If there is overlap on weekend, always push to later business days.
Question: how to implement it and avoid performance issue? I am afraid that if I use a function, it greatly reduce the performance. This view already contains more than 100K rows.
thank you in advance!
Sam
Edited by: lxiscas on Dec 18, 2012 7:55 AM
Edited by: lxiscas on Dec 18, 2012 7:56 AMYou are already using a function, since you're using TRUNC on invdate and whendate.
If you have indexes on those columns, then they will not be used because of the TRUNC.
Consider omitting the TRUNC or testing with Function Based Indexes.
Regarding business days:
If you search this forum, you'll find lots of examples.
Here's another 'golden oldie': http://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:185012348071
Regarding performance:
Steps to take are explained from the links you find here: {message:id=9360003}
Read them, they are more than worth it for now and future questions. -
Poor query performance when I have select ...xmlsequence in from clause
Here is my query and it takes forever to execute. When I run the xmlsequence query alone, it takes a few seconds. But once added to from clause and I execute the query below together, it takes too long. Is there a better way to do the same?
What I am doing here is what is there in t1 should not be there in what is returned from the subquery (xmlsequence).
select distinct t1.l_type l_type,
TO_CHAR(t1.dt1,'DD-MON-YY') dt1 ,
TO_CHAR(t2.dt2,'DD-MON-YY') dt2, t1.l_nm
from table1 t1,table2 t2,
select k.column_value.extract('/RHS/value/text()').getStringVal() as lki
from d_v dv
, table(xmlsequence(dv.xmltypecol.extract('//c/RHS')))k
) qds
where
t2.l_id = t1.l_id
and to_char(t1.l_id) not in qds.lkiSQL> SELECT *
2 FROM TABLE(DBMS_XPLAN.DISPLAY);
Plan hash value: 2820226721
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 7611M| 907G| | 352M (1)|999:59:59 |
| 1 | HASH UNIQUE | | 7611M| 907G| 2155G| 352M (1)|999:59:59 |
|* 2 | HASH JOIN | | 9343M| 1113G| | 22M (2)| 76:31:45 |
| 3 | TABLE ACCESS FULL | table2 | 1088 | 17408 | | 7 (0)| 00:00:0
| 4 | NESTED LOOPS | | 8468M| 883G| | 22M (1)| 76:15:57 |
| 5 | MERGE JOIN CARTESIAN | | 1037K| 108M| | 4040 (1)| 00:00:49 |
| 6 | TABLE ACCESS FULL | D_V | 1127 | 87906 | | 56 (0)| 00
| 7 | BUFFER SORT | | 921 | 29472 | | 3984 (1)| 00:00:48 |
| 8 | TABLE ACCESS FULL | table1 | 921 | 29472 | | 4 (0)| 00:00:01 |
|* 9 | COLLECTION ITERATOR PICKLER FETCH| XMLSEQUENCEFROMXMLTYPE | | | |
Predicate Information (identified by operation id):
2 - access("t2"."L_ID"="t1"."L_ID")
9 - filter(TO_CHAR("t1"."L_ID")<>"XMLTYPE"."GETSTRINGVAL"("XMLTYPE"."EXTRACT"(VALUE(KOKB
alue/text()')))Message was edited by:
M@$$@cHu$eTt$ -
Suppress "Distinct" Clause from Answers Query
Hello All:
How can I suppress the "Distinct" clause that the BI Server issues to the Oracle Database?
For Ex:
I have a customer Table with FirstName and LastName as columns. If I Query FirstName and LastName in Answers, BI server sends a Query like this:
select distinct T8944.fname as c1,
T8944.lname as c2
from
CUSTOMER T8944
order by c1, c2
This will suppress customers having the same name even if their customer Id is different. I want their names included also.
I would appreciate your help.
Thanks
rkingmduAFAIK, there's no config setting or so to prevent this implicit grouping. What you can do though is to add the customer Id ot the report and make the column hidden (Column Format - Hide).
Hth,
Chris
Edited by: Christian Berg on Sep 25, 2008 4:39 PM:
Or what Matt said ;-) Should type faster... -
How to improve the query performance in to report level and designer level
How to improve the query performance in to report level and designer level......?
Plz let me know the detail view......first its all based on the design of the database, universe and the report.
at the universe Level, you have to check your Contexts very well to get the optimal performance of the universe and also your joins, keep your joins with key fields, will give you the best performance.
at the report level, try to make the reports dynamic as much as you can, (Parameters) and so on.
and when you create a paremeter try to get it match with the key fields in the database.
good luck
Amr -
QUERY PERFORMANCE AND DATA LOADING PERFORMANCE ISSUES
WHAT ARE QUERY PERFORMANCE ISSUES WE NEED TO TAKE CARE PLEASE EXPLAIN AND LET ME KNOW T CODES...PLZ URGENT
WHAT ARE DATALOADING PERFORMANCE ISSUES WE NEED TO TAKE CARE PLEASE EXPLAIN AND LET ME KNOW T CODES PLZ URGENT
WILL REWARD FULL POINT S
REGARDS
GURUBW Back end
Some Tips -
1)Identify long-running extraction processes on the source system. Extraction processes are performed by several extraction jobs running on the source system. The run-time of these jobs affects the performance. Use transaction code SM37 Background Processing Job Management to analyze the run-times of these jobs. If the run-time of data collection jobs lasts for several hours, schedule these jobs to run more frequently. This way, less data is written into update tables for each run and extraction performance increases.
2)Identify high run-times for ABAP code, especially for user exits. The quality of any custom ABAP programs used in data extraction affects the extraction performance. Use transaction code SE30 ABAP/4 Run-time Analysis and then run the analysis for the transaction code RSA3 Extractor Checker. The system then records the activities of the extraction program so you can review them to identify time-consuming activities. Eliminate those long-running activities or substitute them with alternative program logic.
3)Identify expensive SQL statements. If database run-time is high for extraction jobs, use transaction code ST05 Performance Trace. On this screen, select ALEREMOTE user and then select SQL trace to record the SQL statements. Identify the time-consuming sections from the results. If the data-selection times are high on a particular SQL statement, index the DataSource tables to increase the performance of selection (see no. 6 below). While using ST05, make sure that no other extraction job is running with ALEREMOTE user.
4)Balance loads by distributing processes onto different servers if possible. If your site uses more than one BW application server, distribute the extraction processes to different servers using transaction code SM59 Maintain RFC Destination. Load balancing is possible only if the extraction program allows the option
5)Set optimum parameters for data-packet size. Packet size affects the number of data requests to the database. Set the data-packet size to optimum values for an efficient data-extraction mechanism. To find the optimum value, start with a packet size in the range of 50,000 to 100,000 and gradually increase it. At some point, you will reach the threshold at which increasing packet size further does not provide any performance increase. To set the packet size, use transaction code SBIW BW IMG Menu on the source system. To set the data load parameters for flat-file uploads, use transaction code RSCUSTV6 in BW.
6)Build indexes on DataSource tables based on selection criteria. Indexing DataSource tables improves the extraction performance, because it reduces the read times of those tables.
7)Execute collection jobs in parallel. Like the Business Content extractors, generic extractors have a number of collection jobs to retrieve relevant data from DataSource tables. Scheduling these collection jobs to run in parallel reduces the total extraction time, and they can be scheduled via transaction code SM37 in the source system.
8). Break up your data selections for InfoPackages and schedule the portions to run in parallel. This parallel upload mechanism sends different portions of the data to BW at the same time, and as a result the total upload time is reduced. You can schedule InfoPackages in the Administrator Workbench.
You can upload data from a data target (InfoCube and ODS) to another data target within the BW system. While uploading, you can schedule more than one InfoPackage with different selection options in each one. For example, fiscal year or fiscal year period can be used as selection options. Avoid using parallel uploads for high volumes of data if hardware resources are constrained. Each InfoPacket uses one background process (if scheduled to run in the background) or dialog process (if scheduled to run online) of the application server, and too many processes could overwhelm a slow server.
9). Building secondary indexes on the tables for the selection fields optimizes these tables for reading, reducing extraction time. If your selection fields are not key fields on the table, primary indexes are not much of a help when accessing data. In this case it is better to create secondary indexes with selection fields on the associated table using ABAP Dictionary to improve better selection performance.
10)Analyze upload times to the PSA and identify long-running uploads. When you extract the data using PSA method, data is written into PSA tables in the BW system. If your data is on the order of tens of millions, consider partitioning these PSA tables for better performance, but pay attention to the partition sizes. Partitioning PSA tables improves data-load performance because it's faster to insert data into smaller database tables. Partitioning also provides increased performance for maintenance of PSA tables for example, you can delete a portion of data faster. You can set the size of each partition in the PSA parameters screen, in transaction code SPRO or RSCUSTV6, so that BW creates a new partition automatically when a threshold value is reached.
11)Debug any routines in the transfer and update rules and eliminate single selects from the routines. Using single selects in custom ABAP routines for selecting data from database tables reduces performance considerably. It is better to use buffers and array operations. When you use buffers or array operations, the system reads data from the database tables and stores it in the memory for manipulation, improving performance. If you do not use buffers or array operations, the whole reading process is performed on the database with many table accesses, and performance deteriorates. Also, extensive use of library transformations in the ABAP code reduces performance; since these transformations are not compiled in advance, they are carried out during run-time.
12)Before uploading a high volume of transaction data into InfoCubes, activate the number-range buffer for dimension IDs. The number-range buffer is a parameter that identifies the number of sequential dimension IDs stored in the memory. If you increase the number range before high-volume data upload, you reduce the number of reads from the dimension tables and hence increase the upload performance. Do not forget to set the number-range values back to their original values after the upload. Use transaction code SNRO to maintain the number range buffer values for InfoCubes.
13)Drop the indexes before uploading high-volume data into InfoCubes. Regenerate them after the upload. Indexes on InfoCubes are optimized for reading data from the InfoCubes. If the indexes exist during the upload, BW reads the indexes and tries to insert the records according to the indexes, resulting in poor upload performance. You can automate the dropping and regeneration of the indexes through InfoPackage scheduling. You can drop indexes in the Manage InfoCube screen in the Administrator Workbench.
14)IDoc (intermediate document) archiving improves the extraction and loading performance and can be applied on both BW and R/3 systems. In addition to IDoc archiving, data archiving is available for InfoCubes and ODS objects.
Hope it Helps
Chetan
@CP.. -
Weblogic 8.1.6 and Oracle 9.2.0.8 - query performance
Folks,
We are upgrading WebLogic from 8.1.5 to 8.1.6 and Oracle from 9.2.0.6 to 9.2.0.8. We use the Oracle thin client driver for 9.2.0.8 to connect from the application to Oracle.
When we use the following combination of the stack we see SQL query performance degradation: -
Oracle 9.2.0.8 database, Oracle 9.2.0.8 driver, WL 8.1.6
Oracle 9.2.0.8 database, Oracle 9.2.0.1 driver, WL 8.1.6
We do not see the degradation in case of the following: -
Oracle 9.2.0.8 database, Oracle 9.2.0.1 driver, WL 8.1.5
Oracle 9.2.0.6 database, Oracle 9.2.0.1 driver, WL 8.1.5
This shows that the problem could be with the WL 8.1.6 version and I was wondering if any of you have faced this before? The query retrieves a set of data from Oracle none of which contain the AsciiStream data type, which is noted as a problem in WL 8.1.6, but that too, only for WL JDBC drivers.
Any ideas appreciated.Folks,
We are upgrading WebLogic from 8.1.5 to 8.1.6 and Oracle from 9.2.0.6 to 9.2.0.8. We use the Oracle thin client driver for 9.2.0.8 to connect from the application to Oracle.
When we use the following combination of the stack we see SQL query performance degradation: -
Oracle 9.2.0.8 database, Oracle 9.2.0.8 driver, WL 8.1.6
Oracle 9.2.0.8 database, Oracle 9.2.0.1 driver, WL 8.1.6
We do not see the degradation in case of the following: -
Oracle 9.2.0.8 database, Oracle 9.2.0.1 driver, WL 8.1.5
Oracle 9.2.0.6 database, Oracle 9.2.0.1 driver, WL 8.1.5
This shows that the problem could be with the WL 8.1.6 version and I was wondering if any of you have faced this before? The query retrieves a set of data from Oracle none of which contain the AsciiStream data type, which is noted as a problem in WL 8.1.6, but that too, only for WL JDBC drivers.
Any ideas appreciated.
Maybe you are looking for
-
When I open firefox,on one user account, it opens my media center and not firefox? any ideas?
When I try to open up Firefox it tries to start in or opens my media center. But only on one of the accounts on my laptop.
-
Creating SQLSERVER source module in OWB 10gR2
I need to import the data from Sqlserver2005. For that I followed the steps: 1. Crated the System DSN. 2. Modified the tnsnames.ora,listner.ora. 3. Add a file like 'inithsodbc.ora' in /hs/admin 4. Listner started at server machine. 5. Created a datab
-
Problem when loading pdf files from Shared Content
When I load pdf files from Shared Content, I got the following problem: "The selected document could not be retrieved, please try uploading the document again." Anyone knows this? Thank you very much in advanced.
-
Reset code is being rejected. It will not allow m...
Skype rejected my password, I received a new code to access account via ail from skype, skype will not recognise code. Please help....
-
Oracle VM VirtualBox need~ VT-x /Intel Virtulization Technology missing in BIOS?
My laptop is lenovo S440,which is running Intel® Core™ i5-4200U Processor. I checked that Intel official website,the processor support Intel® Virtualization Technology (VT-x). However,by the time I running Oracle VM VirtualBOX,it prompt that "VT-x/AM