HierarchyID Indexing/Performance
Hi experts of the internet! I've got a weird issue, and I'm hoping you guys can help me out. I'm sure I'm just missing something fundamental
here, but I'm at my wits end.
Take the following scenario...
CREATE TABLE #ParentItems (
ItemID UNIQUEIDENTIFIER,
HID HIERARCHYID,
TreeID UNIQUEIDENTIFIER,
HIDLevel AS HID.GetLevel()
CREATE TABLE #ChildItems (
ItemID UNIQUEIDENTIFIER,
HID HIERARCHYID,
TreeID UNIQUEIDENTIFIER,
HIDLevel AS HID.GetLevel()
CREATE UNIQUE NONCLUSTERED INDEX PInd ON #ParentItems(TreeID, HIDLevel, HID)
CREATE UNIQUE NONCLUSTERED INDEX CInd ON #ChildItems(TreeID, HIDLevel, HID)
So two tables, the parent table has a bunch of IDs/HIDs which I know are ancestors of items in the child tables. There are multiple different
tree structures in each, hence the TreeId (all HIDs for the same tree, share the same treeID).
Assume #ParentItems and #ChildItems have 20,000+ rows in each.
The following query is reasonable fast (for what it's doing...) it takes around a second to complete.
SELECT DISTINCT LIParent.ItemID as ParentID, LIChild.ItemID as ChildID FROM #ParentItems LIParent
INNER JOIN #ChildItems LIChild ON LIChild.HID.IsDescendantOf(LIParent.HID)=1
However, this ignores the TreeID relationship as well. Now the second I add "WHERE LIParent.TreeID = LIChild.TreeID" to the query
(or add it to the join) the whole query jumps up to around 30 second execution time.
This seems like a very simple change, and I believe the index is in place... but it slows down massively.
Anyone have any ideas? Thanks in advance!
Can you post images (or XML file) of the execution plans? Thanks.
BOL: " Indexing Strategies for Hierarchical Data
There are two strategies for indexing hierarchical data:
Depth-first
A depth-first index stores the rows in a subtree near each other. For example, all employees that report through a manager are stored near their managers' record.
In a depth-first index, all nodes in the subtree of a node are co-located. Depth-first indexes are therefore efficient for answering queries about subtrees, such as "Find all files in this folder and its subfolders".
Breadth-first
A breadth-first stores the rows each level of the hierarchy together. For example, the records of employees who directly report to the same manager are stored near each other.
In a breadth-first index all direct children of a node are co-located. Breadth-first indexes are therefore efficient for answering queries about immediate children, such as "Find all employees who report directly to this manager".
Whether to have depth-first, breadth-first, or both, and which to make the clustering key (if any), depends on the relative importance of the above types of queries, and the relative importance of SELECT vs. DML operations. For a detailed example of indexing
strategies, see Tutorial: Using the hierarchyid Data Type."
LINK: http://technet.microsoft.com/en-us/library/bb677173.aspx
Kalman Toth Database & OLAP Architect
SELECT Video Tutorials 4 Hours
New Book / Kindle: Exam 70-461 Bootcamp: Querying Microsoft SQL Server 2012
Similar Messages
-
Error in Z-Payroll function (No free numbers found for indexed perform)
Hi SAP-HR Experts .
I tried to write a Payroll Function by Pe04 .
When i saved it into PC40 Packege after that it gives me error that
Message no. P0712 , (" No free numbers found for indexed perform")
Please suggest me How to solve this .
Thanks & Regards : rajneeshHello Rajneesh,
Customer fuctions can not began with z as this is in the SAP namespace.
If you are using the customer namespace then please review SAP Note No. 1324745
Regards,
Manny -
Oracle Text Indexing performance in Unicode database
Forum folks,
I'm looking for overall performance thoughts in Text Indexing within a Unicode database. Part of our internal testing suites includes searching on values using contains filters over indexed binary and text documents. We've architected these tests such that they could be run in a suite or on their own, thus, the data is loaded at the beginning of each test and then the text indexes are created and populated prior to running any of the actual testing.
We have the same tests running on non-unicode instances of Oracle 11gR2 just fine, but when we run them against a Unicode instance, we are almost always seeing timing issues where the indexes haven't finished populating, thus our tests are saying we've only found n number of hits when we are expecting n+ 50 or in some cases n + 150 records to be returned.
We are just looking for some general information in regards to text indexing performance in a unicode database. Will we need to add sleep time to the testing to allow for the indexes to populate? How much time? We would rather not get into having to create different tests for unicode vs non-unicode, but perhaps that is necessary.
Any insight you could provide would be most appreciated.
Thanks in advance,
DanRoger,
Thanks much for your quick reply...
When you talk about Unicode, do you mean AL32UTF8?
--> Yes, this is the Unicode charset we are using.
Is the data the same in both cases, or are you indexing simple 7-bit ascii data in the one database, and foreign text (maybe Chinese?) in the UTF8 database?
With the same data, there should be virtually no difference in performance due to the AL32UTF8 database character set.
--> We have a data generation tool we utilize. For non-unicode data, we generate using all 256 characters in the ISO-8859-1 set. With our Unicode data for clobs, we generate using only the first 1,000 characters of UTF8 by setting up an array of code points...0 - 1000. For Blobs, we have sets of sample word documents and pdfs that are inserted, then indexed.
I'm not sure I understand your testing methodology. Do you run ( load-data, index-data, run-queries ) sequentially?
--> That is correct. We utilize the ctx_ddl package to populate the pending table and then to sync the index....The following is an example of the ddl we generate to create and populate the index:
create index "DBMEARSPARK_ORA80"."RESRESUMEDOC" on "DBMEARSPARK_ORA80"."RESUME" ("RESUMEDOC") indextype is CTXSYS.CONTEXT parameters(' nopopulate sync (every "SYSTIMESTAMP + INTERVAL ''30'' MINUTE" PARALLEL 2) filter ctxsys.auto_filter ') PARALLEL 2;
execute ctx_ddl.populate_pending('"DBMEARSPARK_ORA80"."RESRESUMEDOC"',null);
execute ctx_ddl.sync_index('"DBMEARSPARK_ORA80"."RESRESUMEDOC"',null,null,2);
If so, there should be no way that the indexes can be half-created. If not, don't you have some check to see if the index creation has finished before running the query test?
--> Excellent question....is there such a check? I have not found a way to do that yet...
Were you just lucky with the "non-unicode" tests that the indexing just happened to have always finished by the time you ran the queries?
--> This is quite possible as well. If there is a check to see if the index is ready, then we could add that into our infrastructure.
--> Thanks, again, for responding so quickly.
Edited by: djulson on Feb 12, 2013 7:13 AM -
Using Composite Index Performance Issue
Hi,
Need help:
I have a table 'TABLEA' which has composite index on CODE and DEPT_CODE
While fetching the data from the table 'TABLEA' in the WHERE condition I am using only CODE and not DEPT_CODE.
Is the usage of the WHERE condition by using only the CODE column and not DEPT_CODE column affects the performance?
Any help will be needful for meSee the test case below
SQL> create table test_emp
2 (
3 emp_ssn number,
4 emp_name varchar2(50),
5 emp_state varchar2(15),
6 emp_city varchar2(20)
7 );
Table created
SQL> create index test_emp_idx on test_emp(emp_ssn,emp_name);
Index created
SQL> insert into test_emp values (123456789,'Robben','New York','Buffalo');
1 row inserted
SQL> insert into test_emp values (223456789,'Jack','Florida','Miami');
1 row inserted
SQL> insert into test_emp values (323456789,'Peter','Texas','Dallas');
1 row inserted
SQL> insert into test_emp values (423456789,'Johny','Georgia','Atlanta');
1 row inserted
SQL> insert into test_emp values (523456789,'Carmella','California','San Diego');
1 row inserted
SQL> commit;
Commit complete
SQL> explain plan for select /*+ index(test_emp test_emp_idx) */ * from test_emp where emp_ssn = 323456789;
Explained
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
Plan hash value: 2345760695
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)
| 0 | SELECT STATEMENT | | 1 | 61 | 163 (1)
| 1 | TABLE ACCESS BY INDEX ROWID| TEST_EMP | 1 | 61 | 163 (1)
|* 2 | INDEX RANGE SCAN | TEST_EMP_IDX | 1 | | 2 (0)
Predicate Information (identified by operation id):
2 - access("EMP_SSN"=323456789)
Note
- dynamic sampling used for this statement
18 rows selected
SQL> explain plan for select /*+ INDEX_SS(test_emp test_emp_idx) */ * from test_emp where emp_name = 'Robben';
Explained
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
Plan hash value: 85087452
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)
| 0 | SELECT STATEMENT | | 1 | 61 | 15 (0)
| 1 | TABLE ACCESS BY INDEX ROWID| TEST_EMP | 1 | 61 | 15 (0)
|* 2 | INDEX SKIP SCAN | TEST_EMP_IDX | 1 | | 11 (0)
Predicate Information (identified by operation id):
2 - access("EMP_NAME"='Robben')
filter("EMP_NAME"='Robben')
Note
- dynamic sampling used for this statement
19 rows selectedThanks,
Andy -
Very SLOW IOT secondary indexes performance ORACLE 9i
Hi all,
Is it TRUE that in ORACLE 9i version secondary IOT table indexes are
extremely slow?
I created IOT table of 250 million rows with no OWERFLOW segment
I created secondary index, altered it updating its block references
Now i have that it takes about 12 minutes to get rows using that index on IOT table
And it takes about 14 seconds in regular heap-table using the same index
Is it of the same opera that Oracle states: that secondary indexes on IOT table
are similar to regular indexes on performance. but in reality they are significantly slower?
Does anyone had such situation?
What can be done to speed up access?
Thanks for answers.Hi all,
Is it TRUE that in ORACLE 9i version secondary IOT table indexes are
extremely slow?
I created IOT table of 250 million rows with no OWERFLOW segment
I created secondary index, altered it updating its block references
Now i have that it takes about 12 minutes to get rows using that index on IOT table
And it takes about 14 seconds in regular heap-table using the same index
Is it of the same opera that Oracle states: that secondary indexes on IOT table
are similar to regular indexes on performance. but in reality they are significantly slower?
Does anyone had such situation?
What can be done to speed up access?
Thanks for answers. -
Statistics, Rebuilding Indexes, Performance Issues
I have a query which use to run 2-3 sec
After taking stats and rebuliding indexes
This Query is taking 25 sec
Is there anyway that I can get mt performance back?
Collected Stats
First Using Analyze table Compute stats
Second DBMS_STATS.GATHER_ TABLE_STATS
Level 12 Tracing
TKPROF: Release 9.2.0.6.0 - Production on Sun Jun 29 13:23:11 2008
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Sort options: prsela exeela fchela
count = number of times OCI procedure was executed
cpu = cpu time in seconds executing
elapsed = elapsed time in seconds executing
disk = number of physical reads of buffers from disk
query = number of buffers gotten for consistent read
current = number of buffers gotten in current mode (usually for update)
rows = number of rows processed by the fetch or execute call
SELECT ped.addrs_typ, ped.bnk_addrs_seq_no, ped.clm_case_no,
ped.eft_payee_seq_no, ped.partition_desgntr,
ped.payee_bnk_acct_typ, ped.payee_eft_dtl_no,
ped.paye_bnk_acct_no, ped.paye_bnk_nm, ped.paye_bnk_rtng_no,
ped.row_updt_sys_id, ped.vrsn_no, el.clm_payee_no
FROM payee_eft_detail ped, eft_payee_lnk el, clm_payee cp
WHERE ped.curr_row_ind = 'A'
AND cp.curr_row_ind = 'A'
AND cp.clm_payee_no = el.clm_payee_no
AND cp.mail_zip = 'XXXXXX'
AND ped.paye_bnk_rtng_no = 'XXXXXX'
AND ped.paye_bnk_acct_no = 'XXXXXXX'
AND ped.payee_bnk_acct_typ = 'XXXX'
AND ped.eft_payee_seq_no = el.eft_payee_seq_no
call count cpu elapsed disk query current rows
Parse 1 0.02 0.01 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1 23.46 22.91 0 1292083 0 0
total 3 23.48 22.93 0 1292083 0 0
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 117
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
Parse 1 0.02 0.01 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1 23.46 22.91 0 1292083 0 0
total 3 23.48 22.93 0 1292083 0 0
Misses in library cache during parse: 1
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
Parse 0 0.00 0.00 0 0 0 0
Execute 0 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
total 0 0.00 0.00 0 0 0 0
Misses in library cache during parse: 0
1 user SQL statements in session.
0 internal SQL statements in session.
1 SQL statements in session.
Trace file compatibility: 9.02.00
Sort options: prsela exeela fchela
1 session in tracefile.
1 user SQL statements in trace file.
0 internal SQL statements in trace file.
1 SQL statements in trace file.
1 unique SQL statements in trace file.
41 lines in trace file.
****************-----==========================================*****
PLAN_TABLE_OUTPUT
| Id | Operation | Name | Rows | Bytes | Cost | Pstart| Pstop |
| 0 | SELECT STATEMENT | | 1 | 73 | 5 | | |
| 1 | NESTED LOOPS | | 1 | 73 | 5 | | |
| 2 | NESTED LOOPS | | 12 | 708 | 4 | | |
| 3 | TABLE ACCESS BY GLOBAL INDEX ROWID| PAYEE_EFT_DETAIL_T | 12 | 540 | 1 | ROWID | ROW L |
|* 4 | INDEX RANGE SCAN | TEST_PAYEE_EFT_DETAIL_T_IE21 | 12 | | 3 | | |
| 5 | TABLE ACCESS BY GLOBAL INDEX ROWID| EFT_PAYEE_LNK_T | 1 | 14 | 1 | ROWID | ROW L |
|* 6 | INDEX RANGE SCAN | EFT_PAYEE_LNK_PK | 1 | | 1 | | |
|* 7 | INDEX RANGE SCAN | CLM_PAYEE_T_IE10 | 1 | 14 | | | |
Predicate Information (identified by operation id):
4 - access("PED"."PAYE_BNK_RTNG_NO"='XXXXXXX' AND "PED"."PAYE_BNK_ACCT_NO"='XXXXXXXXXX' AND
"PED"."PAYEE_BNK_ACCT_TYP"='CHK' AND "PED"."CURR_ROW_IND"='A')
6 - access("PED"."EFT_PAYEE_SEQ_NO"="LNK"."EFT_PAYEE_SEQ_NO")
7 - access("LNK"."CLM_PAYEE_NO"="CP"."CLM_PAYEE_NO" AND "CP"."MAIL_ZIP"='XXXXXX' AND "CP"."CURR_ROW_IND"='A')
==++++++++++++++*************************=+++++++++++++----------------=========
TKPROF: Release 9.2.0.6.0 - Production on Sun Jun 29 19:28:39 2008
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Sort options: prsela exeela fchela
count = number of times OCI procedure was executed
cpu = cpu time in seconds executing
elapsed = elapsed time in seconds executing
disk = number of physical reads of buffers from disk
query = number of buffers gotten for consistent read
current = number of buffers gotten in current mode (usually for update)
rows = number of rows processed by the fetch or execute call
SELECT ped.addrs_typ, ped.bnk_addrs_seq_no, ped.clm_case_no,
ped.eft_payee_seq_no, ped.partition_desgntr,
ped.payee_bnk_acct_typ, ped.payee_eft_dtl_no,
ped.paye_bnk_acct_no, ped.paye_bnk_nm, ped.paye_bnk_rtng_no,
ped.row_updt_sys_id, ped.vrsn_no, el.clm_payee_no
FROM payee_eft_detail ped, eft_payee_lnk el, clm_payee cp
WHERE ped.curr_row_ind = 'A'
AND cp.curr_row_ind = 'A'
AND cp.clm_payee_no = el.clm_payee_no
AND cp.mail_zip = 'XXXXXX'
AND ped.paye_bnk_rtng_no = 'XXXXXXXXXX'
AND ped.paye_bnk_acct_no = 'XXXXXXXX'
AND ped.payee_bnk_acct_typ = 'CHK'
AND ped.eft_payee_seq_no = el.eft_payee_seq_no
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1 23.30 22.75 0 1292083 0 0
total 3 23.30 22.75 0 1292083 0 0
Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 117
Rows Row Source Operation
0 NESTED LOOPS
214395 NESTED LOOPS
214395 TABLE ACCESS BY GLOBAL INDEX ROWID PAYEE_EFT_DETAIL_T PARTITION: ROW LOCATION ROW LOCATION
214395 INDEX RANGE SCAN TEST_PAYEE_EFT_DETAIL_T_IE21 (object id 160840)
214395 TABLE ACCESS BY GLOBAL INDEX ROWID EFT_PAYEE_LNK_T PARTITION: ROW LOCATION ROW LOCATION
214395 INDEX RANGE SCAN EFT_PAYEE_LNK_PK (object id 75455)
0 INDEX RANGE SCAN CLM_PAYEE_T_IE10 (object id 71871)
alter session set sql_trace=false
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
total 2 0.00 0.00 0 0 0 0
Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 117
ALTER SESSION SET SQL_TRACE = TRUE
call count cpu elapsed disk query current rows
Parse 0 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
total 1 0.00 0.00 0 0 0 0
Misses in library cache during parse: 0
Misses in library cache during execute: 1
Optimizer goal: CHOOSE
Parsing user id: 117
OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
Parse 2 0.00 0.00 0 0 0 0
Execute 3 0.00 0.00 0 0 0 0
Fetch 1 23.30 22.75 0 1292083 0 0
total 6 23.30 22.75 0 1292083 0 0
Misses in library cache during parse: 1
Misses in library cache during execute: 1
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
Parse 0 0.00 0.00 0 0 0 0
Execute 0 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
total 0 0.00 0.00 0 0 0 0
Misses in library cache during parse: 0
3 user SQL statements in session.
0 internal SQL statements in session.
3 SQL statements in session.
Trace file compatibility: 9.02.00
Sort options: prsela exeela fchela
1 session in tracefile.
3 user SQL statements in trace file.
0 internal SQL statements in trace file.
3 SQL statements in trace file.
3 unique SQL statements in trace file.
57 lines in trace file.
Message was edited by:
user644525I have a query which use to run 2-3 sec
After taking stats and rebuliding indexes
This Query is taking 25 sec
Is there anyway that I can get mt performance back?
PLAN_TABLE_OUTPUT
| Id | Operation | Name | Rows | Bytes | Cost | Pstart| Pstop |
| 0 | SELECT STATEMENT | | 1 | 73 | 5 | | |
| 1 | NESTED LOOPS | | 1 | 73 | 5 | | |
| 2 | NESTED LOOPS | | 12 | 708 | 4 | | |
| 3 | TABLE ACCESS BY GLOBAL INDEX ROWID| PAYEE_EFT_DETAIL_T | 12 | 540 | 1 | ROWID | ROW L |
|* 4 | INDEX RANGE SCAN | TEST_PAYEE_EFT_DETAIL_T_IE21 | 12 | | 3 | | |
| 5 | TABLE ACCESS BY GLOBAL INDEX ROWID| EFT_PAYEE_LNK_T | 1 | 14 | 1 | ROWID | ROW L |
|* 6 | INDEX RANGE SCAN | EFT_PAYEE_LNK_PK | 1 | | 1 | | |
|* 7 | INDEX RANGE SCAN | CLM_PAYEE_T_IE10 | 1 | 14 | | | |
Predicate Information (identified by operation id):
4 - access("PED"."PAYE_BNK_RTNG_NO"='XXXXXXX' AND "PED"."PAYE_BNK_ACCT_NO"='XXXXXXXXXX' AND
"PED"."PAYEE_BNK_ACCT_TYP"='CHK' AND "PED"."CURR_ROW_IND"='A')
6 - access("PED"."EFT_PAYEE_SEQ_NO"="LNK"."EFT_PAYEE_SEQ_NO")
7 - access("LNK"."CLM_PAYEE_NO"="CP"."CLM_PAYEE_NO" AND "CP"."MAIL_ZIP"='XXXXXX' AND "CP"."CURR_ROW_IND"='A')
==++++++++++++++*************************=+++++++++++++----------------=========
SELECT ped.addrs_typ, ped.bnk_addrs_seq_no, ped.clm_case_no,
ped.eft_payee_seq_no, ped.partition_desgntr,
ped.payee_bnk_acct_typ, ped.payee_eft_dtl_no,
ped.paye_bnk_acct_no, ped.paye_bnk_nm, ped.paye_bnk_rtng_no,
ped.row_updt_sys_id, ped.vrsn_no, el.clm_payee_no
FROM payee_eft_detail ped, eft_payee_lnk el, clm_payee cp
WHERE ped.curr_row_ind = 'A'
AND cp.curr_row_ind = 'A'
AND cp.clm_payee_no = el.clm_payee_no
AND cp.mail_zip = '16803'
AND ped.paye_bnk_rtng_no = '111000614'
AND ped.paye_bnk_acct_no = '1586266775'
AND ped.payee_bnk_acct_typ = 'CHK'
AND ped.eft_payee_seq_no = el.eft_payee_seq_no
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1 23.30 22.75 0 1292083 0 0
total 3 23.30 22.75 0 1292083 0 0
Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 117
Rows Row Source Operation
0 NESTED LOOPS
214395 NESTED LOOPS
214395 TABLE ACCESS BY GLOBAL INDEX ROWID PAYEE_EFT_DETAIL_T PARTITION: ROW LOCATION ROW LOCATION
214395 INDEX RANGE SCAN TEST_PAYEE_EFT_DETAIL_T_IE21 (object id 160840)
214395 TABLE ACCESS BY GLOBAL INDEX ROWID EFT_PAYEE_LNK_T PARTITION: ROW LOCATION ROW LOCATION
214395 INDEX RANGE SCAN EFT_PAYEE_LNK_PK (object id 75455)
0 INDEX RANGE SCAN CLM_PAYEE_T_IE10 (object id 71871)Trying to recreate my response to this thread that was posted yesterday...
The query is performing 1.3 million logical reads, per the "query" column in the TKPROF output. 1.3 million 8KB (or 16KB) logical reads takes a fair amount of time, and is consuming 23.30 CPU seconds of time (execution time is 22.75 seconds). The explain plan and the row source execution plan are identical, although the predicted cardinality numbers are much lower than the actual number of rows returned, per the TKPROF output (12 rows versus 214,395 rows). The CLM_PAYEE table seems to have the greatest number of restrictions placed on it, yet Oracle is joining that table last.
You found, per my recommendation, that adding the hint /*+ LEADING(CP) */ significantly decreased the execution time to less than one second. You may not have collected statistics on the indexes, even though you collected statistics on the tables. This may also be a sign that one or more histograms are required for the cardinality estimates to be close to accurate.
Did your GATHER_TABLE_STATS commands look similar to the following:
DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'PAYEE_EFT_DETAIL',CASCADE=>TRUE);
DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'EFT_PAYEE_LNK',CASCADE=>TRUE);
DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'CLM_PAYEE',CASCADE=>TRUE);Charles Hooper
IT Manager/Oracle DBA
K&M Machine-Fabricating, Inc. -
Enq TM Contention vs Poor index performance
I have the following issue: I have a lot of Enq TM contention in my database. I then index the relevant tables/columns. This gets rid of the Enq TM issues, but performance is degraded even more due to the massive index reads. We have a good high performance disk system, so that's not the problem.
Has anyone had a similar issue? Aside from testing 101 different possibilities, does anyone have a recommendation on resolving this?
TIA,
Mikeuser12201192 wrote:
I have the following issue: I have a lot of Enq TM contention in my database. I then index the relevant tables/columns. This gets rid of the Enq TM issues, but performance is degraded even more due to the massive index reads. We have a good high performance disk system, so that's not the problem.
Has anyone had a similar issue? Aside from testing 101 different possibilities, does anyone have a recommendation on resolving this?
you have a mystery; while we have no clues.
post results from following SQL
SELECT * FROM V$VERSION
HOW To Make TUNING request
SQL and PL/SQL FAQ -
How to Improve Local Index Performance ??
I need to store telecom CDR Data, which having following fields
CDR_DATE => Date
Telephone_Num=> Varchar2(20)
A=> Varchar2(40)
B=> Varchar2(10)
The Input Data volume is very High.. At Present 100 Million/Day
So i created the Oracle Partition Table with Date Range Partition.
The application will run always one type query
select * from CDR where Telephone_Num='&TNUM' AND CDR_DATE between JAN09 AND MAR09;
Question1
what will be Best way to create Index? Which can provide best performance and not degrade Daily Loading of Data.
Question2- For this I created the LOCAL Index
Create Index ABC ON CDR (CDR_DATE,Telephone_Num) LOCAL;
The Data fetching is using the index but I can see in Trace, the count of CONSISTENT GETS & PHYSICAL READS are very High.
So please suggest, Creating LOCAL INDEX is wise decision or not. Or any other way.
Thanks in advance.
Sumit
Edited by: Sumit2 on Jul 31, 2010 6:27 PMSumit2 wrote:
The Input Data volume is very High.. At Present 100 Million/Day
So i created the Oracle Partition Table with Date Range Partition.
The application will run always one type query
select * from CDR where Telephone_Num='&TNUM' AND CDR_DATE between JAN09 AND MAR09;
Question1
what will be Best way to create Index? Which can provide best performance and not degrade Daily Loading of Data.
Question2- For this I created the LOCAL Index
Create Index ABC ON CDR (CDR_DATE,Telephone_Num) LOCAL;
The Data fetching is using the index but I can see in Trace, the count of CONSISTENT GETS & PHYSICAL READS are very High.
You've created the index with the columns in the wrong order - your equality condition is on telephone number and the range-based condition is on the date, so you need the telephone number first in the index.
In fact, if your query is always going to be for whole days, you might as well exclude the date column from the index because it adds no precision to the query.
Another option to consider is to create the table as an index-organized table that starts with a primary key that (telephone number, cdr_date) so that all the data for a given telephone number is contained within a very small number of index leaf blocks. (However, if you rarely have more than a couple of calls per number per day then the supporting strategies for this approach will cost more than the benefit you get from building the data structure to match the query requirements.)
As far as data loading is concerned, your best strategy is to look at playing games with local indexes and partition exchange.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
There is a +"Preview"+ tab at the top of the text entry panel. Use this to check what your message will look like before you post the message. If it looks a complete mess you're unlikely to get a response. (Click on the +"Plain text"+ tab if you want to edit the text to tidy it up.)
+"Science is more than a body of knowledge; it is a way of thinking"+
+Carl Sagan+ -
Oracle Text ALTER INDEX Performance
Greetings,
We have encountered some ehancement issues with Oracle Text and really need assistance.
We are using Oracle 9i (Release 9.0.1) Standard Edition
We are using a very simple Oracle text environmet, with CTXSYS.CONTEXT indextype on Domain Indexes.
We have indexed two text columns in one table, one of these columns is CLOB.
Currently if one of these columns is modified, we are using a trigger to automatically ALTER the index.
This is very slow, it is just like dropping the index and creating it again.
Is this right? should it be this slow?
We are also trying to use the ONLINE parameter for ALTER INDEX and CREATE INDEX, but it gives an error saying this feature is not enabled.
How can we enable it?
Is there any way in improving the performance of this automatic update of the indexes?
Would using a trigger be the best way to do this?
How can we optimize it to a more satifactory performance level?
Also, are we able to use the language lexers for indexes with the Standard Edition. If so, how do you enable the CTX_DLL?
Many thanks for any assistance.
Chi-Shyan WangIf you are going to sync your index on every update, you need to make sure that you are optmizing it on a regular basis to remove index fragmentation and remove deleted rows.
you can set up a dmbs_job to do a ctx_ddl.optmize and run a full optmize periodically.
Also, depending on the number of rows you have, and also the size of the data, you might want to look at using a CTXCAT index, which is transactional, stays in sync automatically and does not need to be optimized. CTXCAT indexes do not work well on large text objects (they are good for a couple lines of text at most) so they may not suit your dataset. -
Non-Batch Index Performance Issue
I have two tables that have identical data, one is receiving the data via a batch import, the other is receiving data real time at about 10 rows per second, every second, every day. Both tables also have the same index applied and both tables have a similar number of rows.
The batch import table seems to make use of the index and responds to queries (using 'where') very quickly while the table with real time data flow takes considerably longer to respond to the query.
What is the best way to optimize the index on tables that do not receive batch imports?Thanks for the replies,
Stats are gathered in the DB Maintenance Window every night - using the Oracle recommended 'auto' sample size. The stats on the table and indexes are no more than 24 hours old at any given time.
Also, the tkprof ouput is below:
select to_date(MSGDATETIME, 'YYYY-MM-DD HH24:MI:SS') DTG, MGHOSTNAME, PIXIDN, PIXTXT
from dmblue.fw16master
where msghostname = '10.104'
and to_char(to_date(msgdatetime, 'YYYY-MM-DD HH24:MI:SS'), 'Mon DD YYYY / HH24') like 'Sep 29 2006 / 09'
and msglevel = 'Info'
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 90 26.85 237.42 750925 757238 0 1335
total 92 26.85 237.43 750925 757238 0 1335
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: SYS
Rows Row Source Operation
1335 TABLE ACCESS FULL FW16MASTER (cr=757238 pr=750925 pw=0 time=242940908 us)
Any ideas? -
If anyone can look at this that would be of great help.
The context index creation for a million rows is taking forever. We had let it run for 10 or so hours and it still was not finished so we had to cancel it.
We are creating the index as follows:
create index content_version_text_idx on content_version
(dummy_indexed_column)
indextype IS ctxsys.context parameters('datastore concat_cols_datastore sync (on commit)
section group ctxsys.auto_section_group');
The procedure used above in indexing is this:
CREATE OR REPLACE procedure concat_cols
(p_rowid IN ROWID,
p_clob IN OUT clob)
AS
v_clob CLOB :='';
title varchar2(7) := '<title>';
titlec varchar2(8) := '</title>';
detail varchar2(6) := '<DATA>';
detailc varchar2(7) := '</DATA>';
meta varchar2(6) := '<META>';
metac varchar2(7) := '</META>';
localecode varchar2(8) := '<LOCALE>';
localecodec varchar2(9) := '</LOCALE>';
BEGIN
FOR c1 IN
(SELECT content_id, content_data_id, version_number, content_title, isactive_flag
from content_version
WHERE ROWID = p_rowid
LOOP
FOR c0 IN
(SELECT ' ' || locale_iso_code as data
FROM content a1
WHERE c1.content_id = a1.content_id
LOOP
v_clob := v_clob || localecode;
v_clob := v_clob || c0.data;
v_clob := v_clob || localecodec;
END LOOP;
v_clob := v_clob || title;
v_clob := v_clob || c1.content_title;
v_clob := v_clob || titlec;
FOR c3 IN
(SELECT ' ' || content_data as data
FROM content_data b
WHERE b.content_data_id = c1.content_data_id
AND c1.isactive_flag = 1)
LOOP
v_clob := v_clob || detail;
v_clob := v_clob || c3.data;
v_clob := v_clob || detailc;
END LOOP;
FOR c4 IN
(SELECT ' ' || short_string_value || ' ' || long_string_value as data
FROM meta_value d
WHERE d.content_id = c1.content_id
AND d.version_number = c1.version_number
AND c1.isactive_flag = 1)
LOOP
v_clob := v_clob || meta;
v_clob := v_clob || c4.data;
v_clob := v_clob || metac;
END LOOP;
END LOOP;
p_clob := v_clob;
END concat_cols;
Anyone any ideas ?OK have managed to reduce the indexing time by removing the xml tagging in the loop, instead am now using the sql to construct the tags when doing the select. so the sqls have now changed to
SELECT ' <data>'||content_data||'</data>' as data
FROM content_data b
WHERE b.content_data_id = c1.content_data_id
This change has brought the time for 30MB of data from around 8 mins to around 4.30 mins. Have noticed in the log that there are some rows that is taking lot of time to be indexed and because of that think that it is loosing around 40 seconds. Have turned on the logging to print the rowid but not sure what is causing these rows longer indexing time, these rows do not seem to contain any more data then the other rows. -
Using GUIDs : Performance of Indexes/Queries?
Hi,
We are building a new database. The database tables is coupled together with key values which are GUIDs. The database size may grow to in excess of 2Billion records in a few months.
I am not sure here about how would Oracle Indexes perform given the very random nature of values of GUIDs. If I create an Index on a GUID column, would Oracle really use the Index when I have queries with the GUIDs in the where clause? Or would it do a full table scan for those millions of rows? Please let me know your views.
If anyone know of some best practices surrounding GUIDs or some good article on using GUIDs and possible issues associated with that, how to plan such huge database with keys with are GUIDs in nature, etc.., then even that would help me out.
Thanks a lot!
Biren.Hi Biren,
By GUID, are you referring to the Oracle Portal product? -
Use CONTEXT index on mview or use mview's rewrite directly
Please let me explain what I mean in Subject. I have 9 tables. Each of these tables has about 40,000 rows and one table has 2 million rows. Using first approach, I can build a join-only materialized view on top of nine table's mview log. then query on these nine tables directly. Advantage for doing that is use rewrite.
<p>
for second approach, I build a CONETXT index on several columns on the mview, and then query through CONTEXT index on mview directly. This is pretty much like Barbara did on CREATE INDEX book_idx ON book_search
[http]Indexing multiple columns of multiple tables using CTXCAT
but she used CTXCAT instead of CONTEXT index.
<p>
My question is will second approach better than first one and why. Unlike basic join several tables which gives you predictable performance, I often feel that CONTEXT index performance is unpredictable when tables have more than several thousands rows (maybe I did something wrong, but still looking for document regarding performance) .
<p>
I will appreciate someone could show hints on the issue.
<p>
Message was edited by:
qwe15933
Message was edited by:
qwe15933
Message was edited by:
qwe15933
Message was edited by:
qwe15933The best method to find out what is best for any individual situation is to test and compare. In general, Oracle Text is best at searching unstructured data, such as large documents and has a lot of features that enable you to do different kinds of searches. If you only have structured data, with each column only containing one short thing and you only need simple searches, then you probably don't need Text. There are also a few things that can be done to indexes and/or queries to optimize performance. It would help to have an idea what your typical data is like and what sorts of searches you anticipate needing.
-
Auto-indexing is slow for arrays with 1 dimensions
Hi,
I was looking at the performance of operations on all individual elements in 3D arrays, especially the difference between auto-indexing (left image) and manual-indexing (right image, calling "Index array" and "Replace array subset" in the innermost loop). I'm a bit puzzled by the results and post it here for discussion and hopefully somebody's benefit in the future.
Left: auto-indexing; right: manual-indexing
In the tests I added a different random number to all individual elements in a 3D array. I found that auto-indexing is 1.2 to 1.5 times slower than manual-indexing. I also found that the performance of auto-indexing is much more dependent on the size the dimensions: an array with 1000x200x40 elements is 20% slower than an array with 1000x40x200 elements. For manual-indexing there is hardly any difference. The detailed results can be found at the end of this post.
I was under the impression that auto-indexing was the preferred method for this kind of operation: it achieves exactly the same result and it is much clearer in the block diagram. I also expected that auto-indexing would have been optimized in LabView, but the the tests show this is clearly not the case.
What is auto-indexing doing?
With two tests I looked at how auto-index works.
First, I looked if auto-indexing reorganizes the data in an inefficient way. To do this I made a method "fake-auto-indexing" which calls "Array subset" and "Reshape array" (or "Index array" for a 1D-array) when it enters _every_ loop and calls "Replace array subset" when exiting _every_ loop (as opposed to manual-indexing, where I do this only in the inner loop). I replicated this in a test (calling it fake-auto-indexing) and found that the performance is very similar to auto-indexing, especially looking at the trend for the different array lengths.
Fake-auto-indexing
Second, I looked if Locality of reference (how the 3D array is stored in memory and how efficiently you can iterate over that) may be an issue. Auto-indexing loops over pages-rows-columns (i.e. pages in the outer for-loop, rows in the middle for-loop, columns in the inner for-loop). This can't be changed for auto-indexing, but I did change it for manual and fake-indexing. The performance of manual-indexing is now similar to auto-indexing, except for two cases that I can't explain. Fake-auto-indexing performs way worse in all cases.
It seems that the performance problem with auto-indexing is due to inefficient data organization.
Other tests
I did the same tests for 1D and 2D arrays. For 1D arrays the three methods perform identical. For 2D arrays auto-indexing is 15% slower than manual-indexing, while fake-auto-indexing is 8% slower than manual-indexing. I find it interesting that auto-indexing is the slowest of the three methods.
Finally, I tested the performance of operating on entire columns (instead of every single element) of a 3D array. In all cases it is a lot faster than iterating over individual elements. Auto-indexing is more than 1.8 to 3.4 times slower than manual-indexing, while fake-auto-indexing is about 1.5 to 2.7 times slower. Because of the number of iterations that has to be done, the effect of the size of the column is important: an array with 1000x200x40 elements is in all cases much slower than an array with 1000x40x200 elements.
Discussion & conclusions
In all the cases I tested, auto-indexing is significantly slower than manual-indexing. Because auto-indexing is the standard way of indexing arrays in LabView I expected the method to be highly optimized. Judging by the tests I did, that is not the case. I find this puzzling.
Does anybody know any best practices when it comes to working with >1D arrays? It seems there is a lack of documentation about the performance, surprising given the significant differences I found.
It is of course possible that I made mistakes. I tried to keep the code as straightforward as possible to minimize that risk. Still, I hope somebody can double-check the work I did.
Results
I ran the tests on a computer with a i5-4570 @ 3.20 GHz processor (4 cores, but only 1 is used), 8 GB RAM running Windows 7 64-bit and LabView 2013 32-bit. The tests were averaged 10 times. The results are in milliseconds.
3D-arrays, iterate pages-rows-columns
pages x rows x cols : auto manual fake
40 x 200 x 1000 : 268.9 202.0 268.8
40 x 1000 x 200 : 276.9 204.1 263.8
200 x 40 x 1000 : 264.6 202.8 260.6
200 x 1000 x 40 : 306.9 205.0 300.0
1000 x 40 x 200 : 253.7 203.1 259.3
1000 x 200 x 40 : 307.2 206.2 288.5
100 x 100 x 100 : 36.2 25.7 33.9
3D-arrays, iterate columns-rows-pages
pages x rows x cols : manual fake
40 x 200 x 1000 : 277.6 457
40 x 1000 x 200 : 291.6 461.5
200 x 40 x 1000 : 277.4 431.9
200 x 1000 x 40 : 452.5 572.1
1000 x 40 x 200 : 298.1 460.4
1000 x 200 x 40 : 460.5 583.8
100 x 100 x 100 : 31.7 51.9
2D-arrays, iterate rows-columns
rows x cols : auto manual fake
200 x 20000 : 123.5 106.1 113.2
20000 x 200 : 119.5 106.1 115.0
10000 x 10000 : 3080.25 2659.65 2835.1
1D-arrays, iterate over columns
cols : auto manual fake
500000 : 11.5 11.8 11.6
3D-arrays, iterate pages-rows, operate on columns
pages x rows x cols : auto manual fake
40 x 200 x 1000 : 83.9 23.3 62.9
40 x 1000 x 200 : 89.6 31.9 69.0
200 x 40 x 1000 : 74.3 27.6 62.2
200 x 1000 x 40 : 135.6 76.2 107.1
1000 x 40 x 200 : 75.3 31.2 68.6
1000 x 200 x 40 : 133.6 71.7 108.7
100 x 100 x 100 : 13.0 5.4 9.9
VI's
I attached the VI's I used for testing. "ND_add_random_number.vi" (where N is 1, 2 or 3) is where all the action happens, taking a selector with a method and an array with the N dimensions as input. It returns the result and time in milliseconds. Using "ND_add_random_number_automated_test.vi" I run this for a few different situations (auto/manual/fake-indexing, interchanging the dimensions). The VI's starting with "3D_locality_" are used for the locality tests. The VI's starting with "3D_norows_" are used for the iterations over pages and columns only.
Attachments:
3D_arrays_2.zip 222 KBRobert,
the copy-thing is not specific for auto-indexing. It is common for all tunnels. A tunnel is first of all a unique data space.
This sounds hard, but there is an optimization in the compiler trying to reduce the number of copies the VI will create. This optimization is called "in-placeness".
The in-placeness algorithm checks, if the wire passing data to the is connected to anything else ("branch"). If there is no other connection then the tunnel, chance is high that the tunnel won't create an additional copy.
Speaking of loops, tunnels always copies. The in-placeness algorithm cannot opt that out.
You can do a small test to elaborate on this: Simply wire "0" (or anything less than the array sizy of the input array) to the 'N' terminal.......
Norbert
PS: Auto-indexing is perfect for a "by element" operation of analysis without modification of the element in the array. If you want to modify values, use shift registers and Replace Array Subset.
CEO: What exactly is stopping us from doing this?
Expert: Geometry
Marketing Manager: Just ignore it. -
Index taking too much time... anyway to enhance that??
I am creating an index on clob data column .... it s getting too much time any way to enhance that..
Giving script as below :
CREATE INDEX myindex ON sgvd_doc_ver(DOC_FILE) INDEXTYPE IS CTXSYS.CONTEXT parameters ('sync (every "sysdate+(1/24/60)")');Dom Brooks wrote:
Notably see also "FAQ about Indexing Performance":
http://download.oracle.com/docs/cd/E11882_01/text.112/e16594/aoptim.htm#i1006756
Spot on with the choice of link.
Very large memory also means fewer entries per word, which can make queries more efficient. (Oracle calls this "fragmentation" but only mentions it with respect to updating the context index.
Another detail that helps is to add as many words to the stop-list as possible before you create the index.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
A general reminder about "Forum Etiquette / Reward Points": http://forums.oracle.com/forums/ann.jspa?annID=718
If you never mark your questions as answered people will eventually decide that it's not worth trying to answer you because they will never know whether or not their answer has been of any use, or whether you even bothered to read it.
It is also important to mark answers that you thought helpful - again it lets other people know that you appreciate their help, but it also acts as a pointer for other people when they are researching the same question, moreover it means that when you mark a bad or wrong answer as helpful someone may be prompted to tell you (and the rest of the forum) what's so bad or wrong about the answer you found helpful.
Maybe you are looking for
-
hello i ve got an apogee condenser mic to record my acoustic guitar and singing. in box there was a cable for connection to ipad&iphone. but that cable was not able to plug in to my ipad 4 (retina display 16 gb wi-fi).i need an adapter or an another
-
Different font quality in preview mode
Hello, is it normal that fonts become so bulky in preview mode? I´m using a Webfont in this case (PT Sans Narrow Bold) Especially the button text.
-
Photoshop Elements 13 how to open an image as a layer?
Hello, I recently received as asked Photoshop Elements 13 for Christmas, and have been using Gimp for about a year now as a substitute. However, when I went to start a project, I couldnt figure out how to open an image as a layer. Any help would be a
-
Ellipsis Jetpack won't connect and now won't even turn on
I have been having a problem with my Toshiba running Vista , connecting to my Ellipsis jetpack. I called customer service and while the rep was very courteous and helpful, she stated the problem was a security issue with my laptop. I took several mea
-
Hi, I'm a fairly new user of Oracle XMLDB 9.2.0.4 so please bear with me. I registered a schema and inserted about 130k rows into an XMLType table. Retrieval time took a severe hit as expected, but I created some indexes and it seemed to solve the pr