Context Index Optimization

hi ,
This is w.r.to the "maxtime" parameter used in the Context Index Optimization.
As per optimization norms Oracle should optimize the index until the number of minutes specified in the "maxtime" parameter.
For e:g
ctx_ddl.optimize_index('item_ctxdesc', 'FULL', 30);
Here Oracle should optimize the index for 30 minutes as specified above. But it runs for arround 4 hours and sometimes 5 hours.
Could anyone please clarify or explain on this ?
thanx and regards
Naresh
Oracle DBA

This is certification forum. Post your query under:
Database - General
General Database Discussions

Similar Messages

Oracle Text Context index keeps growing. Optimize seems not to be working

Hi,
In my application I needed to search through many varchar columns from differents tables.
So I created a materialized view in which I concatenate those columns, since they exceed the 4000 characters I merged them concatenating the columns with the TO_CLOBS(column1) || TO_CLOB(column)... || TO_CLOB(columnN).
The query is complex, so the refresh is complete on demand for the view. We refresh it every 2 minutes.
The CONTEXT index is created with the sync on commit parameter.
The index then is synchronized every two minutes.
But when we run the optimize index it does not defrag the index. So it keeps growing.
Any idea ?
Thanks, and sorry for my poor english.
Edited by: detryo on 14-mar-2011 11:06

What are you using to determine that the index is fragmented? Can you post a reproducible test case? Please see my test of what you described below, showing that the optimization does defragment the index.
SCOTT@orcl_11gR2> -- table:
SCOTT@orcl_11gR2> create table test_tab
2    (col1 varchar2 (10),
3      col2 varchar2 (10))
4 /
Table created.
SCOTT@orcl_11gR2> -- materialized view:
SCOTT@orcl_11gR2> create materialized view test_mv3
2 as
3 select to_clob (col1) || to_clob (col2) clob_col
4 from   test_tab
5 /
Materialized view created.
SCOTT@orcl_11gR2> -- index with sync(on commit):
SCOTT@orcl_11gR2> create index test_idx
2 on test_mv3 (clob_col)
3 indextype is ctxsys.context
4 parameters ('sync (on commit)')
5 /
Index created.
SCOTT@orcl_11gR2> -- inserts, commits, refreshes:
SCOTT@orcl_11gR2> insert into test_tab values ('a', 'b')
2 /
1 row created.
SCOTT@orcl_11gR2> commit
2 /
Commit complete.
SCOTT@orcl_11gR2> exec dbms_mview.refresh ('TEST_MV3')
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> insert into test_tab values ('c a', 'b d')
2 /
1 row created.
SCOTT@orcl_11gR2> commit
2 /
Commit complete.
SCOTT@orcl_11gR2> exec dbms_mview.refresh ('TEST_MV3')
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> -- query works:
SCOTT@orcl_11gR2> select * from test_mv3
2 where contains (clob_col, 'ab') > 0
3 /
CLOB_COL
ab
c ab d
2 rows selected.
SCOTT@orcl_11gR2> -- fragmented index:
SCOTT@orcl_11gR2> column token_text format a15
SCOTT@orcl_11gR2> select token_text, token_first, token_last, token_count
2 from   dr$test_idx$i
3 /
TOKEN_TEXT      TOKEN_FIRST TOKEN_LAST TOKEN_COUNT
AB                        1          1           1
AB                        2          3           2
C                         3          3           1
3 rows selected.
SCOTT@orcl_11gR2> -- optimizatino:
SCOTT@orcl_11gR2> exec ctx_ddl.optimize_index ('TEST_IDX', 'REBUILD')
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> -- defragmented index after optimization:
SCOTT@orcl_11gR2> select token_text, token_first, token_last, token_count
2 from   dr$test_idx$i
3 /
TOKEN_TEXT      TOKEN_FIRST TOKEN_LAST TOKEN_COUNT
AB                        2          3           2
C                         3          3           1
2 rows selected.
SCOTT@orcl_11gR2>

Use CONTEXT index on mview or use mview's rewrite directly

Please let me explain what I mean in Subject. I have 9 tables. Each of these tables has about 40,000 rows and one table has 2 million rows. Using first approach, I can build a join-only materialized view on top of nine table's mview log. then query on these nine tables directly. Advantage for doing that is use rewrite.
<p>
for second approach, I build a CONETXT index on several columns on the mview, and then query through CONTEXT index on mview directly. This is pretty much like Barbara did on CREATE INDEX book_idx ON book_search
[http]Indexing multiple columns of multiple tables using CTXCAT
but she used CTXCAT instead of CONTEXT index.
<p>
My question is will second approach better than first one and why. Unlike basic join several tables which gives you predictable performance, I often feel that CONTEXT index performance is unpredictable when tables have more than several thousands rows (maybe I did something wrong, but still looking for document regarding performance) .
<p>
I will appreciate someone could show hints on the issue.
<p>
Message was edited by:
qwe15933
Message was edited by:
qwe15933
Message was edited by:
qwe15933
Message was edited by:
qwe15933

The best method to find out what is best for any individual situation is to test and compare. In general, Oracle Text is best at searching unstructured data, such as large documents and has a lot of features that enable you to do different kinds of searches. If you only have structured data, with each column only containing one short thing and you only need simple searches, then you probably don't need Text. There are also a few things that can be done to indexes and/or queries to optimize performance. It would help to have an idea what your typical data is like and what sorts of searches you anticipate needing.

Context Index Peformance

Hi,
I want to create a context index on one column which contains large text. And the table contains millions of records and daily inserts happen into the same table. My question is
1.Do we need to run any procedures after inserting the records daily i.e synchronization??
2.Is there any problem from performace point of view creating context index on the table??
Thanks,
Sri

hi,
pls have a take a look this doc
http://docs.oracle.com/cd/B19306_01/text.102/b14217/ind.htm
http://stackoverflow.com/questions/1916244/how-to-sync-and-optimize-an-oracle-text-index
regards,

Suggestion: Oracle text CONTEXT index on one or more columns ?

Hi,
I'm implementing Oracle text using CONTEXT ..... and would like to ask you for performance suggestion ...
I have a table of Articles .... with columns .. TITLE, SUBTITLE , BODY ...
Now is it better from performance point of view to move all three columns into one dummy column ... with name like FULLTEXT ... and put index on this single column,
and then use CONTAINS(FULLTEXT,'...')>0
Or is it almost the same for oracle if i put indexes on all three columns and then call:
CONTAINS(TITLE,'...')>0 OR CONTAINS(SUBTITLE,'...')>0 OR CONTAINS(BODY,'...')>0
I actually don't care if the result is a match in TITLE OR SUBTITLE OR BODY ....
So if i move into some FULLTEXT column, then i have duplicate data in a article row ... but if i create indexes for each column, than oracle has 2x more to index,optimize and search ... am I wright ?
Table has 1.8mil records ...
Thank you.
Kris

mackrispi wrote:
Now is it better from performance point of view to move all three columns into one dummy column ... with name like FULLTEXT ... and put index on this single column,
and then use CONTAINS(FULLTEXT,'...')>0What version of Oracle are you on? If 11 then you could use a virtual column to do this, otherwise you'd have to write code to maintain the column which can get messy.
mackrispi wrote:
Or is it almost the same for oracle if i put indexes on all three columns and then call:
CONTAINS(TITLE,'...')>0 OR CONTAINS(SUBTITLE,'...')>0 OR CONTAINS(BODY,'...')>0Benchmark it and find out :)
Another option would be something like this.
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:9455353124561
Were i you, i would try out those 3 approaches and see which meet your performance requirements and weigh that with the ease of implementation and administration.

Substring search with Oracle context indexes

Hi,
i would like to know if it is possibile to do a substring search with one of the obtion offer with the context indexes.
(ctxcat,ctxrule,context)
example:
i would like to search the word 'berub' in a column A in table_example.
the value in the column a are :
The betther
berube
A.berube
berub
Berub
BERUB
R berube
S tartif
Y Thibeault
the rows return should be :
berube
A.berube
berub
Berub
BERUB
R berube
A simple sql could be
select * from table_example where upper(a) like upper('%berub%' );
How i can do this same action with the context indexes and a select (catsearch, contains, matches), if it is possible?
A example will be welcome
Thanks

I know how to do explain plan.
my point is not the query i post, it's just a example.
I have many query on my production we optimize many times (they past from 3min to 15 sec with optimisation, but we want to have better result). At this point we are looking to implant the context indexes to make them more efficient.
Do make this sql more efficient we have to deal with like '%xxxxxx%' and the context indexes like to be a option, but we have to be able to do some substring search with context option.
Is it possible to do it and how?
This is my question and why i post it here. The query is just a simple example to illsutrate what i want.
Thanks to anyone who can answer my question.

Portal Context Index Creation Performance issue

Recreating Portal Context Indexes takes around 36 hours at our site (after portal upgrade from 3.0.9.8.2 to 3.0.9.8.5 as per release notes). I was following the Note:158368.1 to rebuild the indexes. Is there anything that i can do to tune this ?
thanks
subu

Unfortunately indexing is generally a fairly intensive operation and can be time consuming.
There are some things that you can do to optimize the performance of your database as a whole which may in turn help the performance of your indexing operation. Look at the Performance Guide and Reference book in the database documentation.
Much of the time spent indexing is taken up by filtering binary documents and fetch content identified by URL attributes. In the case of the later, it might be worth checking in the ctx_user_index_errors view to ensure that you don't have a lot of URL requests that are timing out. The timeout is set to 30 seconds and if there are a lot or URLs where the host cannot be resolved or the fetch times out it might be costing a lot of time during the indexing operation. This is often the case if a proxy is required to reach the URLs but the proxy has not be configured correctly.

Optimizing context index

I have created a context index and want to optimize it regularly. I issue the following statement:
ctx_ddl.optimize_index('MY_INDEX','FAST');
In the Enterprise Manager I can see, that there are a lot of Sort/Merge-operations, but the optimize process doesn't finish.
(During optimize there are inserts into the table and the index is updated by the ctx_ddl.sync_index procedure).
Can anyone tell me why the optimize doesn't finish? How can I avoid this situation?

How long did you leave it running?Optimizing ran more than 10 hours. So I cancelled it, because this is not acceptable.
Optimization is an intensive process, and can sometimes take longer than the original index creation.
If you want it to complete in less time, consider using FULL optimize, but setting a time limit on it. That way it will do as much as it can within the time limit, and then start again next time from where it finished last time.During the night, I start a FULL optimize with MAXTIME = 60 minutes. But this seems not to be enough. If I don't start FAST optimizing during the day, the index get to much fragmentated.
Alternatively, if optimization is taking a lot longer than creating the index, consider dropping the index and recreating it from scratch. This will require downtime on the system, unless you get clever and use the USER_DATASTORE to create two indexes on different, dummy, columns - and switch searching between them when you want to rebuild the index. Make sure you use a generous setting for INDEX_MEMORY to avoid fragmentation.
- Roger

ORACLE TEXT Context index so big, why ?

Hi,
One year ago, I've created a context index, for a table that had 2 million rows , and the context index was around 2.8 Mil rows ....
Today we have 2.9mil rows but the Context index has 95 Mil rows ??
Last weekend I did a FULL optimize index and it took 28hours and it was still not finished .... so this weekend I droped the index and re-created
it .... it took 6 hours ....
Now I started CTX_REPORT.INDEX_STATS to see how it looks like .... cause I exprected 95mil to become like 3.5mil max not more ....
Can anyone please tell me what should I check to see why did this happen so that the last 900.000 records created 91 mil rows in index ?
If there would be something wrong, why would the last 900.000 rows create something that prev 2mil rows did not ...
HEre is index creation :
CREATE INDEX ART_IDX ON MS_ARTICLE(ORATEXT) INDEXTYPE IS CTXSYS.CONTEXT FILTER BY ID,ART_DATE,MEDIA_CODE ORDER BY
ART_DATE DESC PARAMETERS (' LEXER ART_LEX STOPLIST CTXSYS.EMPTY_STOPLIST sync(ON COMMIT) DATASTORE DS_ART');If anyone needs definition of lexer od datastore, then please tell me what do I have to do to get definitions of that.
Thank you.
KRis

Hi,
Ok there must be something I did wrong .... cause after running INDEX_STATS ...I got this results :
===========================================
                    STATISTICS FOR "PRESCLIP"."ART_IDX"
===========================================
indexed documents:                                              2,876,228
allocated docids:                                               2,876,228
$I rows:                                                       96,110,561
                             TOKEN STATISTICS
unique tokens:                                                  5,584,188
average $I rows per token:                                          17.21
tokens with most $I rows:
JE (0:TEXT)                                                      11,650
V (0:TEXT)                                                       11,183
IN (0:TEXT)                                                      10,085So I have expected Average rows per token to be 1 after droping and re-creating index ..... so what else must I do to get the number 96mil to 5.5mil ?
Thank you.
Kris

Creation of context index on index-organized table

I encountered a problem when creating a domain index(intermediate text context index) on a index-organised table in oracle 8i.
The description of the error is stated below:
"ORA-29866: cannot create domain index on a column of index-organized table "
I have configured intermediate text properly and even it worked for those tables which are not index-organised(ordinary tables).
This problem has occured only when i made the tables as index organised.
Please provide us a solution to this problem as early as possible.
In case if you require any more details i shall provide them.

Please ask questions about Oracle Text (formerly interMedia text) in the Oracle Text forum. You will get a quicker, more expert answer there.

Creation of context index on index-organized tables

I encountered a problem when creating a domain index(intermediate text context index) on a index-organised table in oracle 8i.
The description of the error is stated below:
"ORA-29866: cannot create domain index on a column of index-organized table "
I have configured intermediate text properly and even it worked for those tables which are not index-organised(ordinary tables).
This problem has occured only when i made the tables as index organised.
Please provide us a solution to this problem as early as possible.
In case if you require any more details i shall provide them.

creation of domain indexes (such as context) on iot's
is not currently supported in oracle.

Error in Updating ctxsys.context index using ctxsrv

Environment :
Linux 6.2,
Oracle 8.1.6
character set : zhs16gbk or zhs16cgb231280
I can create an ctxsys.context index(using CHINESE_VGRAM_LEXER) on a long column, and i can rebuild this index with no parameters.
When i want to rebuild this index automatic using ctxsrv, this error occur ---
DRG-50857 Oracle error in drtldml(dml)
Ora-2000 error
anybody can help me?
null

the complete error:
Dml batch: cid=1023
End batch: return code is 1
DRG-50857: oracle error in drtldml(dml)
ORA_20000: interMedia text 4mNs
DRG-50857: oracle error in drsxsopen
ORA-01480: STR83V51dA?H1IY?U:sW:
null

CONTEXT index vs a CTXCAT index

I'm puzzled as to which of these indexes is meant for what purpose. I like the auto-update feature of the CTXCAT index, so I'm leaning that direction.
Here's our situation. We have a table with about 2 million rows, where there's an EMPLOYER_NAME field. Users are constantly executing queries like:
where EMPLOYER_NAME like '%CHEVRON%'The CTXCAT index is also described as "Typically, with this index type, you index small documents or text fragments". I would consider this EMPLOYER_NAME column (VARCHAR2(50)) to fit this description, more or less.
The main thing that has me confused is that our vendor is pushing for a CONTEXT index, as they've got experience setting these up. We're not storing "text consists of large coherent documents.", but we are storing plain text.
Could someone offer some guidance? Thanks for your help,
--=Chuck

Depends on whether the words are predictable. You indicated in your first post they are employer names. If they are then presumably there is a table/list of valid values. Otherwise a CTX index may be your best shot but understand that you will be indexing every word and, I assume, most will be irrelevant.

Performance of context index with sorting

Dear All,
I've got a problem and don't know how to solve this.
there has a table which have a XMLTYPE field to store the unstructred xml, and created with context index.
When I try to select a record from it by using contains (res, '[searchingfield]')>0, the response time is quick, but when I try to order by another field which in the same table, the response time is drop down slightly. (ex. select id, path, res, update_date from testingtbl where contains(res, 'shopper')>0 order by update_date desc.
Actually there is a context index build for field 'res', any other index build for field 'update_date', when sql without 'order by update_date', the context index will use, but the update_date index will not be used even have ordering criteria.
Is there any expect can tell how to solve this? how to keep the performance even doing the sorting process?
Thanks and Regards
Raymond

Thanks for your quick reply.
The mentions information provide after back to office, actually I just want to know if there is any method(s) which can use the context index (with contains keyword) and sorting without slow down the performance.
Thanks and Regards
Raymond

Context index and contains operator syntax how it works ?

Hi
I create a context index on four collumns (text_prof, text_gest, text_citizen, text)
of the same table content.
When i have more than one collumn being queryed using the contains syntax, oracle display the ora 29907 error saying found duplicated labels in primary invocations .
This query works:
SELECT * FROM content WHERE cod_type = '1'
AND (UPPER(title) LIKE UPPER('%tabagismo%')
OR contains (text, 'tabagismo',1)>0
This not works:
SELECT * FROM content
WHERE cod_type = '1' AND (
UPPER(title) LIKE UPPER('%tabagismo%')
OR contains (text, 'tabagismo',1)>0
OR contains (text_citizen,'tabagismo',1)>0
OR contains (text_gest,'tabagismo',1)>0
OR contains (text_prof,'tabagismo',1)>0
How can i fix it ?
I need to query all these colluns !
Does the contains operator can be used only in one collumn?
Thank´s in advance

Hi
I create a context index on four collumns (text_prof, text_gest, text_citizen, text)
of the same table content.
When i have more than one collumn being queryed using the contains syntax, oracle display the ora 29907 error saying found duplicated labels in primary invocations .
This query works:
SELECT * FROM content WHERE cod_type = '1'
AND (UPPER(title) LIKE UPPER('%tabagismo%')
OR contains (text, 'tabagismo',1)>0
This not works:
SELECT * FROM content
WHERE cod_type = '1' AND (
UPPER(title) LIKE UPPER('%tabagismo%')
OR contains (text, 'tabagismo',1)>0
OR contains (text_citizen,'tabagismo',1)>0
OR contains (text_gest,'tabagismo',1)>0
OR contains (text_prof,'tabagismo',1)>0
How can i fix it ?
I need to query all these colluns !
Does the contains operator can be used only in one collumn?
Thank´s in advance

Context Index Optimization

Similar Messages

Maybe you are looking for