Oracle text catsearch sub index query

Hello,
I wonder if you can help me with a query about Oracle Text Catsearch.
I have a database which has 10Gb of data.
There is a text column in the database on which I have to find a partial match on the data contained in it
I have indexed this column with a CTXSYS.CTXCAT index.
In addition I have added a sub index to the index set for a Date Field and ran EXEC DBMS_STATS.GATHER_TABLE_STATS to make sure the query execution path is optimised
Here's my Question:
How can I make sure that the Date sub query always runs before the finding the Partial Match on the text column?
Caveat I am a programmer not a DBA, but I've ended up doing some databasey type stuff, apologies if question is thick.
Cheers
Mark
p.s. Performance is good, but I have a feeling that the Date subquery is not being used as efficiently as it should be (the subquery should massively reduce the result set to be searched for the partial match)

You can't - ctxcat doesn't support the "functional invocation" which would be needed if another index is used first. So reducing the set of docs to index doesn't help.
If you can find a way to denormalize the information used in the sub-query such that it can be included in the main query index set, that should help performance considerably.

Similar Messages

Oracle Text - CTX Context Index Soundex Problem

Hi,
I'm running into a problem with Oracle Text when searching using the ! (soundex) option. I've created a simple test example to highlight the issue.
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit
Windows 2008 Server 64-bit
create table test_tab (test_col varchar2(200));
insert all
into test_tab (test_col) values ('ab-tönes')
into test_tab (test_col) values ('ab-tones')
into test_tab (test_col) values ('abtones')
into test_tab (test_col) values ('ab tones')
into test_tab (test_col) values ('ab-tanes')
select * from dual
select * from test_tab
begin
      ctx_ddl.create_preference ('test_lex1', 'basic_lexer');
      ctx_ddl.set_attribute ('test_lex1', 'whitespace', '/\|-_+&''');
      ctx_ddl.set_attribute('test_lex1','base_letter','YES');
      -- ctx_ddl.set_attribute('test_lex1','skipjoins','-');
end;
create index test_idx on test_tab (test_col)
indextype is ctxsys.context
    parameters
      ('lexer        test_lex1'
select token_text from dr$test_idx$i;
TOKEN_TEXT
AB
ABTONES
TANES
TONES
select * from test_tab where contains (test_col, '!ab tones') > 0;
TEST_COL
ab-tönes
ab-tones
ab tones
select * from test_tab where soundex(test_col) = soundex('ab tones');
TEST_COL
ab-tönes
ab-tones
abtones
ab tones
ab-tanes
So my question is, can anyone suggest an approach whereby I can get the Oracle Text Context index (or CTXCAT index if it's more appropriate) to return all 5 rows like the simple Soundex is doing?
I can't really use soundex as this search query will form part of a search screen for a multi-language application. Soundex is limited to English sounding words, so I need the solution to be able to compare strings that may not "sound" English.
It must be an attribute of the BASIC_LEXER, and I've tried skipjoins, start/end-joins, stop lists, but I just cannot get the Soundex feature of Oracle Text to function like the SOUNDEX() function!
Looking at how the tokens are stored dr$test_idx$i I need Oracle Text to almost concat 'AB' and 'TONES' to search as a single string.
Any help greatly appreciated.
Thanks,

I am not getting the same problem that you are getting with the umlat, but I don't see what is different. Please post the result of:
select ctx_report.create_index_script ('test_idx') from dual;
Here are the results on my system. Perhaps you can spot the difference. I added an empty_stoplist, so that it won't print out a long list of stopwords.
SCOTT@orcl12c> create table test_tab (test_col    varchar2(200))
2 /
Table created.
SCOTT@orcl12c> insert all
2    into test_tab (test_col) values ('ab-tönes')
3    into test_tab (test_col) values ('ab-tones')
4    into test_tab (test_col) values ('abtones')
5    into test_tab (test_col) values ('ab tones')
6    into test_tab (test_col) values ('ab-tanes')
7 select * from dual
8 /
5 rows created.
SCOTT@orcl12c> select * from test_tab
2 /
TEST_COL
ab-tönes
ab-tones
abtones
ab tones
ab-tanes
5 rows selected.
SCOTT@orcl12c> begin
2    ctx_ddl.create_preference ('test_lex1', 'basic_lexer');
3    ctx_ddl.set_attribute('test_lex1','base_letter','YES');
4 end;
5 /
PL/SQL procedure successfully completed.
SCOTT@orcl12c> create or replace procedure test_proc
2    (p_rowid in          rowid,
3      p_clob    in out nocopy clob)
4 as
5 begin
6    select replace (translate (test_col, '/\|-_+&''', '      '), ' ', '')
7    into   p_clob
8    from   test_tab
9    where rowid = p_rowid;
10 end test_proc;
11 /
Procedure created.
SCOTT@orcl12c> show errors
No errors.
SCOTT@orcl12c> begin
2    ctx_ddl.create_preference ('test_ds', 'user_datastore');
3    ctx_ddl.set_attribute ('test_ds', 'procedure', 'test_proc');
4 end;
5 /
PL/SQL procedure successfully completed.
SCOTT@orcl12c> create index test_idx on test_tab (test_col)
2    indextype is ctxsys.context
3    parameters
4       ('lexer    test_lex1
5         datastore    test_ds
6         stoplist    ctxsys.empty_stoplist')
7 /
Index created.
SCOTT@orcl12c> select token_text from dr$test_idx$i
2 /
TOKEN_TEXT
ABTANES
ABTONES
2 rows selected.
SCOTT@orcl12c> variable search_string varchar2(100)
SCOTT@orcl12c> exec :search_string := 'ab tones'
PL/SQL procedure successfully completed.
SCOTT@orcl12c> select * from test_tab
2 where contains
3            (test_col,
4             '!' || replace (:search_string, ' ', ' !') ||
5             ' or !' || replace (:search_string, ' ', '')) > 0
6 /
TEST_COL
ab-tönes
ab-tones
abtones
ab tones
ab-tanes
5 rows selected.
SCOTT@orcl12c> exec :search_string := 'abtones'
PL/SQL procedure successfully completed.
SCOTT@orcl12c> /
TEST_COL
ab-tönes
ab-tones
abtones
ab tones
ab-tanes
5 rows selected.
SCOTT@orcl12c> exec :search_string := 'ab tönes'
PL/SQL procedure successfully completed.
SCOTT@orcl12c> /
TEST_COL
ab-tönes
ab-tones
abtones
ab tones
ab-tanes
5 rows selected.
SCOTT@orcl12c> select ctx_report.create_index_script ('test_idx') from dual
2 /
CTX_REPORT.CREATE_INDEX_SCRIPT('TEST_IDX')
begin
ctx_ddl.create_preference('"TEST_IDX_DST"','USER_DATASTORE');
ctx_ddl.set_attribute('"TEST_IDX_DST"','PROCEDURE','"SCOTT"."TEST_PROC"');
end;
begin
ctx_ddl.create_preference('"TEST_IDX_FIL"','NULL_FILTER');
end;
begin
ctx_ddl.create_section_group('"TEST_IDX_SGP"','NULL_SECTION_GROUP');
end;
begin
ctx_ddl.create_preference('"TEST_IDX_LEX"','BASIC_LEXER');
ctx_ddl.set_attribute('"TEST_IDX_LEX"','BASE_LETTER','YES');
end;
begin
ctx_ddl.create_preference('"TEST_IDX_WDL"','BASIC_WORDLIST');
ctx_ddl.set_attribute('"TEST_IDX_WDL"','STEMMER','ENGLISH');
ctx_ddl.set_attribute('"TEST_IDX_WDL"','FUZZY_MATCH','GENERIC');
end;
begin
ctx_ddl.create_stoplist('"TEST_IDX_SPL"','BASIC_STOPLIST');
end;
begin
ctx_ddl.create_preference('"TEST_IDX_STO"','BASIC_STORAGE');
ctx_ddl.set_attribute('"TEST_IDX_STO"','R_TABLE_CLAUSE','lob (data) store as (
cache)');
ctx_ddl.set_attribute('"TEST_IDX_STO"','I_INDEX_CLAUSE','compress 2');
end;
begin
ctx_output.start_log('TEST_IDX_LOG');
end;
create index "SCOTT"."TEST_IDX"
on "SCOTT"."TEST_TAB"
      ("TEST_COL")
indextype is ctxsys.context
parameters('
    datastore       "TEST_IDX_DST"
    filter          "TEST_IDX_FIL"
    section group   "TEST_IDX_SGP"
    lexer           "TEST_IDX_LEX"
    wordlist        "TEST_IDX_WDL"
    stoplist        "TEST_IDX_SPL"
    storage         "TEST_IDX_STO"
begin
ctx_output.end_log;
end;
1 row selected.

Oracle Text location of Indexes

I am creating an Oracle Text index as per example given in documentation. I have set my storage preferences to point all the created tables / indexes to a given tablespace. This works fine, however I still get one index (domain) being created in the SYSTEM tablespace. Is this normal ? Can it be moved ?

The command used is
create index abstract_text_idx on abstracts(merge_text)
indextype is ctxsys.context
parameters('lexer abstract_lexer storage abstract_store memory 52428800')
The index left in SYSTEM tablespace is ABSTRACT_TEXT_IDX. All the DR$$ indexes have used the storage parameters in the abstract_store preference O.K.

Oracle text - issue with contains query

Hello,
Need urgent help.
Following code in my procedure is giving me error.
TYPE c_1 is ref cursor;
result_cursor c1;
i_text2 := 'NEW%';
open result_cursor for
'select /*+ INDEX_SS_DESC(e cad_addr_idx2 )*/
from cad_address
where
contains(text, {:i_text2}, 1) > 0
and rec_type in (1,2,3,4)
order by occur_count desc'
using
i_text2;
ORA-00936: missing expression
ORA-06512: at "AV_OWNER.MY_PROC", line 43
ORA-06512: at line 6
Oracle version is 11.2.0.3.0.
Thanks,

check your table is 'text indexed' on this 'Text' column.To knoow more about 'text index' go to
http://docs.oracle.com/cd/B19306_01/text.102/b14217/ind.htm
Also refer to the below thread where someone had faced issues with CONTAINS clause.
ORA-20000: Oracle Text error: DRG-10599: column is not indexed

Oracle Text, multi_column_datastore describe index

Referring to Re: Oracle Text, ctxsys.context problem with number column
In TOAD or SQL Navigator you can click on some object (like table, index) and click on tab "script" where is generated the code to create it.
Assuming that I am a person who sees the index for the first time where/how I can find multi_columns/parameters with which the index was created ?
For example:
exec ctx_ddl.drop_preference( 'myds' );
exec ctx_ddl.create_preference( 'myds', 'MULTI_COLUMN_DATASTORE' );
exec ctx_ddl.set_attribute( 'myds', 'COLUMNS', 'item_barcode, item_title, item_subtitle' );
CREATE INDEX i_index_test
   ON item (item_title)
   INDEXTYPE IS ctxsys.context
   PARAMETERS ( 'datastore myds' );And script says:
CREATE INDEX PLSQL.I_INDEX_TEST ON PLSQL.ITEM
(ITEM_TITLE)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('datastore myds')
NOPARALLEL;
Where I can find datastore "myds" ?

You can use ctx_report to get all the information about an index.
I use it from SQL*Plus like this:
set pagesize 0
set heading off
set trimspool on
set long 500000
spool index.sql
select ctx_report.create_index_script('MyIndexName') from dual;
spool offThe only thing this won't give you is the source for any procedures used in the user_datastore. You can get that from user_sources.

Oracle Text - procedure refreshing index doesn't run

Hi,
I'm experiencing problems refreshing an index using a procedure (see below). When running the command instead of the procedure everything is fine.
Reading through this forum I found that ctxapp role is needed, which I have. Since this didn't work the admin granted execute privileges on any procedures and programmes (for a short test). Even that didn't help.
The admin can run the procedure without any problems.
Help appreciated
Franziska
---The procedure---
create or replace procedure indizes_erneuern
as
begin
ctx_ddl.optimize_index('ind_nachname_assistent', 'REBUILD');
end;---The error message
Connecting to the database FAM.
ORA-20000: Oracle Text error:
ORA-01031: insufficient privileges
ORA-06512: at "CTXSYS.DRUE", line 160
ORA-06512: at "CTXSYS.CTX_DDL", line 630
ORA-06512: at "FAM.INDIZES_ERNEUERN", line 4
ORA-06512: at line 2
Process exited.
Disconnecting from the database FAM.

When running the command instead of the procedureYou'll have to either issue direct grants or define the procedure with AUTHID CURRENT_USER.

Problem with blob column index created using Oracle Text.

Hi,
I'm running Oracle Database 10g 10.2.0.1.0 standard edition one, on windows server 2003 R2 x64.
I have a table with a blob column which contains pdf document.
Then, I create an index using the following script so that I can do fulltext search using Oracle Text.
CREATE INDEX DMCS.T_DMCS_FILE_DF_FILE_IDX ON DMCS.T_DMCS_FILE
(DF_FILE)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('DATASTORE CTXSYS.DEFAULT_DATASTORE');
However, the index is not searchable and I check the following tables created by database for my index and found them to be empty as well !!
DR$T_DMCS_FILE_DF_FILE_IDX$I
DR$T_DMCS_FILE_DF_FILE_IDX$K
DR$T_DMCS_FILE_DF_FILE_IDX$N
DR$T_DMCS_FILE_DF_FILE_IDX$R
I wonder what's wrong with it.
My user has been granted the ctx_app role and I have other tables that store plain text which I use Oracle Text are fine. I even output the blob column and save as pdf file and they are fine.
However the database seems like not indexing my blob column although the index can be created without error.
Please advise.
Really appreciate anyone who can help.
Thank you.

The situation is I have already loaded a few pdf document into the table's blob column.
After I create the Oracle text index on this blob column, I find the system generated index tables listed in my earlier posting are empty, except for the 4th table.
Normally we'll see words inside the table where those are the words indexed by oracle text on my document.
As a result, no matter how i search for the index using select statement with contains operator, it will not give me any result.
I feel weird why the blob is not indexed. The content of the blob are actually valid because I tested this by export the content back to pdf and I can still view and search within the pdf.
Regards,
Jap.

Problem creating Oracle text index

Hi,
I am trying to create an index in Oracle 9i using Oracle Text.
First i gave this grant as SYSDBA:
"GRANT ALL ON CTX_DDL TO <USERNAME>"
Then i executed the following :
EXECUTE CTX_DDL.CREATE_SECTION_GROUP('MYPATHGROUP','PATH_SECTION_GROUP');
CREATE INDEX SDS_SLIDE_XML_IDX ON SDS_SLIDE_DATA (SLIDE_XML)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('SECTION GROUP MYPATHGROUP');
but I got the following error :
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000: Oracle Text error:
DRG-50857: oracle error in dricon.get_primary_key
ORA-00980: synonym translation is no longer valid
ORA-06512: at "CTXSYS.DRUE", line 157
ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 186
Any ideas?
Rgds
Vikram.

Oracle Text will not index the content inside the portlet of the pages. The portlet is treated as a portlet instance item and only the relevant attributes are searched for, like display name of the portlet, etc.

Oracle Text Help with XML column values

Hello. In addition to being new to Oracle Text, I am inheriting an Oracle Text application and have a couple of questions.
First, A context-based index has been set-up on a CLOB column which contains an XML formatted document. The Auto Section Group parameter has been set to created zones for each tag of the XML document. I have found that when using a browser to display the content of the CLOB, some of the column values have trouble displaying in the browser, where I receive an XML processing error. I believe this is due to the fact that some of the XML document rows contain URLs that are not embedded in the CDATA tag. In any case, if the browser has trouble displaying the XML, will oracle text have trouble indexing the XML and creating the section group zones?
Second, I understand that the NOT operator takes a right operand term and left operand term. Can either of the terms be the results of the WITHIN operator, i.e. "dogs not (cats within animals)".
Thank you.

I bet you just whipped that out, and I thank you with all my
heart, its amazing to me how many ways I tried to do what you did.
Thanks
I have a second question relating to the same problem and
that is in referencing the over state. Currently, I can write
'text' into the text field and see what I have coming in from xml
in its place during the 'up' state.
However, when the timeline hits the 'over' state, the
textfield will display nothing, or 'text' if I have that written
in. I suspect that I am not referencing the'over' state correctly.
Should I add one line of code sort of referencing the text
field and not just the button while in the over state?

Index rules in oracle text and query using matches

Dear All,
I would like to ask about rules and matches function in oracle text.
I followed an example in oracle text application developer's guide.
I have a rule table like this :
1 oracle
2 larry or ellison
3 oracle and text
4 market share
then, I create an index to that table. This is needed for calling matches function. Here is the syntax :
create index queryx on queries(query_string)
indextype is ctxsys.ctxrule;
then, I noticed that the result on DR$QUERYX$I table as follows :
LARRY 0 2 2 1 (BLOB)
MARKET 0 4 4 1 (BLOB) {MARKET} {SHARE}
ORACLE 0 1 1 1 (BLOB)
ORACLE 0 3 3 1 (BLOB) {TEXT}
ELLISON 0 2 2 1 (BLOB)
What I want to ask is why doesn't the words 'share' and 'text' appear in the DR$QUERYX$ table?
When we use matches function, it then search on the index result and consequently it wion't find the 'share' word. so when for example I do query like this :
select query_id from queries where matches(query_string,' It only share ten percent of all products sold')>0
it will give 0 result since the no word in ' It only share ten percent of all products sold' was in index table. But actually it could possibly be categorized as the 4 category which rules is 'market share'
I tried this in a larger set of data and get same result.
Here is my generated rules from my document collection :
1 {REQUIREMENTS} & {ELICITATION}
1 {REQUIREMENTS} ~ {ELICITATION} & {ACTOR}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} & {FURPS}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} & {PROC}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} & {SPEED}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} & {DOCUME}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} ~ {DOCUME} & {PLACED}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} ~ {DOCUME} ~ {PLACED} & {UNNECESSARY}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} ~ {DOCUME} ~ {PLACED} ~ {UNNECESSARY} & {MISUSE}
1 {INTERPRETATION} ~ {REQUIREMENTS}
2 {DESIGN} & {REPRESENTATION}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} & {OCTOBER}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} & {PROCEDURAL}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} & {STRICT}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} ~ {STRICT} & {GRASP}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} ~ {STRICT} ~ {GRASP} & {MANY} & {LAYER}
2 {DESIGN} ~ {REPRESENTATION} ~ {MAY}
3 {PM} & {TESTING} & {ATTRIBUTI}
And this is the index table result with ctxrule :
(only the token_text column shown)
PM
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
INTERPRETATION
so when I try to classify a document with the word ouline inside it, it should produce category 1 (based on the rules) but since there are no word 'outline' in index tabel, the matches will return 0 means that the document is not classifiedto any category. I don't understand why it happen. Anybody knows about this? I would really appreciate any help.
Thank you very much.

Hm, I see. It do make sense. so nice to know.
But then in the second example I gift where I used larger table, as shown below :
Here is my generated rules from my document collection :
1 {REQUIREMENTS} & {ELICITATION}
1 {REQUIREMENTS} ~ {ELICITATION} & {ACTOR}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} & {FURPS}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} & {PROC}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} & {SPEED}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} & {DOCUME}
1 {INTERPRETATION} ~ {REQUIREMENTS}
2 {DESIGN} & {REPRESENTATION}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} & {OCTOBER}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} & {PROCEDURAL}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} & {STRICT}
2 {DESIGN} ~ {REPRESENTATION} ~ {MAY}
3 {PM} & {TESTING} & {ATTRIBUTI}
As far as I know, the sign ' ~ ' means 'OR' and '&' means 'and' . So based on the 4th line in my table :
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE}
it can be concluded that if any of the words stated there been queried, so the category '1' will appear as a result. But then before we can use 'matches' to query it, we need ti create index for the rules table . I did it and the result were :
(only the token_text column shown)
PM
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
INTERPRETATION
there were no words other than PM, DESIGN< REQUIREMENTS and INTERPRETATION. Why the words REQUIREMENTS, ELICITATION, ACTOR, FURPS, OUTLINE don't appear in the index result?

Oracle Text Index Query

Hello
Had a query on oracle Text Index.
1. I have created a table with column ID as sequence & column description as Varchar2(500)
2. Inserted 3 records
3. I created the Oracle Text index on column "description" as INDEXTYPE IS CTXSYS.CONTEXT
4. Run a select using "contains" , get the records as required.
5. I insert another new record & commit;
6. Run the same old query using contains but for the new records I get 0 records
7. Drop the Index and Recreate the index using same old index on column "description" as INDEXTYPE IS CTXSYS.CONTEXT
8. Run the same old query using contains but for the new records and now I get the records as required.
Does that mean every time I insert new records I have to rebuild the Oracle Text Index ?
Your Help will be much appreciated
Regards
Harshad

You need to use some method of updating your index. You can drop and recreate or you can rebuild or you can use ctx_ddl.sync_index or you can include sync(on commit) in your parameters when you create the index or various other methods of periodic synchronization and optimization.

Using multiple sub indexes in a catsearch query

Hi,
I have a ctxcat index on a table with orgname as the indexed column and city and postal code as sub indexes.
I could do a
select *
from xxx_org_search_v
where 1 =1
and catsearch(org_name,'green*','postal_code=''38016''') > 0;
and
select *
from xxx_org_search_v
where 1 =1
and catsearch(org_name,'green*','city=''CORDOVA''') > 0;
but I how do I do this one?
select *
from xxx_org_search_v
where 1 =1
and catsearch(org_name,'green*','postal_code=''38016'' city=''CORDOVA''') > 0;
whats the syntax for acheiving the above?
Thanks
Guru

SCOTT@orcl_11g> CREATE TABLE xxx_org_search_v
2    (org_name     VARCHAR2(15),
3      city          VARCHAR2(15),
4      postal_code VARCHAR2(15))
5 /
Table created.
SCOTT@orcl_11g> INSERT ALL
2 INTO xxx_org_search_v VALUES ('GREEN1', 'CITY1', '38016')
3 INTO xxx_org_search_v VALUES ('GREEN2', 'CORDOVA', 'POST2')
4 INTO xxx_org_search_v VALUES ('GREEN3', 'CITY3', 'POST3')
5 INTO xxx_org_search_v VALUES ('GREEN4', 'CORDOVA', '38016')
6 SELECT * FROM DUAL
7 /
4 rows created.
SCOTT@orcl_11g> BEGIN
2    CTX_DDL.CREATE_INDEX_SET ('iset');
3    CTX_DDL.ADD_INDEX ('iset', 'city');
4    CTX_DDL.ADD_INDEX ('iset', 'postal_code');
5 END;
6 /
PL/SQL procedure successfully completed.
SCOTT@orcl_11g> CREATE INDEX ctxcat_index ON xxx_org_search_v (org_name)
2 INDEXTYPE IS CTXSYS.CTXCAT
3 PARAMETERS ('INDEX SET iset')
4 /
Index created.
SCOTT@orcl_11g> select *
2 from   xxx_org_search_v
3 where catsearch (org_name, 'green*',
4          'postal_code=''38016''') > 0
5 /
ORG_NAME        CITY            POSTAL_CODE
GREEN1          CITY1           38016
GREEN4          CORDOVA         38016
SCOTT@orcl_11g> select *
2 from   xxx_org_search_v
3 where catsearch (org_name, 'green*',
4          'city=''CORDOVA''') > 0
5 /
ORG_NAME        CITY            POSTAL_CODE
GREEN2          CORDOVA         POST2
GREEN4          CORDOVA         38016
SCOTT@orcl_11g> select *
2 from   xxx_org_search_v
3 where catsearch (org_name, 'green*',
4          'postal_code=''38016'' AND city=''CORDOVA''') > 0
5 /
ORG_NAME        CITY            POSTAL_CODE
GREEN4          CORDOVA         38016
SCOTT@orcl_11g>

Performance issue with Oracle Text index

Hi Experts,
We are on Oracle 11.2..0.3 on Solaris 10. I have implemented Oracle Text in our environment and I am facing a strange performance issue that is happening in our environment.
One sql having CONTAINS clause is taking forever - more than 20 minutes and still does not complete. This sql has a contains clause and an exists clause and a not exists clause.
Now if I remove the exists clause and a not exists clause , it completes fast. but with those two clauses it is just taking forever. It is late night so i am not able to post the table and sql query details and will do so tomorrow but based on this general description, are there any pointers for me to review?
sql query doing fine:
SELECT
    U.CLNT_OID, U.USR_OID, S.MAILADDR
FROM
    access_usr U
    INNER JOIN access_sia S
        ON S.USR_OID = U.USR_OID AND S.CLNT_OID = U.CLNT_OID
    WHERE U.CLNT_OID = 'ABCX32S'
    AND CONTAINS(LAST_NAME , 'TO%' ) >0
--sql query that hangs forever:
SELECT
    U.CLNT_OID, U.USR_OID, S.MAILADDR
FROM
    access_usr U
    INNER JOIN access_sia S
        ON S.USR_OID = U.USR_OID AND S.CLNT_OID = U.CLNT_OID
    WHERE U.CLNT_OID = 'ABCX32S'
    AND CONTAINS(LAST_NAME , 'TO%' ) >0
and exists (--one clause here wiht a few table joins)
and not exists (--one clause here wiht a few table joins);
--Now another strange thing I found is if instead of 'TO%' in this sql, if I were to use 'ZZ%' or 'L1%' it works fast but for 'TO%' it goes slow with those two exists not exists clauses!
I will be most thankful for the inputs.
OrauserN

Hi Barbara,
First of all, thanks a lot for reviewing the issue.
Unluckily making the change to empty_stoplist did not work out. I am today copying the entire sql here that has this issue and will be most thankful for more insights/pointers on what can be done.
Here is the entire sql:
SELECT U.CLNT_OID,
       U.USR_OID,
       S.EMAILADDRESS,
       U.FIRST_NAME,
       U.LAST_NAME,
       S.JOBCODE,
       S.LOCATION,
       S.DEPARTMENT,
       S.ASSOCIATEID,
       S.ENTERPRISECOMPANYCODE,
       S.EMPLOYEEID,
       S.PAYGROUP,
       S.PRODUCTLOCALE
FROM    ACCESS_USR U
       INNER JOIN
          ACCESS_SIA S
       ON S.USR_OID = U.USR_OID AND S.CLNT_OID = U.CLNT_OID
WHERE     U.CLNT_OID = 'G39NY3D25942TXDA'
       AND EXISTS
              (SELECT 1
                 FROM ACCESS_USR_GROUP_XREF UGX
                      INNER JOIN ACCESS_GROUP RELG
                         ON     RELG.CLNT_OID = UGX.CLNT_OID
                            AND RELG.GROUP_OID = UGX.GROUP_OID
                      INNER JOIN ACCESS_GROUP G
                         ON     G.CLNT_OID = RELG.CLNT_OID
                            AND G.GROUP_TYPE_OID = RELG.GROUP_TYPE_OID
                WHERE     UGX.CLNT_OID = U.CLNT_OID
                      AND UGX.USR_OID = U.USR_OID
                      AND G.GROUP_OID = 920512943
                      AND UGX.INCLUDED = 1)
       AND NOT EXISTS
                  (SELECT 1
                     FROM    ACCESS_USR_GROUP_XREF UGX
                          INNER JOIN
                             ACCESS_GROUP G
                          ON     G.CLNT_OID = UGX.CLNT_OID
                             AND G.GROUP_OID = UGX.GROUP_OID
                    WHERE     UGX.CLNT_OID = U.CLNT_OID
                          AND UGX.USR_OID = U.USR_OID
                          AND G.GROUP_OID = 920512943
                          AND UGX.INCLUDED = 1)
       AND CONTAINS (U.LAST_NAME, 'Bon%') > 0;
Like I said before if the EXISTS and NOT EXISTS clause are removed it works in sub-second. But with those EXISTS and NOT EXISTS CLAUSE IT TAKES ANY WHERE FROM 25 minutes to more than one hour.
NOte also that it was not TO% but Bon% in the CONTAINS clause that is giving the issue - sorry that was wrong on my part.
Also please see below the ORACLE TEXT index defined on the table ACCESS_USER:
--definition of preferences used in the index:
SET SERVEROUTPUT ON size unlimited
WHENEVER SQLERROR EXIT SQL.SQLCODE
DECLARE
   v_err       VARCHAR2 (1000);
   v_sqlcode   NUMBER;
   v_count     NUMBER;
BEGIN
   ctxsys.ctx_ddl.create_preference ('cust_lexer', 'BASIC_LEXER');
   ctxsys.ctx_ddl.set_attribute ('cust_lexer', 'base_letter', 'YES'); -- removes diacritics
EXCEPTION
   WHEN OTHERS
   THEN
      v_err := SQLERRM;
      v_sqlcode := SQLCODE;
      v_count := INSTR (v_err, 'DRG-10701');
      IF v_count > 0
      THEN
         DBMS_OUTPUT.put_line (
            'The required preference named CUST_LEXER with BASIC LEXER is already set up');
      ELSE
         RAISE;
      END IF;
END;
DECLARE
   v_err       VARCHAR2 (1000);
   v_sqlcode   NUMBER;
   v_count     NUMBER;
BEGIN
   ctxsys.ctx_ddl.create_preference ('cust_wl', 'BASIC_WORDLIST');
   ctxsys.ctx_ddl.set_attribute ('cust_wl', 'SUBSTRING_INDEX', 'true'); -- to improve performance
EXCEPTION
   WHEN OTHERS
   THEN
      v_err := SQLERRM;
      v_sqlcode := SQLCODE;
      v_count := INSTR (v_err, 'DRG-10701');
      IF v_count > 0
      THEN
         DBMS_OUTPUT.put_line (
            'The required preference named CUST_WL with BASIC WORDLIST is already set up');
      ELSE
         RAISE;
      END IF;
END;
--now below is the code of the index:
CREATE INDEX ACCESS_USR_IDX3 ON ACCESS_USR
(FIRST_NAME)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('LEXER cust_lexer WORDLIST cust_wl SYNC (ON COMMIT)');
CREATE INDEX ACCESS_USR_IDX4 ON ACCESS_USR
(LAST_NAME)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('LEXER cust_lexer WORDLIST cust_wl SYNC (ON COMMIT)');
The strange thing is that, like I said, If I remove the exists clause the query returns very fast. Also if I modify the query to use only one NOT EXISTS clause and remove the other EXISTS clause it returns in less than one second. Also if I remove the EXISTS clause and use only the NOT EXISTS clause it returns in less than 4 seconds. But with both clauses it runs forever!
When I tried to get dbms_xplan.display_cursor to get the query plan (for the case of both exists and not exists clause in the query), it said that previous statement's sql id was 0 or something like that so that I was not able to see the query plan. I will keep trying to get this plan (it takes 25 minutes to one hour each time but will get this info soon). Again any pointers are most helpful.
Regards
OrauserN

ORACLE TEXT INDEX ON VARCHAR2 COLUMN

Hello All,
I find a search in our application very slow so i thought of using ORACLE TEXT CTXCAT index based search but i find certain inconsistencies . How can this be avoided....The following query should not return result if i can replace with oracle text but i find few values....why is that...i have also given few sample results below....
SELECT first_name
FROM uc_partner_ms
WHERE
Upper(first_name) LIKE '%WIE%'
minus
SELECT first_name
FROM uc_partner_ms
WHERE CATSEARCH (first_name,'*wie*', null) > 0
RESULTS ....
Hans-Werner Mrowiec
Heinz Oesterwiemann GmbH
Helmut Froitzheim GmbH, Neuwied
Heribert Schwies
Hermann Twieling GmbH & Co. KG
Horst Breitwieser
Horst-Dieter Swie
The script used for creating index is
begin
ctx_ddl.create_preference('mylex', 'BASIC_LEXER');
ctx_ddl.set_attribute ( 'mylex', 'index_themes', 'NO');
ctx_ddl.set_attribute ( 'mylex', 'mixed_case', 'NO');
end;
CREATE INDEX partner_index ON uc_partner_ms (first_name)
INDEXTYPE IS CTXSYS.CTXCAT
parameters ( 'LEXER mylex' );
Where am i wrong i could not guess a trend in the results other than all being in lower case.....

Catsearch does not support leading wildcards. As a workaround, you can use a query template with context grammar. Please see the reproduction and solution below.
SCOTT@orcl_11g> -- test environment:
SCOTT@orcl_11g> CREATE TABLE uc_partner_ms
2    (first_name VARCHAR2 (60))
3 /
Table created.
SCOTT@orcl_11g> SET DEFINE OFF
SCOTT@orcl_11g> INSERT ALL
2 INTO uc_partner_ms VALUES ('Hans-Werner Mrowiec')
3 INTO uc_partner_ms VALUES ('Heinz Oesterwiemann GmbH')
4 INTO uc_partner_ms VALUES ('Helmut Froitzheim GmbH, Neuwied')
5 INTO uc_partner_ms VALUES ('Heribert Schwies')
6 INTO uc_partner_ms VALUES ('Hermann Twieling GmbH & Co. KG')
7 INTO uc_partner_ms VALUES ('Horst Breitwieser')
8 INTO uc_partner_ms VALUES ('Horst-Dieter Swie')
9 SELECT * FROM DUAL
10 /
7 rows created.
SCOTT@orcl_11g> begin
2    ctx_ddl.create_preference('mylex', 'BASIC_LEXER');
3    ctx_ddl.set_attribute ( 'mylex', 'index_themes', 'NO');
4    ctx_ddl.set_attribute ( 'mylex', 'mixed_case', 'NO');
5 end;
6 /
PL/SQL procedure successfully completed.
SCOTT@orcl_11g> CREATE INDEX partner_index ON uc_partner_ms (first_name)
2 INDEXTYPE IS CTXSYS.CTXCAT
3 parameters ( 'LEXER mylex' )
4 /
Index created.
SCOTT@orcl_11g> -- reproduction:
SCOTT@orcl_11g> SELECT first_name
2 FROM uc_partner_ms
3 WHERE
4 Upper(first_name) LIKE '%WIE%'
5 minus
6 SELECT first_name
7 FROM uc_partner_ms
8 WHERE CATSEARCH (first_name,'*wie*', null) > 0
9 /
FIRST_NAME
Hans-Werner Mrowiec
Heinz Oesterwiemann GmbH
Helmut Froitzheim GmbH, Neuwied
Heribert Schwies
Hermann Twieling GmbH & Co. KG
Horst Breitwieser
Horst-Dieter Swie
7 rows selected.
SCOTT@orcl_11g> -- solution:
SCOTT@orcl_11g> SELECT first_name
2 FROM uc_partner_ms
3 WHERE
4 Upper(first_name) LIKE '%WIE%'
5 minus
6 SELECT first_name
7 FROM   uc_partner_ms
8 WHERE CATSEARCH
9            (first_name,
10             '<query>
11             <textquery grammar="CONTEXT">
12               %wie%
13             </textquery>
14           </query>',
15             null) > 0
16 /
no rows selected
SCOTT@orcl_11g>

Oracle text indexes

Can anybody know is it possible to create two oracle text indexes on one column, for example, CTXCAT index and CTXRULE index and what will be during the querying of that column? is it a good practise?
Thanks in advance.

When in doubt, test and see. Yes, you can create two different types of Oracle Text indexes on the same column. If you create a CTXCAT index and a CTXRULE index, then queries using CATSEARCH will use the CTXCAT index and queries using MATCHES will use the CTXRULE index. When querying with CATSEARCH, it will find all rows where the terms searched for are found within the column value. When querying with CTXRULE, it does the opposite, and finds all rows where the column values are found within the terms searched for. Please see the demonstration below. As to whether it is a good practice, it depends on what you need. If you need both types of searches, then yes. If not, then no, it would be unnecessary overhead.
SCOTT@orcl_11gR2> create table test_tab (test_col varchar2(60))
2 /
Table created.
SCOTT@orcl_11gR2> insert all
2 into test_tab values ('test')
3 into test_tab values ('data')
4 into test_tab values ('test data')
5 into test_tab values ('other stuff')
6 select * from dual
7 /
4 rows created.
SCOTT@orcl_11gR2> create index ctxcat_idx on test_tab (test_col)
2 indextype is ctxsys.ctxcat
3 /
Index created.
SCOTT@orcl_11gR2> create index ctxrule_idx on test_tab (test_col)
2 indextype is ctxsys.ctxrule
3 /
Index created.
SCOTT@orcl_11gR2> set autotrace on explain
SCOTT@orcl_11gR2> select * from test_tab
2 where catsearch (test_col, 'test data', null) > 0
3 /
TEST_COL
test data
1 row selected.
Execution Plan
Plan hash value: 399706479
| Id | Operation                   | Name       | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |            |     1 |    44 |     3   (0)| 00:00:01 |
|   1 | TABLE ACCESS BY INDEX ROWID| TEST_TAB   |     1 |    44 |     3   (0)| 00:00:01 |
|* 2 |   DOMAIN INDEX              | CTXCAT_IDX |       |       |            |          |
Predicate Information (identified by operation id):
   2 - access("CTXSYS"."CATSEARCH"("TEST_COL",'test data',NULL)>0)
Note
   - dynamic sampling used for this statement (level=2)
SCOTT@orcl_11gR2> select * from test_tab
2 where matches (test_col, 'test data') > 0
3 /
TEST_COL
test
data
test data
3 rows selected.
Execution Plan
Plan hash value: 1476734355
| Id | Operation                   | Name        | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |             |     1 |    44 |     1   (0)| 00:00:01 |
|   1 | TABLE ACCESS BY INDEX ROWID| TEST_TAB    |     1 |    44 |     1   (0)| 00:00:01 |
|* 2 |   DOMAIN INDEX              | CTXRULE_IDX |       |       |     0   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("CTXSYS"."MATCHES"("TEST_COL",'test data')>0)
Note
   - dynamic sampling used for this statement (level=2)
SCOTT@orcl_11gR2>
Message was edited by: BarbaraBoehmer

Oracle text catsearch sub index query

Similar Messages

Maybe you are looking for