WildCard with And

Question

Can someone please help me. I know some stuff about Oracle Text and then it seems like when I get it figured out something else breaks. I know "And" is a stopword so I've made my own stopword list and made "and" a regular word
This is my contains statement;
AND CONTAINS (location, '%BRICK {AND}%
OR $(SYN(BRICK) AND SYN({AND}))
OR ?(SYN(BRICK) AND SYN({AND}))
OR !(SYN(BRICK) AND SYN({AND})) ',1) > 0)
The Problem is I get a "WildCard error to many"
However
If I do this;
AND CONTAINS (location, '{%BRICK AND%}
OR $(SYN(BRICK) AND SYN({AND}))
OR ?(SYN(BRICK) AND SYN({AND}))
OR !(SYN(BRICK) AND SYN({AND})) ',1) > 0)
No Errors. I get two records back.
or this
AND CONTAINS (location, '%AND%
OR $(SYN({AND}))
OR ?(SYN({AND}))
OR !(SYN({AND})) ',1) > 0)
No Errors. I get records back.
I don't understand this. In the first contains it looks for a word or a word with "brick" in it and then the word "and" and then another word.
The second contains looks for any words that has a word with or without "brick" , "and" in a word or not, and then anything else.
Is this right or am I misunderstanding something. I only have about 47,000 records in the table so I can't understand why I am getting this error.
These are my preferences
begin
ctx_ddl.create_preference('PREF_SOUNDEX', 'BASIC_WORDLIST');
ctx_ddl.set_attribute('PREF_SOUNDEX','PREFIX_INDEX','TRUE');
ctx_ddl.set_attribute('PREF_SOUNDEX','PREFIX_MIN_LENGTH',3);
ctx_ddl.set_attribute('PREF_SOUNDEX','PREFIX_MAX_LENGTH',15);
ctx_ddl.set_attribute('PREF_SOUNDEX','SUBSTRING_INDEX','TRUE');
ctx_ddl.set_attribute('PREF_SOUNDEX','WILDCARD_MAXTERMS',15000);
ctx_ddl.create_preference('MAF_LEXER','BASIC_LEXER');
ctx_ddl.set_attribute('MAF_LEXER','printjoins','-#&');
ctx_ddl.set_attribute('MAF_LEXER','skipjoins','(.),');
end;
How exactly does the substring index work? Are there to many words with "and" in them and that is why it gives me this error?
Any help is greatly appreciated
Thanks
Jeff

Answer 1

I believe the problem is that you cannot combine escaping with {} and wildcarding with %. It seems to have the effect of attempting to expand to include every indexed term, as if you had searched for just %. Even if you have a small enough dataset and/or wildcard_maxterms set high enough, although it will not produce an error, it also does not return any rows. Please see the demonstration below.
SCOTT@10gXE> -- table:
SCOTT@10gXE> CREATE TABLE your_table (location VARCHAR2 (30))
  2  /
Table created.
SCOTT@10gXE> -- data:
SCOTT@10gXE> INSERT ALL
  2  INTO your_table VALUES ('WORD')
  3  INTO your_table VALUES ('WORDS')
  4  INTO your_table VALUES ('WHATEVER')
  5  SELECT * FROM DUAL
  6  /
3 rows created.
SCOTT@10gXE> -- preferences:
SCOTT@10gXE> begin
  2    ctx_ddl.create_preference('PREF_SOUNDEX', 'BASIC_WORDLIST');
  3    ctx_ddl.set_attribute('PREF_SOUNDEX','PREFIX_INDEX','TRUE');
  4    ctx_ddl.set_attribute('PREF_SOUNDEX','PREFIX_MIN_LENGTH',3);
  5    ctx_ddl.set_attribute('PREF_SOUNDEX','PREFIX_MAX_LENGTH',15);
  6    ctx_ddl.set_attribute('PREF_SOUNDEX','SUBSTRING_INDEX','TRUE');
  7    ctx_ddl.set_attribute('PREF_SOUNDEX','WILDCARD_MAXTERMS', 6);
  8    ctx_ddl.create_preference('MAF_LEXER','BASIC_LEXER');
  9    ctx_ddl.set_attribute('MAF_LEXER','printjoins','-#&');
10    ctx_ddl.set_attribute('MAF_LEXER','skipjoins','(.),');
11  end;
12  /
PL/SQL procedure successfully completed.
SCOTT@10gXE> -- index:
SCOTT@10gXE> CREATE INDEX your_index ON your_table (location)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS
  4    ('WORDLIST pref_soundex
  5        LEXER       maf_lexer
  6        STOPLIST CTXSYS.EMPTY_STOPLIST')
  7  /
Index created.
SCOTT@10gXE> -- This is what is tokenized, indexed, and searchable:
SCOTT@10gXE> SELECT token_text FROM dr$your_index$i
  2  /
TOKEN_TEXT
WHA
WHAT
WHATE
WHATEV
WHATEVE
WHATEVER
WHATEVER
WOR
WORD
WORD
WORDS
WORDS
12 rows selected.
SCOTT@10gXE> -- This works:
SCOTT@10gXE> SELECT * FROM your_table
  2  WHERE  CONTAINS (location, 'WORD%', 1) > 0
  3  /
LOCATION
WORD
WORDS
SCOTT@10gXE> -- the following queries expand to more than 6 of the above tokens:
SCOTT@10gXE> SELECT * FROM your_table
  2  WHERE  CONTAINS (location, '{WORD}%', 1) > 0
  3  /
SELECT * FROM your_table
ERROR at line 1:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-51030: wildcard query expansion resulted in too many terms
SCOTT@10gXE> SELECT * FROM your_table
  2  WHERE  CONTAINS (location, '%', 1) > 0
  3  /
SELECT * FROM your_table
ERROR at line 1:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-51030: wildcard query expansion resulted in too many terms
SCOTT@10gXE> -- increase wildcard_maxterms:
SCOTT@10gXE> exec ctx_ddl.set_attribute('PREF_SOUNDEX','WILDCARD_MAXTERMS', 50)
PL/SQL procedure successfully completed.
SCOTT@10gXE> drop index your_index
  2  /
Index dropped.
SCOTT@10gXE> CREATE INDEX your_index ON your_table (location)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS
  4    ('WORDLIST pref_soundex
  5        LEXER       maf_lexer
  6        STOPLIST CTXSYS.EMPTY_STOPLIST')
  7  /
Index created.
SCOTT@10gXE> -- now there is no error:
SCOTT@10gXE> SELECT * FROM your_table
  2  WHERE  CONTAINS (location, '{WORD}%', 1) > 0
  3  /
no rows selected
SCOTT@10gXE> SELECT * FROM your_table
  2  WHERE  CONTAINS (location, '%', 1) > 0
  3  /
LOCATION
WORD
WORDS
WHATEVER
SCOTT@10gXE>