Is a Stop Word stopping my query?

Question

I'm executing a query which searches for IN-LAW not NEXT OF KIN. I get a zillion returns on LAW but none on IN-LAW.
I know that we have text with IN-LAW in it.
My question is two-part:
1) are the Stop Words, in this case IN, interfering?
2) What can I do to get around this? How do I form my query to accomodate this?
NEXT OF KIN works just fine but no examples of IN-LAW are returned.
Thanks.

Answer 1

IN and OF are default stopwords, so it you want them to be tokenized, indexed, and searchable, you need to specify a stoplist that does not include them. A hyphen has special meaning to Oracle Text and is the same as MINUS, which tells it to subtract the score of the second term from the score of the first term. If you want characters that have special meaning to be treated like regular characters, you need to escape them, either by putting a backslash in front of an individual character or by surrounding a whole phrase in curly braces. Please see the demonstration below. By default, once you have specified that a hyphen is to be treated as a regular character, it is regarded a whitespace, so IN and LAW are tokenized separately and "IN-LAW" and "IN LAW" are treated the same. If you want to treat them separately, then you would also need to specify the hyphen as a printjoin in a lexer and specify that lexer in your index parameters.
SCOTT@10gXE> CREATE TABLE test_tab  (test_col  VARCHAR2 (30))
  2  /
Table created.
SCOTT@10gXE> INSERT ALL
  2  INTO test_tab VALUES ('IN-LAW')
  3  INTO test_tab VALUES ('IN LAW')
  4  INTO test_tab VALUES ('NEXT OF KIN')
  5  INTO test_tab VALUES ('IN')
  6  INTO test_tab VALUES ('LAW')
  7  INTO test_tab VALUES ('NEXT OF KIN')
  8  INTO test_tab VALUES ('NEXT')
  9  INTO test_tab VALUES ('OF')
10  INTO test_tab VALUES ('KIN')
11  INTO test_tab VALUES ('IN-LAW NEXT OF KIN')
12  INTO test_tab VALUES ('IN-LAW NEXT WHATEVER OF KIN')
13  INTO test_tab VALUES ('IN-LAW KIN')
14  INTO test_tab VALUES ('IN LAW KIN')
15  SELECT * FROM DUAL
16  /
13 rows created.
SCOTT@10gXE> CREATE INDEX test_ind ON test_tab (test_col)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS
  4    ('STOPLIST CTXSYS.EMPTY_STOPLIST')
  5  /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$test_ind$i
  2  /
TOKEN_TEXT
IN
KIN
LAW
NEXT
OF
WHATEVER
6 rows selected.
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, 'IN-LAW not NEXT OF KIN') > 0
  2  /
TEST_COL
IN
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, 'IN\-LAW not NEXT OF KIN') > 0
  2  /
TEST_COL
IN-LAW
IN LAW
IN-LAW NEXT WHATEVER OF KIN
IN-LAW KIN
IN LAW KIN
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '{IN-LAW} not NEXT OF KIN') > 0
  2  /
TEST_COL
IN-LAW
IN LAW
IN-LAW NEXT WHATEVER OF KIN
IN-LAW KIN
IN LAW KIN
SCOTT@10gXE>
SCOTT@10gXE> spool off