Special Characters in search terms and in data

Question

I'm having a hard time making a decision about what types of special characters I should support in my Contains search.
I know that some of my data has dashes in it like '1034-AMFM' but if a user puts that in the search term it will be interpretted as a special character by OracleText correct?
What I have resorted to at the moment is stripping all special characters out of the users search terms but this severely limits the capabilities of Oracle Text and what I'm stuck with is a very basic search tool.
Is there a way to find a happy medium here?

Answer 1

You need to use a backslash, not a forward slash. The following demonstrates the behavior under different circumstances, such as specifying the hyphen as a printjoin or skipjoin or whitespace, which is the default, with and without escaping:
SCOTT@10gXE> CREATE TABLE test_tab (test_col  VARCHAR2 (30))
  2  /
Table created.
SCOTT@10gXE> INSERT ALL
  2  INTO test_tab VALUES ('1034-AMFM')
  3  INTO test_tab VALUES ('1034 AMFM')
  4  INTO test_tab VALUES ('1034AMFM')
  5  INTO test_tab VALUES ('95.1-FM')
  6  SELECT * FROM DUAL
  7  /
4 rows created.
SCOTT@10gXE> -- without specifying hyphen as printjoin or skipjoin or whitespace
SCOTT@10gXE> -- or continuation or punctuation or whatever:
SCOTT@10gXE> CREATE INDEX test_idx ON test_tab (test_col)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$test_idx$i
  2  /
TOKEN_TEXT
1034
1034AMFM
95.1
AMFM
FM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034-AMFM') > 0
  2  /
no rows selected
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034\-AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034{-}AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '{1034-AMFM}') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034 AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034AMFM') > 0
  2  /
TEST_COL
1034AMFM
SCOTT@10gXE> -- with hyphen as printjoin:
SCOTT@10gXE> DROP INDEX test_idx
  2  /
Index dropped.
SCOTT@10gXE> EXEC CTX_DDL.CREATE_PREFERENCE ('test_lex', 'BASIC_LEXER')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.SET_ATTRIBUTE ('test_lex', 'PRINTJOINS', '-')
PL/SQL procedure successfully completed.
SCOTT@10gXE> CREATE INDEX test_idx ON test_tab (test_col)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS ('LEXER test_lex')
  4  /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$test_idx$i
  2  /
TOKEN_TEXT
1034
1034-AMFM
1034AMFM
95.1-FM
AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034-AMFM') > 0
  2  /
no rows selected
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034\-AMFM') > 0
  2  /
TEST_COL
1034-AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034{-}AMFM') > 0
  2  /
no rows selected
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '{1034-AMFM}') > 0
  2  /
TEST_COL
1034-AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034 AMFM') > 0
  2  /
TEST_COL
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034AMFM') > 0
  2  /
TEST_COL
1034AMFM
SCOTT@10gXE> -- with hyphen as skipjoin:
SCOTT@10gXE> DROP INDEX test_idx
  2  /
Index dropped.
SCOTT@10gXE> EXEC CTX_DDL.DROP_PREFERENCE ('test_lex')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.CREATE_PREFERENCE ('test_lex', 'BASIC_LEXER')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.SET_ATTRIBUTE ('test_lex', 'SKIPJOINS', '-')
PL/SQL procedure successfully completed.
SCOTT@10gXE> CREATE INDEX test_idx ON test_tab (test_col)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS ('LEXER test_lex')
  4  /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$test_idx$i
  2  /
TOKEN_TEXT
1034
1034AMFM
95.1FM
AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034-AMFM') > 0
  2  /
no rows selected
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034\-AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034{-}AMFM') > 0
  2  /
TEST_COL
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '{1034-AMFM}') > 0
  2  /
TEST_COL
1034-AMFM
1034AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034 AMFM') > 0
  2  /
TEST_COL
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034AMFM
SCOTT@10gXE> -- with hyphen as whitespace:
SCOTT@10gXE> DROP INDEX test_idx
  2  /
Index dropped.
SCOTT@10gXE> EXEC CTX_DDL.DROP_PREFERENCE ('test_lex')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.CREATE_PREFERENCE ('test_lex', 'BASIC_LEXER')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.SET_ATTRIBUTE ('test_lex', 'WHITESPACE', ' -')
PL/SQL procedure successfully completed.
SCOTT@10gXE> CREATE INDEX test_idx ON test_tab (test_col)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS ('LEXER test_lex')
  4  /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$test_idx$i
  2  /
TOKEN_TEXT
1034
1034AMFM
95.1
AMFM
FM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034-AMFM') > 0
  2  /
no rows selected
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034\-AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034{-}AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '{1034-AMFM}') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034 AMFM') > 0
  2  /
TEST_COL
1034-AMFM
1034 AMFM
SCOTT@10gXE> SELECT * FROM test_tab WHERE CONTAINS (test_col, '1034AMFM') > 0
  2  /
TEST_COL
1034AMFM
SCOTT@10gXE>