Searching on high ascii characters

Question

Hi all,
I am writing a search engine built on Oracle text (otherwise why would i be posting here??) to search lists of medical articles. Many of the article titles have special characters in the title like cedilla's and umlaut's. Some of my users will have european keyboards and will be searching using these characters, other will not and would demote such characters to the low ascii value. Currently all special characters are stored in the table as html encoded values, but this could easily be changed to something else that is supported by Oracle Text.
Example: I have a article called "Behçet's syndrome" this needs to match if someone search with Behçet or Behcet.
EMP has two records one with ename = Behcet and the other ename = Behçet
SELECT ENAME
FROM EMP
WHERE contains (ENAME, 'Behcet', 100) > 0
only returns one row.
I have looked at using SYN to provide this functionality and it works but means that i continually have to update and maintain that list of synonyms which is a chore and something i want to avoid.
Is there a way to build a index that covers both possibilities? So provide a list of special characters and what they would degrade into? I feel that there is a very simple elegant solution to this just waiting for me, any suggestions very welcome.
thanks
Toby

Answer 1

SCOTT@10gXE> CREATE TABLE articles (id NUMBER, title VARCHAR2 (30))
  2  /
Table created.
SCOTT@10gXE> INSERT ALL
  2  INTO articles VALUES (1, 'Behçet''s syndrome')
  3  INTO articles VALUES (2, 'Behcet''s syndrome')
  4  INTO articles VALUES (2, 'compulsive tuning disorder')
  5  SELECT * FROM DUAL
  6  /
3 rows created.
SCOTT@10gXE> EXEC CTX_DDL.CREATE_PREFERENCE ('your_lexer', 'BASIC_LEXER')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.SET_ATTRIBUTE ('your_lexer', 'BASE_LETTER', 'YES')
PL/SQL procedure successfully completed.
SCOTT@10gXE> CREATE INDEX articles_idx ON articles (title)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS ('LEXER your_lexer')
  4  /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$articles_idx$i
  2  /
TOKEN_TEXT
BEHCET
COMPULSIVE
DISORDER
SYNDROME
TUNING
SCOTT@10gXE> SELECT * FROM articles WHERE CONTAINS (title, 'Behçet') > 0
  2  /
        ID TITLE
         1 Behçet's syndrome
         2 Behcet's syndrome
SCOTT@10gXE> SELECT * FROM articles WHERE CONTAINS (title, 'Behcet') > 0
  2  /
        ID TITLE
         1 Behçet's syndrome
         2 Behcet's syndrome
SCOTT@10gXE>