About Multi_language features of Oracle Text.

Question

I have a customer who has to store into one table docs in different languages and
use contains index to perform some text search.
He would like to use the multi_language feature of Oracle Text.
The database we are using is Oracle 10gR2
We create a table with doc and language column, and then we have to create the context index.
In documentation I found some info's about how set different lexer (MULTI_LEXER) for language that have different lexer,
different stoplist (MULTI_STOPLIST) for different languages stop words,
but I don't understand if it
is possible use the stemmer features for different languages, and if there are some other features that I can set for using multi_language properties.
Thank you in advance
Paola

Answer 1

According to the online documentation: "The Oracle Text stemmer, licensed from Xerox Corporation's XSoft Division, supports the following languages with the BASIC_LEXER: English, French, Spanish, Italian, German, and Dutch. Japanese stemming is supported with the JAPANESE_LEXER."
Please see the demonstration below. Also, if you are using 10g, you can specify the language in the query, instead of changing the language for the session. 10g also has a world_lexer.
scott@ORA92> CREATE TABLE your_table
  2    (id         NUMBER,
  3       doc         CLOB,
  4       lang         VARCHAR2 (3),
  5       CONSTRAINT  your_table_id_pk PRIMARY KEY (id))
  6  /
Table created.
scott@ORA92> INSERT ALL
  2  INTO your_table VALUES (1, 'They say only the good die young.', 'eng')
  3  INTO your_table VALUES (2, 'The dogs like the cats.',          'eng')
  4  INTO your_table VALUES (3, 'cats and dogs',               'eng')
  5  INTO your_table VALUES (4, 'cat and dog',                    'eng')
  6  INTO your_table VALUES (5, 'chats et chiens',               'fre')
  7  INTO your_table VALUES (6, 'chat et chien',               'fre')
  8  INTO your_table VALUES (7, 'Die Hunde mögen die Katzen',          'ger')
  9  INTO your_table VALUES (8, 'Katzen und Hunde',               'ger')
10  INTO your_table VALUES (9, 'Katze und Hund',               'ger')
11  SELECT * FROM DUAL
12  /
9 rows created.
scott@ORA92> BEGIN
  2    ctx_ddl.create_preference ('english_lexer','basic_lexer');
  3    ctx_ddl.set_attribute      ('english_lexer','index_themes','yes');
  4    ctx_ddl.set_attribute      ('english_lexer','theme_language','english');
  5 
  6    ctx_ddl.create_preference ('french_lexer','basic_lexer');
  7    ctx_ddl.set_attribute      ('french_lexer','index_themes','yes');
  8    ctx_ddl.set_attribute      ('french_lexer','theme_language','french');
  9 
10    ctx_ddl.create_preference ('german_lexer','basic_lexer');
11    ctx_ddl.set_attribute      ('german_lexer','composite','german');
12    ctx_ddl.set_attribute      ('german_lexer','alternate_spelling','german');
13 
14    CTX_DDL.CREATE_PREFERENCE ('global_lexer', 'MULTI_LEXER');
15    ctx_ddl.add_sub_lexer      ('global_lexer','english','english_lexer', 'eng');
16    ctx_ddl.add_sub_lexer      ('global_lexer','french','french_lexer', 'fre');
17    ctx_ddl.add_sub_lexer      ('global_lexer','german','german_lexer','ger');
18    ctx_ddl.add_sub_lexer      ('global_lexer','default','english_lexer');
19 
20    CTX_DDL.CREATE_STOPLIST ('global_stoplist', 'MULTI_STOPLIST');
21    CTX_DDL.ADD_STOPWORD    ('global_stoplist', 'and', 'english');
22    CTX_DDL.ADD_STOPWORD    ('global_stoplist', 'und', 'german');
23    CTX_DDL.ADD_STOPWORD    ('global_stoplist', 'et', 'french');
24    CTX_DDL.ADD_STOPWORD    ('global_stoplist', 'the', 'ALL');
25    CTX_DDL.ADD_STOPWORD    ('global_stoplist', 'die', 'german');
26  END;
27  /
PL/SQL procedure successfully completed.
scott@ORA92> CREATE INDEX your_table_doc_idx
  2  ON your_table (doc)
  3  INDEXTYPE IS CTXSYS.CONTEXT
  4  PARAMETERS
  5    ('LEXER           global_lexer
  6        LANGUAGE COLUMN lang
  7        STOPLIST      global_stoplist')
  8  /
Index created.
scott@ORA92> ALTER SESSION SET NLS_LANGUAGE = 'AMERICAN'
  2  /
Session altered.
scott@ORA92> SELECT * FROM your_table WHERE CONTAINS (doc, 'die') > 0
  2  /
        ID DOC                                                                              LAN
         1 They say only the good die young.                                                eng
scott@ORA92> SELECT * FROM your_table WHERE CONTAINS (doc, 'cat AND dog') > 0
  2  /
        ID DOC                                                                              LAN
         4 cat and dog                                                                      eng
scott@ORA92> SELECT * FROM your_table WHERE  CONTAINS (doc, '$cat AND $dog') > 0
  2  /
        ID DOC                                                                              LAN
         4 cat and dog                                                                      eng
         3 cats and dogs                                                                    eng
         2 The dogs like the cats.                                                          eng
scott@ORA92> ALTER SESSION SET NLS_LANGUAGE = 'FRENCH'
  2  /
Session altered.
scott@ORA92> SELECT * FROM your_table WHERE CONTAINS (doc, 'chat AND chien') > 0
  2  /
        ID DOC                                                                              LAN
         6 chat et chien                                                                    fre
scott@ORA92> SELECT * FROM your_table WHERE  CONTAINS (doc, '$chat AND $chien') > 0
  2  /
        ID DOC                                                                              LAN
         6 chat et chien                                                                    fre
         5 chats et chiens                                                                  fre
scott@ORA92> ALTER SESSION SET NLS_LANGUAGE = 'GERMAN'
  2  /
Session altered.
scott@ORA92> SELECT * FROM your_table WHERE CONTAINS (doc, 'Die') > 0
  2  /
no rows selected
scott@ORA92> SELECT * FROM your_table WHERE CONTAINS (doc, 'Katze AND Hund') > 0
  2  /
        ID DOC                                                                              LAN
         9 Katze und Hund                                                                   gerMessage was edited by:
Barbara Boehmer