Parse multilingual search string

Question

I'm using Oracle Text to search a multilingual database. The languages include English & Spanish (= white-space delimited) and Chinese (= NON-white-space delimited).
Suppose a user enters the following search string (quotes are mine): "word1 word2 word3"
Then I want to search something like this (thanks to Barbara Boehmer!):
"SELECT id, title, short_description FROM books
WHERE CONTAINS (dummy, 'word1 AND word2 AND word3') > 0;"
If the string is in English, Spanish, German..., no problem.
But does anyone know how I could parse the search string so that Chinese, Japanese, etc. will be handled properly too? In that case, the words consist of one or more characters (instead of letters) and they are NOT separated by spaces.
I've been searching the web and can't figure this out.

Answer 1

I misunderstood the problem. If you want to parse the search string and put spaces between the tokens, then perhaps you could use ctx_doc.policy_tokens on the search string, something like below.
BEGIN
  CTX_DDL.CREATE_PREFERENCE
    ('searchstring_lexer',
     'WORLD_LEXER');
  CTX_DDL.CREATE_POLICY
    (policy_name => 'searchstring_policy',
     lexer       => 'searchstring_lexer');
END;
CREATE OR REPLACE FUNCTION parse_searchstring
  (p_searchstring IN VARCHAR2)
  RETURN VARCHAR2
AS
  rtab            CTX_DOC.TOKEN_TAB;
  v_searchstring  VARCHAR2 (200);
BEGIN
  CTX_DOC.POLICY_TOKENS
    ('searchstring_policy',
     p_searchstring,
     rtab);
  FOR i IN 1 .. rtab.COUNT LOOP
    v_searchstring := v_searchstring || rtab(i).token || ' ';
  END LOOP;
  RETURN RTRIM (v_searchstring);
END parse_searchstring;
SELECT parse_searchstring ('英语 翻译') FROM DUAL
/