Extracting stem words from text index

Question

Hello all,
I am trying to categorize some records in a table. I wonder if Oracle Text has some searching capabilities inside the text index. So, what I'm trying to achieve is to find the minimum amount of stem words that can be found in a set of records. Basically, it's kind of reverse searching, I have a subset of records from a table that can be found using a regular query (no text query) - the table has a text index on one column - and I want to find, using the text index, the minimum amount of stem words in that column that can generate a hit for the whole subset if queried using only the text query.
Thanks,
Danny

Answer 1

Here is a method for viewing the stem word of any given word by using a function that inserts one row into a table, dynamically rebuilds the index, then selects the stem word from the domain index table. I have then added some code to use that to loop through all the words in the original domain index table, insert them and their roots into another table, and select the roots and corresponding concatenated words.
SCOTT@orcl_11gR2> SELECT banner FROM v$version
  2  /
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE     11.2.0.1.0     Production
TNS for 64-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
5 rows selected.
SCOTT@orcl_11gR2> CREATE TABLE test_tab (test_col VARCHAR2 (40))
  2  /
Table created.
SCOTT@orcl_11gR2> INSERT ALL
  2  INTO test_tab VALUES ('The cats ran quickly from the dogs.')
  3  INTO test_tab VALUES ('The mice were running from the cats.')
  4  INTO test_tab VALUES ('Some people walk their dogs every day.')
  5  INTO test_tab VALUES ('The dogs chased the cats.')
  6  SELECT * FROM DUAL
  7  /
4 rows created.
SCOTT@orcl_11gR2> BEGIN
  2    CTX_DDL.CREATE_PREFERENCE ('test_lex', 'AUTO_LEXER');
  3    CTX_DDL.SET_ATTRIBUTE ('test_lex', 'INDEX_STEMS', 'YES');
  4  END;
  5  /
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> CREATE INDEX test_idx ON test_tab (test_col)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS
  4    ('LEXER       test_lex
  5        STOPLIST CTXSYS.EMPTY_STOPLIST')
  6  /
Index created.
SCOTT@orcl_11gR2> CREATE TABLE stem_tab
  2    (test_word  VARCHAR2 (4000))
  3  /
Table created.
SCOTT@orcl_11gR2> CREATE INDEX stem_idx on stem_tab (test_word)
  2  INDEXTYPE IS CTXSYS.CONTEXT
  3  PARAMETERS
  4    ('LEXER        stem_lex
  5        STOPLIST  CTXSYS.EMPTY_STOPLIST')
  6  /
Index created.
SCOTT@orcl_11gR2> CREATE OR REPLACE FUNCTION get_stem
  2    (p_word IN VARCHAR2)
  3    RETURN VARCHAR2
  4  AS
  5    v_word       VARCHAR2 (32767);
  6  BEGIN
  7    DELETE FROM stem_tab;
  8    COMMIT;
  9    INSERT INTO stem_tab (test_word) VALUES (p_word);
10    COMMIT;
11    EXECUTE IMMEDIATE 'ALTER INDEX stem_idx REBUILD';
12    SELECT MIN (token_text)
13    INTO   v_word
14    FROM   dr$stem_idx$i;
15    RETURN v_word;
16  EXCEPTION
17    WHEN NO_DATA_FOUND THEN RETURN p_word;
18  END get_stem;
19  /
Function created.
SCOTT@orcl_11gR2> SHOW ERRORS
No errors.
SCOTT@orcl_11gR2> CREATE TABLE words_and_stems
  2    (word  VARCHAR2 (20),
  3       stem  VARCHAR2 (20))
  4  /
Table created.
SCOTT@orcl_11gR2> SET SERVEROUTPUT ON
SCOTT@orcl_11gR2> DECLARE
  2    v_stem VARCHAR2 (32767);
  3  BEGIN
  4    FOR r IN
  5        (SELECT DISTINCT token_text
  6         FROM      dr$test_idx$i
  7         WHERE  token_text != '.')
  8    LOOP
  9        v_stem := get_stem (r.token_text);
10        INSERT INTO words_and_stems
11        VALUES (r.token_text, v_stem);
12    END LOOP;
13    COMMIT;
14  END;
15  /
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> COLUMN words FORMAT A45 WORD_WRAPPED
SCOTT@orcl_11gR2> SELECT stem,
  2           LISTAGG (word, ', ') WITHIN GROUP (ORDER BY word)
  3             AS words
  4  FROM   words_and_stems
  5  GROUP  BY stem
  6  /
STEM                 WORDS
BE                   BE, WERE
CAT                  CAT, CATS
CHASE                CHASE, CHASED
DAY                  DAY
DOG                  DOG, DOGS
EVERY                EVERY
FROM                 FROM
MICE                 MICE
MOUSE                MOUSE
PEOPLE               PEOPLE
QUICKLY              QUICKLY
RAN                  RAN
RUN                  RUN, RUNNING
SOME                 SOME
THE                  THE
THEIR                THEIR
WALK                 WALK
17 rows selected.
SCOTT@orcl_11gR2>