Treat plurals as singular

Question

I have a sinking feeling I know the answer to this one, but I hope I'm wrong...
I want to ask if there is a way to get Oracle Text to regard plural and singular forms of a word as equivalent that does NOT involve creating synonyms in a thesaurus (because I don't fancy re-writing a dictionary, basically!).
I have no problem using a thesaurus to make roof=rooves or mouse=mice, but I'm looking for a way to generically allow pencil=pencils or paper=papers. Right now, they're not:
SQL> create table texttst (col1 varchar2(20));
Table created.
SQL> insert into texttst values ('pencil');
1 row created.
SQL> insert into texttst values ('pencils');
1 row created.
SQL> insert into texttst values ('paper');
1 row created.
SQL> insert into texttst values ('papers');
1 row created.
SQL> commit;
Commit complete.
SQL> create index txtidx on texttst(col1) indextype is ctxsys.context;
Index created.
SQL> select count(*) from texttst where contains(col1,'pencil')>0;
  COUNT(*)
         1I want the answer to that last query to be 2!
I am aware I can do this:
SQL> select count(*) from texttst where contains(col1,'$pencil')>0;
  COUNT(*)
         2But the use of 'fuzzy' searches goes much wider than simply seeing singular as the same as plural forms of a word, and that's not something I particularly want to do if I can help it.
Is there some switch that says "plural=singular", please?

Answer 1

The default basic_wordlist with english stemmer merely makes stemming searches possible. The basic_lexer with index_stems makes stemming searches faster. It does this by determining during indexing which tokens correspond to which stems. These values are added to the dr$...$i domain index table with a token_type of 9. Then when a stemming search is done it can quickly find which tokens are associated with which stems by just searching for those with token_type = 9. Your stemming search may then even end up faster than your non-stemming search. You will need to do some tests on your own system to determine the impact. Please see the demonstration below. There are some conflicts between the documentation and what I see on my system. I don't see any way to modify the stemming dictionary or display values from it.
SCOTT@orcl_11g> -- test table and data:
SCOTT@orcl_11g> create table texttst (col1 varchar2(30))
  2  /
Table created.
SCOTT@orcl_11g> insert all
  2  into texttst values ('pen pencil')
  3  into texttst values ('pens pencils')
  4  into texttst values ('pens pencilcase')
  5  select * from dual
  6  /
3 rows created.
SCOTT@orcl_11g> insert into texttst select object_name from all_objects
  2  /
68796 rows created.
SCOTT@orcl_11g> column token_text format a30
SCOTT@orcl_11g> -- without basic_lexer and index_stems:
SCOTT@orcl_11g> create index txtidx on texttst(col1) indextype is ctxsys.context
  2  /
Index created.
SCOTT@orcl_11g> select token_text, token_type, token_first, token_last, token_count
  2  from   dr$txtidx$i
  3  where  token_text like 'PEN%'
  4  /
TOKEN_TEXT                     TOKEN_TYPE TOKEN_FIRST TOKEN_LAST TOKEN_COUNT
PEN                                     0           1          1           1
PENCIL                                  0           1          1           1
PENCILCASE                              0           3          3           1
PENCILS                                 0           2          2           1
PENDING                                 0         235      61171          49
PENDINGEXCEPTION                        0       28535      28536           2
PENS                                    0           2          3           2
7 rows selected.
SCOTT@orcl_11g> exec dbms_stats.gather_table_stats (user, 'TEXTTST')
PL/SQL procedure successfully completed.
SCOTT@orcl_11g> set timing on
SCOTT@orcl_11g> set autotrace on explain
SCOTT@orcl_11g> select * from texttst where contains (col1, 'pen pencil') > 0
  2  /
COL1
pen pencil
1 row selected.
Elapsed: 00:00:00.07
Execution Plan
Plan hash value: 472101605
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |         |     1 |    24 |     0   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| TEXTTST |     1 |    24 |     0   (0)| 00:00:01 |
|*  2 |   DOMAIN INDEX              | TXTIDX  |       |       |     0   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("CTXSYS"."CONTAINS"("COL1",'pen pencil')>0)
SCOTT@orcl_11g> select * from texttst where contains (col1, '$pen $pencil') > 0
  2  /
COL1
pen pencil
pens pencils
2 rows selected.
Elapsed: 00:00:00.08
Execution Plan
Plan hash value: 472101605
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |         |     2 |    48 |     1   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| TEXTTST |     2 |    48 |     1   (0)| 00:00:01 |
|*  2 |   DOMAIN INDEX              | TXTIDX  |       |       |     1   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("CTXSYS"."CONTAINS"("COL1",'$pen $pencil')>0)
SCOTT@orcl_11g> set autotrace off
SCOTT@orcl_11g> set timing off
SCOTT@orcl_11g> -- with basic_lexer and index_stems:
SCOTT@orcl_11g> drop index txtidx
  2  /
Index dropped.
SCOTT@orcl_11g> begin
  2    ctx_ddl.create_preference ('test_lex', 'basic_lexer');
  3    ctx_ddl.set_attribute ('test_lex', 'index_stems', 'english');
  4  end;
  5  /
PL/SQL procedure successfully completed.
SCOTT@orcl_11g> create index txtidx on texttst(col1) indextype is ctxsys.context
  2  parameters ('lexer test_lex')
  3  /
Index created.
SCOTT@orcl_11g> select token_text, token_type, token_first, token_last, token_count
  2  from   dr$txtidx$i
  3  where  token_text like 'PEN%'
  4  /
TOKEN_TEXT                     TOKEN_TYPE TOKEN_FIRST TOKEN_LAST TOKEN_COUNT
PEN                                     0           1          1           1
PEN                                     9           2          3           2
PENCIL                                  0           1          1           1
PENCIL                                  9           2          2           1
PENCILCASE                              0           3          3           1
PENCILS                                 0           2          2           1
PENDING                                 0         235      61171          49
PENDINGEXCEPTION                        0       28535      28536           2
PENNSYLVANIA                            9       46853      65539           7
PENS                                    0           2          3           2
10 rows selected.
SCOTT@orcl_11g> exec dbms_stats.gather_table_stats (user, 'TEXTTST')
PL/SQL procedure successfully completed.
SCOTT@orcl_11g> set timing on
SCOTT@orcl_11g> set autotrace on explain
SCOTT@orcl_11g> select * from texttst where contains (col1, 'pen pencil') > 0
  2  /
COL1
pen pencil
1 row selected.
Elapsed: 00:00:00.03
Execution Plan
Plan hash value: 472101605
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |         |     1 |    24 |     0   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| TEXTTST |     1 |    24 |     0   (0)| 00:00:01 |
|*  2 |   DOMAIN INDEX              | TXTIDX  |       |       |     0   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("CTXSYS"."CONTAINS"("COL1",'pen pencil')>0)
SCOTT@orcl_11g> select * from texttst where contains (col1, '$pen $pencil') > 0
  2  /
COL1
pen pencil
pens pencils
2 rows selected.
Elapsed: 00:00:00.03
Execution Plan
Plan hash value: 472101605
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |         |     2 |    48 |     1   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| TEXTTST |     2 |    48 |     1   (0)| 00:00:01 |
|*  2 |   DOMAIN INDEX              | TXTIDX  |       |       |     1   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("CTXSYS"."CONTAINS"("COL1",'$pen $pencil')>0)
SCOTT@orcl_11g> set autotrace off
SCOTT@orcl_11g> set timing off
SCOTT@orcl_11g>