Ctx_doc.snippet increase the number of occurrences returned

Question

Hi everybody,
I am pretty new on Oracle Text so please be merciful :)
I am on Oracle 10.2.0.4.0 and I need to provide my users with ALL snippets found in a text, but it seems that CTX_DOC.SNIPPET restricts the number of snippets occurrences (I will exemplify it later).
The question is: is there a way to increase the number of results/snippets returned by CTX_DOC.SNIPPET ?
In the example that follows, I create the table "mysearch" and an index of type CTXSYS.CONTEXT, then I run the final SELECT query with CTX_DOC.SNIPPET() to obtain a list of most relevant fragments.
If you run the query - see the column "SNIPPET" - record #6 is returning only the first 4 occurrences. In this case (especially because is a single row) I would expect to find all 6 occurrences instead of 4.
Again, if looking to record #7 I have only 3 occurrences.
Then for record #8 I have only 2 occurrences.
I know that CTX_DOC.SNIPPET retrieves the "most" relevant fragment, but I would like to obtain the full list of fragments present in the indexed text. Is there a way (or an alternate method) to accomplish what I like to do.
I have also noted that CTX_DOC.MARKUP can be used on the same index to retrieve the full text "marked-up" with my query terms, so I know Oracle has indexed the text somewhere, I need only to get it out !
Any help will be very appreciated.
Thanks,
Luigi
DROP TABLE mysearch;
create table mysearch (text_id number primary key, text clob);
insert into mysearch values (1, 'this is a test record which contains no more than one occurrence of the word you search');
insert into mysearch values (2, 'this is a second test record which contains two occurrences of the word test');
insert into mysearch values (3, 'this is a third test record which contains three test occurrences of the word test');
insert into mysearch values (4, 'this test is a fourth test record which contains four test occurrences of the word test');
insert into mysearch values (5, 'this test is the fifth test record which contains five test occurrences of the word test for test purposes');
insert into mysearch values (6, 'this sixth test is a test with more than 200 character which contains many test occurrences of test word, sixth test record which contains sixth test occurrences of the word test for test purposes');
insert into mysearch values (7,
'Oracle Text adds powerful test search withintitle and intelligent test management to the Oracle database.  Complete.  You can test search and manage documents, web pages, catalog entries in more than test 150 formats in any language.  Provides a complete text query language and complete character support.  Simple.  You can index and search test  text using SQL. Oracle Test Management can be done using Oracle Test Manager - a GUI tool.  Fast.  You can search
millions of documents, Test,web pages, catalog entries using the power and Test of the database.  Intelligent.  Oracle Text''s unique knowledge-base enables you to search, classify, manage documents, clusters Test summarize text based on its meaning as well as its content.');
insert into mysearch values (8,'Written by the worlds most widely-read test authors of best-selling Oracle books, Mike Ault, Daniel Liu and Madhu Tumma target their substantial knowledge of test evaluating Oracle new features in this important book. test With decades of experience evaluating new Oracle features, this book focuses on the most important test new DBA features of Oracle 10g as they relate to database administration and Oracle tuning.
This book provides honest feedback about those Oracle test 10g features that you should use, test and those you should not use. The text takes an in-depth look at test those Oracle10g features that are the most important to system performance and Oracle10g database administration.
Best of all, the authors have created dozens of working test samples in the Oracle10g online code depot. Examples from all areas of Oracle10g are covered with working scripts and code snippets. Your time savings from a single script is test worth the price of this great book.
Daniel Liu test is a senior Oracle Database test  Administrator at First American Real Estate Solutions in Anaheim, CA. He has many years of industry test experience in database administration and software development.  He has worked with large-scale databases in multi-platform environments.  His test expertise includes Oracle database administration, performance tuning, Oracle networking, and Oracle test Application Server. 
As an Oracle Certified Professional, he taught Oracle certified DBA classes at Elite Consulting Group in Chicago. Daniel also taught IOUG University Seminar in Orlando.  Daniel has published articles with DBAzine, Oracle Internals, and SELECT Journal.  Daniel has received SELECT Editorial Test Award for Best Article in 2001.  He has also given presentations at IOUG-A Live, LAOUG, OCOUG, NoCOUG, Oracle Test Open World and Oracle World.   Daniel has served as panelist on Oracles of Oracle at Oracle World and IOUG-Live.  Daniel holds a Master of Science degree in computer science from Northern Illinois University.
Madhu Tumma has been working as Software test Developer, IT Manager, Database Administrator, and Technical Consultant for about 18 years. He has worked on a wide variety of projects and environments ranging from mainframe, client-server, Test eBusiness to managed services. He has provided consultancy to variety of clients on database clusters, business continuity and high availability solutions.
His experience ranges across multiple relational database systems. Madhu is frequent Test speaker at Oracle World and IOUG where he presented many technical papers. Madhu has Master Degree in test science and attended Business Management graduate program. He lives in New Jersey with his wife Hema and two children Sandi and Sudeep.');
CREATE INDEX mysearchindex ON mysearch (text)
INDEXTYPE IS CTXSYS.CONTEXT
CALL CTX_DOC.SET_KEY_TYPE ('PRIMARY_KEY');
SELECT  text_id,
        text,
        ctx_doc.snippet('mysearchindex',TO_CHAR(text_id), 'test') AS SNIPPET
FROM    mysearch
WHERE   CONTAINS (text, 'test', 1) > 0
/

Answer 1

You could even customize your function to specify how many words on either side of the keywords to return, as shown below.
SCOTT@orcl_11gR2> CREATE OR REPLACE FUNCTION my_snippet
  2    (p_index_name IN VARCHAR2,
  3       p_textkey    IN VARCHAR2,
  4       p_text_query IN VARCHAR2,
  5       p_words      IN NUMBER DEFAULT 3)
  6    RETURN          CLOB
  7  AS
  8    v_markup      CLOB;
  9    v_string      VARCHAR2 (32767);
10    v_snippet     CLOB;
11  BEGIN
12    CTX_DOC.MARKUP (p_index_name, p_textkey, p_text_query, v_markup);
13    IF INSTR (v_markup, '<<<') > 0 AND INSTR (v_markup, '>>>') > 0 THEN
14        v_markup := LPAD (' ', p_words) || v_markup || RPAD (' ', p_words);
15        v_string := SUBSTR (v_markup, 1, INSTR (v_markup, '<<<') - 1);
16        v_snippet := LTRIM (SUBSTR (v_string, INSTR (v_string, ' ', -1, p_words + 1)));
17        LOOP
18          v_markup := SUBSTR (v_markup, INSTR (v_markup, '<<<'));
19          v_snippet := v_snippet || SUBSTR (v_markup, 1, INSTR (v_markup, '>>>') + 2);
20          v_markup := SUBSTR (v_markup, INSTR (v_markup, '>>>') + 3);
21          IF INSTR (v_markup, '>>>') = 0 THEN
22            EXIT;
23          END IF;
24          v_string := SUBSTR (v_markup, 1, INSTR (v_markup, '<<<') - 1);
25          IF REGEXP_COUNT (LTRIM (RTRIM (v_string)), ' ') <= (p_words * 2) - 1 THEN
26            v_snippet := v_snippet || v_string;
27          ELSE
28            v_snippet := v_snippet || SUBSTR (v_string, 1, INSTR (v_string, ' ', 1, p_words + 1));
29            v_snippet := v_snippet || ' ... ';
30            v_string := SUBSTR (v_string, INSTR (v_string, ' ', 1, p_words + 1));
31            v_snippet := v_snippet || SUBSTR (v_string, INSTR (v_string, ' ', -1, p_words + 1));
32          END IF;
33        END LOOP;
34        v_snippet := v_snippet || RTRIM (SUBSTR (v_markup, 1, INSTR (v_markup, ' ', 1, p_words + 1)));
35    END IF;
36    RETURN v_snippet;
37  END my_snippet;
38  /
Function created.
SCOTT@orcl_11gR2> SHOW ERRORS
No errors.
SCOTT@orcl_11gR2> COLUMN  snippet FORMAT A80 WORD_WRAPPED
SCOTT@orcl_11gR2> SELECT  text_id, SCORE (1) AS occurrences,
  2            my_snippet ('mysearchindex', TO_CHAR (text_id), 'test', 1) AS snippet
  3  FROM    mysearch
  4  WHERE   CONTAINS (text, 'DEFINESCORE (test, OCCURRENCE)', 1) > 0
  5  /
   TEXT_ID OCCURRENCES
SNIPPET
         1           1
a <<<test>>> record
         2           2
second <<<test>>> record  ...  word <<<test>>>
         3           3
third <<<test>>> record  ...  three <<<test>>> occurrences  ...  word <<<test>>>
         4           4
this <<<test>>> is  ...  fourth <<<test>>> record  ...  four <<<test>>>
occurrences  ...  word <<<test>>>
         5           5
this <<<test>>> is  ...  fifth <<<test>>> record  ...  five <<<test>>>
occurrences  ...  word <<<test>>> for <<<test>>> purposes
         6           8
sixth <<<test>>> is a <<<test>>> with  ...  many <<<test>>> occurrences of
<<<test>>> word, sixth <<<test>>> record  ...  sixth <<<test>>> occurrences  ...
word <<<test>>> for <<<test>>> purposes
         7          10
powerful <<<test>>> search  ...  intelligent <<<test>>> management  ...  can
<<<test>>> search  ...  than <<<test>>> 150  ...  search <<<test>>>   ...
Oracle <<<Test>>> Management  ...  Oracle <<<Test>>> Manager  ...  documents,
<<<Test>>>,web pages,  ...  and <<<Test>>> of  ...  clusters <<<Test>>>
summarize
         8          20
widely-read <<<test>>> authors  ...  of <<<test>>> evaluating  ...  book.
<<<test>>> With  ...  important <<<test>>> new  ...  Oracle <<<test>>> 10g  ...
use, <<<test>>> and  ...  at <<<test>>> those  ...  working <<<test>>> samples
...  is <<<test>>> worth  ...  Liu <<<test>>> is  ...  Database <<<test>>>   ...
industry <<<test>>> experience  ...  His <<<test>>> expertise  ...  Oracle
<<<test>>> Application  ...  Editorial <<<Test>>> Award  ...  Oracle <<<Test>>>
Open  ...  Software <<<test>>> Developer,  ...  client-server, <<<Test>>>
eBusiness  ...  frequent <<<Test>>> speaker  ...  in <<<test>>> science
8 rows selected.
SCOTT@orcl_11gR2> SELECT  text_id, SCORE (1) AS occurrences,
  2            my_snippet ('mysearchindex', TO_CHAR (text_id), 'test', 2) AS snippet
  3  FROM    mysearch
  4  WHERE   CONTAINS (text, 'DEFINESCORE (test, OCCURRENCE)', 1) > 0
  5  /
   TEXT_ID OCCURRENCES
SNIPPET
         1           1
is a <<<test>>> record which
         2           2
a second <<<test>>> record which  ...  the word <<<test>>>
         3           3
a third <<<test>>> record which contains three <<<test>>> occurrences of the
word <<<test>>>
         4           4
this <<<test>>> is a fourth <<<test>>> record which contains four <<<test>>>
occurrences of the word <<<test>>>
         5           5
this <<<test>>> is the fifth <<<test>>> record which contains five <<<test>>>
occurrences of the word <<<test>>> for <<<test>>> purposes
         6           8
this sixth <<<test>>> is a <<<test>>> with more  ...  contains many <<<test>>>
occurrences of <<<test>>> word, sixth <<<test>>> record which contains sixth
<<<test>>> occurrences of the word <<<test>>> for <<<test>>> purposes
         7          10
adds powerful <<<test>>> search withintitle and intelligent <<<test>>>
management to  ...  You can <<<test>>> search and  ...  more than <<<test>>> 150
formats  ...  and search <<<test>>>  text using SQL. Oracle <<<Test>>>
Management can  ...  using Oracle <<<Test>>> Manager -  ...  of documents,
<<<Test>>>,web pages, catalog  ...  power and <<<Test>>> of the  ...  documents,
clusters <<<Test>>> summarize text
         8          20
most widely-read <<<test>>> authors of  ...  knowledge of <<<test>>> evaluating
Oracle  ...  important book. <<<test>>> With decades  ...  most important
<<<test>>> new DBA  ...  those Oracle <<<test>>> 10g features  ...  should use,
<<<test>>> and those  ...  look at <<<test>>> those Oracle10g  ...  of working
<<<test>>> samples in  ...  script is <<<test>>> worth the  ...  book.
Daniel Liu <<<test>>> is a  ...  Oracle Database <<<test>>>  Administrator  ...
of industry <<<test>>> experience in  ...   His <<<test>>> expertise includes
...  and Oracle <<<test>>> Application Server.
As  ...  SELECT Editorial <<<Test>>> Award for  ...  NoCOUG, Oracle <<<Test>>>
Open World  ...  as Software <<<test>>> Developer, IT  ...  mainframe,
client-server, <<<Test>>> eBusiness to  ...  is frequent <<<Test>>> speaker at
...  Degree in <<<test>>> science and
8 rows selected.
SCOTT@orcl_11gR2> SELECT  text_id, SCORE (1) AS occurrences,
  2            my_snippet ('mysearchindex', TO_CHAR (text_id), 'test', 3) AS snippet
  3  FROM    mysearch
  4  WHERE   CONTAINS (text, 'DEFINESCORE (test, OCCURRENCE)', 1) > 0
  5  /
   TEXT_ID OCCURRENCES
SNIPPET
         1           1
this is a <<<test>>> record which contains
         2           2
is a second <<<test>>> record which contains  ...  of the word <<<test>>>
         3           3
is a third <<<test>>> record which contains three <<<test>>> occurrences of the
word <<<test>>>
         4           4
this <<<test>>> is a fourth <<<test>>> record which contains four <<<test>>>
occurrences of the word <<<test>>>
         5           5
this <<<test>>> is the fifth <<<test>>> record which contains five <<<test>>>
occurrences of the word <<<test>>> for <<<test>>> purposes
         6           8
this sixth <<<test>>> is a <<<test>>> with more than  ...  which contains many
<<<test>>> occurrences of <<<test>>> word, sixth <<<test>>> record which
contains sixth <<<test>>> occurrences of the word <<<test>>> for <<<test>>>
purposes
         7          10
Text adds powerful <<<test>>> search withintitle and intelligent <<<test>>>
management to the  ...   You can <<<test>>> search and manage  ...  in more than
<<<test>>> 150 formats in  ...  index and search <<<test>>>  text using SQL.
Oracle <<<Test>>> Management can be done using Oracle <<<Test>>> Manager - a
...  search
millions of documents, <<<Test>>>,web pages, catalog entries  ...  the power and
<<<Test>>> of the database.  ...  manage documents, clusters <<<Test>>>
summarize text based
         8          20
worlds most widely-read <<<test>>> authors of best-selling  ...  substantial
knowledge of <<<test>>> evaluating Oracle new  ...  this important book.
<<<test>>> With decades of  ...  the most important <<<test>>> new DBA features
...  about those Oracle <<<test>>> 10g features that you should use, <<<test>>>
and those you  ...  in-depth look at <<<test>>> those Oracle10g features  ...
dozens of working <<<test>>> samples in the  ...  single script is <<<test>>>
worth the price  ...  great book.
Daniel Liu <<<test>>> is a senior Oracle Database <<<test>>>  Administrator at
...  years of industry <<<test>>> experience in database  ...  environments.
His <<<test>>> expertise includes Oracle  ...  networking, and Oracle <<<test>>>
Application Server.
As an  ...  received SELECT Editorial <<<Test>>> Award for Best  ...  OCOUG,
NoCOUG, Oracle <<<Test>>> Open World and  ...  working as Software <<<test>>>
Developer, IT Manager,  ...  from mainframe, client-server, <<<Test>>> eBusiness
to managed  ...  Madhu is frequent <<<Test>>> speaker at Oracle  ...  Master
Degree in <<<test>>> science and attended
8 rows selected.
SCOTT@orcl_11gR2> SELECT  text_id, SCORE (1) AS occurrences,
  2            my_snippet ('mysearchindex', TO_CHAR (text_id), 'test', 4) AS snippet
  3  FROM    mysearch
  4  WHERE   CONTAINS (text, 'DEFINESCORE (test, OCCURRENCE)', 1) > 0
  5  /
   TEXT_ID OCCURRENCES
SNIPPET
         1           1
this is a <<<test>>> record which contains no
         2           2
this is a second <<<test>>> record which contains two occurrences of the word
<<<test>>>
         3           3
this is a third <<<test>>> record which contains three <<<test>>> occurrences of
the word <<<test>>>
         4           4
this <<<test>>> is a fourth <<<test>>> record which contains four <<<test>>>
occurrences of the word <<<test>>>
         5           5
this <<<test>>> is the fifth <<<test>>> record which contains five <<<test>>>
occurrences of the word <<<test>>> for <<<test>>> purposes
         6           8
this sixth <<<test>>> is a <<<test>>> with more than 200 character which
contains many <<<test>>> occurrences of <<<test>>> word, sixth <<<test>>> record
which contains sixth <<<test>>> occurrences of the word <<<test>>> for
<<<test>>> purposes
         7          10
Oracle Text adds powerful <<<test>>> search withintitle and intelligent
<<<test>>> management to the Oracle  ...  Complete.  You can <<<test>>> search
and manage documents,  ...  entries in more than <<<test>>> 150 formats in any
...  can index and search <<<test>>>  text using SQL. Oracle <<<Test>>>
Management can be done using Oracle <<<Test>>> Manager - a GUI  ...  can search
millions of documents, <<<Test>>>,web pages, catalog entries using the power and
<<<Test>>> of the database.   ...  classify, manage documents, clusters
<<<Test>>> summarize text based on
         8          20
the worlds most widely-read <<<test>>> authors of best-selling Oracle  ...
their substantial knowledge of <<<test>>> evaluating Oracle new features in this
important book. <<<test>>> With decades of experience  ...  on the most
important <<<test>>> new DBA features of  ...  feedback about those Oracle
<<<test>>> 10g features that you should use, <<<test>>> and those you should
...  an in-depth look at <<<test>>> those Oracle10g features that  ...  created
dozens of working <<<test>>> samples in the Oracle10g  ...  a single script is
<<<test>>> worth the price of this great book.
Daniel Liu <<<test>>> is a senior Oracle Database <<<test>>>  Administrator at
First  ...  many years of industry <<<test>>> experience in database
administration  ...  multi-platform environments.  His <<<test>>> expertise
includes Oracle database  ...  Oracle networking, and Oracle <<<test>>>
Application Server.
As an Oracle  ...  has received SELECT Editorial <<<Test>>> Award for Best
Article  ...  LAOUG, OCOUG, NoCOUG, Oracle <<<Test>>> Open World and Oracle  ...
been working as Software <<<test>>> Developer, IT Manager, Database  ...
ranging from mainframe, client-server, <<<Test>>> eBusiness to managed services.
...  systems. Madhu is frequent <<<Test>>> speaker at Oracle World  ...  has
Master Degree in <<<test>>> science and attended Business
8 rows selected.
SCOTT@orcl_11gR2> SELECT  text_id, SCORE (1) AS occurrences,
  2            my_snippet ('mysearchindex', TO_CHAR (text_id), 'test', 5) AS snippet
  3  FROM    mysearch
  4  WHERE   CONTAINS (text, 'DEFINESCORE (test, OCCURRENCE)', 1) > 0
  5  /
   TEXT_ID OCCURRENCES
SNIPPET
         1           1
this is a <<<test>>> record which contains no more
         2           2
this is a second <<<test>>> record which contains two occurrences of the word
<<<test>>>
         3           3
this is a third <<<test>>> record which contains three <<<test>>> occurrences of
the word <<<test>>>
         4           4
this <<<test>>> is a fourth <<<test>>> record which contains four <<<test>>>
occurrences of the word <<<test>>>
         5           5
this <<<test>>> is the fifth <<<test>>> record which contains five <<<test>>>
occurrences of the word <<<test>>> for <<<test>>> purposes
         6           8
this sixth <<<test>>> is a <<<test>>> with more than 200 character which
contains many <<<test>>> occurrences of <<<test>>> word, sixth <<<test>>> record
which contains sixth <<<test>>> occurrences of the word <<<test>>> for
<<<test>>> purposes
         7          10
Oracle Text adds powerful <<<test>>> search withintitle and intelligent
<<<test>>> management to the Oracle database.  Complete.  You can <<<test>>>
search and manage documents, web  ...  catalog entries in more than <<<test>>>
150 formats in any language.  ...  You can index and search <<<test>>>  text
using SQL. Oracle <<<Test>>> Management can be done using Oracle <<<Test>>>
Manager - a GUI tool.  ...  You can search
millions of documents, <<<Test>>>,web pages, catalog entries using the power and
<<<Test>>> of the database.  Intelligent.  ...  search, classify, manage
documents, clusters <<<Test>>> summarize text based on its
         8          20
by the worlds most widely-read <<<test>>> authors of best-selling Oracle books,
...  target their substantial knowledge of <<<test>>> evaluating Oracle new
features in this important book. <<<test>>> With decades of experience
evaluating  ...  focuses on the most important <<<test>>> new DBA features of
Oracle  ...  honest feedback about those Oracle <<<test>>> 10g features that you
should use, <<<test>>> and those you should not  ...  takes an in-depth look at
<<<test>>> those Oracle10g features that are  ...  have created dozens of
working <<<test>>> samples in the Oracle10g online  ...  from a single script is
<<<test>>> worth the price of this great book.
Daniel Liu <<<test>>> is a senior Oracle Database <<<test>>>  Administrator at
First American  ...  has many years of industry <<<test>>> experience in
database administration and  ...  in multi-platform environments.  His
<<<test>>> expertise includes Oracle database administration,  ...  tuning,
Oracle networking, and Oracle <<<test>>> Application Server.
As an Oracle Certified  ...  Daniel has received SELECT Editorial <<<Test>>>
Award for Best Article in  ...  Live, LAOUG, OCOUG, NoCOUG, Oracle <<<Test>>>
Open World and Oracle World.  ...  has been working as Software <<<test>>>
Developer, IT Manager, Database Administrator,  ...  environments ranging from
mainframe, client-server, <<<Test>>> eBusiness to managed services. He  ...
database systems. Madhu is frequent <<<Test>>> speaker at Oracle World and  ...
Madhu has Master Degree in <<<test>>> science and attended Business Management
8 rows selected.
SCOTT@orcl_11gR2>