How do I best perform ctx_doc.markup for a batch/set of documents?

Question

Hi,
I'm are working on a system that stores chat messages in a table where the message body is stored in a CLOB column. I'm starting to implement a full text message search using Oracle Text and am looking to use the ctx_doc.markup feature to markup the search terms. The message table is pretty much like:
CREATE TABLE message (
  id NUMBER(19,0),
  sender NUMBER(19,0),
  recipient NUMBER(19,0),
  received_at TIMESTAMP (6),
  data CLOB,
and we have a context text index on the 'data' column and 'contains' queries against it work just fine. When I was looking to add the ability to markup the search terms, I was a bit surprised that there doesn't seem to be an easy way to markup a whole set of results.
What I'm planning doing is basically:
begin
  ctx_doc.markup(index_name => 'MESSAGE_DATA_TXT_IDX',
                      textkey => '2523992',
                      text_query => 'test"',
                      restab => 'message_search_result_markup',
                      query_id => '4',
                      tagset => 'TEXT_DEFAULT');
end
So in my case, a search could result in hundreds of messages being returned. Now in order to mark all of them up, I have to do a 'ctx_doc.markup' for each one, i.e. hundreds of times which seems horribly inefficient. Also keep in mind that all of this is done in a Java web service.
So my first question is, why doesn't 'markup' allow a list of ids to be passed in for the 'textkey'? That would make things much simpler like:
  ctx_doc.markup(index_name => 'MESSAGE_DATA_TXT_IDX',
                      textkey => '2523992,2523993,2523994,2523995',
                      text_query => 'test"',
                      restab => 'message_search_result_markup',
                      query_id => '4',
                      tagset => 'TEXT_DEFAULT');
which would then end up with 4 rows with query_id 4 in the message_search_result_markup table.
I then figured well since I generate the SQL in Java, I can just add a whole bunch of these calls into the begin/end block like
begin
     ctx_doc.markup(...);
     ctx_doc.markup(...);
     ctx_doc.markup(...);
     ctx_doc.markup(...);
end
basically one for each found message. However now I have the problem of tying a marked up result back to the actual message since the marked up result doesn't store the message primary key that's passed in as the 'textkey'. If the schema of the restab table would be something like
    create table message_search_result_markup (query_id  number, textkey varchar2, document  clob);
Then I could have a unique query_id for each query and be able to easily retrieve all the markup results and return them along with other data from the matched messages like the sender, recipient, timestamp, etc.
So now what I'm thinking is that for each 'textkey' I have, I have to create a unique query_id which isn't that straight forward since everything is multithreaded and multiprocess and different queries can return the same messages, so I couldn't just use the textkey as the query_id.
Does anybody have any better suggestions/ideas?
Keep in mind that I want to minimize the number of SQL queries I have to make from Java, ideally only having to make 1 query for the message search, 1 query to markup the found messages and 1 more query to get the marked up results.

Answer 1

You could write a user-defined wrapper function for the ctx_doc.markup procedure, so that you could use it in a SQL query.  Please see the demonstration below.
SCOTT@orcl12c> -- table, data, and index for testing:
SCOTT@orcl12c> CREATE TABLE message
  2    (id         NUMBER(19,0),
  3      sender         NUMBER(19,0),
  4      recipient    NUMBER(19,0),
  5      received_at  TIMESTAMP (6),
  6      data         CLOB)
  7  /
Table created.
SCOTT@orcl12c> INSERT ALL
  2  INTO message (id, data) VALUES
  3    (1, 'I''m are working on a system that stores chat messages in a table where the message body is
  4       stored in a CLOB column. I''m starting to implement a full text message search using Oracle
  5       Text and am looking to use the ctx_doc.markup feature to markup the search terms.
  6       The message table is pretty much like:')
  7  INTO message (id, data) VALUES
  8    (2, 'and we have a context text index on the ''data'' column and ''contains'' queries against it
  9      work just fine. When I was looking to add the ability to markup the search terms, I was a bit
10      surprised that there doesn''t seem to be an easy way to markup a whole set of results. ')
11  SELECT * FROM DUAL
12  /
2 rows created.
SCOTT@orcl12c> CREATE INDEX message_data_idx ON message (data) INDEXTYPE IS CTXSYS.CONTEXT
  2  /
Index created.
SCOTT@orcl12c> -- user-defined wrapper function for ctx_doc.markup procedure:
SCOTT@orcl12c> CREATE OR REPLACE FUNCTION your_markup
  2    (p_index_name IN VARCHAR2,
  3      p_textkey    IN VARCHAR2,
  4      p_text_query IN VARCHAR2,
  5      p_plaintext  IN BOOLEAN  DEFAULT TRUE,
  6      p_starttag   IN VARCHAR2 DEFAULT '<<<',
  7      p_endtag     IN VARCHAR2 DEFAULT '>>>',
  8      p_key_type   IN VARCHAR2 DEFAULT 'ROWID')
  9    RETURN        CLOB
10  AS
11    v_clob        CLOB;
12  BEGIN
13    CTX_DOC.SET_KEY_TYPE (p_key_type);
14    CTX_DOC.MARKUP
15       (index_name => p_index_name,
16        textkey    => p_textkey,
17        text_query => p_text_query,
18        restab     => v_clob,
19        plaintext  => p_plaintext,
20        starttag   => p_starttag,
21        endtag     => p_endtag);
22    RETURN v_clob;
23  END your_markup;
24  /
Function created.
SCOTT@orcl12c> SHOW ERRORS
No errors.
SCOTT@orcl12c> -- query:
SCOTT@orcl12c> COLUMN kwic FORMAT A60 WORD_WRAPPED
SCOTT@orcl12c> SELECT id,
  2          your_markup
  3            ('message_data_idx',
  4             ROWID,
  5             'column') kwic
  6  FROM   message
  7  WHERE  CONTAINS (data, 'column') > 0
  8  /
        ID KWIC
         1 I'm are working on a system that stores chat messages in a
           table where the message body is
           stored in a CLOB <<<column>>>. I'm starting to implement a
           full text message search using Oracle
           Text and am looking to use the ctx_doc.markup feature to
           markup the search terms.
           The message table is pretty much like:
         2 and we have a context text index on the 'data' <<<column>>>
           and 'contains' queries against it
           work just fine. When I was looking to add the ability to
           markup the search terms, I was a bit
           surprised that there doesn't seem to be an easy way to
           markup a whole set of results.
2 rows selected.