How can I eliminate HTML tags from Oracle Text Snippet?

Question

I perform a search on many tables and on many columns of those tables.
Some of those columns are VARCHAR2 and some CLOB.
Also, some of the searchable data are HTML and some are plain text.
My problem is that ctx_doc.snippet fetches the HTML tags.
For example I get this, as a snippet result in one of my searches: Qual Germany n1 <p>Test Qual Germany n1</p>
I want the result to be fetched without the HTML tags.
In my index configuration I have used NULL FILTER and HTML_SECTION_GROUP.With that configuration I managed to eliminate the HTML tags but not in all cases!
For example:
I search table CONTENTS columns TITLE(VARCHAR2) and MAIN_TEXT(CLOB)
I created the following procedure that concatenates the two columns:
CREATE OR REPLACE PROCEDURE CONTENTS_PROC( p_id in rowid, p_lob IN OUT clob)
IS
BEGIN
FOR c1 IN (SELECT main_text||' '||title data FROM contents WHERE ROWID = p_id)
LOOP
dbms_lob.copy( p_lob, c1.data,
dbms_lob.getlength( c1.data ));
END LOOP;
END;
I created a user Datastore:
BEGIN
ctx_ddl.create_preference( 'content_trans_datastore', 'user_datastore' );
ctx_ddl.set_attribute( 'content_trans_datastore', 'procedure', 'CONTENTS_PROC' );
END;
and finally I create the index:
CREATE INDEX content_trans_ot_idx ON contents(ORACLE_TEXT_COLUMN)
INDEXTYPE IS ctxsys.CONTEXT PARAMETERS ('datastore content_trans_datastore SYNC(ON COMMIT) STORAGE INDEX_STORAGE filter ctxsys.null_filter section group ctxsys.html_section_group');
When I perform the search on those data: <p> <strong>Test Doc-Test </strong> </p> the snippet I get is: Test Doc-Test.
That's fine, the html tags are removed!
In another case I search table NCP columns NAME(VARCHAR2) and BODY(VARCHAR2)
I created the following procedure that concatenates the two columns:
CREATE OR REPLACE PROCEDURE NCP_PROC( p_id in rowid, p_lob IN OUT clob)
IS
BEGIN
FOR c1 IN (SELECT name||' '||body data FROM ncp WHERE ROWID = p_id)
LOOP
dbms_lob.copy( p_lob, c1.data,
dbms_lob.getlength( c1.data ));
END LOOP;
END;
I created a user Datastore:
BEGIN
ctx_ddl.create_preference( 'ncp_trans_datastore', 'user_datastore' );
ctx_ddl.set_attribute( 'ncp_trans_datastore', 'procedure', 'NCP_PROC' );
END;
and finally I create the index:
CREATE INDEX ncp_trans_ot_idx ON ncp(ORACLE_TEXT_COLUMN)
INDEXTYPE IS ctxsys.CONTEXT PARAMETERS('datastore ncp_trans_datastore SYNC(ON COMMIT) STORAGE INDEX_STORAGE filter ctxsys.null_filter section group ctxsys.html_section_group');
When I perform the search on those data: test <strong> </strong>http://deleteme.com the snippet I get is: test <strong> </strong>http://deleteme.com!!!!!!!!!!
How is this possible? Why in the first case the HTML tags are eliminated and in the second case they are not?
Thanks,
Margarita
Edited by: user13312701 on 07-Sep-2010 08:51

Answer 1

Doing various tests I found out that the problem is when I need to search in multiple columns of a table.
That is when I create a user_datastore that uses a procedure that concatenates the columns.
And especially when the data with the html tags is in a VARCHAR2 column.
e.g
--create the table*
CREATE TABLE CONTENT_TRANS (content_trans_id NUMBER,
main_text CLOB,
title vARCHAR2(2000),
oracle_text_column VARCHAR2(1));
alter table "CONTENT_TRANS" add constraint CONTENT_PK primary key("CONTENT_TRANS_ID") ;
--Insert dummy data*
Insert into CONTENT_TRANS
(CONTENT_TRANS_ID,MAIN_TEXT,TITLE)
values
(1,'lorem','lorem <p>qualification</p> 2.1 ');
Insert into CONTENT_TRANS
(CONTENT_TRANS_ID,MAIN_TEXT,TITLE)
values
(2,'lorem','lorem <br>qualification</br> 2.1 ');
--CREATE THE procedure that concatenates main_text(CLOB) and title(VARCHAR2)*
CREATE OR REPLACE PROCEDURE CONTENT_TRANS_PROC( p_id in rowid, p_lob IN OUT clob)
IS
BEGIN
FOR c1 IN (SELECT main_text||' '||title data FROM content_trans WHERE ROWID = p_id)
LOOP
dbms_lob.copy( p_lob, c1.data,
dbms_lob.getlength( c1.data ));
END LOOP;
END;
--Create the user datastore*
BEGIN
ctx_ddl.create_preference( 'content_trans_datastore', 'user_datastore' );
ctx_ddl.set_attribute( 'content_trans_datastore', 'procedure', 'CONTENT_TRANS_PROC' );
END;
--Create the index*
CREATE INDEX content_trans_ot_idx ON content_trans(ORACLE_TEXT_COLUMN)
INDEXTYPE IS ctxsys.CONTEXT PARAMETERS ('datastore content_trans_datastore SYNC(ON COMMIT) filter ctxsys.null_filter section group ctxsys.html_section_group');
exec ctx_doc.set_key_type('PRIMARY_KEY');
--Perform the query
SELECT SCORE(1),ct.content_trans_id, ctx_doc.snippet('content_trans_ot_idx', ct.content_trans_id, 'lorem') as snippet
from content_trans ct
where contains(ct.ORACLE_TEXT_COLUMN, 'lorem', 1) > 1;
Results WITH NOT WANTED HTML TAGS:
6     1     <b>lorem</b> <b>lorem</b> &lt;p&gt;qualification&lt;/p&gt; 2.1
6     2     <b>lorem</b> <b>lorem</b> &lt;br&gt;qualification&lt;/br&gt; 2.1
Edited by: user13312701 on 13-Oct-2010 01:18