Document Classification application

Question

I trying to build a document classification application and have read through Chapter 6 of Oracle Text Application Developer’s Guide 11g Release 1. I currently have a document retrieval application that uses Oracle text to search documents. Our documents are stored on a file system and not in the database. We created and maintain a context index on a column of a table that contains the full path to each document. We use queries with the contains clause to do our searches and everything works fine.
Trying to build a document classification application, I have not had much success in following the example in chap 6. Therefore, I have two questions:
1. The example in chapter 6 under rule-based classification, creates table to store the documents to classified. Is it possible to classify documents that are on a file system?
2. Must the documents be text documents? I have a mixture of pdf, doc, txt and xls?
I would appreciate any help you can provide.

Answer 1

1. Yes, it is possible to classify documents that are on a file system. In your classification procedure you will need to load them into a temporary blob using dbms_lob.loadfromfile.
2. No, the documents do not have to be text documents. You can use any supported format by using ctx_doc.ifilter to convert the temporary blob to a temporary clob containing a plain text version of the pdf or whatever to use with matches.
Please see the example below that uses two pdf files. Both contain the phrase "fruit of the month". One has recipes that use bananas and the other has recipes that use cranberries. As you can see, this is a modification of the example in the online documentation. Note the significant changes to the classifier.this procedure.
SCOTT@orcl_11g> create table news_table (
  2           tk           number primary key not null,
  3           title     varchar2(1000),
  4           file_name varchar2(100));
Table created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> insert into news_table values
  2    (1, 'test1', 'banana.pdf');
1 row created.
SCOTT@orcl_11g> insert into news_table values
  2    (2, 'test2', 'cranberry.pdf');
1 row created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> create table news_categories (
  2            queryid  number primary key not null,
  3            category varchar2(100),
  4            query    varchar2(2000));
Table created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> insert into news_categories values
  2    (10,'banana','banana');
1 row created.
SCOTT@orcl_11g> insert into news_categories values
  2    (20,'cranberry','cranberry');
1 row created.
SCOTT@orcl_11g> insert into news_categories values
  2    (30,'fruit','banana or cranberry');
1 row created.
SCOTT@orcl_11g> insert into news_categories values
  2    (40,'fruit of the month','fruit of the month');
1 row created.
SCOTT@orcl_11g>
SCOTT@orcl_11g>
SCOTT@orcl_11g> create table news_id_cat (
  2            tk number,
  3            category_id number);
Table created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> create index news_cat_idx on news_categories(query)
  2  indextype is ctxsys.ctxrule;
Index created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> create or replace directory my_dir
  2  as 'c:\oracle11g'
  3  /
Directory created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> create or replace package classifier as
  2    procedure this;
  3  end classifier;
  4  /
Package created.
SCOTT@orcl_11g> show errors
No errors.
SCOTT@orcl_11g> create or replace package body classifier as
  2    procedure this
  3    is
  4        v_bfile       bfile;
  5        v_docblob     blob;
  6        v_document    clob;
  7    begin
  8        for doc in (select tk, file_name from news_table)
  9        loop
10          v_bfile := bfilename ('MY_DIR', doc.file_name);
11          dbms_lob.open (v_bfile);
12          dbms_lob.createtemporary (v_docblob, true, dbms_lob.session);
13          dbms_lob.loadfromfile
14            (v_docblob,
15             v_bfile,
16             dbms_lob.getlength (v_bfile));
17          dbms_lob.createtemporary (v_document, true, dbms_lob.session);
18          ctx_doc.ifilter (v_docblob, v_document);
19          for c in
20            (select queryid from news_categories
21             where  matches (query, v_document) > 0 )
22          loop
23            insert into news_id_cat values (doc.tk, c.queryid);
24          end loop;
25          dbms_lob.freetemporary (v_document);
26          dbms_lob.freetemporary (v_docblob);
27          dbms_lob.fileclose (v_bfile);
28        end loop;
29    end this;
30  end classifier;
31  /
Package body created.
SCOTT@orcl_11g> show errors
No errors.
SCOTT@orcl_11g>
SCOTT@orcl_11g> exec classifier.this
PL/SQL procedure successfully completed.
SCOTT@orcl_11g>
SCOTT@orcl_11g> select * from news_id_cat
  2  order  by 1, 2
  3  /
        TK CATEGORY_ID
         1          10
         1          30
         1          40
         2          20
         2          30
         2          40
6 rows selected.
SCOTT@orcl_11g>