Oracle Text Concatenated Datastore

I have read this:
http://www.oracle.com/technology/sample_code/products/text/htdocs/concatenated_text_datastore/cdstore_readme.html
I've been trying to follos the 'Installation' section.
I've downloaded cdstore.sql but I get error ORA-00942 (table does not exist) because ctx_user_cdstore_cols does not exist (at line 618 in the file).
Indeed, the table created is 'ctx_cdstore_cols' and not 'ctx_user_cdstore_cols'.
I've changed it to ctx_cdstore_cols and now get ORA-00904 because CDSTORE_NAME is not a column of ctx_cdstores.
Anyway, I believe that this code should work as is so there is something big I must be missing.
Has anyone managed to install this package and how please?

It's not a problem with the concatenated datastore, it's about operator precedence.
If you search for 'A or B within SECTION', "within" has a higher precedence than "or", so this becomes 'A or (B within SECTION)'. What you need to say is '(A or B) within SECTION', or in your case '(BROOKS or BONDS) within name'
Hope this helps.
Roger

Similar Messages

Concatenated Datastore doesn't work for me

I'm using oracle InterMedia Text.
Now I'm trying to build a Concatenated Datastore.
I executed the sql-file cdstore.sql. 0 Errors.
But when I'm trying to execute
"exec ctx_cd.add_column('my_cdstore',columnname)
I get this error:
ORA-01749: you may not GRANT/REVOKE privileges to/from yourself
ORA-06512: at "TWEB.CTX_cd", line 315
ORA-06512: at "TWEB.CTX_cd", line 443
ORA-06512: at "TWEB.CTX_cd", line 816
ORA-06512: at line 2
I'm fairly confused. I read that "The Concatenated Datastore is an additional datastore for Oracle Text".
Can I use it when I'm working with Intermedia, or what am I doing wrong???
THX for help!

There are two possibilities here. I think the most likely one is that you've accidently run cdstore.sql as the user "tweb" at some stage. The ctx_cd package is therefore over-riding the version owned by user CTXSYS.
To check this, log on as user tweb, and do this:
DROP PACKAGE CTX_CD;
if that succeeds - then this is the problem. You'll also want to do all this as tweb:
DROP PACKAGE FRIEDMAN;
DROP TABLE CTX_CDSTORES;
DROP TABLE CTX_CDSTORE_COLS;
DROP SEQUENCE CTX_CDSTORE_SEQ;
DROP VIEW CTX_USER_CDSTORES;
DROP VIEW CTX_USER_CDSTORE_COLS;
The other possibility is that you've modified the package so that it runs under "invoker rights" rather than "definer rights", using the AUTHID clause. This will not work.
Please let me know if this solves your problem.
- Roger

Fuzzy searching and concatenated datastore query performance problems.

I am using the concatenated datastore and indexing two columns.
The query I am executing includes an exact match on one column and a fuzzy match on the second column.
When I execute the query, performance should improve as the exact match column is set to return less values.
This is the case when we execute an exact match search on both columns.
However, when one column is an exact match and the second column is a fuzzy match this is not true.
Is this normal processing??? and why??? Is this a bug??
If you need more information please let me know.
We are under a deadline and this is our final road block.
TIA
Colleen GEislinger

I see that you have posted the message in the Oracle text forum, good! You should get a better, more timely answer there.
Larry

Scoring messed up using concatenated datastore Index

Hi,
Here is my table structure....
CREATE TABLE SRCH_KEYWORD_SEARCH_SME
SYS_ID NUMBER(10) NOT NULL,
PAPER_NO VARCHAR2(10),
PRODIDX_ID VARCHAR2(10),
RESULT_TITLE VARCHAR2(255),
RESULT_DESCR VARCHAR2(1000) NOT NULL,
ABSTRACT CLOB,
SRSLT_CATEGORY_ID VARCHAR2(10) NOT NULL,
SRSLT_SUB_CATEGORY_ID VARCHAR2(10) NOT NULL,
ACTIVE_FLAG VARCHAR2(1) DEFAULT 'Y' NOT NULL,
EVENT_START_DATE DATE,
EVENT_END_DATE DATE,
Here is the Concatenated Datastore preference...
   -- Drop any existing storage preference.
   CTX_DDL.drop_preference('SEARCH_STORAGE_PREF');
   -- Create new storage preference.
   CTX_DDL.create_preference('SEARCH_STORAGE_PREF', 'BASIC_STORAGE');
      CTX_DDL.set_attribute('SEARCH_STORAGE_PREF', 'I_TABLE_CLAUSE', 'tablespace searchidx');
      CTX_DDL.set_attribute('SEARCH_STORAGE_PREF', 'K_TABLE_CLAUSE', 'tablespace searchidx');
      CTX_DDL.set_attribute('SEARCH_STORAGE_PREF', 'R_TABLE_CLAUSE', 'tablespace searchidx lob (data) store as (disable storage in row cache)');
      CTX_DDL.set_attribute('SEARCH_STORAGE_PREF', 'N_TABLE_CLAUSE', 'tablespace searchidx');
      CTX_DDL.set_attribute('SEARCH_STORAGE_PREF', 'I_INDEX_CLAUSE', 'tablespace searchidx compress 2');
      CTX_DDL.set_attribute('SEARCH_STORAGE_PREF', 'P_TABLE_CLAUSE', 'tablespace searchidx');
   -- Drop any existing datastore preference.
   CTX_DDL.drop_preference('SEARCH_DATA_STORE');
   CTX_DDL.DROP_SECTION_GROUP('SEARCH_DATA_STORE_SG');
   -- Create new multi-column datastore preference.
   CTX_DDL.create_preference('SEARCH_DATA_STORE','MULTI_COLUMN_DATASTORE');
   CTX_DDL.set_attribute('SEARCH_DATA_STORE','columns','abstract, srslt_category_id, srslt_sub_category_id, active_flag');
   CTX_DDL.set_attribute('SEARCH_DATA_STORE', 'FILTER','N,N,N,N');
   -- Create new section group preference.
   CTX_DDL.create_section_group ('SEARCH_DATA_STORE_SG','BASIC_SECTION_GROUP');
   CTX_DDL.add_field_section('SEARCH_DATA_STORE_SG', 'abstract',              'abstract',             TRUE);
   CTX_DDL.add_field_section('SEARCH_DATA_STORE_SG', 'srslt_category_id',     'srslt_category_id',    TRUE);
   CTX_DDL.add_field_section('SEARCH_DATA_STORE_SG', 'srslt_sub_category_id', 'srslt_sub_category_id',TRUE);
   CTX_DDL.add_field_section('SEARCH_DATA_STORE_SG', 'active_flag',           'active_flag',          TRUE);
Here is the context Index
CREATE INDEX SRCH_KEYWORD_SEARCH_I ON SRCH_KEYWORD_SEARCH_SME(ABSTRACT)
   INDEXTYPE IS CTXSYS.CONTEXT
      PARAMETERS('STORAGE search_storage_pref DATASTORE SEARCH_DATA_STORE SECTION GROUP SEARCH_DATA_STORE_SG' )
Here is the Query # 1 I am trying out...
SELECT /*+ FIRST_ROWS(10) */
       SCORE(1) score_nbr,
       k.SYS_ID,
       k.RESULT_TITLE,
FROM   SRCH_KEYWORD_SEARCH_SME k
WHERE CONTAINS (k.ABSTRACT, '<query><textquery><progression><seq>{hitchhiker} WITHIN abstract</seq></progression></textquery></query>',1) > 0
ORDER BY SCORE(1) DESC;
Here is the result for Query # 1...
score_nbr   sys_id     result_title
54          99220      SME Releases New Book The Hitchhiker's Guide to Lean                                                                                                                                                                                                     72
43          116583     Lean Leadership Package                                                                                                                                                                                                                                         72
32          132392     The Hitchhikers Guide to Lean: Lessons from the Road                                                                                                                                                                                                           72
11          132017     Lean Manufacturing A Plant Floor Guide Book Summary                                                                                                                                                                                                            72
11          137106     Managing Factory Maintenance, Second Edition                                                                                                                                                                                                                    72
11          132082     Lean Pocket GuideHere is the Query # 2 I am trying out...
SELECT /*+ FIRST_ROWS(10) */
       SCORE(1) score_nbr,
       k.SYS_ID,
       k.RESULT_TITLE,
FROM   SRCH_KEYWORD_SEARCH_SME k
WHERE CONTAINS (k.ABSTRACT, '<query><textquery><progression><seq>{hitchhiker} WITHIN abstract AND Y WITHIN active_flag</seq></progression></textquery></query>',1) > 0
ORDER BY SCORE(1) DESC
Here is the result for Query # 2...
score_nbr sys_id     result_title
3         132017     Lean Manufacturing: A Plant Floor Guide Book Summary                                                                                                                                                                                                            72
3         137106     Managing Factory Maintenance, Second Edition                                                                                                                                                                                                                    72
3         132082     Lean Pocket Guide                                                                                                                                                                                                                                               72
3         132083     The Toyota Way: 14 Management Principles From the World's Greatest...                                                                                                                                                                                           72
3         132417     Lean Manufacturing: A Plant Floor Guide                                                                                                                                                                                                                         72
3         132091     Breaking the Cost Barrier: A Proven Approach to Managing and...                                                                                                                                                                                                 72
3         99318      Conflicting pairs                                                                                                                                                                                                                                               72
3         132393     One-Piece Flow: Cell Design for Transforming the Production Process                                                                                                                                                                                             72
3         137091     Learning to See: Value Stream Mapping to Create Value & Eliminate MUDA                                                                                                                                                                                          72
3         137090     The Purchasing Machine: How the Top 10 Companies Use Best Practices...                                                                                                                                                                                          72
3         137393     Passion for Manufacturing My question is, why did the scoring went all the way to 3 for ALL the results the above query returned when I used the AND clause
and added the 2nd column used in the datastore for my query condition..
Also I want to use progressive relaxation technique in the queries to use stemming & fuzzy search option too.
Help me out please....
Thanks in advance.
- Richard.

Yes, it's in the doc - it's known as the weight operator.
http://download.oracle.com/docs/cd/B28359_01/text.111/b28304/cqoper.htm#i998379
"term*n      Returns documents that contain term. Calculates score by multiplying the raw score of term by n, where n is a number from 0.1 to 10."
We're just using the operator twice as the limit on "n" is 10 (for no obvious reason I know of!). This is perfectly safe, and common practice.

Concatenated datastore fuzzy searches and performance...

Oracle 8.1.7:
I am using the concatenated datastore and indexing two columns.
The query I am executing includes an exact match on one column and a fuzzy match on the second column.
When I execute the query, performance should improve as the exact match column is set to return less values.
This is the case when we execute an exact match search on both columns.
However, when one column is an exact match and the second column is a fuzzy match this is not true.
Is this normal processing??? and why??? Is this a bug??
If you need more information please let me know.
We are under a deadline and this is our final road block.
TIA
Colleen GEislinger

This is more information about our scenario:
We have two groups in the datastore:
concat:
1.) hierarchy:(example text) 321826 325123 543123
2.) page: Actual document text.
321826 325123 543123 represents ids in a hierarchy structure. As you move from left to right the number of times the number occurs is less so there should be less exact matches.
Example: In this index all pages have 321826 as the first value. A few pages have 543123 and all others will have some other number as the last value.
if I do this query:
contains(concat,(321826 within hierarchy ) and ('personnel') within page)
it takes about 10 seconds because it 321826 will hit all pages.
if I do this query:
contains(concat,(543123 within hierarchy ) and ('personnel') within page)
it takes only about 1 second because it 543123 will hit just a few pages.
BUT:::::::
Fuzzy search....
if I do this query:
search A.) contains(concat,(321826 within hierarchy ) and ?('personnel') within page)
it takes about 30 seconds because it 321826 will hit all pages. This is okay for performance for this.
BUT if I do this query:
search B.) contains(concat,(543123 within hierarchy ) and ?('personnel') within page)
it takes about 30 seconds even though 543123 will hit only a few pages.
This should be faster than 30 seconds because you're searching over only a fraction of material for the fuzzy search part.
We've played with different variations on the () and the '' but nothing seems to change this.
Any advice on how to make search B.) faster??
We don't understand why see the different speeds in the exact match and we DON'T see the different speeds in the fuzzy search...
I can send you some test data with the index and query scripts if you want.
Our indexes are on large tables (2,000,000) rows.
TIA
Colleen Geislinger.

Problem with Concatened Datastore

Hello,
We try to implement the "Concatened Datastore". And we've got errors.
So we download the zip file from the link "Download the kit" on the page http://www.oracle.com/technology/sample_code/products/text/htdocs/concatenated_text_datastore/cdstore_readme.html#Installation. We unzip the file and got several files.
1) We try the "full_example" file, but the script "cdstore_10g_user" (line 12) don't exist ; sick.
2) So, to workaround, we try to follow the instructions in the "full_example.sql" file, with little corrections :-)). We create a user test_user, we load the cdstore.sql file, and here, we'got some errors. One for table they don't exist (ORA-00942 ; the first time it's normal. New tables don't exist), and one other for an invalid identifier (ORA-00904). Here the exact log :
Warning: Package Body created with compilation errors.
Errors for PACKAGE BODY CTX_CD:
LINE/COL ERROR
48/7 PL/SQL: SQL Statement ignored
51/16 PL/SQL: ORA-00904: "CDSTORE_NAME": invalid identifier
3) We continue the "full_example.sql" file. Create a table mytab for the test_user, insert data into this table, drop data into cdstore. And when we try to create a new cdstore, we've got this error :
SQL> prompt new...
begin
ctx_cd.create_cdstore('my2_cdstore', 'mytab');
end;
new...
SQL> 2 3 4 begin
ERROR at line 1:
ORA-04063: package body "TEST_USER.CTX_CD" has errors
ORA-06508: PL/SQL: could not find program unit being called: "TEST_USER.CTX_CD"
ORA-06512: at line 2
Is there anyone, who can tell us what's wrong ?
Best regards
Laurent PELLETIER

ATTENTION ROGER FORD
It looks like you need to make a few fixes to your full_example.sql and cdstore.sql. It looks like there are some mismatches between names of scripts and what is called and between names of tables created and selected from and so forth and some missing values for inserts. When I initially ran it with the cdstore.sql under the scott schema it ran o.k. because it was using the old definitions from the previous version, but then it wouldn't run under the test_user, so I suspect it was using the old definitions when you tested as well. After making the following corrections, it ran successfully with the test_user.
-- changes to full_example.sql:
--   changed password for system (not a bug)
--   changed @cdstore_10g_user to @cdstore
-- changes to cdstore.sql:
--   first:
--     changed each occurrence of ctx_cdstore to ctx_user_cdstore
--   then:
--     added column cdstore_name varchar2(30) to ctx_user_cdstore_cols table:
--       create table ctx_user_cdstore_cols (... cdstore_name varchar2(30) ...)
--     added cdstore_name and l_name to both inserts into ctx_user_cdstore_cols:
--       insert into ctx_user_cdstore_cols (... cdstore_name ...)
--       values (... l_name ...)

How do I get Oracle Text to index files on a file server?

I am new to Oracle (I'm a MS-SQL DBA looking for a Full-Text Search solution that is better than linking to a MS index server.)
So - Here's the objective:
I have Oracle Server(Express) installed on a Windows server.
I would like for Oracle to build a Full-Text Catalog of the files on a separate file server based on file paths in a table in the database.
(No desire to store terabytes of images and documents inside the database)
I can get Oracle text up and running, using the URL_Datastore:
CREATE TABLE files (id NUMBER PRIMARY KEY, issue_id NUMBER, path VARCHAR(255) UNIQUE, ot_format VARCHAR(6), ot_version VARCHAR(10));
The Compaq server is a remote windows server on my local workgroup, so the fully qualified path is just "compaq" and the URL is valid:
INSERT INTO files VALUES (9,9,'file://Compaq/FTQ/00000003.pdf',NULL,NULL);
INSERT INTO files VALUES (13,13,'file://Compaq/FTQ/01.txt',NULL,NULL);
CREATE INDEX file_index ON files(path) INDEXTYPE IS ctxsys.context
PARAMETERS ('datastore ctxsys.URL_DATASTORE format column ot_format');
but when I enter:
Select * from CTX_User_Index_errors, I see the following errors:
DRG-11609: URL store: unable to open local file specified by file://Compaq/FTQ/00000003.pdf
DRG-11609: URL store: unable to open local file specified by file://Compaq/FTQ/01.txt
Did I miss something?
Do I need to install anything on the file server?
I would like to convince my company that Oracle can be much quicker than Microsoft's Indexing Service because it can avoid joining two large result sets (one result set from Full_text (indexing service) and one for specific data contained in fields in the MS-SQL database.) Full Text Searches commonly take 40 - 60 seconds where there are 1.5 million multi-page PDF files for a particular set that I sample search on. Without this massive join, I believe I can get the search to run in under 10 seconds.

Thank you!
File_Datastore worked fine.
I was staying away from File_Datastore because the information I gathered from googling suggested that file_datastore would only work locally.
Now I just have to get Oracle to pull data out of tables in a MS-SQL database on the local network (don't have a clue yet), and then have it index compiled file paths.
Then MS-SQL can query Oracle with index and full-text criteria and Oracle can send back a result set
It may sound like a bad way of performing Full-Text Queries, but anything will be better than the way things are currently running. We are currently performing Full Text Searches on a table that is rebuilt nightly, so the table containing millions of file paths is not live..
It would be so much better if we just migrated to Oracle, but we currently do not have the resources.

Using Oracle Text to search through WORD, EXCEL and PDF documents

Hello again,
What I would like to know is if I have a WORD or PDF document stored in a table. Is it possible to use Oracle Text to search through the actual WORD or PDF document?
Thanks
Doug

Yes you can do context sensitive searches on both PDF and Word docs. With the PDF you need to make sure they are text and not images. Some scanners will create PDFs that are nothing more than images of document.
Below is code sample that I made some time back to demonstrate the searching capabilities of Oracle Text. Note that the example makes use of the inso_filter that is no longer shipped with Oracle begging with Patch set 10.1.0.4. See metalink note 298017.1 for the changes. See the following link for more information on developing with Oracle Text.
http://download-west.oracle.com/docs/cd/B14117_01/text.101/b10729/toc.htm
begin example.
-- The following needs to be executed
-- as sys.
DROP DIRECTORY docs_dir;
CREATE OR REPLACE DIRECTORY docs_dir
AS 'C:\sql\oracle_text\documents';
GRANT READ ON DIRECTORY docs_dir TO text;
-- End sys ran SQL
DROP TABLE db_docs CASCADE CONSTRAINTS PURGE;
CREATE TABLE db_docs (
id NUMBER,
format VARCHAR2(10),
location VARCHAR2(50),
document BLOB,
CONSTRAINT i_db_docs_p PRIMARY KEY(id)
-- Several notes need to be made about this anonymous block.
-- First the 'DOCS_DIR' parameter is a directory object name.
-- This directory object name must be in upper case.
DECLARE
f_lob BFILE;
b_lob BLOB;
document_name VARCHAR2(50);
BEGIN
document_name := 'externaltables.doc';
INSERT INTO db_docs
VALUES (1, 'binary', 'C:\sql\oracle_text\documents\externaltables.doc', empty_blob())
RETURN document INTO b_lob;
f_lob := BFILENAME('DOCS_DIR', document_name);
DBMS_LOB.FILEOPEN(f_lob, DBMS_LOB.FILE_READONLY);
DBMS_LOB.LOADFROMFILE(b_lob, f_lob, DBMS_LOB.GETLENGTH(f_lob));
DBMS_LOB.FILECLOSE(f_lob);
COMMIT;
END;
-- build the index
-- Note that this index differs than the file system stored file
-- in that paramter datastore is ctxsys.defautl_datastore and not
-- ctxsys.file_datastore. FILE_DATASTORE is for documents that
-- exist on the file system. DEFAULT_DATASTORE is for documents
-- that are stored in the column.
create index db_docs_ctx on db_docs(document)
indextype is ctxsys.context
parameters (
'datastore ctxsys.default_datastore
filter ctxsys.inso_filter
format column format');
--search for something that is known to not be in the document.
SELECT SCORE(1), id, location
FROM db_docs
WHERE CONTAINS(document, 'Jenkinson', 1) > 0;
--search for something that is known to be in the document.
SELECT SCORE(1), id, location
FROM db_docs
WHERE CONTAINS(document, 'Albright', 1) > 0;

Oracle Text and APEX

Hello
Tried the Oracle White Paper - Oracle Text Web Applications
Created the table and populated with relevant url links
create table htmldb_documentation(
id number,
doc_title varchar2(4000),
doc_url varchar2(4000))
then created the index
create index htmldb_doc_ctxidx on htmldb_documentation(doc_url)
indextype is ctxsys.context
parameters ('datastore CTXSYS.URL_DATASTORE')
Then ran my SQL for the report in Toad and APEX SQL Workshop>SQL Commands before creating an APEX Region based on a SQL Report
select score(1) relevance, doc_title, doc_url
from htmldb_documentation
where CONTAINS (doc_url, :P1_SEARCH, 1) > 0
order by 1 desc
After running the APEX Report I get error
report error:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50901: text query parser syntax error on line 1, column 1
I also ran these grant commands after I received this error
grant ctxapp to demo;
grant execute on ctx_cls to demo;
grant execute on ctx_ddl to demo;
grant execute on ctx_doc to demo;
grant execute on ctx_output to demo;
grant execute on ctx_query to demo;
grant execute on ctx_report to demo;
grant execute on ctx_thes to demo;
grant execute on ctx_ulexer to demo;
Any ideas ?? I'm running APEX 3.1.0.00.32 on Oracle 10.2.0.1 on WindowsXP
If I replace the bind variable :P1_SEARCH, with a literal value the error disappears

Couple of things to check:
1) do you have an item called P1_SEARCH in your application?
2) If so, make sure that it has a value; otherwise, Oracle Text gets confused and will throw that error.
You may want to consider using a PL/SQL Function Returning SQL Query that will only append the CONTAINS clause if P1_SEARCH has some value.
Thanks,
- Scott -
http://sumnertechnologies.com
http://spendolini.blogspot.com

How to use Oracle text

I'm storing files in a blob field in a 9i database, sometimes I need to query using the details stored in the database about the file and sometimes I need to search the files to find matches with some text (like search engine), I was told that oracle text can help me accomplish this functionality , however I don't know if it supports arabic text and I don't know how to use it from my application developed in 9i.
Regards.

Friend by using these step you can easily use Oracle inter text media
j a h a n z e b
[email protected]
Oracle Developer
6th Floor, State Bank of Pakistan
I.I.Chundrigar Road, Karachi.
Please note that in SqlPlus you can use '?' in stead of $ORACLE_HOME, and this works on Unix and Windows so if you want to execute $ORACLE_HOME/rdbms/admin/catalog.sql you can simply use:
on Unix sql> @?/rdbms/admin/catalog.sql
on Windows sql> @?\rdbms\admin\catalog.sql
5.2.1 Explanation of installation steps
1. Connected to database as SYSDBA and create CTXSYS user:
Ctxsys user is created by calling following script:
@?/ctx/admin/dr0csys.sql <ctxsys> <system> <temp>
Where:
change_on_install - is the ctxsys user password
DRSYS - is the default tablespace for ctxsys
TEMP - is the temporary tablespace for ctxsys
This will create user CTXSYS and grants full privileges to CTXSYS in order to create and insert into result tables, execute callbacks, rewrite queries, and perform system cleanup. At this point CTXSYS will not own any objects.ss
2. Connected to database as CTXSYS and create all necessary objects
All necessary object are creates by calling following script:
connect CTXSYS/change_on_install
@?/ctx/admin/dr0inst <replace with $ORACLE_HOME>/ctx/lib/libctxx9.so;
Please not that you have to put full path to your ORACLE_HOME, for example home as paramter
On Solaris/Aix/Linux with $ORACLE_HOME of /u01/app/oracle/product/8.1.7
@?/ctx/admin/dr0inst.sql /u01/app/oracle/product/8.1.7/ctx/lib/libctxx8.so
On HP-UX with $ORACLE_HOME of /u01/app/oracle/product/8.1.7
@?/ctx/admin/dr0inst.sql /u01/app/oracle/product/8.1.7/ctx/lib/libctxx8.sl
Windows NT/2000 with D:\oracle\product\8.1.7
@?/ctx/admin/dr0inst.sql D:\oracle\product\8.1.7\bin\oractxx8.dll
This will installs all Oracle database objects required by the Oracle Text system. This includes:
a) Data dictionary tables, views, sequence, packages
b) Server management tables, views and packages
c) Dispatcher packages
d) Service queue objects
3) Install appropriate language-specific default preferences.
The next step is to install appropriate language-specific default preferences.When you use CREATE INDEX to create an index or ALTER INDEX to manage an index, you can optionally specify indexing preferences in the parameter string. There are seven preference classes:
- Lexer, defines the language being indexed. ( language specific )
- Wordlist, defines the expantion of stem and fuzzy queries. ( language specific )
- Stoplist, defines words and themes that are not be indexed. ( language specific )
- Datastore, defines document storage.
- Filter, defines standards for converion of documents to plaintext.
- Storage, defines the storage of the index tables.
- Section group, enables possibilities to define document sections.
There is script which creates language-specific default preferences for every language Oracle text supports in <ORACLE_HOME>/ctx/admin/defaults directory, such as English(US), Danish(DK), Dutch(NL), Finnish(SF), French(FR), German(DE), Italian(IT), Portuguese(PR), Spanish(ES), and Swedish(S). They are named in the form drdefXX.sql, where XX is the language code. To manually install US default preferences, for example, log into sqlplus as CTXSYS, and run 'drdefus.sql' as described below:
@?/ctx/admin/defaults/drdefus.sql
create user textuser identified by textuser
default tablespace users
temporary tablespace temp;
-- You must grant 'ctxapp' role to textuser
grant connect, resource, ctxapp to textuser;
connect textuser/textuser
drop table quick;
create table quick (
quick_id number
constraint quick_pk primary key,
text varchar2(80) );
insert into quick ( quick_id, text ) values (1,'The cat sat on the mat');
insert into quick ( quick_id, text ) values (2,'The quick brown fox jumps over the lazy dog' );
insert into quick ( quick_id, text ) values (3,'The dog barked like a dog');
commit;
create index quick_text on quick ( text )
indextype is ctxsys.context;
col text format a45
col s format 999
select text, score(42) s from quick
where contains ( text, 'dog', 42 ) > 0
order by s desc;

Problem full-text in blob column index created using Oracle Text

Hi,
I'm running Oracle Database 10g 10.2 on solaris
I configure Oracle text if i look for in a varchar2 column is ok, but with blob column doesn't works the search.
I have a table with a blob column which contains document. I load document with Oracle UCM (stellent)
My index scripts is:
CREATE INDEX ORAUCM.FT_IDCCOLL1 ON ORAUCM.IDCCOLL1
(DDOCFULLTEXT)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('DATASTORE CTXSYS.DEFAULT_DATASTORE FILTER CTXSYS.AUTO_FILTER FORMAT COLUMN DFULLTEXTFORMAT CHARSET
COLUMN DFULLTEXTCHARSET LEXER OCS_IDCCOLL1_LEXER SYNC (ON COMMIT)')
NOPARALLEL;
And my select retunm 0 rows although it will be many documents:
SELECT IdcColl2.dID, dDocName, dDocTitle, dDocType, dRevisionID, dSecurityGroup, dDocAuthor,
dDocAccount, dRevLabel, dFormat, dOriginalName, dExtension, dWebExtension, dInDate, dOutDate,
dPublishType, dRendition1, dRendition2, VaultFileSize, WebFileSize, URL, dFullTextFormat,
dFullTextCharset, DocMeta.*
FROM IdcColl1, DocMeta
WHERE IdcColl1.dID=DocMeta.dID AND (CONTAINS(dDocFullText,'SUBIR') > 0 )
ORDER BY dInDate Desc
Thanks in advance.

Thank you for your answer.
I response your question:
- yes DDOCFULLTEXT is a BLOB column.
- The document that word, excels, whatever. We load the document with UCM (universal Content Management)
because i need full-test search form UCM tool.
- Yes 'subir' containts in the word document.
- select * from CTX_USER_INDEX_ERRORS ;
No rows returned.
- SELECT TOKEN_TEXT FROM DR$FT_IDCCOLL1$I
No rows returned.
- I tried create symplifying index and doen't work.
I tried create table and index context on oracle 10.2.0.3 (test database)and works ok.
I compared both context (test database and ucm database) and i saw a difference:
In ucm database there are these preferences "analyze text"
BEGIN ctx_ddl.create_preference('ORAUCM.', 'WORLD_LEXER'); end;
BEGIN ctx_ddl.create_preference('ORAUCM.', 'DETAIL_DATASTORE'); end;
I don't know if is important diference or no.
Please if you need more information, tell me.
Thanks for your time.

Oracle Text - Problem with filtering binary documents (.doc, .pdf, etc...)

Hi, I have a problem with filtering binary documents (.doc, .pdf, etc...). I use SQL*PLUS for remote access to Oracle 10.2 on Linux and I create table:
CREATE TABLE test (id NUMBER PRIMARY KEY, text VARCHAR2(100));
I insert to this table:
INSERT into test values(1, 'PATH/text1.doc‘);
INSERT into test values(2,'PATH/text2.doc‘);
and then:
CREATE INDEX test_index ON test(text) indextype is ctxsys.context
parameters (’datastore ctxsys.file_datastore
filter ctxsys.auto_filter’);
Message "Index created" is displayed, but objects: DR$test_index$I, DR$test_index$K, DR$test_index$N, DR$test_index$R and DR$test_index$P are empty => index wasn´t created probably.
I don´t know, where is bug, either bug is somewhere in this code or on the server (wrong installation oracle or constraint privileges). Do you know in what is bug?

The following is an excerpt from the 10g online documentation. Note the items that I have put in bold.
"FILE_DATASTORE
The FILE_DATASTORE type is used for text stored in files accessed through the local file system.
Note:
FILE_DATASTORE may not work with certain types of remote mounted file systems.
FILE_DATASTORE has the following attribute(s):
Table 2-4 FILE_DATASTORE Attributes
Attribute Attribute Value
path path1:path2:pathn
path
Specify the full directory path name of the files stored externally in a file system. When you specify the full directory path as such, you need only include file names in your text column.
You can specify multiple paths for path, with each path separated by a colon (:) on UNIX and semicolon(;) on Windows. File names are stored in the text column in the text table.
If you do not specify a path for external files with this attribute, Oracle Text requires that the path be included in the file names stored in the text column.
PATH Attribute Limitations
The PATH attribute has the following limitations:
If you specify a PATH attribute, you can only use a simple filename in the indexed column. You cannot combine the PATH attribute with a path as part of the filename. If the files exist in multiple folders or directories, you must leave the PATH attribute unset, and include the full file name, with PATH, in the indexed column.
On Windows systems, the files must be located on a local drive. They cannot be on a remote drive, whether the remote drive is mapped to a local drive letter."
With accessible paths and files, you get something like:
SCOTT@orcl_11g> CREATE TABLE test (id NUMBER PRIMARY KEY, text VARCHAR2(100));
Table created.
SCOTT@orcl_11g>
SCOTT@orcl_11g>
SCOTT@orcl_11g> INSERT into test values(1,'c:\oracle11g\banana.pdf');
1 row created.
SCOTT@orcl_11g> INSERT into test values(2,'c:\oracle11g\cranberry.pdf');
1 row created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> CREATE INDEX test_index ON test(text) indextype is ctxsys.context
2 parameters ('datastore ctxsys.file_datastore
3 filter ctxsys.auto_filter');
Index created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> select count(*) from dr$test_index$i
2 /
COUNT(*)
608
SCOTT@orcl_11g> In the following, I used a non-existent path and non-existent file name, which produces the same results as when you use a remote path that does not exist locally.
SCOTT@orcl_11g> CREATE TABLE test (id NUMBER PRIMARY KEY, text VARCHAR2(100));
Table created.
SCOTT@orcl_11g>
SCOTT@orcl_11g>
SCOTT@orcl_11g> INSERT into test values(3,'c:\nosuchpath\nosuchfile.pdf');
1 row created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> CREATE INDEX test_index ON test(text) indextype is ctxsys.context
2 parameters ('datastore ctxsys.file_datastore
3 filter ctxsys.auto_filter');
Index created.
SCOTT@orcl_11g>
SCOTT@orcl_11g> select count(*) from dr$test_index$i
2 /
COUNT(*)
0
SCOTT@orcl_11g>

Oracle Text omitting the words as, is, do, or etc in searches

I am trying to do a search using Oracle Text for the country code for Dominican Republic and I get no hits back. This is also the case for some other countries: IS (Iceland), NO (Norway), IT (Italy). Here is my where clause:
where contains(tablename, 'DO within country')
Is there any workaround for this?

Those words are stopwords, words that Oracle does not tokenize and index and are not searchable. If you do not specify a stoplist, then Oracle uses the default stoplist, that contains these words. If you use an empty stoplist or a stoplist that does not contain those words, then they are tokenized, inexed, and searchable. Please see the demonstration below that first recreates your situation, or something similar since I don't know what sort of section group you are using, then drops the index and recreates it using an empty stoplist, then shows that DO is tokenized, indexed, and searchable.
SCOTT@10gXE> CREATE TABLE your_table
2    (address   VARCHAR2 (30),
3      country   VARCHAR2 (2),
4      tablename VARCHAR2 (30))
5 /
Table created.
SCOTT@10gXE> INSERT INTO your_table VALUES ('somewhere', 'DO', NULL)
2 /
1 row created.
SCOTT@10gXE> EXEC CTX_DDL.CREATE_PREFERENCE ('my_multi', 'MULTI_COLUMN_DATASTORE')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.SET_ATTRIBUTE ('my_multi', 'COLUMNS', 'address, country')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.CREATE_SECTION_GROUP ('my_section_group', 'BASIC_SECTION_GROUP')
PL/SQL procedure successfully completed.
SCOTT@10gXE> EXEC CTX_DDL.ADD_FIELD_SECTION ('my_section_group', 'country', 'country')
PL/SQL procedure successfully completed.
SCOTT@10gXE> CREATE INDEX your_index ON your_table (tablename)
2 INDEXTYPE IS CTXSYS.CONTEXT
3 PARAMETERS
4    ('DATASTORE     my_multi
5       SECTION GROUP my_section_group')
6 /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$your_index$i
2 /
TOKEN_TEXT
SOMEWHERE
SCOTT@10gXE> SELECT * FROM your_table
2 WHERE CONTAINS (tablename, 'DO WITHIN country') > 0
3 /
no rows selected
SCOTT@10gXE> DROP INDEX your_index
2 /
Index dropped.
SCOTT@10gXE> CREATE INDEX your_index ON your_table (tablename)
2 INDEXTYPE IS CTXSYS.CONTEXT
3 PARAMETERS
4    ('DATASTORE     my_multi
5       SECTION GROUP my_section_group
6       STOPLIST CTXSYS.EMPTY_STOPLIST')
7 /
Index created.
SCOTT@10gXE> SELECT token_text FROM dr$your_index$i
2 /
TOKEN_TEXT
DO
SOMEWHERE
SCOTT@10gXE> SELECT * FROM your_table
2 WHERE CONTAINS (tablename, 'DO WITHIN country') > 0
3 /
ADDRESS                        CO TABLENAME
somewhere                      DO
SCOTT@10gXE>

Oracle Text USER_LEXERについて

Oracle Text を USER_LEXER で使用したいと考えております。
後述のサイトを参考にプロシージャをcreateしようとしたのですが
以下のエラーになってしまい実行出来ませんでした。
対処方法ご存じでしたら教えて頂けないでしょうか。
＠エラー内容
PROCEDURE QUERY_OTのエラーです。
LINE/COL ERROR
0/0 PL/SQL: Compilation unit analysis terminated
4/20 PLS-00904:
オブジェクト'PUBLIC.CTX_ULEXER'にアクセスするには権限が不十分です
＠実行したSQL文
CREATE OR REPLACE PROCEDURE query_ot
p_target IN VARCHAR2,
p_tab IN CTX_ULEXER.WILDCARD_TAB,
p_result IN OUT VARCHAR2
AUTHID CURRENT_USER
IS
cursor cur_oti is select token from oti;
BEGIN
p_result := '<TOKENS>';
for now in cur_oti loop
if instr(p_target, now.token) > 0 then
p_result := p_result || '<WORD>' || now.token || '</WORD>';
end if;
end loop;
p_result := p_result || '</TOKENS>';
END query_ot;
＠参考サイト
http://www.nacky.info/wiki/index.php?OracleText
＠環境
OS: Ubuntu 12.04LTS
Oracle: 11.2.0-1.0 (64ビット版)
Edited by: 955082 on 2012/08/26 20:42

ご回答ありがとうございます。
FORMAT COLUMNとUSER_DATASTOREを試したところ、
当方の望みの動作を確認出来ました。
（マテリアライズド・ビューは後に確認させて頂きます）
ただ、検索対象のデータを更新したの索引の同期が取れず
また問題につまずいてしまいました。
度々で恐れ入りますが、同期の方法ももしよろしければご教授頂けないでしょうか。
USER_DATASTOREを実装したSQLと確認手順は以下の通りです。
＠データやプロシージャ作成SQL
--検索対象テーブル
drop table ot;
create table ot
( id number primary key,
text varchar2(80),
del_flg number,
type varchar2(80) );
insert into ot ( id, text, del_flg, type ) values ( 1, 'The cat sat on the mat', 0, 'TEXT' );
insert into ot ( id, text, del_flg, type ) values ( 2, 'The dog barked like a dog', 0, 'TEXT' );
insert into ot ( id, text, del_flg, type ) values ( 3, '日本オラクル株式会社', 0, 'IGNORE' );
insert into ot ( id, text, del_flg, type ) values ( 4, '全日本自動車協会', 1, 'TEXT' );
commit;
-- 辞書テーブル
drop table oti;
create table oti
( token varchar2(80) );
insert into oti ( token ) values ( '日本' );
insert into oti ( token ) values ( 'オラクル' );
insert into oti ( token ) values ( '日本オラクル' );
insert into oti ( token ) values ( '自動車協会' );
insert into oti ( token ) values ( 'Ora' );
commit;
-- 索引作成プロシージャ
CREATE OR REPLACE PROCEDURE ind_ngram
v_a IN VARCHAR2,
v_b IN OUT VARCHAR2,
v_c IN BOOLEAN
IS
cursor cur_oti is select token from oti;
BEGIN
v_b := '<tokens>';
for now in cur_oti loop
if instr(v_a, now.token) > 0 then
v_b := v_b || '<word>' || now.token || '</word>';
end if;
end loop;
v_b := v_b || '</tokens>';
END ind_ngram;
--索引検索プロシージャ
CREATE OR REPLACE PROCEDURE que_ngram
v_a IN VARCHAR2,
v_b IN CTX_ULEXER.WILDCARD_TAB,
v_c IN OUT VARCHAR2
IS
cursor cur_oti is select token from oti;
BEGIN
v_c := '<tokens>';
for now in cur_oti loop
if instr(v_a, now.token) > 0 then
v_c := v_c || '<word>' || now.token || '</word>';
end if;
end loop;
v_c := v_c || '</tokens>';
END que_ngram;
--データストアプロシージャ
create or replace procedure myproc(rid in rowid, ret in out nocopy varchar2) is
begin
ret := null;
for c1 in (select text from ot
where rowid = rid and del_flg = 0)
loop
ret := c1.text;
end loop;
end;
--CTX_DDL
BEGIN
CTX_DDL.drop_PREFERENCE('my_lexer');
CTX_DDL.CREATE_PREFERENCE('my_lexer', 'user_lexer');
CTX_DDL.SET_ATTRIBUTE('my_lexer', 'index_procedure', 'ind_ngram');
CTX_DDL.SET_ATTRIBUTE('my_lexer', 'input_type', 'varchar2');
CTX_DDL.SET_ATTRIBUTE('my_lexer', 'query_procedure', 'que_ngram');
END;
BEGIN
CTX_DDL.drop_STOPLIST('my_stoplist');
CTX_DDL.CREATE_STOPLIST('my_stoplist', 'basic_stoplist');
END;
BEGIN
ctx_ddl.drop_preference('my_datastore');
ctx_ddl.create_preference('my_datastore', 'user_datastore');
ctx_ddl.set_attribute('my_datastore', 'procedure', 'myproc');
ctx_ddl.set_attribute('my_datastore', 'output_type', 'varchar2');
END;
--索引作成
CREATE INDEX a ON ot(text)
INDEXTYPE IS ctxsys.context
PARAMETERS
LEXER my_lexer
STOPLIST my_stoplist
datastore my_datastore
＠確認手順１：索引を表示
# USER_DATASTORE実装前はid=3とid=4のデータが索引に格納されていたが、
# id=4のdel_flgを1に設定してINSERTしたためid=3 のデータのみが格納されている(正常)
SQL> select token_text from dr$a$i;
TOKEN_TEXT
オラクル
日本
日本オラク
＠確認手順２：id=3のdel_flgをオン
SQL> update ot set del_flg=1 where id=3;
1行が更新されました
SQL> commit;
コミットが完了しました。
＠確認手順３：同期コマンドを実行
SQL> exec CTX_DDL.SYNC_INDEX('a');
PL/SQLプロシージャが正常に完了しました。
SQL> exec CTX_DDL.OPTIMIZE_INDEX('a','full');
PL/SQLプロシージャが正常に完了しました。
＠確認手順４：再び索引を表示
# id=3のdel_flgに1を立てていたため索引はでないはず・・・だが表示されてしまった。更新されていない模様(異常)
SQL> select token_text from dr$a$i;
TOKEN_TEXT
オラクル
日本
日本オラクル
---

Oracle text indexed view is possible

Oracle text indexed view is possible???

ok,
My table name is T_DOC :
ID----------------> NUMBER(30)
DESCRIPTION-------> VARCHAR2(2000 BYTE)
DOC---------------> BLOB
FILENAME----------> VARCHAR2(2000 BYTE)
MIMETYPE----------> VARCHAR2(2000 BYTE)
LAST_UPDATE_DATE--> DATE
T_DOC
| Id | DESCRIPTION | DOC | FILENAME | MIMETYPE | LAST_UPDATE_DATE |
| 1 | THE DOG | *(!BLOB) | THE_CAT.PDF | application/pdf | 20/05/2010 15:06:15 |
| 2 | THE BIRD | **(!BLOB) | THE_BIRD.PDF | application/pdf | 20/05/2010 15:06:15 |
| 3 | THE HUMAN AND CAT | ***(!BLOB) | THE_HUMAN.PDF | application/pdf | 20/05/2010 15:06:15 |
* is a document .pdf with content: "the dog and cat"
** is a document .pdf with content: "the bird in house"
*** is a document .pdf with content: "the human from USA"
Index the columns DESCRIPTION, DOC (document content), FILENAME
begin
ctx_ddl.create_preference('idxDoc_lx', 'BASIC_LEXER');
ctx_ddl.set_attribute (' idxDoc_lx ', 'MIXED_CASE', 'NO');
end;
begin
ctx_ddl.create_preference('idxDoc_ds', 'MULTI_COLUMN_DATASTORE');
ctx_ddl.set_attribute ('idxDoc_ds', 'COLUMNS', 'DOC, FILENAME, DESCRIPTION');
end;
CREATE INDEX IDX_DOC
ON T_DOC (FILENAME)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS ('lexer idxDoc_lx
datastore idxDoc_ds
filter CTXSYS.AUTO_FILTER
sync (on commit)');Search Query:
select ID
from T_DOC
where CONTAINS (DOCUMENTO, 'CAT', 1) > 0 RESULT ID = 1
WHY NOT ALSO Returned ID 3 ??????

Oracle Text Concatenated Datastore

Similar Messages

Maybe you are looking for