Index creation in Oracle Text ???

Hi,
I have never used oracle text before.
The background info is got corruption errors on database alert log, identified it is an oracle9.2.0.5 bug in aix 5.2, need to apply 9.2.0.6 patchset. Temporary solution provided by oracle support was to drop and recreate an index.
My approach was to identify the index name, associated column, reverse engineer the object ddl, drop the index and re-create the index. Should have been straight forward. The difference is the column is of BLOB datatype. My index script failed with ORA-02327. I get errors on alert log saying missing index.
Don't know how to re-create the index on BLOB column in a table.
Any help, much appreciated.
Thanks
Murali

Hi,
For your recreation, check out CTX_REPORT.CREATE_INDEX_SCRIPT. It will generate all of the ddl that you need.
Thanks,
Ron

Similar Messages

Index rules in oracle text and query using matches

Dear All,
I would like to ask about rules and matches function in oracle text.
I followed an example in oracle text application developer's guide.
I have a rule table like this :
1 oracle
2 larry or ellison
3 oracle and text
4 market share
then, I create an index to that table. This is needed for calling matches function. Here is the syntax :
create index queryx on queries(query_string)
indextype is ctxsys.ctxrule;
then, I noticed that the result on DR$QUERYX$I table as follows :
LARRY 0 2 2 1 (BLOB)
MARKET 0 4 4 1 (BLOB) {MARKET} {SHARE}
ORACLE 0 1 1 1 (BLOB)
ORACLE 0 3 3 1 (BLOB) {TEXT}
ELLISON 0 2 2 1 (BLOB)
What I want to ask is why doesn't the words 'share' and 'text' appear in the DR$QUERYX$ table?
When we use matches function, it then search on the index result and consequently it wion't find the 'share' word. so when for example I do query like this :
select query_id from queries where matches(query_string,' It only share ten percent of all products sold')>0
it will give 0 result since the no word in ' It only share ten percent of all products sold' was in index table. But actually it could possibly be categorized as the 4 category which rules is 'market share'
I tried this in a larger set of data and get same result.
Here is my generated rules from my document collection :
1 {REQUIREMENTS} & {ELICITATION}
1 {REQUIREMENTS} ~ {ELICITATION} & {ACTOR}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} & {FURPS}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} & {PROC}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} & {SPEED}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} & {DOCUME}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} ~ {DOCUME} & {PLACED}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} ~ {DOCUME} ~ {PLACED} & {UNNECESSARY}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} ~ {DOCUME} ~ {PLACED} ~ {UNNECESSARY} & {MISUSE}
1 {INTERPRETATION} ~ {REQUIREMENTS}
2 {DESIGN} & {REPRESENTATION}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} & {OCTOBER}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} & {PROCEDURAL}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} & {STRICT}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} ~ {STRICT} & {GRASP}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} ~ {STRICT} ~ {GRASP} & {MANY} & {LAYER}
2 {DESIGN} ~ {REPRESENTATION} ~ {MAY}
3 {PM} & {TESTING} & {ATTRIBUTI}
And this is the index table result with ctxrule :
(only the token_text column shown)
PM
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
INTERPRETATION
so when I try to classify a document with the word ouline inside it, it should produce category 1 (based on the rules) but since there are no word 'outline' in index tabel, the matches will return 0 means that the document is not classifiedto any category. I don't understand why it happen. Anybody knows about this? I would really appreciate any help.
Thank you very much.

Hm, I see. It do make sense. so nice to know.
But then in the second example I gift where I used larger table, as shown below :
Here is my generated rules from my document collection :
1 {REQUIREMENTS} & {ELICITATION}
1 {REQUIREMENTS} ~ {ELICITATION} & {ACTOR}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} & {FURPS}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} & {PROC}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} & {SPEED}
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE} ~ {PROC} ~ {SPEED} & {DOCUME}
1 {INTERPRETATION} ~ {REQUIREMENTS}
2 {DESIGN} & {REPRESENTATION}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} & {OCTOBER}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} & {PROCEDURAL}
2 {DESIGN} ~ {REPRESENTATION} & {MAY} & {FOUNDATI} ~ {OCTOBER} ~ {PROCEDURAL} & {STRICT}
2 {DESIGN} ~ {REPRESENTATION} ~ {MAY}
3 {PM} & {TESTING} & {ATTRIBUTI}
As far as I know, the sign ' ~ ' means 'OR' and '&' means 'and' . So based on the 4th line in my table :
1 {REQUIREMENTS} ~ {ELICITATION} ~ {ACTOR} ~ {FURPS} ~ {OUTLINE}
it can be concluded that if any of the words stated there been queried, so the category '1' will appear as a result. But then before we can use 'matches' to query it, we need ti create index for the rules table . I did it and the result were :
(only the token_text column shown)
PM
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
DESIGN
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
REQUIREMENTS
INTERPRETATION
there were no words other than PM, DESIGN< REQUIREMENTS and INTERPRETATION. Why the words REQUIREMENTS, ELICITATION, ACTOR, FURPS, OUTLINE don't appear in the index result?

Problem full-text in blob column index created using Oracle Text

Hi,
I'm running Oracle Database 10g 10.2 on solaris
I configure Oracle text if i look for in a varchar2 column is ok, but with blob column doesn't works the search.
I have a table with a blob column which contains document. I load document with Oracle UCM (stellent)
My index scripts is:
CREATE INDEX ORAUCM.FT_IDCCOLL1 ON ORAUCM.IDCCOLL1
(DDOCFULLTEXT)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('DATASTORE CTXSYS.DEFAULT_DATASTORE FILTER CTXSYS.AUTO_FILTER FORMAT COLUMN DFULLTEXTFORMAT CHARSET
COLUMN DFULLTEXTCHARSET LEXER OCS_IDCCOLL1_LEXER SYNC (ON COMMIT)')
NOPARALLEL;
And my select retunm 0 rows although it will be many documents:
SELECT IdcColl2.dID, dDocName, dDocTitle, dDocType, dRevisionID, dSecurityGroup, dDocAuthor,
dDocAccount, dRevLabel, dFormat, dOriginalName, dExtension, dWebExtension, dInDate, dOutDate,
dPublishType, dRendition1, dRendition2, VaultFileSize, WebFileSize, URL, dFullTextFormat,
dFullTextCharset, DocMeta.*
FROM IdcColl1, DocMeta
WHERE IdcColl1.dID=DocMeta.dID AND (CONTAINS(dDocFullText,'SUBIR') > 0 )
ORDER BY dInDate Desc
Thanks in advance.

Thank you for your answer.
I response your question:
- yes DDOCFULLTEXT is a BLOB column.
- The document that word, excels, whatever. We load the document with UCM (universal Content Management)
because i need full-test search form UCM tool.
- Yes 'subir' containts in the word document.
- select * from CTX_USER_INDEX_ERRORS ;
No rows returned.
- SELECT TOKEN_TEXT FROM DR$FT_IDCCOLL1$I
No rows returned.
- I tried create symplifying index and doen't work.
I tried create table and index context on oracle 10.2.0.3 (test database)and works ok.
I compared both context (test database and ucm database) and i saw a difference:
In ucm database there are these preferences "analyze text"
BEGIN ctx_ddl.create_preference('ORAUCM.', 'WORLD_LEXER'); end;
BEGIN ctx_ddl.create_preference('ORAUCM.', 'DETAIL_DATASTORE'); end;
I don't know if is important diference or no.
Please if you need more information, tell me.
Thanks for your time.

Problem with blob column index created using Oracle Text.

Hi,
I'm running Oracle Database 10g 10.2.0.1.0 standard edition one, on windows server 2003 R2 x64.
I have a table with a blob column which contains pdf document.
Then, I create an index using the following script so that I can do fulltext search using Oracle Text.
CREATE INDEX DMCS.T_DMCS_FILE_DF_FILE_IDX ON DMCS.T_DMCS_FILE
(DF_FILE)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('DATASTORE CTXSYS.DEFAULT_DATASTORE');
However, the index is not searchable and I check the following tables created by database for my index and found them to be empty as well !!
DR$T_DMCS_FILE_DF_FILE_IDX$I
DR$T_DMCS_FILE_DF_FILE_IDX$K
DR$T_DMCS_FILE_DF_FILE_IDX$N
DR$T_DMCS_FILE_DF_FILE_IDX$R
I wonder what's wrong with it.
My user has been granted the ctx_app role and I have other tables that store plain text which I use Oracle Text are fine. I even output the blob column and save as pdf file and they are fine.
However the database seems like not indexing my blob column although the index can be created without error.
Please advise.
Really appreciate anyone who can help.
Thank you.

The situation is I have already loaded a few pdf document into the table's blob column.
After I create the Oracle text index on this blob column, I find the system generated index tables listed in my earlier posting are empty, except for the 4th table.
Normally we'll see words inside the table where those are the words indexed by oracle text on my document.
As a result, no matter how i search for the index using select statement with contains operator, it will not give me any result.
I feel weird why the blob is not indexed. The content of the blob are actually valid because I tested this by export the content back to pdf and I can still view and search within the pdf.
Regards,
Jap.

Index document with Oracle Text from an ECM without saving the content

Hi,
I have documents in a ECM (Alfresco, UCM and more) and I would like Oracle Text to index the document without saving the content. I want to save space and not have redundant information. I would use Oracle Text to search for document's identification (ID) and fetch the document from the ECM using the ID.
Is it possible ?
Do I have to use Secure Enterprise Search ?
Thanks
Simon

I want to save space and not have redundant information.The database space or the disk space (in OS)?
If it the database space, it is not possible to index/serach without storing the file conetents.
using , FILE_DATASTORE you can save the file in the disk (OS) and index them.
When you remove the file, you need to re-index it.
I donot see any other ways.
Do I have to use Secure Enterprise Search ?SES also uses Oracle Text as its base. It also uses FILE_DATASTORE. But the re-indexing part is automated using crawlers.

Error When Creating Oracle Text index using Lexer Keyword

Hi All,
I am getting following error when i creating oracle text index using lexer & stoplist keyword.
Pls Help me if any body know.
Thanks in Advance.
Error starting at line 1 in command:
CREATE INDEX TXT_INX_TEXT_SEARCH ON TEXT_SEARCH (BFILE_DOC)
Post INDEXTYPE IS "CTXSYS"."CONTEXT" LOCAL (
PARTITION "BEFORE_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)') ,
PARTITION "Q1_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "THE_REST" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)')
Error at Command Line:1 Column:13
Error report:
SQL Error: ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000: Oracle Text error:
DRG-11000: invalid keyword LEXER
ORA-06512: at "CTXSYS.DRUE", line 160
ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 365
29855. 00000 - "error occurred in the execution of ODCIINDEXCREATE routine"
*Cause: Failed to successfully execute the ODCIIndexCreate routine.
*Action: Check to see if the routine has been coded correctly.
Regards,
Jack R.

Hi,
it works if you put an extra PARAMETERS clause at the end so the creation looks like:
CREATE INDEX TXT_INX_TEXT_SEARCH ON TEXT_SEARCH (BFILE_DOC)
INDEXTYPE IS "CTXSYS"."CONTEXT" LOCAL (
PARTITION "BEFORE_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)') ,
PARTITION "Q1_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "THE_REST" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)')
PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)') <== Added
Hope this helps
Herald ten Dam

Problem with oracle text indexes during import

We have a 9.2.0.6 database using oracle text features on a server with windows 2000 5.00.2195 SP4.
We need to export its data ( user ARIANE only ) and then import the result into another 9.2.0.6 database.
The import never comes to an end.
The only way to make it work is to use the "indexes=n" clause.
Then ( without the indexes ), we tried to create manually the oracle text indexes.
We get this error :
CREATE INDEX ARIANE.DOSTEXTE_DTTEXTE_CTXIDX ON ARIANE.DOSTEXTE (DTTEXTE)
INDEXTYPE IS CTXSYS.CONTEXT PARAMETERS('lexer ariane_lexer stoplist ctxsys.default_stoplist storage ariane_storage');
ORA-29855: erreur d'exécution de la routine ODCIINDEXCREATE
ORA-20000: Erreur Oracle Text :
DRG-10700: préférence inexistante : ariane_lexer
ORA-06512: à "CTXSYS.DRUE", ligne 157
ORA-06512: à "CTXSYS.TEXTINDEXMETHODS", ligne 219
We then tried to uninstall Oracle text and install it ( My Oracle Support [ID 275689.1] ). The index creation above still fails.
We also checked our Text installation and setup through My Oracle Support FAQ ( ID 153264.1 ) and everything seems ok.
Do we have to create some ARIANE* lexer preferences through specific pl/sql ( ctx_report* ? ) before importing anything from the ARIANE user ?
What do we need to do exactly when exporting data with oracle text features from one database to another given we used to restore the database through a copy of the entire windows files ?
Is there a specific order to follow to succeed an import ?
Thank you for your help.
Jean-michel, Nemours, FRANCE

Hi
index preferences are not exported, ie ariane_lexer + ariane_storage, only the Text index metada, thus the DRG-10700 from index DDL on target/import DB.
I recommend to use ctx_report.create_index_script on source/export DB, see Doc ID 189819.1 for details, export with indexes=N and then create text indexes manually after data import.
-Edwin

Strange: Conditional Error in Intermedia Text Index Creation

Hi All,
I am facing a strange error while creating the Intermedia Text index on 8.1.6 release for NT 4.0.
I have one database instance installed on the server. When i connect to Oracle through SQL plus as: connect username/password;
The index creation works fine!
But, if i connect using: username/password@servicename;
The index creation does not work!
The following error occurs.
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000:intermedia Text error:
ORA-50704:Net8 listener is not running or cannot start external procedures
ORA-28575:unable to open RPC connection to external procedure agent
ORA-12154:TNS:could not resolve service name
ORA-06512:at "CTXSYS.DRUE", line 126
ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 54
ORA-06512:at line 1
Could anyone please help as to why it works without when connected without the service name but not otherwise?
Thanking you in advance!
Thanks and Regards
Anurag Agarwal

Hi All,
I am facing a strange error while creating the Intermedia Text index on 8.1.6 release for NT 4.0.
I have one database instance installed on the server. When i connect to Oracle through SQL plus as: connect username/password;
The index creation works fine!
But, if i connect using: username/password@servicename;
The index creation does not work!
The following error occurs.
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000:intermedia Text error:
ORA-50704:Net8 listener is not running or cannot start external procedures
ORA-28575:unable to open RPC connection to external procedure agent
ORA-12154:TNS:could not resolve service name
ORA-06512:at "CTXSYS.DRUE", line 126
ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 54
ORA-06512:at line 1
Could anyone please help as to why it works without when connected without the service name but not otherwise?
Thanking you in advance!
Thanks and Regards
Anurag Agarwal

Update Results not Displayed in Oracle Text search with Transactional Index

Hi,
I am working on a solution utilising Oracle Text to give me a probable list of matching records. The problem I have the table I am searching on is prepopulated with seed data and the application we are building is assigning a record and updating the details(columns) against it. This detail is what we are searching on using an Multi Column Datastore index which is refreshed every hr and also has the transactional parameter specified. Unfortunately the Transactional Index does not pick up the updated details, it only seems to work if I insert a new record (which will never happen). This to me sounds like a bug. Any assistance would be greatly appreciated.

Barbara,
I think you may have eluded to my problem. I haven't updated the "dummy" column
The table structure is as follows:
CREATE TABLE WAGN (
     WAGN               VARCHAR2(8) NOT NULL PRIMARY KEY,
     last_name          VARCHAR2(240),
     first_name          VARCHAR2(240),
     middle_name          VARCHAR2(240),
     date_of_birth     DATE,
     gender               VARCHAR2(1),
     status               VARCHAR2(1) NOT NULL,
     signature          RAW(64));
The preference creation is:
BEGIN
     ctx_ddl.create_preference('WAGN_NAME_SRCH', 'MULTI_COLUMN_DATASTORE');
     ctx_ddl.set_attribute('WAGN_NAME_SRCH', 'columns', 'last_name, first_name, middle_name, date_of_birth, gender');
END;
The Index Creation statement is:
CREATE INDEX wagn_srch_idx1 ON WAGN(signature) --Dummy Column
INDEXTYPE IS ctxsys.CONTEXT
PARAMETERS ('DATASTORE WAGN_NAME_SRCH SYNC(EVERY "SYSDATE+60/24/60" PARALLEL 10) TRANSACTIONAL');
And a typical update statement is (contained with PL/SQL):
     UPDATE WAGN
          SET status = x_wagn_assigned_status,
               last_name = p_employee_details.last_name,
               first_name = p_employee_details.first_name,
               middle_name = p_employee_details.middle_name,
               date_of_birth = p_employee_details.date_of_birth,
               gender = p_employee_details.gender
          WHERE WAGN = l_wagn;
So my guess is that because the dummy column (signature) is not updated it is not being reflected in the transactional memory area.

Create Oracle Text Index from Java via JDBC

Hi, I have a question relevant to creation of the Text Index in Oracle Text via Java JDBC.
In Java I create client application and I connect to remote Oracle server. (there is Oracle
10g Enterprise Edition 10.2.0.3.0 on Debian Linux OS). In Java I create query:
CREATE INDEX index_name ON source_tab(text) indextype is ctxsys.context
parameters('datastore ctxsys.file_datastore filter ctxsys.null_filter');
and I start it by:
PreparedStatement stmt = db.prepareStatement(query);
ResultSet rs = stmt.executeQuery();
There is problem that Text Index is created empty and when I write out error log then:
DRG-11513: unable to open or write to file "path"
I have on my computer Oracle XE 10g (Windows XP OS) here this application in Java functions
without problems. Without problems it´s with that same query in SQLPlus when I connect at
this remote Oracle server via PuTTY.
Do you know anybody where could be error? Or at worst case where find I more detailed
manual about JDBC Oracle mainly possible errors and problems with compatibility?

Does it work if you don't use "file_datastore"?
Eg. try the following in sqlplus:
create table t (text varchar2(2000));
insert into t values ('hello world');
create index ti on t(text) indextype is ctxsys.context;
select * from t where contains (text, 'hello') > 0;
Then drop the index ti and try the same "create index" through JDBC. Does that work?
If so, then we have some kind of issue with the file_datastore. I assume you are aware that the file will be read by the SERVER, not from the client?
So if your database resides on a Linux box, but you are calling from a Java program on a windows PC, the file names in source_tab(text) must be in Linux format, and must refer to files which are available on the Linux server (and readable by the owner of the Oracle database process).
So /home/me/file.txt might work, but C:\Documents\file.txt will not.

Oracle 9i(9.2.0.5.0) - Oracle Text - Indexing files on FTP Server

I am using Oracle 9i(9.2.0.5.0) and I am unable to upgrade to a newer version of Oracle DB.
I am new to this technology and I have not tried it yet myself.
I was reading some articles, documents or references about Oracle text technology and I have find out that Oracle text should be able to create a context index over the files which resides on the FTP server.
I have also found out, that for this purpose an "URL_DATASTORE" should be used.
I would be pleased if someone can answer my question before I decide to start using this technology:
- Is there any limitation which I should be aware of when creating context index over files which resides on FTP server? (file size limit, supported file types limitation)
- During index creation process are the indexed files downloaded and copied to the Oracle database permanently or only temporary until index is created?
- Is any incremental indexing possible(when I add new files to the datastore I do not have to rebuild the whole index)?
- Is there any formula between context index disk size and indexed files disk size?
Regards,
Michal

- Is there any limitation which I should be aware of when creating context index over files which resides on FTP server? (file size limit, supported file types limitation)
Max file size is configurable up to 2GB. No limitation on the file type from the datastore itself, but if you want to process binary files the normal list of suported filter file formats will apply (see the appendix in the admin guide)
- During index creation process are the indexed files downloaded and copied to the Oracle database permanently or only temporary until index is created?
Only temporarily
- Is any incremental indexing possible(when I add new files to the datastore I do not have to rebuild the whole index)?
From the question, I suspect you're seeing this as a crawler - you expect to provide the address of an FTP site and have it fetch all the documents. That's not how it works. Rather, you must put all the URLs into a table, and Text will index those URLs (and only those URLs)
If new files are added, you must arrange somehow to have the new rows added to your table. Then Text will do an incremental update, it won't have to rebuild the whole index.
- Is there any formula between context index disk size and indexed files disk size?
It varies quite a lot depending on types of data and indexing options chosen, but a typical result is that the index will be 40% of the total file size. However, if the documents are formatted (eg Word, PDF) the percentage will be much smaller.

Oracle Text Indexing performance in Unicode database

Forum folks,
I'm looking for overall performance thoughts in Text Indexing within a Unicode database. Part of our internal testing suites includes searching on values using contains filters over indexed binary and text documents. We've architected these tests such that they could be run in a suite or on their own, thus, the data is loaded at the beginning of each test and then the text indexes are created and populated prior to running any of the actual testing.
We have the same tests running on non-unicode instances of Oracle 11gR2 just fine, but when we run them against a Unicode instance, we are almost always seeing timing issues where the indexes haven't finished populating, thus our tests are saying we've only found n number of hits when we are expecting n+ 50 or in some cases n + 150 records to be returned.
We are just looking for some general information in regards to text indexing performance in a unicode database. Will we need to add sleep time to the testing to allow for the indexes to populate? How much time? We would rather not get into having to create different tests for unicode vs non-unicode, but perhaps that is necessary.
Any insight you could provide would be most appreciated.
Thanks in advance,
Dan

Roger,
Thanks much for your quick reply...
When you talk about Unicode, do you mean AL32UTF8?
--> Yes, this is the Unicode charset we are using.
Is the data the same in both cases, or are you indexing simple 7-bit ascii data in the one database, and foreign text (maybe Chinese?) in the UTF8 database?
With the same data, there should be virtually no difference in performance due to the AL32UTF8 database character set.
--> We have a data generation tool we utilize. For non-unicode data, we generate using all 256 characters in the ISO-8859-1 set. With our Unicode data for clobs, we generate using only the first 1,000 characters of UTF8 by setting up an array of code points...0 - 1000. For Blobs, we have sets of sample word documents and pdfs that are inserted, then indexed.
I'm not sure I understand your testing methodology. Do you run ( load-data, index-data, run-queries ) sequentially?
--> That is correct. We utilize the ctx_ddl package to populate the pending table and then to sync the index....The following is an example of the ddl we generate to create and populate the index:
create index "DBMEARSPARK_ORA80"."RESRESUMEDOC" on "DBMEARSPARK_ORA80"."RESUME" ("RESUMEDOC") indextype is CTXSYS.CONTEXT parameters(' nopopulate sync (every "SYSTIMESTAMP + INTERVAL ''30'' MINUTE" PARALLEL 2) filter ctxsys.auto_filter ') PARALLEL 2;
execute ctx_ddl.populate_pending('"DBMEARSPARK_ORA80"."RESRESUMEDOC"',null);
execute ctx_ddl.sync_index('"DBMEARSPARK_ORA80"."RESRESUMEDOC"',null,null,2);
If so, there should be no way that the indexes can be half-created. If not, don't you have some check to see if the index creation has finished before running the query test?
--> Excellent question....is there such a check? I have not found a way to do that yet...
Were you just lucky with the "non-unicode" tests that the indexing just happened to have always finished by the time you ran the queries?
--> This is quite possible as well. If there is a check to see if the index is ready, then we could add that into our infrastructure.
--> Thanks, again, for responding so quickly.
Edited by: djulson on Feb 12, 2013 7:13 AM

Error When Creating OR Rebuilding Oracle Text index using Lexer Keyword

Hi All,
I am getting following error when i creating oracle text index using lexer & stoplist keyword.
Pls Help me if any body know.
Thanks in Advance.
Error starting at line 1 in command:
CREATE INDEX TXT_INX_TEXT_SEARCH ON TEXT_SEARCH (BFILE_DOC)
Post INDEXTYPE IS "CTXSYS"."CONTEXT" LOCAL (
PARTITION "BEFORE_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)') ,
PARTITION "Q1_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "THE_REST" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)')
Error at Command Line:1 Column:13
Error report:
SQL Error: ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000: Oracle Text error:
DRG-11000: invalid keyword LEXER
ORA-06512: at "CTXSYS.DRUE", line 160
ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 365
29855. 00000 - "error occurred in the execution of ODCIINDEXCREATE routine"
*Cause: Failed to successfully execute the ODCIIndexCreate routine.
*Action: Check to see if the routine has been coded correctly.
Regards,
Jack R.

Hi,
it works if you put an extra PARAMETERS clause at the end so the creation looks like:
CREATE INDEX TXT_INX_TEXT_SEARCH ON TEXT_SEARCH (BFILE_DOC)
INDEXTYPE IS "CTXSYS"."CONTEXT" LOCAL (
PARTITION "BEFORE_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)') ,
PARTITION "Q1_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2007" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2008" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q1_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q2_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q3_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "Q4_2009" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)'),
PARTITION "THE_REST" PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)')
PARAMETERS ('LEXER dd_lexer STOPLIST dd_stoplist SYNC (ON COMMIT)') <== Added
Hope this helps
Herald ten Dam

Online help index error w.r.t oracle text

Hi,
Error creating online help index.
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine ORA-20000: Oracle Text error: DRG-10700: preference does not exist: CTXSYS.DEFAULT_LEXER
when i click on help on htmldb and then when i click on find. i get this following error.
how do i fix this error.
Can someone help me please.
THanks,
Philip.

Steps to resolve the domian index creation problem:
Step 1: Ensure the schema CTXSYS does not exist, if exists ignore the execution of “step 2”.
Step 2: Add the below mentioned statement above the existing statement(also mentioned below) in the catctx.sql file which is in the $ORACLE_HOME/ctx/admin" location.
Existing statement : Rem CTXDEF.sql - ctx default object creation
Statement to be added: grant execute on CTX_DDL to ctxsys ;
     In SYS user login, execute the script to create CTXSYS schema and its related objects after confirming below parameters to this script
     ctxsys           - password of CTXSYS schema
     SYSAUX     - default tablespace
     TEMP           - default temp tablespace (Replace if your database default temporary tablespace is different)
SQL>@$ORACLE_HOME/ctx/admin/catctx.sql ctxsys SYSAUX TEMP NOLOCK
Step 3: In SYS user login execute the below script to recompile user objects
     SQL>@$ORACLE_HOME/rdbms/admin/utlrp.sql
Step 4: In CTXSYS user login, execute the below script to create default preference for language ENGLISH
     SQL>@$ORACLE_HOME/ctx/admin/defaults/drdefus.sql
Step 5 : In SYS User login ,execute the below script to give grant permission
SQL>grant ctxapp to DBusername identified by Password;
SQL>grant select on ctxsys.dr$preference to DBusername;
SQL>grant execute on CTX_DDL to DBusername;
Step 5: Login as DB user and execute the below drop index script if any already exists in same name
     SQL>DROP INDEX LEI_CATALOG_SEARCH_SFIELD
Step 6: Login as DB user and create the below domain index
     SQL>CREATE INDEX LEI_CATALOG_SEARCH_SFIELD ON LET_CATALOG_SEARCH (SEARCH_FIELD) INDEXTYPE IS CTXSYS.CONTEXT;
Regards,
Kamal.C
Edited by: 871818 on Jul 12, 2011 2:17 AM

About index memory parameter for Oracle text indexes

Hi Experts,
I am on Oracle 11.2.0.3 on Linux and have implemented Oracle Text. I am not an expert in this subject and need help about one issue. I created Oracle Text indexes with default setting. However in an oracle white paper I read that the default setting may not be right. Here is the excerpt from the white paper by Roger Ford:
URL:http://www.oracle.com/technetwork/database/enterprise-edition/index-maintenance-089308.html
"(Part of this white paper below....)
Index Memory                                  As mentioned above, cached $I entries are flushed to disk each time the indexing memory is exhausted. The default index memory at installation is a mere 12MB, which is very low. Users can specify up to 50MB at index creation time, but this is still pretty low.
This would be done by a CREATE INDEX statement something like:
CREATE INDEX myindex ON mytable(mycol) INDEXTYPE IS ctxsys.context PARAMETERS ('index memory 50M');
Allow index memory settings above 50MB, the CTXSYS user must first increase the value of the MAX_INDEX_MEMORY parameter, like this:
begin ctx_adm.set_parameter('max_index_memory', '500M'); end;
The setting for index memory should never be so high as to cause paging, as this will have a serious effect on indexing speed. On smaller dedicated systems, it is sometimes advantageous to temporarily decrease the amount of memory consumed by the Oracle SGA (for example by decreasing DB_CACHE_SIZE and/or SHARED_POOL_SIZE) during the index creation process. Once the index has been created, the SGA size can be increased again to improve query performance."
(End here from the white paper excerpt)
My question is:
1) To apply this procedure (ctx_adm.set_parameter) required me to login as CTXSYS user. Is that right? or can it be avoided and be done from the application schema? This user CTXSYS is locked by default and I had to unlock it. Is that ok to do in production?
2) What is the value that I should use for the max_index_memory should it be 500 mb - my SGA is 2 GB in Dev/ QA and 3GB in production. Also in the index creation what is the value I should set for index memory parameter - I had left that at default but how should I change now? Should it be 50MB as shown in example above?
3) The white paper also refer to rebuilding an index at some interval like once in a month:   ALTER INDEX DR$index_name$X REBUILD ONLINE;
--Is this correct advice? i would like to ask the experts once before doing that. We are on Oracle 11g and the white paper was written in 2003.
Basically while I read the paper, I am still not very clear on several aspects and need help to understand this.
Thanks,
OrauserN

Perhaps it's time I updated that paper
1. To change max_index_memory you must be a DBA user OR ctxsys. As you say, the ctxsys account is locked by default. It's usually easiest to log in as a DBA and run something like
exec ctxsys.ctx_adm.set_parameter('MAX_INDEX_MEMORY', '10G')
2. Index memory is allocated from PGA memory, not SGA memory. So the size of SGA is not relevant. If you use too high a setting your index build may fail with an error saying you have exceeded PGA_AGGREGATE_LIMIT. Of course, you can increase that parameter if necessary. Also be aware that when indexing in parallel, each parallel process will allocated up to the index memory setting.
What should it be set to? It's really a "safety" setting to prevent users grabbing too much machine memory when creating indexes. If you don't have ad-hoc users, then just set it as high as you need. In 10.1 it was limited to just under 500M, in 10.2 you can set it to any value.
The actual amount of memory used is not governed by this parameter, but by the MEMORY setting in the parameters clause of the CREATE INDEX statement. eg:
create index fooindex on foo(bar) indextype is ctxsys.context parameters ('memory 1G')
What's a good number to use for memory? Somewhere in the region of 100M to 200M is usually good.
3. No - that's out of date. To optimize your index use CTX_DDL.OPTIMIZE_INDEX. You can do that in FULL mode daily or weekly, and REBUILD mode perhaps once a month.

Index creation in Oracle Text ???

Similar Messages

Maybe you are looking for