Oracle Full text Performance

Hi,
I created the indexes using the following commands. I have a few questions:
1. Is the way i have created the indexes is correct? Is it must that i need to use the data store?If yes how do I use that given that there are multiple tables involved
 CREATE INDEX TITLESEARCH ON SCS_PRODUCT(title_printed) INDEXTYPE IS ctxsys.context; 
CREATE INDEX SUBTITLESEARCH ON SCS_PRODUCT(subtitle_printed) INDEXTYPE IS ctxsys.context; 
CREATE INDEX FIRSTNAMESEARCH ON SCS_CONTRIBUTOR(first_name) INDEXTYPE IS ctxsys.context; 
CREATE INDEX LASTNAMESEARCH ON SCS_CONTRIBUTOR(LAST_name) INDEXTYPE IS ctxsys.context; 
CREATE INDEX PRINTEDNAMESEARCH ON SCS_CONTRIBUTOR(PRINTED_name) INDEXTYPE IS ctxsys.context; 
CREATE INDEX ESSNSEARCH ON SCS_JOURNAL(ESSN) INDEXTYPE IS ctxsys.context; 
CREATE INDEX ISSNSEARCH ON SCS_JOURNAL(ISSN) INDEXTYPE IS ctxsys.context; 
CREATE INDEX ISBN10SEARCH ON scs_sku(isbn_old) INDEXTYPE IS ctxsys.context; 
CREATE INDEX ISBN13SEARCH ON scs_sku(isbn) INDEXTYPE IS ctxsys.context; 
CREATE INDEX AVAILABLEUSSEARCH ON SCS_PRODUCT(AVAILABLE_SAGE) INDEXTYPE IS ctxsys.context; 
CREATE INDEX AVAILABLEUKSEARCH ON SCS_PRODUCT(AVAILABLE_SAGE_UK) INDEXTYPE IS ctxsys.context; 
CREATE INDEX KEYWORDSEARCH ON DCS_PRD_KEYWRDS(KEYWORD) INDEXTYPE IS ctxsys.context; 
2. The performace is very slow. 
3. I am getting the following error sometimes when i try to search though my web application. 
ORA-29861: domain index is marked LOADING/FAILED/UNUSABLE
Any help would be really appreciated!
Thanks
Gautam

1) It is correct - but may be not so performance (you using default options).
I would not use oracle text for columns isbn and isbn_old. Normal index it better.
2) Could you post sql, execution plan, more info about computer.
3) When your computer is too slow for Oracle it can occurs... (LOADING).
oerr ora 29861
29861, 00000, "domain index is marked LOADING/FAILED/UNUSABLE"
// *Cause: An attempt has been made to access a domain index that is
// being built or is marked failed by an unsuccessful DDL
// or is marked unusable by a DDL operation.
// *Action: Wait if the specified index is marked LOADING
// Drop the specified index if it is marked FAILED
// Drop or rebuild the specified index if it is marked UNUSABLE.

Similar Messages

Oracle full text, the syntax of CONTAINS()

Hi guys,
contains(columnname, 'ABC')=0this means search for the data which doesn't contain word 'ABC'
contains(columnname, 'ABC and XYZ')=0
contains(columnname, 'ABC or XYZ')=0what do these 2 sql mean? I tested them, there's no syntax error, but they didn't work as I expected, the 'and' seems like an 'or', and 'or' seems like an 'and', could anyone help to explain this? all doc found in google are for contains()>0, those're not what I need.
thanks in advance.

What proof do you have that the two are or should be the same?
Why do you think that Oracle should parse the content of the second parameter & behave differently depending upon content of single string?because, 'and' is a operator in oracle text CONTAINS query:
http://docs.oracle.com/cd/E11882_01/text.112/e24436/cqoper.htm#CHDEGDDF
contains(columnname, 'abc and xyz')>0 means exactly searching for data contains 'abc' and 'xyz'
you can also write it as contains(columnname, 'abc & xyz')>0 as you can see in the link above.

Full text search for web ? Yes or no ?

Hi,
I have a DB that has more then 1.8mil records in a single table .... and would like to implement full text search or some sort of caching for quicker Web search ....
Let me describe you what I have .... The table that holds 1.8mil records is made out of 30 clob columns ... each holding text .... actually these are alphabetic columns ... so words that start with char 'A' are in the first clob ... 'B' in second 'C' in third and so forth ....
Searching is always done first by customerID and CreateDate which are both indexed columns , and then clobs are searched using instr ...
Execution plan was good ... but searching times started to increase ....
So therefor I would like to improve the search ... by implementing some sort of caching mechanism ....
I read a lot about this and found an example where I would create a table containing unique words and table for occurrences of the words ... but this would then
make like 1.8mil articles containing approc 500 words , which would then repeat through articles ... so ok there would be less then 50.000 unique words (in our language ) , but the occurrences would dramatically increase cause every word inside article has to have a link in occurrences table ... so this would then be like 900mil records inside table ..
Is this at all possible to have so many records inside single table ? And still make it quick ?
Is the Oracle Full text search the only right way in this situation ?
Any suggestions ? Did anyone implement anything like this ?
Thanks,
Kris

Let's start with your Oracle version. Please specify which version you run because Text capabilities vary dramatically between releases.
>
I tried using Oracle Text as suggested ... now if I understand correctly ....
CTXCAT - would be great because when new records are added, index is updated automatically .... but doesn't support CLOBs ... so no go
>
CTXCAT is a concatenated transactional index that is supposed to optimize combined searches on text and other columns. No go for you as it indeed does not support CLOB columns.
>
CONTEXT - supports CLObs , but I need to explicitly synchronize index ....
There are like 4000 inserts per day ..... and they all need to be indexed in a real-time ...
>
Not true, at least since 10g: SYNC(ON COMMIT) parameter makes this index type transactional (it's synchronized automatically on commit with this parameter set.)
>
If CTX_DLL.SYNC_INDEX procedure synchronize the whole table which is now 1.8mil records, this can take a while ... so it can't be run after inserts ....
>
It does not, it only synchronizes changed data since last sync operation.
So CONTEXT is actually perfectly suited for your needs (just redesign those 30 columns into one document column and index it.) Note that you need to regularly maintain CONTEXT indexes by scheduling CTX_DDL.OPTIMIZE_INDEX to run at off-hours and purge stale/removed data and rebuild its own internal index bitmaps for better performance. Otherwise you will see performance degrade as changes to the indexed data accumulate. You might also want to tweak initial indexing parameters, especially MEMORY parameter, as it greatly affects resulting index fragmentation - the more memory you give for initial indexing or optimization, the less fragmented and the more performant the index will be all other things equal.

Clarifications needed for full text search

Hi,
I need some clarification regarding full text search.
1) Is japanese part of the standard Oracle full text search?
2) if it is not, how to install the japanese lexer?
3) how oracle is sorting international characters. If a column contains both english, japanese and french, how will be the output?
Thanks
Muneer

Follwoing is the sql statement and the result i got
select language, description,lengthb(description) bytes, length(description) length, vsize(description) vsize from t2;
LANGUAGE      DESCRIPTION      BYTES      LENGTH      VSIZE
English      abcdefghij      10      10      10
English      zyxwvutsrq      10      10      10
French      dÃ©signÃ©ess      16      12      16
French      rÃ©concilia      13      11      13
German      EinfuhrzÃ¶l      13      11      13
German      mÃ¼ÃtÃ¤mpfer      19      13      19
Greek      Î´Î·Î¼Î¿ÏÎ¹Î¿Î³ÏÎ±      40      20      40
Greek      Î±ÏÎ¿ÎºÎ»ÎµÎ¹ÏÏÎ¹      42      20      42
Russian      Ð¿ÑÐµÑÑÑÐ¿Ð½Ð¸Ðº      42      20      42
Russian      Ð¿ÑÐ¸Ð²ÐµÐ´ÐµÐ½Ð¸Ñ      41      20      41
Japanese      å¥éå¸ã®ä¼ç¤¾ã®éè¡å£      65      30      65
Japanese      ç¥æ¸å¸ä¸å¤®åºã®æ±éå      62      30      62
Korean      ì¶ë°ì ë¶í°ì¶ë°ì ë¶í°      64      30      64
Korean      ë³´ì¢ê´ìì¶ë°ì ë¶í°ê²½      64      30      64
Hindi      à¤à¤¤à¤à¤¨à¤¤à¤®à¤¨à¤à¤¤à¤¶à¥à¤°      73      36      73
Hindi      à¤¨à¥à¤à¥à¤¨à¥à¤à¥à¤¨à¥à¤à¥à¤¨à¥à¤à¥à¤¨à¥à¤à¥      130      60      130 I think it explains a lot. I am facing another problem in searching blob columns when it contains japanese or korean characters. I tried with multi lexer (adding japanese as sub lexer and making english as default lexer). But it is not searching the column. Do i have to set any other parameters (editing registry, changing enviornment setting etc). I used the following script to set the lexer.
begin
     ctx_ddl.create_preference('english_lexer','basic_lexer');
     ctx_ddl.set_attribute('english_lexer','index_themes','yes');
     ctx_ddl.set_attribute('english_lexer','theme_language','english');
     ctx_ddl.create_preference('german_lexer','basic_lexer');
     ctx_ddl.set_attribute('german_lexer','composite','german');
     ctx_ddl.set_attribute('german_lexer','mixed_case','yes');
     ctx_ddl.set_attribute('german_lexer','alternate_spelling','german');
     ctx_ddl.create_preference('japanese_lexer','japanese_vgram_lexer');
     ctx_ddl.create_preference('korean_lexer','KOREAN_MORPH_LEXER');
     ctx_ddl.set_attribute('korean_lexer','COMPOSITE','NGRAM');
     ctx_ddl.create_preference('global_lexer', 'multi_lexer');
     ctx_ddl.add_sub_lexer('global_lexer','default','english_lexer');
     ctx_ddl.add_sub_lexer('global_lexer','german','german_lexer','ger');
     ctx_ddl.add_sub_lexer('global_lexer','japanese','japanese_lexer','jpn');
     ctx_ddl.add_sub_lexer('global_lexer','korean','Korean_lexer');
end;Hope i presented enough details.

Oracle XE 10.2.0.1.0 – Performance with BIG full-text indexes

I would like to use Oracle XE 10.2.0.1.0 only for the full-text searching of the files residing outside the database on the FTP server.
Recently I have found out that size of the files to be indexed is 5GB.
As I have read somewhere on this forum before size of the index should be 30-40% of the indexed text files (so with formatted documents like PDF or DOC even less).
Lets say that the CONTEXT index size over these files will be 1.5-2GB.
Number of the concurrent user will be max. 5.
Does anybody have any experience with Oracle XE performance with the CONTEXT index this BIG?
(Oracle XE license limitations: 1 GB RAM and 1 CPU)
Regards.
Edited by: user10543032 on May 18, 2009 11:36 AM
Edited by: user10543032 on May 18, 2009 12:10 PM

I have used the 100% same configuration as above, but now for the Oracle Database 11g R1 11.1.0.7.0 – Production instead of Oracle 10g XE.
The result is that AUTO_FILTER for Oracle 11g is able to parse Czech language characters from the sample PDF file without any problems.
The problem with Oracle Text 10g R2 may be I guess:
1. In embedded fonts as mentioned in the Link: [documentation | http://download-west.oracle.com/docs/cd/B12037_01/text.101/b10730/afilsupt.htm] (I tried to embbed all fonts and the whole character set, but it did not helped)
2. in the character encoding of the text within the PDF documents.
I would like to add that also other third party PDF2Text converters have similar issues with the Czech characters in the PDF documents – after text extraction Czech national characters were displayed incorrectly.
If you have any other remarks, ideas or conclusions please reply :-)

"Oracle text" performance Problem

Architecture for Performance on a web site Search!
I wanna use text service of ORACLE.But I am worried about the performance ....
How should I design the system if I want the best perfomance and scalability ?
1.Should I build a seperate coloumn in my every table and merge all the information into one coloumn and full text index that column.
2.Put a full text index in all column in the table and use OR clause and reverse rank it for AND clause,using CONTAINSTABLE function.
3.Make a different table and put ID,TYPE and _VALUE fields and search in that table with less coloumns.
4.Seperate the full text database and search in a seperate db so that I can scale better?
did anybody have a similiar problem ? Any books on full text search ?

The number of indexes is irrelevant as such. If you really need 100 tables and you really need full text search on all of them, you need 100 indexes. When you are inserting data in any given table, the fact that there are 99 other tables with 99 other Text indexes is irrelevant.
That being said, I would seriously question whether a data model that involves doing full-text searches on 100 separate tables was actually a proper data model. That strikes me as highly unlikely.
Justin

Unable to full text index the contents in Oracle 11g UCM

Hi,
I am new to the Oracle UCM 11g.
i am unable to full text index the content files that are check-in into the Oracle UCM.
I have added the below entries in config.cfg file:
SearchIndexerEngineName=OracleTextSearch
IndexerDatabaseProviderName= SystemDatabase
AdditionalEscapeChars=-;#
While performing the indexing operation using Repository Manager only, metadata of the content files are indexed, but full text is not getting indexed.
What is missing here in Oracle UCM for not fulltext indexing the contents? What configurations do i need to do for this so that i can search perform the full text search on the Contents in Oracle UCM?
Thanks in Advance
Dipesh

Hi Srinath,
Collection rebuild cycle runs perfectly fine. After enabling tracing for Indexer and systemdatabse, i got the below info in the log:
"Finished rebuilding the search index with a total of 123 files successfully indexed. A total of 0 files had a full text index."
The below is the details of the activeindex.hda:
<?hda version="11gR1-11.1.1.3.0-idcprod1-100505T121221" jcharset=UTF8 encoding=utf-8?>
@Properties LocalData
UseImplicitZonedSecurityField=true
blFieldTypes=
ActiveIndex=index1
blDateFormat=M/d{yy}{ h:mm[:ss]{ a}}!mAM,PM!tGMT+05:30
@end
@ResultSet SearchCollections
7
sCollectionID
sDescription
sVerityLocale
sProfile
sLocation
sFlag
sUrlScript
TestHost
!csSearchDefaultSearchCollection
English-US
local
index1
enabled
<$URL$>
@end
Is it possible that OracleTestSearch Component is missing in Oracle UCM?
Thanks
Dipesh

How to Enable Full text search in UCM with Oracle 10g

Hi,
Can you please guide me on how to enable full text search in UCM after I have installed it. My database is Oracle 10g. It would be great if you could guide me through the steps since I am very new to this product.
Regards
Ashish

Hi
Are you sure that your indexer is finishing without any errors? Check activeindex.hda under \search directory to see what is the variable set for activeindex variable? If Oracle Text is the indexer search enabled then the variable activeindex should either OTS1 or OTS2. If anything else then you have indexer problems that is not indexing documents.
Check the CS log file to see if indexer errors are shown there. If yes, then put it over here so that we can analyse it.
Hope it helps
Thanks
Srinath

Full text search with XML input in Oracle

Hi,
I need to develope a web application which can do full text search for a library. We will use XML as the input data format and stored XML file in Oracle. How can I do full text search to the XML according todifferent tags or elements. e.g search from title or from article.
Do not only convert XML to Ascii and do the full text search. It should utilize the XML tag and structure.
InterMedia text in 8i or Oracle text in 9i? Or any other toolkit?
Thanks a lot
Jack

Please refer to Oracle Text documentation

Problems using and configuring Oracle 10gR2 database full-text search

I am having problems trying to set up full-text indexing and search with Universal Content Management (UCM). I followed the Oracle Content Server Installation Guide for windows at [http://download-west.oracle.com/docs/cd/E10316_01/cs/cs_doc_10/documentation/integrator/install_cserver_win_10en.pdf].
What I did was:
1. Modify E:\oracle\ucm\server\config\config.cfg by adding SearchIndexerEngineName=DATABASE.FULLTEXT to the end of the file.
2. Restart the content server.
3. Rebuild the search indexing using Repository Manager.
However, I keep seeing the following error when I query by entering words in the "Full-Text Search" box.
Unable to retrieve search results. Unable to retrieve search results. Unable to create result set for query 'SELECT IdcColl1.dID, dDocName, dDocTitle, dDocType, dRevisionID, dSecurityGroup, dDocAuthor, dDocAccount, dRevLabel, dFormat, dOriginalName, dExtension, dWebExtension, dInDate, dOutDate, dCreateDate, dPublishType, dRendition1, dRendition2, VaultFileSize, WebFileSize, URL, dFullTextFormat, dFullTextCharset, DocMeta.*
FROM IdcColl1, DocMeta
WHERE IdcColl1.dID=DocMeta.dID AND (((CONTAINS(dDocFullText,'test') > 0 ))) ORDER BY dInDate Desc'. ORA-20000: Oracle Text error:
DRG-10599: column is not indexed
Some web searches suggested the following (all of which I have tried but not resolved this problem).
1. Publish the schema using Configuration Manager (applet) and then rebuild index
2. Set the dDocFullText as a "zone field". This is not possible, because dDocFullText does not show up under the list of fields under "Database" or "DatabaseFullText" for the Search Engine drop down (when using Zone Fields Configuration).
3. Reboot the server (did not work either).
I logged onto the Oracle database and checked the IdcColl1 table. There is indeed, no index for the field, dDocFullText. There is only 1 index for the field, did. The field, dDocFullText, is a BLOB. The question is, if I am supposed to create an index manually for this field, how would I do it? A web search has not been fruitful in answering this question.
Here are my server settings.
For UCM:
Operating System: Windows 2003 Enterprise
UCM : 10gR3
Memory: 1 GB
Web Server: Apache 2.2.11
For Oracle:
Operating System: Windows 2003 Enterprise
Oracle: 10gR2
Memory: 1 GB
Thanks.

I found out what the problem was. The problem was that I had to create the role, stellent_role, as described in the installation manual. After I created this role and assigned the database user to this role, a restart of the Content Server services and collection rebuild of the index fixed the problem.
However, I did notice one thing. I checked in 3 PDF files, and when I used Repository Manager to do a collection rebuild, I noticed that for Indexer Counters, the count for Full Text was 0 and the count for Meta Only was 3.
Anyone have any ideas? Is there something else that I missed? From reading the installation manual, it was not clear how database full-text indexing/searching would handle PDF files.

When can oracle support full text search for Simplified Chinese?

When I create index using the create index clause,the following errors appears
"ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000: interMedia Text error:
DRG-11440: operation not supported for the SIMPLIFIED CHINESE language"
Maybe I have to use like to query words...
Did somebody encounter the same problem and have a good solutions?I am expecting your help!
null

Hi,
Full-text search capabilities is not currently not supported and Unfortunately, we do not have a timeline as to when it will be available.
You can post a feedback on the below link.
http://feedback.azure.com/forums/217321-sql-database
Regards,
Mekh.

Large text with national caracters and full text index

Hello,
I have some large portion of string to index. So of these strings contains national characters so I have stored it into a NCLOB column. I would like to put a full text index of this column but I see that it is not possible for NCLOB column.
Does anybody have the problem as me? If yes, how did you proceed? Could the XMLType be a solution ?
Thanks for your help.
Kind regards,
Fred

Hi Fred,
"I do not think that I can manage all these characters with only one character set."
Sure you can. The main reason for using the NLS characterset has nothing to do with whether you can/cannot support multiple languages with your database charset. It is there primarily in case your db characterset is not some unicode/multibyte charset and you need to add support for it while maintaining your original db charset. A secondary reason is for certain performance considerations, but that is for specific application architectures and doesn't seem to be the case here. One of my apps supports virtually every language on Earth - not using the 'N''s'. I'm using AL32UTF8 - it is up to Unicode 4.0 in version 10gR2 of the db.
Choose a unicode character set for your db and you're fine unless you have some overwhelming reason why you are unable to do so. Any Text index will work for you and your problem is solved (and ctxxpath and context index will both work - just compare your requirements to the functionality provided by the index type).
Further reading:
Here's an OracleMag article from last Fall -
http://www.oracle.com/technology/oramag/oracle/05-sep/o55global.html
Here's a white paper -
http://www.oracle.com/technology/tech/globalization/pdf/TWP_AppDev_Unicode_10gR2.pdf
Thanks,
Ron

Cannot do a full text search on pdf or Microsoft office Documents.

I am using Oracle Content Server 10gR3
Apart from the standard components and other components, I have InBound refinery Support and Oracle text Search Components enabled.
I get the following error when I check in a pdf or a Microsoft office document.
Text Conversion of the file '<repo_home>/.pdf' failed.*
Content has been indexed with Info only. Resubmit should only be performed if the problem has been resolved.
I am able to perform a full text search on *.txt and *.htm files.

Here are some more errors from Console output log
requestaudit 10.28 12:20:00.011 Audit Request Monitor Request Audit Report over the last 120 Seconds****
requestaudit 10.28 12:20:00.011 Audit Request Monitor -Num Requests 349 Errors 345 Reqs/sec. 2.911 Avg. Latency (secs)0.309 Max Thread Count 2
requestaudit 10.28 12:20:00.011 Audit Request Monitor 1 Service DOC_INFO Total Elapsed Time (secs) 106.944 Num requests 345 Num errors 345 Avg. Latency (secs) 0.31
requestaudit 10.28 12:20:00.011 Audit Request Monitor 2 Service COLLECTION_GET_REFERENCE Total Elapsed Time (secs) 0.701 Num requests 2 Num errors 0 Avg. Latency (secs) 0.35
requestaudit 10.28 12:20:00.011 Audit Request Monitor 3 Service COLLECTION_DISPLAY Total Elapsed Time (secs) 0.211 Num requests 1 Num errors 0 Avg. Latency (secs) 0.211
requestaudit 10.28 12:20:00.011 Audit Request Monitor 4 Service COLLECTION_GET_INFO Total Elapsed Time (secs) 0.02 Num requests 1 Num errors 0 Avg. Latency (secs) 0.02
requestaudit 10.28 12:20:00.011 Audit Request Monitor ****End Audit Report*****

TREX Full-Text search on Solution Directory

Hi Experts,
I have a problem with Full-Text search on the Solution Directory. Basically it doesn't work at all.
In transaction SOLUTION_MANAGER I choose a solution and then ->Operation Setup -> Solution Settings. At that point from the top menu I choose Solution Directory -> Find Document...
When I trigger the search using normal attributes the process finds documents related to this Solution without any problem. However, when I search using the "Full-Text Search" feature, the process doesn't find any document at all even though they exist.
Why is this functionality not working? Is this problem related to TREX?
What should i do to fix it? How do I create an index for these documents?
I haven't been able to solve this issue as TREX seams to work fine... can anybody help me out?
Any idea?
Many thanks,
Aldo

David:
I want to know if Oracle Context Option to need run over Oracle Application Server for to work search over multiple tables???
Exist some method for this (Context) run without to use Oracle Application Server???
select * from table_name where
contains(column_name1, 'keywords')>0 AND
contains(column_name2, 'keywords')>0However, using the Concatenated Datastore to create one index that has all the columns to be searched as sections will get better performance, since your query then only needs to use one contains statement across one index.
select * from table_name where
contains(indexed_column_name,
'(keywords WITHIN section_name1) AND
(keywords WITHIN section_name2) AND
keyords')>0Note: The last keyword in this query is so you search the indexed column as well, which is not a section. You could also create a dummy column to create the index on, and make all columns to search sections of that.
Cheers
David<HR></BLOCKQUOTE>
null

Kazehakase with full-text search in history using Hyper Estraier

A guide for Kazehakase with full-text history search using Hyper Estraier
I adopted qdbm and submitted hyperestraier in AUR, so you can enable full-text search frature by installing Hyper Estraier from AUR and rebuilding Kazehakase using srcpac or yaourt.
1. Install QDBM and Hyper Estraier from AUR. The easiest way is using yaourt. (If you prefer not to use yaourt, download tarball and do makepkg && pacman -U manually.)
yaourt -S qdbm hyperestraier
2. Rebuild Kazehakase using srcpac.
srcpac -Sb kazehakase
Of cource you can rebuild Kazehakase using yaourt.
yaourt -Sb kazehakase
You don't have to modify configure option in PKGBUILD of Kazehakase, because "--enable-hyper-estraier" is implied by default. If Hyper Estraier is installed successfully, you'll get "Hyper Estraier: yes" in configure messages.
3. Configure Kazehakase. To enable full-text search in history, run Kazehakase and go Edit>Preference>General and change UI Level to "Expert" and apply settings. Next, go Edit>Preference>History and set Search engine name to "hyper-estraier" and restart Kazehakase. Then you'll see "History Search" box next to "Internet Search" box.
Sorry for my poor English.

"ctxsrv" is no longer supported at version 10.1.
Instead PARAMETERS clause has SYNC option, and you can specify ON COMMIT for this.
If you created the database with DBCA and chose Oracle Text option, then you have no need to perform any further operations to configure Oracle Text.

Oracle Full text Performance

Similar Messages

Maybe you are looking for