Significance of enabling stemming while creating lexer + oracle text

Hi all,
I am creating a basic lexer with stemming enabled.
begin
ctx_ddl.create_preference('mylexer', 'basic_lexer');
ctx_ddl.set_attribute('mylexer', 'index_stems', 'ENGLISH');
end;
when i create context index on clob column, default 4 tables get created, one among them, ie., "dr$<index_name>$i", will have a column token_type with 9 has a root words.
Where else this 'index_stems' is getting used?
are we using anywhere else ?
I just want to know the purpose of enabling stemming.
Please help me in understanding the purpose.
Thanks in Advance.

Purpose of stemming:
Stemming enables you to match words with the same linguistic root.
Suppose indexed column contains words like speak,spoke and spoken.
If stemming is enable for english then search on speak should give you the records containing all the three results.
We can enable stemming for a specific language using the word list preference.
We can find regarding the word list in Oracle Documentation (b14217 in case of 10g).
Hope this helps.

Similar Messages

Problem with blob column index created using Oracle Text.

Hi,
I'm running Oracle Database 10g 10.2.0.1.0 standard edition one, on windows server 2003 R2 x64.
I have a table with a blob column which contains pdf document.
Then, I create an index using the following script so that I can do fulltext search using Oracle Text.
CREATE INDEX DMCS.T_DMCS_FILE_DF_FILE_IDX ON DMCS.T_DMCS_FILE
(DF_FILE)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('DATASTORE CTXSYS.DEFAULT_DATASTORE');
However, the index is not searchable and I check the following tables created by database for my index and found them to be empty as well !!
DR$T_DMCS_FILE_DF_FILE_IDX$I
DR$T_DMCS_FILE_DF_FILE_IDX$K
DR$T_DMCS_FILE_DF_FILE_IDX$N
DR$T_DMCS_FILE_DF_FILE_IDX$R
I wonder what's wrong with it.
My user has been granted the ctx_app role and I have other tables that store plain text which I use Oracle Text are fine. I even output the blob column and save as pdf file and they are fine.
However the database seems like not indexing my blob column although the index can be created without error.
Please advise.
Really appreciate anyone who can help.
Thank you.

The situation is I have already loaded a few pdf document into the table's blob column.
After I create the Oracle text index on this blob column, I find the system generated index tables listed in my earlier posting are empty, except for the 4th table.
Normally we'll see words inside the table where those are the words indexed by oracle text on my document.
As a result, no matter how i search for the index using select statement with contains operator, it will not give me any result.
I feel weird why the blob is not indexed. The content of the blob are actually valid because I tested this by export the content back to pdf and I can still view and search within the pdf.
Regards,
Jap.

Enabling Failover While creating the vNIC Template

Hello all,
As per my understanding enabling fail-over will create the secondary virtual path, which gives me the failover of vNIC automatically with the secondary virtual path that got created & the traffic will be routed towards another FI.
While creating the vNIC templates & in most of the deployments i have seen.
No one is considering the enable failover checkbox & I would like to know the pro's & corn's.
In which situation do i need consider it ? What is the recommended practice?
Looking forward for the suggestions.
Regards,
Gopi G

Hi Gopi
This failover flag made a lot of sense in the past, e.g. when Window server only supported one vnic.
For VMware ESXi whether its vswitch, DVS, N1000v, it is recommended to not use this hardware failover flag, and let the OS switch handle it.
see eg
https://supportforums.cisco.com/discussion/12211096/vnic-enable-failover-or-use-vmware
https://supportforums.cisco.com/discussion/12257101/cisco-ucs-network-uplink-aggregation-layer
https://supportforums.cisco.com/document/72501/understanding-fabric-failure-and-failover-ucs
https://supportforums.cisco.com/discussion/12387816/lan-or-san-pin-group-ucsm
Walter.

Problems While Creating A Oracle Instance In Windows 7

Hi All,
I am trying to create a primary standby configuration in windows 7. When I am trying to create a oracle instance using oradim I am facing the below issue:-
I am using Oracle version 10.2.0.1.0.
C:\>ORADIM -new -sid STDBY -SRVC OracleServiceSTDBY -STARTMODE auto -SRVCSTART s
ystem -SPFILE;
DIM-00003: An argument is missing for the parameter.
I am not sure which parameter is missing. can you help me in this..
Regards,
Arijit

Hi Abdul,
I did the same thing mentioned by you but now I am facing some different error:-
C:\>ORADIM -edit -sid STDBY -STARTMODE manual -SRVCSTART system -pfile E:\Arijit
\oraclestby\dbs\initstdby.ora;
DIM-00077: Failed to change service configuration.
O/S-Error: (OS 5) Access is denied.
When I tried the first step I am getting another error:-
C:\>oradim -edit -sid STDBY -startmode auto -srvcstart system
DIM-00077: Failed to change service configuration.
O/S-Error: (OS 5) Access is denied.
Please suggest.
Regards,
Arijit

ORA-12560 while creating database Oracle 11.1.0.6

Hi,
I tried to create a Oracle Database using Oracle installer.
I got the error ORA-12560
This is part of the logfile:
INFO: Configuration assistant "Oracle Net Configuration Assistant" succeeded
INFO: Command = oracle.assistants.server.DbcaCfgPlugIn C:\Windows\system32\cmd /c call E:\app\Administrator\product\11.1.0\db_1/bin/dbca.bat -progress_only -createDatabase -templateName General_Purpose.dbc -sid labor -gdbName labor.world -emConfiguration LOCAL -datafileJarLocation E:\app\Administrator\product\11.1.0\db_1\assistants\dbca\templates -datafileDestination E:\app\Administrator\oradata -responseFile NO_VALUE -characterset WE8MSWIN1252 -obfuscatedPasswords false -sampleSchema true -recoveryAreaDestination NO_VALUE -totalMemory 818 -automaticMemoryManagement true -maskPasswords false -sysPassword ${S_SYSPASS} -systemPassword ${S_SYSTEMPASS} -dbsnmpPassword ${S_DBSNMPPASS} -sysmanPassword ${S_SYSMANPASS} ${S_SYSASMPASS} ${S_HOSTUSERPASS}
Command = oracle.assistants.server.DbcaCfgPlugIn has failed
INFO: Configuration assistant "Oracle Database Configuration Assistant" failed
*** Starting OUICA ***
Oracle Home set to E:\app\Administrator\product\11.1.0\db_1
Configuration directory is set to E:\app\Administrator\product\11.1.0\db_1\cfgtoollogs. All xml files under the directory will be processed
INFO: The "E:\app\Administrator\product\11.1.0\db_1\cfgtoollogs\configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
INFO:
The Runconfig command constructed is E:\app\Administrator\product\11.1.0\db_1\oui\bin\runConfig.bat ORACLE_HOME=E:\app\Administrator\product\11.1.0\db_1 MODE=perform ACTION=configure RERUN=false $*
INFO: Since there is an Internal Plugin Invocation or a Java Plugin Invocation tool in the Oracle Home E:\app\Administrator\product\11.1.0\db_1 we use the runConfig Command instead of the plugin's command
INFO: Created a new file E:\app\Administrator\product\11.1.0\db_1\cfgtoollogs\configToolFailedCommands
INFO: Since the option is to overwrite the existing E:\app\Administrator\product\11.1.0\db_1\cfgtoollogs\configToolFailedCommands file, backing it up
INFO: The backed up file name is E:\app\Administrator\product\11.1.0\db_1\cfgtoollogs\configToolFailedCommands.bak
SEVERE: OUI-25031:Some of the configuration assistants failed/cancelled. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any "Recommended" assistants means your system will not be correctly configured.
1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures.
2. Fix the errors causing these failures.
3. Select the failed assistants and click the 'Retry' button to retry them.
INFO: User Selected: Yes/OK
Can anybody help me
Operating system Windows Server 2008
Regards
siegwiin

Good morning Siegwin,
First, you should probably post to this forum: [Database - General|http://forums.oracle.com/forums/forum.jspa?forumID=61]
However, the error you are getting is a TNS error. You should check the following information in your TNSNAMES.ORA on your system and on the Database Server.
<li>Name of the database
<li>Port Number of the database as defined in the TNSNAMES - compare against the port number that the database is actually listening on
<li>The SID defined in the TNSNAMES file.
On the Database Server:
<li>Check that there is a listener running and that it is listening for your database.
Don.
REWARDS: Please remember to mark helpful or correct posts on the forum, not just for my answers but for everyone! :)

Problem full-text in blob column index created using Oracle Text

Hi,
I'm running Oracle Database 10g 10.2 on solaris
I configure Oracle text if i look for in a varchar2 column is ok, but with blob column doesn't works the search.
I have a table with a blob column which contains document. I load document with Oracle UCM (stellent)
My index scripts is:
CREATE INDEX ORAUCM.FT_IDCCOLL1 ON ORAUCM.IDCCOLL1
(DDOCFULLTEXT)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('DATASTORE CTXSYS.DEFAULT_DATASTORE FILTER CTXSYS.AUTO_FILTER FORMAT COLUMN DFULLTEXTFORMAT CHARSET
COLUMN DFULLTEXTCHARSET LEXER OCS_IDCCOLL1_LEXER SYNC (ON COMMIT)')
NOPARALLEL;
And my select retunm 0 rows although it will be many documents:
SELECT IdcColl2.dID, dDocName, dDocTitle, dDocType, dRevisionID, dSecurityGroup, dDocAuthor,
dDocAccount, dRevLabel, dFormat, dOriginalName, dExtension, dWebExtension, dInDate, dOutDate,
dPublishType, dRendition1, dRendition2, VaultFileSize, WebFileSize, URL, dFullTextFormat,
dFullTextCharset, DocMeta.*
FROM IdcColl1, DocMeta
WHERE IdcColl1.dID=DocMeta.dID AND (CONTAINS(dDocFullText,'SUBIR') > 0 )
ORDER BY dInDate Desc
Thanks in advance.

Thank you for your answer.
I response your question:
- yes DDOCFULLTEXT is a BLOB column.
- The document that word, excels, whatever. We load the document with UCM (universal Content Management)
because i need full-test search form UCM tool.
- Yes 'subir' containts in the word document.
- select * from CTX_USER_INDEX_ERRORS ;
No rows returned.
- SELECT TOKEN_TEXT FROM DR$FT_IDCCOLL1$I
No rows returned.
- I tried create symplifying index and doen't work.
I tried create table and index context on oracle 10.2.0.3 (test database)and works ok.
I compared both context (test database and ucm database) and i saw a difference:
In ucm database there are these preferences "analyze text"
BEGIN ctx_ddl.create_preference('ORAUCM.', 'WORLD_LEXER'); end;
BEGIN ctx_ddl.create_preference('ORAUCM.', 'DETAIL_DATASTORE'); end;
I don't know if is important diference or no.
Please if you need more information, tell me.
Thanks for your time.

Error while running the Oracle Text optimize index procedure (even as a dba user too)

Hi Experts,
I am on Oracle on 11.2.0.2 on Linux. I have implemented Oracle Text. My Oracle Text indexes are fragmented but I am getting an error while running the optimize_index error. Following is the error:
begin
ctx_ddl.optimize_index(idx_name=>'ACCESS_T1',optlevel=>'FULL');
end;
ERROR at line 1:
ORA-20000: Oracle Text error:
ORA-06512: at "CTXSYS.DRUE", line 160
ORA-06512: at "CTXSYS.CTX_DDL", line 941
ORA-06512: at line 1
Now I tried then to run this as DBA user too and it failed the same way!
begin
ctx_ddl.optimize_index(idx_name=>'BVSCH1.ACCESS_T1',optlevel=>'FULL');
end;
ERROR at line 1:
ORA-20000: Oracle Text error:
ORA-06512: at "CTXSYS.DRUE", line 160
ORA-06512: at "CTXSYS.CTX_DDL", line 941
ORA-06512: at line 1
Now CTXAPP role is granted to my schema and still I am getting this error. I will be thankful for the suggestions.
Also one other important observation: We have this issue ONLY in one database and in the other two databases, I don't see any problem at all.
I am unable to figure out what the issue is with this one database!
Thanks,
OrauserN

How about check the following?
Bug 10626728 - CTX_DDL.optimize_index "full" fails with an empty ORA-20000 since 11.2.0.2 upgrade (DOCID 10626728.8)

Error while creating a Oracle Database

Hi,
I was recently gifted with a Dell Vostro, Windows 7pro, Core i5 processor. I was trying to install Oracle in the machine and have been receiving the errors as:
The procedure entry point GetProcessImageFileNameW could not be located in the dynamic link library PSAPI.DLL
The value of the environment variable PERL5LIB is more than 1023 chars. THis value cannot be set.I ignored the above errors and continued the process of installation which got installed successfully. Then tried installing the patch which was also successful.
When I started creating the DB it throws me an error as: Cannot create directory "C:\oracle\product\10.2.0\db_1\cfgtoollogs\dbca\<DBNAME> and there's no Database created. I've admin rights on the machine and have checked the compatibility as well.
I've tried installing Oracle and creating DB in other Windows 7 pro machines with Core i2 duo it has been successful.
Kindly need help on urgent basis.

811638 wrote:
Hey thanks a lot for ur prompt reply.
But I tried installing the Oracle10g in another Windows7 pro machine(HP) and it worked successfully..
Urgent coz, if me not able to install Oracle10g here then I would exchange this DELL to Toshiba(Core2 Duo) tomo..
Edited by: 811638 on Nov 15, 2010 1:38 PMJust because it installs does not necessarily mean that it will work as advertised. As stated above, base 10gR2 is not certified on Win 7 - if you insist on using 10gR2 on Win 7, you are on your own.
This is a forum of volunteers who spend their personal time answering questions without any renumeration. For all "urgent" issues, please open an SR with Oracle Support - they are paid to respond to "urgent" issues ;-)
Srini

Oracle Text Index Error - DRG-10700: preference does not exist: CTXSYS.DEFA

Hi we are having a Oracle 11g db.And we have installed CTX SYS package and provided the grants to the required schema.
But while creating the oracle text index, following error is thrown:
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000: Oracle Text error:
DRG-10700: preference does not exist: CTXSYS.DEFAULT_LEXER
ORA-06512: at "CTXSYS.DRUE", line 160
ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 365
Following grnats are given :
GRANT EXECUTE ON CTXSYS.CTX_DDL TO <schema_name>;
GRANT APPOWN TO <schema_name>;
GRANT CTXAPP TO <schema_name>;
I have verifyed the ctxsys.CTX_PREFERENCES view which is having an entry as :
SELECT PRE_OWNER, PRE_NAME
FROM CTXSYS.CTX_PREFERENCES
WHERE PRE_NAME = 'DEFAULT_LEXER';
The Result is :
PRE_OWNER     PRE_NAME
SYS     DEFAULT_LEXER
Can you please assist if there is a way to update the permission on this DEFAULT_LEXER preference to set the PRE_OWNER as CTXSYS ?
Thanks

That's odd. If I run the same query I see:
SQL> SELECT PRE_OWNER, PRE_NAME
2 FROM CTXSYS.CTX_PREFERENCES
3 WHERE PRE_NAME = 'DEFAULT_LEXER';
PRE_OWNER                      PRE_NAME
CTXSYS                         DEFAULT_LEXERAre all your defaults like thsi? What do you see if you run
select pre_owner, pre_name from ctxsys.ctx_preferences where pre_name like 'DEFAULT%';
It would be easy enough to update the underlying table DR$PREFERENCES to fix the owner, but who knows what else might be wrong.
Has this database been updated from a 9i installation by any chance?

Help with creating oracle text index on 2 columns with partial html data

Hi,
I need to create an oracle text index on 2 columns.
TITLE - varchar(255) = contains plain text data
DESCRIPTION - CLOB = contains partial HTML data
This is what I created.
begin
ctx_ddl.create_preference ('Title_Description_Pref', 'MULTI_COLUMN_DATASTORE');
ctx_ddl.set_attribute('Title_Description_Pref', 'columns', 'TITLE, DESCRIPTION');
end;
begin
ctx_ddl.create_preference ('bid_lexer', 'BASIC_LEXER');
ctx_ddl.set_attribute('bid_lexer', 'index_stems', 'ENGLISH');
ctx_ddl.create_section_group('htmgroup', 'HTML_SECTION_GROUP');
end;
create index Bid_Title_Index on Bid(title) indextype is ctxsys.context parameters ('LEXER bid_lexer sync (every "sysdate+(1/24)")');
create index Bid_Title_Desc_Index on Bid(description) indextype is ctxsys.context parameters ('LEXER bid_lexer DATASTORE Title_Description_Pref sync (every "sysdate+(1/24)") filter ctxsys.null_filter section group htmgroup');
The problem is when I do a CONTAINS(description, '$(auction)')>0. I get results where the descriptions have the "auction" word (which is correct). But, the results also returned rows where the search word is inside an IMG tag. e.g. <img src="http://auction.de/120483" alt="Auction Logo"/>.
What I would like is to exclude rows where the search word is inside HTML tag attributes, results expected are rows having <a>Auction</a> or <p>For Auction</p> ... etc. Basically stripping the html tags and leave the text contents.
I'd appreciate some input.
Thanks,
Amiel

Hi,
I need to create an oracle text index on 2 columns.
TITLE - varchar(255) = contains plain text data
DESCRIPTION - CLOB = contains partial HTML data
This is what I created.
begin
ctx_ddl.create_preference ('Title_Description_Pref', 'MULTI_COLUMN_DATASTORE');
ctx_ddl.set_attribute('Title_Description_Pref', 'columns', 'TITLE, DESCRIPTION');
end;
begin
ctx_ddl.create_preference ('bid_lexer', 'BASIC_LEXER');
ctx_ddl.set_attribute('bid_lexer', 'index_stems', 'ENGLISH');
ctx_ddl.create_section_group('htmgroup', 'HTML_SECTION_GROUP');
end;
create index Bid_Title_Index on Bid(title) indextype is ctxsys.context parameters ('LEXER bid_lexer sync (every "sysdate+(1/24)")');
create index Bid_Title_Desc_Index on Bid(description) indextype is ctxsys.context parameters ('LEXER bid_lexer DATASTORE Title_Description_Pref sync (every "sysdate+(1/24)") filter ctxsys.null_filter section group htmgroup');
The problem is when I do a CONTAINS(description, '$(auction)')>0. I get results where the descriptions have the "auction" word (which is correct). But, the results also returned rows where the search word is inside an IMG tag. e.g. <img src="http://auction.de/120483" alt="Auction Logo"/>.
What I would like is to exclude rows where the search word is inside HTML tag attributes, results expected are rows having <a>Auction</a> or <p>For Auction</p> ... etc. Basically stripping the html tags and leave the text contents.
I'd appreciate some input.
Thanks,
Amiel

Error Creating Oracle Text Index giving DRG-50857: oracle error in drvxtab

Hello,
I'm trying to create an Oracle text indexd as follows:
Create index tt.idx_emp on tt.emp(emp_name)
Indextype is ctxsys.context
Parameters(‘
Datastore idx_emp_dst
Filter idx_emp_fil
Section group idx_emp_sgp
Lexer idx_emp_lex
Wordlist idx_emp_wdl
Stoplist idx_emp_spl
Storage idx_emp_sto
Sync (on commit)
But, I get the following error:
Create index tt.idx_emp
ERROR at line 1:
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-20000: Oracle Text error:
DRG-50857: oracle error in drvxtab.create_index_tables
ORA-00905: missing keywork
ORA-06512: at “CTXSYS.DRUE”, line 160
ORA-06512: at “CTXSYS.TEXTINDEXMETHODS”, line 364
Has anyone encountered this error before? What did you do to fix this?
Thanks.

In what version?
Have you checked metalink?

Does Oracle Text need to be "enabled"?

We want to start using Oracle Text. Does it need to be "enabled"? Any scripts that need to be run first?
Oracle version 10.1.0.4

Hi,
There is nothing to "enable" in order to use Oracle Text. But you need to create domain indexes(CONTEXT, CTXCAT, CTXRULE etc) depending on which features of Oracle Text you want to use. Also, in order to use some of the Oracle Text procedures and packages, you will need the CTXAPP role assigned to you.
Enjoy searching!
Regards,
VenkatR

Trouble creating Oracle Text index

Hey all,
I am having trouble creating an Oracle Text index. I am receiving the following error when I try to sync the index after inserting numerous rows (@170,000). I have also tried to insert the rows and then create the index afterword. See below.
ORACLE_TEXT>insert into varchar_test_ot select doc from varchar_test;
171327 rows created.
Elapsed: 00:06:40.08
ORACLE_TEXT>exec ctx_ddl.sync_index('VT_TEST_IDX_OT');
BEGIN ctx_ddl.sync_index('VT_TEST_IDX_OT'); END;
ERROR at line 1:
ORA-20000: Oracle Text error:
DRG-50857: oracle error in drekmap (lob write)
ORA-22990: LOB locators cannot span transactions
ORA-06512: at "CTXSYS.DRUE", line 157
ORA-06512: at "CTXSYS.CTX_DDL", line 1328
ORA-06512: at line 1
Elapsed: 00:02:09.07

CREATE TABLE varchar_test
(doc VARCHAR2(4000)
CREATE TABLE varchar_test_ot
(doc VARCHAR2(4000)
begin
ctx_ddl.create_preference('storage_pref','BASIC_STORAGE');
ctx_ddl.set_attribute('storage_pref', 'I_TABLE_CLAUSE',
'tablespace ot_test_idx_space storage (initial 1M next 1M)');
ctx_ddl.set_attribute('storage_pref', 'K_TABLE_CLAUSE',
'tablespace ot_test_idx_space storage (initial 1M next 1M)');
ctx_ddl.set_attribute('storage_pref', 'R_TABLE_CLAUSE',
'tablespace ot_test_idx_space storage (initial 1M next 1M)');
ctx_ddl.set_attribute('storage_pref', 'N_TABLE_CLAUSE',
'tablespace ot_test_idx_space storage (initial 1M next 1M)');
ctx_ddl.set_attribute('storage_pref', 'I_INDEX_CLAUSE',
'tablespace ot_test_idx_space storage (initial 1M next 1M)');
ctx_ddl.set_attribute('storage_pref', 'P_TABLE_CLAUSE',
'tablespace ot_test_idx_space storage (initial 1M next 1M)');
ctx_ddl.create_section_group('auto','AUTO_SECTION_GROUP');
ctx_ddl.create_preference('mywordlist', 'BASIC_WORDLIST');
ctx_ddl.set_attribute('mywordlist','SUBSTRING_INDEX', 'YES');
end;
CREATE INDEX vt_test_idx_ot
ON varchar_test_ot(doc)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS ('datastore ctxsys.default_datastore
filter ctxsys.null_filter
wordlist mywordlist
storage storage_pref
section group auto
memory 50M');

Oracle Text and MINUS character

Hi all,
I have following problem:
- I have created a Oracle Text Index on the VARCHAR2 column:
BEGIN
ctx_ddl.create_preference('SUBSTRING_PREF','BASIC_WORDLIST');
ctx_ddl.set_attribute('SUBSTRING_PREF','SUBSTRING_INDEX','TRUE');
END;
CREATE INDEX IDX_TEXT_1 ON MY_TABLE
(COLUMN1)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS('wordlist SUBSTRING_PREF memory 50m')
NOPARALLEL;- I execute the following SELECT:
SELECT mt.*
FROM MY_TABLE mt
WHERE contains(mt.COLUMN1, 'test%') > 0;It returns all records where the column1 contains the entry "test" + something else.
BUT the records where column1 contains the entries "my-test" + something else or "owr-test" + something else, a.s.o.
It should return only the records with "test" + something else entries.
How can I change the Index or query to achieve it?
Best regards

By default, the hyphen is treated as a break character and the words on either side are indexed as separate tokens. To change this behavior, you can create a lexer, set the printjoins attribute of the lexer to include the hyphen, then use that lexer in your index parameters. Then, strings of characters containing a hyphen will be indexed as one token, including the hyphen. Please see the example below.
SCOTT@orcl_11gR2> CREATE TABLE my_table (column1 VARCHAR2(60))
2 /
Table created.
SCOTT@orcl_11gR2> INSERT ALL
2 INTO my_table VALUES ('test')
3 INTO my_table VALUES ('testing')
4 INTO my_table VALUES ('my-test')
5 INTO my_table VALUES ('owr-test')
6 SELECT * FROM DUAL
7 /
4 rows created.
SCOTT@orcl_11gR2> BEGIN
2    ctx_ddl.create_preference('SUBSTRING_PREF','BASIC_WORDLIST');
3    ctx_ddl.set_attribute('SUBSTRING_PREF','SUBSTRING_INDEX','TRUE');
4    CTX_DDL.CREATE_PREFERENCE ('test_lex', 'BASIC_LEXER');
5    CTX_DDL.SET_ATTRIBUTE ('test_lex', 'PRINTJOINS', '-');
6 END;
7 /
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> CREATE INDEX IDX_TEXT_1 ON MY_TABLE (COLUMN1)
2 INDEXTYPE IS CTXSYS.CONTEXT
3 PARAMETERS
4    ('wordlist SUBSTRING_PREF
5       LEXER        test_lex
6       memory    50m')
7 NOPARALLEL
8 /
Index created.
SCOTT@orcl_11gR2> SELECT token_text FROM dr$idx_text_1$i
2 /
TOKEN_TEXT
MY-TEST
OWR-TEST
TEST
TESTING
4 rows selected.
SCOTT@orcl_11gR2> SELECT mt.*
2 FROM   MY_TABLE mt
3 WHERE contains (mt.COLUMN1, 'test%') > 0
4 /
COLUMN1
test
testing
2 rows selected.

Oracle text performance with context search indexes

Search performance using context index.
We are intending to move our search engine to a new one based on Oracle Text, but we are meeting some
bad performance issues when using search.
Our application allows the user to search stored documents by name, object identifier and annotations(formerly set on objects).
For example, suppose I want to find a document named ImportSax2.c: according to user set parameters, our search engine format the following
search queries :
1) If the user explicitely ask for a search by document name, the query is the following one =>
     select objid FROM ADSOBJ WHERE CONTAINS( OBJFIELDURL , 'ImportSax2.c WITHIN objname' , 1 ) > 0;
2) If the user don't specify any extra parameters, the query is the following one =>
     select objid FROM ADSOBJ WHERE CONTAINS( OBJFIELDURL , 'ImportSax2.c' , 1 ) > 0;
Oracle text only need around 7 seconds to answer the second query, whereas it need around 50 seconds to give an answer for the first query.
Here is a part of the sql script used for creating the Oracle Text index on the column OBJFIELDURL
(this column stores a path to an xml file containing properties that have to be indexed for each object) :
begin
Ctx_Ddl.Create_Preference('wildcard_pref', 'BASIC_WORDLIST');
ctx_ddl.set_attribute('wildcard_pref', 'wildcard_maxterms', 200) ;
ctx_ddl.set_attribute('wildcard_pref','prefix_min_length',3);
ctx_ddl.set_attribute('wildcard_pref','prefix_max_length',6);
ctx_ddl.set_attribute('wildcard_pref','STEMMER','AUTO');
ctx_ddl.set_attribute('wildcard_pref','fuzzy_match','AUTO');
ctx_ddl.set_attribute('wildcard_pref','prefix_index','TRUE');
ctx_ddl.set_attribute('wildcard_pref','substring_index','TRUE');
end;
begin
ctx_ddl.create_preference('doc_lexer_perigee', 'BASIC_LEXER');
ctx_ddl.set_attribute('doc_lexer_perigee', 'printjoins', '_-');
ctx_ddl.set_attribute('doc_lexer_perigee', 'BASE_LETTER', 'YES');
ctx_ddl.set_attribute('doc_lexer_perigee','index_themes','yes');
ctx_ddl.create_preference('english_lexer','basic_lexer');
ctx_ddl.set_attribute('english_lexer','index_themes','yes');
ctx_ddl.set_attribute('english_lexer','theme_language','english');
ctx_ddl.set_attribute('english_lexer', 'printjoins', '_-');
ctx_ddl.set_attribute('english_lexer', 'BASE_LETTER', 'YES');
ctx_ddl.create_preference('german_lexer','basic_lexer');
ctx_ddl.set_attribute('german_lexer','composite','german');
ctx_ddl.set_attribute('german_lexer','alternate_spelling','GERMAN');
ctx_ddl.set_attribute('german_lexer','printjoins', '_-');
ctx_ddl.set_attribute('german_lexer', 'BASE_LETTER', 'YES');
ctx_ddl.set_attribute('german_lexer','NEW_GERMAN_SPELLING','YES');
ctx_ddl.set_attribute('german_lexer','OVERRIDE_BASE_LETTER','TRUE');
ctx_ddl.create_preference('japanese_lexer','JAPANESE_LEXER');
ctx_ddl.create_preference('global_lexer', 'multi_lexer');
ctx_ddl.add_sub_lexer('global_lexer','default','doc_lexer_perigee');
ctx_ddl.add_sub_lexer('global_lexer','german','german_lexer','ger');
ctx_ddl.add_sub_lexer('global_lexer','japanese','japanese_lexer','jpn');
ctx_ddl.add_sub_lexer('global_lexer','english','english_lexer','en');
end;
begin
     ctx_ddl.create_section_group('axmlgroup', 'AUTO_SECTION_GROUP');
end;
drop index ADSOBJ_XOBJFIELDURL force;
create index ADSOBJ_XOBJFIELDURL on ADSOBJ(OBJFIELDURL) indextype is ctxsys.context
parameters
('datastore ctxsys.file_datastore
filter ctxsys.inso_filter
sync (on commit)
lexer global_lexer
language column OBJFIELDURLLANG
charset column OBJFIELDURLCHARSET
format column OBJFIELDURLFORMAT
section group axmlgroup
Wordlist wildcard_pref
Oracle created a table named DR$ADSOBJ_XOBJFIELDURL$I which now contains around 25 millions records.
ADSOBJ is the table contaings information for our documents,OBJFIELDURL is the field that contains the path to the xml file containing
data to index. That file looks like this :
<?xml version="1.0" encoding="UTF-8" ?>
<fields>
<OBJNAME><![CDATA[NomLnk_177527o.jpgp]]></OBJNAME>
<OBJREM><![CDATA[Z_CARACT_141]]></OBJREM>
<OBJID>295926o.jpgp</OBJID>
</fields>
Can someone tell me how I can make that kind of request
"select objid FROM ADSOBJ WHERE CONTAINS( OBJFIELDURL , 'ImportSax2.c WITHIN objname' , 1 ) > 0;"
run faster ?

Below are the execution plan for both the 2 requests :
select objid FROM ADSOBJ WHERE CONTAINS( OBJFIELDURL , 'ImportSax2.c WITHIN objname' , 1 ) > 0
PLAN_TABLE_OUTPUT
|     Id     | Operation                              |Name                         |Rows     |Bytes     |Cost (%CPU)|
|     0     | SELECT STATEMENT                    |                              |1272     |119K     |     4     (0)     |
|     1      | TABLE ACCESS BY INDEX ROWID     |ADSOBJ      |1272     |119K     |     4     (0)     |
|     2      |     DOMAIN INDEX                    |ADSOBJ_XOBJFIELDURL     |          |          |     4     (0)     |
Note
- 'PLAN_TABLE' is old version
Executed in 2 seconds
select objid FROM ADSOBJ WHERE CONTAINS( OBJFIELDURL , 'ImportSax2.c' , 1 ) > 0
PLAN_TABLE_OUTPUT
|     Id     |Operation                              |Name                         |Rows     |Bytes     |Cost (%CPU)|
|     0     | SELECT STATEMENT                    |                              |1272     |119K     |     4     (0)     |
|     1     | TABLE ACCESS BY INDEX ROWID     |ADSOBJ                         |1272     |119K     |     4     (0)     |
|     2     | DOMAIN INDEX                    |ADSOBJ_XOBJFIELDURL     |          |          |     4     (0)     |
Sorry for the result formatting, I can't get it "easily" readable :(

Significance of enabling stemming while creating lexer + oracle text

Similar Messages

Maybe you are looking for