Fuzzy Operator with Contains (Oracle Text)

Hi !
I want to know If Fuzzy Operator works with FRENCH language ? In the doc, Oracle said you can created the Context Index in French but for the Fuzzy Operator, the FRENCH language are not displayed in supported language (Chap. 3).
Thanks

It was a problem with sqlplus.exe (not happening to sqlplusw.exe) displaying wrong caracters,
because of the code page it was using.
I've solved executing one of these:
"set NLS_LANG=american_america.US8PC437" or "chcp 1252", before sqlplus.exe.
Joaquin Gonzalez

Similar Messages

Using fuzzy operator in contains

I am new to all of this and would like to get more comfortable that the approach I've taken is correct. Our problem is taking user input which may not be 100% correct or slighty off and attempting to search the database with that input. The idea is the user provided '201 Wilshire Blvd' when the databse contains '201 Wiltshire Ave'. Can we find this customer in the system even if they left the 't' out of Wiltshire and not searching for Ave, Blvd, St, etc? Oracle Text fuzzy operators seems the right solution. We are using Oracle 9i. The app is written in Java/JDBC (1.3 jdk).
I tried to keep it as simple as possible but here is what I did:
1- create a table specifically for searching containing an id, varchar2(360), varchar2(666)
2- create a CONTEXT index on the first varchar
3- create a CONTEXT index on the second varchar
4- the varchar2(666) contains the address information in the following format: 'zip state city addr1 addr2'
5- triggers are defined to keep the search table in sync with its source tables
6- the indexes are re-sync'ed nightly
I created a separate search table because I was concerned over performance if I were to create indexes on the source tables. The select statements I construct look like the following:
Where CONTAINS(address, 24032 & MD & Frostburg, 10) > 0 AND
CONTAINS(address, 1616, 20) > 0 AND
CONTAINS(address, fuzzy(Pullman, 60, 30, weight), 30) > 0 AND
CONTAINS(customer_name, fuzzy(Acme, 60, 30, weight), 40) > 0
So, zip, state and city must match and street address must match. Terms extracted from the address and name are searched for using the fuzzy operator.
My concerns are:
1- performance: My search table has over 2.1M records. What can I do to improve lookups.
2- ignorance: Like I said, I'm new to all of this; am I correct that using the fuzzy operator for numbers makes no sense? So if they type '30' but meant '300' too bad?
3- accuracy: How can I use the input parameters to improve my hit rate? Given that I am indexing varchars and not a document set, does it make sense to change the min score from the conatins clause?
Sorry this is so long but I appreciate any comments/suggestions. Hope I haven't left out anything important.
David

Hi,
Here's a start - taking your questions one at a time:
(1) "Can we find this customer in the system even if they left the 't' out of Wiltshire..."
create table z_test (col1 varchar2(100));
insert into z_test values ('wiltshire');
commit;
create index z_test_idx on z_test(col1)
indextype is ctxsys.context;
-- SQL> column col1 format a20
-- SQL> select score(1), col1
-- 2 from z_test
-- 3 where contains(col1, '!wilshire', 1) > 0;
-- SCORE(1) COL1
-- 3 wiltshire
(2) "...and not searching for Ave, Blvd, St, etc"
truncate table z_test;
insert into z_test values ('wiltshire blvd');
insert into z_test values ('wiltshire ave');
insert into z_test values ('wilshire ave');
commit;
exec ctx_ddl.sync_index('Z_TEST_IDX')
-- SQL> COLUMN COL1 FORMAT A20
-- SQL> select score(1), col1
-- 2 from z_test
-- 3 where contains(col1, '!wilshire, ave', 1) > 0
-- 4 order by 1 desc;
-- SCORE(1) COL1
-- 52 wiltshire ave
-- 52 wilshire ave
-- 2 wiltshire blvd
-- wiltshire blvd is returned even though the search was
-- for ave - see the comma separating tokens in this case.
-- Note also the difference in score as a result.
(3) "5- triggers are defined to keep the search table in sync with its source tables"
That has to be expensive. More on this to come.
(4) "I created a separate search table because I was concerned over
performance if I were to create indexes on the source tables. "
Please explain where you anticipate performance problems that prompted
the separate table. If search, do a trace on the search and see
where Oracle spends its time. You'd be better off creating a storage
preference, storing the DR$ tables in a separate tablespace.
(5) "CONTAINS(address, 24032 & MD & Frostburg, 10) > 0 AND
CONTAINS(address, 1616, 20) > 0 AND
CONTAINS(address, fuzzy(Pullman, 60, 30, weight), 30) > 0 AND
CONTAINS(customer_name, fuzzy(Acme, 60, 30, weight), 40) > 0 "
You can (and should) simplify this query. There are three
passes at the same column and it isn't necessary. Read up on
searching using contains some more.
-Ron

Search with % in oracle text

How can i retrieve the records which has "80%" "and" keyword using oracle text . Because oracle consider " % , and , or " as the stopwords ... But i want to do the search on "80%" not "80" how to do that ....
below query retrieving both "80" and "80%" records....but i want only "80%"
SELECT ID, AUTHOR, DOCUMENT, PATH, PATH1
FROM DATASTORES_TAB a
WHERE contains ( dummy_col,'{80%}' ) > 0;

Try this...
exec ctx_ddl.drop_preference('my_basic_lexer')
BEGIN
ctx_ddl.create_preference
preference_name => 'my_basic_lexer',
object_name => 'basic_lexer'
ctx_ddl.set_attribute
preference_name => 'my_basic_lexer',
attribute_name => 'printjoins',
attribute_value => '_%'
END;
drop table t1;
create table t1 (text varchar2(80));
insert into t1 values ('My example is 80% complete');
insert into t1 values ('This contains 80 without percent');
insert into t1 values ('This contains 801 without percent');
insert into t1 values ('Na%me1');
insert into t1 values ('Narme1');
CREATE INDEX t1_index ON t1 ( text )
indextype IS ctxsys.context
parameters ( 'lexer my_basic_lexer' );
select text from t1 where contains (text, '80%') > 0;
select text from t1 where contains (text, 'Na%me1') > 0;
select text from t1 where contains (text, '{80%}') > 0;
select text from t1 where contains (text, '{Na%me1}') > 0;
Output:
SQL> select text from t1 where contains (text, '80%') > 0;
TEXT
My example is 80% complete
This contains 80 without percent
This contains 801 without percent
Elapsed: 00:00:00.01
SQL> select text from t1 where contains (text, 'Na%me1') > 0;
TEXT
Na%me1
Narme1
Elapsed: 00:00:00.01
SQL> select text from t1 where contains (text, '{80%}') > 0;
TEXT
My example is 80% complete
Elapsed: 00:00:00.00
SQL> select text from t1 where contains (text, '{Na%me1}') > 0;
TEXT
Na%me1

Help with creating oracle text index on 2 columns with partial html data

Hi,
I need to create an oracle text index on 2 columns.
TITLE - varchar(255) = contains plain text data
DESCRIPTION - CLOB = contains partial HTML data
This is what I created.
begin
ctx_ddl.create_preference ('Title_Description_Pref', 'MULTI_COLUMN_DATASTORE');
ctx_ddl.set_attribute('Title_Description_Pref', 'columns', 'TITLE, DESCRIPTION');
end;
begin
ctx_ddl.create_preference ('bid_lexer', 'BASIC_LEXER');
ctx_ddl.set_attribute('bid_lexer', 'index_stems', 'ENGLISH');
ctx_ddl.create_section_group('htmgroup', 'HTML_SECTION_GROUP');
end;
create index Bid_Title_Index on Bid(title) indextype is ctxsys.context parameters ('LEXER bid_lexer sync (every "sysdate+(1/24)")');
create index Bid_Title_Desc_Index on Bid(description) indextype is ctxsys.context parameters ('LEXER bid_lexer DATASTORE Title_Description_Pref sync (every "sysdate+(1/24)") filter ctxsys.null_filter section group htmgroup');
The problem is when I do a CONTAINS(description, '$(auction)')>0. I get results where the descriptions have the "auction" word (which is correct). But, the results also returned rows where the search word is inside an IMG tag. e.g. <img src="http://auction.de/120483" alt="Auction Logo"/>.
What I would like is to exclude rows where the search word is inside HTML tag attributes, results expected are rows having <a>Auction</a> or <p>For Auction</p> ... etc. Basically stripping the html tags and leave the text contents.
I'd appreciate some input.
Thanks,
Amiel

Hi,
I need to create an oracle text index on 2 columns.
TITLE - varchar(255) = contains plain text data
DESCRIPTION - CLOB = contains partial HTML data
This is what I created.
begin
ctx_ddl.create_preference ('Title_Description_Pref', 'MULTI_COLUMN_DATASTORE');
ctx_ddl.set_attribute('Title_Description_Pref', 'columns', 'TITLE, DESCRIPTION');
end;
begin
ctx_ddl.create_preference ('bid_lexer', 'BASIC_LEXER');
ctx_ddl.set_attribute('bid_lexer', 'index_stems', 'ENGLISH');
ctx_ddl.create_section_group('htmgroup', 'HTML_SECTION_GROUP');
end;
create index Bid_Title_Index on Bid(title) indextype is ctxsys.context parameters ('LEXER bid_lexer sync (every "sysdate+(1/24)")');
create index Bid_Title_Desc_Index on Bid(description) indextype is ctxsys.context parameters ('LEXER bid_lexer DATASTORE Title_Description_Pref sync (every "sysdate+(1/24)") filter ctxsys.null_filter section group htmgroup');
The problem is when I do a CONTAINS(description, '$(auction)')>0. I get results where the descriptions have the "auction" word (which is correct). But, the results also returned rows where the search word is inside an IMG tag. e.g. <img src="http://auction.de/120483" alt="Auction Logo"/>.
What I would like is to exclude rows where the search word is inside HTML tag attributes, results expected are rows having <a>Auction</a> or <p>For Auction</p> ... etc. Basically stripping the html tags and leave the text contents.
I'd appreciate some input.
Thanks,
Amiel

Search with contains where text contains a query operator such as NT

I have a query that uses contains and the text I am searching for has the word 'NT' in it (ie CONTAINS(my_name, 'Windows NT') > 0). This and any other operator such as 'PT' causes the error
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: ConText error:
DRG-50901: text query parser syntax error on line 1, column 3
How do I make it search the string without parsing it? I don't know what the strings will be so the code needs to accept any string.
Thanks,
Evelyn Dobkin

There are some example query preprocessors on: http://technet.oracle.com/sample_code/products/intermedia/htdocs/query_syntax_translators/query_syntax_translators.html
null

NEAR operator alternative when not using. oracle Text ?

hi,
I'm working on a project where i would need a Oracle Text 'NEAR like' operator ...
here is my scenario ...
in db we have Customers ... and every customer has some criterias like different search words( names, towns,cars,etc...) so for every customer i can create an SQL query out of criterias . ....
now .... we can have a criteria like. ...... WHERE fulltext like 'john%'. or even distance search line NEAR inside CONTAINS. ... but then the Oracle text index is needed .....
the only tAble on which Text index is created is our storage table that holds more then 4mil records and growing...
my question is ... is there any way to have a query that would do the same thing as NEAR but without Text index ?
here is how I start ....
I get full newspaper article text from our OCR library ......
then i need to check customer's criterias against this text to see which article is for which customer and then bind the article to the customer
I could do it without Oracle using RegEx , but criterias can get really complicated ... like customer wants only specific MEDIA, or specific category , type , only articles that are from medias that are from specific country etc ... and many more different criterias ... and all this can be wrapped inside brackets with ANDs, ORs, NOT. ....
So the only way to do it is to put it in Oracle and execute the correct query and let Oracle decide if the result is true or false .... but due to NEAR operator I need Oracle text ...
So if I decide to first insert article into our storage table which has Oracle text index to be able to do the correct search .... how fast will this be ????
will the the search become slower when there are 6mil records ? I know I can use FILTER BY to help Text index to do a better and quicker seach ... and how to optimize index ....but still
I'm always asking my self..... why insert the article in a table where there are already 6mil articles and execute query when I only need to check data on one single article and. i already know this article ...
I see two solutions :
- if there is alternative for NEAR without using Oracle text index then i would insert data into temporary table and execute query on this table..... table would always contain only this one article. maybe one option would be to have one 'temp' table with Oracle text index in which i insert this one article and with help of Oracle text based on this one article do the search , and then maybe on a daily basis clear index ..... or when the article is removed from the table ... but this would mean having two Orcle text indexes, cause we already have Oracle text index on our storage table anyway....
- another is to use Oracle text index and insert it into our storage table and hope for the best quick results ....
Maybe I'm exaggerating and query like WHERE id=1234 and CONTAINS(...). will execute faster then I think
If anyone would have any other suggestion I will be happy to try it ..
thanks,
Kris

Hi,
this is to my knowledge not possible. It is hard for Oracle to do, think about a table with many rows, every row with that column must be checked. So I think only a single varchar2 is possible. Maybe for you will a function work. It is possible to give a function as second parameter.
function return_signup
return varchar2
is
l_signup_name signup.signup_name%type;
begin
select signup_name
into l_signup_name
from signup
where signup_id = 1
and rownum = 1
return l_signup_name;
exception
when no_data_found
then
l_signup_name := 'abracadabra'; -- hope does not exist
return l_signup_name;
end;Now you can use above function in the contains.
select * from user_history_view users --, signup new_user
--where new_user.signup_id = 1
where contains(users.user_name, return_signup)>0;I didn't test the code! Maybe you have to adjust the function for your needs. But it is a idea how this can be done.
Otherwise you must make the check by normaly check the columns by simple using a join:
select * from user_history_view users, signup new_user
where new_user.signup_id = 1
and users.user_name = new_user.signup_name;Herald ten Dam
htendam.wordpress.com

Performance issues and options to reduce load with Oracle text implementation

Hi Experts,
My database on Oracle 11.2.0.2 on Linux. We have Oracle Text implemented for fuzzy search. Our oracle text indexes are defined as sync on commit as we can not afford to have stale data. Now our application does literally thousands of inserts/updates/deletes to those columns where we have these Oracle text indexes defined. As a result, we are seeing a lot of performance impact due to the oracle text sync routines being called on each commit. We are doing the index optimization every night (full optimization every night at 3 am). The oracle text index related internal operations are showing up as top sql in our AWR report and there are concerns that it is causing lot of load on the DB. Since we do the full index optimization only once at night, I am thinking should I change that , and if I do so, will it help us?
For example here are some data from my one day's AWR report:
Elapsed Time (s)
Executions
Elapsed Time per Exec (s)
%Total
%CPU
%IO
SQL Id
SQL Module
SQL Text
27,386.25
305,441
0.09
16.50
15.82
9.98
ddr8uck5s5kp3
begin ctxsys.drvdml.com_sync_i...
14,618.81
213,980
0.07
8.81
8.39
27.79
02yb6k216ntqf
begin ctxsys.syncrn(:idxownid,...
Full Text of above top sql:
ddr8uck5s5kp3
begin ctxsys.drvdml.com_sync_index(:idxname, :idxmem, :partname);
end
02yb6k216ntqf
begin ctxsys.syncrn(:idxownid, :idxoname, :idxid, :ixpid, :rtabnm, :flg); end;
Now if I do the full index optimization more often and not just once at night 3 PM, will that mean, the load on DB due to sync on commit will decrease? If yes how often should I optimized and doesn't the optimization itself lead to some load? Can someone suggest?
Thanks,
OrauserN

You can query the ctx_parameters view to see what your default and maximum memory values are:
SCOTT@orcl12c> COLUMN bytes    FORMAT 9,999,999,999
SCOTT@orcl12c> COLUMN megabytes FORMAT 9,999,999,999
SCOTT@orcl12c> SELECT par_name AS parameter,
2          TO_NUMBER (par_value) AS bytes,
3          par_value / 1048576 AS megabytes
4 FROM   ctx_parameters
5 WHERE par_name IN ('DEFAULT_INDEX_MEMORY', 'MAX_INDEX_MEMORY')
6 ORDER BY par_name
7 /
PARAMETER                               BYTES      MEGABYTES
DEFAULT_INDEX_MEMORY               67,108,864             64
MAX_INDEX_MEMORY                1,073,741,824          1,024
2 rows selected.
You can set the memory value in your index parameters:
SCOTT@orcl12c> CREATE INDEX EMPLOYEE_IDX01
2 ON EMPLOYEES (EMP_NAME)
3 INDEXTYPE IS CTXSYS.CONTEXT
4 PARAMETERS ('SYNC (ON COMMIT) MEMORY 1024M')
5 /
Index created.
You can also modify the default and maximum values using CTX_ADM.SET_PARAMETER:
http://docs.oracle.com/cd/E11882_01/text.112/e24436/cadmpkg.htm#CCREF2096
The following contains general guidelines for what to set the max_index_memory parameter and others to:
http://docs.oracle.com/cd/E11882_01/text.112/e24435/aoptim.htm#CCAPP9274

Searching from beginning of a line/string with Oracle Text ...

Oracle Database 10.2.0.3, Solaris
Hi,
what sounds very easy with the LIKE operator seems to be impossible with the Oracle Text Operator Contains ;
Searching for 'Deutsche%' results with LIKE ->
'Deutsche Bank'
'Deutsche Post'
'Deutsche Oracle Community'
But with Contains-Operator which is token based the result is following ('$Deutsche%')
'Deutsche Bank'
'Armin Deutscher'
We want to get results starting with 'Deutscher...' But with Contains and some configuration i did not find a way. Combining LIKE with Contains did not help too because Contains expands the word in more instances then LIKE.
Indexed Columns are varchar2 typed
any idea?
kind regards
Karl
Message was edited by:
kreitsch

Have you evaluated the query rewrite template with CONTAINS?
http://download.oracle.com/docs/cd/B19306_01/text.102/b14218/csql.htm#sthref122

Oracle Text in LOV

I'm afraid we're still using 10.1.3.5...
Is it possible for an LOV to perform a fuzzy search against an Oracle Text index (i.e. use the contains operator)?
We have previously done something similar by overriding the advancedSearch method to insert new clauses into the VO's where clause. Would we be able to do something similar in an LOV?
thanks,
M

Yes, that should be possible but it will come down to custom coding, check out this blog post:
http://adfplus.blogspot.nl/2012/05/google-like-search-and-lovs-using.html#mor
Steven Davelaar,
Jheadstart Team.

Using Oracle Text to Data Mine

Can someone provide me with an idea of how to Data Mine with just using Oracle Text and not the data mining option. I need to search a column of customer complaints and then put it in a category based on that. It would be best if the categories were auto generated. It has to be done in PL/SQL.
Thanks,

You cannot have the categories created automatically without data mining. However, if you are willing to create the categories and queries that determine them, then you can do it with just Oracle Text. I posted an example on the 2nd page of the following thread:
Re: New to Oracle Text search

Oracle Text

Hi Expert,
how can i use the Oracle text to do searching with the XML documents?
as i shredded the xml documents into the SQL view already,
i cannot create index onto those views.
what can i do with the Oracle text search?
THX a lot~!
Edith

Hi Edith:
Oracle Text can index document in UTF8 or any other encoding supported by Oracle.
Assuming that your table is LRPAPER_XMLTYPE_TBL you can create a Text index with:
create index LRPAPER_XMLTYPE_TBL_idx on LRPAPER_XMLTYPE_TBL p (value(p)) indextype is ctxsys.context;If you have documents with different languages stored in the same table you has to find some scalar column to be used as discriminator value. In my example "lang" is an scalar column of type varchar2. See the annotated schema at:
http://www.dbprism.com.ar/xsd/document-v20-ann.xsd
This annotated schema creates these Oracle types:
SQL> desc "document"
Name                                      Null?    Type
SYS_XDBPD$                                         XDB.XDB$RAW_LIST_T
id                                                 VARCHAR2(4000 CHAR)
lang                                               VARCHAR2(4000 CHAR)
header                                             headerType
body                                               CLOB
footer                                             footerType
SQL> desc "headerType"
"headerType" is NOT FINAL
Name                                      Null?    Type
id                                                 VARCHAR2(4000 CHAR)
lang                                               VARCHAR2(4000 CHAR)
title                                              VARCHAR2(4000 CHAR)
subtitle                                           VARCHAR2(4000 CHAR)
version                                            versionType
type                                               VARCHAR2(4000 CHAR)
authors                                            authorsType
notice                                             VARCHAR2(4000 CHAR)
abstract                                           VARCHAR2(4000 CHAR)
meta                                               metaListSo the syntax "XMLDATA"."lang" is referencing to the attribute (column) "lang" of the type document.
Best regards, Marcelo.

Oracle Text contain query limit with 9i

The Oracle Text contain query is defined as this:
CONTAINS(
[schema.]column,
text_query VARCHAR2
[,label NUMBER])
RETURN NUMBER;
Is the size limit of text_query 4000 bytes with 9i? Is it increased with 10g or is it accepting CLOB in 10g?
I always got the error: "ORA-01460: unimplemented or unreasonable conversion requested" when I do the following where p_var was dynamically passed with size of more than 4000 bytes:
p_statement varchar2(20000);
p_var varchar2(8000);
p_statement := select count(*) from dewey_table where contains(concat, :x)>0;
open m_cursor for p_statement using p_var;
Thanks very much,
Kevin

The limit is definitely 4000 characters in 9i and the 10g documentation shows no change in the calling spec for that parameter.

Querying Oracle Text using phrase with equivalence operator and NEAR

Hello,
I have two queries I'm running that are returning puzzling results. One query is a subset of the other. The queries use a NEAR operator and an equivalence operator.
Query 1:
NEAR((sister,father,mother=yo mama=mi madre),20) This is returning 3 results
I believe Query 1 should return all records containing the words sister AND father AND (mother OR yo mama OR mi madre) that are within 20 words of each other.
Query 2 (a subset of Query 1):
NEAR((sister,father,mother=yo mama),20) This is returning 5 results
I believe Query 2 should return all records containing the words sister AND father AND (mother OR yo mama) that are within 20 words of each other.
Why would Query 1 be returning fewer results than Query 2, when Query 2 is a subset of Query 1? Shouldn't Query 1 return at least the same amount or more results than Query 2?
~Mimi

For future questions about Oracle Text, you can try the Oracle Text forum at: Text
There you have more chances of recieveing an awnser.

Oracle text - issue with contains query

Hello,
Need urgent help.
Following code in my procedure is giving me error.
TYPE c_1 is ref cursor;
result_cursor c1;
i_text2 := 'NEW%';
open result_cursor for
'select /*+ INDEX_SS_DESC(e cad_addr_idx2 )*/
from cad_address
where
contains(text, {:i_text2}, 1) > 0
and rec_type in (1,2,3,4)
order by occur_count desc'
using
i_text2;
ORA-00936: missing expression
ORA-06512: at "AV_OWNER.MY_PROC", line 43
ORA-06512: at line 6
Oracle version is 11.2.0.3.0.
Thanks,

check your table is 'text indexed' on this 'Text' column.To knoow more about 'text index' go to
http://docs.oracle.com/cd/B19306_01/text.102/b14217/ind.htm
Also refer to the below thread where someone had faced issues with CONTAINS clause.
ORA-20000: Oracle Text error: DRG-10599: column is not indexed

Select List with a Contains clause (oracle text)

Hi, I am trying to create a dynamic select list, but am running into problems using bind variables.
I am trying to do this:
select distinct country d, country r
from table1
where contains(:P6_LIST_VALUE,''''|| :P6_NAME||'''');
I keep getting the error: "LOV query is invalid, a display and return value are needed, the column names need to be different...."
If I hard code the :P6_LIST_VALUE to be 'NAME' (which is what I named my context index) it works fine:
where contains(name, ''''||:P6_NAME||'''');
Is there any way to get around this error?

anonymous - Please tell us your first name.
CONTAINS Operator
Syntax
CONTAINS(
[schema.]column,
text_query VARCHAR2
[,label NUMBER])
RETURN NUMBER;
So you can't use a bind variable for the column name, just like you can't do "SELECT :COL FROM DUAL".
Try &P6_LIST_VALUE. (with trailing period). This effects a textual substitution before the SQL is parsed.
Scott

Fuzzy Operator with Contains (Oracle Text)

Similar Messages

Maybe you are looking for