Oracle Text Context index keeps growing. Optimize seems not to be working

Hi,
In my application I needed to search through many varchar columns from differents tables.
So I created a materialized view in which I concatenate those columns, since they exceed the 4000 characters I merged them concatenating the columns with the TO_CLOBS(column1) || TO_CLOB(column)... || TO_CLOB(columnN).
The query is complex, so the refresh is complete on demand for the view. We refresh it every 2 minutes.
The CONTEXT index is created with the sync on commit parameter.
The index then is synchronized every two minutes.
But when we run the optimize index it does not defrag the index. So it keeps growing.
Any idea ?
Thanks, and sorry for my poor english.
Edited by: detryo on 14-mar-2011 11:06

What are you using to determine that the index is fragmented? Can you post a reproducible test case? Please see my test of what you described below, showing that the optimization does defragment the index.
SCOTT@orcl_11gR2> -- table:
SCOTT@orcl_11gR2> create table test_tab
  2    (col1  varchar2 (10),
  3       col2  varchar2 (10))
  4  /
Table created.
SCOTT@orcl_11gR2> -- materialized view:
SCOTT@orcl_11gR2> create materialized view test_mv3
  2  as
  3  select to_clob (col1) || to_clob (col2) clob_col
  4  from   test_tab
  5  /
Materialized view created.
SCOTT@orcl_11gR2> -- index with sync(on commit):
SCOTT@orcl_11gR2> create index test_idx
  2  on test_mv3 (clob_col)
  3  indextype is ctxsys.context
  4  parameters ('sync (on commit)')
  5  /
Index created.
SCOTT@orcl_11gR2> -- inserts, commits, refreshes:
SCOTT@orcl_11gR2> insert into test_tab values ('a', 'b')
  2  /
1 row created.
SCOTT@orcl_11gR2> commit
  2  /
Commit complete.
SCOTT@orcl_11gR2> exec dbms_mview.refresh ('TEST_MV3')
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> insert into test_tab values ('c a', 'b d')
  2  /
1 row created.
SCOTT@orcl_11gR2> commit
  2  /
Commit complete.
SCOTT@orcl_11gR2> exec dbms_mview.refresh ('TEST_MV3')
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> -- query works:
SCOTT@orcl_11gR2> select * from test_mv3
  2  where  contains (clob_col, 'ab') > 0
  3  /
CLOB_COL
ab
c ab d
2 rows selected.
SCOTT@orcl_11gR2> -- fragmented index:
SCOTT@orcl_11gR2> column token_text format a15
SCOTT@orcl_11gR2> select token_text, token_first, token_last, token_count
  2  from   dr$test_idx$i
  3  /
TOKEN_TEXT      TOKEN_FIRST TOKEN_LAST TOKEN_COUNT
AB                        1          1           1
AB                        2          3           2
C                         3          3           1
3 rows selected.
SCOTT@orcl_11gR2> -- optimizatino:
SCOTT@orcl_11gR2> exec ctx_ddl.optimize_index ('TEST_IDX', 'REBUILD')
PL/SQL procedure successfully completed.
SCOTT@orcl_11gR2> -- defragmented index after optimization:
SCOTT@orcl_11gR2> select token_text, token_first, token_last, token_count
  2  from   dr$test_idx$i
  3  /
TOKEN_TEXT      TOKEN_FIRST TOKEN_LAST TOKEN_COUNT
AB                        2          3           2
C                         3          3           1
2 rows selected.
SCOTT@orcl_11gR2>

Similar Messages

  • Suggestion: Oracle text CONTEXT index on one or more columns ?

    Hi,
    I'm implementing Oracle text using CONTEXT ..... and would like to ask you for performance suggestion ...
    I have a table of Articles .... with columns .. TITLE, SUBTITLE , BODY ...
    Now is it better from performance point of view to move all three columns into one dummy column ... with name like FULLTEXT ... and put index on this single column,
    and then use CONTAINS(FULLTEXT,'...')>0
    Or is it almost the same for oracle if i put indexes on all three columns and then call:
    CONTAINS(TITLE,'...')>0 OR CONTAINS(SUBTITLE,'...')>0 OR CONTAINS(BODY,'...')>0
    I actually don't care if the result is a match in TITLE OR SUBTITLE OR BODY ....
    So if i move into some FULLTEXT column, then i have duplicate data in a article row ... but if i create indexes for each column, than oracle has 2x more to index,optimize and search ... am I wright ?
    Table has 1.8mil records ...
    Thank you.
    Kris

    mackrispi wrote:
    Now is it better from performance point of view to move all three columns into one dummy column ... with name like FULLTEXT ... and put index on this single column,
    and then use CONTAINS(FULLTEXT,'...')>0What version of Oracle are you on? If 11 then you could use a virtual column to do this, otherwise you'd have to write code to maintain the column which can get messy.
    mackrispi wrote:
    Or is it almost the same for oracle if i put indexes on all three columns and then call:
    CONTAINS(TITLE,'...')>0 OR CONTAINS(SUBTITLE,'...')>0 OR CONTAINS(BODY,'...')>0Benchmark it and find out :)
    Another option would be something like this.
    http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:9455353124561
    Were i you, i would try out those 3 approaches and see which meet your performance requirements and weigh that with the ease of implementation and administration.

  • ORACLE TEXT  Context index so big, why ?

    Hi,
    One year ago, I've created a context index, for a table that had 2 million rows , and the context index was around 2.8 Mil rows ....
    Today we have 2.9mil rows but the Context index has 95 Mil rows ??
    Last weekend I did a FULL optimize index and it took 28hours and it was still not finished .... so this weekend I droped the index and re-created
    it .... it took 6 hours ....
    Now I started CTX_REPORT.INDEX_STATS to see how it looks like .... cause I exprected 95mil to become like 3.5mil max not more ....
    Can anyone please tell me what should I check to see why did this happen so that the last 900.000 records created 91 mil rows in index ?
    If there would be something wrong, why would the last 900.000 rows create something that prev 2mil rows did not ...
    HEre is index creation :
    CREATE INDEX ART_IDX ON MS_ARTICLE(ORATEXT) INDEXTYPE IS CTXSYS.CONTEXT  FILTER BY ID,ART_DATE,MEDIA_CODE ORDER BY
    ART_DATE DESC PARAMETERS (' LEXER ART_LEX STOPLIST CTXSYS.EMPTY_STOPLIST sync(ON COMMIT) DATASTORE DS_ART');If anyone needs definition of lexer od datastore, then please tell me what do I have to do to get definitions of that.
    Thank you.
    KRis

    Hi,
    Ok there must be something I did wrong .... cause after running INDEX_STATS ...I got this results :
    ===========================================
                        STATISTICS FOR "PRESCLIP"."ART_IDX"
    ===========================================
    indexed documents:                                              2,876,228
    allocated docids:                                               2,876,228
    $I rows:                                                       96,110,561
                                 TOKEN STATISTICS
    unique tokens:                                                  5,584,188
    average $I rows per token:                                          17.21
    tokens with most $I rows:
      JE (0:TEXT)                                                      11,650
      V (0:TEXT)                                                       11,183
      IN (0:TEXT)                                                      10,085So I have expected Average rows per token to be 1 after droping and re-creating index ..... so what else must I do to get the number 96mil to 5.5mil ?
    Thank you.
    Kris

  • Oracle Text, create index (indextype is ctxsys.context)

    Dear sirs,
    I am a new user of Oracle Text (Oracle 11g release 11.2) and I am unable to create an index of type ctxsys.context). Any suggestions?:
    code:
    drop table mytable;
    drop index myindex force;
    create table mytable(id number primary key, docs clob);
    insert into mytable values(111555,'this text will be indexed');
    insert into mytable values(111556,'this is a default datastore example');
    commit;
    create index myindex on mytable(docs)
    indextype is ctxsys.context
    parameters ('DATASTORE CTXSYS.DEFAULT_DATASTORE');
    +++++++
    error messages:
    create index myindex on mytable(docs)
    ERROR at line 1:
    ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
    ORA-20000: Oracle Text error:
    ORA-06508: PL/SQL: could not find program unit being called
    ORA-06512: at "CTXSYS.DRUE", line 160
    ORA-06512: at "CTXSYS.TEXTINDEXMETHODS", line 366

    Please check for invalid objects. Log on as sys or system and run:
    select owner, object_name, object_type from all_objects where status='INVALID';
    Post the results here and we'll advise the next step.
    If any objects owned by CTXSYS are invalid you may need to recompile the CTXSYS schema.

  • Having upgraded all my devices to IOS 6, including iPad And upgraded no montan limón no my macbookpro, iCloud seems not to be working as smoothly anymore And some impagos docs have not transferred. Help

    Having upgraded all my devices to IOS 6, including iPad And upgraded no montan limón no my macbookpro, iCloud seems not to be working as smoothly anymore And some impagos docs have not transferred. Help

    Having upgraded all my devices to IOS 6, including iPad And upgraded no montan limón no my macbookpro, iCloud seems not to be working as smoothly anymore And some impagos docs have not transferred. Help

  • HT4628 My airport seems not to be working anymore. I have internet but now I can't play music through Itunes remotely. Airport kept flashing amber. Now I have tried to reset it it only flashes briefly amber and then no more light and it seems disconnected

    My airport seems not to be working anymore. I have internet but now I can't play music through Itunes remotely. Airport kept flashing amber. Now I have tried to reset it it only flashes briefly amber and then no more light and it seems disconnected?? Any help out there?

    Unfortunately the Netgear DG834N router is rubbish with Macbook Pro's. I've had a few netgear routers, which were all good with my mac, which is why I bought the 'N' series one as an upgrade to the 'G' version, but basically it crashes my Mac constantly. I get the fade up grey screen forcing my to hard power off my book. everything freezes. It plain just doesn't work. wireless 'N' is still just in draft form and the difference between the Netgear and Mac version on the macbook pro just clash. Netgear haven't updated the firmware on the router since last year, so hopefully the will soon, but I've just gone back to my old 'G' version, and stuck the DG834N on ebay. I tried all the configs, with and with out security and no luck, it always disconnected, froze and crashed.

  • Oracle Text ALTER INDEX Performance

    Greetings,
    We have encountered some ehancement issues with Oracle Text and really need assistance.
    We are using Oracle 9i (Release 9.0.1) Standard Edition
    We are using a very simple Oracle text environmet, with CTXSYS.CONTEXT indextype on Domain Indexes.
    We have indexed two text columns in one table, one of these columns is CLOB.
    Currently if one of these columns is modified, we are using a trigger to automatically ALTER the index.
    This is very slow, it is just like dropping the index and creating it again.
    Is this right? should it be this slow?
    We are also trying to use the ONLINE parameter for ALTER INDEX and CREATE INDEX, but it gives an error saying this feature is not enabled.
    How can we enable it?
    Is there any way in improving the performance of this automatic update of the indexes?
    Would using a trigger be the best way to do this?
    How can we optimize it to a more satifactory performance level?
    Also, are we able to use the language lexers for indexes with the Standard Edition. If so, how do you enable the CTX_DLL?
    Many thanks for any assistance.
    Chi-Shyan Wang

    If you are going to sync your index on every update, you need to make sure that you are optmizing it on a regular basis to remove index fragmentation and remove deleted rows.
    you can set up a dmbs_job to do a ctx_ddl.optmize and run a full optmize periodically.
    Also, depending on the number of rows you have, and also the size of the data, you might want to look at using a CTXCAT index, which is transactional, stays in sync automatically and does not need to be optimized. CTXCAT indexes do not work well on large text objects (they are good for a couple lines of text at most) so they may not suit your dataset.

  • How do I get Oracle Text to index files on a file server?

    I am new to Oracle (I'm a MS-SQL DBA looking for a Full-Text Search solution that is better than linking to a MS index server.)
    So - Here's the objective:
    I have Oracle Server(Express) installed on a Windows server.
    I would like for Oracle to build a Full-Text Catalog of the files on a separate file server based on file paths in a table in the database.
    (No desire to store terabytes of images and documents inside the database)
    I can get Oracle text up and running, using the URL_Datastore:
    CREATE TABLE files (id NUMBER PRIMARY KEY, issue_id NUMBER, path VARCHAR(255) UNIQUE, ot_format VARCHAR(6), ot_version VARCHAR(10));
    The Compaq server is a remote windows server on my local workgroup, so the fully qualified path is just "compaq" and the URL is valid:
    INSERT INTO files VALUES (9,9,'file://Compaq/FTQ/00000003.pdf',NULL,NULL);
    INSERT INTO files VALUES (13,13,'file://Compaq/FTQ/01.txt',NULL,NULL);
    CREATE INDEX file_index ON files(path) INDEXTYPE IS ctxsys.context
    PARAMETERS ('datastore ctxsys.URL_DATASTORE format column ot_format');
    but when I enter:
    Select * from CTX_User_Index_errors, I see the following errors:
    DRG-11609: URL store: unable to open local file specified by file://Compaq/FTQ/00000003.pdf
    DRG-11609: URL store: unable to open local file specified by file://Compaq/FTQ/01.txt
    Did I miss something?
    Do I need to install anything on the file server?
    I would like to convince my company that Oracle can be much quicker than Microsoft's Indexing Service because it can avoid joining two large result sets (one result set from Full_text (indexing service) and one for specific data contained in fields in the MS-SQL database.) Full Text Searches commonly take 40 - 60 seconds where there are 1.5 million multi-page PDF files for a particular set that I sample search on. Without this massive join, I believe I can get the search to run in under 10 seconds.

    Thank you!
    File_Datastore worked fine.
    I was staying away from File_Datastore because the information I gathered from googling suggested that file_datastore would only work locally.
    Now I just have to get Oracle to pull data out of tables in a MS-SQL database on the local network (don't have a clue yet), and then have it index compiled file paths.
    Then MS-SQL can query Oracle with index and full-text criteria and Oracle can send back a result set
    It may sound like a bad way of performing Full-Text Queries, but anything will be better than the way things are currently running. We are currently performing Full Text Searches on a table that is rebuilt nightly, so the table containing millions of file paths is not live..
    It would be so much better if we just migrated to Oracle, but we currently do not have the resources.

  • APEX app using Oracle Text  to index pages that require authorzation

    Hi Gurus and APEX Dev team
    My team need to develop an APEX App that will index all our documents spread across various servers. Some of the documents require Single sign on access (e.g. KIX.oraclecorp.com) and some require other authorization methods (e.g. Metalink) . The Question is , Is it possible to index the pages that require authorization using Oracle text. If yes How? I have implemented the demo app which can index pages that do not require authorization.
    Thanks a million
    regards
    Bala

    Hello,
    Unless I misunderstand you, the fact that the pages require authentication doesn't really matter, it is the underlying data you want to index correct? If so then you would index them in exactly the same way that you would index any table data using Oracle Text/interMedia.
    John.
    Blog: http://jes.blogs.shellprompt.net
    Work: http://www.apex-evangelists.com
    Author of Pro Application Express: http://tinyurl.com/3gu7cd
    REWARDS: Please remember to mark helpful or correct posts on the forum, not just for my answers but for everyone!

  • Oracle 11g flash recovery keeps growing

    Oracle 11g flash recovery (E:\ partition in a Windows 2003 server) keeps growing. It already used 200GB of disk space and only 46GB of disk space is left in that partition. Can we tell how often Oracle 11g delete the old backups?
    Thanks.
    Andy

    andychow wrote:
    SQL> set linesize 1000
    SQL> select substr(name,1,30) name,
    2         space_limit,
    3         space_used,
    4         space_reclaimable,
    5         number_of_files
    6  from V$RECOVERY_FILE_DEST
    7  ;
    NAME                           SPACE_LIMIT SPACE_USED SPACE_RECLAIMABLE NUMBER_OF_FILES
    E:\flash_recovery_area_11       1.0486E+11 1.6752E+10                 0               5
    SQL> select *
    2  from V$FLASH_RECOVERY_AREA_USAGE
    3  ;
    FILE_TYPE            PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
    CONTROL FILE                          0                         0               0
    REDO LOG                              0                         0               0
    ARCHIVED LOG                        .09                         0               2
    BACKUP PIECE                      15.88                         0               3
    IMAGE COPY                            0                         0               0
    FLASHBACK LOG                         0                         0               0
    FOREIGN ARCHIVED LOG                  0                         0               0
    7 rows selected.
    SQL>
    It shows only 5 files in E:\flash_recovery_area_11. Can you verify ?

  • A Cooling Fan seems not to be working

    A message from HP: A cooling Fan seems non to be working, the system will shut dowin in 15 seconds

    Hi ErmannoFelici,
    Thank you for visiting the HP Support Forums and Welcome. I have read your thread on your HP Notebook and getting a cooling fan error. Here is a link to troubleshoot overheating.
    I would be happy to assist if needed. How Do I Find My Model Number or Product Number?
    Please respond with which Operating System you are running:
    Which Windows Operating System am I running?
    Please let me know.
    Thanks.
    Please click “Accept as Solution ” if you feel my post solved your issue, it will help others find the solution.
    Click the “Kudos, Thumbs Up" on the bottom to say “Thanks” for helping!

  • My loop statement in smartforms seems not to be working...

    Hello Experts,
    I am currently practicing smartforms and I am trying to display the records of my internal table.
    What I did was in my table under MAIN window I have put the statement:
    it_spfli into wa_spfli and also in my main area of my table I have a loop statement. Now, in one of my text
    i put wa_spfli-carrid, wa_spfli-countryfr, etc. But it still does not show any data.Help would be greatly appreciated.

    u will be getting a function module name for the smartform then
    goto se37-->enter the FM name -->search for the ITAB you r using then put a break point.
    come back to u r program the run.. controle will stops over there then check whether the data is populating or not.
    INPUT and OUTPUT patameters are for program line and INITIALIZATION tabs.. when you want to modify some data in these you need to give the veriable or ITABs u r using inside. so that system will determine the global declarations(veriable or ITAbs).
    for more info refer these links
    http://www.sap-img.com/smartforms/sap-smart-forms.htm
    http://www.sap-img.com/smartforms/smartform-tutorial.htm
    http://www.sapgenie.com/abap/smartforms.htm

  • Changing a mapplet select seems not to be working

    I am trying to change the mapplet select for the BOM Header extraction task. I've copied the mapping from the ORA R12 Adaptor to my custom folder and I've altered the mapplet in my custom folder. I've also copied the informtica task and workflow.
    In DAC I've pointed the task BOM Header to pick the task from my custom folder.
    Session Log says custom folder... but it seems that the mapplet is still running the original select. (I've changed the mapplet to limit the number of rows ROWNUM < 10 but I can see in DS table more than 400K records...
    Anything I am forgotting ?
    Txs. a lot.
    Antonio

    Sometimes the SQL in a mapping is ignored because there is a session SQL override in the workflow? In workflow manager, select workflow, select task, select tab mapping, select source qualifier, look at property "SQL Query".

  • Items BulkAction - "Copy" seems NOT to be working?

    I AM TRYING TO COPY MANY ITEMS USING THE "BULK ACTION" FROM ONE FOLDER TO ANOTHER IN THE SAME CONTENT AREA.
    I FOLLOWED THE CORRECT STEPS BUT NO ITEM IS COPIED.
    THE BULK ACTION "DELETE" SEEMS TO BE WORKING OK.
    DID ANYBODY EXPERIENCE THE SAME PROBLEM?
    I AM USING PORTAL 3.0.9 / W2000
    I WORK IN SPANISH LANGUAGE.
    ANOTHER THING, COPYING ONE ITEM AT A TIME (USING THE NORMAL "COPY ITEM" FUNCTION WORKS FINE).
    TKS!

    Maria,
    It works fine for me. Have you applied all the relevant 3.0.9 patches? Please check Metalink. Note that some of the patches include other patches, so read the patch release notes carefully.
    Regards,
    Jerry

  • Subwoofer seems not to be working?

    Last week, I purchased Best Buy's daily deal, the Insignia rocketboost wireless subwoofer.  With it I purchased the Rocketfish-rocketboost wireless receiver/transmitter.  I hooked up the transmitter to the back audio out of my Sony Bravia tv.  I plugged in the subwoofer across the room.  I was able to easily connect them.  Both blue lights are on the back of the subwoofer, and all lights are steady on the transmitter.  But I hear nothing coming from the subwoofer?  I tried changing the input on my tv, but none other worked than the HMD1.  I then plugged in the transmitter to the audio out of my dvd player and plugged that back into my tv where it was, but still nothing.  Not sure what is wrong? Can anyone help?
    Thanks!
    Olivia

    You might have to post some pictures of your set up, that might make this easier. Or atleast post some model numbers as products are different even within manufactures product lines.  
    But, I still believe on Sony tv's you have to tell them you are using some time of audio out. 
    I dont' know why you're using ANY type of external sound without a A/V reciever anyways. Unless it's a soundbar. 
    Former BBY Home Theater Associate 2010 - 2012. Now I work as a graphic designer in the print and prepress industry.

Maybe you are looking for