Browse folders

Question

Hi again,
great! Now that I've discovered this wonderful forum, I can ask all the questions I've always wanted to ask about XML DB but never dared to ;-)
I frequent use case with XML DB is that you want to find out about folders and subfolders from your application.
I'm wondering what is the recommended way to list them?
Say we've got:
SELECT any_path FROM resource_view WHERE UNDER_PATH(res,'/public')=1 AND EXISTSNODE(res,'/r:Resource[@Container="true"]','xmlns:r="http://xmlns.oracle.com/xdb/XDBResource.xsd"')=1
This will list me all folders under public. But it is rather slow and will take a few minutes to complete.
As an aside, interestingly, when I try:
SELECT any_path FROM resource_view WHERE EQUALS_PATH(res,'/public')=1 AND EXISTSNODE(res,'/r:Resource[@Container="true"]','xmlns:r="http://xmlns.oracle.com/xdb/XDBResource.xsd"')=1
I get ORA-31038: Invalid hexBinary value: "true"
Why? Weird, indeed!
OK. Trying now this:
SELECT any_path FROM resource_view WHERE UNDER_PATH(res,'/public')=1 AND EXTRACTVALUE(res,'/r:Resource/@Container','xmlns:r="http://xmlns.oracle.com/xdb/XDBResource.xsd"')='true'
Only this finishes in an acceptable time.
Unfortunately, when I try:
SELECT any_path FROM resource_view WHERE UNDER_PATH(res,1,'/public')=1 AND EXTRACTVALUE(res,'/r:Resource/@Container','xmlns:r="http://xmlns.oracle.com/xdb/XDBResource.xsd"')='true'
it doesnt use the XMLIndex I've created on the repository and it is slow again.
The depth parameter of UNDER_PATH seems to make it forget about the XMLIndex.
So, it's faster to read all folders from the whole repository than just one level.
Is there any way I can force it to use the XMLIndex anyway? Or is there a different way to find out only about folders?
Thank you,
republic

Answer 1

Some different attempts, probably no use yet, but enabling text indexing...
http://download.oracle.com/docs/cd/E11882_01/appdev.112/e16659/xdb18res.htm#insertedID5
SQL> conn / as sysdba
Connected.
SQL>  @?/rdbms/admin/dbmsxdbt
Grant succeeded.
Grant succeeded.
Grant succeeded.
Package created.
No errors.
Package body created.
No errors.
Procedure created.
No errors.
SQL> host cat $ORACLE_HOME/rdbms/admin/dbmsxdbt.sql
Rem
Rem $Header: rdbms/admin/dbmsxdbt.sql /main/11 2009/10/08 17:21:09 attran Exp $
Rem
Rem dbmsxdbt.sql
Rem
Rem Copyright (c) 2002, 2009, Oracle and/or its affiliates.
Rem All rights reserved.
Rem
Rem    NAME
Rem      dbmsxdbt.sql - XDB conText index setup
Rem
Rem    DESCRIPTION
Rem      Useful routines and scripts for conText indexes on XDB
Rem
Rem    NOTES
Rem      Must be connected as SYS to run this script
Rem
Rem    MODIFIED   (MM/DD/YY)
Rem    attran      09/09/09 - 8915200 - IFILTER -> PolicyFilter
Rem    badeoti     03/19/09 - dbms_xdbz.get_username moved to dbms_xdbz0
Rem    llsun       06/19/08 - bug 7137308: xdb.dbms_xdbt
Rem    petam       02/04/05 - fix for bug 4148632
Rem    najain      06/05/03 - dont use XDB_RESINFO
Rem    najain      08/01/03 - 3071986: index more fields
Rem    smuralid    01/08/03 - dbms_xdbt: ctx security rewrite
Rem    sichandr    10/03/02 - getCharsetId: handle Oracle names too
Rem    rshaikh     08/08/02 - bug2456600: change IndexMemory to be
Rem                           equal to MAX_INDEX_MEMORY
Rem    smuralid    01/28/02 - globalization support
Rem    smuralid    01/24/02 - Merged smuralid_repos_search
Rem    smuralid    01/23/02 - Created
Rem
Rem
Rem Make sure that XDB can see these packages
Rem
GRANT EXECUTE ON ctxsys.ctx_ddl TO xdb;
GRANT EXECUTE ON ctxsys.ctx_output TO xdb;
GRANT ctxapp TO xdb;
Rem
Rem Package to simplify context index creation on XDB
Rem
CREATE OR REPLACE PACKAGE xdb.dbms_xdbt AUTHID CURRENT_USER IS
  --  OVERVIEW
  --    This package provides utilities for creating and managing conText
  --    indexes on the XDB repository.
  --    The preferred mode of operation is as follows
  --    (a) drop any existing preferences. dbms_xdbt.dropPreferences
  --    (b) re-create preferences for the index
  --        (dbms_xdbt.createPreferences)
  --    (c) Create the index
  --        (dbms_xdbt.createIndex)
  --        Verify that things have gone smoothly using
  --          "select * from ctx_user_index_errors"
  --    (d) Setup automatic sync'ing of the index
  --        (dbms_xdbt.configureAutoSync)
  --    (e) Sit back and relax
  --    The package spec contains a a list of package variables that
  --    describe the configuration settings. These are intended to
  --    cover some of the more basic customizations that installations
  --    might require, but is not intended to be a complete set.
  --    There are 2 ways to customize this package.
  --    (a) Use a PL/SQL procedure to set the appropriate package variables
  --        that control the relevant configurations, and then execute
  --        the package. Obviously, this only applies to the set of existing
  --        package variables
  --    (b) The more general approach is to modify (in place, or as a copy)
  --        this package to introduce the appropriate customizations
  --    For instance, if you need to change the amount of memory available
  --    for indexing, you could use option (a).
  --    NOTES:
  --      If you're using this package as is, please note the following
  --    (a) Make sure that the LOG_DIRECTORY parameter is set using
  --        ctx_adm.set_parameter
  --        Alternately, turn off rowid logging by setting the
  --        'LogFile' package variable to the empty string.
  --    (b) Make sure that the MAX_INDEX_MEMORY parameter is at least
  --        128M. Other change the package variable 'IndexMemory'
  --        appropriately
TYPE varcharset IS TABLE OF VARCHAR2(100);
-- CONSTANTS
-- FILTERING OPTIONS
-- The following constants describe the kinds of filtering we may want
-- to do.
-- USE_NULL_FILTER simply sends the document over to the charset converter
-- USE_INSO_FILTER uses the IFILTER api of INSO to convert the document
--    into HTML
-- SKIP_DATA is used to completely ignore the document's contents for
-- filtering. (The document metadata is indexed, however)
USE_NULL_FILTER            CONSTANT PLS_INTEGER := 1;
USE_INSO_FILTER            CONSTANT PLS_INTEGER := 2;
SKIP_DATA                  CONSTANT PLS_INTEGER := 3;
-- Sync options
-- There are basically two mechanisms of automatic sync provided here.
-- SYNC_BY_PENDING_COUNT indicates that when the number of entries in the
--   pending queue reaches a threshold, it is time to sync up the index
-- SYNC_BY_TIME indicates that the index should be synced up at regular
--   intervals
-- SYNC_BY_PENDING_COUNT_AND_TIME is a combination of both these strategies
SYNC_BY_PENDING_COUNT      CONSTANT PLS_INTEGER := 1;
SYNC_BY_TIME               CONSTANT PLS_INTEGER := 2;
SYNC_BY_PENDING_COUNT_AND_TIME CONSTANT PLS_INTEGER := 3;
SyncTimeOut                NUMBER := NULL;
-- CONFIGURATION SETTINGS
-- This section contains the default settings for the index. This
-- section should be changed as appropriate.
-- The name of the context index
IndexName         CONSTANT VARCHAR2(32) := 'XDB$CI';
-- The default memory to be used for index creation and sync
-- NOTE: This must be less than (or equal to) the MAX_INDEX_MEMORY
--       parameter
IndexMemory       CONSTANT VARCHAR2(32) := '50M';
-- SYNC OPTIONS
-- The following section describes the automatic sync policy.
-- By default, the auto-sync policy (once it has been configured)
-- is to sync based on the pending count.
-- Should we sync up based on pending count, or time or both ?
AutoSyncPolicy             PLS_INTEGER := SYNC_BY_PENDING_COUNT;
-- This parameter determines the maximum size of the pending queue
--   before the index is sync-ed. Applies only when the sync policy
--   is SYNC_BY_PENDING_COUNT or SYNC_BY_TIME_AND_PENDING_COUNT
MaxPendingCount            PLS_INTEGER := 2;
-- This parameter determines the interval - in minutes - at
-- which the "regular" sync should be performed on the index.
-- Applies only to the SYNC_BY_TIME and the SYNC_BY_PENDING_COUNT_AND_TIME
-- policies
SyncInterval               PLS_INTEGER := 60;
-- This parameter determines how frequently - in minutes - the pending
-- queue is polled. Applies only to the SYNC_BY_PENDING_COUNT
-- (and the SYNC_BY_PENDING_COUNT_AND_TIME) policies
CheckPendingCountInterval  PLS_INTEGER := 10;
-- LOGGING OPTIONS
-- Please ensure that the LOG_DIRECTORY parameter is
-- already set if you need rowid logging
-- Logging options. This parameter determines the logfile used - for
-- rowid logging during index creation etc.
-- Set this parameter to NULL to avoid rowid-logging
LogFile                    VARCHAR2(32) := '';
-- FILTER OPTIONS
-- The following classes determine the filtering options based on the
-- mime type of the document
-- The skipFilter_Types list contains a list of regular expressions
-- that describe mime types for which the document is *not* to be
-- filtered/indexed. (The document metadata, however, is still indexed)
-- Use this for document types that cannot really be indexed. Good
-- examples of this class are images, audio files etc
-- The NullFilter_Types list contains a list of regular expressions
-- that describe mime types of documents for which no INSO filtering
-- is required - these documents should be basically text formats
-- and the only filtering required should be character set conversion
-- (if needed)
-- For any given document, the skipFilter_Types list is first scanned
-- to determine if any regular expression in that list matches the
-- document's content type. If it does, then the document content is
-- not indexed.
-- Failing this, the NullFilter_Types list is then scanned. If any
-- regular expression in this list matches the document's content type,
-- the document is sent through character-set conversion
-- failing this, the document is filtered using the INSO filter (using the
-- IFILTER interfaces)
SkipFilter_Types varcharset := varcharset('image/%', 'audio/%', 'video/%',
                                          'model/%');
NullFilter_Types varcharset := varcharset('%text/plain', '%text/html',
                                          '%text/xml');
-- STOPWORD Settings
-- This list describes the set of stopwords over and above that
-- specified by the CTXSYS.DEFAULT_STOPLIST stoplist
StopWords        varcharset := varcharset('0','1','2','3','4','5','6',
                                          '7','8','9',
                                          'a','b','c','d','e','f','g','h','i',
                                          'j','k','l','m','n','o','p','q','r',
                                          's','t','u','v','w','x','y','z',
                                          'A','B','C','D','E','F','G','H','I',
                                          'J','K','L','M','N','O','P','Q','R',
                                          'S','T','U','V','W','X','Y','Z'
-- LEXER preferences
-- This parameter determines if multi-language lexers can be
-- used.
-- Not supported currently, and this parameter should always be FALSE
UseMultiLexer   BOOLEAN := false;
-- SECTION GROUP
-- This parameter determines the sectioner to use.
-- By default, this is an HTML section group. No zone sections have been
-- created - (ie) WITHIN searches are not possible
-- If the vast majority of documents are XML or XML-like, consider using
-- the AUTO_SECTION_GROUP or the PATH_SECTION_GROUP or even a
-- NULL_SECTION_GROUP
SectionGroup    VARCHAR2(100) := 'HTML_SECTION_GROUP';
-- PUBLIC INTERFACES
-- The public APIs exposed by this package
  -- This procedure drops all preferences required by the context index
  PROCEDURE dropPreferences;
  -- This procedure creates all preferences required by the context index
  -- on the XDB repository.
  -- The set of preferences include Datastore, Storage, Filter, Lexer,
  -- SectionGroup, Stoplist and Wordlist preferences
  -- NOTE: This will raise exceptions if any of the preferences already
  --       exist
  PROCEDURE createPreferences;
  -- Creates the datastore preference
  -- Will raise an exception if the datastore already exists
  PROCEDURE createDatastorePref;
  -- Creates the storage preferences
  -- Will raise an exception if the preference already exists
  PROCEDURE createStoragePref;
  -- Creates the section group
  -- Will raise an exception if the preference already exists
  PROCEDURE createSectiongroupPref;
  -- Creates the filter preference
  -- Will raise an exception if the preference already exists
  PROCEDURE createFilterPref;
  -- Creates the lexer preference
  -- Will raise an exception if the preference already exists
  PROCEDURE createLexerPref;
  -- Creates the stoplist
  -- Will raise an exception if the preference already exists
  PROCEDURE createStoplistPref;
  -- Creates the wordlist
  -- Will raise an exception if the preference already exists
  PROCEDURE createWordlistPref;
  -- Creates the index
  -- This requires the above preferences to have already been created.
  -- (a) The LOG_DIRECTORY parameter must be set (to enable
  --     rowid logging during index creation)
  -- (b) Ensure that the memory size specified to index creation is less than
  --     the MAX_INDEX_MEMORY parameter
  PROCEDURE createIndex;
  -- Syncs up the index
  -- This can be used to explicitly sync up the index.
  -- The preferred mechanism is to set up automatic sync'ing with
  -- the "configureAutoSync" procedure
  PROCEDURE syncIndex(myIndexName VARCHAR2 := Indexname,
                      myIndexMemory VARCHAR2 := IndexMemory);
  -- Set a suggested time limit on the SYNC operation, in minutes.
  -- SYNC_INDEX will process as many documents in the queue as possible
  -- within the time limit.
  -- The maxtime value of NULL is equivalent to CTX_DDL.MAXTIME_UNLIMITED.
  PROCEDURE setSyncTimeout(timeout IN INTEGER := NULL);
  -- Optimizes the index
  PROCEDURE optimizeIndex;
  -- Configures for automatic sync of the index
  -- NOTE: The system must be configured for job queues. Also, the
  --   number of job queue processes must be non-zero
  PROCEDURE configureAutoSync;
  -- Procedure used by dbms_job to automatically sync up the context
  -- index
  -- Don't use this directly
  PROCEDURE autoSyncJobByCount(myIndexName VARCHAR2, myMaxPendingCount NUMBER,
                               myIndexMemory VARCHAR2);
  -- Procedure used by dbms_job to automatically sync up the context
  -- index
  -- Don't use this directly
  PROCEDURE autoSyncJobByTime(myIndexName VARCHAR2, myIndexMemory VARCHAR2);
  -- The user-datastore procedure
  -- Do *not* call this directly
  PROCEDURE xdb_datastore_proc(rid IN ROWID, outlob IN OUT NOCOPY CLOB);
end dbms_xdbt;
show errors;
CREATE OR REPLACE PACKAGE BODY xdb.dbms_xdbt AS
-- CONFIGURATION SETTINGS
-- This section contains the default settings for the index. This
-- section should be changed as appropriate.
-- The following are the default values for the policy_filter
-- used to filter out HTML, from BLOB into CLOB.
-- 'Policy-based' procedures do not make use of the Ctx index.
etc
Rem
Rem Create the "real" user-datastore procedure
Rem Note: This is around only for backward compatibility. In 9iR2, the
Rem user-datastore procedure needed to be owned by CTXSYS. In 10i, it needs
Rem to be owned by the index owner. Changing the user-datastore-proc name
Rem may require a rebuild of the index - which we don't really want, and
Rem hence we create a procedure of the same name in XDB's schema
Rem
CREATE OR REPLACE PROCEDURE xdb.xdb_datastore_proc(rid IN ROWID,
                                                   outlob IN OUT NOCOPY CLOB)
  AUTHID CURRENT_USER IS
BEGIN
  xdb.dbms_xdbt.xdb_datastore_proc(rid, outlob);
END;
show errors;
Rem Remember to set these
Rem exec ctxsys.ctx_adm.set_parameter('MAX_INDEX_MEMORY', '128M');
Rem exec ctxsys.ctx_adm.set_parameter('LOG_DIRECTORY', '/tmp');
SQL>  exec ctxsys.ctx_adm.set_parameter('MAX_INDEX_MEMORY', '128M');
PL/SQL procedure successfully completed.
SQL> exec ctxsys.ctx_adm.set_parameter('LOG_DIRECTORY', '/tmp');
PL/SQL procedure successfully completed.
[oracle@marcoxdb ~]$ sqlplus xdb/xdb
SQL*Plus: Release 11.2.0.2.0 Production on Mon Nov 8 16:14:44 2010
Copyright (c) 1982, 2010, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> CREATE INDEX xdb$resource_ctx_i ON XDB.XDB$RESOURCE(OBJECT_VALUE)
  INDEXTYPE IS CTXSYS.CONTEXT;
  2
Index created.
SQL>