Using a byte[] as a secondary index's key within the Collection's API

I am using JE 4.1.7 and its Collections API. Overall I am very satisfied with the ease of using JE within our applications. (I need to know more about maintenance, however!) My problem is that I wanted a secondary index with a byte[] key. The key contains the 16 bytes of an MD5 hash. However, while the code compiles without error when it runs JE tell me
Exception in thread "main" java.lang.IllegalArgumentException: ONE_TO_ONE and MANY_TO_ONE keys must not have an array or Collection type: example.MyRecord.hash
See test code below. I read the docs again and found that the only "complex" formats that are acceptable are String and BigInteger. For now I am using String instead of byte[] but I would much rather use the smaller byte[]. Is it possible to trick JE into using the byte[]? (Which we know it is using internally.)
-- Andrew
package example;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.PrimaryIndex;
import com.sleepycat.persist.SecondaryIndex;
import com.sleepycat.persist.StoreConfig;
import com.sleepycat.persist.model.Entity;
import com.sleepycat.persist.model.PrimaryKey;
import com.sleepycat.persist.model.Relationship;
import com.sleepycat.persist.model.SecondaryKey;
import java.io.File;
@Entity
public class MyRecord {
@PrimaryKey
private long id;
@SecondaryKey(relate = Relationship.ONE_TO_ONE, name = "byHash")
private byte[] hash;
public static MyRecord create(long id, byte[] hash) {
MyRecord r = new MyRecord();
r.id = id;
r.hash = hash;
return r;
public long getId() {
return id;
public byte[] getHash() {
return hash;
public static void main( String[] args ) throws Exception {
File directory = new File( args[0] );
EnvironmentConfig environmentConfig = new EnvironmentConfig();
environmentConfig.setTransactional(false);
environmentConfig.setAllowCreate(true);
environmentConfig.setReadOnly(false);
StoreConfig storeConfig = new StoreConfig();
storeConfig.setTransactional(false);
storeConfig.setAllowCreate(true);
storeConfig.setReadOnly(false);
Environment environment = new Environment(directory, environmentConfig);
EntityStore myRecordEntityStore = new EntityStore(environment, "my-record", storeConfig);
PrimaryIndex<Long, MyRecord> idToMyRecordIndex = myRecordEntityStore.getPrimaryIndex(Long.class, MyRecord.class);
SecondaryIndex<byte[], Long, MyRecord> hashToMyRecordIndex = myRecordEntityStore.getSecondaryIndex(idToMyRecordIndex, byte[].class, "byHash");
// END

We have highly variable length data that we wish to use as keys. To avoid massive index sizes and slow key lookup we are using MD5 hashes (or something more collision resistant should we need it). (Note that I am making assumptions about key size and its relation to index size that may well inaccurate.)Thanks for explaining, that makes sense.
It would be the whole field. (I did consider using my own key data design using the @Persistent and @KeyField annotations to place the MD5 hash into two longs. I abandoned that effort because I assumed (again) that lookup with a custom key design would slower than the built-in String key implementation.)A composite key class with several long or int fields will not be slower than a single String field, and will probably result in a smaller key since the UTF-8 encoding is avoided. Since the byte array is fixed size (I didn't realize that earlier), this is the best approach.
--mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           

Similar Messages

  • Error while renaming indexes and keys in the tables

    hi gems...good afternoon..
    i am using oracle 11gr2 database.
    when i am trying to rename the primary key, foreign key, indexes of all the tables as per the standard, then the following error is given...
    ORA-00604: error occurred at recursive SQL level 1
    ORA-01000: maximum open cursors exceeded
    ORA-00604: error occurred at recursive SQL level 1
    ORA-01000: maximum open cursors exceeded
    ORA-01000: maximum open cursors exceeded
    currently the maximum open cursors parameter is set to 300.
    please help..thanks in advance

    thanks a lot for your help..
    there are total 162 records in the v$open_cursor for that user.
    among that the open cursor is only 4 and open-recursive cursor is only 6 for that user.
    in my session i am executing only the renaming of the constraints and nothing more.
    When i have queried the v$open_cursor for cursor_type='OPEN' for all the users, then i got that it has exceeded the value of open_cursors parameter.
    So how can i resolve that....

  • How is index data managed within the cluster?

    Hello
    We are in the process of evaluating Coherence.
    With regard to indexes, am I correct in thinking that the data for the index is stored in an (internally managed) cache on the same node as the attendant indexed values? I'm not interested in getting at the index data, but rather just understanding how the index data itself is treated/stored within the cluster.
    So in a 2 node cluster where an index is applied to a distributed cache that only has 2 entries, will there be an index cache on each node that has 1 {Binary,key} entry where the key points to the value that is stored on that same node? That is, I trust that the index data is spread out across the cluster in the case of a partitioned topology?
    Regards
    Peter

    user11279467 wrote:
    Hello
    We are in the process of evaluating Coherence.
    With regard to indexes, am I correct in thinking that the data for the index is stored in an (internally managed) cache on the same node as the attendant indexed values? I'm not interested in getting at the index data, but rather just understanding how the index data itself is treated/stored within the cluster.
    So in a 2 node cluster where an index is applied to a distributed cache that only has 2 entries, will there be an index cache on each node that has 1 {Binary,key} entry where the key points to the value that is stored on that same node? That is, I trust that the index data is spread out across the cluster in the case of a partitioned topology?
    Regards
    PeterHi Peter,
    up to 3.4.x indexes were supported only on partitioned topology, and yes, indexes on each node contain only data corresponding to entries which have their primary copy on that node (referred to as local entries from now on).
    The indexes are not stored in a cache but they are managed as part of the information Coherence maintains about the backing map (the map which contains local entries).
    The indexes are made of 2 parts:
    - forward index: this is a mapping from the cache key in an internal representation (backing map keys from now on) to the value extracted with the extractor used to create the index from the entry belonging to the cache key (extracted value from now on)
    - reverse index (aka. reverse map): this is a mapping from the extracted value to a set of backing map keys of entries from which the reverse index key was extracted with the extractor used to create the index
    The index can be sorted which means that the reverse index will be a SortedMap (so sorting order is the ordering of extracted values)
    Let's see an example. Assuming you have the following cache content within a particular storage-enabled node :
    A --> Object (getX=5 ,...)
    B --> Object (getA=3 ,...)
    C --> Object (getA=3 ,...)
    D --> Object (getA=4 ,...)
    The forward index in that node will contain:
    Binary(A) --> 5
    Binary(B) --> 3
    Binary(C) --> 3
    Binary(D) --> 4
    The reverse index will contain:
    3 --> { Binary(B), Binary(C) }
    4 --> { Binary(D) }
    5 --> { Binary(A) }
    Where Binary(...) means the internal (binary) representation of the ... object.
    If the index is not ordered, then the order of iteration on entries in the reverse index are not deterministic.
    If the index is ordered, then the iteration of entries in the reverse index will be as shown above. Also, you can cast the reverse index to SortedMap so you have the very useful headMap and tailMap and firstKey and lastKey methods.
    Hope this helps.
    Best regards,
    Robert

  • Using CTRL + F not working to find any text within the query result

    Hi friends,
    I am trying to use find option to find any text within the result section of a query but when I do CTRL+F  the find window appears and I can input the text that I wanted. But when I press Enter it is not giving me any result though the entered text
    is in the query result. I am using Sql Server 2012 Management studio. Any Body please help me. There is a job that I have to search certain things after running a script and now I am doing copy paste to excel and doing searching from there. 

    Prashant,
    Looks like you are trying to do in SSMS, and it is because the result set is in Grid view , if you want to use CTL + F then your result should be in Text format, try to do using CTL + T ( To get result in Text) and then use CTL + F to find any
    text.
    Thanks
    Manish
    Please click Mark as Answer if my post solved your problem and click
    Vote as Helpful if this post was useful.

  • Issue to send email (using sp_send_dbmail) with an attachement located in other server within the same network

    I have two servers. One is for databases (DB) with the mail server configured and the other is for SSIS without mail server. Both servers are in the same network. On the SSIS server the SSIS package exports data from the DB server table
    to xls file. The file is saved in SSIS server. When I created a task to send the email (using sp_send_dbmail) to attach that file, I got the error message:
    Error: Executing the query "EXECUTE [msdb].[dbo].[sp_send_dbmail]
     @profile_na..." failed with the following error: "Attachment file H:\test.xls is invalid.".
    Any help will be appreciated. Thanks.
    A Fan of SSIS, SSRS and SSAS

    Where are you running this?
    If its in the other server (DB) you need to use UNC path to the SSIS server folder where file resides
    Also the executing account should have access to the folder as well as the file. So If you're running this from a job make sure you either provider folder and file access to sql agent job service account or create a separate proxy account with the required
    permissions and configure the job to use the account
    see
    http://www.databasejournal.com/features/mssql/article.php/3789881/Proxy-Accounts-in-SQL-Server.htm
    http://www.codeproject.com/Articles/28918/SQL-Server-Agent-Proxy
    Please Mark This As Answer if it solved your issue
    Please Vote This As Helpful if it helps to solve your issue
    Visakh
    My Wiki User Page
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • Can I use a separate internal drive as a back up within the same computer?

    Hello
    I am doing a lot of cloning on my drives, and running out of space on my main external back up drive.
    Along with my main drive in my G4, I have 2 additional internal western digital dives. So I am wondering if it is safe to use one of these extra drives as a back up drive for the other western digital working drive instead of using a separate external drive.
    Please advise.
    Thanks,
    Anneita

    Hi Tom
    My external G Tech hard drive is firewire, so it's bootable. But I'm not using it in this case...just going from one internal drive to another internal drive.
    However, as a back up for this G Tech back up drive, I have a w 2T western digital that I have re-formatted as os extended journaled. I am assuming this western digital drive is not bootable since it is a usb drive, correct?
    So my question is, since the western digital is not bootable, what is the best way to back up my first G Tech back up drive (which will have 5 partitions from 5 different drives on it) to the western digital? (Another words, the G Tech would be my source drive and the western digital would be my destination drive.)
    Could I just drag and drop since the western digital drive is not bootable? What do you think is the best way to go?
    FYI, I have also have super duper.
    Thanks for your help.
    Anneita

  • My MacBook Pro will suddenly go to sleep while I am using it. Sometimes if I press any key on the keyboard it will wake back up quickly, other times I have to hold down the power button until it forces a shut down and then restart. Does anyone know why??

    Has anyone else had this problem?? How did you fix it?

    Perform a #1 SMC and #2 PRAM resets
    Step by Step to fix your Mac

  • Difference between primary eindex and secondary index?

    hi experts
    pls answer me
    difference between primary eindex and secondary index?
    rewads apply.
    thanks.
    naresh.

    hi,
    check this link.
    http://help.sap.com/saphelp_47x200/helpdata/en/cf/21eb2d446011d189700000e8322d00/frameset.htm
    A difference is made between Primary & Secondary indexes to a table. the primary index consists of the key fields of the table and a pointer to the non-keys-fields of the table. The Primary index is generated automatically when a table is created and is created in the datebase as the same times as the table. It is also possible to define further indexes to a table in the ABAP/4 dictionary, which are then referred to as Secondary indexes.
    Always it is not mandatory that an index should have all the key fields of a table. To see the index of a table
    goto SE11->specify table name->click on the indexes... button on the application toolbar.
    Based on your requirement you can you any of those index fields in the where clause of your query. Always its a better practice to use the index fields in the order specified. While selecting the records from a table it is always better to select the fields in the same order as specified in the table.

  • Regarding secondary index

    how and when we create secondary indexes and what are the advantages and disadvantages of secondary indexes?

    Hi
    Index: Technical key of a database table.
    Primary index: The primary index contains the key fields of the table and a pointer to the non-key fields of the table. The primary index is created automatically when the table is created in the database.
    Secondary index: Additional indexes could be created considering the most frequently accessed dimensions of the table.
    Structure of an Index
    An index can be used to speed up the selection of data records from a table.
    An index can be considered to be a copy of a database table reduced to certain fields. The data is stored in sorted form in this copy. This sorting permits fast access to the records of the table (for example using a binary search). Not all of the fields of the table are contained in the index. The index also contains a pointer from the index entry to the corresponding table entry to permit all the field contents to be read.
    When creating indexes, please note that:
    An index can only be used up to the last specified field in the selection! The fields which are specified in the WHERE clause for a large number of selections should be in the first position.
    Only those fields whose values significantly restrict the amount of data are meaningful in an index.
    When you change a data record of a table, you must adjust the index sorting. Tables whose contents are frequently changed therefore should not have too many indexes.
    Make sure that the indexes on a table are as disjunctive as possible.
    (That is they should contain as few fields in common as possible. If two indexes on a table have a large number of common fields, this could make it more difficult for the optimizer to choose the most selective index.)
    Accessing tables using Indexes
    The database optimizer decides which index on the table should be used by the database to access data records.
    You must distinguish between the primary index and secondary indexes of a table. The primary index contains the key fields of the table. The primary index is automatically created in the database when the table is activated. If a large table is frequently accessed such that it is not possible to apply primary index sorting, you should create secondary indexes for the table.
    The indexes on a table have a three-character index ID. '0' is reserved for the primary index. Customers can create their own indexes on SAP tables; their IDs must begin with Y or Z.
    If the index fields have key function, i.e. they already uniquely identify each record of the table, an index can be called a unique index. This ensures that there are no duplicate index fields in the database.
    When you define a secondary index in the ABAP Dictionary, you can specify whether it should be created on the database when it is activated. Some indexes only result in a gain in performance for certain database systems. You can therefore specify a list of database systems when you define an index. The index is then only created on the specified database systems when activated
    Regards

  • SELECT QUERY  BASED ON SECONDARY INDEX

    Hi all,
    CAN ANYONE TELL ME HOW TO WRITE SELECT QUERY BASED ON SECONDARY INDEX.
    IN WHAT WAY DOES IT IMPROVE PERFORMANCE.
    i KNOW WHEN CREATING SECONDARY INDEX I NEED TO GIVE AN INDEX NO -iT SHOULD BE ANY NUMBER RIGHT?
    I HAVE TO LIST ALL PRIMARY KEYS FIRST AND THEN THE FIELD FOR WHICH I AM CREATING SECONDARY INDEX RIGHT?
    LETS SAY I HAVE 2 PRIMARY KEYS AND I WANT TO CREATE SEONDARY INDEX FOR 2 FIELDS THEN
    I NEED TO CREATE A SEPERTE SECONDARY INDEX FOR EACH ONE OF THOSE FIELDS OR ONE SHOULD BE ENOUGH
    pLS LET ME KNOW IF IAM WRONG

    HI,
    If you cannot use the primary index to determine the result set because, for example, none of the primary index fields occur in the WHERE or HAVINGclauses, the system searches through the entire table (full table scan). For this case, you can create secondary indexes, which can restrict the number of table entries searched to form the result set.
    You create secondary indexes using the ABAP Dictionary. There you can create its columns and define it as UNIQUE. However, you should not create secondary indexes to cover all possible combinations of fields.
    Only create one if you select data by fields that are not contained in another index, and the performance is very poor. Furthermore, you should only create secondary indexes for database tables from which you mainly read, since indexes have to be updated each time the database table is changed. <b>As a rule, secondary indexes should not contain more than four fields</b>, <b>and you should not have more than five indexes for a single database table</b>.
    <b>What to Keep in Mind for Secondary Indexes:</b>
    http://help.sap.com/saphelp_nw04s/helpdata/en/cf/21eb2d446011d189700000e8322d00/content.htm
    http://www.sap-img.com/abap/quick-note-on-design-of-secondary-database-indexes-and-logical-databases.htm
    Regards
    Sudheer

  • Partial insert with secondary index

    I'd like to use partial inserts (DB_DBT_PARTIAL).
    However, I also need secondary indexes. Now when I insert partial data, the secondary index function is immediately called, but still I do not have enough data to calculate the secondary key. I tried to supply some app_data do the DBT, but it does not reach the secondary index function.
    Is there any way I can use partial inserts together with secondary indexes?

    When writing a partial record to the primary database, here's what we do:
    1. Look up the existing record via get()
    2. Using the existing record and the partial DBT, construct the new record
    3. Pass that newly constructed record to the secondary key generation function
    You don't need to do anything special, your callback will receive the full record, even though you passed us a partial DBT. If that's not the case, there's a bug in BDB and we'd need to see your code.
    app_data is a private field, you should not try to use it or expect it to work predictably. In this situation, we're passing a brand new DBT to the callback function, not the DBT you gave us. That's why app_data is empty.
    Thanks,
    Bogdan Coman

  • Help me with Points to be noted While creating a Secondary Index.

    Hi,
    There is a Secondary index created on a Z table which already have 9 secondary indexes. This is the 10th index that is created.
    The Index is created as follows
    MANDT             Client
    DOCNUM_REF     Document number
    The Table data is huge and it is more than 4Lakh records.
    I have done the following analyses.
    1) The fields to be used are not in any other indexes.
    2) The Data is mostly different, i.e. The document reference field is having distinct data
      select a~docnum a~werks_o a~docnum_ref a~natop
             a~m_icms a~m_ipi
             b~lgort_out b~mov_est
             b~item b~matnr
        appending table ti_requisicoes
        from zsytmm_reqnfcb as a
       inner join zsytmm_reqnfit as b
          on a~branch_o eq b~branch_o
         and a~requi    eq b~requi
         and a~mask     eq b~mask
         for all entries in ti_doc_saida_del
       where a~docnum_ref   eq ti_doc_saida_del-docnum.
    What else I need to check , so that the index improves the performance. The table is using in 300 reports.

    Ravishankar Lanjewar wrote:
    @SAP LEARNER,
    >
    > There is a Secondary index created on a Z table which already have 9 secondary
    > indexes. This is the 10th index that is created.
    >
    >
    > I will not recommand to create index any more on this table. If you want to create secondary index for on above table which you have mention. You don't create index only for hamper of performance of single program/report.
    >
    > Don't create more than 4 to 5 index on sinlge table.
    >
    > Refere the tread Link:[Why to create Secondary index ?|Re: When is a secondary index used (in select or where)]
    Sounds like another myth that is widely spread across SAP community.
    Remember, complex systems like modern DBMS do not like generalizations. I personally know standard SAP tables with 9 standard secondary indexes. In the meantime the hardware became fast enough to update several secondary indexes without really significant problems. Of course indexes require space, of course one should create indexes that are not similar and differ from each other with more than one field for example. In other words, think before creating!
    But it's again incorrect to say "do not create because you already have 9 others". You did not even ask what fields are in the affected table, what are the existing indexes.
    Ravishankar Lanjewar wrote:
    > I will recommand to delete all the index and create 4 to 5 fresh index while doing analysis for all SQL statement using where use list  depends on where condition of SQL.
    I am now trying to imagine the effort to analyze 300 reports using this table and all corresponding selects. And probably you will drop supporting indexes for several reports. Some of them may be CEO/CFO-relevant reports that will run minutes/hours instead of seconds. And I would like to see the reaction of CEO when he finds out that it was the result of your "optimization" activities.

  • How does BDB update its secondary index?

    I am asking how does Berkeley DB update its secondary index?
    For example, in a BDB table with key value pair ``<k,v1>``, the index database would have reversed pair like ``<v1,k>``. Upon an insert (or essentially an update) ``<k,v2>``, base table record ``<k,v1>`` should be overwritten to ``<k,v2>``.
    But how would BDB deal with ``<v1,k>`` in the index table? Will it delete it immediately?

    Hi,
    As long as you correctly open the primary database and secondary index database, and associate the secondary index database with the primary, any update on the primary database will be reflected in the secondary index.
    For more information, please refer to Reference Guide for Berkeley DB -> Chapter 3.  Access Method Operations -> Secondary Indexes section.
    Regards,
    Cindy Zeng

  • Guidelines to create secondary index

    Hi Guys,
    I'm trying to get a guidelines to create an optimize secondary index. For eg, how to analyze the list of fields should be included into the secondary index. What can be done and not.
    Pls place your comment.
    Thanks in advance.

    Hi,
    Basically index are provided to improver performance. i.e with index select on database tables retrieves data much faster.
    If you are writting select statement on the table where in where clause you have non-primary keys then it takes longer to retriev data from Database table.
    If you have select in dbtable on non-primary key you can create Secondary index with keys as per your where clause which result in faster Database table access.
    But remember not to create too many secondary indexes on same table which could result in slow / degrage performance..
    and also check this link :
    http://help.sap.com/saphelp_470/helpdata/en/cf/21eb20446011d189700000e8322d00/frameset.htm
    Regards
    Appana

  • When is a secondary index used (in select or where)

    HI All,
    We are confused (and new to ABAP) on the various postings on SDN concerning this topic.  Replies seem to vary.
    We have the following select statement:
    SELECT date1 FROM table2
    WHERE material (from a stored internal table) = material-table2
    AND order (from a stored internal table) = order-table2
    AND delivery method (from a stored internal table) = specific value of table2.
    For a second index to be created, would we create the index for only date1 (since it's the only field we are selecting from table 2) or an index with date1, material, order, delivery method or just what's in the WHERE clause of material, order and delivery method?
    How can you be sure that an index will be used?
    Thank you!

    > We are confused (and new to ABAP)
    seem you are very very new.
    Your question is actually not ABAP indexes etc are SQL and can be found in many textbooks also on Wiki pages. ABAP uses SQL, but does not change much.
    The use index is determined nearly only by the WHERE clause. You want to know the phone number of somebody and you know the name and the address. The Index uses the name and the address as a telephone book. The phone number comes from the table not the index! This is not the secondary index, but the primary index.
    If you search in the internet for phone number, then you can use also a secondary index, i.e. you know only the complete address but not the name. A phone book would not help here.
    Which index is used, if often very simple as in the example, but can become very difficult. That is done by a sophisticated program of the database, the optimizer. Check other resoucres for details.
    Siegfried

Maybe you are looking for