Using a byte[] as a secondary index's key within the Collection's API
I am using JE 4.1.7 and its Collections API. Overall I am very satisfied with the ease of using JE within our applications. (I need to know more about maintenance, however!) My problem is that I wanted a secondary index with a byte[] key. The key contains the 16 bytes of an MD5 hash. However, while the code compiles without error when it runs JE tell me
Exception in thread "main" java.lang.IllegalArgumentException: ONE_TO_ONE and MANY_TO_ONE keys must not have an array or Collection type: example.MyRecord.hash
See test code below. I read the docs again and found that the only "complex" formats that are acceptable are String and BigInteger. For now I am using String instead of byte[] but I would much rather use the smaller byte[]. Is it possible to trick JE into using the byte[]? (Which we know it is using internally.)
-- Andrew
package example;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.PrimaryIndex;
import com.sleepycat.persist.SecondaryIndex;
import com.sleepycat.persist.StoreConfig;
import com.sleepycat.persist.model.Entity;
import com.sleepycat.persist.model.PrimaryKey;
import com.sleepycat.persist.model.Relationship;
import com.sleepycat.persist.model.SecondaryKey;
import java.io.File;
@Entity
public class MyRecord {
@PrimaryKey
private long id;
@SecondaryKey(relate = Relationship.ONE_TO_ONE, name = "byHash")
private byte[] hash;
public static MyRecord create(long id, byte[] hash) {
MyRecord r = new MyRecord();
r.id = id;
r.hash = hash;
return r;
public long getId() {
return id;
public byte[] getHash() {
return hash;
public static void main( String[] args ) throws Exception {
File directory = new File( args[0] );
EnvironmentConfig environmentConfig = new EnvironmentConfig();
environmentConfig.setTransactional(false);
environmentConfig.setAllowCreate(true);
environmentConfig.setReadOnly(false);
StoreConfig storeConfig = new StoreConfig();
storeConfig.setTransactional(false);
storeConfig.setAllowCreate(true);
storeConfig.setReadOnly(false);
Environment environment = new Environment(directory, environmentConfig);
EntityStore myRecordEntityStore = new EntityStore(environment, "my-record", storeConfig);
PrimaryIndex<Long, MyRecord> idToMyRecordIndex = myRecordEntityStore.getPrimaryIndex(Long.class, MyRecord.class);
SecondaryIndex<byte[], Long, MyRecord> hashToMyRecordIndex = myRecordEntityStore.getSecondaryIndex(idToMyRecordIndex, byte[].class, "byHash");
// END
We have highly variable length data that we wish to use as keys. To avoid massive index sizes and slow key lookup we are using MD5 hashes (or something more collision resistant should we need it). (Note that I am making assumptions about key size and its relation to index size that may well inaccurate.)Thanks for explaining, that makes sense.
It would be the whole field. (I did consider using my own key data design using the @Persistent and @KeyField annotations to place the MD5 hash into two longs. I abandoned that effort because I assumed (again) that lookup with a custom key design would slower than the built-in String key implementation.)A composite key class with several long or int fields will not be slower than a single String field, and will probably result in a smaller key since the UTF-8 encoding is avoided. Since the byte array is fixed size (I didn't realize that earlier), this is the best approach.
--mark
Similar Messages
-
Error while renaming indexes and keys in the tables
hi gems...good afternoon..
i am using oracle 11gr2 database.
when i am trying to rename the primary key, foreign key, indexes of all the tables as per the standard, then the following error is given...
ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-01000: maximum open cursors exceeded
currently the maximum open cursors parameter is set to 300.
please help..thanks in advancethanks a lot for your help..
there are total 162 records in the v$open_cursor for that user.
among that the open cursor is only 4 and open-recursive cursor is only 6 for that user.
in my session i am executing only the renaming of the constraints and nothing more.
When i have queried the v$open_cursor for cursor_type='OPEN' for all the users, then i got that it has exceeded the value of open_cursors parameter.
So how can i resolve that.... -
How is index data managed within the cluster?
Hello
We are in the process of evaluating Coherence.
With regard to indexes, am I correct in thinking that the data for the index is stored in an (internally managed) cache on the same node as the attendant indexed values? I'm not interested in getting at the index data, but rather just understanding how the index data itself is treated/stored within the cluster.
So in a 2 node cluster where an index is applied to a distributed cache that only has 2 entries, will there be an index cache on each node that has 1 {Binary,key} entry where the key points to the value that is stored on that same node? That is, I trust that the index data is spread out across the cluster in the case of a partitioned topology?
Regards
Peteruser11279467 wrote:
Hello
We are in the process of evaluating Coherence.
With regard to indexes, am I correct in thinking that the data for the index is stored in an (internally managed) cache on the same node as the attendant indexed values? I'm not interested in getting at the index data, but rather just understanding how the index data itself is treated/stored within the cluster.
So in a 2 node cluster where an index is applied to a distributed cache that only has 2 entries, will there be an index cache on each node that has 1 {Binary,key} entry where the key points to the value that is stored on that same node? That is, I trust that the index data is spread out across the cluster in the case of a partitioned topology?
Regards
PeterHi Peter,
up to 3.4.x indexes were supported only on partitioned topology, and yes, indexes on each node contain only data corresponding to entries which have their primary copy on that node (referred to as local entries from now on).
The indexes are not stored in a cache but they are managed as part of the information Coherence maintains about the backing map (the map which contains local entries).
The indexes are made of 2 parts:
- forward index: this is a mapping from the cache key in an internal representation (backing map keys from now on) to the value extracted with the extractor used to create the index from the entry belonging to the cache key (extracted value from now on)
- reverse index (aka. reverse map): this is a mapping from the extracted value to a set of backing map keys of entries from which the reverse index key was extracted with the extractor used to create the index
The index can be sorted which means that the reverse index will be a SortedMap (so sorting order is the ordering of extracted values)
Let's see an example. Assuming you have the following cache content within a particular storage-enabled node :
A --> Object (getX=5 ,...)
B --> Object (getA=3 ,...)
C --> Object (getA=3 ,...)
D --> Object (getA=4 ,...)
The forward index in that node will contain:
Binary(A) --> 5
Binary(B) --> 3
Binary(C) --> 3
Binary(D) --> 4
The reverse index will contain:
3 --> { Binary(B), Binary(C) }
4 --> { Binary(D) }
5 --> { Binary(A) }
Where Binary(...) means the internal (binary) representation of the ... object.
If the index is not ordered, then the order of iteration on entries in the reverse index are not deterministic.
If the index is ordered, then the iteration of entries in the reverse index will be as shown above. Also, you can cast the reverse index to SortedMap so you have the very useful headMap and tailMap and firstKey and lastKey methods.
Hope this helps.
Best regards,
Robert -
Using CTRL + F not working to find any text within the query result
Hi friends,
I am trying to use find option to find any text within the result section of a query but when I do CTRL+F the find window appears and I can input the text that I wanted. But when I press Enter it is not giving me any result though the entered text
is in the query result. I am using Sql Server 2012 Management studio. Any Body please help me. There is a job that I have to search certain things after running a script and now I am doing copy paste to excel and doing searching from there.Prashant,
Looks like you are trying to do in SSMS, and it is because the result set is in Grid view , if you want to use CTL + F then your result should be in Text format, try to do using CTL + T ( To get result in Text) and then use CTL + F to find any
text.
Thanks
Manish
Please click Mark as Answer if my post solved your problem and click
Vote as Helpful if this post was useful. -
I have two servers. One is for databases (DB) with the mail server configured and the other is for SSIS without mail server. Both servers are in the same network. On the SSIS server the SSIS package exports data from the DB server table
to xls file. The file is saved in SSIS server. When I created a task to send the email (using sp_send_dbmail) to attach that file, I got the error message:
Error: Executing the query "EXECUTE [msdb].[dbo].[sp_send_dbmail]
@profile_na..." failed with the following error: "Attachment file H:\test.xls is invalid.".
Any help will be appreciated. Thanks.
A Fan of SSIS, SSRS and SSASWhere are you running this?
If its in the other server (DB) you need to use UNC path to the SSIS server folder where file resides
Also the executing account should have access to the folder as well as the file. So If you're running this from a job make sure you either provider folder and file access to sql agent job service account or create a separate proxy account with the required
permissions and configure the job to use the account
see
http://www.databasejournal.com/features/mssql/article.php/3789881/Proxy-Accounts-in-SQL-Server.htm
http://www.codeproject.com/Articles/28918/SQL-Server-Agent-Proxy
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page -
Can I use a separate internal drive as a back up within the same computer?
Hello
I am doing a lot of cloning on my drives, and running out of space on my main external back up drive.
Along with my main drive in my G4, I have 2 additional internal western digital dives. So I am wondering if it is safe to use one of these extra drives as a back up drive for the other western digital working drive instead of using a separate external drive.
Please advise.
Thanks,
AnneitaHi Tom
My external G Tech hard drive is firewire, so it's bootable. But I'm not using it in this case...just going from one internal drive to another internal drive.
However, as a back up for this G Tech back up drive, I have a w 2T western digital that I have re-formatted as os extended journaled. I am assuming this western digital drive is not bootable since it is a usb drive, correct?
So my question is, since the western digital is not bootable, what is the best way to back up my first G Tech back up drive (which will have 5 partitions from 5 different drives on it) to the western digital? (Another words, the G Tech would be my source drive and the western digital would be my destination drive.)
Could I just drag and drop since the western digital drive is not bootable? What do you think is the best way to go?
FYI, I have also have super duper.
Thanks for your help.
Anneita -
Has anyone else had this problem?? How did you fix it?
Perform a #1 SMC and #2 PRAM resets
Step by Step to fix your Mac -
Difference between primary eindex and secondary index?
hi experts
pls answer me
difference between primary eindex and secondary index?
rewads apply.
thanks.
naresh.hi,
check this link.
http://help.sap.com/saphelp_47x200/helpdata/en/cf/21eb2d446011d189700000e8322d00/frameset.htm
A difference is made between Primary & Secondary indexes to a table. the primary index consists of the key fields of the table and a pointer to the non-keys-fields of the table. The Primary index is generated automatically when a table is created and is created in the datebase as the same times as the table. It is also possible to define further indexes to a table in the ABAP/4 dictionary, which are then referred to as Secondary indexes.
Always it is not mandatory that an index should have all the key fields of a table. To see the index of a table
goto SE11->specify table name->click on the indexes... button on the application toolbar.
Based on your requirement you can you any of those index fields in the where clause of your query. Always its a better practice to use the index fields in the order specified. While selecting the records from a table it is always better to select the fields in the same order as specified in the table. -
how and when we create secondary indexes and what are the advantages and disadvantages of secondary indexes?
Hi
Index: Technical key of a database table.
Primary index: The primary index contains the key fields of the table and a pointer to the non-key fields of the table. The primary index is created automatically when the table is created in the database.
Secondary index: Additional indexes could be created considering the most frequently accessed dimensions of the table.
Structure of an Index
An index can be used to speed up the selection of data records from a table.
An index can be considered to be a copy of a database table reduced to certain fields. The data is stored in sorted form in this copy. This sorting permits fast access to the records of the table (for example using a binary search). Not all of the fields of the table are contained in the index. The index also contains a pointer from the index entry to the corresponding table entry to permit all the field contents to be read.
When creating indexes, please note that:
An index can only be used up to the last specified field in the selection! The fields which are specified in the WHERE clause for a large number of selections should be in the first position.
Only those fields whose values significantly restrict the amount of data are meaningful in an index.
When you change a data record of a table, you must adjust the index sorting. Tables whose contents are frequently changed therefore should not have too many indexes.
Make sure that the indexes on a table are as disjunctive as possible.
(That is they should contain as few fields in common as possible. If two indexes on a table have a large number of common fields, this could make it more difficult for the optimizer to choose the most selective index.)
Accessing tables using Indexes
The database optimizer decides which index on the table should be used by the database to access data records.
You must distinguish between the primary index and secondary indexes of a table. The primary index contains the key fields of the table. The primary index is automatically created in the database when the table is activated. If a large table is frequently accessed such that it is not possible to apply primary index sorting, you should create secondary indexes for the table.
The indexes on a table have a three-character index ID. '0' is reserved for the primary index. Customers can create their own indexes on SAP tables; their IDs must begin with Y or Z.
If the index fields have key function, i.e. they already uniquely identify each record of the table, an index can be called a unique index. This ensures that there are no duplicate index fields in the database.
When you define a secondary index in the ABAP Dictionary, you can specify whether it should be created on the database when it is activated. Some indexes only result in a gain in performance for certain database systems. You can therefore specify a list of database systems when you define an index. The index is then only created on the specified database systems when activated
Regards -
SELECT QUERY BASED ON SECONDARY INDEX
Hi all,
CAN ANYONE TELL ME HOW TO WRITE SELECT QUERY BASED ON SECONDARY INDEX.
IN WHAT WAY DOES IT IMPROVE PERFORMANCE.
i KNOW WHEN CREATING SECONDARY INDEX I NEED TO GIVE AN INDEX NO -iT SHOULD BE ANY NUMBER RIGHT?
I HAVE TO LIST ALL PRIMARY KEYS FIRST AND THEN THE FIELD FOR WHICH I AM CREATING SECONDARY INDEX RIGHT?
LETS SAY I HAVE 2 PRIMARY KEYS AND I WANT TO CREATE SEONDARY INDEX FOR 2 FIELDS THEN
I NEED TO CREATE A SEPERTE SECONDARY INDEX FOR EACH ONE OF THOSE FIELDS OR ONE SHOULD BE ENOUGH
pLS LET ME KNOW IF IAM WRONGHI,
If you cannot use the primary index to determine the result set because, for example, none of the primary index fields occur in the WHERE or HAVINGclauses, the system searches through the entire table (full table scan). For this case, you can create secondary indexes, which can restrict the number of table entries searched to form the result set.
You create secondary indexes using the ABAP Dictionary. There you can create its columns and define it as UNIQUE. However, you should not create secondary indexes to cover all possible combinations of fields.
Only create one if you select data by fields that are not contained in another index, and the performance is very poor. Furthermore, you should only create secondary indexes for database tables from which you mainly read, since indexes have to be updated each time the database table is changed. <b>As a rule, secondary indexes should not contain more than four fields</b>, <b>and you should not have more than five indexes for a single database table</b>.
<b>What to Keep in Mind for Secondary Indexes:</b>
http://help.sap.com/saphelp_nw04s/helpdata/en/cf/21eb2d446011d189700000e8322d00/content.htm
http://www.sap-img.com/abap/quick-note-on-design-of-secondary-database-indexes-and-logical-databases.htm
Regards
Sudheer -
Partial insert with secondary index
I'd like to use partial inserts (DB_DBT_PARTIAL).
However, I also need secondary indexes. Now when I insert partial data, the secondary index function is immediately called, but still I do not have enough data to calculate the secondary key. I tried to supply some app_data do the DBT, but it does not reach the secondary index function.
Is there any way I can use partial inserts together with secondary indexes?When writing a partial record to the primary database, here's what we do:
1. Look up the existing record via get()
2. Using the existing record and the partial DBT, construct the new record
3. Pass that newly constructed record to the secondary key generation function
You don't need to do anything special, your callback will receive the full record, even though you passed us a partial DBT. If that's not the case, there's a bug in BDB and we'd need to see your code.
app_data is a private field, you should not try to use it or expect it to work predictably. In this situation, we're passing a brand new DBT to the callback function, not the DBT you gave us. That's why app_data is empty.
Thanks,
Bogdan Coman -
Help me with Points to be noted While creating a Secondary Index.
Hi,
There is a Secondary index created on a Z table which already have 9 secondary indexes. This is the 10th index that is created.
The Index is created as follows
MANDT Client
DOCNUM_REF Document number
The Table data is huge and it is more than 4Lakh records.
I have done the following analyses.
1) The fields to be used are not in any other indexes.
2) The Data is mostly different, i.e. The document reference field is having distinct data
select a~docnum a~werks_o a~docnum_ref a~natop
a~m_icms a~m_ipi
b~lgort_out b~mov_est
b~item b~matnr
appending table ti_requisicoes
from zsytmm_reqnfcb as a
inner join zsytmm_reqnfit as b
on a~branch_o eq b~branch_o
and a~requi eq b~requi
and a~mask eq b~mask
for all entries in ti_doc_saida_del
where a~docnum_ref eq ti_doc_saida_del-docnum.
What else I need to check , so that the index improves the performance. The table is using in 300 reports.Ravishankar Lanjewar wrote:
@SAP LEARNER,
>
> There is a Secondary index created on a Z table which already have 9 secondary
> indexes. This is the 10th index that is created.
>
>
> I will not recommand to create index any more on this table. If you want to create secondary index for on above table which you have mention. You don't create index only for hamper of performance of single program/report.
>
> Don't create more than 4 to 5 index on sinlge table.
>
> Refere the tread Link:[Why to create Secondary index ?|Re: When is a secondary index used (in select or where)]
Sounds like another myth that is widely spread across SAP community.
Remember, complex systems like modern DBMS do not like generalizations. I personally know standard SAP tables with 9 standard secondary indexes. In the meantime the hardware became fast enough to update several secondary indexes without really significant problems. Of course indexes require space, of course one should create indexes that are not similar and differ from each other with more than one field for example. In other words, think before creating!
But it's again incorrect to say "do not create because you already have 9 others". You did not even ask what fields are in the affected table, what are the existing indexes.
Ravishankar Lanjewar wrote:
> I will recommand to delete all the index and create 4 to 5 fresh index while doing analysis for all SQL statement using where use list depends on where condition of SQL.
I am now trying to imagine the effort to analyze 300 reports using this table and all corresponding selects. And probably you will drop supporting indexes for several reports. Some of them may be CEO/CFO-relevant reports that will run minutes/hours instead of seconds. And I would like to see the reaction of CEO when he finds out that it was the result of your "optimization" activities. -
How does BDB update its secondary index?
I am asking how does Berkeley DB update its secondary index?
For example, in a BDB table with key value pair ``<k,v1>``, the index database would have reversed pair like ``<v1,k>``. Upon an insert (or essentially an update) ``<k,v2>``, base table record ``<k,v1>`` should be overwritten to ``<k,v2>``.
But how would BDB deal with ``<v1,k>`` in the index table? Will it delete it immediately?Hi,
As long as you correctly open the primary database and secondary index database, and associate the secondary index database with the primary, any update on the primary database will be reflected in the secondary index.
For more information, please refer to Reference Guide for Berkeley DB -> Chapter 3. Access Method Operations -> Secondary Indexes section.
Regards,
Cindy Zeng -
Guidelines to create secondary index
Hi Guys,
I'm trying to get a guidelines to create an optimize secondary index. For eg, how to analyze the list of fields should be included into the secondary index. What can be done and not.
Pls place your comment.
Thanks in advance.Hi,
Basically index are provided to improver performance. i.e with index select on database tables retrieves data much faster.
If you are writting select statement on the table where in where clause you have non-primary keys then it takes longer to retriev data from Database table.
If you have select in dbtable on non-primary key you can create Secondary index with keys as per your where clause which result in faster Database table access.
But remember not to create too many secondary indexes on same table which could result in slow / degrage performance..
and also check this link :
http://help.sap.com/saphelp_470/helpdata/en/cf/21eb20446011d189700000e8322d00/frameset.htm
Regards
Appana -
When is a secondary index used (in select or where)
HI All,
We are confused (and new to ABAP) on the various postings on SDN concerning this topic. Replies seem to vary.
We have the following select statement:
SELECT date1 FROM table2
WHERE material (from a stored internal table) = material-table2
AND order (from a stored internal table) = order-table2
AND delivery method (from a stored internal table) = specific value of table2.
For a second index to be created, would we create the index for only date1 (since it's the only field we are selecting from table 2) or an index with date1, material, order, delivery method or just what's in the WHERE clause of material, order and delivery method?
How can you be sure that an index will be used?
Thank you!> We are confused (and new to ABAP)
seem you are very very new.
Your question is actually not ABAP indexes etc are SQL and can be found in many textbooks also on Wiki pages. ABAP uses SQL, but does not change much.
The use index is determined nearly only by the WHERE clause. You want to know the phone number of somebody and you know the name and the address. The Index uses the name and the address as a telephone book. The phone number comes from the table not the index! This is not the secondary index, but the primary index.
If you search in the internet for phone number, then you can use also a secondary index, i.e. you know only the complete address but not the name. A phone book would not help here.
Which index is used, if often very simple as in the example, but can become very difficult. That is done by a sophisticated program of the database, the optimizer. Check other resoucres for details.
Siegfried
Maybe you are looking for
-
Anyway to find out if/when cabinet will go fiber?
Hi All, It's been awhile now I moved into my new build development couple of years ago, all houses/flats are now fully occupied for the past year, so the business justification is there to upgrade the cabinet, is there anyway to find out when it wil
-
Firefox opens when downloading PDF file while running IE9
Having problems displaying a PDF file (very low resolution & bad contrast). So I tried IE9. When I click on the link, Firefox opens to display the PDF "Swett/Sweet", even when Firefox is not running. Page with the file "Swett/Sweet" is pg 34 at this
-
Quad core Pro in a confined space - running hot. Problem?
I moved into a new home with a nice desk already there. It's the kind with a specific cubby built in for a CPU - which my quad-core fits into nicely. I've noticed a lot of heat coming off of it since moving it in there though. Should I be worried abo
-
I have the "Hanvon 5040 drawing tablet",it was working perfectly with mac till last month.As soon as i plug the USB of my tablet into my macbook pro the computer is blocked and an error apears: "you need to restart your computer..."nothing works.Any
-
I could use some help w/ Flash Buttons please...
I am creating my site in Flash... All is well. Buttons work fine. Then all the sudden buttons do not work when created inside a movie clip. If I choose to enable simple buttons, they work fine when I am within the clip but when I go back to Scene the