Berkeley DB JE and Sharding
Hello.
Does BDB JE support sharding? I wouldn't like that all replicas have the same data in a HA, but just part of the records to improve read operations.
Thanks.
Hi,
No, JE does not support sharding. Replication is intended for failover as well as load balancing, and of course failover requires that the complete data set is replicated. Oracle NoSQL Database, which is built on top of JE, does support sharding. See:
http://www.oracle.com/technetwork/products/nosqldb/overview/index.html
--mark
Similar Messages
-
About Berkeley DB Structure And Header Segment
Hi,
I want to know about the structure of BDB.
for example: The header segment of the DB file ,etc...
because i have a db file , i confirm it is using BDB,but it's header was rewritted, the DATA segment not changed,
so it cannot be opened normally by BDB,
so i want to Consider this file's header ,how can i change it to correct header format that can be opened by BDB.
This file's structure is same to BDB' File structure except header segment.
I put the file's HEX string bellow , I hope someone can help me to format the file header correctly,
000000000200000001000000ce26004c00000000000000009026004c0000000002000000000000000300000003000000020000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000401f00004c1d0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000004c0079006e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007e007f0000000000000000000000010000000000000000000000000001003000160016000000080000004c0079006e006e0004040000040000000000000000000000000000000000000000000000f80f000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000
FileSize: 12,288 bytes
And the Top 100 of the byte array are bellow:
[0]: 0
[1]: 0
[2]: 0
[3]: 0
[4]: 2
[5]: 0
[6]: 0
[7]: 0
[8]: 1
[9]: 0
[10]: 0
[11]: 0
[12]: 206
[13]: 38
[14]: 0
[15]: 76
[16]: 0
[17]: 0
[18]: 0
[19]: 0
[20]: 0
[21]: 0
[22]: 0
[23]: 0
[24]: 144
[25]: 38
[26]: 0
[27]: 76
[28]: 0
[29]: 0
[30]: 0
[31]: 0
[32]: 2
[33]: 0
[34]: 0
[35]: 0
[36]: 0
[37]: 0
[38]: 0
[39]: 0
[40]: 3
[41]: 0
[42]: 0
[43]: 0
[44]: 3
[45]: 0
[46]: 0
[47]: 0
[48]: 2
[49]: 0
[50]: 0
[51]: 0
[52]: 1
[53]: 0
[54]: 0
[55]: 0
[56]: 0
[57]: 0
[58]: 0
[59]: 0
[60]: 0
[61]: 0
[62]: 0
[63]: 0
[64]: 0
[65]: 0
[66]: 0
[67]: 0
[68]: 0
[69]: 0
[70]: 0
[71]: 0
[72]: 0
[73]: 0
[74]: 0
[75]: 0
[76]: 0
[77]: 0
[78]: 0
[79]: 0
[80]: 0
[81]: 0
[82]: 0
[83]: 0
[84]: 0
[85]: 0
[86]: 0
[87]: 0
[88]: 0
[89]: 0
[90]: 0
[91]: 0
[92]: 0
[93]: 0
[94]: 0
[95]: 0
[96]: 0
[97]: 0
[98]: 0
[99]: 0
this file contains 2 records,values are: Lynn, Lynn2
I'm sorry for this so long content,
I hope someone can help me to change the header to a correct format that can using BDB to open it.
Thanks and waiting for your reply.
Very thanks,
Yours Lynn.Hi Lynn,
Can you use the db_dump utility with the -R flag to retrieve the data?
http://www.oracle.com/technology/documentation/berkeley-db/db/api_reference/C/db_dump.html
The header format of Berkeley DB is complicated. If you really need to create a new, compatible header. The best I can suggest is creating and empty database with the same setup as the corrupted file and copying the header information over to the corrupted file. That is NOT a supported or tested way to recover data from a corrupted database, and may or may not work.
Regards,
Alex Gorrod,
Oracle Berkeley DB -
Berkeley DB, SQLite and virtual tables
Is it possible to compile support for the RTree module and the fts3 module with this version?
Thanks
SebastianOk Sorry for disturbing the peace.
I compiled the distribution with -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS
And it appears to work just fine.
I think this should be compiled into the binary distribution since both are very important features
Thanks again
Sebastian -
A tale of a Berkeley DB project: Success and future developements
The site http://www.electre.com is an on-line catalog of all published French books (close to 1 million of them), from the 1970’s to today, with pricing and availability information for professional users.
The entire book database and search engine was developed with Berkeley DB. The site runs relatively problem free, and owes much to the stability and quality of the Berkeley DB source code. Moreover, the software is sold as an intranet version and runs on remote configurations that are out of our direct control. And yet, we don’t get much problem reports from these remote sites.
Development started in 2002. Using Berkeley DB was my decision. It was, with hindsight, a very good decision, but it would be hypocritical to affirm I knew all along it would work out so well. It was partly a gamble, and partly a calculated risk. I am not ashamed to confess that when I finally made the decision to go with Berkeley DB, there were some key areas in the technical design I had no clue how to tackle. Needless to say, any flaws that remain in this area are due to lack of knowledge and lack of foresight on my part, not because of some flaw in Berkeley DB.
What convinced me 4 years ago to “go for it” was Sleepycat’s no-nonsense approach to support from guys like Michael Cahill, Keith Bostic, John Merrells, Dave Seglau, Michael Ubell, Liz Pennel and others. Support was sending a message to [email protected]. Replies were never more than a couple of days away (including time zone differences). There was always something comforting when you read a reply from someone who says he “owns your problem” (especially when you also saw that someone’s name in the source code <g>). The majority of support questions were asked during evaluation and product development, before even a single dime was paid to Sleepycat. (Now that we sell the product as a closed source Intranet solution, we gladly pay the required license fees, of course). The fact that the project development went so well was in no small part thanks to the competence and reliability of the people I communicated with. I never expressed my thanks, so now is as good a time as any to do so. Thanks, guys (and gals).
When Sleepycat was bought by Oracle, it was rumored that the goal was ultimately “to kill the product”. I thought that was ridiculous and an exaggeration by some paranoid people (especially from the MySQL-camp). I understand that Oracle is a big company, and that things are done differently now. Besides, how do you “kill” a product? I didn’t know until I hit my first support question “post-Sleepycat”. So now I do: you kill a product not directly, but, but by a process of what I call “incremental discouragement”. It’s a subtle, 3-pronged approach:
1) Erect bureaucratic barriers between competent people and your customers. No more [email protected], but a slow and unwieldy web site and a support procedure hiding behind numbers and requiring a 2 hour training session. I’m not sure how I’m going to motivate to management a 2 hour training session for a product we’ve been using for the past 4 years, and which I now master sufficiently to reduce my support requests to about 3 a year, and for which the only difference is a change of company name. There’s a “free” forum, but the primary motivation for it seems to be to allow users to support themselves. Yes, questions are answered on the forum… but compared to the level of support I’ve been accustomed to from the same people in the past, I can’t help noticing something has changed… and not for the better. Some questions aren’t even acknowledged, though they are read over 300 times.
2) Add useless levels of indirection: when we sell an intranet, we asked Sleepycat to mail us an invoice, which we got within the week. Now we ask Oracle the same thing: the last time we did this was on November 27 of last year, and we still waiting for someone from “the Belgian office” to contact us. The only reply we got was from someone telling us our request would be passed on: a level of indirection not exactly adding value.
3) Increase useless Information Noise: www.sleepycat.com was all about Berkeley DB and related products. Look at any page on the www.oracle.com site supposedly about Berkely DB and count the number of items and links pointing to products that have nothing to do with Berkeley DB. It’s like that famous analogy: you’re only interested in a banana, and you must take the whole gorilla.
Berkeley DB is the fire behind the electre.com site. Our customers are satisfied, so this is not going to change for this version. But this developer is now convinced to view Berkeley DB as a medium to long-term liability. The fire will be kept alive for the remainder of this electre.com version, but for future versions and other similar projects, other solutions will have to be found. These other solutions will involve a similar gamble as the one I made a few years ago with Berkeley DB. My only hope is to meet similar competence, friendliness and professionalism, but without the overhead and bureaucracy of an organization for which the product is not even part of the core business.Vincent,
I hear the frustration and concern in your post. I'd like to assure you that we're still here doing what we've done in the past and that your business is valuable to us, don't give up yet! :)
Moving from a company of 30 people to a company of >50,000 people has been an interesting transition for us as well. I have to say that honestly, things are pretty good here inside the realm of Oracle. Of course, some things were bound to change. Oracle has processes in place to acquire and grow companies and that is their intended goal with Sleepycat's Berkeley DB products. As such, some customer facing processes have changed and our home on the internet has been incorporated into the oracle.com site. The interesting thing to me is that even within Oracle's infrastructure you, the customer, are not really much further away from our engineers and support staff than before. Your suggestions, concerns, questions and bugs go directly to us via the same people pre-acquisition for the most part, although that staff is growing. With these OTN forums you have a way to speak directly to most of the Sleepycat staff, 99% of whom are still here at Oracle nearly one year post acquisition. So, I'd argue that we've done a great job of keeping that small company feeling in one of the largest software providers around.
Our web content is 80% identical to that which was on the sleepycat.com sites. Bookmarking one or two locations within Oracle and OTN will get you straight to that information. Sure, there are other product references floating around on the same page and we hope that over time Berkeley DB products and other Oracle products complement each other when used in combination. In general, I believe that most of the web experience is identical with different style sheets and a few extra links.
Support for eval customers continues to be something we provide free of charge via these forums and in private email conversations with our staff. This is unchanged. Ownership of customer issues is much the same as well and we do still use '[email protected]' for tracking issues once opened. We agree that '[email protected]' method of communication was simple and highly effective for us and our customers. We are working with Oracle's support infrastructure to consider adopting similar methods. This is another reason Oracle purchased Sleepycat, to learn from our efficient effective operational model. This is something that is ongoing, in the mean time we still track issues behind the scenes the same way as before with the added information provided by TARs so that we fit into the overall Oracle support infrastructure.
The rumors of our early demise are highly exaggerated. Berkeley DB products are alive and well in Oracle. Just look at the releases we've made in the past 12 months.
As for our three pronged attack. ;-)
1. The distance between first contact with support and helpful information has, in some ways, increased. This is due to the Oracle infrastructure for support that manages all Oracle products. The forums are the replacement for the discussion email lists we managed at sleepycat.com. We're managing a much larger amount of traffic than in the past and most of it is done with the same Berkeley DB engineering team you've come to know and love. Maybe we need to make it more obvious when we answer a question by having a signature indicating who we are, sometimes that's not obvious. As to the particular question you posed, I don't know what happened but the same thing can happen on an email list. My apologies for the lack of a response.
2. Oracle has a huge sales force worldwide. We're only beginning to fully function within this new infrastructure. Sorry we dropped the ball on your sales inquiry, rest assured that we're interested in all commercial deals. Once connected into the Belgium office you'll have a direct relationship, as before, with a sales rep for your use of Berkeley DB.
3. I've already talked about this. Oracle has many products, we're just one of those. We have to fit within the overall site structure. For the most part I'd give us an A- or B+ for transitioning our sleepycat.com information into oracle.com and OTN.com. If you have suggestions as to how we might improve that, drop me a line I'm all ears.
In general I'd like to believe that you and other developers like you will find that our place in Oracle doesn't prevent you from choosing Berkeley DB. Certainly it is a bigger company with some additional process overhead, but hopefully not so much that it prevents you from remaining a loyal customer and someone who would recommend us to others.
regards,
-greg
Gregory Burd [email protected]
Product Manager, Berkeley DB/JE/XML Oracle Corporation -
Berkeley DB Sessions at Oracle OpenWorld Sept 19 - 23
All,
Just posting some of the Berkeley DB related sessions at Oracle OpenWorld this year. Hope to see you there.
Session ID: S317033
Title: Oracle Berkeley DB: Enabling Your Mobile Data Strategy
Abstract: Mobile data is everywhere. Deploying applications and updates, as well as collecting data from the field and synchronizing it with the Oracle Database server infrastructure, is everyone?s concern today in IT. Mobile devices, by their very nature, are easily damaged, lost, or stolen. Therefore, enabling secure, rapid mobile deployment and synchronization is critically important. By combining Oracle Berkeley DB 11g and Oracle Database Lite Mobile Server, you can easily link your mobile devices, users, applications, and data with the corporate infrastructure in a safe and reliable manner. This session will discuss several real-world use cases.
Speaker(s):
Eric Jensen, Oracle, Principal Product Manager
Greg Rekounas, Rekounas.org,
Event: JavaOne and Oracle Develop
Stream(s): ORACLE DEVELOP, DEVELOP
Track(s): Database Development
Tags: Add Berkeley DB
Session Type: Conference Session
Session Category: Case Study
Duration: 60 min.
Schedule: Wednesday, September 22, 11:30 | Hotel Nikko, Golden Gate
Session ID: S318539
Title: Effortlessly Enhance Your Mobile Applications with Oracle Berkeley DB and SQLite
Abstract: In this session, you'll learn the new SQL capabilities of Oracle Berkeley DB 11g. You'll discover how Oracle Berkeley DB is a drop-in replacement for SQLite; applications get improved performance and concurrency without sacrificing simplicity and ease of use. This hands-on lab explores seamless data synchronization for mobile applications using the Oracle Mobile Sync Server to synchronize data with the Oracle Database. Oracle Berkeley DB is an OSS embedded database that has the features, options, reliability, and flexibility that are ideal for developing lightweight commercial mobile applications. Oracle Berkeley DB supports a wide range of mobile platforms, including Android.
Speaker(s):
Dave Segleau, Oracle, Product Manager
Ashok Joshi, Oracle, Senior Director, Development
Ron Cohen, Oracle, Member of Technical Staff
Eric Jensen, Oracle, Principal Product Manager
Event: JavaOne and Oracle Develop
Stream(s): ORACLE DEVELOP, DEVELOP
Track(s): Database Development
Tags: Add 11g, Berkeley DB, Embedded Development, Embedded Technology
Session Type: Hands-on Lab
Session Category: Features
Duration: 60 min.
Schedule: Wednesday, September 22, 16:45 | Hilton San Francisco, Imperial Ballroom A
Session ID: S317032
Title: Oracle Berkeley DB: Adding Scalability, Concurrency, and Reliability to SQLite
Abstract: Oracle Berkeley DB and SQLite: two industry-leading libraries in a single package. This session will look at use cases where the Oracle Berkeley DB library's advantages bring strong enhancements to common SQLite scenarios. You'll learn how Oracle Berkeley DB?s scalability, concurrency, and reliability significantly benefit SQLite applications. The session will focus on Web services, multithreaded applications, and metadata management. It will also explore how to leverage the powerful features in SQLite to maximize the functionality of your application while reducing development costs.
Speaker(s):
Jack Kreindler, Genie DB,
Scott Post, Thomson Reuters, Architect
Dave Segleau, Oracle, Product Manager
Event: JavaOne and Oracle Develop
Stream(s): ORACLE DEVELOP, DEVELOP
Track(s): Database Development
Tags: Add Berkeley DB
Session Type: Conference Session
Session Category: Features
Duration: 60 min.
Schedule: Monday, September 20, 11:30 | Hotel Nikko, Nikko Ballroom I
Session ID: S317038
Title: Oracle Berkeley DB Java Edition: High Availability for Your Java Data
Abstract: Oracle Berkeley DB Java Edition is the most scalable, highest performance Java application data store available today. This session will focus on the latest features, including triggers and sync with Oracle Database as well as new performance and scalability enhancements for high availability, with an emphasis on real-world use cases. We'll discuss deployment, configuration, and maximized throughput scenarios. You'll learn how you can use Oracle Berkeley DB Java Edition High Availability to increase the reliability and performance of your Java application data storage.
Speaker(s):
Steve Shoaff, UnboundID Corp, CEO
Alex Feinberg, Linkedin,
Ashok Joshi, Oracle, Senior Director, Development
Event: JavaOne and Oracle Develop
Stream(s): ORACLE DEVELOP, DEVELOP
Track(s): Database Development
Tags: Add Berkeley DB
Session Type: Conference Session
Session Category: Features
Duration: 60 min.
Schedule: Thursday, September 23, 12:30 | Hotel Nikko, Mendocino I / II
Session ID: S314396
Title: Java SE for Embedded Meets Oracle Berkeley DB at the Edge
Abstract: This session covers a special case of edge-to-enterprise computing, where the edge consists of embedded devices running Java SE for Embedded in combination with Oracle Berkeley DB Java Edition, a widely used embedded database. The approach fits a larger emerging trend in which edge embedded devices are "smart"--that is, they come equipped with an embedded (in-process) database for structured persistent storage of data as needed. In addition, these devices may optionally come with a thin middleware layer that can perform certain basic data processing operations locally. The session highlights the synergies between both technologies and how they can be utilized. Topics covered include implementation and performance optimization.
Speaker(s): Carlos Lucasius, Oracle , Java Embedded Engineering
Carlos Lucasius works in the Java Embedded and Real-Time Engineering product team at Oracle Corporation, where he is involved in development, testing, and technical support. Prior to joining Sun (now Oracle), he worked as an consultant to IT departments at various companies in both North-America and Europe; specific application domains he was involved in include artificial intelligence, pattern recognition, advanced data processing, simulation, and optimization as applied to complex systems and processes such as intelligent instruments and industrial manufacturing. Carlos has presented frequently at scientific conferences, universities/colleges, and corporations across North-America and Europe. He has also published a number of papers in refereed international journals covering applied scientific research in abovementioned areas.
Event: JavaOne and Oracle Develop
Stream(s): JAVAONE
Track(s): Java for Devices, Card, and TV
Session Type: Conference Session
Session Category: Case Study
Duration: 60 min.
Schedule: Tuesday, September 21, 13:00 | Hilton San Francisco, Golden Gate 1
Session ID: S313952
Title: Developing Applications with Oracle Berkeley DB for Java and Java ME Smartphones
Abstract: Oracle Berkeley DB is a high-performance, embeddable database engine for developers of mission-critical systems. It runs directly in the application that uses it, so no separate server is required and no human administration is needed, and it provides developers with fast, reliable, local persistence with zero administration. The Java ME platform provides a new, rich user experience for cell phones comparable to the graphical user interfaces found on the iPhone, Google Android, and other next-generation cell phones. This session demonstrates how to use Oracle Berkeley DB and the Java ME platform to deliver rich database applications for today's cell phones.
Speaker(s): Hinkmond Wong, Oracle, Principal Member of Technical Staff
Hinkmond Wong is a principal engineer with the Java Micro Edition (Java ME) group at Oracle. He was the specification lead for the Java Community Process (JCP) Java Specification Requests (JSRs) 36, 46, 218, and 219, Java ME Connected Device Configuration (CDC) and Foundation Profile. He holds a B.S.E degree in Electrical Engineering from the University of Michigan (Ann Arbor) and an M.S.E degree in Computer Engineering from Santa Clara University. Hinkmond's interests include performance tuning in Java ME and porting the Java ME platform to many types of embedded devices. His recent projects include investigating ports of Java ME to mobile devices, such as Linux/ARM-based smartphones and is the tech lead of CDC and Foundation Profile libraries. He is the author of the book titled "Developing Jini Applications Using J2ME Technology".
Event: JavaOne and Oracle Develop
Stream(s): JAVAONE
Track(s): Java ME and Mobile, JavaFX and Rich User Experience
Tags: Add Application Development, Java ME, Java Mobile, JavaFX Mobile, Mobile Applications
Session Type: Conference Session
Session Category: Tips and Tricks
Duration: 60 min.
Schedule: Monday, September 20, 11:30 | Hilton San Francisco, Golden Gate 3
I think I have them all. If I have missed any, please reply and I can update the list, or just post the info in the reply.
Thanks,
Greg Rekounasare any links to access these Seminars??
-
Re-licensing Berkeley DB back to
Hi guys,
I'm not sure if this has already been discussed here or even if this is right place to request such thing. If there is a better place, just point me there, please.
My question is simple -- can upstream even think about re-licensing Berkeley DB 6.0 back to LGPLv2? It's not hard to imagine how many problems with licensing incompatibility were introduced by re-licensing Berkeley DB to AGPLv3. There are so many projects that use Berkeley DB and cannot be re-licensed as well, which basically means Berkeley DB started to die since 6.0, because such projects will need to switch to some alternative.. If this was desired by upstream to remove BDB from real life, then I can understand it. But anyway, I'd like to hear there is no way to convince upstream to re-licence back.
Regards,
HonzaI am Jesús Cea, maintainer of the Python Berkeley DB bindings.
The bindings are licensed as BSD 3-clauses, so my code is incompatible with BDB 6.0. No Python program, under any license, can use BDB 6.0, then. I can't change the license because it is inherited code and it is impossible to contact with every past author.
I plan to do a release that ONLY allow linking with BDB 6.0 IF you define a environment variable saying "yes, I have a Oracle commercial license for this". Ugly and hacky, but hopefully safe for everybody.
Personaly I use Berkeley DB HA and two phases commit A LOT in my internal projects. It is my main mode, in fact. I use it for everything, from mail storage to application deployment. -
Some php + berkeley db questions
I have some questions about using php with bdb.
First, I compiled Berkeley db and then linked php to it using the configure directive.
Then I accessed bdb through the php standard dba_* APIs. This works, but it seems like locking is broken. The php documentation (and common sense) says that calls to dba_open() with a write lock will block when another such call has succeeded in another process. But my tests show many concurrent processes all getting write locks no problem.
So then I compiled the native php_db4 extension that ships with bdb. I tried to use the API documented here:
http://www.oracle.com/technology/documentation/berkeley-db/db/ref/ext/php.html
Can anybody direct me to a more complete (and more correct) fuller documented version of this API? For instance, the put method is not shown in the Db4 class, but it does exist.
I'm trying to infer how the php API works from the C API docs, but it's not very easy, particularly when it comes to error codes returned. Is there a db_strerror in the php?
I can get the simple demos that come in the db4 php dir to work, but what I need is a locking environment, much like the one documented here:
http://www.oracle.com/technology/documentation/berkeley-db/db/ref/cam/intro.html
However, when I try to open the DBENV with the DB_INIT_CDB and DB_INIT_MPOOL flags, as directed, the call fails in the php. I cannot figure out why or how to get an errorr code or message I can debug?
Any help will be much appreciated. If you could just point me at any real-world examples of php and berkeley db that would be a great start.Hi,
From what I'm aware of, there is no extra documentation on php & BDB (maybe just php.net :) ). Also, I don't know if is there anyone who published his source code.
What kind of application do you want to build? I think that a good option for the moment is to try to use BDB XML version ( Berkeley DB XML and http://www.oracle.com/technology/products/berkeley-db/xml/index.html ), since there are many cases there in which the BDB XML product is used via PHP, and this is why you can have a better support. I think that you can try to achieve the same approach using XML, please let me know if you agree or not.
BDB XML's PHP API's are mapped over C++ API, and you'll have the ability to use XML and XQuery rather than tables and SQL.
If you can point me with a specific issue in PHP APIs for BDB, and provide me with test cases, I can try to work them out. Also, in the next weeks, I 'll try to have a look on PHP APIs in my spare time, and maybe I'll be able to work on supporting the latest BDB APIs. If there is somebody working on a PHP app and is willing to help on testing and maintaining the PHP APIs, please post here.
Regards,
Bogdan Coman, Oracle -
Berkeley DB JDBC Build on Linux/ARM Errors?
Trying to build a version of Berkeley DB 5.3.15 with dbsql and jdnc on a Linux/Arm device.
From the build_unix directory issued the following configure command:
../dist/configure enable-sql enable-jdbc
then
make
The build fails while trying to compile sqlite_jni.c. The last lines of the build look as follows:
native/sqlite_jni.c: In function 'Java_SQLite_Database__1exec__Ljava_lang_String_2LSQLite_Callback_2_3Ljava_lang_String_2':
native/sqlite_jni.c:1622: error: incompatible type for argument 2 of 'sqlite3_vmprintf'
../../lang/sql/generated/sqlite3.h:1986: note: expected 'va_list' but argument is of type 'char *'
native/sqlite_jni.c: In function 'Java_SQLite_Database_vm_1compile_1args':
native/sqlite_jni.c:3282: error: incompatible type for argument 2 of 'sqlite3_vmprintf'
../../lang/sql/generated/sqlite3.h:1986: note: expected 'va_list' but argument is of type 'char *'
make[1]: *** [sqlite_jni.lo] Error 1
make[1]: Leaving directory `/root/db-5.3.15/build_unix/jdbc'
make: *** [jdbc] Error 2
Also tried including --enable-java in the configure script with the same effect.
Any suggestions?
Thanks,
-- Jim CI have done some porting work of Berkeley db4.6.21 on vxworks6.3, and I think my experiences may help:
If you decide to use db-4.5.21, simply open the WorkBench2.5, create a "downloadable kernal module", and then import the db source tree, make sure not to import the files not needed. To see what files to be imported, you can open the db/dist/srcfiles and retrieve the lines with "vx" string, and import only the files listed in the retrieved list.
Then, you can build the project. Note that there are some data type conficts between berkeley db source and the vxworks os, that is the u_int8_t and the like, you will have to remove the definitions in the headers belonging to vxworks. -
Fault Tolerance in Berkeley DB
Please, can you help me with the following situation?
I want to load a big data source (about 20 million of records) in Berkeley DB. I programmed in C++ all the data loader (using the Berkeley DB API) and I began to run it. However, I estimate that it will need about 30 hours to finish.
The Electricity Distribution company announced a power cut of 3 hours in the next 30 minutes, and the program have not finished yet. When the power will go up again, and I re-run the program: will it be able to recover (from the .db files created through the Berkeley DB API) the data previously inserted and continue from the point it was interrupted? Or will it restart the whole process (overwriting the .db files)?
The files were created with the DB_CREATE flag.
Thanks in advance!Hi,
If you need recoverability guarantees you will need to set up your application/data loader as TDS (Transactional Data Store). This way, after the power failure, when restarting the loader, you will be able to recover the data that has not been flushed to stable storage yet.
Please have a look over the following resources:
[http://www.oracle.com/technology/documentation/berkeley-db/db/gsg_txn/C/index.html]
[http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/transapp_fail.html]
[http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/transapp_recovery.html]
[http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/transapp_reclimit.html]
Now, you should program the loader so that it knows where to resume from (for example, keeping a separate file with the last inserted record, or using a cursor to position on the last inserted record in the database and retrieve it, and continue from that record/key onwards). Otherwise, if the loader starts from the beginning it will overwrite any previously inserted records.
Regards,
Andrei -
Problem sqlite+berkeley PANIC: fatal region error detected.
Excuse for my English and if this wrong place to explain the problem
I'm checking to replace Berkeley DB and SQLite for testing the stability against involuntary interruptions of program I have encountered the following error:
Berkeley DB trust in knowing his rnedimiento and stability with Subversion, but I doubt the API bridge SQLITE
I'm testing the Berkeley DB database using the API Berkeley DB SQLITE and I did a small test program:;
/* Open database. */
sqlite3 *db;
sqlite3_open("data/basedatos.db", &db);
sqlite3_exec(db,"CREATE TABLE [test] ([key] INTEGER, [dat] varchar(64), PRIMARY KEY ([key]))",NULL,0,NULL);
err_code = SQLITE_BUSY;
while (err_code != SQLITE_OK ) {
sqlite3_exec( db, "delete from test", NULL, 0, NULL );
err_code = sqlite3_errcode( db );
sqlite3_exec( db, "BEGIN", NULL, 0, NULL );+
for( int i=0; i<_numCartones; i++ ) {
char buf[1024];
sprintf_s( buf, sizeof(buf), "insert into test( key, dat) values ( %d, 'test%d' )", i, i );
sqlite3_exec( db, buf, NULL, 0, NULL );
sqlite3_exec( db, "COMMIT", NULL, 0, NULL );
sqlite3_close(db);I launched the program and insert about 150000 records in 17 seconds. Perfect!
I created a file basedatos.db and basedatos.db-journal subdirectory with files: log.0000000016, __db.001, __db.002, __db.003, __db.004, __db.005, __db.006 and __db.register.
Open it and prove the usefulness dbsql
c: dbsql basedatos.db
select count(*) from test;
150000 ← Ok.
Without closing the program again dbsql run the test program and this will get stuck in the call:
sqlite3_exec( db, "delete from test", NULL, 0, NULL );I close dbsql and automatically releases the "delete from" and the test program again inserted 150,000 records
While this by inserting 150,000 records run it again
c: dbsql basedatos.db
select count(*) from test; [WAIT]
and select count (*) remains locked until you finish the test program, normal thing locks.Once you finish the select TEST responds to 150,000
150000 ← Ok.Without closing the program again dbsql run the test program and this will get stuck in the call:
sqlite3_exec( db, "delete from test", NULL, 0, NULL );I close dbsql and automatically releases the "delete from" and the test program again inserted 150,000 records
while inserting test rerun:
c: dbsql basedatos.db
select count(*) from test;
Error: database disk image is malformed
and in my test program : PANIC: fatal region error detected; run recovery.
Reviewing the files are only: badatos.db, log.0000000031, log.0000000032, log.0000000033, log.0000000034, log.0000000035, log.0000000036, __db.register.
and __db*.* files?Had accidentally opened the program dbsql.exe while doing speed tests data insertion.
While in a shell to make a select count (*) and I realized that was blocked waiting for the COMMIT release, normal thing in a process BEGIN / COMMIT.
In one test was corrupt database which reduced the test software and simplify the test to repeat the problem.
Today I repeated the test and the situation has changed
1) Run test (all OK, inserted in 18 seconds 150000 entries)
2) DBSQL run and I make a select count (*) (All Ok)
3) DBSQL unsealed test run and stay lock on DELETE FROM
4) DELETE FROM I close DBSQL and not released as yesterday.Repeat several times and I have the same behavior
Move in the test code from "delete from..." begin and commit
sqlite3_exec( db, "BEGIN", NULL, 0, NULL );
err_code = sqlite3_errcode( db );
err_code = SQLITE_BUSY;
while (err_code != SQLITE_OK ) {
sqlite3_exec( db, "delete from test", NULL, 0, NULL );
err_code = sqlite3_errcode( db );
for( int i=0; i<_numCartones; i++ ) {Repeat tests
1)Test run, everything ok in 25 seconds. While inserting test this, I run realizao dbsql and a select count (*) and remains lock until test ends. Everything ok, 150000 records
2)Dbsql unsealed test run it again and stay lock on delete until you close the program dbsql.
3)I close dbsql and releasing the lock of seconds to delete the test the error "PANIC ..." like yesterdayRepeat several times and the behavior is the same, except that no desparencen db files.
If I can not run dbsql test run multiple times without problems.
Could be the problem dbsql and simultaneous access to the database?
I'm going to migrate an application in production since SQLITE to Berkeley DB for testing.
I have confidence in the performance of Berkeley DB and I know the proper functioning it does with subversion, but the subversion server on a server is protected with uninterrupted power.
If I avoid using dbsql while the test software that I have that theoretical security operation will be correct when using Berkeley DB SQLITE layer and especially with unexpected off the machine?
Thanks again for your help -
Perforamance comparison of various available Berkeley DB versions
Hi All,
I am currently using Berkeley DB 4.5.20 in my project for message storage. It is handling 1500 messages/sec. I want to improve the performance to 3000 messages/sec.
I have tried to check the change logs for all the latest and available versions of Berkeley DB. I am not able to find any significant information that talks about the performance enhancement in the Berkeley DB with respect to processing and optimization of existing algorithms.
If someone is using Berkeley DB for similar kind of usage or has data that explains performance comparison between Berkeley DB 4.5.20 and all latest versions available. Please share.
Thanks
RahulHi Rahul,
We don't have performance comparisons for the various versions of Berkeley DB. As a general statement, we work on performance projects in each new release of Berkeley DB. However, it is difficult to say whether you'll get a 2X performance improvement for your workload simply by upgrading the BDB version.
Having said that, I'd definitely recommend that
a) you upgrade to the latest release at the earliest convenient opportunity. We've added many new features and enhancements since BDB 4.5.
b) you look into optimizing your existing Berkeley DB configuration and application. Quite often, we find that performance can be improved significantly as a result. We're happy to help with that, but we'll need a lot more detail about your app and set up. My suggestion is that you spend some time analyzing and tuning the application and then post your questions to the forum.
Hope this helps.
Thanks and warm regards.
ashok -
Building Berkeley DB for 32 bit Windows
Hi,
I downloaded Berkeley DB 5.1.25.msi Windows installer, with AES encryption (27M) md5 from
http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html
I followed the instructions below to build a Debug Version: (My comments in bold. I noticed that libdb51d.lib was not avail to build the Debug version)
I am using Visual Studio Express 2008
Any help would be greatly appreciated.
Thanks in advance.
Steve Lee
1. Choose File -> Open -> Project/Solution.... In the build_windows directory, select
Berkeley_DB.sln and click Open. *(I notice that the directory only contained Berkeley_DB_examples. I did not find Berkeley_DB.sln)*
2. The Visual Studio Conversion Wizard will open automatically. Click the Finish button.
3. On the next screen click the Close button.
4. Choose the desired project configuration from the drop-down menu on the tool bar
(either Debug or Release).
5. Choose the desired platform configuration from the drop-down menu on the tool bar
(usually Win32 or x64).
6. To build, right-click on the Berkeley_DB solution and select Build Solution.
Build results
The results of your build will be placed in one of the following Berkeley DB subdirectories,
depending on the configuration that you chose:
build_windows\Win32\Debug
build_windows\Win32\Release
build_windows\Win32\Debug_static *(This option was not avail)*
build_windows\Win32\Release_static *(This option was not avail)*
When building your application during development, you should normally use compile options
"Debug Multithreaded DLL" *(This option was not avail)* and link against build_windows\Debug\libdb51d.lib. *(libdb51d.lib was not avail)*
You can also build using a release version of the Berkeley DB libraries and tools, which will be placed
in build_windows\Win32\Release\libdb51.lib. When linking against the release build,
you should compile your code with the "Release Multithreaded DLL" compile option *(Release Multithreaded DLL was not avail)*. You
will also need to add the build_windows directory to the list of include directories of your
application's project, or copy the Berkeley DB include files to another location.
Edited by: 840755 on Mar 1, 2011 10:21 AM
Edited by: 840755 on Mar 1, 2011 10:21 AM
Edited by: 840755 on Mar 1, 2011 10:22 AMHi,
I am new to Berkely DB and I want to see a working example on Win 7 64 bit on a VC++ 64 bit project.
I downloaded the Berkeley DB 5.1.25.zip file and when I try to biuld the Berkeley_DB_examples_vs2010 on VC++ 2010 64 bit
I get the following error:
LINK : fatal error LNK1181: cannot open input file 'libdb51d.lib'
I can find the libdb51.lib' on my computer but not the libdb51d.lib'
1) How can I get the file libdb51d.lib?
2) As I understand, my other option to see a working example is to install the Berkely DB using a Windows installer and to try to build a project from the provided .cpp (without .sln) files examples, I tried it but then I have configuration problems with the VC++ project, Which option of the 2 options shouls I use?
Thanks! -
3rd party HA middleware for Berkeley DB
Are there HA middlewares(such like OpenClovis) that support Berkeley DB? And if any, what's pros/cons of using those HA middlewares?
Thanks
Message was edited by:
user610522
Message was edited by:
user610522
Message was edited by:
user610522I've also looked at using BerkeleyDB under the control of an AMF system, and I think that the two are compatible, but they would not be aware of each other, and it would be up to you to in your application to coordinate them.
The concepts are compatible though. An SI assigned to a 2N service group could be running 2 SUs, each of which corresponds to a BerkeleyDB environment in a 2-site replication group. The AMF would assign active/standby roles for the SI, and the application can translate that into MASTER/CLIENT calls to the repmgrStart() functions.
There are tricky details you need to work out though. You have two independent systems doing fault detection. i.e. If the BerkeleyDB repmgr framework detects a disconnect, it will conduct an election without asking for your permission to do so. What you can do with the AMF is watch for changes to the master/client role. If you are demoted from master to client, but still assigned the 'active' role to the SI by the AMF, you can report an AMF error, so that the AMF will failover the SI. This would keep the SI-active assignment in-sync with the BerkeleyDB assignment of the Master site, for a 2N service group, anyway.
As this note might imply, I have looked at doing this myself. -
Cursors throwing Unexpected lock status: 6
I am using a Berkeley DB instance and have a multithreaded clients each performing various types of operations on the database. In general there are three types of operations being performed, reads, writes (which use RMW locks), and (non-transactional) cursors. After some time my application dumps the following:
Unexpected lock status: 6
PANIC: Invalid argument
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__os_stack+0x2b) [0xb7ef085f]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__os_abort+0x1d) [0xb7eecbb1]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__env_panic+0xa8) [0xb7e80b8e]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__lock_get_internal+0x1855) [0xb7e2e925]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__lock_vec+0x167) [0xb7e2bd3c]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__db_lget+0x45c) [0xb7e8bfe2]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so [0xb7d850a5]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so [0xb7d816ce]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__dbc_get+0x598) [0xb7e76a29]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__dbc_get_pp+0x11f) [0xb7e870ec]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(_ZN3Dbc3getEP3DbtS1_j+0x3f) [0xb7d71feb]
...(non-berkeley db layers)...
/lib/tls/i686/cmov/libpthread.so.0 [0xb7a2350f]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xb7b20a0e]
Aborted
In studying the core dumps and doing some testing this behavior only occurs when cursors are used, when those operations are removed the error no longer manifests itself so either the cursor is the issue, or how the cursors are interacting with other operations. Below is a sample of cursor code being run:
void run_cursor()
Dbc *curp = NULL;
int ret;
try
int i = 0;
Dbt key(&i,sizeof(int));
Dbt data(buffer, 0);
data.set_flags(DB_DBT_USERMEM);
data.set_ulen(MAX_BUFFER_SIZE);
db->cursor(NULL, &curp, 0);
while ((ret = curp->get(&key, &data, DB_NEXT)) == 0) {
//keep a list of all the keys in the db
keys.push_back(i);
if (ret != DB_NOTFOUND) {
cerr << "Cursor Error" << endl;
curp->close();
catch (DbException &ex) {
cout << ex.what() << endl;
if( curp != NULL )
curp->close();
I have tried wrapping the cursor in a transaction and it seems to worsen the issue (it happens more frequently). In all methods I am printing error messages when exceptions are thrown by the database (e.g. DbExceptions) and the program prints no errors so I do not believe it is a side effect of some other error. Here is the relevant parts of the case where is RMW lock is used:
void update()
DbTxn *txn = NULL;
try
env->txn_begin(NULL, &txn, 0);
int id;
Dbt key( &id, sizeof(int) );
Dbt value(buffer, 0);
value.set_flags(DB_DBT_USERMEM);
value.set_ulen(MAX_BUFFER_SIZE);
int err = db->get(txn, &key, &value, DB_RMW);
if( err == DB_NOTFOUND )
cerr << "Error trying to read key" << endl;
txn->abort(0);
return;
//code to modify the value here
db->put(txn, &key, &value, 0);
txn->commit(0);
} catch (DbDeadlockException &de) {
cout << de.what() << endl;
if( txn != NULL )
txn->abort();
Just in case it helps, here is the output of db_stat -CA on the database after it has crashed:
Default locking region information:
12 Last allocated locker ID
0x7fffffff Current maximum unused locker ID
9 Number of lock modes
5000 Maximum number of locks possible
5000 Maximum number of lockers possible
1000 Maximum number of lock objects possible
1 Number of lock object partitions
6 Number of current locks
9 Maximum number of locks at any one time
4 Maximum number of locks in any one bucket
0 Maximum number of locks stolen by for an empty partition
0 Maximum number of locks stolen for any one partition
11 Number of current lockers
13 Maximum number of lockers at any one time
3 Number of current lock objects
9 Maximum number of lock objects at any one time
2 Maximum number of lock objects in any one bucket
0 Maximum number of objects stolen by for an empty partition
0 Maximum number of objects stolen for any one partition
3786734 Total number of locks requested
3786664 Total number of locks released
0 Total number of locks upgraded
1 Total number of locks downgraded
7089 Lock requests not available due to conflicts, for which we waited
61 Lock requests not available due to conflicts, for which we did not wait
0 Number of deadlocks
0 Lock timeout value
0 Number of locks that have timed out
1000000 Transaction timeout value
0 Number of transactions that have timed out
1MB 592KB The size of the lock region
0 The number of partition locks that required waiting (0%)
0 The maximum number of times any partition lock was waited for (0%)
0 The number of object queue operations that required waiting (0%)
0 The number of locker allocations that required waiting (0%)
165 The number of region locks that required waiting (0%)
2 Maximum hash bucket length
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock REGINFO information:
Lock Region type
5 Region ID
__db.005 Region name
0xb781b000 Original region address
0xb781b000 Region address
0xb781b0c0 Region primary address
0 Region maximum allocation
0 Region allocated
Region allocations: 11006 allocations, 0 failures, 0 frees, 1 longest
Allocations by power-of-two sizes:
1KB 11003
2KB 0
4KB 0
8KB 0
16KB 1
32KB 0
64KB 1
128KB 1
256KB 0
512KB 0
1024KB 0
REGION_JOIN_OK Region flags
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock region parameters:
104 Lock region region mutex [165/7025066 0% 5979/3075574672]
8191 locker table size
1031 object table size
600 obj_off
62608 locker_off
0 need_dd
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock conflict matrix:
0 0 0 0 0 0 0 0 0
0 0 1 0 1 0 1 0 1
0 1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0 0
0 1 1 0 0 0 0 1 1
0 0 1 0 0 0 0 0 1
0 1 1 0 0 0 0 1 1
0 0 1 0 1 0 1 0 0
0 1 1 0 1 1 1 0 1
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by lockers:
Locker Mode Count Status ----------------- Object ---------------
4 dd= 9 locks held 1 write locks 0 pid/thread 5979/3080828608
4 READ 1 HELD ./db.db handle 0
5 dd= 8 locks held 0 write locks 0 pid/thread 5979/3072416656
6 dd= 7 locks held 0 write locks 0 pid/thread 5979/3073469328
7 dd= 6 locks held 0 write locks 0 pid/thread 5979/3073469328
8 dd= 5 locks held 0 write locks 0 pid/thread 5979/3073469328
9 dd= 4 locks held 0 write locks 0 pid/thread 5979/3072416656
a dd= 3 locks held 1 write locks 0 pid/thread 5979/3073469328
a READ 1 WAIT ./db.db page 21
a READ 2 HELD ./db.db page 20
b dd= 2 locks held 1 write locks 0 pid/thread 5979/3075574672
b READ 1 WAIT ./db.db page 21
b READ 2 HELD ./db.db page 20
c dd= 1 locks held 0 write locks 0 pid/thread 5979/3075574672
800054d9 dd= 0 locks held 1 write locks 1 pid/thread 5979/3074522000expires 05-05-12:41:36.048050000
800054d9 WRITE 2 HELD ./db.db page 21
800054da dd= 0 locks held 0 write locks 0 pid/thread 5979/3072416656
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by object:
Locker Mode Count Status ----------------- Object ---------------
800054d9 WRITE 2 HELD ./db.db page 21
a READ 1 WAIT ./db.db page 21
b READ 1 WAIT ./db.db page 21
a READ 2 HELD ./db.db page 20
b READ 2 HELD ./db.db page 20
4 READ 1 HELD ./db.db handle 0
The number of locks and lockers is somewhat low at 5,000 each, this was intentional since the same error occurs with larger numbers of locks/lockers it simply takes longer to occur. I have reviewed this output and nothing out of the ordinary seems to be occurring as far as I can tell.
Any additional insight would be appreciated.Thanks to both of you for your suggestions.
I have still noticing the same issue with very similar output etc so I wont repost that. After running it some more I have started seeing another error message, it looks like its very similar and hopefully with make the issue more clear.
Here is the output of the error file:
TAS unlock failed: lock already unlocked
PANIC: Permission denied
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__os_stack+0x2b) [0xb7ee585f]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__os_abort+0x1d) [0xb7ee1bb1]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__env_panic+0xa8) [0xb7e75b8e]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__db_tas_mutex_unlock+0xa5) [0xb7d72245]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so [0xb7e27d04]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__lock_detect+0x129) [0xb7e268bf]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__lock_get_internal+0x1434) [0xb7e23504]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__lock_vec+0x167) [0xb7e20d3c]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__db_lget+0x45c) [0xb7e80fe2]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so [0xb7d7a0a5]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so [0xb7d766ce]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__dbc_get+0x598) [0xb7e6ba29]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(__dbc_get_pp+0x11f) [0xb7e7c0ec]
/usr/local/BerkeleyDB.4.7/lib/libdb_cxx-4.7.so(_ZN3Dbc3getEP3DbtS1_j+0x3f) [0xb7d66feb]
...(user code stack layers)...
/lib/tls/i686/cmov/libpthread.so.0 [0xb79a950f]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xb7a9ea0e]
Here is the output of the waits for table:
Waiter: Waiting on:
80002249/50: 957608
6/50: 80002249 957208
Waitsfor array
Waiter: Waiting on:
8000224a/28: 957288
a/28: 8000224a 957608
Waitsfor array
Waiter: Waiting on:
8000224c/22: 957608
6/22: 8000224c 957368
Waitsfor array
Waiter: Waiting on:
8000224c/22: 957608
6/22: 8000224c 957368
4/22: 8000224c 957048
Waitsfor array
Waiter: Waiting on:
8000224d/25: 957528
7/25: 8000224d 957608
Waitsfor array
Waiter: Waiting on:
8000224d/25: 957528
7/25: 8000224d 957608
6/25: 8000224d 957208
Waitsfor array
Waiter: Waiting on:
8000224d/25: 957528
7/25: 8000224d 957608
6/25: 8000224d 957208
4/25: 8000224d 957288
Waitsfor array
Waiter: Waiting on:
80002251/10: 957048
a/10: 80002251 957128
Waitsfor array
Waiter: Waiting on:
80002257/39: 80002254 957368
80002254/39: 957288
Waitsfor array
Waiter: Waiting on:
80002258/14: 957048
4/14: 80002258 957608
Waitsfor array
Waiter: Waiting on:
80002258/14: 957048
8/14: 80002258 957208
4/14: 80002258 957608
Waitsfor array
Waiter: Waiting on:
80002259/39: 80002257 957448
80002258/14: 957048
80002257/39: 957368
8/14: 80002258 957208
4/14: 80002258 957608
Waitsfor array
Waiter: Waiting on:
80002259/39: 957448
8/39: 80002259 957288
7/39: 80002259 957048
6/39: 80002259 957528
Waitsfor array
Waiter: Waiting on:
80002259/0: 0
80002257/0: 0
Waitsfor array
Waiter: Waiting on:
8000225a/40: 956968
7/40: 8000225a 956888
Waitsfor array
Waiter: Waiting on:
8000225a/40: 956968
7/40: 8000225a 956888
4/40: 8000225a 957608
It looks like there is a similar pattern here at the end with the two entries waiting on zero ("80002259/0: 0 80002257/0: 0"), which is why I suspect its the same error just manifesting itself in a slightly different manner.
I am using lock detection, I currently have it set to use DB_LOCK_MAXLOCKS, I have tried others as well with not change in behavior.
Most of my transaction code takes the following form, so if the get is successful (the id exists) then the data is updated and written back to the database. If an exception is thrown then it is caught and the transaction is aborted. As far as I can tell this should not leave and transaction handles left open or anything along those lines as the error message suggests.
void do_txn()
DbTxn *txn = NULL;
try
db->txn_begin(NULL, &txn, 0);
Data d;
//does a db->get using the txn pointer and a DB_RMW lock
//returns do_get returns false if the id does not exist
bool success = do_get(id, &d, txn);
if( !success )
txn->abort();
return;
//update data as necessary
//does a db->put using the txn pointer
do_put(id, &d, txn);
txn->commit(0);
} catch (DbDeadlockException &de) {
cout << de.what() << endl;
if( txn != NULL )
txn->abort();
} -
Users need to add Projects into Planning. I know they can if they create a Application Type "Projects Financial Planning"...
The problem is we are developing a "Public Sector" Application Type.
What are the ways I can move metadata, "new Projects", in the "Projects" application to the Public Sector cube?
Thanks,
PRHi PR,
Can't you have both Public Sector and Projects Financial Planning as separate apps and then let the data flow between apps using Map Reporting Application? UC Berkeley owns PSBP and we have anoter plan type that PSBP data flows into. M2 Dynamic and UCB did couple of presentations at ODTUG in the last couple of years. I uploaded last year's presentation into Dropbox. You might find it useful. Here is the link: https://www.dropbox.com/s/ehz3b5kuzaklsd4/ttymczak.enterprisepublic.pdf
Cheers,
Mehmet
Maybe you are looking for
-
500GB Hard drive failure - for the 4th time
Ok, So i bought a macbook pro from Apple in about may 2010. It is a 500GB Serial ATA Drive @ 7200 2.66GHz Intel Core i7 GB 1066MHz DDR3 SDRM SuperDrive 8X DL. Since then the hard drive has failed 3 times and now it has failed for the 4th time this mo
-
Hi, In MIRO FI entry passed Vendor Cr. GRIR Clearing Dr. Now We are having a scenario where above entries should be GL code Cr. GRIR Clearing Dr. Now it should be automatic that when we click Post that Vendor should automatic substituted with GL. Ple
-
PM G4/667 doesn't boot after 10.4.11 update
Hello, We've got a Powermac G4/667 (Digital Audio, 384Mb RAM) equiped with a SATA card on a PCI slot, with 2 hard disks. On the biggest one (250 Gb, 60Gb free), we've got 17 users, as the mac is used as a backup server using AppleTalk under 10.4 (sta
-
Following 455398.1 Using Oracle RAC and ASM with EBS 11i and 11g Finished step 3.6 Post-Oracle RAC Migration Steps Now ASM listener will not come ONLINE. When started manually, it terminates, no error message displayed. Questions: 1- Is there a log f
-
Camera app on iPad only gives me a blank screen. It worked fine earlier today. .
The camera app on my iPad gives me a blank screen so I can't take pics or videos. It worked earlier today.