[PATCH] Fix find()/count() lookups on dbstl set/map

Question

Applies to 5.2.36
Fix find()/count() lookups on dbstl set/map
A bug was discovered that manifests when a custom marshaller
is in use for objects serving as a key for either a db_map,
or the item in a db_set. It may affect other subsystems
(multimap/sets, etc) but this has not been explicitly confirmed.
The symptom is that items that are inserted cannot be properly
queried for (e.g. via .find()/.count()) within a container, yet
the items can be iterated. It was determined that the root cause
of this problem is related to the fact that items are inserted
based on their custom marshaller, but are queried for using
the default shallow/contiguous method. It is highly unlikely that
both methods would return a comparable key, and therefore this
explains why queries (such as via .find()) would not result
in a match.
The reason for the discrepancy relates to how the DBT is formed
for a given query. In the insert path, the DBT "onstack"
optimization was disabled, and this resulted in the DBT formation
properly invoking the copy_function registered in the DbstlElemTraits.
In the query path, the "onstack" optimization was enabled. This
optimization avoids a malloc/memcpy cycle by just pointing the DBT
to the object buffer. This is great when the buffer is already
properly formed. However, when a copy_function is registered, this
will hardly ever be the case. Therefore, the fix is to check
if there is a custom marshaller registered when determining if
the "onstack" optimization can be taken, and to skip the optimization
when a marshaller is present. The ramification of this change is
that DBTs formed under a custom marshaller will always be forced
to take the malloc/memcpy hit, but this is unavoidable and logical
anyway.
Signed-off-by: Gregory Haskins <[email protected]>
lang/cxx/stl/dbstl_dbt.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/lang/cxx/stl/dbstl_dbt.h b/lang/cxx/stl/dbstl_dbt.h
index 43c5a19..691a9bd 100644
--- a/lang/cxx/stl/dbstl_dbt.h
+++ b/lang/cxx/stl/dbstl_dbt.h
@@ -408,7 +408,7 @@ public:
else
sz = sizeof(dt);
- if (onstack) {
+ if (onstack && EM::instance()->get_copy_function() == NULL) {
freemem();
pdbt->data = ((void*)&dt);
// We have to set DB_DBT_USERMEM for DB_THREAD to work.

Answer 1

Hi Greg,
Thank you for addressing this. Your fix is very valuable.
We've made some changes(including yours) to fix this kind of issue, and these changes will be in next major release.
Regards,
Winter Zhang
Oracle Berkeley DB