Write-behind cache not removing entries after upgrade to 3.2

Question

We recently upgraded tangosol.jar and coherence.jar from version 3.0 to version 3.2. After the upgrade, our write-behind caches began consuming all available memory and crashing the JVMs because the entries were not being removed from the cache after being written to the database. We rolled back to the 3.0 jars without making any other modifications and the caches behave as expected. We'd really like to move to 3.2 for the improved network fault tolerance, but we need to resolve this issue first.
What changes were made in 3.2 with respect to write-behind caches that might cause this issue? I've reviewed our configuration and our code and can't find anything unusual, but I'm not sure what I should be looking for.
Any ideas?

Answer 1

I've opened an SR, but I haven't heard back. In the meantime, I've continued digging and I've noticed something strange - in the store() method of our backing map implementation, we take the entry that we just persisted and remove it from the backing map.
In my small-scale local tests, the size of the map is 1 when we enter store() and is 0 when we leave, as expected. If we process another entry using the 3.0 jars, it's again 1 and then 0. However, it gets more interesting with the 3.2 jars - the size of the map is 1 when we enter store() the first time and 0 when we leave, but if we process another entry, the size is 2 when we enter and 1 when we leave. This pattern continues such that both values increase by 1 every time we process an entry.
This would imply that we're either removing the entries incorrectly, or they're somehow being reinserted into the map.
Any ideas?
Here's the body of our method (with a bunch of sysouts added to the normal logging because this app won't run correctly under a debugger):
        * Store the specified value under the specific key in the underlying
        * store, then remove the specific key from the internal map and hence
        * the cache itself. This method is intended to support both key/value
        * creation and value update for a specific key.
        * @param oKey   key to store the value under
        * @param oValue value to be stored
        * @throws UnsupportedOperationException if this implementation or the
        *                                       underlying store is read-only
        public void store(Object oKey, Object oValue)
            RemoveOnStoreRWBackingMap mapBacking = RemoveOnStoreRWBackingMap.this;
            System.out.println("map storing  " + oKey);
            System.out.println("Size before = " + mapBacking.entrySet().size());
            Iterator entries = mapBacking.entrySet().iterator();
            while (entries.hasNext()) {
                System.out.println("entry = " + entries.next());   
            String storeClassName = getCacheStore().getClass().getName();
            Logger log = Logger.getLogger(storeClassName);
            log.debug(storeClassName + ": In store method.  Storing " + oKey);
            long cFailuresBefore = getStoreFailures();
            log.debug(storeClassName + ": failures before=" + cFailuresBefore);
            super.store(oKey, oValue);
            long cFailuresAfter = getStoreFailures();
            log.debug(storeClassName + ": failures afer=" + cFailuresAfter);
            if (cFailuresBefore == cFailuresAfter)  {
                log.debug(storeClassName + ": About to remove");
                mapBacking = RemoveOnStoreRWBackingMap.this;
                Converter converter = mapBacking.getContext().getKeyToInternalConverter();
                System.out.println("removed " + mapBacking.remove(converter.convert(oKey)));
//                System.out.println("removed " + mapBacking.getInternalCache().remove(converter.convert(oKey)));
                log.debug(storeClassName + ": Removed");
            Converter converter = RemoveOnStoreRWBackingMap.this.getContext().getKeyFromInternalConverter();
            System.out.println("Size after = " + mapBacking.entrySet().size());
        }