Coherence entries do not expire

Question

Hello,
I have a replicated scheme and set the expiration time (30 minutes) in the put method from NamedCache. In the first hour and a half, the behavior was as expected, from the logs I could see the entries expired after 30 minutes. But after that they seem to not expire. My problem is that some old data comes from the cache, although I could see the new data was stored. So i'm affraid the old data remains somewhere, maybe on some nodes and then is probably replicated back to all nodes.
I'm not sure what happens, but I only found this post: Problem with Coherence replicated cache and put operation with cMillis para where it says that replicated cache does not fully support per-entry expiration. Has anyone heard or experienced something similar?
And do you know how can I remove the entries from the cache?

Answer 1

Hi,
Given that the reply in this thread Problem with Coherence replicated cache and put operation with cMillis para was from Jason who works in the Coherence Engineering Team, and it was only a few months ago, I would be inclined to believe him when he says per-entry expiry is not supported in replicated caches.
An alternative to replicated caches is to use a Continuous Query Cache that is backed by a distributed cache. This will work like a replicated cache but gets around a lot of the limitations you have with replicated cache, like per-entry expiry.
The easiest way to make a replicated cache using a CQC is to use a class-scheme and custom wrapper class like this.
Your cache configuration would look something like this...
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
    <defaults>
        <serializer>pof</serializer>
    </defaults>
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>Replicated*</cache-name>
            <scheme-name>cqc-scheme</scheme-name>
            <init-params>
                <init-param>
                    <param-name>underlying-cache</param-name>
                    <param-value>ReplicatedUnderlying*</param-value>
                </init-param>
            </init-params>
        </cache-mapping>
        <cache-mapping>
            <cache-name>ReplicatedUnderlying*</cache-name>
            <scheme-name>replicated-underlying-scheme</scheme-name>
        </cache-mapping>
    </caching-scheme-mapping>
    <caching-schemes>
        <class-scheme>
            <scheme-name>cqc-scheme</scheme-name>
            <class-name>com.thegridman.coherence.CQCReplicatedCache</class-name>
            <init-params>
                <init-param>
                    <param-type>{cache-ref}</param-type>
                    <param-value>{underlying-cache}</param-value>
                </init-param>
                <init-param>
                    <param-type>java.lang.String</param-type>
                    <param-value>{cache-name}</param-value>
                </init-param>
            </init-params>
        </class-scheme>
        <distributed-scheme>
            <scheme-name>replicated-underlying-scheme</scheme-name>
            <service-name>ReplicatedUnderlyingService</service-name>
            <backing-map-scheme>
                <local-scheme/>
            </backing-map-scheme>
        </distributed-scheme>
    </caching-schemes>
</cache-config>Any cache with a name prefixed by "Replicated", e.g. ReplicatedTest, would map to the cqc-scheme, which is our custom class scheme. This will create another distributed cache prefixed with "ReplicatedUnderlying", e.g. for the ReplicatedTest cache we would also get the ReplicatedUnderlyingTest cache.
Now, because the data will go into the distributed scheme we can do all the normal things with this cache that we can do with any distributed cache, e.g. we could add cache stores, listeners work properly, we know were entry processors will go, etc...
The com.thegridman.coherence.CQCReplicatedCache is our custom NamedCache implementation that is basically a CQC wrapper around the underlying cache.
The simplest form of this class is...
package com.thegridman.coherence;
import com.tangosol.net.NamedCache;
import com.tangosol.net.cache.ContinuousQueryCache;
import com.tangosol.net.cache.WrapperNamedCache;
import com.tangosol.util.filter.AlwaysFilter;
public class CQCReplicatedCache extends WrapperNamedCache {
    public CQCReplicatedCache(NamedCache underlying, String cacheName) {
        super(new ContinuousQueryCache(underlying, AlwaysFilter.INSTANCE), cacheName);
}Now in your application code you can still get caches just like normal, you code has no idea that the cache is really a wrapper around a CQC. So you can just do normal stuff like this in your code...
NamedCache cache = CacheFactory.getCache("ReplicatedTest");
cache.put("Key-1", "Value-1");This though still does not quite support per-entry expiry reliably. The way that expiry works in Coherence is that entries are only expired when another action happens on a cache, such as a get, put, size, entrySet and so on. The problem with the code above is that say you do this...
NamedCache cache = CacheFactory.getCache("ReplicatedTest");
cache.put("Key-1", "Value-1", 5000);
Thread.sleep(6000);
Object value = cache.get("Key-1");...you would expect to get back null, but you will still get back "Value-1". This is because the value will be from the CQC and not hit the underlying cache. To make everything work for eviction then for certain CQC operations we need to poke the underlying cache in our wrapper, so we change the CQCReplicatedCache class like this
package com.thegridman.coherence;
import com.tangosol.net.NamedCache;
import com.tangosol.net.cache.ContinuousQueryCache;
import com.tangosol.net.cache.WrapperNamedCache;
import com.tangosol.util.Filter;
import com.tangosol.util.filter.AlwaysFilter;
import java.util.Collection;
import java.util.Comparator;
import java.util.Map;
import java.util.Set;
public class CQCReplicatedCache extends WrapperNamedCache {
    private NamedCache underlying;
    public CQCReplicatedCache(NamedCache underlying, String cacheName) {
        super(new ContinuousQueryCache(underlying, AlwaysFilter.INSTANCE), cacheName);
        this.underlying = underlying;
    public NamedCache getUnderlying() {
        return underlying;
    @Override
    public Set entrySet(Filter filter) {
        underlying.size();
        return super.entrySet(filter);
    @Override
    public Set entrySet(Filter filter, Comparator comparator) {
        underlying.size();
        return super.entrySet(filter, comparator);
    @Override
    public Map getAll(Collection colKeys) {
        underlying.size();
        return super.getAll(colKeys);
    @Override
    public Object invoke(Object oKey, EntryProcessor agent) {
        underlying.size();
        return super.invoke(oKey, agent);
    @Override
    public Map invokeAll(Collection collKeys, EntryProcessor agent) {
        underlying.size();
        return super.invokeAll(collKeys, agent);
    @Override
    public Map invokeAll(Filter filter, EntryProcessor agent) {
        underlying.size();
        return super.invokeAll(filter, agent);
    @Override
    public Set keySet(Filter filter) {
        underlying.size();
        return super.keySet(filter);
    @Override
    public boolean containsValue(Object oValue) {
        underlying.size();
        return super.containsValue(oValue);
    @Override
    public Object get(Object oKey) {
        underlying.size();
        return super.get(oKey);
    @Override
    public boolean containsKey(Object oKey) {
        underlying.size();
        return super.containsKey(oKey);
    @Override
    public boolean isEmpty() {
        underlying.size();
        return super.isEmpty();
    @Override
    public int size() {
        return underlying.size();
}We have basically poked the underlying cache, by calling size() on it prior to doing any other operation that would not normally hit the underlying cache, so causing eviction to trigger. I think I have covered all the methods above but it should be obvious how to override any others I have missed.
If you now run this code
NamedCache cache = CacheFactory.getCache("ReplicatedTest");
cache.put("Key-1", "Value-1", 5000);
Thread.sleep(6000);
Object value = cache.get("Key-1");it should work as expected.
JK