Running a task locally or invoking it once

Question

Hi,
I want to run a quartz job on the entries of my cache deleting them if they apply to some constraint. I guess I can do that in 2 ways:
1. run the quartz job on only 1 node which will call cache.invokeAll(filter, myProcessor). This node will send the entryProcessor to the other nodes that contain the relevant data.
Question: How can I make sure a quartz job is run on only one node as all nodes are running the same code and use the same configuration? I don't want to specify the same quartz job an all nodes because then the job would be invoked once for each node. Do I need to define some sort of election policy for that? Or are there better ways to handle a similar situation where only one node of a cluster needs to trigger some action?
2. run the quartz job on all nodes of the cluster. Each job will call cache.invokeAll(new KeyAssociatedFilter(filter), myProcessor) on every node. This will make sure the entryProcessor will only be executed on the own node's data. Will this work?
Best regards
Jan

Answer 1

Jan,
could you please describe your eviction criteria? If you want to remove the entries, then removing it by cache.removeAll() is the fastest approach, and for local only keys it automatically knows not to send to remote nodes, so you don't need the partition filter there.
On the other hand, as you described your eviction criteria in an earlier post to require decision on the content of the entire cache, in which case the prefiltering should not have a partitioned filter.
If your eviction criteria is the following: evict the number of keys from each category as the number of the surplus entries in the category, then PartitionFilter is not something you would want to use.
Instead, you can first do an aggregation on the count of keys in each category to find out the number of keys to be removed. This can easily be done by a GroupAggregator and a Count:
categorySizesByDiscriminator = cache.aggregate(GroupAggregator.createInstance("getCategoryDiscriminator", new Count()));where getCategoryDiscriminator is the name of the method (parentheses intentionally missing) which returns a different value for each category (was it getColor? ).
After this you know how many entries the surplus is.
You can now send a custom parallel aggregation which returns as many keys as the surplus from the category is from each node and throws away the unnecessary from the result (this is necessary because of the arbitrary distribution of the entries), something like this:
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Set;
import com.tangosol.io.ExternalizableLite;
import com.tangosol.io.pof.PofReader;
import com.tangosol.io.pof.PofWriter;
import com.tangosol.io.pof.PortableObject;
import com.tangosol.util.ExternalizableHelper;
import com.tangosol.util.InvocableMap;
import com.tangosol.util.InvocableMap.EntryAggregator;
import com.tangosol.util.InvocableMap.ParallelAwareAggregator;
import com.tangosol.util.extractor.ReflectionExtractor;
public class ByCategoryToBeEvictedKeyHarvester implements
          ParallelAwareAggregator, PortableObject, ExternalizableLite {
     public static class PerNodeResultElement implements PortableObject, ExternalizableLite {
          private String categoryDiscriminator;
          private int currentCountPerCategory;
          private List<Object> keys;
          private transient int toBeEvictedFromCategory;
          public PerNodeResultElement() {
               super();
          private PerNodeResultElement(String categoryDiscriminator, int toBeEvictedFromCategory) {
               super();
               this.categoryDiscriminator = categoryDiscriminator;
               this.currentCountPerCategory = 0;
               this.keys = new ArrayList<Object>(toBeEvictedFromCategory);
               this.toBeEvictedFromCategory = toBeEvictedFromCategory;
          private void addEntry(InvocableMap.Entry entry) {
               if (toBeEvictedFromCategory < keys.size()) {
                    keys.add(entry.getKey());
               ++currentCountPerCategory;
          @Override
          public void readExternal(PofReader reader) throws IOException {
               categoryDiscriminator = reader.readString(0);
               currentCountPerCategory = reader.readInt(1);
               keys = (List<Object>) reader.readCollection(2, new ArrayList<Object>());
          @Override
          public void writeExternal(PofWriter writer) throws IOException {
               writer.writeString(0, categoryDiscriminator);
               writer.writeInt(1, currentCountPerCategory);
               writer.writeCollection(2, keys);
          @Override
          public void readExternal(DataInput input) throws IOException {
               categoryDiscriminator = ExternalizableHelper.readSafeUTF(input);
               currentCountPerCategory = ExternalizableHelper.readInt(input);
               int numKeys = ExternalizableHelper.readInt(input);
               keys = numKeys > 0 ? new ArrayList<Object>(numKeys) : Collections.EMPTY_LIST;
               while (numKeys > 0) {
                    keys.add(ExternalizableHelper.readObject(input));
                    --numKeys;
          @Override
          public void writeExternal(DataOutput output) throws IOException {
               ExternalizableHelper.writeSafeUTF(output, categoryDiscriminator);
               ExternalizableHelper.writeInt(output, currentCountPerCategory);
               ExternalizableHelper.writeInt(output, keys.size());
               for (Object key : keys) {
                    ExternalizableHelper.writeObject(output, key);
     private Map<String,Integer> toEvictByCategories;
     private String categoryDiscriminatorGetMethodName;
     private transient int toEvictAbove;
     public ByCategoryToBeEvictedKeyHarvester(
               String categoryDiscriminatorGetMethodName, Map<String, Integer> toEvictByCategories, int toEvictAbove) {
          super();
          this.toEvictByCategories = toEvictByCategories;
          this.categoryDiscriminatorGetMethodName = categoryDiscriminatorGetMethodName;
          this.toEvictAbove = toEvictAbove;
     private static class PerCategoryCounter {
          private int elementsPerCategory = 0;
          private LinkedList<Collection<Object>> keys = new LinkedList<Collection<Object>>();
          void addPerNodeResultElement(PerNodeResultElement element) {
               elementsPerCategory += element.currentCountPerCategory;
               keys.add(element.keys);
     @Override
     public Object aggregateResults(Collection results) {
          Map<String, PerCategoryCounter> temp = new HashMap<String, PerCategoryCounter>();
          int numKeysToRemove = 0;
          for (Object result : results) {
               Collection<PerNodeResultElement> perNodeResults = (Collection<PerNodeResultElement>) result;
               for (PerNodeResultElement perNodeResultElement : perNodeResults) {
                    String categoryDiscriminator = perNodeResultElement.categoryDiscriminator;
                    PerCategoryCounter counter = temp.get(categoryDiscriminator);
                    if (counter == null) {
                         counter = new PerCategoryCounter();
                         temp.put(categoryDiscriminator, counter);
                    int oldCount = counter.elementsPerCategory;
                    counter.addPerNodeResultElement(perNodeResultElement);
                    int newCount = counter.elementsPerCategory;
                    if (newCount > toEvictAbove) {
                         numKeysToRemove += newCount - (oldCount < toEvictAbove ? toEvictAbove : oldCount);
          // we now have a current view of how many elements there are (sort of)
          // and hopefully enough keys to evict (if there were not really too many entries added since then)
          ArrayList keysToRemove = new ArrayList(numKeysToRemove);
          for (PerCategoryCounter perCategoryCounter : temp.values()) {
               int elementsPerCategory = perCategoryCounter.elementsPerCategory;
               Iterator<Collection<Object>> iter = perCategoryCounter.keys.iterator();
               while (elementsPerCategory > toEvictAbove && iter.hasNext()) {
                    Collection<Object> keys = iter.next();
                    Iterator<Object> keyIter = keys.iterator();
                    while (elementsPerCategory > toEvictAbove && keyIter.hasNext()) {
                         keysToRemove.add(keyIter.next());
                         --elementsPerCategory;
          return keysToRemove;
     @Override
     public EntryAggregator getParallelAggregator() {
          return this;
     @Override
     public Object aggregate(Set entries) {
          final ReflectionExtractor extractor = new ReflectionExtractor(categoryDiscriminatorGetMethodName);
          Map<String,PerNodeResultElement> temp = new HashMap<String, PerNodeResultElement>();
          for (Object object : entries) {
               InvocableMap.Entry entry = (InvocableMap.Entry) object;
               String categoryDiscriminator = (String) entry.extract(extractor);
               Integer toBeEvictedFromCategory = toEvictByCategories.get(categoryDiscriminator);
               if (toBeEvictedFromCategory != null && toBeEvictedFromCategory > 0) {
                    PerNodeResultElement element = temp.get(categoryDiscriminator);
                    if (element == null) {
                         element = new PerNodeResultElement(categoryDiscriminator, toBeEvictedFromCategory);
                         temp.put(categoryDiscriminator, element);
                    element.addEntry(entry);
          return temp.values();
     @SuppressWarnings("unchecked")
     public void readExternal(PofReader reader) throws IOException {
          categoryDiscriminatorGetMethodName = reader.readString(0);
          toEvictByCategories = (Map<String, Integer>) reader.readMap(1, new HashMap<String, Integer>());
     @Override
     public void writeExternal(PofWriter writer) throws IOException {
          writer.writeString(0, categoryDiscriminatorGetMethodName);
          writer.writeMap(1, toEvictByCategories, String.class, Integer.class);
     @Override
     public void readExternal(DataInput input) throws IOException {
          categoryDiscriminatorGetMethodName = ExternalizableHelper.readSafeUTF(input);
          int numEntries = ExternalizableHelper.readInt(input);
          toEvictByCategories = new HashMap<String, Integer>();
          while (numEntries > 0) {
               String key = ExternalizableHelper.readSafeUTF(input);
               Integer value = ExternalizableHelper.readInt(input);
               toEvictByCategories.put(key, value);
               --numEntries;
     @Override
     public void writeExternal(DataOutput output) throws IOException {
          ExternalizableHelper.writeSafeUTF(output, categoryDiscriminatorGetMethodName);
          ExternalizableHelper.writeInt(output, toEvictByCategories.size());
          for (Map.Entry<String, Integer> entry : toEvictByCategories.entrySet()) {
               ExternalizableHelper.writeSafeUTF(output, entry.getKey());
               ExternalizableHelper.writeInt(output, entry.getValue());
}You can send this aggregator and remove the entries afterwards with:
String categoryDiscriminatorGetMethodName = "getColor"; // do I remember well?
// this is the map which you populate based on the result of the first aggregation,
// keys should be colors,
// values should be the number of entries in the category in the entire cache minus the eviction limit (the number of entries you want to evict from the category)
Map<String,Integer> toEvictByCategories = ...;
Collection<Object> keysToRemove = cache.aggregate(AlwaysFilter.INSTANCE, new ByCategoryToBeEvictedKeyHarvester(categoryDiscriminatorGetMethodName, toEvictByCategories, toEvictAbove));
cache.removeAll(keysToRemove);Best regards,
Robert