Query Cache to Derive Summary Data - Advice?

Question

Good Evening, I am faced with a problem and I was hoping to get a nudge in the right direction.
     I have a cache that is storing the status of orders and their respective times they took to process in that status going though our system. I need to retrieve from that some average times. I need to calculate the average time it took for an order to go from status "A" (endtime) to status "B" (starttime). So essentially i will need to retrieve the difference between variables of two different statuses for every order then get the average of those. I know how to query the cache to get my result set and then obviously iterate across them and calculate the average. However I was wondering/hoping there was a more eloquent solution.
     I have been reading through the user guide (http://wiki.tangosol.com/display/COH32UG/Provide+a+Data+Grid) and reading up on Aggregators and Agents but there are no complete examples for me to view and am unsure if these approaches would help in my goal.
     Knowing my task, is there any approach i should focus on and if so is there documentation anywhere to support that?
     Thanks in Advance!
     -- Grant
     Our data structure looks something like this:
     class ClientReport{
          public long lasttimeaccessed;
     public long clientid;
          ...some other stuff....
          Map Orders; //map of Order Objects, keyed on orderid
     class Order{
          public long orderID;
          ...some other stuff....
          Map statuses //map of status changes, keyed on status
     class Status{
          public String status;
          public long starttime;
          public long endtime;     
     }

Answer 1

Hi Grant,
     first of all, you might want to create an index on the status changes, so that you can prefilter for the status changes, but for that you would need to implement a custom ValueExtractor implementation, because the status is twice nested in maps, and use that to create the index.
     You can add the index with that ValueExtractor with the addIndex(ValueExtractor, boolean, Comparator) method of the cache.
     The ValueExtractor would need to collect the status strings from each status change in each order, and return them as a collection.
     You can then use this index to filter for those report objects which do have at least one status changes ending up in the expected state.
     You could also create a similar ValueExtractor (let's name it ReportStatusObjectExtractor) which returns each Status objects from all orders in a report, and create an index also with this ValueExtractor.
     Please be aware that the ValueExtractor-s need to implement the equals() and the hashCode() method properly. In this case, because the extractors will have no parameters, you can implement equals() so that true is returned for all objects of the same class, and hashCode() can return 1.
     This index could then be used to speed up evaluation of the aggregator because you would not need to deserialize the entire report instance.
     You can now create an own InvocableMap.EntryAggregator implementation which should also implement the InvocableMap.ParallelAwareAggregator.
     The getParallelAggregator() method should return an instance of a Serializable class again implementing EntryAggregator, and in the aggregate method it should use the extract method on each entry with an instance of the second mentioned ValueExtractor (should be a constant) to extract the status objects belonging to the report instance from the index:
              public class ReportStatusStringExtractor implements ValueExtractor, Serializable {
           public static final ReportStatusStringExtractor INSTANCE = new ReportStatusStringExtractor();
           public Object extract(Object oTarget) {
             ClientReport report = (ClientReport) oTarget;
             // extract each status strings from each order object and return all of them in a collection
           public boolean equals(Object other) {
             return other!=null && other.getClass().equals(this.getClass());
           public int hashCode() {
             return 1;
         public class ReportStatusObjectExtractor implements ValueExtractor, Serializable {
           public static final ReportStatusObjectExtractor INSTANCE = new ReportStatusObjectExtractor();
           public Object extract(Object oTarget) {
             ClientReport report = (ClientReport) oTarget;
             // extract each status objects from each order object and return all of them in a collection
           public boolean equals(Object other) {
             return other!=null && other.getClass().equals(this.getClass());
           public int hashCode() {
             return 1;
         public class OrderAverageAggregator implements InvocableMap.ParallelAwareAggregator {
           public static final OrderAverageAggregator  INSTANCE = new OrderAverageAggregator();
           public InvocableMap.EntryAggregator getParallelAggregator() {
             return OrderAverageParallelAggregator.INSTANCE;
           public Object aggregateResults(Collection collResults) {
             // join results from all servers
             // the object returned from this method will be returned from the cache.aggregate call
           public Object aggregate(Set setEntries) {
             // the object returned from this method will be returned
             // from the cache.aggregate call if invoked on a replicated cache
             return aggregateResults(
               Collections.singletonList(
                 OrderAverageParallelAggregator.INSTANCE.aggregate(
                   setEntries)));
         public class OrderAverageParallelAggregator implements InvocableMap.EntryAggregator, Serializable {
           public static final OrderAverageParallelAggregator INSTANCE = new OrderAverageParallelAggregator();
         // aggregate method of the parallel aggregator
           public Object aggregate(Set setEntries) {
             Iterator iter = setEntries.iterator();
             while (iter.hasNext()) {
               InvocableMap.Entry entry = iter.next();
               Collection statusObjects = (Collection)  
                 entry.extract(ReportStatusObjectExtractor.INSTANCE);
               // ... do averaging ...
             // the objects returned from this method on each storage-enabled node will be passed in a collection
             // to the aggregateResults method of the OrderAverageAggregator class
             return (Serializable) result;
         // code doing the querying
         NamedCache cache = CacheFactory.getCache(...);
         Object result = cache.aggregate(new ContainsFilter(ReportStatusStringExtractor.INSTANCE, "expectedStatus"),
               OrderAverageAggregator.INSTANCE);
         // code adding the indexes, this needs to run only once after the cluster is started
         NamedCache cache = CacheFactory.getCache(...);
         cache.addIndex(ReportStatusStringExtractor.INSTANCE, false, null);
         cache.addIndex(ReportStatusObjectExtractor.INSTANCE, false, null);
                   The class files for the two extractor classes and the OrderAverageParallelAggregator class must reside in the classpath of the storage-enabled cache JVMs.
     I hope this helps,
     Robert