Suggestion of a possibly useful feature

Question

Hi Gene and Dimitri,
     in our project we face a problem that we should try to find out a subset of tree nodes in a tree hierarchy filtered by transitive parent-child relationship to a specific node.
     The amount of nodes are possibly in the couple of hundred thousands, so probably they would not economically fit into a single JVM, so probably it would need to be stored in a partitioned cache. Also the tree hierarchy occasionally changes (it is a normal use case).
     I expect it could be useful if the children of each non-leaf node could be maintained in a data structure on each cache client node, effectively a reverse index of all entries in the cache based on the parent id attribute value as extracted value.
     However, for this, it would be necessary that I can quickly and consistently build the reverse index upon startup.
     The 3 parameter constructor of the ContinuousQueryCache gives an opportunity to do this, however it would also mean an enormous overhead because it would also store each entry in the JVM, which as it was already mentioned would be quite disastrous to memory footprint, not to mention it is not required anyway.
     A quick first solution to this problem would be if the ContinousQueryCache backing map could be disabled, but it would still act as an ObservableMap. This way the memory footprint overhead would be much less, after the handling of the initial events, but we would still be able receive exactly one event per entry and per subsequent events.
     Another improvement would be allowed if at the construction of the ContinousQueryCache, the events corresponding to the entries returned by the initial query result could be delivered together, not one-by-one, as it would allow optimizing the index structure lists memory consumption by not having to gradually increase the collections holding the reverse mapped keys to an extracted value, but we could create it with the correct size (and some spare space) at once. However this would require two passes on the events corresponding to the initial query, therefore they would need to be delivered together to allow two passes.
     Practically it is even worth extracting this functionality alone (initial list of events or values corresponding to entries in the cache delivered together to a synchronous-listener with all following events delivered normally) to a helper class which can create a listener initializable as mentioned without any functionality additionally provided by the ContinousQueryCache.
     If such a feature is created, then it would be easy to implement arbitrary custom (userspace) indexes remotely to the cache backing maps, or allow indexing replicated caches.
     Is indexing replicated caches actually scheduled as a feature anytime soon?
     A third possible optimization would be if the initial query would not return entire entries but would run an user specifiable aggregation instead, which can return only the relevant extracted attributes. This would allow reducing the initial traffic and also possibly allow leveraging existing indexes in the cache servers therefore bypassing deserialization of the entries altogether.
     Best regards,
     Robert

Answer 1

Hi Peter,
     > Hi Rob,
     >
     > I have managed to create a wrapper over any
     > NamedCache that maintains it's own local indexes and
     > optimizes Queries when an IndexAwareFilter is used.
     > Using only one listener registered for listening to
     > "light" events I just retrieve the key from MapEvent
     > (I'm not even interested in the type of event) and
     > then look-up for what the underlying cache says about
     > the key (whether it is currently deleted or whether
     > it has some value associated) and update the indexes
     > accordingly.
     You should be aware, that the listener must NOT call back to the same cache service that it received the event from, otherwise you are subjecting yourself to deadlocks under heavy load. Therefore you should not be looking up cached values from a listener.
     Also, the fact that you are doing a lookup means that your index is updated asynchronously to the change event. This means that the cache content is not guaranteed to always be in sync with your indexes.
     E.g. when your processing is slow and you have two change events for the same entry, and your lookup sees only the effect of the second change, then your index will not reflect the intermittent existence of the value after the first change, therefore it was not in sync with the cache when it contained the value received in the first change.
     This can make a problem, when you are doing queries for which not all filters are indexed, and therefore some of the filters are checking the index and some are checking the values. They might be inconsistent and lead to false results.
     For queries like that, you must check the values in a state as they were in when the index was created from them. This means that you must be able to provide a snapshot of entries which is consistent with the snapshot of the indexes the filter will see (and of course the indexes must also be in a consistent state with each other). For two changes to the same entry, going from state A through state B to state C, the filter must see index contents (let's assume two indexes) the indexes and the entry either all in state A or all in state B or all in state C together, but not in a mixed-up state.
     This is quite complex logic, and I don't believe this is possible to do with index updating being asynchronous to the changes.
     Also, there is no point in using lite events and getting from the cache in a replicated cache, as the data has already traveled to your node when you received the event, then why do another lookup instead of getting it at once, except for seeing the latest version, which as I mentioned leads to inconsistencies.
     > The initial loading of the indexes is done after the
     > "light" listener is registered and it is simply a
     > mater of iterating over all keys of the map and
     > producing a stream of key-only events that are
     > processed in parallel with events that are produced
     > by the ObservableMap. The order of these events is
     > not important since they only specify "something has
     > changed under that key". If I get two events for the
     > same change/initial load it doesn't matter since they
     > are idempotent.
     Please see my previous comment about the consistency of the index with the cached values.
     > In each index I maintain a map from extracted value
     > to the set of keys (which is the index content) and
     > also a reverse map from key to the extracted value
     > (which is important to be able to update the index
     > when I only have current value and not the old one).
     >
     Coherence terminology is actually just the other way round.
     Reverse map is mapping from the extracted value to the cache keys from which the extracted value is extracted from (and this is called index content in IndexAwareFilter-s).
     Forward map is mapping from the cache key to the extracted value (you can use this by casting the entry to QueryMap.Entry in an index-aware filter or a parallel-aggregator, and calling QueryMap.Entry.extract(ValueExtractor) which will consult an index forward map if it exists.
     > It works perfectly for replicated cache.
     >
     Have you tried stress testing it under high load, when querying for data when you were actually changing that data, and making changes on multiple attributes in a single put, with some of the changed attributes indexed, and some of them are not, and verifying for not getting false results in the query result set?
     Due to the possibility of the cache not being in sync with the index, I would believe, those race conditions would expose errors (or deadlocks).
     Best regards,
     Robert