Querying cache with data that is large enough to fit into available memory

Question

Query on a cache would be applicable on the data that can fit into the cache storage as per the link http://coherence.oracle.com/display/COH35UG/Query+the+Cache. However there is also a mention about restricting the cache content along a specific dimension, so that the query actually knows what to expect from the cache.
Generally any enterprise application would generate huge amount of data and practically it would be difficult to set the fix up the limit to a specific dimension. Fixing the limit is to decide on what else has to be taken from the persistent store.
Please let me know if there are best practices defined in this regard?
Edited by: Mahesh Kamath on Feb 5, 2010 3:16 AM

Answer 1

Hi,
Thanks for the clarification.
Okay. So in a nutshell it sounds like you have data spanning a memory cache and living in a database and
you want to be able to query the entire dataset, so you need someway to figure out what
is in the cache and what is in the database?
In general queries in Coherence only work against the cached data, and because of the LRU characteristics
of memory caches its going to be really difficult to know at a point in time what set of information is living
in memory and which has been evicted and persisted in the underlying database. If the dataset that spills
over into a database is fringe (i.e. a small percentage of your total data), then the simplest solution is to
add new cache servers to get you out of this problem (i.e. avoid databases). If the cache is just the tip
of the iceberg of all your data than you probably want to have either a write-through or write-behind
configuration and query the database directly (write-through will get you everything you put into a cache,
while write-behind will get everything except for the most recent puts which are working their way to being written in batch).
My guess is that retrieval of keys out of the database, and then retrieval of data through the cache using
the keys would not be a good idea. For one thing if the database is substantially larger than the cache,
the entire cache will get evicted by the query activity, which generally is a bad idea. More importantly the
code path to get the data will be much longer and would result in N number of invocations of SQL as opposed
to a single invocation of SQL with a cursor.
Regards,
Bob