Preventing write-coalescing with write-behind

Question

Dear all,
I am very interested in the write-behind feature but I would like to disable the write-coalescing optimization to see each.
Is there any way to do this ?
I feel the fact that CacheStore.storeAll(Map entries) works on cache-key/cache-value makes this impossible to have several cache-value for the same key :-( Not coalescing the consecutive changes on a given entry would require to have a method like CacheStore.storeAll(List<Change>) with Change holding the modification (insert/update/delete), the key and the value.
Thanks,
Cyrille
Cyrille Le Clerc
[email protected]
http://blog.xebia.fr

Answer 1

Thanks for the feedback Robert,
I implemented this "batch processing without coalescing" thanks a "command queue" colocated with my data.
Sample : perform async processing on each modification without coalescence of MyEntity identified by MyEntityKey store in "my-entity-cache".
In my agent/entry-processor, I simultaneously modify my data "MyEntity" and put an entry MyEntityCommand (stored in "my-entity-command-cache"), MyEntityCommand holds enough information to do my async processing. This processing is done asynchronously by MyEntityCommandcacheStore.
MyEntityCommand is associated with a key MyEntityCommandKey which is composed of MyIdentityKey+sequence-number, MyEntityCommandKey has a KeyAssociation with MyEntityKey to ensure colocation.
Benefits :
* There is no coalescence because MyEntityCommandKey contains a unique sequence number.
* The overweight of this "command queue" is limited because the command object only contains that limited piece of data I need for my async processing (foreign key on the entity + few things) and the queue is self purging thanks to a <expiry-delay> on "my-entity-command-cache".
Here is a configuration extract
<distributed-scheme>
  <scheme-name>entity-partitionned</scheme-name>
  <service-name>EntityDistributedCache</service-name>
  <serializer>
  </serializer>
  <backing-map-scheme>
    <local-scheme>
      <scheme-ref>unlimited-backing-map</scheme-ref>
    </local-scheme>
  </backing-map-scheme>
  <thread-count>10</thread-count>
  <autostart>true</autostart>
</distributed-scheme>
<distributed-scheme>
  <scheme-name>entity-command-partitionned</scheme-name>
  <service-name>EntityDistributedCache</service-name>
  <serializer>
  </serializer>
  <backing-map-scheme>
    <read-write-backing-map-scheme>
      <cachestore-scheme>
        <class-scheme>
          <class-name>... EntityCommandStore</class-name>
        </class-scheme>
      </cachestore-scheme>
      <internal-cache-scheme>
        <local-scheme>
          <expiry-delay>30s</expiry-delay>
        </local-scheme>
      </internal-cache-scheme>
      <write-delay>10s</write-delay>
      <write-batch-factor>0.5</write-batch-factor>
    </read-write-backing-map-scheme>
  </backing-map-scheme>
  <thread-count>10</thread-count>
  <autostart>true</autostart>
</distributed-scheme>
{code}
Please note that I had to play with <write-batch-factor> to increase the batching factor (ie the number of entries in each CacheStore.store()/storeAll() invocation). Under very high write load, <write-batch-factor> default value of 0 gave me an average of 1.2 entry per CacheStore.store()/storeAll() invocation. My first try of a 0.5 <write-batch-factor> largely increased this average to probably hundreds (my stats are done on an underlying layer, I don't have the exact average).
Cyrille
Cyrille Le Clerc
[email protected]
http://blog.xebia.fr