HibernateCacheStore and cache backup

Question

Hi all,
Our scenario is a bit like this:
1) two node distributed cluster
2) a third machine is a database server
What I'm trying to achieve is the have the database server act as a backup for the cache so that in the event that both nodes go down, the cache can be restored to the point just prior to the failure. I've been experimenting by modifying the examples (http://wiki.tangosol.com/download/attachments/16730/hibernate-sample-src.jar), here is my configuration:
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
     <!--
          Sample configuration file for Coherence Hibernate CacheStore See the
          following links for more documentation:
          http://wiki.tangosol.com/display/COH33UG/Read-Through%2C+Write-Through%2C+Refresh-Ahead+and+Write-Behind+Caching
          http://wiki.tangosol.com/display/COH33UG/Using+Coherence+and+Hibernate
     -->
<cache-config>
     <caching-scheme-mapping>
          <cache-mapping>
               <cache-name>*</cache-name>
               <scheme-name>distributed-hibernate</scheme-name>
          </cache-mapping>
     </caching-scheme-mapping>
     <caching-schemes>
          <distributed-scheme>
               <scheme-name>distributed-hibernate</scheme-name>
               <backing-map-scheme>
                    <read-write-backing-map-scheme>
                         <internal-cache-scheme>
                              <local-scheme></local-scheme>
                         </internal-cache-scheme>
                         <cachestore-scheme>
                              <class-scheme>
                                   <class-name>com.tangosol.coherence.hibernate.HibernateCacheStore</class-name>
                                   <init-params>
                                        <init-param>
                                             <param-type>java.lang.String</param-type>
                                             <param-value>{cache-name}</param-value>
                                        </init-param>
                                   </init-params>
                              </class-scheme>
                         </cachestore-scheme>
                         <!--
                              Set this to 0 to make writes synchronous, > 1 to make writes
                              asynchronous
                         -->
                         <write-delay>1</write-delay>
                    </read-write-backing-map-scheme>
               </backing-map-scheme>
          </distributed-scheme>
     </caching-schemes>
</cache-config>I've written some simple code to populate the cache from the database:
          LOG.info("loading...");
          HibernateCacheLoader loader = new HibernateCacheLoader(SampleEntity.class.getName());
          Session session = loader.getSessionFactory().openSession();
          int start = 0;
          long total = 0;
          while(true) {
               Stopwatch batchStopwatch = new Stopwatch();
               batchStopwatch.start();
               Query query = session.createQuery("SELECT id FROM SampleEntity");
               query = query.setFirstResult(start);
               query = query.setMaxResults(BUFFER_SIZE);          
               query = query.setFetchSize(BUFFER_SIZE);                              
               List<String> keys = query.list();          // get keys from DB
               if (keys.size() == 0) break;
               cache.putAll(loader.loadAll(keys));          // use CacheLoader to load
               batchStopwatch.stop();
               total += keys.size();
               LOG.info("loaded: "+(start+keys.size())+", "+batchStopwatch.toString());
               start += BUFFER_SIZE;
          stopwatch.stop();
          LOG.info("done, total loaded = "+total+", "+stopwatch.toString());
          LOG.info("average load speed: "+(total/stopwatch.getSeconds())+" rows/sec");I've also been creating dummy data as per the example:
          // Seed the cache/database with sample values
          for (int i = 0; i < ROWS; i++) {
               SampleEntity se = new SampleEntity();
               se.setId(UUID.randomUUID().toString());
               se.setName(getRandomString());
               se.setDate(new Timestamp(System.currentTimeMillis()));
               se.setAge(i);
               dao.saveSampleEntity(se);
               if (i%1000 == 0) LOG.info("saved "+i);
          }So, several questions I have:
1) Have I taken the correct approach to achieve my goals? Is there a better way?
2) I would like to use <write-delay> to enable async writing to the DB to lower the wait time of hitting the cache - however, as I have found in the DB populator application I need to give the application time to clear the queue (ie, the DB populator program quits very quickly when writing asynchronously, and as a result not all records are written to the database). Given that I need to account for hardware failures to the cluster which means the queue may not be cleared, am I stuck with using synchronous writes, and hence each write/update to the cache will result in a database-hit length wait time for the user?
Edited by: user13362605 on Jul 20, 2010 7:48 PM

Answer 1

Hi,
In the Coherence console there is a way to do this with CohQL - see here http://download.oracle.com/docs/cd/E15357_01/coh.360/e15723/api_cq.htm#CEGBCJHD
I have never used this but I suspect what this will do is stream the contents of you cache through the CohQL node to disk. This sounds fine at first but I can imagine it being slow for very large caches.
We recently had a similar requirement for the project I work on but the number one feature was that it had to be fast. In the end we could dump a 100GB cluster to disk in about a minute and load it back in about five minutes. The reason it is fast is because the write/read happens in parallel across all the storage nodes. We just stream the Binary keys and values straight from the backing maps to disk and on reload stream the Binary back in again.
It was complicated by the fact that we have triggers on our caches that we needed to bypass when reloading. We also needed to bypass cache stores - but only for data that had not already been written by any write-behind cache stores.
There are probably other ways of doing what you want, but as I said our number one requirement was speed. At the time on our project reloading from the DB took about 2 hours, so especially in Test environments where we were redeploying a cluster quite often over 2 hours turn-around was not good - and rolling restart of a clusters between 150 - 400 nodes takes even longer.
JK