Write-through Cache behavior during Transactional Operation

If a put is called on a write-through cache during a transaction(with Optimistic Read-Committed settings) that involves multiple caches some set to write-through and others to write-behind, when will the store operation on the corresponding CacheStore be attempted?
     a) Immediately after the put() is called on the cache but before the transaction commit
     or
     b) Immediately after the transaction is committed irrespective of when the put is called

Hi Abhay,
     The backing map (in this case, <tt>com.tangosol.net.cache.ReadWriteBackingMap</tt>) is responsible for calling the CacheStore implementation. When "commit" is called, Coherence will synchronously send the data to the backing map; the backing map then determines what to do with the data. In the case of ReadWriteBackingMap, it will either (depending on its configuration) synchronously call CacheStore (meaning that a store exception will interrupt your transaction) or queue the update for later (meaning that any store exception will occur after the cache transaction has completed).
     In 3.0, the <rollback-cachestore-failures> element under <read-write-backing-map-scheme> controls whether CacheStore exceptions are propagated back to the client. If you are using a release prior to 3.0, please see this FAQ Item on CacheStore Exceptions.
     Jon Purdy
     Tangosol, Inc.

Similar Messages

  • Pre-load a write-through cache

    This probably is a common problem.
    I want to use write-through cache to keep the cache and the db in sync. But at application start-up, I'd like to pre-load the cache. Is there anyway I can disable the write-through behavior during pre-loading, and enable it after pre-loading is complete?

    Wouldn't you just be better to load the data into the database first, and then just query it to get it into the cache? That way, the cache and the db are 'in sync', and no write operations have occurred; also, you don't need to 'flip' any cache settings. Subsequent updates to cache items would then be written to the database as normal.
    Surely if the data wasn't in the db first, you'd end up with a large 'write' operation to place it there once you've loaded the cache and 'flipped' it over, which could be quite a lengthy process. I can't see the advantage of this over putting the data in the db first?
    I'd be interested to know more about your specific use-case, as I'm just to embark on a similar 'loader' program. In my case, I was planning on loading the db first. I'd be interested in any alternative approaches and the reasoning behind them (or, likewise, if you can't actually do it the way I was planning! :)).
    Steve

  • Write-through cache error kills coherence

    Hi, we have a write through cache, and when there's an error writing to the cache store, the cache dies. The first error we see is:
    3.5.3/465 | Coherence(3) - 2012-07-19 07:28:35.918/34.768 Oracle Coherence GE 3.5.3/465 <Info> (thread=DistributedWriteThroughWorker:3, member=1): (Wrapped: Failed to store key="1342708115777") java.lang.RuntimeException: Failed to store hibernate entity : This is normal, since there's a legitimate reason why this couldn't get stored, but after the stack trace, we see the below. So the 1st thing is the "Terminating DistributedCache", that just kills the node and makes it unusable, then there's the "unknown user type" message, as if it's trying to send something over the wire, but it can't; although this "WrapperException" is not one of our classes. Clearly we don't want the cache dying, we want to continue to use it, but any subsequent request to the cache fails. Any ideas?
         [java] ERROR | 07-19-2012 07:28:36.116 | Logger@9230760 3.5.3/465 | Coherence(3) - 2012-07-19 07:28:35.919/34.769 Oracle Coherence GE 3.5.3/465 <Error> (thread=DistributedWriteThroughWorker:3, member=1): Terminating DistributedCache due to unhandled exception: java.lang.IllegalArgumentException
         [java] ERROR | 07-19-2012 07:28:36.117 | Logger@9230760 3.5.3/465 | Coherence(3) - 2012-07-19 07:28:35.919/34.769 Oracle Coherence GE 3.5.3/465 <Error> (thread=DistributedWriteThroughWorker:3, member=1):
         [java] java.lang.IllegalArgumentException: unknown user type: com.tangosol.util.WrapperException
         [java]      at com.tangosol.io.pof.ConfigurablePofContext.getUserTypeIdentifier(ConfigurablePofContext.java:400)
         [java]      at com.tangosol.io.pof.ConfigurablePofContext.getUserTypeIdentifier(ConfigurablePofContext.java:389)
         [java]      at com.tangosol.io.pof.PofBufferWriter.writeObject(PofBufferWriter.java:1432)
         [java]      at com.tangosol.io.pof.ConfigurablePofContext.serialize(ConfigurablePofContext.java:338)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.writeObject(Service.CDB:4)
         [java]      at com.tangosol.coherence.component.net.Message.writeObject(Message.CDB:1)
         [java]      at com.tangosol.coherence.component.net.message.DistributedCacheResponse.write(DistributedCacheResponse.CDB:2)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher.packetizeMessage(PacketPublisher.CDB:137)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher$InQueue.add(PacketPublisher.CDB:8)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.dispatchMessage(Grid.CDB:50)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.post(Grid.CDB:53)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache.onPutRequest(DistributedCache.CDB:146)
         [java]      at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache$PutRequest.run(DistributedCache.CDB:1)
         [java]      at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:1)
         [java]      at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:32)
         [java]      at com.tangosol.coherence.component.util.DaemonPool$Daemon.onNotify(DaemonPool.CDB:63)
         [java]      at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
         [java]      at java.lang.Thread.run(Thread.java:662)

    Ok, figured it out. I failed to include the standard "coherence-pof-config.xml" when loading in our pof config file, and that's what caused the troubles. Including it solved the problem of the dying cache.

  • Several processes write to the SGA during database operation.

    Greeting to all ;
    From following official document  says  Memory Architecture
    Several processes write to the SGA during database operation.
      I thinks DBWn writes updated information from dbbc to data files.
       Server process is fetching required information from data files.
    Several processes write to the SGA during database operation !
    - What are they ? 

    8f953842-815b-4d8c-833d-f2a3dd51e602 wrote:
    Thanks sybrand_b
    Several processes write to the SGA during database operation.
    So your answer is  client process for DML opearations. Ok , i got it.
    From the same link ,  Memory Architecture
    The server and background processes do NOT reside within the SGA, but exist in a separate memory space.
            Partially i can understand above statement. Here  i have little confusion also - please clarify.
    SGA + background process = INSTANCE.
    Server process is only available when user is intracting with database.
    From official document , bgprocess , server process exist in a separate memory space  - 
    -  if so ,  what is  name of the  memory area ?
        Thanks in advance ..
    Which memory area?  The one holding the SGA?  The one holding the executing code for any one of the background processes?  The one holding the code for any of the several server processes?
    Go here: Oracle Database Instance
    and take a look at figure 13-1 and its surrounding discussion.

  • Re: Write-through caching in Forte

    Hello Mark,
    Just one point more. May be you can add an Event Notifier to the lock
    manager to send the new instance of Obj1 to the clients (here client2)
    who use it in their cache.
    Hope this helps.
    Daniel Nguyen.
    Mark S. Potts wrote:
    >
    Andrew
    This is a mixture of a cache strategy and object locking. If I
    understand what you have said I have some suggestions;
    The cache should hold copies of the object and the object should be
    returned to the client. The obect that is returned to the client should
    be version stamped ( optimistic locking ).
    A) Client1 request Obj1
    B) Obj1 is instantiated from the persistent store
    C) Obj1 is version stamped via a lock manager service.
    D) Obj1 is placed in the cache and copy returned to Client1
    Client1 can now work on Obj1
    When Client2 selects Obj2 - the cache size being 1 - the Obj1 is
    replaced with Obj2.
    Obj2 is selected stamped and returned to the client as per the steps
    above.
    When Client 2 now selects Obj1, no longer in the cache, the same steps
    need to be completed as above.
    The cache now contains the same version of Obj1 as give to Client1.
    Now the important part, becuase this is an optimistic locking strategy -
    two clients can have different version of the same object, it is only
    when the object is saved - returned to the persistent store, that the
    version stamp need to be checked. Lets say Client 2 saves before client
    1
    A) Client2 initiates a save on Obj1
    B) Obj1 checks the lock manger to see if anyone has saved a new version
    of Obj1 since it was selected.
    C) If there have been no saves of Obj1 since Obj1 was selected ie the
    version of Obj1 selected does not conflict with the last version saved -
    then save Obj1
    D) Update the version stamp for Obj1 via the Lock manager
    E) Update Obj1 in the cache.
    When Client1 now tries to save the version of Obj1 a conflict will
    result and an exception needs to be raised - and if necessary the new
    version of Obj1, from the cache, returned to Client1.
    The version control can be done more easily if you are prepared to do
    the locking in the database - I do not recommend this for a number of
    well documented reasons.
    However if you choose this alternative instead of using a seperate Lock
    manager you could simply time stamp the row in the database iether on
    that table or a separate lock table and when saving the Obj1 check the
    time stamp on the object against the time stamp on the row. If they are
    the same save the object and update the time stamp to the current time (
    granularity of time stamp determined by number of concurrent users and
    usage patterns ). The time stamp on the row acts as the version stamp
    for the object and is selected into the object as a private attribute at
    time of selection.
    Hope this is of some help.
    Mark Potts
    SCAFFOLDS Product Manager
    Sage IT Partners
    A) Client1 requests Obj1.
    B) Obj1 is instantiated from a persistent store and placed in the cache
    and a reference to Obj1 is
    returned to Client1.
    C) As part of the instantiation of Obj1 the object is version stamped
    through a lock manager service.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2 the
    reference to the Obj1 that Client1 has.
    Faibishenko, Andrew wrote:
    Has anyone out there been successful at implementing a cache which
    maintains updateable objects.
    Due to financial considerations, we cannot buy an off-the-shelf
    framework.
    What we are trying to build is some kind of object persistence
    mechanism
    and the cache would be a layer in that service.
    Our big issue is maintaining consistency within the cache, for
    multiple
    clients performing updates to an object.
    Example:
    A) Client1 requests Obj1.
    B) Obj1 is de-serialized, placed in the cache and a reference to Obj1
    is
    returned to Client1.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and
    a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we
    either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2
    the
    reference to the Obj1 that Client1 has.
    Is this something we should ask Forte Consulting about?
    -Andy
    ============================================
    Andy Faibishenko (312)251-3267
    Senior Consultant (800)462-6301
    Metamor Technologies, Inc. [email protected]

    Hello Mark,
    Just one point more. May be you can add an Event Notifier to the lock
    manager to send the new instance of Obj1 to the clients (here client2)
    who use it in their cache.
    Hope this helps.
    Daniel Nguyen.
    Mark S. Potts wrote:
    >
    Andrew
    This is a mixture of a cache strategy and object locking. If I
    understand what you have said I have some suggestions;
    The cache should hold copies of the object and the object should be
    returned to the client. The obect that is returned to the client should
    be version stamped ( optimistic locking ).
    A) Client1 request Obj1
    B) Obj1 is instantiated from the persistent store
    C) Obj1 is version stamped via a lock manager service.
    D) Obj1 is placed in the cache and copy returned to Client1
    Client1 can now work on Obj1
    When Client2 selects Obj2 - the cache size being 1 - the Obj1 is
    replaced with Obj2.
    Obj2 is selected stamped and returned to the client as per the steps
    above.
    When Client 2 now selects Obj1, no longer in the cache, the same steps
    need to be completed as above.
    The cache now contains the same version of Obj1 as give to Client1.
    Now the important part, becuase this is an optimistic locking strategy -
    two clients can have different version of the same object, it is only
    when the object is saved - returned to the persistent store, that the
    version stamp need to be checked. Lets say Client 2 saves before client
    1
    A) Client2 initiates a save on Obj1
    B) Obj1 checks the lock manger to see if anyone has saved a new version
    of Obj1 since it was selected.
    C) If there have been no saves of Obj1 since Obj1 was selected ie the
    version of Obj1 selected does not conflict with the last version saved -
    then save Obj1
    D) Update the version stamp for Obj1 via the Lock manager
    E) Update Obj1 in the cache.
    When Client1 now tries to save the version of Obj1 a conflict will
    result and an exception needs to be raised - and if necessary the new
    version of Obj1, from the cache, returned to Client1.
    The version control can be done more easily if you are prepared to do
    the locking in the database - I do not recommend this for a number of
    well documented reasons.
    However if you choose this alternative instead of using a seperate Lock
    manager you could simply time stamp the row in the database iether on
    that table or a separate lock table and when saving the Obj1 check the
    time stamp on the object against the time stamp on the row. If they are
    the same save the object and update the time stamp to the current time (
    granularity of time stamp determined by number of concurrent users and
    usage patterns ). The time stamp on the row acts as the version stamp
    for the object and is selected into the object as a private attribute at
    time of selection.
    Hope this is of some help.
    Mark Potts
    SCAFFOLDS Product Manager
    Sage IT Partners
    A) Client1 requests Obj1.
    B) Obj1 is instantiated from a persistent store and placed in the cache
    and a reference to Obj1 is
    returned to Client1.
    C) As part of the instantiation of Obj1 the object is version stamped
    through a lock manager service.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2 the
    reference to the Obj1 that Client1 has.
    Faibishenko, Andrew wrote:
    Has anyone out there been successful at implementing a cache which
    maintains updateable objects.
    Due to financial considerations, we cannot buy an off-the-shelf
    framework.
    What we are trying to build is some kind of object persistence
    mechanism
    and the cache would be a layer in that service.
    Our big issue is maintaining consistency within the cache, for
    multiple
    clients performing updates to an object.
    Example:
    A) Client1 requests Obj1.
    B) Obj1 is de-serialized, placed in the cache and a reference to Obj1
    is
    returned to Client1.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and
    a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we
    either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2
    the
    reference to the Obj1 that Client1 has.
    Is this something we should ask Forte Consulting about?
    -Andy
    ============================================
    Andy Faibishenko (312)251-3267
    Senior Consultant (800)462-6301
    Metamor Technologies, Inc. [email protected]

  • Write-through caching in Forte

    Has anyone out there been successful at implementing a cache which
    maintains updateable objects.
    Due to financial considerations, we cannot buy an off-the-shelf
    framework.
    What we are trying to build is some kind of object persistence mechanism
    and the cache would be a layer in that service.
    Our big issue is maintaining consistency within the cache, for multiple
    clients performing updates to an object.
    Example:
    A) Client1 requests Obj1.
    B) Obj1 is de-serialized, placed in the cache and a reference to Obj1 is
    returned to Client1.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2 the
    reference to the Obj1 that Client1 has.
    Is this something we should ask Forte Consulting about?
    -Andy
    ============================================
    Andy Faibishenko (312)251-3267
    Senior Consultant (800)462-6301
    Metamor Technologies, Inc. [email protected]

    Andrew
    This is a mixture of a cache strategy and object locking. If I
    understand what you have said I have some suggestions;
    The cache should hold copies of the object and the object should be
    returned to the client. The obect that is returned to the client should
    be version stamped ( optimistic locking ).
    A) Client1 request Obj1
    B) Obj1 is instantiated from the persistent store
    C) Obj1 is version stamped via a lock manager service.
    D) Obj1 is placed in the cache and copy returned to Client1
    Client1 can now work on Obj1
    When Client2 selects Obj2 - the cache size being 1 - the Obj1 is
    replaced with Obj2.
    Obj2 is selected stamped and returned to the client as per the steps
    above.
    When Client 2 now selects Obj1, no longer in the cache, the same steps
    need to be completed as above.
    The cache now contains the same version of Obj1 as give to Client1.
    Now the important part, becuase this is an optimistic locking strategy -
    two clients can have different version of the same object, it is only
    when the object is saved - returned to the persistent store, that the
    version stamp need to be checked. Lets say Client 2 saves before client
    1
    A) Client2 initiates a save on Obj1
    B) Obj1 checks the lock manger to see if anyone has saved a new version
    of Obj1 since it was selected.
    C) If there have been no saves of Obj1 since Obj1 was selected ie the
    version of Obj1 selected does not conflict with the last version saved -
    then save Obj1
    D) Update the version stamp for Obj1 via the Lock manager
    E) Update Obj1 in the cache.
    When Client1 now tries to save the version of Obj1 a conflict will
    result and an exception needs to be raised - and if necessary the new
    version of Obj1, from the cache, returned to Client1.
    The version control can be done more easily if you are prepared to do
    the locking in the database - I do not recommend this for a number of
    well documented reasons.
    However if you choose this alternative instead of using a seperate Lock
    manager you could simply time stamp the row in the database iether on
    that table or a separate lock table and when saving the Obj1 check the
    time stamp on the object against the time stamp on the row. If they are
    the same save the object and update the time stamp to the current time (
    granularity of time stamp determined by number of concurrent users and
    usage patterns ). The time stamp on the row acts as the version stamp
    for the object and is selected into the object as a private attribute at
    time of selection.
    Hope this is of some help.
    Mark Potts
    SCAFFOLDS Product Manager
    Sage IT Partners
    A) Client1 requests Obj1.
    B) Obj1 is instantiated from a persistent store and placed in the cache
    and a reference to Obj1 is
    returned to Client1.
    C) As part of the instantiation of Obj1 the object is version stamped
    through a lock manager service.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2 the
    reference to the Obj1 that Client1 has.
    Faibishenko, Andrew wrote:
    Has anyone out there been successful at implementing a cache which
    maintains updateable objects.
    Due to financial considerations, we cannot buy an off-the-shelf
    framework.
    What we are trying to build is some kind of object persistence
    mechanism
    and the cache would be a layer in that service.
    Our big issue is maintaining consistency within the cache, for
    multiple
    clients performing updates to an object.
    Example:
    A) Client1 requests Obj1.
    B) Obj1 is de-serialized, placed in the cache and a reference to Obj1
    is
    returned to Client1.
    C) Client1 modifies the state of Obj1 trough its reference.
    D) Client2 requests Obj2.
    E) Obj2 is de-serialized, placed in the cache, knocking out Obj1, and
    a
    reference to Obj2 is returned to Client2.
    F) Client2 requests Obj1. Since it is no longer in the cache, we
    either
    need to de-serialize Obj1 from some persistent store, in which case we
    now have two out of sync copies of Obj1, or we need to give Client2
    the
    reference to the Obj1 that Client1 has.
    Is this something we should ask Forte Consulting about?
    -Andy
    ============================================
    Andy Faibishenko (312)251-3267
    Senior Consultant (800)462-6301
    Metamor Technologies, Inc. [email protected]

  • Switching from write through to write behind automatically

    Hi,
    We are considering a Coherence solution to protect a customer facing application from outages due to database failures. This is for a financial company and the monetary value of each transaction is large and we want to provide 100% guarantee against data loss while not incurring any outages. We want to provide a write-through persistence to the database through Coherence which can switch to a write-behind automatically at runtime if the database persistence fails. Is this doable automatically and would it solve the problem I am trying to solve without losing any inflight transactions? Are there any real customer cases that were successful in achieving this using Coherence?
    Thanks
    Sairam
    Edited by: SKR on Feb 16, 2012 3:14 PM
    Edited by: SKR on Feb 16, 2012 3:15 PM

    SKR wrote:
    Jonathan.Knight wrote:
    Hi Sairam
    I know you can change the write-delay in JMX for a cache using write-behind but I pretty certauin you cannot make a write-through cache suddenly become a write-behind cache.
    I'm not sure why you think changing from write-through to write-behind will allow you to guarantee 100% no data loss - do you mean no loss of updates to the DB or no loss of data in the cache cluster? There are certainly scenarios that can occur where you can loose data from either the cluster or the DB that write-through or write-behind will not save you from. Presumably you want to use write-behind to allow for the DB to go down, although you will still need to configure Coherence to properly retry failed write-behind calls CacheStore behaviour on failure. What happens to your data if you are using write-behind and you loose a partition from you cluster (i.e. you loose a physical machine or two or more JVMs in a short space of time) - you have data loss - you cannot guarantee against this you can only mitigate it and have a recovery policy/procedure.
    JKJK,
    Thanks for your reply. I must have explained the scenario better. What we are trying to do is to have our transactions commit to the database synchronously using write-through, so that during normal operation, the data will be committed, persisted and durable in the database. But our RW database becomes a single point of failure and if some problem occurs to the database during the peak load time, we run the risk of an outage till we fix the database problem or failover to the standby (We don't have RAC architecture or automatic failover and the manual switchover takes about 10 - 15 mins minimum). We want to avoid this by providing a cache-only operation mode during such a failure, where the customers can continue to transact and the writes will get queued in the cache. I do understand that losing both the database and the cache or losing the primary and the backup in the cache would result in a data loss. But I am assuming such a dual failure is rare.
    We do not want to run write-behind all the time but only during the database failure window. From what you mentioned, it seems the runtime switching from write-through to write-behind is not available as an option.
    SairamHi Sairam,
    I would suggest that you configure write-behind to have a fairly short write-delay, and you only return a confirmation to the client
    - either after the write-behind succeeded (you can use a backing map listener to listen for the removal of the decoration which meant that the entry was dirty)
    - or if the database went down (noticeable from the failure), then it is up to you whether you send a confirmation which also mentions that it is not persisted to disk yet, or not at all
    Best regards,
    Robert

  • Transactional Caches and Write Through

    I've been trying to implement the use of multiple caches, each with write through, all within a transaction.
         The CacheFactory.commitTransactionCollection(..) method only seems to work correctly if the first transactionMap throws an exception in the database code.
         If the second transactionMap throws exceptions, the caches do not appear to rollback correctly.
         I can wrap the whole operation in a JDBC transaction that rolls back the database correctly but the caches are not all rolled back because they are committed one by one?
         For example, I write to two transaction maps, each one created from separate caches. When commiting the transaction maps, the second transaction map causes a database exception. It appears the first transaction map has already committed its objects and doesn't roll back.
         Is it possible to use Coherence with multiple transaction maps and get all the caches and databases rolled back?
         I've also been trying to look at using coherence-tx.rar as described in the forums within WebLogic but I'm getting @@@@@ Failed to commit: javax.transaction.SystemException: Could not contact coordinator at null+SMARTPC:7001+null+t3+
         (SMARTPC being my pc name)
         Has anybody else had this problem? Bonus points for describing how to fix it!
         Mike

    The transaction support in Coherence is for Local     > Transactions. Basically, what this means is that the
         > first phase of the commit ("prepare") acquires locks
         > and ensures that there are no conflicts. The second
         > phase ("commit") does nothing but push data out to
         > the caches.
         This means that once prepare succeeds (all locks acquired), commit will try to copy local data into the base map. If there is a failure on any put, rollback will undo any changes made. All locks are cleared at the end.
         > The problem is that when you are using a
         > CacheStore module, the exception is occurring during
         > the second phase.
         If you start using a CacheStore module, then database update has to be part of the atomic procedure.
         >
         > For this reason, write-through and cache transactions
         > are not a supported combination.
         This is not true for a cache transaction that updaets a single cache entry, right?
         >
         > For single-cache-entry updates, CacheStore operations
         > are fully fault-tolerant in that the cache and
         > database are guaranteed to be consistent during any
         > server failure (including failures during partial
         > updates). While the mechanisms for fault-tolerance
         > vary, this is true for both write-through and
         > write-behind caches.
         For the write-thru case, I believe Database and cache are atomically updated.
         > Coherence does not support two-phase CacheStore
         > operations across multiple CacheStore instances. In
         > other words, if two cache entries are updated,
         > triggering calls to CacheStore modules sitting on
         > separate servers, it is possible for one database
         > update to succeed and for the other to fail.
         But once we have multiple CacheStore modules, then once one atomic write-thru put succeeds that means database is already updated for that specific put. There is no way to roll back the database update (although we can roll back the cache update). Therefore, you may end up in partial commits in such situations where multiple cache entries are updated across different CacheStore modules.
         If I use write-behind CacheStore modules, I can roll back entirely and avoid partial commits? Since writes are not immediately propagated to the database? So in essence, write-behind cache stores are no different than local transactions... Is my understanding correct?

  • Write-Behind batch behavior in EP partition level transactions

    Hi,
    We use EntryProcessors to perform updates on multiple entities stored in the same cache partition. According to the documentation, Coherence handles all the updates in a "sandbox" and then commits them atomically to the cache backing map.
    The question is, when using write-behind, does Coherence guarantee that all entries updated in the same "partition level transaction" will be present in the same "storeAll" operation?
    Again, according to the documentation, the write-behind thread behavior is the following:
    The thread waits for a queued entry to become ripe.
    When an entry becomes ripe, the thread dequeues all ripe and soft-ripe entries in the queue.
    The thread then writes all ripe and soft-ripe entries either via store() (if there is only the single ripe entry) or storeAll() (if there are multiple ripe/soft-ripe entries).
    The thread then repeats (1).
    If all entries updated in the same partition level transaction become ripe or soft-ripe at the same instant they will all be present in the storeAll operation. If they do not become ripe/soft-ripe in the same instant, they may not be all present.
    So, it all depends on the behavior of the commit of the partition level transaction, if all entries get the same update timestamp, they will all become ripe at the same time.
    Does anyone know what is the behavior we can expect regarding this issue?
    Thanks.

    Hi,
    That comment is still correct for 12.1 and 3.7.1.10.
    I've checked Coherence APIs and the ReadWriteBackingMap behavior, and although partition level transactions are atomic, the updated entries will be added one by one to the write behind queue. In each added entry coherence uses current time to calculate when each entry will become ripe, so, there is no guarantee that all entries in the same partition level transaction will become ripe at the same time.
    This leads me to another question.
    We have a use case where we want to split a large entity we are storing in coherence into several smaller fragments. We use EntryProcessors and partition level transactions to guarantee atomicity in operations that need to update more than one fragment of the same entity. This guarantees that all fragments of the same entity are fully consistent. The cached fragments are then persisted into database using write-behind.
    The problem now is how to guarantee that all fragments are fully consistent in the database. If we just relly on coherence write-behind mecanism we will have eventual consistency in DB, but in case of multi-server failure the entity may become inconsistent in database, which is a risk we wouldnt like to take.
    Is there any other option/pattern that would allow us to either store all updates done on the entity or no update at all?
    Probably if in the EntryProcessor we identify which entities were updated and if we place them in another persistency queue as a whole, we will be able to achieve this, but this is a kind of tricky workaround that we wouldnt like to use.
    Thanks.

  • Thread pool configuration for write-behind cache store operation?

    Hi,
    Does Coherence have a thread pool configuration for the Coherence CacheStore operation?
    Or the CacheStore implementation needs to do that?
    We're using write-behind and want to use multiple threads to speed up the store operation (storeAll()...)
    Thanks in advance for your help.

    user621063 wrote:
    Hi,
    Does Coherence have a thread pool configuration for the Coherence CacheStore operation?
    Or the CacheStore implementation needs to do that?
    We're using write-behind and want to use multiple threads to speed up the store operation (storeAll()...)
    Thanks in advance for your help.Hi,
    read/write-through operations are carried out on the worker thread (so if you configured a thread-pool for the service the same thread-pool will be used for the cache-store operation).
    for write-behind/read-ahead operations, there is a single dedicated thread per cache above whatever thread-pool is configured, except for remove operations which are synchronous and still carried out on the worker thread (see above).
    All above is of course per storage node.
    Best regards,
    Robert

  • Should OS/FileSystem caching be write-through?

    I have a question. I use Ubuntu. Should I mount my filesystem (which holds BDB's content) with "-o sync" option? That is, should my file system cache be write-through?
    I have this question because, if I turn on the logging feature in Berkeley DB but let the file system cache be write-back, I don't exactly know if the log is properly flushed to the disk or not.

    Thanks George. I agree that mature applications would be better off mounting their filesystem with "-o sync" option.
    But here is a thing: I ran an example test case where I inserted 10 million key-value pairs with logging enabled, and saw that the average response time per insertion was 10 milli seconds, and I did the same experiment with logging disabled and saw that it too took 10 milliseconds per insertion on an average.
    For the experiment with logging enabled, I create the environment with DB_INIT_LOG and DB_INIT_TXN flags but don't surround the insertion requests with txn_begin() and txn->commit(). I guess this way of doing insertions is called autocommit. I am hoping I am doing this experiment right.
    Thanks for the pointers about set_flags() and DB_TXN_NOSYNC, I am going to look them up.

  • Write-through limitation and putAll

    Please find the quote below from developer guide, particularly this one In other words, if two cache entries are updated, triggering calls to CacheStore modules sitting on separate cache servers, it is possible for one database update to succeed and for the other to fail.If a putAll is called on a cache, will it result in one CacheStore.storeAll or many storeAll triggered from different coherence nodes/servers? (assume a distributed topology coherence 3.7.1)
    Will the store transaction failure lead to putAll transaction failure?
    Are there any patterns that shows how this coherence works with typical databases?
    14.7.2 Write-Through LimitationsCoherence does not support two-phase CacheStore operations across multiple CacheStore instances. In other words, if two cache entries are updated, triggering calls to CacheStore modules sitting on separate cache servers, it is possible for one database update to succeed and for the other to fail. In this case, it may be preferable to use a cache-aside architecture (updating the cache and database as two separate components of a single transaction) with the application server transaction manager. In many cases it is possible to design the database schema to prevent logical commit failures (but obviously not server failures). Write-behind caching avoids this issue as "puts" are not affected by database behavior (as the underlying issues have been addressed earlier in the design process).

    gs100 wrote:
    Thanks for the input, I have further questions based on these suggestions.
    1. Let us say one of the putAll fails we would know that it has failed due to underlying one or more store/storeAll. And even if we rollback the coherence transaction, the store/storeAll that succeeded would not be rolled back automatically, is that correct? If true, this means that it would leave the underlying DB/store in the inconsistent state with that of in-memory cache?I guess that is one of the reasons why the transaction framework does not support cache stores... also, write-behind would coalesce updates which would have funny consequences with regards to the transactional context...
    2. How do we get the custom implementation of putAll, that you suggested to handle specific errors? any pointers on this would be helpful.I guess it is not going to be posted, the Coherence team may or may not add something which is a bit more deterministic with regards to error.
    A few aspects of Coherence behaviour (a.k.a pitfalls) which you need to be aware of to be able to implement your own solution:
    Exceptions propagating back to the client can happen in:
    - entry-processor (not for putAll specifically)
    - result serialization code (not for putAll specifically, but for processAll/aggregate for example)
    - deserialization code (indexes/filter-based backing map listeners/cache stores lead to deserialization even for putAll)
    - triggers (intentionally, too)
    - cache stores
    There is no place where you could catch any exceptions from inside the NamedCache call, so they will come out.
    Coherence may execute the operation on one thread per partition or one thread per multiple partitions, but never on multiple threads per partition. This means there may be multiple exceptions even from a single storage node, but only at most one exception would be generated per partition (starting with 3.5).
    If you send multiple partitions with the same NamedCache call, you can lose exceptions as you wouldn't know if an exception would have or wouldn't have happened with a partition if it was sent alone instead of together with another on the same node.
    As you need to be able to return all exceptions from your method call, you have to produce and catch all of them and collect them otherwise you would lose all but one. To produce and catch all exceptions you have to produce all exceptions independently, i.e. different partitions must be operated on independently.
    To send an operation to a single partition only, you can separate the operations to different partitions by separating the keysets for different partitions with key-based operations, or applying a PartitionedFilter for filter-based operations.
    It is up to you where and how you iterate through the partitions. You can do it on the caller, you can do it on storage node from an Invocable sent via an InvocationService (in this case you can be either optimistic with ownership or chase a partition).
    3. Because we are thinking putAll that coherence implemented is most optimized (parallelism). I am not sure how the custom implementation can be as optimal (hope we don't end up calling one by one).You cannot implement it as optimally as Coherence itself does as it interleaves operations (Messages) to independent partitions/nodes (does not have to wait for the return message) from a single thread without waiting for the responses from individual nodes/partitions.
    You can either parallelize operations to multiple threads, or do the iteration on the single thread at the cost of higher latency.
    Best regards,
    Robert

  • Write behind cache, DB down, when should the system stop taking new data in

    Hello:
    We are trying to use Coherence for our custom ESB, which is brokering payloads of various size between consumer and provider applications.
    Before Coherence, stopping our DB meant organization-wide outage for critically important business services.
    Since we have at least 40G of RAM in production environment, we believe that our app
    can use Coherence write-behind option for tolerating at least several hours worth of DB outage.
    We are currently using a near cache backed by distributed cache in write-behind mode.
    9 business service JVMs (storage enabled=false) use 30 storage enabled JVMs.
    IMPORTANT: We need to create an automated alerting facility determining when
    amount of unsaved data reaches critical level since DB goes down. This alert should help us decide when our application stops accepting inbound traffic.
    It is hard to use QueueSize parameter for that because our payload memory footprint can vary from 1KB to 3MB.
    We do not expire any entries in order to enable support queries against the cache during DB outage.
    Our experiments with trying various flavors of overflow-scheme resulted in OutOfMemoryError, therefore
    we decided to implement RAM-only cache as a first step.
    <near-scheme>
    <scheme-name>message_payload_scheme</scheme-name>
    <front-scheme>
    <local-scheme>
    <scheme-ref>limited_entities_front_scheme</scheme-ref>
    <high-units>100</high-units>
    </local-scheme>
    </front-scheme>
    <back-scheme>
    <distributed-scheme>
    <backing-map-scheme>
    <read-write-backing-map-scheme>
    <internal-cache-scheme>
    <local-scheme>
    <scheme-ref>limited_bytes_scheme</scheme-ref>
    <high-units>199229440</high-units>
    </local-scheme>
    </internal-cache-scheme>
    <cachestore-scheme>
    <class-scheme>
    <class-name>com.comp.MessagePayloadStore</class-name>
    </class-scheme>
    </cachestore-scheme>
    <read-only>false</read-only>
    <write-delay-seconds>3</write-delay-seconds>
    <write-requeue-threshold>2147483646</write-requeue-threshold>
    </read-write-backing-map-scheme>
    </backing-map-scheme>
    <autostart>true</autostart>
    </distributed-scheme>
    </back-scheme>
    </near-scheme>
    <local-scheme>
    <scheme-name>limited_entities_front_scheme</scheme-name>
    <eviction-policy>LRU</eviction-policy>
    <unit-calculator>FIXED</unit-calculator>
    </local-scheme>
    <local-scheme>
    <scheme-name>limited_bytes_scheme</scheme-name>
    <eviction-policy>HYBRID</eviction-policy>
    <unit-calculator>BINARY</unit-calculator>
    </local-scheme>

    Good info ... I feel like I need to restate my original question along with a couple of new questions caused by the discussion above.
    Q1. Does Coherence evict 'dirty', or 'queued', or 'unsaved' objects for cache configuration provided above?
    The answer should be 'NO', otherwise Coherence is unsafe to use as a system of record,
    it should not just drop unsaved information on the floor.
    Q2. What happens to the front tier of the near+partitioned write behind cache described above when amount of unsaved data exceeds max cache capacity defined via high-units?
    I would expect that map.put starts throwing exceptions: cache storage is full, so it should not accept more data
    Q3. How can I determine a moment when amount of dirty data in bytes(!), not in objects, hits 85% of
    max allowed cache capasity configured in bytes (using high-units param and BINARY calculator).
    'DirtyUnits' counter can probably be built with some lower-level Coherence API. Can we use
    this API?
    Please, understand, that we purchased Coherence for reliability, for making our
    system independent from short DB outages, for keeping our business services up
    and running when DBA need some time for admin operations like rebuilding an index.
    Performance benefits are secondary and are not as obvious for our system which
    uses primary keys only and has a well-tuned co-located Oracle back-end.
    We simply cannot put Coherence to production unless we prove that Coherence
    can reliably hold the data and give us information about approaching crisis
    (the cache full of unsaved data).
    If possible, forward this message to Cameron Purdy,
    who was presenting Coherence to our team several moths ago.
    Thanks,
    Vasili Smaliak
    Applications Architect, Enterprise App Integration
    GMAC ResCap
    [email protected]

  • Error in File name or class name not found during Automation operation: 'CreateObj​ect'

    Hello Team,
    When I am trying to execute the below code i am getting the following error. Any help would be greatly appreciated.
    Set oFSO = CreateObject("Scripting.FileSystemObject")
    If Not oFSO.FolderExists(SavePath) Then
    Set f = oFSO.CreateFolder(SavePath)
    Else
    End If
    53 4/11/2014 12:27:22 PM Error:
    Error in <NoName(4).VBS> (Line: 9, Column: 1):
    File name or class name not found during Automation operation: 'CreateObject'
    I have googled through the error and tried to re-register the scrrun.dll using regsvr32 eventhogh it is successfully registered, i am getting the following error. My PC is windows 7 32bit OS.
    any help is greatly appreciated.

    The following script class will write a log file entry. See if it will run for you.
    The script is using a class object that you might not have seen before. A little intro:  The top section is just for testing the class. Normally I just comment this out after the class is working well.  It should run right way. I would save the vbs file in the editor, That way when you are using autoactpath or currentscriptpath variables they will be able resolve the paths.
    Paul
    Attachments:
    LoggingCode_V2.VBS ‏5 KB

  • Map listeners and write-through strategy.

    Hi.
    Write-through strategy seems to be synchronious operations and if it fails, no data should appear in cache. Logically this means, that no events will be produced if the persisting fails (that's what we exactly need). But could not find any mention about this in documentation. Can anyone verify this?
    Thanks, Anton.

    If you are talking about throwing an exception in your CacheStore code,
    it will happen before anything occurs in the internal cache managed by
    Coherence and no events will be generated (that would have been generated
    under normal cases where the operation succeeded.)

Maybe you are looking for

  • User Exits for Document Info Record : CV01N in DMS

    Hi Experts My scenario is: We have Configured DMS in which new document type created Z01 is for MS word documents. I have assigned default Work Station Application : DOC to Z01 document type. But my problem is : while creating Info record : CV01n wit

  • Ipod has stopped

    My ipod has recently been freezing up. I would reset it and then have to set up my playlists again. Now it just will not turn on. I've tried resetting it but the sick Ipod just keeps showing up on the screen. How can I turn my ipod back on?

  • Are the Yellow banding iMac problems fixed?

    I read the posts about the iMac screen flickering, tearing, and having yellow bands on it. I haven't been able to find anything though that says the problem has been resolved. Are the iMacs safe to buy now? Thanks

  • How can I restore bookmarks?

    I have hundreds of bookmarks which I've acquired over a number of years, most in folders on the bookmark bar on my iMac. I use Chrome as a browser. I also have a Nexus 7 and have an app called Chromesync, to sync the bookmarks from my iMac to my othe

  • Inserting data into a MySQL database

    Hi. I am currently working on a group project and know very little about java to be honest I'm trying to insert records into a local MySQL database, but I'm having trouble. When the application is started, a connection is made to the database, which,