How to handle Keys (Sequences) generated in Database

Question

Need some pointers about how to achieve following:
I am building a cache for TRADES and have a CacheStore supporting LOAD/STORE/REMOVE methods.
The TRADES cache is associated with TRADE table on the database. I have most of my business logic related to adding a trade in a Database Package.
If a new trade is being added to the table, Stored Procedure in my package will use a sequence number to assign it a UNIQUE KEY and will return the key with the call
Now, my cache is using TRADE KEY as a key for the trade objects. If my application is putting a new trade to the cache, then how should this be designed?
I do not really want to have sequences put in the cache as my stored procedure does it for me and there are few other applications which are using same SP.
Has anyone implemented something like this where the key is now known until the object is entered in the database?
I can have a TEMPORARY KEY assigned to the object and can replace the record once the trade has been successfully added to the table. As I can not do replace/update cache records in CacheStore, is there any way to achieve this?
Following is my cache-config:
<distributed-scheme>
     <scheme-name>PartitionedTradePOFScheme</scheme-name>
     <service-name>SweetTradeSvc</service-name>
     <serializer>
          <class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
          <init-params>
               <init-param>
                    <param-type>String</param-type>
                    <param-value>position-pof-config.xml</param-value>
               </init-param>
          </init-params>
     </serializer>
     <backing-map-scheme>
          <read-write-backing-map-scheme>
               <internal-cache-scheme>
                    <local-scheme></local-scheme>
               </internal-cache-scheme>
                    <cachestore-scheme>
                    <class-scheme>
                         <class-name>com.db.spg.sweet.coherenceutil.DBCacheStoreTrade</class-name>
                         <init-params>
                              <init-param>
                                   <param-type>java.lang.String</param-type>
                                   <param-value>{cache-name}</param-value>
                              </init-param>
                         </init-params>
                    </class-scheme>
               </cachestore-scheme>
          </read-write-backing-map-scheme>
     </backing-map-scheme>
     <autostart>true</autostart>
</distributed-scheme>I am trying to be as clear as possible. If anything here is confusing, let me know.
Thanks

Answer 1

Jonathan.Knight wrote:
<li>The AQ messaging is asynchronous so there will be a time-lag between the original put of the trade and it being updated with the version from the DB.
<li>How does the AQ listener know what the key is of the temporary trade is that it needs to remove, presumably you write this to the DB then send it back in the AQ message?Agreed. From my CacheStore, I call Stored Procedure inside Database Package. I can pass this TEMP key to this SP as 1 extra parameter. So, I do not necessarily need to store it in the database, but just use merely as an extra parameter to be sent back to AQ. Also, when the client puts a new trade, the business logic for most of the trade processing is in our stored procedures. There, it will enrich that trade with additional information from other tables, it will also adjust the trader position and then will return with the new trade key to the client. GUI/Client Applications will provide the data required to create a trade. But SP will use this data and using other local stored procedures, it will obtain extra information for the trade, enrich all the information in a single trade record and then will create that new record.
<li>If the code the listens to the AQ messages removes the original version of the trade from the cache before putting the new version into the cache you will have a short period of time where there is that trade does not exist in the cache
<li>Alternatively if your AQ listener works the other way round and puts the new version into the cache before removing the original version you will have a short period where there are two copies of the same trade in the cache.
<li>Depending on what you use-cases for reading these trades back out of the cache you could have issues with the above times when there may be one, zero or two copies of the trade.Yes. I have been thinking about that as well. But I think, as the trade entered from GUI is merely an indicator for the background/database processing. If and only if the database processing is successful, that is considered a valid trade. So as far as I know, the duplicate/old copy of the trade would still be considered as in "waiting database update" state. I am designing my GUI accordingly. I will probably check with the users about this requirement once again though. You have a valid point.
<li>I assume your use-cases for accessing trades from the cache are not using the keys as they could not rely on having obtained the correct key for a trade as it might change. At this point, I am using these TRADE keys as my cache keys. Because, it is really easier for us to manage TRADES in the cache, especially when there are lot of lookups involved from our users
One more thing someone I had worked with long time back suggested that, having a staging/metadata cache for all these trades to be processed from the client. Add these trades in the database and then in QueueListener, add a new trade in Trade-Cache and remove the corresponding old trade from staging/metadata cache. This also seems good, though I need to see if that is going to complicate matters in terms of processing.
Thanks a lot JK for your suggestions. Let me know if the logic I explained makes sence.