How to reduce clone/merge overhead in high cardinality 1:M relationships?

Question

Our application has numerous parent/child type 1:M relationships, where the collection size can be very large, and often the child object is itself very large. For these relationships, we have a catch-22 when using Toplink. Using the 1:M relationship with lazy loading and batch-reading is ideal for code that may or may not need the child objects. TopLink caching saves much SQL overhead when traversing the relationship in a read-only context. However, when making updates, the relationship becomes a burden, due to the overhead of cloning and merging the entire collection of objects. When we simply need to add a new object to the collection, TopLink automatically instantiates the collection, and makes clones and backup copies of all the contained objects. The database query is not needed, because the collection is stored in the cached parent object, but the copying overhead really hurts performance.
What we need is a way to add or remove child objects in a UOW without this overhead, but still maintain the collection in the cached parent. I have been trying various things without any success. Is there documented pattern for this problem? Do I need to resort to using the AttributeChangeTrackingPolicy and/or a custom CopyPolicy, or is there a way I can avoid the cloning entirely?
Thanks in advance.
- Bruno

Answer 1

Of the options you've suggested, I suspect the simplest and most performant option would be to use a postMerge handler to maintain the parent collection. This would work in a replicated cache as well, correct? How do I avoid introducing race conditions? Would I need to lock the parent object using the IdentityMapManager while I add or remove from the collection?
As an aside, I must say that the documentation is really lacking for information about what is implied by a "bi-directional" mapping. What I have found through experimentation is that when the "relationshipPartner" is set on the OneToOne mapping for the back-reference, and basicIndirection is used, TopLink will automatically put the object in the parent's OneToMany collection when setValue() is called on the ValueHolderInterface, and thereafter it is essentially conformant within the UOW. Conversely if I create a child object and add it to the parent's collection, the back-reference will be set automatically on the child, as long as I use transparent indirection. I haven't tried it with basic indirection, but I suspect it won't work.
However, it does all this by instantiating the ValueHolder on the parent container and thereby triggering the clone-merge of all the objects in the collection. This is the symptom I'd hoped to avoid. If I don't set the relationshipPartner, on the child mapping I can avoid this, but now my data access logic gets more complicated. What I really want to do is have something like a UnitOfWork within the collection itself. I want add and remove operations to be deferred. When I add or remove something to the collection, it would be kept in a holding area which would be merged into the parent cache when committed, but the ValueHolder would not be triggered. If/when the entire contents of collection is needed within the UnitOfWork, the ValueHolder would be instantiated, the collection contents would be cloned, and then the added/removed objects in the holding area would be merged into the collection. Am I missing any reasons why this approach is technically not feasible?
The other suggestion to use the AttributeChangeTrackingPolicy has been considered, but I think that still won't be as good as what could be achieved by the above approach. There would still be a need to make clones for every object in the collection, and in order to avoid the drudgery of updating all my domain objects I'd have to resort to code-generation, or runtime method interception (we're not using CMP). I'm not quite ready to leap into that approach until all other alternatives have failed.
As for the query cache, I would have to invalidate the entire query cache whenever objects are added or removed either locally or remotely, correct? TopLink won't check each query cache when an object is merged to determine if the object still matches the query criteria, correct? Are you saying that I would need to create my own cache invalidation policy to do this? I could handle removed objects that way, but I'd still need to invalidate the cache for any additions, right?
I know this message is long. Please, any help is much appreciated.
- Bruno