Backing Map Access : Cross-cache joins

Question

Hi,
I have been experimenting with cross-cache joins using Entry Processors in Coherence 3.7.1.
(I have already sent a query to Dave Felcey regarding this - I will post any response from him here - but I just wondered if anyone else has had the same problem)
h3. Scenario
A simplified version of the problem is:
We have two NamedCaches, one called "PARENT_CACHE" and one called "CHILD_CACHE"
Each cache stores the following types of object respectivley...
*class Parent implements PortableObject {*
long id;
String description;
*class Child implements PortableObject {*
long id;
long parentId;
String description;
I want an entry processor that I can send to an instance of "Parent" that will return the "Parent" object together with the associated "Child" objects. The reason I want this is because I do not want to do two out-of-process calls (one to get the parent and one to get it's children - in the real world, we will need to get several different child types and we do not want to do one call for each type) for example...
*class ExampleEntryProcessor extends AbstractProcessor implements PortableObject {*
public ParentAndChildrenResult process(Entry entry) {
I wrote an implementation of this based on Ben Stopfold's blog (see here - particularly the post by "Jonathan Knight")
So I thought I needed something like this...
     *public ParentAndChildrenResult process(Entry entry) {*
          ParentAndChildrenResult result = new ParentAndChildResult();
          Parent parent = (Parent) entry.getValue();
          result.setParent(parent);
          Filter parentIdFilter = new EqualsFilter(new PofExtractor(Long.class, Child.PARENT_ID_FIELD), parent.getId());
          BinaryEntry binaryEntry = (BinaryEntry) entry;
          Set<java.util.Map.Entry> childEntrySet = queryBackingMap("CHILD_CACHE", parentIdFilter, binaryEntry);
          Converter valueUpConverter = binaryEntry.getContext().getValueFromInternalConverter();
          for (java.util.Map.Entry childEntry : childEntrySet) {
               result.addChild((Child) valueUpConverter.convert(childEntry.getValue()));
          return result;
     *public Set<Map.Entry> queryBackingMap(String nameOfCacheToSearch, Filter filter, BinaryEntry entry) {*
     BackingMapContext backingMapContext = entry.getContext().getBackingMapContext(nameOfCacheToSearch);
     Map indexMap = backingMapContext.getIndexMap();
     return InvocableMapHelper.query(backingMapContext.getBackingMap(), indexMap, filter, true, false, null);
I set up key association so I can ensure that the child objects are on the same node as the parent, the keys for each cache look like this...
*class Key implements KeyAssociation, PortableObject {*
private long id;
private long associatedId;
*public Key() {*
*public Key(Parent parent) {*
this.id = parent.getId();
this.associatedId = parent.getId();
*public Key(Child child) {*
this.id = child.getId();
this.associatedId = child.getParentId();
*public Object getAssociatedKey() {*
return associatedId;
When I send this entry processor to a parent object, I am getting the following exception when the "InvocableMapHelper.query" method is called...
+"Portable(java.lang.UnsupportedOperationException): PofExtractor must be used with POF-encoded Binary entries; the Map Entry is not a BinaryEntry"+
I was not expecting this as our cluster is POF enabled and I thought that the backing maps always contained BinaryEntries.
Has anyone else had a similar problem? Has anyone found any simple examples of how to do this anywhere on the web that work?
Once I figure out how to get this to work, I want to post the solution somewhere (probably here) because there are bound to be other people who want to do something similar to this.
Thanks in advance,
-Bret

Answer 1

Forgot the output...
STORAGE NODE OUTPUT
2012-01-12 18:38:09.860/0.766 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/C:/users/vclondon/.m2/repository/com/tangosol/coherence/3.7.1.0b27797/coherence-3.7.1.0b27797.jar!/tangosol-coherence.xml"
2012-01-12 18:38:09.860/0.766 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "file:/C:/dev2/tech-trading-workspace/backing-map-access/bin/tangosol-coherence-override.xml"
2012-01-12 18:38:09.860/0.766 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
Oracle Coherence Version 3.7.1.0 Build 27797
Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
2012-01-12 18:38:10.016/0.922 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Loaded cache configuration from "file:/C:/dev2/tech-trading-workspace/backing-map-access/bin/coherence-cache-config.xml"; this document does not refer to any schema definition and has not been validated.
2012-01-12 18:38:10.344/1.250 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Loaded Reporter configuration from "jar:file:/C:/users/vclondon/.m2/repository/com/tangosol/coherence/3.7.1.0b27797/coherence-3.7.1.0b27797.jar!/reports/report-group.xml"
2012-01-12 18:38:23.610/14.516 Oracle Coherence GE 3.7.1.0 <D4> (thread=main, member=n/a): TCMP bound to /172.23.0.26:8088 using SystemSocketProvider
2012-01-12 18:38:54.454/45.360 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=n/a): Created a new cluster "LOCAL" with Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode, Edition=Grid Edition, Mode=Development, CpuCount=2, SocketCount=1) UID=0xAC17001A00000134D2FFC167532F1F98
2012-01-12 18:38:54.454/45.360 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Started cluster Name=LOCAL
WellKnownAddressList(Size=1,
WKA{Address=172.23.0.26, Port=8088}
MasterMemberSet(
ThisMember=Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode)
OldestMember=Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode)
ActualMemberSet=MemberSet(Size=1
Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode)
MemberId|ServiceVersion|ServiceJoined|MemberState
1|3.7.1|2012-01-12 18:38:54.454|JOINED
RecycleMillis=1200000
RecycleSet=MemberSet(Size=0
TcpRing{Connections=[]}
IpMonitor{AddressListSize=0}
2012-01-12 18:38:54.501/45.407 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=1): Loaded POF configuration from "file:/C:/dev2/tech-trading-workspace/backing-map-access/bin/pof-config.xml"; this document does not refer to any schema definition and has not been validated.
2012-01-12 18:38:54.516/45.422 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=1): Loaded included POF configuration from "jar:file:/C:/users/vclondon/.m2/repository/com/tangosol/coherence/3.7.1.0b27797/coherence-3.7.1.0b27797.jar!/coherence-pof-config.xml"
2012-01-12 18:38:54.579/45.485 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
2012-01-12 18:38:54.876/45.782 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1
2012-01-12 18:38:54.891/45.797 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:InvocationService, member=1): Service InvocationService joined the cluster with senior service member 1
2012-01-12 18:38:54.907/45.813 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=1):
Services
ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.7.1, OldestMemberId=1}
InvocationService{Name=Management, State=(SERVICE_STARTED), Id=1, Version=3.1, OldestMemberId=1}
PartitionedCache{Name=DistributedCache, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}
InvocationService{Name=InvocationService, State=(SERVICE_STARTED), Id=3, Version=3.1, OldestMemberId=1}
Started DefaultCacheServer...
2012-01-12 18:39:03.438/54.344 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member(Id=2, Timestamp=2012-01-12 18:39:03.274, Address=172.23.0.26:8090, MachineId=21295, Location=site:,machine:J67LQ2J,process:14408, Role=TestBackingmapMainRunTheTest) joined Cluster with senior member 1
2012-01-12 18:39:03.610/54.516 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member 2 joined Service Management with senior member 1
2012-01-12 18:39:03.907/54.813 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member 2 joined Service DistributedCache with senior member 1
2012-01-12 18:39:04.032/54.938 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): TcpRing disconnected from Member(Id=2, Timestamp=2012-01-12 18:39:03.274, Address=172.23.0.26:8090, MachineId=21295, Location=site:,machine:J67LQ2J,process:14408, Role=TestBackingmapMainRunTheTest) due to a peer departure; removing the member.
2012-01-12 18:39:04.032/54.938 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member 2 left service Management with senior member 1
2012-01-12 18:39:04.032/54.938 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member 2 left service DistributedCache with senior member 1
2012-01-12 18:39:04.032/54.938 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member(Id=2, Timestamp=2012-01-12 18:39:04.032, Address=172.23.0.26:8090, MachineId=21295, Location=site:,machine:J67LQ2J,process:14408, Role=TestBackingmapMainRunTheTest) left Cluster with senior member 1
PROCESS NODE OUTPUT
2012-01-12 18:39:02.266/0.328 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/C:/users/vclondon/.m2/repository/com/tangosol/coherence/3.7.1.0b27797/coherence-3.7.1.0b27797.jar!/tangosol-coherence.xml"
2012-01-12 18:39:02.266/0.328 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "file:/C:/dev2/tech-trading-workspace/backing-map-access/bin/tangosol-coherence-override.xml"
2012-01-12 18:39:02.266/0.328 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
Oracle Coherence Version 3.7.1.0 Build 27797
Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
2012-01-12 18:39:02.407/0.469 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Loaded cache configuration from "file:/C:/dev2/tech-trading-workspace/backing-map-access/bin/coherence-cache-config.xml"; this document does not refer to any schema definition and has not been validated.
2012-01-12 18:39:02.501/0.563 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Loaded Reporter configuration from "jar:file:/C:/users/vclondon/.m2/repository/com/tangosol/coherence/3.7.1.0b27797/coherence-3.7.1.0b27797.jar!/reports/report-group.xml"
2012-01-12 18:39:03.063/1.125 Oracle Coherence GE 3.7.1.0 <D4> (thread=main, member=n/a): TCMP bound to /172.23.0.26:8090 using SystemSocketProvider
2012-01-12 18:39:03.438/1.500 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=n/a): This Member(Id=2, Timestamp=2012-01-12 18:39:03.274, Address=172.23.0.26:8090, MachineId=21295, Location=site:,machine:J67LQ2J,process:14408, Role=TestBackingmapMainRunTheTest, Edition=Grid Edition, Mode=Development, CpuCount=2, SocketCount=1) joined cluster "LOCAL" with senior Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode, Edition=Grid Edition, Mode=Development, CpuCount=2, SocketCount=1)
2012-01-12 18:39:03.485/1.547 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=n/a): Member 1 joined Service Management with senior member 1
2012-01-12 18:39:03.485/1.547 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=n/a): Member 1 joined Service DistributedCache with senior member 1
2012-01-12 18:39:03.485/1.547 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=n/a): Member 1 joined Service InvocationService with senior member 1
2012-01-12 18:39:03.485/1.547 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Started cluster Name=LOCAL
WellKnownAddressList(Size=1,
WKA{Address=172.23.0.26, Port=8088}
MasterMemberSet(
ThisMember=Member(Id=2, Timestamp=2012-01-12 18:39:03.274, Address=172.23.0.26:8090, MachineId=21295, Location=site:,machine:J67LQ2J,process:14408, Role=TestBackingmapMainRunTheTest)
OldestMember=Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode)
ActualMemberSet=MemberSet(Size=2
Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode)
Member(Id=2, Timestamp=2012-01-12 18:39:03.274, Address=172.23.0.26:8090, MachineId=21295, Location=site:,machine:J67LQ2J,process:14408, Role=TestBackingmapMainRunTheTest)
MemberId|ServiceVersion|ServiceJoined|MemberState
1|3.7.1|2012-01-12 18:38:23.719|JOINED,
2|3.7.1|2012-01-12 18:39:03.477|JOINED
RecycleMillis=1200000
RecycleSet=MemberSet(Size=0
TcpRing{Connections=[1]}
IpMonitor{AddressListSize=0}
2012-01-12 18:39:03.501/1.563 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=2): Loaded POF configuration from "file:/C:/dev2/tech-trading-workspace/backing-map-access/bin/pof-config.xml"; this document does not refer to any schema definition and has not been validated.
2012-01-12 18:39:03.532/1.594 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=2): Loaded included POF configuration from "jar:file:/C:/users/vclondon/.m2/repository/com/tangosol/coherence/3.7.1.0b27797/coherence-3.7.1.0b27797.jar!/coherence-pof-config.xml"
2012-01-12 18:39:03.594/1.656 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:Management, member=2): Service Management joined the cluster with senior service member 1
2012-01-12 18:39:03.891/1.953 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=2): Service DistributedCache joined the cluster with senior service member 1
Exception in thread "main" Portable(com.tangosol.util.WrapperException): (Wrapped: Failed request execution for DistributedCache service on Member(Id=1, Timestamp=2012-01-12 18:38:23.719, Address=172.23.0.26:8088, MachineId=21295, Location=site:,machine:J67LQ2J,process:13600, Role=TestBackingmapStartStorageNode)) PofExtractor must be used with POF-encoded Binary entries; the Map Entry is not a BinaryEntry
at com.tangosol.util.Base.ensureRuntimeException(Base.java:288)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.tagException(Grid.CDB:36)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onInvokeRequest(PartitionedCache.CDB:68)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$InvokeRequest.run(PartitionedCache.CDB:1)
at com.tangosol.coherence.component.net.message.requestMessage.DistributedCacheKeyRequest.onReceived(DistributedCacheKeyRequest.CDB:12)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:34)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:33)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
at java.lang.Thread.run(Unknown Source)
at <process boundary>
at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:57)
at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
at com.tangosol.io.pof.ConfigurablePofContext.deserialize(ConfigurablePofContext.java:368)
at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.readObject(Service.CDB:1)
at com.tangosol.coherence.component.net.Message.readObject(Message.CDB:1)
at com.tangosol.coherence.component.net.message.SimpleResponse.read(SimpleResponse.CDB:6)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.deserializeMessage(Grid.CDB:19)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:31)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
at java.lang.Thread.run(Unknown Source)
Caused by: Portable(java.lang.UnsupportedOperationException): PofExtractor must be used with POF-encoded Binary entries; the Map Entry is not a BinaryEntry
at com.tangosol.util.extractor.PofExtractor.extractInternal(PofExtractor.java:175)
at com.tangosol.util.extractor.PofExtractor.extractFromEntry(PofExtractor.java:146)
at com.tangosol.util.InvocableMapHelper.extractFromEntry(InvocableMapHelper.java:315)
at com.tangosol.util.SimpleMapEntry.extract(SimpleMapEntry.java:168)
at com.tangosol.util.filter.ExtractorFilter.evaluateEntry(ExtractorFilter.java:93)
at com.tangosol.util.InvocableMapHelper.evaluateEntry(InvocableMapHelper.java:262)
at com.tangosol.util.InvocableMapHelper.query(InvocableMapHelper.java:452)
at test.backingmap.main.GetParentAndChildrenEntryProcessor.queryBackingMap(GetParentAndChildrenEntryProcessor.java:60)
at test.backingmap.main.GetParentAndChildrenEntryProcessor.process(GetParentAndChildrenEntryProcessor.java:47)
at test.backingmap.main.GetParentAndChildrenEntryProcessor.process(GetParentAndChildrenEntryProcessor.java:1)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.invoke(PartitionedCache.CDB:10)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onInvokeRequest(PartitionedCache.CDB:52)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$InvokeRequest.run(PartitionedCache.CDB:1)
at com.tangosol.coherence.component.net.message.requestMessage.DistributedCacheKeyRequest.onReceived(DistributedCacheKeyRequest.CDB:12)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:34)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:33)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
at java.lang.Thread.run(Unknown Source)
at <process boundary>
at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:57)
at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
at com.tangosol.io.pof.PortableException.readExternal(PortableException.java:150)
at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:59)
at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
at com.tangosol.io.pof.ConfigurablePofContext.deserialize(ConfigurablePofContext.java:368)
at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.readObject(Service.CDB:1)
at com.tangosol.coherence.component.net.Message.readObject(Message.CDB:1)
at com.tangosol.coherence.component.net.message.SimpleResponse.read(SimpleResponse.CDB:6)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.deserializeMessage(Grid.CDB:19)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:31)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
at java.lang.Thread.run(Unknown Source)
2012-01-12 18:39:04.016/2.078 Oracle Coherence GE 3.7.1.0 <D4> (thread=ShutdownHook, member=2): ShutdownHook: stopping cluster node
Ta,
-Bret