Cluster nodes discover peers outside cluster domain

Question

All, cross-posted from the ColdFusion Server Administration
forum:
I've run into an issue with CFMX7 clustering on a subnet with
multicast disabled. In our configuration, we have two physical
Windows Server 2003 Enterprise Edition servers hosting nine
ColdFusion MX 7 Enterprise clusters. Each server hosts one of two
instances in a cluster. i.e.:
server1 [1.2.3.4] - instance1-1 <- cluster1 -> server2
[1.2.3.5] - instance1-2
server1 [1.2.3.4] - instance2-1 <- cluster2 -> server2
[1.2.3.5] - instance2-2
server1 [1.2.3.4] - instance3-1 <- cluster3 -> server2
[1.2.3.5] - instance3-2
server1 [1.2.3.4] - instance4-1 <- cluster4 -> server2
[1.2.3.5] - instance4-2
server1 [1.2.3.4] - instance5-1 <- cluster5 -> server2
[1.2.3.5] - instance5-2
server1 [1.2.3.4] - instance6-1 <- cluster6 -> server2
[1.2.3.5] - instance6-2
server1 [1.2.3.4] - instance7-1 <- cluster7 -> server2
[1.2.3.5] - instance7-2
server1 [1.2.3.4] - instance8-1 <- cluster8 -> server2
[1.2.3.5] - instance8-2
server1 [1.2.3.4] - instance9-1 <- cluster9 -> server2
[1.2.3.5] - instance9-2
My first step in enabling peer discovery was to add the
unicastPeer attribute to the ClusterManager service under each
instance.
e.g. jrun.xml on instance1-1:
<service class="jrunx.cluster.ClusterManager"
name="ClusterManager">
<attribute name="bindToJNDI">true</attribute>
<attribute name="enabled">true</attribute>
<attribute
name="clusterDomain">cluster1</attribute>
<!-- While we will discover nearby peers automatically
without prior knowledge -->
<!-- of them, you can also add as many specific hosts as
you wish; these unicast -->
<!-- peers do not need to be nearby or reachable via
multicast. -->
<!--EXAMPLE: <attribute
name="unicastPeer">sneville</attribute> -->
<attribute name="unicastPeer">1.2.3.5</attribute>
<service class="jrunx.cluster.ClusterDeployerService"
name="ClusterDeployerService">
<attribute
name="deployDirectory">{jrun.server.rootdir}/SERVER-INF/cluster</attribute>
<attribute name="deactivated">false</attribute>
</service>
</service>
e.g. jrun.xml on instance1-2:
<service class="jrunx.cluster.ClusterManager"
name="ClusterManager">
<attribute name="bindToJNDI">true</attribute>
<attribute name="enabled">true</attribute>
<attribute
name="clusterDomain">cluster1</attribute>
<!-- While we will discover nearby peers automatically
without prior knowledge -->
<!-- of them, you can also add as many specific hosts as
you wish; these unicast -->
<!-- peers do not need to be nearby or reachable via
multicast. -->
<!--EXAMPLE: <attribute
name="unicastPeer">sneville</attribute> -->
<attribute name="unicastPeer">1.2.3.4</attribute>
<service class="jrunx.cluster.ClusterDeployerService"
name="ClusterDeployerService">
<attribute
name="deployDirectory">{jrun.server.rootdir}/SERVER-INF/cluster</attribute>
<attribute name="deactivated">false</attribute>
</service>
</service>
. . . and so on for each instance and cluster. This is where
the problem begins. When I start the instances, every instance
discovers every other instance as a cluster peer, regardless of
cluster domain.
Another forum user suggested using host:port, where port is
the JNDI listening port. That doesn't work. Using the Jini
listening port, however, does work, e.g.:
<attribute
name="unicastPeer">1.2.3.4:4160</attribute>
That presents another problem. The Jini listening port
defaults to 4160. If 4160 is taken, a port is chosen at random.
I can't find documentation on setting a static Jini listening
port, if that's even the correct action to take.
Thoughts?
From what I can tell, the version of Reggie (the Jini lookup
service) shipped with JRun only supports setting the unicast
listening port programmatically. Reggie is started by
jrunx.cluster.ClusterManager.init--actually, the private method
startLookupService--and JRun doesn't appear to ever call Reggie's
setUnicastPeer method.
Assuming we can't tweak Reggie, I guess a more appropriate
question is how do we get JRun's RMI service (?) to honor
groups/domains in a call to getPeers? I'll cross-post to the JRun
forums and investigate JRun Updater 6.
Trev

Answer 1

. . . and it appears I'm exposing my ignorance of Jini in
general. :-)
If I now understand the Jini discovery process correctly, a
multicast request includes one or more service IDs and one or more
groups. The registrar will respond if and only if its service ID is
not in the request and its group memberships exactly match one or
more of the groups in the request.
A unicast request includes nothing more than the protocol
version, and the registrar will respond as if a valid multicast
request had been received.
In both cases, the response packet includes a marshalled copy
of the ServiceRegistrar object and the names of all groups of which
the registrar is a member.
Without looking at more of JRun, I'm guessing that in some,
if not all cases, either JRun's discovery implementation assumes
that any response from a unicast query is valid, regardless of the
server IDs or group names received, or the logic that sorts out the
response isn't 100% correct.