Multicast vs Well known address

Question

Couple of questions regarding the multicast communication in a Coherence 3.3 cluster
- What are the downsides of using the well known address scheme compared to multicast for cluster communication. One i could think of was having to explicitly specify all the addresses that need to be part of the cluster. Was wondering if there were any performance related differences?
- What is the amount of multicast traffic that is handled by a member of the Coherence cluster which participates without having any local data stored? Is every message that is sent to the data storage members also sent to the non-data members
Thanks
Ramdas

Answer 1

Hi Ramdas,
What are the downsides of using the well known address scheme compared to multicast for cluster communication. One i could think of was having to explicitly specify all the addresses that need to be part of the cluster. Was wondering if there were any performance related differences?
You don't need to list all addresses, only a subset of servers that is sufficiently large to guarantee that at least one server will be up at all times. Membership data in Coherence is entirely dynamic, so once a client connects to any of those servers, it will make introductions to the rest. For more details, please see:
http://wiki.tangosol.com/display/COH33UG/well-known-addresses
Using well-known-addresses (WKA) completely disables multicast, both for discovery of members and for communication of data. Discovery itself is a fairly inexpensive process. While Coherence relies mostly on point-to-point communication (it automatically selects the most efficient combination of unicast and multicast), there are use cases that require messages to be sent to all nodes (for example, cluster membership information). As you can imagine, if the sending node must send the same data to all other nodes, using unicast is going to be far more expensive that multicast as the cluster size increases (e.g. we've found that in clusters of 1000+ JVMs, the load on the sending member is a significant problem).
Another way of looking at this ... for one-to-all communications (or even one-to-many), with multicast enabled, the switch is effectively acting as a "hardware accelerator" ... if you disable multicast, then you are emulating that behavior in software, which is obviously going to be less efficient (the sender will have the send the same packet to the switch N times (e.g. 1000 in the example above) rather than once).
Fortunately, for a lot of workloads, this isn't as critical, especially for ones that have strong data affinity (e.g. only a few nodes are working with each piece of data).
What is the amount of multicast traffic that is handled by a member of the Coherence cluster which participates without having any local data stored? Is every message that is sent to the data storage members also sent to the non-data members
Aside from membership events (which are global), storage-enabled nodes really only use multicast to notify other nodes when data changes, and this only occurs if multicast is deemed more efficient than unicast (e.g. if there are a lot of subscribers interested in the changed data).
Storage-disabled nodes (clients) will receive (some) multicast traffic for change events. If there isn't good data affinity (e.g. a lot of client-side near caches have overlapping data), then chances are that the storage nodes will be broadcasting data updates rather than sending them individually, and so there will be some multicast packets that need to be ignored (which is relatively inexpensive).
Jon Purdy
Oracle