Low latency, high throughput RMI, Any suggestions welcome.

Question

I am working on a module which is designed to support low latency synchronous RMI and high throughput asynchronous RMI.
It uses TCP, NIO, custom serialization, custom RMI, Java 5 concurrency thread pools.
Its current latency is 5 to 15 micro-seconds slower than ping times (depending on the arguments/return values). For throughput, it gets 80K-140K calls per second on a single channel.
I have looked at JGroups and Stream.
I was wondering if any one has tried this sort of thing, what suggestions you might have and what open source libraries I might use for compassion.

Answer 1

ejp wrote:
Using custom serialization most messages are under 256 bytes, if I send smaller packets of 128 bytes, it doesn't make much difference.I wouldn't expect it to make any difference. An Ethernet packet is around 1500 bytes. Whatever amount you sent below that is still an Ethernet packet.The maximum packet size is around 1500 bytes, however it won't send more data than needed. If you send ping with different packet sizes you will see different average latencies.
I have used the generic RMI Proxy class, and use simple Reflections to lookup methods to call on the server. I believe I can cut 2-4 microseconds if I generate my own proxies and method callers. This is a bit of work generating byte code for not much saving.You will save time by not using reflection at all.Correct, but for a generic implementation I need to use something.
I was also wondering how much work had been done using non-blocking RMI calls.RMI calls aren't non-blocking. Even if the methods are void they still have to return a a success/failure indication, e.g. 'true' or an exception. You can make them asynchronous by using more threads, but that's more cost.This is a custom implementation, not the built in Java RMI which is much slower (and a pain to use IMHO). When a callback is provided as the first argument, the result is processed in another thread as the results becomes available. In this way it can support many concurrent requests over the same channel. Yes, an additional thread is required and there is a small impact on latency, however asynchronous calls can reduce the impact of latency on an application.
CORBA supports async calls and you may be able to get at that via Java IDL and idlj. Not with RMI/IIOP though.
This is how I get the high throughputs on a single channel.But you're not in control of the number of channels. RMI will use as many channels (i.e. TCP connections) as it needs to execute whatever remote calls are in progress. Making your own calls 'nonblocking' or async doesn't really affect that.I wrote the custom RMI so I make sure I control the number of channels. :) In fact I use different channels for synchronous, asynchronous calls, and asynchronous events (where the server pushes events to the client) as required.