Azure WCF SSL latency problems

Question

I have a WCF service running in Azure Worker Role. The service waits for client requests, then fetchs some data from SQL database and finally returns data to client. First, service was using plain unencrypted NetTcpBinding for communication, and latency
for each request was about 200ms which was acceptable. Lately I switched binding to use SSL (still with NetTcpBinding) and latency jumped to 500ms (which was expected, of course). However, usability of client suffered greatly because clients are doing requests
very frequently (like, there could be a burst of 10 requests going on at the same time) and almost all requests are really light at server side so increase of 300ms in latency really hurt.
Now I am not sure what I could do to help latency. Connection pooling does not seem to work very well with Azure because there is 1min idle timeout and I cannot realiably tell if connection has timeouted before doing new request. Also I am not sure how connection
pooling affects load balancer and scaling out instances: if I force every client to open 10 connections to WCF service and keep connections alive artificially, is it possible that load balancer does not work as expected?
Are there any other options? I was also thinking that maybe I could use SSL only when logging in and then exchange symmetric crypto-key and afterwards use unencrypted connection and encrypt messages in code, but this is probably a bad idea (maybe it would
be secure enough but then I couldn't say that all connections are encrypted with SSL which unfortunately is requirement for me).
Thank you for help!

Answer 1

Hello,
Thank you for your answers! However, the problem still remains so let me explain it with more details.
We're developing a new version of older software for our customers. Previously customers had to own and maintain their own servers which were running our software. Because we wanted to take this burden away from customers we decided to move all servers into
(Azure) cloud.
Now imagine following scenario:
A user is 5 button clicks away from doing whatever he wants to do. Every button click has to do one query to server, usually in order to fetch some data from database. In our old application this was very fast since customers had their own servers running
literally few meters from their workstations, so latency was minimal.
Then we moved servers to cloud which is 500km (not 5 or 50 meters) away so latency jumped to approximately 200ms per query (as expected). Now clicking 5 buttons would add 1 second as latency in total, and although our customers were not particularly happy
about this, they could still accept it. However, now we have to turn SSL on, and since it seems to add about 500ms latency per query, clicking 5 buttons add 2.5 seconds as latency in total which is simply too much, and the application becomes very sluggish
to use.
We are already trying to combine small requests into big ones but it is not possible in many cases because we don't know what action user is going to take before previous action has finished. We could try to collect data how users are using the software
and make decision based on it, and although it would surely help, it would not remove the problem entirely.
That is why I am hoping I could find a way to minimize the latency between client and server running in Azure cloud. Biggest part of latency comes from TCP or SSL handshaking: opening a channel, doing a request and closing the channel takes about 200ms when
using plain TCP and 500ms when using SSL. However, if I don't close the channel but instead of reuse it, latency is only about 60ms. The problem is that Azure tries to prevent me from reusing channels. In perfect world I would simply open 10 SSL connections
whenever application is started and reuse these channels throughout the lifetime of application, as this would result in approx. 60ms latency in every request (I chose number 10 simply because 10 simultaneous queries might be possible in some
cases). Now, 60ms is a LOT shorter time period than 500ms or even 200ms!
Now back to my original questions. Azure has 1min timeout for idle channels but there are libraries (e.g. http://code.msdn.microsoft.com/WCF-Azure-NetTCP-Keep-Alive-09f50fd9) that keep channels artificially open for longer time by sending empty packets every
X seconds so that Azure thinks that channel is active. We are already using this library to prevent Azure from closing channels during some of our longer queries that take more than 1min to finish (like generating a monthly report with lots of data). Now I
was wondering if I could extend this functionality so that I could open 10 connections at startup, keep them alive for X minutes (5, 10, 30, whole day?) and reuse them, as this would reduce latency to a good level. Some problems:
1) How is load balancer affected if I keep channels alive? What if I simply open 10 channels per user (lets say there are 1000 users) is it possible that all channels (10000) are opened to same server instance as there is no significant CPU load on the server
(at this point)?
2) If Azure still closes the connection for some reason, it is hard to know what happened at client side. Basically it seems that if connection is closed and I try to reuse it (note that at client side WCF does not know that connection has been closed before
I try to use that connection) WCF simply throws "timeout exception" with absurd timeout value (e.g. if I have configured timeout to be 5mins WCF throws exception "timeout 0.001 seconds"). Now I could catch this exception and parse the timeout value and see
if it is "too low" and then decide that "ok, Azure closed this channel, I will close it and open new one" but this seems hacky (see 4).
3) Is it ok to open this many connections to Azure? Would Azure think that it is under DoS attack if this many connections are opened in small time window (usually ppl come to work at specific time and so basically the connections would be opened in small
time window). Also, we expect our userbase to grow to at least 10000 simultaneous users (after which this approach would require 100k simultaneous channels)
4) This feels very hacky! Is it really so that applications don't usually behave like ours do? Are there other ways to achieve what I want?
Thanks!