Why do I get IOException on nio read from channel before all data is read?

Question

I am working on an NIO based http client library but I've noticed two problems.
#1: I have been testing against an http server that has authentication turned on. Hence, any request that comes its way will respond with a '401 Authentication Required' response. In the case of a POST, I've noticed that this particular http server does not wait for the body to arrive before responding. As soon as it gets the headers, it dumps the 401 response onto the channel and closes the connection. My library then gets an IOException on the next write to the channel even though I've received write interest event. (I'm using non blocking mode)
Whether I get the exception depends on timing. Sometimes, my writes work and the exception is averted. Sometimes not. The longer I delay since the time I get the write interest event, the more likely it is the exception will occur.
Now, in this situation, I really think it's the http server that is violating the http protocol. I give a valid content length that is not zero but it does not wait for me to finish my request. The moment I write the last empty line of the header section, it responds and closes the channel. Even though I don't think this is correct, my library has to deal with it. So in this situation, I've decided my library will abandon the write phase and move on to reading the response.
NOTE: In the majority of cases, a client would have already authenticated by landing on some page that could produce a POST in the first place. However, for technical reasons I can't mention here, I must support the case that the very first (unathenticated) request to the server is a POST.
2) After #1 above occurs, reading from the channel will also throw an IOException but only on Windows. Sometimes I get the headers only, sometimes the whole document, sometimes nothing. Again, the longer I delay reading (since the time I get the read interest event) the more likely it is the exception will be thrown. This NEVER happens on Linux. I always get the full response on Linux after the #1 happens even if I put long delays between reads.
I can understand #1 happening but #2 should not. I can always see the complete response data on the wire using a network protocol analyser. The data is there every time but my library doesn't always get it.
In situation 2, there is nothing I can do. If I don't receive the data, I have to respond with an error even though the request was successful.
I have stripped down my library to the bare essentials and can reproduce the problem in a self-contained environment every time.
Any ideas why the discrepency between Windows/Linux? Does this seem right?

Answer 1

I read through the section you mentioned. I realize the connection may be dropped at any time. Handling this is no problem. It is reported as an exception to the client. However, this situation is different. I still see no provision that allows a server to respond to a half written request. A request is defined to include an optional message body (Section 5). The client dictates whether it is optional, not the server. Furthermore, section 6 clearly states "After receiving and interpreting a request message, a server responds with an HTTP response message.". To me, this implies the server must wait for the entire request before responding even if its a waste of time. This particular server is attempting to short circuit what I see is the required request/response cycle. In doing so, it is causing me some pain in trying to deliver its response since it seems on windows this interferes with my ability to read it.
The simple answer to this is to process reads before writes, i.e. OP_READ before OP_WRITE. I tried this as well. Originally, I was not even registering READ interest while I was writing my request. (I saw no reason to. I don't want to give my client a response while they are writing their request.) But even when I do register READ interest while I'm writing (and process read events before write), the read event still produces no data and no eof (-1) indication on the read. The discrepency between linux and windows is bothering me. No matter what I do, I can't deliver the response even though its written to the channel by my peer. I think I will supply some code and see if anyone can get it to work on windows.Edited by: r2rossi on Sep 24, 2008 10:42 AMEdited by: r2rossi on Sep 24, 2008 10:43 AM
Edited by: r2rossi on Sep 24, 2008 10:45 AM
Edited by: r2rossi on Sep 24, 2008 10:46 AM