File transfer using non-blocking sockets - data mysteriously vanish

Question

Hello,
I need to pass some big amount of data using sockets. I my appliaction I have noticed that sometimes I didn't get all bytes. To check it out I wrote simple client + server to figure out what is happening. So:
- I have a sender and receiver application
- I'm trying to transfer 5MB text file.
- On receiver side, output file is never larget than 3MB
- If I put some timeout on sender side (1ms timeout between write operations) everything works fine.
Could someone tell me what I do wrong? Why data disappears and when? The same file transfered using old sockets goes always fine...
Thanks in advance!
Here is complete source for receiver and sender:
RECEIVER:
import java.io.FileOutputStream;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
public class ReceiverA {
      * @param args
     public static void main(String[] args) {
          String outputFile = "c:\\outputA.txt", host = "127.0.0.1";          
          int port = 8001, bufferSize = 10240;
          try {
               ByteBuffer buffer = ByteBuffer.allocate(bufferSize);
               Selector selector = Selector.open();
               ServerSocketChannel server = ServerSocketChannel.open();
               server.configureBlocking(false);
               server.socket().bind(new InetSocketAddress(host, port));
               server.register(selector, SelectionKey.OP_ACCEPT);
               System.out.println("Server started");
               while(true)
                    selector.select();
                    Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
                    while (iterator.hasNext()) {
                         SelectionKey key = (SelectionKey) iterator.next();
                         iterator.remove();
                         if (key.isAcceptable()) {                              
                              SocketChannel client = server.accept();
                              client.configureBlocking(false);
                              client.register(selector, SelectionKey.OP_READ);                              
                              continue;
                         SocketChannel channel = (SocketChannel) key.channel();
                         int counter = 1;
                         if ( key.isReadable() ) {
                              FileOutputStream os = new FileOutputStream(outputFile);
                              int res;
                              do
                                   buffer.clear();
                                   res = channel.read(buffer);
                                   counter += res;
                                   System.out.println(res);
                                   buffer.flip();
                                   os.write(buffer.array(), 0, buffer.limit());
                              while( res >= 0 );          
                              channel.close();
                              os.close();          
                              System.out.println("Receiver: " + counter);
                              return;
          } catch (Exception e) {
               e.printStackTrace();
}SENDER:
import java.io.FileInputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
public class SenderA {
     public static void main(String[] args) {
          String inputFile = "c:\\inputA.txt" , host = "127.0.0.1";          
          int port = 8001, bufferSize = 10240;
          try {
               ByteBuffer buffer = ByteBuffer.allocate(bufferSize);
               byte[] byteArr = new byte[buffer.capacity()];
               Selector selector = Selector.open();
               SocketChannel connectionClient = SocketChannel.open();     
               connectionClient.configureBlocking(false);
               connectionClient.connect(new InetSocketAddress(host, port));
               connectionClient.register(selector, SelectionKey.OP_CONNECT);
               while(true) {
                    selector.select();
                    Iterator<SelectionKey> iterator = selector.selectedKeys()
                              .iterator();
                    while (iterator.hasNext()) {
                         SelectionKey key = (SelectionKey) iterator.next();
                         iterator.remove();
                         SocketChannel client = (SocketChannel) key.channel();
                         if (key.isConnectable()) {
                              if (client.isConnectionPending()) {
                                   System.out.println("Trying to finish connection");
                                   try {
                                        client.finishConnect();
                                   } catch (IOException e) {
                                        e.printStackTrace();
                              client.register(selector, SelectionKey.OP_WRITE);
                              continue;
                         if(key.isWritable()) {
                              FileInputStream is = new FileInputStream(inputFile);
                              int res;
                              int counter = 0;
                              do
                                   buffer.clear();
                                   res = is.read(byteArr, 0, byteArr.length);                                   
                                   System.out.println(res);
                                   if ( res == -1 ) break;
                                   counter += res;
                                   buffer.put(byteArr, 0, Math.min(res, buffer.limit()));
                                   buffer.flip();
                                   client.write(buffer);
                                    * When I remove comment file transfer goes OK!
                                   //Thread.sleep(1);
                              while( res != -1 );          
                              client.close();
                              is.close();          
                              System.out.println("Receiver: " + counter);
                              return;
          catch(Exception e) {
               e.printStackTrace();
}

Answer 1

There are at least two problems here.
A. In the receiver you should test for -1 from the read() immediately, rather than continue with the loop and try to write -1 bytes to the file.
B. In the sender you are ignoring the return value of client.write(), which can be anything from 0 to the buffer length. If you get 0 you should wait for another OP_WRITE to trigger; if you get a 'short write' you need to retry it until you've got nothing left to write from the current buffer, before you read any more data. This is where the data is vanishing.