Data loss in the secure file transfer using java

Question

Hi,
My java Appllication uses the sftp(secure file transfer protocol) for file transferring. when the load is high like 150 GB , data loss occuring for some files especially last chunk of data is missing.
Core file transfer code
while (true)
int read = bis.read(buffer); //Reading bytes into the byte array (buffer) from network stream (bis).
if (read==-1) {
break; //Break the loop, if there are no more bytes available in the InputStream.
bos.write(buffer,0,read); //Writing the same number of bytes what we read into buffer byte array into OutputStream
complete.update(buffer,0,read); //Giving the same number of bytes what we read above to the checksum calculation.
bytesSoFar += read; //Incrementing the bytes that we have read from the source.
//if the bytes transferred are equal to the source file size
If (bytesSoFar == fileInfo.getFileSize().intValue())
// Calculate the file checksum.
// Update DbAuditLog with checksum and no. of bytes read
else
// Update DbAuditLog with no. of bytes read
bos.flush();
Please help , why the data loss is happening when the big size files transfer
Note:
1.150 gb not a single file multiple files.
2.SFTP no java specific API, So connection is happening through java .io.sockect connection

Answer 1

so, if you are not getting any exceptions, yet the data is incorrect, it sounds like you are having actual network issues. i'd start looking at things like the actual network cards in the computer as well other equipment involved in the transfer. since you are not getting exceptions, the network stack obviously thinks nothing was lost over the wire, so either the actual hardware has issues, or the network stack, or the issue is on the sending end. the last possibility is that the client is somehow munging the data it is sending. do you control the client code? if so, i'd start checking the client side code for bugs (and/or post it here).
also, i'm assuming that all of the variables included in your original code are thread local (you aren't sharing any of those buffers/streams/etc between multiple threads).