Downloading a binary file using sockets without length known

Question

Hi all,
I'm trying to download a binary(.exe) file using socket, where the length of the file is not known. Please take a look at the code I'm using:
      var readBin = socketBin.read();
     <-- Here comes the code that checks for http Content-length header field
     ; If content-length field is available, then I'll use it to download file, else proceed with following code -->
    pBar.reset("Downloading plugin..",null);pBar.hit(readBin.length);       
            while (1)
                binFil.write(readBin);     //'binFil' is the file, into which downloaded file is written.
                readBin = socketBin.read();
                pBar.hit(readBin.length);
                if( socketBin.eof || readBin.length<=0){
                    break;}          
            binFil.write(readBin);
            binFil.close();
            socketBin.close();
Problem is: I'm able to download file within 10 seconds when content-length is known. But when content-length is not known, its taking about 1 and half minute to download, also the progress bar gets struck for much of time.
FYI: socket is opened in binary mode, file is getting downloaded correctly(even though it takes abt a minute). BTW Im using CS5.5 extendscript
I'm not able to figure out where the bug is.

Answer 1

Hi, Srikanth:
Sevaral points.
#1 When posting code, please use the web interface and the >> icon and choose Syntax Highlighting >> Java. Otherwise your code is just too hard to read and gets misformatted.
#2 Apropos of #1, your script as written does not work, because this line:
url=url.replace(/([a-z]*):\/\/([-\._a-z0-9A-Z]*)(:[0-9]*)?\/?(.*)/,"$ 1/$2/$3/$4");
has an extra space in the $1 and should be this:
url=url.replace(/([a-z]*):\/\/([-\._a-z0-9A-Z]*)(:[0-9]*)?\/?(.*)/,"$1/$2/$3/$4");
Please take an extra moment to make sure that when you asking for help, you do so in a way that makes sense. Otherwise it takes too much effort to help you, and that is frustrating.
#3 If we instrument your script by adding a line to the while() loop:
            while (1)
                binFil.write(readBin);
                readBin = socketBin.read();
                $.writeln(Date()+" Read "+readBin.length+" chars, eof is "+socketBin.eof);
                if(  readBin.length<=0){         break;}              
We get output like this:
Mon Jul 11 2011 12:06:56 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:06:56 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:06:56 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:06:56 GMT-0400 Read 631 chars, eof is false
Mon Jul 11 2011 12:07:06 GMT-0400 Read 0 chars, eof is false
Result: undefined
Therefore, we can reasonably conclude that the last read at the end of the data stream returns a short read when the other side blocks.
Unfortunately, that's clearly insufficient since you can get short reads all the time.
I'm not sure why you say the length of the file is not known, HTTP provides it to you in the Content-Length field of the response.
But the easy answer is to get the server to close it for you. This is easy, enough, with Connection: close. Why were you specifying
Connection: keep-alive, anyhow?:
            var requestBin =
            "GET /" + parsedURLBin.path + " HTTP/1.1\n" +
            "Host: " + parsedURLBin.address + "\n" +
            "User-Agent: InDesign ExtendScript\n" +
            "Accept: */*\n" +
            //"Connection: keep-alive\n\n";
            "Connection: close\n"+
            "\n";
This yields a nice tidy:
Mon Jul 11 2011 12:26:19 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:26:19 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:26:19 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:26:19 GMT-0400 Read 735 chars, eof is true
Mon Jul 11 2011 12:26:19 GMT-0400 Read 0 chars, eof is true
I suspect you're also better off using something like socketBin.read(64*1024) for a 64k buffer size, but it doesn't seem to effect the on-the-wire protocol, so perhaps its not so important.
If you don't want to reply on the server