Solaris HTTP - MS IIS EOF bug, timeout failures

Question

Greetings,
Experiencing what appears to be an "HTTP EOF" problem
running on Solaris 2.7 (Apache ws) for several months.
When my Perl script requests image files, from select remote
servers, the requests hang as if waiting on the last packet to be received.
The images are exclusivly owned by the people that run my cgi script. This
is strictly enforced.
Problem is recreated it using LWP, WGET and LYNX,all fail
the same way. (please see trace data below)
I have a simple script with blocks of urls from several sites
using LYNX with trace on that shows the failure every time.
(please see below)
It works on all non Solaris platforms that I, and others, have tried
(WIN, AIX, etc)
An HTTP 200 is always recieved, with the proper length specified in the
headers.
The hang occurs on what appears to be the last packet (as seen from
LYNX trace)
Sometiimes requests complete after several minutes, with a partial image
file. Occasionally a few requests work fine.
I've seen other newsgroup appends with the exact problem, no solution
Others equate the problem strictly with requests from Solaris to an MS
web server (see related append below)
I can't get my web hosting company to move on this, they've run my
sample app and agree "there's a problem". I've appended this on several
newsgroups in the past, others have run my script, lot of discussion and agreement that there's a problem but to date no solutions.
Looks like an operating system bug to me.
I've tried searching Sun's site for a possible answer but got lost each
time. Not sure how to searh for known fixes, assume I can't open a problem report since I don't own the OS license.
Another related append is included below along with my test script and
trace output.
Any assistance or advice is most appreciated.
Thank you,
Steve Borruso
From a libwww newsgroup append .....On Solaris, an LWP request sometimes fails (ie. reports a server error)
when the server is fine, b/c of some interaction between the OS and the
server response. (See below for more info.) In fact, LWP is receiving data,
but the data is improperly terminated and LWP times out.
There is an interaction between Solaris and certain web servers (notably
Microsoft-IIS/4.0) where the response to a GET comes back improperly
terminated. You can observe this independently of LWP by doing a "telnet
www.ndu.edu 80",
and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
You will see telnet receive the response (it's a 404), but then (usually
but not always) fail to terminate. As far as I know, you will only see this
on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried
same under Linux and haven't seen this problem. I have seen this on a
number of other sites.
Trace output from a past run of my script ...
GET /Sell/SSProfiles/10011921/Images/hugedragonbeads.JPG HTTP/1.0
Host: images.auctions.goto.com
Accept: text/html, text/plain, text/sgml, text/x-sgml,
application/x-wais-source
, application/html, */*;q=0.001
Accept-Encoding: gzip, compress
Accept-Language: en
Negotiate: trans
User-Agent: Lynx/2.7.1 libwww-FM/2.14
Sending HTTP request.
HTTP: WRITE delivered OK
HTTP request sent; waiting for response.
HTTP: Trying to read 1023
HTTP: Read 1023
Read 1023 bytes of data.
HTTP: Rx: HTTP/1.1 200 OK
HTTP: Scanned 2 fields from line_buffer
--- Talking HTTP1.
HTTP/1.1 200 OK
HTFormat: Constructing stream stack for www/mime to www/present
StreamStack: found weak wildcard match: www/present
StreamStack: found exact match: www/mime
HTMIME: Server: Microsoft-IIS/4.0
Date: Fri, 24 Nov 2000 02:38:17 GMT
Content-Type: image/jpeg
Accept-Ranges: bytes
Last-Modified: Tue, 15 Aug 2000 22:32:20 GMT
ETag: "38bfa4ac87c01:1165"
Content-Length: 17935
����
HTMIME: Got 'S' at beginning of line, state now S
HTMIME: Was S, found E, state now SE'
HTMIME: Was SE, foun

Answer 1

<pre>
Sending trace output and code again (hopefully formatted)
Trace output from a past run of my script ...
GET /Sell/SSProfiles/10011921/Images/hugedragonbeads.JPG HTTP/1.0
Host: images.auctions.goto.com
Accept: text/html, text/plain, text/sgml, text/x-sgml,
application/x-wais-source
, application/html, */*;q=0.001
Accept-Encoding: gzip, compress
Accept-Language: en
Negotiate: trans
User-Agent: Lynx/2.7.1 libwww-FM/2.14
Sending HTTP request.
HTTP: WRITE delivered OK
HTTP request sent; waiting for response.
HTTP: Trying to read 1023
HTTP: Read 1023
Read 1023 bytes of data.
HTTP: Rx: HTTP/1.1 200 OK
HTTP: Scanned 2 fields from line_buffer
--- Talking HTTP1.
HTTP/1.1 200 OK
HTFormat: Constructing stream stack for www/mime to www/present
StreamStack: found weak wildcard match: www/present
StreamStack: found exact match: www/mime
HTMIME: Server: Microsoft-IIS/4.0
Date: Fri, 24 Nov 2000 02:38:17 GMT
Content-Type: image/jpeg
Accept-Ranges: bytes
Last-Modified: Tue, 15 Aug 2000 22:32:20 GMT
ETag: "38bfa4ac87c01:1165"
Content-Length: 17935
����
HTMIME: Got 'S' at beginning of line, state now S
HTMIME: Was S, found E, state now SE'
HTMIME: Was SE, found R, checking for 'ver'
HTMIME: PICKED UP Server: 'Microsoft-IIS/4.0'
HTMIME: Got 'D' at beginning of line, checking for 'ate:'
HTMIME: PICKED UP Date: 'Fri, 24 Nov 2000 02:38:17 GMT'
HTMIME: Got 'C' at beginning of line, state now C
HTMIME: Was C, found O, state now CO'
HTMIME: Was CO, found N, state now CON
HTMIME: Was CON, found T, checking for 'ent-'
HTMIME: in case CONTENT_
HTMIME: Was CONTENT_, found T, state now CONTENT_T
HTMIME: in case CONTENT_T
HTMIME: Was CONTENT_T, found Y, checking for 'pe:'
HTMIME: PICKED UP Content-Type: 'image/jpeg'
HTMIME: Got 'A' at beginning of line, state now A
HTMIME: Was A, found C, checking for 'cept-ranges:'
HTMIME: PICKED UP Accept-Ranges: 'bytes'
HTMIME: Got 'L' at beginning of line, state now L
HTMIME: Was L, found A, checking for 'st-modified:'
HTMIME: PICKED UP Last-Modified: 'Tue, 15 Aug 2000 22:32:20 GMT'
HTMIME: Got 'E' at beginning of line, state now E
HTMIME: Was E, found T, checking for 'ag:'
HTMIME: PICKED UP ETag: '38bfa4ac87c01:1165'
HTMIME: Got 'C' at beginning of line, state now C
HTMIME: Was C, found O, state now CO'
HTMIME: Was CO, found N, state now CON
HTMIME: Was CON, found T, checking for 'ent-'
HTMIME: in case CONTENT_
HTMIME: Was CONTENT_, found L, state now CONTENT_L
HTMIME: in case CONTENT_L
HTMIME: Was CONTENT_L, found E, checking for 'ngth:'
HTMIME: PICKED UP Content-Length: '17935'
Converted to integer: '17935'
HTMIME: MIME Content-Type is 'image/jpeg', converting to 'www/present'
HTFormat: Constructing stream stack for image/jpeg to www/present
StreamStack: found weak wildcard match: www/present
StreamStack: Using www/present
Read 437 of 17935 bytes of data.
Read 1897 of 17935 bytes of data.
Read 3357 of 17935 bytes of data.
Read 4817 of 17935 bytes of data.
Read 6541 of 17935 bytes of data.
Read 8001 of 17935 bytes of data.
Read 10209 of 17935 bytes of data.
Read 11669 of 17935 bytes of data.
Read 13877 of 17935 bytes of data.
Read 15337 of 17935 bytes of data.
Read 17140 of 17935 bytes of data.
(never completes after this, if I "cntrl C" I get a partial image file)
Here's my script .......
#!/usr/bin/perl -w
@URLValue=();
# Images from auctions.goto -
#$URLValue[0] =
'http://images.auctions.goto.com/Sell/SSProfiles/10011921/Images/hugedragonbeads.JPG';
#$URLValue[1] =
'http://images.auctions.goto.com/Sell/SSProfiles/10011921/Images/yellowribbona.JPG';
#$URLValue[2] =
'http://images.auctions.goto.com/Sell/SSProfiles/10011921/Images/jcharmsNov12E.JPG';
#$URLValue[3] =
'http://images.auctions.goto.com/Sell/SSProfiles/10011921/Images/turquoise8mma.JPG';
# Images from AOL -
#$URLValue[0] = 'http://members.aol.com/shasta4737/nicole1.jpg';
#$URLValue[