URL Performance Issue

Question

Folks,
I have the following code:
URL url = new URL("http://www.apage.com/ws/" + txtHTML.getText());
URLConnection urlConnection = url.openConnection();
BufferedReader htmlPage = new BufferedReader(new InputStreamReader(url.openStream()));
read the html page....I would like to run this 1000's of times. What would be quicker? How can I speed it up? I know that I can use threading, however this code itself is taking too long.
Thanks
Angus

Answer 1

I've been using a package HTTPClient http://www.innovation.ch/java/HTTPClient/ as the JDK classes are a bit inflexible and (IMHO) rather unstable at the sort of useage rates you are talking about.
Using this package (I think you can also do this with the JDK classes) under HTTP 1.1 you can create a single connection and request lots of URL's throught it. This improves performance by only creating the connection once for your 1000s of pages rather than 1000s of times.
Code goes something like this;
HTTPConnection conn = new HTTPConnection(host, port);
while (true) {
request = "/app/page.jsp?param=value";
try {
HTTPResponse resp = conn.Get(request);
} catch (Esception ex) {
In my case I'm using it to stress test and app server.
HH