Problems in Downloading a file from a web site using HttpClient

Question

Hi,
My requirement is to download a file from a website using a java program. I have written a program using HttpClient API but I am unable to do so. Reality is that I don't know how to proceed with the HttpClient API to get the file downloaded. Same file can also be downloaded manually by login to that website, below mentioned steps are to be followed to download the file manually.
1. Login to the website using Login/Password (I have valid login/password)
2. Two options (links) are there (a) Report (b) Search [I am chosing report option]
3. Newly opened window shows two tabs (a) Today (b) From & To [I am selection Today tab]
4. Every tab has two radio button (a) File (b) Destination [Destination is selected by me]
5. When Search button is pressed. A link gets created when that link is clicked file download popup is opened which allows facility to open/save the file.
All these 5 step are to be followed in a java program to download the file. Please help in doing the same.

Answer 1

// first URL which is used to open the website, this will have two text fields userName and password
String finalURL = "http://www.xyz.in/mstatus/index.php";
SMSGatewayComponentImpl obj = new SMSGatewayComponentImpl();
obj.sendMessage(finalURL);
public void sendMessage(String finalURL) {
String ipAddrs = "a.b.c.d";
int port = 8080;
boolean flag = false;
try {
// Create an instance of HttpClient.
HttpClient client = new HttpClient();
HostConfiguration hostConfig = new HostConfiguration();
hostConfig.setProxy(ipAddrs,port);
client.setHostConfiguration(hostConfig);
// Create a method instance.
String cookieName=null;
String cookieValue=null;
// Here URL for the login page is passed and passing the user/password
PostMethod method = new PostMethod(finalURL);
method.setRequestHeader("userid","userName");
method.setRequestHeader("passwd","pwd");
// Execute the method.
int statusCode = client.executeMethod(method);
Cookie[] cookies = client.getState().getCookies();
for (int i = 0; i < cookies.length; i++) {
Cookie cookie = cookies;
cookieName = cookie.getName();
cookieValue = cookie.getValue();
System.err.println(
"Cookie: " + cookie.getName() +
", Value: " + cookie.getValue() +
", IsPersistent?: " + cookie.isPersistent() +
", Expiry Date: " + cookie.getExpiryDate() +
", Comment: " + cookie.getComment());
NameValuePair[] respParameters = method.getResponseHeaders();
String cookie = "";
for(NameValuePair o : respParameters){
     System.out.println("Name : "+o.getName());
     System.out.println("Value : "+o.getValue());
     if("Set-Cookie".equalsIgnoreCase(o.getName()))
          cookie = o.getValue();
NameValuePair[] footParameters = method.getResponseFooters();
System.out.println("****************** Footer Values *******************");
for(NameValuePair o1 : footParameters){
System.out.println("Name : "+o1.getName());
System.out.println("Value : "+o1.getValue());
// This is jthe URL which comes when login/passowrd is entered and Login button is pressed.
// I am trying to get the cookie from the first URL and pass this cookie for the second URL so that the session can be maintained
// Here I may be wron..don't know is this the right way to download the file like this.....????
finalURL = "http://www.xyz.in/mstatus/mainmenugrsms.php";
     method = new PostMethod(finalURL);
     method.setRequestHeader(cookieName,cookieValue);
     method.setRequestHeader("userid","userName");
     method.setRequestHeader("passwd","pwd");
     method.setRequestHeader("Set-Cookie",cookie);
     statusCode = client.executeMethod(method);
     respParameters = method.getResponseHeaders();
for(NameValuePair o : respParameters){
System.out.println("Name : "+o.getName());
     System.out.println("Value : "+o.getValue());
// and this is the final URL which i got when that final link which enabled file download from the website have been copied as a shortcut and
// pasted in a notepad. I was thinking that this will return the file as an input stream. but its not happening.
     finalURL = "http://www.xyz.in/mstatus/dlr_date.php#";
     method = new PostMethod(finalURL);
     method.setRequestHeader("Set-Cookie",cookie);
     method.setRequestHeader("userid","userName");
// userid and passwd field are obtained when login/password page contents are seen using view source of that html
     method.setRequestHeader("type","1");
     // trying to set the cookie so that session can be maintained
method.setRequestHeader(cookieName,cookieValue);
     method.setRequestHeader("passwd","pwd");
     statusCode = client.executeMethod(method);
     ObjectInputStream objRetInpuStream = new ObjectInputStream(method.getResponseBodyAsStream());
     System.out.println("objRetInpuStream : "+objRetInpuStream);
     if(objRetInpuStream!=null)
     System.out.println("objRetInpuStream available bytes : "+objRetInpuStream.available());
     String returnFile=(String)objRetInpuStream.readObject();
     System.out.println("Returned value \n : "+returnFile);
     respParameters = method.getResponseHeaders();
     for(NameValuePair o : respParameters){
     byte[] responseBody = method.getResponseBody();
     System.out.println("Response Body : "+new String(responseBody));
     if (statusCode != HttpStatus.SC_OK) {
          System.out.println("Error: " + method.getStatusLine());
     } else {
          System.out.println(method.getStatusLine());     
     } catch(Exception nfe) {
               System.out.println("Exception " + nfe);
Output
=====
/home/loguser/batch> sh run.sh SMSGatewayComponentImpl
Classname : SMSGatewayComponentImpl
run.sh[4]: test: 0403-004 Specify a parameter with this command.
final URL : http://www.xyz.in/mstatus/index.php
client is :org.apache.commons.httpclient.HttpClient@190e190e
Cookie: PHPSESSID, Value: anqapu83ktgp8hlot06jtbmdf1, IsPersistent?: false, Expiry Date: null, Comment: null
Name : Date
Value : Thu, 06 May 2010 09:08:47 GMT
Name : Server
Value : Apache/2.2.3 (Red Hat)
Name : X-Powered-By
Value : PHP/5.1.6
Name : Set-Cookie
Value : PHPSESSID=anqapu83ktgp8hlot06jtbmdf1; path=/
Name : Expires
Value : Thu, 19 Nov 1981 08:52:00 GMT
Name : Cache-Control
Value : no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Name : Pragma
Value : no-cache
Name : Content-Length
Value : 4792
Name : Content-Type
Value : text/html; charset=UTF-8
Name : X-Cache
Value : MISS from dcp.pas.abc.in
Name : X-Cache-Lookup
Value : MISS from dcp.pas.abc.in:8080
Name : Via
Value : 1.0 dcp.pas.abc.in:8080 (squid/2.6.STABLE21)
Name : Proxy-Connection
Value : keep-alive
Cookie Value : PHPSESSID=anqapu83ktgp8hlot06jtbmdf1; path=/
****************** Footer Values *******************
Name-2 : Date
Value-2: Thu, 06 May 2010 09:08:47 GMT
Name-2 : Server
Value-2: Apache/2.2.3 (Red Hat)
Name-2 : X-Powered-By
Value-2: PHP/5.1.6
Name-2 : Expires
Value-2: Thu, 19 Nov 1981 08:52:00 GMT
Name-2 : Last-Modified
Value-2: Thu, 06 May 2010 09:08:47 GMT
Name-2 : Cache-Control
Value-2: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Name-2 : Pragma
Value-2: no-cache
Name-2 : Location
Value-2: index.php
Name-2 : Content-Length
Value-2: 0
Name-2 : Content-Type
Value-2: text/html; charset=UTF-8
Name-2 : X-Cache
Value-2: MISS from dcp.pas.abc.in
Name-2 : X-Cache-Lookup
Value-2: MISS from dcp.pas.abc.in:8080
Name-2 : Via
Value-2: 1.0 dcp.pas.abc.in:8080 (squid/2.6.STABLE21)
Name-2 : Proxy-Connection
Value-2: keep-alive
Cookie Value second time : PHPSESSID=anqapu83ktgp8hlot06jtbmdf1; path=/
**Exception java.io.EOFException**
Is my approach to download the file fromthe website ok????? please help me.