POST in Java

Question

Hello,
I am trying to read a web page in Java that requires a POST. I can read normal web pages but I cannot determine how to POST data to a page.
I am trying to read is:
http://www.ohr.psu.edu/EMPLMENT/faculty.cfm
If you use the drop down menu at the top you will notice you can select job descriptions based on the locations listed in the drop down menu.
However when you select on these pages and click the Change button, you are redirected to the same URL but with different results. According to the HTML, this new page depends on a POST. How do I access the new pages requiring the post methods?
I can see in the HTML the values passed to the POST page. However, I cannot figure out how to pass these values to the URL in java. I have attached my code below. Again, I can connect to the first page, collect the POST required values from the drop down menu, but cannot determine how to post one of these values.
What I am attempting to do in the code below is use a regular expression to extract the POST values from the drop down menu and then connect to the POST page using the value. However, I have not met success. The problem arise where I begin to comment my code : ( I would really appreaciate any help.
import java.net.URLConnection.*;
import java.net.HttpURLConnection.*;
import java.net.*;
import java.io.*;
import java.util.regex.*;
public class test
    private StringBuffer hyperStringBuff;
    private URL url;   
    private URLConnection urlConn;   
    private BufferedReader buffReader;
    private String allHtml;
    private StringBuffer secHyperStringBuff;
    private URL secUrl;
    private HttpURLConnection secUrlConn;
    private BufferedReader secBuffReader;
    private String secAllHtml;
    public String doIt() throws MalformedURLException, UnknownHostException, IOException
        hyperStringBuff = new StringBuffer();
        url = new URL("http://www.ohr.psu.edu/EMPLMENT/faculty.cfm");
        urlConn = url.openConnection();
        urlConn.connect();
        buffReader = new BufferedReader(new InputStreamReader((urlConn.getInputStream())));
        String line;
        while((line = buffReader.readLine()) != null)
        { hyperStringBuff.append(line); }
        buffReader.close();    
        allHtml = hyperStringBuff.toString();
        //get the options
        Pattern optionPatt = Pattern.compile("<OPTION VALUE=(.+?)>(.+?)\\s*?<");
        Matcher optionMatch = optionPatt.matcher(allHtml);
        //output all matches
        while(optionMatch.find())
            //System.out.println(optionMatch.group(1) + " " + optionMatch.group(2));
            //attempt to post with the option value (group 1)
            //create a new URL connection
            secUrl = new URL(url.toString());
            secUrlConn = (HttpURLConnection)secUrl.openConnection();
            //try to post
            secUrlConn.setDoInput(true);
            secUrlConn.setRequestProperty("name", optionMatch.group(1));
            secUrlConn.setRequestMethod(secUrlConn.getRequestMethod());
            //read the hypertext into a buffer
            secBuffReader = new BufferedReader(new InputStreamReader((secUrlConn.getInputStream())));
            //read line by line
            String secLine;
            while((secLine = secBuffReader.readLine()) != null)
                { secHyperStringBuff.append(secLine); }
        }   //end while find()       
        return allHtml;
    public static void main(String args[])
        test testObj = new test();
        try
        { testObj.doIt(); }
        catch (Exception e)
        { System.out.println(e); }
        System.out.println("Done ; )");
    }   //end main
}   //end class test

Answer 1

this worked for me
System.setProperty("http.proxyHost", "proxy");
System.setProperty("http.proxyPort", "82");
URL url;
//communications link between the application and a URL
URLConnection urlConn;
/*A data output stream lets an application write primitive Java
data types to an output stream in a portable way. An application
can then use a data input stream to read the data back in. */
DataOutputStream printout;
DataInputStream input;
url = new URL ("http://tech-supp.esatclear.ie/cgi-bin/dslinfo");
urlConn = url.openConnection();
urlConn.setDoInput (true);
urlConn.setDoOutput (true);
// turns off caches
urlConn.setUseCaches (false);
urlConn.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
printout = new DataOutputStream (urlConn.getOutputStream ());
String content = "cli=" + URLEncoder.encode(getNumber());
printout.writeBytes (content);
printout.flush();
printout.close();
input = new DataInputStream (urlConn.getInputStream ());
String str;
String urlString = "\n\n";
while (null != ((str=input.readLine())))
System.out.println (str);
urlString += str;
StringBuffer sb = new StringBuffer(urlString);
if (sb.indexOf("No cli") != -1) {
JOptionPane.showMessageDialog(null, "Error: user does not exist in database","User Details", JOptionPane.ERROR_MESSAGE);          
else { 
createFrame(urlString);
input.close();
searchField.setText("");