How to extract elements from an HTML page?

Question

Hi:
If we have the following returned page which is the web query result from a library: http://library.fsi.uiuc.edu/dbtw-wpd/exec/dbtwpub.dll?XC=%2Fdbtw-wpd%2Fexec%2Fdbtwpub.dll&BU=http%3A%2F%2Flibrary.fsi.uiuc.edu%2Fdbtw-wpd%2FnewWeb%2Fauthor.asp&submit=Submit+Query&QB0=AND&QF0=Author&QI0=william&QB1=AND&QF1=Corporate+Author&QI1=&MR=20&TN=Catalog&DF=Display-Web&RF=Brief+Title+-+Web&DL=1&RL=1&NP=1&AC=QBE_QUERY
If we want to store all the returned information in the local memory,
how can Java be used to extract the records out of the html page?
What kind of classes should be used?
For example, should I just use regular expression to do this (java.util.regex class),
build-in html parser (javax.swing.text.html.parser.Parser, or even javax.swing.text.html.HTMLEditorKit.Parser), or any other classes? Thanks.

Answer 1

I been working on a simular project but i might not be using a method that you would want.
I am reading the webpage code and then getting the data by looking for certain words. The problem with this is that it is reading the page "blindly" so it could read the wrong data or screw up on an error on the page. but here is a bit of code i am using. I am using ti to get data that appears in " " (quotes) so i can get urls, images, etc
This code isnt totaly mine, i amended it from some one and dont know the original owner :(
try {
        is =                    new BufferedInputStream(new URL(jTextField1.getText()).openStream());
      while ((record = is.read()) != -1) {
        buffer.append((char) record);
        length++;
        System.out.print((char)record);
      }     is.close();
} catch // error management herei am using a seperate thread running at same time to enter data
do {
        while (lengthposition != length) {
          //System.out.print(buffer.charAt(lengthposition));
          temp[0] = "" + buffer.charAt(lengthposition);
          if (buffer.charAt(lengthposition) != /*34*/64 && tolist == true)
            temp[1] += ""+ buffer.charAt(lengthposition);
          else if (buffer.charAt(lengthposition) == /*34*/64 && tolist == false)
            tolist = true;
          else if (buffer.charAt(lengthposition) == /*34*/64 && tolist == true) {
            tolist = false;
            list1.addItem(temp[1]);
            temp[1] = "";
          lengthposition++;
      } while (buffer.charAt(lengthposition) != 32);I hope you find out a solution to your problem :)