Accessing seured website using java URL/Socket programming

Question

Hi,
I am working on an intranet search engine using java URL and Authenticator class. This is a secured web-site using SSL in http server. I am not able to access the password protected pages. Following is the code:
URL url = new URL("https://*************");
URLConnection connection = url.openConnection();
PasswordAuthentication pa = Authenticator.requestPasswordAuthentication( InetAddress.getByName(connection.getURL().getHost()),
connection.getURL().getPort(), connection.getURL().getProtocol(), null, "Basic");
StringBuffer buf = new StringBuffer("*****************");
buf.append(":");
buf.append("************");
String encoded = new sun.misc.BASE64Encoder().encode(buf.toString().getBytes());
connection.setRequestProperty ("Authorization", "Basic" + encoded);
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
Let me know if there is any other alternative
tnx,
Biswajit Biswal

Answer 1

Below is my first test program. I first take the home page of a website (this can be automated later) and save it to "myTest.html". Then I try to see if this page contains the keywords that a company website often has.
Could you give your opinion on this approach?
If this approach is fine, then any idea about the keywords to select a company website from other sites is welcome.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
public class MyTest {
      * @param args
      * @throws IOException
     public static void main(String[] args) throws IOException {
          InputStream in = MyTest.class.getResourceAsStream("/myTest.html");
          Reader reader = new InputStreamReader(in, "UTF-8");
          BufferedReader bufferedReader = new BufferedReader(reader);
          String line;
          while ((line = bufferedReader.readLine()) != null) {
               line=line.toLowerCase();
               if (line.contains("contact us")) {
                    System.out.println("contact us: line = "+line);
               if (line.contains("career")) {
                    System.out.println("career: line = "+line);
               if (line.contains("company")) {
                    System.out.println("company: line = "+line);
               if (line.contains("about us")) {
                    System.out.println("about us: line = "+line);
               if (line.contains("product")) {
                    System.out.println("product: line = "+line);
          bufferedReader.close();
          reader.close();
}