Using java to interact with a website

I'm looking to build a program that will function as an alternate front end for an online database so that users can perform batched operations. Is there a way to have java navigate and interacte with a website? This would be something a macro would do, except that I need to be able to interact with the desktop, among other things. Also, I'm stuck with internet explorer, as the IT people don't want Firefox on any of the machines, so that excludes the iMacros add on.
anyone got an idea? thanks

DrClap wrote:
Unless you also control the website. However, even in that case I think that web services are superior in the long run to HTML-scraping.Agreed. I was assuming that if the website was one's own, one wouldn't resort to scraping, but you're right, that needs to be stated.

Similar Messages

Using java to interact with html.

I want to create a program that goes to html pages and presses some buttons. where do I look?

Hi duo,
please try and use the code snippet i am pasting below.The code is accessing a page and then submitting a form on the page by making the url in the code itself.
import java.io.*;
import java.net.*;
public class SubmitHTML
     public static void main(String[] args)
          System.out.println("*********** GOING TO ACCESS PAGE START ***********");
          SubmitHTML s1 = new SubmitHTML();
          s1.getHTMLPage();
          System.out.println("*********** GOING TO ACCESS PAGE END ***********");
     private void getHTMLPage()
          //phase 1
          try
               String str = "";
               char ch1=(char)0x0d; //\r
               String strURL = "http://tahiti.oracle.com/pls/tahiti/tahiti.error_search";
               URL url = new URL(strURL);
               HttpURLConnection.setFollowRedirects(false);
               HttpURLConnection urlConn = (HttpURLConnection)url.openConnection();
               //urlConn.setRequestProperty ("Content-Type", "application/x-www-form-urlencoded");
               urlConn.setDoInput(true);
               //urlConn.setUseCaches(false);
               System.out.println("LOGIN: Reading..." + urlConn.getHeaderField(0));
               System.out.println("getContentEncoding ..." + urlConn.getContentEncoding());
               System.out.println("getContent..." + urlConn.getContent());
               System.out.println("getContentType..." + urlConn.getContentType());
               BufferedReader br = new BufferedReader(new InputStreamReader(urlConn.getInputStream()));
               String line;
               boolean flag = false;
               while((line=br.readLine())!=null) {
                    //System.out.println(line);
                    str = str + line + String.valueOf(ch1);
               System.out.println("*********** THE FIRST PAGE RETURNED BY GOOGLE IS START ***********");
               System.out.println(str);
               System.out.println("*********** THE FIRST PAGE RETURNED BY GOOGLE IS END ***********");
               String strQuery = "ORA-9989";
               strURL = "http://tahiti.oracle.com/pls/tahiti/tahiti.error_search?search="+strQuery+"";
               System.out.println("The url is ::" + str);
               url = new URL(strURL);
               HttpURLConnection.setFollowRedirects(false);
               urlConn = (HttpURLConnection)url.openConnection();
               //urlConn.setRequestProperty ("Content-Type", "application/x-www-form-urlencoded");
               urlConn.setDoInput(true);
               //urlConn.setUseCaches(false);
               br = new BufferedReader(new InputStreamReader(urlConn.getInputStream()));
               str = "";
               while((line=br.readLine())!=null) {
                    //System.out.println(line);
                    str = str + line + String.valueOf(ch1);
               System.out.println("\n\n\n");
               System.out.println("*********** THE PAGE RETURNED BY GOOGLE AFTER SUBMITTING START ***********");
               System.out.println("LOGIN: Reading..." + urlConn.getHeaderField(0));
               System.out.println("getContentEncoding ..." + urlConn.getContentEncoding());
               System.out.println("getContent..." + urlConn.getContent());
               System.out.println("getContentType..." + urlConn.getContentType());
               System.out.println(str);
               System.out.println("*********** THE PAGE RETURNED BY GOOGLE AFTER SUBMITTING END ***********");
          catch (Exception e)
anjani

Using Java SE 6 with Java FX 2.2 on Mac OS X

Hello,
Has anyone tried using Java SE 6 with JavaFX 2.2 on Mac OS X?
JavaFX 2.2 System Requirements doc states that "For Mac OS X, use Java SE 7 Update 6 for Mac OS X. JavaFX 2.2 has not been formally tested with Apple's Java SE 6 implementation for Mac OS and support is offered only when used with Java SE 7 Update 6 or later.".
So I guess that means Java SE 6 may or may not work well with JavaFX 2.2 on Mac.
For some 'bugs' in my app, they simply don't exist if I switch from Java 7 to Java 6?
One such bug is mentioned in this thread: Re: Unable to play mp3 files with non-English alphabets/characters in the name
Can people with experiences in this area please share your thoughts?
Has Oracle done additional Java SE 6 and JavaFX 2.2 compatibility testings since the JavaFX 2.2 System Requirements doc was published (http://docs.oracle.com/javafx/2/system_requirements_2-2/jfxpub-system_requirements_2-2.htm)?
Thanks.

Java 6 on Mac is not supported officially and not going to be supported.
Standalone JavaFX app might work as JDK 6 on Mac was used as dev platform before JDK 7 on Mac became available.
But this was mostly on older Mac OS X version (up to 10.6) and you may easily hit something that was fixed in Java 7 to support JavaFX better.
Webstart/Plugin will not work. You need JRE 7 from Oracle.

Can I use Java Creator 2 with old JRE/JDK 1.4.2_07??

I am planning to install new Java Creator 2 but I am having JRE and JDK 1.4.2_07 and I am using mysql 3.23.55 and I want to integrate it.
Currently my OS is Windows XP SP2 platform with processor Intel Centrino 1.4 and RAM 256 MB... I know the ram is not sufficient and i am planning to upgrade it.. but the problem is can i still remain to use the old JRE and JDK 1.4.2_07 and i am also using mysql 3.23.55... all the driver and engine are out to date but can i still use it and integrate it with java creator 2?
The questions are:
1) Can I use Java creator 2 with those old engine?
2) What java compiler or development tools i can use to develop java standalone application?
Thank you

When the jar file is expanded within Eclipse, it shows that the mail.jar file is built with 1.5.0; I think? Here is the header contents of the MANIFEST.MF file, note "Created by."
Manifest-Version: 1.0
Implementation-Version: 1.4
Specification-Title: JavaMail(TM) API Design Specification
Specification-Version: 1.3
Implementation-Title: javax.mail
Extension-Name: javax.mail
Created-By: 1.5.0 (Sun Microsystems Inc.)
Implementation-Vendor-Id: com.sun
Implementation-Vendor: Sun Microsystems, Inc.
Specification-Vendor: Sun Microsystems, Inc.
SCCS-ID: @(#)javamail.mf 1.6 05/12/09
This is causing problems, because anytime I try to use this file, I get signing errors. I have several Web Services built by BAE and Plumtree that are running on 1.4.2; and therefore I am stuck with 1.4.2. I did try to remove the SUN_MICR.* files, thinkning it would resolve the signing issues, but then that caused other problems. I can provide you with code that demonstrates the problem.
Is there a place where I can get a mail 1.4 jar file built with 1.4.2? that would most likely solve my problems.

Pacman frontend in Java - Howto interact with alpm

I plan to write a(nother :-) ) GUI frontend to Pacman using Java + Swing. However I need some advice regarding how should I interact with Pacman.
Basically my requirements are:
1. Reading/Searching package database
2. Showing a download progress bar
3. Showing a progress bar when Pacman is installing/removing package(s)
As 'alpm' is a C library it's not easily accessible from Java. The only and perhaps dummest solution that I can think of is to write my own pseudo-alpm library to handle the requirements but I'd seriously rather avoid the re-work and bugs of such a process. Is there any other way?
I'd appreciate any idea/hint. TIA,
Bahman

If you have some success creating jni bindings to libalpm you might consider creating an AUR package to join the lua, perl and python packages that are there now.

Using Java 1.5 with TopLink Mapping Workbench 10g(9.0.4.9) - Exception

We're getting a ExternalClassNotFoundException when refreshing classes in 10g(9.0.4.9) workbench. The classes were recently compiled in JAVA 1.5. We have just migrated from RAD6 using Java 1.4 to RAD8 using Java 1.5. Is there a way to use this verion of the workbench with JAVA 1.5? Or is our only alternative to upgrade to a later version on the workbench.

I strongly suggest to upgrade to AS 10.1.3 to get this.
Think of future support of AS 9.0.4. You will get not critical patch updates anymore.
--olaf

Using Java DB (integrated with GlassFish) with Web Space Server

Hello everybody,
Just a question:
Instead MySQL can I use the Java DB (derby) already integrated with GF as data store?
Happy if to get an answer soon.
Thanks, aski

Frank, Thanks a lot for your quick answer.
"Maybe you should look at using database proxy users with ADF BC so you can use the J2EE authenticated user principal as the database schema to connect through. I am working on documenting this approach, which however needs some more time of writing and testing."
Yes, that's exactly what I need!!!
Can you give me some tips about how to do that.. Do you have any idea of when you will finish that document?.
By the way, I'm using FORM authentication, but I don't know how to modify what’s in the “How To Support Dynamic JDBC Credentials” document, to use the J2EE authenticated user principal as the database schema to connect through.
One of the problems is that when using this approach, I cannot get the J2EE security to redirect to the login page...
Any help would be really appreciated

Login codes using java database (validates with Microsoft Access File)

hi all pro-programmer, can you show me the code to login with the username and password using java database. When the user enters the username and password in the login page then it will go to the requested page. may i know how to do it?

no one will give you complete code.
i'll lay out the pieces for you, though:
(1) start with a User object. give it username and password attributes.
(2) write a UserDAO interface with CRUD operations for a User object.
(3) write a UserDAOImpl for your Microsoft Access database
(4) write an AuthenticationService interface
(5) write an implementation of the AuthenicationService that works with the UserDAO to authorize a User.
Use a servlet to accept request from your login page and pass it off to the service. Voila.
PS - Here's skeleton to start with. UI, servlet, and controller are your responsibility:
package model;
public class User implements Serializable
    private String username;
    private String password;
    public User(String u, String p)
        this.username = u;
        this.password = p;
    public String getUsername() { return username; }
    public String getPassword() { return password; }
public interface UserDAO
    public User findByUsername(String username);
    public void saveOrUpdate(User user);
    public void delete(User user);
public class UserDAOImpl implements UserDAO
    private Connection connection;
    public UserDAOImpl(Connection connection)
        this.connection = connection;
    public User findByUsername(String username)
        String password = "";
        // logic for querying the database for a User
        return new User(username, password);
    public void saveOrUpdate(User user)
        // save or update a User
    public void delete(User user)
        // delete a User
public interface AuthenticationService
    public boolean isAuthorized(String username);
public class AuthenticationServiceImpl implements AuthenticationService
    private UserDAO userDAO;
    public AuthenticationServiceImpl()
        // Create a database connection here and the UserDAO, too.
    public boolean isAuthorized(String username)
        boolean isAuthorized = false;
        // Add logic to do the database query and decide if the username is authorized
        return isAuthorized;
}

Can a java code interact with trigger

Hi guyz ,
is there any way by which a trigger can interact with JDBC ?
I want to check the database each time its updated and if some condition is violated , the info((row from table which has voilated some rule) is passed to java code which can then take necessary action . Can anyone help me out in this because i am not aware of any way by which a trigger can interact with java code .
Regards,
Nikhi

nikhil456123 wrote:
Hi guyz ,
is there any way by which a trigger can interact with JDBC ?
I want to check the database each time its updated and if some condition is violated , the info((row from table which has voilated some rule) is passed to java code which can then take necessary action . Can anyone help me out in this because i am not aware of any way by which a trigger can interact with java code .Note that even though it might be possible you might want to consider that it isn't necessarily a good idea.
Say the action is "send email". Then every time an insert/update/delete occurs then an email is sent. That would obviously slow down those operations a lot. Not to mention that if 1000 inserts occur then someone is going to get 1000 emails.
There can be alternatives that are much better.

How to use Java Web Start with EJB ?

hi forum...
how to use Java WebStart with EJB ?
examples ?
thanks...
mindu

Mindu,
WebStart will work with EJB. I use it with WebLogic Server. You have to make sure and deploy the weblogic.jar file - sign it if needed.
For signing information, look here,
http://java.sun.com/products/javawebstart/1.2/docs/developersguide.html
Dan

Using cfform to interact with databases

hi there, I am just trying to get ito using the cfform tag,
and I thought it would be as easy to do insert into table ect using
server behaviours, but clearly not! could somebody point me the
right direction of some good tutorials please, to show me how to do
all the wonderful interaction with Databases that I have been able
to do with normal dreamweaver html forms...
Many thanks

I have found that <cfform> is a bit of a pain. I prefer
using the standard <form>
and writing the SQL myself. Much cleaner!
<cfquery ...
INSERT INTO t_someTable
(column1, column2)
VALUES
'#trim(form.field1)#',
'#trim(form.field2)#',
</cfquery>

Help! Websites are coming up, wifi signal is strong, but I cannot interact with the websites.

No problem with apps... just cannot interact with sites using Safari. The sites come up and are "live" but are "dim." Any suggestions?

Tap Settings / Safari. Clear the History, Cookies, and Cache.
Then restart your iPad.
If that doesn't help, tap Settings / General / Reset / Reset Network Settings.

Problem using Java Mail API with WLS 7.0

Hi All,
I am trying to use the Java Mail API provided by WLS 7.0. I have made the
settings metioned in the WLS 7.0 docs. However when I try to run the program I
am getting the following error:
javax.naming.NoInitialContextException: Need to specify class name in environment
or system property, or as an applet parameter, or in an application resource file:
java.naming.factory.initial
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:6
46)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:246
at javax.naming.InitialContext.getURLOrDefaultInitCtx(InitialContext.jav
a:283)
at javax.naming.InitialContext.lookup(InitialContext.java:350)
The code that I have written is as follows
import java.util.*;
import javax.activation.*;
import javax.mail.*;
import javax.mail.internet.*;
import javax.naming.*;
import java.io.*;
import java.net.InetAddress;
import java.util.Properties;
import java.util.Date;
public class MailTo {
     public static void main(String args[])
          try
               //Context ic = getInitialContext();
               InitialContext ic = new InitialContext();
/* My jndi name is "testSession" */
               Session session = (Session) ic.lookup("testSession"); /* THE PROBLEM IS SHOWN
IN THIS LINE */
               Properties props = new Properties();
               props.put("mail.transport.protocol", "smtp");
               props.put("mail.smtp.host", "XX.XX.XX.XX");
/* For security reasons I have written the ip add in this format */
               props.put("mail.from", "[email protected]"); /* for security reasons i have
changed the mail address */
               Session session2 = session.getInstance(props);
               Message msg = new MimeMessage(session2);
               msg.setFrom();
               msg.setRecipients(Message.RecipientType.TO, InternetAddress.parse("[email protected]",
false));
               msg.setSubject("Test Message");
               msg.setSentDate(new Date());
               MimeBodyPart mbp = new MimeBodyPart();
               mbp.setText("This is a mail sent to you using JAVA Mail API and Weblogic Server");
               Multipart mp = new MimeMultipart();
               mp.addBodyPart(mbp);
               msg.setContent(mp);
               Transport.send(msg);
          catch(Exception e)
               e.printStackTrace();
     }//end of main
public static Context getInitialContext()
     throws NamingException
          Properties p = new Properties();
          p.put(Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
          p.put(Context.PROVIDER_URL, "t3://localhost:7501/testWebApp");
               p.put(Context.SECURITY_PRINCIPAL, "weblogic");
               p.put(Context.SECURITY_CREDENTIALS, "weblogic");
          return new InitialContext(p);
}//end of class
Can anyone please tell me what is the problem. I thought that we cannot directly
do
InitialContext ic = new InitialContext();
So I had written a method getInitialContext() as shown in the above piece of code,
but that too did not work.
Eagerly awaiting a response.
Jimmy Shah

You can use InitialContext ic = new InitialContext() only if you are using a startup class, servlet or a JSP i.e
server side code.
If you are using a java client you need to use Context ic = getInitialContext();
Try this code
import java.util.*;
import javax.activation.*;
import javax.mail.*;
import javax.mail.internet.*;
import javax.naming.*;
import java.io.*;
import java.net.InetAddress;
import java.util.Properties;
import java.util.Date;
public class MailTo {
public static void main(String args[])
try {
Properties h = new Properties();
h.put(Context.INITIAL_CONTEXT_FACTORY,"weblogic.jndi.WLInitialContextFactory");
h.put(Context.PROVIDER_URL, "t3://localhost:7001");
Context ic = new InitialContext(h);
Session session = (Session) ic.lookup("testSession");
Properties props = new Properties();
props.put("mail.transport.protocol", "smtp");
props.put("mail.smtp.host", "XX.XX.XX.XX");
props.put("mail.from", "[email protected]");
Session session2 = session.getInstance(props);
Message msg = new MimeMessage(session2);
msg.setFrom();
msg.setRecipients(Message.RecipientType.TO,InternetAddress.parse("[email protected]",false));
msg.setSubject("Test Message");
msg.setSentDate(new Date());
MimeBodyPart mbp = new MimeBodyPart();
mbp.setText("This is a mail sent to you using JAVA Mail API and Weblogic Server");
Multipart mp = new MimeMultipart();
mp.addBodyPart(mbp);
msg.setContent(mp);
Transport.send(msg);
catch(Exception e)
e.printStackTrace();
}//end of main
}//end of class
We have shipped a javamail example in the samples\server\src\examples\javamail folder.
Jimmy Shah wrote:
Hi All,
I am trying to use the Java Mail API provided by WLS 7.0. I have made the
settings metioned in the WLS 7.0 docs. However when I try to run the program I
am getting the following error:
javax.naming.NoInitialContextException: Need to specify class name in environment
or system property, or as an applet parameter, or in an application resource file:
java.naming.factory.initial
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:6
46)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:246
at javax.naming.InitialContext.getURLOrDefaultInitCtx(InitialContext.jav
a:283)
at javax.naming.InitialContext.lookup(InitialContext.java:350)
The code that I have written is as follows
import java.util.*;
import javax.activation.*;
import javax.mail.*;
import javax.mail.internet.*;
import javax.naming.*;
import java.io.*;
import java.net.InetAddress;
import java.util.Properties;
import java.util.Date;
public class MailTo {
public static void main(String args[])
try
//Context ic = getInitialContext();
InitialContext ic = new InitialContext();
/* My jndi name is "testSession" */
Session session = (Session) ic.lookup("testSession"); /* THE PROBLEM IS SHOWN
IN THIS LINE */
Properties props = new Properties();
props.put("mail.transport.protocol", "smtp");
props.put("mail.smtp.host", "XX.XX.XX.XX");
/* For security reasons I have written the ip add in this format */
props.put("mail.from", "[email protected]"); /* for security reasons i have
changed the mail address */
Session session2 = session.getInstance(props);
Message msg = new MimeMessage(session2);
msg.setFrom();
msg.setRecipients(Message.RecipientType.TO, InternetAddress.parse("[email protected]",
false));
msg.setSubject("Test Message");
msg.setSentDate(new Date());
MimeBodyPart mbp = new MimeBodyPart();
mbp.setText("This is a mail sent to you using JAVA Mail API and Weblogic Server");
Multipart mp = new MimeMultipart();
mp.addBodyPart(mbp);
msg.setContent(mp);
Transport.send(msg);
catch(Exception e)
e.printStackTrace();
}//end of main
public static Context getInitialContext()
throws NamingException
Properties p = new Properties();
p.put(Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
p.put(Context.PROVIDER_URL, "t3://localhost:7501/testWebApp");
p.put(Context.SECURITY_PRINCIPAL, "weblogic");
p.put(Context.SECURITY_CREDENTIALS, "weblogic");
return new InitialContext(p);
}//end of class
Can anyone please tell me what is the problem. I thought that we cannot directly
do
InitialContext ic = new InitialContext();
So I had written a method getInitialContext() as shown in the above piece of code,
but that too did not work.
Eagerly awaiting a response.
Jimmy Shah--
Rajesh Mirchandani
Developer Relations Engineer
BEA Support

Using Java 6 classes with Java 5

I ran into a little problem with the portability of Java (ironic eh?) across computers. The program I wrote on my home computer uses GroupLayout for the GUI. However, much to my surprise, the computers at my school only had Java 5 installed so the program could not run (GroupLayout is a class found only in Java 6). Is there any way to somehow package the GroupLayout class into my program?

C_Zhao wrote:
I ran into a little problem with the portability of Java (ironic eh?) across computers.Not as ironic as you might think.
The program I wrote on my home computer uses GroupLayout for the GUI. However, much to my surprise, the computers at my school only had Java 5 installed so the program could not run (GroupLayout is a class found only in Java 6). Is there any way to somehow package the GroupLayout class into my program?Technically, yes, but I'd imagine that rewriting your code to use a different layout manager (such as one of those available in J2SE 5) would be quicker / easier. In fact, I'm almost willing to guarantee that refactoring would be easier than repackaging.

Examples of Using PHP to interact with R/3

Possible example:
For small businesses wanting to interract with an R/3 backend that my be run by a service provider, then they have a low cost alternative to Java development.
A simple example could be a small online reseller that does not maintain their own stock. By connecting their PHP based website to their suppliers R/3 system they can provide stock and price details to the customer.

Just take a look at the weblogs listed in this thread
Weblogs related to PHP and SAP

Using java to interact with a website

Similar Messages

Maybe you are looking for