Java interaction with websites?

Hey,
I'm been learning java for a few months now, and know the basics well, I have written a few small programs, including one which transfers songs from iPod to computer.
I need some advice on where to start on my next project. Basically I want to make a program which automatically performs tasks in the background on websites, for example listing an item on ebay, or uploading a video to youtube. And I'd also like it to read information from different pages, so that I could maybe gather a database of prices from ebay, or somthing along those lines.
Which classes could I use java to interact with html?
Where do I start? :)
My java vocabulary isn't great yet ;)

Ryanz wrote:
Ok ... I'm checking it out at the moment, are there any other maybe simpler ways? Or can somone explain how to get the JAX-WS installed and ready to use with eclipse?Ryanz,there are few things involved here.
1).First and foremost you need to findout whether if respective website offers you any kind of Web-Services to serve your cause.
Just as an example eBay offers set of services to serve your cause to know more about this you can visit the below links
[http://www.xml.com/pub/a/2005/09/28/ebay-metadata-web-services.html]
[http://developer.ebay.com/community/featured_projects/default.aspx?name=Code%20Samples%20Project]
Similarly identify whether if respective other focused webservice providers offer any specific services serving your cause.
2).Analyse their service specifications
3).Identify and Plan for list of Data Transfer Objects to which we can bind this service
4).Implement for a Stub (may be using Apache Axis) or use JAX-RPC or use JAX-WS or any other related technologies to implement your Webservice Client.
Coming back to using tools like Apache Axis,JAX- RPC or JAX-WS.
you might be intrested in looking towards the below set of turtorials
[http://www.onjava.com/pub/a/onjava/2002/06/05/axis.html]
[http://www.ibm.com/developerworks/webservices/library/ws-castor/]
[http://java.sun.com/j2ee/1.4/docs/tutorial/doc/JAXRPC.html]
[http://www.ibm.com/developerworks/edu/j-dw-java-jaxrpc2-i.html]
[http://java.sun.com/webservices/docs/2.0/tutorial/doc/]
[http://www.myarch.com/create-jax-ws-service-in-5-minutes]
It is not nessary that a provider has to provide relvant data in form offering webservices and sometimes they can offer you different specs which could be Like regular Java RMI services or DCOM object or something like Feeds(RSS 2.0/ATOM) for providing Latest updates on your account.I'd say it'd be more work for us to identify what kind of service the provider is provding us and how best make use of it so that we can use the relavant data in our application.
and believe me in most of the cases only point 4 changes after going through point 1,2 & 3.
Hope that helps :)
REGARDS,
RaHuL

Similar Messages

  • Can Java Interact with Web Pages??

    I wonder if it possible to develop an application or applet that can automate the process of signing up for courses at Notre Dame.
    Currently, we have to use a pretty plane text-based web system. I wonder if it's feasible to use Java to provide a more user-friendly front-end to this system. The program would need to read HTML, view different web pages, and write to web forms. Any tips on where to go from here?
    I realize that some other language might work too, but I'd like to advance my Java skills ;-)
    Thanks!!!
    - Brian

    From what you've posted so far it sounds like you want to make a specialized web browser. Is this what you had in mind? The Java API includes two classes that can be used to construct such a beast relatively easily: JEditorPane and JTextPane. The Swing tutorial has a page on how they can be used:
    http://java.sun.com/docs/books/tutorial/uiswing/components/editorpane.html
    However, their capabilities are rather limited (only support HTML 3.2). They will let you change text styles and the way pages look, but will hardly be more "user friendly" than the original site because it'll be only the old thing with a new look.
    There's also a HTML parser that can be used independently of the gui components, in principle you can use it to extract information from the site and show it in a completely new user interface that has nothing to do with the original site but I haven't used it much and i'm too tired to look up and example on the 'net (been a long day, i'm off to bed)

  • Interacting with Quark Files

    Is it possible in java interacting with quark..
    i am learning what is quark. but i wanted to know can i make some plugin kind of thing for quark using java.

    This is the only Quark I am familiar with. Do you have a url for your Quark?

  • Using java to interact with a website

    I'm looking to build a program that will function as an alternate front end for an online database so that users can perform batched operations. Is there a way to have java navigate and interacte with a website? This would be something a macro would do, except that I need to be able to interact with the desktop, among other things. Also, I'm stuck with internet explorer, as the IT people don't want Firefox on any of the machines, so that excludes the iMacros add on.
    anyone got an idea? thanks

    DrClap wrote:
    Unless you also control the website. However, even in that case I think that web services are superior in the long run to HTML-scraping.Agreed. I was assuming that if the website was one's own, one wouldn't resort to scraping, but you're right, that needs to be stated.

  • Interacting with MS Word through Java

    Hi friends,
    How to interact with a MS Word file through the java program. please send me some sample codes. Thanks in advance.....
    Regards,
    Prakash.....

    Prakashjava wrote:
    Hi friends,
    How to interact with a MS Word file through the java program. please send me some sample codes. Thanks in advance.....
    Can't be done in pure Java.

  • Interacting with perl through java.

    I'm trying to do something with Java that I haven't managed to do until now. I'm trying to interact with a perl process giving him input and reading output.
    I have a perl module that has several functions and the idea is to launch a process in a java program with the following line :
    perl -e 'while(<STDIN>) { eval $_ ; }'
    and then give the process the necessary input and the read the given output.
    For instance with the above line you can do the following:
    [user@host ~]$ perl -e 'while(<STDIN>) { eval $_ ; }'
    print "Hello World\n";
    Hello World
    Here is the code I'm using:
    import java.io.BufferedReader;
    public class ExecProgram {
    private static Runtime runtime;
    private static Process process;
    public static void main(String[] args) {
         runtime = Runtime.getRuntime();
         try {
         process = runtime.exec("/usr/bin/perl -e 'while(<STDIN>) { eval $_ ; }'");
              process = runtime.exec(cmd);
         } catch (IOException e) {
              System.err.println("Error executing process");
         PrintWriter out = new PrintWriter(process.getOutputStream());
         String commandLine = "print \"Hello World\n\"";
         out.println(commandLine);
         BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
         try {
         String line;
         while ((line = in.readLine()) != null)
              System.out.println("Output: "+line);
         } catch (IOException e) {
         System.err.println("Error reading output");
    As you can see I'm using Runtime class to interact with the process but nothing happens if replace the exec line with:
    process = runtime.exec("ls");
    I can see in the output the listing of the current directory.
    Have you ever tried something like this? Is this the correct way for doing this?

    Have you ever tried something like this? I have. Here's a rough sample:
        public static void main(String[] args) throws Exception {       
            String[] cmd = new String[]{
                "/usr/bin/perl",
                "-e",
                "while(<>){ eval or die $@; }"
            final Process p = Runtime.getRuntime().exec(cmd);
            new Thread(){
                public void run(){
                    try{
                        BufferedReader r = new BufferedReader(new InputStreamReader(p.getInputStream()));
                        for(String line; (line = r.readLine()) != null; System.out.println("ProcessOUT:: "+line));
                    catch (Throwable t){
    //                    t.printStackTrace();
            }.start();
            new Thread(){
                public void run(){
                    try{
                        BufferedReader r = new BufferedReader(new InputStreamReader(p.getErrorStream()));
                        for(String line; (line = r.readLine()) != null; System.err.println("ProcessERR:: "+line));
                    catch (Throwable t){
    //                    t.printStackTrace();
            }.start();
            new Thread(){
                public void run(){
                    try{
                        OutputStream out = p.getOutputStream();
                        for(int ii = 0; ii < commands.length; ii++){
                            System.err.println("Sending command: "+commands[ii]);
                            out.write(commands[ii].getBytes());
                            out.write('\n');
                            out.flush();
                    catch (Throwable t){
    //                    t.printStackTrace();
            }.start();
            int exit = p.waitFor();
            System.err.println("Process exited with code: "+exit);
        static final String[]
            commands = {
                "print \"The road goes ever\\n\";",
                "print \"ever\\n\";",
                "print \"on.\\n\";",
                "exit 13;"
        ;

  • Pacman frontend in Java - Howto interact with alpm

    I plan to write a(nother :-) ) GUI frontend to Pacman using Java + Swing.  However I need some advice regarding how should I interact with Pacman.
    Basically my requirements are:
    1.  Reading/Searching package database
    2.  Showing a download progress bar
    3.  Showing a progress bar when Pacman is installing/removing package(s)
    As 'alpm' is a C library it's not easily accessible from Java.  The only and perhaps dummest solution that I can think of is to write my own pseudo-alpm library to handle the requirements but I'd seriously rather avoid the re-work and bugs of such a process.  Is there any other way?
    I'd appreciate any idea/hint.  TIA,
    Bahman

    If you have some success creating jni bindings to libalpm you might consider creating an AUR package to join the lua, perl and python packages that are there now.

  • Interacting with Powershell from Java

    Hi,
    I'm trying to run a Java application which creates a new powershell process on startup and then later on interacts with it multiple times. Calling powershell.exe and have it execute a single command and return the output works fine for me. The problem arises if I don't want the powershell process to immediately finish/exit but to stay open so I can write to its outputStream and receive results back from the inputStream.
    String input = "dir";
    String[] commandList = {"powershell.exe", "-Command", "dir"};
    ProcessBuilder pb = new ProcessBuilder(commandList);
    Process p = pb.start();
    if(input != null) { 
    PrintWriter writer = new PrintWriter(new OutputStreamWriter(new BufferedOutputStream(p.getOutputStream())), true);
    writer.println(input);
    writer.flush();
    writer.close();
    //p.getOutputStream().close();
    Gobbler outGobbler = new Gobbler(p.getInputStream());
    Gobbler errGobbler = new Gobbler(p.getErrorStream());
    Thread outThread = new Thread(outGobbler);
    Thread errThread = new Thread(errGobbler);
    outThread.start();
    errThread.start();
    System.out.println("Waiting for the Gobbler threads to join...");
    outThread.join();
    errThread.join();
    System.out.println("Waiting for the process to exit...");
    int exitVal = p.waitFor();
    System.out.println("\n****************************");
    System.out.println("Command: " + "cmd.exe /c dir");
    System.out.println("Exit Value = " + exitVal);
    List<String> output = outGobbler.getOuput();
    input = "";
    for(String o: output) { 
    input += o;
    System.out.println("Final Output:");
    System.out.println(input);
    This code returns the result of the "dir" command from a powershell - fine. But as you can see, I'm trying to run a second "dir" command using
    PrintWriter writer = new PrintWriter(new OutputStreamWriter(new BufferedOutputStream(p.getOutputStream())), true);
    writer.println(input);
    writer.flush();
    This has no effect whatsoever - no second dir output is shown when I run my code. I've also experimented with a powershell.exe option to open the powershell but not close it immediately:
    String[] commandList = {"powershell.exe", "-NoExit", "-Command", "dir"};
    But then my code hangs, meaning the Gobbler's who consume the process's inputStream don't read anything - strangely enough: they don't even read the first line - there must be at least some output....
    I've also tried to close the process's outputStream after writing the second "dir" command to it - didn't change anything.
    But when I initially call the cmd.exe using the /k (keep open) switch:
    String[] commandList = {"cmd.exe", "/k", "dir"};
    I can then still write to that outputstream and invoke the second "dir" command and get the output of both "dir" commands from the inputstream fo that process.
    Any help is highly appreciated.
    Thanks
    Kurt

    user4491593 wrote:
    BUT: my Gobblers only read the output of all my commands Then why don't change your Gobbler code ? ;)
    Test this, it's ugly and needs improvemens, but by now works fine on linux and windows:
    public class Gobbler implements Runnable {
        private PrintStream out;
        private String message;
        private BufferedReader reader;
        public Gobbler(InputStream inputStream, PrintStream out) {
            this.reader = new BufferedReader(new InputStreamReader(inputStream));
                   this.out = out;
            this.message = ( null != message ) ? message : "";
        public void run() {
            String line;
            try {
                while (null != (line = this.reader.readLine())) {
                    out.println(message + line);
                this.reader.close();
            } catch (IOException e) {
                System.err.println("ERROR: " + e.getMessage());
    public class PowerConsole {
        private ProcessBuilder pb;
        Process p;
        boolean closed = false;
        PrintWriter writer;
        PowerConsole(String[] commandList) {
            pb = new ProcessBuilder(commandList);
            try {
                p = pb.start();
            } catch (IOException ex) {
                throw new RuntimeException("Cannot execute PowerShell.exe", ex);
            writer = new PrintWriter(new OutputStreamWriter(new BufferedOutputStream(p.getOutputStream())), true);
            Gobbler outGobbler = new Gobbler(p.getInputStream(), System.out);
            Gobbler errGobbler = new Gobbler(p.getErrorStream(), System.out);
            Thread outThread = new Thread(outGobbler);
            Thread errThread = new Thread(errGobbler);
            outThread.start();
            errThread.start();
        public void execute(String command) {
            if (!closed) {
                writer.println(command);
                writer.flush();
            } else {
                throw new IllegalStateException("Power console has ben closed.");
        public void close() {
            try {
                execute("exit");
                p.waitFor();
            } catch (InterruptedException ex) {
        public static void main(String[] args) throws IOException, InterruptedException {
            /*   PowerConsole pc = new PowerConsole(new String[]{"/bin/bash"});
            PowerConsole pc = new PowerConsole(new String[]{"/bin/bash"});
            pc.execute("pwd");
            pc.execute("ls");
            pc.execute("cd /");
            pc.execute("ls -l");
            pc.execute("cd ~");
            pc.execute("find . -name 'test.*' -print");
            pc.close();
            //      PowerConsole pc = new PowerConsole(new String[]{"cmd.exe"});
            PowerConsole pc = new PowerConsole(new String[]{"powershell.exe", "-NoExit", "-Command", "-"});
            System.out.println("========== Executing dir");
            pc.execute("dir");
            System.out.println("========== Executing cd\\");
            pc.execute("cd \\"); Thread.sleep(2000);
            System.out.println("========== Executing dir");
            pc.execute("dir"); Thread.sleep(2000);
            System.out.println("========== Executing cd \\temp");
            pc.execute("cd \\temp"); Thread.sleep(2000);
            System.out.println("========== Executing dir");
            pc.execute("dir"); Thread.sleep(2000);
            System.out.println("========== Executing cd \\bubba");
            pc.execute("cd \\bubba"); Thread.sleep(2000);
            System.out.println("========== Exiting .... bye.");
            pc.close();
    }I tested this and there is still a little problem -look at the test below.
    It seems that when thecommand
    executed in the powershell prints only a one ot two lines,
    powershell doesn't flush the output stream
    .... but this rather problem of powershell, not the java code
    I have not a clue how to force powershell to flush
    it's output stream after each command.
    C:\temp>java -jar PowerShell.jar
    ========== Executing dir
        Directory: Microsoft.PowerShell.Core\FileSystem::C:\temp
    Mode                LastWriteTime     Length Name
    -a---        2012-01-09     01:16       5290 PowerShell.jar
    ========== Executing cd\
    ========== Executing dir
        Directory: Microsoft.PowerShell.Core\FileSystem::C:\
    Mode                LastWriteTime     Length Name
    d----        2012-01-08     02:56            61587b977687a6e22fbe
    d----        2011-12-14     03:19            Documents and Settings
    d----        2011-12-15     00:05            oraclexe
    d-r--        2012-01-08     03:44            Program Files
    d----        2012-01-05     19:59            sqldeveloper
    d----        2012-01-09     01:15            temp
    d----        2012-01-09     01:13            WINDOWS
    -a---        2011-12-14     03:12          0 AUTOEXEC.BAT
    -a---        2011-12-14     03:12          0 CONFIG.SYS
    ========== Executing cd \temp
    ========== Executing dir
        Directory: Microsoft.PowerShell.Core\FileSystem::C:\temp
    Mode                LastWriteTime     Length Name
    -a---        2012-01-09     01:16       5290 PowerShell.jar
    ========== Executing cd \bubba
    Set-Location : Cannot find path 'C:\bubba' because it does not exist.
    At line:1 char:3
    + cd  <<<< \bubba
    ========== Exiting .... bye.
    C:\temp>

  • Can a java code interact with trigger

    Hi guyz ,
    is there any way by which a trigger can interact with JDBC ?
    I want to check the database each time its updated and if some condition is violated , the info((row from table which has voilated some rule) is passed to java code which can then take necessary action . Can anyone help me out in this because i am not aware of any way by which a trigger can interact with java code .
    Regards,
    Nikhi

    nikhil456123 wrote:
    Hi guyz ,
    is there any way by which a trigger can interact with JDBC ?
    I want to check the database each time its updated and if some condition is violated , the info((row from table which has voilated some rule) is passed to java code which can then take necessary action . Can anyone help me out in this because i am not aware of any way by which a trigger can interact with java code .Note that even though it might be possible you might want to consider that it isn't necessarily a good idea.
    Say the action is "send email". Then every time an insert/update/delete occurs then an email is sent. That would obviously slow down those operations a lot. Not to mention that if 1000 inserts occur then someone is going to get 1000 emails.
    There can be alternatives that are much better.

  • Interact with exe in java

    Hi!
    I got a old exe file, that no more maintain by any team.
    Unfortunately, requirement has it that I need to run this exe, do some interaction, such as "enter" and "wait" and "input value", get "value"
    And it has to be retrive by java,
    can anyone give idea how should this done?
    I know I can excute this program, but I don't know how to fetch the program any more key board input...
    Thanks

    What sort of exe?
    If it is a GUI, then you will need to use JNI to interact with it. You might find a 3rd party tool that already does this for the GUI/OS that you are targetting. If not you have to write it yourself.
    If it is a command line application then you can use Runtime.exec(). That creates a process without output and input streams. The input stream allows you to send text to the process.

  • How to interact with a COM component from a Java class

    Hi, could someone give a hint on what API I should explore in order to interact with a COM component from a Java class?
    Thanks in advance
    Luis

    jacob sounds nice...http://danadler.com/jacob/

  • Help! Websites are coming up, wifi signal is strong, but I cannot interact with the websites.

    No problem with apps... just cannot interact with sites using Safari. The sites come up and are "live" but are "dim." Any suggestions?

    Tap Settings / Safari. Clear the History, Cookies, and Cache.
    Then restart your iPad.
    If that doesn't help, tap Settings / General / Reset / Reset Network Settings.

  • Java interface with Crystal Reports

    Post Author: [email protected]
    CA Forum: JAVA
    Hello everyone,I need to build a Java interface for JSP to interact with crystal reports.Could anyone recommend any book or forward me the url's where I can find the related material with examples. Eagerly Waiting for reply!Thanks,Prathima.

    Post Author: MJ@BOBJ
    CA Forum: JAVA
    The latest version of the Java Report Component (JRC) is available from the Diamond website.  This download is actually a plugin for Eclipse but you can still use the JRC runtime jars to use the SDK APIs in your JSP page.  You will also find lots of resources such as samples, videos, and forums to help you get started.  You can also refer to the DevLibrary for more information.

  • How do I create an app to interact with my Web ? like skype or msn ?

    I need to create a downloadable interface like Skype or Msn which will interact with my website that is created on PHP and MySQL...
    What will be the best practices for this ? thanks in advance...

    You might take a look at Adobe's AIR platform which can use HTML, JavaScript, and Flash to develop applications that work on the desktop like a regular app but can also connect/interact with a web service. But that question is more in line with the Dreamweaver Application Development forum.

  • Problem in Interacting with Weblogic JMS through Informatica ETL server

    Hi All,
    We are trying to configure Informatica PowerCenter server to interact with JMS queues created on Weblogic 11gR1 server. We have copied the necessary
    jar file - wlfullclient.jar to the classpath of the Informatica server.
    Now when i try to read contents of a JMS queue created on Weblogic server through Informatica, i get this error:-
    Failed to get the Queue Connection Factory [weblogic.jms.common.DestinationImpl]. Reason: Failed to look up object [While trying to lookup 'weblogic.jms.common.DestinationImpl' didn't find subcontext 'common'. Resolved 'weblogic.jms'Exception Stack: javax.naming.NameNotFoundException: While trying to lookup 'weblogic.jms.common.DestinationImpl' didn't find subcontext 'common'. Resolved 'weblogic.jms' [Root exception is javax.naming.NameNotFoundException: While trying to lookup 'weblogic.jms.common.DestinationImpl' didn't find subcontext 'common'. Resolved 'weblogic.jms']; remaining name 'common/DestinationImpl'
         at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:237)
         at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:464)
         at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:272)
         at weblogic.jndi.internal.ServerNamingNode_1211_WLStub.lookup(Unknown Source)
         at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:418)
         at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:406)
         at javax.naming.InitialContext.lookup(InitialContext.java:392)
         at com.informatica.powerconnect.jms.server.common.JNDIConnection.getJNDIObject(JNDIConnection.java:84)
         at com.informatica.powerconnect.jms.server.common.PlginJMSConnection$PlginJMSQueueConnection.createConnection(PlginJMSConnection.java:363)
         at com.informatica.powerconnect.jms.server.common.PlginJMSConnection.<init>(PlginJMSConnection.java:90)
         at com.informatica.powerconnect.jms.server.common.PlginJMSConnection$PlginJMSQueueConnection.<init>(PlginJMSConnection.java:352)
         at com.informatica.powerconnect.jms.server.common.PlginJMSConnection.create(PlginJMSConnection.java:115)
         at com.informatica.powerconnect.jms.server.reader.JMSReaderSQDriver.createPartitionDriver(JMSReaderSQDriver.java:557)
    Caused by: javax.naming.NameNotFoundException: While trying to lookup 'weblogic.jms.common.DestinationImpl' didn't find subcontext 'common'
    I have checked that this class file exists in the above said location :- "weblogic/jms/common/DestinationImpl.class" within the wlfullclient.jar but still the
    code is not able to access the class. Similar error came initially while doing JNDI connection using "weblogic.jndi.WLInitialContextFactory" class but it vanished,
    when i explicitly named the jar file on classpth, instead of just the folder containing the jar file.
    To all the users/experts/moderators of Weblogic - Could you please help me understand why Weblogic is not able to look up subcontext 'common'
    for the class - "weblogic.jms.common.DestinationImpl" ??
    Thanks & Regards
    Raj
    Edited by: user8931188 on May 17, 2012 7:39 AM

    You are experiencing a NameNotFoundException, not ClassNotFoundException so it seems there is something wrong about the client connection code or WebLogic config.
    Can you share your client connection connection config/code?

Maybe you are looking for