Can I use Java to download webpages from the internet?

I need to be able to download webpages from different servers, search the webpage for certain data, save that data to disk and manipulate that data to create complimentary data to save as well. An example of what I need to download is at
http://www.gamestop.com/search.asp?keyword=final+fantasy&lookin=title&platform=var&genre=0&searchtype=adv&sortby=title
I want to do the same thing programmatically as when I point my IE browser at the page and "Save As..." the page to disk.
Does the JDK 1.2 have all the tools I need to get a connection to this URL and download the webpage there? If so, could you tell me what packages to look in or point me to some good docs?
I'm desparate, so please help!

Okay, I got that to work on another webpage, but on the webpage i specified on the previous post gave me the wrong web page, one that state that "cookies" aren't enable and the website cannot work without cookies. Is there any way to programmatically enable cookies?
Here's what I got so far...
import java.net.*;
import java.io.*;
class Test
public static void main(String args[])
try
// this URL works...
URL sourceURL = new URL
("http://www.gamedude.com/textonly.html");
// but this url requires cookies to work...
/*URL sourceURL = new URL("http://www.gamestop.com/search.asp?keyword=final+fantasy&lookin=title&platform=0&genre=0&searchtype=adv&sortby=title");*/
BufferedReader in = new BufferedReader(
new InputStreamReader(
sourceURL.openStream()));
PrintWriter out = new PrintWriter(
new BufferedWriter(
new FileWriter(
new File("C:/page.html"))));
String buffer;
while (!(null == (buffer = in.readLine())))
out.println(buffer);
in.close();
out.close();
catch (MalformedURLException e)
System.out.println("Bad URL:\n" + e);
catch (IOException e)
System.out.println("File error:\n" + e);
}

Similar Messages

  • How do I download videos from the internet using Adobe?

    I have already installed your Flasher in my computer. I want to use it to download videos from the internet. Please help me. I have two email addresses. Both you have. I hope you hear from you. My name is Gomez and my email address is [removed] Hope to hear from you.

    Gomez,
    I hope you are referring to Flash Player and you have installed it successfully. Ideally, you should be able to view Flash videos available in Internet.
    Do you have any issues in viewing videos?
    If so, please let us know your operating system (Windows XP, 7 or Mac) and the browser you use (Internet Explorer, Firefox, Safari).
    Also, let us know what you see in the following link: http://helpx.adobe.com/flash-player/kb/find-version-flash-player.html
    The following doc has useful information: http://helpx.adobe.com/flash-player/kb/flash-player-faq.html
    Thanks,
    Sunil

  • Hello.  I need help.  I use Photoshop Elements 10 on my MAC but it no longer reads discs.  Can I download it from the internet with my original package serial number?

    Hello.  I need help.  I use Photoshop Elements 10 on my MAC but it no longer reads discs.  Can I download it from the internet with my original package serial number?  thanks

    Yes, from here:
    Download Photoshop Elements products | 13, 12, 11, 10

  • Can LabVIEW download files from the Internet? I know the URLs of the files.

    Can LabVIEW download files from the Internet? I know the URLs of the files.

    Hi bmihura,
    I'm not sure what you what to do - download or open URLs - but I've put together the attached VI.
    I hope it is of some use.
    Charlie Rodway
    Test Design Engineer
    Rolls-Royce Controls and Data Services Ltd
    Attachments:
    WebBrowser.vi ‏21 KB

  • I am receiving a new ipad (3 i guess) from my mom who is in the U.S but i am currently in the philippines. I ofcourse, want to have apps,games,movies etc. can i use an itunes giftcard (also from the U.S) to buy my apps without any problems?

    i am receiving a new ipad (3 i guess) from my mom who is in the U.S but i am currently in the philippines. I ofcourse, want to have apps,games,movies etc. on it. can i use an itunes giftcard (also from the U.S) to buy my apps without any problems?

    No.
    iTunes terms of service:
    The iTunes Service is available to you only in the United States, its territories, and possessions. You agree not to use or attempt to use the iTunes Service from outside these locations. Apple may use technologies to verify your compliance.

  • HT1338 I have a problem with opening downloaded files from the internet; zip types. Can you tell me an app to open these types of formats of files?

    I have a problem with opening downloaded files from the internet; zip types. Can you tell me an app to open these types of formats of files?

    It should unzip if you double-cick.
    What are you seeing happen when you do that?
    Select a .zip file in the Finder and Get Info (cmd-i). What is set for it to Open With. It should be Archive Utility.

  • HELP!!!i can download files from the internet but i can not open them,help!

    I am running mac os x 10.5 i can download files from the internet but adobe reader 9 keeps telling me they can not decode them,( Adobe Reader could not open 'dap93.exe' because it is either not a supported file type or because the file has been damaged) (for example, it was sent as an email attachment and wasn't correctly decoded). help!
    Message was edited by: kjsk8n

    Adobe Reader does not open .EXE files, it opens .PDF files.  If you were on Windows I would tell you to fix the file association, but I have no idea how to do that on a Mac.

  • Can a user download something from the internet onto the IPad, without an app or a computer?

    Can a user download something from the internet onto the IPad, without an app or a computer?

    You download apps from the Apple AppStore.
    Music is from iTunes
    For other types of downloads, try iCab.
    Robert

  • I have Adobe Acrobat version 11.0.0 installed with Adobe CC.  I cannot download PDFs from the internet (in this case from PubMed National Library of Medicine).  If I try to do this  I get a popup message saying that "ADOBE ACROBAT NPAPI PLUG-IN, VERSION 1

    I have Adobe Acrobat version 11.0.0 installed with Adobe CC.  I cannot download PDFs from the internet (in this case from PubMed National Library of Medicine).  If I try to do this  I get a popup message saying that "ADOBE ACROBAT NPAPI PLUG-IN, VERSION 11.0.0 NEEDS YOUR PERMISSION TO RUN.  I DON'T KNOW HOW TO GIVE THIS PERMISSION OR WHAT TO DO.  CAN YOU PLEASE HELP?

    What buttons do you get to click on with this message? What browser are you using (e.g. Safari, Chrome, Internet Explorer...)

  • I downloaded music from the internet and edited it in Itunes. How do I keep the details of the song in windows explorer?

    So basically I downloaded music from the internet and edited the artist, album, album art and all that stuff on Itunes. But when I go back to windows explorer, i dont have those details

    Possibilities are that you are working with copies of the originals in which case it is only the copies that get updated, or that the songs are in .wav format and cannot carry a tag, or are in some other format and didn't include a tag so iTunes hasn't created one even though it has recorded details internally, or that there are mp3 files with multiple tags. iTunes doesn't cope well with multiple tags (despite these conforming to standards) and in this case may be making changes to one copy while Windows Explorer continues to show unchanged data from another. Use a 3rd party tag tool to inspect a problem album, or select the tracks and use Right-click > Convert ID3 Tags > None two or three times to strip out the embedded tags, then use Convert ID3 Tags > v2.3 to recreate them.
    Note the process of cleaning and rewriting the tags will remove any embedded art. You can use my script CreateFolderArt before and after converting the tags to preserve and then embed the artwork, provided that you don't store songs from different albums in the same folder. Test a single track or album first to make sure the process has the desired result and only apply where needed.
    tt2

  • Is there any other way to buy pro apps like logic pro x or final cut pro x without downloading straight from the internet?

    Hi,
    Just straigt to the issue, I'm not really sure with my internet connection in my country. I mean its so slow and sometimes it disconnects and reconnect. So I'm not trusting the internet connection here to download such a big software like Logic Pro x or Final Cut Pro x. So is there any way for me to buy the pro apps without downloading straight from the internet? I mean like the oldschool method, come with a box. I bought Logic pro 7 and 8 like that and has no problem with it. But now apple selling all the pro apps from the app store, its cool and easy but im afraid if i bought it online and im pretty sure i will encounter problems during the download period. It might took me like a week or more maybe to download 40GB file, i dont know. And more over, what if my internet connection cut off while i download the software?
    So please do help me with this. I would love to have Logic Pro X on my apple computer.
    Thanks.

    ahavu wrote:
    And to my knowledge I can't sort of 'drag and drop' the clips from the sealed Final Cut event files either.
    The events are not "sealed". Your clips – if you copied rather than leaving in place  – are in the Original Media folder. If you used 10.1.x, you right click on the library bundle and choose Show Package Contents. In early versions, just open the event folder.
    There are multiple ways to get media on ones' computer. One way is to use a card reader and copy the entire card structure to whatever destination is used for a media drive. Then import into the NLE.
    I don't have any experience with Hitfilm but I assume their user forums would be a good place to get an understanding of what workflows work best for that application.
    Good luck.
    Russ

  • Problems with downloading videos from the internet....please help

    sorry if this the wrong forum to post but I don't know where else to post my issue.
    Normally when I download videos from the internet I click on "window" then "activity" then the largest file ie ? of ? (which is the MB file) and it downloads then I use isquint to put it to mp4 format. However now when I do that the file I download says that "video _ts folders can't be directly converted" and there is a button that says "Inadequacy". I can't even download videos from youtube. Any help would be great.
    thank you

    Other than the videos on YouTube, can you post a link or URL of the videos you having problems with for us to test out for you?
    I use Firefox which has quite a few video converter add ons that allows downloads of any type of video from the net. The one I use is called +Flash Video Downloader.+
    "Is there anything else I need to have installed in order to be able to download videos?"
    The latest versions of the following:
    QuickTime
    RealPlayer
    Flip4Mac
    VLC Media Player
    Perian (AVI and FLV support)
    "I also have divx and 3ivx installed"
    You can get rid of those if you have any of the above installed.
    " Mac OS X (10.5.2)"
    Use Software Update to update your OS which is seriously out of date. The last version of Leopard I believe was either v5.6 or v5.8. Also, update all the other things listed that pertain to the OS & your model computer if applicable to your needs.
    !http://i50.tinypic.com/izvwo1.gif!

  • How do I download videos from the internet to my mac

    How do I download videos from the internet to my MAC?

    You can use SnapZ Pro X, which will screen record with sound any animation on the monitor into a QuickTime file, but it's expensive.
    If you look for Add-ons for Firefox, there are some that download Flash and YouTube videos to a file on your machine.
    https://www.mozilla.org/en-US/plugincheck/
    You Mac likely will come across sites and content that won't play in Quicktime, on your Mac or in your web browser unless you install additonal codecs
    You can get them here
    Perian
    http://perian.org/
    VLC
    http://www.videolan.org/vlc/download-macosx.html
    Flip4Mac $29
    http://www.telestream.net/flip4mac-wmv/overview.htm
    Bookmark and download Flash from here, DO NOT trust any website telling you to install anything or follow their links.
    Flash update
    http://get.adobe.com/flashplayer/
    Silverlight update
    http://www.microsoft.com/silverlight/get-started/install/default.aspx
    There is a FlashBack trojan making the rounds which tries to con people into installing it by looking like it's the real thing.
    Make sure your Flash (and Silverlight) is current by running these checks
    http://flashbuilder.eu/flash-player-version.html
    https://www.mozilla.org/en-US/plugincheck/
    How to install
    When you click a link, a download of the installer file occurs to your Downloads Folder usually located next to your Trash Can or in your Users Folder.
    Open the folder and double click (or File: Open) the download and icon can appear on your desktop which you open and inside is a installer package which you double click on to launch the installer.
    OS X will check the download to make sure you intended to start it and that it's not malware it knows about.
    Follow the installer instructions to install the software onto your computer and quit when finished.
    Some require the browsers to be closed first, others require the machine to be restarted to enable the software.
    Once the install is completed, remove any and all downloads from your Download folder, it's safer to keep this folder empty so you can keep track of what appears in there.
    Drive by downloads of malware can occur and reside in the Downloads folder, mixing in with files residing there and await your click to install.
    For backup, restore, performance and other tips I've got, read here
    https://discussions.apple.com/message/16276201#16276201

  • I get the following error everytime I download something from the internet:-"/Users/vinaypunjabi/Downloads/Ontario Colleges Chart(2)(1).pdf could not be opened, because an unknown error occurred.  Try saving to disk first and then opening the file".

    I get the following error everytime I download something from the internet:-"/Users/vinaypunjabi/Downloads/Ontario Colleges Chart(2)(1).pdf could not be opened, because an unknown error occurred.  Try saving to disk first and then opening the file".
    Then it shows as downloading failed.But if I click on it twice ,it opens up .
    this has started happening since last week.Before this ,it never happened.
    Please suggest what has gone wrong somewhere and how can I rectify it.

    Please install the following add-on to see if it helps.
    https://addons.mozilla.org/en-US/firefox/addon/pdfjs/

  • Problems with downloading files from the internet?

    I have Mozilla Firefox and Safari on my macbook. For some reason it will not let me download things from the internet. Whenever i keep a download link, the page never loads....can anyone put some insight into my problem?
    Justin Evans

    Since a few hours, I have the same problem like you. I uninstalled Firefox, because I thought it was a problem with it, but then i started Safari to redownload Firefox and guess what happened.. nothing. I can't download anything. don't know why. does anyone know whats wrong??

Maybe you are looking for