Help to download files from a website with authentication
Hi,
I need to develop a process in java to be able to connect to a website using a username and password and download a zip file on to a local server.
I am kinda new to socket programming and i am not sure how to go about achieving this.
I would really appreciate help on this.
Thanks
Ankit
Have a look at the Jakarta Commons HttpClient project. That library includes classes for making HTTP requests with different authentication schemes.
Or, if you need to download from an FTP server, look at Jakarta Commons Net.
Similar Messages
-
Cannot download files from certain websites: log-in prompt
Hi, I am for some reason getting a log-in prompt when clicking on downloadable files from certain websites (pdf files, etc.). It is especially strange because it is happening on multiple sites, which gave me no problem downloading in the past. This is particular to my computer, and it is happening in different locations, so it can't be a problem with my isp or something.
I am getting this prompt:
"To view this page, you need to log in to area "PAGE TITLE" on "www.address.com:80"
Your password will be sent in the clear."
(Where PAGE TITLE and address.com are specific to the page I'm trying to access)
I've tried giving a username and password, but it didn't work. I tried emptying my cache and deleting cookies, renewing my ip address, to no avail. I do not use any proxies, so I am really confused now.
I've found no documentation of anyone else having this issue.
Any help would be appreciated!i figured it out... it was just a strange coincidence, nothing wrong after all.
-
Why can't I download file from any websites?
Whenever I try to download files from filesharing sites like mediafire, I click the download button and the page appears to be loading and nothing happens.
After several tries, I found out that while the page is loading, it is actually downloading the file because the time it takes for the page to load before the "open" or "save file" windows come out depends on the size of the file. And immediately after I click "save file" the download is done, and I can open whatever I downloaded. The time ranges between 5min to 30min depending on the file size.
Usually the "save file" window will display immediately when I click on the download button, the the fire fox download window will show the progress of the download and % of completion. But no, for me nothing appears in the download window, and I don't get to see the progress of my download until it is apparently "completed" after the webpage finished loading.
I have tried every solution that I know, cleaning my browser history and cookies, resetting firefox, clearing up my computer's registry, even did a clean reinstall of firefox, wiping everything including my bookmarks, but none of these solves the problem. I am also pretty sure that my firefox extension are not the ones to blame as I did do a test download after a clean reinstall, which did not work as well.My problem is fixed after I set the firefox connection settings from "use system proxy setting" to "no proxy". After setting to no proxy, the downloads happens as it should.
but my question now is what is system proxy? and what is the difference between using these 2 different options? -
I cannot download files from some websites
Since upgrading to ver 5.0, I cannot download statements from many of my financial websites. I am forced to start IE or Opera to download. Sometimes I am able to right click the link, and do 'Save Link as' and it will download, but not all. My Verizon bill is the latest failure today. When I clock on the link, I get a new tab opening that shows 'about:blank' in the web link, and says 'done' in the status bar. It does nothing after that.
When reading about the antivirus, I didn't equate that to the iPhone. Since antivirus software cannot run in the background on an iPhone, that isn't it and if pop-up blockers was enabled before I doubt that has anything to do with it either.
Is the statement in PDF format? -
With Firefox 4, I cannot download files from the internet. Please help me !
I cannot download files from the internet (with Firefox 4). I was able to until this upgrade. I am disappointed.
The site specifically is www.c64g.com/games.php?q=pWhat buttons do you get to click on with this message? What browser are you using (e.g. Safari, Chrome, Internet Explorer...)
-
Cannot download pdf files from internet site with either acrobat pro X or pro XI on windows 8 comput
cannot download pdf files from internet website with either acrobat pro x or proXI on windows 8 computer. windows 7, no problem.
Hi valerieross,
Acrobat can be used to view pdf's or edit pdf's not download them.
Check your browser settings.
If you are referring to viewing pdf's in browser, then please check the following : http://helpx.adobe.com/acrobat/kb/cant-view-pdf-web.html -
Having problems installing Canon MG3170. Anyone have any solutions? Here is what I have tried:-
-Installed from provided Canon disk, firstly with wireless connection, then using USB connection. Printer could not be added because the Macbook returns that the software is not available from Apple for both types of connections.
-Uninstalled software from the Canon disk, checked for software updates, then downloaded drivers (printer and scanner) from Canon Website. Again tried to install using wireless connection then USB connecton. Printer could not be added because Macbook returns that the software is not available from Apple
-Uninstalled drivers downloaded direct from Canon Website and plugged printer into Macbook using USB, Macbook recognised the printer and searched for the software, again Macbook returns that hte software is not available from Apple.
Help!nrg adjusta,
Sorry it took me a while to repsond, Ive been pretty busy with school and work. But for my hard drive problem, I did hours and hours of more research and trying countless ways of installing and tweaking things in my laptop and absolutely nothing worked that I could do. It was very irritating because I felt like I was so close to fixing it and I was just missing a step somewhere. Though It still boggles me why the disk utility shows a different capacity for the HD's. One thing I'm going to try next is replacing the hard drive cable that goes from the internal HD to the macbook. I read somewhere that sometimes those can go bad over time, and also if you get a new, modern hard drive, sometimes the old cable (for some odd reason) might not be compatible with the new HD. So i want to test it out for myself to see it that is the actual culprit.
http://www.ifixit.com/MacBook-Parts/MacBook-Hard-Drive-Cable/IF186-018
Once I get the cable and install it I'll let you know the results and if it helped at all.
Have you made any progress on your hard drive at all? If worst comes worst for you, you can probably get a SATA to USB cable and try to reformat it that way (this is the way that worked for me, but the only problem was it wouldn't be recognized when I installed it internally. Thats why i want to try getting the new cable), and if it still wont recognize it after that, you can just leave it outside of your mac, and just boot from that. The only thing that ***** about that is you have this big hard drive handing off the side of your mac when ever you need to use it.
Heres a link for a SATA to USB. this one is good because is compact and protects the HD from damage and most static electricity.
http://www.thinkgeek.com/computing/usb-gadgets/e6d9/?srp=3
Im still continuing my search for the answer and once i find it (if ever!) I'll post it. -
So im trying to download a mixtape from Datpiff.com but the download option does not "load" and doesnt download. I am able to download music from other websites but not on some. Are there any settings that i might've changed?
Start Firefox in <u>[[Safe Mode|Safe Mode]]</u> to check if one of the extensions (Firefox/Tools > Add-ons > Extensions) or if hardware acceleration is causing the problem (switch to the DEFAULT theme: Firefox/Tools > Add-ons > Appearance).
*Do not click the Reset button on the Safe mode start window or otherwise make changes.
*https://support.mozilla.org/kb/Safe+Mode
*https://support.mozilla.org/kb/Troubleshooting+extensions+and+themes
Please Update to the latest Firefox 16.0.x version.
*Help > About Firefox
*https://support.mozilla.org/kb/update-firefox-latest-version
The Firefox 13.0.1 version that you currently run is no longer supported with security updates.<br />
It is important to update Firefox and add-ons to the latest versions to get the latest bug fixes and security updates. -
Help required with downloading files from inside WEB-INF folder
I am working on a web application and I have an upload script which uploads different files to a foler called 'uploads' . To stop users from accessing the folder content I have put it in the /WEB-INF/.
In my JSP file, I used to link to the uploaded files like this before for users to download them :
if(filesListIterator.hasNext()
files = (FilesBean)filesListIterator.next();
<tr><td><a target="_new" href="<%=files.getFileName()%>">"<%=files.getFileName()%></a></td></tr>
}I need to know how to alter the above link to make it access/download files from the /WEB-INF/uploads/ while still restricting direct access to the files.
Thanks
Message was edited by:
topiwalAnything under WEB-INF is not directly accessible via the browser/ client. You could try defining the JSP/ Servlet inside web.xml deployment descriptor and see if you can access the file from there.
I think a better way to accomplish this would be to keep the file outside WEB-INF , and use other techniques to protect the folder that contains the file - for example password-protection, or defining a .htaccess file and serving a 403 Forbidden error. -
Alright, the question I have has to deal with how to link an app to a website to download files. How, or is it even possible to link individual picker view selections to download an individual file from a website or other location such as dropbox using xcode. Sorry if this is the wrong place to post this, but if anyone can help or can send a link to a place that can, I would greatly appreciate it!
I am not getting anywhere with deploying my application or
applet.
I have set up my bc4j project. It contains all my VO info,
links, application module. (proj a)
I then have another project with DbInfo in it(has all my rowset
info), Multiple Frames, and my Applet.java file.
Actually I have an Applet.java file and a Application.java file
because I was seeing if both/either worked. Anyway they seem the
same, except for that extra window that comes up when you run the
applet.
I follow the steps in the oracle directions (from earlier post).
And all seems ok. But at ---->
[*] Select the subdirectory under myhtml where your applet's HTML
file
is located, and enter the directory path of the 'staging'
directory you
created in step 3 above, if different from the default.</li>
[*]Select the HTML files that JDeveloper created to run your
applet.</li>
[*]Select all of the Java source files in your project that make
up the
applet.</li>
I have no HTML file associated with my applet, at least that I
know of.
So do I need to create one, or should it of been done
automatically.
Also, I trying to figure out what will be the best way to deploy
my project. Applet or stand alone application is what my first
choices have been so far. I have read that there is some issues
with applets being served from a different server than the
database. So a stand alone application was my front runner, but
I haven't gotten either way to work yet. -
I used to be able to download files from the Harddrive of my Sony Handycam, but now it won't let me import them while going through "log and transfer." I have tried to update my settings, but this doesn't seem to work. Any help would be much appreciated
Hard Drive: The Hard Drive is on my camera (Sony Handycam)
Just needed some info on what is going on with your particular system.
Knowing the exact camera model will also assist us.
If this camera is standard AVCHD you should be able to connect with USB to Mac with FCE open and use Log and Transfer without all the fiddling around converting etc.
What FCE does during ingest is convert files to AIC. (Apple Intermediate Codec)
FCE cannot read the individual files in the BDMV folder, that's why all the converting is required when you use that method.
BTW: regards formatting drives/cards/memory on cameras; it is wise to use the actual device to format rather than a computer. This is a prime cause of read/write goof ups with solid state media. This applies to still or video cameras.
Keeping the COMPLETE card/memory structure in tact is vital for successful transfers.
Try here for some trouble shooting tips:
https://discussions.apple.com/message/12682263#12682263
Al -
Hi,
My scenario this time is to download a file (using http protocol) but in the way of file adapter (ftp) way of connection (polling and schedule).
File should be polled x times a day.
I search forum and one solution is to create a BPM started by batch job with dummy message which will pick up file from the website by java proxy. I would like to avoid BPM and dummy message from SAP system.
I do some investigation, and find out that I can use a java proxy and use it in a client which will be scheduled on XI server. Does anyone created client application on WAS server which is scheduled? I found something which is called Timeout Service, but I do not know if this is right way to use it.
I would also want to know which will be easier and less time intensive to develop this scheduled java client or new adapter?
best regards
Dawidhi,
>>>>I search forum and one solution is to create a BPM started by batch job with dummy message which will pick up file from the website by java proxy. I would like to avoid BPM and dummy message from SAP system.
you can also do it like this:
start a BPM with a dummy message - this BPM will only start an
async java proxy that will fetch the file and post it to XI (with no BPM - just to
a java proxy server)
this way you will only use a BPM to call fetching the file
but it will not process it anymore
what you need to consider it the use of application acknowledgments
(so you will know that the proxy worked)
also the configuration for the proxy call (http dest etc)
will have to be stored somewhere (web interface to configure)
Regards,
michal
<a href="/people/michal.krawczyk2/blog/2005/06/28/xipi-faq-frequently-asked-questions"><b>XI / PI FAQ - Frequently Asked Questions</b></a> -
firefox keeps crashing when I try to launch it. I uninstalled the program, but now when I try to download it from your website the download freezes with 1 second left. This has happened several times.
Hello Deb,
Something fishy going on with those particular files. But without looking at them directly I couldn't say what. Usually in a case like this the problem is some of the metadata that is attached to the file. Where did these files come from? A camera, a scanner, the internet?
An explanation about the Watched Folder function: It looks for changes to a folder, not just automatically grabbing whatever files are in it. Generally, you want to select an empty folder as a Watch Folder. Once this has been set up, as you add files to this folder they will be imported into the Organizer. If you start with a folder full of images, those will be ignored, any new items will be added.
-Brett -
Corrupted content error when trying to download .pdf file from one website.
Unable to download .pdf file from a website, but IE8 provided the download without hesitation.
Both v7 and v8 produced the same message:
"Corrupted content error"
The page you are trying to view cannot be shown because an error in the data transmission was detected.
This may be a particular website problem, or it may be common to the particular industry segment and their protocols.
I have followed the help leads and examined the profile for Firefox as used in this instance. Nothing untoward seemed to pop out as a causal factor.Reworked some of the javascript for our webserver and fixed the problem.
-
HHi having trouble viewing a downloaded file from a site my iPad is telling me I don have a app to open file . And have downloaded loads of apps and still no joy . Have tried this on my android phone to and still nothing. Please can anyone help as slowly going crazy. Thanks tim
What type of file? From what site? What Apps have you downloaded for it?
Perhaps if you provide details of what you are doing we can provide more precise help.
Maybe you are looking for
-
The wheel click button does not open a new window since the latest update (9.0.1)
Hi I just want to report this bug in hope it will be fixed asap The wheel click button does not open a new window since the latest update (9.0.1) Thank you
-
I am trying to import photo's on i photo. It downloads all of the pictures on my camera, how do i only select the photos i want? Is there a way to do this without selecting one photo at a time? is there a way to select multiple images? thanks!
-
Camera Raw Plugin "Not Recognized File Type"
It was suggested that I post this here... I'm trying to install the camera raw update for the Canon Rebel XS in CS4. I downloaded the newest RAW update, and the files still didn't work, and then when I went back and downloaded the camera-specific
-
I have a function that I created and am trying to use it to output data using a ref cursor. My function call is declare ret_cursor crsiweb.types.cursorType; type output_rec_type is record( location varchar2(100), client varchar2(100), number_in long,
-
I am currently working in SCM 5.0 and trying to figure out how to use the Process Chains in transaction RSPC. I understand the logic behind a process chain and am trying to set up some simple ones but am running into problems. Do any of you have ex