Download multiple files with 1 HttpURLConnection

Hi,
We have an application that downloads small text files (< 2 Kb) from a webserver. These files are downloaded 1 by 1 when needed. We open a HttpURLConnection, download 1 file and close the connection immediately afterwards. This is done for each file and hence produces a significant overhead.
For performance reasons (both client and server side), we would like to open a connection to the webserver once, download a number of files and then close the connection.
Currently, our code roughly looks like:
InputStream is;
HttpURLConnection con;
int resp;
try {
    URL url = new URL(location);
    con = (HttpURLConnection)url.openConnection();
    resp=con.getResponseCode();
    if(resp==HttpURLConnection.HTTP_OK) {
        is = con.getInputStream();
        // ... read data from the InputStream
        is.close();
} catch (MalformedURLException murle) {
    murle.printStackTrace();
} catch (IOException ioe) {
    ioe.printStackTrace();
}Is it even possible to download multiple files from a web server using a HttpURLConnection? If so, how? :-)
Please note that the solution should be compliant with J2SE 1.1, since we are writing an MHP application!
Thanks in advance...

Seerix,
You might be able to fudge the settings of the URL class :
Class URLStreamHandler
Method protected void setURL
Sets the fields of the URL argument to the indicated values. Only classes derived from URLStreamHandler are supposed to be able to call the set method on a URL.
But this is not the best way, as URLs are supposed to be constant.
Your alternative, is to create a Socket, deal with the protocol requirements
(i.e. HTTP GET, POST and so on) then you can request any number files
on that same connection. The only thing that this saves is connection build up and tear down.
You could also multi-thread your original code ....... Although that might
only save you some network transit time.
-Cludge

Similar Messages

  • Download multiple files in one step

    Hi,
    my first try with acrobat.com with uploading some pictures which I like to share with friends led me to the question, how they can download my 18 pictures in one step. I put them in a workspace and gave them access. You can mark all files, but there is no menu point which offers you to download all files in one step.
    Is there a way to do this in one step?
    Greetings
    Mario

    Hello Mario,
    Thank you for posting. Unfortunately, there is currently no way to download multiple files at once; this functionality is high on our list of planned enhancements, so I encourage you to stay tuned for future releases.
    Best regards,
    Rebecca

  • Aria2 does not download multiple files at a time

    I'm trying to speed up my system upgrades by using aria2, but it won't speed up anything as it does not download multiple files at a time.
    As per this wiki post, https://wiki.archlinux.org/index.php/Im … erformance , "using aria2", I've installed aria2 and added the line
    XferCommand = /usr/bin/aria2c --allow-overwrite=true -c --file-allocation=none --log-level=error \
    -m2 --max-connection-per-server=2 --max-file-not-found=5 --min-split-size=5M --no-conf \
    --remote-time=true --summary-interval=60 -t5 -d / -o %o %u
    to the options section in pacman.conf. Pacman still works, but it isn't sped up. When pacman -Syu'ing, a normal line shows:
    #1 SIZE:1.3MiB/2.0MiB(63%) CN:1 SPD:50.0KiBs ETA:15s
    What's going on?
    P.S. I use the line reflector -l 5 --sort rate --save /etc/pacman.d/mirrorlist before doing a system upgrade.
    Last edited by bruno321 (2011-05-15 15:27:45)

    Using aria2c will download files over 5MB with multiple connections but will not download multiple files at the same time. Pacman calls aria2c with the first url, waits for it to finish, then calls aria2c with the second etc...
    If you want to download multiple packages simultaneously, see http://xyne.archlinux.ca/projects/pacman2aria2/

  • OSD: Downloading multiple files is very slow

    I has a exe software. It is not a single file package, it contains a lot of files. when I installed this software in OSD, I found it would spend 15~20minutes to download files.
    It was very strange. The software is only 750 mb size. the speed of downloading 6 GB OS image is also much faster than this software.
    I know downloading multiple files is slower than downloading a single file, if their size are same.
    But the downloading of multiple files in OSD is "really" very slow.
    Why?  Cause of BITS or RDC?  How to fix it?
    PS:I update to SCCM 2012 R2 with CU1,  add the below into SCCM
    Windows Registry Editor Version 5.00
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP]
    "RamDiskTFTPBlockSize"=dword:00004000
    http://windowsdeployments.net/how-to-speed-up-pxe-boot-in-wds-and-sccm/
    Now, the download speed up to 4 minutes, but it still slower than the normal copy & paste. 
    Test result:
    OSD: 2.6G
    single image, 2 minutes
    OSD: 750m multiple files, 4 mintues
    after OSD, copy & paste from network drive, 750 multiple files, 1 mintue

    I had updated to CU1 and add this into regedit . It is quicker than before, but it still slower than the normal.
    Windows Registry Editor Version 5.00
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP]
    "RamDiskTFTPBlockSize"=dword:00004000
    http://windowsdeployments.net/how-to-speed-up-pxe-boot-in-wds-and-sccm/
    Download 2.63 G OS image:
    2min
    Download 750 M multiple
    files 4min

  • Download multiple files

    hi,all
    downloading files from server has been discussed for many times, i have known how to download one file after reading some older topics,but now i want to download
    multiple files,and I want browser prompt "Save As" only one time, then all the files would be downloaded to the same location,can anybody give me some advice?
    thanks & regards

    Hi,
    I don't think it sounds like what you're tyring to do would be possible for security reasons. I'd be quite worried if a browser was allowed to save multiple files after only asking once.
    I guess one solution would be to zip all the files up into one zip file (see JavaDocs for java.util.zip) and then transfer this zip file (this would also probably help with download times). This may not be the ideal solution for you though.

  • FTP Activity - mget multiple files with dynamic names

    Hello All
    Hope all is well.
    I was wondering if there was a way to configure FTP activity to transfer multiple files with dynamic names from source location at daily intervals.
    For example, my source files are affixed with date values (source_filename_<mmddyyyy>_<hh>.txt):
    source_filename_12252009_00.txt
    source_filename_12252009_06.txt
    source_filename_12252009_12.txt
    source_filename_12252009_18.txt
    What I would like my process flow to do would be to download just these 4 files once daily based on a dynamic ftp script as follows:
    open ${Remote.Host}
    ${Remote.User}
    ${Remote.Password}
    cd ${Remote.RootPath}
    prompt off
    mget ????????
    quit
    In DOS batch script you would have the mget command with a wildcard as:
    mget source_filename_%DATE:~0,2%%DATE:~2,2%%DATE:~4,4%*.txt
    How does one go about implementing this in OWB? Does it require a user defined substitution varilable of some sort?
    FYI, I am using version 10gR2.
    Cheers.

    Hi ,
    If you are able to achieve your objective from a DOS batch script then you can call that script for an User Defined Activity in OWB process flow .
    Hope this helps.
    Thanks,
    Sutirtha

  • Upload multiple files WITH correct pairs of form fields into Database

    In my form page, I would like to allow 3 files upload and 3 corresponding text fields, so that the filename and text description can be saved in database table in correct pair. Like this:
    INSERT INTO table1 (filename,desc) VALUES('photo1.jpg','happy day');
    INSERT INTO table1 (filename,desc) VALUES('photo2.jpg','fire camp');
    INSERT INTO table1 (filename,desc) VALUES('photo3.jpg','christmas night');
    However, using the commons fileupload, http://commons.apache.org/fileupload/, I don't know how to reconstruct my codes so that I can acheieve this result.
    if(item.isFormField()){
    }else{
    }I seems to be restricted from this structure.
    The jsp form page
    <input type="text" name="description1" value="" />
    <input type="file" name="sourcefile" value="" />
    <input type="text" name="description2" value="" />
    <input type="file" name="sourcefile" value="" />The Servlet file
    package Upload;
    import sql.*;
    import user.*;
    import javax.servlet.*;
    import javax.servlet.http.*;
    import java.util.Map;
    import java.util.HashMap;
    import java.util.Date;
    import java.util.List;
    import java.util.Iterator;
    import java.io.File;
    import java.io.PrintWriter;
    import java.io.IOException;
    import java.text.SimpleDateFormat;
    import org.apache.commons.fileupload.servlet.ServletFileUpload;
    import org.apache.commons.fileupload.disk.DiskFileItemFactory;
    import org.apache.commons.fileupload.*;
    public class UploadFile extends HttpServlet {
    private String fs;
    private String category = null;
    private String realpath = null;
    public String imagepath = null;
    public PrintWriter out;
    private Map<String, String> formfield = new HashMap<String, String>();
      //Initialize global variables
      public void init(ServletConfig config, ServletContext context) throws ServletException {
        super.init(config);
      //Process the HTTP Post request
      public void doPost(HttpServletRequest request,HttpServletResponse response) throws ServletException, IOException {
        request.setCharacterEncoding("utf-8");
        response.setCharacterEncoding("utf-8");
        response.setContentType("text/html");
        PrintWriter out = response.getWriter();
        Thumbnail thumb = new Thumbnail();
        fs = System.getProperty("file.separator");
        this.SetImagePath();
         boolean isMultipart = ServletFileUpload.isMultipartContent(request);
         if(!isMultipart){
          out.print("not multiple part.");
         }else{
             FileItemFactory factory = new DiskFileItemFactory();
             ServletFileUpload upload = new ServletFileUpload(factory);
             List items = null;
             try{
                items = upload.parseRequest(request);
             } catch (FileUploadException e) {
                e.printStackTrace();
             Iterator itr = items.iterator();
             while (itr.hasNext()) {
               FileItem item = (FileItem) itr.next();
               if(item.isFormField()){
                  String formvalue = new String(item.getString().getBytes("ISO-8859-1"), "utf-8");
                  formfield.put(item.getFieldName(),formvalue);
                  out.println("Normal Form Field, ParaName:" + item.getFieldName() + ", ParaValue: " + formvalue + "<br/>");
               }else{
                 String itemName = item.getName();
                 String filename = GetTodayDate() + "-" + itemName;
                 try{
                   new File(this.imagepath + formfield.get("category")).mkdirs();
                   new File(this.imagepath + formfield.get("category")+fs+"thumbnails").mkdirs();
                   //Save the file to the destination path
                   File savedFile = new File(this.imagepath + formfield.get("category") + fs + filename);
                   item.write(savedFile);
                   thumb.Process(this.imagepath + formfield.get("category") +fs+ filename,this.imagepath + formfield.get("category") +fs+ "thumbnails" +fs+ filename, 25, 100);
                   DBConnection db = new DBConnection();
                   String sql = "SELECT id from category where name = '"+formfield.get("category")+"'";
                   db.SelectQuery(sql);
                    while(db.rs.next()){
                      int cat_id = db.rs.getInt("id");
                      sql = "INSERT INTO file (cat_id,filename,description) VALUES ("+cat_id+",'"+filename+"','"+formfield.get("description")+"')";
                      out.println(sql);
                      db.RunQuery(sql);
                 } catch (Exception e){
                    e.printStackTrace();
            HttpSession session = request.getSession();
            UserData k = (UserData)session.getAttribute("userdata");
            k.setMessage("File Upload successfully");
            response.sendRedirect("./Upload.jsp");
      //Get today date, it is a test, actually the current date can be retrieved from SQL
      public String GetTodayDate(){
        SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
        String today = format.format(new Date());
        return today;
      //Set the current RealPath which the file calls for this file
      public void SetRealPath(){
        this.realpath = getServletConfig().getServletContext().getRealPath("/");
      public void SetImagePath(){
        this.SetRealPath();
        this.imagepath = this.realpath + "images" +fs;
    }Can anyone give me some code suggestion? Thx.

    When one hits the submit button - I then get a 404 page error.What is the apaches(?) error log saying? Mostly you get very useful information when looking into the error log!
    In any case you may look at how you are Uploading Multiple Files with mod_plsql.

  • Receiving multiple files with dual extension using file adapter

    Hi,
    The scenario where I am implementing requires multiple files of different names to be picked(ABC.txt) and dropped at destination(ABC.txt.pgp) up by my File Sender and Receiver adapters.  Any idea on about how the configuration for receiving multiple files with dual extension for receiver adapter is to be done?
    Note:- currently, The scenario is working fine without dual extension. i.e Its picking all files which starts with ABC and creating at destination as it is. even I configured as ABC.txt.pgp, but its not creating the second extension.
    Thanks in Advance
    Manmadha

    Hi,
    Try to concatenate '.pgp ' to the source file name to create the target file name for the receiver file adapter, by accessing the Adapter Specific Attributes using Java user defined function. This might work.
    Reference links:
    <a href="http://help.sap.com/saphelp_nw04/helpdata/en/43/03612cdecc6e76e10000000a422035/frameset.htm">SAP Help - Accessing Adapter Specific Attributes</a>
    <a href="/people/michal.krawczyk2/blog/2005/11/10/xi-the-same-filename-from-a-sender-to-a-receiver-file-adapter--sp14 - XI: The same filename from a sender to a receiver file adapter - SP14</a>
    Regards,
    S.Santhosh Kumar

  • Firefox doesn't reconvert special characters in the file names when download a file with any special characters in the file name

    <blockquote>Locking duplicate thread.<br>
    Please continue here: [/questions/815207]</blockquote><br>
    if i try to download a file with any special characters in file name (e.g. File_Name.pdf), it doesn't reconvert them from the "sanitize url" process and download the file an incorrect name (e.g. File%5FName.pdf).
    This is really annoying.
    Thank you for your patient

    Start Firefox in <u>[[Safe Mode]]</u> to check if one of the extensions is causing the problem (switch to the DEFAULT theme: Firefox (Tools) > Add-ons > Appearance/Themes).
    * Don't make any changes on the Safe mode start window.
    * https://support.mozilla.com/kb/Safe+Mode
    * [[Troubleshooting extensions and themes]]

  • Selecting Multiple Files With Shift Key

    Am I right in understanding that the only way to select multiple files with the Shift key in Finder is by using File view? Seems kinda silly.
    Thank you, Scott

    I'm at my Snow Leopard machine right now, and I'm not sure that nothing was changed in Mavericks, but I never noticed that it was.
    I have four view modes: Icon, List, Column and CoverFlow.
    I almost always use Column view, but I am guessing that you are calling Icon view File view. In Icon view, Shift clicking seems to work the way Command clicking does in Column view (and probably the other views as well): it allows you to add randomly selected files to the selection. In Column view, Shift clicking works like it does for most things (like text selection); it selects every thing between the previous click and the Shift click.

  • How To Split File In to Multiple Files With out using B.P.M

    Hi Guys,
    How To Split File In to Multiple Files With out using B.P.M.
    Thanks in advance
    Regards's
    KIran.B

    Hello
    below r the links were u will find message spilitting by graphicaaly i.e without using BPM.
    /people/claus.wallacher/blog/2006/06/29/message-splitting-using-the-graphical-mapping-tool
    Sender File Adapter with file conversion  Multimapping --file content conversion with split messg mapping
    https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/2748---- [original link is broken] [original link is broken] [original link is broken]
    multimappig
    /people/narendra.jain/blog/2005/12/30/various-multi-mappings-and-optimizing-their-implementation-in-integration-processes-bpm-in-xi
    hope this resolve your problem
    thank's
    Chetan

  • To create multiple files with same content but with different names

    Hi SapAll.
    here i have got a tricky situation on Idoc to File Scenario.
    in my interface of an Idoc to file ,there  is requirement to create multiple files with different file names but with same content based on one Idoc Segment.
    which means there will be one Zsegment with two fields in the idoc,where one field with (content refers to the name which file name should start with .so lets say if this segment is repeated for 3 times then PI should create 3 files in the same directory with same content but with different file names (from the filed).
    so here for now iam using one reciever file communication channel.
    can any body give me the quick answer.
    regards.
    Varma

    What do you mean by different names?
    when i make proper setting in the Receiver Channel....on how to create the filename (what to append) like add Timestamp, counter, date, messageid.....even in this case you will ahve file with different names and that too from same File channel.
    You can perform multi-mapping in XI/ PI and then your File channel will place the files in the target folder with relevant names. You cannot use Dynamic Configuration with Multi-Mapping!
    If you intend to use different File channels, then do the configuration as required (normal)...even over here you can follow multi-mapping.
    Do not use a BPM!
    Regards,
    Abhishek.

  • Read multiples files with same extension

    how to read multiples files with same extension in java.
    for ex : i would like to read all .DAT files from C drive using java.
    How is it done

    - You create the filter
    - You get the list of files
    - You open and read each file.
    For the first two above you look at java.io.File and listFiles(FileFilter filter).
    For the third you find whatever input stream is appropriate from java.io.*

  • Batch Rename Multiple files with different names

    Hi,
    Is there any way to batch rename multiple files with individual names? I.e
    IMG_123 changed to  RSP45AS
    IMG_124 changed to MOL157A
    IMG_125 changed to AGKH135
    IMG_126 changed to MNOLH13
    IMG_127 changed to ASFBLUG
    Etc.
    Are they any programs or scripts or plug ins that would do that job?
    Thanks

    HI there Onemorewave,
    It looks like there is a batch remname feature included in OS X 10.10 Yosimte if you are planing on updating. 
    Rename files, folders, and disks - Mac Help
    Rename multiple items
    Select the items, then Control-click one of them.
    In the shortcut menu, select Rename Items.
    In the pop-up menu below Rename Folder Items, choose to replace text in the names, add text to the names, or change the name format.
    Replace text: Enter the text you want to remove in the Find field, then enter the text you want to add in the “Replace with” field.
    Add text: Enter the text to you want to add in the field, then choose to add the text before or after the current name.
    Format: Choose a name format for the files, then choose to put the index, counter, or date before or after the name. Enter a name in the Custom Format field, then enter the number you want to start with.
    Click Rename.
    Note: To batch rename, you would want to choose the "Format" option.
    -Griff W

  • HT5701 Cannot download pdf files with Safari 6.0.4?

    Cannot download pdf files with Safari 6.0.4

    Back up all data.
    Triple-click the line of text below to select it, the copy the selected text to the Clipboard (command-C):
    /Library/Internet Plug-ins
    In the Finder, select
    Go ▹ Go to Folder
    from the menu bar, or press the key combination shift-command-G. Paste into the text box that opens (command-V), then press return.
    From the folder that opens, remove any items that have the letters “PDF” in the name. You may be prompted for your login password. Then quit and relaunch Safari, and test.
    The "Silverlight" web plugin distributed by Microsoft can also interfere with PDF display in Safari, so you may need to remove it as well, if it's present.
    If you still have the issue, repeat with this line:
    ~/Library/Internet Plug-ins
    If you don’t like the results of this procedure, restore the items from the backup you made before you started. Relaunch Safari again.

Maybe you are looking for

  • ITunes 11.0 won't install correctly when I try to install it

    So, I've had multiple problems with this now, first it wouldn't update and gave me an error message saying "Error writing to file: C:\ProgramData\188F432-103A-4ffb-80F1-36B633C5C9E1\x86\x86\GEARAspiWDM.sys. Verify that you have access to that directo

  • Time Machine and iWeb file

    I have used time machine for weeks. Yesterday I deleted an iWeb file I was preparing for publishing and decided I wanted to do it differently. Now I want to restore the iWeb file I had 2 days ago. How do I find that file in Time Machine? I have no id

  • SOS help please. Macbook Pro won't turn on. Blinking Light

    Hello to everyone this is my first post here and sadly its about a problem with my macbook pro. The notebook in question is a 17'' Macbook Pro I bought in September 2008. I simply closed it (without turning it off, just let it go on sleep) and took i

  • Invalid COMMIT WORK in an update function module.

    Hi,Experts,      Currently We got a problem for posting PO's GR with t-code MIGO. Our requirement is that can create Delivery by automaitcly after PO's GR is saving. Then I have tailored a program for creating delivery.The program will be triggered i

  • Customizing Error Pages

    Hi, When I define a Exception.jsp script to handle exceptions, it applies for all applications. I want to apply exceptions sepearatly for different aplications so I created the Exception.jsp under /apps/<mysitename>/sling/servlet/errorhandler/, but i