Buffered Downloading of a File using a URL

Hi All,
I have been trying to download a file (online - using URL) using the Data Socket Read. But on downloading the file (Minimum Size: 250 MB) the labview memory gets built up i.e I will have to wait till all the packets pertaining to that file are received by the client. I tried using buffered read in Data Socket Open, so that i would to able to obtain instantaneous packets received. But still the memory builds up during the download process. Can anyone please suggest me a solution so that i would be able to obtain the packets as and when they are received by the client instead of waiting for the entire file to be received by the client ?
The data downloaded will be in unformatted text and i have attached the VI for reference. Thanks in advance.
Regards,
S.P.Prasaanth

If you are keeping the file in memory as you are downloading it then it will of course use lots of memory (essentially an array of bytes). I think the key thing here is to periodically flush the chunks data received to disk.
I don't know if this is possible using the DataSocket functions (it maybe buffers the network data, not the file part?) but it should be possible using the raw TCP/IP functions or perhaps even a .NET library (HttpWebRequest?).
Edit: On looking at your VI, I can't see any obvious cause for the whole file to be in memory unless it's something relating to how DataSockets are implemented internally. In which case - my point still stands that you could try the .NET library or doing the http by raw TCP/IP.

Similar Messages

How to download a text file using classes

Hi Guys
I want to download a text file using classes.
Any idea how to do it ?
Thanks
Sameer

HI sameer,
   It is no different from using a Function module.
example:
CALL METHOD cl_gui_frontend_services=>gui_download
    EXPORTING
      filename                  = w_fpath
    append                    = SPACE
      write_field_separator     = 'X'
    CHANGING
      data_tab                  = it_download
    EXCEPTIONS
      file_write_error          = 1
      no_batch                  = 2
      gui_refuse_filetransfer   = 3
      invalid_type              = 4
      no_authority              = 5
      unknown_error             = 6
      header_not_allowed        = 7
      separator_not_allowed     = 8
      filesize_not_allowed      = 9
      header_too_long           = 10
      dp_error_create           = 11
      dp_error_send             = 12
      dp_error_write            = 13
      unknown_dp_error          = 14
      access_denied             = 15
      dp_out_of_memory          = 16
      disk_full                 = 17
      dp_timeout                = 18
      file_not_found            = 19
      dataprovider_exception    = 20
      control_flush_error       = 21
      not_supported_by_gui      = 22
      error_no_gui              = 23
      OTHERS                    = 24
Regards,
ravi

Why can't I download a pdf file using internet explorer as my browser.I can download pdfs using fire

Why can't I download a pdf file using internet explorer as my browser. I can download pdfs using firefox.

Gilad D,
Thanks for the support.
I would guess the plugin is enabled since when I use Foxfire as my browser, all pdfs download and are readable. (just doesn’t happen when I use IE as my browser, I get a blank screen with a small “x” in the upper left corner)
I will try your proposed solution.
Charlie

How to download video files using http urls through OSB

Hi all,
I am working on a requirement where I need to download the video using http url of that video through OSB. Is it possible to convert the video files hosted in a server into base64 code using OSB ? Kindly help me on this.
Thanks
Edited by: 887737 on Jul 25, 2012 3:26 AM

Hi,
I believe you can get the video file using file transport and then convert it into base64 using a java call out...
This may help...
https://blogs.oracle.com/ateamsoab2b/entry/an_example_of_how_to
Cheers,
Vlad

Download a file using a url

Hi all,
is there a class that allows to download a file giving an url as parameter?
Thanks a lot, it will make me win a bet I made with a .net supporter.
Marco

No there is no class, but you can write the stuff yourself
URL downloadURL = new URL("http://www.download.xxx/index.html");
FileWriter local = new FileWriter("C:\index.html");
InputStream internet = downloadURL.openStream();
byte[] buffer = new byte[1000];
int bytesRead;
while (true) {
    bytesRead = internet.read(buffer);
    if (bytesRead == -1) break;
    local.write(buffer, 0, bytesRead);
}The code reads the file from "http://www.download.xxx/index.html" and writes it to "C:\index.html".

Downloading a binary file using sockets without length known

Hi all,
I'm trying to download a binary(.exe) file using socket, where the length of the file is not known. Please take a look at the code I'm using:
      var readBin = socketBin.read();
     <-- Here comes the code that checks for http Content-length header field
     ; If content-length field is available, then I'll use it to download file, else proceed with following code -->
    pBar.reset("Downloading plugin..",null);pBar.hit(readBin.length);
            while (1)
                binFil.write(readBin);     //'binFil' is the file, into which downloaded file is written.
                readBin = socketBin.read();
                pBar.hit(readBin.length);
                if( socketBin.eof || readBin.length<=0){
                    break;}
            binFil.write(readBin);
            binFil.close();
            socketBin.close();
Problem is: I'm able to download file within 10 seconds when content-length is known. But when content-length is not known, its taking about 1 and half minute to download, also the progress bar gets struck for much of time.
FYI: socket is opened in binary mode, file is getting downloaded correctly(even though it takes abt a minute). BTW Im using CS5.5 extendscript
I'm not able to figure out where the bug is.

Hi, Srikanth:
Sevaral points.
#1 When posting code, please use the web interface and the >> icon and choose Syntax Highlighting >> Java. Otherwise your code is just too hard to read and gets misformatted.
#2 Apropos of #1, your script as written does not work, because this line:
url=url.replace(/([a-z]*):\/\/([-\._a-z0-9A-Z]*)(:[0-9]*)?\/?(.*)/,"$ 1/$2/$3/$4");
has an extra space in the $1 and should be this:
url=url.replace(/([a-z]*):\/\/([-\._a-z0-9A-Z]*)(:[0-9]*)?\/?(.*)/,"$1/$2/$3/$4");
Please take an extra moment to make sure that when you asking for help, you do so in a way that makes sense. Otherwise it takes too much effort to help you, and that is frustrating.
#3 If we instrument your script by adding a line to the while() loop:
            while (1)
                binFil.write(readBin);
                readBin = socketBin.read();
                $.writeln(Date()+" Read "+readBin.length+" chars, eof is "+socketBin.eof);
                if( readBin.length<=0){         break;}
We get output like this:
Mon Jul 11 2011 12:06:56 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:06:56 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:06:56 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:06:56 GMT-0400 Read 631 chars, eof is false
Mon Jul 11 2011 12:07:06 GMT-0400 Read 0 chars, eof is false
Result: undefined
Therefore, we can reasonably conclude that the last read at the end of the data stream returns a short read when the other side blocks.
Unfortunately, that's clearly insufficient since you can get short reads all the time.
I'm not sure why you say the length of the file is not known, HTTP provides it to you in the Content-Length field of the response.
But the easy answer is to get the server to close it for you. This is easy, enough, with Connection: close. Why were you specifying
Connection: keep-alive, anyhow?:
            var requestBin =
            "GET /" + parsedURLBin.path + " HTTP/1.1\n" +
            "Host: " + parsedURLBin.address + "\n" +
            "User-Agent: InDesign ExtendScript\n" +
            "Accept: */*\n" +
            //"Connection: keep-alive\n\n";
            "Connection: close\n"+
            "\n";
This yields a nice tidy:
Mon Jul 11 2011 12:26:19 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:26:19 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:26:19 GMT-0400 Read 1024 chars, eof is false
Mon Jul 11 2011 12:26:19 GMT-0400 Read 735 chars, eof is true
Mon Jul 11 2011 12:26:19 GMT-0400 Read 0 chars, eof is true
I suspect you're also better off using something like socketBin.read(64*1024) for a 64k buffer size, but it doesn't seem to effect the on-the-wire protocol, so perhaps its not so important.
If you don't want to reply on the server

Download O365 installer files using Office Deployment Tool times out...

Hi,
We need to deploy O365ProPlus as part of an operating system image using MDT. I was pointed to this page for instructions:
https://technet.microsoft.com/en-us/library/dn314789.aspx
We first needed to download the source files, so I first downloaded the Office Deployment Tool and used this to download the O365ProPlus image to my MDT server.
I created the following config file (O365x64.xml):
<Configuration>
<Add SourcePath="X:\O365\" OfficeClientEdition="64">
   <Product ID="O365ProPlusRetail" >
     <Language ID="nb-no" />
   </Product>
</Add>
</Configuration>
And I invoke the command to run the Office Deployment Tool:
.\setup.exe /download X:\O365.x64.xml
Then the process creates a folder X:\O365\Office\Data with a couple of 16kb CAB files, but nothing happens apart from this.
What am I doing wrong...? I expect the Office folder to be alteast a gigabyte...
Thanks for comments
Tor

Please first check your network or switch to a different network, then try again.
Additionally, you might want to check if your network setting/firewall setting is blocking any Office 365 IP address/URL:
https://technet.microsoft.com/en-us/library/hh373144.aspx
Regards,
Kapaal

When I download a PDF file using Adobe I get a black screen when attempting to print and if I save the file I get a corrupted response when opening it.

This does not always happen, for example if I get a PDF file by email, it appears to be primarilly when I download from the internet.
I use Adobe Reader X1. Can you assist please. Thanks

Thanks Eric, no it does not help the matter. I opened the file using preview and the same message appeared. Thanks.

HT4101 downloading only JPEG files using SD card reader

I am new to Apple-iPad world,, mostly working on WinXP.
I am looking to buy a tablet to view photos while I'm on vacation and traveling. I shoot both RAW and JPEG files of each image. When I connect my SD card to an iPad Mini, I'd like to be able to download only the JPEG files, since the RAW files are too large and would quickly fill up my device.
I've been told that when you plug in the Apple SD card reader, it treats the RAW and JPEG files of an image as one, so when you download you have no choice to just select the JPEGs. Seems like a huge oversight on Apple's part, to offer camera and card connection, but then be so controlling about it, that it doesn't work for people. None of the 3-4 Apple sales reps I've spoken to know of a way around this.
At this point, I'm lookng at an Android tablet instead, since it doesn't have this limiting feature--though I much prefer the look and size of the iPad Mini screen. .
Has anyone else run into this issue? Any thoughts on a work-around?
(Two solutions that I've thought of--though I don't know if they will work--are: 1) download all of the images, then bring them up in a program like Snapseed, where I can save (really a 'save as') a new JPEG version of my image, then delete the original files that I downloaded, or 2) see if I can download a file manager app that would let me view the different files by size and by type (to identify the RAW files), then delete just these from me iPad)
Best, David.

Thanks Tom, at last an answer!
So it seems I would need to first download the image files to my tablet--including RAW files--(or whatever the constraits are in working with the Apple SD card reader), and then after the fact, use PMP to delete just the RAW files.
I assume then that PMP can view the 'standard' photo folders (called 'Albums on my iPod) created by the Apple photo app (or by the iOS, I'm not sure), is that correct?
This might be an acceptable work-around for me.
Best, David.

How do you download a static file using a FileReference and URLRequest and show the progress?

All,
   I'm trying to download static content (PDFs) off of my server using tutorials from adobe's site and some online as well and i'm having no luck showing an updated progress bar. I attached the meat of my code below. If a user clicks a hyperlink, it calls the downloadPDF method passing in a constant url defined.
When debugging, i can see that the event.bytesLoaded is incrementing on each call to the progress method, however the bytesTotal is only set to the intitial number of bytes transferred on the first pass through the method. (Ex: First time the method is called, bytesLoaded and bytesTotal will be 4,000... which i dont understand... the bytesTotal should be the total size of the file being downloaded.. definitely not the case... the next time the method is called, bytesLoaded will increases to some number while bytesTotal will remain at 4,000). This happens during the entire file transfer.
Am i missing something? Are the examples incorrect? Also to note, i'm using sdk 3.2 (Not sure if that makes a difference.)
Thanks for any response or insight.
Damian
<mx:Script>
        <![CDATA[
            import com.ctc.fema.resources.Buttons;
            import flash.net.FileReference;
            private var fileRef:FileReference;
            private var urlReq:URLRequest;
             * Constant values for user manual pdfs
            private static const PDF_STRING:String = "blahblahblah.pdf";
            private function init():void
                /* Define file reference object and add a bunch of event listeners. */
                fileRef = new FileReference();
                fileRef.addEventListener(Event.COMPLETE, completeHandler);
                fileRef.addEventListener(Event.OPEN, openHandler);
                fileRef.addEventListener(ProgressEvent.PROGRESS, progressHandler);
            private function doEvent(evt:Event):void
                /* Create shortcut to the FileReference object. */
                var fr:FileReference = evt.currentTarget as FileReference;
                try
                    /* Update the Model. */
                    fileRefModel.creationDate = fr.creationDate;
                    fileRefModel.creator = fr.creator;
                    fileRefModel.modificationDate = fr.modificationDate;
                    fileRefModel.name = fr.name;
                    fileRefModel.size = fr.size;
                    fileRefModel.type = fr.type;
                    /* Display the Text control. */
                catch (err:*)
                    /* uh oh, an error of sorts. */
            private function downloadPDF(url:String):void
                /* Begin download. */
                urlReq = new URLRequest(url);
                fileRef.download(urlReq);
             * When the OPEN event has dispatched, change the progress bar's label
             * and enable the "Cancel" button, which allows the user to abort the
             * download operation.
            private function openHandler(event:Event):void
                downloadStatusContainer.visible = true;
                downloadStatusContainer.includeInLayout = true;
                downloadProgressBar.label = "DOWNLOADING %3%%";
                cancelButton.enabled = true;
             * While the file is downloading, update the progress bar's status.
            private function progressHandler(event:ProgressEvent):void
                downloadProgressBar.setProgress(event.bytesLoaded, event.bytesTotal);
             * Once the download has completed, change the progress bar's label one
             * last time and disable the "Cancel" button since the download is
             * already completed.
            private function completeHandler(event:Event):void
                downloadStatusContainer.visible = false;
                downloadStatusContainer.includeInLayout = false;
                downloadProgressBar.label = "DOWNLOAD COMPLETE";
                cancelButton.enabled = false;
             * Cancel the current file download.
            public function cancelDownload():void
                fileRef.cancel();
                downloadProgressBar.label = "DOWNLOAD CANCELLED";
                cancelButton.enabled = false;
                downloadStatusContainer.visible = false;
                downloadStatusContainer.includeInLayout = false;
        ]]>
    </mx:Script>

The tick counter has a resolution of 1 ms. Resolution, accuracy, and the responsiveness of the OS are three different things. The issue is that OS latency can be 10s of milliseconds or occasionally longer. If the OS decides to index the hard drive between the time you read the tick count and send the start audio command, your tone could be quite late. Sending the TTL pulse is a third call to the OS. So you have two latency times for each trial. Unless you have a real-time operating system, this latency issue will introduce randomness into your data.
Try sending TTL pulses of 50 ms duration every 200 ms, software timed, for a few minutes and look at the variation in the edge timing. Then try it again with tones thrown into the mix and see if the variation changes.
That is where hardware timing and synchronization pays off.
Lynn

Upload/Download speed of files using Java

Hey All,
I have created a signed Java applet which, among various other things, allows uploading and downloading of files between the server and client. Both Upload and Download is handled through HTTP. For Upload I create a multi-part POST request to transfer the data to a script on the server which intercepts it. For download I craft a URL to http://myserver.com/filedirectory/file.extension. From that I form a HttpURLConnection and read from the stream to the file system of the client. This applet is only run on an internal network, which currently has 100mbit connection. However, I am noticing upload/download speeds of less than 1MB/s (I believe 100mbit = 12MB/s approx). I am currently using a buffer size of 8kb.
Can anyone think of any ways to increase download speed? Would it help to increase the buffer size, knowing we will always be on a closed network? Here Is my upload code, and my download is similar but in the opposite direction.
int bytesRead = 0;
byte b[] = new byte[BUFFER_SIZE];
BufferedInputStream bufin = new BufferedInputStream(new FileInputStream(f));
while ((bytesRead = bufin.read(b)) != -1) {
        * write output
     out.write(b, 0, bytesRead);
     out.flush();
out.close();
bufin.close();I have not included the rest of the code which creates the POST request because the upload is otherwise working perfectly despite the speed, and I have confined the bottleneck to this part using debug output.
Thanks much

Will removing the flush after every loop somehow decrease accuracy?No.
Ideally, files should be the same exact size on the server as the client after upload has finished.Deferring the flush() can't change that. It just lets you write in larger chunks, which is always good.
Which one is the socket receive buffer?The buffer whose size is set by Socket.setReceiveBufferSize().
Currently the size of BUFFER_SIZE in the example above is set to 8k.That's the application buffer. Good size.

When I download a .pdf file using mozilla, I cannot open that file despite having adobe reader on my desktop. I cannot save the file to open at a later time

I wished to view a .pdf file of a vehicle brochure. Mozilla found the brochure ok, but would not let me view it or save it. I ended up having to use IE to download and then view the file. How do I enable Mozilla to allow me to view .pdf files< I have Adobe Reader 9 installed on my PC (using Dell and Window 7)

https://support.mozilla.com/en-US/kb/Using+the+Adobe+Reader+plugin+with+Firefox
https://support.mozilla.com/en-US/kb/Opening+PDF+files+within+Firefox

How to download a text file using jsp i

I want to provide download facility in jsp page.
if i have exe download is working . If I have a text / html file it is just displaying I want to make it download How to do it?
regards
subrahmanayma

If you want to get an "Save as" dialogue, then you need to set the content-disposition in the header to "attachment". The default value is "inline".
Best solution is to create kind of a FileServlet and let it stream the right file as a download to the client. You may find the downloadFile() snippets at the bottom of this article useful: http://balusc.xs4all.nl/srv/dev-jep-pdf.html Put it in a FileServlet and call it using for example .

Heading during downloading into local file using GUI_DOWNLOAD fun.module

Hi Guru's
we have a requirement that we want the plant description as a heading (first line of the file) in the local file. iam using "GUI_DOWNLOAD" function module for downloading data for which iam passing the charecter type internal table. before downloading iam passing all the filed headings to that table and then appending the internal table data into it. now iam getting data properly with field headings . but before that heading i want one more description for a plant field which iam using in my selection screen
in the fun.module "GUI_DOWNLAOD"
we have HEADER file but it is of XSTRING type so it is taking only 2 char.
so how to use this . Plz help me.
thanks well in advance.
UR's
GSANA

Hi,
Please check the below link,
header in 'gui_download'
Also check Manoj kumar's reply in the link,
header information to gui_download
Hope this helps.
Best Regards.

Problem downloading .doc, .xls file using servlet

i have a servlet which downloads the files to client(browser). It works file for .txt file but .doc, .xls files are opend but they contain garbage.
plz letr me know if u know problem!!!!!
-amit

Probably because you output them using the Writer you got from getWriter instead of using the OutputStream you can get from getOutputStream.

Buffered Downloading of a File using a URL

Similar Messages

Maybe you are looking for