Problem when streaming out a binary file

Hi,
I am trying to stream out a binary file to an output stream (not a file, but a socket). My file is a gzip file, and I was initially simply trying to open the
file, read it to a a byte array, and writing it out to my output stream. However, I got corrupted data at my other end and it took me quite a bit of
debugging hours to find out why.
I tried writing to a file the same information I was writing to my output stream but, although my file was correct, my output stream still contained
corrupted data.
I think I have finally narrowed down my problem to the fact that when I try to write a 0x00 value to my output stream, everything gets ignored from this
point until I write a 0x0a byte to my output stream. Obviously this produces corrupted data, since I need all the bytes of my binary file.
If somebody can direct me as to how to write a 0x00 value to a socket output stream, I would really appreciate your input.
This is a sample of my code (Right now I am writing byte by byte: not very efficient, but was the only way I found where I was getting a problem):
// out is of type OutputStream
if (Zipped){
     // In this case we just have to stream the file out
File myFile = new File(fileName);
     FileInputStream inputFile = new FileInputStream(myFile);
     FileOutputStream fos = new FileOutputStream(new File("/tmp/zip.gz")); //zip.gz results in a good (not corrupted) file.
     ByteArrayOutputStream out1 = new ByteArrayOutputStream(1024);
     int bytes = (int)myFile.length();
     byte[] buffer = new byte[bytes];
     bytes = inputFile.read(buffer);
     while (bytes != -1) {
          out1.write(buffer,0,bytes);
          bytes = inputFile.read(buffer);
     inputFile.close();
     try {
          int count=0;
          bytes=out1.size();
          while (count<bytes){
               fos.write(buffer,count,1);
out.write(buffer,count,1); // when buffer[count] == 0x00, everything after it gets ignored until a 0x0a byte is written.
               count++;
     } catch (IOException ioe) {
          SysLog.event("IOEXCEPTION: "+ioe);
     } catch (Exception e) {
          SysLog.event("EXCEPTION: "+e);
     fos.flush();
     fos.close();
     out.flush();
out.close();
     return;
}

Actually, I had thought about that and for some time I tried getting rid of some of the header information in the gzipped file, and then I stopped doing that when I realized that part of the gzipped file is a CRC value which I believe is computed taking into account both the data of the file and the header information. If this is the case, then getting rid of part of the header would produce a corrupted file ... but then, I might be wrong in this issue.
Anyway, to make my application more clear, what I am doing is writing the code for a cgi command which is suppossed to access information from a database, create an xml file, gzip it, and then send it to the client who requested the information.
Therefore, what I am sending is an xml file (Content-Type: text/xml) in compressed format (Content-Encoding: gzip). Note that if I don't gzip the file and then send it uncompressed to the client, I have no problems. However, for me it is very important to gzip it because the size of the file can get very large and that would just take bandwidth unnecessarily.
On the other hand, if I try to gunzip the file locally, I have no problem either (the file is not corrupt at the server's end). Therefore, my problem is when I stream it out.
Any further help would be really appreciated!

Similar Messages

  • Error when reading back a binary file

    Hello,
    I am writing out a binary file consisting of U32 arrays, packed strings, and clusters. I get error 116 when reading back. As long as I am synchronized on the correct data position why should LV care what I write to the file? In the example I write a simple 16 element array and a time stamp cluster. When reading back the array I get the error. What is the correct way to do mixed structures? Thanks.
    Attachments:
    file write.PNG ‏44 KB

    David,
    You can solve the problem in various ways. The easiest might be to modify the writer VI by wiring a True constant into the "header (F)" input of your first Write File node. That forces LabVIEW to include a four-byte header that specifies the length of the array you are writing.
    When you wire up the reader code the way you did, the length of the array you wire to the "byte stream type " input is irrelevant; by wiring nothing into the "count" input, you are implicitly telling LabVIEW to look for a header to determine the length of the array you are asking it to read. Because you didn't write any header, LabVIEW is interpreting the first four bytes of the array itself as a length. That length doesn't match the actual amount of data present in the datafile, so LabVIEW generates your error.
    An alternative solution would be to leave the writer unchanged and modify the reader VI: wire an I32 constant into "byte stream type" and wire a 16 into "count". The choice depends on what you prefer and whether or not these binary files need to be compatible with some other program or not.
    Another aside: you can dispense with your case that checks whether the datafile already exists. Just change the function input of the Open/Create/Replace to "create or replace" and wire a False constant into its "advisory dialog" input.
    Hope that's clear,
    John

  • Problem when streaming with the CLI flash media encoder 2.5

    I have a problem when streaming with Flash Media Encoder 2.5. When I run the GUI, everything works, but when I use the commandline with the same profile I have created using the GUI, it says it can't connect to the server.
    I have to use this version of the encoder, because the other is not working with the server, and I have to create a program to remotely control the flash media encoder, so I have to use the CLI. What can i do to go over this problem?
    Any help would be really appreciated. Thankyou.

    First of all, thankyou.
    There is no authentication needed. I can't use Flash Live Media Encoder 3 because it is not supported by the people which manages the server i have to connect to.
    In fact, I don't know why they dont support it, but that is the way it is.
    Through the GUI, i connect directly and without problems, without user/pass.
    Thankyou again.

  • Severe buffering problems when streaming

    In the past couple of days, I have begun experiencing severe buffering problems when streaming video on the web. The video plays for like 6 seconds and then it pauses for like 6 seconds and continues this pattern over and over. This problem occurs regardless of which web browser I am using. I know the problem is not caused by the internet connection because I did a speed test and it is currently operating at 12mbps.  I have tried streaming video on another computer and it worked fine.

    Welcome to the Apple Community.
    Intermittent problems are often a result of interference. Interference can be caused by other networks in the neighbourhood or from household electrical items.
    You can download and install iStumbler (NetStumbler for windows users) to help you see which channels are used by neighbouring networks so that you can avoid them, but iStumbler will not see household items.
    Refer to your router manual for instructions on changing your wifi channel or adjusting your multicast rate.
    There are other types of problems that can affect networks, but this is by far the most common, hence worth mentioning first. Networks that have inherent issues can be seen to work differently with different versions of the same software. You might also try moving the Apple TV away from other electrical equipment.

  • My apple Tv has audio problems when streaming from my NAS server.

    My Apple TV has audio problems when streaming from my NAS server or computer and even internet. I can hear the click sound it makes when you click on the icons, but when the movie starts there is no audio. This even happens when using XBMC. I have HMDI directly to my TV suing Apple TV 2. I have tried resetting, turning off both TV and Apple TV, and changing the HDMI cable. It is driving me nuts.

    XMBC requires a hack, we cannot help with hacked devices.

  • Why does Acrobat Pdf converter file slow down my 2003, Windows  Word Program.  I only experience this problem when i convert a pdf file to a doc file.

    Why does Acrobat Pdf converter file slow down my 2003, Windows  Word Program.  I only experience this problem when i convert a pdf file to a doc file.

    Hi Bill -- thanks for your reply!
    When I check the Document Properties on Acrobat I can see that the fonts used in the document (Cambria, Times and Windings) are listed as "Embedded Subset" in the Fonts panel. The machine it was created on did use an earlier version of OS X and an old version of Word, but it seems to have the proper fonts...
    -nick

  • Streaming out a zipped file

    I am interested in unzipping a binary file (pdf) and stream it out to a user via a servlet. If possible, I do not want to make a hard copy of the file on the server when it is unzipped. My Servlet code would start like this:
    PrintWriter out = res.getWriter();
    res.setContentType("application/pdf");
    //unzip a file and stream it out.
    How could I approach this / Is this possible?
    Thanks very much in advance.

    I think you should use package java.util.zip.
    There are classes ZipInputStream and ZipEntry.
    Try to play with them.
    E.g.:
    OutputStream out = ... // writes bytes into HTTP-Body
    ZipInputStream in = new ZipInputStream(
      new FileInputStream("file.pdf.zip")
    while(in.available() > 0) {
      out.write(in.read());
    }

  • When it comes to binary files ...

    I can't say I have a good grasp of the real differences between files written in text mode and files written in binary mode.
    For (C) example:
    struct data
    int a, b;
    } d;
    d.a = 432;
    d.b = 581924;
    char s[14] = "hello world!\n";
    FILE *fd = fopen("test.txt", "wt");
    fwrite(&d, sizeof(data), 1, fd);
    fwrite(s, sizeof(char), 14, fd);
    fclose(fd);
    fd = fopen ("test.bin", "wb");
    fwrite(&d, sizeof(data), 1, fd);
    fwrite(s, sizeof(char), 14, fd);
    fclose(fd);
    test.txt hexdump:
    B0 01 00 00 24 E1 08 00 68 65 6C 6C 6F 20 77 6F 72 6C 64 21 0D 0A 00
    test.bin hexdump:
    B0 01 00 00 24 E1 08 00 68 65 6C 6C 6F 20 77 6F 72 6C 64 21 0A 00
    What I know (and see):
    strings containing '\n', when are written in text mode to a file, have the '\n' replaced by CR+LF / LF / CR depending on the platform; in binary mode it is left unchanged
    binary files are used to represent the image of the memory used by a program; so if a chain of structures are wrritten in a known order in a binary file, that chain can be restored on the next startup of the program, if the reading takes place in the order specified so no parsing and type casting is needed
    Am I right? Other than these things what else should I know?
    The reason I want to clear these things out is because I have a file which contains the data collected and interpreted by a counter device from some sensors.
    I believe this file is the binary footprint of the memory used to hold the recordings because I can't decipher the hexdump.
    Now, I have a program to read this type of file and see the recordings and export them to a file in a human readable format (and no, it doesn't have a command line argument for this so that I can make a script). The thing is, I want to put all the recordings in a database and manipulate them further for all kinds of statistics and I don't want to waste time exporting whenever there is a data to be inserted.
    I have no clue if what I'm saying has a bit of truth so I would be grateful if someone can correct or enlighten me on how I should approach this  ... is there a way to "parse" the binary file directly not knowing the types of the placeholders for the recordings?
    Last edited by knob (2011-03-03 22:41:04)

    You're right about what happens when a "text" file is written to disk. However, files ("text" or "binary") can contain anything, not necessarily a "memory footprint". (Edit: well, technically, anything you write to disk comes from memory... but it doesn't have to come from a struct or an array. It doesn't matter anyway...)
    That said, as far as I understand it, your problem is that your "counter device" uses an undocumented file format and closed-source software. First step would be to ask the developer for documentation and source code.
    If the developer doesn't want to give the required information or update the software as needed, you'll have to reverse-engineer the file format, either as previously suggested by "guessing" the contents, or by decompiling the executables that access the files.
    Last edited by stqn (2011-03-04 17:36:59)

  • DMA 2100 - Audio Quality problems when playing mp3 or wma files

    I recently purchased a DMA 2100 - two weeks ago.
    Plays my streamed DVDs (VOB files) along with streamed video off the internet wonderfully.  I'm impressed that it actually seems to upscale my DVDs and the Audio quality is perfect.
    But when I try to play a simple mp3 or wma file  I experience pops, clicks and short dropouts during playback.
    Obviously my network is more then sufficient to push DVDs across reliably it should be able to push mp3s/wma files.  Network performance monitor shows me in the low end of HDTV capable and stable.  MP3s and WMAs are stored on the same disk as my DVD VOB files hence I don't expect there is a problem with my media center PC serving them. (PIII 3.0GHz, 2.5GBs, 2TB RAID 5 SATA Array)/  I was experiencing some problems with playback on my PC but that was resolved by upgrading the drivers for my audio card SB Audigly 2
    Any ideas?

    Looking further into this I also notice that my Media Center PCs processor is running all out 70%-100% processor utilization when streaming the MP3s or WMA files.
    Meanwhile when streaming DVDs the processor is at a rather lazy 10% or so.
    Anyone, any ideas why when Streaming DVDs my audio would be perfect and processor idle.... but when streaming WMAs my processor is pegged and the audio pops and crackels as it plays?

  • Problem saving 2D array to binary file

    I am getting some strange results when trying to write a transposed 2D array of Unisgned 8-bit data to a binary file. Attached is an example program ("071026_ArraySave_Bug.vi") which demonstrates the suspicious behavior.
    The program generates a small 2D array of U8 data, then saves it to two temporary files using two different methods.
    1) Row-by-row: The 2D array is auto-indexed, and the indexed row is written to file at once (with the prepended size option disabled).
    2) Point-by-point: Two For loops are used to auto-index each element and write it to file individually.
    The saved files are then reopened and the data is read into two 1D arrays for display.
    The bug occurs when the generated 2D array is transposed before being saved to file. In the point-by-point method, the data is always saved to the file as desired, i.e. by row or by column depending on whether the data was transposed. However, in the row-by-row method the data is incorrectly saved when the input data is transposed. Without transposing the array, the row-by-row results are correct.
    Saving the array row-by-row is significantly faster for larger arrays, so this method is desired.
    Note that the option to transpose the input data in the attached VI is performed using a case structure with a boolean constant ("Bug Control"). This is important because:
    1) The bug does not occur if the boolean constant is replaced with a control.
    2) The bug does not occur if the case structure is replaced with a "Select" node from the Comparison Palette.
    3) The bug still occurs if the case structure is removed such that the array is always transposed (i.e. keep the True case).
    For now, I have been able to circumvent the bug in practice by transposing the array and then passing it through a Select with a constant True (the False case uses the untransposed array). This allows the faster row-by-row method to be used without error, but requires new memory allocation for the transposed array.
    I will appreciate any help that can be given regarding this problem, or simply confirmation that it is indeed a bug in Labview 8.5. It is worth noting that the same program did not produce any errors when run in Labview 7.
    Thank you for your help,
    David Viggiano
    Attachments:
    071026_ArraySave_Bug.vi ‏29 KB

    This is a known bug (CAR : 4DP855N3) , related to memory management. see this thread
    If I remember well, Altenbach shown somewhere that forcing the transpose operation to produce a copy of the memory was a proper workaround.
    Message Edité par chilly charly le 10-26-2007 08:40 PM
    Chilly Charly    (aka CC)
             E-List Master - Kudos glutton - Press the yellow button on the left...        
    Attachments:
    071026_ArraySave_Bug[1].png ‏2 KB

  • Coded UI - Test Results/Out - The binary files are getting copied to the TestResults/Out folder

    Hi,
    I am using Visual Studio 2013 for developing the Coded UI Automation Scripts.
    Whenever I execute a Coded UI Automation Script, the binary files are getting copied to the TestResults/Out folder.
    I do not want these binary files to be copied to OUT folder, and I have tried disabling the Deployment option under Test Settings > Deployment, but still the behavior is the same.
    Could you please advice, how to disable copying binary files into the Out directory.
    Thanks in Advance.
    Regards,
    Karthick K

    Hi Karthick K,
    From your description, as far as I know that when we enable Tracing and HtmlLogger for Coded UI Test and then run the coded UI test, we will get the Test Results/Out folder automatically.
    So it is default that the Out folder includes some files like codeduitestproject6.dll, CodedUITestProject6.pdb etc, we could not disable copying binary files into the Out directory.
    However, there have a replace workaround is I suggest you can disable Tracing and HtmlLogger for Coded UI Test. After you disable the Tracing and HtmlLogger for Coded UI Test, it will not generate the Out folder.
    Similarly, it will not get the binary files.
    http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/08/enabling-htmllogger-in-coded-ui-test.aspx
    If you still want to this feature, I suggest you could submit this feature request:
    http://visualstudio.uservoice.com/forums/121579-visual-studio. The Visual Studio product team is listening to user voice there. You can send your idea there and people can vote. If
    you submit this suggestion, I hope you could post that link here, I will help you vote it.
    Thanks for your understanding.
    Best Regards,
    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click
    HERE to participate the survey.

  • Filename problems when checking out documents

    Hi,Client machines are using Windows NT4 SP6 and Internet Explorer 6.0.2800.1106CO. Environment is Plumtree 4.5 SP1, Collab Server 2.02.We have an issue whereby when users check out documents with periods in the filename, the default filename for the downloaded file is different from the original.Eg. If a document named test 0.1.docis uploaded, when this file is checked out, Internet Explorer will try and save it at test (0)0.1.doc. This means that users have to either manually change the filename in the Save As dialog box, or rename the file to it's correct name before checking it back in.Is this an issue Plumtree is aware of, and is there a workaround or fix?Thanks.Mun

    Further investigation into this problem shows it does not occur when downloading using a straight A link. Eg. Internet Explorer does not change the filename if excutable 0.1.exe is used; only when downloading through Plumtree (checking out files).This is Plumtree Collaboration Server core functionality.Can anyone from Plumtree shed any light on this?Cheers,Mun

  • Problem when showing MS power point files on browser.

    Hi
         I wrote the code in JSP to show MS power point files on browser. And
    I logged session for security as follow.
    <%@ page language = "java" import="javax.servlet.*, java.util.*, java.io.*" %>
    <%
         String filename = request.getParameter("fileName");
         String path = request.getParameter("path");
         session = request.getSession(true);
         if(session.isNew()) {
              response.sendRedirect("login.jsp");
         }else {
              response.setContentType ("application/vnd.ms-powerpoint");
              int iRead;
              FileInputStream stream = null;
              try {
                   File f = new File(path+filename);
                   stream = new FileInputStream(f);
                   while ((iRead = stream.read()) != -1){
                        out.write(iRead);
                   out.flush();
              }finally {
                   if (stream != null) { stream.close(); }     
    %>
         I got problem with showing in Windows XP with MS Office XP only.
    Could anyone please give me suggestion.
    Thank you.
    Atthapon

    Hi!
    I don't have Office XP. But why you don't try to use application/octet-stream instead of vnd.ms-powerpoint? It should work with it.

  • Deployment problems when using several ejb-jar files

    Hello All,
    I am having problems with deployment of my J2EE application. My EAR file has several EJB-JARs, one for each entity bean. However I refer to the other EJBs in one EJB. Hence I included the <ejb-ref> element in the deployment descriptor of the entity bean. But I am getting the deployment error that "the bundle does not contain the bean name". I tried to change the <ejb-name> value (within the <ejb-ref> element) to the JNDI name of the EJB I am referring to, and also added the <ejb-link> tag specifying the location of the JAR file that has the EJB. None of the attempts worked.
    Can someone help me out on how I am supposed to go about when I have several ejb-jar files as opposed to a single one? I am deploying my app on Sun ONE AS 7.0.
    Thank you,
    PS: I am posting this message on several forum topics. Please disregard this message if you have seen/answered this message. Thanks.

    Hi ,
    It would help if you could post the relevant snippets of your
    ejb-jar.xml. In general, you definitely shouldn't use jndi-name
    for any of the linking information. jndi-name is not part of the
    standard deployment descriptor. The syntax for the ejb-link is
    [ejb-jar uri]#ejb-name
    K

  • I have a problem when cropping 8 but tiff files in elements 12. Once area is selected and I crop the image it disappears. The file no. sill shows but the image is not viewable. I have saved the file from that point and tries reopening but the image isn't

    Has anyone experienced this before ? I have previously been able to crop images with no problem. Recently when I open an image and crop it and confirm the action the image 'disappears' If I save and go to reopen the file is there but it will be  zero meg in size.

    Hi,
    Try resetting the crop tool.
    To the right of the tool option bar, click on the little horizontal lines and you should see reset tool option.
    Good luck
    Brian

Maybe you are looking for