Strange Binary File Write Behaviour

Something a bit strange going on when I attempt to write data to a binary file. I've replicated the issue in StrangeBinaryFileBehaviour.vi which is attached below...
I'm merely writing two bytes to a binary file yet my hex editor/viewer is telling me I have 6 bytes of data, a screenie of the hex editor dump of the binary file is also attached
Maybe I'm losing the plot and am missing something, forgive it's late on a friday
Strokes
Solved!
Go to Solution.
Attachments:
StrangeBinaryFileBehaviour.vi ‏23 KB
Hex Editor.PNG ‏7 KB

altenbach wrote:
Wire a FALSE to the "prepend array or string size ...?" terminal. The default is TRUE so you get additional bytes, depending on what you wire.
For a 1D array, you would get 4 additional bytes for the array size (I32) and tha's what you are seeing.
I can't tell you how many times that has bit me in the arse. I understand why the default it TRUE, but I have yet to run into a case where I actually wanted it to be TRUE.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines

Similar Messages

The first binary file write operation for a new file takes progressively longer.

I have an application in which I am acquiring analog data from multiple
PXI-6031E DAQ boards and then writing that data to FireWire hard disks
over an extended time period (14 days). I am using a PXI-8145RT
controller, a PXI-8252 FireWire interface board and compatible FireWire
hard drive enclosures. When I start acquiring data to an empty
hard disk, creating files on the fly as well as the actual file I/O
operations are both very quick. As the number of files on the
hard drive increases, it begins to take considerably longer to complete
the first write to a new binary file. After the first write,
subsequent writes of the same data size to that same file are very
fast. It is only the first write operation to a new file that
takes progressively longer. To clarify, it currently takes 1 to 2
milliseconds to complete the first binary write of a new file when the
hard drive is almost empty. After writing 32, 150 MByte files,
the first binary write to file 33 takes about 5 seconds! This
behavior is repeatable and continues to get worse as the number of
files increases. I am using the FAT32 file system, required for
the Real-Time controller, and 80GB laptop hard drives. The
system works flawlessly until asked to create a new file and write the
first set of binary data to that file. I am forced to buffer lots
of data from the DAQ boards while the system hangs at this point.
The requirements for this data acquisition system do not allow for a
single data file so I can not simply write to one large file.
Any help or suggestions as to why I am seeing this behavior would be
greatly appreciated.

I am experiencing the same problem. Our program periodically monitors data and eventually save it for post-processing. While it's searching for suitable data, it creates one file for every channel (32 in total) and starts streaming data to these files. If it finds data is not suitable, it deletes the files and creates new ones.
On our lab, we tested the program on windows and then on RT and we did not find any problems.
Unfortunately when it was time to install the PXI on field (an electromechanic shovel on a copper mine) and test it, we've come to find that saving was taking to long and the program screwed up. Specifically when creating files (I.E. "New File" function). It could take 5 or more seconds to create a single file.
As you can see, field startup failed and we will have to modify our programs to workaround this problem and return next week to try again, with the additional time and cost involved. Not to talk about the bad image we are giving to our costumer.
I really like labview, but I am particularly upset beacuse of this problem. LV RT is supposed to run as if it was LV win32, with the obvious and expected differences, but a developer can not expect things like this to happen. I remember a few months ago I had another problem: on RT Time/Date function gives a wrong value as your program runs, when using timed loops. Can you expect something like that when evaluating your development platform? Fortunately, we found the problem before giving the system to our costumer and there was a relatively easy workaround. Unfortunately, now we had to hit the wall to find the problem.
On this particular problem I also found that it gets worse when there are more files on the directory. Create a new dir every N hours? I really think that's not a solution. I would not expect this answer from NI.
I would really appreciate someone from NI to give us a technical explanation about why this problem happens and not just "trial and error" "solutions".
By the way, we are using a PXI RT controller with the solid-state drive option.
Thank you.
Daniel R.
Message Edited by Daniel_Chile on 06-29-2006 03:05 PM

Binary file write cancatenation during writing

Hi,
I'm trying to write a 2D array into a binary file and save the whole complete array in 2D. As shown in the picture, data is written and appended to the binary file during each loop. However, each loop would generate a 5x800 2D array (5 rows, 800 elements in each row). Right now I have to use another vi to process the binary file I saved to concatenate all the arrays in the binary file to finally obtaion a 5 x 800n (n is the number of loops) array. Is there a way to save the data into binary file in that format during each loop instead of writing into different segments and I have to concatenate different segments together after that?
On the other hand, if there's a way to do that, would that increase the loop cycle time because it's more computationally expensive? Thank you very much for your help.
Thank you,
Charles
Attachments:
picture2.png ‏37 KB

See the attached examples.
If you have the file open you can basically just write data to it whenever you have it available, just do not include the length. It can cost a bit to write single elements so the row option is more commonly used, but it depends on how large each row is (if they are huge then it will cost to much to keep them in memory and you write parts of each row instead...).
I have not covered cases where you need to close the file and add data later, but the difference then is just that you need to set the write position after you have re-opened the file.
MTO
Attachments:
Write and read 2D array.vi ‏26 KB

Write output to binary file in C/C++?

How can I write output to a file in binary format instead of ascii?
Or... if I am using a buffer.. to write a buffer to binary format instead of ascii?
I am writing a lot of data to a text file and I would like to reduce the size by writing it all into a binary file instead of ascii?
Last edited by kdar (2012-12-13 14:36:41)

Basic I/O?
char buffer[100];
// C++ style
ofstream file;
file.open ("data.bin", ios::out | ios::binary);
file.write(buffer, sizeof buffer);
/* C style */
FILE* file2 = fopen("data2.bin", "wb");
fwrite(buffer, sizeof buffer, 1, file2);
A lot of people prefer the old C style functions for binary i/o even in C++ (and iostream for formatted i/o). Writing and readong more complex structures requires some castin, in C++ usually done with reinterpret_cast<char*>(someStructure).

Write structure to binary file and append

I have a very simple problem that I cannot seem to figure out, which I guess doesn't make it so simple, for me at least. I've made a little example .vi which basically breaks down what I want to to. I have a structure, in this instance 34 bytes of various data types. I would like to iteratively write this structure to a binary file as the data in the ACTUAL structure in my .vi will be changing through time. I'm having trouble getting the data to be appended to the binary file rather than overwriting it since the file size is always 34 bytes regardless of how many times I run the .vi or run the for loop.
I'm not an expert, but it seems if I write a 34 byte structure to a file 10 times, the final product should be a binary file of 340 bytes (assuming I haven't padded or prepended with size).
One really strange thing is that I keep getting error #1544 when I wire the refnum out wire to the file dialog input on the write function, but it works fine when I wire the file path directly to the write function.
Can someone please swoop in and save me from this remedial task?
Thanks for all the help. NI Forum rules!
Solved!
Go to Solution.
Attachments:
struct_to_bin.vi ‏13 KB

Did you consider reading the text of the error message? Don't set the "disable buffering" input to true - just leave that input unwired. Why do you want to disable buffering?
As a general practice, the file refnum should be stored in a shift register around the for loop instead of using loop tunnels, that way if you have zero iterations you will still get the file refnum passed through correctly. Also, there is no need for Set File Position inside the loop, since the file location is always the end of the last write unless it's specifically moved to another location. You might want it once outside the loop after you open the file, if you're appending to an existing file.

How to read/write a binary file from/to a table with BLOB column

I have create a table with a column of data type BLOB.
I can read/write an IMAGE file from/to the column of the table using:
READ_IMAGE_FILE
WRITE_IMAGE_FILE
How can I do the same for other binary files, e.g. aaaa.zip?

There is a package procedure dbms_lob.readblobfromfile to read BLOB's from file.
http://download-east.oracle.com/docs/cd/B19306_01/appdev.102/b14258/d_lob.htm#sthref3583
To write a BLOB to file you can use a Java procedure (pre Oracle 9i R2) or utl_file.put_raw (there is no dbms_lob.writelobtofile).
http://asktom.oracle.com/pls/ask/f?p=4950:8:1559124855641433424::NO::F4950_P8_DISPLAYID,F4950_P8_CRITERIA:6379798216275

Error when Write/read Cluster in binary file.

hi all,
I implemented a code (using the Labview example) to save and read cluster data (see attached file). The write part works properly but when I want to read the file, there is an error : "Error 4 occurred at Read from Binary File in main_save_data_V2.vi; LabVIEW: End of file encountered'.
BUT the data are correctly open. Why is there such message? any suggestions?
thank you.
Cedric
Attachments:
main_save_data_V2.vi ‏19 KB

Instead of predicting how many clusters to read, just set the count in the binary read to -1. That will read all that it can and I got no error. Here is all you need in order to read the file.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
Attachments:
Read clusters.png ‏12 KB

Bug in LV8 : 'Save for previous version' and 'Write to Binary File' VI

Hello
I am using LabVIEW 8's revamped 'Write to Binary File' VI with a 'TRUE' boolean constant wired to the optional 'prepend array or string size?' to write non-trivial structures to a binary file. I then read the file with the 'Read from Binary File' VI and everything is fine. I don't wire anything to the 'file (use dialog)' input (don't know if this can help).
However, after saving my VI for LabVIEW 7.1, I cannot read the binary files created with the LV7 version of the VI anymore. After examining the LV7 converted version of the VI, there is a 'FALSE' boolean constant that is wired to the equivalent of the 'prepend array or string size' input, which breaks the binary format that is expected.
The attached files are LV8 and 'saved for LV7' versions of a dummy VI that writes an array of 5 integers into a binary file. To test the bug, start LV8, open the LV8 version, run it and create a 'test-lv8.bin' file, then open the LV7 version, run it and create a 'test-lv7.bin' file. Check the content of the two files : the size of the array is indeed missing from the 'test-lv7.bin' file, which can be assimilated as a bug in my opinion.
I think I found another one too : if in LV8 I wire the 'cancelled' boolean output of the 'Open/Create/Replace file' to the selector of a case structure, the 'converted to LV7' version VI will have an error, saying the Case Structure selector is not wired.
Could someone please confirm these are indeed bugs ?
Thanks in advance and have a nice day.
Eric Batut
Attachments:
Test Binary File v7-v8 LV7.vi ‏15 KB
Test Binary File v7-v8 LV8.vi ‏7 KB

I'm using LV8.6 and need to read a .bin file created in MATLAB. This file obviously does not contain the 4 byte header that LabVIEW prepends .bin files with. So when I use Read from Binary File, I get no data (I'm trying to read an array of doubles). I've tried making my .bin file both in native and big-endian format and changing the representation (double, int64, float64) but none of this works. I noted that if I create the same array in a .bin file in LabVIEW and wire a FALSE to the "prepend array or string size?", my VI for reading .bin files can't read this file either.
Any work-arounds here?
(I'll try attaching my write & read VI's)
Attachments:
ReadWriteBinFile.zip ‏19 KB

Write to binary file different from 8.0 to 8.2?

I am communicating with a device via USB and am using a method setup by the USB driver programmer. To communicate, I write to a binary file with a filename of \\?\COMx. This has been working fine for about a year in mfg with Labview 8.0. I recently "upgraded" to 8.2 and now it does not work. If I save for a previous version and run 8.0 it works fine. How can I make this work with 8.2?

Yi Y,
The USB device is one of the instruments we manufacture. The USB driver is written so writing to the USB port is just like writing to a file. I use the binary read/write VI's to send and receive commands to this USB device. The filename I use is \\?\COM#. I am told by the USB driver creator that this is common for low level C. I have been using this VI for that last year and recently upgraged from 8.0 to 8.2 and it does not work. I run the same vi under 8.0 and it works fine. This is not a common way to communicate, so will probably not get an answer.
My next question is, is there any way to run an 8.0 dll or vi from an 8.2 executable?
Thanks
Hi dgtest1,
Which USB device you are using, and which driver are you using? I'm trying to figure out specifically how you are communicating with the USB device by writing a binary file and how was the binary file written. Could you please post more information to describe in more details about what your set up is doing?
Thanks!
Yi Y.
Applications Engineer
National Instruments
http://www.ni.com/support

Write to binary file

I have a relatively large 3D array (Phase Settling), which I want to store in a Binary File. The array is a 15000 point sweep (after decimation), repeated up to 48 by 48 times (in two FOR loops, one inside the other), meaning it can end up having as many as 34,560,000 data points. At the moment, I am attempting to dump that 3D array into a Binary File in one go, using the following VI. There are two possile versions of this array (both the same size as each other) and, depending on which has just been tested, they will be written to their own individual file. Once both tests have been run and their results stored, the idea was to open both files, subtract one array from the other, and store the new 3D array to another Binary File whilst also displaying this latest data elsewhere in a higher-level VI.
After its first trial run, LabVIEW presents me with an error message saying something along the lines of "LabVIEW had run out of memory". I assume it would be to do with this VI, since that's where the first error code appeared (Error Code: 2), apparently originating in a 'Write to Binary File' function.
Any ideas as to how I could get round this first hurdle, or perhaps an alternative method I could try?
Thank you.
- James
Message Edited by James Mamakos on 04-08-2010 10:07 AM
Never say "Oops." Always say "Ah, interesting!"
Attachments:
Running out of LabVIEW memory when writing to binary file.PNG ‏30 KB

Well, as shown in the attached screen-shot, the error indicator shows an error code of 2, and that the error occurred in one of the Write to Binary File functions.
However, when trying to save this particular VI after a couple of tweaks, I was presented with the following pop-up alert message:
LabVIEW: Memory is full.
Cannot save VI "Compare Amp - calculate.vi".
LabVIEW Save error code 10: Default data space.
Anyway, returning back to the main problem at hand, I've though of a couple of possible solutions...
We don't really need a particularly high resolution to the data points for the comparison, so I could convert the array doubles into an array of singles to save space.
I should be able to implement the 'Write to Binary File' for the initial arrays into the outer FOR loop, thereby reducing the amount data written in each instance by a factor of up to 48. The only problem I can foresee in this is that I'm not sure how to append 2D arrays into a binary file such that, when the file is read, they would appear as (or be easily converted into) a 3D array.
As for the 'True' case in my case structure, am I being grossly inefficient in my passing of data from one place to the next? I know there are a lot of branches (and buffers?) of 3D arrays; is there a decent way to reduce this overhead by tweaking the data paths or something?
- James
P.S. The computer I'm using has 1GB RAM.
Message Edited by James Mamakos on 04-08-2010 12:39 PM
Never say "Oops." Always say "Ah, interesting!"
Attachments:
LabVIEW - error code 2.PNG ‏35 KB

Write to Binary File strips carriage returns (LV 8.2.1)

   I was using Write to Binary File to write a combination of strings, numbers, and time stamps. When I later went to read the filel, it didn't quite make sense. Although I had written strings that had lines ending in <CR><LF>, when I read them back in, they only had <LF>. I checked this using other programs to read the file. The <CR>'s have been stripped out! Why would a "binary" file VI kindly edit your strings for you? I needed the strings to be written exactly as they were. The VI corrupted my data.
   I couldn't find any explanation for this in the help, nor any way to turn this behavior off. Is this a known issue, a bug, or a feature?
Regards,
   Dave T.
David Thomson Original Code Consulting
www.originalcode.com
National Instruments Alliance Program Member
Certified LabVIEW Architect
There are 10 kinds of people: those who understand binary, and those who don't.

   Thanks, Altenbach and Jarrod, for your help.
   Time to eat crow. At Altenbach's prompting, I created a sample program that would demonstrate the problem. It didn't. Turns out the files were fine. I was reading them with the "Read from Text File" vi, with the EOL option enabled! I had used another program to check the data in the file, and was basing my conclusion on that. Unfortunately, I didn't use a good hex editor, and it turns out the program I used was also stripping the <CR> off. So the data in the file was good, I was reading it wrong, and my double-check was reading it wrong.
Cheers,
    Dave
David Thomson Original Code Consulting
www.originalcode.com
National Instruments Alliance Program Member
Certified LabVIEW Architect
There are 10 kinds of people: those who understand binary, and those who don't.

Binary file is greyed out in Write Measurement File express VI.

I am running labview 8.2 on Fedora 7 Core. I have also installed it on an earlier version of fedora that was in line with the supported red hat distribution. The problem is that when I try to use the Write Measurement File express VI, the binary file option becomes greyed out after the dialog box opens up. It is present for a second but it is impossible to select. When I have tried bringing over a vi written in Windows (where the binary option was available), the option is also disabled. I am trying to avoid writing the file out in lower level code since I want to use the auto file name generation available under the express VI. If any body knows a workaround or has some sample code that writes out a new file for each measurement with the date and time in the filename let me know.
Thanks

Hi Ikiller,
Is this still occurring?
Could you please provide a screen shot to show the behavior?
Thanks!!
Aashish M
Applications Engineer
National Instruments
http://www.ni.com/support/

Write-read binary file Error 116

Hi all,
I am saving double, numeric array data into a binary file and then trying to read it back but keep on getting Error 116 (cannot read from binary file).
I have attached screenshots of the way I am writing my data into the binary file and then the way I am trying to read it. Basically my data is chunks of 2D double arrays, coming at a rate of 1hz and this is why I have to use the get and set file size functions prior to saving into the file (i.e. so that each time I can append my file with new data).
I have tried all combinations for the write to binary and read to binary file functions, thus meaning I have tried little, big and native endian options but I keep getting the same error. Also played with the way I append my data, i.e. I have used both the "offset in bytes" and "end of file" options, just in case this made any difference, but again no luck.
Any help would be much much appreciated.
Regards,
Harry
Solved!
Go to Solution.
Attachments:
Write_Bin.JPG ‏154 KB
Read_Bin.JPG ‏100 KB

Just as an amendment to my original post, I can succesfully read my data in MATLAB using the standard fread function with double precision. I presume that this means that there is nothing wrong with the way I write the data into the file. However, it would be useful for me to be able to read it directly from Labview without having to build matlab executables etc etc...
cheers again

Multiple writes of a DBL 2D array to binary file

Hello.
Does anyone have an example of writing a 2D array to a binary file multiple times?
Specifically, I need to write a 2D array of DBL to a binary file mulitple times and then (later) be able to read it back (reconvert to DBL). All of the examples that I've found only do one write and then close the file. I need to leave the file reference open and write data to it over the period of a few days. The data comes out of a queue that is fed from a DAQ loop. (In an acquistion loop I feed a queue and in a slower loop I dequeue the data and write it to disk)
I believe it has something to do with the pos mode and offset mode but I can't figure it.
I haven't been able to successfully read the data back yet from the .bin file so I'm not sure if I'm even writing it correctly.
The closest example that I've found is the High Speed Data Logger and High Speed Data Reader but I haven't had any luck with those.
Thanks in advance.
Steve
LabVIEW 7.1
Solved!
Go to Solution.

To change the representation of a number, use the Numeric > Conversion sub-palette.
The example below shows that the wires contain the representation info independently of the destination indicators : you can connect different types of numerics to the same wire and LabVIEW will automatically adapt to the type (the coercion dots indicate that a convertion has been done). Everything you should want to know about converting numbers is explained in the LabVIEW Help : "Conversion VIs and Functions"
Message Edited by chilly charly on 07-30-2005 10:57 AM
Chilly Charly (aka CC)
E-List Master - Kudos glutton - Press the yellow button on the left...
Attachments:
Convert to single.png ‏27 KB

Can TestStand open, read, write and close binary files?

From a TestStand sequence, I need to open, read, write and close binary files. Does TestStand support this capability?

Christine -
In the past I have used the C/C++ Adapter to call directly into the CVI RTE functions. The CVI functions I have used are OpenFile, WriteFile and CloseFile. The functions are exported from the DLL as CVI_OpenFile, CVI_WriteFile, and CVI_CloseFile. I just had to make sure that on termination of the sequence, that the integer handle was properly released by calling the close function.
If you are using LabVIEW, you could call directly into the low level VIs to do a similar set of operations.
Scott Richardson (NI)
Scott Richardson
National Instruments

Strange Binary File Write Behaviour

Similar Messages

Maybe you are looking for