Write to binary file

I have a relatively large 3D array (Phase Settling), which I want to store in a Binary File. The array is a 15000 point sweep (after decimation), repeated up to 48 by 48 times (in two FOR loops, one inside the other), meaning it can end up having as many as 34,560,000 data points. At the moment, I am attempting to dump that 3D array into a Binary File in one go, using the following VI. There are two possile versions of this array (both the same size as each other) and, depending on which has just been tested, they will be written to their own individual file. Once both tests have been run and their results stored, the idea was to open both files, subtract one array from the other, and store the new 3D array to another Binary File whilst also displaying this latest data elsewhere in a higher-level VI.
After its first trial run, LabVIEW presents me with an error message saying something along the lines of "LabVIEW had run out of memory". I assume it would be to do with this VI, since that's where the first error code appeared (Error Code: 2), apparently originating in a 'Write to Binary File' function.
Any ideas as to how I could get round this first hurdle, or perhaps an alternative method I could try?
Thank you.
- James
Message Edited by James Mamakos on 04-08-2010 10:07 AM
Never say "Oops." Always say "Ah, interesting!"
Attachments:
Running out of LabVIEW memory when writing to binary file.PNG ‏30 KB

Well, as shown in the attached screen-shot, the error indicator shows an error code of 2, and that the error occurred in one of the Write to Binary File functions.
However, when trying to save this particular VI after a couple of tweaks, I was presented with the following pop-up alert message:
LabVIEW: Memory is full.
Cannot save VI "Compare Amp - calculate.vi".
LabVIEW Save error code 10: Default data space.
Anyway, returning back to the main problem at hand, I've though of a couple of possible solutions...
We don't really need a particularly high resolution to the data points for the comparison, so I could convert the array doubles into an array of singles to save space.
I should be able to implement the 'Write to Binary File' for the initial arrays into the outer FOR loop, thereby reducing the amount data written in each instance by a factor of up to 48. The only problem I can foresee in this is that I'm not sure how to append 2D arrays into a binary file such that, when the file is read, they would appear as (or be easily converted into) a 3D array.
As for the 'True' case in my case structure, am I being grossly inefficient in my passing of data from one place to the next? I know there are a lot of branches (and buffers?) of 3D arrays; is there a decent way to reduce this overhead by tweaking the data paths or something?
- James
P.S. The computer I'm using has 1GB RAM.
Message Edited by James Mamakos on 04-08-2010 12:39 PM
Never say "Oops." Always say "Ah, interesting!"
Attachments:
LabVIEW - error code 2.PNG ‏35 KB

Similar Messages

Bug in LV8 : 'Save for previous version' and 'Write to Binary File' VI

Hello
I am using LabVIEW 8's revamped 'Write to Binary File' VI with a 'TRUE' boolean constant wired to the optional 'prepend array or string size?' to write non-trivial structures to a binary file. I then read the file with the 'Read from Binary File' VI and everything is fine. I don't wire anything to the 'file (use dialog)' input (don't know if this can help).
However, after saving my VI for LabVIEW 7.1, I cannot read the binary files created with the LV7 version of the VI anymore. After examining the LV7 converted version of the VI, there is a 'FALSE' boolean constant that is wired to the equivalent of the 'prepend array or string size' input, which breaks the binary format that is expected.
The attached files are LV8 and 'saved for LV7' versions of a dummy VI that writes an array of 5 integers into a binary file. To test the bug, start LV8, open the LV8 version, run it and create a 'test-lv8.bin' file, then open the LV7 version, run it and create a 'test-lv7.bin' file. Check the content of the two files : the size of the array is indeed missing from the 'test-lv7.bin' file, which can be assimilated as a bug in my opinion.
I think I found another one too : if in LV8 I wire the 'cancelled' boolean output of the 'Open/Create/Replace file' to the selector of a case structure, the 'converted to LV7' version VI will have an error, saying the Case Structure selector is not wired.
Could someone please confirm these are indeed bugs ?
Thanks in advance and have a nice day.
Eric Batut
Attachments:
Test Binary File v7-v8 LV7.vi ‏15 KB
Test Binary File v7-v8 LV8.vi ‏7 KB

I'm using LV8.6 and need to read a .bin file created in MATLAB. This file obviously does not contain the 4 byte header that LabVIEW prepends .bin files with. So when I use Read from Binary File, I get no data (I'm trying to read an array of doubles). I've tried making my .bin file both in native and big-endian format and changing the representation (double, int64, float64) but none of this works. I noted that if I create the same array in a .bin file in LabVIEW and wire a FALSE to the "prepend array or string size?", my VI for reading .bin files can't read this file either.
Any work-arounds here?
(I'll try attaching my write & read VI's)
Attachments:
ReadWriteBinFile.zip ‏19 KB

Write to Binary File strips carriage returns (LV 8.2.1)

   I was using Write to Binary File to write a combination of strings, numbers, and time stamps. When I later went to read the filel, it didn't quite make sense. Although I had written strings that had lines ending in <CR><LF>, when I read them back in, they only had <LF>. I checked this using other programs to read the file. The <CR>'s have been stripped out! Why would a "binary" file VI kindly edit your strings for you? I needed the strings to be written exactly as they were. The VI corrupted my data.
   I couldn't find any explanation for this in the help, nor any way to turn this behavior off. Is this a known issue, a bug, or a feature?
Regards,
   Dave T.
David Thomson Original Code Consulting
www.originalcode.com
National Instruments Alliance Program Member
Certified LabVIEW Architect
There are 10 kinds of people: those who understand binary, and those who don't.

   Thanks, Altenbach and Jarrod, for your help.
   Time to eat crow. At Altenbach's prompting, I created a sample program that would demonstrate the problem. It didn't. Turns out the files were fine. I was reading them with the "Read from Text File" vi, with the EOL option enabled! I had used another program to check the data in the file, and was basing my conclusion on that. Unfortunately, I didn't use a good hex editor, and it turns out the program I used was also stripping the <CR> off. So the data in the file was good, I was reading it wrong, and my double-check was reading it wrong.
Cheers,
    Dave
David Thomson Original Code Consulting
www.originalcode.com
National Instruments Alliance Program Member
Certified LabVIEW Architect
There are 10 kinds of people: those who understand binary, and those who don't.

How to read/write a binary file from/to a table with BLOB column

I have create a table with a column of data type BLOB.
I can read/write an IMAGE file from/to the column of the table using:
READ_IMAGE_FILE
WRITE_IMAGE_FILE
How can I do the same for other binary files, e.g. aaaa.zip?

There is a package procedure dbms_lob.readblobfromfile to read BLOB's from file.
http://download-east.oracle.com/docs/cd/B19306_01/appdev.102/b14258/d_lob.htm#sthref3583
To write a BLOB to file you can use a Java procedure (pre Oracle 9i R2) or utl_file.put_raw (there is no dbms_lob.writelobtofile).
http://asktom.oracle.com/pls/ask/f?p=4950:8:1559124855641433424::NO::F4950_P8_DISPLAYID,F4950_P8_CRITERIA:6379798216275

Write to binary file different from 8.0 to 8.2?

I am communicating with a device via USB and am using a method setup by the USB driver programmer. To communicate, I write to a binary file with a filename of \\?\COMx. This has been working fine for about a year in mfg with Labview 8.0. I recently "upgraded" to 8.2 and now it does not work. If I save for a previous version and run 8.0 it works fine. How can I make this work with 8.2?

Yi Y,
The USB device is one of the instruments we manufacture. The USB driver is written so writing to the USB port is just like writing to a file. I use the binary read/write VI's to send and receive commands to this USB device. The filename I use is \\?\COM#. I am told by the USB driver creator that this is common for low level C. I have been using this VI for that last year and recently upgraged from 8.0 to 8.2 and it does not work. I run the same vi under 8.0 and it works fine. This is not a common way to communicate, so will probably not get an answer.
My next question is, is there any way to run an 8.0 dll or vi from an 8.2 executable?
Thanks
Hi dgtest1,
Which USB device you are using, and which driver are you using? I'm trying to figure out specifically how you are communicating with the USB device by writing a binary file and how was the binary file written. Could you please post more information to describe in more details about what your set up is doing?
Thanks!
Yi Y.
Applications Engineer
National Instruments
http://www.ni.com/support

Write-read binary file Error 116

Hi all,
I am saving double, numeric array data into a binary file and then trying to read it back but keep on getting Error 116 (cannot read from binary file).
I have attached screenshots of the way I am writing my data into the binary file and then the way I am trying to read it. Basically my data is chunks of 2D double arrays, coming at a rate of 1hz and this is why I have to use the get and set file size functions prior to saving into the file (i.e. so that each time I can append my file with new data).
I have tried all combinations for the write to binary and read to binary file functions, thus meaning I have tried little, big and native endian options but I keep getting the same error. Also played with the way I append my data, i.e. I have used both the "offset in bytes" and "end of file" options, just in case this made any difference, but again no luck.
Any help would be much much appreciated.
Regards,
Harry
Solved!
Go to Solution.
Attachments:
Write_Bin.JPG ‏154 KB
Read_Bin.JPG ‏100 KB

Just as an amendment to my original post, I can succesfully read my data in MATLAB using the standard fread function with double precision. I presume that this means that there is nothing wrong with the way I write the data into the file. However, it would be useful for me to be able to read it directly from Labview without having to build matlab executables etc etc...
cheers again

How to write a 2-D Array of Doubles to a binary file in LabView 8.5?

Okay, this is driving me nuts. I got a program that worked fine in LabView 8.0 but refused to write any data after my institute upgraded to LabView 8.5. The data is stored in a 2-D array of doubles and is supposed to be written to a binary file, that has been correctly opened and got a header written to it containing some meta-data of the measurement. But when the doubles from the array should be written to the file, nothing happens. All I get is an (except for the header) empty file of 786 kB. I found out that writing works if I convert the data from the array to singles right before wiring them to the "write to binary file" VI, but for several reasons I need the data as doubles. Can anyone help me? I've tried everything anyone has written here about writing to binary files and more.
Remember, it worked perfectly fine with an older version of LabView. Any ideas?

It is possible that you run into a known memory optimization bug.
Try to place an "always copy" primitive as discussed here.
Message Edited by altenbach on 11-18-2008 09:11 AM
LabVIEW Champion . Do more with less code and in less time .

Report using Binary file

Hi All , can anyone help me here ...
I used “Write to Binary” to write my report , at first stage this file is writing headers , including column headings . at 2nd writing stage it is writing data in columns. This file can be viewed in .doc or .xls format. I have three issues
1.         If I want to print this file from Front Panel , what I should do?
2.         other thing is if I stop logging and then start for another span , it starts writing from the first line instead of from last line , because of this problem it is causing over-writing .
3          another thing , lets say if for 2nd or 3rd logging , I want to display sub-headings (test-1 , test-2) , how to insert , for example :
MAIN HEADING (COMPANY NAME , REPORT TITLE )
DATE , PRODUCT INFO , OPERATOR NAME
COL1              COL2              COL3              COL4
Subheading (test-1)
Value1             Value2             Value3             Valye4
Value1             Value2             Value3             Valye4
Value1             Value2             Value3             Valye4
Subheading (Test-2)
Value1             Value2             Value3             Valye4
Value1             Value2             Value3             Valye4
Value1             Value2             Value3             Valye4
Any help in this regard will be highly appreciated.
Regards
Faiyaz
Attachments:
report-writing-binaryfile.vi ‏68 KB
02-display-subVI.vi ‏12 KB

Hi Faiyaz,
1. How Do I Print a File Programmatically From LabVIEW?
2. Please the Programming >> File I/O >> Advanced File Functions >> Set File Position function on the block diagram after the Open/Create/Replace File function and set the from input to "end." This will append new data to the end of the file.
3. You can simply add that information to the Format into String you are writing to the second Write to Binary file.
Michael K.
| Michael K | Project Manager | LabVIEW R&D | National Instruments |

Read an avi file using "Read from binary file" vi

My question is how to read an avi file using "Read from binary file" vi .
My objective is to create a series of small avi files by using IMAQ AVI write frame with mpeg-4 codec of 2 second long (so 40 frames in each file with 20 fps ) and then send them one by one so as to create a stream of video. The image are grabbed from USB camera. If I read those frames using IMAQ AVI read frame then compression advantage would be lost so I want to read the whole file itself.
I read the avi file using "Read from binary file" with unsigned 8 bit data format and then sent to remote end and save it and then display it, however it didnt work. I later found that if I read an image file using "Read from binary file" with unsigned 8 bit data format and save it in local computer itself , the format would be changed and it would be unrecognizable. Am I doing wrong by reading the file in unsined 8 bit integer format or should I have used any other data types.
I am using Labview 8.5 and Labview vision development module and vision acquisition module 8.5
Your help would be highly appreciated.
Thank you.
Solved!
Go to Solution.
Attachments:
read avi file in other data format.JPG ‏26 KB

Hello,
Check out the (full) help message for "write to binary file"
The "prepend array or string size" input defaults to true, so in your example the data written to the file will have array size information added at the beginning and your output file will be (four bytes) longer than your input file. Wire a False constant to "prepend array or string size" to prevent this happening.
Rod.
Message Edited by Rod on 10-14-2008 02:43 PM

Binary file with File adapter

Hi gurus,
I am getting binary file as base64binary (not a text/xml file) in a request XML tag. I need to write this file to a file directory with the file adapter after retrieving it from the request message by a message/java mapping and use another field in XML as the file name.
I followed the blog `how to send binary data through PI` but in that case, the request message is also the same. Mine requires a mapping to be executed and in this case I cannot use dummy names for message types/interfaces as Enterprise Repository development is required.
How can I achieve this?
Thanks
Gokhan

Just solved the problem actually
As I was trying to write a binary file directly with the receiver file adapter, I wasn't sure how to define a data type / message type for it. I developed a java mapping that decodes the base64 to binary data and writes it to the output stream, and used a dummy message type / data type for the service interface as the target in Operation Mapping
And it worked!
Regards,
Gökhan

Writing binary file for fixed amount of time

Hello, I'm trying to write a binary file of multiple channels for a fixed amount of time before I perform my analysis. Does anyone know the best way to accomplish this?

Here is an example of one way to do what I think you are trying to do. There may be a more elegant way of doing it, but this is pretty simple and straight-forward. It basically writes a 5-integer array for 5 seconds and then stops. Let me know if you have any questions.
Attachments:
timed write.vi ‏27 KB

Error when Write/read Cluster in binary file.

hi all,
I implemented a code (using the Labview example) to save and read cluster data (see attached file). The write part works properly but when I want to read the file, there is an error : "Error 4 occurred at Read from Binary File in main_save_data_V2.vi; LabVIEW: End of file encountered'.
BUT the data are correctly open. Why is there such message? any suggestions?
thank you.
Cedric
Attachments:
main_save_data_V2.vi ‏19 KB

Instead of predicting how many clusters to read, just set the count in the binary read to -1. That will read all that it can and I got no error. Here is all you need in order to read the file.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
Attachments:
Read clusters.png ‏12 KB

The first binary file write operation for a new file takes progressively longer.

I have an application in which I am acquiring analog data from multiple
PXI-6031E DAQ boards and then writing that data to FireWire hard disks
over an extended time period (14 days). I am using a PXI-8145RT
controller, a PXI-8252 FireWire interface board and compatible FireWire
hard drive enclosures. When I start acquiring data to an empty
hard disk, creating files on the fly as well as the actual file I/O
operations are both very quick. As the number of files on the
hard drive increases, it begins to take considerably longer to complete
the first write to a new binary file. After the first write,
subsequent writes of the same data size to that same file are very
fast. It is only the first write operation to a new file that
takes progressively longer. To clarify, it currently takes 1 to 2
milliseconds to complete the first binary write of a new file when the
hard drive is almost empty. After writing 32, 150 MByte files,
the first binary write to file 33 takes about 5 seconds! This
behavior is repeatable and continues to get worse as the number of
files increases. I am using the FAT32 file system, required for
the Real-Time controller, and 80GB laptop hard drives. The
system works flawlessly until asked to create a new file and write the
first set of binary data to that file. I am forced to buffer lots
of data from the DAQ boards while the system hangs at this point.
The requirements for this data acquisition system do not allow for a
single data file so I can not simply write to one large file.
Any help or suggestions as to why I am seeing this behavior would be
greatly appreciated.

I am experiencing the same problem. Our program periodically monitors data and eventually save it for post-processing. While it's searching for suitable data, it creates one file for every channel (32 in total) and starts streaming data to these files. If it finds data is not suitable, it deletes the files and creates new ones.
On our lab, we tested the program on windows and then on RT and we did not find any problems.
Unfortunately when it was time to install the PXI on field (an electromechanic shovel on a copper mine) and test it, we've come to find that saving was taking to long and the program screwed up. Specifically when creating files (I.E. "New File" function). It could take 5 or more seconds to create a single file.
As you can see, field startup failed and we will have to modify our programs to workaround this problem and return next week to try again, with the additional time and cost involved. Not to talk about the bad image we are giving to our costumer.
I really like labview, but I am particularly upset beacuse of this problem. LV RT is supposed to run as if it was LV win32, with the obvious and expected differences, but a developer can not expect things like this to happen. I remember a few months ago I had another problem: on RT Time/Date function gives a wrong value as your program runs, when using timed loops. Can you expect something like that when evaluating your development platform? Fortunately, we found the problem before giving the system to our costumer and there was a relatively easy workaround. Unfortunately, now we had to hit the wall to find the problem.
On this particular problem I also found that it gets worse when there are more files on the directory. Create a new dir every N hours? I really think that's not a solution. I would not expect this answer from NI.
I would really appreciate someone from NI to give us a technical explanation about why this problem happens and not just "trial and error" "solutions".
By the way, we are using a PXI RT controller with the solid-state drive option.
Thank you.
Daniel R.
Message Edited by Daniel_Chile on 06-29-2006 03:05 PM

Blob for binary file, read/write problems

Hi,
I am relatively new to this type of development so apologies if this question is a bit basic.
I am trying to write a binary document (.doc) to a blob and read it back again, constructing the original word file. I have the following code for reading and writing the file:
private void save_addagreement_Click(object sender, EventArgs e)
// Save the agreement to the database
int test_setting = 0;
// create an OracleConnection object to connect to the
// database and open the connection
string constr;
if (test_setting == 0)
constr = "User Id=royalty;Password=royalty;data source=xe";
else
constr = "User ID=lob_user;Password=lob_password;data source=xe";
OracleConnection myOracleConnection = new OracleConnection(constr);
myOracleConnection.Open();
// create an OracleCommand object to hold a SQL statement
OracleCommand myOracleCommand = myOracleConnection.CreateCommand();
myOracleCommand.CommandText = "insert into blob_content(id, blob_column) values 2, empty_blob())";
OracleDataReader myOracleDataReader = myOracleCommand.ExecuteReader();
// step 2: read the row
OracleTransaction myOracleTransaction = myOracleConnection.BeginTransaction();
myOracleCommand.CommandText =
"SELECT id, blob_column FROM blob_content WHERE id = 2";
myOracleDataReader = myOracleCommand.ExecuteReader();
myOracleDataReader.Read();
Console.WriteLine("myOracleDataReadre[\"id\"] = " + myOracleDataReader["id"]);
OracleBlob myOracleBlob = myOracleDataReader.GetOracleBlobForUpdate(1);
Console.WriteLine("OracleBlob = " + myOracleBlob.Length);
myOracleBlob.Erase();
FileStream fs = new FileStream(agreement_filename.Text, FileMode.Open, FileAccess.Read);
Console.WriteLine("Opened " + agreement_filename.Text + " for reading");
int numBytesRead;
byte[] byteArray = new byte[fs.Length];
numBytesRead = fs.Read(byteArray, 0, (Int32)fs.Length);
Console.WriteLine(numBytesRead + " read from file");
myOracleBlob.Write(byteArray, 0, byteArray.Length);
Console.WriteLine(byteArray.Length + " written to blob object");
Console.WriteLine("Blob Length = " + myOracleBlob.Length);
fs.Close();
myOracleDataReader.Close();
myOracleConnection.Close();
This gives the following console output:
myOracleDataReadre["id"] = 2
OracleBlob = 0
Opened D:\sample_files\oly_in.doc for reading
56832 read from file
56832 written to blob object
Blob Length = 56832
My write to file code is:
private void save_agreement_to_disk_Click(object sender, EventArgs e)
string filename;
SaveFileDialog savedoc = new SaveFileDialog();
if (savedoc.ShowDialog() == DialogResult.OK)
filename = savedoc.FileName;
// create an OracleConnection object to connect to the
// database and open the connection
OracleConnection myOracleConnection = new OracleConnection("User ID=royalty;Password=royalty");
myOracleConnection.Open();
// create an OracleCommand object to hold a SQL statement
OracleCommand myOracleCommand = myOracleConnection.CreateCommand();
myOracleCommand.CommandText =
"SELECT id, blob_column " +
"FROM blob_content " +
"WHERE id = 2";
OracleDataReader myOracleDataReader = myOracleCommand.ExecuteReader();
myOracleDataReader.Read();
Console.WriteLine("myOracleDataReader[id] = " + myOracleDataReader["id"]);
//Step 2: Get the LOB locator
OracleBlob myOracleBlob = myOracleDataReader.GetOracleBlobForUpdate(1);
Console.WriteLine("Blob size = " + myOracleBlob.Length);
//Step 3: get the BLOB data using the read() method
byte[] byteArray = new byte[500];
int numBytesRead;
int totalBytes = 0;
FileStream fs = new FileStream(filename, FileMode.OpenOrCreate, FileAccess.Write);
while ((numBytesRead = myOracleBlob.Read(byteArray, 0, 500)) > 0)
totalBytes += numBytesRead;
fs.Write(byteArray, 0, byteArray.Length);
Console.WriteLine("numBytes = " + numBytesRead + " totalBytes = " + totalBytes);
Console.WriteLine((int)fs.Length + " bytes written to file");
fs.Close();
myOracleDataReader.Close();
myOracleConnection.Close();
This gives the following console output:
myOracleDataReader[id] = 2
Blob size = 0
0 bytes written to file
If I manually add the blob file using the following:
DECLARE
my_blob BLOB;
BEGIN
-- load the BLOB
my_bfile := BFILENAME('SAMPLE_FILES_DIR', 'binaryContent.doc');
SELECT blob_column
INTO my_blob
FROM blob_content
WHERE id = 1 FOR UPDATE;
DBMS_LOB.FILEOPEN(my_bfile, dbms_lob.file_readonly);
DBMS_LOB.LOADFROMFILE(my_blob, my_bfile, DBMS_LOB.GETLENGTH(my_bfile), 1, 1);
DBMS_LOB.FILECLOSEALL();
COMMIT;
END;
COMMIT;
The write to file works perfectly. This tells me that there must be something wrong with my code that is writing the blob to the database. I tried where possible to following the Oracle article using large objects in .NET but that (along with most things on the internet) focus on uploading text files.
Thanks in advance.
Chris.

myOracleCommand.CommandText = "insert into blob_content(id, blob_column) values 2, empty_blob())";
OracleDataReader myOracleDataReader = myOracleCommand.ExecuteReader();
This looks wrong, you shouldn't be using ExecuteReader unless you expect to get a result back. Try using ExecuteNonQuery to do the insert.

Write structure to binary file and append

I have a very simple problem that I cannot seem to figure out, which I guess doesn't make it so simple, for me at least. I've made a little example .vi which basically breaks down what I want to to. I have a structure, in this instance 34 bytes of various data types. I would like to iteratively write this structure to a binary file as the data in the ACTUAL structure in my .vi will be changing through time. I'm having trouble getting the data to be appended to the binary file rather than overwriting it since the file size is always 34 bytes regardless of how many times I run the .vi or run the for loop.
I'm not an expert, but it seems if I write a 34 byte structure to a file 10 times, the final product should be a binary file of 340 bytes (assuming I haven't padded or prepended with size).
One really strange thing is that I keep getting error #1544 when I wire the refnum out wire to the file dialog input on the write function, but it works fine when I wire the file path directly to the write function.
Can someone please swoop in and save me from this remedial task?
Thanks for all the help. NI Forum rules!
Solved!
Go to Solution.
Attachments:
struct_to_bin.vi ‏13 KB

Did you consider reading the text of the error message? Don't set the "disable buffering" input to true - just leave that input unwired. Why do you want to disable buffering?
As a general practice, the file refnum should be stored in a shift register around the for loop instead of using loop tunnels, that way if you have zero iterations you will still get the file refnum passed through correctly. Also, there is no need for Set File Position inside the loop, since the file location is always the end of the last write unless it's specifically moved to another location. You might want it once outside the loop after you open the file, if you're appending to an existing file.

Write to binary file

Similar Messages

Maybe you are looking for