Building array and writing to multiple files

Greetings,
I have a VI wherein, I am collecting data from 3 channels (using Global Virtual Channels with scaling). I am collecting 100 samples at a time with sampling frequency of 1000 Hz. I am then taking average and st. dev of 100 data points/channel and write it to a file (‘*.dat’). I would like to limit the file size to 1000 entries, and so after that limit is reached I would like to create a new file and write the subsequent set of data to the next file and so on. I am doing this in a case structure, wherein I build an array of the values until the condition of file limit is reached and then I write that array to a ‘*.dat’ file. (I’m hoping that this logic is correct, if not please suggest the correct method)
I am also taking average and st. dev of data/channel over period of 1 sec and write it to another ‘*.sum’ file. The VI has other features like writing time stamp to file, stopping the VI after preset time limit is reached or certain safety limit is reached.
When I run the VI, I get an error message:
Error 7 occurred at Open/Create/Replace File in Global Channel with Multiple file Write.vi
Possible reason(s):
LabVIEW: File not found. The file might have been moved or deleted, or the file path might be incorrectly formatted for the operating system. For example, use \ as path separators on Windows, : on Mac OS X, and / on Linux. Verify that the path is correct using the command prompt or file explorer.
=========================
NI-488: Nonexistent GPIB interface.
I have indicated the location of the error in the attached picture, as well as I’m attaching the VI.
I would really appreciate if someone would tell me what is causing the error and how to fix it.
Thanks,
Dinesh
Solved!
Go to Solution.
Attachments:
Block Diag.png ‏121 KB
Global Channel with Multiple file Write.vi ‏515 KB

I did try that but it does not solve the problem. I think I'm missing something while building the array in the Case Structure. I would like to write the average & st. dev values/channel in an array while the loop coount < file limit, and once the file limit is reached write that array to a *.dat file. And keep repeating this procedure till the target time is reached.
Another issue that I discovered is that TestFile_1.dat is not generted, after TestFile_0.dat TestFile_2.dat gets generated. I'm not sure why this is happening.
I would really appreciate if you would take another look and suggest some debugging tips.
Thanks.
P.S. I am attaching the updated VI
Attachments:
Global Channel with Multiple file Write.vi ‏515 KB

Similar Messages

Build Array and Output Values to Text or Excel File

I know this is a simple question but I need some help. I'm reading a DC voltage in LabVIEW a while loop. I want to store all the read values into an array and export that array as an text or Excel file. I had a VI that I build before for this but I cannot seem to find it and I can't remember how I did it before. Any help is appreciated. I think I can do the exporting part but I do help with building the array (storing all the data values).

I run into a problem while using the "Write to Text File Function". Initially I took about 60 measurements and wrote to a text file. That works but I increased the amount of measurements to be taken to 600 and when I did that the output in the text file are all Chinese letters (or that's what it seems like). Is this because I'm writing too much data?
When I use the "Write To Spreadsheet File VI" to write the measurments it works fine for the 600 measurements. The problem with this is I cannot insert any text. Using the "Write to Text File Function" I inserted some text before the measurements and "end of lines", to format the data. Attached is a screenshot of my VI.
Attachments:
measurements.PNG ‏45 KB

Need help optimizing the writing of a very large array and streaming it a file

Hi,
I have a very large array that I need to create and later write to a TDMS file. The array has 45 million entries, or 4.5x10^7 data points. These data points are of double format. The array is created by using a square pulse waveform generator and user-defined specifications of the delay, wait time, voltages, etc.
I'm not sure how to optimize the code so it doesn't take forever. It currently takes at least 40 minutes, and I'm still running it, to create and write this array. I know there needs to be a better way, as the array is large and consumes a lot of memory but it's not absurdly large. The computer I'm running this on is running Windows Vista 32-bit, and has 4GB RAM and an Intel Core 2 CPU @ 1.8Mhz.
I've read the "Managing Large Data Sets in LabVIEW" article (http://zone.ni.com/devzone/cda/tut/p/id/3625), but I'm unsure how to apply the principles here. I believe the problem lies in making too many copies of the array, as creating and writing 1x10^6 values takes < 10 seconds, but writing 4x10^6 values, which should theoretically take < 40 seconds, takes minutes.
Is there a way to work with a reference of an array instead of a copy of an array?
Attached is my current VI, Generate_Square_Pulse_With_TDMS_Stream.VI and it's two dependencies, although I doubt they are bottlenecking the program.
Any advice will be very much appreciated.
Thanks
Attachments:
Generate_Square_Pulse_With_TDMS_Stream.vi ‏13 KB
Square_Pulse.vi ‏13 KB
Write_TDMS_File.vi ‏27 KB

Thanks Ravens Fan, using replace array subset and initializing the array beforehand sped up the process immensely. I can now generate an array of 45,000,000 doubles in about one second.
However, when I try to write all of that out to TDMS at the end LV runs out of memory and crashes. Is it possible to write out the data in blocks and make sure memory is freed up before writing out the next block? I can use a simple loop to write out the blocks, but I'm unsure how to verify that memory has been cleared before proceeding. Furthermore, is there a way to ensure that memory and all resources are freed up at the end of the waveform generation VI?
Attached is my new VI, and a refined TDMS write VI (I just disabled the file viewer at the end). Sorry that it's a tad bit messy at the moment, but most of that mess comes from doing some arithmetic to determine which indices to replace array subsets with. I currently have the TDMS write disabled.
Just to clarify the above, I understand how to write out the data in blocks; my question is: how do I ensure that memory is freed up between subsequent writes, and how do I ensure that memory is freed up after execution of the VI?
@Jeff: I'm generating the waveform here, not reading it. I guess I'm not generating a "waveform" but rather a set of doubles. However, converting that into an actual waveform can come later.
Thanks for the replies!
Attachments:
Generate_Square_Pulse_With_TDMS_Stream.vi ‏14 KB
Write_TDMS_File.vi ‏27 KB

Recording data at particular iterations and writing to text file

Hi all,
this is my first time posting on the NI boards. I'm running into a couple problems I can't seem to figure out how to fix.
I'm collecting data using a LabJack U3-HV daq. I've taken one of the out-of-the-box streaming functions that comes with the LabJack and modified it for my purposes. I am attempting to simply save the recorded data to a text file in columns, one for each of my 4 analog strain gauge inputs, and one for time. For some reason when the 'write to measurement file.vi' executes it is puts everything in rows, and the data is unintelligible.
The 2nd issue I am facing, which is not currently visible in my vi, is that I am running my test for 60,000 cycles, which generates a ton of data. I'm measuring creep/fatigue with my strain gages so I don't need data for every cycle, probably for the first 1000, then the 2k, 4k, 6k, 8k, 10k, 20k, etc. More of an exponential curve. I tried using some max/min functions and then matching the 'write to measurement file.vi' with a case structure that only permitted it to write for particular iterations, but can't seem to get it to work.
Thanks in advance for any help!
Attachments:
3.5LCP strain gages v2.vi ‏66 KB

Hey carfreak,
I've attached a screenshot that shows three different ways of trying to keep track of incrementing data and/or time in a while loop. The top loop just shows a program that demonstrates how shift registers can be used to transfer data between loops. This code just writes the iteration value to the array using the Build Array function.
The first loop counts iterations in an extremely round-about way... the second shows that really you can just build the array directly using the iteration count (the blue "i" in the while loop is just an iteration counter).
The final loop shows how you can use a time stamp to actually keep track of the relative time when a loop executes.
Please note that these three should not actually be implemented together in one VI. I just built them in one BD for simplicity's sake for the screenshot. As described above, the producer-consumer architecture should be used when running parallel loops.
Does that answer your question?
Chris G
Applications Engineer
National Instruments
Attachments:
While Loops and Iterations.JPG ‏83 KB

Reading and writing from a file

I am trying to read and write to a file so that I may store an array of string and a double in my application, but I have come upon a couple of problems; first when I put my file path into the path constant it will say that the file is not there ever though it is (so I have to force the issue and when the application asks for a file I have to browse and add the files), secondly when I run the program and it reads from the file it does not display the strings or doubles immediately instead I have to hit the submit button and than they finally come up, thirdly when I reset my values in the .txt files and put in my string and double and hit submit the array jumps one spot and starts at the index of 1 (array[1]) if anyone can help I would appreciate it a lot.
here is my code:
Harold Timmis
[email protected]
Orlando,Fl
*Kudos always welcome
Solved!
Go to Solution.
Attachments:
testingfileIO.vi ‏30 KB
double.txt ‏1 KB
string.txt ‏1 KB

Harold Timmis wrote:
I am trying to read and write to a file so that I may store an array of string and a double in my application, but I have come upon a couple of problems; first when I put my file path into the path constant it will say that the file is not there ever though it is (so I have to force the issue and when the application asks for a file I have to browse and add the files),
I don't see that behavior at all about forcing the issue. Why are you using Not a Path constants? Why not turn those constants into controls you put on your front panel?
secondly when I run the program and it reads from the file it does not display the strings or doubles immediately instead I have to hit the submit button and than they finally come up,
Put your array indicators before your event structure rather than after. Think dataflow. The code pauses at the event structure waiting for an even to fire. Only when it does (such as hitting the submit button) does the data get written to the array indicators.
thirdly when I reset my values in the .txt files and put in my string and double and hit submit the array jumps one spot and starts at the index of 1 (array[1]) if anyone can help I would appreciate it a lot.
I don't see this behavior either. How come your double.txt file has names in it, and your string.txt file has doubles in it?

Preview and Texteditor open multiple files

Preview and Texteditor often open multiple files when I only clicked a single file. It seems to open one or more previously opened files at the same time. Is this a bug, or can I disable this behaviour somewhere?

"Restore windows when quitting and reopening apps" worked for me. However, it will not work for any apps currently open when you change the preference. Once you quit ans recopen the app, it should work. At least mine did!

ARRAY and records at multiple level ? please help

Hi experts!!
I am struck up with a problem. If u can please help me.
I need a structure of this type.
create type GRADE as object(
grade varchar2(30)
/ -- works fine , creates
create type GRADE_ARRAY as VARRAY(10) of GRADE;
/ -- works fine and creates
create type SPECIES as object
Species_number number,
array_of_grade GRADE_ARRAY
/ works fine
create type SPECIES_ARRAY as VARRAY(20) of SPECIES;
-- error comes here.. Can not have multiple level..type error
and so can not go ahead. In fect I wanted to use next level also. like this.
The next command remains my dream only then because I could not create the SPECIES ARRAY it self..
create type TIMBER as object
timber_mark varchar2(6),
no_species number,
array_of_species(20) SPECIES_ARRAY
the problem is for multiple level ARRAY AND RECORD/object combination..
I tried with OBJECT AND VARRAY it does only for one level.. not even two level.
my Mail ID:
[email protected]
Thanks and Regards..
Virendra chauhan

I think multi-level collections was first implemented in 9.2. You failed to mention what version of the DB you are using.

How to write an array and a scalar to file?

I would like to write my data to a file but am having trouble approaching the problem. In the file I would like to write both the x and y values of my data points in two columns, however, the y data is in an array and the x data is made up of 3 pieces of information: an initial x, an x incremented value, and the number of points in my data sample. How could I go about this?
Solved!
Go to Solution.

The number of points is redundant, because it will be the same as the "other" array.
Here's what I usually do:
LabVIEW Champion . Do more with less code and in less time .
Attachments:
ramper.png ‏13 KB

Open and write to multiple files in labview 6i

I want to open multiple files in a single application in labview and write the data I retrive via gpib in the files

To create multiple files in a directory you can use the attached example.
[email protected]
Attachments:
creating_multiple_files.vi ‏13 KB

Reading and Writing large Excel file using JExcel API

hi,
I am using JExcelAPI for reading and writing excel file. My problem is when I read file with 10000 records and 95 columns (file size about 14MB), I got out of memory error and application is crashed. Can anyone tell me is there any way that I can read large file using JExcelAPI throug streams or in any other way. Jakarta POI is also showing this behaviour.
Thanks and advance

Sorry when out of memory error is occurred no stack trace is printed as application is crashed. But I will quote some lines taken from JProfiler where this problem is occurred:
          reader = new FileInputStream(new File(filePath));
          workbook = Workbook.getWorkbook(reader);
          *sheeet = workbook.getSheet(0);* // here out of memory error is occured
           JProfiler tree:
jxl.Workbook.getWorkBook
      jxl.read.biff.File
             jxl.read.biff.CompoundFile.getStream
                   jxl.read.biff.CompoundFile.getBigBlockStream Thanks

Very slow painting while reading and writing doubles into file

for 15MB length file i = 7662080
for 50MB length file i = 12414368
Part of Code for writing into file follows like this:
try{
fos = new FileOutputStream("Angel.txt");
File f = new File("Angel.txt");
     if(f.length() >=4)
     f.delete();
fos = new FileOutputStream("Angel.txt");
dos = new DataOutputStream(new BufferedOutputStream(fos,1000000));
int x=0;
double y_last, y_new;
for(int j=0 ;j<i ;j++)
if(some condition)
y_new = ....;
try{
//previously in vectors
y_last = y_new;
vect.add(new Line2D.Double(x, y_last, x, y_new)_;
dos.writeDouble(y_new);
     }catch(Exception e){System.out.println(e);}
dos.close();
fos.close();
x++;
}catch(Exception excp){System.out.println(excp);}
part of code for reading from file follows like this:
public void paint(Graphics g)
try{
     double y1, y2 =0;
     Line2D.Double doub;
     raf = new RandomAccessFile("Angel.txt","r");
     dis = new DataInputStream(new BufferedInputStream(new FileInputStream(raf.getFD(),1000000)));
     raf.seek((rect.x*8));
     for (int i = 0/any value; (i < value as per choice); i++)
          g2.setStroke(new BasicStroke(0)); //2
          y1 = y2;
     y2 =dis.readDouble();
          doub=new Line2D.Double(i,y1,i,y2);
          g2.draw(doub);
dis.close();
     raf.close();
}catch(Exception excp){System.out.println(excp);}
I tried using Object Streams but NotSerializable Exception is thrown as Line2D.Double objects
are not serialized.
Any idea to make reading and writing into file specially from MB files faster is appreciated.

Why are you reading in the file in the paint method ?
Create your data once before painting.
I think you should explain what is your goal and what behavior you want.
Denis

Help on reading from and writing to a file !!!!!!!!!!!!!!!!

hi there
anyone can help me on how to write and read from a file? how can i read one string at a time instead of a char. thank you.

You can do this with classes FileReader and FileWriter directly, but it's easiest to wrap these with other classes and call methods on those to read and write a string. All of the classes used below are in the java.io package. The code fragments just print any I/O exceptions to stderr. Replace filename with the name of the file you want to use.
To write a string to a file:
String s = "hello";
try {
FileWriter fw = new FileWriter("filename");
PrintWriter pr = new PrintWriter(fw);
pr.println(s); // write a string with a newline at the end
// pr.print(s); // write a string without a newline at the end
pr.close(); // must close for string to be written
} catch (IOException e) {
System.out.println(e);
To read a string from a file:
String s;
try {
BufferedReader br = new BufferedReader(new FileReader("filename"));
s = br.readLine();
} catch (IOException e) {
System.out.println(e);

Help on reading and writing to a file !!!!!!!!!!!!!!!!

hi there
anyone can help me on how to write and read from a file? how can i read one string at a time instead of a char. thank you.

Use the StringTokenizer to break up the line
File newFile = new File("text.txt");
BufferedReader in = new BufferedReader(new FileReader (newFile));
String line;
while((line = in.readLine()) != null){
StringTokenizer st = new StringTokenizer(inputLine, " ");
while(st.hasMoreTokens()){
// do something with the token
// e.g:
// String lc = st.nextToken().toLowerCase()
hope that helps

READING AND WRITING FROM A FILE(WRITTEN IN ANOTHER LANGUAGE)

# Comment
INTEGER Declares integer type variable
STRING Declares string type variable
LET Assigns a variable
CALCULATE Performs an arithmetic calculation
PRINT Writes data to the console
PRINTLN As print, but followed by a new line
END Terminates the program
Each line of the program must begin with one of these words, and the language should not be case sensitive.
Tim Brailsford, 2007
G64ICP
COMMENT
Any line beginning with the # character is completely ignored.
Example:
# ignore this line!
INTEGER
The INTEGER reserved word declares an Integer variable, and assigns a default value of 0 to it.
Example:
INTEGER myInt
This example is equivalent to the Java statment "int myInt=0;"
STRING
The STRING reserved word declares a String variable, and assigns an empty string to it.
Example:
STRING myString
This is equivalent to the Java statement "String myString;"
LET
This assigns values to variables (either integers, or strings enclosed
in double quotes).
Examples:
LET myInt=42
LET myString="Hello World!"
CALCULATE
This performs numeric calculations upon values or variables, and assigns the result to a variable (which must have
been declared earlier). Four operators are supported: + *
and / (for addition, subtraction multiplication and
integer division respectively).
Examples:
CALCULATE myInt=2*2
CALCULATE myInt=myInt+24
CALCULATE myInt=intA/intB
PRINT and PRINTLN
These words both print to the console the
only difference being that PRINTLN appends a new line (as does
System.out.println in Java). These can print either text provided as an argument, or variables (string or integer).
Examples:
PRINTLN "Hello World!"
PRINT myString
PRINTLN myInt
END
This terminates the program, and prints a message to that effect.
TPL Example
To the left is an example TPL input file , to
calculate the factorial of 5. This should produce
an output similar to that shown overleaf (except
that calculates the factorial of 10).
# A TPL Program to calculate the factorial of 5
INTEGER myInt
INTEGER factorial
STRING myString
LET mystring="Factorial Program"
LET myInt=5
CALCULATE factorial=myInt*4
CALCULATE factorial=factorial*3
CALCULATE factorial=factorial*2
Tim Brailsford, 2007
G64ICP
PRINTLN mystring
PRINTLN
PRINT "The factorial of "
PRINT myInt
PRINT " is "
PRINTLN factorial
END
Documentation
Document your code by writing instructions explaining how to run it, listing all of the files you have submitted,
and specifying any limitations (eg you might want to say what platforms you would expect it to run on, and what
platforms you have tested it on). This document should be written as a plain text file (ie ASCII � MS Word binary
files are not acceptable) � and this file should be called README.TXT. This file should constitute the user
manual of your program, and should explain exactly how to use your program.
The README file should be the user manual for your language as implemented.
NB:
I want it to read values from the file as given above and execute the answer in the command prompt( as intented in the file above)
Thanks
Alam Ikenna

Hai I have one doubt in doing other peoples homework for them. Why bother?

Need Help: UTL_FILE Reading and Writing to Text File

Hello I am using version 11gR2 using the UTL_FILE function to read from a text file then write the lines where it begins with word 'foo' and end my writing to the text file where the line with the word 'ZEN' is found. Now, I have several lines that begin with 'foo' and 'ZEN' Which make for one full paragraph, and in this paragraph there's a line that begins with 'DE4.2'. Therefore,
I need to write all paragraphs that include the line 'DE4.2' in their beginning and ending lines 'foo' and 'ZEN'
FOR EXAMPLE:
FOO/234E53LLID
THIS IS MY SECOND LINE
THIS IS MY THIRD LINE
DE4.2 THIS IS MY FOURTH LINE
THIS IS MY FIFTH LINE
ZEN/DING3434343
FOO/234E53LLID
THIS IS MY SECOND LINE
THIS IS MY THIRD LINE
THIS IS MY FIFTH LINE
ZEN/DING3434343
I am only interested in writing the first paragraph tha includes line DE4.2 in one of ther lines Not the Second paragraph that does not include the 'DE4.2'
Here's my code thus far:
CREATE OR REPLACE PROCEDURE my_app2 IS
infile utl_file.file_type;
outfile utl_file.file_type;
buffer VARCHAR2(30000);
b_paragraph_started BOOLEAN := FALSE; -- flag to indicate that required paragraph is started
BEGIN
-- open a file to read
infile := utl_file.fopen('TEST_DIR', 'mytst.txt', 'r');
-- open a file to write
outfile := utl_file.fopen('TEST_DIR', 'out.txt', 'w');
-- check file is opened
IF utl_file.is_open(infile)
THEN
-- loop lines in the file
LOOP
BEGIN
utl_file.get_line(infile, buffer);
     --BEGINPOINT APPLICATION
IF buffer LIKE 'foo%' THEN
          b_paragraph_started := TRUE;
     END IF;
     --LOOK FOR GRADS APPS
          IF b_paragraph_started AND buffer LIKE '%DE4%' THEN
          utl_file.put_line(outfile,buffer, FALSE);
END IF;
     --ENDPOINT APPLICATION
          IF buffer LIKE 'ZEN%' THEN
     b_paragraph_started := FALSE;
          END IF;
utl_file.fflush(outfile);
EXCEPTION
WHEN no_data_found THEN
EXIT;
END;
END LOOP;
END IF;
utl_file.fclose(infile);
utl_file.fclose(outfile);
EXCEPTION
WHEN OTHERS THEN
raise_application_error(-20099, 'Unknown UTL_FILE Error');
END my_app2;
When I run this code I only get one line: DE4.2 I AM MISSING THE ENTIRE PARAGRAPH
PLEASE ADVISE...

Hi,
Look at where you're calling utl_file.put_line. The only time you're writing anything is immediately after you find the the key word 'DE4', and then you're writing just that line.
You need to store the entire paragraph, and when you reach the end of the paragraph, write the whole thing only if you found the key word, like this:
CREATE OR REPLACE PROCEDURE my_app2 IS
    TYPE line_collection
    IS       TABLE OF VARCHAR2 (30000)
           INDEX BY BINARY_INTEGER;
    infile               utl_file.file_type;
    outfile                      utl_file.file_type;
    input_paragraph          line_collection;
    input_paragraph_cnt          PLS_INTEGER     := 0;          -- Number of lines stored in input_paragraph
    b_paragraph_started      BOOLEAN      := FALSE;     -- flag to indicate that required paragraph is started
    found_key_word          BOOLEAN          := FALSE;     -- Does this paragraph contain the magic word?
BEGIN
    -- open a file to read
    infile := utl_file.fopen('TEST_DIR', 'mytst.txt', 'r');
    -- open a file to write
    outfile := utl_file.fopen('TEST_DIR', 'out.txt', 'w');
    -- check file is opened
    IF utl_file.is_open(infile)
    THEN
     -- loop lines in the file
     LOOP
         BEGIN
          input_paragraph_cnt := input_paragraph_cnt + 1;
             utl_file.get_line (infile, input_paragraph (input_paragraph_cnt));
          --BEGINPOINT APPLICATION
          IF LOWER (input_paragraph (input_paragraph_cnt)) LIKE 'foo%' THEN
              b_paragraph_started := TRUE;
          END IF;
          --LOOK FOR GRADS APPS
          IF b_paragraph_started
          THEN
              IF input_paragraph (input_paragraph_cnt) LIKE '%DE4%'
              THEN
               found_key_word := TRUE;
              END IF;
              --ENDPOINT APPLICATION
              IF input_paragraph (input_paragraph_cnt) LIKE 'ZEN%' THEN
                  b_paragraph_started := FALSE;
               IF found_key_word
               THEN
                   FOR j IN 1 .. input_paragraph_cnt
                   LOOP
                       utl_file.put_line (outfile, input_paragraph (j), FALSE);
                   END LOOP;
               END IF;
               found_key_word := FALSE;
               input_paragraph_cnt := 0;
              END IF;
          ELSE     -- paragraph is not started
              input_paragraph_cnt := 0;
          END IF;
          EXCEPTION
              WHEN no_data_found THEN
               EXIT;
          END;
      END LOOP;
    END IF;
    utl_file.fclose (infile);
    utl_file.fclose (outfile);
--EXCEPTION
--    WHEN OTHERS THEN
--        raise_application_error(-20099, 'Unknown UTL_FILE Error');
END my_app2;
SHOW ERRORSIf you don't have an EXCEPTION section, the default error handling will print an error message, spcifying exactly what the error was, and which line of your code caused the error. By using your own EXCEPTION section, you're hiding all that information. I admit, the error messages aren't always as informative as we'd like, but they're never less informative than "Unknown UTL_FILE Error'. Don't use your own EXCEPTION handling unless you can improve on the default.
Remember that anything inside quotes is case-sensitive. If your file contains upper-case 'FOO', then it won't be "LIKE 'foo%' ".
Edited by: Frank Kulash on Dec 7, 2011 1:35 PM
You may have noticed that this site normally doesn't display multiple spaces in a row.
Whenever you post formatted text (such as your code) on this site, type these 6 characters:
\{code}
(small letters only, inside curly brackets) before and after each section of formatted text, to preserve spacing.

Building array and writing to multiple files

Similar Messages

Maybe you are looking for