Parsing Quicker

Question

Hi everyone,
I am fairly new to LabVIEW and here is my task: I need to import an immensely large data file (16 megabytes) of a long chain of 16-bit binary numbers.  i would have to parse this data into 4 columns of 16-bit numbers until the end of the file.
elaboration: If the data were a counter (represented in hex) and it started from 0000, 0001 .... FFFF, 0000, 0001, etc. the data would look like this:
0000000100020003000400050006000700080009000A000B000C (and so on for a 16 megabyte file). 
I would have to turn it into this:
0000  0001  0002  0003
0004  0005  0006  0007
0008  0009  000A  000B (and so on, delimited by tabs).
I have the VI attached to do this function, with the flexibility of parsing it into any number of columns, while counting and shifting accordingly.   I additionally used the for loop iteration "i" as the shift register by multiplying it by the number of columns needed using to "Write to Spreadsheet File."  I then compared the counter times 4 (in this case) and compared it to the byte length of the total file to detect the EOF to terminate the loop.
This program works.  The problem is, it takes too long to parse it this way for a 16mb file.  The addition of the tab delimiters and carriage returns increase the output file to a 30mb file and it takes approximately 2 hours to parse in total.  I made this as eloquent and simple as possible but Im just wondering if there is another method to do this to make this quicker (aiming for less than an hour).  Is there a limitation in LabVIEW that im overlooking?  or is my CPU just too slow?  are working with arrays usually this slow?  is each comparision of length of bytes and counter after each for loop iteration taking up most of the time?
i feel my lack of experience with labVIEW is finally taking its toll when working with such large files.
Solved!
Go to Solution.
Attachments:
SCI parse edit.vi ‏29 KB

Answer 1

It seems to me that all you're doing is reshaping a 1D array into a 2D array with a user-selected number of columns. You do not need a loop to do this. The Reshape Array function does this.
Note that to get the 4 digit format you need to use %04x as the format string, not %x. 
Message Edited by smercurio_fc on 10-08-2008 03:30 PM
Attachments:
untitled.PNG ‏7 KB