Is there a file size limit when using Read From Spreadsheet File?

I'm trying to read in a large file, about 52mb, 525600 lines with 27 fileds in each line using "Read From SpreadsheetFile.vi". I then search the 2D array for -999, which represents bad/no data in that field and total the number of fields with -999 in it. This all works on 3 months worth of data. The program is giving me an out of memory error and stopping in the case where the read takes place on the large one year file. So my question is does the Read from Spreadsheet file have size limitations? Is there a better way to do this? Thanks in advance for the help.
ssmith
Solved!
Go to Solution.

Camerond--
       Thanks for the help. I recreated the VI you posted and unfortunately it doesnt work. It looks like the index array vi is only looking at the zero index and not the other 26 fields in each row. This would probably require a FOR LOOP to go through each field index. This is what I have and it works on a smaller version of the file. For some reason LV is using up bucket loads of memory to run this smaller file and crashes running the 60MB file. I've attached one of my vi's to solve this problem and the smaller data set. You'll see in the beginning that I trim some columns and rows to leave just the data I want to scan. Thanks again for the help.
    I just tried send 3 months worth of one minute data and it failed. So here is a really small version, 20 minutes worth of data. Does anyone see anything that would cause memory usage issues here?
ssmith
Attachments:
Find-999NoLoops.vi ‏17 KB
TEST2MET.txt ‏3 KB

Similar Messages

  • Format precision apparently ignored when using read from spreadsheet file

    I hope there is ridiculously simple solution to this problem but so far I can't find it. Using the Read from Spreadsheet File function in LabVIEW7.1, I can't get the floating point format (%f) to work; the decimal (%d) seems to work fine. For example, the first value in the attached file is 399.5853. When I read it in using %.2f format I would think I should get 399.58 but instead I get 399.58529663... Oddly if I use %d as the format, I get an as expected value of 399. Can anyone see what I am missing?
    Attachments:
    ReadFileTEST.vi ‏23 KB
    test.txt ‏1 KB

    The solution to the problem is the internal representation of floating point numbers: with a limited number of bytes only a limited number of the (infinitely many) real numbers can be represented. The nearest representable number to 399.5853 is 399.58529663 if the so-called single precision representation (abbr. SGL in Labview) is used.
    It seems that (ridiculously!) NI has chosen to use SGL as the data format in the Read from Spreadsheet VI which causes the unexpected behaviour you observed.
    You can change the representation from SGL (which uses 4 bytes) to DBL (double precision, using 8 bytes) in the Read from Spreadsheet VI and you will observe a better approximation then, but still not 'exact' since the number now reads 399.585300000000018. (With t
    he EXT representation, 10 bytes, you could go to even higher precision)
    I hope someone from NI reads this and they fix the unnecessary limitation of the precision in the Read from Spreadsheet VI.

  • Problem using read from spreadsheet file and polar plotting

    Hi to all labview users,
    i am a beginner in labview and i am trying to do a polar plot.
    i read the polar plotting example in labview and it was straightforward.
    I used "write to spreadsheet file" to gather data.
    and they are in the following format
    13  10
    4  20
    8 30
    ....etc
    now. i tried using "read from spreadfile" to get the data into a array, then using "array to cluster" to convert the array into cluster, so i could connect it to the polar plot block
    however, it kept saying i couldnt connect that way, because polar plot uses 1-d array with cluster of 2 element and my source is a cluster of 9 elements....
    but doesnt the "read from spreadfile" block give me a 1-d array? and where does that 9 come from? i only have 3 rows and 2 columns in my data file....
    any guidance would be greatly appreciated.
    thx alot
    Happy guy
    ~ currently final year undergraduate in Electrical Engr. Graduating soon! Yes!
    ~ currently looking for jobs : any position related to engineering, labview, programming, tech support would be great.
    ~ humber learner of LabVIEW lvl: beginner-intermediate

    Helllo,
    I've made an example to try to help you  with that question.
    Notes:
     - the file must have values separeted by tab
     - reading the values from file as you mentioned using "read from spreadfile" you'll get a 2D array and not 1D;
    Software developer
    www.mcm-electronics.com
    PS: Don't forget to rate a good anwser ; )
    Currently using Labview 2011
    PORTUGAL
    Attachments:
    Read Table and plot polar graph.vi ‏26 KB
    teste.txt ‏1 KB

  • EOF detection in "Read From Spreadsheet File"

    Hi folks, I need help in detecting EOF in a "Read From Spreadsheet File". I' d like to stop a while loop by wiring the EOF? indicator to the stop condition, but this doesn't work. For design resons I can't substitute the while loop with a for loop and I DO need the while loop. I read something about creating custom EOF detection but I'm wondering why the builtin function in the vi doesn' t work and if there isn't some simple solution. Many thanks, hazibuli  

    I'm guessing that you want to process one line at a time from the spreadsheet.  Use Read From Spreadsheet File to read in the entire spreadsheet as Dennis says.  Then wire the 2D output to a For Loop with indexing enabled.  Each iteration of the For Loop will allow you to process one line at a time.  When the last line is processed, the For Loop will exit automatically.  Here is an example to get string length of each element in a spreadsheet. Message Edited by tbob on 03-15-2010 09:46 AM

  • Read From Spreadsheet File.vi in windows 7

    Hi,
    I use Read From Spreadsheet File.vi in windows 7 to read a table, but when it executes, i get a dialogue box asking for the filename, (The box is properly wired to the path).
    However, when i execute exactly the same programme in windows Xp it works with no problem.
    ¿Any known issues with this vi and xindows 7?
    I am using labview 8.6.
    Thanks for your help.

    I am using a relative path as shown in the figure, files are user1.xls user2.xls .... they all exist.
    I have build the program and instaled on other xp machine, it works. Then installed in a win 7 machine and is not working.
    Maybe there is a problem with ..\..\?
    Any other way to go to a parent folder not using ..\ ?
    Thanks for you help
    Attachments:
    ejemplo.JPG ‏8 KB

  • Read from spreadsheet file

    Hi! I am a beginner labview user and I have a problem reading data from a
    ..xls file. I have written in columns the data that i want to present in a
    waveform graph through labview. I give the path to the Read From Spreadsheet
    File tool and i connect a true constant in the "transpose" terminal. The Run
    button shows me no error but all I get in my waveform graph is a x-axis
    line. Could someone help me? Thanx

    You say that you want to read data from a .xls file. You use Read From Spreadsheet File to get the data into LV.
    You have mixed some things. A .xls file is NOT a spreadsheet file. It is a file which could be read by MS Excel and some other tools which have import filters for this file. Read from Spreadsheet File is not such a filter.
    A spreadsheet file in the context of LV is a text file where the cells of a row are separeted by a tab and the rows are separated by a end of line.
    A save thing is use Active-X automation to communicate with MS Excel. Let it load the file and read out the cells. The are examples in LV help and in the dev zone. You must have installed and licenced MS Ecxel on your target machine. Maybe someone has written such a fil
    ter for LV but I don't know it.
    The other way will only work if you use US local settings and the data is in one worksheet. Save the file with the extention .txt. So Excel will generate a text file of the correct format. Excel will use the regional settings of your machine. You can read in this file with Read From Spreadsheet File. Unfortunatly this will use fixed settings which are the same as for US regional settings.
    Waldemar
    Waldemar
    Using 7.1.1, 8.5.1, 8.6.1, 2009 on XP and RT
    Don't forget to give Kudos to good answers and/or questions

  • Is there a Maximum file size limit when combining pdf files

    I am trying to combine files 8 PDF files created with i Explore using Acrobat XI Standard, the process is completed but when opening the file not all 8 individual files have been combined only the first two have been combined, the process shows all eight being read and combined.
    I can open and view each of the original 8 individual files and read the content without any issue, the largest of the eight files is 15,559kb and the smallest is 9,435kb
    if anyone can assist that would be most appreciated

    Hi Tayls450,
    This should not have happened as there is no maximum file size limit when combining PDFs.
    Please let me know if there are some password protected PDFs also?
    Also, I would suggest you to choose 'Repair Acrobat Installation' option from the Help menu.
    Regards,
    Anubha

  • Read an avi file using "Read from binary file" vi

    My question is how to read an avi file using "Read from binary file" vi .
    My objective is to create a series of small avi files by using IMAQ AVI write frame with mpeg-4 codec of 2 second long (so 40 frames in each file with 20 fps ) and then send them one by one so as to create a stream of video. The image are grabbed from USB camera. If I read those frames using IMAQ AVI read frame then compression advantage would be lost so I want to read the whole file itself.
    I read the avi file using "Read from binary file" with unsigned 8 bit data format and then sent to remote end and save it and then display it, however it didnt work. I later found that if I read an image file using "Read from binary file" with unsigned 8 bit data format and save it in local computer itself , the format would be changed and it would be unrecognizable. Am I doing wrong by reading the file in unsined 8 bit integer format or should I have used any other data types.
    I am using Labview 8.5 and Labview vision development module and vision acquisition module 8.5
    Your help would be highly appreciated.
    Thank you.
    Solved!
    Go to Solution.
    Attachments:
    read avi file in other data format.JPG ‏26 KB

    Hello,
    Check out the (full) help message for "write to binary file"
    The "prepend array or string size" input defaults to true, so in your example the data written to the file will have array size information added at the beginning and your output file will be (four bytes) longer than your input file. Wire a False constant to "prepend array or string size" to prevent this happening.
    Rod.
    Message Edited by Rod on 10-14-2008 02:43 PM

  • Read From Spreadsheet File help

    I am saving all my data to a text file (See attached file). The problem is that when I try and use the Read From Spreadsheet File.vi I am unclear on how to make it show all my data.
    Does anyone know a away for me to read the whole file and display it all at once reather than line by line?
    Solved!
    Go to Solution.
    Attachments:
    test data 1.txt ‏6 KB

     The read from spread sheet Vi has two data outputs : the top right retrieves the whole file as a 2D array, if you leave the "number of rows" input on the left unwired.
    Message Edité par chilly charly le 01-07-2009 11:22 AM
    Chilly Charly    (aka CC)
             E-List Master - Kudos glutton - Press the yellow button on the left...        

  • Read from spreadsheet file with multiple delimiters

    Is there a way to specify multiple delimitors in the read from spreadsheet file vi? I have a file that i need to read in that contains both space and comma delimitors and would like to read that data into an array using both delimitors (or not and). Below is the data I'm trying to read.
    ;attenuator data table
    att00:   db       000h,015h,017h,035h,03Ch,03Eh,03Eh,05Ch,05Eh,05Eh
    att10:   db       07Ch,07Eh,07Fh,09Dh,09Fh,09Fh,0BDh,015h,017h,035h
    att20:   db       03Eh,03Eh,05Ch,05Ch,05Eh,07Ch,07Ch,07Eh,09Dh,09Dh
    att30:   db       09Fh,0BDh,000h,000h,000h,002h,002h,002h,002h,003h
    att40:   db       021h,021h,021h,021h,021h,023h,023h,023h,023h,023h
    att50:   db       041h,041h,048h,048h,048h,04Ah,04Ah,04Ah,04Ah,068h
    att60:   db       068h,068h,068h,068h,068h,06Ah,06Bh,06Bh,06Bh,089h
    att70:   db       089h,089h,089h,08Bh,08Bh,08Bh,08Bh,0A9h,0A9h,0A9h
    att80:   db       0A4h,0A6h,0A6h,0A6h,0A6h,0C4h,0C4h,0C4h,0C4h,0C6h
    att90:   db       0C6h,0C6h,0C6h,0E4h,0E4h,0E5h,0E5h,0E7h,0E7h,0E7h
        END
    I'm looking to just read in the data adjust the hex values and then save the data in the exact form which I read it in. If read from spreadsheet file can not recognize multiple delimiters that is all I need to know. I do not want to spend time reading it in using a single delimitor and doing a bunch of string manipulation. I'm also working with LabView 8.5 if that makes a difference.

    You should use "scan string for tokens", and wire an array of delimiters.
    One nice behavior is the fact that consecutive delimiters are contracted into one (by default), so e.g. if your delimiters is an array containing a space and a comma, a sequence of three spaces and a comma would still count as one delimiter.
    For some ideas, have a look at my old example here:
    http://forums.ni.com/ni/board/message?board.id=170&message.id=192847#M192847
    LabVIEW Champion . Do more with less code and in less time .

  • For Mike Porter: Read from Spreadsheet File.vi" function

    On 2/17/03 you answered my question about reading and viewing a test done last week back into Labview. I can open the "Read from Spreadsheet File.VI" in Labview and read the file "data.txt" into the VI. The data comes back into the rows & cells on the VI screen but I don't know how to open the graph to see the data as curves. I can see about a 3 x 4 array of cells at a time and the correct data is there but I want to plot it. Do I need to create a new block diagram and VI? I apologize for the vagueness of my question - I'm new at Labview.
    Attachments:
    DATA2.TXT ‏183 KB

    Create a VI with a Waveform Graph on the front panel. On the block diagram, insert a Read From Spreadsheet File function. Connect the All Rows output to the Waveform Graph.
    Attachments:
    Graph.jpg ‏3 KB

  • 求助: 如何使用write to spreadsheet file.vi 和 read from spreadsheet file.vi, 并且如何定义它们的路径???

    请问如何使用write to spreadsheet file.vi 和 read from spreadsheet file.vi,我知道spreadsheet file.vi只支持.txt文件,但我应该在哪里设置它们的路径呢,请各位帮忙指点,能不能具体说明下,谢谢...

    write to spreadsheet file.vi 和 read from spreadsheet file.vi,支持.xls文件.
    可以在path处设置它们的路径.

  • Is there a file size limit when creating iPad apps?

    When creating an iPad app that has a lot of video in it, what is the file size limit?

    ask on the developer forums at http://devforums.apple.com

  • Labview 8 crashes when using Read Labview Measurement File

    Hi All,
    I am a new user for Labview. I installed Labview 8.0 from the CD (evaluation copy). I was just testing the sample code provided by NI. I opened the "Load from File and Display" example and I was trying to open the properties for the Read Labview Measurement File component when Labview crashed. I tried it a couple of times and everytime it happens the same way (when I double click the component or right click on it).
    I am using Windows XP with all the updates done. In the Windows crash report, the application version is 8.0.0.4005.
    Other problem I encountered is a missing component on the Input Palette (no DAQ Assistant present) and the Instrument I/O Assistant raises the error "Measurement & Automation Explorer or the Instrument I/O Assistant is not installed correctly. Please install these from the Labview Driver CD". For the last problem, I will try to run the Labview Driver CD, once I get it. But I am not sure if the DAQ Assistant will show up then or not.
    Any ideas?
    Regards,
    Nick

    Hello Nick,
    While it is not my area of expertise, I would expect the DAQ Assistant would appear after installing the driver CD as you said you would be trying.
    What caught my attention was the crash.  The easiest way to resolve the crash is to delete the instance of the Read LabVIEW Measurement File on the diagram and place a new one from the Express Input palette (now called "Read From Measurement File").  In that situation, I would expect just pressing OK to accept the default configuration should work.  The crash occurs in certain circumstances when the Read / Write Measurement File Express VIs are dropped from a previous version of LabVIEW and then their config page is opened in 8.0.x.  This issue should be resolved in a future release, sorry for the inconvenience.

  • How to open saved files using 'read from text file' function

    Hi everyone, I am having a hard time trying to solve the this particular problem ( probably because I am a newb to lanbview ). Anyway , I am able to save the acquired waveforms by using the 'Write to text file' icon. I did manually modify the block diagram of the 'Write to text file' icon and create the correct number of connector so as to make my program work. But now I have no idea on how to modify the block diagram of the 'Read from text file' block diagram to make my program 'open' my saved waveforms. Or i do not have to modify anything from the block diagram of the 'Read from text file'? Can anyone teach/help me connect up? Do i need the build array on the "open" page?
    Here are some screenshots on part of my program  
    let me know if you guys would need more information / screenshots thank you!
    Attachments:
    ss_save.jpg ‏94 KB
    ss_open.jpg ‏94 KB
    modified_writetotextfile.jpg ‏99 KB

    Ohmy, thanks altenbach. oh yeah i forgot about those sub VIs. will upload them now. Was rather demoralized after reading the comments and really struck me on how weak i'm at on labview really hope to get this done. But of course i have to study through and see how it works. Actually i am going to replace those 'signal generators sub vi' with ThoughtTechonology's sample code so i can obtain data waveforms real-time using Electrocardiography (ECG) ,Electromyography (EMG ) and Electroencephalography (EEG) hopefully i can find out how to connect the sample code.
    ( ps . cant connect it now unless my program is working otherwise labview will crash ) 
    ( p.s.s the encoder of my biofeedback trainer already acts as an DAQ so i wont need to place an DAQ assistant in my block diagram i suppose )
    The sample code of ThoughtTechnology is named as attachment.ashx.vi. too bad i cant use it and present it as my project
    Attachments:
    frequency detactor.vi ‏53 KB
    signal generator.vi ‏13 KB
    attachment.ashx.vi ‏40 KB

Maybe you are looking for