Read from csv file and plot particular columns

Hello,
I`m a new user of Labview and here it comes...my first major problem.
Maybe this has been discussed before. I’ve made a search to solve my problem first but I couldn`t find anything helpful so I `ve decided to post a new message.
So here is my problem:
I`m working in a small semiconductor lab where different types of nitrides are grown using proprietary reactor. The goal is to read the collected csv files from each growth in Labview and plot the acquired data in appropriate graphs.
I have a bunch of csv files and I have to make a Labview program to read them.
The first part of my project I`ve decided to be displaying the csv file (growth log file) under labview (which I think works fine).
The second one is to be able to plot particular columns from the recipe in graphs in Labview (that one actually gives me a lot of trouble):
1. Timestamp vs Temperature /columns B and D/
2. Timestamp vs Gas flow /columns L to S/
3. Timestamp vs Pressure /columns E,K,T,U,V/
I`ve got one more problem. How can I convert the Timestamp shown in csv file to human readable date in labview? This actually is a big problem, because the timestamp is my x axis and I want to know at what time a particular process took place and I also want to be able to see the converted timestamp when displaying csv file at first. I`ve read a lot about time stamping in excel and timestamp in labview but I`m still confused how to convert it in my case.
I don`t have problems displaying csv file under Labview. My problems are with the timestamp and the graphs.
Sorry for my awful English. I hope you can understand my problems since English is not my mother language.
Please find the attached files.
If you have any ideas or suggestions I`ll be more than happy to discuss them.
Thank you in advance.
Have a nice day!
Attachments:
growth log.csv ‏298 KB
Read from growth log.vi ‏33 KB

Hello again,
I`m having problems with converting the first column in the attached above file Growth Log.csv.
I have a code converting xl timestamp to time and using Index Array traying to grab a particular column out of it but the attached file is written in strings so I guess I have to redo it in array but I don`t know how.Would you help me with this one?
Attachments:
Xl Timestamp to Time.vi ‏21 KB

Similar Messages

Read from csv file and graph at certain interval

Hi,
I've been trying all day to modify a labView program so it reads in data from my csv file to graph temperatures from 4 ovens. The temperatures are logged in the csv file at 3s intervals and i wish to display them on 4 graphs. I've written a vi (or modified one i should say) to take in a temperature value from a numeric control. Im just having extreme difficulty with the reading part.
If someone can help me it would be greatly appreciated.
Best regards,
Dawn
Attachments:
Oven temps.csv ‏2 KB
Oven_level_meas.vi ‏58 KB

Hi there,
I have a similar problem here. I see your point that you should read the file at once but I don't seem to be able to manage it. Shouldn't you use some kind of
a buffer to achieve this? Let me tell you what I am trying to do: I have a csv file with several columns that I want to display in several waveform charts. Each column is a wavechart. I should be able to change the speed at which the data is displayed. I am able to do this because I change the wait until (ms) in the for loop. However the application is too slow. I think because I open the csv file in the for loop and display it while I am reading it. I have tried to put it outside the for loop but this doesn't work. I think I need some kind of a buffer in which the data is stored after reading it. And then this buffer should be used in the for loop. I have looked at the producer-consumer structure but I dont fully understand it as I am not that experienced with labview. Any help or changes to my VI would be appreciated. I have attached the VI and the file with meaningless data I am using for testing.
Thanks
Attachments:
read_data_file_dynamically.vi ‏39 KB
test.CSV ‏332 KB

Read from text file and plot in real time

Hello to everybody,
I recently started using labview and so I know very few features, I apologize for any crap I'm about to write.
My problem is as follows:
_ A wireless node is sampling, with a known frequency, the voltage and the current of an electronic device
_ These data are sent to a coordinator connected to the PC
_ Each packet sent includes a single column with: Letter V followed by voltage samples, Letter I followed by current samples (all in a single column as in the attached example)
_ The coordinator saves it in a .txt file
I want to do a labview application that:
_ Reads that file (continuosly update by the coordinator)
_ Plots on two different graphs the values (over the time) of current and voltage
Thanks for your help
Danilo
Attachments:
TextData.txt ‏1 KB

There are File IO examples that ship with LV which show you how to write to red from a file . A more fundamental question though is why have the file at all? Is it serving strictly as a data communication channel, or will it be used for something else?
The "coordinator" that you mention, can it be accessed directly? Serial IO? TCP/IP? Telnet? ActiveX? .NET? Anything?
If so, that would be the preferred way to handle this situation.
Mike...
PS: Sorry for the next bit, but I used to be a magazine editor and my grandmother was a school teacher so I am just a tiny bit anal retentive about word usage: In English "data" is a funny word in that while it is a singular form, it is (depending upon context) often understood to referring to multiple things and so can be sort of plural. Consequently, it is proper to say: "This data is sent.. " or "The data indicates...", but not, "The data are showing..". Likewise it is always incorrect to try to make it plural by saying "datas".
PPS: The same goes for the word "code" when used as a noun. It can also be a verb, but that's a whole different rant...
PPPS: Also be aware that many of the people who know me well would object to the "tiny bit" part...
Certified Professional Instructor
Certified LabVIEW Architect
LabVIEW Champion
"... after all, He's not a tame lion..."
Be thinking ahead and mark your dance card for NI Week 2015 now: TS 6139 - Object Oriented First Steps

Read a CSV file and dynamically generate the insert

I have a requirement where there are multiple csv's which needs to be exported to a sql table. So far, I am able to read the csv file and generate the insert statement dynamically for selected columns however, the insert statement when passed as a parameter
to the $cmd.CommandText
does not evaluate the values
How to evaluate the string in powershell
Import-Csv -Path $FileName.FullName | % {
# Insert statement.
$insert = "INSERT INTO $Tablename ($ReqColumns) Values ('"
$valCols='';
$DataCols='';
$lists = $ReqColumns.split(",");
foreach($l in $lists)
$valCols= $valCols + '$($_.'+$l+')'','''
#Generate the values statement
$DataCols=($DataCols+$valCols+')').replace(",')","");
$insertStr =@("INSERT INTO $Tablename ($ReqColumns) Values ('$($DataCols))")
#The above statement generate the following insert statement
#INSERT INTO TMP_APPLE_EXPORT (PRODUCT_ID,QTY_SOLD,QTY_AVAILABLE) Values (' $($_.PRODUCT_ID)','$($_.QTY_SOLD)','$($_.QTY_AVAILABLE)' )
$cmd.CommandText = $insertStr #does not evaluate the values
#If the same statement is passed as below then it execute successfully
#$cmd.CommandText = "INSERT INTO TMP_APL_EXPORT (PRODUCT_ID,QTY_SOLD,QTY_AVAILABLE) Values (' $($_.PRODUCT_ID)','$($_.QTY_SOLD)','$($_.QTY_AVAILABLE)' )"
#Execute Query
$cmd.ExecuteNonQuery() | Out-Null
jyeragi

Hi Jyeragi,
To convert the data to the SQL table format, please try this function out-sql:
out-sql Powershell function - export pipeline contents to a new SQL Server table
If I have any misunderstanding, please let me know.
If you have any feedback on our support, please click here.
Best Regards,
Anna
TechNet Community Support

Read a csv file and read the fiscal yr in the 4th pos?

Hello ABAP Experts,
how to write a code for read a csv file and read the fiscal year in the 4th position.
any suggestions or code highly appreciated.
Thanks,
BWer

Hi Bwer,
Declare table itab with the required fields...
Use GUI UPLOAD to get the contents of the file (say abc.csv) in case if the file is on the presentation server...
CALL FUNCTION 'GUI_DOWNLOAD'
EXPORTING
    filename                        = 'c:\abc.csv'
   FILETYPE                        = 'ASC'
    WRITE_FIELD_SEPARATOR           = 'X'
tables
    data_tab                        = itab
EXCEPTIONS
   FILE_WRITE_ERROR                = 1
   NO_BATCH                        = 2
   OTHERS                          = 22
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
         WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
Use OPEN DATASET in case if the file is on the application server..
After that USE SPLIT command at comma to get the contents of the 4th field...
Regards,
Tanveer.
<b>Please mark helpful answers</b>

How to read from excel file and write it using implicit jsp out object

our code is as below:Please give us proper solution.
we are reading from Excel file and writing in dynamicaly generated Excel file.it is writing but not as original excel sheet.we are using response.setContentType and response.setHeader for generating pop up for saveing the original file in to dynamically generated Excel file.
<%@ page contentType="application/vnd.ms-excel" %>
<%
     //String dLoadFile = (String)request.getParameter("jspname1");
     String dLoadFile = "c:/purge_trns_nav.xls" ;
     File f = new File(dLoadFile);
     //set the content type(can be excel/word/powerpoint etc..)
     response.setContentType ("application/msexcel");
     //get the file name
     String name = f.getName().substring(f.getName().lastIndexOf("/") + 1,f.getName().length());
     //set the header and also the Name by which user will be prompted to save
     response.setHeader ("Content-Disposition", "attachment;     filename="+name);
     //OPen an input stream to the file and post the file contents thru the
     //servlet output stream to the client m/c
          FileInputStream in = new FileInputStream(f);
          //ServletOutputStream outs = response.getOutputStream();
          int bit = 10;
          int i = 0;
          try {
                    while (bit >= 0) {
                    bit = in.read();
                    out.write(bit) ;
} catch (IOException ioe) { ioe.printStackTrace(System.out); }
          out.flush();
out.close();
in.close();
%>

If you want to copy files as fast as possible, without processing them (as the DOS "copy" or the Unix "cp" command), you can try the java.nio.channels package.
import java.nio.*;
import java.nio.channels.*;
import java.io.*;
import java.util.*;
import java.text.*;
class Kopy {
     * @param args [0] = source filename
     *        args [1] = destination filename
    public static void main(String[] args) throws Exception {
        if (args.length != 2) {
            System.err.println ("Syntax: java -cp . Kopy source destination");
            System.exit(1);
        File in = new File(args[0]);
        long fileLength = in.length();
        long t = System.currentTimeMillis();
        FileInputStream fis = new FileInputStream (in);
        FileOutputStream fos = new FileOutputStream (args[1]);
        FileChannel fci = fis.getChannel();
        FileChannel fco = fos.getChannel();
        fco.transferFrom(fci, 0, fileLength);
        fis.close();
        fos.close();
        t = System.currentTimeMillis() - t;
        NumberFormat nf = new DecimalFormat("#,##0.00");
        System.out.print (nf.format(fileLength/1024.0) + "kB copied");
        if (t > 0) {
            System.out.println (" in " + t + "ms: " + nf.format(fileLength / 1.024 / t) + " kB/s");
}

Read two CSV files and remove the duplicate values within them.

Hi,
I want to read two CSV files(which contains more than 100 rows and 100 columns) and remove the duplicate values within that two files and merge all the unique values and display it as a single file.
Can anyone help me out.
Thanks in advance.

kirthi wrote:
Can you help me....Yeah, I've just finished... Here's a skeleton of my solution.
The first thing I think you should do is write a line-parser which splits your input data up into fields, and test it.
Then fill out the below parse method, and test it with that debugPrint method.
Then go to work on the print method.
I can help a bit along the way, but if you want to do this then you have to do it yourself. I'm not going to do it for you.
Cheers. Keith.
package forums.kirthi;
import java.util.*;
import java.io.PrintStream;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import krc.utilz.io.ParseException;
import krc.utilz.io.Filez.LineParser;
import krc.utilz.io.Filez.CsvLineParser;
public class DistinctColumnValuesFromCsvFiles
public static void main(String[] args) {
    if (args.length==0) args = new String[] {"input1.csv", "input2.csv"};
    try {
      // data is a Map of ColumnNames to Sets-Of-Values
      Map<String,Set<String>> data = new HashMap<String,Set<String>>();
      // add the contents of each file to the data
      for ( String filename : args ) {
        data.putAll(parse(filename));
      // print the data to output.csv
      print(data);
    } catch (Exception e) {
      e.printStackTrace();
private static Map<String,Set<String>> parse(String filename) throws IOException, ParseException {
    BufferedReader reader = null;
    try {
      reader = new BufferedReader(new FileReader(filename));
      CsvLineParser.squeeze = true; // field.trim().replaceAll("\\s+"," ")
      LineParser<String[]> parser = new CsvLineParser();
      int lineNumber = 1;
      // 1. read the column names (first line of file) into a List
      // 2. read the column values (subsequent lines of file) into a List of Set's of String's
      // 3. build a Map of columnName --> columnValues and return it
    } finally {
      if(reader!=null)reader.close();
private static void debugPrint(Map<String,Set<String>> data) {
    for ( Map.Entry<String,Set<String>> entry : data.entrySet() ) {
      System.out.println("DEBUG: "+entry.getKey()+" "+Arrays.toString(entry.getValue().toArray(new String[0])));
private static void print(Map<String,Set<String>> data) {
    // 1. get the column names from the table.
    // 2. create a List of List's of String's called matrix; logically [COL][ROW]
    // 3. print the column names and add the List<String> for this col to the matrix
    // 4. print the matrix by inerating columns and then rows
}

Automator: Create new folders from .csv file and put images inside.

I have a challenging Automator task to achieve. I need to
1) create a set of folders labeled with the contents in column A of a csv file. (example: column A1 JoeBrown A2 SuzyBrown A3 JimBrown etc..) resulting in
3 folders titled JoeBrown, SuzyBrown, and JimBrown
2) take the image represented by an image number in column B of the same csv file and move that file into the folder that was created (example: A1 JoeBrown gets a folder created by the name JoBrown and column B IMG_1234.jpg is physically moved from within the same folder as the .csv file into the folder created by column A.
Both the csv and img files would be in the same folder. I have the script to create a set of folders from a csv list just can't complete the moving of files based on the contents of column B.
SUMMARY: I have a folder containing several images img_0001, img_0002, img_0003 etc. and a csv file that contains columnA a series of names and column B a series of jpg image numbers. I need to have a script that will create folders labeled whatever name is in column A (see the script below. it works great!) then move the images from column B to the folders generated from column A.
The csv script is:
tell application "Finder"
set mgFolder to container of mgCSVfile as string
repeat with x from 1 to count paragraphs of mgList
    set text item delimiters of AppleScript to ","
    set mgThisList to text items of paragraph x of mgList as list
    set text item delimiters of AppleScript to ""
    set mgTopFolder to item 1 of mgThisList
    if (exists folder mgTopFolder of folder mgFolder) is false then
      make new folder at mgFolder with properties {name:mgTopFolder}
    end if
    set mgNewFolder to (folder mgTopFolder of folder mgFolder) as alias
    repeat with i from 2 to count mgThisList
      if item i of mgThisList is not "" then
        set mgSubFolder to item i of mgThisList
        if (exists folder mgSubFolder of folder mgNewFolder) is false then
          make new folder at mgNewFolder with properties {name:mgSubFolder}
        end if
      end if
    end repeat
end repeat
end tell
Thanks for any help that may come my way.

This should work
(assumes a true "," seperated file with only two items per line, i.e.: Test,Test.jpg):
The Run Shell Script Action is:
cd "${1%/*}"
while read line
do
     FolderName=${line%,*}
     ImageName=${line#*,}
     mkdir "$FolderName"
     mv "$ImageName" "$FolderName"
done < "$1"

Scenario to read from txt-file and create a sales order

Hi,
i have started creating a scenario to read from an txt-file and to create a sales order.
When i activate my scenario, i get the following message :
    no scenario step (vBIU) associated for this step for the incoming system (SysId)
When i look into the detailed xml-file i see that all the records are read from the file, but there is no next step to be treated.
Has somebody any idea how this come ?
-<Msg MessageLog="true" msglogexcl="false" logmsg="0009" recording="true" BeginTimeStamp="20111202095144" MessageId="11120209514499820828C0A801674F24" xmlns="urn:com.sap.b1i.vplatform:entity">-<Header><msglog b1ifactive="true" always="false" step="Default message log"/>-<Resumption><starter ipo="/vP.0010000138.in_FEAN/com.sap.b1i.vplatform.runtime/INB_FI_EXST_ASYN_NAM/INB_FI_EXST_ASYN_NAM.ipo/proc"/></Resumption><IPO tid="11120205535899820808C0A801678C54" Id="INB_FI_EXST_ASYN_NAM"/><Sender Id="0010000138"/><Inbound file="ORDERS_TEST" ext="csv" path="C:\TEMP\In" pltype="txt" wrap="" deli=";"/></Header>-<Body><Payload Type="File exist" Role="T"/>-<Payload Role="S">-<io xmlns="urn:com.sap.b1i.bizprocessor:bizatoms" pltype="txt">
-<row>
<col>OH</col>
<col>0000087077</col>
<col>201110041205</col>
<col>220</col>
<col>9</col>
<col>201110191702</col>
<col>8710624300012</col>
<col>8714252008609</col>
<col>8710624300012</col>
<col>8714252008609</col>
<col>8710624300012</col>
<col>N</col>
<col>N</col>
<col>N</col>
</row>
-<row>
<col>OL</col>
<col>1</col>
<col>8711715844378</col>
<col>20</col>
</row>-<row>
<col>OL</col>
<col>2</col>
<col>8711715844392</col>
<col>60</col>
</rowrow>
<col>OL</col>
<col>16</col>
<col>8710251791092</col>
<col>280</col>
</row>
</io>
</Payload>
</Body>
</Msg>
This is my final atom :
<?xml version="1.0" encoding="UTF-8"?><xsl:stylesheet xmlns:b1e="urn:com.sap.b1i.sim:b1event" xmlns:b1ie="urn:com.sap.b1i.sim:b1ievent" xmlns:bfa="urn:com.sap.b1i.bizprocessor:bizatoms" xmlns:jdbc="urn:com.sap.b1i.adapter:jdbcadapter" xmlns:rfc="urn:sap-com:document:sap:rfc:functions" xmlns:sim="urn:com.sap.b1i.sim:entity" xmlns:utils2="com.sap.b1i.bpc_tools.Utilities" xmlns:vpf="urn:com.sap.b1i.vplatform:entity" xmlns:xci="urn:com.sap.b1i.xcellerator:intdoc" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" bfa:force="" vpf:force="" jdbc:force="" rfc:force="" b1ie:force="" b1e:force="" xci:force="" sim:force="" utils2:force=""><xsl:output method="xml" encoding="UTF-8" indent="yes"></xsl:output><xsl:param name="atom"></xsl:param><xsl:param name="sessionid"></xsl:param><xsl:variable name="msg" select="/vpf:Msg/vpf:Body/vpf:Payload[./@Role='S']"></xsl:variable><xsl:variable name="vpSender" select="/vpf:Msg/vpf:Header/vpf:Sender/@Id"></xsl:variable><xsl:variable name="vpObject" select="/vpf:Msg/vpf:Header/vpf:Sender/@ObjId"></xsl:variable><xsl:variable name="vpReceiver" select="/vpf:Msg/vpf:Header/vpf:ReceiverList/vpf:Receiver[./@handover='P']/@Id"></xsl:variable><xsl:template match="/">
<Msg xmlns="urn:com.sap.b1i.vplatform:entity">
<xsl:copy-of select="/vpf:Msg/@*"></xsl:copy-of>
<xsl:copy-of select="/vpf:Msg/vpf:Header"></xsl:copy-of>
<Body>
<xsl:copy-of select="/vpf:Msg/vpf:Body/*"></xsl:copy-of>
<Payload Role="R" id="{$atom}">
<xsl:call-template name="transform"></xsl:call-template>
</Payload>
</Body>
</Msg>
</xsl:template><xsl:template name="transform">
<FinalAtomResult xmlns="">
<BOM>
<BO>
<AdmInfo>
<Object>17</Object>
<Version>2</Version>
</AdmInfo>
     <Documents>
     <row>
     <DocDate>
          <xsl:copy-of select="$msg/io/row[0]/col[2]/text()"></xsl:copy-of>
     </DocDate>
     <DocDueDate>
          <xsl:copy-of select="$msg/io/row[0]/col[5]/text()"></xsl:copy-of>
     </DocDueDate>
     <CardCode>KD10251</CardCode>
     <NumAtCard>
          <xsl:copy-of select="$msg/io/row[0]/col[1]/text()"></xsl:copy-of>
     </NumAtCard>
     <U_PMX_JD_COMP>32</U_PMX_JD_COMP>
     </row>
                      </Documents>
     <Document_Lines>
     <xsl:for-each select="$msg/io/row">
     <row>
     <BarCode>
          <xsl:copy-of select="$msg/io/row[*]/col[2]/text()"></xsl:copy-of>
     </BarCode>
     <Quantity>
          <xsl:copy-of select="$msg/io/row[*]/col[3]/text()"></xsl:copy-of>
     </Quantity>
     </row>
     </xsl:for-each>
     </Document_Lines>

Mike,
you were right, you may not specify the extension of a file.
Another thing i detected is that the loop of my data start with the index 1 instead of 0 !!!
In this scenario, i have to read all the lines of the file and each time i should do a SQL-query,
so that i can use the result to build my sales document.
Do you know if it this is possible or have you any idea how to do this ?
thx,
Mario

Reading a csv file and bind to a data grid

hi. doing a school project and been searching. the application reads a csv file from c:\stocklist.csv, and then this in a button called btnLoadData, and now need to read in a data control called DmgDisplayData. do i put the code in the button, or in the
data grid. been searching, but cannot seem to find any thing. so, where do i put the code, in the data control, and does any one have an example code how to read into the data fields. also need to have one field, able to edit, the other three or four fields,
read only. can any one help me out. never covered this in the subject, but did do file streams a few years ago in vb, but usin g c#,a dn the help in visual studio, not that helpful, with a blind person using a screen reader, jaws for windows from http://www.freedomscientific.com,
and using visual studio 2013 community edition. can any one help me out, been searching and trawling about 15 to 20 pages so far, and did try a couple of sites, but could not find, any help. thanks. the application is to read a csv file from a button, and
load into a data grid, then have a message box, saying file load successful, then have one field, order on as edit, but the other fields, read only, so do i need the navigator buttons, for next, back, previous, etc, and how do i code that as well. not to do
it for me, want to learn, but maybe some sample code, did do navgiator controls, years ago for a vb project, but need the c # example for that, thanks. then you have a button, Save data, that saves the csv file in the data grid. so can google for that. then
have a toolbar, with a button saying, Sort Items, a tool strip, and when you click on that button, you have a drop down list, of three items, then a sort button, which will then sort the array in the data grid. so, do i need another form, or just do the combo
box as an invisible control, then just refrence, that in the toolbar. so need to use th file class and an array, learnt about single and multi arrays. any ideas. thanks.
http://startrekcafe.stevesdomain.net http://groups.yahoo.com/groups/JawsOz

Hi Marvin,
-->where do i put this, in the data grid click event. or in the form load event. thanks.
You could use this code after you initialize the DataGridView. you could put it in the form load event.
-->what about how get the tool bar and the combo box and another button, then sort from the combo box on the array for the collumns, for to set focus to the first read collumn for the data grid. how do i do that, close the parent form, and have another
form on the toolbar.
Since this is another issue of this thread, I would recommend you posting it with
another thread. We will focus on that thread to help you. Thanks for your
understanding.
BTW, Before you asking questions, I suggest you could learn to make it by yourself. You could begin to learn winforms in MSDN articles:
https://msdn.microsoft.com/en-us/library/dd30h2yb%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396. Or Google it, you will get many answers. In that way, you will learn more from the questions.
Best regards,
Youjun Tang
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

Read from .txt file and output the content as two arrays

I am using the contoured move to control the x-y stage. The trajectory datas for x and y axis are generated using my interpolation program and it is stored in a .txt file as two columns. What I want to do is read .txt file and output the content of this file as two arrays. Is there anyone has any ideas? Thanks, the .txt file is attached.
Attachments:
R.75.txt ‏172 KB

Hi Awen,
This is quite easy to do, you can merely use the "read from spreadsheet file" function to get a 2D array (2 columns and n rows) and then use the index array function to get whatever row/colums you want..
Hope the attached VI helps you
When my feet touch the ground each morning the devil thinks "bloody hell... He's up again!"
Attachments:
read sprdsheet file.vi ‏27 KB

Reading a CSV file and producing an array of objects

Hi everybody,
I am writing an application which will read a cvs file and produce an array of objects.
Can any body help me to solve this problem?
Thanks
Finny

Hello,
Have you tried this link?
http://ostermiller.org/utils/CSV.html

"Read from Binary File" and efficiency

For the first time I have tried using Read from Binary File on sizable data files, and I'm seeing some real performance problems. To prevent possible data loss, I write data as I receive them from DAQ, 10 times per second. What I write is a 2-D array of double, 1000 data points x 2-4 channels. When reading in the file, I wish I could read it as a 3-D array, all at once. That doesn't seem supported, so I repeatedly do reads of 2-D array, and use a shift register with Build Array to assemble the 3-D array that I really need. But it's incredibly slow! It seems that I can only read a few hundred of these 2-D items per second.
It also occurred to me that the Build Array being used in a shift register to keep adding on to the array, could be a quadratic-time operation depending on how it is implemented. Continually and repeatedly allocating bigger and bigger chunks of memory, and copying the growing array at every step.
So I'm looking for suggestions on how to efficiently store, efficiently read, and efficiently reassemble my data into the 3-D array that I need. Perhaps I could simplify life if I had "write raw data" and "read raw data" operations that write and read only the numbers and no metadata.then I could write the file and read it back in any size chunks I please -- and read it with other programs besides. But I don't see them in the menus.
Suggestions?
Ken
Solved!
Go to Solution.

I quote the detailed help from Read from Binary File:
data type sets the type of data the function uses to read from
the binary file. The function interprets the data starting at the current file
position to be count instances of data type.
If the type is an array, string, or cluster containing an array or string, the
function assumes that each instance of that data type contains size information.
If an instance does not include size information, the function misinterprets the
data. If LabVIEW determines that the data does not match the type, it sets data
to the default for the specified type and returns an error.
So I see how I could write data without any array metadata by turning off "prepend array or string size information", but I don't see any way to read it back in such bare form. If I did, I'd have to tell it how long an array to read, and I don't see where to do that. If I could overcome this, I could indeed read in much larger chunks.
I'll try the auto-indexing tunnel anyway. I didn't tell you the whole truth, the 3-D array is actually further sliced up based on metadata that I carry, and ends up as a 4-D array of "runs". But I can do that after the fact instead of trying to do it with shift registers as I'm reading.
Thanks,
Ken

Reading a text file and plot it

Hi Friends,
I have a file, the screen shot is being attached with this mail, what I need is, I want to read this file and plot it in a graph
(In the file in between both values there is a tap specifier)
Could you please provide a sample labview program for that?
Thank you,
Best Regards,
Rathan
Solved!
Go to Solution.
Attachments:
TextFile.GIF ‏5 KB

Your delimiter is "two consecutive tabs". Here's a quick way to graph the file.
Message Edited by altenbach on 07-31-2009 01:03 AM
LabVIEW Champion . Do more with less code and in less time .
Attachments:
GraphTwoColumns.png ‏17 KB
TwoColumnsToXY.vi ‏12 KB

To read & save data from serial port (write to .csv file), and plot data as waveform

Hi,
I've been stuck with this problem for weeks and really need some help. I've been searching a lot but I can't find a proper solution.
I am trying to use Labview to plot and display some parameters of a data acquisition system for a project.
The data acquisition system is connected to a computer via serial connection through COM 1 port.
The data acquisition system sends a serial stream of data (ASCII string with baud rate 19200), as shown below (from HyperTerminal),
01:00:26,933.24,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
01:00:27,931.54,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
The serial data stream consists of a time stamp (as 1st data), followed by 16 individual channel data, separated by comma "," and each stream ending in return "\n". Each value of the data ranges from 1 to 6 digits (value of -5000 to 15000).
I have no problem reading this values correctly using HyperTerminal, but I have problem to plot and display individual parameters of this serial string in Labview. I want to save the entire data in a “.csv” file corresponding to all the 16 channels with time stamp and plot each individual channel parameter in a graph with time on X-axis.
I am relatively new to Labview hope I can learn a lot from your help.
　Thank you

Thank you Dennis.........
Now I had modified the program incorporating1D array for gauge display.....but currently I have a new problem. The program is running updating the file, waveform graph and Gauge very slowly. Approximately once in every 15 seconds 16 data streams are getting updated in a bunch with gauge sticking to the last value rather than indicating real time value.
eg.
12:33:41,-3728.78,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:42,-3730.47,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:43,-3729.81,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:44,-3733.80,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:45,-3733.57,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:46,-3729.27,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:47,-3725.97,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:48,-3730.14,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:49,-3733.20,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:50,-3733.60,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:51,-3732.24,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:52,-3734.51,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:53,-3728.07,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:54,-3732.54,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:55,-3732.24,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
12:33:56,-3731.58,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,
gets updated in a bunch after 15 seconds, rather than in every second. I thought the problem is with while loop but, i realised there is no delay associated with it..... Can u kindly help me in modifing the program so as it can update the waveform, file data and gauge in real time? (I have attached the modified VI file "suitcase new.vi" and the data gathered after executing the same as in "Trail.txt".)
Thank You,
Rahul
Attachments:
trail.txt ‏85 KB
suitcase new.vi ‏44 KB

Read from csv file and plot particular columns

Similar Messages

Maybe you are looking for