Best way to read data sources in parallel

Hi,
I'm looking for conceptual help as I start a project. I am trying to figure out the best way to get data from several sources at different timings and deliver them to a main vi.
I have 4 systems, which each work well on their own (OK, one doesn't work yet, but let's assume that can be fixed
One system reads from 2 pH meters on serial ports. The meters are slow to respond, so it takes about 2 minutes to read 4 channels of data. I save these data to a file every 10 minutes
One system reads from a CO2 meter on the USB port. It reads the data every second, and does some averaging. Every 2-10 minutes, it saves the average to a file and then sends a command to the parallel port to switch the input to the meter.
The third system reads from 6 valves, each on a serial port. These also take time, probably several minutes to poll all 6. These data will also be saved to a file.
The 4th system reads a bank of temperature probes on the USB port. These get polled every few seconds and saved to a file every few minutes.
Now that these individual routines are working, I am trying to create a front end that will display all the data in one place and allow me to set all the parameters from a single place. I would also like the possibility of using the data from one source at another place (for instance, having the output of the temperature probes sent to the pH meters to adjust their calibration). At this point, I get confused as to the best way to proceed.
It seems like if I just want to read the data from each source, I could simply put all 4 routines together in a single vi (oh, what a mess that would be to read). Maybe I should start this way?
However, if I want to have any communication between the different data sources, it seems like I will either need to use queues or VI server. I sort of envision a vi that lets me configure the various ports and the file operations and then can turn on monitoring of any or all of the various inputs. Each of them will do their thing at their own time and the main routine will simply display whatever data they deliver whenever they have new data. Fortunately, nothing is particularly time-critical, nor does it need to run fast.
My questions: Am I correct in how I'm thinking about getting this to work?   Is there a clear choice between queues or vi server? I've looked at several examples of each, but without having done something like this before, it is hard for me to tell which is better.
Thanks for any suggestions.
mike

Hi Mike,
I think that you are on the right track with your thinking process. You might be able to implement this using queues. I'm not exactly sure how you would do it with VI server since it is just a set of functions that allows you to dynamically control front panel objects, VIs, and the LabVIEW environment. However, there are some great resources available with using queues for this type of application. I'm including the link to another discussion forum that had a very similar question to yours. There is a good example of using queues within this forum post. Also, there is a great example in NI Developer Zone about using queues and some other good ones in the NI Example Finder (just search 'queues' and you should get a few results). I hope this helps!
Carla
National Instruments
Applications Engineer

Similar Messages

  • How is the best way to read data from an iphone if you lost your itunes data after a crash?

    How is the best way to read data from an iphone if you lost your itunes data after a crash?

    How is the best way to read data from an iphone if you lost your itunes data after a crash?

  • What is the best way to read and manipulate large data in excel files and show them in Sharepoint

    Hi ,
    I have a large excel file that has 700,000 records in it. The excel file has a few columns that change every day.
    What is the best way to read the data form the excel file in fastest and most efficient way.
    2 nd Problem,
    I have one excel file that has many rows each row contain some data that has certain keywords.
    What I want is  to segregate the data of rows into respective sheets(tabs ) in the workbook.
    for example in rows have following data 
    1. Alfa
    2beta
    3 gama
    4beta
    5gama
    6gama
    7alfa
    in excel
    I want there to be 3 tabs now with each of the key words alfa beta and gamma.

    Hi,
    I don't really see any better options for SharePoint. SharePoint use other production called 'Office Web App' to allow users to view/edit Microsoft Office documents (word, excel etc.). But the web version of excel doesn't support that much records as well
    as there's size limitations (probably the default max size is 10MB).
    Regarding second problem, I think you need some custom solutions (like a SharePoint timer job/webpart ) to read and present data.
    However, if you can reduce the excel file records to something near 16k (which is supported rows in web version of excel) then you can use SharePoint Excel service to refresh data automatically in the excel file in SharePoint from some external sources.
    Thanks,
    Sohel Rana
    http://ranaictiu-technicalblog.blogspot.com

  • What's the best way to read JSON data?

    Hi all;
    What is the best way to read in JSON data? And is the best way to use it once read in to turn it into XML and apply XPath?
    thanks - dave

    jtahlborn wrote:
    without having a better understanding of what your definition of "use it" is, this question is essentially unanswerable. Jackson is a fairly popular library for translating json to/from java objects. the json website provides a very basic library for parsing to/from xml. which one is the "best" depends on what you want to do with it.Good point. We have a reporting product ([www.windward.net|http://www.windward.net]) and we've had a number of people ask us for JSON support. But how complex the data is and what they want to pull is all over the place. The one thing that's commin is they generally want to pull down the JSON data, and then put specific items from that in the report.
    XML/XPath struck me as a good way to do this for a couple of reasons. First it seems to map well to the JSON data layout. Second it provides a known query language. Third, we have a really good XPath wizard and we could then use it for JSON also.
    ??? - thanks - dave

  • Best way to log data

    I have an application that receives data from a source, and I would
    like to create and maintain a log file with this data. Small bits of
    data will be received at least every minute, and often once every
    second. The user can also view the logged data at any time.
    Lastly, each entry is timestamped, and I would like to be able to
    remove log entries that are x hour/days old.
    What I would like advice on is what is the bext way to accomplish this?
    Clearly a bad way to do this is every time I get data, read the log in,
    remove old entries, rewrite all the remaining entries, and finally
    append the new entry. This uses way too many file operations.
    I have briefly looked at the java.util.logging stuff, but it appears that
    the purpose of that is more for code debugging/etc. It also doesn't
    appear that there are methods to read the logs. So, this does not
    look like an "easy way out".
    I know how to append to a file, but I am not sure if I should leave
    the file open during execution of the program and simply flush
    the stream/buffer, or open and close the file everytime I write.
    The latter is clearly more expensive, but safer.
    I don't know how to delete data from the beginning of a file.
    To summarize, what is the best way to read and write data to
    a file such that:
    1) the data is appended to the end of the file (easy)
    2) the data is removed from the beginning of the file
    3) the data that is written is not lost of the program crashes/etc.
    4) the whole logging business is not too expensive
    Any suggestions?
    Thanks,
    Chuck.

    Here's what we did at my last job:
    Incoming messages are logged to disk via FileOutputStream (maybe with BufferedOutputStream encasing that, I don't remember) this outputstream is not closed until the applet is stopped. System.out is also reassigned to this fileoutputstream to capture all messages.
    At the same time, there is an InputStream reading from that file. As data is read, we stick the text into a TextArea.
    When the TextArea size hits some certain amount (maybe 20,000 characters) we'd remove ... maybe 5000 from the top. Log stays the same tho
    When the day rolls over, we switch log files
    You're questions:
    1) the data is appended to the end of the file (easy)
    Yes. But if you're not using streams, you should check that out.
    2) the data is removed from the beginning of the file
    That's pretty easy. Open the file for reading, open a new file for writing. Read the first n bytes but don't write them. Then write the rest
    3) the data that is written is not lost of the program crashes/etc.
    Data should be fine if it's logged to disk
    4) the whole logging business is not too expensive
    Saving to disk isn't very expensive at all. In fact, normally writing to disk is slow enough that stuff just gets put into a buffer somewhere to get sent to the disk and the processor goes on with the program.

  • Best way to pass data sets to another program

    Hey
    I want to connect another (maths) program with my java application. Therefore I need to paste data (some kind of tab separated table) to this program.
    I try now to save these data in a separate newly generated file and to pass a command with Java's Runtime.exec() method to this program to read these data. Is this a good idea or might there be better ways?
    If I do so, is there a way in Java to generate some kind of a "temporary" file which will be deleted automatically after usage or is this nothing else than to save it in a common file and delete it afterwards. What's the best way to pass data generally?

    Well, the connection will not be over a network, so I'd rather think it's not a Socket or RMI problem (unless someone convinces me).
    Yes it's very external, it's a C or C++ written program, I don't have any source codes. So far I generated a file for Input command and data, I passed that on to the maths program and returned the output into another file.
    Now I would like to separate the output and like to obtain some tables and graphical things like charts at the output. Do I have to generate three different types of outputfiles? How to store some graphics e.g. some distributions. I thought even of generating a database. I never thought about XML, I don't know if this works for that kind of problem?!

  • Best Way to Load Data in Hash Partition

    Hi,
    I have partitioning by Hash on a Large Table of 5 TB. We have to load Data say more than 500GB daily on that table from ETL.
    What is the best way to Load data into that Big Table which has hash Partition .
    Regards
    Sahil Soni

    Do you have any specific requirements to match records to lookup tables or it just a straight load - that is an insert?
    Do you have any specific performance requirements?
    The easiest and fastest way to load data into Oracle is via external file and parallel query/parallel insert. Remember that parallel DML is not enabled by default and you have to do so via alter session command. You can leverage multiple CPU cores and direct path operation to perform the load.
    Assuming your database is on a linux/unix server - you could NFS load the file if it is on a remote system, but then you will be most likely limited by network transfer speed.

  • Best way to import data to multiple tables in oracle d.b from sql server

    HI All am newbie to Oracle,
    What is the Best way to import data to multiple tables in Oracle Data base from sql server?
    1)linked server?
    2)ssis ?
    If possible share me the query to done this task using Linked server?
    Regards,
    KoteRavindra.

    check:
    http://www.mssqltips.com/sqlservertip/2011/export-sql-server-data-to-oracle-using-ssis/
          koteravindra     
    Handle:      koteravindra 
    Status Level:      Newbie
    Registered:      Jan 9, 2013
    Total Posts:      4
    Total Questions:      3 (3 unresolved)
    why so many unresolved questions? Remember to close your threads marking them as answered.

  • What could be the best way to Export data from 11.5.8 instance to 12.1.2?

    Hi All
    What could be the best way to Export data from 11.5.8 instance to 12.1.2?
    Release: 11.5.8
    OS: Oracle Solaris on SPARC (32-bit) verison 9
    DB: 9.2.0.1
    Thanks in advance

    What kind of data you are looking to move?
    Database export/import is only supported for full database export/import and the application release should be the same on the source/target nodes.
    You can move the setup using iSetup or FNDLOAD.
    Thanks,
    Hussein

  • What's the best way for reading this binary file?

    I've written a program that acquires data from a DAQmx card and writes it on a binary file (attached file and picture). The data that I'm acquiring comes from 8 channels, at 2.5MS/s for, at least, 5 seconds. What's the best way of reading this binary file, knowing that:
    -I'll need it also on graphics (only after acquiring)
    -I also need to see these values and use them later in Matlab.
    I've tried the "Array to Spreadsheet String", but LabView goes out of memory (even if I don't use all of the 8 channels, but only 1).
    LabView 8.6
    Solved!
    Go to Solution.
    Attachments:
    AcquireWrite02.vi ‏15 KB
    myvi.jpg ‏55 KB

    But my real problem, at least now, is how can I divide the information to get not only one graphic but eight?
    I can read the file, but I get this (with only two channels):
    So what I tried was, using a for loop, saving 250 elements in different arrays and then writing it to the .txt file. But it doesn't come right... I used 250 because that's what I got from the graphic: at each 250 points it plots the other channel.
    Am I missing something here? How should I treat the information coming from the binary file, if not the way I'm doing?
    (attached are the .vi files I'm using to save in the .txt format)
    (EDITED. I just saw that I was dividing my graph's data in 4 just before plotting it... so It isn't 250 but 1000 elements for each channel... Still, the problem has not been solved)
    Message Edited by Danigno on 11-17-2008 08:47 AM
    Attachments:
    mygraph.jpg ‏280 KB
    Read Binary File and Save as txt - 2 channels - with SetFilePosition.vi ‏14 KB
    Read Binary File and Save as txt - with SetFilePosition_b_save2files_with_array.vi ‏14 KB

  • Best ways to view data, total records of an application table ie VBAK

    Hi all,
    What is the best way to view data of an application table in the source system?
    I know about SE16....but are there other ways to know details ie the total no of records and different field information about a
    application table ie VBAK in source R3?
    Also, using SE16 when i checked for VBAK and clicked on the "number of enteries" then it showed 0...however
    when i directly checked from the sqlplus then i found about 5000 records in there in VBAK. I am not sure why
    via SE16 it showed 0. Does anybody have any idea what i missed here?
    Thanks...will give points for ur input.
    ak

    I tried "number of enteried" on se16 and it shows 0 enteries without any selection criterion...i cheked by putting relevant time range as well but it shows 0...
    As i told that when i checked VBAK separately via logging to database directly then i did find 5678 rows there.
    Please note that this is a new demo version....so i thought that i first need to activate the table which i did using tcode SE11. Now the VBAK table is active but still via SE16 shows 0 nuber of enteris....
    Can anybody please advise here..
    Thx
    ak

  • Best way to export data with r.t. prompts and have dense dim mbrs on rows?

    Hi All-
    What is the best way to export data with Run time prompts out of Essbase?
    One thought was to use Business Rules with run time variables and DATAEXPORT command, but I came across at least one limitation where I cannot have months (part of dense Time Periods dimension) on rows.
    I have only two dense dimensions: Accounts and Time Periods and I need both of these on rows. This would come handy when user enter Start and End month and year for data to be exported e.g. If start period is Feb 2010 and end is Jan 2011, I get data for all months in 2010 and 2011.
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000",14202.24,14341.62,14560,13557.54,11711.92,10261.58,12540.31,15307.83,16232.88,17054.62,18121.76,18236
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000",19241,21372.84,21008.4,18952.75,23442.13,19938.18,22689.61,23729.29,22807.48,23365,23915.3,24253
    "CORP1","0173","FY11","Working","Budget","Local","HSP_InputValue","404000",21364,22970.37,23186,27302,25144.38,27847.91,27632.11,29007.39,24749.42,27183.39,26599,27112.79
    where ideally I would need to get the following:
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Feb",14341.62
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Mar",14560
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Apr",13557.54
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","May",11711.92
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Jun",10261.58
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Jul",12540.31
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Aug",15307.83
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Sep",16232.88
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Oct",17054.62
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Nov",18121.76
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","402000","Dec",18236
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Feb",21372.84
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Mar",21008.4,
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Apr",18952.75
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","May",23442.13
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Jun",19938.18
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Jul",22689.61
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Aug",23729.29
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Sep",22807.48
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Oct",23365
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Nov",23915.3
    "CORP1","0173","FY10","Working","Budget","Local","HSP_InputValue","403000","Dec",24253
    "CORP1","0173","FY11","Working","Budget","Local","HSP_InputValue","404000","Jan",21364
    Thank you in advance for any tips.

    Have a read of the following post :- export data to sql
    It may give you a further option.
    Cheers
    John
    http://john-goodwin.blogspot.com/

  • Best way to read chars from InputStream

    Hope this is not a too newbie question.
    Suppose I have an unbuffered InputStream inputStream, what is the best way to read chars from it (in terms of performance)?
    Reader reader = new BufferedReader(new InputStreamReader(inputStream));
    reader.read()
    or
    Read reader = new InputStreamReader(new BufferedInputStream(inputStream))
    reader.read()
    Is there a difference between the two and if so, which one is better?
    thanks.

    If you are reading using a buffer of your own, then adding a buffer for binary data is a bad idea.
    However for text, using a BufferedInputStream could be better as it reduces calls to the OS.
    If it really matters, I suggest you do a simple performance test which runs for at least a few seconds to see what the difference is. (You should runt he test mroe than once)
    Edited by: Peter__Lawrey on 20-Feb-2009 21:37

  • Best way to read from a file

    What would be the best way to read from a file. Which classes do I need to use?
    I have to write a program, which reads data from a comma separated flat file, parse it and after inserting some busineess logic insert into a databse .
    I will have to read the data line by line.
    Any help????

    I would use:
         public void readData()
              try
                   data = new String[this.countRows("comp.txt")][];
                   BufferedReader br = new BufferedReader(new FileReader("comp.txt"));
                   for(int x = 0; x < data.length; x++)
                        StringTokenizer temp = new StringTokenizer(br.readLine(), "?");
                        data[x] = new String[temp.countTokens()];
                        for(int y = 0; y < data[x].length; y++)
                             data[x][y] = temp.nextToken();
              catch(Exception e)
                   System.out.println(e.toString());
         private int countRows(String f)
              int t = 0;
              try
                   BufferedReader brCountRows = new BufferedReader(new FileReader(f));
                   while(brCountRows.readLine() != null)
                        t++;
              catch(Exception e)
                   System.out.println(e.toString());
                   return t;
              return t;
         }It works deliciously!

  • Best way to mark data that has changed when it is edited in  a spark datagrid?

    I am not sure if this is the best way to mark data that has changed (isModifiedClientSide ) when it is edited in  a spark datagrid:
    <GridColumn  width="140" headerText="Margin "dataField="margin" editable="false"  editable.editMode="true"  >
                        <itemRenderer>
                            <fx:Component>
                                <DefaultGridItemRenderer textAlign="right"   background="true" backgroundColor="#ffffff"  alpha="1.0" color="#000000" >
                                    <focusOut>
                                        <![CDATA[
                                        this.data.isModifiedClientSide = 1;
                                        ]]>
                                    </focusOut>
                                </DefaultGridItemRenderer>
                            </fx:Component>
                        </itemRenderer>
                    </GridColumn>

    I tried this but the value commit doesnt't get fired bro:
        <GridColumn  width="140" headerText="Margin (disabled)" headerText.editMode="Margin (editing)" dataField="margin" editable="false"  editable.editMode="true"  >
                        <itemEditor>
                            <fx:Component>
                                <DefaultGridItemEditor>
                                    <valueCommit>
                                        <![CDATA[
                                        this.data.isModifiedClientSide = 1;
                                        ]]>
                                    </valueCommit>
                                </DefaultGridItemEditor>
                            </fx:Component>
                        </itemEditor>
                    </GridColumn>
    what do you think?

Maybe you are looking for