Techniques for a synchronous long-life data acquisition

Hello All
I have a digitizer, which needs to run at a constant rate for ever. When some condition occur (reading exceed treshold level), other data processing must take place.
The data processing last for much longer time that the successive sampling, which means that I can not fit data processing in the same thread as the sampling process.
It looks like I need some kind of buffer, which will be filled and keep track of the sampled data, and another thread for data processing and analizying.
Can somebody give me a link, or advice about this kind of problems. What are the techniques for long-life data acquisition.
Thank you in advance.
Pawel

Indeed a challenging application! Here is what has been done successfully in other applications:
1) to separate the DAQ task from other tasks, put the data into a Queue. This creates a second software buffer for your data.
2) for large buffers, so you can extract pre and post trigger data, use two files (a double buffered file buffer) and stream the data to each file successively within your DAQ VI.
3) when you detect the threshold trigger in your DAQ loop, note the block number and pass this as part of the queue
4) your processing loop, can then use the block number to get pre and post trigger blocks for further analysis.
Hope this helps you get started.
Preston Johnson
Preston Johnson
Principal Sales Engineer
Condition Monitoring Systems
Vibration Analyst III - www.vibinst.org, www.mobiusinstitute.com
National Instruments
[email protected]
www.ni.com/mcm
www.ni.com/soundandvibration
www.ni.com/biganalogdata
512-683-5444

Similar Messages

Where are the guidelines for how to interface customized data acquisition equipment to LabVIEW?

I have a need to do some very specialized data acquisition that may require building special hardware. I would still like to use LabVIEW as the front end and perhaps even carry portions of the interface over into the final product. However, I cannot find the documentation. Where is that kept? I have seen references to a number of generic protocols, without any links back to the details, that may be suitable. The data is high bandwidth, but bursty with sampling rates up to 40 MHz and events that last about 100 milliseconds. Can anyone recommend which of the various methods would best suit this application?
Solved!
Go to Solution.

Once you have figured out your communication bus, it is just a matter of how to format your commands and data. You will need to do this regardless of programming language you choose. Formating the data is quite simple in LabVIEW. You just have to write the data to the right thing.
Quickly based on the limited parameters you have given us, I would go with an Ethernet connection. I'm not completely sure if the Ethernet bus will be able to handle your bandwidth, though. If not, then you will need to go to a PCIe or PXIe connection.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions

Timed structure for output in producer consumer data acquisition

Hello LabVIEW community,
I have a bit of a problem. I am writing a program that is primarily for data aquisition but has a few control features as well. I need the program to aquire and write several channels of data at a relitively high speed. The program does this fine ( in the top 3 loops fo the program). I need the program to also send a seires of two output signals in response to a particular condition. When a certain channel registers a value of above a specified number, I need an output channel to write an anologue signal for 225 seconds and then a signal of a differnt value to the same channel untill the condition occurs again (not for several hours).
I put this action in a timed structure inside of a case structure. The case structure and timed structure are inside of a second consumer loop. I have no idea if this is ligitimate. This event takes much longer than any of the other loops which run at 1 or 2 seconds. When I exicute the program with "highlight exexutuion" this longer (case/ timed ) loop never executes. I asssume this has to do with the mismatch in time scales of this loop and the other loops in the program. I also didn't really understand the help information on timed structures, so its possible that I just need better/ different inputs to the timed stuctures.
If anyone can see any obvious problem in the code or suggest a better way to do the output function I would really appreciate the help. My code is attached.
Thanks,
Jo

I can't figure out how to fix your code, but I can point out the things I see wrong with it.
1. You are dequeueing elements from the same queue in the same loop. In one you are not taking the element, and in parallel, you are taking an element. I see that in your bottom two loops. Why? That code won't execute until the queue gets two elements in it. And which dequeue gets it first and which gets it second is an arbitrary race condition. So there is no certainly of order (first 4 vs. last 4 currents in the one loop), or which gets kept and which gets discarded (in the other loop.)
2. You are creating and/or clearing DAQ tasks in every loop iteration. Tasks should be created before a loop, used inside the loop (read or write), then cleared once the loop is done. Anything else is wasteful, time consuming, and could lead you to run out of resources.
3. You are using the STOP function which is pretty much like hitting the abort button on the toolbar. No program should ever stop that way. All loops should exit gracefully.
4. You have a data dependncy between your bottom two loops because of the boolean wire running from one to the other. That bottom loop will only run once (if the value is True), or run forever (if the value is false).
5. Use of the dynamic datatype. You are taking waveforms, converting them to a blue wire, then coercing them into another datatype such as just displaying a scalar in an indicator. A lot of unnecessary conversions there.
6. Your thermometer indicators have a digital display you can make visible. Then you won't need the separate numeric indicator to display the value.
7. For loop that runs only 1 time because of the constant wired to the N terminal. Get rid of the For Loop.
8. Check your spelling. The word "Temperatures" on the graph, and one instance of "distillate" is missing an "l".
Until all of these problems are fixed, it is not worth talking about timing of timed structures. Some of these problems were discussed in your other thread. http://forums.ni.com/t5/LabVIEW/producer-consumer-missing-channels/m-p/3159757#M911255 But now it seems like you've made things even more complicated without fixing the basic problems.

Data acquisition in a state machine

I've recently learned how to implement state machines, but I have a question that I haven't been able to find an aswer to.
Question:
How do I record uninterupted data (presently i'm using the DAQ assistant to record a single channel at 10kS/s at 1kS intervals) without interfering with my state machine?
Background:
I have a time sensitive test setup (measuring pressure in a rocket chamber), and I'm using labview to both record a single pressure transducer as well as operate the valves (O2, N2, igniter, etc). I'm using a state machine Labview architecture (just learned how). Presently, the DAQ assistant sits in the "Idle" state, and timers (valves need to be turned on at specific times) and manual buttons control when the other states are activated. I have a 10ms timer controlling how often each state is run, and this is the root of my concern.
Main concerns:
I'm worried that:
1) I'm forced to change my data recording to 10ms intervals to match the 10ms timer, and
2) The 10ms timer will interrupt my data recording when it's in another state.
It's very important that the pressure data is recorded without interruption but that I have high time resolution in controlling my valves as well.
Thoughts:
1) Maybe I put the 10ms timer in the "Idle" state only, and this way the other states won't interrupt the data acqusition
2) Maybe there is a way to completely isolate the state machine for controlling the valves and the data acqusition. I'm not sure how to do this.
3) People on the forums have been talking about Queuing their data acqusition, but I'm not sure how this would benefit my situation.
Thanks to everyone for their feedback!

The data acquisition and state machine timer do not have to be equal, you should however have the same data write rate as you have data read rate, otherwise over the course of long running programs you can build up a large buffer of data to be written using a lot of memory, or lose data.
Your data acquisition should be continuous, the most common way of doing this is every time you receive data write it to a queue, then when you are in other states except your data write stage you simply build up data in the queue that is to be written when you enter the write state again.
Think of it as traffic reaching a set of lights:
The traffic represents the data acquisition, there is a constant flow of it always coming through, the state of your state machine is the lights, green is the data write state clearing the queue, red is when in any other state, letting the traffic build up behind the lights.
It is just important that the light is green long enough to let the traffic not build up too much, otherwise you have a jam.
EDIT: For the basics of how queues work, look at the Queue basics example from the example finder.

Copying data to an external server during data acquisition

I need to copy out data to an external server for backup during an extended data acquisition session (days long). What is the safest data format for this operation that won't corrupt the primary data file? *.lvm, *.tdm, *.tdms?
Thanks

Hi Velveeta,
I would think that you would rather save to multiple files and then back them up on the server rather than try to make a copy of a file in the middle of its creation; as this would be more likely to cause corruption issues. If you are very concerned about this, I'd recommend breaking up the data to a couple of files and then backing them up. So, to answer your questions:
1. You can set up the express VI Write to Measurement File to create a new file every X data points. I'm unsure of how you are planning to programmatically copy this to a back up disk.
2. Don't copy a file in the middle of its creation (for example, if you were creating one file and tried to back it up in the middle of your acquisition).
3. The file format shouldn't matter for this functionality in particular.
Cheers,
Marti C
Applications Engineer
National Instruments
NI Medical

I2C interface (Sensor Data Acquisition) LabVIEW

Hi all!
Hope you are doing great!
Well I have a question which is more about asking all you for an idea!
The Situation:
I have a circuit board which has an On-Off Valve, Digital pressure sensors (manufacturer AMD) and Humidity/Temperature Sensors (make- IST Hygrosens). On the board all the sensors communicate as I2C slave devices and all the data from the sensors is read into an I2C --> USB adapter chip which further connects to the PC via normal USB cable.
Additional to this board, There is a Relay circuit with a simple 1-pole relay which controls an on-off valve on the above Circuit board. This valve is controlled totally separate via a coaxial cable from the relay directly to the Valve. But the relay board has a I2C interface and it also acts as a slave device. The relay board has the same I2C --> USB adapter chip.
Both the Relay board and Sensor board connect via USB to the PC which I suppose is the Master device.
The software code written for this arrangement and Sensor data acquisition is too old and there are a lot of problems coming. I have almost given up troubleshooting..
I now want to translate this automation system onto LabVIEW. I searched the NI website where there is a DAC card called - USB8451 which supports I2C interface... I am a beginner in LabVIEW and cant really make sense out of how should I go about implementing this system on LabVIEW..
If you guys can please help me out to atleast start (like what all hardware I would need etc..), to have a clear picture, it would be great help!!
Looking forward to your inputs and Thank you so much in advance!
Cheers!
Pramit

NI provides a LabVIEW API for the USB8451. If you use the USB8451, you would use the provided API to write a program that controls the USB8451 and you would do all of the I2C communication in your program. This would mean using functions / SubVI's to connect to the USB8451 and then perform I2C operations through it.
If you use USB already on your device, then you would probably use NI-VISA as the driver and have to get / write your own API to talk to the specific device. The manufacturer may have a LabVIEW (or other) API available for talking to the device that you could get. If not, then you would have to understand the details of how to communicate with the device and then write an API using NI-VISA serial functions. This would mean making NI-VISA be the assigned driver for the device and then using VISA Serial functions / SubVI's to send the messages and receive the responses.

Multi Channel USB Data Acquisition

I am using the following VI for arduino HC-06 bluettoth data acquisition. I would like to acquire multicahnnel data
Could you suggest me how to acquire multichannel data from USB? To do this, in what way i have to program the arduino?

Thank you very much for your suggestions. its working.
But if i give input to channel 2 or 3 or 4, they are not responding. only channel 1 is responding irrespective of input pin. what could be the reason for this?
earlier, i was acquiring single channel data at 57600 baudrate, i was getting pure sine wave signal upto 90Hz, after that it was started distorting.
Now, it started distoring above 20Hz, i think i need to increase the baudrate to 230400 to get 4 channel undistorted signal upto 100Hz,
but arduino supports upto 115200 only, what shall i do??

Data Acquisition Stops Randomly

We are using a timed loop to read, display, and write data from a CompactDAQ device. Normally, the data reads, displays, and writes correctly, however there have been several occurrences where the write function failed without notice. When we went back to open up the data after a test, we found that the data ended early, ranging from 30 seconds in to a minute into the test. Has anyone encountered an error like this? Thank you for your help.

Thank you for the quick response. I have attached our VI for reference. To answer your questions:
Data is read with the DAQ assistant. Write to measurement file is used for writing the data. Both are within the same timed loop.
Errors are propogated across the timed loop and through the DAQ read and write functions.
Not sure if there were errors reported. How can I tell?
The data I opened after running the test was logged into a .lvm file. I have some code in the VI to automatically generate a timestamped filename, however it does not seem to work so I have been naming files manually in the settings of the "write to measurement file" function.
I apologize for the vague first post. Hopefully the VI sheds some light on the problem. Thank you for your help.
Attachments:
Data Acquisition - ANelson.vi ‏149 KB

Aeronautic Data Acquisition

Dear Brother ?
I have job for reseach asisstant to davelop Data Acquisition for experiment aeronautic Control Pilot data acquisition system.
I use PCI-6014 B-series Multifunction DAQ connect with BNC-2120 and 3 sensor potentiometer, each sensor connect with Aileron,elevator and rudder. +DC Power Supply for input Voltage
from control Stick (pilot) could be move aileron and elevator. meanwhile from pedals could be move Rudder.
DAQ get from moving sensors.
my question how to design Visualization program with LABVIEW ?
note:
when I open NI example finder and click example always Message " "An error occured-The NI Service Locator is Running".
how solution? ..?..? for my problem.
thanks.
Solved!
Go to Solution.

Thank's for you answer Lauren L
I still have problem with my DAQ programming can't sucess when running in my block diagram
error 00284 occured at DAQmx Read .
please see picture in attach file
how is solution? may somebody can help me?
Thanks
Note:
I used Labview 8.20 software B-series multifunction DAQ-PCI-6014 connect with BNC-2120, from Ai5 bnc cable connect Sensor Potentiometer and DC-Power Supply
Attachments:
error.png ‏68 KB

Top level application structure with parallel data acquisition

Hey all LabVIEW-forum members!
I have some experience programing LabVIEW, but mostly concerning smaller for the simplest form of data acquisition. Now I need to develop a larger application that might be continuously run for several weeksin a row and for that reason I woiuld like to make sure the application structure is stable! I turn to you in hope of some input regarding the most suitable approach to my problem or some pointing in a good direction.
-The project involves making hardware communication with a few different hardware interfaces, such as GPIB and NI’s DAQ-card (analog and digital reads/writes on PCI-6221).
-Save acquired data to disk.
-The communication and data save are based on condition of elapsed time (for example there could be a need to make parallel analog measurements med DAQ-card every 5th minute but GPIB-measurements every 30th second).
-At the mean time I would have to regularly the status of the test object and update the user interface. (However, while in test the user won’t be allowed to interact with the application other than to monitor the UI or pause the test).
The project isn’t really time critical (in case I need to make one type of data-acquisition while another one is already initiated, they can be carried out in sequence).
I’ve started with programmatically control the user interface by using the “Top Level Application Using Events”-template and produced code that will let the user set up a new test or load an existing test. All the data concerning the status and configuration of the test I’ve saved in two clusters that can be passed along to different events.
I’ve reserved one subdiagram in the event structure for the actual execution of the test. It is the structure of this subdiagram I am a bit confused about…
Could I use another while-loop/event-structure inside this subdiagram and use user-registered events for the different data acquisition? Or is there some type of state-machine that is better suited to monitor time elapsed and can make a transition to a suitable state when it’s time to make any of the hardware measurement required? It would be nice to avoid using polling loops but maybe that is best way to go?
I’ve checked the “Continuously Generate Occurrences.vi”-example and that might be a way to go? What seems to be good with this approach is that I could let the different parallel hardware acquisitions actually be parallel, right?
I am really confused right now, hope someone has some helpful inputs!
Regards
Oscar

This post here can be proved as a great starting point for your application.
I am not allergic to Kudos, in fact I love Kudos.
Make your LabVIEW experience more CONVENIENT.

MultiThread for data acquisition (basic problem)

Hi there,
I want to create a data acquisition VI that perform acquiring data in an independant thread, and when the data is ready, displaying it in a graph. In another word, one thread for acquisition and another one for UI events (button to start / stop acquisition, display cursor...). Data acquisition is higher priority than displaying data. Any VI concrete sample for multithread ?? Thanks in advance.

> Hi there,
> I want to create a data acquisition VI that perform acquiring data in
> an independant thread, and when the data is ready, displaying it in a
> graph. In another word, one thread for acquisition and another one for
> UI events (button to start / stop acquisition, display cursor...).
> Data acquisition is higher priority than displaying data. Any VI
> concrete sample for multithread ?? Thanks in advance.
By default, the UI runs in the UI thread. The execution of the VI runs
in the Standard execution system, which has its own thread. By default,
these threads are at the same priority. If you want your DAQ to run at
a higher priority, you can place your DAQ code in a subVI. Then set the
subVI to run in any execution system you like at a higher
priority.
Call the subVI in parallel with the rest of your UI loop, and determine
how you want the DAQ data to get to the UI. You can use a global, a
queue, etc. Don't overlook simply using the terminal or local for the
graph. All of these other methods have their advantages, but they also
add overhead. At some point, you will need to copy the data from the
DAQ code to the UI code. This is all that writing to a terminal or
local does as long as synchronous display is not turned on.
Greg McKaskle

"Advance Synchronization Techniques for Data Acqusition is not simple.

I want to use a 6534 DIO to drive my instrument and a synchronized 6502E to do AI from the instrument. The developer zone tutorial "Advanced Synchronization Techniques for Data Acquisition" suggests some of what is necessary to change from using AO as the master to using some other operation. After installing an RTSI cable, "Synchronized AI,AO,DI,DO,CTR.vi" runs as-is. Has anyone successfully converted the "Synchronized AI,AO,DI,DO,CTR.vi" or something similar to having some other master than AO? I want to use DO as master, and have moved the start vi's to the initialization cases except that for DO, have removed DI completely,and have tried to get DO to transmit REC1 on RTSI1 and AI and AO to clock o
n the RTSI1 signal. The modified VI runs to completion, but I get no input at AI.

Hi Bhanu,
I ran the program SAP_FACTVIEWS_RECREATE.It is also not fetching any data.In the monitor, the extraction shows yellow status and the Processing says NO DATA.I have not given any selections at infopackage level.
Please let me know what do i do.
Thanks in advance.
Regards,
Venkat

Choice of design pattern for data acquisition system

Hello all
I have a trouble about selecting the suitable design pattern / architecture for a data acquisition system.
Here is the details of the desired system:
There is data acquisition hardware and I need to use it by observing parameters on User interface.
the data acquisiton period, channel list to scan should be chosen on User interface. Besides, there are many user interface interactions. e.g. if user selects a channel to add scanlist, then I need to enable and make visible some other parts on user interface.
When user completes the channel selection, then he will press the button to start data acquisition. Then I also need to show the scanned values on a graph in real time and log them in txt file.
I know that I cannot use producer consumer pattern here. because the data acquisition loop should wait for parameters to scan channels. and it works in a given period by user. so the user interface loop performs higher rate then consumer loop (data acquisition loop). it means queue will be bigger bigger. if I use notifier it will loss some data come from user interface.
is there any idea about that ? is there any suitable design pattern for this case ?
Thanks in advance
best regards
Veli BAYAR
Embedded Systems Software and Hardware Engineer
"You live in a graphical world. Why not program in one?"
Solved!
Go to Solution.

johnsold wrote:
Veli,
I recommend the Producer/Consumer model with some modifications.
You might need three loops. I cannot tell for sure from your brief description.
The User Interface loop responds to the user inputs for configuration and start/stop of acquisition. The parameters and commands are passed to the Data Acquisition loop via a queue. In this loop is a state machine which has Idle, Configuration, Acquisition, and Shutdown states (and perhaps others). The data is sent to the Processing loop via a different queue. The Processing loop performs any data processing, displays the data to the user, and saves it to file. A notifier can be used to send the Stop or shutdown command from the User Interface loop to the other loops. If the amount of processing is minimal and the file write times are not too long, the Processing loop functions might be able to occur in the Timeout case of the UI loop Event structure. This simplifies things somewhat but is not as flexible when changes need to be made.
I am not sure that a Design Pattern for this exact setup exists but it is basically a combination of the Producer/Consumer (Events) and Producer/Consumer (Data) Design Patterns.
Lynn
Check out this thread: http://forums.ni.com/t5/LabVIEW/Multiple-poll-case-structures-to-event-help/td-p/2551309
There are discussions there about a 3-loop architecture that may help you.
Jeff
Jeffrey Zola

Real-time data acquisition for HR datasources

Dear Experts,
I have a couple of questions on real-time data acquisition...
a) Can you tell me if any standard HR datasources support real-time data acquisition?
b) Can we apply real-time data acquisition for generic datasources? If yes, is there any difference in the process compared to working with business content datasources when using real-time data acquisition?
Hope you can provide some answers...as always, points will be awarded for answers that effectively address the questions.
Thanks a bunch.
K

Hi Karthik,
a)The decision to go for SAP remote cube depends on the volume of data and the frequency of queries. It is advicible not to use the SAP remote cube if the data volume is high and also, the query frequency is high. Which module in HR are yuo implementing? In HR, the data volume is generally high. So, if you go for loading the data from R/3 every 3 hours, you are asfe as long as the loading volumes are less. For example, for implementing Time management, I would not advice frequent loads as it is time consuming process. So, make a decision based on the above mentioned lines. If the data volume is not high, I would prefer the SAP ermote cube as it will reduce managing the loads.
b)I mentioned FM extractor just for the sake of control of data selection. You can even go for view/table extractor.
Hope this helps.
Thanks and Regards
Subray Hegde

Data Acquisition for Position Transducer

I just purchased an Analog Position Transducer from SpaceAge Control, and was wondering if anyone had any code to gather the data. I must admit i'm lost when it comes to this, I thought the transducer would come like an instron where you just plug it into the computer. I have a six pronged plug that exits the device. I have a couple TI data acquisition boards at my disposal. I am hoping someone has used a similar product or has some advice. Thank You
Keith

We have lots of code to gather data in all sorts of ways. Building measurement applications is what we do. Since you are lost when it comes to this, you would do well to hire a guide in the form of an Alliance Member company. We do this stuff every day.
Your TI DAQ boards may do the trick, but you would have to show someone the specs for them. Also, you need to think about what kind of speed you require and where you want the position data to go. Table? Plot? Database? A good LabVIEW developer can set you up with some code that does exactly what you want.
Daniel L. Press
PrimeTest Corp.
www.primetest.com

Techniques for a synchronous long-life data acquisition

Similar Messages

Maybe you are looking for