2 producers/ 1 consumer for a Producer/Consumer Architecture

I am trying to aquire two seperate tasks (one with analog and one with digital signals). The digital signal needs to be aquired at a much higher rate (10 times higher). The data from both of these aquisitions is required for writing to the same file and doing calculations. I was wondering if it is possible to use 2 Producer loops and 1 Consumer loop? I would need to use an enqueue in each producer loop and then dequeue them both in the consumer loop. Will this work?
Thanks.
--Robert

knapkerd wrote:
I am trying to aquire two separate tasks (one with analog and one with digital signals). The digital signal needs to be aquired at a much higher rate (10 times higher). The data from both of these aquisitions is required for writing to the same file and doing calculations. I was wondering if it is possible to use 2 Producer loops and 1 Consumer loop? I would need to use an enqueue in each producer loop and then dequeue them both in the consumer loop. Will this work?
Thanks.
--Robert
Certainly. You can pass the reference wire for the queue you obtained into 2 different producer loops where you enqueue your data. The same wire will pass into the consumer loop where you dequeue your data. Of course with this method, you have no control over which loop puts its data into the queue first.
It really comes down to what you are going to do with the data once you get it into the consumer loop. Do you just want to write the data as you get it? What do you do with the calculations, if you get 8 or 9 pieces of data from the one producer, then get a piece from the other producer, then get 11-12 pieces from the first producer, do you just hold on to the previous values from the older slower loop?
Do you want to do this with 2 different queues? If so, you may need to put some timeout functions on your dequeue functions in your consumer loop otherwise it will only run as fast as it gets the data from the slower queue.
Message Edited by Ravens Fan on 05-07-2008 11:38 PM

Similar Messages

AJAX producer/consumer and JAVA producer/consumer in same project

I've already successfully integrated and customized the code for both an AJAX and JAVA producer/consumer client in JMS (using ActiveMQ as the JMS broker and Tomcat as the webapp container). While both have been working well in separate projects, unfortunately, I'm having a bit of trouble trying to integrate them for the purpose of running an AJAX message listener and JAVA message listener in the same project.
What I'm trying to do is have JAVA act as the failover for when the AJAX JMS consumer is not listening (i.e. when the browser is closed). When a browser client opens, the (constantly running) JAVA client can ignore the messages... but... when no browser client is initialized, the JAVA client should respond to the messages on behalf of the AJAX client.
I've tried using a ServletContextListener pattern, which I was hoping would "automagically" launch the JAVA listener via its main method, whenever the Servlet gets initialized (i.e. it would auto-start when Tomcat gets restarted, or, anytime the Servlet gets redeployed) but it seems Tomcat is not thread-safe and this might be causing some problems.
In any case, I can't seem to get the JAVA client to reliably start running and stay running for the lifetime of my AJAX client (webapp .war), and without including them in the same project and same J2EE container sharing the same ServletContext, I don't know how its possible to ensure they are running and starting and stopping at the same time.
Is there perhaps a better way to accomplish this?
Edited by: bcmoney on May 17, 2010 1:25 PM

I've already successfully integrated and customized the code for both an AJAX and JAVA producer/consumer client in JMS (using ActiveMQ as the JMS broker and Tomcat as the webapp container). While both have been working well in separate projects, unfortunately, I'm having a bit of trouble trying to integrate them for the purpose of running an AJAX message listener and JAVA message listener in the same project.
What I'm trying to do is have JAVA act as the failover for when the AJAX JMS consumer is not listening (i.e. when the browser is closed). When a browser client opens, the (constantly running) JAVA client can ignore the messages... but... when no browser client is initialized, the JAVA client should respond to the messages on behalf of the AJAX client.
I've tried using a ServletContextListener pattern, which I was hoping would "automagically" launch the JAVA listener via its main method, whenever the Servlet gets initialized (i.e. it would auto-start when Tomcat gets restarted, or, anytime the Servlet gets redeployed) but it seems Tomcat is not thread-safe and this might be causing some problems.
In any case, I can't seem to get the JAVA client to reliably start running and stay running for the lifetime of my AJAX client (webapp .war), and without including them in the same project and same J2EE container sharing the same ServletContext, I don't know how its possible to ensure they are running and starting and stopping at the same time.
Is there perhaps a better way to accomplish this?
Edited by: bcmoney on May 17, 2010 1:25 PM

Functinal global variable for producer - consumer architecture

Hi all,
I am using a Producer - consumer architecture for my data acquistion as in the belwo diagram.. at some time i am stuck insde the while loop continously acquiring data .. is there any way i use one stop button as a functional gloabl variable and stop inside (consumer while loop) as well as producer architecture....
Why i need this ?? so many design rules say that global variables are not a good idea...
Thanks in advance...

FGVs will work fine for your application. You can also wire your error cluster to your stop button. Your producer loop should throw an error 1 when the stop button is pressed, which will in turn stop your consumer loop. But you would need a slightly different design for this. Your consumer loop is designed to not advance until the stop button is pressed. Instead of using and enum, you can use your producer loop to control the trigger to take a measurement. No need for the while loop in the consumer loop this way. You could set up your consumer loop to take a number of measurements, and then quit or take measurements for a certain amount of time. Either way, you can stop your while loop when it times out.
If you choose to stay with this design, a simple state machine archetecure might be better based on what I see.
Reese, (former CLAD, future CLD)
Some people call me the Space Cowboy!
Some call me the gangster of love.
Some people call me MoReese!
...I'm right here baby, right here, right here, right here at home

Progmatically stop vi with producer/consumer architecture

I can't seem to get my VI to fully stop, so that I can go on to do some more things in the program. I am trying to progmatically stop the vi (pictured below), so that I can return to my main gui (not pictured). The main gui just consists of a login, exit test system, and run test. Once the user presses run test, the vi pictured below opens.
I am using a producer/consumer architecture with a state machine. The problem I run into is that when I press the "Main Menu" button, it should close the pictured vi and return to the main gui. I tried using the vi server, and when the user presses the Main Menu button, it fires an event and then it goes to the Stop state in the consumer loop, which opens a reference to the vi's (one for the strip chart vi and another for the gage vi), closes the front panel using an invoke node and then closing the reference. I also tried adding in the abort vi method after the close front panel and it still gave me the same result.
Here is the catch, it does close the vi's, but it does not stop the while loops (producer and consumer), or it stops only one of the loops. I thought of putting in a constant in the event structure for when the Main Menu event is fired, but that will stop the producer loop before the consumer loop reveices the info to go to the stop state, so the consumer loop does not end.
I also tried a "dummy" control that was connected to the comsumer loop Stop terminal (for the while loop), and then progmatically changed the value in the stop case to true and then read that value in the producer loop to, hopefully, stop both loops, but it did not work either.
I am sure that it is something so stuipidly simple that I cant see it right now
Kenny
Kenny
Attachments:
test system.gif ‏252 KB

Hi Kenny,
I think of two solutions to your problem. First, use the message queue to pass the message "stop" or something from producer to consumer or second (and I think, better), use occurrences. I made a little draft how it could work.
You have to pass the occurence- Refnum to both VIs, the producer and the consumer, when the producer finishes execution, then use "Set Occurence".
In the consumer blockdiagram you react on the generated occurrence.
If you set in the VI- properties "Open Frontpanel when called" and "Close if originally..." the VI Frontpanel should disappear itself when execution is stopped.
Hope this helps,
Dave
Message Edited by daveTW on 08-14-2006 04:53 PM
Greets, Dave
Attachments:
occurrence 1.png ‏1 KB
occurrence 2.png ‏5 KB

Producer/Consumer Architecture

Hello All,
I am currently working on an application using State Machine Architecture where I have to monitor various parameters (7 temperature values & 3 pressure values), along with this I have to On/Off a Clutch using a digital Output for set time (e.g. On for "X" seconds OFF for "Y" seconds), but as soon as the pressure and temperature values go beyond upper or lower limits or whenever the user presses the Emergency Stop (that means I have to monitor Digital Input as well) so I am moving to different error cases depending on which values go beyond limits or emregency pressed etc..
But this realisation is not working fine as I have to monitor and control several other things in addition to acquiring data....till now I just have used State Machine Architecture, I have seen Producer/Consumer Template but not quite sure how to realise such kind of application which involves data acquisition along with user interaction and monitoring.
Could someone please put some light on giving similar kind of example or what else I can try with this problem.
Regards
James

Hello James,
A Producer/Consumer architecture will certainly do the trick for you. I am sending you an example that has the Data Acquisition section in the producer loop. However, you can also put your User Interface in the Producer loop and have the Data Acquisition in the Consumer loop. The Queues are used for sending data from the Producer to the Consumer(s) loop(s) (you can have multiple comsumer loops).
You will just have to design specific cases to react to different user inputs.
By the way, this example is from LabVIEW Basics II training course. If you have never taken LabVIEW Basics I and II, I highly recommend them. You can self-pace the class or take it with an instructor.
LabVIEW Courses
Hope this helps!
Message Edited by Kalin T on 03-23-2007 09:57 AM
Kalin T.
National Instruments
Attachments:
ImplementedProducerConsumer.zip ‏250 KB

Producer consumer loop for serial comms

I have an application where I am trying to read and write to a serial port using a producer consumer loop. My producer consumer loop seems to work fine until I put the serial comm VIs in the consumer loop. Prior to that my event structure in the producer loop seems to work fine and all the controls on the front panel send commands into the queue and the expected reaction happens in the consumer loop. When I put serial comm VIs into the consumer loop the application will read the serial port but none of the other controls on the front panel ever get any reaction from the consumer loop. I am not sure if I am using the producer consumer loops correctly when using comms at the same time. Any help would be appreciated!!
I have attached the VI for your inspection.
Attachments:
pinger comms.vi ‏65 KB

You could probably continue with your original architecture as well. Just put the check for bytes at port in the Read case of the consumer loop. As you said, it takes 1-2 seconds for the device to send data. So the original architecture didn't work because you were queueing up 20 reads per second in the timeout case, but the consumer loop was taking up to a second for a single byte to come in.
If you check for bytes in the consumer loop, if none are there, the Read case will end quickly and won't wait for that byte to come in.
You should also probably increase the timeout of the event structure to 100-200 mseconds or even more. The event structure would still respond instantaneously to user interface events. And you would only be generating the timeout case a few times per second. Which means when a byte comes in, it would probably be sitting there only 100 mseconds or so before you read it. Unless you go click happy and generate lots of other events not allowing the timeout case to run.

Producer/consumer architecture - timing

Hello.
I have some question about producer consumer pattern.
People say that consumer shound NOT add elements into producer queue. Lets say that some states in in consumer need to stay active for eg 5 minutes. We also should be able to pause/resume state execution and exit program in any time.
I ve used functional global timer but consumer adds elements into queue and this should be avoided.
How to solve timing problem? Some people have written about creating another thread - watchdog or asynchornous calls to vi...
Could someone explain this to me? :

People have consumers add to their own queue all the time. It is usually in a Queued State Machine though. If you have a process that just needs to run for so long, then have the consumer run another while loop until the time is up. It can then process the next item in the queue.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines

Producer/consumer architecture over network variables with Real-Time target.

Hi all,
I am maintaining a producer/consumer data acquisition program to be deployed on a real-time target. The main code is deployed to and run on the real-time target during experiments, but was having trouble because the program was originally designed to write all experimental data to disk on the real time during acquisition, which puts the whole experiment at the mercy of the hard drive. I am now trying to rework the code so that the host takes care of logging, so that my time-critical loops don't have to wait.
I am currently using LabVIEW 8.5
I have two questions:
First, how can I programmatically call the data-logging subvi on the host so that it runs in parallel with the main vi which runs the experiment and collects data on the real-time? I have attached the test code that I have been working with to figure this out, but it does not run the logging vi continuously in the background. I am aware that there is better functionality for this in newer versions of LabVIEW, but I would prefer not to upgrade unless there is no other option. I would like to be able to run my data-generating vi and have it start the data logging remotely.
Second, is there a way in the host VI to read values off the network variable using an event structure rather than polling it for updates?
Any help would be sincerely appreciated!
Attachments:
testRemoteLogging.zip ‏124 KB

VI server
Mark the target VI as served on the machine on which it will execute and use VI server Call by reference to invoke the served VI.
This used to be taught as THE way to communicate syncronously with an RT app.
Ben
Ben Rayner
I am currently active on.. MainStream Preppers
Rayner's Ridge is under construction

Timed structure for output in producer consumer data acquisition

Hello LabVIEW community,
I have a bit of a problem. I am writing a program that is primarily for data aquisition but has a few control features as well. I need the program to aquire and write several channels of data at a relitively high speed. The program does this fine ( in the top 3 loops fo the program). I need the program to also send a seires of two output signals in response to a particular condition. When a certain channel registers a value of above a specified number, I need an output channel to write an anologue signal for 225 seconds and then a signal of a differnt value to the same channel untill the condition occurs again (not for several hours).
I put this action in a timed structure inside of a case structure. The case structure and timed structure are inside of a second consumer loop. I have no idea if this is ligitimate. This event takes much longer than any of the other loops which run at 1 or 2 seconds. When I exicute the program with "highlight exexutuion" this longer (case/ timed ) loop never executes. I asssume this has to do with the mismatch in time scales of this loop and the other loops in the program. I also didn't really understand the help information on timed structures, so its possible that I just need better/ different inputs to the timed stuctures.
If anyone can see any obvious problem in the code or suggest a better way to do the output function I would really appreciate the help. My code is attached.
Thanks,
Jo

I can't figure out how to fix your code, but I can point out the things I see wrong with it.
1. You are dequeueing elements from the same queue in the same loop. In one you are not taking the element, and in parallel, you are taking an element. I see that in your bottom two loops. Why? That code won't execute until the queue gets two elements in it. And which dequeue gets it first and which gets it second is an arbitrary race condition. So there is no certainly of order (first 4 vs. last 4 currents in the one loop), or which gets kept and which gets discarded (in the other loop.)
2. You are creating and/or clearing DAQ tasks in every loop iteration. Tasks should be created before a loop, used inside the loop (read or write), then cleared once the loop is done. Anything else is wasteful, time consuming, and could lead you to run out of resources.
3. You are using the STOP function which is pretty much like hitting the abort button on the toolbar. No program should ever stop that way. All loops should exit gracefully.
4. You have a data dependncy between your bottom two loops because of the boolean wire running from one to the other. That bottom loop will only run once (if the value is True), or run forever (if the value is false).
5. Use of the dynamic datatype. You are taking waveforms, converting them to a blue wire, then coercing them into another datatype such as just displaying a scalar in an indicator. A lot of unnecessary conversions there.
6. Your thermometer indicators have a digital display you can make visible. Then you won't need the separate numeric indicator to display the value.
7. For loop that runs only 1 time because of the constant wired to the N terminal. Get rid of the For Loop.
8. Check your spelling. The word "Temperatures" on the graph, and one instance of "distillate" is missing an "l".
Until all of these problems are fixed, it is not worth talking about timing of timed structures. Some of these problems were discussed in your other thread. http://forums.ni.com/t5/LabVIEW/producer-consumer-missing-channels/m-p/3159757#M911255 But now it seems like you've made things even more complicated without fixing the basic problems.

Basic Producer/Consumer loops for a newbie

Labview neewbie here.
I was looking at Producer Consumer loops.
Can anybody give me a very basic explanation of how to set one up.
I want to take readings from a HP 53131A counter and store them in an excel spreadsheet.
I understand the part about labview slowing down as it opens the Excel file, and a seperate loop will help solve this.
But I dont understad how to set up the Que function.
Thanks.

Multithreading will not make excell open faster but using concurrency you can have more than one operation operate ate different speeds. The nice thing about the produce consumer model is that the if the consumer is slow or unreliable (freeses up) the producer doesnt care and will continue. To use a queue is very simple, 1. create the queue, name it(optional) and give it a data type (ie int, array of floats ...) in the produce loop use the enqueue vi to add items as needed to the queue, in the consumer dequeue these items (using a timeout = -1 the loop will wait for a new item and not consume resources) the consumer can process the data acquired in the producer loop.
Paul Falkenstein
Coleman Technologies Inc.
CLA, CPI, AIA-Vision
Labview 4.0- 2013, RT, Vision, FPGA

Database producer consumer architecture

Dear NI supporters,
I am trying to devolope Producer/Consumer program to communicate with MS SQL database.
The Producer loop select data from database1 and process them.The Consumer loop insert proccesed data to database2.
The problem is, fast Producer loop must wait for slow Consumer loop and I have no idea why.
I have attached really simplified VI. There are two parallel processes (while loops). First loop read large data and the execution time is high. The second loop read really small data, so execution time is low. So if I execute the VI, I would expect the fast while loop (FAST PROCESS) will get much more iterations then the slow one (SLOW PROCESS). But the reallity is, the FAST PROCESS will get the same number of iteration as SLOW PROCESS. It looks like the processes don´t run in parallel or did I just missed something?
Could you explain me, why this occur, and how can I improve my code?
Thanks in advance.

vasicekv wrote:
Thank you Jim, it really helps me!
Do you also know any other library to communicate with database in labview which is reentrant?
I mean I am focused on speed my labview-databse communcation up.
Perhaps you should be considering why your consumer loop is so slow? Is your database running on a dedicated server? Is it getting bogged down?
Please share a little of your real code so we can try to improve the performance of your consumer loop.

Help? Looking for better approaches to mutlithreading, parallelism (producer/consumer)

Hi,
   So II would like to do some parallel processing of data using the multiple cores of my computer, and I'm trying to find a way to do this in labview. I deally, I would liek to be able to take a given task, put the work into a work queue, have threads from each processor take a task from the work queue and process it in parallel until the work queue is empty, and put the result into a results queue where I could then write the results to a file. So in a text-based languange, the implmentation would be straightforward.. but I'm not sure how to go about this in Labview.
       I've read about the different designs patterns, and was trying to use a Producer/Consumer approach with queues. What I wanted to do was read a list of files from a directory, stuff the paths into a queue (Work Producer), then have a second loop (Work consumer/Results Producer) dequeue each file, process it, and send the results to a second queue. Then at the end, a final loop (Results consumer) would read all of the results.
My question now is.. is there a way to do this more cleanly than what I have on the diagram? Right now, I have to know in advance how many parallel tasks I want to use, then I have to wire up each task before executing the program. If I move to a different computer, I need to change the wiring digram. Is there a way to abstract this to avoid having to make huge changes to the block diagram? I was considering trying to make the parallel portion fit into a subVI so that I could just copy and paste several sub-vi's to easily add and remove parallel threads..    but its not so simple to have to keep track of all the queue references, and etc..
Has anyone already done somoething like this? I feel like what i'm trying now is becomming a big mess
Thanks

You can run clones (instaciate from tempaltes) of the crunching code where each get a ref to the queue based on its name. When they see the queue is empty the terminate and close.
Make sure your queue is full be fore you lauch them.
There is supposed to be new property that tells you how many CPU you have but I can't find it at the moment.
Have fun!
Ben
Ben Rayner
I am currently active on.. MainStream Preppers
Rayner's Ridge is under construction

State machine VS producer consumer architecture - Time Analysis

After learning various methods to program efficiently and learn how to use SM and Producer consumer. I built a program to control stepper motor in both these techniques.
Here is the RESULT. As we can see a simple state machine without any complicated producer consumer technique performs faster than the second one.
I am not sure which is still the best based on performance and optimization. Please advise which one should I keep and why.
Abhilash S Nair
Research Assistant @ Photonic Devices and Systems lab
[ LabView professional Development System - Version 11.0 - 32-bit ]
LabView Gear:
1. NI PXI-7951R & NI 5761
2. The Imaging Source USB 3.0 monochrome camera with trigger : DMK 23UM021
OPERATING SYSTEM - [ MS windows 7 Home Premium 64-bit SP-1 ]
CPU - [Intel Core i7-2600 CPU @ 3.40Ghz ]
MEMORY - [ 16.0 GB RAM ]
GPU - [ NVIDIA GeForce GT 530 ]

This is with state machine alone
Abhilash S Nair
Research Assistant @ Photonic Devices and Systems lab
[ LabView professional Development System - Version 11.0 - 32-bit ]
LabView Gear:
1. NI PXI-7951R & NI 5761
2. The Imaging Source USB 3.0 monochrome camera with trigger : DMK 23UM021
OPERATING SYSTEM - [ MS windows 7 Home Premium 64-bit SP-1 ]
CPU - [Intel Core i7-2600 CPU @ 3.40Ghz ]
MEMORY - [ 16.0 GB RAM ]
GPU - [ NVIDIA GeForce GT 530 ]
Attachments:
Control 1.ctl ‏6 KB
Control 2.ctl ‏6 KB
Motor-UNI_Directiona Dev 3-TIME_ANALYSIS-2.vi ‏54 KB

Calling producer/consumer subvi using main vi for image grab acquire

Hi everyone,
I am not sure if I get the producer/consumer concept wrong. I tried to write a producer/consumer subvi that simulate continuously grab acquire image. When I call the subvi from the main while loop with a main vi, it seems to run into a infinite loop. I am wondering does anyone know how else can I activate a producer/consumer subvi using main vi? Thanks in advance.
Kind regards,
Han Yen
Solved!
Go to Solution.
Attachments:
MainTrial.vi ‏7 KB
ProducerConsumerGrab.vi ‏18 KB

Hi Han Yen,
a few remarks :
- your Top VI has no function what so ever. So skip that.
- your image queue has no name, so that gives problems from the minute you're going to use more queues.
- It does not matter which command you send to the queue, it will start the bottom loop anyway
- You don't have a check whether "live trial" is true yes or no, so everytime you switch it from T=>F or F=>T is will activate the command.
I've tweeked your producer-consumer. Maybe this will give you an idea. I put some remarks in some places
Kind regards,
- Bjorn -
Have fun using LabVIEW... and if you like my answer, please pay me back in Kudo's
LabVIEW 5.1 - LabVIEW 2012
Attachments:
ProducerConsumerGrab.vi ‏21 KB

Using BlockingQueue for Producer - Consumer

I am trying to use the BlockingQueue in a Producer-Consumer
pattern instead of creating a Q and using wait() and notify().
The output what I want is:
Put : 1
Got : 1
Put : 2
Got : 2But the output I am getting is :
Put 0
Put 2
Put 4
Got 3
Got 5
Got 7This is my Java code using BlockingQueue
Have I missed any point?
Can I not use BlockingQueue instead of using wait() and notify()?
class Producer2 implements Runnable {
     private BlockingQueue<String> queue;
     public Producer2(BlockingQueue<String> q){
          this.queue = q;
     public void run(){
          int i=0;
          while(true){
               try {
                    String putValue = Integer.toString(i++);
                    queue.put(Integer.toString(i++));
                    System.out.println("Put " + putValue);
               } catch (InterruptedException e) {
                    e.printStackTrace();
class Consumer2 implements Runnable {
     private BlockingQueue<String> queue;
     public Consumer2(BlockingQueue<String> q){
          this.queue = q;
          public void run(){
               while(true){
                    try {
                         String value = queue.take();
                         System.out.println("Got " + value);
                    } catch (InterruptedException e) {
                         e.printStackTrace();
public class ProducerConsumer_BlockingQueue_Test {
       public static void main(String[] args) throws Exception {
              BlockingQueue<String> q =
                 new LinkedBlockingQueue<String>();
              Thread p1 = new Thread(new Producer2(q));
              Thread c1 = new Thread(new Consumer2(q));
              p1.start();
              c1.start();
}

Nothing to do with the queue, in your producer you're incrementing i twice, once after seting putValue, again after queuing the item.

2 producers/ 1 consumer for a Producer/Consumer Architecture

Similar Messages

Maybe you are looking for