Vi/data managment

Question

Hi all.
2 years ago I graduated and 1year aog I started developping a typical stimulus-response set-up for behavioural neurofysiological experiments in LV, without knowing anything about LV (or programming in general). I'm a pretty quick learner however, and today, the set-up (1 stimul pc, 1 control (LV) pc and a dsp all connected through tcp/ip) is used intensively in our lab, although i'm stil debugging/developping new features for it constantly,
I'm getting managment problems however, a few examples: the dsp sends everything it samples (now 3channels @ 20kHz) to LV in blocks of 200samples, for saving to disk and display). At the same time it sends/receives control commands to/from the stimul pc and eventually from other pc's on the internal network.
When the system is running for a long time, the LV display can suffer from delays up to 10seconds. eg i stop execution on the dsp, but the LV display keeps on going for 10seconds displaying data.
Also, every time the histograms get updated, the main vi's display freezes for half a second.
Sometimes it takes way too long before LV (or is it windows? I don't know how the tcp/ip is handled) reads the tcp/ip data, what results in crashing the dsp (I'm at the edge of cpu usage there) which is the worst that can happen because the experiment has to be stopped then.
After reading up a bit on data managment etc, I easily figured out my vi probabely is one big mess that would scare most programmers away (I cannot post it for internal reasons, but I posted the fp of the vi unning a test so you can imagine there's a lot behind it.) Since the vi is still growing every day, I don't have much time to refactor it completely, but I feel I have to do something or I will get lost in the future (we're planning to go to 16channels which all have to be saved to disk by the vi, later it will be 100 channels, all containg brain activity)
So I have some questions I'd like to be answered by more experienced users so I can get some ideas for a new implementation.
Currently, I have 4 while loops containing tcp readers. 3 of them also display the data received and write it to file.
The writing is buffered however, eg first filling a shift register with 10000 samples, then post an event and pass the data through the event to another loop writing it to disk. Would it be better to store the data in a local/gloabel, common between the two loops, and just set an event to warn the scond one? Or is the queueing system better (haven't looked at it yet)? Same question goes for getting the data from the main vi to the histograms, I do this with a global event.
For the displaying, not every single point is displayed (because screen refresh rate is way slower then data rate), but would it benefit putting the display in a seperate loop and use events/... to pass data?
Also, I use like 100 local variables that have to be commonly accessible between the several while loops I have running, I've read locals aren't that efficient, but i see no other solution for it?
The squares on the eye/spike displays you see in the attachment, can be positioned/resized by the mouse, i have seperate while loops for that, recalculating the position every 100mSec depending on zoom etc, every display on the fp is a layer of several pictures/graphs. So the updating of the layers is never synchronous (eg displays data every 20mSec, but the squares every 100mSec), is this a drawback or does LV automatically sync it since it can only display at refresh rate anyway?
Almost every user input (menu/buttons) is handled through events, so I have 3 while loops with an event structure, each handling 20events with sometimes lots of work in it. Is a system with event handling, then passing it to a task handler or so preferable and why?
And is every while loop in LV running in a seperate thread?
I have several more questions, considering things that look fundamental LV stuff to me. Maybe I should follow a course? but I doubt there's a course focussing on all my problems. Or should I get a private teacher somewhere? :-]
Thanks for reading this little book, and thanks in advance for any answers!
Stijn
(LV 7.0, very clean win2k system on a dual xeon 3Ghz 1gb ram dual screen dell, i get up to 60%cpu usage during experiments, most of it kernel time)
ps very important note for dual nvidia screen users: do *not* use dual display, only horizontal span, or windows will eat all your kernel time while running eg vi's, hence slowing down everything really, really bad. Horizontal span is better since then windows thinks it's one (huge) screen only. It took me weeks to figure out why my vi couldn't read/display all data in time, well, it could, but it was just waiting for windows to do freaky secret stuff with my display driver.
Attachments:
VI.zip ‏755 KB

Answer 1

Hi Stijn,
First of all, there is a specific course, called the LabVIEW Intermediate I course, that discusses these issues. This course has been created for customers that want to write professional and large applications.
There is a course scheduled in Belgium the 29th of August, btw.
I have one big tip actually: Seperate your User Interface handling from the data processing and put them into separate parallel loops.
Use the Producer/Consumer Design Pattern with Events (template in LabVIEW). Use the Queue VIs to make sure that no event on the User interface is lost and use User events to communicate back to the User interface loop.
Queue your data coming from the TCP/IP communication. Use separate loops to split-up the acquisition of your data and the processing of your data. Master/Slave designs are ideal for this. Queueing will make sure that that you don't lose the data while processing. Make sure that none of the loops are blocked by shared resources. For example, make sure that the acquisition loop doesn't have to wait for the processing loop to finish because there is a shared resource between the loop. Queues don't have that problem.
Locals can create "race conditions". They also create a copy of the data which is memory intensive. Alternatives for locals are Queue Vis or Notifiers. (Check the design templates in LabVIEW). You could also use "functional globals". These are VI's that replace the normal global variable by a state machine that has a "get data" and a "set data" state and where the value is stored in the "shift register".
Multithreading ---> Check the link below:
http://zone.ni.com/devzone/conceptd.nsf/webmain/D2​E196C7416F373A862568690074C759
Most of your questions are handled by the Intermediate I and Intermediate II course.
Regards.
JorisV
TL AE
NIBE