Why replay history data from NI-DAQ buffer?

Question

LabVIEW 8.6.1f1, NI-DAQ 8.8, Win Vista, two cDAQ chassis.  (I can't move to LV09 or LV10 for this, not yet anyway).
I have a client with two cDAQ banks, each with 16 TC channels (on 4 modules) and 16 voltage channels (on 1 module), 64 channels all told.
The config operation is like this:
This is called twice, with TC CHANNELS and V CHANNELS describing the channel arrangement.  Both channel descriptors are the same except for device number. The Sample rate works out to 1000 / 100 = 10 Hz. I am using CONTINUOUS SAMPLES, with no need for a buffer, but I understand that one is created anyway.
One loop of my main program is waiting on the READ function:
You can see the explanation of why the loops are there.
After reading the data (at 10 Hz), each frame is sent via UDP to somewhere, and every 5th frame is displayed, via another loop.
This program runs 24/7 on the same computer as another program.
All this works fine in normal circumstances.
However, occasionally, a weirdness creeps in:
In the mornings, after waking the screensaver (or the CPU itself, I'm not sure), we sometimes (not every time) get a very rapid display - the displays are changing much faster than the expected 2 Hz.
I've been wondering whether the data shown was historical, or simply faster updates of live data.
Today, it so happened that a blower was turned on just before waking the computer. The device takes 15 seconds to ramp from zero up to operating pressure normally, and then it stays there.
The readings on screen were frozen (unchanging).  When the computer was awakened, what my client saw was the reading for the blower pressure ramp from zero to running pressure in 2-3 seconds, stay there for 2-3 seconds, drop quickly to zero for 4-5 seconds, and start the ramp again.
Over and over and over it replayed this data, way faster than 2 Hz.  At least 40 times it replayed this same data, while the real pressure was just variations around the operating pressure.  As time went on, instead of dropping to zero, it would drop to some point on the ramp up and repeat from there.  
I don't know if it would eventually straighten itself out - the client gives up and restarts the program.  All is well after that.
I've gone over my code several times, and I can't see a mechanism for me to store this data and re-play it.
Here's my best guess as to what's happening:
Since I am not explicitly setting the buffer size, it defaults to 1000 samples.
Since I'm running at 10 Hz, 1000 samples = 100 sec.
At some point, because of the screensaver, or because of CPU sleepage, or because of integer rollover, or because of zombie infiltration during the night, NI-DAQ has become confused. 
When the mouse was wiggled, the gates were opened. After that, whenever I ask for a sample, it gives me one, without regard to read/write position. It continually gives me the next sample from the buffer and wraps around, again and again, and again, as fast as I can read / process / display.
It is also putting new data into the buffer, so that the historical data eventually gets wiped out.
That fits the facts:
  It shows only recent (last 100 sec) data, not ancient data, and not live data too fast, as I had considered.
  The data changes with time - oldest data gets wiped out.
  It plays the SAME data over and over and over.
But that means a bug in NI-DAQ, I think.  
QUESTIONS:
Any flaws in my thinking?
Any previous sightings of this symptom?
Any ideas for workarounds?
Steve Bird
Culverson Software - Elegant software that is a pleasure to use.
Culverson.com
Blog for (mostly LabVIEW) programmers: Tips And Tricks

Answer 1

The rapidly changing data, is that on the viewing machine or on a display on the device acquiring the data?
The "device acquiring the data" is a pair of cDAQ chassis.  There is no display there. It's the LabVIEW program that is reading and displaying the data.
Does your computer go into any sort of power saving mode where the USB controllers are disabled during hibernation/standby?
on MY machine, I have tried this with CPU sleepage and such.  Didn't know you could sleep the USB, but on MY machine, that's disabled.
But I haven't made it happen on MY machine.
My CLIENT's machine is actually running XP, and is set to turn off monitor NEVER, turn off hard disk NEVER, and system standby NEVER.
What version of DAQmx do you have installed?
On MY machine, it's 8.8 - I haven't found out from my client yet what his is, but it won't be far away from what came with LV 8.6.
Can you post your code?
Well, OK:
Here's two of the loops running (the third loop deals with TCP/UDP and is not relevant, I think).
The TCM DAQ manager 's READ function is pictured in the original post.
Here we wait on a sample, scale it (volts into EU), and feed it to the PROCESS TCM DATA vi (which does averaging).
Here's the NEW DATA function for PROCESS TCM DATA:
The ST (short-term) HISTORY is set up as an N channel by 5-sample array, and ST HISTORY SAMPLES is set to 5.
The DAQ loop, after processing this data, triggers a DISPLAY event, which triggers a DISPLAY action, which lands here:
The GET DISPLAY function retrieves the latest averages:
That data (64 channels worth) is split into two banks and displayed:
The display process simply distributes the array data into clusters (channels can be user-configured to be in various locations on the screen):
The UPDATE TIP action causes a TIP STRIP string to be updated:
The TIP STRIP shows the average and min and max values (which might be hidden from the display itself), based on which display cell the mouse is in.
A MOUSE ENTER event on any display cell sets the CLUSTER CONTAINING MOUSE variable to the CTL REF of the event, a MOUSE LEAVE event for the overall display bank (containing 32 cells) sets the CLUSTER CONTAINING MOUSE variable to Not-A-RefNum.
the DISPLAY looks like this (only a few cells shown).  The display for this program occupies an entire 1680x1050 LCD display, another program runs on the main monitor.
As you can see, the only storage in this process is the AVERAGER, and that is limited to 5 samples (0.5 sec). It cannot store any more than that.
I've thought about the possibility of the mouse oscillating between two cells (as mice are sometimes known to do) when it is parked, and causing a flood of MOUSE ENTER events.
But 1) it happens more often than the odds would dictate  (a random location is not likely to be on a cell border), and
2)  When that cleared up, the queue would be processed, but it would display CURRENT data more rapidly than 2 Hz. NOT historical data.
Given that it seems to be about 100 sec of history (judging by my client's recap of what he saw, and how it operates), that points to the 1000-sample buffer within NI-DAQ.  That's the only place that I see to store that data.
I'll admit that I'm not watching the ERROR output from the NIDAQ READ function. Perhaps I should.
The only scenario I can come up with is that NI-DAQ's READ pointer is confused.
When I ask for a sample it tells me that there is ONE and only ONE available.  So I deal with it.
When I ask again, it tells me the same thing.
When I ask again, it tells me the same thing.
It goes round and round the READ buffer.  At the same time, new data is coming in, replacing the oldest stuff.
But it (apparently) never gets back to the "normal" condition, where it WAITS on a sample at the RAD function.
Nobody has waited long enough to see if it would do that eventually, after a few minutes (50+ replays), they give up and restart.
Restarting the program resets the cDAQ drivers, among other things, so the problem goes away.
If there's another scenario to cause these symptoms, I'm all ears. But I cannot think of it.
If it was a USB SLEEP problem, I wouldn't think the data would replay from BEFORE the wake-up. But that's what happened.
Steve Bird
Culverson Software - Elegant software that is a pleasure to use.
Culverson.com
Blog for (mostly LabVIEW) programmers: Tips And Tricks