How to handle ever expanding array?

Question

Hi,
I'm looking for help with a temperature control experiment...
I have a program to aquire data from 24 channels through a while loop.
I am displaying the data in a waveform graph. Currently, I am using
shift registers and build array to add new data to the graph every time
through. This worked fine while I was prototyping the hardware, but now
I'd like it to run for days on end; as everyone can imagine, the array
keeps getting bigger and the build array function keeps hogging memory
until the whole thing crashes.
I'm guessing that I should be writing the data to a file every so
often, which would keep the size manageable, and would allow me to know
the size of the array and could use other opperations rather than build
array. The array would need to stay pretty small so the write operation
didn't slow down the loop too much. The program is pretty slow, since
I'm limited in how quickly I can
change temperatures. Currently, I go through all 24 probes about every
2.5 seconds (this is set just by how long it takes the routine to read
each temperature and calculate the control parameters). If it takes a
little bit longer each time through, that's not a huge problem, but a
big pause every so often would make the control routine behave oddly.
What the save-to-file option takes away, however, is the ability to
walk up to the machine in the morning and see at a glance how well it
regulated the temperature during the night. I've thought about keeping
some reduced set of the data (every minute or so) to give me a
low-frequency glance at the data, but it seems like this will simply
postpone the inevitable memory loading. I don't know if I could have a
separate program that opened up the files that the control routine
wrote, but this would probably be too much for the poor old PII machine
that is running this.
Does anyone have a simple solution to this probelm? Failing that, a
complex solution might do, although it might get beyond my programming
abilities.
I only have LV 6.0, so I can't see examples saved for versions later than this.
Thanks for any suggestions.
cheers,
mike

Answer 1

Mike,
Writing data to an array every so often is definitely what you want to do. Then if power fails or the program shuts down for some reason, the data to the last save is still in the file. At the 2.5 second sample rate you only get ~35000 points per channel per day, so data files will not get too large. Decide how often to write by how valuable the data is: What is the penalty for losing data for the last minute, hour, day...?
Another point is that the panel display only occupies a few hundred pixels in each direction. Plotting more points than that is meaningless. If you plot one point per minute you will have 1440 points per day (per channel). You could always show the most recent 24 hours using a circular buffer. With more programming effort, you could keep previous days plots available for selection by the user or read them back from the file.
Lynn