Reducing data rate written to a LVM file

Question

I have a LabVIEW 7.1 question regarding data logging. I have designed a system that reads the pressure of 6 4-20Ma pressure transducers. This data is then filtered to remove any noise, and displayed to the user on the computer's screen.
I am currently reading the data at 1000HZ using the DAQ Assistant, which seems to give a clear picture of the pressure changes. My supervisor has asked me to allow the system to log the results for what may be 3 or 4 days. As you ca imagine, 48 hours at 1000 samples a second is a lot of data! I need some method to reduce the data rate written to the file. Ideally, I would like to take one second of data, find the highest reading, and then file a single entry and a timestamp interval of 1 second. So, for each hour, there would be 3600 rows with 7 columns (1 time stamp and 6 sensors). It would be nice if the 1 second average could be adjustable, so that if long term logging is to be done (a week, for example), the interval can be changed to 5 seconds or whatever.
I have tried reducing the data read rate as much as possible, but it seems that 100hz is the smallest that is possible. That is still too high... as the hard drive would fill up in no time.
Any help in this area would be appreciated. I have tried a few things, and all to no success. I have included a copy of the code for anyone to review.
Hardware:
- 1 X P3 Laptop running LabVIEW 7.1 data acquisition software
- 1 X NI DAQCard-6062E interface PCard
- 1 X NI SC-2345 data acquisition hardware with
- 3 X SCC-CI20 current input module
- 6 X Omega PX 0-600PSIA 4-20mA pressure transducer
thanks so much!
Andrew
Attachments:
Testing.vi ‏812 KB

Answer 1

You would have to talk with your supervisor first to determine what he intends to do with the log data and what degree of resolution he actually needs first. You probably want the 1000Hz Sampling rate so you can do decent filtering on the signal (hopefully your pressures aren't actually changing at a faster rate than that). I'm assuming you are returning a single result for those 1000 readings for each sensor. Specify some file logging duration of n-seconds (or minutes or whatever). Between file writes, pop each filtered measurement into an array (either 1 array of 6 dimensions or 6 1-dimension arrays). After n-seconds have passed, determine the min, avg, and max values from each sensor (array) and log those with your timestamp. So if you set your log timer for 1 minute, you would log a single min, max, and average reading of 60 readings for 6 sensors (this would only require 1 row with say a timestamp and 3x6 (18) columns for each sensor's min, max, average data). After a 24hour period you will have logged 1440 rows of data. In 3 days that would only be 4320 rows of data. All that is as easy as using a timer and a case structure around your logging function which would be triggered every n-seconds. Everything else you're doing would be the same. None of this really has much to do with labview as it is more of a logical explanation of how and when to acquire and log your data. What method are you using for storing your data? CSV, BIN, etc.. If you also want to display the data in a chart, I would recommend charting the same data you're logging, otherwise your chart will probably crash your system at 1000samps/second for 60 hours.. Once again, it depends on how your supervisor is analyzing the logged data. Make your log duration programmable and change it until he is happy. If he's(she's?) using Excel, your maximum log timer would be 9 seconds (Equates to ~6.67 Logs per Minute, ~400 Logs per Hour, ~9600 Logs Per Day, for a total of 28,800 Logs(rows) for 3 days -- Excel is limited to 32000 rows).