WARNING TDMS memory leak in LV 2010

Question

Hopefully this will save someone the headache that I've been through the last couple of days.  I have a very large applicaiton that is running a final verification test on a production line.  In my testing I noticed a memory leak in the application and after 2 days of debug discovered that the TDMS logging is the culprit.  This is very disappointing since I am (I mean was) a huge fan of the TDMS file format and the LabVIEW functions.  The attached VI reproduces the leak by checking and unchecking the Memory Leak checkbox.  Its a bit ugly but I was just coping and pasting the sections from my application and trying to reproduce the issue.  Luckily for me in this particular application I was only writing to the TDMS log files so I was able to eliminate the problem by switching to the gTDMS versions of the write functions.  I found these referenced in another post about TDMS memory leak, but in that case the leak was caused by the indexing and the fact that the SAME file was continuously written too over a very long period.  As you can see in my case, a log file is opened and closed for each "Test".
gTDMS link
Thanks,
Brian
Brian Gangloff
DataAct Incorporated
Attachments:
TDMS Memory testing.vi ‏31 KB

Answer 1

YongqingYe wrote:
Hi Brian,
I'm one of the developers of TDMS in NI R&D. Well, this is a problem of TDMS which has been complained by some customers. The reason you see the "memory leak" or the memory usage increment is because TDMS needs to bookkeep some information in memory and when you writing more and more data values, the information we keep in memory will keep increasing.
There are some workarounds, gTDMS probably is also one of them, but the original purpose of creating gTDMS is to support writing TDMSs on Linux, Mac and other platforms:
Using "NI_MinimumBufferSize" propertie on channels, you can find the details of the help documentation of TDMS Set Property. It can not eliminate this problem, but would reduce the memory usage significantly. Normally we would set it as 1,000 to 10,000.
From LV 2009, if you writing to the file always with the same layout, like same channels same number of data values, you will not have memory increament.
If you are using LV 2010 and later, you can try to play with TDMS Advanced API, this API will not have any memory increasing problem at all.
Thank you!
Yongqing Ye
NI R&D
Hello Yongqing,
Apparently you did not bother to look at the examples that I provided or read any of the description either.  As Hooovahh has already pointed out, INDEXING is NOT the issue.  The example writes an array to multiple channels ONE time and then the reference is CLOSED.  In the case that does NOT leak, there are multiple waveform arrays written to the file which would require some indexing but the memory does NOT increase.  The problem is when an array of strings is written to multiple channels and the reference is CLOSED.  Unfortunately this type of quick assumption about the problem is why the real issue was overlooked back in 2009.
Thanks,
Brian
Brian Gangloff
DataAct Incorporated