.tdm file storage vi's slow editing vi's

I am developing software using Labview 8.5 and have recently added some file storage vi's for the .tdm format for use in DIAdem 10.2. Upon using the open storage, set file properties, write channel group, write channel, and close storage vi's, the editing has become extremely slow - for any operation I try to perform there is a 10-15seconds delay before it occurs. If I remove the .tdm vi's from my diagram then the editing goes back to normal and changes/operations occur almost instantaneously as expected. My project is not large as far as I am concerned (Front panel - 121k, block diagram=1538k, code=0.2k, data=957k, total=2617k, total vi size on disk=389k)
I saw a similar post to the .tdm storage vi's causing this to occur back in 2005 and was supposed to have been on the fix list but it does not appear that it has 3 years later. I am sure many others are using these vi's so not sure what I may have done differently to cause the editing to slow down so much.
Anyone with ideas and fix would be greatly appreciated as I cannot afford to sit around 10-15 seconds for every step of my developement.

Yes, sometimes Storage VIs are slow. But They are Express VIs, supplying the easy configuration for users. If you feel they are too slow, I suggest that you can use TDMS files instead.

Similar Messages

Editing String in.TDM file

With Set Properties it is possible to add one or more strings to a .TDM file (or is it to another file that is linked to the .TDM file?). Similarly it is possible to read that string with Get Properties.
My current impossibilities are:
- If each measurement is appended to the preceding measurement how do I read a record that is not the last one saved?
- If one or more strings has to be edited how can the modification replace only the affected strings? This implies a Read VI that is also capable to Write (replace).
I am not finding much information on this file format and would appreciate knowledgeable insight.
Regards,

TDM = 2 Files
The Storage VIs create a .TDM file (XML-based, for descriptive properties) and a .TDX file (binary, contains waveform / array data).
One or many channel groups
Each measurement can be appended to a file as a new channel group, or the values can be appended to channels that were stored earlier (resulting in one channel group per file). Use the "Always create new channel group/channel" switch to decide which way of appending data you want to use. You might want to use the "Data viewer" example shipping with LabVIEW to have a look at the hierarchy of channel groups and channels.
Note that when appending values to a channel, "Write Data" uses the channel name to find the right channel to append data to.
Get and set properties on particular channel groups
To modify data on a particular channel group, you need to obtain a "storage refnum (channel group)" to this channel group. There are two ways of doing that:
- Use "Read Data", configured to object type "channel group". The "channel groups" output will give you the array of refnums you need (the attached VI shows how to obtain and use these refnums).
- Wire the "storage refnum (file)" that comes out of "Open Storage" to "Get Properties". The "channel groups" property will be an array of all channel group refnums in your file (the "Write channel groups" and "Read channel groups" examples shipping with LabVIEW show how to do that).
"Read Data" should be used if you want to apply a query condition or if you are e.g. looking up channels across channel groups. "Get Properties" can give you all channel groups in the order they were stored in, or e.g. all channels of a particular group.
"Read Data" and "Set Properties"
"Set Properties" will replace all properties that you have selected and that show up on the block diagram as terminals. All other properties remain unchanged. The properties will be written only to the one file, channel group or channel that the incoming refnum refers to. "Read Data" doesn't write anything by itself, but you can use the output refnums of "Read data" to apply "Set Properties" to whatever you have read.
Attachments:
GetAndSetPropertiesInTdmFile.vi ‏115 KB

Storage VIs internal error, TDMS file

I use LabView 8.5.1, DAQmx 9.0.2. and a module NI-USB6216.
I configure on my application a task as follows: 15 channels activated with a sample rate of 1sample/10s (very slow). My acquisition has to run for a few days or more.
Sometimes, I obtain the error "Storage VIs internal" error (code -2500). then I cannot read my TDMS file (see attached the file) and I cannot treat my data.
Do you know the exact meaning of this error on the case of a TDMS file. Do yo have a solution to treat my data?
Best regards, Daniel.
Solved!
Go to Solution.
Attachments:
Data.zip ‏436 KB

I tried with LabVIEW 2010 and didn't see the problem when reading your attached TDMS file.
Please download the latest TDMS installer from here:
http://zone.ni.com/devzone/cda/tut/p/id/9995
Please backup tdms.dll on your computer like in c:\program files\national instruments\shared\tdms\ and then run the installer.

Error reading TDMS file with inconsistend dt

Hello all,
I would like to ask your help to understand how to read a TDMS file which I am opening and closing based on a user input. As a result the data in the TDMS file does not have a consistent time step, which may be causing my problem, but I really don't know. I keep getting the error 2501, that I have an invalid reference.
Or, if I continue and bypass the error, the extract portion VI gives me an error stating that I need to define an integer multiple of dt.
I am not sure how to solve the first point as I have the reference wired directly to the read TDMS vi.
I have attached a screen shot of the error and the portion of the VI with the offending error (it is the code connected to the read TDMS read vi).
My goad is to extract a portion of the data to do calculations on and display a result to the user.
thanks for any comments.
A. Lopez
Attachments:
error-2501.JPG ‏198 KB

I think you open multiple times the same TDMS file, you should open it before the while loop. And I don't see a close tdms function...
Also check for errors on the open TDMS function.
Ton
Message Edited by TonP on 02-14-2007 07:45 AM
Free Code Capture Tool! Version 2.1.3 with comments, web-upload, back-save and snippets!
Nederlandse LabVIEW user groep www.lvug.nl
My LabVIEW Ideas
LabVIEW, programming like it should be!

I have the G Drive 1 TB (GEN4 1TB) external drive purchased in 2010. I need to connect to my new MacBook Pro. I'm using the external drive for media storage of my video editing projects. Will a 800fw to thunderbolt adapter cable work?

I have the G Drive 1 TB (GEN4 1TB) external drive purchased in 2010. I need to connect to my new MacBook Pro. I'm using the external drive for media storage of my video editing projects. Will a 800fw to thunderbolt adapter cable work? I understand that using a USB port connection is not fast enough for video editing. I read somewhere the G Drives don't connect well to MacBook Pro.
thanks
larry

I doubt this can be relocated or moved so you might want to just repost in the macbook pro area.
MacBook Pro
https://discussions.apple.com/community/notebooks/macbook_pro
https://discussions.apple.com/community/mac_os?view=discussions
Can you afford a new Thunderbolt case and cable? any case with FW800 is going to be slow 72MB/sec and less, and any new 1TB drive is capable of 75MB/s minimum up to 125MB/sec.
Thunderbolt storage can of course when setup properly reach 100's of MB higher using multiple drives.
Use what you have for a system backup or for secondary backup storage.
you might be able to take the drive - most cases allow you to open and swap drives - and use this:
G-Technology 1TB G-DRIVE mini High-Speed Portable Hard Drive
or this:
http://store.apple.com/us/product/HB137VC/A/g-technology-4tb-g-raid-professional -high-performance-dual-drive-hard-drive?fnode=5f

How do I select more then one channel to view in tdms file viewer graph panel

I have a TDMS file that I can view with the labview 2013 TDMS file viewer and I can select one signal out of 15 signals to view on the graph panel. however I would like be able to look at 2 or 3 signals
on the same graph but can't seem to be able to select more then one signal at a time. Is there a way to select more then one to view?

I've modified the TDMS viewer in the past to allow selecting multple channels. It's a little bit of a pain because the tree control in the viewer only allows you to select one item. You have to edit the tree to allow multiple items. Now your selection is an array instead of a scalar. This means the rest of the code has to be updated, and the references to the tree in subVIs need to be updated. Not impossible but it takes some time. Alternativly open it in Excel with the free add-in then you can select the two columns you want and insert a graph. Not as simple for the user but using Excel, which users should be familiar with might mean users need less training on data files.
Unofficial Forum Rules and Guidelines - Hooovahh - LabVIEW Overlord
If 10 out of 10 experts in any field say something is bad, you should probably take their opinion seriously.

Tdms-files: Is it possible to prevent LabVIEW and DIAdem from creating .tdms_index files?

Hello,
is it possible to prevent LabVIEW and DIAdem from creating .tdms_index files when opening/creating/editing a .tdms file?
I think I have no benefit from the .tdms_index files because our applications create a lot of little .tdms-files (repeat measurements). With the additional .tdms_index files we have unnecessary memory consumption and it takes longer for Windows to open the containing folder. Also it´s confusing when searching for a certain file.
Best Regards
Daniel
Solved!
Go to Solution.

Hi Baui,
I'm afraid there's a direct way to disable creating .tdms_index file now from LabVIEW and DIAdem. You can use TDMS Advacend API in LabVIEW, which doesn't create any index file, or you can just make some simple programming and delete the .tdms_index file after closing the TDMS file.
This is a request for a long time, we'll consider to add this feature for TDMS in future releases.
Yongqing Ye
NI R&D

I write digital port by 'DAQmx Configure Logging.vi' and receive TDMS file with 8 boolean channels. How can I write to 1 integer channel?

Hello!
I want to write 1 digital port from PXI-6536 with streaming to TDMS file.
I'm writing by 'DAQmx Configure Logging.vi' and become TDMS file with 8 boolean channels.
How can I write to 1integer channel?
Attachments:
1.JPG ‏27 KB

Hey Atrina,
The actual data stored on disk is just the raw data (that is, a byte per sample in your case). It's really just a matter of how that data is being represented in LabVIEW whenever you read back the TDMS file.
I'm not sure if there is a better way to do this, but here is a way to accomplish what you're wanting:
Read back the TDMS file as a digital waveform. Then there's a conversion function in LabVIEW called DWDT Digital to Binary. This function will convert that set of digital channels into the "port format" that you're wanting. I've attached an example of what I mean.
Note: When looking at this VI, there are a few things that the downgrade process did to the VI that I would not recommend for these TDMS files. It added a 1.0 constant on the TDMS Open function, and it set "disable buffering" on the TDMS Open function to false; you can get rid of both of those constants.
Message Edited by AndrewMc on 01-27-2010 11:21 AM
Thanks,
Andy McRorie
NI R&D
Attachments:
digitalconvert.vi ‏13 KB

How can i add a time stamp to tdms file?

I want the data and the time for the tdms file generated?
how can i do that?
should i cinvert .tdms to .lvm and use the data storage option or excel add on option for .lvm file created?
cant i directly have the time stamp for tdms file generated?

How are you acquiring the data in the first place? That will matter in how we want to approach this.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines

TDMS File Viewer not working in LabVIEW2009

Anyone know how to get this to work in 2009. I believe this came from the forums and quit working when I upgrade to 2009.
Solved!
Go to Solution.
Attachments:
TDMS - File Viewer.vi ‏63 KB

Hi bassinbc
The TDMS file Viewer ia a VI that comes with LV. I just tested it in LV2009 and it worked fine. Have you tried removing it and replacing it from the palette?
Best Regards
David
NISW
Message Edited by Davidek on 08-18-2009 09:40 AM

Blank Spaces in TDMS file Properties

hi,
I want to re write the property (in labview )" Storage date/time " with the actual value(datetime)
but what does diadem expect the property name to be as it doent accept blank spaces etc (converts everything to "_"
is there any trick for this how to insert the propertyname.
names without blank r not an issue

Hi aparab,
Check out this white paper that talks about this property and lots of other points regarding writing TDMS files with LabVIEW that work well with DIAdem and the DataFinder.
http://www.ni.com/white-paper/14252/en/
Brad Turpin
DIAdem Product Support Engineer
National Instruments

Read time of tdm file increases

Hello,
I recently recorded a lot of data in tdm format. I originally saved it in an excessively organised and complex way which resulted in large header files and was very slow. I changed it and most of the data is fine, except 6 files in the old format. Just trying to read them to get the data out and then convert it into an easier format is taking far too long. Each file has ~15000 channel groups, each with 3 channels (I won't do this again...). I have witten a small vi to test the reading speed. Excluding opening and closing of the file, if I just find a channel group by name, reading one channel from that group takes about 1 second. This would still take days to read it all but would be acceptable. The strange thing is, is that if I do this in a loop for successive channel groups which is clearly necessary, it takes longer and longer each iteration. This is behaviour that I have always seen when reading tdm files, but it hasn't caused a serious problem before. I can't find any information about why though, so I hope I'm just doing something stupid. I have attached the speed testing vi (Labview 2010). Running it for 5 iterations, the time taken to read the channels is
0.82304 s
1.58809 s
2.42514 s
3.56820 s
5.60632 s
The channels it is reading all have the same number of values, and it makes no difference if I start at a different channel group (by adding a number to i in the loop).
Does anyone have any explanation for this behaviour?
Thank you very much for your help, and I'm sorry if it's something tht has been asked before,
James
Solved!
Go to Solution.
Attachments:
ReadBadTDMSpeedTest.vi ‏44 KB

TDMS is an all binary format whereas TDM posses an XML header, otherwise they are very similar. Being a binary format you would expect it to read faster especially as you mentioned that the header was complex.
There are a number of un-closed references in your code and I wonder if this was partly the issue, although this wouldn't explain why converting to TDMS fixed the problem?
Beyond that, it would be nice to try this without using the express VIs to see if the problem still occurs.
Also, where you able to see if this was a memory issue? And if the read time continued to increase beyond what you posted?
Anyway, I am glad we found a solution to your problem.
Nick C.
Cardiff University

TDMS Data storage problem

I want to use TDMS file formats to store data from the DAQ. There are dozens of data every 10 ms, at the same time write to TDMS file. I want to ask to do so will be bad for the hard disk?

胖胖龙 wrote:
I want to use TDMS file formats to store data from the DAQ. There are dozens of data every 10 ms, at the same time write to TDMS file. I want to ask to do so will be bad for the hard disk?
Getting back to the question posted. There is a potential that the file may become fragmented as it reaches large size. There is a defragmenting tool available. There will be no harm done to the storage media on any type of hard drive you are likely to have.
That being said, I would not try it with anything left over from the 1980's
Jeff

Can't un-write protect TDM files

I am using the storage VIs to manipulate a TDM file that was created using the TDM header writer API. When I try to do any operations on it (specifically, deleting a channel), i get error -2561 "Data storage is write-protected.". I have checked the permissions on the file and they should be correct.
I ran some tests and found that i can create a TDM file using the storage VIs and delete a channel without any issues. According to LabVIEW, the file created using this method has the same permissions of the problem file.
Do you think this could be a bug with the TDM header writer VIs? Or am i missing something obvious?
I have attached files showcasing the issue.
Thanks in advance for any help.
Attachments:
TDM test.zip ‏131 KB

Bazookazuz-
The standard is described in the Developer Zone described above. TDM files are binary files written such that it follows a particular pattern. That way the data can be easily retrieved and interpreted later. It is written in the fashion to save disk space and to be created/modified quickly. All TDM files acquiring data from multiple channels follow the same format alternating between each channel taking each sample (also described in the Developer Zone article). By converting the data to End-to-End, it makes all the data for each channel together.
In order to distinguish between the modified and unmodified tdm files, there is no way to just look at the file type and know without a naming scheme. Programmatically, you can parse out the first few data sets and see if they are the same channel and then you can determine from there.
Regards,
Mike S
NI AE

Retrieving TDMS files in Directory

In the script below, I have 2 issues with the usage of DataFileLstGet :
1. The wildcard specified is (*.tdms) which I think should only look for (.tdms) files, but it is also picking up (.tmds_index) files,
     I am temporarily filtering this out by checking for the file extension length as four.
2. The function only returns that there are 2 tdms files in the directory pointed to, while in reality I have more than 2 files in this directory.
Can anyone give me an idea of what the issue could be ? I am using Diadem version 10.2 Base Edition, VB script engine is 5,5,0,5207.
Thanks.
Option Explicit
Dim intCount
Dim ExcelFilename
Dim TDMSFilename
Dim Ret
Dim FileExt
Dim FileExt1
Dim NumFiles
FileExt1 = "TDMS"
Ret = PathNameGet("TDMSFiles Location", "D:\MFT\datamine\pmm\testresults\")
Call DataDelAll(1)
If FilEx(OutPutPath & "*.TDMS") Then
Call DataFileLstGet(OutPutPath & "*.TDMS")
' Call MsgBoxDisp (DataFileLstLen)
For intCount = 1 to DataFileLstLen
' Call MsgBoxDisp (DataFileLst(intCount))
    FileExt = NameSplit(DataFileLst(intCount), "E")
' Call MsgBoxDisp (FileExt)
' Call MsgBoxDisp (Len(FileExt))
    FileName = NameSplit(DataFileLst(intCount), "N")
    If Len(FileExt) = 4 Then
        TDMSFilename = OutPutPath & DataFileLst(intCount)
        Call DataFileLoadSel(TDMSFilename,"TDMS","*/*","Load")
        ExcelFilename = OutPutPath & FileName & ".xls"
        Call ExcelExport(ExcelFilename,"", 0, OutPutPath & "EXCEL.STP")
        Call DataDelAll(1)
    End If
Next
Else
MsgBoxDisp("No TDMS files found")
End If

try this Code instead:
Option Explicit
Dim s
Dim oFSO
Dim oFolder
Dim oFiles
Dim oFile
Dim Ret
Dim ExcelFilename
Dim TDMSFilename
Ret = PathNameGet("TDMSFiles Location", "D:\MFT\datamine\pmm\testresults\")
Call DataDelAll(1)
Set oFSO = CreateObject("Scripting.FileSystemObject")
Set oFolder = oFSO.GetFolder(OutPutPath)
For Each oFile in oFolder.Files
' Do-It-Yourself Extension Filter
If UCase(NameSplit(oFile.name, "E")) = "TDMS" Then
TDMSFilename = OutPutPath & oFile.name
Call DataFileLoadSel(TDMSFilename,"TDMS","*/*","Load")
ExcelFilename = OutPutPath & FileName & ".xls"
Call ExcelExport(ExcelFilename,"", 0, OutPutPath & "EXCEL.STP")
Call DataDelAll(1)
End If
Next
Matthias Alleweldt
Project Engineer / Projektingenieur
Twigeater?

.tdm file storage vi's slow editing vi's

Similar Messages

Maybe you are looking for