Number of elements in TDMS file

Is there a way to determine the number of elements in a TDMS file? The files I am creating have several signals all of the same length. The only way I can think of is to do this is put it in a loop and read one element at a time until it reaches the end of the file. But, my files will have millions of elements. Any suggestions?
Thanks.
--Robert

Thanks Herbert
For readers having similar problems, here are my experiences:
It works either without wiring a property name and datatype, as you mentionned before.
Resulting is a variant array. Result see attached solutionVariant.jpg
It also works wiring the right datatype as in the TDMS File Viewer and your first explanations:
Resulting is the channel length as Integer I64 or U64. Datatype constant representation and indicator representation must match!
See attached solution.jpg
My error has been wiring a string "I64" as datatype and a string indicator as value indicator. So stupid...
Regards
Klaus
Attachments:
solutionVariant.jpg ‏30 KB
solution.jpg ‏8 KB

Similar Messages

Number of elements in XML file

Hi,
I have an xml file as follows
<xml>
<head>
<h1>xxxx</h1>
</head>
<line>
<name>ravi</name>
<dept>Oracle</dept>
</line>
<line>
<name>Kamal</name>
<dept>Java</dept>
</line>
<line>
<name>Ajay</name>
<dept>Linux</dept>
</line>
</xml>
I need to insert the above xml file into a table.
For that i have used the following query.
insert into dest_table
SELECT     EXTRACTVALUE (t2.xml_doc, '/xml/head/h1') head,
EXTRACTVALUE (t2.xml_doc, '/xml/line['||level||']/name') name,
EXTRACTVALUE (t2.xml_doc, '/xml/line['||level||']/dept')dept
     FROM xml_table t2
     CONNECT BY LEVEL <= n
Here n in CONNECT BY clause is the number "<line>" elements (3 in the above example).
If i know the number of <LINE> elements,then i can easily select and insert the xml data.
BUT How to know the number of particular elements(<line> in above xml) in the above xml using oracle xml oracle.
any idea psl.
cheers
RRK

select count(1) from table(
    xmlsequence(
        extract(
            XMLType('<root><a>b</a><a>c</a><a>d</a><x>z</x></root>'),
            '/root/a'
)

Number of Groups in TDMS file?

Hello,
Is there a way I can read the number of groups in a pre-existing TDMS file? I want the groups to be named numerically so if I know how many groups there are in a file that was opened, I can automatically set the group name for the next one.
Kenny
Solved!
Go to Solution.

TDMS List Contents is what you're looking for.
You can then do an array size on the Group names list that comes out.

How does DIADEM import TDMS files? How gets every channel his number and groupindex? How can I determine which channel has which groupindex and number?

I store different channels in a TDMS file.
I like to have a time channel at the first position with group index 1 and number 1.
When I read the TDMS file with DIADEM the time channel (Float64) is on a differernt position, and the channels are not sorted alphabetically.
Here are my questions:
How does DIADEM import TDMS files?
How gets every channel his number and groupindex?
How can I determine which channel has which groupindex and number?
Best regards
Joerg

Hi Jörg,
i suppose that you´re programme whose create the *.tdms file is writing on false position. Try to create datas with timechannel on first indes in diadem, then save it and then open it again. you see that all is correct. So please tell me what programm in what version do you use and please attache it here.
Did you use the library for creating *.tdms files like in the link ?
http://zone.ni.com/devzone/cda/tut/p/id/6471
Here you find the gtdms_8.x.zip - when you extract it and opened the *.llb you find vi´s for all functions e.g. writing 2d array of strings to *.tdms file
when you open the subvi´s then you see how created and writing datas/structure to *.tdms files. Because *.tdms is binary you can´t see structure with open it in editor.
When you don´t have Labview you can use the 30 days test of current version 8.5 under following link
german version download link
https://lumen.ni.com/nicif/d/lveval/content.xhtml
english version download link
https://lumen.ni.com/nicif/us/lveval/content.xhtml
Hope it helps
Best Regards

Best way how to write FPGA data in rt cRIO system in tdms file

Hej,
I am struggling to write measured data from an analog input (NI 9215) sampled at up to 20 kHz to a tdms file in the rt system (crio-9022). I just need to save several periods of 4 arbitrary analog signals at frequencies between 5 Hz and 1KHz. So storing up to 50k values should already be enough.
I use a high priority and a low priority loop. First I tried to adapt the example from the "Getting Started with CompactRIO - Logging Data to Disk" (http://zone.ni.com/devzone/cda/tut/p/id/11198). But when I used this in my high priority loop (running at 1ms), the loop runs out of time and the rt system becomes unresponsible. If I change the number of elements to write (the number of elements to wait for in the fifo read block) it becomes better, but still data is lost because the loop finishes late.
So I was thinking to create a RT fifo and to store all the values from the measurement first in this memory inside the high priority loop and then write the values to the tdms file in the low priority loop. This time I used the read/write fpga block instead of the FPGA fifo block. It was already working better but writing the files to the tdms file took a lot of time since each value was read and written to the tdms file individually. Unfortunately I could not find a possibility how to write the whole rt fifo to the tdms file at once. Is there a block available or is it possible to create a big array first and then write the data to the tdms file at once? My code I tried is in this second picture.
I hope someone can give me some tips which method should be better for my project and a hint what I did wrong or what I can optimize. I stucked for days now on how to save my measurements on the cRIO system.
Thank you very much in advance. Have a nice weekend.
Best regards
Andy
Solved!
Go to Solution.

HiXiebo and Christian,
thank you very much for your answers. Actually, my high priority loop is much slower. I run it with a maximum loop time of 50us = 20kHz or slower, depending on my Signal I want to measure. So my data producing rate is maximum 4*8*20k=640 KB/s. My low priority loop runs at 2ms to 5 ms (much slower then the high priority loop), since I am doing just some simple math calculation there and control the front panel in this loop.
I understand that it is much more efficient to write blocks of data (e.g. 1024*32KB instead of just 32KB) to a file with TDMS. But is it also the same for a queue or RT FIFO, i.e. does the block size of the data chunks also matter for the queue and RT FIFO?
@Xiebo: I understand that caching the read data from the FPGA in the high priority loop first will improve my code. But I do not know how I can cache the data I read? I was thinking to do it with the FPGA FIFO, but the FPGA read/write blocks seem to be faster for me and I do not know why? Can you tell me a block/vi to cache the data I read from the FPGA or maybe even an example?
@Christian: This NI_MinimumBufferSize property looks exactly what I was looking for. But my question is now if I should put the tdms write VI's in my high priority loop and read directly from the FPGA FIFO buffer to the file as it is done in the Disk logging example at http://zone.ni.com/devzone/cda/tut/p/id/11198? Or is it better to read the data from the FPGA via the FPGA read/write function, write the data to a RT FIFO in the high priority loop and then write the data with the NI_MinimumBufferSize property option to the tdms file from the RT FIFO in the low priority loop?
In summary, I am still unsure if the FPGA FIFO or the FPGA read/write function with a queue or RT FIFO is better for me and how I can create a cache to build chunks of data blocks to write.
Thank you very much in advance for your help.
Best regards
Andy

Concatonate TDMS file data to live data

I am monitoring 20 or 30 channels at any given time and storing all data to a TDMS file (LV 8.6). At any given time we are only displaying 3 channels on a waveform chart. The chart displays 5 minutes of data. say for the first minute I am viewing channels 1, 2, & 3. Then I switch over to 4 - 6 but in addition to the live data coming in I want to see what has happened since the test started a minute ago (or up to 5 minutes ago), as if I had been viewing these three channels all along.
What I have tried is to read the stored data from the TDMS file, copy the most recent data (up to 15K samples = 5 min at 50Hz). Then I take the live acquired waveforms, break out the array of Y values, concatonate with the Y values from file and rebuild the wavform. But it isn't workin. Seems like it should but I don't know what is wrong.
Any ideas?
Lawrence M. David Jr.
Certified LabVIEW Architect
cell: 516.819.9711
http://www.aleconsultants.com
[email protected]
Attachments:
extendedWaveforms.vi ‏15 KB

well I tried using the "ignore attributes" property, both above the 'value' property, and below (top down ordering - didn't make any difference). Also I verified that the data is being concatonated; normally we are adding one point to the plot every 50mS, when we toggle to a different set of channels, the Y component of the waveform is not an array of one element but of (in this shot anyway) over 9000 elements. the screne seems to flash with a block of data and then the plot resumes filling 1 point at a time from the present time stamp. Maybe I can set an attribute of the x axis to be an earlier timestamp based on the number of points in the Y array?
Lawrence M. David Jr.
Certified LabVIEW Architect
cell: 516.819.9711
http://www.aleconsultants.com
[email protected]
Attachments:
propertyNode.JPG ‏11 KB
probes.JPG ‏45 KB

How to get the last data point from a TDM file in LabVIEW?

Hello,
I am using LabVIEW to analyze some rather large TDM files, and I need a way to get only the last data point. So far, the only way I have been able to accomplish this is by reading the entire file. Is there a property in the TDM file or a function in LabVIEW that will allow me to get the index of the last item in a channel?
Thanks!
Christina

Do you want to avoid reading whole file and want to be able to reach or get the index of last value of channel? is there any specific reason? I am not sure you could do it without loading the whole file. But the easiest way would be just to use array functions "array size" would give you the index of last element.
-Nilesh
Kudos are (always) welcome for the good post. :-)

Number of elements at logical level / odbc driver

Hi everyone,
I'm new in BI area and I cannot understand one basic thing.
I study documentation about BI server, exactly chapter about "creating dimensions" and I'm confused about setting "number of elements at logical level"
It looks for me that "number of elements at logical level" is static.
What will be when someone update data in the table and number of elements will be different than number of elements in our logical level?
example:
current column with distinct values:
channel_desc:
direct sales
tele sales
catalog
internet
partners
in this case I set number of elements at logical level 5.
after a while some update data in the table and our column looks (see below)
column after update with distinct values:
channel_desc:
direct sales
tele sales
catalog
internet
partners
others (added value)
Does it mean that I have to change it to current numbers of elements?
I'm sure that this is misunderstanding in my case, but I don't know where?
Second problem which I have:
BI with oracle database is on linux
client software on windows
I can import data from database like sybase or flat files to the physical model, I can view data, but when I create whole model (physical, bussines model and mapping, presentation) and I try access data from "Answers" I have this error:
"Data source name not found and no default driver specified"
regards
Bart

Number of elements is just an indication used by the BI Server (e.g. when generating LOV's in Answers). It won't cause any problems when it doesn't hold the exact number of elements available in the dataset.

How to store data (in tdm files)

I'm looking for advice on how to store data in tdm/tdx files.
The main challenge is like this: At regular time intervals there is new data available (say every 10 minutes there are 10 minutes of new data available). All data is time based. So every ten minutes I may create a new tdm/tdx file containing this data segment. However, when i want to analyze the data i dont necessarily want to view 10 minutes of data from all channels, but maybe 20 hours of data from one particular channel.
The way I've achieved this so far is to manually load each 10 minute segment of this particular channel and then add these parts together. This is both time consuming and cumbersome. Is there a better way to do this that I've simply not discovered yet? Is there a better way to store data to simplify this process?
One solution is of course to save data from several 10-minute frames in one file, but seeing as there is a never ending supply of data i cant simply save all data in one giant file, it has to be split up at some point, and the problem will still remain.
One factor to have in mind here is that this is rather large amounts of data (maybe 10GB each day), so the option to simply load all the data into memory goes out the window rather quickly.
Feel free to ask if I've not made myself very clear

Hi salte,
If your test ever ended, and if you had LabVIEW 8.20 or higher, I would recommend using a TDMS file, which handles data appeands flawlessly. DIAdem 10.1 now also does data reduction and index windowing during file loading for TDM / TDMS / DataPlugin files, so that you could easily load out only the part of the file you wanted to look at. But since you describe your data acquisition as never stopping and amassing 10 GB per day, I agree that it would be impractical to use only 1 data file. So we are stuck with some number of files which each contain a part of your measurement. This approach can have advantages, since you can save operational properties for each "buffer" such as average value, dominant frequency, ambient room temperature, etc., and later on you can use the DataFinder to query out only the data buffers which meet specific conditions based on these properties. The problem remains to load and assemble data based on multiple files. This is an old problem in DIAdem, and one for which I have an efficient and I hope satisfactory workaround application. It does what you describe already doing in the minimum amount of time and with the minimum amount of user interaction, and it can be highly parametrized to suit your particular situation. The ideal way to start the application would be as part of a ResultsList custom menu. This would enable you to query out the buffers you wanted, highlight those rows in the ResultsList, right-click and choose your custom menu, and WHAM! the selected buffers for your queried channel(s) are automatically appended together in the DataPortal. Launching from a ResultsList custom menu would mean that you could skip the file dialog and just read out the data sources directly from the ResultsList selection.
Let me know what you think,
Brad Turpin
DIAdem Product Support Engineer
National Instruments
Attachments:
Importing Data from Multiple DataPlugin Files.zip ‏198 KB

Reading TDMS files in Matlab

I am trying to read TDMS files directly into Matlab using the NI nilibddc.dll. I have read most of the posts on here but still have a problem...
Within Matlab I can open and read a file and get the data but I cant find anything that tells me what the sample interval (between samples) is for a particular file. I guess its in the header somewhere but how do I access it?
I cant find anything that tells what function to use to get it from the header.
In fact i cant find anyhting that tells me what the functions are in the DLL.
I tried opening the CHM help file, this has a list of all the functions but when you click on one in the contents it reports 'the address is not valid'. It seems the link is borken somehow in the help file (this is on XP)!
any help gratefully receieved!
mike

hi mike,
if your data is stored in form of waveforms, this information can be retrieved from the channel:
Entity
Property
Data type
Description
tdm_channel
wf_xname
string
Name of the x-channel of the waveform
tdm_channel
wf_xunit_string
string
Unit text of the x-channel
tdm_channel
wf_start_time
time
Time stamp for the waveform
tdm_channel
wf_start_offset
double
Starting offset (to be added to the time stamp)
tdm_channel
wf_increment
double
Increment
tdm_channel
wf_samples
long
Number of samples in one shot (needed by LabVIEW when concatenating
more consecutive shots). This property is used by DIAdem and DataFinder to decide whether the tdm_channel is a waveform (wf_samples > 0) or not (wf_samples = 0, or not existing).
tdm_channel
wf_time_pref
string
(enum: "absolute" or "relative") Additional information for LabVIEW to
determine if the x-channel is treated as time (absolute) or as index
(relative). (not available in LVM)

How do you save arrays in tdm files?

Hi,
As part of a university project I am trying to save some program data to a .tdm file. On each channel I wanted a number of different types of data including an analogue wave form and a couple of 1-D DBL arrays. I have found an NI presentation that says that you can store arrays in them but dont seem to be having any luck. Any ideas what Im doing wrong? I am using labview 8.5.
Regards,
James

Hi James,
in a channel, you can save only one array as data. If you want to save more arrays then use several channels. You can organize them in groups. If the arrays must be in the same channel, then convert them to a spreadsheet string (in string palette) and save the string as a property.
Greetings,
shb

DIAdem not saving all samples from TDMS file

Hello to all,
I have several large TDMS files (1.8Gb each) containing approx. 120 million sampels. When I try to open these fiels in Diadem it previews the correct amount of samples (120 Million), but when I try to save tha data in Matlab format it saves me only half the samples, 60 Million. Also, I have a Matlab script that opens the TDMS files and does exactly the same thing, it is not saving all samples.
Can you please advise me on this issue? Is there a way to automatically fragment large TDMS files into smaller ones?
Best regards,
Ion

Hello Jean Baptiste,
It seems your TDMS file is fine since you have the correct number of data in DIAdem.
It may comes from the conversion to the MatLab format. Maybe MatLab can't accept more than 60 millions data. You should ask that to the MatLab expert on their forum.
However, if you want to fragment your TDMS file, you have to do it manually in DIAdem, when you load the file into the Data Portal, or you can automate the function by using VBScript. (Last panel in DIAdem)
An other solution would be to fragment your TDMS file directly where you write it. (In LabVIEW maybe?)
Regards,
Benoit S. - Field Sales Engineer
Certified LabVIEW Developer
Certified TestStand Developer
National Instruments France
#adMrkt{text-align: center;font-size:11px; font-weight: bold;} #adMrkt a {text-decoration: none;} #adMrkt a:hover{font-size: 9px;} #adMrkt a span{display: none;} #adMrkt a:hover span{display: block;}
Été de LabVIEW 2014
12 présentations en ligne, du 30 juin au 18 juillet

Number of elements in the FP affecting the performance of my application ?

I developed a simple application which consists of aquiring a signal through a channel of a PXI 5102 module and commanding some relays of a SCXI 1161 module.
This application has the following features :
-The user can, at any time, change the configuration of the PXI 5102 module.
-The signal acquired can be saved to a file whenever the user wants.
-Instead of showing the signal acquired through the PXI 5102 channel, the user can load a given file which contains a saved signal.
-The user can increase or decrease the pressure on a line through a VI that commands SCXI 1161 relays.
As you can see, it is a fairly simple application but the problem is that it have been running for a time the fron
t panel crashes. Is it because the high number of elements I have in the front panel ?
To give you more information I tell you that the processes of saving, loading, decreasing and increasing the pressure on the line are event cases of an event structure.
The front panel consists of a tab structure, 20 controls and indicators and a cluster consisting of 9 controls.

No error take place. By crashing I mean that after a period of time the frontpanel control´s response to inputs of the user become so slow that after a short time there´s no response at all.
The problem is that I´ve been dealing with this problem in this aplication for quite a while and although I´ve optimized, I think, its execution time and the way it deals with memory usage this problem persists.
Since most of the operations performed by it depends on inputs by the user I´ve decided to use an event structure to manage them and decided to load the VIs related to them dynamically into memory to somewoh optimize memory usage.
In order to make things more clear to you I´m sending you a zipped file containing pics that completely describe the frontpane
l ad the block diagram of the appliction.
Any help you can give will be deeply appreciated.
PS : I´ve tried to sent the zipped file but awindow appeared notifying me that Developer Exchange is down for maintenance. If you need any more information to clarify what I mean please let me know.
Sincerely,
Giovani Marcelino Nesi
DRV Desenvolvimento e Representação Virtual Ltda.

Total number of elements in an array

Hello,
I am trying to count the total number of elements in an array.
I have a simulate signal VI, that simulates a square pulse, this is being written into an excel file using the write to measurement VI, all for a given period of time.
I want to further chop a window of the signal depending on different paramenters (a trigger, which I intend to incorperate later, for now I am working of learning how to chop the data)
In order to chop the data I need to know the array size, I use the array size funtion, I right click on the appeneded array in the front pannel and click show last element (array size -1) it doesnt match with the number of rows in the excel file written by the write to measurement VI, in fact it macthes with the "number of samples" I selected in the simulate signal VI.
Am I doing something wrong?
Why is there a miss match??
I have attached my VI, Please help
Thank You
SP
Attachments:
FlashERGTrialVI 150218.vi ‏93 KB

Hello RavensFan
Thank you for the reply, I did incorperate the changes, but the same number is displayed in the indicator. I've attahced my VI
SP
Attachments:
FlashERGTrialVI 150218.vi ‏106 KB

How to store configuration data from clusters to TDMS files

It is a common use case to write measured data to a TDMS file, while
the configuration information for your test is stored in a cluster (and
that cluster might contain additional clusters, arrays etc). TDMS does not have
native support for clusters and some other data types.
This LAVA discussion thread
contains a VI library that breaks clusters of arbitrary complexity down
to their basic elements and writes their contents to TDMS properties
either at file, group or channel level. It also contains code that
reads your data back from TDMS and recreates the original cluster type.
Please review the whole thread in order to understand what to expect and to make sure you have the latest version.
Herbert

Change
while (rs.next()) {
     System.out.println(rs.getString("piece_id") + "," + rs.getString("piece_num")+ "," + rs.getString("piece_ver"));
     FileOutputStream fileobj = new FileOutputStream(f);
     DataOutputStream writeInsideFileObj = new DataOutputStream(fileobj);
     writeInsideFileObj.writeBytes(rs.getString("piece_id"));
     writeInsideFileObj.writeBytes(",");
     writeInsideFileObj.writeBytes(rs.getString("piece_num"));
     writeInsideFileObj.writeBytes(",");
     writeInsideFileObj.writeBytes(rs.getString("piece_ver"));
     fileobj.close();     //here my data is replacing and only last row is storing..Kindly help me it is very urgent
}to
FileOutputStream fileobj = new FileOutputStream(f);
DataOutputStream writeInsideFileObj = new DataOutputStream(fileobj);
while (rs.next()) {
     System.out.println(rs.getString("piece_id") + "," + rs.getString("piece_num")+ "," + rs.getString("piece_ver"));
     writeInsideFileObj.writeBytes(rs.getString("piece_id"));
     writeInsideFileObj.writeBytes(",");
     writeInsideFileObj.writeBytes(rs.getString("piece_num"));
     writeInsideFileObj.writeBytes(",");
     writeInsideFileObj.writeBytes(rs.getString("piece_ver"));
fileobj.close();- Roy

Number of elements in TDMS file

Similar Messages

Maybe you are looking for