Concatenat​ing TDMS Files

Question

Hello,
I'm writing a script in DIAdem 2011 which combines data from multiple sources into a single TDMS-File. Exporting into multiple TDMS files works fine( I can open them in DIAdem) but I'm having trouble combining all files to one TDMS-File. Speed is important, so I tried to avoid loading all data into the dataportal and saving again (this works though).
If I understood correctly it is possibly to directly concatenate the binary contents of the files into a single file. Is this correct? Because using the MS-Dos copy command or using a VBS-Script in DIAdem with Stream-Objects both results in TDMS-Files that I can't open anymore.
The error message says:
"TDS Exception in Intialize: Tds Error: TdsErrNotSupported(53):"
It seems only one of the files produces these problems. When I load this one file and save it again as TDMS I can sucessfully combine them and load the resulting TDMS without errors.
Perhaps the file is fragmented and this leads to the error (the offending file is measurement data)? Is this expected behaviour for fragmented files?
How can i efficiently combine these files? I have to be able to combine multiple hundreds of files ( in pairs. e.g. combine 400 TDMS in pairs to 200 TDMS files) fast and i guess opening 200 files, saving again and combining is rather slow.
Is there an efficient way to solve this task?
As the offending file is actually TDMS-Data from a BLOB-Datafield in a SQL-database another solution might be loading the TDMS directly from the database into the dataportal and saving from DIAdem, instead of dumping the BLOB-data into a TDMS-File on the HDD( i guess this results in a defragmented file as opening and saving resolves the problems).
But I'm not sure how to achieve this. I'm using ADO to interface with the SQL-DB. The code looks approximately like this:
Set recordSet = CreateObject("ADODB.Recordset")
recordSet.Source = 'QUERY
recordSet.open
Set fso = CreateObject("Scripting.FileSystemObject")
'Create Stream object
Set BinaryStream = CreateObject("ADODB.Stream")
do
chunkSize=100
lngOffset=0
lngFileSize=recordSet.Fields.Item("Field").ActualS​ize
'Open the stream And write binary data To the object
BinaryStream.Open
do while lngOffset<lngFileSize
varChunk=RecordSet.Fields.Item("TDMS_DataInBLOB").​GetChunk(chunkSize)
Call BinaryStream.Write(varChunk)
lngOffset=lngOffset+chunkSize
loop
'Save binary data To disk
Call BinaryStream.SaveToFile("path", adSaveCreateOverWrite)
BinaryStream.Close
Call RecordSet.MoveNext
loop until RecordSet.EOF
 How can I change this code to write the data into the dataportal?
Thanks in adavance.
Best regards,
grmume

Answer 1

Hi Brad Turpin,
I have already tried concatenating the files with the copy command. I don't have a hex-editor at work but by looking at the filesizes and checking the contents with a texteditor the script seems to work but I can't open the created file.
The script I used looks like this:
private Sub mergeFiles(filePath1,filePath2,outFilePath,bDelete​Input)
if bDeleteInput then
Call ExtProgram("cmd.exe","/Q /c copy /b """&filePath1&""" + """&filePath2&""" /b """&outFilePath&""" &del """&filePath1&"""&del """&filePath2&"""")
else
Call ExtProgram("cmd.exe","/Q /c copy /b """&filePath1&""" + """&filePath2&""" /b """&outFilePath&"""")
end if
End Sub
 To test your script I defined the following variables:
NewFileName="C:\Documents and Settings\username\Desktop\new folder\combined.TDMS"
ChanNames(0)="file1"
ChanNames(1)="file2"
Folder="C:\Documents and Settings\username\Desktop\new folder"
DosDrv="C:\Documents and Settings\username\Desktop\BatchFile"
 It completed sucessfully but the result is the same. I can open the two seperate files(file1.tdms and file2.tdms) but I cannot open the combined.tdms.
Sadly I can't share the original files and I can't use DIAdem to fill the files with dummydata because opening in DIAdem and saving resolves the problems.
Do you have an idea what causes these problems? The possibility to combine tdms-files like that was a big bonus for me...
Can I somehow fill the files with dummydata without opening them in DIAdem? Or tell DIAdem to keep the original file structure but change the values?
Best regards,
grmume