A File Reference and its evolving life!

Question

Hi all,
I've noticed something that came as a little bit of a surprise to me, but I think I have the explanation, at a hand-wavy higher level anyway.  What I have not established is if this is a 'bug' or a 'feature', and if there are any ways the following issue can be avoided at the NI function/api layer.
Consider the file open and file close function.  You open a file, you use the reference to the file to write/read data, then at some point you close the reference and the close function spits out the file-path.  Here are a couple of tid-bits you may not be aware of (that are easy to test):
Q1) After your application opens/creates a file and starts using the file-reference to make file writes, if an external source changes the file-name of that file... guess what will happen on your next write function call?
A1::  The write successfully updates the newly re-named file with your new data without producing an error or a warning.  (At least this is the case if your program is running on a vxWorks cRIO target and the file-name is changed directly on the cRIO via an FTP browser.)  
Did this surprise you? It did surprise me!  -My handwavy explanation is that the file-pointer is perhaps managed/maintained by the OS, so when the OS tells the file-system to rename that file, the pointer that LabVIEW holds remains valid and the contents of the memory at the pointer location was updated by the OS.
Q2) Continuing from the situation setup in Q1, after writing several new chunks of data to a file now currently named something completely different than when the file reference was originally created, you use the close function to close the file-reference.  What do you expect on the file-path output from the close function??  What do you actually get??
A2::  The close function will 'happily' return the ORIGINAL file-name, not the actual file-name it has been successfully writing to(!).   This has some potentially significant ramifications on how/what you can use that output for.  At this point there is a ton of room for pontifications and more or less 'crazy' schemes for what one could do, but I argue that the bottom line is that your application has at that point completely lost the ability to accurately and securely track your file(s).  Yes, you could list a folder and try and 'figure out' if your file-name was re-named during writing and you can in various ways make more or less good 'guesses' on which file you in reality just had open, but you can never really know for sure.
So, what do you guys think?? Is the behavior of returning the (incorrect) original file-path when you close the handle a BUG or a FEATURE??  Would it not be possible for LabVIEW to read back the data contained in the (OS?) pointer location and as needed update the file out path data when it closes a reference?  Should we not EXPECT that this would be the behavior?
Q3)  Again, continuning from the above situation, lets assume we are back at the state in Q1, writing data to a (re)named file.  What happens if the file is deleted by an external process? What happens to the file reference? File function calls using the reference?
A3::  This one is less surprising.  The file reference remains 'valid' (because it is a valid reference), but depending on the file function you are calling, you will get error such as error 6 (binary write reports this), or error 4 (a TDMS write will report this error), etc.  So as long as you don't rely on file ref-num tests to establish if you are good to go with a file-write or file-action, you should be safe to recover in an appropriate way.. Just don't forget to close the file-reference, even if the file is 'gone', the reference will still remain in memory until you 'close' it (with an error)(?I might be wrong about this last part?)
I am not sure if the above is possible on e.g. Windows, Windows would probably prevent you from re-naming a file that has an open file-handle to it, but this is definitley observable on at least vxWorks cRIO targets.  (I don't have PharLap ETS or RTLinux devices so I can't test on those targets.. if you want to test its pretty straigth forward to make a simple test app for it.)
[begin rant-mode related to why I found this out and why this behavior BITES]
There are situations where the above situation could cause some rather annoying issues that, for somewhat contrived reasons related to cRIO file API performance, CPU and memory resource management, are non-trivial to work around.  for example, using the NI "list folder" to listing folders take a very hefty chunk of time at 100% cpu that you cannot break up, so polling/listing folders after every file update (or even on a less regular interval) is a big challenge, and if you are really unlucky (or didn't know any better) and gave the list command in a folder with 1000's of files (as opposed to less than about 100 files), the list will lock your CPU at 100% for 10's of seconds...  Therefore, you might be tempted to maintain your own look-up table of files so that your application can upload/push/transfer and/or delete files as dictated by your application specific conditions... except that only works until some prankster or well-intention person remotes in and starts changing file-names, because then your carefully maintained list of file-names/paths' suddenly fall appart.
[\end rant]
QFang
CLD LabVIEW 7.1 to 2013

Answer 1

Hey guys, thanks for turning out your comments on this thread!
-Deny Access : still able to re-name (and delete) the file via FTP browser (didn't test other file avenues).  I think this is for the same reason that NI vxWorks targets (such as cRIO-9014) do not support the concept of different users with different rights, as such, everyone have access rights to everything at the OS level.  Another issue for me would be that "Deny Access" does not work on TDMS file references, so even if it worked, it would not help me.
--> I strongly suspect that these things are non-issues or issues that can be properly managed, on the new NI LinuxRT targets since (the ftp is disabled by default) it supports user accounts and user restrictions on files/folders.  The controller could simply create the files in a tree where 'nobody else' has write access.
Obviously nobody should mess around with files on a (running) cRIO, but customers don't always do what they are supposed to do.   
As far as the 'resources' or overhead to update the file-refnum with the new information, this would not be needed to be done in a polling fashion, simply, when the file-close function is called, as part of that call it updates its internal register from the pointer data, so this should be a low overhead operation I would think?  If that is a true concern, a boolean input defaulting to not updating or a separate 'advanced close' could be created?
I've included a zip with the LV2013 project and test VI's (one for tdms one for binary) that I've used. nothing fancy, but in the interest of full disclosure.  The snippet is the 'binary file' test vi, in case you just want a quick peak:
Steve Bird's findings of (yet) another behavior on Pharlap systems is also very interesting, I think!!
[EDIT]  JUST TO CLARIFY, on vxWorks, the re-named file keeps being successfully written to, unlike the PharLaps' empty file that Steve Bird found.
QFang
CLD LabVIEW 7.1 to 2013
Attachments:
cRIO Tests.7z ‏30 KB