Using auto-indexing tunnels to write data files?

Hi,
I am using ni-motion controller to control a servo motor and collect position and torque data. I want to write the collected data to TDMS files.
Recently I learned about the producer/consumer design pattern and I figured that would be a good approach to ensure that writing the files didn't slow down my data collection timed loop.
However, I also figured out that my program seems to run well if I wire the data I collect to auto-indexing tunnels. Then I use a structure that only executes after all the data collection is done to write the arrays I built to TDMS files.
Is there any reason that the latter method would be advised against? Can the auto idexing tunnels slow down my loop enough for it cause concern? I am only collecting about 5000 data points for each channel.
Cheers,
Kenny
Solved!
Go to Solution.

Well, autoindexing tunnels don't write data files, they just accumulate data until the loop completes. If this is a FOR loop with a known number of iterations, the size of the output data can be allocated in one swoop, which is very efficient. If you are autoindexing on a WHILE loop, the final array size is not known, so LabVIEW needs to make a guess and and requires occasional new memory allocations whenever the last guess is exceeded. This is inefficient.
One problem with these approaches is if the program or computer crashes. In this case the data in the shift register is lost forever, while if you would stream it to disk, you would have most of the data acquired so far.
If you use a proper producer/consumer architecture, you should be able to asynchronously write the data and it will not slow down your acquisition. No need ot wait for the competion of data gathering.
LabVIEW Champion . Do more with less code and in less time .

Similar Messages

Auto index only shows partial data

Hi everyone,
Please see the attached file. I've added notes where the data goes missing. I know it has something to do with auto-indexing and I spent quite a bit of time reading about it and trying different things but I really need some help now please.
Thank you!
Solved!
Go to Solution.
Attachments:
For forum.vi ‏47 KB
For forum.xlsx ‏12 KB

Think about what you're trying to do.
You have three arrays you're indexing. Let's say they're lengths 1, 2, and 3.
In the first iteration, we'd index 0 and get back a value from all three.
In the second iteration, we'd index 1 and get back two values and have a bounds conflict.
In the third iteration, we'd index 2 and get back a single value while having two bounds conflicts.
Anything you process in the second and third iteration is garbage. You have bad inputs so the output isn't useful to you. In some languages, this will throw an error at runtime for indexing out of bounds. Some will just let you process the garbage. In no case do you WANT to run this.
If you want to process all the elements of the array of size 3, you need to fill in values for the first to arrays to make them all have a length of 3. Once you do that, the loop will run as you desire. If you have auto-indexed tunnels with fewer elements than the constant you wire, they will override the terminal, as they should.

Satellite P500/01R reads data but cannot write data files to CD

I recently bought a Satellite P500/01R laptop. I am able to read a data CD but cannot write data files to CD. The CD is read/write.
Any ideas?

Hi
Did you try different burning software?
Im using the Nero and in my opinion one of the best burning applications.
There I can choose between CD, DVD, DL or Blueray (only possible if CD/DVD drive supports this too.)
Did you also check different CDs from different manufacturers?
You should do this since not all disks are 100% compatible.
I had many issues with different CD or DVD in most cases it was an compatibility issue I tested TDK and these kind of disk is compatible with my CD/DVD drive
Im using the R standard.

I used a partitioned HDD for time machine, using a partition already containing other data files. I am now no longer able to view that partition in Finder. Disk Utility shows it in grey and "not mounted". Any suggestions of how to access the files?

I used a partitioned HDD for time machine, using a partition already containing other data files. I am now no longer able to view that partition in Finder. Disk Utility shows it in grey and "not mounted". Any suggestions of how to access the files? Does using time machine mean that that partition is no longer able to be used as it used to be?
HDD is a Toshiba 1TB, partitioned into two 500GB partitions.
OS X version 10.9.2

Yes, sharing a TM disk is a bad idea, and disks are cheap enough so that you don't need to.
Now
Have you tried to repair the disk yet

Array Size doesn't work correctly after "Auto-indexed Tunnel"

Array Size returns value even if Empty Array is detected after "Auto-indexed Tunnel" tha generates 2D array.
See example + VI with correct output as I expected.
OR
There is problem with generating 2D array --- It's empty. 1D array is generated in right way. 100 x 0 but not Emptu Array.
Attachments:
ArraySize_Bug.vi ‏9 KB
ArraySize_Corrected.vi ‏19 KB

Not a bug. This is correct behavior that is consistent with other parts of LabVIEW.
The decision to handle multi-dimensional arrays like this was made long ago and is an enshrined part of the LabVIEW language spec. It will not be changed.

Creating an array without using "auto-indexing"

Hello,
it is terrible! I'm new in working with LV5.0 (Evaluation) and I can't
find any examples how to write data into an array without using the
function "auto-indexing" of the "WHILE" or "FOR"-loops..
The values do not appear all at the same time, so I can't use the
"create_array" function.
So I want to "collect" all occured values and then write them e.g. in a
file.
Did I miss anything in the description ???
Who can help me?
Thanks, CHZ

CHZ wrote in message news:[email protected]..
> Hello,
> it is terrible! I'm new in working with LV5.0 (Evaluation) and I can't
> find any examples how to write data into an array without using the
> function "auto-indexing" of the "WHILE" or "FOR"-loops..
> The values do not appear all at the same time, so I can't use the
> "create_array" function.
> So I want to "collect" all occured values and then write them e.g. in a
> file.
The "Create array" function can be configured to take arrays as well as
elements. You have to use a loop that iterates as often as necessary,
checking for new data. When it receives new data, it adds the new data to
the existing array. It's probably easier to attach a small example than to
describe it in more
detail.
[Attachment Array.zip, see below]
Attachments:
Array.zip ‏10 KB

How can I read/write data files (text file) from PL/SQL Script

I had an oracle forms pl/sql program to read/write a data file (text file). When this code is run on a command line as a PL/SQL script using the SQL*Plus I am getting an error:
-- sample.sql
DECLARE
vLocation                VARCHAR2(50) := 'r:\';
vFilename                VARCHAR2(100) := 'sample.dat';
vTio                   TEXT_IO.FILE_TYPE;
vLinebuf               VARCHAR2(2000);
vRownum               NUMBER        := 0;
-- use array to store data FROM each line of the text file
TYPE           array_type IS VARRAY(15) OF VARCHAR2(100);
vColumn      array_type := array_type('');
PROCEDURE prc_open_file(p_filename IN VARCHAR, p_access IN VARCHAR2) is
BEGIN
    vTio := TEXT_IO.FOPEN(vLocation||p_filename,p_access);
EXCEPTION
    WHEN OTHERS then
      -- raise_application_error(-20000,'Unable to open '||p_filename);
      message(sqlerrm);pause;
END;
PROCEDURE prc_close_file is
BEGIN
    IF TEXT_IO.IS_OPEN(vTio) then
       TEXT_IO.FCLOSE(vTio);
    END IF;
END;
BEGIN
--extend AND initialize the array to 4 columns
vColumn.EXTEND(4,1);
prc_open_file(vFilename,'r');
LOOP
      LTEXT_IO.GET_LINE(vTio,vLinebuf);
      vColumn(1) := SUBSTR(vLineBuf, 1, 3);
      vColumn(2) := SUBSTR(vLineBuf, 5, 8);
      vColumn(3) := SUBSTR(vLineBuf,10,14);
      Insert Into MySampleTable
      Values
        (vColumn(1), vColumn(2), vColumn(3));
      EXIT WHEN vLinebuf IS NULL;
   END LOOP;
   prc_close_file;
END;
SQL> @c:\myworkspace\sql\scripts\sample.sql;
PLS-00201: identifier 'TEXT_IO.FILE_TYPE' must be declaredIt works on the oracle forms but not on the SQL*Plus. Is there an alternative method using a PL/SQL script? A simple sample would help. Thanks.

Did you ever noticed the search box at the right side of the forum?
A quick search (limited to this years entries) brought up this thread for example
Re: UTL_FILE Examples

Configuration "use next available filename" in "write measurement file" function

My project recording signal and saving using "Write to Measurement File" function. I want to perform record multiple segments signals should have configured option "use next available filename". For example, if the original file is data.tdms then it will save the file into: data_1.tdms, data_2.tdms, data_3.tdms, ...
To do that I use a button control enable and disable write functionality.
In the past I have done normally. Now, I can not make it anymore, now every write new data were add into one file created without the next new file as desired, it has almost become a option "Append to file".
I use LabVIEW 2013
Solved!
Go to Solution.
Attachments:
write measurement file.vi ‏54 KB

Hello ducta9,
It looks like you currently have the Write to Measurement File Express VI configured to "save to one file" - this means that all data from a given session will be saved to the same file. The file number will only be incremented once the application is restarted or the Reset Boolean input is set to True. If you want to save to a series of files, select the "save to a series of files" option and configure the Express VI as needed to generate new files at intervals.
If you want to write variable-size chunks of data to a new file every time the user clicks the Enable button, you might be able to get away with just wiring a True constant to the Reset. I would still recommend taking a look at the options available in the save to a series of files settings.
Best Regards,
Tom L.

Using HS.Exp function to write data to a Consolidated Account

Hi,
Just wanted to know whether i can use the HS.Exp function to write data to a consolidated account?
Thanks

Hi Mau,
In order to write data to a consolidated account, you can use HS.Exp function in rules which puts data into a combination of Account, ICP, and Custom1...4 members. The syntax is: HS.Exp "DestPOV = Expression"
You might find useful the HS.Con function which puts data into the [Proportion] and [Elimination] Value dimension members and can also be used in Consolidation rules. The syntax is: Call HS.Con ("V#[Elimination]",-1*dPCon,"")
Regards,
Thanasis

Db- put blocks sometimes, mulit-thread uses Db- put to write DB file

My environment : redhat Linux 5, gcc version 4.1.2
BDB version: 4.8.24
Hi Guys
I used multi-thread application to write DB file. The application used Transactional Data Store and I used flags DB_TXN_NOSYNC . The DB file type is BTREE. I give 180M for the buffer cache and 8M for the log cache. And I used Db->put to write DB file ; five threads is used.
But the Db->put will block for one or two seconds sometimes.
Now I have found that the problem is in the lock/lock.c :: __lock_get_internal , line 953 : MUTEX_LOCK(env, newl->mtx_lock);. My application blocks here .+_
Could anyone give me some advice to avoid these blocks as possible as we can?
By the way, I have tried many of the lock set APIs which BDB offered. No effect -_- .
Thanks

Hi,
You should have posted this new piece of information in the original thread you opened:
Db->put block sometimes , when mulit-thread uses Db->put to write DB file
In the future please do not open a new thread for a problem you already reported.
Now I have found that the problem is in the lock/lock.c :: __lock_get_internal , line 953 : MUTEX_LOCK(env, newl->mtx_lock);. My application blocks here .+_This only confirms that one of your threads is blocked waiting for a requested lock, which is already held by another thread. Until the thread that holds the lock will not have released it, your thread waiting on it will not make progress.
You can collect the locking statistics at the moment you observe the slowness using either the db_stat utility with the -CA or -Co options (if the environment is not opened with DB_PRIVATE, that is, backing its region files into per-process private/heap memory) or programmatically using DB_ENV->lock_stat_print():
[http://www.oracle.com/technology/documentation/berkeley-db/db/api_reference/C/db_stat.html]
[http://www.oracle.com/technology/documentation/berkeley-db/db/api_reference/C/lockstat_print.html]
Examining the locking statistics will help determine what is the problematic lock, what is the locked object and which thread is holding it.
In addition to standard deadlock detection you can look into configuring lock timeouts. If your lock request is not satisfied within a given time it will time out and return DB_LOCK_DEADLOCK or DB_LOCK_NOTGRANTED. Lock timeouts may be configured per environment wide basis or per-lock or per-transaction basis:
[http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/lock_timeout.html]
[http://www.oracle.com/technology/documentation/berkeley-db/db/gsg_txn/C/lockingsubsystem.html#configdeadlkdetect]
As Mike already suggested it would help if you could put together a test case program to demonstrate this issue.
Regards,
Andrei

Need info on using external tables load/write data

Hi All,
We are planning to load conversion/interface data using external tables feature available in the Oracle database.
Also for outbound interfaces, we are planning to use the same feature to write data in the file system.
If you have done similar exercise in any of your projects, please share sample code units. Also, let me know if there
are any cons/limitations in this approach.
Thanks,
Balaji

Please see old threads for similar discussion -- http://forums.oracle.com/forums/search.jspa?threadID=&q=external+AND+tables&objID=c3&dateRange=all&userID=&numResults=15&rankBy=10001
Thanks,
Hussein

How to use a value int the DAT file as name of the PDF file

I need to use a value present in my DAT file as name of the PDF File.
I am using the ,u argument in the Print Agent and I know this causes the pdf document to get a generic filename.
Please help me.
Thanks

See my response to this same question you posed in the Output Designer forum.
http://www.adobeforums.com/webx/.59b585c2/0

Create sql loader data file dynamically

Hi,
I want a sample program/approach which is used to create a sql loader data file.
The program will read table name as i/p and will use
select stmt will column list derived from user_tab_columns from data dictionary
assuming multiple clob columns in the column list.
Thanks
Manoj

I 'm writing clob and other columns to a sql loader dat file.
Below sample code for writing clob column is giving file write error.
How can I write multiple clobs to dat file so that control file will handle it correctly
offset NUMBER := 1;
chunk VARCHAR2(32000);
chunk_size NUMBER := 32000;
WHILE( offset < dbms_lob.getlength(l_rec_type.narrative) )
LOOP
chunk := dbms_lob.substr(l_rec_type.narrative, chunk_size, offset );
utl_file.put( l_file_handle, chunk );
utl_file.fflush(l_file_handle);
offset := offset + chunk_size;
END LOOP;
utl_file.new_line(l_file_handle);

Read/Write Jar files?

This is really a newbie question, but since there's a forum specifically for jar questions I figured it was better to start here and move it to the newbie section if y'all deem it appropriate.
I have written a desktop application using Java 1.4.1 class libraries and intend the application to run on multiple platforms including Mac OS X, Windows XP and Linux. The application works standalone (not yet jarred) on the machine it was developed on, but now it's time to begin figuring out the distribution method.
Right now, the application reads several files from a flat text-file database, allows the user to peruse and display the information in a variety of ways, and gives the user a method to add to the flat-file database as needed. The file is about a megabyte and is excerpted into memory at initialization, then not referred to again unless the user writes additional data. When the user generates new data, write traffic to the file is fairly light, maybe 2-3kbytes per session.
I searched the forums for the best way to handle read/write data files for distribution and so far I haven't found anything that seems relevant, but surely the question must have been asked and answered before - maybe I'm using the wrong keywords?
Anyway, I have three basic questions:
1) Can I both read and write a file that's enclosed in a jar file? Or are files read-only once "jarred"?
2) Assuming I can both read and write a file within my jar file, is reading and rewriting within the jar so inefficient as to make that a non-preferred approach?
3) How do other folks who have a local read/write datafile in a desktop application deal with distribution? Keep the read/write datafile within the jar? Make a copy of it outside the first time the application is run and always read/write the copy outside the jar? Or some other strategy?
Thanks for any suggestions you can give a newbie at the Java game.
Jon

Thanks, that was pretty much what I suspected.
I have several data files and configuration files for this project, so I was trying to make the distribution as clean as possible. At least some of the config information can be hidden in Preferences, but I was struggling with the data files. I'll include the data files in my jar, then unpack them to the user's directory when launched the first time and work with them thereafter in the user's directory.
Enjoy the Dukes!
Jon

TDMS Advanced Asynchronous Write (Data Ref)

Hello,
Im using the TDMS Advanced Asynchronous Write (Data Ref).vi to write Data directly from some DMA FIFO´s to TDMS file. I save the Files on a NI 8260 1TB HDD Array.
Now the question. I´m interested on how this Advanced Asynchronous Write (Data Ref) works. What is done with the external Data Ref from the DMA FIFO and how does LabVIEW work with the data?
I searched the NI Forums and NI page but found no information on how this works.
Maybe someone can help me,
Daniel@amc

Hi,
Indeed, there is little information besides the help file online:
"Writes data to the specified tdms file asynchronously. This function can initiate multiple asynchronous writes that take place in the background."
Is there something in specific you want to know? Or is the function not behaving as expected (performance issues etc.)
Regards,
René

Using auto-indexing tunnels to write data files?

Similar Messages

Maybe you are looking for