Best way to transfer large datasets between actors (Actor framework)

Hi everyone
I am in the planning / design phase of a larger application for which I wish to use the Actor Framework. It seems like a good "design pattern", easy for multiple developers to work on, easy to expand and easy to maintain.
In this application I need to transfer data betweeen actors/modules at a considerable rate. The data is the continous measurements from a DAQ device (or multiple devices) sampling at 200kHz. That's 200 kHz * 8 byte = 1.5 Mb/s per device.
There is no way this is done using the messages of the actor framework - those messages are designed to signal start, stop, select tasks etc. within the application. So how to transfer that amount of data between actors?
I've thought about using TCP/IP on localhost - this could also easily be expanded to allow different parts of the program to run on different machines on a network. I've even tested it and it works quite well. But I would like to hear some opinions or alternatives before I decide to go with this solution. Any thoughts or alternatives?
Thanks
Jonas

JonasCJ wrote:
Could this be done without TCP/IP and just a LABVIEW queue? Passing the queue in a message from one actor to another?
If it's in the same process then yes, and it would be more efficient, because with TCP you have to create a copy when converting the data to a string, another copy when receiving and a third one when converting back. Unless you really are planning on distribtuing it, I would suggest avoiding that. Sidebar - AQ has been talking about networked actors and I believe there should even be an experimental branch in the AF group in the community area. You might want to check that out.
Incidentally, if you are planning on splitting it and don't actually need the real time processing, you might consider saving the data to a network drive and then simply having the other side loading the data from the file. It should be simpler.
Try to take over the world!

Similar Messages

What is the Best way to move large mailboxes between datacenters?

What is the Best way to move large mailboxes between datacenters?

Hi,
Are you asking with regards to on-premises Exchange? With Microsoft Online SaaS services (aka Exchange Online) there is no control and no need to control which data center a mailbox resides in.
With regard to on-premises Exchange, you have two choices: you can move it over the WAN in which case you would either do a native mailbox move (assuming you have Exchange 2010 or later you can suspend the move after the copy so you can control the
time of the cutover) or create a database copy in the second data center and once the database copies have synchronized change the active copy.
The other choice is to move is out of band which would usually involve an offline seed of the database (you could conceivably move via PST file but that would disrupt access to the mailbox and is not really the 'best way').
In general, Exchange on-premises questions are best asked on the Exchange forum: http://social.technet.microsoft.com/Forums/office/en-US/home?category=exchangeserver
Thanks,
Guy

Best way to transfer large files over the Internet?

What is the best way to transfer files that are too large to send by e-mail over the internet, preferably ways that are free or very cheap and are easiest for sender and receiver to work with?
I assume one way is by ftp, but would that necessarily mean buying space on a server somewhere that both I and my recipients can get to? or is there a way to avoid this cost?

you can use a cloud storage like Mobileme (not free) or DropBox (free up to 2GB).
you can also use direct file sharing over internet. no extra software or 3rd party services required. that's what I use to move files between my home and work computers.

What's the best way to transfer large files to a fellow Mac user?

… ftp, file sharing, dropbox…?

Is this a one-time transfer?
If so, what is your email file size limit?
Depending on the limit, try Compressing the file. If that doesn't work, try splitting the file into smaller chunks with Tar.
In Terminal (assumes file is on Desktop, if not, just change cd)
Note: don't use full path in filename unless you want it stored in the Tar
This will break the file into 10MG chunks (-L10000):
cd Desktop
/usr/bin/gnutar -L10000 -cMv --file=tar_archive.{tar,tar-{2..25}} [FileToTar]
to Extract on the receivers end:
/usr/bin/gnutar -xMv --file=tar_archive.{tar,tar-{2..25}}

Best way to copy large folders to external drive?

What is the best way to transfer large folders across to external drives?
I'm trying to clean up my internal hard drive and want to copy a folder (including its sub folders) of approx 10GB of video files to an external firewire drive.
I've tried drag and drop in finder (both copy and move) and it gets stuck on a particular file and then aborts the whole process.
1. Surely it should copy everything it can, and simply not copy the problem file.
2. Is there a way I can do copy and verify, a bit like when I burn a disk so I can be sure the video files have transferred safely before I delete them from my internal drive?
Many thanks in advance for any advice.

What you are trying to do makes perfect sense to me and I have done the same prior to getting myself a Time Machine system in place.
1. Surely it should copy everything it can, and simply not copy the problem file.
The fact that it is getting stuck on a particular file suggests that there is a problem with it. Try to identify which one it is and deal with that file on it's own. It could be that there is a disk error where that file is stored.
2. Is there a way I can do copy and verify....
The copy process you are using does that implicitly as I understand it.
Chris

Best way to transfer video between iphone and mac wirelessly?

Please suggest me the best way to transfer files between mac and iphone wirelessly.Mainly for videos.

If you want to transfer photos and videos from the Mac to the iPhone (but not in the other direction) you can use iTunes and sync via wi-fi: http://support.apple.com/kb/HT1386
If you want to wirelessly tranfer videos in the other direction you may want to look into a 3rd party app such as http://www.photosync-app.com Note that I have not used that app so be sure to investigate others as well.

Best way to transfer data between loops?

Hi, whats the best way to transfer data between loops(state machines, regular while loop etc) in labview. For so many years ive been using local variables but these can cause race conditions. Anyone advise?
Stu

It depends one the structure of the loops and how they need to exchange data (unidirectional, bidirectional, broadcasting).
The simple case of one loop passing data to the other loop: producer/consumer design pattern using a queue
If the structure already contains an event structure: User Events
The way I code is documented in my community nugget on events
I often mix this with producer/consumer.
Felix
www.aescusoft.de
My latest community nugget on producer/consumer design
My current blog: A journey through uml

Best way to transfer data between iMac early 2006 and iMac late 2012

What is the best way to transfer data between iMac early 2006 and iMac late 2012?

HI, is this for Migrating all data ove, or just File Sharing?
What OSX version is in he older one?
Do you have any backup external drive for the older one?

I have a large kodak easyshare library, what is the best way to transfer them into iPhoto ?

I have about 5k pictures in the kodak easyshare software system on a dell laptop - what is the best way to transfer or import them into iPhoto on my iMac ? any help will be greatly appreciated...

Simply, export the data from this esy share system to standard formats like jpeg, making sure to preserve all metadata on the way. How to do that? YOu'll need to post on a Kodak forum I'm afraid.
Then copy the files to the Mac and import them.
Regards
TD

Best way to transfer data between loops in FPGA VIs

Hi,
I'm using a FlexRio Board PXIe 7962, with an I/O module 5781.
I've set the Top level Clock to 80MHz (Base Clk 40M), and I'm implementing two loops in the FPGa VI: In the first one, I'm using a SCTL (20M) where I'm getting Digitized Data (1.5MHz) from the NI 5781, then multiplying it by a DDS Signal (1.8MHz), then I'm filtering (band-pass around 300kHz)) the product.
My question is: If what I've implemented is correct, what is the best way to transfer the first loop data running at 20MHz, to the second one running at 1MHz ? (without loosing Data of course since the signal interest is at 300kHz)
and similarly : what is the best way to transfer the second loop data running at 1MHz, to a Host VI ? DMA FIFOs?
Thank you in advance,
Zouhair
Attachments:
Diagram.png ‏158 KB

Zouhair wrote:
I've set the loop timer in the first frame to 4 clock ticks of 80MHz, so I imagine that what follows should not work under 20MHz, Am I mistaken?
Yes you are. That SCTL will execute in the 20MHz clock time. Everything else in the loop is just adding to that time.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines

What is the best way to transfer my music library from my windows pc to my new macbook air?

what is the best way to transfer my large iTunes library from my windows pc to my new macbook air? Not all purchased, many from CDs I owned

Moving a library: Quick answer if you use iTunes' default preferences settings: Copy the entire iTunes folder (and in doing so all its sub-folders and files) intact to the other drive. Hold down the option (alt) key (shift on Windows) and open iTunes. At the prompt to create or choose a library choose the copied iTunes folder.
If this is to a new computer and you put the copied iTunes folder in the default location of Macintosh HD > Users > *User Name* > Music then you don't even need to start with the option key held down, iTunes will automatically look for it there. (Make sure there isn't anything already in the iTunes folder there that you want to keep since you will be replacing it with the one you are moving.)
Windows users see tip at: https://discussions.apple.com/message/18879381#18879381
Consider moving everything from your Windows PC to your Mac in a single move, including your iTunes library: About Windows Migration Assistant - http://support.apple.com/kb/ht4796 - Learn about how to use Windows Migration Assistant to migrate your files from a Windows PC to a Mac.

Best way to transfer data to new internal hard drive

Hi,
I just got a new internal hard drive for my macbook and I'm trying to figure out the best way to transfer all the data from my old hd to the new one. Basically, I would just like to make an exact copy of my old hd and put it on the new larger one. Thanks for the help.

Will I simply be able to do a firewire transfer on the new internal HD is installed
Yes. Use Carbon Copy Cloner with the sopurce and destination volumes reversed after installing the new drive.
Additionally do I need a bootable copy of OSX on my external HD to do the transfer?
Yes; the clone can act as this copy. If you copy everything with Carbon Copy Cloner, you will be told whether the drive will be bootable or not prior to the clone.
(33676)

Best way to transfer a 10g database from HP9000 to Linux Redhat?

What is the best way to transfer a 10g databasse from HP9000 to Linux Redhat?

Hi Bill,
What is the best way to transfer a 10g databasse from HP9000 to Linux Redhat?Define "best"? There are many choices, each with their own benefits . . .
Fastest?
If you are on an SMP server, parallel CTAS over a databaee link can move large amnunts of tables, fast:
http://www.dba-oracle.com/t_create_table_select_ctas.htm
I've done 100 gig per hours . . .
Easiest?
If you are a beginner, data pump is good, and I have siome tips on doing it quickly:
http://www.dba-oracle.com/oracle_tips_load_speed.htm
Also,, make sure to check the Linux kernel settings. I query http://www.tpc.org and search for the server type . . .
The full disclosure reports show optimal kernel settings.
Finally, don't forget to set direct I/O in Linux:
http://www.dba-oracle.com/t_linux_disk_i_o.htm
Hope this helps . . .
Donald K. Burleson
Oracle Press author
Author of "Oracle Tuning: The Definitive Reference" http://www.rampant-books.com/book_2005_1_awr_proactive_tuning.htm

What is the best way to transfer my date from my old iMac to my new iMac?

what is the best way to transfer my date from my old iMac to my new iMac? I don't have a time capsule, so I cannot use time machine.
thanks

Your best bet by far is to use Setup Assistant when the new Mac first starts up to transfer directly from the old one.
It's similar to Migration Assistant, but doing it that way will avoid having an extra user account.
Either way, you can connect them via FireWire, Thunderbolt, or your network, but avoid wireless if at all possible. You can connect both to your netwrork via Ethernet, or put a single cable between the two.
See Using Setup Assistant on Mountain Lion or Lion for details.

Best way to transfer files without using migration assistant

upgrading from snow leopard to mavericks goes ok, but migrating files hangs with "less than a minute to go". What is the best way to transfer user files without using migration assistant?

I keep backups of essential documents in Dropbox, and then the files can sync when I reinstall Dropbox.
For larger collections of files, for example, my Documents folder, or iTunes or iPhoto libraries, I transfer them from a Time Machine Backup- just by going into the backup manually and restoring the folders.
If you have files that can fit on a flash drive, then you can use that, which is very fast.

Best way to transfer large datasets between actors (Actor framework)

Similar Messages

Maybe you are looking for