How to save large amount of data and memory management

hi guys,
I prepared a vi to take measurement of temperature and pressure simultaneously from Agilent 34970A with 34901A. The sampling rate is 1 sample/per second for each channel (20 temperature channel and 2 pressure channel (current). The device has 20 channels and most probably the test will lasts 2 days. So, it means 86400 sample per day totally. Could the attached vi do this task? I really do not know how the "write to text file.vi" works and use the memory of the pc. And after collecting all data, could I export these data to microsoft office excel to analyse safely? And I really do not know can I work this vi without interrupting. is it possible? I am also open for all other suggests relevant the vi, too. And I wonder that is it possible to build a standalone application to take measurements from DAQ devices (here Agilent 34970A-34901A) by a pc without labview?
Egemen
Attachments:
Agilent_Labview_v2.4.vi ‏85 KB

Hey,
What I meant by check for an error while measuring is the following:
you start the program
If one of your Agilent VIs returns an error (maybe bad initialisation or so) you will still try to continue to measure.
Even if it is not working, you still continue measuring. Instead you could also check at the end of each loop if an error occured, like this:
this will only look for an error and stop right away. If you encounter typical errors with apperatus you might want to handle it. A sufficient way to do so might be a state machine that just re-initializes the setup if an error occurs and then continues with its task. Check out this article: http://www.ni.com/white-paper/3024/en
Anyway, I would recommend to just try it out and see what problems arise, as you might spend hours of improving well working code (:

Similar Messages

Is the only way to import large amount of data and database objects into a primary database is to shutdown the standby, turn off archive log mode, do the import, then rebuild the standby?

I have a primary database that need to import large amount of data and database objects. 1.) Do I shutdown the standby? 2.) Turn off archive log mode? 3.) Perform the import? 4.) Rebuild the standby? or is there a better way or best practice?

Instead of rebuilding the (whole) standby, you take an incremental (from SCN) backup from the Primary and restore it on the Standby. That way, if, for example
a. Only two out of 12 tablespaces are affected by the import, the incremental backup would effectively be only the blocks changed in those two tablespaces (and some other changes in system and undo) {provided that there are no other changes in the other ten tablespaces}
b. if the size of the import is only 15% of the database, the incremental backup to restore to the standby is small
Hemant K Chitale

Advice needed on how to keep large amounts of data

Hi guys,
Im not sure whats the best way is to make large amounts of data available to my android app on the local device.
For example records of food ingredients, in the 100's?
I have read and successfully created .db's using this tutorial.
http://help.adobe.com/en_US/AIR/1.5/devappsflex/WS5b3ccc516d4fbf351e63e3d118666ade46-7d49. html
However to populate the database I use flash? So this kind of defeats the purpose of it. No point in me shifting a massive array of data from flash to a sql database, when I could access the data direct from the as3 array?
So maybe I could create the .db with an external program? but then how would I include that .db in the apk file and then deploy it to users android device.
Or maybe I create a as3 class with an xml object init and use that as a means of data storage?
Any advice would be appreciated

You can use any means you like to populate your SQLite database, including using external programs, (temporarily) embedding a text file with SQL statements, executing some SQL from AS3 code etc etc.
Once you have populated your db, deploy it with your project:
http://chrisgriffith.wordpress.com/2011/01/11/understanding-bundled-sqlite-databases-in-ai r-for-mobile/
Cheers, - Jon -

What java collection for large amount of data and user customizable record

I'm trying to write an application which operates on large amount of data. I want user could customize data structure (record) from different types of variables(float,int,bool,string,enums). These records should be stored in some kind of Array. Size of record: 1-200 variables; size of Array of those records: about 100000 items (one record every second through whole day). I want these data stored in some embedded database (sqlite, hsqldb) - access using simple JDBC. Could you give me some advise how to design thoses data strucures. Sincerely yours :)
Ok, maybe I give some example. This will be some C++ code.
I made an interface:
class ParamI {
virtual string toString() = 0;
virtual void addValue( ParamI * ) = 0;
virtual void setValue( ParamI * ) = 0;
virtual BYTE getType() = 0;
Than I made some template class derived from interface ParamI:
template <class T>
class CParam : CParamI {
public:
     void setValue( T val );
     T getValue();
     string toString();
     void setValue( ParamI *src ) {
          if ( itemType == src->getType() ) {
               CParam<T> ptr = (CParam<T>)src;
               value = ptr->value;
private:
     BYTE itemType;
     T value;
sample constructor of <int> template:
template<> CParam<int>::CParam() {
     itemType = ParamType::INTEGER;
This solution makes me possible to write collection of CParamI:
std::vector<CParamI*> myCollection;
CParam<int> *pi = new CParam<int>();
pi->setValue(10);
myCollection.push_back((CParamI*)pi);
Is this correct solution?. My main problem is to get data from the collection. I have to check its data type using getType() method of CParamI interface.
Please could give me some advise, some idea to make it right using java.

If you have the requirement that you have to be able to configure on the fly, then what I've done in the past is just put everything into data pairs into a list: something along the line of: (<Vector>, <String>), where the Vector would store your data and String would contain a data type. I would then make a checker to validate the input according to the SQL databypes that I want to support on the project. It's not a big deal with the amount of data you are talking about.
The problem you're going to have is when you try to allow dynamic definition, on the fly, of data being input to a table that has already been defined. Your DB will not support that, unless you just store that data pair--which I do not suggest.

How to pass large amount of data between steps

Hi all，
I have some LabVIEW VIs for data acquisition。
I need to pass large amount of data(array size >5000000 each time) from one step to another.
But it is not allowed to set array size larger than 5000000.
Any suggestion?
czhen
Win 7 SP1 & LabVIEW 2012 SP1, Teststand 2012 SP1
Solved!
Go to Solution.
Attachments:
Array Size Limits.png ‏34 KB

In your LabVIEW code, put the data into a data value reference. Pass this reference between your TestStand steps. As an added bonus, you will not get an extra copy of the data at each step. You will need to use the InPlace element structure to get your data out of the data value reference.
This account is no longer active. Contact ShadesOfGray for current posts and information.

Real time application and large amount of data

Hi,
I've a real-time application that needs to allocate large amount of data in memory (more than 5GB). I'm using 1.6 version of JRE and I'm planning to migrate to 1.8 because the application experiment a lot of "Stop of the world" every day. I spent a lot of time analysing and testing all the parameters and policies of the GC.
The question is, with the 1.8 version dissapears the pauses on the application caused by the GC cleaning process?
Thanks.

Just noting that the GC only really needs to do something when objects are created and then are no longer in use. So if your application needs a large amount of memory then keeping it, rather than discarding it might be a better solution.

MY phone is using large amounts of data, when i then go to system services, it s my mapping services thats causing it. what are mapping services and how do i swithch them off. i really need help.

MY phone is using large amounts of data, when i then go to system services, it s my mapping services thats causing it. what are mapping services and how do i swithch them off. i really need help.

I Have the same problem, I switched off location services, maps in data, whatever else maps could be involved in nd then just last nite it chewed 100mb... I'm also on vodacom so I'm seeing a pattern here somehow. Siri was switched on however so I switched it off now nd will see what happens. but I'm gonna go into both apple and vodacom this afternoon because this must be sorted out its a serious issue we have on our hands and some uproar needs to be made against those responsible!

With journaling, I have found that my computer is saving a large amount of data, logs of all the changes I make to files; how can I clean up these logs?

With journaling, I have found that my computer is saving a large amount of data, logs of all the changes I make to files; how can I clean up these logs?
For example, in Notes, I have written three notes; however if I click on 'All On My Mac' on the side bar, I see about 10 different versions of each note I make, it saves a version every time I add or delete a sentence.
I also noticed, that when I write an email, Mail saves about 10 or more draft versions before the final is sent.
I understand that all this journaling provides a level of security, and prevents data lost; but I was wondering, is there a function to clean up journal logs once in a while?
Thanks
Roz

Are you using Microsoft word? Microsoft thinks the users are idiots. They put up a lot of pointless messages that annoy & worry users. I have seen this message from Microsoft word. It's annoying.
As BDaqua points out...
When you copy information via edit > copy, command + c, edit > cut, or command +x, you place the information on the clipboard. When you paste information, edit > paste or command + v, you copy information from the clipboard to your data file.
If you edit > cut or command + x and you do not paste the information and you quite Word, you could be loosing information. Microsoft is very worried about this. When you quite Word, Microsoft checks if there is information on the clipboard & if so, Microsoft puts out this message.
You should be saving your work more than once a day. I'd save every 5 minutes. command + s does a save.
Robert

How do I pause an iCloud restore for app with large amounts of data?

I am using an iPhone app which is holding 10 Gb of data (media files) .
Unfortunately, although all data was backed up, my iPhone 4 was faulty and needed to be replaced with a new handset. On restore, the 10Gb of data takes a very long time to restore over wi-fi. If interrupted (I reached the halfway point during the night) to go to work or take the dog for a walk, I end up of course on 3G for a short period of time.
Next time I am in a wi-fi zone the app is restoring again right from the beginning
How does anyone restore an app with large amounts of data or pause a restore?

You can use classifications but there is no auto feature to archive like that on web apps.
In terms of the blog, Like I have said to everyone that has posted about blog preview images:
http://www.prettypollution.com.au/business-catalyst-blog
Just one example of an image at the start of the blog post rendering out, not hard at all.

Pull large amounts of data using odata, client API and so takes a long time in project server 2013

We are trying to pull large amounts of data in project server 2013 using both client API and odata calls, but it seem to take a long time. How is this done
In project server 2010 we did this creating SQL views in both the reporting database and for list creating a view in the content database. Our IT dept is saying we can't do this anymore. How does a view in Project database or content database create issues?
As long as we don't add a field in the table. So how's one to do this with creating a view?

Hello,
If you are using Project Server 2013 on premise I would recommend using T-SQL against the dbo. schema in the Project Web Database for your reports, this will be far quicker that the APIs. You can create custom objects in the dbo. schema, see the link below:
https://msdn.microsoft.com/en-us/library/office/ee767687.aspx#pj15_Architecture_DAL
It is not supported to query the SharePoint content database directly with T-SQL or add any custom objects to the content database.
Paul
Paul Mather | Twitter |
http://pwmather.wordpress.com | CPS |
MVP | Downloads

How can I copy large amount of data between two HD ?

Hello !
Which command could I user to copy large amount of data between two hard disk drives ?
How Lion identify the disk drives when you want to write some script, for example in Windows I sue
Robocopy D:\folder source\Files E:\folder destination
I just want to copy files and if the files/folders exist in destination the files should be overwritted.
Help please, I bougth my first MAC 4 days ago.
Thanks !

Select the files/folders on one HD and drag & drop onto the other HD. The copied ones will overwrite anything with the same names.
Since you're a newcomer to the Mac, see these:
Switching from Windows to Mac OS X,
Basic Tutorials on using a Mac,
Mac 101: Mac Essentials,
Mac OS X keyboard shortcuts,
Anatomy of a Mac,
MacTips,
Switching to Mac Superguide, and
Switching to the Mac: The Missing Manual,
Snow Leopard Edition.&
Additionally, *Texas Mac Man* recommends:
Quick Assist,
Welcome to the Switch To A Mac Guides,
Take Control E-books, and
A guide for switching to a Mac.

Is there any way to connect time capsule to a MacBook Pro directly via USB. I have a large amount of data that I want to back up and it is taking a very long time (35GB is taking 3 hrs, I have 2TB if files in total)...)?

Perhaps via USB. I have a large amount of data that I want to back up and it is taking a very long time (35GB is taking 3 hrs, I have 2TB if files in total)...? I want to use TimeCapsule as back-up for an archive which is curently stored on a 2 TB WESC HD.

No, you cannot backup via direct usb connection..
But gigabit ethernet is much faster anyway.. are you connected directly by ethernet?
Is the drive you are backing up from plugged into the TC? That will slow it down something chronic.. plug that drive in by its fastest connection method.. WESC sorry I have no idea. If ethernet use that.. otherwise USB direct to the computer.. always think what way the files come and go.. but since you are copying from the computer everything has to go that way.. it makes things slower if they go over the same cable.. if you catch the drift.

How can I edit large amount of data using Acrobat X Pro

Hello all,
I need to edit a catalog that contains large amount of data - mainly the product price. Currently I can only export the document into excel file and then paste the new price onto the catalog using Acrobat X Pro one by one, which is extremely time-consuming. I am sure there's a better way to make this faster while keeping the accuracy of the data. Thanks a lot in advance if any one's able to help!

Hi Chauhan,
Yes I am able to edit text/image via tool box, but the thing is the catalog contains more than 20,000 price data and all I can do is deleteing the orginal price info from catalog and replace it with the revised data from excel. Repeating this process over 20,000 times would be a waste of time and manpower... Not sure if I make my situation clear enough? Pls just ask away, I really hope to sort it out, Thanks!

Scrolling of large amount of data with Unscrolled HEADER .How to do???

I am in much need of scrolling a large amount of data without scrolling the header,i.e. while scrolling,only data will be scrolled,so that I can see the header as well.I am using JSP in struts framework.Waiting for your help friends.Thanks in advance.

1) Install Google at your machine.
2) Open it in your favourite web browser.
3) Note the input field and the buttons.
4) Enter smart keywords like "scrollable", "table", "fixed" and "header" in that input field.
5) Press the first button. You'll get a list of links.
6) Follow the relevant links and read it thouroughly.

JSP and large amounts of data

Hello fellow Java fans
First, let me point out that I'm a big Java and Linux fan, but somehow I ended up working with .NET and Microsoft.
Right now my software development team is working on a web tool for a very important microchips manufacturer company. This tool handles big amounts of data; some of our online reports generates more that 100.000 rows which needs to be displayed on a web client such as Internet Explorer.
We make use of Infragistics, which is a set of controls for .NET. Infragistics allows me to load data fetched from a database on a control they call UltraWebGrid.
Our problem comes up when we load large amounts of data on the UltraWebGrid, sometimes we have to load 100.000+ rows; during this loading our IIS server memory gets killed and could take up to 5 minutes for the server to end processing and display the 100.000+ row report. We already proved the database server (SQL Server) is not the problem, our problem is the IIS web server.
Our team is now considering migrating this web tool to Java and JSP. Can you all help me with some links, information, or past experiences you all have had with loading and displaying large amounts of data like the ones we handle on JSP? Help will be greatly appreciated.

Who in the world actually looks at a 100,000 row report?
Anyway if I were you and I had to do it because some clueless management person decided it was a good idea... I would write a program in something that once a day, week, year or whatever your time period produced the report (in maybe a PDF fashion but you could do it in HTML if you really must have it that way) and have it as a static file that you link to from your app.
Then the user will have to just wait while it downloads but the webserver or web applications server will not be bogged down trying to produce that monstrosity.

How to save large amount of data and memory management

Similar Messages

Maybe you are looking for