Labview file load memory problem

Hi Guys,
I am loading a binay different binary files and am having error of "Memory full"
When I load a 43,837 and 110,196kb file this error comes .
and when I load 5,811 kb file it works fine.
Please note that 43,837kb file has 5611000 samples and 110,196kb file has 8060000
can anyone tell me the solution to fix it ?
Rgs
M Omar Tariq

Please post your code so we can give you real suggestions. It sounds like you are make extra memory copies you probably don't need. To get a jump start on the problem, take a look at the tutorial Managing Large Data Sets in LabVIEW. Newer versions of LabVIEW have this information in the help files.
This account is no longer active. Contact ShadesOfGray for current posts and information.

Similar Messages

E62 file manager & memory problem

Hello
I have an E62 from cingular, with a 2-gig extended memory.
I have a problem with invisible data -- mostly email messages -- that are using up all but 80 mb of the phone memory.
It appears I mucked things up by uninstalling blackberry connect and then reinstalling it to the 2gig extended memory card. There are still messages somewhere on the phone memory but I cannot access them. There are some other things too, but absolutely no data shows up in folders through file manager. It's as if all files are empty, this is true for both phone memory and the extended memory and has been since I got the phone -- it didn't raise any questions for me because I didn't quite understand the OS and I had not yet added much stuff to the phone then. But it seems to me that stuff ought to be visible. For example, I had created some audio files with the recorder. They were visible through the recorder software, but they did not turn up in file manager. So I'm wondering if that program even works.
I would like to avoid resetting the phone completely, but if that is the only way to get rid of this invisible data, or to get file manager to work properly, I guess I'll have to do that. Has anyone ever had this problem? Suggestions?

Try using the Y-Browser to access these files:
http://www.allaboutsymbian.com/software/item/Y-Browser_for_S60_3rd_Edition.php
You would need to set the certificate check in the app-manager to not to validate the certificate to install this.

CS5 Generic File Info Memory Problem

We have been using a very large custom generic panel in CS4 File Info. The MRU file has grown to 2.5 mb, the properties are 1.5 mb -- but they have been working great in CS4. In CS5 they frequently do not work . The panel will either be empty when you open it, although there are xmp values in file, or after entering data, the panel will spin for a while, show empty again, often after trying to process the MRU's.
Are panels in CS5 products OK to be used for this large of a metadata set of attributes and values?
Using latest Macs, 4GB of ram, happens in all CS5 apps.
Thanks for your help.
Todd

Todd,
I understand your requirement. Controlled vocabularies can be a
critical component of a metadata strategy.
I know people that use a semicolon delimited file to pre-load the CS3
File Info MRU of a bag or seq field via a single copy and paste. Its
limit appears to be fifteen entries. Like you they are simulating a
controlled vocabulary for a list.
Pound Hill's MetaGrove Plug-in provides controlled vocabularies for
lists (bag and seq). These are displayed as a table of one column,
popup at the top, open or closed choice.
The largest single (internal) MetaGrove vocabulary I have implemented
has 4004 entries in one language, 1 mb in size. PHS's format is more
verbose than Adobe's MRU or File Info panel formats. It takes 20
seconds to initialize.
MetaGrove vocabularies can optionally be external text files, with tab
delimited value/label pairs, local or on the network. These are single
language, by definition. This is intended for an enterprise where the
external vocabulary file is typically updated monthly or quarterly.
The plug-in will automatically cache the update within a week
(default). The vocabulary can be maintained using a spreadsheet or
output from an application like yours.
Regarding your test with repeated red, green blue, for 100,000 entries:
In File Info v0 and v1 (CS through CS3), "All names and keyphrases
(values) in the list must be unique." This constraint is not mentioned
in the v2/3 File Info document. Does repeated red, green blue work
with 9 entries or so?
Most CS3 applications have odd behaviors with more than 99 entries in
a File Info popup. This appears to be fixed in Acrobat 10. I haven't
attempted more than 200 entries in a CS3 panel popup.
I suspect that building a in-memory copy of a vocabulary from a panel
file, or from the MRU file, is a serial process and does not scale
linearly. Each incremental entry takes longer to add to the in-memory
representation because the entire in-memory copy must be scanned each
time. The design goals of tens to entries probably did not warrant
using database techniques for the internal representation. The
performance in CS4 is your benchmark.
CS4 and 5 File Info appears to be doing some central caching.
Sometimes, I need to remove the panel entirely, Exit all File Info
applications, run an application and its File Info, Exit and then add
the panel in again to get changes to take. You probably need to get
everyone-out-of-the-pool before you touch the MRU file.
The CS3 copy and paste technique mentioned above might be an
alternative to let FIle Info manage the MRU. However, in your case, it
will test the implementation limits of the menu field payload capacity.
Carl crambert at poundhill dot com

Accessing a file and memory problem

hi gurus;
here's the thing:
when i initializing a file like
File a = new File(xxx);
is the file save to memory yet? or it is just initializing only?
or when i accessing using randomaccessfile its just save to memory?
any good link that teach this kind of things..
any input will much appreciate
Thanks

Here you are
http://java.sun.com/docs/books/tutorial/java/javaOO/objectcreation.html
This is a good basic tutorial.

Webutil file load/upload problem

Hi,
When i upload/download file from DB or AS and filename contains only english letters there is no problem. But when i wanna upload file which filename contains baltic leters webutil demo showing thah all ok,no errors, but blob column is empty.
I try to debug code and found that after this code clientFileInfo always = '0|0'
function UploadInt in webutil.pll
WebUtil_Core.setProperty (WebUtil_Core.WUT_PACKAGE,'WUT_FILE_INFO', clientFileInfo);
-- get the size and chunk info of the client side file
clientFileInfo := WebUtil_Core.getProperty(WebUtil_Core.WUT_PACKAGE,'WUT_FILE_INFO');
How i can fix this problem if user wants upload file wich filename contains not only english leters?
Andriejus

This is being tracked on BUG:3454831

Coherence Bulk Load Memory Problem

Hi,
I am trying to load like 35 million object(java object) into stand alone cache servers. According to reports of coherence, average item size is 360 byte. the thing is, to do this implementation I am using invocationservice's. And it means that, my invocable threads are running on on separate nodes(I have arranged them to be run on storage enabled nodes).
But the thing is; I am getting some memory probems liks:
8 nodes with 5 GB memory for each. Means 40 GB but even with this config, it makes full gc s several times in a day.
So to avoid full gc a little bit, I did not use invocalble threads, instead, I used regular threads in my application and it makes the loading work but it does not seem nice I think.
Any ideas that which one is better?(or maybe a new method)
Thanks

Have you taken a look at your GC logs to see what is happening?
Also, my rough calculations put you at an under sized grid:
35M objects @ 360 bytes means you need ~12GB of storage, not taking into account indexes, etc.
Optimistically, you'd need over 50GB of heap space in your grid. I'd suggest over 60GB. Why? Because you need space for backups copies of your data, (assuming you're running distributed services), working heap for serialisation, entry processes, etc, and your grid should never be more than 80% full.
My guess? You need more nodes/capacity in your grid - hence GC.

Memory problem with loading a csv file and displaying 2 xy graphs

Hi there, i'm having some memory issues with this little program.
What i'm trying to do is reading a .csv file of 215 mb (6 million lines more or less), extracting the x-y values as 1d array and displaying them in 2 xy graphs (vi attacked).
I've noticed that this process eats from 1.6 to 2 gb of ram and the 2 x-y graphs, as soon as they are loaded (2 minutes more or less) are really realy slow to move with the scrollbar.
My question is: Is there a way for use less memory resources and make the graphs move smoother ?
Thanks in advance,
Ierman Gert
Attachments:
read from file test.vi ‏106 KB

Hi lerman,
how many datapoints do you need to handle? How many do you display on the graphs?
Some notes:
- Each graph has its own data buffer. So all data wired to the graph will be buffered again in memory. When wiring a (big) 1d array to the graph a copy will be made in memory. And you mentioned 2 graphs...
- load the array in parts: read a number of lines, parse them to arrays as before (maybe using "spreadsheet string to array"?), finally append the parts to build the big array (may lead to memory problems too).
- avoid datacopies when handling big arrays. You can show buffer creation using menu->tools->advanced->show buffer allocation
- use SGL instead of DBL when possible...
Message Edited by GerdW on 05-12-2009 10:02 PM
Best regards,
GerdW
CLAD, using 2009SP1 + LV2011SP1 + LV2014SP1 on WinXP+Win7+cRIO
Kudos are welcome

Memory problem if OLE-object references to WMF files

Hi there,
I have a report with an OLE object containing WMFs.
The graphic files are variable and their name is loaded from the database during runtime (path + filename).
Running the report leads to 185 pages, each one containing a different WMF.
If I preview the report in CR, everything looks fine.
If I print the report, the OLE object / graphic is left empty....
If I export the report to PDF (as an example) I get the error message 'memory full'. Reducing the data set to ~50, the PDF is created. But the pictures get resized (much bigger) and only parts are visible.
The machine I'm using doesn't have any memory problems.
The WM files are only 3 to 12 KB each.
If I convert the WMFs to JPG and use these within the report it works...
Problem with this: a loss of quality (it is necessary to stretch the pictures to certain size)
Thanks in advance for any ideas!
Susanne
I'm using CR 2008 SP 3 on Windows 2003 Server

Format the pictures outside of CR for best results.

Load an existing Berkeley DB file into memory

Dear Experts,
I have created some Berkeley DB (BDB) files onto disk.
I noticed that when I issue key-value retrievals, the page faults are substantial, and the CPU utilization is low.
One sample of the time command line output is as follow:
1.36user 1.45system 0:10.83elapsed 26%CPU (0avgtext+0avgdata 723504maxresident)k
108224inputs+528outputs (581major+76329minor)pagefaults 0swaps
I suspect that the bottleneck is the high frequency of file I/O.
This may be because of page faults of the BDB file, and the pages are loaded in/out of disk fairly frequently.
I wish to explore how to reduce this page fault, and hence expedite the retrieval time.
One way I have read is to load the entire BDB file into main memory.
There are some example programs on docs.oracle.com, under the heading "Writing In-Memory Berkeley DB Applications".
However, I could not get them to work.
I enclosed below my code:
--------------- start of code snippets ---------------
/* Initialize our handles */
DB *dbp = NULL;
DB_ENV *envp = NULL;
DB_MPOOLFILE *mpf = NULL;
const char *db_name = "db.id_url"; // A BDB file on disk, size 66,813,952
u_int32_t open_flags;
/* Create the environment */
db_env_create(&envp, 0);
open_flags =
DB_CREATE | /* Create the environment if it does not exist */
DB_INIT_LOCK | /* Initialize the locking subsystem */
DB_INIT_LOG | /* Initialize the logging subsystem */
DB_INIT_MPOOL | /* Initialize the memory pool (in-memory cache) */
DB_INIT_TXN |
DB_PRIVATE; /* Region files are not backed by the filesystem.
* Instead, they are backed by heap memory. */
* Specify the size of the in-memory cache.
envp->set_cachesize(envp, 0, 70 * 1024 * 1024, 1); // 70 Mbytes, more than the BDB file size of 66,813,952
* Now actually open the environment. Notice that the environment home
* directory is NULL. This is required for an in-memory only application.
envp->open(envp, NULL, open_flags, 0);
/* Open the MPOOL file in the environment. */
envp->memp_fcreate(envp, &mpf, 0);
int pagesize = 4096;
if ((ret = mpf->open(mpf, "db.id_url", 0, 0, pagesize)) != 0) {
envp->err(envp, ret, "DB_MPOOLFILE->open: ");
goto err;
int cnt, hits = 66813952/pagesize;
void *p=0;
for (cnt = 0; cnt < hits; ++cnt) {
db_pgno_t pageno = cnt;
mpf->get(mpf, &pageno, NULL, 0, &p);
fprintf(stderr,"\n\nretrieve %5d pages\n",cnt);
/* Initialize the DB handle */
db_create(&dbp, envp, 0);
* Set the database open flags. Autocommit is used because we are
* transactional.
open_flags = DB_CREATE | DB_AUTO_COMMIT;
dbp->open(dbp, // Pointer to the database
NULL, // Txn pointer
NULL, // File name -- NULL for inmemory
db_name, // Logical db name
DB_BTREE, // Database type (using btree)
open_flags, // Open flags
0); // File mode. defaults is 0
DBT key,data; int test_key=103456;
memset(&key, 0, sizeof(key));
memset(&data, 0, sizeof(data));
key.data = (int*)&test_key;
key.size = sizeof(test_key);
dbp->get(dbp, NULL, &key, &data, 0);
printf("%d --> %s ", *((int*)key.data),(char*)data.data );
/* Close our database handle, if it was opened. */
if (dbp != NULL) {
dbp->close(dbp, 0);
if (mpf != NULL) (void)mpf->close(mpf, 0);
/* Close our environment, if it was opened. */
if (envp != NULL) {
envp->close(envp, 0);
/* Final status message and return. */
printf("I'm all done.\n");
--------------- end of code snippets ---------------
After compilation, the code output is:
retrieve 16312 pages
103456 --> (null) I'm all done.
However, the test_key input did not get the correct value retrieval.
I have been reading and trying this for the past 3 days.
I will appreciate any help/tips.
Thank you for your kind attention.
WAN
Singapore

Hi Mike
Thank you for your 3 steps:
-- create the database
-- load the database
-- run you retrievals
Recall that my original intention is to load in an existing BDB file (70Mbytes) completely into memory.
So following your 3 steps above, this is what I did:
Step-1 (create the database)
I have followed the oracle article on http://docs.oracle.com/cd/E17076_02/html/articles/inmemory/C/index.html
In this step, I have created the environment, set the cachesize to be bigger than the BDB file.
However, I have some problem with the code that opens the DB handle.
The code on the oracle page is as follow:
* Open the database. Note that the file name is NULL.
* This forces the database to be stored in the cache only.
* Also note that the database has a name, even though its
* file name is NULL.
ret = dbp->open(dbp, /* Pointer to the database */
NULL, /* Txn pointer */
NULL, /* File name is not specified on purpose */
db_name, /* Logical db name. */
DB_BTREE, /* Database type (using btree) */
db_flags, /* Open flags */
0); /* File mode. Using defaults */
Note that the open(..) API does not include the BDB file name.
The documentation says that this is so that the API will know that it needs an in-memory database.
However, how do I tell the API the source of the existing BDB file from which I wish to load entirely into memory ?
Do I need to create another DB handle (non-in-memory, with a file name as argument) that reads from this BDB file, and then call DB->put(.) that inserts the records into the in-memory DB ?
Step-2 (load the database)
My question in this step-2 is the same as my last question in step-1, on how do I tell the API to load in my existing BDB file into memory?
That is, should I create another DB handle (non-in-memory) that reads from the existing BDB file, use a cursor to read in EVERY key-value pair, and then insert into the in-memory DB?
Am I correct to say that by using the cursor to read in EVERY key-value pair, I am effectively warming the file cache, so that the BDB retrieval performance can be maximized ?
Step-3 (run your retrievals)
Are the retrieval API, e.g. c_get(..), get(..), for the in-memory DB, the same as the file-based DB ?
Thank you and always appreciative for your tips.
WAN
Singapore

How to Enable Plugin before File Opened And Load PDF File From memory?

hello everyone,
there have two problems,all i want to do is to show a pdf file encrypted by AES algorithm.
1.one is how to enable plugin before file opened?
i developed another two plugin,and each enabled after one pdf file was opened,and now i want to replace the file open command and can do some word by meself,such as decrypt the pdf ?
2.another is how to load pdf file from memory?
if i can enable plugin before file opened,and than i can read the file and decrypt it,and now i want to do is do load the decrypt file (in memory) into the adobe?
i tried many method,but all in vain.who can help me or give me some advice?
thanks.
merry chrismas to everyone.

The proper way to handle other file formats (even where it is really a PDF that has been "wrapped") is to create an AVConversionHandler.
To load a PDF from a source other than the file system, you need to create a custom AVFileSys.

Nokia C5-03 Low memory problems ( Other Files )

Please help. I have a nokia C5-03 the handset is showing phone memory is full. I have checked and uninstalled all unecessay applications. But problem is still there. when i check the phone memory it indicates that i have other files of 42mb installed and using phone memory. however i am unable to check what other files are installed or classfied as other files. Can you please assist me so i can know what these other files consist of or where they are located.

Hi Thetao,
I typed a more extensive reaction before, but it got lost when I pressed "post". Therefore I just respond to the main points that you mentioned (and some I found out myself).
Strange: I can't find the 40 MB Maximum User Storage on the Nokia website anymore (nor the 75 MB). But it sounds very familiar to me. It looks to me as if they removed this from the phone specs, also of other Smartphones by the way.
Yesterday, I deleted some small apps that I don't use (anymore) such as InternetRadio and I also removed Nokia email. Although the apps were below 2 MB together, this freed up over 7 MB of Phone memory (24 MB free now)! I think there were still some old emails stored in C: which I couldn't delete any other way. This helped me a great deal already but I tried your suggestions as well.
1. No map data or CITIES folder on C: 2. Switched messages memory to phone (and phone to offline mode) and I did indeed find a forgotten email account with 30 email messages. Not much but I had 24,7 MB free after that. Of course, I put messages memory back to the memorycard. 3. Used the free edition of Y-Browser to manually delete the cache folder. Not much data in that, but 25,1 MB free after that. Nice tool, with which you can reach folders that normally stay hidden! Used YBrowser to search all C: for files over 300 kB. Only 2 files: boot_space.txt in C:\ (500 kB, contains only the X repeatedly as far as I see, but is probably essential for the operating system) and C:\resource\python25\pyton25.zip (1 MB). It looks like an installation package, but I'm not sure if I can delete it. By the way: YBrowser hasn't made a shortcut in one of the menus. Only way that I found to start it was to look for it using the Phone's search function. Is there a way to make this shortcut myself?
4. Yes I did. No Images folder on C: anymore, nor other big files (see point 3)
5. I use Bluetooth for file transfer sometimes, mainly for installation files (such as YBrowser.sis, but I did this one via USB-cable). However, no big files are left on C: so I don't think I have this problem.
6. I tried to delete Nokia Chat yesterday as well (with the other apps), but it won't be uninstalled the normal way as it says "Uninstall cancelled" (not sure about the exact translation since my phone 'speaks' Dutch) Do you know if there's another way to get rid of this 3 MB app that I don't use at all?
I think I may have found an explanation and a solution for the memory problem while navigating. You mentioned the "memory in use" in the map settings. Above that option there's a slide bar for the % of memory that the navigation can use. Standard is 70%. I always thought this was about storage memory on (in my case) the memorycard. Another topic mentioned that this the working memory (so the RAM) that the navigation may use. Setting it to 70% means there's only 30% left for other apps that run in the background. The other topic states that this is nog enough so the slider should be set to for instance 30% for navigation leaving 70% free for "the phone". From behind my computer, navigation seems much more stable. I'll try this setting in my car soon and let you know how it works.
Thanks a lot for thinking along with me so far! There's already 25,1 MB of space, which is great since it was only 7 MB last Sunday. And navigation looks more stable. I'd appreciate if you have some more answers to my latest questions, but if not I think my phone will work a lot better already!
Regards, Paul

How to load a BDB file into memory?

The entire BDB database needs to reside in memory for performance reasons, it needs to be in memory all the time, not paged in on demand. The physical memory and virtual process address space are large enough to hold this file. How can I load it into memory just before accessing the first entry? I've read the C++ API reference, and it seems that I can do the following:
1, Create a DB environment;
2, Call DB_ENV->set_cachesize() to set a memory pool large enough to hold the BDB file;
3, Call DB_MPOOLFILE->open() to open the BDB file in memory pool of that DB environment;
4, Create a DB handle in that DB environment and open the BDB file (again) via this DB handle.
My questions are:
1, Is there a more elegant way instead of using that DB environment? If the DB environment is a must, then:
2, Does step 3 above load the BDB file into memory pool or just reserve enough space for that file?
Thanks in advance,
Feng

Hello,
Does the documentation on "Memory-only or Flash configurations" at:
http://download.oracle.com/docs/cd/E17076_02/html/programmer_reference/program_ram.html
answer the question?
From there we have:
By default, databases are periodically flushed from the Berkeley DB memory cache to backing physical files in the filesystem. To keep databases from being written to backing physical files, pass the DB_MPOOL_NOFILE flag to the DB_MPOOLFILE->set_flags() method. This flag implies the application's databases must fit entirely in the Berkeley DB cache, of course. To avoid a database file growing to consume the entire cache, applications can limit the size of individual databases in the cache by calling the DB_MPOOLFILE->set_maxsize() method.
Thanks,
Sandra

I just got a new Mac Mini, and now when I try to import photos into Lightroom, iPhoto try to load them first, then Lightroom can't find the files. Big problem. What to do?

I just got a new Mac Mini, and now when I try to import photos into Lightroom, iPhoto ties to load them first, then Lightroom can't find the files. Big problem. What to do?

Simple:
File -> Export to get your photos out of the iPhoto Library. In the Export dialogue you can set the kind to Original and you'll get back exactly what you put in.
Apps like iPhoto2Disk or PhotoShare will help you export to a Folder tree matching your Events.
Once exported, you can then trash the iPhoto Library - just drag it from the Pictures Folder to the trash and empty it.
After that, if you're entirely neurotic about it, just put the iPhoto app in the trash and empty it.
Regards
TD

WAV file won't replay in LabView or load in WMP, but works in VLC

I'm creating an audio file for an interview with the code snippet attached (VI is very large and does a lot of unrelated processing). This audio is recorded in the "interview" subvi. At completion, this subvi is closed and the "process" subvi is opened. If this process is followed, labview will open and play the WAV file with no problem. Alternatively, "process" can be run as a stand-alone. If this is done, I get the error 4810 at Sound File Read Open (possible reason: cannot recognize sound format). If I try to open the file in Windows Media Player, the wav will open, but will have a length of 0. If I open the wav in VLC, it plays with no problem. I haven't tried saving as a new file, but this is a sortof untenable solution anyway. For what it's worth, this problem arose when the audio recording was changed to a queue-based process. I've attached the old (working) code for reference.
Thanks,
Brendan
Solved!
Go to Solution.
Attachments:
interview sound recording.JPG ‏65 KB
playback audio.JPG ‏21 KB
old audio storage.JPG ‏94 KB

I'm also adding a screen cap of the working (right) and non-working (left) headers created by the snippets attached above. I don't have a lot of experience with this, but it strikes me as curious that the non-working (left) header has no value for what I think should be the File Size (data) section (as referenced here http://www.topherlee.com/software/pcm-tut-wavformat.html). I note also that the header seems to be 12 bytes longer that most references suggest, but this is the case in both the working and non-working headers (I'm looking specifically at offset 36-47 in both).
Attachments:
WAV headers.JPG ‏61 KB

Flat-File Loading problem

Hi Friends,
I am struggling with flat-file loading problem. I am trying to load a .csv file into data target. I took all pre-cautions while loading data. I look into preview and simulate the data. Everything is ok, but when i schedule the data, i found 0 records in the monitor. The following is the STATUS message of the above problem:
   No data available
Diagnosis
The data request was a full update.
In this case, the corresponding table in the source system does not
contain any data.
System response
Info IDoc received with status 8.
Procedure
Check the data basis in the source system.
         Can anybody help me what is the problem and procdure to resolve it?
Regards,
Mahesh

Hi Eugene,
Thanks for the quick reply. The following screen-shot tells you the messages of detail tab;
OVER ALL STATUS MISSING WITH MESSAGES OR WARNINGS
REQUEST: MISSING MESSAGES
EXTRACTION
EVERYTHING IS OK
DATA REQUEST RECEIVED
NO DATA AVAILABLE DATA ELECTION ENDED.
PROCESSING
NO DATA
           The above message was shown in details tab. Pls guide me to locate the problem.
Regards,
Mahesh

Labview file load memory problem

Similar Messages

Maybe you are looking for