Large Memory Leak?

I've got a container mc that I'm duplicating and
loading videos into. I have 15 swf videos I'm loading
into each mc. Each swf loaded is about 2mb in size
which equates to 30mb of total loaded swfs.
Each swf contains about 90 keyframes, each with
one image in a sequence that plays an animation.
After all the videos are loaded there are no other
movieclips created or duplicated.
I have arrow controls that allow me to rewind these
videos and fast forward them.
Everything works fine. But I'm noticing now that
as I rewind / fastworward these swfs my memory
is being gobbled up. It seems that my memory
goes up almost per frame that I rewind / fastworward,
and once the whole animation has played the memory
plateus, until I start to play one of the other unplayed
animations.
The only thing I'm doing with the rewind and fastforward
buttons is just a gotoAndPlay(prev / next frame).
I don't understand why my memory is being eaten up,
Seems like if the files loaded are only 30mb then the
memory shouldn't be terribly more than that.
Any ideas or help in finding what would be causing the
memory leak?
Thanks.

To rewind / fastforward the clips I am just calling
gotoAndPlay(back / forward 1 frame).
Thats all I call. It seems like the jpgs that are on each
keyframe are compressed
inside the SWF but then when they are shown on the screen
they are loaded into
memory. As I go through each new frame I can see my memory
going up...once I
play through the entire movie my memory plateaus...then the
process repeats once
I start to play the next movie...it just goes up and up.
Any way to clear memory or something like that without
unloading the movies?
Thanks.

Similar Messages

Memory leak in a function registered using set_restore_function()

I experience a problem with memory leak caused by the following function:
void RestorePhonemesSet(PhonemesSetStructType &phonemesSet, const void *src) {
char p = (char ) src;
memcpy(&phonemesSet.len, p, sizeof (int));
p += sizeof (int);
memcpy(&phonemesSet.whichFile, p, sizeof (int));
p += sizeof (int);
memcpy(&phonemesSet.whichPosition, p, sizeof (int));
p += sizeof (int);
phonemesSet.phonemes = (int *)malloc(sizeof(int)*phonemesSet.len);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ here the problematic code
memcpy(phonemesSet.phonemes, p, sizeof (int) * phonemesSet.len);
This function is registered using a following call: DbstlElemTraits<PhonemesSetStructType>::instance()->set_restore_function(RestorePhonemesSet);
The culprit it the malloc memory allocation. If I leave out the malloc the program crashes. If I free the phonemeSet.phonemes memory segment at the end of the restore function I lost data, and if I use the malloc there is a large memory leak while reading every record from the database.
What should I do to prevent the memory leak?
Regards,
markur
Edited by: 904259 on 2011-12-24 05:42

the solution is using the memory allocated for p, no need to allocate new memory space for the variable: the problematic line should look like .phonemes = (int *)p;
the memcpy function in the following line is thus superfluous.
Edited by: 904259 on 2011-12-24 05:43

Applescript Image Events appears to have memory leak (or I don't know what I am doing)

Using Image Events to create images and thumbnails for a website. The following code results in very large memory leaks in Image Event process. I have documented results of test runs in the comments.
The leak (or poor coding on my part) results in a total system halt if I attempt to process more than about 400 images at a time. Basically, I run out ot physical memory. I am running a new model 13" MBP with 8GM RAM. I have to manually stop Image Events to reclaim the memory (or reboot of course).
Any help/suggestions would be appreciated.
(* test memory leak in Image Events *)
          tests with 58 photos selected in iPhoto
          5.7 MB left in Image Events after run with only open and close
          22.0 MB left in Image Events after run with open, save and close
          45.9 MB left in Image Events after run with open, scale, save and close
          A run with 382 photos selected used OVER 3.8 GB (gigabytes) and the
          mac ran out of physical memory so I had to stop the test.
tell application "Finder"
          set imageFolder to folder "test" of home as alias
end tell
tell application "iPhoto"
          set currPhotoList to the selection
          repeat with currPhoto in currPhotoList
                    log name of currPhoto as string
                    set theImagePath to image path of currPhoto
                    tell application "Image Events"
launch
                              set theImage to open theImagePath
scale theImage to size 128
save theImage in imageFolder as JPEG with icon
close theImage
                    end tell
          end repeat
end tell
--- end of code example

Does the following code do any difference? Most likely no I guess… but better try than nothing.
tell application "Finder"
    set imageFolder to folder "test" of home as text -- instead of “as alias” (see theTargetPath below)
end tell
set theImagePaths to {}
set theImagePathsRef to a reference to theImagePaths -- faster with big lists
tell application "iPhoto"
    set currPhotoList to the selection
    repeat with currPhoto in currPhotoList
        log name of currPhoto as string
        get POSIX file (image path of currPhoto) as alias -- faster ?
        copy result to the end of theImagePathsRef
    end repeat
end tell
tell application "Image Events"
    launch
    repeat with theImagePath in theImagePaths
        set theImage to open theImagePath
        scale theImage to size 128
        set theTargetPath to (imageFolder & name of theImagePath)
        save theImage in theTargetPath as JPEG with icon
        close theImage
    end repeat
end tell

Memory leak in JCO when calling an ABAP-function that returns larg tables

Hello everybody,
I think discovered a memory leak in JCO when the calling functionions that have exporting tables with large datasets. For example the ABAP-function RFC_READ_TABLE, which in this example I use to retrieve data from a table called "RSZELTTXT", which contains ~ 120000 datasets. RFC_READ_TABLE exports the data as table "DATA".
Here a simple JUnit test:
http://pastebin.ca/1420451
When running it with Sun Java 1.6 with standard heap size of 64mb I get a heapsize OutOfMemory error:
http://pastebin.ca/1420472
Looking at the heap dump (which I unfortunately cannot post here, because of it' size), I can see that I've 65000 char[512] array objects in my heap, which don't get cleaned up. I think, each char[512] array stands for one dataset in the exporting table "DATA", since the table contains 120000 datasets, the heap is full after the first 65000 datasets are parsed. Apparently, JCO tries to read all datasets in memory instead of justing reading the dataset to which the pointer (JCoTable.setRow(i)) currently points to and releasing it from memory after the pointer moves forward ...
Did anybody else experience this?
Is SAP going to remove to issue in upcoming versions of JCO?
regards Samir

Hi,
Check Below links
1) How To Analyze Performance Problems JCO
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/3fbea790-0201-0010-6481-8370ebc3c17d
2) How to Avoid Memory Leaks
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c3e598fe-0601-0010-e990-b8622840c8c2
Salil...
Edited by: salil chavan on Jun 2, 2009 5:21 AM

Memory leak with large files and this code?

Okay, so the following code will not complete.
Basically what I am doing is opening 5 files and generating a PDF document from the already opened Report file, which is renewed prior to the PDF export operation. I left out the general stuff at the start. I am also writing out the location of the PDF to a HTML file, which acts as an index of the exported PDF documents.
I believe it is due to some sort of memory leak issue, but it could be something else or just my code. I have not stepped through this code, but for small file sizes of say less than 40 megs per file, it works fine. For file sizes along these lines:
File 1 = 230 megs
File 2,3 = 26 megs
File 4,5 = 8 megs
it will belch erors at the end of about the 5th iteration through the For loop. It will complete the 5th iteration however. Here are the errors generated:
84   10/26/2006 9:35:15 AM Error:
85   File handle for "W:\TR-2051-2100\TR-2060 - BAS Arctic PTC\Vnom\00-99\VnomCombined00-99\p1.TDM" file is invalid.
     (Error no. 6)
86   10/26/2006 9:40:19 AM Error:
87   File handle for "W:\TR-2051-2100\TR-2060 - BAS Arctic PTC\Vnom\00-99\VnomCombined00-99\p2.TDM" file is invalid.
     (Error no. 6)
88   10/26/2006 9:45:17 AM Error:
89   File handle for "W:\TR-2051-2100\TR-2060 - BAS Arctic PTC\Vnom\00-99\VnomCombined00-99\p3.TDM" file is invalid.
     (Error no. 6)
90   10/26/2006 9:53:07 AM Error:
91   File handle for "W:\TR-2051-2100\TR-2060 - BAS Arctic PTC\Vnom\00-99\VnomCombined00-99\p8.TDM" file is invalid.
     (Error no. 6)
92   10/26/2006 10:00:39 AM Error:
93   File handle for "W:\TR-2051-2100\TR-2060 - BAS Arctic PTC\Vnom\00-99\VnomCombined00-99\p9.TDM" file is invalid.
     (Error no. 6)
94   10/26/2006 10:01:01 AM Error:Error occured during file creation.
95   10/26/2006 10:01:01 AM Error:
     Error in <GM315 Pa...sing.VBS> (Line: 185, Column: 5):
     File handle for "W:\TR-2051-2100\TR-2060 - BAS Arctic PTC\Vnom\00-99\VnomCombined00-99\p1.TDM" file is invalid.
     (Error no. 6)
For files of a larger size, like:
File 1 = 7500 megs
File 2,3 = 80 megs
File 4,5 = 25 megs
This occurs after about 2-3 iterations through the loop.
see attachment for code.
Attachments:
ForLoopCode.txt ‏11 KB

i am having a similar error randomly inserted in the log
25   10/1/2009 1:58:36 PM Error:
26   File handle for "C:\Program Files\Summitek\Spartan\data\tdm\S Parameter Test $SSN$ 49.TDMS" file is invalid.
     (Error no. 6)
and
31   10/1/2009 1:58:37 PM Error:
32   File handle for "C:\Program Files\Summitek\Spartan\www\temp\Test_$SSN$ 49_602313172.pdf" file is invalid.
     (Error no. 6)
it doesn't seem to be causing an immediate problem, but i have had several unexplained Diadem lockups.
they occur in log after i use CALL DATAFILELOADSEL and CALL PICpdfexport
help? what does this mean
jim

[Bug?] X-Control Memory Leak with Large Data Array

[LV2009]
[Cross-posted to LAVA]
I have found that if I pass a large data array (~4MB in this example) into an X-Control, it causes massive memory allocations (1 GB+).
Is this a known issue?
The X-Control in the video was created, then the Data.ctl was changed to 2D Array - it has not been edited in any other way.
I also compare the allocations to that of a native 2D Array (which is only ~4MB).
Note: I jiggled the Windows Task Manager about so that JING would update correctly, its a bit slow, but it essentially just keeps rolling up and doesn't stop.
Demo code attached.
Cheers
-JG
Unable to display content. Adobe Flash is required.
Certified LabVIEW Architect * LabVIEW Champion
Attachments:
X Control Bug [LV2009].zip ‏42 KB

Hi Jon (cool name)
Thank you very much for your reply. We came to this conclusion in the cross post and it is good to have it confirmed by LabVIEW R&D. Your response is also similar to that of my AE which I got this morning as well - see below:
Note: Your reference number is included in the Subject field of this
message. It is very important that you do not remove or modify this
reference number, or your message may be returned to you.
Hi Jon,
You probably found some information from the forum. The US engineer has gotten back and he said that unfortunately that's expected behaviour after they have conducted some tests and this is what he replied:
"X Controls in the background use events structures. In particular the Data Change Event is called when the value of the XControl changes (writing to the terminal, local variable, or value change property). What is happening in this case is the XControl is getting called to fast with a large set of data that the event structure is queuing the events and data that a memory leak is produced. It is, unfortunately, expect behavior. The main work around for the customer in this case is not call the XControl as often. Another possibility is to use the Synchronous Display Property to defer updates to the Xcontrol, this might slow down a leak."
He would also like to know if you can provide with more details how you are using the Xcontrol, perhaps there is a better way. Please refer to the link below for synchronous display. Thank you.
http://zone.ni.com/reference/en-XX/help/371361G-01/lvprop/control_synchronous_display/
In my application I updated the X-Control @ 1Hz and it allocated at MBs/s up to 1+GB before it crashed, all within a few hours. That is why I called it a leak. I am really worried that if this CAR gets killed, there will still be an issue lingering that makes using X-Controls a major problem under the above conditions. I have had to pull two sets of libraries from my code because of this - when they got replaced with native LabVIEW controls the leak when away (but I lost reuse and encapsulation etc...).
Anyways, I really want to use X-Control tho (now and in the future) as I like all other aspect of them. If you do not consider this a leak, can a different #CAR be raised that may modify the existing behavior? I offer the suggestion (in the cross-post) that the data be ignored rather than queued? Similar to Christian's idea, but for X-Controls. Maybe as an option?
I look forward to discussing this with you further.
Regards
-Jon
Certified LabVIEW Architect * LabVIEW Champion

How can i detect "Memory leak" with large LabVIEW projects.

Hi,
I have a huge LabVIEW application that runs out of memory after running continuously for some time. I am not able to find out the VI that is hogging up memory. Is there any tool that dynamically detects the VI that is leaking memory.
Or, is there a tool or a way to identify the critical areas which can be potential culprits that is leaking memory.
Regards
Bharath

Bdev wrote:
Thanks Dennis.
I think Desktop Execution toolkit should solve the problem.
Wayne Wrote
Have you tried Tools»Profile»Performance and Memory ? http://zone.ni.com/reference/en-XX/help/371361F-01/lvdialog/profile/
But this will just give me the amount of memory used by the VIs and not the amount of memory that is not getting released.
And where is the problem about that? Just try to find what VIs keep increasing in memory size. That are the culprits. If you have real memory leaks, meaning there is memory that is not managed by LabVIEW directly but for instance by a DLL somewhere and that DLL looses references to memory, so it goes really lost, then the only way to find that is by successively exclude functionality in your application until you can find the culprit.
There is no other simple way to find out about who is loosing memory references than by doing debugging by exclusion until the problem disappears. The only way to speed this up, which quite often works for me is doing an educated guess, about what components are most likely to do this misbehaviour.
Not knowing anything about your application and if you are talking about memory hogs (fairly easily identifiable by the mentioned Performance and Memory monitor) or actual memory leaks, it is hard to tell how to go about it. Memory hogs are usually the first thing I suspect escpecially with software I inherit somehow from people from whom I'm not sure they know all the ins and outs of LabVIEW programming.
If a leak seems likely the first culprit usually are custom DLLs (yes even DLLs I have written myself), then NI DLLs such as DAQmx, etc. and last there come leaks in LabVIEW itself. This last category is very seldom but it has happened to me. However before going to scream about LabVIEW having a memory leak you really, really should make sure you have very intensivly researched all the other possibilities. The chance that you run into a memory leak in LabVIEW, while not impossible, is so small compared to the other ways of causing either a memory hog or running into a leak in an external component to LabVIEW, that in 99.9% of the cases where someone screams about a LabVIEW memory leak, he is simply wrong.
Rolf Kalbermatter
CIT Engineering Netherlands
a division of Test & Measurement Solutions

I am getting a memory leak that sometimes leads to an unresponsive Firefox, but does not crash per se.

Hello, For the last couple of months, I am having issues with one of my tabs or the program itself causing a memory leak. I was hoping that subsequent releases would fix the problem, but when I downloaded V.11 it did not help.
I use tab mix plus and at any time, usually have about 25 tabs open. Everything functioned okay for 8 or so months up until recently.
I am wondering if there is a way to try to track down what is causing the leak. If it is one of my open pages, i will get rid of it. I tried opening one page at a time from scratch, but could not find the issue. .
I always have flash block enabled to cut down on the website junk.
using OSX firefox v11.

Now, I'm not going to say it's an Add-On problem because from the research I've been doing on this problem for the last half hour shows that everyone has DIFFERENT add-ons, but everyone's having the SAME problem....
So I went through my add-ons and disabled them one by one, and the single add-on that has been giving me grief is the latest WOT add-on. So, I have Firefox 11 (so does my wife) and we both have the WOT add-on. But that's where the similarity ends... I have Windows 7 x64, she has Windows XP x86.... but she doesn't have the memory leak problem.
What I see is a sawtooth pattern over time. Memory goes up a little over 30+ seconds, then drops down. But over half an hour, the peaks of the sawtooth are larger, and it doesn't drop back down to the same level again - always a little more than before. And before you know it, FF is peaking at 1.5+GB, dropping down to 1.2GB... and FF is running very, very slowly.... excessive disk accesses (paging probably, though I apparently I still have 1.5 to 2.0 GB of free RAM). Killing FF frees it all up, and if I open FF again, it's back to using 250MB of RAM.
So, it's not the add-ons per se, but how they're interacting with FF (or the other way round).... most likely, it's this plug-in container they created to stop add-ons from taking FF with them when they crashed. Seems to have created more problems than it has solved..... would be great if you could choose not to use it....

DataSocket memory leak problem (2VO0SF00) -- more info?

When upgrading to LabVIEW 8.5 recently, I noticed the following known issue in the readme file:
"ID: 2VO0SF00
DataSocket/OPC Leaks Memory using ActiveX VIs to perform open-write-close repeatedly
If you call the DataSocket Open, DataSocket Write, and DataSocket Close functions in succession repeatedly, LabVIEW leaks memory. Workaround — To correct this problem, call the DataSocket Open function once, use the DataSocket Write function to write multiple times, and then use the DataSocket Close function."
Looking back, I think this problem may have been present in previous LabVIEW releases as well, and might be giving rise to a problem that's been dogging me for quite some time (see my thread, "Error 66 with DataSockets", http://forums.ni.com/ni/board/message?board.id=170&thread.id=187206), in addition to general slow/glitchy behaviour when my VI's have been running continuously for a long time. But in order to determine whether or not this issue affects me, and how I should go about fixing it in the context of my own programs, I need a bit more information about the nature of the issue itself and the inner workings of the DataSocket VI's. Any help or insight the community can provide into this would be greatly appreciated!
Here are my questions:
It is my understanding from the "known issue" description above that the memory leak happens when you have a DS Open wired to a DS Write wired to a DS Close, all inside a loop (example 1), and that the suggested workaround would be to move the DS Open and DS Close functions out of the loop on opposite sides, wired to the DS Write which remains inside the loop (example 2). Is this correct?
Does this leak also happen when performing DS open-read-close's repeatedly (example 3)?
What happens when a DS Write (or DS Read) is called without a corresponding DS Open and DS Close (examples 4a and 4b)? Does it implicitly do a DS open before doing the write operation and a DS close afterwards? What I'm getting at is this: would having an isolated DS Write (or DS Read) inside a loop, not connected to any DS Open or DS Close functions at all, cause this same memory leak?
If one computer is running the DS server and a second computer is running the VI with the repeated open-write-close's, on which computer does the memory leak occur?
In my question #1 workaround (example 2), the DS Open and DS Close outside the loop are routed through a shift register and in to and out of the DS Write inside the loop. If the DS connection id goes into the DS Write "connection in" and then splits and goes around the DS Write and out to the DS Close, without coming out of the DS Write "connection out" (example 5), will the memory leak still be avoided? I.e. if the DS Write function doesn't have anything connected to its "connection out", will it try to do an implicit DS Close?
If the VI causing the memory leak is stopped, but LabVIEW stays running, will the leaked memory be reclaimed? What if the VI is closed? What if all of LabVIEW is closed?
FYI, in the examples above "x1a" is a statically-defined DataSocket on the DS server running on the computer Max, to which the computer running the example VI's has read/write access. My actual application has numerous VI's and hundreds of DataSocket items, many of which are written to / read from every 50-100 ms in the style of examples 4a and 4b.
Does anyone have any idea about this stuff?
Thanks in advance,
Patrick
Attachments:
examples_jpg1.zip ‏63 KB
examples_vi1.zip ‏40 KB

Hi Meghan,
Yes, some of the larger VIs in my application do write to / read from several hundred DataSockets, so it's not feasible to use shift registers for each one individually, and hence why I'm passing the references into an array, etc.
Your Alternate Solution 2 is more along the lines of something that would work for me. However, my actual code has a lot of nested loops, sequences and DataSocket items which are not all written to in the same frame, so this solution would still be difficult to implement: it would be cumbersome to unpack the entire 500-element reference id array and build a new one (maintaining the positions and values of the unaffected elements) every time I write to some small subset of the DataSockets.
I think I have a solution which solves the problem and is also scalable to the size of my application -- I've attached it as Example 7. Do you think this will avoid the memory leak? It's the same as your Alternate Solution 2, except that instead of building a new array out of the DS Write reference outs, each reference out replaces the appropriate element of the original array.
If I understand you correctly, in order to avoid implicit reference opens and closes, a DS Write needs to have both it's reference in and reference out wired to something. Thus, even though my Example 7 replaces an element of the array with an identical value, and therefore doesn't actually change the array (which would be a silly thing to do normally), the DS Writes have their reference outs wired to something, and eventually in a convoluted way to a DS Close, so it should avoid the memory leak.
Just out of curiosity (I don't think anything like this would apply to my application or any fixes I implement), when would the implicit reference close happen in the attached Example 8? The DS Write has its reference in and reference out both connected to temporally "adjacent" DS Writes via the shift register, so perhaps it wouldn't try to close the reference on each loop iteration? Or would it look into the future and see that there is no DS Close and decide to implicitly do that itself? Or maybe only the DS Write on the last loop iteration does this?
Thanks for bearing with me through this,
Patrick
Attachments:
example73.JPG ‏40 KB
example83.JPG ‏14 KB

Memory leak using xslprocessor.valueof in 11.1.0.6.0 - 64bit ??

My company has made the decision to do all of our internal inter-system communication using XML. Often we may need to transfer thousands of records from one system to another and due to this (and the 32K limit in prior versions) we're implementing it in 11g. Currently we have Oracle 11g Enterprise Edition Release 11.1.0.6.0 on 64 bit Linux.
This is a completely network/memory setup - the XML data comes in using UTL_HTTP and is stored in a CLOB in memory and then converted to a DOMDocument variable and finally the relevant data is extracted using xslprocessor.valueof calls.
While this is working fine for smaller datasets, I've discovered that repeated calls with very large documents cause the xslprocessor to run out of memory with the following message:
ERROR at line 1:
ORA-04030: out of process memory when trying to allocate 21256 bytes
(qmxdContextEnc,)
ORA-06512: at "XDB.DBMS_XSLPROCESSOR", line 1010
ORA-06512: at "XDB.DBMS_XSLPROCESSOR", line 1036
ORA-06512: at "XDB.DBMS_XSLPROCESSOR", line 1044
ORA-06512: at "SCOTT.UTL_INTERFACE_PKG", line 206
ORA-06512: at line 28
Elapsed: 00:03:32.45
SQL>
From further testing, it appears that the failure occurs after approximately 161,500 calls to xslprocessor.valueof however I'm sure this is dependent on the amount of server memory available (6 GB in my case).
I expect that we will try and log a TAR on this, but my DBA is on vacation right now. Has anyone else tried calling the xslprocessor 200,000 times in a single session?
I've tried to make my test code as simple as possible in order to track down the problem. This first block simply iterates through all of our offices asking for all of the employees at that office (there are 140 offices in the table).
DECLARE
CURSOR c_offices IS
SELECT office_id
FROM offices
ORDER BY office_id;
r_offices C_OFFICES%ROWTYPE;
BEGIN
OPEN c_offices;
LOOP
FETCH c_offices INTO r_offices;
EXIT WHEN c_offices%NOTFOUND;
utl_interface_pkg.get_employees(r_offices.office_id);
END LOOP;
CLOSE c_offices;
END;
Normally I'd be returning a collection of result data from this procedure, however I'm trying to make things as simple as possible and make sure I'm not causing the memory leak myself.
Below is what makes the SOAP calls (using the widely circulated UTL_SOAP_API) to get our data and then extracts the relevant parts. Each office (call) should return between 200 and 1200 employee records.
PROCEDURE get_employees (p_office_id IN VARCHAR2)
l_request utl_soap_api.t_request;
l_response utl_soap_api.t_response;
l_data_clob CLOB;
l_xml_namespace VARCHAR2(100) := 'xmlns="' || G_XMLNS_PREFIX || 'EMP.wsGetEmployees"';
l_xml_doc xmldom.DOMDocument;
l_node_list xmldom.DOMNodeList;
l_node xmldom.DOMNode;
parser xmlparser.Parser;
l_emp_id NUMBER;
l_emp_first_name VARCHAR2(100);
l_emp_last_name VARCHAR2(100);
BEGIN
--Set our authentication information.
utl_soap_api.set_proxy_authentication(p_username => G_AUTH_USER, p_password => G_AUTH_PASS);
l_request := utl_soap_api.new_request(p_method => 'wsGetEmployees',
p_namespace => l_xml_namespace);
utl_soap_api.add_parameter(p_request => l_request,
p_name => 'officeId',
p_type => 'xsd:string',
p_value => p_office_id);
l_response := utl_soap_api.invoke(p_request => l_request,
p_url => G_SOAP_URL,
p_action => 'wsGetEmployees');
dbms_lob.createtemporary(l_data_clob, cache=>FALSE);
l_data_clob := utl_soap_api.get_return_clob_value(p_response => l_response,
p_name => '*',
p_namespace => l_xml_namespace);
l_data_clob := DBMS_XMLGEN.CONVERT(l_data_clob, 1); --Storing in CLOB converted symbols (<">) into escaped values (<, &qt;, >). We need to CONVERT them back.
parser := xmlparser.newParser;
xmlparser.parseClob(parser, l_data_clob);
dbms_lob.freetemporary(l_data_clob);
l_xml_doc := xmlparser.getDocument(parser);
xmlparser.freeparser(parser);
l_node_list := xslprocessor.selectNodes(xmldom.makeNode(l_xml_doc),'/employees/employee');
FOR i_emp IN 0 .. (xmldom.getLength(l_node_list) - 1)
LOOP
l_node := xmldom.item(l_node_list, i_emp);
l_emp_id := dbms_xslprocessor.valueOf(l_node, 'EMPLOYEEID');
l_emp_first_name := dbms_xslprocessor.valueOf(l_node, 'FIRSTNAME');
l_emp_last_name := dbms_xslprocessor.valueOf(l_node, 'LASTNAME');
END LOOP;
xmldom.freeDocument(l_xml_doc);
END get_employees;
All of this works just fine for smaller result sets, or fewer iterations (only the first two or three offices). Even up to the point of failure the data is being extracted correctly - it just eventually runs out of memory. Is there any way to free up the xslprocessor? I've even tried issuing DBMS_SESSION.FREE_UNUSED_USER_MEMORY but it makes no difference.

Replying to both of you -
Line 206 is the first call to xslprocessor.valueof:
LINE TEXT
206 l_emp_id := dbms_xslprocessor.valueOf(l_node, 'EMPLOYEEID');
This is one function inside of a larger package (the UTL_INTERFACE_PKG). The package is just a grouping of these functions - one for each type of SOAP interface we're using. None of the others exhibited this problem, but then none of them return anywhere near this much data either.
Here is the contents of V$TEMPORARY_LOBS immediately after the crash:
SID CACHE_LOBS NOCACHE_LOBS ABSTRACT_LOBS
132 0 0 0
148 19 1 0
SID 132 is a SYS session and SID 148 is mine.
I've discovered with further testing that if I comment out all of the xslprocessor.valueof calls except for the first one the code will complete successfully. It executes the valueof call 99,463 times. If I then uncomment one of those additional calls, we double the number of executions to a theoretical 198,926 (which is greater than the 161,500 point where it usually crashes) and it runs out of memory again.

JBoss EAP 6 On JRockit - Memory Leak

hello team.
I have memory leak problem on jboss and jrockit.
My Environment :
1. OS :
CentOS release 6.4 (Final)
2. JRockit :
java version "1.6.0_45"
     Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
     Oracle JRockit(R) (build R28.2.7-7-155314-1.6.0_45-20130329-0641-linux-x86_64, compiled mode)
3. Application Server:
JBoss EAP 6.2.0.GA
4. Application
Large EJB Application (100 and more EJB Beans (Stateless, Stateful, MDB, Timers and so on)
Everything works fine on older application server versions (4.3 , 4.2)
But now I have Problem
Of course I know that problem is new version - and i have discussion on JBoss forums.
but guys I have question about jrockit with you:
What is the option "Other" in memory ??
[jboss@jboss-new bin]$ jrcmd 17114 print_memusage
17114:
Total mapped                       8457864KB           (reserved=2983100KB)
-              Java heap              3145728KB           (reserved=0KB)
-              GC tables            105232KB
-          Thread stacks       46412KB           (#threads=138)
-          Compiled code       1048576KB           (used=12257KB)
-               Internal                   1480KB
-                     OS       170324KB
-                  Other       3639056KB
-            Classblocks         10496KB           (malloced=9631KB #28393)
-        Java class data       289536KB           (malloced=192391KB #133697 in 28393 classes)
- Native memory tracking     1024KB           (malloced=294KB #10)
[jboss@jboss-new bin]$
This size increases every time - and took entire amount of RAM.
Thank in Advance.
Regards,
Paata Lominadze

Not sure what the 'other' is, but it is probably best shown by using an example. By using displayMap we can display a memory map of various JVM subsystems and libraries that are loaded, including third-party libraries.
./jrcmd 4523 print_memusage displayMap
Total mapped                  3984796KB           (reserved=2978740KB)
-              Java heap       524288KB           (reserved=0KB)
-              GC tables        17548KB
-          Thread stacks        20276KB           (#threads=39)
-          Compiled code      1048576KB           (used=14224KB)
-               Internal         1672KB
-                     OS       146924KB
-                  Other      2092648KB
-            Classblocks         7424KB           (malloced=7381KB #20064)
-        Java class data       124416KB           (malloced=124411KB #91048 in 20064 classes)
- Native memory tracking         1024KB           (malloced=118KB #10)
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    OS                          *java    r x 0x0000000000400000.(     76KB)
    OS                          *java    rw 0x0000000000612000.(      4KB)
    OS                        *[heap]    rw 0x00000000007c1000.(    132KB)
   INT                           Poll    r   0x000000007fffe000 (      4KB)
   INT                         Membar    rw 0x000000007ffff000.(      4KB)
   MSP              Classblocks (1/2)    rw 0x00000000df8c0000 (   6912KB)
   MSP              Classblocks (2/2)    rw 0x00000000dff80000 (    512KB)
HEAP                      Java heap    rw 0x00000000e0000000.( 524288KB)
    OS                    *ld-2.12.so    r x 0x0000003664400000.(    128KB)
    OS                    *ld-2.12.so    r   0x000000366461f000 (      4KB)
    OS                    *ld-2.12.so    rw 0x0000003664620000 (      4KB)
    OS                   **ld-2.12.so    rw 0x0000003664621000.(      4KB)
   OS           *gconv-modules.cache    r   0x00007f8f2e4a0000 (     28KB)
THREAD                     Stack 4630    rwx 0x00007f8f2e4a7000 (      8KB)
THREAD                     Stack 4630        0x00007f8f2e4a9000 (     12KB)
THREAD                     Stack 4630    rwx 0x00007f8f2e4ac000 (    244KB)
   MSP         Java class data (5/37)    rw 0x00007f8f2e4e9000 ( 14336KB)
   MSP         Java class data (9/37)    rw 0x00007f8f2fa40000 (   5888KB)
                                         rw 0x00007f8f30000000 (    188KB)
                                             0x00007f8f3002f000 ( 65348KB)
                                         rw 0x00007f8f34000000 (    132KB)
                                             0x00007f8f34021000 ( 65404KB)
                                         rw 0x00007f8f38000000 (    412KB)
                                             0x00007f8f38067000.( 65124KB)
   MSP        Java class data (10/37)    rw 0x00007f8f3c034000 ( 34048KB)
                                         rw 0x00007f8f3e174000 (   8200KB)
   MSP        Java class data (11/37)    rw 0x00007f8f3e976000 (    256KB)
    OS                     *libhpi.so    rw 0x00007f8fb37fc000 (      8KB)
    OS                    **libhpi.so    rw 0x00007f8fb37fe000 (      4KB)
CODE                  Compiled code    rwx 0x00007f8fb37ff000 (     64KB)
CODE                  Compiled code    rwx 0x00007f8fb380f000 (    128KB)
CODE                  Compiled code    rwx 0x00007f8fb382f000 (     64KB)
MSP        Java class data (37/37)    rw 0x00007f8ff83a1000 (    512KB)
    GC Modified Union Set (committed)    rw 0x00007f8ff8421000 (    132KB)
    GC                     Card table    rw 0x00007f8ff8442000 (   1024KB)
    GC        Object bits (committed)    rw 0x00007f8ff8542000 (   8196KB)
Here
- thread is thread related (such as thread stacks)
- int, internal use (such as pointer pages)
- heap, chunk used by JRockit for the Java heap
- os, mapped directly from the operating system, such as third party DLLs or shared objects
- msp, memory space. a memory space is a native heap with a specific purpose, for example native memory allocation inside the JVM
- gc, garbage collection related, for example live bits
- code, compiled code
The 'other' memory space looks to me (from the blank entries in the above print-out) like they are a memory pages to are still not used. When the JVM starts it mappes an amount of memory. To my knowledge JRockit uses mmap (mmap(2) - Linux manual page), which creates a new mapping in the virtual address space. JRockit reserves an amount of memory (Java Heap (heap where your object instances go) + its own runtime (all the others)).
To see where the growth is in the various memory spaces, you can use 'print_memusage baseline', after which you can execute print_memusage again, for example,
./jrcmd 4523 print_memusage scale=M
4523:
Total mapped                     3896MB      +4MB (reserved=2905MB -3MB)
-              Java heap          512MB           (reserved=0MB)
-              GC tables           17MB
-          Thread stacks           19MB           (#threads=39)
-          Compiled code         1024MB           (used=14MB +1MB)
-               Internal            1MB
-                     OS          143MB
-                  Other         2043MB
-            Classblocks            7MB           (malloced=7MB #20596 +532)
-        Java class data          126MB      +4MB (malloced=125MB +4MB #93640 +2592 in 20596 classes)
- Native memory tracking            1MB           (malloced=0MB #20 +10)
Note the additional column that prints out the difference in memory usage in relation to the baseline. You can also monitor native allocations by using trace_alloc_sites, memory allocations can then be displayed with different levels of detail using the level argument.
./jrcmd 4523 print_memusage trace_alloc_sites=true
4523:
Total mapped                  3989660KB   +4864KB (reserved=2974732KB -4008KB)
-              Java heap       524288KB           (reserved=0KB)
-              GC tables        17548KB
-          Thread stacks        20276KB           (#threads=39)
-          Compiled code      1048576KB           (used=15265KB +1040KB)
-               Internal         1672KB
-                     OS       146924KB
-                  Other      2092648KB
-            Classblocks         7680KB    +256KB (malloced=7669KB +287KB #20596 +532)
-        Java class data       129024KB   +4608KB (malloced=128967KB +4555KB #93640 +2592 in 20596 classes)
- Native memory tracking         1024KB           (malloced=236KB +118KB #20 +10)
./jrcmd 4523 print_memusage level=3
4523:
Total mapped                  3989660KB   +4864KB (reserved=2974732KB -4008KB)
-              Java heap       524288KB           (reserved=0KB)
-              GC tables        17548KB
-          Thread stacks        20276KB           (#threads=39)
-          Compiled code      1048576KB           (used=15265KB +1040KB)
-               Internal         1672KB
-                     OS       146924KB
-                  Other      2092648KB
-            Classblocks         7680KB    +256KB (malloced=2KB -7379KB #4 -20060) Not fully traced
-        Java class data       129024KB   +4608KB (malloced=26KB -124385KB #16 -91032 in 20596 classes) Not fully traced.
- Native memory tracking         1024KB           (malloced=118KB #10) Not fully traced.
     gather_memorymap_database                     memtrace.c: 206         91KB     +91KB (#1 +1)
           gather_memory_usage                  osal_mspace.c:5142          7KB      +7KB (#4 +4)
msGatherMSpacesUsageDatabase                  osal_mspace.c:6128          2KB      +2KB (#1 +1)
msGatherMSpacesUsageDatabase                  osal_mspace.c:6134         16KB     +16KB (#1 +1)
Note this is more on the JVM level, in your case in might be beneficial to investigate what is happening on the java heap. By using print_object_summary you can get insight how memory on the heap is used on a per-class basis. To get to the bottom of where the memory leak is you can use the memory-leak-detector (an example of its use can be found here Middleware Snippets: Fast, Faster, JRockit). You can also obtain a heapdump that can be analyzed by using for example MAT (see for an example here Middleware Snippets: Visualizing Class Loading). To obtain a heapdump you can run the command, for example,
[weblogic@machine1 bin]$ ./jrcmd 4523 runsystemgc full=true fullcompact=true
4523:
[weblogic@machine1 bin]$ ./jrcmd 4523 hprofdump filename=/home/weblogic/dump.hprof
4523:
Wrote dump to /home/weblogic/dump.hprof
Note that this first issues a full GC by using the runsystemgc command.

HTTPService + XML Load + Memory Leak

Hi all....
I have noticed a memory leak in my application. This leak was
not apparent when the application was completed some months back
which is what left me a little confused as all I have done since
was upgrade to Flex 3 and possibly updated / changed my Flash
player.
I think I have found the cause to this problem (below) but am
not 100% sure that it is the "actual" problem or any reasons to
back my thoughts up, so have listed what I have checked / tried
along the way (maybe I have missed something)....
My Discovery Process:
I started profiling my application but did not find anything
out of the ordinary. I did a code walk-through double checking I
had cleaned up after myself, removing and even nulling all items
etc but still to now success - the leak is still there.
I have profiled the app in the profiler for reasonably long
periods of time.
All the classes etc being used within the app are consistent
in size and instance amount and there is no sign of any apparent
leak.
I am using a HTTPService that is loading XML data which I am
refreshing every 5 seconds. On this 5 second data refresh some
class instances are incremented but are restored to the expected
amount after a GC has occurred. The GC seems to take longer, the
longer the app is running, therefore more and more instances are
being added to the app, but when the GC eventually runs it "seems"
to clear these instances to the expected amount.
After scratching my head for a while I decided to make a copy
of my application, rip everything out, and focus in my data load,
where I found a problem!
I have now just a HTTPService that loads an XML file every 5
seconds, and this is all I currently have in the app (as I ripped
the rest of the code out), e.g:
Code:
<mx:Application xmlns:mx="
http://www.adobe.com/2006/mxml"
....... creationComplete="initApp()" >
<mx:HTTPService
id="httpServiceResults"
url="
http://myIP:myPort/myRoot/myXML.cfm"
resultFormat="e4x"
result="httpResultHandler(event)" />
<script....... >
private var timerPulse:Timer;
private function initApp():void
httpServiceResults.send();
timerPulse = new Timer(5000, 0);
private function httpResultHandler(event:ResultEvent):void
timerPulse.start();
timerPulse.addEventListener(TimerEvent.TIMER, timerRefresh);
public function timerRefresh(eventObj:TimerEvent):void
timerPulse.stop();
timerPulse.removeEventListener(TimerEvent.TIMER,
timerRefresh);
timerPulse.reset();
httpServiceResults.send();
</script>
</mx:Application>
This is pretty much the code I am currently using and it
leaks.
I ran and monitored this in both the profiler and the
activity / task manager, and after running the app for 1800 seconds
(30 min) in the profiler, the memory size grew from 50mg to 165mg
just sending the HTTPService.
I tried loading the service in multiple ways including in AS
rather than MXML creating new instances of it each time, resetting
it, nulling it etc... but nothing prevented this memory increase.
I then tried to load the XML using different methods such as
using the URLRequest and URLLoader which again caused a memory
leak.
What still confuses me is that this leak did not exist in the
previous version and not a great deal has changed since then apart
from upgrading to Flex 3 and possibly upgrading my Flash payer
(which I believe is a possible cause)
After looking into this issue a bit more deeply, I read a few
blogs / forums and other people are experiencing the same problems
- even with a lot bigger leaks in some cases all when reloading
large sets of XML data into Flex - however, I as of yet found no
solution to this leak - people with a similar problem believe it is
not due to a memory leak more a GC error, and others pointing
towards the Flash Player itself that is leaking - I don't really
know.
Findings so far during investigation of this issue:
* App leaks for both HTTPService and ULRRequest / URLLoader
methods
* App only leaks when calling a data loader
* The size of the leak seems to depend on the size of the
XML being loaded
* The size of the leak also seems to be affected by the
applications heaviness - the greater seems to enhance the leak
An interesting factor I have noticed is that if I copy the
XML from my "myXML.cfm" that I link to in my HTTPService and copy
the contents of the file into my own XML file stored within the
Flex project root itself: ""myXML.xml"" the leak disappears... like
it seems to link when loading the XML over a network, however as my
network knowledge is not great I am not sure what to make of this -
any ideas???
Could the connection to the XML document cause leaks??? is
there anything else that could cause this leak??? have I something
in my code sample that could cause this leak??? or could any of the
other things I have mentioed cause this leak???
Any help / ideas would be greatly appreciated.
Thanks,
Jon.

I also observed heavy memory leak from using httpservice with
XML data. I am using Flex3 builder under Linux. My Flex application
polls httpservice every 10 seconds. The reply is a short XML
message less than 100 bytes. This simple polling will consume 30+
MB of memory every hour. I leave it idling for several hours and it
took 200 MB of memory. No sign of garbage collection at all.

Memory leak in WLS6.1 sp3 on Solaris?

We are running WLS 6.1 sp3 on Solaris 8 using Sun's 1.3.1 JVM with BEA's
JDriver to connect to Oracle (8.1.7). We have set the JVM to a maximum heap
of 512M (-Xms256m -Xmx512m).
The issue is that we are now seeing (via pmap) the resident memory growing
to ~1GB. Yet, the WLS management console is reporting memory usage of ~200M.
So, if we are to believe WLS, then the JVM (or some native library) is
chewing up the extra memory.
Having examined the results of "pmap -x <pid>", I see the heap (and some
anon) mem/file are quite large.
Address Kbytes Resident Shared Private Permissions Mapped File
00026000 418912 416096 - 416096 read/write/exec [ heap ]
D8C00000 524288 447928 - 447928 read/write/exec [ anon ]
total Kb 1083832 981136 37960 943176
Questions:
- Are there any known memory leaks with 6.1 sp3 on Solaris?
- Should it be possible for pmap's value of "heap" to grow larger than the
"Xms" setting? We have observed this recently, but the numbers above don't
reflect this.
- Any idea what the "anon" is or how to determine what it is?
- Do you know of any tools that might help me determine what is using that
memory or what is in it?
Any suggestions will be greatly appreciated...
Thanks in advance,
Erik

If you set -Xmx512m then the java heap will grow no larger than 512MB.
WLS is reporting the size of the java heap. (It is a java program after
all and doesn't know a lot about what native code might be doing.)
My first thought is try running with Oracle's thin (type-4) driver.
Since this is java code, it will use the java heap. If the memory usage
drastically changes, then you can point the finger at the jDriver.
Are you using any other native code / libraries?
-- Rob
Erik Westland wrote:
We are running WLS 6.1 sp3 on Solaris 8 using Sun's 1.3.1 JVM with BEA's
JDriver to connect to Oracle (8.1.7). We have set the JVM to a maximum
heap
of 512M (-Xms256m -Xmx512m).
The issue is that we are now seeing (via pmap) the resident memory growing
to ~1GB. Yet, the WLS management console is reporting memory usage of
~200M.
So, if we are to believe WLS, then the JVM (or some native library) is
chewing up the extra memory.
Having examined the results of "pmap -x ", I see the heap (and some
anon) mem/file are quite large.
Address Kbytes Resident Shared Private Permissions Mapped File
00026000 418912 416096 - 416096 read/write/exec [ heap ]
D8C00000 524288 447928 - 447928 read/write/exec [ anon ]
total Kb 1083832 981136 37960 943176
Questions:
- Are there any known memory leaks with 6.1 sp3 on Solaris?
- Should it be possible for pmap's value of "heap" to grow larger than the
"Xms" setting? We have observed this recently, but the numbers above don't
reflect this.
- Any idea what the "anon" is or how to determine what it is?
- Do you know of any tools that might help me determine what is using that
memory or what is in it?
Any suggestions will be greatly appreciated...
Thanks in advance,
Erik

How to fix huge iTunes memory leak in 64-bit Windows 7?

iTunes likes to allocate as much as 1.6GB of memory on my dual-quad XEON 8GB 64-Bit Windows computer and then becomes unresponsive.
This can happen several times a day and has been going on for as long as I can remember. No other software that I use does this - only Apple's iTunes. Each version I have installed of iTunes appears to have this same memory leak. Currently I am running version 10.7.0.21.
I love iTunes when it works. But having to constantly kill and relaunch the app throughout the day is bringing me down.
Searching for a fix for this on the internet just surfaces more and more complaints about this problem - but without a solution.
Having written shrinkwrapped software for end users as well as for large corporations and governments for more than 25 years I know a thing or two about software. A leak like this should take no more than a day or two to locate using modern software tools and double that to fix it. So why with each new version of iTunes does this problem persist? iTunes for Windows is the flagship software product Apple makes for non-Mac users - yet they continue to pass up each opportunity they have had over the years with each new release to fix this issue. Why is this?
Either the software engineers are not that good or they have been told NOT to spend time on this issue. I personally believe that the engineers at Apple are very good, and therefore am left thinking that the latter is more likely the case. Maybe this is to coax people to purchase a Mac so that they can finally run iTunes without these egregious memory leaks. I would like to offer another issue to consider.
Just as Amazon sold Kindles and Google sold Nexus tablets at low cost - not counting on margin for profit - but instead they wanted to saturate the marketplace with tools for making future purchases of content almost trivial to do with their devices. Apple also counts on this model with their pricer hardware - but they also have iTunes. Instead of trying to get people to switch to a MAC by continuing to avoid fixing this glaring issue in iTunes for Windows I would like to suggest that by allowing their engineers to address this issue that Apple will help keep Windows users from jumping ship to another music app. The profit to be made by keeping those Windows users happy and wedded to the iTunes store is obvious.
By continuing to keep this leak in iTunes for Windows all it does is lower my esteem for the company and start to make me wonder if the software is just as buggy on Macs.

I have same issue. Ongoing for more than 1 year and currently running iTunes 11.3.
My PC is Dell OptiPlex 990 I7 processor, 8GB ram, W7 64 [always keep things patched up to latest OS updates etc]
I use this iTunes install to stream music videos etc to multiple appleTVs, ipads, iphones etc .. via Home Sharing
Store all my media including music, videos and apps on separate NAS .. so the iTunes running on PC is only doing the traffic cop role and streaming / using files stored on NAS .. creates lots of IO across my network
Previous troubleshooting suggest possible contributing causes include
a) podcast updates .. until recently I had this auto updates on multiple podcast subscriptions, presumably the iTunes would flow this from the PC to save on the NAS across the network .. if the memory leak is in the iTunes network communication layer (?bonjour?) this may be sensitive to IO that would not normally occur if the iTunes file saving was local on the same PC
b) app updates .. have 200+ apps in my library and there is always a batch of updates .. some updates 100s of MB is size .. routinely see 500MB to 1GB of updates in single update run .. all my apps are
c) streaming music / movies .. seems when we ramp up streamlining of music or movies . memory leak grows faster .. ie within hours of clean start
c) large syncs of music or videos to ipads or iphones .. noticed that get big problems when I rebuild an ipad .. I typically have 60+ GB of data in terms of apps / music / videos to load .. have to do rebuild in phases due to periodic lockups

Does the 3dpc line plot have a memory leak? (ver 8.6.1)

I have a very large application with a very small memory leak and the 3dpc line plot appears to be the most likely candidate.
I create a line plot of data (100-200 points of XYZ) pass this to a Create 3dpc_surfaceplot:xctlcreate.vi in a loop (updated at a few hz).
This code appears to grow in memory slowly over time (not at a regular pace sometimes a garbage collector must kick in). Is this possible, is there anything I have to do to use this function in a loop, there is no help file provided for this vi.
Can I avoide this situation (I am having trouble replicating te out of memory my client is seeing but again this is the most likely cause)?
Paul Falkenstein
Coleman Technologies Inc.
CLA, CPI, AIA-Vision
Labview 4.0- 2013, RT, Vision, FPGA

Hi Paul,
Thank you for bring this to our attention. Do you have a small piece of code that exhibits this memory leak? If so, can you please it so that I can try to reproduce the issue here? If I can, then I can report the issue to R&D for further investigation.
Aaron P
National Instruments
Applications Engineer
http://www.ni.com/support

Large Memory Leak?

Similar Messages

Maybe you are looking for