PyOpenCL - leaking memory at each kernel execution

I found a frustrating problem in pyopencl - after each kernel execution host memory consumption increases by approx 1.5 MB. Taking into account program workflow (modelling some amount of iterations on card, finishing kernel, reading data via enqueue_copy, writing it to file, then starting kernel again), I run out of my 4 GB of RAM after some thousands of such calls.
In a previous project which was on C++ I solved this problem this way. An event object was dynamically created and passed to enqueueNDRangeKernel(). After reading data from GPU event object was deleted. But in Python it's impossible to use this method.
Also it's worth to notice that the memory leak occures when using both CPU or GPU. I'm using Intel CPU and nVidia GPU.
What can be done to fix the problem?

I have the same problem. Searching this forum reveals we are not alone. It appears a few people have submitted a bug report to Apple but it can't hurt to send more.

Similar Messages

Huge memory leaks after upgrading from kernel 2.6.38(?)

My system began leaking memory some months ago and I've little to no idea what is causing it. The closest hint I can give is that I think these problems started around upgrading from Linux 2.6.38 to 2.6.39 and have continued with 3.0 aswell. Sometimes my whole system freezes due to extensive swapping (HDD usage led stays lit), sometimes it stops after consuming almost all of my 4 GB RAM.
I think the leak has to come from inside the kernel because all sysmon apps claim no application is eating too much RAM yet the system memory consumption is at 100 % or even using swap. Even after I kill Xorg the memory usage will stay at least at 2.0 GB, when it should be less than half a gig.
I'd like to guess the blame is on Nouveau, because it was giving me a hell of a time back with Linux 2.6.39. It had a habit of freezing my system when re-enabling suspended compositing (both with Kwin and Mutter). Also video playback was corrupted sometimes. These probs have since been fixed with Linux 3.0.
I cannot name any spesific app that might trigger this memory leaking. I don't play any games and mainly use Firefox, Chromium and VLC day in, day out.
There seems to be absolutely no way of reclaiming the reserved memory than rebooting the whole system. I have even tried unloading Nouveau.
I am currently running the [testing] repo with ~daily upgrades.
Last edited by Verge (2011-08-16 14:04:09)

OK, this kind of surprised me. No wonder I couldn't find any relevant stuff while googling for kernel issues. I have had Strigi disabled for some time now because it is miserably broken, but I now disabled the whole Nepomuk too. Let's see if my problem now goes away or what...
The release of KDE 4.7 Beta 1 actually lines up with the launch of Linux 2.6.39 so this really might be the case. I moved to KDE 4.7 already with the first beta.
Last edited by Verge (2011-08-16 14:30:37)

Question on use of shared memory objects during CIF executions

We have a CIF that runs in background via program RIMODACT that is invoked from our external job scheduler. (The schedulere kicks off a job - call it CIFJOB - and the first step of this job executes RIMODACT.)
During the execution of RIMODACT, we call a BAdI (an implementation of SMOD_APOCF005.)
In the method of this BAdI, we load some data into a shared memory object each time the BAdI is called. (We create this shared memory object the first time the BAdI is called.)
After program RIMODACT finishes, the second step of CIFJOB calls a wrapper program that calls two APO BAPI's.
Will the shared memory object be available to these BAPIs?
Reason I'm asking is that the BAPIs execute on the APO app server, but the shared memory object was created in a CIF exit called from a program executing on the ECC server (RIMODACT).
Edited by: David Halitsky on Feb 20, 2008 3:56 PM

I know what you're saying, but it doesn't apply in this case (I think.)
The critical point is that we can tie the batch job to one ECC app server. In the first step of this job (the one that executes RIMODACT to do the CIF), we build the itab as an attribute of the "root" shared memory object class.
In the second step of the batch job, we attach to the root class we built in the first step, extract some data from it, and pass these data to a BAPI that we call on the APO server. (This is what I meant by a "true" RFC - the APO BAPI on the APO server is being called from a program on the ECC server.)
So the APO BAPI never needs access to the ECC shared memory object - it gets its data passed in from a program on the ECC server that does have access to the shared memory object.
Restated this way, is the solution correct ???

GetOracleDecimal seems to leak memory?

Hello, I am using the C# statement below to retrieve a decimal value from an OracleDataReader, adjust the precision, then assign it to a .NET decimal type:
decimal volume = (decimal)OracleDecimal.SetPrecision(reader.GetOracleDecimal(5), 28);
Over time, the service that hosts this method leaks memory. After some investigation, I found that "Oracle.DataAccess.Types.OpoDecCtx" types were collecting in the finalize queue. Any ideas on how to resolve this? I can't explicitly Dispose() an OracleDecimal...
I'm using ODP.NET version 2.102.2.20.
Thanks

Thanks for the reply. I am using .NET 2.0 SP2. I upgraded the ODP.NET client to 2.111.7.20 but am still seeing leakage of the OracleDecimal structs.
Here's a snippet of output from WinDbg:
Ready for finalization 32874 objects (0b4de540->0b4fe6e8)
Statistics:
MT Count TotalSize Class Name
00fb8cec 32816 656320 Oracle.DataAccess.Types.OpoDecCtx
With each pass of the service the Count and TotalSize grow in increments equal to the number of rows I'm querying...
Just for laughs I changed the GetOracleDecimal statement to GetDecimal and the memory issue went away (because I'm no longer creating OracleDecimals). But I need to use the higher precision type to prevent overflows..
Thanks again..

Imapd service "Leak Memory"

I already have Java Communications Suite 6 Working , but, I have a Leak Memory problems with imapd process.
I have the server with 30 imapd process, 30 mshttpd process, and 30 pop process running to balance the load of the users.
All work fine(the CPU is 100% idle, the RAM is 80% available=6GB) when I access via browser(Chrome,Firefox), too when I access it with the traditional clients IMAP (Outlook, ThunderBird,etc).
The problem is that if one user want to copy(not just view) locally all your emails to your laptops/pc, the imapd process eat more than 3GB(instead of 80MB) of RAM and the system is getting slower.
If the user cancel this process(copy via IMAP all your emails) the process dont freedom doesn't release the memory and the system getting slower.
Actually my solution is:
# kill -9 <PID_imapd>
Some times when I kill the process the problem going to other imapd process, so in this case the solution is:
/opt/sun/comms/messaging/sbin/stop-msg
/opt/sun/comms/messaging/sbin/start-msg
I think that this problem is a bug but I dont know what is the patch.
May you help me? Any idea?
My system is:
A X2100 with one socket AMD Opteron 2600Mhz, 8GB RAM, 2 x disks 750GB in mirror with Solaris Volume Manager, Soalris 10 HW 10/08.
Thanks.
Jaime

Jorth, maybe we are not well explained,
the normal operation of imapd process(one process) need no more that 200MB, when a user is linked to it.
Check my example:
================================================================================================================
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
874 root 610M 421M sleep 59 0 0:14:51 0.1% java/160
818 webservd 413M 285M sleep 59 0 0:16:43 0.3% webservd/95
607 root 326M 45M sleep 59 0 0:07:01 0.1% webservd/58
16719 mailsrv 283M 50M sleep 59 0 0:00:06 0.0% imapd/5
798 root 237M 37M sleep 59 0 0:04:04 0.1% ns-slapd/48
*16673 mailsrv 220M 97M sleep 59 0 0:00:06 0.0% imapd/6*
*16677 mailsrv 210M 42M sleep 59 0 0:00:05 0.0% imapd/5*
*16696 mailsrv 205M 36M sleep 59 0 0:00:04 0.0% imapd/5*
*16695 mailsrv 200M 38M sleep 59 0 0:00:04 0.0% imapd/6*
*16685 mailsrv 197M 38M sleep 59 0 0:00:08 0.0% imapd/5*
*16682 mailsrv 195M 40M sleep 59 0 0:00:04 0.0% imapd/5*
*16681 mailsrv 193M 46M sleep 59 0 0:00:04 0.0% imapd/5*
*16690 mailsrv 187M 52M sleep 59 0 0:00:03 0.1% imapd/6*
*16687 mailsrv 187M 40M sleep 59 0 0:00:05 0.0% imapd/5*
*902 icsuser 185M 24M sleep 59 0 0:00:05 0.0% cshttpd/2*
NPROC USERNAME SWAP RSS MEMORY TIME CPU
78 mailsrv 2439M 2329M 28% 0:11:53 0.5%
14 icsuser 1064M 233M 2.8% 0:06:55 0.0%
136 root 965M 727M 8.9% 2:15:20 50%
1 webservd 264M 326M 4.0% 0:16:43 0.3%
1 noaccess 90M 100M 1.2% 0:01:13 0.0%
Total: 237 processes, 1042 lwps, load averages: 1.91, 1.90, 1.89
================================================================================================================
When we run an aplication like OutLook to copy locally to the laptopn all the mail via IMAP, "ONE imapd process" eat more memory that need it and never released.
Example:
================================================================================================================
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
*16673 mailsrv 2856M 2534M sleep 59 0 0:00:06 45% imapd/6* <---------------this process eat memory and slow the system(CPU/RAM)
874 root 610M 421M sleep 59 0 0:14:51 0.1% java/160
818 webservd 413M 285M sleep 59 0 0:16:43 0.3% webservd/95
607 root 326M 45M sleep 59 0 0:07:01 0.1% webservd/58
16719 mailsrv 283M 50M sleep 59 0 0:00:06 0.0% imapd/5
798 root 237M 37M sleep 59 0 0:04:04 0.1% ns-slapd/48
*16677 mailsrv 210M 42M sleep 59 0 0:00:05 0.0% imapd/5*
*16696 mailsrv 205M 36M sleep 59 0 0:00:04 0.0% imapd/5*
*16695 mailsrv 200M 38M sleep 59 0 0:00:04 0.0% imapd/6*
*16685 mailsrv 197M 38M sleep 59 0 0:00:08 0.0% imapd/5*
*16682 mailsrv 195M 40M sleep 59 0 0:00:04 0.0% imapd/5*
*16681 mailsrv 193M 46M sleep 59 0 0:00:04 0.0% imapd/5*
*16690 mailsrv 187M 52M sleep 59 0 0:00:03 0.1% imapd/6*
*16687 mailsrv 187M 40M sleep 59 0 0:00:05 0.0% imapd/5*
*902 icsuser 185M 24M sleep 59 0 0:00:05 0.0% cshttpd/2*
NPROC USERNAME SWAP RSS MEMORY TIME CPU
78 mailsrv 2439M 2329M 80% 0:00:5 0.5%
78 mailsrv 2439M 2329M 10% 0:11:53 0.5%
14 icsuser 1064M 233M 2.8% 0:06:55 0.0%
136 root 965M 727M 8.9% 2:15:20 50%
1 webservd 264M 326M 4.0% 0:16:43 0.3%
Total: 237 processes, 1042 lwps, load averages: 1.91, 1.90, 1.89
===================================================================================================
The rest of the imapd process(29) in this moment just load 40-50MB(when those are free).
If we kill the process 16673 the user connect to the next imapd process and the problem persist whit the same patology: eat memory and CPU, and slow the system.
So, we think that is not normal and maybe we need patch the system to solve the problem.
We will to test the 64Bit version in this moment and we will post the results.
Thank
Jaime

HT3919 In the past was able to adjust the memory for each individual application, my Illustrator CS5 appears is doesn't have enough memory to complete a particular project. How do I boost the amount of memory that Illustrator uses so I can complete my pro

In the past I have always been able to adjust the memory of each individual application,
my Adobe Illustrator CS5 appears that is doesn't have enough memory to complete a particular project.
How do I boost the amount of memory that Illustrator uses so I can complete my project?

With OS X, it automatically allocates RAM and it can't be allocated manually.
What model Mac do you have and what OS X version is it running. Your profile shows a PowerMac and OS 10.6.8. A PowerMac can't run OS 10.6.8. It's max OS is 10.5.x
How much RAM is installed in your Mac? You need to max out the RAM and close other open apps if you are having problems with Illustrator.
 Cheers, Tom

The memory for each PDF file does not add up to the Adobe Reader App size

Why does the memory for each individual PDF file (about 250 MB) not add up to the total memory (690 MB) of the Reader app for my iPad 2?

What version of Reader and OS X?

Leak memory in xerces DomParser

Hey,
I have " leak memory " in my Java application (Stand alone).
I suspect that this leak caused by xerces DomParser.
i reached to this conculsion after using in Jprobe as a memory debuger.
Does any one has a similar problem?
Thank you.
code:
DOMParser parser = new DOMParser();
parser.parse(new InputSource(new FileInputStream(schemaFile)));

Hey, which Version of Xerces do you use?

Why does Firefox 4 (4.0.1 specifically) leak memory?

'''Answer:''' I figured it out.
History lesson first. Ever since the 4.0.1 update, Firefox 4 has been crashing A LOT. It's gotten so bad that FF4.0.1 will eat memory with no end in sight and eventually Windows (Win 7 64-bit) forcefully close the program due to an out of memory error.
I have 8 GB of RAM installed. And, it would crash once it exceeds 1.5 GB of RAM used by Firefox 4. I've seen Firefox 4 go up to 1.85 GB of RAM in Task Manager. This is with 20 tabs open.
With the add-on Leak-Monitor, I've narrowed down the issue. Having only Leak-Monitor as the extension installed, all pop-ups point to how '''poorly Firefox 4 handles Javascript'''!
MSNBC Leak Monitor issue:
[http://www.octoberasian.com/tech/text/FF4-Leak-Monitor_MSNBC.txt Leak Monitor log from MSNBC.com]
I've gotten more pop-ups from even support.mozilla.com because of the Javascript on this website.
Firefox 4.0.1 is leaking memory from every Javascript on every website I load. It's leaking memory as we speak with only this page loaded (one tab).
FIX IT! Firefox 4 is poorly handling Javascript, especially when multiple tabs are opened.

octoberasian: I, too, am experiencing this phenomenon. I shut my system down, pulled the EIC cable, held down the power button - then replugged and rebooted. I closed down every unneeded application that I have set to auto-start. Finally, I opened a single Firefox 4.0.1 window and let it sit. It began with consuming 43 Mb of memory. By afternoon - about 5 hours later - it was up to 203 Mb. There is definitely something very wrong with this version. It does the same on all the machines we have, sans our server. We don't use the server for anything but being a server & it has some version of Firefox 3 on it anyway. No Java Scripts were running, nor were any Adobe ARM or any plugins. I noticed when plugins run - the memory leakage goes into overdrive. Hey, I'm just sayin'...

Skype 6.4 continues to leak memory at an astonishing rate? Hard crashes ALL my Macs

Last version of Skype 6.3 leaked memory like a river full of rapids. And once it gobbled up 1.5gigs or so of memory, it HARD crashed my iMac requiring a hard reboot. It only takes about 30 minutes for skype to do this. My iMac (2012 Core i7 3.4ghz 32 gigs RAM) is at Mac OSX 10.7.5 and it is repeatable every time. I have to set timers to remind me to restart Skype every 30 minutes.
We thought 6.4 would fix the issue but no luck doing exact same thing. My colleague is across the country on an iMac with 10.7.5 same issues. Repeatable on my Macbook Pro running 10.8. If he let's it run, he gets the black screen of death and have to hard restart. Just like me.
This issue has been around for several weeks and is making skype unusuable for us and we rely on it heavily. Note we are PAYING customers. to Skype.
This ONLY happens if we use the video feature. Submitted to Skype, wondering if anyone else having issue?

Last version of Skype 6.3 leaked memory like a river full of rapids. And once it gobbled up 1.5gigs or so of memory, it HARD crashed my iMac requiring a hard reboot. It only takes about 30 minutes for skype to do this. My iMac (2012 Core i7 3.4ghz 32 gigs RAM) is at Mac OSX 10.7.5 and it is repeatable every time. I have to set timers to remind me to restart Skype every 30 minutes.
We thought 6.4 would fix the issue but no luck doing exact same thing. My colleague is across the country on an iMac with 10.7.5 same issues. Repeatable on my Macbook Pro running 10.8. If he let's it run, he gets the black screen of death and have to hard restart. Just like me.
This issue has been around for several weeks and is making skype unusuable for us and we rely on it heavily. Note we are PAYING customers. to Skype.
This ONLY happens if we use the video feature. Submitted to Skype, wondering if anyone else having issue?

DS 5.1sp2 SLAPI plugins leak memory

I am running DS 5.1sp2, on Solaris 9/SPARC, and have been encountering problems with the ns-slapd process leaking memory when using SLAPI plugins.
We have written our own bind pre-operation plugin function which, when enabled, causes ns-slapd to leak memory at a pretty rapid rate. I have a simple test where I perform a simple ldapsearch in an endless loop. The ns-slapd process starts off with a VSS of ~617MB (we have a fair amount of cache enabled) ; after a few minutes of my test, the VSS grows by several MB. With the plugin disabled, I can run the same test for hours with no growth in the VSS of ns-slapd.
I have also performed this test with my plugin disabled, and the "testbind" plugin function (the one provided in <server root></server>/plugins/slapd/slapi/examples/testbind.c) enabled. I see even more rapid memory leakage than with my plugin.
Are there known issues with SLAPI memory management? Is there anything I can do other than restarting ns-slapd on a regular basis (in production, this memory leak causes VSS growth of several hundred MB in the course of a few days).

I think if you want to modify the result before it is returned, your plugin needs to be invoked post-op as well. Your plugin can be both at the same time, just make sure your init function knows what to do when called a second time.
Moreover, without looking at the stack trace, it is hard to make any progress. Do you still have the core file? Can you paste the pstack and pflags output ?
# Run "file" on the core to see if its a 32 or 64 bit core
# cd /path/to/ns-slapd
# pstack <corefile>
# pflags corefile
Alternatively, see if you can reproduce the problem with a small plugin. Smaller the better. I'll give it a go.

Does ReportDocument leak memory?

Using CrystalDecisions.CrystalReports.Engine.ReportDocument XI R2. w3wp.exe memory use raises while a report is generated (which is expected), yet it does not seem to be ever released after. If you keep generating same report again and again, IIS eventually runs out of memory. A report does not call any external DLLs or COM objects. Calling repDoc.Dispose() does not seem to help. Please help. 

Methods:
Dispose()
GC.WaitForPendingFinalizers()
GC.Collect()
Details:
I created a C# application that connects to BOXI.
It does not specifically use the ReportDocument object, but was also encountering serious memory issues.
I was also using the Dipose method to mark objects for garbage collection.
.Net garbage collection happens automatically (at some time or another), but you can force it using GC.Collect(). I tried this, but was still leaking memory.
Until I found GC.WaitForPendingFinalizers(), which causes the garbage collector to wait until all disposed objects have completely finished their destructors. I think this made the difference with BOXI objects, because they are nested.
After I was using all three methods, I had no more memory issues. I realize this information may be out of date, but I posted it for future seekers.

Flash is killing Safari, which is leaking memory ...

I just saw in Activity Monitor that the Flash plugin is using 147% of the CPU, with one Flash movie playing in one of 12 open Tabs
And Safari seems to leak memory as well, after a day or two it easily can use up to 3GB(!) of RAM.
I am now quitting Safari routinely every 2 to 3 hours to get the CPU usage down and free RAM for other stuff, but I don't think this method makes for a great user experience.

Hi,
Use ClickToFlash... that way you have control of Flash content.
http://clicktoflash.com/
Carolyn

How to Free Up Memory Allocated by Kernel without reboot

Hi
In one of the Production Server we are facing issue in which server is up for last 365 days.
While checking we found that kernel memory utilization is itself 64 %.
I want solution by which we can free up Memory used by kernel which should be normally 12-15 %.
I am aware about that rebooting the server will resolve the issue.
echo ::memstat | mdb -k
>
Page Summary Pages MB %Tot
Kernel 166378 1299 64%
Anon 38042 297 15%
Exec and libs 1161 9 0%
Page cache 39329 307 15%
Free (cachelist) 12152 94 5%
Free (freelist) 1326 10 1%
Total 258388 2018
Physical 254514 1988Please suggest.
Thanks
Rajan

hi
see this URL and alternatively you can add temp SWAP
http://developers.sun.com/solaris/articles/solaris_memory.html

Simple JSP leaks memory?

Simple HelloWorld.jsp leaks memory significantly. Tested with SoapUI setup with 40 clients driving load. At around invocation 100,000 the WLS server simply leaks enough memory to hit the upper heap size and dies. I'm running V10.3.2 (latest) in Production mode with the JRocket JVM. I also tried the same test with HelloWorld.HTML file and found no memory leaks, it runs HTML test fine up to 1,000,000 invocations with no memory or thread leaks.
This test can not be any simpler.
Also, performance goes down significantly as memory leaks! I go from 500tps down to 100tps just prior to running out of memory.
The JSP file is extremely simple, here's the code:
<%@ page language="java" contentType="text/html; charset=ISO-8859-1" pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Hello World JSP</title>
</head>
<body>
Hello World!
</body>
</html>
Why would such a simple JSP file cause the WebContainer to leak memory? I'm at a loss here.
Edited by: user9200447 on Feb 26, 2010 10:47 AM

It depends on the test being run. One possibility is it may be due to implicit session objects.
Try using session=false otherwise session objects are created implicitly for this jsp. Dependng on sesson timeout, hits and java heap memory size you can run out of memory
HTH

PyOpenCL - leaking memory at each kernel execution

Similar Messages

Maybe you are looking for