Memory leak under GNU/Linux when using exec()

Question

Hi,
We detected that our application was taking all the free memory of the computer when we were using intensively and periodically the method exec() to execute some commands of the OS. The OS of the computer is a GNU/Linux based OS.
So, in order to do some monitoring we decided to wrote a simple program that called exec() infinite number of times, and using the profiler tool of Netbeans we saw a memory leak in the program because the number of surviving generations increased during all the execution time. The classes that have more surviving generations are java.lang.ref.Finalizer, java.io.FileDescriptor and byte[].
We also decided to test this simple program using Windows, and in that OS we saw that the memory leak disappeared: the number of surviving generations was almost stable.
I attach you the code of the program.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
public class testExec
    public static void main(String args[]) throws IOException, InterruptedException
        Runtime runtime = Runtime.getRuntime();
        while (true)
            Process process = null;
            InputStream is = null;
            InputStreamReader isr = null;
            BufferedReader br = null;
            try
                process = runtime.exec("ls");
                //process = runtime.exec("cmd /c dir");
                is = process.getInputStream();
                isr = new InputStreamReader(is);
                br = new BufferedReader(isr);
                String line;
                while ((line = br.readLine()) != null)
                    System.out.println(line);
            finally
                process.waitFor();
                if (is != null)
                    is.close();
                if (isr != null)
                    isr.close();
                if (br != null)
                    br.close();
                if (process != null)
                    process.destroy();
}¿Is anything wrong with the test program we wrote? (we know that is not usual to call infinite times the command ls/dir, but it's just a test)
¿Why do we have a memory leak in Linux but not in Windows?
I will appreciate any help or ideas. Thanks in advance.

Answer 1

Hi Joby,
From our last profiling results, we haven't found yet a proper solution. We think that probably the problem is caused by the byte[]'s/FileInputStreams created by the class UNIXProcess that manage the stdin, stdout and stderr streams. It seems that these byte arrays cannot be removed correctly by the garbage collector and they become bigger and bigger, so at the end they took all the memory of the system.
We downloaded the last version of OpenJDK 6 (build b19) and modified UNIXProcess.java.linux so when we call its method destroy(), we assign to null those streams. We did that because we wanted to indicate to the garbage collector that these objects could be removed, as we saw that the close() methods doesn't do anything on their implementation.
public void destroy() {
     // There is a risk that pid will be recycled, causing us to
     // kill the wrong process!  So we only terminate processes
     // that appear to still be running.  Even with this check,
     // there is an unavoidable race condition here, but the window
     // is very small, and OSes try hard to not recycle pids too
     // soon, so this is quite safe.
     synchronized (this) {
         if (!hasExited)
          destroyProcess(pid);
        try {
            stdin_stream.close();
            stdout_stream.close();
            stderr_stream.close();
            // LINES WE ADDED
            stdin_stream = null;
            stdout_stream = null;
            stderr_stream = null;
        } catch (IOException e) {
            // ignore
            e.printStackTrace();
    }But this didn't work at all. We saw that we were able to execute for a long time our application and that the free memory of the system wasn't decreasing as before, but we did some profiling with this custom JVM and the test application and we still see more or less the same behaviour: lots of surviving generations, at some point increase of the used heap to the maximum allowed, and finally the crash of the test app.
So sadly, we still don't have a solution for that problem. You could try to compile OpenJDK 6, modify it, and try it with your program to see if the last version works for you. Compiling OpenJDK 6 in Linux is quite easy: you just have to download the source and the binaries from here and configure your environment with something like this:
export ANT_HOME=/opt/apache-ant-1.7.1/
export ALT_BOOTDIR=/usr/lib/jvm/java-6-sun
export ALT_OUTPUTDIR=/tmp/openjdk
export ALT_BINARY_PLUGS_PATH=/opt/openjdk-binary-plugs/
export ALT_JDK_IMPORT_PATH=/usr/lib/jvm/java-6-sun
export LD_LIBRARY_PATH=
export CLASSPATH=
export JAVA_HOME=
export LANG=C
export CC=/usr/bin/gcc-4.3
export CXX=/usr/bin/g++-4.3Hope it helps Joby :)
Cheers.