Too many trace files apparently created on behalf of Health Monitor

Every ~10 minutes I see two pairs of trc+trm files that appear to be nothing more than a nuisance. A nuisance because if you don't get rid of them, then commands such as "ls -lt | head" take too long to complete. Even after setting disablehealth_check=true, the files still keep coming. The only thing I can find is a correspondence between the rows in v$hm_run and these trace files.
The RDBMS appears to be working fine and there's nothing in the alert log that would explain their existence. Having said that, I'll go ahead and admit that I don't these are false negatives but it sure looks like they are.
Below is a sample of the two trace files in the 4 files that get created every 10 minutes. Thanks.
Here is V11106_m001_8138.trc (including the preamble)
Trace file /opt/oracle/diag/rdbms/v11/V11106/trace/V11106_m001_8138.trc
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /opt/oracle/product/11.1.0.6
System name: Linux
Node name: rhel01.dev.method-r.com
Release: 2.6.18-92.1.13.el5xen
Version: #1 SMP Thu Sep 4 04:20:55 EDT 2008
Machine: i686
Instance name: V11106
Redo thread mounted by this instance: 1
Oracle process number: 26
Unix process pid: 8138, image: [email protected] (m001)
*** 2009-04-03 00:31:36.650
*** SESSION ID:(139.36) 2009-04-03 00:31:36.650
*** CLIENT ID:() 2009-04-03 00:31:36.650
*** SERVICE NAME:(SYS$BACKGROUND) 2009-04-03 00:31:36.650
*** MODULE NAME:(MMON_SLAVE) 2009-04-03 00:31:36.650
*** ACTION NAME:(DDE async action) 2009-04-03 00:31:36.650
========= Dump for error ORA 1110 (no incident) ========
----- DDE Action: 'DB_STRUCTURE_INTEGRITY_CHECK' (Async) -----
Here is V11106_m000_8136.trc (minus the preamble)
*** 2009-04-03 00:31:36.471
*** SESSION ID:(139.34) 2009-04-03 00:31:36.471
*** CLIENT ID:() 2009-04-03 00:31:36.471
*** SERVICE NAME:(SYS$BACKGROUND) 2009-04-03 00:31:36.471
*** MODULE NAME:(MMON_SLAVE) 2009-04-03 00:31:36.471
*** ACTION NAME:(Monitor Tablespace Thresholds) 2009-04-03 00:31:36.471
DDE rules only execution for: ORA 1110
----- START Event Driven Actions Dump ----
---- END Event Driven Actions Dump ----
----- START DDE Actions Dump -----
----- DDE Action: 'DB_STRUCTURE_INTEGRITY_CHECK' (Async) -----
Successfully dispatched
----- (Action duration in csec: 1) -----
----- END DDE Actions Dump -----

Oracle Database 11g by Sam R. Alapati, Charles Kim seems to be a better resource. Here's a link.
http://books.google.com/books?id=14ZH0eZV6G8C&pg=PA60&lpg=PA60&dq=oracle+disable+adr&source=bl&ots=brbhVP05RD&sig=WpaASLcGzJHgBB8Q-RqHu0Efy3k&hl=en&ei=AybaSemkNJSwywWSptXuDg&sa=X&oi=book_result&ct=result&resnum=7#PPA81,M1
When I list hm rows (from x$dbkrun I presume), it shows this detail (which repeats many, many times)
HM RUN RECORD 1
RUN_ID 184481
RUN_NAME HM_RUN_184481
CHECK_NAME DB Structure Integrity Check
NAME_ID 2
MODE 2
START_TIME 2009-04-05 22:44:43.385054 -05:00
RESUME_TIME <NULL>
END_TIME 2009-04-05 22:44:43.718198 -05:00
MODIFIED_TIME 2009-04-05 22:44:43.718198 -05:00
TIMEOUT 0
FLAGS 0
STATUS 5
SRC_INCIDENT_ID 0
NUM_INCIDENTS 0
ERR_NUMBER 0
REPORT_FILE <NULL>
The corresponding data from v$hm_run (and the view definition itself) show that status = 5 means 'COMPLETED'. The interesting thing is that RUN_MODE.RUN_MODE is 'REACTIVE'. This seems to say that it's not a proactive thing. But if something is reacting, then why is it showing no error (i.e., ERR_NUMBER = 0)?

Similar Messages

  • Too many trace files generated by program ORACLE.EXE (J001) in "bdump"

    Hi,
    Please help!
    the following trace file messages have been created in my "bdump" folder about every 6 minutes, so it generates about 200 files per hours. Can someone tell me how to solve the error or stop the trace files be generated. Thanks.
    Windows thread id: 5520, image: ORACLE.EXE (J001)
    *** 2009-05-22 12:49:21.372
    *** ACTION NAME:() 2009-05-22 12:49:21.372
    *** MODULE NAME:() 2009-05-22 12:49:21.372
    *** SERVICE NAME:(SYS$USERS) 2009-05-22 12:49:21.372
    *** SESSION ID:(312.292) 2009-05-22 12:49:21.372
    java.io.IOException: service early exit: code=1 : err=The system cannot find the path specified.
    : out=
    at oracle.wh.runtime.server.Util.execRuntimeService(Util.java:122)
    CJ

    No version number and not enough information in what you posted to help you.
    Did this just start?
    If so what actions preceded it?
    Or is this a new install?
    Have you tried bouncing the instance?
    If it were my system I would have already searched the knowledgebase at metalink and opened an SR if I couldn't find a solution. Did you?

  • Too many trace files 11g

    Thanks for taking my question!
    I just installed 11g and I have a ton of .trc and .trm files. A trc file and trm file is created for just about everything - even Rman backups. I prefer these jobs to show in the alert log not independent trc files.
    Has anyone experienced this issue with 11g? Is there a way to stop the trace files from taking over:)
    Thanks, Kahtie

    There's no way to stop,but to maintain ADR (Automatic Diagnostic Repository) :
    http://download.oracle.com/docs/cd/B28359_01/server.111/b28310/diag.htm#adminChapterDiagnosability
    Werner

  • Too many class files are creating.....

    Hi,
    I am using SwingUtilities.invokeLater() method to update my JTextPane.it is working fine but it is creating number of class files depends upon how many times i am calling SwingUtilities.invokeLater() method....In my application i am calling SwingUtilities.invokeLater() method 5 times so it is creating 5 class files.
    So pl let me know the solution to avoid the same.............thnx

    In my application i am calling
    g SwingUtilities.invokeLater() method 5 times so it
    is creating 5 class files.
    So pl let me know the solution to avoid the
    same............So why is this a problem?

  • "java.io.IOException: Too many open files"  in LinuX

    Hi Developers,
    * I am continiously running and processing more than 2000 XML files by using SAX and DOM.....
    * My process is as follows,
    - Converting the XML file as Document object by DOM....
    - And that DOM will be used while creating log file report, that log file will be created after executing all XML files..
    * After processing approx 1000 files, it throws *"java.io.IOException: Too many open files" in LinuX system* ....
    * I have googled more and more in all sites including sun forum also, but they are telling only to increase the system config by ULIMIT in linux....If i increase that its executing well without exception........
    * My question is, Is it possible to do it by JAVA code itself or any other VM arguments like -Xms512m and -Xmx512m.....
    * Please let me know , if you have any idea.....
    Thanks And Regards,
    JavaImran

    Doh! I forgot to post my little code sample...
    package forums.crap;
    import java.io.*;
    import java.util.*;
    public class TooManyFileHandles
      private static final int HOW_MANY = 8*1024;
      public static void main(String[] args) {
        List<PrintWriter> writers = new ArrayList<PrintWriter>(HOW_MANY);
        try {
          try {
            for (int i=1; i<=HOW_MANY; i++ ) {
              writers.add(new PrintWriter("file"+i+".txt"));
          } finally {
            for (PrintWriter w : writers) {
              if(w!=null)w.close();
        } catch (Exception e) {
          e.printStackTrace();
    }... and the problem still isn't OOME ;-)
    Cheers. Keith.

  • Intermittent too many open files error and Invalid TLV error

    Post Author: jam2008
    CA Forum: General
    I'm writing this up in the hopes of saving someone else a couple of days of hair-pulling...
    Environment: Crystal Reports XI Enterprise / also runtime via Accpac ERP 5.4
    Invalid TLV error in Accpac
    "too many open files" error in event.log file
    Situation:
    Invalid TLV error occurs seemingly randomly on report created in CR Professional 11.  Several days of troubleshooting finally lead to the following diagnosis:
    This error occurs in a report that contains MORE THAN 1 bitmap image.
    The error only shows up after 20 or more reports have been generated sequentially, WITHOUT CLOSING the application that is calling the report.  In our case the Invoice Report dialog within Accpac.  This same error occurred in a custom 3rd party VB.NET app that also called the report through an Accpac API.

    after getting this message you need to do 2 things:
    1. delete the current workspace because it contains some bad data in one the config files - failure to delete the workspace will result the error message to appear even if trying to upload a single file.
    2. add to DTR files in groups - no more than 500 in a single add.

  • RE: (forte-users) [UNIX] "Too many open files" 3.0.M2question

    Actually, the stuff works in interpreted mode.
    It's only when having the server partition compiled that this happen.
    j-p
    -----Message d'origine-----
    De: Adamek, Zenon [mailto:ZAdamekpurolator.com]
    Date: lundi 25 septembre 2000 17:13
    &Agrave;: 'Jean-Paul.Gabriellisema.fr'
    Cc: Forte-userslists.xpedior.com
    Objet: RE: (forte-users) [UNIX] "Too many open files" 3.0.M2 question
    see Technote 10981
    -----Original Message-----
    From: Jean-Paul Gabrielli [SMTP:Jean-Paul.Gabriellisema.fr]
    Sent: Monday, September 25, 2000 11:02 AM
    To: zeForte-users
    Subject: (forte-users) [UNIX] "Too many open files" 3.0.M2 question
    Hi,
    running a server partition that reads a configuration file,
    and apparently doen't close it after, I have that exception:
    SYSTEM ERROR: System Error: Too many open files, opening '....'with mode
    'r'
    Class: qqos_FileResourceException
    1) Is there such a limit, or does this rely only on the OS one ?
    2) How is this error not trapped, as I only got itinteractively, whereas
    my server log does a exception trap/segmentation fault,
    thanlks
    j-p
    For the archives, go to: http://lists.xpedior.com/forte-users and use
    the login: forte and the password: archive. To unsubscribe,send in a new
    email the word: 'Unsubscribe' to: forte-users-requestlists.xpedior.com

    Hi Jean-Paul,
    As described in the Technote 10981 some Forte programs (Nodemanager and
    router) handle correct the high-file descriptor-use problem. It is possible
    that Forte interpreter do it correct too.
    Zenon
    -----Original Message-----
    From: Jean-Paul Gabrielli [SMTP:Jean-Paul.Gabriellisema.fr]
    Sent: Monday, September 25, 2000 12:11 PM
    To: Adamek, Zenon
    Cc: Forte-userslists.xpedior.com
    Subject: RE: (forte-users) [UNIX] "Too many open files" 3.0.M2
    question
    Actually, the stuff works in interpreted mode.
    It's only when having the server partition compiled that this happen.
    j-p
    -----Message d'origine-----
    De: Adamek, Zenon [mailto:ZAdamekpurolator.com]
    Date: lundi 25 septembre 2000 17:13
    &Agrave;: 'Jean-Paul.Gabriellisema.fr'
    Cc: Forte-userslists.xpedior.com
    Objet: RE: (forte-users) [UNIX] "Too many open files" 3.0.M2 question
    see Technote 10981
    -----Original Message-----
    From: Jean-Paul Gabrielli [SMTP:Jean-Paul.Gabriellisema.fr]
    Sent: Monday, September 25, 2000 11:02 AM
    To: zeForte-users
    Subject: (forte-users) [UNIX] "Too many open files" 3.0.M2 question
    Hi,
    running a server partition that reads a configuration file,
    and apparently doen't close it after, I have that exception:
    SYSTEM ERROR: System Error: Too many open files, opening '....'with mode
    'r'
    Class: qqos_FileResourceException
    1) Is there such a limit, or does this rely only on the OS one ?
    2) How is this error not trapped, as I only got itinteractively, whereas
    my server log does a exception trap/segmentation fault,
    thanlks
    j-p
    For the archives, go to: http://lists.xpedior.com/forte-users and use
    the login: forte and the password: archive. To unsubscribe,send in a new
    email the word: 'Unsubscribe' to:
    forte-users-requestlists.xpedior.com
    >
    For the archives, go to: http://lists.xpedior.com/forte-users and use
    the login: forte and the password: archive. To unsubscribe, send in a new
    email the word: 'Unsubscribe' to: forte-users-requestlists.xpedior.com

  • WLS 92MP1: Application Poller issue Too many open files

    Hi,
    We have a wls92mp1 domain on linux AS4(64bit) with Sun jdk 1.5.0_14. It contains only Admin server where we have deployed the application. Over a period of time the server start showing up below message in the logs. We have not deployed the application from autodeploy directory. And the file "/home/userid/wls92/etg/servers/userid_a/cache/.app_poller_lastrun " is available in the location, still it throws FileNotFoundException.
    <Error> <Application Poller> <BEA-149411> <I/O exception encountered java.io.FileNotFoundException: /home/userid/wls92/etg/servers/userid_a/cache/.a
    pp_poller_lastrun (Too many open files).
    java.io.FileNotFoundException: /home/userid/wls92/etg/servers/userid_a/cache/.app_poller_lastrun (Too many open files)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
    at java.io.FileWriter.<init>(FileWriter.java:73)
    at weblogic.management.deploy.GenericAppPoller.setLastRunFileMap(GenericAppPoller.java:423)
    Any help regarding this would be highly appreciated.
    Thanks.

    Hi,
    By above seeing in error, this error code (BEA-149411) describe the following
    149411: I/O exception encountered {0}.
    L10n Package: weblogic.management.deploy.internal
    I18n Package: weblogic.management.deploy.internal
    Subsystem: Application Poller
    Severity: Error
    Stack Trace: true
    Message Detail: An I/O exception denotes a failure to perform a read/write operation on the application files.
    Cause: An I/O exception can occur during file read/write while deploying an application.
    Action: Take corrective action based on the exception message details.
    i think it helps u.
    -abhi

  • Getting java.io.IOException: Too many open files+ClassNotFoundException

    Dear All
    We have a web application deployed on Rational Application Developer 6.0 (Operating System is Windows 2000 professional) our users are randomly getting java.io.IOException: Too many open files+ClassNotFoundException when they click on some servlet or jsp link, but they are getting this error randomly for example when they click some link they may get these exceptions but refreshing page or clicking once again on same link executes servlet successfully.If anyone could help on this topic we will be grateful
    Thanks

    I think this these two exceptions are occuring in differrent environment
    java.io.IOException is occuring under heavy load to web server its stack trace is as follows:
    JSPG0225E: An error occurred at line: 2 in the statically included file: /SessionCheck.jsp
    JSPG0093E: Generated servlet error from file: /Admin/AdminInsuranceCertificates.jsp
    E:\WebSphere_6\AppServer\profiles\AUSECert\temp\centraNode04\server1\AUSECert\Vero.war\Admin\_AdminInsuranceCertificates.java:259: cannot access com.bplus.natmar.LoginDetails
    bad class file: E:\WebSphere_6\AppServer\java\jre\lib\core.jar(java/io/Writer.class)
    unable to access file: E:\WebSphere_6\AppServer\profiles\AUSECert\installedApps\centraNode04Cell\AUSECert.ear\Vero.war\WEB-INF\classes\com\bplus\natmar\LoginDetails.class (Too many open files)
    Please remove or make sure it appears in the correct subdirectory of the classpath.
    (source unavailable)
    1 error
    ]: com.ibm.ws.jsp.JspCoreException: JSPG0049E: /Admin/AdminInsuranceCertificates.jsp failed to compile :
    this error always occurs in reference to logindetails class this clas is used to make a session check on different roles in our project for e.g., user having end user role should not be able to log in as a user having admin role
    we have included a sessioncheck.jsp in our every jsp page in this jsp we have simply used logindetails class as useBean and called its getresource() method
    above stacktrace is from our live application server
    while testing same project on our local system we are not getting too many open files exception but we are getting following ClassNotFoundException
    [11/30/05 17:11:42:797 EST] 0000004a SystemOut O SELECT count(*) as NoofRecs FROM resourcerolebindings WHERE ResourceName = 'mainEdit.jsp' and IsEndUser=1
    [11/30/05 17:12:50:891 EST] 000001eb SystemOut O SELECT count(*) as NoofRecs FROM resourcerolebindings WHERE ResourceName = 'InsuranceCertificates.jsp' and IsEndUser=1
    [11/30/05 17:17:40:828 EST] 0000008d SystemOut O AppURL is: http://www.VeroECert.com/Vero/indexU.jsp
    [11/30/05 17:17:58:141 EST] 0000008b SystemOut O SELECT count(*) as NoofRecs FROM resourcerolebindings WHERE ResourceName = 'InsuranceCertificates.jsp' and IsEndUser=1
    [11/30/05 17:20:41:703 EST] 00000034 ServletWrappe E SRVE0026E: [Servlet Error]-[com.servlet.UserHelpServlet]: java.lang.ClassNotFoundException: com.servlet.UserHelpServlet
         at com.ibm.ws.classloader.CompoundClassLoader.findClass(CompoundClassLoader.java(Compiled Code))
         at com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java(Compiled Code))
         at java.lang.ClassLoader.loadClass(ClassLoader.java(Compiled Code))
         at java.beans.Beans.instantiate(Beans.java:202)
         at java.beans.Beans.instantiate(Beans.java:63)
         at com.ibm.ws.webcontainer.servlet.ServletWrapper$3.run(ServletWrapper.java:1384)
         at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java(Compiled Code))
         at com.ibm.ws.webcontainer.servlet.ServletWrapper.loadServlet(ServletWrapper.java(Compiled Code))
         at com.ibm.ws.webcontainer.servlet.ServletWrapper.initialize(ServletWrapper.java:1312)
         at com.ibm.wsspi.webcontainer.extension.WebExtensionProcessor.createServletWrapper(WebExtensionProcessor.java:84)
         at com.ibm.ws.webcontainer.extension.InvokerExtensionProcessor.handleRequest(InvokerExtensionProcessor.java:238)
         at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:2841)
         at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:220)
         at com.ibm.ws.webcontainer.VirtualHost.handleRequest(VirtualHost.java:204)
         at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java(Compiled Code))
         at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java(Compiled Code))
         at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java(Compiled Code))
         at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java(Compiled Code))
         at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java(Compiled Code))
         at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java(Compiled Code))
         at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java(Compiled Code))
         at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java(Compiled Code))
         at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java(Compiled Code))
         at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code))
    this error is occuring on concurrent clicks suppose three users click on same button then it is possible that one user gets correct output while other users experience this exception
    Also one more point we are not using web.xml for calling servlets we are directly calling it giving its full path.
    Thanks

  • "Too many open files" - Mac OS X 10.7.5

    I'm getting errors about "Too many open files" when trying to use Berkeley DB on Mac OS X 10.7.5.
    The BerkeleyDB site mentions this, but it goes back to 2003 and refers to a file which doesn’t exist in OS X 10.7.5:
    http://docs.oracle.com/cd/E17076_03/html/installation/build_unix_macosx.html
    I tried ulimit, but it doesn't seem to have any affect:
    $ sudo ulimit -n 1024 1024
    Password:
    $ ulimit -a
    core file size          (blocks, -c) 0
    data seg size           (kbytes, -d) unlimited
    file size               (blocks, -f) unlimited
    max locked memory       (kbytes, -l) unlimited
    max memory size         (kbytes, -m) unlimited
    open files                      (-n) 256
    pipe size            (512 bytes, -p) 1
    stack size              (kbytes, -s) 8192
    cpu time               (seconds, -t) unlimited
    max user processes              (-u) 709
    virtual memory          (kbytes, -v) unlimited
    Any suggestions?
    Thanks,
    James.

    sudo sysctl -w kern.maxfiles=32768
    sudo sysctl -w kern.maxprocperuid=16384
    seemed to work.
    Created file /etc/sysctl.conf containing:
    kern.maxfiles=32768
    kern.maxprocperuid=16384
    so that the setting would survive a reboot.
    Hopefully that permanently fixes it.

  • Berkeley DB: Too Many Open Files

    I've installed BerkelyDB 6.0.3 on Mavericks but when I try to use it I get errors about there being too many open files:
    Apr-10-14 11:20:45 [Worker_10001] BerkeleyDB-CRT-ENV-ERROR (1) in HASH Delay , on file /Applications/assp/delaydb.bdb : BDB:Too many open files
    Apr-10-14 11:20:45 [Worker_10001] BerkeleyDB-ENV-ERROR Delay:  - BDB:Too many open files
    Apr-10-14 11:20:45 [Worker_10001] Delaydb database error: BerkeleyDB-ENV-ERROR Delay:  - BDB:Too many open files
    Does anyone know how to (permanently) increase these limits in Mavericks?
    Thanks,
    James.

    sudo sysctl -w kern.maxfiles=32768
    sudo sysctl -w kern.maxprocperuid=16384
    seemed to work.
    Created file /etc/sysctl.conf containing:
    kern.maxfiles=32768
    kern.maxprocperuid=16384
    so that the setting would survive a reboot.
    Hopefully that permanently fixes it.

  • What have "Too many open Files" to do with FIFOs?

    Hi folks.
    I've just finished a middleware service for my company, that receives files via a TCP/IP connection and stores them into some cache-directory. An external program gets called, consumes the files from the cache directory and puts a result-file there, which itself gets sent back to the client over TCP/IP.
    After that's done, the cache file (and everything leftover) gets deleted.
    The middleware-server is multithreaded and creates a new thread for each request connection.
    These threads are supposed to die when the request is done.
    All works fine, cache files get deleted, threads die when they should, the files get consumed by the external program as expected and so on.
    BUT (there's always a butt;) to migrate from an older solution, the old data gets fed into the new system, creating about 5 to 8 requests a second.
    After a time of about 20-30 minutes, the service drops out with "IOException: Too many open files" on the very line where the external program gets called.
    I sweeped through my code, seeking to close even the most unlikely stream, that gets opened (even the outputstreams of the external process ;) but the problem stays.
    Things I thought about:
    - It's the external program: unlikely since the lsof-command (shows the "list of open files" on Linux) says that the open files belong to java processes. Having a closer look at the list, I see a large amount of "FIFO" entries that gets bigger, plus an (almost) constant amount of "normal" open file handles.
    So perhaps the handles get opened (and not closed) somehwere else and the external program is just the drop that makes the cask flood over.
    - Must be a file handle that's not closed: I find only the "FIFO" entries to grow. Yet I don't really know what that means. I just think it's something different than a "normal" file handle, but maybe I'm wrong.
    - Must be a socket connection that's not closed: at least the client that sends requests to the middleware service closes the connection properly, and I am, well, quite sure that my code does it as well, but who knows? How can I be sure?
    That was a long description, most of which will be skipped by you. To boil it down to some questions:
    1.) What do the "FIFO" entries of the lsof-command under Linux really mean ?
    2.) How can I make damn sure that every socket, stream, filehandle etc. pp. is closed when the worker thread dies?
    Answers will be thanked a lot.
    Tom

    Thanks for the quick replies.
    @BIJ001:
    ls -l /proc/<PID>/fdGives the same information as lsof does, namely a slowly but steadily growing amount of pipes
    fuserDoesn't output anything at all
    Do you make exec calls? Are you really sure stdout and stderr are consumed/closed?Well, the external program is called by
    Process p = Runtime.getRuntime().exec(commandLine);and the stdout and stderr are consumed by two classes that subclass Thread (named showOutput) that do nothing but prepending the corresponding outputs with "OUT:" and "ERR" and putting them into a log.
    Are they closed? I hope so: I call the showOutput's halt method, that should eventually close the handles.
    @sjasja:
    Sounds like a pipe.Thought so, too ;)
    Do you have the waitFor() in there?Mentioning the waitFor():
    my code looks more like:
    try  {
         p = Runtime.getRuntime.exec(...);
         outshow = new showOutput(p.getInputStream(), "OUT").start;
         errshow = new showOutput(p.getErrorStream(), "ERR").start;
         p.waitFor();
    } catch (InterruptedException e) {
         //can't wait for process?
         //better go to sleep some.
         log.info("Can't wait for process! Going to sleep 10sec.");
         try{ Thread.sleep(10000); } catch (InterruptedException ignoreMe) {}
    } finally {
         if (outShow!=null) outShow.halt();
         if (errShow!=null) errShow.halt();
    /**within the class showOutput:*/
    /**This method gets called by showOutput's halt:*/
    public void notifyOfHalt() {
         log.debug("Registered a notification to halt");
         try {
              myReader.close(); //is initialized to read from the given InputStream
         } catch (IOException ignoreMe) {}
    }Seems as if the both of you are quite sure that the pipes are actually created by the exec command and not closed afterwards.
    Would you deem it unlikely that most of the handles are opened somewhere else and the exec command is just the final one that crashes the prog?
    That's what I thought.
    Thanks for your time
    Tom

  • Java.io.IOException: Too many open files while deploying in soa 11g

    hi all,
    I am getting a strange error while deploying any composite .. it's a hello world kinda composite but while i am trying to deploy it i am getting "java.io.IOException: Too many open files" while deployment.. i have tried to deploy it in 2-3 ways but all of them resulted in the same error..bouncing the soa server might be an option but can someone give an insight as why it is happening and can it be resolved without restarting the server..
    Thanks

    yes..so this problem is with unix only ..coz i previously worked in Windows ..never got this problem..

  • STARTING DATABASE : PROBLEM OF Linux Error: 23: Too many open files in syst

    Hi everybody,
    I am running an RMAN script and get this error,
    9> @/u01/app/oracle/admin/devpose/backup/configuration.rcv
    RMAN> ###################################################################
    2> # Configuration file used to set Rman policies.
    3> #
    4> ###################################################################
    5>
    6> CONFIGURE DEFAULT DEVICE TYPE TO DISK;
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of configure command at 08/26/2009 20:03:30
    RMAN-06403: could not obtain a fully authorized session
    ORA-01034: ORACLE not available
    RMAN> CONFIGURE RETENTION POLICY TO REDUNDANCY 1;
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of configure command at 08/26/2009 20:03:30
    RMAN-06403: could not obtain a fully authorized session
    ORA-01034: ORACLE not available
    RMAN> #CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
    2> CONFIGURE DEVICE TYPE DISK PARALLELISM 2;
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of configure command at 08/26/2009 20:03:30
    RMAN-06403: could not obtain a fully authorized session
    ORA-01034: ORACLE not available
    RMAN>
    RMAN> CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '/u01/app/oracle/backup/db/ora_df%t_s%s_s%p';
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of configure command at 08/26/2009 20:03:30
    RMAN-06403: could not obtain a fully authorized session
    ORA-01034: ORACLE not available
    But this problem is understandable, as the database is not running. The main problem why database is not running, I have found the reason but do not understand how to solve the problem.
    Since, the database was not running, I tried to startup the database, I then came across the following which is my problem (Why so many files are open? Linux OS error says too many files open. See below,
    SQL> conn /as sysdba
    Connected to an idle instance.
    SQL> startup
    ORACLE instance started.
    Total System Global Area 419430400 bytes
    Fixed Size 779516 bytes
    Variable Size 258743044 bytes
    Database Buffers 159383552 bytes
    Redo Buffers 524288 bytes
    Database mounted.
    ORA-00313: open failed for members of log group 2 of thread 1
    ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/devpose/redo02.log'
    ORA-27041: unable to open file
    Linux Error: 23: Too many open files in system
    Can anybody has run into such problem and guide me to a solution, please?
    Thanks

    Hi,
    yes, this DB was functioning o.k. this configuration script was part of RMAN daily backup.
    Last night the backup failed. So, when I opened "Failed job" in the EM, I saw this type of messages.
    That was the starting point. Gradually, I tried to narrow down on to the actual problem and found the findings as I have posted.
    One way of sovling problem, I thought that, all these processes I would kill and then try to open the database, it might startup. However, that wouldnot lead me in ensuring this won't occur again.
    That's why I am trying to understand why it should open, so many processes (why spawn so many .flb files?) Any thoughts you have around this?
    I will try to restart the OS as the last resort.
    Thanks for your help and suggestions.
    Regards,

  • WLS 10.3.5 on RHEL 5.4, SocketException: Too many open files

    Hi
    I'm running Weblogic server 10.3.5 on Red Hat Enterprise Linux Server release 5.4 (Tikanga), with Java jdk1.6.0_27.
    My order handling application, when receiving client orders, needs to make outbound SOAP calls to fulfill the order. During a performance test, we got following errors:
    ####<Feb 10, 2012 2:28:41 PM ICT> <Critical> <Server> <KKMOMAPP2> <KKMOMPE2> <DynamicListenThread[Default]> <<WLS Kernel>> <> <> <1328858921806> <BEA-002616> <Failed to listen on channel "Default" on 172.24.106.81:4095, failure count: 1, failing for 0 seconds, java.net.SocketException: Too many open files>
    I monitored the java process of this application, when the "Too many open files" error happened, it had 1388 open file descriptors, among which 655 were sockets.
    I also monitored the total open file descriptors of the weblogic user account, the count was around 6300 during this error.
    These numbers are far smaller than the file limits configured on OS:
    - Under weblogic account, ulimit -n shows 65536
    - /proc/sys/fs/file-max shows 772591
    - Following lines are already in /etc/security/limits.conf
    weblogic soft nofile 65536
    weblogic hard nofile 65536
    weblogic soft nproc 16384
    weblogic hard nproc 16384
    I did another test using a simple java program to open large number of sockets under weblogic account. It has no problem to open 15,000 sockets. It seems the file descriptor limit is indeed quite high, but for some reasons, the Weblogic process fails even when it has merely 1388 open files. Are there other Linux or Weblogic parameters I should tune? Or anything else I missed?
    Thank you very much
    Ning

    Hi All,
    Any help on this issue ?
    Thank you,
    Ram

Maybe you are looking for

  • Macbook pro - was doing my work and black screen appeared, shut down, and won't turn back on

    i was doing my work, plugged a usb in the port, and then my screen went black and turned itself off. i used my power plug on a different mac and it still worked, but when i plugged it into my macbook pro the green like flashes once and then dissappea

  • SAP- AR Report by customer, account number and bucket

    Hello SAP Guru's, I am looking for a report that has Customer name, Account number and balance by bucket(30 day, 60, 90, 120).  I have seen this several times but cannot find it anywhere.  Can anyone tell me/ help me with this? Will award points. THA

  • ITunes runs but won't play music anymore

    Since upgrading to 6.0.2 audio files no longer play at all (I don't hear any audio and the diamond in the progress bar does not move) in iTunes for Windows. I'll hit the PLAY button and it changes color, acknowledging I clicked on it, but it won't pl

  • Oracle Parallel Server

    This is a question about availability of Oracle Parallel Server on linux. I tried to ask OTN but their answer was really strange : they adviced me to go to discussion board! I suppose that Parallel was not realized for linux oficially yet but this is

  • Mass unset an adjustment (white balance), aperture 3

    Hi all, I upgraded to Aperture 3 and everything went smoothly, even the library upgrade. Except for a small adjustment thing: After "reprocess masters" every image I had white balance set manually (most of my tens of thousands of images), looks now g