Number of open files
Is there any way I can read kernel data structure that contains the number of open files on the system? I know I can traverse the /proc tree or
use the lsof program, but it creates too much overhead for my application.
This parameter must be somewhere in the kernel - I can't beleive that Solaris reads all /proc tree every time I open a file :-)
...hi Alex. I may not have a complete answer to your question, but I did want to clarify one thing.
It's not like the kernel every has to read the /proc file system tree because the /proc file system IS part of the kernel. It's simply a virtual file system that makes certain parts of the kernel's memory look like a bunch of files.
So, except for the overhead associated with the system calls necessary to "read" through that space with your application, the performance bind is minimal.
That brings us to the open file handles limit. Here's what I see in the "Solaris Internals" book.
Systemwide limit for the number of files a process may open:
rlim_fd_max - systemwide hard limit
rlim_fd_cur - systemwide soft limit (or default, or current if you will)
You can change the per-process limit (maybe bump up from the default to the hard limit) with limit(1) and ulimit(1) commands, or programatically with the function setrlimit(2).
Further, if you are using the <stdio.h> interface to access a file, there is a limit on 32-bit systems of 256 because an 8-bit number is used to represent the file handle. On a 64-bit system the number is closer to 65000. (16-bit number in use)
So that brings us to your real problem, which is how to test for the number of filehandles a process has open at this moment. I don't know that I have a good answer, but I presume that you've already ruled out the pfiles(1) command. This program (/usr/proc/bin/pfiles) gives good output and does a nice job of getting you really good information about open files.
You may not want to execute it from your own process (otherwise you have the execution overhead and you still have to parse the output) but you may find the source code for it to be instructive.
I haven't looked at it, but I would imagine that you could probably trace it to the source of information about how the kernel represents open file information.
If you don't have Solaris source code, perhaps you know someone who has a license. I'll poke around and see if I can find any other useful information that is related to this question.
---v
Similar Messages
-
ORA-22290: operation would exceed the maximum number of opened files or LOB
i am getting this error in a procedure.
ORA-22290: operation would exceed the maximum number of opened files or LOBs
22290, 00000, "operation would exceed the maximum number of opened files or LOBs"
// *Cause: The number of open files or LOBs has reached the maximum limit.
// *Action: Close some of the opened files or LOBs and retry the operation.
NAME TYPE VALUE
session_max_open_files integer 10
Procuedure:
CREATE OR REPLACE PROCEDURE WMSOWN."PROC_WMS_XML_READ"
P_EVENT_KEY IN VARCHAR2,
X_STATUS_MSG OUT VARCHAR2,
X_STATUS OUT NUMBER
)AS
l_parser dbms_xmlparser.Parser;
domdoc xmldom.DOMDocument;
nodelist XMLDOM.DOMNODELIST;
node XMLDOM.DOMNODE;
n_child XMLDOM.DOMNODE;
elements XMLDOM.DOMELEMENT;
name_node_map XMLDOM.DOMNAMEDNODEMAP;
parent_seg varchar2(4000);
tag_name_bkp varchar2(4000); -- LOOK OUT BRAD IS CODING AGAIN
chile_seg VARCHAR2(4000);
p_seg VARCHAR2(4000);
p_seg1 VARCHAR2(4000);
p_seg2 VARCHAR2(30);
p_int_name VARCHAR2(50);
col_value VARCHAR2(100):=NULL;
len1 NUMBER;
cnt NUMBER;
seg_id_bkp NUMBER; -- LOOK OUT BRAD IS CODING AGAIN
sequence_bkp NUMBER; -- LOOK OUT BRAD IS CODING AGAIN
prev_sequence NUMBER; -- LOOK OUT BRAD IS CODING AGAIN
prev_seq_set VARCHAR2(3); --brad coding
parent_id number; ---brad coding
valid_seg NUMBER; -- LOOK OUT BRAD IS CODING AGAIN
data_status VARCHAR2(10);
v_main_seg VARCHAR2(50);
v_seq_no NUMBER;
V_CLOBLOCATOR CLOB;
V_FILELOCATOR BFILE;
v_amount_to_load NUMBER;
dest_offset NUMBER := 1;
src_offset NUMBER := 1;
lang_context NUMBER := DBMS_LOB.DEFAULT_LANG_CTX;
warning NUMBER;
v_event_name USR_OUB_FILE_PROCESS_DETAILS.EVENT_NAME%TYPE;
v_file_name USR_OUB_FILE_PROCESS_DETAILS.FILE_NAME%TYPE;
DIRECTORY_PATH_INVALID EXCEPTION;
PRAGMA EXCEPTION_INIT(DIRECTORY_PATH_INVALID,-22285);
NO_PRIVILEGES EXCEPTION;
PRAGMA EXCEPTION_INIT(NO_PRIVILEGES,-22286);
INVALID_DIRECTORY EXCEPTION;
PRAGMA EXCEPTION_INIT(INVALID_DIRECTORY,-22287);
FILE_NOT_FOUND EXCEPTION;
PRAGMA EXCEPTION_INIT(FILE_NOT_FOUND,-22289);
P_DIRECTORY VARCHAR2(50) :='WMS_XML_DIR_OUB';
v_whid poldat_view.wh_id%type;
BEGIN
--NAME : PROC_WMS_XML_READ.PLS
--DESCRIPTION :
-- Procedure PROC_WMS_XML_READ search XML files from remote location.
-- Open,Parse and Read XML files. Store all XML values into tables.
-- Developed by Dharmesh Patidar(jw782)
-- History: New condition is added i.e. p_seg:=parent_seg to maintain PARENT and CHILD relationship
-- by Vishwanath Dubey(jl246) on 17-June-2011
-- BRAD_XML_DEBUG table removed for CLEANING Activity by DHARMESH PATIDAR(JW782) ON 29-JUNE-2011.
/*BLOCK FOR CAPTURING EVENT NAME BASED ON EVENT ID START*/
BEGIN
SELECT event_name,file_name,WAREHOUSE_ID
INTO v_event_name, v_file_name,v_whid
FROM usr_oub_file_process_details
WHERE event_id=p_event_key
AND process_flag='U';
EXCEPTION
WHEN NO_DATA_FOUND THEN
x_status_msg:=SQLCODE||':'||' Error while selecting event name and event id in Procedure PROC_WMS_XML_READ : Record is not available in USR_OUB_FILE_PROCESS_DETAILS table for event id '|| P_EVENT_KEY;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
WHEN TOO_MANY_ROWS THEN
x_status_msg:=SQLCODE||':'||' Error while selecting event name and event id in Procedure PROC_WMS_XML_READ : More than one Records found in USR_OUB_FILE_PROCESS_DETAILS table for event id '|| P_EVENT_KEY;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
WHEN VALUE_ERROR THEN
x_status_msg:=SQLCODE||':'||' Error while selecting event name and event id in Procedure PROC_WMS_XML_READ : Varibale length is small or data type mismatch while selecting event id and event name in USR_OUB_FILE_PROCESS_DETAILS table for event id '|| P_EVENT_KEY;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
WHEN OTHERS THEN
x_status_msg:=SQLCODE||':'||'Error in Procedure PROC_WMS_XML_READ while selecting event name and event id ';
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
END;
/*BLOCK FOR CAPTURING EVENT NAME BASED ON EVENT ID END*/
/*LOGIC TO READ XML FROM REMOTE LOCATION START*/
DBMS_LOB.CREATETEMPORARY(V_CLOBLOCATOR, TRUE);
V_FILELOCATOR := BFILENAME(P_DIRECTORY,V_FILE_NAME);
DBMS_LOB.OPEN(V_FILELOCATOR,DBMS_LOB.FILE_READONLY);
V_AMOUNT_TO_LOAD := DBMS_LOB.GETLENGTH(V_FILELOCATOR);
DBMS_LOB.LOADCLOBFROMFILE(V_CLOBLOCATOR,
V_FILELOCATOR ,
V_AMOUNT_TO_LOAD,
DEST_OFFSET,
SRC_OFFSET,
0,
LANG_CONTEXT,
WARNING);
dbms_lob.close(V_FILELOCATOR);
/*LOGIC TO READ XML FROM REMOTE LOCATION END*/
/*Temporary Code to help with debug Clear the table before populating it with new data*/
--delete table BRAD_XML_DEBUG;
cnt:=1;
seg_id_bkp:=0;
data_status:='N';
v_seq_no:=0;
prev_seq_set:='NO';
/*create new parser.*/
l_parser := dbms_xmlparser.newParser;
dbms_xmlparser.parseClob(l_parser, replace(V_CLOBLOCATOR,'&','1x2x3x4x5'));
/*Parse the document and create a new DOM document.*/
domdoc :=dbms_xmlparser.getDocument(l_parser);
/* get all elements in the DOM*/
nodelist := XMLDOM.getElementsByTagName(DOMDoc, '*');
len1 := XMLDOM.getLength(nodelist);
/* loop through elements of the DOM */
FOR j in 1..len1-1 LOOP --MAIN LOOP START
BEGIN
/*below sql will fetch Node from table to travel xml data*/
BEGIN
SELECT int_name,tag_name
INTO p_int_name, p_seg1
FROM usr_wms_tag_det
WHERE int_name=v_event_name
AND seq_no =cnt;
EXCEPTION
--PLEASE DO NOT HANDLE ANY EXCEPTION APART MENTIONED BELOW
WHEN OTHERS THEN
NULL;
END;
IF cnt=1 THEN
v_main_seg:=p_seg1;
END IF;
EXCEPTION
--PLEASE DO NOT HANDLE ANY EXCEPTION APART MENTIONED BELOW
WHEN no_data_found THEN
null;
WHEN OTHERS THEN
x_status_msg:=SQLCODE||':'||'Error in Procedure PROC_WMS_XML_READ while selecting interface name and tag name'||sqlerrm;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
END;
/*LOGICS TO READ XML START*/
node:=XMLDOM.item(nodelist, j);
elements:=XMLDOM.makeElement(node);
parent_seg:=(xmldom.getTagName(elements));
tag_name_bkp:=(xmldom.getTagName(elements));
name_node_map:=xmldom.getAttributes(node);
n_child:=xmldom.getFirstChild(node);
col_value:=xmldom.getNodeValue(n_child);
/*get the sequence number from the interface hierarchy table */
SELECT count(1)
INTO valid_seg
FROM usr_wms_tag_det
WHERE int_name=v_event_name
AND tag_name = tag_name_bkp;
if valid_seg>0 then
begin
SELECT seq_no
INTO sequence_bkp
FROM usr_wms_tag_det
WHERE int_name=v_event_name
AND tag_name = tag_name_bkp;
seg_id_bkp:=seg_id_bkp+1;
p_seg:=parent_seg;--Modified by Vishwanath Dubey dated 16-jun-2011
end;
end if;
if prev_seq_set = 'NO' then
begin
prev_sequence := sequence_bkp;
prev_seq_set := 'YES';
end;
end if;
if sequence_bkp < prev_sequence then --you just moved up level(s) in the message structure
begin
select max(seg_id)
into parent_id
from usr_wms_global_xml_det
where seg_sequence = sequence_bkp-1;
prev_sequence := sequence_bkp;
end;
end if;
if sequence_bkp > prev_sequence then --you just moved down a level in the message structure
parent_id := seg_id_bkp-1;
prev_sequence := sequence_bkp;
end if;
/*end getting the hierarchy table sequence */
/*LOGICS TO READ XML END */
IF (parent_seg =p_seg1) or (parent_seg=p_seg2) THEN
if parent_seg=v_main_seg then
v_seq_no:=v_seq_no+1;
end if;
BEGIN
/* INSERTING DATA LOGICS TO READ XML END */
INSERT INTO usr_wms_global_xml_det values(p_int_name,tag_name_bkp,parent_seg,seg_id_bkp,sequence_bkp,parent_id,'','','',J,v_seq_no,data_status,cnt);
EXCEPTION
WHEN OTHERS THEN
x_status_msg:=SQLCODE||' : Error in Procedure PROC_WMS_XML_READ while inserting records in USR_WMS_GLOBAL_XML_DET table for interface name and parent segment '||P_INT_NAME||','||PARENT_SEG;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
END ;
p_seg:=parent_seg;
p_seg2:=P_SEG1;
cnt:=cnt+1;
ELSE
chile_seg:=parent_seg;
BEGIN
/* INSERTING DATA LOGICS TO READ XML END */
INSERT INTO usr_wms_global_xml_det values(p_int_name,tag_name_bkp,p_seg,seg_id_bkp,sequence_bkp,parent_id,'',chile_seg,replace(TRIM(Col_Value),'1x2x3x4x5','&'),J,v_seq_no,data_status,cnt);
EXCEPTION
WHEN OTHERS THEN
x_status_msg:=SQLCODE||' : Error in Procedure PROC_WMS_XML_READ while inserting records in USR_WMS_GLOBAL_XML_DET table for interface name and parent segment '||P_INT_NAME||','||PARENT_SEG;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
END;
END IF;
END LOOP; --MAIN LOOP END
dbms_xmldom.freeDocument(DOMDoc);
x_status:=0;
EXCEPTION
WHEN DIRECTORY_PATH_INVALID THEN
x_status_msg:=SQLCODE||' : Error in Procedure PROC_WMS_XML_READ DIRECTORY PATH IS INVALID';
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
WHEN FILE_NOT_FOUND THEN
x_status_msg:=SQLCODE||' : Error in Procedure PROC_WMS_XML_READ INVALID XML FILE NAME OR FILE DOES NOT EXISTS';
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
WHEN NO_PRIVILEGES THEN
x_status_msg:=SQLCODE||' : Error in Procedure PROC_WMS_XML_READ Insufficient privileges on file or directory NAME- '||p_directory||' to perform FILEOPEN operation.';
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
RETURN;
WHEN OTHERS THEN
x_status_msg:=SQLCODE||' : Error in Procedure PROC_WMS_XML_READ '|| SQLERRM;
x_status:=SQLCODE;
proc_wms_error_trace(v_whid, --warehouse id
null , --event id
v_event_name , --event name
x_status, --error code
x_status_msg ); --error message
dbms_xmlparser.freeParser(l_parser);
dbms_xmldom.freeDocument(DOMDoc);
RETURN;
END PROC_WMS_XML_READ;
Edited by: user13427480 on Feb 8, 2013 7:08 PMwhen you post sql statement use also similar threads :
ORA-22290: operation would exceed the maximum number of opened files or LOB
https://kr.forums.oracle.com/forums/thread.jspa?messageID=10842417 -
Maximum number of open files..
I'm looking for some help...probably a consultant to give us a call.
I need to know the following:
For 2.6 and 7, number of open files per process default and maximum setting.
Procedure to change the default setting to the maximum
Amont of RAM required to handle the max setting.
Risks inherent in setting this parameter to the max.
Any info on test environments where max setting has been utilized (e.g. datase TPC benchmarks, etc..)
Feel free to call 408.861.1103 - happy to pay for the advice.Hi!
The maximum number of file descriptors per process is set by two parameters:
rlim_fd_cur (soft limit, defaults to 64)
rlim_fd_max (hard limit, defaults to 1024)
Processes may raise their soft limit up to the hard limit using setrlimit(2).
Setting rlim_fd_cur high is not a problem as the file desciptors are allocated in chunks of 24 as required, and so not all in one go. They don't actually require that much memory either.
As administrator you may set the limits by adding an entry to /etc/system, eg:
set rlim_fd_max=600
and rebooting.
Note however on 32 bit solaris, the significant limitation is that the stdio library FILE structure limits your process to 256 fds. This is increased to 65536 for 64bit programs on solaris 7.
Select(3c) can use up to 65536 fds (#define FD_SETSIZE 65536 in your code for 32bit solaris 7).
Hope that helps.
Ralph
SUN DTS -
How do I increase the number of opened files in LabWindows?
There's a Win32 function _setmaxstdio, but it doesn't seem to be included in the stdio.h version in LabWindows. I'm using the fopen call and it's limited to 512 opened files. MSDN help says to use the _setmaxstdio function. Is there anyway to do this with the LabWindows compiler?
Thanks!This number is defined in stdio.h as FOPEN_MAX and it is set to 255. Are you using fopen to open your files? _setmaxstdio is not supported for CVI. Why do you need to have more than 512 files open at one time?
I am going to go ahead and say that this is not recommended. This can put a heavy strain on your system and can also suffocate your OS of file descriptors (remember that other applications and your OS itself will be opening files as well). Likely there is another way to architect your application so that you do not need that many files open at one time. Typically you do not leave files open. You open, read/write, then close.
National Instruments
Product Support Engineer -
Number of Open Files in DB Server
Hi everyone,
I need some advise from Unix and Oracle experts on a query that my customer raised.
There are 3 additional application servers added recently. Before the app servers are brought up, I increased the Oracle processes and sessions parameters.
Then, my customer's UNIX administrator asked this question which made me ponder if it's true :
"Changing some of the Oracle parameters can also effect the Unix kernel parameters. For example if there are 256 data files and 4 application servers each with 99 work processes that means that SAP can have at least 101,376 open files on the DB server without the additional Oracle open files and those need for the OS etc. "
I just wonder, does SAP processes really perform read-write on the Oracle datafiles ? Isn't it that the SAP processes logon to Oracle server processes, and it is the Oracle instance that perform the IO on the datafiles ?
The DB server has 343 processes which corresponds to the actual number of total processes in all the app servers. And there are 202 datafiles.
Does that means only 343 processes will open datafiles ? Or will it be 202 files open only ? Or is it like what the Unix administrator said in the example ?
Can someone please enlighten me ?
Thank you,
Kent PehHi again
I give you some example with commands on a real system.
SQL> select count(*) from dba_data_files;
COUNT(*)
262
SQL> select count(*) from dba_temp_files;
COUNT(*)
8
SQL> show parameter processes
NAME TYPE VALUE
processes integer 620
So we have a system with 270 data/tempfiles and a maxium of 620 processes, so the theoretical maximum of open files can be over 270x620. Lets check how many processes we really have at the moment:
orasid> ps -ef | grep ora | wc -l
330
This includes, all shadow and background processes one shell (mine) and even my ps command )
But we do have slightly above 300 oracle processes here. So we might have roughly 270x300 = 81000 files open at the moment. Lets see, i use lsof to list the open files of the oracle command and redirect the output to a file, just because there is really a lot of files open on this system:
orasid> lsof -c oracle > /tmp/lsof.out
This includes directory entries, looks like there is some shared memory stuff, even tcp connections as well (we forgot about these until now). I think all these need file handles on unix as well, so be it. So in our example we have 46557 files open:
orasid> wc -l lsof.out
46557 lsof.out
You can dig further in the lsof.out, for example how many times is one single data file open:
orasid> grep /oracle/SID/sapdata2/stabd_4/stabd.data4 lsof.out | wc -l
209
I hope i did not do too many mistakes here, have fun!
Best regards, Michael
BTW: there is a maximum of 131072 (= 0x20000) open files on this system (HP-UX IA64)
orasid> /usr/sbin/kctune -v nfile
Tunable nfile
Description Maximum number of file descriptors (system-wide)
Module fs
Current Value 0x20000
Edited by: mho on Feb 7, 2008 9:26 AM -
Unable to clear the recently open files list
Currently, in order to clear the opened files list, the user has to go into the Windows registry. This by itself can be bad if the user is not familiar with the registry; could mess up the registry if not very careful. There appears to be no plug-in/addon for this option nor is there any user option to select to clear the list. There is a setting in the options/permissions to limit the number of opened files to the list; the minimum number being one(1). Cannot set this number to zero(0) unless maybe there is a registry fix for this. I want to suggest that in the next release, or at least as soon as possible, to add to the Adobe Reader (by default) a user option to be able to clear this list with having to enter the registry. Or, maybe create a plugin/addon to perform this action. That's it.
Hi Ineedtoknownow
You can raise your feature request on below link : https://www.adobe.com/cfusion/mmform/index.cfm?name=wishform -
"Too many open files" Exception on "tapestry-framework-4.1.1.jar"
When a browser attempts accessing to my webwork, the server opens a certain number of file descriptors to "tapestry-framework-4.1.1.jar" file and don't release them for a while.
Below is the output from "lsof | grep tapestry":
java 26735 root mem REG 253,0 62415 2425040 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-portlet-4.1.1.jar
java 26735 root mem REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
java 26735 root mem REG 253,0 320546 2425036 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-contrib-4.1.1.jar
java 26735 root mem REG 253,0 49564 2424979 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-annotations-4.1.1.jar
java 26735 root 28r REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
java 26735 root 29r REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
java 26735 root 30r REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
These unknown references are sometimes released automatically, but sometimes not.
And I get "Too many open files" exception after using my application for a few hours.
The number of the unknown references increases as I access to my webwork or just hit on "F5" key on my browser to reload it.
I tried different types of browsers to see if I could see any differences in consequence, and in fact it differed by the browser I used.
When viewed by Internet Explorer it increased by 3 for every access.
On the other hand it increased by 7 for each attempt when accessed by FireFox.
I have already tried optimizing the max number of file discriptors, and it solved the "Too many open files" exception.
But stil I'm wondering who actually is opening "tapestry-framework-4.1.1.jar" this many.
Could anyone figure out what is going on?
Thanks in advance.
The following is my environmental version info:
- Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
- Java: 1.5.0_11
- Tomcat: 5.5.20
- Tapestry: 4.1.1Hi,
Cause might The server got an exception while trying to accept client connections. It will try to backoff to aid recovery.
The OS limit for the number of open file descriptor (FD limit) needs to be increased. Tune OS parameters that might help the server to accept more client connections (e.g. TCP accept back log).
http://e-docs.bea.com/wls/docs90/messages/Server.html#BEA-002616
Regards,
Prasanna Yalam -
Linux Error: 23: Too many open files in system
I crashed the my oracle instance with the following error:
Tue Feb 13 22:15:16 2001
Errors in file /home/oracle/product/8.1.6/admin/v2qa1/bdump/lgwr_14175.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/db03/v2qa1/system/log/redo02.log'
ORA-27041: unable to open file
Linux Error: 23: Too many open files in system
Additional information: 2
LGWR: terminating instance due to error 313
Instance terminated by LGWR, pid = 14175
Is the number of open files adjustable?
Why am I opening files?
Could the fact that our java stored procedures are trying, unsuccessfully, be leaving files open?
Thanks - Craig
nullincreasing the oracle users ulimit to unlimited fixed that error for me. Type
ulimit -a
and look at the open files parameter. See your linux doc's to increase it to unlimited (will probably be increased from 1024 to 4096). -
[SOLVED] Maximun TCP open connections and Open File Descriptors
Hi,
back when I was using XP I needed to patch a system file to increase max amount of open TCP connections - you surely know about this. I'm wondering how do I do the same in GNU/Linux and if that's real important here.
Also, by default open files are set to 1024, how do I change that? I'm running x86_64 + KDE SC 4.4.2
Last edited by martin77 (2010-04-12 04:17:27)Thanks for replying.
I mean "open file handler" or in a better GNU/Linux terminology "open file descriptors" aka the maximun number of files that can be accessed at a given time.
For instance, VMWare will need you increase them to 4096 to work properly - and I presume something in the order of 5120 would be ok.
Following The Arch Way, already found the solution and want to share it with all of you fellow n00bs:
1. you need to open for edit /etc/security/limits.conf file with root privileges
2. at the end of the file add:
* soft nofile nn
* hard nofile nn
where nn is the number of open file descriptors you want. I set them to 8196 (probably too high) so for instance it should read:
* soft nofile 8192
* hard nofile 8192
Read the embedded help for a better understanding of this crucial configuration file.
As usual, thank you very much to this great community, devs and users, best!
Last edited by martin77 (2010-04-12 04:18:47) -
Sudden increase in open file descriptors
Our system is live since an year and half and for the first time I encountered the following exception
"java.net.SocketException: Too many open files"
When our internet application stopped responding I immediately checked the number of open file descriptors on my Solaris machine by using the "lsof" command and found that it was an abnormal 600 value and as I continued monitoring it reached 1024 in matter of minutes and WebLogic gave the above exception. The current setting for file descriptors is 1024. But all these days the average number of open file descriptors was well below 220.
I also took thread dumps and found that most of the threads were stuck at the following location
""ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x00883a90 nid=0x10 runnable [6d080000..6d0819c0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sybase.jdbc2.timedio.RawDbio.reallyRead(RawDbio.java:202)
at com.sybase.jdbc2.timedio.Dbio.doRead(Dbio.java:243)
at com.sybase.jdbc2.timedio.InStreamMgr.readIfOwner(InStreamMgr.java:512)
at com.sybase.jdbc2.timedio.InStreamMgr.doRead(InStreamMgr.java:273)
at com.sybase.jdbc2.tds.TdsProtocolContext.getChunk(TdsProtocolContext.java:561)
at com.sybase.jdbc2.tds.PduInputFormatter.readPacket(PduInputFormatter.java:229)
at com.sybase.jdbc2.tds.PduInputFormatter.read(PduInputFormatter.java:62)
at com.sybase.jdbc2.tds.TdsInputStream.read(TdsInputStream.java:81)
at com.sybase.jdbc2.tds.TdsInputStream.readUnsignedByte(TdsInputStream.java:114)
at com.sybase.jdbc2.tds.Tds.nextResult(Tds.java:1850)
at com.sybase.jdbc2.jdbc.ResultGetter.nextResult(ResultGetter.java:69)"
There is no file descriptor leak in the application. My question is since all the threads are hung at JDBC and socket level, does it mean that a faulty query would have triggered this problem(may be the database was too busy executing a faulty query).
I suspect this because I recieved a database exception soon after the problem appeared. One of my database insert transaction had timed out after 300 seconds. Also this was the first time I recieved this kind of an exception
java.sql.SQLException: The transaction is no longer active - status: 'Marked rollback. [Reason=weblogic.transaction.internal.TimedOutException: Transaction timed out after 299 seconds
Xid=BEA1-11FE69525362E51BFA16(6404670),Status=Active,numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds since begin=299,seconds left=60,activeThread=Thread[ExecuteThread: '22' for queue: 'weblogic.kernel.Default',5,Thread Group for Queue: 'weblogic.kernel.Default'],XAServerResourceInfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(ServerResourceInfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(state=started,assigned=none),xar=weblogic.jdbc.wrapper.JTSXAResourceImpl@a68c0),SCInfo[Mizuho-RWS+myserver]=(state=active),properties=({weblogic.jdbc=t3://10.104.8.81:7001}),OwnerTransactionManager=ServerTM[ServerCoordinatorDescriptor=(CoordinatorURL=myserver+10.104.8.81:7001+Mizuho-RWS+t3+, XAResources={},NonXAResources={})],CoordinatorURL=myserver+10.104.8.81:7001+Mizuho-RWS+t3+)]'. No further JDBC access is allowed within this transaction.
at weblogic.jdbc.wrapper.JTSConnection.checkIfRolledBack(JTSConnection.java:118)
at weblogic.jdbc.wrapper.JTSConnection.checkConnection(JTSConnection.java:127)
Any inputs regarding this problem?Raghu S wrote:
Hi,
I am using WebLogic 8.1 SP2 on a Solaris machine.Ok, good enough. Once WebLogic times out a transaction, it rolls it back on the
connection. That Sybase driver's rollback doesn't unfortunately affect it's
own running statements. For 81sp3 we added code to explicitly cancel any ongoing
statement during a rollback. This may be what you need to free up those
threads and the socekts the driver may be keeping open. If you can upgrade
to a newer version of 8.1, this code will free you up. Alternately, you can
try either upgrading to Sybase'e latest driver or to our BEA driver for Sybase.
Ask support for the latest BEA driver package for 8.1.
Joe
>
Stacktrace of one of the threads at the time I took the thread dump....All the threads at the time of taking the thread dump are stuck in a similar fashion.
ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x00883a90 nid=0x10 runnable [6d080000..6d0819c0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sybase.jdbc2.timedio.RawDbio.reallyRead(RawDbio.java:202)
at com.sybase.jdbc2.timedio.Dbio.doRead(Dbio.java:243)
at com.sybase.jdbc2.timedio.InStreamMgr.readIfOwner(InStreamMgr.java:512)
at com.sybase.jdbc2.timedio.InStreamMgr.doRead(InStreamMgr.java:273)
at com.sybase.jdbc2.tds.TdsProtocolContext.getChunk(TdsProtocolContext.java:561)
at com.sybase.jdbc2.tds.PduInputFormatter.readPacket(PduInputFormatter.java:229)
at com.sybase.jdbc2.tds.PduInputFormatter.read(PduInputFormatter.java:62)
at com.sybase.jdbc2.tds.TdsInputStream.read(TdsInputStream.java:81)
at com.sybase.jdbc2.tds.TdsInputStream.readUnsignedByte(TdsInputStream.java:114)
at com.sybase.jdbc2.tds.Tds.nextResult(Tds.java:1850)
at com.sybase.jdbc2.jdbc.ResultGetter.nextResult(ResultGetter.java:69)
at com.sybase.jdbc2.jdbc.SybStatement.nextResult(SybStatement.java:204)
at com.sybase.jdbc2.jdbc.SybStatement.nextResult(SybStatement.java:187)
at com.sybase.jdbc2.jdbc.SybStatement.executeLoop(SybStatement.java:1698)
at com.sybase.jdbc2.jdbc.SybStatement.execute(SybStatement.java:1690)
at com.sybase.jdbc2.jdbc.SybCallableStatement.execute(SybCallableStatement.java:129)
at weblogic.jdbc.wrapper.PreparedStatement.execute(PreparedStatement.java:68)
at com.mizuho.rws.report.business.dao.FIReportsDAO.getReportList(FIReportsDAO.java:3463)
at com.mizuho.rws.report.business.businessObject.FIReports.getReportList(FIReports.java:98)
at com.mizuho.rws.report.business.ejb.FIReportsBean.getReportList(FIReportsBean.java:96)
at com.mizuho.rws.report.business.ejb.FIReports_4f92ds_EOImpl.getReportList(FIReports_4f92ds_EOImpl.java:270)
at com.mizuho.rws.report.client.delegates.FIReportsBusinessDelegates.getReportList(FIReportsBusinessDelegates.java:173)
at com.mizuho.rws.report.client.web.FIReportsAction.handleFIReportsBean(FIReportsAction.java:1759)
at com.mizuho.rws.report.client.web.FIReportsAction.performAction(FIReportsAction.java:349)
at com.mizuho.foundation.presentation.AppBaseAction.perform(AppBaseAction.java:143)
at com.mizuho.foundation.presentation.AppActionServlet.processActionPerform(AppActionServlet.java:518)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1586)
at com.mizuho.foundation.presentation.AppActionServlet.doPost(AppActionServlet.java:562)
at com.mizuho.foundation.presentation.AppActionServlet.doGet(AppActionServlet.java:544)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:971)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:402)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:305)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6350)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:317)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3635)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2585)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
And the stack trace of the exception I recieved.
java.sql.SQLException: The transaction is no longer active - status: 'Marked rollback. [Reason=weblogic.transaction.internal.TimedOutException: Transaction timed out after 299 seconds
Xid=BEA1-11FE69525362E51BFA16(6404670),Status=Active,numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds since begin=299,seconds left=60,activeThread=Thread[ExecuteThread: '22' for queue: 'weblogic.kernel.Default',5,Thread Group for Queue: 'weblogic.kernel.Default'],XAServerResourceInfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(ServerResourceInfo[weblogic.jdbc.wrapper.JTSXAResourceImpl]=(state=started,assigned=none),xar=weblogic.jdbc.wrapper.JTSXAResourceImpl@a68c0),SCInfo[Mizuho-RWS+myserver]=(state=active
),properties=({weblogic.jdbc=t3://10.104.8.81:7001}),OwnerTransactionManager=ServerTM[ServerCoordinatorDescriptor=(CoordinatorURL=myserver+10.104.8.81:7001+Mizuho-RWS+t3+, XAResources={},NonXAResources={})],CoordinatorURL=myserver+10.104.8.81:7001+Mizuho-RWS+t3+)]'. No further JDBC access is allowed within this transaction.
at weblogic.jdbc.wrapper.JTSConnection.checkIfRolledBack(JTSConnection.java:118)
at weblogic.jdbc.wrapper.JTSConnection.checkConnection(JTSConnection.java:127)
at weblogic.jdbc.wrapper.Statement.checkStatement(Statement.java:222)
at weblogic.jdbc.wrapper.PreparedStatement.setString(PreparedStatement.java:414)
at com.mizuho.rws.services.mail.business.dao.FIReportMailDAO.insertMailClientDetails(FIReportMailDAO.java:2790)
at com.mizuho.rws.services.mail.business.businessObject.FIReportMail.sendMail(FIReportMail.java:645)
at com.mizuho.rws.services.mail.business.ejb.MailerBean.sendFIReportMail(MailerBean.java:87)
at com.mizuho.rws.services.mail.business.ejb.Mailer_fyyt2g_EOImpl.sendFIReportMail(Mailer_fyyt2g_EOImpl.java:662)
at com.mizuho.rws.services.mail.client.delegates.MailerBeanBusinessDelegates.sendFIReportMail(MailerBeanBusinessDelegates.java:153)
at com.mizuho.rws.services.mail.business.businessObject.SendMailHandler.sendFIReportMail(SendMailHandler.java:181)
at com.mizuho.rws.services.mail.business.businessObject.SendMailHandler.resolveMail(SendMailHandler.java:429)
at com.mizuho.rws.services.mail.business.businessObject.SendMailHandler.notify(SendMailHandler.java:651)
at com.mizuho.foundation.utils.AppNotificationListener.handleNotification(AppNotificationListener.java:66)
at weblogic.time.common.internal.TimerListener$1.run(TimerListener.java:48)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:317)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
at weblogic.time.common.internal.TimerListener.deliverNotification(TimerListener.java:44)
at weblogic.management.timer.Timer.deliverNotifications(Timer.java:578)
at weblogic.time.common.internal.TimerNotification$1.run(TimerNotification.java:118)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:317)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
Regards
Raghu -
Hi,
We are running WL 6.1 SP4 in a clustered environment....2 Quad CPU Sparc SUN boxes and one Quad CPU Intel box running Red Hat Linux (kernel 2.4.20) and have noticed that Linux has a LOT more open files than the Sun boxes (103281 for the Linux Box versus 2558 for the Solaris box). This seems to have the side affect of making the Linux box use lots more system CPU time compared to the SUN boxes.....my guess is that this is also not very efficient. Is there something that can be done to reduce the number of open files? We are using the Native I/O performance pack and the Sun 1.3.1_07 JVM.Finally we resolved this issue. It was oracle driver which had some compatibility issue, we upgraded our Oracle client driver to newer version, and this fixed the problem. Base line, there was nothing wrong with application code, code was doing good resource clean up, but oracle driver was leaking handles per every connection.
-
DirectoryService[67]: socket(PF_ROUTE) failed: Too many open files
Hello!
On MacOSX 10.4.10 which is an OD master, my log are filed with this:
DirectoryService[67]: socket(PF_ROUTE) failed: Too many open files
It happens at exactly every hours 44 minutes 18 seconds 161 times. At the same time, it makes a lot dns request for "kerberos-master.udp.XXXXXXXXX.COM IN SRV +"
The server works fine, but there's probably a cronjob that go crazy and i would like to know why it's happening.
Thanks a lot!
FredHi,
Cause might The server got an exception while trying to accept client connections. It will try to backoff to aid recovery.
The OS limit for the number of open file descriptor (FD limit) needs to be increased. Tune OS parameters that might help the server to accept more client connections (e.g. TCP accept back log).
http://e-docs.bea.com/wls/docs90/messages/Server.html#BEA-002616
Regards,
Prasanna Yalam -
Limit on number of open RandomAccessFiles
I'm implementing a small database using J2SDK1.4 NIO FileChannel and MappedByteBuffer as follows:
FileChannel fc = new RandomAccessFile(dataFile, "rw").getChannel();
MappedByteBuffer mbb = fc.map (FileChannel.MapMode.READ_WRITE, 0L, (int) fc.size());My problem is that there is a certain limit on the maximum number of open files and the native method RandomAccessFile.open(String name, int mode) throws a FileNotFoundException with message:(Too many open files).
This limit is not system/platform dependent, I experienced the same exception on both NT and Linux machines. Is there a way to raise this limit?
My workaround now is to close the FileChannel immediately after having mapped it to ByteBuffer but then I can only read/write to the buffer and cannot append more data to the file.
All help will be greatly appreciated.I have the exact same problem. Any solutions please?
-
My mac is becoming too slow. It takes long to open word documents, pdf files or excel documents or even safari. Can anybody suggest something? I have tried to reduce the number of open applications, but does not seem to work.
Hi ...
Checked to see how much free space there is on the startup disk lately?
Right or control click the MacintoshHD icon. Click Get Info. In the Get Info window you will see Capacity and Available. Make sure there's a minimum of 15% free disk space.
Freeing Up Hard Disk Space - Mac GuidesFreeing Up Hard Disk Space - Mac Guides
If disk space is not the issue, booting in Safe Mode deletes system caches that may help.
A Safe Mode boot takes longer then a normal boot so be patient.
Once you see the Desktop, click the Apple menu icon top left corner of the screen.
From the drop down menu click Restart.
See if that makes a difference ... -
I'm using acrobat pro in my project after debuging the project and after opening a certain number of PDF files I receive the message: the maximum number of files opened has been reached, you have to close some files to continu.even doing that, I steel receive the same message.Some one can tel what to do please? Thanks
Hi Memalyn
Essentially, the bare issue is that you have a 500GB hard drive with only 10GB free. That is not sufficient to run the system properly. The two options you have are to move/remove files to another location, or to install a larger hard drive (eg 2TB). Drive space has nothing to do with SMC firmware, and usually large media files are to blame.
My first recommendation is this: download and run the free OmniDiskSweeper. This will identify the exact size of all your folders - you can drill down into the subfolders and figure out where your largest culprits are. For example, you might find that your Pictures folder contains both an iPhoto Library and copies that you've brought in from a camera but are outside the iPhoto Library structure. Or perhaps you have a lot of purchased video content in iTunes.
If you find files that you KNOW you do not need, you can delete them. Don't delete them just because you have a backup, since if the backup fails, you will lose all your copies.
Don't worry about "cleaners" for now - they don't save much space and can actually cause problems. Deal with the large file situation first and see how you get on.
Let us know what you find out, and if you manage to get your space back.
Matt
Maybe you are looking for
-
When i go to the epson website it says the printer driver is available from apple. When i go to the printer selection it sees the printer but says that apple has no drivers for this printer. In the updates section of the app store it shows that the p
-
OneDrive (SkyDrive) only sync's to the roaming profile of 1 pc?
Hi, We got a problem with our Onedrive (we got the latest version), we make use of SharePoint 2013 and Office 2013 (we do not use Ofice 365). The SharePoint libraries we sync are stored in our local roaming file, but if i log on into another pc these
-
Wher does one see the setting for the ABAp debugger
Do you know where one can see if the debugger is set to update debugging?
-
HT204406 Itunes match subscription question
I have recently moved countries, but I need to cancel my itunes match subscripton before I can change my region in itunes how do I do this?
-
Why can't I favorite a two-page spread?
I wanted to perform an Auto Layout with two page spreads as part of the mix but when I try to "favorite" one of them, the option is greyed out. Is this happening to everyone? Any thoughts?