Max number of streams

Question

Hello,
My application is running out of stdio streams. There is a note in stdio man pages: no more than 255 files may be opened using fopen(), and only file descriptors 0 through 255 can be used in a stream.
The application creates a bunch of sockets (>255) and eventually calls fopen() that is failing because of the 255 limit. The application increases number of file descriptors that a process may create. This helps for socket() and open() calls but not for fopen(). Is there any workaround for this problem (other than use open() instead of fopen()?)
Regards,
--Stas                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           

Answer 1

Hi
First of all lot depends on which OS you are using. For all 32 bit
OS stdio(3S) the limit is 256. This is because the fd itself is
defined as an unsigned 8 bit value. On Sparc Solaris 7 and beyond with
64 bit option turned on it is 65536 (64K).
Having said that however there are ways to manipulate it. No guarantees that it will work, but try it anyways.
All versions of Solaris (including Solaris 7 64-bit) have a default
"soft" limit of 64 and a default "hard" limit of 1024.
Processes may need to open many files or sockets as file
descriptors. Standard I/O (stdio) library functions have
a defined limit of 256 file descriptors as the fopen() call,
datatype char, will fail if it can not get a file descriptor between
0 and 255.
The open() system call is of datatype "int", removing this limitation.
However, if open() has opened 0-255 file descriptors without closing
any, fopen() will not be able to open any file descriptors as all the
low-numbered ones have been used up. Applications that need to use
many file descriptors to open a large number of sockets, or
other raw files, should be forced to use descriptors
numbered above 256. This allows system functions such as
name services, to work as they depend upon stdio routines.
(See p 368 "Performance and Tuning - Java and the Internet").
There are limitations on the number of file descriptors
available to the current shell and its descendents. (See the ulimit
man page). The maximum number of file descriptors that can
be safely used for the shell and Solaris processes is 1024.
This limitation has been lifted for Solaris 7 64-bit which
can be 64k (65536) as explained before.
Therefore the recommended maximum values to be added to /etc/system
are:
set rlim_fd_cur=1024
set rlim_fd_max=1024
The in the shell :
Use the limit command with csh:
% limit descriptors 1024
Use the ulimit command with Bourne or ksh:
$ ulimit -n 1024
However, some third-party applications need the max raised.
A possible recommendation would be to increase rlim_fd_max,
but not the default (rlim_fd_cur). Then rlim_fd_cur can be
raised on a per-process basis if needed, but the higher setting
for rlim_fd_max doesn't affect all processes.
Let me know how it goes
-Manish
SUN-DTS