TIME_WAIT nio socket leaks
Hi All,
I've read many posting and sites regarding this problem, but still can't find a good solution. I really hope someone can help here.
I have a fairly simple client server setup. When the client disconnects the socket is being left in TIME_WAIT beyond the 2MSL.
TCP exchange goes something like this
client server
----------------------- connect
syn
syn ack
ack
----------------------- send message to server, send reply to client
psh ack
ack
psh ack
ack
----------------------- disconnect
fin ack
ack
fin ack
ack
but then the socket on the client side is never closed, stays in TIME_WAIT until JVM termination. The exchange looks a bit different from what I expected from reading the rfc on tcp ip. Anything obviously wrong with the above?
Cheers
Peter
JDK 1.5
problem discovered under suse, currently debugging under osx
There was a fd leak in early releases of JDK5 that arose with applications using the socket adaptor. This was fixed in 5.0u7 so if you are running an earlier update that this then it could be the problem you are seeing. You mention you are on SuSE then there is a second issue can arise if you have channel registered with a Selector and then Selector doesn't wakeup regularly. If the channels are closed (by invoking close, not by the remote peer) then the close will be delayed until the Selector wakes up. In a busy system this tends not to be noticed because you are continuously waking up for events but in an idle system it can be noticed. This problem is specific to the poll based Selector. If you are a 2.6 kernel and can use jdk6 then it uses the epoll facility which does not have this problem. If you can't move to jdk6 for some reason then you'll see in the 5.0 release notes there is a note explaining how to use the epoll facility.
Similar Messages
-
NIO sockets - cant get it to block!
Hi,
I'm trying to use NIO sockets in a program which sends data to a socket on another machine. I want it to block so that it must wait for a response before it continues. However this doesnt happen! I used Ethereal to see the packets being sent and recieved and the socket clearly doesnt block. Am I correct in thinking that NIO sockets are configured to block by default? I even tried using the configureBlocking(true) method to make sure but it still didnt block. Can anyone shed some light onto why this could be happening? Has this happened to anyone else?
Cheersthomfost wrote:
I've tried using regular I/O but its a bit too slow for what I need, im trying to see if using NIO improves this.Well you can stop now because it won't. NIO, as mentioned, is non-blocking IO, so a thread that is titled "NIO sockets - cant get it to block!" is not ultra promising. More importantly NIO will not make your IO "faster" by some magic. That's not what it does. It helps you write programs that scale because you don't need to have a thread dedicated to every client who connects to your system.
At any rate you have multiple mistakes here which suggest the other slowness problem you refer to is a logical problem in your code. So why don't you tell us about that instead? -
I think I'm being really dense here but for the life of me, I can't get this code to work properly. I am trying to build a NIO socket server using JDK 1.4.1-b21. I know about how the selector OP_WRITE functionality has changed and that isn't my problem. My problem is that when I run this code:
if (key.isAcceptable()) {
System.out.println("accept at " + System.currentTimeMillis());
socket = server.accept();
socket.configureBlocking(false);
socket.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
if (key.isWritable()) {
SocketChannel client = (SocketChannel)key.channel();
System.out.println("write at " + System.currentTimeMillis());
if (key.isReadable()) {
SocketChannel client = (SocketChannel)key.channel();
readBuffer.clear();
client.read(readBuffer);
System.out.println("read at " + System.currentTimeMillis());
}the isWritable if statement will always return (which is fine) and the isReadable if statement will NEVER return (which is most certainly NOT FINE!!). The readBuffer code is there just to clear out the read buffer so isReadable is only called once per data sent.
This SEEMS to be a bug in how the selector works? I would expect to see isReadable return true whenever data is sent, but that is not the case. Now here is the real kicker ;) Go ahead and change this line:
socket.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);to this:socket.register(selector, SelectionKey.OP_READ);And now it appears that isReadable is running as expected?! To let people run this code on their own, I have uploaded a copy of the entire java file here:
http://www.electrotank.com/lab/personal/mike/NioTest.java
Please forgive the code, it's just the smallest test harness I could make and much of it is lifted from other posts on this forum. You can test this by using Telnet and connecting to 127.0.0.1:8080. You can test the isReadable piece by just typing in the Telnet window.
Someone else has listed something as a bug in the Bug Parade, but the test case is flawed:
http://developer.java.sun.com/developer/bugParade/bugs/4755720.html
If this does prove to be a bug, has someone listed this already? Is there a nice clean workaround? I'm getting really desperate here. This bug makes the NIO socket stuff pretty unusable. Thanks in advance for the help!!
Mike Grundvig
[email protected]
Electrotank, Inc.Yeah, isReadable crashed for me too.
My solution was to not call it. I set up two selectors for the two operations I wanted notifying (accept and read) and used them independently in different threads. The accept thread passes them over to the read thread when they're accepted.
This way I don't need to call isReadable since it is bound to be readable otherwise it wouldn't have returned, as read is the only operation I'm being notified about.
--sam -
JSP+RMI = socket leak (with tomcat 5)
Tried this in the RMI forum, no answer, so let's try here...
When I use RMI from a JSP servlet, I have a "socket leak" problem. When I create an RMI unicast remote object, the RMI library (of course) opens up a server/listening socket. In a normal application, I can unbind the remote object, but then the next object I create will re-use the same server socket, so the server socket never closes, but only one is ever created, so it's not a big deal.
Now for the problem: When I use the Tomcat manager to unload my webapp, the socket is still there. Sure, fine. But then when I reload the webapp, instead of re-using the same server socket, a new one is created! Furthermore, if I try to use explicit port numbers to force the server socket to be reused, I get "port already in use" errors. It seems that somehow tomcat won't let RMI recycle its entry points in between webapp restarts. Does anybody know how I can either:
a) Get RMI to re-use the same ports
b) Get RMI to completely shut down its ports when the webapp is unloaded (this would be even better)
Thanks. It's irritating to see 10 open but unused server sockets on my system if I have to restart the webapp.Ah, I left out something: I construct the RMI object a servlet context listener in the contextInitialized() call, and I call UnicastRemoteObject.unexportObject() in the contextDestroyed() call. I added log messages, I am definitely correctly unbinding the object. The problem isn't that it is failing to be unbound - the problem is that the RMI system is refusing to reuse the same port after it is unbound. After unbinding, the port is still open, but it is never reused, each time I re-instantiate the remote object, it is making a new server socket and leaving the old one sitting open.
As for why I'm doing this, it is a bit like a homegrown EJB, but just so small that this was the simplest way. I have a largish non-web based application, then I have a few JSP pages that do database queries to provide information to browsers. The JSP pages are not at all central to the application though, and just show some database info. The JSP system caches some SQL data, and I use RMI to tell JSP when to flush cached data due to database changes. It's all very simple, and it works fantastic, except for the issue that every time I reload the webapp I leave these server sockets hanging open. Even that is just an irritation, I don't reload much except when I'm testing, but it's a big enough irritation that I would really like to fix it. -
Hello,
Im having difficulty understanding the WRITE issue of nio sockets. Ive read most of the nio-related posts on this forum but I still cant figure it out. Ive also looked at some examples by PkWooster but they seems really complicated.
I have a server which accepts connections and echos messages to all clients. Everything works fine except my client cannot write to the server. I know I need to switch back and forth between Op interests since they can block the cpu if set incorrectly but I dont really understand what to set when. Since my server seems to be working fine Ill post my Clients code:
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
import java.net.InetSocketAddress;
import java.util.*;
import java.nio.charset.*;
public class GameClient implements Runnable {
Game game;
SocketChannel sc;
ByteBuffer buffer;
Selector selector;
ByteBuffer sendBuffer=null;
String host = "192.168.1.102";
int port = 8888;
private CharsetDecoder asciiDecoder;
private CharsetEncoder encoder;
private LinkedList sendQ = new LinkedList();
SelectionKey sk;
LinkedList invocations;
public GameClient (Game g) {
game = g;
buffer = ByteBuffer.allocateDirect(1024);
Charset charset = Charset.forName("ISO-8859-1");
asciiDecoder = charset.newDecoder();
encoder = charset.newEncoder();
game.CharMSGArrival("Starting..."); // transfers data to the game thread
public void run(){
try{
// Create a nonblocking socket channel and connect to server.
sc = SocketChannel.open();
sc.configureBlocking(false);
game.CharMSGArrival("Connecting to server..."+"\n");
InetSocketAddress addr = new InetSocketAddress(host, port);
sc.connect(addr); // Nonblocking
while (!sc.finishConnect()) {
game.CharMSGArrival("I am waiting ..."+"\n" );
game.CharMSGArrival("Connection acqired"+ "\n" );
// Send initial message to server.
buffer.put((new Date().toString()+ "\n").getBytes());
buffer.flip();
sc.write(buffer);
// Create a new selector for use.
selector = Selector.open();
// Register the socket channel with the selector.
sk = sc.register(selector, SelectionKey.OP_READ);
while (true) {
//System.out.println("Listening for server on port " +
// remotePort);
// Monitor the registered channel.
int n = selector.select();
if (n == 0) {
continue; // Continue to loop if no I/O activity.
// Get an iterator over the set of selected keys.
Iterator it = selector.selectedKeys().iterator();
// Look at each key in the selected set.
while (it.hasNext()) {
// Get key from the selected set.
SelectionKey key = (SelectionKey) it.next();
// Remove key from selected set.
it.remove();
// Get the socket channel from the key.
SocketChannel keyChannel = (SocketChannel) key.channel();
// Is there data to read on this channel?
if (key.isReadable()) {
replyServer(keyChannel);
if (key.isWritable()) {
doSend();
} catch (IOException ioe) {
private void replyServer(SocketChannel socketChannel)
throws IOException {
// Read from server.
buffer.flip( );
int count = socketChannel.read(buffer);
if (count <= 0) {
// Close channel on EOF or if there is no data,
// which also invalidates the key.
socketChannel.close();
return;
buffer.flip( );
String str = asciiDecoder.decode(buffer).toString( );
game.CharMSGArrival(str);
public void doSend(){
sk.interestOps(SelectionKey.OP_WRITE);
if(sendBuffer != null)write(sendBuffer);
writeQueued();
public void trySend(String text){ // this method is invoked when a send buttong is pressed in the 'game' thread
sendQ.add(text); // first put it on the queue
writeQueued(); // write all we can from the queue
private void writeQueued()
while(sendQ.size() > 0) // now process the queue
String msg = (String)sendQ.remove(0);
write(msg); // write the string
private void write(String text)
try
ByteBuffer buf = encoder.encode(CharBuffer.wrap(text));
write(buf);
catch(Exception e){e.printStackTrace();}
* write out a byte buffer
private void write(ByteBuffer data)
SocketChannel scc = (SocketChannel)sk.channel();
if(scc.isOpen())
int len=0;
if(data.hasRemaining())
try{len = sc.write(data);}
catch(IOException e){e.printStackTrace(); //closeComplete();
if(data.hasRemaining()) // write would have blocked
game.CharMSGArrival("write blocked"+"/n");
//writeReady = false;
sk.interestOps(SelectionKey.OP_READ | SelectionKey.OP_WRITE); // select OP_WRITE
sendBuffer = data; // save the partial buffer
else {sendBuffer = null;
sk.interestOps(SelectionKey.OP_READ); }
else game.CharMSGArrival("write closed"+"/n");
}public void doSend(){
sk.interestOps(SelectionKey.OP_WRITE);
if(sendBuffer != null)write(sendBuffer);
writeQueued();
}This method makes no sense. You only get into it if the key is writable, which means it is already registered for OP_WRITE, so why would you register it again?
Also you're frequently ignoring the result of many write calls, of which you have too many.
See the thread 'Taming the NIO circus': http://forum.java.sun.com/thread.jspa?threadID=459338,
ignoring all the contributions except those by pkwooster and me.
I'm not that crazy about Mina myself. It has its own buffer classes and it quietly uses threads, i.e. missing the entire point of NIO. -
Wierdness with NIO socket on Solaris 2.10 part I
i tried the following NioClient and MockServer, and saw some weird behavior.
1. If i run both the client and server on the same machine on Windows, the client connects to the server, queries the instrument and gets the list of instruments back.
2. if i run both client and server on the same Solaris 2.10 box, the NioClient doesn't get anything back from the MockServer, not even an ACCEPT
3. if i run the client and the server on different solaris 2.10 machines, they work fine.
have anyone seen this before? can sometone sheds some lights?
import java.net.*;
import java.io.*;
public class MockServer
public static int counter = 2;
public static void main(String[] args) throws IOException {
int portNumber = Integer.parseInt(args[0]);
ServerSocket serverSocket = null;
try {
serverSocket = new ServerSocket(portNumber);
} catch (IOException e) {
System.err.println("Could not listen on port: " + portNumber);
System.exit(1);
System.out.println ("Listening on socket " + portNumber);
do {
Socket clientSocket = null;
try {
clientSocket = serverSocket.accept();
System.out.println ("starting a new client....");
MyThread mt = new MyThread (clientSocket);
mt.run ();
} catch (IOException e) {
System.err.println("Accept failed.");
System.exit(1);
while (true);
class MyThread
private final Socket clientSocket;
MyThread (Socket clientSocket)
this.clientSocket = clientSocket;
public void run ()
new Thread (new Runnable () {
public void run ()
try
boolean instrumentquery = false;
PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(
clientSocket.getInputStream()));
String inputLine;
String successoutputLine =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><return><returncode>success</returncode><detail>everything is good</detail></return>";
String failoutputLine =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><return><returncode>failure</returncode><detail>something is not good</detail></return>";
String instrumentsoutput =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><instruments><instrument><contract>usg-505Y</contract><cusip>12131121</cusip><deactivated>false</deactivated><halted>false</halted></instrument><instrument><contract>usg-305Y</contract><cusip>121312342</cusip><deactivated>false</deactivated><halted>false</halted></instrument></instruments>";
while ((inputLine = in.readLine()) != null) {
System.out.println ("Receiving the following" + inputLine);
if (inputLine.contains("queryInstrument")) instrumentquery = true;
if (inputLine.contains("</a>"))
if (instrumentquery)
instrumentquery = false;
System.out.println ("Sending " + instrumentsoutput);
out.println (instrumentsoutput);
else
if ((MockServer.counter % 2) == 0)
System.out.println ("Sending " + successoutputLine);
out.println(successoutputLine);
else
System.out.println ("Sending " + failoutputLine);
out.println(failoutputLine);
MockServer.counter++;
out.close();
in.close();
clientSocket.close();}
catch (Exception ex)
}).start (); }
}please see topic "wierdness with NIO socket on Solaris 2.10 part II" for the NioClient code as the maximum per topic is 5000.code for NioClient.java
import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.channels.spi.SelectorProvider;
import java.util.*;
public class NioClient implements Runnable {
private InetAddress hostAddress;
private int port;
private Selector selector;
private ByteBuffer readBuffer = ByteBuffer.allocate(8192);
private List pendingChanges = new LinkedList();
private Map pendingData = new HashMap();
private Map rspHandlers = Collections.synchronizedMap(new HashMap());
public NioClient(InetAddress hostAddress, int port) throws IOException {
this.hostAddress = hostAddress;
this.port = port;
this.selector = this.initSelector();
public void send(byte[] data, RspHandler handler) throws IOException {
SocketChannel socket = this.initiateConnection();
this.rspHandlers.put(socket, handler);
synchronized (this.pendingData) {
List queue = (List) this.pendingData.get(socket);
if (queue == null) {
queue = new ArrayList();
this.pendingData.put(socket, queue);
queue.add(ByteBuffer.wrap(data));
this.selector.wakeup();
public void run() {
while (true) {
try {
synchronized (this.pendingChanges) {
Iterator changes = this.pendingChanges.iterator();
while (changes.hasNext()) {
ChangeRequest change = (ChangeRequest) changes.next();
switch (change.type) {
case ChangeRequest.CHANGEOPS:
SelectionKey key = change.socket
.keyFor(this.selector);
key.interestOps(change.ops);
break;
case ChangeRequest.REGISTER:
change.socket.register(this.selector, change.ops);
break;
this.pendingChanges.clear();
this.selector.select();
Iterator selectedKeys = this.selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
if (!key.isValid()) {
continue;
if (key.isConnectable()) {
this.finishConnection(key);
} else if (key.isReadable()) {
this.read(key);
} else if (key.isWritable()) {
this.write(key);
} catch (Exception e) {
e.printStackTrace();
private void read(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
this.readBuffer.clear();
int numRead;
try {
numRead = socketChannel.read(this.readBuffer);
} catch (IOException e) {
key.cancel();
socketChannel.close();
return;
if (numRead == -1) {
key.channel().close();
key.cancel();
return;
this.handleResponse(socketChannel, this.readBuffer.array(), numRead);
private void handleResponse(SocketChannel socketChannel, byte[] data,
int numRead) throws IOException {
byte[] rspData = new byte[numRead];
System.arraycopy(data, 0, rspData, 0, numRead);
RspHandler handler = (RspHandler) this.rspHandlers.get(socketChannel);
if (handler.handleResponse(rspData)) {
socketChannel.close();
socketChannel.keyFor(this.selector).cancel();
private void write(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
synchronized (this.pendingData) {
List queue = (List) this.pendingData.get(socketChannel);
while (!queue.isEmpty()) {
ByteBuffer buf = (ByteBuffer) queue.get(0);
socketChannel.write(buf);
if (buf.remaining() > 0) {
break;
queue.remove(0);
if (queue.isEmpty()) {
key.interestOps(SelectionKey.OP_READ);
private void finishConnection(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
try {
socketChannel.finishConnect();
} catch (IOException e) {
System.out.println(e);
key.cancel();
return;
key.interestOps(SelectionKey.OP_WRITE);
private SocketChannel initiateConnection() throws IOException {
SocketChannel socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false);
socketChannel
.connect(new InetSocketAddress(this.hostAddress, this.port));
synchronized (this.pendingChanges) {
this.pendingChanges.add(new ChangeRequest(socketChannel,
ChangeRequest.REGISTER, SelectionKey.OP_CONNECT));
return socketChannel;
private Selector initSelector() throws IOException {
return SelectorProvider.provider().openSelector();
public static void main(String[] args) {
try {
System.out.println ("the host name is " + args[0]);
NioClient client = new NioClient(
InetAddress.getByName(args[0]), 4444);
Thread t = new Thread(client);
t.setDaemon(true);
t.start();
RspHandler handler = new RspHandler();
client.send(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><a><queryInstrument/></a>\n"
.getBytes(), handler);
handler.waitForResponse();
} catch (Exception e) {
e.printStackTrace();
} -
NIO Socket implementation - delay between select and get data from socket
Hi all,
I have implemented a internal CallAPI for RPC over a socket connection. It works fine but if there are more than five clients and some load I have the phenomena that the READ selector returns a SelectorKey but I did not get any data from the socket.
My implementation is based on NIO has following components:
+ Accept-Thread
Thread handles new clients.
+ Read-Thread
Thread handles the data from the socket for the registered client. Each request is handled in an own Process-Thread. A Thread-Pool implementation is used for processing.
+ Process-Thread
The Process-Thread reads the data from the socket and starts the processing of the logical request.
In my tests I get the notification of data at the socket. The Process-Thread want to read the data for the socket, but no data are available. In some situations if have to read about 20 times and more to get the data. Between each read attempt I have inserted a sleep in the Process-Thread if no data was available. This have improved the problem, but it already exists. I tested the problem with several systems and jvm's but it seams that it is independent from the system.
What can I to do improve the situation?
I already include the read implementation from the grizzly-Framework. But it doesn't improve the situation.
Socket - Init
protected void openSocket( String host, int port ) throws IOException
serverChannel = ServerSocketChannel.open();
serverChannel.configureBlocking( false );
serverSocket = serverChannel.socket();
serverSocket.setReuseAddress( true );
this.serverhost = host;
this.serverport = port;
this.srvAcceptSelector = Selector.open();
this.srvReadSelector = Selector.open();
InetSocketAddress isa = null;
if ( serverhost != null )
isa = new InetSocketAddress( this.serverhost, this.serverport );
else
isa = new InetSocketAddress( this.serverport );
serverSocket.bind( isa, 50 );
serverChannel.register( this.srvAcceptSelector, SelectionKey.OP_ACCEPT );
}New Client � Init
// New Client
if ( key.isAcceptable())
keyCountConnect++;
ServerSocketChannel actChannel =
(ServerSocketChannel) key.channel();
// Socket akteptieren
SocketChannel actSocket = actChannel.accept();
if ( actSocket != null )
actSocket.finishConnect();
actSocket.configureBlocking( false );
actSocket.socket().setTcpNoDelay( true );
this.registerSocketList.add( actSocket );
this.srvReadSelector.wakeup();
}Read Data from Socket
protected int readDatafromSocket( ByteArrayOutputStream socketdata )
throws IOException
int readedChars = 0;
int count = -1;
Selector readSelector = null;
SelectionKey tmpKey = null;
if ( sc.isOpen())
ByteBuffer inputbuffer = null;
try
inputbuffer = bufferpool.getBuffer();
while (( count = sc.read( inputbuffer )) > 0 )
readedChars += count;
inputbuffer.flip();
byte[] tmparray=new byte[inputbuffer.remaining()];
inputbuffer.get( tmparray );
socketdata.write( tmparray );
inputbuffer.clear();
if ( count < 0 )
this.closeSocket();
if( readedChars == 0 )
readedChars = -1;
if ( log.isDebug())
log.debug( "Socket is closed! " );
else if ( readedChars == 0 )
if ( log.isDebug())
log.debug( "Reread with TmpSelector" );
// Glassfish/Grizzly-Implementation
readSelector = SelectorFactory.getSelector();
if ( readSelector == null )
return 0;
count = 1;
tmpKey = this.sc.register( readSelector, SelectionKey.OP_READ );
tmpKey.interestOps(
tmpKey.interestOps() | SelectionKey.OP_READ );
int code = readSelector.select( 500 );
tmpKey.interestOps(
tmpKey.interestOps() & ( ~SelectionKey.OP_READ ));
if ( code == 0 )
return 0;
// Return on the main Selector and try again.
while (( count = sc.read( inputbuffer )) > 0 )
readedChars += count;
inputbuffer.flip();
byte[] tmparray=new byte[inputbuffer.remaining()];
inputbuffer.get( tmparray );
socketdata.write( tmparray );
inputbuffer.clear();
if ( count < 0 )
this.closeSocket();
if( readedChars == 0 )
readedChars =-1;
else if ( count == 0 )
// No data
finally
if ( inputbuffer != null )
bufferpool.releaseBuffer( inputbuffer );
inputbuffer = null;
// Glassfish-Implementierung
if ( tmpKey != null )
tmpKey.cancel();
if ( readSelector != null)
// Bug 6403933
try
readSelector.selectNow();
catch (IOException ex)
SelectorFactory.returnSelector( readSelector );
return readedChars;
}Thanks for your time.I've commented on that blog before. It is rubbish:
- what does 'overloading the main Selector' actually mean? if anything?
- 'Although this not clearly stated inside the NIO API documentation': The API documentation doesn't say anything about which Selector you should register channels with. Why would it? Register it with any Selector you like ...
- 'the cost of maintaining multiple Selectors can reduce scalability instead of improving it' Exactly. So what is the point again?
- 'wrapping a ByteBuffer inside a ByteBufferInputStream:' Code is rubbish and redundant. java.nio.channels.Channels has methods for this.
There is no a priori advantage to using multiple Selectors and threads unless you have multiple CPUs. And even then not much, as non-blocking reads and writes don't consume significant amounts of CPU. It's the processing of the data once you've got it that takes the CPU, and that should be done in a separate thread.
So I would re-evaluate your strategy. I suspect you're getting the channel registered with more than one Selector at a time. Implement it the simple way first then see if you really have a problem with 'overloading the main Selector' ... -
Server Socket Leaks on stopping Java Processes
I have a strange problem on Windows 2000 Advanced Server with JDK1.5.10. Our application runs on a Tomcat 5.5.12 and is opening 2 server sockets along with Tomcat's own 8080 port. Also we deployed a stand alone application that also opens a server socket. Whenever we stop Tomcat service or our Application service (Windows services) these server sockets are not getting closed. If we use "netstat" to check if any sockets are still open it is NOT showing any sockets.
But when we use "netstat -a" command it is showing entries like
TCP 10.40.1.162:8080 10.40.1.162:0 LISTENING
TCP 10.40.1.162:22222 10.40.1.162:0 LISTENING
TCP 10.40.1.162:33334 10.40.1.162:0 LISTENINGHere 8080 (tomcat), 22222, 33334 are the ports on which the server sockets are opened earlier. We check in Task Manager but there is no java process running. If we try to restart the same java process it is throwing JVM_Bind exception. If we reboot the box the we are able to restart the services. Similar behaviour we observed sometimes on Linux also. I guess some settings in the OS we need to tweak to fix this issue. Can somebody please help me what are those settings for Windows and Linux platforms which reclaims unused server sockets faster??The state is LISTENING not TIME_WAIT. There is a cleanly closure of sockets provided in the code. It happens to Tomcat's own ports also. If you telnet to these sockets from another box, it is connecting and staying instead of "Connection Failed" error. But in Task Manager or "Services.msc" Console no service or process running! One more observation is that there is heavy load of threads and a huge number of clients socktets were interacting with these sockets just before shutdown.
-
Working with NIO and non-NIO sockets
Hello,
Trying to write a network manager kind of program where the client initiates a number of connections to different servers. Need to manage these connections through the life time of the session ( time the client is connected to the server). The client connections originate from a single machine, the servers are geographically distributed and about 1 to 5 connections between each client and server.
I have tried and example NIO client/server application and it works fine. I have tried the KnockKnock client/server example it works. But when I try to use the NIO client with the multithreaded knockknock server, the socket connection goes through okay. However the data written to socket channel on the client(NIO) side doesn't seem to reach the server(non-NIO). I can't change the servers since they already exist in the field. I am just trying to write the manager application to manage them.
Basically I am trying to avoid writing a multithreaded socket connection management program.
Thanks for any input.Not the greatest NIO code. The author seems unaware that closing a channel also cancels all its selection keys so he laboriously programs both operations every time he needs to close the channel. His discussion of OP_CONNECT makes little sense: the important point is that OP_CONNECT is only used in the connection phase and OP_WRITE is only used in the connected phase.
His code registers the channel for OP_WRITE as soon as the connection completes, when there is no pending data to write. This will just cause the selector to spin uselessly doing nothing. The channel should be registered with zero interest-ops and OP_WRITE should be registered when there is something to write, i.e. during the send() operation; and it should be deregistered when the write has completed successfully and there is no more data to write. -
Need help with java.nio.socket program.
Can someone please help me here? I am trying to create a class that listens on a socket for an incoming connection, but I am getting the following error on compile:
SocketIn.java:48: incompatible types
found : java.nio.channels.SocketChannel
required: java.net.Socket
Socket requestSocket = requestChannel.accept();
Here is my code:
package NETC;
import java.net.*;
import java.util.*;
import java.io.*;
import java.nio.*;
import java.nio.charset.*;
import java.nio.channels.*;
import java.nio.channels.spi.*;
public class SocketIn{
// Buffer for any incoming data.
private static ByteBuffer inBuffer = ByteBuffer.allocateDirect(1024);
CharBuffer charBuffer = null;
public SocketIn(int intPort)throws Exception{
// Create a non-blocking server socket.
ServerSocketChannel serverChannel = ServerSocketChannel.open();
serverChannel.configureBlocking(false);
// Use the host and port to bind the server socket.
InetAddress inetAddress = InetAddress.getLocalHost();
InetSocketAddress socketAddress = new InetSocketAddress(inetAddress, intPort);
serverChannel.socket().bind(socketAddress);
// The selector for incoming requests.
Selector requestSelector = SelectorProvider.provider().openSelector();
// Put the server socket on the selectors 'ready list'.
serverChannel.register(requestSelector, SelectionKey.OP_ACCEPT);
while(requestSelector.select() > 0){
System.out.println("Connection Accepted...");
// A request has been made and is ready for IO.
Set requestKeys = requestSelector.selectedKeys();
Iterator iterator = requestKeys.iterator();
while(iterator.hasNext()){
SelectionKey requestKey = (SelectionKey)iterator.next();
// Get the socket that's ready for IO.
ServerSocketChannel requestChannel = (ServerSocketChannel)requestKey.channel();
// This shouldn't block.
Socket requestSocket = requestChannel.accept();
inBuffer.clear();
requestSocket.read(inBuffer);
inBuffer.flip();
Charset charset = Charset.forName("ISO-8859-1");
CharsetDecoder decoder = charset.newDecoder();
charBuffer = decoder.decode(inBuffer);
iterator.remove();
}Well its been about a month and a half so I guess you have your answer. In case you don't...
// This shouldn't block.
Socket requestSocket = requestChannel.accept();
The problem is quite simple. The ServerSocketChannel.accept() returns a SocketChannel. (Channel -> Channel).
You can get access to the socket using the SocketChannel.
SocketChannel requestSocketChannel = requestChannel.accept();
Socket requestSocket = requestSocketChannel.socket();
But you don't need the socket to read and write. -
NIO Socket Reading - Intermittent Latency in High-Speed Data Reading
Our application is reading data very fast over TCP/IP sockets in Java. We are using the NIO library with a non-blocking Sockets and a Selector to indicate readiness to read. On average, the overall processing times for reading and handling the read data is sub-millisecond. However we frequently see spikes of 10-20 milliseconds. (running on Linux).
Using tcpdump we can see the time difference between tcpdump's reading of 2 discreet messages, and compare that with our applications time. We see tcpdump seems to have no delay, whereas the application can show 20 milliseconds.
We are pretty sure this is not GC, because the GC log shows virtually no Full GC, and in JDK 6 (from what I understand) the default GC is parallel, so it should not be pausing the application threads (unless doing Full GC).
It looks almost as if there is some delay for Java's Selector.select(0) method to return the readiness to read, because at the TCP layer, the data is already available to be read (and tcpdump is reading it).Have you tried profiling your application to confirm that the wait is indeed in the Selector.select() method, as you think?
PS: Selector.select(0) will not return immediately if no channels are selected as you may assume. select(0) is the same as a regular select(): it will block until at least 1 channel is selected. -
Wierdness with NIO socket on Solaris 2.10 part I I
import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.channels.spi.SelectorProvider;
import java.util.*;
public class NioClient implements Runnable {
private InetAddress hostAddress;
private int port;
private Selector selector;
private ByteBuffer readBuffer = ByteBuffer.allocate(8192);
private List pendingChanges = new LinkedList();
private Map pendingData = new HashMap();
private Map rspHandlers = Collections.synchronizedMap(new HashMap());
public NioClient(InetAddress hostAddress, int port) throws IOException {
this.hostAddress = hostAddress;
this.port = port;
this.selector = this.initSelector();
public void send(byte[] data, RspHandler handler) throws IOException {
SocketChannel socket = this.initiateConnection();
this.rspHandlers.put(socket, handler);
synchronized (this.pendingData) {
List queue = (List) this.pendingData.get(socket);
if (queue == null) {
queue = new ArrayList();
this.pendingData.put(socket, queue);
queue.add(ByteBuffer.wrap(data));
this.selector.wakeup();
public void run() {
while (true) {
try {
synchronized (this.pendingChanges) {
Iterator changes = this.pendingChanges.iterator();
while (changes.hasNext()) {
ChangeRequest change = (ChangeRequest) changes.next();
switch (change.type) {
case ChangeRequest.CHANGEOPS:
SelectionKey key = change.socket
.keyFor(this.selector);
key.interestOps(change.ops);
break;
case ChangeRequest.REGISTER:
change.socket.register(this.selector, change.ops);
break;
this.pendingChanges.clear();
this.selector.select();
Iterator selectedKeys = this.selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
if (!key.isValid()) {
continue;
if (key.isConnectable()) {
this.finishConnection(key);
} else if (key.isReadable()) {
this.read(key);
} else if (key.isWritable()) {
this.write(key);
} catch (Exception e) {
e.printStackTrace();
private void read(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
this.readBuffer.clear();
int numRead;
try {
numRead = socketChannel.read(this.readBuffer);
} catch (IOException e) {
key.cancel();
socketChannel.close();
return;
if (numRead == -1) {
key.channel().close();
key.cancel();
return;
this.handleResponse(socketChannel, this.readBuffer.array(), numRead);
private void handleResponse(SocketChannel socketChannel, byte[] data,
int numRead) throws IOException {
byte[] rspData = new byte[numRead];
System.arraycopy(data, 0, rspData, 0, numRead);
RspHandler handler = (RspHandler) this.rspHandlers.get(socketChannel);
if (handler.handleResponse(rspData)) {
socketChannel.close();
socketChannel.keyFor(this.selector).cancel();
private void write(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
synchronized (this.pendingData) {
List queue = (List) this.pendingData.get(socketChannel);
while (!queue.isEmpty()) {
ByteBuffer buf = (ByteBuffer) queue.get(0);
socketChannel.write(buf);
if (buf.remaining() > 0) {
break;
queue.remove(0);
if (queue.isEmpty()) {
key.interestOps(SelectionKey.OP_READ);
private void finishConnection(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
try {
socketChannel.finishConnect();
} catch (IOException e) {
System.out.println(e);
key.cancel();
return;
key.interestOps(SelectionKey.OP_WRITE);
private SocketChannel initiateConnection() throws IOException {
SocketChannel socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false);
socketChannel
.connect(new InetSocketAddress(this.hostAddress, this.port));
synchronized (this.pendingChanges) {
this.pendingChanges.add(new ChangeRequest(socketChannel,
ChangeRequest.REGISTER, SelectionKey.OP_CONNECT));
return socketChannel;
private Selector initSelector() throws IOException {
return SelectorProvider.provider().openSelector();
public static void main(String[] args) {
try {
System.out.println ("the host name is " + args[0]);
NioClient client = new NioClient(
InetAddress.getByName(args[0]), 4444);
Thread t = new Thread(client);
t.setDaemon(true);
t.start();
RspHandler handler = new RspHandler();
client.send(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><a><queryInstrument/></a>\n"
.getBytes(), handler);
handler.waitForResponse();
} catch (Exception e) {
e.printStackTrace();
}Edited by: LuriRon on Mar 16, 2009 10:42 AMimport java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.channels.spi.SelectorProvider;
import java.util.*;
public class NioClient implements Runnable {
private InetAddress hostAddress;
private int port;
private Selector selector;
private ByteBuffer readBuffer = ByteBuffer.allocate(8192);
private List pendingChanges = new LinkedList();
private Map pendingData = new HashMap();
private Map rspHandlers = Collections.synchronizedMap(new HashMap());
public NioClient(InetAddress hostAddress, int port) throws IOException {
this.hostAddress = hostAddress;
this.port = port;
this.selector = this.initSelector();
public void send(byte[] data, RspHandler handler) throws IOException {
SocketChannel socket = this.initiateConnection();
this.rspHandlers.put(socket, handler);
synchronized (this.pendingData) {
List queue = (List) this.pendingData.get(socket);
if (queue == null) {
queue = new ArrayList();
this.pendingData.put(socket, queue);
queue.add(ByteBuffer.wrap(data));
this.selector.wakeup();
public void run() {
while (true) {
try {
synchronized (this.pendingChanges) {
Iterator changes = this.pendingChanges.iterator();
while (changes.hasNext()) {
ChangeRequest change = (ChangeRequest) changes.next();
switch (change.type) {
case ChangeRequest.CHANGEOPS:
SelectionKey key = change.socket
.keyFor(this.selector);
key.interestOps(change.ops);
break;
case ChangeRequest.REGISTER:
change.socket.register(this.selector, change.ops);
break;
this.pendingChanges.clear();
this.selector.select();
Iterator selectedKeys = this.selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
if (!key.isValid()) {
continue;
if (key.isConnectable()) {
this.finishConnection(key);
} else if (key.isReadable()) {
this.read(key);
} else if (key.isWritable()) {
this.write(key);
} catch (Exception e) {
e.printStackTrace();
private void read(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
this.readBuffer.clear();
int numRead;
try {
numRead = socketChannel.read(this.readBuffer);
} catch (IOException e) {
key.cancel();
socketChannel.close();
return;
if (numRead == -1) {
key.channel().close();
key.cancel();
return;
this.handleResponse(socketChannel, this.readBuffer.array(), numRead);
private void handleResponse(SocketChannel socketChannel, byte[] data,
int numRead) throws IOException {
byte[] rspData = new byte[numRead];
System.arraycopy(data, 0, rspData, 0, numRead);
RspHandler handler = (RspHandler) this.rspHandlers.get(socketChannel);
if (handler.handleResponse(rspData)) {
socketChannel.close();
socketChannel.keyFor(this.selector).cancel();
private void write(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
synchronized (this.pendingData) {
List queue = (List) this.pendingData.get(socketChannel);
while (!queue.isEmpty()) {
ByteBuffer buf = (ByteBuffer) queue.get(0);
socketChannel.write(buf);
if (buf.remaining() > 0) {
break;
queue.remove(0);
if (queue.isEmpty()) {
key.interestOps(SelectionKey.OP_READ);
private void finishConnection(SelectionKey key) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
try {
socketChannel.finishConnect();
} catch (IOException e) {
System.out.println(e);
key.cancel();
return;
key.interestOps(SelectionKey.OP_WRITE);
private SocketChannel initiateConnection() throws IOException {
SocketChannel socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false);
socketChannel
.connect(new InetSocketAddress(this.hostAddress, this.port));
synchronized (this.pendingChanges) {
this.pendingChanges.add(new ChangeRequest(socketChannel,
ChangeRequest.REGISTER, SelectionKey.OP_CONNECT));
return socketChannel;
private Selector initSelector() throws IOException {
return SelectorProvider.provider().openSelector();
public static void main(String[] args) {
try {
System.out.println ("the host name is " + args[0]);
NioClient client = new NioClient(
InetAddress.getByName(args[0]), 4444);
Thread t = new Thread(client);
t.setDaemon(true);
t.start();
RspHandler handler = new RspHandler();
client.send(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><a><queryInstrument/></a>\n"
.getBytes(), handler);
handler.waitForResponse();
} catch (Exception e) {
e.printStackTrace();
}Edited by: LuriRon on Mar 16, 2009 10:42 AM -
I see some socket applications in my work, the client side applications usually don't bother to explicitly close the socket connection. TCP/IP doesn't specify that if a socket is not used for a long time, the socket will be closed automatically. i wonder if the client doesn't close the connection, will there be a resource leak? and will the server application hold the connection forever because the client neglects to notify it?
The client side applications need fixing. This is a basic error. Yes there is a resource leak, several in fact: the Socket object, its input and output streams, any buffers associated with them, the socket handle, the kernel structures associated with the socket, the kernel socket send and receive buffers, ... and at the server side, all that plus either a Thread or a SelectionKey and whatever is attached to it.
Socket.setKeepAlive(true) and Socket.setSoTimeout() should both be used at the server to defend against this sort of thing, but it shouldn't be permitted to enter the client application in the first place. What else don't they 'bother' to release? This is a major quality issue. -
Identifying and managing clients in nonblocking NIO socket programming
Here's my beef. When I write a Server using a thread for every new connection that comes in, I can remember everything for each client in its respective thread. I can have a table entry in a MYSQL database for each client which each thread can look up by a remembered id. I can direct packets from a different application to a specific client by passing a message to the thread associated with the client. But, in this implementation, the client-server setup will need to have a master-slave relationship where one is always reading, and can talk on the network only in response.
I like the NIO selector way of doing things, but I do not get a memory method to deal with specific clients. How do I, for example,
direct packets to a specific client.?
recognize incoming packet info and make sense of its context in relation to past messages to and from the same client?
maintain one-to-one correspondence between a MYSQL database table entry and a specific client?
Could somebody explain this to me how I can maintain a client environment, and a client state machine with memory?
Thanks
Anilcloud9ine wrote:
Here's my beef. When I write a Server using a thread for every new connection that comes in, I can remember everything for each client in its respective thread. I can have a table entry in a MYSQL database for each client which each thread can look up by a remembered id. I can direct packets from a different application to a specific client by passing a message to the thread associated with the client. But, in this implementation, the client-server setup will need to have a master-slave relationship where one is always reading, and can talk on the network only in response.
I like the NIO selector way of doing things, but I do not get a memory method to deal with specific clients. How do I, for example,
direct packets to a specific client.?
recognize incoming packet info and make sense of its context in relation to past messages to and from the same client?
maintain one-to-one correspondence between a MYSQL database table entry and a specific client?
Could somebody explain this to me how I can maintain a client environment, and a client state machine with memory?
Thanks
AnilYou will use select() and SelectionKey when you deal with NIO. The SelectionKey has an attach method that you can use to associate the client with something, e.g. a session. Your Session class can contain all that you now have in your thread.
Kaj -
File descriptor leak in socket programming
We have a complex socket programming client package in java using java.nio (Selectors, Selectable channel).
We use the package to connect to a server.
Whenever the server is down, it tries to reconnect to the server again at regular intervals.
In that case, the number of open file descriptors build up with each try. I am able to cofirm this using "pfile <pid>" command.
But, it looks like we are closing the channels, selectors and the sockets properly when it fails to connect to the server.
So we are unable to find the coding that causes the issue.
We run this program in solaris.
Is there a tool to track down the code that leaks the file descriptor.
Thanks.Don't close the selector. There is a selector leak. Just close the socket channel. As this is a client you should then also call selector.selctNow() to have the close take final effect. Otherwise there is also a socket leak.
Maybe you are looking for
-
Hello I recently bought an mac air 2012 and want to upgrade the internal memory HDD and simms ( from 4 to 8 gigs) is it possible and also worth it.
-
HT6114 Kernel Panic in OS X 10.9.2 copying a file to a file server
Hi, i've made the update to OS X 10.9.2 3 days ago and it looked like we had all our problems with this OS X solved. Today, I deleted from a share in a NAS (file server) a small file and tried to copy another one with the same name, but the moment th
-
ABAP Proxy response not showing data in SXMB_MONI
hi all, We have a scenario of web service to abap proxy,synchronous. when the response comes back from ABAP,the xml in SXMB_MONI is not coming fully.Only the top level xml node and the Message type node are being dispalyed.the data nodes are not comi
-
"Unknown User" account recovery, Please help
Hi, Inadvertently i have been deleted an "unknown user" from the (System Properties-->User Profiles) this mistake deleted my "Desktop" and "My Document" files and other settings. How can i resolve this problem or recover my files, i tried to use (Sys
-
Hi Guys, I have 2 (SQL Query's) VO's with primary key -foriegn key relation for the underlying tables but when i create a VL the view link query dosent showup correctly ... is it necessary to have VO's based on Eo's for view links to work properly ?