NIO problems.
We have developed a NIO C/S server, the client side is pluged into Tomcat.
We have encounted an out memory exception in tomcat, and dump the tomcat, and find NIO Client thread is blocked. Is the block normal? or its some bugs?
"Thread-2862" daemon prio=5 tid=0x00dabf60 nid=0x35dd runnable [ce481000..ce4819c8]
at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:136)
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:70)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:59)
- locked <0xf1b373d8> (a java.util.HashSet)
- locked <0xf1b37460> (a java.util.HashSet)
- locked <0xf1b37308> (a sun.nio.ch.DevPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:70)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:74)
at com.televigation.databus.io.SelectorThread.run(SelectorThread.java:429)
at java.lang.Thread.run(Thread.java:534)
Thank you for your concern, the problem for the "POST " method is solved ( the request was constructed in a wrong way), but I'm still facing the zero problem , I have three cases which I receive zero bytes from the server:
1. The last chunk, in this case I received the whole responce correctly and this indicates the end of stream
2. I received some of the response and when I register into the selector to read again I got another chunk till I receive the whole response.
3. This is the actuall problem, I receive zero bytes and register for read again I got nothing and this key remains into the selector !! with no other chunks , and I didn't receive the whole response correctly !!!!
Thanks again
Similar Messages
-
I had a NIO problem.
I tried to open a socket channel, and grep a HTML page from HTTP GET.
I did it success if I open and close socket channel everytime I send a
request and get the response.
However, if I keep open the socket channel and send multiple requests,
I'll get IOException.
Does anybody know how to solve this problem? Thanks in advance!
Note: I don't want to fire requests in parallel but one by one
sample code:
=============================================================
private Charset charset = Charset.forName("ISO-8859-1");
private SocketChannel channel;
try
// do connection
InetSocketAddress socketAddress =
new InetSocketAddress( "www.mydomain.com", 80);
channel = SocketChannel.open(socketAddress);
// send request #1
channel.write(charset.encode("GET /page1.html HTTP/1.0\r\n\r\n") );
// read response #1
ByteBuffer buffer = ByteBuffer.allocate(1024);
while ((channel.read(buffer)) != -1)
buffer.flip();
System.out.println(charset.decode(buffer));
buffer.clear();
// send request #2
channel.write(charset.encode("GET /page2.html HTTP/1.0\r\n\r\n") );
// read response #2
// program threw "java.io.IOException: Read failed" error
while ((channel.read(buffer)) != -1)
buffer.flip();
System.out.println(charset.decode(buffer));
buffer.clear();
catch (IOException e)
e.printStackTrace();
finally
if (channel != null)
try
channel.close();
} catch (IOException e) {}
=============================================================I am pretty sure that most webservers only do one request per connection by default. Try looking up some information on HTTP request headers. I am pretty sure there is a header that you need to send to the server to let the server know that you want to do multiple requests with the same connection. I believe the header is called 'keep alive' or some other such thing.
I'm not an expert on http, so I can't help you any further that that. -
I think I'm being really dense here but for the life of me, I can't get this code to work properly. I am trying to build a NIO socket server using JDK 1.4.1-b21. I know about how the selector OP_WRITE functionality has changed and that isn't my problem. My problem is that when I run this code:
if (key.isAcceptable()) {
System.out.println("accept at " + System.currentTimeMillis());
socket = server.accept();
socket.configureBlocking(false);
socket.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
if (key.isWritable()) {
System.out.println("write at " + System.currentTimeMillis());
if (key.isReadable()) {
SocketChannel client = (SocketChannel)key.channel();
readBuffer.clear();
client.read(readBuffer);
System.out.println("read at " + System.currentTimeMillis());
}the isWritable if statement will always return (which is fine) and the isReadable if statement will NEVER return (which is most certainly NOT FINE!!). The readBuffer code is there just to clear out the read buffer so isReadable is only called once per data sent.
This SEEMS to be a bug in how the selector works? I would expect to see isReadable return true whenever data is sent, but that is not the case. Now here is the real kicker ;) Go ahead and change this line:socket.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);to this:socket.register(selector, SelectionKey.OP_READ);And now it appears that isReadable is running as expected?! To let people run this code on their own, I have uploaded a copy of the entire java file here:
http://www.electrotank.com/lab/personal/mike/NioTest.java
Please forgive the code, it's just the smallest test harness I could make and much of it is lifted from other posts on this forum. You can test this by using Telnet and connecting to 127.0.0.1:8080. You can test the isReadable piece by just typing in the Telnet window.
Someone else has listed something as a bug in the Bug Parade, but the test case is flawed:
http://developer.java.sun.com/developer/bugParade/bugs/4755720.html
If this does prove to be a bug, has someone listed this already? Is there a nice clean workaround? I'm getting really desperate here. This bug makes the NIO socket stuff pretty unusable. Thanks in advance for the help!!
Mike Grundvig
[email protected]
Electrotank, Inc.I'm not sure that a key will ever return with more than one operation bit selected. I expect that the selector simply puts the key into the selected set as soon as it sees any one of the interestedOps that match the current state.
The general problem you are having, it seems, is that isWritable(), in many cases, will constantly return true for a low volume socket. This being the case, the only solution I see is to either deal with output with blocking calls or to dynamically change your interestedOps when there is data to write. That's what I did. I don't know yet how well it will scale. This is certainly not a bug, the application must deal with it. Perhaps it would be nice to have some kind of interface, like "SelectionKeyAttachment" and have the SelectionKey.attach() method take one of those. The SelectionKeyAttachment interface could implement something like isWritable(), isReadable(), isAcceptable() so that the Selector can also invoke the callback before putting it into the selected set. I haven't really thought this through, just initial thoughts. -
Hello.
I am using a NIO server for my application with a pool of WorkingThreads to handle client requests.
The read part seems to work fine (I am deregistering the OP_READ interest when I begin to read from the socketchannel and I register it back when I finish processing the data). This way I am able to process data coming in chunks down the stream.
Now, the problem I have is with writing data back to the client. If I send the data with only 1 socketChannel.write(ByteBuffer) method the client gets it correctly. But if I try to write in a while() loop more messages it happens that the client receives the first chunk correctly but SOMETIMES it receives rubbish as the next chunks. My guess is that after the first write(), the server may send some zeros to the client just before the next chunk so the client interprets that as valid data.
I have tried to unregister the key from writing (key.interestOps(key.interestOps() & (~SelectionKey.OP_WRITE))) immediatelly after I've done a write() and reregister it for writing just before I want to send the data but this didn't help. I'm sure this is solvable but I haven't met anywhere code that deals with writing more than 1 buffer (like writing a big file for example).
Any help would be very much appreciated.
Thank you.
BogdanThank you for replying, ejp, but it seems that what you said is not quite true. Here is my send method:
public void send(SelectionKey key) {
try {
if (sendMessages.size() != 0) {
String message = null;
while (sendMessages.size() > 0) {
synchronized (sendMessages) {
message = (String) sendMessages.removeFirst();
ByteBuffer theBuffer = ByteBuffer.wrap(message.getBytes());
theBuffer.put(message.getBytes());
theBuffer.flip();
((SocketChannel) key.channel()).write(theBuffer);
theBuffer.flip();
log.debug("Sent message:" + decoder.decode(theBuffer));
key.interestOps(key.interestOps() | SelectionKey.OP_READ);
key.interestOps(key.interestOps() & (~SelectionKey.OP_WRITE));
} catch (IOException e) {
key.cancel();
}I verify what I send by outputting with the line log.debug("Sent message:" + decoder.decode(theBuffer));The messages are strings which are being taken from a LinkedList. The chunks I'm sending are of size 4096 bytes.
I forgot to mention that the client uses simple IO (blocking mode). In between the sendings from the server it receives zeros and considers them as part of the message.
I am using Windows 2000 + JDK 1.4.2_01 -
NIO problems, no answer from aljazeera
I'm sending a HTTP request (shown below) to www.aljazeera.net
GET /NR/EXERES/0065325B-D83C-43BB-8B1A-DD15E0A05608.htm HTTP/1.1
Host: www.aljazeera.net
and it hangs indefinitely.
The exact same code works for a bbc news site, returning me the webpage I want. My code is posted below, thanks in advance for your advice. This code is the run method of a thread which has "link" set to " www.aljazeera.net/NR/EXERES/0065325B-D83C-43BB-8B1A-DD15E0A05608.htm" when this code is run.
try
SocketChannel sChannel = SocketChannel.open();
sChannel.configureBlocking(false);
URL url = new URL(link);
// System.out.println(url.toString() + " is the link");
sChannel.connect(new InetSocketAddress(url.getHost(), 80));
while (!sChannel.finishConnect()) {
if(this.interrupted())
return;
ByteBuffer buf = ByteBuffer.allocateDirect(1024);
CharBuffer cBuf = CharBuffer.allocate(1024);
//System.out.println("request " + url.getQuery());
String s = "GET "+ url.getPath() + " HTTP/1.1\r\n";
s += "Host: " + url.getHost() + "\r\n\r\n";
System.out.println(s);
Charset charset = Charset.defaultCharset();
CharsetEncoder encoder = charset.newEncoder();
CharsetDecoder decoder = charset.newDecoder();
sChannel.write(encoder.encode(CharBuffer.wrap(s)));
String resp = "";
while (sChannel.read(buf) != -1) {
if(this.interrupted())
return;
buf.flip();
decoder.decode(buf,cBuf,false);
cBuf.flip();
resp += cBuf;
buf.clear();
cBuf.clear();
System.out.println(resp);
sChannel.close();
catch(Exception e)
e.printStackTrace();
System.exit(0);
}sChannel.configureBlocking(false);
while (!sChannel.finishConnect()) {
if(this.interrupted())
return;
} Why not just connect in blocking mode with a timeout?
sChannel.write(encoder.encode(CharBuffer.wrap(s)));Here you are ignoring the write-count returned by the write() method. You have no assurance that anything got written at all, let alone the complete request string. -
This problem has only appeared in the past few days before that I had nio problems with itunes. Firstly I discovered that I could only open Itunes once after reboot - thereafter it would not reopen until I reboot again. I have spent considerable hours yesterday talking to the Apple help by phone and tried all there suggestions including of course reinstalling itunes amongst other things. Eventually I thought the problem had been solved when I discovered that I could suddenly open Itunes repeatedly without rebooting. After a short time I then discovered that the reason it was working agai was that I had rebooted with the internet disconnected and itunes would open repeatedly as normal provide I left the internet disconnected. When I reconnect to internet the problem resurfaces. HELP
For those still wondering how to install ios 6.1 on 5th gen iPod, you need to download it off of apples official website and then shift click restore from backup and choose the downloaded ios 6.1
apples one year warrant covers any hardware problems like these right? -
EAL replicated Essbase cube empty
Hi,
I am trying to build a replicated Essbase cube using EAL from HFM, all 11.1.2.2. The EAL bridge shows green across the board, so all good. When I build a transparent cube, I can see the data in Essbase, nio problems. Now I am trying to build a replicated cube, using the same region and Essbase grid definition as for the transparent. The outline gets built fine, and when I go to retrieve, it does the retrieve, shows the status bar, and eventually comes back with Successful.
However when I look at the replicated cube there is no data in it, no blocks were created, nothing. When I look in the EAL log, there are no errors, the last entry refers to Extract Region with a list of the dimensions, nothing more. I tried using both Embedded and a specific APS URL in the Essbase cube link in EAL. Like I said the transparent cube is fine, so I believe the region and grid definitions are valid, so I cannot understand why the data is not in the replicated cube.
Anyone has experienced anything similar and might have some hints?Could the level 0 data have been zeroed out, and no agg run to propagate the zeros (or #Missing) to upper levels?
Did you at one time have some data in the database, then lock n send to overwrite?
If you had performed a database reset, that would leave no page file. If you simply overwrote data, then you could have some fragmentation (when data is overwritten, the data is written to a new space in the page file with the index pointing to new data. This leaves the prior data behind with no index pointer to it, but it's still in the page file.)
Was the export file of all data, or level zero data?
If it was level 0 export, perform an export all (3mb is not that much) to verify if there is any data. If not, I'd chalk it up to fragmentation.
Robert -
I have Roadrunner hooked to a WRT54GS LinkSys router. As host I can surf the web with nio problem. Although the computer I'm sharing is a Dell B110. They can sign on to AOL the provider of choice. Although they have an icon that there connected. They can't surf at all, no Eplorer either. All connections are also ethernet, Can some tell me what to look for that would solve this situation?
As you are able to go online through the router on one of the comps that means the router is configured properly.Even on the 2nd comp you are able to sign on to AOL which indicates that you are connected to the internet.If you cannot see the web pages try this : Open IE, go to internet options, click delete cookies , delete files. Then go to connections tab.At the bottom you would have LAN settings.Click it and then you would find a new window.No check box should be checked on this window.Click OK and then OK again.Then type the website address
-
NIO: Strange problem when using ByteBuffer with non-blocking SocketChannel
Hi,
I have a server that uses multiplexed, non-blocking I/O with java.nio. When a client connects, the server waits for the message: <system cmd="knock"/>, returns a message and disconnects the client. The clients are shortly serviced in less than a second.
But the server newer receive anything from about 20% of the clients - even though it is sent. Or with other words: it is received and the data is contained in the ByteBuffer - SocketChannel.read(ByteBuffer) - but a call to ByteBuffer.remaing() returns 0 !!
ByteBuffer receiveBuf = ByteBuffer.allocate(65536);
receiveBuf.clear(); // the code is elsewhere used for longer living clients
int readBytes = channel.read(receiveBuf);
receiveBuf.flip();
StringBuffer sb = new StringBuffer();
System.out.println(" * Remaining: "+receiveBuf.remaining()); // writes: ' * Remaining: 0'
System.out.println(" * Received: "+new String(receiveBuf.array())); // writes: ' * Received: <system cmd="knock"/>'
while(receiveBuf.remaining() >= 2) {
byte b = receiveBuf.get();
sb.append((char)b);
System.out.println(" * sb content: "+sb.toString()); // writes: ' * sb content: 'The ByteBuffer clearly receives the correct data, but the ByteBuffer.remaining() returns 0 and therefore the StringBuffer will never have any content.
The problem seems to occur randomly and for about 20% of the clients (simulated from the same computer and therefore has the same bandwidth and so on).
Anyone knows what is going on, and how to solve the problem !?It's always possible in any read that the number of bytes read is less than the number of bytes requested. You need to keep reading until you have got everything you expected, and cope with every possible error condition on each iteration.
EJP -
NIO - Selector problem - when using more than one in same application
Hi,
I'm developing a multiplayer-server, which shall be able to handle at least 2000 concurrent clients. To avoid the use of 4000 threads for 2000 clients (a thread for each stream), I would like to use NIO.
Problem:
The server has of course a ServerSocketChannel registered with a Selector, and when clients connects, the SocketChannel's is received. IF these channels is registered (OP_READ) with the same Selector-instance as the ServerSocketChannel, there is no problem and the Selector recognizes when a registered SocketChannel is ready for OP_READ - BUT when the received SocketChannels is registered with another Selector-instance than the one the ServerSocketChannel is registered with, the Selector NEVER recognizes, when a SocketChannel is ready for OP_READ - why not and how do I solve the problem? Only one thread can service the same Selector-instance, and when both receiving many connections and read/write, this could easily be a bottleneck
Following is the used code; 2 classes: ClientListener (has a ServerSocketChannel registered with a Selector) and ClientHandler (has another Selector where the SocketChannels is registered and a thread-pool that services all the SocketChannels, when they are ready for read/write):
public class ClientListener {
private static final int BACKLOG = 32;
private int port;
private Thread internalThread;
private volatile boolean noStopRequested;
private ServerSocketChannel ssc;
private static Selector selector;
private ClientHandler clientHandler;
public ClientListener(ClientHandler clientHandler, String bindAddress, int port) throws InternalErrorException {
this.clientClass = clientClass;
this.clientHandler = clientHandler;
this.noStopRequested = true;
this.port = port;
try {
InetAddress inetAddress;
if (bindAddress.equals(""))
inetAddress = InetAddress.getLocalHost();
else
inetAddress = InetAddress.getByName(bindAddress);
ssc = ServerSocketChannel.open();
ssc.socket().bind(new InetSocketAddress(inetAddress, port), BACKLOG);
ssc.configureBlocking(false);
Runnable r = new Runnable() {
public void run() {
try {
start();
} catch (Exception e) {
Log.echoError("ClientListener: Unexpected error: "+e.getMessage());
internalThread = new Thread(r, "ClientHandler");
internalThread.start();
} catch (Exception e) {
throw new InternalErrorException(e.getMessage());
public static Selector selector() {
return selector;
private void start() {
Log.echoSystem("ClientListener: Listening started at port "+port);
try {
selector = Selector.open();
SelectionKey acceptKey = ssc.register(selector, SelectionKey.OP_ACCEPT);
int keysAdded;
while (noStopRequested) {
SelectionKey key = null;
keysAdded = selector.select(10000);
if (keysAdded == 0)
continue;
Set readyKeys = selector.selectedKeys();
Iterator i = readyKeys.iterator();
while (i.hasNext()) {
try {
key = (SelectionKey)i.next();
i.remove();
if (key.isAcceptable()) {
ServerSocketChannel nextReady = (ServerSocketChannel)key.channel();
SocketChannel sc = nextReady.accept();
try {
clientHandler.registerClient(sc);
} catch (Exception e) { e.printStackTrace(); }
} catch (CancelledKeyException cke) {
System.out.println("ClientListener: CancelledKeyException");
} catch (Exception e) {
e.printStackTrace();
try {
ssc.close();
} catch (IOException ioe) {/* Ignore */}
Log.echoSystem("ClientListener: stopped");
public void stop() {
public class ClientHandler {
private static final int INITIAL_WORKER_COUNT = 5;
private int port;
private Thread internalThread;
private volatile boolean noStopRequested;
private static Selector selector;
private Manager manager;
private WorkerPool pool;
private ClientListener clientListener;
public ClientHandler(Manager manager, String bindAddress, int port) throws InternalErrorException {
this.manager = manager;
this.noStopRequested = true;
this.port = port;
clientListener = new ClientListener(this, bindAddress, port);
// initiating load-balanced worker-pool
pool = new WorkerPool(INITIAL_WORKER_COUNT, (int)(manager.getMaxClients() * ClientWorker.CLIENT_FACTOR), 0.75f);
for (int i=0; i < pool.getMinCount(); i++) {
pool.addWorker(new ClientWorker(pool, manager));
Runnable r = new Runnable() {
public void run() {
try {
start();
} catch (Exception e) {
Log.echoError("ClientHandler: Unexpected error: "+e.getMessage());
internalThread = new Thread(r, "ClientHandler");
internalThread.start();
public static Selector selector() {
return selector;
private void start() {
Log.echoSystem("ClientHandler: Started");
try {
selector = Selector.open();
int keysAdded;
while (noStopRequested) {
SelectionKey key = null;
try {
keysAdded = selector.select();
Log.echoDebug("ClientHandler: out of select()");
if (keysAdded == 0)
continue;
Set readyKeys = selector.selectedKeys();
Iterator i = readyKeys.iterator();
while (i.hasNext()) {
try {
key = (SelectionKey)i.next();
i.remove();
if (key.isReadable()) {
key.interestOps(key.interestOps() & (~SelectionKey.OP_READ));
ClientWorker worker = (ClientWorker)pool.getWorker();
worker.process(key);
} else if (key.isWritable()) {
key.interestOps(key.interestOps() & (~SelectionKey.OP_WRITE));
ClientWorker worker = (ClientWorker)pool.getWorker();
worker.process(key);
} else {
} catch (CancelledKeyException cke) {
Client client = (Client)key.attachment();
if (client != null) {
client.disconnect();
key.selector().wakeup();
} catch (CancelledKeyException cke) {
if (key != null) {
Client client = (Client)key.attachment();
if (client != null) {
client.disconnect();
key.selector().wakeup();
} catch (Exception e) {
e.printStackTrace();
Log.echoSystem("ClientHandler: stopped");
public void registerClient(SocketChannel sc) throws Exception {
sc.configureBlocking(false);
Client client = new Client(...);
SelectionKey key = sc.register(selector, SelectionKey.OP_READ, client);
key.selector().wakeup();
public void stop() {
}Ok, found the solution here: http://forum.java.sun.com/thread.jsp?forum=31&thread=282069
"The select() method holds a lock on the selector that
the register() method wants to acquire. The register()
method cannot continue until the lock is relased and
the select() method will not release it until some key
is triggered. If this is the first key you're adding
then select has nothing that will wake it up, ever.
The result is, obviously, a deadlock.
The solution is to have a queue of pending connections.
You put your new connection in that queue. Then you
wake up the selector and let that thread go through
the queue, registering connections." -
Hello there,
I've got a odd problem here, and it's going to be hard to explain so please excuse me if I haven't explained correctly. I'm writing a Java application, which uses nio, with non blocking sockets, to handle its networking. I presumably have a problem with writing data. Quick recap of the problem is that for some reason, data gets concatenated to each other and then sent. So for example, let's say I'd want to send 'hello' and 'there'. What my application sometimes does, is sending 'hellothere' instead of sending 'hello', and then, in a new packet, 'there'.
What now happens on the server side is that it receives 'hellothere' and doesn't know what do with it, while it would have worked fine if they were separated.
This is a snippet coming from the server. This is always running on background (in a thread), in a loop.
synchronized (m_ChangeRequests)
Iterator changes = m_ChangeRequests.iterator();
while (changes.hasNext())
ChangeRequest change = (ChangeRequest)changes.next();
switch (change.getType())
case ChangeRequest.CHANGEOPS:
SelectionKey key = change.getSocket().keyFor(m_Selector);
try
key.interestOps(change.getOps());
catch (NullPointerException e)
disconnectClient(getClientWithChannel(change.getSocket()));
m_ChangeRequests.clear();
// Waiting for events
m_Selector.select(1000);
// Get keys
Set keys = m_Selector.selectedKeys();
Iterator i = keys.iterator();
// For each keys...
while(i.hasNext())
SelectionKey key = (SelectionKey) i.next();
// Remove the current key
i.remove();
// if isAccetable = true
// then a client required a connection
if (!key.isValid())
continue;
if (key.isAcceptable())
accept(key);
else if (key.isReadable())
read(key);
else if (key.isWritable())
write(key);
}Now, I suppose the important method here is write().
private void write(SelectionKey key)
SocketChannel socketChannel = (SocketChannel)key.channel();
synchronized (m_PendingData)
List queue = (List)m_PendingData.get(socketChannel);
while (!queue.isEmpty())
try
ByteBuffer buf = (ByteBuffer)queue.get(0);
System.out.println(socketChannel.write(buf));
if (buf.hasRemaining())
continue;
queue.remove(0);
catch (IOException ioe)
ioe.printStackTrace();
if (queue.isEmpty())
key.interestOps(SelectionKey.OP_READ);
As you can see I'm using a variable m_PendingData there, which is simply a HashMap. There is another important method, which is the send method.
public void send(SocketChannel socket, byte[] data)
synchronized (m_ChangeRequests)
m_ChangeRequests.add(new ChangeRequest(socket, ChangeRequest.CHANGEOPS, SelectionKey.OP_WRITE));
synchronized (m_PendingData)
List queue = (List)m_PendingData.get(socket);
if (queue == null)
queue = new ArrayList();
m_PendingData.put(socket, queue);
queue.add(ByteBuffer.wrap(data));
m_Selector.wakeup();
}You might have noticed the m_ChangeRequests variable. Please see the first code snippet for what it does. It's a LinkedList.
Sorry if I have not explained it clear enough. I suppose this problem could also be in the read method, I assume it is in the write method though.
Thanks,
LarsBasically you can't think of reading and writing to/from sockets in terms of packets - you write some bytes to the socket at one end, and read some bytes at the other. This is true for both blocking and non blocking sockets.
If you want your bytes to be split into meaningful packets, then you have to encode the packet format yourself. A really simple way to do that is to start each message with a fixed number of bytes that contain the number of data bytes in the packet - from your example this would give:
5 hello 5 there
On the reading end, your server will then be able to read the initial byte count of each packet and know how much data is expected. -
Nio write problem: server data sent isn't fully read by client
Hi everyone,
still writing away with my nio server and once again have run into
some problems. Essentially my main problem is that when the server
writes to the client, the write appears to output all the bytes in the
write operation, but the client never accepts them all, even if a
buffer has been manually allocated to the correct size of the data.
As background my server will accept connections. When a connection
is established I register OP_READ on the key. When a OP_READ trigger
occurs the server accepts the clients request, processes it, then
attaches the output to the SelectionKey as a ByteBuffer that has
already been encoded. At this point I then register OP_WRITE on that
SelectionKey (as a side note i'm running this server on XP and had
read that registering OP_READ and OP_WRITE on the same selector was
bad, but then it looked like there was a work around to this) and wait
for an OP_WRITE trigger.
When an OP_WRITE occurs on that key I run a new method (with heavy
influences from the thread: http://forum.java.sun.com/thread.jsp?forum=11&thread=530825 and the taming the nio circus thread) which will grab the attachment and attempt to send it. The code has been written that IF the send cannot complete it should re-attach the remaining bytebuffer to the key and wait for another OP_WRITE to occur so it can send the remainder.
The problem is that whenever I write (and for this test the amount im writing is approx 10100 bytes) the server appears to send it all (by checking the int returned from socketchannel.write()), but at the client end it never reads all the data that is sent.
If i'm using a blocking socket client, then I get a java.net.SocketException: Connection Reset exception, whilst if i'm using a non-blocking client, I get no exception, just not the whole amount of data, even when i've statically allocated a receiving bytebuffer that is big enough.
The following code is a class that is used to do the writing from the server:
/* code for nio write model referenced from:
* http://forum.java.sun.com/thread.jsp?forum=11&thread=530825
class NIOWriteHandler {
private ByteBuffer sendBuffer;
private SelectionKey localSelectionKey;
NIOWriteHandler(SelectionKey currentKey) {
localSelectionKey = currentKey;
public void doWrite() {
localSelectionKey.interestOps(SelectionKey.OP_READ); //deselect write,
sendBuffer = (ByteBuffer)localSelectionKey.attachment();
// perform the writing
SocketChannel writingChannel = (SocketChannel)localSelectionKey.channel();
if (writingChannel.isOpen()) {
int len = 0;
if (sendBuffer.hasRemaining()) {
try {
System.out.println("Sending chunks o data");
len = writingChannel.write(sendBuffer);
System.out.println("value of len: " + len);
} catch (IOException ioe) {
ioe.printStackTrace();
// call close method
System.out.println("CLOSE INVOKED at POINT 8");
closeComplete(localSelectionKey);
System.out.println("Second IF coming...");
if (sendBuffer.hasRemaining()) {
// if we get here then the previous write did not fully
// complete, so need to save data etc
System.out.println("Couldn't send all data this time...");
localSelectionKey.interestOps(SelectionKey.OP_WRITE|SelectionKey.OP_READ);
localSelectionKey.attach(sendBuffer);
} else {
sendBuffer = null;
closeComplete(localSelectionKey);
// write complete at this stage
}This is the basic block client that is incredibly dumb:
import java.net.*;
import java.util.*;
import java.io.*;
import java.nio.charset.*;
import java.nio.channels.*;
import java.nio.*;
import java.nio.ByteBuffer;
public class ServerTest {
/* args 0 - the IP of the machine to connect to
args 1 - the port number
Simple client that connects to a specified IP & port, takes a line
of input via stdin, sends that to the connected machine and prints
out the response.
Error handling and such isn't accounted for.
public static void main (String args[]) throws Exception{
Socket socket = new Socket(args[0], Integer.parseInt(args[1]));
BufferedReader buffRead = new
BufferedReader(new InputStreamReader((socket.getInputStream())));
PrintStream ps =
new PrintStream(socket.getOutputStream());
Charset charset = Charset.forName("ISO-8859-1");
BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in));
System.out.println("[CLIENT]Data to send: ");
String data = stdin.readLine();
ps.println(data);
String returned = buffRead.readLine();
while (returned != null) {
System.out.println(returned);
returned = buffRead.readLine();
System.out.println("[CLIENT]End server response");
buffRead.close();
ps.close();
socket.close();
}And here is the non-blocking basic client (which dosn't actually close at the moment):
import java.net.*;
import java.util.*;
import java.io.*;
import java.nio.charset.*;
import java.nio.channels.*;
import java.nio.*;
public class ServerTestNonBlock {
/* args 0 - the IP of the machine to connect to
args 1 - the port number
Simple client that connects to a specified IP & port, takes a line
of input via stdin, sends that to the connected machine and prints
out the response.
Error handling and such isn't accounted for.
public static void main (String args[]) throws Exception{
InetSocketAddress addr = new InetSocketAddress
(args[0], Integer.parseInt(args[1]));
SocketChannel sc = SocketChannel.open();
sc.configureBlocking(false);
Selector selector = Selector.open();
System.out.println("Starting connection...");
sc.connect(addr);
while(!sc.finishConnect()) {
System.out.println("1,2,3,4 I will keep on counting...");
System.out.println("Connection established..");
Charset charset = Charset.forName("ISO-8859-1");
CharsetEncoder encoder = charset.newEncoder();
sc.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
while (true) {
int n = selector.select();
if (n==0) {
continue;
Set keys = selector.selectedKeys();
Iterator it = keys.iterator();
while (it.hasNext()) {
SelectionKey selKey = (SelectionKey)it.next();
if (selKey.isReadable()) {
// time to setup read
ByteBuffer incomingData =
ByteBuffer.allocateDirect(102000);
incomingData.clear();
int count;
while ((count = sc.read(incomingData)) > 0) {
System.out.println("Value of count: " + count);
// reading the data
System.out.println("Count value: " + count);
int pos = incomingData.position();
incomingData.flip();
CharBuffer content = charset.decode(incomingData);
String inData = content.toString();
System.out.println(inData.trim());
System.out.println("[CLIENT]End server response");
System.out.println("Count value: " + count);
System.out.println("Position value: " + pos);
//sc.close();
//break;
if (selKey.isWritable()) {
BufferedReader stdin = new BufferedReader
(new InputStreamReader(System.in));
System.out.println("[CLIENT]Data to send: ");
String data = stdin.readLine();
ByteBuffer byteBufferOut = encoder.encode
(CharBuffer.wrap(data));
int length = sc.write(byteBufferOut);
System.out.println("Wrote: " + length + " bytes.");
selKey.interestOps(SelectionKey.OP_READ);
}I'm kinda stuck at the moment and am making change for the sake of change without getting a good grasp of what is going on. If anyone can provide any help that'd be fantastic. If in the mean time I figgure something out i'll post a response.
If you've gotten this far thanks a bunch for reading :)
Cheers,
PeteHi Meesum,
thanks for the reply :)
I'm not convinced this is the error - as i've got two clients listed there, and the odd behaviour from both is that neither is getting through to the last of the data that is sent.
If the null were the problem (which is only checked for in the basic dumb blocking client) i'd be expecting some sort of infinite loop or wait for more data from the server, not an abnormal termination (ala connection reset) from the server. I'll give it a shot anyhow, but I know that under a blocking write operation that that code worked fine.
Thanks again though for the post, has got some of my cogs slowly turning :)
Cheers,
Pete -
I have written an NIO server which gets data from a Static queue and sends it to the clients... (this is my intention.. but the server doesn't)...
The problem is the server sends data to one client... and if another client is connected, the server sends data to the latest client and stops sending data to the older client...
But the printouts says that it has sent data to two clients...
I couldn't figure out the problem.. can anyone help me..
I have posted the server code here...
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.nio.channels.spi.SelectorProvider;
import java.util.Iterator;
public class StreamingServer implements Runnable{
// The channel on which we'll accept connections
private ServerSocketChannel serverChannel;
// The selector to be monitored for events
private Selector selector;
int numRead;
//public static boolean isAvailable = false;
int write = 0;
public int socInit(int port){
try{
this.selector = this.initSelector(port);
System.out.println("Streaming engine started... " + port);
return 1;
catch(Exception e){
return -1;
private Selector initSelector(int port) throws IOException {
// Create a new selector
Selector socketSelector = SelectorProvider.provider().openSelector();
// Create a new non-blocking server socket channel
this.serverChannel = ServerSocketChannel.open();
serverChannel.configureBlocking(false);
// Bind the server socket to the specified address and port
InetSocketAddress isa = new InetSocketAddress(port);
serverChannel.socket().bind(isa);
System.out.println("Streaming server bound to port ---> " + isa.getHostName() + " : " + isa.getPort());
// Register the server socket channel, indicating an interest in
// accepting new connections
serverChannel.register(socketSelector, SelectionKey.OP_ACCEPT);
return socketSelector;
public synchronized void run() {
while (true) {
try {
//System.out.println("Streaming server Waiting for an event...");
// Wait for an event one of the registered channels
this.selector.select();
// Iterate over the set of keys for which events are available
Iterator selectedKeys = this.selector.selectedKeys().iterator();
int count = 0;
write = 0;
ByteBuffer readBuffer = null;
int queueSize = InternalEngine.queue.size();
if( queueSize >= 1){
readBuffer = (ByteBuffer)InternalEngine.queue.get(queueSize - 1);
InternalEngine.queue.removeElementAt(queueSize - 1);
else{
Thread.sleep(10);
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
if (!key.isValid()) {
System.out.println("This key is invalid...");
continue;
// Check what event is available and deal with it
if (key.isAcceptable()) {
System.out.println("Accepting Connection...");
this.accept(key);
else if(key.isWritable()){
System.out.println("Key is writable ...");
if(readBuffer != null){
System.out.println("Readbuffer has data... Writing to client...");
//write the data
write(key,readBuffer);
count ++;
}else{
System.out.println("No data in readBuffer... NOT writing to client");
} //End of inner While
/*System.out.println("clients connected ---> " + count);
System.out.println("Written to clients --> " + write);*/
} catch (Exception e) {
e.printStackTrace();
} // End of outer while
* @param key
* @throws IOException
private void write(SelectionKey key, ByteBuffer readBuffer) throws IOException {
SocketChannel socketChannel = (SocketChannel) key.channel();
System.out.println("socketChannel assigned ---> " + socketChannel.socket().getRemoteSocketAddress());
System.out.println("Scoket channel is connected ? " + socketChannel.socket().isConnected());
// Attempt to write to the channel
try {
while(readBuffer.hasRemaining()){
System.out.println(socketChannel.write(readBuffer) + " bytes written to client");
write ++;
} catch (Exception e) {
// The remote forcibly closed the connection, cancel
// the selection key and close the channel.
e.printStackTrace();
key.cancel();
socketChannel.close();
//return;
private void accept(SelectionKey key) throws IOException {
// For an accept to be pending the channel must be a server socket channel.
ServerSocketChannel serverSocketChannel = (ServerSocketChannel) key.channel();
// Accept the connection and make it non-blocking
System.out.println("Waiting for connections ... ");
SocketChannel socketChannel = serverSocketChannel.accept();
//Socket socket = socketChannel.socket();
socketChannel.configureBlocking(false);
// Register the new SocketChannel with our Selector, indicating
// we'd like to be notified when there's data waiting to be read
System.out.println("Registering OP_WRITE after OP_ACCEPT");
socketChannel.register(this.selector, SelectionKey.OP_WRITE);
}Well, it's your code!not exactly... its an example code found when googling and I made some changes...
This code will write whatever comes off the queue to
whichever SocketChannel is writable next. Surely
that's not what you meant? Surely there's something
in the queue data that tells you where it should be
written to?my intention is to write whatever comes off the queue to all the clients that has been connected...
Normally selectiing on OP_WRITE is a bad idea as it
is almost always 'ready'. Normally you just write to
the required channel and only set OP_WRITE if you get
a zero length write. When OP_WRITE triggers for that
channel you then repeat the write and if it succeeds
you then clear OP_WRITE.I don't get this thing into my brain.. I am a newbie for NIO...
what I assume is that
-- I shouldn't register for OP_WRITE at all in the first thing...
-- I should write to clients whichever comes in... without registering the channel for OP_WRITE
am I right?
registering and clearing OP_WRITE is killing me... Could you please make changes in the code so that it can register and clear OP_WRITE and write to the clients...
See
http://forum.java.sun.com/thread.jspa?threadID=459338
for a very complete discussion.I have seen this discussion and tried to understand what's what... but its a bit too complicated for a newbie like me... -
Problems with nio decoder need help plz
I want to make a flash/php chat Client with Java Server with the new nio.
Problem is to Convert a Bytebuffer into a normal string.
I done this with the charsetdecoder and it works...
if i give the string with system.println out it will be the right string. but i cant use the string with the if method..
my source:
php:
$nick="Nerrik";
$host = "localhost";
$data="-".$nick.";$REMOTE_ADDR";
$fp = fsockopen($host,1120);
fwrite($fp, $data,strlen($data));
fclose($fp);
java:
import java.io.*;
import java.net.*;
import java.util.*;
import java.nio.*;
import java.nio.channels.*;
import java.nio.charset.*;
public class newserver
Charset charset = Charset.forName("ISO-8859-1");
CharsetDecoder decoder = charset.newDecoder();
CharsetEncoder encoder = charset.newEncoder();
ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
CharBuffer charBuffer = CharBuffer.allocate(1024);
String[] user=new String[100];
String[] user2=new String[100];
private Selector readSelector;
public void start() throws Exception
System.out.println("Server Gestartet...");
System.out.println("Protokoll aktiv");
SocketChannel client;
readSelector=Selector.open();
ServerSocketChannel channel=ServerSocketChannel.open();
ServerSocket socket=channel.socket();
socket.bind(new InetSocketAddress ("localhost",1120));
channel.configureBlocking(false);
while (true)
client=channel.accept();
if (client!=null)
client.configureBlocking(false);
client.register(readSelector,SelectionKey.OP_READ);
handleClientRequest();
public void handleClientRequest() throws Exception
Set keys;
Iterator it;
SelectionKey key;
SocketChannel client;
if (readSelector.selectNow()>0)
keys=readSelector.selectedKeys();
it=keys.iterator();
while(it.hasNext())
key=(SelectionKey)it.next();
if(key.isReadable())
client=(SocketChannel)key.channel();
buffer.clear();
charBuffer.clear();
client.read(buffer);
buffer.flip();
decoder.decode(buffer, charBuffer, false);
charBuffer.flip();
String em = charBuffer.toString();
String aktion = em.substring(0,1);
if(aktion=="-")
//this don't come and em.substring(0,1) is -
StringTokenizer st = new StringTokenizer(em,";");
System.out.println("New Client:"+st.nextToken()+" on IP:"+st.nextToken());
client.close();
public static final void main(String args[]) throws Exception
(new newserver()).start();
sorry for my bad bad english ;)The code segment...
if(aktion=="-")
//this don't come and em.substring(0,1) is -
StringTokenizer st = new StringTokenizer(em,";");
System.out.println("New Client:"+st.nextToken()+" on IP:"+st.nextToken());
will always return false. You are comparing the object reference of aktion to "-". The code should read:
if(aktion.equals("-"))
//this don't come and em.substring(0,1) is -
StringTokenizer st = new StringTokenizer(em,";");
System.out.println("New Client:"+st.nextToken()+" on IP:"+st.nextToken());
The reason why it works when you hard code the values is because the Java comiler will only keep one object for each String literal. Thus:
String aTest = "test";
aTest=="test"; // This will return true but
String aTest = getStringFromWhereElseThatIsTest();
aTest=="test"; // Will most likely return false
- Chris -
Hi guys,
I experience very strange problem. Once a new connection is established and the key for the channel is registered for READ operations, the selector starts to continuosly provide me with the key having OP_READ set up:
if (selector.select() == 0)
continue;
Iterator<SelectionKey> it = selector.selectedKeys().iterator();
while (it.hasNext())
SelectionKey key = (SelectionKey) it.next();
// Accept new connection
if (key.isAcceptable())
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept();
if (channel == null)
continue;
// Set the new channel nonblocking
channel.configureBlocking(false);
// Register it with the selector
SelectionKey newKey = channel.register(selector, SelectionKey.OP_READ);
// initialization of attachment ...
if (key.isReadable())
System.out.print("session index = " +((ConnectionDescriptor)key.attachment()).session.sessionIndex);+
+System.out.print(", counter "+ ++counter);+
+System.out.println(", ops="+ key.readyOps());
// read the data from channel
if( counter == 10 ) {
setEnabled(false);
break;
it.remove();
}I use a python script in order to test the server:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect("localhost", 28000)
s.send(b'\x00\x01')
s.close()When I run the script I see the following output from the server:
session index = 1, counter 1, ops=1
session index = 1, counter 2, ops=1
session index = 1, counter 3, ops=1
header limit = 2
body limit = 0
process message 1
session index = 1, counter 4, ops=1
session index = 1, counter 5, ops=1
session index = 1, counter 6, ops=1
session index = 1, counter 7, ops=1
session index = 1, counter 8, ops=1
session index = 1, counter 9, ops=1
session index = 1, counter 10, ops=1some parts of the code are omitted in order to keep it clear. If I do not stop it at 10th iteration it runs into endless loop.
Why the key always says that the channel is "ready" to be read while I actually have already read the data?
Thanks a lot in advance.
Edited by: coderman on Jan 28, 2010 5:44 AMThis is an example class you can reproduce the issue with:
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.nio.ByteBuffer;
import java.nio.channels.ClosedChannelException;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Collection;
import java.util.Iterator;
public class MyServer
private Selector selector;
private int sessionSequence;
public MyServer() throws IOException
final ServerSocketChannel mainServerChannel = ServerSocketChannel.open();
ServerSocket mainSocket = mainServerChannel.socket();
mainSocket.bind(new InetSocketAddress(28000));
mainServerChannel.configureBlocking(false);
selector = Selector.open();
mainServerChannel.register(selector, SelectionKey.OP_ACCEPT);
int counter = 0;
while (true)
if (selector.select(1000) == 0)
continue;
Iterator<SelectionKey> it = selector.selectedKeys().iterator();
while (it.hasNext())
SelectionKey key = (SelectionKey) it.next();
if (key.isAcceptable())
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept();
if (channel == null)
continue;
channel.configureBlocking(false);
SelectionKey newKey = channel.register(selector, SelectionKey.OP_READ );
if (key.isReadable())
System.out.print("counter " + counter);
System.out.print(", ops=" + key.readyOps());
ByteBuffer header = ByteBuffer.allocateDirect(10);
try
((SocketChannel) key.channel()).read(header);
} catch (ClosedChannelException cce)
// the channel has been closed by the server
} catch (IOException ex)
// networking problem. Might be also connection lost.
header.flip();
System.out.println(", header limit = " + header.limit());
if (++counter == 15)
System.exit(0);
it.remove();
public static void main(String[] argv)
try
new MyServer();
} catch (IOException ex)
ex.printStackTrace();
}Here is the python script:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("localhost", 28000))
s.send(b'\x00\x01\x02')
s.shutdown(1)This is what I get on my computer:
counter 0, ops=1, header limit = 3
counter 1, ops=1, header limit = 0
counter 2, ops=1, header limit = 0
counter 3, ops=1, header limit = 0
counter 4, ops=1, header limit = 0
counter 5, ops=1, header limit = 0
counter 6, ops=1, header limit = 0
counter 7, ops=1, header limit = 0
counter 8, ops=1, header limit = 0
counter 9, ops=1, header limit = 0
counter 10, ops=1, header limit = 0
counter 11, ops=1, header limit = 0
counter 12, ops=1, header limit = 0
counter 13, ops=1, header limit = 0
counter 14, ops=1, header limit = 0
Maybe you are looking for
-
How to install Boot Camp on Win/XP SP3?
Is Boot Camp even supported on SP3? I can't run the BC that's on the Leopard install DVD (blue screen of death), nor can I run the BCUpdateXP.exe on Apple's website (doesn't launch). I've looked at the other threads here but none addresses this issue
-
Manual BP creation in Interaction center
Hi all, We got a new requirement stating, that we need to create a bp with basic details(address and bank account details) manually in interaction center. Manually means, user will give the bp and we need to create a business partner with the same bp
-
What are the Basic Features of Enterprise Portal
What are the default, given or otherwise features of SAP netweaver/enterprise portal. I do not want to redo a function or feature/template of the feature/basic iview (portlet) already exists. And where I can find/get these features. Thanks.
-
Shipping Condition Type by Volume
Hi all for business requirement I have defined 2 new condition types in LE-Transportation shipment costspricingpricing control--define condition types: 1 Following is config setting for condition type by Volume wise : Cond. class: B Prices Calc
-
Reports for Database metriks: what is possible?
Hello, we are using the GridControl since several months. For our databases we are collecting several metrics and keep them for 365 days. I know how to look up the metrics for historic data (e.g. the first week in january), but for reporting purposes