Thread blocking on java.nio.charset.CoderResult

Hello all,
I have a multi-threaded app which does some fairly intestive string operations (basically extracts text from documents for indexing a search system).
I am seeing a massive bottleneck around the java.nio.charset.CoderResult class. When profiling, I see a whole stack of threads blocking on (waiting for) a monitor on this java.nio.charset.CoderResult class. Seems to be a result of string encoding/decoding (I am often encoding strings as UTF-8).
Anyone know why the JVM would want my threads to sync on this class? It's creating a huge performance issue for my app. Approximately 15% of ALL the processing time is spent waiting for this class.
Help!

I would guess that you're using some of the static methods in the CoderResult class. The static methods CoderResult.unmappableCache(), CoderResult.malformedForLength() and CoderResult.malformedCache all use a static inner class called Cache. Its get() method is synchronized on Cache.class. Since the Cache inner-class is static, any part of your multi-threaded application that goes through the Cache.get() method is going to be waiting for the lock on Cache.class.
Could you create a CoderResult instance for each thread? That would mean that there would be a different static Cache class for each thread, reducing the number of threads competing for the Cache.class lock.
I'd have to see some of your code to give a better answer.
Brian

Similar Messages

  • Looking for performance boost to java.nio.charset.encode(String) method

    hi ,
    i have run jprofiler on my application and so that this method takes 4% of the overall CPU time (!) .
    is there any other , faster way to do this encoding ?
    thanks!

    yjavaman wrote:
    i susspect that in my application (multi threaded ) , it is actually a larger bottle neck than i think right now.
    i ask about maybe the caching of the encoder , decoder , maybe i can save this time by doing other stuff .Not sure if you are already buffereing, but from javadoc:
    java.io.OutputStreamWriter
    Each invocation of a write() method causes the encoding converter to be invoked on the given character(s). The resulting bytes are accumulated in a buffer before being written to the underlying output stream. The size of this buffer may be specified, but by default it is large enough for most purposes. Note that the characters passed to the write() methods are not buffered.
    For top efficiency, consider wrapping an OutputStreamWriter within a BufferedWriter so as to avoid frequent converter invocations. For example:
    Writer out = new BufferedWriter(new OutputStreamWriter(System.out));
    Just something simple to use.

  • Javac compiler throws java.nio.BufferOverflowException!!Why?

    An exception has occurred in the compiler (1.4.1). Please file a bug at the Java Developer Connection (http://java.sun.com/cgi-bin/bugreport.cgi) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you.
    java.nio.BufferOverflowException
    at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
    at java.lang.StringCoding$CharsetSD.decode(StringCoding.java:186)
    at java.lang.StringCoding.decode(StringCoding.java:222)
    at java.lang.StringCoding.decode(StringCoding.java:228)
    at java.lang.String.<init>(String.java:383)
    at java.lang.String.<init>(String.java:404)
    at java.io.UnixFileSystem.list(Native Method)
    at java.io.File.list(File.java:914)
    at com.sun.tools.javac.v8.code.ClassReader.list(ClassReader.java:1224)
    at com.sun.tools.javac.v8.code.ClassReader.listAll(ClassReader.java:1320)
    at com.sun.tools.javac.v8.code.ClassReader.fillIn(ClassReader.java:1340)
    at com.sun.tools.javac.v8.code.ClassReader.complete(ClassReader.java:1049)
    at com.sun.tools.javac.v8.code.Symbol.complete(Symbol.java:332)
    at com.sun.tools.javac.v8.comp.Enter.visitTopLevel(Enter.java:467)
    at com.sun.tools.javac.v8.tree.Tree$TopLevel.accept(Tree.java:390)
    at com.sun.tools.javac.v8.comp.Enter.classEnter(Enter.java:442)
    at com.sun.tools.javac.v8.comp.Enter.classEnter(Enter.java:456)
    at com.sun.tools.javac.v8.comp.Enter.complete(Enter.java:588)
    at com.sun.tools.javac.v8.comp.Enter.main(Enter.java:574)
    at com.sun.tools.javac.v8.JavaCompiler.compile(JavaCompiler.java:334)
    at com.sun.tools.javac.v8.Main.compile(Main.java:520)
    at com.sun.tools.javac.Main.compile(Main.java:36)
    at com.sun.tools.javac.Main.main(Main.java:27)

    See this bug:
    http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4949631

  • Java.nio.BufferOverflowException on big Strings ( 87MB)

    Hi!
    When calling getBytes() on a big String the following exception is thrown:
    java.nio.BufferOverflowException
         at java.nio.charset.CoderResult.throwException(Unknown Source)
         at java.lang.StringCoding$CharsetSD.decode(Unknown Source)
         at java.lang.StringCoding.decode(Unknown Source)
         at java.lang.StringCoding.decode(Unknown Source)
         at java.lang.String.<init>(Unknown Source)
         at java.lang.String.<init>(Unknown Source)
         at com.db.janus.Janus.main(Janus.java:99)
    This happens as soon as the size of the String gets bigger than 92003848 bytes.
    I can reproduce it with the following code:
    String x = new String(new byte[92003848]); // --> OK
    String x = new String(new byte[92003849]); // --> Exception
    JDK Version: 1.4.2_06
    OS: Windows XP
    Is this a known bug / limitation? I found a similar bug related to Java on Linux / Solaris, but this one was about getBytes() with Strings > 16MB.
    Thanks for your help!

    Is there any alternative to solve this problem.I am getting the error
    java.nio.BufferOverflowException .This happens in my mail server when the file six\ze of xml where i have stored data about spam/nonspam grows upto 22mb.I have stored this xml in database as blob.at server startup uploading in in my class.Keeps througout in JVM after every 50 th request saving information back to database(about spam/nonspam).Now if xml size grows i am storing this in string to parse etc.it gives this bug error.Sholud i cahnge my datatype or use something as XML Parser to split my xml in pieces(mechanism i am not aware of at all) so that string just takes some size of xml (but here can i use string builder etc.). Now is there any proper solution to this kind of problem whenever i removes some daya from xml spam mail goes to inbox and vice versa.
    Please suggest me some solution if possible.
    complete error is:
    Servlet initialization failed:
    java.nio.BufferOverflowException
    at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
    at java.lang.StringCoding$CharsetSD.decode([BII)[C(Unknown Source)
             at java.lang.StringCoding.decode(Ljava.lang.String;[BII)[C(Unknown Source)
             at java.lang.String.<init>([BIILjava.lang.String;)V(Unknown Source)
             at java.lang.String.<init>([BLjava.lang.String;)V(Unknown Source)
             at zerocode.core.ByteBlock.toString(ByteBlock.java:89)
             at zcCollab.mailProcessing.NaiveBayesClassifier._getInstanceData(NaiveBayesClassifier.java:275)
             at zcCollab.mailProcessing.NaiveBayesClassifier.initialize(NaiveBayesClassifier.java:86)
             at zcCollab.mailProcessing.SimpleDeliveryManager._setupClassifiers(SimpleDeliveryManager.java:409)
             at zcCollab.mailProcessing.SimpleDeliveryManager.<init>(SimpleDeliveryManager.java:62)
             at zcCollab.zcApp.InfoManagerApp._createDeliveryProcessor(InfoManagerApp.java:307)
             at zcCollab.zcApp.InfoManagerApp.initialize(InfoManagerApp.java:294)
             at zerocode.udm.Application._createInstance(Application.java:168)
             at zerocode.udm.Application.createInstanceForServlet(Application.java:111)
             at zerocode.servlet.UdmServlet._createApplication(UdmServlet.java:67)
             at zerocode.servlet.UdmServlet._initialize(UdmServlet.java:52)
             at zerocode.servlet.GenericServlet.init(GenericServlet.java:116)
             at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:292)
             at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:329)
             at org.mortbay.jetty.servlet.ServletHandler.dispatch(ServletHandler.java:657)
             at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
             at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
             at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
             at org.mortbay.http.HttpServer.service(HttpServer.java:879)
             at org.mortbay.http.handler.ForwardHandler.handle(ForwardHandler.java:130)
             at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
             at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
             at org.mortbay.http.HttpServer.service(HttpServer.java:879)
    Thanks
    Vijendra                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

  • Java.nio select() method return 0 in my client application

    Hello,
    I'm developing a simple chat application who echo messages
    But my client application loop because the select() method return 0
    This is my code
    // SERVER
    package test;
    import java.io.IOException;
    import java.net.InetSocketAddress;
    import java.nio.channels.SelectionKey;
    import java.nio.channels.Selector;
    import java.nio.channels.ServerSocketChannel;
    import java.nio.channels.SocketChannel;
    import java.util.Iterator;
    import java.util.Set;
    public class Server {
         private int port = 5001;
         public void work() {               
              try {
                   ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
                   serverSocketChannel.configureBlocking(false);
                   InetSocketAddress isa = new InetSocketAddress(port);               
                   serverSocketChannel.socket().bind(isa);
                   Selector selector = Selector.open();
                   serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
                   System.out.println("Listing on "+port);
                   while(selector.select()>0) {
                        Set keys = selector.selectedKeys();
                        for(Iterator i = keys.iterator(); i.hasNext();) {
                             SelectionKey key = (SelectionKey) i.next();
                             i.remove();
                             if (key.isAcceptable()) {
                                  ServerSocketChannel keyChannel = (ServerSocketChannel)key.channel();                              
                                  SocketChannel channel = keyChannel.accept();
                                  channel.configureBlocking(false);                              
                                  channel.register(selector, SelectionKey.OP_READ );
                             } else if (key.isReadable()) {
                                  SocketChannel keyChannel = (SocketChannel) key.channel();
                                  String m = Help.read(keyChannel );
                                  Help.write(m.toUpperCase(), keyChannel );
              } catch (IOException e) {                                             
                   e.printStackTrace();                         
         public static void main(String[] args) {
              Server s = new Server();
              s.work();
    // CLIENT
    package test;
    import java.awt.event.KeyAdapter;
    import java.awt.event.KeyEvent;
    import java.awt.event.WindowAdapter;
    import java.awt.event.WindowEvent;
    import java.io.IOException;
    import java.net.InetSocketAddress;
    import java.nio.channels.SelectionKey;
    import java.nio.channels.Selector;
    import java.nio.channels.SocketChannel;
    import java.util.Iterator;
    import java.util.Set;
    import javax.swing.JFrame;
    import javax.swing.JScrollPane;
    import javax.swing.JTextArea;
    import javax.swing.JTextField;
    import javax.swing.SwingUtilities;
    public class Client extends JFrame  {
         private String host = "localhost";
         private int port = 5001;
         private SocketChannel socketChannel;
         private Selector selector;
         public void work() {               
              try {
                   socketChannel = SocketChannel.open();
                   socketChannel.configureBlocking(false);
                   InetSocketAddress isa = new InetSocketAddress(host, port);               
                   socketChannel.connect(isa);
                   selector = Selector.open();
                   socketChannel.register(selector, SelectionKey.OP_CONNECT | SelectionKey.OP_READ );
                   while(true) {
                        selector.select();
                        Set keys = selector.selectedKeys();
                        for(Iterator i = keys.iterator(); i.hasNext();) {
                             SelectionKey key = (SelectionKey) i.next();
                             i.remove();
                             if (key.isConnectable()) {
                                  SocketChannel keyChannel = (SocketChannel) key.channel();
                                  if (keyChannel.isConnectionPending()) {
                                       System.out.println("Connected "+keyChannel.finishConnect());                                                                           
                             } else if (key.isReadable()) {                                                                                                                                                           
                                  SocketChannel keyChannel = (SocketChannel) key.channel();                                             
                                  String m = Help.read(keyChannel);
                                  display(m);                                                                                                                                                                                                                   
              } catch (IOException e) {                                             
                   e.printStackTrace();                         
         private void display(final String m) {
              SwingUtilities.invokeLater(new Runnable() {
                   public void run() {
                        area.append(m+"\n");
                        textFieed.setText("");
         private void sendMessage(final String m) {
              Thread t = new Thread(new Runnable() {               
                   public void run() {                                                                                
                        try {                         
                             Help.write(m, socketChannel);
                        } catch (IOException e) {               
                             e.printStackTrace();
              t.start();                    
         public Client() {
              addWindowListener(new WindowAdapter() {
                   public void windowClosing(WindowEvent e) {
                        System.exit(1);
              textFieed.addKeyListener(new KeyAdapter() {
                   public void keyPressed(KeyEvent e) {
                        if (e.getKeyCode()== KeyEvent.VK_ENTER) {
                             String m = textFieed.getText();
                             sendMessage(m);     
              area.setEditable(false);
              getContentPane().add(textFieed, "North");
              getContentPane().add(new JScrollPane(area));
              setBounds(200, 200, 400, 300);
              show();
         private String messageToSend;
         private JTextArea area = new JTextArea();
         JTextField textFieed = new JTextField();
         public static void main(String[] args) {
              Client s = new Client();
              s.work();
    // HELPER CLASS
    package test;
    import java.io.IOException;
    import java.nio.ByteBuffer;
    import java.nio.CharBuffer;
    import java.nio.channels.SocketChannel;
    import java.nio.charset.Charset;
    import java.nio.charset.CharsetDecoder;
    import java.nio.charset.CharsetEncoder;
    public class Help {
         private static Charset charset = Charset.forName("us-ascii");
         private static CharsetEncoder enc = charset.newEncoder();
         private static CharsetDecoder dec = charset.newDecoder();
         private static void log(String m) {
              System.out.println(m);
         public static String read(SocketChannel channel) throws IOException {
              log("*** start READ");                              
              int n;
              ByteBuffer buffer = ByteBuffer.allocate(1024);
              while((n = channel.read(buffer)) > 0) {
                   System.out.println("     adding "+n+" bytes");
              log("  BUFFER REMPLI : "+buffer);
              buffer.flip();               
              CharBuffer cb = dec.decode(buffer);          
              log("  CHARBUFFER : "+cb);
              String m = cb.toString();
              log("  MESSAGE : "+m);          
              log("*** end READ");
              //buffer.clear();
              return m;                    
         public static void write(String m, SocketChannel channel) throws IOException {          
              log("xxx start WRITE");          
              CharBuffer cb = CharBuffer.wrap(m);
              log("  CHARBUFFER : "+cb);          
              ByteBuffer  buffer = enc.encode(cb);
              log("  BUFFER ALLOUE REMPLI : "+buffer);
              int n;
              while(buffer.hasRemaining()) {
                   n = channel.write(buffer);                         
              System.out.println("  REMAINING : "+buffer.hasRemaining());
              log("xxx end WRITE");

    Here's the fix for that old problem. Change the work method to do the following
    - don't register interest in things that can't happen
    - when you connect register based on whether the connection is complete or pending.
    - add the OP_READ interest once the connection is complete.
    This doesn't fix all the other problems this code will have,
    eg.
    - what happens if a write is incomplete?
    - why does my code loop if I add OP_WRITE interest?
    - why does my interestOps or register method block?
    For code that answers all those questions see my obese post Taming the NIO Circus
    Here's the fixed up Client code
    // CLIENT
    package test
    import java.awt.event.KeyAdapter;
    import java.awt.event.KeyEvent;
    import java.awt.event.WindowAdapter;
    import java.awt.event.WindowEvent;
    import java.io.IOException;
    import java.net.InetSocketAddress;
    import java.nio.channels.SelectionKey;
    import java.nio.channels.Selector;
    import java.nio.channels.SocketChannel;
    import java.util.Iterator;
    import java.util.Set;
    import javax.swing.JFrame;
    import javax.swing.JScrollPane;
    import javax.swing.JTextArea;
    import javax.swing.JTextField;
    import javax.swing.SwingUtilities;
    public class Client extends JFrame  {
         private String host = "localhost";
         private int port = 5001;
         private SocketChannel socketChannel;
         private Selector selector;
         public void work() {
              try {
                   socketChannel = SocketChannel.open();
                   socketChannel.configureBlocking(false);
                   InetSocketAddress isa = new InetSocketAddress(host, port);
                   socketChannel.connect(isa);
                   selector = Selector.open();
                   int interest = 0;
                   if(socketChannel.isConnected())interest = SelectionKey.OP_READ;
                   else if(socketChannel.isConnectionPending())interest = SelectionKey.OP_CONNECT;
                   socketChannel.register(selector, interest);
                   while(true)
                        int nn = selector.select();
                        System.out.println("nn="+nn);
                        Set keys = selector.selectedKeys();
                        for(Iterator i = keys.iterator(); i.hasNext();)
                             SelectionKey key = (SelectionKey) i.next();
                             i.remove();
                             if (key.isConnectable())
                                  SocketChannel keyChannel = (SocketChannel) key.channel();
                                  System.out.println("Connected "+keyChannel.finishConnect());
                                  key.interestOps(SelectionKey.OP_READ);
                             if (key.isReadable())
                                  SocketChannel keyChannel = (SocketChannel) key.channel();
                                  String m = Help.read(keyChannel);
                                  display(m);
              } catch (IOException e) {
                   e.printStackTrace();
         private void display(final String m) {
              SwingUtilities.invokeLater(new Runnable() {
                   public void run() {
                        area.append(m+"\n");
                        textFieed.setText("");
         private void sendMessage(final String m) {
              Thread t = new Thread(new Runnable() {
                   public void run() {
                        try {
                             Help.write(m, socketChannel);
                        } catch (IOException e) {
                             e.printStackTrace();
              t.start();
         public Client() {
              addWindowListener(new WindowAdapter() {
                   public void windowClosing(WindowEvent e) {
                        System.exit(1);
              textFieed.addKeyListener(new KeyAdapter() {
                   public void keyPressed(KeyEvent e) {
                        if (e.getKeyCode()== KeyEvent.VK_ENTER) {
                             String m = textFieed.getText();
                             sendMessage(m);
              area.setEditable(false);
              getContentPane().add(textFieed, "North");
              getContentPane().add(new JScrollPane(area));
              setBounds(200, 200, 400, 300);
              show();
         private String messageToSend;
         private JTextArea area = new JTextArea();
         JTextField textFieed = new JTextField();
         public static void main(String[] args) {
              Client s = new Client();
              s.work();

  • The java.nio circus - what a headache

    Hi All
    I've been working now with the java.nio classes for a bit and trying to get a non-blocking client HTTP socket going. The requirement: no 3rd-party code and must either read fully from the socket OR timeout quickly (say 10sec).
    Unfortunately, I haven't been able to find ANY nio code that actually does this. There was one example by a John Z. that came really close, but it hung after a full reading of an HTTP stream because it could not determine if it was done reading or not!
    Anyway, here's my full code. It's very hacky and piecemeal, but i'm just looking for a prototype.
    Here's the main processing loop, cobbled together from a mix of examples on the web, plus my own little flourishes... :)
    public static void createSelector(URL url) throws IOException
    // Create a selector and register two socket channels
    Selector selector = null;
    // Create the selector
    selector = Selector.open();
    SocketChannel sChannel = SocketChannel.open();
    sChannel.configureBlocking(false);
    int port = url.getPort();
    if (port == -1) { port = 80; }
    sChannel.connect(new InetSocketAddress(url.getHost(), port));
    // Before the socket is usable, the connection must be completed
    // by calling finishConnect(), which is non-blocking
    int iTime = 0;
    while (!sChannel.finishConnect())
         if (iTime > 10)
              System.out.println("Timeout!");
              throw new IOException("Connection timed out!");
         iTime++;
    System.out.println("Waiting for connection...");
    try { Thread.sleep(1000); } catch (InterruptedException interex) {}
    // Register the channel with selector, listening for all events
    //sChannel.register(selector, sChannel.validOps());
    sChannel.register(selector, SelectionKey.OP_CONNECT | SelectionKey.OP_READ);
    boolean allDone = false;
    while (selector.select(5000) > 0)
    // Get list of selection keys with pending events
    Iterator it = selector.selectedKeys().iterator();
    // Process each key at a time
    while (it.hasNext())
    // Get the selection key
    SelectionKey selKey = (SelectionKey)it.next();
    // Remove it from the list to indicate that it is being processed
    it.remove();
    allDone = processSelectionKey(selKey, url);
    if (allDone) { break; }
    } // while(true)
    if (allDone)
    System.out.println("All done!");
    else
         System.out.println("Timeout!");
    Now, the processSelectionKey method returns a true only if we get a -1 on our read. It looks like this:
    public static boolean processSelectionKey(SelectionKey selKey, URL url) throws IOException
    // Since the ready operations are cumulative,
    // need to check readiness for each operation
    if (selKey.isValid() && selKey.isConnectable())
    // Get channel with connection request
    SocketChannel sChannel = (SocketChannel)selKey.channel();
    //System.out.println("Sending request for "+url.getPath()+" using Connectable stream");
    String request = "GET "+url.getPath()+" \r\n\r\n";
    //System.out.println("Writing request "+request);
    sChannel.write(NonblockSocket.encoder.encode(CharBuffer.wrap(request)));
    //sentRequest = true;
    if (selKey.isValid() && selKey.isReadable())
         System.out.println("Readable key!");
    // Get channel with bytes to read
    SocketChannel sChannel = (SocketChannel)selKey.channel();
    int bytesRead = sChannel.read(buf);
    if (bytesRead == -1)
         return true;
    else
         System.out.println("Read "+bytesRead+" bytes...");
    buf.flip();
    System.out.println(decode(buf));
    buf.clear();
    cbuf.clear();
    OK, now this code does not work. There are a couple of problems with it. First of all, the "GET "+urlpath+" \r\n\r\n" is called a WHOLE BUNCH of times, not the usual once-per-request, as you would expect. But, if I programmatically enforce only one GET, it hangs. Still, it's strange.
    Secondly (and this is the real sticky point), it almost never reads in a full HTTP stream from the server before I get the following UNSETTLING exception during a read from the Channel:
    java.io.IOException: An existing connection was forcibly closed by the remote host
    I usually get either one or two successful reads from a readable channel and then the server disconnects me! NEVER saw this one coming. All in all, has ANYONE been able to use nio for client-side stuff to ANY degree at all? Maybe I'm doing something wrong? Any help would be excellent. BTW I'm running Win2K with JDK1.4.1.
    If I can't get this one to work, maybe I'll check out the new additions to the "old-school" java.net Socket.

    Actually,
    I just wrote a if..then to ignore the null accept.
    Even without using non-blocking connect i got two accepts.
    import java.net.*;
    import java.nio.*;
    import java.nio.channels.*;
    import java.nio.channels.spi.*;
    import java.nio.charset.*;
    import java.io.*;
    import java.util.*;
    * @author Aashish
    * To change the template for this generated type comment go to
    * Window>Preferences>Java>Code Generation>Code and Comments
    public class NonBlockingServerSocket
         public static void main(String[] args)
              try
                        ServerSocketChannel serv = ServerSocketChannel.open();
                        serv.configureBlocking(false);
                        serv.socket().bind(new InetSocketAddress(33333));
                        System.out.println("bind complete");
                        Selector selector = Selector.open();
                        serv.register(selector,SelectionKey.OP_ACCEPT);
                        while(selector.select() > 0)
                             Iterator iter = selector.selectedKeys().iterator();
                             while(iter.hasNext())
                                  SelectionKey key = (SelectionKey)iter.next();
                                  if(key.isAcceptable())
                                       ServerSocketChannel servChan = (ServerSocketChannel)key.channel();
                                       SocketChannel sockChan = servChan.accept();
                                       if(sockChan == null)
                                            System.out.println("Just got a null channel in accept");
                                       else
                                            System.out.println("just accepted a sockChan");
                                            sockChan.configureBlocking(false);
                                           sockChan.register(selector,SelectionKey.OP_READ);
                                           if(sockChan.isConnected())
                                                 System.out.println("Channel is connected");
                                  if(key.isReadable())
                                                        //something here
              catch (Exception ex)
                   ex.printStackTrace();

  • Need help with java.nio.socket program.

    Can someone please help me here? I am trying to create a class that listens on a socket for an incoming connection, but I am getting the following error on compile:
    SocketIn.java:48: incompatible types
    found : java.nio.channels.SocketChannel
    required: java.net.Socket
    Socket requestSocket = requestChannel.accept();
    Here is my code:
    package NETC;
    import java.net.*;
    import java.util.*;
    import java.io.*;
    import java.nio.*;
    import java.nio.charset.*;
    import java.nio.channels.*;
    import java.nio.channels.spi.*;
    public class SocketIn{
         // Buffer for any incoming data.
         private static ByteBuffer inBuffer = ByteBuffer.allocateDirect(1024);
         CharBuffer charBuffer = null;
         public SocketIn(int intPort)throws Exception{
              // Create a non-blocking server socket.
              ServerSocketChannel serverChannel = ServerSocketChannel.open();
              serverChannel.configureBlocking(false);
              // Use the host and port to bind the server socket.
              InetAddress inetAddress = InetAddress.getLocalHost();
              InetSocketAddress socketAddress = new InetSocketAddress(inetAddress, intPort);
              serverChannel.socket().bind(socketAddress);
              // The selector for incoming requests.
              Selector requestSelector = SelectorProvider.provider().openSelector();
              // Put the server socket on the selectors 'ready list'.
              serverChannel.register(requestSelector, SelectionKey.OP_ACCEPT);
              while(requestSelector.select() > 0){
                   System.out.println("Connection Accepted...");
                   // A request has been made and is ready for IO.
                   Set requestKeys = requestSelector.selectedKeys();
                   Iterator iterator = requestKeys.iterator();
                   while(iterator.hasNext()){
                        SelectionKey requestKey = (SelectionKey)iterator.next();
                        // Get the socket that's ready for IO.
                        ServerSocketChannel requestChannel = (ServerSocketChannel)requestKey.channel();
                        // This shouldn't block.
                        Socket requestSocket = requestChannel.accept();
                        inBuffer.clear();
                        requestSocket.read(inBuffer);                    
                        inBuffer.flip();
                        Charset charset = Charset.forName("ISO-8859-1");
                        CharsetDecoder decoder = charset.newDecoder();
                        charBuffer = decoder.decode(inBuffer);
                        iterator.remove();                    
    }

    Well its been about a month and a half so I guess you have your answer. In case you don't...
    // This shouldn't block.
    Socket requestSocket = requestChannel.accept();
    The problem is quite simple. The ServerSocketChannel.accept() returns a SocketChannel. (Channel -> Channel).
    You can get access to the socket using the SocketChannel.
    SocketChannel requestSocketChannel = requestChannel.accept();
    Socket requestSocket = requestSocketChannel.socket();
    But you don't need the socket to read and write.

  • Questions about jdk nio charset

    During load test, work threads all wait for reading charset classes from Classloader. Why not keep the stable result in cache? Thanks a lot!
    "ExecuteThread: '25' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0x5c941800 nid=0x38ee waiting for monitor entry [5d697000..5d6988c8]
         at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:49)
         - waiting to lock <0x5113ebe0> (a sun.net.www.protocol.jar.JarFileFactory)
         at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:85)
         at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:107)
         at java.net.URL.openStream(URL.java:913)
         at sun.misc.Service.parse(Service.java:203)
         at sun.misc.Service.access$100(Service.java:111)
         at sun.misc.Service$LazyIterator.hasNext(Service.java:257)
         at java.nio.charset.Charset$1.getNext(Charset.java:303)
         at java.nio.charset.Charset$1.hasNext(Charset.java:318)
         at java.nio.charset.Charset$2.run(Charset.java:361)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.nio.charset.Charset.lookupViaProviders(Charset.java:358)
         at java.nio.charset.Charset.lookup(Charset.java:385)
         at java.nio.charset.Charset.isSupported(Charset.java:407)
         at java.lang.StringCoding.lookupCharset(StringCoding.java:82)
         at java.lang.StringCoding.encode(StringCoding.java:363)
         at java.lang.StringCoding.encode(StringCoding.java:380)
         at java.lang.String.getBytes(String.java:590)

    who can tell me how to avoid this? Thanks a lot!

  • Troubles with timeout using java.nio.channels and non-blocking sockets

    Hello.
    I have a server application that employs java.nio.channels with non-blocking sockets.
    The server waits for connections. The client should connect and be first in sending data.
    Timeouts are significant! If client exceeds the allowed time to send data, the server should break the connection.
    The huge trouble I've discovered that I cannot control the timeout when client connects but remains silent.
    My code looks as follows:
    <pre>
    Selector oSel;
    SocketChannel oSockChan;
    Socket oSock;
    SelectionKey oSelKey;
    Iterator<SelectionKey> oItSelKeys;
    int iCurrState, iMask, iCount;
    iCurrState = INT_SERVER_WORKING;
    iMask = SelectionKey.OP_ACCEPT | SelectionKey.OP_CONNECT | SelectionKey.OP_READ | SelectionKey.OP_WRITE;
    while ( iCurrState == INT_SERVER_WORKING )
    try
    *// retrieving next action*
    iCount = oSel.select();
    if ( iCount > 0 )
    oItSelKeys = oSel.selectedKeys().iterator();
    while ( oItSelKeys.hasNext() )
    oSelKey = oItSelKeys.next();
    oItSelKeys.remove();
    if ( oSelKey.isValid() )
    switch ( oSelKey.readyOps() & iMask ) {
    case SelectionKey.OP_ACCEPT :
    oSockChan = oSSockChan.accept();
    oSockChan.configureBlocking(false);
    oSock = oSockChan.socket();
    oSock.setKeepAlive(true);
    oSockChan.register(oSel,SelectionKey.OP_READ,new MyPacket(oSock.getInetAddress(),oSock.getPort()));
    break;
    case SelectionKey.OP_READ :
    oSelKey.interestOps(0);
    ((MyPacket) oSelKey.attachment()).inRequest(); *// preparing request*
    this.getReader().add(oSelKey); *// sending key to reading thread*
    break;
    case SelectionKey.OP_WRITE :
    oSelKey.interestOps(0);
    ((MyRequest) oSelKey.attachment()).inResponse(); *// preparing response*
    this.getWriter().add(oSelKey); *// sending key to writing thread*
    break;
    case SelectionKey.OP_CONNECT :
    default :
    *// nothing to do*
    catch ( IOException oExcept )
    *// do some actions*
    </pre>
    Timeouts are easily controlled by reading and writing threads (see OP_READ and OP_WRITE ).
    But when a client just connects without consequent data send, the state of this connection remains as OP_ACCEPT. The connection remains open for arbitrarily large time and I cannot control it!
    Please help with idea how can I terminate such connections!

    How can I process the keys that weren't selected at the bottom of the loop? Should I use the method keys() ?Yes. Form a new set from keys() and removeAll(selectedKeys()). Do that before you process selectedKeys().
    And the second moment: as I understood a single key may contain several operations simultaneously? Thus I should use several if's (but not if/else 'cause it's the equivalent of switch ... case ).If there is anything unclear about 'your switch statement is invalid. You need an if/else chain' I fail to see what it is. Try reading it again. And if several ifs were really the equivalent of "switch ... case", there wouldn't be a problem in the first place. They're not, and there is.

  • Socket seems too slow...maybe java.nio?

    Hello
    In our system I have to receive Multicast Packets repeats very quick(1-5 ms). I have to link them one after an other in the order they sent. And if I miss a packet, something will go wrong...
    I have to listen to different IPs and I use different threads for different IPs. If I use only one thread (and listens to only one IP) everything seems ok.
    But if I starts listening to an other ip too, I miss 2 packets in a row, or only one if I turn off parsing the message (XML).
    Here is the code I use in the Threads:
    try{
                   socket = new MulticastSocket(port);
                   socket.setSoTimeout(1000);
                   inetAddress = InetAddress.getByName(ip);
                   NetworkInterface nInterface=
    NetworkInterface.getByName(networkInterface);
                   if(nInterface!=null)socket.setNetworkInterface(nInterface);
                   socket.joinGroup(inetAddress);
              catch(IOException ioe){
                   logger.error(ioe.getMessage(), ioe);
    return;
              try{
                   while (!interrupted()) {
                        try{
                             byte[] buffer = new byte[1480];
                             packet = new DatagramPacket(buffer, buffer.length);
                             socket.receive(packet);
    parse(buffer);
                        }catch(SocketTimeoutException stoe){
                             //     DO NOTHING
                        }catch(IOException ioe){
                             logger.error(ioe.getMessage(), ioe);
              }finally{
                   try {
                        if(socket!=null){
                             socket.leaveGroup(inetAddress);
                   } catch (IOException ioe) {}
    Every Thread has its own parsing object.
    Any tips, what is wrong?
    Maybe java.nio could solve the problem somehow. There is a sample server in [Java Home]/sample/nio/server and it suggest that there are quicker methods to receive messages from different IPs (maybe Blocking/Pooled-Thread Server). But I can't understand the API and the Sample while I was reading it (20-30 minutes).
    Could it be quicker? Does it worth toying with the idea?
    Thanks:
    Bence

    In our system I have to receive Multicast Packets
    repeats very quick(1-5 ms). I have to link them one
    after an other in the order they sent. And if I miss
    a packet, something will go wrong...There is no guarantee anywhere in the system that you won't miss a datagram. UDP doesn't make such guarantees. If you need all the packets you will have to build ACK or NACK into your protocol.
    You can alleviate the problem by running a very large socket receive buffer. But you can't eliminate it. Rethink this.
    NIO is not significantly quicker for applications like this, it is more scalable.

  • Java NIO client

    I need to make the server is able to hold about 500 connections and operates on a single thread. The server itself should make all the connections. Where can I find examples of finished implementations?

    I have an example, but it does not work
    import java.io.IOException;
    import java.net.InetAddress;
    import java.net.InetSocketAddress;
    import java.net.Socket;
    import java.nio.ByteBuffer;
    import java.nio.channels.SelectionKey;
    import java.nio.channels.Selector;
    import java.nio.channels.SocketChannel;
    import java.nio.channels.spi.SelectorProvider;
    import java.util.*;
    public class NioClient implements Runnable {
         // The host:port combination to connect to
         private InetAddress hostAddress;
         private String host;
         private int port;
         // The selector we'll be monitoring
         private Selector selector;
         // The buffer into which we'll read data when it's available
         private ByteBuffer readBuffer = ByteBuffer.allocate(8192);
         // A list of PendingChange instances
         private List pendingChanges = new LinkedList();
         // Maps a SocketChannel to a list of ByteBuffer instances
         private Map pendingData = new HashMap();
         // Maps a SocketChannel to a RspHandler
         private Map rspHandlers = Collections.synchronizedMap(new HashMap());
         public NioClient() {
              try {this.selector = this.initSelector();} catch(IOException e){}
         public void connect(String host, int port, RspHandler handler) throws IOException {
              this.hostAddress = hostAddress;
              this.host = host;
              this.port = port;
              this.send("$Hello |".getBytes(), handler);
         public void send(byte[] data, RspHandler handler) throws IOException {
              // Start a new connection
              SocketChannel socket = this.initiateConnection();
              // Register the response handler
              this.rspHandlers.put(socket, handler);
              // And queue the data we want written
              synchronized (this.pendingData) {
                   List queue = (List) this.pendingData.get(socket);
                   if (queue == null) {
                        queue = new ArrayList();
                        this.pendingData.put(socket, queue);
                   queue.add(ByteBuffer.wrap(data));
              // Finally, wake up our selecting thread so it can make the required changes
              this.selector.wakeup();
              handler.waitForResponse();
         public void run() {
              while (true) {
                   try {
                        // Process any pending changes
                        synchronized (this.pendingChanges) {
                             Iterator changes = this.pendingChanges.iterator();
                             while (changes.hasNext()) {
                                  ChangeRequest change = (ChangeRequest) changes.next();
                                  switch (change.type) {
                                  case ChangeRequest.CHANGEOPS:
                                       SelectionKey key = change.socket.keyFor(this.selector);
                                       key.interestOps(change.ops);
                                       break;
                                  case ChangeRequest.REGISTER:
                                       change.socket.register(this.selector, change.ops);
                                       break;
                             this.pendingChanges.clear();
                        // Wait for an event one of the registered channels
                        this.selector.select();
                        // Iterate over the set of keys for which events are available
                        Iterator selectedKeys = this.selector.selectedKeys().iterator();
                        while (selectedKeys.hasNext()) {
                             SelectionKey key = (SelectionKey) selectedKeys.next();
                             selectedKeys.remove();
                             if (!key.isValid()) {
                                  continue;
                             // Check what event is available and deal with it
                             if (key.isConnectable()) {
                                  this.finishConnection(key);
                             } else if (key.isReadable()) {
                                  this.read(key);
                             } else if (key.isWritable()) {
                                  this.write(key);
                   } catch (Exception e) {
                        e.printStackTrace();
         private void read(SelectionKey key) throws IOException {
              SocketChannel socketChannel = (SocketChannel) key.channel();
              // Clear out our read buffer so it's ready for new data
              this.readBuffer.clear();
              // Attempt to read off the channel
              int numRead;
              try {
                   numRead = socketChannel.read(this.readBuffer);
              } catch (IOException e) {
                   // The remote forcibly closed the connection, cancel
                   // the selection key and close the channel.
                   key.cancel();
                   socketChannel.close();
                   return;
              System.out.println("READ");
              if (numRead == -1) {
                   // Remote entity shut the socket down cleanly. Do the
                   // same from our end and cancel the channel.
                   key.channel().close();
                   key.cancel();
                   return;
              // Handle the response
              this.handleResponse(socketChannel, this.readBuffer.array(), numRead);
         private void handleResponse(SocketChannel socketChannel, byte[] data, int numRead) throws IOException {
              // Make a correctly sized copy of the data before handing it
              // to the client
              byte[] rspData = new byte[numRead];
              System.arraycopy(data, 0, rspData, 0, numRead);
              // Look up the handler for this channel
              RspHandler handler = (RspHandler) this.rspHandlers.get(socketChannel);
              // And pass the response to it
              if (handler.handleResponse(rspData)) {
                   // The handler has seen enough, close the connection
                   socketChannel.close();
                   socketChannel.keyFor(this.selector).cancel();
         private void write(SelectionKey key) throws IOException {
              SocketChannel socketChannel = (SocketChannel) key.channel();
              synchronized (this.pendingData) {
                   List queue = (List) this.pendingData.get(socketChannel);
                   // Write until there's not more data ...
                   while (!queue.isEmpty()) {
                        ByteBuffer buf = (ByteBuffer) queue.get(0);
                        socketChannel.write(buf);
                        if (buf.remaining() > 0) {
                             // ... or the socket's buffer fills up
                             break;
                        queue.remove(0);
                   if (queue.isEmpty()) {
                        // We wrote away all data, so we're no longer interested
                        // in writing on this socket. Switch back to waiting for
                        // data.
                        key.interestOps(SelectionKey.OP_READ);
         private void finishConnection(SelectionKey key) throws IOException {
              SocketChannel socketChannel = (SocketChannel) key.channel();
              // Finish the connection. If the connection operation failed
              // this will raise an IOException.
              try {
                   socketChannel.finishConnect();
              } catch (IOException e) {
                   // Cancel the channel's registration with our selector
                   System.out.println(e);
                   key.cancel();
                   return;
              // Register an interest in writing on this channel
              key.interestOps(SelectionKey.OP_WRITE);
         private SocketChannel initiateConnection() throws IOException {
              // Create a non-blocking socket channel
              SocketChannel socketChannel = SocketChannel.open();
              socketChannel.configureBlocking(false);
              // Kick off connection establishment
              socketChannel.connect(new InetSocketAddress(this.host, this.port));
              // Queue a channel registration since the caller is not the
              // selecting thread. As part of the registration we'll register
              // an interest in connection events. These are raised when a channel
              // is ready to complete connection establishment.
              synchronized(this.pendingChanges) {
                   this.pendingChanges.add(new ChangeRequest(socketChannel, ChangeRequest.REGISTER, SelectionKey.OP_CONNECT));
              return socketChannel;
         private Selector initSelector() throws IOException {
              // Create a new selector
              return SelectorProvider.provider().openSelector();
    public class RspHandler {
         private byte[] rsp = null;
         public synchronized boolean handleResponse(byte[] rsp) {
              this.rsp = rsp;
              this.notify();
              return true;
         public synchronized void waitForResponse() {
              while(this.rsp == null) {
                   try {
                        this.wait();
                   } catch (InterruptedException e) {
              System.out.println(new String(this.rsp));
    }          NioClient NioClient = new NioClient();
              Thread t = new Thread(NioClient);
              t.setDaemon(true);
              t.start();
              RspHandler handler = new RspHandler();          
              NioClient.connect("69.28.156.250", 27040, handler);
              NioClient.connect("72.165.61.188", 27040, handler);
              NioClient.connect("208.111.133.84", 27011, handler);
              NioClient.connect("72.165.61.136", 27012, handler);
    Edited by: 915967 on 01.08.2012 7:07

  • Java NIO's puzzle

    hi,all
    In traditonal thread-pool based programming way,the Server can process multi-clients by thread(time-sharing),all clients's time is equal.
    In Java NIO programming way,the [select] will process multi-clients without thread pool.So,the the second client must be processed after the first client be complished.if the first client is cost much time,other cients must be long time waiting?
    plz help me - thanks for your time
    Tony.
    Hangzhou,china

    As an NIO server in non-blocking mode obviously never blocks (or rather it only blocks in select(), when nothing is happening), it can implement any kind of scheduling it likes between the various SocketChannels it has accepted as connected clients. It isn't obliged in any way to completely finish with one client before starting on the next, and it generally can't do so because data from any particular client doesn't necessarily arrive in a single chunk.

  • Help "ascending" to Java NIO

    Hello there.
    I've been using Java for quite some time, but the few really networked client/server applications I have used, have used java.io and java.net. Now, even I have heard the rumours about Java NIO, and I am curious. I have tried reading up and down the API on java.nio.* , but I must say I am still kind of confused.
    Basically, I am looking for some example code, or a thorough explanation of how all the classes and components come together to form efficient asynchronous non-blocking I/O.
    What I picture is a server over TCP/IP meant to handle 100+ clients, written with an emphasis on performance. Think MMORPG, I guess (no no, I am not ambitious enough to want to write one, but I can imagine that gives a good analogy of how I would have the communication set up). The server needs to quickly figure out which clients need to be handled at any given time, with as little overhead or reduntant iteration as possible.
    Ok, I am asking alot perhaps.. I just wonder if anyone could offer links to example source-code, write something to show me, or basically just try to explain to me how this all works.
    I would be very very grateful if anyone stepped up to the plate.
    Thanks,
    -Terje

    Take a look at my thread Taming the NIO Circus. It starts with simple examples of a single thread echo server and a swing client.

  • Java.nio.channels.Pipe?

    Someone explain what java.nio.channels.Pipe should be uses for, and how to use it?

    You can use this class to transfer the content of a channel from one thread to another thread.
    The writing thread blocks when the pipe is full.
    package com.desoft.pipetest;
    import java.io.*;
    import java.nio.*;
    import java.nio.channels.*;
    public class PipeTest {
    public static void main (String args[])
    try
    FileInputStream fis = new FileInputStream(args[0]);
    FileChannel fc = fis.getChannel();
    Pipe p = Pipe.open();
    PipeReader pr = new PipeReader( p );
    Thread t = new Thread( pr );
    t.start();
    fc.transferTo(0, fc.size(), p.sink() );
    catch( IOException ioe )
    ioe.printStackTrace();
    * Read from a Pipe and write Content to System.out
    static class PipeReader implements Runnable
    Pipe p;
    public PipeReader( Pipe p )
    this.p = p;
    public void run()
    try
    ByteBuffer buffer = ByteBuffer.allocate( 10 );
    int len = 0;
    while( (len=p.source().read( buffer )) > 0 )
    buffer.rewind();
    for( int i = 0; i < len; i++ )
    System.out.print( (char)buffer.get());
    buffer.rewind();
    catch( IOException ioe )
    ioe.printStackTrace();

  • Java.io vs. java.nio

    Hi all,
    this is my first post in this forum.
    I would like to know your opinion about the "old" java.io and the "new" java.nio, I have executed many tests using both the packages and I haven't seen so many differences in terms of time and performance.
    My tests consist of doing file copy for example, using some files of different size and reading them in different block size also.
    Other tests were the hash value calculation, also in this case I did different tests reading the files in different blocks size.
    Finally I can say that I haven't seen so many differences in terms of time and performance like said in other articles found on the net.
    I haven't yet tried executing test of reading/writing files from many threads. Is there someone that have tried it?
    Your opinion?
    Best regards.
    Raffaele

    Hi all,
    this is my first post in this forum.
    I would like to know your opinion about the "old" java.io and the "new" java.nio, I have executed many tests using both the packages and I haven't seen so many differences in terms of time and performance.
    My tests consist of doing file copy for example, using some files of different size and reading them in different block size also.
    Other tests were the hash value calculation, also in this case I did different tests reading the files in different blocks size.
    Finally I can say that I haven't seen so many differences in terms of time and performance like said in other articles found on the net.
    I haven't yet tried executing test of reading/writing files from many threads. Is there someone that have tried it?
    Your opinion?
    Best regards.
    Raffaele

Maybe you are looking for