Java.nio.BufferOverflowException on big Strings ( 87MB)

Hi!
When calling getBytes() on a big String the following exception is thrown:
java.nio.BufferOverflowException
     at java.nio.charset.CoderResult.throwException(Unknown Source)
     at java.lang.StringCoding$CharsetSD.decode(Unknown Source)
     at java.lang.StringCoding.decode(Unknown Source)
     at java.lang.StringCoding.decode(Unknown Source)
     at java.lang.String.<init>(Unknown Source)
     at java.lang.String.<init>(Unknown Source)
     at com.db.janus.Janus.main(Janus.java:99)
This happens as soon as the size of the String gets bigger than 92003848 bytes.
I can reproduce it with the following code:
String x = new String(new byte[92003848]); // --> OK
String x = new String(new byte[92003849]); // --> Exception
JDK Version: 1.4.2_06
OS: Windows XP
Is this a known bug / limitation? I found a similar bug related to Java on Linux / Solaris, but this one was about getBytes() with Strings > 16MB.
Thanks for your help!

Is there any alternative to solve this problem.I am getting the error
java.nio.BufferOverflowException .This happens in my mail server when the file six\ze of xml where i have stored data about spam/nonspam grows upto 22mb.I have stored this xml in database as blob.at server startup uploading in in my class.Keeps througout in JVM after every 50 th request saving information back to database(about spam/nonspam).Now if xml size grows i am storing this in string to parse etc.it gives this bug error.Sholud i cahnge my datatype or use something as XML Parser to split my xml in pieces(mechanism i am not aware of at all) so that string just takes some size of xml (but here can i use string builder etc.). Now is there any proper solution to this kind of problem whenever i removes some daya from xml spam mail goes to inbox and vice versa.
Please suggest me some solution if possible.
complete error is:
Servlet initialization failed:
java.nio.BufferOverflowException
at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
at java.lang.StringCoding$CharsetSD.decode([BII)[C(Unknown Source)
         at java.lang.StringCoding.decode(Ljava.lang.String;[BII)[C(Unknown Source)
         at java.lang.String.<init>([BIILjava.lang.String;)V(Unknown Source)
         at java.lang.String.<init>([BLjava.lang.String;)V(Unknown Source)
         at zerocode.core.ByteBlock.toString(ByteBlock.java:89)
         at zcCollab.mailProcessing.NaiveBayesClassifier._getInstanceData(NaiveBayesClassifier.java:275)
         at zcCollab.mailProcessing.NaiveBayesClassifier.initialize(NaiveBayesClassifier.java:86)
         at zcCollab.mailProcessing.SimpleDeliveryManager._setupClassifiers(SimpleDeliveryManager.java:409)
         at zcCollab.mailProcessing.SimpleDeliveryManager.<init>(SimpleDeliveryManager.java:62)
         at zcCollab.zcApp.InfoManagerApp._createDeliveryProcessor(InfoManagerApp.java:307)
         at zcCollab.zcApp.InfoManagerApp.initialize(InfoManagerApp.java:294)
         at zerocode.udm.Application._createInstance(Application.java:168)
         at zerocode.udm.Application.createInstanceForServlet(Application.java:111)
         at zerocode.servlet.UdmServlet._createApplication(UdmServlet.java:67)
         at zerocode.servlet.UdmServlet._initialize(UdmServlet.java:52)
         at zerocode.servlet.GenericServlet.init(GenericServlet.java:116)
         at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:292)
         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:329)
         at org.mortbay.jetty.servlet.ServletHandler.dispatch(ServletHandler.java:657)
         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
         at org.mortbay.http.HttpServer.service(HttpServer.java:879)
         at org.mortbay.http.handler.ForwardHandler.handle(ForwardHandler.java:130)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
         at org.mortbay.http.HttpServer.service(HttpServer.java:879)
Thanks
Vijendra

Similar Messages

Javac compiler throws java.nio.BufferOverflowException!!Why?

An exception has occurred in the compiler (1.4.1). Please file a bug at the Java Developer Connection (http://java.sun.com/cgi-bin/bugreport.cgi) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.nio.BufferOverflowException
at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
at java.lang.StringCoding$CharsetSD.decode(StringCoding.java:186)
at java.lang.StringCoding.decode(StringCoding.java:222)
at java.lang.StringCoding.decode(StringCoding.java:228)
at java.lang.String.<init>(String.java:383)
at java.lang.String.<init>(String.java:404)
at java.io.UnixFileSystem.list(Native Method)
at java.io.File.list(File.java:914)
at com.sun.tools.javac.v8.code.ClassReader.list(ClassReader.java:1224)
at com.sun.tools.javac.v8.code.ClassReader.listAll(ClassReader.java:1320)
at com.sun.tools.javac.v8.code.ClassReader.fillIn(ClassReader.java:1340)
at com.sun.tools.javac.v8.code.ClassReader.complete(ClassReader.java:1049)
at com.sun.tools.javac.v8.code.Symbol.complete(Symbol.java:332)
at com.sun.tools.javac.v8.comp.Enter.visitTopLevel(Enter.java:467)
at com.sun.tools.javac.v8.tree.Tree$TopLevel.accept(Tree.java:390)
at com.sun.tools.javac.v8.comp.Enter.classEnter(Enter.java:442)
at com.sun.tools.javac.v8.comp.Enter.classEnter(Enter.java:456)
at com.sun.tools.javac.v8.comp.Enter.complete(Enter.java:588)
at com.sun.tools.javac.v8.comp.Enter.main(Enter.java:574)
at com.sun.tools.javac.v8.JavaCompiler.compile(JavaCompiler.java:334)
at com.sun.tools.javac.v8.Main.compile(Main.java:520)
at com.sun.tools.javac.Main.compile(Main.java:36)
at com.sun.tools.javac.Main.main(Main.java:27)

See this bug:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4949631

String.getBytes() & String(byte[]) - java.nio.BufferOverflowException

The application in question uses JNI for legacy integration and I suspect the legacy code is corrupting the stack causing the above error. However, the error does not occur in Java 1.3, only Java 1.4.
Is there some way to suppress 1.4's use of the native IO API when encoding and decoding byte streams? This would at least provide a workaround in the meantime.
Thanks.

This is beginning to make a little sense. The problem is that you got a String and you don't want one. A String wraps an array of chars, which your app needs, right? Specifically they're chars because you need 16-bit char sets.
Presumably the getBytes() method call is used to get an array of bytes for some data transfer operation. java.nio was probably added in 1.4 as it has some very efficient ways of handling buffers as simultaneously of two or more types. It's trying to use the underlying char array as a byte array and there's a straight up bug someplace.
Workaround is strange to contemplate, but I'm pretty sure it will work: use String.getChars() to get an array of chars, and then use java.nio yourself to create your byte array! If you've never been there, it's not very hard. I use nio all the time and it's never been a problem.

Java.nio select() method return 0 in my client application

Hello,
I'm developing a simple chat application who echo messages
But my client application loop because the select() method return 0
This is my code
// SERVER
package test;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;
public class Server {
     private int port = 5001;
     public void work() {
          try {
               ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
               serverSocketChannel.configureBlocking(false);
               InetSocketAddress isa = new InetSocketAddress(port);
               serverSocketChannel.socket().bind(isa);
               Selector selector = Selector.open();
               serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
               System.out.println("Listing on "+port);
               while(selector.select()>0) {
                    Set keys = selector.selectedKeys();
                    for(Iterator i = keys.iterator(); i.hasNext();) {
                         SelectionKey key = (SelectionKey) i.next();
                         i.remove();
                         if (key.isAcceptable()) {
                              ServerSocketChannel keyChannel = (ServerSocketChannel)key.channel();
                              SocketChannel channel = keyChannel.accept();
                              channel.configureBlocking(false);
                              channel.register(selector, SelectionKey.OP_READ );
                         } else if (key.isReadable()) {
                              SocketChannel keyChannel = (SocketChannel) key.channel();
                              String m = Help.read(keyChannel );
                              Help.write(m.toUpperCase(), keyChannel );
          } catch (IOException e) {
               e.printStackTrace();
     public static void main(String[] args) {
          Server s = new Server();
          s.work();
// CLIENT
package test;
import java.awt.event.KeyAdapter;
import java.awt.event.KeyEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;
public class Client extends JFrame {
     private String host = "localhost";
     private int port = 5001;
     private SocketChannel socketChannel;
     private Selector selector;
     public void work() {
          try {
               socketChannel = SocketChannel.open();
               socketChannel.configureBlocking(false);
               InetSocketAddress isa = new InetSocketAddress(host, port);
               socketChannel.connect(isa);
               selector = Selector.open();
               socketChannel.register(selector, SelectionKey.OP_CONNECT | SelectionKey.OP_READ );
               while(true) {
                    selector.select();
                    Set keys = selector.selectedKeys();
                    for(Iterator i = keys.iterator(); i.hasNext();) {
                         SelectionKey key = (SelectionKey) i.next();
                         i.remove();
                         if (key.isConnectable()) {
                              SocketChannel keyChannel = (SocketChannel) key.channel();
                              if (keyChannel.isConnectionPending()) {
                                   System.out.println("Connected "+keyChannel.finishConnect());
                         } else if (key.isReadable()) {
                              SocketChannel keyChannel = (SocketChannel) key.channel();
                              String m = Help.read(keyChannel);
                              display(m);
          } catch (IOException e) {
               e.printStackTrace();
     private void display(final String m) {
          SwingUtilities.invokeLater(new Runnable() {
               public void run() {
                    area.append(m+"\n");
                    textFieed.setText("");
     private void sendMessage(final String m) {
          Thread t = new Thread(new Runnable() {
               public void run() {
                    try {
                         Help.write(m, socketChannel);
                    } catch (IOException e) {
                         e.printStackTrace();
          t.start();
     public Client() {
          addWindowListener(new WindowAdapter() {
               public void windowClosing(WindowEvent e) {
                    System.exit(1);
          textFieed.addKeyListener(new KeyAdapter() {
               public void keyPressed(KeyEvent e) {
                    if (e.getKeyCode()== KeyEvent.VK_ENTER) {
                         String m = textFieed.getText();
                         sendMessage(m);
          area.setEditable(false);
          getContentPane().add(textFieed, "North");
          getContentPane().add(new JScrollPane(area));
          setBounds(200, 200, 400, 300);
          show();
     private String messageToSend;
     private JTextArea area = new JTextArea();
     JTextField textFieed = new JTextField();
     public static void main(String[] args) {
          Client s = new Client();
          s.work();
// HELPER CLASS
package test;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.nio.charset.CharsetEncoder;
public class Help {
     private static Charset charset = Charset.forName("us-ascii");
     private static CharsetEncoder enc = charset.newEncoder();
     private static CharsetDecoder dec = charset.newDecoder();
     private static void log(String m) {
          System.out.println(m);
     public static String read(SocketChannel channel) throws IOException {
          log("*** start READ");
          int n;
          ByteBuffer buffer = ByteBuffer.allocate(1024);
          while((n = channel.read(buffer)) > 0) {
               System.out.println("     adding "+n+" bytes");
          log(" BUFFER REMPLI : "+buffer);
          buffer.flip();
          CharBuffer cb = dec.decode(buffer);
          log(" CHARBUFFER : "+cb);
          String m = cb.toString();
          log(" MESSAGE : "+m);
          log("*** end READ");
          //buffer.clear();
          return m;
     public static void write(String m, SocketChannel channel) throws IOException {
          log("xxx start WRITE");
          CharBuffer cb = CharBuffer.wrap(m);
          log(" CHARBUFFER : "+cb);
          ByteBuffer buffer = enc.encode(cb);
          log(" BUFFER ALLOUE REMPLI : "+buffer);
          int n;
          while(buffer.hasRemaining()) {
               n = channel.write(buffer);
          System.out.println(" REMAINING : "+buffer.hasRemaining());
          log("xxx end WRITE");

Here's the fix for that old problem. Change the work method to do the following
- don't register interest in things that can't happen
- when you connect register based on whether the connection is complete or pending.
- add the OP_READ interest once the connection is complete.
This doesn't fix all the other problems this code will have,
eg.
- what happens if a write is incomplete?
- why does my code loop if I add OP_WRITE interest?
- why does my interestOps or register method block?
For code that answers all those questions see my obese post Taming the NIO Circus
Here's the fixed up Client code
// CLIENT
package test
import java.awt.event.KeyAdapter;
import java.awt.event.KeyEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;
public class Client extends JFrame {
     private String host = "localhost";
     private int port = 5001;
     private SocketChannel socketChannel;
     private Selector selector;
     public void work() {
          try {
               socketChannel = SocketChannel.open();
               socketChannel.configureBlocking(false);
               InetSocketAddress isa = new InetSocketAddress(host, port);
               socketChannel.connect(isa);
               selector = Selector.open();
               int interest = 0;
               if(socketChannel.isConnected())interest = SelectionKey.OP_READ;
               else if(socketChannel.isConnectionPending())interest = SelectionKey.OP_CONNECT;
               socketChannel.register(selector, interest);
               while(true)
                    int nn = selector.select();
                    System.out.println("nn="+nn);
                    Set keys = selector.selectedKeys();
                    for(Iterator i = keys.iterator(); i.hasNext();)
                         SelectionKey key = (SelectionKey) i.next();
                         i.remove();
                         if (key.isConnectable())
                              SocketChannel keyChannel = (SocketChannel) key.channel();
                              System.out.println("Connected "+keyChannel.finishConnect());
                              key.interestOps(SelectionKey.OP_READ);
                         if (key.isReadable())
                              SocketChannel keyChannel = (SocketChannel) key.channel();
                              String m = Help.read(keyChannel);
                              display(m);
          } catch (IOException e) {
               e.printStackTrace();
     private void display(final String m) {
          SwingUtilities.invokeLater(new Runnable() {
               public void run() {
                    area.append(m+"\n");
                    textFieed.setText("");
     private void sendMessage(final String m) {
          Thread t = new Thread(new Runnable() {
               public void run() {
                    try {
                         Help.write(m, socketChannel);
                    } catch (IOException e) {
                         e.printStackTrace();
          t.start();
     public Client() {
          addWindowListener(new WindowAdapter() {
               public void windowClosing(WindowEvent e) {
                    System.exit(1);
          textFieed.addKeyListener(new KeyAdapter() {
               public void keyPressed(KeyEvent e) {
                    if (e.getKeyCode()== KeyEvent.VK_ENTER) {
                         String m = textFieed.getText();
                         sendMessage(m);
          area.setEditable(false);
          getContentPane().add(textFieed, "North");
          getContentPane().add(new JScrollPane(area));
          setBounds(200, 200, 400, 300);
          show();
     private String messageToSend;
     private JTextArea area = new JTextArea();
     JTextField textFieed = new JTextField();
     public static void main(String[] args) {
          Client s = new Client();
          s.work();

Java NIO locking and NTFS network resources

Hi all - just ran into a really nasty situation and I was wondering if anyone else has hit it and might have some suggestions.
Platform: JRE 1.4_02 on a Win XP machine
The following test code locks a file, then copies it to another location using NIO.
When I run it with source path on my local drives (C), it works fine. If I run it with source path on a network shared resource, it fails with an IOException with description 'Error performing inpage operation'.
If I disable the lock immediately before the copy operation, it works fine.
My conclusion is that there is something about the NIO locking implementation that prevents it from working properly with NTFS volumes on other hosts. Can this be right? I've found the following bug report:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4774175
but this seems like a huge problem that would prevent folks from using NIO in many, many applications. Maybe I'm wrong on something here...
Anyway, here's the test code:
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
* Created on May 28, 2004
* (c) 2004 Trumpet, Inc.
* @author kevin
public class test {
     private void createFile(File f) throws IOException{
          FileOutputStream os = new FileOutputStream(f);
          for(int i = 0; i < 10; i++){
               os.write(i);
          os.close();
     public test() {
          boolean testWithReleasingLockPriorToCopy = false;
          final File f1= new File("w:/temp/test2.lok");
          final File f2 = new File("w:/temp/test.lok");
          f1.delete();
          f2.delete();
          try {
               createFile(f1);
               RandomAccessFile raf1 = new RandomAccessFile(f1, "rw");
               RandomAccessFile raf2 = new RandomAccessFile(f1, "rw");
               FileChannel ch1 = raf1.getChannel();
               FileChannel ch2 = raf2.getChannel();
               FileLock flock1 = ch1.lock();
              if (!f2.getParentFile().exists() && !f2.getParentFile().mkdirs())
                   throw new IOException("Unable to create directories for destination file '" + f2 + "'");
              if (testWithReleasingLockPriorToCopy)
                   flock1.release();
               ch1.transferTo(0, raf1.length(), ch2);
               raf1.close();
               raf2.close();
          } catch (Exception e) {
               // TODO Auto-generated catch block
               e.printStackTrace();
     public static void main(String[] args) {
          test t = new test();
}Does anyone have any pointers here? I need to be able to exclusively lock a file on a network drive (preventing any other applications from opening it), then make a copy of it. I can't use regular stream operations, because the lock prevents them from working properly (it appears that, once you grab a file lock using NIO, the only way your application can use the file is via the NIO operations - using stream operations fails...).
Thanks in advance for any help!
- Kevin

i've run into the same problem recently, channels working fine for local file locking, but when you turn to the network, they fail to accurately handle locks.
i ended up writing a jni utility to ship with my java application that locks files using native windows calls.
my .c file ends up looking something like this:
JNIEXPORT jint JNICALL Java_Mapper_NativeUtils_LockFile
(JNIEnv *env, jobject obj, jstring filename)
const char* ntvFilename = (*env)->GetStringUTFChars(env, filename, 0);
int retVal = (int)CreateFile
ntvFilename
, GENERIC_WRITE
, FILE_SHARE_READ
, 0
, OPEN_EXISTING
, FILE_FLAG_SEQUENTIAL_SCAN
, 0
//add code to throw java exceptions based on retVal
if (retVal == (int)INVALID_HANDLE_VALUE)
return retVal;
(*env)->ReleaseStringUTFChars(env, filename, ntvFilename);
return retVal;
JNIEXPORT jboolean JNICALL Java_Mapper_NativeUtils_UnlockFile
(JNIEnv *env, jobject obj, jint handle)
     CloseHandle((void *)handle);
return 1;
it's a little shy on the error checking side, but it provides support for network file locking that java seems to lack.

Java NIO client

I need to make the server is able to hold about 500 connections and operates on a single thread. The server itself should make all the connections. Where can I find examples of finished implementations?

I have an example, but it does not work
import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.channels.spi.SelectorProvider;
import java.util.*;
public class NioClient implements Runnable {
     // The host:port combination to connect to
     private InetAddress hostAddress;
     private String host;
     private int port;
     // The selector we'll be monitoring
     private Selector selector;
     // The buffer into which we'll read data when it's available
     private ByteBuffer readBuffer = ByteBuffer.allocate(8192);
     // A list of PendingChange instances
     private List pendingChanges = new LinkedList();
     // Maps a SocketChannel to a list of ByteBuffer instances
     private Map pendingData = new HashMap();
     // Maps a SocketChannel to a RspHandler
     private Map rspHandlers = Collections.synchronizedMap(new HashMap());
     public NioClient() {
          try {this.selector = this.initSelector();} catch(IOException e){}
     public void connect(String host, int port, RspHandler handler) throws IOException {
          this.hostAddress = hostAddress;
          this.host = host;
          this.port = port;
          this.send("$Hello |".getBytes(), handler);
     public void send(byte[] data, RspHandler handler) throws IOException {
          // Start a new connection
          SocketChannel socket = this.initiateConnection();
          // Register the response handler
          this.rspHandlers.put(socket, handler);
          // And queue the data we want written
          synchronized (this.pendingData) {
               List queue = (List) this.pendingData.get(socket);
               if (queue == null) {
                    queue = new ArrayList();
                    this.pendingData.put(socket, queue);
               queue.add(ByteBuffer.wrap(data));
          // Finally, wake up our selecting thread so it can make the required changes
          this.selector.wakeup();
          handler.waitForResponse();
     public void run() {
          while (true) {
               try {
                    // Process any pending changes
                    synchronized (this.pendingChanges) {
                         Iterator changes = this.pendingChanges.iterator();
                         while (changes.hasNext()) {
                              ChangeRequest change = (ChangeRequest) changes.next();
                              switch (change.type) {
                              case ChangeRequest.CHANGEOPS:
                                   SelectionKey key = change.socket.keyFor(this.selector);
                                   key.interestOps(change.ops);
                                   break;
                              case ChangeRequest.REGISTER:
                                   change.socket.register(this.selector, change.ops);
                                   break;
                         this.pendingChanges.clear();
                    // Wait for an event one of the registered channels
                    this.selector.select();
                    // Iterate over the set of keys for which events are available
                    Iterator selectedKeys = this.selector.selectedKeys().iterator();
                    while (selectedKeys.hasNext()) {
                         SelectionKey key = (SelectionKey) selectedKeys.next();
                         selectedKeys.remove();
                         if (!key.isValid()) {
                              continue;
                         // Check what event is available and deal with it
                         if (key.isConnectable()) {
                              this.finishConnection(key);
                         } else if (key.isReadable()) {
                              this.read(key);
                         } else if (key.isWritable()) {
                              this.write(key);
               } catch (Exception e) {
                    e.printStackTrace();
     private void read(SelectionKey key) throws IOException {
          SocketChannel socketChannel = (SocketChannel) key.channel();
          // Clear out our read buffer so it's ready for new data
          this.readBuffer.clear();
          // Attempt to read off the channel
          int numRead;
          try {
               numRead = socketChannel.read(this.readBuffer);
          } catch (IOException e) {
               // The remote forcibly closed the connection, cancel
               // the selection key and close the channel.
               key.cancel();
               socketChannel.close();
               return;
          System.out.println("READ");
          if (numRead == -1) {
               // Remote entity shut the socket down cleanly. Do the
               // same from our end and cancel the channel.
               key.channel().close();
               key.cancel();
               return;
          // Handle the response
          this.handleResponse(socketChannel, this.readBuffer.array(), numRead);
     private void handleResponse(SocketChannel socketChannel, byte[] data, int numRead) throws IOException {
          // Make a correctly sized copy of the data before handing it
          // to the client
          byte[] rspData = new byte[numRead];
          System.arraycopy(data, 0, rspData, 0, numRead);
          // Look up the handler for this channel
          RspHandler handler = (RspHandler) this.rspHandlers.get(socketChannel);
          // And pass the response to it
          if (handler.handleResponse(rspData)) {
               // The handler has seen enough, close the connection
               socketChannel.close();
               socketChannel.keyFor(this.selector).cancel();
     private void write(SelectionKey key) throws IOException {
          SocketChannel socketChannel = (SocketChannel) key.channel();
          synchronized (this.pendingData) {
               List queue = (List) this.pendingData.get(socketChannel);
               // Write until there's not more data ...
               while (!queue.isEmpty()) {
                    ByteBuffer buf = (ByteBuffer) queue.get(0);
                    socketChannel.write(buf);
                    if (buf.remaining() > 0) {
                         // ... or the socket's buffer fills up
                         break;
                    queue.remove(0);
               if (queue.isEmpty()) {
                    // We wrote away all data, so we're no longer interested
                    // in writing on this socket. Switch back to waiting for
                    // data.
                    key.interestOps(SelectionKey.OP_READ);
     private void finishConnection(SelectionKey key) throws IOException {
          SocketChannel socketChannel = (SocketChannel) key.channel();
          // Finish the connection. If the connection operation failed
          // this will raise an IOException.
          try {
               socketChannel.finishConnect();
          } catch (IOException e) {
               // Cancel the channel's registration with our selector
               System.out.println(e);
               key.cancel();
               return;
          // Register an interest in writing on this channel
          key.interestOps(SelectionKey.OP_WRITE);
     private SocketChannel initiateConnection() throws IOException {
          // Create a non-blocking socket channel
          SocketChannel socketChannel = SocketChannel.open();
          socketChannel.configureBlocking(false);
          // Kick off connection establishment
          socketChannel.connect(new InetSocketAddress(this.host, this.port));
          // Queue a channel registration since the caller is not the
          // selecting thread. As part of the registration we'll register
          // an interest in connection events. These are raised when a channel
          // is ready to complete connection establishment.
          synchronized(this.pendingChanges) {
               this.pendingChanges.add(new ChangeRequest(socketChannel, ChangeRequest.REGISTER, SelectionKey.OP_CONNECT));
          return socketChannel;
     private Selector initSelector() throws IOException {
          // Create a new selector
          return SelectorProvider.provider().openSelector();
public class RspHandler {
     private byte[] rsp = null;
     public synchronized boolean handleResponse(byte[] rsp) {
          this.rsp = rsp;
          this.notify();
          return true;
     public synchronized void waitForResponse() {
          while(this.rsp == null) {
               try {
                    this.wait();
               } catch (InterruptedException e) {
          System.out.println(new String(this.rsp));
}          NioClient NioClient = new NioClient();
          Thread t = new Thread(NioClient);
          t.setDaemon(true);
          t.start();
          RspHandler handler = new RspHandler();
          NioClient.connect("69.28.156.250", 27040, handler);
          NioClient.connect("72.165.61.188", 27040, handler);
          NioClient.connect("208.111.133.84", 27011, handler);
          NioClient.connect("72.165.61.136", 27012, handler);
Edited by: 915967 on 01.08.2012 7:07

Thread blocking on java.nio.charset.CoderResult

Hello all,
I have a multi-threaded app which does some fairly intestive string operations (basically extracts text from documents for indexing a search system).
I am seeing a massive bottleneck around the java.nio.charset.CoderResult class. When profiling, I see a whole stack of threads blocking on (waiting for) a monitor on this java.nio.charset.CoderResult class. Seems to be a result of string encoding/decoding (I am often encoding strings as UTF-8).
Anyone know why the JVM would want my threads to sync on this class? It's creating a huge performance issue for my app. Approximately 15% of ALL the processing time is spent waiting for this class.
Help!

I would guess that you're using some of the static methods in the CoderResult class. The static methods CoderResult.unmappableCache(), CoderResult.malformedForLength() and CoderResult.malformedCache all use a static inner class called Cache. Its get() method is synchronized on Cache.class. Since the Cache inner-class is static, any part of your multi-threaded application that goes through the Cache.get() method is going to be waiting for the lock on Cache.class.
Could you create a CoderResult instance for each thread? That would mean that there would be a different static Cache class for each thread, reducing the number of threads competing for the Cache.class lock.
I'd have to see some of your code to give a better answer.
Brian

Java NIO, ByteBuffers and Linksys router

I have a client server app/game that uses NIO for communication sending ByteBuffers. On a LAN with 5-8 users it runs great. On the internet, through a Linksys router, with one user, it has a blip. I get all my data transmissions except for one buffer. Whenever I chat the buffer contains a size, an int typeID and the encoded string for chat. This particular buffer never makes it to the client on the outside of the router. I have a port forwarded and regular tcp/ip java io sockets stuff works fine. As does al lof the other NIO buffer traffic for locational data, login in and out, etc... ANy thoughts??

But not sure what would be the performance of those clients?? when compared to Java NIO performance....Telnet isn't a high-performance protocol anyway. Don't worry about it. Use existing code. Get it working. Measure. If you have a performance issue, then worry, while at least you have something you can deploy. It won't be a problem. The router is there to route, not to talk high-speed telnet.

Java.nio.channels.NonWritableChannelException

i have been facing this exception, please tell me how to remove it
java.nio.channels.NonWritableChannelException
here is my code
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
public class abc
     public static void main(String arg[])
          try
          File ff=new File("aaa.txt");
          FileInputStream fis=new FileInputStream(ff);
          FileChannel fc=fis.getChannel();
          FileLock lock=fc.lock();
          lock.release();
          fc.close();
          catch(Exception e)
               System.out.println("hello22"+e);
}

what should i do then to make the channel writable . please tell me what to change in the code, i just want to lock the file and after that i will modify it.

Java.nio.channels.FileChannel jar file

Hi
I have used java.nio.channels.FileChannel package in my program.
where can I get the jar file which has the above package?
Thanx

i give here sample code.. use this it will work fine..
import java.nio.*;
import java.nio.channels.*;
import java.io.*;
public class copyImage
public static void main(String [] args) throws IOException
try {
// Create channel on the source
FileChannel srcChannel = new FileInputStream("dragonfly.jpg").getChannel();
FileChannel dstChannel =new FileOutputStream("1.jpg").getChannel();
// Copy file contents from source to destination
dstChannel.transferFrom(srcChannel, 0, srcChannel.size());
// Close the channels
srcChannel.close();
dstChannel.close();
} catch (IOException e) {
}

Java.nio.channels.Pipe?

Someone explain what java.nio.channels.Pipe should be uses for, and how to use it?

You can use this class to transfer the content of a channel from one thread to another thread.
The writing thread blocks when the pipe is full.
package com.desoft.pipetest;
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
public class PipeTest {
public static void main (String args[])
try
FileInputStream fis = new FileInputStream(args[0]);
FileChannel fc = fis.getChannel();
Pipe p = Pipe.open();
PipeReader pr = new PipeReader( p );
Thread t = new Thread( pr );
t.start();
fc.transferTo(0, fc.size(), p.sink() );
catch( IOException ioe )
ioe.printStackTrace();
* Read from a Pipe and write Content to System.out
static class PipeReader implements Runnable
Pipe p;
public PipeReader( Pipe p )
this.p = p;
public void run()
try
ByteBuffer buffer = ByteBuffer.allocate( 10 );
int len = 0;
while( (len=p.source().read( buffer )) > 0 )
buffer.rewind();
for( int i = 0; i < len; i++ )
System.out.print( (char)buffer.get());
buffer.rewind();
catch( IOException ioe )
ioe.printStackTrace();

Performance of java nio with dd in linux.

Hi
I ran this code in java to dump 0s in a file of size 1gb and tried the same with dd in linux.
Java code : I use a preinitialized array (the data) and fill a mappedbytebuffer with it.
package filePersistence.test;
import java.io.File;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class TimeLineTest {
     private static final int BYTE_LENGTH = 1000;
     //this is the input
     static byte[] b = new byte[BYTE_LENGTH];
     public static void main(String[] arg) throws Exception
          //initializing the input
          for (int i=0;i<BYTE_LENGTH;i++)
               b[i] = 0;
          File file = new File(arg[0]);
          RandomAccessFile raFile = new RandomAccessFile(file,"rw");
          FileChannel fChannel = raFile.getChannel();
          int loopCount = 1000000;
          MappedByteBuffer mbuffer = fChannel.map( FileChannel.MapMode.READ_WRITE ,0,loopCount * BYTE_LENGTH);
          //System.out.println(" going to fill in a file" + file.getName());
          long k = 0;
          long startTime = System.currentTimeMillis();
          for (int i=0;i<loopCount;i++)
     //populate the mapped buffer
               mbuffer.put(b);
          //persist into the file
          mbuffer.force();
          long endTime = System.currentTimeMillis();
          System.out.println(" file filled size1 "+file.length());
          System.out.println(" time " + (endTime - startTime));
On a linux machine this takes around 7 secs while dd used as
"dd if=/dev/zero of=mytestfile.out bs=1000 count=1000000"
1000000+0 records in
1000000+0 records out
1000000000 bytes (1.0 GB) copied, 4.618 seconds, 217 MB/s
4.6 and 7 differ quite a lot. Is there a way the java code can be improved to match dd ?(-sever does not help)
Thanks
Sumanta

Hi
Can this be called a dd equivalent code ?
package filePersistence.test;
import java.io.File;
import java.io.FileOutputStream;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class TimeLineTest {
     private static final int BYTE_LENGTH = 1000;
     //this is the input
     static byte[] b = new byte[BYTE_LENGTH];
     public static void main(String[] arg) throws Exception
          //initializing the input
          for (int i=0;i<BYTE_LENGTH;i++)
               b[i] = 0;
          String srcFile = arg[0] + "_src";
          FileOutputStream fos = new FileOutputStream(srcFile);
          fos.write(b);
          fos.getFD().sync();
          fos.flush();
          fos.close();
          RandomAccessFile srcraFile = new RandomAccessFile(srcFile,"rw");
          FileChannel srcChannel = srcraFile.getChannel();
          File file = new File(arg[0]);
          RandomAccessFile raFile = new RandomAccessFile(file,"rw");
          FileChannel raChannel = raFile.getChannel();
          int loopCount = 1000000;
          //MappedByteBuffer mbuffer = fChannel.map( FileChannel.MapMode.READ_WRITE ,0,loopCount * BYTE_LENGTH);
          //System.out.println(" going to fill in a file" + file.getName());
          long startTime = System.currentTimeMillis();
          for (int i=0,position=0;i<loopCount;i++,position+=BYTE_LENGTH)
     //populate the mapped buffer
               //mbuffer.put(b);
               raChannel.position(position);
               srcChannel.transferTo(0, BYTE_LENGTH, raChannel);
          //persist into the file
          //mbuffer.force();
          raFile.getFD().sync();
          raChannel.close();
          raFile.close();
          long endTime = System.currentTimeMillis();
          System.out.println(" file filled size1 "+file.length());
          System.out.println(" time " + (endTime - startTime));
}

Any java.nio guru out there ?

Hi help,
I'm studying new java.io classes. I've compiled and learnt
a basic server like this...
import java.io.*;
import java.net.*;
import java.nio.*;
import java.nio.channels.*;
import java.util.*;
public class Server {
   private static int port = 9999;
   public static void main(String args[])
     throws Exception {
     Selector selector = Selector.open();
     ServerSocketChannel channel =
       ServerSocketChannel.open();
     channel.configureBlocking(false);
     InetSocketAddress isa = new InetSocketAddress(port);
     channel.socket().bind(isa);
     // Register interest in when connection
     channel.register(selector, SelectionKey.OP_ACCEPT);
     // Wait for something of interest to happen
     while (selector.select() > 0) {
       // Get set of ready objects
       Set readyKeys = selector.selectedKeys();
       Iterator readyItor = readyKeys.iterator();
       // Walk through set
       while (readyItor.hasNext()) {
         // Get key from set
         SelectionKey key =
           (SelectionKey)readyItor.next();
         // Remove current entry
         readyItor.remove();
         if (key.isAcceptable()) {
           // Get channel
           ServerSocketChannel keyChannel =
             (ServerSocketChannel)key.channel();
           // Get server socket
           ServerSocket serverSocket = keyChannel.socket();
           // Accept request
           Socket socket = serverSocket.accept();
           // Return canned message
           PrintWriter out = new PrintWriter
             (socket.getOutputStream(), true);
           out.println("Hello, NIO");
           out.close();
         } else {
           System.err.println("Ooops");
     // Never ends
} now I'd like to write a Proxy Server. In other
words the Server must
1. listen for a request
2. forward the request to another server
3. listen its reply
4. forward it back to the original caller.
I've tried to modify the basic class but with no success....
unfortunately I'm not that smart with these new Apis.
anybody can give me some guidelines about how should I
struct it ?
Thanks a lot
Francesco

Not a guru but I'll offer some suggestions. Are these persistent connections to the proxy, or is a connection made per request? You'll probably do things a little bit differently in the two different cases.
In general though, you can have one thread running a request dispatcher which accepts incoming connections and puts read ready channels on a queue. Another thread(or threads) can take requests off the queue and process them. Also, when you accept new connections, I think its better to do it through channels, rather than work directly with the socket. Something like this:
SocketChannel clientChannel = serverChannel.accept();
clientChannel.configureBlocking(false);
SelectionKey newReadKey = clientChannel.register(selector, SelectionKey.OP_READ);
Where are you getting into problems?

Using java.nio

I have copied this, one program from a book that would have to read file of gives to me to you mixed but it does not work, you could me why?
import java.io.FileInputStream;
import java.io.IOException;
import java.io.File;
import java.io.FileNotFoundException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
public class ReadPrimesMixedData {
public static void main(String[] args) {
File aFile = new File("C:/Beg Java Stuff/primes.txt");
FileInputStream inFile = null;
try {
inFile = new FileInputStream(aFile);
} catch(FileNotFoundException e) {
e.printStackTrace(System.err);
System.exit(1);
FileChannel inChannel = inFile.getChannel();
try {
ByteBuffer lengthBuf = ByteBuffer.allocate(8);
int strLength = 0; // Stores the string length
ByteBuffer buf = null; // Stores a reference to the second byte buffer
byte[] strChars = null; // Stores a reference to an array to hold the string
while(true) {
if(inChannel.read(lengthBuf) == -1) // Read the string length, if its EOF
break; // exit the loop
lengthBuf.flip();
strLength = (int)lengthBuf.getDouble(); // Extract the length and convert to int
buf = ByteBuffer.allocate(strLength+8); // Buffer for the string & the prime
if(inChannel.read(buf) == -1) { // Read the string & binary prime value
assert false; // Should not get here!
break; // Exit loop on EOF
buf.flip();
strChars = new byte[strLength]; // Create the array for the string
buf.get(strChars); // Extract string & binary prime value
System.out.println("String length: " + strChars.length+ " String: " +
new String(strChars) + " Binary value: " + buf.getLong());
lengthBuf.clear(); // Clear the buffer for the next read
System.out.println("\nEOF reached.");
inFile.close(); // Close the file and the channel
} catch(IOException e) {
e.printStackTrace(System.err);
System.exit(1);
System.exit(0);
This e' the program that writes the file
import java.io.*;
import java.nio.*;
import java.nio.channels.FileChannel;
public class PrimesToFile2 {
public static void main(String[] args) {
int primesRequired = 100; // Default count
if (args.length > 0) {
try {
primesRequired = Integer.valueOf(args[0]).intValue();
} catch (NumberFormatException e) {
System.out.println("Prime count value invalid. Using default of "
+ primesRequired);
long[] primes = new long[primesRequired]; // Array to store primes
primes[0] = 2; // Seed the first prime
primes[1] = 3; // and the second
// Count of primes found - up to now, which is also the array index
int count = 2;
long number = 5; // Next integer to be tested
outer:
for (; count < primesRequired; number += 2) {
// The maximum divisor we need to try is square root of number
long limit = (long) Math.ceil(Math.sqrt((double) number));
// Divide by all the primes we have up to limit
for (int i = 1; i < count && primes[i] <= limit; i++)
if (number % primes[i] == 0) // Is it an exact divisor?
continue outer; // yes, try the next number
primes[count++] = number; // We got one!
File aFile = new File("C:/Beg Java Stuff/primes.txt");
FileOutputStream outputFile = null;
try {
outputFile = new FileOutputStream(aFile);
} catch (FileNotFoundException e) {
e.printStackTrace(System.err);
System.exit(1);
FileChannel file = outputFile.getChannel();
final int BUFFERSIZE = 100; // Buffer size in bytes
ByteBuffer buf = ByteBuffer.allocate(BUFFERSIZE);
DoubleBuffer doubleBuf = buf.asDoubleBuffer();
buf.position(8);
CharBuffer charBuf = buf.asCharBuffer();
LongBuffer longBuf = null;
String primeStr = null;
for (int i = 0; i < primes.length; i++) {
primeStr = "prime = " + primes; // Create the string
doubleBuf.put(0,(double) primeStr.length());// Store the string length
charBuf.put(primeStr); // Store the string
buf.position(2 * charBuf.position() + 8); // Position for 3rd buffer
longBuf = buf.asLongBuffer(); // Create the buffer
longBuf.put(primes[i]); // Store the binary long value
buf.position(buf.position() + 8); // Set position after last value
buf.flip(); // and flip
try {
file.write(buf); // Write the buffer as before.
} catch (IOException e) {
e.printStackTrace(System.err);
System.exit(1);
buf.clear();
doubleBuf.clear();
charBuf.clear();
try {
System.out.println("File written is " + file.size() + " bytes.");
outputFile.close(); // Close the file and its channel
} catch (IOException e) {
e.printStackTrace(System.err);
System.exit(1);
System.exit(0);
This works correctly

I don't have problems when I compile the program but if I run the program ....I have this
String length: 9 String: prim Binary value: 7277852183226228736
String length: 0 String: Binary value: 162166969980682240
String length: 0 String: Binary value: 7854388801345436928
String length: 0 String: Binary value: 3573983215616
Stjava.lang.IllegalArgumentException
at java.nio.ByteBuffer.allocate(ByteBuffer.java:303)
at ReadPrimesMixedData.main(ReadPrimesMixedData.java:35)
ring length: 0 String: Binary value: 7566167222444367872
String length: 0 String: Binary value: 88089088
String length: 0 String: Binary value: 8214681170873443584
String length: 0 String: Binary value: 1856
String length: 0 String: Binary value: 8070575878335130880
Exception in thread "main"

Java.nio.BufferUnderflowException when send a idoc xml using sapjco3.jar

When I use sapjco3.jar to send a idoc
I have 2 IDOC ,one size is 200kb ,another is 336kb
the first one can send to sap correct, but the bigger one got Exception (java.nio.BufferUnderflowException)
If any thing I have to note when writing java code ? thanks ....
Exception:
send TXLOG-12918467-ToIDOC_20110513161427170_1125436.xml
send idoc D:\TXLOG-12918467-ToIDOC_20110513161427170_1125436.xml to SAP GateWay finished ,IDoc Tid is AC1140540B534DD1EAE61781
send TXLOG-12964100.xml
java.nio.BufferUnderflowException
     at java.nio.Buffer.nextGetIndex(Buffer.java:486)
     at java.nio.HeapCharBuffer.get(HeapCharBuffer.java:129)
     at com.sap.conn.idoc.rt.xml.DefaultIDocXMLParser.parse(DefaultIDocXMLParser.java:365)
     at com.sap.conn.idoc.rt.xml.DefaultIDocXMLProcessor.parse(DefaultIDocXMLProcessor.java:87)
     at com.sap.conn.idoc.rt.xml.DefaultIDocXMLProcessor.parse(DefaultIDocXMLProcessor.java:68)
     at com.testrite.msg.dex.sender.SAPSender.sendMessage(SAPSender.java:61)
     at com.testrite.msg.dex.sender.SAPSender.main(SAPSender.java:37)
My Code :
JCoDestination destination;
String dest = "BCE";
destination = JCoDestinationManager.getDestination(dest);
IDocRepository iDocRepository = JCoIDoc.getIDocRepository(destination);
String tid = destination.createTID();
IDocFactory iDocFactory = JCoIDoc.getIDocFactory();
IDocXMLProcessor processor = iDocFactory.getIDocXMLProcessor();
IDocDocumentList iDocList = processor.parse(iDocRepository, new BufferedInputStream(new FileInputStream("D:
TXLOG-12964100.xml"));
JCoIDoc.send(iDocList, IDocFactory.IDOC_VERSION_DEFAULT, destination, tid);

hi,
it only works with idoc to idoc scenarios I think
when you send from SAP to SAP through XI
have a look at my weblog:
/people/michal.krawczyk2/blog/2005/11/21/xi-idoc-to-idoc-tunneling--how-fast-and-easy-can-you-get
Regards,
michal

Java.nio.BufferOverflowException on big Strings ( 87MB)

Similar Messages

Maybe you are looking for