IO versus NIO
Hi all,
Wow a new look to this forum!!!!
I was goin g through the IO and NIO packages in java.I understand that the new nio package as can do all the thing likethe old IO package but its a bit complex as compare to the old one.I am wondering mostly I see people using (even on this forum) new IO class , channel based, As far as I know the nio is more efficent then the io and is capable of doing more thing than the io package but by its not popular or used in daily coding.So do I really need to under stand this nio capability.
Many Thanks !!!
There are some useful utility classes in NIO, but its main advantage is that you can write
highly scalable servers with NIO. Using traditional java.net means, you most probably will
use lots of threads and this is a scarce system resource. As EJP once said, about 1000
simultaneous clients is the limit of traditional networking code. On the other hand,
it's less straightforward to use NIO. It requires more skill and the resulting code is less
maintainable or requires more skills to maintain.
Similar Messages
-
Dear all,
When reading some data from an input stream e.g. readline(), it is waiting for incoming data. Is it possible to set a timeout that make reading abort after a certain time?
I know how to use timers but I'm not familiar with any aborting procedures for readLine()
Thanks for any help in advance,
Ronimport java.io.*;
* @author Ian Schneider
public class HowFastAreYou implements Runnable {
public void run() {
try {
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.println("you typed " + in.readLine());
} catch (IOException ioe) {
ioe.printStackTrace();
public static final void main(String[] args) throws Exception {
Thread t = new Thread(new HowFastAreYou());
t.start();
Thread.sleep(1000);
t.interrupt();
System.out.println("too late buddy");
}Threads tutorial - there are two threads used in this example. The first thread is spawned by the jvm itself when main is invoked. This thread spawns the input thread. InputStreams are usually blocking (synchronous) , that is, the current thread will no return from a read call until data is present (versus nio, which provides support for an asynchronous API). So the spawned thread is waiting for input, but the main thread carries on, where it then sleeps for one second. If you haven't pressed return by then, the main thread "interrupts" the input thread. This means the blocking operation ends. -
ASCII character/string processing and performance - char[] versus String?
Hello everyone
I am relative novice to Java, I have procedural C programming background.
I am reading many very large (many GB) comma/double-quote separated ASCII CSV text files and performing various kinds of pre-processing on them, prior to loading into the database.
I am using Java7 (the latest) and using NIO.2.
The IO performance is fine.
My question is regarding performance of using char[i] arrays versus Strings and StringBuilder classes using charAt() methods.
I read a file, one line/record at a time and then I process it. The regex is not an option (too slow and can not handle all cases I need to cover).
I noticed that accessing a single character of a given String (or StringBuilder too) class using String.charAt(i) methods is several times (5 times+?) slower than referring to a char of an array with index.
My question: is this correct observation re charAt() versus char[i] performance difference or am I doing something wrong in case of a String class?
What is the best way (performance) to process character strings inside Java if I need to process them one character at a time ?
Is there another approach that I should consider?
Many thanks in advance>
Once I took that String.length() method out of the 'for loop' and used integer length local variable, as you have in your code, the performance is very close between array of char and String charAt() approaches.
>
You are still worrying about something that is irrevelant in the greater scheme of things.
It doesn't matter how fast the CPU processing of the data is if it is faster than you can write the data to the sink. The process is:
1. read data into memory
2. manipulate that data
3. write data to a sink (database, file, network)
The reading and writing of the data are going to be tens of thousands of times slower than any CPU you will be using. That read/write part of the process is the limiting factor of your throughput; not the CPU manipulation of step #2.
Step #2 can only go as fast as steps #1 and #3 permit.
Like I said above:
>
The best 'file to database' performance you could hope to achieve would be loading simple, 'known to be clean', record of a file into ONE table column defined, perhaps, as VARCHAR2(1000); that is, with NO processing of the record at all to determine column boundaries.
That performance would be the standard you would measure all others against and would typically be in the hundreds of thousands or millions of records per minute.
What you would find is that you can perform one heck of a lot of processing on each record without slowing that 'read and load' process down at all.
>
Regardless of the sink (DB, file, network) when you are designing data transport services you need to identify the 'slowest' parts. Those are the 'weak links' in the data chain. Once you have identified and tuned those parts the performance of any other step merely needs to be 'slightly' better to avoid becoming a bottleneck.
That CPU part for step #2 is only rarely, if every the problem. Don't even consider it for specialized tuning until you demonstrate that it is needed.
Besides, if your code is properly designed and modularized you should be able to 'plug n play' different parse and transform components after the framework is complete and in the performance test stage.
>
The only thing that is fixed is that all input files are ASCII (not Unicode) characters in range of 'space' to '~' (decimal 32-126) or common control characters like CR,LF,etc.
>
Then you could use byte arrays and byte processing to determine the record boundaries even if you then use String processing for the rest of the manipulation.
That is what my framework does. You define the character set of the file and a 'set' of allowable record delimiters as Strings in that character set. There can be multiple possible record delimiters and each one can be multi-character (e.g. you can use 'XyZ' if you want.
The delimiter set is converted to byte arrays and the file is read using RandomAccessFile and double-buffering and a multiple mark/reset functionality. The buffers are then searched for one of the delimiter byte arrays and the location of the delimiter is saved. The resulting byte array is then saved as a 'physical record'.
Those 'physical records' are then processed to create 'logical records'. The distinction is due to possible embedded record delimiters as you mentioned. One logical record might appear as two physical records if a field has an embedded record delimiter. That is resolved easily since each logical record in the file MUST have the same number of fields.
So a record with an embedded delimiter will have few fields than required meaning it needs to be combined with one, or more of the following records.
>
My files have no metadata, some are comma delimited and some comma and double quote delimited together, to protect the embedded commas inside columns.
>
I didn't mean the files themselves needed to contain metadata. I just meant that YOU need to know what metadata to use. For example you need to know that there should ultimately be 10 fields for each record. The file itself may have fewer physical fields due to TRAILING NULLCOS whereby all consecutive NULL fields at the of a record do not need to be present.
>
The number of columns in a file is variable and each line in any one file can have a different number of columns. Ragged columns.
There may be repeated null columns in any like ,,, or "","","" or any combination of the above.
There may also be spaces between delimiters.
The files may be UNIX/Linux terminated or Windows Server terminated (CR/LF or CR or LF).
>
All of those are basic requirements and none of them present any real issue or problem.
>
To make it even harder, there may be embedded LF characters inside the double quoted columns too, which need to be caught and weeded out.
>
That only makes it 'harder' in the sense that virtually NONE of the standard software available for processing delimited files take that into account. There have been some attempts (you can find them on the net) for using various 'escaping' techniques to escape those characters where they occur but none of them ever caught on and I have never found any in widespread use.
The main reason for that is that the software used to create the files to begin with isn't written to ADD the escape characters but is written on the assumption that they won't be needed.
That read/write for 'escaped' files has to be done in pairs. You need a writer that can write escapes and a matching reader to read them.
Even the latest version of Informatica and DataStage cannot export a simple one column table that contains an embedded record delimiter and read it back properly. Those tools simply have NO functionality to let you even TRY to detect that embedded delimiters exist let alone do any about it by escaping those characters. I gave up back in the '90s trying to convince the Informatica folk to add that functionality to their tool. It would be simple to do.
>
Some numeric columns will also need processing to handle currency signs and numeric formats that are not valid for the database inpu.
It does not feel like a job for RegEx (I want to be able to maintain the code and complex Regex is often 'write-only' code that a 9200bpm modem would be proud of!) and I don't think PL/SQL will be any faster or easier than Java for this sort of character based work.
>
Actually for 'validating' that a string of characters conforms (or not) to a particular format is an excellent application of regular expressions. Though, as you suggest, the actual parsing of a valid string to extract the data is not well-suited for RegEx. That is more appropriate for a custom format class that implements the proper business rules.
You are correct that PL/SQL is NOT the language to use for such string parsing. However, Oracle does support Java stored procedures so that could be done in the database. I would only recommend pursuing that approach if you were already needing to perform some substantial data validation or processing the DB to begin with.
>
I have no control over format of the incoming files, they are coming from all sorts of legacy systems, many from IBM mainframes or AS/400 series, for example. Others from Solaris and Windows.
>
Not a problem. You just need to know what the format is so you can parse it properly.
>
Some files will be small, some many GB in size.
>
Not really relevant except as it relates to the need to SINK the data at some point. The larger the amount of SOURCE data the sooner you need to SINK it to make room for the rest.
Unfortunately, the very nature of delimited data with varying record lengths and possible embedded delimiters means that you can't really chunk the file to support parallel read operations effectively.
You need to focus on designing the proper architecture to create a modular framework of readers, writers, parsers, formatters, etc. Your concern with details about String versus Array are way premature at best.
My framework has been doing what you are proposing and has been in use for over 20 years by three different major nternational clients. I have never had any issues with the level of detail you have asked about in this thread.
Throughout is limited by the performance of the SOURCE and the SINK. The processing in-between has NEVER been an issu.
A modular framework allows you to fine-tune or even replace a component at any time with just 'plug n play'. That is what Interfaces are all about. Any code you write for a parser should be based on an interface contract. That allows you to write the initial code using the simplest possible method and then later if, and ONLY if, that particular module becomes a bottlenect, replace that module with one that is more performant.
Your intital code should ONLY use standard well-established constructs until there is a demonstrated need for something else. For your use case that means String processing, not byte arrays (except for detecting record boundaries). -
Mysterious Teardown Problem with NIO
I have an NIO proxy server which relays state information from a number of servers to a number of clients. A requirement for this system is that we can restart the proxy server (together with a couple of other components) from a web app.
I tear down the proxy server by
- interrupting out of the read loop
- cancelling all keys associated with the channel
- closing the socket on the channel
- closing the channel.
I do this for for both the input (server side) channel and the output (client side) channel.
I then discard the proxy server, create a new one and start it up.
Now here's the mysterious bit. The new server operates just fine for a minute or two. And then it stops seeing any input.
The thread responsible for input reading freezes on the call to -
myKeysAdded = myAcceptKey.selector().select();and the stack looks something like this:
WindowsSelectorImpl$SubSelector.poll0(long, int, int[], int[], int[], long)
WindowsSelectorImpl$SubSelector.poll() line: 270
WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl$SubSelector) line: 252
WindowsSelectorImpl.doSelect(long) line: 133
WindowsSelectorImpl(SelectorImpl).lockAndDoSelect(long) line: 69
WindowsSelectorImpl(SelectorImpl).select(long) line: 80
WindowsSelectorImpl(SelectorImpl).select() line: 84
ProxyServer$BrowserSelector.run() line: 123If I completely destroy the process (and the jvm) and start a new one everything goes back to working normally.
Has anyone got any suggestions? I'm in hair-tearing mode on this one.
Thx TOTW.I tear down the proxy server by
- interrupting out of the read loopDon't you mean the select() loop?
- cancelling all keys associated with the channelUnnecessary.
- closing the socket on the channel
- closing the channel.Only one of these is necessary. Closing the socket closes the channel and vice versa.
When you have closed all the channels, you then need to call selector.selectNow() (search this forum for why), and then close the selector. -
Java.nio select() method return 0 in my client application
Hello,
I'm developing a simple chat application who echo messages
But my client application loop because the select() method return 0
This is my code
// SERVER
package test;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;
public class Server {
private int port = 5001;
public void work() {
try {
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.configureBlocking(false);
InetSocketAddress isa = new InetSocketAddress(port);
serverSocketChannel.socket().bind(isa);
Selector selector = Selector.open();
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
System.out.println("Listing on "+port);
while(selector.select()>0) {
Set keys = selector.selectedKeys();
for(Iterator i = keys.iterator(); i.hasNext();) {
SelectionKey key = (SelectionKey) i.next();
i.remove();
if (key.isAcceptable()) {
ServerSocketChannel keyChannel = (ServerSocketChannel)key.channel();
SocketChannel channel = keyChannel.accept();
channel.configureBlocking(false);
channel.register(selector, SelectionKey.OP_READ );
} else if (key.isReadable()) {
SocketChannel keyChannel = (SocketChannel) key.channel();
String m = Help.read(keyChannel );
Help.write(m.toUpperCase(), keyChannel );
} catch (IOException e) {
e.printStackTrace();
public static void main(String[] args) {
Server s = new Server();
s.work();
// CLIENT
package test;
import java.awt.event.KeyAdapter;
import java.awt.event.KeyEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;
public class Client extends JFrame {
private String host = "localhost";
private int port = 5001;
private SocketChannel socketChannel;
private Selector selector;
public void work() {
try {
socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false);
InetSocketAddress isa = new InetSocketAddress(host, port);
socketChannel.connect(isa);
selector = Selector.open();
socketChannel.register(selector, SelectionKey.OP_CONNECT | SelectionKey.OP_READ );
while(true) {
selector.select();
Set keys = selector.selectedKeys();
for(Iterator i = keys.iterator(); i.hasNext();) {
SelectionKey key = (SelectionKey) i.next();
i.remove();
if (key.isConnectable()) {
SocketChannel keyChannel = (SocketChannel) key.channel();
if (keyChannel.isConnectionPending()) {
System.out.println("Connected "+keyChannel.finishConnect());
} else if (key.isReadable()) {
SocketChannel keyChannel = (SocketChannel) key.channel();
String m = Help.read(keyChannel);
display(m);
} catch (IOException e) {
e.printStackTrace();
private void display(final String m) {
SwingUtilities.invokeLater(new Runnable() {
public void run() {
area.append(m+"\n");
textFieed.setText("");
private void sendMessage(final String m) {
Thread t = new Thread(new Runnable() {
public void run() {
try {
Help.write(m, socketChannel);
} catch (IOException e) {
e.printStackTrace();
t.start();
public Client() {
addWindowListener(new WindowAdapter() {
public void windowClosing(WindowEvent e) {
System.exit(1);
textFieed.addKeyListener(new KeyAdapter() {
public void keyPressed(KeyEvent e) {
if (e.getKeyCode()== KeyEvent.VK_ENTER) {
String m = textFieed.getText();
sendMessage(m);
area.setEditable(false);
getContentPane().add(textFieed, "North");
getContentPane().add(new JScrollPane(area));
setBounds(200, 200, 400, 300);
show();
private String messageToSend;
private JTextArea area = new JTextArea();
JTextField textFieed = new JTextField();
public static void main(String[] args) {
Client s = new Client();
s.work();
// HELPER CLASS
package test;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.nio.charset.CharsetEncoder;
public class Help {
private static Charset charset = Charset.forName("us-ascii");
private static CharsetEncoder enc = charset.newEncoder();
private static CharsetDecoder dec = charset.newDecoder();
private static void log(String m) {
System.out.println(m);
public static String read(SocketChannel channel) throws IOException {
log("*** start READ");
int n;
ByteBuffer buffer = ByteBuffer.allocate(1024);
while((n = channel.read(buffer)) > 0) {
System.out.println(" adding "+n+" bytes");
log(" BUFFER REMPLI : "+buffer);
buffer.flip();
CharBuffer cb = dec.decode(buffer);
log(" CHARBUFFER : "+cb);
String m = cb.toString();
log(" MESSAGE : "+m);
log("*** end READ");
//buffer.clear();
return m;
public static void write(String m, SocketChannel channel) throws IOException {
log("xxx start WRITE");
CharBuffer cb = CharBuffer.wrap(m);
log(" CHARBUFFER : "+cb);
ByteBuffer buffer = enc.encode(cb);
log(" BUFFER ALLOUE REMPLI : "+buffer);
int n;
while(buffer.hasRemaining()) {
n = channel.write(buffer);
System.out.println(" REMAINING : "+buffer.hasRemaining());
log("xxx end WRITE");Here's the fix for that old problem. Change the work method to do the following
- don't register interest in things that can't happen
- when you connect register based on whether the connection is complete or pending.
- add the OP_READ interest once the connection is complete.
This doesn't fix all the other problems this code will have,
eg.
- what happens if a write is incomplete?
- why does my code loop if I add OP_WRITE interest?
- why does my interestOps or register method block?
For code that answers all those questions see my obese post Taming the NIO Circus
Here's the fixed up Client code
// CLIENT
package test
import java.awt.event.KeyAdapter;
import java.awt.event.KeyEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;
public class Client extends JFrame {
private String host = "localhost";
private int port = 5001;
private SocketChannel socketChannel;
private Selector selector;
public void work() {
try {
socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false);
InetSocketAddress isa = new InetSocketAddress(host, port);
socketChannel.connect(isa);
selector = Selector.open();
int interest = 0;
if(socketChannel.isConnected())interest = SelectionKey.OP_READ;
else if(socketChannel.isConnectionPending())interest = SelectionKey.OP_CONNECT;
socketChannel.register(selector, interest);
while(true)
int nn = selector.select();
System.out.println("nn="+nn);
Set keys = selector.selectedKeys();
for(Iterator i = keys.iterator(); i.hasNext();)
SelectionKey key = (SelectionKey) i.next();
i.remove();
if (key.isConnectable())
SocketChannel keyChannel = (SocketChannel) key.channel();
System.out.println("Connected "+keyChannel.finishConnect());
key.interestOps(SelectionKey.OP_READ);
if (key.isReadable())
SocketChannel keyChannel = (SocketChannel) key.channel();
String m = Help.read(keyChannel);
display(m);
} catch (IOException e) {
e.printStackTrace();
private void display(final String m) {
SwingUtilities.invokeLater(new Runnable() {
public void run() {
area.append(m+"\n");
textFieed.setText("");
private void sendMessage(final String m) {
Thread t = new Thread(new Runnable() {
public void run() {
try {
Help.write(m, socketChannel);
} catch (IOException e) {
e.printStackTrace();
t.start();
public Client() {
addWindowListener(new WindowAdapter() {
public void windowClosing(WindowEvent e) {
System.exit(1);
textFieed.addKeyListener(new KeyAdapter() {
public void keyPressed(KeyEvent e) {
if (e.getKeyCode()== KeyEvent.VK_ENTER) {
String m = textFieed.getText();
sendMessage(m);
area.setEditable(false);
getContentPane().add(textFieed, "North");
getContentPane().add(new JScrollPane(area));
setBounds(200, 200, 400, 300);
show();
private String messageToSend;
private JTextArea area = new JTextArea();
JTextField textFieed = new JTextField();
public static void main(String[] args) {
Client s = new Client();
s.work(); -
Outputting DVI to HDMI input in LCD TV versus using the DVI to RGB ?
I am thinking of buying a mac mini to connect to a Toshiba 20HL85 TV in the lounge primarily to act as a "media server" and accessing Front Row for music, photo's, DVD watching etc. The question is which connection to use to go from the mini to the TV for Video (audio is fine as it will route into my AV Receiver)? The TV has an HDMI input, component video, s-video and a VGA port. The user manual for the TV says:
Note: The HDMI jack is not intended for connection to and should not be used with a personal computer. For PC connection, use VGA cable and connect to the VGA port. The VGA port allows the following resolutions:
Monitor Display modes:
VGA 640x480 60Hz
SVGA 800x600 56.3Hz
SVGA 800x600 60.3Hz
XGA 1024x768 60Hz
WXGA 1280x720 60Hz
WXGA 1280x768 60Hz
I wanted to know a few things as I have no experience with the MAC mini and HDMI:
1. Will I not get a better/crisper/sharper picture if I connect through a digital port like HDMI versus going through the VGA port?
2. I have checked a few LCD TV's user manuals and they all seem to say the same thing i.e. "Do not connect a PC to the HDMI port" - I fail to understand this when the Apple site specifically says under "accessories for the MAC mini" that "HDMI is electrically similar to DVI, but has a different physical connector that may include an audio signal. You’ll need a DVI to HDMI adapter, such as the Belkin PureAV HDMI to DVI cable to use these televisions".
3. What am I missing? Should I just go ahead and buy the DVI to HDMI cable and MAC mini and plug it in i.e. has anyone done this and had it work? Is it better than VGA (I would hope so)?
Bottom line is does the MAC mini connect to any TV with an HDMI port or are there different types of HDMI ports and hence I need to search for the right TV first? If so, what must I look for? Thanks for any help I can get.
Power Mac G5 Dual 2GHz Mac OS X (10.4.5)One word of advice. I just hooked up my mini to my 32" Lowe HD TV using a DVI to HDMI cable and it doesn't just work. The native resolution of the TV is not available for whatever reason so I have to bodge another resolution which looks terrible. When I plug my powerbook in via a DVI to VGA connector it just does work selecting the correct resolution automatically.
Either the problem lies with the mini or with connecting to the HDMI socket. As the HDMI is simply a dvi with audio I can't see why it would cause problems but who knows. Given all the other problems I am currently having with the mini I am pointing the finger in that direction currently. -
NIO Socket implementation - delay between select and get data from socket
Hi all,
I have implemented a internal CallAPI for RPC over a socket connection. It works fine but if there are more than five clients and some load I have the phenomena that the READ selector returns a SelectorKey but I did not get any data from the socket.
My implementation is based on NIO has following components:
+ Accept-Thread
Thread handles new clients.
+ Read-Thread
Thread handles the data from the socket for the registered client. Each request is handled in an own Process-Thread. A Thread-Pool implementation is used for processing.
+ Process-Thread
The Process-Thread reads the data from the socket and starts the processing of the logical request.
In my tests I get the notification of data at the socket. The Process-Thread want to read the data for the socket, but no data are available. In some situations if have to read about 20 times and more to get the data. Between each read attempt I have inserted a sleep in the Process-Thread if no data was available. This have improved the problem, but it already exists. I tested the problem with several systems and jvm's but it seams that it is independent from the system.
What can I to do improve the situation?
I already include the read implementation from the grizzly-Framework. But it doesn't improve the situation.
Socket - Init
protected void openSocket( String host, int port ) throws IOException
serverChannel = ServerSocketChannel.open();
serverChannel.configureBlocking( false );
serverSocket = serverChannel.socket();
serverSocket.setReuseAddress( true );
this.serverhost = host;
this.serverport = port;
this.srvAcceptSelector = Selector.open();
this.srvReadSelector = Selector.open();
InetSocketAddress isa = null;
if ( serverhost != null )
isa = new InetSocketAddress( this.serverhost, this.serverport );
else
isa = new InetSocketAddress( this.serverport );
serverSocket.bind( isa, 50 );
serverChannel.register( this.srvAcceptSelector, SelectionKey.OP_ACCEPT );
}New Client � Init
// New Client
if ( key.isAcceptable())
keyCountConnect++;
ServerSocketChannel actChannel =
(ServerSocketChannel) key.channel();
// Socket akteptieren
SocketChannel actSocket = actChannel.accept();
if ( actSocket != null )
actSocket.finishConnect();
actSocket.configureBlocking( false );
actSocket.socket().setTcpNoDelay( true );
this.registerSocketList.add( actSocket );
this.srvReadSelector.wakeup();
}Read Data from Socket
protected int readDatafromSocket( ByteArrayOutputStream socketdata )
throws IOException
int readedChars = 0;
int count = -1;
Selector readSelector = null;
SelectionKey tmpKey = null;
if ( sc.isOpen())
ByteBuffer inputbuffer = null;
try
inputbuffer = bufferpool.getBuffer();
while (( count = sc.read( inputbuffer )) > 0 )
readedChars += count;
inputbuffer.flip();
byte[] tmparray=new byte[inputbuffer.remaining()];
inputbuffer.get( tmparray );
socketdata.write( tmparray );
inputbuffer.clear();
if ( count < 0 )
this.closeSocket();
if( readedChars == 0 )
readedChars = -1;
if ( log.isDebug())
log.debug( "Socket is closed! " );
else if ( readedChars == 0 )
if ( log.isDebug())
log.debug( "Reread with TmpSelector" );
// Glassfish/Grizzly-Implementation
readSelector = SelectorFactory.getSelector();
if ( readSelector == null )
return 0;
count = 1;
tmpKey = this.sc.register( readSelector, SelectionKey.OP_READ );
tmpKey.interestOps(
tmpKey.interestOps() | SelectionKey.OP_READ );
int code = readSelector.select( 500 );
tmpKey.interestOps(
tmpKey.interestOps() & ( ~SelectionKey.OP_READ ));
if ( code == 0 )
return 0;
// Return on the main Selector and try again.
while (( count = sc.read( inputbuffer )) > 0 )
readedChars += count;
inputbuffer.flip();
byte[] tmparray=new byte[inputbuffer.remaining()];
inputbuffer.get( tmparray );
socketdata.write( tmparray );
inputbuffer.clear();
if ( count < 0 )
this.closeSocket();
if( readedChars == 0 )
readedChars =-1;
else if ( count == 0 )
// No data
finally
if ( inputbuffer != null )
bufferpool.releaseBuffer( inputbuffer );
inputbuffer = null;
// Glassfish-Implementierung
if ( tmpKey != null )
tmpKey.cancel();
if ( readSelector != null)
// Bug 6403933
try
readSelector.selectNow();
catch (IOException ex)
SelectorFactory.returnSelector( readSelector );
return readedChars;
}Thanks for your time.I've commented on that blog before. It is rubbish:
- what does 'overloading the main Selector' actually mean? if anything?
- 'Although this not clearly stated inside the NIO API documentation': The API documentation doesn't say anything about which Selector you should register channels with. Why would it? Register it with any Selector you like ...
- 'the cost of maintaining multiple Selectors can reduce scalability instead of improving it' Exactly. So what is the point again?
- 'wrapping a ByteBuffer inside a ByteBufferInputStream:' Code is rubbish and redundant. java.nio.channels.Channels has methods for this.
There is no a priori advantage to using multiple Selectors and threads unless you have multiple CPUs. And even then not much, as non-blocking reads and writes don't consume significant amounts of CPU. It's the processing of the data once you've got it that takes the CPU, and that should be done in a separate thread.
So I would re-evaluate your strategy. I suspect you're getting the channel registered with more than one Selector at a time. Implement it the simple way first then see if you really have a problem with 'overloading the main Selector' ... -
Billing doc cost PCA versus GL
The billing doc cost should be different for the PCA posting versus the GL posting. We have transfer pricing activated as well as material ledger. On the material master the standard price for legal valuation is different than profit center valuation and that is fine. When the billing doc posts it uses standard price for legal valuation for the GL posting. It should use standard price for profit center valuation. It uses legal valuation instead. Goods issue does use the correct standard price for GL versus PCA though.
How do I make the billing doc use the correct standard price ( profit center valuation ) for PCA?Hi,
I am a bit confuesed on scenario explained by you and I am tring to restate the same:
1. You are using Transfer Pricing and Material Ledger is active.
2. You have different prices updated in Standard Price for Legal and Profit Center Valuations respectively.
3. When you make a goods issue, the system uses the Standard Price in the Legal Valuatoin for accounting.
4. You need the system to use Profit Center Valuation Price for accounting.
Opinion.
The Goods issue as updated in the Financial Accounting should use only the Legal Valuation values. The Profit Center Document anyhow will have the cost of sales based on the profit center valuation. This is by design and correct approach.
If your question is related to values posted in Profitability Analysis. You need to activate profit center valuation in PA and then you will see that both values flow to Profitability Analysis.
Hope this helps you.
Varadharajan -
Why do I get IOException on nio read from channel before all data is read?
I am working on an NIO based http client library but I've noticed two problems.
#1: I have been testing against an http server that has authentication turned on. Hence, any request that comes its way will respond with a '401 Authentication Required' response. In the case of a POST, I've noticed that this particular http server does not wait for the body to arrive before responding. As soon as it gets the headers, it dumps the 401 response onto the channel and closes the connection. My library then gets an IOException on the next write to the channel even though I've received write interest event. (I'm using non blocking mode)
Whether I get the exception depends on timing. Sometimes, my writes work and the exception is averted. Sometimes not. The longer I delay since the time I get the write interest event, the more likely it is the exception will occur.
Now, in this situation, I really think it's the http server that is violating the http protocol. I give a valid content length that is not zero but it does not wait for me to finish my request. The moment I write the last empty line of the header section, it responds and closes the channel. Even though I don't think this is correct, my library has to deal with it. So in this situation, I've decided my library will abandon the write phase and move on to reading the response.
NOTE: In the majority of cases, a client would have already authenticated by landing on some page that could produce a POST in the first place. However, for technical reasons I can't mention here, I must support the case that the very first (unathenticated) request to the server is a POST.
2) After #1 above occurs, reading from the channel will also throw an IOException but only on Windows. Sometimes I get the headers only, sometimes the whole document, sometimes nothing. Again, the longer I delay reading (since the time I get the read interest event) the more likely it is the exception will be thrown. This NEVER happens on Linux. I always get the full response on Linux after the #1 happens even if I put long delays between reads.
I can understand #1 happening but #2 should not. I can always see the complete response data on the wire using a network protocol analyser. The data is there every time but my library doesn't always get it.
In situation 2, there is nothing I can do. If I don't receive the data, I have to respond with an error even though the request was successful.
I have stripped down my library to the bare essentials and can reproduce the problem in a self-contained environment every time.
Any ideas why the discrepency between Windows/Linux? Does this seem right?I read through the section you mentioned. I realize the connection may be dropped at any time. Handling this is no problem. It is reported as an exception to the client. However, this situation is different. I still see no provision that allows a server to respond to a half written request. A request is defined to include an optional message body (Section 5). The client dictates whether it is optional, not the server. Furthermore, section 6 clearly states "After receiving and interpreting a request message, a server responds with an HTTP response message.". To me, this implies the server must wait for the entire request before responding even if its a waste of time. This particular server is attempting to short circuit what I see is the required request/response cycle. In doing so, it is causing me some pain in trying to deliver its response since it seems on windows this interferes with my ability to read it.
The simple answer to this is to process reads before writes, i.e. OP_READ before OP_WRITE. I tried this as well. Originally, I was not even registering READ interest while I was writing my request. (I saw no reason to. I don't want to give my client a response while they are writing their request.) But even when I do register READ interest while I'm writing (and process read events before write), the read event still produces no data and no eof (-1) indication on the read. The discrepency between linux and windows is bothering me. No matter what I do, I can't deliver the response even though its written to the channel by my peer. I think I will supply some code and see if anyone can get it to work on windows.Edited by: r2rossi on Sep 24, 2008 10:42 AMEdited by: r2rossi on Sep 24, 2008 10:43 AM
Edited by: r2rossi on Sep 24, 2008 10:45 AM
Edited by: r2rossi on Sep 24, 2008 10:46 AM -
Problematic frame: # J java.nio.MappedByteBuffer.load()
Helo all...
I'm getting an error when generating a PDF using BIRT Framework in Eclipse. Nothing fancy, just compiling a simple report to pdf.
The problem is, the JVM crashes with the following error:
# A fatal error has been detected by the Java Runtime Environment:
# EXCEPTION_IN_PAGE_ERROR (0xc0000006) at pc=0x0190f0be, pid=5848, tid=2392
# JRE version: 7.0_09-b05
# Java VM: Java HotSpot(TM) Client VM (23.5-b02 mixed mode, sharing windows-x86 )
**# Problematic frame:
# J java.nio.MappedByteBuffer.load()Ljava/nio/MappedByteBuffer;**
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
# An error report file with more information is saved as:
# C:\Users\luis\Desktop\teste\birt\hs_err_pid5848.log
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
I already tried:
1. Run the program in another computers - it worked
2. Reinstall Java on the computer - It did nothing
3. I check the integrity of the disk C: - Nothing
I already spent two days trying to fix this and I don't have any progress. I can't find a solution on the web either.
Thanks in advanceThanks for the help.
But it can not be related to BIRT because it works in every other computer that I tried so far. There is only one computer that not works properly.
If it were BIRT's fault, it wouldn't work in any computer.
Thanks again -
Performance: Reports Builder versus Running on mid tier report server.
Folks are wondering why a report takes 15 minutes when running on the middle tier report server, paper layout, pdf output displayed via adobe, versus running the paper layout from a laptop within Reports Builder.
Any thoughts about the performance differences we now face with the 3 tier set up or suggestions on what we might tweak?hello,
unfortunately your post did not specify what the difference in performance is and whether you are connecting to the same database and what platform your deployment environment as oppsed to your dev environment is.
there are certainly some performance differences expected if the depoloyment platform is significantly different from the development paltform, but those differences should not be significant.
there are certain things that might cause performance differences (e.g. graphs in reports, using DISPLAY vs. DEFAULT_DISPLAY on unix platforms, etc.) so knowing a bit more about your platform would be helpful.
also it would be interesting to know if you see a similar degredation with other reports or even the test.rdf that comes with the product. last, but not least, have you tried to deploy your report to the OC4J that comes with the developer suite ? if so, do you see timings similar to the builder or more to your app server execution times ?
thanks,
philipp -
Authorized versus Associated.....trying to share content
What is the difference between a computer being AUTHORIZED to play content from an iTunes account versus a computer being ASSOCIATED with a device so it can be synced and play the content? It seems multiple computers can be AUTHORIZED to play content from multiple iTunes account but each device can only be ASSOCIATED with one iTunes account (or is it one computer) at a time. Is that correct?
If so, can all the AUTHORIZED content on a computer (even if from multiple iTunes accounts) be synced to one ASSOCIATED iPad?
I have a Work Computer (PC), Associated Work iPad and Work iTunes Account and a separate Home Computer (iMac), Associated Home iPad and Home iTunes account. I am trying to figure out how to share content? Currently all my content is in my Home iTunes Account / Computer. My Work iPad, Work iTunes Account are both new and I want to share some of my Home iTunes content on my Work iPad.
I am concerned when I read that once a device is ASSOCIATED with an iTunes account it can not be ASSOCIATED with a different account for 90 days. I can't risk ASSOCIATING my Work iPad with my Personal iTunes Account and being locked out from syncing my Work iTunes account on my Work iPad for 90 days.Okay, I figured it out myself and will document it here so others may benefit.
A computer can be AUTHORIZED to play/use content purchsed from multiple iTunes Accounts. There is a limit to the number of computers that can be AUTHORIZED per account (5 total) but no limit as to how many accounts a computer can be AUTHORIZED to play. Once you have AUTHORIZED all the accounts AND downloaded all the purchased content from each account to a computer, this content can be also be played on any device ASSOCIATED with the computer. A device can only be ASSOCIATED to one computer but one computer can have many devices ASSOCIATED to it. As an example, you can use apps purchased on from multiple iTunes accounts on an iPad as long as the computer the iPad is ASSOCIATED with is AUTHORIZED to use/play the content for each of the accounts. Another example: if you also purchased music from all the iTunes accounts, you could listen to it on an iPad, iPhone and iPod as long as all three were ASSOCIATED with the computer where all the accounts were AUTHORIZED.
Hope this helps. -
Multiple versus a single collection search with Verity
I have a simple question about Verity search collection.
We have been using verity for a number of years, but we have
never done
any real performance testing in regards to a single
collection versus many.
All documentation and articles argue for multiple small
collections when
indexing for better indexing performance. But what is the
performance
hit when searching 2 collections instead of 1 combined
collection?
How about 4 collections instead of 1?
Thanks
Don Vaillancourt
Director of Software Development
WEB IMPACT INC.
phone: 416-815-2000 ext. 245
fax: 416-815-2001
toll free: 866-319-1573 ext. 245
email: [email protected] <mailto:[email protected]>
blackberry: [email protected]
<mailto:[email protected]>
web:
http://www.web-impact.com
address:
http://maps.google.com
<
http://maps.google.com/maps?f=q&hl=en&q=99+atlantic+ave,+toronto&ie=UTF8&z=15&ll=43.640765 ,-79.420767&spn=0.013448,0.04343&om=1&iwloc=addr>
This email message is intended only for the addressee(s) and
contains
information that may be confidential and/or copyright.
If you are not the intended recipient please notify the
sender by reply
email and immediately delete this email.
Use, disclosure or reproduction of this email by anyone other
than the
intended recipient(s) is strictly prohibited. No
representation is made
that this email or any attachments are free of viruses. Virus
scanning
is recommended and is the responsibility of the recipient.We are searching 7 collections with some 100.000 documents. I
have not noticed any performance issues compared to searching only
one collection. -
Selector in NIO notifies data to be read but bytes read 0
Hi All,
I am facing a problem in NIO which for which I have clue.
The selector object notifies that data is ready to be read from a particular channel after which
I execute the following to read the data into the buffer
readBuffer.clear();
int count = this.sockChannel.read(this.readBuffer);
The value of count which i get is 0 after which which i return without any processing
but this notification goes on and on as I can see in the logs which goes on logging
Count received as 0 infinitely.
Please helpThere's a few things here.
1. I prefer to see only one read statement in these loops, like this:
while ((count = in.read(readBuffer)) > 0)
readBuffer.flip();
// get() from readBuffer into somewhere
// NB the 'len' to use is 'count'. You don't need to get it again via 'position()'.
// it generally makes more sense to use compact rather than clear() here
readBuffer.compact();
}2. count < 0 doesn' t mean the socket is closed, it means the connection is closed. You have to close the socket, or the channel, yourself, and I don't see that:
if (count < 0)
logger.debug("connection closed" ...); // etc
in.close();
}3. You're testing bytes.length > 0 in a place where it can't possibly be zero.
4. I hope you're not really ignoring exceptions as shown. If you are, that would explain the behaviour completely - never do that. If you get any kind of an IOException in this code, the connection is hosed and you must log the exception and close the channel. The only exception to this rule is SocketTimeoutException, which doesn't happen in non-blocking mode.
5. Is there any possibility that the buffer is zero length? That would cause read() to return zero. -
SSLSocket to support NIO - feature request voting
Hello All,
I know there were many discussions here regarding this subject.
I've seen the SSLEngine solution for the problem... But I didn't like it since it is too complicated.
For my understanding SSLSocket inherits Socket so an application that can handle Sockets should be able to handle SSLSocket without writing a specific code besides the factory.
NIO allows Socket to be none blocking... So why SSLSocket which is a Socket does not?
If I write a Web server in Java, why should I deal with none SSL and SSL connections in a different way?
I recently wrote one threaded NIO server and was amazed that I could not use SSL with the same implementation.
I've opened a feature request for Java, but they told me to gain support for this subject here before they will consider it...
As I understand I can re-implement SSLSocket to use SSLEngine and support NIO with blocking and none blocking support... But I think Sun should do it.
One caveat is to make sure that the SSLSocket HandshakeCompletedListener should be called by a daemon thread, so that it will not block other sockets while certificate verification occur.
Of course when I refer to SSLSocket I also refer to SSLServerSocket.
Can you please comment regarding this issue... Every comment will be welcomed!
Best Regards,
Alon Bar-Levnah, I am very glad they released an engine separate from transport allowing encryption on any transport.
I am also working on an abstraction of nio that you may be interested in if you don't want to do the security yourself......it is here...
http://sourceforge.net/projects/channelmanager
Right now, it only contains the api, but there are 3 implementations behind it right now with 3 to be added.
1. Basic implementation that does just nio
2. Secure implementation that implements same api and uses implementation 1
3. Packettizer implementation which puts things in payloads with headers and footers
4. (to be added) Threadpool implementation allowing a threadpool to be inserted in the stack somewhere
5. (to be added) TestProxy implementation allowing exxceptions to be thrown on purpopse to test a system on a bad network....can test failure on bind, connect, read, write, etc....everything.
6. denial of service layer.
7. Exception catch layer to protect #1 and #4 mainly from bad clients that throw exceptions back to the channelmanager.
Each one of these layers (2 - 4) implements the same api and uses the same api so 2 -4 are all proxies. You can reorganize the proxies as you want.
The secure one is almost done. Any comments on the api will be welcome.
thanks,
dean
Maybe you are looking for
-
Is there any way to force purchase an unsupported app?
I'm currently stuck on OS X 1.7.5 since my MacBook Pro isn't compatible with anything more recent than that. However, I'll be upgrading to a new Retina MacBook Pro very soon and I'd like to purchase an OS X app that requires 10.8, since it's currentl
-
Content Items Sorting based on picklist property fails
Content Server 6.0 We have developed a custom presentation template to display a list of all the work items. While displaying the work items, I would like to sort them based on a property and then filter them based on another property. I have a templ
-
Video set to loop, but has a slight pause at end
Hi, I have a video looping via AS3; only thing is the video slightly pauses at the end before it loops. Any one know how I could fix this? I have tried the following, but no matter which method, I get that pause: (METHOD 1) container.myVideo.autoRewi
-
When filling in forms I get invalid email but works on my laptop
Subject: When filling in forms on web sites I get invalid email but it works on my iPhone and laptop
-
How to close one booked sales order when its lines are all closed?
There's one so and its status is booked. But its lines' status is all closed. The so's status should be closed. And How can I do it?