LVOOP DVR with non-blocking Dynamically-dispatched calls
OK, obscure question time again.
I have a current architecture where I have several different self-displaying classes inheriting from a common parent class. I have a dynamic dispatch method defined within the parent class which calls a UI specific for the class which enables me to present the user with the correct UI for edtiing the object properties depending on which object type is currenly being used in my software. Each class may or may not have different methods for manipulating the data, it's not just the UI which changes, the method of editing the data changes also so a common UI is impossible.
I have a central repository of these objects (instantiated according to the Factory method).
What I would like to do is have a method to retrieve the current selected object (Class unknown but certainly an ancestor of the common parent), call the "show" function (Dynamic dispatch) which then opens up the specific UI for editing. I can get this part working OK and have shown that the "By Ref" design is inherently working. But....
I want the editing of the object parameters (Within the Dynamically dispatched VI) updates to run asynchronously with the repository. I want to have the UI be able to run indefinitely but have the values cahnged be reflected in the repository, thus saving a huge amount of headaches in synchronising.
Class 1
Show()
Class 2 (Child of Class 1)
Show()
I want to store a DVR for Class 1 (Edit-time Class 1: Run-time: Class 1 or Class 2), retrieve the DVR, call the Show() method in an unblocking way (i.e. not simply calling Show() within an IPE) so that any changes made on the respective UI will be reflected imemdiately in the repository.
Any ideas?
This (Image below) is all I can come up with but I don't know if this is legit or not. I would have to add some checking for a destroyed DVR int he editor to cover a change oy repository object type at run-time.
Shane
Say hello to my little friend.
RFC 2323 FHE-Compliant
Ben,
I will always take time to try to understand what you mean, guaranteed. Either way, I think anyone who takes the time to try and understand MY posts (which can be severely rambling at times) deserves an award!
Shane.
PS Here's an updated version where each object signals its own changes via user events so that all editors (there may be many active at once, even with different UIs perhaps) stay current.
PPS the Clue is that I use the IPE to give me the ACTUAL object type without using the data within. The DVR passed to the same function is my gateway to the class data, thus I can call the editor asynchronously and have parallel synchronised editors running. This is cool, I think I'll be using this a lot.
Say hello to my little friend.
RFC 2323 FHE-Compliant
Attachments:
DVR Dynamic dispatch (2).zip 167 KB
Similar Messages
-
Broken Pipe with Non-blocking Socket
Hello,
I write a Unix Agent who connect on a Windows Server with socket...
Working well on Linux but on Solaris my problem is:
-When my agent is running before server at connection all seems OK: Connect, Select and Getsockopt but when I try to send data I have always EPIPE Signal but I can receive datas from server !
- When my agent is strarting after the server all it's Ok
I don't unserstand this appears on Solaris SPARC 8 and Solaris 9 Intel ...
Please Help is there something special with non-blocking sockets on Solaris ?
ThanksCan't help you much but what I would recommend is that you
insure that your pipes are opened for both read/write, even
though you are only going to read or write from it. This insures
that the pipe does not close down when you hit EOF. -
Using OCIBindDynamic with non-blocking connections
I need to use an OCI array interface for execute statements more than once per one request to server.
When I have called stored procedure or function in the non-blocking connection context using OCIBindDynamic for parameter binding, application have been crashed at random time.
I don't have any problems using default (blocking) mode.
Environment:
Oracle 8.1.7 release 3 for Windows
MS Visual C++ 6.0 compiler
Could anybody help me ?It's always possible in any read that the number of bytes read is less than the number of bytes requested. You need to keep reading until you have got everything you expected, and cope with every possible error condition on each iteration.
EJP -
NIO: Strange problem when using ByteBuffer with non-blocking SocketChannel
Hi,
I have a server that uses multiplexed, non-blocking I/O with java.nio. When a client connects, the server waits for the message: <system cmd="knock"/>, returns a message and disconnects the client. The clients are shortly serviced in less than a second.
But the server newer receive anything from about 20% of the clients - even though it is sent. Or with other words: it is received and the data is contained in the ByteBuffer - SocketChannel.read(ByteBuffer) - but a call to ByteBuffer.remaing() returns 0 !!
ByteBuffer receiveBuf = ByteBuffer.allocate(65536);
receiveBuf.clear(); // the code is elsewhere used for longer living clients
int readBytes = channel.read(receiveBuf);
receiveBuf.flip();
StringBuffer sb = new StringBuffer();
System.out.println(" * Remaining: "+receiveBuf.remaining()); // writes: ' * Remaining: 0'
System.out.println(" * Received: "+new String(receiveBuf.array())); // writes: ' * Received: <system cmd="knock"/>'
while(receiveBuf.remaining() >= 2) {
byte b = receiveBuf.get();
sb.append((char)b);
System.out.println(" * sb content: "+sb.toString()); // writes: ' * sb content: 'The ByteBuffer clearly receives the correct data, but the ByteBuffer.remaining() returns 0 and therefore the StringBuffer will never have any content.
The problem seems to occur randomly and for about 20% of the clients (simulated from the same computer and therefore has the same bandwidth and so on).
Anyone knows what is going on, and how to solve the problem !?It's always possible in any read that the number of bytes read is less than the number of bytes requested. You need to keep reading until you have got everything you expected, and cope with every possible error condition on each iteration.
EJP -
Combining LVOOP DVR with Asynchronous Dynamic Dispatch and Preserve Run-Time Class
OK, the title sounds like a cornucopia of LVOOP terms. But there's a reason. This is in a way an extension of THIS thread.
What I'm doing recently is creating a LVOOP approach to loading Completely Asynchronous UI elements into subpanels. This I have combined with a global repository for the objects (which are essentially singletons with a UI functionality) which are shared via DVR, thus eliminating a lot of synchronisation headaches).
This means that I can ahve a universal framework for launching the UI elements into a subpanel. The changes made on the Object there are automatically reflected in the global repository.
So far so good.
What I don't like too much is a combination of two seemingly awkward code constructs I need in order to keep things running.
Weird construct 1:
I have defined a "Launch UI" function in my parent class which is Dynamic Dispatch (Allowing each object to take care of launching its own UI). This takes a parent object DVR as a second input which I make sure is of the exact same type as the object type being invoked by using the code shown below. The ACTUAL Type of both inputs to the launch VI within the IPE are identical. This is guaranteed because I require each new class to override this function.
Here I pass the DVR from outside the IPE to the "Launch" VI but the Object obtained within the IPE retains information required for DD thus ensuring that the VI called for launching the UI is identical to the ACTUAL object type in the DVR. This works OK and by placing this weird construct WITHIN the parent class, abuse is minimised, it works fine and seems to have no major side-effects.
So now we have a VI running asynchronously in the background which belongs to a specific object but has a DVR which it *thinks* is of a Parent Type but, because of the steps taken earlier, is actually of the same type as the object itself.
In order to make use of the functionality defined in this actual object type, I need to continuously re-interpret the Object within the IPE as shown below. Otherwise only the Parent functionality is available.
If I am accessing only methods of the parent class, then the Preserve functionality is not needed.
Is there a more elegant way to do this? I find the net result of this code and type-juggling to be really useful and much easier to manage than the non-DVR route since the synchronisation issues disappear. By making the IPE usage near-atomic we remove the chances of deadlock.
All editing done in the UI of the asynchronous VI is reflected automatically in any subsequent usage of the DVR. Even if the DVRs are not shared between VIs, this makes (for me) the headache of synchronisation easier. If you start expanding this beyond the limits of a single VI, the benefits in Synchronisation become really huge. You can even have multiple UI objects operating on the same data in the background without extra synchronisation needs. The only synchronisation required is a global "Data updated" for the object in question whereby the UI elements simply update their indicators and controls from the DVR again. This is trivial.
Thus I am convinced that the net result of this is very beneficial.
My question is if there's a better, safer or more "official" way to do this?
I was about to start a new Idea for combining the "Preserve Run time Class" and the DVR Terminal of the IPE so that the casting is done automatically. We could then have a double input to the IPE, the DVR (of base type) plus the ACTUAL Type of the object but of course returning an error if the types are incompatible. It would be like an "Imposter" DVR input for the IPE which allows a re-interpretation of the object type.
Would all of this go away if we allowed Dynamic Dispatch to work with DVRs? Probably.
Shane
Say hello to my little friend.
RFC 2323 FHE-Compliant
Solved!
Go to Solution.You guys rock!
I didn't even think of casting the DVR like that. Kinds stupid of me but I never would have thought it would work. Cool.
Also, Yeah, the limitation of no IPE in the Launch VI was one I spotted quite early on. this is why my Launch VI also doesn't accept more data than is absolutely neccessara because a DVR access in that VI will of course cause a lockup. I have the system so far now that I can have a SINGLE Launch VI (Which is NOT overridden, so the limitation refers to only a single VI now which is certainly better.
So again guys, thanks for helping out an old newbie. I've been around for quite a while, have made many posts but I love the way I just keep learning from others in the Forum. This is just why I absolutely LOVE it here.
Shane.
Say hello to my little friend.
RFC 2323 FHE-Compliant -
Troubles with timeout using java.nio.channels and non-blocking sockets
Hello.
I have a server application that employs java.nio.channels with non-blocking sockets.
The server waits for connections. The client should connect and be first in sending data.
Timeouts are significant! If client exceeds the allowed time to send data, the server should break the connection.
The huge trouble I've discovered that I cannot control the timeout when client connects but remains silent.
My code looks as follows:
<pre>
Selector oSel;
SocketChannel oSockChan;
Socket oSock;
SelectionKey oSelKey;
Iterator<SelectionKey> oItSelKeys;
int iCurrState, iMask, iCount;
iCurrState = INT_SERVER_WORKING;
iMask = SelectionKey.OP_ACCEPT | SelectionKey.OP_CONNECT | SelectionKey.OP_READ | SelectionKey.OP_WRITE;
while ( iCurrState == INT_SERVER_WORKING )
try
*// retrieving next action*
iCount = oSel.select();
if ( iCount > 0 )
oItSelKeys = oSel.selectedKeys().iterator();
while ( oItSelKeys.hasNext() )
oSelKey = oItSelKeys.next();
oItSelKeys.remove();
if ( oSelKey.isValid() )
switch ( oSelKey.readyOps() & iMask ) {
case SelectionKey.OP_ACCEPT :
oSockChan = oSSockChan.accept();
oSockChan.configureBlocking(false);
oSock = oSockChan.socket();
oSock.setKeepAlive(true);
oSockChan.register(oSel,SelectionKey.OP_READ,new MyPacket(oSock.getInetAddress(),oSock.getPort()));
break;
case SelectionKey.OP_READ :
oSelKey.interestOps(0);
((MyPacket) oSelKey.attachment()).inRequest(); *// preparing request*
this.getReader().add(oSelKey); *// sending key to reading thread*
break;
case SelectionKey.OP_WRITE :
oSelKey.interestOps(0);
((MyRequest) oSelKey.attachment()).inResponse(); *// preparing response*
this.getWriter().add(oSelKey); *// sending key to writing thread*
break;
case SelectionKey.OP_CONNECT :
default :
*// nothing to do*
catch ( IOException oExcept )
*// do some actions*
</pre>
Timeouts are easily controlled by reading and writing threads (see OP_READ and OP_WRITE ).
But when a client just connects without consequent data send, the state of this connection remains as OP_ACCEPT. The connection remains open for arbitrarily large time and I cannot control it!
Please help with idea how can I terminate such connections!How can I process the keys that weren't selected at the bottom of the loop? Should I use the method keys() ?Yes. Form a new set from keys() and removeAll(selectedKeys()). Do that before you process selectedKeys().
And the second moment: as I understood a single key may contain several operations simultaneously? Thus I should use several if's (but not if/else 'cause it's the equivalent of switch ... case ).If there is anything unclear about 'your switch statement is invalid. You need an if/else chain' I fail to see what it is. Try reading it again. And if several ifs were really the equivalent of "switch ... case", there wouldn't be a problem in the first place. They're not, and there is. -
Persistant non-blocking operation problem
When using asynchronous database calls with non-blocking methods, I am getting a persistant error on the CreateSQL method call - the error is "OIP-04153: Non-Blocking operation in progress". This is raised even though the database connection has been closed and restarted and no non-blocking operations called. The methods used are similar to the following example code:
Dim OraDatabase as OraDatabase
Dim OraStmt as OraSQLStmt
Dim stat as long
Dim OraSess as OraSession
Set OraSess = CreateObject("OracleInProcServer.XOraSession")
Set OraDatabase =OraSess.OpenDatabase("ExampleDb", "scott/tiger", 0)
'execute the select statement with NONBLOCKING mode on
set OraStmt = OraDatabase.CreateSQL ("update emp set sal = sal + 1000", ORASQL_NONBLK)
'Check if the call has completed
stat = OraStmt.NonBlockingState
if stat = ORASQL_STILL_EXECUTING
MsgBox "Cancelling the asynchronous operation that is underway"
OraStmt.Cancel
End if
I cannot cancel the operation using OraStmt.Cancel because OraStmt Is Nothing!Actually, that is not correct. All of the 48 port Gig-E line cards for the 6500 are oversubscribed; it does vary by the degree of oversubscription.
In a 6506 or a 6509 (with a Sup720) you have (2) 20 Gbps connections (per slot) to the fabric. With a WS-X6748-GE-TX you have a potential for 48 (Full Duplex) Gbps of traffic, but only 2 connections to the fabric. This yields an oversubscription rate of 1.2:1 (or 48:40) assuming a worst case scenario.
It should be noted that the 24 port 67xx cards are still oversubscribed as well, because they only have one connection to the fabric. There is a potential for 24 Gbps of traffic and only (1) 20 Gbps fabric connection, so it has an identical oversubscription rate.
By using only 24 of the 48 possible ports, depending on which ones you choose, you should be OK. But I do not know which ports use which fabric connection on the backplane, so I can't assist you in picking them.
It should be noted that in most environments it is unlikely every port will be a 100% utilized so the risk of dropping packets because of oversubscription is low. -
Using non blocking connect() call for SCTP sockets in Solaris10
Hi,
I have a problem with non blocking connect call on SCTP socket.
I am using the sctp stack support in Solaris10.
When the connect is successful, I can get the pollout event on the socket.
But there is no event observed when the peer does not exist. In other words, I could not get the pollout event on connection failure. This logic works fine with TCP sockets on both Solaris and Suse10.
I am working with SCTP one-to-one style sockets.
Is there any way to handle this issue?
Do I need to load any patch to resolve this issue?
It will be great if I get a solution in this regard.
Thanks in advance.
Best Regards,
Bipin.There are at least two problems here.
A. In the receiver you should test for -1 from the read() immediately, rather than continue with the loop and try to write -1 bytes to the file.
B. In the sender you are ignoring the return value of client.write(), which can be anything from 0 to the buffer length. If you get 0 you should wait for another OP_WRITE to trigger; if you get a 'short write' you need to retry it until you've got nothing left to write from the current buffer, before you read any more data. This is where the data is vanishing. -
Non-blocking SSLEngine example
Since the example of using SSLEngine with non-blocking IO that comes with Java is quite limited, I have decided to release my own for anyone who wants to see how I solved the various problems that you must face. The example is designed to be a generic non-blocking server that supports SSL.
This is only meant to be an example, as I wrote this mostly in order to learn how to use the SSLEngine, and therefore has certain limitations, and is not thouroughly tested.
You can download the file at: http://members.aol.com/ben77/nio_server2.tar.gz
Here is also the code for SecureIO, which is roughly analagous to the Java example's ChannelIOSecure:
package nio_server2.internalio;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.*;
import java.util.concurrent.*;
import javax.net.ssl.*;
import static javax.net.ssl.SSLEngineResult.HandshakeStatus.*;
* Does IO based on a <code>SocketChannel</code> with all data encrypted using
* SSL.
* @author ben
public class SecureIO extends InsecureIO {
* SSLTasker is responsible for dealing with all long running tasks required
* by the SSLEngine
* @author ben
private class SSLTasker implements Runnable {
* @inheritDoc
public void run() {
Runnable r;
while ((r = engine.getDelegatedTask()) != null) {
r.run();
if (inNet.position() > 0) {
regnow(); // we may already have read what is needed
try {
System.out.println(":" + engine.getHandshakeStatus());
switch (engine.getHandshakeStatus()) {
case NOT_HANDSHAKING:
break;
case FINISHED:
System.err.println("Detected FINISHED in tasker");
Thread.dumpStack();
break;
case NEED_TASK:
System.err.println("Detected NEED_TASK in tasker");
assert false;
break;
case NEED_WRAP:
rereg(SelectionKey.OP_WRITE);
break;
case NEED_UNWRAP:
rereg(SelectionKey.OP_READ);
break;
} catch (IOException e) {
e.printStackTrace();
try {
shutdown();
} catch (IOException ex) {
ex.printStackTrace();
hsStatus = engine.getHandshakeStatus();
isTasking = false;
private SSLEngine engine;
private ByteBuffer inNet; // always cleared btwn calls
private ByteBuffer outNet; // when hasRemaining, has data to write.
private static final ByteBuffer BLANK = ByteBuffer.allocate(0);
private boolean initialHandshakeDone = false;
private volatile boolean isTasking = false;
private boolean handshaking = true;
private SSLEngineResult.HandshakeStatus hsStatus = NEED_UNWRAP;
private boolean shutdownStarted;
private static Executor executor = getDefaultExecutor();
private SSLTasker tasker = new SSLTasker();
private ByteBuffer temp;
* @return the default <code>Executor</code>
public static Executor getDefaultExecutor() {
return new ThreadPoolExecutor(3, Integer.MAX_VALUE, 60L,
TimeUnit.SECONDS, new SynchronousQueue<Runnable>(),
new DaemonThreadFactory());
private static class DaemonThreadFactory implements ThreadFactory {
private static ThreadFactory defaultFactory = Executors
.defaultThreadFactory();
* Creates a thread using the default factory, but sets it to be daemon
* before returning it
* @param r
* the runnable to run
* @return the new thread
public Thread newThread(Runnable r) {
Thread t = defaultFactory.newThread(r);
t.setDaemon(true);
return t;
* @return the executor currently being used for all long-running tasks
public static Executor getExecutor() {
return executor;
* Changes the executor being used for all long-running tasks. Currently
* running tasks will still use the old executor
* @param executor
* the new Executor to use
public static void setExecutor(Executor executor) {
SecureIO.executor = executor;
* Creates a new <code>SecureIO</code>
* @param channel
* the channel to do IO on.
* @param sslCtx
* the <code>SSLContext</code> to use
public SecureIO(SocketChannel channel, SSLContext sslCtx) {
super(channel);
engine = sslCtx.createSSLEngine();
engine.setUseClientMode(false);
int size = engine.getSession().getPacketBufferSize();
inNet = ByteBuffer.allocate(size);
outNet = ByteBuffer.allocate(size);
outNet.limit(0);
temp = ByteBuffer.allocate(engine.getSession()
.getApplicationBufferSize());
private void doTasks() throws IOException {
rereg(0); // don't do anything until the task is done.
isTasking = true;
SecureIO.executor.execute(tasker);
* Does all handshaking required by SSL.
* @param dst
* the destination from an application data read
* @return true if all needed SSL handshaking is currently complete.
* @throws IOException
* if there are errors in handshaking.
@Override
public boolean doHandshake(ByteBuffer dst) throws IOException {
if (!handshaking) {
return true;
if (dst.remaining() < minBufferSize()) {
throw new IllegalArgumentException("Buffer has only "
+ dst.remaining() + " left + minBufferSize is "
+ minBufferSize());
if (outNet.hasRemaining()) {
if (!flush()) {
return false;
switch (hsStatus) {
case FINISHED:
handshaking = false;
initialHandshakeDone = true;
rereg(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
return true;
case NEED_UNWRAP:
rereg(SelectionKey.OP_READ);
break;
case NEED_TASK:
doTasks();
return false;
case NOT_HANDSHAKING:
throw new RuntimeException(
"NOT_HANDSHAKING encountered when handshaking");
SSLEngineResult res;
System.out.println(hsStatus + "1" + handshaking);
switch (hsStatus) {
case NEED_UNWRAP:
int i;
do {
rereg(SelectionKey.OP_READ);
i = super.read(inNet);
if (i < 0) {
engine.closeInbound();
handshaking = false;
shutdown();
return true;
if (i == 0 && inNet.position() == 0) {
return false;
inloop: do {
inNet.flip();
temp.clear();
res = engine.unwrap(inNet, temp);
inNet.compact();
temp.flip();
if (temp.hasRemaining()) {
dst.put(temp);
switch (res.getStatus()) {
case OK:
hsStatus = res.getHandshakeStatus();
if (hsStatus == NEED_TASK) {
doTasks();
// if (hsStatus == FINISHED) {
// // if (!initialHandshakeDone) {
// // throw new RuntimeException(hsStatus
// // + " encountered when handshaking");
// initialHandshakeDone = true;
// handshaking=false;
// key.interestOps(SelectionKey.OP_READ
// | SelectionKey.OP_WRITE);
// TODO check others?
break;
case BUFFER_UNDERFLOW:
break inloop;
case BUFFER_OVERFLOW:
case CLOSED:
throw new RuntimeException(res.getStatus()
+ " encountered when handshaking");
} while (hsStatus == NEED_UNWRAP
&& dst.remaining() >= minBufferSize());
} while (hsStatus == NEED_UNWRAP
&& dst.remaining() >= minBufferSize());
if (inNet.position() > 0) {
System.err.println(inNet);
if (hsStatus != NEED_WRAP) {
break;
} // else fall through
rereg(SelectionKey.OP_WRITE);
case NEED_WRAP:
do {
outNet.clear();
res = engine.wrap(BLANK, outNet);
switch (res.getStatus()) {
case OK:
outNet.flip();
hsStatus = res.getHandshakeStatus();
if (hsStatus == NEED_TASK) {
doTasks();
return false;
// TODO check others?
break;
case BUFFER_OVERFLOW:
outNet.limit(0);
int size = engine.getSession()
.getPacketBufferSize();
if (outNet.capacity() < size) {
outNet = ByteBuffer.allocate(size);
} else { // shouldn't happen
throw new RuntimeException(res.getStatus()
+ " encountered when handshaking");
case BUFFER_UNDERFLOW: // engine shouldn't care
case CLOSED:
throw new RuntimeException(res.getStatus()
+ " encountered when handshaking");
} while (flush() && hsStatus == NEED_WRAP);
break;
case NEED_TASK:
doTasks();
return false;
case FINISHED:
break; // checked below
case NOT_HANDSHAKING:
System.err.println(hsStatus + " encountered when handshaking");
handshaking = false;
initialHandshakeDone = true;
rereg(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
if (hsStatus == FINISHED) {
// if (!initialHandshakeDone) {
// throw new RuntimeException(hsStatus
// + " encountered when handshaking");
initialHandshakeDone = true;
handshaking = false;
rereg(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
System.out.println(hsStatus + "2" + handshaking);
return !handshaking;
* Attempts to flush all buffered data to the channel.
* @return true if all buffered data has been written.
* @throws IOException
* if there are errors writing the data
@Override
public boolean flush() throws IOException {
if (!outNet.hasRemaining()) {
return true;
super.write(outNet);
return !outNet.hasRemaining();
* @return the largest amount of application data that could be read from
* the channel at once.
@Override
public int minBufferSize() {
return engine.getSession().getApplicationBufferSize();
* Begins or proceeds with sending an SSL shutdown message to the client.
* @return true if all needed IO is complete
* @throws IOException
* if there are errors sending the message.
@Override
public boolean shutdown() throws IOException {
if (!shutdownStarted) {
shutdownStarted = true;
engine.closeOutbound();
if (outNet.hasRemaining() && !flush()) {
return false;
SSLEngineResult result;
do {
outNet.clear();
result = engine.wrap(BLANK, outNet);
if (result.getStatus() != SSLEngineResult.Status.CLOSED) {
throw new IOException("Unexpected result in shutdown:"
+ result.getStatus());
outNet.flip();
if (outNet.hasRemaining() && !flush()) {
return false;
} while (result.getHandshakeStatus() == NEED_WRAP);
return !outNet.hasRemaining();
* Reads all possible data into the <code>ByteBuffer</code>.
* @param dst
* the buffer to read into.
* @return the number of bytes read, or -1 if the channel or
* <code>SSLEngine</code> is closed
* @throws IllegalStateException
* if the initial handshake isn't complete *
* @throws IOException
* if there are errors.
* @throws IllegalStateException
* if the initial handshake isn't complete
* @throws IllegalArgumentException
* if the remaining space in dst is less than
* {@link SecureIO#minBufferSize()}
@Override
public int read(ByteBuffer dst) throws IOException {
if (!initialHandshakeDone) {
throw new IllegalStateException("Initial handshake incomplete");
if (dst.remaining() < minBufferSize()) {
throw new IllegalArgumentException("Buffer has only "
+ dst.remaining() + " left + minBufferSize is "
+ minBufferSize());
int sPos = dst.position();
int i;
while ((i = super.read(inNet)) != 0
&& dst.remaining() >= minBufferSize()) {
if (i < 0) {
engine.closeInbound();
shutdown();
return -1;
do {
inNet.flip();
temp.clear();
SSLEngineResult result = engine.unwrap(inNet, temp);
inNet.compact();
temp.flip();
if (temp.hasRemaining()) {
dst.put(temp);
switch (result.getStatus()) {
case BUFFER_UNDERFLOW:
continue;
case BUFFER_OVERFLOW:
throw new Error();
case CLOSED:
return -1;
// throw new IOException("SSLEngine closed");
case OK:
checkHandshake();
break;
} while (inNet.position() > 0);
return dst.position() - sPos;
* Encrypts data and writes it to the channel.
* @param src
* the data to write
* @return the number of bytes written
* @throws IOException
* if there are errors.
* @throws IllegalStateException
* if the initial handshake isn't complete
@Override
public int write(ByteBuffer src) throws IOException {
if (!initialHandshakeDone) {
throw new IllegalStateException("Initial handshake incomplete");
if (!flush()) {
return 0;
int written = 0;
outer: while (src.hasRemaining()) {
outNet.clear(); // we flushed it
SSLEngineResult result = engine.wrap(src, outNet);
outNet.flip();
switch (result.getStatus()) {
case BUFFER_UNDERFLOW:
break outer; // not enough left to send (prob won't
// happen - padding)
case BUFFER_OVERFLOW:
if (!flush()) {
break outer; // can't remake while still have
// stuff to write
int size = engine.getSession().getPacketBufferSize();
if (outNet.capacity() < size) {
outNet = ByteBuffer.allocate(size);
} else { // shouldn't happen
throw new RuntimeException(hsStatus
+ " encountered when handshaking");
continue; // try again
case CLOSED:
throw new IOException("SSLEngine closed");
case OK:
checkHandshake();
break;
if (!flush()) {
break;
return written;
private boolean hasRemaining(ByteBuffer[] src) {
for (ByteBuffer b : src) {
if (b.hasRemaining()) {
return true;
return false;
* Encrypts data and writes it to the channel.
* @param src
* the data to write
* @return the number of bytes written
* @throws IOException
* if there are errors.
* @throws IllegalStateException
* if the initial handshake isn't complete
@Override
public long write(ByteBuffer[] src) throws IOException {
if (!initialHandshakeDone) {
throw new IllegalStateException("Initial handshake incomplete");
if (!flush()) {
return 0;
int written = 0;
outer: while (hasRemaining(src)) {
outNet.clear(); // we flushed it
SSLEngineResult result = engine.wrap(src, outNet);
outNet.flip();
switch (result.getStatus()) {
case BUFFER_UNDERFLOW:
break outer; // not enough left to send (prob won't
// happen - padding)
case BUFFER_OVERFLOW:
if (!flush()) {
break outer; // can't remake while still have
// stuff to write
int size = engine.getSession().getPacketBufferSize();
if (outNet.capacity() < size) {
outNet = ByteBuffer.allocate(size);
} else { // shouldn't happen
throw new RuntimeException(hsStatus
+ " encountered when handshaking");
continue; // try again
case CLOSED:
throw new IOException("SSLEngine closed");
case OK:
checkHandshake();
break;
if (!flush()) {
break;
return written;
private void checkHandshake() throws IOException {
// Thread.dumpStack();
// System.out.println(engine.getHandshakeStatus());
outer: while (true) {
switch (engine.getHandshakeStatus()) {
case NOT_HANDSHAKING:
initialHandshakeDone = true;
handshaking = false;
rereg(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
return;
case FINISHED:
// this shouldn't happen, I don't think. If it does, say
// where.
System.err.println("Detected FINISHED in checkHandshake");
Thread.dumpStack();
break outer;
case NEED_TASK:
if (isTasking) {
while (isTasking) { // TODO: deal with by reg?
Thread.yield();
try {
Thread.sleep(1);
} catch (InterruptedException ex) {
// TODO Auto-generated catch block
ex.printStackTrace();
break;
doTasks();
break;
case NEED_WRAP:
rereg(SelectionKey.OP_WRITE);
break outer;
case NEED_UNWRAP:
rereg(SelectionKey.OP_READ);
break outer;
handshaking = true;
hsStatus = engine.getHandshakeStatus();
* @return true if the channel is open and no shutdown message has been
* recieved.
@Override
public boolean isOpen() {
return super.isOpen() && !engine.isInboundDone();
}That's the reason for checkHandshake(), it is called on every read and write and detects a new handshake and configures the setup properly. As far as the server requesting a new handshake, I did not put that in. It would be simple enough though, you would just need to call SSLEngine.beginHandshake() + then call checkHandshake().
Also, my echo server example had a bug, I forgot to call bu.flip() before queueWrite(), so I fixed that, as well as adding an onConnect method that is called when a connection has been established. The new version should be up at the origional address shortly. -
1. Did I miss something: is there a way in java 1.5 (or even java 1.6) to read files and query file metadata (size, last modified, that kind of thing) in a non-blocking manner?
2. Is there a way to figure out where physical harddrive boundaries lie without resorting to tricks like running 'df' and parsing the output, which is rather un-portable? I obviously don't want to create a thread per file read (one aim is that the maximum number of threads is constant - even with a gazillion requests the amount of threads will never exceed a (low) constant.) but just 1 thread to perform all required reads would possibly end up wasting time if there are multiple harddrives and the CPU is idling anyway. I'd guess at this point that 1 thread per physical harddrive handling all file access is the best way to go, but how do I figure out those boundaries?
3. Is NIO2 (JSR 203) going to add features to work with files in a non-blocking way?
4. Is there a full-stack (templating, database access, writing web services, serving up the data) web server out there someplace that works entirely on a non-blocking methodology? I don't think so, but I might have missed something. JETTY tries something, but the servlet specs fundamentally do not work with non-blocking I/O, so it doesn't count.
background:
I'm writing a non-blocking webserver. KISS is the attitude - so far the basic system and any services (my take on the servlet spec, as the servlet spec doesn't play nice with NIO principles) on it all run inside a single thread, no thread pool.
So far it all works, and stress testing it with 1000 connections, 200 at the same time shows a process time of about half a millisecond per request, on a simple macbook, which is nice, because that beats apache even at serving up 404s.
So far I have only written very simple test apps that for example do calculations and the like, nothing that you'd actually do with web apps, like, say, serving (template-powered) data from files on a harddrive.
Because of the absolute requirement for each service NEVER to block while serving up bytes, manually working with files would be an unbelievable pain on the part of the web service developer. Hence I'm building templating and DB access into the system.
Unfortunately, I realized, reading NIO specs, that there doesn't seem to be any way to perform file operations in a non-blocking manner (not just reading them, but getting file size and last modified timestamps as well). I'll obviously have to use a lot of caching tactics, but before I can cache, I need to read files somehow.
The aim of this webserver is basically to survive massive spikes in traffic, be it from digg/slashdot, or from a DDOS attack, as best as possible, and where that isn't possible, to gracefully fall apart. That is, service as many requests as possible at a certain minimum bandwidth per request, prioritizing repeat visitors and site moderators, and deny the rest instantly, instead of just falling apart in shambles as happends in your usual LAMP or servlet design, not serving even 1 request right most of the time.
Thanks a lot for everyone's time!ejp wrote:
BufferedInputStream will improve the performance by a factor of several thousand because it cuts down on system calls.Well, that might be true if you're reading individual bytes with a call to read(), or making many small read( buffer ) requests. But if you call read( buffer ) with a reasonably sized buffer (BufferedInputStream uses 8192 bytes as a default buffer size), then you're going to get performance equal to or possibly better than a BufferedInputStream. -
Non-blocking execution with Pro*C/C++
Hi readers,
Is there any mechanism available through which Non-blocking execution with Pro*C/C++?
I observe that Pro*C queries are hanged when the DB server (hardware) is down.
I understand that OCI provides non-blocking mechanisms. Pro*C doesn't use this OCI functionality?
Regards,
RajaGopal MaddiYou can use TAF to handle your db down scenario.
http://www.oracle.com/technetwork/database/app-failover-oracle-database-11g-173323.pdf -
Execute query with non database block
How to execute query with non database block when new form instance trigger.
Hi Kame,
Execute_Query not work with non database block. To do this Make a cursor and then assign values to non database block's items programmatically, see following example,
DECLARE
BEGIN
FOR i IN (SELECT col1, col2 FROM Table) LOOP
:block.item1 := i.col1;
:block.item2 := i.col2;
NEXT_RECORD;
END LOOP;
END;
Please mark if it help you or correct
Regards,
Danish -
No payment block with non-valuated GR?
Hello gurus,
in our system, tolerance key PP is maintained to automatically set a payment block for the invoice, if the amount in PO and invoice differs too much.
Unfortunatelly, the payment block is created for items with valuated GR only. For items with non-valuated GR, there is no error, no warning, no nothing.
Any ideas on this?
Thanks
AliciaHi,
but where is the sense in that? In my view, an invoice that differs greatly from the PO should be blocked, no matter how/if we post the GR.
Am I missing something here?
Alicia -
What is wrong with my non-blocking client?
I have two classes here, my abstract base class SingleSocketHandler, and its concrete subclass SingleSocketHandlerImpl. The subclass implements the protocol and parsing of my server.
For some reason, my server is not receiving the packet my client sends to it, and my client is getitng nothing in return (which makes sense, the server is supposed to respond to the logon packet). I make it non-blocking AFTER logon, so I knwo that this is not a problem. Can you see why my server is not receiving the packet my client writes to it? Did I not configure some setting with the SocketChannel that enables it to write? I am sort of unfamiliar with the java.nio.channels package, so the problem may be related to a setting in the SocketChannel or whatnot that I haven't configured.
NOTE: My chat server works fine with my blocking, multi-threaded test clients. Just not for my non-blocking client. The original problem for my blocking clients was that once the server stopped sending them data, they'd get caught in the in.read() loop and never get out of it. That's why I turned to non-blocking.
Just to remind you, my question is: why isn't my client sending the logon packet AND/OR my server receiving+responding to it?
public abstract class SingleSocketHandler extends Thread
/* Subclasses must implement these methods
/* Even though they're not a (public) interface */
/** <------------------------------- */
abstract void parse(int num);
abstract void parseNext();
abstract void doLogon();
/** -------------------------------> */
private SocketChannel sock;
/* Queues for in and out buffers */
private LinkedList <ByteBuffer> qIn;
private LinkedList <ByteBuffer> qOut;
/* Server info */
private String hostname;
private int port;
/* Flags */
protected int flags;
protected final int LOGGED_ON = 0x01;
* Default Constructor
protected SingleSocketHandler()
initQs();
* Constructor that sets socket info
* @param hostname
* @param port
* @param connect
protected SingleSocketHandler(String hostname, int port, boolean connect)
initQs();
if (connect)
connect(hostname, port);
else
setSocket(hostname, port);
* Switches off between reading and writing
protected void handleIO()
try
sock.configureBlocking(false);
} catch (IOException e)
// TODO
readInBuffers(1);
writeOutBuffers(1);
* Read in specified number of buffers into in queue
* Call for parsing
* @param num
protected void readInBuffers(int num)
Reporter.println("READING BUFFER");
for (int i = 0; i < num; i++)
ByteBuffer header = ByteBuffer.allocate(ProtocolCheck.HEADER_LEN);
try
Reporter.println("Reading header...");
sock.read(header);
Reporter.println("Read header.");
} catch (IOException e)
// TODO
/* Only add packet to in queue if it has a valid header */
if (ProtocolCheck.validHeader(header.array()))
Reporter.println("valid header");
ByteBuffer packet = ByteBuffer.allocate(ProtocolCheck.findPacketLen(header.array()));
packet.put(header);
try
Reporter.println("Reading rest of packet...");
sock.read(packet);
Reporter.println("Read packet.");
} catch (IOException e)
// TODO
addInBuffer(packet);
* Write out specified number of buffers from out queue
* And remove from out queue
* @param num
protected void writeOutBuffers(int num)
Reporter.println("WRITING BUFFER");
int i = 0;
while (qOut.size() > 0 && i < num)
try
sock.write(nextOutBuffer());
Reporter.println("Wrote buffer.");
} catch (IOException e)
// TODO
i++;
* Returns and removes next buffer from in queue
* @return ByteBuffer
protected ByteBuffer nextInBuffer()
return qIn.remove();
* Returns and removes next buffer from out queue
* @return ByteBuffer
protected ByteBuffer nextOutBuffer()
return qOut.remove();
* Sees if there is anohter in buffer
* @return boolean
protected boolean hasNextInBuffer()
return qIn.size() > 0;
* Sees if there is another out buffer
* @return ByteBuffer
protected boolean hasNextOutBuffer()
return qOut.size() > 0;
* Add a buffer to in queue
* @param b
public void addInBuffer(ByteBuffer b)
qIn.add(b);
* Add a buffer to in queue
* @param b
public void addInBuffer(Bufferable b)
qIn.add(b.getByteBuffer());
* Add a buffer to out queue
* @param b
public void addOutBuffer(ByteBuffer b)
qOut.add(b);
* Add a buffer to out queue
* @param b
public void addOutBuffer(Bufferable b)
qOut.add(b.getByteBuffer());
* Instantiate queues
protected void initQs()
qIn = new LinkedList <ByteBuffer> ();
qOut = new LinkedList <ByteBuffer> ();
* Set socket info then call connect()
* @param hostname
* @param port
public void connect(String hostname, int port)
setSocket(hostname, port);
connect();
* Connect to server
public void connect()
try
sock = SocketChannel.open();
sock.configureBlocking(true);
sock.connect(new InetSocketAddress(hostname, port));
while (!sock.finishConnect())
} catch (IOException e)
// TODO
* Disconnect from server
public void disconnect()
try
sock.close();
} catch (IOException e)
// TODO
* Set socket info without connecting
* @param hostname
* @param port
public void setSocket(String hostname, int port)
this.hostname = hostname;
this.port = port;
* @return state of connection
public boolean isConnected()
return (sock != null && sock.isConnected());
* @return state of being logged on
public boolean isLoggedOn()
return (sock != null && (flags & LOGGED_ON) == LOGGED_ON);
public final class SingleSocketHandlerImpl extends SingleSocketHandler
private UserDatabase <User> users;
* Constructor that does not set socket info
public SingleSocketHandlerImpl(UserDatabase <User> users)
super();
this.users = users;
* Constructor that does set socket info
* @param hostname
* @param port
* @param connect
public SingleSocketHandlerImpl(String hostname, int port, boolean connect, UserDatabase <User> users)
super(hostname, port, connect);
this.users = users;
* Thread's run method (base class extends Thread)
public void run()
doLogon();
while (isConnected() && isLoggedOn())
handleIO();
* Parses specified number of buffers from in queue
* @param num
/* (non-Javadoc)
* @see client.SingleSocketHandler#parseNext()
@Override
protected void parse(int num)
Reporter.println("Parse(int num) called.");
int i = 0;
while (hasNextInBuffer() && i < num)
parseNext();
i++;
/* (non-Javadoc)
* @see client.SingleSocketHandler#parseNext()
@Override
protected void parseNext()
Reporter.println("Parsing!");
if (!hasNextInBuffer())
Reporter.println("NO IN BUFFER.");
return;
/* Get buffer to work with */
ByteBuffer inBuffer = nextInBuffer();
byte[] data = inBuffer.array();
/* Decide what to do based on message ID */
byte msgid = data[1];
switch (msgid) {
case 0x01:
Reporter.println("0x01 packet.");
/* Determine success of login */
byte success = data[3];
if (success == (byte) 1)
flags |= LOGGED_ON;
Reporter.println("Logged on!");
else
flags &= ~LOGGED_ON;
Reporter.println(" <eChat> Unable to logon. Check the hostname and port settings.");
break;
case 0x02:
/* Parse out text message */
byte[] txtmsgbytes = new byte[data.length - 3];
System.arraycopy(data, 3, txtmsgbytes, 0, txtmsgbytes.length);
String txtmsg = new String(txtmsgbytes);
Reporter.println(txtmsg);
break;
case 0x03:
System.out.println("Packet ID not yet handled.");
break;
case 0x04:
System.out.println("Packet ID not yet handled.");
break;
default:
System.out.println("validID() method is buggy.");
* I make it non-blocking after logon sequences
/* (non-Javadoc)
* @see client.SingleSocketHandler#doLogon()
@Override
protected void doLogon()
Reporter.println("DOING LOGON!");
User myUser = users.getCurr();
addOutBuffer(new ScpLogon(myUser.getUsername(), myUser.getPassword()));
writeOutBuffers(1);
readInBuffers(1);
parseNext();
}Oh, if this helps, this is what gets output to my GUI. I did a lot of outputs for debugging purposes.
[3:29:27 PM]: Connecting...
[3:29:27 PM]: Connected!
[3:29:27 PM]: Logging on...
[3:29:27 PM]: DOING LOGON!
[3:29:27 PM]: WRITING BUFFER
[3:29:27 PM]: Wrote buffer.
[3:29:27 PM]: READING BUFFER
[3:29:27 PM]: Reading header... -
Non-blocking JDBC with Oracle OCI
I have been monitoring performance for a batch type application in a production environment over the last couple of weeks and all indications are that the blocking JDBC actions are the cause of performance issues in the application. The data access layer is built on Hibernate. And while HIbernate is a fine ORM, and does a great job of creating lightweight, fast individual pieces of SQL, I fear it is the sum of these parts that is causing our problems. I consistently see the Hibernate BatcherImpl getResultSet as consuming the most exclusive time in processing. Of course this is related to the structure of the target database as a lot of this time is being spent in execution.
What appears to be happening is that the executing threads are blocking until they get a response from the database. We are currently using Thin connections, on 10.2.x drivers. On last reading our processor queue length was as high as 70!! This is a clear indication of a CPU bottleneck.
I was wondering if it is possible to configure non-blocking JDBC using the OCI type connections instead of Thin connections....
Any advice someone could give on this topic would be much appreciated.
Thanks much in advance,
JayHi Thomas,
Thank you for the reply. I am aware that only one statement can be executed on a connection at any given time, and we have connection pooling configured, so that's not a problem.
More specifically what I am looking for here is that when a thread executes a sql statement using a connection, it seems to block on the processor, it does not switch out, thus not allowing other work to be done while the database satisfies the sql statement. I was wondering if OCI connections were used would the thread switch out until a response was recieved from the database.
Maybe you are looking for
-
XML Publisher Template Builder for Adobe Acrobat
Need a XML Publisher Template Builder for adobe acrobat as is available for word.
-
I get an error when trying to install updates in Photoshop CS6
I get an error when trying to install updates in Photoshop CS6. It says "Update Failed; Updates could not be applied." Anybody else have this probelm?
-
Print Preview Sometimes Inaccurate in 33
MODERATOR, PLEASE DELETE. (The problem persists, but since I keep discovering new aspects to the behavior I decided it would be better if I kept testing and observing for a few days and then post, instead of posting now and editing repeatedly.)
-
How to lock my Primary Email Address
Hey there I have had my skype account hacked more than 5 times and Skype didnt give a **bleep** about it and i had to create a new account. My impersonator has scammed more than $500 from my clients. Is there anyway that I could lock the primary emai
-
Automator - Creating a service - to Copy and Paste Folders
I keep trying to create an Automator service to copy a set of folders and paste them in a location where I have right clicked, but I can not make it happen. Can someone please help me or point me in the right direction of some information. Thank you