Non-blocking write problem

Hi all,
I have an HLS IP that is interracting with memory interface and other IPs. I want to always consume a fifo input to avoid potential deadlock, so here is what my design looks like:
void top(stream<data_t>& in, stream<data_t>& out, ......) {
#pragma HLS DATAFLOW
        static stream<data_t> fifo;
        moveTo(in, fifo);
        main_function(fifo, .......);
main_function is the main processing unit, which interracts with other IPs and memory interface. The input fifo "in" cannot be blocked, so I just have to consume it whenever possible. My solution is to have an internal fifo and use function "moveTo" to consume it. Here is what my moveTo function looks like:
void moveTo(stream<data_t>& in, stream<data_t>& out) {
#pragma HLS PIPELINE II=1
         if (!in.empty()) {
                  data_t temp = in.read();
                  out.write_nb(tmp);
However, in hardware, I can see that the input fifo "in" is valid but not ready, which means the consumer funciton is blocked. I also tried to manually detect full signal like
      if(!out.full())
             out.write(tmp);
, but it doesn't make any difference. Is there any recommended way to do what I am trying to achieve? I used a similar syntax when writing to an external fifo and it was working fine in hardware, so I am not sure why it is not working with internal one. Note that all the interfaces are eventually implemneted with "#pragma HLS resource core=AXI4Stream variable=" pragma to be synthesized as axi_stream interface. 
Thanks,
Jimmy

So I am able to figure out that when the "main_function" is blocked by a write to an external fifo, the start signal of "moveTo" will be disabled too despite the fact that the internal fifo is not full yet. Shouldn't DATAFLOW make each function independent from each other?

Similar Messages

  • Why non-blocking write would block usually?

    I want to know that why my non-blocking socket's write will consume such a long time. sometimes 400ms worst.
    and this is my strace result:
    19:17:48.886276 write(1306, "\n\365\31:h#\t\7\237\20\3\21\337\36\276\317\6\3023\6\204\223,-\306\30\322\"\f\1\30\0\n"..., 121) = 121 <0.403088>
    It use 0.403088 seconds to finish the write, And I am sure of that the fd 1306 is a non-blocking socket.
    Another records shows:
    19:15:27.949736 write(995, "\n\365'\236UN\24\10\26Y\16\30\373\26\306\240\36\267z\10\20\342\f\17\301&\253\240\2\325\\\314\354"..., 160) = 160 <0.368412>
    19:15:28.410003 write(1466, "\n\365't\360`\20\10\20\342&\253\240\36\267z\10\26Y\26\306\240\16\30\373\2\325\\\f\17\301\314\354"..., 194) = 194 <0.199244>
    19:15:28.627854 write(1821, "\n\365'\260*s*\f\17\301\16\30\373\10\26Y\10\20\342\26\306\240\2\325\\&\253\240\36\267z\314\354"..., 160) = 160 <0.397758>
    19:15:29.029411 write(755, "\n\365'f\356\1\1&\253\240\36\267z\10\26Y\26\306\240\16\30\373\f\17\301\2\325\\\10\20\342\314\354"..., 160) = 160 <0.399817>
    19:15:29.440261 write(614, "\n\365'b\345H\6\36\267z&\253\240\10\26Y\2\325\\\26\306\240\10\20\342\f\17\301\16\30\373\314\354"..., 305) = 305 <0.396985>
    19:15:29.846446 write(892, "\n\365'\354\2256t\16\30\373\f\17\301\2\325\\\10\20\342\10\26Y&\253\240\26\306\240\36\267z\314\354"..., 160) = 160 <0.399170>
    19:15:30.250257 write(515, "\n\365'\376\343\214i\10\20\342\26\306\240\f\17\301&\253\240\10\26Y\2\325\\\36\267z\16\30\373\314\354"..., 231) = 231 <0.398990>
    19:15:30.652666 write(262, "\n\365'\274so\6&\253\240\10\26Y\2\325\\\26\306\240\36\267z\16\30\373\f\17\301\10\20\342\314\354"..., 168) = 168 <0.396569>
    19:17:46.885289 write(908, "t\213\16\222\0\0\0\0\240@\33L\0\0\3219\0\0\333J\0\0\2218\0\0\2\0|\203\16\222\0"..., 189) = 189 <0.396330>
    19:17:47.351419 write(1529, "\n\365\31\352\2\10\252\36\276\317\30\322\"\6\204\223\6\3023\f\1\30\7\237\20\3\21\337,-\306\0\n"..., 128) = 128 <0.293829>
    19:17:47.651080 write(683, "\n\365\31~ZG=\30\322\"\6\204\223\7\237\20\36\276\317\6\3023\3\21\337,-\306\f\1\30\0\n"..., 121) = 121 <0.398178>
    19:17:48.062419 write(175, "\n\365\31\224H\260\313\f\1\30\6\3023\3\21\337\36\276\317\6\204\223\7\237\20,-\306\30\322\"\0\n"..., 121) = 121 <0.398811>
    19:17:48.467481 write(569, "\n\365\31v5]H\3\21\337\30\322\"\f\1\30\6\3023\6\204\223\7\237\20,-\306\36\276\317\0\n"..., 121) = 121 <0.192335>
    19:17:48.663023 write(173, "\n\365\31\376\nJ\0\36\276\317,-\306\7\237\20\30\322\"\3\21\337\6\204\223\6\3023\f\1\30\0\n"..., 121) = 121 <0.206703>
    19:17:48.886276 write(1306, "\n\365\31:h#\t\7\237\20\3\21\337\36\276\317\6\3023\6\204\223,-\306\30\322\"\f\1\30\0\n"..., 121) = 121 <0.403088>
    BTW, This application is for the service of 3-4K client.
    And I do want to ask that:
    1.Why non-blocking write will make our process lose the CPU ( I think it is blocked ).
    2.What our process waiting for? Why not just return EAGAIN?
    3.Can we avoid this problem?
    Our server's info:
    Linux hz172-96 2.6.32-bpo.5-amd64 #1 SMP Mon May 2 11:40:03 UTC 2011 x86_64 GNU/Linux
    Looking forward to your answer, Thank you!

    thank you for your answer.
    and i have resolve this ploblem in using the linux kernel 2.6.30, using  kernel  2.6.32 before.
    but, i don't kown Is there some bug in linux kernel 2.6.32 or for other reason.

  • Persistant non-blocking operation problem

    When using asynchronous database calls with non-blocking methods, I am getting a persistant error on the CreateSQL method call - the error is "OIP-04153: Non-Blocking operation in progress". This is raised even though the database connection has been closed and restarted and no non-blocking operations called. The methods used are similar to the following example code:
    Dim OraDatabase as OraDatabase
    Dim OraStmt as OraSQLStmt
    Dim stat as long
    Dim OraSess as OraSession
    Set OraSess = CreateObject("OracleInProcServer.XOraSession")
    Set OraDatabase =OraSess.OpenDatabase("ExampleDb", "scott/tiger", 0)
    'execute the select statement with NONBLOCKING mode on
    set OraStmt = OraDatabase.CreateSQL ("update emp set sal = sal + 1000", ORASQL_NONBLK)
    'Check if the call has completed
    stat = OraStmt.NonBlockingState
    if stat = ORASQL_STILL_EXECUTING
    MsgBox "Cancelling the asynchronous operation that is underway"
         OraStmt.Cancel
    End if
    I cannot cancel the operation using OraStmt.Cancel because OraStmt Is Nothing!

    Actually, that is not correct. All of the 48 port Gig-E line cards for the 6500 are oversubscribed; it does vary by the degree of oversubscription.
    In a 6506 or a 6509 (with a Sup720) you have (2) 20 Gbps connections (per slot) to the fabric. With a WS-X6748-GE-TX you have a potential for 48 (Full Duplex) Gbps of traffic, but only 2 connections to the fabric. This yields an oversubscription rate of 1.2:1 (or 48:40) assuming a worst case scenario.
    It should be noted that the 24 port 67xx cards are still oversubscribed as well, because they only have one connection to the fabric. There is a potential for 24 Gbps of traffic and only (1) 20 Gbps fabric connection, so it has an identical oversubscription rate.
    By using only 24 of the 48 possible ports, depending on which ones you choose, you should be OK. But I do not know which ports use which fabric connection on the backplane, so I can't assist you in picking them.
    It should be noted that in most environments it is unlikely every port will be a 100% utilized so the risk of dropping packets because of oversubscription is low.

  • Non-trasactional write problem

    Hi all,
    I use query to retreive some objects and then i change the values of these
    objects not within a transaction. Later i use query retreive these objects
    again, but i found the values is changed? How can i retrevie back the
    original values?
    Thank you very much

    Try pm.refresh () or pm.evict (). Or begin a transaction as the objects
    will be restored back to their original values.
    On Thu, 14 Aug 2003 11:35:50 +0800, Tim wrote:
    Hi all,
    I use query to retreive some objects and then i change the values of these
    objects not within a transaction. Later i use query retreive these objects
    again, but i found the values is changed? How can i retrevie back the
    original values?
    Thank you very much--
    Steve Kim
    [email protected]
    SolarMetric Inc.
    http://www.solarmetric.com

  • SELECT of OCILobLocator in Non Blocking mode

    Hi all.
    I have problem:
    I try select BLOB field into OCILobLocator* buffer in Non Blocking mode, and receive 'ORA-03106: fatal two-task communication protocol error' sometimes.
    Details:
    Application contains two threads : 1. select blob field, 2. update it.
    I tried separate update and select to different apps (and run its in one time) - problem reproduced.
    problem reproduced for OCIStmtFetch and OCIStmtFetch2.
    If disable non blocking mode - problem disappear.
    if error appear - record level error code contains 1405. Need wait few minutes for error message appearing (~1 minute in my case).
    Using of OCI_DEFAULT in OCIDefineByPos instead OCI_DYNAMIC_FETCH don't solve problem.
    I use oracle server and client library with version 10.2.0.4
    Output sample:
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #877: 1405
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #54: 1405
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #877: 1405
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #54: 1405
    Can anybody help me ?
    m.b. exists workaround ?
    #include <iostream>
    #include <sstream>
    #include <assert.h>
    #include <oci.h>
    #define _DYN 1
    before run this test run next sql at DB:
    CREATE TABLE ADSERVERTEST_BLOB(id NUMBER(10), text BLOB);
    BEGIN
    FOR i IN 0..999 LOOP
    INSERT INTO ADSERVERTEST_BLOB(id, text) VALUES(i, utl_raw.cast_to_raw('init'));
    END LOOP;
    END;
    template<typename Type, unsigned long TypeId>
    class OCIHandlePtr
    public:
    OCIHandlePtr(): handle_(0) {}
    OCIHandlePtr(Type* handle): handle_(handle) {}
    ~OCIHandlePtr()
    reset(0);
    void reset(Type* handle)
    if(handle_)
    sword result;
    if((result = OCIHandleFree(
    handle_,
    TypeId)) != OCI_SUCCESS)
    assert(0);
    std::cerr << "Can't make OCIHandleFree" << std::endl;
    handle_ = handle;
    Type* get()
    return handle_;
    Type*& fill()
    reset(0);
    return handle_;
    private:
    Type* handle_;
    template<typename Type, unsigned long TypeId>
    class OCIDescriptorPtr
    public:
    OCIDescriptorPtr(): handle_(0) {}
    OCIDescriptorPtr(Type* handle): handle_(handle) {}
    ~OCIDescriptorPtr()
    reset(0);
    void reset(Type* handle)
    if(handle_)
    sword result;
    if((result = OCIDescriptorFree(
    handle_,
    TypeId)) != OCI_SUCCESS)
    std::cerr << "Can't make OCIDescriptorFree" << std::endl;
    handle_ = handle;
    Type* get()
    return handle_;
    Type*& fill()
    reset(0);
    return handle_;
    private:
    Type* handle_;
    void throw_oci_error(
    const char* fun,
    const char* oci_op,
    long status,
    OCIError* oci_error_handler,
    const char* query = 0)
    std::cerr << fun << ": can't make " << oci_op << ": status = " << status;
    if(status == OCI_SUCCESS_WITH_INFO ||
    status == OCI_ERROR)
    std::cerr << ", message = '";
    text err_buf[1024] = "";
    sb4 err_code = 0;
    OCIErrorGet(
    oci_error_handler,
    (ub4)1,
    0,
    &err_code,
    err_buf,
    sizeof(err_buf),
    (ub4)OCI_HTYPE_ERROR);
    std::cerr << err_buf << "', error code = " << err_code;
    exit(1);
    if(query)
    std::cerr << ", sql = " << query;
    OCIHandlePtr<OCIEnv, OCI_HTYPE_ENV> environment_handle;
    void connect(
    OCIEnv* environment_handle,
    const char* db,
    const char* user,
    const char* password,
    OCIHandlePtr<OCIError, OCI_HTYPE_ERROR>& error_handle,
    OCIHandlePtr<OCIServer, OCI_HTYPE_SERVER>& server_handle,
    OCIHandlePtr<OCISession, OCI_HTYPE_SESSION>& session_handle,
    OCIHandlePtr<OCISvcCtx, OCI_HTYPE_SVCCTX>& svc_context_handle)
    static const char* FUN = "connect()";
    sword result;
    // allocate an error handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **) &error_handle.fill(),
    OCI_HTYPE_ERROR,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // allocate a server handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **) &server_handle.fill(),
    OCI_HTYPE_SERVER,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    if((result = OCIServerAttach(
    server_handle.get(),
    error_handle.get(),
    (text*)db,
    strlen(db),
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIServerAttach", result, error_handle.get());
    // allocate a service handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **) &svc_context_handle.fill(),
    OCI_HTYPE_SVCCTX,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // set the server attribute in the service context handle
    if((result = OCIAttrSet(
    svc_context_handle.get(),
    OCI_HTYPE_SVCCTX,
    server_handle.get(),
    sizeof(OCIServer*),
    OCI_ATTR_SERVER,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // allocate a user session handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **)&session_handle.fill(),
    OCI_HTYPE_SESSION,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // set username and password attributes in user session handle
    if((result = OCIAttrSet(
    session_handle.get(),
    OCI_HTYPE_SESSION,
    (text*)user,
    strlen(user),
    OCI_ATTR_USERNAME,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    if((result = OCIAttrSet(
    session_handle.get(),
    OCI_HTYPE_SESSION,
    (text*)password,
    strlen(password),
    OCI_ATTR_PASSWORD,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    // start the session
    if((result = OCISessionBegin(
    svc_context_handle.get(),
    error_handle.get(),
    session_handle.get(),
    OCI_CRED_RDBMS,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCISessionBegin", result, error_handle.get());
    // set the user session attribute in the service context handle
    if((result = OCIAttrSet(
    svc_context_handle.get(),
    OCI_HTYPE_SVCCTX,
    session_handle.get(),
    sizeof(OCISession*),
    OCI_ATTR_SESSION,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    ub1 attr_value = 1;
    if((result = OCIAttrSet(
    server_handle.get(),
    OCI_HTYPE_SERVER,
    &attr_value,
    sizeof(attr_value),
    OCI_ATTR_NONBLOCKING_MODE,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    void disconnect(
    OCIEnv* environment_handle,
    OCIError* error_handle,
    OCIServer* server_handle,
    OCISession* session_handle,
    OCISvcCtx* svc_context_handle)
    static const char* FUN = "disconnect()";
    sword result;
    if((result = OCISessionEnd(
    svc_context_handle,
    error_handle,
    session_handle,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCISessionEnd", result, error_handle);
    if((result = OCIServerDetach(
    server_handle,
    error_handle,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIServerDetach", result, error_handle);
    OCILobLocator* FETCH_BUFFER_1[10000*1024];
    char IND_BUFFER_1[10000*1024];
    ub2 ERROR_BUFFER_1[1024];
    OCIDefine* define_handle_1;
    void clear_blob_buf(
    OCILobLocator** buf,
    unsigned long fetch_size)
    for(unsigned long i = 0; i < 1024; ++i)
    OCIDescriptorFree(
    buf,
    (ub4)OCI_DTYPE_LOB);
    ::memset(buf, 0, fetch_size * 1024);
    uint32_t RET_LEN4;
    sb4 oci_blob_callback(
    dvoid* ctx,
    OCIDefine* define,
    ub4 iter,
    dvoid** bufpp,
    ub4** alenpp,
    ub1* piecep,
    dvoid** indpp,
    ub2** rcodepp)
    RET_LEN4 = sizeof(OCILobLocator*);
    *bufpp = FETCH_BUFFER_1[iter];
    *alenpp = &RET_LEN4;
    *piecep = OCI_ONE_PIECE;
    *indpp = IND_BUFFER_1 + iter;
    *rcodepp = ERROR_BUFFER_1 + iter;
    //std::cout << "iter: " << iter << std::endl;
    return OCI_CONTINUE;
    int define_column(
    OCIEnv* environment_handle,
    OCISvcCtx* svc_handle,
    OCIStmt* stmt_handle,
    OCIError* error_handle,
    OCIDefine** define_handle,
    unsigned long pos,
    long oci_type,
    unsigned long fetch_size,
    OCILobLocator** buf,
    char* ind_buf,
    ub2* err_buf)
    static const char* FUN = "define_column()";
    sword result;
    if(oci_type == SQLT_BLOB)
    ::memset(buf, 0, fetch_size * 1024);
    for(int i = 0; i < 1024; ++i)
    if((result = OCIDescriptorAlloc(
    (dvoid*)environment_handle,
    (dvoid**)&buf[i],
    (ub4)OCI_DTYPE_LOB,
    (size_t)0,
    (dvoid**)0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIDescriptorAlloc", result, error_handle);
    ub2 size = 0;
    OCIDescriptorPtr<OCIParam, OCI_DTYPE_PARAM> param_handle;
    // ub2 oci_data_type = 0;
    if((result = OCIParamGet(
    stmt_handle,
    OCI_HTYPE_STMT,
    error_handle,
    reinterpret_cast<void**>(&param_handle.fill()),
    pos)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIParamGet", result, error_handle);
    if((result = OCIAttrGet(
    param_handle.get(),
    OCI_DTYPE_PARAM,
    &size,
    0,
    OCI_ATTR_DATA_SIZE,
    error_handle)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrGet", result, error_handle);
    if((result = OCIDefineByPos(
    stmt_handle,
    define_handle,
    error_handle,
    pos,
    # ifdef _DYN
    0,
    -1,
    #else
    buf,
    fetch_size,
    #endif
    oci_type,
    ind_buf,
    0,
    err_buf, // ptr to array of column-level return codes
    # ifdef _DYN
    OCI_DYNAMIC_FETCH
    # else
    OCI_DEFAULT
    # endif
    )) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIDefineByPos", result, error_handle);
    # ifdef _DYN
    if((result = OCIDefineDynamic(
    *define_handle,
    error_handle,
    0,
    oci_blob_callback)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIDefineByPos", result, error_handle);
    # endif
    return 0;
    int define_columns(
    OCIEnv* environment_handle,
    OCISvcCtx* svc_handle,
    OCIStmt* stmt_handle,
    OCIError* error_handle)
    static const char* FUN = "define_columns()";
    define_handle_1 = 0;
    int pos = 1;
    ::memset(ERROR_BUFFER_1, 0, 1024 * sizeof(ERROR_BUFFER_1[0]));
    define_column(
    environment_handle,
    svc_handle,
    stmt_handle,
    error_handle,
    &define_handle_1,
    pos++,
    SQLT_BLOB,
    sizeof(OCILobLocator*),
    FETCH_BUFFER_1,
    IND_BUFFER_1,
    ERROR_BUFFER_1);
    return 0;
    int execute_select(
    const char* db,
    const char* user,
    const char* password)
    static const char* FUN = "execute_select()";
    const unsigned long FETCH_COUNT = 1024;
    OCIHandlePtr<OCIError, OCI_HTYPE_ERROR> error_handle;
    OCIHandlePtr<OCIServer, OCI_HTYPE_SERVER> server_handle;
    OCIHandlePtr<OCISession, OCI_HTYPE_SESSION> session_handle;
    OCIHandlePtr<OCISvcCtx, OCI_HTYPE_SVCCTX> svc_context_handle;
    connect(
    environment_handle.get(),
    db,
    user,
    password,
    error_handle,
    server_handle,
    session_handle,
    svc_context_handle);
    OCIHandlePtr<OCIStmt, OCI_HTYPE_STMT> stmt_handle;
    sword result;
    if((result = OCIHandleAlloc(
    environment_handle.get(),
    (void**)&stmt_handle.fill(),
    OCI_HTYPE_STMT,
    0,
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    const char QUERY[] = "SELECT text FROM ADSERVERTEST_BLOB";
    ub4 rows = 0;
    if((result = OCIAttrSet(
    stmt_handle.get(),
    OCI_HTYPE_STMT,
    (dvoid *)&rows,
    (ub4)sizeof(ub4),
    OCI_ATTR_PREFETCH_ROWS,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(
    FUN, "OCIAttrSet(OCI_ATTR_PREFETCH_ROWS)", result, error_handle.get());
    if((result = OCIStmtPrepare(
    stmt_handle.get(),
    error_handle.get(),
    (text*)QUERY,
    strlen(QUERY),
    OCI_NTV_SYNTAX,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIStmtPrepare", result, error_handle.get());
    ub2 stmt_type;
    if((result = OCIAttrGet(
    stmt_handle.get(),
    OCI_HTYPE_STMT,
    &stmt_type,
    0,
    OCI_ATTR_STMT_TYPE,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrGet", result, error_handle.get());
    ub4 prev_fetched = 0;
    while((result = OCIStmtExecute(
    svc_context_handle.get(),
    stmt_handle.get(),
    error_handle.get(),
    0,
    0,
    0,
    0,
    OCI_DEFAULT)) != OCI_SUCCESS)
    if(result != OCI_STILL_EXECUTING)
    throw_oci_error(FUN, "OCIStmtExecute", result, error_handle.get());
    define_columns(
    environment_handle.get(),
    svc_context_handle.get(),
    stmt_handle.get(),
    error_handle.get());
    ub4 all_fetched_count = 0;
    while(true)
    while((result = OCIStmtFetch2(
    stmt_handle.get(),
    error_handle.get(),
    FETCH_COUNT,
    OCI_FETCH_NEXT,
    1,
    OCI_DEFAULT)) == OCI_STILL_EXECUTING
    # ifdef _DYN
    // || result == OCI_NEED_DATA
    # endif
    if (result != OCI_SUCCESS &&
    result != OCI_NO_DATA &&
    result != OCI_SUCCESS_WITH_INFO)
    std::cerr << FUN << ": can't make OCIStmtFetch2: status = " << result;
    std::cerr << ", message = '";
    text err_buf[1024] = "";
    sb4 err_code = 0;
    OCIErrorGet(
    error_handle.get(),
    (ub4)1,
    0,
    &err_code,
    err_buf,
    sizeof(err_buf),
    (ub4)OCI_HTYPE_ERROR);
    std::cerr << err_buf << "', error code = " << err_code << ", row errors: ";
    for(const ub2* cur = ERROR_BUFFER_1;
    cur < ERROR_BUFFER_1 + 1024; ++cur)
    if(*cur)
    std::cerr << "#" << (cur - ERROR_BUFFER_1) << ": " <<
    *cur << std::endl;
    ub4 fetched_count = 0;
    if((result = OCIAttrGet (
    stmt_handle.get(),
    OCI_HTYPE_STMT,
    &fetched_count,
    0,
    OCI_ATTR_ROW_COUNT,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrGet", result, error_handle.get());
    all_fetched_count += fetched_count;
    if(fetched_count > 0)
    for(unsigned long i = 0; i < fetched_count - 1; ++i)
    ub4 lob_size;
    while((result = OCILobGetLength(
    svc_context_handle.get(),
    error_handle.get(),
    FETCH_BUFFER_1[i],
    &lob_size)) == OCI_STILL_EXECUTING)
    if(result != OCI_SUCCESS)
    throw_oci_error(FUN, "OCILobGetLength", result, error_handle.get());
    std::cout << "#" << i << ": ind = " << (unsigned long)IND_BUFFER_1[i] <<
    ", len = " << lob_size << std::endl;
    if(fetched_count - prev_fetched < FETCH_COUNT)
    break;
    prev_fetched = fetched_count;
    clear_blob_buf(
    FETCH_BUFFER_1,
    sizeof(OCILobLocator*));
    while((result = OCIStmtFetch(
    stmt_handle.get(),
    error_handle.get(),
    0,
    OCI_FETCH_NEXT,
    OCI_DEFAULT)) == OCI_STILL_EXECUTING
    disconnect(
    environment_handle.get(),
    error_handle.get(),
    server_handle.get(),
    session_handle.get(),
    svc_context_handle.get());
    return 0;
    int execute_update(
    const char* db,
    const char* user,
    const char* password)
    static const char* FUN = "execute_update()";
    OCIHandlePtr<OCIError, OCI_HTYPE_ERROR> error_handle;
    OCIHandlePtr<OCIServer, OCI_HTYPE_SERVER> server_handle;
    OCIHandlePtr<OCISession, OCI_HTYPE_SESSION> session_handle;
    OCIHandlePtr<OCISvcCtx, OCI_HTYPE_SVCCTX> svc_context_handle;
    connect(
    environment_handle.get(),
    db,
    user,
    password,
    error_handle,
    server_handle,
    session_handle,
    svc_context_handle);
    OCIHandlePtr<OCIStmt, OCI_HTYPE_STMT> stmt_handle;
    for(int i = 0; i < 1000; ++i)
    sword result;
    if((result = OCIHandleAlloc(
    environment_handle.get(),
    (void**)&stmt_handle.fill(),
    OCI_HTYPE_STMT,
    0,
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    std::ostringstream sql;
    std::string str;
    std::string et("\xEB\xB2\x88");
    unsigned long rep = ::rand() % 500 + 1;
    str.reserve(rep*et.size());
    for(unsigned long ei = 0; ei < rep; ++ei)
    str.append(et);
    sql << "BEGIN " <<
    "UPDATE ADSERVERTEST_BLOB SET text = "
    "utl_raw.cast_to_raw('" << str << "') "
    "WHERE id = " << i <<
    "END;";
    if((result = OCIStmtPrepare(
    stmt_handle.get(),
    error_handle.get(),
    (text*)sql.str().c_str(),
    sql.str().length(),
    OCI_NTV_SYNTAX,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIStmtPrepare", result, error_handle.get());
    while((result = OCIStmtExecute(
    svc_context_handle.get(),
    stmt_handle.get(),
    error_handle.get(),
    1,
    0,
    0,
    0,
    OCI_DEFAULT)) != OCI_SUCCESS)
    if(result != OCI_STILL_EXECUTING)
    throw_oci_error(FUN, "OCIStmtExecute", result, error_handle.get());
    disconnect(
    environment_handle.get(),
    error_handle.get(),
    server_handle.get(),
    session_handle.get(),
    svc_context_handle.get());
    return 0;
    void* select_thread(void* ctx)
    while(true)
    execute_select(
    "//oraclept.ocslab.com/addbpt.ocslab.com",
    "bs_unittest_4",
    "adserver");
    // std::cout << "select finished" << std::endl;
    return ctx;
    void* update_thread(void* ctx)
    while(true)
    execute_update(
    "//oraclept.ocslab.com/addbpt.ocslab.com",
    "bs_unittest_4",
    "adserver");
    // std::cout << "update finished" << std::endl;
    return ctx;
    int main()
    static const char* FUN = "main()";
    sword result;
    // allocate an environment handle
    if((result = OCIEnvCreate(
    &environment_handle.fill(),
    OCI_OBJECT | OCI_THREADED,
    0, // context
    0, // malloc
    0, // realloc
    0, // free
    0, // extra memory to allocate
    0) // pointer to user-memory
    ) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIEnvCreate", result, 0);
    pthread_t th1, th2;
    pthread_create(&th1, 0, select_thread, 0);
    pthread_create(&th2, 0, update_thread, 0);
    pthread_join(th1, 0);
    pthread_join(th2, 0);
    return 0;
    Edited by: user13011391 on 20.04.2010 5:50
    Edited by: user13011391 on 20.04.2010 6:00
    Edited by: user13011391 on 20.04.2010 6:15

    Hi all.
    I have problem:
    I try select BLOB field into OCILobLocator* buffer in Non Blocking mode, and receive 'ORA-03106: fatal two-task communication protocol error' sometimes.
    Details:
    Application contains two threads : 1. select blob field, 2. update it.
    I tried separate update and select to different apps (and run its in one time) - problem reproduced.
    problem reproduced for OCIStmtFetch and OCIStmtFetch2.
    If disable non blocking mode - problem disappear.
    if error appear - record level error code contains 1405. Need wait few minutes for error message appearing (~1 minute in my case).
    Using of OCI_DEFAULT in OCIDefineByPos instead OCI_DYNAMIC_FETCH don't solve problem.
    I use oracle server and client library with version 10.2.0.4
    Output sample:
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #877: 1405
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #54: 1405
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #877: 1405
    execute_select(): can't make OCIStmtFetch2: status = -1, message = 'ORA-03106: fatal two-task communication protocol error
    ', error code = 3106, row errors: #54: 1405
    Can anybody help me ?
    m.b. exists workaround ?
    #include <iostream>
    #include <sstream>
    #include <assert.h>
    #include <oci.h>
    #define _DYN 1
    before run this test run next sql at DB:
    CREATE TABLE ADSERVERTEST_BLOB(id NUMBER(10), text BLOB);
    BEGIN
    FOR i IN 0..999 LOOP
    INSERT INTO ADSERVERTEST_BLOB(id, text) VALUES(i, utl_raw.cast_to_raw('init'));
    END LOOP;
    END;
    template<typename Type, unsigned long TypeId>
    class OCIHandlePtr
    public:
    OCIHandlePtr(): handle_(0) {}
    OCIHandlePtr(Type* handle): handle_(handle) {}
    ~OCIHandlePtr()
    reset(0);
    void reset(Type* handle)
    if(handle_)
    sword result;
    if((result = OCIHandleFree(
    handle_,
    TypeId)) != OCI_SUCCESS)
    assert(0);
    std::cerr << "Can't make OCIHandleFree" << std::endl;
    handle_ = handle;
    Type* get()
    return handle_;
    Type*& fill()
    reset(0);
    return handle_;
    private:
    Type* handle_;
    template<typename Type, unsigned long TypeId>
    class OCIDescriptorPtr
    public:
    OCIDescriptorPtr(): handle_(0) {}
    OCIDescriptorPtr(Type* handle): handle_(handle) {}
    ~OCIDescriptorPtr()
    reset(0);
    void reset(Type* handle)
    if(handle_)
    sword result;
    if((result = OCIDescriptorFree(
    handle_,
    TypeId)) != OCI_SUCCESS)
    std::cerr << "Can't make OCIDescriptorFree" << std::endl;
    handle_ = handle;
    Type* get()
    return handle_;
    Type*& fill()
    reset(0);
    return handle_;
    private:
    Type* handle_;
    void throw_oci_error(
    const char* fun,
    const char* oci_op,
    long status,
    OCIError* oci_error_handler,
    const char* query = 0)
    std::cerr << fun << ": can't make " << oci_op << ": status = " << status;
    if(status == OCI_SUCCESS_WITH_INFO ||
    status == OCI_ERROR)
    std::cerr << ", message = '";
    text err_buf[1024] = "";
    sb4 err_code = 0;
    OCIErrorGet(
    oci_error_handler,
    (ub4)1,
    0,
    &err_code,
    err_buf,
    sizeof(err_buf),
    (ub4)OCI_HTYPE_ERROR);
    std::cerr << err_buf << "', error code = " << err_code;
    exit(1);
    if(query)
    std::cerr << ", sql = " << query;
    OCIHandlePtr<OCIEnv, OCI_HTYPE_ENV> environment_handle;
    void connect(
    OCIEnv* environment_handle,
    const char* db,
    const char* user,
    const char* password,
    OCIHandlePtr<OCIError, OCI_HTYPE_ERROR>& error_handle,
    OCIHandlePtr<OCIServer, OCI_HTYPE_SERVER>& server_handle,
    OCIHandlePtr<OCISession, OCI_HTYPE_SESSION>& session_handle,
    OCIHandlePtr<OCISvcCtx, OCI_HTYPE_SVCCTX>& svc_context_handle)
    static const char* FUN = "connect()";
    sword result;
    // allocate an error handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **) &error_handle.fill(),
    OCI_HTYPE_ERROR,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // allocate a server handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **) &server_handle.fill(),
    OCI_HTYPE_SERVER,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    if((result = OCIServerAttach(
    server_handle.get(),
    error_handle.get(),
    (text*)db,
    strlen(db),
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIServerAttach", result, error_handle.get());
    // allocate a service handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **) &svc_context_handle.fill(),
    OCI_HTYPE_SVCCTX,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // set the server attribute in the service context handle
    if((result = OCIAttrSet(
    svc_context_handle.get(),
    OCI_HTYPE_SVCCTX,
    server_handle.get(),
    sizeof(OCIServer*),
    OCI_ATTR_SERVER,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // allocate a user session handle
    if((result = OCIHandleAlloc(
    environment_handle,
    (void **)&session_handle.fill(),
    OCI_HTYPE_SESSION,
    0, // extra memory to allocate
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    // set username and password attributes in user session handle
    if((result = OCIAttrSet(
    session_handle.get(),
    OCI_HTYPE_SESSION,
    (text*)user,
    strlen(user),
    OCI_ATTR_USERNAME,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    if((result = OCIAttrSet(
    session_handle.get(),
    OCI_HTYPE_SESSION,
    (text*)password,
    strlen(password),
    OCI_ATTR_PASSWORD,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    // start the session
    if((result = OCISessionBegin(
    svc_context_handle.get(),
    error_handle.get(),
    session_handle.get(),
    OCI_CRED_RDBMS,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCISessionBegin", result, error_handle.get());
    // set the user session attribute in the service context handle
    if((result = OCIAttrSet(
    svc_context_handle.get(),
    OCI_HTYPE_SVCCTX,
    session_handle.get(),
    sizeof(OCISession*),
    OCI_ATTR_SESSION,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    ub1 attr_value = 1;
    if((result = OCIAttrSet(
    server_handle.get(),
    OCI_HTYPE_SERVER,
    &attr_value,
    sizeof(attr_value),
    OCI_ATTR_NONBLOCKING_MODE,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrSet", result, error_handle.get());
    void disconnect(
    OCIEnv* environment_handle,
    OCIError* error_handle,
    OCIServer* server_handle,
    OCISession* session_handle,
    OCISvcCtx* svc_context_handle)
    static const char* FUN = "disconnect()";
    sword result;
    if((result = OCISessionEnd(
    svc_context_handle,
    error_handle,
    session_handle,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCISessionEnd", result, error_handle);
    if((result = OCIServerDetach(
    server_handle,
    error_handle,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIServerDetach", result, error_handle);
    OCILobLocator* FETCH_BUFFER_1[10000*1024];
    char IND_BUFFER_1[10000*1024];
    ub2 ERROR_BUFFER_1[1024];
    OCIDefine* define_handle_1;
    void clear_blob_buf(
    OCILobLocator** buf,
    unsigned long fetch_size)
    for(unsigned long i = 0; i < 1024; ++i)
    OCIDescriptorFree(
    buf,
    (ub4)OCI_DTYPE_LOB);
    ::memset(buf, 0, fetch_size * 1024);
    uint32_t RET_LEN4;
    sb4 oci_blob_callback(
    dvoid* ctx,
    OCIDefine* define,
    ub4 iter,
    dvoid** bufpp,
    ub4** alenpp,
    ub1* piecep,
    dvoid** indpp,
    ub2** rcodepp)
    RET_LEN4 = sizeof(OCILobLocator*);
    *bufpp = FETCH_BUFFER_1[iter];
    *alenpp = &RET_LEN4;
    *piecep = OCI_ONE_PIECE;
    *indpp = IND_BUFFER_1 + iter;
    *rcodepp = ERROR_BUFFER_1 + iter;
    //std::cout << "iter: " << iter << std::endl;
    return OCI_CONTINUE;
    int define_column(
    OCIEnv* environment_handle,
    OCISvcCtx* svc_handle,
    OCIStmt* stmt_handle,
    OCIError* error_handle,
    OCIDefine** define_handle,
    unsigned long pos,
    long oci_type,
    unsigned long fetch_size,
    OCILobLocator** buf,
    char* ind_buf,
    ub2* err_buf)
    static const char* FUN = "define_column()";
    sword result;
    if(oci_type == SQLT_BLOB)
    ::memset(buf, 0, fetch_size * 1024);
    for(int i = 0; i < 1024; ++i)
    if((result = OCIDescriptorAlloc(
    (dvoid*)environment_handle,
    (dvoid**)&buf[i],
    (ub4)OCI_DTYPE_LOB,
    (size_t)0,
    (dvoid**)0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIDescriptorAlloc", result, error_handle);
    ub2 size = 0;
    OCIDescriptorPtr<OCIParam, OCI_DTYPE_PARAM> param_handle;
    // ub2 oci_data_type = 0;
    if((result = OCIParamGet(
    stmt_handle,
    OCI_HTYPE_STMT,
    error_handle,
    reinterpret_cast<void**>(&param_handle.fill()),
    pos)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIParamGet", result, error_handle);
    if((result = OCIAttrGet(
    param_handle.get(),
    OCI_DTYPE_PARAM,
    &size,
    0,
    OCI_ATTR_DATA_SIZE,
    error_handle)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrGet", result, error_handle);
    if((result = OCIDefineByPos(
    stmt_handle,
    define_handle,
    error_handle,
    pos,
    # ifdef _DYN
    0,
    -1,
    #else
    buf,
    fetch_size,
    #endif
    oci_type,
    ind_buf,
    0,
    err_buf, // ptr to array of column-level return codes
    # ifdef _DYN
    OCI_DYNAMIC_FETCH
    # else
    OCI_DEFAULT
    # endif
    )) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIDefineByPos", result, error_handle);
    # ifdef _DYN
    if((result = OCIDefineDynamic(
    *define_handle,
    error_handle,
    0,
    oci_blob_callback)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIDefineByPos", result, error_handle);
    # endif
    return 0;
    int define_columns(
    OCIEnv* environment_handle,
    OCISvcCtx* svc_handle,
    OCIStmt* stmt_handle,
    OCIError* error_handle)
    static const char* FUN = "define_columns()";
    define_handle_1 = 0;
    int pos = 1;
    ::memset(ERROR_BUFFER_1, 0, 1024 * sizeof(ERROR_BUFFER_1[0]));
    define_column(
    environment_handle,
    svc_handle,
    stmt_handle,
    error_handle,
    &define_handle_1,
    pos++,
    SQLT_BLOB,
    sizeof(OCILobLocator*),
    FETCH_BUFFER_1,
    IND_BUFFER_1,
    ERROR_BUFFER_1);
    return 0;
    int execute_select(
    const char* db,
    const char* user,
    const char* password)
    static const char* FUN = "execute_select()";
    const unsigned long FETCH_COUNT = 1024;
    OCIHandlePtr<OCIError, OCI_HTYPE_ERROR> error_handle;
    OCIHandlePtr<OCIServer, OCI_HTYPE_SERVER> server_handle;
    OCIHandlePtr<OCISession, OCI_HTYPE_SESSION> session_handle;
    OCIHandlePtr<OCISvcCtx, OCI_HTYPE_SVCCTX> svc_context_handle;
    connect(
    environment_handle.get(),
    db,
    user,
    password,
    error_handle,
    server_handle,
    session_handle,
    svc_context_handle);
    OCIHandlePtr<OCIStmt, OCI_HTYPE_STMT> stmt_handle;
    sword result;
    if((result = OCIHandleAlloc(
    environment_handle.get(),
    (void**)&stmt_handle.fill(),
    OCI_HTYPE_STMT,
    0,
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    const char QUERY[] = "SELECT text FROM ADSERVERTEST_BLOB";
    ub4 rows = 0;
    if((result = OCIAttrSet(
    stmt_handle.get(),
    OCI_HTYPE_STMT,
    (dvoid *)&rows,
    (ub4)sizeof(ub4),
    OCI_ATTR_PREFETCH_ROWS,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(
    FUN, "OCIAttrSet(OCI_ATTR_PREFETCH_ROWS)", result, error_handle.get());
    if((result = OCIStmtPrepare(
    stmt_handle.get(),
    error_handle.get(),
    (text*)QUERY,
    strlen(QUERY),
    OCI_NTV_SYNTAX,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIStmtPrepare", result, error_handle.get());
    ub2 stmt_type;
    if((result = OCIAttrGet(
    stmt_handle.get(),
    OCI_HTYPE_STMT,
    &stmt_type,
    0,
    OCI_ATTR_STMT_TYPE,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrGet", result, error_handle.get());
    ub4 prev_fetched = 0;
    while((result = OCIStmtExecute(
    svc_context_handle.get(),
    stmt_handle.get(),
    error_handle.get(),
    0,
    0,
    0,
    0,
    OCI_DEFAULT)) != OCI_SUCCESS)
    if(result != OCI_STILL_EXECUTING)
    throw_oci_error(FUN, "OCIStmtExecute", result, error_handle.get());
    define_columns(
    environment_handle.get(),
    svc_context_handle.get(),
    stmt_handle.get(),
    error_handle.get());
    ub4 all_fetched_count = 0;
    while(true)
    while((result = OCIStmtFetch2(
    stmt_handle.get(),
    error_handle.get(),
    FETCH_COUNT,
    OCI_FETCH_NEXT,
    1,
    OCI_DEFAULT)) == OCI_STILL_EXECUTING
    # ifdef _DYN
    // || result == OCI_NEED_DATA
    # endif
    if (result != OCI_SUCCESS &&
    result != OCI_NO_DATA &&
    result != OCI_SUCCESS_WITH_INFO)
    std::cerr << FUN << ": can't make OCIStmtFetch2: status = " << result;
    std::cerr << ", message = '";
    text err_buf[1024] = "";
    sb4 err_code = 0;
    OCIErrorGet(
    error_handle.get(),
    (ub4)1,
    0,
    &err_code,
    err_buf,
    sizeof(err_buf),
    (ub4)OCI_HTYPE_ERROR);
    std::cerr << err_buf << "', error code = " << err_code << ", row errors: ";
    for(const ub2* cur = ERROR_BUFFER_1;
    cur < ERROR_BUFFER_1 + 1024; ++cur)
    if(*cur)
    std::cerr << "#" << (cur - ERROR_BUFFER_1) << ": " <<
    *cur << std::endl;
    ub4 fetched_count = 0;
    if((result = OCIAttrGet (
    stmt_handle.get(),
    OCI_HTYPE_STMT,
    &fetched_count,
    0,
    OCI_ATTR_ROW_COUNT,
    error_handle.get())) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIAttrGet", result, error_handle.get());
    all_fetched_count += fetched_count;
    if(fetched_count > 0)
    for(unsigned long i = 0; i < fetched_count - 1; ++i)
    ub4 lob_size;
    while((result = OCILobGetLength(
    svc_context_handle.get(),
    error_handle.get(),
    FETCH_BUFFER_1[i],
    &lob_size)) == OCI_STILL_EXECUTING)
    if(result != OCI_SUCCESS)
    throw_oci_error(FUN, "OCILobGetLength", result, error_handle.get());
    std::cout << "#" << i << ": ind = " << (unsigned long)IND_BUFFER_1[i] <<
    ", len = " << lob_size << std::endl;
    if(fetched_count - prev_fetched < FETCH_COUNT)
    break;
    prev_fetched = fetched_count;
    clear_blob_buf(
    FETCH_BUFFER_1,
    sizeof(OCILobLocator*));
    while((result = OCIStmtFetch(
    stmt_handle.get(),
    error_handle.get(),
    0,
    OCI_FETCH_NEXT,
    OCI_DEFAULT)) == OCI_STILL_EXECUTING
    disconnect(
    environment_handle.get(),
    error_handle.get(),
    server_handle.get(),
    session_handle.get(),
    svc_context_handle.get());
    return 0;
    int execute_update(
    const char* db,
    const char* user,
    const char* password)
    static const char* FUN = "execute_update()";
    OCIHandlePtr<OCIError, OCI_HTYPE_ERROR> error_handle;
    OCIHandlePtr<OCIServer, OCI_HTYPE_SERVER> server_handle;
    OCIHandlePtr<OCISession, OCI_HTYPE_SESSION> session_handle;
    OCIHandlePtr<OCISvcCtx, OCI_HTYPE_SVCCTX> svc_context_handle;
    connect(
    environment_handle.get(),
    db,
    user,
    password,
    error_handle,
    server_handle,
    session_handle,
    svc_context_handle);
    OCIHandlePtr<OCIStmt, OCI_HTYPE_STMT> stmt_handle;
    for(int i = 0; i < 1000; ++i)
    sword result;
    if((result = OCIHandleAlloc(
    environment_handle.get(),
    (void**)&stmt_handle.fill(),
    OCI_HTYPE_STMT,
    0,
    0)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIHandleAlloc", result, error_handle.get());
    std::ostringstream sql;
    std::string str;
    std::string et("\xEB\xB2\x88");
    unsigned long rep = ::rand() % 500 + 1;
    str.reserve(rep*et.size());
    for(unsigned long ei = 0; ei < rep; ++ei)
    str.append(et);
    sql << "BEGIN " <<
    "UPDATE ADSERVERTEST_BLOB SET text = "
    "utl_raw.cast_to_raw('" << str << "') "
    "WHERE id = " << i <<
    "END;";
    if((result = OCIStmtPrepare(
    stmt_handle.get(),
    error_handle.get(),
    (text*)sql.str().c_str(),
    sql.str().length(),
    OCI_NTV_SYNTAX,
    OCI_DEFAULT)) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIStmtPrepare", result, error_handle.get());
    while((result = OCIStmtExecute(
    svc_context_handle.get(),
    stmt_handle.get(),
    error_handle.get(),
    1,
    0,
    0,
    0,
    OCI_DEFAULT)) != OCI_SUCCESS)
    if(result != OCI_STILL_EXECUTING)
    throw_oci_error(FUN, "OCIStmtExecute", result, error_handle.get());
    disconnect(
    environment_handle.get(),
    error_handle.get(),
    server_handle.get(),
    session_handle.get(),
    svc_context_handle.get());
    return 0;
    void* select_thread(void* ctx)
    while(true)
    execute_select(
    "//oraclept.ocslab.com/addbpt.ocslab.com",
    "bs_unittest_4",
    "adserver");
    // std::cout << "select finished" << std::endl;
    return ctx;
    void* update_thread(void* ctx)
    while(true)
    execute_update(
    "//oraclept.ocslab.com/addbpt.ocslab.com",
    "bs_unittest_4",
    "adserver");
    // std::cout << "update finished" << std::endl;
    return ctx;
    int main()
    static const char* FUN = "main()";
    sword result;
    // allocate an environment handle
    if((result = OCIEnvCreate(
    &environment_handle.fill(),
    OCI_OBJECT | OCI_THREADED,
    0, // context
    0, // malloc
    0, // realloc
    0, // free
    0, // extra memory to allocate
    0) // pointer to user-memory
    ) != OCI_SUCCESS)
    throw_oci_error(FUN, "OCIEnvCreate", result, 0);
    pthread_t th1, th2;
    pthread_create(&th1, 0, select_thread, 0);
    pthread_create(&th2, 0, update_thread, 0);
    pthread_join(th1, 0);
    pthread_join(th2, 0);
    return 0;
    Edited by: user13011391 on 20.04.2010 5:50
    Edited by: user13011391 on 20.04.2010 6:00
    Edited by: user13011391 on 20.04.2010 6:15

  • NIO: Strange problem when using ByteBuffer with non-blocking SocketChannel

    Hi,
    I have a server that uses multiplexed, non-blocking I/O with java.nio. When a client connects, the server waits for the message: <system cmd="knock"/>, returns a message and disconnects the client. The clients are shortly serviced in less than a second.
    But the server newer receive anything from about 20% of the clients - even though it is sent. Or with other words: it is received and the data is contained in the ByteBuffer - SocketChannel.read(ByteBuffer) - but a call to ByteBuffer.remaing() returns 0 !!
    ByteBuffer receiveBuf = ByteBuffer.allocate(65536);
    receiveBuf.clear(); // the code is elsewhere used for longer living clients
    int readBytes = channel.read(receiveBuf);
    receiveBuf.flip();
    StringBuffer sb = new StringBuffer();
    System.out.println(" * Remaining: "+receiveBuf.remaining()); // writes: ' * Remaining: 0'
    System.out.println(" * Received: "+new String(receiveBuf.array())); // writes: ' * Received: <system cmd="knock"/>'
    while(receiveBuf.remaining() >= 2) {
      byte b = receiveBuf.get();
      sb.append((char)b);
    System.out.println(" * sb content: "+sb.toString()); // writes: ' * sb content: 'The ByteBuffer clearly receives the correct data, but the ByteBuffer.remaining() returns 0 and therefore the StringBuffer will never have any content.
    The problem seems to occur randomly and for about 20% of the clients (simulated from the same computer and therefore has the same bandwidth and so on).
    Anyone knows what is going on, and how to solve the problem !?

    It's always possible in any read that the number of bytes read is less than the number of bytes requested. You need to keep reading until you have got everything you expected, and cope with every possible error condition on each iteration.
    EJP

  • How to handle write errors in non blocking sockets

    Hi,
    I'm using sockets registered with a Selector for read operations. I've seen code examples that put the SocketChannel in non blocking mode before registering it with the selector, and in fact not doing so would cause an IllegalBlockingModeException to be thrown.
    My problem is that I can't handle write errors. The call to write() returns inmediately without throwing any exception. Even worse, when the network timeout expires the selector wakes up and I get an exception on read(). So I can't tell the difference between a real read error and a write error.
    What can I do? Is there a magic method I haven't heard about?
    Thanks

    ejp wrote:
    OK, so what happens is this: you write from your ByteBuffer; if there is room in the socket send buffer, the data is transferred and the transfer count is returned. (If there isn''t, the write returns zero and nothing has happened.) Your application code then continues. Meanwhile TCP is trying to send the data in the send buffer and get an ACK from the peer. If the peer is down as per your test, eventually those write attempts will time out. You will then get a connection reset exception on the next read or write.
    Even worse, when the network timeout expires the selector wakes upCorrect, to tell you there is an error condition pending. This is good, not bad.You're right. This way my program can know that something happened.
    But I still don't understand what the difference between a failed write() and a failed read() is. I mean, the error condition may appear during a send attempt, or it may appear after sending. In both cases I get an error when trying to read. How can my program know if data have been received by the other end?
    Do I have to implement ACK messages in the application level protocol??? It'd be nice if TCP could do this for me...

  • About NIO non-blocking socket write method behavior

    Hi all -
    I hope anyone could help in this issue.
    Assume you create a NIO socket and configure its channel in non blocking mode:
    channel.configureBlocking(false);
    then you write a byte buffer into it:
    int sentBytesNum = channel.write(byteBuffer);
    What are the possibilities of sentBytesNum value? I am sure of two possibilities, which are:
    - all data sent so sentBytesNum equals byteBuffer data size (limit minus position before sending) (after sending the limit and position of byteBuffer are the same)
    - no data is sent so sentBytesNum is zero and byteBuffer limit and postion did not change, so we shall use a write selector to test when the channel will be ready for write to write the data.
    What I am not sure about and need someone to confirm is the third possibility:
    - only a part of data is sent (according to the available free space of socket output buffer) so sentBytesNum is more than zero but less than data size and byteBuffer position is advanced by the value of sentBytesNum, might this possibility happen??
    If yes, so we should manage to hold the non sent part of byteBuffer till the channel becomes ready for write (we can know that using write selector) so we can write it to channel???
    Thanks for help,
    Rocka

    Yes, case three can occur. The usual NIO write loop looks like this:
    int count = 0;
    while (buf.position() > 0)
      buf.flip();
      count = ch.write(buf);
      buf.compact();
      if (count == 0)
        break;
    }This will run until buf.position() == 0 or count == 0. If the partial write case happens it will loop one more time and probably get a 0 count.
    At the end of the loop, if buf.position() > 0 there is still unwritten data and at this point you should register for OP_WRITE and return to the selector. Otherwise if there is no data unwritten you should deregister OP_WRITE as you have nothing to write and aren't interested in the event any longer.
    If you are reading something and writing at the same time the logic goes like this:
    while (inch.read(buf) >= 0 || buf.position() > 0)
      // etc
    }This will read until EOF occurs and either buf.position() is zero or a zero length write occurred.

  • Broken Pipe with Non-blocking Socket

    Hello,
    I write a Unix Agent who connect on a Windows Server with socket...
    Working well on Linux but on Solaris my problem is:
    -When my agent is running before server at connection all seems OK: Connect, Select and Getsockopt but when I try to send data I have always EPIPE Signal but I can receive datas from server !
    - When my agent is strarting after the server all it's Ok
    I don't unserstand this appears on Solaris SPARC 8 and Solaris 9 Intel ...
    Please Help is there something special with non-blocking sockets on Solaris ?
    Thanks

    Can't help you much but what I would recommend is that you
    insure that your pipes are opened for both read/write, even
    though you are only going to read or write from it. This insures
    that the pipe does not close down when you hit EOF.

  • Using non blocking connect() call for SCTP sockets in Solaris10

    Hi,
    I have a problem with non blocking connect call on SCTP socket.
    I am using the sctp stack support in Solaris10.
    When the connect is successful, I can get the pollout event on the socket.
    But there is no event observed when the peer does not exist. In other words, I could not get the pollout event on connection failure. This logic works fine with TCP sockets on both Solaris and Suse10.
    I am working with SCTP one-to-one style sockets.
    Is there any way to handle this issue?
    Do I need to load any patch to resolve this issue?
    It will be great if I get a solution in this regard.
    Thanks in advance.
    Best Regards,
    Bipin.

    There are at least two problems here.
    A. In the receiver you should test for -1 from the read() immediately, rather than continue with the loop and try to write -1 bytes to the file.
    B. In the sender you are ignoring the return value of client.write(), which can be anything from 0 to the buffer length. If you get 0 you should wait for another OP_WRITE to trigger; if you get a 'short write' you need to retry it until you've got nothing left to write from the current buffer, before you read any more data. This is where the data is vanishing.

  • Sockets in non-blocking mode

    Hello Techies,
    How to set sockets in the non-blocking mode and what is the main use of setting sockets in non-blocking mode.
    regards,
    Ramu.

    Hello Techies,
    Thanks for u r replies,
    Finally i had understood how to use nio. But the problem is it is going into infinite loop. Here is my code.
    package networking;
    import java.io.IOException;
    import java.net.InetSocketAddress;
    import java.nio.ByteBuffer;
    import java.nio.CharBuffer;
    import java.nio.channels.SelectionKey;
    import java.nio.channels.Selector;
    import java.nio.channels.SocketChannel;
    import java.nio.charset.Charset;
    import java.nio.charset.CharsetDecoder;
    import java.util.Iterator;
    public class testSocketChannel {
          * @param args
         public static ByteBuffer buf ;
         public static SocketChannel clientChannel;
         public static void main(String[] args) {
              // TODO Auto-generated method stub
                        try
                             Selector selector = Selector.open();
                         clientChannel = SocketChannel.open();
                      clientChannel.configureBlocking(false);
                      // Send a connection request to the server; this method is non-blocking
                      clientChannel.connect(new InetSocketAddress("localhost",90));
                      clientChannel.register(selector,clientChannel.validOps());
    //                Wait for events
                      while (true)
                          try {
                              // Wait for an event
                              selector.select();
                          } catch (IOException e) {
                              // Handle error with selector
                              //break;
                          // Get list of selection keys with pending events
                          Iterator it = selector.selectedKeys().iterator();
                          // Process each key at a time
                          //while (it.hasNext())
                               for(int i = 0;i<selector.selectedKeys().size();i++)
                              // Get the selection key
                              SelectionKey selKey = (SelectionKey)it.next();
                              // Remove it from the list to indicate that it is being processed
                              it.remove();
                              try
                          //  processSelectionKey(selKey);
                                    if (selKey.isValid() && selKey.isConnectable()) {
                                       // Get channel with connection request
                                       SocketChannel sChannel = (SocketChannel)selKey.channel();
                                       boolean success = sChannel.finishConnect();
                                       if (!success) {
                                           // An error occurred; handle it
                                           // Unregister the channel with this selector
                                           selKey.cancel();
                                   if (selKey.isValid() && selKey.isReadable()) {
                                       // Get channel with bytes to read
                                        clientChannel = (SocketChannel)selKey.channel();
                                       System.out.println("INSIDE reading");
                                       // See e174 Reading from a SocketChannel
                                       // Clear the buffer and read bytes from socket
    //                                 Clear the buffer and read bytes from socket
                                      // ByteBuffer byteBuffer = ByteBuffer.allocate( BUFSIZE );
                                       buf = ByteBuffer.allocateDirect(1024);
                                       int nbytes = clientChannel.read( buf );
                                       buf.flip();
                                       Charset charset = Charset.forName( "us-ascii" );
                                       CharsetDecoder decoder = charset.newDecoder();
                                       CharBuffer charBuffer = decoder.decode( buf );
                                       String result = charBuffer.toString();
                                   if (selKey.isValid() && selKey.isWritable()) {
                                       // Get channel that's ready for more bytes
                                       //SocketChannel sChannel = (SocketChannel)selKey.channel();
                                       System.out.println("INSIDE writting");
                                       // See e175 Writing to a SocketChannel
                                     String message = "hi"; 
                                      buf = ByteBuffer.wrap( message.getBytes()  );
                                       int nbytes = clientChannel.write( buf );
                                       System.out.println( "Wrote " + nbytes + " to channel." );
                              catch (IOException e) {
                                       // TODO: handle exception
                                   System.out.println("IOException"+e.getMessage());
                        catch (IOException ioe) {
                             // TODO: handle exception
                             System.out.println("IOException"+ioe.getMessage());
    }How to stop this infinite loop.
    Need quick replies.
    Ramu

  • Chaining WritableByteChannels and non-blocking I/O?

    Hello all,
    Suppose I want to implement a data transformation algorithm that works in chunks of say 4 bytes.
    The algorithm would hook into existing I/O paradigms for writing and reading data.
    Ignoring the transformation algorithm for now (it is actually irrelevant for my question), let's just I need to buffer data in 4-byte chunks.
    Since I want to make as little assumptions as possible about the decorated channel, BufferingWritableByteChannel needs to work channels in blocking and in non-blocking mode.
    When the decorated channel is in non-blocking mode, BuffereingWritableByteChannel may behave as in non-blocking mode itself, which is fine by me.
    However the problem is with the residual bytes: they do not get written until the close() method is called.
    I have implemented the close() method to be blocking (the while loop with the ugly yield() calls), so even when the decorated channel is in non-blocking mode everything will get written properly.
    However, writing potentially blocking calls (busy loops) in close() methods seems like a bad idea...
    So the question is:
    Is there a better way of doing this?
    Can I avoid having to write busy loops?
    package nio;
    import java.io.IOException;
    import java.nio.ByteBuffer;
    import java.nio.channels.WritableByteChannel;
    import static nio.ChannelUtils.*;
    public class BufferingWritableByteChannel implements WritableByteChannel {
      private final WritableByteChannel decoratedChannel;
      private final ByteBuffer buffer = ByteBuffer.allocate(4);
      private boolean buffering = true;
      public BufferingWritableByteChannel(WritableByteChannel decoratedChannel) {
        this.decoratedChannel = decoratedChannel;
      public synchronized void close() throws IOException {
        if (buffering) {
          buffer.flip();
        writeBlockingly(decoratedChannel, buffer);
        buffer.clear();
        decoratedChannel.close();
      public synchronized boolean isOpen() {
        return decoratedChannel.isOpen();
      public synchronized int write(ByteBuffer input) throws IOException {
        int bytesWritten = 0;
        if (!buffering) {
          decoratedChannel.write(buffer);
          if (buffer.hasRemaining()) { // May happen when decorated channel in non-blocking mode
            return bytesWritten;
          buffer.clear();
          buffering = true;
        while (input.hasRemaining()) {
          bytesWritten += putAsMuchAsPossible(input, buffer);
          if (!buffer.hasRemaining()) {
            buffer.flip();
            decoratedChannel.write(buffer);
            if (buffer.hasRemaining()) { // May happen when decorated channel in non-blocking mode
              buffering = false;
              return bytesWritten;
            buffer.clear();
        return bytesWritten;
    package nio;
    import java.io.IOException;
    import java.nio.BufferOverflowException;
    import java.nio.ByteBuffer;
    import java.nio.channels.WritableByteChannel;
    public class ChannelUtils {
      private ChannelUtils() {
       * Blockingly writes data to a channel, even if the channel is in non-blocking mode.
      public static final int writeBlockingly(WritableByteChannel channel, ByteBuffer buffer) throws IOException {
        int bytesWritten = 0;
        boolean yield = false;
        while (buffer.hasRemaining()) {
          bytesWritten += channel.write(buffer);
          if (yield)
            Thread.yield();
          else
            yield = true;
        return bytesWritten;
       * Puts as much bytes as possible from an input buffer into an output buffer, avoiding {@link BufferOverflowException}.
      public static final int putAsMuchAsPossible(ByteBuffer input, ByteBuffer output) {
        int bytesWritten = Math.min(input.remaining(), output.remaining());
        ByteBuffer inputSlice = input.slice();
        inputSlice.limit(inputSlice.position() + bytesWritten);
        output.put(inputSlice);
        input.position(input.position() + bytesWritten);
        return bytesWritten;
    }

    Hello all,
    Suppose I want to implement a data transformation algorithm that works in chunks of say 4 bytes.
    The algorithm would hook into existing I/O paradigms for writing and reading data.
    Ignoring the transformation algorithm for now (it is actually irrelevant for my question), let's just I need to buffer data in 4-byte chunks.
    Since I want to make as little assumptions as possible about the decorated channel, BufferingWritableByteChannel needs to work channels in blocking and in non-blocking mode.
    When the decorated channel is in non-blocking mode, BuffereingWritableByteChannel may behave as in non-blocking mode itself, which is fine by me.
    However the problem is with the residual bytes: they do not get written until the close() method is called.
    I have implemented the close() method to be blocking (the while loop with the ugly yield() calls), so even when the decorated channel is in non-blocking mode everything will get written properly.
    However, writing potentially blocking calls (busy loops) in close() methods seems like a bad idea...
    So the question is:
    Is there a better way of doing this?
    Can I avoid having to write busy loops?
    package nio;
    import java.io.IOException;
    import java.nio.ByteBuffer;
    import java.nio.channels.WritableByteChannel;
    import static nio.ChannelUtils.*;
    public class BufferingWritableByteChannel implements WritableByteChannel {
      private final WritableByteChannel decoratedChannel;
      private final ByteBuffer buffer = ByteBuffer.allocate(4);
      private boolean buffering = true;
      public BufferingWritableByteChannel(WritableByteChannel decoratedChannel) {
        this.decoratedChannel = decoratedChannel;
      public synchronized void close() throws IOException {
        if (buffering) {
          buffer.flip();
        writeBlockingly(decoratedChannel, buffer);
        buffer.clear();
        decoratedChannel.close();
      public synchronized boolean isOpen() {
        return decoratedChannel.isOpen();
      public synchronized int write(ByteBuffer input) throws IOException {
        int bytesWritten = 0;
        if (!buffering) {
          decoratedChannel.write(buffer);
          if (buffer.hasRemaining()) { // May happen when decorated channel in non-blocking mode
            return bytesWritten;
          buffer.clear();
          buffering = true;
        while (input.hasRemaining()) {
          bytesWritten += putAsMuchAsPossible(input, buffer);
          if (!buffer.hasRemaining()) {
            buffer.flip();
            decoratedChannel.write(buffer);
            if (buffer.hasRemaining()) { // May happen when decorated channel in non-blocking mode
              buffering = false;
              return bytesWritten;
            buffer.clear();
        return bytesWritten;
    package nio;
    import java.io.IOException;
    import java.nio.BufferOverflowException;
    import java.nio.ByteBuffer;
    import java.nio.channels.WritableByteChannel;
    public class ChannelUtils {
      private ChannelUtils() {
       * Blockingly writes data to a channel, even if the channel is in non-blocking mode.
      public static final int writeBlockingly(WritableByteChannel channel, ByteBuffer buffer) throws IOException {
        int bytesWritten = 0;
        boolean yield = false;
        while (buffer.hasRemaining()) {
          bytesWritten += channel.write(buffer);
          if (yield)
            Thread.yield();
          else
            yield = true;
        return bytesWritten;
       * Puts as much bytes as possible from an input buffer into an output buffer, avoiding {@link BufferOverflowException}.
      public static final int putAsMuchAsPossible(ByteBuffer input, ByteBuffer output) {
        int bytesWritten = Math.min(input.remaining(), output.remaining());
        ByteBuffer inputSlice = input.slice();
        inputSlice.limit(inputSlice.position() + bytesWritten);
        output.put(inputSlice);
        input.position(input.position() + bytesWritten);
        return bytesWritten;
    }

  • Nio write problem: server data sent isn't fully read by client

    Hi everyone,
    still writing away with my nio server and once again have run into
    some problems. Essentially my main problem is that when the server
    writes to the client, the write appears to output all the bytes in the
    write operation, but the client never accepts them all, even if a
    buffer has been manually allocated to the correct size of the data.
    As background my server will accept connections. When a connection
    is established I register OP_READ on the key. When a OP_READ trigger
    occurs the server accepts the clients request, processes it, then
    attaches the output to the SelectionKey as a ByteBuffer that has
    already been encoded. At this point I then register OP_WRITE on that
    SelectionKey (as a side note i'm running this server on XP and had
    read that registering OP_READ and OP_WRITE on the same selector was
    bad, but then it looked like there was a work around to this) and wait
    for an OP_WRITE trigger.
    When an OP_WRITE occurs on that key I run a new method (with heavy
    influences from the thread: http://forum.java.sun.com/thread.jsp?forum=11&thread=530825 and the taming the nio circus thread) which will grab the attachment and attempt to send it. The code has been written that IF the send cannot complete it should re-attach the remaining bytebuffer to the key and wait for another OP_WRITE to occur so it can send the remainder.
    The problem is that whenever I write (and for this test the amount im writing is approx 10100 bytes) the server appears to send it all (by checking the int returned from socketchannel.write()), but at the client end it never reads all the data that is sent.
    If i'm using a blocking socket client, then I get a java.net.SocketException: Connection Reset exception, whilst if i'm using a non-blocking client, I get no exception, just not the whole amount of data, even when i've statically allocated a receiving bytebuffer that is big enough.
    The following code is a class that is used to do the writing from the server:
       /* code for nio write model referenced from:
         * http://forum.java.sun.com/thread.jsp?forum=11&thread=530825
        class NIOWriteHandler {
            private ByteBuffer sendBuffer;
            private SelectionKey localSelectionKey;
            NIOWriteHandler(SelectionKey currentKey) {
                localSelectionKey = currentKey;
            public void doWrite() {
                localSelectionKey.interestOps(SelectionKey.OP_READ);  //deselect write,
                sendBuffer = (ByteBuffer)localSelectionKey.attachment();
                // perform the writing
                SocketChannel writingChannel = (SocketChannel)localSelectionKey.channel();
                if (writingChannel.isOpen()) {
                    int len = 0;
                    if (sendBuffer.hasRemaining()) {
                        try {
                            System.out.println("Sending chunks o data");
                            len = writingChannel.write(sendBuffer);
                            System.out.println("value of len: " + len);
                        } catch (IOException ioe) {
                            ioe.printStackTrace();
                            // call close method
                            System.out.println("CLOSE INVOKED at POINT 8");
                            closeComplete(localSelectionKey);
                    System.out.println("Second IF coming...");
                    if (sendBuffer.hasRemaining()) {
                        // if we get here then the previous write did not fully
                        // complete, so need to save data etc
                        System.out.println("Couldn't send all data this time...");
                        localSelectionKey.interestOps(SelectionKey.OP_WRITE|SelectionKey.OP_READ);
                        localSelectionKey.attach(sendBuffer);
                    } else {
                        sendBuffer = null;
                        closeComplete(localSelectionKey);
                // write complete at this stage
        }This is the basic block client that is incredibly dumb:
    import java.net.*;
    import java.util.*;
    import java.io.*;
    import java.nio.charset.*;
    import java.nio.channels.*;
    import java.nio.*;
    import java.nio.ByteBuffer;
    public class ServerTest {
        /* args 0 - the IP of the machine to connect to
           args 1 - the port number
           Simple client that connects to a specified IP & port, takes a line
           of input via stdin, sends that to the connected machine and prints
           out the response.
           Error handling and such isn't accounted for.
       public static void main (String args[]) throws Exception{
            Socket socket = new Socket(args[0], Integer.parseInt(args[1]));
            BufferedReader buffRead = new
                BufferedReader(new InputStreamReader((socket.getInputStream())));
            PrintStream ps =
                new PrintStream(socket.getOutputStream());
            Charset charset = Charset.forName("ISO-8859-1");
            BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in));
            System.out.println("[CLIENT]Data to send: ");
            String data = stdin.readLine();
            ps.println(data);
            String returned = buffRead.readLine();
            while (returned != null) {
                System.out.println(returned);
                returned = buffRead.readLine();
            System.out.println("[CLIENT]End server response");
            buffRead.close();
            ps.close();
            socket.close();
    }And here is the non-blocking basic client (which dosn't actually close at the moment):
    import java.net.*;
    import java.util.*;
    import java.io.*;
    import java.nio.charset.*;
    import java.nio.channels.*;
    import java.nio.*;
    public class ServerTestNonBlock {
        /* args 0 - the IP of the machine to connect to
           args 1 - the port number
           Simple client that connects to a specified IP & port, takes a line
           of input via stdin, sends that to the connected machine and prints
           out the response.
           Error handling and such isn't accounted for.
       public static void main (String args[]) throws Exception{
            InetSocketAddress addr = new InetSocketAddress
                (args[0], Integer.parseInt(args[1]));
            SocketChannel sc = SocketChannel.open();
            sc.configureBlocking(false);
            Selector selector = Selector.open();
            System.out.println("Starting connection...");
            sc.connect(addr);
            while(!sc.finishConnect()) {
               System.out.println("1,2,3,4 I will keep on counting...");
            System.out.println("Connection established..");       
            Charset charset = Charset.forName("ISO-8859-1");
            CharsetEncoder encoder = charset.newEncoder();
            sc.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
            while (true) {
               int n = selector.select();
               if (n==0) {
                  continue;
               Set keys = selector.selectedKeys();
               Iterator it = keys.iterator();
               while (it.hasNext()) {
                  SelectionKey selKey = (SelectionKey)it.next();
                  if (selKey.isReadable()) {
                     // time to setup read
                     ByteBuffer incomingData =
                        ByteBuffer.allocateDirect(102000);
                     incomingData.clear();
                     int count;
                     while ((count = sc.read(incomingData)) > 0) {
                        System.out.println("Value of count: " + count);
                        // reading the data
                     System.out.println("Count value: " + count);       
                     int pos = incomingData.position();
                     incomingData.flip();
                     CharBuffer content = charset.decode(incomingData);
                     String inData = content.toString();
                     System.out.println(inData.trim());
                     System.out.println("[CLIENT]End server response");
                     System.out.println("Count value: " + count);       
                     System.out.println("Position value: " + pos);       
                     //sc.close();
                     //break;
                  if (selKey.isWritable()) {
                     BufferedReader stdin = new BufferedReader
                       (new InputStreamReader(System.in));
                     System.out.println("[CLIENT]Data to send: ");
                     String data = stdin.readLine();
                     ByteBuffer byteBufferOut = encoder.encode
                        (CharBuffer.wrap(data));
                     int length = sc.write(byteBufferOut);
                     System.out.println("Wrote: " + length + " bytes.");
                     selKey.interestOps(SelectionKey.OP_READ);
    }I'm kinda stuck at the moment and am making change for the sake of change without getting a good grasp of what is going on. If anyone can provide any help that'd be fantastic. If in the mean time I figgure something out i'll post a response.
    If you've gotten this far thanks a bunch for reading :)
    Cheers,
    Pete

    Hi Meesum,
    thanks for the reply :)
    I'm not convinced this is the error - as i've got two clients listed there, and the odd behaviour from both is that neither is getting through to the last of the data that is sent.
    If the null were the problem (which is only checked for in the basic dumb blocking client) i'd be expecting some sort of infinite loop or wait for more data from the server, not an abnormal termination (ala connection reset) from the server. I'll give it a shot anyhow, but I know that under a blocking write operation that that code worked fine.
    Thanks again though for the post, has got some of my cogs slowly turning :)
    Cheers,
    Pete

  • What is wrong with my non-blocking client?

    I have two classes here, my abstract base class SingleSocketHandler, and its concrete subclass SingleSocketHandlerImpl. The subclass implements the protocol and parsing of my server.
    For some reason, my server is not receiving the packet my client sends to it, and my client is getitng nothing in return (which makes sense, the server is supposed to respond to the logon packet). I make it non-blocking AFTER logon, so I knwo that this is not a problem. Can you see why my server is not receiving the packet my client writes to it? Did I not configure some setting with the SocketChannel that enables it to write? I am sort of unfamiliar with the java.nio.channels package, so the problem may be related to a setting in the SocketChannel or whatnot that I haven't configured.
    NOTE: My chat server works fine with my blocking, multi-threaded test clients. Just not for my non-blocking client. The original problem for my blocking clients was that once the server stopped sending them data, they'd get caught in the in.read() loop and never get out of it. That's why I turned to non-blocking.
    Just to remind you, my question is: why isn't my client sending the logon packet AND/OR my server receiving+responding to it?
    public abstract class SingleSocketHandler extends Thread
         /* Subclasses must implement these methods
            /* Even though they're not a (public) interface */
         /** <------------------------------- */
              abstract void parse(int num);
              abstract void parseNext();
              abstract void doLogon();
         /** -------------------------------> */
         private SocketChannel sock;
         /* Queues for in and out buffers */
         private LinkedList <ByteBuffer> qIn;
         private LinkedList <ByteBuffer> qOut;
         /* Server info */
         private String hostname;
         private int port;
         /* Flags */
         protected int flags;
              protected final int LOGGED_ON = 0x01;
          * Default Constructor
         protected SingleSocketHandler()
              initQs();
          * Constructor that sets socket info
          * @param hostname
          * @param port
          * @param connect
         protected SingleSocketHandler(String hostname, int port, boolean connect)
              initQs();
              if (connect)
                   connect(hostname, port);
              else
                   setSocket(hostname, port);
          * Switches off between reading and writing
         protected void handleIO()
              try
                   sock.configureBlocking(false);
              } catch (IOException e)
                   // TODO
              readInBuffers(1);
              writeOutBuffers(1);
          * Read in specified number of buffers into in queue
          * Call for parsing
          * @param num
         protected void readInBuffers(int num)
              Reporter.println("READING BUFFER");
              for (int i = 0; i < num; i++)
                   ByteBuffer header = ByteBuffer.allocate(ProtocolCheck.HEADER_LEN);
                   try
                        Reporter.println("Reading header...");
                        sock.read(header);
                        Reporter.println("Read header.");
                   } catch (IOException e)
                        // TODO
                   /* Only add packet to in queue if it has a valid header */
                   if (ProtocolCheck.validHeader(header.array()))
                        Reporter.println("valid header");
                        ByteBuffer packet = ByteBuffer.allocate(ProtocolCheck.findPacketLen(header.array()));
                        packet.put(header);
                        try
                             Reporter.println("Reading rest of packet...");
                             sock.read(packet);
                             Reporter.println("Read packet.");
                        } catch (IOException e)
                             // TODO
                        addInBuffer(packet);
          * Write out specified number of buffers from out queue
          * And remove from out queue
          * @param num
         protected void writeOutBuffers(int num)
              Reporter.println("WRITING BUFFER");
              int i = 0;
              while (qOut.size() > 0 && i < num)
                   try
                        sock.write(nextOutBuffer());
                        Reporter.println("Wrote buffer.");
                   } catch (IOException e)
                        // TODO
                   i++;
          * Returns and removes next buffer from in queue
          * @return ByteBuffer
         protected ByteBuffer nextInBuffer()
              return qIn.remove();
          * Returns and removes next buffer from out queue
          * @return ByteBuffer
         protected ByteBuffer nextOutBuffer()
              return qOut.remove();
          * Sees if there is anohter in buffer
          * @return boolean
         protected boolean hasNextInBuffer()
              return qIn.size() > 0;
          * Sees if there is another out buffer
          * @return ByteBuffer
         protected boolean hasNextOutBuffer()
              return qOut.size() > 0;
          * Add a buffer to in queue
          * @param b
         public void addInBuffer(ByteBuffer b)
              qIn.add(b);
          * Add a buffer to in queue
          * @param b
         public void addInBuffer(Bufferable b)
              qIn.add(b.getByteBuffer());
          * Add a buffer to out queue
          * @param b
         public void addOutBuffer(ByteBuffer b)
              qOut.add(b);
          * Add a buffer to out queue
          * @param b
         public void addOutBuffer(Bufferable b)
              qOut.add(b.getByteBuffer());
          * Instantiate queues
         protected void initQs()
              qIn = new LinkedList <ByteBuffer> ();
              qOut = new LinkedList <ByteBuffer> ();
          * Set socket info then call connect()
          * @param hostname
          * @param port
         public void connect(String hostname, int port)
              setSocket(hostname, port);
              connect();
          * Connect to server
         public void connect()
              try
                   sock = SocketChannel.open();
                   sock.configureBlocking(true);
                   sock.connect(new InetSocketAddress(hostname, port));
                   while (!sock.finishConnect())
              } catch (IOException e)
                   // TODO
          * Disconnect from server
         public void disconnect()
              try
                   sock.close();
              } catch (IOException e)
                   // TODO
          * Set socket info without connecting
          * @param hostname
          * @param port
         public void setSocket(String hostname, int port)
              this.hostname = hostname;
              this.port = port;
          * @return state of connection
         public boolean isConnected()
              return (sock != null && sock.isConnected());
          * @return state of being logged on
         public boolean isLoggedOn()
              return (sock != null && (flags & LOGGED_ON) == LOGGED_ON);
    public final class SingleSocketHandlerImpl extends SingleSocketHandler
         private UserDatabase <User> users;
          * Constructor that does not set socket info
         public SingleSocketHandlerImpl(UserDatabase <User> users)
              super();
              this.users = users;
          * Constructor that does set socket info
          * @param hostname
          * @param port
          * @param connect
         public SingleSocketHandlerImpl(String hostname, int port, boolean connect, UserDatabase <User> users)
              super(hostname, port, connect);
              this.users = users;
          * Thread's run method (base class extends Thread)
         public void run()
              doLogon();
              while (isConnected() && isLoggedOn())
                   handleIO();
          * Parses specified number of buffers from in queue
          * @param num
         /* (non-Javadoc)
          * @see client.SingleSocketHandler#parseNext()
         @Override
         protected void parse(int num)
              Reporter.println("Parse(int num) called.");
              int i = 0;
              while (hasNextInBuffer() && i < num)
                   parseNext();
                   i++;
         /* (non-Javadoc)
          * @see client.SingleSocketHandler#parseNext()
         @Override
         protected void parseNext()
              Reporter.println("Parsing!");
              if (!hasNextInBuffer())
                   Reporter.println("NO IN BUFFER.");
                   return;
              /* Get buffer to work with */
              ByteBuffer inBuffer = nextInBuffer();
              byte[] data = inBuffer.array();
              /* Decide what to do based on message ID */
              byte msgid = data[1];
              switch (msgid) {
              case 0x01:
                   Reporter.println("0x01 packet.");
                   /* Determine success of login */
                   byte success = data[3];
                   if (success == (byte) 1)
                        flags |= LOGGED_ON;
                        Reporter.println("Logged on!");
                   else
                        flags &= ~LOGGED_ON;
                        Reporter.println(" <eChat> Unable to logon. Check the hostname and port settings.");
                   break;
              case 0x02:
                   /* Parse out text message */
                   byte[] txtmsgbytes = new byte[data.length - 3];
                   System.arraycopy(data, 3, txtmsgbytes,  0, txtmsgbytes.length);
                   String txtmsg = new String(txtmsgbytes);
                   Reporter.println(txtmsg);
                   break;
              case 0x03:
                   System.out.println("Packet ID not yet handled.");
                   break;
              case 0x04:
                   System.out.println("Packet ID not yet handled.");
                   break;
              default:
                   System.out.println("validID() method is buggy.");
             * I make it non-blocking after logon sequences
         /* (non-Javadoc)
          * @see client.SingleSocketHandler#doLogon()
         @Override
         protected void doLogon()
              Reporter.println("DOING LOGON!");
              User myUser = users.getCurr();
              addOutBuffer(new ScpLogon(myUser.getUsername(), myUser.getPassword()));
              writeOutBuffers(1);
              readInBuffers(1);
              parseNext();
    }

    Oh, if this helps, this is what gets output to my GUI. I did a lot of outputs for debugging purposes.
    [3:29:27 PM]: Connecting...
    [3:29:27 PM]: Connected!
    [3:29:27 PM]: Logging on...
    [3:29:27 PM]: DOING LOGON!
    [3:29:27 PM]: WRITING BUFFER
    [3:29:27 PM]: Wrote buffer.
    [3:29:27 PM]: READING BUFFER
    [3:29:27 PM]: Reading header...

  • Multiple non-blocking selectors in threads

    About networking in java and sockets i read alot. I know about the 2 implementations witk thread per client and non-blocking IO (usage of select()). I ran a few tests of my own and i could't decide. So I wrote a server that uses them both. The Ideea is to have no more than 100 clients per each selector and each selector runs in a thread. so when you have 4000 clients you actualy have 40 threads running each 100 clients:D
    However I'm stuck. I wrote this piece of code that has 2 objects one work thread with his own selector and a main thread with an accepting selector. The problem is that when I connect to it via telnet it only works when there is an even number of clients:(( when client nr 1 connects it blocks on channel.register(workSelector,SelectionKey.OP_READ);when client 2 gets connected it works:((
    HELP!!!
    here is the
    import java.nio.*;
    import java.nio.channels.*;
    import java.nio.charset.*;
    import java.io.*;
    import java.net.*;
    import java.util.*;
    import java.util.concurrent.ConcurrentHashMap;
    //work thread with own selector
    class workThread implements Runnable{
         public int userCount=0;
         public int ID=0;
         private Charset charset;
         private CharsetEncoder encoder;
         private CharsetDecoder decoder;
         private ByteBuffer buffer;
         private ConcurrentHashMap userMap;
         public Selector workSelector = null;
         public void addClient(SelectableChannel channel, int ops) throws IOException{
              if (channel == null) return;
              //channel.configureBlocking(false);
              System.out.println("Channel was added in workSelector : "+channel);
              channel.register(workSelector,SelectionKey.OP_READ);
              System.out.println("Client was added in work selector : " + ID);
              userCount++;
              System.out.println("Client nr : "+userCount);
         public boolean find(String name){
              return true;
         public workThread() throws IOException{
              charset = Charset.forName("ISO-8859-1");
              encoder = charset.newEncoder();
              decoder     = charset.newDecoder();     
              buffer = ByteBuffer.allocate(256);
    //          System.out.println("S-a creat selectorul");
              workSelector = Selector.open();
              System.out.println("Selector open ");
         private void doRead(SelectionKey key) throws IOException{
              SocketChannel clientChannel = (SocketChannel)key.channel();
              buffer.clear();
              System.out.println("Speaking to client");
              if(clientChannel.read(buffer) == -1){
                   key.cancel();
                   clientChannel.close();
              }else{
                   buffer.flip();
                   String message = decoder.decode(buffer).toString();
                   System.out.println(message);
                   if(message.trim().equals(".")){
                        clientChannel.write(encoder.encode(CharBuffer.wrap("BYE")));
                        key.cancel();
                        clientChannel.close();
                   }else{
                        buffer.flip();
                        clientChannel.write(buffer);
         private void doWrite(SelectionKey key){
              System.out.println("WRITE!!!");
         public void run(){
              while(true){
                   try{
                   if(workSelector.select() > 0){
                        Set keys1 = workSelector.selectedKeys();
                        for(Iterator itr = keys1.iterator();itr.hasNext();){
                             SelectionKey key = (SelectionKey)itr.next();
                             itr.remove();
                             if(key.isReadable()) {doRead(key);}
                        //     if(key.isWritable()) {doWrite(key);}
              }catch(IOException e){
                   System.out.println("Error in workthread "+ID+" "+e);
    //main thread
    public class msselect implements Runnable{
         private ServerSocketChannel server;
         private ServerSocket sock;
         private Selector acceptSelector;
         private workThread wk;
         public static void main(String args[]){
              try{
              msselect x = new msselect("80.97.89.73",6500,5);
              new Thread(x).start();
              }catch(IOException ex){
                   System.out.println("Probleme"+ex);
         msselect(String host, int port, int numT) throws IOException{
              server = ServerSocketChannel.open();
              sock = server.socket();
              sock.bind(new InetSocketAddress(port));
              server.configureBlocking(false);
              acceptSelector = Selector.open();
              server.register(acceptSelector, SelectionKey.OP_ACCEPT);
    /*          for(int i=0;i<numT;i++){
                   wk[i] = new workThread();
                   wk.ID = i;
                   new Thread(wk[i]).start();
              wk = new workThread();
              wk.ID = 1;
              server.register(wk.workSelector, SelectionKey.OP_ACCEPT);
              new Thread(wk).start();
         public void run(){
              while(true){
                   try{
                   acceptSelector.select();
                   Set keys = acceptSelector.selectedKeys();
                   for(Iterator itr = keys.iterator();itr.hasNext();){
                        SelectionKey key = (SelectionKey)itr.next();
                        itr.remove();
                        if(key.isAcceptable()){
                             SocketChannel client = null;
                             client = server.accept();
                             client.configureBlocking(false);
                             System.out.println("Accepted "+client);
                             wk.addClient(client,SelectionKey.OP_READ);
              }catch(IOException e){
                   System.out.println("Main Thread error"+e);

    I compiled and ran your code.
    The call channel.register(...) should block while selector is blocked in workSelector.select(). The fact that the registration succeeds is that you registered the server channel with both the work selector and the acceptor selector. The work selector is returning from select() because of a new client connection is accepted and is a matter of thread scheduling that the client.register() succeeeds (but only sometimes).
    Possible solution:
    - don't register the server channel with the work selector;
    - probably call workSelector.wakeup() to make the client channel registration possible.

Maybe you are looking for

  • Foreign language input problem

    Hi, I was using the "ITABC" that goes with Mac OSX to input Chinese. It worked pretty well until a few days ago. Now I cannot input Chinese into my computer anymore. Is there a way out? Thank you very much!

  • PSE 7 - open 1 file, OK. Open another file, CRASH!

    I'm running PSE 7 on a Windows XP Pro machine with 3GB RAM. I'd had no problems until recently - I just installed about 15 Windows updates (mostly Office and IE security fixes), a Java update and an Adobe Reader update.  After all these updates, I'm

  • How do I download my iPhoto update from App Store?

    I have iPhoto on my iMac (it is the one that came preinstalled - it has likely had updates made to it since I had the iMac as I have had it a few years now, but I never use iPhoto so don't know which version currently.) I am running 10.8.2. I have ju

  • Help with f:validateLength ... /

    Hi, I'm using the <f:validateLength minimum="6" maximum="45"/> tag in a login form to make sure there is something in a username field before looking up a database. The problem is if I don't type anything in the input field <h:inputText id="username"

  • Program freezes when adding text to clip

    Hi, I have sent a 40 minute slideshow to PE7. I am splitting the slide show at various points and inserting short video clips. After having added text to both photos and video two or three times, I now find that the program freezes when I either try