Getting following exception on TCP ssl at server side

Question

Hi,
I am trying to setup an ssl client and ssl server.
But on ssl client side, My need_wrap going into loop(Buffer overflow)
On my server side, i am getting following error.
Please help me...I am not getting much discussion/articles to solve this problem.
javax.net.ssl.SSLProtocolException: Handshake message sequence violation, state = 1, type = 1
javax.net.ssl.SSLProtocolException: Handshake message sequence violation, state = 1, type = 1
  at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1371)
  at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513)
  at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:790)
  at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:758)
  at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
  at com.ipay.ssl.SSLServerNio.doHandshake(SSLServerNio.java:55)
  at com.ipay.ssl.SSLServerNio.main(SSLServerNio.java:215)
Caused by: javax.net.ssl.SSLProtocolException: Handshake message sequence violation, state = 1, type = 1
  at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:156)
  at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
  at sun.security.ssl.Handshaker$1.run(Handshaker.java:808)
  at sun.security.ssl.Handshaker$1.run(Handshaker.java:806)
  at java.security.AccessController.doPrivileged(Native Method)
  at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1299)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:744)
My server handshake code look like this.
void doHandshake(SocketChannel socketChannel, SSLEngine engine,
         ByteBuffer myNetSendData, ByteBuffer myNetRecieveData) throws Exception {
     // Create byte buffers to use for holding application data
  ByteBuffer myAppDataSend = ByteBuffer.allocate(engine.getSession().getApplicationBufferSize());
  ByteBuffer myAppDataRecieve = ByteBuffer.allocate(engine.getSession().getApplicationBufferSize());
  System.out.println(engine.getPeerPort());
     // Begin handshake
     engine.beginHandshake();
     SSLEngineResult.HandshakeStatus hs = engine.getHandshakeStatus();
     // Process handshaking message
     while (hs != SSLEngineResult.HandshakeStatus.FINISHED &&
         hs != SSLEngineResult.HandshakeStatus.NOT_HANDSHAKING) {
      switch (hs) {
         case NEED_UNWRAP:
          System.out.println("Reached NEED UNWRAP");
             // Receive handshaking data from peer
             if (socketChannel.read(myNetRecieveData) < 0) {
                 // Handle closed channel
              System.out.println("not able toRead data from channel to buffer at client");
             myNetRecieveData.flip();
             // Process incoming handshaking data
             SSLEngineResult res = engine.unwrap(myNetRecieveData, myAppDataRecieve);
           //  myNetRecieveData.compact();
             // Getting handshake status
             hs = res.getHandshakeStatus();
             System.out.println("Debugging in NEED_UNWRAP-->"+hs);
             // Check status
             switch (res.getStatus()) {
             case OK :
                 // Handle OK status
              System.out.println("OK");
                 break;
             case BUFFER_OVERFLOW:
              System.out.println("BUFFER OVERFLOW");
              break;
             case BUFFER_UNDERFLOW:
              System.out.println("BUFFER UNDERFLOW");
              /* if (socketChannel.read(myNetRecieveData) < 0) {
                 // Handle closed channel
              System.out.println("not able toRead data from channel to buffer at client");
              hs=HandshakeStatus.NEED_UNWRAP;
              System.out.println("Read data on underflow condition");
              break;
             case CLOSED:
              System.out.println("CLOSED");
              break;
             // Handle other status: BUFFER_UNDERFLOW, BUFFER_OVERFLOW, CLOSED
             break;
         case NEED_WRAP :
          System.out.println("Reached NEED WRAP");
             // Empty the local network packet buffer.
             // Generate handshaking data
          //myAppDataSend.flip();
             res = engine.wrap(myAppDataSend, myNetSendData);
             // Getting handshake status
             hs = res.getHandshakeStatus();
             System.out.println("Debugging in NEED_WRAP-->"+hs);
             // Check status
             switch (res.getStatus()) {
             case OK :
              System.out.println("OK");
                 myNetSendData.flip();
                 // Send the handshaking data to peer
                 while (myNetSendData.hasRemaining()) {
                     if (socketChannel.write(myNetSendData) < 0) {
                         // closing socket channel
                 break;
             case BUFFER_OVERFLOW:
              System.out.println("BUFFER OVERFLOW");
              //Writing network send buffer
              myNetSendData.flip();
                 while (myNetSendData.hasRemaining()) {
              if(socketChannel.write(myNetSendData) < 0)
              System.out.println("Some thing wrong happened");
              System.out.println("written data");
              hs=HandshakeStatus.NEED_WRAP;
              break;
             case BUFFER_UNDERFLOW:
              System.out.println("BUFFER UNDERFLOW");
              break;
             case CLOSED:
              System.out.println("CLOSED");
              break;
             // Handle other status:  BUFFER_OVERFLOW, BUFFER_UNDERFLOW, CLOSED
              break;
         case NEED_TASK :
          System.out.println("NEED TASK");
          System.out.println("Debugging in NEED_TASK-->"+hs);
          Runnable task;
          while((task=engine.getDelegatedTask()) != null)
          System.out.println("Inside while loop");
          ExecutorService executorService = Executors.newFixedThreadPool(1);
          executorService.execute(task);
             // Handle blocking tasks
          hs=engine.getHandshakeStatus();
          System.out.println("Printing"+engine.getHandshakeStatus());
          break;
         case FINISHED:
          System.out.println("Debugging in FINISHED-->"+hs);
          System.out.println("handshake done");
          hs=HandshakeStatus.FINISHED;
                break;
         // Handle other status:  // FINISHED or NOT_HANDSHAKING
     // Processes after handshaking
brs,
varghese

Answer 1

Hi Zia,
The error is...
Caused by: java.io.InvalidClassException: org.eclipse.persistence.indirection.IndirectList; local class incompatible: stream classdesc serialVersionUID = 4038061360325736360, local class serialVersionUID = -494763524358427112
...which means you have two different versions of org.eclipse.persistence.indirection.IndirectList, one on the server and one in the client (JDeveloper). As these classes have different serialVersionUID values then Java throws an exception when deserializing as they are probably not compatible.
I don't know much about the Eclipse stuff but from looking at your post I can only assume one version of the class is in D:\Coherence\toplink\jlib\eclipselink.jar on the Coherence server side and the other is in JDevloper in D:\OracleSOA\Middleware\modules\org.eclipse.persistence_1.1.0.0_2-1.jar as these are the only jar file containing "eclipse" that I can see on the classpaths.
JK