Java NIO - TCP segment size abnormally low

Hi !
After noticing a weird behaviour on our Linux production server for code that works perfectly on my Windows dev box, I used tcpdump to sniff the packets that are actually sent to our clients.
The code I use to write the data is as simple as :
// using NIO - buffer is at most 135 bytes long
channel.write(buffer);
if (buffer.hasRemaining()) {
// this never happens
}When the buffer is 135 bytes long, this systematically results in two TCP segments being sent : one containing 127 bytes of data, the other one containing 8 bytes of data.
Our client is an embedded system which is poorly implemented and handles TCP packets as whole application messages, which means that the remaining 8 bytes end up being ignored and the operation fails.
I googled it a bit, but couldn't find any info about the possible culprit (buffer sizes and default max tcp segment sizes are of course way larger that 127 bytes !)
Any ideas ?
Thanks !

NB the fragmentation could also be happening in any intermediate router.
All I can suggest is that you set the TCP send buffer size to some multiple of the desired segment size, or maybe just set it very large, like 64k-1, so that you can reduce its effect on segmentation.

Similar Messages

Java NIO, ByteBuffers and Linksys router

I have a client server app/game that uses NIO for communication sending ByteBuffers. On a LAN with 5-8 users it runs great. On the internet, through a Linksys router, with one user, it has a blip. I get all my data transmissions except for one buffer. Whenever I chat the buffer contains a size, an int typeID and the encoded string for chat. This particular buffer never makes it to the client on the outside of the router. I have a port forwarded and regular tcp/ip java io sockets stuff works fine. As does al lof the other NIO buffer traffic for locational data, login in and out, etc... ANy thoughts??

But not sure what would be the performance of those clients?? when compared to Java NIO performance....Telnet isn't a high-performance protocol anyway. Don't worry about it. Use existing code. Get it working. Measure. If you have a performance issue, then worry, while at least you have something you can deploy. It won't be a problem. The router is there to route, not to talk high-speed telnet.

Converting from CP1252 (Windows) to ISO 8859-1 doesn't work with java.nio?

Hi
I'm trying to write some code that checks whether an InputStream contains only characters with a given encoding. I'm using java.nio for that. For tests, I downloaded some character set examples from http://www.columbia.edu/kermit/csettables.html
When creating the CharsetDecoder, I want to get all errors:
    Charset charset = Charset.forName( encoding );
    CharsetDecoder decoder = charset.newDecoder();
    decoder.onMalformedInput( CodingErrorAction.REPORT );
    decoder.onUnmappableCharacter( CodingErrorAction.REPORT );I then read an InputStream and try to convert it. If that fails, it can't contain the desired encoding:
    boolean isWellEncoded = true;
    ByteBuffer inBuffer = ByteBuffer.allocate( 1024 );
    ReadableByteChannel channel = Channels.newChannel( inputStream );
    while ( channel.read( inBuffer ) != -1 )
      CharBuffer decoded = null;
      try
        inBuffer.flip();
        decoded = decoder.decode( inBuffer );
      catch ( MalformedInputException ex )
        isWellEncoded = false;
      catch ( UnmappableCharacterException ex )
        isWellEncoded = false;
      catch ( CharacterCodingException ex )
        isWellEncoded = false;
      if ( decoded != null )
        LOG.debug( decoded.toString() );
      if ( !isWellEncoded )
        break;
      inBuffer.compact();
    channel.close();
    return isWellEncoded;Now I want to check whether a file containing Windows 1252 characters is ISO-8859-1. From my point of view, the code above should fail when it gets to the Euro symbol (decimal 128), since that's not defined in ISO-8859-1.
But all I get is a ? character instead:
(}) 125 07/13 175 7D                 RIGHT CURLY BRACKET, RIGHT BRACE
(~) 126 07/14 176 7E                 TILDE
[?] 128 08/00 200 80 EURO SYMBOL
[?] 130 08/02 202 82 LOW 9 SINGLE QUOTEI also tried to replace the faulty character, using
    decoder.onUnmappableCharacter( CodingErrorAction.REPLACE );
    decoder.replaceWith("!");but I still get the question marks.
I'm probably doing something fundamentally wrong, but I dont get it :-)
Any help is greatly appreciated!
Eric

As a suggestion....create a complete example demonstrating the problem. It shouldn't have channel in it since that wouldn't appear to be the problem (decoding is.) You should create the byte array in the example code - populate it with the byte sequence that you think should work. And your code should then demonstrate that it doesn't. Then post that.

Java.nio.BufferOverflowException on big Strings ( 87MB)

Hi!
When calling getBytes() on a big String the following exception is thrown:
java.nio.BufferOverflowException
     at java.nio.charset.CoderResult.throwException(Unknown Source)
     at java.lang.StringCoding$CharsetSD.decode(Unknown Source)
     at java.lang.StringCoding.decode(Unknown Source)
     at java.lang.StringCoding.decode(Unknown Source)
     at java.lang.String.<init>(Unknown Source)
     at java.lang.String.<init>(Unknown Source)
     at com.db.janus.Janus.main(Janus.java:99)
This happens as soon as the size of the String gets bigger than 92003848 bytes.
I can reproduce it with the following code:
String x = new String(new byte[92003848]); // --> OK
String x = new String(new byte[92003849]); // --> Exception
JDK Version: 1.4.2_06
OS: Windows XP
Is this a known bug / limitation? I found a similar bug related to Java on Linux / Solaris, but this one was about getBytes() with Strings > 16MB.
Thanks for your help!

Is there any alternative to solve this problem.I am getting the error
java.nio.BufferOverflowException .This happens in my mail server when the file six\ze of xml where i have stored data about spam/nonspam grows upto 22mb.I have stored this xml in database as blob.at server startup uploading in in my class.Keeps througout in JVM after every 50 th request saving information back to database(about spam/nonspam).Now if xml size grows i am storing this in string to parse etc.it gives this bug error.Sholud i cahnge my datatype or use something as XML Parser to split my xml in pieces(mechanism i am not aware of at all) so that string just takes some size of xml (but here can i use string builder etc.). Now is there any proper solution to this kind of problem whenever i removes some daya from xml spam mail goes to inbox and vice versa.
Please suggest me some solution if possible.
complete error is:
Servlet initialization failed:
java.nio.BufferOverflowException
at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
at java.lang.StringCoding$CharsetSD.decode([BII)[C(Unknown Source)
         at java.lang.StringCoding.decode(Ljava.lang.String;[BII)[C(Unknown Source)
         at java.lang.String.<init>([BIILjava.lang.String;)V(Unknown Source)
         at java.lang.String.<init>([BLjava.lang.String;)V(Unknown Source)
         at zerocode.core.ByteBlock.toString(ByteBlock.java:89)
         at zcCollab.mailProcessing.NaiveBayesClassifier._getInstanceData(NaiveBayesClassifier.java:275)
         at zcCollab.mailProcessing.NaiveBayesClassifier.initialize(NaiveBayesClassifier.java:86)
         at zcCollab.mailProcessing.SimpleDeliveryManager._setupClassifiers(SimpleDeliveryManager.java:409)
         at zcCollab.mailProcessing.SimpleDeliveryManager.<init>(SimpleDeliveryManager.java:62)
         at zcCollab.zcApp.InfoManagerApp._createDeliveryProcessor(InfoManagerApp.java:307)
         at zcCollab.zcApp.InfoManagerApp.initialize(InfoManagerApp.java:294)
         at zerocode.udm.Application._createInstance(Application.java:168)
         at zerocode.udm.Application.createInstanceForServlet(Application.java:111)
         at zerocode.servlet.UdmServlet._createApplication(UdmServlet.java:67)
         at zerocode.servlet.UdmServlet._initialize(UdmServlet.java:52)
         at zerocode.servlet.GenericServlet.init(GenericServlet.java:116)
         at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:292)
         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:329)
         at org.mortbay.jetty.servlet.ServletHandler.dispatch(ServletHandler.java:657)
         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
         at org.mortbay.http.HttpServer.service(HttpServer.java:879)
         at org.mortbay.http.handler.ForwardHandler.handle(ForwardHandler.java:130)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
         at org.mortbay.http.HttpServer.service(HttpServer.java:879)
Thanks
Vijendra

Java.nio and databases

Please, need help.
I'm using java.nio to get large files contents (txt) and need to put these data into MSSQL SERVER 2000 database tables. NIO have capabilityies to get data and transfer directly to the tables? If so, how can i do it?
Thanks a lot
Aloisio

ChuckBing, i'm connected with database trough JDBC yet and i can retrieve data meaning my connection is OK.
I'm able to read data from a txt file (250mb large) trough MappedByteBuffer and write into another with the same size and content. It's working fine.
My question is if i can read the txt file and, instead write the data into another txt file, put these data into a table.
Thanks for your reply

Help "ascending" to Java NIO

Hello there.
I've been using Java for quite some time, but the few really networked client/server applications I have used, have used java.io and java.net. Now, even I have heard the rumours about Java NIO, and I am curious. I have tried reading up and down the API on java.nio.* , but I must say I am still kind of confused.
Basically, I am looking for some example code, or a thorough explanation of how all the classes and components come together to form efficient asynchronous non-blocking I/O.
What I picture is a server over TCP/IP meant to handle 100+ clients, written with an emphasis on performance. Think MMORPG, I guess (no no, I am not ambitious enough to want to write one, but I can imagine that gives a good analogy of how I would have the communication set up). The server needs to quickly figure out which clients need to be handled at any given time, with as little overhead or reduntant iteration as possible.
Ok, I am asking alot perhaps.. I just wonder if anyone could offer links to example source-code, write something to show me, or basically just try to explain to me how this all works.
I would be very very grateful if anyone stepped up to the plate.
Thanks,
-Terje

Take a look at my thread Taming the NIO Circus. It starts with simple examples of a single thread echo server and a swing client.

Java.nio.channels.FileChannel jar file

Hi
I have used java.nio.channels.FileChannel package in my program.
where can I get the jar file which has the above package?
Thanx

i give here sample code.. use this it will work fine..
import java.nio.*;
import java.nio.channels.*;
import java.io.*;
public class copyImage
public static void main(String [] args) throws IOException
try {
// Create channel on the source
FileChannel srcChannel = new FileInputStream("dragonfly.jpg").getChannel();
FileChannel dstChannel =new FileOutputStream("1.jpg").getChannel();
// Copy file contents from source to destination
dstChannel.transferFrom(srcChannel, 0, srcChannel.size());
// Close the channels
srcChannel.close();
dstChannel.close();
} catch (IOException e) {
}

Java.nio.channels.Pipe?

Someone explain what java.nio.channels.Pipe should be uses for, and how to use it?

You can use this class to transfer the content of a channel from one thread to another thread.
The writing thread blocks when the pipe is full.
package com.desoft.pipetest;
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
public class PipeTest {
public static void main (String args[])
try
FileInputStream fis = new FileInputStream(args[0]);
FileChannel fc = fis.getChannel();
Pipe p = Pipe.open();
PipeReader pr = new PipeReader( p );
Thread t = new Thread( pr );
t.start();
fc.transferTo(0, fc.size(), p.sink() );
catch( IOException ioe )
ioe.printStackTrace();
* Read from a Pipe and write Content to System.out
static class PipeReader implements Runnable
Pipe p;
public PipeReader( Pipe p )
this.p = p;
public void run()
try
ByteBuffer buffer = ByteBuffer.allocate( 10 );
int len = 0;
while( (len=p.source().read( buffer )) > 0 )
buffer.rewind();
for( int i = 0; i < len; i++ )
System.out.print( (char)buffer.get());
buffer.rewind();
catch( IOException ioe )
ioe.printStackTrace();

Performance of java nio with dd in linux.

Hi
I ran this code in java to dump 0s in a file of size 1gb and tried the same with dd in linux.
Java code : I use a preinitialized array (the data) and fill a mappedbytebuffer with it.
package filePersistence.test;
import java.io.File;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class TimeLineTest {
     private static final int BYTE_LENGTH = 1000;
     //this is the input
     static byte[] b = new byte[BYTE_LENGTH];
     public static void main(String[] arg) throws Exception
          //initializing the input
          for (int i=0;i<BYTE_LENGTH;i++)
               b[i] = 0;
          File file = new File(arg[0]);
          RandomAccessFile raFile = new RandomAccessFile(file,"rw");
          FileChannel fChannel = raFile.getChannel();
          int loopCount = 1000000;
          MappedByteBuffer mbuffer = fChannel.map( FileChannel.MapMode.READ_WRITE ,0,loopCount * BYTE_LENGTH);
          //System.out.println(" going to fill in a file" + file.getName());
          long k = 0;
          long startTime = System.currentTimeMillis();
          for (int i=0;i<loopCount;i++)
     //populate the mapped buffer
               mbuffer.put(b);
          //persist into the file
          mbuffer.force();
          long endTime = System.currentTimeMillis();
          System.out.println(" file filled size1 "+file.length());
          System.out.println(" time " + (endTime - startTime));
On a linux machine this takes around 7 secs while dd used as
"dd if=/dev/zero of=mytestfile.out bs=1000 count=1000000"
1000000+0 records in
1000000+0 records out
1000000000 bytes (1.0 GB) copied, 4.618 seconds, 217 MB/s
4.6 and 7 differ quite a lot. Is there a way the java code can be improved to match dd ?(-sever does not help)
Thanks
Sumanta

Hi
Can this be called a dd equivalent code ?
package filePersistence.test;
import java.io.File;
import java.io.FileOutputStream;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class TimeLineTest {
     private static final int BYTE_LENGTH = 1000;
     //this is the input
     static byte[] b = new byte[BYTE_LENGTH];
     public static void main(String[] arg) throws Exception
          //initializing the input
          for (int i=0;i<BYTE_LENGTH;i++)
               b[i] = 0;
          String srcFile = arg[0] + "_src";
          FileOutputStream fos = new FileOutputStream(srcFile);
          fos.write(b);
          fos.getFD().sync();
          fos.flush();
          fos.close();
          RandomAccessFile srcraFile = new RandomAccessFile(srcFile,"rw");
          FileChannel srcChannel = srcraFile.getChannel();
          File file = new File(arg[0]);
          RandomAccessFile raFile = new RandomAccessFile(file,"rw");
          FileChannel raChannel = raFile.getChannel();
          int loopCount = 1000000;
          //MappedByteBuffer mbuffer = fChannel.map( FileChannel.MapMode.READ_WRITE ,0,loopCount * BYTE_LENGTH);
          //System.out.println(" going to fill in a file" + file.getName());
          long startTime = System.currentTimeMillis();
          for (int i=0,position=0;i<loopCount;i++,position+=BYTE_LENGTH)
     //populate the mapped buffer
               //mbuffer.put(b);
               raChannel.position(position);
               srcChannel.transferTo(0, BYTE_LENGTH, raChannel);
          //persist into the file
          //mbuffer.force();
          raFile.getFD().sync();
          raChannel.close();
          raFile.close();
          long endTime = System.currentTimeMillis();
          System.out.println(" file filled size1 "+file.length());
          System.out.println(" time " + (endTime - startTime));
}

Using java.nio

I have copied this, one program from a book that would have to read file of gives to me to you mixed but it does not work, you could me why?
import java.io.FileInputStream;
import java.io.IOException;
import java.io.File;
import java.io.FileNotFoundException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
public class ReadPrimesMixedData {
public static void main(String[] args) {
File aFile = new File("C:/Beg Java Stuff/primes.txt");
FileInputStream inFile = null;
try {
inFile = new FileInputStream(aFile);
} catch(FileNotFoundException e) {
e.printStackTrace(System.err);
System.exit(1);
FileChannel inChannel = inFile.getChannel();
try {
ByteBuffer lengthBuf = ByteBuffer.allocate(8);
int strLength = 0; // Stores the string length
ByteBuffer buf = null; // Stores a reference to the second byte buffer
byte[] strChars = null; // Stores a reference to an array to hold the string
while(true) {
if(inChannel.read(lengthBuf) == -1) // Read the string length, if its EOF
break; // exit the loop
lengthBuf.flip();
strLength = (int)lengthBuf.getDouble(); // Extract the length and convert to int
buf = ByteBuffer.allocate(strLength+8); // Buffer for the string & the prime
if(inChannel.read(buf) == -1) { // Read the string & binary prime value
assert false; // Should not get here!
break; // Exit loop on EOF
buf.flip();
strChars = new byte[strLength]; // Create the array for the string
buf.get(strChars); // Extract string & binary prime value
System.out.println("String length: " + strChars.length+ " String: " +
new String(strChars) + " Binary value: " + buf.getLong());
lengthBuf.clear(); // Clear the buffer for the next read
System.out.println("\nEOF reached.");
inFile.close(); // Close the file and the channel
} catch(IOException e) {
e.printStackTrace(System.err);
System.exit(1);
System.exit(0);
This e' the program that writes the file
import java.io.*;
import java.nio.*;
import java.nio.channels.FileChannel;
public class PrimesToFile2 {
public static void main(String[] args) {
int primesRequired = 100; // Default count
if (args.length > 0) {
try {
primesRequired = Integer.valueOf(args[0]).intValue();
} catch (NumberFormatException e) {
System.out.println("Prime count value invalid. Using default of "
+ primesRequired);
long[] primes = new long[primesRequired]; // Array to store primes
primes[0] = 2; // Seed the first prime
primes[1] = 3; // and the second
// Count of primes found - up to now, which is also the array index
int count = 2;
long number = 5; // Next integer to be tested
outer:
for (; count < primesRequired; number += 2) {
// The maximum divisor we need to try is square root of number
long limit = (long) Math.ceil(Math.sqrt((double) number));
// Divide by all the primes we have up to limit
for (int i = 1; i < count && primes[i] <= limit; i++)
if (number % primes[i] == 0) // Is it an exact divisor?
continue outer; // yes, try the next number
primes[count++] = number; // We got one!
File aFile = new File("C:/Beg Java Stuff/primes.txt");
FileOutputStream outputFile = null;
try {
outputFile = new FileOutputStream(aFile);
} catch (FileNotFoundException e) {
e.printStackTrace(System.err);
System.exit(1);
FileChannel file = outputFile.getChannel();
final int BUFFERSIZE = 100; // Buffer size in bytes
ByteBuffer buf = ByteBuffer.allocate(BUFFERSIZE);
DoubleBuffer doubleBuf = buf.asDoubleBuffer();
buf.position(8);
CharBuffer charBuf = buf.asCharBuffer();
LongBuffer longBuf = null;
String primeStr = null;
for (int i = 0; i < primes.length; i++) {
primeStr = "prime = " + primes; // Create the string
doubleBuf.put(0,(double) primeStr.length());// Store the string length
charBuf.put(primeStr); // Store the string
buf.position(2 * charBuf.position() + 8); // Position for 3rd buffer
longBuf = buf.asLongBuffer(); // Create the buffer
longBuf.put(primes[i]); // Store the binary long value
buf.position(buf.position() + 8); // Set position after last value
buf.flip(); // and flip
try {
file.write(buf); // Write the buffer as before.
} catch (IOException e) {
e.printStackTrace(System.err);
System.exit(1);
buf.clear();
doubleBuf.clear();
charBuf.clear();
try {
System.out.println("File written is " + file.size() + " bytes.");
outputFile.close(); // Close the file and its channel
} catch (IOException e) {
e.printStackTrace(System.err);
System.exit(1);
System.exit(0);
This works correctly

I don't have problems when I compile the program but if I run the program ....I have this
String length: 9 String: prim Binary value: 7277852183226228736
String length: 0 String: Binary value: 162166969980682240
String length: 0 String: Binary value: 7854388801345436928
String length: 0 String: Binary value: 3573983215616
Stjava.lang.IllegalArgumentException
at java.nio.ByteBuffer.allocate(ByteBuffer.java:303)
at ReadPrimesMixedData.main(ReadPrimesMixedData.java:35)
ring length: 0 String: Binary value: 7566167222444367872
String length: 0 String: Binary value: 88089088
String length: 0 String: Binary value: 8214681170873443584
String length: 0 String: Binary value: 1856
String length: 0 String: Binary value: 8070575878335130880
Exception in thread "main"

Java.nio.BufferUnderflowException when send a idoc xml using sapjco3.jar

When I use sapjco3.jar to send a idoc
I have 2 IDOC ,one size is 200kb ,another is 336kb
the first one can send to sap correct, but the bigger one got Exception (java.nio.BufferUnderflowException)
If any thing I have to note when writing java code ? thanks ....
Exception:
send TXLOG-12918467-ToIDOC_20110513161427170_1125436.xml
send idoc D:\TXLOG-12918467-ToIDOC_20110513161427170_1125436.xml to SAP GateWay finished ,IDoc Tid is AC1140540B534DD1EAE61781
send TXLOG-12964100.xml
java.nio.BufferUnderflowException
     at java.nio.Buffer.nextGetIndex(Buffer.java:486)
     at java.nio.HeapCharBuffer.get(HeapCharBuffer.java:129)
     at com.sap.conn.idoc.rt.xml.DefaultIDocXMLParser.parse(DefaultIDocXMLParser.java:365)
     at com.sap.conn.idoc.rt.xml.DefaultIDocXMLProcessor.parse(DefaultIDocXMLProcessor.java:87)
     at com.sap.conn.idoc.rt.xml.DefaultIDocXMLProcessor.parse(DefaultIDocXMLProcessor.java:68)
     at com.testrite.msg.dex.sender.SAPSender.sendMessage(SAPSender.java:61)
     at com.testrite.msg.dex.sender.SAPSender.main(SAPSender.java:37)
My Code :
JCoDestination destination;
String dest = "BCE";
destination = JCoDestinationManager.getDestination(dest);
IDocRepository iDocRepository = JCoIDoc.getIDocRepository(destination);
String tid = destination.createTID();
IDocFactory iDocFactory = JCoIDoc.getIDocFactory();
IDocXMLProcessor processor = iDocFactory.getIDocXMLProcessor();
IDocDocumentList iDocList = processor.parse(iDocRepository, new BufferedInputStream(new FileInputStream("D:
TXLOG-12964100.xml"));
JCoIDoc.send(iDocList, IDocFactory.IDOC_VERSION_DEFAULT, destination, tid);

hi,
it only works with idoc to idoc scenarios I think
when you send from SAP to SAP through XI
have a look at my weblog:
/people/michal.krawczyk2/blog/2005/11/21/xi-idoc-to-idoc-tunneling--how-fast-and-easy-can-you-get
Regards,
michal

TCP segment Overwrite

Recently, I've been seeing a ton of messages on the IPS Event Viewer stating that TCP Segment Overwrites have been detected. The events are from either the IPS-A to our DMZ or vice versa. It just started whenever we added some servers to the network. Does anybody have any suggestions? We traced the packets, it's not hacker related. It seems like maybe a server or app. may be causing the issue. The problem is isolating it. I would appreciate any feedback you could provide. Thanks!

TCP normalization
Through intentional or natural TCP session segmentation, some classes of attacks can be hidden. To make sure policy enforcement can occur with no false positives and false negatives, the state of the two TCP endpoints must be tracked and only the data that is actually processed by the real host endpoints should be passed on. Overlaps in a TCP stream can occur, but are extremely rare except for TCP segment retransmits. Overwrites in the TCP session should not occur. If overwrites do occur, someone is intentionally trying to elude the security policy or the TCP stack implementation is broken. Maintaining full information about the state of both endpoints is not possible unless the sensor acts as a TCP proxy. Instead of the sensor acting as a TCP proxy, the segments will be ordered properly and the normalizer will look for any abnormal packets associated with evasion and attacks.
http://www.cisco.com/en/US/products/hw/vpndevc/ps4077/products_configuration_guide_chapter09186a00804cf52e.html

TCP Hijack/TCP Segment Overwrite false positives?

Hello all,
I was just curious if anyone else has had many false positives with 3 signatures in particular: TCP Hijack (3250.0 - High), TCP Hijack Simplex Mode (3251.0 - High), and TCP Segment Overwrite (1300.0 - High). The reason I think they are false positives is because they occur everyday, and I've also seem them caused by internal network traffic that crosses an IPS sensor (that is, making the potentially dangerous assumption that the internal devices can be trusted). We usually see between a dozen and 3 dozen a day depending on the signature, and we have 8 IPS total deployed internally and on the perimeters.
Has anyone else had similar experiences? If so, do you have any suggestions on how to decrease the number of false positives for these alerts?
Thanks,
Ryan

I get TCP Hijack and TCP Segment Overwrite all the time. I opened a TAC case about it because it was getting out of hand, and the engineer said that TCP Hijack would be very very hard to execute and if it is getting fired a lot odds are it is a false positive.
This was his response:
5769 - Malformed HTTP Request
This signature basically just looks for traffic destined to one of your web ports (defined by the WEBPORTS variable) and containing a valid HTTP request (i.e., GET, POST, HEAD, PUT, DELETE, TRACE, CONNECT) but followed by malformed (i.e., not proper http protocol syntax) URI information. This type of malformed HTTP request can be used for a variety of exploits. Microsoft has malformed HTTP request vulnerabilities, another attack known as "http request smuggling" can be launched using malformed HTTP requests at a Squid web proxy, which may cause the web proxy and an upstream HTTP agent to disagree on the boundary between HTTP requests on a persistent connection. These are a couple of examples.
If you open this signature in IDM and go to "Edit", you can see the regex it looks for within the http payload. Basically, it looks for a valid HTTP request followed by the hex code regex [\x20][\x21-\x7e]+[\x20]?[\x0d\x0a]. A properly formed HTTP request should not contain this hex code.
It's possible that normal traffic could cause this, but unlikely. If you have further concerns about this signature firing, please capture the trigger packet context either by changing the signature action to 'produce verbose alert' or 'log attacker packet' for analysis. If you need assistance in analyzing these alerts, please contact TAC and open a case on this issue.
3250 or 3251 - TCP Hijack and TCP Hijack Simplex Mode
This signature detects attempts to insert packets into a TCP stream by an attacker in an effort to take over this session. However, if you're using inline ips mode, TCP Hijack attacks are impossible. Also, this type of attack is very rare and not easy to do, and is often a false positive. Types of things that can be used by network sniffers to detect that a TCP hijack may be happening is looking for repeated ARP updates, frames sent between client and server with different MAC addresses, or tcp ack storms.
For these two hijack signatures, per MySDN information:
"This signature fires upon detecting out of order ack packets. The most common network event that may trigger this signature is an idle telnet session. The TCP Hijack attack is a low-probability, high level-of-effort event."
Thus, very likely to be false positives and unlikely to be a legitimate attack given the difficulties involved in doing this. However, it's worth checking out the source / destination of the attacks. Again though, if you are running inline mode, these attacks are impossible and you can ignore these signatures.
About the TCP Segment Overwrite, mine is always fired for port 20 traffic from some sort of web cache server. Is that the same for you?

Java.io vs. java.nio

Hi all,
this is my first post in this forum.
I would like to know your opinion about the "old" java.io and the "new" java.nio, I have executed many tests using both the packages and I haven't seen so many differences in terms of time and performance.
My tests consist of doing file copy for example, using some files of different size and reading them in different block size also.
Other tests were the hash value calculation, also in this case I did different tests reading the files in different blocks size.
Finally I can say that I haven't seen so many differences in terms of time and performance like said in other articles found on the net.
I haven't yet tried executing test of reading/writing files from many threads. Is there someone that have tried it?
Your opinion?
Best regards.
Raffaele

Hi all,
this is my first post in this forum.
I would like to know your opinion about the "old" java.io and the "new" java.nio, I have executed many tests using both the packages and I haven't seen so many differences in terms of time and performance.
My tests consist of doing file copy for example, using some files of different size and reading them in different block size also.
Other tests were the hash value calculation, also in this case I did different tests reading the files in different blocks size.
Finally I can say that I haven't seen so many differences in terms of time and performance like said in other articles found on the net.
I haven't yet tried executing test of reading/writing files from many threads. Is there someone that have tried it?
Your opinion?
Best regards.
Raffaele

File Locking using java.nio package

I am trying to lock a file using FileChannel's lock method on a CIFS file system. My application is on windows and the CIFS is accessed using UNC format ( \\1.2.3.4\blah.. ). I get the following error
     java.io.IOException: The network request is not supported
     at sun.nio.ch.FileChannelImpl.lock0(Native Method)
     at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:750)
     at java.nio.channels.FileChannel.lock(FileChannel.java:865)
From the same machine using the same UNC formatm, when I lock the file using windows C API, I can successfull lock.
Anyone has any idea why JVM can not lock even using nio package. Shouldnt it use the same ( or similar ) API under the hood as its clear by the package name java.nio
The C program I am using is
#include <io.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/locking.h>
#include <share.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void main( int argc, char **argv )
int fh,numread;
char buffer[40];
char filename[500];
/* Quit if can't open file or system doesn't
* support sharing.
if ( argc > 1 )
          strcpy(filename,argv[1]);
else
          strcpy(filename,"locking.c");
fh = sopen(filename , O_RDWR, SHDENYNO, //_SH_DENYRW,
SIREAD | SIWRITE );
if( fh == -1 )
exit( 1 );
/* Lock some bytes and read them. Then unlock. */
if( locking( fh, LKNBLCK, 30L ) != -1 )
printf( "No one can change these bytes while I'm reading them\n" );
numread = _read( fh, buffer, 30 );
printf( "%d bytes read: %.30s\n", numread, buffer );
lseek( fh, 0L, SEEK_SET );
     printf( "Press a key to unlock the file.....\n" );
     getchar();
     locking( fh, LKUNLCK, 30L );
printf( "Now I'm done. Do what you will with them\n" );
else
perror( "Locking failed\n" );
_close( fh );
}

It uses LockFile and LockFileEx. Please check in the MSDN documentation the peculiarities of such APIs.
JNIEXPORT jint JNICALL
Java_sun_nio_ch_FileChannelImpl_lock0(JNIEnv *env, jobject this, jobject fdo,
                                      jboolean block, jlong pos, jlong size,
                                      jboolean shared)
    jint fd = fdval(env, fdo);
    HANDLE h = (HANDLE)_get_osfhandle(fd);
    DWORD lowPos = (DWORD)pos;
    long highPos = (long)(pos >> 32);
    DWORD lowNumBytes = (DWORD)size;
    DWORD highNumBytes = (DWORD)(size >> 32);
    jint result = 0;
    if (onNT) {
        DWORD flags = 0;
        OVERLAPPED o;
        o.hEvent = 0;
        o.Offset = lowPos;
        o.OffsetHigh = highPos;
        if (block == JNI_FALSE) {
            flags |= LOCKFILE_FAIL_IMMEDIATELY;
        if (shared == JNI_FALSE) {
            flags |= LOCKFILE_EXCLUSIVE_LOCK;
        result = LockFileEx(h, flags, 0, lowNumBytes, highNumBytes, &o);
        if (result == 0) {
            int error = GetLastError();
            if (error != ERROR_LOCK_VIOLATION) {
                JNU_ThrowIOExceptionWithLastError(env, "Lock failed");
                return sun_nio_ch_FileChannelImpl_NO_LOCK;
            if (flags & LOCKFILE_FAIL_IMMEDIATELY) {
                return sun_nio_ch_FileChannelImpl_NO_LOCK;
            JNU_ThrowIOExceptionWithLastError(env, "Lock failed");
            return sun_nio_ch_FileChannelImpl_NO_LOCK;
        return sun_nio_ch_FileChannelImpl_LOCKED;
    } else {
        for(;;) {
            if (size > 0x7fffffff) {
                size = 0x7fffffff;
            lowNumBytes = (DWORD)size;
            highNumBytes = 0;
            result = LockFile(h, lowPos, highPos, lowNumBytes, highNumBytes);
            if (result != 0) {
                if (shared == JNI_TRUE) {
                    return sun_nio_ch_FileChannelImpl_RET_EX_LOCK;
                } else {
                    return sun_nio_ch_FileChannelImpl_LOCKED;
            } else {
                int error = GetLastError();
                if (error != ERROR_LOCK_VIOLATION) {
                    JNU_ThrowIOExceptionWithLastError(env, "Lock failed");
                    return sun_nio_ch_FileChannelImpl_NO_LOCK;
                if (block == JNI_FALSE) {
                    return sun_nio_ch_FileChannelImpl_NO_LOCK;
            Sleep(100);
    return sun_nio_ch_FileChannelImpl_NO_LOCK;
}

Java NIO - TCP segment size abnormally low

Similar Messages

Maybe you are looking for