TCP out-of-order at IPS

Dear All,
We have a setup the IPS 4510 working inline mode with strict inspection turn on. we have detected some latency issue accessing the internal website. So we did some capture at the IPS interface. We found that there's a lot of out-of-order packet and DUP ACK detected by IPS which causing the normalizer engine buffer full and could not handle anymore request. As a work around we put the IPS in asymmetric mode where it turn off the IPS normalizer engine.
I need some opinion on possibilities why the Out of order and DUP ACK happen.
We are seeing quite a lot of Out-of-order, DUP ACK and TCP zero window in TCP stream that we captured.
The topology is quite straight forward:
Internet ----WAN ROUTER ----- IPS4510 ----- ASA ----- Web server
There's no redundancy or load balance for the ASA or WANROUTER.
Im hoping for some opinion and idea on how to tackle this issue.
Thank you very much

Hi
bumping out an old thread since the issue still on going.
I already discussed with TAC regarding the issue and 2 option that she gave
+ asymmetric mode (Which we rejected as permanent solution)
+ Event action filter
I'm currently looking at this solution and plan to implement it in the IPS.
I need to consider a few things and also suggestion
+ The signature engine involve is Normalizer engine (specifically sig 1330)
+ is it possible to customize this signature or should we just go for Event action filter?
need opinion and pro and cons of this.
Thanks a bunch

Similar Messages

ACE Dup ACK and TCP Out-of-order

Hi,
I have a pair of FT ACE 4710 offloading https traffic to a couple of webservers. We are seeing very high network utilisation when I capture the client facing port of the active ACE. There appears to alot of duplicate ACKs and TCP out-of-order packets (as shown by wireshark). Does anyone know if this is a problem with the ACE or "normal"
Thanks

I've seen some similar behaviour with the ACE Module and Apache webservers. To mitigate this I've configured the following which seems to work.
On the ACE Module
parameter-map type http ALL-HEADERS
persistence-rebalance
parameter-map type connection TCP-OPTIONS
set tcp syn-retry 5
tcp-options timestamp allow
policy-map multi-match test-policy
class http-vip
    loadbalance vip inservice
    loadbalance policy http-test-pm
    loadbalance vip icmp-reply active
    appl-parameter http advanced-options ALL-HEADERS
    connection advanced-options TCP-OPTIONS
On Apache here are the two test results with keepalive on and off
httpd.conf
KeepAlive Off
MaxKeepAliveRequests 1024
KeepAliveTimeout 30
MK-ACE01/001# show serverfarm MK-FARM-sf
serverfarm     : MK-FARM-sf, type: HOST
total rservers : 8
                                                ----------connections-----------
       real                  weight state        current    total      failures
   ---+---------------------+------+------------+----------+----------+---------
   rserver: MK-HOST10
       10.10.1.10:0          8      OPERATIONAL 321        510863     16442
   rserver: MK-HOST11
       10.10.1.11:0          8      OPERATIONAL 304        512718     16276
   rserver: MK-HOST12
       10.10.1.12:0          8      OPERATIONAL 286        524207     17257
   rserver: MK-HOST13
       10.10.1.13:0          8      OPERATIONAL 291        516987     16626
   rserver: MK-HOST14
       10.10.1.14:0          8      OPERATIONAL 291        513016     16594
   rserver: MK-HOST15
       10.10.1.15:0          8      OPERATIONAL 311        510177     16434
   rserver: MK-HOST16
       10.10.1.16:0          8      OPERATIONAL 345        516340     16708
   rserver: MK-HOST17
       10.10.1.17:0          8      OPERATIONAL 282        513046     16418
httpd.conf
KeepAlive On
MaxKeepAliveRequests 1024
KeepAliveTimeout 30
MK-ACE01/001# show serverfarm MK-FARM-sf
serverfarm     : MK-FARM-sf, type: HOST
total rservers : 8
                                                ----------connections-----------
       real                  weight state        current    total      failures
   ---+---------------------+------+------------+----------+----------+---------
   rserver: MK-HOST10
       10.10.1.10:0          8      OPERATIONAL 0          553        0
   rserver: MK-HOST11
       10.10.1.11:0          8      OPERATIONAL 0          551        0
   rserver: MK-HOST12
       10.10.1.12:0          8      OPERATIONAL 0          552        0
   rserver: MK-HOST13
       10.10.1.13:0          8      OPERATIONAL 0          555        0
   rserver: MK-HOST14
       10.10.1.14:0          8      OPERATIONAL 0          554        0
   rserver: MK-HOST15
       10.10.1.15:0          8      OPERATIONAL 0          551        0
   rserver: MK-HOST16
       10.10.1.16:0          8      OPERATIONAL 0          550        0
   rserver: MK-HOST17
       10.10.1.17:0          8      OPERATIONAL 0          550        0
This seems to of reduced the large number or re-transmits and dup-acks.

TCP out of order packets

Hi,
We are getting TCP out of error packets while sending requests to outside. Though we can access the internet and also connectivity is fine. But some of the application is not working due to this error, specially TCP based application as on the remote side they are not accepting two requests from our network. That means two requests are going from our network with each of the request sent to outside network.
We have 4-5 vlans and intervlan routing is configured. Could somebody pls. let me know the reason of this and how can I solve this problem?
Thanks,
Pawan

Good mornning.
We have many application similar to the case you have explained.
Our links use satelite connection and the delayare between: 600 msec to 1000 msec.
Delays over 1000 msec generates delay and connectivitiy problems with tcp applications.
What is the delay between your endpoints ?
Do you have access to internet router ? so can you tell us if there are packets drops in the interfaces ?
Waiting your answer.

ACE duplicate ack and tcp out-of-order errors

Hi,
I have just performed a capture using a NAM in my 6500 on the port attached to my ACE appliance.
What i have noticed in the capture is a lot of duplicate ack errors and tcp out-of-sync errors.
The reason we found this was becuase the link utilisation per session seems higher than we expected, hence are the errors adding to this and is there any way to remedy them?
Thanks
Scott

Hi Scott,
I'm not sure why you would see duplicate packets, although when you use SPAN, I know you can see them when you configure it to capture both directions on a VLAN. This is because you see each packet as it enters and leaves the VLAN. I don't know if that would apply to a NAM.
One thing you could do is use the ACE 4710's built-in capture utility to see if you see the same symptoms from an alternative source. This is covered in the Capturing Packet Information section of the configuration guides.
Hope this helps,
Sean

Help with TCP out-of-order packets Wireshark capture

Hello everyone, we have a bit of an odd issue. Can you take a look at the attached capture file and tell me what's broken? Please change the file extension from .txt to .pcapng and open with Wireshark.
We have a major issue where clients cannot retrieve data from the server at 10.10.7.27.
Server is behind the firewall at 172.18.123.4 which is configured to NAT the traffic coming through.
Please advise.

It's actually from anywhere. The DNS resolves the website address to a global address. So regardless of the source (inside or out), you hit the firewall and get routed to 10.10.7.0 network. The firewall's LAN interface shares the same VLAN as the DMVPN head-end router's LAN interface. From the DMVPN head-end router, it goes over the DMVPN cloud (i.e. back over the internet) to our office in Florida where this site is being hosted.
The capture I grabbed was by SPAN port between the two LAN interfaces showing transactions between the firewall's LAN interface and the server's IP address on the 10.10.7.0 subnet.
Site uses HTTPS and we have other servers in the same subnet (10.10.7.0) that are accessible in the same manner. I did SPAN the ports for another webserver and did see a lot of TCP OOO and re-transmissions however not as bad as this one.
I do have a theory, please feel free to correct me. Request comes in on the WAN interface, gets NATed by the firewall and sent to the DMVPN router, router encrypts the packet and places it on the wire, once the remote DMVPN peer receives the packet, it decrypts it and then sends it out it's DMZ interface connected to another application firewall. This firewall checks the packet and then sends it to the web server hosting the content. The process is reversed for reply traffic. On top of all this, the content is served over HTTPS therefore more encryption/decryption. This seems like too much handling of the packet to me? When the source computer sends a request, it simply times out or spends too much time within our own network causing the source to resend the request?

Signature 1330-X: TCP segment out of order - what does it mean?

Hi,
on a customer's site, on one of their IPS, I get a lot of sig 1330 alerts, mainly those two:
1330-12: TCP segment is out of order. If the signature status is set to disabled, the packet will be passed to all engines that are not stream based.
This signature will not produce an alert in promiscuous mode regardless of the signature status.
1330-17: TCP segment out of state order. If a packet in a stream causes this signature to produce an alert, processing will cease for that stream. This signature will not produce an alert in promiscuous mode regardless of the signature status
I'm not sure how to interpret these alerts correctly and/or how to troubleshoot further. Does anyone have an idea?
Thanks a lot,
Florian

Is your sensor monitoring more than one network segment?
If so then these alarms are common when a TCP connection crosses both networks and gets seen twice by the sensor.
This can confuse the sensor's tracking of the connection.
A common scenario is to have the sensor monitor both the Inside network or a firewall as well as the DMZ. When an internal user connects to the company's web server the traffic gets seen by the sensor both on the Inside network and in the DMZ. The sensor tries to put the packets from both networks together in order to try and monitor it as a single connection. Because the packets get modified by the firewall it often results in inconsitency between traffic on the 2 sides and causes the sensor to be confused about the connection.
The good news is that if this is your problem, then there are 2 easy workarounds.
1) If your sensor supports virtual sensors, then create a second virtual sensor. Assign one network to default vs0, and assign the other network to the new virtual sensor. This way each virtual sensor sees traffic on just one of the networks and won't become confused.
2) If your sensor does not support additional virtual sensors, or you've used up all 4 virtual sensors, then there is a configuration option within the virtual sensor configuration itself:
Inline TCP Session Tracking Mode
By default it is set to Virtual Sensor which is why it tries to put together packets from both networks to try and look at is a single connection and gets confused.
BUT it can also be set to Interface and Vlan. This configuration allows the virtual sensor to treat the traffic on each network independantly. The connection on the first network will be monitored independant of the connection on the second network. This will prevent the virtual sensor from getting confused.
The above is just my guess at what is going on in your network based on what we've seen on other networks. If this doesn't address the reason for the signature triggerings, then please respond back with more information about your network.
It is possible that these could be a hacker trying to avoid detection by the sensor, but more likely something in your deployment is confusing the sensor.

Out of order packets via LWAPP?

I am capturing packets on a wireless client ftp'ing a file from a server. It's showing that there are a lot of out of order packets. Although, the file received is fine, I'm worried that it'll affect voice packets later on. I'm using Wism with Lightweight APs. I tested using autonomous AP to tranfer the file from the same server--> No problems. Any ideas why this is? Is it some thing I mis-configure on the WLC?
Thanks
Binh Dinh

Good mornning.
We have many application similar to the case you have explained.
Our links use satelite connection and the delayare between: 600 msec to 1000 msec.
Delays over 1000 msec generates delay and connectivitiy problems with tcp applications.
What is the delay between your endpoints ?
Do you have access to internet router ? so can you tell us if there are packets drops in the interfaces ?
Waiting your answer.

Data written to socket getting lost? or perhaps out of order?

I'm trying to fix a bug in Flashmog. The bug is described in more detail here.
Basically what is happening is that my Flash client claims that it is calling functions that don't arrive at the server -- or they arrive, but the socket data is out of order and therefore is garbled. I've stared at the source code for hours and tried a lot of trial-and-error type stuff and added trace statements to see if I can find the problem and I'm not having any luck.
In particular, there's class I have called RPCSocket that extends the AS3 Socket class so I can serialize data structures before sending them across a socket. At one point, this RPCSocket class calls super.writeBytes and super.Flush. It is the point at which I send all data out of my client. The data is binary data in AMF3 format.
          public function executeRPC(serviceName:String, methodName:String, methodParams:Array):void {
               if (!this.connected) {
                    log.write('RPCSocket.executeRPC failed. ' + methodName + ' attempted on service ' + serviceName + ' while not connected', Log.HIGH);
                    throw new Error('RPCSocket.executeRPC failed. ' + methodName + ' attempted on service ' + serviceName + ' while not connected.');
                    return;
               var rpc:Array = new Array();
               rpc[0] = serviceName;
               rpc[1] = methodName;
               rpc[2] = methodParams;
               var serializedRPC:ByteArray = serialize(rpc);
               if (!serializedRPC) {
                    log.write('RPCSocket.executeRPC failed. Serialization failed for method ' + methodName + ' on service ' + serviceName, Log.HIGH);
                    dispatchEvent(new IOErrorEvent(IOErrorEvent.IO_ERROR, false, false, 'RPCSocket.executeRPC failed. Serialization failed for method ' + methodName + ' on service ' + serviceName));
               super.writeUnsignedInt(serializedRPC.length);
               super.writeBytes(serializedRPC);
               super.flush();
          } // executeRPC
Can someone recommend a way for me to store without corruption, conversion, or filtering or translation of any kind *all* of the information sent across this socket? I'd like to write it to a file. I'm guessing that keep a global ByteArray var and storing the info there might work, but I'm wondering how I might get the contents of that ByteArray into a file so I can inspect it.
Also, I'm wondering if I might be able to inspect what flash actually sends out on the socket? I have a sneaking suspicion that data I supply to super.writeBytes may be sent out of order or may not actually get sent across the socket. This bug I'm talking about only seems to happen under high-stress situations when I'm sending dozens of messages per second across this one socket.

oops...forgot link to bug description: http://flashmog.net/community/viewtopic.php?f=5&t=549

Photos syncing from folder to iPhone all out of order?

I have 2 folders synced to my iPhone 4 photo app.
They were previous camera rolls before I set up my phone as a new device. They are all named in order and taken in order, for example IMG_001.jpg was my first picture in my old camera roll. IMG_822.jpg was the last photo taken before I did the backup, and is the last in the folder.
The folder I am syncing from on my windows computer displays all of the pictures by filename (and because of that, by date) but when syncing with iTunes they end up on the phone ALL out of order. I can't find any rhyme or reason to how they are displaying. Most of the earlier photos show up at the beginning like they should, and the last at the bottom of the scroll appear to be mostly right, but then it's completely random from there.
I've tried unsyncing them, removing them, and then adding them back again and nothing seems to work. I've even tried renaming the IMG_xxx numbers to different things and they still appear to sync in the same random order.
Am I doing something wrong, or is this a huge bug for someone with OCD like me? lol

This is highly annoying.
I have probably 15-20 Smart Albums in iPhoto 8.1.2 in OS10.6.4 that are syncing to iOS4.0
On the old software, my Smart Albums for photos would show up AS the smart albums I created. Now they show up in the iPhone one giant mess called "Photo Library".
I can view photos broken up under "Events" on the iPhone by specific dates, but the dates don't tell me what the name of the event was, and some Smart Albums had multiple dates grouped together in one Smart Album and I don't really want them broken out separately by date. I want them organized how I organized them...
In short, I want my photos grouped the way I have explicitly grouped them. It used to work, it doesn't now.
Since iOS4.0 I am also getting sync errors all the time in iTunes. "iPhone cannot be synced because the required file cannot be found." WHICH FILE??? tell me and I will delete it...
I've rebuilt the entire database in iPhoto (took 4 hours). I've scanned iTunes and removed all broken links and duplicate files. I've scanned the hard drive with the disk tool. I've emptied the iPhone and reinstalled iOS4 on it 5 times. I've turned syncing off for all non-essential applications and stuff I don't actually sync like voice memos. Same problem.
I think I want my iOS 3 back. I don't really like multitasking anyway, always having to turn off applications for the one or two times in a million where task switching if helpful...
Ugh....

The music I sync into my iPod touch is all literally out of order and in the library it's not. Why is this?

I checked the starting time for songs in the library and on my iPod and they all say 0:00. What I mean when I say it's out of order is for example, the first ten seconds are skipped and placed somewhere in the middle of the song and some other parts are skipped then played later. I didn't have this problem before I updated iTunes for my new iPod, now when I tried syncing onto my old iPod as well, the same thing happens. So there's nothing wrong with my iPod. I want to know how I can fix this on iTunes or if it's just a glitch on my iTunes?

Something you can try is resetting it all Settings > General > Reset > Reset all settings. Or go to Settings > Store & sign out & log back in. Another thing you can do is restore your ipod. This has happened to me but the only thing i did was that i created a new account but try doing the things above i just told you. :)

Video and audio not sync when i play all forms of video. from music videos, to tv shows. but they play perfect on my iPod. and tv shows display out of order on iPod, but in correct order on iTunes/computer. I even ran a virus check just to make sure.

audio and video do not syncronize. all forms of videos. music videos to tv shows. what do i need to do the syncronize them again
they are in sync on iPod, but not on computer.
another issue that might be from the same cause is, the order of episodes of the tv shows i have downloaded, they appear in order on computer, but on iPod they show out of order.
any tips on how to fix this would very much help.
thank you.

Welcome to Apple Discussions!
An old version of iTunes will cause this and mess up the videos on your iPod. Always make sure you always use an up-to-date version of iTunes.
Make sure all the software on your computer is up to date...
iTunes
iPod Updater
Updating iPod's Software
You will then need to restore the iPod and then let iTunes update your iPod and put your music, videos, etc. back onto your iPod...
Restore the iPod
btabz

Opening and closing stock with sales order and with out sales order

hello,
any body please help me my client want to check opening stock and and closing stock in areport.
material contains batch and some material with sales order and some are with out sales order. my client is asking this in a single layout. please tell me isthere any teport or bapi or function module to get this report.
this is very uregent. and layout is requesting like opening stock, production stock, sales stock and closing stock.
please guide me to get this report.
Thanks & Regads
Bhakta

Transaction MB5B
For sales order related use specila stock as E and use radio button indicator in stock type
For stock w/o sales order, use special stock indiactor as "space' and stock type valuated stock.
to get a perticular month,opening stock/closing stock enter the start date/end date as month start and end date.
See the o/p which will give the stock as required by you

Keys out of order -- follow up to earlier post

I ran the install disc Disc Utility Test and the Hardware Test disc on the computer since I posted my first question earlier tonight "Computer won't finish start-up -- any suggestions". The Hardware Test came back fine. The Disc Utility test on the install disc says: Keys out of order. It does not, however, explain how to repair this problem on the hard drive.
Is there any way to fix this problem on the hard drive?
Power Mac G4 Quicksilver Mac OS X (10.3.9)

Invest in DiskWarrior. I had the same problem and DiskWarrior fixed it. No other utility programs worked for this problem. DiskWarrior is very inexpensive. Ordered online in the afternoon. It was delivered to my office the NEXT morning.

Camera roll pictures are out of order after upgrading to iOS 8.1

It's nice that iOS 8.1 brings camera roll back. However, pictures are out of chronological order. I'm using iPhone 5s. I upgraded iOS from 8.0.2 to 8.1, and noticed this issue.

That's weird cause my videos are all interspersed with everything else like they used to be, which unfortunately means it's inconsistently screwing up too. My pics start with 2007 (first year i got an iphone) then run up to about 2014 vaguely in order, then back to 2013 where everything's all out of order, then down to 2012 again (also out of order), then 2011, 2010, 2009 etc like it's running backwards to 2007. Then once it hits 2007 again, it jumps to this past july.
SO aggravating!

Is there a way to reorganize out-of-order tracks in iTunes 11?

Hello all--
In iTunes 10, if an album appeared with the track numbers out of order, I could just click on the track number tab within each individual album and the tracks would be resorted by track number. In iTunes 11, I can't find a way to do the same thing. For example, I imported an album and now it starts out with track 4, then goes to 15, etc., but can't get them to sort sequentially. Any help would be greatly appreciated.
I tried looking at one out-of-order album in the "songs" view, based on a similar question, but every time I try to search for the album under "songs" iTunes shows the result under "albums".

here is a link to macosxhints.com that tells you how to output all of your feed URLs to a terminal window. from there you can cut and paste them into your new feed reader of choice...
http://www.macosxhints.com/article.php?story=20080319094830396
and if you are in the marked for a new reader may i suggest David Watanabe's excellent NewsFire. he had just recently made it totally free as a gift to the macintosh community.
best of luck.

TCP out-of-order at IPS

Similar Messages

Maybe you are looking for