IDSM-2 Inline Vlan Pair - Duplicate Packets

Question

Dear All
We have a setup where two IDSM-2 modules are ether-channeled together in a single 6513 Chassis.
There is an FWSM module also, which acts as the default gateway for all internal VLANs.
Problem: IDSM show stat virtual-sensor command is showing tons of 'Duplicate Packets'
show statistics virtual-sensor | inc Duplic
Duplicate Packets = 2950967
Inline TCP Tracking Mode: Interface and VLAN
Topology:
Assume Client VLAN = 10 and Server VLAN = 60
IPS Inline VLAN Pairs:
10 >> 110 (Client VLAN)
60 >> 160 (Server VLAN)
Client >> Server Flow: (Layer 2):
[ClientPC] >>>> Access Switch (VLAN 10) >>>> Core SW >>>> IDSM-2 (VLAN 10--110 Pair) >>>> Core Sw >>>> FWSM VLAN 110 >>>>
FWSM VLAN 160 >>>> Core Sw >>>> IDSM-2 (VLAN 160--60 Pair) >>>> Server Switch (VLAN 60) >>>> [Server]
Core Switch IPS Etherchannel Setup:
Group 5: IDSM(A) and IDSM(B) Port x/7
Group 6: IDSM(A) and IDSM(B) Port x/8
Some VLAN Pair(s) are on interface x/7 and others are on x/8
Because of the above issue, we see a lot of TCP normalization signatures being fired (as the IPS gets confused with duplicate packets seen for the same flow). Specially signatures 1330:12 :17 and :18.
It is also causing some applications to break (e.g. Veritas Netbackup 6.5). When I removed the DENY action from these signatures, our IPS started having stability issues (This could also be due to E3 upgrade)
Should we change the Tracking mode to 'VLAN' only, OR any other possible solution?. Should not the 'interface and vlan' setting be sufficient?.
Regards
Farrukh

Answer 1

This will take some traffic analysis to determine what is going wrong.
You might need to place a sniffer to watch the traffic on the client where the backup software is running at the same time that you capture the traffic on the sensor.
Look to see if there are any differences in the traffic.
Look for any anomalies in the traffic.
Look to see if maybe the backup software is not using a standard TCP connection (is it jumping the tcp sequence numbers in any abnormal way?)
You might also try some things on the sensor to determine if the sensor itself might have an issue.
Determine if the connction passes through 2 connections (inline vlan pairs) monitored by the sensor.
If you can, you might try removing both of the pairs from the virtual sensor. (don't delete the pairs, just remove them from the virtual sensor so they won't be analyzed)
And see if the backup works.
If it does then just add in one pair, and see if it keeps working.
If it has errors with just the one pair, then the problem is likely not because of the connection being monitored twice.
Something else must be weird about the connection.
If the problems are only seen when having both pairs in the same virtual sensor, then try placing the pairs in different virtual sensors and see if the problem goes away.
If the problem goes away when in different virtual sensors, then there may be an error in the inline tcp session tracking code that should track connections separately for each interface/vlan.