Configuring the Catalyst 6500 Switch for IPS Inline Operation of the IDSM

Question

I understand how to configure the Catalyst 6500 switch so that the monitoring ports are access ports in two separate VLAN's for inline operation.
However, I don't see any documentation that describes how the desired VLAN traffic gets forced through the IPS.
In promiscuous mode, you can use VACL's to copy/capture and forward the desired traffic to the IDSM for analysis. I'm not seeing how to get the desired traffic through the IPS.
Note that the host 6500 is running native IOS 12.2(18)SXE.
Thanks for any assistance.

Answer 1

A tranparent firewall is a fairly good comparison.
Let's say you have vlan 10 with 100 PCs and 1 Router for the network.
If you want to apply a transparent firewall on that vlan you can not simply put one interface of the firewall on vlan 10. Nothing would go through the firewall.
Instead you have to create a new vlan, let's say 1010. Now you place one interface of the firewall on vlan 10 and the other on vlan 1010. Still nothing is going through the firewall. So now you move that Router from vlan 10 to vlan 1010. All you do is change the vlan, the IP Address and netmask of the router stay the same.
The transparent firewall bridges vlan 10 and vlan 1010. The PCs on vlan 10 ae still able to communicate to and through the router, but must go through the transparent firewall to do so.
The firewall is transparent because it does not IP Route between 2 vlans, instead the same IP subnet exists on both vlans and the firewall transparently beidges traffic between the 2 vlans.
The transparent firewall can do firewalling between the PCs on vlan 10 and the Router on vlan 1010. But is PC A on vlan 10 talks to PC B on vlan 10, then the transparent firewall does not see and can not block that traffic.
An InLine sensor is very similar to the transparent firewall and will bridge between the 2 vlans. And similarly an InLine sensor is able to InLine monitor traffic between PCs on vlan 10 and the Router on vlan 1010, but will not be able to monitor traffic between 2 PCs on vlan 10.
Now the router on one vlan and the PCs on the other vlan is a typical deployment for inline sensors, but your vlans do not Have to be divided that way. You could choose to place some servers in one vlan, and desktop PCs in the other vlan. You subdivide the vlans in what ever method makes sense for your deployment.
Now for monitoring multiple vlans the same principle still applies. You can't monitor traffic between machines on the same vlan. So for each of the vlans you want to monitor you will need to create a new vlan and split the machines between the 2 vlans.
In your case with Native IOS you are limited to only 1 pair of vlans for InLine monitoring, but your desired deployment would require 20 vlan pairs.
The 5.1 IPS software has now the capability to handle the 20 pairs, but the Native IOS software does not have the capability to send the 40 vlans (20 pairs) to the IDSM-2.
The Native IOS changes are in testing right now, but I have not heard a release date for those changes.
Now Cat OS has already made these changes. So here is a basic breakdown of what you could do in Cat OS and you can use in preparation for a Native IOS deployment when it gets released.
For vlans 10-20, and 300-310 that you want monitored you will need to break each of those vlans in to 2 vlans.
Let's say we make it simple and add 500 to each vlan in order to create the new vlan for each pair.
So you have the following pairs:
10/510, 11/511, 12/512, etc...
300/800, 301/801, 302/802, etc....
You set up the sensor port to trunk all 40 vlans:
set trunk 5/7 10-20,300-310,510-520,800-810
(Then clear all other vlans off that trunk to keep things clean)
In the IDSM-2 configuration create the 20 inline vlan pairs on interface GigabitEthernet0/7
Nw on each of the 20 original vlans move the default router for each vlan from the original vlan to the 500+ vlan.
At this point you should ordinarily be good to go. The IDSM-2 won't be monitoring traffic that stays within each of the original 20 vlans, but Would monitor traffic getting routed in and out of each of the 20 vlans.
Because of a switch bug you may have to have an additional PC moved to the same vlan as the router if the switch/MSFC is being used as the router and you are deploying with an IDSM-2.