Sniffer Trace on ACE w/VACLs and One-Arm Design

Wow...that was a mouthful of a title!
Here is what I'm trying to accomplish. There is an application that is having issues. This application is being load balanced by the ACE. The ACE is configured in a One-Armed design. Essentially the application flow is as follows:
client --> ACE VIP --> SNAT Pool --> rserver and then the reverse.
The vlan for my ACE is 3002. It is the only vlan in this context. I have a WildPackets OmniEngine connected to port on the 6500. Here is its config:
interface GigabitEthernet x/xx
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
switchport capture
switchport capture allowed vlan 3002
no ip address
no cdp enable
Here is the problem. When I take a trace I only see the back half of the conversation. That is I only see from the SNAT pool IPs to the rservers and back. I need to be able to see the conversation between the client IPs and the VIP. Does anyone know how this can be done? If you need more details or have questions please fire away! Thanks for the help...
bc

This can be done by setting up a monitor session on the Sup, with the
TenGig/1 as SPAN
source, and a trunk port as SPAN destination.
For example, if the ACE is in slot X, the configuration would be:
monitor session 10 source interface TeX/1
monitor session 10 destination interface Giy/z
The configuration for this port would be:
int giy/z
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
Syed Iftekhar Ahmed

Similar Messages

ACE 4700 one-arm design with SSL termination

Hi,
We are evaluating the one-arm design for the ACE 4700 and need some clarifications:
1. Are there any limitations in the one-arm design and the SSL offloading
2. Can the ACE be configured with an IN and an OUT vlan to the router
CLIENT -> Router -> ACE IN -> ACE OUT -> Router -> Server Vlan
so that the SSL and the clear text traffic is in a separate Vlan?
3. In some sample configuration i saw SNAT configuration on the ACE to modify the client IP. This i assume is for instructing the return traffic from the server to go through ACE? Using SNAT we eliminate the requirement for NAT or PBR on the router? Will i still be able to insert the client IP address after the SSL offload?
I would appreciate if you can share some sample configs
Regards,
George Georgiou

There are two ways to implement One Arm topology.
1. One Arm with PBR & 2.One Arm with SRC NAT
PBR/Source Nat is needed to ensure that the return traffic from Real Servers should not bypass ACE.
1. Are there any limitations in the one-arm design and the SSL offloading
The limitations/config issues I can think of are following
One ARM with PBR:
Direct access to Servers require the enabling of Assymtric routing (by turning off Normalization). If direct server access is not required then you dont need to enable assymtric routing. Now for these assymetric connection (Direct Server Access return traffic) its required to purge idle connections more frequently (default being one hour).
One ARM with SRC NAT:
You will loose the client information. Server logs will show the connections initiated from NAT IP Pool configured on ACE.
2. Can the ACE be configured with an IN and an OUT vlan to the router
CLIENT -> Router -> ACE IN -> ACE OUT -> Router -> Server Vlan
so that the SSL and the clear text traffic is in a separate Vlan?
Yes you can do that but wouldnt it make it routed mode topology?
3. In some sample configuration i saw SNAT configuration on the ACE to modify the client IP. This i assume is for instructing the return traffic from the server to go through ACE? Using SNAT we eliminate the requirement for NAT or PBR on the router? Will i still be able to insert the client IP address after the SSL offload?
As I said earlier you loose the Source IP address with SRC NAT. But with ACE you have an option to use header-insert and insert this source ip as an HTTP Header.
Details at
http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A1/configuration/slb/guide/classlb.html#wp1040008
HTH
Syed Iftekhar Ahmed

Virtualisation and One Arm mode

Hi All ,
Is it possible to make one context and one arm mode and rest in normal
-parvees

Yes, no restriction. 1-arm mode is just placing the VIP in server's subnet and using source NAT for clients.

Source Nating in CSM one armed design

What is the best practice for creating Source Nating in CSM One armed design? I am doing CSS to CSM migration. I have created the NATPOOL used the VIP address like natpool CSS0 10.xxx.xx.xxx 10.xxx.xx.xxx netmask 255.255.255.0. I did experience some latency after migrating to CSM. Then I used diffrent Ip address is the NATPOOL that improved the latency. Is there any documentation which clearly explains this issue?
Thanks

the natpool will have no impact on performance.
The problem must come from somewhere else.
You should capture a sniffer trace and verify what is going on.
Gilles.

CSS 11503 One-arm Design and Server Default Gateway

Our problem is determining the correct default gateway for our web servers. All IP addresses are in the same subnet (VIP, interfaces, and servers). Should the servers default gateway be the L3 switch, or the CSS?
Thanks!
Tom

Hi Tom,
If you have one arm mode, you might have problems with asymmetric flows, due that the CSS behaves similar to a firewall when it comes to flows, as it needs to see both sides of the flow ( client and server side ) in order to handle things correctly. Having this kind of setup, and even when the server pointing to the CSS as its default gateway, ICMP redirects might force the traffic to change dynamically.
You can put as default gateway the L3 switch, but you need to force the traffic that has been load balanced by the CSS to go back to the CSS, otherwise the flow would fail. You can do this by using a group on the CSS, adding the service with the following command: 'add destination service xxxx'. This would NAT the client's IP address for the VIP that you use on the group and would force the flow to go back to the CSS.
Another thing that you can do is to use the CSS as the server's DG, but you must make sure that all L3 devices, including the CSS have ICMP redirects turned off on this subnet. If you have a firewall on this subnet, you would need to turn off proxy ARP as well.
I hope you find this helpful. Thanks!
Regards,
Jose Quesada.

CSS11500 one arm design configuration assistance.

Is it possible to configure the CSS11500 as single arm design? if yes how to configure the source nat on the CSS11500, it is not possibe for me to change the default gateway as well as configure CSS as inline.
Regards

yes you can configure CSS in one armed mode. You would do the nat with a group config ie:
service yada
ip address 192.168.20.40
active
content yadayada
vip address 192.168.20.55
add service yada
group yadayadayada
vip address 192.168.20.55
add destination service yada

We are going to Great Britain this summer. Do we need voltage converters as we'll plug adapters for our iPads (one is 2nd generation and one 4th)

We are going o Great Britain this summer. Do we need voltage converters as well as plug adapters for our iPads (one is 2nd generation and one 4th)

No. Since Apple power converter work from 110 to 240 volts, no power converter is reguired.
You will need a pin adaprt. I have found my local Ace hardware store to have the best deal on them.
Allan

How to see the Source IP Address of a client using ACE One-armed-mode to load balance HTTP proxy request

I'm using an Ace 4710 Appliance deployed in One-Armed mode, using Source NAT to loadbalance HTTP request to a couple of Proxy servers.
Everything is working fine, but the thing is that I can't see the Clients IP addresses on Proxy's logs, so I can't keep track of them.
The Interfaces and Nat configs are:
interface vlan 200
description Server-Side-VLAN
bridge-group 5
nat-pool 5 10.1.1.5 10.1.1.5 netmask 255.255.255.0 pat
service-policy input VIPS
interface vlan 300
description Client-Side-VLAN
bridge-group 5
interface bvi 5
ip address 10.1.1.3 255.255.248.0
description Client-Server-Virtual-Interface
ip route 0.0.0.0 0.0.0.0 10.1.1.1
and the policy map looks like this
policy-map multi-match VIPS
class Port80
    loadbalance vip inservice
    loadbalance policy Port80
    nat dynamic 5 vlan 200
Resource assignment:
sticky ip-netmask 255.255.255.255 address both RESOURCE-CLASS
timeout 5
serverfarm Service80
Any suggestions will be appreciated,
Thanks

Hi Kanwal,
Thanks for your quick reply,
I've already tried this but it didn't work. The problem is that I don't manage the proxy servers so I rely on their skills to see the logs.
The Proxies are Squid. Do you know if they need to do something else on the servers to see that field of the HTTP header?
But I'll try again tomorrow and let you know how it goes.
Thank you again.

BPM with 2 JDBC outbound interfaces and one inbound file interface

Hi,
I have a scenario where data is coming from different databases into XI and after processing the data it creates a file at the target side. Some calculations are to be done in the middleware and the result will be sent through a file.
My question is
1. BPM will be used in this scenario?
2. There will 2 outbound interfaces(one for each database access), 2 abstract interfaces and one inbound interface(file). Please correct me if I am wrong.
3. I have to multiply the value of field from one database to a particular value in another database and get the final result. For e.g exchange rate from one database is multiplied to Transaction currency in another database and get the group currency value in the output. Can anyone give the sequence of steps in the intergration process for this scenario as I am new to BPM.
Thanks,
Aparna

Thanks a lot Liang and madhusudana for your responses.
Since I can use JDBC lookup for this in my nornal scenario I think no need for BPM.
I have created the advanced UDF for JDBC lookup code. Below is the code.
String Query = " ";
Channel channel = null;
DataBaseAccessor accessor = null;
DataBaseResult resultSet = null;
MappingTrace importanttrace;
importanttrace = container.getTrace();
// Build the Query String
Query = "Select home_cur_code FROM gltrxcp WHERE company_code = '" + a[0] + "' ";
try{
//Determine a channel, as created in the Configuration
channel = LookupService.getChannel("BS_JDBC_send","cc_JDBClookup_rec");
//Get a system accessor for the channel. As the call is being made to an DB, an DatabaseAccessor is obtained.
accessor = LookupService.getDataBaseAccessor(channel);
//Execute Query and get the values in resultset
resultSet = accessor.execute(Query);
importanttrace.addWarning("resultset" + resultSet );
for(Iterator rows = resultSet.getRows();rows.hasNext();){
Map rowMap = (Map)rows.next();
result.addValue((String)rowMap.get("HOME_CUR_CODE"));
importanttrace.addWarning("result" + result );
catch(Exception ex){
result.addValue(ex.getMessage());
finally{
try{
if (accessor!=null) accessor.close();
catch(Exception e){
result.addValue(e.getMessage());
But I am not getting the desired result. I am getting result as 'Null'.
Can you please help in resolving this issue. I have also used trace for find out where it is passing Null and found that 'result' is coming null. Do i also need to parse it before passing it to 'result'?
Can anyone send sample code for JDBC lookup.

ACE in one-arm model. VIP on Client Side, servers in other vlan

Hello All
i have a LAN whit many servers,but only 2 need to be balanced. So i think in one-arm model, due to the higth trafic that not be pass trought ACE.
i have a vlan 900 where is the client side and the VIP also. (10.0.9.64/26)
the servers are in vlan 503 (10.12.3.0/24)
it mi first design with ONE-arm but i thinks something is missing, because doesn't work.
the configuration is the next:
MSFC:
svclc module 1 vlan-group 1,2,
svclc vlan-group 1 503,900-902
svclc vlan-group 2 511
interface Vlan503
description OSS_&_Otros
ip address 10.12.3.253 255.255.255.0
standby 10 ip 10.12.3.254
standby 10 priority 150
standby 10 preempt delay minimum 305
interface Vlan900
description MSF_<->_ACE
ip address 10.0.9.126 255.255.255.192
end
access-list 101 permit ip 10.12.3.0 0.0.0.255 10.0.9.64 0.0.0.63
access-list 101 deny ip any any
route-map From_Server_OSS_to_ACE permit 10
match ip address 101
set ip next-hop 10.0.9.125
ACE_1/admin#
ip route 0.0.0.0 0.0.0.0 10.0.9.126
context OSS
allocate-interface vlan 511
allocate-interface vlan 900
allocate-interface vlan 902
member Max20
ACE_1/OSS# sh run
Generating configuration....
access-list EVERYONE line 10 extended permit ip any any
access-list EVERYONE line 20 extended permit icmp any any
rserver host OSS_FES_1
description OSS_Front_End_Server_1
ip address 10.12.3.140
inservice
rserver host OSS_FES_2
description OSS_Front_End_Server_2
ip address 10.12.3.150
inservice
serverfarm host SERVER_farm_OSS
rserver OSS_FES_1
inservice
rserver OSS_FES_2
inservice
class-map match-all VIP-OSS
2 match virtual-address 10.0.9.66 any
policy-map type loadbalance first-match OSS-LB-POLICY
class class-default
serverfarm SERVER_farm_OSS
policy-map multi-match OSS-POLICY-MAP
class VIP-OSS
loadbalance vip inservice
loadbalance policy OSS-LB-POLICY
loadbalance vip icmp-reply
interface vlan 900
description Clients-side
ip address 10.0.9.125 255.255.255.192
access-group input EVERYONE
access-group output EVERYONE
service-policy input OSS-POLICY-MAP
no shutdown
ip route 0.0.0.0 0.0.0.0 10.0.9.126
maybe a i need to allocate the vlan 503 in OSS Context, any advice?
Thanks in advace,
Gianni From Chile

Since you server are not behind the ACE in either bridge or routed mode add the follwoing to your config and use nat to get the traffic back to the ace.
This is how one-armed mode works.
ACE_1/OSS# sh run
Generating configuration....
access-list EVERYONE line 10 extended permit ip any any
access-list EVERYONE line 20 extended permit icmp any any
rserver host OSS_FES_1
description OSS_Front_End_Server_1
ip address 10.12.3.140
inservice
rserver host OSS_FES_2
description OSS_Front_End_Server_2
ip address 10.12.3.150
inservice
serverfarm host SERVER_farm_OSS
rserver OSS_FES_1
inservice
rserver OSS_FES_2
inservice
class-map match-all VIP-OSS
2 match virtual-address 10.0.9.66 any
policy-map type loadbalance first-match OSS-LB-POLICY
class class-default
serverfarm SERVER_farm_OSS
policy-map multi-match OSS-POLICY-MAP
class VIP-OSS
loadbalance vip inservice
loadbalance policy OSS-LB-POLICY
loadbalance vip icmp-reply
nat dynamic 10 vlan 900
interface vlan 900
description Clients-side
ip address 10.0.9.125 255.255.255.192
nat-pool 10 0.9.126 10 0.9.126 netmask 255.255.255.192 pat
access-group input EVERYONE
access-group output EVERYONE
service-policy input OSS-POLICY-MAP
no shutdown

[ACE] Real servers and VIP in the same VLAN

Hello.
I´m facing an issue because the real servers and the VIP address are in the same VLAN, when a request comes from an external client to the VIP (crossing an ASA firewall) , the ACK gets back using the IP of one of the real servers instead of the VIP so this traffic is blocked by our WAN firewall probably due the inspection rules.
My question is if there is some way make the VIP the address who ACK´s that requests? Creating a new VLAN would be complicated because there are other services already running on those real servers.
Thanks a lot,
Miquel

Hi Miquel,
Please do source nat on ACE so that return traffic gets sent to ACE and not FW. Pasting an example for you.
     ==========================================================================
     One-Armed Load Balancing with VIP, Servers, & NAT Pool on the Same Subnet
     ==========================================================================
login timeout 0
access-list ANYONE line 10 extended permit ip any any
rserver host SERVER_01
ip address 192.168.1.11
inservice
rserver host SERVER_02
ip address 192.168.1.12
inservice
rserver host SERVER_03
ip address 192.168.1.13
inservice
serverfarm host REAL_SERVERS
rserver SERVER_01
    inservice
rserver SERVER_02
    inservice
rserver SERVER_03
    inservice
class-map match-all VIP-30
2 match virtual-address 192.168.1.30 tcp eq www
class-map type management match-any REMOTE_ACCESS
description remote-access-traffic-match
2 match protocol telnet any
3 match protocol ssh any
4 match protocol icmp any
policy-map type management first-match REMOTE_MGT
class REMOTE_ACCESS
    permit
policy-map type loadbalance first-match SLB_LOGIC
class class-default
    serverfarm REAL_SERVERS
policy-map multi-match CLIENT_VIPS
class VIP-30
    loadbalance vip inservice
    loadbalance policy SLB_LOGIC
    loadbalance vip icmp-reply active
    nat dynamic 1 vlan 451
interface vlan 451
description Servers vlan
ip address 192.168.1.2 255.255.255.0
access-group input ANYONE
service-policy input CLIENT_VIPS
nat-pool 1 192.168.1.10 192.168.1.10 netmask 255.255.255.0 pat
no shutdown
ip route 0.0.0.0 0.0.0.0 192.168.1.1
Let me know if you have any question.
Regards,
Kanwal

Probe fail on Standby ACE in One-armed mode

Hi there
I'm Kilsoo.
I made One-armed mode using ACE.
Real servers are in away Vlan from ACE.
So, I configured the PBR with ACE alias ip address for the next-hop on the real server's gateway interface.
And, the probe from active ACE works well.
But, the probe from standby ACE was fail.
At this point, my first question
Is it normal situation that the probe fail from standby ACE????
So, I made the route-map for PBR like below for temporary solution.
route-map deny PBR 5
match ip address Probe_ACL
route-map permit PBR 10
match ip address L4_ACL
set ip next-hop <Alias IP address>
ip access-list extended Probe_ACL
pemit ip any <Standby ACE's IP address>
ip access-list extended L4_ACL
permit tcp <Real server's IP address> eq 80 any
Second question...
Do you have any other good solutions???
Thanks

Hi Cesar
Thanks for your reply.
But I think I was confuse when I wrote the message.
I used both ace's vlan ip address for next-hop ip address like your advice.
Do you know the standby ace can't check probe without route-map in one-armed mode like below diagram???
Backbone Router
         |
         |
         |
Supervisor --------------------ACE(vserver: 172.19.100.100)
         |         (vlan 200)
         |
         |
         |(vlan 110)
         |
         |
Real servers
(172.19.110.111)

Source IP in One armed Mode ACE

Hi,
How do we find actual Client Source IP address in One armed mode ACE for NON-HTTP application like LDAP,FTP and etc....

It's not possible. Insertion within header works only for HTTP and HTTPS with SSL offload.

ACE One Arm Mode vs Routed Mode

Gents,
When is it required to use the One Arm Mode and one do I use the routed mode? Actually I am confused and would really like to know the pros and cons of each?
Regards,
Hesham

Hi Hesham,
When you do not want to change the physical topology of your network then you usually go with ONE ARM mode.
Such as default gateway on server, IP addressing on servers. In this case client can access the server directly as well.
Its a flat network topology where your VIP and servers are in the same network ( VLAN ).
You use routed mode when you want to segregate the servers in seperate vlan and don't want to allow client to access it directly.
Client and VIP in same VLAN >>> ACE >>>>>> Server VLAN ( In this case we usually point the default gateway to ACE)
hope it helps.
regards,
Ajay Kumar

ACE inline VS one-armed based

Hello Forum, ;-)
I have 2 basic questions I am having doubts about it and would love to have some clarifications:
1) I configure in one ACE4710 (running 4.2.2) context a bridged interface and in another context the same interface, like here below :
---- Context Microsoft ----
ACE1/Microsoft# sh run
interface vlan 503
   bridge-group 3
   access-group input NONIP
   access-group input ALL
   access-group output ALL
   service-policy input POLICY
   no shutdown
interface vlan 1503
   bridge-group 3
   access-group input ALL
   access-group output ALL
   no shutdown
interface bvi 3
   ip address 120.223.22.30 255.255.255.0
   no shutdown
Then I move to the Juniper context and I try to create an interface (either L-2 or L-3) but it doesn’t work:
---- Context Juniper----
ACE1/Juniper(config)# int vlan 503
Error: VLAN creation is not allowed, shared bridged VLAN exists in another context
ACE1/Juniper(config)#
It gives ERROR!!
So if I configure an interface as bridged in one Context, I cannot configure it in another context??
2) If I want to migrate in context Microsoft from One-armed to inline (L-2 bridged), can I migrate one service at the time ( I.e. the config i showed above for context Microsoft, would it work also for one-armed based???)
Thanks so much for your explanations!!
Giulio.

Hello Giulio-
You can only share vlans in one-armed or routed modes. Think of it this way:
Interface vlan 10 and 11 are bridged on context C1. (bridged mode)
Interface vlan 12 and 13 are configured on context C2. (routed mode)
When you have routed mode, your server's gateway is configured to point to the ACE interface IP (or alias if you are have FT.) If a packet comes into the physical interface on the ACE, the processor has to decide which context it belongs to. Since the mac address is the interface on context X, it knows instantly where it goes. It will either hit a VIP, or be routed via the routing table.
If a packet arrived on vlan 12 or 13 and the MAC address did not belong to the ACE, it would drop the packet by basic routing rules. (think a client connected to a hub sees a packet destine to a MAC that is not its own, it drops/ignores the packet.)
In bridged mode, the gateway for your server is the router on the other side of the bridged vlan. I.e., you server is on vlan 10, the gateway is on vlan 11 and ace is bridging them together. When packets arrive to the physical interface, ACE knows the traffic arrived on vlan 10 or 11 which belongs to context C2. If the MAC address is not a VIP, ACE simply hucks the packet out of the other vlan. If you send traffic to the interface MAC that does not belong to a VIP, ACE drops it because it would not make sense to send a packet out the other vlan that has a MAC address that belongs to the interface of the ACE itself.
One-armed mode is simply routed mode with a single vlan and source NAT. Nothing special applies to how ACE handles the traffic versus routed mode with only a single vlan.
Now imagine this:
Interface vlan 10 and 11 are bridged on context C1.
Interface vlan 11 and 12 are configured on context C2.
Remember 3 things:
a.) ACE conserves MAC addresses - so the VIPs share MAC addresses with the interface.
b.) ACE will never communicate between 2 contexts directly.
c.) If you are in a routed mode and share vlans between 2 contexts, ACE will make each vlan have a unique MAC address. If you create unique vlans on each context, ACE uses the same single MAC across all vlans for all contexts.
With traffic that is destine to ACE's MAC address and the IP is a VIP, its not a problem - ACE could figure out which context the traffic belongs to (especially since vlan 11 would have unique mac addresses on each context. However, what if ACE recieved a packet to the interface 10 and 12 MAC address? How would it know if it belonged to the bridged or routed context if it was not a VIP IP? What about traffic that arrives that doesn't have the MAC of any of the interfaces? 2 different entirely behaviors would occur, ACE should drop the packet on the bridged context, and route the packet on the routed context.
So the bottom line is - you can't determine which context a packet would need to apply to in all circumstances if you tried to share vlans in a bridge mode across multiple contexts.
Regards,
Chris Higgins

Sniffer Trace on ACE w/VACLs and One-Arm Design

Similar Messages

Maybe you are looking for