ACE module only inserting X-Forwarder Header on first packet

Hi,
As above, I have a strange problem where if I use my proxy server to access an LB VIP then it is inserting the X-forwarding header for Each GET request.
However if I make the request direct from my PC (not via Proxy) it inserts the header on the first packet but no subsequent packets unless I restart the browser.
Any ideas????
Thanks
Scott

Hi Scott,
In the ACE documentation, check out the section on Configuring the ACE to Modify Headers on Every HTTP Request or Response.
I hope this helps,
Sean

Similar Messages

ACE Module vs ACE Appliance

Hello,
What is the difference between ACE Module and ACE Appliance? why the ACE Module is better? or ACE Appliance, what is the advantage between Module and Appliance.
anyone can explain me?
Best Regards

In the past Cisco has been shipping two line of Loadbalancing products
First line ( modules dedicated for 6500/7600 chassis ) includes CSM & CSM-S & SSLSM (for ssl offloading)
The other line comprises of appliance based CSS series products.
ACE module is a next generation module replacing CSM modules that fits into 6500/7600 chassis.
It gives you upto 16Gbps throughput (versus CSM's 4Gbps throughput).
ACE appliance is a next gen replacement of CSS line of appliance based products.
CSS appliances were used to come in different Hardware models with varied
performance capacities. ACE appliance is a single hardware with various licenses
used to scale the performance/features.Ace appliance supports upto 4Gbps of throughput.
Previously CSS & CSM code terminologies & command set was different. For example a real server
was termed as "service" in CSS & was called "real" in CSM . Similarly "probe" in CSM was "keepalive"
in CSS.
With ACE line of products you get the same terminologies & command sets for both
modules & Appliances.
ACE Appliance & ACE modules are functionality vise coming closer with every new release but
still there are some differences.
For example following ACE appliance features are not available in ACE module:
Appl optimization (flash forward, Delta Encoding)
Embedded Device manager
Http compression
Which one is better than the other really depends on your requirement
From Performance perspective Module give you much higher performance then Appliance.
SO if performance is your criteria the ACE module is better than ACE appliance.(Some performance metrics at the end of the post).
If you are looking for Application optimization & HTTP compression along with Loadbalancing
then it can only be achieved with ACE appliance.
If you are not using 6500/7600 series chassis in your environment then you can only use ACE appliance
(unless you are open to buy module+chassis due to performance requirement).
Some performance metrics
Ace Appliance supports 1 Million concurrent connections where as Ace Module supports 4 Million.
Ace Appliance supports 120K L4 conn/sec where as Ace Module supports 380K L4 conn/sec.
Ace Appliance supports 40K L7 conn/sec where as Ace Module supports 133K L7 conn/sec.
Ace Appliance supports upto 4Gbps throughput where as Ace Module supports 16Gbps throughput .
HTH
Syed Iftekhar Ahmed

ACE CLIENT CERTIFIATE INSERTION IN HEADER

Hy guys.
I have a doubt regarding the client cert insertion in the https header.
The exact problem is that in the old SSL module we had an option like this:
policy http-header cert_pass
client-cert pem
As you can see, we configure the option to pass the complete certificate in pem format in one header.
I'm unable to find this optiono in ace 5.1(3) version.
Any idea?
Thanks!

Hi David,
May be i didn't understand. Is this what are you looking for? You can find it in the same link.
Configuring HTTP Header Insertion of SSL Client Certificate Information
When you configure the ACE for client authentication, you can instruct the ACE to provide the server with information about the client certificate that the ACE receives from the client. This SSL session information enables the server to properly manage the client request and can include certificate information such as the certificate serial number or the public key algorithm used to create the public key in the certificate. To forward the SSL session information to the server, the ACE inserts HTTP headers containing the client certificate fields that you specify into the HTTP requests that it receives over the client connection. The ACE then forwards the HTTP requests to the server.
Note To prevent HTTP header spoofing, the ACE deletes any incoming HTTP headers that match one of the headers that it is going to insert into the HTTP request.
When you instruct the ACE to insert SSL client certificate information, by default, the ACE inserts the HTTP header information into every HTTP request that it receives over the client connection because persistence rebalance is enabled by default. If you do not want the ACE to insert the information into every HTTP request that it receives over the connection, disable persistence rebalance in an HTTP parameter map. You can also instruct the ACE to insert the information into every HTTP request that it receives over the connection by creating an HTTP parameter map with the header modify per-request command enabled. You then reference the parameter map in the policy map that the ACE applies to the traffic. For information about creating an HTTP parameter map, see the Cisco 4700 Series Application Control Engine Appliance Server Load-Balancing Configuration Guide.
Note You must have the ACE configured for client authentication to insert an HTTP header with SSL client certificate field information (see the "Enabling Client Authentication" section). If you configure header insertion but do not configure the ACE for client authentication, no header information is inserted and the counters that track the header insertion operation do not increment (see Chapter 6, "Displaying SSL Information and Statistics").
Regards,
Kanwal

Bizarre ACE module behavior

Hi,
I configured a new serverfarm with leastconns predictor for two servers on our ACE module Version A2(2.3). Probes (show probes XX detail) to the servers are successful and both servers are operational (show serverfarm APPLI detail) but connections are directed only to one server.
When I deactived the server which is receiving the connections (no inservice), the ACE start to direct connection to the second server.
There are several serverfarm, configured the same way, that are Loadbalancing traffic as correctly.
Here is a sample of my config
serverfarm host TEST_443
predictor leastconns
probe TEST_443_PROBE01
rserver TEST_RS01 443
    inservice
rserver TEST_RS02 443
    inservice
sticky http-cookie TEST_HTTPS TEST_443_STKY
cookie insert
timeout 720
replicate sticky
serverfarm TEST_443
probe http TEST_443_PROBE01
port 443
interval 20
passdetect interval 60
passdetect count 5
request method get url /test
expect status 302 302
connection term forced
policy-map type loadbalance first-match TEST_L7PLB_HTTPS
class class-default
    sticky-serverfarm TEST_443_STKY_SF
    insert-http X-Forwarded-Proto header-value "https"
    insert-http X-Forwarded-For header-value "%is"
policy-map multi-match SLB-HTTP-POLICY
class TEST_L4VIP_HTTPS
    loadbalance vip inservice
    loadbalance policy TEST_L7PLB_HTTPS
    loadbalance vip icmp-reply active
    loadbalance vip advertise active
    nat dynamic 1 vlan 202
    appl-parameter http advanced-options PERSIST
    ssl-proxy server TEST_SSL_PROXY_SERVER
PS : ACE uptime is 291days, could that impact ACE behavior ?
Thanks for any troubleshooting hints

Looking at this on my phone but it looks like you L7 policy is referencing a sticky server farm that does not exist.
ie TEST_443_STKY_SF is incorrect name for sticky
If that's not it. Then check that the first server actually has a number of conns on it when a new connection is established. Sometimes when both servers have 0 conns - new incoming conns will always go to the first server
Regards
Stephen
===============================
Free network configuration management software at www.rconfig.com
Sent from Cisco Technical Support iPhone App

ACE Module

Basically we have a running ACE context which works however we are using natting and we have some applications complaining that they can't see the source address of things. So I created a whole new context with the following config but I have the problem of when the client is on the server side network the traffic never makes it there.
ACE1/10.0.0.0_Network# sho run
Generating configuration....
access-list ALL line 8 extended permit ip any any
rserver host CE-565-1
ip address 10.0.2.83
inservice
serverfarm host Content_Engine_SF
rserver CE-565-1
inservice
class-map match-all Content_Engine_VIP
2 match virtual-address 10.0.18.101 any
class-map type management match-any Remote_Management
2 match protocol http any
3 match protocol icmp any
4 match protocol telnet any
5 match protocol ssh any
policy-map type management first-match rmt_mgt_policy
class Remote_Management
permit
policy-map type loadbalance first-match Content_Engine_VIP-l7slb
class class-default
serverfarm Content_Engine_SF
policy-map multi-match int18
class Content_Engine_VIP
loadbalance vip inservice
loadbalance policy Content_Engine_VIP-l7slb
loadbalance vip icmp-reply active
access-group input ALL
interface vlan 3
description Server_Side
ip address 10.0.3.240 255.255.254.0
mac-sticky enable
no shutdown
interface vlan 18
description Client Side Network
ip address 10.0.18.251 255.255.255.0
mac-sticky enable
service-policy input int18
no shutdown
ip route 0.0.0.0 0.0.0.0 10.0.18.1
if I telnet to the vip from my machine 172.16.6.222 it works fine. If I telnet from 10.0.18.30 it works fine. However when I telnet from a machine on the vlan 3 10.0.2.188 it does not work. I would have thought the mac-sticky option would work but it seems to be doing nothing. Any ideas with out using a NAT pool would be great so we can see the originating IP Address.

If you are initiating traffic from serverA to a vip that load balances to serverB in that same vlan you will have an asymmetric flow. ServerA is on the same vlan as serverB. Since both servers are in the same subnet, ServerB will ARP for serverA address and send the response directly to serverA. The traffic will never make it back to the ACE. There are a few things you can do:
1. Use NAT to ensure the return traffice makes it back to ACE.
2. Insert HTTP header with client IP address. This only works for HTTP traffic and your application must be able to recognize this header for logging.
http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A1/configuration/slb/guide/classlb.html#wp1040008
3. Use Direct Server Return (DSR). This feature has been committed to ACE 2.0. This will require the servers to be L2 adjacent to the ACE module and you will need to configure the VIP address as a loopback address on the server. Here is CSM documentation that lists some of the limitations with DSR:
http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/csm/4.2.x/configuration/guide/netwcsm.html#wp1065827

ACE Module Cookie Parsing causes Reset Connection

I am trying to upgrade my ACE Modules from A2(1.3) to A2(3.2) . Unfortunately, the cookie parsing breaks when there are illegal characters and causes a connection reset (RST) when there is an invalid cookie, but only on code later than A2(1.3).
The cookie in question is being passed by a third party so making them change the cookie is not necessarily do-able. The cookie has the following value:
Cookie: CurrentUser={"UserKey":{"Key":"anonymous"},"LastUpdated":"10/13/2010 1:35:52 PM"}
We are using the following parameter map:
parameter-map type http CASE_PARAM
case-insensitive
persistence-rebalance
set header-maxparse-length 20480
length-exceed continue
On the older code, the request is passed on to the server.
Is there a setting similar to "length-exceed continue" that I can give the ACE to tell it to ignore cookies it cannot parse?

HTTP inspection is not enabled.
Did you mean adding a class-default to the policy-map?
Adding it to the policy-map does make it match the class-default. Unfortunately, cookie parse errors result in the inability to parse both the cookie and the host header as well. It seems that rather than just failing to parse the cookie and being unable to do sticky matching - it completely fails the entire header parsing.
Here's our setup:
rserver host test1
ip address 192.168.1.101
inservice
rserver host test2
   ip address 192.168.1.102
   inservice
rserver host test3
   ip address 192.168.1.103
   inservice
rserver host test4
   ip address 192.168.1.104
   inservice
serverfarm host auto
probe HTTP-diagnostic
rserver test1
    inservice
rserver test2
    inservice
serverfarm host news
probe HTTP-diagnostic
rserver test3
    inservice
rserver test4
    inservice
sticky http-cookie autoCookie auto-cookie
cookie insert browser-expire
replicate sticky
serverfarm auto
sticky http-cookie newsCookie news-cookie
cookie insert browser-expire
replicate sticky
serverfarm news
class-map type http loadbalance match-any auto
2 match http header Host header-value "www.auto.local"
3 match http header Host header-value "auto.local"
class-map type http loadbalance match-any news
   2 match http header Host header-value "www.news.local"
   3 match http header Host header-value "news.local"
class-map match-all prod_VIP
2 match virtual-address XXX.XXX.XXX.XXX tcp eq www
policy-map type loadbalance first-match prod_POLICY
class auto
    sticky-serverfarm auto-cookie
class news
    sticky-serverfarm news-cookie
class class-default
    sticky-serverfarm auto-cookie
policy-map multi-match aggregate-slb-apps
class prod_VIP
    loadbalance vip inservice
    loadbalance policy prod_POLICY
    loadbalance vip icmp-reply active
    loadbalance vip advertise
    appl-parameter http advanced-options CASE_PARAM

Want to know about ACE module in 6509 : load-balancing concept

Hi,
I am quite new in this field , where i need to configure and understand the concept of load-balancing through ACE.
In my existing network set-up , i have some application servers as well as some other servers where i am looking for load-balancing.
I have gone through some of the site and cisco site as well and i came across ACE module which can be installed in 6509 switch.
I have 6509 switch as well but before going for installing the ACE module I am keen to understand below things:
1) what is difference between CSM or any other product load-balancer and ACE module :
Gone through site as well , but not getting proper answer or comparison.
1) I have some of the server configured with clustering and getting one virtual IP, In this case , will ACE work ?
2) If suppose i go for configuring different IP address with all server IP :
How do i achieve it ?
3) what is Virtual IP concept in ACE because i do not have and other ACE module then why do i need virtual IP ?
4) will the load-balancing happens based on destination based or session based ?
Please share the knowledge. It would be great help for me to go ahead with ACE and configure it and understand all the application ?

Hello,
1) what is difference between CSM or any other product load-balancer and ACE module :
There are several differences but to say simply, you get higher performance and more features with ACE module/appliance comparing others.
One big difference is that with ACE seriese, you can configure multiple contexts on one box (virtual load-balancers on one box) that makes us possible to provide a virtual load-balancer to a customer. In that way, the customer can access and makes changes on only the virtual box. You can split management domain for each customers. Also using contexts, you can assign certain resources available on the hardware for each contexts according to their service contract.
ACE serise has specific hardware chip for supporting SSL termination but some others do not.
For instance, you need a CSM-S, or a CSM and a SSL module to terminate SSL.
The other thing I should mention is that our most recent product is ACE serise that means it has longer product roadmap.
Let me try clarifying your other questions.
3) what is Virtual IP concept in ACE because i do not have and other ACE module then why do i need virtual IP ?
4) will the load-balancing happens based on destination based or session based ?
I think I'd better to put 3) and 4) first.
Virtual ip address (VIP) is the address to which client accesses.
VIP is tied with a serverfarm or serverfarms, in a serverfarm one or multiple rservers can be configured.
"serverfarm" is a group of "rservers".
"rserver" means real-server that has an ip address and processes transactions.
When a client accesses to the VIP, ACE picks up a rserver according to algorithm.
If you configure a VIP that is tied with a serverfarm where only one rsever is configured, client accesses to the virtual ip address are
all forwarded to the rserver.
If you configure a VIP that is tied with a serverfarm where multiple rsevers are configured, client accesses to the virtual ip address are
balanced among those rservers.
If you configure multiple VIPs, client accesses to those VIPs are forwareded to corresponding rservers according to configuration.
1) I have some of the server configured with clustering and getting one virtual IP, In this case , will ACE work ?
ACE load-balances connections to configured rservers.
If the clustered servers are sharing one virtual ip address and you configure the virtual ip address as a rserver, all connections are
sent to the virtual ip address. That is not "load-balancing" on ACE... You need multiple rservers to which ACE load-balances connections.
2) If suppose i go for configuring different IP address with all server IP :
How do i achieve it ?
You can configure those ip addresses as rserver ip address.
Multiple rservers are tied into a group, "serverfarm".
I'm not certain about your culstered servers but I guess you can configure each ip addresses in the culster as rservers.
Then put those rservers in a serverfarm.Client accesses to a virtual ip address configured on ACE for the serverfarm.
This way connections are load-balanced among those rservers depending on load-balancing algorithm you choose.
Above is just an overveiw. ACE gives you granular control not mentioned above.
I can provide more specific information if you tell me details of what you are trying to archive with ACE.
Regards,
Kimihito.

How to Virtual IP configuration in ACE module?

Hi,
I am in the process of configuring load balancing on ACE module but struggling to configure virtual IP address for ACE module.
I'm working on ACE30 module and using software version A5 (1.2). ACE module is in slot of Catalyst 6504 switch.
Can anybody please post the steps/commands to perform this activity? An early response would be appreciated.
Regards,
Rachit.

Hi Rachit,
Here is a basic configuration example:
access-list Allow_Access line 10 extended permit ip any any
rserver host test
ip address 10.198.16.98
inservice
rserver host test2
ip address 10.198.16.93
inservice
serverfarm host test
rserver test 80
    inservice
rserver test2 80
    inservice
sticky http-cookie test group2
cookie insert
serverfarm test
class-map match-all VIP
2 match virtual-address 10.198.16.122 tcp eq www
policy-map type loadbalance first-match test
class class-default
    sticky-serverfarm group1
policy-map multi-match clients
class VIP
    loadbalance vip inservice
    loadbalance policy test
    loadbalance vip icmp-reply active
    nat dynamic 1 vlan 112
interface vlan 112
ip address 10.198.16.91 255.255.255.192
access-group input Allow_Access
nat-pool 1 10.198.16.122 10.198.16.122 netmask 255.255.255.192 pat
service-policy input NSS_MGMT
service-policy input clients
no shutdown
ip route 0.0.0.0 0.0.0.0 10.198.16.65
Here is the configuration guide:
http://tools.cisco.com/squish/101AD
Cesar R

Load Balancing on ACE Modules

hi,
Is it possible to load balance VIP hits on two ACE Modules in an active/active configuration. Or is it that only per FT group only single context could be active.
Regards.

You can have 1 context active on one ACE and the other context active on the other ACE.
If you have 2 Vip, you can have 1 vip belonging to one context and the other vip belonging to the other context.
Like this, you split the traffic between the 2 devices which allows you to handle more traffic than what 1 device could normally do.
If one device can handle all your traffic, I prefer to only have 1 active unit and 1 standby.
Easier to implement and troubleshoot.
Gilles.

Cisco ACE loadbalancing matching more than one header in L7 class map

Dear All,
This is regarding Cisco ACE loadbalancing matching more than one header in L7 class map. I have a small setup with ACE 30 module in Cisco6500. I have got three webservers. Presently I have following configuration where I am mathing one url header.
class-map type http loadbalance match-all L7_WEB_HEADER_MATCH
description MATCH THE HOST HEADER OF HTTP REQUEST
2 match http header Host header-value ".*abhisar.com*"
So for above configuration, when traffic is coming for abhisar.com, it is working fine.
Now, I have following headers and DNS entry is pointing to same virtual IP for all http url header same as abhisar.com
abhisarindia.com
indiaabhi.com
So new configuration will be
class-map type http loadbalance match-any L7_WEB_HEADER_MATCH
description MATCH THE HOST HEADER OF HTTP REQUEST
2 match http header Host header-value ".*abhisar.com*"
4 match http header Host header-value ".*abhisarindia.com*"
6 match http header Host header-value ".*indiaabhi.com*"
So just want to confirm if this is fine.
Thank You,
Abhisar.

Dear Rajesh,
Thank you for reply. I will let you know once I carry out this activity.
Thank You,
Abhisar.

Reuse of context in ACE module

Hi all, just have a question about som reuse of resources in a ACE module context. I don't want to make a new context, and can reuse most of the existing configuration in one of my context. The config is not complex and difficult, but I'm not sure if I can do this.
The primary goal is to loadbalance 2 webservers with a new vip, new serverfarm, stickygroup, policy-map and different nat-pool.
Since I haven't decided the ip addresses to be used, they are just xx in the config below.
The changes I want to implement are in bold. Will this work for me?
probe http WEBGUI_D2
description Probe for http mot webgui
interval 10
passdetect interval 10
passdetect count 1
request method get url /D2/auth/login.aspx
expect status 200 302
header User-Agent header-value "IDENTITY"
rserver host cwi003
description content server logon
ip address 10.163.22.27
inservice
rserver host cwi004
description content server logon
ip address 10.163.22.28
inservice
rserver host cwi503
description content server logon 2
ip address 10.163.22.23
inservice
rserver host cwi504
description content server logon 2
ip address 10.163.22.24
inservice
serverfarm host SF_LOGON_D2
probe WEBGUI_D2
rserver cwi003 80
   inservice
rserver cwi004 80
   inservice
serverfarm host SF_LOGON2_D2
probe WEBGUI_D2
rserver cwi503 80
   inservice
rserver cwi504 80
   inservice
sticky ip-netmask 255.255.255.255 address source STICKYGROUP1
timeout 20
replicate sticky
serverfarm SF_LOGON_D2
serverfarm SF_LOGON2_D2
class-map match-all VS_LOGON_D2
3 match virtual-address 10.163.22.13 any
class-map match-all VS_LOGON2_D2
3 match virtual-address 10.163.22.xx any
policy-map type loadbalance first-match PM_ONE_ARM_LB
class class-default
   sticky-serverfarm STICKYGROUP1
policy-map multi-match PM_ONE_ARM_MULTI_MATCH
class VS_LOGON_D2
   loadbalance vip inservice
   loadbalance policy PM_ONE_ARM_LB
   nat dynamic 5 vlan 1240
class VS_LOGON2_D2
   loadbalance vip inservice
   loadbalance policy PM_ONE_ARM_LB
   nat dynamic 6 vlan 1240
interface vlan 1240
description Client_server
ip address 10.163.22.11 255.255.255.0
peer ip address 10.163.22.12 255.255.255.0
access-group input INBOUND
nat-pool 5 10.163.22.14 10.163.22.17 netmask 255.255.255.192 pat
nat-pool 6 10.163.22.xx 10.163.22.xx netmask 255.255.255.192 pat
service-policy input PM_ONE_ARM_MULTI_MATCH
no shutdown
ip route 0.0.0.0 0.0.0.0 10.163.22.1
BR
Geir

Thanks for your reply.
Hope I understand you correct. This sould be the config I need to paste into the existing context.
rserver host cwi503
description content server logon 2
ip address 10.163.22.23
inservice
rserver host cwi504
description content server logon 2
ip address 10.163.22.24
inservice
serverfarm host SF_LOGON2_D2
probe WEBGUI_D2
rserver cwi503 80
    inservice
rserver cwi504 80
    inservice
sticky ip-netmask 255.255.255.255 address source STICKYGROUP2
   timeout 20
   replicate sticky
   serverfarm SF_LOGON2_D2
class-map match-all VS_LOGON2_D2
   3 match virtual-address 10.163.22.xx any
policy-map type loadbalance first-match PM_ONE_ARM_LB2
class class-default
    sticky-serverfarm STICKYGROUP2
policy-map multi-match PM_ONE_ARM_MULTI_MATCH
class VS_LOGON2_D2
    loadbalance vip inservice
    loadbalance policy PM_ONE_ARM_LB2
    nat dynamic 6 vlan 1240
interface vlan 1240
nat-pool 6 10.163.22.xx 10.163.22.xx netmask 255.255.255.192 pat
Br
Geir

ACE module - Qos - set ip tos #

All,
Trying to mark traffic to/from L4 rules in the ACE.
Documentation (like always) says it's really easy. Mark traffic by using the "set ip tos <value>" command in Policy/Class configuration. Ok, so I do this, set ip tos 24.
Enable qos globally on the 6500 host, but don't see the traffic being marked.
sh mls qos says that packets are being modified by module 5 (ACE)
But I never see the tos value in any of my captures either via netflow from the host 6500, or at the firewall one hop away.
sh mls qos:
QoS is enabled globally
Policy marking depends on port_trust
QoS ip packet dscp rewrite enabled globally
Input mode for GRE Tunnel is Pipe mode
Input mode for MPLS is Pipe mode
QoS Trust state is CoS on the following interface:
Te3/1
QoS Trust state is DSCP on the following interface:
Gi2/3
Vlan or Portchannel(Multi-Earl) policies supported: Yes
Egress policies supported: Yes
----- Module [5] -----
QoS global counters:
    Total packets: 207147888661
    IP shortcut packets: 0
    Packets dropped by policing: 0
    IP packets with TOS changed by policing: 2663386
    IP packets with COS changed by policing: 4889352
    Non-IP packets with COS changed by policing: 0
    MPLS packets with EXP changed by policing: 0
Can someone explain to me what I've got wrong here? Is the ACE simply marking traffic destined for the servers behind it and not the return traffic? Am I missunderstanding something?

Well... hopefully someone knows how to classify traffic coming from the ACE.
I've given up on using the ACE to mark traffic as I'm fairly certain it won't do it. At least not the way I want.
However, now I've taken to marking ingress on the rserver switch ports... which has resulted in a partially sucessful solution. Problem is, "partially" successful.
You'll have a bunch of little conversations like this with no tos value full of push-acks:
10:29:53.527526 207.161.222.68.2828 > 205.200.114.228.http: P 2954:3455(501) ack 203152 win 65535 (DF)
10:29:53.527698 205.200.114.228.http > 207.161.222.68.2828: . ack 3455 win 32267
10:29:53.555271 207.161.222.68.2828 > 205.200.114.228.http: P 3455:3686(231) ack 203152 win 65535 (DF)
10:29:53.562676 205.200.114.228.http > 207.161.222.68.2828: P 203152:203784(632) ack 3686 win 32768
10:29:53.674758 207.161.222.68.2828 > 205.200.114.228.http: P 3686:4036(350) ack 203784 win 64903 (DF)
10:29:53.690853 205.200.114.228.http > 207.161.222.68.2828: P 203784:205244(1460) ack 4036 win 32768
10:29:53.690863 205.200.114.228.http > 207.161.222.68.2828: P 205244:206704(1460) ack 4036 win 32768
10:29:53.690871 205.200.114.228.http > 207.161.222.68.2828: P 206704:208164(1460) ack 4036 win 32768
10:29:53.690879 205.200.114.228.http > 207.161.222.68.2828: P 208164:209624(1460) ack 4036 win 32768
10:29:53.690887 205.200.114.228.http > 207.161.222.68.2828: P 209624:211084(1460) ack 4036 win 32768
10:29:53.690895 205.200.114.228.http > 207.161.222.68.2828: P 211084:212544(1460) ack 4036 win 32768
But then you'll see another conversation pop up with the correct markings
10:31:53.845287 205.200.114.228.http > 207.161.222.68.2828: . 32753:34213(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845298 205.200.114.228.http > 207.161.222.68.2828: . 34213:35673(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845306 205.200.114.228.http > 207.161.222.68.2828: . 35673:37133(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845313 205.200.114.228.http > 207.161.222.68.2828: . 37133:38593(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845321 205.200.114.228.http > 207.161.222.68.2828: . 38593:40053(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845328 205.200.114.228.http > 207.161.222.68.2828: . 40053:41513(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845335 205.200.114.228.http > 207.161.222.68.2828: . 41513:42973(1460) ack 1082 win 62808 (DF) [tos 0x48]
10:31:53.845343 205.200.114.228.http > 207.161.222.68.2828: . 42973:44433(1460) ack 1082 win 62808 (DF) [tos 0x48]
I think what's happening, is that the conversations full of the P-acks is the load balancer communicating directly with the client (i.e. LB pretending to be the server), whereas the marked traffic is "data only" which the load balancer isn't mangling (like it might/probably is doing with the p-acks) on it's way back to the client.
I also can't modify the configuration of the "virtual ten gig" interface that the 6500 uses as a connection to the ACE module, so can't mark traffic there either. And though I still have a couple of things to try, I don't believe I can do egress marking on a trunk from the 6500 either (connection to the firewalls).
So.... PLEASE... Anyone??? Ideas???

Ace module dropping assymetric layer 2 connections

Hi we had a situation in where the ACE would randomly drop certain tcp connections, and all ICMP packets from a certain windows server. The server in question was using Transmit Load Balancing with Fault Tolerance.
The server has one Nic connected to Access switch1, and the other nic connected to Access switch2. Each access switch connects up to a pair of 6509's, which is active on Core1 on both switches.
I am guessing If the server sends on Nic 2, core1 knows it came in on the downstream trunk port to Switch2, it must reply to these packets based on the teamed mac of the layer 3 address(no idea who is arping for the destination - the ace?), and send them back out the downstream trunk port to switch1. The ace module is in transparent mode. When contacting a server on the other side of the ace, the ace drop packets that came from the second nic - and I am wondering how it "knows" that the return path is out of different downstream port. Does it share some kind of layer 2 RPF check with the 6500 ?
Please note there is no routing involved here. The destination server is just on another vlan on the same subnet, on the other side of the ace.

Bryan,
As long as the server replies back to the ACE the client should only be commmunicating with the VIP address in either of your two examples.
In your first example the flow will look like this.
client > VIP after the ACE client > rserver
the reply would be
rserver > client after the ACE VIP > rserver
In your second example using client nat it will look like this
Client > VIP After ACE Natpool > rserver.
the reply would be
rserver > Nat-pool after ACE VIP > client.
The ACE by default will always nat the vip to the server ip unless you use the command "transparent" under the serverfarm. When using this command we send the packet to the MAC address of the server leaving the destination IP of the VIP. The server would need to have the VIP address configured under the loopback interface.
Regards
Jim

Certificates vanished - ACE Module. Strange!

ACE modules are configured in Active/Standby context mode on two distinct Cat6500's. The feature license is 10,000 SSL tps, 8Gbps throughput.
We ran the application performance tests with 1000 users with https transactions and I noticed that the all the root certificates under the chaingroup disappeared. Only the website certificate remained. When I accessed the website, it gave 'error with the security certificate' i.e. the root was not identifiable due to missing certificates. Eventually, the CPU went 100% on Cat6500 and the ACE module was shutdown by the chassis. It got reenabled automatically in 5 minutes.
I re-added the root certs, removed/added the service policy and after sometime I noticed the root certs disappeared again. STRANGE !
show version output is
Cisco Application Control Software (ACSW)
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2006, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.
Software
loader: Version 12.2[121]
system: Version 3.0(0)A1(6.3a) [build 3.0(0)A1(6.3a) adbuild_02:16:25-2008/02/02_/auto/adbu-rel3/ws/rel_3_0_0_a1_6.3-thr
ottle/REL_3_0_0_A]
system image file: [LCP] disk0:c6ace-t1k9-mz.3.0.0_A1_6_3a.bin
installed license: ACE-08G-LIC ACE-VIRT-020 ACE-SSL-10K-K9
Hardware
Cisco ACE (slot: 2)
cpu info:
number of cpu(s): 2
cpu type: SiByte
cpu: 0, model: SiByte SB1 V0.2, speed: 700 MHz
cpu: 1, model: SiByte SB1 V0.2, speed: 700 MHz
memory info:
total: 957640 kB, free: 347924 kB
shared: 0 kB, buffers: 1588 kB, cached 0 kB
cf info:
filesystem: /dev/cf
total: 1014624 kB, used: 360960 kB, available: 653664 kB
last boot reason: NP 0 Failed : NP ME Hung
configuration register: 0x1
Could you please advise whether there is any bug in the above software version i.e. it removes the root certs due to heavy transaction load.
Thanks.

I wanted to look for more details regarding this bug id. But I got the below message in Bug Toolkit. Please advise...
CSCsl96203 Bug Details
Information contained within bug ID CSCsl96203 is only available to Cisco employees. It is our policy to make all externally-facing bugs available in Bug Toolkit so the system administrators have been automatically alerted to the problem. By choosing to save this bug, you may be notified when the decision to make this bug available to you has been made. Note: Some product enhancement requests and documentation error bugs may not be available in Bug Toolkit.

ANM Server & ACE Module

We have a Custom built tool to manage our existing CSS boxes wherein we shutdown multiple Services at one single instance without affecting the entire VIP. The reason to do that is because of the following scenario. 10 Servers. Each server has multiple interfaces configured to support multiple websites thru IIS. Out of the 10 Web servers if we plan to remove one server for code upgrade/deployment, we take that server and shutdown all its configured services from the CSS using our tool.
I understand in ACE, the control is not at the Rserver level, but at the Server farm wherein you have all you servers configured for multiple ports. If I want to take a server (which is configured for multiple websites and multiple ports), I have to navigate to each server farm in the GUI (ANM) and then select the rserver one at a time..U know, it is time consuming..
Since ANM uses mysql to store the data collected from ACE module, is there a way we can create custom tools to achieve our requirement. If possible, Could you please provide us more information on the ACE/ANM Interaction and the options to customize ANM features?
I checked and found from CISCO Site that ANM 1.2 is the latest and only available Software package to Manage ACE in GUI environment. Do you have any other recommendations are products?

you can do it from the CLI.
Each rserver is defined with just an ip address and you define the port when using the rserver in a serverfarm.
By de-activating the rserver in global, you de-activate it in all serverfarms it is being used.
This does not seem possible with ANM so.
If you don't like CLI, you could use XML commands.
Gilles.

ACE module only inserting X-Forwarder Header on first packet

Similar Messages

Maybe you are looking for