Application Slowness through ACE 4710
We are struggling with an issue where a user can run a report directly from the server but not from ACE loadbalanced url.
Report run via Individual Web server URL’s
The report takes less than 20 minutes (average 15 minutes) to fetch and return the data. This is observed 9 out of 10 times.
Report run via ACE Load Balanced URL
The report keeps on running for more than 20 minutes and never completes. The front end keeps showing report is running.
The data in general when tested directly by running queries against the database (bypassing the platform) completes in 15-18 minutes
The network connectivity for each and every ports involved (Loadbalancer/Servers) have been throulgly checked.
Please advise what is the best way to troubleshoot this issue.
Thx
Ahmad
Hello Jorge,
Here is my response below.
What version are you using? Version A4(2.3a)
Can you upload these outputs?
# show service-policy int63 class-map appbo detail
Status : ACTIVE
Description: -----------------------------------------
Interface: vlan 1 62 63 155 409
service-policy: int63
class: appbo
VIP Address: Protocol: Port:
170.116.253.245 tcp eq 80
loadbalance:
L7 loadbalance policy: appbo-l7slb
VIP ICMP Reply : ENABLED-WHEN-PRIMARY-SF-UP
VIP State: INSERVICE
VIP DWS state: DWS_DISABLED
Persistence Rebalance: DISABLED
curr conns : 0 , hit count : 136348
dropped conns : 21
conns per second : 0
client pkt count : 4579400 , client byte count: 1054651106
server pkt count : 6006908 , server byte count: 7506886155
conn-rate-limit : - , drop-count : -
bandwidth-rate-limit : - , drop-count : -
L7 Loadbalance policy : appbo-l7slb
class/match : class-default
LB action: :
sticky group: appbo
primary serverfarm: appbo
state:UP
backup serverfarm : -
hit count : 136348
dropped conns : 12
compression : off
compression:
bytes_in : 0 bytes_out : 0
Compression ratio : 0.00%
Gzip: 0 Deflate: 0
compression errors:
User-Agent : 0 Accept-Encoding : 0
Content size: 0 Content type : 0
Not HTTP 1.1: 0 HTTP response error: 0
Others : 0
Parameter-map(s):
3600-seconds
# show stats http
+------------------------------------------+
+-------------- HTTP statistics -----------+
+------------------------------------------+
LB parse result msgs sent : 455403292 , TCP data msgs sent : 2434371041
Inspect parse result msgs : 0 , SSL data msgs sent : 1740205587
sent
TCP fin msgs sent : 40338385 , TCP rst msgs sent: : 22697825
Bounced fin msgs sent : 3083341 , Bounced rst msgs sent: : 2062455
SSL fin msgs sent : 53021042 , SSL rst msgs sent: : 89469
Drain msgs sent : 260995432 , Particles read : 749867347
Reuse msgs sent : 0 , HTTP requests : 413618855
Reproxied requests : 232446464 , Headers removed : 0
Headers inserted : 105893583 , HTTP redirects : 1485493
HTTP chunks : 144637122 , Pipelined requests : 42911599
HTTP unproxy conns : 305776225 , Pipeline flushes : 942
Whitespace appends : 720 , Second pass parsing : 0
Response entries recycled : 42882784 , Analysis errors : 0
Header insert errors : 49 , Max parselen errors : 35941
Static parse errors : 261401 , Resource errors : 0
Invalid path errors : 0 , Bad HTTP version errors : 117
Headers rewritten : 49 , Header rewrite errors : 0
SSL headers inserted : 0 , SSL header insert errors : 0
SSL spoof headers deleted : 0 , Unproxy msgs sent : 305776279
HTTP passthrough stat : 0
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Has this ever worked before? YES but the issue is Intermittent
Why do you have a low timeout for your sticky configuration? NOT SURE. WILL HAVE TO CHECK WITH APP OWNERS IF WE CAN INCREASE THAT
If you clear the cookies of the browser and turn off some of the servers to test only with one, do you have the same behavior? Will try to perform this test with app owners
In your tests, are you trying to the same type of query to your database? I mean do you see the problem if you try to update/delete/select in the database? No Refresh is pretty quick.
What are the servers? Oracle, SQL server, MS Access? ORAGLE 11G.
Thanks for your help!
Similar Messages
-
Access Server through VIP (ACE 4710) but very slow
Re: Access Server through VIP (ACE 4710) but very slow
Hi Shiva
Kindly Help .....Accessing the server very slow.., Plz check my real configuration... this configuration is for application server and after this i have to configure more serverfarm for different server like webmail etc. in this ACE 4710. I have only one ACE 4710 .
ACE Version A4(2.0) = is there supports Probe with this version.??? without probe server will work but very slow. And plz guide Nat-pool is required
VIP :-- 172.16.15.8
LB/Admin# sh run
Generating configuration....
no ft auto-sync startup-config
logging enable
logging host 172.29.91.112 udp/514
resource-class RC1
limit-resource all minimum 10.00 maximum unlimited
boot system image:c4710ace-mz.A4_2_0.bin
hostname LB
interface gigabitEthernet 1/1
description Management
speed 1000M
switchport access vlan 1000
no shutdown
interface gigabitEthernet 1/2
description clientside
switchport access vlan 30
no shutdown
interface gigabitEthernet 1/3
description serverside
switchport access vlan 31
no shutdown
interface gigabitEthernet 1/4
no shutdown
context Admin
description Management
member RC1
access-list everyone line 8 extended permit ip any any
access-list everyone line 16 extended permit icmp any any
probe http probe1
description health check
interval 5
passdetect interval 10
request method head
expect status 200 200
open 1
rserver redirect https_redirect
description redirect traffic to https
webhost-redirection / 302
inservice
rserver redirect maintenance_page
description maintenance page displayed
webhost-redirection /sry.html 301
inservice
rserver host web1
ip address 192.168.10.3
inservice
rserver host web2
ip address 192.168.10.4
inservice
rserver host web3
ip address 192.168.10.5
inservice
serverfarm host http
rserver web1
inservice
rserver web2
inservice
rserver web3
inservice
serverfarm redirect https_redirect_farm
description Redirect traffic to https
serverfarm redirect maintenance_farm
description send user to maintenance page
parameter-map type connection paramap_http
description parameter connection tcp
exceed-mss allow
sticky ip-netmask 255.255.255.0 address source Sticky_http
timeout activeconns
serverfarm http
class-map match-all REMOTE-ACCESS
class-map type management match-any remote_access
2 match protocol xml-https any
3 match protocol icmp any
4 match protocol telnet any
5 match protocol ssh any
6 match protocol http any
7 match protocol https any
8 match protocol snmp any
class-map match-all slb-vip
2 match virtual-address 172.16.15.8 tcp eq www
policy-map type management first-match remote_access
class class-default
permit
policy-map type management first-match remote_mgmt_allow_policy
class remote_access
permit
policy-map type loadbalance first-match slb
class class-default
serverfarm http
policy-map type inspect http all-match slb-vip-http
class class-default
permit
policy-map multi-match client-vips
class slb-vip
loadbalance vip inservice
loadbalance policy slb
loadbalance vip icmp-reply active
inspect http policy slb-vip-http
connection advanced-options paramap_http
interface vlan 30
description "Client Side"
ip address 172.16.15.24 255.255.255.0
access-group input everyone
service-policy input client-vips
no shutdown
interface vlan 31
description "Server Side"
ip address 192.168.10.1 255.255.255.0
service-policy input remote_access
no shutdown
interface vlan 1000
description managment
ip address 172.29.91.110 255.255.255.0
service-policy input remote_mgmt_allow_policy
no shutdown
ip route 0.0.0.0 0.0.0.0 172.16.15.1
snmp-server contact "PHQ"
snmp-server community phq group Network-Monitor
snmp-server trap-source vlan 1000
username admin password 5 $1$b2txbc5U$TA74D920oSdd2eOZ4hSFe/ role Admin domain
default-domain
username www password 5 $1$.GuWwQEK$r8Ub4OcE3l190d5GA4kvR. role Admin domain de
fault-domain
username prem password 5 $1$8C7eRKrI$it3UV4URZ26X4S/Bh6OEr0 role Admin domain d
efault-domain
ssh key rsa 1024 force
banner motd # "ro" #
Regards,
PremHi Shiva,
plz guide i'm new with ACE LB, also find my n/w design for connected ace to server. but server accessing very very slow, but when i connect through my old server software LB (with two interface)then accessing very fast. I just replace my old serverLB(with two interface) to ACE4710 and connect the same scenario then why not server accessing smoothly with VIP .Reply soon only I connect ACE's two interface with switch.....
Regards,
Prem -
ACE 4710 responds very slow to CLI commands
I am expericing delayed responses to my CLI commands on the ACE 4710. The delays occur sporadically. I have check the cpu and memory and neither one appeared to show any abnormal behaviour. Has anybody else experienced unsual delay with your CLI commands? If so, where you able to isolate and correct the problem? If not, any suggestions on where and how to look for the problem?
I am experiencing the same exact problem. CLI commands are very slow. Although, i dont get any performance issues for my application when i issue the "show run" or "wr mem" commands at CLI i wait for over 1 minute to receive any output, commands like "show serverfarm" ,"show rserver" "show stats" are working fine. my resource usage is ok and cpu has no problems. The problem exists in all contexts of the specific ACE. I did a test by forcing the standby ace to become active and while the standby ACE had no problem in executing the command when it becomes the active one the problem shows up. is there a way to troubleshoot this?
Thank you in advance -
Need help to Configure Cisco ACE 4710 Cluster Deployment
Dear Experts,
I'm newbie for Cisco ACE 4710, and still I'm in learning stage. Meanwhile I got chance at my work place to deploy a Cisco ACE 4710 cluster which should load balance the traffic between two Application Servers based on HTTP and HTTPS traffic. So I was looking for good deployment guide in Cisco SBA knowledge base then finall found this guide.
http://www.cisco.com/en/US/docs/solutions/SBA/February2013/Cisco_SBA_DC_AdvancedServer-LoadBalancingDeploymentGuide-Feb2013.pdf
This guide totally fine with my required deployment model. I have same deployment environment as this guide contains with ACE cluster that connects to two Cisco 3750X (Stack) switches. But I have some confusion places in this guide
This guide follow the "One-armed mode" as a deployment method. But when I go through it further I have noticed that they have configured server VLAN as a 10.4.49.0/24 (all servers reside in it) and Client side VIP also in same VLAN which is 10.4.49.100/24 (even NAT pool also).
My confusion is, as I have learned about Cisco ACE 4710 one-armed mode deployment method, it should has two VLAN segments, one for Client side which client request come and hit the VIP and then second one for Server side. which means besically two VLANs. So please be kind enough to go through above document then tell me where is wrong, what shoud I need to do for the best. Please this is an urgent, so need your help quickly.
Thanks....!
-Amal-Dear Kanwal,
I need quick help for you. Following are the Application LB requirements which I received from my clinet side.
Following detail required for configuring Oracle EBS Apps tier on HA:
LBR IP and Name required to configure EBS APPS Tier (i.e, ap1ebs & ap2ebs nodes)
Suggested IP and Name for LBR:
IP : 172.25.45.x [should be on same 172.25.45 subnet of ap1ebs & ap2ebs nodes]
ebiz.xxxx.lk [on port 80 for http protocol accessibility]
This LBR IP & name must be resolve and respond on DNS network
Server Farm detail for LBR Setup
Following detail will be use for configuring the LBR:
LBR IP and Name :
IP : 172.25.45.x [should be on same 172.25.45 subnet of ap1ebs & ap2ebs nodes]
ebiz.xxxx.lk [on port 80 for http protocol accessibility]
This LBR IP & name must be resolve and respond on DNS network
Server Farm Detail for LBR setup:
Server 1 (EBS App1 Node, ap1ebs):
IP : 172.25.45.19
Server Name: ap1ebs.xxxx.lk [ap1ebs hostname is an example, actual hostname will be use]
Protocol: http
Port: 8000
Server 2 (EBS App2 Node, ap2ebs):
IP : 172.25.45.20
Server Name: ap2ebs.xxxx.lk [ap2ebs hostname is an example, actual hostname will be use]
Protocol: http
Port: 8000
Since my client needs to access URL ebiz.xxxx.lk which should be resolved by IP 172.25.45.21 (virtual IP) via http (80) before they deploy the app on the two servers I just ran web service on both servers (Linux) and was trying to access http://172.25.45.21 it was working fine and gave me index.html page. Now after my client has deployed the application then when he tries to access the page http://172.25.45.21 he cannot see his main login page. But still my testing web servers are there on both servers when I type http://172.25.45.21 it will get index.html page, but not my client web login page. What can I do for this ?
Following are my latest config :
probe http Get-Method
description Check to url access /OA_HTML/OAInfo.jsp
interval 10
faildetect 2
passdetect interval 30
request method get url /OA_HTML/OAInfo.jsp
expect status 200 200
probe udp http-8000-iRDMI
description IRDMI (HTTP - 8000)
port 8000
probe http http-probe
description HTTP Probes
interval 10
faildetect 2
passdetect interval 30
passdetect count 2
request method get url /index.html
expect status 200 200
probe https https-probe
description HTTPS traffic
interval 10
faildetect 2
passdetect interval 30
passdetect count 2
ssl version all
request method get url /index.html
probe icmp icmp-probe
description ICMP PROBE FOR TO CHECK ICMP SERVICE
rserver host ebsapp1
description ebsapp1.xxxx.lk
ip address 172.25.45.19
conn-limit max 4000000 min 4000000
probe icmp-probe
probe http-probe
inservice
rserver host ebsapp2
description ebsapp2.xxxx.lk
ip address 172.25.45.20
conn-limit max 4000000 min 4000000
probe icmp-probe
probe http-probe
inservice
serverfarm host ebsppsvrfarm
description ebsapp server farm
failaction purge
predictor response app-req-to-resp samples 4
probe http-probe
probe icmp-probe
inband-health check log 5 reset 500
retcode 404 404 check log 1 reset 3
rserver ebsapp1 80
conn-limit max 4000000 min 4000000
probe icmp-probe
inservice
rserver ebsapp2 80
conn-limit max 4000000 min 4000000
probe icmp-probe
inservice
sticky http-cookie jsessionid HTTP-COOKIE
cookie insert browser-expire
replicate sticky
serverfarm ebsppsvrfarm
class-map type http loadbalance match-any default-compression-exclusion-mime-type
description DM generated classmap for default LB compression exclusion mime types.
2 match http url .*gif
3 match http url .*css
4 match http url .*js
5 match http url .*class
6 match http url .*jar
7 match http url .*cab
8 match http url .*txt
9 match http url .*ps
10 match http url .*vbs
11 match http url .*xsl
12 match http url .*xml
13 match http url .*pdf
14 match http url .*swf
15 match http url .*jpg
16 match http url .*jpeg
17 match http url .*jpe
18 match http url .*png
class-map match-all ebsapp-vip
2 match virtual-address 172.25.45.21 tcp eq www
class-map type management match-any remote_access
2 match protocol xml-https any
3 match protocol icmp any
4 match protocol telnet any
5 match protocol ssh any
6 match protocol http any
7 match protocol https any
8 match protocol snmp any
policy-map type management first-match remote_mgmt_allow_policy
class remote_access
permit
policy-map type loadbalance first-match ebsapp-vip-l7slb
class default-compression-exclusion-mime-type
serverfarm ebsppsvrfarm
class class-default
compress default-method deflate
sticky-serverfarm HTTP-COOKIE
policy-map multi-match int455
class ebsapp-vip
loadbalance vip inservice
loadbalance policy ebsapp-vip-l7slb
loadbalance vip icmp-reply active
nat dynamic 1 vlan 455
interface vlan 455
ip address 172.25.45.36 255.255.255.0
peer ip address 172.25.45.35 255.255.255.0
access-group input ALL
nat-pool 1 172.25.45.22 172.25.45.22 netmask 255.255.255.0 pat
service-policy input remote_mgmt_allow_policy
service-policy input int455
no shutdown
ft interface vlan 999
ip address 10.1.1.1 255.255.255.0
peer ip address 10.1.1.2 255.255.255.0
no shutdown
ft peer 1
heartbeat interval 300
heartbeat count 10
ft-interface vlan 999
ft group 1
peer 1
no preempt
priority 110
associate-context Admin
inservice
ip route 0.0.0.0 0.0.0.0 172.25.45.1
Hope you will reply me soon
Thanks....!
-Amal- -
Hi. I'm working on the Cisco ACE 4710 to be able to load balance web Traffic between several web servers. but despite following the steps mentioned on the Cisco configuration guide (specially this link and related docs: http://docwiki.cisco.com/wiki/Cisco_ACE_4700_Series_Appliance_Quick_Start_Guide,_Release_A3(1.0)_--_Creating_a_Virtual_Context) we did not managed to make it. we tested both the "bridged scenario" and "routed scenario" but none of them is working. specifically "configuring Nat" in the above link is very confusing and is not clear; because it's not the same as Cisco IOS, which we used to implement it that way.
Routed Scenario:
==========================================
probe http Http_Probe
description Server Healty Check
port 80
request method head url /index.htm
probe icmp ICMP_Check
interval 10
passdetect interval 5
rserver host NetCad_Server_1
ip address 172.16.1.100
probe ICMP_Check
inservice
rserver host NetCad_Server_2
ip address 172.16.1.101
probe ICMP_Check
inservice
rserver host NetCad_Server_3
ip address 172.16.1.102
probe ICMP_Check
inservice
serverfarm host NetCad_Servers
probe Http_Probe
rserver NetCad_Server_1 80
inservice
rserver NetCad_Server_2 80
inservice
rserver NetCad_Server_3 80
inservice
sticky http-cookie Cookie1 1
serverfarm NetCad_Servers
class-map match-all VS_NetCad
2 match virtual-address 192.168.13.162 255.255.252.0 tcp any
policy-map type management first-match mgmt-pm
class class-default
permit
policy-map type loadbalance first-match VS_NetCad-l7slb
class class-default
serverfarm NetCad_Servers
policy-map multi-match int40
class VS_NetCad
loadbalance vip inservice
loadbalance policy VS_NetCad-l7slb
loadbalance vip icmp-reply
interface vlan 40
description Client Side
ip address 192.168.13.161 255.255.252.0
ip options allow
no normalization
no icmp-guard
access-group input Permit_ALL
service-policy input mgmt-pm
service-policy input int40
no shutdown
interface vlan 41
description Server Side
ip address 172.16.1.1 255.255.255.0
ip options allow
no normalization
no icmp-guard
access-group input Permit_ALL
nat-pool 1 172.16.1.110 172.16.1.110 netmask 255.255.255.255 pat
service-policy input mgmt-pm
no shutdown
ip route 0.0.0.0 0.0.0.0 192.168.12.1
==========================================Hi,
Let me explain you.
Assuming client IP as 1.1.1.1, VIP as 2.2.2.2 and Real Server as 3.3.3.3
Consider the simple situation where client needs to access an application hosted on 3.3.3.3. Client sends a request which comes to VIP.
src 1.1.1.1----->dst------->2.2.2.2. ACE after matching conditions and taking LB decision decides to send it to 3.3.3.3 real server. Performs destination NAT and forwards the client request to 3.3.3.3. So the above packet L3 header will now look like:
src 1.1.1.1 dst 3.3.3.3. When reply comes from server, ACE will change src 3.3.3.3 back to 2.2.2.2 and forwards the request to client 1.1.1.1. SIMPLE LB.
Now comes a situation where let's say you want to hide the client IP from server or let's say server's default GW is not ACE or client and server are in same subnet but need to communicate through VIP on ACE etc.
Src 1.1.1.1 dst 2.2.2.2
After LB ace decides to send it to 3.3.3.3 but also policy multi match has nat rule (nat dynamic 1 vlan x). But packet would be forwarded from server vlan where you have NAT pool defined. So let's say pool IP is 3.3.3.4. So ACE will perform both destination as well as src NAT here before forwarding the packet to server and packet L3 header will look like:
src 3.3.3.4 ----->dst 3.3.3.3
Now when 3.3.3.3 has to send packet back, ACE will answer ARP for 3.3.3.3 and hence packet will come back to ACE which will again change the L3 header IP's and send it out the client VLAN towards client.
So NAT is always applied to server side vlan and that's why pool is chosen from server side subnet.
Let me know if you have any questions.
Regards,
Kanwal -
Hi,
We have to ACE 4710 device in our network and we have facing device hung issue in our Primary ACE. We are not able to get management access or direct console access to the device when the issue is happened and also we are not able to reach the vlan interface IP or/VIP. Please find the below output we got through monitor that we are connected to the ACE.
Booting localboot(c4710ace-t1k9-mz.A5_1_2.bin)
kernel=(hd0,1)/c4710ace-t1k9-mz.A5_1_2.bin ro root=LABEL=/ auto console=ttyS0,9
600n8 quiet bigphysarea=32768
[Linux-bzImage,setup=0x1400,size=0xe75a16c]
Uncompressing linux Ok, booting the kernal.
Issue is resolved after we manually rebooted the ACE. We have collected the sh tech after the reboot.
Software version : A5 1.2
Kindly suggest what may cause this issue.
Thanks in Adavance.
Regards,
RanjithHi,
We have collected the console logs while we done the reboot. Please find the below output.
------------------------------------------------ Boot log -----------------------------------------------------------------------------
ÐS ÀS AMIBIOS(C)2005 American Megatrends, Inc. BIOS Date: 08/25/09 09:37:25 Ver: 08.00.11 CPU : Intel(R) Pentium(R) 4 CPU 3.40GHz Speed : 3.40 GHz Broadcom NetXtreme Ethernet Boot Agent v8.1.53 Copyright (C) 2000-2005 Broadcom Corporation All rights reserved. Press Ctrl-S to Enter Configuration Menu ... Broadcom NetXtreme Ethernet Boot Agent v8.1.53 AMIBIOS(C)2005 American Megatrends, Inc. BIOS Date: 08/25/09 09:37:25 Ver: 08.00.11 CPU : Intel(R) Pentium(R) 4 CPU 3.40GHz Speed : 3.40 GHz Press F2 to run Setup Press F12 for BBS POPUP DDR2 Frequency:667 MHz, ECC Support in Dual-Channel Interleaved Mode Initializing USB Controllers .. Done. 6144MB OK USB Device(s): 1 Keyboard Auto-Detecting Pri Slave...IDE Hard Disk Pri Slave : 1GB CompactFlash Card CF B612J GRUB Loading stage2........ GNU GRUB version 0.95.1 (639K lower / 3144640K upper memory) *************************************************************************** * localboot(ACE_APPLIANCE_RECOVERY_IMAGE.bin) * * localboot(c4710ace-t1k9-mz.A5_1_2.bin) * * localboot(c4710ace-t1k9-mz.A4_2_0.bin) * * * * * * * * * * * * * * * * * * * *************************************************************************** Use the * and * keys to select which entry is highlighted. Press enter to boot the selected OS, 'e' to edit the commands before booting, or 'c' for a command-line. The highlighted entry will be booted automatically in 1 seconds. kernel=(hd0,1)/c4710ace-t1k9-mz.A5_1_2.bin ro root=LABEL=/ auto console=ttyS0,9 600n8 quiet bigphysarea=32768 [Linux-bzImage, setup=0x1400, size=0xe75a16c] INIT: version 2.85 booting
b4 lspci
1 Cavium device(s) found.
Bringing up NP 0
Downloading U-Boot to NP card 0
Downloading DP image to NP card 0
Starting DP image on NP card on all cores
DP image started on NP card
Setting up dynamic memory size
Initializing Shared Memory
INIT: Entering runlevel: 3
Testing PCI path for Octeon(0)....
This may take some time, Please wait ....
PCI test loop , count 0
PCI path is ready
Starting services...
Waiting for 3 seconds to enter setup mode...
Certificate & key are up to date
Installing MySQL
groupadd: group nobody exists
useradd: user nobody exists
MySQL Installed
Installing JRE
JRE Installed
Starting sysmgr processes.. Please wait...Done!!!
IDC4-INTR-ACE-01 login: admin
Password:
Cisco Application Control Software (ACSW)
TAC support: http://www.cisco.com/tac
Copyright (c) 1985-2012 by Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.
We have not found any error related to flash while booting ACE.
Regards,
Ranjith -
ACE 4710 Web Optimization Licnesing
I currently have a 4710 running the 1Gbps package. We are utilizing Application Acceleration and are comg very close to hitting our 10,000 Web Optimization connection limit. I am trying to find out how to upgrade that.
I see in our license usage an option of ACE-AP-OPT-UP1-K9 but can find no information on this part number. Does anyone know if this is even available and what it brings you connection limit to?
ACE01/Admin# show license usage
License Ins Lic Status Expiry Date Comments
Count
ACE-AP-C-UP1 No - Unused -
ACE-AP-C-UP2 No - Unused -
ACE-AP-C-UP3 No - Unused -
ACE-AP-01-LIC No - Unused -
ACE-AP-01-UP1 No - Unused -
ACE-AP-02-LIC No - Unused -
ACE-AP-02-UP1 No - Unused -
ACE-AP-04-LIC No - Unused -
ACE-AP-04-UP1 No - Unused -
ACE-AP-04-UP2 No - Unused -
ACE-AP-VIRT-5 No - Unused -
ACE-AP-500M-LIC No - Unused -
ACE-AP-VIRT-020 No - Unused -
ACE-AP-C-100-LIC No - Unused -
ACE-AP-C-500-LIC Yes 1 In use never -
ACE-AP-C-500-UP1 No - Unused -
ACE-AP-OPT-50-K9 No - Unused -
ACE-AP-C-1000-LIC No - Unused -
ACE-AP-C-2000-LIC No - Unused -
ACE-AP-OPT-LIC-K9 Yes 1 In use never -
ACE-AP-OPT-UP1-K9 No - Unused -
ACE-AP-SSL-05K-K9 Yes 1 In use never -
ACE-AP-SSL-07K-K9 No - Unused -
ACE-AP-SSL-100-K9 No - Unused -
ACE-AP-SSL-UP1-K9 No - Unused -
ACE-AP-SSLUP-5K-K9 No - Unused -
ACE-AP-VIRT-020-UP No - Unused -Unfortunately, ACE-AP-OPT-LIC-K9 is not available on ACE4710 and
ACE 4710 cannot handle more than 10,000 concurrent connections..
When you use the ACE to perform a specific set of application
acceleration and optimization functions, and the ACE reaches the
maximum of 10,000 concurrent connections, the appliance stops
accepting any additional concurrent connections until the count
drops below 10,000.
http://www.cisco.com/en/US/partner/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_x/command/reference/optimize.html#wp1048813
Regards,
Yuji -
ACE 4710: Config Allows all traffic except large HTTP downloads
Hi Folks,
Got an ACE 4710 with a basic config that seems to work for all traffic except large downloads.
I've attached the current config
As I mentioned I can do normal HTTP to a standard destination like google or SSH through the ACE or ICMP
If i try to get a large file from the server side of ACE, then a trace shows that the first and subsequent 1460Byte packets dont go through ACE
I've thought of parse lengths, but i cannot see any that seem to affect the generic L4 maps that I am trying to use
Cheers
AlanI've seen a similar fault. I suppose a lower MSS was sent in the TCP SYN handshake packets (1300 or 1380?) and the packets exceeding that value were dropped by the ACE. This is the default behavior which can be switched to a less strict mode by either
exceed-mss allow
or
no normalization
commands.
In our case, a linux web server was whose replies wouldn't keep to the MSS limit. -
Using the ACE 4710 for loadbalancing a Sharepoint site.
We currently have a HTTP probe setup to check the port 80 status of the rserver.
Is there anyway to get the HTTP probe to check a DNS entry for each of the application sites? For instance http://info vs http://site are two different web sites running on the same IP. One site could have a problem but the actual port 80 for the IP may be still alive.
Thanks for any information.Has anyone figure this out? I am tring to get healthchecks/probes setup in this same fashion. I have 2 servers with 1 IP but have many sites. I want to probe each side and ensure I get a 200 code. I also have to provide credentials to the site. It seems that if i open IE I can log in just fine to the site with the credentials. However there is an active x control box that is wanting to be installed. When I set this up on my ACE it seems I am getting a http 401 unauthorized error. I have done a wireshark capture while I was browsing and I see the 401 however it also reports a 200 code after that. Do you think this is a problem because of the active x control wanting to be downloaded? Or is this an issue with the first http code that is recieved by the probe, that being the 401 and then the 200? Below is my config (cleaned of course).
probe http HTTP-80-OUR.DOMAIN.COM
interval 15
passdetect interval 60
credentials
request method get url http://our.domain.com/default.aspx
expect status 200 200
header Host header-value "our.domain.com"
open 1
rserver host SERVER-A
ip address X.X.X.47
inservice
rserver host SERVER-B
ip address X.X.X.48
inservice
serverfarm host FARM-AB
predictor leastconns
probe HTTP-80-OUR.DOMAIN.COM
rserver SERVER-A
inservice
rserver SERVER-B
inservice
ACE4710# show probe HTTP-80-OUR.DOMAIN.COM detail
probe : HTTP-80-OUR.DOMAIN.COM
type : HTTP
state : ACTIVE
description :
port : 80 address : 0.0.0.0 addr type : -
interval : 15 pass intvl : 60 pass count : 3
fail count: 3 recv timeout: 10
http method : GET
http url : http://our.domain.com
conn termination : GRACEFUL
expect offset : 0 , open timeout : 1
expect regex : -
send data : -
------------------ probe results ------------------
associations ip-address port porttype probes failed passed health
------------ ---------------+-----+--------+--------+--------+--------+------
serverfarm : OUR.DOMAIN.COM-10.25.4.12-L3-FARM
real : SERVER-A[0]
X.X.X.47 80 DEFAULT 414 406 8 FAILED
Socket state : CLOSED
No. Passed states : 1 No. Failed states : 2
No. Probes skipped : 0 Last status code : 401
No. Out of Sockets : 0 No. Internal error: 0
Last disconnect err : Received invalid status code
Last probe time : Wed Jun 2 17:44:18 2010
Last fail time : Wed Jun 2 13:37:04 2010
Last active time : Wed Jun 2 13:34:19 2010
real : SERVER-B[0]
X.X.X.48 80 DEFAULT 414 406 8 FAILED
Socket state : CLOSED
No. Passed states : 1 No. Failed states : 2
No. Probes skipped : 0 Last status code : 401
No. Out of Sockets : 0 No. Internal error: 0
Last disconnect err : Received invalid status code
Last probe time : Wed Jun 2 17:44:20 2010
Last fail time : Wed Jun 2 13:37:06 2010
Last active time : Wed Jun 2 13:34:21 2010 -
With Ajay Kumar and Telmo Pereira
Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn and ask questions about configuration and troubleshooting the Cisco Application Control Engine (ACE) load balancer with Cisco expert Ajay Kumar and Telmo Pereira. The Cisco ACE Application Control Engine Module for Cisco Catalyst 6500 Series Switches and Cisco 7600 Series Routers is a next-generation load-balancing and application-delivery solution. A member of the Cisco family of Data Center 3.0 solutions, the module: Helps ensure business continuity by increasing application availability Improves business productivity by accelerating application and server performance Reduces data center power, space, and cooling needs through a virtualized architecture Helps lower operational costs associated with application provisioning and scaling
Ajay Kumar is a customer support engineer in the Cisco Technical Assistance Center in Brussels, covering content delivery network technologies including Cisco Application Control Engine, Cisco Wide Area Application Services, Cisco Content Switching Module, Cisco Content Services Switches, and others. He has been with Cisco for more than four years, working with major customers to help resolve their issues related to content products. He holds DCASI and VCP certifications.
Telmo Pereira is a customer support engineer in the Cisco Technical Assistance Center in Brussels, where he covers all Cisco content delivery network technologies including Cisco Application Control Engine (ACE), Cisco Wide Area Application Services (WAAS), and Digital Media Suite. He has worked with multiple customers around the globe, helping them solve interesting and often highly complex issues. Pereira has worked in the networking field for more than 7 years. He holds a computer science degree as well as multiple certifications including CCNP, DCASI, DCUCI, and VCP
Remember to use the rating system to let Ajay know if you have received an adequate response.
Ajay and Telmo might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Data Center sub-community discussion forum Application Networking shortly after the event.
This event lasts through July 26, 2013. Visit this forum often to view responses to your questions and the questions of other community members.Hello Krzysztof,
Another set of good/interesting questions posted. Thanks!
I will try to clarify your doubts.
In the output below both resources (proxy-connections and ssl-connections rate) are configured with a min percentage of resources (column Min), while 'Max' is set to equal to the min.
ACE/Context# show resource usage
Allocation
Resource Current Peak Min Max Denied
-- outputs omitted for brevity --
proxy-connections 0 16358 16358 16358 17872
ssl-connections rate 0 626 626 626 23204
Most columns are self explanatory, 'Current' is current usage, 'Peak' is the maximum value reached, and the most important counter to monitor 'Denied' represents the amount of packets denied/dropped due to exceeding the configured limits.
On the resources themselves, Proxy-connections is simply the amount of proxied connections, in other words all connections handled at layer 7 (SSL connections are proxied, as are any connections with layer 7 load balance policies, or inspection).
So in this particular case for the proxy-connections we see that Peak is equal to the Max allocated, and as we have denies we can conclude that you have surpassed the limits for this resource. We see there were 17872 connections dropped due to that.
ssl-connections rate should be read in the same manner, however all values for this resource are in bytes/s, except for Denied counter, that is simply the amount of packets that were dropped due to exceeding this resource.
For your particular tests you have allocated a min percentage and set max equal to min, this way you make sure that this context will not use any other additional resources.
If you had set the max to unlimited during resource allocation, ACE would be allowed to use additional resources on top of those guaranteed, if those resources were available.
This might sound a great idea, but resource planning on ACE should be done carefully to avoid any sort of oversubscription, specially if you have business critical contexts.
We have a good reference for ACE resource planning that contains also description of all resources (this will help to understand the output better):
http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A2/configuration/virtualization/guide/config.html#wp1008224
1) When a resource is utilized to its maximum limit, the ACE denies additional requests made by any context for that resource. In other words, the action is to Drop. ACE should in theory silently drop (No RST is sent back to the client). So unless we changed something on the code, this is what you should see.
To give more context, seeing resets with SSL connections is not necessarily synonym of drops. As it is usual to see them during normal transactions.
For instance Microsoft servers are usually ungracefully terminating SSL connections with RESET. Also when there is renegotiation during an SSL transaction you may see RESETS, but this will pass unnoticed for end users.
2) ACE will simply drop/ignore new connections when we reach the maximum amount of proxied connections for that context. Exisiting connections will continue there.
As ACE doesn't respond back, client would simply retransmit, and if he is lucky maybe in the next attempt he will be able to establish the connection.
To overcome the denies, you will definitely have to increase the resource allocation. This of course, assuming you are not reaching any physical limit of the box.
As mentioned setting max as unlimited might work for you, assuming there are a lot of unused resources on the box.
3) If a new connection comes in with a sticky value, that matches the sticky entry of a real server, which is already in MAXCONNS state, then both the ACE module/appliance should reject the connection and that sticky entry would be removed.
The client would at that point reestablish a new connection and ACE would associate a new sticky entry with the flow for a new RSERVER after the loadbalancing decision.
I hope this makes things clearer! Uff...
Regards,
Telmo -
Configuring ACE 4710 for Load Balancing Speech servers
Hello, I'm configuring ACE 4710's for the first time and I want to load balance my Nuance speech servers on port 554. Here's my configuration on ACE01:
hostname ace471001
interface gigabitEthernet 1/1
switchport access vlan 1000
no shutdown
interface gigabitEthernet 1/2
shutdown
interface gigabitEthernet 1/3
shutdown
interface gigabitEthernet 1/4
shutdown
access-list ALL line 8 extended permit ip any any
rserver host nss01
class-map type management match-any remote_access
2 match protocol xml-https any
3 match protocol icmp any
4 match protocol telnet any
5 match protocol ssh any
6 match protocol http any
7 match protocol https any
8 match protocol snmp any
policy-map type management first-match remote_mgmt_allow_policy
class remote_access
permit
interface vlan 1000
ip address 10.20.17.21 255.255.248.0
access-group input ALL
service-policy input remote_mgmt_allow_policy
no shutdown
How would I configure my speech server to isten on 554?
Thanks in advanceHello Reginald
Currently you have only basic network configuration, there is no loadbalancing config
I'm not sure what exactly you're asking about , but basically you need to have
- real servers configured on ACE (
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/slb/guide/rsfarms.html#wp999495)
- serverfarm configured on ACE (
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/slb/guide/rsfarms.html#wp1014522)
- L7 policy map (
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/slb/guide/classlb.html#wp1171109 ,
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/slb/guide/classlb.html#wp1027248 )
- L4 policy map , class-map (
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/slb/guide/classlb.html#wp1027819)
And then apply it on necessary interface.
This is a general configuration, in your specific case you may need to configure some additinal features (e.g. I think you will need to have stickiness enabled
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/slb/guide/sticky.html but it depends on your application)
links are for old config guids , but basic is pretty much the same for all versions.
Please check them and try to narrow down your question a bit. -
Hi,
Pls can you help me find out where is my error in the below:
I have an ACE 4710. Also I have 2 Bluecoat Proxy SG working in proxy mode. I want the ACE to be the Load Balancer for these 2 Proxy SG. I configure the ACE as below and put the vip-address in the Internet Explorer LAN Settings but it did not work. Also I configure Policy-based Routing on the Core Switch (for any http or https traffic going through core apply set ip next-hop vip-address).
Core SW SVI:
interface Vlan56
description BC Proxy
ip address 10.0.1.33 255.255.255.224
interface Vlan57
description ACE-LB-Alias
ip address 10.0.1.65 255.255.255.224
ACE 4710:
hostname VSS-ACE-BC-01
interface gigabitEthernet 1/1
description Management
speed 1000M
duplex FULL
switchport access vlan 101
no shutdown
interface gigabitEthernet 1/2
description User Side
speed 1000M
duplex FULL
switchport access vlan 56
no shutdown
interface gigabitEthernet 1/3
description BC Proxy Side
speed 1000M
duplex FULL
switchport access vlan 57
no shutdown
interface gigabitEthernet 1/4
description Failover
speed 1000M
duplex FULL
ft-port vlan 900
no shutdown
context Admin
member sticky
access-list external line 10 extended permit ip any any
access-list external line 20 extended permit icmp any any
access-list external line 30 extended permit tcp any any
access-list external line 40 extended permit udp any any
access-list internal line 10 extended permit ip any any
access-list internal line 20 extended permit icmp any any
access-list internal line 30 extended permit tcp any any
access-list internal line 40 extended permit udp any any
probe tcp web443
port 443
interval 30
faildetect 1
passdetect interval 30
passdetect count 1
open 1
probe tcp web8080
port 8080
interval 30
faildetect 1
passdetect interval 30
passdetect count 1
open 1
rserver host BC01
ip address 10.0.1.41
inservice
rserver host BC02
ip address 10.0.1.42
inservice
serverfarm host web443
probe web443
rserver BC01
inservice
rserver BC02
inservice
serverfarm host web8080
probe web8080
rserver BC01
inservice
rserver BC02
inservice
sticky ip-netmask 255.255.255.255 address source group1
replicate sticky
serverfarm web8080
sticky ip-netmask 255.255.255.255 address source group2
replicate sticky
serverfarm web443
class-map type management match-any REMOTE_ACCESS
2 match protocol telnet any
3 match protocol ssh any
4 match protocol icmp any
5 match protocol http any
6 match protocol snmp any
class-map match-all external-web
2 match virtual-address 10.0.1.70 any
class-map match-all external-web443
2 match virtual-address 10.0.1.70 any
class-map match-any nat-class
2 match access-list external
policy-map type management first-match REMOTE_MGMT
class REMOTE_ACCESS
permit
policy-map type loadbalance http first-match slb
class class-default
sticky-serverfarm group1
policy-map type loadbalance http first-match slb443
class class-default
sticky-serverfarm group2
policy-map multi-match external-access
class nat-class
nat dynamic 1 vlan 57
class external-web
loadbalance vip inservice
loadbalance policy slb
class external-web443
loadbalance vip inservice
loadbalance policy slb443
timeout xlate 120
interface vlan 56
description Server-Side
ip address 10.0.1.43 255.255.255.224
ip verify reverse-path
alias 10.0.1.40 255.255.255.224
peer ip address 10.0.1.44 255.255.255.224
mac-address autogenerate
access-group input internal
service-policy input REMOTE_MGMT
no shutdown
interface vlan 57
description VIP-Interface
ip address 10.0.1.67 255.255.255.224
alias 10.0.1.66 255.255.255.224
peer ip address 10.0.1.68 255.255.255.224
mac-address autogenerate
access-group input external
service-policy input external-access
service-policy input REMOTE_MGMT
no shutdown
interface vlan 101
description Management
ip address 10.220.1.131 255.255.255.0
alias 10.220.1.133 255.255.255.0
peer ip address 10.220.1.132 255.255.255.0
mac-address autogenerate
service-policy input REMOTE_MGMT
no shutdown
ft interface vlan 900
ip address 172.20.100.1 255.255.255.252
peer ip address 172.20.100.2 255.255.255.252
no shutdown
ft peer 1
heartbeat interval 300
heartbeat count 20
ft-interface vlan 900
ft group 1
peer 1
priority 200
peer priority 150
associate-context Admin
inservice
ip route 0.0.0.0 0.0.0.0 10.0.1.65I see that you used:
nat dynamic 1 vlan 57
Where is the nat pool on Vlan 57 ?
May be you can try to assign that and that should help.
Something like below:
Interface vlan 57
nat-pool 1 10.0.1.93 10.0.1.93 netmask 255.255.255.224 pat
regards,
Ajay Kumar -
I have ACE 4710 and I need configuration:
I have real web-server with folders : /1/index.html, /2/index.html, /3/index.html
I need to balance virtual service:
If I try to connect URL: http://server/index.html, then ACE balance among
http://real_server/1/index.html,
http://real_server/2/index.html,
http://real_server/3/index.htm
How can I configure ACE ?ACE, can't modify the url.
But it can send redirect.
So you could build 3 redirect rservers, and have ACE loadbalance between them.
rserver redirect HTTP-REDIRECT1
webhost-redirection http://real_server/1/index.html
inservice
rserver redirect HTTP-REDIRECT2
webhost-redirection http://real_server/2/index.html
inservice
rserver redirect HTTP-REDIRECT3
webhost-redirection http://real_server/3/index.html
inservice
serverfarm redirect SF_REDIRECT
rserver HTTP-REDIRECT1
inservice
rserver HTTP-REDIRECT2
inservice
rserver HTTP-REDIRECT3
inservice
But even if it works, this does not sound good.
It seems like a design done by an application server person who does not know how network loadbalancers work.
It seems like all you need is stickyness, which you are trying to achieve by redirecting to /1 or /2 or /3.
But this can be done differently with cookies or by just doing stickyness on source ip address.
Gilles. -
Hi,
Will ACE 4710 support for IPS features?
Regards,
Lingaraj R N
+91-9920944501Hi Lingaraj,
Are you looking for any specific features?
Please go through the below link which is security configuration guide for security features available in ACE.
http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA3_1_0/configuration/security/guide/appinsp.html
Let me know if you have any doubts.
Regards,
Kanwal -
Displaying all application windows through Hot Corners is no longer working.
I have my MacBook Pro and my iMac setup to display APPLICATION WINDOWS through the upper right HOT CORNER. It is no longer working on my iMac. Instead, the system only displays the top/recent used window. The display separates and shows the most recent window floating, but, no longer does it arrange windows of each application on the desktop. There are no other options, it's either set in HOT CORNERS or not. What changed? It still works on my MacBook. Version update = 10.10.2 (14C1514)
Please read this whole message before doing anything.
This procedure is a test, not a solution. Don’t be disappointed when you find that nothing has changed after you complete it.
Step 1
The purpose of this step is to determine whether the problem is localized to your user account.
Enable guest logins* and log in as Guest. Don't use the Safari-only “Guest User” login created by “Find My Mac.”
While logged in as Guest, you won’t have access to any of your documents or settings. Applications will behave as if you were running them for the first time. Don’t be alarmed by this behavior; it’s normal. If you need any passwords or other personal data in order to complete the test, memorize, print, or write them down before you begin.
Test while logged in as Guest. Same problem?
After testing, log out of the guest account and, in your own account, disable it if you wish. Any files you created in the guest account will be deleted automatically when you log out of it.
*Note: If you’ve activated “Find My Mac” or FileVault, then you can’t enable the Guest account. The “Guest User” login created by “Find My Mac” is not the same. Create a new account in which to test, and delete it, including its home folder, after testing.
Step 2
The purpose of this step is to determine whether the problem is caused by third-party system modifications that load automatically at startup or login, by a peripheral device, by a font conflict, or by corruption of the file system or of certain system caches.
Please take this step regardless of the results of Step 1.
Disconnect all wired peripherals except those needed for the test, and remove all aftermarket expansion cards, if applicable. Start up in safe mode and log in to the account with the problem. You must hold down the shift key twice: once when you turn on the computer, and again when you log in.
Note: If FileVault is enabled in OS X 10.9 or earlier, or if a firmware password is set, or if the startup volume is a software RAID, you can’t do this. Ask for further instructions.
Safe mode is much slower to start up and run than normal, with limited graphics performance, and some things won’t work at all, including sound output and Wi-Fi on certain models. The next normal startup may also be somewhat slow.
The login screen appears even if you usually log in automatically. You must know your login password in order to log in. If you’ve forgotten the password, you will need to reset it before you begin.
Test while in safe mode. Same problem?
After testing, restart as usual (not in safe mode) and verify that you still have the problem. Post the results of Steps 1 and 2.
Maybe you are looking for
-
No data in tables after migration
I am migrating SQL Server database to Oracle using using SQL Developer - and all when well until I checked the tables and there is no data. Login as the new schema and do "select count(*) from user_tables" Got 35 tables, but when I checked the indivi
-
My ipod touch has a huge collection of music that I don't want to lose or replace but my computer has now crashed beyond recovery. How can I access my itunes account from another computer or transfer my playlists to another computer from my ipod?
-
Btyahoo homepage not loading with Safari on iPad
Safari will not load my Bt Yahoo homepage, all other web page load ok.
-
To debug the Tcode F110 for script output
Hi, I want to debug the Tcode F110 for the onr particular vendor whose payment has be runned. the script that is attached is the ZScript. I want to debug the attached script through F110 to view the payment output in production. Kindly suggest how to
-
MacBook 10.6.8 Won't See Home WiFi
Here are my symptoms: At only my home office Airport will not even see my Netgear router. This is a new condition as of a week ago. It did start with not seeing router when opening laptop. I would play around in preferences and it would start to