Cisco ACS 4.2 - Server Busy

Hi!
We're authenticating our Desktops and IP-Phones via 802.1x using two Radius-servers running Cisco ACS v4.2 on Win2k8.
From time to time we run into the problem, that one of the servers 'get's too busy' and stops answering authentication requests. That results in many failed authentications with our VoIP-phones (Siemens OpenStage).
What I don't understand is why the ACS acts that way...
TAC says that all 42 or so threads are in use when the server says it's too busy.
While the server is 'busy' the CPU runs at 1 - 2 % !! And there's loads of RAM left...
This is an extract from the CSRadius-Log-File:
RDS 06/09/2011 07:51:13 E 1495 2072 0x0 Server too busy - request from 10.104.204.249 ignoredRDS 06/09/2011 07:51:13 E 1495 5124 0x0 Server too busy - request from 10.104.204.249 ignoredRDS 06/09/2011 07:51:13 E 1495 5124 0x0 Server too busy - request from 10.100.204.22 ignoredRDS 06/09/2011 07:51:13 E 0958 3712 0x0 Error processing accounting request - no response sent to NASRDS 06/09/2011 07:51:13 E 5947 4916 0x0 Failed to update logged on list for IPPhone (UDB_SERVER_BUSY)RDS 06/09/2011 07:51:13 E 1495 5124 0x0 Server too busy - request from 10.100.204.22 ignoredRDS 06/09/2011 07:51:13 E 0958 1880 0x0 Error processing accounting request - no response sent to NASRDS 06/09/2011 07:51:13 E 6025 3560 0x0 Matching class attribute failed for user IPPhone, no further processing will be done assuming this is out-of-order packet due to UDPRDS 06/09/2011 07:51:13 E 1825 1532 0x0 Error UDB_SERVER_BUSY authenticating host/hostname.xxx.yyy - no response sent to NAS...RDS 06/09/2011 07:51:20 E 3089 2704 0x0 Error AS_NO_FREE_CONNECTIONS authenticating IPPhone - no response sent to NAS
Did any of you encounter the same problem? Did you find a workaround or fix? Maybe there's a way to increase the number of authentication threads?
Thanks alot!

The key is to get all of the information needed. Normally when they say it takes too long for the client to answer that is not always the exact fault.
You may seem to get that answer if the ACS is taking a long time to process the request and the switch or client has basically timed out its requests.
The information needed is the following
all of these items really need to be gathered at the same time
switch debugs including
debug radius
debug aaa authen
debug aaa accounting
sniffer capture between the switch and the ACS
logs from ACS with debugs enabled.
If you are going to AD on the backend you may also want a sniffer capture between the ACS and the AD
all of these together should tell you where the delay of failure lays and then at that time some changes can be suggested

Similar Messages

Cisco ACS on Virtual Server

Hi,
I have a virtual server running ACS v4.1, when i try and authenticate against Active Directory i get a failed with the Failure-Code of "Internal Error".
Does anyone know if this is this a compatability issue?
Many thanks
Chris

Hi Chris,
Few things before troubleshooting. According to release notes:-
++++++++++
â¢VMware. ACS 4.1 was tested on the following VMWare platform:
-VMWare ESX server 3.0.0
-Processor-AMD Opteron Dual core
-# of Virtual machines-4
-Guest operating system-Windows 2003 Standard Edition
-RAM for each guest operation system-3 GB
Note The Microsoft JVM is no longer supported. ACS 4.1 supports the Sun Java Run-time Environment (JRE) 1.4.2_04. This is an ACS for Windows web client requirement.
Note ACS is supported on Windows Server 2003 R2.
+++++++++++++++++++++++++++++++++++++
http://www.cisco.com/en/US/partner/docs/net_mgmt/cisco_secure_access_control_server_for_windows/4.1/release/notes/RNacs41.html#wp140886
+++++++++++++++++++++++++++++++++++++
'Internal error is a very generic error and to find out the real cause i need to check the logs from ACS. Auth logs, tacacs/raduis depends upon the authen method alon with time stamp
You need to make sure that ACS is installed on tested platform otherwise you will keep on facing unexpected errors.
Vinay

Integration of Cisco ACS SE 4.2 and RSA SecurID Token Server

Hi,
I would be very appreciated if anyone can share their experience. Thanks in advance.
Issue:
I am trying to configure the ACE SE 4.2 to authenticate using RSA SecurID Token Server.
Problems encountered:
Authentication failed. In the failed logged attempt the error "External Database not operational" was next to the login name.
In the auth.log, there was "External DB [SecurID.dll]: aceclnt.dll callback returned error [23]".
Questions:
1. Please kindly advise how I should resolve this problem.
2. Also, is there any successful message once ACS get the sdconf.rec? Will the "Purge Node Secret" button be enabled?
Troubleshooting steps I have done:
Below is the steps I took to setup the external DB.
1. Verified sdconf.rec is not a garbage file using the Test authentication function in RSA client.
2. FTP sdconf.rec in the external database configuration. (Had used Wireshark and confirm file transfered successfully.)
2. Defined unknown user policy to check RSA SecurID Token Server to authenticate.
Thank you.

I have NO experience with ACS SE 4.2 and
RSA SecurID Token Server BUT I have
experiences with Cisco ACS 4.1 running on
Windows 2003 SP2 Enterprise Edition and
RSA SecurID Token Server.
All the troubleshoot you've done is correct.
In Windows 2003 running Cisco ACS, you can
install the test authentication RSA client
and that you can verify that the setup
is correct (by verifying that the sdconf.rec
is not corrupted).
One thing I can think of is that when you
setup the ACS SE box, under external
database, configure unknown user policy,
did you check it to tell how to define users
when they are not found in the ACS internal
database. Did you select RSA SecurID token
server?
Other than that, from what I understand,
you've done everything correctly.

How to manage the Cisco ACS 3.3 from ciscoworks server ?

i have ACS 3.3 server appliance and want to manage it from the ciscoworks server .i wanted to see it in the campus manager and can i manage ite inventory ?
Cisco Secure Access Control Server Solution Engine.

Hai,
Yes absolutely right. ACS appliance cannot be managed since SNMP cannot be enabled in ACS.
I tried but after raising TAC case with Cisco they told it is not possible.
In campus manager u cannot view the cisco devices ACS,NIDS and PIX firewall
Hope this information is useful

Migrating from Linux based Tacacs+ server to Cisco ACS 1113 appliance

I'm trying to migrate my configuration from a Linux based Tacacs+ server to the Cisco ACS 1113 appliance. Does anyone have any recommendations.
Thanks.

Hi
We (extraxi) offer migration and general consultancy for ACS if you need professional help.
www.extraxi.com/contact.htm

Extra server on cisco ACS engine

I'm a bit curious about the way the cisco ACS engine (the cisco-built hardware) sets up servers initially. Most of the documentation I have is for windows, so I was a bit confused when, after the initial configuration there were two "AAA Servers" shown in the configuration, one called "Self" with the IP address I defined, and the other with the name I defined and a different address.
Has anyone else encountered this? Will it cause problems? and is there a way to get rid of it?
Thanks

That is a known issue with acs appliance, but nothing to worry about. Make sure you have this setting in acs,
acs--->network configuration--->Proxy dis table---> Bring Deleverance1 in the fwd to box and your server name in the left box.
Incase you dont see proxy dis table , then you need to enable it
Interface configuration---> Advance option ---> Put a check in distribution table.
Regards,
~JG
Please do rate helpful posts

Cisco ACS 1121 server configuration

Hi,
Anyone can tell me how to configure LAN teaming in Cisco ACS 1121. My requirement is to have virtual IP in the server with two physical IPs in the available 2 interface in the server.
Regards,
Haja Shajahan.M

Currently Gig 0 is supported. Gig 1 is blocked. Check this link ((Blocked) Gigabit Ethernet 1).
http://www.cisco.com/en/US/partner/docs/net_mgmt/cisco_secure_access_control_system/5.2/installation/guide/csacs_hw_ins.html#wp1119105
Paps

Cisco ACS Server

Hi
I have at present a Cisco ACS server 3.3. I want to upgrade the server to latest version and also cluster it with another one so that we could have a redundant infrastructure as if one fails the other one takes over ..
CAn you provide a suitable solution for this ?
Thanks

Hi,
The Latest version is ACS 4.1. You can upgrade from 3.3.3 build 11 directly to 4.1.
Then you can install another ACS 4.1 on different machine and setup replication between these two. This way you will have to make changes only on one ACS and the secondary will automatically get updated.
Once these two are set, you can define both of these server as Radius/Tacacs server on the devices and there would be a redundancy.
Regards,
Vivek

Cisco ACS Server . Download Evaluation Version For Testing.

Hello.
I want to try to install ACS server for windows to check how this is working with Microsoft AD. Does anyone know where i can download an evaluation version of Cisco ACS Server for Windows ?

Hello Michael-
The ACS version for Windows is no longer available. The product is EOL/EOS:
http://www.cisco.com/c/en/us/products/collateral/security/secure-access-control-server-windows/end_of_life_notice_c51-664639.html
The product was replaced with a Linux based version (5.x) and it is a lot easier of a product to install and manage.
If you want to evaluate the product I would recommend that you contact your local Cisco partner:
https://tools.cisco.com/WWChannels/LOCATR/openBasicSearch.do
Thank you for rating helpful posts!

Limitations of Cisco ACS server

I want to ask about limitations of Cisco ACS server 3.3 .
I use ACS server for Radius authentication, and has a limit 80 authentications per second. But at peak time i need 150-200 authentications per second. Is this a software limitaion or changed due to hardware performance?
Can i also solve this problem with a High Availability configuration.

Hi
ACS performance is a very complex issue and depends largely on
1) auth protocol (anything eap is SLOW)
2) backend (anything external is SLOW)
3) server CPU
We did some performance tests a few years ago and could get up to 1000 auths/sec for MSCHAP against internal DB.
AD authentication/group mapping can take several seconds to complete.
ACSs big problem is limited concurrency when authentication time is high. There are some bottlenecks that effectively limit the number of concurrent authentications to 20. This is the max number of tcp/ip connections between CSRadius/CSTacacs and CSAuth. Inside CSRadius there are 50 dedicated authentication threads multiplexing requests over the 20 tcp/ip connections to CSauth. Messages to CSauth are blocking - so 20 simultaneous authentications that took 1 second would cap performance to 20 auths/sec.
EAP-TLS and now EAP-FAST are really really slow becase they send multiple rounds over RADIUS using challenge/response marshalled between the device and the 802.1x supplicant.
Putting ACS onto a quad CPU server wont reduce back-end external db latency or increase concurrency.
The only way to increase performance is to add more servers... and then you'll also have to get into load balancing :(
IMHO Cisco needs to make a low cost "ACS on a blade" and have one in each device. Have the config pushed down from a central database.
Darran

Cisco ACS Server Tacacs Based on LDAP AND Source IP Possible???

Hi All,
I have used Cisco ACS tacacs for authentication based on Active Directory. Is it possible to use Active Directory as a criteria for authentication AND source IP?
For example, if someone wants to log in to a certain device... they must have correct credentials AND their IP must be sourcing from the acceptable subnet range.
Thanks!

I see your point. This will depend if the user's IP is provided in the authentication request, if this information is provided then you can use the feature called "End Station Filter". This feature is used as a Condition in the Access Policy to deny or allow access. Below are the steps:
1. Create a End Station Filter, here configure the user's IP
2. Customize your Conditions under Access Policies/Authorization to use End Station Filter
3. Define your rule with the required result

VPN client and Cisco ACS

hi,
I'm trying to setup a VPN solution, connecting to a 800 series router and authenticating off a Cisco ACS tacacs server.
I've basically followed the suggested config at http://www.cisco.com/en/US/customer/tech/tk59/technologies_configuration_example09186a00800a393b.shtml and the setup works fine if I use local authentication, but as soon as I switch to using TACACS the client authentication fails.
Debugging tacacs on the router i can see the requests being sent to the server, and the replies coming back - the login detail are definitely correct so I'm guessing that TACACS isn't authorising me to use VPN or IPSEC or something. But there is nothing in the ACS logs to suggest why I'm not getting through - no failed attempts are shown.
Any ideas?

here is some debug from the router:
Feb 24 12:28:58.973 UTC: TPLUS: processing authentication start request id 129
Feb 24 12:28:58.973 UTC: TPLUS: Authentication start packet created for 129(vpngroup)
Feb 24 12:28:58.973 UTC: TPLUS: Using server 10.10.10.10
Feb 24 12:28:58.973 UTC: TPLUS(00000081)/0/NB_WAIT/823A9F04: Started 5 sec timeout
Feb 24 12:28:58.989 UTC: TPLUS(00000081)/0/NB_WAIT: socket event 2
Feb 24 12:28:58.989 UTC: T+: Version 192 (0xC0), type 1, seq 1, encryption 1
Feb 24 12:28:58.989 UTC: T+: session_id 1729330768 (0x67137E50), dlen 16 (0x10)
Feb 24 12:28:58.989 UTC: T+: type:AUTHEN/START, priv_lvl:1 action:LOGIN ascii
Feb 24 12:28:58.989 UTC: T+: svc:LOGIN user_len:8 port_len:0 (0x0) raddr_len:0 (0x0) data_len:0
Feb 24 12:28:58.989 UTC: T+: user: vpntest
Feb 24 12:28:58.989 UTC: T+: port:
Feb 24 12:28:58.989 UTC: T+: rem_addr:
Feb 24 12:28:58.989 UTC: T+: data:
Feb 24 12:28:58.989 UTC: T+: End Packet
Feb 24 12:28:58.989 UTC: TPLUS(00000081)/0/NB_WAIT: wrote entire 28 bytes request
Feb 24 12:28:58.993 UTC: TPLUS(00000081)/0/READ: socket event 1
Feb 24 12:28:58.993 UTC: TPLUS(00000081)/0/READ: Would block while reading
Feb 24 12:28:59.009 UTC: TPLUS(00000081)/0/READ: socket event 1
Feb 24 12:28:59.009 UTC: TPLUS(00000081)/0/READ: read entire 12 header bytes (expect 16 bytes data)
Feb 24 12:28:59.009 UTC: TPLUS(00000081)/0/READ: socket event 1
Feb 24 12:28:59.009 UTC: TPLUS(00000081)/0/READ: read entire 28 bytes response
Feb 24 12:28:59.009 UTC: T+: Version 192 (0xC0), type 1, seq 2, encryption 1
Feb 24 12:28:59.009 UTC: T+: session_id 1729330768 (0x67137E50), dlen 16 (0x10)
Feb 24 12:28:59.009 UTC: T+: AUTHEN/REPLY status:5 flags:0x1 msg_len:10, data_len:0
Feb 24 12:28:59.009 UTC: T+: msg: Password:
Feb 24 12:28:59.009 UTC: T+: data:
Feb 24 12:28:59.009 UTC: T+: End Packet
s9990-cr#
Feb 24 12:28:59.009 UTC: TPLUS(00000081)/0/823A9F04: Processing the reply packet
Feb 24 12:28:59.009 UTC: TPLUS: Received authen response status GET_PASSWORD (8)
"AUTHEN/REPLY status:5" is a permanent fail according to the TACACS RFC
In the VPN Client log it say "User does not provide any authentication data"
So to summarise:
-Same ACS server\router\username combination works fine for telnet access.
-VPN works fine with local authentication.
-No login failures showing in the ACS logs.

Cisco ISE 1.2 and Cisco ACS 5.4 patch 6 and support for snmp version 3

does anyone know if cisco ISE version 1.2 patch 8 and Cisco ACS 5.4 patch 6 support snmp version 3?
ciscoISE/admin(config)# snmp-server ?
community Set community string
contact    Text for mib object sysContact
host       Specify hosts to receive SNMP notifications
location   Text for mib object sysLocation
ciscoISE/admin(config)# snmp-server
Ciscoacs/admin(config)# snmp-server ?
community Set community string
contact    Text for mib object sysContact
host       Specify hosts to receive SNMP notifications
location   Text for mib object sysLocation
Ciscoacs/admin(config)# snmp-server

No support SNMP v3 on ISE v1.2 and 1.3 except for profilling
http://www.cisco.com/c/en/us/td/docs/security/ise/1-2/cli_ref_guide/ise_cli/ise_cli_app_a.html#12768
http://www.cisco.com/c/en/us/td/docs/security/ise/1-3/cli_ref_guide/b_ise_CLIReferenceGuide/b_ise_CLIReferenceGuide_chapter_0100.html#ID-1364-00000d30

Linksys WAP54G connecting to CISCO ACS via LEAP

I understand that Linksys WAP54G support WPA and 802.1x authentication. Will a cisco compatible client card get connected to the WAP54G via LEAP authentication to a Cisco ACS server ?
Connection scenario:-
Cisco compatible client card <-WPA/LEAP-> WAP54G <-WPA/LEAP-> Cisco ACS3.1
Pls advise if such setting is feasible.
Tks

This is really a question for Linksys support. The Cisco wireless BU has no involvement with the Linksy's product line. They operate as a totally separate wholly own subsidiary of Cisco.
As for LEAP, no, to my knowledge the Linksys AP does not support LEAP, which is not tested or part of the WPA certification program. To my knowledge the ONLY APs that support LEAP are Cisco Aironet APs.
If the Linksys supports WPA-Enterprise, then any client that supports WPA-Enterprise should work using EAP-TLS. The Cisco ACS server supports EAP-TLS.
One word of caution. Early CCX cards do not necessarily support WPA. The CCX specification and certification were out before WPA was released. You will need to check with the actual vendor of the card to verify WPA compatibility.;
Also there are two types of WPA. WPA-Personal, which supports only the WPA encryption, and the keys are handles by a Pre-shared Key input system (no radius server) and WPA-Enterprise, which is certified using WPA encryption an 802.1x EAP-TLS radius server (in fact using Microsoft and Funk Software servers). make sure that the Linksys supports WPA-enterprise, or it may not support 802.1x.
Bruce Alexander, Cisco

Using Cisco ACS for Solaris login authentication

Hi all
I am planning to authenticate ssh logins to Solaris 8/9 systems using PAM and radius (while radius is considered the primary solution, tacacs+ could be used, too). The radius/tacacs+ server is provided by a Cisco ACS.
Can anybody out there confirm that the combination "Solaris & PAM & radius/tacacs+ & Cisco ACS" is correctly doing this authentication stuff? Is there anything to specially consider?
Thanks, David

Hard to comment with any certainty but provided the client implementation of RADIUS is sound AND the authentication protocol is one that ACS supports, eg PAP, CHAP, MSCHAP, LEAP, EAP (PEAP/FAST/TLS/GTC/MSCHAP) then should be fine.

Cisco ACS 4.2 - Server Busy

Similar Messages

Maybe you are looking for