CUCM Syslog Message ISSUE (kernel: Exceeded hashlimit)

Hello.
Our Customer using CUCM 9.0 (PUB :1 , Sub : 4) and 4 Voice Gateway Cisco 3945 (16 E1 PRI per each Gateway)
CUCMs have problem with syslog messages.
I saw these messages in rtmt syslog
- kernel: Exceeded hashlimit IN=bond0 OUT= MAC=34:40:b5:d5:63:e8:1c:e6:c7:52:44:40:08:00 SRC=130.1.254.27 DST=130.1.13.11 LEN=204 TOS=0x00 PREC=0x00 TTL=246 ID=19646 PROTO=UDP SPT=19200 DPT=30546 LEN=184
kernel: Exceeded hashlimit IN=bond0 OUT= MAC=6c:ae:8b:67:1a:28:bc:16:65:12:99:7f:08:00 SRC=130.1.254.27 DST=130.1.14.13 LEN=204 TOS=0x18 PREC=0xA0 TTL=253 ID=42621 PROTO=UDP SPT=26694 DPT=26842 LEN=184
What's the problem with these messages ?
And how can I solve this problem
Thanks.

I used to have the same problem, it was a sip trunk against to one CME, just reset the sip trunk in CUCM it fixed the error. it is because the end poing is sending a lot of requests to CUCM

Similar Messages

[worrisome syslog messages after kernel update]

Hi everyone,
I've installed the following two kernel patches on our Solaris 10 Sparc system:
120011-14
127111-02
During installation of 120011-14 I got some warnings:
WARNING: /platform/sun4u/kernel/misc/sparcv9/des <no longer a linked file>
Installation of <SUNWcakr> was successful.
WARNING: /kernel/misc/sparcv9/des <no longer a linked file>
Installation of <SUNWckr> was successful.
added profile to /etc/name_to_major
added n2rng to /etc/name_to_major
added physmem to /etc/name_to_major
added ds_snmp to /etc/name_to_major
added ds_pri to /etc/name_to_major
Installation of <SUNWcsd> was successful.
WARNING: /etc/inet/ipnodes <no longer a regular file>
Installation of <SUNWcsr> was successful.
After the rebooting the system with -r option I recognized a lot of worrisome messages in the syslog:
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'netstack_find_by_stackid'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/ipsecah: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'netstack_rele'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/ipsecah: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'ipsec_rl_strlog'
Nov 8 07:17:23 src@xps1pm krtld: [ID 472681 kern.notice] WARNING: mod_load: cannot load module 'ipsecah'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/ipsecah: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'netstack_register'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/ipsecah: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'secpolicy_ip_config'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/ipsecah: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'kstat_delete_netstack'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/ipsecah: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/aggr: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'mac_capab_get'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/aggr: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'mac_free'
Nov 8 07:17:23 src@xps1pm krtld: [ID 819705 kern.notice] /kernel/drv/sparcv9/aggr: undefined symbol
Nov 8 07:17:23 src@xps1pm krtld: [ID 826211 kern.notice] 'mac_alloc'
Nov 8 07:17:23 src@xps1pm krtld: [ID 472681 kern.notice] WARNING: mod_load: cannot load module 'aggr'
Although the system seems to be ok, I want to know whether those messages are normal or not. Has anybody experienced the same behavior?
Thanks in advance!!
Juergen

The warning messages during installation are not the problem. But what makes me worry is that some modules can not be loaded. Seems to be the wrong drivers for Solaris 10 kernel (or at least unsupported one). Even the path (/kernel/drv/sparcv9) to the modules says that they are for Solaris 9. Do I missunderstand something??
The following modules couldn't be loaded due to undefined symbols:
WARNING: mod_load: cannot load module 'trapstat'
WARNING: mod_load: cannot load module 'ipsecesp'
WARNING: mod_load: cannot load module 'ipsecah'
WARNING: mod_load: cannot load module 'aggr'
Would really appreciate any help! Thanks!
J�rgen

PO message issue above certain PO value

Hi,
I am thinking about a value-related PO message issue. Is this possible by default?
One example:
POs are to be issued via fax, no problem. But if PO value exceeds e.g. 100.000u20AC, the PO should also be issued via e-mail or printout to another (internal) party, just as a notice that this high-value PO has been sent out.
Is there any possibility to implement this w/o big efforts?

by standard setting there is no possibility.
It is possibe with help of ABAP develoment.
No Big effort required

Unterstanding syslog messages from our wlc

Hello,
we use two wlc 4402 (4.1.181.0) and several leightweight accesspoints (AIR-AP1010-E-K9 and AIR-AP1030-E-K9 ) connected to them.
On our syslog server we get a lot of messages from the two wlc, and there are 3 message types which I am a little bit afraid of.
1. ca. 10 times per hour we get the message
apf_80211.c:4792 APF-6-NO_CONFIG_CHANGES: Not saving 'apf.cfg' - no config changes."
Cisco system message guide:
Error Message %APF-6-NO_CONFIG_CHANGES: Not saving '[chars]' - no config changes.
Explanation Not saving - no config changes.
Recommended Action No action is required.
Does anybody know why we get this messages and if it's possibly to suppress them?
2. Intermittently (several times a day) we get the following message types:
a) [ERROR] spam_l2.c 723: Max retransmissions reached on AP 00:0B:85:56:63:40 (CONFIGURE_COMMAND^M , 2)"
b) [ERROR] spam_tmr.c 569: Did not receive hearbeat reply from AP 00:0b:85:56:ae:40"
The MAC address is not every time the same but one of our accesspoints.
On our network management system we get the following trap messages with nearly exactly the same timestamp:
14.01.2008 04:21:56 CET
AP ''00.0b.85.56.63.40'', interface ''0x1'' is down.
When Airespace AP's interface operation status goes down this trap will be sent.
bsnAPDot3MacAddress = 00.0b.85.56.63.40
bsnAPIfSlotId = 0x1
14.01.2008 04:21:56 CET
AP disassociated from Switch.
When an Airespace AP disassociates from a Airespace Switch, the AP disassociated notification will be sent with the dot3 MAC address of the Airespace AP. This will notify the management system to remove Airespace AP from this Airespace Switch.
bsnAPMacAddrTrapVariable =
14.01.2008 04:22:25 CET
AP associated with Switch.
When an Airespace AP Associates to a Airespace Switch, the AP associated notification will be sent with the dot3 MAC address of the Airespace AP. This will help the management system to discover the Airespace AP and add it to system.
bsnAPMacAddrTrapVariable =
bsnAPPortNumberTrapVariable = 1
Cisco system message guide:
a) Error Message %LWAPP-3-TX_ERR3: Max retransmissions for LWAPP control message reached on AP [hex]:[hex]:[hex]:[hex]:[hex]:[hex] for [chars] (number of pending messages is [dec])
Explanation Maximum number of times an LWAPP control packet is transmitted before declaring the AP dead has been reached for this AP. The AP may not be on the network, or might have rebooted.
Recommended Action Check if the AP has rebooted or if it has been removed from the network, or if there are connectivity issues between the AP and the controller.
b) Error Message %LWAPP-3-ECHO_ERR: Did not receive heartbeat reply; AP: [hex]:[hex]:[hex]:[hex]:[hex]:[hex]
Explanation Controller did not get a response for the AP heartbeat message. There may be connectivity issues between the AP and the controller.
Recommended Action Check if the AP has rebooted or if it has been removed from the network, or if there are connectivity issues between the AP and the controller.
Because we don't see any network problems I'm wondering why the connection is lost.
Does anybody have an idea, perhaps CSCsh13928 (http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsh13928, but we don't have much traffic on the wlans) ?
Is there any possibility to remotely check if the accesspoint rebooted?
If you need further information please give me a short feedback.
Many thanks in advance,
Thorsten Steffen

Thanks for the help.
I have set up to send email and syslog messages from the RME applications. LMS server immediately started to send messages to the email server but syslog messages are not forwarded to the syslog server. Everything was done according to your instructions except that the name of the first script (syslog_forward.pl) is made consistent with what the second script (.bat) refer to (forward1.pl). What's the problem? Do RME sends the standard syslog messages via UDP port 514?
Sincerely.

Syslog messages in AAA

I have an issue with a switch's syslog messages showing up in the failed authentication attempts report in the AAA.
If anyone has any thoughts, let me know!!
CHRIS

Do you perhaps have this switch console connected on a terminal server, and if so, does the terminal server have "no exec" configured on the lines used for reverse telnet?
I have seen symptoms similar to what you describe in a situation where I had a switch whose console port was connected to a terminal server and the terminal server lines did not have no exec. It looks like there was some activity on the switch which the terminal server presented a login prompt. The next text displayed on the switch was interpreted by the terminal server as the login id and was logged in the failed attempts log.
HTH
Rick

Syslog messages AP541

Hi community,
to find the reason for my connection problems to our network over a AP541N
I have configured the AP541 to send its syslog messages to a syslog server.
Now I am looking for a document where I can find informations about the received
messages.
For example, what means
hostapd: wlan0: IEEE 802.11 STA 78:a3:e4:3e:f7:19 deauthed from BSSID 00:21:29:03:18:40 reason 3
or
hostapd: wlan0: IEEE 802.11 STA 58:1f:aa:2c:96:4b disassociated from BSSID 00:21:29:03:18:40 reason 8
Are there documents where the messages are explained ?
Regards
Joachim

Here is a document for cisco wireless access controller client reason codes:
http://www.cisco.com/en/US/docs/wireless/controller/3.2/configuration/guide/c32err.html
Client Reason Code…Description…Meaning
0…noReasonCode…Normal operation.
1…unspecifiedReason…Client associated but no longer authorized.
2…previousAuthNotValid…Client associated but not authorized.
3…deauthenticationLeaving…The access point went offline, deauthenticating the client.
4…disassociationDueToInactivity…Client session timeout exceeded.
5…disassociationAPBusy…The access point is busy, performing load balancing, for example.
6…class2FrameFromNonAuthStation…Client attempted to transfer data before it was authenticated.
7…class2FrameFromNonAssStation…Client attempted to transfer data before it was associated.
8…disassociationStaHasLeft…Operating System moved the client to another access point using non-aggressive load balancing.
9…staReqAssociationWithoutAuth…Client not authorized yet, still attempting to associate with an access point.
99…missingReasonCode…Client momentarily in an unknown state.

Recivining and analyzing syslog messages from facility local3 on LMS4.2 soft appliance.

                   HI,
all of our enterprise switches are sert to send syslog messages from facility local3. this is partly because our linux syslog server loggs its boot syslog messages from facility local7 an we could't use the default facility of local7 on our cisco switches. LMS4.2s syslog daemon is set to recieve syslog messages from facility local7. how can i change it so that it can listen for facility local3 and also make sure the syloganalyzer and automated action work fine.
thanks,
Kerim

Hi All,
I thought it is a good idea to share the workaround my colleague came up with for this prolem. there is a file called syslog-entries.txt under /opt/CSCOpx/conf. he added all the entries we needed like :
local3.*     /var/log/syslog_info
local5.*   /var/log/syslog_info
the change was automatically reflected on syslog.conf
now we receve alerts from facilities 3 and 5 besides 7. hope this helps anyone who run into the same issue.

Ciscoworks syslog collector issue

Hi All,
In a central location i have a ciscoworks syslog collector version 3.5. The issue is not all the logs generated in the device are collected by ciscoworks including the devices connected in LAN. The major issue is on Cisco6500 series switches where i see multiple interface flaps in log but only few are found in syslog.
Regards,
Sathvik

Hi,
check here Admin > Collection Settings > Syslog > Syslog Collector Status , see if messages are falling under fitered or Invalid
then check the filter:
Admin > Network > Notification and Action Settings > Syslog Message Filters
I would suggest you to create a filter with all * and see if that helps.
you can look at this thread as well:
https://supportforums.cisco.com/thread/2244888?tstart=60
Thanks-
Afroz
[Do rate the useful post]

#5.3.4 message header size exceeds limit

Hi !
We are getting this bounce error message from our customers trying to send emails to our newly built C370 Ironport box.
Here is the error message;
"The following message to [email protected] was undeliverable.
The reason for the problem:
5.3.0 - Other mail system problem 552 - '#5.3.4 message header size exceeds limit'"
Hope is Delivery Satus Notification will help to identify what the problem is about.
Appreciate your kind response on how to fix this issue.
Best Regards,
Ruveni

Hi Ruveni,
Please check our following knowledge base article which explains this error and provide solution for it.
Message Bounces with "552 #5.3.4 message header size exceeds limit"
http://tinyurl.com/2yw579
Hope this helps!
Regards,
Viquar
Customer Support Engineer

RV110W excessive syslog messages

I bought a RV110W wireless router a couple months ago that I've been pretty happy with.
However, I have one significant problem with it. It is configured to send syslog messages to an internal server. Twice now it has gone into a mode where it starts dumping messages like,
ip_conntrack_is_ipc_allowed: ipc_entry_is_full
continuously, at a rate of about 20 per second. It otherwise seems to function normally, but of course if unnoticed my syslog file quickly grows to hundreds or thousands of megabytes. A reboot restores normal operation. It is running firmware 1.1.0.9. A search on the internet turned up no information about this problem.
It may be some corruption is occuring in the router's OS, or perhaps this is something that can be triggered externally (in which case it would be a weak form of DoS attack? Or maybe worse if in this state it is unable to properly apply the firewall rules.)
Looking for some hints on what might be wrong and how to fix.

I have also experieced the same issue. I did not reboot the Wireless Router but the logging has seemed to stop. I'm not sure what caused it either. I did clear the log and it has not been logging the error "ip_conntrack_is_ipc_allowed: ipc_entry_is_full".
I would like a response from Cisco on this error. How do we get Cisco to respond?

After WCS v4.1.91.0 upgrade: Strange SYSLOG message

Hi
After upgrading WCS to v 4.1.91.0 and Controller 4400 to v.1.185 I get these SYSLOG message:
Emergency <DATE> 1x_ptsm.c:419 DOT1X-1-MAXEAP_RETRANS_FOR_MOBILE: MAX EAP retransmissions reached for mobile <MAC>
Critical <DATE> iapp_socket_task.c:580 IAPP-3-MSGTAG015: iappSocketTask: iappRecvPkt returned error
Can?t find any info on cisco.com or any release notes. Anybody know what it means and what I can/should do?
TIA
Peter

Actually, Maximum EAP Retransmissions message indicates that EAPOL key retransmission to client has failed. Increase the no of error count for failure. But to further trace down the issue, we need a complete syslog output to which this MAX EAP retransmission message is associated with.
Check whether AAA server is UP and running(if external RADIUS server is used). What EAP authentication type you are using?. Let me know these details.

Syslog Message

Hi all,
In my firewall ASA 5540,Every day I am getting the syslog message.
4
Jul 07 2014
08:57:39
[ Scanning] drop rate-2 exceeded. Current burst rate is 0 per second, max configured rate is 8; Current average rate is 7 per second, max configured rate is 4; Cumulative total count is 28683
Please explain about above mentioned syslog.

Hi Kabeer,
That is because of the threat detection value set on your ASA. This might be an attack.
Because of the scanning rate configured and the
threat-detection rate scanning-rate 3600
average-rate 15
command:
%ASA-4-733100: [144.60.88.2] drop rate-2 exceeded. Current burst rate is 0 per
second, max configured rate is 8; Current average rate is 5 per second, max
configured rate is 4; Cumulative total count is 38086
Recommended Action
Perform the following steps
according to the specified
object type that appears
in the message:
1.
If the object in the message is one of the following:
Firewall
Bad pkts
Rate limit
DoS attck
ACL drop
Conn limit
ICMP attck
Scanning
SYN attck
Inspect
Interface
Check whether the drop rate is ac
ceptable for the running environment.
2.
Adjust the threshold rate of the particular drop to an appropriate value by using the
threat-detection rate
xxx command, where
xxx
is one of the following:
acl-drop
bad-packet-drop
conn-limit-drop
dos-drop
fw-drop
icmp-drop
inspect-drop
interface-drop
scanning-threat
syn-attack
3.
If the object in the message is a TCP or UDP port
, an IP address, or a
host drop, check whether
or not the drop rate is accepta
ble for the running environment.
4.
Adjust the threshold rate of the particular drop to an appropriate value by using the
threat-detection rate bad-packet-drop
command.
Note
If you do not want the drop rate exceed warning to appear, you can disable it by using
the
no threat-detection basic-threat command.
You can refer the below mentioned cisco document for more information.
http://www.cisco.com/c/en/us/td/docs/security/asa/syslog-guide/syslogs.pdf
Regards
Karthik

/Sbin/lilo -v error message(new kernel ..problem moot)

The message:
Setup length exceeds 31 maximum
Kernel setup will override boot loader.
Please advise as to what "setup length" is and I may get the problem solved.

This is the extent of the lilo entries which aren't that many. I assume there is an error in the typing somewhere or my boot file is messed up.
# /etc/lilo.conf
boot=/dev/discs/disc0/disc append="devfs=nomount"
# This line often fixes L40 errors on bootup
# disk=/dev/discs/disc0/disc bios=0x80
default= Linux-2.6.10-2
timeout=50
lba32
prompt
image=/boot/vmlinuz26
label=Linux-2.6.10-2
root=/dev/discs/disc0/part3
read-only
image=/boot/vmlinuz26a
label=Custom
root=/dev/discs/disc0/part3
read-only
image=/boot/vmlinuz
label=arch
root=/dev/discs/disc0/part3
read-only
#other=/dev/discs/disc0/part1
# label=dos
# End of file
This fails the /sbin/lilo -v run.

Can't get syslog messages from Remote SA520 over VPN

I'm trying to set up a central logging server on a debian system running rsyslog.
The syslog server is local & I have a branch office connected via a VPN. Both buildings have SA520 routers.
I have set up both firewalls to allow ANY from each network 192.168.150.X & 19.168.160.X
(also tried to add a rule for UDP514 but that didn't help)
The debian system is new & has no iptables set up
I've entered the syslog server IP in remote logging.
I've set up facilities in Send to syslog for both routers.
I am logging messages from the local router but don't see anything from the remote.
I've checked with wireshark & see no syslog packages from the remote (I do see SSL negotiation & others when using the web admin and of course the functioning vpn)
I rebooted the router to see if that mae a difference but no luck.
Any ideas why I can't get the syslog traffic across the VPN?

I do have the correct IP address of the syslog server set up. I do not want email logs so have not enabled that.
My setup is
remote lan > SA520-remote (192.168.160.1) > [ site to site IPSec VPN over WAN ] > SA520-local (192.168.150.1) > syslog server (192.168.150.25) & local lan
Firewall is set up to allow ANY IN & OUT to local lan on both routers.
I have also set up specific rules for UDP 514 Syslog traffic (no difference, currently disabled)
syslog server has -no- firewall at the moment.
Syslog server is receiving messages from the local router with no issues.
Log Severity is set to Information & Log Facility is set up to send to Syslog.
I have also setup a SNMP trap on the syslog server & pointed the remote router to it in hopes of diagnosing the issue.
Both routers have the latest firmware applied.
Using wireshark on the syslog server I see no traffic on UDP 514 (syslog) or UDP 162 (snmp)
I can use the WUI for the remote & ping the 160.1 with no problem. Both ping & TLS/TCP traffic show up in wireshark on the syslog server when I do so.
It looks to me like there is a problem routing the syslog messages out of the router & then back through the VPN.
Worst case I'll set up another syslog server on an old machine at the remote location & then cron the logs to the central syslog server but it really seems I shouldn't have to.

Cisco MARS Syslog messages

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman";
mso-ansi-language:#0400;
mso-fareast-language:#0400;
mso-bidi-language:#0400;}
Hi,
I've recently noticed that ALL the syslog messages that are sent to our Cisco MARS device are then being sent to our syslog server. Besides the messages from our MARS device, the syslog server also gets the original syslog messages from our ASA and PIX firewalls (which, of course, also send to our MARS device). I would like to have MARS send syslog messages to the syslog server that pertain only to changes/events happening directly to the MARS device. Can anyone help me with this?
Thanks in advance!

Kerry;
To have CS-MARS specific incidents forward to your syslog server, you will most likely want to add an action to generate a syslog for the CS-MARS-specific inspection rules. These rules can be found by navigatng to:
RULES>Inspection Rules
from the Group: drop-down choose "System: CS-MARS Issues"
You can then edit the Action: section for the specific rules (one at a time) to add a syslog action. Specifics are outlined here:
http://www.cisco.com/en/US/docs/security/security_management/cs-mars/6.0/user/guide/combo/alerts.html
Scott

CUCM Syslog Message ISSUE (kernel: Exceeded hashlimit)

Similar Messages

Maybe you are looking for