QoS in MPLS Packet drops
Hello
I have experienced the following situation recently. It hasn't fixed yet. If you have any idea, please advise. Any comments are welcome!
The problem was that the voice packets have been dropped in class Voice despite the fact that the interface's utilization is quite low. Please see the policy-map interface output below.
Service-policy output: out_(3072k/3072k)_QoS (4754)
queue stats for all priority classes:
queue limit 192 (packets)
(queue depth/total drops/no-buffer drops) 0/297/0
(pkts queued/bytes queued) 12372/1041424
Class-map: Voice (match-all) (4755/7)
12098 packets, 1042365 bytes
5 minute offered rate 32000 bps, drop rate 3000 bps
Match: ip precedence 5 (4756)
Priority 50 (%) (1536 kbps) burst 1250000 (bytes)
I removed the service-module from the multilink interface and restored. I repeated the procedure for a few times. At some point, I noticed that queue-limit of LLQ changed from 192 to 384. I didn't change it manually. It was changed by itself. Then the drop stopped.
Service-policy output: out_(3072k/3072k)_QoS (6339)
queue stats for all priority classes:
queue limit 384 (packets)
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts queued/bytes queued) 2628/221321
Class-map: Voice (match-all) (6340/7)
2653 packets, 228634 bytes
5 minute offered rate 14000 bps, drop rate 0 bps
Match: ip precedence 5 (6341)
Priority 50 (%) (1536 kbps) burst 1250000 (bytes)
* I omitted some of the output due to the text size in this forum.
The average packet sizes in queue-limit 192 and 384 are almost same (86 bytes/packet).
Does anyone know how does the IOS calculate the queue-limit in LLQ? I know that it changes dynamic in LLQ but would like to know the algorithm.
Did anyone have a similar experience?
Thank you,
Hello Creed,
>> I just think that the P router might not able to police the traffics as it get routed into the tunnels...
With both LDP LSPs and MPLS TE LSPs traffic is forwarded inside MPLS frames and so EXP field of the topmost label is accessible to P routers to provide diffserv QoS treatments.
Usually P routers implement only outgoing scheduling (queueing) combined with congestion avoidance (WRED).
The service policies are applied outbound the physical interfaces.
Rate control of what traffic enters the MPLS cloud both LDP LSPs and MPLS TE LSPs is made on PE at the edge before traffic enters the MPLS space.
DSCP bits should not be accessible inside the MPLS cloud once the ip packet travel inside an MPLS label stack.
Actually some inside inspections can be performed for flow based load-balancing purposes.
So policing on the P routers is not used in general.
Hope to help
Giuseppe
Similar Messages
-
Monitoring dscp ef packet drops
Looking for some guidance please.
I have been tasked by our network team to find a solution to monitor voice traffic specifically for packet drops in dscp ef traffic.
Thinking of using my cacti box as my first port of call but need to know exactly which OIDs i need to be pulling in. I have looked at the various mib sets related to qos cos etc.... but to be honest, they are bit daunting for someone who is not familiar in this area.
Any other options for this would be greatly appreciated - could rmon fulfill this task?
cheersYou can troubleshoot the output drops occuring with priority queuing be following the sugesstions made in http://www.cisco.com/en/US/tech/tk39/tk51/technologies_tech_note09186a0080103e8a.shtml
-
Output packets drops on Router interface
Hi,
As shown in attached diagram we have connected DC and site with two ISP.
RTR1 and RTR2 are in HSRP mode and RTR2 is active router for site
We are seeing packet drops on DC RTR2 interfece connected to Sprint MPLS.
We have a site with two isp (Sprint MPLS and P2P link). Site is running on Sprint ISP.
We are seeing out put errors on DC RTR2 Sprint interface, when we failover site to P2P link from sprint.
And when fail over back to Sprint at site we are not seeing any output packets drops on DC RTR2 sprint link.
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 36/255, rxload 12/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is RJ45
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 1w0d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 82137
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 4722000 bits/sec, 1449 packets/sec
5 minute output rate 14469000 bits/sec, 2088 packets/sec
601690470 packets input, 1836920253 bytes, 0 no buffer
Received 134822 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 134790 multicast, 0 pause input
0 input packets with dribble condition detected
858756749 packets output, 2658785694 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped outSh int
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 36/255, rxload 12/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is RJ45
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 1w0d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 82137
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 4722000 bits/sec, 1449 packets/sec
5 minute output rate 14469000 bits/sec, 2088 packets/sec
601690470 packets input, 1836920253 bytes, 0 no buffer
Received 134822 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 134790 multicast, 0 pause input
0 input packets with dribble condition detected
858756749 packets output, 2658785694 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out -
Hi,
Our customer took 30Mbps metro link from us. Even at 17Mbps link utilization they are facing packet drops. Our side policer is implemented for 30Mbps. There are no errors on customer or our interfaces. But I can see exceed packets under 'show policy-map interface' . Used maximum Bc. Does customer required to implement shaping his end with same CIR and Bc.
Regards
Siva KHi Siva,
It is not a mandatory rule that Customer should also have the same CIR configured with shaping.
When Customer have 30 Mbps circuit SLA with Service Provider, he may be able to pump ta line rate from CE side. But on PE side, it will be policied and excess traffic will be dropped.
To avoid Customer's traffic getting dropped at PE, It is advisible to configure shaping at CE side so that the traffic SLA will be maintained without or with less number of packet loss.
Can you post your config and show policy-map interface output with traffic?.
Regards,
Nagendra -
Packet drop when clients moving from one Access point to another
HI All ,
I am new to wireless . I am using WS-SVC-WISM-1-K9 wism module and using 5 Access points . When my clients are moving from one access point to another we are getting packet drops .
Kindly anyone suggest me what all configuration i need to verify on the controller for Proper client roaming so that i can resolve my issues..
Please let me know in case of any explanations requiered .
Thanks in Advance !!!
Regards
AngusFor radius authenticated SSIDs, you need WPA2-aes or wpa1-tkip-CCKM. It depends on what the client supports.
For pre-shared key, any WPA should be decent enough for roaming speed.
If you're on WEP ... no comment.
If you covered the above point, check if it's not a coverage problem. If the 2 APs coverage zone are not overlapping there will be a hole where you don't have signal and logically will have packet drops. -
Wireless AP 1262 getting packet drops whille buffering videos for 18 users.
Hi Team,
Please help for this issue
We are having 1262 Access point model and we are getting packet drops when 20 users are connected and users do Video streaming and buffering online.
Even our AD IP address also getting packet drops during the users are connected and using youtube or someother video sites.
Please help on this issue.
Best regards,
ArunWell if you have 802.11n enabled and also have 802.11n capable devices, then you would have max of 144mbps on the 2.4ghz and up to 300mbps on the 5ghz with 40 MHz channels. If you are using 20mhz on the 5ghz you will have the same as the 2.4ghz which is again 144mbps.
So if you have clients working fine on the 5ghz and its set to 20mhz, then I would look at interference on the 2.4ghz. See if your SNR is low as that will identify a poor 2.4ghz spectrum.
Sent from Cisco Technical Support iPhone App -
Best Practice QOS in MPLS core
Hi.
Wath is the recommendation in configuring QOS in the core mpls (PE-P)
we have a mixture of traffic of data, cell signalling, O&M, voice R4, IVR.etc. We have 7600 as PE and 12000 as P. The PE's have SIP-GE and WS-GE cards against the P. In some cases the traffic goes through two P's connected via SIP-GE cards. I Had seen that the WS cards have some special considerations about QOS.Hi,
Here is a good starting point regarding the best practices:
http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND/VPNQoS.html
The how to configure QoS for MPLS on 7600 :
WS cards: http://www.cisco.com/en/US/docs/routers/7600/ios/12.2SR/configuration/guide/mplsqos.html
SIP cards: http://www.cisco.com/en/US/docs/interfaces_modules/shared_port_adapters/configuration/7600series/76cfgsip.html#wp1162382
HTH
Laurent. -
N7000 : details of packets dropped by COPP policy (class-default) ?
Hi,
On one of our N7K, we have some packets dropped by the COPP policy in the class-default class-map. :
Partial results of "show policy-map interface control-plane" not so long after clearing the counters :
class-map class-default (match-any)
set cos 0
police cir 100 kbps , bc 250 ms
module 1 :
conformed 12210790 bytes; action: transmit
violated 201870 bytes; action: drop
module 2 :
conformed 8399646 bytes; action: transmit
violated 0 bytes; action: drop
module 3 :
conformed 34518233 bytes; action: transmit
violated 6186895 bytes; action: drop
What would be the best way to figure out what traffic is dropped by the policy ? Is there any logging possible ?
Thanks,
LaurentThere is still no logging possible.
What can be done is piping the class-default-traffic to some port and then analyze it with wireshark or some similar tool. But as far as I know, this still cannot be done by default - at least with NX-OS 4.2(4) we had to reprogram the module with assistance from TAC. I suggest you contact your support partner in this matter. -
EEM -automatic shut down or switch over of WAN link in OSPF when packet drop increase
Hi,
Need help..
can any one help me how can EEM help for automatic shut down or switch over of WAN link in OSPF when packet drop increase a predefined level.
I have a set up different branches connected together...OSPF is the routing protocol and need to communicate with two branches via hub locations.
need to shut or switch some percent of traffic from primary to back up when packet drop in the link.I am not sure EEM can do what you want.
Another option could be to use SLA tacking/monitoring. But you will fall back to the new route when you lose some percentage of pings, you can't switch only part of the traffic.
I hope it helps.
PK -
Signature 1330 causes packet drops
Hello Members,
i see in my IPS-NME module a hign number of packet drops because of the following signatures:
1330-17: TCP segment out of state order
1330-12: TCP segment is out of order.
the targets and the attacers are internal hosts.
are these signatures triggered because of not propper configured policies or is this an indicator for problems in the internal network.
thanks for your inputs.
regards
alexHello Sid,
thanks for your answer. I learned that most of packets where the Signature 1330 triggers are packets from the IPS module to the IPS Express Manager. I added wireshark dump to the case.
That's really odd, i ran a traceroute from the IPS Manager to the IPS Module and vice versa and the flow look ok to me.
Trace from the IPS module to the IPS Manager
# trace 10.0.128.5
traceroute to 10.0.128.5 (10.0.128.5), 4 hops max, 40 byte packets
1 172.16.1.9 (172.16.1.9) 1.479 ms 1.327 ms 1.275 ms
2 172.16.1.1 (172.16.1.1) 3.616 ms 2.952 ms 1.907 ms
3 10.89.27.10 (10.89.27.10) 2.288 ms 2.044 ms 2.136 ms
4 10.89.27.21 (10.89.27.21) 8.106 ms 9.148 ms 8.266 ms
return path
C:\Users\Administrator.NOS-POC>tracert 172.16.1.11
Tracing route to 172.16.1.11 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 10.0.128.1
2 2 ms 3 ms 2 ms 172.16.2.1
3 1 ms 1 ms 1 ms 10.89.27.22
4 9 ms 9 ms 9 ms 10.89.27.9
5 8 ms 8 ms 8 ms 172.16.1.6
6 8 ms 8 ms 8 ms 172.16.1.11
Trace complete.
trace from the IPS module's gateway
#traceroute vrf CENTRAL 10.0.128.5 source 172.16.1.9
Type escape sequence to abort.
Tracing the route to 10.0.128.5
1 172.16.1.1 0 msec 0 msec 0 msec
2 10.89.27.10 0 msec 0 msec 4 msec
3 10.89.27.21 8 msec 8 msec 8 msec
4 172.16.2.6 8 msec 8 msec 4 msec
5 10.0.128.5 4 msec 4 msec 4 msec
what make me wonder is that the IPS module doesn't show hops further than 4 hops.
regards
alex -
Can you carry L3VPN MPLS packets over Ethernet XConnect?
Hi All,
Can you carry L3VPN MPLS packets over an ethernet port-based xconnect???
Current:
POP_1 >> Physical Circuit << POP_2
Proposed:
POP_1 >> Provider_Router_1 << ethernet port-based xconnect >> Provider_Router_2 << POP_2
We are cancelling our physical circuits between each POP and going with another provider who is going to carry all our traffic between the POPS using ethernet xconnects. We have a few L3VPN MPLS customers and I wasn't 100% sure if their L3VPN data would be carried over the proposed xconnects.
Thanks.
AndyThanks guys for your reply...
One thing that I've never fully understood is the MTU setting you need within the Service Provider's core network (and I've read quite a bit about it).
For example we've cut across to the new xconnect circuit last night and I can get a 1500 byte L3VPN MPLS packet through the Service Provider's core from one PE to another PE via the xconnect (see below). I think this is made up of the payload (1492 bytes) + 2 x Tunnel/VC headers (8 bytes) = 1500 bytes total - so the Service Provider's core has MTU of 1500 bytes (correct me if I'm wrong on this).
Now I don't know if this is good or bad??? What should I be looking for? How do I determine what MTU is required through the Service Provider's core???
PE_1#ping vrf NSTEST 172.16.100.17 size 1492 df-bit
Type escape sequence to abort.
Sending 5, 1492-byte ICMP Echos to 172.16.100.17, timeout is 2 seconds:
Packet sent with the DF bit set
Success rate is 100 percent (5/5), round-trip min/avg/max = 24/25/28 ms
PE_1#ping vrf NSTEST 172.16.100.17 size 1493 df-bit
Type escape sequence to abort.
Sending 5, 1493-byte ICMP Echos to 172.16.100.17, timeout is 2 seconds:
Packet sent with the DF bit set
Thanks.
Andy -
Packet drops on v490 production server..help us
Hello...
We have v490 server with ce0 interface configured.. It gets down frequently & after some packet drops it makes itself up...
Can anybody tell me what could be the reason behind this problem...
I have checked switch & router by changing interface cables, still problem persists...no message on /var/adm/messages.
Thanks in advance
gmrajtry a "snoop -d ce0" and verify messages
also, perhaps the NIC is broken
also, perhaps the duplex/speed of the NIC isn't set correctly (autoneg, forced, fullduplex, halfduplex etc.) and you have to define it with a "ndd -set " -
High packet drop over FCoE setup
We have nexus 5k switch connected to storage array through FCoE 10GB interface and with blade chasse support FCoE. We are facing a hug latency on the traffic flow between the server and the storage. Can some one help me to solve this issue? Also do we need to setup the jumbo frame and modify the MTU size?
Sent from Cisco Technical Support iPad AppAymen,
MTU should not be an issue. No need to modify the MTU for regular ethernet traffic, unless you're using IP storage such as iSCSI.
Let's narrow down the problem first.
1. Do you see packet loss/performance issues on other servers connected to the same N5K(s)?
2. Are you seeing any packet drops on the N5K interfaces or GATOs ASIC?
show interface e1/20 counters errors
show interface e1/20 flowcontrol
show interface e1/20 priority-flow-control
show system internal ethpm errors | egrep Ethernet1/20
show hardware internal gatos port ethernet 1/20| egrep -i err
I would check these counters on both the host facing and arrary facing interfaces.
3. What is the exact array that is FCoE attached?
4. Do you have a topology diagram?
5. What are the server side adapters, firmware and driver versions being used (include the OS on the host).
Regads,
Robert -
Hi,
i want to ask. My ASA5520 is generating some packet drops constantly and we have some problems with server aplication that a proccessing of tasks from client to server take a long time (sometime about 15 seconds). Our client application is accessing a server throught IPSec VPN tunnel terminated on two ASA`s. Our connectivity is about 20Mbit/s to internet and responses to ping about 5 ms and our internet load is about 20% on both sides - so i think this parameters are not bad. MTU is configured for 1500 for all interfaces. If this apllication is on local network its is working with no problems. Long responses are only throught VPN tunnel.
Can someone help me where to search for possible reasons? - is a drop rate about 2-4pkts/sec a normal behavior on Outside and Inside interface?
Outside:
received (in 3089.110 secs):
1440158 packets 1318512125 bytes
466 pkts/sec 426825 bytes/sec
transmitted (in 3089.110 secs):
1189541 packets 449651676 bytes
385 pkts/sec 145560 bytes/sec
1 minute input rate 660 pkts/sec, 569735 bytes/sec
1 minute output rate 543 pkts/sec, 194757 bytes/sec
1 minute drop rate, 2 pkts/sec
5 minute input rate 541 pkts/sec, 494752 bytes/sec
5 minute output rate 418 pkts/sec, 115924 bytes/sec
5 minute drop rate, 2 pkts/sec
Inside:
received (in 998799.294 secs):
1207809993 packets 733339825912 bytes
1002 pkts/sec 734002 bytes/sec
transmitted (in 998799.294 secs):
1200125098 packets 882901742659 bytes
1003 pkts/sec 883004 bytes/sec
1 minute input rate 502 pkts/sec, 179984 bytes/sec
1 minute output rate 614 pkts/sec, 564726 bytes/sec
1 minute drop rate, 4 pkts/sec
5 minute input rate 391 pkts/sec, 108899 bytes/sec
5 minute output rate 508 pkts/sec, 490840 bytes/sec
5 minute drop rate, 4 pkts/sec
DMZ:
received (in 998799.984 secs):
58298524 packets 44825759311 bytes
2 pkts/sec 44002 bytes/sec
transmitted (in 998799.984 secs):
46530732 packets 12940381278 bytes
3 pkts/sec 12001 bytes/sec
1 minute input rate 53 pkts/sec, 13049 bytes/sec
1 minute output rate 49 pkts/sec, 3004 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 36 pkts/sec, 5570 bytes/sec
5 minute output rate 33 pkts/sec, 1755 bytes/sec
5 minute drop rate, 0 pkts/sec
Aggregated Traffic on Physical Interface
GigabitEthernet0/0:
received (in 3089.870 secs):
1440885 packets 1346005546 bytes
466 pkts/sec 435618 bytes/sec
transmitted (in 3089.870 secs):
1190187 packets 474475065 bytes
385 pkts/sec 153558 bytes/sec
1 minute input rate 660 pkts/sec, 582256 bytes/sec
1 minute output rate 543 pkts/sec, 206077 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 541 pkts/sec, 504955 bytes/sec
5 minute output rate 418 pkts/sec, 124804 bytes/sec
5 minute drop rate, 0 pkts/sec
GigabitEthernet0/1:
received (in 998800.164 secs):
1207813930 packets 757321051733 bytes
1002 pkts/sec 758002 bytes/sec
transmitted (in 998800.164 secs):
1200125732 packets 906238831947 bytes
1003 pkts/sec 907000 bytes/sec
1 minute input rate 502 pkts/sec, 190546 bytes/sec
1 minute output rate 614 pkts/sec, 576442 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 391 pkts/sec, 117300 bytes/sec
5 minute output rate 508 pkts/sec, 500487 bytes/sec
5 minute drop rate, 0 pkts/sec
GigabitEthernet0/2:
received (in 998800.224 secs):
58298526 packets 45904344202 bytes
2 pkts/sec 45000 bytes/sec
transmitted (in 998800.224 secs):
46530733 packets 13855555976 bytes
3 pkts/sec 13003 bytes/sec
1 minute input rate 53 pkts/sec, 14097 bytes/sec
1 minute output rate 49 pkts/sec, 4018 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 36 pkts/sec, 6271 bytes/sec
5 minute output rate 33 pkts/sec, 2437 bytes/sec
5 minute drop rate, 0 pkts/sec
GigabitEthernet0/3:
received (in 998800.364 secs):
0 packets 0 bytes
0 pkts/sec 0 bytes/sec
transmitted (in 998800.364 secs):
0 packets 0 bytes
0 pkts/sec 0 bytes/sec
1 minute input rate 0 pkts/sec, 0 bytes/sec
1 minute output rate 0 pkts/sec, 0 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 0 pkts/sec, 0 bytes/sec
5 minute output rate 0 pkts/sec, 0 bytes/sec
5 minute drop rate, 0 pkts/secHi,
There is no UDP flow limit configured on this firewall:
asa-hvac# sh local-host router-bacnet
Interface inside: 3 active, 8 maximum active, 0 denied
local host: ,
TCP flow count/limit = 0/unlimited
TCP embryonic count to host = 0
TCP intercept watermark = unlimited
UDP flow count/limit = 2/unlimited
Conn:
UDP out ctrl-delta-maniwaki:47808 in router-bacnet:47808 idle 0:00:15 flags -
UDP out ctrl-delta-laurentienne:47808 in router-bacnet:47808 idle 0:00:00 flags -
Interface outside: 15 active, 33 maximum active, 0 denied
To answer your second question, when the problem appear, there is the same 2 flows when I issue the "show local-host bacnet-router" command. -
Hi,
MY ASA5540 has 40 L2L IPsec VPN tunnels to other sites. One of tunnels has packet drop often ( but the tunnel remind up ). Called ISP and confirm its not ISP issue. Is there any method to troubleshoot the issue ? what should I look at in the configuration ? any help will be appericated.
ThanksVerify that ACLs and NAT configurations are Correct. Some times it may also block the traffic.
Most Common L2L and Remote Access IPSec VPN Troubleshooting Solutions
http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a00807e0aca.shtml
Maybe you are looking for
-
How to get sso_userid from Database
Hi. I use the SSO in my forms application 10g, and All the Users are connected to Data Base with only one resource conexion, and I want to know the SSO_USERID from a Trigger in the database to insert in a log table. I want to chance de "USER VARIABLE
-
How to create a pdf file with CS5
Hello, I'm new to PhotoShop CS5 and haven't figured out yet (despite two hours of trying) how to create a pdf file with pictures and texts. Can someone please help me with this ? The "help" button in CS5 doesn't seem to cover this question. Nor do
-
My movie downloads from itunes take up to 3 hours even with a 2.8 m/s DSL internet connection. Any suggestions as to why?
-
My scanner does not work since mountain lion
Since I installed Mountain Lion my scanner does not work anymore. I have a Konica Minolta Magicolor 1690MF and the printing works. There doesn't seem to be other drivers available at Konica Minolta.
-
How to modify audio output of my sequence
Hi. Let's say I have a sequence that has one video track and one mono audio track. 1) If the audio track is completely empty and I delete all empty tracks, Premiere refuses to delete Audio track 1. How do I delete the audio track, such that my seq