Error disabled. Reason:DCX-No ACK in 100 PDUs

Question

Hi,
  I have a customer who lost all connectivity from the ESX host for both networking and FCoE because (as the title suggests) the interfaces were error disabled.  This happened across all 8, dual ported, dual homed CNAs at the same time.  Does anyone have any idea what this error comes from?  The are using ESX 4.0 and are running Nexus 5020 with 4.2(1)N2(1a).
Thanks,
Thom

Answer 1

DCBX Type Length Values(TLV) are packaged within a LLDP frame which  is exchanged between the switch and the CNA. One such Control Sub-TLV is  used for ACK which is sequence based. For example, the switch sends  this control Sub-TLV with SeqNo of 1 and AckNo of 2. The host is  supposed to inverse this and send a LLDP frame with control sub-TLV with  SeqNo of 2 and AckNo of 1.
We expect this exchange every 30 seconds from the host and if the  switch does not see it for 100 times 30 which is 3000 seconds (or 50  minutes), the switch error disables with following error
2011 May 13 12:03:23 CSX_5020_A1 %ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet115/1/17 is down (Error disabled. Reason:DCX-No ACK in 100 PDUs)
2011 May 13 12:03:27 CSX_5020_A1 %ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet116/1/16 is down (Error disabled. Reason:DCX-No ACK in 100 PDUs)
Some commands on the switch which helps in narrowing down root cause.
F340.24.10-5548-1# show lldp interface ethernet 1/22
Interface Information:
  Enable (tx/rx/dcbx): Y/Y/Y    Port Mac address: 00:05:73:ab:29:bd
Peer's LLDP TLVs:
Type Length Value
001  007    040000c9 9d2372
002  007    030000c9 9d2372
003  002    0078
006  045    456d756c 6578204f 6e65436f 6e6e6563 74203130 4762204d 756c7469
            2066756e 6374696f 6e204164 61707465 72
007  004    00800080
127  055    001b2102 020a0000 00000002 00000001 04110000 c0000001 00003232
            00000000 00000206 060000c0 00080808 0a0000c0 00890600 1b2108
000  000   
F340.24.10-5548-1# show lldp dcbx interface ethernet 1/22
Local DCBXP Control information:
Operation version: 00  Max version: 00  Seq no: 1  Ack no: 2  <<---Our sequence # and Ack #
Type/
Subtype    Version    En/Will/Adv Config
003/000     000        Y/N/Y      0808
004/000     000        Y/N/Y      8906001b21 08
002/000     000        Y/N/Y      0001000032 32000000 00000002
Peer's DCBXP Control information:
Operation version: 00  Max version: 00  Seq no: 2  Ack no: 1  <<---Peer sequence # and Ack # should be reversed.
Type/      Max/Oper
Subtype    Version    En/Will/Err Config
002/000     000/000    Y/Y/N      0001000032 32000000 00000002
003/000     000/000    Y/Y/N      0808
004/000     000/000    Y/Y/N      8906001b21 08
F340.24.10-5548-1#
Root cause for this problem in most cases is misbehaving CNA/server or incorrect firmware/driver on the CNA.