Where is hardware dCEF's adjacency table stored?

Question

For hardware dCEF, based on some documents on the website, at the ingress line card/interface, conceptually it looks up the FIB with dst IP address, and gets a pointer to an adjacency table entry, where L2 rewrite information is stored, e.g. nexthop mac, etc.
But what confuses me is that, doesn't the L2 rewrite happen on egress line card/interface? If so, then why is this adjacency table with L2 rewrite info stored on ingress? Or where is the adjacency table look up happening? ingress or egress? If this is on ingress, are the L2 rewrite information carried over from ingress card to egress line card? Wouldn't that be a waste of fabric bandwidth?

Answer 1

Hello,
It depends on the platform and its operating system...
In the old days, layer 2 protocols like ARP or PPP/HDLC were running on the central RP or RSP. So it was indeed possible for the RP/RSP to distribute the adjacency table to all linecards and have an ingress linecard append the adjacency rewrite. A few years ago, some types of linecard did not have a forwarding ASIC in the egress direction so it was up to the ingress linecard to perform the adjacency rewrite. But some other platforms also had the ingress LC insert an adjacency index in the buffer header so that the egress linecard was doing a lookup on the adjacency index to identify the CEF adjacency.
If we take IOS XR which is running on modern routers like the NCS6000, CRS and ASR9000 , there is a new concept called "2 stage forwarding". For scalability reasons, layer 2 protocol handling has been distributed to the linecard CPUs. So it's the CPU on each linecard which will handle the layer 2 protocol for the local interfaces connected to that linecard. This also means that the adjacency table built from the ARP table for instance will only be known locally on that linecard. There is a first CEF lookup on the ingress forwarding ASIC on the ingress linecard and the result points at what is called a "remote adjacency". A remote adjacency provides enough information to reach the remote interface on the egress linecard: what is the local virtual output queue on the ingress linecard, what is the fabric destination address (egress network processor)... but not the layer 2 rewrite on the egress interface as this is not known on the ingress linecard. Once the packet reaches the egress forwarding ASIC on the egress linecard, there is a second CEF lookup (hence the name "2 stage forwarding") where the local CEF adjacency can be identified and this adjacency contains the MAC rewrite built from the layer 2 protocol running locally on that linecard.
Hope this helps,
JC