What is "Source ID" in Netflow V9 Packet Header

Question

Hi,
My question is regarding the "Source ID" field that appears in Netflow V.9 packet header. Following Cisco link (http://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.pdf) gives Source ID definition as -
"The Source ID field is a 32-bit value that is used to guarantee uniqueness for all flows exported from a particular device. (The Source ID field is the equivalent of the engine type and engine ID fields found in the NetFlow Version 5 and Version 8 headers). The format of this field is vendor specific. In the Cisco implementation, the first two bytes are reserved for future expansion, and will always be zero. Byte 3 provides uniqueness with respect to the routing engine on the exporting device. Byte 4 provides uniqueness with respect to the particular line card or Versatile Interface Processor on the exporting device."
I am using "Source ID" (combined with template id) to uniquely identify options templates exported by different routers. At our new lab setup where we have more than one routers configured to export Netflow, I observed that all the routers were exporting "Source ID" value as "0"(zero). It failed my assumption that I had formed based on definition from above Cisco doc.
I assumed -
SourceID    Template Id  Unique Key
source1       256              source1-256
source1       257              source1-257
source2       256              source2-256
source3       258              source3-258
But, I observed
SourceID    Template Id  Unique Key
0                  256              0-256
0                  257              0-257
0                  256              0-256
0                  258              0-258
Thus, same template id(256) from different routers(source1, source3) eventually form same unique key and breaks my code.
I would like to know if my interpretation that Source ID can be used to uniquely identify templates in this manner is correct or not ? 
Is "Source ID" user configurable attribute ? How does it comply to the definition given in above Cisco doc ?
Thanks,
Deepak

Answer 1

Deepak,
Consider these quotations from the same RFC 3954:
Section 2: Terminology:
Observation Point
An Observation Point is a location in the network where IP packets
can be observed; for example, one or a set of interfaces on a network
device like a router. Every Observation Point is associated with an
Observation Domain.
Observation Domain
The set of Observation Points that is the largest aggregatable set of
flow information at the network device with NetFlow services enabled
is termed an Observation Domain. For example, a router line card
composed of several interfaces with each interface being an
Observation Point.
Section 7: Template Management:
A NetFlow Collector that receives Export Packets from several
Observation Domains from the same Exporter MUST be aware that the
uniqueness of the Template ID is not guaranteed across Observation
Domains.
Section 9: The Collector Side:
At any given time the Collector SHOULD maintain the following for all
the current Template Records and Options Template Records: Exporter,
Observation Domain, Template ID, Template Definition, Last Received.
Note that the Observation Domain is identified by the Source ID field
from the Export Packet.
So in other words, the Source ID is an identifier of the Observation Domain (and in fact, the IPFIX RFC calls this header field directly as Observation Domain ID). Template IDs are unique per Exporter and per Observation Domain, and if a single Exporter uses multiple templates in its different Observation Domains, the IDs of these templates could overlap even in a single Exporter. Observation Domain IDs (that is, Source IDs) identify only the internal structure of a single Exporter, and no provisions are done to preserve their uniqueness across multiple Exporters - for this, the source IP shall be used.
With respect to whether there can be multiple NetFlow instances on a single router, I am getting a feeling that with decentralized, distributed platforms, multiple linecards in a single router could run their own NetFlow analysis for data that pass through them, so each one provides a separate NetFlow collection. Thus, each linecard or each feature card doing its own NetFlow analysis should be assigned its own unique Observation Domain ID.
If it is not user configurable then system should automatically form the value based on router engine and line card. But what I have observed, at more than one routers, is that this value is always 0(zero).
I believe this is strongly dependent to the hardware construction of the router. As a remotely-related example, old 2600 series routers had two WIC slots. If you inserted two WIC-2T modules into these slots, you'd expect that they would be numbered Serial0/0, Serial0/1, Serial1/0, Serial1/1. Very surprisingly, however, these routers considered both slots to be internally connected to a single bus, and the interfaces were named Serial0/0, Serial0/1, Serial0/2 and Serial0/3 - as if they all were installed in a single slot '0'. Something similar may happen to the Observation Domains and their IDs. You would believe that each single linecard constituted a separate Observation Domain. However, the reality may be different, and the whole router can act as a single Observation Domain to the outside world. It's just the way it is constructed - and programmed.
It is not clear why Cisco doc says that one should use both "Source ID" and "Source IP Address" to properly distinguish between flows.
I think it's a poor wording in the RFC. I think what they want to say is that if you use the duplet <Source IP, Source ID> to distinguish between flows, then you're fine both for multiple flows from the same Exporter, and for multiple flows from different Exporters.
Moreover, isn't "Source IP Address" good enough to distinguish between flows from different sources ?
If an Exporter could truly be partitioned into multiple Observation Domains then the source IP would not be sufficient. I am just making up examples with no real-life backup here, but think of, say, a multi-chassis router with each chassis being one Observation Domain, or each linecard of a distributed switch being a standalone Observation Domain, or one router virtualized to several different contexts and virtual routers, each of them being a unique Observation Domain, reporting about the flows using the same source IP... I think you get the point.
I would put it this way... The existence of Source ID in NetFlow v9 (and Observation Domain ID in IPFIX) allows these protocols to nicely cope with situations in which a single physical device can be partitioned into several Observation Domains and perform independent reporting on them using a single source IP. However, the fact that these protocols have this ability does not mean that each and every device, even a Cisco router/switch, must necessarily make use of it.
Best regards,
Peter