Re: [mpls] Concerns about ISD

[Haoyu Song <[e-mail]>]

Hi Tony,  please see inline comments with [HS2]

Best regards,
Haoyu

From: Tony Li <[e-mail]> On Behalf Of Tony Li
Sent: Friday, April 15, 2022 1:04 PM
To: Haoyu Song <[e-mail]>
Cc: John E Drake <jdrake=[e-mail]>; Tianran Zhou <zhoutianran=[e-mail]>; [e-mail]
Subject: Re: [mpls] Concerns about ISD


Hi Haoyu,



But if you check the cost of parsing ISD, this can still be a better choice.

Please explain that.  As previously discussed, parsing the full stack to find the bottom of the stack requires more read/load operations. That would seem to mandate poorer performance.

[HS] Assuming the label stack depth is L,  it needs L steps to reach the EH.  The question here is: what's the reasonable value of L?   The total parsing cost and latency also need to include the parsing of the AD items. If we include this into consideration, we would see PSD would still win overall. Please see pg. 7 of the slides for the analysis https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrac.ietf.org%2Ftrac%2Fmpls%2Fwiki%2F2022-01-27-agenda%23no1&data=05%7C01%7Chaoyu.song%40futurewei.com%7C0f1c419a6fba45b48e2908da1f1b1ff2%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637856498622812126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n9vdFVxMQS2gVOBgD9DMhSO5%2FKLAgSv2L%2FHINECufaw%3D&reserved=0>


You're assuming a TCAM and your only metric here seems to be the storage cost.  Many folks don't do parsing using a TCAM and the performance cost is in the reads, not in the storage.

[HS2] TCAM is actually the better choice to store states involving bitmaps. Otherwise, parsing bitmap requires extra calculations in addition to storage, making it even less favorable.

Moreover, in another draft https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.potaroo.net%2Fietf%2Fall-ids%2Fdraft-andersson-mpls-eh-label-stack-operations-03.txt&data=05%7C01%7Chaoyu.song%40futurewei.com%7C0f1c419a6fba45b48e2908da1f1b1ff2%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637856498622812126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GKad3KmRVlaZmhNeDXCxKth5FfKlgYDkfK%2FqtYFI%2BIo%3D&reserved=0> we describe a way to avoid unnecessary scan of the label stack with the help from the control plane.


If you're going to involve the control plane, then it would seem like Robert's proposal (or what I understand of it), would imply that you push EVERYTHING into the control plane.  Why bother with data plane encodings at all?

[HS2] We propose to use explicit FEC to tell there's no EHs in the packet to avoid the label stack scanning in such cases. This doesn't nullify the need to carry extra metadata for some user packets to support some in-network functions.

For the EH encoding efficiency, I appreciate your input. Currently it follows the IPv6 EH type (i.e., using NH + LENGTH to delineate every EH). There could certainly be room for improvement.


Ok.  If I understand your proposal correctly, there are 4 octets of overhead (HEH) at the start. Then, for each network action, there would seem to be at least 3 octets of overhead, plus any associated data, plus alignment.

Let's suppose that we want to encode NFFRR, entropy, and GISS in one packet.  By my math, the cost is:

EHI: 4 octets
HEH: 4 octets
NFFRR: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 28 octets

Do I have that right?

[HS] It's right if you think 8 octets are needed for Entropy and GISS


If I understand your proposal, and we want >8 bits for both of these fields, then each would require 3 octets of overhead. Storing the EL/GIS value might take 2 or 3 octets.  Alignment would take you to 8.

If I look at the same thing with the FAI draft, I get:

FAI: 4 octets
NFFRR: included in the above, so 0 octets
Entropy: 4 octets (30 bits, plus overhead)
GISS: 4 octets (30 bits, plus overhead)

Total: 12 octets

[HS2] Header overhead is an important dimension for comparison.  To make it apple to apple, let's first clarify several points.

  1.  If an action doesn't need any extra data, there's no point to allocate an EH for it. A flag in EHI is sufficient (this applies to the NFFRR case).
  2.  You are using the bitmap encoding since you mention FAI. Bitmap is more compact than Type/Length for sure. Here the implication is that each ISD must be 4 octets long, otherwise it will need every node to understand the size of every data item, further complicating the design.
  3.  For extensibility, how many ISD data items is planned to be supported? If it exceeds the one label capacity, you will need to extend it. So the actual overhead of FAI could be 8 octets instead of 4.
Let's temporarily put all these details aside and assume the minimum sized FAI. Basically, EH has a fixed 4 byte overhead for HEH. For each use case that does need metadata, EH will add 4 more byte overhead. This is the header overhead comparison. (In your example, it's 24 bytes PSD vs. 12 bytes ISD)
Now let's look at the other dimensions.

  1.  We know some use cases requiring data too big to fit in stack, so PSD is needed anyway. PSD can be used for the use cases with smaller metadata size, although it's relatively inefficient as discussed above. So essentially we can have just one mechanism all everything but to support ISD we end up with two.
  2.  Bitmap, albeit succinct, is inflexible and less extensible. The semantics and order are fixed at the design time; the total size of each data item must be equal to a label size to make bitmap useful in finding the corresponding data item. Use cases identified in the future can only use unused bitmap bits, regardless of its importance. If a use case requires to extend its data size, it's out of luck; if some common use cases also apply to the other types of networks (especially for those still under investigation), the use cases can be seriously limited by such size constraint which hampers design sharing and interoperability.
  3.  Parsing based on bitmap significantly bloats either the parser size or the parsing latency or both (as shown in my analysis). People can write pseudocode by themselves to verify. This is bad for both software and hardware data plane implementation. No wonder bitmap is never used for header encoding before (it's sometimes used in a header for its sub-structure, which is related to the header processing but not the header parsing).
  4.  The encoding of each ISD item itself is awkward. You have 30 bits and only 30 bits at your disposal with a BoS bit in between. It creates a hole in data, which is bad for both software and hardware.

(BTW, I wonder why we want to redefine entropy. We have already have a standard for it and we don't need to do anything more about it ).


It's true that we have a standard for it. It's fairly common, so then the interesting question is what happens when it is present in the label stack along with MNA?  If we do not incorporate EL into MNA, then an implementation has to deal with both MNA and EL independently.  What order are they in? How do they interact? If we use the standard ELI/EL encoding, that's 8 octets just for entropy.  OTOH, if we incorporate EL into MNA, then when using both, we can collapse the encoding.  For example, in the example above, entropy is 4 octets, resulting in a savings of 4 octets.  Half price! And that's assuming that you don't use the smaller entropy/GISS encodings which would give an even greater savings.

[HS2] I think the new design won't nullify the EL/ELI standard so they need to coexist in an incremental deployment scenario given EL/ELI might have been realized (It's possible to provide yet another entropy solution using the new mechanism but the necessity is subject to further discussion IMO). But they don't need to interact with each other because each has its clear function and operational procedure.

Tony