Re: [mpls] Concerns about ISD

[Tony Li <[e-mail]>]

Hi Haoyu,

> You’re assuming a TCAM and your only metric here seems to be the storage cost.  Many folks don’t do parsing using a TCAM and the performance cost is in the reads, not in the storage.
>  
> [HS2] TCAM is actually the better choice to store states involving bitmaps. Otherwise, parsing bitmap requires extra calculations in addition to storage, making it even less favorable. 


Ok, I disagree. I would MUCH rather have normal ALU functionality.


> [HS2] We propose to use explicit FEC to tell there’s no EHs in the packet to avoid the label stack scanning in such cases. This doesn’t nullify the need to carry extra metadata for some user packets to support some in-network functions.  


I will wait to see a concrete complete proposal before commenting.


> 
> For the EH encoding efficiency, I appreciate your input. Currently it follows the IPv6 EH type (i.e., using NH + LENGTH to delineate every EH). There could certainly be room for improvement.
>  
>  
> Ok.  If I understand your proposal correctly, there are 4 octets of overhead (HEH) at the start. Then, for each network action, there would seem to be at least 3 octets of overhead, plus any associated data, plus alignment.
>  
> Let’s suppose that we want to encode NFFRR, entropy, and GISS in one packet.  By my math, the cost is:
>  
> EHI: 4 octets
> HEH: 4 octets
> NFFRR: 4 octets
> Entropy: 8 octets
> GISS: 8 octets
>  
> Total: 28 octets
>  
> Do I have that right?
>  
> [HS] It’s right if you think 8 octets are needed for Entropy and GISS
>  
>  
> If I understand your proposal, and we want >8 bits for both of these fields, then each would require 3 octets of overhead. Storing the EL/GIS value might take 2 or 3 octets.  Alignment would take you to 8.
>  
> If I look at the same thing with the FAI draft, I get:
>  
> FAI: 4 octets
> NFFRR: included in the above, so 0 octets
> Entropy: 4 octets (30 bits, plus overhead)
> GISS: 4 octets (30 bits, plus overhead)
>  
> Total: 12 octets
>  
> [HS2] Header overhead is an important dimension for comparison.  To make it apple to apple, let’s first clarify several points.
> If an action doesn’t need any extra data, there’s no point to allocate an EH for it. A flag in EHI is sufficient (this applies to the NFFRR case).


Ok, that saves you 4 octets, but costs you a bit in EHI. There’s only a finite number of bits available.  What do you do when they’re consumed?


> You are using the bitmap encoding since you mention FAI. Bitmap is more compact than Type/Length for sure. Here the implication is that each ISD must be 4 octets long, otherwise it will need every node to understand the size of every data item, further complicating the design.  


In FAI, the length of the ISD is defined based on the action and may have different lengths depending on bit combinations. Please see the EG bits, for example.

Yes, determining the length requires reading the full ISD.


> For extensibility, how many ISD data items is planned to be supported? If it exceeds the one label capacity, you will need to extend it. So the actual overhead of FAI could be 8 octets instead of 4. 


Absolutely true. Allocate popular functions first. :) If there are too many actions, then we could also consider a second SPL.


> Let’s temporarily put all these details aside and assume the minimum sized FAI. Basically, EH has a fixed 4 byte overhead for HEH. For each use case that does need metadata, EH will add 4 more byte overhead. This is the header overhead comparison. (In your example, it’s 24 bytes PSD vs. 12 bytes ISD)



So EH is twice as expensive as FAI.


> Now let’s look at the other dimensions.
> We know some use cases requiring data too big to fit in stack, so PSD is needed anyway. PSD can be used for the use cases with smaller metadata size, although it’s relatively inefficient as discussed above. So essentially we can have just one mechanism all everything but to support ISD we end up with two.


That’s true.  We’re trading added complexity for better performance.


>   
> Bitmap, albeit succinct, is inflexible and less extensible. The semantics and order are fixed at the design time; the total size of each data item must be equal to a label size to make bitmap useful in finding the corresponding data item.


That’s incorrect. The size can vary. An implementation must support the NAI that are set.  Handling cases where a node only knows some of them still needs to be discussed, but I favor just not supporting that case at all.


> Use cases identified in the future can only use unused bitmap bits, regardless of its importance. If a use case requires to extend its data size, it’s out of luck; if some common use cases also apply to the other types of networks (especially for those still under investigation), the use cases can be seriously limited by such size constraint which hampers design sharing and interoperability.


Yes, future actions can only allocate unused bitmap bits.  Reusing existing bitmap bits would overlap with existing functions.  Your statement seems to be a non-sequitur.

A new action may define whatever length of ISD it likes, including being variable length. It’s not recommended, but it is possible.


>     
> Parsing based on bitmap significantly bloats either the parser size or the parsing latency or both (as shown in my analysis). People can write pseudocode by themselves to verify. This is bad for both software and hardware data plane implementation. No wonder bitmap is never used for header encoding before (it’s sometimes used in a header for its sub-structure, which is related to the header processing but not the header parsing).


Only if you insist on using a TCAM for parsing. From my perspective, that’s the exactly wrong tool for the job. If you use conventional ALU, then parsing become a straightforward ‘if’ chain.

We are not obligated to optimize the solution for the wrong tools.

> The encoding of each ISD item itself is awkward. You have 30 bits and only 30 bits at your disposal with a BoS bit in between. It creates a hole in data, which is bad for both software and hardware.


We already discussed this. Yes, it’s not optimal, but it’s only a few instructions to fix, if necessary.

> (BTW, I wonder why we want to redefine entropy. We have already have a standard for it and we don’t need to do anything more about it ).
>  
>  
> It’s true that we have a standard for it. It’s fairly common, so then the interesting question is what happens when it is present in the label stack along with MNA?  If we do not incorporate EL into MNA, then an implementation has to deal with both MNA and EL independently.  What order are they in? How do they interact? If we use the standard ELI/EL encoding, that’s 8 octets just for entropy.  OTOH, if we incorporate EL into MNA, then when using both, we can collapse the encoding.  For example, in the example above, entropy is 4 octets, resulting in a savings of 4 octets.  Half price! And that’s assuming that you don’t use the smaller entropy/GISS encodings which would give an even greater savings.
>  
> [HS2] I think the new design won’t nullify the EL/ELI standard so they need to coexist in an incremental deployment scenario given EL/ELI might have been realized (It’s possible to provide yet another entropy solution using the new mechanism but the necessity is subject to further discussion IMO). But they don’t need to interact with each other because each has its clear function and operational procedure.
> 


Well, you could do things that way, but IMHO, that’s suboptimal.  If you’re including MNA, then you’re already limting yourself to MNA capable nodes. You might as well take the benefit of the better encoding.

Tony