Re: [mpls] Concerns about ISD

[Haoyu Song <[e-mail]>]

Please see inline [HS2]

-Haoyu

From: Tony Li <[e-mail]> On Behalf Of Tony Li
Sent: Monday, April 18, 2022 1:32 PM
To: Haoyu Song <[e-mail]>
Cc: John E Drake <jdrake=[e-mail]>; Tianran Zhou <zhoutianran=[e-mail]>; [e-mail]
Subject: Re: [mpls] Concerns about ISD


Hi Haoyu,


[HS] Here “expensive” means the performance impact. NP is struggling to squeeze every cycle out of the processing, since the throughput is reversely proportional to the number of cycles a packet can finish its processing, and NP’s throughput already falls behind ASIC. Engineers need to handcraft the code to save a bit here and there. There’s never enough. The cycle is really a precious resource which can’t afford to squander.


First off, network processors and switches are ASICs.  It’s all just silicon. How you get there may vary, but at the end of the day, the performance that you get out given a specific process technology corresponds directly to the effort that you put into optimizing it.

Yes, you’re correct, every cycle is precious.  Memory reads are also precious. And in most every modern architecture, a memory read is far more expensive than an ALU operation. For example, in my history, memory reads took around 12ns and operations took 1ns.  Thus, spending a memory read to save a compute operation was a bad trade-off.

As a result, memorizing the memory reads is far more important that worrying about operations.

[HS2] You are talking about accessing DRAM perhaps. The embedded SRAM today can be read in 1ns, and such header processing operations would use SRAM exclusively if memory is needed.

Yes, I have.  The memory reads for parsing a header chain are extremely high.  No thank you.  The bandwidth requirements for the header chain are extremely high.  No thank you.

[HS] Not sure what you mean here. In ASIC, memory access is really fast. In software, every instruction and data need to be read from memory/cache. The more “if” in the code, the more memory accesses. And why header chain parsing requires higher bandwidth?


I’m sorry, but that’s simply not true. Memory accesses are limited by SRAM cycle times. If it’s in DRAM, it’s far worse than that. That is an order of magnitude higher than ALU operation times. Yes, instructions do need to be fetched. This is why, in some architectures, there are multiple operations per instruction.

Header chain parsing requires more bandwidth because you’ve distributed all of the information over memory.  Let’s look at the example again:

EHI: 4 octets
HEH: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 24 octets

This requires that we perform at least 4 reads just for parsing. There’s two more reads to get the EL and GISS values into registers.  And this is again assuming that all PSD is readable. If your architecture doesn’t keep that much of the packet in its context, you’re in tough shape.

If we look at FAI:

AI: 4 octets
NFFRR: included in the above, so 0 octets
Entropy: 4 octets (30 bits, plus overhead)
GISS: 4 octets (30 bits, plus overhead)

Parsing requires 1 read. Again, there’s two more reads to get the EL and GISS values into registers, so that’s equal.

[HS2] The chip keeps the header buffer in SRAM/register, so the access to it is at the same speed of the system clock.

Well, I prefer proposals where there is more flexibility.

[HS] Header chain is more flexible and extensible. It’s used for all the headers AFAIK.


Except that now you’ve decided that SOME operations get taken out of a fixed number of bits, not out of the header chain.  And once we’ve used that number of bits, we’re stuck, back having this same argument again in 10 years.  No thank you.

[HS2] An 8-bit field can support up to 256 extension headers. I think it’s more than enough for data plane functions.

Tony