Re: [Coin] The Future of P4, Revisited

[Hesham ElBakoury <[e-mail]>]

This is comprehensive survey on "An Exhaustive Survey on P4 Programmable
Data Plane Switches: Taxonomy, Applications, Challenges, and Future" Trends
programmable switches. It is almost 2 years old, therefore, some updates
are needed.
https://ieeexplore.ieee.org/document/9447791

The survey mentioned Cavium (Now Mavell) XPliant switch. Is Octeon 10 based
on Xpliant technology?

Hesham







On Mon, May 15, 2023, 2:45 PM Jeff Tantsura <[e-mail]> wrote:

> Hermann,
>
> You are comparing apples to oranges (or actually apples to blackberries :))
> Trident4 (with 256x 100G SerDes) is a 25T device.
> Jericho (2C+) is 14.4T device.
> Octeon is a smartNIC/DPU NPU - (400G now, 800G in 2 years?) that should
> not be compared to the above.
>
> There’s a number of DPUs on the market that expose P4 APIs (partial spec +
> private extensions), non of them can replace (economically) a switch/router
> ASIC, I believe there a Pensando based Aruba switch (public data) - look at
> the price and power per bit.
>
> P4 - there’s quite some stuff in production that uses P4, Google is also
> pushing PINS in SONiC/SAI, while in the beginning it was a genius strategy
> to (up)sell BF, now the scope is somewhat beyond that.
> P4 is indeed quite limited as compared to “native” SDKs, with an argument
> that is provides normalized/common platform/layer to program HW, one might
> however argue that SAI will eventually be the only winner in this place.
> Lack of wide ecosystem and development of compliers towards different HW
> could eventually be the end of P4 (and potentially push to develop
> something new).
>
> My 0.2
> Jeff
>
> On May 15, 2023, at 1:19 PM, hemant=[e-mail] wrote:
>
> Broadcom NPL is proprietary, and Broadcom develops customer code in NPL
> for switch. Further Trident/Jericho 2 are 10-12 Tbps asics while Octeon 10
> is a 1 Tbps asic. My company’s P4 compiler makes Octeon P4 programmable.
>
> Hemant
>
> *From:* Hesham ElBakoury <[e-mail]>
> *Sent:* Monday, May 15, 2023 3:28 PM
> *To:* [e-mail]; 'Toerless Eckert' <[e-mail]>; 'Marie-Jose
> Montpetit' <[e-mail]>
> *Cc:* 'coin' <[e-mail]>; [e-mail]
> *Subject:* Re: [Coin] Fwd: The Future of P4, Revisited
>
> How you compare Marvel Octeon 10 with Broadcom Trident and Jericho 2
> devices which use NPL (https://nplang.org).
> Hesham
>
> Toerless,
>
>
>
> This is the latest network processor which supports use in a switch or nic.
>
>
>
> https://www.marvell.com/content/dam/marvell/en/company/media-kit/octeon-10/marvell-octeon-10-media-deck.pdf
>
>
>
> This is your Tofino replacement with hardware available from Marvell and P4 compiler and tools chain available from my company.
>
>
>
> Hemant
>
>
>
>
>
> -----Original Message-----
>
> From: Coin <[e-mail]> <[e-mail]> On Behalf Of Toerless Eckert
>
> Sent: Monday, May 15, 2023 1:04 PM
>
> To: Marie-Jose Montpetit <[e-mail]> <[e-mail]>
>
> Cc: coin <[e-mail]> <[e-mail]>; [e-mail]
>
> Subject: Re: [Coin] Fwd: The Future of P4, Revisited
>
>
>
> Thanks, Marie-Jose
>
>
>
> IMHO:
>
>
>
> - P4 in research short-term:
>
>   Researchers can not do real-world relevant PoC with just BMV2, because it has no constraints.
>
>   All those papers claiming some problem can be solved with P4 and then just use BMV2 are just
>
>   so painful to read/listen-to. We really need a BMV2 with the exact constraints of Tofino,
>
>   and given how Intel is deinvesting from Tofino, maybe they would finally be willing to
>
>   publish those constraints outside of NDA so this can be implemented into BMV2.
>
>
>
> - SmartNICs:
>
>   P4 is not the right DSL for SmartNICs even if Intel may say so because they want to
>
>   sell SmartNICs and have a brand with P4. SmartNICs are much more flexible, and programming then
>
>   in P4 constraints you. If you don't know anything better, try eBPF. That is AFAIK the
>
>   most widely adopted DSL today to span general purpose CPU (host/router CPU), and SmartNICs.
>
>
>
>   SmartNICs are a great target FPE for forwarding plane features you do not need on every hop,
>
>   but for example only on few "can be more expensive" hops. WAN interface of site-edge-routers
>
>   (Campus, Home, WAN) for example. But do not try to assume they would be used beyond that
>
>   (on every hop == especially on higher aggregation speed links!).
>
>
>
> - P4/Coin for higher-than-forwarding-plane features:
>
>   With Tofino disappearing, i think we should caution researchers to NOT invest cycles trying to
>
>   implement solutions on P4 with the explicit purpose of "abusing" Tofino, aka: Program a
>
>   non-forwarding-plane problem solution in P4 in the expectation that a fast P4 exeuction
>
>   product like Tofino would be a great alternative/competition to other execution platforms
>
>   (SmartNIC, CPU,...). We had several of those great ideas presented/shown in Coin, they
>
>   where all about higher than forwarding-plane "compute" problems. But very quickly, with
>
>   Tofino aging, i am sure CPU or SmartNIC slutions for the same problems can easily be
>
>   shown to be more cost-effective.
>
>
>
> - Long-term forwarding plane research:
>
>   The rejection of the industry (outside Intel) to support P4 for researchers on their router/switch
>
>   forwarding planes (for a decade now) should primarily be a trigger for research funding
>
>   such as from NSF/EU to better enable long-term forwarding plane research. IMHO there
>
>   should simply be a single well funded effort for a Network Programming Element (NPE) equivalent
>
>   to RISC-V. Aka: fully open source architecture and instruction set for a research NPE,
>
>   competitive to industry products. There are lot of other benefits from such an effort
>
>   that could help make that happen.
>
>
>
> Cheers
>
>     Toerless
>
>
>
> On Mon, May 15, 2023 at 03:33:42AM -0700, Marie-Jose Montpetit wrote:
>
> Interesting article for out community.
>
>
>
> mjm
>
>
>
> Marie-José Montpetit, Ph.D.
>
> [e-mail]
>
>
>
>
>
>
>
> From: Larry Peterson from Systems Approach
>
> <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
>
> Reply: Larry Peterson from Systems Approach
>
> <reply+1zsws7&rqag0&&3ca860fd966ac1a0bf575ed48379b6645b7df5edc0903e4c5
>
> [e-mail]>
>
> <reply+1zsws7&rqag0&&3ca860fd966ac1a0bf575ed48379b6645b7df5edc0903e4c5
>
> [e-mail]>
>
> Date: May 15, 2023 at 3:55:52 AM
>
> To: [e-mail] <[e-mail]> <[e-mail]>
>
> <[e-mail]> <[e-mail]>
>
> Subject:  The Future of P4, Revisited
>
>
>
> The P4 Workshop was a couple weeks ago, and as General Chair, I went
>
> into it with a fair amount of trepidation. My concern was that Intel’s
>
> announcement earlier this year that they’re cancelling development of
>
> the Tofino 3 switching chip would have a chilling effect, not only on
>
> the Workshop, but also on the future of P4. That concern has been
>
> voiced in several forums  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> ‌ ‌ ‌ Open in app
>
> <https://open.substack.com/pub/systemsapproach/p/the-future-of-p4-one- <https://open.substack.com/pub/systemsapproach/p/the-future-of-p4-one-perspective?utm_source=email&redirect=app-store>
>
> perspective?utm_source=email&redirect=app-store> <https://open.substack.com/pub/systemsapproach/p/the-future-of-p4-one-perspective?utm_source=email&redirect=app-store>
>
> or online
>
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> 2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9 <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4T <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> lRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWF <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> YTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmU <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> S50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTI <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> wNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxM <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> zczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGl <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> yZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
>
> The Future of P4, Revisited
>
> <https://substack.com/app-link/post?publication_id=283943&post_id=1206 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> 01591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIj <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> o0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> .tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
>
>
> Larry Peterson
>
> <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j= <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> May 15
>
> <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j= <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
>
>
> <https://substack.com/app-link/post?publication_id=283943&post_id=1206 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
>
> 01591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
>
> VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIs <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
>
> ImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIi <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
>
> wic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&u <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
>
> tm_medium=email> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
>
> <https://substack.com/app-link/post?publication_id=283943&post_id=1206 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
>
> 01591&utm_source=substack&utm_medium=email&isFreemail=true&comments=tr <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
>
> ue&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiO <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
>
> jE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
>
> 6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&ut <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
>
> m_source=substack&utm_medium=email> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
>
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
> 2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXR <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
> tX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvb <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
> W1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQ <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
> sImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1Y <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
> iI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo? <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
> &utm_source=substack&utm_medium=email> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
>
>
>
> Share
>
> <https://substack.com/app-link/post?publication_id=283943&post_id=1206 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> 01591&utm_source=substack&utm_medium=email&utm_content=share&action=sh <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> are&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNC <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> wicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0 <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFE <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
> gLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
>
>
>
>
>
> The P4 workshop has now been chaired by both co-founders of Systems
>
> Approach, but this year the P4 landscape has shifted again with
>
> Intel’s announcement that Tofino 3, its flagship P4-powered switching
>
> chip, would not go ahead. There is much more to P4 than Tofino,
>
> however, as we explore in this week’s newsletter.
>
> ------------------------------
>
>
>
> The P4 Workshop
>
> <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j= <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> was a couple weeks ago, and as General Chair, I went into it with a
>
> fair amount of trepidation. My concern was that Intel’s announcement
>
> <https://substack.com/redirect/a0831b60-fff8-4ded-88c7-f3e6ef7e1e54?j= <https://substack.com/redirect/a0831b60-fff8-4ded-88c7-f3e6ef7e1e54?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/a0831b60-fff8-4ded-88c7-f3e6ef7e1e54?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> earlier this year that they’re cancelling development of the Tofino 3
>
> switching chip would have a chilling effect, not only on the Workshop,
>
> but also on the future of P4. That concern has been voiced in several
>
> forums, including SIGCOMM’s Slack workspace
>
> <https://substack.com/redirect/300dcab9-6056-4c82-9627-1e08ea449b0e?j= <https://substack.com/redirect/300dcab9-6056-4c82-9627-1e08ea449b0e?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/300dcab9-6056-4c82-9627-1e08ea449b0e?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>,
>
> with members of the P4 Advisory Board
>
> <https://substack.com/redirect/2fcc130d-cf74-4d68-be4e-d531cbb379b4?j= <https://substack.com/redirect/2fcc130d-cf74-4d68-be4e-d531cbb379b4?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/2fcc130d-cf74-4d68-be4e-d531cbb379b4?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> making reassuring pronouncements in various settings. (See for
>
> example, Nick McKeown’s post to the P4 Forum
>
> <https://substack.com/redirect/b0f9bc57-686d-4962-8676-c234ebcbd12f?j= <https://substack.com/redirect/b0f9bc57-686d-4962-8676-c234ebcbd12f?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/b0f9bc57-686d-4962-8676-c234ebcbd12f?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>,
>
> and Nick along with Nate Foster and Jennifer Rexford discussing the
>
> future of Network Programmability on The Networking Channel
>
> <https://substack.com/redirect/f2016908-8afb-4538-be3d-5d0e6aab0b3c?j= <https://substack.com/redirect/f2016908-8afb-4538-be3d-5d0e6aab0b3c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/f2016908-8afb-4538-be3d-5d0e6aab0b3c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> ).
>
>
>
> I won’t try to give a point-by-point replay of what Nick, Nate, and
>
> Jen and others have been saying, except to observe that at a high
>
> level it can be summarized as follows:
>
>
>
> *Programmable Networks  >>  P4 Language  >>  Tofino Switching Chip*
>
>
>
> They point out, for example, that Tofino is just one of many
>
> interesting backend targets for P4 programs (SmartNICs
>
> <https://substack.com/redirect/f5d04cdb-5915-42ab-9904-a20c38eeab33?j= <https://substack.com/redirect/f5d04cdb-5915-42ab-9904-a20c38eeab33?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/f5d04cdb-5915-42ab-9904-a20c38eeab33?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> and IPUs
>
> <https://substack.com/redirect/a0ddc803-4ea3-4248-96a1-bdc33f86cd15?j= <https://substack.com/redirect/a0ddc803-4ea3-4248-96a1-bdc33f86cd15?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/a0ddc803-4ea3-4248-96a1-bdc33f86cd15?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> being the next “big deal”) and P4 is one of many tools being used to
>
> inject functionality into the end-to-end network path (DPDK and eBPF
>
> being two active projects that people are integrating with P4).
>
> Ultimately, the value of programmability comes from having visibility
>
> and control over the network, and there are many complementary approaches to making that happen.
>
> With that background, I do have three takeaways from what turned out
>
> to be an interesting and vibrant two days at the P4 Workshop (despite
>
> my initial concerns).
>
>
>
> First, we’re often so focused on P4 as a tool to program the
>
> forwarding pipeline that we forget the other half of its value
>
> proposition: It also provides a way to specify the behavior of a
>
> pipeline (independent of how that pipeline is implemented). We talk
>
> about this idea, and the value of being able to auto-generate the
>
> Control API, in the P4 chapter of our SDN Book
>
> <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
>
> Rob Sherwood made a similar argument
>
> <https://substack.com/redirect/2442a19b-b82c-410c-8692-f50b89733875?j= <https://substack.com/redirect/2442a19b-b82c-410c-8692-f50b89733875?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/2442a19b-b82c-410c-8692-f50b89733875?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> at the P4 Workshop. It is now becoming a reality as companies like
>
> Google are starting to use such behavioral definitions as a Hardware
>
> Abstraction Layer (see Parveen Patel’s Keynote
>
> <https://substack.com/redirect/e656a164-3fef-497b-9628-ffeb700108af?j= <https://substack.com/redirect/e656a164-3fef-497b-9628-ffeb700108af?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/e656a164-3fef-497b-9628-ffeb700108af?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
>
> at the Workshop). This makes me hopeful that we are rapidly
>
> approaching the day when a P4 program (plus the generated P4RT
>
> interface) will become the standard way network providers specify
>
> their requirements to network vendors, and proposed new features
>
> (whether proprietary or standard) will be specified by a P4 program
>
> (potentially augmenting the intuition and design rationale presented in an RFC).
>
>
>
> *As an aside, I couldn’t help but notice the similarities between the
>
> architecture Parveen described and the way P4 has been used to program
>
> the forwarding plane of the 5G Mobile Core
>
> <https://substack.com/redirect/64e4cac9-e6fd-4652-8e36-09f50ff1305c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0> <https://substack.com/redirect/64e4cac9-e6fd-4652-8e36-09f50ff1305c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
>
>