Re: [Coin] Fwd: The Future of P4, Revisited

[Hesham ElBakoury <[e-mail]>]

Thanks Hemant!
Hesham

On Tue, May 16, 2023, 1:41 PM <[e-mail]> wrote:

> Tofino is a switching asic and the Octeon 10 is one too with the SERDES
> bus. Tofino does not support crypto and AI/ML which Octeon supports. The
> only significant difference is Tofino supports up to 16 Tbps while Octeon
> 10 supports up to 1 T bps. Of course, with lower rate, Octeon uses much
> less power. Ericson research, in p4 mailer, asked for a 1 Tbps asic with P4
> support because Tofino at 3 Tbps was overkill for them and using too much
> power. The only solution I could find was Octeon 10 with P4 compiler
> licensed from my company.
>
>
>
> Hemant
>
>
>
>
>
> *From:* Hesham ElBakoury <[e-mail]>
> *Sent:* Tuesday, May 16, 2023 4:06 PM
> *To:* [e-mail]
> *Cc:* Toerless Eckert <[e-mail]>; Marie-Jose Montpetit <
> [e-mail]>; coin <[e-mail]>; coinrg-chairs <
> [e-mail]>
> *Subject:* Re: [Coin] Fwd: The Future of P4, Revisited
>
>
>
> Hemant,
>
> Does Octeon 10 have the same capabilities to replace next generation
> Tofino whose development is halted by Intel?
>
>
>
> Thanks
>
> Hesham
>
>
>
> On Mon, May 15, 2023, 11:05 AM <hemant=[e-mail]> wrote:
>
> Toerless,
>
> This is the latest network processor which supports use in a switch or nic.
>
>
> https://www.marvell.com/content/dam/marvell/en/company/media-kit/octeon-10/marvell-octeon-10-media-deck.pdf
>
> This is your Tofino replacement with hardware available from Marvell and
> P4 compiler and tools chain available from my company.
>
> Hemant
>
>
> -----Original Message-----
> From: Coin <[e-mail]> On Behalf Of Toerless Eckert
> Sent: Monday, May 15, 2023 1:04 PM
> To: Marie-Jose Montpetit <[e-mail]>
> Cc: coin <[e-mail]>; [e-mail]
> Subject: Re: [Coin] Fwd: The Future of P4, Revisited
>
> Thanks, Marie-Jose
>
> IMHO:
>
> - P4 in research short-term:
>   Researchers can not do real-world relevant PoC with just BMV2, because
> it has no constraints.
>   All those papers claiming some problem can be solved with P4 and then
> just use BMV2 are just
>   so painful to read/listen-to. We really need a BMV2 with the exact
> constraints of Tofino,
>   and given how Intel is deinvesting from Tofino, maybe they would finally
> be willing to
>   publish those constraints outside of NDA so this can be implemented into
> BMV2.
>
> - SmartNICs:
>   P4 is not the right DSL for SmartNICs even if Intel may say so because
> they want to
>   sell SmartNICs and have a brand with P4. SmartNICs are much more
> flexible, and programming then
>   in P4 constraints you. If you don't know anything better, try eBPF. That
> is AFAIK the
>   most widely adopted DSL today to span general purpose CPU (host/router
> CPU), and SmartNICs.
>
>   SmartNICs are a great target FPE for forwarding plane features you do
> not need on every hop,
>   but for example only on few "can be more expensive" hops. WAN interface
> of site-edge-routers
>   (Campus, Home, WAN) for example. But do not try to assume they would be
> used beyond that
>   (on every hop == especially on higher aggregation speed links!).
>
> - P4/Coin for higher-than-forwarding-plane features:
>   With Tofino disappearing, i think we should caution researchers to NOT
> invest cycles trying to
>   implement solutions on P4 with the explicit purpose of "abusing" Tofino,
> aka: Program a
>   non-forwarding-plane problem solution in P4 in the expectation that a
> fast P4 exeuction
>   product like Tofino would be a great alternative/competition to other
> execution platforms
>   (SmartNIC, CPU,...). We had several of those great ideas presented/shown
> in Coin, they
>   where all about higher than forwarding-plane "compute" problems. But
> very quickly, with
>   Tofino aging, i am sure CPU or SmartNIC slutions for the same problems
> can easily be
>   shown to be more cost-effective.
>
> - Long-term forwarding plane research:
>   The rejection of the industry (outside Intel) to support P4 for
> researchers on their router/switch
>   forwarding planes (for a decade now) should primarily be a trigger for
> research funding
>   such as from NSF/EU to better enable long-term forwarding plane
> research. IMHO there
>   should simply be a single well funded effort for a Network Programming
> Element (NPE) equivalent
>   to RISC-V. Aka: fully open source architecture and instruction set for a
> research NPE,
>   competitive to industry products. There are lot of other benefits from
> such an effort
>   that could help make that happen.
>
> Cheers
>     Toerless
>
> On Mon, May 15, 2023 at 03:33:42AM -0700, Marie-Jose Montpetit wrote:
> > Interesting article for out community.
> >
> > mjm
> >
> > Marie-José Montpetit, Ph.D.
> > [e-mail]
> >
> >
> >
> > From: Larry Peterson from Systems Approach
> > <[e-mail]> <[e-mail]>
> > Reply: Larry Peterson from Systems Approach
> > <reply+1zsws7&rqag0&&3ca860fd966ac1a0bf575ed48379b6645b7df5edc0903e4c5
> > [e-mail]>
> > <reply+1zsws7&rqag0&&3ca860fd966ac1a0bf575ed48379b6645b7df5edc0903e4c5
> > [e-mail]>
> > Date: May 15, 2023 at 3:55:52 AM
> > To: [e-mail] <[e-mail]>
> > <[e-mail]>
> > Subject:  The Future of P4, Revisited
> >
> > The P4 Workshop was a couple weeks ago, and as General Chair, I went
> > into it with a fair amount of trepidation. My concern was that Intel’s
> > announcement earlier this year that they’re cancelling development of
> > the Tofino 3 switching chip would have a chilling effect, not only on
> > the Workshop, but also on the future of P4. That concern has been
> > voiced in several forums  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> > ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> > ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> > ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> > ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> > ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> > ‌ ‌ ‌ Open in app
> > <https://open.substack.com/pub/systemsapproach/p/the-future-of-p4-one-
> > perspective?utm_source=email&redirect=app-store>
> > or online
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9
> > rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4T
> > lRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWF
> > YTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmU
> > S50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTI
> > wNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxM
> > zczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGl
> > yZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
> > The Future of P4, Revisited
> > <https://substack.com/app-link/post?publication_id=283943&post_id=1206
> > 01591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIj
> > o0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6
> > MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ
> > .tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
> >
> > Larry Peterson
> > <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > May 15
> > <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> >
> > <https://substack.com/app-link/post?publication_id=283943&post_id=1206
> > 01591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2
> > VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIs
> > ImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIi
> > wic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&u
> > tm_medium=email>
> > <https://substack.com/app-link/post?publication_id=283943&post_id=1206
> > 01591&utm_source=substack&utm_medium=email&isFreemail=true&comments=tr
> > ue&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiO
> > jE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI
> > 6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&ut
> > m_source=substack&utm_medium=email>
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXR
> > tX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvb
> > W1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQ
> > sImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1Y
> > iI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?
> > &utm_source=substack&utm_medium=email>
> >
> > Share
> > <https://substack.com/app-link/post?publication_id=283943&post_id=1206
> > 01591&utm_source=substack&utm_medium=email&utm_content=share&action=sh
> > are&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNC
> > wicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0
> > OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFE
> > gLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
> >
> >
> > The P4 workshop has now been chaired by both co-founders of Systems
> > Approach, but this year the P4 landscape has shifted again with
> > Intel’s announcement that Tofino 3, its flagship P4-powered switching
> > chip, would not go ahead. There is much more to P4 than Tofino,
> > however, as we explore in this week’s newsletter.
> > ------------------------------
> >
> > The P4 Workshop
> > <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > was a couple weeks ago, and as General Chair, I went into it with a
> > fair amount of trepidation. My concern was that Intel’s announcement
> > <https://substack.com/redirect/a0831b60-fff8-4ded-88c7-f3e6ef7e1e54?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > earlier this year that they’re cancelling development of the Tofino 3
> > switching chip would have a chilling effect, not only on the Workshop,
> > but also on the future of P4. That concern has been voiced in several
> > forums, including SIGCOMM’s Slack workspace
> > <https://substack.com/redirect/300dcab9-6056-4c82-9627-1e08ea449b0e?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>,
> > with members of the P4 Advisory Board
> > <https://substack.com/redirect/2fcc130d-cf74-4d68-be4e-d531cbb379b4?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > making reassuring pronouncements in various settings. (See for
> > example, Nick McKeown’s post to the P4 Forum
> > <https://substack.com/redirect/b0f9bc57-686d-4962-8676-c234ebcbd12f?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>,
> > and Nick along with Nate Foster and Jennifer Rexford discussing the
> > future of Network Programmability on The Networking Channel
> > <https://substack.com/redirect/f2016908-8afb-4538-be3d-5d0e6aab0b3c?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > ).
> >
> > I won’t try to give a point-by-point replay of what Nick, Nate, and
> > Jen and others have been saying, except to observe that at a high
> > level it can be summarized as follows:
> >
> > *Programmable Networks  >>  P4 Language  >>  Tofino Switching Chip*
> >
> > They point out, for example, that Tofino is just one of many
> > interesting backend targets for P4 programs (SmartNICs
> > <https://substack.com/redirect/f5d04cdb-5915-42ab-9904-a20c38eeab33?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > and IPUs
> > <https://substack.com/redirect/a0ddc803-4ea3-4248-96a1-bdc33f86cd15?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > being the next “big deal”) and P4 is one of many tools being used to
> > inject functionality into the end-to-end network path (DPDK and eBPF
> > being two active projects that people are integrating with P4).
> > Ultimately, the value of programmability comes from having visibility
> > and control over the network, and there are many complementary
> approaches to making that happen.
> > With that background, I do have three takeaways from what turned out
> > to be an interesting and vibrant two days at the P4 Workshop (despite
> > my initial concerns).
> >
> > First, we’re often so focused on P4 as a tool to program the
> > forwarding pipeline that we forget the other half of its value
> > proposition: It also provides a way to specify the behavior of a
> > pipeline (independent of how that pipeline is implemented). We talk
> > about this idea, and the value of being able to auto-generate the
> > Control API, in the P4 chapter of our SDN Book
> > <
> https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0
> >.
> > Rob Sherwood made a similar argument
> > <https://substack.com/redirect/2442a19b-b82c-410c-8692-f50b89733875?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > at the P4 Workshop. It is now becoming a reality as companies like
> > Google are starting to use such behavioral definitions as a Hardware
> > Abstraction Layer (see Parveen Patel’s Keynote
> > <https://substack.com/redirect/e656a164-3fef-497b-9628-ffeb700108af?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > at the Workshop). This makes me hopeful that we are rapidly
> > approaching the day when a P4 program (plus the generated P4RT
> > interface) will become the standard way network providers specify
> > their requirements to network vendors, and proposed new features
> > (whether proprietary or standard) will be specified by a P4 program
> > (potentially augmenting the intuition and design rationale presented in
> an RFC).
> >
> > *As an aside, I couldn’t help but notice the similarities between the
> > architecture Parveen described and the way P4 has been used to program
> > the forwarding plane of the 5G Mobile Core
> > <
> https://substack.com/redirect/64e4cac9-e6fd-4652-8e36-09f50ff1305c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0
> >.
> > Both include a P4-based “abstract forwarding model” that’s independent
> > of the underlying implementation details*.
> >
> > Second, it is common to divide forwarding pipelines into “programmable”
> > versus “fixed function”, but this glosses over what might be the more
> > important distinction: whether the pipeline is *open* or *closed*.
> > Even “fixed function” pipelines are increasingly flexible–it’s just a
> > question of how restrictive the vendor is in who they allow to make
> > changes. This restriction may have the biggest impact on researchers
> > who want to experiment with a new feature (especially ones that do not
> > yet have a proven market), but maybe less so in the commercial world
> > where incentives to make changes are (arguably) well-defined. Using P4
> > as the “spec language” (as I just outlined) has the potential to
> > accelerate the process on the commercial side. On the research side,
> > there is a strong argument in favor of using Tofino 2 to demonstrate
> > the feasibility and value of new ideas (12.8 Tb/s still makes for a
> > credible Proof-of-Concept), and repeating the refrain yet again,
> > P4-as-spec makes for a compelling tech transfer story. If that were to
> > happen, it would be interesting to see how vendors and chip designers
> > adapt to reduce their spec-to-hardware implementation overhead. I
> > would argue that programmable forwarding planes have a time-to-market
> advantage even for closed solutions.
> >
> > Third, our focus on quantifiable metrics makes it easy to forget about
> > the less quantifiable aspects of programmability. At its core, P4 is a
> > programming language that does a good job of abstracting the essence
> > of a packet forwarding pipeline. It is enormously impressive that a P4
> > program can be compiled onto a PISA-based
> > <https://substack.com/redirect/01cfb256-04e7-49f8-9600-55e1471955b8?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > switching chip that has the same performance, die area, cost, and
> > power consumption of a fixed-function ASIC (and that equivalency was
> > probably necessary for P4 to be taken seriously), but hitting that
> > quantifiable mark is not sufficient. Well-designed languages are
> > software tools that bring clarity to the intellectual challenge of
> > programming. For me, the biggest “aha” moment of the Workshop was when
> > Chris Sommers (long-time P4 contributor and new co-Chair of the API
> > Working Group) started rattling off all the functions he’d been
> > involved in writing in P4, and remarking on how natural P4 makes that
> > process. There is certainly room to add new language features as P4
> > expands its domain to include SmartNICs and IPUs—as Chris and the
> > other WG chairs are now pursuing—but having an existing target to evolve
> is a great position to be in.
> >
> > One common thread that weaves its way through these three takeaways is
> > that Intel’s cancellation of the Tofino 3 chip is a potentially
> > helpful forcing
> > function: The P4 community has to demonstrate the value of the
> > language without being buttressed by ever-improving performance
> > numbers that have more to do with 7nm semiconductor technology than
> > anything networking people have done. I saw a lot of evidence that
> > exactly that is happening at last month’s workshop. The march to
> > programmable networks is inevitable (in my view), and I’m still
> > optimistic about the role P4 will play a central role.
> >
> > Systems Approach is reader-supported and we are committed to keeping
> > our books and articles open to all. To receive new posts and support
> > our work, consider becoming a free or paid subscriber.
> >
> > Upgrade to paid
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL3N1YnNjcmliZT90b2tlbj1leUoxYzJWeVgybGtJam8wTmpVM05
> > qSXlOQ3dpYVdGMElqb3hOamcwTVRNM016UTVMQ0psZUhBaU9qRTJPRFkzTWprek5Ea3NJb
> > Wx6Y3lJNkluQjFZaTB5T0RNNU5ETWlMQ0p6ZFdJaU9pSmphR1ZqYTI5MWRDSjkuLW5LYUM
> > tRE1weEw3Smp0bHh4U1I0aEFFLXQ2QWltZ0lENFlFRC1BbWJicyZ1dG1fc291cmNlPXBvc
> > 3QiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsIml
> > hdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6I
> > mxpbmstcmVkaXJlY3QifQ.lWjTeCLgxCt9OYaegbWWppVXvEzTeJBQs6psa45Oowg?&utm
> > _medium=email&utm_source=subscribe-widget-preamble&utm_content=1206015
> > 91>
> >
> > ------------------------------
> >
> > We continue to run into people who want to translate our books into
> > other languages, and if you are one of them, you should definitely
> > reach out to us. The latest entrant is a Portuguese translation
> > <https://substack.com/redirect/e0ad5652-867f-4d2c-8cd6-3fc8b0f8eda0?j=
> > eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> > of our Private 5G book by Edmar Candeia Gurjão. You can find other
> > translations of our books here
> > <
> https://substack.com/redirect/0d0dad50-a3db-42e9-9ad6-95318436a540?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0
> >.
> >
> >
> > You can follow us on Mastodon
> > <
> https://substack.com/redirect/3e6fc5cc-6a89-4a15-97bf-231cf12de9ec?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0
> >.
> >
> >
> > You're currently a free subscriber to Systems Approach
> > <
> https://substack.com/redirect/3d94c242-4758-4bf3-9e1d-62b5ce3f23ab?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0
> >.
> > For the full experience, upgrade your subscription.
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL3N1YnNjcmliZT90b2tlbj1leUoxYzJWeVgybGtJam8wTmpVM05
> > qSXlOQ3dpYVdGMElqb3hOamcwTVRNM016UTVMQ0psZUhBaU9qRTJPRFkzTWprek5Ea3NJb
> > Wx6Y3lJNkluQjFZaTB5T0RNNU5ETWlMQ0p6ZFdJaU9pSmphR1ZqYTI5MWRDSjkuLW5LYUM
> > tRE1weEw3Smp0bHh4U1I0aEFFLXQ2QWltZ0lENFlFRC1BbWJicyZ1dG1fc291cmNlPXBvc
> > 3QiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsIml
> > hdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6I
> > mxpbmstcmVkaXJlY3QifQ.lWjTeCLgxCt9OYaegbWWppVXvEzTeJBQs6psa45Oowg?&utm
> > _source=substack&utm_medium=email&utm_content=postcta>
> >
> > Upgrade to paid
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL3N1YnNjcmliZT90b2tlbj1leUoxYzJWeVgybGtJam8wTmpVM05
> > qSXlOQ3dpYVdGMElqb3hOamcwTVRNM016UTVMQ0psZUhBaU9qRTJPRFkzTWprek5Ea3NJb
> > Wx6Y3lJNkluQjFZaTB5T0RNNU5ETWlMQ0p6ZFdJaU9pSmphR1ZqYTI5MWRDSjkuLW5LYUM
> > tRE1weEw3Smp0bHh4U1I0aEFFLXQ2QWltZ0lENFlFRC1BbWJicyZ1dG1fc291cmNlPXBvc
> > 3QiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsIml
> > hdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6I
> > mxpbmstcmVkaXJlY3QifQ.lWjTeCLgxCt9OYaegbWWppVXvEzTeJBQs6psa45Oowg?&utm
> > _source=substack&utm_medium=email&utm_content=postcta>
> >
> > Like
> > <https://substack.com/app-link/post?publication_id=283943&post_id=1206
> > 01591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2
> > VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIs
> > ImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIi
> > wic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&u
> > tm_medium=email>
> > Comment
> > <https://substack.com/app-link/post?publication_id=283943&post_id=1206
> > 01591&utm_source=substack&utm_medium=email&isFreemail=true&comments=tr
> > ue&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiO
> > jE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI
> > 6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&ut
> > m_source=substack&utm_medium=email>
> > Restack
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXR
> > tX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvb
> > W1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQ
> > sImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1Y
> > iI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?
> > &utm_source=substack&utm_medium=email>
> >
> >
> > © 2023 Systems Approach, LLC
> > 548 Market Street PMB 72296, San Francisco, CA 94104 Unsubscribe
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY
> > 2guc3Vic3RhY2suY29tL2FjdGlvbi9kaXNhYmxlX2VtYWlsP3Rva2VuPWV5SjFjMlZ5WDJ
> > sa0lqbzBOalUzTmpJeU5Dd2ljRzl6ZEY5cFpDSTZNVEl3TmpBeE5Ua3hMQ0pwWVhRaU9qR
> > TJPRFF4TXpjek5Ea3NJbVY0Y0NJNk1UWTROamN5T1RNME9Td2lhWE56SWpvaWNIVmlMVEk
> > 0TXprME15SXNJbk4xWWlJNkltUnBjMkZpYkdWZlpXMWhhV3dpZlEuRXdaUkN3ZEdjdG8xd
> > FdwbmRVd3h2eXZKUk5pVmlOX2h1ZDBVNERDbVZYayZleHBpcmVzPTM2NWQiLCJwIjoxMjA
> > 2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzN
> > zM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJ
> > lY3QifQ.l6lDN2unh8JkUjeYEXjKiNFu7akDZ_FyOQUxaYockVM?>
> >
> > [image: Start writing]
> > <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zdWJzdGFjay5jb20vc
> > 2lnbnVwP3V0bV9zb3VyY2U9c3Vic3RhY2smdXRtX21lZGl1bT1lbWFpbCZ1dG1fY29udGV
> > udD1mb290ZXImdXRtX2NhbXBhaWduPWF1dG9maWxsZWQtZm9vdGVyJmZyZWVTaWdudXBFb
> > WFpbD1tYXJpZUBtam1vbnRwZXRpdC5jb20iLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0Myw
> > iZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5M
> > zQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.GtVGEVhyYZ__pv96
> > bIPsssoL78rJrnvkIgcdwC4WQfs?>
>
> > --
> > Coin mailing list
> > [e-mail]
> > https://www.irtf.org/mailman/listinfo/coin
>
>
> --
> ---
> [e-mail]
>
> --
> Coin mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/coin
> --
> Coin mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/coin
>
>