Re: [Coin] Fwd: The Future of P4, Revisited

[Toerless Eckert <[e-mail]>]

Thanks, Marie-Jose

IMHO:

- P4 in research short-term:
  Researchers can not do real-world relevant PoC with just BMV2, because it has no constraints.
  All those papers claiming some problem can be solved with P4 and then just use BMV2 are just
  so painful to read/listen-to. We really need a BMV2 with the exact constraints of Tofino,
  and given how Intel is deinvesting from Tofino, maybe they would finally be willing to
  publish those constraints outside of NDA so this can be implemented into BMV2.

- SmartNICs:
  P4 is not the right DSL for SmartNICs even if Intel may say so because they want to 
  sell SmartNICs and have a brand with P4. SmartNICs are much more flexible, and programming then
  in P4 constraints you. If you don't know anything better, try eBPF. That is AFAIK the 
  most widely adopted DSL today to span general purpose CPU (host/router CPU), and SmartNICs.

  SmartNICs are a great target FPE for forwarding plane features you do not need on every hop,
  but for example only on few "can be more expensive" hops. WAN interface of site-edge-routers
  (Campus, Home, WAN) for example. But do not try to assume they would be used beyond that
  (on every hop == especially on higher aggregation speed links!).

- P4/Coin for higher-than-forwarding-plane features:
  With Tofino disappearing, i think we should caution researchers to NOT invest cycles trying to
  implement solutions on P4 with the explicit purpose of "abusing" Tofino, aka: Program a
  non-forwarding-plane problem solution in P4 in the expectation that a fast P4 exeuction
  product like Tofino would be a great alternative/competition to other execution platforms
  (SmartNIC, CPU,...). We had several of those great ideas presented/shown in Coin, they
  where all about higher than forwarding-plane "compute" problems. But very quickly, with
  Tofino aging, i am sure CPU or SmartNIC slutions for the same problems can easily be
  shown to be more cost-effective.

- Long-term forwarding plane research:
  The rejection of the industry (outside Intel) to support P4 for researchers on their router/switch
  forwarding planes (for a decade now) should primarily be a trigger for research funding
  such as from NSF/EU to better enable long-term forwarding plane research. IMHO there
  should simply be a single well funded effort for a Network Programming Element (NPE) equivalent
  to RISC-V. Aka: fully open source architecture and instruction set for a research NPE,
  competitive to industry products. There are lot of other benefits from such an effort
  that could help make that happen.

Cheers
    Toerless

On Mon, May 15, 2023 at 03:33:42AM -0700, Marie-Jose Montpetit wrote:
> Interesting article for out community.
> 
> mjm
> 
> Marie-José Montpetit, Ph.D.
> [e-mail]
> 
> 
> 
> From: Larry Peterson from Systems Approach <[e-mail]>
> <[e-mail]>
> Reply: Larry Peterson from Systems Approach
> <reply+1zsws7&rqag0&&[e-mail]>
> <reply+1zsws7&rqag0&&[e-mail]>
> Date: May 15, 2023 at 3:55:52 AM
> To: [e-mail] <[e-mail]> <[e-mail]>
> Subject:  The Future of P4, Revisited
> 
> The P4 Workshop was a couple weeks ago, and as General Chair, I went into
> it with a fair amount of trepidation. My concern was that Intel’s
> announcement earlier this year that they’re cancelling development of the
> Tofino 3 switching chip would have a chilling effect, not only on the
> Workshop, but also on the future of P4. That concern has been voiced in
> several forums
>  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
> Open in app
> <https://open.substack.com/pub/systemsapproach/p/the-future-of-p4-one-perspective?utm_source=email&redirect=app-store>
> or online
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dG9rZW49ZXlKMWMyVnlYMmxrSWpvME5qVTNOakl5TkN3aWNHOXpkRjlwWkNJNk1USXdOakF4TlRreExDSnBZWFFpT2pFMk9EUXhNemN6TkRrc0ltVjRjQ0k2TVRZNE5qY3lPVE0wT1N3aWFYTnpJam9pY0hWaUxUSTRNemswTXlJc0luTjFZaUk2SW5CdmMzUXRjbVZoWTNScGIyNGlmUS50UVhzNmRNSG1GRWdMS0I0dlVHNHVHSlRVU0UwTjdVZEFIMXFzajg2bUNBIiwicCI6MTIwNjAxNTkxLCJzIjoyODM5NDMsImYiOnRydWUsInUiOjQ2NTc2MjI0LCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTAiLCJzdWIiOiJsaW5rLXJlZGlyZWN0In0.6L7-AmSfpdLfVXnKIHzqDqYX9u_tCmRVPJC2N9oP8gU?>
> The Future of P4, Revisited
> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=post-email-title&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
> 
> Larry Peterson
> <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> May 15
> <https://substack.com/redirect/ff29b5a3-7ea9-4b83-b888-b0c517b2bedd?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> 
> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
> 
> Share
> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&utm_content=share&action=share&triggerShare=true&isFreemail=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA>
> 
> 
> The P4 workshop has now been chaired by both co-founders of Systems
> Approach, but this year the P4 landscape has shifted again with Intel’s
> announcement that Tofino 3, its flagship P4-powered switching chip, would
> not go ahead. There is much more to P4 than Tofino, however, as we explore
> in this week’s newsletter.
> ------------------------------
> 
> The P4 Workshop
> <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> was a couple weeks ago, and as General Chair, I went into it with a fair
> amount of trepidation. My concern was that Intel’s announcement
> <https://substack.com/redirect/a0831b60-fff8-4ded-88c7-f3e6ef7e1e54?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> earlier this year that they’re cancelling development of the Tofino 3
> switching chip would have a chilling effect, not only on the Workshop, but
> also on the future of P4. That concern has been voiced in several forums,
> including SIGCOMM’s Slack workspace
> <https://substack.com/redirect/300dcab9-6056-4c82-9627-1e08ea449b0e?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>,
> with members of the P4 Advisory Board
> <https://substack.com/redirect/2fcc130d-cf74-4d68-be4e-d531cbb379b4?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> making reassuring pronouncements in various settings. (See for example,
> Nick McKeown’s post to the P4 Forum
> <https://substack.com/redirect/b0f9bc57-686d-4962-8676-c234ebcbd12f?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>,
> and Nick along with Nate Foster and Jennifer Rexford discussing the future
> of Network Programmability on The Networking Channel
> <https://substack.com/redirect/f2016908-8afb-4538-be3d-5d0e6aab0b3c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> ).
> 
> I won’t try to give a point-by-point replay of what Nick, Nate, and Jen and
> others have been saying, except to observe that at a high level it can be
> summarized as follows:
> 
> *Programmable Networks  >>  P4 Language  >>  Tofino Switching Chip*
> 
> They point out, for example, that Tofino is just one of many interesting
> backend targets for P4 programs (SmartNICs
> <https://substack.com/redirect/f5d04cdb-5915-42ab-9904-a20c38eeab33?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> and IPUs
> <https://substack.com/redirect/a0ddc803-4ea3-4248-96a1-bdc33f86cd15?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> being the next “big deal”) and P4 is one of many tools being used to inject
> functionality into the end-to-end network path (DPDK and eBPF being two
> active projects that people are integrating with P4). Ultimately, the value
> of programmability comes from having visibility and control over the
> network, and there are many complementary approaches to making that happen.
> With that background, I do have three takeaways from what turned out to be
> an interesting and vibrant two days at the P4 Workshop (despite my initial
> concerns).
> 
> First, we’re often so focused on P4 as a tool to program the forwarding
> pipeline that we forget the other half of its value proposition: It also
> provides a way to specify the behavior of a pipeline (independent of how
> that pipeline is implemented). We talk about this idea, and the value of
> being able to auto-generate the Control API, in the P4 chapter of our SDN
> Book
> <https://substack.com/redirect/4b87822a-70c6-4ffa-864a-45637b26909c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
> Rob Sherwood made a similar argument
> <https://substack.com/redirect/2442a19b-b82c-410c-8692-f50b89733875?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> at the P4 Workshop. It is now becoming a reality as companies like Google
> are starting to use such behavioral definitions as a Hardware Abstraction
> Layer (see Parveen Patel’s Keynote
> <https://substack.com/redirect/e656a164-3fef-497b-9628-ffeb700108af?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> at the Workshop). This makes me hopeful that we are rapidly approaching the
> day when a P4 program (plus the generated P4RT interface) will become the
> standard way network providers specify their requirements to network
> vendors, and proposed new features (whether proprietary or standard) will
> be specified by a P4 program (potentially augmenting the intuition and
> design rationale presented in an RFC).
> 
> *As an aside, I couldn’t help but notice the similarities between the
> architecture Parveen described and the way P4 has been used to program the
> forwarding plane of the 5G Mobile Core
> <https://substack.com/redirect/64e4cac9-e6fd-4652-8e36-09f50ff1305c?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
> Both include a P4-based “abstract forwarding model” that’s independent of
> the underlying implementation details*.
> 
> Second, it is common to divide forwarding pipelines into “programmable”
> versus “fixed function”, but this glosses over what might be the more
> important distinction: whether the pipeline is *open* or *closed*. Even
> “fixed function” pipelines are increasingly flexible–it’s just a question
> of how restrictive the vendor is in who they allow to make changes. This
> restriction may have the biggest impact on researchers who want to
> experiment with a new feature (especially ones that do not yet have a
> proven market), but maybe less so in the commercial world where incentives
> to make changes are (arguably) well-defined. Using P4 as the “spec
> language” (as I just outlined) has the potential to accelerate the process
> on the commercial side. On the research side, there is a strong argument in
> favor of using Tofino 2 to demonstrate the feasibility and value of new
> ideas (12.8 Tb/s still makes for a credible Proof-of-Concept), and
> repeating the refrain yet again, P4-as-spec makes for a compelling tech
> transfer story. If that were to happen, it would be interesting to see how
> vendors and chip designers adapt to reduce their spec-to-hardware
> implementation overhead. I would argue that programmable forwarding planes
> have a time-to-market advantage even for closed solutions.
> 
> Third, our focus on quantifiable metrics makes it easy to forget about the
> less quantifiable aspects of programmability. At its core, P4 is a
> programming language that does a good job of abstracting the essence of a
> packet forwarding pipeline. It is enormously impressive that a P4 program
> can be compiled onto a PISA-based
> <https://substack.com/redirect/01cfb256-04e7-49f8-9600-55e1471955b8?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> switching chip that has the same performance, die area, cost, and power
> consumption of a fixed-function ASIC (and that equivalency was probably
> necessary for P4 to be taken seriously), but hitting that quantifiable mark
> is not sufficient. Well-designed languages are software tools that bring
> clarity to the intellectual challenge of programming. For me, the biggest
> “aha” moment of the Workshop was when Chris Sommers (long-time P4
> contributor and new co-Chair of the API Working Group) started rattling off
> all the functions he’d been involved in writing in P4, and remarking on how
> natural P4 makes that process. There is certainly room to add new language
> features as P4 expands its domain to include SmartNICs and IPUs—as Chris
> and the other WG chairs are now pursuing—but having an existing target to
> evolve is a great position to be in.
> 
> One common thread that weaves its way through these three takeaways is that
> Intel’s cancellation of the Tofino 3 chip is a potentially helpful forcing
> function: The P4 community has to demonstrate the value of the language
> without being buttressed by ever-improving performance numbers that have
> more to do with 7nm semiconductor technology than anything networking
> people have done. I saw a lot of evidence that exactly that is happening at
> last month’s workshop. The march to programmable networks is inevitable (in
> my view), and I’m still optimistic about the role P4 will play a central
> role.
> 
> Systems Approach is reader-supported and we are committed to keeping our
> books and articles open to all. To receive new posts and support our work,
> consider becoming a free or paid subscriber.
> 
> Upgrade to paid
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3N1YnNjcmliZT90b2tlbj1leUoxYzJWeVgybGtJam8wTmpVM05qSXlOQ3dpYVdGMElqb3hOamcwTVRNM016UTVMQ0psZUhBaU9qRTJPRFkzTWprek5Ea3NJbWx6Y3lJNkluQjFZaTB5T0RNNU5ETWlMQ0p6ZFdJaU9pSmphR1ZqYTI5MWRDSjkuLW5LYUMtRE1weEw3Smp0bHh4U1I0aEFFLXQ2QWltZ0lENFlFRC1BbWJicyZ1dG1fc291cmNlPXBvc3QiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.lWjTeCLgxCt9OYaegbWWppVXvEzTeJBQs6psa45Oowg?&utm_medium=email&utm_source=subscribe-widget-preamble&utm_content=120601591>
> 
> ------------------------------
> 
> We continue to run into people who want to translate our books into other
> languages, and if you are one of them, you should definitely reach out to
> us. The latest entrant is a Portuguese translation
> <https://substack.com/redirect/e0ad5652-867f-4d2c-8cd6-3fc8b0f8eda0?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>
> of our Private 5G book by Edmar Candeia Gurjão. You can find other
> translations of our books here
> <https://substack.com/redirect/0d0dad50-a3db-42e9-9ad6-95318436a540?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
> 
> 
> You can follow us on Mastodon
> <https://substack.com/redirect/3e6fc5cc-6a89-4a15-97bf-231cf12de9ec?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
> 
> 
> You're currently a free subscriber to Systems Approach
> <https://substack.com/redirect/3d94c242-4758-4bf3-9e1d-62b5ce3f23ab?j=eyJ1IjoicnFhZzAifQ.CwdsLjQyIcRXfYRGUucLsXfrvggM3KKa9Z1jxtXNWH0>.
> For the full experience, upgrade your subscription.
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3N1YnNjcmliZT90b2tlbj1leUoxYzJWeVgybGtJam8wTmpVM05qSXlOQ3dpYVdGMElqb3hOamcwTVRNM016UTVMQ0psZUhBaU9qRTJPRFkzTWprek5Ea3NJbWx6Y3lJNkluQjFZaTB5T0RNNU5ETWlMQ0p6ZFdJaU9pSmphR1ZqYTI5MWRDSjkuLW5LYUMtRE1weEw3Smp0bHh4U1I0aEFFLXQ2QWltZ0lENFlFRC1BbWJicyZ1dG1fc291cmNlPXBvc3QiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.lWjTeCLgxCt9OYaegbWWppVXvEzTeJBQs6psa45Oowg?&utm_source=substack&utm_medium=email&utm_content=postcta>
> 
> Upgrade to paid
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3N1YnNjcmliZT90b2tlbj1leUoxYzJWeVgybGtJam8wTmpVM05qSXlOQ3dpYVdGMElqb3hOamcwTVRNM016UTVMQ0psZUhBaU9qRTJPRFkzTWprek5Ea3NJbWx6Y3lJNkluQjFZaTB5T0RNNU5ETWlMQ0p6ZFdJaU9pSmphR1ZqYTI5MWRDSjkuLW5LYUMtRE1weEw3Smp0bHh4U1I0aEFFLXQ2QWltZ0lENFlFRC1BbWJicyZ1dG1fc291cmNlPXBvc3QiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.lWjTeCLgxCt9OYaegbWWppVXvEzTeJBQs6psa45Oowg?&utm_source=substack&utm_medium=email&utm_content=postcta>
> 
> Like
> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&isFreemail=true&submitLike=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJyZWFjdGlvbiI6IuKdpCIsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMjgzOTQzIiwic3ViIjoicmVhY3Rpb24ifQ.vN6AE7iLQvKkv98V0RaNKEWEX9UdKbdHQdc3nu95Kyg&utm_medium=email>
> Comment
> <https://substack.com/app-link/post?publication_id=283943&post_id=120601591&utm_source=substack&utm_medium=email&isFreemail=true&comments=true&token=eyJ1c2VyX2lkIjo0NjU3NjIyNCwicG9zdF9pZCI6MTIwNjAxNTkxLCJpYXQiOjE2ODQxMzczNDksImV4cCI6MTY4NjcyOTM0OSwiaXNzIjoicHViLTI4Mzk0MyIsInN1YiI6InBvc3QtcmVhY3Rpb24ifQ.tQXs6dMHmFEgLKB4vUG4uGJTUSE0N7UdAH1qsj86mCA&utm_source=substack&utm_medium=email>
> Restack
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL3AvdGhlLWZ1dHVyZS1vZi1wNC1vbmUtcGVyc3BlY3RpdmU_dXRtX3NvdXJjZT1zdWJzdGFjayZ1dG1fbWVkaXVtPWVtYWlsJmFjdGlvbj1yZXN0YWNrLWNvbW1lbnQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.hfc7rMzN53dfpysbLYXxmtkw9siB9dPnC71KIavCOTo?&utm_source=substack&utm_medium=email>
> 
> 
> © 2023 Systems Approach, LLC
> 548 Market Street PMB 72296, San Francisco, CA 94104
> Unsubscribe
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zeXN0ZW1zYXBwcm9hY2guc3Vic3RhY2suY29tL2FjdGlvbi9kaXNhYmxlX2VtYWlsP3Rva2VuPWV5SjFjMlZ5WDJsa0lqbzBOalUzTmpJeU5Dd2ljRzl6ZEY5cFpDSTZNVEl3TmpBeE5Ua3hMQ0pwWVhRaU9qRTJPRFF4TXpjek5Ea3NJbVY0Y0NJNk1UWTROamN5T1RNME9Td2lhWE56SWpvaWNIVmlMVEk0TXprME15SXNJbk4xWWlJNkltUnBjMkZpYkdWZlpXMWhhV3dpZlEuRXdaUkN3ZEdjdG8xdFdwbmRVd3h2eXZKUk5pVmlOX2h1ZDBVNERDbVZYayZleHBpcmVzPTM2NWQiLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.l6lDN2unh8JkUjeYEXjKiNFu7akDZ_FyOQUxaYockVM?>
> 
> [image: Start writing]
> <https://substack.com/redirect/2/eyJlIjoiaHR0cHM6Ly9zdWJzdGFjay5jb20vc2lnbnVwP3V0bV9zb3VyY2U9c3Vic3RhY2smdXRtX21lZGl1bT1lbWFpbCZ1dG1fY29udGVudD1mb290ZXImdXRtX2NhbXBhaWduPWF1dG9maWxsZWQtZm9vdGVyJmZyZWVTaWdudXBFbWFpbD1tYXJpZUBtam1vbnRwZXRpdC5jb20iLCJwIjoxMjA2MDE1OTEsInMiOjI4Mzk0MywiZiI6dHJ1ZSwidSI6NDY1NzYyMjQsImlhdCI6MTY4NDEzNzM0OSwiZXhwIjoxNjg2NzI5MzQ5LCJpc3MiOiJwdWItMCIsInN1YiI6ImxpbmstcmVkaXJlY3QifQ.GtVGEVhyYZ__pv96bIPsssoL78rJrnvkIgcdwC4WQfs?>

> -- 
> Coin mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/coin


-- 
---
[e-mail]