Re: Mnot's Pub/Sub for the Web

[Kevin Marks <[e-mail]>]

Does WebSub (formerly PubSubHubbub) fit this case?

On Tue, 22 Feb 2022, 9:25 pm Mike Bishop, <[e-mail]> wrote:

> This is a very interesting space, and I’m glad we have two such solid
> contenders.  I’m not convinced this fits squarely within HTTP’s mandate, as
> this seems more like a protocol on top of HTTP than a pure extension to
> HTTP.  Perhaps like OHAI, there might be enough interest to warrant a
> dedicated working group?
>
>
>
> *From:* Kévin Dunglas <[e-mail]>
> *Sent:* Tuesday, February 22, 2022 12:56 PM
> *To:* Michael Toomim <[e-mail]>
> *Cc:* HTTP Working Group <[e-mail]>; Mark Nottingham <
> [e-mail]>
> *Subject:* Re: Mnot's Pub/Sub for the Web
>
>
>
> Thanks for bringing this topic to the list again!
>
>
>
> On the Mercure side, the spec has stabilized. Several open-source and
> proprietary implementations are available (
> https://mercure.rocks/spec#implementation-status), and adoption is
> growing: 2.7K stars on GitHub, dozens of open source projects using it,
> large companies publicly declaring use...
>
>
>
> Many new use cases have been reported on the bug tracker over the years,
> and we improved the spec to cover most of them. Some minor issues still
> need to be handled (https://github.com/dunglas/mercure/labels/spec), but
> we're very soon to publish the final version of the specification.
>
>
>
> As demonstrated by the discussions on Hacker News, Mark's great article,
> and by the adoption of Mercure, the community is in demand of a pub/sub
> standard for web resources.
>
>
>
> Even if most discussions occurred on GitHub, Slack, Twitter, and other
> channels instead of on IETF mailing lists, the Mercure spec is now
> implemented by production-grade "running code" and has reached "rough
> consensus".
>
>
>
> Mercure is less ambitious than Braid. Its scope is more limited. It is
> focusing on providing a simple pub/sub protocol for web content proved
> working with the current web infrastructure (web browsers, proxies,,
> firewalls, etc). In its current state, it doesn't require any JS library or
> polyfill client-side.
>
>
>
> The spec is very similar to the WebSub specification from the W3C, but
> mainly targets web browsers instead of servers. As WebSub, Mercure uses a
> hub to distribute web resources, which allows implementing the protocol
> easily even in legacy applications, with languages not designed to handle
> long-living connections (e.g. PHP), and when using modern infrastructure
> such as serverless and edge computing platforms (
> https://dunglas.fr/2019/07/mercure-real-time-apis-for-serverless-and-beyond/).
> Unlike WebSub, Mercure natively supports authorization, end-to-end
> encryption, and state reconciliation. Both clients and servers can be
> publishers.
>
>
>
> Currently, Mercure only allows using SSE as transport, but we'll maybe
> allow using other transports such as WebSockets and Web Transports,
> probably as extensions to the current spec, to cover use case such as
> transmitting non-base64-encoded binary data (
> https://github.com/dunglas/mercure/issues/616).
>
>
>
> Braid is very interesting and has a much broader scope (state
> synchronization, P2P, etc). It also requires more changes to the current
> software stack to be natively supported by the web platform. Mercure
> overlaps only with the "subscribe" feature of Braid, and I've the feeling
> than Braid could use Mercure (and probably WebSub too) for its subscribe
> feature, at least in a first iteration.
>
>
>
> I wonder how we can move forward regarding the standardization of a
> pub/sub protocol for web content and web browsers. Even if Mercure gained
> traction outside of the IETF, it hasn't on this group. I was thinking about
> proposing the final version of the spec as an independent-track RFC, or to
> the W3C as it is very close to WebSub, and is also related to the other
> specs published by the Social Web Working Group (ActivityPub, and even
> Solid). But as the this topic is discussed again, maybe could we work on a
> pub/sub protocol here?
>
>
>
>
>
>
>
> On Sun, Feb 20, 2022 at 10:39 AM Michael Toomim <[e-mail]> wrote:
>
> Hello, HTTP!
>
> Today Mark Nottingham posted a great articulation of the issues
> programmers face when choosing between using SSE, WebSockets, and
> WebTransports:
>
> https://www.mnot.net/blog/2022/02/20/websockets
>
> I'll attempt to summarize Mark's beautiful insight as: in almost all
> cases, what the programmer *really* wants is a Pub/Sub protocol, not an
> arbitrary socket. And we could standardize a Pub/Sub protocol, and that
> would have great benefits.
>
> These benefits are real and I think could improve performance
> dramatically. CDNs could cache realtime updates, not just static data.
>
> However, I'll take Mnot one further, and propose that when a programmer is
> choosing a Pub/Sub protocol, what he *really* wants is a State
> Synchronization protocol, not an arbitrary Pub/Sub protocol.
>
> He wants to Subscribe specifically to *state updates*. He wants to Publish
> specifically *updates to state*.
>
> What we need is not a general Pub/Sub standard, but specifically a State
> Synchronization standard. State Synchronization is a constrained type of
> general Pub/Sub. And we'll need to constrain Pub/Sub in this way to address
> some of the issues Mark brings up, such as:
>
> > There are also some architectural/philosophical concerns about how
> non-final responses **relate to the state of the resource**.
>
> The relationship between a server's "responses" and the "state of the
> resource" is what a State Synchronization protocol defines. And, in fact,
> we have two proposed solutions to State Synchronization in the IETF!
>
> Braid:
> https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http
> Mercure:    https://datatracker.ietf.org/doc/draft-dunglas-mercure/
>
> I am seeing a growing awareness that HTTP needs to add State
> Synchronization abilities, as well as excitement about the new fundamental
> power it gives programmers on the web.
>
> These protocols transform HTTP from a State *Transfer* into a State
> *Synchronization* protocol. Whereas a transfer protocol can move a resource
> from server to client in a single request/response, it requires an
> application programmer to take over if the resource ever changes after the
> response completes. That sucks for programmers. A synchronization protocol
> provides a much better programming abstraction. The programmer just says "I
> want state X", and can assume it will be kept up-to-date by the protocol.
>
> If we standardize this, we also get CDNs that automatically cache dynamic
> content (the stuff currently hidden within websockets), just as easily as
> they cache static content today. We get collaborative editing and offline
> modes available in web apps for free. We also take an important step
> towards decentralizing the web, by creating an open standard for the
> trickiest part of decentralized app development — data synchronization —
> that is compatible with P2P CRDT and OT algorithms.
>
> Since this all seems to be coming together, I would like to know what
> HTTPbis as a group thinks. Is there interest in this topic?
>
> If so, what aspects might we want to work on?
>
>