Re: Proposal: Adopt State Synchronization into HTTPbis

[Kévin Dunglas <[e-mail]>]

As the main author of the Mercure specification (up-to-date spec:
https://mercure.rocks/spec, older I-D:
https://www.ietf.org/archive/id/draft-dunglas-mercure-06.html), I support
this goal.

Our previous attempt to "promote" the Mercure spec as a RFC didn't get much
feedback (
https://lists.w3.org/Archives/Public/ietf-http-wg/2020JulSep/0020.html), so
we are in the process of submitting the spec as independent submission.

We would be glad to work with the Baid and the PREP team to get a single
set of standards instead.

Best regards,

On Wed, Nov 1, 2023 at 3:11 AM Michael Toomim <[e-mail]> wrote:

> At IETF 118 I will present a proposal to adopt State Synchronization work
> into HTTPbis:
>
> Braid-HTTP:
> https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http [1]
>
>
> HTTP currently provides State *Transfer* of a document between client and
> server. This made sense in 1992 when the web was mostly static, and pages
> were written by hand. Caching was simple. However, today's pages are
> dynamic, use Javascript and Ajax, and users expect realtime updates.
> Today's web needs robust State *Synchronization*; not just Transfer.
>
> We have an opportunity to add State Synchronization to the WWW in a robust
> & general way. Four simple extensions (below) transform HTTP into a
> general-purpose State Synchronization protocol—keeping simple cases simple,
> while also enabling the most advanced distributed synchronization (OT and
> CRDT) algorithms. These support multiple writers, making simultaneous
> edits, over arbitrary network conditions (offline or online), while
> guaranteeing consistency with peer-to-peer merge-semantics. HTTP can become
> a general-purpose Distributed State Abstraction: a protocol that guarantees
> your local cache of state is always editable, and always up-to-date.
>
> We first discussed this idea at IETF 104 and 105 in Prague and Montreal.
> It received great enthusiasm and feedback, which was incorporated into
> draft 02 in March 2020. We tested this draft extensively over the past few
> years in the braid.org group, against a variety of algorithms, apps, and
> browsers, and last week we published draft 03, which I feel confident is a
> general and performant approach to state synchronization, supporting every
> known CRDT and OT algorithm (!), and fitting easily into today's web.
>
> But adopting this won't be a monolithic project like WebRTC. The approach
> we propose is actually to spiff up three aspects of existing HTTP:
>
> 1. Subscriptions (sse)
> 2. Version History (etag, max-age)
> 3. Patch Formats (PATCH)
>
> ...and then to tie them together with one new concept:
>
>  4. the Merge-Type header
>
> The first three improvements will be useful on their own. In fact, there
> are already efforts underway to improve them:
>
>    - *Subscriptions* are being proposed in PREP [2] and Mercure [3] to
>    address limitations in SSE and WebSub.
>    - *Patch Formats* are being extended for Byte-Range-Patch [4], which
>    needs a generalized Content-Range header.
>    - Resumeable Uploads [5] needs to send a series of *Patches* (called
>    parts), until the *Version* on the server is complete.
>    - HTTP Cache Invalidation API [6] proposes a standard for servers to
>    push updates to the CDNs that *Subscribe* to their state.
>
> These improvements are needed for State Synchronization too. By adopting
> the higher-level goal of State Synchronization, we gain a unified
> perspective, to generalize these disparate efforts into a more powerful and
> elegant HTTP. We will address the above use-cases, and enable a broad array
> of new ones.
>
> We verified this framework with a number of implementations in the
> braid.org group over the last few years.
>
> - Protocol implementations: braid-http [7], braid-protocol [8], wai-braid
> [9]
> - Algorithm integrations: sync9 [10], diamond-types [11], automerge/yjs
> [12], sharedb [13], shelf [14]
> - Browser extension: braid-chrome [15]
> - Applications: PeeryView.org [16], Simple Braid Chat [17], Braid Wiki
> [18]
> - Developer APIs: statebus [19], redwood [20]
>
> I'm making a demo video to make these more concrete. It will appear on
> this thread soon.
>
> Separately, I am IETF-Dispatching the idea of forming a general State
> Synchronization group. (Email thread here.
> <https://mailarchive.ietf.org/arch/msg/dispatch/wCrAiHwWvlK9netFi71nz3FF7Aw/>)
> I believe the Braid-HTTP draft is ready for standardization, but this
> general group could take on any further issues that are too abstract for
> HTTPbis.
>
> In summary, I am proposing that HTTPbis adopt the general goal of
> supporting State *Synchronization* (not just Transfer) in HTTP, which
> entails making general extensions for (1) Subscriptions, (2) Version
> History, and (3) Patch Formats, and then offering a (4) Merge-Type to tie
> it all together. These will make for a very powerful upgrade.
>
> I kindly request feedback on this idea! See the draft [1] for details.
>
> Michael
>
> References:
> [1] https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http
> [2]
> https://datatracker.ietf.org/doc/draft-gupta-httpbis-per-resource-events
> [3] https://datatracker.ietf.org/doc/draft-dunglas-mercure/
> [4]
> https://datatracker.ietf.org/meeting/117/materials/slides-117-httpapi-byte-range-patch-00
> [5] https://datatracker.ietf.org/doc/draft-ietf-httpbis-resumable-upload/
> [6] https://datatracker.ietf.org/doc/draft-nottingham-http-invalidation/
> [7] https://www.npmjs.com/package/braid-http
> [8] https://github.com/josephg/braid-protocol
> [9] https://github.com/braid-org/wai-braid
> [10] https://braid.org/sync9
> [11] https://github.com/josephg/diamond-types
> [12] https://braid.org/automerge
> [13] https://braid.org/demo/interoperate
> [14] https://braid.org/algorithms/shelf
> [15] https://github.com/braid-org/braid-chrome
> [16] https://peeryview.org/about
> [17] Simple Braid Chat: http://invisible.college:3007
> [18] Antimatter Wiki:
> https://github.com/braid-org/braidjs/tree/master/antimatter_wiki and
> https://wickie.invisible.college
> [19] https://stateb.us/
> [20] https://github.com/brynbellomy/redwood
>