Proposal: Adopt State Synchronization into HTTPbis

[Michael Toomim <[e-mail]>]

At IETF 118 I will present a proposal to adopt State Synchronization work into HTTPbis:

Braid-HTTP: https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http [1]

HTTP currently provides State *Transfer* of a document between client and server. This made sense in 1992 when the web was mostly static, and pages were written by hand. Caching was simple. However, today's pages are dynamic, use Javascript and Ajax, and users expect realtime updates. Today's web needs robust State *Synchronization*; not just Transfer.

We have an opportunity to add State Synchronization to the WWW in a robust & general way. Four simple extensions (below) transform HTTP into a general-purpose State Synchronization protocol—keeping simple cases simple, while also enabling the most advanced distributed synchronization (OT and CRDT) algorithms. These support multiple writers, making simultaneous edits, over arbitrary network conditions (offline or online), while guaranteeing consistency with peer-to-peer merge-semantics. HTTP can become a general-purpose Distributed State Abstraction: a protocol that guarantees your local cache of state is always editable, and always up-to-date.

We first discussed this idea at IETF 104 and 105 in Prague and Montreal. It received great enthusiasm and feedback, which was incorporated into draft 02 in March 2020. We tested this draft extensively over the past few years in the braid.org group, against a variety of algorithms, apps, and browsers, and last week we published draft 03, which I feel confident is a general and performant approach to state synchronization, supporting every known CRDT and OT algorithm (!), and fitting easily into today's web.

But adopting this won't be a monolithic project like WebRTC. The approach we propose is actually to spiff up three aspects of existing HTTP:

	1. Subscriptions (sse)
	2. Version History (etag, max-age)
	3. Patch Formats (PATCH)

...and then to tie them together with one new concept:

	 4. the Merge-Type header

The first three improvements will be useful on their own. In fact, there are already efforts underway to improve them:
*Subscriptions* are being proposed in PREP [2] and Mercure [3] to address limitations in SSE and WebSub.
*Patch Formats* are being extended for Byte-Range-Patch [4], which needs a generalized Content-Range header.
Resumeable Uploads [5] needs to send a series of *Patches* (called parts), until the *Version* on the server is complete.
HTTP Cache Invalidation API [6] proposes a standard for servers to push updates to the CDNs that *Subscribe* to their state.
These improvements are needed for State Synchronization too. By adopting the higher-level goal of State Synchronization, we gain a unified perspective, to generalize these disparate efforts into a more powerful and elegant HTTP. We will address the above use-cases, and enable a broad array of new ones.

We verified this framework with a number of implementations in the braid.org group over the last few years. 

	- Protocol implementations: braid-http [7], braid-protocol [8], wai-braid [9]
	- Algorithm integrations: sync9 [10], diamond-types [11], automerge/yjs [12], sharedb [13], shelf [14]
	- Browser extension: braid-chrome [15]
	- Applications: PeeryView.org [16], Simple Braid Chat [17], Braid Wiki [18]
	- Developer APIs: statebus [19], redwood [20]

I'm making a demo video to make these more concrete. It will appear on this thread soon.

Separately, I am IETF-Dispatching the idea of forming a general State Synchronization group. (Email thread here. <https://mailarchive.ietf.org/arch/msg/dispatch/wCrAiHwWvlK9netFi71nz3FF7Aw/>) I believe the Braid-HTTP draft is ready for standardization, but this general group could take on any further issues that are too abstract for HTTPbis.

In summary, I am proposing that HTTPbis adopt the general goal of supporting State *Synchronization* (not just Transfer) in HTTP, which entails making general extensions for (1) Subscriptions, (2) Version History, and (3) Patch Formats, and then offering a (4) Merge-Type to tie it all together. These will make for a very powerful upgrade.

I kindly request feedback on this idea! See the draft [1] for details.

Michael

References:
[1] https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http
[2] https://datatracker.ietf.org/doc/draft-gupta-httpbis-per-resource-events
[3] https://datatracker.ietf.org/doc/draft-dunglas-mercure/
[4] https://datatracker.ietf.org/meeting/117/materials/slides-117-httpapi-byte-range-patch-00
[5] https://datatracker.ietf.org/doc/draft-ietf-httpbis-resumable-upload/
[6] https://datatracker.ietf.org/doc/draft-nottingham-http-invalidation/
[7] https://www.npmjs.com/package/braid-http
[8] https://github.com/josephg/braid-protocol
[9] https://github.com/braid-org/wai-braid
[10] https://braid.org/sync9
[11] https://github.com/josephg/diamond-types
[12] https://braid.org/automerge
[13] https://braid.org/demo/interoperate
[14] https://braid.org/algorithms/shelf
[15] https://github.com/braid-org/braid-chrome
[16] https://peeryview.org/about
[17] Simple Braid Chat: http://invisible.college:3007
[18] Antimatter Wiki: https://github.com/braid-org/braidjs/tree/master/antimatter_wiki and https://wickie.invisible.college
[19] https://stateb.us/
[20] https://github.com/brynbellomy/redwood