[dispatch] A State Synchronization Working Group

[Michael Toomim <[e-mail]>]

I would like to dispatch the idea of forming a general State 
Synchronization Working Group, to coordinate duplicate efforts in 
defining various state synchronization protocols across existing IETF 
working groups.

Many networking protocols implement some form of state synchronization.  
At the application layer, SCIM, SIP, COAP, ALTO, NETCONF, JMAP, CalDAV, 
and IPP all define state synchronization protocols over HTTP.  Many 
non-IETF protocols also implement sync-over-HTTP for specific uses, such 
as WebSub (to synchronize blogs), ActivityPub (to synchronize social 
feeds), and Matrix (to synchronize chat).  Outside of HTTP, we do state 
synchronization in DNS, IMAP, LDAP, NFS, RSYNC, and Git.  Dropping to 
the lower-level OSI layers, we have protocols to synchronize IP routing 
tables, network reachability, and header compression dictionaries like 
ROHC, OSPFv2, IS-IS, and BGP.  State synchronization also comes up in 
distributed systems and databases, with algorithms such as OT and CRDT, 
and each system defines its own distinct protocol.

We design these as separate protocols because the *general* version of 
the State Synchronization Problem has seemed too challenging to tackle. 
However, the last decade of research has brought breakthroughs in 
general state synchronization algorithms that allow (a) multiple 
editors, (b) editing arbitrary data types, (c) over a distributed 
network, (d) of arbitrary network topology, (e) experiencing arbitrary 
network partitions and delays, (f) with arbitrary merge resolution 
semantics—all with reasonable (and improving) performance.  These 
general algorithms bring up the possibility of designing general 
*protocols* for state sync.

If IETF working groups could rely on general standards for state sync, 
they could eliminate redundant consensus and specification work, as well 
as implementation work, because implementers could re-use common 
libraries and algorithms instead of writing their own algorithms.  
Furthermore, general libraries could afford investment in advanced sync 
features such as peer-to-peer networking, offline modes, 
delta-compression, merge resolution, consistency guarantees, and 
fast-replay; which applications might not implement on their own, but 
now could benefit from for free. This would improve networking 
robustness across the board. Finally, interoperability would improve.  
Network sysadmins, for example, could inspect and debug the state and 
history of their routers, emails, web applications, email, and file 
systems with intercompatible tools over a general protocol.

This new working group would integrate research with practice by working 
in symbiosis with existing IETF Working Groups.  Our new group would 
gather experts and researchers in state synchronization and interface 
with individual Working Groups.  It would learn the needs of individual 
working groups, and produce general knowledge, protocols, and libraries 
in return.

We have had initial success with this model in the informal Braid.org 
group, which was roughly modeled on an IETF Working Group, and has 
produced the Braid-HTTP internet draft that is coming up for discussion 
in HTTPbis.  The Braid group has met every two weeks for 2.5 years on 
zoom, with average attendance of 4 or 5 people, and has produced a 
number of research results in generalizing performant state synchronization:

   - Diamond-Types <https://github.com/josephg/diamond-types>: World's 
fastest P2P text editing CRDT
   - List-CRDTs <https://github.com/josephg/reference-crdts>: First text 
editing CRDT with general pluggable merge semantics (video presentation 
<https://braid.org/video/https://invisiblecollege.s3-us-west-1.amazonaws.com/braid-meeting-10.mp4#305>)
   - Antimatter: World's first history-pruning P2P text editing CRDT 
(draft <https://braid.org/antimatter>) (implementation 
<https://www.npmjs.com/package/@braid.org/antimatter>)
   - Sync9 <https://braid.org/sync9>: First true replace text CRDT
   - Portals <https://braid.org/meeting-62/portals>: Generalized patch 
semantics for moves, replaces, splices
   - Time Machines <https://braid.org/drafts/time-machines>: 
Generalization of OT & CRDT theory
   - Braid-HTTP 
<https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http>: 
Synchronization for HTTP

We could imagine formalizing the Braid group into this State 
Synchronization group. It would look more broadly at synchronization 
than any individual application, and would produce specs that could be 
offered to other groups for standardization, similar to how the QUIC 
group seeded HTTP/3.

I am very curious what the IETF thinks about this idea for a working 
group, and would be very grateful to hear thoughts and opinions from 
everyone.  Thank you very much!

Michael

[1] Relevant Braid-HTTP draft: 
https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http