Re: Publication of the Mercure Protocol

[Michael Toomim <[e-mail]>]

Hi Kévin I'm excited to discover this spec!

A group of us are working on a related draft called Braid, which also adds push-updates to HTTP:

Draft:	https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http <https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http>
Web:	https://braid.news <https://braid.news/>

So it seems we are all tackling the same problem, but aren't aware of each other! Let's solve that.
How about we work together to make our systems interoperable, and find consensus on a single spec?

To start, I've read the Mercure draft and am leaving a review. I'm also cc'ing the braid-http list, in case any Braidly people want to chime in. I'd love to see your review of Braid as well.

# My review of the Mercure draft

Push-updates is an important, and common problem. A number of IETF specs have devised their own solutions: (e.g. Calendar Synchronization <https://tools.ietf.org/html/rfc6578>, JMAP Email Synchronization <https://tools.ietf.org/html/rfc8620>). Our lives would be much improved if we spoke the same push-update (aka "synchronization") language.

I like the general SSE approach. Braid similarly streams updates over a long-running GET, although it doesn't use the other parts of SSE.

However, I think the Mercure spec would be simpler, cleaner, and more general with two major changes:
Combine the separate Discovery and Subscription steps into a single request/response.
Unbundle the Authorization method. Let people use their own methods.
These changes would simplify and generalize the spec, and be more compatible with Braid. I provide a more detailed analysis below, and then conclude with some small suggestions.

## 1. Combine Discovery and Subscription into a single request/response

To get a subscription, this spec requires the client to make two requests:
A GET to the server to discover how to subscribe
A separate GET (using SSE) to a "hub" to actually subscribe

It is actually possible (and I think simpler) to combine these into a single GET request. This is how Braid works. If a GET request includes a "Subscribe" header, then the response will return the current version, but then also stay open, and stream all new versions as updates like SSE does.

Combining these requests eliminates a round trip of network latency, which is great; but it also simplifies the protocol. In particular, there are a number of new concepts introduced in this spec (Topic, Hub, Publisher, Subscriber) that seem like re-inventions of existing HTTP concepts (Server, Resource, Client issuing put, Client issuing get). I presume that you didn't re-use the existing HTTP concepts because you needed these to run over a separate SSE connection, which doesn't have built-in HTTP semantics. However, if you instead stream the updates to any resource within the existing HTTP connection, you can re-use all of its existing HTTP semantics. Then you won't need to re-specify these concepts in the Subscription phase, and the spec will be simpler and easier to digest and adopt.

Is it possible to combine these into a single request? Is there any reason why these need to be separate in the protocol?

## 2. Unbundle Authorization

Another benefit of moving Subscriptions into the existing HTTP requests is that we don't need to invent a new authorization method — the subscriptions can use whatever the existing HTTP requests use.

However, if I'm missing something, and we do have a need to specify a particular style of authorization, can we at least move it into a separate spec?  There are many ways to authenticate, and it would be nice if we could find consensus on authorization separately from how to push updates.

Ok, that's it for the big items! I'd love to hear what you think about these design decisions, and see if we can make these protocols compatible.

## Details

- Spec says the hub is a server, and the Link header specifies the hub, but the examples provided are Links to full resources with a path (e.g. https://example.com/.well-known/mercure), not just to a server (e.g. https://example.com). So which is it: a hub/server, or a full resource with a path?

- Spec says:
    "The URL of the hub MUST be the "well-known" fixed path "/.well-known/mercure"
  But the point of .well-known <https://tools.ietf.org/html/rfc8615> is that you don't need to specify it anywhere, and you don't need the discovery step. So I think you can eliminate this header entirely.

- Last-event-id: If I'm not mistaken, this only handles linear time, with a single writer. Can this spec support multiple writers modifying a resource simultaneously? That would produce two simultaneous last-event-ids, neither of which has occurred more recently than the other.
  - The Braid specification calls this "Parents:".

SSE stream ID:
   - Why must the ID be an IRI?  Does the IRI reference something?
   - It'd be more specific to call this a "Version" instead. It's not just any ID! It's a Version ID.
   - Braid calls this "Version", and allows it to be any unique string, not just an IRI.

Reconnection
  - The Mercure spec provides clients with no guarantee that they will receive all updates upon reconnection.
  - Braid servers, on the other hand, can guarantee a duration of time for a client to reconnect and receive all updates. I would like to support this type of guarantee.

Active Subscriptions
 - Although I agree that this feature is useful, I'd rather this not be a part
   of the spec.  Do we have any interoperability use-cases where this needed?
   It seems like it would be fine for each web app to have its own method of
   tracking active subscriptions.  This method forces you to use JSON-LD.


> On Jul 8, 2020, at 7:44 AM, Kévin Dunglas <[e-mail]> wrote:
> 
> Hi all,
> 
> Late 2018, I published an Internet-Draft specifying a protocol called Mercure:
> Abstract
> 
>    Mercure is a protocol enabling the pushing of data updates to web
>    browsers and other HTTP clients in a fast, reliable and battery-
>    efficient way.  It is especially useful for publishing real-time
>    updates of resources served through web APIs to web and mobile apps.
> https://datatracker.ietf.org/doc/draft-dunglas-mercure/ <https://datatracker.ietf.org/doc/draft-dunglas-mercure/>
> 
> I just published the 7th version of the I-D. The protocol can now be considered stable and feature complete. It is already widely implemented and used, including by popular web frameworks such as Symfony. You can see the full list of implementations in the "Implementation Status" section of the I-D.
> I would like to go one step further and propose it as a RFC.
> 
> I was wondering if the HTTPbis Working Group could host the work on this protocol (this looks allowed by the "Other HTTP-Related Work" section of the charter)?
> 
> Also, I tried to register the link relation (https://github.com/protocol-registries/link-relations/issues/21 <https://github.com/protocol-registries/link-relations/issues/21>) and the "well-known" URI (https://github.com/protocol-registries/well-known-uris/issues/4 <https://github.com/protocol-registries/well-known-uris/issues/4>) used by the protocol, but it's not possible yet because the I-D isn't on any stream.
> I submitted a new version of the XML file with the following header:
> 
> <rfc version="3" ipr="trust200902" docName="draft-dunglas-mercure-07" submissionType="IETF" category="std" xml:lang="en" xmlns:xi="http://www.w3.org/2001/XInclude <http://www.w3.org/2001/XInclude>" consensus="true">
> 
> But it looks like it hasn't been taken into account by the tracker (https://datatracker.ietf.org/doc/draft-dunglas-mercure/ <https://datatracker.ietf.org/doc/draft-dunglas-mercure/>).
> 
> I must admit that the process to propose a RFC is still a bit unclear to me. Is this list the right place to propose and discuss this protocol? Should I create a new submission for the draft or is it possible to "update" the stream on the existing one?
> 
> Best regards,
> -- 
> Kévin Dunglas
> 
> https://dunglas.fr <https://dunglas..fr/> / @dunglas <https://twitter.com/dunglas>
>