Re: Publication of the Mercure Protocol

Michael Toomim <toomim@gmail.com> Wed, 15 July 2020 18:25 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0A21B3A0B00 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 15 Jul 2020 11:25:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.018
X-Spam-Level:
X-Spam-Status: No, score=-3.018 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vmbSGsBJRbCE for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 15 Jul 2020 11:25:52 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0527C3A07F4 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 15 Jul 2020 11:25:51 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1jvm3P-0004wz-Vb for ietf-http-wg-dist@listhub.w3.org; Wed, 15 Jul 2020 18:22:52 +0000
Resent-Date: Wed, 15 Jul 2020 18:22:51 +0000
Resent-Message-Id: <E1jvm3P-0004wz-Vb@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <toomim@gmail.com>) id 1jvm3N-0004wE-ND for ietf-http-wg@listhub.w3.org; Wed, 15 Jul 2020 18:22:49 +0000
Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <toomim@gmail.com>) id 1jvm3L-0001I9-5Y for ietf-http-wg@w3.org; Wed, 15 Jul 2020 18:22:49 +0000
Received: by mail-pf1-x433.google.com with SMTP id a24so2446675pfc.10 for <ietf-http-wg@w3.org>; Wed, 15 Jul 2020 11:22:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=X7p+Iw3CwW9dMPbdbY2dMI/T9JfCp9WS0nnm8hZuvp4=; b=ERE2ZzkHyckZj1Tie1QKBDRrloT1a1O6lhPG7c+1B1nQbecJXf8RlZ43ht1tkzRYGx BSaql1SaV1vllVaHYG/+uE42mqJacJVMjvSt/VdxX2EKdjxa2pD+WZT76vLiC/cOXpzf Urht49fFlcAcBWvhZXa+9WoVHQ6+NLahjA2zKlvitoTJ2kgu1l54SDAWqodgYxHOf+sY Dp2GtmhWTJobN9etaTk3Pi8/NUsTwK+fz9ZdiMm2jR9j/BFdXyxmACWjoFwO39rFn0Xa 4jGz9gJAPLwMFTmi8cCDjQ8DW0N/aH8dsX6rECQ0lYYSqSoFNfeGzOWMwi6+ZypAdigD 8dIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=X7p+Iw3CwW9dMPbdbY2dMI/T9JfCp9WS0nnm8hZuvp4=; b=EZm6zrs7qdLiEm1sMmcoUA1rgT7yr9kB6gFb4yMZW2b7wMRPTH5ggKHTKcaeCA+Qsk 6gMRZP8Wn7xTWxqvMChFX2TQIve//vTmxX+T2NAwR+afap3zskZLob/FB9cFIHmvrNyg R2SB3QrbJAU2D6Hm0Cx4F6urIoi4T/35kOF70Bx+/ncOYbSUgzis+f1s6ZDf1M8rcRxj KghlhYFGTHm6/qZt0r6kETBoe8WrQoIpNa2en3kFUazF9sqCDSwULtoKPg1ja5QjTm4H 5vlsjBVWSUD20NGZzje8rCDPfIRwEGu532+5fRseCel8fKDVChhoP2gmgVbKrz2py8GZ jsag==
X-Gm-Message-State: AOAM531DfxSB6A4upHACK70aiiyLpLuajm/J0weHhYf28pKP2oE89tOH f+vQun9quEWgsOA3zCcIPJQ=
X-Google-Smtp-Source: ABdhPJxkQUw03/+aO6g9TeK5rbRmZTqgGKpt8Cp9TwdSEgltTrt/iMQ5Srxq84Ck1e+hzJcS5WlJtg==
X-Received: by 2002:a63:215e:: with SMTP id s30mr806595pgm.87.1594837355677; Wed, 15 Jul 2020 11:22:35 -0700 (PDT)
Received: from [192.168.42.5] (135-180-96-53.fiber.dynamic.sonic.net. [135.180.96.53]) by smtp.gmail.com with ESMTPSA id f3sm2788330pju.54.2020.07.15.11.22.34 (version=TLS1 cipher=ECDHE-ECDSA-AES128-SHA bits=128/128); Wed, 15 Jul 2020 11:22:35 -0700 (PDT)
Content-Type: multipart/alternative; boundary="Apple-Mail=_84B1550F-5D2D-4043-B9D4-4D73F957A6E7"
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Michael Toomim <toomim@gmail.com>
In-Reply-To: <CADU7aovMvY+k1yhV1TPwpw8FWvZ0of6wMMsjiAV8vq=ZBt+MNQ@mail.gmail.com>
Date: Wed, 15 Jul 2020 11:22:34 -0700
Cc: HTTP Working Group <ietf-http-wg@w3.org>, Braid <braid-http@googlegroups.com>
Message-Id: <0DD01D80-6CDD-4BD7-A5FC-40FD21832FB7@gmail.com>
References: <CADU7aovMvY+k1yhV1TPwpw8FWvZ0of6wMMsjiAV8vq=ZBt+MNQ@mail.gmail.com>
To: =?utf-8?Q?K=C3=A9vin_Dunglas?= <kevin@dunglas.fr>
X-Mailer: Apple Mail (2.3124)
Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=toomim@gmail.com; helo=mail-pf1-x433.google.com
X-W3C-Hub-Spam-Status: No, score=-4.1
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1jvm3L-0001I9-5Y a07e30f2175eeb9675e12ecbba8c2b03
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Publication of the Mercure Protocol
Archived-At: <https://www.w3.org/mid/0DD01D80-6CDD-4BD7-A5FC-40FD21832FB7@gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/37881
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Hi Kévin I'm excited to discover this spec!

A group of us are working on a related draft called Braid, which also adds push-updates to HTTP:

Draft:	https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http <https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http>
Web:	https://braid.news <https://braid.news/>

So it seems we are all tackling the same problem, but aren't aware of each other! Let's solve that.
How about we work together to make our systems interoperable, and find consensus on a single spec?

To start, I've read the Mercure draft and am leaving a review. I'm also cc'ing the braid-http list, in case any Braidly people want to chime in. I'd love to see your review of Braid as well.

# My review of the Mercure draft

Push-updates is an important, and common problem. A number of IETF specs have devised their own solutions: (e.g. Calendar Synchronization <https://tools.ietf.org/html/rfc6578>, JMAP Email Synchronization <https://tools.ietf.org/html/rfc8620>). Our lives would be much improved if we spoke the same push-update (aka "synchronization") language.

I like the general SSE approach. Braid similarly streams updates over a long-running GET, although it doesn't use the other parts of SSE.

However, I think the Mercure spec would be simpler, cleaner, and more general with two major changes:
Combine the separate Discovery and Subscription steps into a single request/response.
Unbundle the Authorization method. Let people use their own methods.
These changes would simplify and generalize the spec, and be more compatible with Braid. I provide a more detailed analysis below, and then conclude with some small suggestions.

## 1. Combine Discovery and Subscription into a single request/response

To get a subscription, this spec requires the client to make two requests:
A GET to the server to discover how to subscribe
A separate GET (using SSE) to a "hub" to actually subscribe

It is actually possible (and I think simpler) to combine these into a single GET request. This is how Braid works. If a GET request includes a "Subscribe" header, then the response will return the current version, but then also stay open, and stream all new versions as updates like SSE does.

Combining these requests eliminates a round trip of network latency, which is great; but it also simplifies the protocol. In particular, there are a number of new concepts introduced in this spec (Topic, Hub, Publisher, Subscriber) that seem like re-inventions of existing HTTP concepts (Server, Resource, Client issuing put, Client issuing get). I presume that you didn't re-use the existing HTTP concepts because you needed these to run over a separate SSE connection, which doesn't have built-in HTTP semantics. However, if you instead stream the updates to any resource within the existing HTTP connection, you can re-use all of its existing HTTP semantics. Then you won't need to re-specify these concepts in the Subscription phase, and the spec will be simpler and easier to digest and adopt.

Is it possible to combine these into a single request? Is there any reason why these need to be separate in the protocol?

## 2. Unbundle Authorization

Another benefit of moving Subscriptions into the existing HTTP requests is that we don't need to invent a new authorization method — the subscriptions can use whatever the existing HTTP requests use.

However, if I'm missing something, and we do have a need to specify a particular style of authorization, can we at least move it into a separate spec?  There are many ways to authenticate, and it would be nice if we could find consensus on authorization separately from how to push updates.

Ok, that's it for the big items! I'd love to hear what you think about these design decisions, and see if we can make these protocols compatible.

## Details

- Spec says the hub is a server, and the Link header specifies the hub, but the examples provided are Links to full resources with a path (e.g. https://example.com/.well-known/mercure), not just to a server (e.g. https://example.com). So which is it: a hub/server, or a full resource with a path?

- Spec says:
    "The URL of the hub MUST be the "well-known" fixed path "/.well-known/mercure"
  But the point of .well-known <https://tools.ietf.org/html/rfc8615> is that you don't need to specify it anywhere, and you don't need the discovery step. So I think you can eliminate this header entirely.

- Last-event-id: If I'm not mistaken, this only handles linear time, with a single writer. Can this spec support multiple writers modifying a resource simultaneously? That would produce two simultaneous last-event-ids, neither of which has occurred more recently than the other.
  - The Braid specification calls this "Parents:".

SSE stream ID:
   - Why must the ID be an IRI?  Does the IRI reference something?
   - It'd be more specific to call this a "Version" instead. It's not just any ID! It's a Version ID.
   - Braid calls this "Version", and allows it to be any unique string, not just an IRI.

Reconnection
  - The Mercure spec provides clients with no guarantee that they will receive all updates upon reconnection.
  - Braid servers, on the other hand, can guarantee a duration of time for a client to reconnect and receive all updates. I would like to support this type of guarantee.

Active Subscriptions
 - Although I agree that this feature is useful, I'd rather this not be a part
   of the spec.  Do we have any interoperability use-cases where this needed?
   It seems like it would be fine for each web app to have its own method of
   tracking active subscriptions.  This method forces you to use JSON-LD.


> On Jul 8, 2020, at 7:44 AM, Kévin Dunglas <kevin@dunglas.fr> wrote:
> 
> Hi all,
> 
> Late 2018, I published an Internet-Draft specifying a protocol called Mercure:
> Abstract
> 
>    Mercure is a protocol enabling the pushing of data updates to web
>    browsers and other HTTP clients in a fast, reliable and battery-
>    efficient way.  It is especially useful for publishing real-time
>    updates of resources served through web APIs to web and mobile apps.
> https://datatracker.ietf.org/doc/draft-dunglas-mercure/ <https://datatracker.ietf.org/doc/draft-dunglas-mercure/>
> 
> I just published the 7th version of the I-D. The protocol can now be considered stable and feature complete. It is already widely implemented and used, including by popular web frameworks such as Symfony. You can see the full list of implementations in the "Implementation Status" section of the I-D.
> I would like to go one step further and propose it as a RFC.
> 
> I was wondering if the HTTPbis Working Group could host the work on this protocol (this looks allowed by the "Other HTTP-Related Work" section of the charter)?
> 
> Also, I tried to register the link relation (https://github.com/protocol-registries/link-relations/issues/21 <https://github.com/protocol-registries/link-relations/issues/21>) and the "well-known" URI (https://github.com/protocol-registries/well-known-uris/issues/4 <https://github.com/protocol-registries/well-known-uris/issues/4>) used by the protocol, but it's not possible yet because the I-D isn't on any stream.
> I submitted a new version of the XML file with the following header:
> 
> <rfc version="3" ipr="trust200902" docName="draft-dunglas-mercure-07" submissionType="IETF" category="std" xml:lang="en" xmlns:xi="http://www.w3.org/2001/XInclude <http://www.w3.org/2001/XInclude>" consensus="true">
> 
> But it looks like it hasn't been taken into account by the tracker (https://datatracker.ietf.org/doc/draft-dunglas-mercure/ <https://datatracker.ietf.org/doc/draft-dunglas-mercure/>).
> 
> I must admit that the process to propose a RFC is still a bit unclear to me. Is this list the right place to propose and discuss this protocol? Should I create a new submission for the draft or is it possible to "update" the stream on the existing one?
> 
> Best regards,
> -- 
> Kévin Dunglas
> 
> https://dunglas.fr <https://dunglas..fr/> / @dunglas <https://twitter.com/dunglas>
>