Re: [Ohai] DoubleCheck and CONNECT-UDP

Christopher Wood <> Thu, 04 August 2022 19:14 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C11E0C157B4F for <>; Thu, 4 Aug 2022 12:14:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.108
X-Spam-Status: No, score=-2.108 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key) header.b=GvQ58OOr; dkim=pass (2048-bit key) header.b=gBH3+AfF
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id fOf-bppgplkz for <>; Thu, 4 Aug 2022 12:13:59 -0700 (PDT)
Received: from ( []) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by (Postfix) with ESMTPS id 15832C14CF06 for <>; Thu, 4 Aug 2022 12:13:58 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal []) by mailout.west.internal (Postfix) with ESMTP id D34CC3200970; Thu, 4 Aug 2022 15:13:55 -0400 (EDT)
Received: from mailfrontend1 ([]) by compute4.internal (MEProxy); Thu, 04 Aug 2022 15:13:56 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1659640435; x= 1659726835; bh=6KPNr9JP0fV2ngJQm01m8sQxRKh+RrQwsSqxIEJ3g94=; b=G vQ58OOrzI2qpXFX6RRIV3+fZ0PrI3KThetemRTowytOYIhSDOinexBxjWLemQR2/ +w0lseUfXH5cD1zOn0dl/r8UaefGDQMUpsYgGQ9AqKstbKxX60miKvztM5HRMjCe 9lBIHh+zBYJlcjnHv9ByQiXStKZstmeYQGrtMUzB3Yet++3M33xzSVfu/QXity7i V/U5JN9n5CMt0A3wULd06rtwMoxpBDgjA6Jnmnf9rzNUdb7U4OW/hrVVFwexXHRV WrnswG8cwxsSFYbT0HrXHXOISGKNRB+GYrIBYHcWZb24W2lJD1UvayYj7pukd7Hb o5efVASOHC80esV54Xsxw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1659640435; x= 1659726835; bh=6KPNr9JP0fV2ngJQm01m8sQxRKh+RrQwsSqxIEJ3g94=; b=g BH3+AfFobf0187w1YjzdIwiYpbyEYwCJu1Kji6NiwALbyf9j0S7b1adro7MQbFvq hpPDcFRMEWDPU42ymwfE2s8CLKjJeMJ9VKq+SwNj/qOTR3DXeQiWRGKdm2/DLOKZ ZiVmaMJi1IQaKbjAHYPDWqtCTLKTGFgRW4J5oTmlZzJd04gwbl5bfds/gwl4SDnp xUX/1BT4Ku4ViIjw4zs0T+iTEcm4EjD9pwC5cZU0mX4zxEiNpx9X89r0hm5UpsFc NEE8qm0iRYdzqMldHuturKpgmdW5OzVG69gQmmQZnAhqaqSmR+aXk4PE1tTZgWzX lObK+rHnWFqJVdebCCHtA==
X-ME-Sender: <xms:cxrsYkJecMoKC6HlL-piCclKHGI1EQua_zKvvZO9F2LDujMtVcMgQQ> <xme:cxrsYkKKVY_m0SS7PjAzQIYTox_AMwMkcCi9pTn-oeMUjf1y7IePGE3BwZ0el7_tO dCRjwce1KJCS55DMYo>
X-ME-Received: <xmr:cxrsYkuC24l_sCWKf4sZy7cy8nLCWGSeMJG0CK8R-lVBQss5SmWCKYlWwtZyCbp3FoEwstzlZMWy0-VSjgjfxDb8qJqmWp0EJHE>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvddvledgudefhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpegtggfuhfgjffevgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhepvehh rhhishhtohhphhgvrhcuhghoohguuceotggrfieshhgvrghpihhnghgsihhtshdrnhgvth eqnecuggftrfgrthhtvghrnhepudeuvddtgfekiedvjefglefhteejudfgleelleevhedv geevvdejhfetgeevteeknecuffhomhgrihhnpehhthhtphhvvghrshhiohhnrdhinhdphh htthhpvhgvrhhsihhonhhsthhhvggvtghoshihshhtvghmfihilhhltghonhhvvghrghgv ohhnshhomhgvthhhihhnghhthhgrthifohhrkhhsrdhithdphhhtthhptghonhhnvggtth hiohhnshgrnhgulhgvthhthhgvvhgvrhhsihhonhgvvhholhhvvgdrihhmnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomheptggrfieshhgvrghpih hnghgsihhtshdrnhgvth
X-ME-Proxy: <xmx:cxrsYhZIjVVEyyN9RPZcVp1al1ikwkuolDHKAJu-dPnIKX0a7d6dwA> <xmx:cxrsYrY5dyBNMbdcnHEwWydROIU8BUlJZd13w_9sqIcHw89yKrlo4g> <xmx:cxrsYtCyS3lWj37wlaIcT9a3CM1ahISVbyFVdvoqRFO8k2wX0MqlZA> <xmx:cxrsYsB9XK26xXns3ShkaxY0vXb581T9si1_FDfM2EKm4jZbJalTaw>
Feedback-ID: i2f494406:Fastmail
Received: by (Postfix) with ESMTPA; Thu, 4 Aug 2022 15:13:54 -0400 (EDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.\))
From: Christopher Wood <>
In-Reply-To: <>
Date: Thu, 4 Aug 2022 15:13:53 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <>
To: Eric Orth <>
X-Mailer: Apple Mail (2.3696.
Archived-At: <>
Subject: Re: [Ohai] DoubleCheck and CONNECT-UDP
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Oblivious HTTP Application Intermediation <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 04 Aug 2022 19:14:03 -0000

> On Aug 4, 2022, at 3:03 PM, Eric Orth <> wrote:
> On Wed, Aug 3, 2022 at 8:56 PM Christopher Wood <> wrote:
> On Thu, Jul 28, 2022, at 7:12 AM, Tommy Pauly wrote:
> >> On Jul 27, 2022, at 7:00 PM, Ben Schwartz <> wrote:
> >> 
> >> 
> >> 
> >> On Wed, Jul 27, 2022 at 5:43 PM Tommy Pauly <> wrote:
> >>> 
> >>> 
> >>>> On Jul 27, 2022, at 3:07 PM, Ben Schwartz <> wrote:
> >>>> 
> >>>> 
> >>>> 
> >>>> On Wed, Jul 27, 2022 at 2:25 PM Tommy Pauly <> wrote:
> >>>>> Trying to lock people into using CONNECT-UDP to an HTTP/3 server in order for key consistency to work seems to be a “not good” state to end in. Maybe better than locking in CONNECT, but locking in any particular protocol or version seems quite limiting.
> >>>> 
> >>>> As a formal matter, I think this is essentially unavoidable in any situation where proxies are in use.  For example, in OHAI, the Relay and Gateway must both support a common HTTP version.  In practice, this likely means that (general-purpose, open) Relays and Gateways will have to support HTTP/1.1 as a de facto baseline.
> >>> 
> >>> There needs to be at least one HTTP version that the relay can use to speak to the gateway, yes. However, I certainly don’t see any reason why these would need to use HTTP/1.1.
> >> 
> >> This is my prediction of how the ecosystem would evolve absent any recommendation in the spec.  Neither Gateway nor Relay operators would feel confident that they could disable HTTP/1.1 without breaking some counterparts, since nothing prohibits a compliant Gateway or Relay from being HTTP/1.1-only.  (This outcome is probably fine, since supporting HTTP/1.1 is mostly harmless.)
> >
> > For a relay and gateway, HTTP/1.1 isn’t quite so harmless from a 
> > resource consumption aspect. If a relay is handling millions of 
> > requests, putting these onto HTTP/2 or HTTP/3 connections that can more 
> > efficiently multiplex is preferable to opening up tons of parallel 
> > sockets between the two boxes, or adding a lot of latency via 
> > head-of-line blocking.
> >
> > This is the same reason that using HTTP/1.1 for DoH is a pretty bad 
> > idea compared to H2 or H3.
> >
> > Gateways and Relays are all new things. I don’t see a reason either 
> > would ever need to support HTTP/1.1. I think the ones we are using are 
> > only ever using at least HTTP/2, and there would be no intention of 
> > going backwards.
> >
> >>  
> >>> Relays can be picky about what gateways they allow
> >> 
> >> My focus here is on Relays that allow (authorized) users to reach any Gateway on the internet (and similarly, Gateways that are reachable from any Relay).  I think such "universal relays" are very natural, as a companion service to privacy proxies that similarly allow access to arbitrary destinations.
> >
> > I’m not entirely sure — I think there will be relays that are not 
> > entirely special-purpose, but they likely will need some established 
> > relationship to verify the stance of a gateway. I.e., to be added to 
> > the ACLs, establish any mutual authentication that is needed, etc. That 
> > may be something that can be automated to a degree, or just audited.
> >
> >> 
> >> Limited-purpose Gateways and Relays, which only support a fixed set of approved counterparts, are in a very different position.  If the client has to know which Gateways are usable with the Relay, the service that provides this information to the client might also be trusted to provide authentic, consistent KeyConfigs.
> >>  
> >>> — and I’d certainly advocate that any sensible implementation SHOULD NOT allow talking to any gateway that doesn’t at least support HTTP/2.
> >> 
> >> We can certainly put that in draft-ietf-ohai-ohttp, but it isn't there yet!
> >
> > I don’t think this needs to be in that draft — I think we let 
> > implementations do what they need to, since there are ways to negotiate 
> > version. But the base OHTTP assumes you know what you’re doing and how 
> > you set up your relays and gateways. If we have new documents that talk 
> > about mixing and matching deployments, that deployment guide can give 
> > more advice.
> >
> >>>>> I don’t think that it follows that if we don’t have CONNECT-UDP, the clients would directly fetch the key config from your proposed SDH resource.
> >>>>> 
> >>>>> To perform double checking, I think it just matters that we have one check that uses a TLS-authenticated connection to the authority for the key config, and one other check that uses a cached version on a TLS-authenticated connection to another entity that is not the authority for the key config; AND that these checks are done in such a way that the TLS-authenticated connection to the config authority does not get to track or identify the user.
> >>>> 
> >>>> I think that's an accurate description of the requirements, but I don't think documenting the requirements is sufficient.  If we don't standardize the proxy/relay protocols and their configuration, that essentially leaves the users locked into a single relay service implementation.  I don't believe that users should have to use the relay service operated by, say, their handset manufacturer or browser vendor.
> >>> 
> >>> I agree that users don’t need to be limited to the relay service provided by a specific manufacturer. But they do need to use protocols that their client implementation is willing to support.
> >> 
> >> If we don't standardize what those protocols are, an independent Relay operator can't be sure that they are compatible with DoubleCheck clients.
> >> 
> >> However, given the feedback so far, I think the most likely outcome is that the draft will mention CONNECT and CONNECT-UDP, without mandating either, and almost everyone will end up implementing both.  That seems like a suboptimal outcome, but it's tolerable.
> >
> > I think the client should be able to use whatever proxying mechanism 
> > its relays/proxies support, and what the config lookup resource 
> > supports. The double-check mechanism shouldn’t need to rely on what the 
> > proxying landscape looks like, just that there is one.
> >
> >> 
> >>> If the requirement is that a client needs to access the resource that is authoritative for the OHTTP key configuration, it should be free to access that using any secure proxy protocol over which it can run a TLS connection to the key configuration resource, and it can use normal ALPN selection to know which HTTP version to use to the key configuration resource.
> >> 
> >> ALPN doesn't cover TCP vs. UDP selection.
> >
> > Doesn’t it? If I get ALPNs of both h2 and h3 in my SVCB records, don’t 
> > I know which to use?
> >
> >> 
> >> In practice, if we don't offer any recommendations regarding HTTP versions, the ecosystem will converge on something that works.  It just might not be the most efficient outcome.
> >
> > Yes, the ecosystem will converge and continue to evolve, and I expect 
> > that it will do the right thing for now, since no one should have an 
> > incentive to support old versions like HTTP/1.1 for these new services.
> >
> >>>>> What method you use to proxy a connection is entirely up to the client’s favorite proxying method, I think. Let’s not try to drag specific transports into this.
> >>>> 
> >>>> We can let the market decide the HTTP version, but I do think we should specify how the transport proxy is configured.  Otherwise it becomes very difficult for there to be multiple interoperable implementations of the relay service.
> >>> 
> >>> For the reasons above, I strongly disagree. We have ALPN and other ways to know what version is supported. There’s a reason we use HTTP and TLS here — we can run secure HTTP connections and let the version evolve.
> >> 
> >> I'm not talking about the HTTP version; I'm talking about an Access Service Description (or an equivalent standardized configuration) for the Relay.  A standardized Relay configuration system is necessary if independent operators are going to reasonably be able to provide clients with an affiliated Relay/cache, transport proxy(s), and DoH server.  This standard can potentially support multiple transport proxy protocols (e.g. CONNECT and CONNECT-UDP), at the risk of partitioning the anonymity set due to varying client transport support.
> >
> > This is a conversation that belongs to the larger scope of how we know 
> > configuration. I think there can be reasonable baselines — that we say 
> > a baseline CONNECT-like relay always supports both CONNECT and 
> > CONNECT-UDP to be able to access both TCP and QUIC upstream hosts that 
> > are allowed by the relay.
> >
> > But, overall, I don’t think that the mechanism of double-check needs to 
> > care — it just says “you have some relay or proxy you’ve learned about, 
> > use it in this way”. That would allow us, for example, to update and 
> > have -bis relay/proxy documents without needing to rev or change the 
> > document that explains that you should double check key configs.
> I agree with Tommy here. I view DoubleCheck less as a “protocol” and more as a pattern. One could implement it in many ways using a myriad of transports. I tend to think we ought to simply state the requirements here and leave it at that.
> I strongly believe we should have both.
> oHTTP is mostly used for "special cases", so there's certainly lots of room for different implementations to have special needs and capabilities and only need general pattern and requirement advice.  draft-wood-key-consistency-02 is good for filling this role (but should maybe be updated a bit to also cover more generalized advice around this doublecheck as a pattern), and I want to see that draft get adopted somewhere (OHAI?).
> But the requirements to do something correctly and safely following the patterns is also very tricky, as evidenced by all the thought that had to go into this draft and various discussions (both public and side) that I have seen regarding this draft since last week.  While we definitely shouldn't force it on all oHTTP implementations, I strongly believe we need at least one standardized full protocol for safely exchanging the oHTTP key, and I support this draft to fill that role.

That’s a reasonable position, but I don’t know what you mean by “standardized protocol” here. Let’s say the gateway exposes a resource containing its configuration for DoubleCheck. I don’t see how we need a new protocol for accessing this resource. Depending on the application requirements, one could fetch it directly from the gateway, use CONNECT through the relay, or just fetch it over Tor. 

What would you expect this “protocol” to say?