Re: QUIC Invariants and QUIC-LB

Martin Duke <> Fri, 22 May 2020 00:00 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id DF7B83A0C0C for <>; Thu, 21 May 2020 17:00:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id JaNfXdITmhxv for <>; Thu, 21 May 2020 17:00:19 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 567853A0BFF for <>; Thu, 21 May 2020 17:00:19 -0700 (PDT)
Received: by with SMTP id 18so9060701iln.9 for <>; Thu, 21 May 2020 17:00:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rqW49QlFgVehG5hYYqBbwT4xU7mQ/uQffn2GTSZGYYk=; b=e4stGwsT2HRVOC13PM6D0C2NRArIzdlcdR4dOmtN5hzr+xbpOysqPouYeCsXGH4xFm WeEMzasU1xXtxKpmqPib8EmbNsMPaXP4+0SNLsijjbyfzZY08OmhLy9NnUmKMyFMfkbv smL12fy3cdG4NtqR6FXR/+wWYh8xQ/fQr2Yfmotlgo+APPhjS0na4JmS7AEv7ykn3peQ 2rwYgyASzLciVZsYUEdXGf9pVTBWgWqO/nlZm8bZsV9NTj6O9HXN8JP/sR2kCAt/ik9g mz99s5hSkdl9LrMPJW6+uEnpU36ydapzjwWlrKZkLB3lDo3SYCRzAupdJ2J28Rxx3pRQ itzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rqW49QlFgVehG5hYYqBbwT4xU7mQ/uQffn2GTSZGYYk=; b=Ki2nvwUbCMoz2HUamE795h68kM2NsmVpcSUrCgoW1sS6SW+67RbbsEHc5fd9cGqy5u /8Vbeszrt8FJIybPjb0Q/XF+vcIgm5Ziqy/extvq9W65CFardv8qb3FaHAZFEMqDTyFw qXcxq9H3t3AboV/vz7trdeEqCKRs2EZtBo0NAW50RhUpTPwfbEGAtdIoR9CtOskxhiQY sijfvcwmMPZ2Wd+YHELb/pPVS0T75ONttUTsgy7SrSx99boYHnfRUOAMpTRubcoSimpz AY/265sRcr25+Y6P/jgFBhSlVLDZ1792Ur2g2Th7nuSceOi17fjqr57O9ftuUYOB8yp7 H3Nw==
X-Gm-Message-State: AOAM5300xUP9lAHPfsfSBnPyqepbfTjGTo70IGY8Cy5rfgYb8Lg/Xlbq kqQIXUsgpXbv+TQXVrwbrwzDqpY0SeUZUgsKGgoaz1Ja
X-Google-Smtp-Source: ABdhPJxtDHBuTSEGWyDq9cJbaPnxP11oLWewWaUvyhK9h9gxFaKH7J/JfnnnsJrlJmlqGOWWfXjaGbNMOQgQ+3a+uYw=
X-Received: by 2002:a05:6e02:965:: with SMTP id q5mr10733831ilt.272.1590105618460; Thu, 21 May 2020 17:00:18 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <>
In-Reply-To: <>
From: Martin Duke <>
Date: Thu, 21 May 2020 17:00:06 -0700
Message-ID: <>
Subject: Re: QUIC Invariants and QUIC-LB
To: Martin Thomson <>
Content-Type: multipart/alternative; boundary="0000000000008b33bc05a6315014"
Archived-At: <>
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 22 May 2020 00:00:22 -0000

I don't think we're going to get a 100% version-invariant QUIC-LB, though
we're pretty close.

It would be nice to give LBs somewhat more specific guidance for what
fields they can use for packets with non-compliant DCIDs. For example, if
we could at least safely assume the DCID won't change until the server
provides one, that would be way better than forcing them to make
assumptions about IP and UDP fields remaining stable.

On Thu, May 21, 2020 at 4:41 PM Martin Thomson <> wrote:

> That's a good point.
> The set of assumptions here are fairly small, which is good.  Essentially,
> these turn into requirements for a new version of QUIC that is intended to
> replace v1, if that version also needs to work with a load balancer (which
> seems quite likely).  As the set is so small, and as I don't think that any
> replacement would be likely to make changes at so fundamental a level, it
> seems like those requirements wouldn't end up being too onerous.
> Basically, this boils down to whether you think the ability to load
> balance is invariant, and while it might be inconceivable for something
> like HTTP, it is not generally so.
> On Fri, May 22, 2020, at 00:31, Ian Swett wrote:
> > Thanks for pointing these out. To me, these constraints seem like
> > natural consequences of using QUIC with a load balancer. But there are
> > use cases for QUIC where a load balancer isn't required(ie: P2P and
> > many datacenter use cases), so I'd prefer not to add more constraints
> > to the invariants.
> >
> > On Wed, May 20, 2020 at 7:28 PM Martin Duke <>
> wrote:
> > > TL;DR We are trying to make QUIC-LB as independent of QUIC version as
> possible, but there are a few gaps in the invariants that make that harder.
> If any of these are oversights, it would be great to add it to the
> invariants.
> > >
> > > *******
> > > A few draft versions ago we did a pass on QUIC-LB to make it truly
> QUIC-version invariant, except for the Retry Services portion.
> > >
> > > After a little more thought, I added this section to the draft:
> > >
> > > which describes a bunch of QUIC behaviors that are not explicitly
> described in the invariants draft, but that if changed might cause QUIC-LB
> not to be fully robust to new versions. An obvious one is a version that
> described a maximum CID length too small for some of the encodings.
> > >
> > > A more subtle one is the handling of "non-compliant CIDs" (Sec 4).
> These are CIDs that are (1) too short for the configured algorithm, (2)
> fail authentication, or (3) don't map to a valid server ID. In v1, these
> will only occur in the client's first flight of Initial and 0RTT packets.
> The rules we have for version-independent LBs to process packets with
> non-compliant CIDs are as follows:
> > > * if a short header, drop it.
> > > * if a long header, use any hash algorithm that will consistently get
> you the same result over short time scales (i.e. a couple of RTTs, by which
> time a server-generated CID will be in use)
> > > Compliant CIDs are always processed using the decoding in the spec.
> > >
> > > There are some assumptions baked into this advice.
> > > (1) Short headers never contain client-generated CIDs. [QUIC-LB
> servers will always provide new CIDs when given the opportunity, but a new
> version could have short 0RTT headers or something]
> > > (2) Use of client-generated DCIDs is limited to a few RTTs at most
> (any more and you run the risk of the number of servers changing in this
> interval, which will generally cause the hash to get a different result)
> > > (3) There's a bunch of stuff that remains constant in the packet
> headers over this short interval. I probably need to tighten up the
> language on what is a suitable input for the LB's hash, but in v1 you're
> currently safe using some combination of Client IP, Client Port, SCID, and
> DCID. If there of these that we're willing to lock down in the invariants,
> I could rewrite the guidance accordingly.
> > >
> > > For example, if the invariants could include a statement like "A
> client will use no more than one DCID that it didn't receive from the
> server", that would simplify the problem space.
> > >
> > > ****
> > >
> > > I'm fine with having this text in the QUIC-LB draft, if need be, but
> if there are things that should be in the invariants but we didn't think
> about them, that would be even better.
> > >
> > > Martin