[ippm] QUIC concerns relating to draft-ietf-ippm-explicit-flow-measurements

Lucas Pardue <lucaspardue.24.7@gmail.com> Wed, 10 May 2023 21:42 UTC

Return-Path: <lucaspardue.24.7@gmail.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id D32A8C1D9FDB; Wed, 10 May 2023 14:42:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.845
X-Spam-Status: No, score=-6.845 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id 4P1AjBtGCSrG; Wed, 10 May 2023 14:41:58 -0700 (PDT)
Received: from mail-ot1-x332.google.com (mail-ot1-x332.google.com [IPv6:2607:f8b0:4864:20::332]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 055B9C1D9FE5; Wed, 10 May 2023 14:41:58 -0700 (PDT)
Received: by mail-ot1-x332.google.com with SMTP id 46e09a7af769-6ab1e0420e8so990742a34.1; Wed, 10 May 2023 14:41:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683754917; x=1686346917; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=P/Gsjm21LrMVDPQidp9aHZhe2LhjhF/1F3tEueQecNs=; b=spASDenR3GiHeJxzS30AC5VTSptLR9LHe+roEC0K2/aUBFy9/LJODt670GOQQXAZU1 s0VPJDZYM4319PIvfWiQBbfzktLhXyiWb0VQs2t5ion73n5RqcrjdvjqXzzru1p4g2j2 thCcCcjxKK9hTYwgssl8gNd5y6E6WRQz4KmIH2zTDaBPjO2fCb8/vZEU4T9BSpiuZXTY nfwZUC4GNX7OoshiENrl7+/K6Lec9ejq8w+5IXLlU6PFGztPgm7isFr5A0bMopcQ3MWg 8TPfuIm9/T5nXEfCeU3514td+bHlGB2c4P184iFfJKJiR1TSLfJ4Q7yrqFYW/JAUx/A2 4ZbA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683754917; x=1686346917; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=P/Gsjm21LrMVDPQidp9aHZhe2LhjhF/1F3tEueQecNs=; b=YJpUuHxaGwsJPGyomRCPu9BrVewmXwl+Oii7O7vnRp1dqdj38NwGJLhACd30hE8NfS UDZWNiAtQ5fPV8QUnljFCQhq7qMAAAFRFIKU4ie7pm7qe6lcGFjQoVrdM/UpQP+a/Dsn fWWQ5bmlfM45d5bsrZMj054KvPkQTwMpWSugSr1hkw31IsaMjJT1Wjdx3N+Qnk1g6IfV qc/FS5IveLMVMPTN3zp3gLF5Fv9FK/YvN7lFjLJ3aWI1LCMoypUjGi6YERe7JBHwRylA g20A5mlN+mfjwfJ5wr/HOgHQ8U5c3sLuwKnEPzrzAYvtOEitofaEC8E0JcmGGrVIW79e 9ktA==
X-Gm-Message-State: AC+VfDwxHpfRzDfP9A8y23KaJerQl25zp/s7FBZwvSBN1DOrn9DHOwpR fwrHXYYF+u744jQPBfqWaE0WaSOle3VvmUicAQZV51wsofY=
X-Google-Smtp-Source: ACHHUZ4UdPs76ciM/qU/sn3ACQrC+NggxzvDSEDOntASLPOJcvPN5JUUqBYZKC3SypG8ZEytks7xtx/53FvRr/5lgeY=
X-Received: by 2002:a05:6870:e1c2:b0:196:45b7:9385 with SMTP id g2-20020a056870e1c200b0019645b79385mr2384260oab.27.1683754916641; Wed, 10 May 2023 14:41:56 -0700 (PDT)
MIME-Version: 1.0
From: Lucas Pardue <lucaspardue.24.7@gmail.com>
Date: Wed, 10 May 2023 22:41:45 +0100
Message-ID: <CALGR9oZxFEXWD5hZZOJB7q+-f766FsjBGBTNjpuc1jyZyucz3Q@mail.gmail.com>
To: ippm@ietf.org, QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000b209c705fb5dbdd8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/RgtxAHmJfANjlfPkn1Jb4Hbbcs8>
Subject: [ippm] QUIC concerns relating to draft-ietf-ippm-explicit-flow-measurements
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 10 May 2023 21:42:01 -0000


For the purpose of this email, I'm speaking as an individual. I'm cross
posting to the QUIC mailing list for visibility of others that might also
be interested. The TL;DR is the draft -03 contains text that indicates how
visible bits of QUIC packets could be used and that raises several concerns
that I think need to be addressed before this document can progress further.

draft-ietf-ippm-explicit-flow-measurements is fairly far progressed through
its IETF journey. To drastically summarise it, it's about the various forms
of marking bits in packets that can be used for performance measurements.
And that requires collaboration between client and server in order to
expose some information that on-path observers might use.

This might seem familiar to the QUIC WG. We have a spin bit formally
defined in version 1. And we last heard, explicitly in QUIC, about the
concepts presented in draft-ietf-ippm-explicit-flow-measurements in a
thread in 2021. The work was adopted to IPPM and progressed there and that
is all good. However, I've not been able to track this draft as closely as
I would have liked and I suspect some other QUIC stakeholders may have
missed it too.

Other folks have raised some concerns during last call already and I'm
generally in agreement with them. However, for clarity of discussion
consider this a new thread that has some overlap and some non-overlap.

The major point that I think remains unaddressed by this document is how
bits in QUIC packets would _actually_ get used in a coordinated and
interoperable manner. The QUIC invariants draft, Section 5.2 [1] states
that short header packets consists of 7 Version-Specific Bits. Per 9000
Section 17.3.1 [1], QUIC version 1 defines the 1-RTT short header packet
with the 7 bits allocated as so

  Fixed Bit (1) = 1,
  Spin Bit (1),
  Reserved Bits (2),
  Key Phase (1),
  Packet Number Length (2),

Where the Reserved Bits have the requirements:

_The value included prior to protection MUST be set to 0. An endpoint MUST
treat receipt of a packet that has a non-zero value for these bits, after
removing both packet and header protection, as a connection error of type

The Intro draft-ietf-ippm-explicit-flow-measurements states that:

_this document proposes adding a small number of dedicated measurement bits
to the clear portion of the transport protocol headers. These bits can be
added to an unencrypted portion of a transport-layer header, e.g. UDP
surplus space (see [UDP-OPTIONS
]) or reserved bits in a QUIC v1 header, as already done with the latency
Spin bit (see [QUIC-TRANSPORT

This is problematic because you can't just change those Reserved Bits at
will and hope that independent implementations will interoperate. Similarly
you can't just substitute the Spin Bit as proposed in Section 6. This
really makes me question how this has been implemented and tested to date
because, to make it function properly and operate safely on the Internet,
you'd either need to have defined new versions of QUIC or defined an
extension. For an example of the latter, see RFC 9287 [3], which describes
a mechanism for repurposing the Fixed Bit. Yes, I understand some people
have trialled these out in a limited fashion but I'm talking about how
would an operator that is watching millions of QUIC flows from heterogenous
clients and servers know which ones are using measurements bits and which

At the point that client and server need to negotiate a version or an
extension, there is now a coordination challenge with these on-path
observers. The QUIC manageability spec has some great text detailing the
considerations that apply here, this draft is sorely lacking a reference to
Section 3 of RFC 9312 [4], and also lacking any commentary on dealing with
those considerations here. (but maybe it shouldn't, see end of email).

In addition to these concerns, I also find the protocol ossification
considerations in Section 5 to be awkward. It's not clear who the onus is
on to do anything because it talks about protocol designers and then says
implementations could decide to do something. It mentions "Latency Spin bit
greasing" in RFC 9000 but that term is not used by that name. I'm going to
presume you're alluding to Section 17.4 [5]. If so there are two important
separate aspects, disabling the spin bit on every 1 in 16 paths or
connection IDs, and randomizing values in the spin bit. The reason this is
awkward is because there are many different permutations of bits described
in the specification and operators need to somehow guess i) what
permutation is being used ii) if the endpoints have it enabled and iii) how
randomization affects analysis.

Finally, the draft consistently cites other QUIC documents (great!) but
lacks deep linking into the specific sections where you really want the
reader to go and focus. That places a lot of expectation on the reader to
read the right thing and do it. It would be helpful to add more-specific
links whereever possible.

All in all, these combination of concerns lead me to believe the document
in its current state is potentially unsafe, because it endangers people
picking it up literally and starting to write or read bits without due
coordination or feature detection. The confusion arises because the intro
implies this is a proposal to change bits but then backs off that
commitment and uses "coulds" etc. I think we need to be crystal clear.
Speak about some bits in the abstract and mention the few protocol-specific
concerns or related examples that each protocol already has defined.
However, remove the implications that we have well thought out how to get
these deployed for QUIC and punt that to future work by explicitly stating
so. I think this takes the form of removing section 6, rewriting section 5
to speak about the higher-level problems and possible approaches, and
removing any straggling sentences about reusing existing or reserved bits.


[1] - https://datatracker.ietf.org/doc/html/rfc8999#section-5.2
[2] - https://datatracker.ietf.org/doc/html/rfc9000#section-17.3.1
[3] - https://datatracker.ietf.org/doc/html/rfc9287.html
[4] - https://datatracker.ietf.org/doc/html/rfc9312#section-3
[5] - https://datatracker.ietf.org/doc/html/rfc9000#section-17.4