Re: [tsvwg] TSVWG: WG adoption of draft-white-tsvwg-nqb!

"Black, David" <> Sun, 15 September 2019 21:40 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 6CCA012026E for <>; Sun, 15 Sep 2019 14:40:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.19
X-Spam-Status: No, score=-0.19 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.01, RCVD_IN_MSPIKE_L5=2.5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key) header.b=xSVrPSEa; dkim=fail (1024-bit key) reason="fail (message has been altered)" header.b=CD5dH7qe
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 3-psz6qK15FQ for <>; Sun, 15 Sep 2019 14:40:27 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 4066712006D for <>; Sun, 15 Sep 2019 14:40:27 -0700 (PDT)
Received: from pps.filterd ( []) by ( with SMTP id x8F8TVsg012368; Sun, 15 Sep 2019 17:40:20 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=smtpout1; bh=Kz08kdCcTeODj20o9Pjgcayye9+pfWKajohqpnhOsFs=; b=xSVrPSEaCWfXYK0hxGAhZG9zZaVi4fZ2srTUoNGdi0MsAo7z3cOFVpoMImXl66L1tIgE Mx9m9FI9xrQwI9XkUcknqaX7igzdNlLKlGVQKxNOEmpQhBTyvQ8nyTIgPa9CsWuMxe1d edaQZ4MSbG70OrbJZe+5oduxrG4ZNYRLsqiOmGuE0B98VtgZdceoqTO0grZwzEPuDvhz rLyLrmwI+hE0lT5SmNnuPewsO6HdYDAiYDWWB7Jb9UK4kzqppESMC6d+UQhIUFsZmEzb o1H4l5eRQwVFZ6r28a4lFRKFO6WUXLmZ8thYu4S5BHCetOj/0Ui0rREDzO2hfiPoN+Hz Vw==
Received: from ( []) by with ESMTP id 2uytdqd541-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 15 Sep 2019 17:40:20 -0400
Received: from pps.filterd ( []) by ( with SMTP id x8FLcKaN133125; Sun, 15 Sep 2019 17:40:19 -0400
Received: from ( [] (may be forged)) by with ESMTP id 2v0tucqbw3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sun, 15 Sep 2019 17:40:19 -0400
Received: from ( []) by (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id x8FLe4t8008393 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 15 Sep 2019 17:40:18 -0400
X-DKIM: OpenDKIM Filter v2.4.3 x8FLe4t8008393
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed;; s=jan2013; t=1568583618; bh=tGFpDjPlW+18n2Z+ONM5urs8B40=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=CD5dH7qe5I8lD6V3JK47QYQTNkN3edHGaVbR4zroeTVJJfy6QZOoEEOsNEVBqW7Y/ N5adyupVjP1mdNSlQFnCW/d6vKBX2gcjRVBcUBpR9OnUMY+ltwtuz+DJqfUbFL+WV+ Id9VgCae35dMqL9K7lFZId+F1JXGRagwOSQwRong=
Received: from ( []) by (RSA Interceptor); Sun, 15 Sep 2019 17:38:37 -0400
Received: from ( []) by (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id x8FLceYb010637 (version=TLSv1.2 cipher=AES128-SHA256 bits=128 verify=FAIL); Sun, 15 Sep 2019 17:38:41 -0400
Received: from ([fe80::849f:5da2:11b:4385]) by ([]) with mapi id 14.03.0439.000; Sun, 15 Sep 2019 17:38:40 -0400
From: "Black, David" <>
To: Sebastian Moeller <>, Greg White <>
CC: "" <>
Thread-Topic: [tsvwg] TSVWG: WG adoption of draft-white-tsvwg-nqb!
Date: Sun, 15 Sep 2019 21:38:39 +0000
Message-ID: <>
References: <> <> <LEJPR01MB1178B6C102455F1F9886D49A9CBB0@LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE> <> <LEJPR01MB11789EE6D8B7C732393BD1439CB70@LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE> <> <> <> <> <> <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
msip_labels: MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Enabled=True; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_SiteId=945c199a-83a2-4e80-9f8c-5a91be5752dd;; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_SetDate=2019-09-15T21:38:39.5784326Z; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Name=External Public; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Application=Microsoft Azure Information Protection; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Extended_MSFT_Method=Manual; aiplabel=External Public
x-originating-ip: []
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.70,1.0.8 definitions=2019-09-15_11:2019-09-11,2019-09-15 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 suspectscore=0 clxscore=1015 bulkscore=0 impostorscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 adultscore=0 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1908290000 definitions=main-1909150236
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 mlxscore=0 suspectscore=0 impostorscore=0 malwarescore=0 bulkscore=0 lowpriorityscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1906280000 definitions=main-1909120181
Archived-At: <>
Subject: Re: [tsvwg] TSVWG: WG adoption of draft-white-tsvwg-nqb!
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 15 Sep 2019 21:40:30 -0000

Resurfacing to top-post and answer Greg's original question:

> On your item 3, could you clarify what you mean?   RFC 8325 does not discuss a “non queue building” service class, and so I don’t see a technical inconsistency between the NQB draft and RFC8325.
>>  3)	RFC 8325 reflects the IETF consensus on how to map between Diffserv and WiFi QoS, hence the 8.3 section of the NQB draft needs to be modified to be consistent with RFC 8325.

The posted version of the NQB draft cites RFC 8325 and then proceeds to ignore it in that Section 8.3 of the NQB draft discusses only what to do with implementations that don't support RFC 8325 (not a good approach).  At a minimum, that needs to be complemented by a discussion of what to do with implementations that do support RFC 8325, and perhaps some discussion of how to cope with the inevitable mix of support/non-support that will occur in practice.  The email discussion that has transpired to date appears to be "digging in a useful place" on how to handle NQB traffic on 802.11 networks, in particular when RFC 8325 is implemented.

Thanks, --David

> -----Original Message-----
> From: tsvwg <> On Behalf Of Sebastian Moeller
> Sent: Friday, September 13, 2019 4:46 AM
> To: Greg White
> Cc:
> Subject: Re: [tsvwg] TSVWG: WG adoption of draft-white-tsvwg-nqb!
> Hi Greg,
> > On Sep 13, 2019, at 01:34, Greg White <> wrote:
> >
> > Sebastian,
> >
> > Since it seems we’re going in circles (or maybe it's a very slowly converging
> spiral), why don't I make the edits that I think you want to see, and then we
> can go from there.
> 	[SM] I am all for that approach, and I want to repeat that I realize that
> I am not the arbiter in this matter and convincing me (while nice from my
> perspective) is not important, convincing the chairs that my objections are
> baseless or handled well seems more important.
> 	Since I seem to be the main person in this discussion with the "how
> does it look from the receiving end/end-user perspective" hat on, I will try to
> stay maximally critical (but let me repeat that I like the idea in general,
> modulo the side-effects I predict from a few of the detail-choices in the
> draft) that might be annoying on an inter-personal social level, but hopefully
> leads to a better technical soution.
> >
> > Based on your arguments, I've agreed to add some detail in a section on
> "Applicability" that indicates more concretely that the intent of the NQB
> marking is not for capacity-seeking traffic.  That text will supplement the
> existing language in section 3 of the draft that currently reads:
> >
> >  There are many applications that send traffic at relatively low data
> >   rates and/or in a fairly smooth and consistent manner such that they
> >   are highly unlikely to exceed the available capacity of the network
> >   path between source and sink.  These applications do not make use of
> >   network buffers, but nonetheless can be subjected to packet delay and
> >   delay variation as a result of sharing a network buffer with those
> >   that do make use of them.  Many of these applications are negatively
> >   affected by excessive packet delay and delay variation.  Such
> >   applications are ideal candidates to be queued separately from the
> >   capacity-seeking applications that are the cause of queue buildup,
> >   latency and loss.
> >
> >   These Non-queue-building (NQB) flows are typically UDP flows that
> >   send traffic at a lower data rate and don't seek the capacity of the
> >   link (examples: online games, voice chat, DNS lookups).  Here the
> >   data rate is essentially limited by the Application itself.  In
> >   contrast, Queue-building (QB) flows include traffic which uses the
> >   Traditional TCP or QUIC, with BBR or other TCP congestion
> >   controllers.
> >
> > If you can point out specifically what in that text led you to believe it was
> intended for capacity-seeking traffic, I would appreciate it.
> 	[SM] Given how much language lawyering is applicable on when
> parsing RFCs I would like to see more precision for the following terms:
> "relatively low data rates" as an application developer how do I decide what
> is relatively low? Or put differently relative to what exactly. Also If I have a
> paced-UDP based video delivery application that does application level
> switches between different bitrates based on experienced
> buffering/capacity/delay (like a paced DASH equivalent over UDP, or given
> that the sending rate for each segment is essentially fixed even standard
> paced-DASH) does this qualify for NQB or not? I believe the draft should
> make it crystal clear what can and what can not be considered to be NQB
> material.
> "such that they are highly unlikely to exceed the available capacity of the
> network path between source and sink" while I understand the thought
> behind this, it is simply impossible for an application to know that, given that
> "available capacity" is not fixed (also a sending application will have very little
> information about the available path capacity when it starts). Maybe I am
> being to pedantic here, but starting with a wrong premise irks me.
> "(examples: online games, voice chat, DNS lookups)" this still has the DNS
> lookups which collide with rfc8325, I understand why giving DNS lookups a
> low latency treatment (as DNS latency this directly translates into browsing
> latency, albeit only for uncached entries), but keeping DNS here has
> consequences on the wifi section IMHO.
> "Queue-building (QB) flows include traffic which uses the Traditional TCP"
> why the qualifier? As far as I understand TCPPraque would also be classified
> to the QB queue by virtue of being capacity seeking. In a sense Prague is
> included in the "other TCP congestion controllers" set, but I would prefer a
> less ambiguous wording like
> "In contrast, Queue-building (QB) flows include all flows which seek to utilize
> the full capacity of the link like all TCP flows, as well as many SCTP  and UDP
> flows, like QUIC"
> If you want you can add "all TCPs independent of the used congestion
> controller".
> If you disagree and believe TCPPrague should be allowed in the NQB queue,
> then at say so explicitly (but as far as I understand you really want to exclude
> all TCPs).
> Or put differently I would like to see a description of the behavior precise
> enough that I can easily deduce from a packet capture whether a given flow
> meets the NQB requirements, in a sense an enumerated list of expected
> behaviors and of behaviors that disqualify for NQB marking/treatment.
> I believe that would be in agreement with the stated goal that "the NQB
> designation and marking would be intended to convey verifiable traffic
> behavior, not needs or wants"
> 	Conceptually my next issue is that I think just trusting the end-points
> to mark correctly is too optimistic and that any scheduler giving special
> treatment according to NQB-ness must be required to ascertain that each
> individual flow is behaving according to the NQB requirements and ideally re-
> mark non-compliant flows CS0 to protect downstream devices (if NQB maps
> to anything but AC_BE). This requirement mainly is driven by the side effects
> NQB will have on wifi, so if the special wifi power of NQB goes away, this
> issue gets less important to me (also there are wifi scenarios where queue
> protection as proposed upstream will not be sufficient).
> 	Final and strongest objection: incompatibility between current wifi
> gear and NQB.
> I hope we can all agree that :
> a) Wifi is a shared system not only between up- and downstream between
> stations and APS, but also due to the fact that the same frequency bands are
> often shared between different wifi networks (either directly be using the
> same center frequency band, or by overlap of the side-bands).
> b) no currently deployed wifi gear (NICs and APs) supports special treatment
> for NQB flows.
> c) most deployed gear supports access classes and will in all likelihood use the
> default DSCP to AC mapping you described in a different post. (as far as I can
> tell WMM is mandatory for wifi >= 802.11n)
> d) The AC priority system is essentially a (weak) precedence system (lower
> ACs might sneak in a tx_op even with higher AC queues full, but that is gong
> to be rare and can result in multi second delays for packets in the lower ACs,
> as observed in my home network). So in traditional thinking using ACs
> requires some sort of access control/rate limiting to avoid undesired
> starvation issues.
> e) For quite a number of end-users the wifi rate is what limits the internet
> access and not the access link itself (at least for higher bandwidth internet
> plans in crowded RF environments like apartment buildings)
> f) NQB is not intended to reduce the "performance of QB flows" but rather
> isolate well behaving sparse and/or paced fixed-rate traffic from the
> transient queue building effects of non-paced non-rate-limited flows.
> g) NQB is intended such that  "that malicious or badly configured nodes can't
> abuse it."
> h) NQB traffic is not rate limited or rate policed. Even if each individual flow is
> self rate-limiting the aggregate clearly is not intended to be.
> i) For QB flows, the QB queue provides better performance (considering
> latency, loss and throughput) than the NQB queue
> j) NQB-marking (hopefully) will one day survive end to end.
> Now, from e) it follows that the ISPs upstream NQB aware AQM will not
> trigger in quite a number of instances, so making sure that NQB does no
> harm is up to the wifi gear. Due to h) it is clear that NQB traffic will be able to
> saturate the wifi link. Together with c) and d) this means that the mapping of
> NQB to AC by the default rules will define the precedence level of the
> saturating traffic. Due to a) good wifi citizens try to be considerate in not
> hogging a channel to themselves (and due to CSMA/CA users on the same
> channel should stochastically share tx_ops "fairly" in each AC). Most wifi
> traffic currently is using AC_BE, but due to j) and h) that might change in the
> future.
> I now argue that using anything but AC_BE for NQB will violate f) and g).
> Essentially in this not unlikely scenario NQB will confer a super-power (lower
> latency and higher bandwidth) without any checks and balances
> (incompatible with g)).
> To make it explicit, with ACs > BE in play, AC_BE will essentially only get the
> left-over bandwidth, which runs counter to i) as now QB flows need to mark
> themselves such that they also get scheduled into the same AC as NQB to get
> a level playing field.
> For me that obviously means that the choice of DSCP for NQB needs to take
> care that the above scenario can not happen.
> So I vote to follow Ruediger's proposal for a 000xx1 codepoint for the
> intended PHB such that the status quo is not negatively affected. With the
> added bonus that this now has a chance of actually surviving end to end
> today.
> NQB aware APs/NICs can then implement proper NQB handling with or
> without selective priority boosting to different ACs while making sure no
> other flows/APs are starved.
> Since I have made this point several times, without convincing you, I would
> be happy to hear your rationale why this scenario is unlikely enough to still
> justify aiming for a DSCP that maps to a AC > BE?
> Ideally that rationale would be more refined than just noting, that this kind of
> abuse is possible already today, the aim should be to at least "do no harm",
> not just "do not make it significantly worse".
> Best Regards
> 	Sebastian