Re: [tsvwg] NQB PHB description and assumptions

<> Tue, 10 September 2019 10:01 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 807C31200A3 for <>; Tue, 10 Sep 2019 03:01:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.288
X-Spam-Status: No, score=-4.288 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_KAM_HTML_FONT_INVALID=0.01] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id trLe6eo9Erp6 for <>; Tue, 10 Sep 2019 03:01:08 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 3077E12009E for <>; Tue, 10 Sep 2019 03:01:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;;; q=dns/txt; s=dtag1; t=1568109660; x=1599645660; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=4diXDfROzekME0NnPZxMH1JlskcC/7USrn7/geBpZS8=; b=Q2kMxXlafy1mc8oASjL39qHaN0dEZNLdeWlgFQKKyeSMKPy7a/JDUz/A ivxIsARnBb0WIQB6idMLLx9khFK7f/6N1kMHUBQSsacBvS6043e+qORBK oqClxw1NJfLqMpqeOBXnMBHb/jiy0h/DlXfXKDmh+v0sKQCEMQWLb+8aZ YSSIrLojy1ceHQlBcxo4RX1kuhJJZx/uFJAEATN9jzsZWy9cnjrw+49Bb bypwVW7xTcznQbUzoIY2QzlDWHu7xP/OCdpDwTTmn/oh3XH7OaJRhODUY CW52f9KmWsgPtw+0Suajbzo3/73yp9fa2Qm3yK/GwBVOqm60V3AcUNHvo Q==;
Received: from ([]) by with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Sep 2019 12:00:56 +0200
X-IronPort-AV: E=Sophos;i="5.64,489,1559512800"; d="scan'208,217";a="359531783"
X-MGA-submission: =?us-ascii?q?MDGmoGEvOjuctTxrepRecnbYL4cAhskxZD339H?= =?us-ascii?q?x/fVk6tDlL1F0DBxy/Cq52FEFebAb2Rjw4iORg+u6zMjU0RJhgR1hEvX?= =?us-ascii?q?5sxcKvYrWpZxCqeNJwLH6thqyvKc+nK853WgfMi4wFpM6T1ixVCndAFc?= =?us-ascii?q?h85Y1qM7o6VmD74UhRh2WoGQ=3D=3D?=
Received: from ([]) by with ESMTP/TLS/AES256-SHA; 10 Sep 2019 12:00:57 +0200
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 10 Sep 2019 12:00:51 +0200
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Tue, 10 Sep 2019 12:00:51 +0200
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 10 Sep 2019 12:00:48 +0200
Received: from LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE ( by LEJPR01MB1226.DEUPRD01.PROD.OUTLOOK.DE ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.20; Tue, 10 Sep 2019 10:00:51 +0000
Received: from LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE ([fe80::987d:70fa:4d72:d6da]) by LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE ([fe80::987d:70fa:4d72:d6da%5]) with mapi id 15.20.2241.018; Tue, 10 Sep 2019 10:00:50 +0000
From: <>
To: <>
CC: <>
Thread-Topic: [tsvwg] NQB PHB description and assumptions
Thread-Index: AdVkfBP29FODUeIhSciHo/YcUWQEYgCl/QiAAChMzhA=
Date: Tue, 10 Sep 2019 10:00:50 +0000
References: <LEJPR01MB11783FACEF3688ED3986EA6C9CBA0@LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE> <>
In-Reply-To: <>
Accept-Language: de-DE, en-US
Content-Language: de-DE
authentication-results: spf=none (sender IP is );
x-originating-ip: []
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a5f6f4f2-1daa-49a1-ff19-08d735d5c2c7
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:LEJPR01MB1226;
x-ms-traffictypediagnostic: LEJPR01MB1226:
x-microsoft-antispam-prvs: <LEJPR01MB122625E17176F15B2FE6ED189CB60@LEJPR01MB1226.DEUPRD01.PROD.OUTLOOK.DE>
x-ms-oob-tlc-oobclassifiers: OLM:5797;
x-forefront-prvs: 01565FED4C
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(376002)(366004)(136003)(39860400002)(396003)(189003)(199004)(446003)(81166006)(478600001)(11346002)(14454004)(66574012)(5660300002)(33656002)(486006)(53936002)(53546011)(66066001)(6306002)(54896002)(102836004)(26005)(790700001)(14444005)(6116002)(3846002)(316002)(256004)(19627235002)(66946007)(2906002)(52396003)(4326008)(55016002)(476003)(8936002)(76176011)(71200400001)(85182001)(76116006)(8676002)(81156014)(7696005)(6916009)(66446008)(64756008)(66556008)(66476007)(85202003)(71190400001)(7736002)(86362001)(186003)(9686003)(777600001); DIR:OUT; SFP:1101; SCL:1; SRVR:LEJPR01MB1226; H:LEJPR01MB1178.DEUPRD01.PROD.OUTLOOK.DE; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None ( does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: 29YY4ZUvotmDhMq8tuPtRE7ZmzNrm5+39CpO9gZWjqNVwcCIL0DGDUJo6gCcdPFxSddnazdlwKPL0eIKaN4FyMyZQBn7kORYdiHpajCLcf5/RFzVi5U4qAhrrMcioCkt/bBtoSZXkasE2Y9bbxxpuaHohgIRuq8FZPFMiLxXcvAO+ai/gGrpoY6SY0kCc8vjuP1UDFwnpJn7LbzJRejOal89uBFVoXVkFRNKDs/myT1a0nndbAeu2K3mKDKjxwcd25pKhK/64i7fCd9j6HePspmGxSMUATz/s4eh5bwQXRw+wxwb5+ozN2kaJT7m1LyF/R0M4hcWI/WywnxwJGNA6JXhTJVpFjaPjTyNM1I2Lx3e0C146agOvcDmyILLrA9UOZ11uK9jl1YEXgonVQAyHt5oSqGSWtKMlGkE5+D56sA=
x-ms-exchange-transport-forked: True
Content-Type: multipart/alternative; boundary="_000_LEJPR01MB117840B7D90565CA4BEDF15B9CB60LEJPR01MB1178DEUP_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: a5f6f4f2-1daa-49a1-ff19-08d735d5c2c7
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2019 10:00:50.7238 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bde4dffc-4b60-4cf6-8b04-a5eeb25f5c4f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 9P2Oq3bvYBoK4cTcu3BDwNPMrw97eg39XNfHDykSOANIM7mhd0QJXq1GoC0utUZdmNaSPuGHpnOOEA/tcHHP3jtFwUk0ODrqt6ves2SOPU0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LEJPR01MB1226
Archived-At: <>
Subject: Re: [tsvwg] NQB PHB description and assumptions
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 10 Sep 2019 10:01:12 -0000

Hi Bob,

a PHB usually isn’t defined to the detail of implementation. AF and EF allow for a coarse understanding how to implement and configure the required functions. For me, NQB isn’t at that level yet.

I remove some of the prior text and reply [RG], marking your statements [BB]


  *   You state that NQB is enabled largely by high bandwidth Internet accesses. The NQB draft doesn’t.
     [BB] I didn't intend hi b/w to be a requirement for NQB.

     [BB] That was merely me saying that, if you have hi bandwidth, NQB could be designed like I proposed in my straw man. There might be other NQB designs for low b/w (obviously more difficult tho).

      [BB] I.e. the more b/w you have, the less you're tied to 'old-school' QoS designs, where low delay depends on bandwidth priority.

[RG] I’d appreciate, if the draft clarifies this point. Is NQB recommended for high bandwidth links? If yes, what’s meant by “high speed”? When is “old-school” QoS recommended and how will NQB work there? Some rough guidance should be given.


On 05/09/2019 19:23, Bob Briscoe wrote:
Background: NQB has become possible largely because Internet access link rates have typically become fast enough that the serialization delay of a packet can be sub-millisecond, and therefore a queue of a few packets introduces delay that is small relative to other non-optional sources of delay like propagation. In these cases we no longer need priority scheduling for low delay.

   [BB] I only said sub-millisecond 'cos it's about an order or magnitude lower than typical propagation delays over the public Internet, and therefore tends towards an insignificant contribution to overall delay.

    [BB] In the context of a data centre with 1ms RTT diameter, that sentence would translate to 'link rates ... fast enough that serialization delay .. can be sub-100us'.

[RG] I’d appreciate text telling a reader, what NQB is about and how it works. In that sense, picking up and mixing your statements, is it something like that:

The bitrate configured for NQB traffic should ensure that a queue of a few MTU sized packets introduces a delay causing only an insignificant contribution to the overall end-to-end delay.


  *   Further, your example suggests a 3ms@accessbandwidth buffer depth for the NQB Queue. ..[snip]

I’d like to add another point: if, in the example below, the NQB traffic stays below 50% link utilization also in the case of access link congestion, it shouldn’t see a queue.
    [BB] Er no... Consider bursts of ten back-to-back 1500B packets sent on average every 30 ms from a 1Gb/s NIC into a 100Mb/s link. That's an average of only 4% utilization (4Mb/s), but at the end of each burst the queue is 9 packets.

[RG] Your explanation is correct. Unfortunately, it doesn’t help me. Do you want to say, the example flow is NQB compliant? Then this shouldn’t be an issue (it adds of 2,4 ms delay in the worst case)? Or this is an issue? But what happens then and what’s the impact on the NQB traffic? None? I’m lost. Your example proposed a configuration and I take the available NQB text giving some examples for NQB traffic by the letter. What’s wrong here?

      [BB] Even if each NQB flow perfectly paces its own packets, the size of queue will depend on the number of uncoordinated flows. That's because the number of flows determines how many packets could arrive at about the same time and cause a queue. My calculation gave the worst case queue where a packet from all n flows happens to arrive at the same time. Statistically unlikely, but I explained I was calculating the tail of the delay distribution.

[RG] Yes. That’s why I’d expect the draft to be more specific on the traffic properties expected by NQB compliant traffic. Current text is

        There are many applications that send traffic at relatively low data
        rates and/or in a fairly smooth and consistent manner such that they
        are highly unlikely to exceed the available capacity of the network
        path between source and sink.
        These Non-queue-building (NQB) flows ..typically ….
        send traffic at a lower data rate and don't seek the capacity of the
         link (examples: online games, voice chat, DNS lookups).  Here the
        data rate is essentially limited by the Application itself.

[RG] What’s a “lower data rate”? What are the multiplexing conditions to avoid a queue build up at a link operating NQB (to quote you:… because the number of flows determines how many packets could arrive at about the same time and cause a queue)? How does an application developer and a service provider decide, which traffic is NQB? Which congestion indication is required by NQB and is an AQM expected? If yes, what is the AQM supposed to do an where’s that specified?

[RG] If the sole signaling of congestion is to forward flows identified as causing congestion by a QB PHB, the draft should state it that way. If the bottleneck is congested and the NQB flows consume more than (say) 80% of the bandwidth set aside for the NQB PHB, all flows could be perfectly well behaving and still a queue starts to build up. How does NQB work then and how to avoid these situations or return to desired NQB operational conditions for all NQB marked flows (please note, I don’t assume misuse and per definition admission control is absent)?