Re: [tsvwg] NQB PHB description and assumptions

Bob Briscoe <> Tue, 10 September 2019 14:29 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 8BBF31201C6 for <>; Tue, 10 Sep 2019 07:29:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.988
X-Spam-Status: No, score=-1.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_KAM_HTML_FONT_INVALID=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id IFzNU9G0uehH for <>; Tue, 10 Sep 2019 07:29:34 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id A91A01201CE for <>; Tue, 10 Sep 2019 07:29:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=TqGHLzcTTEf8m236LiEnNwRwM/GsuZoEqofzeWQSCSA=; b=JLanFFo851zUR2sr39EdYhgTr NZIniJQZnB0g/Y3AFdBJIT1Miex8Y1yNx9TLZV9p6uNWgW8mFrVu2pTRZx6ST4DJZwe/Lrr0R1uoL J5uqvKZVN6FNOhY8/1Ij4CnD2/WfT2jddjSCtRzTe63bjIaoWGD4IouX2IwYK2N/3CJ8f9rXNpgr0 Q1MTPV2bZKB5NsX/f16ReNx+Rs0mpzyEKf4xfm1qywRY0c4lKgSUkQzQeUqFfQwnMylq2AB/Vp87r /ovVJcT0KmTsJZVVb172pYFsG39i2UTzHYnj7QPNTveZg06bKYIY+R8MQOvlm6qrU2BftdJHwQEFL 5BXYi0DFA==;
Received: from [] (port=41676 helo=[]) by with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <>) id 1i7h97-0005Pf-SP; Tue, 10 Sep 2019 15:29:30 +0100
From: Bob Briscoe <>
Message-ID: <>
Date: Tue, 10 Sep 2019 15:29:29 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="------------F697F26277DD0EFF2B871CD9"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] NQB PHB description and assumptions
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 10 Sep 2019 14:29:43 -0000


I'm not a co-author. But I did work alongside Greg while he and Thomas 
were developing the idea. So I hope my interventions are helping, but 
they might be adding confusion.

I will start out by saying that people who think they understand QoS 
will not understand NQB unless they wipe most of their 'old-school QoS' 

Responses inline tagged [BB2]...

On 10/09/2019 11:00, wrote:
> Hi Bob,
> a PHB usually isn’t defined to the detail of implementation. AF and EF 
> allow for a coarse understanding how to implement and configure the 
> required functions. For me, NQB isn’t at that level yet.
[BB2] NQB is primarily a property of a single traffic flow.

I believe the description of an NQB PHB would be pretty much null, which 
is perhaps why you think it isn't sufficient yet. As I said, the primary 
property of the PHB is merely isolation, i.e. a separate queue for NQB 
traffic. The sentence in the draft describing the 'useful property' of 
an NQB PHB (that you're worse off mismarking) is really the only 
additional specification it needs.

I asked the authors for examples of how an NQB queue might be scheduled, 
but they would be examples, not definitions of the PHB.

> I remove some of the prior text and reply [RG], marking your 
> statements [BB]
> #######################
>       * You state that NQB is enabled largely by high bandwidth
>         Internet accesses. The NQB draft doesn’t.
>      [BB] I didn't intend hi b/w to be a requirement for NQB.
>      [BB] That was merely me saying that, if you have hi bandwidth, 
> NQB could be designed like I proposed in my straw man. There might be 
> other NQB designs for low b/w (obviously more difficult tho).
>       [BB] I.e. the more b/w you have, the less you're tied to 
> 'old-school' QoS designs, where low delay depends on bandwidth priority.
> [RG] I’d appreciate, if the draft clarifies this point. Is NQB 
> recommended for high bandwidth links? If yes, what’s meant by “high 
> speed”? When is “old-school” QoS recommended and how will NQB work 
> there? Some rough guidance should be given.
> #################################
> On 05/09/2019 19:23, Bob Briscoe wrote:
>         _Background_: NQB has become possible largely because Internet
>         access link rates have typically become fast enough that the
>         serialization delay of a packet can be sub-millisecond, and
>         therefore a queue of a few packets introduces delay that is
>         small relative to other non-optional sources of delay like
>         propagation. In these cases we no longer need priority
>         scheduling for low delay.
>    [BB] I only said sub-millisecond 'cos it's about an order or 
> magnitude lower than typical propagation delays over the public 
> Internet, and therefore tends towards an insignificant contribution to 
> overall delay.
>     [BB] In the context of a data centre with 1ms RTT diameter, that 
> sentence would translate to 'link rates ... fast enough that 
> serialization delay .. can be sub-100us'.
> [RG] I’d appreciate text telling a reader, what NQB is about and how 
> it works. In that sense, picking up and mixing your statements, is it 
> something like that:
> The bitrate configured for NQB traffic should ensure that a queue of a 
> few MTU sized packets introduces a delay causing only an insignificant 
> contribution to the overall end-to-end delay.
[BB2] Again, I used link rate as a condition for the straw man NQB 
implementation I described. I do not believe link rate is (or should be) 
a condition in any general definition of NQB.
> ######################################
>       * Further, your example suggests a 3ms@accessbandwidth buffer
>         depth for the NQB Queue. ..[snip]
>     I’d like to add another point: if, in the example below, the NQB
>     traffic stays below 50% link utilization also in the case of
>     access link congestion, it shouldn’t see a queue.
>     [BB] Er no... Consider bursts of ten back-to-back 1500B packets 
> sent on average every 30 ms from a 1Gb/s NIC into a 100Mb/s link. 
> That's an average of only 4% utilization (4Mb/s), but at the end of 
> each burst the queue is 9 packets.
> [RG] Your explanation is correct. Unfortunately, it doesn’t help me. 
> Do you want to say, the example flow is NQB compliant? Then this 
> shouldn’t be an issue (it adds of 2,4 ms delay in the worst case)? Or 
> this is an issue? But what happens then and what’s the impact on the 
> NQB traffic? None? I’m lost. Your example proposed a configuration and 
> I take the available NQB text giving some examples for NQB traffic by 
> the letter. What’s wrong here?
>       [BB] Even if each NQB flow perfectly paces its own packets, the 
> size of queue will depend on the number of uncoordinated flows. That's 
> because the number of flows determines how many packets could arrive 
> at about the same time and cause a queue. My calculation gave the 
> worst case queue where a packet from all n flows happens to arrive at 
> the same time. Statistically unlikely, but I explained I was 
> calculating the tail of the delay distribution.
> [RG] Yes. That’s why I’d expect the draft to be more specific on the 
> traffic properties expected by NQB compliant traffic. Current text is
> There are many applications that send traffic at relatively low data
> rates and/or in a fairly smooth and consistent manner such that they
>      are highly unlikely to exceed the available capacity of the network
>      path between source and sink.
>         These Non-queue-building (NQB) flows ..typically ….
>      send traffic at a lower data rate and don't seek the capacity of the
>        link (examples: online games, voice chat, DNS lookups).  Here the
>      data rate is essentially limited by the Application itself.
> [RG] What’s a “lower data rate”? What are the multiplexing conditions 
> to avoid a queue build up at a link operating NQB (to quote you:… 
> because the number of flows determines how many packets could arrive 
> at about the same time and cause a queue)? How does an application 
> developer and a service provider decide, which traffic is NQB? Which 
> congestion indication is required by NQB and is an AQM expected? If 
> yes, what is the AQM supposed to do an where’s that specified?
[BB2] Wipe your preconceptions. I don't think NQB is as precise as you 
want it to be. My calculations were precise for the traffic scenario I 
assumed, but the service cannot be defined precisely, because the 
behaviour aggregate could contain an arbitrary number of flows (even 
though there will be a likely number).

It's more like: "NQB is targeted at access links, where flow 
multiplexing will probably be low enough most of the time that it will 
be worthwhile isolating the aggregate of all NQB flows from any QB 
flows, such that queuing delay for all of the NQB flows together, 
without any additional controls."

The main point is that, even if "old-school" QoS folks are tempted to 
over-engineer this, the isolation of the NQB behaviour aggregate gives 
most of the benefit. So let's try to keep it to isolation alone.

> [RG] If the sole signaling of congestion is to forward flows 
> identified as causing congestion by a QB PHB, the draft should state 
> it that way. If the bottleneck is congested and the NQB flows consume 
> more than (say) 80% of the bandwidth set aside for the NQB PHB, all 
> flows could be perfectly well behaving and still a queue starts to 
> build up. How does NQB work then and how to avoid these situations or 
> return to desired NQB operational conditions for all NQB marked flows 
> (please note, I don’t assume misuse and per definition admission 
> control is absent)?
[BB2] As above, yes the queue does build up as the amount of NQB traffic 
increases. But if one assumes all that NQB traffic is necessary, just 
isolating all the NQB from the QB still gives all the NQB traffic the 
lowest delay possible, given the link resource available.

The DOCSIS queue protection function keeps queuing delay below a 
threshold, by reclassifying packets from the least NQB flows to protect 
the most NQB flows. But you might not want that.



> Regards,
> Ruediger

Bob Briscoe