Re: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-06.txt

[Bob Briscoe <[e-mail]>]

Greg,

I've had a look at -07.
Thank you. See [BB] inline...

On 29/07/2021 03:48, Greg White wrote:
>
> Bob,
>
> Thank you very much for your review.  I am preparing a draft-07 that 
> addresses many of these comments. Please see some responses below.
>
> -Greg
>
> *From: *Bob Briscoe <[e-mail]>
> *Date: *Friday, July 16, 2021 at 5:27 AM
> *To: *Greg White <[e-mail]>, "[e-mail]" 
> <[e-mail]>
> *Subject: *Re: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-06.txt
>
> Greg, list,
>
> Here's my review of the nqb draft-06:
>
>
>       ==Gap: Partial Deployment==
>
>
> In §6, it says:
>
>     "In the case of an NQB flow that isn't marked as NQB and ends up
>     in the QB queue, it would only impact its own quality of service,
>     and so it seems to be of lesser concern."
>
> This made me realize that the draft doesn't say more generally what 
> will be the outcome if an NQB flow ends up in a QB queue, just simply 
> because NQB is not supported at the bottleneck. It talks about this in 
> specific cases (e.g. WiFi), but not in general.
>
> [GW] Well there is also some related text in 5.2 and in the paragraph 
> preceding 5.1.  But, I agree, this could be covered more directly.  I 
> will add some text on that topic in 5.2 (Aggregation), which is where 
> it seems most appropriate to me.
>

[BB] Yes, the beginning of §4.2 covers this from the network point of 
view, thank you. I suggest
s/This preservation of the NQB marking/
  /This preservation of the NQB marking distinction/

I was also thinking from the sender's point of view that, if it marks 
its traffic NQB, even if the bottleneck doesn't support NQB, it at least 
won't get worse delay than if it had marked its traffic as Default (BE). 
I guess it would be futile to require non-NQB nodes not to selectively 
block or discard NQB traffic. But the sender does need some reassurance 
that the service will never be worse than Default, so that it doesn't 
have to think twice before marking as NQB.

>
>       ==SHOULDs or MUSTs?==
>
>
> It might help to add some reasoning for why the draft allows 
> implementers /not/ to comply with the 'SHOULDs'.
> To help in this process, below I've tried to describe the exceptions 
> to each SHOULD, so it could be written as "MUST do x..., except if y...".
>
> · 5. DSCP Marking of NQB Traffic
>
> o"...SHOULD identify themselves to the network using a Diffserv Code 
> Point (DSCP) of 45..."
>
> §Doesn't say what valid cases would be for not using 45 and how that 
> would interwork with other networks
>
> §e.g. some other DSCP agreed locally with the access network would 
> work fine, but some other DSCP to work with an unmanaged home WiFi 
> might not work with the rest of the path.
>
> [GW] Ok, I added “In networks where another (e.g. a local-use) 
> codepoint is designated for NQB traffic, or where specialized PHBs are 
> available that can meet specific application requirements (e.g. a 
> guaranteed-latency path for voice traffic), it may be preferred to use 
> another DSCP.”
>

[BB] "May be preferred" is sort-of worse. I was trying to replace 
vagueness with more certainty.

I was thinking more of something like. 'The "SHOULD" is intended not to 
preclude use of a different DSCP by arrangement with the local operator.'

> o"If the application's traffic exceeds more than a few packets per 
> RTT, or exceeds approximately 1 Mbps on an instantaneous 
> (inter-packet) basis, the application SHOULD NOT mark its traffic with 
> the NQB DSCP."
>
> §If this were 'MUST NOT' it would be too prescriptive as the Internet 
> scales. But perhaps it's better to say 'MUST NOT' and allow more 
> flexibility in the quantification. For instance, instead of 'a few' 
> and 'approximately 1 Mbps', explain how these numbers might scale as 
> the Internet scales.
>
> [GW] I understand your point.  But am going to hold off on making this 
> change.  It isn’t immediately clear to me how to structure this as a 
> MUST NOT.  Maybe we can discuss this in the WG.
>

[BB] I agree that it would probably be impossible to come up with a MUST 
NOT here. But with a SHOULD NOT, I still think the quantified 
requirement will become outdated unless it is somehow explained how it 
will scale in the future. Yes, for WG discussion.


> o"the application SHOULD instead implement a congestion control 
> mechanism, for example as described in Section 3.1 of [RFC8085] or 
> [I-D.ietf-tsvwg-ecn-l4s-id]."
>
> §I think it's best to say neither SHOULD nor MUST here, but instead 
> refer to drafts that say normative things. Something like this: "the 
> application has to instead implement a relevant congestion control 
> mechanism, for example...". After all, an NQB RFC is not really 
> entitled to say what non-NQB traffic should do.
>
> [GW] I agree.
>
> o"...nodes that do not support the NQB PHB SHOULD treat NQB marked 
> traffic the same as traffic marked "Default", and SHOULD preserve the 
> NQB marking."
>
> §Again, this is putting requirements on non-NQB nodes, but in this 
> case the requirement is not on the implementer, rather it's about the 
> classifier config of the operator. So this ought to be phrased as 
> operator guidance (e.g. RECOMMENDED).
>
> §And it needs to say what will happen in pre-existing networks, where 
> the operator is not aware of either of these recommendations (see 
> earlier point about partial deployment).
>
> [GW] I believe that I’ve got this covered now in the Aggregation section.
>

[BB] Yup

> · 5.1. End-to-end usage and DSCP Re-marking
>
> o"Absent an explicit agreement to the contrary, networks that support 
> the NQB PHB SHOULD preserve a DSCP marking distinction between NQB 
> traffic and Default traffic when forwarding via an interconnect from 
> or to another network. To facilitate the default treatment of NQB 
> traffic in backbones and core networks, networks SHOULD remap NQB 
> traffic (DSCP 45) to DSCP 5 prior to interconnection, unless agreed 
> otherwise between the interconnecting partners."
>
> §Both these SHOULDs could be MUSTs if the sentence started with "To 
> support NQB, ..." because they both allow a locally agreed exception.
>
> §One reason for not complying would be lack of space for the marking 
> (e.g. MPLS or Ethernet header space). So it would be worth saying that 
> one way to preserve the distinction would be encapsulation, with the 
> distinction only preserved in the inner.
>
> o"In order to enable interoperability with WiFi equipment, networks 
> SHOULD remap NQB traffic (e.g. DSCP 5) to DSCP 45..."
>
> §Again, this could be made into a MUST by adding "To support NQB, ..." 
> and allowing locally agreed exceptions.
>
> [GW] For the first one (preserving a marking distinction), I agree 
> with you (and made it a MUST).  For the others that discuss specific 
> DSCPs, I think we need to leave them as SHOULDs because (by 
> definition) a PHB spec only “recommends” DSCPs and does not mandate them.
>

[BB] Then another way to tighten this up would be to try to describe 
valid exceptions.

> · 5.2. Aggregation of the NQB PHB with other Diffserv PHBs
>
> o"the NQB DSCP SHOULD be preserved"
>
> §As above, explain that you are talking about encapsulation to 
> preserve the distinction by preserving the inner
>
> §(and it would be worth clarifying that the degradations discussed in 
> the previous para are only within the aggregation network, not after 
> traffic has been disaggregated again).
>
> [GW] I’ll cover the first sub-bullet, but I’m not sure what more I 
> could say on the second sub-bullet. The degradations 
> (loss/latency/jitter), while they would be introduced in an 
> aggregation node, clearly are felt end-to-end.  The statement around 
> preserving NQB DSCP already mentions that it should be preserved “in 
> order to limit the negative impact that such networks would have on 
> end-to-end performance for NQB traffic.”  Maybe I’m misunderstanding 
> what you mean?
>

[BB] First, I've just realized that perhaps this should say "the NQB 
DSCP /distinction/ SHOULD be preserved".

Then, yes, I didn't mean performance degradations after the tunnel 
egress. I meant that it might be useful to explain how the DSCP 
distinction can be preserved both across the tunnel and after it. Not 
essential, because such tutorial is really the job of a Diffserv 
aggregation RFC. I meant that encapsulation can be used to aggregate 
DSCPs together (e.g. where MPLS or Ethernet provide fewer Traffic Class 
codepoints), but still preserve DSCP distinctions within the 
encapsulation (inner headers) so that, beyond the scope of the 
encapsulation the distinct DSCPs can be revealed again.


> · 6. Non-Queue-Building PHB Requirements
>
> oThere are lots of SHOULDs here, without any reasoning given for why 
> exceptions are allowed, or what sort of exceptions are not allowed. 
> They all relate to preserving vs. enforcing incentives. So I think it 
> could be said explicitly that where SHOULD rather than MUST is used in 
> this section, it is because the details depend on the threat level in 
> the deployment environment and whether incentives are working 
> sufficiently.
>
> [GW] I think your comment mainly applies to two of the SHOULDs 
> (equivalent forwarding preference requirement and fair scheduler 
> requirement), so I added it there.  The one that I’m debating is the 
> “… SHOULD NOT be rate limited or rate policed separately from QB 
> traffic …” requirement.  I don’t think this one is entirely related to 
> incentives.  Even in a controlled environment where there is no threat 
> of mismarking, I think this would still be the recommendation.  But, 
> I’m having trouble justifying why it should be a SHOULD as opposed to 
> a MUST.  My original rationale was that there is always a corner case, 
> but I think I would be ok with it either way.
>

[BB] Interesting that you've brought that up. I was looking at that and 
wondering whether it was intended to mean "...need not be rate limited 
or policed separately..." There is an obvious downside of rate policing 
two traffic classes separately when it's unnecessary: it unnecessarily 
limits users. But if the operator wants to limit its users 
unnecessarily, surely we don't need to stop it? Is there an interop 
concern here?

> · 11. Example Use Cases
>
> o"To support the NQB PHB, the mobile network SHOULD be configured to 
> give UEs a dedicated, low-latency, non-GBR, EPS bearer"
>
> §I assume the SHOULD is because it depends on how things are done 
> locally. This could be handled by describing this non-normatively, as 
> a way to satisfy the 'MUST' requirement in §6 to provide a separate 
> NQB queue.
>
> §Similarly "SHOULD be routed" and "SHOULD NOT be routed" can be 
> non-normative as ways to satisfy earlier requirements in current 
> mobile networks.
>
> o"WiFi equipment SHOULD map the NQB codepoint 45 into a separate queue 
> that shares an Access Category"
>
> §As for mobile, this seems to be a way to satisfy the earlier 
> requirements for the /current/ way that WiFi works. These are 
> examples, not normative requirements, aren't they?
>
> §Ditto for "SHOULD deploy a policing function" and "SHOULD map the 
> recommended NQB code point 45 to UP_5"
>
> oHowever, I understand that you might want to draw implementers' 
> attention to what they have to do, by using capitalized requirements.
>
> [GW] Yes, I like these being called out as requirement statements.
>
>
>       ==Flow Queuing in scope or not?==
>
>
> I think the draft ought to specify that its scope is primarily shared 
> queues, although there would be some merit in saying how NQB would 
> work with an FQ. Per-flow queues provide the necessary isolation, but 
> there's no need to make any of the other requirements apply to an FQ 
> {however, I'm not sure - see Note 1}. Stated another way, there has 
> been no standardization of how FQ systems handle DSCPs, but this draft 
> would be the wrong place to introduce that subject for one specific DSCP.
>
> {Note 1}: Except, the isolation provided by an FQ is not absolute. 
> E.g. an attacker can spoof flow identifiers on well-known ports in the 
> hope it will match a flow ID being used by a genuine NQB flow and 
> create a long-standing queue (which might otherwise just exist as a 
> series of transient queue instances for sparse packets). I doubt it 
> would be possible to provide local per-queue traffic protection 
> against such traffic (nothing to distinguish it). But this is where 
> other protection techniques come in, like scrubbing farms that can 
> identify such attack traffic closer to the spoofed sources.
>
> [GW] Good suggestion.  I added a sentence in the Introduction.
>
>
>       ==Partially described scenarios==
>
> I didn't find the following scenarios easy to understand:
>
> ·"Active Queue Management (AQM) mechanisms (such as PIE [RFC8033], 
> DOCSIS-PIE [RFC8034], or CoDel [RFC8289]) can improve the quality of 
> experience for latency sensitive applications, but there are practical 
> limits to the amount of improvement that can be achieved without 
> impacting the throughput of capacity-seeking applications, 
> particularly when only a few of such flows are present."
>
> oFor all the text after the 'but', I think there's some scenario in 
> the authors' heads that isn't fully explained.
>
> ·"Either approach comes with trade-offs: aggregating with 
> Default/Elastic traffic could result in a degradation of 
> loss/latency/jitter performance for NQB traffic, while aggregating 
> with Real-Time risks creating an incentive for mismarking of 
> non-compliant traffic as NQB."
>
> oThe scenario isn't clear - is it talking about a bottleneck within 
> the aggregated network, or after it?
>
> oThe sentence about r-t incentives needs to explain how it reaches 
> that conclusion, rather than expecting the reader to recreate the 
> explanation.
>
> [GW] Ok, I added some more detail for those.
>

[BB] I understand now. Much better.

> ·"In the case of the pipe model, any DSCP manipulation (re-marking) of 
> the outer header by intermediate nodes would be discarded at tunnel 
> egress, potentially improving the possibility of achieving NQB 
> treatment in subsequent nodes."
>
> o...Or potentially making it worse? It all depends whether the inner 
> marking is meaningful edge-to-edge. I presume the word 'potentially' 
> alludes to this, but the sentence is rather opaque as it stands.
>
> ·"...tunnel protocols that are sensitive to reordering can result in 
> undesirable interactions..."
>
> o"undesirable interactions" is rather opaque. Does it mean within the 
> tunnel? or after the egress?
>
> [GW] The language comes from RFC2983, and the sentence begins with “As 
> is discussed in RFC2983, …”  I think it is ok to assume a confused 
> reader will know where to look to figure this one out.
>

[BB] It wasn't the pipe model I didn't understand. The pipe model 
certainly preserves the inner, but that's only useful if the mapping of 
the inner DSCP to a meaning  is shared between the operators of the two 
ends of the pipe, which are not necessarily neighbours. Or are you 
meaning that they can refer to RFC2983 for that as well?

> ·"into a separate queue that shares an Access Category with default 
> traffic "
>
> oThe word 'shares' threw me at first - I think I've now understood; it 
> must just mean 'it is within the same Access Category'. I was 
> wondering how it could both be separate and be shared.
>
> [GW] Ok, fixed.
>
> ·"This document recommends the implementation of a traffic protection 
> mechanism to achieve this goal, but recognizes that other options may 
> be more desirable in certain situations."
>
> oI don't think 'other options' have been described in the doc (yet). 
> So this seems opaque
>
> [GW] Two other options are provided in the PHB Requirements section.
>

[BB] Found them. Thanks.

>
>       ==Relationship with L4S (§9)==
>
>
> Ought to say that a non-relationship to L4S is also fine.
> And ought to explain that L4S is experimental, while NQB is stds track.
>
> [GW] Done.
>

[BB] Thx.
This section cites l4s-arch and aqm-dualq-coupled. Altho both those 
mention NQB, ecn-l4s-id is the most relevant L4S draft. It says how to 
interwork with NQB traffic (specifically §5.4.1). I suggest you cite it 
instead of, or in addition to l4s-arch.


>       ==Structure==
>
> There were times when I couldn't really see a logical flow.
>
> Here's the current ToC for reference:
>  Table of Contents
> 1.  Introduction
> 2.  Requirements Language
> 3.  Non-Queue-Building Behavior
> 4.  The NQB PHB and its Relationship to the Diffserv Architecture
> 5.  DSCP Marking of NQB Traffic
> 5.1.  End-to-end usage and DSCP Re-marking
> 5.2.  Aggregation of the NQB PHB with other Diffserv PHBs
> 6.  Non-Queue-Building PHB Requirements
> 7.  Impact on Higher Layer Protocols
> 8.  The NQB PHB and Tunnels
> 9.  Relationship to L4S
> 10. Configuration and Management
> 11. Example Use Cases
> etc
>
> §3 serves both as initial tutorial material and as the specification 
> of the sender behaviour that is normatively defined as a SHOULD at the 
> start of §5. Perhaps the text could be split in two, for these two 
> purposes, then most of this section could be moved to a new §5.1, 
> where it could be merged with the text in §5 prior to §5.1. Then §5 
> has a more logical structure for all the normative DSCP handling 
> (source nodes, interconnect nodes, aggregation nodes)
>
> 5.  DSCP Marking of NQB Traffic
> 5.1.  Non-Queue-Building Source Behavior
> 5.2   End-to-end usage and DSCP Re-marking
> 5.3.  Aggregation of the NQB PHB with other Diffserv PHBs
>
> Also, the para just before the current 5.1 ought really to be in the 
> PHB section (§6). It follows the incentives logic of the previous 
> paras, but I'm not sure whether that's the more important logical flow 
> (?).
>
> Also, I would have thought "8. The NQB PHB and Tunnels" is most 
> closely related to "Aggregation of the NQB PHB with other Diffserv 
> PHBs" and ought to be perhaps at §5.4. But I'm not sure whether §8 is 
> meant to be more about the PHB than the DSCP. The title says PHB, but 
> it seems to be more about the DSCP, and therefore rightly belongs in §5.
>
> The single para section "7.  Impact on Higher Layer Protocols" is 
> really just a specific point about reordering caused by traffic 
> protection. I don't see the need for a separate section.
>
> It also seems odd to have "Relationship to Diffserv" and "Relationship 
> to L4S" at opposite ends of the document. I suspect this is a result 
> of the original purpose being tutorial, and now it's turned into a spec.
>
> Pulling that all together, might look sthg like this:
>
> 1.  Introduction
> 2.  Requirements Language
> 3.  Context
> 3.1.  Non-Queue-Building Behavior
> 3.2.  Relationship to the Diffserv Architecture
> 3.3.  Relationship to L4S
> 4.  DSCP Marking of NQB Traffic
> 4.1.  Non-Queue-Building Sender Behavior
> 4.2   End-to-end usage and DSCP Re-marking
> 4.3.  Aggregation of the NQB PHB with other Diffserv PHBs
> 4.4.  The NQB DSCP and Tunnels
> 5.  Non-Queue-Building PHB Requirements
> 6. Configuration and Management
> 7. Example Use Cases
> etc
>
> [GW] I largely agree, and thanks for taking the time to think through 
> and suggest this.  I’ve kept the Impact on Higher Layer Protocols as a 
> separate section (I think it’s worth highlighting, if for no other 
> reason than it is a mandatory section per RFC2745 Guidelines), but 
> otherwise reorganized the material along these lines.
>

[BB] OK, glad that worked out. And I didn't realize that was a mandatory 
section - that explains it.


Bob


>
> HTH
> Regards
>
>
>
> Bob
>
> On 12/07/2021 22:30, Greg White wrote:
>
>     Hi all,
>
>     This update to the NQB draft includes the following changes:
>
>     - Recommends DSCPs 5 & 45 based on the analysis presented by Ana Custura at the last IETF
>
>     - Clarifies what the PHB provides, in the abstract & intro
>
>     - A more concrete description of NQB behavioral characteristics (including a rule-of-thumb value of 1 Mbps maximum burst rate)
>
>     - Moved the text on DSCP re-marking pathologies to an Appendix
>
>     - Further restructuring of the WiFi section
>
>     - Security section mentions default WiFi configuration
>
>     - Moved several references to Normative
>
>     - Multiple other editorial fixes
>
>     Many of these changes were spurred by off list comments received from David Black, Stuart Cheshire and Bob Briscoe (thanks!).
>
>     -Greg
>
>     On 7/12/21, 2:47 PM, "tsvwg on behalf [e-mail]  <mailto:[e-mail]>"<[e-mail] on behalf of [e-mail]>  <mailto:[e-mail][e-mail]>  wrote:
>
>          A New Internet-Draft is available from the on-line Internet-Drafts directories.
>
>          This draft is a work item of the Transport Area Working Group WG of the IETF.
>
>                  Title           : A Non-Queue-Building Per-Hop Behavior (NQB PHB) for Differentiated Services
>
>                  Authors         : Greg White
>
>                                    Thomas Fossati
>
>                 Filename        : draft-ietf-tsvwg-nqb-06.txt
>
>                 Pages           : 18
>
>                 Date            : 2021-07-12
>
>          Abstract:
>
>             This document specifies properties and characteristics of a Non-
>
>             Queue-Building Per-Hop Behavior (NQB PHB).  The purpose of this NQB
>
>             PHB is to provide a separate queue that enables smooth, low-data-
>
>             rate, application-limited traffic flows, which would ordinarily share
>
>             a queue with bursty and capacity-seeking traffic, to avoid the
>
>             latency, latency variation and loss caused by such traffic.  This PHB
>
>             is implemented without prioritization and without rate policing,
>
>             making it suitable for environments where the use of either these
>
>             features may be restricted.  The NQB PHB has been developed primarily
>
>             for use by access network segments, where queuing delays and queuing
>
>             loss caused by Queue-Building protocols are manifested, but its use
>
>             is not limited to such segments.  In particular, applications to
>
>             cable broadband links, Wi-Fi links, and mobile network radio and core
>
>             segments are discussed.  This document recommends a specific
>
>             Differentiated Services Code Point (DSCP) to identify Non-Queue-
>
>             Building flows.
>
>          The IETF datatracker status page for this draft is:
>
>          https://datatracker.ietf.org/doc/draft-ietf-tsvwg-nqb/  <https://datatracker.ietf.org/doc/draft-ietf-tsvwg-nqb/>
>
>          There is also an HTML version available at:
>
>          https://www.ietf.org/archive/id/draft-ietf-tsvwg-nqb-06.html  <https://www.ietf.org/archive/id/draft-ietf-tsvwg-nqb-06.html>
>
>          A diff from the previous version is available at:
>
>          https://www.ietf.org/rfcdiff?url2=draft-ietf-tsvwg-nqb-06  <https://www.ietf.org/rfcdiff?url2=draft-ietf-tsvwg-nqb-06>
>
>          Internet-Drafts are also available by anonymous FTP at:
>
>          ftp://ftp.ietf.org/internet-drafts/  <ftp://ftp.ietf.org/internet-drafts/>
>
>
>
> -- 
> ________________________________________________________________
> Bob Briscoehttp://bobbriscoe.net/  <http://bobbriscoe.net/>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/