Re: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-06.txt

[Greg White <[e-mail]>]

Bob,

Thank you very much for your review.  I am preparing a draft-07 that addresses many of these comments.  Please see some responses below.

-Greg


From: Bob Briscoe <[e-mail]>
Date: Friday, July 16, 2021 at 5:27 AM
To: Greg White <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-06.txt

Greg, list,

Here's my review of the nqb draft-06:
==Gap: Partial Deployment==

In §6, it says:
"In the case of an NQB flow that isn't marked as NQB and ends up in the QB queue, it would only impact its own quality of service, and so it seems to be of lesser concern."
This made me realize that the draft doesn't say more generally what will be the outcome if an NQB flow ends up in a QB queue, just simply because NQB is not supported at the bottleneck. It talks about this in specific cases (e.g. WiFi), but not in general.
[GW] Well there is also some related text in 5.2 and in the paragraph preceding 5.1.  But, I agree, this could be covered more directly.  I will add some text on that topic in 5.2 (Aggregation), which is where it seems most appropriate to me.

==SHOULDs or MUSTs?==

It might help to add some reasoning for why the draft allows implementers /not/ to comply with the 'SHOULDs'.
To help in this process, below I've tried to describe the exceptions to each SHOULD, so it could be written as "MUST do x..., except if y...".
·          5. DSCP Marking of NQB Traffic
o    "...SHOULD identify themselves to the network using a Diffserv Code Point (DSCP) of 45..."
§  Doesn't say what valid cases would be for not using 45 and how that would interwork with other networks
§  e.g. some other DSCP agreed locally with the access network would work fine, but some other DSCP to work with an unmanaged home WiFi might not work with the rest of the path.
[GW] Ok, I added “In networks where another (e.g. a local-use) codepoint is designated for NQB traffic, or where specialized PHBs are available that can meet specific application requirements (e.g. a guaranteed-latency path for voice traffic), it may be preferred to use another DSCP.”
o    "If the application's traffic exceeds more than a few packets per RTT, or exceeds approximately 1 Mbps on an instantaneous (inter-packet) basis, the application SHOULD NOT mark its traffic with the NQB DSCP."
§  If this were 'MUST NOT' it would be too prescriptive as the Internet scales. But perhaps it's better to say 'MUST NOT' and allow more flexibility in the quantification. For instance, instead of 'a few' and 'approximately 1 Mbps', explain how these numbers might scale as the Internet scales.
[GW] I understand your point.  But am going to hold off on making this change.  It isn’t immediately clear to me how to structure this as a MUST NOT.  Maybe we can discuss this in the WG.
o    "the application SHOULD instead implement a congestion control mechanism, for example as described in Section 3.1 of [RFC8085] or [I-D.ietf-tsvwg-ecn-l4s-id]."
§  I think it's best to say neither SHOULD nor MUST here, but instead refer to drafts that say normative things. Something like this: "the application has to instead implement a relevant congestion control mechanism, for example...". After all, an NQB RFC is not really entitled to say what non-NQB traffic should do.
[GW] I agree.
o    "...nodes that do not support the NQB PHB SHOULD treat NQB marked traffic the same as traffic marked "Default", and SHOULD preserve the NQB marking."
§  Again, this is putting requirements on non-NQB nodes, but in this case the requirement is not on the implementer, rather it's about the classifier config of the operator. So this ought to be phrased as operator guidance (e.g. RECOMMENDED).
§  And it needs to say what will happen in pre-existing networks, where the operator is not aware of either of these recommendations (see earlier point about partial deployment).
[GW] I believe that I’ve got this covered now in the Aggregation section.
·          5.1. End-to-end usage and DSCP Re-marking
o    "Absent an explicit agreement to the contrary, networks that support the NQB PHB SHOULD preserve a DSCP marking distinction between NQB traffic and Default traffic when forwarding via an interconnect from or to another network. To facilitate the default treatment of NQB traffic in backbones and core networks, networks SHOULD remap NQB traffic (DSCP 45) to DSCP 5 prior to interconnection, unless agreed otherwise between the interconnecting partners."
§  Both these SHOULDs could be MUSTs if the sentence started with "To support NQB, ..." because they both allow a locally agreed exception.
§  One reason for not complying would be lack of space for the marking (e.g. MPLS or Ethernet header space). So it would be worth saying that one way to preserve the distinction would be encapsulation, with the distinction only preserved in the inner.
o    "In order to enable interoperability with WiFi equipment, networks SHOULD remap NQB traffic (e.g. DSCP 5) to DSCP 45..."
§  Again, this could be made into a MUST by adding "To support NQB, ..." and allowing locally agreed exceptions.
[GW]  For the first one (preserving a marking distinction), I agree with you (and made it a MUST).  For the others that discuss specific DSCPs, I think we need to leave them as SHOULDs because (by definition) a PHB spec only “recommends” DSCPs and does not mandate them.

·          5.2. Aggregation of the NQB PHB with other Diffserv PHBs
o    "the NQB DSCP SHOULD be preserved"
§  As above, explain that you are talking about encapsulation to preserve the distinction by preserving the inner
§  (and it would be worth clarifying that the degradations discussed in the previous para are only within the aggregation network, not after traffic has been disaggregated again).
[GW] I’ll cover the first sub-bullet, but I’m not sure what more I could say on the second sub-bullet. The degradations (loss/latency/jitter), while they would be introduced in an aggregation node, clearly are felt end-to-end.  The statement around preserving NQB DSCP already mentions that it should be preserved “in order to limit the negative impact that such networks would have on end-to-end performance for NQB traffic.”  Maybe I’m misunderstanding what you mean?
·          6. Non-Queue-Building PHB Requirements
o    There are lots of SHOULDs here, without any reasoning given for why exceptions are allowed, or what sort of exceptions are not allowed. They all relate to preserving vs. enforcing incentives. So I think it could be said explicitly that where SHOULD rather than MUST is used in this section, it is because the details depend on the threat level in the deployment environment and whether incentives are working sufficiently.
[GW] I think your comment mainly applies to two of the SHOULDs (equivalent forwarding preference requirement and fair scheduler requirement), so I added it there.  The one that I’m debating is the “… SHOULD NOT be rate limited or rate policed separately from QB traffic …” requirement.  I don’t think this one is entirely related to incentives.  Even in a controlled environment where there is no threat of mismarking, I think this would still be the recommendation.  But, I’m having trouble justifying why it should be a SHOULD as opposed to a MUST.  My original rationale was that there is always a corner case, but I think I would be ok with it either way.

·          11. Example Use Cases
o    "To support the NQB PHB, the mobile network SHOULD be configured to give UEs a dedicated, low-latency, non-GBR, EPS bearer"
§  I assume the SHOULD is because it depends on how things are done locally. This could be handled by describing this non-normatively, as a way to satisfy the 'MUST' requirement in §6 to provide a separate NQB queue.
§  Similarly "SHOULD be routed" and "SHOULD NOT be routed" can be non-normative as ways to satisfy earlier requirements in current mobile networks.
o    "WiFi equipment SHOULD map the NQB codepoint 45 into a separate queue that shares an Access Category"
§  As for mobile, this seems to be a way to satisfy the earlier requirements for the /current/ way that WiFi works. These are examples, not normative requirements, aren't they?
§  Ditto for "SHOULD deploy a policing function" and "SHOULD map the recommended NQB code point 45 to UP_5"
o    However, I understand that you might want to draw implementers' attention to what they have to do, by using capitalized requirements.
[GW] Yes, I like these being called out as requirement statements.
==Flow Queuing in scope or not?==

I think the draft ought to specify that its scope is primarily shared queues, although there would be some merit in saying how NQB would work with an FQ. Per-flow queues provide the necessary isolation, but there's no need to make any of the other requirements apply to an FQ {however, I'm not sure - see Note 1}. Stated another way, there has been no standardization of how FQ systems handle DSCPs, but this draft would be the wrong place to introduce that subject for one specific DSCP.

{Note 1}: Except, the isolation provided by an FQ is not absolute. E.g. an attacker can spoof flow identifiers on well-known ports in the hope it will match a flow ID being used by a genuine NQB flow and create a long-standing queue (which might otherwise just exist as a series of transient queue instances for sparse packets). I doubt it would be possible to provide local per-queue traffic protection against such traffic (nothing to distinguish it). But this is where other protection techniques come in, like scrubbing farms that can identify such attack traffic closer to the spoofed sources.
[GW] Good suggestion.  I added a sentence in the Introduction.
==Partially described scenarios==
I didn't find the following scenarios easy to understand:
·         "Active Queue Management (AQM) mechanisms (such as PIE [RFC8033], DOCSIS-PIE [RFC8034], or CoDel [RFC8289]) can improve the quality of experience for latency sensitive applications, but there are practical limits to the amount of improvement that can be achieved without impacting the throughput of capacity-seeking applications, particularly when only a few of such flows are present."
o    For all the text after the 'but', I think there's some scenario in the authors' heads that isn't fully explained.
·         "Either approach comes with trade-offs: aggregating with Default/Elastic traffic could result in a degradation of loss/latency/jitter performance for NQB traffic, while aggregating with Real-Time risks creating an incentive for mismarking of non-compliant traffic as NQB."
o    The scenario isn't clear - is it talking about a bottleneck within the aggregated network, or after it?
o    The sentence about r-t incentives needs to explain how it reaches that conclusion, rather than expecting the reader to recreate the explanation.
[GW] Ok, I added some more detail for those.

·         "In the case of the pipe model, any DSCP manipulation (re-marking) of the outer header by intermediate nodes would be discarded at tunnel egress, potentially improving the possibility of achieving NQB treatment in subsequent nodes."
o    ...Or potentially making it worse? It all depends whether the inner marking is meaningful edge-to-edge. I presume the word 'potentially' alludes to this, but the sentence is rather opaque as it stands.
·         "...tunnel protocols that are sensitive to reordering can result in undesirable interactions..."
o    "undesirable interactions" is rather opaque. Does it mean within the tunnel? or after the egress?
[GW] The language comes from RFC2983, and the sentence begins with “As is discussed in RFC2983, …”  I think it is ok to assume a confused reader will know where to look to figure this one out.
·         "into a separate queue that shares an Access Category with default traffic "
o    The word 'shares' threw me at first - I think I've now understood; it must just mean 'it is within the same Access Category'. I was wondering how it could both be separate and be shared.
[GW] Ok, fixed.
·         "This document recommends the implementation of a traffic protection mechanism to achieve this goal, but recognizes that other options may be more desirable in certain situations."
o    I don't think 'other options' have been described in the doc (yet). So this seems opaque
[GW] Two other options are provided in the PHB Requirements section.

==Relationship with L4S (§9)==

Ought to say that a non-relationship to L4S is also fine.
And ought to explain that L4S is experimental, while NQB is stds track.

[GW] Done.
==Structure==
There were times when I couldn't really see a logical flow.

Here's the current ToC for reference:
 Table of Contents
1.  Introduction
2.  Requirements Language
3.  Non-Queue-Building Behavior
4.  The NQB PHB and its Relationship to the Diffserv Architecture
5.  DSCP Marking of NQB Traffic
5.1.  End-to-end usage and DSCP Re-marking
5.2.  Aggregation of the NQB PHB with other Diffserv PHBs
6.  Non-Queue-Building PHB Requirements
7.  Impact on Higher Layer Protocols
8.  The NQB PHB and Tunnels
9.  Relationship to L4S
10. Configuration and Management
11. Example Use Cases
etc

§3 serves both as initial tutorial material and as the specification of the sender behaviour that is normatively defined as a SHOULD at the start of §5. Perhaps the text could be split in two, for these two purposes, then most of this section could be moved to a new §5.1, where it could be merged with the text in §5 prior to §5.1. Then §5 has a more logical structure for all the normative DSCP handling (source nodes, interconnect nodes, aggregation nodes)

5.  DSCP Marking of NQB Traffic
5.1.  Non-Queue-Building Source Behavior
5.2   End-to-end usage and DSCP Re-marking
5.3.  Aggregation of the NQB PHB with other Diffserv PHBs

Also, the para just before the current 5.1 ought really to be in the PHB section (§6). It follows the incentives logic of the previous paras, but I'm not sure whether that's the more important logical flow (?).

Also, I would have thought "8. The NQB PHB and Tunnels" is most closely related to "Aggregation of the NQB PHB with other Diffserv PHBs" and ought to be perhaps at §5.4. But I'm not sure whether §8 is meant to be more about the PHB than the DSCP. The title says PHB, but it seems to be more about the DSCP, and therefore rightly belongs in §5.

The single para section "7.  Impact on Higher Layer Protocols" is really just a specific point about reordering caused by traffic protection. I don't see the need for a separate section.

It also seems odd to have "Relationship to Diffserv" and "Relationship to L4S" at opposite ends of the document. I suspect this is a result of the original purpose being tutorial, and now it's turned into a spec.

Pulling that all together, might look sthg like this:

1.  Introduction
2.  Requirements Language
3.  Context
3.1.  Non-Queue-Building Behavior
3.2.  Relationship to the Diffserv Architecture
3.3.  Relationship to L4S
4.  DSCP Marking of NQB Traffic
4.1.  Non-Queue-Building Sender Behavior
4.2   End-to-end usage and DSCP Re-marking
4.3.  Aggregation of the NQB PHB with other Diffserv PHBs
4.4.  The NQB DSCP and Tunnels
5.  Non-Queue-Building PHB Requirements
6. Configuration and Management
7. Example Use Cases
etc

[GW] I largely agree, and thanks for taking the time to think through and suggest this.  I’ve kept the Impact on Higher Layer Protocols as a separate section (I think it’s worth highlighting, if for no other reason than it is a mandatory section per RFC2745 Guidelines), but otherwise reorganized the material along these lines.



HTH
Regards



Bob

On 12/07/2021 22:30, Greg White wrote:

Hi all,



This update to the NQB draft includes the following changes:



- Recommends DSCPs 5 & 45 based on the analysis presented by Ana Custura at the last IETF

- Clarifies what the PHB provides, in the abstract & intro

- A more concrete description of NQB behavioral characteristics (including a rule-of-thumb value of 1 Mbps maximum burst rate)

- Moved the text on DSCP re-marking pathologies to an Appendix

- Further restructuring of the WiFi section

- Security section mentions default WiFi configuration

- Moved several references to Normative

- Multiple other editorial fixes



Many of these changes were spurred by off list comments received from David Black, Stuart Cheshire and Bob Briscoe (thanks!).



-Greg





On 7/12/21, 2:47 PM, "tsvwg on behalf of [e-mail]<mailto:[e-mail]>" <[e-mail] on behalf of [e-mail]><mailto:[e-mail][e-mail]> wrote:





    A New Internet-Draft is available from the on-line Internet-Drafts directories.

    This draft is a work item of the Transport Area Working Group WG of the IETF.



            Title           : A Non-Queue-Building Per-Hop Behavior (NQB PHB) for Differentiated Services

            Authors         : Greg White

                              Thomas Fossati

           Filename        : draft-ietf-tsvwg-nqb-06.txt

           Pages           : 18

           Date            : 2021-07-12



    Abstract:

       This document specifies properties and characteristics of a Non-

       Queue-Building Per-Hop Behavior (NQB PHB).  The purpose of this NQB

       PHB is to provide a separate queue that enables smooth, low-data-

       rate, application-limited traffic flows, which would ordinarily share

       a queue with bursty and capacity-seeking traffic, to avoid the

       latency, latency variation and loss caused by such traffic.  This PHB

       is implemented without prioritization and without rate policing,

       making it suitable for environments where the use of either these

       features may be restricted.  The NQB PHB has been developed primarily

       for use by access network segments, where queuing delays and queuing

       loss caused by Queue-Building protocols are manifested, but its use

       is not limited to such segments.  In particular, applications to

       cable broadband links, Wi-Fi links, and mobile network radio and core

       segments are discussed.  This document recommends a specific

       Differentiated Services Code Point (DSCP) to identify Non-Queue-

       Building flows.





    The IETF datatracker status page for this draft is:

    https://datatracker.ietf.org/doc/draft-ietf-tsvwg-nqb/



    There is also an HTML version available at:

    https://www.ietf.org/archive/id/draft-ietf-tsvwg-nqb-06.html



    A diff from the previous version is available at:

    https://www.ietf.org/rfcdiff?url2=draft-ietf-tsvwg-nqb-06





    Internet-Drafts are also available by anonymous FTP at:

    ftp://ftp.ietf.org/internet-drafts/









--

________________________________________________________________

Bob Briscoe                               http://bobbriscoe.net/