Re: [tsvwg] WGLC comments draft-ietf-tsvwg-nqb-15

[Greg White <[e-mail]>]

Thanks for the in-depth review and detailed comments.  It is appreciated. 

Please see my responses below for the first 22 points, marked [GW]. I'll tackle the rest tomorrow.

-Greg


On 2/21/23, 3:50 PM, "tsvwg on behalf of gorry Fairhurst" <[e-mail] <mailto:[e-mail]> on behalf of [e-mail] <mailto:[e-mail]>> wrote:




I would like to add my own comments to this WGLC for the NQB ID. These 
are added as an individual during the WGLC.


To me this draft seems like it addresses a useful specification, 
although I think it still requires work.


I hope the comments/queries below help can be discussed or addressed 
from the WGLC,


Best wishes,


Gorry Fairhurst


=====


1. The definition of “low-rate data-rate, application-limited traffic 
flows” is still problematic for me. The text later describes this (in 
more than one place) as /relatively low data rate applications/ … which 
I am less sure about what that means? how relative is this? Can we be 
more concrete?

[GW] Section 4.1 describes an upper bound using a rate equation, where the rate R is described as 'about 1 percent of "typical" network path capacity' and further provides that today that works out to 1 Mbps.  Would it help if we provided that definition earlier on in the document?  Or, it your concern that even this definition is not concrete enough?

I was particularly surprised by this: “Current examples include many 
online games” ... which clearly covers a wide range of rates.

[GW] As has been discussed in a separate thread, the intent of that example was to refer to the extremely common multiplayer online PC/console type games, which may in fact cover a wide range of rates, but that range is roughly 1 kbps to 300 kbps.  Cloud-rendered game services are definitely different, and can have downstream data rates measured in the 10s of Mbps.   I have clarified this in https://github.com/gwhiteCL/NQBdraft/commit/8bb01560897f59383032f7d5fb82907414269522 let me know if you still find that problematic. 

It seems that it argues that the traffic carried by the PHB consumes a 
limited proportion of the available capacity, and can see how if this 
is the case and the flow/flows contributing the traffic are paced, then 
this might be acceptable for some people deploying diffserv. The 
traffic I would have in mind would be limited to a few packets over an 
RTT, or an average rate less than, say, 1 packet/second - such things as 
DNS, upper protocol control messages, infrequent exchanges, etc. Even 
when aggregated up as many many flows, such traffic seems relatively 
benign compared to the rate of modern Internet paths in general (aka >1% 
of typical path capacity). Yet it mentions, video conferencing, or 
Broadcast Video, which although rate-limited and paced are much more 
than the rates I would have expected, and can aggregate to appreciable 
traffic without traffic conditioning.

[GW] Yes, it was a mistake to mention the option of aggregating CS3 (Broadcast Video) with NQB-marked traffic in the NQB PHB.  Ruediger pointed this out as well.  I've eliminated it in https://github.com/gwhiteCL/NQBdraft/commit/106ec7ac09ea334327fc90225ba82b01ff4cc3f0.
Video conferencing was not explicitly mentioned in draft-15.  Could you point out the text that your comment refers to?

I have a feeling of discomfort that stems from this PHB not requiring a 
conditioning method, and yet potentially relating to a sizeable 
proportion of the traffic on some portions of the Internet. While 
operators can use IETF specs and configure what they need for their 
service, I would like to think the IETF as a whole is careful about what 
is explicitly endorses.

[GW] This is a shallow-buffered Best Effort service, not a guaranteed service. The draft does recommend a "conditioning method", although it describes it in terms of identifying flows that are inconsistent with the sender requirements, as opposed to handling cases where the aggregate of compliant flows exceeds available capacity.  In Best Effort services, overloads can happen.  Whether the impact of overload (typically packet loss and/or latency) is only felt by a subset of the users/flows or is felt by all of them could be different in different implementations. I think this is ok.  

====


2. In understand the goal of “This PHB is implemented without 
prioritization” and that is not in dispute, but the side effects of the 
currently discussed DSCP value is that it could be assigned by accident 
to a lower layer that treats traffic with this DSCP as having this 
priority. I expect that somehow needs extra diligence with the way we 
define the traffic profile.

[GW] That topic is the subject of section 4.3.1, which provides recommendations on how to prevent or mitigate problems that could otherwise result.  Do you have any thoughts on what changes there would help eliminate this concern?

===


3. I understand the goal “can be implemented without rate policing”, and 
I would hope that is the case for individual flows. None-the-less the 
conditioning of traffic matching this PHB is likely to be important for 
many deployments, and the later section do talk about how to condition 
the traffic, so I was unsure what was intended.

[GW] Ok, I was trying to keep the abstract short, but still describe the important differences from other PHBs (primarily EF).  The intent was to say that, in a node that supports the PHB, the aggregate of NQB traffic is not limited to a subset of the link capacity that it shares with Default (i.e. if there happens to be no Default traffic, NQB could consume the entire capacity that the two PHBs share).  I added the "can be" in that phrase since we've added discussion in 4.3.1 about certain cases (interconnection with unmanaged networks that don't support the PHB) where it could be needed.  Strictly speaking, I think the sentence was correct without the "can be", since it is referring to implementations of the PHB (where rate policing is not defined) as opposed to interoperability with networks that don't support the PHB.  Maybe the issue is that the abstract doesn't explicitly state that this PHB is paired with the Default PHB and that the two of them could be rate policed (or rate shaped) together.  Do you think that would solve it?

====


4. I agree with the desire that they are “such that they are highly 
unlikely to exceed the available capacity of the network path between 
source and sink.”, but reading this left me feel really uncomfortable 
with respect to operators and networks offering less than 100 Mbps - I 
am not sure I like the idea of the IETF projecting a minimal service 
rate, so I'd encourage much more care in how this is all framed.

[GW] The intent wasn't to project a minimal service rate in general for the Internet.  But, the ability to deliver on the promise of the NQB PHB does depend on the relative rates between the applications sending NQB-marked traffic (in aggregate on a link) and the link rate where the PHB is supported.  We could enable the benefits of the NQB PHB to extend to more networks by setting this application rate guidance lower, but that would then limit the applications that could take advantage of it.  This is a judgement call, but 1% of "typical" capacity (1 Mbps today) to me seems like a reasonable place to draw the line and provides some headroom in cases where the capacity is less than typical.  I think setting it to 500 kbps would be ok, but I don't think we would want to go a lot lower than that.  

This also occurs later, "“In today's network, where access network data 
rates are typically on the order of 100 Mbps, this implies 1 Mbps as an 
upper limit. “ - typically yes in many parts of the world for many this 
is not the case still.

[GW] And this is covered in Section 5.3.  Just put NQB traffic and Default traffic into the same queue (as is the case for those networks today and would be the case if the NQB PHB never existed).  If L4S isn't supported in the network, then arguably this isn't needed since I think that the outcome of splitting the traffic into two equal priority queues (one shallow buffered and one deep buffered) even on low rate links isn't necessarily worse than using a single queue.  In any case, I'll try to find a place in (or before) this section to plant the seed that this is covered later. 


====


5. I think the statement below does not represent current IETF consensus:


“ While this architecture is powerful and flexible enough to be
configured to meet the performance requirements of a variety of
applications and traffic categories, or to achieve differentiated
service offerings, it has proven problematic to enable its use for
these purposes end-to-end across the Internet.”
This is currently not appropriate to a standards-track document, unless 
we explicitly seek to gain that consensus, and I don’t particularly see 
why that paragraph is needed, or suggest something of the flavour:
“ While this architecture is powerful and flexible enough to be
configured to meet the performance requirements of a variety of
applications and traffic categories, or to achieve differentiated
service offerings, at the time of writing there is no specification 
to enable its use for
these purposes end-to-end across the Internet.”

[GW] Ok.  But, is it correct to say that the issue is the lack of a specification? I wouldn't agree that this is the problem.  Maybe we could simply say "it has not been used for these purposes end-to-end across the Internet."

====
6. I am not sure this really matches how I see the diffserv 
architecture. Please check the wording:
“They also
significantly simplify access control and admission control
functions, reducing them to simple verification of behavior.”
- If it is indeed intended to talk about admission control, then please 
separate this from the para on diffserv and use language that maps to 
that network function.

[GW] I'm not sure I fully understand what you are getting at. The "They" in that sentence was referring to "These attributes" of the NQB PHB in the previous sentence. Is that how you interpreted it? 


===
7. I query what is intended by the word policing, or would like more 
description for what is exactly intended, if this is different to 
traffic conditioning?
“less stringent policing than they would with either codepoint alone”

[GW] Hmm, that statement was provided by Bob, and I had interpreted it to mean the traffic protection function (in the context of PHB implementations) and re-marking/traffic policing (in the context of section 4.3.1). 
https://mailarchive.ietf.org/arch/msg/tsvwg/ohoTqN2olPG_kyML-9WsbIclkwM/
I see that Ruediger asked a similar question:
https://mailarchive.ietf.org/arch/msg/tsvwg/Q97SEK2sLCCSdlX4H9eQwfleZQk/
I think the correct statement should be: "Packets marked with both codepoints SHOULD be treated the same as packets marked with either codepoint alone by the traffic protection function and by any re-marking/traffic policing function designed to protect unmanaged networks (as described in Section 4.3.1)."  Does this leave any gaps?

===
8. “In addition, these applications send their traffic in a smooth (i.e. 
paced) manner,”
- I know smooth pacing can be true in some cases , but is it necessarily 
true. If I take a Zone transfer or download a large DNS record is this 
paced? What are the implications here?

[GW] Well, I think we need to draw the line somewhere.  Based on the current text, a burst greater than 1500 bytes above "R" would not be compliant with the sender requirements.  Do you think 1500 bytes isn't the right number?

===


9. “Note that, while such flows ordinarily don't implement a 
traditional congestion control mechanism, they”


- The word ordinarily made me worried about how this could be read. 
Could you perhaps consider turning this around and avoid stating the 
norm. (and avoid the "flow" implementing something) e.g. When such a 
flow is generated by a sender that does not implement a traditional 
congestion control mechanism...".

[GW] Sure. That is an improvement.   Actually, I'm still a bit uneasy with how that is presented. As I mentioned in 
https://mailarchive.ietf.org/arch/msg/tsvwg/i4sz-cMb5MOLBCzDv47RFB0cn7g/
I don't see why we wouldn't want to encourage ALL NQB-marked applications to also implement an L4S-compliant congestion controller.  Later in this section we recommend that NQB non-compliant applications consider implementing L4S, but why not low-data rate ones too?

===


10. Consider: “To be clear, the description of NQB-marked flows in this 
document


should not be interpreted as suggesting that such flows are in any
way exempt from this responsibility.”
/should not/is not/ ?

[GW] Yes.


===


11. At points there are long paras that span multiple topics, please 
break these where it is possible. e.g. Could we break the para here to 
separate the local-use and recommended approaches:


/In networks where another (e.g., a local-use) codepoint/

[GW] Ok.

====
12. This expectation for applications might be OK, but it does not help 
an operator that has to support a lower rate link
/If the application's traffic exceeds the rate equation provided in
the first paragraph of this section, the application SHOULD NOT mark
its traffic with the NQB DSCP.


/


There is text on this later, that I think really ought to be called out 
earlier in the document.

[GW] Ok, I'll find a spot to mention this earlier. 

====


13.
/At the time
of writing, it is believed that 1 Mbps is a reasonable upper bound on
instantaneous traffic rate for an NQB-marked application, but this
value is of course subject to the context in which the application is
expected to be deployed./
- This is an IETF PS, and needs to have IETF consensus: I’ve argued this 
is high in the past fro a PS designed for the Internet as a whole, and I 
will argue again the same. Although I can see that in some regions this 
value would be legitimate, I fear in other places it would not. I think 
this is something I where I would like to see a strong consensus.

[GW] Per my comment above I think if we reduce it too much it won't provide a benefit for very many applications.  Do you have a number in mind?

===


14.
I have two queries form this phrase:
/happens to exceed
the available path capacity (even on an instantaneous basis) runs the
risk of being subjected to a traffic protection algorithm/
- 1: Why exceed capacity? rather than its share of the capacity?
- 2: Where is the term traffic protection defined and explained in 
section 4.1 ? (can we try to have the term defined before it is used?)

[GW] How about: "An application that marks its traffic as NQB runs the risk of being subjected to a traffic protection algorithm (see Section 5.2) if it contributes to the formation of a queue in a node that supports the PHB. This could result in the excess traffic being discarded or queued separately as default traffic (and thus potentially delivered out of order)."


====
15. “The sender requirements outlined in this section are all related to
observable attributes of the packet stream, which makes it possible
for network elements (including nodes implementing the PHB) to
monitor for inappropriate usage of the DSCP, and re-mark traffic that
does not comply.
“


- how does this work I can see how a specific operator might know 
specific details, and that might even include the RTT between some 
source/destination pairs. But I don’t know how an operator would 
reasonably condition or police Internet traffic in terms of RTT. Maybe 
you understand better? It seems to me that an application developer 
(unless for a closed platform), would likely not know RTTs or what 
other traffic will be aggregated, so advice here to limit the type of 
traffic admitted seems hard to grasp.

[GW] I actually don't think there are any cases where the RTT comes into play. The current sender requirements today are:  ((at most a few pkts per RTT) || (no more than 1 Mbps)) && (less than (1 Mbps)*T+1500B over any interval T).  Maybe we should simplify the requirements, and remove the mention of RTT?

===


16.
/but is recommended nonetheless in order to /
… Ought this to be a RFC2119 keyword? it might be or it might not? If 
this PS makes a recommendation it needs to be RFC2119 compliant.
… It would be really nice to remove the /in order/ term to avoid an 
ambiguity.

[GW] The RFC2119 recommendation is in the previous paragraph, this sentence is simply explaining why we made that recommendation.  Could you take a look again and see if you think that is unclear?  
[GW] I will delete the /in order/.


===


17. controlled environments


I would also like to see a para break between the IETF-specified domains 
and any local-use description of controlled environments:


/An alternative, in controlled environments/


- perhaps advice on controlled environments would be better placed in a 
separate subsection?

[GW] How big a concern is this for you?  It does seem like it would make things difficult to explain.  I'd probably prefer deleting any mention of controlled environments rather than trying to create a new subsection. 

===


18. Please explain:


/would be to aggregate NQB-marked
traffic with real-time, latency sensitive traffic./


- How would aggregation be performed … would this traffic with this set 
of DSCP values be forwarded as the same treatment aggregate?

[GW] That is what I had in mind, but this sentence has pretty low overall value in the draft.  If it is problematic, I'd probably delete it rather than spending a lot of time wordsmithing it. 

===


19. The following text is still to me is problematic for a PS.
/Similarly,
networks and nodes that aggregate service classes as discussed in
[RFC5127] and [RFC8100] might not be able to provide a PDB/PHB that
meets the requirements of this document./
- We can define in this PS, what is to be done to comply with the PS, we 
should not provide text that seems to hint that complying with the spec 
allows non-compliance.
- This text needs to be in a separate section to clearly differentiate 
it from the other treatment

[GW] I see your point.  How separate does it need to be?  An appendix?
===


20. All operators seems like a strong statement:


/ is recognized by all operators/ - not quite, all operators configuring 
a diffserv domain? something else?

[GW] How about we obfuscate that sentence a little:  "is recognized and mapped across network boundaries accordingly." ?

===


21. Is this an RFC2119 requirement:
/ To
ensure reliable end-to-end NQB PHB treatment, the appropriate NQB
DSCP should be restored when forwarding to another network./
-Do you have an RFC that you could quote to support this?

[GW] It seems like this is something that RFC2475 would say.  It's not jumping out at me on a quick scan though.  I'll keep looking.
  
===


22. I objected to the following term in my last review, and I'm going to 
push-back again on the use of "management" to characterise these 
networks, I don't think the text is anything to do with management of 
network elements:


/Unmanaged Networks/
I also do not find this term in RFC2475, presumably because that's not 
the appropriate term?


This term needs explained or another term used.


Later it also seems to link residential as unmanaged? This seems a 
mix-up of terms again.

[GW] Sorry if I missed this on an earlier review.  I agree this could be improved.  I'll take a stab at it. 



====
23. I can’t understand what this means in terms of diffserv:
/there may be cases where a
network operator is delivering traffic into a network outside of
their control,/
- its this any operator? one that uses diffserv? what does control mean 
in this context?
====
24. Also, what does this mean:
/where there is no knowledge of the traffic management
capabilities of the downstream domain/
- what is traffic management? Which part of the spec do you mean?
====
25. /In such cases, the
network operator should presume that the existing network equipment
in the downstream domain does not support the NQB PHB and might
instead prioritize traffic marked with the NQB DSCP./
- Explain the should here clearly.
- If this is a recommendation what exactly is the requirement?


====


26. What does this actually mean, please clarify:


/networks SHOULD ensure NQB traffic is marked DSCP
45 prior to a customer access link, subject to the safeguards
described below and in that section./
- Does it mean remark the DSCP in all traffic this way (i.e. are you 
calling for a multi-field classifier and DPI or something else?
- Could this be only to remark traffic that is marked with a DSCP that 
is associated with the NQB PHB treatment (and potentially carried using 
another DSCP)? I don't know from the text.
===
27. /In these cases,
the network operator SHOULD take precautions to prevent issues that
could be caused by excessive NQB traffic and/or traffic mismarked as
NQB./
-I don’t see this as sufficiently clear for a standards-track 
recommedation. Much more clarity is needed and the recommendations 
clarified within the sentence that contains the keyword!
===
28. Traffic protection


To me at least there is a topic in 4.3.1 on traffic protection that 
seems to need to be in a separate section so it can be found.


===


29. I think the polid=cing and shaping text is still unclear:
What does / policing function or a rate shaping/ actually do to the 
traffic that is excess - is this dropped, ECN-marked, queued, DSCP 
re-marked, or something else? IT isn’t clear to me what the 
recommendation is.


===


30. Please explain why is there approximately 100 ms of burst tolerance? 
- what is the reason, and is that independent of the rate of the link?


===


31. I just do nor understand this clause:


/support for at least 5 simultaneous NQB streams,/ - what is an NQB 
stream? and why 5?


===


32. There is a sentence that says:
/Such a function SHOULD be implemented
in an objective and verifiable manner, basing its decisions upon the
behavior of the flow rather than on application-layer constructs.


/ - to me this sentence is hard to understand. Please ensure the 
sentence with RFC2119 language is self-contained and clearly verifiable.


===


33. The following clasue doesn’t seem verifiable, please say what “fail 
gracefully” means…
/The traffic
protection function MUST be designed to fail gracefully in the case
that the flow state is exhausted./


===


34. Editorial note, this note needs to be at the front of the document 
please, so that we do not miss this request.


[NOTE (to
be removed by RFC-Editor): This ISE submission draft is approved for
publication as an RFC, the NQB draft should be held for publication
until the queue protection RFC can be referenced.]


===


35. Please explain why this is not necessary, rather than stating this:
/ There are some situations where such a function is potentially not
necessary. For example, a network element designed for use in
controlled environments (e.g., enterprise LAN).
/
- Is it better that this statement is said in the controlled 
environments section,
- or at least this section ought to be referenced there, it can not be 
expected that
- readers will try to read the whole document to extract guidance related to


- specific networks.


====


36. Section 5.3 says it provides
/Guidance for Very Low-Rate Links/
- I am doubtful that people providing 10 Mbps service would regard their


- service as


- I'll reiterate that I really felt uncomfortable with labeling this as 
/very low rate/. The title here is clearly not likely to attract the 
reader who needs to pay a lot of attenetion to this section. In some 
places in the world 10 Mbps would still be regarded as high speed!!


====
37. Section 5.3 says:
/To limit the consequences of this scenario, operators of such
networks SHOULD utilize a traffic protection function that is more
tolerant of burstiness (i.e., a temporary queue). Alternatively,
operators of such networks MAY choose to disable NQB support (and
thus aggregate NQB-marked traffic with Default traffic) on these low-
speed links. For links that are less than ten percent of "typical"
path rates, it is RECOMMENDED that NQB support be disabled and for
NQB-marked traffic to thus be carried using the default PHB./


Is there a standards-track specification for this function?
Is there a clear indication of what this is important?
What does disable mean here .. does that mean drop? remark? or
simply use the default PHB?
What is the impact when these networks are lower speed and use the now
deprecated IP precedence QoS model?


====


38.


/As discussed in [RFC8325], most existing WiFi
implementations use a default DSCP to User Priority mapping that
utilizes the most significant three bits of the Diffserv Field to
select "User Priority" which is then mapped to the four WMM Access
Categories. /
- I am unsure the assertion that /most/ do anything is a correct thing 
to state in this PS.


- It is quite correct though to state /at the time of writing some 
equipment/ does this, or some other forms of words.


====


39.




/All known WiFi gear has hardware support (albeit generally not
exposed for user control) for adjusting the EDCA parameters in
order to meet the equal priority recommendation.
/
- This might be so, but can this be said without making a statement 
about what equipment does.
Perhaps:/WiFi gear typically has hardware support (albeit generally not
exposed for user control) for adjusting the EDCA parameters in
order to meet the equal priority recommendation./ ???


====


40.The next sentence is not at all clear to me:
/If left unchanged, the prioritization of Video Access Category
traffic (particularly in the case of traffic originating outside of
the WiFi network as mentioned above) could erode the principle of
alignment of incentives./
- I suspect it is about the default mapping discussed earlier? then what 
is alignment of incentives? - please can you rewrite this?


====


41.


/ The NQB signal (DSCP) is not integrity protected and could be changed
by an on-path attacker. / … I read this several times, but are you 
actually just saying that
the design of Diffserv permits, and in some cases, expects DSCPs to 
change as packets
are forwarded by diffserv nodes? - Or are actually saying something 
different?


====
Examples of NiTs I saw in the current draft:


* Please be clear whether than value 45 is decimal, hex or something 
else? :-)
* Should /application limited/ be hyphenated in all uses?
* /as NQB but happens to / - insert comma before but?
* /re-marking NQB traffic as Default/re-marking NQB traffic with the 
Default DSCP/
* /the NQB marking/the NQB DSCP value/
* / NQB-marked traffic/traffic marked with the NQB DSCP/
* /the same as Default/the same as traffic with a Default DSCP/
* /treating NQB-marked traffic as Default/forwarding packets with the 
NQB DSCP using the Default treatment/
* /performance but is/ - insert comma before but?
* /mismarking of traffic/ This seems wrong in diffserv we should talk 
about /classifying the traffic/ or refer to the DSCP value carried?
* /the value 45/the DSCP value 45/
* /with DSCPs 40-47/ explain the base - decimal?
* /Such equipment is to support the ability to configure the 
re-marking/It is RECOMMENDED that this equipment supports the ability 
to configure the re-marking/ since this is about compliance rather than 
protocol action? - it would be super helpful if the /equipment/ was 
clearly identified by a term that was agreed.
* /This characteristic provides one disincentive for mismarking of 
traffic./This characteristic provides one disincentive for incorrectly 
setting the NQB DSCP for this traffic./
* /codepoint 45/DSCP 45/
* /So, the choice of 45/So, the choice of using DSCP 45/
* /Recommended NQB codepoint 45/The NQB DSCP (45) / ?