Re: [tsvwg] TSVWG: WG adoption of draft-white-tsvwg-nqb!

[Sebastian Moeller <[e-mail]>]

Hi Greg,

thanks for your response, comments prefixed with [SM]

> On Sep 10, 2019, at 23:28, Greg White <[e-mail]> wrote:
> 
> Hi Sebastian, 
> 
> See below marked [GW].
> 
> 
> 
> On 9/10/19, 1:23 PM, "Sebastian Moeller" <[e-mail]> wrote:
>    Hi Greg,
>> On Sep 10, 2019, at 20:29, Greg White <[e-mail]> wrote:
>> Yes, my focus (both from the perspective of the systems that I work on, and from the perspective of where I think NQB differentiation is most valuable) is on last mile and WiFi.       
>    	[SM] Except with the current very loose definition of which flows qualify for NQB marking, NQB seems to only merit AC_BE, which is already the default, so is NQB actually going to be valuable for wifi?
>> In terms of interconnection, there could be multiple ways to handle this.  My view is that the endpoint application has the best a priori knowledge of its sending behavior, so it is good to leverage that (and either validate it or not, depending on what is most appropriate for the situation).  
>    	[SM] not validating the behavior but giving special powers (like the originally proposed AC-VI/AC_VO on wifi) seems like a genuinely optimistic ideas. 
> 
> 
> [GW] What validation is done today for applications that mark their packets as AC_VO or AC_VI?  

	[SM] As far as I know almost no validation, but do to the traditional DSCP bleaching over the internet and the reluctance of most applications to twiddle with DSCP bits, most traffic will be in the AC_BE, but admittedly rather by chance than by design. In addition the IEEE's default DSCP to AC mappings are quite conservative (like mapping EF to AC_VI not AC_VO) making high volume traffic in the higher ACs relatively unlikely with such DSCPs that might survive internet passage. But with NQB's desirable approach to be transmitted e2e, that is going to change.

> I think that WMM is another one of those cases where the tragedy of the commons seems imminently possible, yet it largely hasn't occurred.  

	[SM] Mmmh, with 2.4GHz radios in a dense AP-concentration situation, we are close (and this is by beacon traffic alone).

> But, I hear you about being more clear about which flows are intended to be marked NQB. We certainly don't want to recommend something that creates the tragedy.

	[SM] Great, I think we can all agree on this ;)


> Perhaps we've left it too ambiguous in the draft, so that needs some review.  It was not the intent that capacity-seeking flows (even L4S) mark their packets as NQB.

	[SM] Great, maybe this can be made more explicit?

>  The target flows would be ones where the sender has a degree of confidence that it will not exceed the available capacity of the path.  

	[SM] With a variable bandwidth/rate path like wifi, the sender's confidence might not be the best measure? I would guess the hops operating the RF link would be in a better position to predict the path capacity in the immediate future?

> Also, I think validation of NQB behavior is a very useful tool, and it is in fact mandatory to implement for DOCSIS links.

	[SM] +1.

>  Other links (including WiFi) could implement it as well.  

	[SM] I fear I am less optimistic, and would argue that without this implemented wifi should be very careful to prioritize NQB (unless we can agree on NQB qualifying behavior that makes harm very unlikely).

> For example, WiFi APs or Stations could monitor queue depth or queue latency for the AC_VI (or AC_VO, whichever NQB maps to) queue and re-direct NQB traffic to AC_BE, either in a flow-aware way (probably preferred) or possibly even in a flow-blind way.  In other situations it may not be necessary (e.g. in controlled environments or on links that support FQ), or it may not be feasible.  

	[SM] All good ideas; the question to me is more in light of the lack of such mechanisms would it not be safer to default to AC_BE and only selectively elevate NQB traffic if the radio controller/AP is NQB aware? 
	I ask because I just did a flent rrul_CS8 (that is bi-directional greedy TCP traffic with one flow dscp-marked for each CS) test-run on my home net, and saw my macbook's sent CS6 and CS7 flows almost starve all other traffic, including CS6/CS7 marked traffic from the AP; I am not confident, that the hardware out there is ready for significant traffic volumes on AC_VO/AC_VI yet... (happy to share the flent data file on request).
	Now, I realize that NQB is not intended for bulk flows, but as long as the marking is not actively verified (and maybe even bleached on detected mis-marking) the intention does not matter that much, NQB-mismarking will give a considerable bandwidth and latency improvement. With access links getting faster and faster, wifi is becoming more and more a (transient) bottleneck making this issue sensitive.

> 
> 
> 
>> This would argue for networks to adopt a policy of not bleaching the NQB value on traffic arriving from interconnections, and utilizing queue protection if necessary to validate NQB behavior.   That said, there are other ways that network operators can identify NQB traffic without solely relying on endpoint marking (and DSCP preservation across interconnects).  I know some operators are investigating such techniques.      
>    	[SM] if the name has any relevance, one could treat each flow as non-queue-building until there is enough evidence to change that classification (like fq_codel treats all new flows as sparse initially until their observed bahavior merits a different classification).
> 
> 
> [GW] Exactly.  That is one mechanism that has been discussed.  A possible downside to that approach is that once the flow has demonstrated that it is QB (by building a queue in the NQB buffer) and subsequent packets are classified to the QB buffer, it is possible that the QB buffer was empty when those packets arrived, and they end up being transmitted before the tail packets in the NQB buffer, i.e. out of order delivery.  Flow startup for traditional TCP flows (even those that support RACK) can be sensitive to out of order delivery, so if this happens frequently it could be an issue.

	Interestingly it was a similar observation that made me initially think, DOCSIS queue protection would track each flow's packets in the queue so that on data-driven change of classification all queued packets of that flow could be moved from one queue to the other (I obviously did not consider the cost of tracking very deeply). Even though at that point it might be easier to directly go for stochastic flow queueing, as in that configuration re-ordering is not an issue and it should be much easier to verify compliance to required behavior if the flows are isolated... 

Best Regards
	Sebastian