Re: [tsvwg] Traffic protection as a hard requirement for NQB

[Sebastian Moeller <[e-mail]>]

Hi Bob,


> On Sep 10, 2019, at 23:25, Bob Briscoe <[e-mail]> wrote:
> 
> Sebastian,
> 
> On 06/09/2019 10:51, Sebastian Moeller wrote:
>> 
>>> On Sep 5, 2019, at 23:46, Bob Briscoe <[e-mail]> wrote:
>>> 
>>> Sebastian, (sorry, I mis-spelled your name previously), inline...
>>> 
>>> On 03/09/2019 10:14, Sebastian Moeller wrote:
>>>> Dear Bob,
>>>> 
>>>> allow me to chime in.
>>>> 
>>>>> On Sep 2, 2019, at 16:47, Bob Briscoe <[e-mail]> wrote:
>>>>> 
>>>>> David,
>>>>> 
>>>>> Thanks for your closing remarks on the NQB adoption call.
>>>>> You say your last point is open for discussion, so I will dive straight in to start that discussion.
>>>>> 
>>>>> On 30/08/2019 16:40, Black, David wrote:
>>>>>> 	• [snip]
>>>>>> 	• The criticisms on this list of the “queue protection” requirement in the draft are largely accurate.   The draft needs at least an Editor’s Note that this material will be revised, as while the DOCSIS mechanism is an example of how to do queue protection, it is not appropriate to require implementation of that mechanism.   A plausible plan that I have discussed with the authors is to write a set of functional/behavioral requirements for NQB “traffic protection” that can be satisfied by a “queue protection” mechanism such as the DOCSIS mechanism, or by a suitably configured FQ AQM implementation. [snip]
>>>>>>  In addition, related to item 2), my expectation (which is open to further discussion) that “traffic protection” will be a “MUST” requirement, perhaps with some well-specified exceptions (including explanations of why the exceptions are ok).   This is because “traffic protection” (e.g., “queue protection” or a suitably configured FQ AQM) appears to be necessary in general to keep queue-building traffic out of the NQB traffic aggregate, as allowing such traffic degrades the properties of the NQB PHB.
>>>>> I think we should be wary of making traffic protection a hard requirement at such an early stage in our knowledge of the NQB behaviour. I believe this is a case where the market, not the IETF, ought to decide whether protection is required.
>>>> 	[SM] The draft says "... it is worthwhile to note that the NQB designation and marking would be intended to convey verifiable traffic behavior, not needs or wants." in the light of this requirement it seems obvious that a hop willing to honor that DSCP should/must actually verify the traffic behavior, no? Requiring behavior to according to a set of requirements but not enforcing these requirements seems very very optimistic.
>>> [BB] Just because the behaviour is verifiable does not mean it always has to be verified.
>>> For instance, if someone claims a car is 17 years old, you might trust it is likely to be truthful because the speaker knows you could verify it. But it doesn't mean you have to actually look up the engine and chassis serial numbers to check the date of manufacture.
>> 	[SM] I fail to see how this is a fitting analogy to this issue at hand.
> [BB] It is a highly fitting analogy, so if you fail to see it, please try harder. My wife just read this, and said, "What's not to understand? That's an extremely relevant analogy."

	[SM] Okay, I tried to avoid this unnecessary detour, but since you seem to insist:
NQB, according to this draft is more than a simple claim of a property, it is rather a "promise" to behave in a specific way. In turn this draft proposes to give something special to NQB-marked flows as an exchange for that specific behavior. 

IMHO this is not exactly the same as me claiming my car to be 17 years old, which by itself will most likely have zero effect on anything, except maybe the verbosity of my story (and hence will not require verification). Now, let's see what happens if we actually make the age of the car relevant:
	Say, there is a specific tax exemption for cars older than 16 years (or a special tax increase, the direction does not really matter); and, say I want to buy the car you claim to be 17 years old, do you really expect me to not verify your claim? (And how could I actually not realize the age of the car, given that the car's papers will state the manufacturing date explicitly (at least here in Germany they do)? It sees to me that once the age of a car actually has actionable meaning verification will in all likelihood happen. Otherwise, I have a 17 year old car to sell to you...
	Now, unlike real-world persistent object like the car above, where we can do the verification post-hoc and drag each other to court if the given information was inaccurate yet relevant, in the NQB case there really only is a single entity which can verify the behavior, that is also is the same entity that gives special treatment to NQB marked flows. 
	Excuse me, if I still believe your car analogy (at least in its original form) to be ill fitting to the topic at hand. Sure I agree that there are verifiable facts that do not need to constantly be verified, but NQB-ness is not one of them.

> 
> In life, most trust is based on verifiability, but rarely actually verified.

	[SM] Sure, but for important things we do verify and we have post-hoc measures to ensure compliance (search for penal law). But that is not applicable to transient flows on the internet, you either enforce compliance on the spot or the horse has left the barn. There is a reason why "trust, but verify" and "speak softly, but carry a big stick" entered the political dialog, enforcement of relevant facts is much less rare than you might believe.

>> The NQB draft defines NQB-ness as " the NQB designation and marking would be intended to convey verifiable traffic behavior, not needs or wants." IMHO either get rid of this section or mandate the verification.
> No, it's important that it is verifiable. It's also important that it doesn't have to be verified, for reasons I've given in other emails.
> 
> For example, the case of a mobile uplink where:
> a) apps are verified before being allowed into the app-store

	[SM] That does not help, as the NQB draft so eloquently observed non-que-buildingness is only partly a function of each individual sender, it effectively depends on the aggregate ingress traffic (in the NQB class) versus the available egress rate, if the former exceeds the latter all NQB-classified flows are doing what they claimed they will not, namely building a queue. It is the bottleneck that sees the queue first and is in the best position to react, no amount of app-store pre-verification is going to change that, and that is under the optimistic assumption that app-store verification is perfect.... (Just see the examples of root kits distibuted vi)

> b) and the only place where the operator could apply verification prior to the bottleneck radio link would be on the UE

	[SM] UE = User Equipment? Which still runs a baseband process that the user has exactly zero influence on (this is a stronghold of the telco). Surely that baseband software should do the verification in the upstream direction? In addition the fact that for the upstream direction it is the CPE that should do the NQB handling is also true for DSL and DOCSIS. What is so special about mobile in this regard?

> 
> Here continual run-time verification is both complex (b) and unnecessary (a). So not appropriate to mandate it.

	[SM] Well, then simply do not propose a scheme that is complex in the first place? That it is necessary has been argued multiple times in this thread (Jonathan's NUR example, my DDOD as a service examples).

> 
> I'm afraid I cannot continue spending time on your responses if you just snip the text of my arguments that you haven't argued against and then continue to argue against the conclusion I've drawn from those arguments with no rationale, just IYHO assertions.

	[SM] ATM we are both exchanging assertions, and I would have thought that you would simply shut me up with real-world data, instead of strawman constructs and Gedankenexperimenten.

> 
>> 
>>> In the case of NQB, there is less need to verify behaviour because applications or users have no incentive to mismark - they would be worse off.
>> 	[SM] I still do not believe that to be true, and no amount of Gedankenexperimente is going to change that, as this is rather a data question.
>> As I wrote in another mail, any application that can gracefully deal with the shallow buffer of the NQB queue will be fine to misclassify itself into it even if its behaviors would be verifiably capacity-seeking. The only argument for misalignment of incentives to mismark is the shallowness of the buffer, once that is solved (by following Jonathan's BBR example) the argument that "for NQB flows, the NQB queue provides better performance (considering latency, loss and throughput) than the QB queue; and for QB flows, the QB queue provides better performance (considering latency, loss and throughput) than the NQB queue." is shown to be overly optimistic unless backed by queue protection.
> [BB] Well, let's see what Jonathan comes up with in response to my request to reconstruct his argument, given he had misunderstood the config.
> 
> But remember that, even if the incentives are not clear-cut, I am saying that it is still not appropriate for the IETF to mandate a protection mechanism.

	[SM] Yes, I get that, you are convinced a MUST for queue protection must be avoided, that much is clear, what is lacking to me is a clear and understandable rationale for that position that is more elaborate than "this might rule out a few desirable configurations" since I believe you underestimate the likely "undesirable configurations" that will come along from not enforcing what ever behavior a NQB flow needs to fulfill. Conceding that the incentives are not as clearly aligned as you initially posited makes this enforcement even more important IMHO. I think it was your argument that enforcement can be lax, since there is no incentive to game the NQB marking. Once you agree that the incentive situation is not that clear what else makes abuse improbable?

> 
>> 
>>> However, there is still the risk of accidents or malice.
>> 	[SM] With malice actually being a market place with its own economics. IMHO that factor alone should make queue protection mandatory, as this proposal will allow denial of service attacks much cheaper (no need to clogg the full pipe as long as one can drive the NQB queue into dropping mode often enough and that, in your example requires at most 31 full-MTU packets...)
> [BB] You seem to be treating me as if the economics of DDoS and traffic security is not my area of expertise.

	[SM] Well, then why did I have to bring this up? 

> 
> I assume you are aware that nearly every RFC is vulnerable to DDoS. So now please explain why each RFC does not mandate protection against DDoS?

	[SM] And why is your only defense for advocating for more attack surface to state that everybody else is not perfect either? But we are discussing the merits of this RFC draft and not all other RFCs so, lets's leave these out unless specifically related.

> 
> It's not an oversight. It's deliberate, because one doesn't have to implement protection at every point in a network and at all times (think of the example of app-store verification above).

	[SM] Deliberately, introducing a new cheaper DDOS attack vector that nicely isolates easy to disrupt traffic on a silver platter does not make it any better than doing it accidentally, rather the opposite. Requirng app-store verification basically proposes to make commercial entities the back-stop safety valve for a potentially overly optimistic RFC, is that the IETF's typical approach?.
 
> 
> 
>> 
>>> Nonetheless, the harm from any accidental or malicious NQB mismarking will be confined to other applications of the same customer {Note 1}.
>> 	[SM] The draft mentions access links, it does not restrict itself to access links though, so these need to be covered as well, no?
> I guess so. But that's not relevant to whether 'MUST protect' is necessary.
> 
> To prove MUST is necessary, you have to prove protection is necessary in every scenario.

	[SM] Do I? From https://www.ietf.org/rfc/rfc2119.txt:

"Imperatives of the type defined in this memo must be used with care and sparingly.  In particular, they MUST only be used where it is actually required for interoperation or to limit behavior which has potential for causing harm (e.g., limiting retransmisssions)  For example, they must not be used to try to impose a particular method on implementors where the method is not required for interoperability."

"limit behavior which has potential for causing harm" is a far cry from your interpretation that as long as there exists one scenario without "potential for causing harm" a MUST is not merited. 
@chairs, is my reading of rfc2119 in agreement with the official stance, or is Bob's interpretation correct?

I believe that your proposed level of absoluteness is not what engineering is about, the question is not the existence of scenarios that do or do not require behavioral verification, but rather the IETF will have to make a judgement call how much potential damage to risk for how much potential gain.
 
	And David Black already indicated he leans towards a MUST with a set of explicitly enumerated exceptions, so why not start enumerating those cases that should not be required to activate queue protection, instead of boxing yourself into a corner?

> That means you have to argue against the cases where I have shown it's /not/ necessary.

	[SM] No, I do not need to do this, as I do not believe that your assumption "necessary in every scenario" is in compliance with rfc2119's definition of the term.

> Unless you have done that, there's no point you introducing more cases where you think protection /is/ necessary.

	[SM] Well played, well played, to demonstrate why this line of dogmatic argumentation will lead to absurd conclusions, how about:
Not all motorcyclist are not involved in accidents and crashes and hence should not be required to wear a helmet because not all of them need that protection! 
Not all TCP's need congestion control, since as long as the window size restricts the max achievable bandwidth for the flow below the available bandwidth for the path no "collapse" will happen. 

I could go on, but I hope I made my point.

> 
>> But in the DOS case it is not an application of the customer that degrades the NQB queue but the attacker's.
> [BB] My sentence is about the application that is harmed, not the application doing the harm.

	[SM] Ah, sorry, I misunderstood.

>> 
>>> Given there's no incentive to mismark,
>> 	[SM] Which I am not convinced to be true yet.
>> 
>>> and the effect of accidents or malice are contained, an operator might decide that traffic protection is unnecessary - more cost than benefit.
>> 	[SM] Sure, but in that case our hypothetical operator will have no problem whatsoever to simply ignore the RFC's MUST/SHOULD language. But making it SHOULD means the operator can do so and actually crow about it to its end customers.
> [BB] The argument right from the start has been about SHOULD/MUST implement, not SHOULD/MUST enable. But it's similar. If it's SHOULD implement, the implementer can crow about it. But there's still a market for NQB without protection (e.g. for a mobile uplink).

	Erm, the last proposal was MUST implement SHOULD enable, at least that was the last proposal I remember. 
	Why does is a mobile uplink better off without queue protection than with? Really an honest question, as the owner of a mobile device I would be happy if the system would handle misbehaving flows gracefully and localized instead of spreading the pain to all flows sharing the queue. Look, I belong to those folks that consciously run fq AQMs on both upstream and downstream direction of my fixed link and that would happily do so for my mobile devices as well, and I would pay extra to be able to do so, so there might also be a market for full flow isolation on mobile uplinks as well, for all I know (maybe with a customer pool of one).

>> 
>>> The point about verifiability is merely that, if an operator does choose to protect flows from each other, an implementation can objectively determine which flows are out of compliance, because NQB behaviour is verifiable. Objective verifiability is neutral, which is important in jurisdictions with net neutrality legislation.
>> 	[SM] If a tree falls in a forrest and no one is there to witness it does it still make a sound? Only the bottleneck is guaranteed to have sufficient information to actually perform the verification, if that bottleneck does not do this it is game over;
> [BB] Yes.
> 
> That is still not an argument for 'MUST protect', which would force implementations of NQB to protect in order to claim RFC compliance, even if protection is not appropriate for the deployment scenario of the particular NQB implementation.

	[SM] And that is why it was proposed to go for MUST implement SHOULD enable instead of the stricter MUST implement & enable. So is your point that you see the implementation cost as too high so it will hinder deployment? In that case I would argue the fact that cable labs both mandates (implementation) of queue protection and champions the NQB codepoint, indicates that your sentiment is not shared universally, no?
	But back to the MUST, I see clearly a "potential for causing harm" from unprotected NQB queues, so IMHO a MUST protect is perfectly justified by rfc2119, no? (Note how interoperability is not the only clause that justifies a MUST or SHOULD, and how the "not required for interoperability" sentence is only given as an example).


> 
>> with the exception of the very narrow model of NQB-scheduling only on the access link in which case that observable behavior at the end-user will be strongly correlated with the behavior at the bottleneck (but even there it is not exactly clear which packets were dropped by the upstream NQB-scheduler and why).
> [BB] Access links are the primary and often expected to be the only point on a path through the Internet where NQB would be deployed.

	[SM] Sure, unless those cases when they are not... For wifi with its variable rates see my rationale for mapping NQB not higher than AC_BE for example. And for end-users more often then desired bottlenecks appear in transit/peering links between ISPs (which admittedly are not likely to special-case NQB anytime soon).

> See earlier response about how operators design their networks to ensure the bottleneck is usually at a single point that they can easily control.

	[SM] Some do and some do not, see e.g. the discussion about peering conflicts between eyeball-T1s and "pure" T1-ISPs, where it is implied that keeping capacity too low for peak traffic is used as a tool to sell direct access links for content providers affected by the rush-hour congestion, but I digress.
 
>> 
>>> 
>>> {Note 1}: The draft says NQB is targeted at access links. That implies NQB scheduling is between the classes of one customer and that a higher level in the network scheduling hierarchy isolates customers from each other.
>> 	[SM] Given your explanation, the draft maybe should clarify that it only targets access links, otherwise this argument is not very convincing.
> [BB] At present the draft concerns the bottleneck link of a path, which it says is frequently the access network. I think an NQB per-hop-behaviour could be applicable to any bottleneck, but the discussion of incentives and protection in the draft is mostly specific to cases where the bottleneck is an access network link (as in the three use-cases given).
> 
> So if you mean that the authors could say that the protection discussion applies primarily to access networks, I would agree.

	[SM] I would prefer to either fully restrict the scope of NQB to access links or also discuss the implications for other link types, yes.

Best Regards
	Sebastian

> [...]