Re: [tsvwg] Follow-up to your DSCP and ECN codepoint comments at tsvwg interim

[Bob Briscoe <[e-mail]>]

Jonathan,

[Gorry asked us to hold this thread for a week or so - I've now asked 
him to allow it to proceed]
Inline...

On 12/03/2020 13:07, Jonathan Morton wrote:
>> On 12 Mar, 2020, at 2:19 pm, Gorry Fairhurst <[e-mail]> wrote:
>>> On 12/03/2020 11:45, Jonathan Morton wrote:
>>>> On 12 Mar, 2020, at 10:18 am, Lars Eggert <[e-mail]> wrote:
>>>>> [LE] Is it at worst a no-op, or can it degrade performance for 
>>>>> non-participating flows? 
>>>> [JM] Non-SCE flows see only an RFC-3168 compliant AQM on the
>>>>   path, and ignore the SCE signals.  SCE flows at worst have a tendency
>>>> to defer to existing traffic - the opposite to what you fear - but with a
>>>>   well-designed bottleneck qdisc, this effect is limited or eliminated.
>>>> The SCE proposal presents several such qdiscs as examples.
[BB] As SCE AQMs are deployed, they SCE-mark any ECT0 IP header to ECT1, 
whether or not the source is taking part in the SCE experiment.

Currently nearly all packets on connections to & from Apple devices use 
ECT0. So an SCE AQM would routinely alter a high proportion of current 
Apple-customer-traffic to ECT1.

That may not be a problem. But it is not 'safety first'. If there are 
ECT1 black holes in the Internet, they will currently be benign to 
traffic traversing them, because no-one is using ECT1. However, if these 
black-holes do exist, deploying an SCE AQM will open up these holes 
beneath a potentially high proportion of any Apple-customer's traffic 
traversing them.

I have checked the four main large-scale ECN measurement studies (2011, 
2013, 2015, 2018) and none reported any experiments testing for ECT1 
black holes. Have any been conducted to your knowledge?


In the L4S approach, the source opts into the experiment by setting 
ECT1, so any L4S source that hits a black hole can opt back out of the 
experiment - for that path.

>>> [GF] ... Or rewrite ECT(1), as I understand for some existing 
>>> tunnels/encaps. I recall from the early L4S discussion that this 
>>> consideration was also be important, and hence we charted work to 
>>> try and improve ECN-marking in the IP layers. 
> [JM] I think we will need to have a focused discussion about tunnels at some point.  For the moment, I'll just note that if the SCE signal is erased (on either the forward or feedback path), then the flow will revert to conventional RFC-3168 behaviour, which is conservatively safe.
>
>>> [JM] SCE enabled middleboxes are easy to detect, by passing ECT(0) traffic through it and seeing if ECT(1) marks appear alongside (or before) CE marks. If CE marks or an AQM-like (or tail-drop-like) pattern of drops appears but ECT(1) does not, it's not an SCE enabled middlebox.  This is probably sufficient to reclaim the spare ECN codepoint.

[BB] Bear in mind that multiple parts need to deploy (sender, AQM, 
receiver) before there's any benefit.

Whether ECT1 is assigned to SCE or L4S, I think we should be honest with 
ourselves and plan for the codepoint not being reclaimable for at least 
20 years, even if deployment of all three parts never occurs (see nonce 
data point below). If there is widespread deployment but the experiment 
still fails despite this (e.g. an insurmountable technical flaw), I 
suspect at least double that (40 years) before use of the codepoint 
tails off enough to be reclaimed.

We should not understate the stakes.

>> [GF] Probably worth writing down. I've found it hard to test for ECN-marking on operational paths, but insight in this is always going to be welcome.
> [JM] Granted, there is the corner case where the middlebox in question is not a bottleneck on the path under test - so SCE or ECN functionality may be present but not visible.  That would appear to be the chief difficulty with achieving full probe coverage.

[BB] Even if the SCE AQM was the bottleneck, the measurement would have 
to be done just after the SCE AQM, because:

  * If measured at the receiver, absence of ECT(1) marks might still
    occur if an SCE AQM introduces ECT(1), but a subsequent tunnel decap
    eats them all up again.
  * If measured at the sender, absence of ECT(1) feedback could be due
    to the above tunnel problem, or due to the absence of a receiver (at
    least in TCP) that feeds back ECT(1).

    That's because an SCE TCP sender never knows whether it's paired up
    with a receiver that has ECT(1) feedback logic or not.
    (An SCE sender could test the receiver by sending fake ECT(1)
    markings to see if they cause feedback. But then that in turn would
    make in-network measurement think there was an SCE AQM when there
    wasn't.)


> But I think enough confidence could be obtained by such tests to permit reclaiming the codepoint for another experiment in a reasonable timeframe.  We can try paying more rigorous attention to this question if the WG deems it necessary.

[BB]  "Reasonable timeframe" needs to be understood in relation to the 
following data point. AFAICT there was one implementation of the ECN 
nonce on a private testbed, and possibly one instance deployed somewhere 
in Norway. But it still took 16 years of absence of measurements before 
we could reclaim the codepoint.


That said, when assigning a scarce codepoint in the IP header, designing 
an experiment for reversal is far less important than minimizing the 
chance the experiment will need to be reversed. That means a need for 
pre-existing strong (i.e. business, not just techie) interest in 
deploying /all/ the necessary parts of the experiment. That was what the 
ECN nonce failed on. With industry backing, even if there are teething 
problems, there will be energy (and money) to fix them.

I know the IETF doesn't normally look beyond 'techie', but it is capable 
of doing so.

Cheers



Bob

>
>> This was to be an important topic of the face-to-face meeting in Vancouver, where we have been planning to allow time for different viewpoints to be aired and the consensus of the group to be sought. We still plan to do that, and are considering how best to proceed.
>>
>> Comments in-line to show the expected direction of travel, but please do avoid growing this discussion thread for a week or so, until we have a plan to conclude it (see end of email).
>> The TSVWG Chairs are now aware that we need to plan to make progress and will be in touch in the next week or so, with a plan move forward in the absence of an IETF face-to-face meeting.
> Noted.
>
>   - Jonathan Morton
>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/