Re: [tsvwg] RFC 4301 on ECN codepoints (was RE: L4S vs SCE (Evolvability))

[Ingemar Johansson S <[e-mail]>]

Hi
Yes I am aware that SCE works with RFC6040 tunnels, it does however seem to 
work well with RFC4301 ditto.
Now I have no idea as regards to how large extent RFC4301 is deployed on the 
internet and whether this is a large problem for SCE or not. I notice however 
that RFC4301 is no problem for L4S.

/Ingemar

> -----Original Message-----
> From: Black, David <[e-mail]>
> Sent: den 2 januari 2020 17:26
> To: Ingemar Johansson S <[e-mail]>; Bob Briscoe
> <[e-mail]>; Roland Bless <[e-mail]>
> Cc: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-bell-
> labs.com>; Kyle Rose <[e-mail]>; [e-mail]; tsvwg-
> [e-mail]; Black, David <[e-mail]>
> Subject: RE: RFC 4301 on ECN codepoints (was RE: [tsvwg] L4S vs SCE
> (Evolvability))
>
> See RFC 6040, which updates this area of RFC 4301.
>
> In addition, see  draft-ietf-tsvwg-rfc6040update-shim and 
> draft-ietf-tsvwg-ecn-
> encap-guidelines, both of which apply RFC 6040 to additional
> encapsulations/tunnels ... and both of which are on Bob Briscoe's "round 
> tuit"
> list to write better fragmentation text (what's currently in both drafts 
> does not
> comply with RFC 3168's fragmentation provisions, hence has to be changed).
>
> Thanks, --David
>
> > -----Original Message-----
> > From: Ingemar Johansson S <[e-mail]>
> > Sent: Thursday, January 2, 2020 5:13 AM
> > To: Black, David; Bob Briscoe; Roland Bless
> > Cc: De Schepper, Koen (Nokia - BE/Antwerp); Kyle Rose; [e-mail];
> > [e-mail]; Ingemar Johansson S
> > Subject: RFC 4301 on ECN codepoints (was RE: [tsvwg] L4S vs SCE
> > (Evolvability))
> >
> > Hi
> >
> > I put my question in a separate thread to avoid messing up the
> > original thread.
> >
> > I read in https://tools.ietf.org/html/rfc4301#section-5.1.2.1  the
> > following
> >
> >       (6) If the ECN field in the inner header is set to ECT(0) or
> >           ECT(1), where ECT is ECN-Capable Transport (ECT), and if the
> >           ECN field in the outer header is set to Congestion Experienced
> >           (CE), then set the ECN field in the inner header to CE;
> >           otherwise, make no change to the ECN field in the inner
> >           header.  (The IPv4 checksum changes when the ECN changes.)
> >
> > So, if we have an SCE flow that is tunneled:
> > 1) At the tunnel ingress, the ECN bits are copied to the outer header.
> > 2) Somewhere along the tunneled path an SCE compatible AQM remarks
> > from ECT '10' to SCE '01'.
> > 3) At tunnel egress, given rule #6 above, I understand that the inner
> > header will still be ECT '10'.
> >
> > In order words, the SCE congestion marks along the tunneled interface
> > become ignored and the queue will grow until packets are either CE
> > marked, or dropped.
> >
> > Is this a correct interpretation ?
> >
> > /Ingemar
> >
> >
> > > -----Original Message-----
> > > From: Black, David <[e-mail]>
> > > Sent: den 2 januari 2020 00:32
> > > To: Bob Briscoe <[e-mail]>; Roland Bless
> > <[e-mail]>
> > > Cc: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-
> > bell-
> > > labs.com>; Ingemar Johansson S <[e-mail]>;
> > Kyle
> > > Rose <[e-mail]>; [e-mail]; [e-mail];
> > > Black,
> > David
> > > <[e-mail]>
> > > Subject: RE: [tsvwg] L4S vs SCE (Evolvability)
> > >
> > > Bob,
> > > (posting as an individual)
> > >
> > > The assertion that SCE traffic will "starve" is just plain wrong in
> > > the following ...
> > >
> > > > > The SCE markings can be used independently whether you have
> > multiple
> > > > > queues or just a single queue.
> > > > [BB] This seems to be the opposite of reality. Perhaps you've used
> > > > the word 'can' because you think it might be possible. But there's
> > > > good reason to doubt that. As follows...
> > > >
> > > > Packets from non-SCE and from SCE senders are indistinguishable by
> > > > just looking at them. So, if there's a single queue, they have to
> > > > be mixed together in it {Note 1}. But a single queue can only have
> > > > one length.
> > > > Any transports that don't understand SCE drive the single queue to
> > > > a deeper operating point (defined by CE or drop).
> > >
> > > The reasoning appears to be sound up to this point, but the next
> > > sentence does not follow from the above:
> > >
> > > > This overruns the SCE
> > > > operating point, and causes SCE marking to approach 100% {Note 2}.
> > > > Then those transports that do understand SCE will starve.
> > >
> > > Actually, it causes both SCE and non-SCE senders to operate at that
> > "deeper
> > > operating point (defined by CE or drop)" because SCE traffic has an
> > > RFC-3168-
> > > like response to CE marks, and the usual reaction to drops.  The
> > > result is a deeper queue than would have resulted in the absence of
> > > non-SCE traffic,
> > but
> > > the result is definitely *not* starvation.
> > >
> > > Thanks, --David
> > >
> > > > -----Original Message-----
> > > > From: Bob Briscoe <[e-mail]>
> > > > Sent: Tuesday, December 31, 2019 6:11 PM
> > > > To: Roland Bless
> > > > Cc: De Schepper, Koen (Nokia - BE/Antwerp); Ingemar Johansson S;
> > > > Kyle Rose; [e-mail]; [e-mail]
> > > > Subject: Re: [tsvwg] L4S vs SCE (Evolvability)
> > > >
> > > >
> > > > [EXTERNAL EMAIL]
> > > >
> > > > Roland,
> > > >
> > > > As I just said, I noticed that Koen's useful response snipped some
> > > > of your responses about evolvability that I ought to have addressed.
> > > > Sorry, you'll need to reload state from November...
> > > >
> > > > On 22/11/2019 02:32, Roland Bless wrote:
> > > > > Hi Bob,
> > > > >
> > > > > see inline.
> > > > >
> > > > > On 21.11.19 at 15:44 Bob Briscoe wrote:
> > > > >> Roland,
> > > > >>
> > > > >> I'm not getting it....
> > > > >>
> > > > >> On 21/11/2019 09:32, Roland Bless wrote:
> > > > >>> Hi Bob,
> > > > >>>
> > > > >>> see inline.
> > > > >>>
> > > > >>> On 21.11.19 at 19:34 Bob Briscoe wrote:
> > > > >>>> On 20/11/2019 21:22, Roland Bless wrote:
> > > > >>>>> Yes, but as I also expressed my concerns w.r.t. the L4S
> > > > >>>>> codepoint earlier, at the cost of binding this to a quite
> > > > >>>>> fixed set of L4S behaviors and "burning" the last ECT codepoint.
> > > > >>>>> Personally, I like concepts with a little bit more potential
> > > > >>>>> to be useful for future development (evolvability) of
> > > > >>>>> congestion controls, e.g., BBRv2 and LoLa could also benefit
> > > > >>>>> from an SCE-like
> > > marking...
> > > > >>>>>
> > > > [BB] BBRv2 has not used SCE, but it has used L4S marking. And the
> > > > authors of the LoLa draft have not used SCE but they have merged
> > > > with the NQB draft, so that they can work with L4S as well. So
> > > > perhaps your belief that SCE is "more evolvable" is just that - a 
> > > > belief.
> > > >
> > > >
> > > > >> This last sentence... please really spell it out what you could
> > > > >> possibly be meaning here. What has made you suddenly think this
> > > > >> particular marking scheme has got magical properties that
> > > > >> bestow evolvability? I'm serious, if you can't explain why
> > > > >> you've said a sentence like this, it implies you are under the
> > > > >> spell of some cult or fad.
> > > > > Not sure what you are trying to indicate by your last sentence,
> > > > > but sure, I can elaborate a bit on my last sentence.
> > > > > I find it architecturally cleaner to have an additional ECN
> > > > > codepoint used for indicating "SCE" rather than (ab)using it as
> > > > > classifier for the dualQ AQM.
> > > > [BB] I just forked another thread 'ECN as a classifier' to address
> > > > that assertion.
> > > > > The SCE markings can be used independently whether you have
> > multiple
> > > > > queues or just a single queue.
> > > > [BB] This seems to be the opposite of reality. Perhaps you've used
> > > > the word 'can' because you think it might be possible. But there's
> > > > good reason to doubt that. As follows...
> > > >
> > > > Packets from non-SCE and from SCE senders are indistinguishable by
> > > > just looking at them. So, if there's a single queue, they have to
> > > > be mixed together in it {Note 1}. But a single queue can only have
> > > > one length.
> > > > Any transports that don't understand SCE drive the single queue to
> > > > a deeper operating point (defined by CE or drop). This overruns
> > > > the SCE operating point, and causes SCE marking to approach 100% {Note
> 2}.
> > > > Then those transports that do understand SCE will starve.
> > > >
> > > > In contrast, L4S works in dual queues or multiple queues (where 
> > > > 'works'
> > > > means it gives very low latency without losing utilization).
> > > >
> > > > Also, there is considerable scope for evolvability with both:
> > > > * the dualQ is a framework that is intended to encompass different
> > > > AQMs in future.
> > > > * and there is considerable scope for developing the ways FQ uses L4S.
> > > >
> > > > {Note 1}: Unless L4 identifiers are used to separate out
> > > > microflows, and identify low latency flows by their behaviour. But
> > > > then it's not really a single queue, unless you make it look like
> > > > a single queue, but still treat it like multiple queues, as in
> > > > LFQ. However, schemes like LFQ are "single-queue" in name only.
> > > > They still have all the downsides of multiple queues.
> > > >
> > > > {Note 2}: Unless the SCE operating point is hardly any shallower
> > > > than that for CE, in which case SCE gives hardly any improvement.
> > > >
> > > > > If one leaves the coupling aside the algorithm for marking will
> > > > > probably not differ so much from what is proposed in L4S.
> > > > [BB] You still haven't explained (rather than asserted a belief)
> > > > why the choice between SCE and L4S has anything to do with 
> > > > evolvability.
> > > >
> > > > >>>> My whole purpose in solving the problem of deploying scalable
> > > > >>>> CCs
> > > > over
> > > > >>>> the Internet was to re-juvenate evolution (to widen the range
> > > > >>>> of applications that could be supported by different
> > > > >>>> transport behaviours, particularly for real-time with low
> > > > >>>> latency and high throughput at the same time). One of the
> > > > >>>> main things that has stopped CCs evolving so
> > > > far
> > > > >>>> is the need for friendliness with the Reno behaviour that was
> > > > >>>> not scaling over the years.
> > > > >>> FWIW, our delay-based approach has deployment problems at the
> > > > >>> other end of the spectrum: it gets suppressed by loss-based CCs...
> > > > >>> (and we don't want to sacrifice our low delay goal).
> > > > >>>
> > > > >>>> If SCE is primarily supported in FQ AQMs, that will constrain
> > > > >>>> flows to be capped at the rate that FQ gives them. How is
> > > > >>>> that doing anything
> > > > >>> I was solely referring to the marking behavior of SCE, not a
> > > > >>> particular implementation based on FQ-AQMs.
> > > > >> This implies you believe all that silliness about a shallower
> > > > >> SCE threshold not starving an SCE flow in a single queue,
> > > > >> because it makes SCE flows not want to use the queue.
> > > > > I do not understand what you are saying here.
> > > > [BB] I've now explained this above (after "But there's good reason
> > > > to doubt that...").
> > > >
> > > > The silliness I'm referring to is the statement by Jonathan M in
> > > > his tsvwg talk at IETF-106 that CNQ is backward compatible with
> > > > RFC3168 traffic, because its performance for SCE traffic when
> > > > mixed with 3168 is bad enough that it discourages use of SCE alongside
> 3168.
> > > >
> > > > >>>> other than massively constraining future evolution of CCs,
> > > > >>>> especially real-time ones? See Per-Flow Scheduling and the
> > > > >>>> End-to-End
> > > > Argument
> > > > >>>> <https://protect2.fireeye.com/v1/url?k=10e7b9ba-4c6d6cbf-
> > 10e7f921
> > > > >>>> -861010bc36ff-2b3c729b7d8a378e&q=1&e=35be9991-3367-47d3-
> > b023-
> > > ed85
> > > > >>>>
> > >
> > 01dddd52&u=http%3A%2F%2Fbobbriscoe.net%2Fprojects%2Flatency%2Fpe
> > r
> > > > >>>> -flow_tr.pdf>. I don't
> > > > need
> > > > >>>> to tell you that the e2e argument is all about giving end
> > > > >>>> systems the power to innovate without permission.
> > > > >>>> Anyway, what are you imagining would stop CCs evolving
> > > > >>>> alongside
> > > > other
> > > > >>>> scalable CCs? In much the same way CCs have always evolved.
> > > > >>>> With L4S
> > > > you
> > > > >>>> have a clean slate that seems just like a FIFO with a shallow
> > > > >>>> ECN-only immediate AQM. And other flows are causing you
> > > > >>>> hardly any delay and
> > > > very
> > > > >>>> rarely any loss. Think of all the things you can do with
> > > > >>>> that. Go evolve, Roland!
> > > > >>> I already said that the Dual Queue AQM is nice, but comes with
> > > > >>> the problem due to its built-in dropping law. Other CCs may
> > > > >>> react differently and BBR was one example of newer CCs that
> > > > >>> did not work in the L4S
> > > > queue
> > > > >>> without further adaptation.
> > > > >> The built in dropping law is for the old traffic ('pre-evolved'
> > > > >> if you
> > > > >> like) that doesn't respond to anything else but loss. That's
> > > > >> what drop-based AQMs were for - to fool Reno etc into thinking
> > > > >> they had hit the buffers, so to speak.
> > > > > Yes, I understand that. But what happens if the classic traffic
> > > > > once vanishes and we only have low-delay congestion control and
> > > > > want to re-use the "second" queue for other purposes, e.g.,
> > > > > using it for flows that do not use loss-based CC?
> > > > [BB] Let me repeat back the question, 'cos it seems rather odd, so
> > > > perhaps I've misunderstood.
> > > >
> > > > You want me to assume that the classic behaviour of repeatedly
> > > > seeking out more capacity until loss occurs doesn't exist on the
> > > > Internet any more. There's only low-delay CC, by which I think you
> > > > mean CC based on L4S ECN (but I'm not sure that's your meaning).
> > > >
> > > > Then, it sounds like everything would be working well. So why do
> > > > you want to re-use the "second" queue for other purposes? Nothing
> > > > is wasted if you don't use it. It has no capacity associated with it.
> > > > it's just some lines of code sitting there in case packets appear
> > > > from a classic
> > > CC.
> > > >
> > > >
> > > > >>>    > The key thing here is the wording of the Prague
> > > > >>> requirements
> > > > >>>> <https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-08#s
> > > > >>>> ection-
> > 4>.
> > > > >>>> We have a session in the 'Prague' side-meeting tomorrow to
> > > > >>>> review
> > > > them
> > > > >>>> (and I encourage this on this list too).
> > > > >>>>
> > > > >>>> Later down the line, if the L4S experiment is successful,
> > > > >>>> there will be an opportunity to review these requirements if
> > > > >>>> a standards track doc replaces the experimental (it is easier
> > > > >>>> to relax CC requirements than tighten them). So, for the expt
> > > > >>>> track, the requirements are designed to protect competing
> > > > >>>> flows from harm in a tighter way than you might find in RFC5033 
> > > > >>>> or
> similar.
> > > > >>>>
> > > > >>>>
> > > > >>>> Nonetheless, even the 1/p isn't tightly spec'd, quoting:
> > > > >>>>
> > > > >>>>      The inverse proportionality requirement above is worded
> > > > >>>>      as a 'SHOULD' rather than a 'MUST' to allow reasonable
> > > > >>>> flexibility
> > > > >>>>      when defining these specifications.
> > > > >>> Yes, but something has to be implemented (in hardware) and
> > > > >>> therefore
> > > > it
> > > > >>> is fixed for some time and it should be consistent along a
> > > > >>> path, so I don't see a viable path for evolvement of this coupling 
> > > > >>> law...
> > > > >> Why does the coupling law need to evolve? It's a relationship
> > > > >> between something invariant (scalable) and the past (which is
> > > > >> over, by definition, so it's not going to change now).
> > > > > See above, probably we want to put something into queue which is
> > > > > using a non-loss-based congestion control and/or we need to
> > > > > change the (1/p,1/p^2) marking.
> > > > [BB] The queue doesn't induce loss, the CC does. The queue isn't
> > > > 1/p or
> > > > 1/p^2 or whatever, the different CCs are. So, if a CC is using the
> > > > C queue without inducing any loss (e.g. a delay-based CC might be
> > > > able to keep the queue under the AQM's target delay), then the
> > > > coupling won't couple any marking across to the L queue.
> > > >
> > > > But I can't imagine why a delay-based CC that can keep the queue
> > > > below the target of the AQM would want to classify itself into a
> > > > queue designed for CCs that need to induce a decent queue (in
> > > > order to maintain full utilization), given such queue-inducing
> > > > traffic might start up at any time.
> > > >
> > > > >> If you want to evolve, you select what's best for you -
> > > > >> probably the nice clean L4S ECN queue. I still don't get what
> > > > >> sort of evolution you can't do? Evolvability isn't about a
> > > > >> researcher being able to do what they'd like.
> > > > > I also don't understand it in this way, but investigating
> > > > > invariables and degrees of freedom prudently may enable or
> > > > > facilitate deployment of new stuff. If that new stuff cannot be
> > > > > deployed it never gets a chance of being weeded out later either.
> > > > [BB] If you have something specific in mind, please say it. This
> > > > is all getting rather abstract.
> > > >
> > > > This does make me want to question your notion of evolvability.
> > > > Usually, when we make sure the Internet supports evolvability, we
> > > > mean evolvability of new applications. We don't mean the ability
> > > > for researchers to come up with different ways to solve problems
> > > > that are already solved. There seems to be a hint of a complaint
> > > > in your emails that L4S doesn't leave room for researchers to
> > > > solve the same problem differently. That sort of evolvability is
> > > > rather a luxury isn't it? If one approach solves an enduring
> > > > problem with the Internet, it would be rather churlish to say "No,
> > > > we can't use this solution, because it might preclude some ideas
> > > > that might lead to other solutions to the same problem in the future, 
> > > > but
> we're not sure."
> > > >
> > > > Having said that, I hope I (and Koen) have shown that L4S still
> > > > leaves scope for complementary delay-based CCs, not to mention
> > > > scope for different AQMs and for different FQ-based solutions.
> > > >
> > > >
> > > > Regards
> > > >
> > > >
> > > >
> > > > Bob
> > > >
> > > >
> > > >
> > > > >
> > > > > Roland
> > > > >
> > > >
> > > > --
> > > >
> > __________________________________________________________
> > > > ______
> > > > Bob
> > > > Briscoehttps://protect2.fireeye.com/v1/url?k=9834ac6d-c4be7968-
> > 9834ecf
> > > > 6-861010bc36ff-d256b2de1d12a128&q=1&e=35be9991-3367-47d3-b023-
> > > ed8501dd
> > > > dd52&u=http%3A%2F%2Fbobbriscoe.net%2F