Re: [tsvwg] path forward on L4S issue #16

[Ingemar Johansson S <[e-mail]>]

Hi
Inline...
/Ingemar

> -----Original Message-----
> From: Sebastian Moeller <[e-mail]>
> Sent: den 11 juni 2020 12:40
> To: Ingemar Johansson S <[e-mail]>
> Cc: Jonathan Morton <[e-mail]>; [e-mail]
> Subject: Re: [tsvwg] path forward on L4S issue #16
> 
> Hi Ingemar,
> 
> as usual more below in-line.
> 
> > On Jun 11, 2020, at 10:23, Ingemar Johansson S
> <[e-mail]> wrote:
> >
> > Hi
> >
> > Please see inline [IJ]
> > /Ingemar
> >
> >> -----Original Message-----
> >> From: Sebastian Moeller <[e-mail]>
> >> Sent: den 10 juni 2020 22:05
> >> To: Ingemar Johansson S <[e-mail]>
> >> Cc: Jonathan Morton <[e-mail]>; [e-mail]
> >> Subject: Re: [tsvwg] path forward on L4S issue #16
> >>
> >> Hi Ingemar,
> >>
> >> more below in-line.
> >>
> >>> On Jun 10, 2020, at 19:15, Ingemar Johansson S
> >> <[e-mail]> wrote:
> >>>
> >>> Hi
> >>>
> >>> As regards to dualQ (below), do you see any specific reason why it
> >>> would not be possible to upstream (complexity/memory whatever) it or
> >>> is your argument that it is just not done yet ?
> >>
> >> 	[SM] Caveat: I am not involved in the Linux kernel and hence can not
> >> predict with any certainty dualQ's likelihood of getting included.
> >> BUT as
> > you
> >> know I am arguing for some time now, that there is a discrepancy
> >> between the dualQ claims and its actual performance. In other words
> >> it is not
> > done,
> >> and given the responses from the L4S team to this issue (like the
> >> 15ms
> > hack
> >> in TCP Prague's CC response) I am more and more of the opinion, that
> >> this
> > is
> >> not simply a case of dualQ just not being finished, but rather that
> >> the
> > current
> >> dualQ design simply can not meet its promises and someone would need
> >> to go back to the drawing board.
> >
> > [IJ] Yes, and understand that there are widely differing views about
> > the severity of this possible problem, if it really is a problem.
> 
> 	[SM] Sorry, not accepting that starving nonECT(1) flows at short
RTTs
> is a severe problem seems to be a position not even team L4S took. It has
> been demonstrated from early on (it is visible in allmost all L4S papers
and in
> the measurements from team SCE) that the coupling heuristic that is an
> essential component of dualQ's design simply does not give the required
> isolation guarantees needed to save flows in the non-LL queue from the
> consequences of its ~15:1 LL:no-LL queue scheduler. At its core dualQ is a
> ~16:1 scheduler that by applying a number of heuristics will give rough
> equivalence between the queues under a number of realistic conditions,
> unfortunately it fails to do so under a number of different equally
realistic
> conditions. Team L4S went so far to propose an absolute ugly hack in TCP
> Prague to paper over this mis-design in dualQ, accepting that this is a
severe
> problem.
> 	So are you really taking the position that this is not really a
problem?
> Especially since, as i mentioned before, L4S really only has seen testing
for
> short RTT/few hop scenarios, and is expected to be rolled-out/used by
> typically close by CDNs?
>

[IJ] I believe that this discussion belongs to issue #28 so I wont delve
into it in this thread. 

> 
> 
> >
> >>
> >>>
> >>> Also, do you have any comments to my three other questions, please
> >>> refer to earlier email in the thread for the context.
> >>
> >> 	[SM] I snipped these out of my reply since I had nothing meaningful
> >> to add to those.
> >>
> >>
> >>> 1) Do you have any public sources that confirm the numbers you quote
> >>> below ?. I tried to look up data on this but it surely is not easy.
> >>
> >> 	[SM] I do not know where Jonathan's numbers come from. But
> >> https://datatracker.ietf.org/meeting/98/materials/slides-98-maprg-tcp
> >> -ecn-
> >> experience-with-enabling-ecn-on-the-internet-padma-bhooma-00 has
> some
> >> numbers from Apple, I believe Bob cited these numbers multiple times
> >> in
> > the
> >> past.
> >> Given the fact that 3gpp contains quite a lot of large carriers maybe
> >> that would be a forum to ask for numbers?
> >
> > [IJ] OK, that is the same reference that I have found, I was thinking
> > that there is perhaps something more up to date available?
> 
> 	[SM] I believe the problem is, that almost nobody (except maybe for
> the ripe atlas project) has the required testing nodes deployed required
to
> assess the likelihood/prevalence on CE marking on the internet. Ideally
you
> have a network of widely distributed testing nodes in leaf networks and
try
> to run saturating ECN enabled transfers between them, while recording the
> number of CE marks per path (pair of testing nodes), nobody in academia
> really has such a network, but a few major carriers should be able to set
> something like this up easily, given the big names in support of L4S, I
would
> find it marvelous if these could cooperate to get the CE measurements
done,
> before proceeding with the L4S drafts.
> 
> 
> > As regards to 3GPP
> > access then I can very safely say that we can stop worrying, there is
> > AFAIK no ECN marking (RFC3168 or anything else) in 3GPP base station
> > nodes so there are no interop concerns here. There is a remote
> > possibility that ECN marking can be enabled in e.g. industrial Cisco
> > LTE routers I tried this a while back but had problems with ECN
> > bleaching which is a configuration matter in 3GPP networks.
> 
> 	[SM] I meant asking the 3GPP members to help in setting up an
> internet wide trawl for CE marked packets, for this leaf/edge or close to
the
> leaf/edge mesurement nodes seem required, and hence operators/carrieres
> that connect end-users would be in a good position to measure CE mark
> probabilities, no?

[IJ] OK, my misunderstanding. I leave this uncommented as I don't personally
know of any such activities. 

> 
> 
> >
> >>
> >>
> >>> 2) Which foras are the vendors that manufacture CPEs active in (if
> >>> any)
> > ?.
> >>
> >> 	[SM] I believe that OpenWrt certainly supports rfc3168 behaviour,
> >> and there are CPE that run on modified OpenWrt, so the OpenWrt forum
> >> might be a decent starting point?
> >
> > [IJ] OK, thanks, I was of the impression, that OpenWrt is on the table
> > in the implementation of fq-codel and the like, is this correct?,
> 
> 	[SM] SQM-scripts in OpenWrt allows to choose between cake and
> fq_codel, but cake can be run in single queue mode. QOS-scripts in OpenWrt
> IIRC defaults to fq_codel.
> 
> 
> > if that is
> > true then it perhaps fits bets with L4S issue #17 but perhaps it is
> > relevant for #16 and #17?.
> 
> 	[SM] I believe that Pete Heist's recent data indicates that thanks
to
> the birthday paradoxon, stochastic fq as used in fq_codel and cake is not
a
> full remedy for L4S's changed response to CE-marks, so maybe that
> separation needs to be reverted again.
> 
> > Has this extended beyond the early adopter do-it-your-self community?.
> > I recall from earlier in this long tread that you found off the shelf
> > routers in stores, were these shipped and ready with RFC3168 support
> > or was an update necessary to make it RFC3168 capable?.
> 
> 	[SM] There is Evenroute's IQrouter, that you can order at amazon,
> that runs a modified OpenWrt and does use fq (I think it defaults to cake
> nowadays, and cake is an rfc3168 AQM), then there is the Turris Omnia (and
> also the MOX I believe) that use fq_codel for their guest networks. And
since
> both of these are OpenWrt based they will default to fq_codel for all
> interfaces.

[IJ] Thanks, looked up the Turris Omnia, it mentions automatic updates with
rationale to protect against cyber threats. I interpret this nice feature as
a possibility to update also for support L4S later on. This is course
pending interest from the developer community. But atleast there are options
to avoid the effects of issue #16 and #17 and avoid some of the issues with
legacy only RFC3168 compatible AQMs, right ?.
Something similar seem to apply to the Evenroute quote "Over-the-air
firmware updates to stay current" .

> 
> 
> >
> >>
> >>> 3) As regards to endpoints implementing RFC3168, do you refer to
> >>> servers and clients or something else?. My interpretation is servers
> >>> and clients and I don't believe that they are central  to this
> >>> discussion, or do I miss something ?.
> >>
> >> 	[SM] Well, it is these end-points that are going to suffer, when L4S
> >> gets it wrong (when, not if). So these numbers give you an estimate
> >> of the potential fall-out area.
> >
> > [IJ] OK, then I understand.
> 
> 	[SM] Please note, that I can not speak for Jonathan, so this was
just
> my opinion on that matter.
> 
> 
> > I was a bit mislead as I assumed that we discussed network nodes that
> > do RFC3168 ECN marking, not end hosts that negotiate ECN. I would like
> > to see these as separate parts of the problem.
> 
> 	[SM] Fair enough. To assess the total risk from L4S's changed CE-
> response however, we need to look at all these things in integrated form.

[IJ] Sure, but it is still important to keep these things separate, after
all one can alter the argument and say that end hosts can update to use L4S
with BBRv2 or Prague..