Re: [tsvwg] plan for L4S issue #29

[Pete Heist <[e-mail]>]

On Mon, 2020-09-28 at 09:54 +0000, De Schepper, Koen (Nokia -
BE/Antwerp) wrote:
> Hi Sebastian,
> 
> > > 	How confident are we that the "wide variety of conditions"
> > > under which this supposedly works is actually representative for
> > > the existing internet?
> 
> Only real world deployments will answer this question. I guess any
> other answer will be purely speculation...
> 
> > > 	rfc3168 present; rfc3168 detected: 	hit (true positive)
> > > 	rfc3168 present; rfc3168 not-detected:	miss	(false
> > > negative)
> > > 	rfc3168 absent; rfc3168 detected:		false alarm
> > > (false positive)
> > > 	rfc3168 absent; rfc3168 not-detected:	correct rejection (true
> > > negative)
> 
> Thanks for this. I like this classification and the tabular
> representation a lot. I guess it is ok then to use in the draft
> "missed detection" and "false detection"? I assume the other correct
> cases will be less referred to.

Personally, "missed detection" and "false detection" sound fine to me,
as I get the right sense of both the bottleneck and the result in both
cases.

> > > 	That is why I believe that rfc3168 detection in TCP Prague is a
> > > red herring that distracts from fixing L4S true issues. Like
> > > demonstrating that the current implementation actually performs
> > > as inteded over long holti-hop high RTT high Bandwdth links, over
> > > asymmetric links, and over uni- and bidirectionally saturated
> > > links.
> 
> These are all known problems that also exist in Classic TCP on FIFO
> queues. Solutions are known for several (all?) of them, which can as
> well be implemented for L4S (Prague CCs and L4S AQMs). So I wouldn't
> blame L4S for those problems or expect that a reference
> implementation is including all solutions to all known problems.
> 
> Koen.
> 
> -----Original Message-----
> From: Sebastian Moeller <[e-mail]> 
> Sent: Wednesday, September 23, 2020 7:47 PM
> To: De Schepper, Koen (Nokia - BE/Antwerp) <
> [e-mail]>
> Cc: Pete Heist <[e-mail]>; Wesley Eddy <[e-mail]>; 
> [e-mail]
> Subject: Re: [tsvwg] plan for L4S issue #29
> 
> Hi Koen,
> 
> 
> > On Sep 23, 2020, at 17:46, De Schepper, Koen (Nokia - BE/Antwerp) <
> > [e-mail]> wrote:
> > 
> > Hi Pete,
> > 
> > I don't think the goal is to wait for a perfect RFC3168 detection
> > solution and we already have an implementation that works well
> > under a wide variety of conditions without additional
> > configuration.
> 
> 	How confident are we that the "wide variety of conditions"
> under which this supposedly works is actually representative for the
> existing internet?
> 
> 
> > This also means that it can still be improved, which I think can be
> > done more appropriate when facing real world issues. The goal of
> > the operational guidelines is to provide a larger set of tools for
> > solving problems with classic ECN bottlenecks, exactly to avoid the
> > need to rely on perfect end-host detection mechanism only.
> 
> 	I grudgingly agree that trying to fix L4S's AQM short comings
> by mandating fancy heuristics in the end points seems to be a loosing
> proposition. 
> 
> > Related to the "false negatives" and "false positives" naming, I
> > agree that it is very confusing. As the goal is to detect Classic
> > ECN network AQM behavior, maybe better and shorter names could be
> > "false-detect" and "false-non-detect"? 
> 
> 	I do not think that that is helpful, false-positive and false
> positive have much better well-known definitions that actually apply
> here. So if other nomenclature should be used, let's make it clearly
> different or better yet use already established terms. 
> 
> 	We could switch to signal detection theory terms (which is just
> one way to think about a classifier with just two options), then the
> four combinations of truth and classification for our RFC3168
> detector would be described with the following terms:
> 
> rfc3168 present; rfc3168 detected: 	hit (true positive)
> rfc3168 present; rfc3168 not-detected:	miss	(false
> negative)
> rfc3168 absent; rfc3168 detected:		false alarm (false
> positive)
> rfc3168 absent; rfc3168 not-detected:	correct rejection (true
> negative)
> 
> these terms are pretty much standard for similar detection problems
> (and also offer a decent approach to assess the effectiveness of the
> detector).
> 
> Let's stick to some already established nomenclature, whether that is
> the true/false positive/negative one or the DST one, please (see e.g.
> https://en.wikipedia.org/wiki/Evaluation_of_binary_classifiers).
> 
> The bigger issue I have with this is that this is still just a
> "fudge" to be able write in the internet draft "transports are
> required to implement RFC3168 detection, as demonstrated in TCP
> Prague" even though everybody here should know that that is not going
> to happen. People will mark whatever with ECT(1) and we will have to
> live with the fall-out. IMHO this is a sufficient reason to put the
> smarts in the AQM, instead of the current approach of assuming that
> end-points will just do this out of goodness of their heart.
> 
> That is why I believe that rfc3168 detection in TCP Prague is a red
> herring that distracts from fixing L4S true issues. Like
> demonstrating that the current implementation actually performs as
> inteded over long holti-hop high RTT high Bandwdth links, over
> asymmetric links, and over uni- and bidirectionally saturated links.
> 
> Best Regards
> 	Sebastian
> 
> > Koen.
> > 
> > -----Original Message-----
> > From: Pete Heist <[e-mail]>
> > Sent: Wednesday, September 23, 2020 2:19 PM
> > To: De Schepper, Koen (Nokia - BE/Antwerp) 
> > <[e-mail]>; Wesley Eddy 
> > <[e-mail]>
> > Cc: [e-mail]
> > Subject: Re: [tsvwg] plan for L4S issue #29
> > 
> > Hi Koen,
> > 
> > I can definitely understand the need to turn bottleneck detection
> > on and off for testing, or for additional knobs during development.
> > 
> > Overall, I suspect that there will be more questions about
> > potential problems if bottleneck detection is not a MUST for
> > implementations in the draft, or not baked into the final
> > implementation in a way that works well under a wide variety of
> > conditions without additional configuration.
> > 
> > On an easier topic, I wonder if we shouldn't change the "false 
> > negatives" and "false positives" terminology to something clearer, 
> > like "mis-identification of RFC 3168 bottlenecks as L4S", or "mis- 
> > identification of L4S bottlenecks as RFC 3168", respectively. I
> > might 
> > have opened up a can of worms there in trying to save a few words.
> > :)
> > 
> > Pete
> > 
> > On Tue, 2020-09-15 at 12:43 +0000, De Schepper, Koen (Nokia -
> > BE/Antwerp) wrote:
> > > Hi Wes, Pete,
> > > 
> > > I think to make progress on avoiding both false negatives and
> > > false 
> > > positives, a good view on the conditions that cause problems is 
> > > needed. So we better have the means to detect the real life
> > > impact of 
> > > Classic-ECN-FIFO deployments. This means we need to be able to
> > > switch 
> > > off Classic ECN detection (under controlled or even known
> > > conditions).
> > > 
> > > Another point is that it would be useful also to have all
> > > control 
> > > variables of the existing implementation configurable for
> > > everyone 
> > > willing to further experiment (without necessarily needing to
> > > change 
> > > code). As I understood, the right tuning of these can bring a lot
> > > of 
> > > further improvement opportunities. Also depending on a typical 
> > > deployment, these parameters could be tuned for that specific 
> > > targeted case.
> > > 
> > > So the resolution of this issue is exactly to facilitate further 
> > > improving the detection algorithm (preferably via tuning), and
> > > being 
> > > able to disable it when conditions are controlled or safe to
> > > avoid 
> > > these false negatives.
> > > 
> > > I think these are topics that can be covered by the Operational 
> > > Guidelines draft.
> > > 
> > > Regards,
> > > Koen.
> > > 
> > > -----Original Message-----
> > > From: tsvwg <[e-mail]> On Behalf Of Pete Heist
> > > Sent: Friday, July 31, 2020 8:53 PM
> > > To: Wesley Eddy <[e-mail]>
> > > Cc: [e-mail]
> > > Subject: Re: [tsvwg] plan for L4S issue #29
> > > 
> > > Hi Wesley,
> > > 
> > > One thing I noticed during testing was that the current 
> > > implementation of TCP Prague in Linux allows disabling
> > > bottleneck 
> > > detection through the prague_ecn_fallback kernel module parameter
> > > (
> > > https://github.com/L4STeam/linux/blob/0e7cf8acb318873c3f61084453f8da1
> > > 5 b2e398be/net/ipv4/tcp_prague.c , line 158). I don’t know if
> > > that 
> > > was left in only for testing.
> > > 
> > > In section 6.3.3 of l4s-arch, there is discussion around classic 
> > > bottleneck detection. Since I don’t see an explicit MUST that it 
> > > remain enabled (although I do see the text “an L4S sender will
> > > have 
> > > to fall back to…”), it’s not completely clear to me if it’s
> > > actually 
> > > required to be implemented and permanently enabled in all 
> > > implementations. If it is, I suppose the implementation should 
> > > reflect that also.
> > > 
> > > While I feel it best that detection identifies both types of
> > > queues 
> > > accurately, if bottleneck detection were both an explicit MUST in
> > > the 
> > > text *and* not possible to disable in any implementation, I
> > > think 
> > > that would make the misidentification of L4S queues as classic
> > > ECN 
> > > queues less of a safety concern, since it would be impossible to
> > > turn 
> > > off. It would remain an issue for the architecture overall
> > > though.
> > > 
> > > Hope that helps...
> > > 
> > > Pete
> > > 
> > > > On Jul 31, 2020, at 5:41 PM, Wesley Eddy <[e-mail]>
> > > > wrote:
> > > > 
> > > > Hello, ticket #29 for the L4S documents is about classic
> > > > bottleneck 
> > > > detection misidentifying L4S queues as classic ECN queues.
> > > > 
> > > > https://trac.ietf.org/trac/tsvwg/ticket/29
> > > > 
> > > > In contrast to other issues, it doesn't seem like this should
> > > > block 
> > > > a WGLC on the L4S drafts.
> > > > 
> > > > 	• It is specific to classic bottleneck detection
> > > > algorithm, which 
> > > > is planned to be worked on in the Prague ICCRG draft.
> > > > 	• The result is sometimes failing to achieve the best
> > > > possible L4S 
> > > > behavior, but doesn't seem to be an Internet safety
> > > > issue.  This 
> > > > resulting in people turning off classic bottleneck detection
> > > > would 
> > > > be a different issue, and something maybe the operator
> > > > guidelines 
> > > > would address.
> > > > 	• It seems like it can be worked on further in the
> > > > course of L4S 
> > > > experimentation, without negative effects to others.
> > > > So, I believe we should track this work in the ICCRG, and close
> > > > the 
> > > > ticket here.  Please let me know in the next week if I've 
> > > > misunderstood any aspect of this and it should remain open.
> > > > 
> > > >