Re: ECN in QUIC connection migration

Gorry Fairhurst <gorry@erg.abdn.ac.uk> Wed, 21 February 2018 16:58 UTC

Return-Path: <gorry@erg.abdn.ac.uk>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF3B112D87C for <quic@ietfa.amsl.com>; Wed, 21 Feb 2018 08:58:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.21
X-Spam-Level:
X-Spam-Status: No, score=-4.21 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K1uk0pUKoUp2 for <quic@ietfa.amsl.com>; Wed, 21 Feb 2018 08:58:20 -0800 (PST)
Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 04E8512D875 for <quic@ietf.org>; Wed, 21 Feb 2018 08:58:20 -0800 (PST)
Received: from Gs-MacBook-Pro.local (at-zeroshell-1.erg.abdn.ac.uk [139.133.217.68]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPA id 6B9EE1B00067; Wed, 21 Feb 2018 16:57:37 +0000 (GMT)
Message-ID: <5A8DA500.2070101@erg.abdn.ac.uk>
Date: Wed, 21 Feb 2018 16:57:36 +0000
From: Gorry Fairhurst <gorry@erg.abdn.ac.uk>
Reply-To: gorry@erg.abdn.ac.uk
Organization: University of Aberdeen
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:12.0) Gecko/20120428 Thunderbird/12.0.1
MIME-Version: 1.0
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
CC: "De Schepper, Koen (Nokia - BE/Antwerp)" <koen.de_schepper@nokia-bell-labs.com>, =?UTF-8?B?TWlyamEgS8O8aGxld2luZA==?= <mirja.kuehlewind@tik.ee.ethz.ch>, QUIC WG <quic@ietf.org>, Christian Huitema <huitema@huitema.net>, Ian Swett <ianswett@google.com>, "Lubashev, Igor" <ilubashe@akamai.com>, "ekinnear@apple.com" <ekinnear@apple.com>
Subject: Re: ECN in QUIC connection migration
References: <HE1PR0702MB36252D61A7A51B848B5DA9E7C2FA0@HE1PR0702MB3625.eurprd07.prod.outlook.com> <639A6936-34F2-4C27-973A-EEAE878677FA@apple.com> <VI1PR0701MB2126E79A17DB715F6C91EA05B9F90@VI1PR0701MB2126.eurprd07.prod.outlook.com> <HE1PR0702MB36256D92C008302CBD908DE9C2F20@HE1PR0702MB3625.eurprd07.prod.outlook.com> <EFA61DBC-1F6F-49C2-906C-D0054F2341C7@tik.ee.ethz.ch> <HE1PR0702MB3625AF6F741A89D4BD68E7CCC2F70@HE1PR0702MB3625.eurprd07.prod.outlook.com> <5CC94D22-1375-40A2-8084-BA4A43F6FA94@tik.ee.ethz.ch>, <HE1PR0702MB3625310E4FC43F4BCA7717EEC2F50@HE1PR0702MB3625.eurprd07.prod.outlook.com> <VI1PR0701MB21268D5DE0F26BF9A54530F9B9F50@VI1PR0701MB2126.eurprd07.prod.outlook.com> <HE1PR0702MB36252D23A798375FE12607FBC2CE0@HE1PR0702MB3625.eurprd07.prod.outlook.com>
In-Reply-To: <HE1PR0702MB36252D23A798375FE12607FBC2CE0@HE1PR0702MB3625.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/xGIPO6HVR7XHzLt6auArt-n-9bc>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Feb 2018 16:58:25 -0000

So I worked carefully through the text. Most of what I saw looked sound 
- well done!

At places the language looked inconsistent with either other parts of 
the document or with other RFCs. I updated the wiki to resulve thses 
issues. Hoepfully the new text is more close to what we need,

Gorry

On 21/02/2018, 14:37, Ingemar Johansson S wrote:
>
> Hi
>
> Trying to pick up this thread again.
>
> Do we have some kind of minimal consensus around the proposed ECN 
> functionality in QUIC ?
>
> https://github.com/quicwg/base-drafts/wiki/ECN-in-QUIC#suggested-additions-to-to-become-rfcs
>
> /Ingemar
>
> *From:* De Schepper, Koen (Nokia - BE/Antwerp) 
> [mailto:koen.de_schepper@nokia-bell-labs.com]
> *Sent:* den 14 februari 2018 17:49
> *To:* Mirja Kühlewind <mirja.kuehlewind@tik.ee.ethz.ch>ch>; Ingemar 
> Johansson S <ingemar.s.johansson@ericsson.com>
> *Cc:* ekinnear@apple.com; Lubashev, Igor <ilubashe@akamai.com>om>; Ian 
> Swett <ianswett@google.com>om>; QUIC WG <quic@ietf.org>rg>; Christian 
> Huitema <huitema@huitema.net>et>; Ingemar Johansson S 
> <ingemar.s.johansson@ericsson.com>
> *Subject:* RE: ECN in QUIC connection migration
>
> Hi Mirja,
>
> Why would you like to see the distinction?
>
> It was indeed decided in the side meeting that we should not foresee 
> extra protocol complexity for signalling that the receiver cannot read 
> the ecn bits.
>
> For me the reasons were: It is not a desired situation which will 
> hopefully disappear in the future, and I don't think it is so useful 
> for a sender to know. We now assume reading 00 bits and echo all 
> counters unincremented with 0 values in that case, just as if the path 
> bleaches it. If we make an explicit protocol exchange for it, it feels 
> like it is an extra feature ;-). Also people might start making their 
> congestion control more complex with up to 2 more conditional 
> parameters (sender-cannot-set-ecn and receiver-cannot-read-ecn 
> booleans, next to no-ecn). Not to mention extra interop testcases with 
> such sender's/receiver's extra functionality.
>
> To make your code simple and platform independent, not being able to 
> set/read echo bits should be invisible until the point of the missing 
> socket API call, which will most probably have conditional compiler 
> directives per platform at that point.
>
> So conclusion was: Any higher layer protocol code should not make any 
> distinction between host inability and path bleaching, and we only 
> have functionality for general "end2end ecn inability".
>
> Agreed that the receiver code should be independent from how ecn is 
> used. This is the case now. For the receiver we only increment and 
> echo the counters all the time. That's it, very simple. Only the 
> sender determines how ecn capability is checked and how ecn is used.
>
> Maybe also a good point that the current capability check is the 
> minimal?? And needs some extra checks?
>
> Regards,
>
> Koen.
>
> Outlook voor Android <https://aka.ms/ghei36> downloaden
>
> ------------------------------------------------------------------------
>
> *From:*Ingemar Johansson S <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>
> *Sent:* Wednesday, February 14, 2018 2:08:30 PM
> *To:* Mirja Kühlewind
> *Cc:* De Schepper, Koen (Nokia - BE/Antwerp); ekinnear@apple.com 
> <mailto:ekinnear@apple.com>; Lubashev, Igor; Ian Swett; QUIC WG; 
> Christian Huitema; Ingemar Johansson S
> *Subject:* RE: ECN in QUIC connection migration
>
> Hi
>
> I attach the minutes from the side meeting in Singapore, I believe 
> that they were only posted on the ecn-quic mailing list ☹ .
>
> I agree that the sender/received issues are probably known a-priory 
> and if a sender does know that its OS stack does not have the 
> necessary API support for ECN then it does not make much sense to try it.
>
> /Ingemar
>
>
> > -----Original Message-----
> > From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
> > Sent: den 13 februari 2018 13:29
> > To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>
> > Cc: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-
> <mailto:koen.de_schepper@nokia-%0b>> bell-labs.com>; 
> ekinnear@apple.com <mailto:ekinnear@apple.com>; Lubashev, Igor
> > <ilubashe@akamai.com <mailto:ilubashe@akamai.com>>; Ian Swett 
> <ianswett@google.com <mailto:ianswett@google.com>>; QUIC WG
> > <quic@ietf.org <mailto:quic@ietf.org>>; Christian Huitema 
> <huitema@huitema.net <mailto:huitema@huitema.net>>
> > Subject: Re: ECN in QUIC connection migration
> >
> > Hi Ingemar,
> >
> > see below.
> >
> > > Am 12.02.2018 um 22:11 schrieb Ingemar Johansson S
> > <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>:
> > >
> > > Hi
> > >
> > > Please see inline
> > >
> > > BR
> > > Ingemar
> > >
> > >> -----Original Message-----
> > >> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
> > >> Sent: den 12 februari 2018 16:26
> > >> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>
> > >> Cc: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-
> <mailto:koen.de_schepper@nokia-%0b>> >> bell-labs.com>; 
> ekinnear@apple.com <mailto:ekinnear@apple.com>; Lubashev, Igor
> > >> <ilubashe@akamai.com <mailto:ilubashe@akamai.com>>; Ian Swett 
> <ianswett@google.com <mailto:ianswett@google.com>>; QUIC WG
> > >> <quic@ietf.org <mailto:quic@ietf.org>>; Christian Huitema 
> <huitema@huitema.net <mailto:huitema@huitema.net>>
> > >> Subject: Re: ECN in QUIC connection migration
> > >>
> > >> Hi Ingemar,
> > >>
> > >> two points.
> > >>
> > >> I think it would be good to be as explicit as possible here,
> > >> therefore I would prefer if an endpoint that knows that it can not
> > >> access the ECN IP bits indicates this directly in the handshake. In
> > >> this case no further testing is needed.
> > > <IJ> Yes, this was the original idea, we discussed this at the ECN
> > > QUIC side meeting though and afterwards and as far as I understand it,
> > > there was a consensus that it was unnecessarily complicated. So we
> > > selected a method where we just verify that the ECT(0),ECT(1),CE
> > > counters have reasonable values. If that is not the case, then it is
> > > an indication that either the path or the endpoints do not support
> > > ECN. </IJ>
> >
> > I checked the minutes and could not really find a conclusion there. 
> I agree
> > that there should be no negotiation in the handshake (both endpoint need
> > to implement the ECN logic as part of v1) but what would be helpful 
> is to let
> > the other end know if you already know that you never will be able to
> > provide feedback because you can just not read the bits in the IP 
> header.
> > That is not complicated at all.
> >
> > I think what we need to specify in v1 is a way to provide ECN 
> feedback (or
> > indicate that this is not possible). However, we actually would not 
> need to
> > say anything about how ECN should be used. That’s a sender side 
> decision.
> > As long as you can get feedback, we are flexible in how to use and 
> test ECN in
> > v1 or anytime later. Of course it would be nice to provide some default
> > recommendations with some easy fallback logic but that's the part we can
> > experiment with and change any time without any need to change the
> > protocol spec (as long as ECN feedback is provided, when ECN is used 
> by the
> > sender).
> >
> > >
> > >>
> > >> Further, the path testing itself is independent of QUIC and can be
> > >> used for any protocol. Moreover, it’s more complicated than just
> > >> testing ECT. I guess the general advice here is to remember which
> > >> codepoints were send and compare this to the feedback provided. If
> > >> discrepancies are observed that indicate that a middlebox has changed
> > >> these bits, all packets should be marked as no-ECT (while additional
> > >> testing MAY be performed at any later point in time). However, how
> > >> exactly this testing should look like, should not be specified in 
> this doc
> > (but more generally).
> > > <IJ> It is possible that the actual path verification can be put as a
> > > separate document, not sure if that is accepted by the group or it is
> > > desired with a more elaborate description directly in the QUIC drafts.
> > > As regards to verification, I do believe that the proposed counters
> > > should be able to detect bleaching, accidental remarking from ECT(0)
> > > to ECT(1) and also cases where packets are not CE marked at all
> > > despite other signs of congestion </IJ>
> >
> > We tried to design a more elaborated testing scheme where we would send
> > 3 ECT0, the 3 ECT1, and the 3 CE (which we would of cause not count as a
> > congestion indication as the CE was set by the sender). However, 
> this was
> > mostly to detect black-whaling which is probably less a concern than re-
> > marking. I think today the most common case it that the bits will be 
> set to
> > zeros which is only a problem when there was some CE marking previously
> > on the path that then would get lost. Thus in this case it would be
> > recommend to not use ECN/not set packets as ECT at sender side.
> >
> > However, as I said above, we probably should make recommendation how to
> > use ECN by default but need to leave room for other usage/testing 
> schemes.
> > But after all the receiver being not able to read the bits and 
> remarking on the
> > path are to different things and should not be mixed up. If the 
> sender can
> > not read the bits this is know a priori and there is no ambiguity 
> that needs to
> > be tested. So why not signal it directly. The question if the sender 
> can write
> > ECN does not need to be communicated. There is no difference in the
> > sender decided to not use ECN or it just can not use it; in both 
> cases it is not
> > necessary to send ECN feedback back. Only if a non-non-ECT is 
> observed at
> > the receiver, ECN feedback needs to be provided. However, knowing that
> > the receiver can’t not provide feedback is important because in this 
> case the
> > sender MUST not set the packets as ECT as it will not be able to 
> react to that
> > signal. Testing the path in independent of that.
> >
> > Mirja
> >
> >
> > >
> > >
> > >>
> > >> Mirja
> > >>
> > >>
> > >>> Am 09.02.2018 um 08:34 schrieb Ingemar Johansson S
> > >> <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>:
> > >>>
> > >>> Hi Eric, Koen + others
> > >>>
> > >>> Thanks for the comments and the update. A question to the rest of
> > >>> the
> > >> audience, does the presented text explain sufficiently enough how ECN
> > >> works with connection migration ?
> > >>> https://github.com/quicwg/base-drafts/wiki/ECN-in-QUIC#transport-
> > >> draft-connection-migration
> > >>>
> > >>>
> > >>> /Ingemar
> > >>>
> > >>>
> > >>> From: De Schepper, Koen (Nokia - BE/Antwerp)
> > >> [mailto:koen.de_schepper@nokia-bell-labs.com]
> > >>> Sent: den 2 februari 2018 15:44
> > >>> To: ekinnear@apple.com <mailto:ekinnear@apple.com>; Ingemar 
> Johansson S
> > >> <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>
> > >>> Cc: QUIC WG <quic@ietf.org <mailto:quic@ietf.org>>; Ian Swett 
> <ianswett@google.com <mailto:ianswett@google.com>>;
> > >> Lubashev, Igor <ilubashe@akamai.com 
> <mailto:ilubashe@akamai.com>>; Christian Huitema
> > >> <huitema@huitema.net <mailto:huitema@huitema.net>>
> > >>> Subject: RE: ECN in QUIC connection migration
> > >>>
> > >>> Hi Ingemar,
> > >>>
> > >>> I also read again through the document, and had following comments:
> > >>>
> > >>> I replaced:
> > >>> “The ACK_ECN frame is used to convey ACKs when the ECN capability
> > >> exchange concludes that ECN should be used for the given connection.”
> > >>> Originally it sounds as if the counters are not echoed anymore if
> > >>> the ECN
> > >> capability check fails. I assume this was not the intention, but if
> > >> it is: First, we don’t have a protocol or decision point defined when
> > >> that condition is met (sender should let the receiver know in the
> > >> next packet), and second, I would always send the echo, as we might
> > >> “try again later (with another ECT?)”, or might decide to use the 
> ECN bits
> > differently later.
> > >>>
> > >>> I also changed the text around the connection migration, I think
> > >>> before Eric
> > >> reviewed it. I made it such that every new path creates a new
> > >> connection state with an initialized congestion control state and ECN
> > >> counters (at least ECN counters reset to 0s). Not sure if this
> > >> matches with the other sections in the QUIC draft.
> > >>>
> > >>> I also had the thought whether it was really needed to have the 2
> > >>> cases. I
> > >> agree we should simplify, by just starting over and doing the check
> > >> unconditionally whether it was successful or not in a previous path.
> > >>>
> > >>> Regards,
> > >>> Koen.
> > >>>
> > >>> From: ekinnear@apple.com <mailto:ekinnear@apple.com> 
> [mailto:ekinnear@apple.com]
> > >>> Sent: Friday, February 2, 2018 12:02 AM
> > >>> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>>
> > >>> Cc: QUIC WG <quic@ietf.org <mailto:quic@ietf.org>>; Ian Swett 
> <ianswett@google.com <mailto:ianswett@google.com>>;
> > >> Lubashev, Igor <ilubashe@akamai.com 
> <mailto:ilubashe@akamai.com>>; Christian Huitema
> > >> <huitema@huitema.net <mailto:huitema@huitema.net>>; De Schepper, 
> Koen (Nokia - BE/Antwerp)
> > >> <koen.de_schepper@nokia-bell-labs.com 
> <mailto:koen.de_schepper@nokia-bell-labs.com>>
> > >>> Subject: Re: ECN in QUIC connection migration
> > >>>
> > >>> Hi Ingemar,
> > >>>
> > >>> This generally looks good to me!
> > >>> A few questions and observations:
> > >>>
> > >>> There are two unique cases:
> > >>>
> > >>>                • ECN capability check successful at initial 
> connection setup
> > >>>                • ECN capability check failed at initial connection
> > >>> setup
> > >>>
> > >>> Are these actually any different in terms of action taken by an 
> endpoint?
> > >>> It seems like on the new network path the endpoint would go through
> > >>> the
> > >> same process of: (a) verify ECN capability and if successful to
> > >> whatever degree then (b) run the rest of the ECN machinery as
> > appropriate.
> > >>>
> > >>> In other words, as long as you allow people to start using ECN upon
> > >> migration (in other words, to allow starting it after the initial
> > >> connection establishment), then the specifics for ECN with migration
> > >> is essentially one statement, which is “do the ECN process on each
> > >> new network path that you use”.
> > >>>
> > >>> Connection migration has impact on the number of reported CE marked
> > >> packets. A new connection state with new congestion control state and
> > >> ECN counters is instantiated at the sender and receiver. The ECN
> > >> counters MUST start from zero again.
> > >>>
> > >>> This seems fine to me, it’s inline with the rest of the congestion
> > >> control/loss recovery state that also needs to be reset for the 
> new path.
> > >>>
> > >>> Given that each migration requires at least one exchange of packets
> > >>> at the
> > >> beginning (at the very least for address validation from the side of
> > >> the endpoint that didn’t migrate), it seems like that’s an excellent
> > >> moment to mark the packets and perform the ECN capability detection.
> > >> Those packets are also likely padded, etc. similar to initial packets
> > >> since they’re on a previously unused network path.
> > >>>
> > >>> I think the current proposed requirement of just using the “first”
> > >>> packets
> > >> and marking those is sufficient (and good to avoid any requirement as
> > >> to which frames are contained in those packets), since we are always
> > >> exchanging packets in both directions immediately after migration.
> > >>>
> > >>> This has the added benefit such that any endpoint “probing” possible
> > >> network paths could include the bits necessary to detect ECN
> > >> capability on the path being probed, so it could even know before
> > >> migrating whether or not it would be able to use ECN on that new 
> path.
> > >>>
> > >>> Thanks,
> > >>> Eric
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Feb 1, 2018, at 3:07 AM, Ingemar Johansson S
> > >> <ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>> wrote:
> > >>>
> > >>> Hi
> > >>> I have written up different aspects of connection migration and how
> > >>> it
> > >> plays with ECN.
> > >>> https://github.com/quicwg/base-drafts/wiki/ECN-in-QUIC#transport-
> > >> draft-connection-migration
> > >>>
> > >>> A caveat.. I am a bit unsure about the definition of connection 
> migration
> > .
> > >> My interpretation is that a client moves to a new IP address due to
> > >> e.g. a NAT rebind or switch from WiFi to cellular, right ?. Is a new
> > >> connection instantiated at a connection migration ?
> > >>>
> > >>> /Ingemar
> > >>>
> > >>> ==================================
> > >>> Ingemar Johansson  M.Sc.
> > >>> Master Researcher
> > >>>
> > >>> Ericsson Research
> > >>> Network Protocols & E2E Performance
> > >>> Labratoriegränd 11
> > >>> 971 28, Luleå, Sweden
> > >>> Phone +46-1071 43042
> > >>> SMS/MMS +46-73 078 3289
> > >>> ingemar.s.johansson@ericsson.com 
> <mailto:ingemar.s.johansson@ericsson.com>
> > >>> www.ericsson.com <http://www.ericsson.com>
> > >>>
> > >>>                 You can't start a fire  Worrying 'bout your little
> > >>> world falling apart
> > >>>               Bruce Springsteen
> > >>> ==================================
> > >
>