Re: ECN in QUIC

"Brian Trammell (IETF)" <ietf@trammell.ch> Wed, 20 September 2017 14:57 UTC

Return-Path: <ietf@trammell.ch>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E6FF13207A for <quic@ietfa.amsl.com>; Wed, 20 Sep 2017 07:57:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gMRI6WAiQy1C for <quic@ietfa.amsl.com>; Wed, 20 Sep 2017 07:57:51 -0700 (PDT)
Received: from capri.iway.ch (capri.iway.ch [IPv6:2001:8e0:40:325::45]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DAB72124F57 for <quic@ietf.org>; Wed, 20 Sep 2017 07:57:50 -0700 (PDT)
Received: from gozo.iway.ch (localhost [127.0.0.1]) by localhost (Postfix) with ESMTP id 22EB2340A45; Wed, 20 Sep 2017 16:57:49 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1]) by localhost (ACF/18338.22864); Wed, 20 Sep 2017 16:57:48 +0200 (CEST)
Received: from switchplus-mail.ch (switchplus-mail.ch [212.25.8.236]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by gozo.iway.ch (Postfix) with ESMTPS; Wed, 20 Sep 2017 16:57:44 +0200 (CEST)
Received: from [195.176.111.14] (account ietf@trammell.ch HELO public-docking-cx-0363.ethz.ch) by switchplus-mail.ch (CommuniGate Pro SMTP 6.1.18) with ESMTPSA id 30266344; Wed, 20 Sep 2017 16:57:44 +0200
From: "Brian Trammell (IETF)" <ietf@trammell.ch>
Message-Id: <08876D99-50DC-466A-ADB1-53A06C921664@trammell.ch>
Content-Type: multipart/signed; boundary="Apple-Mail=_71311556-3F32-4AAA-9359-6F94364BA640"; protocol="application/pgp-signature"; micalg="pgp-sha512"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: ECN in QUIC
Date: Wed, 20 Sep 2017 16:57:44 +0200
In-Reply-To: <59C27B81.2020609@erg.abdn.ac.uk>
Cc: Frank Kastenholz <fkastenholz@google.com>, Magnus Westerlund <magnus.westerlund@ericsson.com>, "Bob Briscoe (research@bobbriscoe.net)" <research@bobbriscoe.net>, Praveen Balasubramanian <pravb@microsoft.com>, Ian Swett <ianswett@google.com>, Ingemar Johansson S <ingemar.s.johansson@ericsson.com>, "Eggert, Lars (lars@netapp.com)" <lars@netapp.com>, marcelo bagnulo braun <marcelo@it.uc3m.es>, Mirja Kühlewind <mirja.kuehlewind@tik.ee.ethz.ch>, Piers O'Hanlon <piers.ohanlon@cs.ox.ac.uk>, QUIC IETF mailing list <quic@ietf.org>, Colin Perkins <csp@csperkins.org>, "De Schepper, Koen (Nokia - BE/Antwerp)" <koen.de_schepper@nokia-bell-labs.com>
To: Gorry Fairhurst <gorry@erg.abdn.ac.uk>
References: <AM3PR07MB34048E916A7B18695313171C2820@AM3PR07MB340.eurprd07.prod.outlook.com> <8E718F38-88DB-4E1D-BC1B-1C0F0E9E5C34@csperkins.org> <DB4PR07MB348F7E2CFAF1574F838ED8CC2610@DB4PR07MB348.eurprd07.prod.outlook.com> <CAKcm_gMbxEMcq4UaQ9R_iGgXSpKJHzA67VMg2ZXDg2K1OhdUNA@mail.gmail.com> <DB4PR07MB348D95672E48A338758E32DC2610@DB4PR07MB348.eurprd07.prod.outlook.com> <CAD3dRjoz7XNPZB3HY2tHL6iJvqKf=0EgVCmc20nFkB0F914jkA@mail.gmail.com> <59C27B81.2020609@erg.abdn.ac.uk>
X-Mailer: Apple Mail (2.3273)
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/wjPbDigQEH_EL9M4JEiuBwYscOQ>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Sep 2017 14:57:55 -0000

> On 20 Sep 2017, at 16:30, Gorry Fairhurst <gorry@erg.abdn.ac.uk> wrote:
> 
> On 20/09/2017, 15:21, Frank Kastenholz wrote:
>> Hi
>> 
>> I wonder if the ACK can be kept simple by including only the ECN information from the most recently received packet, or maybe N (where N is fairly small) most recently received packets? This would reduce the amount of information to be carried as well as the complexity of including it.
>> 
>> I confess that I am not familiar with the various congestion control algorithms ... but I would imagine that more recent ECN information would be more useful than older ECN info. No?
> No - there are two styles of ECN CC. The more recent requires accurate feedback of the ECN marks - think of this more as understanding the "rate of marking from the bottleneck" and in some case "was this (test) packet marked". This requires per-packet ECN info - at least one bit per received packet, and preferably 2 bits.

I'll also note that AccECN (draft-ietf-tcpm-accurate-ecn) would seem to be far easier to add to QUIC than to TCP. In the TCP case, it needs to reallocate a flag bit and add an option, both of which need to be defensively specified against path interference, *all* of which simply go away when the information is carried in encrypted QUIC ACK frames.

AccECN also shows the way to handle the recency of information issues raised above: the counters echoed are all the least significant N bits of a cumulative counter, which is both recent and robust against ACK loss.

Cheers,

Brian

> Gorry
> 
>> 
>> Frank Kastenholz
>> 
>> On Wed, Sep 20, 2017 at 9:27 AM, Ingemar Johansson S <ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com>> wrote:
>> 
>>    Thanks Ian for the prompt response.
>> 
>>    I must admit that I have perhaps overlooked possible combos of
>>    loss and ECN, the price you pay for working too much with
>>    simulators. Your arguments are very strong indeed.
>> 
>>    In the SCReAM congestion control (RMCAT work) the ECN and loss
>>    info comes in the same feedback and the handling of loss and ECN
>>    is straightforward there. Personally I would prefer ECN in the ACK
>>    frames and the earlier drafts suggested formats for that, I
>>    however got the impression that there is a certain resistance
>>    against additional things in the ACK frame ?.
>> 
>>    /Ingemar
>> 
>>    *From:* Ian Swett [mailto:ianswett@google.com
>>    <mailto:ianswett@google.com>]
>>    *Sent:* den 20 september 2017 14:41
>>    *To:* Ingemar Johansson S <ingemar.s.johansson@ericsson.com
>>    <mailto:ingemar.s.johansson@ericsson.com>>
>>    *Cc:* Colin Perkins <csp@csperkins.org
>>    <mailto:csp@csperkins.org>>; Magnus Westerlund
>>    <magnus.westerlund@ericsson.com
>>    <mailto:magnus.westerlund@ericsson.com>>; Bob Briscoe
>>    (research@bobbriscoe.net <mailto:research@bobbriscoe.net>)
>>    <research@bobbriscoe.net <mailto:research@bobbriscoe.net>>;
>>    Praveen Balasubramanian <pravb@microsoft.com
>>    <mailto:pravb@microsoft.com>>; Eggert, Lars (lars@netapp.com
>>    <mailto:lars@netapp.com>) <lars@netapp.com
>>    <mailto:lars@netapp.com>>; marcelo bagnulo braun
>>    <marcelo@it.uc3m.es <mailto:marcelo@it.uc3m.es>>; Mirja Kühlewind
>>    <mirja.kuehlewind@tik.ee.ethz.ch
>>    <mailto:mirja.kuehlewind@tik.ee.ethz.ch>>; Piers O'Hanlon
>>    <piers.ohanlon@cs.ox.ac.uk <mailto:piers.ohanlon@cs.ox.ac.uk>>;
>>    QUIC IETF mailing list <quic@ietf.org <mailto:quic@ietf.org>>; De
>>    Schepper, Koen (Nokia - BE/Antwerp)
>>    <koen.de_schepper@nokia-bell-labs.com
>>    <mailto:koen.de_schepper@nokia-bell-labs.com>>
>> 
>> 
>>    *Subject:* Re: ECN in QUIC
>> 
>>    On Wed, Sep 20, 2017 at 4:39 AM, Ingemar Johansson S
>>    <ingemar.s.johansson@ericsson.com
>>    <mailto:ingemar.s.johansson@ericsson.com>> wrote:
>> 
>>        Hi
>> 
>>        And sorry for the sloth-ish response (you should see me type         a text message..).
>> 
>>        To summarize the two main questions
>> 
>>         1. ECN in ACK frame or in a separate frame : Given the recent
>>            discussion on the list and fear that the ACK frame is
>>            already enough complex, it seems unwise to try to cram in
>>            additional info in the ACK frames. So my personal feeling
>>            is that we should use a separate frame type for ECN , but
>>            this frame type must be a MUST to implement and support !
>>            Comments are very welcome.
>> 
>>    I was neutral on this before, but I started thinking about
>>    implementing this, and separating the frames makes it more
>>    complex, and potentially really, really difficult to get the
>>    implementation correct.  As an example, if the frames were
>>    accidentally sent in two separate packets, I would count the
>>    packet as acked, and potentially increase the congestion window
>>    and other state variables.  I might even send an extra packet
>>    based on this information.  Then in the next packet I'd realize
>>    that packet had an ECN marking, and actually should have been
>>    treated as lost from a congestion controller perspective.  In
>>    order to do this correctly, I'd have to undo a bunch of things.     My experience with undo is that it's terribly error prone, and I
>>    would never recommend it.
>> 
>>    Alternately, the ack could get lost, and the next packet would
>>    convey ECN marking information for packets that hadn't been
>>    acknowledged yet.  Which is just weird.
>> 
>>    If we require support for ECN, then the implementation has a
>>    certain extra level of complexity.  As illustrated above, putting
>>    it in one frame is actually simpler overall if the congestion
>>    controller needs to use all the information at once to be
>>    implemented correctly.  This same logic applies to timestamps as
>>    well, in my opinion.
>> 
>>        1.
>>         2. As regards to the level of detail : There are pros and
>>            cons with both alternatives (marked bytes or detailed
>>            indication). I would like to hear the opinions from others
>>            as well on this matter .
>> 
>>        /Ingemar
>> 
>>        *From:* Colin Perkins [mailto:csp@csperkins.org
>>        <mailto:csp@csperkins.org>]
>>        *Sent:* den 16 augusti 2017 12:53
>>        *To:* Ingemar Johansson S <ingemar.s.johansson@ericsson.com
>>        <mailto:ingemar.s.johansson@ericsson.com>>
>>        *Cc:* QUIC IETF mailing list <quic@ietf.org
>>        <mailto:quic@ietf.org>>; Magnus Westerlund
>>        <magnus.westerlund@ericsson.com
>>        <mailto:magnus.westerlund@ericsson.com>>; Bob Briscoe
>>        (research@bobbriscoe.net <mailto:research@bobbriscoe.net>)
>>        <research@bobbriscoe.net <mailto:research@bobbriscoe.net>>;
>>        Praveen Balasubramanian <pravb@microsoft.com
>>        <mailto:pravb@microsoft.com>>; Eggert, Lars (lars@netapp.com
>>        <mailto:lars@netapp.com>) <lars@netapp.com
>>        <mailto:lars@netapp.com>>; marcelo bagnulo braun
>>        <marcelo@it.uc3m.es <mailto:marcelo@it.uc3m.es>>; Mirja
>>        Kühlewind <mirja.kuehlewind@tik.ee.ethz.ch
>>        <mailto:mirja.kuehlewind@tik.ee.ethz.ch>>; Piers O'Hanlon
>>        <piers.ohanlon@cs.ox.ac.uk
>>        <mailto:piers.ohanlon@cs.ox.ac.uk>>; De Schepper, Koen (Nokia
>>        - BE/Antwerp) <koen.de_schepper@nokia-bell-labs.com
>>        <mailto:koen.de_schepper@nokia-bell-labs.com>>
>>        *Subject:* Re: ECN in QUIC
>> 
>>            On 16 Aug 2017, at 10:33, Ingemar Johansson S
>>            <ingemar.s.johansson@ericsson.com
>>            <mailto:ingemar.s.johansson@ericsson.com>> wrote:
>> 
>>            Hi
>> 
>>            Finally back from vacation, and very grateful for the
>>            support to continue the work to add ECN in QUIC.
>> 
>>            Just to recap.. there were two main topics raised at the
>>            meeting
>> 
>>            1) ECN info in ACK frame or in dedicated frame : There
>>            were concerns about adding extra complexity in an already
>>            potentially complex ACK frame, one can have differing
>>            opinions about the complexity but can understand the
>>            concerns. As far as I am concerned, a separate frame type
>>            for ECN is possible, possibly one need to add information
>>            about the amount of not-ECT marked packets as well to keep
>>            the signaling robust, this needs further investigation
>>            though. One concern with a separate ECN frame is that it
>>            becomes a not-implemented or optional feature, is there
>>            any reason to be worried about this ?
>> 
>>            2) More detailed ECN information : Earlier versions of the
>>            ECN in QUIC draft
>>            (seehttps://tools.ietf.org/id/draft-johansson-quic-ecn-01.txt
>>            <https://tools.ietf.org/id/draft-johansson-quic-ecn-01.txt>)
>>            provided with examples. We (Myself, Koen, Mirja and
>>            Praveen) discussed this and we could not come up with any
>>            use case where it is beneficial to know exactly how each
>>            packet is ECN marked. I know that this kind of detailed
>>            ECN information is suggested for the generic feedback for
>>            RMCAT and I personally have a problem to see the gain with
>>            the detailed ECN information also here. Input from others
>>            is very welcome here.
>> 
>>        For the RMCAT format, we wanted per-packet loss and timing
>>        information, and it was as easy to feedback per-packet ECN
>>        information along with it as to design something different.
>> 
>>        A benefit of per-packet ECN marking could be to allow a
>>        congestion controller that reacted differently to bursts of
>>        consecutive ECN marks than it did to isolated ECN marks, given
>>        the same fraction of marked packets (i.e., that reacted to ECN
>>        marking events rather than ECN marking rate, like how TCP
>>        responds to loss events). I don’t think we have such a thing,
>>        but certainly in the context of RMCAT where we’re
>>        experimenting with novel congestion control schemes for
>>        traffic that has very different characteristics to traditional
>>        bulk flows, it might be plausible. Per-packet marking
>>        information is also useful for troubleshooting.
>> 
>>        Certainly we need to know number of NotECT, ECT(0), ECT(1),
>>        and ECN-CE marks since the last report, but I guess that’s
>>        already possible.
>> 
>>        Cheers,
>> 
>>        Colin
>> 
>>            There are a consequences with detailed ECN marking
>>            information.
>> 
>>            a) necessary to correlate with the list of transmitted
>>            packets, this increases amount of code on sender side, not
>>            sure of that is a large concern as lookup is anyway needed
>>            to process incoming ACKs
>> 
>>            b) necessary to embed ECN information in ACK frame ?, at
>>            least this was my conclusion when I devised the detailed
>>            ECN marking info in the 01 version of the draft.
>> 
>>            Comments are welcome
>> 
>>            /Ingemar
>> 
>>            ==================================
>> 
>>            Ingemar Johansson  M.Sc.
>> 
>>            Master Researcher
>> 
>>            Ericsson AB
>> 
>>            Wireless Access Networks
>> 
>>            Labratoriegränd 11
>> 
>>            971 28, Luleå, Sweden
>> 
>>            Phone +46-1071 43042
>> 
>>            SMS/MMS +46-73 078 3289 <tel:+46%2073%20078%2032%2089>
>> 
>>            ingemar.s.johansson@ericsson.com
>>            <mailto:ingemar.s.johansson@ericsson.com>
>> 
>>            www.ericsson.com <http://www.ericsson.com>
>> 
>>            A mistake is to commit a misunderstanding
>>                                 Bob Dylan
>>            ==================================
>> 
>> 
>> 
>>        --         Colin Perkins
>>        https://csperkins.org/
>> 
>> 
>> 
>> 
>> --
>> Thanks
>> Frank Kastenholz
>> 
>