Re: [iccrg] draft-briscoe-iccrg-prague-congestion-control: CE-marked bytes or packets?

Neal Cardwell <ncardwell@google.com> Thu, 11 August 2022 14:36 UTC

Return-Path: <ncardwell@google.com>
X-Original-To: iccrg@ietfa.amsl.com
Delivered-To: iccrg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 13BD9C159481 for <iccrg@ietfa.amsl.com>; Thu, 11 Aug 2022 07:36:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.606
X-Spam-Level:
X-Spam-Status: No, score=-17.606 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VnSN6njWyA30 for <iccrg@ietfa.amsl.com>; Thu, 11 Aug 2022 07:36:03 -0700 (PDT)
Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com [IPv6:2607:f8b0:4864:20::f35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9C218C14F74D for <iccrg@irtf.org>; Thu, 11 Aug 2022 07:36:03 -0700 (PDT)
Received: by mail-qv1-xf35.google.com with SMTP id d1so13475211qvs.0 for <iccrg@irtf.org>; Thu, 11 Aug 2022 07:36:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=JChqF2ksdS7xwGU2mthcu1ayEcX637R+N3vijZ1vGLI=; b=SLTYZ1LJ5JGj+7m0QpIjh2OeOWwgdfWQw/xvMxaCKaUH0ibvxOcDII3mEwFKQ0dCJG nYVj+gqrvRIuq0vRcAVoGDxIUhau2R+hRLjH9hD18ddgitjARji+djRYTGkotVsGR+Pi 1PMkz6VYuck2rczdpyW8BIAKtvBmmlHAIPUbNot1fmXi+Y6aR9STNnEEgsklmSJXi4wd xK6/AsyZAfi3E3AGaRXEWcJbIFboxCFiHZTgT7huUPAjtycWYw8SyaKLcKWoyx3q7Xg1 zbtZd8ca4nWxu/GlfUrSx5YzkdyugpmETm8U86VSW+oLUpAUWKm+qtYl+Tj8IvpqBLns Wt7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=JChqF2ksdS7xwGU2mthcu1ayEcX637R+N3vijZ1vGLI=; b=YcDIZOg8wke1+M7IxtQyXGCmUhqueAOCKsDvcE5lQipixLhCAdJAajKDStby9PA9Is yharXHVFLb2ilMnryFhKDdwpGMyRjnGNcEyIZJMZsnS4eIYL/9ILKyL9GKvaqF0T120w 777vzAv1AtX92hPqSb2uWCI1bwLcSdbP+A6Vj0uTl9kgyOZ9rqwTbstGX9T7AZTAhJtp D8godGyiOWmP8+TI9emm7LfXGNLxP7FJJhG0+9sdQVCPZaWbHg5C9mjmHE5Ec3j73D4s bg3sZ4K/iWRQ1qZ6N1+I+x5tQhx7d8UXoH0p1JA9Q/sZXr+PrlaMxOku/ZJeaZ+L+rzq YjbQ==
X-Gm-Message-State: ACgBeo0tRDo1HG4eVOAbK9kwdhyv/rLCslffsoMz59mtNzYVb+BCxOXQ GP2aX38nC+XGF21AILJc+CyBC/xpMaDrtbd0ZodYzg==
X-Google-Smtp-Source: AA6agR7S4phB8rDVKWmNonMdbXifZ1JJOZBY4mT8Hp1JMcNFYjvHxXJl85dY9qefp2OUo4eeW+nJp+hKsUkpMF52W6M=
X-Received: by 2002:a05:6214:27e4:b0:476:be6a:91c1 with SMTP id jt4-20020a05621427e400b00476be6a91c1mr28037403qvb.39.1660228562149; Thu, 11 Aug 2022 07:36:02 -0700 (PDT)
MIME-Version: 1.0
References: <CADVnQykxwaqZTGXR-ZMYLEem0rKfAcT7KkHYgsF4dBdWvi2k4w@mail.gmail.com> <cfc8e089-32cc-5b54-9361-889b76ed0d82@lakerest.net>
In-Reply-To: <cfc8e089-32cc-5b54-9361-889b76ed0d82@lakerest.net>
From: Neal Cardwell <ncardwell@google.com>
Date: Thu, 11 Aug 2022 10:35:45 -0400
Message-ID: <CADVnQy=JesSC5ABkArTzgpv01Uvfi08+LqcorFp01oa2bKveXw@mail.gmail.com>
To: rrs <rrs@lakerest.net>
Cc: Bob Briscoe <ietf@bobbriscoe.net>, "Tilmans, Olivier (Nokia - BE/Antwerp)" <olivier.tilmans@nokia-bell-labs.com>, "De Schepper, Koen (Koen)" <koen.de_schepper@nokia.com>, iccrg IRTF list <iccrg@irtf.org>
Content-Type: multipart/alternative; boundary="000000000000b1ed0505e5f8159c"
Archived-At: <https://mailarchive.ietf.org/arch/msg/iccrg/M08-t7KDLEntCESrSWIF1hjqN7c>
Subject: Re: [iccrg] draft-briscoe-iccrg-prague-congestion-control: CE-marked bytes or packets?
X-BeenThere: iccrg@irtf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Discussions of Internet Congestion Control Research Group \(ICCRG\)" <iccrg.irtf.org>
List-Unsubscribe: <https://www.irtf.org/mailman/options/iccrg>, <mailto:iccrg-request@irtf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/iccrg/>
List-Post: <mailto:iccrg@irtf.org>
List-Help: <mailto:iccrg-request@irtf.org?subject=help>
List-Subscribe: <https://www.irtf.org/mailman/listinfo/iccrg>, <mailto:iccrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 Aug 2022 14:36:06 -0000

On Thu, Aug 11, 2022 at 9:30 AM rrs <rrs@lakerest.net> wrote:

> Neal:
>
>
> Just a couple of thoughts on your toy example  .. inline :)
>
> On 8/10/22 6:11 PM, Neal Cardwell wrote:
> > Re:
> >
> https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control-01
> >
> > and the passages:
> >
> > "2.3.2.  Moving Average of ECN Feedback
> > ...it measures the fraction, frac, of ACKed bytes that carried ECN
> > feedback over the previous round trip. ...
> >
> > 2.4.3.  Additive Increase and ECN Feedback
> > ...a Prague CC applies additive increase irrespective of its CWR
> > state, but only for bytes that have been ACK'd without ECN feedback.
> > ...  This approach reduces additive increase as the marking
> > probability increases..."
> >
> > I was curious about the design choice to specify that the algorithm
> > reacts to the fraction of *bytes* that have been CE-marked instead of
> > the fraction of *packets*. IMHO it would be useful for the document to
> > outline the motivation.
> >
> > Apologies if I have missed this in previous e-mail discussions or
> > presentations. I may well have. :-)
> >
> > I can imagine a number of potential reasons why it could be
> > advantageous to react to the fraction of packets CE-marked rather than
> > the fraction of bytes CE-marked:
> >
> > (1) AFAICT byte counters distort the path's ECN marking probability
> > more than using packet counters. For example, suppose we have a round
> > trip with 100 packets sent at roughly uniform intervals across the
> > round trip time:
> >
> > o  99 packets of 1 byte each, all CE-marked
> > o 1 packet of 1000 bytes that was not CE-marked
> >
> > Then the byte-based Prague "frac" ("the fraction, frac, of ACKed bytes
> > that carried ECN feedback over the previous round trip") is:
> >
> >    99 bytes / 1099 bytes ~= .09
> >
> > Whereas the fraction of ACKed packets that carried ECN feedback is:
> >
> >     99 packets / 100 packet = .99
> >
> > So in this toy example there is a >10x difference in the CE "frac"
> > signal depending on whether bytes or packets are counted.
> >
> > And given that these packets were spaced uniformly across the round
> > trip, 99% of the time the bottleneck had excess queuing. This 99%
> > number is well reflected in a packet-based "frac", but seems to imply
> > that the byte-based "frac" approach dramatically underestimates the
> > probability that a packet will encounter excessive queuing, aka the
> > packet CE marking probability.
>
> So lets make this a bit more concrete of a toy. Lets say the bottleneck
> is 20Mbps.
>
> Now your 99 packets of 1 byte (assuming v4 and no timestamps enabled)
> are going to each take 22 microseconds to traverse the bottleneck
> (assuming bytes of 14<enet>, 20 <ip>, 20 <tcp> 1 <data>). Your 1000 byte
> packet is going to take 421 microseconds. So you have a delay through
> the bottleneck for a total of 2,599 microseconds which other packets
> will feel.
>
> Now lets compare that to someone who sent all 1000 byte packets i.e. 421
> x 100. So the delay held by those packets preventing the bottleneck from
> forwarding other packets is 42,100 microseconds or about 1:16.
>
> I suspect when you look at the number of bytes as indicating how long
> your packets take to send through the link thus blocking others the
> situation looks a bit different (at least not near as lopsided as your
> simple example illustrated)
>

I'm not sure what can be concluded from this kind of analysis, since the
delay "which other packets will feel" is entirely dependent upon whether
those other packets arrive at the bottleneck soon enough after the packets
in question (with an inter-arrival time less than the serialization time of
the packet in question, so that the other packets are likely to be queued)?

neal