Re: [tcpm] [Fwd: New Version Notification for draft-gomez-tcpm-ack-pull-00.txt]

Yuchung Cheng <ycheng@google.com> Wed, 17 July 2019 17:32 UTC

Return-Path: <ycheng@google.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9FACC120833 for <tcpm@ietfa.amsl.com>; Wed, 17 Jul 2019 10:32:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.5
X-Spam-Level:
X-Spam-Status: No, score=-17.5 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5-Qk9df7FRfB for <tcpm@ietfa.amsl.com>; Wed, 17 Jul 2019 10:32:07 -0700 (PDT)
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D9F3612081A for <tcpm@ietf.org>; Wed, 17 Jul 2019 10:32:06 -0700 (PDT)
Received: by mail-wr1-x42f.google.com with SMTP id x4so25694831wrt.6 for <tcpm@ietf.org>; Wed, 17 Jul 2019 10:32:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=hGH6eXtAXehFroDCW8VbmTbUW3fm38kxM7vJiUUjtho=; b=MiNTTjdz+UGpEOu19GUaB1NhIbEk0ogNPW8G53wwnJk8IatECQabJiSC3kNXnsTYkT V5AkOhwLLEt3/2uPlOK+aGNrd/aJ1aRpdpvykHAtSmeh1BUQQkOvjY7QScPh3jB5yAJx fxjE0+zkpmvYWmgfAckpeORqsm2ldDnpHELVnkr0TSAAFsPYLdqoasYLN4FHcblbQJbt 7Z1rsblG+Iq/EHoaXiwATA1zxdAgHjXID9fcdNZk7p/rh1YaAc+3CtGZXx9QjB4YYJGZ xhJI/u3/lciM2xsaatRAVL58s/SEg3Rpdqn9Flg5HVwLg0MWl0gAm+Gkd2h4UfoCUNea YALA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=hGH6eXtAXehFroDCW8VbmTbUW3fm38kxM7vJiUUjtho=; b=norueFZdU/wv904F1FirtIIkx1GJ969PckWX6tO6/ZBaLv4fripkLn2ST6Ig7QTTVU FZ7yEY0+McTBVhLTv7YOWZ8I/Btcah3YJ/8Yrb4BLD8W+n2GWsBStLtag0KhvZfz7kkM w/gNIocha/qUgo8nXcT6xxL/KzUbjLkbRtZZYTzQeq03+kCoVDWna2kQDXqM714w8IK1 arD4vvxk4vcNNfQlh9wweH3Ygan9s1UaSJwjPgSGRZJ8fE9fBIUtphVoRdd6aY1FUcdU Lq4aGaP3S5ebfGUteCJQ4yfI9kWs6Gvdgdx3DSHSc0it4hNqwa1UBGfVNR098phmBMbo jPqg==
X-Gm-Message-State: APjAAAWOxNMILatLALhb58ShfKDle++utOJMKCGqqx6ki/VbGd54vQgb jaSoR0lvtcHUj0WXYsEeQtffxc/XSVJSqrjCizJIOQ==
X-Google-Smtp-Source: APXvYqyApao0M0UEGI30Sgv3CbWPBzx7MvkwNYzyBrQU3+4CtFxjyOxe4vUD2gjU+XNB+fXd8iDLbj11uicp1DkM0rA=
X-Received: by 2002:adf:cf0d:: with SMTP id o13mr31646558wrj.291.1563384724888; Wed, 17 Jul 2019 10:32:04 -0700 (PDT)
MIME-Version: 1.0
References: <6EC6417807D9754DA64F3087E2E2E03E2D3B2F41@rznt8114.rznt.rzdir.fht-esslingen.de> <437D3782-04FB-4D5B-9A38-4BA9DB7C2ECF@lurchi.franken.de> <MW2PR2101MB1049CF516B565A37F13FD0F4B6C90@MW2PR2101MB1049.namprd21.prod.outlook.com> <CADVnQy=1Q-iZQxDAvKx2zeQYF=7Ss4JoSJxyvm1xw-1bzmmx9g@mail.gmail.com>
In-Reply-To: <CADVnQy=1Q-iZQxDAvKx2zeQYF=7Ss4JoSJxyvm1xw-1bzmmx9g@mail.gmail.com>
From: Yuchung Cheng <ycheng@google.com>
Date: Wed, 17 Jul 2019 10:31:27 -0700
Message-ID: <CAK6E8=fb+2xRMv9nB+-F97ptWpHUxp7nDgW+sG67gh_bOK2-fw@mail.gmail.com>
To: Neal Cardwell <ncardwell=40google.com@dmarc.ietf.org>
Cc: Praveen Balasubramanian <pravb=40microsoft.com@dmarc.ietf.org>, Michael Tuexen <michael.tuexen@lurchi.franken.de>, "tcpm@ietf.org Extensions" <tcpm@ietf.org>, "jon.crowcroft@cl.cam.ac.uk" <jon.crowcroft@cl.cam.ac.uk>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/kOcUI7YsbJc-Rs61g5fDmzbhSj4>
Subject: Re: [tcpm] [Fwd: New Version Notification for draft-gomez-tcpm-ack-pull-00.txt]
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Jul 2019 17:32:10 -0000

On Wed, Jul 17, 2019 at 9:53 AM Neal Cardwell
<ncardwell=40google.com@dmarc.ietf.org> wrote:
>
> Indeed, Linux also does not generate an immediate ACK upon receiving
> PSH, and also sets PSH on sendmsg() boundaries.
>
> One perspective I am not sure has been discussed in this thread: Those
> behaviors to generate PSH on sendmsg() boundaries are widely deployed.
> There are lots of workloads that have a large number of small packets
> with PSH set (RPC, web, ...). If we change the semantics of PSH to
> cause an immediate ACK, then we will effectively turn off delayed ACK
> acks for many RPC and web workloads. This will increase CPU and
> network loads for such workloads. For some RPC workloads it would
> double the number of packets  sent by servers (the server would now
> send 1 ACK for every request, and then 1 response packet, instead of
> piggybacking ACKs and replies). That cost has to be weighed against
> any latency advantages.
+1 on similar concern.

It's worth noting a recent issue we discovered while testing BBRv2
which uses DCTCP-ECN protocol. The Linux delayed-ACK implementation does not
always ACK every other data packet. It may delay sending an ACK until
the available receive window has at least matched or grown beyond to
the previous window announced. Under heavy (data-path) network
congestion, a slower receiver could frequently delay the ACKs when
most data packets have CE marks.

This ACK delay causes the sender to further slow down in additional to
the CE events being returned since both BBRv2 and DCTCP are still
ACK-clocked. The large RPC is received more slowly (compared to a
policy that acks immediately every other packet). The application
goodput is zero until the RPC is finally completely received. This
cause much longer RPC tail latency visible to the users. So this
design causes all RPCs to progress more slowly in parallel, vs fewer
RPCs to progress more quickly in serial.

Arguably this is an implementation / optimization issue, but I think
it's very relevant to this thread. Neal may talk about that in the
upcoming ICCRG bbr.v2 talk.


>
> Using a new bit would have the advantage of allowing senders to only
> force more ACKs when their available cwnd or rwnd is low enough that
> they need an immediate ACK to allow good performance.
>
> Just a thought.
>
> cheers,
> neal
>
>
>
> On Wed, Jul 17, 2019 at 12:21 PM Praveen Balasubramanian
> <pravb=40microsoft.com@dmarc.ietf.org> wrote:
> >
> > Windows TCP on sender side will set PSH at the message boundary i.e. when it finishes generating sends as part of a posted send() API call or equivalent. On receiver side however it does not generated immediate ACK upon receiving PSH. IIRC Linux also sets PSH on sendmsg() boundary. Not sure about other implementations. If most major implementations are doing this then generating immediate ACK upon PSH will be an improvement.
> >
> > I do think reusing PSH to require eliciting ACKs is a reasonable alternative than using up a reserved bit. One concern with both the original draft and the PSH proposal is that we'll need clear recommendations on when a receiver will request this. Otherwise (selfish) receivers will always request it and senders will have to build complicated heuristics to detect such behavior and fall back to doing delayed ACKs.
> >
> > -----Original Message-----
> > From: tcpm <tcpm-bounces@ietf.org> On Behalf Of Michael Tuexen
> > Sent: Wednesday, July 17, 2019 6:41 AM
> > To: Scharf, Michael <Michael.Scharf@hs-esslingen.de>
> > Cc: tcpm@ietf.org Extensions <tcpm@ietf.org>; jon.crowcroft@cl.cam.ac.uk
> > Subject: Re: [tcpm] [Fwd: New Version Notification for draft-gomez-tcpm-ack-pull-00.txt]
> >
> > > On 17. Jul 2019, at 14:44, Scharf, Michael <Michael.Scharf@hs-esslingen.de> wrote:
> > >
> > > Is my understanding correct that this could be formalized as follows:
> > >
> > >   A TCP MAY not delay ACKs for data segments with the PSH flag.
> > I think this is what Carsten was referring to.
> >
> > My point was: Is this already deployed?
> >
> > The reason I'm asking is that I have heard several time that the semantic of the PSH bit is to send out an ACK immediately. I could not find the source of this statement...
> >
> > Best regards
> > Michael
> > >
> > > If that was the intention, I believe that the wording of RFC 1122 (and
> > > draft-ietf-tcpm-rfc793bis) would allow such a receiver-side heuristic
> > > already. Delayed ACKs are a SHOULD in RFC 1122 and the exact logic is
> > > not specified. Thus, taking the PSH flag into account inside a
> > > receiver-side delayed ACK heuristic may not even be a change of the
> > > TCP semantics…
> > >
> > > Michael
> > >
> > >
> > > Von: Carsten Bormann
> > > Gesendet: Mittwoch, 17. Juli 2019 09:56
> > > An: Yoshifumi Nishida
> > > Cc: jon.crowcroft@cl.cam.ac.uk; tcpm@ietf.org Extensions
> > > Betreff: Re: [tcpm] [Fwd: New Version Notification for
> > > draft-gomez-tcpm-ack-pull-00.txt]
> > >
> > > On Jul 17, 2019, at 08:58, Yoshifumi Nishida <nsd.ietf@gmail.com> wrote:
> > > >
> > > >  using a reserved flag is a bit expensive
> > >
> > > The option could simply redefine existing PSH as having the AKP semantics.
> > > (And possibly all packets having what used to be the PSH semantics.)
> > > They are close enough anyway…
> > >
> > > Grüße, Carsten
> > >
> > > _______________________________________________
> > > tcpm mailing list
> > > tcpm@ietf.org
> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> > > ietf.org%2Fmailman%2Flistinfo%2Ftcpm&amp;data=02%7C01%7Cpravb%40micros
> > > oft.com%7Cbfdb4e02ba524e0c707408d70abc6d1e%7C72f988bf86f141af91ab2d7cd
> > > 011db47%7C1%7C0%7C636989676733913763&amp;sdata=YxD0gSXU6owQeXgKqYp462j
> > > mS9ihLrI5RDocqpS8CfQ%3D&amp;reserved=0
> > >
> > > _______________________________________________
> > > tcpm mailing list
> > > tcpm@ietf.org
> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> > > ietf.org%2Fmailman%2Flistinfo%2Ftcpm&amp;data=02%7C01%7Cpravb%40micros
> > > oft.com%7Cbfdb4e02ba524e0c707408d70abc6d1e%7C72f988bf86f141af91ab2d7cd
> > > 011db47%7C1%7C0%7C636989676733913763&amp;sdata=YxD0gSXU6owQeXgKqYp462j
> > > mS9ihLrI5RDocqpS8CfQ%3D&amp;reserved=0
> >
> > _______________________________________________
> > tcpm mailing list
> > tcpm@ietf.org
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&amp;data=02%7C01%7Cpravb%40microsoft.com%7Cbfdb4e02ba524e0c707408d70abc6d1e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636989676733923755&amp;sdata=DLucPcy4x04sZslmF8Egm9BiggxVxu5gCuW%2F8T70yQo%3D&amp;reserved=0
> > _______________________________________________
> > tcpm mailing list
> > tcpm@ietf.org
> > https://www.ietf.org/mailman/listinfo/tcpm
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm