Re: [tsvwg] RDMA Support by UDP FRAG Option

"C. M. Heard" <heard@pobox.com> Sat, 19 June 2021 23:40 UTC

Return-Path: <heard@pobox.com>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E5563A0D40 for <tsvwg@ietfa.amsl.com>; Sat, 19 Jun 2021 16:40:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.199
X-Spam-Level:
X-Spam-Status: No, score=-0.199 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=pobox.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qge_QSbZin1J for <tsvwg@ietfa.amsl.com>; Sat, 19 Jun 2021 16:40:14 -0700 (PDT)
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AACC43A0D3F for <tsvwg@ietf.org>; Sat, 19 Jun 2021 16:40:14 -0700 (PDT)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id AB4CC1444CE for <tsvwg@ietf.org>; Sat, 19 Jun 2021 19:40:07 -0400 (EDT) (envelope-from heard@pobox.com)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h= mime-version:references:in-reply-to:from:date:message-id:subject :to:cc:content-type; s=sasl; bh=1VYNwaZBoN9TBrV9e7+dO7jIXv6j9ttS c5olKJNrIe8=; b=GlrO+Io5o1lyhb1nx9JTmD78E9EgoaheXBb7Q5r/Am+93U1i jI7twhI6MJ8PrJQef3AMVOt1/MnNA9hqML44XNzxHJuKVjNULL4APxTbchsQ9th/ jKOi5Ipc4nl1E9ODsoJVpLTcFNleN2b0qoAzzwzfkXhocDrHI4qOCEbV5AI=
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id A07F31444CD for <tsvwg@ietf.org>; Sat, 19 Jun 2021 19:40:07 -0400 (EDT) (envelope-from heard@pobox.com)
Received: from mail-il1-f171.google.com (unknown [209.85.166.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 3C4091444C8 for <tsvwg@ietf.org>; Sat, 19 Jun 2021 19:40:05 -0400 (EDT) (envelope-from heard@pobox.com)
Received: by mail-il1-f171.google.com with SMTP id u2so1494296ilk.7 for <tsvwg@ietf.org>; Sat, 19 Jun 2021 16:40:05 -0700 (PDT)
X-Gm-Message-State: AOAM530NCXC8G5e2DuiA9TOiYBLcZuMBmE9D8gx1p+nwyYx8MwNKNaVm EJEO+BmQtYQztIEZF3W8GVsX7LLLlVDf9sN20ao=
X-Google-Smtp-Source: ABdhPJxE+TN1Hgkv/O+xSNbgmyslqd20U7qd1wDeSpmeFT64VHYbKTMVDNhU2+oNl4+uY9F1PlS1AMXvQRGuglMZpsk=
X-Received: by 2002:a05:6e02:ee6:: with SMTP id j6mr7115872ilk.143.1624146003943; Sat, 19 Jun 2021 16:40:03 -0700 (PDT)
MIME-Version: 1.0
References: <CACL_3VEyLdQZ-3hvzXxyA8ehtWs2hXESZ2OqyAx+BeSg85+-cA@mail.gmail.com> <CACL_3VFE4TjKvmkfZjvNpWo6vVfKjz5w85=Q+yqnYZKcwbYLmQ@mail.gmail.com> <63FFC34B-2179-47F1-B325-21CAC3D1543A@strayalpha.com> <CACL_3VHTfxWaBj7TFEmBXBqovrrAj7XuFEZFUag_iBHr3Hx09g@mail.gmail.com> <0EBFC9B0-591A-4860-B327-6E617B83F4D1@strayalpha.com> <CALx6S34pT81TbfQDk2vKF8wBrXL312As79K=rEzUQ3Lmg7UvpA@mail.gmail.com> <7C51D926-9DBB-41F5-93B2-10F716F672B1@strayalpha.com> <CALx6S37uN8TsXQZ3cv5jmxwxSyBRjK=-GQ_MsWxPWSs21XoGHw@mail.gmail.com> <CACL_3VEx7+VnLz7OLdXyhZU41e+-oBz3dc8JdMV_7pLMfic6=w@mail.gmail.com> <fcc8762f-c042-7999-d2e4-f28384950a19@erg.abdn.ac.uk> <CALx6S36sWGcZmFpAhF4DfOMyf6Z0w5F9bemNfeM1yWV-r0M+BA@mail.gmail.com> <8af3abf9-943f-13c1-e239-5efca27cf68c@erg.abdn.ac.uk> <CACL_3VHdyLAmzMbWsTVfJD+4tTzsMvcTzKS1B1CAdZ3k5U957g@mail.gmail.com> <CALx6S34DUrUBYd94LPPg4Hgh0FnZYZjZ4eKEYuaxb-7zbzb=pQ@mail.gmail.com> <CACL_3VEq9R=HmWXGbu_zcrgWfG0=q0z+HWM3cQ9Vh68hTCUR-w@mail.gmail.com> <CALx6S35bdGwY8FagGn8x5CaO4O3zW3U+NnB5ejC7bB6BHsXtJg@mail.gmail.com>
In-Reply-To: <CALx6S35bdGwY8FagGn8x5CaO4O3zW3U+NnB5ejC7bB6BHsXtJg@mail.gmail.com>
From: "C. M. Heard" <heard@pobox.com>
Date: Sat, 19 Jun 2021 16:39:52 -0700
X-Gmail-Original-Message-ID: <CACL_3VFwUJzT7uiXh33gBffboqqb51uFWJAEh290SsD0=aAzaQ@mail.gmail.com>
Message-ID: <CACL_3VFwUJzT7uiXh33gBffboqqb51uFWJAEh290SsD0=aAzaQ@mail.gmail.com>
To: Tom Herbert <tom@herbertland.com>
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Joseph Touch <touch@strayalpha.com>, TSVWG <tsvwg@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000a0e78c05c526f554"
X-Pobox-Relay-ID: AC931B7A-D157-11EB-9896-FA9E2DDBB1FC-06080547!pb-smtp21.pobox.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/dmZIWmyBBjUShNwfPVvxQipdc88>
Subject: Re: [tsvwg] RDMA Support by UDP FRAG Option
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 19 Jun 2021 23:40:19 -0000

On Sat, Jun 19, 2021 at 11:09 AM Tom Herbert wrote:

> On Sat, Jun 19, 2021 at 11:01 AM C. M. Heard wrote:
> > Is there a use case for employing UDP options with GRE/UDP? Or GUE?
>
> We can divide UDP encapsulations into two flavors. Those that are
> extensible and contain their own TLVs or equivalent method to encode
> ancillary data, and those that don't have that. GUE and Geneve are
> examples of UDP encapsulation protocols that have their own
> extensibility mechanisms and hence wouldn't be used with UDP options,
> GRE/UDP and VXLAN don't have extensibility so UDP options could be
> useful with those protocols.
>

OK, thanks. To loop back to your original point:


>  The stack will attempt to offload the innermost checksum which is the

TCP checksum. The TCP checksum is the one offloaded regardless of
> whether or not the outer UDP checksum is zero (if it's non-zero then
> the stack would set it using local checksum offload (LCO)). The
> offloaded TCP checksum computation would start the computation at the
> first byte of the TCP header through the end of the whole packet. So
> unless the surplus area is properly checksummed then the computed TCP
> checksum will be invalid and the packet will be dropped at the
> receiver. This is not just a problem for offload for offload, I
> believe that this wouldn't work properly in existing software stacks
> without some major changes.
>
> So to make all uses of transport checksum computation and offload
> reasonably robust, when the UDP surplus area is being used both the
> UDP checksum and checksum over the surplus area MUST always be set.
>
> FYI, here is some nice background checksum offload is
> https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html
>
>
This was also helpful:
https://www.ietf.org/proceedings/95/slides/slides-95-nvo3-3.pdf

Even after looking at the referenced material, however, I must confess that
I still need clarification on the following points:

1.) I do not understand why the UDP checksum (or anything prior to
csum_start) would matter for the inner packet checksum offload.

2.) Assuming that the trailer checksum (OCS) is present, it is not clear
how offload for the inner packet checksum could be expected to work
properly without change. Remember, OCS does not cause the data in the
trailer to sum to zero. It causes it to sum to the ones-complement of the
trailer length.

The modifications needed to handle point #2 are small -- specifically,
adding the trailer length to whatever would normally be preloaded into the
inner packet. The FRAG proposal
in draft-ietf-tsvwg-udp-options-13#section-5.5, which proposes to sandwich
the payload in between option headers, would make it a whole lot harder.

Looking forward to your reply.

Mike