Re: [ippm] [iccrg] [tsvwg] New Internet Draft: Congestion Signaling (CSIG)

Tom Herbert <tom@herbertland.com> Sun, 18 February 2024 18:50 UTC

Return-Path: <tom@herbertland.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3480BC14EB19 for <ippm@ietfa.amsl.com>; Sun, 18 Feb 2024 10:50:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.106
X-Spam-Level:
X-Spam-Status: No, score=-2.106 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=herbertland.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 32M_-qPMb5TD for <ippm@ietfa.amsl.com>; Sun, 18 Feb 2024 10:50:36 -0800 (PST)
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF2B7C14F5EB for <ippm@ietf.org>; Sun, 18 Feb 2024 10:50:36 -0800 (PST)
Received: by mail-ed1-x533.google.com with SMTP id 4fb4d7f45d1cf-56439b7c7a9so1005843a12.3 for <ippm@ietf.org>; Sun, 18 Feb 2024 10:50:36 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=herbertland.com; s=google; t=1708282235; x=1708887035; darn=ietf.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=N7wkXxAmuUWczpwW0vkcyuIAO4CfRWczRUJje3Gv4QU=; b=AvalE6E2SzW2w0aTfnSBggztujTBGflc0qfgqHKepumiy2i1sQJbLivnMjG5bSsrXT R+DZLcDZcoYx/31wgmRDGIqMa2cmRdmpag7ZtJ/JMFw92qnySbZ5mwBeQ0KuuQYnTogL yXfCcXn8g29Q8jP4067O4qVL9+D6R4PVk5r1oC/PG39PfsItIjmJlYwzdw40NmPWKo1F E8MC6ekET38M6bmD4Kb/lDFT9bbE/deOhwPpR3qUdobCCn/Zv1GAfblCV0xgaxxp4efr lMYm5Ebh7+V+edaCNoD/lXyFbjkrWLTnXy4C10scYA41/2mF3E/eirv1N4cCA5fVett1 Dnew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708282235; x=1708887035; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N7wkXxAmuUWczpwW0vkcyuIAO4CfRWczRUJje3Gv4QU=; b=JsBjLH1vRn4kd8UaqMop1Wm9+rlX+oWnSoeA8hDJ4Lsqp5hd8oF21oWv+hVGdb15wD WqbP8vEA5G9Hes65ssUoxGv+nb1tzLCFDPFaaz+KFfYr1n5OtQjMkKvyuRayrqitgYqB SgWuYyj5Uaybpz+6S4aDwYcKEjAtkBn9E+ja7IHXNtnScQp+7svwufCzeoJZ7w083BWW O7yRsvtdeYo7MZvlF9AcWelU2SibWZhyUXfLeZ7YodmtOZeup+izIjkiQyYHGTpFhJHQ 1CB0U2mNo/OyeRIbTe5YLONS0k01zcp0l2f6HCXjMISmdCuCjTO6gBjW84K70Ko8j41d S1XA==
X-Forwarded-Encrypted: i=1; AJvYcCUPMJczeM3wJUGu5BKrHX+RQbG7WSFGPCxjn2Je1zdYC2HmaB6az2EhubJh8deMdc/yPt4Tdy7pZp5SUR7m
X-Gm-Message-State: AOJu0YwmhYXhqG9AUtQad+r4ukaafIAE+/7cK47TVa3zJuLTtnrTsncB 9Ja1RN1wYxCmzyFiof+lKoSZGP23MVMW0R4Ey6q6s80kD2V7RT2PCryqhrgw7HHYN0ot9tCeUa6 +QRKYKPngU/FFsAXTgYk8/U6h0trleWSs3DQ/
X-Google-Smtp-Source: AGHT+IGkHBviyGdPPP0toDtNhX6kuQ2ErZR/AL7PHHm8pzL/SXka0erEato/eyZviLrXu29i4FxUfpXGTt/s/BrlqaI=
X-Received: by 2002:a50:cd58:0:b0:563:e5e0:85e1 with SMTP id d24-20020a50cd58000000b00563e5e085e1mr6313107edj.25.1708282234655; Sun, 18 Feb 2024 10:50:34 -0800 (PST)
MIME-Version: 1.0
References: <CALx6S347exOZhHtsJCwDA3mZEBjMW=c5GYRgOiAJUNUbEG=SOQ@mail.gmail.com> <7CD9EBE2-8517-4FD4-9EB5-054D65DCA79C@broadcom.com>
In-Reply-To: <7CD9EBE2-8517-4FD4-9EB5-054D65DCA79C@broadcom.com>
From: Tom Herbert <tom@herbertland.com>
Date: Sun, 18 Feb 2024 10:50:23 -0800
Message-ID: <CALx6S35O57iAVD9ROawzvUUPwywGwB_msORBrAmvFfN_xtoMGA@mail.gmail.com>
To: Jai Kumar <jai.kumar@broadcom.com>
Cc: Matt Mathis <mattmathis@measurementlab.net>, Nandita Dukkipati <nanditad@google.com>, Abhiram Ravi <abhiramr@google.com>, IETF IPPM WG <ippm@ietf.org>, tsvwg <tsvwg@ietf.org>, ccwg@ietf.org, iccrg@irtf.org, Naoshad Mehta <naoshad@google.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/Dr4vD4hCVIQ5rOytxQv-A-n8u0s>
Subject: Re: [ippm] [iccrg] [tsvwg] New Internet Draft: Congestion Signaling (CSIG)
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Feb 2024 18:50:41 -0000

On Sun, Feb 18, 2024 at 9:14 AM Jai Kumar <jai.kumar@broadcom.com> wrote:
>
> Hello Tom,
>
> As a major switch vendor provider, I can with confidence say that your statement is not correct.
> Quote:
> “ we insert more information into L2 that makes it a variable length header which actually makes processing *harder*, not easier especially for legacy devices that don't support CSIG.”
>
> Layer 3 parsing is much more complex specifically when packet traverses thru multiple tunnel domains where packet need to be encap’ed or decap’ed. Not only the parsing gets impacted, transit nodes need to worry about checksum and/or encrypted, auth packet as well.

Jai,

Those cases do not affect Layer 3 parsing at intermediate nodes.
Packets are forwarded based on the addresses in the outer network
layers in the presence of tunnels. If the argument is that updating
the IPv4 header checksum is too much trouble then use IPv6 or just
make your network one big Ethernet domain and switch on Ethernet E2E.
In the latter case you may as well make the return path go over L2 to
make the protocol symmetric in both directions; that solves the
multiple upper layer protocols problem and you wouldn't even need to
take the proposal to IETF at all  :-).

>
> HW implement is one of the challenges IOAM is facing and that’s why it has such a sparse (or no) support from switch vendors.

I'll let the IOAM and Segment Routing advocates respond to that.

Tom

>
> Keeping CSIG in layer 2 or 2.5 (like MPLS) simplifies complexity of hw implementation by multitude.
>
> Best,
> -Jai
>
>
> > On Feb 18, 2024, at 8:49 AM, Tom Herbert <tom@herbertland.com> wrote:
> >
> > On Sat, Feb 17, 2024 at 11:17 AM Matt Mathis
> > <mattmathis@measurementlab.net> wrote:
> >>
> >
> > Hi Matt,
> >
> >> I think the L2/L4 split is brilliant.
> >
> > As opposed to the L1-L7 split being brilliant? :-)
> >
> >> Putting the forward instrumentation as low as possible in the stack permits easy processing in HW w/o parsing any L3.
> >
> > A switch is already doing IP routing which requires parsing of L3. If
> > we insert more information into L2 that makes it a variable length
> > header which actually makes processing *harder*, not easier,
> > especially for legacy devices that don't support CSIG. If this is done
> > in a Hop-by-Hop Option, then all the protocols a router looks at are
> > at fixed offsets and fixed length (Ethernet header could be at offset
> > 0, IPv6 header is at offset 14, and the CSIG information is at offset
> > 54)-- that's the best case scenario for Hw router processing.
> >
> >> Putting the replies in L4 only requires a handful of implementations to cover all possible paths,
> >
> > I'm not sure "a handful of implementations" could cover all possible
> > paths. The draft only talks about TCP and QUIC, but that is hardly a
> > complete set of protocols-- there's RTP, GTP, ROCEE, and various
> > tunneling protocols to name a few. On the other hand, if this
> > information is in the Network layer then there are at most two
> > protocols that need to be extended: IPv4 and IPv6. This also true at
> > the link layer, this proposal would need to extend Ethernet, Mobile,
> > and tunnels. The fundamental premise of a network layer protocol is
> > that it works with *any* transport layer above it, and *any* link
> > layer protocol below it (that to me is the brilliant thing :-) ).
> >
> >> and piggybacks on existing solutions to session layer issues, such as authentication and authorization.
> >
> > But that only considers the return path of the signals, not the
> > forward path so it could only solve half the problem. If the forward
> > path is in L2 then we can't leverage Internet protocols for
> > authentication (maybe there's ways to do this at L2 but then we have
> > to consider every L2 protocol and regardless the the path
> > characteristics are very different in both directions of
> > communications and the protocols aren't even under the same SDO). If
> > we use L3 in both directions then we can leverage all of the IETF
> > protocols. For instance, if E2E authentication is required, then it's
> > just a matter of using AH in both directions (note the presence of AH
> > in packet doesn't affect router processing).
> >
> >>
> >> I would consider mentioning but then temporarily excluding alternet placements: either as a shim at the top of L2, sort of like VLAN tags, or within an L3 option.   Both of these have their own challenges, but might be extremely valuable in some environments.
> >
> > There have been a *lot* of discussions and work in IETF on how to
> > optimize router processing for network to host signaling like this
> > draft is proposing.  Please look at IOAM, the work being done in 6man
> > to make Hop-by-Hop Options viable, and work done in tunneling
> > protocols that are intended to be processed in HW.
> >
> > Tom
> >
> >>
> >>> On Sat, Feb 10, 2024 at 7:42 AM Tom Herbert <tom=40herbertland.com@dmarc.ietf.org> wrote:
> >>>
> >>> On Fri, Feb 9, 2024 at 10:53 PM Nandita Dukkipati <nanditad@google.com> wrote:
> >>>>
> >>>> Hi Tom,
> >>>>
> >>>> We updated the draft, correcting some nit errata, and to not let the draft expire. It's not discussed in any other mailing lists.
> >>>
> >>> Thanks Nandita.
> >>>
> >>> I still have fundamental concerns about the protocol layering in this
> >>> draft, please see my previous comments on that. The draft defines a
> >>> protocol for end-to-end network to host signaling and IMO, such a
> >>> protocol belongs in the network layer but the draft puts the protocol
> >>> in L2 and L4 and seems to avoid L3 without explanation. IOAM defines a
> >>> very similar method of signaling and RFC9486 is a good model for
> >>> network layer protocol that provides network to host signaling.
> >>>
> >>> Tom
> >>>
> >>>>
> >>>> Nandita
> >>>>
> >>>> On Thu, Feb 8, 2024 at 3:53 PM Tom Herbert <tom@herbertland.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I noticed there is now an -01 version of the draft posted on Feb. 2.
> >>>>> Is this draft being discussed on some other list?
> >>>>>
> >>>>> Thanks,
> >>>>> Tom
> >>>>>
> >>>>> On Sat, Sep 9, 2023 at 9:09 AM Tom Herbert <tom@herbertland.com> wrote:
> >>>>>>
> >>>>>> Hi, thanks for draft!
> >>>>>>
> >>>>>> The first thing that stands out to me is the carrier of the new packet headers. In the forward path it would be in L2 and in reflection it would be L4. As the draft describes, this would entail having to support the protocol in multiple L2 and multiple L4 protocols-- that's going to be a pretty big lift! Also, L2 is not really an end-to-end protocol (would legacy switches in the path also forward the header)l?).
> >>>>>>
> >>>>>> The signaling being described in the draft is network layer information, and hence IMO should be conveyed in network layer headers. That's is L3 which conveniently is the average of L2+L4 :-)
> >>>>>>
> >>>>>> IMO, the proper carrier of the signal data is Hop-by-Hop Options. This is end-to-end and allows modification of data in-flight. The typical concern with Hop-by-Hop Options is high drop rates on the Internet, however in this case the protocol is explicitly confined to a limited domain so I don't see that as a blocking issue for this use case.
> >>>>>>
> >>>>>> The information being carried seems very similar to that of IOAM (IOAM uses Hop-by-Hop Options and supports reflection). I suppose the differences are that this protocol is meant to be consumed by the transport Layer and the data is a condensed summary of path characteristics. IOAM seems pretty extensible, so maybe it could be adapted to carry the signals of this draft?
> >>>>>>
> >>>>>> A related proposal might be FAST draft-herbert-fast. Where the CSIG is network to host signaling, FAST is host to network signaling for the purposes of requesting network services. These might be complementary and options for both may be in the same packet. FAST also uses reflection, so we might be able to leverage some common implementation at a destination.
> >>>>>>
> >>>>>> Tom
> >>>>>>
> >>>>>> On Fri, Sep 8, 2023, 7:43 PM Abhiram Ravi <abhiramr=40google.com@dmarc.ietf.org> wrote:
> >>>>>>>
> >>>>>>> Hi IPPM folks,
> >>>>>>>
> >>>>>>> I am pleased to announce the publication of a new internet draft, Congestion Signaling (CSIG): https://www.google.com/url?q=https://datatracker.ietf.org/doc/draft-ravi-ippm-csig/&source=gmail-imap&ust=1708879744000000&usg=AOvVaw2NqlnBohcTmRfPfCpkG028
> >>>>>>>
> >>>>>>> CSIG is a new end-to-end packet header mechanism for in-band signaling that is simple, efficient, deployable, and grounded in concrete use cases of congestion control, traffic management, and network debuggability. We believe that CSIG is an important new protocol that builds on top of existing in-band network telemetry protocols.
> >>>>>>>
> >>>>>>> We encourage you to read the CSIG draft and provide your feedback and comments. We have also cc'd the TSVWG, CCWG, and ICCRG mailing lists, as we believe that this work may be of interest to their members as well.
> >>>>>>>
> >>>>>>> Thank you for your time and consideration.
> >>>>>>>
> >>>>>>> Sincerely,
> >>>>>>> Abhiram Ravi
> >>>>>>> On behalf of the CSIG authors
> >>>
> >>> _______________________________________________
> >>> iccrg mailing list
> >>> iccrg@irtf.org
> >>> https://www.google.com/url?q=https://mailman.irtf.org/mailman/listinfo/iccrg&source=gmail-imap&ust=1708879744000000&usg=AOvVaw1WMuz9B16Iot8u2lJtzb-U
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> --MM--
> >> Evil is defined by mortals who think they know "The Truth" and use force to apply it to others.
>
> --
> This electronic communication and the information and any files transmitted
> with it, or attached to it, are confidential and are intended solely for
> the use of the individual or entity to whom it is addressed and may contain
> information that is confidential, legally privileged, protected by privacy
> laws, or otherwise restricted from disclosure to anyone else. If you are
> not the intended recipient or the person responsible for delivering the
> e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.