Re: [tsvwg] [iccrg] New Internet Draft: Congestion Signaling (CSIG)

[Tom Herbert <[e-mail]>]

On Mon, Feb 19, 2024 at 11:59 PM Sebastian Moeller <moeller0=
[e-mail]> wrote:
>
> Hi Tom,
>
>
> > On 19. Feb 2024, at 21:09, Tom Herbert <tom=
[e-mail]> wrote:
> >
> > On Mon, Feb 19, 2024 at 11:31 AM Sebastian Moeller
> > <moeller0=[e-mail]> wrote:
> >>
> >> Hi Tom,
> >>
> >>
> >>> On 19. Feb 2024, at 18:53, Tom Herbert <tom=
[e-mail]> wrote:
> >>>
> >>> On Sun, Feb 18, 2024 at 11:34 PM Sebastian Moeller
> >>> <moeller0=[e-mail]> wrote:
> >>>>
> >>>> Hi Matt,
> >>>>
> >>>>> On 17. Feb 2024, at 20:17, Matt Mathis <
[e-mail]> wrote:
> >>>>>
> >>>>> I think the L2/L4 split is brilliant.
> >>>>
> >>>> [SM] Respectfully, the brilliance depends very much on the
goal/gamer plan. Is this purely aimed at data center traffic this looks
like a sweet solution that is 'organically' confined to the domain with
appropriately capable L2 elements? Or is the end-game here an (overdue)
improvement of end-to-end loads/congestion information? In the former case
L2/L4 seems a decent solution, in the latter case less so (not that getting
a common L3 solution would be guaranteed or easy).
> >>>>
> >>>>
> >>>>> Putting the forward instrumentation as low as possible in the stack
permits easy processing in HW w/o parsing any L3.
> >>>>
> >>>> [SM] Sweet, but that really means that this solution is unlikely to
survive over a full internet path.
> >>>>
> >>>>> Putting the replies in L4 only requires a handful of
implementations to cover all possible paths,
> >>>>
> >>>> [SM] Mmmh, that might be but partly because the L2 solution
noticeably restricts the set of possible paths, no?
> >>>>
> >>>>> and piggybacks on existing solutions to session layer issues, such
as authentication and authorization.
> >>>>
> >>>> [SM] What is the threat model here? I would guess an attacker that
knows the full path might just as well probe the congestion level and an
attacker that does not know the path might not be able to do much with the
congestion information? (Any attacker that can modify the congestion
information might as well drop the packet directly).
> >>>>
> >>>>>
> >>>>> I would consider mentioning but then temporarily excluding alternet
placements: either as a shim at the top of L2, sort of like VLAN tags, or
within an L3 option.   Both of these have their own challenges, but might
be extremely valuable in some environments.
> >>>>
> >>>> [SM] Some environments, like the internet? I know that the I in IETF
is not a strict limiter of scope, but still it would be nice if drafts
would have a viable path of being implemented over the internet... That
said, well possible that the current state does not merit
use-over-the-internet yet and so maybe starting with an L2/L4 solution
might be considered a safety back-stop?
> >>>
> >>> Sebastian,
> >>>
> >>> There's no reason to believe that Congestion Signaling isn't of
> >>> interest to use on an internet (lower case 'i' is explicit here).
> >>
> >> [SM] Well, in my mind that still would keep this out of scope for the
capital I ETCopy of Copy of Enfabrica-SiPandaF,
>
> [SM2] Not sure how that 'Copy of Copy of Enfabrica-SiPanda' string ended
up in that paragraph, I do not recall writing that... either autocompletion
run wild or some other issue.
>
>
> >> I am interested in improving end to end congestion signalling over the
Internet so I desire these signals to sink and source at my endpoints...
Again, I understand that my position is in the rough regarding what the
IETF should care about.
> >>
> >>> This is almost certainly beneficial for 6G for instance which is an
> >>> internet composed for various link layer technologies.
> >>> Neither is this
> >>> the only protocol of this nature there are and will be others-- from
> >>> an IETF POV I believe we want a extensible protocol solution that
> >>> benefits multiple use cases and works in different environments.
> >>
> >> [SM] That way lies madness IMHO. Getting enough routers/switches
support one signal and hence make it useful is already almost a Sisyphus
task expecting them to support multiple signals selected individually per
packet seems like a recipe of never getting this to work end to end (which
is my motivator here). If we do not know which single signal to use here, I
guess keeping this private and do more research seems like a productive way
forward.
> >
> > Sebastian,
> >
> > I would agree with that if this was the first protocol ever trying to
> > do something like this, but it's not. IOAM is already a published RFC.
>

Hi Sebastian,

> [SM2] There are a number of points that are against IOAM being a suitable
encapsulation here:

Yes, I wasn't suggesting that IOAM could be used without modification. I
was suggesting that a compressed format could be derived where we can
leverage the underlying mechanisms and implementation.

>
> https://datatracker.ietf.org/doc/html/rfc9378 :
> "IOAM is focused on "limited domains", as defined in [RFC8799]. IOAM is
not targeted for a deployment on the global Internet."

That's more of a statement of security and not feasibility. There's simply
no security in the Internet, so we cannot trust or validate that anonymous
intermediate nodes are going to write correct information. Any plain text
in a packet on the Internet is subject to inspection and modification if
the data isn't authenticated, and in the worst case this could be a DoS
vector by writing bad information.

>
> IMHO that already disqualifies IOAM here as the goal needs to be a fully
end to end method having the Internet as its scope.
>

Because of the fundamental security issues on the global Internet, it's
unlikely we'll ever have a usable network-thost signaling protocol on the
Internet.

> [SM2] This is already handled in the ' IOAM Proof of Transit Type 0'
field, the facrt that you did not notice that supports my argument that
IOAM is too complicated an RFC for this simple use-case, no?
>
>
> > If that constraint is
> > removed then the only remaining argument against IOAM seems to be that
> > it's easier for hardware to handle L3 rather than L2 in hardware.
>
> [SM2] No, it is still way to complicated (offering options all
intermediary nodes will need to check before setting values) and way way to
overheady...

We can define a fixed length L3 format for the fast path that would be
quite efficient. See below.

>
> > I don't believe there is currently consensus that that is generally
> > true.
>
> [SM2] Unclear what the consensus is. IMHO is is not really a question of
consensus, if we want this to be a first class end 2 end signal, L2 is
simply not an option independent of our wishes.

I agree, but we have seen these sort of statements from vendors before that
processing L3 in hardware is overly complex and difficult to make
efficient.Instead of blindly accepting that, or just dismissing the point,
I believe it's in everyone's best interest to openly discuss this, and note
there's already a lot of work to address known inefficiencies in IP
processing see draft-ietf-6man-hbh-processing for instance.

>
> > And, if this is why IOAM "has such a sparse (or no) support from
> > switch vendors" as Jai claims then it seems like this is maybe
> > something that should be discussed instead of just arbitrarily
> > dismissing IOAM. Why exactly is IOAM in HW such a problem and can it
> > be fixed? (a quick look at ippm archives didn't reveal any
> > discussions).
>
> [SM2] As shown above IOAM seems to be one of these everything and the
kitchensink solution, probably great for its intended use cases, but
efficient end/network to end signalling IM;HO is not one of these use
cases. Sure it could be bent into shape to support that use case, but I
would rather see a meaner and leaner signal design for the congestion
information aggregation along a path use case. If I had control, I would
propose a new IPv6 header type (with a new next header number) mutually
exclusive with hop by hop (so that confirming the next header number is the
only check required before updating data at a known offset). There is value
in simplicity... (I guess this scheme will fail due to firewalls only
permitting a limited set of next header values...)

We don't need to define a new extension header (and realistically we'd
never get a new EH through IETF anyway), CSIG can be efficiently done as a
Hop-by-Hop Option. For example:



+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
      |                                                   Src Mac address
                                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Dest Mac address                      |
   +                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                               |             0x86dd            |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   6   | Traffic Class |              Flow Label               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Payload Length        |      0        |   Hop Limit   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                                                               +
   |                                                               |
   +                         Source Address                        +
   |                                                               |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                                                               +
   |                                                               |
   +                      Destination Address                      +
   |                                                               |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      0        |  Next header  |      50       |       4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Type  |         LM            |               S               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |              ULP (e.g. TCP header and payload                 |

This assumes 0x32 HBH Option type is assigned to CSIG.

* Is this encoding efficient? Yes. Compared to the compact format in the
CSIG draft there is an additional four bytes of on-the-wire overhead, and
it's the same overhead as the Expanded format which is eight bytes.

* Is this friendly to hardware processing in the fast path? Yes. The
Ethernet header, IPv6 header, and Hop-by-Hop Options header with a single
option for CSIG form a fixed header of 42 bytes which will fit well within
a router's parsing buffer. that can be quickly processed. The match of this
format is Bytes 12-31 == 0x86dd && Byte 20 == 0 && Byte 34 ==0 Bytes 36-37
== 0x3204. Upon receiving a packet, these fields can be extracted and run
through a TCAM. If there's a hit that this is a packet with CSIG then it
can be processed. Note that the CSIG option can be processed in parallel
with processing the Ethernet header and IP header.

* Is this format extensible? Yes. The Type field allows CSIG formats with
different semantics and structure (the CSIG draft defines three types, so
thirteen are remaining for extensibility).

* Is the format routable? Yes, The format is contained in the Network layer
so it is properly end-to-end data that is routable over a limited domain
internet and the Internet (modulo that packets with extension headers might
be dropped in the Internet)

* Can the format be authenticated? Yes. AH header can be used. AH would
authenticate the option but not the data since the intent is for routers
along the path to write CSIG data in the packet. AH prevents someone from
inserting or removing the option in flight, the data written by routers
isn't authenticated and there's really no way to do that since we'd have to
establish and SA between routers and hosts; the typical answer to this is
that we should restrict protocols like this to trusted limited domains (for
instance, using CSIG over the Internet is pretty pointless since nothing
prevents intermediate devices from writing bad information into a packet--
this could even be a DoS vector).

* Can this format be encrypted? No. We'd need an SA established network and
hosts which doesn't seem plausible. However, the return path for reflecting
the signal could be done in a Destination Option which could be encrypted
by an IPsec header.

Tom

>
>
> Regards
>         Sebastian
>
>
> >
> > Tom
> >
> >>
> >> Regards
> >>        Sebastian
> >>
> >>
> >>>
> >>> Tom
> >>>
> >>>>
> >>>> Regards
> >>>>       Sebastian
> >>>>
> >>>>>
> >>>>> On Sat, Feb 10, 2024 at 7:42 AM Tom Herbert <tom=
[e-mail]> wrote:
> >>>>> On Fri, Feb 9, 2024 at 10:53 PM Nandita Dukkipati <
[e-mail]> wrote:
> >>>>>>
> >>>>>> Hi Tom,
> >>>>>>
> >>>>>> We updated the draft, correcting some nit errata, and to not let
the draft expire. It's not discussed in any other mailing lists.
> >>>>>
> >>>>> Thanks Nandita.
> >>>>>
> >>>>> I still have fundamental concerns about the protocol layering in
this
> >>>>> draft, please see my previous comments on that. The draft defines a
> >>>>> protocol for end-to-end network to host signaling and IMO, such a
> >>>>> protocol belongs in the network layer but the draft puts the
protocol
> >>>>> in L2 and L4 and seems to avoid L3 without explanation. IOAM
defines a
> >>>>> very similar method of signaling and RFC9486 is a good model for
> >>>>> network layer protocol that provides network to host signaling.
> >>>>>
> >>>>> Tom
> >>>>>
> >>>>>>
> >>>>>> Nandita
> >>>>>>
> >>>>>> On Thu, Feb 8, 2024 at 3:53 PM Tom Herbert <[e-mail]>
wrote:
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I noticed there is now an -01 version of the draft posted on Feb.
2.
> >>>>>>> Is this draft being discussed on some other list?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Tom
> >>>>>>>
> >>>>>>> On Sat, Sep 9, 2023 at 9:09 AM Tom Herbert <[e-mail]>
wrote:
> >>>>>>>>
> >>>>>>>> Hi, thanks for draft!
> >>>>>>>>
> >>>>>>>> The first thing that stands out to me is the carrier of the new
packet headers. In the forward path it would be in L2 and in reflection it
would be L4. As the draft describes, this would entail having to support
the protocol in multiple L2 and multiple L4 protocols-- that's going to be
a pretty big lift! Also, L2 is not really an end-to-end protocol (would
legacy switches in the path also forward the header)l?).
> >>>>>>>>
> >>>>>>>> The signaling being described in the draft is network layer
information, and hence IMO should be conveyed in network layer headers.
That's is L3 which conveniently is the average of L2+L4 :-)
> >>>>>>>>
> >>>>>>>> IMO, the proper carrier of the signal data is Hop-by-Hop
Options. This is end-to-end and allows modification of data in-flight. The
typical concern with Hop-by-Hop Options is high drop rates on the Internet,
however in this case the protocol is explicitly confined to a limited
domain so I don't see that as a blocking issue for this use case.
> >>>>>>>>
> >>>>>>>> The information being carried seems very similar to that of IOAM
(IOAM uses Hop-by-Hop Options and supports reflection). I suppose the
differences are that this protocol is meant to be consumed by the transport
Layer and the data is a condensed summary of path characteristics. IOAM
seems pretty extensible, so maybe it could be adapted to carry the signals
of this draft?
> >>>>>>>>
> >>>>>>>> A related proposal might be FAST draft-herbert-fast. Where the
CSIG is network to host signaling, FAST is host to network signaling for
the purposes of requesting network services. These might be complementary
and options for both may be in the same packet. FAST also uses reflection,
so we might be able to leverage some common implementation at a destination.
> >>>>>>>>
> >>>>>>>> Tom
> >>>>>>>>
> >>>>>>>> On Fri, Sep 8, 2023, 7:43 PM Abhiram Ravi <abhiramr=
[e-mail]> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi IPPM folks,
> >>>>>>>>>
> >>>>>>>>> I am pleased to announce the publication of a new internet
draft, Congestion Signaling (CSIG):
https://datatracker.ietf.org/doc/draft-ravi-ippm-csig/
> >>>>>>>>>
> >>>>>>>>> CSIG is a new end-to-end packet header mechanism for in-band
signaling that is simple, efficient, deployable, and grounded in concrete
use cases of congestion control, traffic management, and network
debuggability. We believe that CSIG is an important new protocol that
builds on top of existing in-band network telemetry protocols.
> >>>>>>>>>
> >>>>>>>>> We encourage you to read the CSIG draft and provide your
feedback and comments. We have also cc'd the TSVWG, CCWG, and ICCRG mailing
lists, as we believe that this work may be of interest to their members as
well.
> >>>>>>>>>
> >>>>>>>>> Thank you for your time and consideration.
> >>>>>>>>>
> >>>>>>>>> Sincerely,
> >>>>>>>>> Abhiram Ravi
> >>>>>>>>> On behalf of the CSIG authors
> >>>>>
> >>>>> _______________________________________________
> >>>>> iccrg mailing list
> >>>>> [e-mail]
> >>>>> https://mailman.irtf.org/mailman/listinfo/iccrg
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Thanks,
> >>>>> --MM--
> >>>>> Evil is defined by mortals who think they know "The Truth" and use
force to apply it to others.
>
>