Re: [iccrg] [tsvwg] [CCWG] New Internet Draft: Congestion Signaling (CSIG)

[Hesham ElBakoury <[e-mail]>]

What is the answer to the question: "So, we ask: what is the way ahead?
Should congestion control really stay end-to-end?"

Hesham

On Fri, Sep 15, 2023, 4:09 AM Bob Briscoe <[e-mail]> wrote:

> Hesham,
>
> See earlier discussion of that paper on iccrg:
> https://mailarchive.ietf.org/arch/msg/iccrg/UK4sg6MsjI8ijgtVJyNVdL5iMGQ/
>
>
> Bob
>
> On 15/09/2023 08:37, Hesham ElBakoury wrote:
>
> Speaking of Internet congestion control, have you looked at this paper:
> Future Internet Congestion Control: The Diminishing Feedback Problem [
> https://arxiv.org/pdf/2206.06642]? The abstract of this paper says: *<we
> argue that congestion control mechanisms should move away from their strict
> “end-to-end”*
> *adherence. This change would benefit from avoiding a “one size fits all
> circumstances” approach, and moving towards a more*
> *selective set of mechanisms that will result in a better performing
> Internet.*>
>
> Hesham
>
> On Thu, Sep 14, 2023, 3:23 AM Sebastian Moeller <[e-mail]> wrote:
>
>> Hi Ruediger,
>>
>>
>> > On Sep 14, 2023, at 09:25, <[e-mail]> <
>> [e-mail]> wrote:
>> >
>> > Sebastian, Ingemar,
>> >
>> > the draft says "limited domain". The method suggested resembles IPPM's
>> IOAM (but the packet size remains fixed, certainly an advantage).
>> Information like a port ID only makes sense, if the evaluation unit is
>> aware of the domain-topology.
>>
>>         [SM] Not sure I understand this, if on an internet-wide path all
>> nodes would simply include the minimum of their egress bitrate (that is
>> only include this if it is lower than the existing value), then an end-2end
>> protocol like TCP could even as part of the initial 3 way handshake figure
>> out a hard limit for maximum rasinsable cwin (by simply calculating the
>> BDP). SUre that would not stop slow-start fuully from overshooting, but it
>> should limit the maximum overshoot considerably....
>>
>>
>> > That will be restricted to a single domain, I think.
>>
>>         [SM] As long as the forward and collection path requires L2
>> elements, sure this will not work over the internet.
>>
>>
>> > I'm also not sure, what is to be optimized on an operational carrier
>> backbone.
>>
>>         [SM] I trust you on this, as that is out of my league.
>>
>>
>> > Under expectable conditions, these are operated without congestion.
>> Bottlenecks are likely at the access and, in a rather limited number of
>> cases, at peering. There you'd need to know the RTT so that the available
>> bandwidth calculation makes sense.
>>
>>         [SM] But the protocol that is supposed to consume the congestion
>> information will know the RTT, so can take this into account, the network
>> really only needs to give information about its current state. That said, I
>> can see network nodes that might want to know a flow's RTT, but I do not
>> think there is a way of supplying that this is not both easy and efficient
>> and not easily gamed. Passing on congestion informatin however is far less
>> abusable (after all the node could just as well have dropped the packet
>> instead of bothering to fudge the congestion info).
>>
>>
>> > Or you need to calculate several values for several RTTs and provide
>> this information (RTTs and corresponding available bandwidths). Requires a
>> fair amount of bytes, so then trust may be an issue (I'm not sure whether a
>> carrier would be willing to add an extended Ethernet header to every
>> encrypted IP packet indicating interest in getting CSIG information on
>> bottleneck links).
>>
>>         [SM] Yes encryption/privacy are reasons to stick to L2/L3. I
>> would guess in a limited domain deploying CSIG as a pseudo VLAN header
>> seems like a nice work-around, but this information really should be inside
>> IP header to go end to end. IMHO it also should not be in an optional
>> header, but should have been part of the mandatory header, but that ship
>> has sailed long ago... It turns out that the mechanism intended here IPv6
>> extension headers was mis-deployed... (It should have been initiated with a
>> really desirable use-case so it would actrually see usage immediately,
>> protocol design is a bit like evolution, in both cases "use it or loose it"
>> applies to some degree.)
>>
>> Regards
>>         Sebastian
>>
>>
>> >
>> > Regards, Ruediger
>> >
>> > -----Ursprüngliche Nachricht-----
>> > Von: Sebastian Moeller <[e-mail]>
>> > Gesendet: Donnerstag, 14. September 2023 08:49
>> > An: Ingemar Johansson S <[e-mail]>
>> > Cc: Geib, Rüdiger <[e-mail]>; [e-mail];
>> [e-mail]; [e-mail]; [e-mail]
>> > Betreff: Re: [tsvwg] [iccrg] New Internet Draft: Congestion Signaling
>> (CSIG)
>> >
>> > Hi Ingemar
>> >
>> >> On Sep 14, 2023, at 08:42, Ingemar Johansson S <
>> [e-mail]> wrote:
>> >>
>> >> Hi Ruediger
>> >>
>> >> Even though CSIG is on ethernet, it appears to be e2e as the feedback
>> is on L4. So I guess somehow the CSIG info needs to jump from domain to
>> domain somewhow, and that sounds to me like L3, albeit perhaps brief jumps
>> ?
>> >
>> >       [SM] rfc3168 and L4S ECN signaling faces the same hurdle, forward
>> path uses a different layer than feed-back path. And I do not think that it
>> is conceptually all that cleaner to have fewer layers between forward and
>> reverse signaling... "clean" would be to put both forward and reverse
>> information into the same layer, but that is not on the table as far as I
>> can see.
>> >       The other difference however, richness of signal, is a clear
>> difference between 1 bit ECN and multibit CSIG (and alternatives).
>> (Side-note, sure the ECN bitfield is two bits but it only carries 2
>> congestion states when ECN is in use, which makes it IMHO a 1-bit
>> information channel, if we had followed the SCE proposal with its 3
>> congestion states we would be at 1.5 bits ;))
>> >
>> > Regards
>> >       Sebastian
>> >
>> >
>> >
>> >>
>> >> /Ingemar
>> >>
>> >>> -----Original Message-----
>> >>> From: [e-mail] <[e-mail]>
>> >>> Sent: Wednesday, 13 September 2023 14:20
>> >>> To: [e-mail]; Ingemar Johansson S
>> >>> <[e-mail]>
>> >>> Cc: [e-mail]; [e-mail]; [e-mail];
>> >>> [e-mail]; [e-mail]; [e-mail];
>> >>> [e-mail]; [e-mail]; [e-mail];
>> >>> [e-mail]; [e-mail]; [e-mail]
>> >>> Subject: AW: [tsvwg] [iccrg] New Internet Draft: Congestion Signaling
>> >>> (CSIG)
>> >>>
>> >>> Hi Ingemar, hi Sebastian,
>> >>>
>> >>> Skimming over the draft only, isn't CSIG about Ethernet-domains,
>> >>> while L4S is E2E?
>> >>>
>> >>> Regards,
>> >>>
>> >>> Ruediger
>> >>>
>> >>> -----Ursprüngliche Nachricht-----
>> >>> Von: tsvwg <[e-mail]> Im Auftrag von Sebastian Moeller
>> >>> Gesendet: Mittwoch, 13. September 2023 13:42
>> >>> An: Ingemar Johansson S
>> >>> <ingemar.s.johansson=[e-mail]>
>> >>> Cc: Vidhi Goel <vidhi_goel=[e-mail]>;
>> >>> Shihang(Vincent) <shihang9=[e-mail]>; tsvwg
>> >>> <[e-mail]>; Jai Kumar <[e-mail]>; IETF IPPM WG
>> >>> <[e-mail]>; Tom Herbert <tom=[e-mail]>;
>> >>> [e-mail]; Abhiram Ravi <abhiramr=[e-mail]>;
>> >>> Nandita Dukkipati <[e-mail]>; [e-mail]; Huangyihong
>> >>> (Rachel) <rachel.huang=[e-mail]>; Naoshad Mehta
>> >>> <[e-mail]>
>> >>> Betreff: Re: [tsvwg] [iccrg] New Internet Draft: Congestion Signaling
>> >>> (CSIG)
>> >>>
>> >>> Hi Ingemar,
>> >>>
>> >>>
>> >>>> On Sep 13, 2023, at 12:30, Ingemar Johansson S
>> >>> <ingemar.s.johansson=[e-mail]> wrote:
>> >>>>
>> >>>> Hi
>> >>>>
>> >>>> I agree with Vihdi
>> >>>>
>> >>>> L4S is recently standardised
>> >>>
>> >>>     [SM] In experimental track, the goal is currently to test whether
>> it
>> >>> can/should be deployed at scale....
>> >>>
>> >>>> and it is definitely gaining traction also in 3GPP. We have an echo
>> >>> system that is looking forward to having L4S widely deployed.
>> >>>> Still, the congestion control aspects are not fully explored yet.
>> >>>> One
>> >>> interesting topic is if L4S allows to more safely deviate from
>> >>> additive increase to make congestion control algorithms more quickly
>> >>> converge to higher link capacity. There are a number of study topics
>> >>> around L4S congestion control that are listed in e.g the TCP Prague
>> draft.
>> >>>>
>> >>>> I cannot dictate what others should do with their time and money but
>> >>> personally I'd prefer that the IETF explores L4S and its
>> >>> possibilities and downsides before jumping on the next idea.
>> >>>
>> >>>     [SM] L4S can be described as taking the ideas behind DCTCP and
>> >>> making them fit for use over the internet*. Yet the signaling
>> >>> discussed here is to be used in e.g. data center contexts where DCTCP
>> >>> is already used and found lacking compared to newer methods operating
>> >>> on richer congestion information (HPCC, Swift, Poseidon, ...).
>> >>>     Given that L4S essentially uses a multi-packet signal(**) to
>> report
>> >>> the "queue filling state" that is then stochastically distributed
>> >>> over all concurrent flows, it seems obvious to me that reconstructing
>> >>> a reliable estimate of on-path queueing will take some time and
>> >>> averaging for each individual flow, I would guess that in some
>> >>> environments this delay simply is too costly.
>> >>>     So L4S and CSIG seem complementary and in no way mutually
>> exclusive.
>> >>>
>> >>>
>> >>> Regards
>> >>>     Sebastian
>> >>>
>> >>>
>> >>>
>> >>> *) I will not further discuss whether that is achieved or not as it
>> >>> seems irrelevant here.
>> >>> **) In essence transmission of congestion state via a 1-bit serial
>> >>> channel, clocked at the (variable***) packet rate at the bottleneck.
>> >>> ***) as packets are not of uniform size
>> >>>
>> >>>>
>> >>>> CSIG sounds to me like something that belongs more in ICCRG or ?
>> >>>>
>> >>>> /Ingemar
>> >>>>
>> >>>> From: tsvwg <[e-mail]> On Behalf Of Vidhi Goel
>> >>>> Sent: Wednesday, 13 September 2023 00:59
>> >>>> To: Shihang(Vincent) <shihang9=[e-mail]>
>> >>>> Cc: Huangyihong (Rachel) <rachel.huang=[e-mail]>;
>> >>> Tom Herbert <tom=[e-mail]>; Abhiram Ravi
>> >>> <abhiramr=[e-mail]>; IETF IPPM WG <[e-mail]>;
>>
>> >>> tsvwg <[e-mail]>; [e-mail]; [e-mail]; Nandita
>> >>> Dukkipati <[e-mail]>; Naoshad Mehta <[e-mail]>;
>> >>> Jai Kumar <[e-mail]>
>> >>>> Subject: Re: [tsvwg] [iccrg] New Internet Draft: Congestion
>> >>>> Signaling
>> >>> (CSIG)
>> >>>>
>> >>>> Not sure why we are coming up with so many new techniques when ECN
>> >>> just works fine.
>> >>>> ECN is a 2 bit field (not 1 bit) and seems to be sufficient to
>> >>> indicate extent of congestion by marking it per packet. Adding more
>> >>> complexity to any layer whether it is L2 or L3 doesn’t work well in
>> >>> deployments. Our goal should be to simplify things and only add new
>> >>> headers if absolutely necessary.
>> >>>>
>> >>>> Vidhi
>> >>>>
>> >>>>
>> >>>> On Sep 12, 2023, at 3:12 AM, Shihang(Vincent)
>> >>> <shihang9=[e-mail]> wrote:
>> >>>>
>> >>>> Hi,
>> >>>> I agree L2 may not be the best choice to carry the congestion
>> >>> signaling end-to-end and more bits are needed. We have submitted a
>> >>> draft to carry the multi-bits congestion signaling in L3. We call it
>> >>> Advanced ECN. See
>> https://datatracker.ietf.org/doc/draft-shi-ccwg-advanced-ecn/.
>> >>>>
>> >>>> Thanks,
>> >>>> Hang
>> >>>>
>> >>>> From: CCWG <[e-mail]> On Behalf Of Huangyihong (Rachel)
>> >>>> Sent: Tuesday, September 12, 2023 5:41 PM
>> >>>> To: Tom Herbert <tom=[e-mail]>; Abhiram Ravi
>> >>> <abhiramr=[e-mail]>
>> >>>> Cc: IETF IPPM WG <[e-mail]>; tsvwg <[e-mail]>;
>> >>> [e-mail]; [e-mail]; Nandita Dukkipati
>> >>> <[e-mail]>; Naoshad Mehta <[e-mail]>; Jai Kumar
>> >>> <[e-mail]>
>> >>>> Subject: Re: [CCWG] [iccrg] [tsvwg] New Internet Draft: Congestion
>> >>> Signaling (CSIG)
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> I also have the same feeling. Implementing in L2 may be difficult to
>> >>> be used in e2e transport. Of course it can work well in limited
>> >>> domain, like DC or HPC clusters. However, I also look for some
>> >>> solutions that could be able to go through internet. We have
>> >>> submitted a draft to describe the transport challenges. See
>> >>> https://datatracker.ietf.org/doc/html/draft-huang-tsvwg-transport-
>> >>> challenges.
>> >>>>
>> >>>> I share the same opinion that the congestion signal is useful and
>> >>> current 1-bit ECN solution is not fully sufficient. But I also feel
>> >>> like the more straight way is to extend L3, or l4, like update IOAM,
>> >>> to carry the information. For L2 solution, it should be developed
>> >>> together with IEEE 802.1.
>> >>>>
>> >>>> BR，
>> >>>> Rachel
>> >>>>
>> >>>> 发件人: iccrg <[e-mail]> 代表 Tom Herbert
>> >>>> 发送时间: 2023年9月10日 0:10
>> >>>> 收件人: Abhiram Ravi <abhiramr=[e-mail]>
>> >>>> 抄送: IETF IPPM WG <[e-mail]>; tsvwg <[e-mail]>;
>> >>> [e-mail]; [e-mail]; Nandita Dukkipati
>> >>> <[e-mail]>; Naoshad Mehta <[e-mail]>; Jai Kumar
>> >>> <[e-mail]>
>> >>>> 主题: Re: [iccrg] [tsvwg] New Internet Draft: Congestion Signaling
>> >>> (CSIG)
>> >>>>
>> >>>> Hi, thanks for draft!
>> >>>>
>> >>>> The first thing that stands out to me is the carrier of the new
>> >>>> packet
>> >>> headers. In the forward path it would be in L2 and in reflection it
>> >>> would be L4. As the draft describes, this would entail having to
>> >>> support the protocol in multiple L2 and multiple L4 protocols--
>> >>> that's going to be a pretty big lift! Also, L2 is not really an
>> >>> end-to-end protocol (would legacy switches in the path also forward
>> the header)l?).
>> >>>>
>> >>>> The signaling being described in the draft is network layer
>> >>> information, and hence IMO should be conveyed in network layer
>> headers.
>> >>> That's is L3 which conveniently is the average of L2+L4 :-)
>> >>>>
>> >>>> IMO, the proper carrier of the signal data is Hop-by-Hop Options.
>> >>>> This
>> >>> is end-to-end and allows modification of data in-flight. The typical
>> >>> concern with Hop-by-Hop Options is high drop rates on the Internet,
>> >>> however in this case the protocol is explicitly confined to a limited
>> >>> domain so I don't see that as a blocking issue for this use case.
>> >>>>
>> >>>> The information being carried seems very similar to that of IOAM
>> >>>> (IOAM
>> >>> uses Hop-by-Hop Options and supports reflection). I suppose the
>> >>> differences are that this protocol is meant to be consumed by the
>> >>> transport Layer and the data is a condensed summary of path
>> >>> characteristics. IOAM seems pretty extensible, so maybe it could be
>> >>> adapted to carry the signals of this draft?
>> >>>>
>> >>>> A related proposal might be FAST draft-herbert-fast. Where the CSIG
>> >>>> is
>> >>> network to host signaling, FAST is host to network signaling for the
>> >>> purposes of requesting network services. These might be complementary
>> >>> and options for both may be in the same packet. FAST also uses
>> >>> reflection, so we might be able to leverage some common
>> >>> implementation at a destination.
>> >>>>
>> >>>> Tom
>> >>>>
>> >>>> On Fri, Sep 8, 2023, 7:43 PM Abhiram Ravi
>> >>> <abhiramr=[e-mail]> wrote:
>> >>>> Hi IPPM folks,
>> >>>>
>> >>>> I am pleased to announce the publication of a new internet draft,
>> >>> Congestion Signaling (CSIG): https://datatracker.ietf.org/doc/draft-
>> >>> ravi-ippm-csig/
>> >>>>
>> >>>> CSIG is a new end-to-end packet header mechanism for in-band
>> >>>> signaling
>> >>> that is simple, efficient, deployable, and grounded in concrete use
>> >>> cases of congestion control, traffic management, and network
>> >>> debuggability. We believe that CSIG is an important new protocol that
>> >>> builds on top of existing in-band network telemetry protocols.
>> >>>>
>> >>>> We encourage you to read the CSIG draft and provide your feedback
>> >>>> and
>> >>> comments. We have also cc'd the TSVWG, CCWG, and ICCRG mailing lists,
>> >>> as we believe that this work may be of interest to their members as
>> well.
>> >>>>
>> >>>> Thank you for your time and consideration.
>> >>>>
>> >>>> Sincerely,
>> >>>> Abhiram Ravi
>> >>>> On behalf of the CSIG authors
>> >>
>> >
>> > _______________________________________________
>> > iccrg mailing list
>> > [e-mail]
>> > https://www.irtf.org/mailman/listinfo/iccrg
>>
>> --
>> CCWG mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/ccwg
>>
>
> --
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
>
>