Re: [tsvwg] Network feedback and security

[Gorry Fairhurst <[e-mail]>]

On 21/10/2023 11:09, Sebastian Moeller wrote:
> HI Gorry,
>
> top posting because it is a request for clarification on a single point.
>
> You call L4S a '1-bit' signal. Yet an L4S end point is expected to 
> basically translate a stream of of such 1-bit messages (one per 
> packet) into 'marking probability' and for that it needs to operate 
> over multiple such messages. So I would argue that L4S is not using a 
> 1-bit signal (it uses a 1 bit/packet carrier) but a form of multibit 
> signal. However, and in the context of timely response to congestion 
> this becomes relevant, extracting the marking probability requires 
> collecting multiple packets and hence time....*
> Now, I ask as this is not my aerea of expertise: is it still useful to 
> consider L4S a 1-bit signal or not?

So that's true. Howver in the context of packet headers it signals with 
1 bit on the wire.

Gorry

>
> Regards
>         Sebastian
>
> *) In addition the sender needs to translate some queue occupancy 
> information into a marking probability which introduces additional 
> delay.  I am beginning to get an intuition, I think, why in short RTT 
> environments DCTCP does not seem to be the last word in congestion 
> signaling...
>
>
> On 21 October 2023 11:31:58 CEST, Gorry Fairhurst 
> <[e-mail]> wrote:
>
>     On 20/10/2023 19:16, Tom Herbert wrote:
>
>         On Fri, Oct 20, 2023 at 8:44 AM Gorry Fairhurst
>         <[e-mail]> wrote:
>
>             See later, On 20/10/2023 16:12, Tom Herbert wrote:
>
>                 On Fri, Oct 20, 2023 at 1:11 AM Sebastian Moeller
>                 <[e-mail]> wrote:
>
>                     Hi Tom,
>
>                         On Oct 20, 2023, at 01:13, Tom Herbert
>                         <[e-mail]> wrote: On Thu, Oct 19,
>                         2023, 7:56 AM Sebastian Moeller
>                         <[e-mail]> wrote:
>
>                             Hi Tom,
>
>                                 On Oct 19, 2023, at 16:41, Tom Herbert
>                                 <[e-mail]> wrote: On Thu,
>                                 Oct 19, 2023 at 12:28 AM Sebastian
>                                 Moeller <[e-mail]> wrote:
>
>                                     Hi Ingemar, On 19 October 2023
>                                     07:19:40 CEST, Ingemar Johansson S
>                                     <[e-mail]>
>                                     wrote:
>
>                                         Hi I have followed this
>                                         discussi a bit. I focus only
>                                         on the congestion indication
>                                         and the discussion around the
>                                         multi-bit congestion
>                                         signaling. L4S is just
>                                         recently standardized and we
>                                         have just starated to get the
>                                         community onboard to implement
>                                         and evaluate it on a larger
>                                         scale. We should give it some
>                                         time and get more data before
>                                         we jump on the next big thing.
>                                         It is of course possible for
>                                         IETF to start working on the
>                                         next big thing with multi-bit
>                                         congestion indication. But as
>                                         I see it, we currently have
>                                         more than enough tasks on the
>                                         table with L4S. 
>
>                                     [SM] Respectfully, that task is
>                                     clearly on the proponents of L4S
>                                     and IMHO should have been done
>                                     before granting RFC status. L4S
>                                     already hindered one higher
>                                     resolution ECN approach (SCE)
>                                     without actual evidence of
>                                     superiority, let's leave it at
>                                     that one victim please. In my
>                                     opinion L4S is too little too
>                                     late, as we already know multi-bit
>                                     per packet information is
>                                     superior, so we arguably should
>                                     focus on that. 
>
>                                 Sebastian, There's a fundamental
>                                 problem with mulit-bit per packet
>                                 information: there's no place in
>                                 packets to put the data that will work
>                                 across the Internet! (other than the
>                                 eight bit TOS/Priority field). 
>
>                             [SM2] Yes, I agree. The most fitting place
>                             would be in the IP header, but there we
>                             are out of (re-)usable bits...
>
>                                 The correct mechanism for network to
>                                 hosts signaling is Hop-by-Hop options
>                                 but those currently see a 99% drop rate. 
>
>                             [SM] Again I agree, for IPv6 any of the
>                             extension headers would seems the next
>                             best place. It should be one that exists
>                             along the full path, and is visible by the
>                             end points, if Hop-byHop fits that bill,
>                             great. Or not so great, 1% passage is
>                             sobering even for a glass half full kind
>                             of person.
>
>                                 There are proposed alternatives like
>                                 putting information in transport
>                                 payload, TCP and now UDP options,
>                                 overloading flow label or IPv6
>                                 addresses; but all of these have
>                                 drawbacks that will preclude widescale
>                                 deployment (these are discussed in
>                                 section 5 of draft-herbert-host2netsig). 
>
>                             [SM] Yes deployment of any of these
>                             including IP6 extension headers is going
>                             to be an uphill battle, as the network
>                             will only gain little by this, so
>                             deployment likely will start in
>                             environments where "little" already is
>                             worth it, and that I agree is likely not
>                             the wider internet. But darn it, it would
>                             be great if having to access a service
>                             over a congested link that would result in
>                             lowish throughput but with acceptable
>                             latency, as compared t today's low
>                             throughput and high latency and jitter.
>                             That might be a way of pitching this, if
>                             this allows more graceful operation
>                             at/close to saturation this might be
>                             interesting for infrastructure folks ()
>
>                                 Since network to host signaling
>                                 modifies data in flight limits the
>                                 solution space even more, for instance
>                                 it might be okay for a switch to read
>                                 a transport payload like QUIC spin
>                                 bit, but writing data in the transport
>                                 payload risks systematic data corruption. 
>
>                             [SM] Well, all handling of packets
>                             involved reading and writing memory
>                             already, and also changes in header fields
>                             is performed routinely, TTL/Hop limit come
>                             to mind, as does NATP... so there is ample
>                             precedence even if we exclude ECN, no?
>
>                                 I agree that multi-bit data should
>                                 facilitate more precise congestion
>                                 control with faster reaction time, but
>                                 I also think we need that more in the
>                                 data center than over the Internet and
>                                 the problem of defining a multi-bit
>                                 packet field for the data is solvable
>                                 in a limited domain. 
>
>                             [SM] I respectfully disagree, I do not run
>                             a data center, but I do run a home network
>                             and use the internet, I think this is a
>                             case where all domains would profit.
>                             Realistically though I expect this to come
>                             to fruition in limited domains/data
>                             centers first. Would be sad if it would
>                             stay there however. 
>
>                         Sebastian, There may be another option.
>                         Fortunately, the definition of a limited
>                         domain is rather loose. For instance, a
>                         provider network could deploy switches that
>                         support multi-bit congestion signals in HBH
>                         options could be considered a limited domain.
>                         So if a host connects to a server in the
>                         provider's domain, like a hosted YT server for
>                         instance, then the signal is viable
>                         end-to-end. The provider network could be
>                         quite large and it could partner with
>                         connected networks to extend the limited
>                         domain. Conceivably, over time the "limited
>                         domain" could encompass a significant portion
>                         of the Internet. This is part of the plan to
>                         extend the reach FAST. 
>
>                     [SM2] OK, "starting from a limited domain" seems
>                     viable then.
>
>                         There's a problem with this though: the signal
>                         is only viable in the limited domain. For
>                         instance, if a packet with HBH options leaves
>                         the limited domain then there's a high
>                         probability the packet will be dropped.
>                         There's no easy means for a host to determine
>                         if it's sending within a limited domain, so
>                         the sending hosts can't decide on using EH. 
>
>                     [SM2] I respectfully disagree, the signal clearly
>                     should be end2end and be part of the negotiation
>                     then, so endpoints check whether the signal passes
>                     between the endpoints and disale its usage if they
>                     do not. 
>
>                 Sebastian, That was my first intuition as well-- to
>                 have hosts do some sort of "Happy Eyeballs" like
>                 probing of path capabilities, and maybe even creating
>                 a mash-up map of the capabilities of paths across the
>                 Internet so that hosts can get a priori information on
>                 the path capabilities to destinations. But thinking it
>                 through, I've come to believe that this is an
>                 inordinate amount of complexity and logic we'd have to
>                 add to host stacks just to use a protocol that is
>                 defined in an Internet standard. Note that this may
>                 coincide with negotiation, but the process itself is
>                 probing the capabilities of the path to each
>                 destination. Presumably, the host could start by
>                 sending initial packets with EH and wait for a
>                 response.If packets come back then it knows the path
>                 is viable, if it doesn't get a reply then it knows is
>                 that a packet was dropped somewhere but not why it was
>                 dropped (RFC8883 which would allows routers to send an
>                 ICMP error when packets with EH are dropped, but we
>                 know ICMP is a lost cause on the Internet). The host
>                 could try to back off and not use EH to see what
>                 happens. But even if the first packet succeeded,
>                 there's no guarantee that all packets will take the
>                 same path and some later packets in the connection may
>                 be dropped so probing might need to be a continuous
>                 process over the life of a connection. But there's a
>                 more insidious problem to deal with. Instead of
>                 dropping packets, some routers will relegate EH to
>                 slow path processing in a CPU. To the end hosts EH
>                 will appear viable, however when that happens packets
>                 will go through different queues and be subject to
>                 greater latency. When packets go through the slow path
>                 the likelihood of congestion increases substantially.
>                 So, ironically, by trying to measure congestion we may
>                 be the cause of congestion!
>
>                         The solution we're proposing is to remove the
>                         bits when packets leave the limited domain at
>                         an egress router. 
>
>                     [SM2] That I disagree with, this needs to be
>                     end2end, unless your proposal entails the end
>                     points being informed about that removal. 
>
>                 I believe you also mentioned that it is also best
>                 effort. If we remove EH at egress routers then we
>                 don't get feedback on congestion for those
>                 destinations, but at least communications are still
>                 viable. The firewalls where this happens already exist
>                 and the implementation of EH removal is very
>                 straightforward. IMO, this EH removal is a more
>                 feasible solution than pushing complexity with the
>                 necessary probing logic into all host stacks. Tom 
>
>             Somehow this thread seems to have now developed a walk
>             through various topics. Musing on the future direction for
>             IPv6 EH I'is more appropriate to discuss in v6ops (if it's
>             just operationla practice) or 6man (if the desire is
>             really to change the specification of an EH). I'm sure it
>             will get lots more comment there. That sort of discussion
>             isn't really so appropriate here, where the focus ought to
>             be on the end-to-end transport or end-to-network use of
>             these signals. 
>
>         Hi Gorry, Sebastian's proposal is for network devices to
>         produce multi-bit signals meant for consumption by the
>         transport layer at end points. It's an interesting idea, but
>         it seems reasonable to ask if it's feasible for the network
>         layer to provide those signals (right now on the Internet it's
>         not). If it's not feasible, then we don't need to spend the
>         WG's time considering the transport layer details of the
>         solution. More generally, this isn't only an example of
>         network devices generating signals that would be consumed by
>         the transport layer. For example, there's also
>         draft-ravi-ippm-csig which was posted to tsvwg (in that case
>         the carrier is Ethernet which isn't in scope of IETF). We've
>         also seen a couple of drafts proposing to use UDP options to
>         convey network layer information in the UDP surplus space.
>         Network to host signaling and host to networking signaling
>         inevitably need to cross protocol layers, and it's going to be
>         hard if the carrier and the content of signals are developed
>         in isolation. If this isn't in the charter of tsvwg that's
>         fine, but I don't know that this fits into any other WG.
>         Please advise. Thanks, Tom 
>
>     I see Tom, you're right to ask: where would standards work happen?
>     A method to transport the signals could be a 6man/intarea topic
>     topic or could be a transport one, depending on the design of
>     signal. Some of the thoughts seem to go beyond the current
>     definitions and any adaption of EH or transport protocols would
>     need to be in the respective groups. I think the signals
>     concerning CC or any capacity/latency indications would fall in
>     transport. I sympathise with the pain of coordinating between WGs
>     - the chairs can help to avoid going backwards and forwards. But,
>     the first step is to show the research results, and gather
>     interest in deploying a new standard. TSVWG is now starting to
>     complete work on a 1-but signal in the form of L4S.  I see the use
>     of multi-bit CC signals as tempting, and research on this topic
>     appears from time-to-time, especially in controlled enviroments.
>     That likely would be interesting to discuss in ICCRG, and to
>     present prototype experiments. If there's finally a need for a
>     standard, then TSVWG would be one place where a spec. could be
>     developed. Best wishes, Gorry Gorry
>
>             Gorry (YSVWG Chgair)
>
>                         as described in
>                         draft-herbert-eh-inflight-removal. This also
>                         has some nice security properties and
>                         coincides with existing network firewalls. 
>
>                     [SM2] Will need to read that, when I find time.
>                     Regards Sebastian
>
>                         Tom
>
>                                 Tom
>
>                                     Regards Sebastian
>
>                                         /Ingemar
>
>                                             -----Original Message-----
>                                             From: tsvwg
>                                             <[e-mail]>
>                                             On Behalf Of Christian
>                                             Huitema Sent: Tuesday, 17
>                                             October 2023 17:28 To:
>                                             Sebastian Moeller
>                                             <[e-mail]>;
>                                             [e-mail]
>                                             Cc: [e-mail]
>                                             Subject: [tsvwg] Network
>                                             feedback and security
>                                             (was: New Version
>                                             Notification for
>                                             draft-huang-tsvwg-transport-challenges-00.txt)
>                                             On 10/17/2023 3:17 AM,
>                                             Sebastian Moeller wrote:
>
>                                                 Personally, ever since
>                                                 I read Arslan, Serhat,
>                                                 and Nick McKeown. 
>
>                                             ‘Switches Know the Exact
>                                             Amount of Congestion’. In
>                                             Proceedings of the 2019
>                                             Workshop on Buffer Sizing,
>                                             1–6, 2019. I came to the
>                                             prediction that something
>                                             like max(bufferoccupancy)
>                                             over a network path in
>                                             either predicted
>                                             sojourntime or percentual
>                                             buffer filling could
>                                             really improve congestion
>                                             control if included in all
>                                             packets. That paper and
>                                             follow ups on that idea
>                                             are to my quite
>                                             convincing. I see no real
>                                             security concern as the
>                                             network node adding this
>                                             information might as well
>                                             have dropped the packet if
>                                             it wanted to harm that
>                                             flow, so little increase
>                                             in attack surface in that
>                                             direction. What I hope
>                                             something like this might
>                                             allow is a gentler exist
>                                             from slow start (assuming
>                                             one can measure and
>                                             compare the dynamics of
>                                             the congestion indicator
>                                             with those of the
>                                             slow-starting flow to
>                                             predict when to exist slow
>                                             start without first having
>                                             to dump ~2 too much data
>                                             into the network in one
>                                             RTT). The network could
>                                             indirectly profit if
>                                             end-points use this
>                                             information to better
>                                             reign in congestion
>                                             pro-actively, but that
>                                             clearly is not guaranteed,
>                                             especially over the open
>                                             internet. Yes, network
>                                             feedback is useful. You
>                                             mention
>                                             "max(bufferoccupancy) over
>                                             a network path" as a
>                                             signal, but I would argue
>                                             that ECN is pretty much a
>                                             1 bit version of exactly
>                                             that. So your argument is
>                                             really for "more ECN
>                                             bits". That's plausible,
>                                             but the whole "L4S/Prague"
>                                             effort seems to indicate
>                                             that 1 bit is enough,
>                                             because you can observe
>                                             that bit on many
>                                             consecutive packets. The
>                                             conservative side of me
>                                             would say we should first
>                                             deploy the 1 bit version,
>                                             and then study whether we
>                                             really need many more
>                                             bits. The "gentler exit
>                                             from slow start" issue is
>                                             largely addressed by
>                                             Hystart, which uses the
>                                             variations of the RTT as a
>                                             signal. And yes, Hystart
>                                             can be improved by also
>                                             monitoring the ECN bits,
>                                             not just the RTT. But
>                                             there are in fact security
>                                             concerns. We already know
>                                             about ICMP attacks in
>                                             which attackers spoof the
>                                             network feedback and
>                                             inject fake ICMP packets,
>                                             for example to disrupt
>                                             PMTU discovery. "Man on
>                                             the side" attackers can
>                                             easily send a second copy
>                                             of a packet with modified
>                                             header bits and race it to
>                                             the destination. If
>                                             transports accept that
>                                             copy as genuine, they will
>                                             heed the faked congestion
>                                             signals and the attacker
>                                             will succeed in disrupting
>                                             the connection. --
>                                             Christian Huitema 
>
>                                     -- Sent from my Android device
>                                     with K-9 Mail. Please excuse my
>                                     brevity. 
>
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.