Re: [tsvwg] Network feedback and security (was: New Version Notification for draft-huang-tsvwg-transport-challenges-00.txt)

[Sebastian Moeller <[e-mail]>]

Hi Tom,


> On Oct 20, 2023, at 01:13, Tom Herbert <[e-mail]> wrote:
> 
> On Thu, Oct 19, 2023, 7:56 AM Sebastian Moeller <[e-mail]> wrote:
>> 
>> Hi Tom,
>> 
>> 
>>> On Oct 19, 2023, at 16:41, Tom Herbert <[e-mail]> wrote:
>>> 
>>> On Thu, Oct 19, 2023 at 12:28 AM Sebastian Moeller <[e-mail]> wrote:
>>>> 
>>>> Hi Ingemar,
>>>> 
>>>> On 19 October 2023 07:19:40 CEST, Ingemar Johansson S <[e-mail]> wrote:
>>>>> Hi
>>>>> 
>>>>> I have followed this discussi a bit. I focus only on the congestion indication and the discussion around the multi-bit congestion signaling.
>>>>> 
>>>>> L4S is just recently standardized and we have just starated to get the community onboard to implement and evaluate it on a larger scale.
>>>>> We should give it some time and get more data before we jump on the next big thing.
>>>>> 
>>>>> It is of course possible for IETF to start working on the next big thing with multi-bit congestion indication. But as I see it, we currently have more than enough tasks on the table with L4S.
>>>> 
>>>> [SM] Respectfully, that task is clearly on the proponents of L4S and IMHO should have been done before granting RFC status. L4S already hindered one higher resolution ECN approach (SCE) without actual evidence of superiority, let's leave it at that one victim please.
>>>> In my opinion L4S is too little too late, as we already know multi-bit per packet information is superior, so we arguably should focus on that.
>>> 
>>> Sebastian,
>>> 
>>> There's a fundamental problem with mulit-bit per packet information:
>>> there's no place in packets to put the data that will work across the
>>> Internet! (other than the eight bit TOS/Priority field).
>> 
>>        [SM2] Yes, I agree. The most fitting place would be in the IP header, but there we are out of (re-)usable bits...
>> 
>> 
>>> The correct
>>> mechanism for network to hosts signaling is Hop-by-Hop options but
>>> those currently see a 99% drop rate.
>> 
>>        [SM] Again I agree, for IPv6 any of the extension headers would seems the next best place. It should be one that exists along the full path, and is visible by the end points, if Hop-byHop fits that bill, great. Or not so great, 1% passage is sobering even for a glass half full kind of person.
>> 
>> 
>>> There are proposed alternatives
>>> like putting information in transport payload, TCP and now UDP
>>> options, overloading flow label or IPv6 addresses; but all of these
>>> have drawbacks that will preclude widescale deployment (these are
>>> discussed in section 5 of draft-herbert-host2netsig).
>> 
>>        [SM] Yes deployment of any of these including IP6 extension headers is going to be an uphill battle, as the network will only gain little by this, so deployment likely will start in environments where "little" already is worth it, and that I agree is likely not the wider internet. But darn it, it would be great if having to access a service over a congested link that would result in lowish throughput but with acceptable latency, as compared t today's low throughput and high latency and jitter. That might be a way of pitching this, if this allows more graceful operation at/close to saturation this might be interesting for infrastructure folks ()
>> 
>>> Since network to
>>> host signaling modifies data in flight limits the solution space even
>>> more, for instance it might be okay for a switch to read a transport
>>> payload like QUIC spin bit, but writing data in the transport payload
>>> risks systematic data corruption.
>> 
>>        [SM] Well, all handling of packets involved reading and writing memory already, and also changes in header fields is performed routinely, TTL/Hop limit come to mind, as does NATP... so there is ample precedence even if we exclude ECN, no?
>> 
>> 
>>> I agree that multi-bit data should facilitate more precise congestion
>>> control with faster reaction time, but I also think we need that more
>>> in the data center than over the Internet and the problem of defining
>>> a multi-bit packet field for the data is solvable in a limited domain.
>> 
>>        [SM] I respectfully disagree, I do not run a data center, but I do run a home network and use the internet, I think this is a case where all domains would profit. Realistically though I expect this to come to fruition in limited domains/data centers first. Would be sad if it would stay there however.
> 
> 
> Sebastian,
> 
> There may be another option. Fortunately, the definition of a limited
> domain is rather loose. For instance, a provider network could deploy
> switches that support multi-bit congestion signals in HBH options
> could be considered a limited domain. So if a host connects to a
> server in the provider's domain, like a hosted YT server for instance,
> then the signal is viable end-to-end. The provider network could be
> quite large and it could partner with connected networks to extend the
> limited domain. Conceivably, over time the "limited domain" could
> encompass a significant portion of the Internet. This is part of the
> plan to extend the reach FAST.

	[SM2] OK, "starting from a limited domain" seems viable then.


> 
> There's a problem with this though: the signal is only viable in the
> limited domain. For instance, if a packet with HBH options leaves the
> limited domain then there's a high probability the packet will be
> dropped. There's no easy means for a host to determine if it's sending
> within a limited domain, so the sending hosts can't decide on using
> EH.

	[SM2] I respectfully disagree, the signal clearly should be end2end and be part of the negotiation then, so endpoints check whether the signal passes between the endpoints and disale its usage if they do not.


> The solution we're proposing is to remove the bits when packets
> leave the limited domain at an egress router.

	[SM2] That I disagree with, this needs to be end2end, unless your proposal entails the end points being informed about that removal.


> as described in
> draft-herbert-eh-inflight-removal. This also has some nice security
> properties and coincides with existing network firewalls.

	[SM2] Will need to read that, when I find time.

Regards
	Sebastian


> 
> Tom
> 
>> 
>> 
>> 
>>> 
>>> Tom
>>> 
>>>> 
>>>> Regards
>>>>        Sebastian
>>>> 
>>>> 
>>>>> 
>>>>> /Ingemar
>>>>> 
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: tsvwg <[e-mail]> On Behalf Of Christian Huitema
>>>>>> Sent: Tuesday, 17 October 2023 17:28
>>>>>> To: Sebastian Moeller <[e-mail]>; [e-mail]
>>>>>> Cc: [e-mail]
>>>>>> Subject: [tsvwg] Network feedback and security (was: New Version
>>>>>> Notification for draft-huang-tsvwg-transport-challenges-00.txt)
>>>>>> 
>>>>>> 
>>>>>> On 10/17/2023 3:17 AM, Sebastian Moeller wrote:
>>>>>>> Personally, ever since I read Arslan, Serhat, and Nick McKeown.
>>>>>> ‘Switches Know the Exact Amount of Congestion’. In Proceedings of the
>>>>>> 2019 Workshop on Buffer Sizing, 1–6, 2019. I came to the prediction that
>>>>>> something like max(bufferoccupancy) over a network path in either
>>>>>> predicted sojourntime or percentual buffer filling could really improve
>>>>>> congestion control if included in all packets. That paper and follow ups
>>>>>> on that idea are to my quite convincing. I see no real security concern
>>>>>> as the network node adding this information might as well have dropped
>>>>>> the packet if it wanted to harm that flow, so little increase in attack
>>>>>> surface in that direction. What I hope something like this might allow
>>>>>> is a gentler exist from slow start (assuming one can measure and compare
>>>>>> the dynamics of the congestion indicator with those of the slow-starting
>>>>>> flow to predict when to exist slow start without first having to dump ~2
>>>>>> too much data into the network in one RTT). The network could indirectly
>>>>>> profit if end-points use this information to better reign in congestion
>>>>>> pro-actively, but that clearly is not guaranteed, especially over the
>>>>>> open internet.
>>>>>> 
>>>>>> Yes, network feedback is useful. You mention "max(bufferoccupancy) over
>>>>>> a network path" as a signal, but I would argue that ECN is pretty much a
>>>>>> 1 bit version of exactly that. So your argument is really for "more ECN
>>>>>> bits". That's plausible, but the whole "L4S/Prague" effort seems to
>>>>>> indicate that 1 bit is enough, because you can observe that bit on many
>>>>>> consecutive packets. The conservative side of me would say we should
>>>>>> first deploy the 1 bit version, and then study whether we really need
>>>>>> many more bits.
>>>>>> 
>>>>>> The "gentler exit from slow start" issue is largely addressed by
>>>>>> Hystart, which uses the variations of the RTT as a signal. And yes,
>>>>>> Hystart can be improved by also monitoring the ECN bits, not just the
>>>>>> RTT.
>>>>>> 
>>>>>> But there are in fact security concerns. We already know about ICMP
>>>>>> attacks in which attackers spoof the network feedback and inject fake
>>>>>> ICMP packets, for example to disrupt PMTU discovery. "Man on the side"
>>>>>> attackers can easily send a second copy of a packet with modified header
>>>>>> bits and race it to the destination. If transports accept that copy as
>>>>>> genuine, they will heed the faked congestion signals and the attacker
>>>>>> will succeed in disrupting the connection.
>>>>>> 
>>>>>> -- Christian Huitema
>>>> 
>>>> --
>>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>>> 
>>