Re: [tsvwg] signaling packet importance [was Re: New Version Notification for draft-herbert-fast]

[Tom Herbert <[e-mail]>]

On Wed, Aug 9, 2023, 2:22 PM Dan Wing <[e-mail]> wrote:

> Yes, there is a lot of interest in marking 'important' packets.  Thanks
> for your comments on the list and at the meeting and thanks for updating
> FAST.  Thanks to Mike for being a nudnik and poking all the co-authors
> earlier this week, too.
>

Hi Dan,

Thanks for the great discussion! Comments in line.


>
>
> Regarding inapplicability of UDP trailers to TCP, TCP has head of line
> blocking so marking a packet within a TCP packet differently from other
> packets in that same flow provides no benefit to the flow.
>

Not necessarily. If a host sees a connection is having problems, like
seeing retransmission or high variance in RTT, it's their prerogative to
try to affect routing to fix the problem. For instance, this is implemented
in Linux as a feature where the IPv6 flow label can be changed mid-flow to
see if there's a better route. We can do the same thing for signaling in
HBH to affect QoS. While we don't want to be sending every packet
differently, the ability to change things occasionally for a TCP connection
could be very beneficial.

A *separate* TCP flow could get marked differently (e.g., a file transfer
> TCP flow versus an interactive audio/video TCP flow), but that differential
> marking is only possible with multiple TCP connections -- causing its
> drawbacks of NAPT port consumption, connection set-up time, and
> per-connection congestion control.  Those drawbacks were some of the
> justification for creating QUIC, and resolved by QUIC.
>

Right, connection setup/teardown is expensive which is why there's
incentive to try to save failing connections as I described above.


> The problem user "A" will mark all their packets as important starving
> user "B" (the "everyone is an ambulance" problem) requires some sort of
> permission (or quota or "admission") system -- like FAST.  But that is a
> problem no matter if IPv6 HbH option, UDP option, DSCP, RSVP, etc.
>

Yep, that's a fundamental problem in host network signaling! This also
where the ticket analogy is helpful, if concert tickets didn't have
features to prevent forgery then everyone would be sitting in the front row
for the Taylor Swift concert :-)


>
> Tom Herbert wrote:
>
> Can you comment on the requirements and pragmatics of network devices
> to process UDP Options. As I mentioned, high performance network
> devices process protocol _headers_, it's going to be difficult to
> teach them to process trailers efficiently.
>
>
> The IP header length and UDP header length are examined to determine if
> there is a UDP trailer.  Examining the trailer can be helped by encoding
> "backwards" to ease parsing as was done by SRTP EKT,
> https://datatracker.ietf.org/doc/html/rfc8870#section-4.1.
>

But I believe that's an E2E protocol. Do you have any examples of protocols
with trailers that are explicitly intended to be processed by intermediate
nodes?

Middle boxes, especially hardware implementation, aren't designed to handle
trailers. They may have to defer processing of UDP Options to a software,
which as we've seen, time and time, again dooms a data path protocol to the
bit bucket! Hosts probably can process trailers, but it's still not nearly
as efficient as just processing headers; and host devices with programmable
datapath, SmartNICs like P4 programmable programambaility, probably won't
be able to process trailers without some major hardware changes. UDP
Options might be functional on the Internet, but it's yet to be proven they
can be implemented to be sufficiently performant.


> I agree UDP trailers are not headers, and I agree network devices are
> quite used to dealing with headers.
>

> The differentiated packet handling would occur near the edge of the
> network close to the subscriber.  In the case of Wi-Fi it occurs on the
> subscriber's premises; in the case of 5G it is the radio access network.
> Those are where queues build up (due to congestion caused by other users)
> and where radio interference occurs.  Those devices can improve the user
> experience by dropping less-important packets in favor of more-important
> packets.  Today, those devices have little-to-no idea which packets are
> important.  They know TCP packets will eventually get re-transmitted, and
> they can make guesses that UDP/443 is carrying QUIC and -- today -- has
> TCP-like behavior and will also re-transmit packets. But as unreliable QUIC
> [RFC9221] becomes A Thing, the existing treatment of QUIC by those devices
> will cause harm.  I believe it is use-cases for unreliable QUIC that is
> what is causing the influx of new IETF proposals.  Of course, (S)RTP has
> long wanted this sort of behavior, too, long prior to unreliable QUIC --
> it's just been less interesting as the likes of Zoom, Cisco WebEx, Teams,
> and Skype have tried to quietly just get their job done.  But they, too,
> could benefit from UDP trailers.
>

Reliable protocols need QOS just like any other protocols. Just relying on
TCP or QUIC retransmissions would set an awfully low bar for QoS! Customers
want latency/throughput/jitter guarantees. And if someone is seeing a lot
of retransmissions, that's more likely to be a bad thing than a good thing
and some action is needed. In E2E protocols, only end hosts have the state
to know what's happening to connections, but it's really the network where
the problems are and where things can be fixed (like trying to influence
routing like described above). That's exactly why we need host network
signaling-- the host has the E2E state and understanding of application
requirements; the network has the mechanisms and services to satisfy those
requirements.


> If we use fragment options
>
> to force the UDP Options into headers then the problem becomes that we
> have to change all end hosts to support that. There's also potential
> problems mixing transport and network options, and in particular this
> could affect security and DoS considerations.
>
>
> Changing OS stacks is already necessary for UDP trailers:  the user- and
> kernel-mode interfaces to UDP don't provide the flexibility to add a UDP
> trailer (on Windows); instead, raw sockets are necessary.  Once the entire
> packet is built by hand with raw sockets, fragment options become viable.
> That said, I agree we don't want to make changes to host stacks -- and is
> one thing I don't like about UDP trailers!
>

Yes, frankly that has long been a concern of mine wrt UDP Options. We
haven't seen a lot of running code or deployment. I believe there is a
FreeBSD patch, but I haven't seen patches sent to netdev for Linux. Neither
do I know of any active deployment of UDP Options. And this concern is only
about the current use case of transport options, using UDP Options for
network consumption hasn't even been brought up until recently. This
situation can be contrasted to a protocol like Segment Routing that
referenced at least ten implementations, had Linux patches accepted
upstream, and I believe there was significant deployment when SR was
published as RFC-- see
https://datatracker.ietf.org/doc/html/draft-matsushima-spring-srv6-deployment-status-15
for a great example of demonstaring the "running code" requirement of IETF;
it is quite thorough in describing the implementation, deployment, and
interoperability status of segment routing (I really wish there was an
equivalent for UDP Options especially if their scope is being expanded to
be processed by network devices).


>
> On survivability of UDP trailers:  the sender can probe to determine if
> UDP trailers survive a path, and avoid sending them the UDP trailer causes
> failure to receive the packet.  This works because the UDP trailer is an
> optimization to attempt to help the network deliver improved service.  We
> have do similar probing for IPv6/IPv4 (Happy Eyeballs), TCP/QUIC, and a
> slew of other protocols as they are pushed out.  Such probing works fine
> for UDP trailers because they are unlikely to be dropped on the Internet
> (that is, on transit networks) because security devices and UDP checksum
> validation does not occur on such transit networks.
>
Rather, networks close to the subscriber is where security devices and
> checksum validation might occur and those networks close to the subscriber
> are the very same networks where packet priority is more interesting
> because it's the same place queues build up and packet importance would
> influence dropping less-important packets.  This creates alignment of the
> network operator wanting the benefit of UDP trailer (to improve user
> experience) who can also influence their security vendor and their UDP
> checksum validation device to permit UDP trailers.
>

That same logic can be applied to HBH Options. The ability of operators to
offer differentiated services and monetize them, should be sufficient
incentive (alignment ) to fix their bugs and make host to network signaling
work HBH Options, UDP Options, etc.


The "don't care about UDP trailers" behavior of transit networks contrasts
> from IPv6 HbH options where the transit networks are, today, interfering
> with IPv6 HbH options, and do not have an aligned incentive to change their
> IPv6 HbH operations.
>

Probing can be done for HBH Options as well (see draft). There are some
differences. UDP Options probing is for establishing feasibility across the
path as well as the end host, HBH options probing and Happy Eyeballs are
really only probing the path. One likely difference in practice is that UDP
Options probing probably needs to be done on a 4-tuple basis because the
end host might have an Anycast address that is behind a load balancer
(might be a minor point, but will require testing and consideration to
implement a robust probing algorithm).

Without any deployment of UDP Options and not a lot of implementations, I'm
quite skeptical of any statement that UDP Options are any more survivable
than HBH Options. We don't really know. And, even if it were true today,
there's a very real possibility the providers will block them if they sense
the slightest security issue, especially if they can't parse these because
they are trailers (i.e. trailers might be perceived to be a covert channel
to firewalls that can't process trailers).

Tom




> -d
>
>