Re: [tsvwg] I-D Action: draft-ietf-tsvwg-udp-options-23.txt - what is the intended use of APC

[Sebastian Moeller <[e-mail]>]

Hi Gory,

> On Sep 27, 2023, at 11:53, Gorry Fairhurst <[e-mail]> wrote:
> 
> I think I better understand the design options for APC, thanks Joe.
> 
> Let's see if I can work out the 2 candidate use-cases:
> 
> 1. Replacement for the UDP checksum.
> 
> This has requirements:
> - a CRC is needed.
> - a tag/identifier is also needed to (probabilistically) confirmthe flow demux (as in the VTAG in SCTP). This is just an ID i(t could be set randomly or could be a hash of the port/address info at the sender). At the receiver an app checks the ID is consistent, to avoid accepting data that was mapped to the wrong port. 
> - In UDP-Opt this implies an UNSAFE option and as Joe notes the packet needs to be placed in the surplus area.
> - There is a rule that states: "UNSAFE options MUST be used only with the FRAG option" ??? *
> - Apps do still receive null packets when there the payload is corrupted, but not the payload.
> 
> If it wishes this, a sender can then disable the UDP checksum and still have a meaningful integrity check.
> 
> 2. Additional protection above the UDP Checksum
> 
> This has requirements:
> - a CRC is needed.
> - a checksum is also still needed and avoids accepting data that was mapped to the wrong port. 
> - Apps receive packets when there the payload is corrupted, and need to check the CRC value when present and differentiate packets with no CRC option, as in a legacy receiver  (the receiver likely has to discarding these or tell the sender that the CRC was not present if this was needed for the app).
> 
> 
> The current text seems to need a small amount of work either way, and as Tom says, you could allow both options, if we think both have utility.
> 
> When we experimented with our code, my thoughts were more like (1), where the UDP receiver only sent app data that was valid. The WG could do (2), although to me at least, it seems like it might be better to embed this integrity check in the application data where it can be associated with the block data it is protecting.
> 
> * I think I am missing something: I don't understand why the spec says that UNSAFE options MUST be used only with the FRAG option, and therefore why UENC and perhaps this APC option  would need to have a FRAG option?

 UNSAFE options are not safe to ignore and can be used
   unidirectionally or without soft-state confirmation of UDP option
   capability. They are always used only when the user data occurs
   inside a reassembled set of one or more UDP fragments, such that if
   UDP fragmentation is not supported, the enclosed UDP user data would
   be silently dropped anyway.


I think this is to ensure that legacy apps see reliably 0 size packets instead of seeing the real data and being able to ignore the unsafe option...


Regards
	Sebastian



> 
> Gorry
> 
> 
> On 27/09/2023 01:32, [e-mail] wrote:
>> If that’s what the WG wants, sure - but note that UNSAFE can’t be used without FRAG.
>> 
>> But even with FRAG, if the APC fails, the receiver STILL gets a packet (just an empty payload one).
>> 
>> It HAS to behave like it would for a legacy receiver.
>> 
>> Joe
>> —
>> Dr. Joe Touch, temporal epistemologist
>> www.strayalpha.com
>> 
>>> On Sep 26, 2023, at 5:06 PM, Tom Herbert <tom=[e-mail]> wrote:
>>> 
>>> A simple answer to this would just be to define an UNSAFE variant of APC along side the SAFE variant and let the user decide what to use.
>>> 
>>> Tom
>>> 
>>> 
>>> On Tue, Sep 26, 2023, 4:52 PM Tom Herbert <[e-mail]> wrote:
>>> On Tue, Sep 26, 2023 at 4:05 PM Joe Touch <[e-mail]> wrote:
>>> >
>>> > The rules require that, for safe option, the receiver decides what it accepts it accepts.
>>> >
>>> > Different receivers are absolutely allowed to have different views. The default MUST be legacy receivers do”.
>>> >
>>> > That’s not at all analogous to how quantum mechanics works, btw.
>>> 
>>> Joe,
>>> 
>>> It is, except for the fact that an external observer can intercept the
>>> packet with the CRC and just based on the contents to determine if the
>>> packet is corrupted. In that case there is *only* *one* right answer
>>> that can be deduced solely based on the rules governing the system and
>>> no observer bias.
>>> 
>>> >From a more practical perspective, the draft: " The Alternate Payload
>>> Checksum (APC, Kind=2) option provides a stronger alternative to the
>>> checksum". Whether it's stronger than checksum seems to be conditional
>>> on whether the receiver processes the option. If a receiver just
>>> ignores the option then clearly the checksum is stronger. The problem
>>> is there is no mechanism for a sender to know whether the CRC is going
>>> to be ignored by the receiver if it is a SAFE option. If I were
>>> deploying a  UDP application in a network that is sensitive to data
>>> corruption, I really want a CRC that is guaranteed to work. The rule
>>> is simple: either the receiver validates that CRC is correct or the
>>> packet is dropped. If UDP options allow a third possibility where the
>>> packet could be accepted even though it's corrupted then I wouldn't
>>> use APC. I would try to get the required behavior by putting the CRC
>>> in the application data (which a lot of application layer protocols do
>>> anyway).
>>> 
>>> Tom
>>> 
>>> 
>>> 
>>> 
>>> >
>>> > Joe
>>> >
>>> > > On Sep 26, 2023, at 1:13 PM, Tom Herbert <[e-mail]> wrote:
>>> > >
>>> > > On Tue, Sep 26, 2023 at 12:36 PM [e-mail]
>>> > > <[e-mail]> wrote:
>>> > >>
>>> > >> Hi, Tom,
>>> > >>
>>> > >> Remember that (as currently defined) APC succces/failure is passed to the application. An application that wants to enforce it can. It’s not simply ignored. The receiver doesn’t decide whether to check APC or not, it (currently) decides what to do if/when APS fails.
>>> > >>
>>> > >> Yes, this allows two endpoints to act differently. There are a variety of reasons why this could be the case - e.g., APC would fail if there were bit errors, but it would also fail if the data were somehow otherwise manipulated by a node that doesn’t support UDP options, e.g., for port rewriting. An endpoint that accepts and allows that can do what it wants.
>>> > >>
>>> > >
>>> > > So rules of the protocol allow the same packet to simultaneously be
>>> > > considered both corrupted and uncorrupted, it is only when a biased
>>> > > receiver observes the packet that we decide which one it is. I'm going
>>> > > to call that Schrodinger's packet :-)
>>> > >
>>> > >> That’s not non-determinism; it’s self-determinism. Again, this is a USER protocol and the user decides.
>>> > >>
>>> > >> However, the current definition of all SAFE options *is based on acting like a legacy node* unless explicitly configured otherwise. That is the definition of SAFE.
>>> > >>
>>> > >> Joe
>>> > >>
>>> > >> —
>>> > >> Dr. Joe Touch, temporal epistemologist
>>> > >> www.strayalpha.com
>>> > >>
>>> > >> On Sep 26, 2023, at 10:52 AM, Tom Herbert <[e-mail]> wrote:
>>> > >>
>>> > >> On Tue, Sep 26, 2023 at 8:49 AM Joe Touch <[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >> I disagree with use of unsafe option formats simply to force option support.
>>> > >>
>>> > >>
>>> > >> Joe,
>>> > >>
>>> > >> It's not simply to force option support, it's to enforce correctness
>>> > >> and determinism of the protocol. If a packet contains a CRC that isn't
>>> > >> checked and the packet data is corrupted such that data is accepted
>>> > >> that would otherwise have been dropped had the CRC been checked, then
>>> > >> I claim the packet has not been processed correctly. Furthermore, the
>>> > >> draft allows that two nodes could receive the exact same UDP options
>>> > >> and UDP payload in a packet where one of them drops the packet as
>>> > >> being corrupted and the other accepts the packet. Which is right? If
>>> > >> you say they are both right then I claim UDP Options is a
>>> > >> non-deterministic protocol .
>>> > >>
>>> > >> Tom
>>> > >>
>>> > >> On Mon, Sep 25, 2023 at 11:58 PM Gorry Fairhurst  wrote:
>>> > >>
>>> > >>
>>> > >> On 26/09/2023 05:30, Joe TOuch wrote:
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Sep 25, 2023, at 7:36 PM, Tom Herbert <tom=[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Mon, Sep 25, 2023, 7:35 PM Tom Herbert <[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Mon, Sep 25, 2023, 5:31 PM [e-mail] <[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >> See below.
>>> > >>
>>> > >> —
>>> > >> Dr. Joe Touch, temporal epistemologist
>>> > >> www.strayalpha.com
>>> > >>
>>> > >> On Sep 25, 2023, at 5:21 PM, Tom Herbert <tom=[e-mail]> wrote:
>>> > >>
>>> > >> On Mon, Sep 25, 2023 at 5:08 PM [e-mail]
>>> > >> <[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >> On Sep 25, 2023, at 4:14 PM, Tom Herbert <tom=[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Mon, Sep 25, 2023, 3:55 PM [e-mail] <[e-mail]> wrote:
>>> > >>
>>> > >>
>>> > >> FWIW,  you can also “require ACS” at the receiver, if you want. That would ensure that ACS was both present and valid.
>>> > >>
>>> > >> UDP options are based on the core rule that senders decide what to offer and receivers decide what’s required.
>>> > >>
>>> > >>
>>> > >> Joe,
>>> > >>
>>> > >> For something like CRC, the sender is in a much better position to decide if it's needed.
>>> > >>
>>> > >>
>>> > >> Whether it is or is not, that’s not something UDP options supports - what UDP supports is based on a bunch of principles, including the need to support legacy receivers. So if something is “required”, the receiver has to demand and check for it.
>>> > >>
>>> > >> For example, if two hosts were communicating over a rack switch, the ACS is probably pointless since Ethernet CRC is sufficient. But, if the hosts were communicating over an unreliable satellite link then ACS might be critical.
>>> > >>
>>> > >>
>>> > >> Or the link/phy layer might have its own checks anyway, e.g., block checksums. End to end errors happen during reassembly or in memory, not all that often on links that aren’t checked, AFAIR.
>>> > >>
>>> > >> In both those cases, it's the sender that has the best view as to what's required because it may require knowledge of the path to the destination. So, if the sender computes the ACS, then the receiver should honor that and verify it.
>>> > >>
>>> > >>
>>> > >> In some other protocol where those semantics can be demanded, perhaps. But remember that anything the sender “demands”, except for encryption, is at best a “request”; receivers always get to decide what they enforce anyway.
>>> > >>
>>> > >>
>>> > >> Joe,
>>> > >>
>>> > >> I believe the general consensus in IETF is that checksum or CRCs may
>>> > >> be optional to send, but not optional to validate when received. The
>>> > >> precedent was established in RFC1122: "If a UDP datagram is received
>>> > >> with a checksum that is non-zero and invalid, UDP MUST silently
>>> > >> discard the datagram."
>>> > >>
>>> > >>
>>> > >> That’s for checksums you know are coming. A legacy receiver won’t know or care.
>>> > >>
>>> > >> UDP option-aware receivers work like legacy receivers UNTIL they are explicitly told otherwise. That’s the basic operational principle.
>>> > >>
>>> > >> So you can tell your receiver to care about ACS if it shows up all the time. Or not.
>>> > >>
>>> > >> Here’s the current text, which follows these principles:
>>> > >>
>>> > >> UDP packets with incorrect APC checksums MUST be passed to the
>>> > >>
>>> > >>  application by default, e.g., with a flag indicating APC failure.
>>> > >>
>>> > >>  Like all SAFE UDP options, APC needs to be silently ignored when
>>> > >>  failing by default, unless the receiver has been configured to do
>>> > >>  otherwise. Although all UDP option-aware endpoints support APC
>>> > >>  (being in the required set), this silently-ignored behavior ensures
>>> > >>  that option-aware receivers operate the same as legacy receivers
>>> > >>  unless overridden.
>>> > >>
>>> > >> For user perspective, if I set the ACS in my packets but the receiver
>>> > >> may or may not validate it then it doesn't seem very useful as a check
>>> > >> for corruption of the sender's data.
>>> > >>
>>> > >>
>>> > >> It is useful when the receiver agrees it is useful.
>>> > >>
>>> > >> I'd probably put a CRC in the UDP
>>> > >> payload instead that I can assure it is always verified by the
>>> > >> receiving application.
>>> > >>
>>> > >>
>>> > >> You can do that if you want, but then it means that it cannot be silently ignored by a legacy receiver.
>>> > >>
>>> > >>
>>> > >>
>>> > >> Right, that's exactly the desired non-deterministic
>>> > >>
>>> > >>
>>> > >>
>>> > >> Deterministic  I meant
>>> > >>
>>> > >> behavior that we want with regards to a CRC. Silently ignoring the CRC also means that data corruption would also be silently ignored.
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >> So you have two choices:
>>> > >>
>>> > >> 1. Use UDP options with ACS (which is now APC)
>>> > >> a cooperating receiver would know to set APC as required and would drop packets if APC fails
>>> > >>
>>> > >> 2. Create a new application that knows to require APC
>>> > >> so that the application can enforce CRC in the payload
>>> > >>
>>> > >> But if you can do (2), why not do (1) and have the application enforce APC-required?
>>> > >>
>>> > >> Joe
>>> > >>
>>> > >> I'm reading this with some doubt now.
>>> > >>
>>> > >> I think option the above option 2 needs to be strongly discouraged: it provides a place to send an integrity check, but it doesn't protect the parsing of the other options nor their data, which would leave all other fields unprotected. That check would better be provided within the application data, and we ought to remove this option if that is all it does.
>>> > >>
>>> > >> Here is what I thought had been designed:
>>> > >>
>>> > >> I expect a sender chooses whether to add an APC option.
>>> > >>
>>> > >> If a receiver receives the APC option, the receiver must to either:
>>> > >>
>>> > >> - discard all datagrams that contain an APC that it decides not to check, or when the APC fails.
>>> > >>
>>> > >> - forward datagrams that contain an APC that passes. (It MIGHT be configured to always expect an APC: which I think is the ONLY receiver decision).
>>> > >>
>>> > >> ... This differs from the benign "use or ignore" processing that applies to most options, but it is exactly the required logic that I would expect and I thought we previously discussed.
>>> > >>
>>> > >>
>>> > >>
>>> > >> Gorry and Tom, the behaviour for which you advocate COULD be achieved if, instead of a SAFE option APC, we specify instead an UNSAFE option UAPC, and stipulate in its definition that the packet is discarded if the integrity check fails. In that case:
>>> > >>
>>> > >> The option, being UNSAFE, would be sent only in the options trailer of a fragmented packet (I refer here to UDP fragmentation, not IP fragmentation). Having the packet encapsulated in (one or more) UDP fragments would ensure that a legacy receiver would not process the packet at all, and the fact that the option is UNSAFE would ensure that an options-aware receiver that does not implement it would discard the packet.
>>> > >> An options-aware receiver that does understand the option would, after successful reassembly, check that to see if the UAPC is correct and would discard the packet if it is not.
>>> > >>
>>> > >> This would somewhat expand the usage of UNSAFE from "options that modify the user data" to "options that must be understood in order to process the user data as intended." In my view (which I know is not universal) this expansion is likely necessary to cover use cases that will actually arise; the proposed protocol number in draft-daiya-tsvwg-udp-options-protocol-number seems to me to be one such use case.
>>> > >>
>>> > >> Mike Heard
>>> > >>
>>> > >>
>>> > >
>>> >
>> 
> 
>