Re: [tsvwg] I-D Action: draft-ietf-tsvwg-udp-options-23.txt - what is the intended use of APC

[Gorry Fairhurst <[e-mail]>]

I think I better understand the design options for APC, thanks Joe.

Let's see if I can work out the 2 candidate use-cases:

1. Replacement for the UDP checksum.

This has requirements:
- a CRC is needed.
- a tag/identifier is also needed to (probabilistically) confirmthe flow 
demux (as in the VTAG in SCTP). This is just an ID i(t could be set 
randomly or could be a hash of the port/address info at the sender). At 
the receiver an app checks the ID is consistent, to avoid accepting data 
that was mapped to the wrong port.
- In UDP-Opt this implies an UNSAFE option and as Joe notes the packet 
needs to be placed in the surplus area.
- There is a rule that states: "UNSAFE options MUST be used only with 
the FRAG option" ??? *
- Apps do still receive null packets when there the payload is 
corrupted, but not the payload.

If it wishes this, a sender can then disable the UDP checksum and still 
have a meaningful integrity check.

2. Additional protection above the UDP Checksum

This has requirements:
- a CRC is needed.
- a checksum is also still needed and avoids accepting data that was 
mapped to the wrong port.
- Apps receive packets when there the payload is corrupted, and need to 
check the CRC value when present and differentiate packets with no CRC 
option, as in a legacy receiver  (the receiver likely has to discarding 
these or tell the sender that the CRC was not present if this was needed 
for the app).


The current text seems to need a small amount of work either way, and as 
Tom says, you could allow both options, if we think both have utility.

When we experimented with our code, my thoughts were more like (1), 
where the UDP receiver only sent app data that was valid. The WG could 
do (2), although to me at least, it seems like it might be better to 
embed this integrity check in the application data where it can be 
associated with the block data it is protecting.

* I think I am missing something: I don't understand why the spec says 
that UNSAFE options MUST be used only with the FRAG option, and 
therefore why UENC and perhaps this APC option  would need to have a 
FRAG option?

Gorry


On 27/09/2023 01:32, [e-mail] wrote:
> If that’s what the WG wants, sure - but note that UNSAFE can’t be used 
> without FRAG.
>
> But even with FRAG, if the APC fails, the receiver STILL gets a packet 
> (just an empty payload one).
>
> It HAS to behave like it would for a legacy receiver.
>
> Joe
> —
> Dr. Joe Touch, temporal epistemologist
> www.strayalpha.com
>
>> On Sep 26, 2023, at 5:06 PM, Tom Herbert 
>> <tom=[e-mail]> wrote:
>>
>> A simple answer to this would just be to define an UNSAFE variant of 
>> APC along side the SAFE variant and let the user decide what to use.
>>
>> Tom
>>
>>
>> On Tue, Sep 26, 2023, 4:52 PM Tom Herbert <[e-mail]> wrote:
>>
>>     On Tue, Sep 26, 2023 at 4:05 PM Joe Touch <[e-mail]>
>>     wrote:
>>     >
>>     > The rules require that, for safe option, the receiver decides
>>     what it accepts it accepts.
>>     >
>>     > Different receivers are absolutely allowed to have different
>>     views. The default MUST be legacy receivers do”.
>>     >
>>     > That’s not at all analogous to how quantum mechanics works, btw.
>>
>>     Joe,
>>
>>     It is, except for the fact that an external observer can
>>     intercept the
>>     packet with the CRC and just based on the contents to determine
>>     if the
>>     packet is corrupted. In that case there is *only* *one* right answer
>>     that can be deduced solely based on the rules governing the
>>     system and
>>     no observer bias.
>>
>>     >From a more practical perspective, the draft: " The Alternate
>>     Payload
>>     Checksum (APC, Kind=2) option provides a stronger alternative to the
>>     checksum". Whether it's stronger than checksum seems to be
>>     conditional
>>     on whether the receiver processes the option. If a receiver just
>>     ignores the option then clearly the checksum is stronger. The problem
>>     is there is no mechanism for a sender to know whether the CRC is
>>     going
>>     to be ignored by the receiver if it is a SAFE option. If I were
>>     deploying a  UDP application in a network that is sensitive to data
>>     corruption, I really want a CRC that is guaranteed to work. The rule
>>     is simple: either the receiver validates that CRC is correct or the
>>     packet is dropped. If UDP options allow a third possibility where the
>>     packet could be accepted even though it's corrupted then I wouldn't
>>     use APC. I would try to get the required behavior by putting the CRC
>>     in the application data (which a lot of application layer
>>     protocols do
>>     anyway).
>>
>>     Tom
>>
>>
>>
>>
>>     >
>>     > Joe
>>     >
>>     > > On Sep 26, 2023, at 1:13 PM, Tom Herbert
>>     <[e-mail]> wrote:
>>     > >
>>     > > On Tue, Sep 26, 2023 at 12:36 PM [e-mail]
>>     > > <[e-mail]> wrote:
>>     > >>
>>     > >> Hi, Tom,
>>     > >>
>>     > >> Remember that (as currently defined) APC succces/failure is
>>     passed to the application. An application that wants to enforce
>>     it can. It’s not simply ignored. The receiver doesn’t decide
>>     whether to check APC or not, it (currently) decides what to do
>>     if/when APS fails.
>>     > >>
>>     > >> Yes, this allows two endpoints to act differently. There are
>>     a variety of reasons why this could be the case - e.g., APC would
>>     fail if there were bit errors, but it would also fail if the data
>>     were somehow otherwise manipulated by a node that doesn’t support
>>     UDP options, e.g., for port rewriting. An endpoint that accepts
>>     and allows that can do what it wants.
>>     > >>
>>     > >
>>     > > So rules of the protocol allow the same packet to
>>     simultaneously be
>>     > > considered both corrupted and uncorrupted, it is only when a
>>     biased
>>     > > receiver observes the packet that we decide which one it is.
>>     I'm going
>>     > > to call that Schrodinger's packet :-)
>>     > >
>>     > >> That’s not non-determinism; it’s self-determinism. Again,
>>     this is a USER protocol and the user decides.
>>     > >>
>>     > >> However, the current definition of all SAFE options *is
>>     based on acting like a legacy node* unless explicitly configured
>>     otherwise. That is the definition of SAFE.
>>     > >>
>>     > >> Joe
>>     > >>
>>     > >> —
>>     > >> Dr. Joe Touch, temporal epistemologist
>>     > >> www.strayalpha.com <http://www.strayalpha.com/>
>>     > >>
>>     > >> On Sep 26, 2023, at 10:52 AM, Tom Herbert
>>     <[e-mail]> wrote:
>>     > >>
>>     > >> On Tue, Sep 26, 2023 at 8:49 AM Joe Touch
>>     <[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >> I disagree with use of unsafe option formats simply to force
>>     option support.
>>     > >>
>>     > >>
>>     > >> Joe,
>>     > >>
>>     > >> It's not simply to force option support, it's to enforce
>>     correctness
>>     > >> and determinism of the protocol. If a packet contains a CRC
>>     that isn't
>>     > >> checked and the packet data is corrupted such that data is
>>     accepted
>>     > >> that would otherwise have been dropped had the CRC been
>>     checked, then
>>     > >> I claim the packet has not been processed correctly.
>>     Furthermore, the
>>     > >> draft allows that two nodes could receive the exact same UDP
>>     options
>>     > >> and UDP payload in a packet where one of them drops the
>>     packet as
>>     > >> being corrupted and the other accepts the packet. Which is
>>     right? If
>>     > >> you say they are both right then I claim UDP Options is a
>>     > >> non-deterministic protocol .
>>     > >>
>>     > >> Tom
>>     > >>
>>     > >> On Mon, Sep 25, 2023 at 11:58 PM Gorry Fairhurst  wrote:
>>     > >>
>>     > >>
>>     > >> On 26/09/2023 05:30, Joe TOuch wrote:
>>     > >>
>>     > >>
>>     > >>
>>     > >> On Sep 25, 2023, at 7:36 PM, Tom Herbert
>>     <tom=[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >>
>>     > >> On Mon, Sep 25, 2023, 7:35 PM Tom Herbert
>>     <[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >>
>>     > >>
>>     > >> On Mon, Sep 25, 2023, 5:31 PM [e-mail]
>>     <[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >> See below.
>>     > >>
>>     > >> —
>>     > >> Dr. Joe Touch, temporal epistemologist
>>     > >> www.strayalpha.com <http://www.strayalpha.com/>
>>     > >>
>>     > >> On Sep 25, 2023, at 5:21 PM, Tom Herbert
>>     <tom=[e-mail]> wrote:
>>     > >>
>>     > >> On Mon, Sep 25, 2023 at 5:08 PM [e-mail]
>>     > >> <[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >> On Sep 25, 2023, at 4:14 PM, Tom Herbert
>>     <tom=[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >>
>>     > >> On Mon, Sep 25, 2023, 3:55 PM [e-mail]
>>     <[e-mail]> wrote:
>>     > >>
>>     > >>
>>     > >> FWIW,  you can also “require ACS” at the receiver, if you
>>     want. That would ensure that ACS was both present and valid.
>>     > >>
>>     > >> UDP options are based on the core rule that senders decide
>>     what to offer and receivers decide what’s required.
>>     > >>
>>     > >>
>>     > >> Joe,
>>     > >>
>>     > >> For something like CRC, the sender is in a much better
>>     position to decide if it's needed.
>>     > >>
>>     > >>
>>     > >> Whether it is or is not, that’s not something UDP options
>>     supports - what UDP supports is based on a bunch of principles,
>>     including the need to support legacy receivers. So if something
>>     is “required”, the receiver has to demand and check for it.
>>     > >>
>>     > >> For example, if two hosts were communicating over a rack
>>     switch, the ACS is probably pointless since Ethernet CRC is
>>     sufficient. But, if the hosts were communicating over an
>>     unreliable satellite link then ACS might be critical.
>>     > >>
>>     > >>
>>     > >> Or the link/phy layer might have its own checks anyway,
>>     e.g., block checksums. End to end errors happen during reassembly
>>     or in memory, not all that often on links that aren’t checked, AFAIR.
>>     > >>
>>     > >> In both those cases, it's the sender that has the best view
>>     as to what's required because it may require knowledge of the
>>     path to the destination. So, if the sender computes the ACS, then
>>     the receiver should honor that and verify it.
>>     > >>
>>     > >>
>>     > >> In some other protocol where those semantics can be
>>     demanded, perhaps. But remember that anything the sender
>>     “demands”, except for encryption, is at best a “request”;
>>     receivers always get to decide what they enforce anyway.
>>     > >>
>>     > >>
>>     > >> Joe,
>>     > >>
>>     > >> I believe the general consensus in IETF is that checksum or
>>     CRCs may
>>     > >> be optional to send, but not optional to validate when
>>     received. The
>>     > >> precedent was established in RFC1122: "If a UDP datagram is
>>     received
>>     > >> with a checksum that is non-zero and invalid, UDP MUST silently
>>     > >> discard the datagram."
>>     > >>
>>     > >>
>>     > >> That’s for checksums you know are coming. A legacy receiver
>>     won’t know or care.
>>     > >>
>>     > >> UDP option-aware receivers work like legacy receivers UNTIL
>>     they are explicitly told otherwise. That’s the basic operational
>>     principle.
>>     > >>
>>     > >> So you can tell your receiver to care about ACS if it shows
>>     up all the time. Or not.
>>     > >>
>>     > >> Here’s the current text, which follows these principles:
>>     > >>
>>     > >> UDP packets with incorrect APC checksums MUST be passed to the
>>     > >>
>>     > >>  application by default, e.g., with a flag indicating APC
>>     failure.
>>     > >>
>>     > >>  Like all SAFE UDP options, APC needs to be silently ignored
>>     when
>>     > >>  failing by default, unless the receiver has been configured
>>     to do
>>     > >>  otherwise. Although all UDP option-aware endpoints support APC
>>     > >>  (being in the required set), this silently-ignored behavior
>>     ensures
>>     > >>  that option-aware receivers operate the same as legacy
>>     receivers
>>     > >>  unless overridden.
>>     > >>
>>     > >> For user perspective, if I set the ACS in my packets but the
>>     receiver
>>     > >> may or may not validate it then it doesn't seem very useful
>>     as a check
>>     > >> for corruption of the sender's data.
>>     > >>
>>     > >>
>>     > >> It is useful when the receiver agrees it is useful.
>>     > >>
>>     > >> I'd probably put a CRC in the UDP
>>     > >> payload instead that I can assure it is always verified by the
>>     > >> receiving application.
>>     > >>
>>     > >>
>>     > >> You can do that if you want, but then it means that it
>>     cannot be silently ignored by a legacy receiver.
>>     > >>
>>     > >>
>>     > >>
>>     > >> Right, that's exactly the desired non-deterministic
>>     > >>
>>     > >>
>>     > >>
>>     > >> Deterministic  I meant
>>     > >>
>>     > >> behavior that we want with regards to a CRC. Silently
>>     ignoring the CRC also means that data corruption would also be
>>     silently ignored.
>>     > >>
>>     > >>
>>     > >>
>>     > >>
>>     > >> So you have two choices:
>>     > >>
>>     > >> 1. Use UDP options with ACS (which is now APC)
>>     > >> a cooperating receiver would know to set APC as required and
>>     would drop packets if APC fails
>>     > >>
>>     > >> 2. Create a new application that knows to require APC
>>     > >> so that the application can enforce CRC in the payload
>>     > >>
>>     > >> But if you can do (2), why not do (1) and have the
>>     application enforce APC-required?
>>     > >>
>>     > >> Joe
>>     > >>
>>     > >> I'm reading this with some doubt now.
>>     > >>
>>     > >> I think option the above option 2 needs to be strongly
>>     discouraged: it provides a place to send an integrity check, but
>>     it doesn't protect the parsing of the other options nor their
>>     data, which would leave all other fields unprotected. That check
>>     would better be provided within the application data, and we
>>     ought to remove this option if that is all it does.
>>     > >>
>>     > >> Here is what I thought had been designed:
>>     > >>
>>     > >> I expect a sender chooses whether to add an APC option.
>>     > >>
>>     > >> If a receiver receives the APC option, the receiver must to
>>     either:
>>     > >>
>>     > >> - discard all datagrams that contain an APC that it decides
>>     not to check, or when the APC fails.
>>     > >>
>>     > >> - forward datagrams that contain an APC that passes. (It
>>     MIGHT be configured to always expect an APC: which I think is the
>>     ONLY receiver decision).
>>     > >>
>>     > >> ... This differs from the benign "use or ignore" processing
>>     that applies to most options, but it is exactly the required
>>     logic that I would expect and I thought we previously discussed.
>>     > >>
>>     > >>
>>     > >>
>>     > >> Gorry and Tom, the behaviour for which you advocate COULD be
>>     achieved if, instead of a SAFE option APC, we specify instead an
>>     UNSAFE option UAPC, and stipulate in its definition that the
>>     packet is discarded if the integrity check fails. In that case:
>>     > >>
>>     > >> The option, being UNSAFE, would be sent only in the options
>>     trailer of a fragmented packet (I refer here to UDP
>>     fragmentation, not IP fragmentation). Having the packet
>>     encapsulated in (one or more) UDP fragments would ensure that a
>>     legacy receiver would not process the packet at all, and the fact
>>     that the option is UNSAFE would ensure that an options-aware
>>     receiver that does not implement it would discard the packet.
>>     > >> An options-aware receiver that does understand the option
>>     would, after successful reassembly, check that to see if the UAPC
>>     is correct and would discard the packet if it is not.
>>     > >>
>>     > >> This would somewhat expand the usage of UNSAFE from "options
>>     that modify the user data" to "options that must be understood in
>>     order to process the user data as intended." In my view (which I
>>     know is not universal) this expansion is likely necessary to
>>     cover use cases that will actually arise; the proposed protocol
>>     number in draft-daiya-tsvwg-udp-options-protocol-number seems to
>>     me to be one such use case.
>>     > >>
>>     > >> Mike Heard
>>     > >>
>>     > >>
>>     > >
>>     >
>>
>