Re: [tsvwg] draft-ietf-udp-options issues from IETF 104

[Joe Touch <[e-mail]>]

Mike, 

Did you had read my earlier post today about using OCS only on transmit for LITE - as a user choice?

That costs on transmit *but not receive* and allows errors in the LITE data *if* the packet doesn’t cross a buggy middlebox afterwards (which kills the packet no matter what we do anyway).

And in that context, can you update your response below?

Joe

> On Jul 15, 2019, at 10:43 PM, C. M. Heard <[e-mail]> wrote:
> 
>> On Mon, Jul 15, 2019 at 3:01 PM Joe Touch wrote:
> 
>>> On 2019-07-15 14:43, C. M.. Heard wrote:
>>> ...
>>> > NOTE2: *as repeatedly already noted*, again - GIVEN THIS CONTEXT -
>>> > there is no utility in the additional pre-option header proposed in
>>> > draft-herbert-udp-space-hdr-01.
>>>  
>>> Actually, there is something in the draft that IMO has great utility --
>>> not the pre-option header per-se, but the **functionality** provided by
>>> the draft's protocol header packet format. Specifically, that packet
>>> format allows options to share fate with the payload data that they
>>> accompany (i.e., either both are accepted and processed or neither is)..
>>  
>> How is that true? We can't redefine the UDP checksum, which covers the 
>> payload.
> 
> The answer is that for this variant, the the UDP length is set to 8,
> so the UDP checksum just covers the UDP header (as with FRAG+LITE in
> draft-ietf-tsvwg-udp-options-07). Everything else -- the pre-option
> header, options, and user data go in the surplus space, and is covered
> by the checksum in the pre-option header. I think this this clip from
> the draft will help to clarify:
> 
> 2.2 Protocol header format (Extended UDP header)
> 
>    The UDP Surplus Header can be used as a protocol header. Effectively,
>    this creates an extended UDP header format. The logical protocol
>    layering is:
> 
>                       +-+-+-+-+-+-+-+-+-+-+-+ \
>                       |      UDP header     |  |
>                     / +-+-+-+-+-+-+-+-+-+-+-+  | Extended
>                    |  |   UDP space header  |  | UDP header
>            Surplus |  +-+-+-+-+-+-+-+-+-+-+-+  |
>            space   |  | Type specific data  |  |
>                    |  +-+-+-+-+-+-+-+-+-+-+-+ /
>                    |  |       UDP data      |
>                     \ +-+-+-+-+-+-+-+-+-+-+-+
> 
>    The packet format containing an extended UDP header is:
> 
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
>    |        Source port            |      Destination port         | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP
>    |           Length              |         Checksum              | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
>    |     Type      |   TSLength    |         Checksum              |\
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
>    |                                                               | |
>    ~                      Type Specific Data                       ~ USH
>    |                                                               | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /
>    |                                                               |
>    ~                           UDP data                            ~
>    |                                                               |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
>    Notes:
> 
>       - Since the UDP header is aligned and a multiple of four bytes, no
>         padding for USH is necessary.
> 
>       - The UDP length is fixed to be eight so that all bytes beyond the
>         UDP header are contained in the surplus space.
> 
>       - The UDP checksum covers the eight bytes of UDP header and the
>         checksum pseudo header. The USH checksum covers the entire
>         surplus space which includes the UDP Surplus Header and UDP
>         data.
> 
> The USH checksum does needs to include a compensation pseudo-header that
> is equal to the length of the surplus area (the draft omits that through
> oversight, but that is easily fixed and does not affect the discussion
> in the draft in a fundamental way).  
>  
>> A legacy receiver doesn't care about the surplus area and can ignore it. 
>> At which point, whether the surplus is corrupted or not, a legacy 
>> receiver will accept it.
> 
> Yes, but what it will see is a zero-length UDP datagram.
>  
>> A UDP-option-aware receiver would do the same thing with OCS (as 
>> currently defined) as with the one in draft-herbert. Remember, OCS was 
>> already redefined before the last IETF to have the same "zero out" 
>> property as in draft-herbert-00.
> 
> Agreed -- subject of course to adding the compensation pseudo-header.
>  
>>> ...
>>> Besides defining a UDP surplus header (USH, Section 2), the draft also
>>> defines two variations of is usage: a protocol trailer variant (Section
>>> 2.1) and a protocol header variant (Section 2.2).
>>>  
>>> In the trailer variant the UDP user data (and padding bytes as needed
>>> for alignment) precede the USH and trailing options appear at the the
>>> offset indicated by the UDP Length. It applies only when UDP Length > 8.
>>> It provides functionality similar to the packet format defined in
>>> draft-ietf-tsvwg-udp-options-07.
>>>  
>>> In the header variant the UDP Length is set to 8. The USH immediately
>>> follows the UDP header (with no padding), the options are next, and the
>>> payload is last. This is functionally similar to what you get if you
>>> start with draft-ietf-tsvwg-udp-options-07 and add a "user data" option
>>> that consumes all trailing bytes, like the modified FRAG proposed in
>>> https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg.
>>>  
>>> One can reasonably ask, why on earth would we want both formats? My
>>> answer is that they provide different but complementary functions.
>>  
>> Didn't you just prove they're equivalent to, in essence, LITE and non-LITE? 
> 
> No, not at all. By definition, LITE data is excluded from OCS. The
> protocol header packet format (aka extended UDP packer format) includes
> the payload data in its checksum, along with the options. One could
> certainly do that with OCS, but it would be necessary to add an
> "payload data" option of some sort.
>  
>>> - The trailer variant can be sent to a legacy receiver with a reasonable
>>> expectation that the user data will be processed normally and the
>>> options will be ignored.
>>  
>> that's how UDP options already work...
> 
> Exactly so. To expand on this a bit, here is the packet format:
> 
> 2.1 Protocol trailer format
> 
>    When used as a protocol trailer, the UDP Surplus Header immediately
>    follows the UDP data. The logical protocol layering is:
> 
>                       +-+-+-+-+-+-+-+-+-+-+-+
>                       |      UDP header     |
>                       +-+-+-+-+-+-+-+-+-+-+-+
>                       |       UDP data      |
>                     / +-+-+-+-+-+-+-+-+-+-+-+ \
>            Surplus |  |   USH base header   |  |
>            space   |  +-+-+-+-+-+-+-+-+-+-+-+  |  USH
>                    |  | Type specific data  |  |
>                     \ +-+-+-+-+-+-+-+-+-+-+-+ /
>    The packet format of UDP Surplus Header as a protocol trailer is:
> 
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
>    |        Source port            |      Destination port         | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP
>    |           Length              |         Checksum              | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
>    |                                                               |
>    ~                           UDP data                            ~
>    |                                                               |
>    +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |               |                      Padding                  |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |     Type      |   TSLength    |         Checksum              |\
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
>    |                                                               | |
>    ~                      Type Specific Data                       ~ USH
>    |                                                               | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /
> 
>    Notes:
> 
>       - The offset of the UDP Surplus Header from the start of the UDP
>         header, including possible padding for the USH, is equal the UDP
>         Length.
> 
>       - The number of padding bytes is 3 - ((udp_length - 1) % 4), where
>         udp_length is equal to the UDP Length field. The offset of the
>         Type field of the USH is 4 * ((udp_length - 1) / 4 + 1).
> 
>       - If the size of the USH header (four plus four times TSLength) is
>         less than the size of the UDP surplus space in a packet, then
>         the USH is considered to be malformed (see section 3.2).
> 
>       - The UDP checksum covers the UDP header and UDP data. The USH
>         checksum covers the entire UDP surplus space.
> 
>       - A legacy receiver, one that does not understand the UDP Surplus
>         Header, will ignore the contents of the UDP surplus space and
>         process the UDP data as normal. Protocol data that cannot
>         correctly be ignored by a receiver, such as the fragmentation
>         option in the [UDPOPT], MUST NOT be in a surplus space trailer.
> 
> hese are the main functional differences between the protocol trailer
> packet format and what is currently in draft-ietf-tsvwg-udp-options-07:
> 
> - The checksum is not optional; it is required. It covers all data
> from the padding preceding the fixed surplus header to the end of
> the packet. As a consequence, full checksum compensation is provided.
> 
> - Option data (not including fixed surplus header) is limited to 1020
> bytes, and it must be aligned on a four-byte boundary.
> 
> - Options that affect the handling of the data preceding the trailer
> are not allowed there. In other words, an option would not be allowed
> in the trailer if the interpretation of the data preceding the trailer
> would be different depending on whether the option was processed or
> ignored. Examples include FRAG (for obvious reasons), ACS (as data
> failing the alternate checksum would normally be discarded), and AE
> (as data failing authentication or decryption would normally be
> discarded). Such options are allowed in the header variant only.
> 
> - LITE data is not accommodated, both because of the above rule and
> also because of the mandatory nature of the surplus space checksum.
> 
> EVALUATION: I do understand from what you wrote elsewhere on this thread
> that OCS will incorporate the essential checksum functionality described 
> above in the -08 rev, which I applaud. Indeed, I already proposed that
> OCS, if present, should cover the entire UDP options trailer (excluding
> LITE data), and that it should be present whenever the checksum in the
> UDP header is non-zero (and I'd still to see "required when UDP CS<>0"
> instead of "required"). The other functional requirement that I'd
> like to see levied is to ban, or at least strongly discourage, options
> from the trailer if they affect the handling of the payload. I include
> FRAG, ACS, and AE in this category (see comments below about LITE).
> Neither of those changes require the use of a fixed pre-option header. 
> 
> <OPINIONATED COMMENTARY>
> For the protocol trailer packet format I see only downsides to the
> fixed pre-option header as compared to what is currently defined in
> draft-ietf-tsvwg-udp-options-07, including OCS as a 3-byte option.
> When sending I can align OCS with NOPs if I need to, and when receiving
> I don't care if it's aligned or not, since I'll use a standard Internet
> checksum calculation (with compensation pseudo-header) over the whole
> trailer area, conceptually prepending or appending zero bytes if it
> starts or ends on an odd byte boundary. If the checksum passes, I don't
> have to do anything with OCS when I go looping through the option TLVs.
> So, the protocol trailer packet format has no advantage when it comes to
> checksum handling. Limiting the size of the trailer area limits the
> ability of DPLPMTUD to pad user data packet for MTU probing (that is a
> good use for EOL, please keep it). And it precludes LITE, though given
> that an OCS failure causes LITE data to be discarded while the non-LITE
> data is delivered, it may be better to tell folks to use UDP-Lite if
> they need a partial checksum coverage capability.
> </OPINIONATED COMMENTARY>
>  
>>> ...
>>> - The header variant can be used to safely send options that need to
>>> share the fate of the payload data that they accompany (the options
>>> and payload share the same checksum, so both will be discarded if
>>> the checksum fails).
>>  
>> How is this not LITE?
> 
> If you mean LITE with UDP Length = 8, the answer is because LITE data
> is not checksummed. In the usual case, one wants the user data to be
> protected by a checksum.
>  
>> Note, however, that a packet still shows up (albeit zero length) with 
>> either approach. So this isn't exactly the same as true fate-sharing.
>  
> That's true; as I noted above, it is possible for a the zero-length UDP
> datagram to affect a legacy receiver. However, it need not affect an
> options-aware receiver; the transport module can certainly distinguish
> between UDP Length = 8 and no additional data vs. UDP Length = 8
> and a additional data that fails OCS or the USH checksum.
>  
>>> Options that cause the payload to be handled
>>> differently when they are present fall into that category. Examples
>>> include FRAG (for obvious reasons), ACS (as data failing the alternate
>>> checksum would normally be discarded instead of being passed to the
>>> application), and AE (as data failing authentication or decryption would
>>> normally be discarded instead of being passed to the application).
>>>  
>>> Note that draft-herbert-udp-space-hdr-01 stipulates that options which
>>> cause the payload to be handled differently when they are present may
>>> reside in the header variant only.
>>>  
>>> Why is this important?
>>>  
>>> Clearly, for legacy receivers, options in the trailer cannot be
>>> guaranteed to share fate with the payload because they discard all
>>> options. Similarly, if a new option option that affects data
>>> handling is defined in the future, option-aware receivers that do
>>> not recognize it will skip over it. These issues can in principle
>>> be dealt with by insisting (as draft-ietf-tsvwg-udp-options-07 does)
>>> that options that are not safe to ignore must not be sent without
>>> prior knowledge (obtained by negotiation or configuration) that the
>>> target receiver can properly handle them.
>>>  
>>> BUT, it's not just legacy receivers that feel the impact; option-aware
>>> receivers treat the entire options trailer as being absent if the option 
>>> checksum fails. If the UDP checksum passes, and there are options that
>>> change the handling of the preceding payload data, then that data will
>>> be handled incorrectly. In other words, a correctly implemented receiver
>>> that recognizes all options nonetheless has the potential to pass
>>> corrupted data to the application. In my opinion that's not acceptable.
>>  
>> That's not a feature of draft-herbert vs the exising UDP options; that 
>> is a choice in Sec 8 of udp-options.
>>  
>> We chose to have that case ignore options to emulate legacy receivers; 
>> if that's not what we want, we change that rule.
>>  
>> If that rule is changed, then you'd get the fate sharing you seek.
>>  
>> But you can't have it both ways - with either solution. They're the same 
>> here. Either failed options means "act like legacy" or it means "fail", but 
>> that means that incorrect options would succeed in a legacy receiver but 
>> fail in an options-aware one - EITHER WAY.
> 
> It seems to me that you are actually making the case for devoting a bit
> in the Option Kind that specifies what to when the option is not
> recognized (or recognized but not supported). This shows up in the IETF
> 104 slides under the rubric of a "require/ignore" but that misrepresents
> what the advocates (myself included) had in mind.. What we (or I, at
> least) had in mind was something similar to what's in RFC 8200:
> 
>       0 - skip over this option and continue processing the packet
> 
>       1 - discard the packet.
> 
> With this capability available, we could assign an Option Kind with the
> flag set if the option affects the handling of the accompanying payload
> (FRAG, AE, and ACS, at a minimum) and an Option Kind with the flag clear
> if the option can be safely ignored by a legacy receiver (EOL, NOP, OCS,
> MSS, TIME, REQ, RSP). There should be an EXP option of each flavor to
> be used as appropriate for the experiment. Then the rule is, only those
> options whose flag is clear can be in an options trailer; options whose
> flag is set have to be in the options header. Then the desired behavior
> results: an option whose flag is set will be discarded, along with its
> data, either by a legacy receiver that does not understand options at
> all (thought it will see a zero-length datagram) or by an options-aware
> receiver that does not understand the option.
> 
> I'm of two minds about LITE: I don't see a way to make it work with the
> protocol header format or something similar, but I don't see a way to
> make it safe with the trailer format of draft-ietf-tsvwg-udp-options-07.
> I could accept seeing it remain with a "for mutually consenting parties
> only" health warning, but unless it solves actual problems that
> UDP-Lite does not, maybe it's best to say "just use UDP-Lite." It does
> seem as if the fond hopes of easy traversal of middleboxes that block
> protocol 136 are not to be, as LITE defeats checksum compensation.
> 
>>> AFAICT, draft-ietf-tsvwg-udp-options-07 currently has no means for
>>> providing the fate-sharing of options and payload.
>> 
>> It does, as shown above.
> 
> LITE data is delivered or dscarded depending on the fate if the LITE
> option, but only at the cost of having no checksum protection for the
> data. The protocol header packet format in draft-herbert does not
> have that drawback.
>  
>>> The protocol header
>>> format in draft-herbert-udp-space-hdr-01 does. I'd like to see
>>> draft-ietf-tsvwg-udp-options-07 revised to provide that functionality.
>>>  
>>> I have my own ideas on how best to do that, but I am going to hold them
>>> in abeyance for a later time because I think it's much more important
>>> for us to discuss whether I am correct to insist on the need for an
>>> option and payload fate-sharing capability for protocol correctness.
>>  
>> I agree with that - let's decide what functionality we want and THEN 
>> figure out how to deliver it. Or at least separate those two discussions 
>> where possible.
> 
> Excellent, thank you.
>  
>> However, header/trailer is a misnomer. It's really about whether and how 
>> much to use LITE.
> 
> Sorry, but I disagree with that. LITE does not and cannot deliver the
> functionality of the protocol header packet format in draft-herbert.
> LITE  does not checksum the payload data; the protocol header packet
> format in draft-herbert does. I do believe that improvements can be
> made to the latter; details for those who care are below my .sig
> 
> Mike
> 
> --
> The main properties of of the protocol header packet format in
> draft-herbert are:
> 
> - The UDP Length is required to be 8. The fixed-size surplus header
> begins immediately after the UDP header, without padding.
> 
> - The checksum is not optional; it is required. It covers all data
> from the the fixed surplus header to the end of the packet (as above).
> As a consequence, full checksum compensation is provided.
> 
> - Option data (not including the fixed surplus header) is limited to
> 1020 bytes, and it must be aligned on a four-byte boundary (as above).
> 
> - LITE data is not accommodated.
> 
> - All other options are allowed.
> 
> - Data following the end of the options area (as indicated by the length
> field in the surplus header) starts on a four-byte boundary. It is
> processed as indicated in the options and passed to the application
> (if appropriate) following such processing. In many cases there is no
> processing and it's just passed upward, but that would not be the case
> for FRAG, ACS, and AE.
> 
> The limitation to 1020 bytes of options seems needless and arbitrary,
> and it interferes with the capability of this format to support DPLPMTUD
> probe packets that consist of padding only (for more information see
> draft-ietf-tsvwg-datagram-plpmtud-08 Section 4.1). This limitation is
> a result of reserving a byte in the pre-option header as a type for
> extensibility; if we agree that the options themselves provide enough
> flexibility, then this issue can be overcome by modifying the format
> to be as follows:
> 
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
>    |        Source port            |      Destination port         | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -Base Hdr
>    |        UDP Length = 8         |   UDP Header Checksum         | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
>    |        Payload Offset         |  Option+Payload Checksum      |\
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
>    |                                                               | |
>    ~                           UDP Options                           ~Ext Hdr
>    |                                                               | |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /
>    |                                                               |
>    ~                         Payload Data                          ~
>    |                                                               |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
> If alignment of the payload data is seen as desirable (which is likely
> the case), then Payload Offset can be restricted to be a multiple of 2,
> 4, or 8 as appropriate, with a minimum value of 4. Fragments should be
> similarly aligned.
> 
> Several alternatives to this format were considered, notably including
> the currently-defined 3-byte OCS (one byte Kind, two byte checksum)
> with a to-be-defined one-byte "user data" option that would appear at
> the end and consume all following data. That combination uses at least
> as much space and will use more if NOPs are inserted for alignment. So
> in this case (unlike the trailer case), a fixed header appears to be
> advantageous compared to possible alternatives. FRAG would be as in
> https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg
> but with the interpretation changed to have it consume all payload
> data instead of all data that follows.
>