Re: [tsvwg] Last Call: <draft-ietf-tsvwg-datagram-plpmtud-15.txt> (Packetization Layer Path MTU Discovery for Datagram Transports) to Proposed Standard

[Gorry Fairhurst <[e-mail]>]

Thanks, your careful reading was really helpful in improving the spec.

Gorry

On 30/03/2020 16:08, Marc Petit-Huguenin wrote:
> Thanks.  All my concerns, minus the state machine thing (which I admit I am in the minority here insisting that standards should be written in a way that simplify the developer job), are addressed.
>
> On 3/30/20 7:45 AM, Gorry Fairhurst wrote:
>> Thank you for checking the revision. Please see below.
>>
>> On 30/03/2020 14:56, Marc Petit-Huguenin wrote:
>>> Hi Gorry,
>>>
>>> Thanks for the new version of the draft and your explanations below.
>>>
>>> Let go in the same order than my review:
>>>
>>> I think that your reading of RFC 4821 section 6.1 as applicable only for TCP is too restrictive.  The text explicitly uses "such as" as a way to show that it is just an example of how to "robustly distinguish between the isolated loss of just a probe and other losses...".
>> I read this section as mainly urging PLPMTUD not to rely on TCP RTO, and instead the such-as motivates SACK-based loss detection, or DCCP ACK Vectots. It could be read as saying more, but either way we don't refer to this explicitly.
> OK.
>
>>> RFC 6884 imposes a rate limit on non-atomic IPv4 packet, which are exactly what probes are made of.
>>>     That rate limit is between two IPv4 address, and per transport.  That means that *all* the probes between these IPv4 addresses (independently of the ports) are rate limited.  Admittedly that limit won't be reached in normal cases, but it is still, IMO, useful to remind implementers of that (or to switch to IPv6, that removes the rate limit).
>> I think this is a misunderstanding - all DPLPMTUD probes are atomic and not subject to this upper limit, because they set IPv4 DF.
> Ugh, I got it completely wrong.  Sorry for that.
>
>>> About the state diagram I do not really care one way or another, because these are mostly non-normative.  The useful part is the normative description of each state and each transition, which is where, IMO, the text is lacking.
>>>
>>> I notice that not all my comments were addressed, which would have been either by modifying the text or by giving an explanation why they were not applicable.
>>>
>>> Minor comments not addressed:
>>>
>>> - Section 3, Bullet point 8:  Why not "MUST NOT"?
>> Sorry for not explaining this: "Loss of a probe packet SHOULD NOT be treated as an indication of congestion and SHOULD NOT trigger a congestion control reaction [RFC4821], because this could result in unnecessary reduction of the sending rate."
>>
>> I would say this does not warrant a MUST.  It is a recommendation for useful performance, and if an implementation happens to respond to some probes as congestion, this does not itself result in congestion collapse or another serious defect that impacts other traffic.
> OK.
>
>>> - Section 4.4: "The MPS is smaller than the PLPMTU because of the presence of Pl headers and any IP options or extensions added to the PL packet."  Obviously also because of the presence of the IP header itself, as shown in the diagram.
>> Sorry, for previous confusion here, in the latest rev this is clearer in Figure 1.
> OK.
>
>>> - Section 5.1.1: "An implementation..." Should be replaced by a more general statement saying that implementers can do whatever they want, as long as the external behavior of the implementation behaves exactly as the external behavior of how that state machine would behave.
>> I had no preference here, so will suggest:
>>
>> /An implementation that supports PTB messages MUST validate messages before they are further processed./A PL that supports PTB messages MUST validate these messages before they are further processed./
>>
>> /An implementation that only reduces the PLPMTU to a suitable size would be sufficient to ensure reliable operation,/A method that only reduces the PLPMTU to a suitable size would be sufficient to ensure reliable operation,/
>>
>> /An implementation could implement the various timers using a single timer./The various timers could be implemented using a single timer./
>>
>> /To be robust to these paths an implementation could implement the Error State./The Error State could be implemented to provide rubustness to such paths./
> OK.
>
>>> - Section 5.2: "uses an unacknowledged PL": I do not know what that is.
>> This was defined in section 2.
>>> Nits not addressed:
>>>
>>> - Section 4.3: s/up-to-data/up-to-date/
>> Missed, sorry. Will be fixed.
>>> - Section 4.6.2: s/to trigger enabling a resilience/to enable a resilience/
>> Missed, sorry. Will be fixed.
>>> Thanks.
>> Best wishes,
>>
>> Gorry
>>
>> P.S. Note to self: We also need to fix /MPS: MPS: /MPS:/.
>>
>>> On 3/25/20 10:17 AM, Gorry Fairhurst wrote:
>>>> Hi Marc,
>>>>
>>>> We thought that we would let you know that we have just made a revision of the spec, and what this includes.
>>>>
>>>> This took a little longer to process than we expected because we wanted to really address the under-lying issue of the terms "PMTU and "PLPMTU" that had been with us since the start of this story. We think the new revision is much more concrete on these terms. Similar questions were raised in the SECDIR review concerning the MPS, and have also been resolved here.
>>>>
>>>> There's an SCTP version of the spec heading for FreeBSD and we wanted to also be sure that when that implementation was done, it didn't make different assumptions to what we now write!
>>>>
>>>> Concerning the state diagram - that's been something that another people have used along with the text to make implementations, it's maybe not perfect in capturing every possibility (as you note) but the people writing code found it helpful and proposed small changes at the WG meetings, which we have incorporated as the document progressed. We didn't introduce Cosmogol, and I am myself unsure that significant changes to the current structure would
>>>>
>>>> We didn't understand your comment on RFC6864, because I wasn't sure how this proposed a new rate limit, other than avoiding wrapping MSL in an IP flow when sending fragmentable packets. Did we miss something here?
>>>>
>>>> Last, you mention about  "Using the possibility in RFC 4821 section 6.1 to take in account the packets surrounding a Probe (including probes of different size sent at the same time) to differentiate between congestion and a probe lost because of its size."  - To me the text referenced in 6.1 of PLPMTUD seems rather TCP-focussed. I guess it is possible to do this within the DPLPMTUD spec for a congestion controlled PL, and count the packets against the congestion window as in, bullet 7, section 3. For a PL that does not perform CC we have kept the restriction that it should probe one per RTT (as per RFC8085). That's a constraint, but we we also don't know of any running code in TCP that does this. There still is a lot of lattitude in how DPLPMTUS searches and how to map this to different PLs - e.g., a bunch probes do not have to all be the same size, although it is useful to ensure that at least one probe is likely to succeed in a round of tests.
>>>>
>>>> We also addressed the typos and mistakes you noted - so thanks again for seeing these when we obviously were focussed on other aspects. Sorry for not realising and fixing these earlier.
>>>>
>>>> Best wishes,
>>>>
>>>> Gorry (as an individual) and my co-editors
>>>>
>>>> On 11/03/2020 13:02, Gorry Fairhurst wrote:
>>>>> Thank you for reading this and the review comments. We now plan to look at each of these turn and prepare a new revision. We will also get back in touch to note the corrections and ask where we need clarification.
>>>>>
>>>>> Best wishes,
>>>>>
>>>>> Gorry and the other editors for datagram-plpmtud.
>>>>>
>>>>>
>>>>> On 10/03/2020 22:00, Marc Petit-Huguenin wrote:
>>>>>> Please find below my Last Call review of draft-ietf-tsvwg-datagram-plpmtud-15.  Note that this review does not cover sections 6.2, 6.3 and 9.  Also I believe that an RFC should be implementable without reading the informative parts, so I skipped the abstract and section 1.
>>>>>>
>>>>>> Let's start with the most general comments:
>>>>>>
>>>>>> It seems that the goal of this standard track document is to prescribe one single method (from now on: "method") to find the effective PMTU, something that RFC 4821 did not do.  By doing so, this draft effectively restricts the number of ways that RFC 4821 can be implemented.  A non-exhaustive list of things that the method would prevent could be:
>>>>>>
>>>>>> - Doing parallel probing, i.e. sending a few probes of different sizes at the same time.  Instead the method uses a lockstep mechanism so a new size can be tried only when an acknowledgement is received or the PROBE_TIMER expired MAX_PROBES times.
>>>>>> - Using the possibility in RFC 4821 section 6.1 to take in account the packets surrounding a Probe (including probes of different size sent at the same time) to differentiate between congestion and a probe lost because of its size.
>>>>>>
>>>>>> As a software developer specialized in communication protocols, I do not particularly like the idea that my options to implement a protocol are constrained, especially when the constraints are that I can only do things sequentially.  I think that a better option would be to simply constrain RFC 4821 by defining some limits (like the number of retransmission, and the rate probes should be sent) and let developers do their job.  That said that draft certainly has value for a beginner or unsupervised developer, in which case that whole state machine would be useful in an Informative draft, as the simplest and safest way to do PLPMTUD.
>>>>>>
>>>>>> Now going more in detail about the draft:
>>>>>>
>>>>>> - I would suggest to say something about RFC 6864, which would rate-limits the probes sent between a pair of IPv4 addresses for a particular protocol (in that case UDP).
>>>>>>
>>>>>> - MAX_PMTU is defined as the minimum of the local link MTU and the destination link MTU.  From the top of my mind I could not find a protocol that actually carries that value back to the local side, but I suppose that can be easily done.  It would be useful to say something about that, that the size of the packet used to retrieve that value (also the size of the packet used for connectivity check) should be lower than MIN_MTU, and also what happen when that value becomes available when the state machine is in another state than DISABLED.
>>>>>>
>>>>>> - About MAX_PMTU, this name and others are defined after their first use.  Maybe adding all these to section 2 would make it easier to find definitions (and may even result in discovering some unnecessary aliasing).
>>>>>>
>>>>>> - It could be useful to state that a probe should carry a unique identifier, and that it needs to be reflected in the acknowledgement, so to be able to process out-of-order and delayed packets.  In that case an additional variable in section 5.1.3 would contain the last probe identifier used.
>>>>>>
>>>>>> - From a developer point of view, the information needed to implement PLPMTUD seems to be spread in different sections, making it difficult to get a complete picture of what is going on.  In fact I had to convert the text into a Petri Net -- a non-trivial and time-consuming task -- to be able to understand how bits from various sections fit together.
>>>>>>
>>>>>> So I would suggest to merge sections 4.6.2, 5.1.1, 5.1.2, 5.1.3, 5.2 and 5.3 into one single state machine, listing (a) the set of states, (b) the state context (aka variables, adding PLPMTU to it), (c) the list of transitions conditions (effectively merging timers and packet types received -- destination MTU size, connectivity acknowledgment, probe acknowledgement, and PTB) and finally (d) the exhaustive list of transitions between states, including for each the list of actions on the context and/or the packets sent.  I would either forgo completely the state machine diagram, or use Cosmogol (draft-bortzmeyer-language-state-machines) to include a formal state machine that can be converted into an SVG picture.
>>>>>>
>>>>>> Having such exhaustive list of transitions between states would 1) put all the information needed in one single place and 2) add more clarity to the whole state machine.  E.g. it is not clear if a Probe should also be sent when entering the Base and Search state, or just when PROBE_TIMER expires (delaying the first probe by PROBE_TIMER).  There is other ambiguities like this that could be resolved by a systematic listing of the transitions actions.  And the formalization would permit to check the model for completeness and a few other properties, which cannot be a bad thing in itself.
>>>>>>
>>>>>> Some minor comments:
>>>>>>
>>>>>> - Section 3, Bullet point 8:  Why not "MUST NOT"?
>>>>>> - Section 4.4: "The MPS is smaller than the PLPMTU because of the presence of Pl headers and any IP options or extensions added to the PL packet."  Obviously also because of the presence of the IP header itself, as shown in the diagram.
>>>>>> - Figure 2: "UDPO" is never defined.
>>>>>> - Section 5.1.1: "When an acknowledged PL is used..."  I do not understand what an "acknowledged PL" is.
>>>>>> - Section 5.1.1: "An implementation..." Should be replaced by a more general statement saying that implementers can do whatever they want, as long as the external behavior of the implementation behaves exactly as the external behavior of how that state machine would behave.
>>>>>> - Section 5.1.4: "sends an acknowledged probe packet"  I do not know what that is.
>>>>>> - Section 5.2: "Not all changes are shown to simplify the diagram."  See above.
>>>>>> - Section 5.2: "uses an unacknowledged PL": I do not know what that is.
>>>>>>
>>>>>> Some nits:
>>>>>>
>>>>>> - Section 3, first bullet point: s/For datagram PLs,]/For datagram PLs,]/
>>>>>> - Section 4.3: s/MUST NOT rely soley/MUST NOT rely solely/
>>>>>> - Section 4.3: s/up-to-data/up-to-date/
>>>>>> - Section 4.6.1: s/speed at the which/speed at which/
>>>>>> - Section 4.6.2: s/(e. g.  PLPMTU/(e.g. PLPMTU/
>>>>>> - Section 4.6.2: s/to trigger enabling a resilience/to enable a resilience/
>>>>>> - Section 5.2: s/This state is left, once/This state is left once/
>>>>>> - Section 6.1.3: s/A probe packet that could/A probe packet could/
>>>>>> - Section 6.1.6: s/the application to check each/the application checks that/
>>>>>>
>>>>>> On 2/25/20 6:14 AM, The IESG wrote:
>>>>>>> The IESG has received a request from the Transport Area Working Group WG
>>>>>>> (tsvwg) to consider the following document: - 'Packetization Layer Path MTU
>>>>>>> Discovery for Datagram Transports'
>>>>>>>      <draft-ietf-tsvwg-datagram-plpmtud-15.txt> as Proposed Standard
>>>>>>>
>>>>>>> The IESG plans to make a decision in the next few weeks, and solicits final
>>>>>>> comments on this action. Please send substantive comments to the
>>>>>>> [e-mail] mailing lists by 2020-03-10. Exceptionally, comments may
>>>>>>> be sent to [e-mail] instead. In either case, please retain the beginning
>>>>>>> of the Subject line to allow automated sorting.
>>>>>>>
>>>>>>> Abstract
>>>>>>>
>>>>>>>
>>>>>>>       This document describes a robust method for Path MTU Discovery
>>>>>>>       (PMTUD) for datagram Packetization Layers (PLs).  It describes an
>>>>>>>       extension to RFC 1191 and RFC 8201, which specifies ICMP-based Path
>>>>>>>       MTU Discovery for IPv4 and IPv6.  The method allows a PL, or a
>>>>>>>       datagram application that uses a PL, to discover whether a network
>>>>>>>       path can support the current size of datagram.  This can be used to
>>>>>>>       detect and reduce the message size when a sender encounters a packet
>>>>>>>       black hole (where packets are discarded).  The method can probe a
>>>>>>>       network path with progressively larger packets to discover whether
>>>>>>>       the maximum packet size can be increased.  This allows a sender to
>>>>>>>       determine an appropriate packet size, providing functionality for
>>>>>>>       datagram transports that is equivalent to the Packetization Layer
>>>>>>>       PMTUD specification for TCP, specified in RFC 4821.
>>>>>>>
>>>>>>>       The document updates RFC 4821 to specify the method for datagram PLs,
>>>>>>>       and updates RFC 8085 as the method to use in place of RFC 4821 with
>>>>>>>       UDP datagrams.  Section 7.3 of RFC4960 recommends an endpoint apply
>>>>>>>       the techniques in RFC 4821 on a per-destination-address basis.  RFC
>>>>>>>       4960, RFC 6951 and RFC 8261 are updated to recommend that SCTP, SCTP
>>>>>>>       encapsulated in UDP and SCTP encapsulated in DTLS use the method
>>>>>>>       specified in this document instead of the method in RFC 4821.
>>>>>>>
>>>>>>>       The document also provides implementation notes for incorporating
>>>>>>>       Datagram PMTUD into IETF datagram transports or applications that use
>>>>>>>       datagram transports.
>>>>>>>
>>>>>>>       When published, this specification updates RFC 4960, RFC 4821, RFC
>>>>>>>       8085 and RFC 8261.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The file can be obtained via
>>>>>>> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-datagram-plpmtud/
>>>>>>>
>>>>>>> IESG discussion can be tracked via
>>>>>>> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-datagram-plpmtud/ballot/
>>>>>>>
>>>>>>>
>>>>>>> No IPR declarations have been submitted directly on this I-D.
>>>>>>>
>>>>>>>
>