Re: [multipathtcp] MPTCP implementation feedback for RFC6824bis

[Christoph Paasch <[e-mail]>]

Hello Alan,

> On Nov 28, 2019, at 8:16 AM, Alan Ford <[e-mail]> wrote:
> 
> Hi Christoph,
> 
> Thank you for the feedback. Comments inline...
> 
>> On 27 Nov 2019, at 19:29, Christoph Paasch <[e-mail] <mailto:[e-mail]>> wrote:
>> 
>> Hello,
>> 
>> as I mentioned during the meeting last week in Singapore, the team working on upstreaming MPTCP into the Linux Kernel has some feedback on RFC6824bis that would be good to be factored into the RFC before publication.
>> 
>> Here is the list of comments/changes the team is suggesting:
>> 
>> Section 3.1, page 19, "If B has data to send first,..."
>> 
>> The first phrase states that when A receives a DSS from B it indicates successful delivery of the MP_CAPABLE. However, it is not the DSS but rather the DATA_ACK that indicates this. Digging through some past e-mail exchanges I had with Alan I see that that was indeed the intention.
>> 
>> We are thus suggesting to change this text to:
>> If B has data to send first, then the reliable delivery of the ACK+MP_CAPABLE can be inferred by the receipt of this data with a MPTCP Data Sequence Signal (DSS) option that includes a DATA_ACK (Section 3.3). Further, whenever A receives a DATA_ACK from B it is a signal of the reliable delivery of A's MP_CAPABLE. After this, A can send data to B with a regular DSS option.
> 
> That seems entirely logical clarification, and was the intention anyway. A developer probably infers the meaning but clarification is no bad thing.

Sounds good! It would be good to clarify this.

> 
>> Section 3.3.1, page 32 & 33, "A data sequence mapping does not need..."
>> 
>> This paragraph states that it is permissive to send a mapping in advance. Late-mapping is specified a bit higher around the sentence 
>> "Implementations MAY hold onto such unmapped data for a short while in the expectation that a mapping will arrive shortly"
>> 
>> This kind of early/late mapping announcements are difficult to handle in an implementation. The Linux Kernel implementation of multipath-tcp.org <http://multipath-tcp.org/> has always disallowed such kind of mappings. Meaning, whenever a DSS-option is received such that the range specified by the relative subflow-sequence number in the DSS-option and the data-length does not (partially) cover the TCP sequence number of the packet itself, the subflow will be killed with a TCP-RST. The problem around handling such early/late mappings is that it is unclear for how long the stack needs to remember these mappings (in the early-mapping case), or for how long he needs to hold on to the data (in the late-mapping case).
>> 
>> We thus suggest to change this to the following:
>> Whenever a DSS-option is received on a packet such that the mapping of the subflow-sequence space does not partially cover the TCP-sequence number of  the packet itself, the host MUST discard this mapping and MAY destroy the subflow with a TCP-RST. It should be noted that a DATA_FIN that does not come with data has a relative subflow-sequence number of 0 and thus should be handled separately.
> 
> This one I do have an issue with:
> 
> - It is a technical change
> - Wording to this effect has been in the document since pretty much the beginning
> - It is a MAY which might as well say “there is no guarantee this would work”

The problem with the MAY is that the sender can't really know if the receiver accepts it (more regarding this below)

> Most importantly, the replacement text seems not to address this issue at all. If I read it correctly, it says that the data sequence mapping option MUST partially cover the subflow sequence space of the packet itself. But that has nothing to do with late or early mapping, both could partially cover the subflow sequence space and preceding data.
> 
> Can you clarify exactly what you want to permit and prevent, here?

Let me try to clarify what exactly we mean with early/late mapping so that we are all on the same terms here:

Early mapping:

A TCP-segment with sequence-number 1 holds a DSS-option with subflow-sequence number 1001 and data-length 100. This means we need to allocate space to store this DSS-option so that when the TCP-segment with seqno 1001 arrives we can know the mapping. There may be coming more of these DSS-options which all need to be stored in allocated memory. It is unclear what the limit to this is and there is no way to communicate this limit to the sender.

Late mapping:

The receiver receives data without DSS-options with TCP-sequence 1 to 1001. The corresponding DSS-option however arrives with the TCP-segment with seqno 2001. Here, the receiver needs to hold on to this data, waiting for the TCP-segment with the DSS-option. At one point the receiver needs to drop the data due to memory limits. Again, the sender has no way for knowing what this limit is.


When the DSS-option comes together with the corresponding TCP-sequence it is straight-forward to store it together with the data. There are no issues with memory-allocation,... as all of this is accounted together with the announced window (yes, the memory is not counted against the window, but the receiver can foresee the DSS-option overhead when computing what window he should announce).

When a receiver gets data without a DSS-option, he can store it for up to 64KB of data as that is the maximum data-level length and the last segment of the 64KB-train could be holding the DSS-option. After that he has to drop the data.

When the mapping partially covers the segment it also isn't a problem as the unmapped part can safely be dropped and the mapped part can be passed on to the MPTCP-layer.

All of this does not imply that every segment of a mapping needs to hold the DSS-option. Just one of them needs to have it.


Does this help? It is all about making things more deterministic as it is very hard for the receiver to handle these late/early mappings.

Thanks,
Christoph