Re: [multipathtcp] RFC6824bis edits based on implementation feedback

[Matthieu Baerts <[e-mail]>]

Hi all,

First, thank you all for having contributed all these years to make 
clear MPTCP RFCs.

I'm sorry to interrupt the discussions with my message but because it 
seems there are still some opened questions related to these changes, I 
would like to express my view from the implementer side.

As a maintainer of both the MPTCP Linux out-of-tree implementation and 
the new MPTCP Linux upstream one, I fully endorse both changes.

The first one avoids confusions while the second one avoids senders to 
generate segments that are not supported by the two MPTCP Linux 
implementations and certainly by most/all existing ones. Accepting such 
Data Segment Mappings are not supported and will probably never be 
because it is complex, middleboxes don't seem to modify packets to have 
a situation like that and it can be a source of security issues. 
Avoiding senders to create such mappings would avoid issues if most 
network stacks don't implement this specific support for technical reasons.

Thank you Alan for taking care of this!

Cheers,
Matt

On 04/02/2020 13:12, Alan Ford wrote:
> Hi Yoshi,
> 
> The proposed new text enforces a mapping to be sent on a packet whose 
> subflow sequence number is included within the mapping provided. This 
> does not directly contradict with the MAY case. Indeed this scenario 
> could easily happen if there is a 20 byte packet followed by a 10 byte 
> packet. The first 10 bytes of the first packet are covered by the 
> mapping on packet 1, then the remaining 10 bytes and the following 10 
> bytes are covered by the mapping on packet 2. That would be valid.
> 
> To re-iterate, the primary purpose of this proposed text is to stop 
> people sending mappings before they are needed.
> 
> Best regards,
> Alan
> 
>> On 4 Feb 2020, at 07:38, Yoshifumi Nishida <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Hi Anil,
>>
>> Thanks for pointing it out. I overlooked this one.
>> This looks an interesting point.
>>
>> It seems to me that whether RST is happen or not depends on the size 
>> of receive window according to the text.
>> If the receive window size is big enough to accommodate segment 1 and 
>> segment 3, the text "Implementations MAY hold onto such unmapped data 
>> for a short while in the expectation that a mapping will arrive 
>> shortly. " can be applied to the segment 1. As a result, the segment 1 
>> won't be discarded.
>> However, this might be contradict with the new texts Alan proposed? 
>> Or, am I missing something?
>>
>> Thanks,
>> --
>> Yoshi
>>
>>
>>
>> On Sun, Feb 2, 2020 at 8:42 AM V Anil Kumar <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>>     Hi Yoshi,
>>
>>     Thanks for this point. In fact, I had initially not thought of a
>>     scenario, where the map is being delivered through a retransmitted
>>     data packet while its first transmission did not include the map.
>>     Now I am just seeing the document (RFC 6824-bis) in this context.
>>
>>     My understanding is that in scenarios like what I described in my
>>     previous mail, RST is likely to happen whether we explicitly state
>>     so or not. Please see the paragraph containing the below text in
>>     RFC 6824-bis.
>>
>>     "If a mapping for that subflow-level sequence space does not
>>     arrive within a receive window of data, that subflow SHOULD be
>>     treated as broken, closed with a RST, and any unmapped data
>>     silently discarded."
>>
>>     if we assume that the map is included while retransmitting the
>>     data (even though the first transmission did not contain the map
>>     for some reasons),  we could argue that RST could be avoided
>>     provided that the retransmission is triggered within a receive
>>     window of data. But the question here would be how and when will
>>     the retransmission take place. In this case, the subflow may not
>>     initiate the retransmission of data by its own (i.e., no
>>     retransmission due to three duplicate ACKs or RTO expiry at
>>     subflow level) as there is no segment loss at subflow level
>>     sequence space. So there could be a high possibility of RST
>>     happening even before the map delivery through retransmission.
>>
>>     With regards,
>>
>>     Anil
>>     ------------------------------------------------------------------------
>>     *From: *"Yoshifumi Nishida" <[e-mail]
>>     <mailto:[e-mail]>>
>>     *To: *"V Anil Kumar" <[e-mail] <mailto:[e-mail]>>
>>     *Cc: *"alan ford" <[e-mail]
>>     <mailto:[e-mail]>>, "multipathtcp"
>>     <[e-mail] <mailto:[e-mail]>>
>>     *Sent: *Saturday, February 1, 2020 3:39:51 AM
>>     *Subject: *Re: [multipathtcp] RFC6824bis edits based on
>>     implementation feedback
>>
>>     Hi Anil,
>>
>>     I have a question about your proposed text.
>>     I am actually wondering if we really want to terminate connection
>>     here.
>>
>>     The packets without proper mappings will be treated as invalid and
>>     will be discarded.
>>     If an implementation failed to attach proper mapping for some
>>     reasons (e.g. option space), it might be able to attach the proper
>>     one when it retransmits the packets. This also looks ok to me.
>>
>>     I don't have strong preference for this. But, do we have a reason
>>     to terminate connection?
>>
>>     Thanks,
>>     --
>>     Yoshi
>>
>>     On Mon, Jan 13, 2020 at 10:28 AM V Anil Kumar <[e-mail]
>>     <mailto:[e-mail]>> wrote:
>>
>>         Hi,
>>
>>         I have some points related to the  modifications (Change 2)
>>         being proposed on data sequence map. Please see them inline.
>>         Though I am putting forward the below points, if the consensus
>>         is in favour of the proposed change for reducing
>>         implementation complexity, I am also OK with that as well.
>>
>>         ------------------------------------------------------------------------
>>         *From: *"alan ford" <[e-mail]
>>         <mailto:[e-mail]>>
>>         *To: *[e-mail] <mailto:[e-mail]>
>>         *Sent: *Thursday, January 2, 2020 4:21:32 AM
>>         *Subject: *[multipathtcp] RFC6824bis edits based on
>>         implementation feedback
>>
>>         Hi all,
>>         We’d love to get this to a state of completion as soon as
>>         possible, and to this end I am starting a new thread on this
>>         topic. In discussion with the chairs, it /is /possible to make
>>         the desired changes in AUTH48 as long as there is WG
>>         consensus. The discussion so far has been fairly limited in
>>         terms of participation.
>>
>>         I would ask the chairs please if it was possible to specify a
>>         time bound for this discussion and a default conclusion.
>>
>>         Regarding the changes, in summary, there are two areas where
>>         changes have been requested by the implementation community.
>>         As we are the IETF we obviously have strong focus on “running
>>         code” and so ease of implementing standards-compliant code is
>>         strongly desirable. However, we do not wish to reduce
>>         functionality agreed by the IETF community if it is considered
>>         a required feature by the community.
>>
>>
>>         *Change 1*
>>         *
>>         *
>>         Change the sentence reading:
>>
>>         /   If B has data to send first, then the reliable delivery of
>>         the ACK + MP_CAPABLE can be inferred by the receipt of this
>>         data with an MPTCP Data Sequence Signal (DSS) option (Section
>>         3.3).
>>         /
>>         To:
>>
>>         /   If B has data to send first, then the reliable delivery of
>>         the ACK + MP_CAPABLE is ensured by the receipt of this data
>>         with an MPTCP Data Sequence Signal (DSS) option (Section 3.3)
>>         containing a DATA_ACK for the MP_CAPABLE (which is the first
>>         octet of the data sequence space).
>>         /
>>         What this means:
>>
>>         The current text is concerned only with ensuring a path is
>>         MPTCP capable, and so only cares that DSS option occurs on a
>>         data packet. However, the MP_CAPABLE option is defined to
>>         occupy the first octet of data sequence space and thus, if
>>         analogous to TCP, must be acknowledged. >From an
>>         implementation point of view it would make sense not to have
>>         this hanging around forever and instead define it is
>>         acknowledged at the connection level as soon as received. This
>>         change ensures the first data packet also DATA_ACKs this
>>         MP_CAPABLE octet.
>>
>>
>>         *Change 2*
>>         *
>>         *
>>         Change the sentence reading:
>>
>>         /   A Data Sequence Mapping does not need to be included in
>>         every MPTCP packet, as long as the subflow sequence space in
>>         that packet is covered by a mapping known at the receiver./
>>
>>         To:
>>
>>         /   The mapping provided by a Data Sequence Mapping MUST apply
>>         to some or all of the subflow sequence space in the TCP
>>         segment which carries the option. It does not need to be
>>         included in every MPTCP packet, as long as the subflow
>>         sequence space in that packet is covered by a mapping known at
>>         the receiver.
>>         /
>>         What this means:
>>
>>         The current text does not place any restrictions on where a
>>         mapping could appear. In theory a sender could define a
>>         thousand different mappings up front, send them all, and
>>         expect a receiver to store this and reassemble data according
>>         to these mappings as it arrives. Indeed, this was never
>>         explicitly disallowed since it “might have been useful”. The
>>         implementation community, however, has expressed concerns over
>>         the difficulty of implementing this open-endedly. How many
>>         mappings is it reasonable to store? Is there a DoS risk here?
>>         Instead, it has been requested that thee specification
>>         restricts the placement of the DSS option to being within the
>>         subflow sequence space to which it applies.
>>
>>         Below are my comments on this. I had shared some of these
>>         points in a previous thread that you had initiated in the same
>>         context.
>>
>>         Transmitting large number of non-contiguous data sequence maps
>>         could be a misbehaviour (map-flooding), though it is not clear
>>         whether this can go to the extent of causing a potential DoS
>>         to the data receiver. So some sort of restriction on this
>>         could be useful.  One approach could be to insist that the
>>         data sender should ensure that the map being transmitted is
>>         for in-window data, as per the receiver advertised window. A
>>         receiver should anyhow be willing to store the maps for
>>         in-window data to deal with packet loss. For example, when a
>>         window of data segments (say 1 to 64) is transmitted, each
>>         carrying its corresponding map, and segment-1 is lost, the
>>         maps for the remaining 63 need to be stored till the lost
>>         segment is retransmitted. Of course, in this case the maps
>>         will be stored at the receiver side along with their
>>         corresponding data. But the need to store multiple maps for
>>         in-window data would still be there.
>>
>>         The problem with the proposed change (restriction) is that a
>>         data sender may find it difficult, in case a need arise to
>>         slightly delay the map delivery by few segments, i.e., sending
>>         some data first without map, and then send the corresponding
>>         map in a later segment, as shown below:
>>
>>         subflow-1:      segment-1                   segment-3         
>>                   segment-4                       segment-7
>>                               bytes:1-100                
>>         bytes:201-300              bytes:301-400                
>>         bytes:601-700
>>                               no map                        map for
>>         1-100              map for 201-400             map for 601-700
>>
>>
>>
>>         subflow-2:       segment-2                  segment-5        
>>                     segment-6                       segment-8
>>                                bytes: 101-200          
>>         bytes:401-500               bytes: 501-600              
>>          bytes:701-800
>>                                map for 101-200       map for 401-600  
>>                  no map                            map for 701-800
>>
>>
>>         In the above case, segment-1 goes without map and its map is
>>         included later in segment-3, the next data segment in the same
>>         subflow. Further,  in the above scheduling pattern, the map in
>>         segment-3 cannot cover the  data in segment-1 and segment-3,
>>         as some  data in between (segment-2) is transmitted through
>>         another subflow.  With the proposed change, the map in
>>         segment-3 will become invalid and this will eventually break
>>         subflow-1, though this could be a corner case.
>>
>>         The question at this stage is why would segment-1 be
>>         transmitted without its map. In the case of bidirectional data
>>         transfer, there could be a need to pack both timestamp and
>>         SACK  options in a data segment, i.e., piggybacking of  SACK
>>         with data. If we consider that timestamp takes 12 bytes and
>>         SACK, even with single block,  takes another 10 bytes, the
>>         remaining 18 bytes option space is not adequate to carry data
>>         sequence signal with map, especially when DSN is 64 bit long.
>>         So the delivery of either of the two (SACK or map) would be
>>         delayed.
>>
>>         As far as I understand, RFC 2018 (TCP Selective
>>         Acknowledgement Options) implies that SACK should not be
>>         delayed. It states "If sent at all, SACK options SHOULD be
>>         included in all ACKs which do not ACK the highest sequence
>>         number in the data receiver's queue". It also says "If data
>>         receiver generates SACK options under any circumstance, it
>>         SHOULD generate them under all permitted circumstances".   So,
>>         as part of meeting the RFC 2018 requirements, if the
>>         combination of SACK and timestamp is given preference over
>>         DSS, data segments could be transmitted without their map.
>>
>>         Another case of delaying map could arise if the data sender
>>         prefers to send ADD_ADDR option, instead of map, in a data
>>         segment. It is nice that ADD_ADDR option can be delivered
>>         reliably in a pure ACK, but I think this is not the case with
>>         DSS at present.
>>
>>         If we adopt the proposed change, I think it might also be
>>         helpful to spell out how the receiver is supposed to behave,
>>         if it gets maps not meeting the MUST condition in the proposed
>>         change.  For example termination of the subflow with MP_TCPRST
>>         option (section 3.6 in RFC 6824-bis) with appropriate reason
>>         code and T flag value to intimate the data sender the cause
>>         for subflow termination.
>>         With regards,
>>
>>         Anil
>>
>>         Please can members of the WG express whether they are happy
>>         with these changes, or concerned.
>>
>>         Best regards,
>>         Alan
>>
>>
>>         _______________________________________________
>>         multipathtcp mailing list
>>         [e-mail] <mailto:[e-mail]>
>>         https://www.ietf.org/mailman/listinfo/multipathtcp
>>
>>
>>
>>         _______________________________________________
>>         multipathtcp mailing list
>>         [e-mail] <mailto:[e-mail]>
>>         https://www.ietf.org/mailman/listinfo/multipathtcp
>>
>>
>>
>>
> 
> 
> _______________________________________________
> multipathtcp mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/multipathtcp
> 

-- 
Matthieu Baerts | R&D Engineer
[e-mail]
Tessares SA | Hybrid Access Solutions
www.tessares.net
1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium