Re: [manet] Mirja Kühlewind's Discuss on draft-ietf-manet-olsrv2-multipath-12: (with DISCUSS and COMMENT)

["Mirja Kuehlewind (IETF)" <[e-mail]>]

Hi Jiazi,

unfortunately, the following sentence is not correct, as basically any transport can be encapsulated over UDP:

> Per-Datagram scheduling is recommended for transport layer protocols that have less constraint on datagram arriving order such as UDP.

I would propose the following sentence:

> Per-Datagram scheduling is only recommended if it is known that the traffic is not sensitive to reordering, e.g. for non-reliable transmission of media traffic.

In this case I would further recommend to add the following sentence:

> Note that the use of paths with highly different delays can still have a negative impact on these kind of transmissions, as packets that arrive too late may be ignored and therefore have been transmitted unnecessarily.

Further I also don’t agree to this sentence:

> In lossy networks, per-datagram scheduling is also recommended.

Even in lossy networks, per-datagram scheduling is still not recommended for TCP (if the delay of the different paths are too diverse).

Finally, I would further recommend you to not say that round-robin needs to be used in section 8.4. I also don’t think you need to have the two bullet points that explain what per-datagram and per-flow means. It’s enough to say that per-flow scheduling means that a decision is made on the first packet which path to use and all subsequent packets of the same flow (e.g. identified by the 5- or 6-tuple) need to use the same path.

Further please also check section 4 as this also makes assumptions on the path selection.

Thanks,
Mirja
 


> Am 23.05.2017 um 13:49 schrieb Jiazi Yi <[e-mail]>:
> 
> Dear Mirja, 
> 
> Thank very much for your reply. 
> 
> We understand your concern on the performance, especially the issue of packet reordering when the multipath protocol is applied to transport protocol like TCP. Therefore, we propose to have some guidelines to indicate when per-flow scheduling is recommended, and when per-packet scheduling is recommended. 
> 
> More specifically, in the draft, we propose the text:
> 
> 
> ==========
> In section 1.1, Experiments to be conducted:
> 
> 	• Different path-selection schedulers. Depending on the application type and transport layer type either per-flow scheduler or per-datagram scheduler is applied. By default, round-robin scheduling is used to select a path to be used. In some scenarios, weighted scheduling can be considered: for example, the paths with lower metrics (i.e., higher quality) can transfer more datagrams or flows compared to paths with higher metrics.
> 
> 
> In section 8.4, Datagram Processing at the MP-OLSRv2 Originator
> 
> If a matching Multipath Routing Tuple is obtained, the Path Tuples of the Multipath Routing Tuple are applied to the datagrams using round-robin scheduling. The scheduling policy can be either per-flow or per-datagram, depending on the transport layer protocol and the application used: 
> 
> 	- In per-flow scheduling, the datagrams of the same flow is transmitted through the same path. Different flows are assigned to different paths according to round-robin scheduling. For example, there are 2 path Tuples (Path-1, Path-2) for the destination router D. A series of flows (Flow-1, Flow-2, Flow-3, ... etc.) are to be sent to router D. Path-1 is then chosen for Flow-1, Path-2 for Flow-2, Path-1 for Flow 3, etc. 
> 
> 	- In per-datagram scheduling, different datagrams are transmitted through different paths according to round-robin scheduling. For example, there are 2 path Tuples (Path-1, Path-2) for the destination router D. A series of datagrams (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent to router D. Path-1 is then chosen for Packet-1, Path-2 for Packet-2, Path-1 for Packet 3, etc. 
> 
> Per-flow scheduling is recommended for transport layer protocols that require strict ordering of the datagrams such as TCP. By default, per-flow scheduling is used. Per-Datagram scheduling is recommended for transport layer protocols that have less constraint on datagram arriving order such as UDP. In lossy networks, per-datagram scheduling is also recommended.  
> Other path scheduling mechanisms are also possible and will not impact the interoperability of different implementations.
> 
> ===========
> 
> If it looks good for you, we will make the modification in the next revision of the draft. 
> 
> regards
> 
> Jiazi
> 
> 
> 
>> On 22 May 2017, at 12:17, Mirja Kuehlewind (IETF) <[e-mail]> wrote:
>> 
>> Hi Jiazi,
>> 
>> sorry for my very late reply. Please see below.
>> 
>>> Am 10.05.2017 um 00:51 schrieb Jiazi Yi <[e-mail]>:
>>> 
>>> Dear Mirja, 
>>> 
>>> Thanks very much for your review and comments. Please find the reply inline: 
>>> 
>>>> ----------------------------------------------------------------------
>>>> DISCUSS:
>>>> ----------------------------------------------------------------------
>>>> 
>>>> The following text in section 4 seems to indicate that scheduling is done
>>>> on a per-packet basis:
>>>> "When there is a
>>>>  datagram to be sent to a destination, the source router acquires a
>>>>  path from the Multi-path Routing Set (MAY be Round-Robin, or other
>>>>  scheduling algorithms)."
>>>> This seems not appropriate as e.g. TCP packets routed on links with
>>>> largely different delays may suffer performance. ECMP usually hashes the
>>>> 5-tuple or 6-tuple (incl. DiffServ Codepoint) to setup state and routes
>>>> all packets belonging to the same flow on the same route. I recommend to
>>>> apply the same here.
>>>> 
>>>> Also related is this text in section 8.4 that should explain Round-Robin
>>>> on a per flow basis instead. Further this should only be an example
>>>> scheduling alogirthm while text belong seems to assume that Round-Robin
>>>> is always used.
>>>> "If a matching Multi-path Routing Tuple is obtained, the Path Tuples
>>>>  of the Multi-path Routing Tuple are applied to the datagrams using
>>>>  Round-robin scheduling.  For example, there are 2 path Tuples
>>>>  (Path-1, Path-2) for destination router D. A series of datagrams
>>>>  (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent router D.
>>>>  Path-1 is then chosen for Packet-1, Path-2 for Packet-2, Path-1 for
>>>>  Packet 3, etc.  Other path scheduling mechanisms are also possible
>>>>  and will not impact the interoperability of different
>>>>  implementations.”
>>> 
>>> In fact, the per-packet scheduling is an intentional choice because:
>>> 
>>> 1. The aimed scenario is mobile ad hoc networks, which have high packet loss by nature. One of the most important reasons is route failure due to mobility — in such case, if a per-flow scheduling is applied, the whole flow will lost. On the other hand, if we send the packets in disjoint paths (not necessarily equal cost), we can still make use partial information of the flow, or even reconstruct the flow with erasure coding. This is especially interesting for video/audio streaming. 
>> 
>> This is only true for unreliable or partially reliable traffic. I see your use case. However, will if you have TCP traffic, you can probably assume that the traffic is reliable and should not route on a per-packet basis. This case is not covered in your draft. Also for other non-TCP you often might not know much about the traffic characteristics and therefore cannot know if per-packet scheduling is good or bad. Therefore default should be per-flow.
>> 
>>> 
>>> 2. In such kind of lossy networks, the traditional TCP performs very poor [1]. So as far as I know, TCP is not very popular in ad hoc networks. On the other hand, based on our study, the multi-path routing can actually reduce the overall jitter [2]. 
>> 
>> I didn’t have time to read the whole paper but on a brief look, it looks like you take the delay into account for TCP traffic. Which is what I said above: either you have two links which have more or less the same delay or re-ordering will be wrongly interpreted as congestion. The other problem is TCP reduces its sending rate due to loss. So if both links are congested and losses occurred (in a non-synchronized way), TCP will react to both signals and reduce its sending rate more than necessary. That's why I would recommend to schedule TCP as well as other reliable and congestion-controlled traffic on a per-flow base. Further for UDP traffic there is no good way to actually know if the traffic is reliably transmitted and congestion control, so it’s hard to make such a decision in the network in a general case.
>> 
>>> 
>>> We understand your concern on this issue (and you can imagine that you are not the first one who raises this ;). It is also something that we want to have further experience from this *experimental* draft, as we stated in the “experiments to be conducted” section:
>>> 
>>> 	• Different path-selection schedulers. By default, round-robin scheduling is used to select a path to be used for datagrams. In some scenarios, weighted scheduling can be considered: for example, the paths with lower metrics (i.e., higher quality) can transfer more datagrams compared to paths with higher metrics. 
>>> 	• The impacts of the delay variation due to multipath routing. [RFC2991] brings out some concerns of multipath routing, especially variable latencies. Although current experiment results show that multipath routing can reduce the jitter in dynamic scenarios, some transport protocols or applications may be sensitive to the datagram re-ordering. 
>>> 	• The disjoint multipath protocol has interesting application with erasure coding, especially for services like video/audio streaming [WPMC11]. The combination of erasure coding mechanisms and this extension is thus encouraged.
>> 
>> That’s fine but then you cannot assume in the rest of the document that per-packet scheduling is used.
>> 
>> There are two options to handle this: either you remove the text that is cited above and do not talk about scheduling at all other than in the experimentation part, or you explain carefully when potentially per-packet scheduling could be used and make clear that by default and especially for TCP per-flow scheduling should be used.
>> 
>> 
>>> 
>>> 
>>> [1] Performance Evaluation of TCP over Mobile Ad-hoc Networks https://arxiv.org/pdf/1002.2189.pdf 
>>> [2] Multipath optimized link state routing for mobile ad hoc networks", In Elsevier Ad Hoc Journal, vol.9, n. 1, 28-47, January, 2011.
>>> 
>>>> 
>>>> Related is this text in section 8.4.:
>>>> "If datagrams without source routing header need to be forwarded using
>>>>  multiple paths (for example, based on the information of DiffServ
>>>>  Code Point [RFC2474])"
>>>> RFC2474 does not specify any application requirements on multipath use
>>>> and as such the DiffServe field should not be used to determine if the
>>>> flow can be routed on multiple paths. The ability to profit from
>>>> multipath routing depends not only on the application and protocols used
>>>> but also on the characteristics of the multipath link(s); so it's hard to
>>>> make any implicit assumptions here. However, if routing would only be
>>>> recommended on a per-flow basis this problem does not occur and the
>>>> brackets above could be remove. Further, if routed on a per flow basis
>>>> would be done, DiffServ could actually be used to decide which path to
>>>> use, if e.g. one path has a lower delay, but that seem to need further
>>>> discussion as well.
>>> 
>>> As we mentioned before, the multipath routing has special interests in audio/video streaming applications. For applications that requires strict ordering, single path can be used. For streaming and time-critical applications, the multi-path application might be more interesting, which can be identified by the DiffServe information. 
>> 
>> Which DiffServ codepoints are you taking about? Some of the code points require low delay but they either say nothing about reordering or require a bounded jitter as well which mean forwarding on a per-packet basis over two links which highly different delays should not be done.
>> 
>> Mirja
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> ----------------------------------------------------------------------
>>>> COMMENT:
>>>> ----------------------------------------------------------------------
>>>> 
>>>> Minor comments/questions:
>>>> 
>>>> 1) section 8.4: this sentence is not clear:
>>>> "It is RECOMMENDED to use MTU sizes considering the source routing
>>>>  header to avoid fragmentation."
>>>> MAYBE
>>>> "It is NOT RECOMMENDED to fragment the IP packet if the packet with the
>>>> source routing header would  exceed the minimum MTU along the path. In
>>>> this case source routing and therefore the additional path calculated by
>>>> MP-OLSRv2 SHOULD NOT be used.”
>>> 
>>> Fixed. 
>>> 
>>>> 
>>>> 2) section 9:
>>>> "For IPv6 networks, it MUST
>>>>     be set to 0, i.e., no constraint on maximum number of hops."
>>>> Why is that?
>>> 
>>> Because the current RFC6554 supports only IPv6 strict source routing, we thus need to keep all the path information in the source routing header, in which case we don’t know the number of hops. 
>>> 
>>> On the other hand, we noticed that the IPv6 segment routing header is being discussed at 6man, we thus have the following text in the “Experiments to be conducted” section:
>>> 
>>> 	• Use of IPv6 loose source routing. In the current specification, only strict source routing is used for IPv6 based on [RFC6554]. In [I‑D.ietf‑6man‑segment‑routing‑header], the use of loose source routing is also proposed in IPv6. In scenarios where the length of the source routing header is critical, the loose source routing can be considered.
>>> 
>>>> 
>>>> 3) Not sure why section 12.1. is there? Can this be removed?
>>> 
>>> Removed. 
>>> 
>>> Agains, we appreciate your valuable comments and hope that our reply addresses your concern. 
>>> 
>>> regards
>>> 
>>> Jiazi
>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> manet mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/manet
>