Re: [manet] Mirja Kühlewind's Discuss on draft-ietf-manet-olsrv2-multipath-12: (with DISCUSS and COMMENT)

[Abdussalam Baryun <[e-mail]>]

 I did not reference 8.4, just reply to the discussing about TCP, so we
don't do routing on TCP ports, yes we can have data plane using manet paths
but not reliable, so the discussion about reliability was not related. I
think the multipath work needs clear assignments for IPv4 and IPv6 not just
following RPL 6554 .

I should not think about what is implemented/used-RFC now, but we may think
of what if all our RFCs become implemented over night, so it should be
perfect or workable otherwise our review failed. My attention was that we
have allocated issues and I am talking as if thy are implemented (MANET
protocols usually are used, but no one will tell you he implemented it if
he/she did, they just use it).

AB

On Fri, May 26, 2017 at 6:11 PM, Ratliff, Stanley <[e-mail]>
wrote:

> AB,
>
>
>
> The text you reference in Section 8.4 is talking about the data being
> routed; not about the control plane (OLSR) traffic. For example, if regular
> browser traffic were to be carried by an OLSR network with the multipath
> enhancement, said browser traffic (which is TCP-based) MUST be delivered by
> the network in-sequence. The restriction stated in Section 8.4 preserves
> the ordering, since the traffic follows the same path.
>
>
>
> As to RFC5444 – IIRC, the WG decision at the time was to allow for both
> operation using UDP as a transport between routers, as well as for Layer 3
> communication (as OSPF does). The decision of whether to support UDP
> communication or Layer 3 communication is implementation-dependent. AFAIK,
> there are no Layer 3 implementations presently, but if anyone on the list
> knows of one, please correct me. However, that is the reason both a UDP
> port number and an IP number were allocated.
>
>
>
> Regards,
>
> Stan
>
>
>
> *From:* manet [mailto:[e-mail]] *On Behalf Of *Abdussalam
> Baryun
> *Sent:* Friday, May 26, 2017 11:52 AM
> *To:* Mirja Kuehlewind (IETF) <[e-mail]>
> *Cc:* manet <[e-mail]>; The IESG <[e-mail]>;
> [e-mail]
> *Subject:* Re: [manet] Mirja Kühlewind's Discuss on
> draft-ietf-manet-olsrv2-multipath-12: (with DISCUSS and COMMENT)
>
>
>
> Hi Mirja, and IESG,
>
>
>
> You first mentioned TCP or reliability in your first message and authors
> replying too, but IMO, in MANET WG drafts under IETF we still have no
> routing through TCP because our MANET-Packet as in RFC5444, is only having
> as mentioned in RFC5498 the IP number 138 and the UDP port number. So our
> OLSRv2 routers don't do any TCP ports. Regarding reliability, I don't think
> it is right to mention the reliability in MANET/this-draft, because it may
> need many other considerations.
>
>
>
> I think that we need to clarify in this  draft the manet packet rfc5444,
> and that it takes the manet messages including the multipaths. So in the
> draft mentions datagram, but not manet packet which is 5444 packet, is our
> source routing at ip layer or transport layer do we need ip number or
> transport port number.
>
>
>
> We need to add the below to section 12 as it was done in rfc7181 and 7722:
>
>
>
> Expert Review: Evaluation Guidelines:
>    For the registry where an Expert Review is required, the designated
>    expert SHOULD take the same general recommendations into
>    consideration as are specified by [RFC5444], [RFC5498], [RFC7631]
> and [RFC7722].
>
>
>
> This draft is using the same header type of rfc6554 which I may disagree.
> Furthermore, in RFC6554 routing has mentioned in its IANA section the
> assigned ipv6 number is 3 for RPL source routing (this draft is not for
> RPL), so for this multipath-protocol, we need to assign an IPv4 number for
> the manet-source-routing, and  we should mention it in section 12 as using
> 138 for IPv6 for manet packets 5444. As we have done in MANET DSR source
> routing RFC4728 assigned it for IPv4 number 48.
>
>
>
> In section 12 we need to update the table -1 to have a title similar to
> 7722 which is:
>
>
>
> Type 7 Message TLV Type Extensions
>
>
>
> however, and that we add the references and type extensions of 0 and 1
> which were allocated already. Add references in table 1 as 7181 and 7722.
>
>
>
> take care,
>
> AB
>
>
>
> On Wed, May 24, 2017 at 1:22 PM, Mirja Kuehlewind (IETF) <
> [e-mail]> wrote:
>
> Thanks! That’s much better!
>
> Two nits below.
>
> Mirja
>
>
> > Am 24.05.2017 um 00:26 schrieb Jiazi Yi <[e-mail]>:
> >
> > Hi Mirja,
> >
> > Thanks very much for your input. Now we have the text for section 8.4:
> >
> >> If a matching Multipath Routing Tuple is obtained, the Path Tuples of
> the Multipath Routing Tuple are applied to the datagrams using either
> per-flow scheduling or per-datagram scheduling, depending on the transport
> layer protocol and the application used. By default, per-flow scheduling is
> used, especially for the transport protocols that are sensitive to
> reordering, such as TCP. The path selection decision is made on the first
> datagram and all subsequent datagrams of the same flow use the same path.
> If the path is detected broken before the flow is closed, another path with
> the most similar metric is used. Per-datagram scheduling is recommended if
> the traffic is insensitive to reordering such as non-reliable transmission
> of media traffic, or when erasure coding is applied. In such case, each
> datagram selects their paths independently.
>
> s/each datagram selects their paths/each datagram selects its paths/
>
> >>
> >> By default, the traffic load is equally distributed in multiple paths.
> Other path scheduling
>
> Maybe
> s/the traffic load is equally distributed/the traffic load should be
> equally distributed/
>
>
> >>  mechanisms (e.g., assigning more traffic over better paths) are also
> possible and will not impact the interoperability of different
> implementations.
> >
> >
> > And we will also rephrase the text in section 4.
> >
> > Let us know if it’s OK for you.
> >
> > best
> >
> > Jiazi
> >
> >
> >> On 23 May 2017, at 14:10, Mirja Kuehlewind (IETF) <[e-mail]>
> wrote:
> >>
> >> Hi Jiazi,
> >>
> >> unfortunately, the following sentence is not correct, as basically any
> transport can be encapsulated over UDP:
> >>
> >>> Per-Datagram scheduling is recommended for transport layer protocols
> that have less constraint on datagram arriving order such as UDP.
> >>
> >> I would propose the following sentence:
> >>
> >>> Per-Datagram scheduling is only recommended if it is known that the
> traffic is not sensitive to reordering, e.g. for non-reliable transmission
> of media traffic.
> >>
> >> In this case I would further recommend to add the following sentence:
> >>
> >>> Note that the use of paths with highly different delays can still have
> a negative impact on these kind of transmissions, as packets that arrive
> too late may be ignored and therefore have been transmitted unnecessarily.
> >>
> >> Further I also don’t agree to this sentence:
> >>
> >>> In lossy networks, per-datagram scheduling is also recommended.
> >>
> >> Even in lossy networks, per-datagram scheduling is still not
> recommended for TCP (if the delay of the different paths are too diverse).
> >>
> >> Finally, I would further recommend you to not say that round-robin
> needs to be used in section 8.4. I also don’t think you need to have the
> two bullet points that explain what per-datagram and per-flow means. It’s
> enough to say that per-flow scheduling means that a decision is made on the
> first packet which path to use and all subsequent packets of the same flow
> (e.g. identified by the 5- or 6-tuple) need to use the same path.
> >>
> >> Further please also check section 4 as this also makes assumptions on
> the path selection.
> >>
> >> Thanks,
> >> Mirja
> >>
> >>
> >>
> >>> Am 23.05.2017 um 13:49 schrieb Jiazi Yi <[e-mail]>:
> >>>
> >>> Dear Mirja,
> >>>
> >>> Thank very much for your reply.
> >>>
> >>> We understand your concern on the performance, especially the issue of
> packet reordering when the multipath protocol is applied to transport
> protocol like TCP. Therefore, we propose to have some guidelines to
> indicate when per-flow scheduling is recommended, and when per-packet
> scheduling is recommended.
> >>>
> >>> More specifically, in the draft, we propose the text:
> >>>
> >>>
> >>> ==========
> >>> In section 1.1, Experiments to be conducted:
> >>>
> >>>     • Different path-selection schedulers. Depending on the
> application type and transport layer type either per-flow scheduler or
> per-datagram scheduler is applied. By default, round-robin scheduling is
> used to select a path to be used. In some scenarios, weighted scheduling
> can be considered: for example, the paths with lower metrics (i.e., higher
> quality) can transfer more datagrams or flows compared to paths with higher
> metrics.
> >>>
> >>>
> >>> In section 8.4, Datagram Processing at the MP-OLSRv2 Originator
> >>>
> >>> If a matching Multipath Routing Tuple is obtained, the Path Tuples of
> the Multipath Routing Tuple are applied to the datagrams using round-robin
> scheduling. The scheduling policy can be either per-flow or per-datagram,
> depending on the transport layer protocol and the application used:
> >>>
> >>>     - In per-flow scheduling, the datagrams of the same flow is
> transmitted through the same path. Different flows are assigned to
> different paths according to round-robin scheduling. For example, there are
> 2 path Tuples (Path-1, Path-2) for the destination router D. A series of
> flows (Flow-1, Flow-2, Flow-3, ... etc.) are to be sent to router D. Path-1
> is then chosen for Flow-1, Path-2 for Flow-2, Path-1 for Flow 3, etc.
> >>>
> >>>     - In per-datagram scheduling, different datagrams are transmitted
> through different paths according to round-robin scheduling. For example,
> there are 2 path Tuples (Path-1, Path-2) for the destination router D. A
> series of datagrams (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent
> to router D. Path-1 is then chosen for Packet-1, Path-2 for Packet-2,
> Path-1 for Packet 3, etc.
> >>>
> >>> Per-flow scheduling is recommended for transport layer protocols that
> require strict ordering of the datagrams such as TCP. By default, per-flow
> scheduling is used. Per-Datagram scheduling is recommended for transport
> layer protocols that have less constraint on datagram arriving order such
> as UDP. In lossy networks, per-datagram scheduling is also recommended.
> >>> Other path scheduling mechanisms are also possible and will not impact
> the interoperability of different implementations.
> >>>
> >>> ===========
> >>>
> >>> If it looks good for you, we will make the modification in the next
> revision of the draft.
> >>>
> >>> regards
> >>>
> >>> Jiazi
> >>>
> >>>
> >>>
> >>>> On 22 May 2017, at 12:17, Mirja Kuehlewind (IETF) <
> [e-mail]> wrote:
> >>>>
> >>>> Hi Jiazi,
> >>>>
> >>>> sorry for my very late reply. Please see below.
> >>>>
> >>>>> Am 10.05.2017 um 00:51 schrieb Jiazi Yi <[e-mail]>:
> >>>>>
> >>>>> Dear Mirja,
> >>>>>
> >>>>> Thanks very much for your review and comments. Please find the reply
> inline:
> >>>>>
> >>>>>> ------------------------------------------------------------
> ----------
> >>>>>> DISCUSS:
> >>>>>> ------------------------------------------------------------
> ----------
> >>>>>>
> >>>>>> The following text in section 4 seems to indicate that scheduling
> is done
> >>>>>> on a per-packet basis:
> >>>>>> "When there is a
> >>>>>> datagram to be sent to a destination, the source router acquires a
> >>>>>> path from the Multi-path Routing Set (MAY be Round-Robin, or other
> >>>>>> scheduling algorithms)."
> >>>>>> This seems not appropriate as e.g. TCP packets routed on links with
> >>>>>> largely different delays may suffer performance. ECMP usually
> hashes the
> >>>>>> 5-tuple or 6-tuple (incl. DiffServ Codepoint) to setup state and
> routes
> >>>>>> all packets belonging to the same flow on the same route. I
> recommend to
> >>>>>> apply the same here.
> >>>>>>
> >>>>>> Also related is this text in section 8.4 that should explain
> Round-Robin
> >>>>>> on a per flow basis instead. Further this should only be an example
> >>>>>> scheduling alogirthm while text belong seems to assume that
> Round-Robin
> >>>>>> is always used.
> >>>>>> "If a matching Multi-path Routing Tuple is obtained, the Path Tuples
> >>>>>> of the Multi-path Routing Tuple are applied to the datagrams using
> >>>>>> Round-robin scheduling.  For example, there are 2 path Tuples
> >>>>>> (Path-1, Path-2) for destination router D. A series of datagrams
> >>>>>> (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent router D.
> >>>>>> Path-1 is then chosen for Packet-1, Path-2 for Packet-2, Path-1 for
> >>>>>> Packet 3, etc.  Other path scheduling mechanisms are also possible
> >>>>>> and will not impact the interoperability of different
> >>>>>> implementations.”
> >>>>>
> >>>>> In fact, the per-packet scheduling is an intentional choice because:
> >>>>>
> >>>>> 1. The aimed scenario is mobile ad hoc networks, which have high
> packet loss by nature. One of the most important reasons is route failure
> due to mobility — in such case, if a per-flow scheduling is applied, the
> whole flow will lost. On the other hand, if we send the packets in disjoint
> paths (not necessarily equal cost), we can still make use partial
> information of the flow, or even reconstruct the flow with erasure coding.
> This is especially interesting for video/audio streaming.
> >>>>
> >>>> This is only true for unreliable or partially reliable traffic. I see
> your use case. However, will if you have TCP traffic, you can probably
> assume that the traffic is reliable and should not route on a per-packet
> basis. This case is not covered in your draft. Also for other non-TCP you
> often might not know much about the traffic characteristics and therefore
> cannot know if per-packet scheduling is good or bad. Therefore default
> should be per-flow.
> >>>>
> >>>>>
> >>>>> 2. In such kind of lossy networks, the traditional TCP performs very
> poor [1]. So as far as I know, TCP is not very popular in ad hoc networks.
> On the other hand, based on our study, the multi-path routing can actually
> reduce the overall jitter [2].
> >>>>
> >>>> I didn’t have time to read the whole paper but on a brief look, it
> looks like you take the delay into account for TCP traffic. Which is what I
> said above: either you have two links which have more or less the same
> delay or re-ordering will be wrongly interpreted as congestion. The other
> problem is TCP reduces its sending rate due to loss. So if both links are
> congested and losses occurred (in a non-synchronized way), TCP will react
> to both signals and reduce its sending rate more than necessary. That's why
> I would recommend to schedule TCP as well as other reliable and
> congestion-controlled traffic on a per-flow base. Further for UDP traffic
> there is no good way to actually know if the traffic is reliably
> transmitted and congestion control, so it’s hard to make such a decision in
> the network in a general case.
> >>>>
> >>>>>
> >>>>> We understand your concern on this issue (and you can imagine that
> you are not the first one who raises this ;). It is also something that we
> want to have further experience from this *experimental* draft, as we
> stated in the “experiments to be conducted” section:
> >>>>>
> >>>>>   • Different path-selection schedulers. By default, round-robin
> scheduling is used to select a path to be used for datagrams. In some
> scenarios, weighted scheduling can be considered: for example, the paths
> with lower metrics (i.e., higher quality) can transfer more datagrams
> compared to paths with higher metrics.
> >>>>>   • The impacts of the delay variation due to multipath routing.
> [RFC2991] brings out some concerns of multipath routing, especially
> variable latencies. Although current experiment results show that multipath
> routing can reduce the jitter in dynamic scenarios, some transport
> protocols or applications may be sensitive to the datagram re-ordering.
> >>>>>   • The disjoint multipath protocol has interesting application with
> erasure coding, especially for services like video/audio streaming
> [WPMC11]. The combination of erasure coding mechanisms and this extension
> is thus encouraged.
> >>>>
> >>>> That’s fine but then you cannot assume in the rest of the document
> that per-packet scheduling is used.
> >>>>
> >>>> There are two options to handle this: either you remove the text that
> is cited above and do not talk about scheduling at all other than in the
> experimentation part, or you explain carefully when potentially per-packet
> scheduling could be used and make clear that by default and especially for
> TCP per-flow scheduling should be used.
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>> [1] Performance Evaluation of TCP over Mobile Ad-hoc Networks
> https://arxiv.org/pdf/1002.2189.pdf
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__arxiv.org_pdf_1002.2189.pdf&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=kVrk59fKkcL1V7Qr7L9glnGQ9yChwWSXhj0aCNFn2do&e=>
> >>>>> [2] Multipath optimized link state routing for mobile ad hoc
> networks", In Elsevier Ad Hoc Journal, vol.9, n. 1, 28-47, January, 2011.
> >>>>>
> >>>>>>
> >>>>>> Related is this text in section 8.4.:
> >>>>>> "If datagrams without source routing header need to be forwarded
> using
> >>>>>> multiple paths (for example, based on the information of DiffServ
> >>>>>> Code Point [RFC2474])"
> >>>>>> RFC2474 does not specify any application requirements on multipath
> use
> >>>>>> and as such the DiffServe field should not be used to determine if
> the
> >>>>>> flow can be routed on multiple paths. The ability to profit from
> >>>>>> multipath routing depends not only on the application and protocols
> used
> >>>>>> but also on the characteristics of the multipath link(s); so it's
> hard to
> >>>>>> make any implicit assumptions here. However, if routing would only
> be
> >>>>>> recommended on a per-flow basis this problem does not occur and the
> >>>>>> brackets above could be remove. Further, if routed on a per flow
> basis
> >>>>>> would be done, DiffServ could actually be used to decide which path
> to
> >>>>>> use, if e.g. one path has a lower delay, but that seem to need
> further
> >>>>>> discussion as well.
> >>>>>
> >>>>> As we mentioned before, the multipath routing has special interests
> in audio/video streaming applications. For applications that requires
> strict ordering, single path can be used. For streaming and time-critical
> applications, the multi-path application might be more interesting, which
> can be identified by the DiffServe information.
> >>>>
> >>>> Which DiffServ codepoints are you taking about? Some of the code
> points require low delay but they either say nothing about reordering or
> require a bounded jitter as well which mean forwarding on a per-packet
> basis over two links which highly different delays should not be done.
> >>>>
> >>>> Mirja
> >>>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> ------------------------------------------------------------
> ----------
> >>>>>> COMMENT:
> >>>>>> ------------------------------------------------------------
> ----------
> >>>>>>
> >>>>>> Minor comments/questions:
> >>>>>>
> >>>>>> 1) section 8.4: this sentence is not clear:
> >>>>>> "It is RECOMMENDED to use MTU sizes considering the source routing
> >>>>>> header to avoid fragmentation."
> >>>>>> MAYBE
> >>>>>> "It is NOT RECOMMENDED to fragment the IP packet if the packet with
> the
> >>>>>> source routing header would  exceed the minimum MTU along the path.
> In
> >>>>>> this case source routing and therefore the additional path
> calculated by
> >>>>>> MP-OLSRv2 SHOULD NOT be used.”
> >>>>>
> >>>>> Fixed.
> >>>>>
> >>>>>>
> >>>>>> 2) section 9:
> >>>>>> "For IPv6 networks, it MUST
> >>>>>>   be set to 0, i.e., no constraint on maximum number of hops."
> >>>>>> Why is that?
> >>>>>
> >>>>> Because the current RFC6554 supports only IPv6 strict source
> routing, we thus need to keep all the path information in the source
> routing header, in which case we don’t know the number of hops.
> >>>>>
> >>>>> On the other hand, we noticed that the IPv6 segment routing header
> is being discussed at 6man, we thus have the following text in the
> “Experiments to be conducted” section:
> >>>>>
> >>>>>   • Use of IPv6 loose source routing. In the current specification,
> only strict source routing is used for IPv6 based on [RFC6554]. In
> [I‑D.ietf‑6man‑segment‑routing‑header], the use of loose source routing
> is also proposed in IPv6. In scenarios where the length of the source
> routing header is critical, the loose source routing can be considered.
> >>>>>
> >>>>>>
> >>>>>> 3) Not sure why section 12.1. is there? Can this be removed?
> >>>>>
> >>>>> Removed.
> >>>>>
> >>>>> Agains, we appreciate your valuable comments and hope that our reply
> addresses your concern.
> >>>>>
> >>>>> regards
> >>>>>
> >>>>> Jiazi
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> manet mailing list
> >>>>>> [e-mail]
> >>>>>> https://www.ietf.org/mailman/listinfo/manet
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_manet&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=Jhd9pV0Cw_wXKbnlFstVZICKvmTHz_Fz5G1AyjYvksI&e=>
> >>>
> >>
> >
> >
>
> _______________________________________________
> manet mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/manet
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_manet&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=Jhd9pV0Cw_wXKbnlFstVZICKvmTHz_Fz5G1AyjYvksI&e=>
>
>
> ------------------------------
> This electronic message and any files transmitted with it contains
> information from iDirect, which may be privileged, proprietary
> and/or confidential. It is intended solely for the use of the individual
> or entity to whom they are addressed. If you are not the original
> recipient or the person responsible for delivering the email to the
> intended recipient, be advised that you have received this email
> in error, and that any use, dissemination, forwarding, printing, or
> copying of this email is strictly prohibited. If you received this email
> in error, please delete it and immediately notify the sender.
>