Re: [6lo] [6tisch] Thomas' review of draft-thubert-6lo-forwarding-fragments-04

[Rahul Jadhav <[e-mail]>]

Hi Pascal,

Since you checked regarding WG interest ... I for one, would definitely
like to see the fragmentation problem getting worked upon. You have already
presented the problem statement and few design considerations during IET98
and I fully agree with those slides. If there is some way to avoid buffer
bloat on intermediate and end nodes during frag & reassembly, then it will
be really great!

Fragmentation at 6lo layer as it is today is almost unusable for a
moderately sized network with lower MTUs such as 127B.
If required, I can help to work on these issues from design as well as
implementation point of view.
Secondly, just want to add that during the relevant poll in 6lo meeting it
was very evident that WG considers this as an important problem.

Regards,
Rahul


On 5 Apr 2017 8:49 p.m., "Pascal Thubert (pthubert)" <[e-mail]>
wrote:

> Hello Thomas :
>
>
>
> All in all, your points are well understood, and I will restructure a bit
> and add words. But, before I spend too much of my time on this draft, I’d
> like to see the WG interest clearly expressed. So far I saw you and Michel
> expressing clear support. Is the group behind us?
>
>
>
>
>
> Please see below ;
>
>
>
> point 1: clearer intro needed
>
>
>
> from my understanding, this draft introduces two things: (1) a way of
> forwarding fragments so as to avoid to have to reassemble at every hop and
> (2) an end-to-end protocol to request lost fragments to be retransmitted. I
> think item (1) is really the main contribution of the draft, and (2) is a
> necessary corollary to making (1) work. The abstract and intro, however,
> seem to focus almost exclusively on (2), not clearly stating (1). A single
> sentence at the very beginning stressing the two contributions would go a
> long way IMO.
>
>
>
> [Pascal] OK, I’ll improve that on the next revision. Note that the initial
> title of the draft was draft-thubert-6lowpan-simple-fragment-recovery
> <https://tools.ietf.org/html/draft-thubert-6lowpan-simple-fragment-recovery-00>
> ; since then, I moved the focus on the capability to streamli,ne the
> fragments over a mesh. But both are really important. Forwarding
> fragments is now the title, and the draft improves the throughput by
> streamlining while avoiding buffer bloat inside the network.
>
> Yet, recovering the missing frags is important as well, avoids the bloat
> in the end point. Looking at the big picture, it’s very hard to do one
> without the other, the techniques benefit from one another and are not
> separable.
>
>
>
> ---
>
>
>
> point 2: justification
>
>
>
> Section 3 provides a rationale as to why this draft is needed. It focuses
> almost exclusively on reliability: if the link is 99% reliable, then the
> probability that 5 fragments, etc. IMO that doesn't make much sense as each
> fragment will be link-layer ACK'ed and retransmitted (at the link layer,
> i.e."transparently" to 6LoWPAN). There is of course a probability that all
> retries fail, but justifying the draft by assuming there are no link-layer
> ACKs make little sense. I may completely misunderstand, in which case,
> please point that out.
>
>
>
> [Pascal] We want to make the transmission reliable over multihop. I agree
> with what you are saying over one hop. But in the multihop scenario, if a
> hop down the path temporarily fails, the retries there will exhaust, but
> the node that did the fragmentation will not be aware (no nack all the way
> back), as opposed to the single hop case. So it will keep sending the next
> fragments, contributing to the problem, wasting time and network resources,
> etc...
>
>
>
> The major rationale IMO for this draft is that it doesn't require
> intermediary nodes to reassemble! Why not just say that? Reassembly at each
> hop yields several problems, the biggest one in my experience is that of
> memory management (if I'm missing one fragment from a packet, and a
> fragment from a second packet arrive, and I have only one assembly buffer,
> what should I do?).
>
>
>
> [Pascal] Yes, this is one of the problems I illustrated the most at the
> 6lo meeting, as you saw. Now, the bloat may happen in the receive end node,
> even when fragments are streamlined. It happens as soon as one fragment is
> lost. So forwarding fragments protects the network, making the fragments
> reliable protects the packet and the end point.
>
>
>
> I feel like I'm missing something, or even reading a draft that is not
> what I expected it to be...
>
>
>
> [Pascal] Can’t say. But both are needed and help one another to the point
> that they cannot be separated. The draft addresses all the points in my
> presentation at 6lo (https://www.ietf.org/proceedings/98/slides/slides-
> 98-6lo-fragmentation-forwarding-issues-in-6lowpan-00.pdf), but the hidden
> terminal which is not an issue for 6TiSCH and is only alleviated by the
> proposed flow control and the window of one fragment. The value that you
> focus on is discussed slides 9-12, and was a focus of the discussion.
>
> ---
>
>
>
> point 3: Section 5 Overview is not an overview
>
>
>
> Section 5 is called "Overview" and appears before the actual proposal is
> described. From the title, I was expecting a couple of paragraphs with the
> main idea, maybe a diagram or two to make the big picture clear. Instead,
> the section contains a discussion about very subtle points, including
> congestion control, interaction with link-layer protocols, timeout
> calculation and security.
>
>
>
> To summarize, the text in this section is interesting and should certainly
> appear in the specification, but as discussion points AFTER the main idea
> is presented.
>
>
>
> [Pascal] Ack
>
> ---
>
>
>
> point 4: the confusing use of the word "acknowledgment"
>
>
>
> While understandable, the use of acknowledgment which can refer to L2 ACKs
> or RFRAG-ACK messages is confused. I suggest to use the term "RFRAG-ACK"
> rather than the generic "Acknowledgement".
>
>
>
> [Pascal] Ack
>
>
>
>
>
> ---
>
>
>
> point 5: label switching
>
>
>
> The label switching mechanism is elegant as the labels are locally
> significant only, with no need to maintain a network-wide label registry.
> The document should state so.
>
>
>
> [Pascal] Ack
>
>
>
> For the rest I answered in line, please see below.
>
>
>
> Take care,
>
>
>
> Pascal
>
>
>
>
>
> ===
>
>
>
> 6lo                                                      P. Thubert, Ed.
>
> Internet-Draft                                             Cisco Systems
>
> Intended status: Standards Track                                  J. Hui
>
> Expires: July 14, 2017                                         Nest Labs
>
>                                                         January 10, 2017
>
>
>
>
>
>                   LLN Fragment Forwarding and Recovery
>
>                draft-thubert-6lo-forwarding-fragments-04
>
>
>
> Abstract
>
>
>
>    In order to be routed, a fragmented 6LoWPAN packet must be
>
>    reassembled at every hop of a multihop link where lower layer
>
>    fragmentation occurs.  Considering that the IPv6 minimum MTU is 1280
>
>    bytes, and that an 802.15.4 frame can have a payload limited to 74
>
>    bytes in the worst case, a packet might end up fragmented into as
>
>    many as 18 fragments by 6LoWPAN.
>
> TW> not sure the term "shim layer" is widely used. I suggest to replace
> simply by "6LoWPAN" (done already)
>
>    If a single one of
>
>    those fragments is lost in transmission, all fragments must be
>
>    resent, further contributing to the congestion that might have caused
>
>    the initial packet loss.  This specification introduces a simple
> protocol to
>
>    forward and recover individual fragments that might be lost over
>
>    multiple hops between 6LoWPAN endpoints.
>
>
>
> Status of This Memo
>
>
>
>    This Internet-Draft is submitted in full conformance with the
>
>    provisions of BCP 78 and BCP 79.
>
>
>
>    Internet-Drafts are working documents of the Internet Engineering
>
>    Task Force (IETF).  Note that other groups may also distribute
>
>    working documents as Internet-Drafts.  The list of current Internet-
>
>    Drafts is at http://datatracker.ietf.org/drafts/current/.
>
>
>
>    Internet-Drafts are draft documents valid for a maximum of six months
>
>    and may be updated, replaced, or obsoleted by other documents at any
>
>    time.  It is inappropriate to use Internet-Drafts as reference
>
>    material or to cite them other than as "work in progress."
>
>
>
>    This Internet-Draft will expire on July 14, 2017.
>
>
>
> Copyright Notice
>
>
>
>    Copyright (c) 2017 IETF Trust and the persons identified as the
>
>    document authors.  All rights reserved.
>
>
>
>    This document is subject to BCP 78 and the IETF Trust's Legal
>
>    Provisions Relating to IETF Documents
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 1]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
>    (http://trustee.ietf.org/license-info) in effect on the date of
>
>    publication of this document.  Please review these documents
>
>    carefully, as they describe your rights and restrictions with respect
>
>    to this document.  Code Components extracted from this document must
>
>    include Simplified BSD License text as described in Section 4.e of
>
>    the Trust Legal Provisions and are provided without warranty as
>
>    described in the Simplified BSD License.
>
>
>
> Table of Contents
>
>
>
>    1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
>
>    2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
>
>    3.  Rationale . . . . . . . . . . . . . . . . . . . . . . . . . .   4
>
>    4.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   5
>
>    5.  Overview  . . . . . . . . . . . . . . . . . . . . . . . . . .   6
>
>    6.  New Dispatch types and headers  . . . . . . . . . . . . . . .   8
>
>      6.1.  Recoverable Fragment Dispatch type and Header . . . . . .   8
>
>      6.2.  Fragment acknowledgment Dispatch type and Header  . . . .   8
>
>    7.  Fragments Recovery  . . . . . . . . . . . . . . . . . . . . .  10
>
>    8.  Forwarding Fragments  . . . . . . . . . . . . . . . . . . . .  11
>
>      8.1.  Upon the first fragment . . . . . . . . . . . . . . . . .  12
>
>      8.2.  Upon the next fragments . . . . . . . . . . . . . . . . .  13
>
>      8.3.  Upon the fragment acknowledgments . . . . . . . . . . . .  13
>
>    9.  Security Considerations . . . . . . . . . . . . . . . . . . .  14
>
>    10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  14
>
>    11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  14
>
>    12. References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
>
>      12.1.  Normative References . . . . . . . . . . . . . . . . . .  14
>
>      12.2.  Informative References . . . . . . . . . . . . . . . . .  15
>
>    Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  16
>
>
>
> 1.  Introduction
>
>
>
>    In most Low Power and Lossy Network (LLN) applications, the bulk of
>
>    the traffic consists of small chunks of data (in the order few bytes
>
>    to a few tens of bytes) at a time.  Given that an 802.15.4 frame can
>
>    carry 74 bytes or more in all cases, fragmentation is usually not
>
>    required.  However, and though this happens only occasionally, a
>
>    number of mission-critical applications do require the capability to
>
>    transfer larger chunks of data, for instance to support firmware
>
>    update of the LLN nodes, or an extraction of logs from LLN nodes.
>
>    In the former case, the large chunk of data is transferred to the LLN
>
>    node, whereas in the latter, the large chunk flows away from the LLN
>
>    node.  In both cases, the size can be on the order of 10 kB or
>
>    more, and an end-to-end reliable transport is required.
>
>
>
>    Mechanisms such as TCP or application-layer segmentation will be used
>
>    to support end-to-end reliable transport.  One option to support bulk
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 2]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
>    data transfer over a frame-size-constrained LLN is to set the Maximum
>
>    Segment Size to fit within the link maximum frame size.  Doing so,
>
>    however, can add significant header overhead to each 802.15.4 frame.
>
>    This causes the end-to-end transport to be intimately aware of the
>
>    delivery properties of the underlaying LLN, which is a layer
>
>    violation.
>
>
>
>    An alternative mechanism uses 6LoWPAN fragmentation in
>
>    addition to transport or application-layer segmentation.  Increasing
>
>    the Maximum Segment Size reduces header overhead by the end-to-end
>
>    transport protocol.  It also encourages the transport protocol to
>
>    reduce the number of outstanding datagrams, ideally to a single
>
>    datagram, thus reducing the need to support out-of-order delivery
>
>    common to LLNs.
>
>
>
>    [RFC4944] defines a datagram fragmentation mechanism for LLNs.
>
>    However, because [RFC4944] does not define a mechanism for recovering
>
>    fragments that are lost, datagram forwarding fails if even one
>
>    fragment is not delivered properly to the next IP hop.  End-to-end
>
>    transport mechanisms will require retransmission of all fragments,
>
>    wasting resources in an already resource-constrained network.
>
>
>
>    Past experience with fragmentation has shown that mis-associated or
>
>    lost fragments can lead to poor network behavior and, eventually,
>
>    trouble at application layer.  The reader is encouraged to read
>
>    [RFC4963] and follow the references for more information.  That
>
>    experience led to the definition of the Path MTU discovery [RFC1191]
>
>    protocol that limits fragmentation over the Internet.
>
>
>
>    For one-hop communications, a number of media propose a local
>
>    acknowledgment mechanism that is enough to protect the fragments.  In
>
>    a multihop environment, an end-to-end fragment recovery mechanism
>
>    might be a good complement to a hop-by-hop MAC level recovery.  This
>
>    draft introduces a simple protocol to recover individual fragments
>
>    between 6LoWPAN endpoints.  Specifically in the case of UDP, valuable
>
>    additional information can be found in UDP Usage Guidelines for
>
>    Application Designers [RFC5405].
>
>
>
> 2.  Terminology
>
>
>
>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>
>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>
>    document are to be interpreted as described in [RFC2119].
>
>
>
>    Readers are expected to be familiar with all the terms and concepts
>
>    that are discussed in "IPv6 over Low-Power Wireless Personal Area
>
>    Networks (6LoWPANs): Overview, Assumptions, Problem Statement, and
>
>
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 3]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
>    Goals" [RFC4919] and "Transmission of IPv6 Packets over IEEE 802.15.4
>
>    Networks" [RFC4944].
>
>
>
>    ERP
>
>
>
>       Error Recovery Procedure.
>
>
>
>    6LoWPAN endpoints
>
>
>
>       The LLN nodes in charge of generating or expanding a 6LoWPAN
>
>       header from/to a full IPv6 packet.  The 6LoWPAN endpoints are the
>
>       points where fragmentation and reassembly take place.
>
> TW> should we add a sentence that states that one of those point is NOT
> necessarily the LBR?
>
>
>
>
>
>
>
> [Pascal]Not sure there is a need; this is not a ROLL document, there is
> not necessary such a thing as a 6LBR
>
>
>
>
>
> 3.  Rationale
>
>
>
>    There are a number of use cases for large packets in Wireless Sensor
>
>    Networks.  Such usages may not be the most typical or represent the
>
>    largest amount of traffic over the LLN; however, the associated
>
>    functionality can be critical enough to justify extra care for
>
>    ensuring effective transport of large packets across the LLN.
>
>
>
>    The list of those use cases includes:
>
>
>
>    Towards the LLN node:
>
>
>
>       Packages of Commands:  A number of commands or a full
>
>          configuration can be packaged as a single message to ensure
>
>          consistency and enable atomic execution or complete rollback.
>
>          Until such commands are fully received and interpreted, the
>
>          intended operation will not take effect.
>
>
>
>       Firmware update:  For example, a new version of the LLN node
>
>          software is downloaded from a system manager over unicast or
>
>          multicast services.  Such a reflashing operation typically
>
>          involves updating a large number of similar LLN nodes over a
>
>          relatively short period of time.
>
>
>
>    From the LLN node:
>
>
>
>       Waveform captures:  A number of consecutive samples are measured
>
>          at a high rate for a short time and then transferred from a
>
>          sensor to a gateway or an edge server as a single large report.
>
>
>
>       Data logs:  LLN nodes may generate large logs of sampled data for
>
>          later extraction.  LLN nodes may also generate system logs to
>
>          assist in diagnosing problems on the node or network.
>
>
>
>
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 4]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
>       Large data packets:  Rich data types might require more than one
>
>          fragment.
>
>
>
>    Uncontrolled firmware download or waveform upload can easily result
>
>    in a massive increase of the traffic and saturate the network.
>
>
>
>    When a fragment is lost in transmission, all fragments are resent,
>
>    further contributing to the congestion that caused the initial loss,
>
>    and potentially leading to congestion collapse.
>
>
>
>    This saturation may lead to excessive radio interference, or random
>
>    early discard (leaky bucket) in relaying nodes.  Additional queuing
>
>    and memory congestion may result while waiting for a low power next
>
>    hop to emerge from its sleeping state.
>
>
>
>    To demonstrate the severity of the problem, consider a fairly
>
>    reliable 802.15.4 frame delivery rate of 99.9% over a single 802.15.4
>
>    hop.  The expected delivery rate of a 5-fragment datagram would be
>
>    about 99.5% over a single 802.15.4 hop.  However, the expected
>
>    delivery rate would drop to 95.1% over 10 hops, a reasonable network
>
>    diameter for LLN applications.  The expected delivery rate for a
>
>    1280-byte datagram is 98.4% over a single hop and 85.2% over 10 hops.
>
> TW> I don't think this paragraph makes much sense.
>
> TW> You would have link-layer ACKs and retries to ensure that all
> fragments reach the next hop.
>
>
>
>
>
> [Pascal] In order to fit within transport time outs, you want to limit the
> number of retries to an acceptable time. In the 99.9% I assume 90%
> delivery, 3 tries, and complete independence of the 3 tries. All of these
> assumptions are completely wrong, but that gives an idea of how the
> problems adds up in a mesh. Still I can remove the paragraph, since, after
> all , it is not needed in a solution draft.
>
>
>
>
>
>    Considering that [RFC4944] defines an MTU is 1280 bytes and that in
>
>    most incarnations (but 802.15.4G) a 802.15.4 frame can limit the MAC
>
>    payload to as few as 74 bytes, a packet might be fragmented into at
>
>    least 18 fragments by 6LoWPAN.  Taking into account
>
>    the worst-case header overhead for 6LoWPAN Fragmentation and Mesh
>
>    Addressing headers will increase the number of required fragments to
>
>    around 32.  This level of fragmentation is much higher than that
>
>    traditionally experienced over the Internet with IPv4 fragments.  At
>
>    the same time, the use of radios increases the probability of
>
>    transmission loss and Mesh-Under techniques compound that risk over
>
>    multiple hops.
>
>
>
> 4.  Requirements
>
>
>
>    This specification proposes a method to recover individual fragments
> between
>
>    LLN endpoints.
>
> TW> add that endpoints can be multiple hops away from one another?
>
>
>
>
>
> [Pascal] Exactly, I’ll add the words
>
>
>
>    The method is designed to fit the following
>
>    requirements of a LLN (with or without a Mesh-Under routing
>
>    protocol):
>
>
>
>    Number of fragments
>
>
>
>       The recovery mechanism must support highly fragmented packets,
>
>       with a maximum of 32 fragments per packet.
>
> TW> resulting length of the packet? The abstract implies a 32*74=2368 byte
> limit. Is that correct?
>
>
>
> [Pascal] Yes, depends on the security overhead which is variable. But I’d
> say that the reasonable goal of MTU=2048, which is the usual IEEE bar, is
> easily achieved.
>
>
>
>
>
>    Minimum acknowledgment overhead
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 5]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
>       Because the radio is half duplex, and because of silent time spent
>
>       in the various medium access mechanisms, an acknowledgment
>
>       consumes roughly as many resources as data fragment.
>
>
>
>       The recovery mechanism should be able to acknowledge multiple
>
>       fragments in a single message and not require an acknowledgment at
>
>       all if fragments are already protected at a lower layer.
>
>
>
> TW> maybe specify here that we are talking about end-to-end ACKs between
> endpoints, not L2 ACKs
>
>
>
>
>
> [Pascal] Will do
>
>
>
>
>
>    Controlled latency
>
>
>
>       The recovery mechanism must succeed or give up within the time
>
>       boundary imposed by the recovery process of the Upper Layer
>
>       Protocols.
>
>
>
>    Support for out-of-order fragment delivery
>
>
>
>       A Mesh-Under load balancing mechanism such as the ISA100 Data Link
>
>       Layer can introduce out-of-sequence packets.
>
>
>
> TW> I don't fully understand why we focus so much on mesh-under.
>
> TW> If you have a fully mesh-under network, 4944 fragmentation results in
> fragment forwarding.
>
> TW> I would remove the repeated mentioning on mesh-under (as well as
> ISA100)
>
>
>
>
>
> [Pascal] I’ll move the whole section to appendix; the draft is more needed
> in route over but still beneficial to mesh under, for the benefits that are
> not the buffer bloat. These include flow control, loss detection, and
> recovery.
>
>
>
>
>
>       The recovery mechanism must account for packets that appear lost
>
>       but are actually only delayed over a different path.
>
>
>
>    Optional congestion control
>
>
>
>       The aggregation of multiple concurrent flows may lead to the
>
>       saturation of the radio network and congestion collapse.
>
>
>
>       The recovery mechanism should provide means for controlling the
>
>       number of fragments in transit over the LLN.
>
>
>
> 5.  Overview
>
>
>
>    Considering that a multi-hop LLN can be a very sensitive environment
>
>    due to the limited queuing capabilities of a large population of its
>
>    nodes, this specification recommends a simple and conservative approach
> to
>
>    congestion control, based on TCP congestion avoidance.
>
> TW> "based on" or "inspired by"
>
>
>
> TW> The "overview" section delves immediately into subtle discussions
> about congestion control.
>
> TW> At this point, the reader (e.g. me) have no idea what the basic
> principle is.
>
> TW> Discussions about subtleties such as congestion control seem out of
> place here.
>
>
>
>
>
>
>
> [Pascal] Understood, will restructure, add words; that’s a bit of work so
> please give me some time.
>
>
>
>
>
>    Congestion on the forward path is assumed in case of packet loss, and
>
>    packet loss is assumed upon time out.  This specification allows to
> control
>
>    the number of outstanding fragments, that have been transmitted but
>
>    for which an acknowledgment was not received yet.  It must be noted
>
>    that the number of outstanding fragments should not exceed the number
>
>    of hops in the network, but the way to figure the number of hops is
>
>    out of scope for this document.
>
>
>
>    Congestion on the forward path can also be indicated by an Explicit
>
>    Congestion Notification (ECN) mechanism.  Though whether and how ECN
>
>    [RFC3168] is carried out over the LoWPAN is out of scope, this draft
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 6]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
>    provides a way for the destination endpoint to echo an ECN indication
>
>    back to the source endpoint in an acknowledgment message as
>
>    represented in Figure 5 in Section 6.2.
>
>
>
> TW> The following paragraph talks about how a link-layer transmits
> multiple frames in a row.
>
> TW> The link-layer does so whether those frames are fragments or regular
> packets.
>
> TW> I would remove this discussion, which seems to be link-layer only, and
> not specific to fragment forwarding.
>
>
>
> [Pascal] I can remove the text, but the problem remains, and your point,
> while largely correct, is not fully so. The transport layer may do some
> flow control based on command response, windowing and ECN, and thus protect
> against bursts of packets, but it cannot do a thing against bursts of
> fragments. So essentially, we are echoing some of the transport
> functionality at L2+ to provide those service to the fragments.
>
>
>
>
>
>    It must be noted that congestion and collision are different topics.
>
>    In particular, when a mesh operates on a same channel over multiple
>
>    hops, then the forwarding of a fragment over a certain hop may
>
>    collide with the forwarding of a next fragment that is following over
>
>    a previous hop but in a same interference domain.  This draft enables
>
>    an end-to-end flow control, but leaves it to the sender stack to pace
>
>    individual fragments within a transmit window, so that a given
>
>    fragment is sent only when the previous fragment has had a chance to
>
>    progress beyond the interference domain of this hop.  In the case of
>
>    6TiSCH [I-D.ietf-6tisch-architecture], which operates over the
>
>    TimeSlotted Channel Hopping [I-D.ietf-6tisch-tsch] (TSCH) mode of
>
>    operation of IEEE802.14.5, a fragment is forwarded over a different
>
>    channel at a different time and it make full sense to fire a next
>
>    fragment as soon as the previous fragment has had its chance to be
>
>    forwarded at the next hop, retry (ARQ) operations included.
>
>
>
>    From the standpoint of a source 6LoWPAN endpoint, an outstanding
>
>    fragment is a fragment that was sent but for which no explicit
>
>    acknowledgment was received yet.  This means that the fragment might
>
>    be on the way, received but not yet acknowledged, or the
>
>    acknowledgment might be on the way back.  It is also possible that
>
>    either the fragment or the acknowledgment was lost on the way.
>
>
>
>    Because a meshed LLN might deliver frames out of order, it is
>
>    virtually impossible to differentiate these situations.  In other
>
>    words, from the sender's standpoint, all outstanding fragments might
>
>    still be in the network and contribute to its congestion.  There is
>
>    an assumption, though, that after a certain amount of time, a frame
>
>    is either received or lost, so it is not causing congestion anymore.
>
>    This amount of time can be estimated based on the round trip delay
>
>    between the 6LoWPAN endpoints.  The method detailed in [RFC6298] is
>
>    recommended for that computation.
>
>
>
>    The reader is encouraged to read through "Congestion Control
>
>    Principles" [RFC2914].  Additionally [RFC2309] and [RFC5681] provide
>
>    deeper information on why this mechanism is needed and how TCP
>
>    handles Congestion Control.  Basically, the goal here is to manage
>
>    the amount of fragments present in the network; this is achieved by
>
>    to reducing the number of outstanding fragments over a congested path
>
>    by throttling the sources.
>
>
>
>    Section 7 describes how the sender decides how many fragments are
>
>    (re)sent before an acknowledgment is required, and how the sender
>
>    adapts that number to the network conditions.
>
>
>
>
>
>
>
> Thubert & Hui             Expires July 14, 2017                 [Page 7]
>
> Internet-Draft    LLN Fragment Forwarding and Recovery      January 2017
>
>
>
>
>
> 6.  New Dispatch types and headers
>
>
>
>    This specification extends "Transmission of IPv6 Packets over IEEE
>
>    802.15.4 Networks" [RFC4944] with 4 new dispatch types, for
>
>    Recoverable Fragments (RFRAG) headers with or without Acknowledgment
>
>    Request, and for the Acknowledgment back, with or without ECN Echo.
>
>
>
>             Pattern    Header Type
>
>           +------------+-----------------------------------------------+
>
>           | 11  101000 | RFRAG      - Recoverable Fragment             |
>
>           | 11  101001 | RFRAG-AR   - RFRAG with Ack Request           |
>
>           | 11  101010 | RFRAG-ACK  - RFRAG Acknowledgment             |
>
>           | 11  101011 | RFRAG-AEC  - RFRAG Ack with ECN Echo          |
>
>           +------------+-----------------------------------------------+
>
>
>
>              Figure 1: Additional Dispatch Value Bit Patterns
>
>
>
>    In the following sections, the semantics of "datagram_tag",
>
>    "datagram_offset" and "datagram_size" and the reassembly process are
>
>    changed from [RFC4944] Section 5.3.  "Fragmentation Type and Header".
>
>    The size and offset are expressed on the compressed packet per
>
>    [RFC6282] as opposed to the uncompressed - native packet - form.
>
>
>
> 6.1.  Recoverable Fragment Dispatch type and Header
>
>
>
>                             1                   2                   3
>
>         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>        |1 1 1 0 1 0 0 X|datagram_offset|         datagram_tag          |
>
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>        |Sequence |    datagram_size    |
>
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>                                                   X set == Ack Requested
>
>
>
>           Figure 2: Recoverable Fragment Dispatch type and Header
>
>
>
>    X: 1 bit; When set, the sender requires an Acknowledgment from the
>
>       receiver
>
>
>
>    Sequence:  5 bits; The sequence number of the fragment.  Fragments
>
>       are numbered [0..N] where N is in [0..31].
>
>
>
> 6.2.  Fragment acknowledgment Dispatch type and Header
>
>
>
>    The specification also defines a 4-octet ackn
>
> _______________________________________________
> 6tisch mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/6tisch
>
> ...