Re: [6lo] Benjamin Kaduk's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)

[Benjamin Kaduk <[e-mail]>]

On Mon, Mar 09, 2020 at 05:12:11PM +0000, Pascal Thubert (pthubert) wrote:
> Hello Again Benjamin,
> 
> Continuing with the COMMENT piece of your really appreciated deep review below: 
> 
> And as usual please find the diffs, and since this impacts both drafts we get:
> 
> https://www.ietf.org/rfcdiff?url2=draft-ietf-6lo-minimal-fragment-14
> https://www.ietf.org/rfcdiff?url2=draft-ietf-6lo-fragment-recovery-15 
>  
> Please find the discussion below
> 
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> > 
> > It might be worth mentioning that whereas the RFC 4944 fragmentation
> > procedures uses different dispatch types for the first fragment and subsequent
> > fragments, we have no need to do so (the first fragment is identified by
> > sequence number zero).
> 
> I agree with your statement; that's because with RFC 4944 the first and next fragments have a different format to save a few bits in the first fragment. 
> 
> Added the least sentence in:
> "
>    "LLN Minimal Fragment Forwarding" [FRAG-FWD] specifies the generic
>    behavior that all FF techniques including this specification follow,
>    and presents the associated caveats.  In particular, the routing
>    information is fully indicated in the first fragment, which is always
>    forwarded first.  With this specification, the first fragment is
>    identified by a Sequence of 0 as opposed to a dispatch type in
>    [RFC4944].  
> "
> 
> 
> 
> > 
> > I have a question about the "stickiness" of per-fragment ECN markings.
> > That is, suppose that fragment 1 is marked as having experienced congestion
> > when it arrives at the recipient.  The sender requests an ACK after fragment 4,
> > and the recipient duly sets the E bit in the ack.
> > Suppose that fragments 5 through 9 then arrive with no congestion indicated,
> > and fragment 9 requests and ACK.  Does this new ACK still have to set the E bit
> > since, as discussed in Section 5.2, "at least one of the acknowledged fragments
> > was received with an Explicit Congestion Notification"?  My (very very non-
> > expert) recollection is that we tend to say that any given ECN marking should
> > produce at most one "congestion present" response on the return path.
> 
> 
> I experimented with FECNs and BECNs at the times of frame relay. I found that the number of times the bit was set was unreliable, could get many in a row not really because of the receiving end point but because the congested node sets it a lot. 
> I expect the 6LoWPAN network to look more like FR than IP. 
> Bottom line is that it is for the most part on the shoulder of the sender and that is out of scope (clarified with Mirja's DISCUSS).
> 
> To your point I suggest adding in the next section " 4.3.  Flow Control "
> "
>    An intermediate node that experiences a congestion MAY set the ECN
>    bit in a fragment, and the reassembling endpoint echoes the ECN
>    bit at most once at the next opportunity to acknowledge back.
> 
> "
> In section 6, the normative text becomes:
> "
> 
>    The RFRAG Acknowledgment carries an ECN indication for flow control
>    (see Appendix C).  The reassembling endpoint of a fragment with the
>    'E' (ECN) flag set MUST echo that information at most once by setting
>    the 'E' (ECN) flag in the next RFRAG Acknowledgment.
> 
> "
> 

Thanks!

> For my FR implementation I ended up applying at most one action every RTT (that was an IBM FRTE / FRSE).
> 
> > 
> > Section 1
> > 
> >    LLNs.  One critical issue with this original design is that routing
> >    an IPv6 [RFC8200] packet across a route-over mesh requires
> >    reassembling the full packet at each hop, which may cause latency
> >    along a path and an overall buffer bloat in the network.  The "6TiSCH
> > 
> > I think I encountered at least one good reference for this statement during my
> > reading this week (posibly just -minimal-fragment, but maybe something
> > referenced from there?); it might be nice to help with cross-linking.
> 
> The 2 next sentences are just those references : )  (see 4.6.3.  Fragment Forwarding  of https://tools.ietf.org/html/draft-ietf-6tisch-architecture-28 which may be your memory).
> 
> Changed the text to read like:
> 
> "
> 
> "Transmission of IPv6 Packets over IEEE 802.15.4 Networks" [RFC4944]
>    defines the original 6LoWPAN datagram fragmentation mechanism for
>    LLNs.  One critical issue with this original design is that routing
>    an IPv6 [RFC8200] packet across a route-over mesh requires the
>    reassembly of the packet at each hop.  The "6TiSCH Architecture"
>    [I-D.ietf-6tisch-architecture] indicates that this may cause latency
>    along a path and impact critical resources such as memory and
>    battery; to alleviate those undesirable effects it recommends using a
>    fragment forwarding (FF) technique .
> 
>    "LLN Minimal Fragment Forwarding" [FRAG-FWD] specifies the generic
>    behavior that all FF techniques including this specification follow,
>    and presents the associated caveats.
> 
> "

Nicely done :)

> > 
> >    That problem is exacerbated when forwarding fragments over multiple
> >    hops since a loss at an intermediate hop will not be discovered by
> >    either the source or the destination, and the source will keep on
> >    sending fragments, wasting even more resources in the network and
> >    possibly contributing to the condition that caused the loss to no
> >    avail since the datagram cannot arrive in its entirety.  [...]
> > 
> > nit: I'm not sure that "to no avail" is placed properly.
> 
> Restructured as follows:
> 
> "
>   That problem is exacerbated when forwarding fragments over multiple
>    hops since a loss at an intermediate hop will not be discovered by
>    either the source or the destination endpoints, and the source will
>    keep on sending fragments, wasting even more resources in the network
>    since the datagram cannot arrive in its entirety, and possibly
>    contributing to the condition that caused the loss.
> 
> "
> 
> 
> > 
> > Section 2.2
> > 
> >    "LLN Minimal Fragment Forwarding" [I-D.ietf-6lo-minimal-fragment]
> >    introduces the generic concept of a Virtual Reassembly Buffer (VRB)
> > 
> > It's perhaps debatable whether -minimal-fragment vs. lwig-vrb really
> > "introduces" the concept -- draft-ietf-6lo-minimal-fragment-12 doesn't really
> > say very much about it, to me.
> 
> True. Reassembly Buffer is an old concept, see https://tools.ietf.org/html/draft-ietf-intarea-frag-fragile-17#section-3.1 
> => s/ introduces/discusses /
> 
> 
> > 
> >    and specifies behaviours and caveats that are common to a large
> >    family of FF techniques including this, which fully inherits from
> >    that specification.
> > 
> > nit: I'd suggest rewording "including this" for clarity, perhaps "including the
> > mechanism specified by this document".
> 
> Done; also a bit of restructuring, giving:
> 
> "
> 
>    In this document, readers will encounter terms and concepts that are
>    discussed in "IPv6 over Low-Power Wireless Personal Area Networks
>    (6LoWPANs): Overview, Assumptions, Problem Statement, and Goals"
>    [RFC4919], "Transmission of IPv6 Packets over IEEE 802.15.4 Networks"
>    [RFC4944], and "Problem Statement and Requirements for IPv6 over
>    Low-Power Wireless Personal Area Network (6LoWPAN) Routing"
>    [RFC6606].
> 
>    "LLN Minimal Fragment Forwarding" [FRAG-FWD] discusses the generic
>    concept of a Virtual Reassembly Buffer (VRB) and specifies behaviors
>    and caveats that are common to a large family of FF techniques
>    including the mechanism specified by this document, which fully
>    inherits from that specification.  It also defines terms used in this
>    document: 6LoWPAN endpoints, Compressed Form, Datagram_Tag,
>    Datagram_Size, and Fragment_Offset.
> 
> "
> 
> 
> > 
> > Section 2.3
> > 
> >    6LoWPAN endpoints:  The LLN nodes in charge of generating or
> >       expanding a 6LoWPAN header from/to a full IPv6 packet.  The
> >       6LoWPAN endpoints are the points where fragmentation and
> >       reassembly take place.
> > 
> > [-minimal-fragment also defines this, and many of the other terms we define
> > here, with minimal-but-nonzero variation.  Given that we already have a
> > normative dependency on that document, perhaps we should pull definitions
> > from it as well, instead of repeating them?  If not, I note that on -minimal-
> > fragment, I commented:
> > nit: I think they are only "the [only] points" where fragmentation/reassembly
> > happen if the entire newtork is using fragment forwarding; in other cases some
> > intermediate nodes will also reassemble and (re)fragment.]
> 
> Yes, done already for another review. 
> 
> For the nit, I realize that I missed your point at the time. 
> 
> For the purpose of FF, the intermediate nodes you mention become the fragmentation endpoints, which are the points we care about here. I need to change both specs to use "6LoWPAN FF endpoints" as opposed to "6LoWPAN endpoints".
> 
> In minimal:
> "
> 
>    6LoWPAN Fragment Forwarding endpoints:  The 6FF endpoints are the
>       first and last nodes in an unbroken string of 6LoWPAN fragment
>       forwarding nodes.  They are also the only points where the
>       fragmentation and reassembly operations take place.
> "
>  And later
> "
>    We use Figure 1 to illustrate 6LoWPAN fragmentation.  We assume node
>    A forwards a packet to node B, possibly as part of a multi-hop route
>    between 6LoWPAN Fragment Forwarding endpoints which may be neither A
>    nor B, though 6LoWPAN may compress the IP header better when they are
>    both the 6FF and the 6LoWPAN compression endpoints.
> 
> "
> 
> 
> > 
> > Section 4.1
> > 
> >    This specification cannot allow this operation since fragments are
> >    recovered end-to-end based on a sequence number.  This means that the
> > 
> > I suggest s/this operation/such a refragmentation operation/.
> 
> Done
> 
> > 
> >    fragments that contain a 6LoWPAN-compressed header MUST have enough
> >    slack to enable a less efficient compression in the next hops that
> >    still fits in one MAC frame.  For instance, if the IID of the source
> > 
> > (This assumes a homogeneous MAC mesh, an assumption that might be worth
> > making explicit.
> 
> Well in fact the assumption is that the MIN MTU is known, since the fragmentation can adapt to that. 
> 4.1 becomes:
> 
> "
> 4.1.  Slack in the First Fragment
> 
>    [FRAG-FWD] allows for refragmenting in intermediate nodes, meaning
>    that some bytes from a given fragment may be left in the VRB to be
>    added to the next fragment.  The need for more space in the outgoing
>    fragment than was needed for the incoming fragment arises when the
>    6LoWPAN Header Compression is not as efficient on the outgoing link
>    or the Link MTU is reduced.
> 
>    This specification cannot allow such a refragmentation operation
>    since the fragments are recovered end-to-end based on a sequence
>    number.  The Fragment_Size MUST be tailored to fit the minimal MTU
>    along the path, and the first fragment that contains a 6LoWPAN-
>    compressed header MUST have enough slack to enable a less efficient
>    compression in the next hops to still fits within the Link MTU.  If
>    the fragmenting endpoint is also the 6LoWPAN compression endpoint, it
>    will elide the IID of the source IPv6 address if it matches the Link-
>    Layer address [RFC6282].  In a network with a consistent MTU, it MUST
>    compute the Fragment_Size as if the MTU was 8 bytes less, so the next
>    hop can expand the IID within the same fragment.
> "
> 
> 
> But the MTU discovery is out of scope.

Surely!

> Changed a bit section 6.2
> "
>    This specification does not provide a method to discover the number
>    of hops or the minimal value of MTU along those hops.  In a typical
>    case, the MTU is constant and the same across the network.  But
>    should the minimal MTU along the path decrease,
> "

Excellent, thank you

> 
> > 
> >    IPv6 address is elided by the originator, then it MUST compute the
> >    Fragment_Size as if the MTU was 8 bytes less.  This way, the next hop
> >    can restore the source IID to the first fragment without impacting
> >    the second fragment.
> > 
> > Is it worth noting that this to large extent obviates the benefits of header
> > compression?
> 
> On the next hops it does, yes. It is a  generic thing for all 6FF techniques. 
> It would be better to say it in minimal I guess. Text in minimal section 5 becomes:
> 
> "
>    Fragment forwarding obviates some of the benefits of the 6LoWPAN
>    header compression [RFC6282] in intermediate hops.  In return, the
>    memory used to store the packet is distributed along the path, which
>    limits the buffer bloat effect.  Multiple fragments may progress
>    simultaneously along the network as long as they do not interfere.
> "
> 
> > 
> > Section 4.2
> > 
> >    This specification introduces a concept of an inter-frame gap, which
> >    is a configurable interval of time between transmissions to the same
> > 
> > I think draft-ietf-6lo-minimal-fragment also talks about an "inter-frame gap",
> > though in less detail.
> 
> Yes, already changed for another review. Now reads as
> 
> "
> 
> [FRAG-FWD] requires that a configurable interval of time is inserted
>    between transmissions to the same next hop and in particular between
>    fragments of a same datagram.  In the case of half duplex interfaces,
>    this inter-frame gap ensures that the next hop is done forwarding the
>    previous frame and is capable of receiving the next one.
> 
> "
> 
> 
> 
> 
> > Section 4.3
> > 
> >    The compression of the Hop Limit, of the source and destination
> >    addresses in the IPv6 Header, and of the Routing Header may change en
> >    route in a Route-Over mesh LLN.  If the size of the first fragment is
> >    modified, then the intermediate node MUST adapt the Datagram_Size to
> >    reflect that difference.
> > 
> > Hmm, if this really only affects the first fragment, then maybe Section
> > 4.1 could be more clear that the need for extra "slack" in practice only applies
> > to the first fragment, as well.
> 
> Yes, see the revised section 4.1 above
> 
> 
> > 
> >    The intermediate node MUST also save the difference of Datagram_Size
> >    of the first fragment in the VRB and add it to the Datagram_Size and
> >    to the Fragment_Offset of all the subsequent fragments for that
> >    datagram.
> > 
> > The RFRAG header does not carry the Datagram_Size, so I think this needs
> > rewording (we do, however, need to store the updated size to detect the final
> > fragment).
> > 
> 
> It does, actually, well hidden in the Fragment_Offset of the first fragment. 
> Clearly you found that later in the review.

I assume I was just doing a "find in page"... :-/

> > Section 5
> > 
> >    This specification provides a technique that is derived from MPLS to
> >    forward individual fragments across a 6LoWPAN route-over mesh without
> >    reassembly at each hop.  The Datagram_Tag is used as a label; it is
> >    locally unique to the node that owns the source MAC address of the
> >    fragment, so together the MAC address and the label can identify the
> >    fragment globally.  A node may build the Datagram_Tag in its own
> > 
> > (More "globally unique" with implied "lifetime of the packet".)
> 
> Yes, moved it up:
> 
> "
> 
>    A 6LoWPAN Fragment Forwarding [FRAG-FWD] technique derived from MPLS
>    enables the forwarding of individual fragments across a 6LoWPAN
>    route-over mesh without reassembly at each hop.  The Datagram_Tag is
>    used as a label; it is locally unique to the node that owns the
>    source Link-Layer address of the fragment, so together the Link-Layer
>    address and the label can identify the fragment globally within the
>    lifetime of the datagram.  A node may build the Datagram_Tag in its
>    own locally-significant way, as long as the chosen Datagram_Tag stays
>    unique to the particular datagram for its lifetime.
> "
> 
> > 
> >    locally-significant way, as long as the chosen Datagram_Tag stays
> >    unique to the particular datagram for the lifetime of that datagram.
> > 
> > [okay, less implied and more explicit here than it was above :) ]
> > 
> indeed
> 
> > Section 5.1
> > 
> > We discuss the header format, but I don't think we actually say that "you put
> > the fragment payload after the header" :)
> > 
> >                              1                   2                   3
> >         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> >                                        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >                                        |1 1 1 0 1 0 0|E|  Datagram_Tag |
> >        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >        |X| Sequence|   Fragment_Size   |       Fragment_Offset         |
> >        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > 
> > I strongly suggest a stronger indicator that the field marked "Fragment_Offset"
> > is also used to carry the Datagram_Size (for sequence number zero).
> 
> Reading your review I can see why. So why not adapt 4.4 as follows:
> 
> "
> 
> 4.4.  Modifying the First Fragment
> 
>    The compression of the Hop Limit, of the source and destination
>    addresses in the IPv6 Header, and of the Routing Header may change en
>    route in a Route-Over mesh LLN.  If the size of the first fragment is
>    modified, then the intermediate node MUST adapt the Datagram_Size,
>    encoded in the Fragment_Size field, to reflect that difference.
> 
>    The intermediate node MUST also save the difference of Datagram_Size
>    of the first fragment in the VRB and add it to the Fragment_Offset of
>    all the subsequent fragments that it forwards for that datagram.
> 
> "

Thanks.  (I know the diagram is a bit tight on space, so I can understand
not wanting to change it.)

> > 
> >    The format of the fragment header is shown in Figure 1.  It is the
> >    same for all fragments.  The format has a length and an offset, as
> > 
> > (Similarly, this could mention the slight exception to "all fragments".)
> 
> Shuffing a bit the text and adding the end of this sentence:
> "
>    The format of the fragment header is shown in Figure 1.  It is the
>    same for all fragments though the Fragment_Offset is overloaded.
> 
> "
> 
> 
> >    order in which the fragments are received.  This enables out-of-
> >    sequence subfragmenting, e.g., a fragment seq. 5 that is retried end-
> >    to-end as smaller fragments seq. 5, 13 and 14 due to a change of MTU
> >    along the path between the 6LoWPAN endpoints.
> > 
> > Per the Discuss, reusing a sequence number to mean a different fragment
> > (here, a smaller one) seems pretty risky.  I recall that one of the big advantages
> > of QUIC over TCP is that we don't reuse sequence numbers for retransmissions
> > in QUIC, to get better clarity about what exactly is lost vs. ack'd; reusing a
> > sequence number to mean something with a different *size* seems
> > qualitatively worse (riskier) to me.
> 
> Yes, that was changed for the discuss and published already.
> 
> 
> 
> 
> > 
> >    There is no requirement on the receiver to check for contiguity of
> >    the received fragments.  The sender knows that the datagram is fully
> >    received when the acknowledged fragments cover the whole datagram.
> > 
> > But what if there's ambiguity about which version of a fragment (identified by
> > sequence number) is ack'd?
> 
> There's no more with the change you suggested, since there is no reuse.
> There was none either id the reuse was for a smaller fragment but that's a thing of the past.
> 
> 
> > Also, I'm confused by "no requirement" to check for contiguity; presumably the
> > receiver also is going to check that it has all the data in the datagram
> > somehow...
> 
> 
> Yes, I meant that the sequence in the fragment matches a perfect tiling of the compressed packet
> 
> What about
> 
> "
> 
>    There is no requirement on the reassembling endpoint to check that
>    the received fragments are consecutive and non-overlapping.
> 
> "

That seems to cover it, though of course there are the usual potential
security considerations if it doesn't check, which IIRC are in the
companion document now.

> 
> 
> 
> >    The first fragment is recognized by a Sequence of 0; it carries its
> >    Fragment_Size and the Datagram_Size of the compressed packet before
> >    it is fragmented, whereas the other fragments carry their
> >    Fragment_Size and Fragment_Offset.  The last fragment for a datagram
> >    is recognized when its Fragment_Offset and its Fragment_Size add up
> >    to the Datagram_Size.
> > 
> > I suggest "add up to the stored Datagram_Size of the packet identified by the
> > sender and Datagram_Tag".
> 
> Done
> 
> "
> 
>    The first fragment is recognized by a Sequence of 0; it carries its
>    Fragment_Size and the Datagram_Size of the compressed packet before
>    it is fragmented, whereas the other fragments carry their
>    Fragment_Size and Fragment_Offset.  The last fragment for a datagram
>    is recognized when its Fragment_Offset and its Fragment_Size add up
>    to the stored Datagram_Size of the packet identified by the sender
>    Link-Layer address and the Datagram_Tag.
> 
> "
> 
> 
> > 
> >    E:  1 bit; Explicit Congestion Notification; the "E" flag is reset by
> >       the source of the fragment and set by intermediate routers to
> >       signal that this fragment experienced congestion along its path.
> > 
> > nit: I suggest s/reset/cleared/.
> 
> done
> 
> 
> > 
> >    Datagram_Tag:  8 bits; an identifier of the datagram that is locally
> >       unique to the sender.
> > 
> > RFC 4944 had a 16-bit tag field; I would expect some discussion of why the
> > shorter tag is acceptable now.  I guess that's in the security considerations,
> > which is okay (albeit not where I would have first looked).
> 
> Text was added with Eric' s review deeper in the spec, and I see you found it.
> 
> > 
> >    Sequence:  5-bit unsigned integer; the sequence number of the
> >       fragment in the acknowledgement bitmap.  Fragments are numbered
> >       [0..N] where N is in [0..31].  A Sequence of 0 indicates the first
> > 
> > This implies that a one-fragment packet is allowed (N of 0).  Wouldn't you just
> > not use the fragmentation framing for the case where the packet fits in a single
> > frame?
> 
> Each bit counts when the MTU is 80 to 100 octets.
> You'd do that if you want an ack from the edge of the 6LoPWAN network.
> But a full packet that fails on path A can be be rerouted, so the L2 ack usually suffices.

Ah, fair enough.

> 
> > 
> >       *  if a VRB already exists and is not broken, the fragment is to
> >          be forwarded along the associated Label Switched Path (LSP) as
> >          described in Section 6.1.2, but regardless of the value of the
> >          Sequence field;
> > 
> > I'm not sure I'm parsing "but regardless of the value of the Sequence field"
> > properly; if the intent is to say that it is forwarded without checking the value
> > of the Sequence field, I would remove the "but".
> 
> picked
> 
> > Also, we seem to be intending "is not broken" to match up with "the next hop
> > is still reachable" in the preceding paragraph; I'd suggest harmonizing the
> > wording between the two instances.
> 
> done
> 
> > 
> >       *  else, if the Sequence is 0, then the fragment is to be routed
> >          as described in Section 6.1.1, but no state is conserved
> >          afterwards.  In that case, the session if it exists is aborted
> >          and the packet is also forwarded in an attempt to clean up the
> >          next hops along the path indicated by the IPv6 header (possibly
> >          including a routing header).
> > 
> > Do we want an "*  else (the Sequence is nonzero and either no VRB exists or
> > the next hop is unavailable), the fragment is discarded and an abort RFRAG-
> > ACK is sent back [...]" to replace the last paragraph of the section?
> 
> Much nicer, thanks!
> 
> "
> 
>       *  else (the Sequence is nonzero and either no VRB exists or the
>          next hop is unavailable), the fragment cannot be forwarded or
>          routed; the fragment is discarded and an abort RFRAG-ACK is
>          sent back to the source as described in Section 6.1.2.
> "
> 
> 
>  
> > Section 5.2
> > 
> >    This specification also defines a 4-octet RFRAG Acknowledgment bitmap
> >    that is used by the reassembling endpoint to confirm selectively the
> >    reception of individual fragments.  A given offset in the bitmap maps
> >    one-to-one with a given sequence number and indicates which fragment
> >    is acknowledged as follows:
> > 
> > [Except it *doesn't* indicate that, since the sequence number is not a unique
> > fragment identifier if the fragment size can change!]
> 
> Point taken and applied : )
> 
> 
> 
> > 
> >         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> >                                        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >                                        |1 1 1 0 1 0 1|E|  Datagram_Tag |
> >        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >        |          RFRAG Acknowledgment Bitmap (32 bits)                |
> >        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > 
> > I think we need to explicitly say here that the Datagram_Tag, in a reversal of
> > the previous usage, is scoped to the link-layer *recipient* of the RFRAG_ACK
> > frame.
> 
> It is indeed. Added:
> "
>    Datagram_Tag:  8 bits; an identifier of the datagram that is locally
>       unique to the Link-Layer recipient.
> 
> "

Hmm, the -16 is showing this still as "sender", not "recipient" (maybe
conflicting edits in intermediate versions?).

> > 
> > Section 6
> > 
> >    that was the originator of the fragments.  To achieve this, each hop
> >    that performed an MPLS-like operation on fragments reverses that
> >    operation for the RFRAG_ACK by sending a frame from the next hop to
> >    the previous hop as known by its MAC address in the VRB.  The
> > 
> > I'm not 100% sure that the previous work in this space (e.g.,
> > draft-ietf-6lo-minimal-fragment) framed things such that there was a
> > requirement to be able to perform this "reverse lookup".  It might be worth
> 
> Agreed
> 
> > calling this out as an additional requirement over the prevoius fragment-
> > forwarding methodologies...
> > (We do seem to do so in Section 6.1.1, though I am not sure if it merits an
> > earlier mention in the document.)
> 
> 
> Modifying 2.2:
> 
> "
> 
>    Quoting the "Multiprotocol Label Switching (MPLS) Architecture"
>    [RFC3031]: with MPLS, 'packets are "labeled" before they are
>    forwarded' along a Label Switched Path (LSP).  At subsequent hops,
>    there is no further analysis of the packet's network layer header.
>    Rather, the label is used as an index into a table which specifies
>    the next hop, and a new label".  [FRAG-FWD] leverages MPLS to forward
>    fragments that actually do not have a network layer header, since the
>    fragmentation occurs below IP, and this specification makes it
>    reversible so the reverse path can be followed as well. 
> 
> "

Nicely done; thanks!  (I would have ended up using many more words.)

> > 
> >    limiting the number of outstanding fragments, which are the fragments
> >    that have been sent but for which reception or loss was not
> >    positively confirmed by the reassembling endpoint.  The maximum
> > 
> > (Do we have a way for *loss* to be "positively confirmed"?  Appendix C notes
> > that we infer loss from timeout without acknowledgment.)
>  
> If a particular network guarantees the order of frames then yes. But we do not mandate that.
> Otherwise, time out, though it is not really positive. Not sure what text to change though.

I'm not really sure what text to change, either, unfortunately.
The only thing that really comes to mind would be to drop "positively", but
that might have drawbacks of its own.

> > 
> >    The Ack-Request ('X') set in an RFRAG marks the end of a window.
> >    This flag MUST be set on the last fragment if the sender wishes to
> >    protect the datagram, and it MAY be set in any intermediate fragment
> >    for the purpose of flow control.
> > 
> > Is this usage of "protect" conventional or should it get clarified somewhere?
> 
> Clarified as
> 
> "
> 
>    The Ack-Request ('X') set in an RFRAG marks the end of a window.
>    This flag MUST be set on the last fragment if the fragmenting
>    endpoint wishes to perform an automatic repeat request (ARQ) process
>    for the datagram, and it MAY be set in any intermediate fragment for
>    the purpose of flow control.
> 
> "

Ah, thanks.

>  
> >    This automatic repeat request (ARQ) process MUST be protected by a
> >    Retransmission Time Out (RTO) timer, and the fragment that carries
> > 
> > nit: I'm not sure that "this" is appropriate yet, since we haven't really talked
> > about "repeat"ing or retransmission yet.  (Perhaps also give a bit more
> > discussion about what this ARQ process specifically entails, too.)
> 
> With the above it's now done.
> 
> > 
> >    of times (see Section 7.1).  Upon exhaustion of the retries the
> >    sender may either abort the transmission of the datagram or retry the
> >    datagram from the first fragment with an 'X' flag set in order to
> >    reestablish a path and discover which fragments were received over
> >    the old path in the acknowledgment bitmap.  When the sender of the
> > 
> > I'm not sure I'm understanding properly what's meant by "retry the datagram
> > from the first fragment with an 'X' flag set".  My best guess is that it's saying
> > "re-send the fragment with sequence number 0, and set the 'X' flag in this new
> > transmission", in which case I'd suggest s/from the first fragment with an 'X'
> > flag set/from the first fragment, setting the 'X' flag in the retransmitted
> > fragment,/.
> 
> Not from frag 0; the idea is to resent Frag 0 with ack request. That forms a new path, and enables the other end point to ack what it has received. From there, the remainder can be set over the new path.
> 
> Per a prior review it was reworded to
> "
> Upon exhaustion of the retries the
>    fragmenting endpoint may either abort the transmission of the
>    datagram or resend the first fragment with an 'X' flag set in order
>    to establish a new path for the datagram and obtain the list of
>    fragments that were received over the old path in the acknowledgment
>    bitmap.
> 
> "

This helps me a lot!

> > 
> >    The receiver MAY issue unsolicited acknowledgments.  An unsolicited
> >    acknowledgment signals to the sender endpoint that it can resume
> >    sending if it had reached its maximum number of outstanding
> >    fragments.  Another use is to inform the sender that the reassembling
> > 
> > How would the sender get into a situation where it has saturated its transmit
> > window but not requested an ACK?  Is this just the case where the fragment
> > requesting an ACK gets lost and thus we have to rely on some timeout
> > (whether at the sender or the receiver) to recover?  It seems like we may need
> > to define a symbolic constant for the "unsolicited ACK" timer after no
> > fragments are received, and give some guidance on how to determine it.
> 
> "if" was meant to say " in case ", replacing. This can happen. E.g., the window may be smaller than the capacity of the network (related to the nb of hops). 
> 
>  (even more after Mirja's review) I do not want to dig into flow control and this is borderline. I just wanted to leave the door open for future mechanisms that are out of scope and that would do it.
> 

Okay, I will not insist on it.

> > 
> >    The RFRAG Acknowledgment can optionally carry an ECN indication for
> >    flow control (see Appendix C).  The receiver of a fragment with the
> >    'E' (ECN) flag set MUST echo that information by setting the 'E'
> >    (ECN) flag in the next RFRAG Acknowledgment.
> > 
> > I'm not sure I'd use the word "optionally" here -- IIUC the idea is to make it
> > mandatory to implement the ECN behavior, and the ECN indication is only
> > "optional" in that it is sometimes set, but when it is set and when it is not set
> > are mandatory parts of the protocol, not subject to implementation discretion.
> > If that's correct, then I'd just say "The RFRAG Acknowledgment carries and ECN
> > indication for flow control".
> 
> Done, a slight update from what we did for 14
> "
> 
>    The RFRAG Acknowledgment carries an ECN indication for flow control
>    (see Appendix C).  The reassembling endpoint of a fragment with the
>    'E' (ECN) flag set MUST echo that information by setting the 'E'
>    (ECN) flag in the next RFRAG Acknowledgment
> 
> "
> 
> >    round-trip delay in the network.  As the timer runs, the receiving
> >    endpoint absorbs the fragments that were still in flight for that
> >    datagram without creating a new state.  The receiving endpoint aborts
> >    the communication if it keeps going on beyond the duration of the
> >    timer.
> > 
> > (This "absorbs" the behavior is to catch retransmissions, right?  Might be worth
> > saying so, though also might not.) I'd suggest expanding "if it keeps going",
> > perhaps as "if fragments with matching source and tag continue to be received
> > after the timer expires".
> 
> "
>    When enough fragments are received to cover the whole datagram, the
>    reassembling endpoint reconstructs the packet, passes it to the upper
>    layer, sends an RFRAG Acknowledgment on the reverse path with a FULL
>    bitmap, and arms a short timer, e.g., on the order of an average
>    round-trip delay in the network.  The FULL bitmap is used as opposed
>    to a bitmap that acknowledges only the received fragments to let the
>    intermediate nodes know that the datagram is fully received.  As the
>    timer runs, the reassembling endpoint absorbs the fragments that were
>    still in flight for that datagram without creating a new state,
>    acknowledging the ones that that bear an Ack-Request with an FRAG
>    Acknowledgment and the FULL bitmap.  The reassembling endpoint aborts
>    the communication if fragments with matching source and Datagram-Tag
>    continue to be received after the timer expires.
> 
> 
> 
> "
> 
> 
> 
> > 
> >    Fragments are sent in a round-robin fashion: the sender sends all the
> >    fragments for a first time before it retries any lost fragment; lost
> >    fragments are retried in sequence, oldest first.  This mechanism
> >    enables the receiver to acknowledge fragments that were delayed in
> >    the network before they are retried.
> > 
> > Is this round-robin within a given "window" or within the entire packet?
> 
> Entire packet is more efficient and intended here;
> 
> "
> 
>    Fragments are sent in a round-robin fashion: the fragmenting endpoint
>    sends all the fragments of the datagram for a first time before it
>    retries any lost fragment; lost fragments are retried in sequence,
>    oldest first through the whole datagram.  This mechanism enables the
>    reassembling endpoint to acknowledge fragments that were delayed in
>    the network before they are retried.
> 
> "
> 
> > 
> >    When a single frequency is used by contiguous hops, the sender should
> > 
> > nit: "radio frequency"?  Retransmissions have a frequency, too (though several
> > orders of magnitude different!).
> 
> Radio : ) added.
> 
> 
> > 
> > Section 6.1
> > 
> >    label, which is swapped in each hop.  All fragments follow the same
> >    path and fragments are delivered in the order in which they are sent.
> > 
> > Where do we make a normative requirement for intermediate nodes to retain
> > order in transmitting fragments?
> 
> we do not but I'm not even sure that this sentence meant that. Removed.
> Changed the beginning as:
> 
> 
> "
>    This specification extends the Virtual Reassembly Buffer (VRB)
>    technique to forward fragments with no intermediate reconstruction of
>    the entire packet.  It inherits operations like Datagram_Tag
>    switching and using a timer to clean the VRB once the traffic ceases.
>    The first fragment carries the IP header and creates a path from the
>    fragmenting endpoint to the reassembling endpoint that all the other
>    fragments follow.
> 
> "

Sounds good ("creating the path" is important; "in-order" less so).

> > 
> > Section 6.1.1
> > 
> >    In Route-Over mode, the source and destination MAC addresses in a
> > 
> > It feels surprising to mention Route-Over mode and not have a corresponding
> > mention of Mesh-Under.
> 
> In mesh under, fragments are always forwarded since they are inside a frame and the frame is what gets forwarded. This draft serves a route-over-only problem. Recently we have changed the abstract to:
> 
> "
>    This draft updates RFC 4944 with a simple protocol to recover
>    individual fragments across a route-over mesh network, with a minimal
>    flow control to protect the network against bloat.
> 
> "

Well I feel silly for missing that and blindly applying the comparison
between route-over and mesh-under.  Thanks for the update.

> 
> > 
> >    (and unique) for that source MAC address.  Upon receiving the first
> >    fragment (i.e., with a Sequence of zero), an intermediate router
> >    creates a VRB and the associated LSP state for the tuple (source MAC
> >    address, Datagram_Tag) and the fragment is forwarded along the IPv6
> >    route that matches the destination IPv6 address in the IPv6 header as
> >    prescribed by [I-D.ietf-6lo-minimal-fragment], where the receiving
> >    endpoint allocates a reassembly buffer.
> > 
> > nit: please check the comma usage here (the last one looks misplaced, to my
> > eye).
>  Removed the comma, changed the last words, which were misleading since the reassemble may not be the IPv6 destination.
> "
>    Upon receiving the first fragment (i.e., with a Sequence of 0), an
>    intermediate router creates a VRB and the associated LSP state
>    indexed by the incoming interface, the previous-hop Link-Layer
>    address, and the Datagram_Tag, and forwards the fragment along the
>    IPv6 route that matches the destination IPv6 address in the IPv6
>    header until it reaches the reassembling endpoint, as prescribed by
>    [FRAG-FWD].  The LSP state enables to match the next incoming
>    fragments of a datagram to the abstract forwarding information (next
>    interface, source and next-hop Link-Layer addresses, swapped
>    Datagram_Tag). 
> "

That should work well (though I expect the RFC Editor to tweak the wording
further).

> >    Datagram_Tag.  This reverse LSP state also points at the VRB and
> >    enables matching the (next MAC address, swapped_Datagram_Tag) found
> >    in an RFRAG Acknowledgment to the tuple (previous MAC address,
> > 
> > nit(?): doesn't this "reverse lookup" only depend on the
> > swapped_Datagram_Tag?  I don't think "next MAC address" is needed.
> 
> The trick is that the node may have more than one interface and more than one MAC address on that interface (think privacy MAC). The whole text is written with the assumption of a single interface because that's usually the case in 6LoWPAN but at least there could be more than one MAC sued as source by this node in parallel for difference datagrams. A node must remember which MAC addresses it uses for a datagram and that's part of the key used as index for the rfrag ack. I revisited the text a bit.
> 
> "
>   In addition, the router also forms a reverse LSP state indexed by the
>    interface to the next hop, the Link-Layer address the router uses as
>    source for that datagram, and the swapped Datagram_Tag.  This reverse
>    LSP state enables matching the (next interface, destination Link-
>    Layer address, swapped_Datagram_Tag) found in an RFRAG Acknowledgment
>    to the abstract forwarding information (previous interface, previous
>    Link-Layer address, Datagram_Tag) used to forward the Fragment
>    Acknowledgment (RFRAG-ACK) back to the fragmenting endpoint.
> 
> "

Ah, that makes sense.  I think I had envisoned the datagram_tag as being
global to the node, across all interfaces and MAC addresses, but there's
not a reason to do so (and in fact reason to not do so, as you discuss
somewhere about how to cope with the limited 8-bit space).

> 
> > 
> >    The first fragment may be received a second time, indicating that it
> >    did not reach the destination and was retried.  In that case, it
> >    SHOULD follow the same path as the first occurrence.  It is up to
> >    sending endpoint to determine whether to abort a transmission and
> >    then retry it from scratch, which may build an entirely new path.
> > 
> > This seems to be alluding to the text in Section 6 about "or retry the datagram
> > from the first fragment with an 'X' flag set in order to reestablish a path",
> > however, my reading of that text is that the same datagram tag value is used in
> > the retry, in which case the retransmission of the first fragment seems like it
> > would be indistinguishable from any other retransmission, to the intermediate
> > node.  If a new tag was to be allocated, then we would not be able to get
> > information about fragments received over the old path as Section 6 discusses.
> > So it seems like this recommendation here ("use the same
> > path") is at odds with the implied recommendation in the earlier text ("use a
> > potentially different path"), and it would be worth some discussion to resolve
> > the apparent disparity.
> > 
> 
> It is indeed. Let me remove the whole paragraph.
> 
> 
> 
> > Section 6.1.2
> > 
> >    [I-D.ietf-6lo-minimal-fragment] indicates that the receiving endpoint
> >    stores "the actual packet data from the fragments received so far, in
> >    a form that makes it possible to detect when the whole packet has
> >    been received and can be processed or forwarded".  How this is
> >    computed is implementation specific but relies on receiving all the
> >    bytes up to the Datagram_Size indicated in the first fragment.  An
> >    implementation may receive overlapping fragments as the result of
> >    retries after an MTU change.
> > 
> > This text leaves me feeling unsatisfied, for a couple reasons:
> > (1) I have to think to infer that "receiving endpoint" means "the intermediate
> > router receiving the fragments" and not the ultimate (IPv6) destination
> 
> There was wrong stuff that was cleaned with your review above. In particular:
> 
> - The term "receiving endpoint" is gone. I'm using "reassembling endpoint" throughput and the endpoints are the 6FF endpoints not the 6LoWPAN compression or the IPv6 endpoints. 
> - That guy may not be the node that undoes the 6loWPAN compression but a node in between as you noticed. This makes sense with RFC 8138 that forwards in the compressed form.
> - The final IPv6 destination might be even farther, far, far away in the internet.

Sounds good.

> 
> 
> > (2) overlapping fragments that contain different data can be a security risk,
> > and we don't discuss that here (whether directly or by reference).
> 
> We do now. Minimal frag section 7:
> 
> "
>    *  Overlapping fragment attacks are possible with 6LoWPAN fragments
>       but there is no known firewall operation that would work on
>       6LoWPAN fragments at the time of this writing, so the exposure is
>       limited.  An implementation of a firewall SHOULD NOT forward
>       fragments but instead should recompose the IP packet, check it in
>       the uncompressed form, and then forward it again as fragments if
>       necessary.  Overlapping fragments are acceptable as long as they
>       contain the same payload.  The firewall MUST drop the whole packet
>       if overlapping fragments are encountered that result in different
>       data at the same offset.
> 
> "
> And this spec has
> 
> "
> An implementation
>    may receive overlapping fragments as the result of retries after an
>    MTU change.
> "
> and
> "
> There is no requirement on the reassembling endpoint to check that
>    the received fragments are consecutive and non-overlapping.
> "
> We do not mandate checking because it is more code and the attack is difficult "
>                   since the
>       fragmentation happens within the LLN, meaning that the intruder
>       should already be inside to perform the attack.
> "

Yup, given the MAC-layer protection an attacker that can do this can do
more interesting (read: harmful) things.

> 
> 
> > Section 6.2
> > 
> >    Upon receipt of an RFRAG-ACK, the router looks up a reverse LSP
> >    indexed by the tuple (MAC address, Datagram_Tag), which are
> > 
> > (As above, is the MAC address needed here?)
> 
> You'd be correct yes.
> 
> Still we want a MAC address but the destination MAC of the RFRAG ACK not the source, and that's in case this node uses more than one MAC e.g., to reuse the tag address space.
> Also we need to introduce the interface. This is a change that spans to minimal-fragments in fact, I'll publish a new one as well.
> 
> Rewording that piece to
> "
> 
>    Upon receipt of an RFRAG-ACK, the router looks up a reverse LSP
>    indexed by the interface and destination Link-Layer address of the
>    received frame and the received Datagram_Tag in the RFRAG-ACK.  If it
>    is found, the router forwards the fragment using the associated VRB
>    as prescribed by [FRAG-FWD], but using the reverse LSP so that the
>    RFRAG-ACK flows back to the fragmenting endpoint.
> 
> "
> 
> > 
> >    Either way, if the RFRAG-ACK indicates that the fragment was entirely
> >    received (FULL bitmap), it arms a short timer, and upon timeout, the
> >    VRB and all the associated state are destroyed.  Until the timer
> > 
> > Is there anything that could cancel this timer (e.g., an RFRAG-ACK that with
> > non-FULL bitmap)?
> 
> I cannot see that? Logically that one would have been sent before and received after. 
> Once the FULL is achieved the fragmenting node knows that the reassembling endpoint is done.

I couldn't see anything either, but it resembles an edge case that we like
to double-check.  So, thanks for double-checking :)

> 
> > 
> >    This specification does not provide a method to discover the number
> >    of hops or the minimal value of MTU along those hops.  But should the
> >    minimal MTU decrease, it is possible to retry a long fragment (say
> >    Sequence of 5) with first a shorter fragment of the same Sequence (5
> >    again) and then one or more other fragments with a Sequence that was
> >    not used before (e.g., 13 and 14).  Note that Path MTU Discovery is
> >    out of scope for this document.
> > 
> > [another mention of sequence number reuse]
> > 
> 
> The text is now (since -14)
> "
> But
>    should the minimal MTU along the path decrease, it is possible to
>    retry a long fragment (say Sequence of 5) with several shorter
>    fragments with a Sequence that was not used before (e.g., 13 and 14).
>    Fragment 5 is marked as abandoned and will not be retried anymore.
> "
> 
> 
> > Section 6.3
> > 
> >    A reset is signaled on the forward path with a pseudo fragment that
> >    has the Fragment_Offset, Sequence, and Fragment_Size all set to 0,
> >    and no data.
> > 
> > The text in Section 5.1 indicates that an abort is characterized by
> > Fragment_Offset of zero.  I don't think that it has a requirement to set the
> > other fields to zero in order to be considered an abort.
> 
> True:
> 
> "
> 
>    A reset is signaled on the forward path with a pseudo fragment that
>    has the Fragment_Offset set to 0.  The sender of a reset SHOULD also
>    set the Sequence and Fragment_Size field to 0.
> 
> "
> 
> 
> 
> >    When the sender or a router on the way decides that a packet should
> > 
> > nit: I don't think we've used "on the way" previously in this sense (and do use
> > "on the path").  Though, it looks like we do use "on the way"
> > again a few paragraphs later.
> 
> on the path looks better
> 
> > 
> >    The other way around, the receiver might need to abort the process of
> >    a fragmented packet for internal reasons, for instance if it is out
> >    of reassembly buffers, already uses all 256 possible values of the
> >    Datagram_Tag, or if it keeps receiving fragments beyond a reasonable
> >    time while it considers that this packet is already fully reassembled
> >    and was passed to the upper layer.  In that case, the receiver SHOULD
> > 
> > nit: s/process/processing/?
> 
> done
> 
> > 
> > Section 7
> > 
> > We can probably inherit discussion of integrity and confidentiality (non-
> > )protection from -minimal-fragment, assuming my recommended changes
> > there are made.  It remains the case, though, that intermediate nodes can
> > always modify the payload and disrupt/deny service, and I, for one, would not
> > turn down another reminder of that.
> 
> Done some in 14
> 
> "
> 
>   This document specifies an instantiation of a 6FF technique and
>    inherits from the generic description in [FRAG-FWD].  The
>    considerations in the Security Section of [FRAG-FWD] equally apply to
>    this document.
> 
>    In addition to the threats detailed therein, an attacker that is on-
>    path can prematurely end the transmission of a datagram by sending a
>    RFRAG Acknowledgment to the fragmenting endpoint.  It can also cause
>    extra transmissions of fragments by resetting bits in the RFRAG
>    Acknowledgment bitmap, and of RFRAG Acknowledgments by forcing the
>    Ack-Request bit in fragments that it forwards.  As indicated in
>    [FRAG-FWD], Secure joining and the Link-Layer security are REQUIRED
>    to protect against those attacks.
> "
> 
> Do we want more?

That looks good.  The only thing I'd consider adding is another clause at
the very end, ", as the fragmentation protocol does not include any native
security mechanisms".  But it's up to you.

> > 
> >    This specification extends "On Forwarding 6LoWPAN Fragments over a
> >    Multihop IPv6 Network" [I-D.ietf-6lo-minimal-fragment] and requires
> >    the same parameters in the receiver and on intermediate nodes.  There
> >    is no new parameter as echoing ECN is always on.  These parameters
> >    typically include the reassembly time-out at the receiver and an
> >    inactivity clean-up timer on the intermediate nodes, and the number
> >    of messages that can be processed in parallel in all nodes.
> > 
> > Do we have a new parameter for the "timer for a receiver node to send an
> > unsolicited ack if no additional fragments are received"?
> > 
> 
> I'd rather not if that's OK. Too close to the flow control that we stay away from.

Okay.  (I feel like we had this conversation already; sorry for duplicating
it.)

> 
> >    The configuration settings introduced by this specification only
> >    apply to the sender, which is in full control of the transmission.
> > 
> > (Similarly, this is "full control" modulo acks.)
> 
> The acks are not commands, they are information : ) The  fragmenting endpoint (sender) is the boss.

I think I was considering that the node that's supposed to send the ack
could decline to do so for some reason, or send spontaneous acks.  But "no
contest" here.

> 
> > 
> > Section 7.1
> > 
> >    OptWindowSize:  The OptWindowSize is the value for the Window_Size
> >       that the sender should use to start with.  It is greater than or
> >       equal to MinWindowSize.  It is less than or equal to
> >       MaxWindowSize.  The Window_Size should be maintained below the
> >       number of hops in the path of the fragment to avoid stacking
> >       fragments at the bottleneck on the path.  If an inter-frame gap is
> >       used to avoid interference between fragments then the Window_Size
> >       should be at most on the order of the estimation of the trip time
> >       divided by the inter-frame gap.
> > 
> > Let's do some dimensional analysis here: the window size is measured in
> > number of frames, the trip time is measured in time, and the inter-frame gap is
> > discussed above as an "amount of time" (though whether this is an absolute
> > time value or a count of transmit windows is perhaps subject to
> > interpretation).  The ratio of trip time deivided by inter-frame gap, then, is
> > dimensionless, which we then try to interpret as a number of frames.  That's
> > probably okay, though mathematics pedants might want some additional
> > clarifying discussion.
> 
> Me who thought that the physics exams were behind!

:)
Must be my chemistry degree at work...

> We transmit at most 1 frame per interframe gap. So the number of interframe gaps per RTT gives us the max number of frames that can be sent by RTT, that is till an ack comes in. So we fill the network with the number given, any more than that is a potential addition to the bloat but to no added throughput.
> 
> The typo in divided was fixed too...
> 
> > 
> >    An implementation may be capable of performing flow control based on
> >    ECN; see in Appendix C.  This is controlled by the following
> >    parameter:
> > 
> >    UseECN:  Indicates whether the sender should react to ECN.  The
> >       sender may react to ECN by varying the Window_Size between
> >       MinWindowSize and MaxWindowSize, varying the Fragment_Size between
> >       MinFragmentSize and MaxFragmentSize, and/or by increasing the
> >       inter-frame gap.
> > 
> > Please forgive the naivite, but shouldn't this parameter merely control
> > *how* an implementation responds to ECN markings, not *whether or not it
> > should respond at all* (since obviously it should respond in some fashion)?
> 
> The spec provides either do nothing or reset the window to one and live with it till the end of the datagram. The goal is that other specs provide a better flow control.
> 
> "
> 
>    UseECN:  Indicates whether the fragmenting endpoint should react to
>       ECN.  The fragmenting endpoint may react to ECN by varying the
>       Window_Size between MinWindowSize and MaxWindowSize, varying the
>       Fragment_Size between MinFragmentSize and MaxFragmentSize, and/or
>       by increasing or reducing the inter-frame gap.  With this
>       specification, if UseECN is set and a fragmenting endpoint detects
>       a congestion, it resets the Window_Size to 1 till the end of the
>       datagram, whereas if UseECN is reset, the endpoint does not react
>       to congestion.  Future specifications may provide additional
>       parameters and capabilities.
> 
> "
> 
> > 
> > Section 11
> > 
> > I'm not seeing why RFC 6554 (as referenced in one location only) is a normative
> > reference.
> 
> fixed
> 
> > 
> > Section 12
> > 
> > Given that "Readers are expected to be familiar with all the terms and
> > concepts that are discussed in "IPv6 over Low-Power Wireless Personal Area
> > Networks (6LoWPANs): Overview, Assumptions, Problem Statement, and
> > Goals" [RFC4919]", RFC 4919 seems like it is more properly categorized as
> > Normative.  Similarly for RFC 6606.
> > 
> 
> OK, will be the usual downref
> 
> > I don't have strong feelings about RFC 8200's status, though I suppose one
> > could argue that one needs to parse the network header in order to route the
> > initial fragment.
> 
> Sure, moved 8200 to normative
> 
> > 
> > Appendix A
> > 
> >    Considering that RFC 4944 defines an MTU is 1280 bytes and that in
> >    most incarnations (but 802.15.4g) a IEEE Std. 802.15.4 frame can
> >    limit the MAC payload to as few as 74 bytes, a packet might be
> > 
> > I'm not sure I'm parsing this properly, but I think "except" is more approriate
> > than "but" in the parenthetical.
> > 
> 
> Applied
> 
> > Appendix C
> > 
> >    From the standpoint of a source 6LoWPAN endpoint, an outstanding
> >    fragment is a fragment that was sent but for which no explicit
> >    acknowledgment was received yet.  This means that the fragment might
> > 
> > Is there also a timer that has to have not expired yet?
> > 
> 
> Added " and are still covered by the ARQ timer "
> 
> Whao, that was a lot!
> 
> Again and again, so many thanks!

And thanks again for the updates; they are always very thoughtfully done.

I will go reballot now; I had a few more nits that I saw while reading the
diff.

-Ben