Re: [6lo] Benjamin Kaduk's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)

[Benjamin Kaduk <[e-mail]>]

Hi Pascal,

On Thu, Mar 05, 2020 at 06:13:38PM +0000, Pascal Thubert (pthubert) wrote:
> Hello again Benjamin
> 
> You really made those drafts I'm editing a lot better by your keen reviews and this is another incredible one. Many thanks!!!
> 
> Let's go with the Discuss to start with?

I guess we could have taken them all at once, my schedule was so hectic...

> > ----------------------------------------------------------------------
> > DISCUSS:
> > ----------------------------------------------------------------------
> > 
> > In Section 2.3 we refer to the datagram_tag plus layer-2 sender address as
> > being "a globally unique identifier for the datagram", but I think this can only
> > hold within some time-bounded window (e.g., the lifetime of the packet), since
> > the tag space is finite and reuse somewhat inevitable.  [The simplest way to
> > resolve this is probably to just remove the definition from this document and
> > refer to draft-ietf-6lo-minimal-fragment for definitions.]
> 
> Rightly so. I changed the text on the minimal draft as 
> "
> 
>    "LLN Minimal Fragment Forwarding" [I-D.ietf-6lo-minimal-fragment]
>    introduces the generic concept of a Virtual Reassembly Buffer (VRB)
>    and specifies behaviours and caveats that are common to a large
>    family of FF techniques including this, which fully inherits from
>    that specification.  It also defines terms used in this document:
>    6LoWPAN endpoints, Compressed Form, Datagram_Tag, Datagram_Size, and
>    Fragment_Offset.
> 
> "

Thanks!

> > 
> > I think we should be more clear about whether a "FULL bitmap" always has
> > 32 bits set to one, or if "merely" having as many bits as the sender sent
> > fragments set to one also counts as "FULL".  The current text seems to invite
> > different interpretations by implementations.  (If FULL does mean all 32 bits,
> > then the semantics of the other case seem unclear to
> > me.)
> 
> There's a definition and we need to stick to it:
> 
>    FULL bitmap:  Refers to a bitmap with all bits set to one.

To answer a question that was raised in a different thread, I think the
part that made me most uncertain of the intended semantics here was the
"all bits" part -- there's a tendency to extrapolate to "all bits with
defined semantic meaning", but of course as discussed in the new text you
added, which bits will have semantic meaning can change with
refragmentation.  It might be possible to specify "all 32 bits set to one"
to avoid this risk of misinterpretation, but the current formulation of the
document leaves the concept of bitmap more general when "FULL bitmap" is
defined, and allows for future expansion, so I'm inclined to say that the
current text in the -16 is good and should stay as-is.

> The problem is that we do not say (or even know in advance) how many fragments there are, total. The MTU may change.
> So even if all fragments are acknowledged with a bit set, an intermediate node cannot fathom if that's the end or if there's more to come. But the end-points know. The receiver will see that all bytes up to Datagram-Size are received so it has it all.
> The spec asks the receiver to use a FULL bitmap instead of setting only the bits corresponding to the received fragments so the intermediate nodes know that the process is complete.
> 
> To clarify I'm adding the last sentence in the text below:
> 
> "
>    When enough fragments are received to cover the whole datagram, the
>    receiving endpoint reconstructs the packet, passes it to the upper
>    layer, sends an RFRAG Acknowledgment on the reverse path with a FULL
>    bitmap, and arms a short timer, e.g., on the order of an average
>    round-trip delay in the network.  The FULL bitmap is used as opposed
>    to a bitmap that acknowledges only the received fragments to let the
>    intermediate nodes know that the datagram is fully received.
> 
> "

This is good, thanks.
I guess this means that a recipient of a "bitmap with as many bits set as
fragments have been sent but not all bits set" has to assume that
refragmentation has occurred and that the full datagram is not acknowledged
yet, which is fine.

> 
> 
> > 
> > What's the transition/backwards-compatibility story?  That is, how does a
> > sender know that all nodes on the path support the RFRAG dispatch types, and
> > what happens if they are sent anyway and get to a node that doesn't
> > implement them?
>  
> It has to be management and configuration or alliance games. There is no negotiation.
> There is no overarching 6LoWPAN protocol that can advertise capabilities though we're building one for RPL.
> Same goes for all the prior changes to 6LoWPAN, e.g., RFC 8025.
> In practice there is no 6LoWPAN network. There's an ISA100.11a network, a ZigbeeIP network, a Thread network, or a WiSUN network. Those alliances specify a particular bundle of functionalities from a bunch of RFCs and build their compliance tests on that. If Thread decides that the RFRAGs are used, then all the nodes are capable of it.
> But that's a long story to make in the RFC just to say we do nothing. Do we really want that?

I see your point, but we are also in theory supposed to consider the IAB's
guidance from RFC 8170, including among other things incremental
deployability.  So my inclination is to include a brief note that since
deployments are expected to be managed and homogeneous, incremental
transition requires a flag day.

> 
> > I have grave misgivings about allowing a packet (as identified by sender and
> > tag) to be refragmented by the sender so that a single fragment sequence
> > number is used for fragments of different lengths.  We do not seem to provide
> > a mechanism to distinguish which variant of that fragment is being ack'd,
> > which could lead to disagreement between sender and receiver as to whether a
> > full packet is reconstructed.
> 
> Yes, that makes sense. And the change is simple to do and appears harmless. Note that if the second is smaller than and included within the first, it does not matter which is received. I just did not expect that people would reuse the seq for a larger fragment in which case there may be an issue. I just hope people did not implement that piece yet.
> 
> Note that the FULL bitmap hides the lack of continuity from intermediate nodes. 
> I also wish to avoid duplication in the example so we get
> 
> "
>                                                                                             This enables
>    refragmenting to cope with an MTU deduction, see the example of the
>    fragment seq. 5 that is retried end-to-end as smaller fragments seq.
>    13 and 14 in Section 6.2."
> 
> And in Section 6.2
> 
> "
>    This specification does not provide a method to discover the number
>    of hops or the minimal value of MTU along those hops.  But should the
>    minimal MTU decrease, it is possible to retry a long fragment (say
>    Sequence of 5) with several shorter fragments with a Sequence that
>    was not used before (e.g., 13 and 14).  Fragment 5 is marked as
>    abandoned and will not be retried anymore.  Note that Path MTU
>    Discovery is out of scope for this document.
> 
> "

This sounds good, thanks.

It did occur to me while reading the diff that we do treat the fragment
with sequence 0 specially, and I didn't fully think through the
possibilities if it is that first packet that is "too long" and needs to be
subdivided.  There may be special considerations for the initial fragment
where it gets some special handling, and though I suspect that there are
not any real problems here, and in any case the datagram itself could be
re-sent, I mention it just in case there are some new problems (e.g., we
get stuck in a case where we have to send something that gets treated as a
reset even if we don't want it to).

> > Brainstorming, it might be possible to allow such refragmenting at the sender
> > by using a Fragment_Size of zero to indicate "this fragment is superseded" and
> > allocating new sequence number for all its components.
> > (I didn't attempt to do an exhaustive check on whether that proposal is flawed
> > and Fragment_Size of zero already has some existing semantics that would be
> > in conflict.)
> > 
> 
> Well the proposal is just to ignore the original fragment. The set of fragments is used to figure if everything is sent and what may need resend. It is expected that there will be overlaps and now gaps in the acknowledgements. The FULL bitmap (all ones) is the indicator that enough was received without ambiguity and regardless of which fragments were used.
> 
> Works?

Agreed that the FULL bitmap should be a clear sign that "all data is
received"

Thanks,

Ben