Re: [Roll] review of dao-projection -21

["Pascal Thubert (pthubert)" <[e-mail]>]

Hello Michael

Many thanks for your deep review! This may take a few rounds, in particular if we want more figures.
Please see the discussion below and the first set of diffs in https://github.com/roll-wg/dao-projection/commit/2e160f93408127799dc5fa8bbbb9e4d0a36ee8dc


> -----Original Message-----
 
> 
> Rahul and I (via hypothys.is) want to know why "anistropic" was part of
> the introduction sentence.  It's not wrong, but it seems funny to
> suddenly have this so prominent.

I opened a separate thread for this. What is hypothys.is ?


> 
> I have made a number of editorial suggestions, most are typos, some
> word re-ordering, which I put into: https://github.com/roll-wg/dao-
> projection/pull/10
> rather than bore you with minutia.

Already applied, much appreciated


> 
> On the whole, the document is much improved, but I think that it still
> needs some significant rework.

OK, let's do it then


> 
> You might want to consider upgrading to kramdown for your source, and
> then change your ascii art to something goat can upgrade to SVG.
>   https://github.com/blampe/goat


I'm not that fond of kramdown, that's an extra step for me, and I use it when I must. 
Goat seems orthogonal, I love it but lack time for fun to learn through the IETF procedure.
Look at https://www.ietf.org/archive/id/draft-ietf-rift-rift-13.pdf; good yes, perfect, no. Worth the effort ?

> 
> A major issue for me is how things get turned on.
> Does this document wait for capex?  probably?  I know that I'm probably
> the slow/weak link in the capex chain.

It would have largely benefitted from capex if capex was there. 
For now it will be configuration / management. 

See section 3.7 "

   The "6TiSCH Architecture" [6TiSCH-ARCHI] leverages a centralized
   model that is similar to that of "Deterministic Networking
   Architecture" [RFC8655], whereby the device resources and
   capabilities are exposed to an external controller which installs
   routing states into the network based on its own objective functions
   that reside in that external entity.
"

The management magic tells the root which node supports this and their capacity. It's all done outside RPL.

> 
> The intro paragraph:
>   > With this specification, a Path Computation Element [PCE] in an
> external
>   > controller interacts with the RPL Root to compute centrally shorter
> Peer
>   > to Peer (P2P) paths within a pre-existing RPL Main DODAG. The
> topological
>   > information that is passed to the PCE is derived from the DODAG
> that is
>   > already available at the Root in RPL Non-Storing Mode. This
> specification
>   > introduces protocol extensions that enrich the topological
> information
>   > that is available at the Root and passed to the PCE.
> 
> and I am concerned about the emphasis on PCE.   Of course, we can
> support
> offloading to a PCE.  But, we don't have to, do we?
> As far as I understand it, the PCE/DODAG-Root protocol is yet-to-be-
> standardized.
> What we are standardizing is Root->6LR communication.
> I will suggest some text after my read-through.  What I don't want is
> for someone to read the intro, say, "I don't have a PCE, so I can't use
> this"
> 

Makes sense. I need to say abstract. PCE is just the term for some logical function that will compute the paths.
What about 

"
   With this specification, an abstract routing function called a Path
   Computation Element [PCE] (e.g., located in an central controller or
   collocated with the Root) interacts with the RPL Root to compute Peer
   to Peer (P2P) paths within a pre-existing RPL Main DODAG. "

> The document says, "Root or an associated PCE " a few times, so maybe
> we could just call it the "P-DAO engine" or something like that.

I like using the standard term if that's abstract enough

> 
> _Domain Terms_
> Terminology for path.  Maybe don't "", since there is an embedded
> "path".
>             maybe just indent?
> 

Good point. I'll wrap the overall quotations with <blockquote> instead. Will look like this:

"

   Quoting section 1.1.3 of [INT-ARCHI]:

   |  At a given moment, all the IP datagrams from a particular source
   |  host to a particular destination host will typically traverse the
   |  same sequence of gateways.  We use the term "path" for this
   |  sequence.  Note that a path is uni-directional; it is not unusual
   |  to have different paths in the two directions between a given host
   |  pair.

   Section 2 of [I-D.irtf-panrg-path-properties] points to a longer,
   more modern definition of path, which begins as follows:

   |  A sequence of adjacent path elements over which a packet can be
   |  transmitted, starting and ending with a node.  A path is
   |  unidirectional.  Paths are time-dependent, i.e., the sequence of
   |  path elements over which packets are sent from one node to another
   |  may change.  A path is defined between two nodes.

   It follows that the general acceptance of a path is a linear sequence
   of nodes, as opposed to a multi-dimensional graph.  In the context of
   this document, a path is observed by following one copy of a packet
   that is injected in a Track and possibly replicated within.
"


> Maybe, instead of trying to make a definition list for this section,
> make subsections, because I think that each term here actually requires
> more than a sentence.
> 
> _subTrack_ or sub-track?
>   1) Not fond of camelcase in specifications.

This is common with the RAW architecture. I can uplift both but it's work to do and undo, so I'd like to see a consensus.
Maybe make it an item to resolve for the WGLC?



>   2) too much definition here.  Supports above suggest to make these
> sub-sections.


OK, done

> 
> Probably need to define what is East-West before we use it.

That was the definition : ) reworded as:

"
   This specification builds Tracks that are DODAGs oriented towards a
   Track Ingress, and the forward direction for packets (aka East-West)
   is from the Track Ingress to one of the possibly multiple Track
   Egress Nodes, which is also down the DODAG.
"
> 
> What is "routing stretch"?

Adding

" 
2.4.4.  Routing Stretch:

   RPL is anisotropic, meaning that it is directional, or more exactly
   polar.  RPL does not behave the same way "down" with multicast DIO
   messages that form the DODAG and "up" with unicast DAO messages that
   follow the DODAG.  This is in contrast with traditional IGPs that
   operate the same in all directions and are thus called isotropic.

   The term Routing Stretch denotes the length of a path, as compared
   with a shortest path, which can be a abstract concepts in RPL when
   the metrics are statistical and dynamic, and the concept of short
   varies with the Objective Function.

   The RPL DODAG optimizes the P2MP (from Root) and MP2P (to Root)
   paths, but the P2P (node to node) traffic has to follow the same
   DODAG.  Following the DODAG, the RPL datapath passes via a common
   parent in Storing Mode and via the Root in Non-Storing Mode.  This
   typically involves more hops and more latency than the minimum
   possible for a direct P2P path that an isotropic protocol would
   compute.  We refer to this elongated path as stretched.

"

> 
> section 3.2:
>   } Packets flow via the common parent and the routing stretch is
> reduced
>   } vs. Non-Storing, for a better P2P connectivity, while the internet
>   } connectivity is restored more slowly, time for the DV operation to
> operate
>   } hop-by-hop.
> 
> Too many thoughts in the same sentence.  Could be a word missing?
> I think that I understand Storing is better for P2P connectivity, but
> the other thought about connectivity is lost in the mix.


The other point is that a new route takes longer to propagate. What about:

"
             Packets flow via the common parent and the
   routing stretch is reduced vs. Non-Storing, for a better P2P
   connectivity.  On the other hand, a new route takes a longer time to
   propagate to the Root, time for the Distance-Vector protocol to
   operate hop-by-hop, and the Internet connectivity is restored more
   slowly upon movement.
"

> 
> section 3.3.1:
> 
>  } It results that the Root, or then some associated centralized
> computation  } engine such as a PCE, can determine the amount of
> packets that reach a  } destination in the RPL domain, and thus the
> amount of energy and bandwidth  } that is wasted for transmission,
> between itself and the destination, as well  } as the risk of
> fragmentation, any potential delays because of a paths longer  } than
> necessary (shorter paths exist that would not traverse the Root).
> 
> I tried to re-write this, putting a period after "RPL domain", but what
> was left over didn't make sense.  Please rewrite with shorter
> sentences.
> 

OK then. Note that this impacts the 3 paragraphs, what about:

"
   Based on the parent-children relationships expressed in the Non-
   Storing DAO messages, the Root possesses topological information
   about the whole network, though this information is limited to the
   structure of the DODAG for which it is the destination.  A packet
   that is generated within the domain will always reach the Root, which
   can then apply a source routing information to reach the destination
   if the destination is also in the DODAG.  Similarly, a packet coming
   from the outside of the domain for a destination that is expected to
   be in a RPL domain reaches the Root.  It results that the wireless
   bandwidth near the Root is the gating factor for all transmissions
   towards or within the domain, and that the Root is a single point of
   failure for all connectivity to nodes within its domain.

   The RPL Root must add a source routing header to all downward
   packets.  As a network grows, the size of the source routing header
   augments with the depth of the nodes.  In some use cases, a RPL
   network forms long lines along physical structures such as streets
   for lighting.  Limiting the packet size is directly beneficial to the
   energy budget, but, mostly, it reduces the chances of frame loss and
   packet fragmentation, which are highly detrimental to the LLN
   operation.  A limited amount of well-targeted routing state would
   allow the source routing operation to be loose as opposed to strict,
   and save packet size.  Because the capability to store a routing
   state in every node is limited, the decision of which route is
   installed where can only be optimized with a global knowledge of the
   system, a knowledge that the Root or an associated PCE may possess by
   means that are outside of the scope of this specification.

   Being on path for all packets in Non-Storing mode, the Root may
   determine the number of P2P packets in its RPL domain per source and
   destination, the latency incurred, and the amount of energy and
   bandwidth that is consumed to reach the self and then down, including
   a possible fragmentation when encapsulating larger packets.  Enabling
   a shorter path that would not traverse the Root for select P2P
   source/destinations may improve the latency, lower the consumption of
   constrained resources, free bandwidth at the bottleneck near the
   Root, improve the delivery ratio and reduce the latency for those P2P
   flows with a global benefit for all flows of reducing the load at the
   Root.
"



> section 3.3.2, is "INternet" capitalized like this for a reason?

Taïpo 😊

> 
> The diagram in section 3.3.2 would be a good SVG candidate.
> Have you tried "goat"?

Quite neat (attached)

> 
> "The amount of stretch depends on the Mode of Operation:"
> add reference RFC9008?

Added (I realized too late that this was in your PULL Req that I had forgotten to pull first. Gee it hurts).

> 
> section 3.4:
> "so-called Track Legs" ... I think the terminology says they are called
> Track Legs, so I think "so-called" is unneeded text.
> (ps: so now I have ZZtop in my head:
> https://www.youtube.com/watch?v=eUDcTLaWJuo )

Excellent! Done. The text can be a bit more explicit, more in the full excerpt below;

> 
> "With this specification, a Track forms DODAG that is directed towards
> a Track Ingress."
> 

> There is perhaps a word missing ("a DODAG"?), but even with the
> article, I don't understand what this means.
> 


Yes the "a" was missing. The D of DODAG is a directed, and in this case following the directed links leads to the egress.
We'd get:

"

3.4.1.  Building Tracks With RPL

   The concept of a Track was introduced in the "6TiSCH Architecture"
   [6TiSCH-ARCHI], as a collection of potential paths that leverage
   redundant forwarding solutions along the way.  This can be a DODAG or
   a more complex structure that is only partially acyclic (e.g., per
   packet).

   With this specification, a Track is shaped as a DODAG, and following
   the directed edges leads to a Track Ingress.  Storing Mode P-DAO
   messages follow the direction of the edges to set up routes for
   traffic that flows the other way, towards the Track Egress(es).  If
   there is a single Track Egress, then the Track is reversible to form
   another DODAG by reversing the direction of each edge.  A node at the
   Ingress of more than one Segment in a Track may use one or more of
   these Segments to forward a packet inside the Track.

   A RPL Track is a collection of (one or more) parallel loose source
   routed sequences of nodes ordered from Ingress to Egress, each
   forming a Track Leg.  The nodes that are directly connected,
   reachable via existing Tracks as illustrated in Section 3.5.2.3 or
   joined with strict Segments of other nodes as shown in
   Section 3.5.1.3.  The Legs are expressed in RPL Non-Storing Mode and
   require an encapsulation to add a Source Route Header, whereas the
   Segments are expressed in RPL Storing Mode.
"


> Overall, I think that I need a gentler introduction to Segments, Legs,
> Track Legs.  I think that section 3.4 needs a diagram for examples,
> such as those which were in the presentations that we had.  There are
> diagrams in 3.5, but I think that maybe section 3.4/3.5 needs to be
> reworked so that the definitions come out with the diagrams.

OK, first TODO. I'll need to propose off line ascii art and will update when we agree


> 
> The last two paragraphs of 3.4 are about how Tracks are indexed.
> I think that section should be a new section.
> The different cases in the last paragraph need to be subsubsections so
> that they can be referred to.

Done

> 
> "stitching" point... needs a definition, I think.

"
2.4.9.  Stitching:

   This specification using the term stitching to indicate that a track
   is piped to another one, meaning that traffic out of the first is
   injected in the other.
"
Works?

> 
> I didn't find table 2 (RIB setting) useful without a better diagram to
> attach
> the information to.   InstanceID 129 just pops up, and I'm not sure
> exactly why.

I picked any value, let me add text on that.


> 
> section 3.5.1.1 and 3.5.1.2 are different examples, I suggest that the
> P-DAO messages are numbered differently. (1,2,3) and (4,5,6).
> I would understand this only by getting out paper and coloured pens and
> reading each word carefully.  I think it needs to be a bit easier to
> understand.




> 
> section 3.6 at the end, *detnet* appears, and this is surprising.
> I think section 3.6 should be 4.0, or maybe before section 8.
> 

It's good that it's before current 4.0. I split 3.7 in
       3.7.1.  External Dependencies . . . . . . . . . . . . . . . .  32
       3.7.2.  Positioning vs. IETF Standards  . . . . . . . . . . .  32

And moved the paragraph with DetNet in there.
3.7.2.2.  Mapping to DetNet

   DetNet Forwarding Nodes only understand the simple 1-to-1 forwarding
   sublayer transport operation along a segment whereas the more
   sophisticated Relay nodes can also provide service sublayer functions
   such as Replication and Elimination.

   One possible mapping between DetNet and this specification is to
   signal the Relay Nodes as the hops of a Leg and the forwarding Nodes
   as the hops in a Segment that join the Relay nodes as illustrated in
   Figure 6.


> section 4.1:
> "A new P-DAO Request (PDR) message (see Section 5.1) enables a Track
> Ingress to request the Track from the __Root for which it is the Root__
> and it owns the address that serves as TrackID, as well as the
> associated namespace from which it selects the TrackID."
> 
> too long sentence, and I can't understand the __middle__ part.

And there was a typo which did not help. Reworded as

"                                                                 A new
   P-DAO Request (PDR) message (see Section 5.1) enables a Track Ingress
   to request the Track from the Root.  The resulting Track is also a
   DODAG for which the Track Ingress is the Root, the owner the address
   that serves as DODAGID and authoritative for the associated namespace
   from which the TrackID is selected.
"

> 
> I suggest section 4 identify when it Updates, whether it is Amending,

Argll you're killing me :) I still do not understand what "updates" means and now we amend?
We do not break anything in existence if that is the question. 
But "extends" is the result if reviews with Alvaro on another spec, where I used to use "updates".


> See-Also or Extending.  4.1.1. is, for instance, Extends.
> Whereas the last paragraph of 4.1 is Amends.

So Amend means provide more specific recommendations on how to use the existing?
Following your line of thought I should make that a different section, but it seems to me with this is really too picky.
I already created many sections in the definition ad in section 3. This will be the draft of one sentence per section.

> I think that section 4.1.1 should just tell us about P-DAO, not define
> it.
> I was expecting section 5 to define it.

Section 5 and 6 both provide new signaling. But 5 is new stuff in existing structures that are extended whereas 6 is new messages and/or structures.
P dao is new stuff in existing structure (one bit), that's why it is defined in section 5. I 'm not too keen changing that now but I note your vote and when/if more come I'll comply.

> Last paragraph of 4.1.1 is about TIO/RIO, and deserves it's own Amends
> section.

I do not see why that is an amendment? The TIO/RIO discussion is today's RPL as is.


> section 6.
> The PDR message came as a surprise. Probably I wasn't paying attention.

Ys, that was on the ML for a while.

> I think that the Root sends a P-DAO, and then the node in question
> replies with a PDR as a maintenance function.  A kind of 3-way
> handshake on creating the track, I think.

There's a use case when the device A knows it needs to talk to device B and asks the root to set it up.
Also, the trackId comes from a namespace that is associated with IP address of the device and for which the device is ; the flow starting at the device (and your 3-way that makes full sense but we do not have for now) allow the device to be authoritative on assigning from that name space.

> I think that the 3-way-ness of this should be in the intro.
Let's take that as a potential addition to be discussed on the ML, too big for me to yes/no just here

> 
> Figure 14: I think a better diagram is needed, rather than repeating
> the LLN diagram.

Second TODO, to be taken of line

> 
> section 6.4:
>   The SRH-6LoRH with the Via Addresses in the NSM-VIO are not needed
> and MUST be omitted.
> ->
>   The SRH-6LoRH with the Via Addresses in the NSM-VIO are not needed
> and MUST
>   be ignored by the receiver.
> 
> (I always prefer to tell the receiver what to discard, rather than tell
> the sender what to omit)

OK still sending it wastes energy and may cause a packet loss. What about
                                    
"                                     The SRH-6LoRH with the Via Addresses in
    the NSM-VIO are not needed; it SHOULD not be placed in the message and MUST
    be ignored by the receiver.
"

> 
> 6.5.2:
>         "It makes sense to add a new Leg before removing one that is
>         misbehaving, and switch to the new Leg before removing the old.
> "
> 
> does it? if the old one is misbehaving, wouldn't removing it send
> traffic up to the root (according to non-optimized flow), allowing
> traffic to flow rather than get dropped?  yes, there are some cases
> where this doesn't work.

"
                                                     It makes sense to
   add a new Leg before removing one that is becoming excessively lossy,
   and switch to the new Leg before removing the old.  Dropping a Track
   before the new one is installed would reroute the traffic via the
   root; this may augment the latency beyond acceptable thresholds, and
   load the network near the root.  This may also cause loops in the
   case of stitched Tracks; the packets that cannot be injected in the
   second Track may be routed back at reinjected at the Ingress of the
   first."


> 
> section 6.6:
>         "When forwarding a packet to a destination for which a router
>         determines that routing happens via a Track for which it is
> Ingress,"
> 
> can you reword?  Sounds like the Marx Brothers sketch about a the party
> of the first party... (_A Day at the Races_)
> 
"
   When injecting a packet in a Track, the Ingress router must
   encapsulate the packet ...
 " 


> I think that the MTU statement deserves its own section, probably in
> section 4.

Made it a global prereq section
"

6.1.  RPL Network Setup

   To avoid the need of Path MTU Discovery, 6LoWPAN links are normally
   defined with a MTU of 1280 (see section 4 of [6LoWPAN]).  Injecting
   packets in a Track typically involves an IP-in-IP encapsulation and
   additional IPv6 Extension Headers.  This may cause a fragmentation if
   the resulting packets exceeds the MTU that is defined for the RPL
   domain.

   Though fragmentation is possible in a 6LoWPAN LLN, e.g., using
   [6LoWPAN], [RFC8930], and/or [RFC8931], it is RECOMMENDED to allow an
   MTU that is larger than 1280 in the main DODAG and allows for the
   additional headers while exposing only 1280 to the 6LoWPAN Nodes.
"
> 
> section 7: how are the extra DAOs turned on in storing mode?
>         CAPEX seems to be in order here?
> 

Yes, another to discuss with the list.

Added "
   Since there is no capability negotiation, the expectation is that all
   the nodes in the network support this specification in the same
   fashion, or are configured the same way through management.
" 
But the autonomic spirit of RPL and the need to avoid flag day suggests that at least we turn this on with a bit in the DIO.
So: should we consume the last bit in https://www.iana.org/assignments/rpl/rpl.xhtml#dodag-config-option-flags to indicate this spec is activated?


> section 8: i like these profiles.  We do need diagrams.
>         (again, SVG and goat may be very nice to use) Do the profiles
> need to be announced in some place in the protocol?

Argl, I do not see that. Hopefully not?

> 
> section 9: I don't think that the new risk for forged P-DAOs is
> adequately
>         explained.   Can we at least authenticate P-DAO messages
> because they
>         come from the Root? Sometimes the Root IP address is the same
> as the DODAGID.
>         (but not always? I forget)

For P-DAO it is, yes. But we do not secure messages from the root. That would be nice. How can we do that?

> 
> 
> I would like to see some section that explains the additions/changes
> *just* from the 6LR point of view.  What new attributes need to be
> added to the routes?  Most of the data path stuff is, I think, longest
> prefix match, and then possible encapsulation.  

Well that's not a change, any router should to that. RPL is not limited to host and default routes.


> RFC9008 probably has
> provided all the primitives.  (I hope!) But, I'd like to see a state
> machine of sorts that relates the P-DAO, DAO-ACK and PDR/PDR-ACK
> messages.

Again, I'd like to see this agreed by the WG. Would that be normative? 


> I fear that the SHOULD/MUSTs/new-ICMPs messages are too spread over the
> document.

The previous review caused some restructuring. The more we do that the more we spread. What do we prefer, stricture vs. MUSt concentration?

Again; many thanks!


Take care, Pascal