Re: [Detnet] Benjamin Kaduk's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

[Benjamin Kaduk <[e-mail]>]

On Sat, Mar 02, 2019 at 09:12:23PM +0100, János Farkas wrote:
> Hi Benjamin,
> 
> Coming to your detailed comments.
> Please see in-line.
> 
> Thank you,
> Janos
> 
> 
> On 2/20/2019 4:44 AM, Benjamin Kaduk wrote:
> > ----------------------------------------------------------------------
> > COMMENT:
> > ----------------------------------------------------------------------
> >
> >
> >
> > Abstract
> >
> >                                   DetNet operates at the IP layer and
> >     delivers service over sub-network technologies such as MPLS and IEEE
> >     802.1 Time-Sensitive Networking (TSN).
> >
> > I don't know what "sub-network technologies" means.  (Should I?  Is it
> > defined somewhere we can reference?)
> >
> > More generally, is DetNet supposed to be a "sub-layer" and/or "sub-network"
> > that lies between specific layers or classes of layer?  Does DetNet
> > itself have component "sub-layers" that provide distinct DetNet
> > functionality?  These are good questions to address early on in the
> > document so the reader is familiar with the concepts as they progress
> > through the document.
> Please note that the WG has reached consensus on introducing the term 
> "sub-layer", in particular using the terms "DetNet Service sub-layer" 
> and DetNet Forwarding sub-layer", see: 
> https://mailarchive.ietf.org/arch/msg/detnet/TR6oXu7IMWhelrH_I-tOuHGfDqo.
> These are explained in detail in Section 4. DetNet Architecture. I do 
> not see how to bring it earlier in the document.

That's a fair point.  I guess I would consider adding a Terminology entry
for "Sub-layer" that notes that "The DetNet architecture divides the DetNet
functionality into two sub-layers, the forwarding sub-layer and the service
sub-layer.", but use your judgement.

> Section 4. DetNet Architecture also explains what is meant by 
> sub-network. Figure 3 shows it and the text above Figure 3 explains.
> I don't see how could be this explanation earlier in the document.
> 
> As for the abstract, would it help to replace?:
> 
> OLD:
> DetNet operates at the IP layer and
>     delivers service over sub-network technologies such as MPLS and IEEE
>     802.1 Time-Sensitive Networking (TSN).
> 
> with
> 
> NEW:
> DetNet operates at the IP layer and
>     delivers service over lower layer technologies such as MPLS and IEEE
>     802.1 Time-Sensitive Networking (TSN).
> 
> Similar replacement could be done in the introduction:
> 
> OLD:
>     DetNet operates at the IP layer and delivers service over sub-network
>     technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
>     (TSN).
> 
> NEW:
>     DetNet operates at the IP layer and delivers service over lower layer
>     technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
>     (TSN).
> 
> 
> The next occurrence of sub-network is with a cross-reference to the 
> section with Figure 3 and description. I think it is fine this way as 
> the reader can jump there and check the details.

I agree; these two replacements in abstract+introduction would alleviate
the main question of whether this usage is some existing well-known thing
that the reader should be familiar with, and the rest of the document is
well-contained.

> 
> >
> > Section 1
> >
> >                            DetNet is for networks that are under a single
> >     administrative control or within a closed group of administrative
> >     control; these include campus-wide networks and private WANs.  DetNet
> >     is not for large groups of domains such as the Internet.
> >
> > side note: Campus-wide networks at educational institutions are basically
> > guaranteed to have untrusted entities participating in them, just as a
> > backdrop for security considerations.
> >
> > Section 3.1
> >
> >                          This mechanism distributes the contents of
> >     DetNet flows over multiple paths in time and/or space, so that the
> >     loss of some of the paths does need not cause the loss of any
> >
> > The failure models for which this statement is absolutely true as opposed
> > to probabilistically true seem rather unrealistic models of real physical
> > systems.
> >
> > Section 3.2.1.1
> >
> >     The primary means by which DetNet achieves its QoS assurances is to
> >     reduce, or even completely eliminate packet loss due to output packet
> >     contention within a DetNet node as a cause of packet loss.  [...]
> >
> > editing error?
> >
> >                                       Note that App-flows are generally
> >     not expected to be responsive to implicit [RFC2914] or explicit
> >     congestion notification [RFC3168].
> >
> > I note that the word "implicit" does not appear in RFC 2914; it may be
> > worth a bit more detailed of a mapping from concept to reference.
> > (This text/reference also appears in Section 4.3.2.)
> Would it help to change the text?:
> 
> OLD:
>     There is no expectation in DetNet for App-flows to be responsive to
>     implicit [RFC2914] or explicit congestion notification [RFC3168].
> 
> NEW:
>     There is no expectation in DetNet for App-flows to be responsive to
>     congestion control [RFC2914] or explicit congestion notification 
> [RFC3168].

Sure, thanks.

> 
> >
> > Section 3.2.1.2
> >
> >                                                                     In
> >     general, users are encouraged to use, instead of, "do this when you
> >     get the packet," a combination of:
> >
> > It seems that an architecture would be within its rights to *mandate* such
> > application design, rather than just encourage it.  What sorts of
> > exceptions would cause us to not want to mandate this design?
> This is not mandatory.
> For instance, it is not mandatory to use time synchronization. There may 
> be DetNet services that do not require time synchronization.

I was basically asking if you could list an example for me (here in email,
not necessarily in the document).

> 
> >
> > Section 3.2.2.2
> >
> > Please expand SRLG (it is only used once, so the abbreviation itself may
> > not be needed at all).
> Replace "SRLG" with "Shared Risk Link Group (SRLG)"
> 
> >
> > Section 3.2.3
> >
> >     Out-of-order packet delivery can be a side effect of distributing a
> >     single flow over multiple paths especially when there is a change
> >     from one path to another when combining the flow.  [...]
> >
> > nit: comma before "especially".
> Add comma.
> 
> >
> >     Resource allocation
> >             The DetNet forwarding sub-layer provides resource allocation.
> >             See Section 4.5.  The actual queuing and shaping mechanisms
> >             are typically provided by underlying subnet, these can be
> >
> > nit: is this usage of "subnet" common?
> Figure 3 shows it and the text above Figure 3 explains sub-net.
> 
> > Also, this comma looks to be a comma splice.
> Make it two sentences.
> 
> >
> >             closely associated with the means of providing paths for
> >             DetNet flows, the path and the resource allocation are
> >             conflated in this figure.
> >
> > nit: Hmm, actually, is this comma *also* a comma splice?
> Make it two sentences.
> 
> >
> >     Operations, Administration, and Maintenance (OAM) leverages in-band
> >     and out-of-band signaling that validates whether the service is
> >     effectively obtained within QoS constraints.  [...]
> >
> > nit: is there a singular/plural mismatch here ("the service"/"service" vs.
> > "effectively within"/"effectively obtained within")?
> Seems good to me.
> 
> >
> > Section 4.1.1
> >
> > This figure would have helped me a lot several sections earlier.
> >
> > Section 4.1.2
> >
> >     A "Deterministic Network" will be composed of DetNet enabled end
> >     systems, DetNet edge nodes, DetNet relay nodes and collectively
> >     deliver DetNet services.  DetNet relay and edge nodes are
> >
> > Nit: I think this is intended to be:
> > A "Deterministic Network" will be composed of DetNet-enabled end
> > systems, DetNet edge nodes, and DetNet relay nodes, which collectively
> > deliver DetNet services.  DetNet relay and edge nodes are
> Fine by me to make the cahnge.
> 
> >
> >                                                               Examples of
> >     sub-networks include MPLS TE, IEEE 802.1 TSN and OTN.  [...]
> >
> > nit: are these sub-networks or protocols used by sub-networks?
> These are sub-network technologies. They may include/use multiple protocols.

Should this be "examples of sub-networking technologies" then?

> 
> >
> >     Distinguishing the function of two DetNet data plane sub-layers, the
> >     DetNet service sub-layer and the DetNet forwarding sub-layer, helps
> >     to explore and evaluate various combinations of the data plane
> >     solutions available, some are illustrated in Figure 4.  This
> >
> > nit: this last comma is a comma splice.
> Suggest to replace with:
> "DetNet data plane is divided into two sub-layers: the
> DetNet service sub-layer and the DetNet forwarding sub-layer.
> This helps to explore and evaluate various combinations of the data plane
> solutions available. Some of them are illustrated in Figure 4."
> (https://mailarchive.ietf.org/arch/msg/detnet/JvPTAFVTJwxjhzZYEF9uCnUTFBA)

Thanks.

> >
> >     There are many valid options to create a data plane solution for
> >     DetNet traffic by selecting a technology approach for the DetNet
> >     service sub-layer and also selecting a technology approach for the
> >     DetNet forwarding sub-layer.  There are a high number of valid
> >     combinations.
> >
> > nit: I think "large number" is more conventional prose.
> Replace "high number" with "large number".
> 
> >
> > Section 4.3.1
> >
> > I think I'm confused about how, for these flows that "require the <foo>
> > feature", whether that means that the DetNet implementation must provide
> > <foo>, or that it is required for the application to have implemented the
> > <foo> feature.  A mapping (if it makes sense) to the categorization of end
> > systems in Section 4.2.1 would be a big help.
> The notation is crafted such that to provide mapping.

I inferred that intent, yet.  I'm claiming that "It only requires" confused
me about which entity required what.  Changing to "It only uses" would
remove that confusion (but would implicitly deny the ability for
opportunistic use of other features, so is not a perfect replacement).

> 
> >
> > Section 4.3.2
> >
> >     Asynchronous DetNet flows are characterized by:
> >
> >     o  A maximum packet size;
> >
> >     o  An observation interval; and
> >
> >     o  A maximum number of transmissions during that observation
> >        interval.
> >
> > Is there necessarily only a single tier of observation interval/rate?
> > (E.g., could there be a burst cap in a small interval and then a lower
> > overall baseline rate over large intervals?)
> We intentionally have a simple model here.

OK.

> 
> >
> >                       That is, while any useful application is written to
> >     expect a certain number of lost packets, the real-time applications
> >     of interest to DetNet demand that the loss of data due to the network
> >     is a rare event.
> >
> > (I might even go with "vanishingly rare".)
> Yes, there are extremely low loss requirements in certain use cases. 
> That's why we have PREOF.
> 
> >
> > Section 4.4.1
> >
> > (Is there a standard reference for "Northbound"?  I know we're all used to
> > it, but it's probably best to have a reference if we can.)
> RFC7426 is a reference. It is cited at the very beginning of the section.
> 
> >
> > Section 4.4.2
> >
> >                                           The deterministic sequence can
> >     typically be more complex than a direct sequence and include
> >     redundancy path, with one or more packet replication and elimination
> >     points.  [...]
> >
> > nit: "redundancy paths", plural?
> 
> Replace "redundancy paths", "redundant paths".
> 
> 
> >
> > Section 4.8
> >
> > How does *provisioning* require knowledge of *dynamic* state?
> I guess the idea was that the resource may change dynamically.
> 
> Would it help to delete "dynamic"?
> Thus get: "The state of a DetNet node's DetNet resources."

That would probably help, yes.  Maybe even "The current state", but I
didn't think about it very hard.

> >
> > Section 4.9
> >
> > Does aggregation like this pose a risk of all the aggregatees getting
> > affected when one exceeds their allocation substantially (so as to also
> > cause the aggregate to exceed the aggregate's allocation)?
> One way could be to apply rate limitation for each member flow being 
> aggregated at the input of the aggregate flow. The need to apply rate 
> limitation is explained at multiple places in the document.

OK

> >
> > Section 6
> >
> > The ability for an attacker to use QoS markings as part of traffic
> > correlation/inspection is not new with DetNet, but is probably still worth
> > mentioning explicitly.
> Do you mean mention it in Section 5 Security Considerations?

That's not what I had in mind, no. I was thinking

OLD:
   DetNet is provides a Quality of Service (QoS), and as such, does not
   directly raise any new privacy considerations.

NEW:
   DetNet provides a Quality of Service (QoS) mechanism, and the generic
   considerations for such mechanisms apply.  In particular, such markings
   allow for an attacker to correlate flows or to select particular types
   of flow for more detailed inspection.

> The text in my previous mail 
> (https://mailarchive.ietf.org/arch/msg/detnet/mOzgdXOJ5lGr4gLBGAapYt6Azl8) 
> has a paragraph starting:
> "To provide uninterrupted availability of the DetNet quality of service, 
> provisions must be made against DOS attacks and delay attacks."
> 
> Would it be good to extend that paragraph with a sentence, e.g.,?: 
> "Malicious QoS marking should be also considered".

I wouldn't oppose something like that, but it may not be needed in the
architecture document (as opposed to the documents that actually describe
the QoS markings).

-Benjamin