Re: [Detnet] Benjamin Kaduk's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

[János Farkas <[e-mail]>]

Hi Benjamin,

Please see in-line some further suggested changes based on your comments.

On 3/4/2019 4:41 AM, Benjamin Kaduk wrote:
> On Sat, Mar 02, 2019 at 09:12:23PM +0100, János Farkas wrote:
>> Hi Benjamin,
>>
>> Coming to your detailed comments.
>> Please see in-line.
>>
>> Thank you,
>> Janos
>>
>>
>> On 2/20/2019 4:44 AM, Benjamin Kaduk wrote:
>>> ----------------------------------------------------------------------
>>> COMMENT:
>>> ----------------------------------------------------------------------
>>>
>>>
>>>
>>> Abstract
>>>
>>>                                    DetNet operates at the IP layer and
>>>      delivers service over sub-network technologies such as MPLS and IEEE
>>>      802.1 Time-Sensitive Networking (TSN).
>>>
>>> I don't know what "sub-network technologies" means.  (Should I?  Is it
>>> defined somewhere we can reference?)
>>>
>>> More generally, is DetNet supposed to be a "sub-layer" and/or "sub-network"
>>> that lies between specific layers or classes of layer?  Does DetNet
>>> itself have component "sub-layers" that provide distinct DetNet
>>> functionality?  These are good questions to address early on in the
>>> document so the reader is familiar with the concepts as they progress
>>> through the document.
>> Please note that the WG has reached consensus on introducing the term
>> "sub-layer", in particular using the terms "DetNet Service sub-layer"
>> and DetNet Forwarding sub-layer", see:
>> https://mailarchive.ietf.org/arch/msg/detnet/TR6oXu7IMWhelrH_I-tOuHGfDqo.
>> These are explained in detail in Section 4. DetNet Architecture. I do
>> not see how to bring it earlier in the document.
> That's a fair point.  I guess I would consider adding a Terminology entry
> for "Sub-layer" that notes that "The DetNet architecture divides the DetNet
> functionality into two sub-layers, the forwarding sub-layer and the service
> sub-layer.", but use your judgement.
I suggest to extend the definitions of the two sub-layers instead of 
introducing a new one:

OLD:

    DetNet forwarding sub-layer
            The DetNet layer that optionally provides resource allocation
            for DetNet flows over paths provided by the underlying
            network.

    DetNet service sub-layer
            The DetNet sub-layer at which A DetNet service, e.g., service
            protection is provided.


NEW:

    DetNet forwarding sub-layer:
                   DetNet functionality is divided into two sub-layers. 
One of
                   them is the DetNet forwarding sub-layer, which optionally
                   provides resource allocation for DetNet flows over paths
                   provided by the underlying network.

    DetNet service sub-layer:
                   DetNet functionality is divided into two sub-layers. 
One of
                   them is the DetNet service sub-layer, at which a 
DetNet service,
                   e.g., service protection is provided.


>
>> Section 4. DetNet Architecture also explains what is meant by
>> sub-network. Figure 3 shows it and the text above Figure 3 explains.
>> I don't see how could be this explanation earlier in the document.
>>
>> As for the abstract, would it help to replace?:
>>
>> OLD:
>> DetNet operates at the IP layer and
>>      delivers service over sub-network technologies such as MPLS and IEEE
>>      802.1 Time-Sensitive Networking (TSN).
>>
>> with
>>
>> NEW:
>> DetNet operates at the IP layer and
>>      delivers service over lower layer technologies such as MPLS and IEEE
>>      802.1 Time-Sensitive Networking (TSN).
>>
>> Similar replacement could be done in the introduction:
>>
>> OLD:
>>      DetNet operates at the IP layer and delivers service over sub-network
>>      technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
>>      (TSN).
>>
>> NEW:
>>      DetNet operates at the IP layer and delivers service over lower layer
>>      technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
>>      (TSN).
>>
>>
>> The next occurrence of sub-network is with a cross-reference to the
>> section with Figure 3 and description. I think it is fine this way as
>> the reader can jump there and check the details.
> I agree; these two replacements in abstract+introduction would alleviate
> the main question of whether this usage is some existing well-known thing
> that the reader should be familiar with, and the rest of the document is
> well-contained.
>
>>> Section 1
>>>
>>>                             DetNet is for networks that are under a single
>>>      administrative control or within a closed group of administrative
>>>      control; these include campus-wide networks and private WANs.  DetNet
>>>      is not for large groups of domains such as the Internet.
>>>
>>> side note: Campus-wide networks at educational institutions are basically
>>> guaranteed to have untrusted entities participating in them, just as a
>>> backdrop for security considerations.
>>>
>>> Section 3.1
>>>
>>>                           This mechanism distributes the contents of
>>>      DetNet flows over multiple paths in time and/or space, so that the
>>>      loss of some of the paths does need not cause the loss of any
>>>
>>> The failure models for which this statement is absolutely true as opposed
>>> to probabilistically true seem rather unrealistic models of real physical
>>> systems.
>>>
>>> Section 3.2.1.1
>>>
>>>      The primary means by which DetNet achieves its QoS assurances is to
>>>      reduce, or even completely eliminate packet loss due to output packet
>>>      contention within a DetNet node as a cause of packet loss.  [...]
>>>
>>> editing error?
>>>
>>>                                        Note that App-flows are generally
>>>      not expected to be responsive to implicit [RFC2914] or explicit
>>>      congestion notification [RFC3168].
>>>
>>> I note that the word "implicit" does not appear in RFC 2914; it may be
>>> worth a bit more detailed of a mapping from concept to reference.
>>> (This text/reference also appears in Section 4.3.2.)
>> Would it help to change the text?:
>>
>> OLD:
>>      There is no expectation in DetNet for App-flows to be responsive to
>>      implicit [RFC2914] or explicit congestion notification [RFC3168].
>>
>> NEW:
>>      There is no expectation in DetNet for App-flows to be responsive to
>>      congestion control [RFC2914] or explicit congestion notification
>> [RFC3168].
> Sure, thanks.
>
>>> Section 3.2.1.2
>>>
>>>                                                                      In
>>>      general, users are encouraged to use, instead of, "do this when you
>>>      get the packet," a combination of:
>>>
>>> It seems that an architecture would be within its rights to *mandate* such
>>> application design, rather than just encourage it.  What sorts of
>>> exceptions would cause us to not want to mandate this design?
>> This is not mandatory.
>> For instance, it is not mandatory to use time synchronization. There may
>> be DetNet services that do not require time synchronization.
> I was basically asking if you could list an example for me (here in email,
> not necessarily in the document).
There may be a DetNet service with very strict loss requirement, but not 
that strict delay and jitter requirement. In this case, we may not need 
any fine tuned QoS technique to address delay. However, we need PREOF to 
address loss. Time synchronization is not needed in this example.


>
>>> Section 3.2.2.2
>>>
>>> Please expand SRLG (it is only used once, so the abbreviation itself may
>>> not be needed at all).
>> Replace "SRLG" with "Shared Risk Link Group (SRLG)"
>>
>>> Section 3.2.3
>>>
>>>      Out-of-order packet delivery can be a side effect of distributing a
>>>      single flow over multiple paths especially when there is a change
>>>      from one path to another when combining the flow.  [...]
>>>
>>> nit: comma before "especially".
>> Add comma.
>>
>>>      Resource allocation
>>>              The DetNet forwarding sub-layer provides resource allocation.
>>>              See Section 4.5.  The actual queuing and shaping mechanisms
>>>              are typically provided by underlying subnet, these can be
>>>
>>> nit: is this usage of "subnet" common?
>> Figure 3 shows it and the text above Figure 3 explains sub-net.
>>
>>> Also, this comma looks to be a comma splice.
>> Make it two sentences.
>>
>>>              closely associated with the means of providing paths for
>>>              DetNet flows, the path and the resource allocation are
>>>              conflated in this figure.
>>>
>>> nit: Hmm, actually, is this comma *also* a comma splice?
>> Make it two sentences.
>>
>>>      Operations, Administration, and Maintenance (OAM) leverages in-band
>>>      and out-of-band signaling that validates whether the service is
>>>      effectively obtained within QoS constraints.  [...]
>>>
>>> nit: is there a singular/plural mismatch here ("the service"/"service" vs.
>>> "effectively within"/"effectively obtained within")?
>> Seems good to me.
>>
>>> Section 4.1.1
>>>
>>> This figure would have helped me a lot several sections earlier.
>>>
>>> Section 4.1.2
>>>
>>>      A "Deterministic Network" will be composed of DetNet enabled end
>>>      systems, DetNet edge nodes, DetNet relay nodes and collectively
>>>      deliver DetNet services.  DetNet relay and edge nodes are
>>>
>>> Nit: I think this is intended to be:
>>> A "Deterministic Network" will be composed of DetNet-enabled end
>>> systems, DetNet edge nodes, and DetNet relay nodes, which collectively
>>> deliver DetNet services.  DetNet relay and edge nodes are
>> Fine by me to make the cahnge.
>>
>>>                                                                Examples of
>>>      sub-networks include MPLS TE, IEEE 802.1 TSN and OTN.  [...]
>>>
>>> nit: are these sub-networks or protocols used by sub-networks?
>> These are sub-network technologies. They may include/use multiple protocols.
> Should this be "examples of sub-networking technologies" then?
That would be clearer phrasing.
Replace
"Examples of sub-networks include MPLS TE, IEEE 802.1 TSN and OTN."
with
"Examples of sub-network technologies include MPLS TE, IEEE 802.1 TSN 
and OTN."

(sub-networking sounds a bit strange to me, not used in the document)


>
>>>      Distinguishing the function of two DetNet data plane sub-layers, the
>>>      DetNet service sub-layer and the DetNet forwarding sub-layer, helps
>>>      to explore and evaluate various combinations of the data plane
>>>      solutions available, some are illustrated in Figure 4.  This
>>>
>>> nit: this last comma is a comma splice.
>> Suggest to replace with:
>> "DetNet data plane is divided into two sub-layers: the
>> DetNet service sub-layer and the DetNet forwarding sub-layer.
>> This helps to explore and evaluate various combinations of the data plane
>> solutions available. Some of them are illustrated in Figure 4."
>> (https://mailarchive.ietf.org/arch/msg/detnet/JvPTAFVTJwxjhzZYEF9uCnUTFBA)
> Thanks.
>
>>>      There are many valid options to create a data plane solution for
>>>      DetNet traffic by selecting a technology approach for the DetNet
>>>      service sub-layer and also selecting a technology approach for the
>>>      DetNet forwarding sub-layer.  There are a high number of valid
>>>      combinations.
>>>
>>> nit: I think "large number" is more conventional prose.
>> Replace "high number" with "large number".
>>
>>> Section 4.3.1
>>>
>>> I think I'm confused about how, for these flows that "require the <foo>
>>> feature", whether that means that the DetNet implementation must provide
>>> <foo>, or that it is required for the application to have implemented the
>>> <foo> feature.  A mapping (if it makes sense) to the categorization of end
>>> systems in Section 4.2.1 would be a big help.
>> The notation is crafted such that to provide mapping.
> I inferred that intent, yet.  I'm claiming that "It only requires" confused
> me about which entity required what.  Changing to "It only uses" would
> remove that confusion (but would implicitly deny the ability for
> opportunistic use of other features, so is not a perfect replacement).
We can make it clearer. What about the following update:

OLD:

    A DetNet flow can have different formats while its packets are
    forwarded between the peer end systems.  Therefore, the following
    possible types / formats of a DetNet flow are distinguished in this
    document:

    o  App-flow: native format of the data carried over a DetNet flow.
       It does not contain any DetNet related attributes.


NEW:

    A DetNet flow can have different formats while its packets are
    forwarded between the peer end systems depending on the type
    of the end systems.  Corresponding to the end system types,
    the following possible types / formats of a DetNet flow are
    distinguished in this document. The different flow types have
    different requirements to DetNet nodes.

    o  App-flow: native format of the data carried over a DetNet flow
       between DetNet unaware end systems. An app-flow does not
       contain any DetNet related attributes and does not imply any
       specific requirement on DetNet nodes.




>
>>> Section 4.3.2
>>>
>>>      Asynchronous DetNet flows are characterized by:
>>>
>>>      o  A maximum packet size;
>>>
>>>      o  An observation interval; and
>>>
>>>      o  A maximum number of transmissions during that observation
>>>         interval.
>>>
>>> Is there necessarily only a single tier of observation interval/rate?
>>> (E.g., could there be a burst cap in a small interval and then a lower
>>> overall baseline rate over large intervals?)
>> We intentionally have a simple model here.
> OK.
>
>>>                        That is, while any useful application is written to
>>>      expect a certain number of lost packets, the real-time applications
>>>      of interest to DetNet demand that the loss of data due to the network
>>>      is a rare event.
>>>
>>> (I might even go with "vanishingly rare".)
>> Yes, there are extremely low loss requirements in certain use cases.
>> That's why we have PREOF.
>>
>>> Section 4.4.1
>>>
>>> (Is there a standard reference for "Northbound"?  I know we're all used to
>>> it, but it's probably best to have a reference if we can.)
>> RFC7426 is a reference. It is cited at the very beginning of the section.
>>
>>> Section 4.4.2
>>>
>>>                                            The deterministic sequence can
>>>      typically be more complex than a direct sequence and include
>>>      redundancy path, with one or more packet replication and elimination
>>>      points.  [...]
>>>
>>> nit: "redundancy paths", plural?
>> Replace "redundancy paths", "redundant paths".
>>
>>
>>> Section 4.8
>>>
>>> How does *provisioning* require knowledge of *dynamic* state?
>> I guess the idea was that the resource may change dynamically.
>>
>> Would it help to delete "dynamic"?
>> Thus get: "The state of a DetNet node's DetNet resources."
> That would probably help, yes.  Maybe even "The current state", but I
> didn't think about it very hard.
>
>>> Section 4.9
>>>
>>> Does aggregation like this pose a risk of all the aggregatees getting
>>> affected when one exceeds their allocation substantially (so as to also
>>> cause the aggregate to exceed the aggregate's allocation)?
>> One way could be to apply rate limitation for each member flow being
>> aggregated at the input of the aggregate flow. The need to apply rate
>> limitation is explained at multiple places in the document.
> OK
>
>>> Section 6
>>>
>>> The ability for an attacker to use QoS markings as part of traffic
>>> correlation/inspection is not new with DetNet, but is probably still worth
>>> mentioning explicitly.
>> Do you mean mention it in Section 5 Security Considerations?
> That's not what I had in mind, no. I was thinking
>
> OLD:
>     DetNet is provides a Quality of Service (QoS), and as such, does not
>     directly raise any new privacy considerations.
>
> NEW:
>     DetNet provides a Quality of Service (QoS) mechanism, and the generic
>     considerations for such mechanisms apply.  In particular, such markings
>     allow for an attacker to correlate flows or to select particular types
>     of flow for more detailed inspection.
Looks good to me. (the word "attacker" made me unsure whether it goes to 
security)
Combining with the change in 
https://mailarchive.ietf.org/arch/msg/detnet/QqUWlFnG9iJl5qXnuprPC6J4rDo, 
we'd get:

NEW:
    DetNet provides a Quality of Service (QoS) mechanism, and the generic
    considerations for such mechanisms apply.  In particular, such markings
    allow for an attacker to correlate flows or to select particular types
    of flow for more detailed inspection.

    DetNet provides a Quality of Service (QoS), and as such, is not 
expected to
    directly raise any new privacy considerations, the generic 
considerations
    for such mechanisms apply.  In particular, such markings allow for an
   attacker to correlate flows or to select particular types of flow for 
more
   detailed inspection.


>
>> The text in my previous mail
>> (https://mailarchive.ietf.org/arch/msg/detnet/mOzgdXOJ5lGr4gLBGAapYt6Azl8)
>> has a paragraph starting:
>> "To provide uninterrupted availability of the DetNet quality of service,
>> provisions must be made against DOS attacks and delay attacks."
>>
>> Would it be good to extend that paragraph with a sentence, e.g.,?:
>> "Malicious QoS marking should be also considered".
> I wouldn't oppose something like that, but it may not be needed in the
> architecture document (as opposed to the documents that actually describe
> the QoS markings).
OK, leave it then out from the architecture document.

Thank you,
Janos

>
> -Benjamin
>
> _______________________________________________
> detnet mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/detnet