[Tsv-art] Fwd: Re: Tsvart last call review of draft-ietf-detnet-architecture-08

[János Farkas <[e-mail]>]

Hi,

Coming back to the aspects that have not been picked up in the ongoing 
email discussions.
Combining the Michael's email and David's add-on to that.

Some comments, suggestions are in line, marked <JF> </JF>.

Best regards,
Janos

ps: In this month, I have difficulties with accessing emails. Sorry 
about that. I trust that the WG, including co-authors. continue the 
discussions.



On 11/4/2018 6:21 PM, Black, David wrote:
> Trying to help with these Transport issues.  I generally agree with Michael Sharf's
> first two concerns.  Like Michael, I will defer to the IESG on appropriate use of
> RFC 7426.
>
> In each case, I've extracted a key piece of text to try to make the point crisp ...
>
> [1] Minimum required level of DetNet support.
>
>> Does "some of the DetNet capabilities
>> (not necessarily all)" include e.g. vanilla traffic engineering? For instance, if a
>> network node can ensure that the QoS requirements are met by leveraging
>> existing standardized TE without any additional DetNet awareness, is this
>> enough for deploying DetNet? Or does this node have to implement additional
>> DetNet-specific functionality? Specifically, please explain if an existing DetNet-
>> unaware router on the path taken by DetNet traffic can be used to forward
>> DetNet traffic, or if it is mandatory that any router forwarding DetNet traffic
>> indeed implements new capabilities specific to DetNet.
> It's the former, as indicated by this text in Section 4.2.2 (I'm working from the -09 draft):
>
>      In general, if a DetNet flow passes through one or more DetNet-
>     unaware network nodes between two DetNet nodes providing the DetNet
>     transport sub-layer for that flow, there is a potential for
>     disruption or failure of the DetNet QoS.  A network administrator
>     needs to ensure that the DetNet-unaware network nodes are configured
>     to minimize the chances of packet loss and delay, and provision
>     enough extra buffer space in the DetNet transit node following the
>     DetNet-unaware network nodes to absorb the induced latency
>     variations.
>
> Perhaps a forward reference to Section 4.2.2 from the new text in the Introduction would help.
<JF>
David is right.
Forward reference will be added. Furthermore, the last sentence will be 
updated in order to make it clear.

The suggested new text:

A goal of DetNet is a converged network in all respects. That is, the 
presence of DetNet flows does not preclude non-DetNet flows, and the 
benefits offered DetNet flows should not, except in extreme cases, 
prevent existing QoS mechanisms from operating in a normal fashion, 
subject to the bandwidth required for the DetNet flows. A single 
source-destination pair can trade both DetNet and non-DetNet flows. End 
systems and applications need not instantiate special interfaces for 
DetNet flows. Networks are not restricted to certain topologies; 
connectivity is not restricted. Any application that generates a data 
flow that can be usefully characterized as having a maximum bandwidth 
should be able to take advantage of DetNet, as long as the necessary 
resources can be reserved. Reservations can be made by the application 
itself, via network management, by an application’s controller, or by 
other means, e.g., a dynamic control plane (e.g., [RFC2205]). Network 
nodes of a DetNet domain, i.e., DetNet nodes have to implement DetNet 
capabilities in order to treat DetNet flows such that their QoS 
requirements are met. DetNet nodes can be interconnected with different 
sub-network technologies (4.1.2), where the nodes of the subnet are not 
DetNet aware (4.2.2).



Furthermore, the the following text is suggested to be added at the end 
of section 4.5:

All nodes in a DetNet domain are expected to support the data behavior 
required to deliver a particular DetNet service.  If a node itself  is 
not DetNet service aware, the DetNet nodes that are adjacent to such 
non-DetNet aware nodes must ensure that the non-DetNet aware node is 
provisioned  to appropriately support the DetNet service.  For example, 
an IEEE 802.1 TSN node may be used to interconnect DetNet aware nodes, 
and these DetNet nodes can map DetNet flows to 802.1 TSN flows. An other 
example, and MPLS-TE or TP domain may be used to interconnect DetNet 
aware nodes, and these DetNet nodes can map DetNet flows to TE LSPs 
which can provide the Quality of Service requirements of the DetNet 
service.
</JF>



On 10/22/2018 11:44 PM, Scharf, Michael wrote:
> Hi Janos,
>
> Thanks for the reply. My comments are inline [ms].
>
> I have omitted all proposed changes that work for me.
>
> Michael
>
>
> -----Original Message-----
> From: János Farkas<[e-mail]>  
> Sent: Wednesday, October 17, 2018 10:56 AM
> To: Scharf, Michael<[e-mail]>
> Cc:[e-mail];[e-mail]; DetNet WG<[e-mail]>
> Subject: Re: Tsvart last call review of draft-ietf-detnet-architecture-08
>
> Hi Michael,
>
> Thank you very much for the thorough review and good comments!
>
> Please find below in-line responses and how we plan to update the draft to resolve your comments.
>
> Best regards,
> Janos
>
>
> On 9/29/2018 12:24 AM, Michael Scharf wrote:
>> Reviewer: Michael Scharf
>> Review result: Ready with Issues
>>
>> The document "Deterministic Networking Architecture"
>> (draft-ietf-detnet-architecture-08) defines an overall framework for
>> Deterministic Networking.
>>
>> As TSV-ART reviewer, I believe that this document has issues as
>> detailed below.
>>
>> Michael
>>
>> Major issues:
> [...]
>
>> DetNet
>> basically requires support in (almost) all network devices
>> transporting DetNet traffic. That assumption should be explicitly
>> spelt out early in the document, e.g., in the introduction.
> Actually, some form of DetNet support is required from each node that transports DetNet traffic. For instance, DetNet flows have to be recognized in order not to spoil their QoS at the minimum.
>
> [ms] I look for an clear explanation in the architecture document of what "some form of DetNet support" exactly means.
<JF>
The key point is that each DetNet node of a DetNet MUST support DetNet.
There is text making this point, e.g., the definition of DetNet domain.
However, you are right, further clarifications are needed in the text. 
See suggestions below.
</JF>

> For instance, is standard MPLS TE without any DetNet support good enough for an LSR on the IP/MPLS path used by DetNet traffic, or not? As an example, when I read the following paragraph in draft-ietf-detnet-dp-sol-mpls-01 ...
>
>     Transit nodes are normal MPLS Label Switching Routers (LSRs).  They
>     are generally unaware of the special requirements of DetNet flows,
>     although they need to provide traffic engineering services and proper
>     QoS to the LSPs associated with DetNet flows to enhance the prospect
>     of the LSPs meeting the DetNet service requirements.  Some
>     implementations of transit nodes may be DetNet aware, but such nodes
>     just support the DetNet transport layer.
>
> ... I could maybe assume that actually there can be "transit nodes" that do not require *any* sort of DetNet support or DetNet awareness. Is that correct?
<JF>
It is correct.
DetNet nodes of a DetNet domain can be interconnected by various subnet 
technologies, e.g., MPLS TE. The nodes of the subnet, e.g., and LSR are 
not DetNet aware.
</JF>

> I think the architecture document has to be clear on such fundamental questions.
<JF>
See suggestions for clarification below.
</JF>


> The Introduction could be extended to clarify, e.g., the third paragraph:
>
> OLD:
> A goal of DetNet is a converged network in all respects. That is, the presence of DetNet flows does not preclude non-DetNet flows, and the benefits offered DetNet flows should not, except in extreme cases, prevent existing QoS mechanisms from operating in a normal fashion, subject to the bandwidth required for the DetNet flows. A single source-destination pair can trade both DetNet and non-DetNet flows. End systems and applications need not instantiate special interfaces for DetNet flows. Networks are not restricted to certain topologies; connectivity is not restricted. Any application that generates a data flow that can be usefully characterized as having a maximum bandwidth should be able to take advantage of DetNet, as long as the necessary resources can be reserved. Reservations can be made by the application itself, via network management, by an application’s controller, or by other means, e.g., a dynamic control plane (e.g., [RFC2205]).
>
> NEW:
> A goal of DetNet is a converged network in all respects. That is, the presence of DetNet flows does not preclude non-DetNet flows, and the benefits offered DetNet flows should not, except in extreme cases, prevent existing QoS mechanisms from operating in a normal fashion, subject to the bandwidth required for the DetNet flows. A single source-destination pair can trade both DetNet and non-DetNet flows. End systems and applications need not instantiate special interfaces for DetNet flows. Networks are not restricted to certain topologies; connectivity is not restricted. Any application that generates a data flow that can be usefully characterized as having a maximum bandwidth should be able to take advantage of DetNet, as long as the necessary resources can be reserved. Reservations can be made by the application itself, via network management, by an application’s controller, or by other means, e.g., a dynamic control plane (e.g., [RFC2205]). Network nodes supporting DetNet flows have to implement some of the DetNet capabilities (not necessarily all) in order to treat DetNet flows such that their QoS requirements are met.
>
> [ms] To me, the last sentence is not clear. Does "some of the DetNet capabilities (not necessarily all)" include e.g. vanilla traffic engineering? For instance, if a network node can ensure that the QoS requirements are met by leveraging existing standardized TE without any additional DetNet awareness, is this enough for deploying DetNet? Or does this node have to implement additional DetNet-specific functionality? Specifically, please explain if an existing DetNet-unaware router on the path taken by DetNet traffic can be used to forward DetNet traffic, or if it is mandatory that any router forwarding DetNet traffic indeed implements new capabilities specific to DetNet. In the latter case, I think the document would need to discuss incremental deployment strategies as well.
>
>> * It is surprising that there is hardly any discussion on network
>> robustness
>> and safety; this probably also relates to security. For instance,
>> misconfiguration or errors of functions performing packet replication
>> could
>> severely and permantly congest a network and cause harm. How does the
>> DetNet
>> architecture ensure that a network stays fully operational e.g. if the
>> topology
>> changes or there are equipment failures? Probably this can be solved by
>> implementations (e.g., dynamic control plane), but why are corresponding
>> requirements not spelt out? Section 3.3.2 speculates that filters and
>> policers
>> can help, and that may be true, but that probably still assumes
>> consistently
>> and correctly configured (and well-behaving) devices. And Section 3.3.2 is
>> vague and mentions a "infinite variety of possible failures" without
>> stating
>> any requirements or recommendations. There may be further solutions,
>> such as
>> circuit breakers and the like. Why are such topics not discussed?
> Security issues and considerations are addressed by the DetNet Security
> draft:https://datatracker.ietf.org/doc/draft-ietf-detnet-security.
> Reference to the  DetNet Security draft will be added.
>
> There will be a document dedicated to Operations, Administration and
> Maintenance (OAM), which will address operational, misconfiguration, and
> failure detection issues.
>
> The topology changes have to be solved by the control plane, either
> centralized or distributed.
>
> The filters and policers described in Section 3.3.2 also provide
> robustness by eliminating/mitigating traffic coming from malfunctioning
> devices.
>
> RFC2475 is recommended for traffic policing functions to increase
> robustness.
>
> The text  in Section 3.3.2 could be made clearer, e.g., by updating the
> first paragraph to:
>
> OLD:
> One key to building robust real-time systems is to reduce the
> infinite variety of possible failures to a number that can be
> analyzed with reasonable confidence. DetNet aids in the process by
> allowing for filters and policers to detect DetNet packets received
> on the wrong interface, or at the wrong time, or in too great a
> volume, and to then take actions such as discarding the offending
> packet, shutting down the offending DetNet flow, or shutting down the
> offending interface.
>
> NEW:
> Robust real-time systems require to reduce the number of possible
> failures. Filters and policers should be used in a DetNet network to
> detect if DetNet packets are received on the wrong interface, or at
> the wrong time, or in too great volume. Furthermore, filters and
> policers can take action to discard offending packets or flows, or
> trigger shutting down the offending DetNet flow, or shutting down
> the offending interface.
>
> [ms] That does not address my concern. As I wrote already, I believe this document has to discuss the applicability of circuit breakers according to RFC 8084.
>
>> * Somewhat related, the document only looks at impact of failures to
>> the QoS of
>> DetNet traffic. What is missing is a discussion how to protect
>> non-DetNet parts
>> of a network from any harm caused by DetNet mechanisms. Solutions to this
>> probably exist. But why is the impact on non-DetNet traffic (e.g., in
>> case of
>> topology changes or failures of DetNet functions) not discussed at all
>> in the
>> document?
> Actually the regulation of DetNet traffic by polices and filters
> protects both other DetNet traffic and non-DetNet traffic.
>
> Section 3.3.1 could be extended to make it clear, e.g., by extending the
> last paragraph:
>
> OLD:
> Ideally, the net effect of the presence of DetNet flows in a network
> on the non-DetNet packets is primarily a reduction in the available
> bandwidth.
>
> NEW:
> Traffic policing functions (e.g. [RFC2475]) are used to avoid the
> starvation of non-DetNet traffic. Thus, the net effect of the presence
> of DetNet flows in a network on non-DetNet flows is primarily a
> reduction in the available bandwidth.
>
> [ms] I believe a stronger wording is needed, e.g. along the lines of "a DetNet deployment must avoid starvation of non-DetNet traffic ...".
<JF>
Along your suggestion, the text will be updated to:
Starvation of non-DetNet traffic must be avoided, e.g., by traffic
policing functions (e.g. [RFC2475]). Thus, the net effect of the
presence of DetNet flows in a network on non-DetNet flows is
primarily a reduction in the available bandwidth.
</JF>

> [...]
>
>> * Section 4.4 refers to RFC 7426, which is an informational RFC on
>> IRTF stream,
>> and the document uses the concepts introduced there (e.g., "planes").
>> This is
>> very confusing. First, an IETF Proposed Standard should probably refer to
>> documents having IETF consensus. An example would be RFC 7491, albeit
>> there is
>> other related work as well, e.g., in the TEAS WG. Second, Section 4.4
>> is by and
>> large decoupled from the rest of the document and not specific to DetNet.
>> Neither do other sections of the document refer to the concepts
>> introduced in
>> Section 4.4, nor does Section 4.4 use the DetNet terminology or discuss
>> applicability to DetNet. Section 4.4 even mentions explicitly at the
>> end that
>> it discusses aspects that are orthogonal to the DetNet architecture.
>> It is not
>> at all clear why Section 4.4 is in this document. Section 4.4 could be
>> removed
>> from the document without impacting the rest of the document.
> The document should point out to traffic engineering and centralized
> control plane aspects, so it is better to keep Section 4.4.
>
> RFC 7426 provides the full picture of SDN architecture, which needs to
> be referred from the DetNet architecture. No other RFC provides such
> detailed picture. Therefore, we need to keep the reference to RFC 7426,
> which is an informative reference.
>
> [ms] I'll leave that up to the IESG to decide whether an IETF proposed standard should build an architecture based on research work instead of, say, documents with IETF consensus. As a side note, I believe the understanding in industry what "SDN architecture" really means and how it is implemented may have evolved since publication of IRTF RFC 7426, which was published in Jan. 2015. For instance, IRTF RFC 7426 does not discuss the implications of segment routing, BGP-LS, and other recent IETF techniques. And, based on what I read in the DetNet architecture, I believe DetNet could actually be implemented without any "SDN architecture", e.g. by distributed traffic engineering. As a result, I don't even understand the need to discuss "SDN".
>
> [ms] For the record: As TSV-ART reviewer, I question whether IRTF RFC 7426 is indeed an appropriate reference, I have doubts that IRTF RFC 7426 is really up-to-date, and I am concerned about the content quality of section 4.4, e.g., regarding the lack of alignment with the rest of the document. The IESG will certainly have a better understanding that than me how to deal with such topics.

<JF>
The subject is introduced with TEAS in the beginning of Section 4.4. The 
architectural principles described in the DetNet architecture document 
are according to RFC 8453 and RFC 7149, as well as RFC 7426. It would be 
good to make this point clearer by adding references to RFC 8453 and RFC 
7149. As the architectural principles  are the same, there is no need to 
remove RFC 7426.

Suggested modifications:

Section 3.3:

OLD:
Explicit routes can be established in
    various ways, e.g., with RSVP-TE [RFC3209], with Segment Routing (SR)
    [RFC8402], via a Software Defined Networking approach [RFC7426], with
    IS-IS [RFC7813], etc.  Explicit routes are typically used in MPLS TE
    LSPs.

NEW:
Explicit routes can be established in
    various ways, e.g., with RSVP-TE [RFC3209], with Segment Routing (SR)
    [RFC8402], via a Software Defined Networking approach [RFC7426],
    [RFC8453], and [RFC7149], with IS-IS [RFC7813], etc. Explicit routes
   are typically used in MPLS TE LSPs.

Section 4.4

OLD:
    The Deterministic Networking architecture is thus composed of three
    planes, a (User) Application Plane, a Controller Plane, and a Network
    Plane, which echoes that of Figure 1 of Software-Defined Networking
    (SDN): Layers and Architecture Terminology [RFC7426].:


NEW:
    The Deterministic Networking architecture is thus composed of three
    planes, a (User) Application Plane, a Controller Plane, and a Network
    Plane, which echoes that of Figure 1 of Software-Defined Networking
    (SDN): Layers and Architecture Terminology [RFC7426] , and the
    Controllers identified in [RFC8453] and [RFC7149].

</JF>

>   
> The related work in TEAS is already referred to.
>
> [...]
>
>> Minor issues:
>>
>> * Terminology "DetNet transport layer"
>>
>> The term "transport layer" has a well-defined meaning in the IETF, e.g.
>> originating from RFC 1122. While "transport" and e.g. "transport
>> network" is
>> used in the IETF for different technologies in different areas, I
>> think the
>> term "transport layer" is typically understood to refer to transport
>> protocols such as TCP and UDP. As such, I personally find the term "DetNet
>> transport layer" misleading and confusing. The confusion is easy to
>> see e.g.
>> in Figure 4, where UDP (which is a transport protocol as per RFC 1122)
>> sits
>> on top of "transport".
>>
>> Based on the document it also may be solution/implementation specific
>> whether
>> the "DetNet transport layer" is actually a separate protocol layer
>> compared
>> to the "DetNet service layer". Thus it is not clear to me why the word
>> "layer" has to be used, specifically in combination "transport layer".
>>
>> To me as, the word "transport layer" (and "transport protocol") should be
>> used for protocols defined in TSV area, consistent with RFC 1122. But
>> this is
>> probably a question to be sorted out by the IESG.
> "transport" is used here as in "packet transport networks" not as OSI L4
> transport.
>
> [ms] At the risk of stating the obvious: The relevant section of RFC 1122 defining "transport layer" is entitled by "The Internet Architecture". And, according e.g. to RFC 5921, "packet transport network" is a term defined by ITU-T. In an IETF document, I believe IETF terminology should be used.
>
> I suggest to use the terms "DetNet transport sub-layer" and "DetNet
> service sub-layer" throughout the document, which would hopefully avoid
> confusion.
>
> [ms] "DetNet transport sub-layer" somewhat solves my concern of avoiding the expression "transport layer". To really avoid confusion, I believe it would be better to avoid the ambiguous term "transport" altogether.  But I will be fine if all DetNet documents consistently avoid the term "transport layer".
<JF>
The plan is to avoid "transport layer".

Based on your comment, it seems better to make the following change in 
3.2.2.2:

OLD:
    o  Providing sequencing information to the packets of a DetNet
       compound flow.  This may be done by adding a sequence number or
       time stamp as part of DetNet, or may be inherent in the packet,
       e.g., in a Layer-4 transport protocol, or associated to other
       physical properties such as the precise time (and radio channel)
       of reception of the packet.  This is typically done once, at or
       near the source.

NEW:
    o  Providing sequencing information to the packets of a DetNet
       compound flow.  This may be done by adding a sequence number or
       time stamp as part of DetNet, or may be inherent in the packet,
       e.g., in a higher layer protocol, or associated to other
       physical properties such as the precise time (and radio channel)
       of reception of the packet.  This is typically done once, at or
       near the source.
</JF>

> Michael