Re: [Detnet] Congestion Protection name change

Lou Berger <lberger@labn.net> Tue, 11 December 2018 12:39 UTC

Return-Path: <lberger@labn.net>
X-Original-To: detnet@ietfa.amsl.com
Delivered-To: detnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE6D6130DD6 for <detnet@ietfa.amsl.com>; Tue, 11 Dec 2018 04:39:54 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level:
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (768-bit key) header.d=labn.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CmxDGSvnIZ2B for <detnet@ietfa.amsl.com>; Tue, 11 Dec 2018 04:39:50 -0800 (PST)
Received: from gproxy2-pub.mail.unifiedlayer.com (gproxy2-pub.mail.unifiedlayer.com [69.89.18.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B7375130DD0 for <detnet@ietf.org>; Tue, 11 Dec 2018 04:39:50 -0800 (PST)
Received: from cmgw14.unifiedlayer.com (unknown [10.9.0.14]) by gproxy2.mail.unifiedlayer.com (Postfix) with ESMTP id 3E2701E116D for <detnet@ietf.org>; Tue, 11 Dec 2018 05:39:50 -0700 (MST)
Received: from box313.bluehost.com ([69.89.31.113]) by cmsmtp with ESMTP id WhKHg1oK46vZjWhKHgbtrU; Tue, 11 Dec 2018 05:39:50 -0700
X-Authority-Reason: nr=8
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=labn.net; s=default; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Subject: References:In-Reply-To:Message-ID:Date:CC:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=bZDI8nx23lNp2qvtUHJ0MQuk7EUJLP2vzeBFJpOIE9I=; b=BF87Kw+YW2u/M/pXH2vE4xWcI4 gVmgBNpgWYePQ1n0sJqm0oYnW2fjLygcmO1tdt3n8i3+PhfN4C0hjjfA/2z9SmjYX0grfsmuZvS+5 Dj96YUN7hWXdcu3MWWFJ40ckb;
Received: from pool-100-15-82-67.washdc.fios.verizon.net ([100.15.82.67]:59669 helo=[11.4.0.65]) by box313.bluehost.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from <lberger@labn.net>) id 1gWhKH-000a1Z-HK; Tue, 11 Dec 2018 05:39:49 -0700
From: Lou Berger <lberger@labn.net>
To: "Pascal Thubert (pthubert)" <pthubert@cisco.com>
CC: Balázs Varga A <balazs.a.varga@ericsson.com>, "Black, David" <David.Black@dell.com>, Janos Farkas <Janos.Farkas@ericsson.com>, draft-ietf-detnet-architecture.all@ietf.org, detnet@ietf.org
Date: Tue, 11 Dec 2018 07:39:46 -0500
Message-ID: <1679d47c650.27ce.9b4188e636579690ba6c69f2c8a0f1fd@labn.net>
In-Reply-To: <2A75E68D-2A59-4528-B008-418A613227B6@cisco.com>
References: <153817345967.27205.135001179751151278@ietfa.amsl.com> <fdf872d6-08a6-2c33-de21-9dd1506c1d21@labn.net> <6EC6417807D9754DA64F3087E2E2E03E2D16A4D3@rznt8114.rznt.rzdir.fht-esslingen.de> <e38ab4d6-0924-ab60-b1dc-4ac26600044c@labn.net> <090ae5ba-e44c-f8fa-7259-5ab1b01fb23c@ericsson.com> <167991e4c98.27ce.9b4188e636579690ba6c69f2c8a0f1fd@labn.net> <59a55a2e-45ac-ec18-8a7b-7b65490812e6@ericsson.com> <CE03DB3D7B45C245BCA0D243277949363038AC73@MX307CL04.corp.emc.com> <f9916f6612f14c68aa08bc96ebc27768@XCH-RCD-001.cisco.com> <AM5PR0701MB2514C0BBEAAA0271839B89B5ACA60@AM5PR0701MB2514.eurprd07.prod.outlook.com> <1679d161f60.27ce.9b4188e636579690ba6c69f2c8a0f1fd@labn.net> <2A75E68D-2A59-4528-B008-418A613227B6@cisco.com>
User-Agent: AquaMail/1.17.0-1324 (build: 101700010)
MIME-Version: 1.0
Content-Type: text/plain; format="flowed"; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - box313.bluehost.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - labn.net
X-BWhitelist: no
X-Source-IP: 100.15.82.67
X-Source-L: No
X-Exim-ID: 1gWhKH-000a1Z-HK
X-Source:
X-Source-Args:
X-Source-Dir:
X-Source-Sender: pool-100-15-82-67.washdc.fios.verizon.net ([11.4.0.65]) [100.15.82.67]:59669
X-Source-Auth: lberger@labn.net
X-Email-Count: 1
X-Source-Cap: bGFibm1vYmk7bGFibm1vYmk7Ym94MzEzLmJsdWVob3N0LmNvbQ==
X-Local-Domain: yes
Archived-At: <https://mailarchive.ietf.org/arch/msg/detnet/MGpnAzdsDG4fejI8tZO66Ks3j64>
Subject: Re: [Detnet] Congestion Protection name change
X-BeenThere: detnet@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions on Deterministic Networking BoF and Proposed WG <detnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/detnet>, <mailto:detnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/detnet/>
List-Post: <mailto:detnet@ietf.org>
List-Help: <mailto:detnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/detnet>, <mailto:detnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Dec 2018 12:39:55 -0000

Protection has very specific meaning in transport networks and I think 
would lead to great confusion.

I see no issue with the term management implying control(er) plane, as 
there will be some level of control here whether it be centralized or 
distributed.

This whole discussion is sort of why 'resource allocation' ended up being 
used in the TE context.  If that term and 'flow management' don't work, I 
think 'resource management' is at the top of my personal list.

Lou


----------
On December 11, 2018 6:59:26 AM "Pascal Thubert (pthubert)" 
<pthubert@cisco.com> wrote:

> Management looks like control plane, Lou, don’t you think?
> David’s flow protection also indicates the interesting security properties 
> that we are effectively bringing in...
>
>
> Regards,
>
> Pascal
>
>> Le 11 déc. 2018 à 15:45, Lou Berger <lberger@labn.net> a écrit :
>>
>> Balázs,
>>
>> In some systems that work on  more than just queue management is needed, so 
>> I personally feel pretty strongly that we should not include the word 
>> queue. How about:
>>
>> 'flow management'
>>
>> Lou
>>
>>
>>
>> ----------
>>> On December 11, 2018 2:21:19 AM Balázs Varga A 
>>> <balazs.a.varga@ericsson.com> wrote:
>>>
>>> Hi,
>>>
>>> "queuing loss avoidance".
>>>
>>> I am partly with You. I think the to include the term "flow" is a good idea.
>>> David has right, here we are focusing on flows.
>>>
>>> However I have problems with the "protection" part of the term, due to other
>>> terminology we have already defined, namely service protection (e.g., PREOF).
>>> I think for many people "DetNet" + "protection" = "PREOF".
>>>
>>> As Janos wrote:
>>>> We wanted to have a brief collective term for the mechanisms used
>>>> to avoid queuing related packet loss (including buffer overflow etc.).
>>> and Pascal extended:
>>> - and at the same time we want to fulfill the latency requirement of the flow.
>>>
>>> So, let's lists the functions we intend to describe with the term:
>>> - selecting queuing method used and the queue used by the flow
>>> - properly configuring queue parameters (e.g., egress speed, queue size, etc.),
>>> based on latency and (zero) loss requirements for the flow/hop
>>>
>>> A possible way to put all these pieces together:
>>> "flow queuing regulation " or "flow queuing management"
>>>
>>> Any thoughts?
>>>
>>> Cheers
>>> Bala'zs
>>>
>>> -----Original Message-----
>>> From: Pascal Thubert (pthubert) <pthubert@cisco.com>
>>> Sent: Tuesday, December 11, 2018 5:21 AM
>>> To: Black, David <David.Black@dell.com>; Janos Farkas 
>>> <Janos.Farkas@ericsson.com>; Lou Berger <lberger@labn.net>
>>> Cc: draft-ietf-detnet-architecture.all@ietf.org; detnet@ietf.org
>>> Subject: RE: [Detnet] Congestion Protection name change
>>>
>>> "Flow Protection" looks good to me too...
>>>
>>>> -----Original Message-----
>>>> From: Black, David <David.Black@dell.com>
>>>> Sent: mardi 11 décembre 2018 05:40
>>>> To: János Farkas <janos.farkas@ericsson.com>; Lou Berger
>>>> <lberger@labn.net>
>>>> Cc: draft-ietf-detnet-architecture.all@ietf.org; detnet@ietf.org
>>>> Subject: RE: [Detnet] Congestion Protection name change
>>>>
>>>> > >>     Congestion protection operates by allocating resources along the path
>>>> > >>     of a DetNet flow, e.g., buffer space or link bandwidth.  Congestion
>>>> > >>     protection greatly reduces, or even eliminates entirely, packet loss
>>>> > >>     due to output packet congestion within the network, but it can only
>>>> > >>     be supplied to a DetNet flow that is limited at the source to a
>>>> > >>     maximum packet size and transmission rate.
>>>>
>>>> > >> We wanted to have a brief collective term for the mechanisms used
>>>> > >> to avoid queuing related packet loss (including buffer overflow etc.).
>>>>
>>>> Perhaps "flow protection" as it protects individual flows and is
>>>> configured/administered/managed at flow granularity??
>>>>
>>>> The use of the word "resource" proposed below seems easy to mis-read
>>>> as involving resources beyond the flow-by-flow scope of this functionality.
>>>>
>>>> Thanks, --David
>>>>
>>>>
>>>> > -----Original Message-----
>>>> > From: detnet [mailto:detnet-bounces@ietf.org] On Behalf Of János
>>>> > Farkas
>>>> > Sent: Monday, December 10, 2018 12:59 PM
>>>> > To: Lou Berger
>>>> > Cc: draft-ietf-detnet-architecture.all@ietf.org; detnet@ietf.org
>>>> > Subject: Re: [Detnet] Congestion Protection name change
>>>> >
>>>> >
>>>> > [EXTERNAL EMAIL]
>>>> >
>>>> > I'm of course open to other alternatives. I'm interested to avoid
>>>> > the confusion; just proposed an initial idea.
>>>> > I'm afraid we won't find the ideal term, but it would be good to
>>>> > have a good enough one.
>>>> >
>>>> > On 12/10/2018 3:47 PM, Pascal Thubert (pthubert) wrote:
>>>> > > The problem János is that we do not only avoid loss. We also
>>>> > > control
>>>> > latency. So "queuing loss" is too limitation.
>>>> > > We are protecting the resources that are necessary, or providing
>>>> > guarantees that they are available. To provide the service.
>>>> > > Maybe "service guarantees" could be good?
>>>> > > Pascal
>>>> > I know, and it is also there in the document:
>>>> >
>>>> > "   Congestion protection addresses two of the DetNet QoS requirements:
>>>> >     latency and packet loss.  Given that DetNet nodes have a finite
>>>> >     amount of buffer space, congestion protection necessarily
>>>> > results in
>>>> >     a maximum end-to-end latency.  It also addresses the largest
>>>> >     contribution to packet loss, which is buffer congestion."
>>>> >
>>>> > also:
>>>> > "   The low-level mechanisms described in Section 4.5 provide the
>>>> >     necessary regulation of transmissions by an end system or DetNet
>>>> > node
>>>> >     to provide congestion protection.  The allocation of the
>>>> > bandwidth
>>>> >     and buffers for a DetNet flow requires provisioning.  A DetNet
>>>> > node
>>>> >     may have other resources requiring allocation and/or scheduling,
>>>> > that
>>>> >     might otherwise be over-subscribed and trigger the rejection of
>>>> > a
>>>> >     reservation."
>>>> >
>>>> > In other words, we need adequate queuing mechanism with appropriate
>>>> > queue management plus resource allocation.
>>>> >
>>>> > On 12/10/2018 6:15 PM, Lou Berger wrote:
>>>> > > Fwiw in the te world, we normally call this "resource allocation".
>>>> > > As a contributor, I'd be comfortable with that term or "resource
>>>> > > management".
>>>> >
>>>> > I'd prefer "resource management" out of these two because I can talk
>>>> > into it that it is the combination of "resource allocation" and
>>>> > queue management".
>>>> >
>>>> > Thanks,
>>>> > Janos
>>>> >
>>>> >
>>>> > >
>>>> > > Lou
>>>> > >
>>>> > >
>>>> > >
>>>> > > ----------
>>>> > > On December 10, 2018 9:30:20 AM János Farkas
>>>> > > <janos.farkas@ericsson.com> wrote:
>>>> > >
>>>> > >> Hi,
>>>> > >>
>>>> > >> As I understand, there is confusion around two DetNet terms.
>>>> > >> We are removing "transport".
>>>> > >>
>>>> > >> The other one is "congestion" and "congestion protection".
>>>> > >>
>>>> > >> Brief description of congestion protection in Section 3.1:
>>>> > >>
>>>> > >>     Congestion protection operates by allocating resources along
>>>> > >> the path
>>>> > >>     of a DetNet flow, e.g., buffer space or link bandwidth.
>>>> > >> Congestion
>>>> > >>     protection greatly reduces, or even eliminates entirely,
>>>> > >> packet loss
>>>> > >>     due to output packet congestion within the network, but it
>>>> > >> can only
>>>> > >>     be supplied to a DetNet flow that is limited at the source to
>>>> > >> a
>>>> > >>     maximum packet size and transmission rate.  Note that
>>>> > >> congestion
>>>> > >>     protection provided via congestion detection and notification
>>>> > >> is
>>>> > >>     explicitly excluded from consideration in DetNet, as it
>>>> > >> serves a
>>>> > >>     different set of applications.
>>>> > >>
>>>> > >>     Congestion protection addresses two of the DetNet QoS requirements:
>>>> > >>     latency and packet loss.  Given that DetNet nodes have a
>>>> > >> finite
>>>> > >>     amount of buffer space, congestion protection necessarily
>>>> > >> results in
>>>> > >>     a maximum end-to-end latency.  It also addresses the largest
>>>> > >>     contribution to packet loss, which is buffer congestion.
>>>> > >>
>>>> > >> Detailed description is in Section 3.2.1.
>>>> > >>
>>>> > >> We wanted to have a brief collective term for the mechanisms used
>>>> > >> to avoid queuing related packet loss (including buffer overflow etc.).
>>>> > >>
>>>> > >> Based on the discussions, we should have a term that does not
>>>> > >> include "congestion".
>>>> > >> ("Service Protection" is another DetNet term, hence we may
>>>> > >> consider a term without "protection" to minimize confusion.)
>>>> > >>
>>>> > >> I suggest to replace "congestion protection" with "queuing loss
>>>> > >> avoidance".
>>>> > >>
>>>> > >> After agreeing in the terminology cahnge (if any), the text has
>>>> > >> to be updated accordingly.
>>>> > >>
>>>> > >> What do you think?
>>>> > >>
>>>> > >> Best regards,
>>>> > >> Janos
>>>> > >>
>>>> > >>
>>>> > >> On 11/20/2018 1:11 PM, Lou Berger wrote:
>>>> > >>> Michael,
>>>> > >>>
>>>> > >>> I think we're getting somewhere and identifying where we have
>>>> > >>> disconnects and what may (and what may not) need to change in
>>>> > >>> the document.  My takeaways are:
>>>> > >>>
>>>> > >>> - The document needs a good 'scrub' of the congestion related
>>>> > >>> references to ensure that the document only makes statements on
>>>> > what
>>>> > >>> is actually done within a DetNet and the relationship with
>>>> > >>> transport protocols that use detnet (which are in fact outside
>>>> > >>> the scope of the DetNet WG).  I'll work with the authors and WG
>>>> > >>> on this -- I see this change as important, but editorial in nature.
>>>> > >>>
>>>> > >>> - We have a perception issue with at least one member of the TSV
>>>> > >>> area on the meaning and more importantly, implication, of the
>>>> > >>> term "DetNet
>>>> > >>> *Transport* sub-layer".  While I don't disagree that a good
>>>> > >>> portion of the IETF thinks transport protocol (UDP/TCP) when
>>>> > >>> they hear "transport" there are plenty others, particularly in
>>>> > >>> the routing area, who understand that "transport" can refer to
>>>> > >>> Transport Networks.  And Transport Network is a well understood
>>>> > >>> general industry term. The IETF even has a bunch of RFCs that
>>>> > >>> relate to Transport networks. This said, I think it reasonable
>>>> > >>> to go back to the DetNet WG and discuss changing the name of the
>>>> > >>> "DetNet Transport sub-layer"  to avoid the word "transport".  --
>>>> > >>> BTW we made a parallel change in the TEAS WG when producing
>>>> RFC8453.
>>>> > >>>
>>>> > >>> See below for detail response in-line.
>>>> > >>>
>>>> > >>> On 11/19/2018 5:15 PM, Scharf, Michael wrote:
>>>> > >>>> Lou,
>>>> > >>>>
>>>> > >>>>> --
>>>> > >>>>> I wanted to take a step back from the multiple discussions
>>>> > >>>>> that were spawned by your review -- from a doc shepherd
>>>> > >>>>> perspective, and see where we are.   I know that the authors
>>>> > >>>>> have sent a -09 version that addresses some, but not all issues.
>>>> > >>>>>
>>>> > >>>>>   From the exchanges I've seen, I think the key remaining
>>>> > >>>>> issues are related to:
>>>> > >>>>>
>>>> > >>>>> (a) possibly introduction of congestion in the general
>>>> > >>>>> internet if packets were somehow to escape a detnet domain.
>>>> > >>>>> The source of
>>>> > this
>>>> > >>>>> congestion would be inelastic traffic using DetNet or due to
>>>> > >>>>> congestion loss that is masked by PREOF.
>>>> > >>>> These are two major issues that need to be addressed. Note that
>>>> > >>>> it may not be sufficient just to add a section on operational
>>>> > >>>> and deployment considerations. Also the existing text in the
>>>> > >>>> document will need to get aligned to normative guidance on how
>>>> > >>>> to avoid a congestion collapse.
>>>> > >>>>
>>>> > >>>> In -09, one example would be Section 3.1. "Primary goals
>>>> > >>>> defining the DetNet QoS"
>>>> > >>>>
>>>> > >>>>     Congestion protection operates by allocating resources
>>>> > >>>> along the path
>>>> > >>>>     of a DetNet flow, e.g., buffer space or link bandwidth.
>>>> > >>>> Congestion
>>>> > >>>>     protection greatly reduces, or even eliminates entirely,
>>>> > >>>> packet loss
>>>> > >>>>     due to output packet congestion within the network, but it
>>>> > >>>> can only
>>>> > >>>>     be supplied to a DetNet flow that is limited at the source
>>>> > >>>> to a
>>>> > >>>>     maximum packet size and transmission rate.  Note that
>>>> > >>>> congestion
>>>> > >>>>     protection provided via congestion detection and
>>>> > >>>> notification is
>>>> > >>>>     explicitly excluded from consideration in DetNet, as it
>>>> > >>>> serves a
>>>> > >>>>     different set of applications.
>>>> > >>>
>>>> > >>>> At least the last sentence would contradict a better discussion
>>>> > >>>> of congestion in the document. For instance, it could just be removed.
>>>> > >>>> In any case, the current wording in the last sentence is not
>>>> > >>>> correct, as the IETF term for what is described in the last
>>>> > >>>> sentence is "congestion control".
>>>> > >>>>
>>>> > >>>> Another example would be  Section 3.2.1.1. "Eliminate congestion loss"
>>>> > >>>>       The primary means by which DetNet achieves its QoS
>>>> > >>>> assurances is to
>>>> > >>>>     reduce, or even completely eliminate, congestion within a
>>>> > >>>> DetNet node
>>>> > >>>>     as a cause of packet loss.  This can be achieved only by
>>>> > >>>> the
>>>> > >>>>     provision of sufficient buffer storage at each node through
>>>> > >>>> the
>>>> > >>>>     network to ensure that no packets are dropped due to a lack
>>>> > >>>> of buffer
>>>> > >>>>     storage.  Note that a DetNet flow cannot be throttled,
>>>> > >>>> i.e., its
>>>> > >>>>     transmission rate cannot be reduced via explicit congestion
>>>> > >>>>     notification.
>>>> > >>>>
>>>> > >>>> This section IMHO has to include a discussion of what happens
>>>> > >>>> in the (not expected) case that packets get dropped or that ECN
>>>> > >>>> marks are received. It is understood that this would not happen
>>>> > >>>> in normal operation of a DetNet network, but I believe just
>>>> > >>>> considering the error-free operation of a DetNet network is not
>>>> > >>>> sufficient for this document. At least for the risk of traffic
>>>> > >>>> that may escape from a DetNet network is inherently not
>>>> > >>>> sufficient to assume that the DetNet network is always error-free.
>>>> > >>>
>>>> > >>> I think these are examples of text that needs to be cleanup up
>>>> > >>> and to delineate what is done with a DetNet.
>>>> > >>>
>>>> > >>>
>>>> > >>>> As a result, addressing my concerns will most likely require
>>>> > >>>> editing several parts of the document.
>>>> > >>>>
>>>> > >>>> In addition, I'd like to emphasize that my review comment "It
>>>> > >>>> is surprising that there is hardly any discussion on network
>>>> > >>>> robustness and safety"
>>>> > >>>
>>>> > >>> I have no idea what you mean by safety here.  Can you elaborate.
>>>> > >>>
>>>> > >>>
>>>> > >>>> covers more than just inelastic traffic that escapes from a
>>>> > >>>> DetNet network and masking of packet loss. Given that DetNet
>>>> > >>>> traffic may be extremely critical traffic, I really wonder why
>>>> > >>>> the document doesn't emphasize more the required robustness
>>>> > >>>> against failures *inside* the DetNet network as well as
>>>> > >>>> counter-measures. But this is something the WG needs to decide.
>>>> > >>>> As TSV-ART reviewer, I will be fine if the document clearly
>>>> > >>>> describes how the impact of failures will be isolated inside
>>>> > >>>> the DetNet network and will not put the general Internet at risk.
>>>> > >>>
>>>> > >>> I agree - I think, the document should be clear on it's scope
>>>> > >>> and relationship to general internet usage.
>>>> > >>>
>>>> > >>>
>>>> > >>>>
>>>> > >>>>> (b) The use of the term 'transport' in DetNet to refer to what
>>>> > >>>>> is basically a Traffic Engineered sub-network layer, such as
>>>> > >>>>> is provided with MPLS-TE or Optical Transport Networks.
>>>> > >>>> In the Internet architecture, the term 'transport' refers to
>>>> > >>>> Internet transport protocols. I doubt that the document can
>>>> > >>>> avoid discussing the implications of and interactions with
>>>> > >>>> Internet transport protocols such as UDP or TCP. As a result, I
>>>> > >>>> disagree that the document can use the term 'transport' to
>>>> > >>>> refer to traffic engineered sub-network layers.
>>>> > >>>
>>>> > >>> I think this is covered by my comment above.
>>>> > >>>
>>>> > >>>
>>>> > >>>>  From a TSV-ART point of view, the document can either only use
>>>> > >>>> the term "transport" for Internet transport protocols and use
>>>> > >>>> another term for sub-network layers (as handled in the
>>>> > >>>> *routing* area of the IETF), or the document has to clearly
>>>> > >>>> distinguish between the Internet transport layer and other uses
>>>> > >>>> of the term "transport" and explain the overlap. I believe the
>>>> > >>>> former would be less confusing, but I will leave it up to the
>>>> > >>>> TSV ADs to discuss terminology overlap in the IESG. As TSV-ART
>>>> > >>>> reviewer I insist that the document uses the terms "transport
>>>> > >>>> layer" and "transport protocol" only when referring to the Internet 
>>>> transport layer.
>>>> > >>>
>>>> > >>> I'm personally okay with a name change and even willing to push
>>>> > >>> this discussion within the WG, but as said above, "Transport
>>>> > >>> Network" is a generally understood industry term that is also
>>>> > >>> used in RFCs -- so we'll have to see what where WG consensus ends up.
>>>> > >>>
>>>> > >>>>
>>>> > >>>>> Do you have any other issues that that are critical to be
>>>> > >>>>> addressed before this work moves forward?  If so which?
>>>> > >>>> Regarding Section 4.4 I have already deferred the discussion to
>>>> > >>>> the IESG. The TSV-ART review comment is that the IESG needs to
>>>> > >>>> carefully look at the concepts, terminology, and references in
>>>> > >>>> section
>>>> 4.4.
>>>> > >>>>
>>>> > >>>> Regarding my other comments, I acknowledge that -09 is a step
>>>> > >>>> forward. But given the cross-dependencies e.g. regarding
>>>> > >>>> terminology and definitions, I will need to read the text
>>>> > >>>> completely once there is a proposal how to address my review.
>>>> > >>>> As noted in my review, I believe the document must use
>>>> > >>>> terminology
>>>> clearly and consistently.
>>>> > >>>> As example, a statement in -09 such as "Network nodes
>>>> > >>>> supporting DetNet flows have to implement some of the DetNet
>>>> > >>>> capabilities (not necessarily all) in order to treat DetNet
>>>> > >>>> flows such that their QoS requirements are met" is IMHO too
>>>> > >>>> vague. But in such cases it depends whether there is precise
>>>> > >>>> normative guidance elsewhere. And this requires looking at the text 
>>>> as a whole.
>>>> > >>>
>>>> > >>> I think the next steps lie with me and the WG.  We'll let you
>>>> > >>> know once there is a new version.  Of course, you can also
>>>> > >>> contribute to the WG discussion on the topic.
>>>> > >>>
>>>> > >>> Thanks,
>>>> > >>>
>>>> > >>> Lou
>>>> > >>>
>>>> > >>>>
>>>> > >>>> Best regards
>>>> > >>>>
>>>> > >>>> Michael
>>>> > >>>>
>>>> > >>>>
>>>> > >>>>
>>>> > >>>>> Thank you,
>>>> > >>>>>
>>>> > >>>>> Lou
>>>> > >>>>>
>>>> > >>>>> On 9/28/2018 6:24 PM, Michael Scharf wrote:
>>>> > >>>>>> Reviewer: Michael Scharf
>>>> > >>>>>> Review result: Ready with Issues
>>>> > >>>>>>
>>>> > >>>>>> The document "Deterministic Networking Architecture"
>>>> > >>>>>> (draft-ietf-detnet-architecture-08) defines an overall
>>>> > >>>>>> framework for Deterministic Networking.
>>>> > >>>>>>
>>>> > >>>>>> As TSV-ART reviewer, I believe that this document has issues
>>>> > >>>>>> as
>>>> > >>>>> detailed below.
>>>> > >>>>>> Michael
>>>> > >>>>>>
>>>> > >>>>>> Major issues:
>>>> > >>>>>>
>>>> > >>>>>> * It seems that DetNet cannot easily be deployed in the
>>>> > >>>>>> Internet
>>>> > >>>>> without
>>>> > >>>>>> additional means. Thus, for a baseline document, one could
>>>> > >>>>>> expect
>>>> > >>>>> some
>>>> > >>>>>> explanation on the requirements of deploying DetNet in a network.
>>>> > >>>>> DetNet
>>>> > >>>>>> basically requires support in (almost) all network devices
>>>> > >>>>> transporting DetNet
>>>> > >>>>>> traffic. That assumption should be explicitly spelt out early
>>>> > >>>>>> in the
>>>> > >>>>> document,
>>>> > >>>>>> e.g., in the introduction. There also needs to be an explicit
>>>> > >>>>> discussion of the
>>>> > >>>>>> implications if not the whole network is aware of or supports
>>>> > >>>>>> DetNet.
>>>> > >>>>> There is
>>>> > >>>>>> some text in Section 4.2.2 and Section 4.3.3, but I believe
>>>> > >>>>> additional explicit
>>>> > >>>>>> discussion is needed at a prominant place. For instance, can
>>>> > >>>>>> use of
>>>> > >>>>> DetNet do
>>>> > >>>>>> harm to parts of a network not supporting DetNet? As a side
>>>> > >>>>>> note,
>>>> > >>>>> when TCPM
>>>> > >>>>>> published RFC 8257, the following disclaimer was added:
>>>> > >>>>>> "DCTCP, as
>>>> > >>>>> described in
>>>> > >>>>>> this specification, is applicable to deployments in
>>>> > >>>>>> controlled
>>>> > >>>>> environments
>>>> > >>>>>> like data centers, but it must not be deployed over the
>>>> > >>>>>> public
>>>> > >>>>> Internet without
>>>> > >>>>>> additional measures." I wonder if a similar disclaimer is
>>>> > >>>>>> needed for
>>>> > >>>>> DetNet. If
>>>> > >>>>>> there is an implicit assumption that DetNet will  be used in
>>>> > >>>>> homogenous
>>>> > >>>>>> environments with mostly DetNet-aware devices within the same
>>>> > >>>>> organization,
>>>> > >>>>>> such an assumption should be made explicit.
>>>> > >>>>>>
>>>> > >>>>>> * It is surprising that there is hardly any discussion on
>>>> > >>>>>> network
>>>> > >>>>> robustness
>>>> > >>>>>> and safety; this probably also relates to security. For
>>>> > >>>>>> instance, misconfiguration or errors of functions performing
>>>> > >>>>>> packet replication
>>>> > >>>>> could
>>>> > >>>>>> severely and permantly congest a network and cause harm. How
>>>> > does
>>>> > >>>>>> the
>>>> > >>>>> DetNet
>>>> > >>>>>> architecture ensure that a network stays fully operational e.g.
>>>> > >>>>>> if
>>>> > >>>>> the topology
>>>> > >>>>>> changes or there are equipment failures? Probably this can be
>>>> > solved
>>>> > >>>>> by
>>>> > >>>>>> implementations (e.g., dynamic control plane), but why are
>>>> > >>>>> corresponding
>>>> > >>>>>> requirements not spelt out? Section 3.3.2 speculates that
>>>> > >>>>>> filters and
>>>> > >>>>> policers
>>>> > >>>>>> can help, and that may be true, but that probably still
>>>> > >>>>>> assumes
>>>> > >>>>> consistently
>>>> > >>>>>> and correctly configured (and well-behaving) devices. And
>>>> > >>>>>> Section
>>>> > >>>>> 3.3.2 is
>>>> > >>>>>> vague and mentions a "infinite variety of possible failures"
>>>> > >>>>>> without
>>>> > >>>>> stating
>>>> > >>>>>> any requirements or recommendations. There may be further
>>>> > solutions,
>>>> > >>>>> such as
>>>> > >>>>>> circuit breakers and the like. Why are such topics not discussed?
>>>> > >>>>>>
>>>> > >>>>>> * Somewhat related, the document only looks at impact of
>>>> > >>>>>> failures
>>>> > to
>>>> > >>>>> the QoS of
>>>> > >>>>>> DetNet traffic. What is missing is a discussion how to
>>>> > >>>>>> protect
>>>> > >>>>>> non-
>>>> > >>>>> DetNet parts
>>>> > >>>>>> of a network from any harm caused by DetNet mechanisms.
>>>> > Solutions to
>>>> > >>>>> this
>>>> > >>>>>> probably exist. But why is the impact on non-DetNet traffic
>>>> > >>>>>> (e.g., in
>>>> > >>>>> case of
>>>> > >>>>>> topology changes or failures of DetNet functions) not
>>>> > >>>>>> discussed at
>>>> > >>>>> all in the
>>>> > >>>>>> document?
>>>> > >>>>>>
>>>> > >>>>>> * Regarding security, an architecture like DetNet probably
>>>> > >>>>>> requires
>>>> > >>>>> that only
>>>> > >>>>>> authenticated and authorized end systems have access to the
>>>> > >>>>>> data
>>>> > >>>>> plane. The
>>>> > >>>>>> security considerations only briefly mention the control
>>>> > >>>>>> aspect ("the authentication and authorization of the
>>>> > >>>>>> controlling systems").
>>>> > >>>>>>
>>>> > >>>>>> * For an architecture document, the lack of clarity and
>>>> > >>>>>> consistency
>>>> > >>>>> regarding
>>>> > >>>>>> terminology is concerning. This specifically applies to the
>>>> > >>>>>> case of
>>>> > >>>>> incomplete
>>>> > >>>>>> networks (as per Section 4.2.2 and 4.3.3) that include
>>>> > >>>>>> "DetNet-
>>>> > >>>>> unaware nodes".
>>>> > >>>>>> The document introduces terms such as "DetNet intermediate
>>>> > nodes"
>>>> > >>>>>> but
>>>> > >>>>> then
>>>> > >>>>>> repeatedly uses generic terms such as "node" or "hop" that
>>>> > >>>>>> may
>>>> > >>>>> include
>>>> > >>>>>> DetNet-unaware nodes. For instance, for incomplete networks,
>>>> > >>>>>> a
>>>> > >>>>> sentence such as
>>>> > >>>>>> "The primary means by which DetNet achieves its QoS
>>>> > >>>>>> assurances is
>>>> > to
>>>> > >>>>> reduce, or
>>>> > >>>>>> even completely eliminate, congestion within a node as a
>>>> > >>>>>> cause of
>>>> > >>>>> packet loss"
>>>> > >>>>>> seems to only apply to "DetNet transit nodes" but not
>>>> > >>>>>> "DetNet-unaware
>>>> > >>>>> nodes".
>>>> > >>>>>> Similar ambiguity exist for other use of the terms "hop" and
>>>> > >>>>>> "node",
>>>> > >>>>> which may
>>>> > >>>>>> or may not include DetNet-unaware nodes. It is unclear why
>>>> > >>>>>> the
>>>> > >>>>> document does
>>>> > >>>>>> not consistently use the terminology introduced in Section
>>>> > >>>>>> 2.1 in all
>>>> > >>>>> sections
>>>> > >>>>>> and clearly distinguishes cases with and without DetNet support.
>>>> > >>>>>>
>>>> > >>>>>> * Section 4.4 refers to RFC 7426, which is an informational
>>>> > >>>>>> RFC on
>>>> > >>>>> IRTF stream,
>>>> > >>>>>> and the document uses the concepts introduced there (e.g.,
>>>> > >>>>>> "planes").
>>>> > >>>>> This is
>>>> > >>>>>> very confusing. First, an IETF Proposed Standard should
>>>> > >>>>>> probably
>>>> > >>>>> refer to
>>>> > >>>>>> documents having IETF consensus. An example would be RFC
>>>> > >>>>>> 7491, albeit
>>>> > >>>>> there is
>>>> > >>>>>> other related work as well, e.g., in the TEAS WG. Second,
>>>> > >>>>>> Section
>>>> > >>>>>> 4.4
>>>> > >>>>> is by and
>>>> > >>>>>> large decoupled from the rest of the document and not
>>>> > >>>>>> specific to
>>>> > >>>>> DetNet.
>>>> > >>>>>> Neither do other sections of the document refer to the
>>>> > >>>>>> concepts
>>>> > >>>>> introduced in
>>>> > >>>>>> Section 4.4, nor does Section 4.4 use the DetNet terminology
>>>> > >>>>>> or
>>>> > >>>>> discuss
>>>> > >>>>>> applicability to DetNet. Section 4.4 even mentions explicitly
>>>> > >>>>>> at the
>>>> > >>>>> end that
>>>> > >>>>>> it discusses aspects that are orthogonal to the DetNet architecture.
>>>> > >>>>> It is not
>>>> > >>>>>> at all clear why Section 4.4 is in this document. Section 4.4
>>>> > >>>>>> could
>>>> > >>>>> be removed
>>>> > >>>>>> from the document without impacting the rest of the document.
>>>> > >>>>>>
>>>> > >>>>>> Minor issues:
>>>> > >>>>>>
>>>> > >>>>>> * Terminology "DetNet transport layer"
>>>> > >>>>>>
>>>> > >>>>>>     The term "transport layer" has a well-defined meaning in
>>>> > >>>>>> the IETF,
>>>> > >>>>> e.g.
>>>> > >>>>>>     originating from RFC 1122. While "transport" and e.g.
>>>> > >>>>>> "transport
>>>> > >>>>> network" is
>>>> > >>>>>>     used in the IETF for different technologies in different
>>>> > >>>>>> areas, I
>>>> > >>>>> think the
>>>> > >>>>>>     term "transport layer" is typically understood to refer
>>>> > >>>>>> to
>>>> > >>>>> transport
>>>> > >>>>>>     protocols such as TCP and UDP. As such, I personally find
>>>> > >>>>>> the term
>>>> > >>>>> "DetNet
>>>> > >>>>>>     transport layer" misleading and confusing. The confusion
>>>> > >>>>>> is easy
>>>> > >>>>> to see e.g.
>>>> > >>>>>>     in Figure 4, where UDP (which is a transport protocol as
>>>> > >>>>>> per RFC
>>>> > >>>>> 1122) sits
>>>> > >>>>>>     on top of "transport".
>>>> > >>>>>>
>>>> > >>>>>>     Based on the document it also may be
>>>> > >>>>>> solution/implementation
>>>> > >>>>> specific whether
>>>> > >>>>>>     the "DetNet transport layer" is actually a separate
>>>> > >>>>>> protocol layer
>>>> > >>>>> compared
>>>> > >>>>>>     to the "DetNet service layer". Thus it is not clear to me
>>>> > >>>>>> why the
>>>> > >>>>> word
>>>> > >>>>>>     "layer" has to be used, specifically in combination
>>>> > >>>>>> "transport
>>>> > >>>>> layer".
>>>> > >>>>>>     To me as, the word "transport layer" (and "transport
>>>> > >>>>>> protocol")
>>>> > >>>>> should be
>>>> > >>>>>>     used for protocols defined in TSV area, consistent with
>>>> > >>>>>> RFC 1122.
>>>> > >>>>> But this is
>>>> > >>>>>>     probably a question to be sorted out by the IESG.
>>>> > >>>>>>
>>>> > >>>>>> * Page 9
>>>> > >>>>>>
>>>> > >>>>>>      A DetNet node may have other resources requiring
>>>> > >>>>>> allocation
>>>> > >>>>> and/or
>>>> > >>>>>>      scheduling,
>>>> > >>>>>>
>>>> > >>>>>>     This is just one of several examples for inconsistent use
>>>> > >>>>>> of
>>>> > >>>>> terminology.
>>>> > >>>>>>     What is a "DetNet node"? That term is not introduced in
>>>> > >>>>>> Section
>>>> > >>>>> 2.1
>>>> > >>>>>> * Page 14
>>>> > >>>>>>
>>>> > >>>>>>      A DetNet network supports the dedication of a high
>>>> > >>>>>> proportion
>>>> > >>>>> (e.g.
>>>> > >>>>>>      75%) of the network bandwidth to DetNet flows.
>>>> > >>>>>>
>>>> > >>>>>>     The 75% value is not reasoned. What prevents using 99% of
>>>> > >>>>>> the
>>>> > >>>>> bandwidth for
>>>> > >>>>>>     DetNet traffic?
>>>> > >>>>>>
>>>> > >>>>>> * Page 15: Figure 2
>>>> > >>>>>>
>>>> > >>>>>>     If the term "transport layer" cannot be avoided, the
>>>> > >>>>>> labels in
>>>> > >>>>> this figure
>>>> > >>>>>>     should at least be expanded to "DetNet transport layer".
>>>> > >>>>>>
>>>> > >>>>>> * Page 18: Figure 4
>>>> > >>>>>>
>>>> > >>>>>>     As already mentioned earlier, Figure 4 is confusing. UDP
>>>> > >>>>>> is a
>>>> > >>>>> transport
>>>> > >>>>>>     protocol. If the term "transport" cannot be avoided, the
>>>> > >>>>>> labels in
>>>> > >>>>> this
>>>> > >>>>>>     figure should at least be expanded to "DetNet transport".
>>>> > >>>>>>
>>>> > >>>>>> * Page 23
>>>> > >>>>>>
>>>> > >>>>>>      If the source transmits less data than this limit
>>>> > >>>>>>      allows, the unused resource such as link bandwidth can
>>>> > >>>>>> be made
>>>> > >>>>>>      available by the system to non-DetNet packets.
>>>> > >>>>>>
>>>> > >>>>>>     Could there be additional requirements on the use of
>>>> > >>>>>> unused
>>>> > >>>>> resources by
>>>> > >>>>>>     non-DetNet packets, e.g., regarding preemption? I am just
>>>> > >>>>> wondering... If
>>>> > >>>>>>     that was possible, a statement like "... can be made
>>>> > >>>>>> available by
>>>> > >>>>> the system
>>>> > >>>>>>     to non-DetNet packets as long as all guarantees are fulfilled"
>>>> > >>>>> would be on
>>>> > >>>>>>     the safe side, no?
>>>> > >>>>>>
>>>> > >>>>>> * Page 27:
>>>> > >>>>>>
>>>> > >>>>>>      DetNet achieves congestion protection and bounded
>>>> > >>>>>> delivery
>>>> > >>>>> latency by
>>>> > >>>>>>      reserving bandwidth and buffer resources at every hop
>>>> > >>>>>> along the
>>>> > >>>>> path
>>>> > >>>>>>      of the DetNet flow.
>>>> > >>>>>>
>>>> > >>>>>>     Why does this sentence use the word "hop"? As far as I
>>>> > >>>>>> understand,
>>>> > >>>>> in DetNet
>>>> > >>>>>>     bandwidth and buffer resources are reserved in each
>>>> > >>>>>> DetNet
>>>> > >>>>> intermediate node.
>>>> > >>>>>>     If there were hops over IP routers not being DetNet
>>>> > >>>>>> intermediate
>>>> > >>>>> nodes, no
>>>> > >>>>>>     resources would be reserved there. As per Section 4.3.3,
>>>> > >>>>>> it is
>>>> > >>>>> possible to
>>>> > >>>>>>     deploy DetNet this way. And obviously there can be
>>>> > >>>>>> resource
>>>> > >>>>> bottlenecks below
>>>> > >>>>>>     IP, on devices that are not routers... So does "hop" here
>>>> > >>>>>> refer to
>>>> > >>>>> IP router
>>>> > >>>>>>     hops or also to devices not processing IP (or IP/MPLS)?
>>>> > >>>>>>
>>>> > >>>>>> * Page 27:
>>>> > >>>>>>
>>>> > >>>>>>      Standard queuing and transmission selection algorithms
>>>> > >>>>>> allow a
>>>> > >>>>>>      central controller to compute the latency contribution
>>>> > >>>>>> of each
>>>> > >>>>>>      transit node to the end-to-end latency, ...
>>>> > >>>>>>
>>>> > >>>>>>     The text does not explain why a _central_ controller is
>>>> > >>>>>> needed for
>>>> > >>>>> this
>>>> > >>>>>>     computation. Why would a distributed control plane not be
>>>> > >>>>>> able to
>>>> > >>>>> realize
>>>> > >>>>>>     this computation. Isn't this implementation-specific?
>>>> > >>>>>>
>>>> > >>>>>> * Page 32
>>>> > >>>>>>
>>>> > >>>>>>     To somebody who is not deeply familiar with DetNet, it is
>>>> > >>>>> impossible to parse
>>>> > >>>>>>     the description of the examples in Section 4.7.3. For
>>>> > >>>>>> instance,
>>>> > >>>>> "VID +
>>>> > >>>>>>     multicast MAC address" is not introduced. I think this
>>>> > >>>>>> example
>>>> > >>>>> must be
>>>> > >>>>>>     expaned with additional context and explanation to be
>>>> > >>>>>> useful to
>>>> > >>>>> readers.
>>>> > >>>>>> * Page 34
>>>> > >>>>>>
>>>> > >>>>>>      There are three classes of information that a central
>>>> > >>>>>> controller
>>>> > >>>>> or
>>>> > >>>>>>      distributed control plane needs to know that can only be
>>>> > >>>>>> obtained
>>>> > >>>>>>      from the end systems and/or nodes in the network.
>>>> > >>>>>>
>>>> > >>>>>>     Wouldn't it be sufficient to state "Provisioning of
>>>> > >>>>>> DetNet
>>>> > >>>>> requires knowledge
>>>> > >>>>>>     about ...". Does it matter in this context whether the
>>>> > >>>>> provisioning is done
>>>> > >>>>>>     by a central controller or a distributed control plane?
>>>> > >>>>>> For
>>>> > >>>>> instance, could
>>>> > >>>>>>     the same paragraph also apply to a network that uses
>>>> > >>>>>> _multiple_
>>>> > >>>>> central
>>>> > >>>>>>     controllers, or hybrid combinations of central
>>>> > >>>>>> controllers and
>>>> > >>>>> distributed
>>>> > >>>>>>     control planes? In general, an architecture document
>>>> > >>>>>> should be
>>>> > >>>>> agnostic to
>>>> > >>>>>>     implementation aspects unless there is a specific need.
>>>> > >>>>>> In this
>>>> > >>>>> specific
>>>> > >>>>>>     case, I fail to see a need to discuss the realization of
>>>> > >>>>>> the
>>>> > >>>>> control plane of
>>>> > >>>>>>     a network.
>>>> > >>>>>>
>>>> > >>>>>> Editorial nits:
>>>> > >>>>>>
>>>> > >>>>>> * Page 9:
>>>> > >>>>>>
>>>> > >>>>>>      The low-level mechanisms described in Section 4.5
>>>> > >>>>>> provide the
>>>> > >>>>>>      necessary regulation of transmissions by an end system
>>>> > >>>>>> or
>>>> > >>>>>>      intermediate node to provide congestion protection. The
>>>> > >>>>> allocation
>>>> > >>>>>>      of the bandwidth and buffers for a DetNet flow requires
>>>> > >>>>> provisioning
>>>> > >>>>>>      A DetNet node may have other resources requiring
>>>> > >>>>>> allocation
>>>> > >>>>> and/or
>>>> > >>>>>>      scheduling, that might otherwise be over-subscribed and
>>>> > >>>>>> trigger
>>>> > >>>>> the
>>>> > >>>>>>      rejection of a reservation.
>>>> > >>>>>>
>>>> > >>>>>>     Probably a full stop is missing after "provisioning".
>>>> > >>>>>>
>>>> > >>>>>> * Page 11: "... along separate (disjoint non-SRLG) paths ..."
>>>> > >>>>>>
>>>> > >>>>>>     I find this confusing. I would understand e.g. "along
>>>> > >>>>>> separate
>>>> > >>>>>>     (SRLG-disjoint) paths".
>>>> > >>>>>>
>>>> > >>>>>> * Page 34:
>>>> > >>>>>>
>>>> > >>>>>>      When using a peer-
>>>> > >>>>>>      to-peer control plane, some of this information may be
>>>> > >>>>>> required
>>>> > >>>>> by a
>>>> > >>>>>>      system's neighbors in the network.
>>>> > >>>>>>
>>>> > >>>>>>     Would "acquired" be a better term?
>>>> > >>>>>>
>>>> > >>>>>> * Page 34:
>>>> > >>>>>>
>>>> > >>>>>>      o  The identity of the system's neighbors, and the
>>>> > >>>>> characteristics of
>>>> > >>>>>>         the link(s) between the systems, including the length
>>>> > >>>>>> (in
>>>> > >>>>>>         nanoseconds) of the link(s).
>>>> > >>>>>>
>>>> > >>>>>>     "Latency" or "delay" would probably be a better terms if
>>>> > >>>>>> the value
>>>> > >>>>> is
>>>> > >>>>>>     measured in nanoseconds.
>>>> > >>>>>>
>>>> > >>>>>> * Page 35:
>>>> > >>>>>>
>>>> > >>>>>>      DetNet is provides a Quality of Service (QoS), and as
>>>> > >>>>>> such, does
>>>> > >>>>> not
>>>> > >>>>>>      directly raise any new privacy considerations.
>>>> > >>>>>>
>>>> > >>>>>>     Broken sentence
>>>> > >>>>>>
>>>> > >>>>>> * Please expand acronyms on first use (e.g., OTN)
>>>> > >>>>>>
>>>> > >>>>>>
>>>> > >>>> _______________________________________________
>>>> > >>>> detnet mailing list
>>>> > >>>> detnet@ietf.org
>>>> > >>>> https://www.ietf.org/mailman/listinfo/detnet
>>>> > >>
>>>> > >> _______________________________________________
>>>> > >> detnet mailing list
>>>> > >> detnet@ietf.org
>>>> > >> https://www.ietf.org/mailman/listinfo/detnet
>>>> > >
>>>> > >
>>>> >
>>>> > _______________________________________________
>>>> > detnet mailing list
>>>> > detnet@ietf.org
>>>> > https://www.ietf.org/mailman/listinfo/detnet
>>> _______________________________________________
>>> detnet mailing list
>>> detnet@ietf.org
>>> https://www.ietf.org/mailman/listinfo/detnet
>>
>>