Re: [Detnet] New Version Notification for draft-liu-detnet-large-scale-requirements-03.txt

[Toerless Eckert <[e-mail]>]

Hi Lou,

thanks a lot for the review!

On Fri, Oct 07, 2022 at 09:24:39AM -0400, Lou Berger wrote:
> Hi Peng,
>
> Thank you for the update.  I think the clarifications are quite helpful.  I
> still have a few comments:
> 
> 
> - I remain a bit uncomfortable the use of relative measures as a means to
> define "large-scale" - e.g, "The link delay between network nodes is
> larger..." - I think this is largely answered by section where specific
> technical requirements are defined.  I think it would be best if the
> description in the introduction (more) closely aligned with the requirement
> areas/topics called out in that section (4).

How about adding a pargraph like the following to the Intro to introduce
the term "large scale networks" ? Of course, we should strive to find
more accurate cutover points for all points as much as possible, but
hopefully we can do this as part of WG work (e.g.: links longer than 10
miles at 100Gbps to precise the link delay. TSN participant knowledge would
be handy here!).

| Large-scale networks in the context of this document are networks that
| require DetNet solutions for typically more than one of the requirements
| in this document: multiple administrative domains, link speed in excess of
| 10Gbps, no or relaxed clock synchronization, Links with large propagation
| latencies and possible jitter, support for large number of flows exceeding
| the ability to support per-hop, per-flow forwarding state and resiliency in
| the face of link/node and other failures as well as topology changes.

> - A more minor comment - While the reference to "Local area network" may be
> applicable to TSN, I think the current text mistakenly reads as if current
> DetNet only to "Local area network".

If you have specific places in the doc, let us know. Right now i can see
three places where local network / LAN are used, and it seems they could also
say "TSN network". Would that solve your point ?

> - You include high speed links as part of what what defines a "Large-scale"
> DetNet.  My understanding is that any rate link is supported via current
> DetNet definitions.  I do agree that number of flows can be used to identify
> a "large-scale" DetNet (independent of link speeds).

How about inseting this into 4.3:

| While the current DetNet architecture is not withstanding the use of
| higher speed links, the available and not DetNet standardized, mostly TSN
| derived dependent technology aspects are considered by this document
| to not be  sufficient to support higher speed links at low hardware cost.
| These aspects include the following.
|
| [existing]: With the increasing of the data rate, the network scheduling cycle

> - Similarly, I don't understand how asymmetric links are new to the
> large-scale problem, e.g., consider the RAW use case.

I have to admit i am not clear about the RAW use-case details right now.

Did you read speed asymmetry ? I think the text means propagation latency
asymmetry. That is a pain for all clock synchronization method.

Aka: we'll write propagation latency asymmetry. If there is more to your
point, pls. explain.

> I suspect there will be more refinement in the document besides the points
> above once this document is accepted as a WG document,but do think the above
> should be clarified before WG adoption so that the scope of the work is
> clear to all.
> 
> 
> Thank you,
> 
> Lou
> 
> 
> On 10/4/2022 6:31 AM, Peng Liu wrote:
> > Hi Lou,
> > 
> > Thanks. According to your suggestion, we modified the text as follows,
> > which have the definition and the key attributes of large-scale
> > deterministic network, and each requirements proposed in Section 4 could
> > map to the attributes directly or indirectly. We hope this could work.
> > The new version of the document and the diff could be found in
> > attachment.
> > 
> > In this document we define a large-scale DetNet network as a network
> > that has longer transmission distance, more network devices and more
> > kinds of traffic flows than a local area network, it includes one or
> > more of the following key attributes:
> > 
> >    * There are many types of network nodes which may introduce disparate
> > local time variation and buffering capabilities.
> >    * The link delay between network nodes is larger, and the end to end
> > path is a combination of short and long distance hops.
> >    * There are various transmission rate supported at the different
> > ports and on different network node, varying from tens of Mbps to tens
> > of Gbps or higher.
> >    * The path between two neighbor nodes can be asymmetric in terms of
> > the link delay.
> >    * There are a large number of flows which may travel across multiple
> > Detnet domains.
> >    * The mechanisms used to ensure bounded latency (e.g. queuing
> > mechanism) may be different or have different configuration/parameter in
> > each domain.
> > 
> > Such domains are normally within a single administrative control network
> > or multiple cooperating administrative networks within a closed group of
> > administrative control [RFC8655].  However they may belong to different
> > AS domains and be controlled by multiple peering or cascaded
> > controllers, and at the same time they may not have the same time clock
> > source. Maintaining per flow status becomes impractical in the large
> > scale network.  Aggregation and disaggregation of flows take place at
> > the boundaries of Detnet omains as well as within a Detnet domain with
> > various rules. The flow identification and packet treatment should take
> > care of one or combined changes introduced by the large-scale network.
> > 
> > Based on the defination and characteristics above, this document
> > describes requirements for large-scale deterministic networks which
> > demands the enhancement based on the existing bounded latency mechanisms
> > and the corresponding data plane to ensure the detnet service for single
> > administrative network or multiple (cooperating) administrative networks
> > that defined in the detnet charter. The deterministic network for open
> > internet is not within current scope.
> > 
> > Regards,
> > Peng
> > ------------------------------------------------------------------------
> > [e-mail]
> > 
> >     *From:* Lou Berger <mailto:[e-mail]>
> >     *Date:* 2022-09-30 20:32
> >     *To:* Peng Liu <mailto:[e-mail]>
> >     *CC:* detnet <mailto:[e-mail]>;
> >     draft-liu-detnet-large-scale-requirements
> >     <mailto:[e-mail]>
> >     *Subject:* Re: [Detnet] New Version Notification for
> >     draft-liu-detnet-large-scale-requirements-03.txt
> > 
> >     Hi Peng,
> > 
> >     Thank you for the response. I think your text below will help, but
> >     still does not provide a precise definition of the term
> >     'large-scale'. I think such a precise definition is required to
> >     ensure we all understand the problem being described.
> > 
> > 
> >     I suggest having text that starts with something along the lines
> >     of "In this document we define a large-scale DetNet network as a
> >     network that includes one or more of the following...." and then
> >     expanding a bit on each of the identified key attributes.
> > 
> > 
> >     Thank you!
> > 
> >     Lou
> > 
> > 
> >     On 9/27/2022 4:35 AM, Peng Liu wrote:
> > >     Hi Lou,
> > > 
> > >     Thanks for your comments. We think the large-scale deterministic
> > >     network aims at the wide area network, which may also has more
> > >     network devices, detnet flows and cross multi domains. Those
> > >     characteristics bring more challenges to ensure the detnet service.
> > > 
> > >     As you said, the current version is not clear about what
> > >     large-scale is, and it is inaccurate to say like 'detnet is
> > >     limited to a single network domain'. So we propose the
> > >     modification at the last paragraphs in the Introduction Section:
> > > 
> > >     A large-scale network usually has a longer transmission distance,
> > >     more network devices and more kinds of traffic flows than a Local
> > >     area network. It also has the characteristics including:
> > > 
> > >     The link delay between network nodes is larger. The end to end
> > >     path is a combination of short and long distance hops. The path
> > >     between two neighbor nodes can be asymmetric in terms of the link
> > >     delay. There are many types of network nodes which may introduce
> > >     disparate local time variation and buffering capabilities. At the
> > >     same time, there are various transmission rate supported at the
> > >     different ports and on different network node, varying from tens
> > >     of Mbps to tens of Gbps or higher. A large number of flows can
> > >     travel across multiple Detnet domains.
> > > 
> > >     Such domains are normally within a single administrative control
> > >     network or multiple cooperating administrative networks within a
> > >     closed group of administrative control [RFC8655]. However
> > >     they may belong to different AS domains and be controlled by
> > >     multiple peering or cascaded controllers, and at the same time
> > >     they may not have the same time clock source. Maintaining
> > >     per flow status becomes impractical in the large scale network.
> > >     Aggregation and disaggregation of flows take place at the
> > >     boundaries of Detnet domains as well as within a Detnet domain
> > >     with various rules. The flow identification and packet treatment
> > >     should take care of one or combined changes introduced by the
> > >     large-scale network.
> > > 
> > > 
> > >     This document describes requirements for large-scale
> > >     deterministic networks which demands the enhancement based on the
> > >     existing bounded latency mechanisms and the corresponding data
> > >     plane to ensure the detnet service for single administrative
> > >     network or multiple (cooperating) administrative networks that
> > >     defined in the detnet charter. The deterministic network for open
> > >     internet is not within current scope.
> > > 
> > > 
> > >     Moreover, we will remove/refine the uncorrect description related
> > >     to the multi domains in the whole text.
> > > 
> > >     Hope that could address the issues about the 'large-scale' and
> > >     'multi domain'.
> > > 
> > >     Thanks again~
> > >     Peng&co authors
> > >     ------------------------------------------------------------------------
> > >     [e-mail]
> > > 
> > >         *From:* Lou Berger <mailto:[e-mail]>
> > >         *Date:* 2022-09-24 00:52
> > >         *To:* Peng Liu <mailto:[e-mail]>;
> > >         draft-liu-detnet-large-scale-requirements
> > >         <mailto:[e-mail]>
> > >         *CC:* DetNet WG <mailto:[e-mail]>
> > >         *Subject:* Re: [Detnet] New Version Notification for
> > >         draft-liu-detnet-large-scale-requirements-03.txt
> > >         Authors,
> > > 
> > >         Thank you for this update. I have a few comments on this draft:
> > > 
> > >         - The first question I have is what is the documents
> > >         definition of "large-scale deterministic network"?
> > > 
> > >         In the abstract you seems to imply that "large-scale"
> > >         translates to "wide area":
> > >            ... this document
> > >            describes the technical and operational requirements when the
> > >            different deterministic levels of applications co-exist
> > >         and are
> > >            transported over a wide area.
> > > 
> > >         But the intro, you seem to state that "large-scale"
> > >         translates to "multiple domains":
> > > 
> > >           ... The current DetNet is limited to a
> > >            single administrative domain network, and there are technical
> > >            elements necessary for application to a large-scale
> > >         network spanning
> > >            multiple domains.
> > > 
> > >            This document describes requirements for large-scale
> > >         deterministic
> > >            networks where different deterministic levels of
> > >         applications co-
> > >            exist and large-scale deterministic networking across multiple
> > > 
> > >         administrative domains is possible.
> > > 
> > > 
> > >         Can you clarify as well as ensure you are consistent within
> > >         the document?
> > > 
> > > 
> > >         FWIW, based on the later sections of the document, I suspect
> > >         large-scale is used in the document to cover number of flows
> > >         supported on a device and/or network, as well as longer
> > >         latency of a link and/or across the detnet domain. No matter
> > >         what, the document really needs to be explicit.
> > > 
> > > 
> > >         - The following implies that detnet works only within a
> > >         single network domain.  I think it would be good to clarify
> > >         how you define "single *network* domain" in the context of
> > >         this document.
> > > 
> > >            DetNet, of which architecture is defined in RFC 8655
> > >         [RFC8655],
> > >            provides a capability to carry specified unicast or
> > >         multicast data
> > >            flows for real-time applications with extremely low data
> > >         loss rates
> > >            and bounded latency within a network domain.
> > > 
> > >         Note that the charter says multiple (cooperating) networks
> > >         that have different administrative control was part of the
> > >         scope of initial detnet definition, and RFC  8655 defines a
> > >         "DetNet domain" that is independent from areas/groups/domains
> > >         of administrative control.
> > >         - I think the following should be aligned with however you
> > >         address the previous comment:
> > >            .. As documented in RFC 8578 [RFC8578],
> > >            the scope of networks addressed by the current DetNet is
> > >         limited to
> > >            networks that can be centrally controlled, i.e., an
> > >         "enterprise" (aka
> > >            "corporate") network, excluding "the open Internet,"
> > >         explicitly.
> > > 
> > >         RFC 8578 defines *example* use cases.  The appropriate
> > >         reference for current DetNet *solution* scope is RFC8655 or
> > >         the other solution oriented documents.  DetNet solutions are
> > >         not currently limited to central control, enterprise, or
> > >         corporate networks.  I do agree that the open internet is not
> > >         within current scope.
> > > 
> > >            ..The current DetNet is limited to a
> > >            single administrative domain network, and there are technical
> > >            elements necessary for application to a large-scale
> > >         network spanning
> > >            multiple domains.
> > > 
> > > 
> > >         As previously noted, the current definition of DetNet is
> > >         limited to networks that are under a single administrative
> > >         control or within a closed group of administrative control.
> > > 
> > > 
> > >         I think clarifying these points would be very helpful (and
> > >         are necessary) to evaluate the remainder of the document.
> > > 
> > > 
> > > 
> > >         Thank you,
> > > 
> > >         Lou (as contributor)
> > > 
> > > 
> > >         On 8/17/2022 4:50 AM, Peng Liu wrote:
> > > >         Dear All,
> > > > 
> > > >         We have posted the new version of
> > > >         draft-liu-detnet-large-scale-requirements-03
> > > >         <https://datatracker.ietf.org/doc/draft-liu-detnet-large-scale-requirements/> based
> > > >         on the discussions and suggestions, and we would like to
> > > >         thanks Bala'zs, Tianran and Fan for their comments. Here are
> > > >         the the major changes:
> > > > 
> > > >         1. Add some descriptions of the requirements
> > > >         and clarifications not to specify solutions in Section 4.
> > > > 
> > > >         2. Change the name of Section 5.2 as "Support Information
> > > >         used by Functions ensuring Deterministic Latency".
> > > > 
> > > >         3. Add some clarifications text in Section 5.1 and Section 5.2.
> > > > 
> > > >         4. Add Shiyin Zhu as the co author and Lei Zhou as the
> > > >         contributor.
> > > > 
> > > >         5. Correct some formatting and spelling errors.
> > > > 
> > > >         Most of the solution draft presented in IETF 114 reference
> > > >         this document as the base requirement draft. We tried to put
> > > >         the common requirements together and made some updates here.
> > > >         There were around 20 people showing they had read the
> > > >         document during 114 session polling. We think it is a
> > > >         reasonable number to use this draft as the starting point
> > > >         for the milestone of Enhanced DetNet Data Plane Requirements
> > > >         document.
> > > > 
> > > >         So we encourage the group to review the document. We plan to
> > > >         ask for a call for adoption maybe next month upon chairs’
> > > >         agreement.
> > > > 
> > > > 
> > > >         Regards,
> > > > 
> > > >         Peng&co authors
> > > > 
> > > >         ------------------------------------------------------------------------
> > > >         [e-mail]
> > > > 
> > > >             *From:* internet-drafts <mailto:[e-mail]>
> > > >             *Date:* 2022-08-17 15:51
> > > >             *To:* Jeong-dong Ryoo <mailto:[e-mail]>; Peng Liu
> > > >             <mailto:[e-mail]>; Quan Xiong
> > > >             <mailto:[e-mail]>; Shiyin Zhu
> > > >             <mailto:[e-mail]>; Toerless Eckert
> > > >             <mailto:[e-mail]>; Yizhou Li
> > > >             <mailto:[e-mail]>; zhushiyin
> > > >             <mailto:[e-mail]>
> > > >             *Subject:* New Version Notification for
> > > >             draft-liu-detnet-large-scale-requirements-03.txt
> > > >             A new version of I-D,
> > > >             draft-liu-detnet-large-scale-requirements-03.txt
> > > >             has been successfully submitted by Peng Liu and posted
> > > >             to the
> > > >             IETF repository.
> > > >             Name: draft-liu-detnet-large-scale-requirements
> > > >             Revision: 03
> > > >             Title: Requirements for Large-Scale Deterministic Networks
> > > >             Document date: 2022-08-17
> > > >             Group: Individual Submission
> > > >             Pages: 18
> > > >             URL:
> > > >             https://www.ietf.org/archive/id/draft-liu-detnet-large-scale-requirements-03.txt
> > > >             Status:
> > > >             https://datatracker.ietf.org/doc/draft-liu-detnet-large-scale-requirements/
> > > >             Htmlized:
> > > >             https://datatracker.ietf.org/doc/html/draft-liu-detnet-large-scale-requirements
> > > >             Diff:
> > > >             https://www.ietf.org/rfcdiff?url2=draft-liu-detnet-large-scale-requirements-03
> > > >             Abstract:
> > > >                Aiming at the large-scale deterministic network, this
> > > >             document
> > > >                describes the technical and operational requirements
> > > >             when the
> > > >                different deterministic levels of applications
> > > >             co-exist and are
> > > >                transported over a wide area.  This document also
> > > >             describes the
> > > >                corresponding Deterministic Networking (DetNet) data
> > > >             plane
> > > >                enhancement requirements.
> > > >             The IETF Secretariat
> > > > 
> > > > 
> > > >         _______________________________________________
> > > >         detnet mailing list
> > > >         [e-mail]
> > > >         https://www.ietf.org/mailman/listinfo/detnet
> > > 

-- 
---
[e-mail]