Re: [Tsv-art] [Detnet] Tsvart last call review of draft-ietf-detnet-architecture-08

["Scharf, Michael" <[e-mail]>]

> On Thu, Nov 15, 2018 at 01:18:09PM +0000, Scharf, Michael wrote:
> > My comments are about the question what happens if DetNet packets get
> dropped despite that this should _never_ happen in normal operation.
> There can be different reasons for packet drops in a managed network.
> One root cause would be that the DetNet traffic gets routed erroneously
> over a DetNet unaware node somewhere in the Internet and then causes
> harm there.
> 
> Understood.
> 
> > For inelastic traffic, if such packet loss persists, that flow (at
> least on the path where losses occur) may need to be blocked for safety
> reasons. I assume that is what you suggest in your last sentence?
> 
> Yes. Of course, i can not speak for the authors, and i don't think so
> far this type of detail for the  PREOF functions have been required,
> but i certainly think it would make the PEF/POF a lot more useful if it
> would include diagnostics about packet loss - and that could then be
> used to trigger some control/management-plane circuit breaker.

Thanks a lot for the clarification. Such a safety mechanism would address a lot of my concerns.

Michael


> 
> Cheers
>     Toerless
> 
> > Michael
> >
> >
> > -----Original Message-----
> > From: Toerless Eckert [mailto:[e-mail]]
> > Sent: Thursday, November 15, 2018 4:57 AM
> > To: Scharf, Michael
> > Cc: Black, David; Lou Berger; 'J?nos Farkas'; [e-mail];
> DetNet WG; [e-mail]
> > Subject: Re: [Detnet] [Tsv-art] Tsvart last call review of draft-
> ietf-detnet-architecture-08
> >
> > On Fri, Nov 09, 2018 at 06:38:30AM +0000, Scharf, Michael wrote:
> > > Indeed.
> > >
> > > And, as noted in my earlier way, depending on how the PRF is
> realized DetNet may even shield DetNet traffic from packet losses due
> to congestion, i.e., even traffic that would be able to react to
> congestion may not react due to DetNet. If DetNet is able to mask
> congestion from the carried traffic, special care is needed inside
> DetNet.
> >
> > I don't think we should complicate the matters by talking about
> DetNet
> > flows that may react to congestion. I don't think this is in scope of
> > the current architecture.
> >
> > AFAIK, all DetNet flows have a known envelope, and DetNet flows
> should
> > be shutdown (filtered/blocked) by the DetNet service whenever they
> exceed
> > the envelope (*).
> >
> > Within the envelope of DetNet flows, there is no useful support
> > for congestion loss, because congestion constitutes a
> misconfiguration
> > or misbehavior of the DetNet service.
> >
> > PEF/POF could actually act as a circuit breaker using the maximum
> > permissible loss of the envelope as a trigger to make the PRF stop
> > the flow. That would be a good feature for the PREOF.
> >
> > Cheers
> >     Toerless
> >
> > (*) I am not sure if DetNet intends to make such a strong
> > statement, but from what i heard, this is usually done in TSN
> > applications:
> >
> > If you have an interlocked system and a sender exceeds its
> > envelope, that indicates a malfunctioning sender, and there is
> > no way to design a safe&secure system with randomnly
> > malfunctioning components. Shutting down a component by
> > stopping its traffic is a supported system behavior resulting
> > typically in bringing up redundant components or shutting down
> > parts of the system. Unknown component behavior results in
> > unpredictable system behavior, often catastrophic.
> >
> > > > That quoted comment is followed by a "reduce to previous case"
> argument
> > > > about networking in general:
> > > >
> > > > > > WRT escaped traffic, as I mentioned in today's session, I
> really
> > > > think
> > > > > > this is no difference between escaped DetNet traffic and
> traffic
> > > > > > delivered over an uncongested network to a peer (congested or
> > > > > > uncongested) network.
> > > >
> > > > Unfortunately, that argument leads to a contradiction, namely a
> > > > conclusion that congestion control is not necessary for the
> Internet,
> > > > which is known to be wrong.  Having arrived at a result that
> > > > mathematicians would call "proof by contradiction," one of the
> > > > assumptions that lead to this point has to be wrong, but which
> one?
> > > >
> > > > The "clear, simple and wrong" (Mencken) approach to resolving
> this
> > > > would be to focus on the inelastic traffic and require that all
> DetNet
> > > > traffic MUST be congestion controlled.  I'm not advocating this,
> and
> > > > neither is Michael.
> > > >
> > > > A more subtle approach (with credit to Deb Brungard for an
> insightful
> > > > discussion after RTGAREA yesterday) is to recognize that DetNet
> is a
> > > > Controlled Environment, to which different rules apply than the
> > > > Internet and networks in general.  For the definition of
> Controlled
> > > > Environment, see Section 3.6 of RFC 8085
> > > > (https://tools.ietf.org/html/rfc8085#section-3.6).  For now,
> please
> > > > ignore the material in that section on safety, robustness and
> > > > corruption, as inapplicable to this discussion.  FWIW, that
> material
> > > > was motivated by the incessant attempts to optimize out UDP
> checksums
> > > > without serious consideration of the resulting packet corruption
> risks
> > > > and the potential consequences thereof.  Viewing DetNet as a
> Controlled
> > > > Environment, this discussion is then primarily about what to
> require/do
> > > > at the boundary between the DetNet Controlled Environment and
> other
> > > > networks.  I've been there, done that for other Controlled
> Environments
> > > > ... and have the scars ... from the worked examples in RFC 7510
> > > > (MPLS/UDP) and RFC 8086's TMCE (Traffic Management Controlled
> > > > Environment) applicability scenario for GRE/UDP.  Those two RFCs
> may be
> > > > helpful background reading.
> > > >
> > > > The good news here is that we are not as far from agreeing on a
> > > > solution as might appear from the length of this discussion.
> Going
> > > > back to a couple of chunks of text that Janos proposed earlier:
> > > >
> > > > 	NEW:
> > > > 	This document provides the overall Architecture for
> Deterministic
> > > > 	Networking (DetNet), which provides a capability for the
> delivery
> > > > of
> > > > 	data flows with extremely low packet loss rates and bounded
> end-
> > > > to-
> > > > 	end delivery latency within a network domain. DetNet is for
> > > > networks
> > > > 	that are under a single administrative control or within a
> closed
> > > > group
> > > > 	of administrative control; these include campus-wide
> networks and
> > > > 	private WANs. DetNet is not for large groups of domains
> such as
> > > > the
> > > > 	Internet.
> > > >
> > > > That reads to me as a clear statement that a DetNet network is a
> > > > Controlled Environment, so I would hope that we're in agreement
> on
> > > > that.
> > > >
> > > > 	NEW:
> > > > 	Robust real-time systems require to reduce the number of
> possible
> > > > 	failures. Filters and policers should be used in a DetNet
> network
> > > > to
> > > > 	detect if DetNet packets are received on the wrong
> interface, or
> > > > at
> > > > 	the wrong time, or in too great volume. Furthermore,
> filters and
> > > > 	policers can take action to discard offending packets or
> flows,
> > > > or
> > > > 	trigger shutting down the offending DetNet flow, or
> shutting down
> > > > 	the offending interface.
> > > >
> > > > That proposed text actually contains two circuit breakers,
> "shutting
> > > > down the offending DetNet flow" or "shutting down the offending
> > > > interface."  So far, so good, but what's missing is text on
> deployment
> > > > requirements of the mechanisms described in that text,
> particularly at
> > > > the edges of DetNet networks.
> > >
> > > Yep, strict deployment requirements are needed.
> > >
> > > > Text that requires deployment of "filters and policers ... [that]
> > > > discard offending [DetNet] packets and flows" at the boundaries
> between
> > > > DetNet and non-DetNet networks would go a long way towards
> resolving
> > > > this discussion.  I would suggest expanding "offending" into text
> that
> > > > includes all unauthorized DetNet traffic and all unexpected
> DetNet
> > > > traffic.  Once that's done, the "circuit breaker" discussion
> becomes
> > > > more tractable, as the discarding of all DetNet traffic that
> would
> > > > otherwise escape the DetNet network reduces the "circuit breaker"
> > > > discussion to the existing DetNet material measures that need to
> be
> > > > taken inside a DetNet network to protect delivery of DetNet
> service.
> > >
> > > As already mentioned, it is possible that some solutions don't need
> any "circuit breaker" as they achieve sufficient robustness by some
> other means. I only ask for text that explains how to avoid harm from
> non-DetNet networks, not for a specific solution.
> > >
> > > Michael
> > >
> > >
> > > >
> > > > Thanks, --David
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Scharf, Michael [mailto:[e-mail]]
> > > > > Sent: Thursday, November 8, 2018 1:12 PM
> > > > > To: Lou Berger; Black, David; 'János Farkas'
> > > > > Cc: [e-mail]; DetNet WG; draft-ietf-detnet-
> > > > [e-mail]
> > > > > Subject: RE: [Tsv-art] [Detnet] Tsvart last call review of
> draft-
> > > > ietf-detnet-
> > > > > architecture-08
> > > > >
> > > > >
> > > > > [EXTERNAL EMAIL]
> > > > > Please report any suspicious attachments, links, or requests
> for
> > > > sensitive
> > > > > information.
> > > > >
> > > > >
> > > > > Inline
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Tsv-art [mailto:[e-mail]] On Behalf Of
> Lou
> > > > Berger
> > > > > > Sent: Thursday, November 8, 2018 8:45 AM
> > > > > > To: Black, David; Scharf, Michael; 'János Farkas'
> > > > > > Cc: [e-mail]; DetNet WG; draft-ietf-detnet-
> > > > > > [e-mail]
> > > > > > Subject: Re: [Tsv-art] [Detnet] Tsvart last call review of
> draft-
> > > > ietf-
> > > > > > detnet-architecture-08
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > See below for in-line responses
> > > > > >
> > > > > > On 11/5/2018 12:21 AM, Black, David wrote:
> > > > > > > ...
> > > > > >
> > > > > > >>> * It is surprising that there is hardly any discussion on
> > > > network
> > > > > > robustness
> > > > > > >>> and safety; this probably also relates to security. For
> > > > instance,
> > > > > > >>> misconfiguration or errors of functions performing packet
> > > > > > replication could
> > > > > > >>> severely and permanently congest a network and cause
> harm.
> > > > > > > [... snip ...]
> > > > > > >> NEW:
> > > > > > >> Robust real-time systems require to reduce the number of
> > > > possible
> > > > > > >> failures. Filters and policers should be used in a DetNet
> > > > network to
> > > > > > >> detect if DetNet packets are received on the wrong
> interface, or
> > > > at
> > > > > > >> the wrong time, or in too great volume. Furthermore,
> filters and
> > > > > > >> policers can take action to discard offending packets or
> flows,
> > > > or
> > > > > > >> trigger shutting down the offending DetNet flow, or
> shutting
> > > > down
> > > > > > >> the offending interface.
> > > > > > >>
> > > > > > >> [ms] That does not address my concern. As I wrote already,
> I
> > > > believe
> > > > > > this
> > > > > > >> document has to discuss the applicability of circuit
> breakers
> > > > > > according to RFC
> > > > > > >> 8084.
> > > > > > > I concur with Michael's implied concern that the "should"
> in the
> > > > NEW
> > > > > > text
> > > > > > > is woefully inadequate.  The provisions to prevent escaped
> non-
> > > > > > elastic traffic
> > > > > > > from causing damage elsewhere need to be mandatory
> particularly
> > > > if
> > > > > > DetNet
> > > > > > > and non-DetNet networks are interconnected.  The above
> > > > > > > text on filters doesn't appear to cover escape from the
> DetNet
> > > > > > network,
> > > > > > > and needs to do so.  Deployment of filters at DetNet nodes
> that
> > > > drop
> > > > > > all
> > > > > > > DetNet traffic that tries to escape, and that drop all
> > > > misdirected
> > > > > > DetNet
> > > > > > > traffic should address most of this problem.
> > > > > > >
> > > > > > > Once that is done, I think a discussion of circuit breakers
> is
> > > > > > needed, in the
> > > > > > > form of Detnet service mechanisms that can shut down (cease
> > > > > > transmission
> > > > > > > of) a flow that is being sent entirely into the bit-bucket
> by a
> > > > > > downstream filter,
> > > > > > > due to misconfiguration, failure, etc.
> > > > > >
> > > > > > I think we have a bit of a disconnect of what DetNet service
> > > > delivers.
> > > > > >
> > > > > > DetNet does not itself impact any Transport protocol  (e.g.,
> UDP,
> > > > TCP)
> > > > > > so itself does not *produce* elastic or non-elastic traffic.
> > > > >
> > > > > First, the question whether DetNet *produces* elastic or non-
> elastic
> > > > traffic does
> > > > > not matter for the applicability of circuit breakers. Section 3
> of
> > > > RFC 8084 states:
> > > > >
> > > > >    CBs are RECOMMENDED for IETF protocols and tunnels that
> carry non-
> > > > >    congestion-controlled Internet flows and for traffic
> aggregates.
> > > > >    This includes traffic sent using a network tunnel.
> Designers of
> > > > >    other protocols and tunnel encapsulations also ought to
> consider
> > > > the
> > > > >    use of these techniques as a last resort to protect traffic
> that
> > > > >    shares the network path being used.
> > > > >
> > > > > Section 3.2.1.1 of draft-ietf-detnet-architecture-09 explains
> > > > explicitly that
> > > > > DetNet can carry non-congestion-controlled traffic:
> > > > >
> > > > >    Note that a DetNet flow cannot be throttled, i.e., its
> > > > >    transmission rate cannot be reduced via explicit congestion
> > > > >    notification.
> > > > >
> > > > > So, clearly the condition of carrying non-congestion-controlled
> flows
> > > > or traffic
> > > > > aggregates applies to the DetNet architecture. And, as a
> result, the
> > > > > RECOMMENDED in RFC 8084 is in principle applicable to DetNet.
> Also,
> > > > note that
> > > > > the RECOMMENDED in RFC 8084 is not limited to tunnels.
> > > > >
> > > > > As pointed out by David, the key question is whether, and how,
> the
> > > > DetNet
> > > > > architecture prevents that non-congestion controlled traffic
> escapes
> > > > from a
> > > > > controlled environment into the Internet. As also explained in
> RFC
> > > > 8084, this
> > > > > question in particular matters for failure scenarios.
> > > > >
> > > > > As TSV-ART reviewer, I insist in a discussion of this problem
> in
> > > > draft-ietf-detnet-
> > > > > architecture. And I believe that a reference to RFC 8084 is
> needed in
> > > > draft-ietf-
> > > > > detnet-architecture.
> > > > >
> > > > > Note that the wording in RFC 8084 is RECOMMENDED and not MUST,
> i.e.,
> > > > it is
> > > > > possible that circuit breakers may not be needed for certain
> > > > solutions. As an
> > > > > example, MPLS traffic may not easily escape from a managed MPLS
> > > > network.
> > > > >
> > > > >
> > > > > Second, in my TSV-ART review I have explicitly mentioned the
> DetNet
> > > > Packet
> > > > > Replication Function (PRF). Replicating non-elastic traffic may
> > > > create additional
> > > > > challenges if the replicated packets can somehow escape into
> the
> > > > Internet, e.g.,
> > > > > in failure cases. For instance, as far as I understand draft-
> ietf-
> > > > detnet-
> > > > > architecture, I assume the PRF can replicate non-congestion-
> > > > controlled traffic
> > > > > and forward it along different paths. If one of these paths
> escapes
> > > > to the
> > > > > Internet, to the Internet the PRF could basically be a producer
> of
> > > > non-congestion
> > > > > controlled traffic, and that traffic could cause significant
> harm in
> > > > the Internet.
> > > > > Also, note that in that case it may not necessarily help if the
> > > > traffic carried by
> > > > > DetNet is elastic, e.g., if DetNet carries TCP traffic. For
> instance,
> > > > if other paths
> > > > > used by the PRF within the DetNet domain would deliver all
> packets,
> > > > the
> > > > > congestion control in TCP would not never be triggered as each
> packet
> > > > gets
> > > > > indeed delivered over one of the paths. So, an unelastic
> traffic
> > > > stream would
> > > > > escale into the Internet and never back off. Unless I miss
> something,
> > > > DetNet
> > > > > needs counter-means to prevent this sort of issues.
> > > > >
> > > > > As TSV-ART reviewer, I believe draft-ietf-detnet-architecture
> needs a
> > > > dedicated
> > > > > discussion whether the PRF can cause harm and, if so, how to
> avoid
> > > > such harm.
> > > > >
> > > > > > DetNet operates at Layer 3 and basically provides detnet
> flows with
> > > > a
> > > > > > service equivalent to IntServ's Guaranteed Quality of
> Service.  As
> > > > > > such,
> > > > > > DetNet flows basically operate as if the network is
> uncongested.
> > > > >
> > > > > RFC 8084 exactly addresses that case. The question is not the
> normal
> > > > operation
> > > > > of the network, but abnormal cases.
> > > > >
> > > > > > WRT escaped traffic, as I mentioned in today's session, I
> really
> > > > think
> > > > > > this is no difference between escaped DetNet traffic and
> traffic
> > > > > > delivered over an uncongested network to a peer (congested or
> > > > > > uncongested) network.
> > > > >
> > > > > That comes down to how the DetNet network deals with abnormal
> > > > situations at
> > > > > the edges, e.g., for the problems mentioned in RFC 8084:
> > > > >
> > > > >    o  anomalous traffic that exceeds the provisioned capacity
> (or
> > > > whose
> > > > >       traffic characteristics exceed the threshold configured
> for the
> > > > >       CB);
> > > > >
> > > > >    o  traffic generated by an application at a time when the
> > > > provisioned
> > > > >       network capacity is being utilized for other purposes;
> > > > >
> > > > >    o  routing changes that cause additional traffic to start
> using
> > > > the
> > > > >       path monitored by the CB;
> > > > >
> > > > >    o  misconfiguration of a service/network device where the
> capacity
> > > > >       available is insufficient to support the current traffic
> > > > >       aggregate;
> > > > >
> > > > >    o  misconfiguration of an admission controller or traffic
> policer
> > > > >       that allows more traffic than expected across the path
> > > > monitored
> > > > >       by the CB.
> > > > >
> > > > > According to the last point, misconfiguration of
> filters/policies is
> > > > one of the
> > > > > challenges to look at.
> > > > >
> > > > > > Based on all this, I'm not sure how or why a  discussion of
> circuit
> > > > > > breakers is needed.
> > > > >
> > > > > RFC 8084 represents IETF community consensus. And in my reading
> the
> > > > DetNet
> > > > > architecture falls into the type of solutions discussed in RFC
> 8084.
> > > > >
> > > > > Thanks
> > > > >
> > > > > Michael
> > > > >
> > > > >
> > > > > >
> > > > > > Lou
> > > > > >
> > > > > > >
> > > > > > > Thanks, --David
> > > > > > >
> > > > > > >> -----Original Message-----
> > > > > > >> From: Tsv-art [mailto:[e-mail]] On Behalf
> Of
> > > > Scharf,
> > > > > > Michael
> > > > > > >> Sent: Monday, October 22, 2018 5:44 PM
> > > > > > >> To: 'János Farkas'
> > > > > > >> Cc: [e-mail]; DetNet WG; draft-ietf-detnet-
> > > > > > [e-mail]
> > > > > > >> Subject: Re: [Tsv-art] Tsvart last call review of draft-
> ietf-
> > > > detnet-
> > > > > > architecture-08
> > > > > > >>
> > > > > > >>
> > > > > > >> [EXTERNAL EMAIL]
> > > > > > >> Please report any suspicious attachments, links, or
> requests for
> > > > > > sensitive
> > > > > > >> information.
> > > > > > >>
> > > > > > >>
> > > > > > >> Hi Janos,
> > > > > > >>
> > > > > > >> Thanks for the reply. My comments are inline [ms].
> > > > > > >>
> > > > > > >> I have omitted all proposed changes that work for me.
> > > > > > >>
> > > > > > >> Michael
> > > > > > >>
> > > > > > >>
> > > > > > >> -----Original Message-----
> > > > > > >> From: János Farkas <[e-mail]>
> > > > > > >> Sent: Wednesday, October 17, 2018 10:56 AM
> > > > > > >> To: Scharf, Michael <[e-mail]>
> > > > > > >> Cc: [e-mail]; draft-ietf-detnet-
> > > > [e-mail];
> > > > > > DetNet WG
> > > > > > >> <[e-mail]>
> > > > > > >> Subject: Re: Tsvart last call review of draft-ietf-detnet-
> > > > > > architecture-08
> > > > > > >>
> > > > > > >> Hi Michael,
> > > > > > >>
> > > > > > >> Thank you very much for the thorough review and good
> comments!
> > > > > > >>
> > > > > > >> Please find below in-line responses and how we plan to
> update
> > > > the
> > > > > > draft to
> > > > > > >> resolve your comments.
> > > > > > >>
> > > > > > >> Best regards,
> > > > > > >> Janos
> > > > > > >>
> > > > > > >>
> > > > > > >> On 9/29/2018 12:24 AM, Michael Scharf wrote:
> > > > > > >>> Reviewer: Michael Scharf
> > > > > > >>> Review result: Ready with Issues
> > > > > > >>>
> > > > > > >>> The document "Deterministic Networking Architecture"
> > > > > > >>> (draft-ietf-detnet-architecture-08) defines an overall
> > > > framework
> > > > > > for
> > > > > > >>> Deterministic Networking.
> > > > > > >>>
> > > > > > >>> As TSV-ART reviewer, I believe that this document has
> issues as
> > > > > > >>> detailed below.
> > > > > > >>>
> > > > > > >>> Michael
> > > > > > >>>
> > > > > > >>> Major issues:
> > > > > > >> [...]
> > > > > > >>
> > > > > > >>> DetNet
> > > > > > >>> basically requires support in (almost) all network
> devices
> > > > > > >>> transporting DetNet traffic. That assumption should be
> > > > explicitly
> > > > > > >>> spelt out early in the document, e.g., in the
> introduction.
> > > > > > >> Actually, some form of DetNet support is required from
> each node
> > > > > > that
> > > > > > >> transports DetNet traffic. For instance, DetNet flows have
> to be
> > > > > > recognized in
> > > > > > >> order not to spoil their QoS at the minimum.
> > > > > > >>
> > > > > > >> [ms] I look for an clear explanation in the architecture
> > > > document of
> > > > > > what "some
> > > > > > >> form of DetNet support" exactly means. For instance, is
> standard
> > > > > > MPLS TE
> > > > > > >> without any DetNet support good enough for an LSR on the
> IP/MPLS
> > > > > > path used
> > > > > > >> by DetNet traffic, or not? As an example, when I read the
> > > > following
> > > > > > paragraph in
> > > > > > >> draft-ietf-detnet-dp-sol-mpls-01 ...
> > > > > > >>
> > > > > > >>     Transit nodes are normal MPLS Label Switching Routers
> > > > (LSRs).
> > > > > > They
> > > > > > >>     are generally unaware of the special requirements of
> DetNet
> > > > > > flows,
> > > > > > >>     although they need to provide traffic engineering
> services
> > > > and
> > > > > > proper
> > > > > > >>     QoS to the LSPs associated with DetNet flows to
> enhance the
> > > > > > prospect
> > > > > > >>     of the LSPs meeting the DetNet service requirements.
> Some
> > > > > > >>     implementations of transit nodes may be DetNet aware,
> but
> > > > such
> > > > > > nodes
> > > > > > >>     just support the DetNet transport layer.
> > > > > > >>
> > > > > > >> ... I could maybe assume that actually there can be
> "transit
> > > > nodes"
> > > > > > that do not
> > > > > > >> require *any* sort of DetNet support or DetNet awareness.
> Is
> > > > that
> > > > > > correct? I
> > > > > > >> think the architecture document has to be clear on such
> > > > fundamental
> > > > > > questions.
> > > > > > >>
> > > > > > >> The Introduction could be extended to clarify, e.g., the
> third
> > > > > > paragraph:
> > > > > > >>
> > > > > > >> OLD:
> > > > > > >> A goal of DetNet is a converged network in all respects.
> That
> > > > is,
> > > > > > the presence of
> > > > > > >> DetNet flows does not preclude non-DetNet flows, and the
> > > > benefits
> > > > > > offered
> > > > > > >> DetNet flows should not, except in extreme cases, prevent
> > > > existing
> > > > > > QoS
> > > > > > >> mechanisms from operating in a normal fashion, subject to
> the
> > > > > > bandwidth
> > > > > > >> required for the DetNet flows. A single source-destination
> pair
> > > > can
> > > > > > trade both
> > > > > > >> DetNet and non-DetNet flows. End systems and applications
> need
> > > > not
> > > > > > >> instantiate special interfaces for DetNet flows. Networks
> are
> > > > not
> > > > > > restricted to
> > > > > > >> certain topologies; connectivity is not restricted. Any
> > > > application
> > > > > > that generates
> > > > > > >> a data flow that can be usefully characterized as having a
> > > > maximum
> > > > > > bandwidth
> > > > > > >> should be able to take advantage of DetNet, as long as the
> > > > necessary
> > > > > > resources
> > > > > > >> can be reserved. Reservations can be made by the
> application
> > > > itself,
> > > > > > via network
> > > > > > >> management, by an application???s controller, or by other
> means,
> > > > e.g.,
> > > > > > a dynamic
> > > > > > >> control plane (e.g., [RFC2205]).
> > > > > > >>
> > > > > > >> NEW:
> > > > > > >> A goal of DetNet is a converged network in all respects.
> That
> > > > is,
> > > > > > the presence of
> > > > > > >> DetNet flows does not preclude non-DetNet flows, and the
> > > > benefits
> > > > > > offered
> > > > > > >> DetNet flows should not, except in extreme cases, prevent
> > > > existing
> > > > > > QoS
> > > > > > >> mechanisms from operating in a normal fashion, subject to
> the
> > > > > > bandwidth
> > > > > > >> required for the DetNet flows. A single source-destination
> pair
> > > > can
> > > > > > trade both
> > > > > > >> DetNet and non-DetNet flows. End systems and applications
> need
> > > > not
> > > > > > >> instantiate special interfaces for DetNet flows. Networks
> are
> > > > not
> > > > > > restricted to
> > > > > > >> certain topologies; connectivity is not restricted. Any
> > > > application
> > > > > > that generates
> > > > > > >> a data flow that can be usefully characterized as having a
> > > > maximum
> > > > > > bandwidth
> > > > > > >> should be able to take advantage of DetNet, as long as the
> > > > necessary
> > > > > > resources
> > > > > > >> can be reserved. Reservations can be made by the
> application
> > > > itself,
> > > > > > via network
> > > > > > >> management, by an application???s controller, or by other
> means,
> > > > e.g.,
> > > > > > a dynamic
> > > > > > >> control plane (e.g., [RFC2205]). Network nodes supporting
> DetNet
> > > > > > flows have to
> > > > > > >> implement some of the DetNet capabilities (not necessarily
> all)
> > > > in
> > > > > > order to treat
> > > > > > >> DetNet flows such that their QoS requirements are met.
> > > > > > >>
> > > > > > >> [ms] To me, the last sentence is not clear. Does "some of
> the
> > > > DetNet
> > > > > > capabilities
> > > > > > >> (not necessarily all)" include e.g. vanilla traffic
> engineering?
> > > > For
> > > > > > instance, if a
> > > > > > >> network node can ensure that the QoS requirements are met
> by
> > > > > > leveraging
> > > > > > >> existing standardized TE without any additional DetNet
> > > > awareness, is
> > > > > > this
> > > > > > >> enough for deploying DetNet? Or does this node have to
> implement
> > > > > > additional
> > > > > > >> DetNet-specific functionality? Specifically, please
> explain if
> > > > an
> > > > > > existing DetNet-
> > > > > > >> unaware router on the path taken by DetNet traffic can be
> used
> > > > to
> > > > > > forward
> > > > > > >> DetNet traffic, or if it is mandatory that any router
> forwarding
> > > > > > DetNet traffic
> > > > > > >> indeed implements new capabilities specific to DetNet. In
> the
> > > > latter
> > > > > > case, I think
> > > > > > >> the document would need to discuss incremental deployment
> > > > strategies
> > > > > > as well.
> > > > > > >>
> > > > > > >>> * It is surprising that there is hardly any discussion on
> > > > network
> > > > > > robustness
> > > > > > >>> and safety; this probably also relates to security. For
> > > > instance,
> > > > > > >>> misconfiguration or errors of functions performing packet
> > > > > > replication could
> > > > > > >>> severely and permantly congest a network and cause harm.
> How
> > > > does
> > > > > > the DetNet
> > > > > > >>> architecture ensure that a network stays fully
> operational e.g.
> > > > if
> > > > > > the topology
> > > > > > >>> changes or there are equipment failures? Probably this
> can be
> > > > > > solved by
> > > > > > >>> implementations (e.g., dynamic control plane), but why
> are
> > > > > > corresponding
> > > > > > >>> requirements not spelt out? Section 3.3.2 speculates that
> > > > filters
> > > > > > and policers
> > > > > > >>> can help, and that may be true, but that probably still
> assumes
> > > > > > consistently
> > > > > > >>> and correctly configured (and well-behaving) devices. And
> > > > Section
> > > > > > 3.3.2 is
> > > > > > >>> vague and mentions a "infinite variety of possible
> failures"
> > > > > > without stating
> > > > > > >>> any requirements or recommendations. There may be further
> > > > > > solutions, such as
> > > > > > >>> circuit breakers and the like. Why are such topics not
> > > > discussed?
> > > > > > >> Security issues and considerations are addressed by the
> DetNet
> > > > > > Security
> > > > > > >> draft: https://datatracker.ietf.org/doc/draft-ietf-detnet-
> > > > security.
> > > > > > >> Reference to the  DetNet Security draft will be added.
> > > > > > >>
> > > > > > >> There will be a document dedicated to Operations,
> Administration
> > > > and
> > > > > > >> Maintenance (OAM), which will address operational,
> > > > misconfiguration,
> > > > > > and
> > > > > > >> failure detection issues.
> > > > > > >>
> > > > > > >> The topology changes have to be solved by the control
> plane,
> > > > either
> > > > > > >> centralized or distributed.
> > > > > > >>
> > > > > > >> The filters and policers described in Section 3.3.2 also
> provide
> > > > > > >> robustness by eliminating/mitigating traffic coming from
> > > > > > malfunctioning
> > > > > > >> devices.
> > > > > > >>
> > > > > > >> RFC2475 is recommended for traffic policing functions to
> > > > increase
> > > > > > >> robustness.
> > > > > > >>
> > > > > > >> The text  in Section 3.3.2 could be made clearer, e.g., by
> > > > updating
> > > > > > the
> > > > > > >> first paragraph to:
> > > > > > >>
> > > > > > >> OLD:
> > > > > > >> One key to building robust real-time systems is to reduce
> the
> > > > > > >> infinite variety of possible failures to a number that can
> be
> > > > > > >> analyzed with reasonable confidence. DetNet aids in the
> process
> > > > by
> > > > > > >> allowing for filters and policers to detect DetNet packets
> > > > received
> > > > > > >> on the wrong interface, or at the wrong time, or in too
> great a
> > > > > > >> volume, and to then take actions such as discarding the
> > > > offending
> > > > > > >> packet, shutting down the offending DetNet flow, or
> shutting
> > > > down
> > > > > > the
> > > > > > >> offending interface.
> > > > > > >>
> > > > > > >> NEW:
> > > > > > >> Robust real-time systems require to reduce the number of
> > > > possible
> > > > > > >> failures. Filters and policers should be used in a DetNet
> > > > network to
> > > > > > >> detect if DetNet packets are received on the wrong
> interface, or
> > > > at
> > > > > > >> the wrong time, or in too great volume. Furthermore,
> filters and
> > > > > > >> policers can take action to discard offending packets or
> flows,
> > > > or
> > > > > > >> trigger shutting down the offending DetNet flow, or
> shutting
> > > > down
> > > > > > >> the offending interface.
> > > > > > >>
> > > > > > >> [ms] That does not address my concern. As I wrote already,
> I
> > > > believe
> > > > > > this
> > > > > > >> document has to discuss the applicability of circuit
> breakers
> > > > > > according to RFC
> > > > > > >> 8084.
> > > > > > >>
> > > > > > >>> * Somewhat related, the document only looks at impact of
> > > > failures
> > > > > > to the QoS of
> > > > > > >>> DetNet traffic. What is missing is a discussion how to
> protect
> > > > non-
> > > > > > DetNet parts
> > > > > > >>> of a network from any harm caused by DetNet mechanisms.
> > > > Solutions
> > > > > > to this
> > > > > > >>> probably exist. But why is the impact on non-DetNet
> traffic
> > > > (e.g.,
> > > > > > in case of
> > > > > > >>> topology changes or failures of DetNet functions) not
> discussed
> > > > at
> > > > > > all in the document?
> > > > > > >> Actually the regulation of DetNet traffic by polices and
> filters
> > > > > > >> protects both other DetNet traffic and non-DetNet traffic.
> > > > > > >>
> > > > > > >> Section 3.3.1 could be extended to make it clear, e.g., by
> > > > extending
> > > > > > the
> > > > > > >> last paragraph:
> > > > > > >>
> > > > > > >> OLD:
> > > > > > >> Ideally, the net effect of the presence of DetNet flows in
> a
> > > > network
> > > > > > >> on the non-DetNet packets is primarily a reduction in the
> > > > available
> > > > > > >> bandwidth.
> > > > > > >>
> > > > > > >> NEW:
> > > > > > >> Traffic policing functions (e.g. [RFC2475]) are used to
> avoid
> > > > the
> > > > > > >> starvation of non-DetNet traffic. Thus, the net effect of
> the
> > > > > > presence
> > > > > > >> of DetNet flows in a network on non-DetNet flows is
> primarily a
> > > > > > >> reduction in the available bandwidth.
> > > > > > >>
> > > > > > >> [ms] I believe a stronger wording is needed, e.g. along
> the
> > > > lines of
> > > > > > "a DetNet
> > > > > > >> deployment must avoid starvation of non-DetNet traffic
> ...".
> > > > > > >>
> > > > > > >> [...]
> > > > > > >>
> > > > > > >>> * Section 4.4 refers to RFC 7426, which is an
> informational RFC
> > > > on
> > > > > > IRTF stream,
> > > > > > >>> and the document uses the concepts introduced there
> (e.g.,
> > > > > > "planes"). This is
> > > > > > >>> very confusing. First, an IETF Proposed Standard should
> > > > probably
> > > > > > refer to
> > > > > > >>> documents having IETF consensus. An example would be RFC
> 7491,
> > > > > > albeit there is
> > > > > > >>> other related work as well, e.g., in the TEAS WG. Second,
> > > > Section
> > > > > > 4.4 is by and
> > > > > > >>> large decoupled from the rest of the document and not
> specific
> > > > to
> > > > > > DetNet.
> > > > > > >>> Neither do other sections of the document refer to the
> concepts
> > > > > > introduced in
> > > > > > >>> Section 4.4, nor does Section 4.4 use the DetNet
> terminology or
> > > > > > discuss
> > > > > > >>> applicability to DetNet. Section 4.4 even mentions
> explicitly
> > > > at
> > > > > > the end that
> > > > > > >>> it discusses aspects that are orthogonal to the DetNet
> > > > > > architecture. It is not
> > > > > > >>> at all clear why Section 4.4 is in this document. Section
> 4.4
> > > > could
> > > > > > be removed
> > > > > > >>> from the document without impacting the rest of the
> document.
> > > > > > >> The document should point out to traffic engineering and
> > > > centralized
> > > > > > >> control plane aspects, so it is better to keep Section
> 4.4.
> > > > > > >>
> > > > > > >> RFC 7426 provides the full picture of SDN architecture,
> which
> > > > needs
> > > > > > to
> > > > > > >> be referred from the DetNet architecture. No other RFC
> provides
> > > > such
> > > > > > >> detailed picture. Therefore, we need to keep the reference
> to
> > > > RFC
> > > > > > 7426,
> > > > > > >> which is an informative reference.
> > > > > > >>
> > > > > > >> [ms] I'll leave that up to the IESG to decide whether an
> IETF
> > > > > > proposed standard
> > > > > > >> should build an architecture based on research work
> instead of,
> > > > say,
> > > > > > documents
> > > > > > >> with IETF consensus. As a side note, I believe the
> understanding
> > > > in
> > > > > > industry what
> > > > > > >> "SDN architecture" really means and how it is implemented
> may
> > > > have
> > > > > > evolved
> > > > > > >> since publication of IRTF RFC 7426, which was published in
> Jan.
> > > > > > 2015. For
> > > > > > >> instance, IRTF RFC 7426 does not discuss the implications
> of
> > > > segment
> > > > > > routing,
> > > > > > >> BGP-LS, and other recent IETF techniques. And, based on
> what I
> > > > read
> > > > > > in the
> > > > > > >> DetNet architecture, I believe DetNet could actually be
> > > > implemented
> > > > > > without
> > > > > > >> any "SDN architecture", e.g. by distributed traffic
> engineering.
> > > > As
> > > > > > a result, I don't
> > > > > > >> even understand the need to discuss "SDN".
> > > > > > >>
> > > > > > >> [ms] For the record: As TSV-ART reviewer, I question
> whether
> > > > IRTF
> > > > > > RFC 7426 is
> > > > > > >> indeed an appropriate reference, I have doubts that IRTF
> RFC
> > > > 7426 is
> > > > > > really up-
> > > > > > >> to-date, and I am concerned about the content quality of
> section
> > > > > > 4.4, e.g.,
> > > > > > >> regarding the lack of alignment with the rest of the
> document.
> > > > The
> > > > > > IESG will
> > > > > > >> certainly have a better understanding that than me how to
> deal
> > > > with
> > > > > > such topics.
> > > > > > >>
> > > > > > >> The related work in TEAS is already referred to.
> > > > > > >>
> > > > > > >> [...]
> > > > > > >>
> > > > > > >>> Minor issues:
> > > > > > >>>
> > > > > > >>> * Terminology "DetNet transport layer"
> > > > > > >>>
> > > > > > >>> The term "transport layer" has a well-defined meaning in
> the
> > > > IETF,
> > > > > > e.g.
> > > > > > >>> originating from RFC 1122. While "transport" and e.g.
> > > > "transport
> > > > > > network" is
> > > > > > >>> used in the IETF for different technologies in different
> areas,
> > > > I
> > > > > > think the
> > > > > > >>> term "transport layer" is typically understood to refer
> to
> > > > > > transport
> > > > > > >>> protocols such as TCP and UDP. As such, I personally find
> the
> > > > term
> > > > > > "DetNet
> > > > > > >>> transport layer" misleading and confusing. The confusion
> is
> > > > easy to
> > > > > > see e.g.
> > > > > > >>> in Figure 4, where UDP (which is a transport protocol as
> per
> > > > RFC
> > > > > > 1122) sits
> > > > > > >>> on top of "transport".
> > > > > > >>>
> > > > > > >>> Based on the document it also may be
> solution/implementation
> > > > > > specific
> > > > > > >> whether
> > > > > > >>> the "DetNet transport layer" is actually a separate
> protocol
> > > > layer
> > > > > > compared
> > > > > > >>> to the "DetNet service layer". Thus it is not clear to me
> why
> > > > the
> > > > > > word
> > > > > > >>> "layer" has to be used, specifically in combination
> "transport
> > > > > > layer".
> > > > > > >>>
> > > > > > >>> To me as, the word "transport layer" (and "transport
> protocol")
> > > > > > should be
> > > > > > >>> used for protocols defined in TSV area, consistent with
> RFC
> > > > 1122.
> > > > > > But this is
> > > > > > >>> probably a question to be sorted out by the IESG.
> > > > > > >> "transport" is used here as in "packet transport networks"
> not
> > > > as
> > > > > > OSI L4
> > > > > > >> transport.
> > > > > > >>
> > > > > > >> [ms] At the risk of stating the obvious: The relevant
> section of
> > > > RFC
> > > > > > 1122 defining
> > > > > > >> "transport layer" is entitled by "The Internet
> Architecture".
> > > > And,
> > > > > > according e.g.
> > > > > > >> to RFC 5921, "packet transport network" is a term defined
> by
> > > > ITU-T.
> > > > > > In an IETF
> > > > > > >> document, I believe IETF terminology should be used.
> > > > > > >>
> > > > > > >> I suggest to use the terms "DetNet transport sub-layer"
> and
> > > > "DetNet
> > > > > > >> service sub-layer" throughout the document, which would
> > > > hopefully
> > > > > > avoid
> > > > > > >> confusion.
> > > > > > >>
> > > > > > >> [ms] "DetNet transport sub-layer" somewhat solves my
> concern of
> > > > > > avoiding the
> > > > > > >> expression "transport layer". To really avoid confusion, I
> > > > believe
> > > > > > it would be
> > > > > > >> better to avoid the ambiguous term "transport" altogether.
> But
> > > > I
> > > > > > will be fine if
> > > > > > >> all DetNet documents consistently avoid the term
> "transport
> > > > layer".
> > > > > > >>
> > > > > > >> Michael
> > > > > > >> _______________________________________________
> > > > > > >> Tsv-art mailing list
> > > > > > >> [e-mail]
> > > > > > >> https://www.ietf.org/mailman/listinfo/tsv-art
> > > > > > > _______________________________________________
> > > > > > > detnet mailing list
> > > > > > > [e-mail]
> > > > > > > https://www.ietf.org/mailman/listinfo/detnet
> > > > > >
> > > > > > _______________________________________________
> > > > > > Tsv-art mailing list
> > > > > > [e-mail]
> > > > > > https://www.ietf.org/mailman/listinfo/tsv-art
> > > _______________________________________________
> > > detnet mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/detnet
> >
> >
> > _______________________________________________
> > detnet mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/detnet
> 
> --
> ---
> [e-mail]