Re: [Detnet] [6tisch] comments on draft-finn-detnet-architecture-00

["Pascal Thubert (pthubert)" <[e-mail]>]

Yes,

But we can absorb that difference of latency at staging points along the path, which do not have to be at the egress, so the requirement on buffer capacity that is classically required at the egress could be lowered.

Cheers,

Pascal


> -----Original Message-----
> From: 6tisch [mailto:[e-mail]] On Behalf Of
> [e-mail]
> Sent: lundi 13 avril 2015 12:04
> To: Rodney Cummings
> Cc: Thomas Watteyne; Patrick Wetterwald (pwetterw); Norman Finn (nfinn);
> [e-mail]; [e-mail]; Christian Boiger
> Subject: Re: [6tisch] [Detnet] comments on draft-finn-detnet-architecture-00
> 
> Hi,
> 
> The math for Option 1 can really get interesting if we combine zero jitter
> pursuit with seamless redundancy to improve the message reliability.
> Different paths can present dissimilar jitter to the "virtual" end-point
> which is expected to give jitter-free stream to the listener, apart from
> removing duplicates. With more path diversity comes more complexity in
> smoothening the stream.
> 
> Anand
> 
> 
> 
> > I agree with Norm's conclusion, but you are correct that this is a
> > tradeoff in complexity (and therefore to some degree cost).
> >
> > As a baseline assumption, I take it as a given that every host is synced
> > in time (e.g. IEEE 1588). Power does this, industrial does this, and
> > automotive is doing it.
> >
> > Let's also assume that the application requires zero jitter. Both Option 1
> > and 2 meet that requirement.
> >
> > Option 1 is simply what Norm says... use the packet at the point when your
> > application's jitter is zero in synced time. In the host (end-station),
> > this requires a buffer per DetNet (zero-jitter) flow, and the ability to
> > identify each flow. That is not rocket science, even for an FPGA. For
> > large topologies, the tradeoff is that there is some complex math to
> > figure out the worst-case latency, but let's be clear... that's just
> > math... not higher cost hardware. There are ways to mitigate this math
> > using certain techniques in the network... in the same direction as you go
> > with Option 2, but not "full blown" (not zero jitter at every hop).
> >
> > Option 2 sounds appealing at first, but it implies identification and
> > buffering of each DetNet flow at every hop. You are essentially moving a
> > minor requirement for the host to every switch/router in the network,
> > which greatly expands the overall complexity. You also don't avoid the
> > math problem, because now there is a complex calculation to figure out the
> > schedule for every flow at every hop.
> >
> > The main difference is that Option 2 is more complex to implement,
> > probably by order(s) of magnitude. I agree with Norm. If you are designing
> > a new application for zero jitter, Option 1 is the way to go.
> >
> > All of this being said, 802.1 TSN is providing hooks to enable either
> > option. Option 1 is the baseline assumption for most applications, but
> > Option 2 is certainly do-able. Hopefully IETF DetNet will move in the same
> > direction.
> >
> >
> >
> > From:   Christian Boiger <[e-mail]>
> > To:     "Norman Finn (nfinn)" <[e-mail]>, Qin Wang
> > <[e-mail]>, "Patrick Wetterwald (pwetterw)"
> > <[e-mail]>, Thomas Watteyne <[e-mail]>,
> > "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>,
> > Date:   04/10/2015 06:32 AM
> > Subject:        Re: [Detnet] [6tisch] comments on
> > draft-finn-detnet-architecture-00
> > Sent by:        "detnet" <[e-mail]>
> >
> >
> >
> > Actually the packet delay variation is not only important for the
> > applications, it is also an important factor to a achieve really low
> > deterministic latency. In networks where the packet delay variation for a
> > packet grows at each hop it gets very hard to guarantee a very low
> > latency. AVB is working that way and this has its limitations, especially
> > in arbitrary big and complex networks (and the goal of AVB was only 2ms
> > over 7 hops). In order to get much smaller latencies and simpler ways to
> > calculate the worst case latency in a network, a small and bounded (not
> > only bounded by the worst case latency) packet delay variation is very
> > important.
> >
> > Regards
> > Christian
> >
> >
> > Von: detnet [mailto:[e-mail]] Im Auftrag von Norman Finn
> > (nfinn)
> > Gesendet: Donnerstag, 9. April 2015 22:53
> > An: Qin Wang; Patrick Wetterwald (pwetterw); Thomas Watteyne;
> > [e-mail]; [e-mail]
> > Betreff: Re: [Detnet] [6tisch] comments on
> > draft-finn-detnet-architecture-00
> >
> > There are (at least!) two basic models in use, today, for running real
> > time applications over ethernet:
> >
> >  1. Do what the packet says to do when you get the packet.
> >     That allows a maximally-stupid end node, but constrains the network to
> > have near-0 jitter, or things can happen at the wrong time.  This is not
> > trivial for the network to do, can require considerable buffering
> > capability, especially applied to multicasts, and VERY quickly ceases to
> > scale.
> >  2. Synchronize your clock with everyone else's, then do what the packet
> > says to do when the packet says to do it.
> >     That requires a somewhat smarter end node, but means that the network
> > only has to guarantee a worst-case maximum latency, not a worst-case
> > minimum latency.
> >
> > If I were designing a new application, I would lean towards the second
> > scheme.  But, one size definitely doesn’t fit all.
> >
> > — Norm
> >
> > From: Qin Wang <[e-mail]>
> > Reply-To: Qin Wang <[e-mail]>
> > Date: Thursday, April 9, 2015 at 08:09 AM
> > To: "Patrick Wetterwald (pwetterw)" <[e-mail]>, Thomas
> Watteyne
> > <[e-mail]>, "[e-mail]" <[e-mail]>, "
> > [e-mail]" <[e-mail]>
> > Subject: Re: [Detnet] [6tisch] comments on
> > draft-finn-detnet-architecture-00
> >
> > Hi Patrick,
> >
> > I don't understand why communication jitter = 0 is so important. Assume
> > DetNet can guarantee the maximum end-to-end latency, and there is a well
> > designed buffer in the Listener device, then I think the Listener can
> > consume data just on time, in another word, from application point of
> > view, jitter =0. Please point out if I'm wrong.
> >
> > Thanks
> > Qin
> >
> >
> > On Thursday, April 9, 2015 5:03 PM, Patrick Wetterwald (pwetterw) <
> > [e-mail]> wrote:
> >
> > Norm, Pascal,
> >
> > Some other comments:
> >
> > If you look at
> > https://tools.ietf.org/html/draft-wetterwald-detnet-utilities-reqs you
> > will see that for some important use cases, the jitter should very close
> > to zero. In the architecture draft, you are mainly focussing on the
> > maximum value of the latency and the packet loss (at least in the text).
> > This is not sufficient for power automation and tele protection use case.
> > We need to be able to deploy deterministic network with Jitter = 0.
> > Second: may be good to reference the requirement drafts in the doc. :-)
> > Thanks,
> >
> > Patrick
> >
> >
> > From: Thomas Watteyne <[e-mail]>
> > Date: Monday 23 March 2015 18:44
> > To: "[e-mail]" <[e-mail]>, "[e-mail]" <
> > [e-mail]>
> > Subject: [Detnet] comments on draft-finn-detnet-architecture-00
> >
> > Norm, Pascal,
> >
> > Please find below a number of comment on
> > draft-finn-detnet-architecture-00.
> >
> > Overall, the draft is very well written and highlights important points
> > for deterministic networking.
> >
> > My general comments are:
> > - in the context of a low-power wireless network (e.g. 6TiSCH), I wonder
> > what you thoughts are on reserving a fixed path. Maybe some extra text on
> > redundancy is needed?
> > - would it make sense to discuss the difference between hard and soft
> > real-time, also wrt wireless systems?
> >
> > Thomas
> >
> > ---
> >
> >
> > DetNet                                                           N. Finn
> > Internet-Draft                                                P. Thubert
> > Intended status: Standards Track                                   Cisco
> > Expires: September 10, 2015                                March 9, 2015
> >
> >
> >                  Deterministic Networking Architecture
> >                    draft-finn-detnet-architecture-00
> >
> > Abstract
> >
> >    Deterministic Networking (DetNet) provides a capability to carry
> >    specified unicast or multicast data streams for real-time
> >    applications with extremely low data loss rates and maximum latency.
> > TW> it looks like you aim for maximum latency :)
> >    Techniques used include: 1) reserving data plane resources for
> >    individual (or aggregated) DetNet streams in some or all of the relay
> >    systems (bridges or routers) along the path of the stream; 2)
> >    providing fixed paths for DetNet streams that do not rapidly change
> >    with the network topology; and 3) sequentializing, replicating, and
> >    eliminating duplicate packets at various points to ensure the
> >    availability of at least one path.  The capabilities can be managed
> >    by configuration, or by manual or automatic network management.
> >
> > Status of This Memo
> >
> >    This Internet-Draft is submitted in full conformance with the
> >    provisions of BCP 78 and BCP 79.
> >
> >    Internet-Drafts are working documents of the Internet Engineering
> >    Task Force (IETF).  Note that other groups may also distribute
> >    working documents as Internet-Drafts.  The list of current Internet-
> >    Drafts is at http://datatracker.ietf.org/drafts/current/.
> >
> >    Internet-Drafts are draft documents valid for a maximum of six months
> >    and may be updated, replaced, or obsoleted by other documents at any
> >    time.  It is inappropriate to use Internet-Drafts as reference
> >    material or to cite them other than as "work in progress."
> >
> >    This Internet-Draft will expire on September 10, 2015.
> >
> > Copyright Notice
> >
> >    Copyright (c) 2015 IETF Trust and the persons identified as the
> >    document authors.  All rights reserved.
> >
> >    This document is subject to BCP 78 and the IETF Trust's Legal
> >    Provisions Relating to IETF Documents
> >    (http://trustee.ietf.org/license-info) in effect on the date of
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 1]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    publication of this document.  Please review these documents
> >    carefully, as they describe your rights and restrictions with respect
> >    to this document.  Code Components extracted from this document must
> >    include Simplified BSD License text as described in Section 4.e of
> >    the Trust Legal Provisions and are provided without warranty as
> >    described in the Simplified BSD License.
> >
> > Table of Contents
> >
> >    1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
> >    2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
> >    3.  Providing the DetNet Quality of Service . . . . . . . . . . .   5
> >      3.1.  Zero Congestion Loss  . . . . . . . . . . . . . . . . . .   6
> >      3.2.  Pinned-down paths . . . . . . . . . . . . . . . . . . . .   7
> >      3.3.  Seamless Redundancy . . . . . . . . . . . . . . . . . . .   7
> >    4.  DetNet Architecture . . . . . . . . . . . . . . . . . . . . .   8
> >      4.1.  The Application Plane . . . . . . . . . . . . . . . . . .  11
> >      4.2.  The Controller Plane  . . . . . . . . . . . . . . . . . .  11
> >      4.3.  The Network Plane . . . . . . . . . . . . . . . . . . . .  12
> >      4.4.  Elements of DetNet Architecture . . . . . . . . . . . . .  13
> >      4.5.  DetNet streams  . . . . . . . . . . . . . . . . . . . . .  14
> >        4.5.1.  Talker guarantees . . . . . . . . . . . . . . . . . .  14
> >        4.5.2.  Incomplete Networks . . . . . . . . . . . . . . . . .  15
> >      4.6.  Data Flow Model through Systems . . . . . . . . . . . . .  16
> >      4.7.  Queuing, Shaping, Scheduling, and Preemption  . . . . . .  16
> >      4.8.  Coexistence with normal traffic . . . . . . . . . . . . .  16
> >      4.9.  Fault Mitigation  . . . . . . . . . . . . . . . . . . . .  16
> >      4.10. Protocol Stack Model  . . . . . . . . . . . . . . . . . .  17
> >      4.11. Advertising resources, capabilities and adjacencies . . .  17
> >      4.12. Provisioning model  . . . . . . . . . . . . . . . . . . .  17
> >        4.12.1.  Centralized Path Computation and Installation  . . .  17
> >        4.12.2.  Distributed Path Setup . . . . . . . . . . . . . . .  17
> >    5.  Related IETF work . . . . . . . . . . . . . . . . . . . . . .  18
> >      5.1.  Deterministic PHB . . . . . . . . . . . . . . . . . . . .  18
> >      5.2.  6TiSCH  . . . . . . . . . . . . . . . . . . . . . . . . .  18
> >    6.  Security Considerations . . . . . . . . . . . . . . . . . . .  19
> >    7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  19
> >    8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  19
> >    9.  Informative References  . . . . . . . . . . . . . . . . . . .  19
> >    Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  23
> >
> > 1.  Introduction
> >
> >    Operational Technology (OT) refers to industrial networks that are
> >    typically used for monitoring systems and supporting control loops,
> >    as well as movement detection systems for use in process control
> >    (i.e., process manufacturing) and factory automation (i.e., discrete
> >    manufacturing).  Due to its different goals, OT has evolved in
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 2]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    parallel but in a manner that is radically different from IT/ICT,
> > TW> define IT and ICT at first use
> >    focusing on highly secure, reliable and deterministic networks, with
> >    limited scalability over a bounded area.
> >
> >    The convergence of IT and OT technologies, also called the Industrial
> >    Internet, represents a major evolution for both sides.  The work has
> >    already started; in particular, the industrial automation space has
> >    been developing a number of Ethernet-based replacements for existing
> >    digital control systems, often not packet-based (fieldbus
> >    technologies).
> >
> >    These replacements are meant to provide similar behavior as the
> >    incumbent protocols, and their common focus is to transport
> > TW> "on transporting"?
> >    a fully
> >    characterized flow over a well-controlled environment (i.e., a
> >    factory floor), with a bounded latency, extraordinarily low frame
> >    loss, and a very narrow jitter.  Examples of such protocols include
> >    PROFINET, ODVA Ethernet/IP, and EtherCAT.
> > TW> It might be good to quantify the target latency, packet loss and
> > jitter.
> > TW> I appreciate this is not easy, but one representatve example might
> > help
> >    In parallel, the need for determinism in professional and home audio/
> >    video markets drove the formation of the Audio/Video Bridging (AVB)
> >    standards effort of IEEE 802.1.  With the explosion of demand for
> >    connectivity and multimedia in transportation in general, the
> >    Ethernet AVB technology has become one of the hottest topics, in
> >    particular in the automotive connectivity.  It is finding application
> >    in all elements of the vehicle
> > TW> ":"
> >    from head units, to rear seat
> >    entertainment modules, to amplifiers and camera modules.  While aimed
> >    at less critical applications than some industrial networks, AVB
> >    networks share the requirement for extremely low packet loss rates
> >    and ensured finite latency and jitter.
> > TW> ensure -> guarantee?
> >
> >    Other instances of in-vehicle deterministic networks have arisen as
> >    well for control networks in cars, trains and buses, as well as
> >    avionics, with, for instance, the mission-critical "Avionics Full-
> >    Duplex Switched Ethernet" (AFDX) that was designed as part of the
> >    ARINC 664 standards.  Existing automotive control networks such as
> >    the LIN, CAN and
> > TW> would TTP/C fit in this list?
> >    FlexRay standards were not designed to cover these
> >    increasing demands in terms of bandwidth and scalability that we see
> >    with various kinds of Driver Assistance Systems (DAS) and new
> >    multiplexing technologies based on Ethernet are now getting traction.
> >
> >    The generalization of the needs for more deterministic networks have
> >    led to the IEEE 802.1 AVB Task Group becoming the Time-Sensitive
> >    Networking (TSN) Task Group (TG), with a much-expanded constituency
> >    from the industrial and vehicular markets.  Along with this
> >    expansion, the networks in consideration are becoming larger and
> >    structured, requiring deterministic forwarding beyond the LAN
> >    boundaries.  For instance, Industrial Automation segregates the
> >    network along the broad lines of the Purdue Enterprise Reference
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 3]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    Architecture (PERA), using different technologies at each level, and
> >    public infrastructures such as Electricity Automation require
> >    deterministic properties over the Wide Area.  The realization is now
> >    coming that the convergence of IT and OT networks requires Layer-3,
> >    as well as Layer-2, capabilities.
> >
> >    The present architecture is the result of a collaboration of the IETF
> >    and the IEEE and implements an abstract model that can be applicable
> >    both at Layer-2 and Layer-3, and along segments of different
> >    technologie.
> > TW> typo
> >    With this new work, a path may span, for instance,
> >    across a (limited) number of 802.1 bridges and then a (limited)
> >    number of IP routers.
> > TW> add somethind like "possibly wireless" somewhere?
> >    In that example, the IEEE 802.1 bridges may be
> >    operating at Layer-2 over Ethernet whereas the IP routers may be
> >    6TiSCH nodes operating at Layer-2 and/or Layer-3 over the IEEE
> >    802.15.4e MAC.
> >
> >    Many applications of interest to Deterministic Networking require the
> >    ability to synchronize the clocks in end systems to a sub-microsecond
> >    accuracy.
> > TW> why this accuracy?
> >    Some of the queue control techniques defined in
> >    Section 4.7 also require time synchronization among relay systems.
> >    The means used to achieve time synchronization are not addressed in
> >    this document.
> >
> > 2.  Terminology
> >
> >    The follwing
> > TW> typo
> >    special terms are used in this document in order to
> >    avoid the assumption that a given element in the archetecture
> > TW> typo
> >    does or
> >    does not have Internet Protocol stack, functions as a router or a
> >    bridge, or otherwise plays a particular role at Layer-3 or higher:
> > TW> I don't get the sentence above
> >
> >    bridge
> >            A Customer Bridge as defined by IEEE 802.1Q
> >            [IEEE802.1Q-2011].
> >
> >    circuit
> >            A trail of configuration from talker to listener(s)
> > TW> sender and receiver might be more common at the IETF?
> >            through
> >            relay systems associated with a DetNet stream, required to
> >            deliver the benefits of DetNet.
> >
> >    end system
> >            Commonly called a "host" in IETF documents, and an "end
> >            station" is IEEE 802 documents.  End systems of interest to
> >            this document are talkers and listeners.
> >
> >    listener
> >            An end system capable of sinking a DetNet stream.
> >
> >    relay system
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 4]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >            A router or a bridge.
> >
> > TW> maybe add "layer 2 bridge" and "layer 3 router" for clarity?
> >
> >
> >    stream
> >            A DetNet stream is a sequence of packets from a single
> >            talker, through some number of relay systems to one or more
> >            listeners, that is limited by the talker in its maximum
> >            packet size and transmission rate, and can thus be ensured
> >            the DetNet Quality of Service (QoS) from the network.
> >
> >    talker
> >            An end system capable of sourcing a DetNet stream.
> >
> > 3.  Providing the DetNet Quality of Service
> >
> >    DetNet Quality of Service is expressed in terms of:
> >
> >    o  Minimum and maximum end-to-end latency from talker to listener;
> >
> >    o  Probability of loss of a packet, assuming the normal operation of
> >       the relay systems and links;
> >
> >    o  Probability of loss of a packet in the event of the failure of a
> >       relay system or link.
> >
> >    It is a distinction of DetNet that it is concerned solely with worst-
> >    case values for all of the above parameters.  Average, mean, or
> >    typical values are of no interest, because they do not affect the
> >    ability of a real-time system to perform its tasks.
> > TW> Would it make sense to discuss hard and soft real-time here, in
> > TW> particular in the context of a wireless system?
> >
> >
> >    Three techniques are employed by DetNet to achieve these QoS
> >    parameters:
> >
> >    a.  Zero congestion loss (Section 3.1).  Network resources such as
> >        link bandwidth, buffers, queues, shapers, and scheduled input/
> > TW> add 6TiSCH cells?
> >        output slots are assigned in each relay system to the use of a
> >        specific DetNet stream or group of streams.  Note that, given a
> >        finite amount of buffer space)
> > TW> extra ")"
> >    , zero congestion loss necessarily
> >        ensures a maximum end-to-end latency.  Depending on the method
> >        employed, a minimum latency can also be achieved.
> >
> >    b.  Pinned-down paths (Section 3.2).  Point-to-point paths or point-
> >        to-multipoint trees through the network from a talker to one or
> >        more listeners can be established, and DetNet streams assigned to
> >        follow a particular path or tree.
> > TW> Using a single path in a wireless system will fail, I would recommend
> > TW> discussing here path redundancy, for example through DAGs
> >
> >    c.  Packet replication and deletion (Section 3.3).  End systems and/
> >        or relay systems can sequence number, replicate, and eliminate
> >        replicated packets at multiple points in the network in order to
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 5]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >        ensure that one (or more) equipment failure events still leave at
> >        least one path intact for a DetNet stream.
> >
> >    These three techniques can be applied independently, giving eight
> >    possible combinations, including none (no DetNet), although some
> >    combinations are of wider utility than others.  This separation keeps
> >    the protocol stack coherent and maximizes interoperability with
> >    existing and developing standards in this (IETF) and other Standards
> >    Development Organizations.  Some examples of typical expected
> >    combinations:
> >
> >    o  Pinned-down paths (a) plus packet replication (b) are exactly the
> >       techniques employed by [HSR-PRP].  Pinned-down paths are achieve
> >       by limiting the physical topology of the network, and the
> >       sequentialization, replication, and duplicate elimination
> >       facilitated by packet tags added at the front or the end of
> >       Ethernet frames.
> >
> >    o  Zero congestion loss (a) alone is is offered by IEEE 802.1 Audio
> >       Video bridging [IEEE802.1BA-2011].  As long as the network suffers
> >       no failures, near-zero (at best, zero) congestion loss can be
> >       achieved through the use of a reservation protocol (MSRP) and
> >       shapers in every relay system (bridge).
> >
> >    o  Using all three together gives maximum protection.
> >
> >    There are, of course, simpler methods available (and employed, today)
> >    to achieve levels of latency and packet loss that are satisfactory
> >    for many applications.  However, these methods generally work best in
> >    the absence of any significant amount of non-critical traffic in the
> >    network (if, indeed, such traffic is supported at all), or work only
> >    if the critical traffic constitutes only a small portion of the
> >    network's theoretical capacity, or work only if all systems are
> >    functioning properly, or in the absence of actions by end systems
> >    that disrupt the network's operations.
> >
> >    There are any number of methods in use, defined, or in progress for
> >    accomplishing each of the above techniques.  It is expected that this
> >    DetNet Architecture will assist various vendors, users, and/or
> >    "vertical" Standards Development Organizations (dedicated to a single
> >    industry) to make selections among the available means of
> >    implementing DetNet networks.
> >
> > 3.1.  Zero Congestion Loss
> >
> >    The primary means by which DetNet achieves its QoS assurances is to
> >    completely eliminate congestion at an output port as a cause of
> >    packet loss.  Given that a DetNet stream cannot be throttled, this
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 6]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    can be achieved only by the provision of sufficient buffer storage at
> >    each hop through the network to ensure that no packets are dropped
> >    due to a lack of buffer storage.
> > TW> and the provision the sufficient link bandwidth?
> >
> >    Ensuring adequate buffering requires, in turn, that the talker, and
> >    every relay system along the path to the listener (or nearly every
> >    relay system -- see Section 4.5.2) be careful to regulate its output
> >    to not exceed the data rate for any stream, except for brief perios
> > TW> typo
> >    when making up for interfering traffic.  Any packet sent ahead of its
> >    time potentially adds to the number of buffers required by the next
> >    hop, and may thus exceed the resources allocated for a particular
> >    stream.
> >
> >    The low-level mechanisms described in Section 4.7 provide the
> >    necessary regulation of transmissions by an edge system or relay
> >    system to ensure zero congestion loss.  Of course, the reservation of
> >    the bandwidth and buffers for a stream requires the provisioning
> >    described in Section 4.12.
> >
> > 3.2.  Pinned-down paths
> >
> >    In networks controlled by typical peer-to-peer protocols such as IEEE
> >    802.1 ISIS bridged networks or ETF OSPF routed networks, a network
> >    topology event in one part of the network can impact, at least
> >    briefly, the delivery of data in parts of the network remote from the
> >    failure or recovery event.  Thus, even redundant paths through a
> >    network, if controlled by the typical peer-to-peer protocols, do not
> >    eliminate the chances of brief losses of contact.  For this reason,
> >    many real-time networks rely on physical rings of two-port devices,
> >    with a relatively simple ring control protocol.  This both minimizes
> >    recovery time and easily supports redundant paths.  Of course, this
> >    comes at the cost of increased hop count, and thus latency, for the
> >    typical path.
> >
> >    In order to get the advantages of low hop count and still ensure
> >    against even brief losses of connectivity, DetNet employs pinned-down
> >    paths, where the path taken by a given DetNet stream does not change,
> >    at least immediately, and likely not at all, in response to network
> >    topology events.  When combined with seamless redundancy
> >    (Section 3.3), this results in a high likelihood of continuous
> >    connectivity.
> >
> > 3.3.  Seamless Redundancy
> >
> >    After congestion loss has been eliminated, the most important causes
> >    of packet loss are random media and/or memory faults and equipment
> >    failures.
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 7]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    Seamless redundancy involves three capabilities:
> >
> >    o  Adding sequence numbers to the packets of a DetNet stream.
> >
> >    o  Replicating these packets and, typically, sending them along at
> >       least two different paths to the listener(s).
> >
> >    o  Discarding duplicated packets.
> >
> >    In the simplest case, this amounts to replicating each packet in a
> >    talker that has two interfaces, and conveying them through the
> >    network, along separate paths, to the similarly dual-homed listeners,
> >    that discard the extras.  This ensures that one path (with zero
> >    congestion loss) remains, even if some relay system fails.
> > TW> this has some energy cost associated?
> >
> >    Alternatively, relay systems in the network can provide replication
> >    and elimination facilities at various points in the network, so that
> >    multiple failures can be accommodated.
> >
> >    This is shown in the following figure, where the two relay systems
> >    each replicate (R) the DetNet stream on input, sending the stream to
> >    both the other relay system and to the end system, and eliminated
> >    duplicates (E) on the output interface to the right-hand end system.
> >    Any one links
> > TW> typo
> >    in the network can fail, and the Detnet stream can
> >    still get through.  Furthermore, two links can fail, as long as they
> >    are in different segments of the network.
> >
> >                      > > > > > > > >   relay    > > > > > > > >
> >                     > /------------+ R system E +------------\ >
> >                    > /                  v + ^                 \ >
> >    end    R +                   v | ^                  + E end
> >    system   +                   v | ^                  +   system
> >                    > \                  v + ^                 / >
> >                     > \------------+ R relay  E +------------/ >
> >                      > > > > > > > >   system   > > > > > > > >
> >
> >                                  Figure 1
> > TW> I don't understand what R and E means
> >
> > 4.  DetNet Architecture
> >
> >    Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
> >    traffic-engineering architectures for generic applicability across
> >    packet and non-packet networks.  From TEAS perspective, Traffic
> >    Engineering (TE) refers to techniques that enable operators to
> >    control how specific traffic flows are treated within their networks.
> >
> >    Because if its very nature of establishing pinned-down optimized
> >    paths, Deterministic Networking can be seen as a new, specialized
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 8]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    branch of Traffic Engineering, and inherits its architecture with a
> >    separation into planes.
> >
> >    The Deterministic Networking architecture is thus composed of three
> >    planes, a (User) Application Plane, a Controller Plane, and a Network
> >    Plane, which echoes that of Software-Defined Networking (SDN):
> > TW> double ":"
> >    Layers
> >    and Architecture Terminology [RFC7426] which is represented below:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015               [Page 9]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >            SDN Layers and Architecture Terminology per RFC 7426
> >
> >                      o--------------------------------o
> >                      |                                |
> >                      | +-------------+   +----------+ |
> >                      | | Application |   |  Service | |
> >                      | +-------------+   +----------+ |
> >                      |       Application Plane        |
> >                      o---------------Y----------------o
> >                                      |
> >        *-----------------------------Y---------------------------------*
> >        |           Network Services Abstraction Layer (NSAL)           |
> >        *------Y------------------------------------------------Y-------*
> >               |                                                |
> >               |               Service Interface                |
> >               |                                                |
> >        o------Y------------------o       o---------------------Y------o
> >        |      |    Control Plane |       | Management Plane    |      |
> >        | +----Y----+   +-----+   |       |  +-----+       +----Y----+ |
> >        | | Service |   | App |   |       |  | App |       | Service | |
> >        | +----Y----+   +--Y--+   |       |  +--Y--+       +----Y----+ |
> >        |      |           |      |       |     |               |      |
> >        | *----Y-----------Y----* |       | *---Y---------------Y----* |
> >        | | Control Abstraction | |       | | Management Abstraction | |
> >        | |     Layer (CAL)     | |       | |      Layer (MAL)       | |
> >        | *----------Y----------* |       | *----------Y-------------* |
> >        |            |            |       |            |               |
> >        o------------|------------o       o------------|---------------o
> >                     |                                 |
> >                     | CP                              | MP
> >                     | Southbound                      | Southbound
> >                     | Interface                       | Interface
> >                     |                                 |
> >        *------------Y---------------------------------Y----------------*
> >        |         Device and resource Abstraction Layer (DAL)           |
> >        *------------Y---------------------------------Y----------------*
> >        |            |                                 |                |
> >        |    o-------Y----------o   +-----+   o--------Y----------o     |
> >        |    | Forwarding Plane |   | App |   | Operational Plane |     |
> >        |    o------------------o   +-----+   o-------------------o     |
> >        |                       Network Device                          |
> >        +---------------------------------------------------------------+
> >
> >                                  Figure 2
> >
> >
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 10]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> > 4.1.  The Application Plane
> >
> >    Per [RFC7426], the Application Plane includes both applications and
> >    services.  In particular, the Application Plane incorporates the User
> >    Agent, a specialized application that interacts with the end user /
> >    operator and performs requests for Deterministic Networking services
> >    via an abstract Stream Management Entity, (SME) which may or may not
> >    be collocated with (one of) the end systems.
> >
> >    At the Application Plane, a management interface enables the
> >    negotiation of streams between end systems.  An abstraction of the
> >    stream called a Traffic Specification (TSpec) provides the
> >    representation.  This abstraction is used to place a reservation over
> >    the (Northbound) Service Interface and within the Application plane.
> >    It is associated with an abstraction of location, such as IP
> >    addresses and DNS names, to identify the end systems and eventually
> >    specify intermediate relay systems.
> >
> > 4.2.  The Controller Plane
> >
> >    The Controller Plane corresponds to the aggregation of the Control
> >    and Management Planes in [RFC7426], though Common Control and
> >    Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
> >    between management and measurement.  When the logical separation of
> >    the Control, Measurement and other Management entities is not
> >    relevant, the term Controller Plane is used for simplicity to
> >    represent them all, and the term controller refers to any device
> >    operating in that plane, whether is it a Path Computation entity or a
> >    Network Management entity (NME).  The Path Computation Element (PCE)
> >    [PCE] is a core element of a controller, in charge of computing
> >    Deterministic paths to be applied in the Network Plane.
> >
> >    A (Northbound) Service Interface enables applications in the
> >    Application Plane to communicate with the entities in the Controller
> >    Plane.
> >
> >
> > TW> could you define NME, SME and PCE in the terminology?
> >
> >    One or more PCE(s) collaborate to implement the requests from the SME
> >    as Per-Stream Per-Hop Behaviors installed in the relay systems for
> >    each individual streams.  The PCEs place each stream along a
> >    deterministic sequence of relay systems so as to respect per-stream
> >    constraints such as security and latency, and optimize the overall
> >    result for metrics such as an abstract aggregated cost.  The
> >    deterministic sequence can typically be more complex than a direct
> >    sequence and include redundancy path, with one or more packet
> >    replication and elimination points.
> >
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 11]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> > 4.3.  The Network Plane
> >
> >    The Network Plane represents the network devices and protocols as a
> >    whole, regardless of the Layer at which the network devices operate.
> >
> >    The network Plane comprises the Network Interface Cards (NIC) in the
> >    end systems, which are typically IP hosts, and relay systems, which
> >    are typically IP routers and switches.  Network-to-Network Interfaces
> >    such as used for Traffic Engineering path reservation in [RFC3209],
> >    as well as User-to-Network Interfaces (UNI) such as provided by the
> >    Local Management Interface (LMI) between network and end systems, are
> >    all part of the Network Plane.
> >
> >    A Southbound (Network) Interface enables the entities in the
> >    Controller Plane to communicate with devices in the Network Plane.
> >    This interface leverages and extends TEAS to describe the physical
> >    topology and resources in the Network Plane.
> >
> >                          Stream Management Entity
> >
> >        End                                                     End
> >            System                                               System
> >
> >       -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
> >
> >                 PCE         PCE              PCE              PCE
> >
> >       -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+--+-+-+-+-+-+-
> >
> >                   Relay      Relay      Relay      Relay
> >                   System     System     System     System
> >        NIC                                                     NIC
> >                   Relay      Relay      Relay      Relay
> >                   System     System     System     System
> >
> >                                  Figure 3
> >
> >    The relay systems (and eventually the end systems NIC) expose their
> >    capabilities and physical resources to the controller (the PCE), and
> >    update the PCE with their dynamic perception of the topology, across
> >    the Southbound Interface.  In return, the PCE(s) set the per-stream
> >    paths up, providing a Stream Characterization that is more tightly
> >    coupled to the relay system Operation than a TSpec.
> >
> >    At the Network plane, relay systems exchange information regarding
> >    the state of the paths, between adjacent systems and eventually with
> >    the end systems, and forward packets within constraints associated to
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 12]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    each stream, or, when unable to do so, perform a last resort
> >    operation such as drop or declassify.
> >
> >    This specification focuses on the Southbound interface and the
> >    operation of the Network Plane.
> >
> > 4.4.  Elements of DetNet Architecture
> >
> >    The DetNet architecture has a number of elements, discussed in the
> >    following sections:
> >
> >    a.  A model for the definition, identification, and operation of
> >        DetNet streams (Section 4.5), for use by relay systems to
> >        classify and process individual packets following per-stream
> >        rules.
> > TW> how is a packet "labeled" as part of stream?
> >
> >    b.  A model for the flow of data from an end system or through a
> >        relay system that can be used to predict the bounds for that
> >        system's impact on the QoS of a DetNet stream, without
> >        significantly constraining the method of implementing that
> >        system, for use by the Controllers to configure policing and
> >        shaping engines in Network Systems over the Southbound interface.
> >        The model includes:
> >
> >        1.  A model for queuing, transmission selection, shaping,
> >            preemption, and timing resources that can be used by an end
> >            system or relay system to control the selection of packets
> >            output on an interface.  These models must have sufficiently
> >            well-defined characteristics, both individually and in the
> >            aggregate, to give predictable results for the QoS for DetNet
> >            packets (Section 4.7).
> >
> >        2.  A model for identifying misbehaving DetNet streams and
> >            mitigating their impact on properly functioning streams
> >            (Section 4.9).
> >
> >    c.  A model for the relay system to inform the controller(s) of the
> >        information it needs for adequate path computations including:
> >
> >        1.  Systems' individual capabilities (e.g. can do replication,
> >            can do precise time).
> >
> >        2.  Link capabilities and resources (e.g. bandwidth, 0 delays,
> >            hardware deterministic support to the physical layer, ...)
> >
> >        3.  hysical
> > TW> typo
> >            resources (total and available buffers, timers,
> >            queues, etc)
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 13]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >        4.  Network Adjacencies (neighbors)
> >
> >    d.  A model for the provision of a service, by end systems, or relay
> >        systems, to forward a DetNet stream over a simple or redundant
> >        path.  The model includes:
> >
> >        1.  A model for an abstract relaying operation of either Routing
> >            or forwarding packets of a DetNet stream to a next-hop relay
> >            system, across Layer boundaries.
> >
> >        2.  A model of next-hop(s) information for replicating the
> >            packets of a DetNet stream, typically at or near the talker,
> >            merging and/or re-replicating those packets at other points
> >            in the network, and finally eliminating the duplicates,
> >            typically at or near the listener(s), in order to provide
> >            high availability (Section 3.3).
> >
> >    e.  The protocol stack model for an end system and/or a relay system
> >        should support the above elements in a manner that maximizes the
> >        applicability of existing standards and protocols to the DetNet
> >        problem, allows for the creation of new protocols where needed,
> >        thus making DetNet an add-on feature to existing networks, rather
> >        than a new way to do networking.  In particular this protocol
> >        stack supports networks in which the path from talker to
> >        listener(s) includes bridges and/or routers in any order
> >        (Section 4.10).
> >
> >    f.  A variety of models for the provisioning of DetNet streams can be
> >        envisioned, including orchestration by a central controller or by
> >        a federation of controllers, provisioning by relay systems and
> >        end systems sharing peer-to-peer protocols, by off-line
> >        configuration, or by a combination of these methods.  The
> >        provisioning models are similar to existing Layer-2 and Layer-3
> >        models, in order to minimize the amount of innovation required in
> >        this area (Section 4.12).
> >
> > 4.5.  DetNet streams
> >
> > 4.5.1.  Talker guarantees
> >
> >    DetNet streams can by synchronous or asynchronous.  The transmission
> >    of packets in synchronous DetNet streams uses time synchronization
> >    among the end and relay systems to control the flow of packets.
> >    Asynchronous DetNet streams are characterized by:
> >
> >    o  A maximum packet size;
> >
> >    o  An observation interval; and
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 14]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    o  A maximum number of transmissions during that observation
> >       interval.
> >
> >    These parameters, together with knowledge of the protocol stack used
> >    (and thus the size of the various headers added to a packet), limit
> >    the number of bit times per observation interval that the DetNet
> >    stream can occupy the physical medium.
> >
> >    The talker promises that these limits will not be exceeded.  If the
> >    talker transmits less data than this limit allows, the unused
> >    resources such as link bandwidth can be made available by the system
> >    to non-DetNet packets.  However, making those resources available to
> >    DetNet packets in other streams would serve no purpose.  Those other
> >    streams have their own dedicated resources, on the assumption that
> >    all DetNet streams can use all of their resources over a long period
> >    of time.
> >
> >    Note that there is no provision in DetNet for throttling streams; the
> >    assumption is that a DetNet stream, to be useful, must be delivered
> >    in its entirety.  That is, while any useful application is written to
> >    expect a certain number of lost packets, the real-time applications
> >    of interest to DetNet demand that the loss of data due to the network
> >    is extraordinarily infrequent.
> >
> >    Although DetNet strives to minimize the changes required of an
> >    application to allow it to shift from a special-purpose digital
> >    network to an Internet Protocol network, one fundamental shift in the
> >    behavior of network applications that is impossible to avoid--the
> >    reservation of resources before the application starts.  In the first
> >    place, a network cannot deliver finite latency and practically zero
> >    packet loss to an arbitrarily high offered load.  Secondly, achieving
> >    practically zero packet loss for unthrottled (though bandwidth
> >    limited) streams means that bridges and routers have to dedicate
> >    buffer resources to specific streams or to classes of streams.  The
> >    requirements of each reservation have to be translated into the
> >    parameters that control each system's queuing, shaping, and
> >    scheduling functions and delivered to the hosts, bridges, and
> >    routers.
> >
> > 4.5.2.  Incomplete Networks
> >
> >    The presence in the network of relay systems that are not fully
> >    capable of offering DetNet services complicates the ability of the
> >    relay systems and/or controller to allocate resources, as extra
> >    buffering, and thus extra latency, must be allocated at each point
> >    that is downstream from the non-DetNet relay system for some DetNet
> >    stream.
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 15]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> > 4.6.  Data Flow Model through Systems
> >
> > 4.7.  Queuing, Shaping, Scheduling, and Preemption
> >
> >    For this reason, the IEEE 802.1 Time-Sensitive Networking Task Group
> >    has defined a set of queuing, shaping, and scheduling algorithms that
> >    enable each bridge or router to compute the exact number of buffers
> >    to be allocated for each stream or class of streams.
> >
> > 4.8.  Coexistence with normal traffic
> >
> >    A DetNet network supports the dedication of at least 75% of the
> >    network bandwidth to DetNet streams.  But, no matter how much is
> >    dedicated for DetNet streams, It is z
> > TW> typo
> >    goal of DetNet to not interfere
> >    excessively with existing QoS schemes.  It is also important that
> >    non-DetNet traffic not disrupt the DetNet stream, of course (see
> >    Section 4.9 and Section 6).  For these reasons:
> >
> >    o  Bandwidth (transmission opportunities) not utilized by a DetNet
> >       stream are available to non-DetNet packets (though not to other
> >       DetNet streams).
> >
> >    o  DetNet streams can be shaped, in order to ensure that the highest-
> >       priority non-DetNet packet also is ensured a maximum latency.
> >
> >    o  When transmission opportunities for DetNet streams are scheduled
> >       in detail, then the algorithm constructing the schedule should
> >       leave sufficient opportunities for non-DetNet packets to satisfy
> >       the needs of the uses of the network.
> >
> >    Ideally, the net effect of the presence of DetNet streams in a
> >    network on the non-DetNet packets is primarily a reductoin
> > TW> typo
> >    in the
> >    available bandwidth.
> >
> > 4.9.  Fault Mitigation
> >
> >    One key to building robust real-time systems is to reduce the
> >    infinite variety of possible failures to a number that can be
> >    analyzed with reasonable confidence.  DetNet aids in the process by
> >    providing filters and policers to detect DetNet packets received on
> >    the wrong interface, or at the wrong time, or in too great a volume,
> >    and to then take actions such as disabling the offending packet,
> >    shutting down the offending DetNet stream, or shutting down the
> >    offending interface.
> >
> >    It is also essential that filters and service remarking
> > TW> please define "service remarking"
> >    be employed
> >    to prevent non-DetNet packets from impinging on the resources
> >    allocated to DetNet packets.
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 16]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    There exist techniques, at present and/or in various stages of
> >    standardization, that can perform these fault mitigation tasks that
> >    deliver a high probability that misbehaving systemd
> > TW> typo
> >    will have zero
> >    impact on well-behaved DetNet streams, except of course, for the
> >    receiving interface(s) immediately downstream of the misbehaving
> >    device.
> >
> > 4.10.  Protocol Stack Model
> >
> >    This section will be further developed.  See [IEEE802.1CB], Annex C,
> >    for a description of the protocol stack.  This is very much a work in
> >    progress, not a standard.  See also [IEEE802.1Qcc].
> >
> > 4.11.  Advertising resources, capabilities and adjacencies
> >
> > 4.12.  Provisioning model
> >
> > 4.12.1.  Centralized Path Computation and Installation
> >
> >    A centralized routing model, such as provided with a PCE (RFC 4655
> >    [RFC4655]), enables global and per-stream optimizations.  The model
> >    is attractive but a number of issues are left to be solved.  In
> >    particular:
> >
> >    o  whether and how the path computation can be installed by 1) an end
> >       device or 2) a Network Management entity,
> >
> >    o  and how the path is set up, either by installing state at each hop
> >       with a direct interaction between the forwarding device and the
> >       PCE, or along a path by injecting a source-routed request at one
> >       end of the path.
> >
> > 4.12.2.  Distributed Path Setup
> >
> >    Whether a distributed alternative without a PCE can be valuable
> >    should be studied as well.  Such an alternative could for instance
> >    inherit from the Resource ReSerVation Protocol [RFC5127] (RSVP)
> >    flows.
> >
> >    In a Layer-2 only environment, or as part of a layered approach to a
> >    mixed environment, IEEE 802.1 also has work, either completed or in
> >    progress.  [IEEE802.1Q-2011] Clause 35 describes SRP, a peer-to-peer
> >    protocol for Layer-2 roughly analogous to RSVP.  Almost complete is
> >    [IEEE802.1Qca], which defines how ISIS can provide multiple disjoint
> >    paths or distribution trees.  Also in progress is [IEEE802.1Qcc],
> >    which expands the capabilities of SRP.
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 17]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> > 5.  Related IETF work
> >
> > 5.1.  Deterministic PHB
> >
> >    [I-D.svshah-tsvwg-deterministic-forwarding] defines a Differentiated
> >    Services Per-Hop-Behavior (PHB) Group called Deterministic Forwarding
> >    (DF).  The document describes the purpose and semantics of this PHB.
> >    It also describes creation and forwarding treatment of the service
> >    class.  The document also describes how the code-point can be mapped
> >    into one of the aggregated Diffserv service classes [RFC5127].
> >
> > 5.2.  6TiSCH
> >
> >    Industrial process control already leverages deterministic wireless
> >    Low power and Lossy Networks (LLNs) to interconnect critical
> >    resource-constrained devices and form wireless mesh networks, with
> >    standards such as [ISA100.11a] and [WirelessHART].
> >
> >    These standards rely on variations of the [IEEE802154e] timeSlotted
> >    Channel Hopping (TSCH) [I-D.ietf-6tisch-tsch] Medium Access Control
> >    (MAC), and a form of centralized Path Computation Element (PCE), to
> >    deliver deterministic capabilities.
> >
> >    The TSCH MAC benefits include high reliability against interference,
> >    low power consumption on characterized streams, and Traffic
> >    Engineering capabilities.  Typical applications are open and closed
> >    control loops, as well as supervisory control streams and management.
> >
> >    The 6TiSCH Working Group focuses only on the TSCH mode of the IEEE
> >    802.15.4e standard.  The WG currently defines a framework for
> >    managing the TSCH schedule.  Future work will standardize
> >    deterministic operations over so-called tracks as described in
> >    [I-D.ietf-6tisch-architecture].  Tracks are an instance of a
> >    deterministic path, and the DetNet work is a prerequisite to specify
> >    track operations and serve process control applications.
> >
> >    [RFC5673] and [I-D.ietf-roll-rpl-industrial-applicability] section
> >    2.1.3.  and next discusses application-layer paradigms, such as
> >    Source-sink (SS) that is a Multipeer to Multipeer (MP2MP) model that
> >    is primarily used for alarms and alerts, Publish-subscribe (PS, or
> >    pub/sub) that is typically used for sensor data, as well as Peer-to-
> >    peer (P2P) and Peer-to-multipeer (P2MP) communications.  Additional
> >    considerations on Duocast and its N-cast generalization are also
> >    provided for improved reliability.
> >
> >
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 18]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> > 6.  Security Considerations
> >
> >    Security in the context of Deterministic Networking has an added
> >    dimension; the time of delivery of a packet can be just as important
> >    as the contents of the packet, itself.  A man-in-the-middle attack,
> >    for example, can impose, and then systematically adjust, additional
> >    delays into a link, and thus disrupt or subvert a real-time
> >    application without having to crack any encryption methods employed.
> >    See [RFC7384] for an exploration of this issue in a related context.
> >
> >    Furthermore, in a control system where millions of dollars of
> >    equipment, or even human lives, can be lost if the DetNet QoS is not
> >    delivered, one must consider not only simple equipment failures,
> >    where the box or wire instantly becomes perfectly silent, but bizarre
> >    errors such as can be coused
> > TW> typo
> >    by software failures.  Because there is
> >    essentiall
> > TW> typo
> >    no limit to the kinds of failures that can occur,
> >    protecting against realistic equipment failures is indistinguishable,
> >    in most cases, from protecting against malicious behavior, whether
> >    accidental or intentional.  See also Section 4.9.
> >
> >    Security must cover:
> >
> >    o  the protection of the signaling protocol
> >
> >    o  the authentication and authorization of the controlling systems
> >
> >    o  the identification and shaping of the streams
> >
> > 7.  IANA Considerations
> >
> >    This document does not require an action from IANA.
> >
> > 8.  Acknowledgements
> >
> >    The authors wish to thank Jouni Korhonen, Erik Nordmark, George
> >    Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
> >    Shitanshu Shah, Craig Gunther, Rodney Cummings, Wilfried Steiner,
> >    Marcel Kiessling, Karl Weber, Ethan Grossman and Pat Thaler, for
> >    their various contribution with this work.
> >
> > 9.  Informative References
> >
> >    [AVnu]     http://www.avnu.org/, "The AVnu Alliance tests and
> >               certifies devices for interoperability, providing a simple
> >               and reliable networking solution for AV network
> >               implementation based on the Audio Video Bridging (AVB)
> >               standards.", .
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 19]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    [CCAMP]    IETF, "Common Control and Measurement Plane",
> >               <https://datatracker.ietf.org/doc/charter-ietf-ccamp/>.
> >
> >    [HART]     www.hartcomm.org, "Highway Addressable Remote Transducer,
> >               a group of specifications for industrial process and
> >               control devices administered by the HART Foundation", .
> >
> >    [HSR-PRP]  IEC, "High availability seamless redundancy (HSR) is a
> >               further development of the PRP approach, although HSR
> >               functions primarily as a protocol for creating media
> >               redundancy while PRP, as described in the previous
> >               section, creates network redundancy.  PRP and HSR are both
> >               described in the IEC 62439 3 standard.",
> >               <http://webstore.iec.ch/webstore/webstore.nsf/
> >               artnum/046615!opendocument>.
> >
> >    [I-D.finn-detnet-problem-statement]
> >               Finn, N. and P. Thubert, "Deterministic Networking Problem
> >               Statement", draft-finn-detnet-problem-statement-01 (work
> >               in progress), October 2014.
> >
> >    [I-D.ietf-6tisch-architecture]
> >               Thubert, P., Watteyne, T., Struik, R., and M. Richardson,
> >               "An Architecture for IPv6 over the TSCH mode of IEEE
> >               802.15.4e", draft-ietf-6tisch-architecture-05 (work in
> >               progress), January 2015.
> >
> >    [I-D.ietf-6tisch-tsch]
> >               Watteyne, T., Palattella, M., and L. Grieco, "Using
> >               IEEE802.15.4e TSCH in an IoT context: Overview, Problem
> >               Statement and Goals", draft-ietf-6tisch-tsch-05 (work in
> >               progress), January 2015.
> >
> >    [I-D.ietf-roll-rpl-industrial-applicability]
> >               Phinney, T., Thubert, P., and R. Assimiti, "RPL
> >               applicability in industrial networks", draft-ietf-roll-
> >               rpl-industrial-applicability-02 (work in progress),
> >               October 2013.
> >
> >    [I-D.svshah-tsvwg-deterministic-forwarding]
> >               Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
> >               draft-svshah-tsvwg-deterministic-forwarding-03 (work in
> >               progress), March 2015.
> >
> >    [IEEE802.1AS-2011]
> >               IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
> >               2011, <http://standards.ieee.org/getIEEE802/
> >               download/802.1AS-2011.pdf>.
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 20]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    [IEEE802.1BA-2011]
> >               IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011,
> >               <http://standards.ieee.org/getIEEE802/
> >               download/802.1BA-2011.pdf>.
> >
> >    [IEEE802.1CB]
> >               IEEE, "Seamless Redundancy (IEEE Draft P802.1CB)", 2015,
> >               <http://p8021:[e-mail]/1/files/private/
> >               cb-drafts/>.
> >
> >    [IEEE802.1Q-2011]
> >               IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2011", 2011,
> >               <http://standards.ieee.org/getIEEE802/
> >               download/802.1Q-2011.pdf>.
> >
> >    [IEEE802.1Qat-2010]
> >               IEEE, "Stream Reservation Protocol (IEEE 802.1Qat-2010)",
> >               2010, <http://standards.ieee.org/getIEEE802/
> >               download/802.1Qat-2010.pdf>.
> >
> >    [IEEE802.1Qav]
> >               IEEE, "Forwarding and Queuing (IEEE 802.1Qav-2009)", 2009,
> >               <http://standards.ieee.org/getIEEE802/
> >               download/802.1Qav-2009.pdf>.
> >
> >    [IEEE802.1Qca]
> >               IEEE, "Path Control and Reservation", 2015,
> >               <http://p8021:[e-mail]/1/files/private/
> >               ca-drafts/>.
> >
> >    [IEEE802.1Qcc]
> >               IEEE, "Stream Reservation Protocol (SRP) Enhancements and
> >               Performance Improvements", 2015,
> >               <http://p8021:[e-mail]/1/files/private/
> >               cc-drafts/>.
> >
> >    [IEEE802.1TSNTG]
> >               IEEE Standards Association, "IEEE 802.1 Time-Sensitive
> >               Networks Task Group", 2013,
> >               <http://www.IEEE802.org/1/pages/avbridges.html>.
> >
> >    [IEEE802154]
> >               IEEE standard for Information Technology, "IEEE std.
> >               802.15.4, Part. 15.4: Wireless Medium Access Control (MAC)
> >               and Physical Layer (PHY) Specifications for Low-Rate
> >               Wireless Personal Area Networks", June 2011.
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 21]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    [IEEE802154e]
> >               IEEE standard for Information Technology, "IEEE std.
> >               802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area
> >               Networks (LR-WPANs) Amendment 1: MAC sublayer", April
> >               2012.
> >
> >    [ISA100.11a]
> >               ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
> >               also IEC 62734", 2011, < http://www.isa100wci.org/en-
> >               US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch-
> >               WEB-ETSI.aspx>.
> >
> >    [ODVA]     http://www.odva.org/, "The organization that supports
> >               network technologies built on the Common Industrial
> >               Protocol (CIP) including EtherNet/IP.", .
> >
> >    [PCE]      IETF, "Path Computation Element",
> >               <https://datatracker.ietf.org/doc/charter-ietf-pce/>.
> >
> >    [Profinet]
> >               http://us.profinet.com/technology/profinet/, "PROFINET is
> >               a standard for industrial networking in automation.",
> >               <http://us.profinet.com/technology/profinet/>.
> >
> >    [RFC2205]  Braden, B., Zhang, L., Berson, S., Herzog, S., and S.
> >               Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
> >               Functional Specification", RFC 2205, September 1997.
> >
> >    [RFC3209]  Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
> >               and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
> >               Tunnels", RFC 3209, December 2001.
> >
> >    [RFC4655]  Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
> >               Element (PCE)-Based Architecture", RFC 4655, August 2006.
> >
> >    [RFC5127]  Chan, K., Babiarz, J., and F. Baker, "Aggregation of
> >               Diffserv Service Classes", RFC 5127, February 2008.
> >
> >    [RFC5673]  Pister, K., Thubert, P., Dwars, S., and T. Phinney,
> >               "Industrial Routing Requirements in Low-Power and Lossy
> >               Networks", RFC 5673, October 2009.
> >
> >    [RFC7384]  Mizrahi, T., "Security Requirements of Time Protocols in
> >               Packet Switched Networks", RFC 7384, October 2014.
> >
> >
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 22]
> > Internet-Draft    Deterministic Networking Architecture       March 2015
> >
> >
> >    [RFC7426]  Haleplidis, E., Pentikousis, K., Denazis, S., Hadi Salim,
> >               J., Meyer, D., and O. Koufopavlou, "Software-Defined
> >               Networking (SDN): Layers and Architecture Terminology",
> >               RFC 7426, January 2015.
> >
> >    [TEAS]     IETF, "Traffic Engineering Architecture and Signaling",
> >               <https://datatracker.ietf.org/doc/charter-ietf-teas/>.
> >
> >    [WirelessHART]
> >               www.hartcomm.org, "Industrial Communication Networks -
> >               Wireless Communication Network and Communication Profiles
> >               - WirelessHART - IEC 62591", 2010.
> >
> > Authors' Addresses
> >
> >    Norm Finn
> >    Cisco Systems
> >    170 W Tasman Dr.
> >    San Jose, California  95134
> >    USA
> >
> >    Phone: [phone]
> >    Email: [e-mail]
> >
> >
> >    Pascal Thubert
> >    Cisco Systems
> >    Village d'Entreprises Green Side
> >    400, Avenue de Roumanille
> >    Batiment T3
> >    Biot - Sophia Antipolis  06410
> >    FRANCE
> >
> >    Phone: [phone]
> >    Email: [e-mail]
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Finn & Thubert         Expires September 10, 2015              [Page 23]
> >
> > _______________________________________________
> > 6tisch mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/6tisch
> >  _______________________________________________
> > detnet mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/detnet
> >
> >
> >
> > --
> > This message has been scanned for viruses and
> > dangerous content by MailScanner, and is
> > believed to be clean.
> >
> > _______________________________________________
> > 6tisch mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/6tisch
> >
> 
> 
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
> 
> _______________________________________________
> 6tisch mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/6tisch