Re: [Detnet] WG Last Call: draft-ietf-detnet-architecture-05

[János Farkas <[e-mail]>]

Hi David,

Please see in-line.

Thank you!
Janos


On 7/6/2018 11:50 PM, Black, David wrote:
>
> -- Bridged path definition
>
> >  Or should we do something else?
>
> > There is only one occurrence of "bridged path" in the text in brackets.
>
> That sounds like a better suggestion – change that occurrence and 
> remove the definition of “bridged path” from the draft. That 
> occurrence is in Section 4.1.1:
>
>    Congestion protection
>
>            The DetNet transport layer provides congestion protection.
>
>            See Section 4.5.  The actual queuing and shaping mechanisms
>
>            are typically provided by underlying subnet layers, these can
>
>            be closely associated with the means of providing paths for
>
>           DetNet flows (e.g., MPLS LSPs or bridged paths), the path and
>
>            the congestion protection are conflated in this figure.
>
> Perhaps: (e.g., MPLS LSPs or bridged paths) -> (MPLS LSPs or other 
> network paths)
>
> or just delete the entire parenthetical example.
>
OK. Let's just delete the entire parenthetical example.

> -- Section 4
>
> > We could add some introductory text to 4.1 or 4.1.1, for instance:
> > The functionality needed for DetNet (Section 3) are implemented in 
> the DetNet layer which comprises the DetNet service layer and the 
> DetNet transport layer.
>
> > The DetNet service layer functions rely on flow identification and 
> sequencing. The DetNet transport layer function may rely on flow 
> identification.
> >
> > What do you think?
>
> I prefer functional (what it does) characterization to mechanistic 
> (how it works) characterization in an overview, e.g., excerpting from 
> what I originally wrote:
>
> ·The DetNet Service layer provides timely reliable in-order DetNet 
> flow delivery to applications that use DetNet.
>
> ·The DetNet Transport layer insulates DetNet flows from interference 
> by network mechanisms, primarily queuing and routing
>
I understand the principle.

What about just copying the two definitions here:
DetNet service layer is the layer at which A DetNet Service, such as 
congestion or service protection is provided.
DetNet transport layer optionally provides congestion protection for 
DetNet flows over paths provided by the underlying network.


>
> > Should we also rearrange, e.g., indent the text after Figure 2 such that
> > Packet sequencing, Duplicate elimination, Flow replication, Flow 
> merging, Packet encoding, and Packet decoding get under Service layer
> > and
> > Congestion protection and Explicit routes get under Transport layer?
>
> No, I don’t think that’s necessary – Figure 2 provides sufficient 
> structure.
>
OK.

> Plus a few more minor comments inline …
>
> Thanks, --David
>
> *From:*János Farkas [mailto:[e-mail]]
> *Sent:* Wednesday, July 4, 2018 2:15 PM
> *To:* Black, David
> *Cc:* DetNet WG
> *Subject:* Re: [Detnet] WG Last Call: draft-ietf-detnet-architecture-05
>
> Hi David,
>
> Thank you very much for your review and comments!
>
> Please see in-line.
>
> On 6/29/2018 4:10 PM, Black, David wrote:
>
>     The -06 version of this draft is much improved over the -05
>     version – lots of diligent work by the authors is evident and
>     appreciated.
>
>     I have two minor content concerns and some smaller editorial items.
>
>     ** Content
>
>     -- Section 2.2
>
>     The definition of “bridged path” looks like a definition of
>     “bridged forwarding” – the definition needs to encompass the
>     notion of a path that involves more than one bridge.
>
>        bridged path
>
>                A VLAN bridge uses the VLAN ID and the destination MAC
>
>                address to select the outbound port hence the path for a
>
>                frame.
>
> You are right, it looks like the emphasis is on bridge forwarding 
> rather than path.
> However, the superposition of the outbound ports provides a path, 
> which was tried to be captured by "hence the path".
> The idea was to keep it simple not dive into the details, which are 
> not necessary in this document.
>
> Would it be good to update to:
>
>    bridged path
>            A VLAN bridge uses the VLAN ID and the destination MAC
>            address to select the outbound port. The superposition
>            of the outbound ports is the path for a frame.
>
> ?
>
> Or should we do something else?
> There is only one occurrence of "bridged path" in the text in brackets.
>
>
>     -- Section 4
>
>     Figure 2 in Section 4.1.1 introduces the Detnet service and
>     transport layers.   The associated text quickly dives into
>     explaining the functionality in those layers, but it would be
>     helpful to start with an initial high level description of each of
>     those layers that helps motivate their separation.  It appears that:
>
>     - The DetNet Service layer provides timely reliable in-order
>     DetNet flow delivery to applications that use DetNet.   A DetNet
>     flow at the top of the DetNet Service layer may or may not be a
>     compound flow at the bottom of that layer.
>
>     - The DetNet Transport layer insulates DetNet flows (at the bottom
>     of the DetNet Service layer) from interference by network
>     mechanisms, primarily queuing and routing, where congestion is an
>     indication of queueing interference.  A DetNet flow at the top of
>     the DetNet Transport layer may or may not be a member flow that is
>     part of a compound flow, depending on what is done in the DetNet
>     Service layer.
>
>
> You are right, section 4 jumps into the details without much of an 
> introduction. I guess we had a kind of chicken and egg problem here. 
> Figure 4 provides a higher level view than Figure 2. However, Figure 4 
> captures different aspects, dives into the details of data plane 
> options. So, it seems better to keep the order.
>
> We could add some introductory text to 4.1 or 4.1.1, for instance:
> The functionality needed for DetNet (Section 3) are implemented in the 
> DetNet layer which comprises the DetNet service layer and the DetNet 
> transport layer. The DetNet service layer functions rely on flow 
> identification and sequencing. The DetNet transport layer function may 
> rely on flow identification.
>
> What do you think?
>
> Should we also rearrange, e.g., indent the text after Figure 2 such that
> Packet sequencing, Duplicate elimination, Flow replication, Flow 
> merging, Packet encoding, and Packet decoding get under Service layer
> and
> Congestion protection and Explicit routes get under Transport layer?
>
>
>
>     There’s a related paragraph of text between Figures 3 and 4 in
>     Section 4.1.2, but it doesn’t provide useful intuition about which
>     functionality belongs in which layer.
>
>     ** Editorial
>
>
>     -- Section 2.1
>
>     “expected” seems off in the definition of reservation:
>
>        reservation
>
>                The set of resources allocated between a source and one or
>
>                more destinations through transit nodes and subnets
>
>                associated with a DetNet flow, to provide the expected
>     DetNet
>
>                Service.
>
>     Suggest either “to provide the provisioned DetNet Service to that
>     flow”
>
> I like this one, will update to this one.
>
>
>     or “to provide the specified DetNet Service to that flow.”
>
>     -- Section 3.1
>
>        o  An upper bound on out-of-order packet delivery.  It is worth
>
>           noting that some DetNet applications are unable to tolerate any
>
>           out-of-order delivery.
>
>     Is that a sequence bound or a time bound or both?   I suspect that
>     a sequence bound (maximum degree of reordering in a flow,
>     independent of flow rate) was intended.  An analogous
>     clarification would be helpful in section 3.2.2.1 .
>
> It is intentionally kept high level in the architecture draft. Details 
> are to be provided by the flow information model draft. Please see my 
> previous mail: 
> https://mailarchive.ietf.org/arch/msg/detnet/AWG2eemg_AWsvF2FwtREyd_xtC8.
>
> */[David>] /*OK.
>
>     -- Section 3.2.3
>
>     I don’t understand “defined” in:
>
>        Even the use of
>
>        redundant paths through a network defined, e.g., by [RFC6372]
>     do not
>
>        eliminate the chances of packet loss.
>
>     Was “e.g., as defined by [RFC6372],” intended?
>
> Yes, this was the intention, will update to this one.
>
>        Many real-time networks rely on physical rings or chains of
>     two-port
>
>        devices, with a relatively simple ring control protocol.
>
>     How does a “ring control protocol” control “chains”? Suggest:
>     “ring control protocol” -> “control protocol”
>
>     Also, are paired chains between common endpoints intended, or a
>     more general notion of chains?
>
> This text refers to typical factory deployment, which is chain or ring.
> It seems better to keep “ring control protocol” because they are often 
> very different from generic control protocols applicable to mesh networks.
>
> Would flipping chain and ring resolve your concern?
>
>    Many real-time networks rely on physical chains or rings of two-port
>    devices, with a relatively simple ring control protocol.
>
>     */[David>] /*It feels like “ring control protocol” is an important
>     phrase to retain.   In the original text, would it be reasonable
>     to add “(applies to both rings and chains)” to the end of the
>     sentence?
>
Well, I guess the point is that there are many specific ring protocols, 
but we are looking for a generic solution.
What about deleting "chains" from the sentence?

>        Explicit routes can be established various ways,
>
>     established various -> established in various
>
> OK.
>
>
>        This is
>
>        irrespective of the distribution method used, also applies to
>     service
>
>        protection over explicit routes.
>
>     used, also -> used, and also
>
> OK.
>
>
>     -- Figure 5 (Section 4.1.3)
>
>     The UNI acronym is introduced without expansion or definition –
>     please correct that, e.g., by defining UNI in Section 2.1.
>
> DetNet-UNI is there among the definitions. It seems enough to me.
>
> */[David>] /*That’s fine – I overlooked that definition, sorry.
>
>     -- Figure 9 (Section 4.7.2)
>
>     I think I understand what’s going on, but the use of “add/remove”
>     in the figure should be explained in associated text.
>
> What about updating the sentence above Figure 9 to:
> A packet with corresponding Flow-IDs is illustrated in Figure 9, which 
> also indicates where Flow-ID can be added or removed.
>
> */[David>] /*Sure, with one minor edit: “where Flow-ID” -> “where each 
> Flow-ID”
>
OK.

>     Thanks, --David
>
>
> Thank you!
> Janos
>
>
>
>     *From:*detnet [mailto:[e-mail]] *On Behalf Of
>     *János Farkas
>     *Sent:* Thursday, June 28, 2018 9:09 PM
>     *To:* Lou Berger; Stewart Bryant; [e-mail]
>     <mailto:[e-mail]> >> DetNet WG
>     *Subject:* Re: [Detnet] WG Last Call:
>     draft-ietf-detnet-architecture-05
>
>     Dear all,
>
>     Off-line discussions among Lou, Stewart, and authors followed the
>     discussions to properly address the WGLC comments, including the
>     detailed comments.
>
>     A new revision of the draft has been uploaded:
>     draft-ietf-detnet-architecture-06.
>
>     In addition to the changes already described in this thread, the
>     following bigger changes have been made to the draft:
>
>
>     *Section 2.1 Terms used in this document*
>
>     Some definitions refined as suggested by the detailed comments
>
>     New definitions have been added:
>
>        "allocation
>                Resources are dedicated to support a DetNet flow. 
>     Depending
>                on an implementation, the resource may be reused by non-
>                DetNet flows when it is not used by the DetNet flow.
>
>
>        PEF     A Packet Elimination Function (PEF) eliminates duplicate
>                copies of packets to prevent excess packets flooding the
>                network or duplicate packets being sent out of the DetNet
>                domain.  PEF can be implemented by an edge node, a relay
>                node, or an end system.
>
>       PRF     A Packet Replication Function (PRF) replicates DetNet flow
>                packets and forwards them to one or more next hops in the
>                DetNet domain.  The number of packet copies sent to
>     each next
>                hop is a DetNet flow specific parameter at the node
>     doing the
>                replication.  PRF can be implemented by an edge node, a
>     relay
>                node, or an end system.
>
>        PREOF   Collective name for Packet Replication, Elimination, and
>                Ordering Functions.
>
>        POF     A Packet Ordering Function (POF) re-orders packets within a
>                DetNet flow that are received out of order. This function
>                can be implemented by an edge node, a relay node, or an end
>                system.
>
>       DetNet service proxy
>                Maps between App-flows and DetNet flows.
>
>        bridged path
>                A VLAN bridge uses the VLAN ID and the destination MAC
>                address to select the outbound port hence the path for a
>                frame."
>
>
>     *Section 3.1 Primary goals defining the DetNet QoS*
>
>     A new QoS aspect has been added:
>        "o  An upper bound on out-of-order packet delivery. It is worth
>           noting that some DetNet applications are unable to tolerate any
>           out-of-order delivery."
>
>
>     The 3rd paragraph on loss on page 8 after the bullet list has been
>     extended to:
>        "After congestion, the most important contributions to packet
>     loss are
>        typically from random media errors and equipment failures.  Service
>        protection is the name for the mechanisms used by DetNet to address
>        these losses.  The mechanisms employed are constrained by the
>        requirement to meet the users' latency requirements. Packet
>        replication and elimination (Section 3.2.2) and packet encoding
>        (Section 3.2.2.3) are described in this document to provide service
>        protection; others may be found.  For instance, packet encoding can
>        be used to provide service protection against random media errors,
>        packet replication and elimination can be used to provide service
>        protection against equipment failures.  This mechanism distributes
>        the contents of DetNet flows over multiple paths in time and/or
>        space, so that the loss of some of the paths does need not
>     cause the
>        loss of any packets."
>
>
>     *3.2.2.  Service Protection*
>
>     Service protection is used as a more generic term. Introductory
>     text added:
>        "Service protection aims to mitigate or eliminate packet loss
>     due to
>        equipment failures, random media and/or memory faults.  These types
>        of packet loss can be greatly reduced by spreading the data over
>        multiple disjoint forwarding paths.  Various service protection
>        methods are described in [RFC6372], e.g., 1+1 linear protection.
>        This section describes the functional details of an additional
>     method
>        in Section 3.2.2.2, which can be implemented as described in
>        Section 3.2.2.3 or as specified in [I-D.ietf-detnet-dp-sol-mpls] in
>        order to provide 1+n hitless protection.  The appropriate service
>        protection mechanism depends on the scenario and the requirements."
>
>
>     New sub-section added:
>
>     "3.2.2.1.  In-Order Delivery
>
>        Out-of-order packet delivery can be a side effect of service
>        protection.  Packets delivered out-of-order impact the amount of
>        buffering needed at the destination to properly process the
>     received
>        data.  Such packets also influence the jitter of a flow.  The
>     DetNet
>        service includes maximum allowed misordering as a constraint.  Zero
>        misordering would be a valid service constraint to reflect that the
>        end system(s) of the flow cannot tolerate any out-of-order
>     delivery.
>        Service protection may provide a mechanism to support in-order
>        delivery."
>
>
>     *3.2.2.2. Packet Replication and Elimination*
>
>     New bullet added as the last one:
>        "o  The Packet Ordering Function (POF) uses the sequencing
>     information
>           to re-order a DetNet flow's packets that are received out of
>           order."
>
>     New sentence added after the bullet list:
>     "The order in which a node applies the PEF and the PRF to a DetNet
>     flow is implementation specific."
>
>     2nd paragraph after the bullet list has been updated to:
>     "Some service protection mechanisms rely on switching from one flow to
>        another when a failure of a flow is detected. Contrarily, packet
>        replication and elimination combines the DetNet member flows sent
>        along multiple different paths, and performs a packet-by-packet
>        selection of which to discard, e.g., based on sequencing
>     information."
>
>
>     *3.2.3.  Explicit routes*
>
>     Out-of-order aspect added to the first paragraph, which is about
>     distributed routing:
>     "Furthermore, out-of-order
>     packet delivery can be a side effect of route changes."
>
>     New sentence added to the 3rd paragraph:
>     "Explicit routes can be established various
>     ways, e.g., with RSVP-TE [RFC3209], with Segment Routing (SR)
>     [I-D.ietf-spring-segment-routing], via a Software Defined Networking
>     approach [RFC7426], with IS-IS [RFC7813], etc."
>
>     New paragraph added:
>        "Out-of-order packet delivery can be a side effect of
>     distributing a
>        single flow over multiple paths especially when there is a change
>        from one path to another when combining the flow. This is
>        irrespective of the distribution method used, also applies to
>     service
>        protection over explicit routes.  As described in Section 3.2.2.1,
>        out-of-order packets influence the jitter of a flow and impact the
>        amount of buffering needed to process the data; therefore, DetNet
>        service includes maximum allowed misordering as a constraint.  The
>        use of explicit routes helps to provide in-order delivery because
>        there is no immediate route change with the network topology,
>     but the
>        changes are plannable as they are between the different explicit
>        routes."
>
>     *
>     4.1.1. Representative Protocol Stack Model*
>
>     "Explicit routes" have been added to Figure 2 with the
>     corresponding explanation:
>      "Explicit routes
>                The DetNet transport layer provides mechanisms to
>     ensure that
>                fixed paths are provided for DetNet flows. These explicit
>                paths avoid the impact of network convergence."
>
>
>     Section 4.11 Connected islands vs. networks of v05 has been
>     deleted because it was just a leftover from early drafts on what
>     DetNet WG should do; which are covered by the charter anyways.
>
>
>     References have been cleaned up and brought up-to-date.
>
>
>     Refinements have been implemented in the draft according to Lou's
>     detailed comments. They are not listed here because they are minor
>     changes.
>
>
>     Best regards,
>     Janos
>
>
>     On 6/12/2018 2:48 PM, Lou Berger wrote:
>
>
>         Balázs,
>
>         Thanks for the response -- please see below.
>
>         ----------
>         On June 12, 2018 4:07:35 AM Balázs Varga A
>         <[e-mail]>
>         <mailto:[e-mail]> wrote:
>
>
>
>             Hi Lou, Thanks for the comments. See reactions inline.
>             Document update in progress. Cheers Bala'zs
>
>             -----Original Message-----
>             From: detnet <[e-mail]>
>             <mailto:[e-mail]> On Behalf Of Lou Berger
>             Sent: 2018. június 1. 22:42
>             To: DetNet WG <[e-mail]> <mailto:[e-mail]>;
>             [e-mail]
>             <mailto:[e-mail]>
>             Subject: [Detnet] Promised comments on
>             draft-ietf-detnet-architecture
>
>             Hi,
>
>                  I have a number of high level comments on the
>             document that I'd like to  raise below.  I also have a
>             number of more editorial/specific comments that  I'd like
>             to review directly with the authors and then have them
>             report back  on changes -- if any turn out to be more
>             substantive discussions from the  author's perspective,
>             I'll raise these on the list separately.
>
>         ...
>
>
>
>             - WRT Section 4.4.3, I think the inclusion of a
>             distributed control plane in the "Network Plane" is
>             inconsistent with other functional definitions and
>             conflates where a function resides from the actual
>             function and that whether control is implemented in a
>             fully centralized, fully distributed or some hybrid form
>             doesn't fundamentally change  the control function's role
>             -- therefore I think the sections 4.4.2 and .3 should be
>             revised accordingly
>             [Balázs Varga A] Agree in principal. Let's discuss the
>             details.
>
>         Okay - I'll work with you off line and we can report back the
>         results/proposed changes.
>
>
>
>             - In several places it's not clear that DetNet service is
>             always a L3 service which is controlled using L3
>             identifiers, i.e., IP addresses -- this is true even in
>             the MPLS service case and the TSN over MPLS case. I think
>             this is an important point to be clear on in the document.
>             [Balázs Varga A] I am not sure. DetNet service is always
>             provided over a L3 network (IP or MPLS), that is fine.
>             However the service itself can be L2 or L3. In case of TSN
>             Ethernet frames are transported, so it is a L2 service. In
>             case of IP end systems IP packets are transported so it is
>             a L3 service.
>
>
>         Humm - While I agree that DetNet is providing an (enhanced)
>         L2VPN service, it is not itself providing control or service
>         of L2 devices -- this is TSN's job.  The fact that DetNet is
>         all about behavior of L3 nodes (i.e., IP and PW/MPLS) and not
>         L2 nodes (i.e., TSN bridges) is something the architecture
>         should make unambiguous.
>
>         Thanks,
>         Lou
>
>
>
>             Please let me know what you think.
>
>             Cheers,
>
>             Lou
>
>
>             _______________________________________________
>             detnet mailing list
>             [e-mail] <mailto:[e-mail]>
>             https://www.ietf.org/mailman/listinfo/detnet
>             _______________________________________________
>             detnet mailing list
>             [e-mail] <mailto:[e-mail]>
>             https://www.ietf.org/mailman/listinfo/detnet
>
>
>
>
>
>     On 6/12/2018 6:27 PM, Stewart Bryant wrote:
>
>         Dear Bala'zs
>
>         Thank you your for your consideration of these points. I will
>         just pick up a few that need some further thought:
>
>
>
>
>                 DetNet transit node
>
>                        A node operating at the DetNet transport layer,
>             that utilizes
>
>                        link layer and/or network layer switching
>             across multiple
>
>                        links and/or sub-networks to provide paths for
>             DetNet service
>
>                        layer functions. Optionally provides congestion
>             protection
>
>                        over those paths.  An MPLS LSR is an example of
>             a DetNet
>
>                        transit node.
>
>             SB> In that example it would have to be a DetNet
>             enable/aware LSR. An
>
>             SB> ordinary LSR would not know anything about DetNet.
>
>             [Balázs Varga A] No, A DetNet aware LSR would be a relay
>             node (S-PE).
>
>
>         I think the confusion is what "DetNet Transport Layer" means.
>         This
>         technology touches on Transport Layer in  the L4 sense, and the
>         Transport Network Layer as in the packet network that carries
>         L3 packets.
>
>         So I think that we need to clarify whether a DetNet transit node
>         is an S-PE (i.e. a a transit node in the DetNet layer), or a P
>         node
>         (i.e. a transit node that is carrying DetNet packets but could be
>         carrying any sort of MPLS packet)
>
>
>
>             ============
>
>                These three techniques can be applied independently,
>             giving eight
>
>                possible combinations, including none (no DetNet),
>             although some
>
>                combinations are of wider utility than others.  This
>             separation keeps
>
>                the protocol stack coherent and maximizes
>             interoperability with
>
>                existing and developing standards in this (IETF) and
>             other Standards
>
>                Development Organizations.  Some examples of typical
>             expected
>
>                combinations:
>
>                o  Explicit routes plus service protection are exactly
>             the techniques
>
>                   employed by [HSR-PRP].  Explicit routes are achieved
>             by limiting
>
>                   the physical topology of the network, and the
>             sequentialization,
>
>                   replication, and duplicate elimination are
>             facilitated by packet
>
>                   tags added at the front or the end of Ethernet frames.
>
>             SB> ER can be done virtually as well as physically. RSVP
>             is a type of
>
>             SB> ER, as is Adj-SID based SR, and we can design other types.
>
>             [Balázs Varga A] Agree, but these are examples. Statement
>             is for HSR-PRP.
>
>         So the question is whether we should expand the set of
>         examples, particularly
>         to more accessible ones.
>
>         ============
>
>
>                                 Packet replication and elimination
>
>                             > > > > > > > > > relay > > > > > > > >
>
>                            > /------------+ R node E +------------\ >
>
>                           > /                  v + ^               \ >
>
>                   end    R +                   v | ^                +
>             E end
>
>                   system +                   v | ^                + system
>
>                           > \                  v + ^               / >
>
>                            > \------------+ R relay E +-----------/ >
>
>                             > > > > > > > > >  node > > > > > > > >
>
>             Figure 1
>
>                Packet replication and elimination does not react to
>             and correct
>
>                failures; it is entirely passive.  Thus, intermittent
>             failures,
>
>             SB> I think it copes with intermittent failures OK.
>
>             [Balázs Varga A] Yes, PRF and PEF can eliminate the effect
>             of such failures. But text
>
>             intends to say that it is passive. It is not reacting to
>             such failures. No change in text.
>
>
>         You say that PREF does not correct failures. I would contend
>         that is exactly
>         what it does. Sure it does not react but it does correct, and
>         it does
>         address intermittent failures.
>
>
>             ===========
>
>                transported between the peer end systems.  Therefore,
>             the following
>
>                possible types / formats of a DetNet flow are
>             distinguished in this
>
>                document:
>
>                o  App-flow: native format of a DetNet flow.  It does
>             not contain any
>
>                   DetNet related attributes.
>
>                o  DetNet-t-flow: specific format of a DetNet flow. 
>             Only requires
>
>                   the congestion / latency features provided by the
>             Detnet transport
>
>                   layer.
>
>                o  DetNet-s-flow: specific format of a DetNet flow. 
>             Only requires
>
>                   the replication/elimination feature ensured by the
>             DetNet service
>
>                   layer.
>
>                o  DetNet-st-flow: specific format of a DetNet flow. 
>             It requires
>
>                   both DetNet service layer and DetNet transport layer
>             functions
>
>                   during forwarding.
>
>             SB> I find the relisting of these types confusing.
>             Wheren't they defined
>
>             SB> in the section above?
>
>             [Balázs Varga A] This is inline with the previous section
>             " Grouping of end systems ".
>
>         Surely if we have defined them we never need to do anything
>         but name them in
>         later sections. Redefinition is never a good idea because it
>         often leads to
>         conflicting definitions.
>
>
>
>             ============
>
>                [HSR-PRP]  IEC, "High availability seamless redundancy
>             (HSR) is a
>
>                           further development of the PRP approach,
>             although HSR
>
>                           functions primarily as a protocol for
>             creating media
>
>                           redundancy while PRP, as described in the
>             previous
>
>                           section, creates network redundancy.  PRP
>             and HSR are both
>
>                           described in the IEC 62439 3 standard.",
>
>                           <http://webstore.iec.ch/webstore/webstore.nsf/
>
>             artnum/046615!opendocument>.
>
>             SB> Not available at the time of this review, but my
>             recollection is
>
>             SB> that this is not readily available without paying a
>             large fee.
>
>
>         If we decide that this is essential as a key reference, there
>         needs to be
>         some suitable text, as this will get raised a number of times
>         between
>         here an publication as first the directorates and then the ADs run
>         into this.
>
>
>
>             ===========
>
>                [ISA95]    ANSI/ISA, "Enterprise-Control System
>             Integration Part 1:
>
>                           Models and Terminology", 2000,
>
>                           <https://www.isa.org/isa95/>.
>
>             SB> Should that be 2000, or 2010.
>
>             SB> Note that this seems to be a very expensive document
>             to access.
>
>         You did not comment on the correctness of the reference.
>
>
>
>         Best Regards
>
>         Stewart
>