Re: [Detnet] Some initial comments on draft-dt-detnet-dp-sol-00

[Stewart Bryant <[e-mail]>]

On 06/06/2017 17:50, Balázs Varga A wrote:
>
> Hi Stewart,
>
> Thanks for the detailed review.
>
> Based on several good comments in the Chicago meeting we have started 
> to update the draft.
>
> The major changes are:
>
> 1, changing encapsulation
>
> - PW-based encapsulation: The PW-based data plane can be run over an 
> MPLS PSN.
>
> - Native-IP encapsulation: This solution is based on IP header fields, 
> namely on IPv6 Flow label
>
> and a new DetNet Control Word extension header option. It is targeted 
> for native IPv6 networks.
>

It seems a shame that it needs to be different from the MPLS design. I 
was pointing out that
PW (MPLS format) over IP had not been deployed. That does not make it not a
nice design.

> 2, PW-label specific requirement is simplified (regarding uniqueness):
>
> - DetNet flows that need to undergo PREF processing MUST have the same 
> PW Label when
>
> they arrive at the DA-*-PE node.
>

That aligns with how PWs work.

> We have also identified that sequence numbering related functions may 
> be challenging, however
>
> it is a must for the PREF (Packet Replication and Elimination 
> Function) implementation. PREF provides
>
> per packet level redundancy and not the so far defined per-PW-segment 
> redundancy.
>

I think we need to draw up some use cases to understand what constraints 
we may need
to add in to make this at all feasible.

> Similarly, your concerns on constraining DetNet packets to a path and 
> an interface in a pure IP network,
>
> is absolutely a candidate topic for further improvements.
>

The obvious solution is IP SR, but it needs to be a much lighterweight 
solution than the
solution we have on the table for V6 with one IPv6 address per specified 
hop along the
path.

Another way is to modify the routing protocol to constrain the path. We 
did that
in not-via, and could make constrain repair paths to avoid a failure, 
and could get
this to work in both link state and dv routing protocols. However we 
need the opposite.

Then I suppose RSVP was originally designed to do what you need. So I 
suppose
we could your RSVP to add explicit IP routing paths to the network.

Interesting problem.

> I will be back soon with further feedback regarding your detailed notes.
>

Well that was only a first pass, but it was somewhere to start.

BR

Stewart

> Thanks & Cheers
>
> Bala’zs
>
> *From:*Stewart Bryant [mailto:[e-mail]]
> *Sent:* 2017. június 5. 22:02
> *To:* [e-mail]; [e-mail]
> *Subject:* Some initial comments on draft-dt-detnet-dp-sol-00
>
> I took an initial look at the detnet dp document.
> I think that PWs are certainly a good way to go, but I am concerned
> about a number of aspects of the proposal.
> My biggest concern is doing the sequence number checking. There have
> not been many attempts to do this, and from work that we have done in
> the OAM space all the feedback I get is that this is a hard problem.
> It might be useful if you constrained the packets to a path and an
> interface using MPLS-TP or MPLS-TE, however I don't see a practical
> way to do that in a pure IP network. You could of course do it
> with SR, but that is v6 only, and the packets could potentially need
> a very large SR header.
> Please see the notes inline.
> -Stewart
>   
>     The PW-based data plane can be run over either an IP
>     or MPLS [RFC4448][RFC6658] Packet Switched Network (PSN).
> SB> I comment on this in detail later
> ============
> 5.1.  DetNet Control Word
>     The DetNet control word (d-CW) is identical to the control word
>     defined for Ethernet over MPLS networks in [RFC4448].  The DetNet
>     control word is illustrated in Figure 4.
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |0 0 0 0|  reserved - set to 0  |   16 bit Sequence Number      |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>                         Figure 4: DetNet Control Word
>     [Editor's note: Shoudl we care about high speed links, here 16 bits
>     of sequence number wraps fast?  For example, in a case of 100Gb/s
>     link, 16 bits of sequence number will wrap in ~6.6ms assuming 1250
>     octets of packets and ~3.3ms for 625 octets packets.  Both numbers
>     mean quite long fiber distances, though.]
> SB> This worries me. Doing read modify write on the sequence number is
> SB> difficult in the general case, particularly without path constraint
> SB> since a packet can arrive on any interface, and this interface can
> SB> change.
> SB>
> SB> Are you thinking that there will be so few DN PWs that you can
> SB> put the counters in registers? That might fly at the T-PEs, but
> SB> I would be worried at the S-PEs.
> SB>
> SB> BTW shouldn't you consider much smaller packets, or do you imagine
> SB> that DN will be constrained to applications using large packets.
> ===========
> 5.3.  DetNet encapsulation
>     The DetNet data plane follows PW encapsulation.  This document
>     specifies a single encapsulation that can be used over both MPLS and
>     IP packet switched Networks (PSN).  The DetNet data plane
>     encapsulation consists of a
>     o  DetNet control word (d-CW): contains sequencing information for
>        packet replication and duplicate elimination purposes.  There is a
>        separate sequence number space per each DetNet label.
> SB> Do you mean per DetNet flow ID, or per PW label?
>   
>   o  DetNet flow-ID (f-ID): uniquely identifies a DetNet flow within a
>        DetNet network.  Multiple DetNet PWs with different PW labels may
>        have the same f-ID, which then implies the PWs are actually
>        subflows of one compound flow.
> SB> I am not sure I understand the definition of a DetNet network yet.
> SB> I presume that it is Fig 5 from the architecture draft, which
> SB> is a tunnel between two service instances.
> SB> However I am having difficulty understanding the scope of the
> SB> uniqueness. It sounds as if it needs to be unique between a
> SB> pair of service instances is that the case, or does the
> SB> uniqueness have greater scope?
> SB> PWs are not subflows, they are "A mechanism that carries the
> SB> essential elements of an emulated service from one PE to one or
> SB> more other PEs over a PSN."
>   o  PseudoWire Label (PW Label;): a standard PW label that identifies
>        a PW Instance within a (DA-)T-PE or (DA-)S-PE device.
> SB> Just so we are all clear the PW label changes at S-PEs
> ===========
>   o  DetNet topology overlay label (L-label): an optional label used
>        between (DA-)T-PE or (DA-)S-PE nodes.  The main use of L-labels is
>        to tunnel PWs through a PE node and therefore effectively making a
>        PE node to behave like a P node.
> SB> This needs more thought. The reason that S-PEs were created
> SB> was to minimise the burden of running PWs between different
> SB> administrative domains. To make this feasible it was necessary
> SB> for the T-PEs to allocate their own PW label and have the
> SB> S-PEs swap, that way only the boundary nodes (S-PEs) needed
> SB> be worried about the mapping between the PE identity and the
> SB> PW label in the data-plane.
> SB> If a data-plane identifier is used, then we don't really need
> SB> S-PEs as such. So I think that we have to define the new
> SB> purpose of the S-PE more clearly when they are used for Detnet.
>   In a case of MPLS-based PSN, the tunnel labels between LSRs are
>     referred as T-labels.
> SB> I think that they are really LSP labels.
>   The DetNet CW and the Detnet flow-ID together constitute the DetNet
>     PseudoWire encapsulation header.
>        [Editor's note: The current design has the DetNet flow-ID as part
>        of the every DetNet flow packet.  The flow-ID identifies the flow
>        uniquely within the DetNet network and together with the sequence
>        number information from the DetNet control word is used for PREF
>        purposes.  The flow-ID makes is easy for the DA-*-PE node to
>        associate different PWs into one compound flow and perform the
>        elimination of duplicate packets.
> SB> I am not sure this is needed. The DA-*-PE knows the relationship
> SB> between the PW labels and can make the decision based on that.
>        The flow-ID would point at the
>        node internal construct that holds the received packet history for
> Korhonen, et al.       Expires September 14, 2017               [Page 8]
> Internet-Draft         DetNet data plane solution             March 2017
>        each DetNet flow of interest.  However, it could also be possible
>        to associate multiple PWs into one DetNet flow just using the
>        control plane provided information.  In this case different PWs
>        (using any PW label) would be mapped internally within a node to a
>        local-ID (or similar construct), which again points at the
>        internal per DetNet flow received packets history construct.
> SB> I think you have to do this anyway
>        The
>        explicit in-band flow-ID is easy from the processing and control
>        plane point of view.
> SB> Isn't this a bigger change to the forwarder? What normally
> SB> happens is you vector to the instructions and context using
> SB> the PW label as the identifier. So I think the flow-ID just adds
> SB> complexity. Given that it needs to be unique and known at
> SB> each PW node on the path, I do not see what has been gained in
> SB> terms of reduction in control plane activity.
>        The local-ID approach does not need the in-
>        band information (thus has less overhead) but requires more from
>        the control plane and the mapping information has to be stored
>        into the LFIB.  Current design decision is the in-band flow-ID but
>        may be changed to local-ID if there is a strong reason to do the
>        change.]
>     Figure 6 illustrates a DetNet PseudoWire encapsulation using an MPLS
>     PSN.  Similarly, Figure 7 illustrates the DetNet PseudoWire
>     encapsulation when IP PSN is used.  The encapsulation is uniform
>     above the PSN.
>     Depending on the network topology the "overlay label" (L-label) may
>     be part of the label stack.  The L-label tunnels guarantee PW labels
>     remain unchanged between DA-*-PE nodes.
> SB> Please see earlier
>     Furthermore, L-labels
>     tunnels allow selectively exposing the PW label to DA-*-PE nodes,
>     which means some overlay topologies may just pass through specific
>     DA-S-PEs without any DetNet specific processing.
> SB> They can do this anyway. At an SPE we normally just swap the
> SB> PW label, and other than for OAM reasons (trapped by TTL expiry)
> SB> we do no processing. So if you do not teach an SPE that the PW
> SB> is to be processed, it will just pass through.
> SB>
> SB> I think that you need to look seriously at deleting this component
> SB> of your design and building something much closer to a normal
> SB> PW design.
> ===============
>     When IP PSN is used, the label stack it transports is only inspected
>     when the IP packet destination address equals to the IP address of a
>     DA-*-PE or a P node.  Essentially there are one more IP tunnels
>     between a number of DA-*-PE and/or P nodes.  The LFIB and the
>     forwarding information base (FIB) combination determines whether a PW
>     gets terminated at the node or forwarded to another node within a new
>     IP tunnel.
> SB> So, setting aside for the moment the work that we are doing on
> SB> unifying SRv6 and MPLS SR, work which as yet has no official status
> SB> you should understand that there really is no deployment of
> SB> MPLS-PW over IP. All of the deployed PWs are either pure PW over
> SB> MPLS, or LT2Pv3. L2TPv3 has no concept of an S-PE.
> SB>
> SB> Now I think the unified approach is the right one, but so far
> SB> there are no real specifications.
> SB>
> SB> Also you need to think about whether you want ECMP or not, because
> SB> if you do you really do need the interstitial UDP layer (RFC7510)
> SB> shown below.
> ==============
> 6.1.  Forwarded clarifications
>     [Editor's note: The Detnet-aware "extended forwarder" does the heavy
>     lifting on maintaining the sequence numbers associated with the
>     DetNet labels.  Extended forwarder is also responsible for packet
>     replication and duplicate elimination.  See the excerpt from RFC3985
>     Section 4.2.1. about forwarder's functions.  We extend that to PREF:
>        Some applications have to forward payload elements selectively
>        from one or more ACs to one or more PWs.  In such cases, there
>        will also be a need to perform the inverse function on PWE3-PDUs
>        received by a PE from the PSN.  This is the function of the
>        forwarder.
>     ]
> SB> But note that forwarder function only appears in the T-PE, we
> SB> never included it in the S-PE which can be better thought of
> SB> as a two layer MPLS switch - it's not how it works although
> SB> a modern two label lookup MPLS system could do it like that
> SB> but all that happens in an SPE is that you swap two labels
> SB> simultaneously - the LSP label and the PW label.
>      The DetNet specific new functionality in a DA-*-PE PW processing is
>     the packet replication and duplication elimination function (PREF).
>     This functional is a part of the "extended" forwarder.  The PREF
>     processing is triggered by the LFIB actions i.e., not all PWs receive
> Korhonen, et al.       Expires September 14, 2017              [Page 11]
> Internet-Draft         DetNet data plane solution             March 2017
>     DetNet specific processing.  Basically the LFIB has to be extended
>     with a "PREF enabled" boolean configuration switch that is associated
>     with the normal label actions (e.g., swap, push, pop, ..).  The
>     output of the PREF elimination function is always a single packet.
>     The output of the PREF replication function is always one or more
>     packet (i.e., 1:M replication).  The replicated packets MUST share
>     the same DetNet PW control word sequence number and flow identity
>     word flow-id.
>     The complex part of the DetNet PREF processing is tracking the
>     history of received packets for multiple PWs.  These PWs do not have
>     the same PW label value while they still share the same PW sequence
>     number counter and the history information.  That is where the DetNet
>     encapsulation header flow-ID plays an important role and binds the
>     control word sequence number to the flow specific shared counter and
>     history information within the PREF function.
> SB> That is certainly one way of doing it, although given that you need
> SB> to provision the PW anyway, I an not sure it is needed.
>     The DetNet flow word contains a D flag bit (see Section 5.2), which
>     makes the DA-*-PE node aware of the direction the flow-ID arrived
>     from.  If the node, based on the local policy, checks for the D bit
>     setting that effectively means the sequence number history has to
>     contain also the D bit information.
> SB> I am really not sure why you need the D bit a PW received on a given PW
> SB> label only goes one way.
>   ==============
> 7.  Other DetNet considerations
> 7.1.  Class of Service
>     [Editor's note: Discuss the CoS.. and how that is archived when using
>     MPLS or IP PSN.]
> SB> Don't all your packets need to go on the highest class of service?
> ===========
> 7.3.  Time synchronization
>     o  PTP with on-path support: in this approach PTP packets are sent as
>        DetNet flows, and intermediate nodes take part in the protocol as
>        Transparent Clocks or Boundary Clocks [IEEE1588].  The on-path PTP
>        support by intermediate nodes provides a higher degree of accuracy
>        than the previous approach.  The actual accuracy depends on
>        whether all intermediate nodes are PTP-capable, or only a subset
>        of them.
> SB> RFC8169 shows how to do TC in an MPLS network.
> SB> I am not sure anyone knows how to do this in an IP network.
> ===========
> 7.4.  Bidirectional traffic
>     Some DetNet applications generate bidirectional traffic and may
>     require symmetric flows.  There are already mechanisms that can be
>     used to create bidirectional tunnels at the transport network level,
>     such as MPLS-TP.  The data plane solution SHOULD allow establishing
>     bidirectional symmetric flows.  Control plane mechanisms would need
>     to also support this, though this is out of scope of this document.
>     [Summary of existing mechanisms to create bidirectional tunnels that
>     can be used.]
> SB> PWs are always bidirectional of course.
>   
>   8.1.  PW Label assignment and distribution
>     The PW label distribution follows the same mechanisms specified for
>     MS-PW [RFC6073].
> SB> This will need extensions to support DETNET