[Detnet] Some initial comments on draft-dt-detnet-dp-sol-00

[Stewart Bryant <[e-mail]>]

I took an initial look at the detnet dp document.

I think that PWs are certainly a good way to go, but I am concerned
about a number of aspects of the proposal.

My biggest concern is doing the sequence number checking. There have
not been many attempts to do this, and from work that we have done in
the OAM space all the feedback I get is that this is a hard problem.
It might be useful if you constrained the packets to a path and an
interface using MPLS-TP or MPLS-TE, however I don't see a practical
way to do that in a pure IP network. You could of course do it
with SR, but that is v6 only, and the packets could potentially need
a very large SR header.

Please see the notes inline.

-Stewart

  
    The PW-based data plane can be run over either an IP
    or MPLS [RFC4448][RFC6658] Packet Switched Network (PSN).

SB> I comment on this in detail later

============


5.1.  DetNet Control Word

    The DetNet control word (d-CW) is identical to the control word
    defined for Ethernet over MPLS networks in [RFC4448].  The DetNet
    control word is illustrated in Figure 4.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |0 0 0 0|  reserved - set to 0  |   16 bit Sequence Number      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


                        Figure 4: DetNet Control Word

    [Editor's note: Shoudl we care about high speed links, here 16 bits
    of sequence number wraps fast?  For example, in a case of 100Gb/s
    link, 16 bits of sequence number will wrap in ~6.6ms assuming 1250
    octets of packets and ~3.3ms for 625 octets packets.  Both numbers
    mean quite long fiber distances, though.]

SB> This worries me. Doing read modify write on the sequence number is
SB> difficult in the general case, particularly without path constraint
SB> since a packet can arrive on any interface, and this interface can
SB> change.
SB>
SB> Are you thinking that there will be so few DN PWs that you can
SB> put the counters in registers? That might fly at the T-PEs, but
SB> I would be worried at the S-PEs.
SB>
SB> BTW shouldn't you consider much smaller packets, or do you imagine
SB> that DN will be constrained to applications using large packets.

===========

5.3.  DetNet encapsulation

    The DetNet data plane follows PW encapsulation.  This document
    specifies a single encapsulation that can be used over both MPLS and
    IP packet switched Networks (PSN).  The DetNet data plane
    encapsulation consists of a

    o  DetNet control word (d-CW): contains sequencing information for
       packet replication and duplicate elimination purposes.  There is a
       separate sequence number space per each DetNet label.

SB> Do you mean per DetNet flow ID, or per PW label?

  
  o  DetNet flow-ID (f-ID): uniquely identifies a DetNet flow within a
       DetNet network.  Multiple DetNet PWs with different PW labels may
       have the same f-ID, which then implies the PWs are actually
       subflows of one compound flow.

SB> I am not sure I understand the definition of a DetNet network yet.
SB> I presume that it is Fig 5 from the architecture draft, which
SB> is a tunnel between two service instances.
SB> However I am having difficulty understanding the scope of the
SB> uniqueness. It sounds as if it needs to be unique between a
SB> pair of service instances is that the case, or does the
SB> uniqueness have greater scope?
SB> PWs are not subflows, they are "A mechanism that carries the
SB> essential elements of an emulated service from one PE to one or
SB> more other PEs over a PSN."


  o  PseudoWire Label (PW Label;): a standard PW label that identifies
       a PW Instance within a (DA-)T-PE or (DA-)S-PE device.

SB> Just so we are all clear the PW label changes at S-PEs

===========

  o  DetNet topology overlay label (L-label): an optional label used
       between (DA-)T-PE or (DA-)S-PE nodes.  The main use of L-labels is
       to tunnel PWs through a PE node and therefore effectively making a
       PE node to behave like a P node.

SB> This needs more thought. The reason that S-PEs were created
SB> was to minimise the burden of running PWs between different
SB> administrative domains. To make this feasible it was necessary
SB> for the T-PEs to allocate their own PW label and have the
SB> S-PEs swap, that way only the boundary nodes (S-PEs) needed
SB> be worried about the mapping between the PE identity and the
SB> PW label in the data-plane.
SB> If a data-plane identifier is used, then we don't really need
SB> S-PEs as such. So I think that we have to define the new
SB> purpose of the S-PE more clearly when they are used for Detnet.


  In a case of MPLS-based PSN, the tunnel labels between LSRs are
    referred as T-labels.

SB> I think that they are really LSP labels.

  The DetNet CW and the Detnet flow-ID together constitute the DetNet
    PseudoWire encapsulation header.

       [Editor's note: The current design has the DetNet flow-ID as part
       of the every DetNet flow packet.  The flow-ID identifies the flow
       uniquely within the DetNet network and together with the sequence
       number information from the DetNet control word is used for PREF
       purposes.  The flow-ID makes is easy for the DA-*-PE node to
       associate different PWs into one compound flow and perform the
       elimination of duplicate packets.

SB> I am not sure this is needed. The DA-*-PE knows the relationship
SB> between the PW labels and can make the decision based on that.



       The flow-ID would point at the
       node internal construct that holds the received packet history for



Korhonen, et al.       Expires September 14, 2017               [Page 8]

Internet-Draft         DetNet data plane solution             March 2017


       each DetNet flow of interest.  However, it could also be possible
       to associate multiple PWs into one DetNet flow just using the
       control plane provided information.  In this case different PWs
       (using any PW label) would be mapped internally within a node to a
       local-ID (or similar construct), which again points at the
       internal per DetNet flow received packets history construct.

SB> I think you have to do this anyway

       The
       explicit in-band flow-ID is easy from the processing and control
       plane point of view.
SB> Isn't this a bigger change to the forwarder? What normally
SB> happens is you vector to the instructions and context using
SB> the PW label as the identifier. So I think the flow-ID just adds
SB> complexity. Given that it needs to be unique and known at
SB> each PW node on the path, I do not see what has been gained in
SB> terms of reduction in control plane activity.


       The local-ID approach does not need the in-
       band information (thus has less overhead) but requires more from
       the control plane and the mapping information has to be stored
       into the LFIB.  Current design decision is the in-band flow-ID but
       may be changed to local-ID if there is a strong reason to do the
       change.]

    Figure 6 illustrates a DetNet PseudoWire encapsulation using an MPLS
    PSN.  Similarly, Figure 7 illustrates the DetNet PseudoWire
    encapsulation when IP PSN is used.  The encapsulation is uniform
    above the PSN.

    Depending on the network topology the "overlay label" (L-label) may
    be part of the label stack.  The L-label tunnels guarantee PW labels
    remain unchanged between DA-*-PE nodes.
SB> Please see earlier

    Furthermore, L-labels
    tunnels allow selectively exposing the PW label to DA-*-PE nodes,
    which means some overlay topologies may just pass through specific
    DA-S-PEs without any DetNet specific processing.

SB> They can do this anyway. At an SPE we normally just swap the
SB> PW label, and other than for OAM reasons (trapped by TTL expiry)
SB> we do no processing. So if you do not teach an SPE that the PW
SB> is to be processed, it will just pass through.
SB>
SB> I think that you need to look seriously at deleting this component
SB> of your design and building something much closer to a normal
SB> PW design.

===============

    When IP PSN is used, the label stack it transports is only inspected
    when the IP packet destination address equals to the IP address of a
    DA-*-PE or a P node.  Essentially there are one more IP tunnels
    between a number of DA-*-PE and/or P nodes.  The LFIB and the
    forwarding information base (FIB) combination determines whether a PW
    gets terminated at the node or forwarded to another node within a new
    IP tunnel.

SB> So, setting aside for the moment the work that we are doing on
SB> unifying SRv6 and MPLS SR, work which as yet has no official status
SB> you should understand that there really is no deployment of
SB> MPLS-PW over IP. All of the deployed PWs are either pure PW over
SB> MPLS, or LT2Pv3. L2TPv3 has no concept of an S-PE.
SB>
SB> Now I think the unified approach is the right one, but so far
SB> there are no real specifications.
SB>
SB> Also you need to think about whether you want ECMP or not, because
SB> if you do you really do need the interstitial UDP layer (RFC7510)
SB> shown below.

==============


6.1.  Forwarded clarifications

    [Editor's note: The Detnet-aware "extended forwarder" does the heavy
    lifting on maintaining the sequence numbers associated with the
    DetNet labels.  Extended forwarder is also responsible for packet
    replication and duplicate elimination.  See the excerpt from RFC3985
    Section 4.2.1. about forwarder's functions.  We extend that to PREF:

       Some applications have to forward payload elements selectively
       from one or more ACs to one or more PWs.  In such cases, there
       will also be a need to perform the inverse function on PWE3-PDUs
       received by a PE from the PSN.  This is the function of the
       forwarder.

    ]
SB> But note that forwarder function only appears in the T-PE, we
SB> never included it in the S-PE which can be better thought of
SB> as a two layer MPLS switch - it's not how it works although
SB> a modern two label lookup MPLS system could do it like that
SB> but all that happens in an SPE is that you swap two labels
SB> simultaneously - the LSP label and the PW label.


     The DetNet specific new functionality in a DA-*-PE PW processing is
    the packet replication and duplication elimination function (PREF).
    This functional is a part of the "extended" forwarder.  The PREF
    processing is triggered by the LFIB actions i.e., not all PWs receive



Korhonen, et al.       Expires September 14, 2017              [Page 11]

Internet-Draft         DetNet data plane solution             March 2017


    DetNet specific processing.  Basically the LFIB has to be extended
    with a "PREF enabled" boolean configuration switch that is associated
    with the normal label actions (e.g., swap, push, pop, ..).  The
    output of the PREF elimination function is always a single packet.
    The output of the PREF replication function is always one or more
    packet (i.e., 1:M replication).  The replicated packets MUST share
    the same DetNet PW control word sequence number and flow identity
    word flow-id.

    The complex part of the DetNet PREF processing is tracking the
    history of received packets for multiple PWs.  These PWs do not have
    the same PW label value while they still share the same PW sequence
    number counter and the history information.  That is where the DetNet
    encapsulation header flow-ID plays an important role and binds the
    control word sequence number to the flow specific shared counter and
    history information within the PREF function.

SB> That is certainly one way of doing it, although given that you need
SB> to provision the PW anyway, I an not sure it is needed.

    The DetNet flow word contains a D flag bit (see Section 5.2), which
    makes the DA-*-PE node aware of the direction the flow-ID arrived
    from.  If the node, based on the local policy, checks for the D bit
    setting that effectively means the sequence number history has to
    contain also the D bit information.

SB> I am really not sure why you need the D bit a PW received on a given PW
SB> label only goes one way.


  ==============





7.  Other DetNet considerations

7.1.  Class of Service

    [Editor's note: Discuss the CoS.. and how that is archived when using
    MPLS or IP PSN.]


SB> Don't all your packets need to go on the highest class of service?

===========

7.3.  Time synchronization


    o  PTP with on-path support: in this approach PTP packets are sent as
       DetNet flows, and intermediate nodes take part in the protocol as
       Transparent Clocks or Boundary Clocks [IEEE1588].  The on-path PTP
       support by intermediate nodes provides a higher degree of accuracy
       than the previous approach.  The actual accuracy depends on
       whether all intermediate nodes are PTP-capable, or only a subset
       of them.

SB> RFC8169 shows how to do TC in an MPLS network.
SB> I am not sure anyone knows how to do this in an IP network.

===========

7.4.  Bidirectional traffic

    Some DetNet applications generate bidirectional traffic and may
    require symmetric flows.  There are already mechanisms that can be
    used to create bidirectional tunnels at the transport network level,
    such as MPLS-TP.  The data plane solution SHOULD allow establishing
    bidirectional symmetric flows.  Control plane mechanisms would need
    to also support this, though this is out of scope of this document.
    [Summary of existing mechanisms to create bidirectional tunnels that
    can be used.]

SB> PWs are always bidirectional of course.


  
  8.1.  PW Label assignment and distribution

    The PW label distribution follows the same mechanisms specified for
    MS-PW [RFC6073].

SB> This will need extensions to support DETNET