Re: [mpls] Review of draft-song-mpls-extension-header-09

[Haoyu Song <[e-mail]>]

Hi Stewart,

Thank you for the review. Please see my feedback below marked with [HS].

Best regards,
Haoyu

From: Stewart Bryant <[e-mail]>
Sent: Wednesday, August 24, 2022 10:24 AM
To: mpls <[e-mail]>
Cc: Stewart Bryant <[e-mail]>; [e-mail]
Subject: Review of draft-song-mpls-extension-header-09



I know that I was involved in the text in the very early days, but I am worried about it. Things have moved on and it is not clear that the approach which is basically to clone an IPv6 EH is the right approach for where we are now.

Certainly there are a lot of things that the OTD needs to consider in detail before committing to this design.

I do not think we should adopt this draft at the stage.

Detailed comments below.

Stewart





[HS] After ODT's work in one year and a half, the adopted use case, requirement, and framework drafts have set a clear demand for the PSD and this draft has also made a lot of progress to meet the requirements and comply with the architecture, based on the ODT discussions and the hard work of the coauthors. The draft takes the common approach and so far is supported by people from all the major vendors. Certainly there are still a lot work to do and some issues need to solve and some design details may subject to change. But we think it's on the right track and there's no fundamental issue. We think it's time to engage  with the WG to work on it together.





MPLS                                                        H. Song, Ed.

Internet-Draft                                    Futurewei Technologies

Intended status: Standards Track                                   Z. Li

Expires: 12 February 2023                                        T. Zhou

                                                                  Huawei

                                                            L. Andersson

                                                Bronze Dragon Consulting

                                                                Z. Zhang

                                                        Juniper Networks

                                                               R. Gandhi

                                                         J. Rajamanickam

                                                         J. Bhattacharya

                                                           Cisco Systems

                                                          11 August 2022



SB> The author list exceeds the allowed number, so needs to be trimmed. It would be useful if this was done at adoption.



[HS] We are aware of this and will conform with the IETF guidelines.



==========







1.  Motivation



   Some applications require adding sizable instructions and/or metadata

   to user packets within a network.  Such examples include In-situ OAM

   (IOAM) [RFC9197] and Service Function Chaining (SFC) [RFC7665].

SB> This is a big change that we are proposing to make to MPLS. Do we have any indication that there is planned deployment of those protocols and whether further use is stalled on this capability?



[HS] We have several use case drafts to detail the schemes for supporting these use cases. We think these use cases are valuable and will be deployed.



==========

   New

   applications are emerging.  It is possible that the instructions and/

   or metadata for multiple applications are stacked together in one

   packet to support a compound service.



   Conceivably, such instructions and/or metadata would be encoded as

   new headers and encapsulated in user packets.  Such headers may

   require to be processed in fast path due to performance

   considerations.  Moreover, such headers may require being attended at

   each hop on the forwarding path (i.e., hop-by-hop or HBH) or at

   designated end nodes (i.e., end-to-end or E2E).

SB> This is all a bit speculative. I think we need to state (or reference) a much tighter use case given the significant cost of putting this in Silicon.



[HS] These are also covered in the use case/requirement/architecture documents.



   The need and requirements to support such applications in MPLS

   networks, i.e., MPLS Network Actions (MNA), are described in

   [I-D.ietf-mpls-mna-usecases] and [I-D.ietf-mpls-mna-requirements].

   It is clear that some headers should be located after the MPLS label

   stack.  We call such a header "post-stack extension header".  The

   encapsulation of post-stack extension header(s) poses some challenges

   to MPLS networks, because the MPLS label stack contains no explicit

   indicator for the upper layer protocols by design.



   While the post-stack extension header needs some in-stack indicator

   to signal its presence, the mechanism specification is out of the

   scope of this document.  The indication for the presence of the post-

   stack extension header can be achieved using several mechanisms,

   including carrying an SPL or signaling it with the label FEC as

   described in [I-D.ietf-mpls-mna-fwk].  In this document, we focus on

   the encoding and encapsulation of the post-stack extension headers in

   an MPLS packet.

SB> The text to this point could usefully be trimmed and made less speculative.

SB> At the time of publication as an RFC (if this is published as an RFC) the situation will be that: this information needs to be carried and this is the/one of the ways that it is to be carried.



[HS] Yes, at the time of publication, the content here needs to be updated. By then, the indicator method should have be decided.



   The similar problem has been tackled for some applications before.

SB> Need to cite the examples with references in the following:

[HS] will do.

==========





   *  Some previous work such as G-ACH [RFC5586] was explicitly defined

      for control channel only but what we need is for the user packets.

SB> We can change that constraint if it is the right thing to do. It was introduced as a result to concern for the h/w likely to be available at the time, but it is not cast in stone.



[HS] Yes it's possible. Here we only describe the current status.



   To solve these problems, we introduce post-stack extension header as

   a general and extensible means to support new MNAs which involve

   instructions and/or meta data.  The concept is similar to IPv6

   extension headers which offer a huge potential for extending IPv6's

   capability (e.g, network security, SRv6 [RFC8754], network

   programming [RFC8986], SFC [I-D.ietf-spring-sr-service-programming],

   etc.).  Thanks to the existence of extension headers, it is

   straightforward to introduce new network services into IPv6 networks.

   For example, it has been proposed to carry IOAM header

   [I-D.ietf-ippm-ioam-ipv6-options] as a new extension header option in

   IPv6 networks.

SB> I think the point is the the eh mechanism is a mechanism, whether that makes it easy to introduce new eh onto a forwarder or into an operating network is a different matter, particularly in the HxH case.



[HS] Agree, the mechanism provides the possibility. Will rephrase.



   Nevertheless, when applying the extension header to MPLS, some issues

   of the IPv6 EH should be avoided:



   *  IPv6's extension headers are chained with the original upper layer

      protocol headers in a flat stack.  One must scan all the extension

      headers to access the upper layer protocol headers and the

      payload.  This is inconvenient and raises some performance

      concerns for some applications (e.g., DPI and ECMP).  The new

      scheme for MPLS header extension needs to improve this.



   *  [RFC8200] enforces many constraints to IPv6 extension headers

      (e.g., EH can only be added or deleted by the end nodes specified

      by the IP addresses in the IPv6 header, and there is only one Hop-

      by-Hop EH that can be processed on the path nodes), which are not

      suitable for MPLS networks.  For example, MPLS label stacks are

      added and changed in network, and there could be tunnel within

      tunnel, so the extension headers need more flexibility.

SB> Be careful here. The constraints in IPv6 are MTU and Security.

SB> MTU is less of an non-issue because of the domain specific nature of most MPLS deployments. The security issue may well apply to us as well.



[HS] Yes, the security issues should be discussed in the security section. We welcome your input on this topic.



2.  MPLS Extension Header



   The concept and design of the MPLS post-stack extension header comply

   with the requirements laid out in [I-D.ietf-mpls-mna-requirements].

   We highlight some specific design requirements for the post-stack

   extension headers in MPLS networks:



   Performance:  Unnecessary label stack scanning for a label and the

      full extension header stack scanning for the upper layer protocol

      should be avoided.  The extension headers a node needs to process

      should be located as close to the MPLS label stack as possible.

SB> How does the design achieve this?



[HS] The design itself has several features to make the EH parsing and accessing faster. The actual performance also depends on the use case itself but that's out of the scope. Here we only define the mechanism for the general container.



      Each extension header is better to serve only one application to

      avoid the need of packing multiple TLV options in one extension

      header.









Song, et al.            Expires 12 February 2023                [Page 4]



Internet-Draft      MPLS Post-Stack Extension Header         August 2022





   Scalability:  New applications can be supported by introducing new

      extension headers.  Multiple extension headers can be easily

      stacked together to support multiple services simultaneously.

SB> ... until you get to the max header length of the on path routers.



[HS]Yes, up to the limit of hardware capability.



   Backward Compatibility:  Legacy devices which do not recognize the

      extension header option should still be able to forward the

      packets as usual.

SB> That may or may not be a good thing.



[HS] We think this is important, otherwise we'll not have an incremental deployment scenario.



      If a device recognizes some of the extension

      headers but not the others in an extension header stack, it can

      process the known headers only while ignoring the others.

SB> ... again which may or may not be a good thing.



[HS] Such flexibility is good for incremental deployment. Of course, its potential implications should be considered. Your input is welcome.



   Flexibility:  A node can be configured to process or not process any

      EH.  Any tunnel end nodes in the MPLS domain can add new EH to the

      packets which shall be removed on the other end of the tunnel.

SB> That is not unique to this proposal



[HS] Here we mean the decouple of LSP and EHP so an action can be applied to a sub-path only.

=========





   Following the MPLS label stack is the 4-octet Header of Extension

   Headers (HEH), which indicates the total number of extension headers

   in this packet, the overall length of the extension headers, the type

   of the original upper layer header, and the type of the next header.

   The format of the HEH is shown in Figure 2.



       0           1         2          3

       0123 4567 89012345 67890123 45678901

      +----+----+--------+--------+--------+

      | R  |EHC |  EHTL  |  OUL   |   NH   |

      +----+----+--------+--------+--------+



                            Figure 2: HEH Format



   The meaning of the fields in an HEH is as follows:



   R:  4-bit reserved.  The nibble value means to avoid conflicting with

      IP version numbers and other well-defined semantics

      [I-D.kbbma-mpls-1stnibble].



   EHC:  4-bit unsigned integer for the Extension Header Counter.  This

      field keeps the total number of extension headers included in this

      packet.  It does not count the original upper layer protocol

      headers.

SB> What do you mean here? The payload headers?

[HS] Yes, it means the original MPLS payload before EHs are added.



      At most 15 EHs are allowed in one packet.



   EHTL:  8-bit unsigned integer for the Extension Header Total Length

      in 4-octet units.  This field keeps the total length of the

      extension headers in this packet, not including the HEH itself.



   OUL:  8-bit Original Upper Layer protocol number indicating the

      original upper layer protocol type.  It can be set to "UNKNOWN" if

      unknown.

SB> This is a different subject, and needs to be called out and explicitly agreed. There is no need for this as far as I can see, and I am not sure why it needs to be in every EH.



[HS]This support the case that there's no any EHs but the HEH. The feature can be useful to indicate the payload type.



   NH:  8-bit indicator for the Next Header.  This field identifies the

      type of the header immediately following the HEH.



   The value of the reserved nibble needs further consideration.  The

   EHC field can be used to keep track of the number of extension

   headers when some headers are inserted or removed at some network

   nodes.

SB> Has the WG agreed to inflight insertion and removal?

[HS]Not yet. This is subject to further discussion. We think it's nice feature to have.

SB> If we do this, which label does this apply to when we do nested tunnels?

[HS]This should be configured through control plane.

   The EHTL field can help to skip all the extension headers in

   one step if the original upper layer protocol headers or payload need

   to be accessed.  The OUL field can help identify the type of the

   original upper layer protocol.

SB> These are capabilities that we have not as yet agreed that we need. I do not recall them being called up the use case or requirement drafts.

[HS]As discussed in the other threads, this features is nice to have with little cost.



   The format of an Extension Header (EH) is shown in Figure 3.













Song, et al.            Expires 12 February 2023                [Page 6]



Internet-Draft      MPLS Post-Stack Extension Header         August 2022





       0          1          2          3

       01234567 89012345 67890123 45678901

      +--------+--------+--------+-------+

      |  NH    |  HLEN  |EXT(opt)|       |

      +--------+--------+--------+       |

      |                                  |

      ~        Header Specific Data      ~

      |                                  |

      +--------+--------+----------------+



                            Figure 3: EH Format



   The meaning of the fields in an EH is as follows:



   NH:  8-bit indicator for the Next Header.  This field identifies the

      type of the EH immediately following this EH.

SB> I am sure that you got the 8 bits by copying IPv6, but do we need 8 bits?

[HS]Ideally we will use the same coding space as the Internet protocols, as suggested in the draft.

SB> We are not operating in Internet scope. How many EH types do we think that we will need? Could we operate in domain scope? Could we use some of these bits to specify the action on an unknown EH, and to specify E2E and HxH?

[HS]Sharing the same coding space can support the same number for the same protocol. Experimental EHs can also be supported. We also have the mechanism to expand the space. The E2E or HbH property should be an inherent feature of an EH type so there's no need to specify it again.



   HLEN:  8-bit unsigned integer for the Extension Header Length in

      4-octet units, not including the first 4 octets.

SB> Are we seriously considering 1KB of EH? That will break pretty much every forwarder.

[HS]We think enough scalability should be preserved because future hardware might be unimaginably powerful.



   EXT:  8-bit optional type extension.  To save the Next Header numbers

      and extend the number space, it is possible to use one "Next

      Header" code to cover a set of sub-types.  Each sub-type is

      assigned a new code in a different name space.  This field is

      optional and it is only specified for some specific EH type.

SB> So are we saying that we need to make provision for 64K EH types?

[HS] No. Only some EH types would need expansion. For example, IOAM has several different options. It's too expensive to assign an EH type for each of it. In this case, it's better to have a single EH type for IOAM, and use the EXT to specify the option types.

SB> EH sub-types could be a concept that is private to a particular EH type, and that might be a more powerful design.

[HS] Yes, in the case of private EH types, EXT can be used to define sub-types.



   Header Specific Data:  Variable length field for the specification of

      the EH.  This field is in length such that the EH is 4-octet

      aligned.



SB> What you do not have here and which may be of use is a flag saying what to do if you do not understand. I also wonder if there needs to be a HxH flag.

[HS]The currently thinking is that any EH that a node doesn't understand can be skipped. As for the HxH feature, it should be inherent to the EH. That is, once you know the EH, you know if it should be processed hop by hop or not.



SB> The way this is laid out we need to parse the whole of the EH chain to find out if we are interested in any EH entry. I wonder if we should have a manifest of some sort so just looking at BoS we can find out if anything is of interest?

[HS]Header chain is easy to parse. We avoid any catalog because it complicates the parser and is inflexible. On the other hand, the control plane can configure the node to tell it what to looks for and what it is capable of, so the parsing process can terminate earlier in some cases.





   The extension headers as well as the first original upper layer

   protocol header are chained together through the NH field in HEH and

   EHs.  The encoding of NH can share the same value registry for IPv4/

   IPv6 protocol numbers.  Values for new EH types (i.e., NH number)

   shall be assigned by IANA from the same registry as for the ipv4 and

   ipv6 protocol numbers (https://www.iana.org/assignments/protocol-<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Fprotocol-&data=05%7C01%7Chaoyu.song%40futurewei.com%7C5353ae291c254699cdbd08da85f57d49%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637969586687824586%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Cncq%2B5uTyBw7gnJrqgzkAAu1LwQdCsHTP4%2FP3FDhGXc%3D&reserved=0>

   numbers/protocol-numbers.xhtml).

SB> ... but is that the right registry to use? Do we really want to be tied to the IP protocol numbers registry and do they want us cluttering up their registry - which has to last the life of the Internet - with our extension system which could use its own registry?



[HS] Many codepoints can be shared (e.g., MPLS and the original upper layer types). The shared codepoints are beneficial to both communities and make the implementation easier. Of course, we can resort to a dedicated registry if the WG decides to do so.



SB> Indeed the result of using our own registry would be a more compact registry space which leads to more efficient forwarding.



[HS] For scalability consideration, 8 bits are not too many. And a few bits less don't help for the efficiency because the header itself is much larger and the entire EH should be word-aligned.



   Specifically, the NH field of the last EH in a chain can have some

   special values, which shall be assigned by IANA as well:



   NONE (No Next Header):  Indicates that there is no other header and

      payload after this header.  This can be used to transport packets

      with only extension header(s), for example, the control packets

      for control or the probe packets for measurements.  Note that

      value 59 was reserved for "IPv6 No Next Header" indicator.  It may

      be possible for MPLS EH to share this value.



SB> That needs discussion in 6man



[HS] If we decide to use value 59, we need to discuss in 6man.





Song, et al.            Expires 12 February 2023                [Page 7]



Internet-Draft      MPLS Post-Stack Extension Header         August 2022





   UNKNOWN (Unknown Next Header):  Indicates that the type of the header

      after this header is unknown.  This is intended to be compatible

      with the original MPLS design in which the upper layer protocol

      type is unknown from the MPLS header alone.



SB> Not quite true, it is known from the FEC of the BoS label.

[HS]Yes payload type can be known through FEC. This statement is just for the general case.



   MPLS:  Indicates that the original upper layer protocol is still MPLS

      and another MPLS label stack follows.

   Note that the original upper layer protocol can be of type "MPLS",

   which implies that in a packet there might be multiple label stacks

   separated by EHs.  Having more than one independent label stack is

   not new.  For example, A Bier header could separate the transport/

   bier labels and the payload labels; An MPLS Pseudo Wire (PW) network

   could be implemented on the top of another infrastructure MPLS

   network.

SB> I do not understand this point.

[HS] This means two logically independent MPLS label stacks each with its own EHs.

After the outer label stack and EHs are processed and removed, the inner one is exposed. In some cases, this can make the EHs more reachable.



   In such cases, we have the flexibility to apply different

   services to different label stacks.



3.  Type of MPLS Extension Headers



   Basically, there are two types of MPLS EHs: HBH and E2E.  E2E means

   that the EH is only supposed to be inserted/removed and processed at

   the MPLS tunnel end points where the MPLS header is inserted or

   removed.  The EHs that need to be processed on path nodes within the

   MPLS tunnel are of the HBH type.  However, any node in the tunnel can

   be configured to ignore an HBH EH, even if it is capable of

   processing it.



   If there are two types of EHs in a packet, the HBH EHs must take

   precedence over the E2E EHs.

SB> I am worried about how we bind the scope of the EH to a label.

SB> If I tunnel a packet by pushing a label are the EH in the scope of that tunnel or not? Is that a ubiquitous decision?

[HS]This could be done through control plane configuration? If the pipe mode is used for a tunnel, we can also apply the multiple MPLS label stacks as discussed before to clarify the EH scope.





   Making a distinction of the EH types and ordering the EHs in a packet

   help improve the forwardidng performance.  For example, if a node

   within an MPLS tunnel finds only E2E EHs in a packet, it can avoid

   scanning the EH list.



   The type of an EH (i.e., HBH or E2E) is an intrinsic property of the

   EH.  In other words, EH type indicates if it needs to be processed on

   each hop or only on edge node.

SB> Is that the optimum design?

[HS] Currently we think so.



4.  Operation on MPLS Extension Headers



   When the first EH X needs to be added to an MPLS packet, an EH

   indicator is inserted into the proper location in the MPLS label

   stack.  A HEH is then inserted after the MPLS label stack, in which

   EHCNT is set to 1, EHTLEN is set to the length of X in 4-octet units,

   and NH is set to the header value of X.  At last, X is inserted after

   the HEH, in which NH and HELN are set accordingly.  Note that if this

   operation happens at a PE device, the upper layer protocol is known







Song, et al.            Expires 12 February 2023                [Page 8]



Internet-Draft      MPLS Post-Stack Extension Header         August 2022





   before the MPLS encapsulation, so its value can be saved in the NH

   field if desired.  Otherwise, the NH field is filled with the value

   of "UNKNOWN".

SB> I am wondering if there are cases, for example as a result of nested tunnels where some may be know and some unknown.

[HS] only the last NH field indicating the original upper layer protocol can be set to "UNKNOWN", all other NH fields must be set to a regular type.



=========



5.  Use Cases



   In this section, we show how MPLS extension header can be used to

   support several new network applications.



   In-situ OAM:  In-situ OAM (IOAM) records flow OAM information within

      user packets while the packets traverse a network.  The

      instruction and collected data are kept in an IOAM header

      [RFC9197].  When applying IOAM in an MPLS network, the IOAM header

      can be encapsulated as an MPLS extension header.



   Network Telemetry and Measurement:  A network telemetry and

      instruction header can be carried as an extension header to

      instruct a node what type of network measurements should be done.

      For example, the method described in [RFC8321] can be implemented

      in MPLS networks since the EH provides a natural way to color MPLS

      packets.



   Network Security:  Security related functions often require user

      packets to carry some metadata.  In a DoS limiting network

      architecture, a "packet passport" header is used to embed packet

      authentication information for each node to verify.

SB> You need to comment on the matter of NS and EH addition removal in the security case. We spent a little time thinking of this in the TCR case (draft-bcx-rtgwg-tcr), but there are probably others that have considered the problem of partial signatures in more detail.



[HS]Here we just mention the possible use cases for network security. The actual application is better to be covered by some other drafts. Of course, if such use cases raises some special requirements, we'd like to know if our scheme has already supported them or not. Your input on this is welcome.





Song, et al.            Expires 12 February 2023                [Page 9]



Internet-Draft      MPLS Post-Stack Extension Header         August 2022





   Segment Routing and Network Programming:  MPLS extension header can

      support the implementation of a new flavor of the MPLS-based

      segment routing, with better performance and richer

      functionalities.  The details will be described in another draft.

SB> Reference?

[HS] We haven't published the draft yet. It's briefly mentioned in the use case draft.



6.  Security Considerations



SB> We need a security architecture or this will not get past first base.

[HS] The major security concerns may come from the actions that encapsulated in the generic EH containers. So we need to be careful to admit actions and take measures to avoid the security threats. We welcome any suggestions on other related considerations.

==========

==========