[Teas] Really late-late WGLC comments [Re: Late WGLC review of draft-ietf-teas-5g-ns-ip-mpls]

[Greg Mirsky <[e-mail]>]

Dear All,

It appears that the WG LC is open, and I have one comment for your
consideration:

   - After a discussion with my colleagues, I found that Figure 6 confuses,
   particularly the part that reflects the relationship between 5G NSO, TN
   Orchestration, and systems that it contains, i.e., Customer Site
   Orchestration and RFC 9543 NSC. Some of us interpreted that the
   down-looking pitchfork indicates that 5G NSO communicates directly with the
   components of the TN Orchestration. In contrast, others assumed that the
   demultiplexing of communications from 5G NSO is the function of the TN
   Orchestration. Although we understand that the figure presents an example,
   it seems like it can be made more straightforward since the scope of the
   document is the interactions between RFC 9543 NSC and the Provider Network.
   Thus, I have two suggestions, each can clarify Figure 6 and help a
   reader:
      - Reduce Figure 6 to RFC 9543 NSC and the Provider Network;
      - Alternatively, remove the pitchfork stopping down-looking arrow
      from 5G NSO at the TN Orchestration (how the TN Orchestration
demultiplexes
      that flow to Customer Site Orchestration and RFC 9543 NSC is outside the
      scope of the document).

I appreciate your consideration and profusely apologize for being so late
with the comment.


Regards,

Greg

On Sun, Apr 28, 2024 at 2:20 PM Adrian Farrel <[e-mail]> wrote:

> Hi,
>
> I'm sorry this review comes in late. It's rather a long document and
> needed some care. My main excuse is obvious in the volume of my
> comments.
>
> Maybe my comments boil down to "there are too many words, and the
> material doesn't arrive in an order that makes it easy to understand
> the document." I don't know! I seem to have a lot to say, and a lot of
> it is, "That's unclear. Oh, it gradually becomes clearer in later
> sections." That combined with wondering why some of the material is
> present at all.
>
> I do have a sticking point on Appendix B.
>
> I've updated my review to cover the most recent revision, just posted.
> I hope that helps. It certainly removes any duplication with previous
> reviews.
>
> Cheers,
> Adrian
>
> ===
>
> In general, I wonder whether there is some confusion between the concept
> of slicing, that of a slice service, and that of connectivity. It may
> help to think about the differences (see RFC 9543) in terms of which
> sits where in the figures you have drawn (which are all pretty much
> point-to-point connectivity).
>
> ---
>
> The document could really benefit from the addition of a section called
> "Scalability Considerations."
>
> draft-ietf-teas-nrp-scalability says...
>
>    It is anticipated that any specification of a network slicing
>    protocol solution will include considerations of scalability and
>    discussion of the applicability of the solution.  This will not
>    denigrate any specific solution, but will help clarify the type of
>    deployment in which the solution is optimal while providing advice
>    about its limitations in other deployments.
>
> That seems like good advice and reasoning (even though your document is
> not actually a specification). That draft also gives a lot of help in
> understanding what the scalability concerns are, so you should be able
> to give god advice to people who want to deploy based on your draft as
> to what they should and should not do, and where they can expect to need
> to impose limits. Appendix A.1 of that draft may be particularly
> relevant to your draft.
>
> ---
>
> Although it still has not been adopted by the working group, you may
> find draft-li-teas-composite-network-slices to be a useful reference for
> the discussion of "horizontal composition" of network slices in an IETF
> context. At the least, it should give you some additional thoughts, but
> you might consider it as an informational reference for its discussion
> of multi-domain slices which certainly cuts into a lot of what you are
> talking about.
>
> ---
>
> I think you might find appendix A.4 of RFC 9543 to be a pertinent
> reference.
>
> ---
>
> 3.1 says
>    A model for the Transport Network based on
>    orchestration domains is introduced in Section 3.4.  This model
>    permits to define more precisely where the RFC 9543 Network Slices
>    apply.
>
> That sent me jumping ahead to 3.4 principally to discover the converse,
> that is, where 9543 slices do not apply. My immediate concern was that
> you would be stating that slicing of networks that use IETF technologies
> could somehow be done using a different approach.
>
> In fact, as far as I can tell, 3.4 only talks about TNs that use IETF
> technologies, and only talks about 9543 as the slicing model for those
> TNs.
>
> So I am unclear about the second sentence quoted. Perhaps it is just
> unnecessary?
>
> ---
>
> 3.1
>
>    The term "Transport Network" is used for disambiguation with 5G
>    network (e.g., IP, packet-based forwarding vs RAN and CN).
>    Consequently, the disambiguation applies to Transport Network Slicing
>    vs. 5G End-to-End Network Slicing (Section 3.2) as well the
>    management domains: RAN, CN, and TN domains.
>
> I thought I understood what was meant by TN in this document until I
> reached this paragraph. The previous text in 3.1 (and in the references)
> seems clear as to what a TN is. This text, however, confuses me and I
> can't extract anything useful from it. After all, haven't you just
> explained that:
>
>    Appendix B provides an overview of 5G network building blocks: the
>    Radio Access Network (RAN), Core Network (CN), and Transport Network
>    (TN).  The Transport Network is defined by the 3GPP as the "part
>    supporting connectivity within and between CN and RAN parts"
>    (Section 1 of [TS-28.530]).
>
> ---
>
> 3.2
>
>    Network slicing has a different meaning in the 3GPP mobile and
>    transport worlds.
>
> Firstly, this reads with some ambiguity. I think you mean to say
>
>    Network slicing has a different meaning in the 3GPP mobile world and
>    the transport world.
>
> Second, I think we are probably limiting ourselves to the world made up
> of transport networks built from IETF technologies. So you might say:
>
>    Network slicing has a different meaning in the 3GPP mobile world and
>    the IETF transport network world.
>
> Lastly, this is a substantial assertion. I think your subsequent text is
> supposed to substantiate this assertion rather than be dependent on it.
> So maybe...
>
> OLD
>    Hence, for the sake of precision and without
>    seeking to be exhaustive, this section provides a brief description
>    of the objectives of 5G Network Slicing and Transport Network
>    Slicing:
> NEW
>    This difference can be seen from the descriptions below that set out
>    the objectives of 5G Network Slicing and Transport Network
>    Slicing.  These descriptions are not intended to be exhaustive.
> END
>
> You go on to say...
>
>       The term "TN slice" is used in this document to refer to a slice
>       in the Transport Network domain of the overall 5G architecture.
>
> That is fine, except that you have asserted "the transport world" yet
> you are limiting yourself to "this document." It would be fine to mean
> "this document" (in which case, fix the scope in the earlier assertion),
> and it is fine to mean "transport world," in which case you need (as you
> did for the 3G slicing case) you need to give a reference.
>
> ---
>
> 3.3
>
>    Figure 2 depicts the reference design used for modelling the
>    Transport Network based on management perimeters (Customer vs.
>    Provider).
>
> Is that "...used in this document for modelling..."? In which case I
> have no objection so long as you make this clear.
> Or are you being more prescriptive for the general case? If so, then
> I have significant concerns because you have imposed a restriction to
> only one of the cases shown in Figure 1 of RFC 9543.
>
> In any case, it appears from your Figure 2 and the definitions in this
> section that your slice intends to include more elements of the customer
> network than just the CE. This is, I think, contrary to RFC 9543 that
> you claim to be consistent with. It's a big difference, and depends
> somewhat on the definition of the TN.
>
> I suspect that this difference could be bridged by a clearer
> representation of the TN in Figure 1 (you could show it mapping to
> 'customer network' as well), and in Figure 2 (you could show that the
> link from NF to CE is potentially multi-hop -- you have used the same
> notation as for the AC). In Figure 2, you might also make it clearer
> where the precise end points of the 'Transport Network' are.
>
> However (!) I looked through the rest of the document for the placement
> of SDPs, and it seems you focus on the SDP in the PE, and you certainly
> don't talk about SDPs that are further inside the customer network. So,
> Where do you really think the TN edges are?
>
> ---
>
> 3.3
>
> Your definition of Attachment Circuit is fine, but it embraces the
> potential that the AC is a non-IETF technology in which case it is hard
> to know how the IETF TN would extend beyond the PE.
>
> ---
>
> I wonder what section 3.3.1 adds to the document. Sure, it is a
> tutorial on distributed CEs and PEs, and I don't find any fault with
> it (although it's a bit odd to not find the provider-managed CE
> present at the customer site as one of the examples).
>
> But you close the section saying...
>
>    In subsequent sections of this document, the terms CE and PE are used
>    for both single and distributed devices.
>
> ...so why do we need this tutorial?
>
> I wonder whether you might successfully collapse each of the definitions
> of "distributed CE", "distributed PE", "co-managed CE", and "service-
> aware CE" into short paragraphs, and just add them to the terminology
> list (removing sections 3.3.1, 3.3.2, and 3.3.3).
>
> ---
>
> Section 3.3.3 seems confused about what technologies might be in use.
> I see IP, MPLS, and SR mentioned at different places in the text. But
> what about ACs that use Ethernet?
>
> You might move Figure 4 to closer to sections 4.3.2 and 4.3.3 where it
> is used and more valuable.
>
> ---
>
> 3.4.1
>
> Accidental indentation of the second paragraph?
>
> Broken reference {#sec-ref-design}. Presume 3.3?
>
> ---
>
> 3.4.1 (notwithstanding reading 3.4.2)
>
> This is confusing.
>
>    This section introduces a global framework for the orchestration of a
>    5G end-to-end slice (a.k.a. 5G Network Slice) with a zoom on TN
>    parts.  This framework helps to delimit the realization scope of RFC
>    9543 Network Slices and identify interactions that are required for
>    the realization of such slices.
>
> I don't see such a delimitation of realization in the text that follows.
> Nor do I understand whether that delimitation is supposed to apply to
> all cases of 9543 realization, or just to the realization described in
> this document.
>
>       This framework is consistent with the management coordination
>       example shown in Figure 4.7.1 of [TS-28.530].
>
>    In reference to Figure 5, a 5G End-to-End Network Slice Orchestrator
>    (5G NSO) is responsible for orchestrating 5G Network Slices end-to-
>    end.  The details of the 5G NSO is out of the scope of this document.
>
> Except that there is an interface between the 5G NSO and the components
> that you describe as fundamental parts of your realisation. You can't
> realise this without understanding that interface.
>
>    The realization of the 5G Network Slices spans RAN, CN, and TN.  As
>    mentioned in Section 3.1, the RAN and CN are under the responsibility
>    of the 3GPP Management System, while the TN is not.  The
>    orchestration of the TN is split into two sub-domains in conformance
>    with the reference design in {#sec-ref-design}:
>
>    Provider Network Orchestration domain:  As defined in [RFC9543], the
>       provider relies on a Network Slice Controller (NSC) to manage and
>       orchestrate RFC 9543 Network Slices in the provider network.  This
>       framework permits to manage connectivity together with SLOs.
>
> I would note that the NSC in 9543 may orchestrate the AC and the CPE as
> well. Do you consider those elements to be part of the provider network?
>
>    Customer Site Orchestration domain:  The Orchestration of TN elements
>       of the customer sites relies upon a variety of controllers (e.g.,
>       Fabric Manager, Element Management System, or VIM).  The
>       realization of this segment does not involve the Transport Network
>       Orchestration.
>
> So, are the TN elements of the customer site part of the Transport
> Network? The term "TN" would seem to suggest so. I would assume that the
> "Transport Network Orchestration" is the orchestration of the Transport
> Network. So how do you orchestrate parts of the TN without being part of
> the Transport Network Orchestration?
>
> ---
>
> Figure 5 finally makes it clear that you are trying to distinguish a
> "network slice" from a "TN slice". In practice, I think you are trying
> to say that the slices of the different domains may be combined to
> form an end-to-end slice in the IP/MPLS technology. This is certainly
> supported by 3.4.2 and is consistent with draft-li-teas-composite-
> network-slices, but you need to work out which way you are slicing (sic)
> this:
>
> - You could be slicing each domain and stitching the slices together, in
>   which case, you don't need nearly as much detail because each domain
>   is sliced under its own slice controller/orchestrator, and the slices
>   are simply joined.
> - You could be performing a variant of the above where multiple customer
>   domain slices may be carried across the provider network by a single
>   provider domain slice in a hierarchical manner.
> - You could be performing a single slicing operation, end-to-end across
>   each of the domains, in which case your SDPs are in the wrong place.
>
> I would say:
> 1. Figure 5 is frightfully late in the document to reach an
>    understanding of your terminology
> 2. You need to work out what model you are trying to use for your
>    realisation, explain it, and stick to it.
>
> ---
>
> 3.4.2
>
>       The realization of this segment is driven by the 5G Network
>       Orchestration and potentially the Customer Site Orchestration.
>       The realization of this segment does not involve the Transport
>       Network Orchestration.
>
> So, Figure 5 shows that parts of the Customer Segment (specifically
> the NF) is under the orchestration of the "3GPP Domains Orchestration",
> but that other parts (actually, the whole customer site, but
> specifically the CE) are under the orchestration of the "Customer Site
> Orchestration". How is that consistent with your "potentially" text?
>
> Further, you say "does not involve the Transport Network Orchestration"
> but Figure 5 clearly shows that "Customer Site Orchestration" is part of
> "TN Orchestration".
>
> What does "driven by" mean?
>
> ---
>
> In 3.4.2 and with reference to Figure 5, it appears that your
> realisation is based on RFC 9543 Figure 1 Type 3. That's great, could
> you say so somewhere early in the document? It would help. (It still
> wouldn't make clear whether you are stitching domain slices or running
> a full end-to-end slicing operation, but it might help drive the
> answer.)
>
> By the time we get to Figure 6, you are talking about "slice segments"
> and that is really helping because now we can consider stitching those
> segments together.
>
> On the other hand, you also say that the customer domain slice can be
> considered part of the RAN/CN domain and sliced accordingly. If they are
> part of the RAN/CN domain, they are not part of the TN domain, so
> perhaps there is a lot here that simply doesn't need to be said.
>
> ---
>
> 3.4.2
>
>    In other words, the main
>    focus for the enforcement of end-to-end SLOs is managed at the
>    Network Slice between PE interfaces connected to the AC.
>
> Would that be more clearly stated with reference to the SDP?
>
> ---
>
> 3.4.2
>
>       Automating the provisioning and management of the AC is
>       recommended.
>
> Hmmm. That probably needs some justification, but when you put in that
> text you might just give the benefits of automation and leave out the
> recommendation.
>
> BTW, do you consider an active control plane to be automation or does
> the communication have to be between the orchestrators/controllers as
> shown in Figure 7?
>
> ---
>
> 3.5
>
> eMBB needs expansion on first use and/or an entry in the Appendix
>
> ---
>
> 3.5
>
> There seems to be a difference between the title of the section...
>       Mapping Schemes Between 5G Network Slices and Transport Network
>       Slices
> ...and the first line of text
>    There are multiple options for mapping 5G Network Slices to TN
>    slices:
> That is, the text talks about a unidirectional mapping (5G to TN)
> while the title says "between".
>
> But I think I object to the word "mapping". While, in one direction, the
> word is fine and clearly describes how one type of slice is projected
> onto another type of slice, the problem is more complicated because in
> the other direction (at the receiving end of the data flow) we need to
> "un-map".
>
> Additionally, I wonder whether the idea of 1:N is real. Of course, the
> example you give is real (carrying CP and UP on different resources
> over the TN), but I wonder whether that is really only one 5G slice or,
> in fact, two. If the traffic uses only one slice in the RAN, then it
> seems that when it is handed off to the TN it would all appear as a
> single flow and could not be separated across the two TN slices. That
> doesn't stop M:N (see 3.6) being credible because in that case the
> 5G slice pairs (UP and CP) are "mapped" to one TN slice for all CP,
> and individual or aggregated TN slices for UP traffic. 3.6 seems to
> recognise this by having a global 5G slice for eMBB (i.e., not just a
> special TN slice).
>
> ---
>
> 3.5
>
>    *  1 to N: A single 5G Network Slice can be mapped to multiple TN
>       slices (1 to N).  For instance, consider the scenario depicted in
>       Figure 8, illustrating the separation of the 5G Control Plane and
>       User Plane in TN slices for a single 5G eMBB network slice.  It is
>       important to note that this mapping can serve as an interim step
>       to N:M mapping.  In this scenario, a subset of the TN slices can
>       be intended for sharing by multiple 5G network slices (e.g., the
>       Control Plane TN slice is shared by multiple 5G network Slices).
>       Further details about this scheme are described in Section 3.6.
>
> s/N:M mapping/M:N mapping/ !
>
> I think that the sentence "In this scenario..." is describing M:N. Move
> it to the definition of M:N.
>
> Is the final sentence talking about 1:N or M:N? I think probably M:N. So
> move that, too.
>
> ---
>
> 3.5
>
>    In practice, for operational and scaling reasons, typically M to N
>    would be used, with M >> N.
>
> This is good.
> If you are considering 1:N and M:1 as actual cases (not subsumed into
> M:N as special cases) then I think you have more to say because 1:N
> clearly has scaling concerns even if N is small when the number of 5G
> slices is large.
> Further, 1:1 and M:1 may have scaling concerns when the number of 5G
> slices is large.
>
> So you are probably saying that:
> - all deployments are M:N
> - M >> N
> - M is a key factor in scaling as the number of 5G slices increases
>
> ---
>
> 3.7
>
> eCPRI needs a reference
>
> ---
>
> 3.7
>
> OLD
>       and a
>       Layer 2 or Layer 3 for fronthaul connections
> NEW
>       and
>       Layer 2 or Layer 3 delivery for fronthaul connections
> END
>
> ---
>
> 3.7
>
>    *  Coarse-grained resource control at the transit (non-attachment
>       circuits) links in the provider network, using a single NRP,
>       spanning the entire provider network.
>
> This is the first time you have mentioned NRPs despite 20 pages
> establishing the realization model. Doesn't that seem a bit strange?
>
> The figure says "base NRP" like it is considering other NRPs.
>
> ---
>
> 3.7
>
>       The role of capacity management is to ensure the provider network
>       capacity can be utilized without causing any bottlenecks.  The
>       toolset used here can range from careful network planning, to
>       ensure a more or less equal traffic distribution (i.e., equal cost
>       load balancing), to advanced TE techniques, with or without
>       bandwidth reservations, to force more consistent load distribution
>       even in non-ECMP friendly network topologies.
>
> This is a bit of a stretch as a description of "toolset". Maybe
> "methods"? Unless you can think of specific tools.
>
> ---
>
> 3.7
>
>    This document does not describe in detail how to manage an L2VPN or
>    L3VPN, as this is already well-documented.
>
> Care to say where?
>
> ---
>
> 4.
>
>    these methods are not
>    reproduced here because of the intrinsic shortcomings of these
>    methods.
>
> I think probably s/reproduced/discussed/
>
> Now, you say "intrinsic shortcomings" and some might find that
> pejorative. Does draft-ietf-teas-5g-network-slice-application describe
> those shortcomings? If so, you might just include a pointer. Otherwise,
> you either have to describe the shortcomings (not recommended) or
> simplify the text because we are not interested in what you don't do
> and more interested in what you do do (and why).
>
> ---
>
> Section 4 is pretty clear and helpful. Thanks. I think it is where the
> real work of the draft begins (23 pages in). I wonder whether we can do
> something to get here more quickly.
>
> ---
>
> Figure 14
>
> I see why you have used 2001:0db8
> Well done for using a documentation range.
> However, your "example" then moves on to use x, t, and d
> So I think you are not really doing an example so much as showing the
> format and you could go to:
>    xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:ttdd:dddd
> And not say "example".
>
> Figure 15 is a different matter, and is good with the documentation
> range.
>
> ---
>
> 4.3.2
>
>    In the BGP control plane, when exchanging service prefixes
>    over attachment circuit, each slice might be represented by a unique
>    BGP community, so tracking label assignment to the slice is possible.
>
> "might be"?
>
> ---
>
> 4.3.2 and 4.3.3
>
> Just wondering whether these use an existing BGP mechanism (in which
> case, add a reference), or need a small protocol extension to define the
> community that matches a slice.
>
> ---
>
> Section 5 is also clear, but seems really, really complex. I wonder
> whether these techniques are likely to be error-prone.
>
> I'd be particularly interested to know what arrangements are needed
> when the TN is comprised of multiple administrative domains (possibly
> from different providers). How are the uses of QoS codepoints
> coordinated and maintained?
>
> ---
>
> 5.2.2.1
>
> I think there is a punctuation error.
> s/provider network inbound policy/provider network, inbound policy/
>
> ---
>
> In Section 6, have you invented the Filter Topology when you use the
> term "transport plane"? I think you have, and it would be helpful
> either:
> - to say "when we say transport plane, this is equivalent to the term
>   Filter Topology defined in RFC 9542"
> - to replace all mentions of "transport plane"
>
> I prefer the second of these.
>
> ---
>
> 7.2.1
>
> I think the solution to handling variations in demand is what is
> sometimes called "metric tweaking".
>
> ---
>
> 7.2.2 and 7.2.3
>
> I'm not sure that SR-TE as described in 9256 is limited to LSPs. You
> might just say "path".
>
> ---
>
> 7.2.3 seems to be floating a few ways of doing things in a rather
> hypothetical way. Shouldn't you be referencing the technologies that
> enable this?
>
> ---
>
> 8.
>
>       SFC OAM [RFC9451] should also be supported for slices that make
>       uses of service function chaining [RFC7665].  An example of SFC
>       OAM technique to Continuity Check, Connectivity Verification, or
>       tracing service functions is specified in [RFC9516].
>
> This paragraph is completely true. But why is it here? You have not
> mentioned SFC anywhere in the document.
>
> ---
>
> Section 8 seems to focus on the provider network. It's all good stuff,
> but your TN slice appears to go outside the provider network. So what
> about OAM for the whole TN slice?
>
> ---
>
> Appendix A possibly includes some terms not used in the document. I see
> CSP and PLMN. You might check the others.
>
> Are we sure
>    SMF: Session Management Function
> Not
>    SMF: Subnet Management Function
> per 3.4.1?
>
> ---
>
> I am worried by the presence of Appendix B. I appreciate it being moved
> out of the body of the text, and I welcome the caveat at the start of
> the appendix, but what are we, the IETF, doing publishing an RFC that
> seeks to explain elements of the 3GPP architecture?
>
> - That's not our business
> - I don't see how we can get IETF consensus on it
> - I think it at the very least needs formal review and approval by 3GPP
>
> For me this is a sticking point. I strongly believe that this appendix
> should be removed from the document.
>
> ---
>
> Contributors
>
> You might want to check John Drake's coordinates.
>
> _______________________________________________
> Teas mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/teas
>