[Teas] Late WGLC review of draft-ietf-teas-5g-ns-ip-mpls

[Adrian Farrel <[e-mail]>]

Hi,

I'm sorry this review comes in late. It's rather a long document and
needed some care. My main excuse is obvious in the volume of my
comments.

Maybe my comments boil down to "there are too many words, and the 
material doesn't arrive in an order that makes it easy to understand
the document." I don't know! I seem to have a lot to say, and a lot of
it is, "That's unclear. Oh, it gradually becomes clearer in later 
sections." That combined with wondering why some of the material is
present at all.

I do have a sticking point on Appendix B.

I've updated my review to cover the most recent revision, just posted.
I hope that helps. It certainly removes any duplication with previous
reviews.

Cheers,
Adrian

===

In general, I wonder whether there is some confusion between the concept
of slicing, that of a slice service, and that of connectivity. It may
help to think about the differences (see RFC 9543) in terms of which 
sits where in the figures you have drawn (which are all pretty much 
point-to-point connectivity).

---

The document could really benefit from the addition of a section called
"Scalability Considerations."

draft-ietf-teas-nrp-scalability says...

   It is anticipated that any specification of a network slicing
   protocol solution will include considerations of scalability and
   discussion of the applicability of the solution.  This will not
   denigrate any specific solution, but will help clarify the type of
   deployment in which the solution is optimal while providing advice
   about its limitations in other deployments.

That seems like good advice and reasoning (even though your document is
not actually a specification). That draft also gives a lot of help in
understanding what the scalability concerns are, so you should be able
to give god advice to people who want to deploy based on your draft as
to what they should and should not do, and where they can expect to need
to impose limits. Appendix A.1 of that draft may be particularly
relevant to your draft.

---

Although it still has not been adopted by the working group, you may
find draft-li-teas-composite-network-slices to be a useful reference for
the discussion of "horizontal composition" of network slices in an IETF
context. At the least, it should give you some additional thoughts, but
you might consider it as an informational reference for its discussion
of multi-domain slices which certainly cuts into a lot of what you are
talking about.

---

I think you might find appendix A.4 of RFC 9543 to be a pertinent 
reference.

---

3.1 says
   A model for the Transport Network based on
   orchestration domains is introduced in Section 3.4.  This model   
   permits to define more precisely where the RFC 9543 Network Slices
   apply.

That sent me jumping ahead to 3.4 principally to discover the converse,
that is, where 9543 slices do not apply. My immediate concern was that
you would be stating that slicing of networks that use IETF technologies
could somehow be done using a different approach. 

In fact, as far as I can tell, 3.4 only talks about TNs that use IETF
technologies, and only talks about 9543 as the slicing model for those 
TNs.

So I am unclear about the second sentence quoted. Perhaps it is just
unnecessary?

---

3.1

   The term "Transport Network" is used for disambiguation with 5G
   network (e.g., IP, packet-based forwarding vs RAN and CN).
   Consequently, the disambiguation applies to Transport Network Slicing
   vs. 5G End-to-End Network Slicing (Section 3.2) as well the
   management domains: RAN, CN, and TN domains.

I thought I understood what was meant by TN in this document until I
reached this paragraph. The previous text in 3.1 (and in the references)
seems clear as to what a TN is. This text, however, confuses me and I
can't extract anything useful from it. After all, haven't you just
explained that:

   Appendix B provides an overview of 5G network building blocks: the
   Radio Access Network (RAN), Core Network (CN), and Transport Network
   (TN).  The Transport Network is defined by the 3GPP as the "part
   supporting connectivity within and between CN and RAN parts"
   (Section 1 of [TS-28.530]).

---

3.2

   Network slicing has a different meaning in the 3GPP mobile and
   transport worlds.

Firstly, this reads with some ambiguity. I think you mean to say

   Network slicing has a different meaning in the 3GPP mobile world and
   the transport world.

Second, I think we are probably limiting ourselves to the world made up
of transport networks built from IETF technologies. So you might say:

   Network slicing has a different meaning in the 3GPP mobile world and
   the IETF transport network world.

Lastly, this is a substantial assertion. I think your subsequent text is
supposed to substantiate this assertion rather than be dependent on it.
So maybe...

OLD
   Hence, for the sake of precision and without
   seeking to be exhaustive, this section provides a brief description
   of the objectives of 5G Network Slicing and Transport Network
   Slicing:
NEW
   This difference can be seen from the descriptions below that set out
   the objectives of 5G Network Slicing and Transport Network
   Slicing.  These descriptions are not intended to be exhaustive.
END

You go on to say...

      The term "TN slice" is used in this document to refer to a slice
      in the Transport Network domain of the overall 5G architecture.

That is fine, except that you have asserted "the transport world" yet 
you are limiting yourself to "this document." It would be fine to mean
"this document" (in which case, fix the scope in the earlier assertion),
and it is fine to mean "transport world," in which case you need (as you
did for the 3G slicing case) you need to give a reference.

---

3.3

   Figure 2 depicts the reference design used for modelling the
   Transport Network based on management perimeters (Customer vs.
   Provider).

Is that "...used in this document for modelling..."? In which case I
have no objection so long as you make this clear.
Or are you being more prescriptive for the general case? If so, then
I have significant concerns because you have imposed a restriction to
only one of the cases shown in Figure 1 of RFC 9543.

In any case, it appears from your Figure 2 and the definitions in this
section that your slice intends to include more elements of the customer
network than just the CE. This is, I think, contrary to RFC 9543 that
you claim to be consistent with. It's a big difference, and depends
somewhat on the definition of the TN.

I suspect that this difference could be bridged by a clearer
representation of the TN in Figure 1 (you could show it mapping to 
'customer network' as well), and in Figure 2 (you could show that the
link from NF to CE is potentially multi-hop -- you have used the same
notation as for the AC). In Figure 2, you might also make it clearer
where the precise end points of the 'Transport Network' are.

However (!) I looked through the rest of the document for the placement
of SDPs, and it seems you focus on the SDP in the PE, and you certainly
don't talk about SDPs that are further inside the customer network. So,
Where do you really think the TN edges are?

---

3.3

Your definition of Attachment Circuit is fine, but it embraces the 
potential that the AC is a non-IETF technology in which case it is hard
to know how the IETF TN would extend beyond the PE.

---

I wonder what section 3.3.1 adds to the document. Sure, it is a 
tutorial on distributed CEs and PEs, and I don't find any fault with 
it (although it's a bit odd to not find the provider-managed CE 
present at the customer site as one of the examples).

But you close the section saying...

   In subsequent sections of this document, the terms CE and PE are used
   for both single and distributed devices.

...so why do we need this tutorial?

I wonder whether you might successfully collapse each of the definitions
of "distributed CE", "distributed PE", "co-managed CE", and "service-
aware CE" into short paragraphs, and just add them to the terminology
list (removing sections 3.3.1, 3.3.2, and 3.3.3).

---

Section 3.3.3 seems confused about what technologies might be in use.
I see IP, MPLS, and SR mentioned at different places in the text. But
what about ACs that use Ethernet?

You might move Figure 4 to closer to sections 4.3.2 and 4.3.3 where it
is used and more valuable.

---

3.4.1

Accidental indentation of the second paragraph?

Broken reference {#sec-ref-design}. Presume 3.3?

---

3.4.1 (notwithstanding reading 3.4.2)

This is confusing. 

   This section introduces a global framework for the orchestration of a
   5G end-to-end slice (a.k.a. 5G Network Slice) with a zoom on TN
   parts.  This framework helps to delimit the realization scope of RFC
   9543 Network Slices and identify interactions that are required for
   the realization of such slices.
   
I don't see such a delimitation of realization in the text that follows.   
Nor do I understand whether that delimitation is supposed to apply to 
all cases of 9543 realization, or just to the realization described in 
this document.

      This framework is consistent with the management coordination
      example shown in Figure 4.7.1 of [TS-28.530].

   In reference to Figure 5, a 5G End-to-End Network Slice Orchestrator
   (5G NSO) is responsible for orchestrating 5G Network Slices end-to-
   end.  The details of the 5G NSO is out of the scope of this document.

Except that there is an interface between the 5G NSO and the components
that you describe as fundamental parts of your realisation. You can't
realise this without understanding that interface.

   The realization of the 5G Network Slices spans RAN, CN, and TN.  As
   mentioned in Section 3.1, the RAN and CN are under the responsibility
   of the 3GPP Management System, while the TN is not.  The
   orchestration of the TN is split into two sub-domains in conformance
   with the reference design in {#sec-ref-design}:

   Provider Network Orchestration domain:  As defined in [RFC9543], the
      provider relies on a Network Slice Controller (NSC) to manage and
      orchestrate RFC 9543 Network Slices in the provider network.  This
      framework permits to manage connectivity together with SLOs.

I would note that the NSC in 9543 may orchestrate the AC and the CPE as
well. Do you consider those elements to be part of the provider network?

   Customer Site Orchestration domain:  The Orchestration of TN elements
      of the customer sites relies upon a variety of controllers (e.g.,
      Fabric Manager, Element Management System, or VIM).  The
      realization of this segment does not involve the Transport Network
      Orchestration.

So, are the TN elements of the customer site part of the Transport 
Network? The term "TN" would seem to suggest so. I would assume that the
"Transport Network Orchestration" is the orchestration of the Transport
Network. So how do you orchestrate parts of the TN without being part of
the Transport Network Orchestration?

---

Figure 5 finally makes it clear that you are trying to distinguish a
"network slice" from a "TN slice". In practice, I think you are trying 
to say that the slices of the different domains may be combined to
form an end-to-end slice in the IP/MPLS technology. This is certainly
supported by 3.4.2 and is consistent with draft-li-teas-composite-
network-slices, but you need to work out which way you are slicing (sic)
this:

- You could be slicing each domain and stitching the slices together, in
  which case, you don't need nearly as much detail because each domain
  is sliced under its own slice controller/orchestrator, and the slices
  are simply joined.
- You could be performing a variant of the above where multiple customer
  domain slices may be carried across the provider network by a single
  provider domain slice in a hierarchical manner.
- You could be performing a single slicing operation, end-to-end across
  each of the domains, in which case your SDPs are in the wrong place.

I would say:
1. Figure 5 is frightfully late in the document to reach an 
   understanding of your terminology
2. You need to work out what model you are trying to use for your
   realisation, explain it, and stick to it.

---

3.4.2

      The realization of this segment is driven by the 5G Network
      Orchestration and potentially the Customer Site Orchestration.
      The realization of this segment does not involve the Transport
      Network Orchestration.

So, Figure 5 shows that parts of the Customer Segment (specifically 
the NF) is under the orchestration of the "3GPP Domains Orchestration",
but that other parts (actually, the whole customer site, but 
specifically the CE) are under the orchestration of the "Customer Site
Orchestration". How is that consistent with your "potentially" text?

Further, you say "does not involve the Transport Network Orchestration"
but Figure 5 clearly shows that "Customer Site Orchestration" is part of
"TN Orchestration".

What does "driven by" mean?

---

In 3.4.2 and with reference to Figure 5, it appears that your
realisation is based on RFC 9543 Figure 1 Type 3. That's great, could
you say so somewhere early in the document? It would help. (It still 
wouldn't make clear whether you are stitching domain slices or running
a full end-to-end slicing operation, but it might help drive the
answer.)

By the time we get to Figure 6, you are talking about "slice segments"
and that is really helping because now we can consider stitching those
segments together. 

On the other hand, you also say that the customer domain slice can be
considered part of the RAN/CN domain and sliced accordingly. If they are
part of the RAN/CN domain, they are not part of the TN domain, so 
perhaps there is a lot here that simply doesn't need to be said.

---

3.4.2

   In other words, the main
   focus for the enforcement of end-to-end SLOs is managed at the
   Network Slice between PE interfaces connected to the AC.

Would that be more clearly stated with reference to the SDP?

---

3.4.2

      Automating the provisioning and management of the AC is
      recommended.

Hmmm. That probably needs some justification, but when you put in that
text you might just give the benefits of automation and leave out the
recommendation.

BTW, do you consider an active control plane to be automation or does
the communication have to be between the orchestrators/controllers as
shown in Figure 7?

---

3.5

eMBB needs expansion on first use and/or an entry in the Appendix

---

3.5

There seems to be a difference between the title of the section...
      Mapping Schemes Between 5G Network Slices and Transport Network
      Slices
...and the first line of text
   There are multiple options for mapping 5G Network Slices to TN
   slices:
That is, the text talks about a unidirectional mapping (5G to TN)
while the title says "between".

But I think I object to the word "mapping". While, in one direction, the
word is fine and clearly describes how one type of slice is projected
onto another type of slice, the problem is more complicated because in
the other direction (at the receiving end of the data flow) we need to
"un-map". 

Additionally, I wonder whether the idea of 1:N is real. Of course, the
example you give is real (carrying CP and UP on different resources 
over the TN), but I wonder whether that is really only one 5G slice or,
in fact, two. If the traffic uses only one slice in the RAN, then it
seems that when it is handed off to the TN it would all appear as a
single flow and could not be separated across the two TN slices. That
doesn't stop M:N (see 3.6) being credible because in that case the 
5G slice pairs (UP and CP) are "mapped" to one TN slice for all CP, 
and individual or aggregated TN slices for UP traffic. 3.6 seems to
recognise this by having a global 5G slice for eMBB (i.e., not just a
special TN slice).

---

3.5

   *  1 to N: A single 5G Network Slice can be mapped to multiple TN
      slices (1 to N).  For instance, consider the scenario depicted in
      Figure 8, illustrating the separation of the 5G Control Plane and
      User Plane in TN slices for a single 5G eMBB network slice.  It is
      important to note that this mapping can serve as an interim step
      to N:M mapping.  In this scenario, a subset of the TN slices can
      be intended for sharing by multiple 5G network slices (e.g., the
      Control Plane TN slice is shared by multiple 5G network Slices).
      Further details about this scheme are described in Section 3.6.

s/N:M mapping/M:N mapping/ !

I think that the sentence "In this scenario..." is describing M:N. Move
it to the definition of M:N.

Is the final sentence talking about 1:N or M:N? I think probably M:N. So
move that, too.

---

3.5

   In practice, for operational and scaling reasons, typically M to N
   would be used, with M >> N.

This is good. 
If you are considering 1:N and M:1 as actual cases (not subsumed into
M:N as special cases) then I think you have more to say because 1:N
clearly has scaling concerns even if N is small when the number of 5G
slices is large.
Further, 1:1 and M:1 may have scaling concerns when the number of 5G
slices is large.

So you are probably saying that:
- all deployments are M:N
- M >> N
- M is a key factor in scaling as the number of 5G slices increases

---

3.7

eCPRI needs a reference

---

3.7

OLD
      and a
      Layer 2 or Layer 3 for fronthaul connections
NEW
      and 
      Layer 2 or Layer 3 delivery for fronthaul connections
END

---

3.7

   *  Coarse-grained resource control at the transit (non-attachment
      circuits) links in the provider network, using a single NRP,
      spanning the entire provider network.

This is the first time you have mentioned NRPs despite 20 pages
establishing the realization model. Doesn't that seem a bit strange?

The figure says "base NRP" like it is considering other NRPs.

---

3.7

      The role of capacity management is to ensure the provider network
      capacity can be utilized without causing any bottlenecks.  The
      toolset used here can range from careful network planning, to
      ensure a more or less equal traffic distribution (i.e., equal cost
      load balancing), to advanced TE techniques, with or without
      bandwidth reservations, to force more consistent load distribution
      even in non-ECMP friendly network topologies.

This is a bit of a stretch as a description of "toolset". Maybe 
"methods"? Unless you can think of specific tools.

---

3.7

   This document does not describe in detail how to manage an L2VPN or
   L3VPN, as this is already well-documented.

Care to say where?

---

4.

   these methods are not
   reproduced here because of the intrinsic shortcomings of these
   methods.

I think probably s/reproduced/discussed/

Now, you say "intrinsic shortcomings" and some might find that
pejorative. Does draft-ietf-teas-5g-network-slice-application describe
those shortcomings? If so, you might just include a pointer. Otherwise,
you either have to describe the shortcomings (not recommended) or 
simplify the text because we are not interested in what you don't do
and more interested in what you do do (and why).

---

Section 4 is pretty clear and helpful. Thanks. I think it is where the
real work of the draft begins (23 pages in). I wonder whether we can do
something to get here more quickly.

---

Figure 14

I see why you have used 2001:0db8 
Well done for using a documentation range.
However, your "example" then moves on to use x, t, and d
So I think you are not really doing an example so much as showing the
format and you could go to:
   xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:ttdd:dddd
And not say "example".

Figure 15 is a different matter, and is good with the documentation 
range.

---

4.3.2

   In the BGP control plane, when exchanging service prefixes
   over attachment circuit, each slice might be represented by a unique
   BGP community, so tracking label assignment to the slice is possible.

"might be"?

---

4.3.2 and 4.3.3

Just wondering whether these use an existing BGP mechanism (in which 
case, add a reference), or need a small protocol extension to define the
community that matches a slice.

---

Section 5 is also clear, but seems really, really complex. I wonder 
whether these techniques are likely to be error-prone.

I'd be particularly interested to know what arrangements are needed
when the TN is comprised of multiple administrative domains (possibly
from different providers). How are the uses of QoS codepoints 
coordinated and maintained?

---

5.2.2.1

I think there is a punctuation error.
s/provider network inbound policy/provider network, inbound policy/

---

In Section 6, have you invented the Filter Topology when you use the
term "transport plane"? I think you have, and it would be helpful
either:
- to say "when we say transport plane, this is equivalent to the term
  Filter Topology defined in RFC 9542"
- to replace all mentions of "transport plane"

I prefer the second of these.

---

7.2.1

I think the solution to handling variations in demand is what is
sometimes called "metric tweaking". 

---

7.2.2 and 7.2.3

I'm not sure that SR-TE as described in 9256 is limited to LSPs. You
might just say "path".

---

7.2.3 seems to be floating a few ways of doing things in a rather
hypothetical way. Shouldn't you be referencing the technologies that
enable this?

---

8.

      SFC OAM [RFC9451] should also be supported for slices that make
      uses of service function chaining [RFC7665].  An example of SFC
      OAM technique to Continuity Check, Connectivity Verification, or
      tracing service functions is specified in [RFC9516].

This paragraph is completely true. But why is it here? You have not 
mentioned SFC anywhere in the document.

---

Section 8 seems to focus on the provider network. It's all good stuff,
but your TN slice appears to go outside the provider network. So what
about OAM for the whole TN slice?

---

Appendix A possibly includes some terms not used in the document. I see
CSP and PLMN. You might check the others.

Are we sure
   SMF: Session Management Function
Not 
   SMF: Subnet Management Function
per 3.4.1?

---

I am worried by the presence of Appendix B. I appreciate it being moved
out of the body of the text, and I welcome the caveat at the start of
the appendix, but what are we, the IETF, doing publishing an RFC that
seeks to explain elements of the 3GPP architecture? 

- That's not our business
- I don't see how we can get IETF consensus on it
- I think it at the very least needs formal review and approval by 3GPP

For me this is a sticking point. I strongly believe that this appendix 
should be removed from the document.

---

Contributors

You might want to check John Drake's coordinates.