Re: [RTG-DIR] RtgDir Early review: draft-ietf-mpls-mna-fwk-06.txt

Toerless Eckert <tte@cs.fau.de> Fri, 03 May 2024 16:53 UTC

Return-Path: <eckert@i4.informatik.uni-erlangen.de>
X-Original-To: rtg-dir@ietfa.amsl.com
Delivered-To: rtg-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2ACB3C1D5C4A; Fri, 3 May 2024 09:53:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.65
X-Spam-Level:
X-Spam-Status: No, score=-1.65 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v-Oa7QNcTQHT; Fri, 3 May 2024 09:53:53 -0700 (PDT)
Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 65989C1654EB; Fri, 3 May 2024 09:53:29 -0700 (PDT)
Received: from faui48e.informatik.uni-erlangen.de (faui48e.informatik.uni-erlangen.de [131.188.34.51]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTPS id 4VWH1d6CXtznkMc; Fri, 3 May 2024 18:53:25 +0200 (CEST)
Received: by faui48e.informatik.uni-erlangen.de (Postfix, from userid 10463) id 4VWH1d59gGzknYm; Fri, 3 May 2024 18:53:25 +0200 (CEST)
Date: Fri, 03 May 2024 18:53:25 +0200
From: Toerless Eckert <tte@cs.fau.de>
To: Tony Li <tony.li@tony.li>
Cc: mpls-chairs <mpls-chairs@ietf.org>, draft-ietf-mpls-mna-fwk@ietf.org, Routing Directorate <rtg-dir@ietf.org>, mpls <mpls@ietf.org>
Message-ID: <ZjUWhbcVAK03Fir2@faui48e.informatik.uni-erlangen.de>
References: <Zg3kupJ9jovwpzbY@faui48e.informatik.uni-erlangen.de> <1E0C0F9D-87A1-4790-925F-DFE1EB480983@tony.li>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <1E0C0F9D-87A1-4790-925F-DFE1EB480983@tony.li>
Archived-At: <https://mailarchive.ietf.org/arch/msg/rtg-dir/7es_CobZUtk-1RRbkle-CE70z_8>
Subject: Re: [RTG-DIR] RtgDir Early review: draft-ietf-mpls-mna-fwk-06.txt
X-BeenThere: rtg-dir@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Routing Area Directorate <rtg-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtg-dir>, <mailto:rtg-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rtg-dir/>
List-Post: <mailto:rtg-dir@ietf.org>
List-Help: <mailto:rtg-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtg-dir>, <mailto:rtg-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 03 May 2024 16:53:56 -0000

Thanks a lot Tony,

I've left points / replies unchanged when your reply has resolved my concern, and commented
only on the points i still have issues with.

Cheers
    Toerless

On Fri, Apr 05, 2024 at 11:28:29AM -0700, Tony Li wrote:
> [WG chair hat: off]
> 
> 
> Hi Toerless,
> 
> Thank you for your comments.  Please see inline.  I have taken the liberty of deleting some of the quoted text.
> 
> 
> > 1.2 I recently managed to observe a meeting in which third parties who attempted to implement proposed
> > solutions where also unclear about core aspects, such as what degree of lookahead into the stack
> > for sub-stacks is assumed to be required, and i can not find this explained in this framework
> > (nor the requirements document).
> 
> Please recall that the framework is not the only document necessary for constructing an implementation.
> The specific solution document, in this case draft-ietf-mpls-mna-hdr, is also necessary. While RLD is
> discussed here in the framework, details about RLD are up to the solution document.

Sorry, i think you misunderstood me, i probably did not describe the problem correctly.

I am just missing any explanation where in the label stack a NAS should be located so that it will
be acted upon. Is it exactly the same lookahead rules as for ECMP ? Also in the presence of multiple NAS ?
(aka: only need to find the first NAS ?)

> > From my little experience i gather that some pre-existing special label functionality does
> > require more lookahead to find them in the label stack than being directly below the TOS LSE,
> > and i don't think this option is written into rfc3031, which is the only relevant reference
> > mentioned here. So i think some text about the expected supported methods to find a NAS
> > would be welcome text to make readers better understand expectations against NAS. Including
> > the appropriate references if it is based on prior mechanisms.
> 
> Lookahead processing in the label stack was introduced by the Entropy Label.

See above. "introduced" does not imply exactly the same way as. And i don't know what other lookahead
processing there has been in MPLS since then and if they all use exactly the same lookahead mechanism.
And if that lookahead mechanism is easily described, then better repeat it here than rely on the
ECP RFC text for it, given how central it is to the concept of MNA ISD.

> Section 1.2 is the terminology section, so I’m unclear as to exactly where in the document this concern arises.
> I will add a comment about label stack processing and reference to the Entropy Label.

I could not locate any new comment in -07, only the reference.  > 

> > 2. The framework document does not explain unambiguously whether a single packet or a single network
> > was supposed to support the simultaneous deployment/use of only one or multiple MNA solutions.
> 
> 
> That is correct.  Again, details are left to solutions documents.  If a particular ISD solution is incompatible 
> with PSD, that would be up to the solution to document.

This question was noy only about PSD i thnk, but also about having ISD substacks from multiple
solutions. Is it permitted to have ISD from two solutions in a single label stack ? Is there
a reason why this would not work ?

In any case, it would be good if the fwk doc would state explicitly these issues about co-existance
of multiple solutions, and what exactly a solution document should specify about that.

> > Technially, the provisions described in the framework do seem to allow supporting to operate
> > more than one solution at the same time by use of different NSI for different MNA solution
> > substacks. Reading draft-ietf-mpls-mna-requirements it sounds a bit as if only one solution
> > is assumed to be deployed at one time, but that of course does not make it clear that the
> > framework expects the same limitation too.
> 
> The framework strives to introduce no unnecessary limitations.  Full details of any limitations
> should be called out by the solutions documents.

Given the complete absence of discussion of potentially multiple solutions and their co-existance, it rather seems
as if there is complete desinterest to ever support more than one.

For example, is there ever a reason why two MNA solutions could not co-exist if individual packets
only have one or more sub-stacks of a single solution ? If not, then why not say that at least.

And what reasons would there be why sub-stacks from two solutions could not be in the same packet ?
And what do we buy if we then say that solutions SHOULD NOT have such a limitation ?

Or else, if there is desinterest to think about this, then the framework could say that the
primary goal is to have one solution in a particular deployment only, and anything beyond that
would be icing on the cake.

Without any such discussion it's really hard to understand what the thinking of the active people in the WG is.

> > Aka: please explain explicitly. I do suggest possible text further below.
> > 
> > 3. Framework re. requirements document
> > 
> > 3.1 It is quite unclear to solution designers which of the requirements document requirements
> > are left unchallenged by the framework and which are replaced/superceeded by more detailled
> > reqiurements from this framework document. It would be a lot better IMHO if solution designers
> > would only have to refer to requirements from this framework document but not have to read
> > the requirements document as well.
> 
> 
> The intent is that the framework document does not challenge any of the requirements, but it may
> bound how the solutions may meet those requirements. 
> 
> I’m very sorry that you dislike how the documents are organized, but this is a decision that was
> taken very long ago and seems to work for most members of the working group.

Sorry, "challenged" was the wrong word, i should have probably said "addressed".

Aka: some requirements are addressed directly by the framework. And there is no text in the
framework listing which requirement is addressed by which framework section(s) for example.
And if framework text narrows the solution space of a requirement, that is also not written.

And if the implementer of a solution in his understanding sees a conflict between some text
in the framework and a requirement, what should he then do ? Ignore the framework and follow
how he reads the requirements ? Or ignore the requirements and follows how he interprets the
framework ? Aka: Any requirements that are "addressed" by the framework seem to be requirments
where implementers shuold only follow the framework, right ?

Other requirements may not be addressed by the framerwork at all. Which is fine.

Without this mapping, requirements and whether they're addressed by thre framework,
it's really hard for implementers IMHO.

> > Aka: Add a section to the framework document, inherit all the requirements from the
> > requirements document, delete all the ones that are superceeded by the more refined requirements/
> > definitions from the framewokr document, and then review the remaining ones and see what
> > to do about them - keep unchanged or refine/add-explanations from framework view etc.. pp.
> > 
> > This is ultimately the check if the framework is considered to be complete against
> > the requirements document, and i don't think a reviewer of the framework document should do the
> > first run for this - but rather the authors by doing this cut & check.
> > 
> > Worst case, if something like this is not done, how would developers of solutions address
> > inconsistencies between framework and requirements document ? Or know what to do about
> > requirements assumed to be unaffected by the framework if those are not clear to developers how
> > to translate into a solution ?
> 
> There should be no inconsistencies between the requirements and framework documents. If there
> are, we should repair them now.

Then please first write down this mapping, aka: what you as the authors of the framerwork
think are the requirements addressed by the framework, and by which section/functionality
accordingly.

> > I have comments about specific requiremnts further down in this review.
> > 
> > It would also be good to make the required reading consistent with the requirements
> > document. E.g.: it mentions rfc3031/rfc3032 and then starts to hand-wave about extensions
> > - making it difficult to undersstand which of those extensions migh be relevant. This framework
> > document does only mention rfc3031/rfc3032 in passing but not in the main text.
> 
> There are no references to RFC 3032.  RFC 3031 is referenced in the section on partial processing. 
> Extensions are only mentioned in the abstract and introduction, referring to future network action 
> specifications.

RFC3032 is mentioned in the requirements. My first point was consistency.
Especially because implementers of solutions will certainly have to understand it.

My second point was about starting the doc with something like "Readers and implementers of solutions
are assumed to be familiar with [RFC3031], [RFC3032], ... - these are the specifications for the
aspects of MPLS that this framework relies upon".

> No hands are waved in the document.

How about "en passant" ?
Aka: one has to go through the whole document to figure out what "base architecture"
is that one has to understand.  Better architectural/framework type documents make this clear in the beginning.

> Could you please be more specific?

See above.

> > 3.2 I tried to validate if this document meets the requirement listed in the requirements
> > document, and stumbled across the following points in the requirements document:
> > 
> >   50.  In-stack ancillary data MUST only be inserted in conjunction
> >        with an operation conforming to [RFC3031].
> > 
> >   51.  Post-stack ancillary data MUST only be inserted in conjunction
> >        with an operation conforming to [RFC3031].
> > 
> > I am guessing that this does not apply to the initial construction of the label
> > stack on the ingress LSR ? It would be good to say so explicitly.
> 
> This would seem to be a comment about the requirements document.

Only an understanding question, yes. Because i was trying to understand if/how the fwk matches the
requirements, but i am unclear about the point i raised. 

But i've raised the question in feedback to the req. doc.

> > Also, RFC3031
> > only seems to define "operations" as something derived from LSR state, aka: NHLFE.
> > And network actions for example do not necessarily require NHLFE (but often could be
> > stateless alternatives). But network actions are also referred to as "operation" in the
> > framework draft. Aka: requirement 50/51 can be read as if a network action itself
> > indicated by a Network SubStack can not modify the MPLS Label Stack unless it has
> > NHLFE associated with it.
> 
> Again, this seems like a comment on the requirements document.

Yes, likewise raised against req-doc. Its mostly about the ?overload? of the term operation
which may unintentionally constrain what we can do with MNA...

> >   6.   Solutions MUST NOT require an implementation to support in-stack
> >        ancillary data, unless the implementation chooses to support a
> >        network action that uses in-stack ancillary data.
> > 
> >   7.   Solutions MUST NOT require an implementation to support post-
> >        stack ancillary data, unless the implementation chooses to
> >        support a network action that uses post-stack ancillary data.
> > 
> > These two requirements seem like NOPs, or at least i can not come up with an idea how
> > to violate them. Maybe some example would help.
> 
> Again, this seems like a comment on the requirements document.

Yes, likewise raised against req-doc.

> > 3.3  There are requirements i do not agree with, but i guess those comments would
> > need to go to the requirements do last call, but just in case, consider as comments:
> > 
> >   36.  If there is post-stack ancillary data, there MUST be an
> >        indication of its presence in the label stack.
> 
> Again, this seems like a comment on the requirements document.

Yes, 
> 
> 
> > I think this requirement can be in contradiction to requirement 8, 9:
> > 
> >   8.   The design of any MNA solution MUST minimise the amount of
> >        processing required to parse the label stack at an LSR.
> > 
> >    9.  Solutions MUST minimize any additions to the size of the MPLS label stack.
> 
> 
> 
> Again, this seems like a comment on the requirements document.

Yes.

> 
> > 3.4  This requirement may be mis-worded / unclear:
> > 
> >  46.  Any MNA solution specification MUST describe whether it can
> >       coexist with existing post-stack data mechanisms e.g. control
> >       words and G-ACH, and if so how this coexistence operates.
> 
> 
> Again, this seems like a comment on the requirements document.

Ack.

> > When i called the DetNet control word a part of MPLS in the DetNet WG in IETF 119,
> > Greg Mirsky educated me on the microphone that this has nothing to do with MPLS, aka:
> > the PseudoWire or DetNet control words are not any more MPLS Post Stack Data than IP
> > IP/IPv6 packet header following the MPLS label stack are.
> > 
> > I don't know about G-ACH, but it would certainly be
> > prudent to have a clear understanding if or what actually does constitute proper current
> > (pre MNA) "MPLS post stack data". And doing so in this framework document might be a good
> > place. So far, my review assumes that there really is no pre-existing proper MPLS PSD.
> 
> 
> As of this writing, we have not decided to proceed with any PSD solution.

Right. This was mostly terminology (aka: what is called PSD). Raised with req. doc.

> > 4. It would be nice if the framework could venture to provide some MPLS WG agreed
> > upon estimates of future growth requirements for MNA solutions, or else solutions
> > will come up with either over or underscaled encodings and reviewers can not really
> > vet how good a solution proposal is.
> > 
> > Aka: Total of N actions that need to be encodeable
> >     N1 actions requring no further parameters
> >     N2 actions requiring 8 bit parameters
> >     N3 actions requiring 32 bit parameters
> >     N4 actions requiring 64 bit parameters
> >     worst case substack: M actions, one with 8, one with 32 bit one with 64 bit parameters
> > 
> > Just as an idea, of course i do not know for any of these parameters the right
> > numbers, but without the framework providing any such bounds its not sufficient
> > for
> 
> 
> As you know well, predicting the future is somewhat challenging. We have no basis
> for making any estimates for any bound on how future actions will be employed.

Well, one could look at the superset of action proposals brought up even if not
agreed upon in requirements yet.

Without ANY requiremens against extensibility, i am sure there will be as much
extenibility as we've seen in IPv6 extension header support so far in forwarding
planes - aka: NONE. Hence also the work on better HBH header requirements
(draft-ietf-6man-hbh-processing-16).


> > 12                     MPLS Network Actions Framework
> > 
> > nit:
> > 
> > I always like for abbreviations to be included in the title, so people
> > trying to find the document belonging to the title will find it in e.g.: rfc-index.txt
> > or other places only including the title. Aka suggest to change to:
> > 
> >    MPLS Network Actions (MNA) Framework
> 
> 
> Sure, done.
> 
> 
> > 17   This document specifies an architectural framework for the MPLS
> > 18   Network Actions (MNA) technologies.  MNA technologies are used to
> > 
> > nit:
> > 
> > I would suggest to add
> > 
> > [ RFC-Editor: Please add "MNA - MPLS Network Action" to RFC Abbreviations List ]
> > 
> > so your new TLA is recognized there. Somewhere in the document where you like it.
> 
> 
> As this has far-reaching and long-lasting effects, I hesitate to proceed with this.
> If MNA turns out to be unpopular, we will have cluttered our namespace indefinitely.
> We can always make this addition later if MNA proves to be popular.

Just for the fun of it:

SR         - sender report (SR) 

No mention of Segment Routing ;-)

Aka: There will always be historical clutter. No reason for not mentioning your abbrebviation IMHO.

If you want to earn a star in the abbreviation list. That's when you need to show
wide adoption/understanding.

But just a suggestion.

> > 19   indicate actions for Label Switched Paths (LSPs) and/or MPLS packets
> > 20   and to transfer data needed for these actions.
> > 
> > nit: 
> > 
> > A bit more explanatory introduction of what "action" on the LSP or packet means would be
> > nice. Maybe something like:
> > 
> > ... to indicate actions that impact the forwarding or other processing (such as monitoring)
> > of the packet along the Label Switched Path (LSP) of the packet or that impact the packet content 
> > itself.
> 
> 
> I think that this is largely addressed in the introduction, where there is a bit more description about actions. 

You're judging from a subject expert side. This is not always helpful
to create text to explain things well to novice readers. My suggestion stands.

> > 109   The document provides the foundation for the development of a common
> > 110   set of network actions and information elements supporting additional
> > 111   operational models and capabilities of MPLS networks.  Some of these
> > 112   actions are defined in existing MPLS specifications, while others
> > 113   require extensions to existing specifications to meet the
> > 114   requirements found in [I-D.ietf-mpls-mna-requirements].
> > 
> > nit:
> > 
> > I am somewhat confused about the above text. Let me ask whether the following text
> > is a correct (if not better) rewrite of the goal of this document and its relationship to
> > requirements and solutions:
> > 
> > ...
> > This document provides the architectural framework for the development of
> > complementary and/or competing solutions for operational models and capabilities of MPLS
> > which are called "MPLS Network Actions" (MNA).
> 
> 
> Several people appear to have an allergy to the word “architecture”, so we’ve tried to avoid it,
> despite that being exactly what we’re trying to do. It is not our intent to produce competing 
> solutions. While it was anticipated that we might have competing solutions during our initial work,
> we were able to merge proposals so there is no remaining competition.

a) Sure, s/architecture// in the proposed text.

b) Rephrase your explanation so it can go into the text. Aka: initial thinking is that
a single solution target may suffice, but longer term do not preclude alternative or
complementary solutions.

> > MNA solutions derived from this framework are intended to support all or a subset
> > of the requirements found in [I-D.ietf-mpls-mna-requirements].
> 
> We are not subsetting requirements.

"address". Another part of the requirements is not explicitly addressed by the framework
but need to be inherited by the solutions directly from the requirements doc.

> > In additions, MNA solutions
> > may also support actions for which prior solutions have been defined through existing MPLS
> > specification, for example to make it easier to combine existing and new actions in the
> > same MPLS packet.
> 
> 
> Yes, we are trying to subsume some existing functionality (e.g., Entropy Label) because it 
> encoding efficiency. Other actions will require new definitions.

What i wrote was suggested explanatory text to make these things clear - which i was just
guessing at when reading the fwk draft.

> > 116   Forwarding actions are instructions to MPLS routers to apply
> > 117   additional actions when forwarding a packet. 
> > 
> > nit:
> > 
> > Given how the term "forwarding action" is used in a lot more IP/IPv6 RFCs than MPLS
> > RFCs, it is somewhat irritating to define the unqualified "forwarding action" to be
> > something that means MPLS.
> 
> 
> I’m happy to add an MPLS qualifier here. I think that clarifies the entire two paragraphs.
> 
> 
> > Suggest to remove MPLS and then add a sentence scoping it to MPLS, e.g.:
> > 
> > For MPLS forwarding, this term was introduced via rfc6639 and rfc8402. In this document,
> > all use of forwarding actions refers to MPLS forwarding actions.
> > 
> > 117   These might include
> > 118   load-balancing a packet given its entropy, whether or not to perform
> > 119   fast reroute on a failure, and whether or not a packet has metadata
> > 120   relevant to the forwarding decisions along the path.
> > 
> > nit:
> > Suggest to avoid introducing unnecessary additional terms (forwarding decisions) without
> > wanting to spend the space for a reference or definition.
> > 
> > suggest: s/forwarding decisions/forwarding actions/
> 
> 
> Ok
> 
> 
> > 122   This document generalizes the concept of "forwarding actions" into
> > 123   "network actions" to include any action that an MPLS router is
> > 124   requested to take on the packet.  That includes any forwarding
> > 125   action, but may include other operations (such as security functions,
> > 126   OAM procedures, etc.) that are not directly related to forwarding of
> > 127   the packet.
> > 
> > minor:
> > I am not so happy with a term as broad as "network action". Are failures/recoveries
> > of components and resulting re-routing, change of link-speeds  and the like network
> > actions even before i look into specific traffic ? The term itself makes me think they
> > are, but i think your definition of "network actions" likely not.
> > 
> > Maybe "network traffic actions" may be better. And/or scoping the definition to
> > "operations triggered by network traffic or its absence".
> 
> 
> Changing our primary acronym and abbreviation is a broader topic that should be
> taken up separately. 

You don't have to change the terminology. You just need define what
can trigger those "network actions" and what type of impact an action can have.

E.g.:

- Can a network action only be triggered by a packet ?
- Can a network action only impact the packet that triggers it ?

Two examples where i am not sure if they could be valid MNA "network actions":

A route-change (e.g.: LFA) triggers an MPLS OAM packet.
In this case, no packet is involved triggereing the action.
I am guessing this is not an MNA "network action", but i assume an
MNA network action can only be triggered by a packet with an MNA substack
and/or MNA PSD. But that's not written explicitly in the framework.

A packet with a "start/stop to encrypt for this label" trigger actions on packets
(for example by changing NHLFE state) that follow this packet but do NOT
have an MNA substack.
I am unclear if such NHLFE state impacting actions are valid MNA "network actions".

And i don't want answers in email. I wan to understand the answers from reading
the draft.

> > 129   MNA technologies may use the Label, Traffic Class (TC), and Time to
> > 130   Live (TTL) fields in an MPLS LSE for an alternative purpose.
> > 
> > nit:
> > LSE - expand before first use (No (*) in RFC editor abbreviation list).
> 
> 
> Fixed.
> 
> > Please browse through all abbreviations and check for expansion on first use, i may
> > not have been complete.
> 
> 
> Done. Found a few.
> 
> 
> > nit:
> > "alternative purpose" took a while to click for me what it is supposed to meant. I first thought
> > that the fields themselves keep their semantic, but that the action itself would do
> > something new with them, such as routing based on TC.
> 
> 
> Reworded.
> 
> 
> > 142   Although this is an Informative document, these conventions are
> > 143   applied to achieve clarity in the requirements that are presented.
> > 
> > minor:
> > 
> > Why then is this document only informational ? An added justification here in
> > the text would be helpfull. "This document is informational only even though it
> > raises normative language requirements against work referring to it because ...."
> 
> 
> It is informational because it does not mandate anything for implementations. The
> solution documents provide those mandates.

Hah! That's a new explanation to me. "Requirements against other documents can not be
normative" ... ??? We now even have normative (abstract) API RFCs, so IMHO this
framework is perfect for standards track.

> > 145 1.2.  Terminology
> > 
> > 147 1.2.1.  Normative Definitions
> > 
> > 149   This document adopts the definitions of the following terms and
> > 150   abbreviations from [I-D.ietf-mpls-mna-requirements] as normative:
> > 151   "Network Action", "Network Action Indication (NAI)", "Ancillary Data
> > 152   (AD)", and "Scope".
> > 
> > nit:
> > 
> > Would be esier for readers to read the definitions here given how few those are.
> 
> 
> The concern is that as they morph, we are challenged to remain synchronized.
> We prefer to have one source of truth.
> 
> 
> > minor:
> > 
> > Not happy about term "as normative" when the document is informational.
> 
> 
> Understood, but we don’t see a better way.
> 
> 
> > 154   In addition, this document also defines the following terms:
> > 
> > 156   *  Network Action Sub-Stack (NAS): A set of related, contiguous Label
> > 
> > nit:
> > 
> > Abbreviation would be easier to remember if it was NASS
> 
> 
> NAS was the acronym chosen by the WG. There is also some concern about accidental obscenity.
> 
> 
> > Also less obvioust TLA overload. How many Terrabytes does your NAS have ?
> 
> 
> TLA overload is always an issue.  The context doesn’t overlap, so this was not a concern.
> 
> 
> > 175    +--------------+---------------+----------------------------------+
> > 176    | ECMP         | Equal Cost    |                                  |
> > 177    |              | Multipath     |                                  |
> > 
> > nit:
> > 
> > Maybe RFC3272 as reference.
> 
> 
> Sure
> 
> 
> > nit:
> > 
> > Misses NSI.
> 
> 
> Added
> 
> 
> > 217 2.  Structure
> > 
> > nit: It would be lovely to include an ASCII graphics of a NAS. And highlight
> > accordingly that the S-bit is maintained to make this more obvious.
> 
> 
> This can be found in draft-ietf-mpls-mna-hdr.

Not an abstract one, only the ones for the solution. Aka: would be lovely
to have the generic/abstract header as ascii showing just the textually defined
elements from the framework.

> 
> > 219   An MNA solution specifies one or more actions to apply to an MPLS
> > 
> > nit: network actions ? (also any further occurrance below)
> 
> 
> Ok
> 
> 
> > mayor:
> > 
> > This is where i as a reader started wondering whether a network or packet is supposed
> > to support only one or multiple MNA solutions. Aka: please clarify here or earlier.
> 
> 
> That’s still up to the WG to decide.  As of right now, a packet may have ISD, PSD, or both.

Issue was not about ISD vs. PSD but just one solution or multiple was
discussed already above. 

> 
> > 220   packet.  These actions and their parameters may be carried in sub-
> > 221   stacks within the MPLS label stack and/or possibly post-stack data.
> > 
> > minor:
> > 
> > The requirements draft sounded as if both ISD and PSD are optional, whereas
> > this sentence sounds as if only PSD was optional in the framework. If this is a conscious
> > choice the discrepancy should be explained somewhere.
> 
> 
> Removed the word ‘possibly’.
> 
> > 
> > nit:
> > 
> > reference for post-stack data would be good, or definition.
> 
> 
> There is no reference as we have not and may not publish anything.  PSD is defined in the
> requirements document.

This was only about a reference for the definition of the term if it exists.
 See my questions about the use of the term in the requirements doc as well,
which refers to PSD as something pre-exist (before requirements document).

Aka: Is an IP header following an MPLS label stack "post-stack data" or not ?
I think here we are really talking about new "MPLS PSD".

But question also raised now as concern against requirements doc.

> 
> > 222   A solution must specify where in the label stack the network actions
> > 223   sub-stacks occur, if and how frequently they should be replicated
> > 224   within the label stack, and how the network action sub-stack and
> > 225   post-stack data are encoded.
> > 
> > mayor:
> > 
> > This is where one wonder about single versus multiple solution PSD in a single packet or network,
> > so again, please make sure the expectation against that are explicitly describe here or earlier.
> 
> 
> Sorry, I don’t understand this point.  Do you have a specific text change that you would like to
> propose?

This was just about me getting completely lost about whether or not a packet can
have sub-stacks from multiple solutions and if that could ever work.

Maybe given what you said so far here, it would be best to write in the start of the
doc that the framework is not necessarily sufficient to fully consider mixing sub-stacks
from more than one solution because that is not seen as a well-defined requirement.

> > 227   A network action sub-stack contains:
> > 
> > 229   *  Network Action Sub-Stack Indicator: The first LSE in the NAS
> > 230      contains a label with special semantics, called the MNA label,
> > 231      that is used to indicate the start of a network action sub-stack.
> > 
> > nit:
> > 
> > Further up in the document you introduce this LSE with the abbreviation NSI.
> > Here you do not use the term NSI but introduce the redundant term MNA label that you
> > then re-use i think 3 times in the document.
> 
> 
> The NSI contains the MNA label but is not synonymous.  The MNA label is 20 bits, 
> the NSI is a 32 bit LSE.

Ok, got it. Still confusing. The terms "NSI LSE" and "NSI Label" would be less
redundant IMHO.

> > nit:
> > 
> > Would be helpful to readers to add the following comment to unconfuse whether
> > special semantics is just another term for special purpose. Suggested text:
> > 
> > possible relationship between special semantics and Special Purpose Labels (SPL)
> > is discussed in section 3.1).
> 
> 
> Reworded to say ’special purpose’ instead of ’special semantics’.
> 
> 
> > 233   *  Network Action Indicators: Optionally, a set of indicators that
> > 234      describes the set of network actions.  If the set of indicators is
> > 235      not in the sub-stack, a solution could encode them in post-stack
> > 236      data.  A network action is said to be present if there is an
> > 237      indicator in the packet that invokes the action.
> > 
> > minor:
> > 
> > This text leaves me to wonder if such an action indicator if encoded
> > ISD would have to be one or more additional LSE, or if it could be encoded
> > also as part of the NASS Indicator LSE. Maybe that question is answered later,
> > but here it leaves the reader think of the text being ambiguous/incomplete.
> 
> 
> The text is intentionally ambiguous. The specifics of how indicators are encoded
> is up to the solution.
> 
> 
> > 239   *  In-Stack Data: A set of zero or more LSEs that carry ancillary
> > 240      data for the network actions that are present.  Network action
> > 241      indicators are not considered ancillary data.
> > 
> > nit: Introduce on first use term ISD here. Also check capitalization of term
> > (inconsistent capitalization of Data across document).
> 
> 
> Ok
> 
> 
> > nit:
> > 
> > previously you talked about "parameters" for network actions, now you talk
> > about "ancillary data". So now i wonder what the parameters are and how they
> > differ from the ancillary data. Eliminate redundant term if it's just that.
> 
> 
> Fixed
> 
> 
> > 243   Each network action present in the network action sub-stack may have
> > 244   zero or more LSEs of in-stack data.  The ordering of the in-stack
> > 245   data LSEs corresponds to the ordering of the network action
> > 246   indicators.  The encoding of the in-stack data, if any, for a network
> > 247   action must be specified in the document that defines the network
> > 248   action.
> > 
> > minor:
> > 
> > This (ordering) sounds as if each LSE can only belong to one action, but
> > it would certainly be useful to share LSE across actions. For example a solution
> > could define multiple actions but also a common authorization ticket LSE applying
> > to the other actions. Aka: There may be a more complex relationship between
> > multiple actions and their parameters in a single solution than a 1:1 mapping
> > between action and in-stack-data for them.
> 
> 
> Fair enough. The way that I would describe this is with an NAI indicating the 
> common authorization ticket and then additional NAI indicating any operations
> to be performed with the ticket. Thus, you still end up with ancillary data that is
> specific to an NAI, and then NAI may interact.

Maybe add something like "in-stack data may be shared by multiple actions".

> > 250   Certain network actions may also specify that data is carried after
> > 251   the label stack.  This is called post-stack data.  The encoding of
> > 252   the post-stack data, if any, for a network action must be specified
> > 253   in the document that defines the network action.  If multiple network
> > 254   actions are present and have post-stack data, the ordering of their
> > 255   post-stack data corresponds to the ordering of the network action
> > 256   indicators.
> > 
> > minor:
> > 
> > Same concern as last comment.
> 
> 
> Same answer.
> 
> 
> > 258   A solution must specify the order that network actions are to be
> > 259   applied to the packet.
> > 
> > minor:
> > 
> > Usually in MPLS i would expect that stack order is execution order. It would
> > be nice to include a simple reasoning (or even example) how MNA becomes better by that
> > not being implied. Yes, i understand now, order of bits in an encoding for example,
> > but on cold read that's not immediately obviously, so would be could to elaborate.
> 
> 
> Flag NAI have no stack order, thus its wholly up to the solution to specify the semantics.

Sure. I was hoping you'd reprase that and add as explanation, e.g. after the sentence
This is necessary, because Flag NAI encoding order does not imply action execution order.

> > 261 2.1.  Scopes
> > 
> > 263   A network action may need to be processed by every node along the
> > 264   path, or some subset of the nodes along its path.  Some of the scopes
> > 265   that an action may have are:
> > 
> > 267   *  Hop-by-hop (HBH): Every node along the path will perform the
> > 268      action.
> > 
> > 270   *  Ingress-to-Egress (I2E): Only the last node on the path will
> > 271      perform the action.
> > 
> > nit: Why is this called I2E if it only applies to egress ?
> 
> 
> That was the consensus of the WG. There was much debate.
> 
> 
> > minor:
> > 
> > how would you distinguish between actions only on
> >  a) ingress, b) ingress/egress c) egress only
> 
> 
> Ingress is silly.  Just do it. There is no point in signaling to yourself.

"Silly" is not immediately obvious to me.After all, the stack/PSD imposition may
be done by a different (MPLS application) layer, and if i can not trigger something
to happen at the MPLS layer in the sending device, i need to come up with a whole
different mechanism to do it.

> I2E only affects the egress.
> 
> > For example consider DetNet PREOF. ingress needs to create control word with sequence
> > number, egresss does duplicate elimination based on control word sequence number
> > (this was actually defined by PALs for psuedowire, but never used with exactly this
> > expensive action, but only simple switchover). In any case, are these two separate
> > actions, one ingress, one egress ? Or one ingress/egress action..
> > 
> > Explanations to that end would be useful in this framework.
> 
> That’s exactly I2E.

I was thinking of such a header where one element for example a sending timestamp, so that
actually is best done as low-level in the forwarding plane as possible to be accurate.

I'd have to think longer to find potentially a more boring older example.

I think we had a bunch of QoS mechanisms where you also did that QoS on the ingress PE
based on appropriate packet marking from the application, not only on egress PE
(aka: not the aggregate QoS you do on the P LSR).

> > 273   *  Select: Only specific nodes along the path will perform the
> > 274      action.
> > 
> > 276   If a solution supports the select scope, it must describe how it
> > 277   specifies the set of nodes to perform the actions.
> > 
> > minor:
> > 
> > I wonder if i am implying what is supposed to be allowed or if i am overimagining:
> > 
> > Performing of action may be based on a combination of ISD/PSD encoding and/or
> > LSR configuration/state.
> 
> 
> Correct.
> 
> 
> > If any of this freedom is meant to be excluded, please write so. Else it would be
> > reconfirming to include the above.
> 
> 
> Specific suggestions?

As i wrote: for example;

Performing of action may be based on a combination of ISD/PSD encoding and/or LSR configuration/state (NHLFE)"

> > 311   In some solutions, an indication may be provided in the packet or in
> > 312   the action as to how the forwarder should proceed if it does not
> > 313   recognize the action.  Where an action needs to be processed at every
> > 314   hop, it is recommended that care be taken not to construct an LSP
> > 315   that traverses nodes that do not support that action.  It is
> > 316   recognised that in some circumstances it may not be possible to
> > 317   construct an LSP that avoids such nodes, such as when a network is
> > 318   re-converging following a failure or when IPFRR [RFC5714] is taking
> > 319   place.
> > 
> > minor:
> > 
> > IMHO another strong reason to have a framework level definition for how to determine
> > end-of-PSD without having to be able to parse the whole ISD.
> 
> 
> That would be up to a PSD proposal.

As i tried to lay out, _if_ support for multiple different
solutions within the same packet was required, then it would require a similar
cross-solution PSD header as the sub-stack NSI LSE is.

Arguably you are preferring standardization of ISD by defining the concept of 
sub-stack format with NSI LSE, which IMHO does allow to mix different solutions,
but you are not doing the same for PSD. I'd call that ISP bias.

(which is fine, if that's what the WG agrees to, i would just like to see all such
 core decisions appropriately be written down in the framework).

> > 321 2.3.  Signaling
> > 
> > 323   A node that wishes to make use of MNA and apply network actions to a
> > 324   packet must understand the nodes that the packet will transit and
> > 325   whether or not the nodes support MNA and the network actions that are
> > 326   to be invoked.
> > 
> > nit:
> > 
> > Should this not always be "MNA solution" instead of "MNA" in this paragraph ?
> 
> 
> I don’t think so.  Why?

A node that inserts ISD and/or PSD of one specific MNA solution needs to understand that
the node that the packet will transit and whether or not those transit nodes support
that specific MNA solution and the network actions invoked by the inserted MNA data.

again: text to clarify the distinction between potentially different MNA solutions.

> > minor:
> > 
> > Why ?
> 
> Predictable results. Not knowing what nodes will support the requested actions
> will result in unintended consequences.

Shouldn't ISD be ignored by a node not supporting it ?
What happens when it passes a legacy node completely unaware of MNA ?

If you expectation is "we don't know", then it would be good to write that
down, because the rest of the document made me believe it would be ignored,
unless it happens to become Top of Stack.

> > Aka: there are all type of cases where good or baad things happen if this is not
> > the case. For example, i should be able to build an action that happens for each
> > hop, but if the hop doesn't support the action, it ignores it, right ? And thats
> > may be a good thing - or maybe not.
> 
> 
> Or maybe it sends the packet to its worst interface. Or maybe it crashes. Or maybe it encrypts the packet.
> 
> If a node sees something that it doesn’t support the results are undefined.

See above. that expectation is not clear from the document. It's also not clear to
me how this would be the case for the best NSI options. Those options should result in
just an undefined MNA Label to be seen, and i thought that MPLS has some good OAM
when that happens.

> > I would suggest rephrase to something like:
> > 
> > Various networking functions such as (nonwithstanding) diagnostics, operations, orchestration
> > and instantiation of network services may need to understand exactly which nodes support
> > which actions to determine which actions and parameters to invoke on which node, which optimal
> > encoding to use and what results to expect.
> > 
> > 326   to be invoked.  These capabilities are presumed to be signaled by
> > 327   protocols that are out-of-scope for this document and are presumed to
> > 328   have per-network action granularity.  If a solution requires
> > 329   alternate signaling, it must specify so explicitly.
> > 
> > nit:
> > 
> > "alternate signaling" meaning what ? The prior sentence allows all signaling,
> > it only expects some specific granularity, so if anything was alternate it would
> > be degree of granularity ??
> 
> 
> Meaning if you need something strong than whatever is cooked up for describing MNA, then you
> need to say so.  I’m expecting a YANG model with a per-action capability. If you need to
> know that a node supports action Z but only on Tuesdays, there will need to be an alternate
> mechanism to describe that.

Aka:

replace

If a solution requires alternate signaling, it must specify so explicitly.

with

If a solution requires signaling with a different granularity than per-network action,
it must specify so explicitly. Time based actions are an example of such different
granularity.

> > Something like:
> > 
> > The signalings should allow to indicate all differences in support and configuration
> > of the MNA solution that can be of interest to any of the aforementioned functions.
> > 
> > 331 2.3.1.  Readable Label Depth
> > 
> > 333   [RFC8662] introduced the concept of Entropy Readable Label Depth
> > 334   (ERLD).  Readable Label Depth (RLD) is the same concept, but
> > 335   generalized and not specifically associated with the Entropy Label
> > 336   (EL) or MNA.  Readable Label Depth is defined as the number of LSEs,
> > 337   starting from the top of the stack, that a router can read in an
> > 338   incoming MPLS packet with no performance impact.
> > 
> > 340   ERLD is not redundant with respect to RLD because ERLD specifically
> > 341   specifies a value of zero if a system does not support the Entropy
> > 342   Label.  Since a system could reasonably support MNA or other MPLS
> > 343   functions and need to advertise an RLD value but not support the
> > 344   Entropy Label, another advertised value is required.
> > 
> > nit:
> > 
> > I would move all mentioning of ERLD from 333-338 to the beginning of 334, and
> > make that paragraph a "Note:"
> 
> 
> I don’t understand how to do that.

typo in line number, sorry.


Readable Label Depth is defined as the number of LSEs,
starting from the top of the stack, that a router can read in an
incoming MPLS packet with no performance impact.

Note: [RFC8662] introduced the concept of Entropy Readable Label Depth
(ERLD).  Readable Label Depth (RLD) is the same concept, but
generalized and not specifically associated with the Entropy Label
(EL) or MNA. ERLD is not redundant with respect to RLD because ERLD specifically
specifies a value of zero if a system does not support the Entropy
Label.  Since a system could reasonably support MNA or other MPLS
functions and need to advertise an RLD value but not support the
Entropy Label, another advertised value is required.

The whole ERLD is an aside, hence the nit proposal to make that textually clearer.

> > 346   A node that pushes an NAS onto the label stack is responsible for
> > 347   ensuring that all nodes that are expected to process the NAS will
> > 348   have the entire NAS within their RLD.  A node SHOULD use signaling
> > 349   (e.g., [RFC9088], [RFC9089]) to determine this.
> > 
> > minor/Q:
> > 
> > Any requirements against nodes if a NASS can only be read partially  by a node
> > because it's too deep ? "All Hell Breaks Loose" ?
> 
> 
> That behavior is undefined, so the latter.

"If a node creating a packet with a NAS is calculating the depths with which
it arrives at a particular processing node incorrectly, then it may be deeper
in the label stack than that nodes RLD, which may cause unexpected behavior.
A reason for that to happen could be in unexpected behavior on an intervening
router that pushes more LSE on the stack than expected.

Aka: i think it helps the reader who tries to understand MNA to write these
type of details down.

> > 351   Per [RFC8662], a node that does not support EL will advertise a value
> > 352   of zero for its ERLD, so advertising ERLD alone does not suffice in
> > 353   all cases.  A node MAY advertise both ERLD and RLD.
> > 
> > minor:
> > 
> > Is this trying to say something/different from 333-338, or is this just redundant ?
> > If not redundant, please rephrase because i can't see a difference. Else delete.
> 
> 
> The point is the last sentence.

Still unclear exactly what it means.

Do you mean "A node MAY advertise RLD even if it is redundany because of the advertised ERLD" ?
(then say so, else explain).

> > 355   RLD is advertised by an IGP MSD-Type value of (TBA) and MAY be
> > 356   advertised as a Node MSD, Link MSD, or both.
> > 
> > minor:
> > 
> > I guess the allocation of a single MSD-Type means that there can only be a single
> > MSD solution be deployed at the same time, or else they would need to be implemented with
> > the same RLD...
> 
> 
> We are not expecting differing solutions to have different RLDs. That should be a property of a node’s
> hardware.

Thanks.
Would be useful to add:

"NAS encoding differences in different MNA solutions are not expected to impact the nodes RLD".

> > Aka: If we do want to allow specification of more than one MNA solution, then i think
> > there should be a per-MNA-solution RLD and the MNA solution should reques the RLD MSD-Type.
> > 
> > 358   An MNA node MUST use the RLD determined by selecting the first
> > 359   advertised non-zero value from:
> > 
> > 361   *  The RLD advertised for the link.
> > 
> > 363   *  The RLD advertised for the node.
> > 
> > 365   *  The non-zero ERLD for the node.
> > 
> > minor:
> > 
> > Why would we ever trust the ERLD to be an authoritative replacement for RLD ?
> 
> 
> This allows us to advertise only the ERLD if RLD == ERLD.
> 
> If a nodes RLD != ERLD and it’s not advertising RLD, then things are already messed up.
> 
> 
> > An MNA solution requires new forwarding plane firmware (AFAIK quite expensive), but we don't
> > want to expect the vendor to also implement the IGP MSD-Type in the control plane ?
> 
> 
> New forwarding plane firmware is not significantly more expensive than new control plane software.
> It’s just a matter of programmer time. 
> 
> We do want the vendor to implement the RLD MSD-Type.

Then you should upgrade the MAY we discussed above to a SHOULD.

> > That's just unnecessary case permutation complexity and guesswork. Heck, didn't the
> > draft state that the supported actions etc. would need to be implemented in the 
> > protocol (i assume also IGP) ???
> > 
> > Aka: suggest to remove 365. KISS.
> > 
> > 374 3.  Encoding
> > 
> > 376   Several possible ways to encode NAIs have been proposed.  In this
> > 377   section, we enumerate the possibilities and some considerations for
> > 
> > nit:
> > 
> > s/enumerate the possibilities/summarize the options proposed/ ??
> 
> 
> That seems purely synonymous. Benefit?

I just do not see the enumeration of NAI possibilitites. Are two-level
subsections an enumeration ? And does the sentence mean its an enumeration
of possibilities or possibilities and considerations... ?

> > 378   the various alternatives.  In this section, we enumerate the
> > 379   possibilities and some considerations for the various alternatives.
> > 
> > nit:
> > 
> > duplicate text ?!
> 
> 
> Fixed
> 
> 
> > 390   [I-D.ietf-mpls-mna-requirements] requires that a solution not add
> > 391   unnecessary LSEs to the sub-stack (Section 3.1, requirement 9).
> > 392   Accordingly, solutions should also make efficient use of the bits
> > 
> > nit:
> > 
> > "...of the available bits (all except S-bit) within the..." ??
> > Is that correct understanding ? If so, would be nice to write for reconfirmation by
> > readers...
> > 
> 
> 
> The S-bit is not available. Noted.
> 
> 
> > 393   within the sub-stack, as inefficient use of the bits could result in
> > 394   the addition of unnecessary LSEs.
> > 
> > 402 3.1.1.  Existing Base SPL
> > 
> > 404   A solution may reuse an existing Base SPL (bSPL).  If it elects to do
> > 405   so, it must explain how the usage is backward compatible, including
> > 406   in the case where there is ISD.
> > 
> > minor:
> > 
> > Are NASS considered to be ISD (or at least part of the NASS) ?
> 
> 
> To be very precise: ISD are just the ancillary data.  NAS contains ISD.

> > If not, then that would be good to define upfront somewhere.
> 
> ISD is defined in section 2. NAS is defined in section 1.2.1.

      In-Stack Data (ISD): A set of zero or more LSEs that carry
      ancillary data for the network actions that are present.  Network
      action indicators are not considered ancillary data.

So, that definition is missing the piece about the repurposed TC/TTL fields then ?

That missing ASCII picture  could maybe show this best...

> > But when ISD also includes
> > some of NASS, but you mean pre-NASS ISD, then maybe
> > 
> > s/ISD/non MNA solution introduced ISD/
> > 
> > (or legacy ISD, or whatever seems most accurately explanatory).
> > 
> > 408   If an existing inactive bSPL is selected and its usage would not be
> > 409   backward compatible, then it must first be retired per [RFC7274] and
> > 410   then reallocated.
> > 
> > 412 3.1.2.  New Base SPL
> > 
> > 414   A solution may select a new bSPL.
> > 
> > suggestion:
> > 
> > I would just ask for allocation of e.g.: 3 bSPL as configurable. Not really that
> > difficult to match consistent configuration of bSPL semantic through control plane
> > between adjacent SLR and not sending MPLS pckets to a neighbor with inconsistent config.
> > That way a network could use up to three new bSPL based specs, MNA or other new semantics
> > in parallel, as long as it's ok. to expect full hop-by-hop configuration consistency. 
> > Just as one option.
> > 
> > But no idea how important partial MNA deployments are. But it would be good to explore exit
> > strategies from shrinking bSPL space other than increasing label stack size.
> 
> 
> I’m not understanding your suggestion.

Network wide configurable bSPL. BUt primarily makes sense when we predict more
MNA solutions than we would want to allocate bSPL for. On the other hand,
i would consider that any second MNA solution should require that its bSPL
(assuming it uses bSPL) is configureable on nodes, so that as soon as then a third
one comes around, we wouldn't need to allocate a third bSPL unless more than one
MNA solution is needed.

> Part of the motivation for MNA is the observation that we keep burning bSPL for various functions.
> It’s pretty clear that we need a better extension mechanism than one bSPL for each function.
> MNA allows us to do everything with one bSPL.
^^^^^

An MNA solution ;-)

> > 421 3.1.4.  User-Defined Label
> > 
> > 423   A solution may allow the network operator to define the label that
> > 424   indicates the network action sub-stack.  This creates management
> > 425   overhead for the network operator to coordinate the use of this label
> > 426   across all nodes on the path using management or signaling protocols.
> > 427   The user-defined label could be network-wide or LSP-specific.  If a
> > 428   solution elects to use a user-defined label, the solution should
> > 429   justify this overhead.
> > 
> > minor:
> > 
> > isn't the problem rather that it may be unclear whether the same label
> > could be configured across different vendors ? If there is sufficient understanding
> > that this can always be possible, it would be good to point to that. My understanding
> > was that the whole label->SID mapping was because there is no consistent use of MPLS
> > label space across vendors.
> > 
> > minor:
> > 
> > If there is no multi-vendor "finding common label" issue, then i think the
> > rest is not really a problem. I would simply add a requirement for an IETF standards based
> > YANG model for the configuration of the user-defined label before such an MNA solution
> > can become standard.
> 
> This is a non-problem as we’ve chosen the bSPL path.
> 
> 
> > 431 3.2.  TC and TTL
> > 
> > 433   In the first LSE of the network action sub-stack, only the 20 bits of
> > 434   Label Value and the Bottom of Stack bit are significant for MNA
> > 435   purposes; the TC field (3 bits) and the TTL (8 bits) are not used.
> > 
> > nit:
> > 
> > The way its written i think it's logically a bit self contradictory. If you would
> > assign TC/TTL to something in the MNA encoding then it would become "significant to
> > MNA purposes". I think instead of "significant for MNA purposes" it should be "used
> > for the NAI”.
> 
> 
> Fair.
> 
> 
> > 492 3.3.2.  Length Field
> > 
> > 494   A solution may opt to have a fixed size length field at a fixed
> > 495   location within the NAS.  The fixed size of the length field may not
> > 496   be large enough to support all possible NAS contents.  This approach
> > 497   may be more efficient if the NAS is longer but not longer than can be
> > 498   described by the length field.
> > 
> > 500   Advice from hardware designers advocates a length field as this
> > 501   minimizes branching in the logic.
> > 
> > minor:
> > 
> > Does that not depend on encoding ? and FPE ?
> > 
> > Aka: If i have action1, action1-parameter, action2, axction2-parameter, ... then
> > continuation bit might be easier to parse this, but if i have 
> > action-set, action1-parameter, action2-parameter, then a length field might be better.
> > Aka: make sure you're not providing one-sided guidance.
> 
> 
> The only guidance we got was from one source.

"_Limited_ advice from hardware ..." ? ;-) (any appropriate wording to not overinterpret the importance
of the sentence).
> 
> Folks with a more general mechanism didn’t seem to care one way or the other.
> 
> 
> > 503 3.4.  Encoding of Scopes
> > 
> > 505   A solution may choose to explicitly encode the scope of the actions
> > 506   contained in a network action sub-stack.  A solution may also choose
> > 507   to have the scope encoded implicitly, based on the actions present in
> > 508   the network action sub-stack.  This choice may have performance
> > 
> > nit:
> > 
> > So lets say i have one type of action called FOO, but i do create two actions out of it,
> > one would be "FOO-on-every-LSR" and the other "FOO-only-on-explicitly-configured-LSR-for-FOO".
> > Which of the two described options would that be ? In other words: i am not sure i do
> > understand the disctinction completely.
> 
> 
> You can either say:
> 
> HBH: Actions A, B, C
> 
> or 
> 
> Action A (HBH), Action B (HBH), Action C (HBH)
> 
> Either encoding is acceptable.

That would be a good example to include into the text to make sure the definition of the
encoding is understood by everybody.

> > 508   the network action sub-stack.  This choice may have performance
> > 509   implications as an implementation might have to parse the network
> > 510   actions that are present in a network action sub-stack only to
> > 511   discover that there are no actions for it to perform.
> > 
> > nit:
> > 
> > Without an example, it is not clear to me what the most obvious encoding option based
> > choice would be to minimize the need to parse network actions unnecessarily. Aka:
> > if you can, please insert such an example.
> 
> 
> Ok, added.

| For example, suppose that an NAS is embedded in a label stack at a	
| depth of 6 LSEs.  Further, suppose that the NAS contains 3 actions.	
| Each of those actions has Select scope and thus is not applicable at	
| the current node.

I do not understand this example. How is the depth relevant. Are you trying to
imply a 6 hop strict segment path ? Is there no
lookahead ? And why does the fact that the actions have select scope imply
they are not applicable at the current node without also describing the set
of nodes that are supposed to perform the action ?

> > 513   Solutions need to consider the order of scoped NAIs and their
> > 514   associated AD within individual sub-stacks and the order of per-scope
> > 515   sub-stacks so that network actions and the AD can be most readily
> > 516   found and not need be processed by nodes that are not required to
> > 517   handle those actions.
> > 
> > nit:
> > 
> > IMHO, 513-517 would make a better start than conclusion of this section.
> 
> In general, IETF denizens seem to prefer discussing things in a bottom up fashion.
>
> > 559 3.6.  Encoding of Post-Stack Data
> > 
> > 561   A solution may carry some NAI and AD as PSD.  For ease of parsing,
> > 562   all AD should be co-located with its NAI.
> > 
> > minor:
> > 
> > If i am not mistaken, today, the only component that is considered to be part of
> > an "MPLS header" is the "MPLS label stack". E.g.: a pseudo-wire header including it's
> > control word is considered to be it's own header. And in result, one has never needed
> > or wanted to use the term "MPLS header", but could always refer to "MPLS label stack".
> > 
> > With the introduction of (even optional) PSD, this changes. And in result i think
> > this document should at an appropriate place (maybe earlier than here) introduce
> > the term "MPLS header" (or any other/better term) to refer to the combination
> > of MPLS label stack plus PSD. Which it effectively refers to in places, but only calls
> > it MPLS label stack.
> 
> 
> We are trying to avoid defining unnecessary terms.  It is not clear that a PSD solution
> would appear immediately after the label stack, so it’s not even clear that there will
> be a ‘header’.

Huh ? What could ever be between label stack and PSD ?

Even if there was such a think, i find it irritating to not have a term to refer
to the whole MPLS packet enchilada, MPLS label stack plus PSD. I don't think it is
redundant.

Also note:

| Alternatively, it also
| offers end-to-end encryption of the MPLS payload with no
|  cryptographic integrity protection of the MPLS headers.

You do use the term MPLS header, even though i guess it is undefined so far as you seem
to say above.

> > 564   If there are multiple instances of post-stack data, they should occur
> > 565   in the same order as their relevant network action sub-stacks and
> > 566   then in the same order as their relevant network actions occur within
> > 567   the network action sub-stacks.
> > 
> > mayor: 
> > 
> > I don't think this is sufficient. The framework needs a definition that allows packet
> > parsers to skip across PSD, and it needs a framework definition for how multiple
> > different solutions could share PSD space. Even if there is only one solution
> > supposed to be useable in one packet.
> 
> 
> It is not up to the framework to mandate PSD encodings.
> 
> We are not seeking multiple PSD solutions. At present, we are not even seeking one.

But you are seeking one framework that at least does not want to close the door on PSD,
and you do not have any text in the draft even saying what you say above, hence i can
only propose how to fix up the draft text as it is.

> > This is not different from what the framework
> > does for ISD: The parser can skip across the label stack using the S-Bit, and the
> > different NAS indicator methods defined in this document introduce the ability to
> > mix & match different solution ISD together with the extraction of Readable Label Depth
> > from all LSR to even create label stacks (whether or not this is a requirement is
> > as mentioned elsewhere unclear from the doc).
> > 
> > If skipping beyond PSD with a simple method from the framework is not included, then the
> > whole parsing with PSD would becomes more fragile. Being able to parse beyond PSD must
> > not be dependent on network wide MNA solution configuration but from framework specification
> > (IMHO). And i also do not see a way to mix two solutions without having an equvalent previously
> > known structure like the NAS indicator labels.
> > 
> > I don't think one wants to repeat the use of S-Bit for within PSD though. One of the
> > big benefits of PSD over ISD would be to easier have contiguous parameters of 32 bits or longer
> > without being interrupted by the S-Bits.
> > 
> > I would suggest to consider including something like the following as PSD separators for
> > the framework:
> > 
> >  0                   1                   2                   3
> >  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > |PSDcod | Rsv   |Total PSD len  | Next Hdr      | Next Hdr len  |
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > 
> >  Aka:  PSDcod would be something no 0, 4, 6 to most easily recognize PSD
> >  for ECMP and other purposes (aka: first Nibble).
> > 
> >  Total PSD len would allow to skip with one lookup beyond all PSD blocks for for operations
> >  such as ECMP that would want to do this in the presence of PSD. 
> >  Next Hdr would be a new registry that could include both registration points
> >  for MNA solution PSD (one or more per solution) as well as existing next-headers,
> >  and Next Hdr len would be those headers/PSD (sub-headers) len. 
> > 
> > Just an example and to demonstrate what seem to be the salient info fields needs:
> > First Nibble compatibility, single length field to skip all PSD, and ability to
> > chain different solution PSD. And likely make it fit into 32 bits.
> > 
> > 
> > 600   A PSD solution should specify the contents of the first nibble, the
> > 601   actions to be taken for the value, and the interaction with post-
> > 602   stack data used concurrently by other MPLS applications.
> > 
> > nit:
> > 
> > I would move 600-602 after 585. Then reconsider why the text about BIER, 587-594
> > exists, because it reads confusingly. I think you want to say that a PSD solution
> > that wants to avoid for pre-existing ECMP to occur and/or that does not want
> > to get confused with a Pseudowire header with control word should use a Nibble value other
> > then 0, 4 or 6 - and one example of a mechanism that does already apply this logic
> > is the BIER encapsulation of RFC8296. And leave it with that.
> > 
> > But i am not sure if this BIER encapsulation is the most easy to understand
> > example approach given how it was heavily tweaked around MPLS (making the last
> > LSE of the MPLS label stack the first 32 bit of its own header), buit through its doc
> > evolution started also to claiming and that it is an MPLS and non-MPLS solution and
> > calling the label BIFT-ID instead of showing the label field, and hence making it
> > severely confusing why those Nibble bits are there to the non-expert reader.
> > If there is another, more simpler  example using a desirable
> > the Nibble approach than BIER, maybe replace the text with such an example.
> 
> 
> Thank you, but I think that the current text is clear as it is.

I continue to disagree on that.

> > minor:
> > 
> > I think the PSD section is jumping into details before making a fundamental statement like:
> > 
> > PSD is a new (optional) element of the MPLS header and hence extending the
> > MPLS header beyond the BoS LSE. In result, an MNA specification using PSD needs to
> > specify how current deployed (and layer violating) MPLS functions that inspect and operate on
> > data beyond BoS LSE (heuristically!) are supposed to operate in the presence of the new PSD.
> > 
> > Then you have a better context of bringing up the example of ECMP and may recommend for
> > PSD to inhibit the ECMP function through an appropriate choice of Nibble. But likewise,
> > if these layer violating existing functions are considered valuable, then the PSD solution
> > should also include an explanation how to parse to the end of the MPLS header, aka:
> > beyond the end of PSD. And/or specify a mechanism for a PSD encoding to allow/prohibit for that
> > to happen (because it may be undesired by the desired packet processing).
> 
> 
> Again, this is up to a PSD solution to propose.

No. The type of text proposed/requested by these two paragraphs is exactly along the lines of requirements
in the framework about what an MNA solution needs to describe about it's ISD encoding. 

> > 604 4.  Semantics
> > 
> > 606   For MNA to be consistent across implementations and predictable in
> > 607   operational environments, its semantics need to be entirely
> > 608   predictable.  An MNA solution MUST specify a deterministic order for
> > 609   processing each of the Network Actions in a packet.  Each network
> > 610   action must specify how it interacts with all other previously
> > 611   defined network actions.  Private network actions MUST be included in
> > 612   the ordering of network actions, but the interactions of private
> > 613   actions with other actions are outside of the scope of this document.
> > 
> > minor:
> > 
> > First and only use of "Private (network action)". Aka: explain a bit of what you think
> > a Private network action is (such as some different type of specification ?), or else
> > remove the last sentence.
> 
> 
> Added a definition.
> 
> 
> > 646 6.  Management Considerations
> > 
> > 648   Network operators will need to be cognizant of which network actions
> > 649   are supported by which nodes and will need to ensure that this is
> > 650   signaled.  Some solutions may require network-wide configuration to
> > 651   synchronize the use of the labels that indicate the start of an NAS.
> > 652   Solution documents must make clear what management considerations
> > 653   apply to the solutions they are describing.  Solutions documents must
> > 654   describe mechanisms for performing network diagnostics in the
> > 655   presence of MNAs.
> > 
> > nit:
> > 
> > Maybe call this "operational considerations”.
> 
> 
> Why?

You are describing what network operators need to do. If you have a reference
that would make you feel safe that everything you describe/ask-for is well
defined to be Management, then you're fine. I just think it is less risky
to have the title match the content ("Network operatos will need ...").

> > minor:
> > 
> > I would love to see a requirement that all MNA solution must have a YANG specification
> > of all the parameters required to build MNA label stacks through them. Aka: including
> > the use of labels that indicate the start of a NAS as well as any other parameters,
> > including (but not limited to) as network actions supported by nodes supporting (a subset of)
> > the MNA solution.
> 
> 
> This is not a requirement that the WG agreed to.

Was it even brought up as a candidate requirement ?

> > I also think that would be a good replacement for 648-651. I don't think we should
> > be happy with just "CLI" network wide configuration, and we should also not end up
> > in a hodgepodge of protocol solutions for signaling. One solution promoting to only
> > use e.g.: IGP, the other only YANG. Aka: make YANG mandatory to specify and let other
> > signaling options be subject to downstream decisions.
> 
> 
> Nothing requires CLI configuration.

"network-wide configuration" is just a woefully underspecified term, and i just choose
to pick the interpretation "CLI", unless the draft text makes it clearer that that
is not the correct interpretation.

> > 657 7.  Security Considerations
> > 
> > minor:
> > 
> > It would be good to go through the security related requirements in the requirements
> > document and determine how to explicitly refer to them and address them in this document.
> > 
> > E.g.:
> > 
> >   12.  The design of any network action solution MUST NOT expose
> >        information that is not already exposed to the PE to the LSRs.
> >        It MUST NOT expose any information that a user of any service
> >        using the LSP considers confidential [RFC6973] [RFC3552] .
> > 
> >   13.  Network action solutions MUST NOT expose information considered
> >        confidential to the user of the MPLS-based service [RFC6973] to
> >        the LSRs.
> > 
> > Could be discussed in the security section as follows:
> > 
> > Network Actions introduce the generic challenge of moving potentially sensitive
> > information from NHLFE to label stacks. In an MPLS network without hop-by-hop
> > encryption, the sensitive information caried in sub stacks or PSD is likely
> > a lot easier to exploit than whats only in LSR NHLFE state (and assumingly
> > signalled via encrypted control plane). Deployments requring such higher confidentiality
> > of MPLS packets because of MNA need to accordingly put stronger emphasis on per-hop
> > encryption of MPLS packets (or other forms of confidentiality).
> > pointing to IMHO a higher need for hop-by-hop encryption if/when such sensitive
> 
> 
> That is not an approach that the WG has accepted.

So then IMHO the framework's security considerations are incomplete because
they do not describe how the framework directly addresses the security relevant
MNA requirements.

> 
> >   14.  Solution specifications MUST document any new security
> >        considerations that they introduce.
> > 
> > As mentionined initially, inherit into this framework text.
> 
> The framework is not meant to be a repeat of the requirements.

No, but it need to describe if/how it addresses individual requirements IMHO.

> > 681   Particular care would be needed when introducing any end-to-end
> > 682   security mechanism to allow an in-stack MNA solution that needed to
> > 683   employ on-path modification of the MNA data, or where post-stack MNA
> > 684   data needed to be examined on-path.
> > 
> > minor:
> > 
> > This makes it sound as if this is a new MNA challenge, whereas AFAIK the
> > same problem exists already since day 1 for the MPLS label stack. So, unless i am
> > not mistaken, it would help to clarify that MNA simply shares this challenge
> > with MPLS itself.
> 
> 
> MNA has changed the situation slightly by allowing mutable data in the label stack.

Please include that explanation:

MNA introduces the option of mutable data in the label stack, hence particular care ...

> > 686   A cornerstone of MPLS security is to protect the network from
> > 687   processing MPLS labels originated outside the network.
> > 
> > 689   Operators have considerable experience in excluding raw MPLS-encoded
> > 
> > nit:
> > 
> > What is a raw MPLS-encoded packet ? Either just say MPLS packet or define the term/difference.
> 
> 
> Removed the word ‘raw’.
> 
> 
> > 701   Within a single well-managed domain, an adjacent domain may be
> > 702   considered to be trusted provided that it is sufficiently shielded
> > 703   from third-party traffic ingress and third-party traffic observation.
> > 704   In such a situation, no new security vulnerabilities are introduced
> > 705   by MNA.
> > 
> > mayor:
> > 
> > But there is no requirement specified so far that such collaborating MPLS domains
> > would need consistent configuration of e.g.: NAS start label, so there may be no
> > malicious attack, but it just wouldn't work. Not to speak of knowing inter-domain
> > thr supported actions. Unless the solution just attempts to MPLS-tunnel through the
> > collaborating MPLS domain, which still wouldn't necessarily work in the presence
> > of axctions with lookahead or PSD unless all that is consistent... 
> > 
> > Maybe put a paragraph to this extend into the management/operational considerations
> > section given how its not about (intentional) attack vectors...
> 
> I’m sorry, I don’t understand this comment.

There is always the question of the scope of "security" (considerations), aka: if
it should only cover malicious attacks or also bringing down the network by stupidity
because of unmanageable operational complexity.

Assuming warning about the latter the later, than the whole point of exchanging all the
necessary information to properly create an MNA labels stack that will parse correctly in a different
(adjacent/trusted) domain will become a lot more prone to errors and limited options than
in a single domain (aka: no shared IGP to disseminate information for example). Aka: much higher risk

> > 718   Several mitigations are available to an operator:
> > 
> > 720   a) Reject all incoming packets containing MNA information that do not
> > 721   come from a trusted network.  Note that it may be acceptable to
> > 722   accept and process MNA information from a trusted network.
> > 
> > 724   b) Fully encapsulate the inbound packet in a new additional MPLS
> > 725   label stack such that the forwarder finds a BoS bit imposed by the
> > 726   carrier network and only finds MNA information added by the carrier
> > 727   network.
> > 
> > minor:
> > 
> > "MPLS label stack" here hould be "MPLS header" (as mentioned above) because it could
> > include PSD.
> 
> 
> We are not defining “MPLS header”, so this would be an inappropriate change.

I continue to believe this is a shortcoming.

> > minor:
> > 
> > In fact, one type of PSD could simply be an indication "end of MPLS header" (along
> > the above described First Nibble options). Maybe we would even want to make it a
> > requirement in this document for MNA solutions to include text about how to "fully encapsulate"
> > an MPLS header when usingthat MNA solution. This does not have to be PSD based, but if it
> > is not PSD based, then the MNA solution would have to describe how any of the
> > other options would work with the ECMP and other post-stack legacy behavior.
> > 
> > I'd certainly be in favor of providing a clear MPLS header separation from following
> > data via PSD to potentially avoid running into these legacy compatibility issues.
> 
> 
> That should be in a PSD proposal.

And as mentioned above, the requirements about what a solution with PSD needs
to describe wrt. backward compatibility need to be in the framework in the same way
as there is a lot of good andn important requirements detail about ISD.

> > 729   A mitigation that we reject as unsafe is having the ingress LSR push
> > 730   sufficient additional labels such that any MNA information received
> > 731   in packets entering the network from a third-party network is made
> > 732   inaccessible due to it being below the RLD.  This is unsafe in the
> > 733   presence of an overly conservative RLD value which can result in the
> > 734   third-party MNA information becoming visible to and acted on by an
> > 735   MNA forwarder in the carrier network.
> > 
> > minor:
> > 
> > Not quite sure about the wording here. I guess the point is that the RLD is only a
> > guarantee that the LSR can at least look as deep as RLD, but it is no guarantee that
> > the LSR will not be able to look deeper ?? If so, then say so. Maybe even revisit the
> > initial text about RLD in this spec (337 ff). E.g.: add that RLD is just a minimum guarantee,
> > and that the LSR may be able to look deeper (but just not consistently enough to guarantee it).
> 
> 
> RLD is about performance, not about absolute guarantees.

Makes a lot more sence than what i read from:

> > 732   This is unsafe in the
> > 733   presence of an overly conservative RLD value which can result in the
> > 734   third-party MNA information becoming visible to and acted on by an
> > 735   MNA forwarder in the carrier network.

Maybe:
  This is unsafe, because RLD only indicates a no performance impact depth. Routers may examine and act
  upon NAS deeper than RLD.

If that's not correct then i do not understand your comment.

> > If instead RLD is really meant to be a very accurate depth (min = max), then the above
> > text makes no sense to me. However i full agree that we should never simply concatenate
> > MPLS label stacks between non-trusting domains. Besides it wouldn't work with PSD on
> > the out MPLS header anyhow.
> 
> 
> It cannot be absolute: almost every box can process the full packet by kicking it up to the CPU.
> The performance is just horrible. RLD is what it can do at rate.

Well. The text you wrote "becoming visible" sounded like a binary thing. And that
actually does provide some easier to understand behavior than having RLD only be about
performance. So it might actually be better to consider promising tonot examine stacks
deeper for NAS than RLD.

> Regards,
> T

Thanks a lot Tony. Such an intermediate document (framework, between reqs & specs) is always difficult.
Appreciate the effort put into it

Cheers
    Toerless

-- 
---
tte@cs.fau.de