[RTG-DIR]Re: RtgDir Early review: draft-ietf-mpls-mna-fwk-06.txt

Tony Li <tony.li@tony.li> Mon, 06 May 2024 21:43 UTC

Return-Path: <tony1athome@gmail.com>
X-Original-To: rtg-dir@ietfa.amsl.com
Delivered-To: rtg-dir@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id DBFB0C169404; Mon, 6 May 2024 14:43:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.647
X-Spam-Status: No, score=-1.647 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, HTML_TAG_BALANCE_BODY=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id 2P6KsF9fAiWc; Mon, 6 May 2024 14:43:45 -0700 (PDT)
Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3B996C165518; Mon, 6 May 2024 14:43:45 -0700 (PDT)
Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-6ed3cafd766so1763987b3a.0; Mon, 06 May 2024 14:43:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715031824; x=1715636624; darn=ietf.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:sender:from:to:cc:subject:date:message-id:reply-to; bh=T3Zs9PaWe6Gks0/pE9Tw3DVTvroGuXPXXl2wGxCDyU8=; b=N4AMcCOyhwGzvs0vn4dUTZL66qgWcP/9Vhd9I+1hLmmZec2ejxgqb8ZdTPcaXWNdPL Gp5ZzU8lIhwpnhIk6a+JCKcll7RHA/dQCMDXo7incx6xS2VParZL1skZ+iXyqsKhCMYN e4Fg/bu4MgEJdOrHkjZYeBnatVCZn7gVUBTHQwXSd5NiuVgl7omj6eEDT+Ly7y0CnCOo Wir9xjTiSowGq0ru7ORY8azGoFXjSmzEHlMtWVTqzA4WFjtKSjQ2NSbEL/BAoYq72Mhs HEB+oZzaj6BugINBPlGk1Fu8QsZNO8mm7HKN6f+RQPm4clBu+aXtQashskhrAa82wzBN j4MA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715031824; x=1715636624; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=T3Zs9PaWe6Gks0/pE9Tw3DVTvroGuXPXXl2wGxCDyU8=; b=V0OfAE/TpT76m9QYJ0J24w1+VlHCHh/rWRoTqw58OOczdJLYQKEWxkyo33t7MMKcGf /jK0JhSWQx1ck8JC/iAwBEIA6IlC/T8U5s5w3Y2YirkKmwuQVw8D6rxlmLoreNunPKxI tdnFj6tpVVanVKeqFEptLwURRC7CMpLRj5hqu3jKZMOFcCyBCaM0SRgmWxodDKnrucSV ZAr7JF6USzoJT8/ReXggbIO8pvM4EYS7RpoIR8K1ieGpGTVKwGMzUAOYDvkcGs7mekAH lBZMUs0JYgpoZuVd1SQJ7fPALgELfuc5RMvND9Jy9rpedgaIVhDFNeBiVDV2r401fmnR j/FA==
X-Forwarded-Encrypted: i=1; AJvYcCXE/pHJakDA/ltJDOwVBeziAZSGD1yQM0G1v8er0pEbpzuwJ5qEiP8otL+Md1HRbW7TCensr75zfWn9VSuJ1zG6x4penzrw9qO17f5659+F2UENvUC0mUf1t/q5afKiV8c68I0sw0aaWzkki5g9HTF5oQ==
X-Gm-Message-State: AOJu0YzPzzNwTKoRcFFh+xUO6sf4dK2/mXdWtpKTTLCABRmI+y/lQFsS IWXqKqPKVBmyqYmdioRvdODQ3jw4RQ2yErV/xYwt5Gig5OmWFcxE
X-Google-Smtp-Source: AGHT+IHmTGunEYKn6YrxtMjagD3T8A3OkuD5+JyQzW6MUsbHg7lLVZmm/uvoNH30bFcNPAKrx0FZMw==
X-Received: by 2002:a05:6a00:3cc6:b0:6e8:3d5b:f3b1 with SMTP id ln6-20020a056a003cc600b006e83d5bf3b1mr11829814pfb.22.1715031823781; Mon, 06 May 2024 14:43:43 -0700 (PDT)
Received: from smtpclient.apple (c-73-93-167-4.hsd1.ca.comcast.net. []) by smtp.gmail.com with ESMTPSA id p40-20020a056a0026e800b006f44d0df062sm6328248pfw.125.2024. (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 May 2024 14:43:42 -0700 (PDT)
Sender: Tony Li <tony1athome@gmail.com>
From: Tony Li <tony.li@tony.li>
Message-Id: <DD5F9477-97EB-4ABC-925F-341F08477134@tony.li>
Content-Type: multipart/alternative; boundary="Apple-Mail=_1E05B9EA-1C04-4127-924D-0A0E41DE0C00"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\))
Date: Mon, 06 May 2024 14:43:31 -0700
In-Reply-To: <ZjUWhbcVAK03Fir2@faui48e.informatik.uni-erlangen.de>
To: Toerless Eckert <tte@cs.fau.de>
References: <Zg3kupJ9jovwpzbY@faui48e.informatik.uni-erlangen.de> <1E0C0F9D-87A1-4790-925F-DFE1EB480983@tony.li> <ZjUWhbcVAK03Fir2@faui48e.informatik.uni-erlangen.de>
X-Mailer: Apple Mail (2.3774.500.171.1.1)
X-MailFrom: tony1athome@gmail.com
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-rtg-dir.ietf.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
CC: mpls-chairs <mpls-chairs@ietf.org>, draft-ietf-mpls-mna-fwk@ietf.org, Routing Directorate <rtg-dir@ietf.org>, mpls <mpls@ietf.org>
X-Mailman-Version: 3.3.9rc4
Precedence: list
Subject: [RTG-DIR]Re: RtgDir Early review: draft-ietf-mpls-mna-fwk-06.txt
List-Id: Routing Area Directorate <rtg-dir.ietf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/rtg-dir/KpUCyBhdVgn7nWG8j_Y6gZj5xSw>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rtg-dir>
List-Help: <mailto:rtg-dir-request@ietf.org?subject=help>
List-Owner: <mailto:rtg-dir-owner@ietf.org>
List-Post: <mailto:rtg-dir@ietf.org>
List-Subscribe: <mailto:rtg-dir-join@ietf.org>
List-Unsubscribe: <mailto:rtg-dir-leave@ietf.org>

[WG chair hat: off]

Hi Toerless,

Thank you for your continued conversation.  Please see inline. I will upload a new version of the document shortly.

>>> 1.2 I recently managed to observe a meeting in which third parties who attempted to implement proposed
>>> solutions where also unclear about core aspects, such as what degree of lookahead into the stack
>>> for sub-stacks is assumed to be required, and i can not find this explained in this framework
>>> (nor the requirements document).
>> Please recall that the framework is not the only document necessary for constructing an implementation.
>> The specific solution document, in this case draft-ietf-mpls-mna-hdr, is also necessary. While RLD is
>> discussed here in the framework, details about RLD are up to the solution document.
> Sorry, i think you misunderstood me, i probably did not describe the problem correctly.
> I am just missing any explanation where in the label stack a NAS should be located so that it will
> be acted upon. Is it exactly the same lookahead rules as for ECMP ?

Almost. The folks that defined the entropy label defined the ERLD as the amount that one should lookahead for
an entropy label.  Unfortunately, they made ERLD specific to the entropy label, so we are simply generalizing
that work: RLD is the maximum amount of lookahead that a given node will do on the stack, so that
needs to be accounted for when composing the original stack.

> Also in the presence of multiple NAS ?
> (aka: only need to find the first NAS ?)

The framework leaves this open to the solutions.  

If you look to draft-jags-mpls-mna-hdr (section 5.3), it specifies one possible encoding for multiple NAS.
To summarize that solution: an implementation should scan up to RLD depth, looking for the first HBH NAS,
a Select NAS after the first label, and an I2E NAS, but only after the first label and at the bottom of stack.

>>> From my little experience i gather that some pre-existing special label functionality does
>>> require more lookahead to find them in the label stack than being directly below the TOS LSE,
>>> and i don't think this option is written into rfc3031, which is the only relevant reference
>>> mentioned here. So i think some text about the expected supported methods to find a NAS
>>> would be welcome text to make readers better understand expectations against NAS. Including
>>> the appropriate references if it is based on prior mechanisms.
>> Lookahead processing in the label stack was introduced by the Entropy Label.
> See above. "introduced" does not imply exactly the same way as. And i don't know what other lookahead
> processing there has been in MPLS since then and if they all use exactly the same lookahead mechanism.

I am unaware of any other lookahead requirements in MPLS.

> And if that lookahead mechanism is easily described, then better repeat it here than rely on the
> ECP RFC text for it, given how central it is to the concept of MNA ISD.

I’m very unclear on what your asking for. Could you please be more specific? Section 2.3.1 already describes
the restrictions that the framework wants to have in place for lookahead.  The details are intentionally left to 
the solution.

>> Section 1.2 is the terminology section, so I’m unclear as to exactly where in the document this concern arises.
>> I will add a comment about label stack processing and reference to the Entropy Label.
> I could not locate any new comment in -07, only the reference.  > 

I added paragraph 2 of section 2. I infer that you consider that insufficient. I will add more random blatherings
in the hope that I somehow hit upon someething satisfactory. ;-)

>>> 2. The framework document does not explain unambiguously whether a single packet or a single network
>>> was supposed to support the simultaneous deployment/use of only one or multiple MNA solutions.
>> That is correct.  Again, details are left to solutions documents.  If a particular ISD solution is incompatible 
>> with PSD, that would be up to the solution to document.
> This question was noy only about PSD i thnk, but also about having ISD substacks from multiple
> solutions. Is it permitted to have ISD from two solutions in a single label stack ? Is there
> a reason why this would not work ?

The presumption is that there will be a single ISD solution. Multiple interoperable ISD solutions would be 
extremely complex.

> In any case, it would be good if the fwk doc would state explicitly these issues about co-existance
> of multiple solutions, and what exactly a solution document should specify about that.

I will add text, but this really seems like it should be part of the requirements document.

>>> Technially, the provisions described in the framework do seem to allow supporting to operate
>>> more than one solution at the same time by use of different NSI for different MNA solution
>>> substacks. Reading draft-ietf-mpls-mna-requirements it sounds a bit as if only one solution
>>> is assumed to be deployed at one time, but that of course does not make it clear that the
>>> framework expects the same limitation too.
>> The framework strives to introduce no unnecessary limitations.  Full details of any limitations
>> should be called out by the solutions documents.
> Given the complete absence of discussion of potentially multiple solutions and their co-existance, it rather seems
> as if there is complete desinterest to ever support more than one.

That is correct. That would be like having a single network layer that contained parts of IPv4 and IPv6, intermingled.
Would you like to see routing headers introduced into IPv4? :-)

> For example, is there ever a reason why two MNA solutions could not co-exist if individual packets
> only have one or more sub-stacks of a single solution ? If not, then why not say that at least.

Then an implementation would have to deal with two encodings, thereby doubling everyone’s implementation effort, 
standardization work, and testing.

Two standards for the same function are always a bad idea.  See OSPF and IS-IS.  The redundancy has been
insanely inefficient and held back both the IETF and the industry.  And those two protocols do not have to

> And what reasons would there be why sub-stacks from two solutions could not be in the same packet ?

Developer sanity.

> And what do we buy if we then say that solutions SHOULD NOT have such a limitation ?

If solutions are adqueately flexible, then they are both nearly equal from a capability perspective, so we buy nothing.

> Or else, if there is desinterest to think about this, then the framework could say that the
> primary goal is to have one solution in a particular deployment only, and anything beyond that
> would be icing on the cake.

There has been zero discussion of this direction within the WG.

> Without any such discussion it's really hard to understand what the thinking of the active people in the WG is.

We like our hair.  We have no wish to pull it out. ;-)

>>> Aka: please explain explicitly. I do suggest possible text further below.
>>> 3. Framework re. requirements document
>>> 3.1 It is quite unclear to solution designers which of the requirements document requirements
>>> are left unchallenged by the framework and which are replaced/superceeded by more detailled
>>> reqiurements from this framework document. It would be a lot better IMHO if solution designers
>>> would only have to refer to requirements from this framework document but not have to read
>>> the requirements document as well.
>> The intent is that the framework document does not challenge any of the requirements, but it may
>> bound how the solutions may meet those requirements. 
>> I’m very sorry that you dislike how the documents are organized, but this is a decision that was
>> taken very long ago and seems to work for most members of the working group.
> Sorry, "challenged" was the wrong word, i should have probably said "addressed".
> Aka: some requirements are addressed directly by the framework. And there is no text in the
> framework listing which requirement is addressed by which framework section(s) for example.
> And if framework text narrows the solution space of a requirement, that is also not written.

I agree with the sentiment here.  Unfortunately, our ability to automatically cross reference between
documents is extremely limited and manual cross referencing would very likely be unmaintainable.

> And if the implementer of a solution in his understanding sees a conflict between some text
> in the framework and a requirement, what should he then do ?

As with any specification, questions should be directed to the WG.

> Ignore the framework and follow
> how he reads the requirements ? Or ignore the requirements and follows how he interprets the
> framework ? Aka: Any requirements that are "addressed" by the framework seem to be requirments
> where implementers shuold only follow the framework, right ?
> Other requirements may not be addressed by the framerwork at all. Which is fine.
> Without this mapping, requirements and whether they're addressed by thre framework,
> it's really hard for implementers IMHO.

I think that you’re ignoring that the solution document provides a great deal of clarity.  An
implementor should NOT be working just from the framework and requirements.

>>> Aka: Add a section to the framework document, inherit all the requirements from the
>>> requirements document, delete all the ones that are superceeded by the more refined requirements/
>>> definitions from the framewokr document, and then review the remaining ones and see what
>>> to do about them - keep unchanged or refine/add-explanations from framework view etc.. pp.
>>> This is ultimately the check if the framework is considered to be complete against
>>> the requirements document, and i don't think a reviewer of the framework document should do the
>>> first run for this - but rather the authors by doing this cut & check.
>>> Worst case, if something like this is not done, how would developers of solutions address
>>> inconsistencies between framework and requirements document ? Or know what to do about
>>> requirements assumed to be unaffected by the framework if those are not clear to developers how
>>> to translate into a solution ?
>> There should be no inconsistencies between the requirements and framework documents. If there
>> are, we should repair them now.
> Then please first write down this mapping, aka: what you as the authors of the framerwork
> think are the requirements addressed by the framework, and by which section/functionality
> accordingly.

The best mechanism that I have for doing this right now would be manual reference to requirement numbers.
If the requirement numbers ever change, it would necessitate recomputing all cross references manually.

I am not filled with joy at this prospect. This is an extremely labor intensive approach to hitting a moving target.

>>> I have comments about specific requiremnts further down in this review.
>>> It would also be good to make the required reading consistent with the requirements
>>> document. E.g.: it mentions rfc3031/rfc3032 and then starts to hand-wave about extensions
>>> - making it difficult to undersstand which of those extensions migh be relevant. This framework
>>> document does only mention rfc3031/rfc3032 in passing but not in the main text.
>> There are no references to RFC 3032.  RFC 3031 is referenced in the section on partial processing. 
>> Extensions are only mentioned in the abstract and introduction, referring to future network action 
>> specifications.
> RFC3032 is mentioned in the requirements. My first point was consistency.
> Especially because implementers of solutions will certainly have to understand it.
> My second point was about starting the doc with something like "Readers and implementers of solutions
> are assumed to be familiar with [RFC3031], [RFC3032], ... - these are the specifications for the
> aspects of MPLS that this framework relies upon".

This seems to be highly redundant.  This is the reason that we have references in the first place, so this
seems about as helpful as saying “Readers should read the references”.

>> No hands are waved in the document.
> How about "en passant" ?
> Aka: one has to go through the whole document to figure out what "base architecture"
> is that one has to understand.  Better architectural/framework type documents make this clear in the beginning.
>> Could you please be more specific?
> See above.
>>> 3.2 I tried to validate if this document meets the requirement listed in the requirements
>>> document, and stumbled across the following points in the requirements document:
>>>  50.  In-stack ancillary data MUST only be inserted in conjunction
>>>       with an operation conforming to [RFC3031].
>>>  51.  Post-stack ancillary data MUST only be inserted in conjunction
>>>       with an operation conforming to [RFC3031].
>>> I am guessing that this does not apply to the initial construction of the label
>>> stack on the ingress LSR ? It would be good to say so explicitly.
>> This would seem to be a comment about the requirements document.
> Only an understanding question, yes. Because i was trying to understand if/how the fwk matches the
> requirements, but i am unclear about the point i raised. 
> But i've raised the question in feedback to the req. doc.

Yes, my understanding is that the LER at the start of the LSP has free rein to compose 
the label stack and PSD as it pleases.

>>> Also, RFC3031
>>> only seems to define "operations" as something derived from LSR state, aka: NHLFE.
>>> And network actions for example do not necessarily require NHLFE (but often could be
>>> stateless alternatives). But network actions are also referred to as "operation" in the
>>> framework draft. Aka: requirement 50/51 can be read as if a network action itself
>>> indicated by a Network SubStack can not modify the MPLS Label Stack unless it has
>>> NHLFE associated with it.
>> Again, this seems like a comment on the requirements document.
> Yes, likewise raised against req-doc. Its mostly about the ?overload? of the term operation
> which may unintentionally constrain what we can do with MNA...

Operation is not a defined term in the framework document and I can find no constraints
related to it.  Thus, I’m still quite confused by the comment and how to address it. If you
have a specific text change in mind, could you please articulate it?

[Many comments where you seem to be agreeing elided.]

>>> 4. It would be nice if the framework could venture to provide some MPLS WG agreed
>>> upon estimates of future growth requirements for MNA solutions, or else solutions
>>> will come up with either over or underscaled encodings and reviewers can not really
>>> vet how good a solution proposal is.
>>> Aka: Total of N actions that need to be encodeable
>>>    N1 actions requring no further parameters
>>>    N2 actions requiring 8 bit parameters
>>>    N3 actions requiring 32 bit parameters
>>>    N4 actions requiring 64 bit parameters
>>>    worst case substack: M actions, one with 8, one with 32 bit one with 64 bit parameters
>>> Just as an idea, of course i do not know for any of these parameters the right
>>> numbers, but without the framework providing any such bounds its not sufficient
>>> for
>> As you know well, predicting the future is somewhat challenging. We have no basis
>> for making any estimates for any bound on how future actions will be employed.
> Well, one could look at the superset of action proposals brought up even if not
> agreed upon in requirements yet.
> Without ANY requiremens against extensibility, i am sure there will be as much
> extenibility as we've seen in IPv6 extension header support so far in forwarding
> planes - aka: NONE. Hence also the work on better HBH header requirements
> (draft-ietf-6man-hbh-processing-16).

We have a number of action proposals, but it is very unclear about how those will
progress.  We have had no complaints about extensibility from the existing 
action proposals.

Again, I’m very unclear on how I can address this comment. Is there specific
text that you would like to see?

>>> 17   This document specifies an architectural framework for the MPLS
>>> 18   Network Actions (MNA) technologies.  MNA technologies are used to
>>> nit:
>>> I would suggest to add
>>> [ RFC-Editor: Please add "MNA - MPLS Network Action" to RFC Abbreviations List ]
>>> so your new TLA is recognized there. Somewhere in the document where you like it.
>> As this has far-reaching and long-lasting effects, I hesitate to proceed with this.
>> If MNA turns out to be unpopular, we will have cluttered our namespace indefinitely.
>> We can always make this addition later if MNA proves to be popular.
> Just for the fun of it:
> SR         - sender report (SR) 
> No mention of Segment Routing ;-)
> Aka: There will always be historical clutter. No reason for not mentioning your abbrebviation IMHO.

Adding to the historical clutter is not our objective.  Rather, we would much rather wait to
see relevance.  If MNA or SR ever become popular, then I would support adding them to
the abbreviations list.  It is not a requirement that this happen in the framework document.

> If you want to earn a star in the abbreviation list. That's when you need to show
> wide adoption/understanding.

That will not happen as a result of making a request in the framework document.

>>> 19   indicate actions for Label Switched Paths (LSPs) and/or MPLS packets
>>> 20   and to transfer data needed for these actions.
>>> nit: 
>>> A bit more explanatory introduction of what "action" on the LSP or packet means would be
>>> nice. Maybe something like:
>>> ... to indicate actions that impact the forwarding or other processing (such as monitoring)
>>> of the packet along the Label Switched Path (LSP) of the packet or that impact the packet content 
>>> itself.
>> I think that this is largely addressed in the introduction, where there is a bit more description about actions. 
> You're judging from a subject expert side. This is not always helpful
> to create text to explain things well to novice readers. My suggestion stands.

Ok, I’ve taken part of your suggestion:

	MNA technologies are used to
	indicate actions that impact the forwarding or other processing (such
	as monitoring) of the packet along the Label Switched Path (LSP) of
	the packet and to transfer any additional data needed for these

I’m unaware of anything that would impact the packet content and I would not support that, 
so I’ve omitted that for now.

>>> 109   The document provides the foundation for the development of a common
>>> 110   set of network actions and information elements supporting additional
>>> 111   operational models and capabilities of MPLS networks.  Some of these
>>> 112   actions are defined in existing MPLS specifications, while others
>>> 113   require extensions to existing specifications to meet the
>>> 114   requirements found in [I-D.ietf-mpls-mna-requirements].
>>> nit:
>>> I am somewhat confused about the above text. Let me ask whether the following text
>>> is a correct (if not better) rewrite of the goal of this document and its relationship to
>>> requirements and solutions:
>>> ...
>>> This document provides the architectural framework for the development of
>>> complementary and/or competing solutions for operational models and capabilities of MPLS
>>> which are called "MPLS Network Actions" (MNA).
>> Several people appear to have an allergy to the word “architecture”, so we’ve tried to avoid it,
>> despite that being exactly what we’re trying to do. It is not our intent to produce competing 
>> solutions. While it was anticipated that we might have competing solutions during our initial work,
>> we were able to merge proposals so there is no remaining competition.
> a) Sure, s/architecture// in the proposed text.
> b) Rephrase your explanation so it can go into the text. Aka: initial thinking is that
> a single solution target may suffice, but longer term do not preclude alternative or
> complementary solutions.

Per the discussion above, I’ve already added a ‘requirement’ that the WG produce at most 
a single ISD and a single PSD solution.

>>> MNA solutions derived from this framework are intended to support all or a subset
>>> of the requirements found in [I-D.ietf-mpls-mna-requirements].
>> We are not subsetting requirements.
> "address". Another part of the requirements is not explicitly addressed by the framework
> but need to be inherited by the solutions directly from the requirements doc.


>>> In additions, MNA solutions
>>> may also support actions for which prior solutions have been defined through existing MPLS
>>> specification, for example to make it easier to combine existing and new actions in the
>>> same MPLS packet.
>> Yes, we are trying to subsume some existing functionality (e.g., Entropy Label) because it 
>> encoding efficiency. Other actions will require new definitions.
> What i wrote was suggested explanatory text to make these things clear - which i was just
> guessing at when reading the fwk draft.

Ok, reworded:

The document provides the foundation for the development of a common
set of network actions and information elements supporting additional
operational models and capabilities of MPLS networks.  MNA solutions
derived from this framework are intended to address the requirements
found in {{I-D.ietf-mpls-mna-requirements}}. In addition, MNA may
support actions that overlap existing MPLS functionality. This may be
beneficial for numerous reasons, such as making it more efficient to
combine existing functionality and new functions in the same MPLS

>>> 122   This document generalizes the concept of "forwarding actions" into
>>> 123   "network actions" to include any action that an MPLS router is
>>> 124   requested to take on the packet.  That includes any forwarding
>>> 125   action, but may include other operations (such as security functions,
>>> 126   OAM procedures, etc.) that are not directly related to forwarding of
>>> 127   the packet.
>>> minor:
>>> I am not so happy with a term as broad as "network action". Are failures/recoveries
>>> of components and resulting re-routing, change of link-speeds  and the like network
>>> actions even before i look into specific traffic ? The term itself makes me think they
>>> are, but i think your definition of "network actions" likely not.
>>> Maybe "network traffic actions" may be better. And/or scoping the definition to
>>> "operations triggered by network traffic or its absence".
>> Changing our primary acronym and abbreviation is a broader topic that should be
>> taken up separately. 
> You don't have to change the terminology. You just need define what
> can trigger those "network actions" and what type of impact an action can have.
> E.g.:
> - Can a network action only be triggered by a packet ?

For MNA, yes.

> - Can a network action only impact the packet that triggers it ?

No.  Section 2.4 already points out that an action can affect state and thus 
can have follow-on ramifications.

> Two examples where i am not sure if they could be valid MNA "network actions":
> A route-change (e.g.: LFA) triggers an MPLS OAM packet.
> In this case, no packet is involved triggereing the action.
> I am guessing this is not an MNA "network action", but i assume an
> MNA network action can only be triggered by a packet with an MNA substack
> and/or MNA PSD. But that's not written explicitly in the framework.
> A packet with a "start/stop to encrypt for this label" trigger actions on packets
> (for example by changing NHLFE state) that follow this packet but do NOT
> have an MNA substack.
> I am unclear if such NHLFE state impacting actions are valid MNA "network actions".
> And i don't want answers in email. I wan to understand the answers from reading
> the draft.

Ok, how about:

   MPLS network actions are always triggered by an MNA
   packet, but may have implications for subsequent traffic, including
   non-MNA packets, as discussed in Section 2.4.

>>> 142   Although this is an Informative document, these conventions are
>>> 143   applied to achieve clarity in the requirements that are presented.
>>> minor:
>>> Why then is this document only informational ? An added justification here in
>>> the text would be helpfull. "This document is informational only even though it
>>> raises normative language requirements against work referring to it because ...."
>> It is informational because it does not mandate anything for implementations. The
>> solution documents provide those mandates.
> Hah! That's a new explanation to me. "Requirements against other documents can not be
> normative" ... ??? We now even have normative (abstract) API RFCs, so IMHO this
> framework is perfect for standards track.

Please feel free to take this to the mailing list.  This is over my pay grade.  Informational
was discussed and was the consensus of the WG.

>>> 217 2.  Structure
>>> nit: It would be lovely to include an ASCII graphics of a NAS. And highlight
>>> accordingly that the S-bit is maintained to make this more obvious.
>> This can be found in draft-ietf-mpls-mna-hdr.
> Not an abstract one, only the ones for the solution. Aka: would be lovely
> to have the generic/abstract header as ascii showing just the textually defined
> elements from the framework.

There are many possible solutions. Presenting one would biased. 
Presenting many would be confusing.

>>> mayor:
>>> This is where i as a reader started wondering whether a network or packet is supposed
>>> to support only one or multiple MNA solutions. Aka: please clarify here or earlier.
>> That’s still up to the WG to decide.  As of right now, a packet may have ISD, PSD, or both.
> Issue was not about ISD vs. PSD but just one solution or multiple was
> discussed already above. 

Answered above.

>>> nit:
>>> reference for post-stack data would be good, or definition.
>> There is no reference as we have not and may not publish anything.  PSD is defined in the
>> requirements document.
> This was only about a reference for the definition of the term if it exists.

There’s already a refence in section 1.2.2.

> See my questions about the use of the term in the requirements doc as well,
> which refers to PSD as something pre-exist (before requirements document).
> Aka: Is an IP header following an MPLS label stack "post-stack data" or not ?
> I think here we are really talking about new "MPLS PSD".
> But question also raised now as concern against requirements doc.

It’s their definition, I will let them respond.

>>> 222   A solution must specify where in the label stack the network actions
>>> 223   sub-stacks occur, if and how frequently they should be replicated
>>> 224   within the label stack, and how the network action sub-stack and
>>> 225   post-stack data are encoded.
>>> mayor:
>>> This is where one wonder about single versus multiple solution PSD in a single packet or network,
>>> so again, please make sure the expectation against that are explicitly describe here or earlier.
>> Sorry, I don’t understand this point.  Do you have a specific text change that you would like to
>> propose?
> This was just about me getting completely lost about whether or not a packet can
> have sub-stacks from multiple solutions and if that could ever work.

This has been addressed above already.

> Maybe given what you said so far here, it would be best to write in the start of the
> doc that the framework is not necessarily sufficient to fully consider mixing sub-stacks
> from more than one solution because that is not seen as a well-defined requirement.

I think what I’ve added is sufficient.

>>> 227   A network action sub-stack contains:
>>> 229   *  Network Action Sub-Stack Indicator: The first LSE in the NAS
>>> 230      contains a label with special semantics, called the MNA label,
>>> 231      that is used to indicate the start of a network action sub-stack.
>>> nit:
>>> Further up in the document you introduce this LSE with the abbreviation NSI.
>>> Here you do not use the term NSI but introduce the redundant term MNA label that you
>>> then re-use i think 3 times in the document.
>> The NSI contains the MNA label but is not synonymous.  The MNA label is 20 bits, 
>> the NSI is a 32 bit LSE.
> Ok, got it. Still confusing. The terms "NSI LSE" and "NSI Label" would be less
> redundant IMHO.

This is the terminology that the WG has reached consensus on.  If you feel strongly
about this, please feel free to take it up on the mailing list.

>>> 243   Each network action present in the network action sub-stack may have
>>> 244   zero or more LSEs of in-stack data.  The ordering of the in-stack
>>> 245   data LSEs corresponds to the ordering of the network action
>>> 246   indicators.  The encoding of the in-stack data, if any, for a network
>>> 247   action must be specified in the document that defines the network
>>> 248   action.
>>> minor:
>>> This (ordering) sounds as if each LSE can only belong to one action, but
>>> it would certainly be useful to share LSE across actions. For example a solution
>>> could define multiple actions but also a common authorization ticket LSE applying
>>> to the other actions. Aka: There may be a more complex relationship between
>>> multiple actions and their parameters in a single solution than a 1:1 mapping
>>> between action and in-stack-data for them.
>> Fair enough. The way that I would describe this is with an NAI indicating the 
>> common authorization ticket and then additional NAI indicating any operations
>> to be performed with the ticket. Thus, you still end up with ancillary data that is
>> specific to an NAI, and then NAI may interact.
> Maybe add something like "in-stack data may be shared by multiple actions".


>>> 258   A solution must specify the order that network actions are to be
>>> 259   applied to the packet.
>>> minor:
>>> Usually in MPLS i would expect that stack order is execution order. It would
>>> be nice to include a simple reasoning (or even example) how MNA becomes better by that
>>> not being implied. Yes, i understand now, order of bits in an encoding for example,
>>> but on cold read that's not immediately obviously, so would be could to elaborate.
>> Flag NAI have no stack order, thus its wholly up to the solution to specify the semantics.
> Sure. I was hoping you'd reprase that and add as explanation, e.g. after the sentence
> This is necessary, because Flag NAI encoding order does not imply action execution order.

Reworded slightly.  

>>> minor:
>>> how would you distinguish between actions only on
>>> a) ingress, b) ingress/egress c) egress only
>> Ingress is silly.  Just do it. There is no point in signaling to yourself.
> "Silly" is not immediately obvious to me.After all, the stack/PSD imposition may
> be done by a different (MPLS application) layer, and if i can not trigger something
> to happen at the MPLS layer in the sending device, i need to come up with a whole
> different mechanism to do it.

Regardlesss, it never leaves the system, so we should not be standardizing the mechanism.

>> I2E only affects the egress.
>>> For example consider DetNet PREOF. ingress needs to create control word with sequence
>>> number, egresss does duplicate elimination based on control word sequence number
>>> (this was actually defined by PALs for psuedowire, but never used with exactly this
>>> expensive action, but only simple switchover). In any case, are these two separate
>>> actions, one ingress, one egress ? Or one ingress/egress action..
>>> Explanations to that end would be useful in this framework.
>> That’s exactly I2E.
> I was thinking of such a header where one element for example a sending timestamp, so that
> actually is best done as low-level in the forwarding plane as possible to be accurate.

Sending is not the issue.  The question is which downstream nodes will need to process it.  If every 
hop needs to process it, then it should be HBH.  If it’s only the exit, then I2E or Select at BOS.

> I'd have to think longer to find potentially a more boring older example.
> I think we had a bunch of QoS mechanisms where you also did that QoS on the ingress PE
> based on appropriate packet marking from the application, not only on egress PE
> (aka: not the aggregate QoS you do on the P LSR).

Then HBH would be an obvious approach.

>>> 273   *  Select: Only specific nodes along the path will perform the
>>> 274      action.
>>> 276   If a solution supports the select scope, it must describe how it
>>> 277   specifies the set of nodes to perform the actions.
>>> minor:
>>> I wonder if i am implying what is supposed to be allowed or if i am overimagining:
>>> Performing of action may be based on a combination of ISD/PSD encoding and/or
>>> LSR configuration/state.
>> Correct.
>>> If any of this freedom is meant to be excluded, please write so. Else it would be
>>> reconfirming to include the above.
>> Specific suggestions?
> As i wrote: for example;
> Performing of action may be based on a combination of ISD/PSD encoding and/or LSR configuration/state (NHLFE)"

I added:

 The precise semantics of an action are dependent on the
contents of the packet, including any ancillary data, and the state of
the router.

>>> 311   In some solutions, an indication may be provided in the packet or in
>>> 312   the action as to how the forwarder should proceed if it does not
>>> 313   recognize the action.  Where an action needs to be processed at every
>>> 314   hop, it is recommended that care be taken not to construct an LSP
>>> 315   that traverses nodes that do not support that action.  It is
>>> 316   recognised that in some circumstances it may not be possible to
>>> 317   construct an LSP that avoids such nodes, such as when a network is
>>> 318   re-converging following a failure or when IPFRR [RFC5714] is taking
>>> 319   place.
>>> minor:
>>> IMHO another strong reason to have a framework level definition for how to determine
>>> end-of-PSD without having to be able to parse the whole ISD.
>> That would be up to a PSD proposal.
> As i tried to lay out, _if_ support for multiple different
> solutions within the same packet was required, then it would require a similar
> cross-solution PSD header as the sub-stack NSI LSE is.

I’m sorry, I can’t parse this comment.

> Arguably you are preferring standardization of ISD by defining the concept of 
> sub-stack format with NSI LSE, which IMHO does allow to mix different solutions,
> but you are not doing the same for PSD. I'd call that ISP bias.

I assume you mean ISD bias.

The framework document represents the consensus of the WG. Currently, the WG 
has decided that PSD is not required. Thus, the framework is NOT trying to be 
unbiased with respect to ISD vs PSD.

> (which is fine, if that's what the WG agrees to, i would just like to see all such
> core decisions appropriately be written down in the framework).

This is reflected in the requirements document.

>>> 321 2.3.  Signaling
>>> 323   A node that wishes to make use of MNA and apply network actions to a
>>> 324   packet must understand the nodes that the packet will transit and
>>> 325   whether or not the nodes support MNA and the network actions that are
>>> 326   to be invoked.
>>> nit:
>>> Should this not always be "MNA solution" instead of "MNA" in this paragraph ?
>> I don’t think so.  Why?
> A node that inserts ISD and/or PSD of one specific MNA solution needs to understand that
> the node that the packet will transit and whether or not those transit nodes support
> that specific MNA solution and the network actions invoked by the inserted MNA data.
> again: text to clarify the distinction between potentially different MNA solutions.

Again, we’re only supporting one solution.  If actions are encoded using something
else, well, those actions and not supported by the transit nodes, so the original text
is still on point.

>>> minor:
>>> Why ?
>> Predictable results. Not knowing what nodes will support the requested actions
>> will result in unintended consequences.
> Shouldn't ISD be ignored by a node not supporting it ?
> What happens when it passes a legacy node completely unaware of MNA ?

That is the point of the entire section.

> If you expectation is "we don't know", then it would be good to write that
> down, because the rest of the document made me believe it would be ignored,
> unless it happens to become Top of Stack.

I’ve added a comment saying that we assume that MNA is ignored by nodes that don’t support it.

>>> Aka: there are all type of cases where good or baad things happen if this is not
>>> the case. For example, i should be able to build an action that happens for each
>>> hop, but if the hop doesn't support the action, it ignores it, right ? And thats
>>> may be a good thing - or maybe not.
>> Or maybe it sends the packet to its worst interface. Or maybe it crashes. Or maybe it encrypts the packet.
>> If a node sees something that it doesn’t support the results are undefined.
> See above. that expectation is not clear from the document. It's also not clear to
> me how this would be the case for the best NSI options. Those options should result in
> just an undefined MNA Label to be seen, and i thought that MPLS has some good OAM
> when that happens.

If it doesn’t come to the ToS, it shouldn’t be relevant.

>>> I would suggest rephrase to something like:
>>> Various networking functions such as (nonwithstanding) diagnostics, operations, orchestration
>>> and instantiation of network services may need to understand exactly which nodes support
>>> which actions to determine which actions and parameters to invoke on which node, which optimal
>>> encoding to use and what results to expect.
>>> 326   to be invoked.  These capabilities are presumed to be signaled by
>>> 327   protocols that are out-of-scope for this document and are presumed to
>>> 328   have per-network action granularity.  If a solution requires
>>> 329   alternate signaling, it must specify so explicitly.
>>> nit:
>>> "alternate signaling" meaning what ? The prior sentence allows all signaling,
>>> it only expects some specific granularity, so if anything was alternate it would
>>> be degree of granularity ??
>> Meaning if you need something strong than whatever is cooked up for describing MNA, then you
>> need to say so.  I’m expecting a YANG model with a per-action capability. If you need to
>> know that a node supports action Z but only on Tuesdays, there will need to be an alternate
>> mechanism to describe that.
> Aka:
> replace
> If a solution requires alternate signaling, it must specify so explicitly.
> with
> If a solution requires signaling with a different granularity than per-network action,
> it must specify so explicitly. Time based actions are an example of such different
> granularity.

I don’t think you mean what you said there.  The first sentence is semantically identical.

The second sentence is problematic because we don’t know what is covered.  For example,
TVR working group is working on scheduled capabilities across YANG models.  That would
actually cover temporally dependent support.

>>> Something like:
>>> The signalings should allow to indicate all differences in support and configuration
>>> of the MNA solution that can be of interest to any of the aforementioned functions.
>>> 331 2.3.1.  Readable Label Depth
>>> 333   [RFC8662] introduced the concept of Entropy Readable Label Depth
>>> 334   (ERLD).  Readable Label Depth (RLD) is the same concept, but
>>> 335   generalized and not specifically associated with the Entropy Label
>>> 336   (EL) or MNA.  Readable Label Depth is defined as the number of LSEs,
>>> 337   starting from the top of the stack, that a router can read in an
>>> 338   incoming MPLS packet with no performance impact.
>>> 340   ERLD is not redundant with respect to RLD because ERLD specifically
>>> 341   specifies a value of zero if a system does not support the Entropy
>>> 342   Label.  Since a system could reasonably support MNA or other MPLS
>>> 343   functions and need to advertise an RLD value but not support the
>>> 344   Entropy Label, another advertised value is required.
>>> nit:
>>> I would move all mentioning of ERLD from 333-338 to the beginning of 334, and
>>> make that paragraph a "Note:"
>> I don’t understand how to do that.
> typo in line number, sorry.
> Readable Label Depth is defined as the number of LSEs,
> starting from the top of the stack, that a router can read in an
> incoming MPLS packet with no performance impact.
> Note: [RFC8662] introduced the concept of Entropy Readable Label Depth
> (ERLD).  Readable Label Depth (RLD) is the same concept, but
> generalized and not specifically associated with the Entropy Label
> (EL) or MNA. ERLD is not redundant with respect to RLD because ERLD specifically
> specifies a value of zero if a system does not support the Entropy
> Label.  Since a system could reasonably support MNA or other MPLS
> functions and need to advertise an RLD value but not support the
> Entropy Label, another advertised value is required.
> The whole ERLD is an aside, hence the nit proposal to make that textually clearer.

I regard it as important context, but I’m happy to reorder the sentences.

>>> 346   A node that pushes an NAS onto the label stack is responsible for
>>> 347   ensuring that all nodes that are expected to process the NAS will
>>> 348   have the entire NAS within their RLD.  A node SHOULD use signaling
>>> 349   (e.g., [RFC9088], [RFC9089]) to determine this.
>>> minor/Q:
>>> Any requirements against nodes if a NASS can only be read partially  by a node
>>> because it's too deep ? "All Hell Breaks Loose" ?
>> That behavior is undefined, so the latter.
> "If a node creating a packet with a NAS is calculating the depths with which
> it arrives at a particular processing node incorrectly, then it may be deeper
> in the label stack than that nodes RLD, which may cause unexpected behavior.
> A reason for that to happen could be in unexpected behavior on an intervening
> router that pushes more LSE on the stack than expected.
> Aka: i think it helps the reader who tries to understand MNA to write these
> type of details down.

We’ve already defined the sanctioned behavior. Everything outside of that is undefined.

>>> 351   Per [RFC8662], a node that does not support EL will advertise a value
>>> 352   of zero for its ERLD, so advertising ERLD alone does not suffice in
>>> 353   all cases.  A node MAY advertise both ERLD and RLD.
>>> minor:
>>> Is this trying to say something/different from 333-338, or is this just redundant ?
>>> If not redundant, please rephrase because i can't see a difference. Else delete.
>> The point is the last sentence.
> Still unclear exactly what it means.
> Do you mean "A node MAY advertise RLD even if it is redundany because of the advertised ERLD" ?
> (then say so, else explain).

I’ve added a bunch of redundant text.

>>> 355   RLD is advertised by an IGP MSD-Type value of (TBA) and MAY be
>>> 356   advertised as a Node MSD, Link MSD, or both.
>>> minor:
>>> I guess the allocation of a single MSD-Type means that there can only be a single
>>> MSD solution be deployed at the same time, or else they would need to be implemented with
>>> the same RLD...
>> We are not expecting differing solutions to have different RLDs. That should be a property of a node’s
>> hardware.
> Thanks.
> Would be useful to add:
> "NAS encoding differences in different MNA solutions are not expected to impact the nodes RLD".

I’ve added something similar.

>>> An MNA solution requires new forwarding plane firmware (AFAIK quite expensive), but we don't
>>> want to expect the vendor to also implement the IGP MSD-Type in the control plane ?
>> New forwarding plane firmware is not significantly more expensive than new control plane software.
>> It’s just a matter of programmer time. 
>> We do want the vendor to implement the RLD MSD-Type.
> Then you should upgrade the MAY we discussed above to a SHOULD.

I must have lost context.  I think that the MAY we discussed above is "A node MAY advertise both ERLD and RLD”.
We do NOT want a node to advertise ERLD and RLD if ERLD == RLD.  It’s redundant and a waste of IGP
space.  Thus, it should not be a ’SHOULD’.  [My usual rant about the IGP not being a dump truck are elided.]

>>> That's just unnecessary case permutation complexity and guesswork. Heck, didn't the
>>> draft state that the supported actions etc. would need to be implemented in the 
>>> protocol (i assume also IGP) ???
>>> Aka: suggest to remove 365. KISS.
>>> 374 3.  Encoding
>>> 376   Several possible ways to encode NAIs have been proposed.  In this
>>> 377   section, we enumerate the possibilities and some considerations for
>>> nit:
>>> s/enumerate the possibilities/summarize the options proposed/ ??
>> That seems purely synonymous. Benefit?
> I just do not see the enumeration of NAI possibilitites. Are two-level
> subsections an enumeration ? And does the sentence mean its an enumeration
> of possibilities or possibilities and considerations... ?


>>> 393   within the sub-stack, as inefficient use of the bits could result in
>>> 394   the addition of unnecessary LSEs.
>>> 402 3.1.1.  Existing Base SPL
>>> 404   A solution may reuse an existing Base SPL (bSPL).  If it elects to do
>>> 405   so, it must explain how the usage is backward compatible, including
>>> 406   in the case where there is ISD.
>>> minor:
>>> Are NASS considered to be ISD (or at least part of the NASS) ?
>> To be very precise: ISD are just the ancillary data.  NAS contains ISD.
>>> If not, then that would be good to define upfront somewhere.
>> ISD is defined in section 2. NAS is defined in section 1.2.1.
>      In-Stack Data (ISD): A set of zero or more LSEs that carry
>      ancillary data for the network actions that are present.  Network
>      action indicators are not considered ancillary data.
> So, that definition is missing the piece about the repurposed TC/TTL fields then ?

The TC and TTL fields are not part of the ISD.  They are part of the NAS.  NAS != ISD.

> That missing ASCII picture  could maybe show this best...

The ASCII pictures are in the ISD solution proposal.

>>> 412 3.1.2.  New Base SPL
>>> 414   A solution may select a new bSPL.
>>> suggestion:
>>> I would just ask for allocation of e.g.: 3 bSPL as configurable. Not really that
>>> difficult to match consistent configuration of bSPL semantic through control plane
>>> between adjacent SLR and not sending MPLS pckets to a neighbor with inconsistent config.
>>> That way a network could use up to three new bSPL based specs, MNA or other new semantics
>>> in parallel, as long as it's ok. to expect full hop-by-hop configuration consistency. 
>>> Just as one option.
>>> But no idea how important partial MNA deployments are. But it would be good to explore exit
>>> strategies from shrinking bSPL space other than increasing label stack size.
>> I’m not understanding your suggestion.
> Network wide configurable bSPL. BUt primarily makes sense when we predict more
> MNA solutions than we would want to allocate bSPL for. On the other hand,
> i would consider that any second MNA solution should require that its bSPL
> (assuming it uses bSPL) is configureable on nodes, so that as soon as then a third
> one comes around, we wouldn't need to allocate a third bSPL unless more than one
> MNA solution is needed.

Again, we have no interest in multiple MNA solutions. One bSPL to rule them all.

>>> 492 3.3.2.  Length Field
>>> 494   A solution may opt to have a fixed size length field at a fixed
>>> 495   location within the NAS.  The fixed size of the length field may not
>>> 496   be large enough to support all possible NAS contents.  This approach
>>> 497   may be more efficient if the NAS is longer but not longer than can be
>>> 498   described by the length field.
>>> 500   Advice from hardware designers advocates a length field as this
>>> 501   minimizes branching in the logic.
>>> minor:
>>> Does that not depend on encoding ? and FPE ?
>>> Aka: If i have action1, action1-parameter, action2, axction2-parameter, ... then
>>> continuation bit might be easier to parse this, but if i have 
>>> action-set, action1-parameter, action2-parameter, then a length field might be better.
>>> Aka: make sure you're not providing one-sided guidance.
>> The only guidance we got was from one source.
> "_Limited_ advice from hardware ..." ? ;-) (any appropriate wording to not overinterpret the importance
> of the sentence).

Reworded to make it singular.

>>> 503 3.4.  Encoding of Scopes
>>> 505   A solution may choose to explicitly encode the scope of the actions
>>> 506   contained in a network action sub-stack.  A solution may also choose
>>> 507   to have the scope encoded implicitly, based on the actions present in
>>> 508   the network action sub-stack.  This choice may have performance
>>> nit:
>>> So lets say i have one type of action called FOO, but i do create two actions out of it,
>>> one would be "FOO-on-every-LSR" and the other "FOO-only-on-explicitly-configured-LSR-for-FOO".
>>> Which of the two described options would that be ? In other words: i am not sure i do
>>> understand the disctinction completely.
>> You can either say:
>> HBH: Actions A, B, C
>> or 
>> Action A (HBH), Action B (HBH), Action C (HBH)
>> Either encoding is acceptable.
> That would be a good example to include into the text to make sure the definition of the
> encoding is understood by everybody.


>>> 508   the network action sub-stack.  This choice may have performance
>>> 509   implications as an implementation might have to parse the network
>>> 510   actions that are present in a network action sub-stack only to
>>> 511   discover that there are no actions for it to perform.
>>> nit:
>>> Without an example, it is not clear to me what the most obvious encoding option based
>>> choice would be to minimize the need to parse network actions unnecessarily. Aka:
>>> if you can, please insert such an example.
>> Ok, added.
> | For example, suppose that an NAS is embedded in a label stack at a	
> | depth of 6 LSEs.  Further, suppose that the NAS contains 3 actions.	
> | Each of those actions has Select scope and thus is not applicable at	
> | the current node.
> I do not understand this example. How is the depth relevant.

The depth is relevant because Select scope only is relevant when the NAS is directly 
under the top label.

> Are you trying to
> imply a 6 hop strict segment path ?


> Is there no
> lookahead ?

Lookahead was discussed under the section on RLD.  It may or may not be available, depending
on the nodes in the path.

> And why does the fact that the actions have select scope imply
> they are not applicable at the current node without also describing the set
> of nodes that are supposed to perform the action ?

Actions with select scope describe the nodes that they apply to by being directly underneath the top most label.

>>> 559 3.6.  Encoding of Post-Stack Data
>>> 561   A solution may carry some NAI and AD as PSD.  For ease of parsing,
>>> 562   all AD should be co-located with its NAI.
>>> minor:
>>> If i am not mistaken, today, the only component that is considered to be part of
>>> an "MPLS header" is the "MPLS label stack". E.g.: a pseudo-wire header including it's
>>> control word is considered to be it's own header. And in result, one has never needed
>>> or wanted to use the term "MPLS header", but could always refer to "MPLS label stack".
>>> With the introduction of (even optional) PSD, this changes. And in result i think
>>> this document should at an appropriate place (maybe earlier than here) introduce
>>> the term "MPLS header" (or any other/better term) to refer to the combination
>>> of MPLS label stack plus PSD. Which it effectively refers to in places, but only calls
>>> it MPLS label stack.
>> We are trying to avoid defining unnecessary terms.  It is not clear that a PSD solution
>> would appear immediately after the label stack, so it’s not even clear that there will
>> be a ‘header’.
> Huh ? What could ever be between label stack and PSD ?

Pseudowire control words. Other types of PSD.  

In general, the entire topic of what comes after the label stack was mishandled by the original
MPLS architecture.  See the entire debate about the 1st nibble.

> Even if there was such a think, i find it irritating to not have a term to refer
> to the whole MPLS packet enchilada, MPLS label stack plus PSD. I don't think it is
> redundant.

Please take it up with the WG.  This is not a document issue.

> Also note:
> | Alternatively, it also
> | offers end-to-end encryption of the MPLS payload with no
> |  cryptographic integrity protection of the MPLS headers.
> You do use the term MPLS header, even though i guess it is undefined so far as you seem
> to say above.

I’ve changed that reference to say ‘label stack’ instead.

>>> 564   If there are multiple instances of post-stack data, they should occur
>>> 565   in the same order as their relevant network action sub-stacks and
>>> 566   then in the same order as their relevant network actions occur within
>>> 567   the network action sub-stacks.
>>> mayor: 
>>> I don't think this is sufficient. The framework needs a definition that allows packet
>>> parsers to skip across PSD, and it needs a framework definition for how multiple
>>> different solutions could share PSD space. Even if there is only one solution
>>> supposed to be useable in one packet.
>> It is not up to the framework to mandate PSD encodings.
>> We are not seeking multiple PSD solutions. At present, we are not even seeking one.
> But you are seeking one framework that at least does not want to close the door on PSD,
> and you do not have any text in the draft even saying what you say above, hence i can
> only propose how to fix up the draft text as it is.

Again, as discussed above.

>>> This is not different from what the framework
>>> does for ISD: The parser can skip across the label stack using the S-Bit, and the
>>> different NAS indicator methods defined in this document introduce the ability to
>>> mix & match different solution ISD together with the extraction of Readable Label Depth
>>> from all LSR to even create label stacks (whether or not this is a requirement is
>>> as mentioned elsewhere unclear from the doc).
>>> If skipping beyond PSD with a simple method from the framework is not included, then the
>>> whole parsing with PSD would becomes more fragile. Being able to parse beyond PSD must
>>> not be dependent on network wide MNA solution configuration but from framework specification
>>> (IMHO). And i also do not see a way to mix two solutions without having an equvalent previously
>>> known structure like the NAS indicator labels.
>>> I don't think one wants to repeat the use of S-Bit for within PSD though. One of the
>>> big benefits of PSD over ISD would be to easier have contiguous parameters of 32 bits or longer
>>> without being interrupted by the S-Bits.
>>> I would suggest to consider including something like the following as PSD separators for
>>> the framework:
>>> 0                   1                   2                   3
>>> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>> |PSDcod | Rsv   |Total PSD len  | Next Hdr      | Next Hdr len  |
>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>> Aka:  PSDcod would be something no 0, 4, 6 to most easily recognize PSD
>>> for ECMP and other purposes (aka: first Nibble).
>>> Total PSD len would allow to skip with one lookup beyond all PSD blocks for for operations
>>> such as ECMP that would want to do this in the presence of PSD. 
>>> Next Hdr would be a new registry that could include both registration points
>>> for MNA solution PSD (one or more per solution) as well as existing next-headers,
>>> and Next Hdr len would be those headers/PSD (sub-headers) len. 
>>> Just an example and to demonstrate what seem to be the salient info fields needs:
>>> First Nibble compatibility, single length field to skip all PSD, and ability to
>>> chain different solution PSD. And likely make it fit into 32 bits.
>>> 600   A PSD solution should specify the contents of the first nibble, the
>>> 601   actions to be taken for the value, and the interaction with post-
>>> 602   stack data used concurrently by other MPLS applications.
>>> nit:
>>> I would move 600-602 after 585. Then reconsider why the text about BIER, 587-594
>>> exists, because it reads confusingly. I think you want to say that a PSD solution
>>> that wants to avoid for pre-existing ECMP to occur and/or that does not want
>>> to get confused with a Pseudowire header with control word should use a Nibble value other
>>> then 0, 4 or 6 - and one example of a mechanism that does already apply this logic
>>> is the BIER encapsulation of RFC8296. And leave it with that.
>>> But i am not sure if this BIER encapsulation is the most easy to understand
>>> example approach given how it was heavily tweaked around MPLS (making the last
>>> LSE of the MPLS label stack the first 32 bit of its own header), buit through its doc
>>> evolution started also to claiming and that it is an MPLS and non-MPLS solution and
>>> calling the label BIFT-ID instead of showing the label field, and hence making it
>>> severely confusing why those Nibble bits are there to the non-expert reader.
>>> If there is another, more simpler  example using a desirable
>>> the Nibble approach than BIER, maybe replace the text with such an example.
>> Thank you, but I think that the current text is clear as it is.
> I continue to disagree on that.

I’m sorry that you disagree.  I don’t understand what would help make it more clear for you.

>>> minor:
>>> I think the PSD section is jumping into details before making a fundamental statement like:
>>> PSD is a new (optional) element of the MPLS header and hence extending the
>>> MPLS header beyond the BoS LSE. In result, an MNA specification using PSD needs to
>>> specify how current deployed (and layer violating) MPLS functions that inspect and operate on
>>> data beyond BoS LSE (heuristically!) are supposed to operate in the presence of the new PSD.
>>> Then you have a better context of bringing up the example of ECMP and may recommend for
>>> PSD to inhibit the ECMP function through an appropriate choice of Nibble. But likewise,
>>> if these layer violating existing functions are considered valuable, then the PSD solution
>>> should also include an explanation how to parse to the end of the MPLS header, aka:
>>> beyond the end of PSD. And/or specify a mechanism for a PSD encoding to allow/prohibit for that
>>> to happen (because it may be undesired by the desired packet processing).
>> Again, this is up to a PSD solution to propose.
> No. The type of text proposed/requested by these two paragraphs is exactly along the lines of requirements
> in the framework about what an MNA solution needs to describe about it's ISD encoding. 

The framework does not present requirements.  That is up to the requirements document.  The framework is
responsible for articulating additional architecture for the MNA solution.  Since the WG has not decided that
PSD is a requirement, we do not have more to say about a PSD solution.

>>> 646 6.  Management Considerations
>>> 648   Network operators will need to be cognizant of which network actions
>>> 649   are supported by which nodes and will need to ensure that this is
>>> 650   signaled.  Some solutions may require network-wide configuration to
>>> 651   synchronize the use of the labels that indicate the start of an NAS.
>>> 652   Solution documents must make clear what management considerations
>>> 653   apply to the solutions they are describing.  Solutions documents must
>>> 654   describe mechanisms for performing network diagnostics in the
>>> 655   presence of MNAs.
>>> nit:
>>> Maybe call this "operational considerations”.
>> Why?
> You are describing what network operators need to do. If you have a reference
> that would make you feel safe that everything you describe/ask-for is well
> defined to be Management, then you're fine. I just think it is less risky
> to have the title match the content ("Network operatos will need ...").

Network operators manage their networks. That is what they do.  That’s why it’s called the 
management plane. This is not just about repairing links when they fail, this is about 
architecting, planning, and designing the specific network. That is, by definition, management.

>>> minor:
>>> I would love to see a requirement that all MNA solution must have a YANG specification
>>> of all the parameters required to build MNA label stacks through them. Aka: including
>>> the use of labels that indicate the start of a NAS as well as any other parameters,
>>> including (but not limited to) as network actions supported by nodes supporting (a subset of)
>>> the MNA solution.
>> This is not a requirement that the WG agreed to.
> Was it even brought up as a candidate requirement ?

I have no idea.  I cannot track everything that happens. I’m very happy if I can remember what
I had for breakfast yesterday.  ;-)

If you would like to participate as a WG member, you’re more than welcome to do so.

>>> I also think that would be a good replacement for 648-651. I don't think we should
>>> be happy with just "CLI" network wide configuration, and we should also not end up
>>> in a hodgepodge of protocol solutions for signaling. One solution promoting to only
>>> use e.g.: IGP, the other only YANG. Aka: make YANG mandatory to specify and let other
>>> signaling options be subject to downstream decisions.
>> Nothing requires CLI configuration.
> "network-wide configuration" is just a woefully underspecified term, and i just choose
> to pick the interpretation "CLI", unless the draft text makes it clearer that that
> is not the correct interpretation.

Well, you’re more than welcome to argue that we should ban the CLI and only do things
via YANG.  However, that would seem to have a broader audience than just the MPLS WG
and this document.

>>> 657 7.  Security Considerations
>>> minor:
>>> It would be good to go through the security related requirements in the requirements
>>> document and determine how to explicitly refer to them and address them in this document.
>>> E.g.:
>>>  12.  The design of any network action solution MUST NOT expose
>>>       information that is not already exposed to the PE to the LSRs.
>>>       It MUST NOT expose any information that a user of any service
>>>       using the LSP considers confidential [RFC6973] [RFC3552] .
>>>  13.  Network action solutions MUST NOT expose information considered
>>>       confidential to the user of the MPLS-based service [RFC6973] to
>>>       the LSRs.
>>> Could be discussed in the security section as follows:
>>> Network Actions introduce the generic challenge of moving potentially sensitive
>>> information from NHLFE to label stacks. In an MPLS network without hop-by-hop
>>> encryption, the sensitive information caried in sub stacks or PSD is likely
>>> a lot easier to exploit than whats only in LSR NHLFE state (and assumingly
>>> signalled via encrypted control plane). Deployments requring such higher confidentiality
>>> of MPLS packets because of MNA need to accordingly put stronger emphasis on per-hop
>>> encryption of MPLS packets (or other forms of confidentiality).
>>> pointing to IMHO a higher need for hop-by-hop encryption if/when such sensitive
>> That is not an approach that the WG has accepted.
> So then IMHO the framework's security considerations are incomplete because
> they do not describe how the framework directly addresses the security relevant
> MNA requirements.

I’m sorry that you feel that way.  Again, the framework is not intended to address requirements.
It bounds solutions.  Solutions must address the requirements.

>>>  14.  Solution specifications MUST document any new security
>>>       considerations that they introduce.
>>> As mentionined initially, inherit into this framework text.
>> The framework is not meant to be a repeat of the requirements.
> No, but it need to describe if/how it addresses individual requirements IMHO.

I’m sorry that you feel that way.  Again, the framework is not intended to address requirements.
It bounds solutions.  Solutions must address the requirements.

>>> 681   Particular care would be needed when introducing any end-to-end
>>> 682   security mechanism to allow an in-stack MNA solution that needed to
>>> 683   employ on-path modification of the MNA data, or where post-stack MNA
>>> 684   data needed to be examined on-path.
>>> minor:
>>> This makes it sound as if this is a new MNA challenge, whereas AFAIK the
>>> same problem exists already since day 1 for the MPLS label stack. So, unless i am
>>> not mistaken, it would help to clarify that MNA simply shares this challenge
>>> with MPLS itself.
>> MNA has changed the situation slightly by allowing mutable data in the label stack.
> Please include that explanation:
> MNA introduces the option of mutable data in the label stack, hence particular care ...

That’s exactly what the above paragraph says.

>>> 701   Within a single well-managed domain, an adjacent domain may be
>>> 702   considered to be trusted provided that it is sufficiently shielded
>>> 703   from third-party traffic ingress and third-party traffic observation.
>>> 704   In such a situation, no new security vulnerabilities are introduced
>>> 705   by MNA.
>>> mayor:
>>> But there is no requirement specified so far that such collaborating MPLS domains
>>> would need consistent configuration of e.g.: NAS start label, so there may be no
>>> malicious attack, but it just wouldn't work. Not to speak of knowing inter-domain
>>> thr supported actions. Unless the solution just attempts to MPLS-tunnel through the
>>> collaborating MPLS domain, which still wouldn't necessarily work in the presence
>>> of axctions with lookahead or PSD unless all that is consistent... 
>>> Maybe put a paragraph to this extend into the management/operational considerations
>>> section given how its not about (intentional) attack vectors...
>> I’m sorry, I don’t understand this comment.
> There is always the question of the scope of "security" (considerations), aka: if
> it should only cover malicious attacks or also bringing down the network by stupidity
> because of unmanageable operational complexity.

Security is hard enough without broadening its definition to include human error.

> Assuming warning about the latter the later, than the whole point of exchanging all the
> necessary information to properly create an MNA labels stack that will parse correctly in a different
> (adjacent/trusted) domain will become a lot more prone to errors and limited options than
> in a single domain (aka: no shared IGP to disseminate information for example). Aka: much higher risk

We are trying to push back very hard on IGP dependencies.  It is my favorite rant.

Typically, multi-domain networks are operate by a single management organizational structure, but 
are hieararchically peers within the administration.  This is typically due to network acquisition, but
also can result from geographic partitioning. While the organizational structure separation certainly
creates operational difficulties and increases the odds of human error, that in itself does not 
constitute a new security problem.

>>> 718   Several mitigations are available to an operator:
>>> 720   a) Reject all incoming packets containing MNA information that do not
>>> 721   come from a trusted network.  Note that it may be acceptable to
>>> 722   accept and process MNA information from a trusted network.
>>> 724   b) Fully encapsulate the inbound packet in a new additional MPLS
>>> 725   label stack such that the forwarder finds a BoS bit imposed by the
>>> 726   carrier network and only finds MNA information added by the carrier
>>> 727   network.
>>> minor:
>>> "MPLS label stack" here hould be "MPLS header" (as mentioned above) because it could
>>> include PSD.
>> We are not defining “MPLS header”, so this would be an inappropriate change.
> I continue to believe this is a shortcoming.

I’m sorry you feel that way. PSD is in no way a ‘header’. For that matter, neither is ISD, despite
what that document says.

>>> minor:
>>> In fact, one type of PSD could simply be an indication "end of MPLS header" (along
>>> the above described First Nibble options). Maybe we would even want to make it a
>>> requirement in this document for MNA solutions to include text about how to "fully encapsulate"
>>> an MPLS header when usingthat MNA solution. This does not have to be PSD based, but if it
>>> is not PSD based, then the MNA solution would have to describe how any of the
>>> other options would work with the ECMP and other post-stack legacy behavior.
>>> I'd certainly be in favor of providing a clear MPLS header separation from following
>>> data via PSD to potentially avoid running into these legacy compatibility issues.
>> That should be in a PSD proposal.
> And as mentioned above, the requirements about what a solution with PSD needs
> to describe wrt. backward compatibility need to be in the framework in the same way
> as there is a lot of good andn important requirements detail about ISD.

Again, requirements belong in the requirements draft.  The WG does not yet see
the need for PSD, thus, there is not a lot of architectural work done on supporting PSD.

>>> 729   A mitigation that we reject as unsafe is having the ingress LSR push
>>> 730   sufficient additional labels such that any MNA information received
>>> 731   in packets entering the network from a third-party network is made
>>> 732   inaccessible due to it being below the RLD.  This is unsafe in the
>>> 733   presence of an overly conservative RLD value which can result in the
>>> 734   third-party MNA information becoming visible to and acted on by an
>>> 735   MNA forwarder in the carrier network.
>>> minor:
>>> Not quite sure about the wording here. I guess the point is that the RLD is only a
>>> guarantee that the LSR can at least look as deep as RLD, but it is no guarantee that
>>> the LSR will not be able to look deeper ?? If so, then say so. Maybe even revisit the
>>> initial text about RLD in this spec (337 ff). E.g.: add that RLD is just a minimum guarantee,
>>> and that the LSR may be able to look deeper (but just not consistently enough to guarantee it).
>> RLD is about performance, not about absolute guarantees.
> Makes a lot more sence than what i read from:
>>> 732   This is unsafe in the
>>> 733   presence of an overly conservative RLD value which can result in the
>>> 734   third-party MNA information becoming visible to and acted on by an
>>> 735   MNA forwarder in the carrier network.
> Maybe:
>  This is unsafe, because RLD only indicates a no performance impact depth. Routers may examine and act
>  upon NAS deeper than RLD.
> If that's not correct then i do not understand your comment.

Some routers can (and do) read the entire packet.  Thus, RLD is not a security mechanism. If one wants
to put a boundary on MNA information, then there needs to be an explicit BoS indication.

>>> If instead RLD is really meant to be a very accurate depth (min = max), then the above
>>> text makes no sense to me. However i full agree that we should never simply concatenate
>>> MPLS label stacks between non-trusting domains. Besides it wouldn't work with PSD on
>>> the out MPLS header anyhow.
>> It cannot be absolute: almost every box can process the full packet by kicking it up to the CPU.
>> The performance is just horrible. RLD is what it can do at rate.
> Well. The text you wrote "becoming visible" sounded like a binary thing.

Actually, I didn’t write it.  I believe that Stewart contributed that section.

> And that
> actually does provide some easier to understand behavior than having RLD only be about
> performance. So it might actually be better to consider promising tonot examine stacks
> deeper for NAS than RLD.

There is no such promise, nor are we asking for one.

> Thanks a lot Tony. Such an intermediate document (framework, between reqs & specs) is always difficult.
> Appreciate the effort put into it

Thank you again for your comments.