Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

[[e-mail]]

Hi Sasha,
great discussion, thank you!
Please find my notes below in-line tagged by GIM>>.

Best regards,
Greg Mirsky
Sr. Standardization Expert
预研标准部/有线研究院/有线产品经营部  Standard Preresearch Dept./Wireline Product R&D Institute/Wireline Product Operation Division
E: [e-mail]
www.zte.com.cn
------------------Original Mail------------------
Sender: AlexanderVainshtein
To: Loa Andersson;
CC: [e-mail];Haoyu Song;Jeffrey (Zhaohui) Zhang;
Date: 2021/06/21 03:56
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS
_______________________________________________
mpls mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/mpls

Loa,
Regarding your question “Would you include adding a copy of the GAL higher up in the stack to make sure that it is within readable depth for any LSR?”  my answer is NO.
I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, 
GIM>> I would not refer to existing implementation that supports GAL only as BoS being standards-complaint. I believe that earlier, we've established that the requirement in RFC 5586 "GAL MUST be BoS" applies only and only to an MPLS-TP environment. RFC 55886 and all standards up to date are underspecified for other scenarios. I understand that developers used their best judgment by implementing GAL as strictly BoS, but that doesn't have any foundation in current RFCs. We seem to agree on this characterization of the state of the use of GAL in a non-MPLS-TP environment. Hence, the next step could be closing the gap in the GAL specification. It appears there are two possible paths:
- have a short document extending GAL is only BoS to all MPLS environments;
- allow GAL as ToS while != BoS.
In either case, GAL indicates that a single ACH immediately follows the BoS.

 it will not know how to handle it since such handling has not ever been defined – not in RFC 5586 and not anywhere else. Stewart has responded that “an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable”.  Neither of these options  sounds optimistic to me.
I also do not favor investing into a technique that would guarantee that packets with GAL in the middle of the stack only pass thru new LSRs that know how to handle them .
However, it is quite possible to do the following IMHO:
1.       Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
2.       Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, “we can carry everything in the associated channel”, including TLVs and Sub-TLVs,  if necessary – it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
GIM>> That, to me, is a logical conclusion of #1. I might propose to allocate a new, different from GAL, SPL to signal that the "new ACH immediately follows the BoS". And we might avoid terminology confusion in the future if the new ACH is named differently, e.g., "MPLS Extension Header (MEH)".

3.       Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
GIM>> If we use a new SPL (why not call it MEHI), we can allow MEHI to be not BoS, and thus an LSR can react to its presence in the stack. Consequently, MEHI  can appear in the stack more than once.

4.       Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169) even if they cannot detect presence of GAL at BoS due to the depth of the stack.
I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere.
My 2c,
Sasha
Office: [phone]
Cell:      [phone]
Email:   [e-mail]
-----Original Message-----
From: Loa Andersson <[e-mail]>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[e-mail]>
Cc: [e-mail]; Haoyu Song <[e-mail]>; Jeffrey (Zhaohui) Zhang <zzhang=[e-mail]>; Stewart Bryant <[e-mail]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS
Sasha,
On 21/06/2021 11:55, Alexander Vainshtein wrote:
> Loa and all,
>
> I fully agree with the proposal "to not tamper with ACH anymore".
>
>  From my POV, this includes (by implication) not tampering also with
> GAL as well.
Would you include adding a copy of the GAL higher up in the stack to make sure that it is within readable depth for any LSR?
>
> As for the question " If the slot immediately after the label stack is
> reserved for the ACH does this mean the no other ancillary data may be
> inserted in this position, e.g. MPLS EH's, given that there is a GAL
> in the stack" the answer, IMHO, is YES.
>
> However, it is quite possible to carry any kind of new information in
> the ACH, similar to the way this has been done in Section 3 of RFC
> 8169
> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3>  where G-ACH is used for residence time measurement.
Logically this means that we can carry everything in the associated channel. However there can only one ACH per packet, right?
/Loa
>
> Regards,
>
> Sasha
>
> Office: [phone]
>
> Cell:      [phone]
>
> Email:    [e-mail]
>
> -----Original Message-----
> From: Loa Andersson <[e-mail]>
> Sent: Monday, June 21, 2021 12:40 PM
> To: Haoyu Song <[e-mail]>; Jeffrey (Zhaohui) Zhang
> <zzhang=[e-mail]>; Alexander Vainshtein
> <[e-mail]>; Stewart Bryant
> <[e-mail]>
> Cc: [e-mail]
> Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary
> data after the BoS
>
> Haoyu, DT
>
> On 17/06/2021 18:56, Haoyu Song wrote:
>
>  > My opinion is to not tamper with ACH anymore because it's designed
> for control channel only and so far for a special scenario. The
> constraints on GAL and format of ACH are hard to adapt to the new use
> case requirements.
>
>  >
>
> I think this is a position that is possible to defend.
>
> One question though.
>
> RFC 5586 specifies "that the ACH appears immediately after the bottom
> of the label stack."
>
> If the slot immediately after the label stack is reserved for the ACH
> does this mean the no other ancillary data maybe inserted in this
> position, e.g. MPLS EH's, given that there is a GAL in the stack?
>
> /Loa
>
>  > Thanks!
>
>  > Haoyu
>
>  >
>
>  > -----Original Message-----
>
>  > From: mpls <[e-mail] <mailto:[e-mail]>>
> On Behalf Of Jeffrey (Zhaohui)
>
>  > Zhang
>
>  > Sent: Thursday, June 17, 2021 8:02 AM
>
>  > To: Loa Andersson <[e-mail] <mailto:[e-mail]>>; Alexander
> Vainshtein
>
>  > <[e-mail]
> <mailto:[e-mail]>>; Stewart Bryant
>
>  > <[e-mail] <mailto:[e-mail]>>
>
>  > Cc: [e-mail] <mailto:[e-mail]>
>
>  > Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and
> ancillary
>
>  > data after the BoS
>
>  >
>
>  > Hi Loa,
>
>  >
>
>  >> but I'd like to see the DT address multiple indicators in the
> stack and multiple sets of ancillary data after the BoS.
>
>  >
>
>  > I think the earlier emails of this email thread were talking about
> multiple indicators in the stack; for multiple set of ancillary data
> after the BoS, either the extended ACH or the proposed MPLS/generic
> extension headers or a merge of those proposals should be able to
> handle it. This is alluded to the DataAfterBOS wiki page.
>
>  >
>
>  > Thanks.
>
>  >
>
>  > Jeffrey
>
>  >
>
>  > -----Original Message-----
>
>  > From: Loa Andersson <[e-mail] <mailto:[e-mail]>>
>
>  > Sent: Thursday, June 17, 2021 10:46 AM
>
>  > To: Jeffrey (Zhaohui) Zhang <[e-mail]
> <mailto:[e-mail]>>; Alexander Vainshtein
>
>  > <[e-mail]
> <mailto:[e-mail]>>; Stewart Bryant
>
>  > <[e-mail] <mailto:[e-mail]>>
>
>  > Cc: [e-mail] <mailto:[e-mail]>
>
>  > Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and
> ancillary
>
>  > data after the BoS
>
>  >
>
>  > [External Email. Be cautious of content]
>
>  >
>
>  >
>
>  > DT,
>
>  >
>
>  > Responded to Jeffrey's mail, but it is intended to address the
> entire discussion.
>
>  >
>
>  > There seem to be enough issues to sort out around the GAL/ACH pair,
> and I was worried about a set of other indicators and the data that
> they might want to put "after the BoS". So far I have seen no real
> effort to address the interference's this might lead to.
>
>  >
>
>  > Further inline
>
>  >
>
>  >
>
>  > On 17/06/2021 16:15, Jeffrey (Zhaohui) Zhang wrote:
>
>  >> Hi,
>
>  >>
>
>  >> It's not clear how we could put a GAL not at a BoS:
>
>  >>
>
>  >>
>
>  >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>  >>
>
>  >>      |                              ACH
> |
>
>  >>
>
>  >>
>
>  >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>  >>
>
>  >>      |                         ACH TLV Header
> |
>
>  >>
>
>  >>
>
>  >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>  >>
>
>  >>      |
> ~
>
>  >>
>
>  >>      ~                     zero or more ACH TLVs
> ~
>
>  >>
>
>  >>      ~
> |
>
>  >>
>
>  >>
>
>  >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>  >>
>
>  >>      |
> ~
>
>  >>
>
>  >>      ~                        G-ACh Message
> ~
>
>  >>
>
>  >>      ~
> |
>
>  >>
>
>  >>
>
>  >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>  >>
>
>  >>                         Figure 2: G-ACh Packet Payload
>
>  >>
>
>  >> If the GAL does not have S-bit set, wouldn't a transit LSR treat
> any
>
>  >> 4-ocet field (i.e. those in the above Figure) after that GAL as a
>
>  >> label+TOS+S+TTL? If that 4-octet field has the S-bit set, the
> transit
>
>  >> LSR will think the label stack ends there even though that's just
>
>  >> part of the ACH.
>
>  >>
>
>  >> Or are you saying that a GAL not at the BoS will not have the ACH
>
>  >> following it?
>
>  >
>
>  > Well, as far as I understand a GAL which does not have the NoS-bit
> set will have other labels after itself. The BoS-bit will be found
> deeper down stack and the ACH will immediately fo9llow the BoS.
>
>  >
>
>  > Yes there are issues here, but I'd like to see the DT address
> multiple indicators in the stack and multiple sets of ancillary data
> after the BoS.
>
>  >
>
>  > I think we need to nail down the relevant questiuons first, and
> start working on solutions after that.
>
>  >
>
>  > /Loa
>
>  >>
>
>  >> Jeffrey
>
>  >>
>
>  >> *From:*mpls <[e-mail] <mailto:[e-mail]>>
> *On Behalf Of *Alexander
>
>  >> Vainshtein
>
>  >> *Sent:* Thursday, June 17, 2021 5:07 AM
>
>  >> *To:* Stewart Bryant <[e-mail]
> <mailto:[e-mail]>>
>
>  >> *Cc:* [e-mail] <mailto:[e-mail]>
>
>  >> *Subject:* Re: [mpls] [EXTERNAL] Indicators in the stack and
>
>  >> ancillary data after the BoS
>
>  >>
>
>  >> *[External Email. Be cautious of content]*
>
>  >>
>
>  >> Stewart,
>
>  >>
>
>  >> I fully agree with your statement that "an old implementation that
>
>  >> received a ToS GAL not at BoS would at best throw an exception or
>
>  >> worst be unpredictable".
>
>  >>
>
>  >> Regarding your statement "it is OK to have multiple GALs and GALs
> not
>
>  >> at BoS IFF the creator of the LSP ensured that all LSRs on the
> LSP,
>
>  >> including ECMP and FRR paths that found the GAL at ToS were known
> to
>
>  >> be able to process it correctly":
>
>  >>
>
>  >>   1. I fully agree with this statement as a general restriction  2.
>
>  >> Quite a lot of things have to be done in order to make this
>
>  >>      restriction work including at least:
>
>  >>
>
>  >>       1. The definition of correct processing of GAL at ToS but
> not at
>
>  >>          BoS must be provided
>
>  >>       2. Advertisement of ability to process GAL not at BoS
> correctly in
>
>  >>          IGP and BGP must be defined
>
>  >>       3. Ability to set up network-wide paths that only cross
> nodes that
>
>  >>          process GAL correctly must be provided for different
> techniques
>
>  >>          (RSVP-TE, SR-TE, FlexAlgo. BGP-LU etc.)
>
>  >>
>
>  >> It is still possible that, after all this work, we shall find out
>
>  >> that the benefits of supporting GAL at ToS but not BoS will be
> only
>
>  >> available in the networks where all the nodes support the new
>
>  >> functionality because presence of non-supporting nodes imposes too
>
>  >> many restrictions on connectivity and/or resilience.
>
>  >>
>
>  >> Regards,
>
>  >>
>
>  >> Sasha
>
>  >>
>
>  >> Office: [phone]
>
>  >>
>
>  >> Cell:      [phone]
>
>  >>
>
>  >> Email:  [e-mail]
> <mailto:[e-mail]>
>
>  >> <mailto:[e-mail]
> <mailto:[e-mail]>>
>
>  >>
>
>  >> *From:*Stewart Bryant <[e-mail]
>
>  >> <mailto:[e-mail]
> <mailto:[e-mail]>>>
>
>  >> *Sent:* Thursday, June 17, 2021 10:36 AM
>
>  >> *To:* Alexander Vainshtein <[e-mail]
>
>  >> <mailto:[e-mail]
> <mailto:[e-mail]>>>
>
>  >> *Cc:* Stewart Bryant <[e-mail]
>
>  >> <mailto:[e-mail]
> <mailto:[e-mail]>>>; [e-mail]
> <mailto:[e-mail]>
>
>  >> <mailto:[e-mail]
> <mailto:[e-mail]>>;
> [e-mail] <mailto:[e-mail]>
>
>  >> <mailto:[e-mail] <mailto:[e-mail]>>
>
>  >> *Subject:* Re: [mpls] [EXTERNAL] Indicators in the stack and
>
>  >> ancillary data after the BoS
>
>  >>
>
>  >>      On 17 Jun 2021, at 07:45, Alexander Vainshtein
>
>  >>      <[e-mail]
>
>  >>      <mailto:[e-mail]
> <mailto:[e-mail]>>> wrote:
>
>  >>
>
>  >>      While that might be the case, I think that the Open DT may
> give it a
>
>  >>      try and investigate how the existing systems will handle GAL
> being
>
>  >>      not the BoS label.
>
>  >>
>
>  >>      */[[Sasha]] Great minds think alike! One useful step could be
>
>  >>      collecting the known actual behavior of popular
> implementations in
>
>  >>      this case, say, by running a survey among the vendors - what
> do you
>
>  >>      think?/*
>
>  >>
>
>  >> That is actually a considerable amount of work that will take a while.
>
>  >>
>
>  >> It seems to me that an old implementation that received a ToS GAL
> not
>
>  >> at BoS would at best throw an exception or worst be unpredictable.
>
>  >>
>
>  >> The original assumed processing model is to take the context of
> the
>
>  >> PW label or PW+FAT label, discover the GAL and then process the
> GAL
>
>  >> in the context of the PW label.
>
>  >>
>
>  >> When we extended GAL to apply to LSPs we again had the model that
> the
>
>  >> GAL operated in the context of the LSP label that preceded it for
>
>  >> context. It was still BoS.
>
>  >>
>
>  >> Putting the GAL further up the stack is a new behaviour.
>
>  >>
>
>  >> If it arrives at an LSR that knows the new semantic all is good.
>
>  >>
>
>  >> If it arrives at an LSR that does not know the new semantic then
>
>  >>
>
>  >> a) An error has occurred either in setting up the LSP, or in forwarding.
>
>  >>
>
>  >> b) The behaviour at the receiving node is unpredictable, but in
> any
>
>  >> well written implementation should just result in the packet being
>
>  >> dropped and counted as with any other Mal-formed packet.
>
>  >>
>
>  >> So I would think that it is OK to have multiple GALs and GALs not
> at
>
>  >> BoS IFF the creator of the LSP ensured that all LSRs on the LSP,
>
>  >> including ECMP and FRR paths that found the GAL at ToS were known
> to
>
>  >> be able to process it correctly.
>
>  >>
>
>  >> A GAL not at BoS and not at ToS should not be inspected or
> processed
>
>  >> by any LSR that did not know what it was doing, and to attempt to
>
>  >> precess it would be a violation of the normal MPLS processing model.
>
>  >>
>
>  >> - Stewart
>
>  >>
>
>  >>
>
>  >> Notice: This e-mail together with any attachments may contain
>
>  >> information of Ribbon Communications Inc. and its Affiliates that
> is
>
>  >> confidential and/or proprietary for the sole use of the intended
>
>  >> recipient. Any review, disclosure, reliance or distribution by
> others
>
>  >> or forwarding without express permission is strictly prohibited.
> If
>
>  >> you are not the intended recipient, please notify the sender
>
>  >> immediately and then delete all copies, including any attachments.
>
>  >>
>
>  >>
>
>  >> Juniper Business Use Only
>
>  >>
>
>  >>
>
>  >> _______________________________________________
>
>  >> mpls mailing list
>
>  >> [e-mail] <mailto:[e-mail]>
>
>  >>
>  https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%
> <https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%
> 25>
>
>  >>
> 2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252
>
>  >> F%252Furld
>
>  >>
> efense.com%2Fv3%2F__https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2
>
>  >> F
>
>  >>
> mpls__%3B!!NEt6yMaO-gk!RVgTGVbknjgIjv3x-q8ob1JglFKOP6qKkgAcCSPbeBMMj2
>
>  >> A
>
>  >>
> nexFnPevXopeK1a6u%24&amp;data=04%7C01%7Chsong%40futurewei.com%7Ccc49d
>
>  >> e
>
>  >>
> 9585a24092e29708d931a0e327%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0
>
>  >> %
>
>  >>
> 7C637595389337881384%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
>
>  >> I
>
>  >>
> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=5et4Juc3Ij
>
>  >> G
>
>  >> dfux%2FR5MsJnuTYDWL6S4pZ8uz3F6h34Q%3D&amp;reserved=0
>
>  >>
>
>  >
>
>  > --
>
>  >
>
>  > Loa Andersson                        email:  [e-mail]
> <mailto:[e-mail]>
>
>  > Senior MPLS Expert  [e-mail] <mailto:[e-mail]>
>
>  > Bronze Dragon Consulting             phone: [phone]
>
>  >
>
>  > Juniper Business Use Only
>
>  > _______________________________________________
>
>  > mpls mailing list
>
>  > [e-mail] <mailto:[e-mail]>
>
>  >
>  https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2
> <https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%
> 252>
>
>  >
> F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%
>
>  >
> 252Fwww.ietf.org%252Fmailman%252Flistinfo%252Fmpls%26data%3D04%257C01%
>
>  >
> 257Chsong%2540futurewei.com%257Ccc49de9585a24092e29708d931a0e327%257C0
>
>  >
> fee8ff2a3b240189c753a1d5591fedc%257C1%257C0%257C637595389337881384%257
>
>  >
> CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
>
>  >
> k1haWwiLCJXVCI6Mn0%253D%257C1000%26sdata%3DXQlRpwkgODLRxcIjyMYyPMiCF2K
>
>  > DC0Y7GG4O8VGESnw%253D%26reserved%3D0
>
>  >
>
> --
>
> Loa Andersson                        email:  [e-mail]
> <mailto:[e-mail]>
>
> Senior MPLS Expert  [e-mail] <mailto:[e-mail]>
>
> Bronze Dragon Consulting             phone: [phone]
>
>
> Notice: This e-mail together with any attachments may contain
> information of Ribbon Communications Inc. and its Affiliates that is
> confidential and/or proprietary for the sole use of the intended
> recipient. Any review, disclosure, reliance or distribution by others
> or forwarding without express permission is strictly prohibited. If
> you are not the intended recipient, please notify the sender
> immediately and then delete all copies, including any attachments.
--
Loa Andersson                        email:  [e-mail]
Senior MPLS Expert                           [e-mail]
Bronze Dragon Consulting             phone: [phone]
Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. and its Affiliates that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.