Re: [Idr] Fwd: [E] New Version Notification for draft-mishra-idr-v4-islands-v6-core-4pe-05.txt

[Igor Malyushkin <[e-mail]>]

Hi Gyan,

I'm OK with this list.
Thank you.

пт, 11 авг. 2023 г. в 18:57, Gyan Mishra <[e-mail]>:

> Hi Igor / all
>
> Here is a summary of 4PE draft updates.
>
> Please review and any comments  much appreciated!
>
>  4PE design Deployment methods:
>
> All of what’s discussed use cases are  applicable to intra-as and inter-as
> options a, b, c, ab with Data planes MPLS, SR-MPLS, SRv6.
>
> 1. Arbitrary and Explicit null value 2 label topmost ipv6 LSP each with 3
> use cases: (6 use cases)
>
> a. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC 8277
> 1/4 service label labeling all IPv4 customer prefixes.
>
> b LERs signal IPv6 topmost LSP with 2 level label stack, BOS set RFC 8277
> 1/4 service label using ingress to egress  PE loopback to loopback LSP
> single BOS label with all global table customer prefixes unlabeled.
>
> c. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC 8277
> 1/4 service label using per CE label table routing context LSP ingress to
> egress  CE PE-CE interface PE side interface LSP single BOS label with per
> CE label table customer prefixes unlabeled.
>
> 2. IPv6 LSP PHP implicit null value 3  with 3 use cases: (3 use cases)
>
> a. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC 8277
> 1/4 service label labeling all IPv4 customer prefixes.
>
> b LERs signal IPv6 topmost LSP with 2 level label stack, BOS set RFC 8277
> 1/4 service label using ingress to egress  PE loopback to loopback LSP
> single BOS label with all global table customer prefixes unlabeled.
>
> c. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC 8277
> 1/4 service label using per CE label table routing context LSP ingress to
> egress  CE PE-CE interface PE side interface LSP single BOS label with per
> CE label table customer prefixes unlabeled.
>
> 3. Arbitrary and explicit null value 2 topmost IPv6 LSP BOS set single
> level label stack with all global table customer prefixes 1/1 unlabeled.
> Caveat that this requires DPI to determine next header inspection for
> protocol type so packets are not dropped. (2 use cases)
>
> Total of 11 use cases.
>
> Thanks
>
> Gyan
>
>
>
> On Tue, Aug 8, 2023 at 9:02 PM Gyan Mishra <[e-mail]> wrote:
>
>> Hi Igor
>>
>> It was a pleasure having this detailed discussion.
>>
>> I am all set!
>>
>> Look out for an updated draft soon.
>>
>> Many Thanks!!
>>
>> Gyan
>>
>> On Mon, Aug 7, 2023 at 4:47 AM Igor Malyushkin <[e-mail]>
>> wrote:
>>
>>> Hi Gayn.
>>>
>>> Agree, we have reached a consensus. I commented on all inline (IM4).
>>>
>>> Thank you for the discussion!
>>>
>>> вс, 6 авг. 2023 г. в 00:37, Gyan Mishra <[e-mail]>:
>>>
>>>>
>>>> Hi Igor
>>>>
>>>> My comments in-line Gyan3>
>>>>
>>>> This last thread really helped clear up the use case of arbitrary
>>>> labels which comes into play if any implementation desired to not signal
>>>> implicit null value 3 by default.  Implicit null default is not a
>>>> requirement so we have to account for both use case possibilities.
>>>>
>>>> We are pretty well in sync!!
>>>>
>>>> On Sat, Aug 5, 2023 at 1:11 PM Igor Malyushkin <[e-mail]>
>>>> wrote:
>>>>
>>>>> Hi Gyan,
>>>>>
>>>>> My comments are inline with the IM3 prefix.
>>>>>
>>>>> сб, 5 авг. 2023 г. в 19:17, Gyan Mishra <[e-mail]>:
>>>>>
>>>>>> Hi Igor
>>>>>>
>>>>>> Responses in-line Gyan2>
>>>>>>
>>>>>> On Thu, Aug 3, 2023 at 2:35 PM Igor Malyushkin <[e-mail]>
>>>>>> wrote:
>>>>>>
>>>>>>> My comments are inline (IM2).
>>>>>>>
>>>>>>> чт, 3 авг. 2023 г. в 18:54, Gyan Mishra <[e-mail]>:
>>>>>>>
>>>>>>>> Hi Igor
>>>>>>>>
>>>>>>>> Yes looks like we have reached a consensus!!
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Aug 3, 2023 at 8:43 AM Igor Malyushkin <
>>>>>>>> [e-mail]> wrote:
>>>>>>>>
>>>>>>>>> Hi Gayn, looks like we have reached a consensus!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> чт, 3 авг. 2023 г. в 09:03, Gyan Mishra <[e-mail]>:
>>>>>>>>>
>>>>>>>>>> Hi Igor
>>>>>>>>>>
>>>>>>>>>> I am good with everything listed below.
>>>>>>>>>>
>>>>>>>>>> As far as the topmost explicit versus arbitrary label I agree
>>>>>>>>>> that based on implementation if you use arbitrary you are ok if 1/4 service
>>>>>>>>>> label is used
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IM> Ok.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> however if you use arbitrary topmost label and use 1/1 for
>>>>>>>>>> service label at the PHP node the native IPv4 packet could be dropped.
>>>>>>>>>> There would be a warning for that particular use case.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IM> Yes. But I would say not at the PHP node. The PHP node just
>>>>>>>>> swaps a label in the case of arbitrary labels. Dropping occurs at the
>>>>>>>>> ultimate node (egress PE) due to any label for IPv6 LSPs (an arbitrary one
>>>>>>>>> or explicit 2) here can be allocated for the IPv6 routing process, but a
>>>>>>>>> packet is an IPv4 one.
>>>>>>>>>
>>>>>>>>
>>>>>>>>     Gyan> My understanding here is that the PHP (Penultimate Hop
>>>>>>>> POP) node is not the Ultimate hop which is the egress LER (PE) but is one
>>>>>>>> hop prior to the egress LER which would be the egress LSR (P) node.
>>>>>>>>
>>>>>>> [IM2] Yes, a PHP node is not an egress LER (PE). I've never heard
>>>>>>> someone calls a penultimate node an egress LSR, but ok, I will follow for
>>>>>>> this discussion.
>>>>>>>
>>>>>>>
>>>>>>>> So when the topmost label is an arbitrary label by default the PHP
>>>>>>>> node egress LSR (P) when it does a label swap the incoming label is
>>>>>>>> arbitrary but the outgoing label is “POP” which signals implicit null value
>>>>>>>> 3.
>>>>>>>>
>>>>>>> [IM2] No. Let me give you an example. Egress LER X allocates the
>>>>>>> label of 3000 and puts it in its ILM table with a POP operation. Then it
>>>>>>> advertises a mapping of its loopback's IPv6 address plus the label (3000)
>>>>>>> via LDPv6. That means that the egress LER expects MPLS traffic with the
>>>>>>> label of 3000 on its LDP-enabled interfaces and will strip this label by
>>>>>>> itself. All its neighbors would either swap something to 3000 or push 3000.
>>>>>>> The former case is when these neighbors are LSRs, the latter is when they
>>>>>>> are iLERs. In other words, we have a continuous single-labeled LSP from
>>>>>>> ingress LER to egress LER. All LSRs just do swapping of labels, but not
>>>>>>> popping.
>>>>>>>
>>>>>>
>>>>>>    Gyan2> Thanks for the clarification on arbitrary label use case
>>>>>> which is why the arbitrary label comes into play. So it appear this is a
>>>>>> vendor implementation difference.  I think this helps shed some light on
>>>>>> the arbitrary label case that you made and helps me understand the use case.
>>>>>>
>>>>> [IM3] Good! But I don't see where the vendor implementation difference
>>>>> comes to play. To me, it is standard behavior for MPLS.
>>>>>
>>>>>>
>>>>>> Since Nokia does not implement implicit null by default, PHP is
>>>>>> disabled, so then the  topmost label then can be any arbitrary label and
>>>>>> the topmost label is now preserved to the egress LER which then pops the
>>>>>> entire stack.
>>>>>>
>>>>> [IM3] I want you to note that I've never said that Nokia does not
>>>>> implement an implicit null. I don't want problems with Nokia fellas :) It
>>>>> is turned off by default, yes, but you can easily turn it back. The rest is
>>>>> correct.
>>>>>
>>>>
>>>>      Gyan3> Ack we are in sync here.  Just for clarity I think both of
>>>> us did not say that Nokia did not implement implicit null. My main point is
>>>> that RFC 3031 does not make implicit null a MUST for implementations by
>>>> default as long as implicit null is supported.  So Nokia supports
>>>> configuration of implicit null, however it is not default and leads to a
>>>> valid corner case with arbitrary labels use case where PHP is disabled.
>>>> PHP disabled default behavior as well provides an alternative to orthogonal
>>>> point that RFC 3270 MPLS QOS Pipe mode using explicit null is not necessary
>>>> as now you can use arbitrary topmost labels which are now preserved and do
>>>> PHB MPLS QOS EXP scheduling is not broken. So that based on implementations
>>>> over the years makes disabling PHP by default by not signaling implicit
>>>> null possibilities as not a corner case at all as there are many advantages
>>>> of doing so as we have seen with this thread. Historically PHP existed to
>>>> not overburden the LER to pop the entire stack offloading the load to the
>>>> egress LSR to POP the topmost label (PHP).  As well two flavors of
>>>> implementation to disable PHP, Cisco / Juniper method of enabling explicit
>>>> null and Nokia method of disabling implicit null by default.  I would like
>>>> to point out that the MPLS QOS EXP scheduling reason for explicit null or
>>>> not configuring implicit null is vendor implementation of being able to PHB
>>>> schedule MPLS QOS EXP and simultaneously with same policy schedule on DSCP
>>>> allows QOS PHB scheduling to not be broken which both Cisco and Juniper
>>>> implements, allowing operators to keep the default PHP behavior and not
>>>> have to signal explicit null Pipe mode to fix the scheduling issue.  So
>>>> there is a quite a number of permutations of designs based on use case.
>>>>
>>> [IM4] Ok, that point's clear.
>>>
>>>>
>>>>>> RFC 3031 section 4.1.5 describes the MPLS requirement to support
>>>>>> implicit null value 3 signaling done via label mapping message from egress
>>>>>> LER to PHP node (egress LSR) to POP the topmost label.  The MPLS standard
>>>>>> does not specify that implicit null MUST be the default behavior, however
>>>>>> most vendors have implemented as Default behavior.
>>>>>>
>>>>>
>>>>>> Cisco and Juniper use implicit null value 3 signaled by LER to PHP
>>>>>> node LSR (egress P) to pop the topmost outer label.  With Nokia implicit
>>>>>> null is not default and has to be enabled manually see links below:
>>>>>>
>>>>> [IM3] See my point above.
>>>>>
>>>>
>>>>      Gyan3> Ack
>>>>
>>>>>
>>>>>> Cisco supports implicit null signaling for PHP by default link below.
>>>>>>
>>>>>>
>>>>>> https://community.cisco.com/t5/mpls/implicit-null-and-explicit-null/td-p/1496792
>>>>>>
>>>>>>
>>>>>> Juniper supports implicit null signaling by default link below.
>>>>>>
>>>>>>
>>>>>> https://www.juniper.net/documentation/us/en/software/junos/mpls/topics/ref/statement/explicit-null-edit-protocols-mpls.html
>>>>>>
>>>>>>
>>>>>> Nokia does NOT support implicit null signaling by default link
>>>>>> below.  Since implicit null is a configured option it’s not enabled by
>>>>>> default to signal PHP.  Given that is the case AFAIK, any arbitrary label
>>>>>> can be preserved and thus you don’t have to use the explicit null label to
>>>>>> preserve the topmost label.  All you need to do is disable PHP which is
>>>>>> done by default my not signaling implicit null.  Cisco and Juniper as
>>>>>> implicit null is default behavior PHP is signaled, however in order to
>>>>>> disable PHP explicit null has to be configured.  So then the arbitrary
>>>>>> label concept only comes into play with vendors that don’t support PHP by
>>>>>> default like Nokia.
>>>>>>
>>>>>>
>>>>>> https://infocenter.nokia.com/public/7750SR217R1A/index.jsp?topic=%2Fcom.nokia.MPLS_Guide_21.7.R1%2Fconfiguring_imp-ai9emdyowp.html
>>>>>>
>>>>>> Configuring Implicit Null Label
>>>>>>
>>>>>> The implicit null label option allows an egress LER to receive MPLS
>>>>>> packets from the previous hop without the outer LSP label. The user can
>>>>>> configure to signal the implicit operation of the previous hop is referred
>>>>>> to as penultimate hop popping (PHP). This option is signaled by the egress
>>>>>> LER to the previous hop during the FEC signaling by the LDP control
>>>>>> protocol.
>>>>>>
>>>>>> Enable the use of the implicit null option, for all LDP FECs for
>>>>>> which this node is the egress LER, using the following command:
>>>>>>
>>>>>> config>router>ldp>implicit-null-label
>>>>>>
>>>>>> When the user changes the implicit null configuration option, LDP
>>>>>> withdraws all the FECs and re-advertises them using the new label value.
>>>>>>
>>>>>>>
>>>>>>> As I previously said, Nokia does it by default for RSVP, LDP, and
>>>>>>> BGP LSP. Huawei has a configuration option at least for LDP to turn this
>>>>>>> feature on. From time to time there are bugs with handling labels 0,2 or 3
>>>>>>> among different vendors. I had a very unpleasant experience with this too
>>>>>>> before. Arbitrary labels allow us to fool the penultimate nodes and make
>>>>>>> them regular LSRs. It also could work with egress LERs when they due to
>>>>>>> a bug cannot handle either unlabeled traffic or traffic with explicit
>>>>>>> labels. Personally, I consider PHP and UHP with explicit labels as
>>>>>>> rudiments and prefer using arbitrary labels instead.
>>>>>>>
>>>>>>
>>>>>>    Gyan2> See above what I mentioned about Nokia and PHP signaling
>>>>>> via implicit null appears to not be enabled by default.
>>>>>>
>>>>> I have tested 0, 2, 3 with Cisco and Juniper and I have not run into
>>>>>> any issues.  Based on what I stated above I agree the arbitrary label
>>>>>> concept I believe only comes into play with vendors such as Nokia that do
>>>>>> not have implicit null signaling to trigger the PHP node to POP the topmost
>>>>>> transport label.
>>>>>>
>>>>> [IM3] I don't get the idea of the last sentence. Arbitrary labels (if
>>>>> they are supported by a vendor in question) come into play when a network
>>>>> engineer decides to turn them on. Their usage is a matter of configuration
>>>>> and a design solution.
>>>>>
>>>>>
>>>>      Gyan3> In the last sentence I forgot to say by “default” vendors
>>>> such as Nokia that enable implicit null signaling for PHP to be enabled by
>>>> default.  The main reason to use arbitrary labels is for UHP to preserve
>>>> the topmost label to the egress LER.  Correct?
>>>>
>>> [IM4] Yes, that's correct.
>>>
>>>
>>>> It’s the exact same reason to use explicit null signaling to preserve
>>>> the topmost label so it’s not popped by the default implicit null PHP
>>>> enabled mode.  Cisco / Juniper both support the latter scenario where
>>>> implicit null is not configurable and cannot be explicitly enabled or
>>>> disabled, however in order to switch from implicit null PHP mode to
>>>> explicit null UHP mode you have to explicitly configure explicit null.  So
>>>> for Cisco and Juniper, arbitrary labels are not possible as implicit null
>>>> enable disable state from a configurable perspective does not exist as
>>>> implicit null is enabled by default. Nokia for example does not support
>>>> implicit null by default and has a configuration command to enable implicit
>>>> null and thus by default arbitrary labels use case is possible.  That is
>>>> why I say arbitrary labels usage and use case is based on vendor
>>>> implementation for the designer to use.  Of course of its not implemented
>>>> like with Cisco and Juniper for example it cannot be used.
>>>>
>>> [IM4] I got the main point of your statement above, that the use of
>>> arbitrary labels is based on their support by a vendor in question. Other
>>> vendors use explicit labels to achieve the same. I agree and always
>>> emphasized the same. If I missed anything, please, let me know.
>>>
>>>>
>>>>
>>>> Since PHP is disabled on Nokia by default you can now fool the PHP node
>>>>>> making it act as a regular LSR.  You are not actually fooling the PHP node,
>>>>>> it’s just that implicit null is not signaled by default so PHP is disabled,
>>>>>> so when LDP distributes the labels via downstream unsolicited from
>>>>>> downstream to upstream nodes and the LSP is instantiated and LIB is
>>>>>> allocated and LFIB data plane forwarding is programmed, the LSP is standard
>>>>>> swapping on every LSR including the PHP nodes as it did not get signaled
>>>>>> for implicit null to POP the transport label so acts as normal LSR swapping
>>>>>> labels.  No fooling and is acting as it should with arbitrary labels.
>>>>>>
>>>>> [IM3] Well, I can not use this term if you don't like it :) By
>>>>> fooling, I meant exactly what you stated above. So, looks like we are in
>>>>> sync here.
>>>>>
>>>>
>>>>      Gyan> Excellent! We are in sync here!
>>>>
>>>>>
>>>>>>>  So once the  PHP node egress LSR does the “POP” processing the
>>>>>>> outer topmost  label in the label stack is stripped or removed leaving the
>>>>>>> exposed payload which if a 2 level label stack would now be a single level
>>>>>>> label stack 1/4 and the labeled packet is successfully forwarded to the
>>>>>>> egress LER (PE) that POPs the remaining labels in the stack in this case
>>>>>>> the BOS bit set service label and the native IPv4 packet is forwarded to
>>>>>>> IPv4 context PE-CE interface via standard forwarding.
>>>>>>> [IM2] That's correct.
>>>>>>>
>>>>>>> So if it’s a single level label stack with topmost IPv6 arbitrary
>>>>>>> label once the PHP processing is completed “POP” operation stripping or
>>>>>>> removing outer topmost IPv6 arbitrary label the IPv4 payload is now exposed
>>>>>>> in the case of 1/1 service and the unlabeled IPv4 packet is dropped by the
>>>>>>> PHP node as it cannot be forwarded onto the egress IPv6 only  core
>>>>>>> interface facing the egress LER (PE). That’s the problem that we would
>>>>>>> address in this specification with 4PE.
>>>>>>> [IM2] That's not.
>>>>>>>
>>>>>>
>>>>>>     Gyan2> Updated arbitrary label use case.  So here  we not
>>>>>> signaling PHP with implicit null as some vendors do not signal implicit
>>>>>> null value 3 by default so PHP is disabled.  So now the topmost arbitrary
>>>>>> label is preserved to the egress LER which pops the entire stack and
>>>>>> forwards the native IPv4 packet to IPv4 routing context.
>>>>>>
>>>>> [IM2] In general, yes, except I'm not sure an IPv4 packet is forwarded
>>>>> to the IPv4 routing context for every implementation. From my
>>>>> understanding, this is vendor-depended behavior. If there is a normative
>>>>> reference to this procedure, please, let me know!
>>>>>
>>>>>>
>>>>>>>> If the egress LER signaled explicit null pipe mode to the egress
>>>>>>>> LSR PHP node, the PHP node seeing the value 2 IPv6 explicit null label
>>>>>>>> being signaled and would preserve the topmost IPv6 label in pipe mode and
>>>>>>>> forward the packet now still with topmost IPv6 label intact preserved over
>>>>>>>> the IPv6 core interface to the egress LER (PE) which would receive the
>>>>>>>> single level label stack and now pop the remaining label in this case the
>>>>>>>> explicit null value 2 topmost label and forward the native IPv4 packet to
>>>>>>>> the outgoing PE-CE ipv4 routing context.
>>>>>>>>
>>>>>>> [IM2] No, it does not forward an IPv4 packet after the stripping of
>>>>>>> the label of 2. I've just checked it on a Juniper, it drops all such
>>>>>>> traffic until we turn off the explicit label (I tried with LDPv6 and family
>>>>>>> inet without an address for the incoming interface). Nokia does the
>>>>>>> additional packet inspection for arbitrary labels that belong to IPv6 LSPs
>>>>>>> and forwards IPv4 packets. I cannot check it with an explicit label, looks
>>>>>>> like it is not supported or I failed to find it.
>>>>>>>
>>>>>>
>>>>>>     Gyan2> I believe IPv6 explicit null label as topmost once popped
>>>>>> cannot determine the network layer protocol of the unlabeled IPv4
>>>>>> underneath and so drops the packet.
>>>>>>
>>>>>
>>>>>
>>>>>> But it appear from your test that when using topmost arbitrary label
>>>>>> it can determine the network layer protocol type.
>>>>>>
>>>>> [IM3] Yes, but we should use this information very cautiously. We
>>>>> don't know the internals of one or another implementation. I'll repeat
>>>>> myself: some vendors do additional packet inspection, while others do not.
>>>>>
>>>>
>>>>      Gyan3>  As this use case digs deep into vendor implementation
>>>> internals I think it’s fair to exclude 1/1 unlabeled use case under the
>>>> topmost both arbitrary or explicit null cases due to the variation in deep
>>>> packet inspection being supported or not by vendors.
>>>>
>>> [IM4] I think that the draft should do exactly the opposite. The
>>> excluding of this case creates some undefined behavior. It is better to
>>> describe this case with the known pitfalls in detail and warn readers.
>>>
>>>
>>>> Based on this we are in sync for 4PE!
>>>>
>>>> 1/1 unlabeled is possible but due variance in support or not from 4PE
>>>> specification perspective we would have the following use cases:
>>>>
>>>> 3 use cases can have topmost be arbitrary or explicit null.
>>>>
>>>>   1st - All CE global table prefixes are labeled
>>>>   2nd - PE loopback to PE loopback LSP BOS set that contains all the CE
>>>> global table prefixes unlabeled
>>>>   3rd - Per CE PE-CE interface label table BOS set  IPv4 routing
>>>> context that contains the respective CE prefixes unlabeled
>>>>
>>>
>>>> 3 use cases of PHP implicit null - have you tested this ?  This should
>>>> work
>>>>
>>>>  1st - All CE global table prefixes are labeled
>>>>   2nd - PE loopback to PE loopback LSP BOS set that contains all the CE
>>>> global table prefixes unlabeled
>>>>   3rd - Per CE PE-CE interface label table BOS set  IPv4 routing
>>>> context that contains the respective CE prefixes unlabeled
>>>>
>>> [IM4] I'm OK with all the above. I've tested PHP for underlying LSPs
>>> when there is an overlay label at the bottom, and it works.
>>>
>>> For the arbitrary or explicit null 1/1  unlabeled IPv4 prefixes case
>>>> confirming we will not include due to the deep packet inspection variable
>>>> or maybe could mention with the implementation requirement of deep packet
>>>> inspection to infer the network protocol type?
>>>>
>>> [IM4] If it is possible to describe these cases as less preferred with a
>>> detailed explanation of why, I'm OK with this option. Talking about the
>>> requirement for DPI, I'm not sure about it. To me, it is too uncertain,
>>> there should be a procedure for such inspection. It is possible to mention,
>>> that vendors can analyze a network layer header and consider forwarding
>>> traffic.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> RFC 3032 section 2.2 talks about determining the network layer
>>>>>> protocol type of the different protocol type exposed underneath topmost
>>>>>>  once the topmost is popped and it says that the network layer protocol
>>>>>> type is inferred from the value of label at bottom of stack. I think since
>>>>>> explicit null is a special label it cannot infer the protocol type but when
>>>>>> using arbitrary it can as proved from your test.
>>>>>>
>>>>> [IM3] No, that is not what I proved. Please, be careful here. Let me
>>>>> explain my point broader. From RFC 3032 (section 2.2) and specifically
>>>>> mentioned by you RFC 4182 (section 2), we can understand that the explicit
>>>>> labels, if and only if they have a BoS set, point exactly to the network
>>>>> protocol they belong to.
>>>>>
>>>>
>>>>> RFC4182 (cases for labels with BoS):
>>>>>
>>>>>        If, on the other hand, the NULL was the only label on the
>>>>>           stack, then the stack is now empty.  The resulting packet is
>>>>>           treated as an IPv4 packet, and its disposition is based on the
>>>>>           IPv4 header.
>>>>>
>>>>>
>>>>>           If, on the other hand, the NULL was the only label on the
>>>>>           stack, then the stack is now empty.  The resulting packet is
>>>>>           treated as an IPv6 packet, and its disposition is based on the
>>>>>           IPv6 header.
>>>>>
>>>>>
>>>>
>>>>     Gyan3> So for IPv6 explicit null it’s expecting an IPv6 packet and
>>>> not IPv4 so the packet would be dropped unless the implementation supports
>>>> deep packet inspection to infer the protocol type underneath.  What you
>>>> quoted from RFC 4182 section 2 is it says null is the only label in the
>>>> label stack so then of course no label with BOS.
>>>>
>>> [IM4] Not exactly. When there is only one label in the stack, this label
>>> must come with BoS. Also, from my understanding, this section talks not
>>> about the exact number of labels for an LSP, but about the parsing process
>>> when a parser eventually has the last label from the stack. I think it does
>>> not matter how many labels there are prior to an explicit null (0 or 2) if
>>> the latter is at the bottom. These labels (except the mentioned explicit
>>> one) will be stripped anyway. So, at the end of the day, a LER has only one
>>> label and this label is of explicit type with a BoS set.
>>>
>>>
>>>> This is the 1/1 use case unlabeled IPv4 packets so no BOS would be
>>>> present since it’s a single level label stack. Please clarify.
>>>>
>>>  [IM4] Yes, the use case is correct, but BoS is present even if this is
>>> a single-level label stack. BoS notifies a system to stop considering the
>>> next bytes after an LSE as MPLS header-related and start processing a
>>> network layer header.
>>>
>>>
>>>>> So, I expect an implementation to drop any IPv4 packets above the
>>>>> label of 2 and any IPv6 packets above the label of 0 according to these
>>>>> RFCs as the default behavior. However, according to the RFC3032, section
>>>>> 2.2:
>>>>>
>>>>
>>>>     Gyan3>. This is for the 1/1 not 1/4 (BOS) set with explicit null
>>>> topmost label so I would expect the IPv4 packets to be dropped as that is
>>>> based on the vendor implementation of deep packet inspection or not.
>>>>
>>> [IM4] Yes, agree. This is for 1/1 and, depending on a vendor DPI's
>>> implementation, packets may be dropped or not.
>>>
>>>
>>>> This is the 1/1 use case so no labeled IPv4 prefixes but you are
>>>> mentioning >=2 . We are talking 4PE which is IPv4 packets over IPv6 core
>>>> not 6PE which is IPv6 packets over IPv4 core, so in 6PE case as well the
>>>> IPv6 packets would be dropped.
>>>>
>>> [IM4] Yes, that's exactly what I tried to state. No matter what type of
>>> explicit label we have, if we do have not the expected network layer
>>> header, traffic may be dropped. This is true both for 4PE and 6PE when the
>>> explicit label is at the bottom of the stack.
>>>
>>>
>>>> So in both 4PE or 6PE unless the implementation supports deep packet
>>>> inspection the 1/1 unlabeled packets would be dropped.
>>>>
>>> [IM4] Yes.
>>>
>>>
>>>>
>>>>
>>> This means that the identity of the network layer protocol
>>>>>    must be inferable from the value of the label which is popped from
>>>>>    the bottom of the stack, possibly along with the contents of the
>>>>>    network layer header itself.
>>>>>
>>>>>
>>>>> We see that also vendors *can *analyze a network layer header, but it
>>>>> is complementary rather than the general behavior (see *possibly*).
>>>>> As I understand it, that is exactly what I have seen with the arbitrary
>>>>> label on Nokia. From my POV, it is also true for the explicit labels. But I
>>>>> think there is no guarantee, that each vendor adheres to this.
>>>>>
>>>>
>>>>    Gyan4> Understood on the deep packet inspection and that it’s
>>>> complementary optional but not generally implemented.  Agreed.  The thread
>>>> above was a bit confusing see above that we are talking about 1/1 not 1/4
>>>> so it’s unlabeled IPv4 prefixes with arbitrary or explicit topmost label.
>>>>
>>> [IM4] Yes, we discussed the case with unlabeled IPv4 traffic, because,
>>> from my POV, with labeled IPv4 traffic, we were already in sync.
>>>
>>>
>>>>
>>>>>   So I can mention this as a valid use case in the draft which is good
>>>>>> reason for arbitrary label.  So IPv6 explicit null is valid and not illegal
>>>>>> with the RFC 4182 update in 2005 to be used for any level of label stack
>>>>>> including topmost, however for case where the network layer under the
>>>>>> topmost is unlabeled you have to use arbitrary label for the IPv4 packet to
>>>>>> be inferred and not dropped.
>>>>>>
>>>>> [IM3] I will highlight it here too for additional clearance. Neither
>>>>> arbitrary nor explicit labels guarantee us that the IPv4 payload will be
>>>>> forwarded. That is why I think, the draft should clearly warn readers.
>>>>>
>>>>
>>>>      Gyan3> Ack and just to be crystal clear again this is 1/1 use case
>>>> for arbitrary or explicit null topmost.
>>>>
>>> [IM4] Yes!
>>>
>>>>
>>>>
>>>>>
>>>>>>> [IM] For example, RFC3032 says:
>>>>>>>
>>>>>>> A value of 2 represents the "IPv6 Explicit NULL Label".
>>>>>>>               This label value is only legal at the bottom of the label
>>>>>>>               stack.  It indicates that the label stack must be popped,
>>>>>>>               and the forwarding of the packet must then be based on the
>>>>>>>               IPv6 header.
>>>>>>>
>>>>>>>
>>>>>>> Despite I believe that requirement of the bottomness is kinda
>>>>>>> relaxed these days, anyway, it indicates that after the label of 2, there
>>>>>>> must be the IPv6 header, not IPv4. So, as I said, some boxes expect only
>>>>>>> IPv6 traffic and drop everything else if there is a label of 2 with a BoS
>>>>>>> set. Others could check packets deeper and forward them.
>>>>>>>
>>>>>>>>
>>>>>>>> No IPv4 dropped packets with IPv6 explicit null signaling value 2
>>>>>>>> but with arbitrary label we drop all IPv4 packets on the PHP node after the
>>>>>>>> “POP” operation.
>>>>>>>>
>>>>>>> [IM] As I explained above, in both cases -- no.
>>>>>>>
>>>>>>>
>>>>>>>> The egress node can do additional packet inspection and avoid the
>>>>>>>>> dropping, but there is no guarantee. This warning should be described in
>>>>>>>>> the document alongside the warning of the PHP case for single-labeled
>>>>>>>>> traffic.
>>>>>>>>> Also, if this document tried to invent and describe procedures on
>>>>>>>>> how to not drop IPv4 unlabeled traffic coming from IPv6 LSPs, I would say,
>>>>>>>>> that deserves to be a Standard :)
>>>>>>>>>
>>>>>>>>> Summary:
>>>>>>>>> * For single-labeled stacks:
>>>>>>>>> * * If the topmost label is an explicit one (value 2) or an
>>>>>>>>> arbitrary one, a drop is possible at an egress PE. The PE could think that
>>>>>>>>> a packet behind the label is IPv6, if there is no additional packet
>>>>>>>>> inspection, the drop will occur.
>>>>>>>>> * * In the PHP case a drop is possible at the penultimate PE when
>>>>>>>>> the topmost label is stripped of the payload. The PE could check the
>>>>>>>>> outgoing interface's families and does not allow IPv4 packets there.
>>>>>>>>> * For two or more labeled stacks:
>>>>>>>>> * * A bottom label must be present and be either an arbitrary one
>>>>>>>>> or explicit (0).
>>>>>>>>> * * Any labels above can be present in a stack with any values or
>>>>>>>>> cannot be present at all.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> The use case of topmost explicit null or implicit null default
>>>>>>>>>> will both work with PE loopback LSP or per CE PE-CE interface LSP context
>>>>>>>>>> label table.  We will allow topmost label PHP with no restriction as this
>>>>>>>>>> is possible with the permutation of use cases using 1/4 for service label
>>>>>>>>>> labeling all prefixes one bucket for all global table prefixes,  as well ws
>>>>>>>>>> works for your use case PE loopback LSP or per CE PE-CE interface LSP
>>>>>>>>>> carrying the per CE unlabeled CE prefixes.  As long as the service prefixes
>>>>>>>>>> are labeled allowing PHP for topmost label is valid permutation without any
>>>>>>>>>> risk of payload traffic drops.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [IM] Yes, agree.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Let me know if you are good with what I have stated.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [IM] I'm good with that, yes.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> As far as standards track I will discuss with the
>>>>>>>>>> co-authors.
>>>>>>>>>>
>>>>>>>>>> I will discuss if in shipping code what is implemented in detail
>>>>>>>>>> and with what is described below if there is anything that requires
>>>>>>>>>> standardization even though existing mechanisms are being used that if any
>>>>>>>>>> procedural or any differences could cause interoperability issues.  One of
>>>>>>>>>> the major goal of the IETF standardization is mix vendor interoperability.
>>>>>>>>>> So if this can be achieved as informational draft or based on
>>>>>>>>>> interoperability issues found with current or future vendor implementation
>>>>>>>>>> do we need to have a standards track draft or not to meet the goal.
>>>>>>>>>> Normative language only comes into play for standards track draft so since
>>>>>>>>>> we do have normative language RFC 2119 does that mean this document should
>>>>>>>>>> be standards track.
>>>>>>>>>>
>>>>>>>>>> I will discuss this topic with the chairs as well.
>>>>>>>>>>
>>>>>>>>>> Many Thanks!
>>>>>>>>>>
>>>>>>>>>> Gyan
>>>>>>>>>>
>>>>>>>>>> On Wed, Aug 2, 2023 at 6:32 AM Igor Malyushkin <
>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Gyan.
>>>>>>>>>>>
>>>>>>>>>>> When I'm talking about transport, I mean some underlay, i.e.,
>>>>>>>>>>> LSPs that points to remote PEs. They could be RSVP/LDP/SR-MPLS in a single
>>>>>>>>>>> domain or the exact same options with BGP LSP (IPv6) as E2E LSP through
>>>>>>>>>>> several domains. This draft should not restrict any label operations in any
>>>>>>>>>>> way for this set. That is what I meant for PHP. With PHP for these LSPs, we
>>>>>>>>>>> can face the situation when the IPv4 payload is exposed at the penultimate
>>>>>>>>>>> node.
>>>>>>>>>>>
>>>>>>>>>>> To avoid the problem with the exposure of the payload this draft
>>>>>>>>>>> offers to use the next level of LSPs -- IPv4 labeled unicast LSPs. And we
>>>>>>>>>>> have spent some time discussing how to organize them. This is where I was
>>>>>>>>>>> talking about the separation of reachability from next-hops signaling. I
>>>>>>>>>>> didn't mean PHP for these LSPs. Here we can use either an explicit label
>>>>>>>>>>> when we want to make an IPv4 lookup in the IPv4 global routing table or an
>>>>>>>>>>> arbitrary label for the rest of the cases (i.e., per-CE, per-prefix, it
>>>>>>>>>>> also supports per-table, SR OS uses it by default for per-table). This is
>>>>>>>>>>> some form of overlay.
>>>>>>>>>>>
>>>>>>>>>>> For both, underlay and overlay, explicit labels *and *arbitrary
>>>>>>>>>>> labels are possible and the latter do not break anything. I hope we are
>>>>>>>>>>> synchronized at this point. For overlay, the selection could be done with a
>>>>>>>>>>> variable granularity. Many vendors allow us to set a per-table label (it
>>>>>>>>>>> can be an explicit one but not necessarily) for the most prefixes, and set
>>>>>>>>>>> dedicated labels for some smaller groups of routes that belong to some
>>>>>>>>>>> interfaces of this table (per-CE). It does not depend on whether we send
>>>>>>>>>>> all customer's routes as labeled or just next-hops for them.
>>>>>>>>>>>
>>>>>>>>>>> Third, despite this draft trying to save people from some
>>>>>>>>>>> pitfalls (losing frames at the penultimate node when the payload is
>>>>>>>>>>> unlabeled after PHP with a single-labeled stack's LSPs) I still think that
>>>>>>>>>>> these options shouldn't be excluded from the consideration. If we advertise
>>>>>>>>>>> IPv4 1/1 routes with IPv6 NHs and PHP is not done, nothing bad happens,
>>>>>>>>>>> right? So, from my POV, it is better not to delve into the kingdom of
>>>>>>>>>>> restrictions, but to make a clear statement that with PHP for
>>>>>>>>>>> transport/underlay in the case of a single-labeled stack, there is a
>>>>>>>>>>> dangerous situation. If a user applies a two-labeled or more stack, PHP is
>>>>>>>>>>> possible for all labels prior to the bottom one. So there is a simple
>>>>>>>>>>> condition for PHP depending on the depth of the stack, and no need to
>>>>>>>>>>> restrict a user with the exact depth.
>>>>>>>>>>>
>>>>>>>>>>> In summary.
>>>>>>>>>>> * The draft should remove all "MUST" corresponding to the
>>>>>>>>>>> procedures of sending customers' routes and the selection of AFI/SAFI for
>>>>>>>>>>> them (IPv4 1/1 vs. 1/4).
>>>>>>>>>>> * The draft should not restrict using any labels for transport
>>>>>>>>>>> LSPs (arbitrary vs. explicit).
>>>>>>>>>>> * The draft should describe the case when the penultimate node
>>>>>>>>>>> drops traffic, warn readers, and suggest (not mandate) a more viable
>>>>>>>>>>> alternative.
>>>>>>>>>>> * With this alternative (IPv4 LU) there should not be strict
>>>>>>>>>>> requirements for sending all customers' routes via 1/4, for some cases, the
>>>>>>>>>>> signaling of next-hops is also suitable.
>>>>>>>>>>> * The draft should be informational, all the mechanics are
>>>>>>>>>>> already available, and the notion of the depth of a label stack is not
>>>>>>>>>>> something new.
>>>>>>>>>>> * The draft should consider other options, not related to MPLS,
>>>>>>>>>>> or related but with nowadays features (e.g., TEA).
>>>>>>>>>>>
>>>>>>>>>>> Thank you!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ср, 2 авг. 2023 г. в 04:13, Gyan Mishra <[e-mail]>:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>
>>>>>>>>>>>> We are making a lot of progress here!
>>>>>>>>>>>>
>>>>>>>>>>>> Responses in-line Gyan5>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Aug 1, 2023 at 5:42 PM Igor Malyushkin <
>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>        Gyan3> I agree this sentence is a bit confusing with
>>>>>>>>>>>>> regards to label 0 and 2 use.  We are not using IPv4 explicit null value 0
>>>>>>>>>>>>> at all in this draft.  So the conveying of the next hop and what the
>>>>>>>>>>>>> sentence it talking about related to the IPv6 LSP egress PE signaling done
>>>>>>>>>>>>> via the IPv6 explicit null label value 2 defined in RFC 4182 to use for
>>>>>>>>>>>>> topmost label so the next hop of the  advertised IPv4 routes have an IPv6
>>>>>>>>>>>>> next hop as defined according to RFC 8950 next hop encoding which is an
>>>>>>>>>>>>> IPv6 labeled address per BGP LU RFC 8277 for the BOS IPv4 prefixes. So here
>>>>>>>>>>>>>  we are talking about using label value 2 IPv6 explicit null to identify
>>>>>>>>>>>>> the IPv4 routing context of the outgoing interface since the next hop is an
>>>>>>>>>>>>> IPv6 next hop for the labeled IPv4 prefixes.  I will cleanup the wording.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Now it is more clear to me. Thank you. I've already expressed
>>>>>>>>>>>>> my thoughts on restriction of the transport. I strongly disagree.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    Gyan5> Understood and points and use cases and reasons for
>>>>>>>>>>>> flexibility of arbitrary label are well taken.
>>>>>>>>>>>> Just to make sure I am crystal clear.  The use of topmost
>>>>>>>>>>>> transport IPv6 explicit null label does not allow you to create the per CE
>>>>>>>>>>>> / LSP IPv4 context tables or does it.  If so what is the technical reason
>>>>>>>>>>>> why you cannot label the loopbacks or CE interface creating an IPv4 LSP to
>>>>>>>>>>>> tunnel the per CE prefixes unlabeled.  I missed that if that’s the case so
>>>>>>>>>>>> want to understand why your use case is not possible with topmost transport
>>>>>>>>>>>> label  IPv6 explicit null.
>>>>>>>>>>>>
>>>>>>>>>>>> I believe you would like the flexibility of PHP as you have use
>>>>>>>>>>>> cases where PHP is necessary.   However that is separate from your primary
>>>>>>>>>>>> use case of per CE / LSP IPv4 context tables providing scalability and
>>>>>>>>>>>> faster convergence.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> P.S. Label value 2 cannot identify the IPv4 routing context.
>>>>>>>>>>>>> With BoS bit it identifies the IPv6 routing context. Without BoS bit this
>>>>>>>>>>>>> label is stripped and a lookup is done for the next label until the bottom
>>>>>>>>>>>>> is reached. The bottom label actually identifies the context in this case.
>>>>>>>>>>>>> So, it is also not the best wording I believe.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    Gyan5> Agreed will fix
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> вт, 1 авг. 2023 г. в 21:20, Gyan Mishra <[e-mail]
>>>>>>>>>>>>> >:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Response on the remaining item I missed in your first email -
>>>>>>>>>>>>>> Gyan3>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Gyan
>>>>>>>>>>>>>> On Tue, Aug 1, 2023 at 11:02 AM Gyan Mishra <
>>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Responses in-line Gyan2>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jul 31, 2023 at 5:20 AM Igor Malyushkin <
>>>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Gyan,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My responses are also inline (IM>).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> пн, 31 июл. 2023 г. в 09:07, Gyan Mishra <
>>>>>>>>>>>>>>>> [e-mail]>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Many thanks to your questions to clarify points in the
>>>>>>>>>>>>>>>>> draft.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Responses in-line Gyan>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sun, Jul 30, 2023 at 8:40 AM Igor Malyushkin <
>>>>>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Gyan,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have several questions w.r.t your draft. But first I
>>>>>>>>>>>>>>>>>> would like to clarify some points from your summary.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "*Customer IPv4 prefixes must be labeled when tunneled
>>>>>>>>>>>>>>>>>> over IPv6 LSP*". I believe there is a misconception
>>>>>>>>>>>>>>>>>> between the "labeled prefixes" and the "labeled traffic". Prefixes cannot
>>>>>>>>>>>>>>>>>> be tunneled via LSPs, traffic can be. Customer IPv4 prefixes can be
>>>>>>>>>>>>>>>>>> dissipated as vanilla IPv4 prefixes without labels. I described the reasons
>>>>>>>>>>>>>>>>>> during the previous discussion of this document.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gyan> I was referring to the bottom of stack service
>>>>>>>>>>>>>>>>> labeling of the IPv4 customer prefixes which the traffic flows are within
>>>>>>>>>>>>>>>>> the payload of packet and not the label stack.  I can change the verbiage
>>>>>>>>>>>>>>>>> to labeled traffic. Good point.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "*If not labeled on the PHP node when the topmost label
>>>>>>>>>>>>>>>>>> is popped the native IPv4 prefix is exposed and is not routable and will be
>>>>>>>>>>>>>>>>>> dropped*", this sentence describes traffic, not BGP
>>>>>>>>>>>>>>>>>> prefixes, and I'm sure we can reach the goal of having two and more labels
>>>>>>>>>>>>>>>>>> in a stack without advertising tons of IPv4 reachability with labels. We
>>>>>>>>>>>>>>>>>> can solely use IPv4 labeled prefixes also, as this draft describes, but
>>>>>>>>>>>>>>>>>> there are other options too.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> What I am describing is the native IPv4 prefixes
>>>>>>>>>>>>>>>>> under the topmost transport IPv6 label once popped that are exposed which
>>>>>>>>>>>>>>>>> the protocol is IPv4, where  the core P nodes are IPv6 so the IPv4 Paulo’s
>>>>>>>>>>>>>>>>> packets will be dropped.  Imagine the scenario where you have an IPv4 core
>>>>>>>>>>>>>>>>> and have native IPv4 to the edge so in that case you would have native IPv4
>>>>>>>>>>>>>>>>> Unicast prefixes 1/1 over a IPv4 core IPv4 LSP topmost label, and with IPv6
>>>>>>>>>>>>>>>>> core IPv6 LSP topmost label would have native IPv4 Unicast prefixes 1/1
>>>>>>>>>>>>>>>>> over IPv6 core.  I can rewrite as “Native IPv4 prefixes are exposed and non
>>>>>>>>>>>>>>>>> routable and IPv4 payload traffic is dropped. RFC 3032 section 2.2 states
>>>>>>>>>>>>>>>>> that the bottom of stack prefixes if a different protocol  then the topmost
>>>>>>>>>>>>>>>>> label as in this case must be labeled to identify the protocol type of the
>>>>>>>>>>>>>>>>> prefixes.  We can discuss the other options.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "*RFC 7948 “6PE” states that the label bound to the IPv4
>>>>>>>>>>>>>>>>>> prefixes may be an arbitrary value or explicit null label which has led to
>>>>>>>>>>>>>>>>>> vendor interoperability issues in the past*". Can you
>>>>>>>>>>>>>>>>>> please elaborate on that topic? I know the exact opposite cases.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> Issue arises if one vendor uses arbitrary label
>>>>>>>>>>>>>>>>> and the other vendor uses IPv4 explicit null label value 1. AFAIK  Both
>>>>>>>>>>>>>>>>> sides using to arbitrary value or both sides signaling explicit null is ok
>>>>>>>>>>>>>>>>> but with mismatch there is an issue.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> The selection of a value for a label is a matter of
>>>>>>>>>>>>>>>> choice of a node that allocates the value for the label. I'm lost what you
>>>>>>>>>>>>>>>> mean by "both sides". There is no such conception as "mismatch" because
>>>>>>>>>>>>>>>> LSRs do not negotiate label values among themselves in general. Yes, there
>>>>>>>>>>>>>>>> are examples of the opposite but for peculiar cases that do not interfere
>>>>>>>>>>>>>>>> with our topic.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    Gyan2>Unidirectional LSP so no concept of both sides
>>>>>>>>>>>>>>> -agreed. One side could signal LSP using arbitrary label and other side
>>>>>>>>>>>>>>> explicit null. There is not related to mismatch and is some other peculiar
>>>>>>>>>>>>>>> case which I have to dig up.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "*4PE draft states that the IPv4 prefixes must use MPLS
>>>>>>>>>>>>>>>>>> QOS RFC 3270 Pipe mode explicit null label bound to the IPv4 prefix and
>>>>>>>>>>>>>>>>>> MUST be used ...*". In my opinion, the idea of getting
>>>>>>>>>>>>>>>>>> rid of arbitrary labels breaks a lot of stuff. I will clarify it later.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> Yes please clarify
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> Well, I've done it below in my previous message.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "*Additional text clarity added related RFC 8950 next
>>>>>>>>>>>>>>>>>> hop encoding...*". Honestly, I don't understand why. It
>>>>>>>>>>>>>>>>>> is just a rephrase of the stuff from the original standard. Can you shed
>>>>>>>>>>>>>>>>>> light on the necessity of this section? Maybe I'm missing something.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Here and after my comments w.r.t the body of the draft.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gyan> It’s just for reference stated explicitly in the
>>>>>>>>>>>>>>>>> draft but I agree it can be removed
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. Section 1. "*This document explains the "4PE" design
>>>>>>>>>>>>>>>>>> procedures and how to interconnect IPv4 islands over a Multiprotocol Label
>>>>>>>>>>>>>>>>>> Switching (MPLS) [RFC3031] LDPv6 enabled IPv6-Only core, Segment Routing
>>>>>>>>>>>>>>>>>> (SR) enabled SR-MPLS [RFC8660] IPv6-Only core or SRv6 [RFC8986] IPv6-Only
>>>>>>>>>>>>>>>>>> core*". Is my understanding correct that the 4PE
>>>>>>>>>>>>>>>>>> solution does not support RSVP-TE LSP signaled to IPv6 tail-ends over an
>>>>>>>>>>>>>>>>>> IPv6-routed core? I see LDPv6 here, but the link goes to RFC3031 which is
>>>>>>>>>>>>>>>>>> the architecture standard that applies not only to LDP. By the way,
>>>>>>>>>>>>>>>>>> 6PE/6vPE perfectly works with RSVP LSPs.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan>  RFC 3031 is referenced for MPLS data plane but I
>>>>>>>>>>>>>>>>> did not specify LDP spec which I can add to be consistent.  Further down in
>>>>>>>>>>>>>>>>> the draft I do mention RSVP-TE but forgot to mention in above text so will
>>>>>>>>>>>>>>>>> add.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. Section 1. "*The 4PE routers exchange the IPv4
>>>>>>>>>>>>>>>>>> reachability information transparently over the core using the
>>>>>>>>>>>>>>>>>> Multiprotocol Border Gateway Protocol (MP-BGP) over IPv6. In doing so, the
>>>>>>>>>>>>>>>>>> BGP Next Hop field egress PE FEC (Forwarding Equivalency Class) is used to
>>>>>>>>>>>>>>>>>> convey the IPv6 address of the 4PE router learned dynamically via IGP...*".
>>>>>>>>>>>>>>>>>> Can a 4PE router, that resides in one IGP domain, use a next-hop address
>>>>>>>>>>>>>>>>>> received from the other domain via, for example, IPv6 labeled unicast?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> Yes it can using as you stated using BGP-LU is
>>>>>>>>>>>>>>>>> exactly how it’s done with inter-as option C where the next hops are
>>>>>>>>>>>>>>>>> imported between the IGP domains.  See section 7.3.2.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> "Between the IGP domains" does not mean "between ASes".
>>>>>>>>>>>>>>>> From my understanding, Option C describes the latter, I was asking about
>>>>>>>>>>>>>>>> the former. So my question is still intact. Can addresses for BGP next-hops
>>>>>>>>>>>>>>>> be learned via means different from IGP as you stated in the body of the
>>>>>>>>>>>>>>>> doc? Consider the Seamless MPLS approach. My point is that you should
>>>>>>>>>>>>>>>> delete "... via IGP", not only IGP can do that, but BGP also can.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Gyan2> Got it will update
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 3. Section 1. "...* over an MPLS [RFC3031] LDP IPv6
>>>>>>>>>>>>>>>>>> core, Segment Routing SR-MPLS [RFC8660] IPv6 core or SRv6 [RFC8986] IPv6
>>>>>>>>>>>>>>>>>> core*". This part is a repetition of the same that was
>>>>>>>>>>>>>>>>>> done previously, and it will be repeated several times later. Moreover,
>>>>>>>>>>>>>>>>>> there are standard numbers enclosed after every term again and again. As I
>>>>>>>>>>>>>>>>>> understand, it is enough to do it one time per every desired abbreviation
>>>>>>>>>>>>>>>>>> during the whole body.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gyan> Agreed will fix
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 4. Section 1. "*The approach requires that the Provider
>>>>>>>>>>>>>>>>>> Edge (PE) routers Provider Edge - Customer Edge (PE-CE) connections to
>>>>>>>>>>>>>>>>>> Customer Edge (CE)*". Honestly, it is difficult to read
>>>>>>>>>>>>>>>>>> such things. From my POV, there should be a section with all unique
>>>>>>>>>>>>>>>>>> definitions pertaining to this draft (e.g., 4PE router). Moreover, things
>>>>>>>>>>>>>>>>>> such as the PE, and CE are well-known and do not require unfolding. There
>>>>>>>>>>>>>>>>>> is a document of such well-known terms on the IETF portal for your
>>>>>>>>>>>>>>>>>> convenience. The last thing, as I see the same abbreviations are unfolded
>>>>>>>>>>>>>>>>>> many times during the document, which is contrary to the idea of an
>>>>>>>>>>>>>>>>>> abbreviation in general. Once explained, an abbreviation should be used
>>>>>>>>>>>>>>>>>> without further explanation.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gyan> Agreed will fix
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 5. Section 3. "*...the 4PE IPv6 address Loopback0 MUST
>>>>>>>>>>>>>>>>>> to be routable within the IPv6 core*". What is "the 4PE
>>>>>>>>>>>>>>>>>> IPv6 address Loopback0"? It was not previously defined in the document.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gyan> Will define and make clear
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 6. Section 3. "*Every ingress 4PE router can signal an
>>>>>>>>>>>>>>>>>> IPv6 MPLS [RFC3031] LSP, SRMPLS [RFC8660] LSP or instantiate an SRv6 Best
>>>>>>>>>>>>>>>>>> Effort (BE) or Segment Routing Traffic Engineering (SR-TE) [RFC9256] path
>>>>>>>>>>>>>>>>>> to send to any egress 4PE router*". It is not necessary
>>>>>>>>>>>>>>>>>> for an ingress router to signal an LSP. For example, LDP DU signals LSPs
>>>>>>>>>>>>>>>>>> from a downstream towards all possible upstreams. RSVP or LDP DoD acts the
>>>>>>>>>>>>>>>>>> opposite (if we talk about the intention, not a label distribution). So it
>>>>>>>>>>>>>>>>>> is better to rephrase this sentence as "every ingress 4PE router can have
>>>>>>>>>>>>>>>>>> an LSP toward..." or something like that.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>   Gyan> I was speaking generically not actually but is
>>>>>>>>>>>>>>>>> confusing as you stated not correct- so will fix with the LDP downstream
>>>>>>>>>>>>>>>>> unsolicited and RSVP-TE DOD.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> Well, one can call me too pedantic, but I believe that
>>>>>>>>>>>>>>>> technical documentation especially the one that wants to be a Standard must
>>>>>>>>>>>>>>>> be streamlined and precise.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     Gyan2> Yep Understood
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 7. Section 3. "*...the IPv6 signaled next hop Loopback0
>>>>>>>>>>>>>>>>>> used to identify the Ingress and Egress 4PE router*".
>>>>>>>>>>>>>>>>>> Now the Loopback0 is somehow connected to some next-hop. I believe this
>>>>>>>>>>>>>>>>>> connection should be clarified too.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>      Gyan> Will fix
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 8. Section 3. "*In doing so, the 4PE routers convey
>>>>>>>>>>>>>>>>>> their IPv6 address FEC label binding as the BGP Next Hop for the advertised
>>>>>>>>>>>>>>>>>> IPv4 prefixes*". The term FEC is pretty general, but the
>>>>>>>>>>>>>>>>>> "IPv6 address FEC label binding" can mislead a reader. From my POV, this
>>>>>>>>>>>>>>>>>> term is related to LDP, but the document includes many options to signal
>>>>>>>>>>>>>>>>>> transport paths.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> Will fix
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 9. Section 3. "*The ingress and egress 4PE router MUST
>>>>>>>>>>>>>>>>>> bind a label to the IPv4 prefix as per [RFC8277] using BGP Labeled Unicast
>>>>>>>>>>>>>>>>>> herinafter called BGP-LU, AFI/SAFI Address Family (AFI) / Subsequent
>>>>>>>>>>>>>>>>>> Address Family Identifier (SAFI) 2-tuple "1/4"*".  Which
>>>>>>>>>>>>>>>>>> is the IPv4 prefix? If I understand it correctly, this requirement is too
>>>>>>>>>>>>>>>>>> strict. As a designer I don't want to send all IPv4 prefixes with labels in
>>>>>>>>>>>>>>>>>> every case, I want to have some flexibility. Moreover, as a part of the
>>>>>>>>>>>>>>>>>> implementation team, I'm sure we won't consider this as the only possible
>>>>>>>>>>>>>>>>>> option too.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan>Lets discuss this in more detail the use cases
>>>>>>>>>>>>>>>>> for not labeling.  Also the issue I mentioned in RFC 3032 section 2.2 that
>>>>>>>>>>>>>>>>> describes the problem with not labeling the prefix if the protocol is
>>>>>>>>>>>>>>>>> different between the topmost label and service prefix.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> I believe there should be a clear distinction between
>>>>>>>>>>>>>>>> reachability advertisements and LSP signaling. BGP LU does both at the same
>>>>>>>>>>>>>>>> time. Every IP prefix here comes with a label that allows us to signal a
>>>>>>>>>>>>>>>> BGP LSP. It seems to me, you consider LU only as the copy-paste of IPv4
>>>>>>>>>>>>>>>> unicast but with labels. I offer you to look at it as an LSP signaling
>>>>>>>>>>>>>>>> protocol (one can think LU is an LDP on steroids).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Let's consider an example. We have a PE-CE link with an
>>>>>>>>>>>>>>>> IPv4 subnet toward an IPv4 CE. A PE router of that PE-CE link is a part of
>>>>>>>>>>>>>>>> the IPv6-only core. We have some v6 LSP mesh among all PEs of that core.
>>>>>>>>>>>>>>>> These v6 LSPs populate the FTN table and allow us to resolve any BGP
>>>>>>>>>>>>>>>> next-hops through the network. The CE advertises IPv4 routes via 1/1. Let's
>>>>>>>>>>>>>>>> call them Customer Prefixes or Reachability for convenience. I believe, at
>>>>>>>>>>>>>>>> this point, the setup is pretty clear.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Your draft restricts me to export these Customer Prefixes
>>>>>>>>>>>>>>>> into IPv4 labeled unicast internal sessions and also restricts me to attach
>>>>>>>>>>>>>>>> an explicit null label (with the value of 0) to them. Anyway, that works,
>>>>>>>>>>>>>>>> and I have never argued
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I don't see a good reason to do that. All that I want is to
>>>>>>>>>>>>>>>> create an IPv4 loopback on the PE and advertise its IPv4 address via IPv4
>>>>>>>>>>>>>>>> labeled unicast toward remote PEs. In doing so I create a BGP LSP from
>>>>>>>>>>>>>>>> every remote PE to this advertising PE. The label from this advertisement
>>>>>>>>>>>>>>>> can be either an explicit null or an arbitrary one. This is my choice. Then
>>>>>>>>>>>>>>>> I just readvertise all Customer Prefixes via internal IPv4 1/1 sessions
>>>>>>>>>>>>>>>> with the next-hop address of my new shiny IPv4 loopback.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Both examples would have the same deep of a label stack.
>>>>>>>>>>>>>>>> Both examples wouldn't have any problems at the penultimate LSR. Both
>>>>>>>>>>>>>>>> examples (well, mechanics, not exact examples) should be a part of the doc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But with my example, I have greater flexibility, I can
>>>>>>>>>>>>>>>> change the next-hop addresses from the loopback address to an address of
>>>>>>>>>>>>>>>> the PE-CE link, allocate an arbitrary label in a per-next-hop fashion and
>>>>>>>>>>>>>>>> get an EPE. Also, it would increase convergence if I want to. The key here
>>>>>>>>>>>>>>>> -- is my desire as a designer. With your solution, it is
>>>>>>>>>>>>>>>> impossible because the next-hop for all Customer Prefixes and the label
>>>>>>>>>>>>>>>> would always be the same. It is impossible from a single value (0)
>>>>>>>>>>>>>>>> to produce many different actions at egress.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>      Gyan2> Thank you for the detailed use case.  Have you
>>>>>>>>>>>>>>> tried to configure this example and with which vendor?  I am not sure I
>>>>>>>>>>>>>>> would call it EPE as SR or BGP-LU are not steering traffic to the egress
>>>>>>>>>>>>>>> but I get it your point EPE like behavior to point to specific CE next
>>>>>>>>>>>>>>> hop.  What is the advantage of doing so.  With my solution you have a
>>>>>>>>>>>>>>> single next hop which is the egress PE however all the prefixes received on
>>>>>>>>>>>>>>> the egress PE in the IPv4 labeled unicast RIB would get advertised to each
>>>>>>>>>>>>>>> CE across the respective next hops. In your example you create and LSP
>>>>>>>>>>>>>>> between the PE loopbacks or use PE-CE interface next hop for the LSP per CE
>>>>>>>>>>>>>>> and then advertise all the prefixes per CE unlabeled.  The  prefixes for
>>>>>>>>>>>>>>> each CE broken down now by IPv4 LSP are going to all get advertised out to
>>>>>>>>>>>>>>> all CEs anyway doing it your way or my way.  So net end result of what gets
>>>>>>>>>>>>>>> advertised to which ingress and egress CE over the LSP is identical. I am
>>>>>>>>>>>>>>> not seeing any benefit to doing so other than added complexity.  I am not
>>>>>>>>>>>>>>> sure how much convergence gain you have as you can do per VRF label
>>>>>>>>>>>>>>> allocation and better scale and convergence. Can you expand on the benefits
>>>>>>>>>>>>>>> of doing so in your example. I do now understand the flexibility you desire.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 10. Section 3. "*The Ingress 4PE router MUST forward
>>>>>>>>>>>>>>>>>> IPv4 NLRI as Labeled prefixes using BGP-LU SAFI over the IPv6-signaled LSP
>>>>>>>>>>>>>>>>>> towards the Egress 4PE router identified by the IPv4 address advertised in
>>>>>>>>>>>>>>>>>> the IPv6 next hop encoding per [RFC8950]*". NLRI is a
>>>>>>>>>>>>>>>>>> term related to BGP message encoding. How can a 4PE router forward an NLRI
>>>>>>>>>>>>>>>>>> over an LSP? Forwarding is a procedure of transmitting traffic, LSP is a
>>>>>>>>>>>>>>>>>> transport entity used for forwarding labeled traffic, not NLRIs.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>   Gyan> I am referring to BGP SAFI in this case IPv4
>>>>>>>>>>>>>>>>> labeled unicast 1/4 which should say “advertised” and not “forwarded”
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 11. Section 4. "*To ensure interoperability between
>>>>>>>>>>>>>>>>>> routers that implement the 4PE design over MPLS [RFC3031] LDP IPv6 Core
>>>>>>>>>>>>>>>>>> described in this document, ingress and egress 4PE MUST support building
>>>>>>>>>>>>>>>>>> the underlay tunneling using IPv6-signaled MPLS LSPs established by LDP
>>>>>>>>>>>>>>>>>> [RFC5036] or Resource Reservation Protocol (RSVP-TE) [RFC3209]*".
>>>>>>>>>>>>>>>>>> Here we see that RSVP is anyhow possible, but I'm still missing the idea of
>>>>>>>>>>>>>>>>>> "MPLS LDP IPv6 Core" and how RSVP is related to it. Is this sentence about
>>>>>>>>>>>>>>>>>> LDP over RSVP tunneling? Or about ships in the night?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> yes two ships in the night two different
>>>>>>>>>>>>>>>>> scenarios will fix
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 12. Section 4. "*The (outer) label imposed MUST
>>>>>>>>>>>>>>>>>> correspond to the IPv6- signaled LSP starting on the ingress 4PE Router and
>>>>>>>>>>>>>>>>>> ending on the egress 4PE Router*". What if I have
>>>>>>>>>>>>>>>>>> segmented LSPs among several domains of the same AS? Pretty sure that in
>>>>>>>>>>>>>>>>>> this case, the outermost label would correspond to an LSP either from an
>>>>>>>>>>>>>>>>>> ingress 4PE to an ABR or from an ABR to the other ABR/egress 4PE router.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> Yes that’s correct and will add that scenario of
>>>>>>>>>>>>>>>>> multiple domains to the wording for end to end scenario
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 13. Section 4. "*The reason for the use of a second
>>>>>>>>>>>>>>>>>> level bottom of stack service label...*". I'm sure that
>>>>>>>>>>>>>>>>>> the stack can be deeper. I made some hints earlier in my notes (e.g., see
>>>>>>>>>>>>>>>>>> the previous bullet), also you can think about emerging MNA solutions. I
>>>>>>>>>>>>>>>>>> think it is better to pay attention not to the number of labels in a stack
>>>>>>>>>>>>>>>>>> but to the fact that the bottom label must pertain to an LSP forwarding
>>>>>>>>>>>>>>>>>> IPv4 traffic and that this label must be present on the egress.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> Agreed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 14. Section 4. "*...it allows for Penultimate Hop
>>>>>>>>>>>>>>>>>> Popping (PHP) on the IPv6 Label Switch Router (LSR) Provider (P) node,...*".
>>>>>>>>>>>>>>>>>> I believe it is better to use one terminology, either pertaining to an MPLS
>>>>>>>>>>>>>>>>>> LSP (LER/LSR) or a BGP service (P/PE) in a single sentence. Here
>>>>>>>>>>>>>>>>>> MPLS-related terms suit you better because you are describing an LSP and
>>>>>>>>>>>>>>>>>> label stack.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> Agreed will fix
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 15. Section 4. "*The label advertised by the egress 4PE
>>>>>>>>>>>>>>>>>> Router with MP-BGP MUST be an explicit Null label Pipe mode Diff-Serv
>>>>>>>>>>>>>>>>>> Tunneling Model use case as defined in [RFC3270], so that the topmost label
>>>>>>>>>>>>>>>>>> can be preserved Ultimate Hop POP (UHP) to the egress PE Edge LSR*".
>>>>>>>>>>>>>>>>>> Why can it be an arbitrary label instead an explicit null?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> The reason for IPv6 explicit null label for
>>>>>>>>>>>>>>>>> topmost label is to prevent the drop issue in case the service prefix is
>>>>>>>>>>>>>>>>> not labeled as I have mentioned in the draft and the IPv4 traffic is
>>>>>>>>>>>>>>>>> dropped.  Using IPv6 explicit null label RFC 3270  pipe mode also fixes the
>>>>>>>>>>>>>>>>> MPLS QOS issue of EXP PHB scheduling on PHP node when the topmost label is
>>>>>>>>>>>>>>>>> popped as mentioned earlier.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> My question was about a different thing. An arbitrary
>>>>>>>>>>>>>>>> label also prevents a penultimate LSR from exposing the payload. An LSR
>>>>>>>>>>>>>>>> would do a simple swap operation not knowing itself as penultimate. The
>>>>>>>>>>>>>>>> second moment, I don't understand why are you restricting the transport
>>>>>>>>>>>>>>>> from PHP if you have already restricted the service from that. One
>>>>>>>>>>>>>>>> shouldn't try to save everybody from every possible pitfall with a single
>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     Gyan2> I am restricting the transport from PHP with
>>>>>>>>>>>>>>> topmost IPv6 LSP explicit null signaling.  The service does not need to be
>>>>>>>>>>>>>>> restricted as it’s only popped at the egress PE.  The PHP node would either
>>>>>>>>>>>>>>> do default PHP and pop the topmost label or if explicit null signaling is
>>>>>>>>>>>>>>> enabled for the LSP the topmost is preserved to egress PE which pops the
>>>>>>>>>>>>>>> entire stack.  All intermediate LSRs would do the normal swap operation
>>>>>>>>>>>>>>> except the PHP node which would use the implicit null value 3 label default
>>>>>>>>>>>>>>> PHP POP operation.  My concern is only with exposing unlabeled service
>>>>>>>>>>>>>>> which would get dropped with the default PHP operation and not using
>>>>>>>>>>>>>>> explicit null signaling on the transport LSP.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 16. Section 4. "*The explicit null label advertised by
>>>>>>>>>>>>>>>>>> the egress PE router with MP-BGP also identifies the IPv4 routing context
>>>>>>>>>>>>>>>>>> or outgoing interface to forward the packet to and ingress 4PE Router which
>>>>>>>>>>>>>>>>>> MUST be able to be accept the "IPv6 Explicit NULL Label" advertised*".
>>>>>>>>>>>>>>>>>> From the very beginning of this statement, I understand that the egress PE
>>>>>>>>>>>>>>>>>> advertises an explicit label (0) for IPv4-labeled unicast routes to
>>>>>>>>>>>>>>>>>> identify either IPv4 routing context or outgoing interface. The end of this
>>>>>>>>>>>>>>>>>> sentence tells us that the ingress 4PE router must accept the IPv6 explicit
>>>>>>>>>>>>>>>>>> label (2). This label (2) cannot be used to identify either IPv4 routing
>>>>>>>>>>>>>>>>>> contexts or any interfaces. So I made an assumption that there is a mistake
>>>>>>>>>>>>>>>>>> and this is actually a 0 label, not 2.
>>>>>>>>>>>>>>>>>> How can the label with the same value (0) identify
>>>>>>>>>>>>>>>>>> several things (context, interface)? If you offer to bind this label to an
>>>>>>>>>>>>>>>>>> exact interface (without a route lookup at egress) instead of doing a
>>>>>>>>>>>>>>>>>> lookup in the global routing context/table, it will break the idea behind
>>>>>>>>>>>>>>>>>> this special purpose label and won't support more than one interface for
>>>>>>>>>>>>>>>>>> your solution. Arbitrary labels solve this issue and also many others.
>>>>>>>>>>>>>>>>>> For example, it allows us to do EPE if and only if we
>>>>>>>>>>>>>>>>>> dissipate IPv4 reachability (separately from its next-hop addresses) as
>>>>>>>>>>>>>>>>>> IPv4 routes with IPv4 next-hops alongside arbitrary labels for these
>>>>>>>>>>>>>>>>>> next-hops distributed as labeled IPv4 routes with IPv6 next-hops (which
>>>>>>>>>>>>>>>>>> this draft does not support and breaks). In this case, these arbitrary
>>>>>>>>>>>>>>>>>> labels are allocated in per next-hop fashion and preclude any IP lookups at
>>>>>>>>>>>>>>>>>> egress. I've have mentioned it several months ago.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> It seems, that you have skipped the crucial part of my
>>>>>>>>>>>>>>>> message :(
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     Gyan2> I will answer in the next email, thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>        Gyan3> I agree this sentence is a bit confusing with
>>>>>>>>>>>>>> regards to label 0 and 2 use.  We are not using IPv4 explicit null value 0
>>>>>>>>>>>>>> at all in this draft.  So the conveying of the next hop and what the
>>>>>>>>>>>>>> sentence it talking about related to the IPv6 LSP egress PE signaling done
>>>>>>>>>>>>>> via the IPv6 explicit null label value 2 defined in RFC 4182 to use for
>>>>>>>>>>>>>> topmost label so the next hop of the  advertised IPv4 routes have an IPv6
>>>>>>>>>>>>>> next hop as defined according to RFC 8950 next hop encoding which is an
>>>>>>>>>>>>>> IPv6 labeled address per BGP LU RFC 8277 for the BOS IPv4 prefixes. So here
>>>>>>>>>>>>>>  we are talking about using label value 2 IPv6 explicit null to identify
>>>>>>>>>>>>>> the IPv4 routing context of the outgoing interface since the next hop is an
>>>>>>>>>>>>>> IPv6 next hop for the labeled IPv4 prefixes.  I will cleanup the wording.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 17. Section 4. "*4PE design MUST use "IPv6 Explicit Null
>>>>>>>>>>>>>>>>>> label" value 2 defined in [RFC4182] Pipe Diff-Serv Tunneling Model as
>>>>>>>>>>>>>>>>>> defined in [RFC3270]*". From this statement, I conclude
>>>>>>>>>>>>>>>>>> that the authors talk about an underlying transport (IPv6 LSP via BGP LU).
>>>>>>>>>>>>>>>>>> Why does the draft about a service mandate a label for transport? I don't
>>>>>>>>>>>>>>>>>> see anything wrong with arbitrary labels for underlying IPv6 LSPs either.
>>>>>>>>>>>>>>>>>> If you want to highlight that there should *not *be PHP
>>>>>>>>>>>>>>>>>> it does not mean there is only option is an explicit label. Some vendors
>>>>>>>>>>>>>>>>>> use arbitrary labels by default with an option to use a label of 2 instead.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> I agree most vendors use arbitrary label by
>>>>>>>>>>>>>>>>> default and an option to use explicit null.  As it’s an easy configuration
>>>>>>>>>>>>>>>>> change to use explicit null with its benefits why not mandate it with the
>>>>>>>>>>>>>>>>> benefits.  As well as solve issues with mix of arbitrary and explicit null
>>>>>>>>>>>>>>>>> usage.  I would like to highlight that there would not be PHP but AFAIK
>>>>>>>>>>>>>>>>> that can only be done if using explicit null. As well as explicit null pipe
>>>>>>>>>>>>>>>>> mode solves the MPLS QOS PHB EXP scheduling issue.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> No, this can be done not only using the explicit null.
>>>>>>>>>>>>>>>> For example, Nokia SR OS allocates arbitrary labels by default for RSVP and
>>>>>>>>>>>>>>>> LDP LSPs. Penultimate routers in that case just swap one label to another
>>>>>>>>>>>>>>>> (our arbitrary one). Egress LER then handles the stack with the transport
>>>>>>>>>>>>>>>> label at the top that it allocated previously. But the difference between
>>>>>>>>>>>>>>>> an explicit label and an arbitrary one is this eLER can do many things with
>>>>>>>>>>>>>>>> the latter: allocate if for a table, for a prefix, for a next-hop address.
>>>>>>>>>>>>>>>> You can not do it with the single-valued label (0 or 2). The labels of 0 or
>>>>>>>>>>>>>>>> 2 are always tightened with their global contexts. Changing that is not
>>>>>>>>>>>>>>>> possible under this draft because it influences the whole MPLS paradigm.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     Gyan2>   In the draft all I am concerned about is the
>>>>>>>>>>>>>>> transport topmost label IPv6 explicit null signaling so the PHP node
>>>>>>>>>>>>>>> preserves the transport label to prevent the IPv4 packets from being
>>>>>>>>>>>>>>> dropped at PHP node.  It sounds like Nokia by default for RSVP and LDP LSP
>>>>>>>>>>>>>>> does not do PHP by default on the PHP node PHP label 3 and uses arbitrary
>>>>>>>>>>>>>>> label as you mentioned.  Cisco, Juniper and most all vendors on the market
>>>>>>>>>>>>>>> do PHP by default.  Egress LER pops the entire stack of remaining labels
>>>>>>>>>>>>>>> regardless of topmost label explicit null signaling or PHP operation.
>>>>>>>>>>>>>>> Since the explicit versus arbitrary relates to the topmost transport label
>>>>>>>>>>>>>>> the behavior of what happens on the LER egress PE is identical popping the
>>>>>>>>>>>>>>> entire stack after which perform per VRF table lookup, per CE next hop or
>>>>>>>>>>>>>>> per prefix label allocation. I think what you are missing is RFC 4182
>>>>>>>>>>>>>>> updates RFC 3032 label stack so that explicit null 0 and 2 can be used for
>>>>>>>>>>>>>>> the transport label where previously was only valid for BOS label.  That is
>>>>>>>>>>>>>>> what I have been referring to using explicit null for topmost label and not
>>>>>>>>>>>>>>> the BOS label.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 18. Section 4. "*BGP/MPLS VPN [RFC4364] defines 3 label
>>>>>>>>>>>>>>>>>> allocation modes for Layer 3 VPN's...*" and later "*The
>>>>>>>>>>>>>>>>>> 4PE design provides the same operator flxeiblity as BGP/MPLS VPN [RFC4798],
>>>>>>>>>>>>>>>>>> 2 level label stack option using Per-CE label allocation mode where the
>>>>>>>>>>>>>>>>>> next hop is label so all prefixes associated with CE get the same label.
>>>>>>>>>>>>>>>>>> The 4PE design provides the same operator flxeiblity as BGP/MPLS VPN
>>>>>>>>>>>>>>>>>> [RFC4798], 2 level label stack option using Per-VRF label allocation mode
>>>>>>>>>>>>>>>>>> where all prefixes within a VRF get the same is label*".
>>>>>>>>>>>>>>>>>> I believe it is not possible with an explicit label for an IPv4 LSP either
>>>>>>>>>>>>>>>>>> because a single value cannot be attached to many different things at the
>>>>>>>>>>>>>>>>>> same time.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> The IPv4 labeled prefixes AFAIK would have a
>>>>>>>>>>>>>>>>> single label per CE or per VRF. Can you elaborate on why not possible?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IM> Yes. We have a single label per VRF but we can't have a
>>>>>>>>>>>>>>>> single label per CE because there are possibly lots of CEs in a VRF. The
>>>>>>>>>>>>>>>> egress router doing a label lookup must understand what to do with every
>>>>>>>>>>>>>>>> label it allocated. With a per VRF label it does an IP lookup in a
>>>>>>>>>>>>>>>> corresponding VRF. With a per CE label it strips the label and sends
>>>>>>>>>>>>>>>> payload via the exact interface without any IP lookups. As you can see, you
>>>>>>>>>>>>>>>> can't bind a single value to many different interfaces (tasks).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    Gyan2> I think it’s actually the opposite that when using
>>>>>>>>>>>>>>> 6VPE or 4VPE you have VRF context but with 6PE or 4PE their is not many VRF
>>>>>>>>>>>>>>> context instances but we do have a single default VRF context so single
>>>>>>>>>>>>>>> label for the default VRF.  With per CE label allocation with 6VPE / 4VPE
>>>>>>>>>>>>>>> versus 6PE / 4PE I don’t see how it’s any different with per CE the label
>>>>>>>>>>>>>>> is stripped by the PE and the native packet payload is forwarded to the
>>>>>>>>>>>>>>> exact CE interface in this case it’s the default VRF many interfaces which
>>>>>>>>>>>>>>> with VPE you can have many interfaces as well - no different.  If per CE
>>>>>>>>>>>>>>> works for 6VPE / 4VPE it should work for 6PE / 4PE.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 19. Section 7.3.1. "*...the 4PE IPv4 BGP-LU labeled
>>>>>>>>>>>>>>>>>> Unicast RIB is not maintained on the ASBR*". I would
>>>>>>>>>>>>>>>>>> further elaborate on this point as these routes must not be installed in
>>>>>>>>>>>>>>>>>> the forwarding table as some vendors do it by default because it breaks the
>>>>>>>>>>>>>>>>>> whole idea of the Option B scenario. Only their labels must be installed
>>>>>>>>>>>>>>>>>> into LFIB. With the VPN-based option B there is no any danger because we do
>>>>>>>>>>>>>>>>>> not create VRFs on an ASBR, but with labeled unicast this is a possible
>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Gyan> 7.3.1 Opt-B I mentioned that labeled unicast RIB
>>>>>>>>>>>>>>>>> is maintained on ASBR but as you stated in the details is LFIB and not RIB
>>>>>>>>>>>>>>>>> agreed is more optimized as what some vendors do by default both RIB and
>>>>>>>>>>>>>>>>> LFIB. Agreed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 20. Section 8. RFC 8950 describes the logic of how to
>>>>>>>>>>>>>>>>>> handle next-hop addresses with different lengths including the routes of
>>>>>>>>>>>>>>>>>> SAFI 4. Does this section bring something new?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  Gyan>  It does not.  It’s just explicitly stating the
>>>>>>>>>>>>>>>>> next hop encoding used.  So this section could be deleted.  As the next hop
>>>>>>>>>>>>>>>>> encoding is critical I had added the section explicitly.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 21. I don't see any description of a scenario when some
>>>>>>>>>>>>>>>>>> vendors install specific routes in auxiliary tables for the sake of further
>>>>>>>>>>>>>>>>>> BGP NH resolution and consider them as LSPs. There should be a warning at
>>>>>>>>>>>>>>>>>> least.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> I can add that to security considerations
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let's imagine a case where there are two or more BNG
>>>>>>>>>>>>>>>>>> routes residing in a single PoP. The core network is single-stack IPv6.
>>>>>>>>>>>>>>>>>> BNGs are dual-stacked because we are still supporting some legacy
>>>>>>>>>>>>>>>>>> connections for v4 eyeballs. PE routes that are connected to these BNGs are
>>>>>>>>>>>>>>>>>> dual-stacked too (you can consider a BNG as a CE router). Now we want to
>>>>>>>>>>>>>>>>>> distribute tons of /32 among PEs because the BNGs share the same IPv4
>>>>>>>>>>>>>>>>>> subnets. That facilitates more optimal allocation of the depleting IPv4
>>>>>>>>>>>>>>>>>> address space and is a pretty common case in the SP world. So, if the PE
>>>>>>>>>>>>>>>>>> routers distribute these specifics by mistake or by design (let's say, to
>>>>>>>>>>>>>>>>>> other PEs of this PoP for CG-NAT purposes) as IPv4 labeled unicast routes.
>>>>>>>>>>>>>>>>>> Possible receivers of these routes may consider them viable LSPs. Imagine
>>>>>>>>>>>>>>>>>> of hundreds thousands of such prefixes. That's actually another reason why
>>>>>>>>>>>>>>>>>> labeled unicast is not good at this task.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     Gyan> Understood I can add to the security
>>>>>>>>>>>>>>>>> considerations section
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank you in advance.
>>>>>>>>>>>>>>>>>> Igor.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> чт, 27 июл. 2023 г. в 19:50, Gyan Mishra <
>>>>>>>>>>>>>>>>>> [e-mail]>:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> All
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have updated the IDR draft below updates as reviewed
>>>>>>>>>>>>>>>>>>> during our IETF  117 meeting.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> https://datatracker.ietf.org/doc/draft-mishra-idr-v4-islands-v6-core-4pe/05/
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>
> <http://www.verizon.com/>
>
> *Gyan Mishra*
>
> *Network Solutions A**rchitect *
>
> *Email [e-mail] <[e-mail]>*
>
>
>
> *M 301 502-1347*
>
>