Re: [Idr] Fwd: [E] New Version Notification for draft-mishra-idr-v4-islands-v6-core-4pe-05.txt

[Gyan Mishra <[e-mail]>]

Thanks Igor!

Gyan

On Fri, Aug 11, 2023 at 12:46 PM Igor Malyushkin <[e-mail]>
wrote:

> Hi Gyan,
>
> I'm OK with this list.
> Thank you.
>
> пт, 11 авг. 2023 г. в 18:57, Gyan Mishra <[e-mail]>:
>
>> Hi Igor / all
>>
>> Here is a summary of 4PE draft updates.
>>
>> Please review and any comments  much appreciated!
>>
>>  4PE design Deployment methods:
>>
>> All of what’s discussed use cases are  applicable to intra-as and
>> inter-as options a, b, c, ab with Data planes MPLS, SR-MPLS, SRv6.
>>
>> 1. Arbitrary and Explicit null value 2 label topmost ipv6 LSP each with 3
>> use cases: (6 use cases)
>>
>> a. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC
>> 8277 1/4 service label labeling all IPv4 customer prefixes.
>>
>> b LERs signal IPv6 topmost LSP with 2 level label stack, BOS set RFC 8277
>> 1/4 service label using ingress to egress  PE loopback to loopback LSP
>> single BOS label with all global table customer prefixes unlabeled.
>>
>> c. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC
>> 8277 1/4 service label using per CE label table routing context LSP ingress
>> to egress  CE PE-CE interface PE side interface LSP single BOS label with
>> per CE label table customer prefixes unlabeled.
>>
>> 2. IPv6 LSP PHP implicit null value 3  with 3 use cases: (3 use cases)
>>
>> a. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC
>> 8277 1/4 service label labeling all IPv4 customer prefixes.
>>
>> b LERs signal IPv6 topmost LSP with 2 level label stack, BOS set RFC 8277
>> 1/4 service label using ingress to egress  PE loopback to loopback LSP
>> single BOS label with all global table customer prefixes unlabeled.
>>
>> c. LERs signal IPv6 topmost LSP with 2 level label stack  BOS set RFC
>> 8277 1/4 service label using per CE label table routing context LSP ingress
>> to egress  CE PE-CE interface PE side interface LSP single BOS label with
>> per CE label table customer prefixes unlabeled.
>>
>> 3. Arbitrary and explicit null value 2 topmost IPv6 LSP BOS set single
>> level label stack with all global table customer prefixes 1/1 unlabeled.
>> Caveat that this requires DPI to determine next header inspection for
>> protocol type so packets are not dropped. (2 use cases)
>>
>> Total of 11 use cases.
>>
>> Thanks
>>
>> Gyan
>>
>>
>>
>> On Tue, Aug 8, 2023 at 9:02 PM Gyan Mishra <[e-mail]> wrote:
>>
>>> Hi Igor
>>>
>>> It was a pleasure having this detailed discussion.
>>>
>>> I am all set!
>>>
>>> Look out for an updated draft soon.
>>>
>>> Many Thanks!!
>>>
>>> Gyan
>>>
>>> On Mon, Aug 7, 2023 at 4:47 AM Igor Malyushkin <[e-mail]>
>>> wrote:
>>>
>>>> Hi Gayn.
>>>>
>>>> Agree, we have reached a consensus. I commented on all inline (IM4).
>>>>
>>>> Thank you for the discussion!
>>>>
>>>> вс, 6 авг. 2023 г. в 00:37, Gyan Mishra <[e-mail]>:
>>>>
>>>>>
>>>>> Hi Igor
>>>>>
>>>>> My comments in-line Gyan3>
>>>>>
>>>>> This last thread really helped clear up the use case of arbitrary
>>>>> labels which comes into play if any implementation desired to not signal
>>>>> implicit null value 3 by default.  Implicit null default is not a
>>>>> requirement so we have to account for both use case possibilities.
>>>>>
>>>>> We are pretty well in sync!!
>>>>>
>>>>> On Sat, Aug 5, 2023 at 1:11 PM Igor Malyushkin <[e-mail]>
>>>>> wrote:
>>>>>
>>>>>> Hi Gyan,
>>>>>>
>>>>>> My comments are inline with the IM3 prefix.
>>>>>>
>>>>>> сб, 5 авг. 2023 г. в 19:17, Gyan Mishra <[e-mail]>:
>>>>>>
>>>>>>> Hi Igor
>>>>>>>
>>>>>>> Responses in-line Gyan2>
>>>>>>>
>>>>>>> On Thu, Aug 3, 2023 at 2:35 PM Igor Malyushkin <
>>>>>>> [e-mail]> wrote:
>>>>>>>
>>>>>>>> My comments are inline (IM2).
>>>>>>>>
>>>>>>>> чт, 3 авг. 2023 г. в 18:54, Gyan Mishra <[e-mail]>:
>>>>>>>>
>>>>>>>>> Hi Igor
>>>>>>>>>
>>>>>>>>> Yes looks like we have reached a consensus!!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Aug 3, 2023 at 8:43 AM Igor Malyushkin <
>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Gayn, looks like we have reached a consensus!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> чт, 3 авг. 2023 г. в 09:03, Gyan Mishra <[e-mail]>:
>>>>>>>>>>
>>>>>>>>>>> Hi Igor
>>>>>>>>>>>
>>>>>>>>>>> I am good with everything listed below.
>>>>>>>>>>>
>>>>>>>>>>> As far as the topmost explicit versus arbitrary label I agree
>>>>>>>>>>> that based on implementation if you use arbitrary you are ok if 1/4 service
>>>>>>>>>>> label is used
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> IM> Ok.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> however if you use arbitrary topmost label and use 1/1 for
>>>>>>>>>>> service label at the PHP node the native IPv4 packet could be dropped.
>>>>>>>>>>> There would be a warning for that particular use case.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> IM> Yes. But I would say not at the PHP node. The PHP node just
>>>>>>>>>> swaps a label in the case of arbitrary labels. Dropping occurs at the
>>>>>>>>>> ultimate node (egress PE) due to any label for IPv6 LSPs (an arbitrary one
>>>>>>>>>> or explicit 2) here can be allocated for the IPv6 routing process, but a
>>>>>>>>>> packet is an IPv4 one.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>     Gyan> My understanding here is that the PHP (Penultimate Hop
>>>>>>>>> POP) node is not the Ultimate hop which is the egress LER (PE) but is one
>>>>>>>>> hop prior to the egress LER which would be the egress LSR (P) node.
>>>>>>>>>
>>>>>>>> [IM2] Yes, a PHP node is not an egress LER (PE). I've never heard
>>>>>>>> someone calls a penultimate node an egress LSR, but ok, I will follow for
>>>>>>>> this discussion.
>>>>>>>>
>>>>>>>>
>>>>>>>>> So when the topmost label is an arbitrary label by default the PHP
>>>>>>>>> node egress LSR (P) when it does a label swap the incoming label is
>>>>>>>>> arbitrary but the outgoing label is “POP” which signals implicit null value
>>>>>>>>> 3.
>>>>>>>>>
>>>>>>>> [IM2] No. Let me give you an example. Egress LER X allocates the
>>>>>>>> label of 3000 and puts it in its ILM table with a POP operation. Then it
>>>>>>>> advertises a mapping of its loopback's IPv6 address plus the label (3000)
>>>>>>>> via LDPv6. That means that the egress LER expects MPLS traffic with the
>>>>>>>> label of 3000 on its LDP-enabled interfaces and will strip this label by
>>>>>>>> itself. All its neighbors would either swap something to 3000 or push 3000.
>>>>>>>> The former case is when these neighbors are LSRs, the latter is when they
>>>>>>>> are iLERs. In other words, we have a continuous single-labeled LSP from
>>>>>>>> ingress LER to egress LER. All LSRs just do swapping of labels, but not
>>>>>>>> popping.
>>>>>>>>
>>>>>>>
>>>>>>>    Gyan2> Thanks for the clarification on arbitrary label use case
>>>>>>> which is why the arbitrary label comes into play. So it appear this is a
>>>>>>> vendor implementation difference.  I think this helps shed some light on
>>>>>>> the arbitrary label case that you made and helps me understand the use case.
>>>>>>>
>>>>>> [IM3] Good! But I don't see where the vendor implementation
>>>>>> difference comes to play. To me, it is standard behavior for MPLS.
>>>>>>
>>>>>>>
>>>>>>> Since Nokia does not implement implicit null by default, PHP is
>>>>>>> disabled, so then the  topmost label then can be any arbitrary label and
>>>>>>> the topmost label is now preserved to the egress LER which then pops the
>>>>>>> entire stack.
>>>>>>>
>>>>>> [IM3] I want you to note that I've never said that Nokia does not
>>>>>> implement an implicit null. I don't want problems with Nokia fellas :) It
>>>>>> is turned off by default, yes, but you can easily turn it back. The rest is
>>>>>> correct.
>>>>>>
>>>>>
>>>>>      Gyan3> Ack we are in sync here.  Just for clarity I think both of
>>>>> us did not say that Nokia did not implement implicit null. My main point is
>>>>> that RFC 3031 does not make implicit null a MUST for implementations by
>>>>> default as long as implicit null is supported.  So Nokia supports
>>>>> configuration of implicit null, however it is not default and leads to a
>>>>> valid corner case with arbitrary labels use case where PHP is disabled.
>>>>> PHP disabled default behavior as well provides an alternative to orthogonal
>>>>> point that RFC 3270 MPLS QOS Pipe mode using explicit null is not necessary
>>>>> as now you can use arbitrary topmost labels which are now preserved and do
>>>>> PHB MPLS QOS EXP scheduling is not broken. So that based on implementations
>>>>> over the years makes disabling PHP by default by not signaling implicit
>>>>> null possibilities as not a corner case at all as there are many advantages
>>>>> of doing so as we have seen with this thread. Historically PHP existed to
>>>>> not overburden the LER to pop the entire stack offloading the load to the
>>>>> egress LSR to POP the topmost label (PHP).  As well two flavors of
>>>>> implementation to disable PHP, Cisco / Juniper method of enabling explicit
>>>>> null and Nokia method of disabling implicit null by default.  I would like
>>>>> to point out that the MPLS QOS EXP scheduling reason for explicit null or
>>>>> not configuring implicit null is vendor implementation of being able to PHB
>>>>> schedule MPLS QOS EXP and simultaneously with same policy schedule on DSCP
>>>>> allows QOS PHB scheduling to not be broken which both Cisco and Juniper
>>>>> implements, allowing operators to keep the default PHP behavior and not
>>>>> have to signal explicit null Pipe mode to fix the scheduling issue.  So
>>>>> there is a quite a number of permutations of designs based on use case.
>>>>>
>>>> [IM4] Ok, that point's clear.
>>>>
>>>>>
>>>>>>> RFC 3031 section 4.1.5 describes the MPLS requirement to support
>>>>>>> implicit null value 3 signaling done via label mapping message from egress
>>>>>>> LER to PHP node (egress LSR) to POP the topmost label.  The MPLS standard
>>>>>>> does not specify that implicit null MUST be the default behavior, however
>>>>>>> most vendors have implemented as Default behavior.
>>>>>>>
>>>>>>
>>>>>>> Cisco and Juniper use implicit null value 3 signaled by LER to PHP
>>>>>>> node LSR (egress P) to pop the topmost outer label.  With Nokia implicit
>>>>>>> null is not default and has to be enabled manually see links below:
>>>>>>>
>>>>>> [IM3] See my point above.
>>>>>>
>>>>>
>>>>>      Gyan3> Ack
>>>>>
>>>>>>
>>>>>>> Cisco supports implicit null signaling for PHP by default link below.
>>>>>>>
>>>>>>>
>>>>>>> https://community.cisco.com/t5/mpls/implicit-null-and-explicit-null/td-p/1496792
>>>>>>>
>>>>>>>
>>>>>>> Juniper supports implicit null signaling by default link below.
>>>>>>>
>>>>>>>
>>>>>>> https://www.juniper.net/documentation/us/en/software/junos/mpls/topics/ref/statement/explicit-null-edit-protocols-mpls.html
>>>>>>>
>>>>>>>
>>>>>>> Nokia does NOT support implicit null signaling by default link
>>>>>>> below.  Since implicit null is a configured option it’s not enabled by
>>>>>>> default to signal PHP.  Given that is the case AFAIK, any arbitrary label
>>>>>>> can be preserved and thus you don’t have to use the explicit null label to
>>>>>>> preserve the topmost label.  All you need to do is disable PHP which is
>>>>>>> done by default my not signaling implicit null.  Cisco and Juniper as
>>>>>>> implicit null is default behavior PHP is signaled, however in order to
>>>>>>> disable PHP explicit null has to be configured.  So then the arbitrary
>>>>>>> label concept only comes into play with vendors that don’t support PHP by
>>>>>>> default like Nokia.
>>>>>>>
>>>>>>>
>>>>>>> https://infocenter.nokia.com/public/7750SR217R1A/index.jsp?topic=%2Fcom.nokia.MPLS_Guide_21.7.R1%2Fconfiguring_imp-ai9emdyowp.html
>>>>>>>
>>>>>>> Configuring Implicit Null Label
>>>>>>>
>>>>>>> The implicit null label option allows an egress LER to receive MPLS
>>>>>>> packets from the previous hop without the outer LSP label. The user can
>>>>>>> configure to signal the implicit operation of the previous hop is referred
>>>>>>> to as penultimate hop popping (PHP). This option is signaled by the egress
>>>>>>> LER to the previous hop during the FEC signaling by the LDP control
>>>>>>> protocol.
>>>>>>>
>>>>>>> Enable the use of the implicit null option, for all LDP FECs for
>>>>>>> which this node is the egress LER, using the following command:
>>>>>>>
>>>>>>> config>router>ldp>implicit-null-label
>>>>>>>
>>>>>>> When the user changes the implicit null configuration option, LDP
>>>>>>> withdraws all the FECs and re-advertises them using the new label value.
>>>>>>>
>>>>>>>>
>>>>>>>> As I previously said, Nokia does it by default for RSVP, LDP, and
>>>>>>>> BGP LSP. Huawei has a configuration option at least for LDP to turn this
>>>>>>>> feature on. From time to time there are bugs with handling labels 0,2 or 3
>>>>>>>> among different vendors. I had a very unpleasant experience with this too
>>>>>>>> before. Arbitrary labels allow us to fool the penultimate nodes and make
>>>>>>>> them regular LSRs. It also could work with egress LERs when they due to
>>>>>>>> a bug cannot handle either unlabeled traffic or traffic with explicit
>>>>>>>> labels. Personally, I consider PHP and UHP with explicit labels as
>>>>>>>> rudiments and prefer using arbitrary labels instead.
>>>>>>>>
>>>>>>>
>>>>>>>    Gyan2> See above what I mentioned about Nokia and PHP signaling
>>>>>>> via implicit null appears to not be enabled by default.
>>>>>>>
>>>>>> I have tested 0, 2, 3 with Cisco and Juniper and I have not run into
>>>>>>> any issues.  Based on what I stated above I agree the arbitrary label
>>>>>>> concept I believe only comes into play with vendors such as Nokia that do
>>>>>>> not have implicit null signaling to trigger the PHP node to POP the topmost
>>>>>>> transport label.
>>>>>>>
>>>>>> [IM3] I don't get the idea of the last sentence. Arbitrary labels (if
>>>>>> they are supported by a vendor in question) come into play when a network
>>>>>> engineer decides to turn them on. Their usage is a matter of configuration
>>>>>> and a design solution.
>>>>>>
>>>>>>
>>>>>      Gyan3> In the last sentence I forgot to say by “default” vendors
>>>>> such as Nokia that enable implicit null signaling for PHP to be enabled by
>>>>> default.  The main reason to use arbitrary labels is for UHP to preserve
>>>>> the topmost label to the egress LER.  Correct?
>>>>>
>>>> [IM4] Yes, that's correct.
>>>>
>>>>
>>>>> It’s the exact same reason to use explicit null signaling to preserve
>>>>> the topmost label so it’s not popped by the default implicit null PHP
>>>>> enabled mode.  Cisco / Juniper both support the latter scenario where
>>>>> implicit null is not configurable and cannot be explicitly enabled or
>>>>> disabled, however in order to switch from implicit null PHP mode to
>>>>> explicit null UHP mode you have to explicitly configure explicit null.  So
>>>>> for Cisco and Juniper, arbitrary labels are not possible as implicit null
>>>>> enable disable state from a configurable perspective does not exist as
>>>>> implicit null is enabled by default. Nokia for example does not support
>>>>> implicit null by default and has a configuration command to enable implicit
>>>>> null and thus by default arbitrary labels use case is possible.  That is
>>>>> why I say arbitrary labels usage and use case is based on vendor
>>>>> implementation for the designer to use.  Of course of its not implemented
>>>>> like with Cisco and Juniper for example it cannot be used.
>>>>>
>>>> [IM4] I got the main point of your statement above, that the use of
>>>> arbitrary labels is based on their support by a vendor in question. Other
>>>> vendors use explicit labels to achieve the same. I agree and always
>>>> emphasized the same. If I missed anything, please, let me know.
>>>>
>>>>>
>>>>>
>>>>> Since PHP is disabled on Nokia by default you can now fool the PHP
>>>>>>> node making it act as a regular LSR.  You are not actually fooling the PHP
>>>>>>> node, it’s just that implicit null is not signaled by default so PHP is
>>>>>>> disabled, so when LDP distributes the labels via downstream unsolicited
>>>>>>> from downstream to upstream nodes and the LSP is instantiated and LIB is
>>>>>>> allocated and LFIB data plane forwarding is programmed, the LSP is standard
>>>>>>> swapping on every LSR including the PHP nodes as it did not get signaled
>>>>>>> for implicit null to POP the transport label so acts as normal LSR swapping
>>>>>>> labels.  No fooling and is acting as it should with arbitrary labels.
>>>>>>>
>>>>>> [IM3] Well, I can not use this term if you don't like it :) By
>>>>>> fooling, I meant exactly what you stated above. So, looks like we are in
>>>>>> sync here.
>>>>>>
>>>>>
>>>>>      Gyan> Excellent! We are in sync here!
>>>>>
>>>>>>
>>>>>>>>  So once the  PHP node egress LSR does the “POP” processing the
>>>>>>>> outer topmost  label in the label stack is stripped or removed leaving the
>>>>>>>> exposed payload which if a 2 level label stack would now be a single level
>>>>>>>> label stack 1/4 and the labeled packet is successfully forwarded to the
>>>>>>>> egress LER (PE) that POPs the remaining labels in the stack in this case
>>>>>>>> the BOS bit set service label and the native IPv4 packet is forwarded to
>>>>>>>> IPv4 context PE-CE interface via standard forwarding.
>>>>>>>> [IM2] That's correct.
>>>>>>>>
>>>>>>>> So if it’s a single level label stack with topmost IPv6 arbitrary
>>>>>>>> label once the PHP processing is completed “POP” operation stripping or
>>>>>>>> removing outer topmost IPv6 arbitrary label the IPv4 payload is now exposed
>>>>>>>> in the case of 1/1 service and the unlabeled IPv4 packet is dropped by the
>>>>>>>> PHP node as it cannot be forwarded onto the egress IPv6 only  core
>>>>>>>> interface facing the egress LER (PE). That’s the problem that we would
>>>>>>>> address in this specification with 4PE.
>>>>>>>> [IM2] That's not.
>>>>>>>>
>>>>>>>
>>>>>>>     Gyan2> Updated arbitrary label use case.  So here  we not
>>>>>>> signaling PHP with implicit null as some vendors do not signal implicit
>>>>>>> null value 3 by default so PHP is disabled.  So now the topmost arbitrary
>>>>>>> label is preserved to the egress LER which pops the entire stack and
>>>>>>> forwards the native IPv4 packet to IPv4 routing context.
>>>>>>>
>>>>>> [IM2] In general, yes, except I'm not sure an IPv4 packet is
>>>>>> forwarded to the IPv4 routing context for every implementation. From my
>>>>>> understanding, this is vendor-depended behavior. If there is a normative
>>>>>> reference to this procedure, please, let me know!
>>>>>>
>>>>>>>
>>>>>>>>> If the egress LER signaled explicit null pipe mode to the egress
>>>>>>>>> LSR PHP node, the PHP node seeing the value 2 IPv6 explicit null label
>>>>>>>>> being signaled and would preserve the topmost IPv6 label in pipe mode and
>>>>>>>>> forward the packet now still with topmost IPv6 label intact preserved over
>>>>>>>>> the IPv6 core interface to the egress LER (PE) which would receive the
>>>>>>>>> single level label stack and now pop the remaining label in this case the
>>>>>>>>> explicit null value 2 topmost label and forward the native IPv4 packet to
>>>>>>>>> the outgoing PE-CE ipv4 routing context.
>>>>>>>>>
>>>>>>>> [IM2] No, it does not forward an IPv4 packet after the stripping of
>>>>>>>> the label of 2. I've just checked it on a Juniper, it drops all such
>>>>>>>> traffic until we turn off the explicit label (I tried with LDPv6 and family
>>>>>>>> inet without an address for the incoming interface). Nokia does the
>>>>>>>> additional packet inspection for arbitrary labels that belong to IPv6 LSPs
>>>>>>>> and forwards IPv4 packets. I cannot check it with an explicit label, looks
>>>>>>>> like it is not supported or I failed to find it.
>>>>>>>>
>>>>>>>
>>>>>>>     Gyan2> I believe IPv6 explicit null label as topmost once popped
>>>>>>> cannot determine the network layer protocol of the unlabeled IPv4
>>>>>>> underneath and so drops the packet.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>> But it appear from your test that when using topmost arbitrary label
>>>>>>> it can determine the network layer protocol type.
>>>>>>>
>>>>>> [IM3] Yes, but we should use this information very cautiously. We
>>>>>> don't know the internals of one or another implementation. I'll repeat
>>>>>> myself: some vendors do additional packet inspection, while others do not.
>>>>>>
>>>>>
>>>>>      Gyan3>  As this use case digs deep into vendor implementation
>>>>> internals I think it’s fair to exclude 1/1 unlabeled use case under the
>>>>> topmost both arbitrary or explicit null cases due to the variation in deep
>>>>> packet inspection being supported or not by vendors.
>>>>>
>>>> [IM4] I think that the draft should do exactly the opposite. The
>>>> excluding of this case creates some undefined behavior. It is better to
>>>> describe this case with the known pitfalls in detail and warn readers.
>>>>
>>>>
>>>>> Based on this we are in sync for 4PE!
>>>>>
>>>>> 1/1 unlabeled is possible but due variance in support or not from 4PE
>>>>> specification perspective we would have the following use cases:
>>>>>
>>>>> 3 use cases can have topmost be arbitrary or explicit null.
>>>>>
>>>>>   1st - All CE global table prefixes are labeled
>>>>>   2nd - PE loopback to PE loopback LSP BOS set that contains all the
>>>>> CE global table prefixes unlabeled
>>>>>   3rd - Per CE PE-CE interface label table BOS set  IPv4 routing
>>>>> context that contains the respective CE prefixes unlabeled
>>>>>
>>>>
>>>>> 3 use cases of PHP implicit null - have you tested this ?  This should
>>>>> work
>>>>>
>>>>>  1st - All CE global table prefixes are labeled
>>>>>   2nd - PE loopback to PE loopback LSP BOS set that contains all the
>>>>> CE global table prefixes unlabeled
>>>>>   3rd - Per CE PE-CE interface label table BOS set  IPv4 routing
>>>>> context that contains the respective CE prefixes unlabeled
>>>>>
>>>> [IM4] I'm OK with all the above. I've tested PHP for underlying LSPs
>>>> when there is an overlay label at the bottom, and it works.
>>>>
>>>> For the arbitrary or explicit null 1/1  unlabeled IPv4 prefixes case
>>>>> confirming we will not include due to the deep packet inspection variable
>>>>> or maybe could mention with the implementation requirement of deep packet
>>>>> inspection to infer the network protocol type?
>>>>>
>>>> [IM4] If it is possible to describe these cases as less preferred with
>>>> a detailed explanation of why, I'm OK with this option. Talking about the
>>>> requirement for DPI, I'm not sure about it. To me, it is too uncertain,
>>>> there should be a procedure for such inspection. It is possible to mention,
>>>> that vendors can analyze a network layer header and consider forwarding
>>>> traffic.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> RFC 3032 section 2.2 talks about determining the network layer
>>>>>>> protocol type of the different protocol type exposed underneath topmost
>>>>>>>  once the topmost is popped and it says that the network layer protocol
>>>>>>> type is inferred from the value of label at bottom of stack. I think since
>>>>>>> explicit null is a special label it cannot infer the protocol type but when
>>>>>>> using arbitrary it can as proved from your test.
>>>>>>>
>>>>>> [IM3] No, that is not what I proved. Please, be careful here. Let me
>>>>>> explain my point broader. From RFC 3032 (section 2.2) and specifically
>>>>>> mentioned by you RFC 4182 (section 2), we can understand that the explicit
>>>>>> labels, if and only if they have a BoS set, point exactly to the network
>>>>>> protocol they belong to.
>>>>>>
>>>>>
>>>>>> RFC4182 (cases for labels with BoS):
>>>>>>
>>>>>>        If, on the other hand, the NULL was the only label on the
>>>>>>           stack, then the stack is now empty.  The resulting packet is
>>>>>>           treated as an IPv4 packet, and its disposition is based on the
>>>>>>           IPv4 header.
>>>>>>
>>>>>>
>>>>>>           If, on the other hand, the NULL was the only label on the
>>>>>>           stack, then the stack is now empty.  The resulting packet is
>>>>>>           treated as an IPv6 packet, and its disposition is based on the
>>>>>>           IPv6 header.
>>>>>>
>>>>>>
>>>>>
>>>>>     Gyan3> So for IPv6 explicit null it’s expecting an IPv6 packet and
>>>>> not IPv4 so the packet would be dropped unless the implementation supports
>>>>> deep packet inspection to infer the protocol type underneath.  What you
>>>>> quoted from RFC 4182 section 2 is it says null is the only label in the
>>>>> label stack so then of course no label with BOS.
>>>>>
>>>> [IM4] Not exactly. When there is only one label in the stack, this
>>>> label must come with BoS. Also, from my understanding, this section talks
>>>> not about the exact number of labels for an LSP, but about the parsing
>>>> process when a parser eventually has the last label from the stack. I think
>>>> it does not matter how many labels there are prior to an explicit null (0
>>>> or 2) if the latter is at the bottom. These labels (except the mentioned
>>>> explicit one) will be stripped anyway. So, at the end of the day, a LER has
>>>> only one label and this label is of explicit type with a BoS set.
>>>>
>>>>
>>>>> This is the 1/1 use case unlabeled IPv4 packets so no BOS would be
>>>>> present since it’s a single level label stack. Please clarify.
>>>>>
>>>>  [IM4] Yes, the use case is correct, but BoS is present even if this is
>>>> a single-level label stack. BoS notifies a system to stop considering the
>>>> next bytes after an LSE as MPLS header-related and start processing a
>>>> network layer header.
>>>>
>>>>
>>>>>> So, I expect an implementation to drop any IPv4 packets above the
>>>>>> label of 2 and any IPv6 packets above the label of 0 according to these
>>>>>> RFCs as the default behavior. However, according to the RFC3032, section
>>>>>> 2.2:
>>>>>>
>>>>>
>>>>>     Gyan3>. This is for the 1/1 not 1/4 (BOS) set with explicit null
>>>>> topmost label so I would expect the IPv4 packets to be dropped as that is
>>>>> based on the vendor implementation of deep packet inspection or not.
>>>>>
>>>> [IM4] Yes, agree. This is for 1/1 and, depending on a vendor DPI's
>>>> implementation, packets may be dropped or not.
>>>>
>>>>
>>>>> This is the 1/1 use case so no labeled IPv4 prefixes but you are
>>>>> mentioning >=2 . We are talking 4PE which is IPv4 packets over IPv6 core
>>>>> not 6PE which is IPv6 packets over IPv4 core, so in 6PE case as well the
>>>>> IPv6 packets would be dropped.
>>>>>
>>>> [IM4] Yes, that's exactly what I tried to state. No matter what type of
>>>> explicit label we have, if we do have not the expected network layer
>>>> header, traffic may be dropped. This is true both for 4PE and 6PE when the
>>>> explicit label is at the bottom of the stack.
>>>>
>>>>
>>>>> So in both 4PE or 6PE unless the implementation supports deep packet
>>>>> inspection the 1/1 unlabeled packets would be dropped.
>>>>>
>>>> [IM4] Yes.
>>>>
>>>>
>>>>>
>>>>>
>>>> This means that the identity of the network layer protocol
>>>>>>    must be inferable from the value of the label which is popped from
>>>>>>    the bottom of the stack, possibly along with the contents of the
>>>>>>    network layer header itself.
>>>>>>
>>>>>>
>>>>>> We see that also vendors *can *analyze a network layer header, but
>>>>>> it is complementary rather than the general behavior (see *possibly*).
>>>>>> As I understand it, that is exactly what I have seen with the arbitrary
>>>>>> label on Nokia. From my POV, it is also true for the explicit labels. But I
>>>>>> think there is no guarantee, that each vendor adheres to this.
>>>>>>
>>>>>
>>>>>    Gyan4> Understood on the deep packet inspection and that it’s
>>>>> complementary optional but not generally implemented.  Agreed.  The thread
>>>>> above was a bit confusing see above that we are talking about 1/1 not 1/4
>>>>> so it’s unlabeled IPv4 prefixes with arbitrary or explicit topmost label.
>>>>>
>>>> [IM4] Yes, we discussed the case with unlabeled IPv4 traffic, because,
>>>> from my POV, with labeled IPv4 traffic, we were already in sync.
>>>>
>>>>
>>>>>
>>>>>>   So I can mention this as a valid use case in the draft which is
>>>>>>> good reason for arbitrary label.  So IPv6 explicit null is valid and not
>>>>>>> illegal with the RFC 4182 update in 2005 to be used for any level of label
>>>>>>> stack including topmost, however for case where the network layer under the
>>>>>>> topmost is unlabeled you have to use arbitrary label for the IPv4 packet to
>>>>>>> be inferred and not dropped.
>>>>>>>
>>>>>> [IM3] I will highlight it here too for additional clearance. Neither
>>>>>> arbitrary nor explicit labels guarantee us that the IPv4 payload will be
>>>>>> forwarded. That is why I think, the draft should clearly warn readers.
>>>>>>
>>>>>
>>>>>      Gyan3> Ack and just to be crystal clear again this is 1/1 use
>>>>> case for arbitrary or explicit null topmost.
>>>>>
>>>> [IM4] Yes!
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>> [IM] For example, RFC3032 says:
>>>>>>>>
>>>>>>>> A value of 2 represents the "IPv6 Explicit NULL Label".
>>>>>>>>               This label value is only legal at the bottom of the label
>>>>>>>>               stack.  It indicates that the label stack must be popped,
>>>>>>>>               and the forwarding of the packet must then be based on the
>>>>>>>>               IPv6 header.
>>>>>>>>
>>>>>>>>
>>>>>>>> Despite I believe that requirement of the bottomness is kinda
>>>>>>>> relaxed these days, anyway, it indicates that after the label of 2, there
>>>>>>>> must be the IPv6 header, not IPv4. So, as I said, some boxes expect only
>>>>>>>> IPv6 traffic and drop everything else if there is a label of 2 with a BoS
>>>>>>>> set. Others could check packets deeper and forward them.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> No IPv4 dropped packets with IPv6 explicit null signaling value 2
>>>>>>>>> but with arbitrary label we drop all IPv4 packets on the PHP node after the
>>>>>>>>> “POP” operation.
>>>>>>>>>
>>>>>>>> [IM] As I explained above, in both cases -- no.
>>>>>>>>
>>>>>>>>
>>>>>>>>> The egress node can do additional packet inspection and avoid the
>>>>>>>>>> dropping, but there is no guarantee. This warning should be described in
>>>>>>>>>> the document alongside the warning of the PHP case for single-labeled
>>>>>>>>>> traffic.
>>>>>>>>>> Also, if this document tried to invent and describe procedures on
>>>>>>>>>> how to not drop IPv4 unlabeled traffic coming from IPv6 LSPs, I would say,
>>>>>>>>>> that deserves to be a Standard :)
>>>>>>>>>>
>>>>>>>>>> Summary:
>>>>>>>>>> * For single-labeled stacks:
>>>>>>>>>> * * If the topmost label is an explicit one (value 2) or an
>>>>>>>>>> arbitrary one, a drop is possible at an egress PE. The PE could think that
>>>>>>>>>> a packet behind the label is IPv6, if there is no additional packet
>>>>>>>>>> inspection, the drop will occur.
>>>>>>>>>> * * In the PHP case a drop is possible at the penultimate PE when
>>>>>>>>>> the topmost label is stripped of the payload. The PE could check the
>>>>>>>>>> outgoing interface's families and does not allow IPv4 packets there.
>>>>>>>>>> * For two or more labeled stacks:
>>>>>>>>>> * * A bottom label must be present and be either an arbitrary one
>>>>>>>>>> or explicit (0).
>>>>>>>>>> * * Any labels above can be present in a stack with any values or
>>>>>>>>>> cannot be present at all.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> The use case of topmost explicit null or implicit null default
>>>>>>>>>>> will both work with PE loopback LSP or per CE PE-CE interface LSP context
>>>>>>>>>>> label table.  We will allow topmost label PHP with no restriction as this
>>>>>>>>>>> is possible with the permutation of use cases using 1/4 for service label
>>>>>>>>>>> labeling all prefixes one bucket for all global table prefixes,  as well ws
>>>>>>>>>>> works for your use case PE loopback LSP or per CE PE-CE interface LSP
>>>>>>>>>>> carrying the per CE unlabeled CE prefixes.  As long as the service prefixes
>>>>>>>>>>> are labeled allowing PHP for topmost label is valid permutation without any
>>>>>>>>>>> risk of payload traffic drops.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [IM] Yes, agree.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Let me know if you are good with what I have stated.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [IM] I'm good with that, yes.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> As far as standards track I will discuss with the
>>>>>>>>>>> co-authors.
>>>>>>>>>>>
>>>>>>>>>>> I will discuss if in shipping code what is implemented in detail
>>>>>>>>>>> and with what is described below if there is anything that requires
>>>>>>>>>>> standardization even though existing mechanisms are being used that if any
>>>>>>>>>>> procedural or any differences could cause interoperability issues.  One of
>>>>>>>>>>> the major goal of the IETF standardization is mix vendor interoperability.
>>>>>>>>>>> So if this can be achieved as informational draft or based on
>>>>>>>>>>> interoperability issues found with current or future vendor implementation
>>>>>>>>>>> do we need to have a standards track draft or not to meet the goal.
>>>>>>>>>>> Normative language only comes into play for standards track draft so since
>>>>>>>>>>> we do have normative language RFC 2119 does that mean this document should
>>>>>>>>>>> be standards track.
>>>>>>>>>>>
>>>>>>>>>>> I will discuss this topic with the chairs as well.
>>>>>>>>>>>
>>>>>>>>>>> Many Thanks!
>>>>>>>>>>>
>>>>>>>>>>> Gyan
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Aug 2, 2023 at 6:32 AM Igor Malyushkin <
>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Gyan.
>>>>>>>>>>>>
>>>>>>>>>>>> When I'm talking about transport, I mean some underlay, i.e.,
>>>>>>>>>>>> LSPs that points to remote PEs. They could be RSVP/LDP/SR-MPLS in a single
>>>>>>>>>>>> domain or the exact same options with BGP LSP (IPv6) as E2E LSP through
>>>>>>>>>>>> several domains. This draft should not restrict any label operations in any
>>>>>>>>>>>> way for this set. That is what I meant for PHP. With PHP for these LSPs, we
>>>>>>>>>>>> can face the situation when the IPv4 payload is exposed at the penultimate
>>>>>>>>>>>> node.
>>>>>>>>>>>>
>>>>>>>>>>>> To avoid the problem with the exposure of the payload this
>>>>>>>>>>>> draft offers to use the next level of LSPs -- IPv4 labeled unicast LSPs.
>>>>>>>>>>>> And we have spent some time discussing how to organize them. This is where
>>>>>>>>>>>> I was talking about the separation of reachability from next-hops
>>>>>>>>>>>> signaling. I didn't mean PHP for these LSPs. Here we can use either an
>>>>>>>>>>>> explicit label when we want to make an IPv4 lookup in the IPv4 global
>>>>>>>>>>>> routing table or an arbitrary label for the rest of the cases (i.e.,
>>>>>>>>>>>> per-CE, per-prefix, it also supports per-table, SR OS uses it by default
>>>>>>>>>>>> for per-table). This is some form of overlay.
>>>>>>>>>>>>
>>>>>>>>>>>> For both, underlay and overlay, explicit labels *and *arbitrary
>>>>>>>>>>>> labels are possible and the latter do not break anything. I hope we are
>>>>>>>>>>>> synchronized at this point. For overlay, the selection could be done with a
>>>>>>>>>>>> variable granularity. Many vendors allow us to set a per-table label (it
>>>>>>>>>>>> can be an explicit one but not necessarily) for the most prefixes, and set
>>>>>>>>>>>> dedicated labels for some smaller groups of routes that belong to some
>>>>>>>>>>>> interfaces of this table (per-CE). It does not depend on whether we send
>>>>>>>>>>>> all customer's routes as labeled or just next-hops for them.
>>>>>>>>>>>>
>>>>>>>>>>>> Third, despite this draft trying to save people from some
>>>>>>>>>>>> pitfalls (losing frames at the penultimate node when the payload is
>>>>>>>>>>>> unlabeled after PHP with a single-labeled stack's LSPs) I still think that
>>>>>>>>>>>> these options shouldn't be excluded from the consideration. If we advertise
>>>>>>>>>>>> IPv4 1/1 routes with IPv6 NHs and PHP is not done, nothing bad happens,
>>>>>>>>>>>> right? So, from my POV, it is better not to delve into the kingdom of
>>>>>>>>>>>> restrictions, but to make a clear statement that with PHP for
>>>>>>>>>>>> transport/underlay in the case of a single-labeled stack, there is a
>>>>>>>>>>>> dangerous situation. If a user applies a two-labeled or more stack, PHP is
>>>>>>>>>>>> possible for all labels prior to the bottom one. So there is a simple
>>>>>>>>>>>> condition for PHP depending on the depth of the stack, and no need to
>>>>>>>>>>>> restrict a user with the exact depth.
>>>>>>>>>>>>
>>>>>>>>>>>> In summary.
>>>>>>>>>>>> * The draft should remove all "MUST" corresponding to the
>>>>>>>>>>>> procedures of sending customers' routes and the selection of AFI/SAFI for
>>>>>>>>>>>> them (IPv4 1/1 vs. 1/4).
>>>>>>>>>>>> * The draft should not restrict using any labels for transport
>>>>>>>>>>>> LSPs (arbitrary vs. explicit).
>>>>>>>>>>>> * The draft should describe the case when the penultimate node
>>>>>>>>>>>> drops traffic, warn readers, and suggest (not mandate) a more viable
>>>>>>>>>>>> alternative.
>>>>>>>>>>>> * With this alternative (IPv4 LU) there should not be strict
>>>>>>>>>>>> requirements for sending all customers' routes via 1/4, for some cases, the
>>>>>>>>>>>> signaling of next-hops is also suitable.
>>>>>>>>>>>> * The draft should be informational, all the mechanics are
>>>>>>>>>>>> already available, and the notion of the depth of a label stack is not
>>>>>>>>>>>> something new.
>>>>>>>>>>>> * The draft should consider other options, not related to MPLS,
>>>>>>>>>>>> or related but with nowadays features (e.g., TEA).
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ср, 2 авг. 2023 г. в 04:13, Gyan Mishra <[e-mail]
>>>>>>>>>>>> >:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are making a lot of progress here!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Responses in-line Gyan5>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 1, 2023 at 5:42 PM Igor Malyushkin <
>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>        Gyan3> I agree this sentence is a bit confusing with
>>>>>>>>>>>>>> regards to label 0 and 2 use.  We are not using IPv4 explicit null value 0
>>>>>>>>>>>>>> at all in this draft.  So the conveying of the next hop and what the
>>>>>>>>>>>>>> sentence it talking about related to the IPv6 LSP egress PE signaling done
>>>>>>>>>>>>>> via the IPv6 explicit null label value 2 defined in RFC 4182 to use for
>>>>>>>>>>>>>> topmost label so the next hop of the  advertised IPv4 routes have an IPv6
>>>>>>>>>>>>>> next hop as defined according to RFC 8950 next hop encoding which is an
>>>>>>>>>>>>>> IPv6 labeled address per BGP LU RFC 8277 for the BOS IPv4 prefixes. So here
>>>>>>>>>>>>>>  we are talking about using label value 2 IPv6 explicit null to identify
>>>>>>>>>>>>>> the IPv4 routing context of the outgoing interface since the next hop is an
>>>>>>>>>>>>>> IPv6 next hop for the labeled IPv4 prefixes.  I will cleanup the wording.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Now it is more clear to me. Thank you. I've already expressed
>>>>>>>>>>>>>> my thoughts on restriction of the transport. I strongly disagree.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    Gyan5> Understood and points and use cases and reasons for
>>>>>>>>>>>>> flexibility of arbitrary label are well taken.
>>>>>>>>>>>>> Just to make sure I am crystal clear.  The use of topmost
>>>>>>>>>>>>> transport IPv6 explicit null label does not allow you to create the per CE
>>>>>>>>>>>>> / LSP IPv4 context tables or does it.  If so what is the technical reason
>>>>>>>>>>>>> why you cannot label the loopbacks or CE interface creating an IPv4 LSP to
>>>>>>>>>>>>> tunnel the per CE prefixes unlabeled.  I missed that if that’s the case so
>>>>>>>>>>>>> want to understand why your use case is not possible with topmost transport
>>>>>>>>>>>>> label  IPv6 explicit null.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe you would like the flexibility of PHP as you have
>>>>>>>>>>>>> use cases where PHP is necessary.   However that is separate from your
>>>>>>>>>>>>> primary use case of per CE / LSP IPv4 context tables providing scalability
>>>>>>>>>>>>> and faster convergence.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> P.S. Label value 2 cannot identify the IPv4 routing context.
>>>>>>>>>>>>>> With BoS bit it identifies the IPv6 routing context. Without BoS bit this
>>>>>>>>>>>>>> label is stripped and a lookup is done for the next label until the bottom
>>>>>>>>>>>>>> is reached. The bottom label actually identifies the context in this case.
>>>>>>>>>>>>>> So, it is also not the best wording I believe.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    Gyan5> Agreed will fix
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> вт, 1 авг. 2023 г. в 21:20, Gyan Mishra <
>>>>>>>>>>>>>> [e-mail]>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Response on the remaining item I missed in your first email
>>>>>>>>>>>>>>> - Gyan3>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Gyan
>>>>>>>>>>>>>>> On Tue, Aug 1, 2023 at 11:02 AM Gyan Mishra <
>>>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Responses in-line Gyan2>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jul 31, 2023 at 5:20 AM Igor Malyushkin <
>>>>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Gyan,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> My responses are also inline (IM>).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> пн, 31 июл. 2023 г. в 09:07, Gyan Mishra <
>>>>>>>>>>>>>>>>> [e-mail]>:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Igor
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Many thanks to your questions to clarify points in the
>>>>>>>>>>>>>>>>>> draft.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Responses in-line Gyan>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sun, Jul 30, 2023 at 8:40 AM Igor Malyushkin <
>>>>>>>>>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Gyan,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have several questions w.r.t your draft. But first I
>>>>>>>>>>>>>>>>>>> would like to clarify some points from your summary.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "*Customer IPv4 prefixes must be labeled when tunneled
>>>>>>>>>>>>>>>>>>> over IPv6 LSP*". I believe there is a misconception
>>>>>>>>>>>>>>>>>>> between the "labeled prefixes" and the "labeled traffic". Prefixes cannot
>>>>>>>>>>>>>>>>>>> be tunneled via LSPs, traffic can be. Customer IPv4 prefixes can be
>>>>>>>>>>>>>>>>>>> dissipated as vanilla IPv4 prefixes without labels. I described the reasons
>>>>>>>>>>>>>>>>>>> during the previous discussion of this document.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Gyan> I was referring to the bottom of stack service
>>>>>>>>>>>>>>>>>> labeling of the IPv4 customer prefixes which the traffic flows are within
>>>>>>>>>>>>>>>>>> the payload of packet and not the label stack.  I can change the verbiage
>>>>>>>>>>>>>>>>>> to labeled traffic. Good point.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "*If not labeled on the PHP node when the topmost label
>>>>>>>>>>>>>>>>>>> is popped the native IPv4 prefix is exposed and is not routable and will be
>>>>>>>>>>>>>>>>>>> dropped*", this sentence describes traffic, not BGP
>>>>>>>>>>>>>>>>>>> prefixes, and I'm sure we can reach the goal of having two and more labels
>>>>>>>>>>>>>>>>>>> in a stack without advertising tons of IPv4 reachability with labels. We
>>>>>>>>>>>>>>>>>>> can solely use IPv4 labeled prefixes also, as this draft describes, but
>>>>>>>>>>>>>>>>>>> there are other options too.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Gyan> What I am describing is the native IPv4
>>>>>>>>>>>>>>>>>> prefixes under the topmost transport IPv6 label once popped that are
>>>>>>>>>>>>>>>>>> exposed which the protocol is IPv4, where  the core P nodes are IPv6 so the
>>>>>>>>>>>>>>>>>> IPv4 Paulo’s packets will be dropped.  Imagine the scenario where you have
>>>>>>>>>>>>>>>>>> an IPv4 core and have native IPv4 to the edge so in that case you would
>>>>>>>>>>>>>>>>>> have native IPv4 Unicast prefixes 1/1 over a IPv4 core IPv4 LSP topmost
>>>>>>>>>>>>>>>>>> label, and with IPv6 core IPv6 LSP topmost label would have native IPv4
>>>>>>>>>>>>>>>>>> Unicast prefixes 1/1 over IPv6 core.  I can rewrite as “Native IPv4
>>>>>>>>>>>>>>>>>> prefixes are exposed and non routable and IPv4 payload traffic is dropped.
>>>>>>>>>>>>>>>>>> RFC 3032 section 2.2 states that the bottom of stack prefixes if a
>>>>>>>>>>>>>>>>>> different protocol  then the topmost label as in this case must be labeled
>>>>>>>>>>>>>>>>>> to identify the protocol type of the prefixes.  We can discuss the other
>>>>>>>>>>>>>>>>>> options.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "*RFC 7948 “6PE” states that the label bound to the
>>>>>>>>>>>>>>>>>>> IPv4 prefixes may be an arbitrary value or explicit null label which has
>>>>>>>>>>>>>>>>>>> led to vendor interoperability issues in the past*".
>>>>>>>>>>>>>>>>>>> Can you please elaborate on that topic? I know the exact opposite cases.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> Issue arises if one vendor uses arbitrary label
>>>>>>>>>>>>>>>>>> and the other vendor uses IPv4 explicit null label value 1. AFAIK  Both
>>>>>>>>>>>>>>>>>> sides using to arbitrary value or both sides signaling explicit null is ok
>>>>>>>>>>>>>>>>>> but with mismatch there is an issue.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> The selection of a value for a label is a matter of
>>>>>>>>>>>>>>>>> choice of a node that allocates the value for the label. I'm lost what you
>>>>>>>>>>>>>>>>> mean by "both sides". There is no such conception as "mismatch" because
>>>>>>>>>>>>>>>>> LSRs do not negotiate label values among themselves in general. Yes, there
>>>>>>>>>>>>>>>>> are examples of the opposite but for peculiar cases that do not interfere
>>>>>>>>>>>>>>>>> with our topic.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    Gyan2>Unidirectional LSP so no concept of both sides
>>>>>>>>>>>>>>>> -agreed. One side could signal LSP using arbitrary label and other side
>>>>>>>>>>>>>>>> explicit null. There is not related to mismatch and is some other peculiar
>>>>>>>>>>>>>>>> case which I have to dig up.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "*4PE draft states that the IPv4 prefixes must use MPLS
>>>>>>>>>>>>>>>>>>> QOS RFC 3270 Pipe mode explicit null label bound to the IPv4 prefix and
>>>>>>>>>>>>>>>>>>> MUST be used ...*". In my opinion, the idea of getting
>>>>>>>>>>>>>>>>>>> rid of arbitrary labels breaks a lot of stuff. I will clarify it later.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> Yes please clarify
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> Well, I've done it below in my previous message.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "*Additional text clarity added related RFC 8950 next
>>>>>>>>>>>>>>>>>>> hop encoding...*". Honestly, I don't understand why. It
>>>>>>>>>>>>>>>>>>> is just a rephrase of the stuff from the original standard. Can you shed
>>>>>>>>>>>>>>>>>>> light on the necessity of this section? Maybe I'm missing something.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Here and after my comments w.r.t the body of the draft.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Gyan> It’s just for reference stated explicitly in the
>>>>>>>>>>>>>>>>>> draft but I agree it can be removed
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. Section 1. "*This document explains the "4PE" design
>>>>>>>>>>>>>>>>>>> procedures and how to interconnect IPv4 islands over a Multiprotocol Label
>>>>>>>>>>>>>>>>>>> Switching (MPLS) [RFC3031] LDPv6 enabled IPv6-Only core, Segment Routing
>>>>>>>>>>>>>>>>>>> (SR) enabled SR-MPLS [RFC8660] IPv6-Only core or SRv6 [RFC8986] IPv6-Only
>>>>>>>>>>>>>>>>>>> core*". Is my understanding correct that the 4PE
>>>>>>>>>>>>>>>>>>> solution does not support RSVP-TE LSP signaled to IPv6 tail-ends over an
>>>>>>>>>>>>>>>>>>> IPv6-routed core? I see LDPv6 here, but the link goes to RFC3031 which is
>>>>>>>>>>>>>>>>>>> the architecture standard that applies not only to LDP. By the way,
>>>>>>>>>>>>>>>>>>> 6PE/6vPE perfectly works with RSVP LSPs.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan>  RFC 3031 is referenced for MPLS data plane but
>>>>>>>>>>>>>>>>>> I did not specify LDP spec which I can add to be consistent.  Further down
>>>>>>>>>>>>>>>>>> in the draft I do mention RSVP-TE but forgot to mention in above text so
>>>>>>>>>>>>>>>>>> will add.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2. Section 1. "*The 4PE routers exchange the IPv4
>>>>>>>>>>>>>>>>>>> reachability information transparently over the core using the
>>>>>>>>>>>>>>>>>>> Multiprotocol Border Gateway Protocol (MP-BGP) over IPv6. In doing so, the
>>>>>>>>>>>>>>>>>>> BGP Next Hop field egress PE FEC (Forwarding Equivalency Class) is used to
>>>>>>>>>>>>>>>>>>> convey the IPv6 address of the 4PE router learned dynamically via IGP...*".
>>>>>>>>>>>>>>>>>>> Can a 4PE router, that resides in one IGP domain, use a next-hop address
>>>>>>>>>>>>>>>>>>> received from the other domain via, for example, IPv6 labeled unicast?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> Yes it can using as you stated using BGP-LU is
>>>>>>>>>>>>>>>>>> exactly how it’s done with inter-as option C where the next hops are
>>>>>>>>>>>>>>>>>> imported between the IGP domains.  See section 7.3.2.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> "Between the IGP domains" does not mean "between
>>>>>>>>>>>>>>>>> ASes". From my understanding, Option C describes the latter, I was asking
>>>>>>>>>>>>>>>>> about the former. So my question is still intact. Can addresses for BGP
>>>>>>>>>>>>>>>>> next-hops be learned via means different from IGP as you stated in the body
>>>>>>>>>>>>>>>>> of the doc? Consider the Seamless MPLS approach. My point is that you
>>>>>>>>>>>>>>>>> should delete "... via IGP", not only IGP can do that, but BGP also can.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Gyan2> Got it will update
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 3. Section 1. "...* over an MPLS [RFC3031] LDP IPv6
>>>>>>>>>>>>>>>>>>> core, Segment Routing SR-MPLS [RFC8660] IPv6 core or SRv6 [RFC8986] IPv6
>>>>>>>>>>>>>>>>>>> core*". This part is a repetition of the same that was
>>>>>>>>>>>>>>>>>>> done previously, and it will be repeated several times later. Moreover,
>>>>>>>>>>>>>>>>>>> there are standard numbers enclosed after every term again and again. As I
>>>>>>>>>>>>>>>>>>> understand, it is enough to do it one time per every desired abbreviation
>>>>>>>>>>>>>>>>>>> during the whole body.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Gyan> Agreed will fix
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 4. Section 1. "*The approach requires that the Provider
>>>>>>>>>>>>>>>>>>> Edge (PE) routers Provider Edge - Customer Edge (PE-CE) connections to
>>>>>>>>>>>>>>>>>>> Customer Edge (CE)*". Honestly, it is difficult to read
>>>>>>>>>>>>>>>>>>> such things. From my POV, there should be a section with all unique
>>>>>>>>>>>>>>>>>>> definitions pertaining to this draft (e.g., 4PE router). Moreover, things
>>>>>>>>>>>>>>>>>>> such as the PE, and CE are well-known and do not require unfolding. There
>>>>>>>>>>>>>>>>>>> is a document of such well-known terms on the IETF portal for your
>>>>>>>>>>>>>>>>>>> convenience. The last thing, as I see the same abbreviations are unfolded
>>>>>>>>>>>>>>>>>>> many times during the document, which is contrary to the idea of an
>>>>>>>>>>>>>>>>>>> abbreviation in general. Once explained, an abbreviation should be used
>>>>>>>>>>>>>>>>>>> without further explanation.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Gyan> Agreed will fix
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 5. Section 3. "*...the 4PE IPv6 address Loopback0 MUST
>>>>>>>>>>>>>>>>>>> to be routable within the IPv6 core*". What is "the 4PE
>>>>>>>>>>>>>>>>>>> IPv6 address Loopback0"? It was not previously defined in the document.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Gyan> Will define and make clear
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 6. Section 3. "*Every ingress 4PE router can signal an
>>>>>>>>>>>>>>>>>>> IPv6 MPLS [RFC3031] LSP, SRMPLS [RFC8660] LSP or instantiate an SRv6 Best
>>>>>>>>>>>>>>>>>>> Effort (BE) or Segment Routing Traffic Engineering (SR-TE) [RFC9256] path
>>>>>>>>>>>>>>>>>>> to send to any egress 4PE router*". It is not necessary
>>>>>>>>>>>>>>>>>>> for an ingress router to signal an LSP. For example, LDP DU signals LSPs
>>>>>>>>>>>>>>>>>>> from a downstream towards all possible upstreams. RSVP or LDP DoD acts the
>>>>>>>>>>>>>>>>>>> opposite (if we talk about the intention, not a label distribution). So it
>>>>>>>>>>>>>>>>>>> is better to rephrase this sentence as "every ingress 4PE router can have
>>>>>>>>>>>>>>>>>>> an LSP toward..." or something like that.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>   Gyan> I was speaking generically not actually but is
>>>>>>>>>>>>>>>>>> confusing as you stated not correct- so will fix with the LDP downstream
>>>>>>>>>>>>>>>>>> unsolicited and RSVP-TE DOD.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> Well, one can call me too pedantic, but I believe that
>>>>>>>>>>>>>>>>> technical documentation especially the one that wants to be a Standard must
>>>>>>>>>>>>>>>>> be streamlined and precise.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     Gyan2> Yep Understood
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 7. Section 3. "*...the IPv6 signaled next hop Loopback0
>>>>>>>>>>>>>>>>>>> used to identify the Ingress and Egress 4PE router*".
>>>>>>>>>>>>>>>>>>> Now the Loopback0 is somehow connected to some next-hop. I believe this
>>>>>>>>>>>>>>>>>>> connection should be clarified too.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>      Gyan> Will fix
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 8. Section 3. "*In doing so, the 4PE routers convey
>>>>>>>>>>>>>>>>>>> their IPv6 address FEC label binding as the BGP Next Hop for the advertised
>>>>>>>>>>>>>>>>>>> IPv4 prefixes*". The term FEC is pretty general, but
>>>>>>>>>>>>>>>>>>> the "IPv6 address FEC label binding" can mislead a reader. From my POV,
>>>>>>>>>>>>>>>>>>> this term is related to LDP, but the document includes many options to
>>>>>>>>>>>>>>>>>>> signal transport paths.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Gyan> Will fix
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 9. Section 3. "*The ingress and egress 4PE router MUST
>>>>>>>>>>>>>>>>>>> bind a label to the IPv4 prefix as per [RFC8277] using BGP Labeled Unicast
>>>>>>>>>>>>>>>>>>> herinafter called BGP-LU, AFI/SAFI Address Family (AFI) / Subsequent
>>>>>>>>>>>>>>>>>>> Address Family Identifier (SAFI) 2-tuple "1/4"*".
>>>>>>>>>>>>>>>>>>> Which is the IPv4 prefix? If I understand it correctly, this requirement is
>>>>>>>>>>>>>>>>>>> too strict. As a designer I don't want to send all IPv4 prefixes with
>>>>>>>>>>>>>>>>>>> labels in every case, I want to have some flexibility. Moreover, as a part
>>>>>>>>>>>>>>>>>>> of the implementation team, I'm sure we won't consider this as the only
>>>>>>>>>>>>>>>>>>> possible option too.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Gyan>Lets discuss this in more detail the use cases
>>>>>>>>>>>>>>>>>> for not labeling.  Also the issue I mentioned in RFC 3032 section 2.2 that
>>>>>>>>>>>>>>>>>> describes the problem with not labeling the prefix if the protocol is
>>>>>>>>>>>>>>>>>> different between the topmost label and service prefix.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> I believe there should be a clear distinction between
>>>>>>>>>>>>>>>>> reachability advertisements and LSP signaling. BGP LU does both at the same
>>>>>>>>>>>>>>>>> time. Every IP prefix here comes with a label that allows us to signal a
>>>>>>>>>>>>>>>>> BGP LSP. It seems to me, you consider LU only as the copy-paste of IPv4
>>>>>>>>>>>>>>>>> unicast but with labels. I offer you to look at it as an LSP signaling
>>>>>>>>>>>>>>>>> protocol (one can think LU is an LDP on steroids).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let's consider an example. We have a PE-CE link with an
>>>>>>>>>>>>>>>>> IPv4 subnet toward an IPv4 CE. A PE router of that PE-CE link is a part of
>>>>>>>>>>>>>>>>> the IPv6-only core. We have some v6 LSP mesh among all PEs of that core.
>>>>>>>>>>>>>>>>> These v6 LSPs populate the FTN table and allow us to resolve any BGP
>>>>>>>>>>>>>>>>> next-hops through the network. The CE advertises IPv4 routes via 1/1. Let's
>>>>>>>>>>>>>>>>> call them Customer Prefixes or Reachability for convenience. I believe, at
>>>>>>>>>>>>>>>>> this point, the setup is pretty clear.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Your draft restricts me to export these Customer Prefixes
>>>>>>>>>>>>>>>>> into IPv4 labeled unicast internal sessions and also restricts me to attach
>>>>>>>>>>>>>>>>> an explicit null label (with the value of 0) to them. Anyway, that works,
>>>>>>>>>>>>>>>>> and I have never argued
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I don't see a good reason to do that. All that I want is
>>>>>>>>>>>>>>>>> to create an IPv4 loopback on the PE and advertise its IPv4 address via
>>>>>>>>>>>>>>>>> IPv4 labeled unicast toward remote PEs. In doing so I create a BGP LSP from
>>>>>>>>>>>>>>>>> every remote PE to this advertising PE. The label from this advertisement
>>>>>>>>>>>>>>>>> can be either an explicit null or an arbitrary one. This is my choice. Then
>>>>>>>>>>>>>>>>> I just readvertise all Customer Prefixes via internal IPv4 1/1 sessions
>>>>>>>>>>>>>>>>> with the next-hop address of my new shiny IPv4 loopback.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Both examples would have the same deep of a label stack.
>>>>>>>>>>>>>>>>> Both examples wouldn't have any problems at the penultimate LSR. Both
>>>>>>>>>>>>>>>>> examples (well, mechanics, not exact examples) should be a part of the doc.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But with my example, I have greater flexibility, I can
>>>>>>>>>>>>>>>>> change the next-hop addresses from the loopback address to an address of
>>>>>>>>>>>>>>>>> the PE-CE link, allocate an arbitrary label in a per-next-hop fashion and
>>>>>>>>>>>>>>>>> get an EPE. Also, it would increase convergence if I want to. The key here
>>>>>>>>>>>>>>>>> -- is my desire as a designer. With your solution, it is
>>>>>>>>>>>>>>>>> impossible because the next-hop for all Customer Prefixes and the label
>>>>>>>>>>>>>>>>> would always be the same. It is impossible from a single value (0)
>>>>>>>>>>>>>>>>> to produce many different actions at egress.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>      Gyan2> Thank you for the detailed use case.  Have you
>>>>>>>>>>>>>>>> tried to configure this example and with which vendor?  I am not sure I
>>>>>>>>>>>>>>>> would call it EPE as SR or BGP-LU are not steering traffic to the egress
>>>>>>>>>>>>>>>> but I get it your point EPE like behavior to point to specific CE next
>>>>>>>>>>>>>>>> hop.  What is the advantage of doing so.  With my solution you have a
>>>>>>>>>>>>>>>> single next hop which is the egress PE however all the prefixes received on
>>>>>>>>>>>>>>>> the egress PE in the IPv4 labeled unicast RIB would get advertised to each
>>>>>>>>>>>>>>>> CE across the respective next hops. In your example you create and LSP
>>>>>>>>>>>>>>>> between the PE loopbacks or use PE-CE interface next hop for the LSP per CE
>>>>>>>>>>>>>>>> and then advertise all the prefixes per CE unlabeled.  The  prefixes for
>>>>>>>>>>>>>>>> each CE broken down now by IPv4 LSP are going to all get advertised out to
>>>>>>>>>>>>>>>> all CEs anyway doing it your way or my way.  So net end result of what gets
>>>>>>>>>>>>>>>> advertised to which ingress and egress CE over the LSP is identical. I am
>>>>>>>>>>>>>>>> not seeing any benefit to doing so other than added complexity.  I am not
>>>>>>>>>>>>>>>> sure how much convergence gain you have as you can do per VRF label
>>>>>>>>>>>>>>>> allocation and better scale and convergence. Can you expand on the benefits
>>>>>>>>>>>>>>>> of doing so in your example. I do now understand the flexibility you desire.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 10. Section 3. "*The Ingress 4PE router MUST forward
>>>>>>>>>>>>>>>>>>> IPv4 NLRI as Labeled prefixes using BGP-LU SAFI over the IPv6-signaled LSP
>>>>>>>>>>>>>>>>>>> towards the Egress 4PE router identified by the IPv4 address advertised in
>>>>>>>>>>>>>>>>>>> the IPv6 next hop encoding per [RFC8950]*". NLRI is a
>>>>>>>>>>>>>>>>>>> term related to BGP message encoding. How can a 4PE router forward an NLRI
>>>>>>>>>>>>>>>>>>> over an LSP? Forwarding is a procedure of transmitting traffic, LSP is a
>>>>>>>>>>>>>>>>>>> transport entity used for forwarding labeled traffic, not NLRIs.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>   Gyan> I am referring to BGP SAFI in this case IPv4
>>>>>>>>>>>>>>>>>> labeled unicast 1/4 which should say “advertised” and not “forwarded”
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 11. Section 4. "*To ensure interoperability between
>>>>>>>>>>>>>>>>>>> routers that implement the 4PE design over MPLS [RFC3031] LDP IPv6 Core
>>>>>>>>>>>>>>>>>>> described in this document, ingress and egress 4PE MUST support building
>>>>>>>>>>>>>>>>>>> the underlay tunneling using IPv6-signaled MPLS LSPs established by LDP
>>>>>>>>>>>>>>>>>>> [RFC5036] or Resource Reservation Protocol (RSVP-TE) [RFC3209]*".
>>>>>>>>>>>>>>>>>>> Here we see that RSVP is anyhow possible, but I'm still missing the idea of
>>>>>>>>>>>>>>>>>>> "MPLS LDP IPv6 Core" and how RSVP is related to it. Is this sentence about
>>>>>>>>>>>>>>>>>>> LDP over RSVP tunneling? Or about ships in the night?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> yes two ships in the night two different
>>>>>>>>>>>>>>>>>> scenarios will fix
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 12. Section 4. "*The (outer) label imposed MUST
>>>>>>>>>>>>>>>>>>> correspond to the IPv6- signaled LSP starting on the ingress 4PE Router and
>>>>>>>>>>>>>>>>>>> ending on the egress 4PE Router*". What if I have
>>>>>>>>>>>>>>>>>>> segmented LSPs among several domains of the same AS? Pretty sure that in
>>>>>>>>>>>>>>>>>>> this case, the outermost label would correspond to an LSP either from an
>>>>>>>>>>>>>>>>>>> ingress 4PE to an ABR or from an ABR to the other ABR/egress 4PE router.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> Yes that’s correct and will add that scenario of
>>>>>>>>>>>>>>>>>> multiple domains to the wording for end to end scenario
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 13. Section 4. "*The reason for the use of a second
>>>>>>>>>>>>>>>>>>> level bottom of stack service label...*". I'm sure that
>>>>>>>>>>>>>>>>>>> the stack can be deeper. I made some hints earlier in my notes (e.g., see
>>>>>>>>>>>>>>>>>>> the previous bullet), also you can think about emerging MNA solutions. I
>>>>>>>>>>>>>>>>>>> think it is better to pay attention not to the number of labels in a stack
>>>>>>>>>>>>>>>>>>> but to the fact that the bottom label must pertain to an LSP forwarding
>>>>>>>>>>>>>>>>>>> IPv4 traffic and that this label must be present on the egress.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> Agreed.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 14. Section 4. "*...it allows for Penultimate Hop
>>>>>>>>>>>>>>>>>>> Popping (PHP) on the IPv6 Label Switch Router (LSR) Provider (P) node,...*".
>>>>>>>>>>>>>>>>>>> I believe it is better to use one terminology, either pertaining to an MPLS
>>>>>>>>>>>>>>>>>>> LSP (LER/LSR) or a BGP service (P/PE) in a single sentence. Here
>>>>>>>>>>>>>>>>>>> MPLS-related terms suit you better because you are describing an LSP and
>>>>>>>>>>>>>>>>>>> label stack.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    Gyan> Agreed will fix
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 15. Section 4. "*The label advertised by the egress 4PE
>>>>>>>>>>>>>>>>>>> Router with MP-BGP MUST be an explicit Null label Pipe mode Diff-Serv
>>>>>>>>>>>>>>>>>>> Tunneling Model use case as defined in [RFC3270], so that the topmost label
>>>>>>>>>>>>>>>>>>> can be preserved Ultimate Hop POP (UHP) to the egress PE Edge LSR*".
>>>>>>>>>>>>>>>>>>> Why can it be an arbitrary label instead an explicit null?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Gyan> The reason for IPv6 explicit null label for
>>>>>>>>>>>>>>>>>> topmost label is to prevent the drop issue in case the service prefix is
>>>>>>>>>>>>>>>>>> not labeled as I have mentioned in the draft and the IPv4 traffic is
>>>>>>>>>>>>>>>>>> dropped.  Using IPv6 explicit null label RFC 3270  pipe mode also fixes the
>>>>>>>>>>>>>>>>>> MPLS QOS issue of EXP PHB scheduling on PHP node when the topmost label is
>>>>>>>>>>>>>>>>>> popped as mentioned earlier.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> My question was about a different thing. An arbitrary
>>>>>>>>>>>>>>>>> label also prevents a penultimate LSR from exposing the payload. An LSR
>>>>>>>>>>>>>>>>> would do a simple swap operation not knowing itself as penultimate. The
>>>>>>>>>>>>>>>>> second moment, I don't understand why are you restricting the transport
>>>>>>>>>>>>>>>>> from PHP if you have already restricted the service from that. One
>>>>>>>>>>>>>>>>> shouldn't try to save everybody from every possible pitfall with a single
>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     Gyan2> I am restricting the transport from PHP with
>>>>>>>>>>>>>>>> topmost IPv6 LSP explicit null signaling.  The service does not need to be
>>>>>>>>>>>>>>>> restricted as it’s only popped at the egress PE.  The PHP node would either
>>>>>>>>>>>>>>>> do default PHP and pop the topmost label or if explicit null signaling is
>>>>>>>>>>>>>>>> enabled for the LSP the topmost is preserved to egress PE which pops the
>>>>>>>>>>>>>>>> entire stack.  All intermediate LSRs would do the normal swap operation
>>>>>>>>>>>>>>>> except the PHP node which would use the implicit null value 3 label default
>>>>>>>>>>>>>>>> PHP POP operation.  My concern is only with exposing unlabeled service
>>>>>>>>>>>>>>>> which would get dropped with the default PHP operation and not using
>>>>>>>>>>>>>>>> explicit null signaling on the transport LSP.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 16. Section 4. "*The explicit null label advertised by
>>>>>>>>>>>>>>>>>>> the egress PE router with MP-BGP also identifies the IPv4 routing context
>>>>>>>>>>>>>>>>>>> or outgoing interface to forward the packet to and ingress 4PE Router which
>>>>>>>>>>>>>>>>>>> MUST be able to be accept the "IPv6 Explicit NULL Label" advertised*".
>>>>>>>>>>>>>>>>>>> From the very beginning of this statement, I understand that the egress PE
>>>>>>>>>>>>>>>>>>> advertises an explicit label (0) for IPv4-labeled unicast routes to
>>>>>>>>>>>>>>>>>>> identify either IPv4 routing context or outgoing interface. The end of this
>>>>>>>>>>>>>>>>>>> sentence tells us that the ingress 4PE router must accept the IPv6 explicit
>>>>>>>>>>>>>>>>>>> label (2). This label (2) cannot be used to identify either IPv4 routing
>>>>>>>>>>>>>>>>>>> contexts or any interfaces. So I made an assumption that there is a mistake
>>>>>>>>>>>>>>>>>>> and this is actually a 0 label, not 2.
>>>>>>>>>>>>>>>>>>> How can the label with the same value (0) identify
>>>>>>>>>>>>>>>>>>> several things (context, interface)? If you offer to bind this label to an
>>>>>>>>>>>>>>>>>>> exact interface (without a route lookup at egress) instead of doing a
>>>>>>>>>>>>>>>>>>> lookup in the global routing context/table, it will break the idea behind
>>>>>>>>>>>>>>>>>>> this special purpose label and won't support more than one interface for
>>>>>>>>>>>>>>>>>>> your solution. Arbitrary labels solve this issue and also many others.
>>>>>>>>>>>>>>>>>>> For example, it allows us to do EPE if and only if we
>>>>>>>>>>>>>>>>>>> dissipate IPv4 reachability (separately from its next-hop addresses) as
>>>>>>>>>>>>>>>>>>> IPv4 routes with IPv4 next-hops alongside arbitrary labels for these
>>>>>>>>>>>>>>>>>>> next-hops distributed as labeled IPv4 routes with IPv6 next-hops (which
>>>>>>>>>>>>>>>>>>> this draft does not support and breaks). In this case, these arbitrary
>>>>>>>>>>>>>>>>>>> labels are allocated in per next-hop fashion and preclude any IP lookups at
>>>>>>>>>>>>>>>>>>> egress. I've have mentioned it several months ago.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> It seems, that you have skipped the crucial part of my
>>>>>>>>>>>>>>>>> message :(
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     Gyan2> I will answer in the next email, thanks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>        Gyan3> I agree this sentence is a bit confusing with
>>>>>>>>>>>>>>> regards to label 0 and 2 use.  We are not using IPv4 explicit null value 0
>>>>>>>>>>>>>>> at all in this draft.  So the conveying of the next hop and what the
>>>>>>>>>>>>>>> sentence it talking about related to the IPv6 LSP egress PE signaling done
>>>>>>>>>>>>>>> via the IPv6 explicit null label value 2 defined in RFC 4182 to use for
>>>>>>>>>>>>>>> topmost label so the next hop of the  advertised IPv4 routes have an IPv6
>>>>>>>>>>>>>>> next hop as defined according to RFC 8950 next hop encoding which is an
>>>>>>>>>>>>>>> IPv6 labeled address per BGP LU RFC 8277 for the BOS IPv4 prefixes. So here
>>>>>>>>>>>>>>>  we are talking about using label value 2 IPv6 explicit null to identify
>>>>>>>>>>>>>>> the IPv4 routing context of the outgoing interface since the next hop is an
>>>>>>>>>>>>>>> IPv6 next hop for the labeled IPv4 prefixes.  I will cleanup the wording.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 17. Section 4. "*4PE design MUST use "IPv6 Explicit
>>>>>>>>>>>>>>>>>>> Null label" value 2 defined in [RFC4182] Pipe Diff-Serv Tunneling Model as
>>>>>>>>>>>>>>>>>>> defined in [RFC3270]*". From this statement, I conclude
>>>>>>>>>>>>>>>>>>> that the authors talk about an underlying transport (IPv6 LSP via BGP LU).
>>>>>>>>>>>>>>>>>>> Why does the draft about a service mandate a label for transport? I don't
>>>>>>>>>>>>>>>>>>> see anything wrong with arbitrary labels for underlying IPv6 LSPs either.
>>>>>>>>>>>>>>>>>>> If you want to highlight that there should *not *be PHP
>>>>>>>>>>>>>>>>>>> it does not mean there is only option is an explicit label. Some vendors
>>>>>>>>>>>>>>>>>>> use arbitrary labels by default with an option to use a label of 2 instead.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Gyan> I agree most vendors use arbitrary label by
>>>>>>>>>>>>>>>>>> default and an option to use explicit null.  As it’s an easy configuration
>>>>>>>>>>>>>>>>>> change to use explicit null with its benefits why not mandate it with the
>>>>>>>>>>>>>>>>>> benefits.  As well as solve issues with mix of arbitrary and explicit null
>>>>>>>>>>>>>>>>>> usage.  I would like to highlight that there would not be PHP but AFAIK
>>>>>>>>>>>>>>>>>> that can only be done if using explicit null. As well as explicit null pipe
>>>>>>>>>>>>>>>>>> mode solves the MPLS QOS PHB EXP scheduling issue.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> No, this can be done not only using the explicit null.
>>>>>>>>>>>>>>>>> For example, Nokia SR OS allocates arbitrary labels by default for RSVP and
>>>>>>>>>>>>>>>>> LDP LSPs. Penultimate routers in that case just swap one label to another
>>>>>>>>>>>>>>>>> (our arbitrary one). Egress LER then handles the stack with the transport
>>>>>>>>>>>>>>>>> label at the top that it allocated previously. But the difference between
>>>>>>>>>>>>>>>>> an explicit label and an arbitrary one is this eLER can do many things with
>>>>>>>>>>>>>>>>> the latter: allocate if for a table, for a prefix, for a next-hop address.
>>>>>>>>>>>>>>>>> You can not do it with the single-valued label (0 or 2). The labels of 0 or
>>>>>>>>>>>>>>>>> 2 are always tightened with their global contexts. Changing that is not
>>>>>>>>>>>>>>>>> possible under this draft because it influences the whole MPLS paradigm.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     Gyan2>   In the draft all I am concerned about is the
>>>>>>>>>>>>>>>> transport topmost label IPv6 explicit null signaling so the PHP node
>>>>>>>>>>>>>>>> preserves the transport label to prevent the IPv4 packets from being
>>>>>>>>>>>>>>>> dropped at PHP node.  It sounds like Nokia by default for RSVP and LDP LSP
>>>>>>>>>>>>>>>> does not do PHP by default on the PHP node PHP label 3 and uses arbitrary
>>>>>>>>>>>>>>>> label as you mentioned.  Cisco, Juniper and most all vendors on the market
>>>>>>>>>>>>>>>> do PHP by default.  Egress LER pops the entire stack of remaining labels
>>>>>>>>>>>>>>>> regardless of topmost label explicit null signaling or PHP operation.
>>>>>>>>>>>>>>>> Since the explicit versus arbitrary relates to the topmost transport label
>>>>>>>>>>>>>>>> the behavior of what happens on the LER egress PE is identical popping the
>>>>>>>>>>>>>>>> entire stack after which perform per VRF table lookup, per CE next hop or
>>>>>>>>>>>>>>>> per prefix label allocation. I think what you are missing is RFC 4182
>>>>>>>>>>>>>>>> updates RFC 3032 label stack so that explicit null 0 and 2 can be used for
>>>>>>>>>>>>>>>> the transport label where previously was only valid for BOS label.  That is
>>>>>>>>>>>>>>>> what I have been referring to using explicit null for topmost label and not
>>>>>>>>>>>>>>>> the BOS label.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 18. Section 4. "*BGP/MPLS VPN [RFC4364] defines 3 label
>>>>>>>>>>>>>>>>>>> allocation modes for Layer 3 VPN's...*" and later "*The
>>>>>>>>>>>>>>>>>>> 4PE design provides the same operator flxeiblity as BGP/MPLS VPN [RFC4798],
>>>>>>>>>>>>>>>>>>> 2 level label stack option using Per-CE label allocation mode where the
>>>>>>>>>>>>>>>>>>> next hop is label so all prefixes associated with CE get the same label.
>>>>>>>>>>>>>>>>>>> The 4PE design provides the same operator flxeiblity as BGP/MPLS VPN
>>>>>>>>>>>>>>>>>>> [RFC4798], 2 level label stack option using Per-VRF label allocation mode
>>>>>>>>>>>>>>>>>>> where all prefixes within a VRF get the same is label*".
>>>>>>>>>>>>>>>>>>> I believe it is not possible with an explicit label for an IPv4 LSP either
>>>>>>>>>>>>>>>>>>> because a single value cannot be attached to many different things at the
>>>>>>>>>>>>>>>>>>> same time.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Gyan> The IPv4 labeled prefixes AFAIK would have a
>>>>>>>>>>>>>>>>>> single label per CE or per VRF. Can you elaborate on why not possible?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IM> Yes. We have a single label per VRF but we can't have
>>>>>>>>>>>>>>>>> a single label per CE because there are possibly lots of CEs in a VRF. The
>>>>>>>>>>>>>>>>> egress router doing a label lookup must understand what to do with every
>>>>>>>>>>>>>>>>> label it allocated. With a per VRF label it does an IP lookup in a
>>>>>>>>>>>>>>>>> corresponding VRF. With a per CE label it strips the label and sends
>>>>>>>>>>>>>>>>> payload via the exact interface without any IP lookups. As you can see, you
>>>>>>>>>>>>>>>>> can't bind a single value to many different interfaces (tasks).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    Gyan2> I think it’s actually the opposite that when
>>>>>>>>>>>>>>>> using 6VPE or 4VPE you have VRF context but with 6PE or 4PE their is not
>>>>>>>>>>>>>>>> many VRF context instances but we do have a single default VRF context so
>>>>>>>>>>>>>>>> single label for the default VRF.  With per CE label allocation with 6VPE /
>>>>>>>>>>>>>>>> 4VPE versus 6PE / 4PE I don’t see how it’s any different with per CE the
>>>>>>>>>>>>>>>> label is stripped by the PE and the native packet payload is forwarded to
>>>>>>>>>>>>>>>> the exact CE interface in this case it’s the default VRF many interfaces
>>>>>>>>>>>>>>>> which with VPE you can have many interfaces as well - no different.  If per
>>>>>>>>>>>>>>>> CE works for 6VPE / 4VPE it should work for 6PE / 4PE.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [e-mail] <[e-mail]>*



*M 301 502-1347*