[bess] [Shepherding AD review] Pre-IETF Last-Call review of draft-ietf-bess-evpn-vpws-fxc-08

["Gunter van de Velde (Nokia)" <[e-mail]>]

Hi Authors,

It unfortunately took some time to start processing this nice fxc write-up.
I started reviewing this document before starting a IETF LC process. 
During this review i spent some time re-editing some paragraphs from 
readability perspective and i hope it will help you to improve the draft.

The review details some questions and some observations that occurred 
when going through the document. 

Please find my review notes below. 

# Gunter Van de Velde, RTG AD, comments for draft-ietf-bess-evpn-vpws-fxc-08

#GENERIC COMMENTS
#================
## Review by AD took more time as anticipated. Apologies for that. 

## During my review i proposed (quiet a number of) readability re-edits. 
I hope this helps to make the document easier to read and implement. It does 
make the review text appear longer as intended. I do believe that for most 
it will be a review of the re-edit proposal and then copy/paste of text blobs

## I was informed that technology is implemented by some vendors already. 
If possible a swift turnaround is appreciated so we can process the 
document further and avoid code-point squatting

## idnits spits up some minor items (outdated refs and old submit date)
https://author-tools.ietf.org/api/idnits?url=https://www.ietf.org/archive/id/draft-ietf-bess-evpn-vpws-fxc-08.txt

## The line numbers shown with this AD review are based upon the idnits tool.

## there was un-responded clarification request in BESS WG
https://mailarchive.ietf.org/arch/msg/bess/YocP1AqChFa4LIWe3GT3FufyFEw/

## Because this draft was stuck for substancial amount of time, it may be worthwhile 
to add that this document is concerning the IP/MPLS based dataplane (and not SRv6)?


#DETAILED COMMENTS
#=================
##classified as [minor] and [major]

105	   node or link failure [I-D.ietf-rtgwg-bgp-pic].  Furthermore, the use
106	   of EVPN BGP constructs eliminates the need for multi-segment PW
107	   auto-discovery and signaling if the VPWS service need to span across
108	   multiple ASes.

[minor]
reference or description context about to describe 'multi-segment PW auto-discovery'?

110	   Some service providers have very large number of ACs (in millions)
111	   that need to be back hauled across their MPLS/IP network.  These ACs

[minor]
Due to the long time this draft has been pending upon AD 
handling, SRv6 progressed substancially. Is this statement 
about MPLS/IP still correct? does this draft assume MPLS transport or 
does it include SRv6 too? It may be helpful to explicit mention this

128	1.1.  Terminology

[major]
Further in the text these definitions are used. 
For example "A-D:  Auto Discovery", but nowhere is there a explanation 
what this exactkly means? Maybe add for such terminology the RFCs 
where the terms are further explained or add a short description what it 
exactly is.

202	   In [RFC8214], a PE signals an AC indirectly by first associating that
203	   AC to a VPWS service tunnel (e.g., a VPWS service instance) and then

[minor]
add description/terminology about VPWS service instance

204	   signaling the VPWS service tunnel via a Ethernet A-D per EVI route

[minor]
Add terminology and reference [RFC7432]

209	   Therefore, a PE device that receives such EVPN routes, can associate
210	   the VPWS service tunnel to the remote Ethernet Segment, and when th

[minor] 
Can the logic be explained more explicit how such association happens?

212	   associated with the failed ES per [RFC7432], it can update its BGP
213	   path list for that VPWS service tunnel quickly and achieve fast
214	   convergence for multi-homing scenarios.  Even if fast convergence

[minor]
How is this 'path list' updated? and how does that translate in the encodings

219	   single- active multi-homing) or to other PEs in the redundancy group
220	   (in case of all-active multi-homing).  In absence of updating the BGP

[minor]
Is there a description or reference to what is redundancy group in this context?

224	   When a single VPWS service tunnel multiplexes many ACs across number
225	   of Ethernet Segments (number of physical interfaces) and the ACs are
226	   not signaled via EVPN BGP to remote PE devices, then the remote PE
227	   devices neither know the association of the received Ethernet Segment
228	   to these ACs (and in turn to their local ACs) nor they know the
229	   association of the VPWS service tunnel (e.g., EVPN service label) to
230	   the far-end ACs - i.e, the remote PEs only know the association of
231	   their local ACs to the VPWS service tunnel but not the far-end ACs.
232	   Thus upon a connectivity failure to the ES, they don't know how to
233	   redirect traffic via another multi-homing PE to that ES.  In other
234	   words, even if an ES failure is signaled via EVPN to the remote PE
235	   devices, they don't know what to do with such message because they
236	   don't know the association among the remote ES, the remote ACs, and
237	   the VPWS service tunnel.

[minor]
What about following rewrite from readability perspective:

"When a single VPWS service tunnel carries multiple ACs across various 
Ethernet Segments (physical interfaces) without signaling the ACs via 
EVPN BGP to remote PE devices, those remote PE devices lack the 
information to associate the received Ethernet Segment with these 
ACs or with their local ACs. They also lack the association between 
the VPWS service tunnel (e.g., EVPN service label) and the far-end 
ACs. This means that while the remote PEs can associate their local 
ACs with the VPWS service tunnel, they cannot make similar associations 
for the far-end ACs.

Consequently, in case of a connectivity failure to the ES, the 
remote PEs are unable to redirect traffic via another multi-homing 
PE to that ES. In other words, even if an ES failure is signaled via 
EVPN to the remote PE devices, they cannot effectively respond because 
they do not know the relationship between the remote ES, the 
remote ACs, and the VPWS service tunnel."

239	   In order to address this issue when multiplexing large number of ACs
240	   onto a single VPWS service tunnel, two mechanisms are devised: one to
241	   support VPWS services between two single-homed endpoints and another
242	   one to support VPWS services where one of the endpoints is multi-
243	   homed.

[minor]
What about following rewriting proposal
"To address this issue when multiplexing a large number of ACs 
onto a single VPWS service tunnel, two mechanisms have been 
developed: one to support VPWS services between two single-homed 
endpoints, and another to support VPWS services where one of 
the endpoints is multi-homed."

245	   For single-homed endpoints, it is OK not to signal each AC in BGP
246	   because upon connection failure to the ES, there is no alternative
247	   path to that endpoint.  However, the ramification for not signaling
248	   an AC failure is that the traffic destined to the failed AC, is sent
249	   over MPLS/IP core and then gets discarded at the destination PE -
250	   i.e., it can waste network resources.
251	   This waste of network resources upon connection failure may be
252	   transient as it is detectable and preventable at the application
253	   layer in some cases.  Section 3.2 describes a solution for such
254	   single-homing VPWS service.

[minor]
What about this rewrite?
"For single-homed endpoints, it is acceptable not to signal each AC 
in BGP because, in the event of a connection failure to the ES, there 
is no alternative path to that endpoint. However, the implication 
of not signaling an AC failure is that the traffic destined for 
the failed AC is sent over the MPLS/IP core and then discarded at 
the destination PE, thereby potentially wasting network resources.

This wastage of network resources during a connection failure may 
be transient, as it can be detected and prevented at the application 
layer in certain cases. Section 3.2 outlines a solution for such 
single-homing VPWS service."

256	   For VPWS services where one of the endpoints is multi-homed, there
257	   are two options:

259	   1) to signal each AC via BGP so that the path list can be updated
260	   upon a failure that impacts those ACs.  This solution is described in
261	   Section 3.3 and it is called VLAN-signaled flexible cross-connect
262	   service.

264	   2) to bundle several ACs on an ES together per destination end-point
265	   (e.g., ES, MAC-VRF, etc.) and associate such bundle to a single VPWS
266	   service tunnel.  This is similar to VLAN-bundle service interface
267	   described in [RFC8214].  This solution is described in Section 3.2.1.

[minor] 
What following rewrite:

"For VPWS services where one of the endpoints is multi-homed, there 
are two options:

1) To signal each AC via BGP, allowing the path list to be updated upon a 
failure affecting those ACs. This solution is described in Section 3.3 and 
is referred to as the VLAN-signaled flexible cross-connect service.

2) To bundle several ACs on an ES together per destination endpoint 
(e.g., ES, MAC-VRF, etc.) and associate such a bundle with a single 
VPWS service tunnel. This approach is similar to the VLAN-bundle 
service interface described in [RFC8214]. This solution is described 
in Section 3.2.1."

271	   This section describes a solution for providing a new VPWS service
272	   between two PE devices where a large number of ACs (e.g., VLANs) that
273	   span across many Ethernet Segments (i.e., physical interfaces) on
274	   each PE are multiplex onto a single P2P EVPN service tunnel.  Since
275	   multiplexing is done across several physical interfaces, there can be
276	   overlapping VLAN IDs across these interfaces; therefore, in such
277	   scenarios, the VLAN IDs (VIDs) MUST be translated into unique VIDs to
278	   avoid collision.  Furthermore, if the number of VLANs that are
279	   getting multiplex onto a single VPWS service tunnel exceed 4095, then
280	   a single tag to double tag translation MUST be performed.  This
281	   translation of VIDs into unique VIDs (either single or double) is
282	   referred to as "VID normalization".

[minor]
potential readability rewrite:
"This section outlines a solution for providing a new VPWS service 
between two PE devices where a large number of ACs (such as VLANs) that 
span across multiple Ethernet Segments (physical interfaces) on each 
PE are multiplexed onto a single P2P EVPN service tunnel. Since the 
multiplexing involves several physical interfaces, there can be 
overlapping VLAN IDs across these interfaces. In such cases, the 
VLAN IDs (VIDs) must be translated into unique VIDs to prevent collisions. 
Furthermore, if the number of VLANs being multiplexed onto a single 
VPWS service tunnel exceeds 4095, then a single tag to double tag 
translation must be performed. This translation of VIDs into unique 
VIDs (either single or double) is referred to as "VID normalization.""

284	   When single normalized VID is used, the lower 12-bit of Ethernet tag
285	   field in EVPN routes MUST be set to that VID and when double
286	   normalized VID is used, the lower 12-bit of Ethernet tag field MUST
287	   be set to inner VID and the higher 12-bit is set to the outer VID.
288	   As in [RFC8214], 12-bit and 24-bit VPWS service instance identifiers
289	   representing normalised VIDs MUST be right-aligned.

[minor]
readability rewrite proposal:
"When a single normalized VID is used, the lower 12 bits of the Ethernet 
tag field in EVPN routes MUST be set to that VID. When a double normalized 
VID is used, the lower 12 bits of the Ethernet tag field MUST be set to 
the inner VID, while the higher 12 bits are set to the outer VID. As 
stated in [RFC8214], 12-bit and 24-bit VPWS service instance identifiers 
representing normalized VIDs MUST be right-aligned."

291	   Since there is only a single EVPN VPWS service tunnel associated with
292	   many normalized VIDs (either single or double) across multiple
293	   physical interfaces, MPLS lookup at the disposition PE is no longer
294	   sufficient to forward the packet to the right egress
295	   endpoint/interface.  Therefore, in addition to an EVPN label lookup
296	   corresponding to the VPWS service tunnel, a VID lookup (either single
297	   or double) is also required.  On the disposition PE, one can think of
298	   the lookup of EVPN label results in identification of a VID-VRF, and
299	   the lookup of normalized VID(s) in that table, results in
300	   identification of egress endpoint/interface.  The tag manipulation
301	   (translation from normalized VID(s) to local VID) SHOULD be performed
302	   either as part of the VID table lookup or at the egress interface
303	   itself

[minor]
proposed readabilty rewrite:
"Since there is only a single EVPN VPWS service tunnel associated with 
many normalized VIDs (either single or double) across multiple 
physical interfaces, an MPLS lookup at the disposition PE is no 
longer sufficient to forward the packet to the correct egress 
endpoint or interface. Therefore, in addition to an EVPN label 
lookup corresponding to the VPWS service tunnel, a VID lookup 
(either single or double) is also required. At the disposition 
PE, the EVPN label lookup identifies a VID-VRF, and the lookup 
of the normalized VID(s) within that table identifies the appropriate 
egress endpoint or interface. The tag manipulation (translation from 
normalized VID(s) to the local VID) SHOULD be performed either as 
part of the VID table lookup or at the egress interface itself."

Is the normative SHOULD here really required? there are too 
many options available here. How does implementator know what exactly to do?

305	   Since VID lookup (single or double) needs to be performed at the
306	   disposition PE, then VID normalization MUST be performed prior to the
307	   MPLS encapsulation on the ingress PE.  This requires that both
308	   imposition and disposition PE devices be capable of VLAN tag
309	   manipulation, such as re-write (single or double), addition, deletion
310	   (single or double) at their endpoints (e.g., their ES's, MAC-VRFs,
311	   IP-VRFs, etc.).  Operators should be informed of possible trade-offs
312	   from performance standpoint, compared to usual PW processing.

[minor]
on line 305 it is stated that VID lookup needs to be performed. 
Is this a normative MUST/SHOULD? I assume that potentially the phrase 
is misleading constructed due to artifact from an earlier construct?
What about following minor readability re-edit:
"Since the VID lookup (single or double) needs to be performed at the 
disposition PE, VID normalization MUST be completed prior to MPLS 
encapsulation on the ingress PE. This requires that both the imposition 
and disposition PE devices be capable of VLAN tag manipulation, such 
as rewriting (single or double), addition, or deletion (single or 
double) at their endpoints (e.g., their ESs, MAC-VRFs, IP-VRFs, etc.). 
Operators should be informed of potential trade-offs from a performance 
standpoint, compared to typical PW processing."

316	   In [RFC8214], a unique value in the context of each PE's EVI is
317	   signaled.  The 32-bit Ethernet Tag ID field MUST be set to this VPWS
318	   service instance identifier value.

[minor]
A unique value for what exactly? I assume that this is for the 
VPWS Service Identifier as indicated by the section title. 
Should consider to add that more explicit in this phrase.

[major]
In RFC8214 i find the following:

   Either (1) the VPWS service instance identifier encoded in the
   Ethernet Tag ID in an advertised per-EVI Ethernet A-D route MUST be
   unique across all ASes or (2) an ASBR needs to perform a translation
   when the per-EVI Ethernet A-D route is re-advertised by the ASBR from
   one AS to the other AS.

So there seems to be option (1) and (2) and option (1) is the only that 
that MUST requires uniqueness. Maybe add indication where the uniqueness 
applies with reference to the correct section of RFC8214. I assume htis is 
a value in the per-EVI Ethernet A-D route?

320	   For FXC, Ethernet Tag ID field value may represent:

[minor]
Where is this Ethernet Tag ID dound for fxc? Is that in the 
advertised per-EVI Ethernet A-D route? for single or multihomed CE?
Can this be clarified?

324	   *  VLAN-Aware Bundle : a unique value for individual VLANs, and may
325	      be considered same as the normalised VID

[minor]
Why use may? is this a normative statement or informational statement? 
If informational, mthen using some other word as may will cause less confusion

327	   Both the VPWS service instance identifier and normalised VID are
328	   carried in the Ethernet Tag ID field of the Ethernet A-D per EVI
329	   route.  For FXC, in the case of a 12-bit ID the VPWS service instance

[major]
When browsing RFC8214 i see the followingtext:

"the Ethernet Tag ID is set to the VPWS 
service instance identifier that identifies the EVPL or EPL service."

Does this not conflict with the text from lines 327-329 where it stated 
that the Ethernet tag contains service instance identifier and normalised VID?
Is this encoding into the Ethernet TAG ID the novelty of FXC? if yes, maybe 
call that out explicitly.

329	   route.  For FXC, in the case of a 12-bit ID the VPWS service instance
330	   identifier is the same as the single-tag normalised VID and will be
331	   the same on both PEs.  Similarly in the case of a 24-bit ID, the VPWS

[minor]
In this paragraph there is mentoining of PEs. What is meant with these? 
Are these routers speaking EVPN that originate some EVPN route? Could a small 
diagram be added to add some context to the procedures discussed?

337	   In this mode of operation, many ACs across several Ethernet Segments
338	   are multiplex into a single EVPN VPWS service tunnel represented by a
339	   single VPWS service ID.  This is the default mode of operation for
340	   FXC and the participating PEs do not need to signal the VLANs
341	   (normalized VIDs) in EVPN BGP.

[minor]
proposed re-edit frm readability perspective:
"In this mode of operation, numerous ACs from multiple 
Ethernet Segments are multiplexed into a single EVPN VPWS 
service tunnel, which is identified by a single VPWS 
service ID. This is the standard operational mode for 
FXC, and the participating PEs are not required to 
signal the VLANs (normalized VIDs) in EVPN BGP."

343	   With respect to the data-plane aspects of the solution, both
344	   imposition and disposition PEs MUST be aware of the VLANs as the
345	   imposition PE performs VID normalization and the disposition PE does
346	   VID lookup and translation.  In this solution, there SHOULD only be a
347	   single P2P EVPN VPWS service tunnel between a pair of PEs for a set
348	   of ACs.

350	   As discussed previously, since the EVPN VPWS service tunnel is used
351	   to multiplex ACs across different ES's (e.g., physical interfaces),
352	   the EVPN label alone is not sufficient for proper forwarding of the
353	   received packets (over MPLS/IP network) to egress interfaces.
354	   Therefore, normalized VID lookup is REQUIRED in the disposition
355	   direction to forward packets to their proper egress end-points -
356	   i.e., the EVPN label lookup identifies a VID-VRF and subsequently,
357	   the normalized VID lookup in that table, identifies the egress
358	   interface.

360	   In this solution, on each PE, the single-homing ACs represented by
361	   their normalized VIDs are associated with a single EVPN VPWS service
362	   tunnel (in a given EVI).  The EVPN route that gets generated is an
363	   Ethernet A-D per EVI route with ESI=0, Ethernet Tag field set to VPWS
364	   service instance ID, MPLS label field set to dynamically generated
365	   EVPN service label representing the EVPN VPWS service tunnel.  This
366	   route is sent with a Route Target (RT) representing the EVI.  This RT
367	   can be auto-generated from the EVI per Section 5.1.2.1 of [RFC8365].
368	   Furthermore, this route is sent with the EVPN Layer-2 Extended
369	   Community defined in Section 3.1 of [RFC8214] with two new flags
370	   (defined in Section 4) that indicate: 1) this VPWS service tunnel is
371	   for default Flexible Cross-Connect, and 2) normalized VID type
372	   (single versus double).  The receiving PE uses these new flags for
373	   consistency check and MAY generate an alarm if it detects
374	   inconsistency but doesn't bring down the VPWS service.

[minor]
Proposed readability re-edit:
"
Regarding the data-plane elements of this solution, both imposition 
and disposition Provider Edge (PE) devices must be aware of the 
VLANs, as the imposition PE performs VLAN ID (VID) normalization 
and the disposition PE carries out VID lookup and translation. 
There SHOULD ideally be a single point-to-point (P2P) EVPN VPWS 
service tunnel between a pair of PEs for a specific set of 
Attachment Circuits (ACs).

As previously mentioned, because the EVPN VPWS service tunnel 
is employed to multiplex ACs across various Ethernet 
Segments (ESs) or physical interfaces, the EVPN label alone does 
not suffice for accurate forwarding of the received packets 
over the MPLS/IP network to egress interfaces. Therefore, a 
normalized VID lookup is REQUIRED in the disposition direction 
to route packets to their correct egress endpoints; the EVPN 
label lookup identifies a VID-VRF, and a subsequent normalized 
VID lookup in that table pinpoints the egress interface.

In this solution, for each PE, the single-homing ACs 
represented by their normalized VIDs are linked to a 
single EVPN VPWS service tunnel within a specific Ethernet 
Virtual Instance (EVI). The generated EVPN route is an 
Ethernet Auto-Discovery (A-D) per EVI route with 
an Ethernet Segment Identifier (ESI) of 0, an Ethernet 
Tag field set to the VPWS service instance ID, and an 
MPLS label field set to a dynamically generated EVPN 
service label representing the EVPN VPWS service tunnel. 
This route is sent with a Route Target (RT) that 
represents the EVI, which can be auto-generated from 
the EVI according to Section 5.1.2.1 of [RFC8365]. 
Additionally, this route includes the EVPN Layer-2 
Extended Community defined in Section 3.1 of [RFC8214], 
with two new flags (outlined in Section 4) that 
indicate: 1) this VPWS service tunnel is for the 
default Flexible Cross-Connect, and 2) the normalized 
VID type (single versus double). The receiving PE 
utilizes these new flags for a consistency check and 
MAY generate an alarm if it detects inconsistencies, 
but it will not disrupt the VPWS service.
"

376	   It should be noted that in this mode of operation, a single
377	   Ethernet A-D per EVI route is sent upon configuration of the first AC
378	   (ie, normalized VID).  Later, when additional ACs are configured and
379	   associated with this EVPN VPWS service tunnel, the PE does not
380	   advertise any additional EVPN BGP routes.  The PE only associates
381	   locally these ACs with the already created VPWS service tunnel.

[minor]
readability proosal re-edit:
"It should be observed that in this mode of operation, a single 
Ethernet Auto-Discovery (A-D) per Ethernet Virtual Instance (EVI) route is 
transmitted upon the configuration of the first Attachment Circuit (AC), 
specifically the normalized VLAN ID (VID). Subsequently, as additional 
ACs are configured and linked to this EVPN VPWS service tunnel, the 
Provider Edge (PE) does not issue any further EVPN BGP routes. 
Instead, the PE merely associates these ACs locally with the 
pre-established VPWS service tunnel.
"

note: that i was not certain you meant ie (=specifically) or eg (=for example)

385	   The default FXC mode can also be used for multi-homing.  In this
386	   mode, a group of normalized VIDs (ACs) on a single Ethernet segment
387	   that are destined to a single endpoint are multiplexed into a single
388	   EVPN VPWS service tunnel represented by a single VPWS service ID.
389	   When the default FXC mode is used for multi-homing, instead of a
390	   single EVPN VPWS service tunnel, there can be many service tunnels
391	   per pair of PEs - i.e, there is one tunnel per group of VIDs per pair
392	   of PEs and there can be many groups between a pair of PEs, thus
393	   resulting in many EVPN service tunnels.

[minor]
readability re-edit proposal:
"The default Flexible Cross-Connect (FXC) mode can also be 
utilized for multi-homing. In this mode, a group of normalized 
VLAN IDs (VIDs) representing Attachment Circuits (ACs) on a 
single Ethernet segment, all destined for a single endpoint, 
are multiplexed into a single EVPN VPWS service tunnel, which 
is identified by a unique VPWS service ID. When employing 
the default FXC mode for multi-homing, rather than using 
a single EVPN VPWS service tunnel, there may be multiple 
service tunnels per pair of Provider Edges (PEs). 
Specifically, there is one tunnel for each group of VIDs 
per pair of PEs, and there can be many such groups between 
a pair of PEs, resulting in numerous EVPN service tunnels.
"

395	3.3.  VLAN-Signaled Flexible Xconnect
396
397	   In this mode of operation, just as the default FXC mode in
398	   Section 3.2, many normalized VIDs (ACs) across several different
399	   ES's/interfaces are multiplexed into a single EVPN VPWS service
400	   tunnel; however, this single tunnel is represented by many VPWS
401	   service IDs (one per normalized VID) and these normalized VIDs are
402	   signaled using EVPN BGP.

404	   In this solution, on each PE, the multi-homing ACs represented by
405	   their normalized VIDs are configured with a single EVI.  There is no
406	   need to configure VPWS service instance ID in here as it is the same
407	   as the normalized VID.  For each normalized VID on each ES, the PE
408	   generates an Ethernet A-D per EVI route where ESI field represents
409	   the ES ID, the Ethernet Tag field is set to the normalized VID, MPLS
410	   label field is set to dynamically generated EVPN label representing
411	   the P2P EVPN service tunnel and it is the same label for all the ACs
412	   that are multiplexed into a single EVPN VPWS service tunnel.  This
413	   route is sent with a Route Target (RT) representing the EVI.  As
414	   before, this RT can be auto-generated from the EVI per section
415	   Section 5.1.2.1 of [RFC8365].  Furthermore, this route is sent with
416	   the EVPN Layer-2 Extended Community defined in Section 3.1 of
417	   [RFC8214] with two new flags (defined in Section 4) that indicate: 1)
418	   this VPWS service tunnel is for VLAN-signaled Flexible Cross-Connect,
419	   and 2) normalized VID type (single versus double).  The receiving PE
420	   uses these new flags for consistency check and MAY generate an alarm
421	   if it detects inconsistency but doesn't bring down the VPWS service.

423	   It should be noted that in this mode of operation, the PE sends a
424	   single Ethernet A-D per EVI route for each AC that is configured -
425	   i.e., each normalized VID that is configured per ES results in
426	   generation of an EVPN Ethernet A-D per EVI.

428	   This mode of operation provides automatic cross checking of
429	   normalized VIDs used for EVPL services because these VIDs are
430	   signaled in EVPN BGP.  For example, if the same normalized VID is
431	   configured on three PE devices (instead of two) for the same EVI,
432	   then when a PE receives the second Ethernet A-D per EVI route, it
433	   generates an error message unless the two Ethernet A-D per EVI routes
434	   include the same ESI.  Such cross-checking is not feasible in default
435	   FXC mode because the normalized VIDs are not signaled.

[minor]
In th etext above there is no normative language, but only 
descriptive informational language. Is that intentional?

Proposed readability rewrite (re-using the informational language):
"In this operational mode, similar to the default Flexible 
Cross-Connect (FXC) mode described in Section 3.2, multiple 
normalized VLAN IDs (VIDs) representing Attachment Circuits (ACs) 
across various Ethernet Segments (ESs)/interfaces are 
multiplexed into a single EVPN VPWS service tunnel. However, 
this single tunnel is represented by multiple VPWS service 
IDs (one per normalized VID), and these normalized VIDs are 
signaled using EVPN BGP.

In this solution, on each Provider Edge (PE), the multi-homing 
ACs represented by their normalized VIDs are configured with 
a single Ethernet Virtual Instance (EVI). There is no need 
to configure a separate VPWS service instance ID here, as 
it corresponds to the normalized VID. For each normalized 
VID on each ES, the PE generates an Ethernet 
Auto-Discovery (A-D) per EVI route where the ESI field 
represents the ES ID, the Ethernet Tag field is set to 
the normalized VID, and the MPLS label field is set to 
a dynamically generated EVPN label representing the 
point-to-point (P2P) EVPN service tunnel. This label 
remains consistent for all ACs multiplexed into a single 
EVPN VPWS service tunnel. This route is sent with a Route 
Target (RT) representing the EVI. As before, this RT can 
be auto-generated from the EVI per Section 5.1.2.1 of [RFC8365]. 
Additionally, this route includes the EVPN Layer-2 Extended 
Community defined in Section 3.1 of [RFC8214] with two 
new flags (outlined in Section 4) that indicate: 1) this 
VPWS service tunnel is for VLAN-signaled Flexible 
Cross-Connect, and 2) the normalized VID type 
(single versus double). The receiving PE uses 
these new flags for a consistency check and may 
generate an alarm if it detects inconsistency, but it 
does not bring down the VPWS service.

It should be noted that in this mode of operation, the 
PE sends a single Ethernet A-D per EVI route for each 
AC configured-i.e., each normalized VID configured per ES 
results in the generation of an EVPN Ethernet A-D per EVI.

This mode of operation enables automatic cross-checking 
of normalized VIDs used for Ethernet Virtual Private Line 
(EVPL) services because these VIDs are signaled in EVPN BGP. 
For instance, if the same normalized VID is configured on 
three PE devices (instead of two) for the same EVI, then 
when a PE receives the second Ethernet A-D per EVI route, it 
generates an error message unless the two Ethernet A-D per 
EVI routes include the same ESI. Such cross-checking is not 
feasible in the default FXC mode because the normalized 
VIDs are not signaled.
"

437	3.3.1.  Local Switching

439	   When cross-connection is between two ACs belonging to two multi-homed
440	   Ethernet Segments on the same set of multi-homing PEs, then
441	   forwarding between the two ACs MUST be performed locally during
442	   normal operation (e.g., in absence of a local link failure) - i.e.,
443	   the traffic between the two ACs MUST be locally switched within the
444	   PE.

446	   In terms of control plane processing, this means that when the
447	   receiving PE receives an Ethernet A-D per-EVI route whose ESI is a
448	   local ESI, the PE does not alter its forwarding state based on the
449	   received route.  This ensures that the local switching takes
450	   precedence over forwarding via MPLS/IP network.  This scheme of
451	   locally switched preference is consistent with baseline EVPN
452	   [RFC7432] where it describes the locally switched preference for
453	   MAC/IP routes.

455	   In such scenarios, the Ethernet A-D per EVI route should be
456	   advertised with the MPLS label either associated with the destination
457	   Attachment Circuit or with the destination Ethernet Segment in order
458	   to avoid any ambiguity in forwarding.  In other words, the MPLS label
459	   cannot represent the same VID-VRF used in Section 3.3 because the
460	   same normalized VID can be reachable via two Ethernet Segments.  In
461	   case of using MPLS label per destination AC, then this same solution
462	   can be used for VLAN-based VPWS or VLAN-bundle VPWS services per
463	   [RFC8214].

[minor]
In the above section there is only a single normative MUST. Is all 
the remainder of the text here informative? no more normative 
blobs of texts and procedures?

Proposed readability rewrite:
"When cross-connection occurs between two Attachment Circuits (ACs) 
belonging to two multi-homed Ethernet Segments on the same set 
of multi-homing Provider Edges (PEs), the forwarding between 
the two ACs must be conducted locally during normal operations 
(e.g., in the absence of a local link failure). This means that 
traffic between the two ACs MUST be switched locally within the PE.

In terms of control plane processing, this indicates that 
when the receiving PE processes an Ethernet Auto-Discovery (A-D) 
per-EVI route with a local Ethernet Segment Identifier (ESI), the 
PE does not modify its forwarding state based on the received 
route. This approach ensures that local switching takes precedence 
over forwarding via the MPLS/IP network. This method of 
prioritizing locally switched traffic aligns with the baseline 
EVPN principles described in [RFC7432], where locally switched 
preference is specified for MAC/IP routes.

In such scenarios, the Ethernet A-D per EVI route should be 
advertised with the MPLS label either associated with the 
destination Attachment Circuit or with the destination Ethernet 
Segment to eliminate any ambiguity in forwarding. In other words, 
the MPLS label cannot represent the same VID-VRF outlined in 
Section 3.3, as the same normalized VLAN ID can be reachable 
via two different Ethernet Segments. In the case of using an 
MPLS label per destination AC, this approach can also be applied 
to VLAN-based VPWS or VLAN-bundle VPWS services as per [RFC8214].
"

465	3.4.  Service Instantiation

467	   The V field defined in Section 4 is OPTIONAL.  However, when
468	   transmitted, its value could be flagging an error condition which may
469	   result in an operational issue.  Notification to operator of an error
470	   is not sufficient, the VPWS service tunnel must not be established.

472	   If both PEs of a VPWS tunnel are signaling a matching Normalised VID
473	   in control plane, yet one is operating in single tag and the other in
474	   double tag mode, the signaling of V-bit allows for detecting and
475	   preventing this tunnel instantiation.

477	   If single VID normalization is signaled in the Ethernet Tag ID field
478	   (12-bits) yet dataplane is operating based double tags, the VID
479	   normalization applies only to outer tag.  If double VID normalization
480	   is signaled in the Ethernet Tag ID field (24-bits), VID normalization
481	   applies to both inner and outer tags.

[minor]
Proposed readability re-edit:

"The V field, defined in Section 4, is OPTIONAL. However, if transmitted, its 
value may indicate an error condition that could lead to operational 
issues. In such cases, merely notifying the operator of an error 
is insufficient; the VPWS service tunnel must not be established.

If both Provider Edges (PEs) of a VPWS tunnel are signaling a 
matching Normalized VLAN ID (VID) in the control plane, but one 
is operating in single-tag mode and the other in double-tag mode, 
the signaling of the V-bit facilitates the detection and 
prevention of this tunnel's instantiation.

If single VID normalization is indicated in the Ethernet 
Tag ID field (12 bits) but the data plane is operating based 
on double tags, the VID normalization applies only to the outer 
tag. Conversely, if double VID normalization is signaled in the 
Ethernet Tag ID field (24 bits), VID normalization applies to 
both the inner and outer tags.
"

483	4.  BGP Extensions

485	   This draft uses the EVPN Layer-2 attribute extended community defined
486	   in [RFC8214] with two additional flags added to this EC as described
487	   below.  This EC is sent with Ethernet A-D per EVI route per
488	   Section 3, and SHOULD be sent for Single-Active and All-Active
489	   redundancy modes.

[minor]
readability re-edit:
"This draft utilizes the EVPN Layer-2 attribute extended community 
as defined in [RFC8214], with two additional flags incorporated into 
this Extended Community (EC) as detailed below. This EC is transmitted 
with the Ethernet Auto-Discovery (A-D) per Ethernet Virtual Instance 
(EVI) route as specified in Section 3 and SHOULDhould be sent for both 
Single-Active and All-Active redundancy modes.
"

528	5.  Failure Scenarios

530	   Two examples will be used as an example to analyze the failure
531	   scenarios.

533	   The first scenario is a default Flexible Xconnect with Multi- Homing
534	   solution and it is depicted in Figure 1.  In this case, the same VID
535	   Normalization as in the previous example is performed, however there
536	   is not an individual Ethernet A-D per EVI route per normalized VID,
537	   but per bundle of ACs on an ES.  That is, PE1 will advertise two
538	   Ethernet A-D per EVI routes: the first one will identify the ACs on
539	   p1's ES and the second one will identify the AC2 in p2's ES.
540	   Similarly, PE2 will advertise two Ethernet A-D per EVI routes.

[major]
It is mentioned "the previous example". What previous example? This is 
the first example in this section. I suspect this is left=over from a 
previous edit of the text? Can this be sanity checked or point better 
to the example intended

566	   The second scenario is depicted in Figure 2 and shows the
567	   VLAN-signaled FXC mode with Multi-Homing.  In this example:

569	   *  CE1 is connected to PE1 and PE2 via (port,vid)=(p1,1) and (p3,3)
570	      respectively.  CE1's VIDs are normalized to value 1 on both PEs,
571	      and CE1 is Xconnected to CE3's VID 1 at the remote end.

573	   *  CE2 is connected to PE1 and PE2 via ports p2 and p4 respectively:

575	      -  (p2,1) and (p4,3) identify the ACs that are used to Xconnect
576	         CE2 to CE4's VID 2, and are normalized to value 2.

578	      -  (p2,2) and (p4,4) identify the ACs that are used to Xconnect
579	         CE2 to CE5's VID 3, and are normalized to value 3.

581	   In this scenario, PE1 and PE2 advertise an Ethernet A-D per EVI route
582	   per normalized VID (values 1, 2 and 3), however only two VPWS Service
583	   Tunnels are needed: VPWS Service Tunnel 1 (sv.T1) between PE1's FXC
584	   service and PE3's FXC, and VPWS Service Tunnel 2 (sv.T2) between
585	   PE2's FXC and PE3's FXC.

[minor] 
re-edit from a readability perspective:

"The second scenario, depicted in Figure 2, illustrates the VLAN-signaled 
Flexible Xconnect (FXC) mode with Multi-Homing. In this example:

* Customer Edge 1 (CE1) is connected to Provider Edge 1 (PE1) and Provider 
Edge 2 (PE2) via (port, VID) = (p1,1) and (p3,3), respectively. CE1's VIDs 
are normalized to value 1 on both PEs, and CE1 is cross-connected to CE3's 
VID 1 at the remote end.

* Customer Edge 2 (CE2) is connected to PE1 and PE2 via ports p2 
and p4, respectively:

** The combinations (p2,1) and (p4,3) identify the Attachment 
Circuits (ACs) used to cross-connect CE2 to CE4's VID 2, which 
are normalized to value 2.

** The combinations (p2,2) and (p4,4) identify the ACs used 
to cross-connect CE2 to CE5's VID 3, which are normalized to 
value 3.

In this scenario, PE1 and PE2 each advertise an Ethernet 
Auto-Discovery (A-D) per Ethernet Virtual Instance (EVI) route 
for each normalized VID (values 1, 2, and 3). However, only two 
VPWS Service Tunnels are required: VPWS Service Tunnel 1 (sv.T1) 
between PE1's FXC service and PE3's FXC, and VPWS Service 
Tunnel 2 (sv.T2) between PE2's FXC and PE3's FXC.
"

618	5.2.  Attachment Circuit Failure

620	   In case of AC Failure, the VLAN-Signaled and default FXC modes behave
621	   in a different way:

623	   *  Default FXC (Figure 1): a VLAN or AC failure is not signaled in
624	      the default mode, therefore in case of an AC failure, e.g.  VID1
625	      on CE2, nothing prevents PE3 from sending CE4's traffic to PE1,
626	      creating a black-hole.  Application layer OAM may be used if per-
627	      VLAN fault propagation is required in this case.

629	   *  VLAN-signaled FXC (Figure 2): a VLAN or AC failure, e.g.  VID1 on
630	      CE2, triggers the withdrawal of the Ethernet A-D per EVI route for
631	      the corresponding Normalized VID, that is, Ethernet-Tag 2.  When
632	      PE3 receives the route withdrawal, it will remove PE1 from its
633	      path-list for traffic coming from CE4.

[minor]
proposed re-edit from readability perspective:

"In the event of an Attachment Circuit (AC) Failure, the VLAN-Signaled 
and default FXC modes exhibit distinct behaviors:

* Default FXC (Figure 1): In the default mode, a VLAN or AC failure 
is not signaled. Consequently, in the event of an AC failure, such 
as VID1 on Customer Edge 2 (CE2), there is nothing to prevent 
Provider Edge 3 (PE3) from directing traffic from Customer 
Edge 4 (CE4) to Provider Edge 1 (PE1), leading to a potential 
black hole. Application layer Operations, Administration, and 
Maintenance (OAM) may be utilized if per-VLAN fault propagation 
is necessary in this scenario.

* VLAN-Signaled FXC (Figure 2): In the case of a VLAN or 
AC failure, such as VID1 on CE2, the failure triggers the 
withdrawal of the Ethernet Auto-Discovery (A-D) per Ethernet 
Virtual Instance (EVI) route for the corresponding Normalized VID, 
specifically Ethernet-Tag 2. Upon receiving the route withdrawal, 
PE3 will remove PE1 from its path list for traffic originating 
from CE4.
"

635	5.3.  PE Port Failure

637	   In case of PE port Failure, the failure will be signaled and the
638	   other PE will take over in both cases:

640	   *  Default FXC (Figure 1): a port failure, e.g. p2, is signaled by
641	      route for sv.T2 will also be withdrawn.  Upon receiving the fault
642	      notification, PE3 will remove PE1 from its path-list for traffic
643	      coming from CE4 and CE5.

645	   *  VLAN-signaled FXC (Figure 2): a port failure, e.g. p2, triggers
646	      the withdrawal of the Ethernet A-D per EVI routes for Normalized
647	      VIDs 2 and 3, as well as the withdrawal of the Ethernet A-D per ES
648	      route for p2's ES.  Upon receiving the fault notification, PE3
649	      will withdraw PE1 from its path-list for the traffic coming from
650	      CE4 and CE5.

[minor]
Proposed re-edit from readability perspective

"In the event of a Provider Edge (PE) port failure, the failure will be 
signaled, and the alternative PE will assume responsibility in both 
scenarios:

* Default FXC (Figure 1): In the case of a port 
failure, such as p2, the route for Service Tunnel 2 (sv.T2) will be 
withdrawn. Upon receiving the fault notification, Provider Edge 3 
(PE3) will remove Provider Edge 1 (PE1) from its path list for 
traffic originating from Customer Edge 4 (CE4) and Customer Edge 5 (CE5).

* VLAN-Signaled FXC (Figure 2): A port failure, such as p2, triggers 
the withdrawal of the Ethernet Auto-Discovery (A-D) per Ethernet 
Virtual Instance (EVI) routes for Normalized VLAN IDs (VIDs) 2 and 3, 
along with the withdrawal of the Ethernet A-D per Ethernet Segment 
(ES) route for p2's ES. Upon receiving the fault notification, PE3 
will remove PE1 from its path list for the traffic originating 
from CE4 and CE5.
"


665	7.  IANA Considerations

667	   This document requests allocation of bits 4-7 in the "EVPN Layer 2
668	   Attributes Control Flags" registry with names M and V:

670	      M     Signaling mode of operation (2 bits)
671	      V     VLAN-ID normalization (2 bits)

[major]
The below confuses me. On the lines 501-505 there are 16 bits, 
and the bits indicated are bits 8-11 (see below). How does this 
correlated with the allocation of bits 4-7 requested by the authors to IANA?

501	                            1 1 1 1 1 1
502	        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
503	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
504	       | MBZ           | V | M |-|C|P|B|    (MBZ = MUST Be Zero)
505	       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

G/