Re: [Bier] AD Review of draft-ietf-bier-te-arch-10

Toerless Eckert <tte@cs.fau.de> Fri, 28 January 2022 17:12 UTC

Return-Path: <eckert@i4.informatik.uni-erlangen.de>
X-Original-To: bier@ietfa.amsl.com
Delivered-To: bier@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5167E3A0BCB; Fri, 28 Jan 2022 09:12:03 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.649
X-Spam-Level:
X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3N18fGvyB62n; Fri, 28 Jan 2022 09:11:59 -0800 (PST)
Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [131.188.34.40]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 46A783A0A06; Fri, 28 Jan 2022 09:11:49 -0800 (PST)
Received: from faui48e.informatik.uni-erlangen.de (faui48e.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:51]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTPS id 4EBDA58C4B2; Fri, 28 Jan 2022 18:11:43 +0100 (CET)
Received: by faui48e.informatik.uni-erlangen.de (Postfix, from userid 10463) id 4AAFE4EA4BD; Fri, 28 Jan 2022 18:11:43 +0100 (CET)
Date: Fri, 28 Jan 2022 18:11:43 +0100
From: Toerless Eckert <tte@cs.fau.de>
To: Alvaro Retana <aretana.ietf@gmail.com>
Cc: draft-ietf-bier-te-arch@ietf.org, "Gengxuesong (Geng Xuesong)" <gengxuesong@huawei.com>, BIER WG Chairs <bier-chairs@ietf.org>, BIER WG <bier@ietf.org>
Message-ID: <YfQjzzWgJLBrDrfF@faui48e.informatik.uni-erlangen.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAMMESsyGszotDPReWZKxCTZ3w6m_zz+3fhU0XUYMPo=DAS0d9w@mail.gmail.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/bier/rkJxZS3KUJn6Pf9ZsHbpoat7G_E>
Subject: Re: [Bier] AD Review of draft-ietf-bier-te-arch-10
X-BeenThere: bier@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "\"Bit Indexed Explicit Replication discussion list\"" <bier.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bier>, <mailto:bier-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bier/>
List-Post: <mailto:bier@ietf.org>
List-Help: <mailto:bier-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bier>, <mailto:bier-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 28 Jan 2022 17:12:09 -0000

Alvaro, *

-12 as just posted should close all outstanding DISCUSS (the one from Ben) as well as
answer all IESG/*directorate review nits/comments, so this could/should be the
version to post to RFC-Editor queue.

No functional changes where incurred through any of the reviews, except of course
fixing of the pseudocode by moving the "~" the right code line through Ben's DISCUSS,
so this should all be benign. 

There are a few language questions open with [RFC-Editor ] notes preceeding them.
Spell check run.

Specifically for your review:

- The leftover fixes from -09 where applied (move SR into appendix, removed
  beauty contest header from example section, which will  be removed in RFC).

- One of your nits made me rewrite the paragraphs of section 4.1 leading up to
  Figure 4, but i felt it was important to eliminate confusion about terminology
  between BIER/BIER-TE, and your reply seemed to indicate the existing text
  (nor your suggestion) did achieve that.

Full diff from -11 to -12:

http://tools.ietf.org//rfcdiff?url1=https://www.ietf.org/archive/id/draft-ietf-bier-te-arch-11.txt&url2=https://www.ietf.org/archive/id/draft-ietf-bier-te-arch-12.txt

Diff with just the deltas for your feedback:

http://tools.ietf.org//rfcdiff?url1=https://raw.githubusercontent.com/toerless/bier-te-arch/fbb26ac4d2cc5dd841e36a00801f58ad5c7eae11/draft-ietf-bier-te-arch.txt&url2=https://raw.githubusercontent.com/toerless/bier-te-arch/3391606088a0a73513e6be40a895afb27493f244/draft-ietf-bier-te-arch.txt

Thanks again for your review.

Toerless

On Tue, Jul 20, 2021 at 03:21:31PM -0400, Alvaro Retana wrote:
> Hi!
> 
> This is a quick review of -10.  Most of the comments are minor or nits.
> 
> Thanks!
> 
> Alvaro.
> 
> 
> 
> [Line numbers from idnits for -10.]
> 
> ...
> 135	1.  Overview
> 
> 137	   BIER-TE is based on architecture, terminology and packet formats with
> 138	   BIER as described in [RFC8279] and [RFC8296].  This document
> 139	   describes BIER-TE in the expectation that the reader is familiar with
> 140	   these two documents.
> 
> [minor] s/based on architecture, terminology and packet formats with
> BIER as described/based on the BIER architecture, terminology and
> packet formats with as described

Fixed.

> ...
> 180	   o  Section 5 describes operational considerations for the BIER-TE
> 181	      controller, foremost how the BIER-TE controller can optimize the
> 182	      use of BP by using specific type of BIER-TE adjacencies for
> 183	      different type of topological situations, but also how to assign
> 184	      bits to avoid loops and duplicates (which in BIER-TE does not come
> 185	      for free), and finally how SI, sub-domains and BFR-ids can be
> 186	      managed by a BIER-TE controller, examples and summary.
> 
> [nit] s/by using specific type of BIER-TE adjacencies for different
> type of topological situations/by using specific BIER-TE adjacencies
> in different topological situations
> 
> 
> 188	   o  Section 6 concludes the technology specific sections of document
> 189	      by further relating BIER-TE to Segment Routing (SR).
> 
> [nit] s/of document/of the document

Fixed already through other reviewer feedback.

> ...
> 335	2.2.  BIER-TE Topology and adjacencies
> ...
> 342	   The BIER-TE Topology consists of the BIFTs of all the BFR and can
> 343	   also be expressed as a directed graph where the edges are the
> 344	   adjacencies between the BFR labelled with the BP used for the
> 345	   adjacency.  Adjacencies are naturally unidirectional.  BP can be
> 346	   reused across multiple adjacencies as long as this does not lead to
> 347	   undesired duplicates or loops as explained further down in the text.
> 
> [nit] s/between the BFR/between the BFRs

Fixed.

> ...
> 357	2.3.  Relationship to BIER
> 
> 359	   BIER-TE is designed so that is forwarding plane is a simple extension
> 360	   to the BIER forwarding plane, hence allowing for it to be added to
> 361	   BIER deployments where it can be beneficial.
> 
> [nit] s/is forwarding/its forwarding

Fixed already through other reviewer feedback.

> ...
> 370	       1.  The fundamental purpose of per-packet signaled packet
> 371	           replication and delivery via a BitString.
> 
> [nit] s/per-packet signaled packet replication/per-packet signaled replication

Fixed.

> ...
> 376	       3.  The supportable encapsulations, [RFC8296] or other (future)
> 377	           encapsulations.
> 
> [nit] s/supportable encapsulations, [RFC8296]/supported encapsulations [RFC8296]

Fixed.

> ...
> 412	       1.  BIRTs on BFR for BIER-TE are not required when using a BIER-
> 413	           TE controller because the controller can directly populate
> 414	           the BIFTs.  In BIER, BIRTs are populated by the distributed
> 415	           routing protocol support for BIER, allowing BFR to populate
> 416	           their BIFTs locally from their BIRTs.  Other BIER-TE control
> 417	           plane or management plane options may introduce requirements
> 418	           for BIRTs for BIER-TE BFR.
> 
> [minor] Expand first occurrence of BIRT.

Fixed.

> 420	       2.  The BIER-TE layer forwarding plane does not require BFR to
> 421	           have a unique BP and therefore also no unique BFR-id.  See
> 422	           for example See Section 5.1.3.
> 
> [nit] s/require BFR/require a BFR

actually meant to say "require BFRs".

> ...
> 436	       1.  The BIER/BIER-TE packet header needs to allow addressing both
> 437	           BIER and BIER-TE BIFT.  Depending on the encapsulation
> 438	           option, the same SD may or may not be reusable across BIER
> 439	           and BIER-TE.  See Section 4.3.  In either case, a packet is
> 440	           always only forwarded end-to-end via BIER or via BIER-TE
> 441	           (ships in the nights forwarding).
> 
> [nit] s/both BIER and BIER-TE BIFT/both the BIER and BIER-TE BIFTs

fixed.

> 443	       2.  BIER-TE deployments will have to assign BFR-ids to BFR and
> 444	           insert them into the BFR-id field of BIER packet headers as
> 445	           BIER does, whenever the deployment uses (unchanged)
> 446	           components developed for BIER that use BFR-id, such as
> 447	           multicast flow overlays or BIER layer control plane elements.
> 448	           See also Section 5.3.3.
> 
> [nit] s/to BFR/to BFRs

already fixed.

> 450	2.4.  Accelerated/Hardware forwarding comparison
> 
> 452	   Forwarding of BIER-TE is designed to easily build/program common
> 453	   forwarding hardware with BIER.  The pseudocode in Section 4.4 shows
> 454	   how existing BIER/BIFT forwarding can be modified to support the
> 455	   REQUIRED BIER-TE forwarding functionality, by using BIER BIFT's
> 456	   "Forwarding Bit Mask" (F-BM): Only the clearing of bits to avoid
> 457	   duplicate packets to a BFR neighbor is skipped in BIER-TE forwarding
> 458	   because it is not necessary and could not be done when using BIER
> 459	   F-BM.
> 
> [major] s/REQUIRED BIER-TE forwarding/required BIER-TE forwarding
> In context, there is no normative action, just a statement of fact.
> Nonetheless, a reference to §4.6 would be nice.

done.

> 501	3.2.  The BIER-TE Control Plane
> ...
> 509	   For BIER-TE, the control plane includes at minimum the following
> 510	   functionality.
> 
> [minor] It would be useful to point at the sections where each piece
> of functionality is described -- even if it is at just a high-level of
> granularity.

Done. I also added one additional paragraph to 3.1, Flow Overlay which
so far just had a measly one line of text:

<t>When a BIER-TE controller is used, then the signalling for the Multicast Flow Overlay may
also be preferred to operate through a central point of control. For BGP based
overlay flow services such as "Multicast VPN Using BIER" (<xref target="RFC8556"/>) this
can be achieved by making the BIER-TE controller operate as a BGP Route
Reflector (<xref target="RFC4456"/>) and combining it with signaling through BGP
or a different protocol for the BIER-TE controller calculated BitStrings.
See <xref target="engineered-bitstrings"/> and <xref target="bitstring-mappings"/>.</t>

Hope this gives a good high-level BIER-TE centric view of where this could
go (beyond all the explanations in rfc8279).

> ...
> 545	3.2.1.  The BIER-TE Controller
> 
> 547	   Notwithstanding other options, this architecture describes the BIER
> 548	   control plane as shown in Figure 3 to consists of:
> 
> [nit] s/to consists/to consist

fixed.

> 550	   o  A single centralized BIER-TE controller.
> 
> [minor] Perhaps remove "single centralized" which gives the impression
> of single-point-of-failure/attack point -- where we all know that in
> general the controller may not be a single box, etc..

fixed.

> 552	   o  Data-models and protocols to communicate between controller and
> 553	      BFR in step 1, such YANG/Netconf/RestConf.
> 
> [?] Step 1?  I think that you're referring to the list in the previous
> section -- please indicate there that they represent "steps".  On
> second thought, it looks like only a couple of places refer back to
> the list as steps; maybe find an alternate way to refer to them.

For anothre reviewer, 3.2 was improved to introduce the terms
"BIER-TE topology control" and "BIER-TE tree control" for the
main bullet points and those are then referred to in this 3.2.1 bullet
point list. I hope this solves your concerns.

> [minor] Only in "step 1"? 

was removed as part of above points modification..

> [minor] There is some programmability-related work to be done in step 2.

I now have "BFR data-models and protocols ... in support of BIER-TE topology/tree control" for both steps.

When you mention "programmability", not sure exactly what you are referring
to, but i stumbled across "The BIFT is programmed into the data plane" in 4.1,
and replaced that with consistent language:

<t>The BIFT is configured for the data plane of a BFR by the BIER-TE
Controller through an appropriate protocol and data-model.

Aka: avoiding "programm(ability)" for any of this and just using it
data-model and protocols.

> 
> 555	   o  Protocols to communicate between controller and BFIR in step 2,
> 556	      such as BIER-TE extensions for [RFC5440].
> 
> [minor] Same question...only step 2?

Also removed.

> [minor] It's not clear to me why (it  any extensions are TBD) a PCE can't be used for step 1, and viceversa.

Not sure i parse the sentence, but indeed i do consider a BIER-TE
controller to include a PCE. 

> 567	3.2.1.1.  BIER-TE Topology discovery and creation
> 
> 569	   Step 1.1 includes network topology discovery and BIER-TE topology
> 570	   creation.  The latter describes the process by which a Controller
> 571	   determines which routers are to be configured as BFR and the
> 572	   adjacencies between them.
> 
> [nit] s/as BFR/as BFRs

fixed.

> 583	   Dynamic creation of the BIER-TE topology can be as easy as mapping
> 584	   the network topology 1:1 to the BIER-TE topology by assigning a BP
> 585	   for every network subnet adjacency.  In larger networks, it likely
> 586	   involves more complex policy and optimization decisions including how
> 587	   to minimize the number of BP required and how to assign BP across
> 588	   different BitStrings to minimize the number of duplicate packets
> 589	   across links when delivering an overlay flow to BFER using different
> 590	   SIs/BitStrings.  These topics are discussed in Section 5.
> 
> [nit] s/number of BP/number of BPs

fixed.

> [nit] s/assign BP/assign BPs

fixed.

> 598	   Communications between the BIER-TE Controller and BFRs (beside
> 599	   topology discovery) is ideally via standardized protocols and data-
> 600	   models such as Netconf/RestConf/Yang/PCEP.  Vendor-specific CLI on
> 601	   the BFRs is also an option (as in many other SDN solutions lacking
> 602	   definition of standardized data model).
> 
> [nit] s/standardized data model/standardized data models

fixed.

> 604	3.2.1.2.  Engineered Trees via BitStrings
> 
> 606	   In BIER, the same set of BFER in a single sub-domain is always
> 607	   encoded as the same BitString.  In BIER-TE, the BitString used to
> 608	   reach the same set of BFER in the same sub-domain can be different
> 609	   for different overlay flows because the BitString encodes the paths
> 610	   towards the BFER, so the BitStrings from different BFIR to the same
> 611	   set of BFER will often be different, and the BitString from the same
> 612	   BFIR to the same set of BFER can different for different overlay
> 613	   flows for policy reasons such as shortest path trees, Steiner trees
> 614	   (minimum cost trees), diverse path trees for redundancy and so on.
> 
> [nit] s/can different/can be different

already fixed.

> 640	3.3.  The BIER-TE Forwarding Plane
> 
> 642	   The BIER-TE Forwarding Plane constitutes of the following components:
> 
> [nit] s/constitutes of/is constituted by

Other reviewers suggested other solutions, i now picked "is constituted from"
and have an RFC-editor note to resolve it with RFC-editor.

> 644	   1.  On BFIR imposition of BIER header for packets from overlay flows.
> 645	       This is driven by a combination of state established by the BIER-
> 646	       TE control plane and/or the multicast flow overlay as explained
> 647	       in Section 3.1.
> 
> [nit] s/On BFIR/On a BFIR,

fixed.

> [nit] s/of BIER header/of the BIER header
> 
> 
> 649	   2.  On BFR (including BFIR and BFER), forwarding/replication of BIER
> 650	       packets according to their BitString as explained below and
> 651	       optionally Entropy.  Processing of other BIER header fields such
> 652	       as DSCP is outside the scope of this document.
> 
> [nit] s/On BFR/On a BFR

fixed.

> [minor] "as explained below"   A specific reference would be nice.

fixed.

> [major] "Processing of other BIER header fields such as DSCP is
> outside the scope of this document."   Better wording might be that it
> is unchanged.  Is that the intent?

No change.

"Unchanged" would make me cringe because it would sound like the reader
would find useful prior information, but rfc8279 doesn't mention
additional fields (such as DSCP) at all, and even rfc8296 just says
that its not used by MPLS encap by default. And both docs use out
of scope for other things too so its a favourite way to define the
boundaries of our docs.

Remember that rfc8296 is a "committee header" to try to apease both
MPLS and IP desires for BIER, but the IP side did not have much interest,
so the idea is "when you use BIER with IP, fields like DSCP and TTL
are probably exactly as with IP... but we haven't really thoroughly
studied or documented that".

> 654	   3.  On BFER removal of BIER header and dispatching of the payload
> 655	       according to state created by the BIER-TE control plane and/or
> 656	       overlay layer.
> 
> [nit] s/On BFER/On a BFER,

Actually changed to BFRs, better matches the fact that it can be multiple (for one original packet after replication).

> ...
> 746	4.1.  The Bit Index Forwarding Table (BIFT)
> ...
> 756	   In [RFC8279], Figure 2, indices into the BIFT are both SI:BitString
> 757	   and BFR-id, where BitString is indicating a BP: BFR-id = SI * 2^BSL +
> 758	   BP.  As shown in Figure 4, in BIER-TE, only SI:BP are used as indices
> 759	   into a BIFT because they identify adjacencies and not BFR.
> 
> [minor] I think that the formula makes things sound more complicated.
> Suggestion>
>    Figure 2 of [rfc8279] uses both SI:BitString and BFR-id as indices into
>    the BIFT.  As shown in Figure 4 (below), in BIER-TE only SI:BP are used
>    as indices into a BIFT because they identify adjacencies and not BFR.

Ok., there is a bug in the formula... but its more difficult to explain everything
without formula. And your suggestion makes me think my text was confusing *sigh*.

So, i ended up rewriting the paragraphs of 4.1 leading up to the BIFT figure,
and also improved the BIFT figure. Got a bit longer *sigh*, but its really difficult
to ensure no confusion ensues when not being detailled about all the terms used and
what they do. Pls. check.

> ...
> 835	4.2.3.  ECMP
> ...
> 840	   A BIER-TE "Equal Cost Multipath" (ECMP) adjacency has a list of two
> 841	   or more non-ECMP adjacencies and a seed parameter.  When a BIER-TE
> 842	   packet is copied onto such an ECMP adjacency, an implementation
> 843	   specific so-called hash function will select one out of the lists
> 844	   adjacencies to which the packet is forwarded.  This ECMP hash
> 845	   function MUST select the same adjacency from that list for all
> 846	   packets with the same entropy parameter.  The seed parameter allows
> 847	   to design hash functions that are easy to implement at high speed
> 848	   without running into polarization issues across multiple consecutive
> 849	   ECMP hops.  See Section 5.1.7 for more explanations.
> 
> [minor] s/non-ECMP/ECMP

That was actually correctly, but i further elaborated the sentence to point to the
picture showing the parameters:

<t>A BIER-TE "Equal Cost Multipath" (ECMP()) adjacency as shown in <xref target="adjacencies"/>
for BIFT-index 7 has a list of two or more non-ECMP adjacencies as parameters and an optional 
seed parameter.

> 861	4.3.  Encapsulation / Co-existence with BIER
> ...
> 881	   With MPLS, it is also possible to reuse the same SD space for both
> 882	   BIER-TE and BIER, so that the same SD has both a BIER BIFT and
> 883	   according range of BIFT-ids and a disjoint BIER-TE BIFT and non-
> 884	   overlapping range of BIFT-ids.
> 
> [minor] s/and according range of BIFT-ids/with a corresponding range
> of BIFT-ids,
> [nit] s/and non-overlapping/and a non-overlapping

Fixed.

> 886	   When a fixed mapping from BSL, SD, SI is used without specifically
> 887	   distinguishing BIER and BIER-TE, such as proposed for non-MPLS
> 888	   forwarding with [RFC8296] in [I-D.ietf-bier-non-mpls-bift-encoding]
> 889	   revision 04, section 5., then it is necessary to allocate disjoint
> 890	   SDs to BIER and BIER-TE BIFT so that both can be addressed by the
> 891	   BIFT-ids.  The encoding proposed in section 6. of the same document
> 892	   does not statically encode BSL or SD into the BIFT-id, but allows for
> 893	   a mapping, and hence could provide for the same freedom as when MPLS
> 894	   is being used (same or different SD for BIER/BIER-TE).
> 
> [nit] s/5.,/5,

fixed.

> [nit] s/BIER and BIER-TE BIFT/BIER and BIER-TE BIFTs

fixed.

> 908	4.4.  BIER-TE Forwarding Pseudocode
> ...
> 914	      void ForwardBitMaskPacket_withTE (Packet)
> 915	      {
> 916	          SI=GetPacketSI(Packet);
> 917	          Offset=SI*BitStringLength;
> 918	          for (Index = GetFirstbit position(Packet->BitString); Index ;
> 919	               Index = GetNextbit position(Packet->BitString, Index)) {
> 920	              F-BM = BIFT[Index+Offset]->F-BM;
> 921	              if (!F-BM) continue;                            [3]
> 922	              BFR-NBR = BIFT[Index+Offset]->BFR-NBR;
> 923	              PacketCopy = Copy(Packet);
> 924	              PacketCopy->BitString &= F-BM;                  [2]
> 925	              PacketSend(PacketCopy, BFR-NBR);
> 926	              // The following must not be done for BIER-TE:
> 927	              // Packet->BitString &= ~F-BM;                  [1]
> 928	          }
> 929	      }
> 
> [nit] s/GetFirstbit position/GetFirstbitPosition
> [nit] s/GetNextbit position/GetNextbitPosition

fixed.

> 974	   This Forwarding Pseudocode can support the REQUIRED BIER-TE
> 975	   forwarding functions (see Section 4.6), forward_connected,
> 976	   forward_routed() and local decap, but not the RECOMMENDED functions
> 977	   DNC flag and multiple adjacencies per bit nor the OPTIONAL function,
> 978	   ECMP adjacencies.  The DNC flag cannot be supported when using only
> 979	   [1] to mask bits.
> 
> [major] s/REQUIRED...RECOMMENDED...OPTIONAL/required...recommended...optional
> This text is not normative, just a description.

Fixed.

> 1144	4.6.  BFR Requirements for BIER-TE forwarding
> 
> 1146	   BFR MUST support to configure the BIFT of sub-domains so that they
> 1147	   use BIER-TE forwarding rules instead of BIER forwarding rules.  Every
> 1148	   BP in the BIFT MUST support to have zero or one adjacency.
> 1149	   Forwarding MUST support the adjacency types forward_connected() with
> 1150	   clear DNC flag, forward_routed() and local_decap.  As explained in
> 1151	   Section 4.4, these REQUIRED BIER-TE forwarding functions can be
> 1152	   implement via the same Forwarding Pseudocode as BIER forwarding
> 1153	   except for one modification (skipping one masking with F-BM).
> 
> [minor] s/support to configure/support configuring

Actually changed the sentence through anothrer reviewer to read easier already.

> [nit] s/clear DNC flag/a clear DNC flag

Also changed by proposal of othre reviewer to "with DNC flag not set"

> [major] s/REQUIRED/required
> Not a normative statement.

fixed.

> 1159	   BIER-TE forwarding SHOULD support more than one adjacency on a bit.
> 1160	   This allows to save bits in hub&spoke scenarios (see Section 5.1.5).
> 
> [nit] s/hub&spoke/hub and spoke

fixed (also in other places).

> 1162	   BIER-TE forwarding MAY support ECMP adjacencies to save bits in ECMP
> 1163	   scenarios, see Section 5.1.7 for an example.  This is a MAY
> 1164	   requirement, because the deployment importance of ECMP adjacencies
> 1165	   for BIER-TE is unclear as one can also leverage ECMP of the routing
> 1166	   underlay via forwarded_routed adjacencies and/or might prefer to have
> 1167	   more explicit control of the path chosen via explicit BP/adjacencies
> 1168	   for each ECMP path alternative.
> 
> [major] s/a MAY requirement/an optional requirement

fixed.

> [minor] s/the deployment importance of ECMP adjacencies for BIER-TE is
> unclear as/for ECMP deployments using BIER-TE

fixed (nice shortening).

> 1183	5.1.1.  P2P Links
> 
> 1185	   On a P2P link that connects two BFR, the same bit position can be
> 1186	   used on both BFR for the adjacency to the neighboring BFR.  A P2P
> 1187	   link requires therefore only one bit position.
> 
> [nit] s/two BFR/two BFRs
> [nit] s/both BFR/both BFRs

fixed.

> 1194	5.1.3.  Leaf BFERs
> ...
> 1209	   A leaf BFERs is one where incoming BIER-TE packets never need to be
> 1210	   forwarded to another BFR but are only sent to the BFER to exit the
> 1211	   BIER-TE domain.  For example, in networks where Provider Edge (PE)
> 1212	   router are spokes connected to Provider (P) routers, those PEs are
> 1213	   Leaf BFERs unless there is a U-turn between two PEs.
> 
> [nit] s/A leaf BFERs/A leaf BFER

fixed already.

> 1492	5.1.9.  Reuse of bit positions (without DNC)
> ...
> 1544	                          Figure 17: Reuse of BP
> 
> 1546	   Reuse may also save BPs in larger topologies.  Consider the topology
> 1547	   shown in Figure 20.  A BFIR/sender (e.g.: video headend) is attached
> 1548	   to area 1, and area 2...6 contain receivers/BFER.  Assume each area
> 1549	   had a distribution ring, each with two BPs to indicate the direction
> 1550	   (as explained before).  These two BPs could be reused across the 5
> 1551	   areas.  Packets would be replicated through other BPs for the Core to
> 1552	   the desired subset of areas, and once a packet copy reaches the ring
> 1553	   of the area, the two ring BPs come into play.  This reuse is a case
> 1554	   of (B), but it limits the topology choices: Packets can only flow
> 1555	   around the same direction in the rings of all areas.  This may or may
> 1556	   not be acceptable based on the desired path steering options: If
> 1557	   resilient transmission is the path engineering goal, then it is
> 1558	   likely a good optimization, if the bandwidth of each ring was to be
> 1559	   optimized separately, it would not be a good limitation.
> 
> [minor] s/Figure 20/Figure 17

Fixed already.

> 1561	5.1.10.  Summary of BP optimizations
> ...
> 1603	   Note that the described list of optimizations is not exhaustive.
> 1604	   Especially when the set of required path steering choices is limited
> 1605	   and the set of possible subsets of BFERs that should be able to
> 1606	   receive traffic is limited, further optimizations of BP are possible.
> 1607	   The hub & spoke optimization is a simple example of such traffic
> 1608	   pattern dependent optimizations.
> 
> [nit] s/hub & spoke/hub and spoke

ack.

> 1757	5.3.3.  Assigning BFR-id with BIER-TE
> 
> 1759	   BIER-TE forwarding does not use the BFR-id, not does it require for
> 1760	   the BFR-id field of the BIER header to be set to a particular value.
> 1761	   However, other parts of a BIER-TE deployment may need a BFR-id,
> 1762	   specifically overlay signaling, and in that case BFR need to also
> 1763	   have BFR-ids for BIER-TE SDs.
> 
> [nit] s/BFR need/BFRs need

fixed.

> 1765	   For example, for BIER overlay signaling, BFIR need to have a BFR-id,
> 1766	   because this BFIR BFR-id is carried in the BFR-id field of the BIER
> 1767	   header to indicate to the overlay signaling on the receiving BFER
> 1768	   which BFIR originated the packet.
> 
> [nit] s/BFIR need/BFIRs need

fixed.

> 1781	   As explained in Section 5.1.3, leaf BFERs do not need such a unique
> 1782	   local_decap() adjacency, likewise, BFIR who are not also BFER may not
> 1783	   have a unique local_decap() adjacency either.  For all those BFIR and
> 1784	   (leaf) BFER, the controller needs to determine unique BFR-ids that do
> 1785	   not collide with the BFR-ids derived from the non-leaf BFER
> 1786	   local_decap() BPs.
> 
> [nit] s/BFIR who are not also BFER/BFIRs that are not also BFERs

fixed.

> [nit] s/BFIR and (leaf) BFER/BFIRs and (leaf) BFERs

fixed.

> 1788	   While this document defines no requirements how to allocate such BFR-
> 1789	   id, a simple option is to derive it from the (SI,BP) of an adjacency
> 1790	   that is unique to the BFR in question.  For a BFIR this can be he
> 1791	   first adjacency only populated on this BFIR, for a leaf-BFER, this
> 1792	   could be the first BP with an adjacency towards that BFER.
> 
> [nit] s/requirements how/requirements on how

fixed

> [nit] s/can be he/can be the

already fixed

> 1794	5.3.4.  Mapping from BFR to BitStrings with BIER-TE
> 
> 1796	   In BIER, applications of the flow overlay on a BFIR can calculate the
> 1797	   (SI,BP) of a BFER from the BFR-id of the BFER and can therefore
> 1798	   easily determine the BitStrings for a BIER packet to a set of BFER
> 1799	   with known BFR-ids.

> [nit] s/set of BFER/set of BFERs

fixed.

> 1817	   If "independent branches" are used, the BIER-TE Controller can signal
> 1818	   to the BFIR flow overlay for every BFER an SI:BitString that
> 1819	   represents the branch to that BFER.  The flow overlay on the BIFR can
> 1820	   then independently of the controller calculate the SI:BitString for
> 1821	   all desired BFER by OR'ing their BitStrings.  This allows for flow
> 1822	   overlay applications to operate independently from the controller
> 1823	   whenever it needs to determine which subset of BFERs need to receive
> 1824	   a particular packet.
> 
> 
> [nit] s/all desired BFER/all desired BFERs

fixed.

> 1838	   Communications between BFIR flow overlay and BIER-TE controller
> 1839	   requires some way to identify BFER.  If BFR-ids are used in the
> 1840	   deployment, as outlined in Section 5.3.3, then those are the natural
> 1841	   BFR identifier.  If BFR-ids are not used, then any other unique
> 1842	   identifier, such as the BFR-prefix of the BFR as of [RFC8279] could
> 1843	   be used.
> 
> [nit] s/BIER-TE controller/the BIER-TE controller

fixed.

> [nit] s/identify BFER/identify the BFER

fixed.

> [minor] s/BFR-prefix of the BFR as of [RFC8279]/BFR-prefix of the BFR [RFC8279]

fixed.

> ...
> 2031	7.  Security Considerations
> ...
> 2046	   The reference model for the BIER-TE layer control plane is a BIER-TE
> 2047	   controller.  When such a controller is used, impairment of individual
> 2048	   BFR in a domain causes no impairment of the BIER-TE control plane on
> 2049	   other BFR.  If a routing protocol is used to support forward_routed()
> 2050	   adjacencies, then this is still an attack vector as in BIER, but only
> 2051	   for BIER-TE forward_routed() adjacencies, and no other adjacencies.
> 
> [nit] s/of individual/of an individual

fixed.

> [nit] s/other BFR/other BFRs

fixed.

> [EoR -10]