Re: [Bier] Comments on draft-chen-bier-frr-02.

Tony Przygienda <> Wed, 17 March 2021 20:34 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 00DC73A1369 for <>; Wed, 17 Mar 2021 13:34:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.996
X-Spam-Status: No, score=-1.996 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_DNSWL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id NwtLD7MtRbfJ for <>; Wed, 17 Mar 2021 13:34:22 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id ADC403A136B for <>; Wed, 17 Mar 2021 13:34:22 -0700 (PDT)
Received: by with SMTP id j26so6101iog.13 for <>; Wed, 17 Mar 2021 13:34:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zQ8k1edYc5yLzQgYuZOpjf+pBX0TV65pOnxlnjBj1hg=; b=t9l85pF51J6uaB2aA6LC9mlcl6uO93F3CAsoPhvDjGVkhG0ZfBvec2onvyvH2+d3Bo N6emi+EWGWT3o3s6EXiWFMbPbYrZqiHomVG3Fys43X79RpyyCZHqevJjCtbiLOx+1WqQ gZCdChGSxP3YmNlfqMalsk5pAebjtCXPE2+8FOUAUpIL1EwBiHOxlufJTJgB5zYODePK 0RxDGEZxZuVaGOylY1lKLGjSt/EURMsnYRhYouNJ5RTcgro/nLmG+ZLxafqsKzg7j+fd 6huf51QdMy0yh0evdTllg5dbzkGi7IDLnk2obXSceoQvVLC81TSFvpNwMwDFDJn6lPYY UlAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zQ8k1edYc5yLzQgYuZOpjf+pBX0TV65pOnxlnjBj1hg=; b=JpXXufYOedVj88NrfTYfBxAn3FOjmYJnHUhsJ3iREqNfHKboxBea7Gccb0JMT3F8ap HyRH30KqpgUNtwlBYbdpRs1GcGZAAfiZlfEa0UK3M/GIWcvngCj01QTwGqzXCY9cXyKP 7A9fHdPSp0zlw9IteX3UybTmQ/ojMGt3lj1/NIVLXSAEqr+n2gHgp2vH9Xze5TLFrbEb vJO9CPDI3lwSH8OsySOiQmxB4oRJX9pTqtGXQWBQRHliVJH8pV1p8KBr4fpDmNObYEeG Io0QCclFQoYbVPD/P0yHXkq/ABZr0EW7k8JlK9rKYZFPjPCHkvF+KySZ3BRBe36D6CJn CxWg==
X-Gm-Message-State: AOAM532lTZ3Mj6BTFtHIn5eRf13V4egUudU3zggZp4Rw3hIk5vmS63Uv eM5Qu0y7bXNKeXO4w6L9CVe98O7twRHfRYZ/5us=
X-Google-Smtp-Source: ABdhPJypHncV5m5N2xgpNJpjRlayDhNziC1TMAJ0r27Fp/cXZygBkYzFi1/73ab7n8K0tSV1FJcJKK2l3ALubg2KH1I=
X-Received: by 2002:a6b:ec14:: with SMTP id c20mr8303039ioh.122.1616013261129; Wed, 17 Mar 2021 13:34:21 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <>
In-Reply-To: <>
From: Tony Przygienda <>
Date: Wed, 17 Mar 2021 21:33:44 +0100
Message-ID: <>
To: Huaimo Chen <>
Cc: "Jeffrey (Zhaohui) Zhang" <>, "" <>
Content-Type: multipart/alternative; boundary="00000000000061cd4b05bdc168fc"
Archived-At: <>
Subject: Re: [Bier] Comments on draft-chen-bier-frr-02.
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "\"Bit Indexed Explicit Replication discussion list\"" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 17 Mar 2021 20:34:26 -0000

agreed on both counts

-- tony

On Wed, Mar 17, 2021 at 9:28 PM Huaimo Chen <>

> Hi Tony,
>     Thanks much for your comments.
>     My responses are inline below with [HC].
> Best Regards,
> Huaimo
> ------------------------------
> *From:* Tony Przygienda <>
> *Sent:* Wednesday, March 17, 2021 7:36 AM
> *To:* Jeffrey (Zhaohui) Zhang <>
> *Cc:* Huaimo Chen <>om>; <
> *Subject:* Re: [Bier] Comments on draft-chen-bier-frr-02.
> as participant
> On Tue, Mar 16, 2021 at 10:41 PM Jeffrey (Zhaohui) Zhang <zzhang=
>> wrote:
> Hi Huaimo,
> Please see my comments below.
> Some are nits, but if I understand it correctly, the idea of using
> multiple per-nbr FRR BIFTs is flawed.
>    [I-D.merling-bier-frr] proposes a tunnel-based fast re-route (FRR)
>    ... Before the
>    primary bit mask is recomputed and updated, some of BIER packets may
>    be forwarded incorrectly.
> Would like to see elaborations on why some of the packets may be
> forwarded incorrectly - especially if you consider the approach
> at the end of this mail.
> it seems to me just a bit of a wild assertion. IGPs are the fastest way to
> distribute topology info unless every node is somehow hooked up to a
> "controller" which is basically a degenerate IGP graph (super edge
> connected AFAIR).
> so one could construct a case where every node before running IGP talks to
> controller and that can compute & install backups first while IGP is
> converging. I doubt for the moment practicality of something like this but
> there is little account for taste in networking ;-)
> [HC]: This may earn some time.
> So I think a framework should explain that a unicast nexthop when running
> IGP has already protection (all LFA variants) with BIER for free if
> implemented correctly and given maturity of that technology it's an easy
> choice. If/when that cannot be used (IGP is not in place) etc then the
> framework document should explain protection schemes that could be built
> directly into BIFT (and I agree as informational for the moment and we see
> whether it goes from there). The same document IMO should also cover TE-FRR
> which is a bit of a more interesting case.
> [HC]: If the framework is a separate document, it should cover BIER FRR
> and BIER-TE FRR.
> If the framework is added into BIER FRR draft, it seems better to just
> cover BIER FRR.
> All the BIER packets are forwarded using the BIFTs and the forwarding
> procedure specific for BIER.
> All the IP packets are forwarded using the normal FIBs and the forwarding
> procedure specific for IP.
> When/if IP FRR is provided, it is through the FIB with LFA and the
> forwarding procedure for IP.
> When there is a failure, IP packets are protected by IP FRR through the
> FIB with LFA and
> the forwarding procedure for IP.
> All the BIER packets are still forwarded using the BIFTs, which do not
> have backup next hops.
> They are not protected since they are not forwarded using the FIB with LFA.
> We can use the LFA backup next hop in the FIB for IP for free in the
> corresponding forwarding
> entry in the BIFT for BIER if IP FRR is supported in the network.
> If/when IP FRR is not provided in the network, the LFA backup next hops
> are computed
> for the BIFTs.
> After the BIFTs have the LFA backup next hops and the forwarding procedure
> for BIER
> is enhanced for BIER FRR, all the BIER packets are protected.
> BTW, the Menthe paper seems bit misleading in this respect, it somehow
> doesn't show that the NH unicast is easily protected via IGP FRR @ no
> additional storage or complexity cost. The section VI on LFA based FRR is
> literally this AFAIS, modulo squeezing the protection into BIFT NH (which
> is with IGP not a _real_ nexthop but indirection to IGP neighbor if
> implemented smartly). yes, all the discussion on compressed backup etc is
> fine if there's no IGP but to try to replicate what IGPs do today already
> does not seem like any saving (unless one costrues again a convoluted case
> of underlying IGP without LFA available/enabled).
> <>
> itself seems very implementation specific (and subtly flawed, I think
> Jeffrey picked @ it carefully in rest of this email) but describes largely
> same concept AFAIS so it looks to me those drafts could be esaily joined
> into a common bier-frr draft convering all the approaches and bier te frr
> as well.
> [HC]: It is OK for me to combine those drafts into one covering all the
> BIER FRR approaches.
> For BIER-TE FRR, it seems different from BIER FRR. It would be better to
> have a separate one.
>    This document describes a mechanism for fast re-route (FRR)
>    protection against the failure of a node or link in the core of a
>    BIER domain, which resolves the above issue.  It is based on LFA,
>    which is called LFA-based BIER-FRR.
> Unicast FRR can be based on many mechanisms, and BIER FRR can simply
> follow suit. It does not have to be limited to LFA or tunnel - simple
> ECMP can also work if there are ECMP paths (if an ECMP branch is no
> longer available you can simply take another one).
> +1
> IGPs provide very reach X-LFA support today already (well, in RFCs and
> solid implementaions)
> [HC]: The draft is about LFA-based BIER FRR. Any X-LFA supported today
> can be included.
> -- tony
>    In normal operations, the normal BIFT is used to forward BIER
>    packets.  When a neighbor fails, the BFR as PLR uses the FRR BIFT for
>    the neighbor to forward BIER packets.  For a BIER packet to traverse
>    the BFR and the failed neighbor, the BFR reroutes the packet around
>    the failed neighbor using the FRR BIFT for the neighbor.  For a BIER
>    packet to traverse the BFR and any other neighbors, the BFR forwards
>    the packet to its expected next hop neighbors using the forwarding
>    entries with these BFR neighbors in the FRR BIFT.
> I do not see why there is a need for per-neighbor FRR BIFT. It is also not
> clear how the switch between normal BIFT and FRR BIFT is done - in fact
> I think it is flawed - more on that later.
>    From the BIRT on the BFR, a "Bit Index Forwarding Table" (BIFT) is
>    derived.  In addition to having a route to a BFER in each row of the
>    BIFT which is the same as the BIRT, it has a "Forwarding Bit Mask"
>    (F-BM) in its each row.  For the rows in the BIRT that have the same
>    SI and the same BFR-NBR, the F-BM for each of these rows in the BIFT
>    is the logical OR of the BitStrings of these rows.
> Need to be a bit more strict here. A row in BIFT is about a
> particular BFER, which could be reached via two ECMP nbrs.
> The F-BM is really for a nbr, not for a row. See Figure 6 in RFC8279.
>    When a BFR receives a packet, for each BFER k (from the rightmost to
>    the leftmost) represented in the SI and BitString of the packet, if
>    BFER k is the BFR itself, the BFR copies the packet, sends the copy
>    to the multicast flow overlay and clears bit k in the original
>    packet; otherwise the BFR finds the row (i.e., forwarding entry) in
>    the BIFT for the sub-domain using the SI and BitString as the key or
>    say index, and then copies, updates and forwards the packet to the
>    BFR-NBR (i.e., the next hop) indicated by the row (i.e., forwarding
>    entry).
> Strictly speaking, the BIFT is for <sub-domain, SI> (if we ignore
> different bitstringLen), and the lookup key is 'k' not the entire
> bitstring.
>    The FRR-BIRT for BFR-NBR X of the BFR considers the failure of X and
>    maps the BFR-id (in the sub-domain) of a BFER to the BFR-prefix of
> A nit here - strictly speaking it's not that you map the bfr-id to bier
> prefix in BIRT. BIRT is built based on how a BIER prefix is reached,
> and the mapping is the other way around - you map the prefix to a BFR-id
> when you derive the BIFTs from BIRT.
>    The BFR may build the FRR-BIRT for BFR-NBR X by copying its BIRT to
>    the FRR-BIRT first, and then change the next hop with value BFR-NBR X
>    in the FRR-BIRT to a backup next hop (BNH) to protect against the
>    failure of X.  In other wards, for the BFR-id of a BFER in the FRR-
>    BIRT for BFR-NBR X, if the next hop BFR-NBR on the path to the BFER
>    is X, it is changed to a BNH when there is a BNH on a backup path to
>    the BFER without going through X and the link from the BFR to X.
>    If there is not any BNH to a BFER to protect against the failure of
>    X, the next hop BFR-NBR X to the BFER in the FRR-BIRT for BFR-NBR X
>    is changed to NULL.  For a multicast packet having the BFER as one of
>    its destinations, if the next hop BFR-NBR to the BFER is NULL, the
>    BFR does not send the packet to the next hop BFR-NBR NULL but drops
>    it when X fails.
> A nit - it's not that the packdet is dropped. Rather, the bit for that
> BFER is simply cleared.
> So for BFERs not affected by X's failure, they're still present in
> X's FRR BIFT and are the same as in the normal BIFT. Keeping these
> per-nbr FRR BIFTs with those unaffected entries
> just does not make sense - you might as well have one BIFT in which
> each row includes ECMP/backup branches. Of course, this is implementation
> details/preferences, but:
> - it's not good to specify this controversial implementation details
> - switching to using FRR BIFT is actually flawed - see later comments.
>    The forwarding procedure defined in [RFC8279] is updated/enhanced for
>    a FRR-BIFT to consider the case where the next hop BFR-NBR to a BFER
>    is NULL.  For a multicast packet with the BitString indicating a BFER
>    as one of its destinations, the updated forwarding procedure checks
>    whether the next hop BFR-NBR to the BFER in the FRR-BIFT is NULL.  If
>    it is NULL, the procedure will not send the packet to this next hop
>    BFR-NBR NULL but drop the packet.
> A nit - not "drop the packet" but simply clear the bit.
> Additionally, even when you do not support FRR you also have situations
> whwere some BFERs are simply not reachable. In other words, clearing
> the bit for those BFERs is the base requirement/assumption in RFC8279,
> not a update/enhancement.
>    The FRR-BIFTs will be pre-computed and installed ready for activation
>    when a failure is detected.  Once the BFR detects the failure of its
>    BFR-NBR X, it activates the FRR-BIFT for X to forward packets with
>    BIER headers and de-activates its BIFT.  After activation of the FRR-
>    BIFT, it remains in effect until it is no longer required.
> Let's say this BFR has several BFR neighbors (including X and W).
> X fails first and W fails instantly after that.
> Which FRR BIFT do you use? X's FRR BIFT does not consider W's failure
> and W's FRR BIFT does not consider X's failure, so using either one alone
> has problems. Notice that this is different from the typical
> "multiple failure" scenario - it's not that X and W may depend on each
> other but it could be that some BFERs have a common backup neighbor Y, yet
> if you use X's FRR BIFT then those BFERs normally reachable via W
> are not protected now that W also fails. Similarly, if you use W's
> FRR BIFT then those BFERs normally reachable via X are not protected
> now that X also fails.
> That's why a single BIFT should be used for both normal and FRR forwarding,
> just like in unicast case.
>    The number of entries in a FRR BIFT is the number of BFERs.  Each FRR
>    BIFT on a BFR can be compressed through combining all the entries
>    with the same BFR-BNR and F-BM into one entry.  The number of entries
>    in a compressed FRR BIFT is the number of neighbors of the BFR minus
>    one.
>    For example, the compressed FRR-BIFT for BFR C on BFR B is shown in
>    Figure 7.  The number of entries in it is three, which equals the
>    number (four) of neighbors of BFR B minus one.
>                  +----------------+---------+------------+
>                  |     BFR-id     |  F-BM   |  BFR-NBR   |
>                  | (SI:BitString) |         | (Next Hop) |
>                  +================+=========+============+
>                  | 1, 4 (0:01001) |  01001  |     G      |
>                  +----------------+---------+------------+
>                  | 2, 3 (0:00110) |  00110  |     E      |
>                  +----------------+---------+------------+
>                  |  5 (0:10000)   |  10000  |     A      |
>                  +----------------+---------+------------+
>              Figure 7: Compressed FRR BIFT for BFR C on BFR B
>    For a BIER packet with a BFR-ID as a destination, the entry
>    containing the BFR-ID is used to forward the packet.
> Not sure of the benefit of compression here. It makes the lookup more
> complicated. If the memory consumption is a concern, just get rid of
> per-nbr FRR-BIT and use a single one. Additionally, the F-BM and BFR-NBR
> information can be considered forwarding information that can be pointed
> at by multiple BIFT entries, so the above compression does not really
> give you much savings.
>    Once BFR B detects the failure of its BFR-NBR X, after receiving the
>    packet from BFR A, BFR B copies, updates and sends the packet using
>    the FRR-BIFT for X on BFR B to avoid X and link from B to X according
>    to the forwarding procedure defined in [RFC8279].
>    For example, once BFR B detects the failure of its BFR-NBR C, after
>    receiving the packet from BFR A, BFR B copies, updates and sends the
>    packet to BFR G and BFR E using the FRR-BIFT for BFR C on BFR B to
>    avoid C and link from B to C.
> See earlier comments about failure of multiple neighbors (who do not
> use each other for backup). Using a per-nbr FRR BIFT not only is
> unnecessary, but also has problems.
> A better way is to simply have a single BIFT for both normal and FRR
> forwarding. Each entry has some ECMP and/or primary/backup branches,
> and you simply use another ECMP branch or a backup branch when the
> normally used branch fails. A backup branch can go through completely
> different nbr, or go through the same neighbor but via a tunnel.
> Jeffrey
> Juniper Business Use Only
> _______________________________________________
> BIER mailing list
> <>