Re: [Bier] Comments on draft-chen-bier-frr-02.

Tony Przygienda <> Wed, 17 March 2021 11:36 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 2DE513A136E for <>; Wed, 17 Mar 2021 04:36:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id jrSZLPyj665Z for <>; Wed, 17 Mar 2021 04:36:43 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 115F13A136C for <>; Wed, 17 Mar 2021 04:36:42 -0700 (PDT)
Received: by with SMTP id r8so1128753ilo.8 for <>; Wed, 17 Mar 2021 04:36:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wcXSGUHCbNb5/ISYkldqRH4tDKkQJ2uF/v8cbax/c2U=; b=m0iWOsr708uiH83Lq5MAMWOyrg0jPA5ELMlg+LzBA16wJEagC3I7tcIxxivmZ3AjoI xz44X89geDzCkrMPLcSRLIhSpTNdaeLSxutaEsUWpFnMkqyQrQqzxbIoek8g68f4YwpG Wn9HyLLUOsMw++6vSwHgLD5XwjknRHQciRKhoHoOnDtAhD7SSJ1Ts+zu+KP7WgG+pTmW B1tI2b2FgOvd03Kb/I6fIP2s0zxzrKxbdQjyBtAQxqciEI7Lif7+UAnw7irxJq+WBKdp 5X/P2SKE6Du4MN3ypgV4i58N/MO0gvZFp/KccCIH9xImihbkYCH5cwPnJ67/K0Xi8ZL4 ifPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wcXSGUHCbNb5/ISYkldqRH4tDKkQJ2uF/v8cbax/c2U=; b=g2FNcTxoR1Zv3bR19Vi01JzJFp0Hy12vWceCLuVZfIZuUtWRhZNz+eN5zE6pxfCoWv 84bHOwUfsx+8HS6SZle2JN9HSyYEyERZbzRgUrRLGBJxfjhbAOZG3bMuZlkAe4YekF2b PLE/JEk3z+PdGRYd0bafYSH+Dn/mZju8SlEZeai8WY+A1ah0876B3fv2A/X8zATm8qgR EwhhEXyOjUlEfGKndpONf4Vc/bBDR19LThwG2ykePjsGkP4KQqPiHSMwUnAMmozB38Op SBsBveQxTu0f4UXEChsNfWHFNx8oNEJJteGYX3OGxIliTbUSEZvUkYQ9zXKIB0BEeOLk 5ejg==
X-Gm-Message-State: AOAM530TygVlmpENzB6Cs/4+sUESOkN2aIciao4OCKJv6B01MNwXd4n0 ykUfRSDDq+zUdbDlYe1xApIVfDKUjkABsG9edlc=
X-Google-Smtp-Source: ABdhPJwEB5WfAYv8LIPn/NhzFd5Igz9MXJmvGnJhv+c5FnEidCf3Xfi+04P/KJJXwkXSoJKRtlTk9+hwd0N51L3RQ38=
X-Received: by 2002:a92:ce03:: with SMTP id b3mr7266916ilo.302.1615981001898; Wed, 17 Mar 2021 04:36:41 -0700 (PDT)
MIME-Version: 1.0
References: <>
In-Reply-To: <>
From: Tony Przygienda <>
Date: Wed, 17 Mar 2021 12:36:06 +0100
Message-ID: <>
To: "Jeffrey (Zhaohui) Zhang" <>
Cc: Huaimo Chen <>, "" <>
Content-Type: multipart/alternative; boundary="00000000000094fef605bdb9e5ca"
Archived-At: <>
Subject: Re: [Bier] Comments on draft-chen-bier-frr-02.
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "\"Bit Indexed Explicit Replication discussion list\"" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 17 Mar 2021 11:36:46 -0000

as participant

On Tue, Mar 16, 2021 at 10:41 PM Jeffrey (Zhaohui) Zhang <zzhang=> wrote:

> Hi Huaimo,
> Please see my comments below.
> Some are nits, but if I understand it correctly, the idea of using
> multiple per-nbr FRR BIFTs is flawed.
>    [I-D.merling-bier-frr] proposes a tunnel-based fast re-route (FRR)
>    ... Before the
>    primary bit mask is recomputed and updated, some of BIER packets may
>    be forwarded incorrectly.
> Would like to see elaborations on why some of the packets may be
> forwarded incorrectly - especially if you consider the approach
> at the end of this mail.

it seems to me just a bit of a wild assertion. IGPs are the fastest way to
distribute topology info unless every node is somehow hooked up to a
"controller" which is basically a degenerate IGP graph (super edge
connected AFAIR).

so one could construct a case where every node before running IGP talks to
controller and that can compute & install backups first while IGP is
converging. I doubt for the moment practicality of something like this but
there is little account for taste in networking ;-)

So I think a framework should explain that a unicast nexthop when running
IGP has already protection (all LFA variants) with BIER for free if
implemented correctly and given maturity of that technology it's an easy
choice. If/when that cannot be used (IGP is not in place) etc then the
framework document should explain protection schemes that could be built
directly into BIFT (and I agree as informational for the moment and we see
whether it goes from there). The same document IMO should also cover TE-FRR
which is a bit of a more interesting case.

BTW, the Menthe paper seems bit misleading in this respect, it somehow
doesn't show that the NH unicast is easily protected via IGP FRR @ no
additional storage or complexity cost. The section VI on LFA based FRR is
literally this AFAIS, modulo squeezing the protection into BIFT NH (which
is with IGP not a _real_ nexthop but indirection to IGP neighbor if
implemented smartly). yes, all the discussion on compressed backup etc is
fine if there's no IGP but to try to replicate what IGPs do today already
does not seem like any saving (unless one costrues again a convoluted case
of underlying IGP without LFA available/enabled). itself seems very
implementation specific (and subtly flawed, I think Jeffrey picked @ it
carefully in rest of this email) but describes largely same concept AFAIS
so it looks to me those drafts could be esaily joined into a common
bier-frr draft convering all the approaches and bier te frr as well.

>    This document describes a mechanism for fast re-route (FRR)
>    protection against the failure of a node or link in the core of a
>    BIER domain, which resolves the above issue.  It is based on LFA,
>    which is called LFA-based BIER-FRR.
> Unicast FRR can be based on many mechanisms, and BIER FRR can simply
> follow suit. It does not have to be limited to LFA or tunnel - simple
> ECMP can also work if there are ECMP paths (if an ECMP branch is no
> longer available you can simply take another one).


IGPs provide very reach X-LFA support today already (well, in RFCs and
solid implementaions)

-- tony

>    In normal operations, the normal BIFT is used to forward BIER
>    packets.  When a neighbor fails, the BFR as PLR uses the FRR BIFT for
>    the neighbor to forward BIER packets.  For a BIER packet to traverse
>    the BFR and the failed neighbor, the BFR reroutes the packet around
>    the failed neighbor using the FRR BIFT for the neighbor.  For a BIER
>    packet to traverse the BFR and any other neighbors, the BFR forwards
>    the packet to its expected next hop neighbors using the forwarding
>    entries with these BFR neighbors in the FRR BIFT.
> I do not see why there is a need for per-neighbor FRR BIFT. It is also not
> clear how the switch between normal BIFT and FRR BIFT is done - in fact
> I think it is flawed - more on that later.

>    From the BIRT on the BFR, a "Bit Index Forwarding Table" (BIFT) is
>    derived.  In addition to having a route to a BFER in each row of the
>    BIFT which is the same as the BIRT, it has a "Forwarding Bit Mask"
>    (F-BM) in its each row.  For the rows in the BIRT that have the same
>    SI and the same BFR-NBR, the F-BM for each of these rows in the BIFT
>    is the logical OR of the BitStrings of these rows.
> Need to be a bit more strict here. A row in BIFT is about a
> particular BFER, which could be reached via two ECMP nbrs.
> The F-BM is really for a nbr, not for a row. See Figure 6 in RFC8279.
>    When a BFR receives a packet, for each BFER k (from the rightmost to
>    the leftmost) represented in the SI and BitString of the packet, if
>    BFER k is the BFR itself, the BFR copies the packet, sends the copy
>    to the multicast flow overlay and clears bit k in the original
>    packet; otherwise the BFR finds the row (i.e., forwarding entry) in
>    the BIFT for the sub-domain using the SI and BitString as the key or
>    say index, and then copies, updates and forwards the packet to the
>    BFR-NBR (i.e., the next hop) indicated by the row (i.e., forwarding
>    entry).
> Strictly speaking, the BIFT is for <sub-domain, SI> (if we ignore
> different bitstringLen), and the lookup key is 'k' not the entire
> bitstring.
>    The FRR-BIRT for BFR-NBR X of the BFR considers the failure of X and
>    maps the BFR-id (in the sub-domain) of a BFER to the BFR-prefix of
> A nit here - strictly speaking it's not that you map the bfr-id to bier
> prefix in BIRT. BIRT is built based on how a BIER prefix is reached,
> and the mapping is the other way around - you map the prefix to a BFR-id
> when you derive the BIFTs from BIRT.
>    The BFR may build the FRR-BIRT for BFR-NBR X by copying its BIRT to
>    the FRR-BIRT first, and then change the next hop with value BFR-NBR X
>    in the FRR-BIRT to a backup next hop (BNH) to protect against the
>    failure of X.  In other wards, for the BFR-id of a BFER in the FRR-
>    BIRT for BFR-NBR X, if the next hop BFR-NBR on the path to the BFER
>    is X, it is changed to a BNH when there is a BNH on a backup path to
>    the BFER without going through X and the link from the BFR to X.
>    If there is not any BNH to a BFER to protect against the failure of
>    X, the next hop BFR-NBR X to the BFER in the FRR-BIRT for BFR-NBR X
>    is changed to NULL.  For a multicast packet having the BFER as one of
>    its destinations, if the next hop BFR-NBR to the BFER is NULL, the
>    BFR does not send the packet to the next hop BFR-NBR NULL but drops
>    it when X fails.
> A nit - it's not that the packdet is dropped. Rather, the bit for that
> BFER is simply cleared.
> So for BFERs not affected by X's failure, they're still present in
> X's FRR BIFT and are the same as in the normal BIFT. Keeping these
> per-nbr FRR BIFTs with those unaffected entries
> just does not make sense - you might as well have one BIFT in which
> each row includes ECMP/backup branches. Of course, this is implementation
> details/preferences, but:
> - it's not good to specify this controversial implementation details
> - switching to using FRR BIFT is actually flawed - see later comments.
>    The forwarding procedure defined in [RFC8279] is updated/enhanced for
>    a FRR-BIFT to consider the case where the next hop BFR-NBR to a BFER
>    is NULL.  For a multicast packet with the BitString indicating a BFER
>    as one of its destinations, the updated forwarding procedure checks
>    whether the next hop BFR-NBR to the BFER in the FRR-BIFT is NULL.  If
>    it is NULL, the procedure will not send the packet to this next hop
>    BFR-NBR NULL but drop the packet.
> A nit - not "drop the packet" but simply clear the bit.
> Additionally, even when you do not support FRR you also have situations
> whwere some BFERs are simply not reachable. In other words, clearing
> the bit for those BFERs is the base requirement/assumption in RFC8279,
> not a update/enhancement.
>    The FRR-BIFTs will be pre-computed and installed ready for activation
>    when a failure is detected.  Once the BFR detects the failure of its
>    BFR-NBR X, it activates the FRR-BIFT for X to forward packets with
>    BIER headers and de-activates its BIFT.  After activation of the FRR-
>    BIFT, it remains in effect until it is no longer required.
> Let's say this BFR has several BFR neighbors (including X and W).
> X fails first and W fails instantly after that.
> Which FRR BIFT do you use? X's FRR BIFT does not consider W's failure
> and W's FRR BIFT does not consider X's failure, so using either one alone
> has problems. Notice that this is different from the typical
> "multiple failure" scenario - it's not that X and W may depend on each
> other but it could be that some BFERs have a common backup neighbor Y, yet
> if you use X's FRR BIFT then those BFERs normally reachable via W
> are not protected now that W also fails. Similarly, if you use W's
> FRR BIFT then those BFERs normally reachable via X are not protected
> now that X also fails.
> That's why a single BIFT should be used for both normal and FRR forwarding,
> just like in unicast case.
>    The number of entries in a FRR BIFT is the number of BFERs.  Each FRR
>    BIFT on a BFR can be compressed through combining all the entries
>    with the same BFR-BNR and F-BM into one entry.  The number of entries
>    in a compressed FRR BIFT is the number of neighbors of the BFR minus
>    one.
>    For example, the compressed FRR-BIFT for BFR C on BFR B is shown in
>    Figure 7.  The number of entries in it is three, which equals the
>    number (four) of neighbors of BFR B minus one.
>                  +----------------+---------+------------+
>                  |     BFR-id     |  F-BM   |  BFR-NBR   |
>                  | (SI:BitString) |         | (Next Hop) |
>                  +================+=========+============+
>                  | 1, 4 (0:01001) |  01001  |     G      |
>                  +----------------+---------+------------+
>                  | 2, 3 (0:00110) |  00110  |     E      |
>                  +----------------+---------+------------+
>                  |  5 (0:10000)   |  10000  |     A      |
>                  +----------------+---------+------------+
>              Figure 7: Compressed FRR BIFT for BFR C on BFR B
>    For a BIER packet with a BFR-ID as a destination, the entry
>    containing the BFR-ID is used to forward the packet.
> Not sure of the benefit of compression here. It makes the lookup more
> complicated. If the memory consumption is a concern, just get rid of
> per-nbr FRR-BIT and use a single one. Additionally, the F-BM and BFR-NBR
> information can be considered forwarding information that can be pointed
> at by multiple BIFT entries, so the above compression does not really
> give you much savings.
>    Once BFR B detects the failure of its BFR-NBR X, after receiving the
>    packet from BFR A, BFR B copies, updates and sends the packet using
>    the FRR-BIFT for X on BFR B to avoid X and link from B to X according
>    to the forwarding procedure defined in [RFC8279].
>    For example, once BFR B detects the failure of its BFR-NBR C, after
>    receiving the packet from BFR A, BFR B copies, updates and sends the
>    packet to BFR G and BFR E using the FRR-BIFT for BFR C on BFR B to
>    avoid C and link from B to C.
> See earlier comments about failure of multiple neighbors (who do not
> use each other for backup). Using a per-nbr FRR BIFT not only is
> unnecessary, but also has problems.
> A better way is to simply have a single BIFT for both normal and FRR
> forwarding. Each entry has some ECMP and/or primary/backup branches,
> and you simply use another ECMP branch or a backup branch when the
> normally used branch fails. A backup branch can go through completely
> different nbr, or go through the same neighbor but via a tunnel.
> Jeffrey
> Juniper Business Use Only
> _______________________________________________
> BIER mailing list