Re: [Bier] draft-ietf-bier-ipv6-requirements-09

Tony Przygienda <> Thu, 26 November 2020 17:38 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 091003A155C for <>; Thu, 26 Nov 2020 09:38:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 41AWXn_WGosM for <>; Thu, 26 Nov 2020 09:38:15 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:4864:20::d41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 772333A155B for <>; Thu, 26 Nov 2020 09:38:15 -0800 (PST)
Received: by with SMTP id d17so2458260ion.4 for <>; Thu, 26 Nov 2020 09:38:15 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=iri90UJBAL+21GSaqTEMGFIAQcje2SmqiUzJUSYIW04=; b=cPiTHqCq1IjPbSMWkxgw/zGtiqYHgPn5oKo5fGLsG0blpJTKeloTE0r7wrEefBxVCA +WVI2/T2eX1Jc1T9WXJUXcSRW3gZ+XMvxWyCrCP9oFmQKRZyIaZUhJ3G0Gnwl+fME8DI hqasrf9B1lpB2lCcrcZsd3kj1xrMI1zJhExW95qObdaj3ZOP1CHenAhJAZVN5cwAMMgm 9biQC+F7gnXFbFH0URWA4XHDJU5ktRhXIjFWeKxvy+Pek9V9hWdCLbDdvVQKVUAxz3fV +Z7Mclksk8SjpE2q2nGuqpZa3iUklsaKvwWmKeoUU506ZUTQpdndM9bidUgdKQyph3wq gk4g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iri90UJBAL+21GSaqTEMGFIAQcje2SmqiUzJUSYIW04=; b=ksjsV5FTQcf8RUiyGnLZLr9Jn/ECV0gC4htmg0mi1KiqSlT4j/sitC86NellrRsk6C NwhbioJQzLOy5/VIsOpwufAyhm4yLnTKAymrN0PVeVedR9DqX+x9VOkbHY2r2e8o09/G ALOSvFEaz2cK2RRAKd5eJLvBj0zDrgmC2yiC1ObTySxCTmGVB5IdjgW/rysr7yTRvqcV kopPdOyLGbQ/MsbUQf5fesdozH3SCWKY4agNXDkbtoJu28w+vjSSRtyLEZB5TpN5fLDl aABMqZwhdaFAhIJMc8arYFnSOKcPZF07K2tGUWPp75NCcVPXzBWbSS1wVm3jG4QpamFK OkDQ==
X-Gm-Message-State: AOAM532JhrshA5Yk6hbVrSAqOkQStkMrKuHgl9LgpbuQ3zyo9cz0kaJ+ nmPkGLzb5pr1sOY0NsCXy6eFAt+Yd/MyoGPvopjnG4E+cvQ=
X-Google-Smtp-Source: ABdhPJzp6UfflUSxRCTIkF3yVSCxWvoQ0OPFrqEYlnBPxXLFdxoUB7/ZTOeGN9J6SEZbk7aM5sZm2N7/UuR8xZ9oiOU=
X-Received: by 2002:a02:2421:: with SMTP id f33mr3891420jaa.113.1606412294684; Thu, 26 Nov 2020 09:38:14 -0800 (PST)
MIME-Version: 1.0
References: <> <> <> <> <> <> <02fd01d6c3f5$a8f23de0$fad6b9a0$> <> <033a01d6c410$92e413f0$b8ac3bd0$>
In-Reply-To: <033a01d6c410$92e413f0$b8ac3bd0$>
From: Tony Przygienda <>
Date: Thu, 26 Nov 2020 09:37:38 -0800
Message-ID: <>
To: Adrian Farrel <>
Cc: "Jeffrey (Zhaohui) Zhang" <>, Greg Shepherd <>, BIER WG <>
Content-Type: multipart/alternative; boundary="00000000000030057505b5060267"
Archived-At: <>
Subject: Re: [Bier] draft-ietf-bier-ipv6-requirements-09
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "\"Bit Indexed Explicit Replication discussion list\"" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 26 Nov 2020 17:38:19 -0000

Quickly chiming in only the address the possible budding confusion on ECMP

1. Adrian, yes, this is hourglass _L2.5_ and hence we have to consider lots
of things which can be hand-waived around in other groups. Things like OAM,
silicon speed/size/offsets/pipeline depth, ECMP that you're bringing up,
_transitory_ solutions to support jumping non-BIER routers until we have an
all native silicon everywhere are all important considerations.
2. ECMP in this case (i.e. encapsulation into v6 header) is NOT control
plane, it's a pure dataplane issue. In case of v6 _encapsulation_ we simply
need to support a _transitory_ encapsulation solution as Greg S. stated
repeatedly (and as footnote: and possibly very cheap silicon like WiFi
where it is desirable to allow any old v6 silicon to pick up a packet that
is BIER and throw it to middle path or slow path to support BIER e.g. as
homenet multicast solution. Speaking of which, maybe HomeNet should go into
requirements document, some noises were made years ago but basically
Homenet answer was, "well, we don't have multicast solution, PIM is
overkill, we probably should work on something". Newest state there escapes
me right now). Given BIER is L2.5 we basically encaps hop-by-hop and the
only ECMP issue here are possibly parallel links where BIER does NOT really
care about the outcome.  So whether we use Flow-ID field or Flow-ID + 5
tuple is really not that interesting but since in case of BIERin6 there is
no 5-tuple (it's just a next-header-protocol in v6 like UDP or whats-not)
that could use Flow-ID copied from BIER entropy for consistency reason and
give a good chance for different flows to use different _parallel_ one hop
links. Without silicon knowing anything about BIER or having to reach into
the packet @ deep offset to find entropy.
3. ECMP in BIER as control-plane is an issue which is irrelevant to
encapsulation since it only exists when looking @ end2end BIER forwarding
to a BFER. This is basically the whole discussion how to compute BIFTs with
the trade-offs going back to yours/Eric observations and my picture of the
fishtail when BIER architecture was discussed around 2014/15 or so. This
does not care about link specific encapsulation at all (which
MPLS/Ether/non-ether/BIERin6 encapsulations are).
4. The ECMP-v6 confusion may be related to the fact that given stuff is in
a v6 packet the temptation to throw packet multiple hops (violating the
specs even if we write them "you shall never do that" ;-) will be
irresistible. And yes, it's a viable _transitory_ solution that
architecture outlines to "jump over non-BIER routers via unicast tunnels".
But ECMP of this unicast tunnels is in a sense that's NOT BIER ECMP since
it will not process the mask so IPv6 can do what it wants in a sense.
Obviously OAM is lost when doing such kind of stuff and it is to observe
that architecture supports _transition_ shortcuts so any other technology
could be used like GRE, v6, IP, SRv6 tunnel here is just a random choice.
Now, however, and that confuses lots of people AFAIS, the perception by
some people rather new to BIER is that v6-encapsulation-work becomes all of
a sudden a charter to write a new multicast-L3 tunneling technology and
first, as Greg & others pointed out many times and they have huge amount of
years in multicast amongst them, that was has proven operationally quite
challenging and ultimately not very successul repeatedly (for manifold
reasons I don't want to spend lots time here) otherwise BIER would not have
happened AFAI.  And second if L3 multicast tunnel is really desired, mLDP,
PIM and other things exist since L3 multicast tunnel needs signalling, leaf
joins, dealing with ethernet multicast, state to pin the tunnels possibly,
specialized OAM and procedures etc, etc so if anything it seems to me like
a BoF, new WG matter calling it
but that's just random musings given BIER today is already a perfect mp2mp
PSMI that plugs in one-for-one-replacement into MVPN/EVPN with procedures
worked out in WG and about to go RFC. All that given that Ice was one of
the brainfathers/implementors of mLDP & hierarchical mLDP and ultimately,
after years of pain, was I think _the_ father of the basic BIER idea to
avoid all the problems that such a solution causes. Seeing BIER as "better
mLDP" is however also a wrorng way to think about it, BIER is an hourglass
technology independent of L3 as much as MPLS is.

PS: I personally still wait for the BIER(SR)v6 authors to address the
fundamental justification questions that Greg formulated in the working
group and were clarified on the mail independently of all those side
threads coming up now (most of them touching good issues though like MVPN
efficiency etc) that help to clarify the requirements.
PPS: I BTW do _strongly_ disapprove of some of the completely off-topic and
discourteous utterances by fairly fresh BIERv6 proponents on some of those
threads but I leave it to my co-chair and AD to deal with that.


-- tony

On Thu, Nov 26, 2020 at 8:24 AM Adrian Farrel <> wrote:

> Hi Jeffery,
> Happy Thanksgiving!
> Are you saying that all IPv6 routers use flow label and other primary
> header for entropy and none of them looks into the payload? (I’m asking cos
> I don’t know what v6 routers do.)
> RFC 6437 implies that routers should use the “traditional” 5-tuple in
> addition to the flow label. But maybe routers don’t follow 6437?
> BTW, I not trying to debate the solutions. I’m trying to find out what
> behavior is required in the IPv6 tunnel between BFRs. It appears that ECMP
> is required (good). Are we asking for any “special” ECMP behavior, or do we
> assume that the tunnel transit nodes are blind legacy nodes that cannot
> tell a BIER packet from any other packet?
> Cheers,
> Adrian
> *From:* Jeffrey (Zhaohui) Zhang <>
> *Sent:* 26 November 2020 15:25
> *To:*; 'Tony Przygienda' <>om>; 'Greg
> Shepherd' <>
> *Cc:* 'BIER WG' <>
> *Subject:* RE: [Bier] draft-ietf-bier-ipv6-requirements-09
> Hi Adrian,
> Looks like I don’t have a life – making arguments for this life and death
> situation on Thanksgiving day 😊
> Anyway, about the ECMP topic, not sure if the following addresses your
> questions/comments.
> At BIER layer itself, BIER packet has an entry field used for hashing to
> decide which ECMP path to the next BFR it will take. This is used by any
> BFR.
> If BFR1 determines that for to reach BFR2 it is going to use tunnel1 (vs.
> tunnel2 or another L2 link), and tunnel1 is an IPv6 tunnel, then with
> BIERin6 an IPv6 header is put on by BFR1, with the BIER header following
> the IPv6 header, and the BIER header’s entropy field copied into IPv6
> header’s flow label field. That flow label field is then used by the
> routers along the tunnel to do ECMP.
> Jeffrey
> *From:* BIER <> *On Behalf Of *Adrian Farrel
> *Sent:* Thursday, November 26, 2020 8:12 AM
> *To:* 'Tony Przygienda' <>om>; 'Greg Shepherd' <
> *Cc:* 'BIER WG' <>
> *Subject:* Re: [Bier] draft-ietf-bier-ipv6-requirements-09
> *[External Email. Be cautious of content]*
> I’ve been reading up on this thread and the three related drafts.
> I don’t dip into BIER often (I’m not a multicast person, and I have a
> life), but this seemed to be a fairly weighty topic which has been bubbling
> away for a while, and the volume of the discussion suggested that this is a
> really important question (it sounded like a life and death decision
> judging by some of the emails!).
> I think Tony captured some really key points in his email below. I
> particularly like his observation that BIER is working at the neck of the
> hourglass: that demands caution and good judgement; it also requires
> everyone to step back and do the right thing regardless of their investment
> (emotional or financial) in their preferred solution.
> It seems to me (again, from the outside, and apologies if this is
> re-opening age-old discussions) that most of this is just protocol
> engineering. We have long experience at making any protocol do anything we
> want. If a particular solution lacks some capability, it can always be
> added with an extra TLV. That makes comparisons of solutions (also known as
> beauty contests) somewhat pointless: if you judge A better than B because B
> lacks some feature, then we just add the feature to B, and the cycle starts
> again.
> That means that, while the requirements work is highly valuable for
> working out what the solution should deliver, it is not so helpful in
> determining which solution the WG should pursue. We are left, IMHO, with
> some of the edge requirements about transiting non-BIER nodes. These are
> nodes that can happily process “normal” IPv6 packets, but don’t know what
> to do with a BIER encapsulation. That looks like Section 3.1.3 of the
> requirements draft.
> Embedded in that requirement is discussion of what an IPv6 router that is
> a transit might do with a packet. On the whole, routers just route on the
> fields in the v6 header itself, but they may look deeper in order to
> perform ECMP functions etc. For example, they may look for the transport
> payload to hash on ports etc. To achieve this, a router must be able to
> step over any additional headers (RH, DOH, etc.) to find the payload or
> must know not to even try. In general, a router that doesn’t understand a
> header will step over it if it can, but will probably give up the hunt for
> hashable fields.
> At this point I ran aground ☹ 8926 doesn’t have anything to say about
> ECMP in a BIER network (with or without BIER-capable routers). But 8279 has
> a nice fat section on ECMP, but this seems to describe how ECMP works when
> processing the BIER encapsulation for equal cost paths between BIER
> routers, not for how the “underlay” (the IPv6 network in this case) might
> handle equal cost paths in its own routing.
> Any clues as to how ECMP is expected to work in the context of the v6
> requirements? Anything that should be added to 3.1.3 or a new section?
> Thanks,
> Adrian
> *From:* BIER <> *On Behalf Of *Tony Przygienda
> *Sent:* 20 November 2020 05:36
> *To:* Greg Shepherd <>
> *Cc:* BIER WG <>rg>; Gyan Mishra <>om>;
> draft-ietf-bier-ipv6-requirements <
>>gt;; <
>>gt;; Alvaro Retana <>om>; Jeffrey
> (Zhaohui) Zhang <>
> *Subject:* Re: [Bier] draft-ietf-bier-ipv6-requirements-09
> Well, I’m glad that the work on requirements draft, albeit as product
> found wanting in AD’s assessment, has led to clarification of the crucial
> questions that e'one seems to agree need to be asked.
> It surprised me then mildly that my co-chair had to explicitly lay out the
> semantics of what was a clear direction spelled out during the meeting but
> that’s all well to get e’one better in sync I guess. Needless to say I am
> sharing his assessment and questions put to the room entirely.
> Some things that I think need explicit spelling out IMO after the last few
> meetings (since I’m not sure e’one in the process internalized that) is
> that WG is not here to tell people they cannot work on something whatever
> the perception seems to be, IETF doesn’t work that way. People go sideways
> and build stuff based on what we publish/develop in open source and for
> their customers in all kind of ways which may be neither fitting into an
> architecture, consensus or interest of a WG all the time. And that’s
> wonderful and more power to them, RFCs are free to download and they are
> just RFCs, they are not stone tablets brought from the mountain. However,
> and that's a big however, _if_ a work is looking for WG adoption and
> ultimately RFC status, the IETF process kicks in and the process has been
> here and well debugged over 30 years and that’s why Internet was built IME.
> The process is unusual in the way that it resists pretty well pressure
> based on non-technical claims, exceedingly poor architectural choices,
> chair shopping, padding of communication channels with “I participated only
> once to send a +1 to a list”, ad-hominem attacks and similar shenanigans
> that have been all tried over and over again. In the same vein the process
> tends to weigh based on reputation of “who said what in which context”';
> such reputation being built on community service and sound work over many
> years. And sometimes hard calls are made based on rough consensus called by
> people that are here to steer stuff and nudge it along the way. Sure, it’s
> easy to standardize and build “something”, it’s very hard to keep it going
> operationally @ Internet scale for 20 years and lots of those lessons are
> unfortunately scar tissue not easily transferred except at level of
> RFC1925. Last point to emphasize is that BIER is not the average set of
> RFCs, we have been handed the permission to go into the hourglass of the
> Internet, something that happens every 15 years or so. The stuff we deliver
> is as fundamental as MPLS or IP forwarding plane and as PS has to meet
> toughest architectural standards to prevent a melt-down of non-orthogonal,
> under’spec’ed solutions leading to poor operational properties @ scale and
> non-interoperable solutions which long-term serves no'one well that relies
> on IP technology to support high quality infrastructure @ scale.
> Juniper Business Use Only