Re: [Bier] draft-ietf-bier-ipv6-requirements-09

Adrian Farrel <> Thu, 26 November 2020 23:53 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 0DE1F3A067A for <>; Thu, 26 Nov 2020 15:53:12 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 2rCVJIxlYQ8b for <>; Thu, 26 Nov 2020 15:53:08 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id C45AE3A0621 for <>; Thu, 26 Nov 2020 15:53:06 -0800 (PST)
Received: from ( []) by (8.14.4/8.14.4) with ESMTP id 0AQNr4HU012728; Thu, 26 Nov 2020 23:53:04 GMT
Received: from (unknown []) by IMSVA (Postfix) with ESMTP id 795A32203B; Thu, 26 Nov 2020 23:53:04 +0000 (GMT)
Received: from (unknown []) by (Postfix) with ESMTPS id 63BE62203A; Thu, 26 Nov 2020 23:53:04 +0000 (GMT)
Received: from LAPTOPK7AS653V ([]) (authenticated bits=0) by (8.14.4/8.14.4) with ESMTP id 0AQNr3Vl012875 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Nov 2020 23:53:03 GMT
Reply-To: <>
From: "Adrian Farrel" <>
To: "'Jeffrey \(Zhaohui\) Zhang'" <>, "'Tony Przygienda'" <>, "'Greg Shepherd'" <>
Cc: "'BIER WG'" <>
References: <> <> <> <> <> <> <02fd01d6c3f5$a8f23de0$fad6b9a0$> <> <033a01d6c410$92e413f0$b8ac3bd0$> <>
In-Reply-To: <>
Date: Thu, 26 Nov 2020 23:53:02 -0000
Organization: Old Dog Consulting
Message-ID: <03c201d6c44f$478cd240$d6a676c0$>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_03C3_01D6C44F.478E7FF0"
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQJW4ZgcgJn9V6dtIlaK/47czvNh1wE/cTJnAeqNA30DXQLSuwINZ543AaWk7vEBlsWxfwEyp29BAipMmkYBDXPuWqhY/rZA
Content-Language: en-gb
X-TM-AS-Product-Ver: IMSVA-
X-TM-AS-Result: No--7.847-10.0-31-10
X-imss-scan-details: No--7.847-10.0-31-10
X-TMASE-Result: 10--7.847000-10.000000
X-TMASE-MatchedRID: 8HTFlOrbAtHxIbpQ8BhdbPVY7U3NX8JgIiTd2l7lf6FWPcnrekBHfLuJ BaiamILRw1DWsvv5I3FyrwOLCH41n7mvMSppeWbNrMcMK3Nm8dl9iuvWn3J8Knxz6Ei4ZX4xI2O nLXnAwVXlrBUldX2ZMcte5ulkcPgwBs6OOQdmb9UsYOarN8c4H6t8JTb7GATYK4YqHgCSopWIus hUx+1X0MAHLDIY2YiGsPHRM3GN+aIjb6jOf5okrMzSKGx9g8xh4nNiagWRHaog+XOQbVTXW5D2y 7vOr03QWMYBlEakkxqsTpB1Wxdh0Q0rMd1m+Q0JLFqCUQ7xhcwX2zxRNhh61T0nqBp2UmIcI3BP FOcw9qOCT1fgQNdYp3d5M5tyqJS/5Wlph4hc0awtMfCdg6KRDXBKi8EjNDaUHdFjikZMLIceNYX nmybWxSYxz8zvRVbyA4S6rJfSNyN9qY2Jd1gxtUQjOqFi1L/nVIv0P7/ytVnvwGUwU4gzmWlpmu YgAcs4aXdcf0WxcLsyMJ4ANEGmEIcp7k3a0wUd0ytzHp3p+JZjCt269TGh+Hrjo3X9e+DHiPMxD l6SgX+qzyyWxHZ3AvBilazGYLz4Bzpq5anXg++5UUv022NJAljWwaUPviNZ7K5p55rm0/P3IUiz 0mDTydKVAorLrVK5R9GaaqhIHHSkGDD954zV5AfBcTvwUB7CrYIJqyGcXHXDOS0FhcAXSryDGOd LfK2qK6LsX8TCsmLRlWHeWPoWcShiJto1D8m8vbqrzSdPiqqpB34ezti/uyBwR3y4lKPEDzDrDF u2B8ZDKSOKGShQXJcq7ydQXn7lAVeYrJDUIaeeAiCmPx4NwGmRqNBHmBveGtkvK5L7RXGw7M6dy uYKg4VH0dq7wY7uDjItjOFWI9Ee539hrEurbrgbkYAIRPQAQSWHZuUmbT3Jz+fpZFsKJnkJvy9F CwyiQpapEqft36yfUpiQdZtfevPqYFBx86EKpe1H2udguDA=
X-TMASE-SNAP-Result: 1.821001.0001-0-1-12:0,22:0,33:0,34:0-0
Archived-At: <>
Subject: Re: [Bier] draft-ietf-bier-ipv6-requirements-09
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "\"Bit Indexed Explicit Replication discussion list\"" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 26 Nov 2020 23:53:12 -0000

Thanks Jeffery,


I think our discussion forks.

In one direction Tony says (possibly) don’t worry about legacy transit routers because we shouldn’t be tunneling through them anyway.

In the other direction we worry about legacy routers, and here you are suggesting we should hope that they are legacy-but-modern 😉 That doesn’t seem to work out for us because *if* we need to tunnel through non-BIER routers we should probably assume that they might be old enough to not be considered modern.


I do like your answer that if the BIER header is encountered as payload it will not be hashed because of the first nibble. I’d missed that and it handles the case of native encapsulation.


But I think that even legacy IPv6 routers that do ECMP are capable of walking the extension headers until they find a header that is a known protocol or until they find one they can’t parse. (But I may be wrong here.)


Now, we have got diverted (again) into discussions of what we can and can’t do with different solutions. We need to come back to the requirement:


Do we need to be able to tunnel through legacy routers? I am sure I hear Tony saying “no, that is explicitly excluded”.

If the answer is “no”, let’s capture that in the requirements doc.

If the answer is “yes” then we have a second question…

Do we need to provide ECMP in that tunnel?

If the answer to that is “no”, we’re done.

But if the answer here is “yes”, we have to ask…

Is it enough to rely on the flow label (and src/dst), or do other fields need to be available.





From: Jeffrey (Zhaohui) Zhang <> 
Sent: 26 November 2020 23:38
To:; 'Tony Przygienda' <>om>; 'Greg Shepherd' <>
Cc: 'BIER WG' <>
Subject: RE: [Bier] draft-ietf-bier-ipv6-requirements-09


Hi Adrian,


Please see zzh> below.


From: Adrian Farrel < <> > 
Sent: Thursday, November 26, 2020 11:24 AM
To: Jeffrey (Zhaohui) Zhang < <> >; 'Tony Przygienda' < <> >; 'Greg Shepherd' < <> >
Cc: 'BIER WG' < <> >
Subject: RE: [Bier] draft-ietf-bier-ipv6-requirements-09


[External Email. Be cautious of content]


Hi Jeffery,


Happy Thanksgiving!


Zzh> Thanks!


Are you saying that all IPv6 routers use flow label and other primary header for entropy and none of them looks into the payload? (I’m asking cos I don’t know what v6 routers do.)


Zzh> No. I just believe/assume the preferred/modern way going forward for ECMP is flow label based. The BIERv6 proponents are emphasizing on “modern” aspect of that solution, too 😊


RFC 6437 implies that routers should use the “traditional” 5-tuple in addition to the flow label. But maybe routers don’t follow 6437?


Zzh> I assume that is only applicable for the typical TCP/UDP payload, and not enough for today’s multi-service networks?


BTW, I not trying to debate the solutions. I’m trying to find out what behavior is required in the IPv6 tunnel between BFRs. It appears that ECMP is required (good). Are we asking for any “special” ECMP behavior, or do we assume that the tunnel transit nodes are blind legacy nodes that cannot tell a BIER packet from any other packet?


Zzh> Understand. Technical discussions are always appreciated, even if it is for debating the solutions. The chairs once requested that the requirements draft does not evaluate solutions, but we can certainly evaluate outside the draft.


Zzh> Tunnel transit nodes may not understand BIER at all, but with BIERin6 they should not mistake a BIER packet as an IP packet, as the first nibble of the BIER header is set to 0101 (would be “IPv5” if one treat it as IP version). This would then be similar to PW ECMP (where a CW with first nibble of 0000 is used).

Zzh> With BIERv6, even though the BIER payload follows the IPv6 header directly, the “traditional” 5-tuple based ECMP may not be used after all for two reasons: 1. The payload may not be IP 2. It seems that to locate the start of real payload (after the extension headers) for ECMP hashing, a router will need to follow the chain of extension headers one by one (since the standard IPv6 header does not have a field for the total length of all headers) and a legacy transit router may not actually do that for implementation/performance reasons.

Zzh> Even if a legacy transit router actually follows the extension headers to do 5-tuple based ECMP, since the BIERv6 proponents emphasizes on the “modern” technology/direction, it would be reasonable to consider the ECMP issue in the context of flow label based solution.

Zzh> Thanks.

Zzh> Jeffrey





From: Jeffrey (Zhaohui) Zhang < <> > 
Sent: 26 November 2020 15:25
To: <> ; 'Tony Przygienda' < <> >; 'Greg Shepherd' < <> >
Cc: 'BIER WG' < <> >
Subject: RE: [Bier] draft-ietf-bier-ipv6-requirements-09


Hi Adrian,


Looks like I don’t have a life – making arguments for this life and death situation on Thanksgiving day 😊 


Anyway, about the ECMP topic, not sure if the following addresses your questions/comments.


At BIER layer itself, BIER packet has an entry field used for hashing to decide which ECMP path to the next BFR it will take. This is used by any BFR.


If BFR1 determines that for to reach BFR2 it is going to use tunnel1 (vs. tunnel2 or another L2 link), and tunnel1 is an IPv6 tunnel, then with BIERin6 an IPv6 header is put on by BFR1, with the BIER header following the IPv6 header, and the BIER header’s entropy field copied into IPv6 header’s flow label field. That flow label field is then used by the routers along the tunnel to do ECMP.




From: BIER < <> > On Behalf Of Adrian Farrel
Sent: Thursday, November 26, 2020 8:12 AM
To: 'Tony Przygienda' < <> >; 'Greg Shepherd' < <> >
Cc: 'BIER WG' < <> >
Subject: Re: [Bier] draft-ietf-bier-ipv6-requirements-09


[External Email. Be cautious of content]


I’ve been reading up on this thread and the three related drafts.


I don’t dip into BIER often (I’m not a multicast person, and I have a life), but this seemed to be a fairly weighty topic which has been bubbling away for a while, and the volume of the discussion suggested that this is a really important question (it sounded like a life and death decision judging by some of the emails!).


I think Tony captured some really key points in his email below. I particularly like his observation that BIER is working at the neck of the hourglass: that demands caution and good judgement; it also requires everyone to step back and do the right thing regardless of their investment (emotional or financial) in their preferred solution.


It seems to me (again, from the outside, and apologies if this is re-opening age-old discussions) that most of this is just protocol engineering. We have long experience at making any protocol do anything we want. If a particular solution lacks some capability, it can always be added with an extra TLV. That makes comparisons of solutions (also known as beauty contests) somewhat pointless: if you judge A better than B because B lacks some feature, then we just add the feature to B, and the cycle starts again.


That means that, while the requirements work is highly valuable for working out what the solution should deliver, it is not so helpful in determining which solution the WG should pursue. We are left, IMHO, with some of the edge requirements about transiting non-BIER nodes. These are nodes that can happily process “normal” IPv6 packets, but don’t know what to do with a BIER encapsulation. That looks like Section 3.1.3 of the requirements draft.


Embedded in that requirement is discussion of what an IPv6 router that is a transit might do with a packet. On the whole, routers just route on the fields in the v6 header itself, but they may look deeper in order to perform ECMP functions etc. For example, they may look for the transport payload to hash on ports etc. To achieve this, a router must be able to step over any additional headers (RH, DOH, etc.) to find the payload or must know not to even try. In general, a router that doesn’t understand a header will step over it if it can, but will probably give up the hunt for hashable fields. 


At this point I ran aground ☹ 8926 doesn’t have anything to say about ECMP in a BIER network (with or without BIER-capable routers). But 8279 has a nice fat section on ECMP, but this seems to describe how ECMP works when processing the BIER encapsulation for equal cost paths between BIER routers, not for how the “underlay” (the IPv6 network in this case) might handle equal cost paths in its own routing.


Any clues as to how ECMP is expected to work in the context of the v6 requirements? Anything that should be added to 3.1.3 or a new section?






From: BIER < <> > On Behalf Of Tony Przygienda
Sent: 20 November 2020 05:36
To: Greg Shepherd < <> >
Cc: BIER WG < <> >; Gyan Mishra < <> >; draft-ietf-bier-ipv6-requirements < <> >; <>  < <> >; Alvaro Retana < <> >; Jeffrey (Zhaohui) Zhang < <> >
Subject: Re: [Bier] draft-ietf-bier-ipv6-requirements-09


Well, I’m glad that the work on requirements draft, albeit as product found wanting in AD’s assessment, has led to clarification of the crucial questions that e'one seems to agree need to be asked. 

It surprised me then mildly that my co-chair had to explicitly lay out the semantics of what was a clear direction spelled out during the meeting but that’s all well to get e’one better in sync I guess. Needless to say I am sharing his assessment and questions put to the room entirely. 

Some things that I think need explicit spelling out IMO after the last few meetings (since I’m not sure e’one in the process internalized that) is that WG is not here to tell people they cannot work on something whatever the perception seems to be, IETF doesn’t work that way. People go sideways and build stuff based on what we publish/develop in open source and for their customers in all kind of ways which may be neither fitting into an architecture, consensus or interest of a WG all the time. And that’s wonderful and more power to them, RFCs are free to download and they are just RFCs, they are not stone tablets brought from the mountain. However, and that's a big however, _if_ a work is looking for WG adoption and ultimately RFC status, the IETF process kicks in and the process has been here and well debugged over 30 years and that’s why Internet was built IME. The process is unusual in the way that it resists pretty well pressure based on non-technical claims, exceedingly poor architectural choices, chair shopping, padding of communication channels with “I participated only once to send a +1 to a list”, ad-hominem attacks and similar shenanigans that have been all tried over and over again. In the same vein the process tends to weigh based on reputation of “who said what in which context”'; such reputation being built on community service and sound work over many years. And sometimes hard calls are made based on rough consensus called by people that are here to steer stuff and nudge it along the way. Sure, it’s easy to standardize and build “something”, it’s very hard to keep it going operationally @ Internet scale for 20 years and lots of those lessons are unfortunately scar tissue not easily transferred except at level of RFC1925. Last point to emphasize is that BIER is not the average set of RFCs, we have been handed the permission to go into the hourglass of the Internet, something that happens every 15 years or so. The stuff we deliver is as fundamental as MPLS or IP forwarding plane and as PS has to meet toughest architectural standards to prevent a melt-down of non-orthogonal, under’spec’ed solutions leading to poor operational properties @ scale and non-interoperable solutions which long-term serves no'one well that relies on IP technology to support high quality infrastructure @ scale. 



Juniper Business Use Only


Juniper Business Use Only