Re: [bess] Éric Vyncke's Discuss on draft-ietf-bess-evpn-proxy-arp-nd-11: (with DISCUSS and COMMENT)

["Eric Vyncke (evyncke)" <[e-mail]>]

Hello Jorge,

Sorry for belated reply… Your email was kind of lost in my post-IETF-110 filled in-tray...

See below for EV> (for the many comments, as you have addressed them, I replied nothing).

Once I am clear about how normal DAD (i.e., non optimized by your document) continues to work, then I am clearing my DISCUSS. So, more explanations by email or in the I-D are welcome.

Regards

-éric


From: "Rabadan, Jorge (Nokia - US/Mountain View)" <[e-mail]>
Date: Thursday, 18 March 2021 at 09:04
To: Eric Vyncke <[e-mail]>, The IESG <[e-mail]>
Cc: "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>, "Bocci, Matthew (Nokia - GB)" <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: Éric Vyncke's Discuss on draft-ietf-bess-evpn-proxy-arp-nd-11: (with DISCUSS and COMMENT)

Hi Éric,

Thanks for this, it is very useful. Please see my comments in-line with [jorge].
We just published a revision, addressing yours and all the comments received in all the reviews.

Thanks again!
Jorge

From: Éric Vyncke via Datatracker <[e-mail]>
Date: Thursday, January 21, 2021 at 3:13 PM
To: The IESG <[e-mail]>
Cc: [e-mail] <[e-mail]>, [e-mail] <[e-mail]>, [e-mail] <[e-mail]>, Bocci, Matthew (Nokia - GB) <[e-mail]>, Bocci, Matthew (Nokia - GB) <[e-mail]>, [e-mail] <[e-mail]>
Subject: Éric Vyncke's Discuss on draft-ietf-bess-evpn-proxy-arp-nd-11: (with DISCUSS and COMMENT)
Éric Vyncke has entered the following ballot position for
draft-ietf-bess-evpn-proxy-arp-nd-11: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bess-evpn-proxy-arp-nd/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Thank you for the work put into this document. This system could indeed be very
useful but I am afraid that this is a very complex system especially for IPv6
NDP.

Minor regret in the shepherd write-up as the WG summary did not include any
comment on the WG consensus.

Thanks to Jean-Michel Combes for its Internet directorate review at:
https://datatracker.ietf.org/doc/review-ietf-bess-evpn-proxy-arp-nd-11-intdir-telechat-combes-2021-01-20/
as Jean-Michel added some important comments, please review them as well as I
support them especially those around DAD that should be a blocking DISCUSS
point.

I also second Erik Kline's DISCUSS points.

Question to the authors and BESS WG chairs: was this document submitted to a
6MAN/V6OPS WGs review ? This is where all IPv6 experts live :-)

Please find below some blocking DISCUSS points, some non-blocking COMMENT
points (but replies would be appreciated), and some nits.

I hope that this helps to improve the document,

Regards,

-éric

== DISCUSS ==

Would RFC 8929 be enough to solve the problem ?
[jorge] I found RFC8929 an interesting reading, thanks for the reference. However, unless I’m missing something the use-case is very different.
It seems RFC8929 tries to reduce broadcast domains by using MLSNs where each link is its own broadcast domain. In EVPN BDs, the idea is reduce the control plane Broadcast/Multicast flooding among PEs of the same BD by replacing them with BGP EVPN routes. For ARP/ND, this basically means we learn at the ingress PE by snooping ARP/NAs and advertise the entries in EVPN MAC/IP routes so that the egress PE learns ARP/ND entries, and can reply to its local ARP-Requests/NS. Also in RFC8929, even for the bridging proxy, it seems that the proxy appears as an IPv6 host on the backbone, which is not the case in this document. Another difference is that the proxy in RFC8929 uses only ND messages to register bindings and in our document, we also use static entries and EVPN messages (in addition to snooped ARP and NA messages).
Please let me know if you see it otherwise.

EV> the use case is indeed different but the handling of new ND code should be the same or similar even if the ‘transport/sharing’ of information is different. Moreover RFC 8929 has been published by an IPv6-heavy WG.


-- Section 3 --
"A Proxy-ARP/ND implementation MAY support all those sub-functions or only a
subset of them.", I am afraid that it is mandatory that the reply and
duplicate-ip must be coupled: either both of them are active or none of them
are active else the system allows for duplicate IP addresses.
[jorge] the new text is as follows, let me know if it is okay. Note that the duplicate ip detection on the PEs is new in this document, and we didn’t want to make it mandatory we allow backwards compatibility with RFC7432 EVPN PEs that do proxy-ARP/ND.

   A Proxy-ARP/ND implementation MUST at least support the Learning,
   Reply and Maintenance sub-functions.  The following sections describe
   each individual sub-function.

EV> this is a progress of course but I am still puzzled how duplicate address detection can work then ? Failing to do DAD can cause very critical operational issues. Or do you rely on other mechanisms ?


-- Section 3.1 --
"A Proxy-ARP/ND implementation SHOULD support static, dynamic and EVPN-learned
entries." why not a MUST ? Or at least for dynamic & EVPN-learned ? or at least
one ?
[jorge] new text is as follows, let me know if it is ok:

   A Proxy-ARP/ND implementation in an EVPN BD MUST support dynamic and

   EVPN-learned entries, and SHOULD support static entries.

EV> Perfect thank you


"Upon receiving traffic from the CE... the PE will activate the IP->MAC and
advertise it in EVPN" it is unspecified how many bindings can be advertised in
the case of multiple static MAC for one IP... only one or all ?
[jorge] good point, thx, changed it to:

Only in that case, the PE will activate the IP->MAC and advertise

   only that IP and MAC in an EVPN MAC/IP Advertisement route.

EV> thank you


-- Section 3.2 --
Why not flooding to all other PEs the ARP/NS with unknown options ? It would be
safer.
[jorge] yes, the new text is as follows, let me know please:

   f.  A PE MUST only reply to ARP-Request and NS messages with the

       format specified in [RFC0826] and [RFC4861] respectively.

       Received ARP-Requests and NS messages with unknown options SHOULD

       be either forwarded (as unicast packets) to the owner of the

       requested IP (assuming the MAC is known in the Proxy-ARP/ND table

       and BD) or discarded.  An option to flood ARP-Requests/NS

       messages with unknown options MAY be used.  The operator should

       assess if flooding those unknown options may be a security risk

       for the EVPN BD.  An administrative option to control this

       behavior ('unicast-forward', 'discard' or 'forward') SHOULD be

       supported.  The 'unicast-forward' option is described in

       Section 3.4.

EV> please note that the ‘forward’ behavior does not seem to be listed as a sub-function
EV> nevertheless with the added text in section 3.6, this appears to be OK for me now.



-- Section 3.6 --
This function MUST be a mandatory part of the list of functions of section 3.
[jorge] the next text is as follows. As discussed, we didn’t want to make it a MUST to allow RFC7432 backwards compatibility.. let me know if it is okay.

   The Proxy-ARP/ND function SHOULD support duplicate IP detection as

   per this section so that ARP/ND-spoofing attacks or duplicate IPs due

   to human errors can be detected.  For IPv6 addresses, CEs will

   continue to carry out the DAD procedures as per [RFC4862].

EV> still a little unsure how DAD could work without this sub-element. Even if appears to me that this sub-element is more an anti-spoofing feature than a DAD proxy...



-- Section 5.2 --
An easy to fix: "Any unknown source MAC->IP entries" isn't it IP->MAC as in the
rest of the document including the terminology section ?
[jorge] fixed this one and a couple of other occurrences. Thanks!

EV> you are welcome


-- Section 5.4 --
"traffic to unknown entries is discarded" which traffic (section 5.5 is much
better to this point suggest to copy the text)? The NDP/ARP or normal data
plane traffic ? Where is this behavior specified in the 6 sub-functions of
section 3 ?
[jorge] added the following text, let me know if it is okay:

   In this scenario, the Learning sub-function is

   limited to static entries, the Maintenance sub-function will not

   require any procedures due to the static entries, and the Flooding

   reduction sub-function will completely suppress Unknown ARP-Requests/

   NS messages as well as GARP and unsolicited-NA messages.

EV> OK



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Consider adding a section about host not doing GARP or doing no DAD or
optimistic DAD.
[jorge] the document does not impose the use of GARP or DAD or ODAD, or its absence. Could you please elaborate what you would like to see in that section?

EV> what would be the impact of a CE moving ‘silently’ (no GARP/DAD/ODAD) from PE1 to PE2 ?


-- Section 1 --
Is there any reason why the terminology section is not alphabetically sorted ?
[jorge] none, I just sorted it.

EV> ;-)

-- Section 2.1 --
I would have assumed that the multicast nature of IPv6 address resolution would
cause more problems than IPv4 ARP. The use of link-local multicast groups do
not usually help as MLD snooping is often disabled in switches for link-local.
Not to mention that there could be more IPv6 addresses per node than IPv4
address and IPv6 addresses keep changing. Do the authors have data to back this
section ?
[jorge] I added a sentence in that respect. As a side note, one of the references that we include claims that the use of SN-multicast addresses in NS messages is actually better than broadcast in ARP, given that SN-multicast IP Das can be easily identified and discarded at the receiving CEs (assuming that the PEs do not have MLD snooping enabled) https://delaat.net/rp/2008-2009/p23/report.pdf

EV> I failed to see the added sentence in -13
EV> the URL you wrote above does not work anymore... Also, quite an old reference


-- Section 2.2 --
Unsure about the meaning of "large layer-2 peering network"... Do we peer at
layer-2 ? Now, I understand what is meant of course but the wording appears
strange to me (not being an English native), may I suggest "large layer-2
network for peering" ?
[jorge] how about “A typical IXP provides access to a large layer-2 Broadcast Domain for peering purposes”

EV> ok


Please expand GARP in "Unsolicited GARP". Also, this is a pleonasm as
gratuitous ARP are by definition "unsolicited" ;-)
[jorge] ok, done. Note that GARP is included in the terminology section though.

EV> then no need to expand indeed, the pleonasm still stands though


The definition of a CE in an IXP network would be welcome.
[jorge] added:
“We refer to these Internet routers as Customer Edge (CE) devices in this section”

EV> thank you


I am afraid that I do not agree with "The issue may be better in IPv6 routers"
even if the IPv6 addresses are static in this environment (i.e., no RFC 4941
addresses).
[jorge] How about “The issue might be better in IPv6 routers if MLD-snooping was enabled, since ND uses SN-multicast address in NS messages”

EV> LGTM


-- Section 3 --
An IPv6 example would also be useful as NS is not like ARP.
[jorge] added:

   In the same example, if we assume IP1, IP2, IP3 and IP4 are now IPv6

   addresses and Proxy-ARP/ND is enabled in BD1:



   1.  PEs will start adding entries in a similar way as for IPv4,

       however there are some differences:



       A.  IP1->M1 and IP2->M2 are learned as dynamic entries in PE1 and

           PE2 respectively, by snooping NA messages and not by snooping

           NS messages.  In the IPv4 case, any ARP frame can be snooped

           to learn the dynamic Proxy-ARP entry.  When learning the

           dynamic entries, the R and O Flags contained in the snooped

           NA messages will be added to the Proxy-ND entries too.



       B.  PE1 and PE2 will advertise those entries in EVPN MAC/IP

           Advertisement routes, including the corresponding learned R

           and O Flags in the ARP/ND Extended Community.



       C.  PE3 also adds IP4->M4 as dynamic, after snooping an NA

           message sent by CE4.



   2.  When CE3 sends an NS message asking for the MAC address of IP1,

       PE3 behaves as in the IPv4 example, by intercepting the NS, doing

       a lookup on the IP and replying with an NA if the lookup is

       successful.  If it is successful the NS is not flooded to the

       EVPN PEs or any other local CEs.



   3.  If the lookup is not successful, PE3 will flood the NS to remote

       EVPN PEs attached to the same BD and the other local CEs as in

       the IPv4 case.


EV> thank you


Should the default behavior/sub-function of flooding be added to the list of 1)
to 6) ?
[jorge] based on the specific section for it, the flooding reduction sub-function can also be configured to not suppress any flooding… so it is sort of implicit? Let me know otherwise.

EV> still an ambiguous name though, what about ‘flood handling’ ? or similar


-- Section 3.1 --
"Upon receiving traffic from the CE"... but with which IP address ? (OK
guessable but let's be clear in a standard specification). It also seems to me
like a local policy / feature that do not require standardization.
[jorge] I clarified by using IP1 and MAC1 as an example, let me know if it clarifies please.

   Upon receiving traffic from the CE, the PE

   will check that the source MAC, E.g., MAC1, is included in the list of allowed

   MACs.  Only in that case, the PE will activate the

   IP1->MAC1 and advertise only that IP1 and MAC1 in an EVPN MAC/IP

   Advertisement route.

EV> thanks


"Note that MAC and IPs with value 0 SHOULD NOT be learned" unsure why it is a
singular MAC and plural IP ;-)
[jorge] changed to “a MAC or an IP address with value 0”

EV> I do not see the change in -13


"only if the ARP/NA message creating the entry was NOT flooded before" what is
meant by 'flooded' ?
[jorge] flooded in the BD to local CEs. If it was already received by other local CEs (if the ARP/NA message was multicast/broadcast) there is no need to flood multicast/broadcast the same information to the local CEs again. Changed the text to reflect that:

      The PE SHOULD send an

      unsolicited GARP/NA message for dynamic entries only if the ARP/NA

      message that previously created the entry on the PE was NOT

      flooded to all the local connected CEs before.  This unsolicited

      GARP/NA message makes sure the CE ARP/ND caches are updated even

      if the ARP/NS/NA messages from CEs connected to remote PEs are not

      flooded in the EVPN network.

EV> ok


Suggestion to add some descriptions of the impact of a rebooting/new PE with an
empty cache while other PE have caches.
[jorge] added the following, let me know if it makes sense.
“In case of a PE reboot, the static and EVPN entries will be re-added as soon as the PE is back online and receives all the EVPN routes for the BD. However, the dynamic entries will be gone. Due to that reason, new NS and ARP Requests will be flooded by the PE to remote PEs and dynamic entries gradually re-learned again.”

EV> thank you


-- Section 3.1.1 --
Should RFC 4861 also be mentioned in "The use of the R Flag in NA messages has
an impact on how hosts select their default gateways when sending packets
off-link" ?
[jorge] added, thx


"Static entries SHOULD have the R Flag information added by the management
interface.", else what is the default setting of te R-flag ?
[jorge] there is this sentence in the same bullet, let me know if it is not clear:
“If the R and O Flags are not configured, the default value is 1.”


"This configured R Flag SHOULD be an administrative choice with a default value
of 1", so all other CE will appear as a router ? Not critical in the case of
IXP as it is a default free zone but in a DC (suggest s/SHOULD/MAY/)?
[jorge] changed, thx


Is there a recommended setting for the O-flag?
[jorge] I modified the sentence as follows:
“These configured R and O Flags MAY be an administrative choice with a default value of 1.”


-- Section 3.2 --
Is "'anycast' is enabled in the BD" specified somewhere in this document ?
[jorge] good point. I added the following in the Learning sub-function section:
“This document specifies an "anycast" capability that can be configured for the proxy-ND function of the PE, and affects how dynamic Proxy-ND entries are learned based on the O Flag of the snooped NA messages. If the O Flag is zero in the received NA message, the IP->MAC SHOULD only be learned in case the IPv6 "anycast" capability is enabled in the BD. Irrespective, an NA message with O Flag = 0 will be normally forwarded by the PE based on a MAC DA lookup.”


Suggest to split the point d) in three items: one for each flag.
[jorge] done, thx


Why is there no IPv6 equivalent of e) ?
[jorge] we think the use of these ARP probes is not that common, whether IPv6 DAD procedures are performed by all CEs, and we want the PEs to reply to DAD messages if they can, to reduce the flooding among PEs. That’s how it has been implemented. Let me know if it is ok.


In point f), "or discarded" can a packet with known IP->MAC mapping be
discarded as well ?
[jorge] do you mean with known options? I don’t think that needs to be specified but let me know if you think differently.


-- Section 3.4 --
Please expand "IRB"
[jorge] done


Should "flushed if the owner is no longer in the network" be complemented with
a BGP withdrawal ?
[jorge] added: “..and flushed (and the associated RT2 withdrawn) if the owner..”


Is there any security exposure (control plane DoS) by forcing the PE without
IRB to have an IPv6 LLA ?
[jorge] if the BD does not have an IRB, the LLA is only used for the purpose of the refreshes. It is not associated to any reachable IP interface.. let me know if it is ok.


-- Section 3.6 --
Strong suggestion to s/the PE MAY send a CONFIRM message to the former owner of
the IP/the PE SHOULD send a CONFIRM message to the former owner of the IP/
[jorge] done.


Unsure why CONFIRM is in uppercase BTW.
[jorge] changed to Confirm


"If the PE does not receive an answer within a given timer" is there a
recommended value for this timer ?
[jorge] Added “The default RECOMMENDED time to receive the confirmation is 30 seconds”


I have re-read three times the "anti-spoofing MAC" part, and, I still do not
understand it... Is MAC-AS the black-hole address (then why not using a all 0
MAC address) or an alternative MAC address (but then who modifies the frame
header to the CE) ?
[jorge] this is about updating all the CE’s ARP/ND caches with the AS-MAC for the IP, to make sure the spoofer does not attract the traffic for the IP. Using an all 0s MAC would not be accepted by the CEs, and we wouldn’t know if there is traffic from the CEs to the ‘suspect’ IP. I re-worded it a bit, let me know if it is better:
“Optionally the PE MAY associate an "anti-spoofing-mac" (AS-MAC) to the duplicate IP in the Proxy-ARP/ND table. The PE will send a GARP/unsolicited-NA message with IP1->AS-MAC to the local CEs as well as an RT2 (with IP1->AS-MAC) to the remote PEs. This will update the ARP/ND caches on all the CEs in the BD, and hence all the CEs in the BD will use the AS-MAC as MAC DA when sending traffic to IP1. This procedure prevents the spoofer from attracting any traffic for IP1. Since the AS-MAC is a managed MAC address known by all the PEs in the BD, all the PEs MAY apply filters to drop and/or log any frame with MAC DA= AS-MAC. The advertisement of the AS-MAC as a "black-hole MAC" (by using an indication in the RT2) that can be used directly in the BD to drop frames is for further study.”


-- Section 5.1 --
"in the peering network" is this use case only valid in the case of IXP ?
[jorge] changed it to “BD”, thx


-- Section 5.2 --
"The Proxy-ARP/ND function is enabled" but what about the sub-functions
enumerated in section 3 ?
[jorge] added:
“This scenario makes use of the Learning, Reply and Maintenance sub-functions, with an optional use of the Unicast-forward and Duplicate IP detection sub-functions. The Flooding reduction sub-function uses the default flooding for unknown ARP-Request/NS messages.”


"by snooping ARP/ND messages issued by the CEs" isn't the learning sub-function
?
[jorge] yes, added the functions.


-- Section 5.3 (and others) --
Why is this section apparently limited to IXP only ?
[jorge] it was written by our co-author IXP :-) but I replaced IXP with “operator” and “peering network” with “BD” in this section.


-- Section 5.5 --
There is a big overlap between this clear/good sub-sections and the previous
ones. Suggest to keep only this one + section 5.6.
[jorge] sections 5.1 to 5.4 are typical scenario types, and 5.5/5.6 refer to them for the particular examples of IXPs and DCs. I re-worded the sections a bit, but prefer to keep them since it was appreciated by some operators.


-- Section 5.6 --
"IPv6 'anycast' may be required in DCs, while it is not a requirement in IXP
networks." I have doubts that anycast is never used in IXP networks. Let's
rather say "seldom used in IXP networks".
[jorge] changed it to “while it is typically not a requirement in IXP networks” based on a previous review.


-- Section 6 --

Nothing is said about putting some limits on the number of entries for an IP
address... Else, this could lead to a DoS against the proxy & BGP sessions.
[jorge] added:
“The Proxy-ARP/ND function specified in this document does not allow the learning of an IP address mapped to multiple MAC addresses in the same table, unless the "anycast" capability is enabled (and only in case of Proxy-ND). When "anycast" is enabled in the Proxy-ND function, the number of allowed entries for the same IP address MUST be limited by the operator to prevent DoS attacks that attempt to fill the Proxy-ND table with a significant number of entries for the same IP.”


  "For example, IXPs should disable all unneeded control protocols, and
   block unwanted protocols from CEs so that only IPv4, ARP and IPv6
   Ethertypes are permitted on the peering network.  In addition, port
   security features and ACLs can provide an additional level of
   security."
While the above text is a good recommendation, I wonder what it the
relationship with this document ?
[jorge] true, however this document is a reference for IXPs (and co-author by one very relevant IXP) so this makes sure people is fully aware that there are other considerations to look at. We prefer to keep the text if it is okay.


== NITS ==

-- Abstract --
s/(DBs)/(BDs)/
[jorge] fixed, thx


-- Section 2.2 --
s/IPv4 layer-3 addresses/IPv4 addresses/
[jorge] fixed, thx


-- Section 3.1 --
Please use lower hexadecimal number, e.g., s/0x86dd/0x86dd/
[jorge] fixed, thx



-- Section 5.5 --
The "IXP-LAN" terminology is only used in this section while others are using
"peering network" or "IXP networks". Let's choose only one ;-)

[jorge] fixed, thx