Re: [RTG-DIR] Rtgdir early review of draft-dmk-rtgwg-multisegment-sdwan-05

[Linda Dunbar <[e-mail]>]

Adrian,

See below for the additional resolutions:

Linda

From: Adrian Farrel <[e-mail]>
Sent: Saturday, February 17, 2024 3:13 AM
To: Linda Dunbar <[e-mail]>; 'Yingzhen Qu' <[e-mail]>
Cc: [e-mail]; [e-mail]; [e-mail]
Subject: RE: [RTG-DIR] Rtgdir early review of draft-dmk-rtgwg-multisegment-sdwan-05

Hi Linda,

Thank you for your careful consideration of my review.

In line…

Cheers,
Adrian

From: rtg-dir <[e-mail]<mailto:[e-mail]>> On Behalf Of Linda Dunbar
Sent: 16 February 2024 20:16
To: Yingzhen Qu <[e-mail]<mailto:[e-mail]>>; Adrian Farrel <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [RTG-DIR] Rtgdir early review of draft-dmk-rtgwg-multisegment-sdwan-05

Adrian,

Thank you very much for the detailed review.
Please see if  the resolution below are acceptable?

Thank you,
Linda


On Sat, Feb 10, 2024 at 1:38 PM Adrian Farrel via Datatracker <[e-mail]<mailto:[e-mail]>> wrote:
Reviewer: Adrian Farrel
Review result: Has Issues

Hello,

draft-dmk-rtgwg-multisegment-sdwan-05.txt

I have been selected to do a routing directorate "early" review of this
draft.

The routing directorate will, on request from the working group chair,
perform an "early" review of a draft before it is submitted for
publication to the IESG. The early review can be performed at any time
during the draft's lifetime as a working group document. The purpose of
the early review depends on the stage that the document has reached.

This review was requested by the RTGWG chairs as part of the process for
considering this draft for working group adoption.

For more information about the Routing Directorate, please see
https://wiki.ietf.org/en/group/rtg/RtgDir

Although these comments are primarily for the use of the RTGWG chairs,
it would be helpful if you could consider them along with any other
adoption poll or mailing list comments that you receive, and strive to
resolve them through discussion or by updating the draft.

Document: draft-dmk-rtgwg-multisegment-sdwan-05.txt
Reviewer: Adrian Farrel
Review Date: 10 February 2024
Intended Status: Standards Track

= Summary =

I believe this document is suitable to be a candidate for an adoption
poll in RTGWG.
[Linda] Thank you.
I have some concerns about this document that I think should be resolved
during the adoption process (or before).

= Overview =

The IETF has documented GENEVE in the Standards Track RFC 8926. It is a
generic encapsulation for network virtualization.

This document proposes using GENEVE to encapsulate IPsec packets between
customer premises equipment (CPEs) and data centre gateways as it is
carried across a variety of underlay networks in support of "SD-WAN"
connectivity services. It is suggested tht this is necessary so that key
transit relay points do not need to decrypt and re-encrypt the packets,
allowing IPsec to run correctly "edge-to-edge" between CPEs.

I assume there is a reason why this document was not taken to NVO3, but
it seems wise to let them know about the proposal to adopt it in RTGWG.
Otherwise, it seems that this draft could be in scope for RTGWG
according to the charter.
[Linda] Yes, the RTGwg Chair already send the notice to NVO3 group asking for feedback. But NVO3 WG seems to be dead, no one responded.
The problem statement is somewhat woolly and is better understood from
the examples than from the Introduction. However, the examples seem to
offer credible scenarios and requirements. What is not clear from the
examples or the introduction is why GENEVE is an appropriate solution
rather than any of the many other tunnelling encapsulations that exist.
It is not until Section 4 that we discover why GENEVE is suggested as a
solution. I have no doubt that using GENEVE in this way could be made to
work (modulo some fixes to the proposal in the draft), and if that is
what the WG wants to work on, then it seems to be OK.
[Linda] How about the following rearrangement of the Intro section?
Enterprises connecting to Cloud DC may find significant benefits in leveraging the Cloud Backbone for transporting traffic between their CPEs, such as

  1.  Enterprises can benefit from the robust and high-performance infrastructure cloud service providers provide by leveraging diverse paths and harnessing cloud backbones' scalability and global reach to reduce the risk of downtime or disruptions.
  2.  The scalability of the Cloud Backbone allows for efficient handling of increased data traffic, accommodating the growing demands of modern enterprises.
  3.  Cloud Backbone's centralized management and orchestration capabilities contribute to simplified network administration, enabling organizations to streamline their operations and respond more effectively to changing business requirements.

To ensure security, enterprise traffic between their CPEs requires encryption. By steering the encrypted traffic through the Cloud Backbone without the need for decryption and re-encryption at Cloud GWs, processing demands at these GWs can be significantly reduced. This streamlined approach not only maintains the integrity of the encrypted traffic but also optimizes processing resources, enhancing overall efficiency within the cloud infrastructure.
This document describes a method for SD-WAN CPEs using GENEVE Encapsulation [RFC8926] to encapsulate the IPsec encrypted packets and send them to their closest Cloud GWs, who can steer the IPsec encrypted payload through the Cloud Backbone without decryption to optimal egress Cloud GWs which then forward the original IPsec encrypted payload to the destination CPEs. This method is for Cloud Backbone to connect multiple segments of SD-WAN without the Cloud GWs decrypting and re-encrypting the payloads.
[AF] This is much better. What is missing (perhaps obvious to you) is why a GW would otherwise need to decrypt and re-encrypt. I think you probably need:

  *   Each GW must look at the {which?} fields in the packet to determine the next hop (i.e., GW) to which to forward the packet
[Linda] added a sentence saying that GW looks at the sub-TLV carried in the GENEVE header (which is specified Section 4 of the document)

  *   If native IPsec is used end-to-end, these fields are obscured
  *   Therefore, if IPsec is to be used, each GW must maintain an SA with each other GW, with the packet being decrypted and re-encrypted at each GW
  *   An alternative, is to use end-to-end IPsec, and to encapsulate the packets in another protocol that contains the necessary information for the GWs to forward the packet without needing to look into the payload
  *   GENEVE is selected in this document as the encapsulation protocol because it is already widely in use in Cloud DC sites

[Linda] Change to the following:

As some packets from an enterprise CPE are destined to services within the Cloud DC, therefore requiring the Cloud DC to decrypt and forward to their respective destinations, and some are steered across the Cloud Backbone to the destination CPEs, the Cloud ingress GW must be able to determine which packets to decrypt and which one to steer across without decryption. This document describes a method for SD-WAN CPEs using GENEVE Encapsulation [RFC8926] to encapsulate the IPsec encrypted packets and send them to their closest Cloud GWs, who can detect if the packet needs to be steered across its backbone without decryption by looking into the Sub-TLVs inside the GENEVE header, which are specified in Section 4. Once it is determined that the packet is for steering across the backbone, the IPsec encrypted payload is steered through the Cloud Backbone without decryption to an optimal egress Cloud GWs, which forward the original IPsec encrypted payload to the destination CPEs. This method is for Cloud Backbone to connect multiple segments of SD-WAN without the Cloud GWs decrypting and re-encrypting the payloads. GENEVE is selected in this document as the encapsulation protocol because it is already widely in use in Cloud DC sites.
= Comments =

The document is relatively readable notwithstanding my concerns about
clarity of motivation. I feel that it could be a lot shorter and cleaner
(possibly the authors are too close to the problem?), but this is
something the WG should be able to polish.

There are a lot of nits in this document. Nits are not significant for
determining adoption and might not need to be fixed before adoption by a
WG (this is a matter for the chairs to determine), but their presence
makes the document much harder to review.

= Major Issues =

4.1 has

   The Protocol Type (16 bits) = 50 (ESP) [RFC4303] indicates
   that IPsec ESP encapsulated data are appended at the end of
   the GENEVE header.

My understanding of RFC 8926 is that you put an Ethertype in the
Protocol Type field.

   Protocol Type (16 bits):  The type of protocol data unit appearing
      after the Geneve header.  This follows the Ethertype [ETYPES]
      convention, with Ethernet itself being represented by the value
      0x6558.

The value 50 (x32) would thus be interpreted as an IEEE802.3 Length
Field. I *think* what you are meant to do is put the ESP after an IP
header (per RFC 4303) and then use the appropriate Ethertype in the
GENEVE header. This probably also makes life a lot easier because the
encapsulating IP header tells the gateway a lot about the intended
destination of the packet (for example, you wouldn't need the SD-WAN
Tunnel Endpoint Sub-TLV and the SD-WAN Tunnel Originator Sub-TLV).
[Linda] Per your suggestion, we removed the Section 3.1, simply stating that Geneve header is specified in Section 3 of [RFC8926].
The draft needs to apply a new GENEVE Option Class (multi-seg-SD-WAN Option Class) at IANA (https://www.iana.org/assignments/nvo3/nvo3.xhtml ).
= Minor Issues =

Section 1 lists a number of ways to connect an enterprise to a cloud DC.
It says that these are described in [Net2Cloud] but a quick search does
not yield matching terms. It would be helpful if the terminology was
aligned and a more specific (section) reference was given.
[Linda] Added: Section 4.1 of [NET2CLOUD]

[AF] That will surely help. Will you also align the terminology?
[Linda] Yes. Will do.
The section introduces "the IPsec encrypted packets" without mentioning
where these are flowing and why IPsec is being used.

A lot of Section 1 seems to be describing reasons for an architecture
(which would make use of the proposed solution) and not simply speaking
to the problem at hand that motivates the solution. Doesn't the
architectural discussion belong in [Net2Cloud]?

All in all, it is very hard to determine a clear motivation for this
work from the text until we get to Section 3.4.
[Linda] Please see above re-arrangement of the paragraphs of Section 1.
---

2.

   SD-WAN      An overlay connectivity service that optimizes
               transport of IP Packets over one or more Underlay
               Connectivity Services by recognizing applications
               (Application Flows) and determining forwarding
               behavior by applying Policies to them. [MEF-70.1]

To be clear, and notwithstanding anything in MEF70.1, it is not
possible (or desirable) to identify end applications. What is
possible, using the fields highlighted in MEF70.1, is to select
traffic flows and apply steering policies to them to direct them
towards interfaces or tunnel endpoints so that they are carried
across the underlay.
[Linda] Joel Halpern asked me to provider reference for SD-WAN. I sent him several options. He thinks referencing MEF is better than using any vendor’s definition.
[AF] I am not objecting to using MEF70.1 as a reference to explain the term “SD-WAN”.
I am objecting to the suggestion that points within the network should be able to identify user’s applications.
The language here is yours (it’s in your draft!) so you can change it to talk about traffic flows, and not mention applications at all.

[Linda] got it. Did a global change to remove all mention of “applications”.

---

3.1 points to the use of cloud security functions and SaaS features. If,
however, end-to-end encryption is in use through IPsec, and that is
carried in GENEVE so that the gateways do not need to decrypt and re-
encrypt, the availability of those security functions will be limited
(because they cannot see into the payload packet). Doesn't that mean
that these pieces of this use case are lost?
[Linda] There are monitoring functions provided by Cloud DC than can check IPsec encrypted traffic.
[AF] OK. Would you like to add references for those?
[Linda] different cloud operators provide different ones. mentioning one vendor’s function is not appropriate for IETF drafts.
Here are the current available monitoring tools by AWS:
 https://docs.aws.amazon.com/vpn/latest/s2svpn/monitoring-overview-vpn.html#monitoring-automated-manual

---

I think 3.3 says, "overlays are useful," and then 3.4 says "tunnels are
useful."
[Linda] Overlay network is built upon tunnels. Is it wrong?
[AF] Well, nothing wrong. Just checking that your many words match my take-away.
Perhaps you could deliver the summary message rather than leaving the reader to derive it?
[Linda] There is only one occurrence of mentioning “overlay connectivity” (from the MEF70.1 citation). where to add?
---

You need much more clarity on your choice to not set the C bit in the
option types. You should state that this is a deliberate choice and
describe how the packets will be processed if the option type is not
recognised.

Won't you need a registry for these option types?
[Linda] Are you talking about the multi-seg-SD-WAN Option Class specified in Section 4.2? (https://www.iana.org/assignments/nvo3/nvo3.xhtml ).

[AF] There are two things going on…

  1.  You are requesting an assignment from the Option Class registry (TBD). No problem.
  2.  You are defining three Type values. Here there are two issues:

     *   According to 3.5 of 8926, the Type field of an Option Class is 8 bits with the high order bit being named the C-bit. The C-bit has specific meaning. Your three values do not choose to set that bit. You should explain why.
     *   You may want to consider having a registry for these Type values so that new SD-WAN Option class types can be defined in future.
[Linda] add the following:
C-bit needs to be set, so that receiving node can drop the packet if it does not recognize the option [RFC8926].


---

4.6 and 4.7

Obviously, you are going to have to specify these TLVs in a future
version. For the include case, you are going to have to say whether this
is an ordered list, whether the inclusion is mandatory, and whether the
list is strict or loose. You may also have to worry about the option
length in the GENEVE header especially in the case of IPv6.
[Linda] Yes. Will add. Just at this moment, Azure is not sure what information they are willing to provide .
[AF] Apologies, I thought this was an IETF spec. If this document is describing a proprietary protocol, then the document needs a lot of work!
[Linda] I meant to say that they may not want internal IP address exposed. Node ID or Region ID are all okay.
Add the TLV
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Include-Transit| length        |Transit_Type   |I|Reserved     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            Transit node ID                    |
~                                                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
            Figure 8 Include-Transit Sub-TLV

Multiple Include-Transit Sub-TLVs can be included in one GENEVE header to represent multiple nodes or regions to be included when the packet is steered through the Cloud Backbone.
Transit_type:
TBD1: when Transit node ID is represented as a numeric number, such as a Cloud Availability Region or Zone numeric identifier.
TBD2: when Transit node ID is represented as string, such as a Cloud Availability Region or Zone name.
TBD3: when Transit node ID is represented as an IP address.
I-bit:
When set to 0: it indicates it needs best effort to steer through the transit node ID.
When set to 1, it indicates that the Transit Node ID must be included through the Cloud Backbone. If the Transit Node ID cannot be traversed, an alert or alarm must be generated to the enterprise via an out-of-band channel. It is out of the scope of this document to specify those alerts or alarms.

---

5.5 What do you mean by an invalid address?
[Linda] When an enterprise purchases the Cloud resources, it must register its valid CPEs addresses. If Cloud GW receives packets from the source addresses that it doesn’t recognize, they are considered as invalid address.
Removed the “invalid”,
Cloud GW SHOULD drop the packets with unregistered source or destination addresses

---

4.5 has...
   The Control Plane protocol is out of the
   scope of this document.
[Linda] meant to say : The detailed Control Plane protocol extension is out of the scope of this document.

And yet...

   7. Control Plane considerations

---

I'm curious why your new Multi Segment SD-WAN Sub-TLVs registry has 0
and 255 as reserved.
[Linda] To be on the safe side. I see many registries has 0 and 255 reserved. Is it okay?

[AF] I am all in favour of safety. However, “I saw someone else did it, so I did it,” is as much reckless as safe! Have a reason for the things you do.
[Linda] In my option, it is a good practice to avoid 0 and FFFF in case some future operation needs to do some math function on the value. .


= Nits =

The very first thing I do when asked to review a document is to run idnits
It is not hard (just click a button).

There is rarely an excuse for any version of a draft to have nits, but when
an author is requesting action (such as adoption or last call) then they
really have a duty to fix all of the nits first.

idnits reports
[Linda] Will fix all those idnits reports.
  Checking nits according to https://www.ietf.org/id-info/checklist :
  ----------------------------------------------------------------------------

  == There are 7 instances of lines with non-RFC6890-compliant IPv4 addresses
     in the document.  If these are example addresses, they should be changed.

  == There are 5 instances of lines with private range IPv4 addresses in the
     document.  If these are generic example addresses, they should be changed
     to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x,
     198.51.100.x or 203.0.113.x.


  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  == The document seems to lack the recommended RFC 2119 boilerplate, even if
     it appears to use RFC 2119 keywords -- however, there's a paragraph with
     a matching beginning. Boilerplate error?

     (The document does seem to have the reference to RFC 2119 which the
     ID-Checklist requires).
  -- The document date (January 24, 2024) is 13 days in the past.  Is this
     intentional?

---

The layout of the file is a bit of a mess in places. This does not make
reviewing it any easier. The problems are around layout of boilerplate
and the use of bullets. (Probably an artefact of using the Word template
but surely easy to fix before submission?

The running footer is also broken probably for the same reason.

---

As a matter of style, the Title, Abstract, and main text should not use
abreviations (SD-WAN, DC, CPE, GW) without expanding them first. This
should happen in each place they are used for the first time (i.e., the
main text cannot inherit from the Abstract, and the Abstract cannot
inherit from the Title.

---

Please decide on "cloud DC" or "Cloud DC"
[Linda] changed all to Cloud DC

---

I'm not sure that wikipedia is the best reference, but if you want to
use it, make it a proper reference.
[Linda] Removed. That was left over from Net2Cloud, where Robert insisted on having a reference for IXP. Over many options I provided, he thinks this one is the best.
---

Clean your terms to only mention ones you actually use.

---

I don't think you should reproduce the GENEVE header from RFC 8926.
Although you probably have this right, you risk issues if you
accidentally introduce a discrepency.

You can just say, "The GENEVE header is defined in Section 3 of
[RFC8926]."
[Linda] Good idea! Changed.
---

Please use consistent case for "TBD"

[Linda] changed.