Re: [RTG-DIR] Rtgdir early review of draft-dmk-rtgwg-multisegment-sdwan-05

[Linda Dunbar <[e-mail]>]

Adrian,

Thank you again. See my resolutions to your comments under [Linda2] in purple.

Linda

From: Adrian Farrel <[e-mail]>
Sent: Tuesday, February 20, 2024 3:43 AM
To: Linda Dunbar <[e-mail]>; 'Yingzhen Qu' <[e-mail]>
Cc: [e-mail]; [e-mail]; [e-mail]
Subject: RE: [RTG-DIR] Rtgdir early review of draft-dmk-rtgwg-multisegment-sdwan-05

Thanks, Linda.

I am now looking at the -06 version you posted.

I am doing some heavy snipping in this thread. Look for [AF2]

Adrian

[Linda] How about the following rearrangement of the Intro section?
Enterprises connecting to Cloud DC may find significant benefits in leveraging the Cloud Backbone for transporting traffic between their CPEs, such as

  1.  Enterprises can benefit from the robust and high-performance infrastructure cloud service providers provide by leveraging diverse paths and harnessing cloud backbones' scalability and global reach to reduce the risk of downtime or disruptions.
  2.  The scalability of the Cloud Backbone allows for efficient handling of increased data traffic, accommodating the growing demands of modern enterprises.
  3.  Cloud Backbone's centralized management and orchestration capabilities contribute to simplified network administration, enabling organizations to streamline their operations and respond more effectively to changing business requirements.

To ensure security, enterprise traffic between their CPEs requires encryption. By steering the encrypted traffic through the Cloud Backbone without the need for decryption and re-encryption at Cloud GWs, processing demands at these GWs can be significantly reduced. This streamlined approach not only maintains the integrity of the encrypted traffic but also optimizes processing resources, enhancing overall efficiency within the cloud infrastructure.
This document describes a method for SD-WAN CPEs using GENEVE Encapsulation [RFC8926] to encapsulate the IPsec encrypted packets and send them to their closest Cloud GWs, who can steer the IPsec encrypted payload through the Cloud Backbone without decryption to optimal egress Cloud GWs which then forward the original IPsec encrypted payload to the destination CPEs. This method is for Cloud Backbone to connect multiple segments of SD-WAN without the Cloud GWs decrypting and re-encrypting the payloads.
[AF] This is much better. What is missing (perhaps obvious to you) is why a GW would otherwise need to decrypt and re-encrypt. I think you probably need:

  *   Each GW must look at the {which?} fields in the packet to determine the next hop (i.e., GW) to which to forward the packet
[Linda] added a sentence saying that GW looks at the sub-TLV carried in the GENEVE header (which is specified Section 4 of the document)
[AF2] Ha! Sorry, I wasn't clear. Yes, with Geneve encapsulation in use, the necessary information to steer the packet is visible in the GENEVE header and all is good. But what I was trying to get at is why you need to encapsulate in GENEVE. So, if you go to the solution before this draft was written, there is a packet trying to get end-to-end through some GWs. That packet is encrypted using IPsec, but when it gets to the GW, the GW has to decrypt the packet to access some information needed in order to steer the packet. I was asking, "When GENEVE is not in use, which fields of the IPsec-encrypted packet does the GW need to access that requires the GW to decrypt the packet?"
Now, your revised text is clearer in some respects, but the goal-posts seem to have moved....
[Linda] Change to the following:

As some packets from an enterprise CPE are destined to services within the Cloud DC, therefore requiring the Cloud DC to decrypt and forward to their respective destinations, and some are steered across the Cloud Backbone to the destination CPEs, the Cloud ingress GW must be able to determine which packets to decrypt and which one to steer across without decryption. This document describes a method for SD-WAN CPEs using GENEVE Encapsulation [RFC8926] to encapsulate the IPsec encrypted packets and send them to their closest Cloud GWs, who can detect if the packet needs to be steered across its backbone without decryption by looking into the Sub-TLVs inside the GENEVE header, which are specified in Section 4. Once it is determined that the packet is for steering across the backbone, the IPsec encrypted payload is steered through the Cloud Backbone without decryption to an optimal egress Cloud GWs, which forward the original IPsec encrypted payload to the destination CPEs. This method is for Cloud Backbone to connect multiple segments of SD-WAN without the Cloud GWs decrypting and re-encrypting the payloads. GENEVE is selected in this document as the encapsulation protocol because it is already widely in use in Cloud DC sites.
[AF2] The first few sentences in this paragraph talk about the GW needing to determine which packets to decrypt before forwarding, and which to forward without decryption. I guess that is fine, although I don't see why this cannot be achieved using the normal IPsec SAs. That is, you are saying what can be achieved using the GENEVE encapsulation, but you are not saying why the GENEVE encapsulation is needed.
[AF2] But at the end of the paragraph you have come back to "without the Cloud GWs decrypting and re-encrypting the payloads" without indicating why that would have been necessary (which was the intent of my original question).

[Linda2] Let me try again. How about the following write-up?
"To ensure security, enterprise traffic between their CPEs is encrypted and remains inaccessible to any third parties, including the Cloud DC. For the encrypted packets to be steered through the Cloud Backbone, the packet header must contain information indicating the packet's intended route. Given that the IPsec SA between CPEs is exclusively maintained between the CPEs and is not accessible to Cloud GWs, the encrypted packet needs to be carried by a tunnel between the source CPE and the ingress Cloud GW. This tunnel can be another layer of IPsec, which adds processing overhead to the Cloud GW to decrypt the outer IPsec tunnel solely for steering the encrypted payload.
By steering the encrypted traffic through the Cloud Backbone without the need for decryption and re-encryption at Cloud GWs, processing demands at these GWs can be significantly reduced. This streamlined approach not only maintains the integrity of the encrypted traffic but also optimizes processing resources, enhancing overall efficiency within the cloud infrastructure.
This document introduces a method for SD-WAN CPEs that utilizes GENEVE Encapsulation [RFC8926] to encapsulate IPsec encrypted packets, directing them to the nearest Cloud GWs. These gateways can determine whether the packet needs to traverse the backbone without decryption by inspecting Sub-TLVs within the GENEVE header, as specified in Section 4. Once determined that the packet is intended for backbone traversal, the IPsec encrypted payload is steered through the Cloud Backbone without decryption to optimal egress Cloud GWs. These gateways then forward the original IPsec encrypted payload to the destination CPEs. This method facilitates the Cloud Backbone's connecting multiple SD-WAN segments without Cloud GWs decrypting and re-encrypting payloads.
GENEVE is selected in this document as the encapsulation protocol due to its widespread usage in Cloud DC sites."

4.1 has

   The Protocol Type (16 bits) = 50 (ESP) [RFC4303] indicates
   that IPsec ESP encapsulated data are appended at the end of
   the GENEVE header.

My understanding of RFC 8926 is that you put an Ethertype in the
Protocol Type field.

   Protocol Type (16 bits):  The type of protocol data unit appearing
      after the Geneve header.  This follows the Ethertype [ETYPES]
      convention, with Ethernet itself being represented by the value
      0x6558.

The value 50 (x32) would thus be interpreted as an IEEE802.3 Length
Field. I *think* what you are meant to do is put the ESP after an IP
header (per RFC 4303) and then use the appropriate Ethertype in the
GENEVE header. This probably also makes life a lot easier because the
encapsulating IP header tells the gateway a lot about the intended
destination of the packet (for example, you wouldn't need the SD-WAN
Tunnel Endpoint Sub-TLV and the SD-WAN Tunnel Originator Sub-TLV).
[Linda] Per your suggestion, we removed the Section 3.1, simply stating that Geneve header is specified in Section 3 of [RFC8926].
The draft needs to apply a new GENEVE Option Class (multi-seg-SD-WAN Option Class) at IANA (https://www.iana.org/assignments/nvo3/nvo3.xhtml ).

[AF2] Right, so you have fixed 4.1. Good. The encap is now Eth-IP-UDP-Geneve-IP-ESP-payload

[AF2] Which takes us back to asking what the Geneve encapsulation an the sub-TLVs buy us as the original IP header is now visible. Perhaps the question is whether the IPsec SA is CPE-CPE or end-to-end. I don't think that has been made clear. If the SA is between CPEs, the SD-WAN end-point sub-TLV seems to only reproduce what is in the encapsulated IP header dst field, and the SD-WAN tunnel originator Sub-TLV seems to reproduce the encapsulated IP header src field. So, perhaps you are trying to build an overlay which:

  *   Has very little to do with whether or not the payload is encrypted
  *   Is all about indicating the entry, exit, and transit GWs in the overlay network
[AF2] Overlays and tunnels are great (q.v.) and I am just trying to understand what it is you are trying to do with this work.
[Linda2] with the above changed write-up, is it more clear?

2.

   SD-WAN      An overlay connectivity service that optimizes
               transport of IP Packets over one or more Underlay
               Connectivity Services by recognizing applications
               (Application Flows) and determining forwarding
               behavior by applying Policies to them. [MEF-70.1]

To be clear, and notwithstanding anything in MEF70.1, it is not
possible (or desirable) to identify end applications. What is
possible, using the fields highlighted in MEF70.1, is to select
traffic flows and apply steering policies to them to direct them
towards interfaces or tunnel endpoints so that they are carried
across the underlay.
[Linda] Joel Halpern asked me to provider reference for SD-WAN. I sent him several options. He thinks referencing MEF is better than using any vendor's definition.
[AF] I am not objecting to using MEF70.1 as a reference to explain the term "SD-WAN".
I am objecting to the suggestion that points within the network should be able to identify user's applications.
The language here is yours (it's in your draft!) so you can change it to talk about traffic flows, and not mention applications at all.
[Linda] got it. Did a global change to remove all mention of "applications".
[AF2] Except in -06 you still have...
   SD-WAN      An overlay connectivity service that optimizes
               transport of IP Packets over one or more Underlay
               Connectivity Services by recognizing applications
               (Application Flows) and determining forwarding
               behavior by applying Policies to them. [MEF-70.1]

[Linda2] will change in -07
3.1 points to the use of cloud security functions and SaaS features. If,
however, end-to-end encryption is in use through IPsec, and that is
carried in GENEVE so that the gateways do not need to decrypt and re-
encrypt, the availability of those security functions will be limited
(because they cannot see into the payload packet). Doesn't that mean
that these pieces of this use case are lost?
[Linda] There are monitoring functions provided by Cloud DC than can check IPsec encrypted traffic.
[AF] OK. Would you like to add references for those?
[Linda] different cloud operators provide different ones. mentioning one vendor's function is not appropriate for IETF drafts.
Here are the current available monitoring tools by AWS:
 https://docs.aws.amazon.com/vpn/latest/s2svpn/monitoring-overview-vpn.html#monitoring-automated-manual

[AF2] I am not too interested in proprietary solutions (although I note that the referenced page makes no mention of IPsec or encryption). May I suggest:
OLD
  - Cloud-based tools and SaaS (Software as a Service) can be
     easily utilized to collect and analyze the threat to the
     traffic.
NEW
  - Proprietary cloud-based tools and SaaS (Software as a Service)
    may be available in specific deployments to collect and analyze
    the threat to the traffic.
END

[Linda2] Thanks for the suggestion, changed in -7
You need much more clarity on your choice to not set the C bit in the
option types. You should state that this is a deliberate choice and
describe how the packets will be processed if the option type is not
recognised.

Won't you need a registry for these option types?
[Linda] Are you talking about the multi-seg-SD-WAN Option Class specified in Section 4.2? (https://www.iana.org/assignments/nvo3/nvo3.xhtml ).

[AF] There are two things going on...

  1.  You are requesting an assignment from the Option Class registry (TBD). No problem.
  2.  You are defining three Type values. Here there are two issues:

     *   According to 3.5 of 8926, the Type field of an Option Class is 8 bits with the high order bit being named the C-bit. The C-bit has specific meaning. Your three values do not choose to set that bit. You should explain why.
     *   You may want to consider having a registry for these Type values so that new SD-WAN Option class types can be defined in future.
[Linda] add the following:
C-bit needs to be set, so that receiving node can drop the packet if it does not recognize the option [RFC8926].
[AF2] OK. This has not shown up in -06, so presumably still in your edit buffer. I don't know whether you are supposed to write "Type 1 with the C-bit set" or "Type 129"
[AF2] Note that 8926 calls it the 'C' bit and the high-order bit.
[AF2] You haven't answered the point about the new registry
[Linda2] Yes, need IANA registry. Reflected in -7
---

4.6 and 4.7

Obviously, you are going to have to specify these TLVs in a future
version. For the include case, you are going to have to say whether this
is an ordered list, whether the inclusion is mandatory, and whether the
list is strict or loose. You may also have to worry about the option
length in the GENEVE header especially in the case of IPv6.
[Linda] Yes. Will add. Just at this moment, Azure is not sure what information they are willing to provide .
[AF] Apologies, I thought this was an IETF spec. If this document is describing a proprietary protocol, then the document needs a lot of work!
[Linda] I meant to say that they may not want internal IP address exposed. Node ID or Region ID are all okay.
Add the TLV

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Include-Transit| length        |Transit_Type   |I|Reserved     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            Transit node ID                    |
~                                                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
            Figure 8 Include-Transit Sub-TLV

Multiple Include-Transit Sub-TLVs can be included in one GENEVE header to represent multiple nodes or regions to be included when the packet is steered through the Cloud Backbone.
Transit_type:
TBD1: when Transit node ID is represented as a numeric number, such as a Cloud Availability Region or Zone numeric identifier.
TBD2: when Transit node ID is represented as string, such as a Cloud Availability Region or Zone name.
TBD3: when Transit node ID is represented as an IP address.
I-bit:
When set to 0: it indicates it needs best effort to steer through the transit node ID.
When set to 1, it indicates that the Transit Node ID must be included through the Cloud Backbone. If the Transit Node ID cannot be traversed, an alert or alarm must be generated to the enterprise via an out-of-band channel. It is out of the scope of this document to specify those alerts or alarms.
[AF2] That is making progress. Thanks. I think you are still missing:

  *   Is this an ordered list or just a set?
[Linda2] No

  *   How is the string encoded?
[Linda2] strictly between the enterprise <-> Cloud Provider.

  *   Will you have a new registry for these Transit_type values?
[Linda2] Yes.

[AF2] Not sure that this discussion is needed prior to adoption, but you will need to have it before you reach RFC.

I'm curious why your new Multi Segment SD-WAN Sub-TLVs registry has 0
and 255 as reserved.
[Linda] To be on the safe side. I see many registries has 0 and 255 reserved. Is it okay?

[AF] I am all in favour of safety. However, "I saw someone else did it, so I did it," is as much reckless as safe! Have a reason for the things you do.
[Linda] In my option, it is a good practice to avoid 0 and FFFF in case some future operation needs to do some math function on the value. .

[AF2] ROFL