Re: [nvo3] Draft Geneve

[Tom Herbert <[e-mail]>]

On Fri, Feb 28, 2014 at 3:24 PM, Sam Aldrin <[e-mail]> wrote:
> Hi all,
>
> Read the draft but have few questions on the same line others have asked.
>
> - Is this draft intended for standardizing within NVo3 WG? The status
> indicates it as informational. Also it is good to have it as draft-nvo3....,
> if it is meant for NVo3 WG.
> - I fail to find good reasoning, in the current version of the draft, on why
> design of encap transport header should be closely associated with metadata
> OR closely tied together? Could you add more details to clarify?

The draft alludes to the general need for extensibility, but does not
provide any example uses, so maybe I can suggest one. We have a real
use case for an encapsulation protocol with security to allow
validation of the virtual network identifier. In their current for
vxlan and nvgre have no provisions for authenticating or integrity
check of vni, existing mechanisms in the network were not deemed
robust enough to guarantee integrity of vni and ensure strict
isolation between tenants. UDP checksum is not sufficient for this. We
need a mechanism to at least have enforce an unpredictable security
token, or possibly at stronger authentication using something like a
message digest. This is intrinsic to the encapsulation, we cannot
deploy network virtualization without this security, hence an
extensible protocol is desirable. Additionally, as the network scales,
new threats emerge, we may have need for further extensions to adapt.
All of this needs to be efficient and amenable to HW performance
optimizations.

> - Has the GAP analysis draft concluded on the assertion you made in your
> draft, for ex:
>      "Existing tunnel protocols have each attempted to solve different
> aspects of these new requirements, only to be quickly rendered out of date
> by changing control plane implementations and advancements."
> - In the past WG sessions and over the emails, the indication given was that
> various encap technologies already exist. Defining a good control plane
> technology is the need for the WG, no?
> - Finally, would like to see more details on why new encap is needed and
> could solve the problem, while extensions to VXLAN or NVGRE or LISP cannot
> solve.
>
> thanks
> -sam
>
>
>
> On Fri, Feb 28, 2014 at 11:00 AM, Anton Ivanov (antivano)
> <[e-mail]> wrote:
>>
>> On 28/02/14 18:28, Pankaj Garg wrote:
>> > The definition of meta-data for a feature should come from their
>> > respective WG, e.g. service-chaining blob design should come from SFC. How,
>> > the meta-data data is carried in an encapsulation comes from the design of
>> > the encapsulation format. The point of Geneve is that it would allow the
>> > meta-data to be carried without breaking NIC hardware offloads, whereas
>> > VXLAN, NVGRE etc. would require NIC hardware changes to carry meta-data.
>>
>> I have read that section about 10 times and I fail to see you doing
>> anything else beside putting it on 32 bit boundary and mandating it ends
>> on a 32 bit boundary.
>>
>> If that is the case, that can be done with any other encaps. No need to
>> define a new one - just insert metadata in front of payload and pad to
>> boundaries if needed.
>>
>> If that is not the case, can you please expand on exactly what it makes
>> it so special in the draft so that an insertion of a metadata blob at 32
>> bit boundary and ending on 32 bit boundary into L2TPv3, GRE, VXLAN, etc
>> does not fulfill that. It is not evident from the draft in its current
>> form.
>>
>> A.
>>
>> >
>> > -----Original Message-----
>> > From: nvo3 [mailto:[e-mail]] On Behalf Of Anton Ivanov
>> > (antivano)
>> > Sent: Friday, February 28, 2014 11:17 PM
>> > To: [e-mail]
>> > Subject: Re: [nvo3] Draft Geneve
>> >
>> > On 28/02/14 16:57, Pankaj Garg wrote:
>> >> We can discuss merits of Geneve vs VXLAN vs NVGRE vs STT vs L2TPV3 vs
>> >> <Name your favorite protocol> when it comes to standardization for network
>> >> virtualization.
>> > I already said what I had to say here. There is no discussion to be had.
>> >
>> >> My point was that Geneve is designed for network virtualization and not
>> >> only for service chaining. We have many use cases of ability to carry
>> >> meta-data to evolve network virtualization that have nothing to do with
>> >> service chaining and hence the right discussion forum for Geneve is NVO3
>> >> (IMHO).
>> > You again missed Kens and my point.
>> >
>> > We do not dispute the need to carry metadata and this is exactly the
>> > discussion we should be having.
>> >
>> > However the need to carry metadata has to be _DECOUPLED_ from
>> > encapsulation requirements.
>> >
>> > That is trivial.
>> >
>> > Use case - I have a Geneve  (+ metadata) on the left, I cross an NVI and
>> > I exit on the right via let's say VXLAN. If metadata is naturally welded to
>> > encapsulation as you propose this means losing all of it.
>> >
>> > Sorry, this does not fly. You asking the IETF under the guise of
>> > standard to grant you monopoly on any deployment where you got a leg in.
>> >
>> > Errr.. that shall be a no then :)
>> >
>> >
>> > A.
>> >
>> >
>> >> -----Original Message-----
>> >> From: Anton Ivanov (antivano) [mailto:[e-mail]]
>> >> Sent: Friday, February 28, 2014 10:20 PM
>> >> To: Pankaj Garg
>> >> Cc: Ken Gray (kegray); Brad Hedlund; [e-mail]
>> >> Subject: Re: [nvo3] Draft Geneve
>> >>
>> >> On 28/02/14 16:26, Pankaj Garg wrote:
>> >>> Geneve options are not specific to service chaining, though a service
>> >>> chain blob can be one of the option in Geneve header. So it does belongs to
>> >>> network virtualization encapsulation format and not to SFC charter.
>> >> Disagree.
>> >>
>> >> >From an encapsulation perspective it is a reduced form of what L2TPv3
>> >> static tunnels can do already:
>> >>
>> >> 1. Smaller VNID
>> >> 2. No cookies so no defence against misconfiguration 3. Unknown
>> >> security interactions and no analysis of interactions with IPSEC so no
>> >> clarity on extending beyond datacenter.
>> >> 4. While offset in L2TPv3 is not part of the standard, it is a most
>> >> common implementation feature allowing for same "metadata" blob behaviour.
>> >> Specifying that blob to hit an aligned address so you can configure offload
>> >> for it is trivial. It is not worth it to waste a draft (and a potential RFC)
>> >> on what is an implementation detail. What if I implement an offload engine
>> >> which works on unaligned addresses? Go and change the draft just because of
>> >> that?
>> >> 5. No silicon implementation, no widely deployed software
>> >> implementations.
>> >>
>> >> So if it is "just an encapsulation" there is no reason for it to exist
>> >> - it has all been done, we should just standardize the existing offset
>> >> feature and be done with it.
>> >>
>> >> I agree with Brad that the encaps part is the least interesting bit -
>> >> the key here is the way this is treated in a system - the meaning of life,
>> >> universe and the metadata blob. That at the moment is being looked at only
>> >> in SFC, so it does indeed belong there despite the fact that they are
>> >> looking at a reduced set of meanings for that.
>> >>
>> >> A.
>> >>
>> >>
>> >>
>> >>> -----Original Message-----
>> >>> From: nvo3 [mailto:[e-mail]] On Behalf Of Ken Gray
>> >>> (kegray)
>> >>> Sent: Friday, February 28, 2014 9:48 PM
>> >>> To: Brad Hedlund; Anton Ivanov (antivano)
>> >>> Cc: [e-mail]
>> >>> Subject: Re: [nvo3] Draft Geneve
>> >>>
>> >>> That work (chaining and metadata) is already being addressed in the
>> >>> SFC workgroup by charter.  At the very least, the draft should move if
>> >>> that's it's primary thrust.  In that work, there is no coupling between the
>> >>> header for forwarding in an overlay and the header for chaining and passing
>> >>> metadata Šand I think that was on purpose.
>> >>>
>> >>> On 2/28/14 10:08 AM, "Brad Hedlund" <[e-mail]> wrote:
>> >>>
>> >>>> The Geneve draft proposes a 24-bit VNI, in addition to 32-bits of
>> >>>> "Variable Length Options".
>> >>>> Those later 32 bits can be used entirely by the system designer for
>> >>>> carrying implementation specific metadata, such as for example
>> >>>> carrying service chaining relevant information from one middle-box to
>> >>>> the next.
>> >>>>
>> >>>> Cheers,
>> >>>> Brad
>> >>>>
>> >>>>
>> >>>> ----- Original Message -----
>> >>>> From: "Anton Ivanov (antivano)" <[e-mail]>
>> >>>> To: "Brad Hedlund" <[e-mail]>
>> >>>> Cc: [e-mail]
>> >>>> Sent: Friday, February 28, 2014 8:40:51 AM
>> >>>> Subject: Re: [nvo3] Draft Geneve
>> >>>>
>> >>>> On 28/02/14 14:14, Brad Hedlund wrote:
>> >>>>> The diagram in the Geneve draft depicts tunnels terminating on both
>> >>>>> virtual switches in a Hypervisor, as well as physical switches
>> >>>>> connecting to non-virtual physical hosts.
>> >>>>> As such, the later brings the physical hosts into the overlay
>> >>>>> without any modification to the software stack on the physical
>> >>>>> hosts.
>> >>>>> This is already happening today with VXLAN, with commercial
>> >>>>> switching silicon providing line rate VXLAN encap/decap.  The
>> >>>>> physical switch vendor can then provide interoperability with a
>> >>>>> third party control plane that can provision tunnels between the
>> >>>>> virtual and physical switches.
>> >>>> Correct. I am not questioning that.
>> >>>>
>> >>>> My point is that in the case of L2TPv3 and GRE it _HAS_ happened.
>> >>>> Many years ago. Different platforms though (BNG, Carrier systems,
>> >>>> etc).
>> >>>>
>> >>>> Also your VXLAN interpretation still means that the host does not
>> >>>> participate in the overlay. It is mediated by an off-host switch
>> >>>> through an interim step as raw Ethernet and you just added a silicon
>> >>>> requirement for that switch as well.
>> >>>>
>> >>>> This is not the host participating in the overlay. There is a
>> >>>> benefit for the host understanding the overlay.
>> >>>>
>> >>>> Trivial example - if the host can understand the overlay it can
>> >>>> present an overlay endpoint as an ethernet interface to a lightweight
>> >>>> container.
>> >>>> No overhead. Direct overlay to visualization instance. Thousands of
>> >>>> instances on a host too if you want to.
>> >>>>
>> >>>>> There's nothing preventing the physical hosts from implementing
>> >>>>> Geneve tunnels in their own software stack and bypassing the need
>> >>>>> for terminating tunnels on a physical switch.
>> >>>>> The diagram just depicts what the authors felt would be most common
>> >>>>> deployment scenario.
>> >>>>>
>> >>>>> As I see it, the point with Geneve is that the tunneling protocol
>> >>>>> is actually the least interesting part of the overall system.
>> >>>> We are in complete agreement. However I am taking this one step
>> >>>> further.
>> >>>> It is not just the least interesting part - it has already been done.
>> >>>> It is in the stack on most OSes. It has been in the stack on most
>> >>>> OSes for
>> >>>> 5 years running. There is no reason whatsoever to reinvent the wheel.
>> >>>>
>> >>>> The protocols are there, present, usable and should be used.
>> >>>>
>> >>>>>  So perhaps the industry can work with a common tunneling format
>> >>>>> that provides both data plane interoperability and sufficient room
>> >>>>> in the header for implementation specific differentiation
>> >>>>> (metadata).
>> >>>> L2TPv3 has 32 bits of what can be used as VNID (session). I do not
>> >>>> see how a proposal with a 24 bit VNID space has any merit over that.
>> >>>> If 24 bits are enough, there should be a valid argument on how
>> >>>> suddenly 32 bit have become smaller than 24.
>> >>>>
>> >>>>> Instead, perhaps the industry can focus on competing and
>> >>>>> differentiating at the control and management planes.  Not too
>> >>>>> dissimilar from what happened when we agreed on Ethernet and IP
>> >>>>> encapsulation.
>> >>>> Again - completely agree. In between themselves VXLAN, L2TPv3 and
>> >>>> GRE cover all key encaps basics and are supported on all platforms
>> >>>> of interest. There is no need for yet another unoriginal
>> >>>> encapsulation.
>> >>>>
>> >>>> A.
>> >>>>
>> >>>>> Cheers,
>> >>>>> Brad
>> >>>>>
>> >>>>>
>> >>>>> ----- Original Message -----
>> >>>>> From: "Anton Ivanov (antivano)" <[e-mail]>
>> >>>>> To: [e-mail]
>> >>>>> Sent: Friday, February 28, 2014 2:30:15 AM
>> >>>>> Subject: [nvo3] Draft Geneve
>> >>>>>
>> >>>>> Hi all,
>> >>>>>
>> >>>>> I just finished reading this draft.
>> >>>>>
>> >>>>> I am going to ignore the caption under the diagram at this point -
>> >>>>> it does not match to the diagram and neither does the rest of the
>> >>>>> draft (we will come back to it later).
>> >>>>>
>> >>>>> So, first and foremost, the diagram. I actually really like it. It
>> >>>>> is very close to what we have been showing to customers as a
>> >>>>> product prototype and will be shipping in the near future. Namely -
>> >>>>> I like the presence of the physical(s) in the top right corner.
>> >>>>> Physicals are an integral part of a datacenter and need to talk to
>> >>>>> the overlay too.
>> >>>>>
>> >>>>> It is quite sad that after getting this right, the draft cripples
>> >>>>> it by limiting the scope and failing to address what is actually
>> >>>>> shown on that diagram.
>> >>>>>
>> >>>>> In order for physicals to talk to the overlay they need to be able
>> >>>>> to connect to the overlay. In real life this means well known and
>> >>>>> well supported encapsulations - EoGRE, EoL2TPv3 and more recently
>> >>>>> VXLAN.
>> >>>>> The former two are supported on most "hosts" as well as nearly all
>> >>>>> network equipment. The latter is getting there too. A new and
>> >>>>> wonderful encapsulation which on top of all is not directly
>> >>>>> supported and mandates me to run a virtual switch everywhere is not
>> >>>>> a "well supported encapsulation". It does not fit the bill and does
>> >>>>> not match the diagram.
>> >>>>>
>> >>>>> There are two ways to approach this from here onwards. One is to
>> >>>>> continue the critique and discussion. IMHO this is pointless and
>> >>>>> violates the spirit of IETF: "We believe in rough consensus and
>> >>>>> working code".
>> >>>>>
>> >>>>> The consensus is there - most of the networking world uses either
>> >>>>> EoGRE or EoL2TPv3 (VXLAN has its established use cases too). These
>> >>>>> are established standards with RFCs. We will now _ADD_ _THE_ _CODE_
>> >>>>> so we can get out of this "I'd like to invent a new encapsulation
>> >>>>> today"
>> >>>>> madness once and for all.
>> >>>>>
>> >>>>>     * We just contributed a GPL2 EoL2TPv3 direct support for
>> >>>>> qemu-kvm to the qemu project.
>> >>>>>     * We also made a binding promise to offer equivalent code to
>> >>>>> UML (we have the code there, just the patch-sets are too big to
>> >>>>> submit at once).
>> >>>>>     * Containers can talk EoL2TPv3 directly already.
>> >>>>>
>> >>>>> All of these can use this to talk directly VM to VM, Physical to VM
>> >>>>> and Switch to VM (as on the diagram).
>> >>>>>
>> >>>>> It is fully supported for switching - you simply configure a
>> >>>>> EoL2TPv3 interface on your local host and bring it into OVS, Bridge
>> >>>>> or other local switching system of your choice. You can also switch
>> >>>>> remotely - the NVE is no longer mandatory co-hosted on the same
>> >>>>> machine. So if you  want to do all of your datacenter switching on
>> >>>>> a "big router/switch" you  can now do so. All of them can talk the
>> >>>>> encaps, most have support for  running thousands (some up to tens
>> >>>>> and hundreds of thousands) of NVEs too.
>> >>>>>
>> >>>>> You also can talk to host - just configure a tunnel "to yourself".
>> >>>>> Most importantly you can talk VM to VM and VM to physical router as
>> >>>>> needed for service chaining.
>> >>>>>
>> >>>>> This brings me back to the caption under the diagram. That caption
>> >>>>> is complete nonsense. You can implement means of communicating
>> >>>>> between all the elements on it without a mandatory switch. Why on
>> >>>>> earth do you need to cripple the spec and specify it after that?
>> >>>>>
>> >>>>> I am aware that EoLT2Pv3 is not a candidate for NVO3 at present. We
>> >>>>> will address that by contribute equivalent GRE code at a later date
>> >>>>> (next couple months).
>> >>>>>
>> >>>>> In fact, anyone can take the driver we contributed and make it
>> >>>>> support GRE, Geneve or carrier pigeons. This is because this
>> >>>>> contribution has produced an interesting side effect - anyone who
>> >>>>> would like to implement in software a new encaps that can be
>> >>>>> presented as a socket interface on a host running Type 2 environment
>> >>>>> can now do so in an afternoon or less.
>> >>>>> It is no longer "state of the art" activity. The barrier to entry
>> >>>>> and the technical merit of "new encaps" if it is only an encaps (no
>> >>>>> control plane component) has just been lowered to somewhere around
>> >>>>> zero. It is now "boring".
>> >>>>>
>> >>>>> This is inclusive of Geneve itself. As it has no other merit
>> >>>>> besides being new encaps I do not see why it should be left to live
>> >>>>> under these circumstances. I'd rather have people include the
>> >>>>> remaining key encaps out there - L2TPv3 and concentrate on getting
>> >>>>> the control plane portion working.
>> >>>>>
>> >>>>> Some final clarifications:
>> >>>>>
>> >>>>> 1. Drafts - L2TPv3 is an RFC and has been around for many years.
>> >>>>> There is no need for extra drafts.
>> >>>>> 2. IPR - the same mechanics for overlay termination have been used
>> >>>>> in storage for many years providing a plethora of prior art.
>> >>>>> Similarly, non-standard versions of the same approach (VDE,
>> >>>>> socket,etc) have been around since the dawn of qemu and UML. There
>> >>>>> is nothing to patent here.
>> >>>>>
>> >>>>> A
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> nvo3 mailing list
>> >>>>> [e-mail]
>> >>>>>
>> >>>>> https://urldefense.proofpoint.com/v1/url?u=https://www.ietf.org/mai
>> >>>>> l
>> >>>>> ma
>> >>>>> n/l
>> >>>>> istinfo/nvo3&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=LfVwmzZtwW32snST
>> >>>>> U
>> >>>>> vj
>> >>>>> gMl
>> >>>>> 4EgksMaIQ6nBSzwpJYhxw%3D%0A&m=HIV6ylhSIvlBLF2JcY4AbvKIp4AytOoWy1TnX
>> >>>>> n
>> >>>>> 9P
>> >>>>> rCc
>> >>>>> %3D%0A&s=4df281d1fecc67d941f33a355364c2e128de78da24a74dfa6af38f8f85
>> >>>>> e
>> >>>>> 09
>> >>>>> 0d2
>> >>>> _______________________________________________
>> >>>> nvo3 mailing list
>> >>>> [e-mail]
>> >>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >>> _______________________________________________
>> >>> nvo3 mailing list
>> >>> [e-mail]
>> >>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> _______________________________________________
>> >> nvo3 mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/nvo3
>> >>
>> > _______________________________________________
>> > nvo3 mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/nvo3
>>
>> _______________________________________________
>> nvo3 mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/nvo3
>
>
>
> _______________________________________________
> nvo3 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nvo3
>