Re: [nvo3-dt-encap] [nvo3] Encap draft published by design team

Tom Herbert <> Thu, 16 February 2017 23:26 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 4DCD91294E9 for <>; Thu, 16 Feb 2017 15:26:16 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id XM_amNAqEGI4 for <>; Thu, 16 Feb 2017 15:26:14 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:400d:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id E7C87129485 for <>; Thu, 16 Feb 2017 15:26:13 -0800 (PST)
Received: by with SMTP id 11so30327664qkl.3 for <>; Thu, 16 Feb 2017 15:26:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=a7gICHU0zFrkWf0PBdW1ppiAeHWLcZEwHpmQl6Rcxjw=; b=dG3MVS5sPO9yCbFOxAAg/1Owsddqq8eJ2MDb8dAxoAG5UWR7cbcK5P7jub2zqWuoso RVm5jPEdHT24EcMlUjogORFw9GZV/Ecq35BFOLJPsqvqJoZNCd8esnvqE6xlHXsVG3oB Z474i6/OwZD2quqd+38GimWe132IAqmfQIy12yD0Gqe3TGPPKA3s9iFDVSc1wmi2vGHJ x5KY6pzjSDzxdVMn4/PoEYRY/HtezeW0xo+6psl9IkhbYWMETNV2tB1skW0DhPP4m3Of zhW2ndEaE41/qRgWJQrnmnPr7HhwmeppBM/+HxwP7XirojxBEvGzfcmaRfAShh+5MJYv nxwA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=a7gICHU0zFrkWf0PBdW1ppiAeHWLcZEwHpmQl6Rcxjw=; b=Y1prgUQDqIgLiN7GVHPycDCRLXcTFX3xTdi6j9Ve+u8tyH9iwC0yQMcjvdH3yPKgFy olAAA+24C4A9adnpxzMqj0OF6mWbsOBYnFdSpaLHoqS5B2Lktmz6TS8+hzZ/+gnl6Uqu wvMxSWNh/1iaHfaJwQGdm9q+/JCfck1oxDeK+RaJ4oIvojFKrp37w/mUvibCUdqSpXnd 3hvHCjobqxZBfLp2/Hj7EuNYa/32/G45Go3B109V67WjDDp3cw4O5aFmhh84VcYvSWDF R6+5ahituFZECLUn/MyCYRNhaJTCUagjD3rdK3vIvL5eEUQN03vVp3rUwlQyUzuL4Tnz 28gg==
X-Gm-Message-State: AMke39n3qtRJVp8byBC9lyaQOZtIhMzH86VGlfxaZQ8yRT6dEfFamn+HVcZMbbGgh2nIJBSIKO0YDo+ec5VF+A==
X-Received: by with SMTP id 2mr5461435qkh.228.1487287572977; Thu, 16 Feb 2017 15:26:12 -0800 (PST)
MIME-Version: 1.0
Received: by with HTTP; Thu, 16 Feb 2017 15:26:12 -0800 (PST)
In-Reply-To: <>
References: <> <> <> <> <> <> <> <> <>
From: Tom Herbert <>
Date: Thu, 16 Feb 2017 15:26:12 -0800
Message-ID: <>
To: Joe Touch <>
Content-Type: text/plain; charset=UTF-8
Archived-At: <>
Cc: Sam Aldrin <>, Sami Boutros <>, "" <>, "" <>, "" <>
Subject: Re: [nvo3-dt-encap] [nvo3] Encap draft published by design team
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Private mailing list for internal NVO3 Encapsulation Design Team discussions <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 16 Feb 2017 23:26:16 -0000

On Thu, Feb 16, 2017 at 1:21 PM, Joe Touch <> wrote:
> On 2/16/2017 1:14 PM, Tom Herbert wrote:
>> On Thu, Feb 16, 2017 at 1:11 PM, Joe Touch <> wrote:
>>> On 2/16/2017 12:27 PM, Tom Herbert wrote:
>>>> The problems of TLVs, particularly that they are unordered, require
>>>> iterative processing,
>>> That's trivially avoided by forcing the order.
>>> As I noted before, all that is required for equivalently easy processing
>>> is that both TLVs and bitfields use only known variants in only known
>>> orders.
>> Joe, do you know of any protocols that enforce such an ordering?
> No, because in most cases the "T" is intended to allow arbitrary reordering.
I'd feel more comfortable with calling this a "trivial" solution for
TLVs if there was a working example of protocol that has implemented
it and had been successfully deployed.

> My point is just that it isn't TLV itself that affects hardware and
> parallelization; it's the potential for variation.
> The same variation and need for serial processing could be true for new
> definitions for previously undefined bitfields values. E.g., consider
> that the first few bits of an IP packet determine whether the addresses
> are 32 bits or 128 bits, etc.
Not in the same way. At question here is how the packet is parsed to
identify the protocol fields, not how individual fields are processed.
Since TLVs are unordered that means the number of discrete packet
header formats including extensions is combinatoric. Specifically, the
number of possible combinations is

SUM(K! * (N choose K)) == SUM(N! / (N-K)!) == N! * SUM(1 / K!), for K
= 0..N which approximates to e * N!.

For bit-fields the number of possible combinations is always 2^N (or
2^(N+1) to allow both plain IPv4 and IPv6)

The former function grows at a much steeper rate. For instance, with
just 8 extensions there are 109601 possible combinations of TLVs in a
packet. For bit-fields there are just 256. So without some constraints
applied to ordering, TLVs are not amenable to use with HW
parallelization techniques such as TCAM. If the problem is
unconstrained, HW processing degenerates to be iterative over the
packet (alignment and making TLVs same size don't help here). The
situation in SW is not particularly better, bit-fields processing can
be loop free, whereas TLVs will require a loop.

Susceptibility of the TLV protocol mechanism to DOS is demonstrated
when an attacker spoofs a packet containing the maximum number of
minimum sized TLVs. An attackers job is made easier if the protocol
allows TLVs that can be ignored. For instance, to attack Geneve they
could just create a list of random tiny "ignore" TLVs and set the
C-bit to ensure the receiver processes the TLVs. An obvious mitigation
to this might be to require mandatory options to appear before
ignorable options (that is introduce a weak ordering requirement).
With that the C-bit could be completely eliminated since it's just as
easy to check the type of the first option to see if mandatory options
are present. Also, if the receiver only processes mandatory options
then they know to stop processing when they hit the first
non-mandatory option. This might be sufficient defense against the DOS
attack I described, but then of course if everyone were to ignore
non-mandatory TLVs in this manner then we'd have to ask what the point
is of even including them in the protocol definition.

Admittedly, without any actual TLVs defined in Geneve all of this is
all just speculation on my part!